Optimize Drain3's Log Matching Speed by Skipping Unnecessary Tokenization

[DaiKiQi]

Current

Drain3 currently tokenizes every log entry first (before tree matching), but:

• Tokenization consumes ~80% of CPU (includes string copy overhead).
• When the Drain3 tree is stable (no new tokens added), most logs align with existing patterns (no new token variations).

Proposed Optimization

Add a "fast path" before tokenization:

1. For incoming logs: Use a two-pointer string comparison to check if it exactly matches the raw string of any existing node in the Drain3 tree.
2. If full match: Directly hit the corresponding pattern (skip tokenization entirely).
3. If no match: Fall back to the original tokenization + matching process.

Value

This cuts the most CPU-heavy step (tokenization) for stable trees, while retaining full compatibility for new/unmatched logs.