perf: share compiled RegexSet cache across scanners

[debugactiveprocess]

Optimize RegexSet Compilation and Reuse Across Scanners

Overview

This PR improves the lifecycle management of RegexSet compilation by moving compiled automata from scanner-local caches to a shared cache associated with Rules.

Previously, RegexSet instances were lazily compiled inside each Scanner through ScanContext::regex_set_matches(). While this avoided unnecessary upfront work, it introduced duplicated compilation costs when multiple scanners operated on the same ruleset.

Previous Behavior

When a RegexSet was first used during scanning:

1. The scanner-local cache was checked.
2. If absent, CompiledRules::get_regex_set() was invoked.
3. Each regex belonging to the set was reconstructed.
4. The regex was parsed into HIR.
5. The HIR was converted back into a string representation.
6. regex::bytes::RegexSetBuilder reparsed those strings and built a new automaton.

This workflow resulted in:

• First-scan latency spikes.
• Duplicate RegexSet compilation across scanners.
• Redundant automata construction in multi-threaded CLI workloads.
• Additional allocations and parsing overhead caused by the HIR → string → parse cycle.

New Approach

Compiled RegexSet instances are now stored at the Rules level and shared across scanners.

Conceptually:

struct Rules {
    regex_sets: FxHashMap<RegexSetId, Vec<RegexId>>,
    compiled_regex_sets: Vec<OnceLock<Arc<regex::bytes::RegexSet>>>,
}

The first scanner requesting a given RegexSet performs the compilation and stores the result in a shared cache. Subsequent scanners reuse the already-built automaton through an Arc, eliminating redundant compilation work.

Benefits

Reduced Cold-Scan Latency

The first file that triggers a grouped regex no longer incurs repeated compilation costs across scanners.

Shared Automata Across Threads

Parallel scans executed by the CLI can reuse the same compiled RegexSet instead of rebuilding identical automata in multiple workers.

Lower CPU Consumption

Compilation is performed once per ruleset rather than once per scanner instance.

Better Scalability for Long-Lived Services

Applications that create multiple scanners from the same Rules object benefit from improved reuse and lower memory churn.

Future Opportunities

This change lays the groundwork for additional optimizations around regex compilation.

A remaining source of overhead is the current workflow:

HIR -> String -> RegexSetBuilder -> Parse Again

Future work could explore direct construction of grouped automata from HIR representations, similar to how individual regexes can be compiled through:

regex_automata::meta::Builder::build_from_hir(...)

Avoiding reparsing would further reduce allocations and compilation overhead for large regex groups.

Expected Impact

Workloads with:

• Large rule sets
• Heavy usage of grouped matches
• Parallel CLI scanning
• Multiple scanners sharing the same rules

should observe lower startup overhead, reduced CPU utilization, and improved scan throughput due to more efficient automata reuse.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[plusvic]

Is this double-lock really necessary? It looks very suspicious that we need a lock for accessing the hash map and then another lock for accessing the RegexSet.

[debugactiveprocess]

good point, I remove the outer OnceLock and now initialize the runtime cache when Rules is built or deserialized. Each RegexSet still has its own OnceLock for lazy compilation, but get_regex_set no longer needs to lazily initialize and lock the whole cache map.