Bugfix/incorrect evaluation of deferred regular expressions

[suryamajhi]

Summary

This PR fixes the incorrect evalution of deferred regular expression.

Problem Statement

In presence of any regex modifier, it always does AND operation among them.
Example sigma:

detection:
      sel:
          fieldA|re: 
              - foo.*bar
              - abc.*xyz
      condition: sel

Current output:

| process regex("foobar", fieldA, "filter=true") | process regex("abcxyz", fieldA, "filter=true")

The result of this query is AND operation as both of them are filtering on truth case. But actually, it has to be an OR operation. Also, there is no regards for NOT operation.

Solution

Instead of directly adding the filter=true in regex process command, let the regex process command execute with group capturing and eval for checking if the group capturing was successful or not. Later use that eval generated field to correctly substitute in the sigma expression.

Sigma Example 1:

detection:
    sel:
        fieldA|re: 
            - foo.*bar
            - abc.*xyz
    condition: sel

Output:

| process regex("(?P<fieldA_match>foo.*bar)", fieldA)
| process eval("fieldA_condition=case(isnotnull(fieldA_match) -> 'true', 'false')") 
| process regex("(?P<fieldA_match2>abc.*xyz)", fieldA)
| process eval("fieldA_condition2=case(isnotnull(fieldA_match2) -> 'true', 'false')")
| search fieldA_condition="true" OR fieldA_condition2="true"

This output correctly express sigma rule as fieldA_condition and fieldA_condition2 will be populated accordingly with correct sigma expression.

Sigma Example 2 (Negation on regex modifier):

detection:
      sel:
          fieldA|re: 127\.0\.0\.1:[1-9]\d{3}
          fieldB: foo
      filter:
          fieldC|re: foo.*bar
      condition: sel and not filter

Output:

| process regex("(?P<fieldA_match>127\.0\.0\.1:[1-9]\d{3})", fieldA)
| process eval("fieldA_condition=case(isnotnull(fieldA_match) -> 'true', 'false')") 
| process regex("(?P<fieldC_match>foo.*bar)", fieldC)
| process eval("fieldC_condition=case(isnotnull(fieldC_match) -> 'true', 'false')")
| search fieldA_condition="true" fieldB="foo" -fieldC_condition="true"

[Copilot]

Pull request overview

This pull request fixes incorrect evaluation of deferred regular expressions in the Logpoint backend, addressing an issue where regex modifiers were incorrectly combined using AND operations instead of OR operations, and adding proper support for NOT operations.

Changes:

• Redesigned regex handling to use intermediate condition fields that are evaluated separately and then combined correctly in the final search query
• Updated the regex expression generation to use named capture groups and eval-based condition checking
• Removed redundant field mapping logic from Azure and M365 pipelines (lowercasing now handled in backend)
• Updated pysigma dependency from ^1.0.0 to ^1.1.1 to support new features

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
sigma/backends/logpoint/logpoint.py	Core implementation of deferred regex expressions using intermediate condition fields, removed convert_value_str override, added finish_query override to handle regex expressions
sigma/pipelines/logpoint/m365.py	Removed snake_case conversion logic (now handled in backend)
sigma/pipelines/logpoint/azure.py	Removed snake_case conversion logic (now handled in backend)
tests/test_backend_logpoint.py	Added comprehensive tests for single regex, OR, AND, and NOT operations with regex patterns
pyproject.toml	Updated pysigma dependency to ^1.1.1
poetry.lock	Updated lock file with new pysigma version and transitive dependency updates

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[d4rk-d4nph3]

LGTM