[Bug] ESQL Remote Validation Ignoring Rule Min-Stack

[terrancedejesus]

Pull Request

Issue link(s):

• [Bug] ESQL Remote Validation for Unknown Indices #6213

Summary - What I changed

Adds a small change to ignore stack version validation in ESQLValidator.remote_validate_rule() if the version is < the defined min_stack_version of the rule. This causes ESQL remote validation errors where the index pattern is not found in locally (integration manifests/schema) data, ultimately failing. Please see related issue for more details.

How To Test

1. Checkout new-rule/azure-ad-graph-potential-roadrecon-enum
2. Run python -m detection_rules view-rule rules/integrations/azure/discovery_aad_graph_roadrecon_aitohttp_enumeration.toml --esql-remote-validation
3. Observe error

Error (EsqlUnknownIndexError): Unknown index pattern(s): logs-azure.aadgraphactivitylogs-*. Known patterns: logs-azure.eventhub*, logs-azure.platformlogs*, logs-azure.activitylogs-*, logs-azure.platformlogs-* ...

4. Add debugger config

  {
      "name": "esql remote validation repro",
      "type": "debugpy",
      "request": "launch",
      "module": "detection_rules",
      "console": "integratedTerminal",
      "justMyCode": false,
      "cwd": "${workspaceFolder}",
      "args": [
          "view-rule",
          "rules/integrations/azure/discovery_aad_graph_roadrecon_aiohttp_enumeration.toml",
          "--esql-remote-validation"
      ]
  }

4. Set breakpoint at rule_validators.py#L928 (notice it starts at lowest 8.19.0 and increments up) -> note this the issue and should not be evaluated here
5. Continue --> error
6. Add the suggested fix to this branch and run again, notice only 9.3+ is evaluated in the for version in versions: loop

Checklist

• Added a label for the type of pr: bug, enhancement, schema, maintenance, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
• Added the meta:rapid-merge label if planning to merge within 24 hours
• Secret and sensitive material has been managed correctly
• Automated testing was updated or added to match the most common scenarios
• Documentation and comments were added for features that require explanation

Contributor checklist

• Have you signed the contributor license agreement?
• Have you followed the contributor guidelines?

[github-actions]

Bug - Guidelines

These guidelines serve as a reminder set of considerations when addressing a bug in the code.

Documentation and Context

• Provide detailed documentation (description, screenshots, reproducing the bug, etc.) of the bug if not already documented in an issue.
• Include additional context or details about the problem.
• Ensure the fix includes necessary updates to the release documentation and versioning.

Code Standards and Practices

• Code follows established design patterns within the repo and avoids duplication.
• Ensure that the code is modular and reusable where applicable.

Testing

• New unit tests have been added to cover the bug fix or edge cases.
• Existing unit tests have been updated to reflect the changes.
• Provide evidence of testing and detecting the bug fix (e.g., test logs, screenshots).
• Validate that any rules affected by the bug are correctly updated.
• Ensure that performance is not negatively impacted by the changes.
• Verify that any release artifacts are properly generated and tested.
• Conducted system testing, including fleet, import, and create APIs (e.g., run make test-cli, make test-remote-cli, make test-hunting-cli)

Additional Checks

• Verify that the bug fix works across all relevant environments (e.g., different OS versions).
• Confirm that the proper version label is applied to the PR patch, minor, major.

[eric-forte-elastic]

Nit. Is get_min_supported_stack_version needed? Its loading from the schema and get_stack_versions also loads verbatim from the schema so it should never be different right?

[eric-forte-elastic]

Also get_min_supported_stack_version already returns a Version object so the cast to string and then back to Version is unnecessary.

[eric-forte-elastic]

If for instance we min stack a rule to 9.4.1, it may not go through this section of the validation if the latest stack version is 9.4.0. Not inherently a problem, just that we need to be sure that the min stacks will not be a min stacked to a version newer than the latest version in the manifest.

[eric-forte-elastic]

Manual review, confirmed functionality accomplishes goal. Few comments, but no blockers.