Skip to content

feat(metras): add Metras external-import connector for EDR telemetry (#7042)#7039

Open
Khidr6G wants to merge 3 commits into
OpenCTI-Platform:masterfrom
Khidr6G:feature/metras-external-import
Open

feat(metras): add Metras external-import connector for EDR telemetry (#7042)#7039
Khidr6G wants to merge 3 commits into
OpenCTI-Platform:masterfrom
Khidr6G:feature/metras-external-import

Conversation

@Khidr6G

@Khidr6G Khidr6G commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Proposed changes

  • Add a new EXTERNAL_IMPORT connector for the Metras EDR platform
    (api.metras.sa, X-API-KEY) under external-import/metras/.
  • Imports three data sources into OpenCTI as STIX:
    • EDR alertsIncident, with Attack-Pattern (uses) from the alert mitre_ids
      and Url observables for external artifacts.
    • File binariesStixFile observables (banned/unsigned only by default to limit noise).
    • Endpoint inventoryIdentity(identity_class="system"). Internal fleet assets are
      modeled as System identities (internal IPs/hostnames kept in the description), not as
      IOC observables or Infrastructure, per maintainer guidance (PR [qualys] Add Qualys CVE enrichment connector #6164).
  • Incremental sync: client-side cursor on last_occurrence_time for the alerts endpoint
    (no server fromTime), server-side fromTime windowing for binaries, full listing for
    endpoints. Deterministic STIX IDs via pycti generate_id. Full Work-API lifecycle.
  • Secrets typed as pydantic.SecretStr; startup API ping; structured logging; CIS-hardened
    docker-compose.yml; non-root python:3.12-alpine image with libmagic.
  • Unit tests for settings, API client, converter, and connector orchestration; pyproject.toml
    coverage config for the codecov/patch gate. black / isort --profile black /
    flake8 --ignore=E,W clean.
  • Validated end-to-end against OpenCTI 7.260529.0 (pycti==7.260529.0,
    connectors-sdk @7.260529.0, support_version ">=7.260529.0").

Related issues

Checklist

  • I consider the submitted work as finished
  • I have signed my commits using GPG key.
  • I tested the code for its functionality using different use cases
  • I added/update the relevant documentation (either on github or on notion)
  • Where necessary I refactored code to improve the overall quality

Further comments

Design notes driven by the Metras API: the EDR alerts endpoint exposes no fromTime
parameter, so incremental import uses a client-side cursor on last_occurrence_time rather
than a server-side window; only the binaries endpoint supports fromTime. Endpoints are
emitted as System identities rather than observables to keep the IOC space clean — this is the
main modeling decision and is consistent with the accompanying Metras enrichment connector.

Copilot AI review requested due to automatic review settings July 17, 2026 18:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Metras EXTERNAL_IMPORT connector under external-import/metras/ to ingest Metras EDR telemetry into OpenCTI as STIX (Incidents + ATT&CK Attack-Patterns + Url/File observables + System identities), with a shared HTTP client, deterministic IDs, and a basic unit-test suite.

Changes:

  • Introduces the Metras feed connector implementation (settings, API client, STIX conversion, import loop with state cursors + Work API lifecycle).
  • Adds unit tests for utility functions, settings, API client, converter, and orchestration.
  • Adds packaging/runtime artifacts for containerized deployment (Dockerfile, compose, entrypoint) plus connector metadata/docs.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
external-import/metras/VERSION Adds connector version marker.
external-import/metras/tests/test-requirements.txt Defines test dependencies for the connector.
external-import/metras/tests/test_main.py Tests pure utility functions (refang, validation, timestamps, severity mapping).
external-import/metras/tests/test_connector/test_settings.py Tests Pydantic settings behavior and defaults.
external-import/metras/tests/test_connector/test_orchestration.py Tests connector import loop behavior with mocked helper/client.
external-import/metras/tests/test_connector/test_converter.py Tests alert/binary/endpoint → STIX conversion outputs.
external-import/metras/tests/test_connector/test_client.py Tests API client request/response handling with mocked HTTP.
external-import/metras/tests/conftest.py Adds src path for test imports.
external-import/metras/src/requirements.txt Adds runtime dependencies (pycti, stix2, requests, connectors-sdk, etc.).
external-import/metras/src/metras_client/api_client.py Implements shared Metras REST API client (retrying session + endpoints).
external-import/metras/src/metras_client/init.py Exposes MetrasClient / MetrasAPIError.
external-import/metras/src/main.py Adds connector entrypoint wiring settings → helper → connector.run().
external-import/metras/src/connector/utils.py Adds dependency-free helpers (refang, validation, timestamp helpers, severity mapping).
external-import/metras/src/connector/settings.py Adds connector + Metras config models (connectors-sdk + pydantic).
external-import/metras/src/connector/converter_to_stix.py Converts Metras alerts/binaries/endpoints into STIX objects + relationships.
external-import/metras/src/connector/connector.py Implements the feed import loop, cursors/state, and Work API status updates.
external-import/metras/src/connector/init.py Exposes connector and settings entrypoints.
external-import/metras/README.md Documents behavior, mapping, incremental sync, and configuration.
external-import/metras/pyproject.toml Adds coverage configuration for this connector.
external-import/metras/entrypoint.sh Adds container entrypoint script.
external-import/metras/Dockerfile Builds a non-root Alpine-based container image for the connector.
external-import/metras/docker-compose.yml Provides a hardened compose example for deployment.
external-import/metras/config.yml.sample Provides sample YAML configuration.
external-import/metras/.dockerignore Excludes unnecessary files from Docker build context.
external-import/metras/metadata/connector_manifest.json Adds connector catalog metadata manifest.

Comment thread external-import/metras/src/connector/converter_to_stix.py
Comment thread external-import/metras/src/metras_client/api_client.py Outdated
Comment thread external-import/metras/__metadata__/connector_manifest.json
@Khidr6G Khidr6G changed the title feat(metras): add Metras external-import connector for EDR telemetry (#7038) feat(metras): add Metras external-import connector for EDR telemetry (#7042) Jul 17, 2026
@romain-filigran romain-filigran added the community Contribution from the community. label Jul 24, 2026
@romain-filigran
romain-filigran requested a review from Copilot July 27, 2026 08:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated 4 comments.

Comment on lines +20 to +36
from pycti import (
AttackPattern,
Identity,
Incident,
StixCoreRelationship,
)

if TYPE_CHECKING:
from pycti import OpenCTIConnectorHelper

_TLP_BY_NAME = {
"clear": stix2.TLP_WHITE,
"white": stix2.TLP_WHITE,
"green": stix2.TLP_GREEN,
"amber": stix2.TLP_AMBER,
"red": stix2.TLP_RED,
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid this kind of issues, it's strongly advised to use the OCTI models from connectors_sdk instead of stix2 objects (see https://github.com/OpenCTI-Platform/connectors/blob/master/connectors-sdk/connectors_sdk/models/tlp_marking.py)

Comment on lines +181 to +185
if len(all_objects) > 1: # more than just the author
bundle = self.helper.stix2_create_bundle(all_objects)
self.helper.send_stix2_bundle(
bundle, work_id=work_id, cleanup_inconsistent_bundle=True
)
Comment on lines +200 to +205
# Advance cursors only for categories that did not hard-fail.
new_state = dict(state)
if "alerts" not in "".join(errors) and new_alerts_max is not None:
new_state["alerts_last_occurrence"] = stix_timestamp(new_alerts_max)
if "binaries" not in "".join(errors) and new_binaries_max:
new_state["binaries_last_seen"] = new_binaries_max
Comment on lines +45 to +48
| `CONNECTOR_SCOPE` | no | `Metras` | Import scope |
| `CONNECTOR_LOG_LEVEL` | no | `info` | Log level |
| `CONNECTOR_DURATION_PERIOD` | yes | `PT1H` | ISO-8601 poll interval |
| `METRAS_API_BASE_URL` | no | `https://api.metras.sa/api` | Metras API base URL |

@Powlinett Powlinett left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Khidr6G , thanks for your contribution.

Before addressing Copilot's comments, please rebase your branch onto the latest version of master to ensure the CI performs the same checks as on master 🙏

@Powlinett

Copy link
Copy Markdown
Member

Also, please rename connector folder so it has a unique name. The rule is: folder name == manifest's slug == docker image's name 👌

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community Contribution from the community.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(metras): Add Metras EDR connector suite (external-import, internal-enrichment, stream)

5 participants