feat(metras): add Metras external-import connector for EDR telemetry (#7042)#7039
feat(metras): add Metras external-import connector for EDR telemetry (#7042)#7039Khidr6G wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new Metras EXTERNAL_IMPORT connector under external-import/metras/ to ingest Metras EDR telemetry into OpenCTI as STIX (Incidents + ATT&CK Attack-Patterns + Url/File observables + System identities), with a shared HTTP client, deterministic IDs, and a basic unit-test suite.
Changes:
- Introduces the Metras feed connector implementation (settings, API client, STIX conversion, import loop with state cursors + Work API lifecycle).
- Adds unit tests for utility functions, settings, API client, converter, and orchestration.
- Adds packaging/runtime artifacts for containerized deployment (Dockerfile, compose, entrypoint) plus connector metadata/docs.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| external-import/metras/VERSION | Adds connector version marker. |
| external-import/metras/tests/test-requirements.txt | Defines test dependencies for the connector. |
| external-import/metras/tests/test_main.py | Tests pure utility functions (refang, validation, timestamps, severity mapping). |
| external-import/metras/tests/test_connector/test_settings.py | Tests Pydantic settings behavior and defaults. |
| external-import/metras/tests/test_connector/test_orchestration.py | Tests connector import loop behavior with mocked helper/client. |
| external-import/metras/tests/test_connector/test_converter.py | Tests alert/binary/endpoint → STIX conversion outputs. |
| external-import/metras/tests/test_connector/test_client.py | Tests API client request/response handling with mocked HTTP. |
| external-import/metras/tests/conftest.py | Adds src path for test imports. |
| external-import/metras/src/requirements.txt | Adds runtime dependencies (pycti, stix2, requests, connectors-sdk, etc.). |
| external-import/metras/src/metras_client/api_client.py | Implements shared Metras REST API client (retrying session + endpoints). |
| external-import/metras/src/metras_client/init.py | Exposes MetrasClient / MetrasAPIError. |
| external-import/metras/src/main.py | Adds connector entrypoint wiring settings → helper → connector.run(). |
| external-import/metras/src/connector/utils.py | Adds dependency-free helpers (refang, validation, timestamp helpers, severity mapping). |
| external-import/metras/src/connector/settings.py | Adds connector + Metras config models (connectors-sdk + pydantic). |
| external-import/metras/src/connector/converter_to_stix.py | Converts Metras alerts/binaries/endpoints into STIX objects + relationships. |
| external-import/metras/src/connector/connector.py | Implements the feed import loop, cursors/state, and Work API status updates. |
| external-import/metras/src/connector/init.py | Exposes connector and settings entrypoints. |
| external-import/metras/README.md | Documents behavior, mapping, incremental sync, and configuration. |
| external-import/metras/pyproject.toml | Adds coverage configuration for this connector. |
| external-import/metras/entrypoint.sh | Adds container entrypoint script. |
| external-import/metras/Dockerfile | Builds a non-root Alpine-based container image for the connector. |
| external-import/metras/docker-compose.yml | Provides a hardened compose example for deployment. |
| external-import/metras/config.yml.sample | Provides sample YAML configuration. |
| external-import/metras/.dockerignore | Excludes unnecessary files from Docker build context. |
| external-import/metras/metadata/connector_manifest.json | Adds connector catalog metadata manifest. |
| from pycti import ( | ||
| AttackPattern, | ||
| Identity, | ||
| Incident, | ||
| StixCoreRelationship, | ||
| ) | ||
|
|
||
| if TYPE_CHECKING: | ||
| from pycti import OpenCTIConnectorHelper | ||
|
|
||
| _TLP_BY_NAME = { | ||
| "clear": stix2.TLP_WHITE, | ||
| "white": stix2.TLP_WHITE, | ||
| "green": stix2.TLP_GREEN, | ||
| "amber": stix2.TLP_AMBER, | ||
| "red": stix2.TLP_RED, | ||
| } |
There was a problem hiding this comment.
To avoid this kind of issues, it's strongly advised to use the OCTI models from connectors_sdk instead of stix2 objects (see https://github.com/OpenCTI-Platform/connectors/blob/master/connectors-sdk/connectors_sdk/models/tlp_marking.py)
| if len(all_objects) > 1: # more than just the author | ||
| bundle = self.helper.stix2_create_bundle(all_objects) | ||
| self.helper.send_stix2_bundle( | ||
| bundle, work_id=work_id, cleanup_inconsistent_bundle=True | ||
| ) |
| # Advance cursors only for categories that did not hard-fail. | ||
| new_state = dict(state) | ||
| if "alerts" not in "".join(errors) and new_alerts_max is not None: | ||
| new_state["alerts_last_occurrence"] = stix_timestamp(new_alerts_max) | ||
| if "binaries" not in "".join(errors) and new_binaries_max: | ||
| new_state["binaries_last_seen"] = new_binaries_max |
| | `CONNECTOR_SCOPE` | no | `Metras` | Import scope | | ||
| | `CONNECTOR_LOG_LEVEL` | no | `info` | Log level | | ||
| | `CONNECTOR_DURATION_PERIOD` | yes | `PT1H` | ISO-8601 poll interval | | ||
| | `METRAS_API_BASE_URL` | no | `https://api.metras.sa/api` | Metras API base URL | |
|
Also, please rename connector folder so it has a unique name. The rule is: folder name == manifest's slug == docker image's name 👌 |
Proposed changes
EXTERNAL_IMPORTconnector for the Metras EDR platform(
api.metras.sa,X-API-KEY) underexternal-import/metras/.Incident, withAttack-Pattern(uses) from the alertmitre_idsand
Urlobservables for external artifacts.StixFileobservables (banned/unsigned only by default to limit noise).Identity(identity_class="system"). Internal fleet assets aremodeled as System identities (internal IPs/hostnames kept in the description), not as
IOC observables or
Infrastructure, per maintainer guidance (PR [qualys] Add Qualys CVE enrichment connector #6164).last_occurrence_timefor the alerts endpoint(no server
fromTime), server-sidefromTimewindowing for binaries, full listing forendpoints. Deterministic STIX IDs via
pyctigenerate_id. Full Work-API lifecycle.pydantic.SecretStr; startup API ping; structured logging; CIS-hardeneddocker-compose.yml; non-rootpython:3.12-alpineimage withlibmagic.pyproject.tomlcoverage config for the
codecov/patchgate.black/isort --profile black/flake8 --ignore=E,Wclean.7.260529.0(pycti==7.260529.0,connectors-sdk @7.260529.0,support_version ">=7.260529.0").Related issues
Checklist
Further comments
Design notes driven by the Metras API: the EDR alerts endpoint exposes no
fromTimeparameter, so incremental import uses a client-side cursor on
last_occurrence_timeratherthan a server-side window; only the binaries endpoint supports
fromTime. Endpoints areemitted as System identities rather than observables to keep the IOC space clean — this is the
main modeling decision and is consistent with the accompanying Metras enrichment connector.