Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 33 additions & 7 deletions stream/microsoft-sentinel-intel/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ The Microsoft Sentinel Intel connector streams OpenCTI indicators to Microsoft S
- [Introduction](#introduction)
- [Installation](#installation)
- [Requirements](#requirements)
- [Authentication](#authentication)
- [Azure AD Application Setup](#azure-ad-application-setup)
- [Configuration variables](#configuration-variables)
- [OpenCTI environment variables](#opencti-environment-variables)
- [Base connector environment variables](#base-connector-environment-variables)
Expand Down Expand Up @@ -44,15 +46,37 @@ Key features:
- Azure subscription with Microsoft Sentinel
- Azure AD Application or Managed Identity with appropriate permissions

### Authentication

The connector supports two authentication modes:

- **App registration** (`tenant_id` / `client_id` / `client_secret`): explicit Azure AD
application credentials. If you set any one of these three variables, you must set all
three.
- **Default Azure credential** (recommended): leave `tenant_id`, `client_id`, and
`client_secret` unset and the connector falls back to
[`DefaultAzureCredential`](https://learn.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential),
which authenticates using, in order, environment variables, a workload identity (e.g.
in AKS), a managed identity (system- or user-assigned, when running on Azure compute),
or the locally logged-in `az login` session.

Whichever identity is used, it must be granted the **Microsoft Sentinel Contributor**
role on the Log Analytics Workspace (see step 3 below). No Microsoft Graph API
permission is required, the connector authenticates against Azure Resource Manager
(`https://management.azure.com/.default`), and access is controlled entirely by that
RBAC role assignment, per the
[official upload API prerequisites](https://learn.microsoft.com/en-us/azure/sentinel/connect-threat-intelligence-upload-api#prerequisites).

### Azure AD Application Setup

1. Register an application in Azure AD (Entra portal)
2. Note the `tenant_id`, `client_id`, and `client_secret`
3. Configure API permissions: **ThreatIndicators.ReadWrite.OwnedBy**
4. In the Log Analytics Workspace, add **Microsoft Sentinel Contributor** role to the application
3. In the Log Analytics Workspace, add **Microsoft Sentinel Contributor** role to the application

> Skip this section if you're using a managed identity, workload identity, or `az login`
> instead — see [Authentication](#authentication) above.

![Sentinel Variables](./doc/sentinel_info_variables.png)
![Sentinel Permissions](./doc/permission_mandatory.png)

View indicators in: **Microsoft Sentinel > Threat Intelligence > Indicators**

Expand Down Expand Up @@ -95,9 +119,11 @@ Configure the connector in `docker-compose.yml`:
- CONNECTOR_LIVE_STREAM_ID=ChangeMe
- CONNECTOR_LIVE_STREAM_LISTEN_DELETE=true
- CONNECTOR_LIVE_STREAM_NO_DEPENDENCIES=true
- MICROSOFT_SENTINEL_INTEL_TENANT_ID=ChangeMe
- MICROSOFT_SENTINEL_INTEL_CLIENT_ID=ChangeMe
- MICROSOFT_SENTINEL_INTEL_CLIENT_SECRET=ChangeMe
# Optional: uncomment and fill in to use app-registration auth instead of DefaultAzureCredential
# (managed identity, workload identity, or `az login`).
# - MICROSOFT_SENTINEL_INTEL_TENANT_ID=ChangeMe
# - MICROSOFT_SENTINEL_INTEL_CLIENT_ID=ChangeMe
# - MICROSOFT_SENTINEL_INTEL_CLIENT_SECRET=ChangeMe
- MICROSOFT_SENTINEL_INTEL_WORKSPACE_ID=ChangeMe
- MICROSOFT_SENTINEL_INTEL_WORKSPACE_NAME=ChangeMe
- MICROSOFT_SENTINEL_INTEL_SUBSCRIPTION_ID=ChangeMe
Expand Down Expand Up @@ -202,7 +228,7 @@ CONNECTOR_LOG_LEVEL=debug
## Additional information

- **Role Propagation**: Role assignments on Log Analytics Workspace can take time to propagate
- **Authentication**: Managed identity is recommended over app registration
- **Authentication**: Managed identity (via `DefaultAzureCredential`) is recommended over app registration — see [Authentication](#authentication)
- **STIX Bundles**: Indicators are sent as STIX bundles to preserve threat intelligence context
- **Extra Labels**: Add comma-separated labels to all indicators sent to Sentinel
- **Delete Extensions**: Set to `true` to remove OpenCTI-specific extensions from bundles
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,6 @@ Below is an exhaustive enumeration of all configurable parameters available, eac
| OPENCTI_URL | `string` | ✅ | Format: [`uri`](https://json-schema.org/understanding-json-schema/reference/string#built-in-formats) | | The base URL of the OpenCTI instance. |
| OPENCTI_TOKEN | `string` | ✅ | string | | The API token to connect to OpenCTI. |
| CONNECTOR_LIVE_STREAM_ID | `string` | ✅ | string | | The ID of the live stream to connect to. |
| MICROSOFT_SENTINEL_INTEL_TENANT_ID | `string` | ✅ | string | | Your Azure App Tenant ID, see the screenshot to help you find this information. |
| MICROSOFT_SENTINEL_INTEL_CLIENT_ID | `string` | ✅ | string | | Your Azure App Client ID, see the screenshot to help you find this information. |
| MICROSOFT_SENTINEL_INTEL_CLIENT_SECRET | `string` | ✅ | Format: [`password`](https://json-schema.org/understanding-json-schema/reference/string#built-in-formats) | | Your Azure App Client secret, See the screenshot to help you find this information. |
| MICROSOFT_SENTINEL_INTEL_WORKSPACE_ID | `string` | ✅ | string | | Your Azure Workspace ID |
| MICROSOFT_SENTINEL_INTEL_WORKSPACE_NAME | `string` | ✅ | string | | The name of the log analytics workspace |
| MICROSOFT_SENTINEL_INTEL_SUBSCRIPTION_ID | `string` | ✅ | string | | The subscription id where the Log Analytics is |
Expand All @@ -21,6 +18,9 @@ Below is an exhaustive enumeration of all configurable parameters available, eac
| CONNECTOR_TYPE | `const` | | `STREAM` | `"STREAM"` | |
| CONNECTOR_LIVE_STREAM_LISTEN_DELETE | `boolean` | | boolean | `true` | Whether to listen for delete events on the live stream. |
| CONNECTOR_LIVE_STREAM_NO_DEPENDENCIES | `boolean` | | boolean | `true` | Whether to ignore dependencies when processing events from the live stream. |
| MICROSOFT_SENTINEL_INTEL_TENANT_ID | `string` | | string | `null` | Your Azure App Tenant ID, see the screenshot to help you find this information. |
| MICROSOFT_SENTINEL_INTEL_CLIENT_ID | `string` | | string | `null` | Your Azure App Client ID, see the screenshot to help you find this information. |
| MICROSOFT_SENTINEL_INTEL_CLIENT_SECRET | `string` | | Format: [`password`](https://json-schema.org/understanding-json-schema/reference/string#built-in-formats) | `null` | Your Azure App Client secret, See the screenshot to help you find this information. |
| MICROSOFT_SENTINEL_INTEL_RESOURCE_GROUP | `string` | | string | `"default"` | The name of the resource group where the log analytics is |
| MICROSOFT_SENTINEL_INTEL_SOURCE_SYSTEM | `string` | | string | `"Opencti Stream Connector"` | The name of the source system displayed in Microsoft Sentinel |
| MICROSOFT_SENTINEL_INTEL_DELETE_EXTENSIONS | `boolean` | | boolean | `true` | Delete the extensions in the stix bundle sent to the SIEM |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,14 +61,17 @@
"type": "boolean"
},
"MICROSOFT_SENTINEL_INTEL_TENANT_ID": {
"default": null,
"description": "Your Azure App Tenant ID, see the screenshot to help you find this information.",
"type": "string"
},
"MICROSOFT_SENTINEL_INTEL_CLIENT_ID": {
"default": null,
"description": "Your Azure App Client ID, see the screenshot to help you find this information.",
"type": "string"
},
"MICROSOFT_SENTINEL_INTEL_CLIENT_SECRET": {
"default": null,
"description": "Your Azure App Client secret, See the screenshot to help you find this information.",
"format": "password",
"type": "string",
Expand Down Expand Up @@ -164,9 +167,6 @@
"OPENCTI_URL",
"OPENCTI_TOKEN",
"CONNECTOR_LIVE_STREAM_ID",
"MICROSOFT_SENTINEL_INTEL_TENANT_ID",
"MICROSOFT_SENTINEL_INTEL_CLIENT_ID",
"MICROSOFT_SENTINEL_INTEL_CLIENT_SECRET",
"MICROSOFT_SENTINEL_INTEL_WORKSPACE_ID",
"MICROSOFT_SENTINEL_INTEL_WORKSPACE_NAME",
"MICROSOFT_SENTINEL_INTEL_SUBSCRIPTION_ID"
Expand Down
8 changes: 5 additions & 3 deletions stream/microsoft-sentinel-intel/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,17 @@

- CONNECTOR_ID=ChangeMe
- "CONNECTOR_NAME=Microsoft Sentinel Intel"
- CONNECTOR_SCOPE=sentinel # MIME type or Stix Object - Not used

Check warning on line 11 in stream/microsoft-sentinel-intel/docker-compose.yml

View workflow job for this annotation

GitHub Actions / Lint stream/microsoft-sentinel-intel

VC104: docker-compose.yml:11: CONNECTOR_SCOPE=sentinel has a default — should be commented out
- CONNECTOR_LOG_LEVEL=error

Check warning on line 12 in stream/microsoft-sentinel-intel/docker-compose.yml

View workflow job for this annotation

GitHub Actions / Lint stream/microsoft-sentinel-intel

VC104: docker-compose.yml:12: CONNECTOR_LOG_LEVEL=error has a default — should be commented out
- CONNECTOR_LIVE_STREAM_ID=ChangeMe # ID of the live stream created in the OpenCTI UI
- CONNECTOR_LIVE_STREAM_LISTEN_DELETE=true

Check warning on line 14 in stream/microsoft-sentinel-intel/docker-compose.yml

View workflow job for this annotation

GitHub Actions / Lint stream/microsoft-sentinel-intel

VC104: docker-compose.yml:14: CONNECTOR_LIVE_STREAM_LISTEN_DELETE=true has a default — should be commented out
- CONNECTOR_LIVE_STREAM_NO_DEPENDENCIES=true

Check warning on line 15 in stream/microsoft-sentinel-intel/docker-compose.yml

View workflow job for this annotation

GitHub Actions / Lint stream/microsoft-sentinel-intel

VC104: docker-compose.yml:15: CONNECTOR_LIVE_STREAM_NO_DEPENDENCIES=true has a default — should be commented out

- MICROSOFT_SENTINEL_INTEL_TENANT_ID=ChangeMe # Azure Tenant ID
- MICROSOFT_SENTINEL_INTEL_CLIENT_ID=ChangeMe # Azure App Client ID
- MICROSOFT_SENTINEL_INTEL_CLIENT_SECRET=ChangeMe # Azure App Client Secret
# Optional: uncomment and fill in to use app-registration auth instead of DefaultAzureCredential
# (managed identity, workload identity, or `az login`).
# - MICROSOFT_SENTINEL_INTEL_TENANT_ID=ChangeMe # Azure Tenant ID
# - MICROSOFT_SENTINEL_INTEL_CLIENT_ID=ChangeMe # Azure App Client ID
# - MICROSOFT_SENTINEL_INTEL_CLIENT_SECRET=ChangeMe # Azure App Client Secret
- MICROSOFT_SENTINEL_INTEL_WORKSPACE_ID=ChangeMe
- MICROSOFT_SENTINEL_INTEL_WORKSPACE_NAME=ChangeMe # For deletion
- MICROSOFT_SENTINEL_INTEL_SUBSCRIPTION_ID=ChangeMe # For deletion
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
from azure.core.exceptions import HttpResponseError
from azure.core.pipeline.policies import BearerTokenCredentialPolicy, RetryPolicy
from azure.core.pipeline.transport._base import HttpRequest, HttpResponse
from azure.identity import ClientSecretCredential
from azure.identity import ClientSecretCredential, DefaultAzureCredential
from microsoft_sentinel_intel.errors import ConnectorClientError
from microsoft_sentinel_intel.settings import ConnectorSettings
from pycti import OpenCTIConnectorHelper
Expand All @@ -22,15 +22,22 @@ def __init__(self, helper: OpenCTIConnectorHelper, config: ConnectorSettings):
f"/providers/Microsoft.OperationalInsights/workspaces/{self.config.microsoft_sentinel_intel.workspace_name}"
f"/providers/Microsoft.SecurityInsights/threatIntelligence/main"
)
if (
config.microsoft_sentinel_intel.tenant_id is not None
and config.microsoft_sentinel_intel.client_id is not None
and config.microsoft_sentinel_intel.client_secret is not None
):
credential = ClientSecretCredential(
tenant_id=config.microsoft_sentinel_intel.tenant_id,
client_id=config.microsoft_sentinel_intel.client_id,
client_secret=config.microsoft_sentinel_intel.client_secret.get_secret_value(),
)
else:
credential = DefaultAzureCredential()

policies = [
BearerTokenCredentialPolicy(
ClientSecretCredential(
tenant_id=config.microsoft_sentinel_intel.tenant_id,
client_id=config.microsoft_sentinel_intel.client_id,
client_secret=config.microsoft_sentinel_intel.client_secret.get_secret_value(),
),
"https://management.azure.com/.default",
credential, "https://management.azure.com/.default"
),
RetryPolicy(),
]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
BaseStreamConnectorConfig,
ListFromString,
)
from pydantic import Field, SecretStr
from pydantic import Field, SecretStr, model_validator


class StreamConnectorConfig(BaseStreamConnectorConfig):
Expand All @@ -31,13 +31,16 @@
Define parameters and/or defaults for the configuration specific to the `MicrosoftSentinelIntelConnector`.
"""

tenant_id: str = Field(
tenant_id: str | None = Field(
default=None,
description="Your Azure App Tenant ID, see the screenshot to help you find this information.",
)
client_id: str = Field(
client_id: str | None = Field(
default=None,
description="Your Azure App Client ID, see the screenshot to help you find this information.",
)
client_secret: SecretStr = Field(
client_secret: SecretStr | None = Field(
default=None,
description="Your Azure App Client secret, See the screenshot to help you find this information.",
)
workspace_id: str = Field(
Expand Down Expand Up @@ -72,7 +75,7 @@
# TODO: remove management_api_version config var (not used anymore)
management_api_version: str = Field(
description="API version of the Microsoft management interface",
default="2025-03-01",

Check warning on line 78 in stream/microsoft-sentinel-intel/src/microsoft_sentinel_intel/settings.py

View workflow job for this annotation

GitHub Actions / Lint stream/microsoft-sentinel-intel

VC105: Field default="2025-03-01" is a hardcoded absolute date
)
query_api_version: str = Field(
description="API version of the Microsoft Sentinel threat-intel /query endpoint",
Expand Down Expand Up @@ -102,6 +105,34 @@
default=False,
)

@model_validator(mode="after")
def check_auth_fields_consistency(self) -> "MicrosoftSentinelIntelConfig":
values = {
"tenant_id": self.tenant_id,
"client_id": self.client_id,
"client_secret": (
self.client_secret.get_secret_value()
if self.client_secret is not None
else None
),
}
blank_fields = [
name
for name, value in values.items()
if value is not None and not value.strip()
]
if blank_fields:
raise ValueError(
f"{', '.join(blank_fields)} must not be empty or whitespace-only."
)

provided = [value is not None for value in values.values()]
if any(provided) and not all(provided):
raise ValueError(
Comment thread
Renizmy marked this conversation as resolved.
"If you provide any of tenant_id, client_id, or client_secret, you must provide all three."
)
return self


class ConnectorSettings(BaseConnectorSettings):
"""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,23 @@
},
id="minimal_valid_settings_dict",
),
pytest.param(
{
"opencti": {"url": "http://localhost:8080", "token": "test-token"},
"connector": {
"id": "connector-id",
"scope": "test, connector",
"log_level": "error",
"live_stream_id": "live",
},
"microsoft_sentinel_intel": {
"workspace_id": "ChangeMe",
"workspace_name": "ChangeMe",
"subscription_id": "ChangeMe",
},
},
id="no_auth_fields_uses_default_credential",
),
],
)
def test_settings_should_accept_valid_input(settings_dict):
Expand Down Expand Up @@ -141,6 +158,46 @@ def _load_config_dict(cls, _, handler) -> dict[str, Any]:
"connector.id",
id="missing_connector_id",
),
pytest.param(
{
"opencti": {"url": "http://localhost:8080", "token": "test-token"},
"connector": {
"id": "connector-id",
"scope": "test, connector",
"log_level": "error",
"live_stream_id": "live",
},
"microsoft_sentinel_intel": {
"tenant_id": "ChangeMe",
"workspace_id": "ChangeMe",
"workspace_name": "ChangeMe",
"subscription_id": "ChangeMe",
},
},
"microsoft_sentinel_intel",
id="partial_auth_fields",
),
pytest.param(
{
"opencti": {"url": "http://localhost:8080", "token": "test-token"},
"connector": {
"id": "connector-id",
"scope": "test, connector",
"log_level": "error",
"live_stream_id": "live",
},
"microsoft_sentinel_intel": {
"tenant_id": " ",
"client_id": "ChangeMe",
"client_secret": "ChangeMe",
"workspace_id": "ChangeMe",
"workspace_name": "ChangeMe",
"subscription_id": "ChangeMe",
},
},
"microsoft_sentinel_intel",
id="blank_auth_field",
),
],
)
def test_settings_should_raise_when_invalid_input(settings_dict, field_name):
Expand Down
Loading