Skip to content

feat(rstthreatlibrary): create rst threat library connector (#7021) #7032

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Winnie-158263 wants to merge 6 commits into
base: master
Choose a base branch
from
Open

feat(rstthreatlibrary): create rst threat library connector (#7021) #7032

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1687369
Feature (rstthreatlibrary): Create RST Threat Library Connector (#7021)
Winnie-158263 Jul 17, 2026
8b801ed
feat(rstthreatlibrary): update rst threat library connector by updati…
Winnie-158263 Jul 23, 2026
e053293
feat(rstthreatlibrary): update connector by updating exception handli…
Winnie-158263 Jul 23, 2026
f68de3f
feat(rstthreatlibrary): update connector by updating stix source name…
Winnie-158263 Jul 23, 2026
ff68cb4
feat(rstthreatlibrary): update connector by updating config schema fo…
Winnie-158263 Jul 24, 2026
2e172af
feat(rstthreatlibrary): update connector by updating max retries and …
Winnie-158263 Jul 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions external-import/rst-threat-library/.dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
__metadata__
**/__pycache__
**/__docs__
**/.venv
**/venv
**/logs
**/config.yml
**/*.egg-info
**/*.gql
28 changes: 28 additions & 0 deletions external-import/rst-threat-library/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
FROM python:3.12-alpine
ENV CONNECTOR_TYPE=EXTERNAL_IMPORT

WORKDIR /opt/opencti-connector-rst-threat-library

COPY src /opt/opencti-connector-rst-threat-library/src

# Install pinned deps first, then connectors-sdk with --no-deps so its pycti
RUN apk --no-cache add --virtual .build-deps git build-base && \
apk --no-cache add libmagic && \
grep -v '^connectors-sdk' src/requirements.txt | grep -v '^#' | grep -v '^$' > /tmp/reqs.txt && \
pip3 install --no-cache-dir -r /tmp/reqs.txt && \
SDK_SPEC=$(grep '^connectors-sdk' src/requirements.txt | sed 's/^connectors-sdk[[:space:]]*@[[:space:]]*//') && \
pip3 install --no-cache-dir --no-deps "$SDK_SPEC" && \
rm -f /tmp/reqs.txt && \
apk del .build-deps

COPY entrypoint.sh /
RUN sed -i 's/\r$//' /entrypoint.sh && chmod +x /entrypoint.sh

RUN addgroup -S opencti && adduser -S opencti -G opencti && \
chown -R opencti:opencti /opt/opencti-connector-rst-threat-library /entrypoint.sh

USER opencti:opencti

HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
CMD sh -c "ps | grep -q '[p]ython3'"
ENTRYPOINT ["/bin/sh", "/entrypoint.sh"]
80 changes: 80 additions & 0 deletions external-import/rst-threat-library/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# OpenCTI RST Threat Library Connector

Ingests intrusion sets, malware, tools, and campaigns from the **RST Cloud Threat Library REST API** (`https://api.rstcloud.net/v1`) into OpenCTI, and keeps them in sync as upstream objects change.

## Introduction

The connector polls `GET /threat-objects/<type>` for each configured type and upserts the results into OpenCTI keyed on `standard_id`.

## Installation

### Requirements

- OpenCTI Platform >= 6.8.12 with a working worker
- Python >= 3.11 (for manual deployment)
- An RST Cloud Threat Library API key
- Docker / Docker Compose

### Configuration variables

Configuration is set either in `docker-compose.yml` (Docker) or in `config.yml` (manual deployment). See `config.yml.sample` for a full annotated example. Field descriptions and examples are also defined on the Pydantic models in `src/connector/settings.py` and mirrored in `__metadata__/connector_config_schema.json`.

#### OpenCTI environment variables

| Parameter | config.yml | Docker environment variable | Mandatory | Example | Description |
| ------------- | ---------- | --------------------------- | --------- | ----------------------- | ------------------------------------------------ |
| OpenCTI URL | `url` | `OPENCTI_URL` | Yes | `http://localhost:8080` | URL of the OpenCTI platform. |
| OpenCTI Token | `token` | `OPENCTI_TOKEN` | Yes | `ChangeMe` | Admin / API token used to bootstrap the connector. |

#### Base connector environment variables

| Parameter | config.yml | Docker environment variable | Default | Mandatory | Example | Description |
| ------------------------------ | -------------------------------------------- | -------------------------------------------------------- | ------------------- | --------- | -------------------------------------------- | --------------------------------------------------------------------------- |
| Connector ID | `id` | `CONNECTOR_ID` | / | Yes | `b7f9a6b4-6b2a-4c3f-9f7b-8b5e5b6f2d1a` | Unique UUID v4 for this connector instance. |
| Connector Type | `type` | `CONNECTOR_TYPE` | `EXTERNAL_IMPORT` | Yes | `EXTERNAL_IMPORT` | Must be `EXTERNAL_IMPORT`. |
| Connector Name | `name` | `CONNECTOR_NAME` | `RST Threat Library`| Yes | `RST Threat Library` | Display name in OpenCTI. |
| Connector Scope | `scope` | `CONNECTOR_SCOPE` | / | Yes | `intrusion-set,malware,tool,campaign` | STIX domain types emitted. |
| Log Level | `log_level` | `CONNECTOR_LOG_LEVEL` | `error` | No | `info` | `debug`, `info`, `warn`, or `error`. |
| Duration Period | `duration_period` | `CONNECTOR_DURATION_PERIOD` | `PT1H` | No | `PT1H` | ISO-8601 interval between runs. |
| Queue Threshold | `queue_threshold` | `CONNECTOR_QUEUE_THRESHOLD` | `500` | No | `500` | Max RabbitMQ queue size (MB) before pausing ingestion. |
| Update Existing Data | `update_existing_data` | `CONNECTOR_UPDATE_EXISTING_DATA` | `true` | No | `true` | Upsert existing STIX objects when `true`. |
| Auto-create Service Account | `auto_create_service_account` | `CONNECTOR_AUTO_CREATE_SERVICE_ACCOUNT` | `false` | No | `true` | Create a Connectors-group service account on first start. |
| Service Account Confidence | `auto_create_service_account_confidence_level` | `CONNECTOR_AUTO_CREATE_SERVICE_ACCOUNT_CONFIDENCE_LEVEL` | `50` | No | `50` | Max confidence for the auto-created service account. |

#### Connector extra parameters

| Parameter | config.yml | Docker environment variable | Default | Mandatory | Example | Description |
| --------------------------------- | ----------------------------------- | ---------------------------------------------------- | -------------------------------------------- | --------- | -------------------------------------------- | --------------------------------------------------------------------------- |
| API base URL | `baseurl` | `RST_THREAT_LIBRARY_BASEURL` | `https://api.rstcloud.net/v1` | Yes | `https://api.rstcloud.net/v1` | RST Cloud API root. |
| API key | `apikey` | `RST_THREAT_LIBRARY_APIKEY` | / | Yes | `ChangeMe` | RST Cloud Threat Library API key. |
| Auth header name | `auth_header` | `RST_THREAT_LIBRARY_AUTH_HEADER` | `x-api-key` | No | `x-api-key` | HTTP header carrying the API key. |
| HTTP proxy | `proxy` | `RST_THREAT_LIBRARY_PROXY` | *(empty)* | No | `http://proxy.example.com:8080` | Forward HTTP proxy URL. Empty = direct egress. |
| Verify TLS | `ssl_verify` | `RST_THREAT_LIBRARY_SSL_VERIFY` | `true` | No | `true` | Verify TLS certificates. |
| Connect timeout | `contimeout` | `RST_THREAT_LIBRARY_CONTIMEOUT` | `30` | No | `30` | HTTP connect timeout in seconds. |
| Read timeout | `readtimeout` | `RST_THREAT_LIBRARY_READTIMEOUT` | `120` | No | `600` | HTTP read timeout in seconds. |
| HTTP fetch retries | `retry` | `RST_THREAT_LIBRARY_RETRY` | `2` | No | `10` | Per-request retry count. |
| OpenCTI push max retries | `max_retries` | `RST_THREAT_LIBRARY_MAX_RETRIES` | `3` | No | `3` | Attempts when pushing to OpenCTI (minimum 1). |
| OpenCTI push retry delay | `retry_delay` | `RST_THREAT_LIBRARY_RETRY_DELAY` | `10` | No | `10` | Initial retry delay in seconds. |
| OpenCTI push backoff multiplier | `retry_backoff_multiplier` | `RST_THREAT_LIBRARY_RETRY_BACKOFF_MULTIPLIER` | `2.0` | No | `2.0` | Exponential backoff multiplier. |
| OpenCTI push mode | `opencti_push_mode` | `RST_THREAT_LIBRARY_OPENCTI_PUSH_MODE` | `bundle` | No | `bundle` | `bundle` (worker) or `api` (GraphQL import). |
| Object types to pull | `object_types` | `RST_THREAT_LIBRARY_OBJECT_TYPES` | `intrusion-sets,malware,tools,campaigns` | No | `intrusion-sets,malware,tools,campaigns` | Comma-separated paths under `/threat-objects/`. |
| Sort field | `order_by` | `RST_THREAT_LIBRARY_ORDER_BY` | `modified` | No | `modified` | Use `modified` for incremental sync. |
| Sort direction | `order_mode` | `RST_THREAT_LIBRARY_ORDER_MODE` | `desc` | No | `desc` | `asc` or `desc`. |
| Page size | `page_size` | `RST_THREAT_LIBRARY_PAGE_SIZE` | `100` | No | `20` | `limit` per request. |
| Intrusion-set merge/split | `merge_split` | `RST_THREAT_LIBRARY_MERGE_SPLIT` | `false` | No | `false` | Reconcile intrusion-set alias merge/split. |
| Respect local user edits | `respect_user_edits` | `RST_THREAT_LIBRARY_RESPECT_USER_EDITS` | `false` | No | `false` | Preserve higher-confidence OpenCTI edits. |
| Intrusion-set import confidence | `intrusion_set_default_confidence` | `RST_THREAT_LIBRARY_INTRUSION_SET_DEFAULT_CONFIDENCE`| *(unset)* | No | `80` | When set (0–100), replaces upstream confidence on intrusion sets. |
| Sync labels on import | `sync_labels` | `RST_THREAT_LIBRARY_SYNC_LABELS` | `RST Threat Library` | No | `RST Threat Library` | Labels merged on import; scopes merge/split. |
| Reconcile exclude labels | `reconcile_exclude_labels` | `RST_THREAT_LIBRARY_RECONCILE_EXCLUDE_LABELS` | *(empty)* | No | `MITRE,manual` | Labels excluded from merge/split fusion. |
| Reconcile createdBy allowlist | `reconcile_allow_created_by` | `RST_THREAT_LIBRARY_RECONCILE_ALLOW_CREATED_BY` | *(empty)* | No | `identity--…` | If set, fuse only entities with these `createdBy` IDs. |
| Initial backfill cutoff | `import_from_date` | `RST_THREAT_LIBRARY_IMPORT_FROM_DATE` | *(empty)* | No | `2024-01-01` | `YYYY-MM-DD`. Empty = full history on first run. |

### Recommended values (RST API reliability)

For deep pagination (especially `tools`), these reduce `504 Gateway Timeout` and `IncompleteRead` errors:

```env
RST_THREAT_LIBRARY_PAGE_SIZE=20
RST_THREAT_LIBRARY_READTIMEOUT=600
RST_THREAT_LIBRARY_RETRY=10
```
45 changes: 45 additions & 0 deletions external-import/rst-threat-library/__metadata__/CONNECTOR_CONFIG_DOC.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Connector Configurations

Below is an exhaustive list of the environment variables supported by the RST
Threat Library connector. Required values must be supplied before starting the
connector.

### Type: `object`

| Property | Type | Required | Possible values | Default | Description |
| -------- | ---- | -------- | --------------- | ------- | ----------- |
| OPENCTI_URL | `string` | Yes | URL | | The base URL of the OpenCTI instance. |
| OPENCTI_TOKEN | `string` | Yes | string | | The API token used to connect to OpenCTI. |
| CONNECTOR_ID | `string` | Yes | UUID v4 | | A unique identifier for this connector instance. |
| CONNECTOR_TYPE | `const` | Yes | `EXTERNAL_IMPORT` | `"EXTERNAL_IMPORT"` | The OpenCTI connector type. |
| CONNECTOR_NAME | `string` | Yes | string | `"RST Threat Library"` | The connector name displayed in OpenCTI. |
| CONNECTOR_SCOPE | `string` | Yes | string | `"intrusion-set,malware,tool,campaign"` | Comma-separated STIX domain types emitted by the connector. |
| CONNECTOR_LOG_LEVEL | `string` | | `debug`, `info`, `warn`, `warning`, `error` | `"error"` | The minimum level of logs to display. |
| CONNECTOR_DURATION_PERIOD | `string` | | ISO-8601 duration | `"PT1H"` | The period between connector runs. |
| CONNECTOR_QUEUE_THRESHOLD | `number` | | Number greater than 0 | `500` | Maximum RabbitMQ queue size in MB before ingestion pauses. |
| CONNECTOR_UPDATE_EXISTING_DATA | `boolean` | | `true`, `false` | `true` | Whether existing STIX objects may be updated. |
| CONNECTOR_AUTO_CREATE_SERVICE_ACCOUNT | `boolean` | | `true`, `false` | `false` | Whether to create a dedicated Connectors-group service account on first start. |
| CONNECTOR_AUTO_CREATE_SERVICE_ACCOUNT_CONFIDENCE_LEVEL | `integer` | | `0` to `100` | `50` | Maximum confidence level assigned to the auto-created service account. |
| RST_THREAT_LIBRARY_BASEURL | `string` | Yes | URL | `"https://api.rstcloud.net/v1"` | The RST Cloud Threat Library API base URL. |
| RST_THREAT_LIBRARY_APIKEY | `string` | Yes | string | | The RST Cloud Threat Library API key. |
| RST_THREAT_LIBRARY_AUTH_HEADER | `string` | | string | `"x-api-key"` | The HTTP header used to send the API key. |
| RST_THREAT_LIBRARY_PROXY | `string` | | URL or empty string | `""` | Optional forward HTTP proxy URL. An empty value uses direct egress. |
| RST_THREAT_LIBRARY_SSL_VERIFY | `boolean` | | `true`, `false` | `true` | Whether TLS certificates are verified for API requests. |
| RST_THREAT_LIBRARY_CONTIMEOUT | `integer` | | Positive integer | `30` | HTTP connection timeout in seconds. |
| RST_THREAT_LIBRARY_READTIMEOUT | `integer` | | Positive integer | `120` | HTTP read timeout in seconds. |
| RST_THREAT_LIBRARY_RETRY | `integer` | | Non-negative integer | `2` | Number of retries for each RST API request. |
| RST_THREAT_LIBRARY_MAX_RETRIES | `integer` | | Positive integer (≥ 1) | `3` | Maximum attempts when sending data to OpenCTI (at least one). |
| RST_THREAT_LIBRARY_RETRY_DELAY | `integer` | | Non-negative integer | `10` | Initial delay in seconds before retrying an OpenCTI push. |
| RST_THREAT_LIBRARY_RETRY_BACKOFF_MULTIPLIER | `number` | | Positive number | `2.0` | Exponential backoff multiplier for OpenCTI push retries. |
| RST_THREAT_LIBRARY_OPENCTI_PUSH_MODE | `string` | | `bundle`, `api` | `"bundle"` | The OpenCTI write path: worker bundle or GraphQL API import. |
| RST_THREAT_LIBRARY_OBJECT_TYPES | `string` | | Comma-separated string | `"intrusion-sets,malware,tools,campaigns"` | Threat-object API paths to poll. |
| RST_THREAT_LIBRARY_ORDER_BY | `string` | | string | `"modified"` | Field used to order API results for incremental synchronization. |
| RST_THREAT_LIBRARY_ORDER_MODE | `string` | | `asc`, `desc` | `"desc"` | Direction used to order API results. |
| RST_THREAT_LIBRARY_PAGE_SIZE | `integer` | | Positive integer | `100` | Maximum number of threat objects requested per page. |
| RST_THREAT_LIBRARY_MERGE_SPLIT | `boolean` | | `true`, `false` | `false` | Whether intrusion-set alias merge/split reconciliation is enabled. |
| RST_THREAT_LIBRARY_RESPECT_USER_EDITS | `boolean` | | `true`, `false` | `false` | Whether higher-confidence OpenCTI edits are preserved. |
| RST_THREAT_LIBRARY_INTRUSION_SET_DEFAULT_CONFIDENCE | `integer` | | `0` to `100` | | Optional confidence value that replaces upstream confidence on imported intrusion sets. |
| RST_THREAT_LIBRARY_SYNC_LABELS | `string` | | Comma-separated string | `"RST Threat Library"` | Labels applied during import and used to scope merge/split reconciliation. |
| RST_THREAT_LIBRARY_RECONCILE_EXCLUDE_LABELS | `string` | | Comma-separated string | `""` | Labels that exclude entities from merge/split fusion. |
| RST_THREAT_LIBRARY_RECONCILE_ALLOW_CREATED_BY | `string` | | Comma-separated identity IDs | `""` | When set, only entities created by these identities may be fused. |
| RST_THREAT_LIBRARY_IMPORT_FROM_DATE | `string` | | Date in `YYYY-MM-DD` format or empty string | `""` | Initial backfill cutoff. An empty value imports the full available history. |
Loading
Loading