Please do not open a public issue for a security problem.
Report it privately through GitHub's private vulnerability reporting (repository Security tab → Report a vulnerability). Only the maintainer (@BenjaminSRussell) can see the report.
If that button is not available, open an issue titled "Security contact request" that contains no details of the problem. The maintainer will reply with a private channel. Send the details there.
Please include:
- what is affected (file, endpoint, workflow, image, or Helm chart) and the commit or tag;
- steps to reproduce or a proof of concept;
- the impact you expect (data exposure, SSRF, code execution, denial of service, ...).
What to expect:
| Step | Target |
|---|---|
| Acknowledgement | within 3 business days |
| Initial assessment (accepted / needs info / not a vulnerability) | within 10 business days |
| Fix or mitigation for accepted reports | as soon as practical; tracked in a private advisory |
You will be credited in the advisory and the CHANGELOG unless you ask not to be.
The project has not cut a stable release yet (pyproject.toml version 0.1.0).
Security fixes land on main only; release images are rebuilt from v* tags
(see docs/RELEASING.md).
| Version | Supported |
|---|---|
main |
yes |
| any earlier commit or tag | no, upgrade to main |
In scope: code under Scraping_project/, the container images built by
.github/workflows/cd-release.yml, the Helm chart and Kubernetes manifests under
Scraping_project/k8s/, and the GitHub workflows.
Out of scope: temp_scripts/ (unsupported one-off experiments), third-party
services (Kafka, Redis, Postgres, Grafana) beyond how this project configures them,
and findings that need an already-compromised host or cluster-admin access.
These rules apply to contributors and operators alike (#482).
- Never commit real credentials. Secrets belong in
Scraping_project/.env(or a Kubernetes Secret / external secret store). The repo-root.gitignoreignores.envand.env.*; only the*.env.exampletemplates are tracked, and they hold placeholders. - Templates:
Scraping_project/.env.example(pipeline, Compose) andScraping_project/kafka-delta-ingest/.env.examplelist every variable and mark required vs optional. - No secrets in
config.yml, Helmvalues*.yaml, workflow files, logs or issues. Pass them as environment variables (DB_PASSWORD,REDIS_PASSWORD,KAFKA_SASL_PASSWORD,GRAFANA_ADMIN_PASSWORD,POSTGRES_PASSWORD) or Helmsecrets.*/ existing Secret references. Log redaction (src/log_redaction.py) is a safety net, not permission to log secrets. - Local-dev defaults are public.
docker-compose.ymlfalls back to well-known passwords when.envdoes not set them: Postgres superuserpostgres(POSTGRES_PASSWORD), app rolescrapy_app_dev(APP_DB_PASSWORD), Grafanaadmin(GRAFANA_ADMIN_PASSWORD), and Redis without AUTH (REDIS_PASSWORD). Set real values before the stack is reachable from anything but your own machine.
If a credential was committed, pasted in an issue/log, or a default above was ever used on a shared host, treat it as compromised:
- Rotate first. Put a new value in
.env(or the Secret), then apply it:- Postgres:
docker compose exec postgres psql -U postgres -c "ALTER ROLE postgres PASSWORD '<new>'"(andALTER ROLE scrapy_app PASSWORD '<new>'forAPP_DB_PASSWORD), thendocker compose up -d --force-recreateso the exporter, Grafana and workers pick it up.POSTGRES_PASSWORDin.envonly seeds a new volume; an existing database keeps its old password until you runALTER ROLE. - Grafana:
docker compose exec grafana grafana cli admin reset-admin-password '<new>'. - Redis: set
REDIS_PASSWORD, thendocker compose up -d --force-recreate. - Kafka SASL / cloud keys: rotate at the provider, then update the Secret.
- Postgres:
- Then clean up. Removing the commit does not un-leak it: forks, clones and caches keep
it. Rewrite history only after rotating (
git filter-repo), and ask GitHub support to purge cached views if needed. - Report it privately as described above if the leak affects anyone besides you.