feat: CipherSwarm → HashHive migration tool

[unclesp1d3r]

Summary

Build a one-shot migration tool that copies data from a running CipherSwarm v2 deployment into a HashHive deployment. The tool reads from CipherSwarm's PostgreSQL database directly (read-only) and writes to HashHive via its public APIs — never by reaching into HashHive's database.

This unblocks existing CipherSwarm operators from adopting HashHive without losing accumulated history (projects, hash lists, cracked plaintexts, resources, agents, completed campaigns).

Goals

• Operators with a live CipherSwarm v2 instance can run a single command to populate a fresh HashHive deployment with their historical data.
• All writes to HashHive go through the public APIs (Control API for human-driven entities, Agent API only where it's the right surface). No direct database writes.
• Idempotent: re-running against the same CipherSwarm snapshot and HashHive target should not produce duplicates.
• Dry-run mode that reports what would be migrated without making any HashHive API calls.

Non-Goals

• Live/continuous sync. This is a one-time cutover tool.
• Migrating in-flight tasks or running campaigns — operators are expected to drain CipherSwarm first.
• Migrating CipherSwarm secrets (agent tokens, API keys, password hashes). Operators reissue credentials in HashHive.
• Supporting unreleased CipherSwarm code. See "Supported Source Versions" below.

Supported Source Versions

The tool supports the latest tagged release of CipherSwarm v2 (i.e. the most recent git tag in the CipherSwarm repository). Untagged commits on main, release candidates, and older tagged releases are explicitly out of scope.

• The tool pins the exact CipherSwarm schema revision (Alembic/migration head) corresponding to that release tag and refuses to run against any other revision with an actionable error.
• When a new CipherSwarm release ships, this tool is updated to track the new tag. We do not maintain matrix support across multiple released versions.

Inputs

• CIPHERSWARM_DATABASE_URL — read-only PostgreSQL connection string for the source CipherSwarm instance.
• HASHHIVE_CONTROL_API_URL — base URL for the HashHive deployment.
• HASHHIVE_CONTROL_API_KEY — cst_* Control API key with permissions to create the entities below.
• Optional flags: --dry-run, --project=<name> (migrate a single project), --since=<ISO timestamp> (filter by creation date), --resume.

Entities In Scope

Order matters — later entities depend on earlier ones.

1. Projects / workspaces — name, description, created_at preserved as metadata.
2. Users — usernames and roles only. Operators reset passwords and issue new API keys post-migration.
3. Hash types — verify the CipherSwarm hash type maps to a known HashHive hash type; fail loudly on unknown types rather than guessing.
4. Hash lists — items + any per-item metadata. Crack results (plaintext + found_at) are migrated alongside.
5. Resources — wordlists, rule lists, masks, charsets. All resources are migrated, with structural conversions applied where CipherSwarm and HashHive shapes differ (see below). File bytes are streamed through the HashHive resources upload protocol.
6. Agents — name + label + project association only. Agents must re-enroll to receive new tokens.
7. Campaigns / attacks — all completed campaigns and their attacks are imported as historical records. Running/queued campaigns are skipped with a warning (operators are expected to drain first). Tasks are not migrated; campaigns land in a terminal state reflecting their CipherSwarm state.

Out of scope for v1, may follow later: audit log entries, benchmark history, agent error logs.

Structural Conversions

Where CipherSwarm and HashHive model the same concept differently (resource categorization, attack mode encoding, hash list metadata shape, etc.), the migration tool applies the documented conversion rather than refusing to migrate. Each conversion lives in a dedicated mapper module with unit tests covering the known shape differences for the supported release tag. Mapper modules are the contract; they get updated when we bump the supported CipherSwarm tag.

Behavior

• Read path: Direct PostgreSQL via a typed query layer. Raw SQL against the documented schema for the supported tag — no dependency on CipherSwarm's ORM.
• Write path: HashHive Control API (RFC 9457 errors, offset/limit pagination, cst_* key). For resource file uploads, follow the resources surface's existing upload protocol (presigned URL → SeaweedFS).
• Idempotency / provenance: Every migrate-able HashHive entity gains two nullable provenance fields, source_system (e.g. "cipherswarm") and source_id (the originating row's stable identifier — typically the CipherSwarm UUID, namespaced per entity type where needed). The pair is uniquely indexed per entity type. The Control API accepts these fields on create and exposes a lookup-by-source endpoint (GET /api/v1/control/<entity>?source_system=...&source_id=...) so the tool can resolve "have I already migrated this row?" without keeping local state. Re-runs are safe because the unique index makes duplicate inserts impossible.
• Why first-class provenance: Beyond migration, source_system + source_id lets us tag any imported, federated, or externally-synced entity going forward (future bulk-import tools, cross-deployment federation, audit trails for entities that originated outside the HashHive deployment). Treating this as a real domain concept now is cheaper than retrofitting later.
• Progress + resumability: Emit structured progress to stdout. Resumability falls out of the provenance lookup — restart simply re-runs every step; already-migrated rows are skipped via the unique constraint.
• Error handling: Fail loudly on schema-revision mismatches and unmapped hash types; continue past per-row API errors with a structured error report at the end.

Packaging

• Lives under packages/migration-tool/ (or similar) as a Bun CLI inside the existing monorepo.
• Distributed primarily as a docker image so operators don't need the HashHive source tree to run it.
• Documented in docs/ with a runbook: pre-flight checks, the exact supported CipherSwarm release tag + schema revision, expected runtime for typical dataset sizes, post-migration validation steps.

Prerequisite Work (separate PRs, land before the tool itself)

• Drizzle migration adding nullable source_system (text) + source_id (text) columns plus a composite unique index (source_system, source_id) per entity type, to: projects, users, hash_lists, resources, agents, campaigns, attacks (and any other migrate-able entity).
• @hashhive/shared Zod schema updates to expose the two fields on Control API request/response shapes, plus the OpenAPI spec updates that mirror them.
• Control API: accept source_system + source_id on each create endpoint; add GET /api/v1/control/<entity>?source_system=...&source_id=... lookup support (or equivalent filter on existing list endpoints).
• Add a dedicated POST /api/v1/control/campaigns/import endpoint that accepts a completed campaign + its attacks + crack results in a single payload and lands them in a terminal state. Keeps the existing create endpoints free of import-only edge cases (terminal state on create, historical timestamps, attached crack results) and gives the migration tool a single round trip per campaign.

Test Plan

• Unit tests for each entity mapper (CipherSwarm row → HashHive API request body), including every documented structural conversion.
• Integration test against a seeded CipherSwarm Postgres fixture (matching the supported release tag's schema) + a real HashHive backend running in CI, asserting end-to-end migration of a small synthetic dataset.
• Schema-revision guard test: pointing the tool at a database with the wrong Alembic head fails fast with an actionable error.
• Dry-run mode produces a structured report without making any HashHive API calls (verified by mock interceptor).
• Idempotency test: run migration twice against the same source + target, assert no duplicates (relies on the (source_system, source_id) unique index) and identical end state.
• Resume test: kill mid-run, restart, assert completion matches a clean run.
• Unknown-hash-type fixture causes a clean failure with actionable error, not silent skip.
• Completed-campaign import lands campaigns in the correct terminal state with crack results attached.
• Provenance fields are queryable through the Control API after migration (source_system=cipherswarm&source_id=<uuid> returns the migrated entity).

[coderabbitai]

🔗 Related PRs

#146 - feat(agent): heartbeat & error handling with audit log [merged]

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!