integration-protect-ffi.yml

name: Integration — protect-ffi (native + WASM)

# The `languages/typescript/packages/protect-ffi/integration-tests/` suite: 19 files of live
# encrypt / decrypt / lock-context / keyset / JSON-SteVec / Postgres /
# WASM-round-trip coverage against real ZeroKMS and a real Postgres.
#
# WHY THIS EXISTS: upstream (`cipherstash/protectjs-ffi`) ran this suite on
# EVERY pull request from `.github/workflows/test.yml` — `mise setup`, then
# `mise run test:integration:all`. The absorption copied the suite in intact and
# left that workflow behind under `languages/typescript/packages/protect-ffi/.github/`, where GitHub
# never looks: it reads workflows from the repository root alone. So from the
# day protect-ffi landed here, none of it ran, and a suite that never starts
# reads exactly like a suite that passes.
# `languages/typescript/packages/protect-ffi/src/integrationSuiteCi.test.ts` asserts this file (or a
# successor) still invokes it.
#
# WHERE THE SUITE'S DEPENDENCIES COME FROM. `integration-tests/` is a pnpm
# workspace member (CIP-3744) and installs with the repo's own
# `pnpm install --frozen-lockfile`, like everything else here. It used to sit
# outside the workspace with its own `package-lock.json` and an `npm ci`, pinning
# `@cipherstash/auth ^0.39.0`, `vitest ^3.1.3` and `@cipherstash/eql 3.0.2`
# against the repo catalog. That last pin was the point: `tests/postgres-v3.test.ts`
# queried SQL installed from the PUBLISHED EQL bundle while the payloads under
# test were emitted by the in-tree `eql-bindings`, so the two halves of EQL could
# disagree — and would have disagreed in a database, not in CI. `@cipherstash/eql`
# now resolves `workspace:^`, `@cipherstash/auth` `workspace:*`, and `vitest`
# `catalog:repo`.
#
# Separate from `tests.yml` on purpose, and separate from `tests-rust.yml`: this
# needs CipherStash credentials AND Docker, and the suites THROW rather than
# skip when unconfigured. Keeping them out of the unit job is what lets
# `pnpm test` stay runnable with neither.

on:
  push:
    branches: [main]
    paths:
      # The suite itself: tests, fixtures, its compose file, its `tasks.toml`
      # and its manifest.
      - 'languages/typescript/packages/protect-ffi/integration-tests/**'
      # The suite's dependency versions, now that it is a pnpm workspace member:
      # `vitest` and `typescript` reach it through
      # `catalog:repo`, so a catalog bump changes what this job runs while
      # editing no file under the suite. Same entry, same reason, as the other
      # integration workflows — and like them, `pnpm-lock.yaml` is deliberately
      # NOT listed: it moves on roughly every dependency bump in the monorepo,
      # and a catalog version cannot reach it without this file changing first.
      - 'pnpm-workspace.yaml'
      # The EQL v3 SQL this suite installs into its database, which
      # `tests/postgres-v3.test.ts` then queries. It came from a published
      # tarball until the suite joined the workspace; it now resolves
      # `workspace:^`, which is what makes it an input to this job rather than a
      # pinned external. Not narrowed to `sql/` — `tasks.toml` reads the bundle
      # through `@cipherstash/eql/sql`, i.e. through the package's tsup build,
      # so the build config and its asset copier are inputs too.
      - 'packages/eql/packages/eql/**'
      # The Rust the suite round-trips every payload through, and the manifests
      # that change what cargo builds without touching a .rs file. Matching the
      # filter in tests-rust.yml.
      - 'languages/typescript/packages/protect-ffi/crates/**'
      - 'languages/typescript/packages/protect-ffi/Cargo.toml'
      - 'languages/typescript/packages/protect-ffi/Cargo.lock'
      # `src/**` is `lib/`, which is the module every test imports, plus
      # `load.cts` — the loader that finds `index.node`.
      - 'languages/typescript/packages/protect-ffi/src/**'
      # `scripts/inline-wasm.mjs` emits `dist/wasm/protect_ffi_inline.js`, which
      # `tests/wasm-round-trip.test.ts` loads at module-graph time. An edit
      # there breaks that suite while touching no Rust and no `src/`.
      - 'languages/typescript/packages/protect-ffi/scripts/**'
      # The other half of the same build: `build:wasm` ends in `tsc -p
      # tsconfig.wasm-errors.json`, and the inliner re-exports what it emits.
      # `scripts/**` above does not reach a root tsconfig, so without this an
      # edit to it rebuilds the bundle that suite loads and starts no job.
      - 'languages/typescript/packages/protect-ffi/tsconfig.wasm-errors.json'
      # package.json carries the build scripts and the exports map the suite
      # resolves through; mise.toml carries the `tasks.toml` include, the
      # toolchain pins, and the PG* connection env.
      - 'languages/typescript/packages/protect-ffi/package.json'
      - 'languages/typescript/packages/protect-ffi/mise.toml'
      # Out of this package, and a real compile input: the cdylib crate carries
      # `eql-bindings = { path = "../../../../../../packages/eql/crates/eql-bindings" }`, so the
      # EQL wire types are linked into `index.node` AND into the wasm32 build
      # this suite's `wasm-round-trip` test loads. Both cache keys in
      # `.github/actions/build-ffi-binding` hash these, so an edit misses the
      # cache and rebuilds the binding — without these entries it would be
      # rebuilt and no suite that loads it would start. Pinned by
      # scripts/__tests__/wasm-build-inputs-paths-filter.test.mjs.
      - 'packages/eql/crates/**'
      - 'packages/eql/Cargo.toml'
      - '.github/workflows/integration-protect-ffi.yml'
      - '.github/actions/build-ffi-binding/**'
      - '.github/actions/build-auth-binding/**'
      # `@cipherstash/auth` is a workspace package, and the SDK loads its napi
      # module (and, for `wasm-inline`, its wasm) on import, so a change to what
      # it ships arrives here rather than in `pnpm-workspace.yaml`. The paths of
      # require-auth-npm-changeset.yml.
      - 'packages/stack-auth/Cargo.toml'
      - 'packages/stack-auth/src/**'
      - 'languages/typescript/packages/auth/**'
      - 'languages/typescript/packages/stack-auth-wasm/**'
      - '.github/actions/require-cs-secrets/**'
  pull_request:
    branches: ['**']
    # Repeated verbatim: GitHub Actions does not support YAML anchors/aliases.
    # A one-sided edit disables the job on pull requests while leaving it green
    # on `main` — the exact inversion of what you want.
    paths:
      - 'languages/typescript/packages/protect-ffi/integration-tests/**'
      - 'pnpm-workspace.yaml'
      - 'packages/eql/packages/eql/**'
      - 'languages/typescript/packages/protect-ffi/crates/**'
      - 'languages/typescript/packages/protect-ffi/Cargo.toml'
      - 'languages/typescript/packages/protect-ffi/Cargo.lock'
      - 'languages/typescript/packages/protect-ffi/src/**'
      - 'languages/typescript/packages/protect-ffi/scripts/**'
      - 'languages/typescript/packages/protect-ffi/tsconfig.wasm-errors.json'
      - 'languages/typescript/packages/protect-ffi/package.json'
      - 'languages/typescript/packages/protect-ffi/mise.toml'
      # Out of this package, and a real compile input: the cdylib crate carries
      # `eql-bindings = { path = "../../../../../../packages/eql/crates/eql-bindings" }`, so the
      # EQL wire types are linked into `index.node` AND into the wasm32 build
      # this suite's `wasm-round-trip` test loads. Both cache keys in
      # `.github/actions/build-ffi-binding` hash these, so an edit misses the
      # cache and rebuilds the binding — without these entries it would be
      # rebuilt and no suite that loads it would start. Pinned by
      # scripts/__tests__/wasm-build-inputs-paths-filter.test.mjs.
      - 'packages/eql/crates/**'
      - 'packages/eql/Cargo.toml'
      - '.github/workflows/integration-protect-ffi.yml'
      - '.github/actions/build-ffi-binding/**'
      - '.github/actions/build-auth-binding/**'
      # `@cipherstash/auth` is a workspace package, and the SDK loads its napi
      # module (and, for `wasm-inline`, its wasm) on import, so a change to what
      # it ships arrives here rather than in `pnpm-workspace.yaml`. The paths of
      # require-auth-npm-changeset.yml.
      - 'packages/stack-auth/Cargo.toml'
      - 'packages/stack-auth/src/**'
      - 'languages/typescript/packages/auth/**'
      - 'languages/typescript/packages/stack-auth-wasm/**'
      - '.github/actions/require-cs-secrets/**'
  workflow_dispatch: {}

permissions:
  contents: read

defaults:
  run:
    shell: bash

# Ref-scoped, so the only thing it cancels is a superseded commit on the same
# branch — which is what you want for a job that holds live credentials and a
# database for tens of minutes. This is NOT the cross-workflow job-level group
# that `.github/actions/integration-db` was written to remove: that one was
# shared by three workflows, and GitHub cancels a third contender outright
# rather than queueing it, which dropped whole signals.
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  integration:
    name: protect-ffi integration suite
    runs-on: blacksmith-4vcpu-ubuntu-2404
    # Generous, because a cold `build-ffi-binding` compiles the cipherstash
    # client graph from scratch. It is also the backstop for `mise run
    # start-db`, whose readiness loop retries `pg_isready` forever rather than
    # giving up.
    timeout-minutes: 45
    # Everything runs EXCEPT a fork PR, which has no secrets: skip that cleanly
    # rather than fail on something the contributor cannot fix — `tests.yml`
    # still gives them a green signal.
    #
    # Stated as "not a fork PR", NOT as a list of the events allowed to run.
    # The previous form (`github.event_name == 'push' || <same-repo check>`)
    # made the `workflow_dispatch:` above decorative: on a manual dispatch the
    # event name is neither `push` nor `pull_request`, and the payload carries
    # no `pull_request` object to reach through, so both operands were false —
    # the run was created, the only job was skipped, and the run reported
    # success having executed nothing. This is the one credentialed suite with
    # no other way to be re-run on demand: its `paths:` filter means an
    # unrelated commit will not start it either. An allowlist of event names
    # would break again the day someone adds `schedule:`.
    # Enforced by scripts/__tests__/workflow-dispatch-job-conditions.test.mjs.
    if: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository }}

    env:
      # Same split as the other integration workflows: the workspace CRN and
      # client id are repo `vars`, the two keys are `secrets`.
      CS_WORKSPACE_CRN: ${{ vars.CS_WORKSPACE_CRN }}
      CS_CLIENT_ID: ${{ vars.CS_CLIENT_ID }}
      CS_CLIENT_KEY: ${{ secrets.CS_CLIENT_KEY }}
      CS_CLIENT_ACCESS_KEY: ${{ secrets.CS_CLIENT_ACCESS_KEY }}
      #
      # CS_ZEROKMS_HOST / CS_CTS_HOST are deliberately NOT set. Upstream's
      # workflow pinned both to `ap-southeast-2`, which was right for the
      # workspace whose credentials it held and is not portable: the client
      # derives both hosts from the workspace CRN, so hardcoding a region here
      # would silently point this repo's shared credentials at the wrong
      # deployment. No other credentialed workflow in this repo sets them.
      #
      # The PG* block is how `tests/postgres*.test.ts` reach the database:
      # they construct `new Client()` with no arguments, so libpq's environment
      # is the entire connection string.
      #
      # 5436, not 5432. `integration-tests/docker-compose.yml` publishes
      # `5436:5432`, so 5436 is the only port that reaches the container from
      # the runner. Upstream's workflow said 5432 and worked anyway because
      # every invocation went through mise, and `[env]` in
      # languages/typescript/packages/protect-ffi/mise.toml (which says 5436) overrides an inherited
      # value — verified, not assumed. The suite step below runs vitest
      # directly, outside mise, so that safety net is gone and this block has to
      # be right. The values here are identical to mise.toml's, which makes the
      # override a no-op for the steps that do go through mise.
      PGHOST: localhost
      PGPORT: '5436'
      PGDATABASE: cipherstash
      PGUSER: cipherstash
      PGPASSWORD: password

    steps:
      # Not `./.github/actions/integration-setup`, and the difference is
      # deliberate: that action builds the `stash` CLI because the stack
      # integration harness installs EQL v3 by shelling out to it. This suite
      # reads the SQL straight out of the workspace `@cipherstash/eql` instead
      # (see `eql:v3:install` in integration-tests/tasks.toml), so the CLI build
      # would be minutes of runner time for nothing.
      - uses: actions/checkout@v6
        with:
          persist-credentials: false

      - uses: pnpm/action-setup@v6.1.0
        with:
          run_install: false

      # Node 22, not the 24 `.github/actions/integration-setup` defaults to.
      # This used to be an npm-version argument — the suite installed with
      # `npm ci` against a lockfile npm 11 reads more strictly than npm 10 — and
      # that argument died with the lockfile. What is left is the plainer one:
      # 22 is `engines`' floor and what every other workflow in this repo pins,
      # so it is the version this job's Rust binding and WASM artifacts are
      # built and cached against everywhere else. 24 is covered for the JS
      # surface by `tests.yml`'s matrix.
      - uses: actions/setup-node@v6.5.0
        with:
          node-version: 22
          cache: 'pnpm'

      # node-pty's install hook falls back to `node-gyp rebuild` when no
      # linux-x64 prebuild matches. pnpm/action-setup v6 no longer ships
      # node-gyp on PATH, so install it explicitly.
      #
      # Nothing in this job uses a pty. node-pty is a dev-dependency of the
      # `stash` CLI and the repo's one entry in `pnpm.onlyBuiltDependencies`, so
      # its install hook fires on every workspace install — and the 1.1.0
      # tarball ships prebuilds for darwin and win32 ONLY, making the fallback
      # unconditional on a Linux runner. Without this step the job dies in
      # `Install dependencies` with `sh: 1: node-gyp: not found`, before the
      # secrets pre-flight, the binding build or a single test.
      # Enforced by scripts/__tests__/workflow-node-gyp.test.mjs.
      - name: Install node-gyp
        run: npm install -g node-gyp

      # The one install for this job, and it now covers the suite too: since
      # CIP-3744 `integration-tests` is a pnpm workspace member, so this is what
      # puts its `vitest`, `pg` and `dotenv` on disk and symlinks
      # `@cipherstash/protect-ffi` and `@cipherstash/eql` to the packages in
      # this tree. It is also required before the binding build —
      # `build-ffi-binding` runs `pnpm --filter @cipherstash/protect-ffi run
      # build` — and is what puts `@neon-rs/load` in
      # languages/typescript/packages/protect-ffi/node_modules.
      #
      # `mise run setup` below runs the same command again, deliberately: the
      # task has to stand on its own for a contributor running it locally. On
      # the runner it is a no-op.
      - name: Install dependencies
        run: pnpm install --frozen-lockfile

      # Fast pre-flight: fail in seconds if a secret was rotated or cleared,
      # before the binding build and the docker pull. Ordered ahead of anything
      # expensive on purpose — a pre-flight that runs after a cold Rust build is
      # not a pre-flight. Enforced by
      # scripts/__tests__/ffi-binding-step-order.test.mjs.
      - name: Require CipherStash secrets
        uses: ./.github/actions/require-cs-secrets
        with:
          workspace-crn: ${{ vars.CS_WORKSPACE_CRN }}
          client-id: ${{ vars.CS_CLIENT_ID }}
          client-key: ${{ secrets.CS_CLIENT_KEY }}
          client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }}

      # `wasm: 'true'` because `tests/wasm-round-trip.test.ts` and
      # `tests/wasm-error-codes.test.ts` load `dist/wasm/protect_ffi_inline.js`
      # from the built package, and both FAIL rather than skip when it is
      # missing (they `existsSync` it and throw a build hint — the dynamic
      # `import` is deliberate, so the artifact is needed by the time the suite
      # runs rather than at collection). No other step in this job produces it.
      #
      # This replaces upstream's `mise run build:debug`, and the profile change
      # is the point. The loader's fallback is `require('../index.node')` at the
      # package root (see src/load.cts), which is exactly where this action's
      # `neon dist` puts the RELEASE build — so a release binary satisfies the
      # suite, and this action caches it on a content hash of the Rust inputs.
      # Going through the mise task instead would recompile the client SDK in
      # the debug profile on every run and overwrite the artifact the rest of
      # CI already paid for. The action also proves both artifacts LOAD, so a
      # restore that landed nothing fails here naming the file rather than as
      # 19 unrelated encryption errors.
      - name: Build the protect-ffi binding
        uses: ./.github/actions/build-ffi-binding
        with:
          wasm: 'true'

      - name: Build the @cipherstash/auth binding
        uses: ./.github/actions/build-auth-binding
        with:
          wasm: 'true'

      # `working_directory` is load-bearing, not tidiness. mise reads config
      # from the current directory and its PARENTS, so an action running at the
      # repo root never sees languages/typescript/packages/protect-ffi/mise.toml — it would install
      # nothing and leave the config untrusted, and the later `mise run` fails
      # with "Config files ... are not trusted", which reads as a toolchain
      # problem rather than a trust one.
      #
      # A bare `install: true`, unlike `.github/actions/build-ffi-binding` which
      # narrows to wasm-pack with `install_args`. Narrowing would not help here:
      # `mise run` auto-installs any declared tool that is missing (verified),
      # so it would only move the zig / cargo-zigbuild build into the middle of
      # `mise run setup` instead of removing it. Installing up front is what
      # mise-action's cache can absorb.
      #
      # SHA-pinned rather than `@v3`, same pin as
      # .github/actions/build-ffi-binding and .github/workflows/tests-rust.yml:
      # mise-action is a third-party trust dependency the absorption introduced,
      # the tag is mutable, and this job holds live CipherStash credentials.
      # Dependabot moves the pin and the comment together.
      - uses: jdx/mise-action@5228313ee0372e111a38da051671ca30fc5a96db # v3.6.3
        with:
          install: true
          working_directory: languages/typescript/packages/protect-ffi
          # Must name runs-on's image; see scripts/__tests__/workflow-mise-cache-image.test.mjs
          cache_key_prefix: mise-v1-ubuntu-2404

      # One task, four things the suite cannot run without (see
      # integration-tests/tasks.toml): `pnpm install --frozen-lockfile` (a
      # no-op here — the step above already ran it), `docker compose up` for
      # Postgres, the EQL **v2** bundle from a GitHub release, and EQL **v3**
      # built from the in-tree `@cipherstash/eql`.
      #
      # BOTH EQL versions are required and nothing else in this repo installs
      # v2: `tests/postgres.test.ts` creates `eql_v2_encrypted` columns and
      # calls `eql_v2.add_encrypted_constraint`, while
      # `tests/postgres-v3.test.ts` needs the `eql_v3_*` domains. The repo's
      # `.github/actions/integration-db` supplies neither — it brings up a
      # compose stack and hands back a URL, leaving the schema to each suite's
      # own setup (stack's harness shells out to `stash eql install`).
      #
      # That is also why this job does NOT use `integration-db`: the two EQL
      # install tasks pipe SQL through `docker exec -i protect-ffi-postgres`,
      # hardwired to the container name in the suite's own compose file, which
      # `integration-db`'s per-job project names deliberately do not produce.
      # The fixed host port (5436) that comes with it cannot collide with the
      # shared stacks, which publish on ephemeral ports.
      - name: Provision the suite (Postgres, EQL v2 + v3)
        working-directory: languages/typescript/packages/protect-ffi
        run: mise run setup

      # vitest directly rather than `mise run test:integration:all`, because
      # that task's first two steps are the debug cargo build and a wasm-pack
      # rebuild that `build-ffi-binding` has already done (above). What it runs
      # after those is exactly this: vitest over the whole directory, INCLUDING
      # `tests/lock-context.test.ts`. Its sibling `test:integration` excludes
      # that file; upstream CI ran `:all` for a reason, and an invocation that
      # inherits the exclusion silently drops the identity-aware coverage.
      #
      # `pnpm exec`, which resolves the catalog `vitest` the workspace install
      # put in this package's `node_modules/.bin`. It was `npx` against the
      # suite's own `package-lock.json` until CIP-3744 retired that lockfile.
      - name: Integration suite (live ZeroKMS + Postgres, native and WASM)
        working-directory: languages/typescript/packages/protect-ffi/integration-tests
        run: pnpm exec vitest run

      # `-v` drops the volume too. The runner is ephemeral, so this is not about
      # reclaiming it: it is so a re-run on a warm runner cannot inherit a
      # half-installed EQL schema and report a failure that only reproduces
      # there. The default compose project name is the directory name, which is
      # what `mise run start-db` brought the stack up under.
      - name: Stop Postgres
        if: always()
        working-directory: languages/typescript/packages/protect-ffi/integration-tests
        run: docker compose down -v