Skip to content

fix: resolve nextest flakiness and expand mise test coverage - #1188

Merged
Manuthor merged 10 commits into
developfrom
test/shared-resource-races
Sep 18, 2026
Merged

Manuthor merged 10 commits into
developfrom
test/shared-resource-races

Conversation

@Manuthor

Copy link
Copy Markdown
Contributor
  • Retry transient SQLite "database is locked" errors during pool instantiation and parameter/state writes, fixing a race exposed by nextest's per-test-process parallelism when a test closes and reopens a KMS on the same sqlite file.
  • Fix a cross-process race in the ckms CLI test binary auto-builder: replace a process-local Once with a cross-process lock file so nextest's many concurrent ckms test processes no longer race to rebuild the binary (was causing intermittent ENOENT failures and redundant cargo build spawns).
  • Expand mise test to run every task under .mise/tasks/test/, auto-skipping groups whose infra/credentials/tools are unavailable (Docker, nix-shell, kubectl/helm/minikube+cluster, cloud creds, the authentication submodule, an OTLP collector, etc.), while avoiding duplicate execution of tasks already covered by a parent task (test:matrix, ui-auth/ui-oidc, spire-pki/spire-sds/ spire-kmip-key-manager/spire-kmip-upstream-authority).
  • Fix test:iris wastefully building a redundant FIPS KMS binary via an unnecessary depends=["build:kms"] before building its own correctly-varianted binary.
  • Add has_cmd/has_env_vars/docker_ready helpers to .mise/lib/common.sh.
  • Include GET /access/obtained wildcard-user (*) permission fix already staged on this branch.

@Manuthor
Manuthor deployed to xks-remote-approval September 11, 2026 15:06 — with GitHub Actions Active
@Manuthor
Manuthor deployed to xks-remote-approval September 12, 2026 08:10 — with GitHub Actions Active
@Manuthor
Manuthor deployed to xks-remote-approval September 12, 2026 09:36 — with GitHub Actions Active
@Manuthor
Manuthor force-pushed the test/shared-resource-races branch from 7a69f4f to 42335ad Compare September 16, 2026 10:51
@Manuthor
Manuthor deployed to xks-remote-approval September 16, 2026 10:52 — with GitHub Actions Active
@Manuthor
Manuthor force-pushed the test/shared-resource-races branch from 42335ad to ea5d5f7 Compare September 16, 2026 12:32
@Manuthor
Manuthor deployed to xks-remote-approval September 16, 2026 12:32 — with GitHub Actions Active
@Manuthor
Manuthor deployed to xks-remote-approval September 16, 2026 13:33 — with GitHub Actions Active
@Manuthor
Manuthor deployed to xks-remote-approval September 16, 2026 15:25 — with GitHub Actions Active
@Manuthor
Manuthor deployed to xks-remote-approval September 16, 2026 15:38 — with GitHub Actions Active
@Manuthor
Manuthor force-pushed the test/shared-resource-races branch from 92dbaf0 to 7786c50 Compare September 16, 2026 21:15
@Manuthor
Manuthor deployed to xks-remote-approval September 16, 2026 21:15 — with GitHub Actions Active
@Manuthor
Manuthor deployed to xks-remote-approval September 17, 2026 04:39 — with GitHub Actions Active
@Manuthor
Manuthor deployed to xks-remote-approval September 17, 2026 09:05 — with GitHub Actions Active
@Manuthor
Manuthor force-pushed the test/shared-resource-races branch from f8c3edf to 6ffc65f Compare September 17, 2026 09:48
@Manuthor
Manuthor deployed to xks-remote-approval September 17, 2026 09:48 — with GitHub Actions Active
- Retry transient SQLite "database is locked" errors during pool
  instantiation and parameter/state writes, fixing a race exposed by
  nextest's per-test-process parallelism when a test closes and
  reopens a KMS on the same sqlite file.
- Fix a cross-process race in the ckms CLI test binary auto-builder:
  replace a process-local Once with a cross-process lock file so
  nextest's many concurrent ckms test processes no longer race to
  rebuild the binary (was causing intermittent ENOENT failures and
  redundant cargo build spawns).
- Expand `mise test` to run every task under .mise/tasks/test/,
  auto-skipping groups whose infra/credentials/tools are unavailable
  (Docker, nix-shell, kubectl/helm/minikube+cluster, cloud creds,
  the authentication submodule, an OTLP collector, etc.), while
  avoiding duplicate execution of tasks already covered by a parent
  task (test:matrix, ui-auth/ui-oidc, spire-pki/spire-sds/
  spire-kmip-key-manager/spire-kmip-upstream-authority).
- Fix test:iris wastefully building a redundant FIPS KMS binary via
  an unnecessary `depends=["build:kms"]` before building its own
  correctly-varianted binary.
- Add has_cmd/has_env_vars/docker_ready helpers to .mise/lib/common.sh.
- Include GET /access/obtained wildcard-user (*) permission fix
  already staged on this branch.
…KMS_TEST_DB backends

Fixes cargo test / cargo nextest failures against shared external
databases (KMS_TEST_DB=postgresql|mysql|redis-findex):

- acquire_shared_test_db_clear_lock now uses an in-process OnceLock
  latch plus a cross-process marker+lock file pair (scoped to one
  NEXTEST_RUN_ID) so only the very first test-server flavor across an
  entire run clears the shared database; every later flavor/process
  reuses the already-bootstrapped state instead of truncating it out
  from under in-flight tests.
- Renamed 5 hardcoded symmetric-key IDs in
  ckms::tests::symmetric::rotation_policy (kst-dec-first,
  kst-dec-gen-n, kst-enc-first, kst-enc-gen-n, kst-invalid-gen) that
  collided verbatim with an independent test_kms_server vector
  manifest using the same IDs.

Verified: KMS_TEST_DB={sqlite,postgresql,mysql,redis-findex} cargo
nextest run --workspace --lib --features non-fips all pass 2085/2085
(51 skipped). cargo clippy-all and cargo fmt --all -- --check clean.
…list_user_operations_granted

zip()-ing two independently-ordered HashMaps (permissions and objects_get
results) instead of joining by key could mismatch a user's granted
operations with the wrong object's owner/state, or silently drop entries
when objects_get returned fewer results than requested. This surfaced in
CI (Test on redis - non-fips) as 'object not found in the granted-access
list' in permissions_granted_includes_wildcard.

Also includes an unrelated mise.toml cleanup (drop unused node/pnpm/usage
[tools] entries not needed for backend testing).
KMIP Locate (used by the Web UI 'Search Objects' page) queried
read_access filtered on the exact requesting user id only, so objects
granted exclusively to the wildcard user '*' were invisible to Locate
even though the user could obtain the grant via GET /access/obtained
and successfully perform the granted operation on the object.

Update the shared SQL query builder (query_from_attributes, used by
SQLite/PostgreSQL/MySQL) to also match read_access rows granted to
'*', consistent with list_user_operations_granted which already
unions direct and wildcard grants. The Redis-findex backend already
resolved permissions via list_user_permissions, which includes
wildcard grants, so it needed no change.

Add a regression test (permissions_find_includes_wildcard) mirroring
the existing permissions_granted_includes_wildcard test.
@Manuthor
Manuthor force-pushed the test/shared-resource-races branch from 6ffc65f to 1984267 Compare September 17, 2026 13:51
@Manuthor
Manuthor deployed to xks-remote-approval September 17, 2026 13:51 — with GitHub Actions Active
@Manuthor
Manuthor force-pushed the test/shared-resource-races branch from 1984267 to 7ad57a2 Compare September 18, 2026 05:08
@Manuthor
Manuthor deployed to xks-remote-approval September 18, 2026 05:08 — with GitHub Actions Active
@Manuthor
Manuthor merged commit 4c916db into develop Sep 18, 2026
77 checks passed
@Manuthor
Manuthor deleted the test/shared-resource-races branch September 18, 2026 06:53

This branch was successfully deployed

1 active deployment
xks-remote-approval 7ad57a27 Deployed Sep 18, 2026 by Manuthor via main / test / AWS XKS — remote server #8665
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant