Merge pull request #1204 from cipherstash/test/eql-selector-review-followups

test(eql): review follow-ups to the SteVec selector derivation

Author: tobyhede

AGENTS.md

@@ -570,17 +570,33 @@ monorepo, which is where the silent failures are.
   `fixtures::v3_doc_integer::field_op_selector` and
   `fixtures::v3_ste_vec::hello_op_selector`, which apply a written rule to the
   loaded fixture and fail, listing every candidate, when zero or several
-  selectors match. `tests/sqlx/tests/selector_literal_guard.rs` fails on a
-  random-looking standalone 32-hex run anywhere in the crate's source, SQL and
-  escaped JSON strings included, and runs DB-free in `mise run test:schema` so
-  fork PRs get it too; a forged selector must be
-  visibly fake (`"00000000000000000000000000000001"`). The suite passes against
-  any workspace. That a keyset and path still produce the same selector bytes is
-  pinned separately, in the encryption test crate
+  selectors match. The jsonb_entry matrix resolves the selector at runtime and
+  queries the literal `payload -> '<sel>'::text` customers write.
+  `tests/sqlx/tests/selector_literal_guard.rs` fails on a random-looking
+  standalone 32-hex run, in any case, in any `.rs` file under the crate's `src/`
+  and `tests/`, SQL and escaped JSON inside Rust strings included, and counts a
+  string escape (`\n`, `\x`), a `0x` prefix or an integer suffix (`0x…u128`)
+  as a boundary. It does not catch a selector split into pieces, such as a
+  `[u8; 16]` array of `0x..` bytes; the generated
+  `fixtures/*.sql` files hold real selectors and are not scanned. It runs
+  DB-free in `mise run test:schema` so fork PRs get it too; a forged selector
+  must be visibly fake (`"00000000000000000000000000000001"`). The suite passes
+  against any workspace. That a keyset and path still produce the same selector
+  bytes is pinned separately, in the encryption test crate
   (`packages/eql/tests/encryption/tests/ste_vec_selectors.rs`, regenerated only
-  with `EQL_UPDATE_STE_VEC_SELECTORS=1`). It pins the cipherstash-client version
-  the EQL workspace resolves; protect-ffi resolves its own from its own
-  lockfile, so the pin covers protect-ffi's only while the two versions agree.
+  with `EQL_UPDATE_STE_VEC_SELECTORS=1`, exactly `1`). It pins the index config
+  production really sends for an EQL v3 JSON column, the unrewritten
+  `prefix: "enabled"` sentinel included (#1203: a fix there changes every stored
+  v3 JSON selector and is meant to fail the pin). It also records the fake index
+  key it derives under: a changed key fails first, with a message naming it a
+  stack-kms test-infrastructure change rather than a stored-data break. It pins
+  the cipherstash-client and cipherstash-config (the crates that compute the
+  selectors) the EQL workspace resolves; protect-ffi resolves its own from its
+  own lockfile, and `scripts/__tests__/cipherstash-client-parity.test.mjs`
+  (`pnpm run test:scripts`, every PR) fails when the two lockfiles disagree on
+  the version, source or checksum of either, so a protect-ffi bump has to move the EQL pins with it. Primitive
+  dependencies such as blake3 are not compared, and already differ at patch
+  level.
 - **`FROZEN_PUBLISHERS` is what stops this repo publishing a package whose
   publisher lives elsewhere, and it is a mechanism, not a status.** `scripts/release-gate.mjs` carries a map of every
   package that lives here but is published from another repository. For each,

packages/eql/Cargo.lock

@@ -32,10 +32,14 @@ postgres = "0.19"
 # depend on lives in cipherstash-client, which protect-ffi encrypts through.
 # Its `JsonIndexer` runs offline over a supplied index key. The same exact pin
 # tests/sqlx names, so this workspace's lockfile gains no second copy.
-# protect-ffi resolves cipherstash-client from its own workspace and lockfile,
-# so the pin covers the version named here, not necessarily the one protect-ffi
-# ships: a protect-ffi bump is covered once this line moves to match it.
-cipherstash-client = "=0.42.0"
+# protect-ffi resolves cipherstash-client from its own workspace and lockfile;
+# scripts/__tests__/cipherstash-client-parity.test.mjs fails on every PR where
+# the two lockfiles disagree on cipherstash-client or cipherstash-config (the
+# crates that compute selectors), so a protect-ffi bump must move this line
+# (and tests/sqlx's) with it, and the pin then covers the versions of those two
+# crates protect-ffi ships. Primitive dependencies such as blake3 are not
+# compared.
+cipherstash-client = "=0.42.3"
 # The tampering test decodes a stored body, changes one byte and re-encodes
 # it. The same line eql-bindings uses, so the lockfile gains no second copy.
 base64 = "0.22"

packages/eql/tests/encryption/Cargo.toml

@@ -31,10 +31,13 @@
 //! document covers every JSON shape the indexer branches on: numbers (integer
 //! and float), a string, a bool, null, a nested object and an array.
 //!
-//! What it pins is the cipherstash-client version this EQL workspace resolves.
-//! protect-ffi, which writes customers' documents, resolves cipherstash-client
-//! from its own workspace and lockfile, so a protect-ffi bump is covered only
-//! once this workspace moves to the same version.
+//! What it pins is the cipherstash-client and cipherstash-config (which parses
+//! the index config) this EQL workspace resolves. protect-ffi, which writes
+//! customers' documents, resolves both from its own workspace and lockfile.
+//! `scripts/__tests__/cipherstash-client-parity.test.mjs` fails on every PR
+//! where the two lockfiles disagree on either, so the versions pinned here are
+//! the ones protect-ffi ships. Primitive dependencies such as blake3 (the
+//! selector MAC) are not compared.
 //!
 //! Regenerate with `EQL_UPDATE_STE_VEC_SELECTORS=1` (exactly `1`; any other
 //! value, `0` included, regenerates nothing), deliberately not the crate's

packages/eql/tests/encryption/tests/ste_vec_selectors.rs

@@ -17,7 +17,7 @@ jsonschema = { version = "0.46.4", default-features = false }
 # The published client is the encryption oracle for fixtures and fresh e2e
 # queries. Pin it exactly so its SteVec envelope cannot drift independently of
 # the strict bindings and SQL contract tested in this repository.
-cipherstash-client = { version = "=0.42.0", features = ["tokio"] }
+cipherstash-client = { version = "=0.42.3", features = ["tokio"] }
 # chrono is already in the tree transitively (cipherstash-client / ore-rs); pin
 # it as a direct dependency so the harness can name `chrono::NaiveDate` for the
 # `date` scalar (Encode/Decode/Type come from the sqlx `chrono` feature above).

packages/eql/tests/sqlx/Cargo.toml

@@ -104,7 +104,9 @@ from the fixture it loaded.
 Each states a rule that singles out one selector and fails, listing every
 candidate, when zero or several match (`src/fixtures/ste_vec_selector.rs`).
 `tests/selector_literal_guard.rs` fails if a selector-shaped hex literal is
-added to `src/` or `tests/`. A forged selector in a literal-payload test must
+added to a `.rs` file under `src/` or `tests/`, including SQL and escaped JSON
+inside Rust strings. The generated `fixtures/*.sql` files hold real selectors
+and are not scanned. A forged selector in a literal-payload test must
 be visibly fake (for example `"00000000000000000000000000000001"`).
 
 ## Writing Tests