Skip to content

Commit d80a33a

Browse files
committed
Merge branch 'feat/json-index-stack-encrypt' into perf/json-tree-walk-and-projection
2 parents 0a3f64b + 6427424 commit d80a33a

10 files changed

Lines changed: 23 additions & 23 deletions

File tree

‎.changeset/eql-json-tree-domain.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,4 @@
44

55
**EQL v4 gains the `json_tree` domain family: `public.eql_v4_json_tree`, an encrypted JSON document stored as one entry per node, with `@>` and `<@` that agree with PostgreSQL's own `jsonb @>` across array elements.** The family is new beside `public.eql_v4_json` and is not a port of the v3 SteVec surface: it is what Stack Encrypt's JSON index writes. A document's entries each carry a path selector `s` and a value selector `vs` (containment is a set comparison over the pair), the node's own ciphertext `c` (its label, kind and scalar value; a document's ciphertext is linear in its size, where a SteVec document repeats every subtree at every level), and a token chain (`p`, `w`, `o`) that lets core PostgreSQL walk the subtree under a node from one client-derived token: `eql_v4.json_tree_subtree(doc, token)`, `json_tree_subtree_at(doc, token, selector)`, `json_tree_children` and `json_tree_child_count`. By default the root is sealed, so no query can hand the server a token for the whole column. Containment projects a stored document into an element-nested shape from its group tags (`n`, `g`), `eql_v4.json_tree_query(doc)`, and compares it with the needle (`eql_v4.query_json_tree`) by native `jsonb @>`, so each needle array element must fit inside one stored element, which closes the cross-element false positives of selector-set containment (#1231) for this family; a GIN index on `(eql_v4.json_tree_query(col)::jsonb) jsonb_path_ops` serves it. `->` by selector returns an entry that opens on its own (`public.eql_v4_json_tree_entry`, with the header and ordinal grafted), and `eql_v4.query_json_tree_leaf` orders an extracted number or string leaf with `<`, `<=`, `>`, `>=`. Every other native jsonb operator on the column is blocked, as on the other EQL domains. EQL v4 only (`-- ONLY EQL VERSION: 4`); the v3 bundle is unchanged. The `eql-bindings` crate carries the hand-written `v4::json_tree` types with Stack Encrypt's `EncryptFrom` / `DecryptInto` for the document and the entry.
66

7-
Why. Stack Encrypt had no JSON index: the operators (`Contains`, `ContainedBy`, `AtPath`) were declared and answered by nothing, and the v4 JSON wire format was unconstrained by anything shipped. This is the spike's Phase 1 and 2 (#1249), implemented against `docs/plans/2026-10-17-json-index-stack-encrypt-spike.md`, whose "Implementation notes" section records where the code deviates from the design.
7+
Why. Stack Encrypt had no JSON index: the operators (`Contains`, `ContainedBy`, `AtPath`) were declared and answered by nothing, and the v4 JSON wire format was unconstrained by anything shipped. This is Phase 1 and 2 of the `json_tree` index's phased plan (#1249), implemented against `docs/plans/2026-10-17-json-tree-index.md`, whose "Implementation notes" section records where the code deviates from the design.

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -689,7 +689,7 @@ monorepo, which is where the silent failures are.
689689
oracle, containment over the encrypted column against PostgreSQL's own
690690
`jsonb @>` over the plaintexts, is `packages/eql/tests/encryption/tests/json_tree.rs`,
691691
run by `mise run test:encryption:postgres`. The design and its
692-
implementation notes are `docs/plans/2026-10-17-json-index-stack-encrypt-spike.md`.
692+
implementation notes are `docs/plans/2026-10-17-json-tree-index.md`.
693693
A change to the derivations on one side (a PAE domain, the header layout,
694694
the projection shape) is a change to the other: the Postgres suite is
695695
what holds them together.

docs/plans/2026-10-17-json-index-stack-encrypt-spike.md renamed to docs/plans/2026-10-17-json-tree-index.md

Lines changed: 17 additions & 17 deletions
Large diffs are not rendered by default.

docs/plans/2026-10-17-json-index-stack-encrypt-spike/containment-oracle.sql renamed to docs/plans/2026-10-17-json-tree-index/containment-oracle.sql

File renamed without changes.

docs/plans/2026-10-17-json-index-stack-encrypt-spike/jsonb-semantics.sql renamed to docs/plans/2026-10-17-json-tree-index/jsonb-semantics.sql

File renamed without changes.

docs/plans/2026-10-17-json-index-stack-encrypt-spike/token-chain.sql renamed to docs/plans/2026-10-17-json-tree-index/token-chain.sql

File renamed without changes.

‎packages/eql/AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ Where the two versions' SQL must differ, the source holds both forms in **versio
6060

6161
#### Files confined to one EQL version
6262

63-
A source file belongs to every version unless it carries the line `-- ONLY EQL VERSION: N` (the payload version, not the schema name, which the source may not write). Two surfaces do today. The generated range surface: `src/v3/scalars/<T>/query_<T>_range_types.sql` and `query_<T>_range_functions.sql` hold the range query domains (`eql_v4.query_<T>_range_ore` / `_range_ope`), their `lo` / `hi` accessors and `eql_v4.between(column, range)` (ADR-0009 decision 7). And the hand-written JSON tree family under `src/v3/json_tree/` (`types.sql`, `functions.sql`, `operators.sql`, `blockers.sql`): `public.eql_v4_json_tree`, `public.eql_v4_json_tree_entry`, `eql_v4.query_json_tree` and `eql_v4.query_json_tree_leaf`, the document Stack Encrypt's JSON index writes, a new family beside `public.eql_v4_json` and not a port of the SteVec surface (design: `docs/plans/2026-10-17-json-index-stack-encrypt-spike.md` at the repository root; bindings: `crates/eql-bindings/src/v4/json_tree.rs`, hand-written like the SteVec structs; the family is not in the catalog yet). Both are `-- ONLY EQL VERSION: 4`. EQL v3 is what cipherstash-client writes, nothing there produces a range or a JSON tree document, and its SQL must not change. `tasks/build.sh` runs the install order through `eql-codegen select <3|4>` before rendering each version, `eql-codegen render` refuses to render a confined file as the other version, `verify_installer_complete.sh` filters its disk side the same way with its own `grep`, the symbol-order checker runs on the v4 order too (on a copy mapped back to `eql_v3` names), the Doxygen filter reads a confined file from its own version's rendered tree, and `docs:validate:documented-sql` runs the whole v4 install order (`build/render/eql_v4/deps-ordered.txt`) in one rolled-back transaction, so every statement of the confined files runs after what it requires, besides running each file of the v3 render on its own. `eql-codegen order` still orders every file, so `src/deps-ordered-v3.txt` lists the v4-only files; the per-version lists are `build/render/eql_v<N>/deps-ordered.txt`.
63+
A source file belongs to every version unless it carries the line `-- ONLY EQL VERSION: N` (the payload version, not the schema name, which the source may not write). Two surfaces do today. The generated range surface: `src/v3/scalars/<T>/query_<T>_range_types.sql` and `query_<T>_range_functions.sql` hold the range query domains (`eql_v4.query_<T>_range_ore` / `_range_ope`), their `lo` / `hi` accessors and `eql_v4.between(column, range)` (ADR-0009 decision 7). And the hand-written JSON tree family under `src/v3/json_tree/` (`types.sql`, `functions.sql`, `operators.sql`, `blockers.sql`): `public.eql_v4_json_tree`, `public.eql_v4_json_tree_entry`, `eql_v4.query_json_tree` and `eql_v4.query_json_tree_leaf`, the document Stack Encrypt's JSON index writes, a new family beside `public.eql_v4_json` and not a port of the SteVec surface (design: `docs/plans/2026-10-17-json-tree-index.md` at the repository root; bindings: `crates/eql-bindings/src/v4/json_tree.rs`, hand-written like the SteVec structs; the family is not in the catalog yet). Both are `-- ONLY EQL VERSION: 4`. EQL v3 is what cipherstash-client writes, nothing there produces a range or a JSON tree document, and its SQL must not change. `tasks/build.sh` runs the install order through `eql-codegen select <3|4>` before rendering each version, `eql-codegen render` refuses to render a confined file as the other version, `verify_installer_complete.sh` filters its disk side the same way with its own `grep`, the symbol-order checker runs on the v4 order too (on a copy mapped back to `eql_v3` names), the Doxygen filter reads a confined file from its own version's rendered tree, and `docs:validate:documented-sql` runs the whole v4 install order (`build/render/eql_v4/deps-ordered.txt`) in one rolled-back transaction, so every statement of the confined files runs after what it requires, besides running each file of the v3 render on its own. `eql-codegen order` still orders every file, so `src/deps-ordered-v3.txt` lists the v4-only files; the per-version lists are `build/render/eql_v<N>/deps-ordered.txt`.
6464

6565
Anything that needs the source as real SQL reads the rendered tree, not `src/`: `docs:validate:documented-sql` runs psql over `build/render/eql_v3/src`, and the Doxygen input filter (`tasks/docs/doxygen-filter.sh`) reads each file's rendered copy from `build/render/eql_v3/`. A new tool that parses `src/v3` must do the same, and depend on `build`; do not substitute the placeholders a second way.
6666

‎packages/eql/crates/eql-bindings/src/v4/json_tree.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616
//! .context("users/profile").encrypt_into::<JsonDocument>()`, and converts
1717
//! into the operand types here; the identifier and the two-segment label
1818
//! are one context, so the selectors agree (see [`crate::encryption`]). The
19-
//! design is `docs/plans/2026-10-17-json-index-stack-encrypt-spike.md` at
19+
//! design is `docs/plans/2026-10-17-json-tree-index.md` at
2020
//! the repository root.
2121
2222
use schemars::{schema_for, JsonSchema, Schema};

‎packages/eql/src/v3/json_tree/types.sql‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
--! - {{prefix}}.query_json_tree_leaf — an ordering operand at a path: a
1919
--! selector and a leaf ordering term.
2020
--!
21-
--! The design is `docs/plans/2026-10-17-json-index-stack-encrypt-spike.md`
21+
--! The design is `docs/plans/2026-10-17-json-tree-index.md`
2222
--! at the repository root. The document's per-entry fields:
2323
--! `s` path selector (16 bytes, hex): every element of an array shares its
2424
--! parent's, so a path reads `$.a[*].x`.

‎packages/stack-encrypt/src/json/mod.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
//! The JSON index: a document stored as one searchable value, with
22
//! node-local ciphertexts, a token chain and exact containment.
33
//!
4-
//! `docs/plans/2026-10-17-json-index-stack-encrypt-spike.md` is the design
4+
//! `docs/plans/2026-10-17-json-tree-index.md` is the design
55
//! this module implements, and the authority on why. In short, a document
66
//! is flattened into one entry per node. Each entry carries:
77
//!

0 commit comments

Comments
 (0)