|
5 | 5 | **The `json_tree` family's read path no longer pays for the document's size per query.** Three changes to the EQL v4 `json_tree` SQL, each measured (cipherstash/benches, `report/JSON_TREE_COMPARISON.md` at 57abcf9) on a 10,000-row corpus of nested documents (about 220 nodes each) where the shipped surface was one to four orders of magnitude behind native `jsonb`: |
6 | 6 |
|
7 | 7 | - **A stored document carries its own containment projection, `q`.** The client writes the document's `{s, vs}` pairs nested by array element beside `sv`, and `eql_v4.json_tree_query(doc)` now reads that field instead of rebuilding the projection from the group tags on every call. The GIN index recheck, which runs the projection once per candidate row, was the whole cost of `@>` on a nested document: 75 ms for ten rows, 16 s for a two-thousand-row result set, 44 s to build the index on 10,000 rows. With the projection stored the same queries measure in the low milliseconds and the index builds in seconds. `json_tree_query` returns `jsonb` rather than the needle domain, since the domain's CHECK would run per recheck; the document's own CHECK validates `q` when the row is written. It is a plpgsql function costed at 1000 rather than an inlinable SQL one: inlined to `value -> 'q'` the planner costs the filter as one operator, and a column of large documents is TOASTed out of the heap, so a sequential scan over it looked nearly free and was chosen over the GIN index while in fact decompressing every row (1.5 s for a one-row match on 10,000 rows; 0.5 to 0.7 ms through the index). The index expression is `(eql_v4.json_tree_query(col)) jsonb_path_ops`; the `::jsonb` cast earlier guidance wrote is harmless and no longer needed. **Wire change:** `q` is required in `public.eql_v4_json_tree`, and a document written by the earlier build of this unreleased family is refused by the domain CHECK. |
8 | | -- **The token-chain walk reads the document once.** `json_tree_subtree`, `json_tree_subtree_at`, `json_tree_children` and `json_tree_child_count` computed each parent tag inside a scan of `sv`, once per (node, ordinal, entry); a 244-entry subtree took ten seconds. The walk now builds one map of the document's entries keyed by parent tag and looks each computed tag up in it, so a level costs its fan-out in SHA-256 evaluations and the document is never scanned per node. The same subtree walks out in 23 ms; `children` and `child_count` in 1.6 ms. The walk is also bounded by the document's size: it probes at most as many ordinals as the document has entries and returns at most as many rows, so a stored document whose unauthenticated chain fields claim a huge ordinal, or wrap a child's token back to its parent's, can no longer keep a query running until `statement_timeout`. |
9 | | -- **A walk has a budget, and past it returns the sealed root.** `json_tree_subtree(doc, token, budget DEFAULT 64)` and `json_tree_subtree_at(doc, token, selector, budget DEFAULT 64)` stop once the subtree exceeds `budget` entries and return the root entry alone, with `d = 0`, when the document's root is sealed: the root's ciphertext is the whole document at about the plaintext's size, where the walk returns some four hundred characters per entry and pays the fan-out in hashes per node, so past a dozen or so entries the whole document is cheaper on both sides. The client opens the root and takes the walked path over it, and gets what the walked rows open to: attach the path with `below(path)` for `json_tree_subtree` rows and `at(path)` for `json_tree_subtree_at` rows (`JsonSubtree` in `stack-encrypt`, `JsonTreeSubtree::new(rows)` in `eql-bindings`). Rows that fell back and carry no path refuse to open. A budget of `0` never falls back; a token that names no node still yields no row; a document whose root is not sealed is always walked out. |
| 8 | +- **The token-chain walk reads the document once.** `json_tree_subtree`, `json_tree_subtree_at`, `json_tree_children` and `json_tree_child_count` computed each parent tag inside a scan of `sv`, once per (node, ordinal, entry); a 244-entry subtree took ten seconds. The walk now builds one map of the document's entries keyed by parent tag and looks each computed tag up in it, so a level costs its fan-out in SHA-256 evaluations and the document is never scanned per node. The same subtree walks out in 23 ms; `children` and `child_count` in 1.6 ms. The walk is also bounded by the document's size, so a stored document whose unauthenticated chain fields claim a huge ordinal, or loop back to an ancestor, can no longer keep a query running until `statement_timeout`. |
| 9 | +- **A walk has a budget, and past it returns the sealed root.** `json_tree_subtree(doc, token, budget DEFAULT 64)` and `json_tree_subtree_at(doc, token, selector, budget DEFAULT 64)` stop once the subtree exceeds `budget` entries and return the root entry alone, with `d = 0`, when the document's root is sealed: the root's ciphertext is the whole document at about the plaintext's size, where the walk returns some four hundred characters per entry and pays the fan-out in hashes per node, so past a dozen or so entries the whole document is cheaper on both sides. The client opens the root and takes the walked path over it, and gets what the walked rows open to. The rows are opened through the key that keyed the walk, which carries the path it was derived from and knows from its gates which walk the rows came from: `key.open(rows)` in `stack-encrypt`, `JsonTreeSubtree::from_key(&key, rows)` in `eql-bindings`. Rows cannot be opened without it, so a document that grows past the budget does not break a caller. A budget of `0` never falls back; a token that names no node still yields no row; a document whose root is not sealed is always walked out. |
10 | 10 |
|
11 | 11 | Also: `eql_v4.json_tree_path_query_first`, the function `->` and the ordering comparisons extract through, is costed at 1000 so a prepared statement's generic plan keeps the functional btree on `eql_v4.ord_term(col -> sel)` instead of filtering a sequential scan (16.9 ms down to 0.11 ms after the fifth execution). `eql_v4_internal.json_tree_group`, the server-side projection builder, is removed. |
12 | 12 |
|
13 | | -The `eql-bindings` crate's `JsonTreeDocument` gains the `q` field and `JsonTreeSubtree` becomes a struct with `rows`, an optional `path` and `below`; `stack-encrypt`'s `JsonDocument` writes `q` on the wire and ignores it on the way in, `JsonPath::select` evaluates a path over a value, and `JsonSubtree::below`, `at`, `path` and `is_sealed_root` carry the fallback. EQL v4 only; the v3 bundle is unchanged. |
| 13 | +The `eql-bindings` crate's `JsonTreeDocument` gains the `q` field and `JsonTreeSubtree` becomes a struct of the rows and the key that keyed the walk; `stack-encrypt`'s `JsonDocument` writes `q` on the wire and ignores it on the way in, `JsonPath::select` evaluates a path over a value, and `JsonSubtreeKey::path` and `open`, `JsonSubtree::path` and `is_sealed_root` carry the fallback. EQL v4 only; the v3 bundle is unchanged. |
| 14 | + |
| 15 | +**The leaf ordering term `op` is opt-in per path.** A stored leaf carried the order-preserving term on every number and string, about 80 bytes each and the largest field of a nested document, whether or not anything ordered by it. Now the JSON index's options name the paths that order (`{"json": {"ordered": ["$.age", "$.orders[*].total"]}}` on the data plan; `JsonOptions::ordering([...])` in Rust), a wildcard covering every element of an array, and no path is ordered by default. A position in an ordered path (`$.scores[2]`), or in the path an ordering asks at, is refused: the comparison extracts by selector, which reads a position as the wildcard and reaches the first element, never the one named. An ordering query at a path the index does not order is refused before any key loads rather than silently matching no row. The SQL is unchanged: `eql_v4.ord_term` was already NULL for a leaf without a term. `eql-bindings` gains `v4::JsonTree`, the `json_tree` layout under options, and `stack-encrypt` gains `target::Layout` and `Plan::value().encrypt_with_layout(layout)`, a plan laid out by a value whose queries run under its options. |
0 commit comments