Shared golden tests for the jMotif SAX stack across Java (jMotif/SAX), R (jMotif/jmotif-R), and Python (seninp/saxpy).
saxpy is the reference generator for expected values. Each implementation runs the same case definitions and is checked against the committed expectations.
Published targets (Jul 2026): saxpy 2.0.1, jmotif 1.3.2, jmotif-sax 2.0.2. Tier-B RRA pins use bundled ecg0606 with seed = 0.
These tables measure Java end-to-end wall time on GrammarViz’s RRA and HOT-SAX paths. They are not golden conformance checks — tier-B RRA asserts region overlap, not identical spans or run time. NN distances between the two algorithms are not comparable (fixed-window z-norm vs length-normalized grammar spans).
Regenerate after stack upgrades (requires ./scripts/bootstrap.sh first):
./scripts/bench_rra_hotsax.sh --update-readmeCase definitions live in cases/bench_rra_hotsax.json. Long tiled rows repeat chfdbchf15_1.csv with a tiny per-cycle drift so cycles are not exact clones (verbatim tiling can stall HOT-SAX).
What drives the crossover. HOT-SAX prunes with SAX-word frequencies but still traverses the full word index. RRA pays upfront parallel SAX, Re-Pair, and rule-interval construction, then evaluates fewer variable-length grammar-rule candidates. On short series that fixed cost dominates wall-clock, which is why the ecg0606 rows below favor HOT-SAX. On the tiled long rows (n ≥ 50k), RRA wins clearly. Between those extremes the outcome depends on (window, PAA, alphabet), hardware, and SAX-string compressibility — at n ≈ 15k either algorithm can be faster on ECG-like data, as the chfdbchf15 row shows.
Generated 2026-07-23 09:00 UTC by ./scripts/bench_rra_hotsax.sh --update-readme (GrammarViz grammarviz2_src, jmotif-sax jmotif-java, k=1, seed=0, NR=NONE, z=0.01).
Short series (ecg0606, n = 2,299)
| Parameters | HOT-SAX | RRA | RRA / HOT-SAX |
|---|---|---|---|
w=120, p=4, a=4 |
50 ms | 76 ms | 1.52× slower |
w=100, p=4, a=4 |
45 ms | 77 ms | 1.71× slower |
w=150, p=7, a=4 |
91 ms | 82 ms | 0.90× faster |
Longer ECG-like series (w=100, p=4, a=4)
| Series | n |
HOT-SAX | RRA | RRA / HOT-SAX |
|---|---|---|---|---|
| ecg0606 baseline | 2,299 | 43 ms | 82 ms | 1.91× slower |
| chfdbchf15 | 15,000 | 235 ms | 317 ms | 1.35× slower |
| chfdb tiled+drift 50k | 50,000 | 10.5 s | 2447 ms | 0.23× faster |
| chfdb tiled+drift 100k | 100,000 | 20.8 s | 7776 ms | 0.37× faster |
Ratios below 1.0× (shown as “faster”) mean RRA had lower wall-clock on that row.
Mirror on the GrammarViz anomaly tutorial §2.4.
Informative comparison of the two GI algorithms as grammar backends for RRA — not a
conformance check. Both algorithms consume the identical single-threaded SAX
discretization per cell, so differences are attributable to grammar induction alone.
Driver: drivers/java/SequiturVsRepairBench.java
(compile against a GrammarViz fat jar + GI classes, like RRAvsHotsaxBench; --dataset,
optional --tile-length). Long real series live in grammarviz2_src/data/ — copy them under
a datasets/ dir and point --repo-root at its parent.
Measured 2026-07-26 on Apple Silicon, JDK 21, GI master (post de-static refactor 21b8ca9),
grid w×p×a ∈ {100,120,160,200}×{4,6,8}×{4,6} (6 cells), k=3, seed=0, NR=EXACT, z=0.01.
Full per-cell JSON in the bench output; summary below is the grid trend, not a single cell.
Grammar inference wall-clock (the consistent Sequitur win — grows with series length)
| Series | n |
Sequitur infer | Re-Pair infer | speedup |
|---|---|---|---|---|
| mitdbx_108 (ECG) | 21,600 | 5–17 ms | 10–41 ms | ~2–6× |
| dutch_power_demand | 35,039 | 11–31 ms | 22–100 ms | ~2–6× |
| ap1 (smooth AP) | 100,001 | 0–3 ms | 1–4 ms | — (tiny grammars) |
| usgs_james_river | 245,839 | 24–149 ms | 127–989 ms | ~4–7× |
| 318_signal1 (physio) | 586,086 | 0.1–2.3 s | 0.4–11.4 s | ~3–5× |
Grammar shape. On real (non-tiled) series Re-Pair is consistently more compact: Sequitur emits ~1.2–1.5× more rules at a comparable size proxy, in a flatter hierarchy. (On drift-tiled periodic data this inverts — Sequitur captures long exact repeats with far fewer rules — so tiled benchmarks overstate Sequitur's compactness.)
Downstream RRA. Mixed: on the largest series RRA over Re-Pair's more compact grammar is
often faster (318_signal1 w=200: 17.1 s vs 29.8 s), which can erase Sequitur's inference
win end-to-end; on mid-size series the backends are comparable. Top-1 discord positions
disagree between backends in nearly every cell (both usually land in the same anomalous
neighborhood, e.g. mitdbx ~10,050–11,160). Grammar choice is a real RRA hyper-parameter, not
an implementation detail — discord-quality scoring against annotated ground truth is the
natural follow-up before preferring either.
Parallel sweeps. Sequitur inferences parallelize cleanly since the de-static refactor
(isolated SequiturGrammar contexts): 6-thread sweep of the grid runs ~2.4–3.5× faster
than sequential on every series tested.
These cases assume the 2.0.0 alignment rules:
- z-normalization uses the population standard deviation (divide by
n) - PAA uses fractional segment boundaries
- a value exactly on a Gaussian breakpoint maps to the symbol above the cut
- discord search compares z-normalized subsequences
- distance ties break on the lowest index
- trivial matches within ±(
window− 1) are excluded
cases/ JSON case definitions + expected outputs
datasets/ shared CSV inputs
drivers/ thin per-language runners
scripts/ bootstrap, run_all, export_goldens
Sibling checkout layout:
~/git/
jmotif-conformance/
SAX/ # or jmotif-java/
GI/
sax-vsm_classic/
grammarviz2_src/ # or GrammarViz2/
jmotif-R/
saxpy/
cd ~/git/jmotif-conformance
./scripts/bootstrap.sh # build/install all three implementations
./scripts/run_all.sh # run every case × every aligned implementation
./scripts/bench_rra_hotsax.sh --update-readme # optional: refresh RRA vs HOT-SAX tables in this READMERun a single implementation or case:
./scripts/run_all.sh --impl python
./scripts/run_all.sh --impl java --case cases/discord_bruteforce_ecg800.jsonWhen all three implementations agree on an intentional behavior change, refresh expectations from saxpy:
cd saxpy && uv pip install -e .
cd ../jmotif-conformance
python3 scripts/export_goldens.py
git diff cases/Review the diff carefully before committing.
Every operation runs on Java × R × Python and is checked against committed expectations. Cases fall into two tiers by how tightly the languages can agree:
- Tier A — exact. Deterministic outputs are compared value-for-value (integer positions and strings exactly; floats within an absolute tolerance).
- Tier B — region-level. RRA uses variable-length grammar-rule intervals, so exact spans legitimately differ; agreement is asserted on the anomaly region with a substantial-overlap threshold plus a ground-truth anchor and a cross-language consensus check.
| Operation | Tier | Datasets | Checked exactly (conform) | Tolerance |
|---|---|---|---|---|
sax_via_window (NONE) |
A | synthetic_window_60, ecg0606 |
SAX word for every window index | exact |
discord_bruteforce |
A | ecg0606 (full / slice / first-800) |
discord position; nn_distance |
nn_distance 1e-6 |
discord_hotsax (NONE) |
A | ecg0606 |
discord position + nn_distance; HOT-SAX top ∈ brute-force top discord |
nn_distance 1e-6 |
repair |
A | abab, paper, ecg_sax, jmotif_r_bugs |
R0 has no repeated digram; decompress == input; exact rule strings/IDs on tie-free inputs | exact |
saxvsm_classify |
A | CBF, Gun_Point | correct, total, accuracy, error |
accuracy 1e-12 |
rra_discord |
B | ecg0606 (window 100, 120) |
see tier-B table below | ≥ 50% overlap |
RePair rule numbering is compared exactly only on tie-free inputs (paper
example, a b a b). Long SAX strings check the decompression round-trip and the
no-repeated-digram guarantee on R0 instead of per-rule IDs.
SAX-VSM cases pin train/test accuracy (correct, total, accuracy, error)
at operating points aligned with sax-vsm 2.0.1 / saxpy 2.0.1.
RRA (Rare Rule Anomaly) grammar-rule intervals vary in length across implementations, so agreement is checked at the region level. Each check uses a ≥ 50% of window overlap threshold (not a single-sample touch):
| Checked (conform) | Not checked (de-conform) |
|---|---|
| RRA top span overlaps the HOT-SAX discord window on ecg0606 by ≥ 50% of the window | Exact start / end of the top discord |
RRA top span overlaps the ground-truth anomaly region [400, 560) by ≥ 50% of the window |
Whether index 430 lies strictly inside the RRA span (Java NewRepair can shift by one vs saxpy/R) |
| All three languages' RRA top spans mutually overlap each other by ≥ 50% of the window (consensus) | rule_id of the winning interval |
| Robustness across two window sizes (100, 120) | Exact nn_distance (early-abandon is approximate) |
| Distance-call count / search trajectory | |
| Multi-discord ordering beyond the primary region |
Thresholds are conservative: on the pinned stack the observed overlaps are 0.84–1.0, so a 0.5 floor leaves margin for the documented one-sample RePair shift while still failing a genuinely divergent region (e.g. window 150, where Java's top region jumps elsewhere).
The Java driver follows the saxpy / jmotif-R pipeline (RePair on the composed SAX string, saxpy-style zero-coverage filtering, seeded phase-2 shuffle). GrammarViz’s pruned-RRA CLI path is intentionally not used here.
To block accidental commits of Cursor/Claude/audit artifacts, run once per clone:
./scripts/install-git-hooks.sh.gitignore already excludes these paths; the hook is a second line of defense.
- JDK 21+
- Maven 3.9+
- R 4.0+ with Rcpp toolchain
- Python 3.10+ with numpy/scipy (saxpy dependencies)
- optional: uv for Python env management
GPL-2.0 — see LICENSE.