[rmorozov]

Requested by Ruslan · project thread

Problem

The parity suite compares only against the pinned oracle (Python 3.12, Jinja2 3.1.6, MarkupSafe 3.0.3). A user on another Jinja2 or Python could not ask whether Jinja2C++ matches their oracle. This is the open half of task 0186. The approved plan (sections 4.2-4.3, phase 1) makes the measurement configurable and leaves the library alone.

Resulting behaviour

• JINJA2CPP_TEST_ORACLE=bundled|python|venv (cmake/parity_oracle.cmake, one include() in the test block), plus JINJA2CPP_TEST_ORACLE_PYTHON and JINJA2CPP_TEST_ORACLE_REQUIREMENTS.
  • bundled is the default and registers nothing new. Every CI job stays on bundled.
  • python and venv add a CTest fixture, jinja2cpp_parity_oracle_generate (cmake/parity_oracle_generate.cmake). It builds a venv when needed and runs generate.py --out <build>/parity-oracle --any-version.
  • The venv is rebuilt when the requirements or the base interpreter change.
  • Generation happens at test time, so configure needs no network.
  • Configure warns when the oracle's Python is not the pinned one.
• generate.py --out DIR writes the expectations plus oracle.json, which records the Python, Jinja2 and MarkupSafe versions.
  • --any-version works only together with --out, so expected/ keeps the pin.
  • It refuses Python < 3.10 and Jinja2 < 3.0, and refuses --out expected/.
• parity_test.cpp reads JINJA2CPP_TEST_ORACLE_DIR.
  • Where the two oracles agree, a case is judged by the allow-list exactly as before.
  • Where they differ, the case is reported as tracks-bundled, tracks-oracle, neither or not-run. It fails only under JINJA2CPP_TEST_ORACLE_STRICT=1 (decision O4).
  • The run writes report.md next to the expectations.
  • ParityRegistry.OracleCorpusMatches catches a stale or malformed oracle directory. It compares every rendering input of each case: template, context, env and templates.
  • ParityRegistry.OracleVerdicts tests the classification table.
• Docs: test/parity/README.md, new section "Other oracles". The 0186 task file records progress.
• Filed: 0372, corpus cases on code points that changed after Unicode 15.0. Without them the Python axis is invisible to the oracle rows.
• 0355 (Python module parity) will reuse generate.py --out, JINJA2CPP_TEST_ORACLE_DIR and these verdicts.

Validation

• Bundled mode: the GCC Debug build, ctest -R parity and generate.py --check (py3.12) pass. Regenerating expected/ is byte-identical.
• Oracle runs:

Oracle	Agree	Differ	Verdicts
python, py3.12, jinja2 3.1.6	2850	0	none
venv, jinja2==3.1.2;markupsafe==2.1.5 on py3.12	2841	9	all tracks-bundled
venv, jinja2 3.0.3 on py3.13	2828	22	all tracks-bundled

These were measured on the 2850-case corpus. On master 2fc3fac the corpus has 2940 cases, and the 3.1.2 row is 2931 agree, 9 differ.
The 9 cases are the 8 from plan section 2 plus string_filters.m168. That satisfies section 8's "done when". The 22 match the plan's measurement. STRICT=1 fails the deltas, as designed.

• Sanitizers: a clang-18 ASan+UBSan build of Parity/*:ParityRegistry.* is clean with and without an oracle dir, and under shuffle/repeat.
• Preflight: scripts/preflight.sh is green: clang-format, clang-tidy 22.1.8 on changed lines, task files.
• CI: green on 8025018: 49 passed, 2 skipped, CodeQL neutral.
• Not run here: MSVC and Apple Clang. CI covers them.

Process

• Roles: the main session explored and implemented. The verifier ran one round, as a general-purpose agent following .claude/agents/verifier.md. It found 0 blocking and 4 non-blocking findings, and all were fixed in the second commit:
  • --out swept unrelated *.json files;
  • invalid UTF-8 in report cells;
  • a malformed oracle.json threw at tear-down;
  • misleading venv message.
• One lifetime bug was caught before the first push: a range-for over a Json::parse(...) temporary.
• Red CI pushes: 0.
  • The first push conflicted with master in the CMakeLists test block, next to the new chat-templates test, so GitHub ran no CI on it.
  • Master 2fc3fac was merged once, at the steward's request.
• Codex: 2 findings, 2 fixed, 0 declined. Both were P2 and both were fixed in a984a03:
  • the venv stamp ignored the base interpreter;
  • the corpus check compared only the template.

Remaining

• Phase 2, the nightly parity-oracles.yml with Jinja2 main, 3.0.x and Python 3.13/3.14 rows, follows in a separate PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AWNL7oJQzYFjjoGQPuDCF2

[rmorozov]

@codex review

---

Generated by Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-02T09:21:26.054921Z	8025018	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 802501815a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".