עדכון דלתא: hash תוכן לכל טבלה במניפסט - #27
Merged
Conversation
…eContentHashes) LogicalContentHasher.computeReport מחשב במעבר יחיד, בשני digests, את ה-hash הכולל ואת ה-hash של כל טבלה בנפרד (sha256 של זרם הבתים של הטבלה בלבד, כולל הקידומת). compute() לא השתנה — הבתים שמוזנים ל-digest הכולל זהים. ReleaseManifestWriter פולט את שתי המפות כאובייקטי JSON בסדר טבלאות ה-hash (מושמטות כשריקות; שתיהן או אף אחת). PatchPipelineCli מחשב את הדוחות ומעביר אותן, ושער האימות משווה גם לפי טבלה. Manifest.kt של delta-updater מקבל את השדות עם ברירת מחדל ריקה. הלקוח משתמש במפות כדי לאמת אחרי apply רק את הטבלאות שה-patch יכול היה לשנות, ולבדוק את השאר אחרי ה-commit בלי לחסום קריאה. בדיקות: LogicalHashContractTest חדש מול fixture משותף עם הלקוח (logical_hash_contract.json, מושווה ב-contract.yml ב-cmp), ReleaseManifestWriterTest מורחב. תיעוד ב-DELTA_UPDATE_WORKFLOW.md §1.4.
This was referenced Sep 7, 2026
palmoni5
pushed a commit
that referenced
this pull request
Sep 8, 2026
…ycle-33987355439 contract.yml was the only conflict: base added the logical-hash fixture pair (env, test -f, cmp, --tests '*LogicalHashContractTest*'); this branch added the otzaria push filter, the @v5 action majors and the JUnit-XML test count. Resolved by keeping both: 6 fixtures present, 3 byte-identical pairs, and the count line labels both test classes. test_ci_workflows.py pins those literals, so its three assertions move in lockstep (4→6, 2→3, combined label). PatchPipelineCli.kt auto-merged (non-overlapping hunks); no script in .github/ parses the delta manifest's new fromTableContentHashes/toTableContentHashes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
palmoni5
pushed a commit
that referenced
this pull request
Sep 9, 2026
…shes only, test counts in every gate, workflow contract tests incl. the run: length guard, __pycache__ untracked and ignored Squashed from the audit branch by file set (3 original commits contributed; their messages follow). --- 15ef349 ci: setup-gradle v5 with real cache reuse; contract.yml on otzaria pushes only; test counts in every gate; native-access flag ends 60 JVM warnings Audit of cycle 33987355439 (items S9 + S7, reports 12 and 07). S9 - ci.yml: gradle/gradle-build-action@v3 (deprecated; no inputs; the audited run executed 67/67 tasks after a ~400 s cache step) -> gradle/actions/setup-gradle@v5 with cache-read-only on non-otzaria refs (write on trunk, read on PRs). contract.yml's setup-gradle v4 -> v5 so the repo runs one generation (test pins them equal). v6 is not taken: it moves caching into a proprietary component under commercial terms and drops configuration-cache support, which this repo enables (operator decision). - contract.yml: `push: branches: [otzaria]` (+ unfiltered pull_request); the pipeline's own release/handoff refs no longer re-trigger it. No tag trigger existed, nothing in the four repos looks up a contract run, and the trunk has no branch protection, so no required check is lost. - Every Python-suite gate prints `tests: <suite> N passed` per suite and a total, and fails on a failing suite AND on a suite collecting zero tests; ci.yml and contract.yml count the JUnit XML (`tests: allTests …`) and fail if no XML exists. The reconcile job's loop now also runs test_ci_workflows.py, test_patch_anchor_schema.py and test_prefetch_patch_anchors.py (hermetic: stub gh, no network) — S8's prefetch work was gated by nothing before; a contract test asserts the set of test_*.py files on disk equals the wired set. - Node 20 deprecations: every ubuntu-latest job moves to the lowest node24 major (checkout v5, setup-java v5, upload-artifact v6, setup-gradle v5), each verified from the action's action.yml; self- hosted jobs untouched. The "Gradle-8" line is KGP's own inert wording (0 .java sources) and is explained in ci.yml rather than silenced. S7 - The 60 `WARNING: A restricted method in java.lang.System has been called … System::load` lines were 15 forked-JVM blocks from org.sqlite.SQLiteJDBCLoader (Generate x9, Dump lines, Apply LINKER, patch fan x4). `--enable-native-access=ALL-UNNAMED` is applied centrally via a CommandLineArgumentProvider on every JavaExec/Test task in the root build (not jvmArgs, which ~25 tasks assign and two lines of which the recovery overlay greps verbatim) and via patchPipelineJvmArgs for the fan's direct java. Proven: 4 warnings -> 0 on the same class; a real JavaExec reaching NativeDB -> 0; the launcher spec publishes the flag; configuration cache stored and reused. - `--warning-mode all` on ci.yml's single Gradle invocation: exactly two deprecations, both from the Android Gradle plugin (lint-gradle, aapt2), none in this repo's build scripts. - The remaining gh stderr leak (prefetch's unexpected seforim.db.zst failure) folds gh's reason into its named line. - Not done, by decision: moving the 45 `run:` bodies (2,403 lines, ~127 KB of echoed script per log) to files before Thursday. QA: action majors and `with:` keys verified per action.yml; loops executed (failing suite, zero-collection, empty XML dir); 137 pytest; 108 step bodies across 9 workflows bash -n; 0 CR in the diff. --- f0c95a1 chore: untrack .github/scripts/__pycache__ (added by mistake in 807b36b) and ignore it --- 3772bac Merge origin/otzaria (PR #27 per-table content hash) into fix/audit-cycle-33987355439 contract.yml was the only conflict: base added the logical-hash fixture pair (env, test -f, cmp, --tests '*LogicalHashContractTest*'); this branch added the otzaria push filter, the @v5 action majors and the JUnit-XML test count. Resolved by keeping both: 6 fixtures present, 3 byte-identical pairs, and the count line labels both test classes. test_ci_workflows.py pins those literals, so its three assertions move in lockstep (4→6, 2→3, combined label). PatchPipelineCli.kt auto-merged (non-overlapping hunks); no script in .github/ parses the delta manifest's new fromTableContentHashes/toTableContentHashes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
מה השינוי
מוסיף למניפסט של כל patch שתי מפות אופציונליות,
fromTableContentHashesו-toTableContentHashes: ה-hash הלוגי של כל טבלה בנפרד (sha256 של זרם הבתים של הטבלה בלבד, כולל הקידומתtable:<t>). ה-hash הכולל הקיים לא משתנה — הבתים שמוזנים לו זהים; המפות מחושבות באותו מעבר יחיד, בשני digests.LogicalContentHasher.computeReport— מעבר יחיד, hash כולל + hash לכל טבלה (LinkedHashMap בסדר טבלאות ה-hash, כולל טבלאות חסרות).compute()מאציל.ReleaseManifestWriter— פולט את המפות כאובייקטי JSON בסדר איטרציה; מושמטות כשריקות; שתיהן או אף אחת.fromContentHash/toContentHashנשארים חובה.PatchPipelineCli— מחשב את הדוחות (ללא מעברים נוספים על ה-DB), מעביר לכותב, ושער האימות משווה גם לפי טבלה (whole-hash נשמר).delta-updater/Manifest.kt— השדות עם ברירת מחדל ריקה; התנהגות ה-applier לא משתנה.DELTA_UPDATE_WORKFLOW.md§1.4 — תיעוד המנגנון וכלל הלקוח.למה
הלקוח מאמת היום את כל הטבלאות (כ-80% מקובץ של 6GB) בתוך ה-transaction, לכל patch בשרשרת. עם המפות הוא מאמת רק את הטבלאות שה-patch יכול היה לשנות, ובודק את השאר אחרי ה-commit בלי לחסום קריאה.
PRים מקושרים (מענפים נפרדים)
contract.ymlמשווה את ה-fixture המשותף מולו.סדר מיזוג: PR זה + ה-updater קודם, ואז אוצריא.
איך נבדק
LogicalHashContractTestחדש מול fixture משותף עם הלקוח (generator/common/src/jvmTest/resources/logical_hash_contract.json, זהה-בתים ל-test/logical_hash_contract.jsonב-updater;contract.ymlמוסיףcmp): hash כולל, hash לכל טבלה,compute()ללא שינוי, hash של טבלה בודדת שווה לרשומה בריצה המלאה.ReleaseManifestWriterTestמורחב (עם מפות / בלי / אחת בלבד → שגיאה).:generator-common:jvmTest --tests '...common.patch.*'— 57/57 ירוק (JDK 25).:delta-updater:jvmTest— כשל אחד קודם ולא קשור (DeltaUpdaterClientEndToEndTest"lucene failure after sqlite commit"), נכשל זהה על עץ נקי שלotzaria.