Skip to content

v0.2.0: an unfinished rebuild cannot be taken back by a stale watch, purge leaves the WAL empty - #11

Closed
puretensor wants to merge 2 commits into
mainfrom
claude/gifted-knuth-jytfax
Closed

puretensor wants to merge 2 commits into
mainfrom
claude/gifted-knuth-jytfax

Conversation

@puretensor

@puretensor puretensor commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Fixes from a max-effort review of #10 (index lock, fail-closed vault switch, purge debt).

Fixes

  • A stale watch could revert a failed or interrupted --rebuild. A model or vault mismatch was allowed whenever the index held 0 files. A --rebuild that emptied the index and then failed or was cut short before its first write left it at 0 files. A puremind watch waiting on the lock with the old model or vault could then claim the index, clear it and index it again with the old model. That contradicts the CHANGELOG. Also, clear() and the set_meta("embedder", …) calls ran in separate transactions, so a crash between them left the old identity on an emptied index.
    • Store.clear(meta) now records the new model, root and a rebuild_pending marker in the transaction that empties the index. Only a run that visits every note removes the marker.
    • While the marker is set, a mismatched run is refused even on an empty index. An index that is empty for any other reason still takes a new model without --rebuild, as in v0.1.4.
  • A successful Store.purge() left the -wal non-empty (8 KB), because the debt DELETE was committed after the checkpoint. That defeats _open_read_only's immutable fallback. purge() now checkpoints again after clearing the debt.
  • Runs with purge=False that inherited debt used to checkpoint on every run. They also logged "text deleted or replaced in this run stays in -wal" even when nothing changed. Such a run now checkpoints only if it removed or replaced text itself, and purge_deferred still reports the debt.
  • compact() in cmd_watch reset owed_since even when a reader kept the purge from finishing. It now keeps the timer running while the debt remains.
  • test_a_failed_vault_switch_leaves_no_old_content could not fail. Its asserts inside the embed hook were wrapped into the EmbedderError the test expects. The hook now records what it sees, and the test asserts on that after the raise. A mutation that moves the clear after embedding now fails the test.
  • Cleanup: one _put_meta helper shared by set_meta and clear.

Tests

  • New regression tests:
    • a stale run cannot take back an unfinished rebuild (model and vault cases);
    • an index emptied by a finished rebuild still takes a new model;
    • a deferring run that changed nothing does not checkpoint or warn;
    • the -wal is 0 bytes after a purging run.
  • Each new test fails on main and passes here.
  • Full suite: 720 passed, 19 skipped. The permission tests skip when run as root.
  • ruff check is clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KNcv9ZmzjUZyZT2Xk6i6mj

…purge leaves the WAL empty

- Store.clear() records the new model, root and a rebuild_pending marker in the
  transaction that empties the index; only a run that visits every note removes it.
  While it is set, a run on another model or vault is refused even on an empty index,
  so a `puremind watch` on the old config can no longer revert a failed or interrupted
  --rebuild. A genuinely empty index still takes a new model without --rebuild.
- Store.purge() checkpoints again after clearing the debt, so the -wal ends empty
  (an immutable read-only open needs that).
- A purge=False run checkpoints only when it removed or replaced text itself; inherited
  debt is reported via purge_deferred without a checkpoint or a misleading warning.
- cmd_watch compact() keeps owed_since while a purge left the debt in place.
- One meta upsert helper (_put_meta) shared by set_meta and clear.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KNcv9ZmzjUZyZT2Xk6i6mj
…gain

An assert in the hook was wrapped into the EmbedderError the test expects, so
the 'emptied before any embedding' check could never fail. The hook now records
what it sees and the test asserts after the raise; the raises match the hook's
error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KNcv9ZmzjUZyZT2Xk6i6mj
@puretensor

Copy link
Copy Markdown
Owner Author

Closing as superseded by #12, which merged to main as squash commit 36f04a7. #12 makes every change this PR makes: the REBUILD_PENDING marker written atomically through clear(meta=...), _put_meta/delete_meta, the second checkpoint in purge(), deferring runs that checkpoint only when they removed or updated text, compact() keeping owed_since, and the vault-switch test fix. The HAL PR sweep (2026-11-04) copied this PR's new tests unchanged into #12's tree, and all of them pass there. #12 also fixes this PR's Important finding F1: the rebuild marker is now cleared only after a clean scan (if not scan.errors), so a stale watch can no longer take the index back after a rebuild whose directories could not be listed. It also covers F2 (_one_line(str(exc)) in cmd_watch), the <db>.lock dot-file name collision, compaction outside the run lock, the root-owned lock file, and LOCK_UN at fork. After #12 this PR conflicts in 6 files, and both claim v0.2.0. The branch is kept.

@puretensor puretensor closed this Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants