Skip to content

Fix IDBMirrorVFS corrupting the database on rollback - #352

Open
lalexdotcom wants to merge 2 commits into
rhashimoto:masterfrom
lalexdotcom:fix/idb-mirror-proxy-write
Open

lalexdotcom wants to merge 2 commits into
rhashimoto:masterfrom
lalexdotcom:fix/idb-mirror-proxy-write

Conversation

@lalexdotcom

Copy link
Copy Markdown

What happens

jWrite stores files other than the main database in a single buffer, and copies into it with:

block.set(pData, iOffset);

pData is a Uint8ArrayProxy, not a Uint8Array. It carries the methods but has no indexed access, so TypedArray.prototype.set treats it as an array-like: it reads pData[0], pData[1], … which are all undefined, and stores zeroes. The class comment in FacadeVFS.js says exactly this — "it is not a real Uint8Array and passing it to functions that expect a Uint8Array may not work. Use subarray()" — and MemoryVFS already calls subarray() for this reason.

What it costs is the rollback journal. SQLite reserves its header, journals the pages it is about to write, and only then stamps the magic and the record count at offset 0. That stamp is a 12-byte write, and it lands as zeroes:

write  off=0 len=12  src=d9d505f920a163d700000002
stored off=0         = 00000000

On rollback SQLite reads the header back, finds a journal that looks empty, concludes there is nothing to undo, and deletes it. The pages already written to the database stay, and the database header no longer describes the file — one measurement had the header claiming 3 pages for a file of 2070, another 2694 pages for a file of 3. SQLite reports database disk image is malformed.

Larger writes are unaffected, which is why this went unseen. A write that does not fit the current buffer takes the growth path, where the copy is newBlock.set(block) between two real arrays; only the faulty block.set(pData, …) line writes the payload. A write that fits the existing buffer takes it directly — and the journal header, written after the journal has grown, is exactly that.

The change

block.set(pData.subarray(), iOffset);

subarray() returns a real Uint8Array over the same bytes. Nothing else moves.

Test

test/vfs_rollback.js, wired into test/IDBMirrorVFS.test.js: a transaction large enough that SQLite must write pages before the commit — which is what makes it journal them — then rolled back. It asserts through SQL alone, PRAGMA integrity_check and the table contents, so it does not depend on how the VFS stores anything.

It has to go through SQLite. A test calling jWrite directly receives a real Uint8Array from Comlink and cannot see the defect; I wrote that test first and it passed against the bug.

Against master it fails with database disk image is malformed. With the change, the file's tests pass and the full suite is 13 files, 0 failures.

Scope

The defect needs no interruption, no concurrency and no savepoint: an ordinary BEGIN / large INSERT / ROLLBACK reaches it. Any transaction big enough to spill the page cache and then rolled back corrupts the database.

Checklist

  • I grant to recipients of this Project distribution a perpetual,
    non-exclusive, royalty-free, irrevocable copyright license to reproduce, prepare
    derivative works of, publicly display, sublicense, and distribute this
    Contribution and such derivative works.
  • I certify that I am legally entitled to grant this license, and that this
    Contribution contains no content requiring a license from any third party.

lalexdotcom and others added 2 commits September 18, 2026 14:46
…ase.

jWrite stores files other than the main database in one buffer and
copies into it with block.set(pData, iOffset). pData is a
Uint8ArrayProxy, not a Uint8Array: it carries the methods but has no
indexed access, so TypedArray.set treats it as an array-like, reads
undefined at every index and stores zeroes. The comment on the class
says as much, and MemoryVFS already calls subarray() for this reason.

What it costs is the rollback journal. SQLite reserves its header,
journals the pages it is about to write, and only then stamps the magic
and the record count at offset 0 -- a 12-byte write, which lands as
zeroes. On rollback SQLite reads that header back, finds a journal that
looks empty, concludes there is nothing to undo, and deletes it. The
pages it had already written to the database stay, and the database
header no longer matches the file: "database disk image is malformed".

Larger writes are unaffected, which is why this went unseen: they extend
the buffer, and the copy inside the growth path uses newBlock.set(block)
between two real arrays. Only a write that fits the existing buffer
takes the faulty path, and the journal header is exactly that.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A transaction big enough that SQLite must write pages before the commit,
then rolled back. That is what makes SQLite journal the pages and stamp
the journal header, and the rollback is what reads the header back.

The test asserts through SQL alone -- PRAGMA integrity_check and the
table contents -- so it does not depend on how the VFS stores anything.
It has to go through SQLite: the defect is in how a Uint8ArrayProxy
reaches the VFS, and a test calling jWrite directly receives a real
Uint8Array from Comlink and cannot see it.

Against the previous behaviour it fails with "database disk image is
malformed".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant