fix: keep Node API responsive during txhashset zip#3890
Conversation
Snapshot under write locks, then package the multi-GB archive under read locks so Node API and other readers stay responsive while a peer is served a txhashset zip. Compaction still waits for the read locks. Addresses mimblewimble#3550.
| let _header_pmmr = self.header_pmmr.read(); | ||
| let _txhashset = self.txhashset.read(); | ||
|
|
||
| let file = txhashset::zip_read(self.db_root.clone(), &header)?; |
There was a problem hiding this comment.
zip_read was previously serialized by the write locks. It can now run concurrently for the same temp directory and zip path, so requests may clobber each other or reuse a partial archive. Could we serialize archive creation separately or use unique temp files with an atomic rename?
| /// at the provided block hash. | ||
| /// | ||
| /// Write locks are only held while rewinding/snapshotting. Packaging the | ||
| /// multi-GB zip is done under *read* locks so the Node API and other |
There was a problem hiding this comment.
The height request mentioned in #3550 goes through Chain::head() and does not take either of these locks. Since the multi-GB copy, zip write, and sync_all() remain unchanged, could we reproduce the original timeout before claiming this fixes it?
| txhashset::zip_read(self.db_root.clone(), &header) | ||
| .map(|file| (header.output_mmr_size, header.kernel_mmr_size, file)) | ||
| }) | ||
| // Shared locks: allow concurrent chain reads (API, header queries) while |
There was a problem hiding this comment.
This only allows concurrent readers until a writer is waiting. With parking_lot::RwLock, later readers then queue behind that writer for the rest of the zip. Could we test the API with a writer waiting during packaging?
2 participants
Summary
Fixes #3550: Node API becomes unresponsive while packaging a
txhashsetzip for a syncing peer.Cause
Chain::txhashset_readheld write locks onheader_pmmrandtxhashsetfor the entire rewind + snapshot + multi-GB zip path. API handlers that need any chain access blocked for the whole packaging duration.Fix
zip_read(copy + package). Concurrent readers (Node API, headers, etc.) can proceed. Compaction/writers still wait until the zip finishes so backend files are not rewritten mid-package.Test plan
cargo test -p grin_chain --test test_txhashset -- --test-threads=1