[WIP] Initial work on pruning old Authority data

[eranrund]

Motivation

Related to #3230 — Enable pruning of historic state.

BlockStorage currently keeps authority-related data forever. As the chain grows, this causes the RocksDB database to grow indefinitely even though consensus only needs recent authority information.

In simple terms, a block contains two different kinds of information:

• Permanent information, such as transactions, headers, hashes, and state data.
• Authority information used by consensus, including quorum subdags, batch certificates and signatures.

This PR keeps the permanent information, but removes old authority information once it falls outside the recent retention window.

What this PR does

When the history feature is not enabled:

• Retain authority data for the latest BatchHeader::MAX_GC_ROUNDS blocks. This is currently 100 blocks.

• Never prune the genesis authority (this appears to be optional and is currently done as an abundance of caution, mimicking he special treatment the genesis block has in the Ledger code).

• When a new block is inserted, atomically remove:

  • The expired entry from AuthorityMap.
  • The corresponding certificate indexes from CertificateMap.

• When an existing database is opened, remove authority and certificate entries that are already outside the retention window.

• Perform startup cleanup in batches of at most 10,000 map deletions so opening an old production database does not create one enormous write batch.

• Restrict the startup random-block integrity check to blocks whose authorities are guaranteed to remain available.

When the history feature is enabled:

• Do not prune anything.
• Preserve the existing behavior where old blocks can be reconstructed through get_block.

The top-level history feature is propagated to snarkvm-ledger, and CI now tests snarkvm-ledger-store with both rocks and rocks,history.

What remains stored

This PR does not prune:

• Transactions or the transaction store.
• Block hashes or heights.
• Block headers.
• State roots or state paths.
• Ratifications.
• Solutions.
• Finalize data.

get_state_path_for_commitment previously reconstructed the complete block through get_block. It now reads only the block height, previous hash, header, and transactions that it actually needs. This keeps historical state paths working even after the block’s authority has been pruned.

Expected effect on get_block

get_block is intentionally unchanged: it still returns a complete Block, and a complete block requires its authority.

Consequently, in a non-history build, get_block will fail for blocks whose authority has been pruned. Returning a partially populated Block would change the meaning and invariants of that API.

Estimated production savings

A temporary one-off estimator was written to measure this using real mainnet data. The estimator was intentionally removed after collecting the results and is not part of this PR but I can reproduce it if needed.

The estimator used the following procedure:

1. Query the Provable Explorer for the latest numerically available mainnet height.
2. Select 10,000 heights spread evenly across the prunable portion of mainnet history.
3. Also include genesis and the complete retained tip window.
4. Download the full block and state root at every selected height from the Provable Explorer.
5. Insert all fetched full blocks into a RocksDB database using a history-enabled build so that nothing was pruned while constructing the sample.
6. Flush and fully compact RocksDB before measuring the baseline.
7. Reopen the same database without history, prune the expired authorities and certificate indexes, verify the retention boundary, and flush and fully compact the database again.
8. Flush and fully compact RocksDB.
9. Measure the on-disk size of the database directory and the time required to prune the sampled data.

The observed results were:

Measurement	Before pruning	After pruning	Difference
RocksDB directory	874,322,436 bytes (0.814 GiB)	102,936,461 bytes (0.096 GiB)	771,385,975 bytes (0.718 GiB, 88.23%)
Authority entries	10,102	102	10,000 removed
Certificate entries	454,385	5,096	449,289 removed

Removing the 10,000 sampled authorities and 449,289 certificate indexes took approximately 412 milliseconds. The forced full compaction used for the subsequent disk measurement took another 310 milliseconds.

For the production storage estimate, the observed on-disk savings were divided by the 10,000 sampled prunable blocks and multiplied by the 20,219,339 blocks that were prunable at the sampled mainnet height:

771,385,975 bytes / 10,000 sampled blocks
    × 20,219,339 production-prunable blocks
    = 1,559,691,452,837 bytes

This estimates approximately:

• 1,559,691,452,837 bytes
• 1,452.576 GiB
• Approximately 1.42 TiB

of reclaimable production storage.

Applying the same linear extrapolation to the measured pruning time gives:

0.412079458 seconds / 10,000 sampled blocks
    × 20,219,339 production-prunable blocks
    = approximately 833 seconds

That is approximately 13.9 minutes for the initial production pruning pass.

This timing estimate covers the deletion pass performed while opening the database. This PR does not force a full RocksDB compaction in production. Deletions create tombstones, and the physical disk space is reclaimed later through RocksDB’s normal compaction process.

The estimator forced a full compaction only so that its before-and-after disk measurements would be comparable. The sample compaction took approximately 310 milliseconds, but it would not be reliable to extrapolate that number directly to a full production database: compaction can touch data outside the pruned maps and depends heavily on the database layout, existing compaction state and storage hardware.

These storage and timing numbers are estimates, not exact promises. Authority sizes vary between blocks, and the sample contains 10,000 evenly distributed blocks rather than the entire chain. The production database may therefore prune faster or slower than the straight-line estimate.

The one-off estimator also retained one additional experimental safety block. This PR retains exactly MAX_GC_ROUNDS blocks, so the storage estimate is slightly conservative relative to the final implementation.

Important remaining work: P2P synchronization

Important

The largest remaining question is how non-history nodes should participate in P2P block synchronization after their old authorities have been pruned.

In snarkOS, a node handling a BlockRequest currently calls ledger.get_blocks. Ledger::get_blocks reconstructs every requested block through get_block.

After this PR:

1. A peer can request an old block range from a non-history node.
2. The node still has the transactions, headers, hashes, and state data.
3. However, it no longer has the old authorities.
4. It therefore cannot reconstruct the complete blocks.
5. ledger.get_blocks fails, and the node cannot return the requested BlockResponse.

This is especially important for a new node attempting to synchronize from old heights. This PR does not yet decide:

• Whether only history-enabled/archive nodes should serve old block ranges.
• How peers discover which nodes retain history.
• Whether block requests should be limited according to the serving peer’s retention window.
• Whether initial synchronization should use another source or mechanism.
• How a non-history node should explicitly communicate that a requested range has been pruned.
• Whether we want to support returning partial blocks (blocks without Authority data) for syncing peers who are looking for old/pruned blocks.

The storage pruning itself is implemented and tested here, but the P2P synchronization policy and corresponding snarkOS changes remain a significant TBD before relying on pruning across all production peers.

Test Plan

The new tests cover the following behavior:

Test area	What it proves
Non-history retention boundary	Height 1 remains available through height 100 and is pruned when height 101 is inserted; genesis and permanent block maps remain available.
History retention	Enabling history preserves old authorities and complete historical blocks.
Certificate pruning	Removing a quorum authority also removes all corresponding certificate indexes.
Atomic insertion failure	If block insertion fails, authority and certificate deletions are rolled back with the rest of the atomic write.
Invalid certificate indexes	Missing indexes are tolerated, while an index pointing to the wrong block height is rejected.
Multi-batch pruning	More than 10,000 entries are pruned correctly and repeated pruning is idempotent.
Storage backends	The same multi-batch assertions run against both BlockMemory and RocksDB.
Startup pruning	Opening an old non-history RocksDB removes stale entries and remains idempotent across repeated opens.
History startup	Opening the same shape of RocksDB with history preserves all authority information.
Historical state paths	A state path can still be produced after the corresponding authority has been removed.

Local validation performed:

cargo +nightly fmt --all -- --check

cargo check -p snarkvm-ledger-store -p snarkvm-ledger
cargo check -p snarkvm-ledger-store -p snarkvm-ledger --features history

cargo test -p snarkvm-ledger-store -p snarkvm-ledger
cargo test -p snarkvm-ledger-store -p snarkvm-ledger --features history

Noting that test_bond_and_unbond_validator failed but this appears unrelated as it also happens without the changes in this PR.

CircleCI continues to run snarkvm-ledger-store with --features=rocks and now has an additional --features=rocks,history job. Together, these configurations should include all new history and non-history tests.

Documentation

The new retention helper, pruning behavior, and tests have code-level documentation.

No external operator documentation is updated in this PR. Once the P2P synchronization policy is decided, operator-facing documentation should explain:

• The difference between history and non-history nodes.
• Which historical queries are unavailable after pruning.
• Whether history-enabled nodes are required to support network synchronization.
• That enabling history after a database has already been pruned cannot recreate the deleted authority data; the database must be restored or rebuilt.

Backwards compatibility

This is an operationally breaking change in two important ways.

P2P block synchronization

A node running this version without the history feature can no longer reconstruct complete blocks outside the authority-retention window. get_block requires the block’s authority, and that authority will have been pruned.

An unupdated peer may request an old block range from a pruning node without knowing that the node no longer retains complete historical blocks. The pruning node will fail while building the BlockResponse, and the requesting peer will not receive the blocks it expected.

This PR does not yet introduce a way for peers to advertise their history-retention capability or avoid requesting historical blocks from pruning nodes. The P2P synchronization behavior must be addressed before pruning can be safely enabled across nodes expected to serve historical sync requests.

Irreversible database pruning

For a build without the history feature, opening an existing database automatically removes expired entries from AuthorityMap and CertificateMap. Subsequent block insertions continue pruning entries as they leave the retention window.

This deletion is irreversible. Enabling history later does not recreate the deleted authority data. An operator who needs the complete historical database would have to rebuild the database from a source that still retains full history.

This does not change consensus rules, block serialization, or network message formats, so I believe it should not require a ConsensusVersion guard. However, the P2P behavior and automatic irreversible database deletion will make the release operationally backwards-incompatible and require explicit upgrade coordination.