diff --git a/doc/assumeutxo.md b/doc/assumeutxo.md index d5da07d6c39b..e5d57c999721 100644 --- a/doc/assumeutxo.md +++ b/doc/assumeutxo.md @@ -1,85 +1,107 @@ -# Assumeutxo Usage +# Using Assumeutxo -Assumeutxo is a feature that allows fast bootstrapping of a validating dashd -instance. +Assumeutxo can make a new Dash Core node usable quickly by loading a recent +chainstate snapshot. The node syncs forward from the snapshot immediately and, +at the same time, validates every block from genesis in the background. For +implementation details, see the [design document](/doc/design/assumeutxo.md). -For notes on the design of Assumeutxo, please refer to [the design doc](/doc/design/assumeutxo.md). +## Availability and trust -## Loading a snapshot +There is currently no canonical snapshot distribution service and this tree +has no mainnet or testnet snapshot parameters. Built-in snapshots and the +`-assumeutxodata` option are for regtest only until release builders generate +and review production parameters. -There is currently no canonical source for snapshots, but any downloaded snapshot -will be checked against a hash that's been hardcoded in source code. If there is -no source for the snapshot you need, you can generate it yourself using -`dumptxoutset` on another node that is already synced (see -[Generating a snapshot](#generating-a-snapshot)). +A snapshot may be transported by an untrusted third party. For production +parameters, Dash Core accepts it only when its network and base block match an +authorized entry and both its UTXO hash and Dash evo-state hash match the +hardcoded values. (Two built-in legacy regtest fixtures retain a test-only null +evo-hash wildcard.) Dash Core also +cross-checks the deterministic masternode list, quorum commitments, and credit +pool against commitments in the base block's CbTx. Background validation then +re-derives the UTXO set and canonical masternode state from genesis before the +snapshot is considered fully validated. -Once you've obtained the snapshot, you can use the RPC command `loadtxoutset` to -load it. +## Obtain or create a snapshot -``` -$ dash-cli loadtxoutset /path/to/input -``` - -After the snapshot has loaded, the syncing process of both the snapshot chain -and the background IBD chain can be monitored with the `getchainstates` RPC. - -### Pruning - -A pruned node can load a snapshot. To save space, it's possible to delete the -snapshot file as soon as `loadtxoutset` finishes. +Obtain a snapshot for an authorized base height from a source you choose, or +create one on a normally synced node with `dumptxoutset`. A current-tip dump is: -The minimum `-prune` setting is 550 MiB, but this functionality ignores that -minimum and uses at least 1100 MiB. +```text +dash-cli -rpcclienttimeout=0 dumptxoutset /path/to/utxo.dat latest +``` -As the background sync continues there will be temporarily two chainstate -directories, each multiple gigabytes in size (likely growing larger than the -downloaded snapshot). +To dump a particular recent height (or block hash), temporarily roll the node +back while writing the snapshot: -### Indexes +```text +dash-cli -rpcclienttimeout=0 -named dumptxoutset /path/to/utxo.dat rollback=HEIGHT +``` -Indexes work but don't take advantage of this feature. They always start building -from the genesis block and can only apply blocks in order. Once the background -validation reaches the snapshot block, indexes will continue to build all the -way to the tip. +The positional form `dumptxoutset /path/to/utxo.dat rollback` selects the +latest base already authorized in chain parameters. A rollback dump requires +all intervening block and undo data, so it can fail on a pruned node. Network +activity is suspended and peers are disconnected during rollback; avoid other +block-storage RPCs until the command finishes. The original tip is reconsidered +after the dump. Use an unlimited or long RPC timeout because hashing and evo +snapshot construction can take several minutes. +The result includes `base_hash`, `base_height`, `txoutset_hash`, `evo_hash`, +`nchaintx`, and the written path. Operators can reproduce an authorized dump at +the same base and compare both hashes. Creating a dump at an arbitrary height +does not authorize it for loading; authorization is supplied by chain +parameters (or, on regtest, an exact `-assumeutxodata` entry). -For indexes that support pruning, note that these indexes only allow blocks that -were already indexed to be pruned. Blocks that are not indexed yet will also -not be pruned. +## Load and monitor a snapshot -This means that, if the snapshot is old, then a lot of blocks after the snapshot -block will need to be downloaded, and these blocks can't be pruned until they -are indexed, so they could consume a lot of disk space until indexing catches up -to the snapshot block. +Wait for the destination node to learn the base block header, ensure its +mempool is empty, and load the file: -## Generating a snapshot +```text +dash-cli loadtxoutset /path/to/utxo.dat +``` -The RPC command `dumptxoutset` can be used to generate a snapshot for the current -tip (using type "latest") or a recent height (using type "rollback"). A generated -snapshot from one node can then be loaded -on any other node. However, keep in mind that the snapshot hash needs to be -listed in the chainparams to make it usable. If there is no snapshot hash for -the height you have chosen already, you will need to change the code there and -re-compile. +Relative paths are resolved under the network data directory. On success the +result reports `coins_loaded`, `tip_hash`, `base_height`, and `path`. The input +file can then be removed; Dash Core has copied its state into +`chainstate_snapshot` and EvoDB. -Using the type parameter "rollback", `dumptxoutset` can also be used to verify the -hardcoded snapshot hash in the source code by regenerating the snapshot and -comparing the hash. +`loadtxoutset` is unavailable when Dash Core is running in masternode mode. +Stop using `-masternodeblsprivkey` and restart as a regular node before loading +a snapshot. Quorum signing is also refused whenever an unvalidated snapshot is +active. -Example usage: +Monitor progress with: -``` -$ dash-cli -rpcclienttimeout=0 dumptxoutset /path/to/output rollback +```text +dash-cli getchainstates ``` -For most of the duration of `dumptxoutset` running the node is in a temporary -state that does not actually reflect reality, i.e. blocks are marked invalid -although we know they are not invalid. Because of this it is discouraged to -interact with the node in any other way during this time to avoid inconsistent -results and race conditions, particularly RPCs that interact with blockstorage. -This inconsistent state is also why network activity is temporarily disabled, -causing us to disconnect from all peers. - -`dumptxoutset` takes some time to complete, independent of hardware and -what parameter is chosen. Because of that it is recommended to increase the RPC -client timeout value (use `-rpcclienttimeout=0` for no timeout). +During background validation, `chainstates` normally contains two entries. The +active snapshot entry has `snapshot_blockhash` and `validated: false`; the +historical entry advances from genesis. The snapshot chain syncs to the network +tip first, so normal wallet and mempool use can begin before background +validation finishes. + +When the historical entry reaches the snapshot base, Dash Core recomputes the +UTXO hash, compares the canonical masternode list and required reconstruction +history, and completes any CbTx/evo checks deferred until the base block was +available. The active entry then reports `validated: true`. Cleanup and +promotion are crash-safe and may finish on the next restart, after which only +one normal chainstate remains. + +If any required hash or derived-state comparison fails, Dash Core invalidates +the snapshot and shuts down rather than continuing on it. Follow the reported +recovery instruction; a reindex may be required for an EvoDB inconsistency. + +## Disk use, pruning, and indexes + +A pruned node can use Assumeutxo, but dual-chainstate operation can temporarily +exceed the configured prune budget. The minimum block-file allowance is needed +for each chainstate, the snapshot base is retained until deferred checks +complete, and index builders retain blocks they have not processed. + +Indexes still build sequentially from genesis; they do not begin at the +snapshot base. Expect extra download time and disk use until background +validation and indexes catch up. Two chainstate databases also remain on disk +until validation succeeds and cleanup completes. diff --git a/doc/design/assumeutxo.md b/doc/design/assumeutxo.md index ba6f1a4926e8..c4eb43ff2485 100644 --- a/doc/design/assumeutxo.md +++ b/doc/design/assumeutxo.md @@ -1,128 +1,203 @@ -# Assumeutxo Design - -For notes on the usage of Assumeutxo, please refer to [the usage doc](/doc/assumeutxo.md). - -## General background - -- [assumeutxo proposal](https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal) -- [Github issue](https://github.com/bitcoin/bitcoin/issues/15605) (Bitcoin) -- [draft PR](https://github.com/bitcoin/bitcoin/pull/15606) (Bitcoin) - -## Design notes - -- The concept of UTXO snapshots is treated as an implementation detail that lives - behind the ChainstateManager interface. The external presentation of the changes - required to facilitate the use of UTXO snapshots is the understanding that there are - now certain regions of the chain that can be temporarily assumed to be valid. - In certain cases, e.g. wallet rescanning, this is very similar to dealing with - a pruned chain. - - Logic outside ChainstateManager should try not to know about snapshots, instead - preferring to work in terms of more general states like assumed-valid. - - -## Chainstate phases - -Chainstate within the system goes through a number of phases when UTXO snapshots are -used, as managed by `ChainstateManager`. At various points there can be multiple -`Chainstate` objects in existence to facilitate both maintaining the network tip and -performing historical validation of the assumed-valid chain. - -It is worth noting that though there are multiple separate chainstates, those -chainstates share use of a common block index (i.e. they hold the same `BlockManager` -reference). - -The subheadings below outline the phases and the corresponding changes to chainstate -data. - -### "Normal" operation via initial block download - -`ChainstateManager` manages a single Chainstate object, for which -`m_from_snapshot_blockhash` is `std::nullopt`. This chainstate is (maybe obviously) -considered active. This is the "traditional" mode of operation for dashd. - -| | | -| ---------- | ----------- | -| number of chainstates | 1 | -| active chainstate | ibd | - -### User loads a UTXO snapshot via `loadtxoutset` RPC - -`ChainstateManager` initializes a new chainstate (see `ActivateSnapshot()`) to load the -snapshot contents into. During snapshot load and validation (see -`PopulateAndValidateSnapshot()`), the new chainstate is not considered active and the -original chainstate remains in use as active. - -| | | -| ---------- | ----------- | -| number of chainstates | 2 | -| active chainstate | ibd | - -Once the snapshot chainstate is loaded and validated, it is promoted to active -chainstate and a sync to tip begins. A new chainstate directory is created in the -datadir for the snapshot chainstate called `chainstate_snapshot`. - -When this directory is present in the datadir, the snapshot chainstate will be detected -and loaded as active on node startup (via `DetectSnapshotChainstate()`). - -A special file is created within that directory, `base_blockhash`, which contains the -serialized `uint256` of the base block of the snapshot. This is used to reinitialize -the snapshot chainstate on subsequent inits. Otherwise, the directory is a normal -leveldb database. - -| | | -| ---------- | ----------- | -| number of chainstates | 2 | -| active chainstate | snapshot | - -The snapshot begins to sync to tip from its base block, technically in parallel with -the original chainstate, but it is given priority during block download and is -allocated most of the cache (see `MaybeRebalanceCaches()` and usages) as our chief -goal is getting to network tip. - -**Failure consideration:** if shutdown happens at any point during this phase, both -chainstates will be detected during the next init and the process will resume. - -### Snapshot chainstate hits network tip - -Once the snapshot chainstate leaves IBD, caches are rebalanced -(via `MaybeRebalanceCaches()` in `ActivateBestChain()`) and more cache is given -to the background chainstate, which is responsible for doing full validation of the -assumed-valid parts of the chain. - -**Note:** at this point, ValidationInterface callbacks will be coming in from both -chainstates. Considerations here must be made for indexing, which may no longer be happening -sequentially. - -### Background chainstate hits snapshot base block - -Once the tip of the background chainstate hits the base block of the snapshot -chainstate, we stop use of the background chainstate by setting `m_disabled`, in -`CompleteSnapshotValidation()`, which is checked in `ActivateBestChain()`). We hash the -background chainstate's UTXO set contents and ensure it matches the compiled value in -`CMainParams::m_assumeutxo_data`. - -| | | -| ---------- | ----------- | -| number of chainstates | 2 (ibd has `m_disabled=true`) | -| active chainstate | snapshot | - -The background chainstate data lingers on disk until the program is restarted. - -### Dashd restarts sometime after snapshot validation has completed - -After a shutdown and subsequent restart, `LoadChainstate()` cleans up the background -chainstate with `ValidatedSnapshotCleanup()`, which renames the `chainstate_snapshot` -datadir as `chainstate` and removes the now unnecessary background chainstate data. - -| | | -| ---------- | ----------- | -| number of chainstates | 1 | -| active chainstate | ibd (was snapshot, but is now fully validated) | - -What began as the snapshot chainstate is now indistinguishable from a chainstate that -has been built from the traditional IBD process, and will be initialized as such. - -A file will be left in `chainstate/base_blockhash`, which indicates that the -chainstate, even though now fully validated, was originally started from a snapshot -with the corresponding base blockhash. +# Assumeutxo design + +For operator instructions, see [Using Assumeutxo](/doc/assumeutxo.md). + +Assumeutxo lets a node become useful from a recent, precomputed chainstate +while it validates the history from genesis in the background. The snapshot is +not a consensus shortcut: its base block and expected state hashes must be +authorized by chain parameters, and the node does not mark it fully validated +until background validation independently reaches the base. + +Dash snapshots contain both the UTXO set and the block-derived state needed to +continue Dash validation above the base. This document describes the format, +trust model, and lifecycle implemented in this tree. + +## File format and authorization + +A snapshot starts with the version-2 `SnapshotMetadata` container defined in +`src/node/utxo_snapshot.h`: + +1. the five bytes `utxo\xff`; +2. metadata version 2; +3. the network message-start bytes; +4. the snapshot base block hash; and +5. the number of serialized coins. + +The body is the declared sequence of `(COutPoint, Coin)` records followed by +the `DASHEVO\0` marker and a `CEvoSnapshot`. The evo payload has its own format +version, currently version 3. Keeping its version separate permits the Dash +section to evolve without changing the outer, upstream-compatible metadata +container. + +`AssumeutxoData` authorizes a base height and block hash together with +`nChainTx`, the serialized UTXO hash (`hash_serialized`), and the canonical evo +hash (`evo_hash`). The hashes commit to the decoded UTXO and evo bodies, not to +the metadata header or the whole file. Network magic prevents loading a file +for another network. Mainnet and testnet currently have no authorized entries; +regtest has built-in test entries and accepts additional exact entries through +the regtest-only debug option +`-assumeutxodata=::::`. +Two legacy regtest fixtures use a null `evo_hash` as an intentional test-only +wildcard; null evo hashes are rejected on every other network. + +Release builders generate and independently reproduce production parameter +entries using the procedure in [release-process.md](/doc/release-process.md). + +## Dash evo snapshot version 3 + +The evo section carries the minimum block-derived state required to validate +and operate above a snapshot base without possessing every earlier block: + +- the canonical deterministic masternode list at the base, including internal + IDs and the total registration counter; +- a diff-encoded, apply-ordered reconstruction history for the older MN lists + needed by quorum member selection; +- active and safety mined quorum commitments for every enabled LLMQ type, + quorum score modifiers, and the four rotation-cycle snapshots needed to + reconstruct the current and retained rotated quorum sets; +- credit-pool state, including the balance/limits and spent asset-unlock index + ranges; and +- MNHF/EHF deployment signals. + +Collections and nested allocations are bounded during decoding. Canonical +ordering covers MNs, historical diffs, quorum types and commitments, score +modifiers, credit-pool ranges, and MNHF signals. Historical MN lists are +reconstructed by applying each diff, rebuilding their indexes, and checking +their canonical hashes. See `src/evo/snapshot.h` and `src/evo/snapshot.cpp` for +the exact codec and bounds. + +Internal MN IDs are part of the commitment deliberately. A node synced from +genesis assigns them in deterministic on-chain registration order and advances +the registration counter identically, so a dump can be reproduced from +genesis. Sorting by full `proTxHash` avoids dependence on in-memory container +iteration order. + +Except for that regtest-only wildcard, the compiled `evo_hash` is SHA256 of the +canonical disk serialization. It has +the same developer-reviewed, hardcoded trust model as `hash_serialized` and is +reproducible by independently synced-from-genesis nodes. In addition, the evo +state is checked against commitments in the base block's coinbase special +transaction (CbTx): the deterministic-MN-list merkle root, quorum merkle root, +and, where present, credit-pool balance. These header-chain-anchored checks +reduce the state that rests only on the hardcoded hashes. + +## Dual-chainstate lifecycle + +Without a snapshot, `ChainstateManager` owns one normal, fully validated +chainstate. Snapshot use adds a second chainstate; both share the block index, +block files, Dash managers, and one `CEvoDB`, but have separate coins databases +and separate EvoDB identities. + +### Activation + +`loadtxoutset` first requires an empty mempool and an authorized base whose +header is known. It parses the metadata and coins, checks the coin count and +`hash_serialized`, parses and validates the evo v3 section, checks `evo_hash`, +and performs every CbTx check possible with the locally available base block. +It then seeds the base MN list, reconstruction history, score modifiers, +quorum commitments and rotation snapshots, credit pool, and MNHF signals. + +Only after these checks and durable writes does the new +`chainstate_snapshot` become active. The former normal chainstate becomes the +historical/background chainstate and continues from genesis. The snapshot +chainstate is explicitly `UNVALIDATED` even though it can sync from its base to +the network tip. `getchainstates` exposes both chainstates and their +`validated` fields. + +### Background validation and completion + +Both chainstates execute normal block and Dash special-transaction validation. +The active snapshot chainstate is prioritized until it reaches the network tip; +cache is then rebalanced toward background validation. When the background +chainstate reaches the snapshot base, completion: + +1. flushes both coins/EvoDB identities; +2. recomputes its serialized UTXO hash and compares it with + `hash_serialized`; +3. reconstructs the retained historical MN lists and compares them with the + independently captured background lists; +4. performs deferred CbTx checks using the now-available base block; +5. compares the snapshot's canonical base MN-list hash with the list derived + by background validation; and +6. verifies both EvoDB best-block markers. + +Any mismatch invalidates the snapshot and shuts the node down with recovery +information. Success changes the snapshot chainstate to `VALIDATED`, records +the computed UTXO hash on the completed background chainstate, and releases +the snapshot-base pruning lock. + +### Crash-safe promotion and cleanup + +Shutdown can occur during loading, background validation, completion, or +cleanup. Persistent base-block and EvoDB lifecycle markers let startup resume +the two-chainstate run. After successful completion, startup promotes the +snapshot coins directory to the normal `chainstate` directory and retires the +background directory through durable rename steps (`chainstate_todelete` is +used as the deletion staging name). EvoDB snapshot markers are promoted only +after both directory renames are durable. + +Load-time recovery recognizes interrupted promotion states and either finishes +cleanup or reports an inconsistency requiring reindex; it does not silently +select one side. After cleanup there is one normal, fully validated chainstate. +The retained `base_blockhash` records its snapshot origin but does not make it +assumed-valid again. Reindex paths remove stale snapshot/invalid/to-delete +directories and rebuild EvoDB. + +## Shared EvoDB and union convergence + +Dash uses one `CEvoDB` rather than a second `evodb_snapshot` directory. Most +derived data is keyed by block hash: snapshot-chainstate writes above the base +and background-chainstate writes through the base are naturally disjoint. Their +union converges to the database produced by validation from genesis, so no +lossy merge is needed when the background coins database is discarded. + +The `NORMAL` and `SNAPSHOT` identities have independent best-block markers and +transaction contexts. Flushes are serialized and committed per identity. +Writes of already seeded block-hash data are verify-or-skip: disagreement +between seeded and independently derived state is validation failure, not an +overwrite. A dual-chainstate marker makes incomplete shared-DB work detectable +at startup. This is the D2 union-convergence design implemented in +`src/evo/evodb.*`, `src/evo/snapshot_load.cpp`, and `src/validation.cpp`. + +## Subsystem behavior before validation + +Validation callbacks identify the chainstate role so background connections do +not drive active-tip side effects. Wallets, indexes, P2P processing, and Dash +subsystems continue to follow the active snapshot tip where appropriate while +the background chainstate performs consensus validation. + +Safety boundaries are stricter for quorum signing: + +- signature-share creation and quorum-signing RPCs refuse to sign while a + snapshot is unvalidated; +- `masternode status` reports quorum participation as disabled in that state; + and +- runtime `loadtxoutset` is refused in masternode mode because the already + constructed active signing contexts cannot be rebound safely. + +Verification and non-signing use of the seeded quorum state remain available. +When requested historical block or quorum data is unavailable because it lies +below an unvalidated snapshot base, serving code treats that as local data +unavailability and does not punish the requesting or serving peer for the +snapshot limitation. The node also advertises limited network service until +background validation restores full historical service. + +## Pruning, indexes, and the mempool + +Pruned nodes may load snapshots. During dual-chainstate operation the prune +budget is split between chainstates, with the per-chainstate minimum enforced; +the effective requirement can therefore exceed the configured budget. A prune +lock retains the base block until deferred CbTx/evo completion checks finish, +and blocks needed by indexes cannot be pruned before those indexes process +them. + +Indexes do not skip history: they build in order from genesis using background +validation and continue through the snapshot base to the active tip. This may +temporarily retain substantial block data. The active snapshot can otherwise +serve normal wallet and mempool operation, but `loadtxoutset` requires the +mempool to be empty so transactions validated against the old active coins view +cannot cross the activation boundary. + +The input snapshot file is no longer needed after `loadtxoutset` returns +successfully. The two coins databases and retained block/index data, however, +remain until validation and cleanup complete. diff --git a/doc/release-notes-assumeutxo-m7-params-polish.md b/doc/release-notes-assumeutxo-m7-params-polish.md new file mode 100644 index 000000000000..cafa8d1c0ab8 --- /dev/null +++ b/doc/release-notes-assumeutxo-m7-params-polish.md @@ -0,0 +1,20 @@ +Assumeutxo +---------- + +Dash Core can now bootstrap from an authorized UTXO and Dash evo-state +snapshot while validating the chain from genesis in the background. The new +`loadtxoutset` RPC loads snapshots created by `dumptxoutset`, and the new +`getchainstates` RPC reports active and background validation progress. +`dumptxoutset` now includes the canonical deterministic-masternode, quorum, +credit-pool, and MNHF state needed to continue Dash validation, and supports +creating a snapshot at a specified rollback height or block hash. + +Production snapshot contents are checked against hardcoded UTXO and evo hashes +and against the base block's CbTx commitments. Masternode-mode nodes cannot call +`loadtxoutset`, and quorum signing remains disabled while a loaded snapshot is +not yet validated. + +This release does not yet include mainnet or testnet snapshot parameters. +Assumeutxo loading is available for built-in regtest entries and exact regtest +entries supplied with the debug-only `-assumeutxodata` option until production +parameters are generated through the release process. diff --git a/doc/release-process.md b/doc/release-process.md index 5752dd7c6129..68c771ecd64d 100644 --- a/doc/release-process.md +++ b/doc/release-process.md @@ -20,6 +20,17 @@ Before every minor and major release: - Testnet should be set some tens of thousands back from the tip due to reorgs there. - This update should be reviewed with a `reindex-chainstate` with `assumevalid=0` to catch any defect that causes rejection of blocks in the past history. +* [ ] Append a new entry to `m_assumeutxo_data` using the values returned by + `dash-cli -rpcclienttimeout=0 -named dumptxoutset utxo.dat rollback=`. + Use the same height-safety considerations as `defaultAssumeValid`, and copy + `base_height`, `base_hash`, `txoutset_hash`, `evo_hash`, and `nchaintx` into + the corresponding `AssumeutxoData` fields. + - Generate the release value on a node synced normally from genesis, without + loading an assumeutxo snapshot. This is required to reproduce `evo_hash`: + deterministic-masternode registration counters and the derived historical + evo state must have been built from genesis. + - Reproduce both `txoutset_hash` and `evo_hash` on a second independently + synced-from-genesis node at the same base block before committing the entry. * [ ] Ensure all TODOs are evaluated and resolved if needed * [ ] Verify Insight works * [ ] Verify p2pool works (unmaintained; no responsible party) diff --git a/src/Makefile.am b/src/Makefile.am index 1637e93d2646..87cdf7b2a121 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -289,6 +289,7 @@ BITCOIN_CORE_H = \ kernel/mempool_limits.h \ kernel/mempool_options.h \ kernel/mempool_persist.h \ + kernel/types.h \ key.h \ key_io.h \ limitedmap.h \ diff --git a/src/bitcoin-chainstate.cpp b/src/bitcoin-chainstate.cpp index 24eb5965c26d..7e2b86f29c65 100644 --- a/src/bitcoin-chainstate.cpp +++ b/src/bitcoin-chainstate.cpp @@ -142,7 +142,7 @@ int main(int argc, char* argv[]) LOCK(chainman.GetMutex()); std::cout << "\t" << "Reindexing: " << std::boolalpha << node::fReindex.load() << std::noboolalpha << std::endl - << "\t" << "Snapshot Active: " << std::boolalpha << chainman.IsSnapshotActive() << std::noboolalpha << std::endl + << "\t" << "Snapshot Active: " << std::boolalpha << bool{chainman.CurrentChainstate().m_from_snapshot_blockhash} << std::noboolalpha << std::endl << "\t" << "Active Height: " << chainman.ActiveHeight() << std::endl << "\t" << "Active IBD: " << std::boolalpha << chainman.ActiveChainstate().IsInitialBlockDownload() << std::noboolalpha << std::endl; CBlockIndex* tip = chainman.ActiveTip(); diff --git a/src/chainlock/handler.cpp b/src/chainlock/handler.cpp index 8d5f88ceda4e..13729d50c5d4 100644 --- a/src/chainlock/handler.cpp +++ b/src/chainlock/handler.cpp @@ -216,9 +216,9 @@ void ChainlockHandler::AcceptedBlockHeader(const CBlockIndex* pindexNew) m_chainlocks.AcceptedBlockHeader(pindexNew); } -void ChainlockHandler::BlockConnected(ChainstateRole role, const std::shared_ptr& pblock, const CBlockIndex* pindex) +void ChainlockHandler::BlockConnected(const kernel::ChainstateRole& role, const std::shared_ptr& pblock, const CBlockIndex* pindex) { - if (role == ChainstateRole::BACKGROUND) return; + if (role.historical) return; if (!m_mn_sync.IsBlockchainSynced()) { return; } diff --git a/src/chainlock/handler.h b/src/chainlock/handler.h index 4a849ee5bf95..af9a4adff712 100644 --- a/src/chainlock/handler.h +++ b/src/chainlock/handler.h @@ -112,7 +112,7 @@ class ChainlockHandler final : public CValidationInterface void TransactionAddedToMempool(const CTransactionRef& tx, int64_t nAcceptTime, uint64_t mempool_sequence) override EXCLUSIVE_LOCKS_REQUIRED(!cs); - void BlockConnected(ChainstateRole role, const std::shared_ptr& pblock, const CBlockIndex* pindex) override + void BlockConnected(const kernel::ChainstateRole& role, const std::shared_ptr& pblock, const CBlockIndex* pindex) override EXCLUSIVE_LOCKS_REQUIRED(!cs); private: diff --git a/src/chainlock/signing.cpp b/src/chainlock/signing.cpp index e89cf2453a05..af52a229deea 100644 --- a/src/chainlock/signing.cpp +++ b/src/chainlock/signing.cpp @@ -185,9 +185,9 @@ void ChainLockSigner::BlockDisconnected(const std::shared_ptr& blo } -void ChainLockSigner::BlockConnected(ChainstateRole role, const std::shared_ptr& block, const CBlockIndex* pindex) +void ChainLockSigner::BlockConnected(const kernel::ChainstateRole& role, const std::shared_ptr& block, const CBlockIndex* pindex) { - if (role == ChainstateRole::BACKGROUND) return; + if (role.historical) return; if (!m_mn_sync.IsBlockchainSynced()) { return; } diff --git a/src/chainlock/signing.h b/src/chainlock/signing.h index 50c498ee68c3..20e9e7bff935 100644 --- a/src/chainlock/signing.h +++ b/src/chainlock/signing.h @@ -72,7 +72,7 @@ class ChainLockSigner final : public llmq::CRecoveredSigsListener, public CValid void UnregisterRecoveryInterface(); // implements validation interface: - void BlockConnected(ChainstateRole role, const std::shared_ptr& block, const CBlockIndex* pindex) override + void BlockConnected(const kernel::ChainstateRole& role, const std::shared_ptr& block, const CBlockIndex* pindex) override EXCLUSIVE_LOCKS_REQUIRED(!cs_signer); void BlockDisconnected(const std::shared_ptr& block, const CBlockIndex* pindex) override EXCLUSIVE_LOCKS_REQUIRED(!cs_signer); diff --git a/src/dbwrapper.cpp b/src/dbwrapper.cpp index 953d49aad549..c63c4400376c 100644 --- a/src/dbwrapper.cpp +++ b/src/dbwrapper.cpp @@ -128,7 +128,7 @@ static leveldb::Options GetOptions(size_t nCacheSize) } CDBWrapper::CDBWrapper(const fs::path& path, size_t nCacheSize, bool fMemory, bool fWipe, bool obfuscate) - : m_name{fs::PathToString(path.stem())}, m_path{path}, m_is_memory{fMemory} + : m_name{fs::PathToString(path.stem())} { penv = nullptr; readoptions.verify_checksums = true; diff --git a/src/dbwrapper.h b/src/dbwrapper.h index 95ef9fb9dca5..f5bb18f7422e 100644 --- a/src/dbwrapper.h +++ b/src/dbwrapper.h @@ -254,11 +254,6 @@ class CDBWrapper std::vector CreateObfuscateKey() const; - //! path to filesystem storage - const fs::path m_path; - - //! whether or not the database resides in memory - bool m_is_memory; public: /** @@ -335,14 +330,6 @@ class CDBWrapper return WriteBatch(batch, fSync); } - //! @returns filesystem path to the on-disk data. - std::optional StoragePath() { - if (m_is_memory) { - return {}; - } - return m_path; - } - template bool Exists(const K& key) const { diff --git a/src/dsnotificationinterface.cpp b/src/dsnotificationinterface.cpp index b80f74c7c765..3844e2d359a9 100644 --- a/src/dsnotificationinterface.cpp +++ b/src/dsnotificationinterface.cpp @@ -78,9 +78,9 @@ void CDSNotificationInterface::TransactionAddedToMempool(const CTransactionRef& m_dstxman.TransactionAddedToMempool(ptx); } -void CDSNotificationInterface::BlockConnected(ChainstateRole role, const std::shared_ptr& pblock, const CBlockIndex* pindex) +void CDSNotificationInterface::BlockConnected(const kernel::ChainstateRole& role, const std::shared_ptr& pblock, const CBlockIndex* pindex) { - if (role == ChainstateRole::BACKGROUND) return; + if (role.historical) return; m_dstxman.BlockConnected(pblock, pindex); } diff --git a/src/dsnotificationinterface.h b/src/dsnotificationinterface.h index 4ab61d486e0b..f2c84f5b6b6b 100644 --- a/src/dsnotificationinterface.h +++ b/src/dsnotificationinterface.h @@ -34,7 +34,7 @@ class CDSNotificationInterface : public CValidationInterface void SynchronousUpdatedBlockTip(const CBlockIndex *pindexNew, const CBlockIndex *pindexFork, bool fInitialDownload) override; void UpdatedBlockTip(const CBlockIndex *pindexNew, const CBlockIndex *pindexFork, bool fInitialDownload) override; void TransactionAddedToMempool(const CTransactionRef& tx, int64_t nAcceptTime, uint64_t mempool_sequence) override; - void BlockConnected(ChainstateRole role, const std::shared_ptr& pblock, const CBlockIndex* pindex) override; + void BlockConnected(const kernel::ChainstateRole& role, const std::shared_ptr& pblock, const CBlockIndex* pindex) override; void BlockDisconnected(const std::shared_ptr& pblock, const CBlockIndex* pindexDisconnected) override; void NotifyMasternodeListChanged(bool undo, const CDeterministicMNList& oldMNList, const CDeterministicMNListDiff& diff) override; void NotifyChainLock(const CBlockIndex* pindex, const std::shared_ptr& clsig) override; diff --git a/src/evo/snapshot_load.cpp b/src/evo/snapshot_load.cpp index 8a030c70137e..dc4086719063 100644 --- a/src/evo/snapshot_load.cpp +++ b/src/evo/snapshot_load.cpp @@ -345,11 +345,12 @@ bool ChainstateManager::PopulateAndValidateSnapshot( // has, hash its already-connected ordinary state now, before any snapshot // seeds enter the shared EvoDB namespace. std::map existing_background_work_hashes; - auto& background_dmnman{m_ibd_chainstate->ChainHelper().DeterministicMNManager()}; + Chainstate& background_chainstate{CurrentChainstate()}; + auto& background_dmnman{background_chainstate.ChainHelper().DeterministicMNManager()}; for (const auto& block_hash : required_work_blocks) { const CBlockIndex* index{m_blockman.LookupBlockIndex(block_hash)}; assert(index != nullptr); - if (m_ibd_chainstate->m_chain.Contains(index)) { + if (background_chainstate.m_chain.Contains(index)) { existing_background_work_hashes.emplace( block_hash, evo::CanonicalMNListHash(background_dmnman.GetListForBlock(index))); } @@ -440,7 +441,7 @@ bool ChainstateManager::PopulateAndValidateSnapshot( LogPrintf("[snapshot] failed to commit required historical MN-list marker\n"); return false; } - m_ibd_chainstate->SetRequiredBackgroundMNListHashes(required_work_blocks); + background_chainstate.SetRequiredBackgroundMNListHashes(required_work_blocks); if (!snapshot_chainstate.m_evoDb.CommitRootTransaction(EvoDbIdentity::SNAPSHOT, /*sync=*/true)) { LogPrintf("[snapshot] failed to commit snapshot EvoDB marker\n"); return false; @@ -465,21 +466,26 @@ bool ChainstateManager::PopulateAndValidateSnapshot( return true; } -SnapshotCompletionResult ChainstateManager::MaybeCompleteSnapshotValidation( +SnapshotCompletionResult ChainstateManager::MaybeValidateSnapshot( + Chainstate& validated_cs, + Chainstate& unvalidated_cs, std::function shutdown_fnc) { AssertLockHeld(cs_main); - if (m_ibd_chainstate.get() == &this->ActiveChainstate() || - !this->IsUsable(m_snapshot_chainstate.get()) || - !this->IsUsable(m_ibd_chainstate.get()) || - !m_ibd_chainstate->m_chain.Tip()) { + if (unvalidated_cs.m_assumeutxo != Assumeutxo::UNVALIDATED || + !unvalidated_cs.m_from_snapshot_blockhash || + validated_cs.m_assumeutxo != Assumeutxo::VALIDATED || + !validated_cs.m_chain.Tip() || + !validated_cs.m_target_blockhash || + *validated_cs.m_target_blockhash != *unvalidated_cs.m_from_snapshot_blockhash || + !validated_cs.ReachedTarget()) { // Nothing to do - this function only applies to the background // validation chainstate. return SnapshotCompletionResult::SKIPPED; } - const auto snapshot_base_height_opt = this->GetSnapshotBaseHeight(); - if (!snapshot_base_height_opt) { - if (!m_snapshot_chainstate->CoinsDB().StoragePath()) { + const CBlockIndex* snapshot_base{unvalidated_cs.SnapshotBase()}; + if (!snapshot_base) { + if (!fs::exists(unvalidated_cs.StoragePath())) { // Some Dash unit fixtures construct a synthetic in-memory snapshot // chainstate before inserting its base block into the block index. return SnapshotCompletionResult::SKIPPED; @@ -487,24 +493,18 @@ SnapshotCompletionResult ChainstateManager::MaybeCompleteSnapshotValidation( LogPrintf("[snapshot] on-disk snapshot base block is missing from the block index\n"); return SnapshotCompletionResult::BASE_BLOCKHASH_MISMATCH; } - const int snapshot_base_height = *snapshot_base_height_opt; - const CBlockIndex& index_new = *Assert(m_ibd_chainstate->m_chain.Tip()); + const int snapshot_base_height{snapshot_base->nHeight}; + const CBlockIndex& index_new{*Assert(validated_cs.m_chain.Tip())}; - if (index_new.nHeight < snapshot_base_height) { - // Background IBD not complete yet. - return SnapshotCompletionResult::SKIPPED; - } - - assert(SnapshotBlockhash()); - uint256 snapshot_blockhash = *Assert(SnapshotBlockhash()); + const uint256 snapshot_blockhash{*unvalidated_cs.m_from_snapshot_blockhash}; // Completion is serialized by cs_main. Flush each identity in sequence so // CEvoDB's single-open-transaction invariant is preserved and marker // promotion can atomically operate on fully committed transaction trees. - m_ibd_chainstate->ForceFlushStateToDisk(); - m_snapshot_chainstate->ForceFlushStateToDisk(); - if (!m_ibd_chainstate->m_evoDb.CommitRootTransaction(EvoDbIdentity::NORMAL, /*sync=*/true) || - !m_snapshot_chainstate->m_evoDb.CommitRootTransaction(EvoDbIdentity::SNAPSHOT, /*sync=*/true)) { + validated_cs.ForceFlushStateToDisk(); + unvalidated_cs.ForceFlushStateToDisk(); + if (!validated_cs.m_evoDb.CommitRootTransaction(EvoDbIdentity::NORMAL, /*sync=*/true) || + !unvalidated_cs.m_evoDb.CommitRootTransaction(EvoDbIdentity::SNAPSHOT, /*sync=*/true)) { LogPrintf("[snapshot] failed to sync completion EvoDB state\n"); return SnapshotCompletionResult::STATS_FAILED; } @@ -515,24 +515,16 @@ SnapshotCompletionResult ChainstateManager::MaybeCompleteSnapshotValidation( assert(handled); }; - if (index_new.GetBlockHash() != snapshot_blockhash) { - LogPrintf("[snapshot] supposed base block %s does not match the " /* Continued */ - "snapshot base block %s (height %d). Snapshot is not valid.", - index_new.ToString(), snapshot_blockhash.ToString(), snapshot_base_height); - handle_invalid_snapshot(); - return SnapshotCompletionResult::BASE_BLOCKHASH_MISMATCH; - } - + assert(index_new.GetBlockHash() == snapshot_blockhash); assert(index_new.nHeight == snapshot_base_height); - int curr_height = m_ibd_chainstate->m_chain.Height(); + int curr_height = validated_cs.m_chain.Height(); assert(snapshot_base_height == curr_height); assert(snapshot_base_height == index_new.nHeight); - assert(this->IsUsable(m_snapshot_chainstate.get())); assert(this->GetAll().size() == 2); - CCoinsViewDB& ibd_coins_db = m_ibd_chainstate->CoinsDB(); + CCoinsViewDB& ibd_coins_db = validated_cs.CoinsDB(); auto maybe_au_data = GetParams().AssumeutxoForHeight(curr_height); if (!maybe_au_data) { @@ -590,7 +582,7 @@ SnapshotCompletionResult ChainstateManager::MaybeCompleteSnapshotValidation( evo::CEvoSnapshot retained_snapshot; CBlock base_block; std::string evo_error; - if (!m_ibd_chainstate->m_evoDb.Read(EVODB_SNAPSHOT_EVO_SECTION, retained_snapshot) || + if (!validated_cs.m_evoDb.Read(EVODB_SNAPSHOT_EVO_SECTION, retained_snapshot) || !ReadBlockFromDisk(base_block, &index_new, GetConsensus()) || base_block.vtx.empty()) { LogPrintf("[snapshot] missing retained evo section/base block at completion\n"); handle_invalid_snapshot(); @@ -602,7 +594,7 @@ SnapshotCompletionResult ChainstateManager::MaybeCompleteSnapshotValidation( if (history_matches) { for (const auto& [block_hash, reconstructed_list] : reconstructed_history) { uint256 background_hash; - if (!m_ibd_chainstate->m_evoDb.ReadBackgroundWorkMNListHash(block_hash, background_hash) || + if (!validated_cs.m_evoDb.ReadBackgroundWorkMNListHash(block_hash, background_hash) || background_hash != evo::CanonicalMNListHash(reconstructed_list)) { evo_error = "missing or mismatched background historical MN-list capture"; history_matches = false; @@ -625,8 +617,8 @@ SnapshotCompletionResult ChainstateManager::MaybeCompleteSnapshotValidation( uint256 snapshot_mn_list_hash; uint256 background_mn_list_block; uint256 background_mn_list_hash; - if (!m_ibd_chainstate->m_evoDb.ReadSnapshotBaseMNListHash(snapshot_mn_list_hash) || - !m_ibd_chainstate->m_evoDb.ReadBackgroundMNListHash( + if (!validated_cs.m_evoDb.ReadSnapshotBaseMNListHash(snapshot_mn_list_hash) || + !validated_cs.m_evoDb.ReadBackgroundMNListHash( background_mn_list_block, background_mn_list_hash) || background_mn_list_block != snapshot_blockhash || snapshot_mn_list_hash != background_mn_list_hash) { @@ -637,9 +629,9 @@ SnapshotCompletionResult ChainstateManager::MaybeCompleteSnapshotValidation( return SnapshotCompletionResult::EVO_STATE_MISMATCH; } - const uint256 snapshot_tip = m_snapshot_chainstate->CoinsTip().GetBestBlock(); - if (!m_ibd_chainstate->m_evoDb.VerifyBestBlock(EvoDbIdentity::NORMAL, snapshot_blockhash) || - !m_snapshot_chainstate->m_evoDb.VerifyBestBlock(EvoDbIdentity::SNAPSHOT, snapshot_tip)) { + const uint256 snapshot_tip = unvalidated_cs.CoinsTip().GetBestBlock(); + if (!validated_cs.m_evoDb.VerifyBestBlock(EvoDbIdentity::NORMAL, snapshot_blockhash) || + !unvalidated_cs.m_evoDb.VerifyBestBlock(EvoDbIdentity::SNAPSHOT, snapshot_tip)) { LogPrintf("[snapshot] EvoDB best-block markers do not match their chainstate tips\n"); handle_invalid_snapshot(); return SnapshotCompletionResult::EVO_STATE_MISMATCH; @@ -648,7 +640,8 @@ SnapshotCompletionResult ChainstateManager::MaybeCompleteSnapshotValidation( LogPrintf("[snapshot] snapshot beginning at %s has been fully validated\n", snapshot_blockhash.ToString()); - m_ibd_chainstate->m_disabled = true; + unvalidated_cs.m_assumeutxo = Assumeutxo::VALIDATED; + validated_cs.m_target_utxohash = AssumeutxoHash{ibd_stats.hashSerialized}; ReleaseSnapshotPruneLock(); this->MaybeRebalanceCaches(); diff --git a/src/index/base.cpp b/src/index/base.cpp index 1e10efde7623..0e188e7af969 100644 --- a/src/index/base.cpp +++ b/src/index/base.cpp @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -21,6 +22,8 @@ using node::PruneLockInfo; using node::ReadBlockFromDisk; using node::fPruneMode; +using kernel::ChainstateRole; + constexpr uint8_t DB_BEST_BLOCK{'B'}; constexpr auto SYNC_LOG_INTERVAL{30s}; @@ -237,7 +240,6 @@ bool BaseIndex::Commit() bool BaseIndex::Rewind(const CBlockIndex* current_tip, const CBlockIndex* new_tip) { - assert(current_tip == m_best_block_index); assert(current_tip->GetAncestor(new_tip->nHeight) == new_tip); if (!CustomRewind({current_tip->GetBlockHash(), current_tip->nHeight}, {new_tip->GetBlockHash(), new_tip->nHeight})) { @@ -259,15 +261,13 @@ bool BaseIndex::Rewind(const CBlockIndex* current_tip, const CBlockIndex* new_ti return true; } -void BaseIndex::BlockConnected(ChainstateRole role, const std::shared_ptr& block, const CBlockIndex* pindex) +void BaseIndex::BlockConnected(const ChainstateRole& role, const std::shared_ptr& block, const CBlockIndex* pindex) { - // Ignore events from the assumed-valid chain; we will process its blocks - // (sequentially) after it is fully verified by the background chainstate. This - // is to avoid any out-of-order indexing. + // Ignore events from not fully validated chains to avoid out-of-order indexing. // // TODO at some point we could parameterize whether a particular index can be // built out of order, but for now just do the conservative simple thing. - if (role == ChainstateRole::ASSUMEDVALID) { + if (!role.validated) { return; } @@ -340,11 +340,10 @@ void BaseIndex::BlockDisconnected(const std::shared_ptr& block, co } } -void BaseIndex::ChainStateFlushed(ChainstateRole role, const CBlockLocator& locator) +void BaseIndex::ChainStateFlushed(const ChainstateRole& role, const CBlockLocator& locator) { - // Ignore events from the assumed-valid chain; we will process its blocks - // (sequentially) after it is fully verified by the background chainstate. - if (role == ChainstateRole::ASSUMEDVALID) { + // Ignore events from not fully validated chains to avoid out-of-order indexing. + if (!role.validated) { return; } @@ -418,7 +417,7 @@ bool BaseIndex::Start() { // m_chainstate member gives indexing code access to node internals. It is // removed in followup https://github.com/bitcoin/bitcoin/pull/24230 - m_chainstate = &WITH_LOCK(::cs_main, return m_chain->context()->chainman->GetChainstateForIndexing()); + m_chainstate = &WITH_LOCK(::cs_main, return m_chain->context()->chainman->ValidatedChainstate()); m_interrupt.reset(); // Need to register this ValidationInterface before running Init(), so that // callbacks are not missed if Init sets m_synced to true. diff --git a/src/index/base.h b/src/index/base.h index 907bb3c59307..e808fd56756d 100644 --- a/src/index/base.h +++ b/src/index/base.h @@ -104,11 +104,11 @@ class BaseIndex : public CValidationInterface std::unique_ptr m_chain; Chainstate* m_chainstate{nullptr}; - void BlockConnected(ChainstateRole role, const std::shared_ptr& block, const CBlockIndex* pindex) override; + void BlockConnected(const kernel::ChainstateRole& role, const std::shared_ptr& block, const CBlockIndex* pindex) override; void BlockDisconnected(const std::shared_ptr& block, const CBlockIndex* pindex) override; - void ChainStateFlushed(ChainstateRole role, const CBlockLocator& locator) override; + void ChainStateFlushed(const kernel::ChainstateRole& role, const CBlockLocator& locator) override; /// Initialize internal state from the database and block index. [[nodiscard]] virtual bool CustomInit(const std::optional& block) { return true; } diff --git a/src/init.cpp b/src/init.cpp index 5f868842629a..6dd9632cc452 100644 --- a/src/init.cpp +++ b/src/init.cpp @@ -353,7 +353,7 @@ void PrepareShutdown(NodeContext& node) // FlushStateToDisk generates a ChainStateFlushed callback, which we should avoid missing if (node.chainman) { LOCK(cs_main); - for (Chainstate* chainstate : node.chainman->GetAll()) { + for (const auto& chainstate : node.chainman->m_chainstates) { if (chainstate->CanFlushToDisk()) { chainstate->ForceFlushStateToDisk(); } @@ -423,7 +423,7 @@ void PrepareShutdown(NodeContext& node) if (node.chainman) { LOCK(cs_main); - for (Chainstate* chainstate : node.chainman->GetAll()) { + for (const auto& chainstate : node.chainman->m_chainstates) { if (chainstate->CanFlushToDisk()) { chainstate->ForceFlushStateToDisk(); chainstate->ResetCoinsViews(); @@ -2005,9 +2005,14 @@ bool AppInitMain(NodeContext& node, interfaces::BlockAndHeaderTipInfo* tip_info) node.chainman = std::make_unique(chainman_opts); ChainstateManager& chainman = *node.chainman; - // This is defined here instead of validation.h to avoid a dependency on - // index/base from libdashconsensus-facing validation code. - chainman.restart_indexes = [&node]() { + // This is defined and set here instead of inline in validation.h to avoid a hard + // dependency on index/base from libdashconsensus-facing validation code. + chainman.snapshot_download_completed = [&node]() { + if (!node::fPruneMode) { + LogPrintf("[snapshot] re-enabling NODE_NETWORK services\n"); + node.connman->AddLocalServices(NODE_NETWORK); + } + LogPrintf("[snapshot] restarting indexes\n"); SyncWithValidationInterfaceQueue(); @@ -2327,14 +2332,19 @@ bool AppInitMain(NodeContext& node, interfaces::BlockAndHeaderTipInfo* tip_info) if (fPruneMode) { if (!fReindex) { LOCK(cs_main); - for (Chainstate* chainstate : chainman.GetAll()) { + for (const auto& chainstate : chainman.m_chainstates) { uiInterface.InitMessage(_("Pruning blockstore…").translated); chainstate->PruneAndFlush(); } } } else { - LogPrintf("Setting NODE_NETWORK on non-prune mode\n"); - nLocalServices = ServiceFlags(nLocalServices | NODE_NETWORK); + // Prior to setting NODE_NETWORK, check if we can provide historical blocks. + if (!WITH_LOCK(chainman.GetMutex(), return chainman.HistoricalChainstate())) { + LogPrintf("Setting NODE_NETWORK on non-prune mode\n"); + nLocalServices = ServiceFlags(nLocalServices | NODE_NETWORK); + } else { + LogPrintf("Running node in NODE_NETWORK_LIMITED mode until snapshot background sync completes\n"); + } } // As PruneAndFlush can take several minutes, it's possible the user diff --git a/src/instantsend/net_instantsend.cpp b/src/instantsend/net_instantsend.cpp index 7fc4642c7085..0edd43b43c2c 100644 --- a/src/instantsend/net_instantsend.cpp +++ b/src/instantsend/net_instantsend.cpp @@ -525,9 +525,9 @@ void NetInstantSend::TransactionRemovedFromMempool(const CTransactionRef& tx, Me m_is_manager.TransactionIsRemoved(tx); } -void NetInstantSend::BlockConnected(ChainstateRole role, const std::shared_ptr& pblock, const CBlockIndex* pindex) +void NetInstantSend::BlockConnected(const kernel::ChainstateRole& role, const std::shared_ptr& pblock, const CBlockIndex* pindex) { - if (role == ChainstateRole::BACKGROUND) return; + if (role.historical) return; if (!m_is_manager.IsInstantSendEnabled()) { return; } diff --git a/src/instantsend/net_instantsend.h b/src/instantsend/net_instantsend.h index 48e6d6671e72..04cae66b3057 100644 --- a/src/instantsend/net_instantsend.h +++ b/src/instantsend/net_instantsend.h @@ -75,7 +75,7 @@ class NetInstantSend final : public NetHandler, public CValidationInterface void TransactionAddedToMempool(const CTransactionRef&, int64_t, uint64_t mempool_sequence) override; void TransactionRemovedFromMempool(const CTransactionRef& ptx, MemPoolRemovalReason reason, uint64_t mempool_sequence) override; - void BlockConnected(ChainstateRole role, const std::shared_ptr& pblock, const CBlockIndex* pindex) override; + void BlockConnected(const kernel::ChainstateRole& role, const std::shared_ptr& pblock, const CBlockIndex* pindex) override; void BlockDisconnected(const std::shared_ptr& pblock, const CBlockIndex* pindexDisconnected) override; void NotifyChainLock(const CBlockIndex* pindex, const std::shared_ptr& clsig) override; diff --git a/src/interfaces/chain.h b/src/interfaces/chain.h index 6be400577644..21918a1ff8a9 100644 --- a/src/interfaces/chain.h +++ b/src/interfaces/chain.h @@ -29,7 +29,6 @@ class Coin; class uint256; enum class MemPoolRemovalReason; enum class RBFTransactionState; -enum class ChainstateRole; struct bilingual_str; struct CBlockLocator; struct FeeCalculation; @@ -39,6 +38,9 @@ struct ChainLockSig; namespace instantsend { struct InstantSendLock; } // namespace instantsend +namespace kernel { +struct ChainstateRole; +} // namespace kernel namespace node { struct NodeContext; } // namespace node @@ -254,6 +256,9 @@ class Chain //! Check if any block has been pruned. virtual bool havePruned() = 0; + //! Get the current prune height. + virtual std::optional getPruneHeight() = 0; + //! Check if the node is ready to broadcast transactions. virtual bool isReadyToBroadcast() = 0; @@ -282,10 +287,10 @@ class Chain virtual ~Notifications() {} virtual void transactionAddedToMempool(const CTransactionRef& tx, int64_t nAcceptTime) {} virtual void transactionRemovedFromMempool(const CTransactionRef& tx, MemPoolRemovalReason reason) {} - virtual void blockConnected(ChainstateRole role, const BlockInfo& block) {} + virtual void blockConnected(const kernel::ChainstateRole& role, const BlockInfo& block) {} virtual void blockDisconnected(const BlockInfo& block) {} virtual void updatedBlockTip() {} - virtual void chainStateFlushed(ChainstateRole role, const CBlockLocator& locator) {} + virtual void chainStateFlushed(const kernel::ChainstateRole& role, const CBlockLocator& locator) {} virtual void notifyChainLock(const CBlockIndex* pindexChainLock, const std::shared_ptr& clsig) {} virtual void notifyTransactionLock(const CTransactionRef &tx, const std::shared_ptr& islock) {} }; @@ -330,7 +335,9 @@ class Chain //! removed transactions and already added new transactions. virtual void requestMempoolTransactions(Notifications& notifications) = 0; - //! Return true if an assumed-valid chain is in use. + //! Return true if an assumed-valid snapshot is in use. Note that this + //! returns true even after the snapshot is validated, until the next node + //! restart. virtual bool hasAssumedValidChain() = 0; //! Get internal node context. Useful for testing, but not diff --git a/src/kernel/chain.cpp b/src/kernel/chain.cpp index a248e4dc3f1d..51351fee0abf 100644 --- a/src/kernel/chain.cpp +++ b/src/kernel/chain.cpp @@ -4,6 +4,7 @@ #include #include +#include #include #include @@ -23,14 +24,15 @@ interfaces::BlockInfo MakeBlockInfo(const CBlockIndex* index, const CBlock* data info.data = data; return info; } -} // namespace kernel std::ostream& operator<<(std::ostream& os, const ChainstateRole& role) { - switch(role) { - case ChainstateRole::NORMAL: os << "normal"; break; - case ChainstateRole::ASSUMEDVALID: os << "assumedvalid"; break; - case ChainstateRole::BACKGROUND: os << "background"; break; - default: os.setstate(std::ios_base::failbit); + if (!role.validated) { + os << "assumedvalid"; + } else if (role.historical) { + os << "background"; + } else { + os << "normal"; } return os; } +} // namespace kernel diff --git a/src/kernel/chain.h b/src/kernel/chain.h index 50b6f5cf9f2d..e08fb5f55a01 100644 --- a/src/kernel/chain.h +++ b/src/kernel/chain.h @@ -32,26 +32,10 @@ struct BlockInfo { } // namespace interfaces namespace kernel { +struct ChainstateRole; //! Return data from block index. interfaces::BlockInfo MakeBlockInfo(const CBlockIndex* block_index, const CBlock* data = nullptr); - -} // namespace kernel - -//! This enum describes the various roles a specific Chainstate instance can take. -//! Other parts of the system sometimes need to vary in behavior depending on the -//! existence of a background validation chainstate, e.g. when building indexes. -enum class ChainstateRole { - // Single chainstate in use, "normal" IBD mode. - NORMAL, - - // Doing IBD-style validation in the background. Implies use of an assumed-valid - // chainstate. - BACKGROUND, - - // Active assumed-valid chainstate. Implies use of a background IBD chainstate. - ASSUMEDVALID, -}; - std::ostream& operator<<(std::ostream& os, const ChainstateRole& role); +} // namespace kernel #endif // BITCOIN_KERNEL_CHAIN_H diff --git a/src/kernel/types.h b/src/kernel/types.h new file mode 100644 index 000000000000..164d4cfda81d --- /dev/null +++ b/src/kernel/types.h @@ -0,0 +1,30 @@ +// Copyright (c) The Bitcoin Core developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +//! @file kernel/types.h is a home for simple enum and struct type definitions +//! that can be used internally by functions in the libbitcoin_kernel library, +//! but also used externally by node, wallet, and GUI code. +//! +//! This file is intended to define only simple types that do not have external +//! dependencies. More complicated types should be defined in dedicated header +//! files. + +#ifndef BITCOIN_KERNEL_TYPES_H +#define BITCOIN_KERNEL_TYPES_H + +namespace kernel { +//! Information about chainstate that notifications are sent from. +struct ChainstateRole { + //! Whether this is a notification from a chainstate that's been fully + //! validated starting from the genesis block. False if it is from an + //! assumeutxo snapshot chainstate that has not been validated yet. + bool validated{true}; + + //! Whether this is a historical chainstate downloading old blocks to + //! validate an assumeutxo snapshot, not syncing to the network tip. + bool historical{false}; +}; +} // namespace kernel + +#endif // BITCOIN_KERNEL_TYPES_H diff --git a/src/net.cpp b/src/net.cpp index c2eb7543dcaf..cc4073a94d30 100644 --- a/src/net.cpp +++ b/src/net.cpp @@ -1969,7 +1969,7 @@ void CConnman::CreateNodeFromAcceptedSocket(std::unique_ptr&& sock, NodeId id = GetNewNodeId(); uint64_t nonce = GetDeterministicRandomizer(RANDOMIZER_ID_LOCALHOSTNONCE).Write(id).Finalize(); - ServiceFlags nodeServices = nLocalServices; + ServiceFlags nodeServices = GetLocalServices(); if (NetPermissions::HasFlag(permission_flags, NetPermissionFlags::BloomFilter)) { nodeServices = static_cast(nodeServices | NODE_BLOOM); } @@ -3645,7 +3645,7 @@ void CConnman::OpenNetworkConnection(const CAddress& addrConnect, bool fCountFai mapSocketToNode.emplace(pnode->m_sock->Get(), pnode); } - m_msgproc->InitializeNode(*pnode, nLocalServices); + m_msgproc->InitializeNode(*pnode, GetLocalServices()); { LOCK(m_nodes_mutex); m_nodes.push_back(pnode); diff --git a/src/net.h b/src/net.h index 1c4919dbb3ef..37f7f3098fbb 100644 --- a/src/net.h +++ b/src/net.h @@ -1501,6 +1501,11 @@ friend class CNode; //! that peer during `net_processing.cpp:PushNodeVersion()`. ServiceFlags GetLocalServices() const; + //! Updates the local services that this node advertises to other peers + //! during connection handshake. + void AddLocalServices(ServiceFlags services) { nLocalServices = ServiceFlags(nLocalServices | services); }; + void RemoveLocalServices(ServiceFlags services) { nLocalServices = ServiceFlags(nLocalServices & ~services); } + uint64_t GetMaxOutboundTarget() const EXCLUSIVE_LOCKS_REQUIRED(!m_total_bytes_sent_mutex); std::chrono::seconds GetMaxOutboundTimeframe() const; @@ -1836,11 +1841,12 @@ friend class CNode; * This data is replicated in each Peer instance we create. * * This data is not marked const, but after being set it should not - * change. + * change. Unless AssumeUTXO is started, in which case, the peer + * will be limited until the background chain sync finishes. * * \sa Peer::our_services */ - ServiceFlags nLocalServices; + std::atomic nLocalServices; std::unique_ptr semOutbound; std::unique_ptr semAddnode; diff --git a/src/net_processing.cpp b/src/net_processing.cpp index bff7d642775f..129c91f73c14 100644 --- a/src/net_processing.cpp +++ b/src/net_processing.cpp @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -79,6 +80,7 @@ #include #include +using kernel::ChainstateRole; using node::ReadBlockFromDisk; using node::fImporting; using node::fPruneMode; @@ -582,7 +584,7 @@ class PeerManagerImpl final : public PeerManager } /** Overridden from CValidationInterface. */ - void BlockConnected(ChainstateRole role, const std::shared_ptr& pblock, const CBlockIndex* pindexConnected) override + void BlockConnected(const ChainstateRole& role, const std::shared_ptr& pblock, const CBlockIndex* pindexConnected) override EXCLUSIVE_LOCKS_REQUIRED(!m_peer_mutex, !m_recent_confirmed_transactions_mutex); void BlockDisconnected(const std::shared_ptr &block, const CBlockIndex* pindex) override EXCLUSIVE_LOCKS_REQUIRED(!m_recent_confirmed_transactions_mutex); @@ -1466,11 +1468,13 @@ void PeerManagerImpl::FindNextBlocksToDownload(const Peer& peer, unsigned int co return; } - // When we sync with AssumeUtxo and discover the snapshot is not in the peer's best chain, abort: - // We can't reorg to this chain due to missing undo data until the background sync has finished, + // When syncing with AssumeUtxo and the snapshot has not yet been validated, + // abort downloading blocks from peers that don't have the snapshot block in their best chain. + // We can't reorg to this chain due to missing undo data until validation completes, // so downloading blocks from it would be futile. - const CBlockIndex* snap_base{m_chainman.GetSnapshotBaseBlock()}; - if (snap_base && state->pindexBestKnownBlock->GetAncestor(snap_base->nHeight) != snap_base) { + const CBlockIndex* snap_base{m_chainman.CurrentChainstate().SnapshotBase()}; + if (snap_base && m_chainman.CurrentChainstate().m_assumeutxo == Assumeutxo::UNVALIDATED && + state->pindexBestKnownBlock->GetAncestor(snap_base->nHeight) != snap_base) { LogDebug(BCLog::NET, "Not downloading blocks from peer=%d, which doesn't have the snapshot block in its best chain.\n", peer.m_id); return; } @@ -2155,7 +2159,7 @@ void PeerManagerImpl::StartScheduledTasks(CScheduler& scheduler) * possibly reduce dynamic block stalling timeout. */ void PeerManagerImpl::BlockConnected( - ChainstateRole role, + const ChainstateRole& role, const std::shared_ptr& pblock, const CBlockIndex* pindex) { @@ -2174,8 +2178,8 @@ void PeerManagerImpl::BlockConnected( } // The following task can be skipped since we don't maintain a mempool for - // the ibd/background chainstate. - if (role == ChainstateRole::BACKGROUND) { + // the historical chainstate. + if (role.historical) { return; } @@ -6714,18 +6718,19 @@ bool PeerManagerImpl::SendMessages(CNode* pto) return std::max(0, MAX_BLOCKS_IN_TRANSIT_PER_PEER - static_cast(state.vBlocksInFlight.size())); }; - // If a snapshot chainstate is in use, we want to find its next blocks - // before the background chainstate to prioritize getting to network tip. + // If there are multiple chainstates, download blocks for the + // current chainstate first, to prioritize getting to network tip + // before downloading historical blocks. FindNextBlocksToDownload(*peer, get_inflight_budget(), vToDownload, staller); - if (m_chainman.BackgroundSyncInProgress() && !IsLimitedPeer(*peer)) { - // If the background tip is not an ancestor of the snapshot block, + auto historical_blocks{m_chainman.GetHistoricalBlockRange()}; + if (historical_blocks && !IsLimitedPeer(*peer)) { + // If the first needed historical block is not an ancestor of the last, // we need to start requesting blocks from their last common ancestor. - const CBlockIndex *from_tip = LastCommonAncestor(m_chainman.GetBackgroundSyncTip(), m_chainman.GetSnapshotBaseBlock()); + const CBlockIndex* from_tip = LastCommonAncestor(historical_blocks->first, historical_blocks->second); TryDownloadingHistoricalBlocks( *peer, get_inflight_budget(), - vToDownload, from_tip, - Assert(m_chainman.GetSnapshotBaseBlock())); + vToDownload, from_tip, historical_blocks->second); } for (const CBlockIndex *pindex : vToDownload) { vGetData.emplace_back(MSG_BLOCK, pindex->GetBlockHash()); diff --git a/src/node/blockstorage.cpp b/src/node/blockstorage.cpp index aad737422f06..cbcc50e7661f 100644 --- a/src/node/blockstorage.cpp +++ b/src/node/blockstorage.cpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -167,8 +168,7 @@ void BlockManager::PruneOneBlockFile(const int fileNumber) void BlockManager::FindFilesToPruneManual( std::set& setFilesToPrune, int nManualPruneHeight, - const Chainstate& chain, - ChainstateManager& chainman) + const Chainstate& chain) { assert(fPruneMode && nManualPruneHeight > 0); @@ -177,7 +177,7 @@ void BlockManager::FindFilesToPruneManual( return; } - const auto [min_block_to_prune, last_block_can_prune] = chainman.GetPruneRange(chain, nManualPruneHeight); + const auto [min_block_to_prune, last_block_can_prune] = chain.GetPruneRange(nManualPruneHeight); int count = 0; for (int fileNumber = 0; fileNumber < this->MaxBlockfileNum(); fileNumber++) { @@ -204,12 +204,20 @@ void BlockManager::FindFilesToPrune( if (chain.m_chain.Height() < 0 || nPruneTarget == 0) { return; } + // Compute `target` value with maximum size (in bytes) of blocks below the + // `last_prune` height which should be preserved and not pruned. The + // `target` value will be derived from the -prune preference provided by the + // user. If there is a historical chainstate being used to populate indexes + // and validate the snapshot, the target is divided by two so half of the + // block storage will be reserved for the historical chainstate, and the + // other half will be reserved for the most-work chainstate. + const int num_chainstates{chainman.HistoricalChainstate() ? 2 : 1}; // Distribute our -prune budget over all chainstates. On regtest, preserve // the lower configured target used by pruning tests. const auto target = std::max( std::min(MIN_DISK_SPACE_FOR_BLOCK_FILES, nPruneTarget), - nPruneTarget / chainman.GetAll().size()); + nPruneTarget / num_chainstates); if (target == 0) { return; @@ -218,7 +226,7 @@ void BlockManager::FindFilesToPrune( return; } - const auto [min_block_to_prune, last_block_can_prune] = chainman.GetPruneRange(chain, last_prune); + const auto [min_block_to_prune, last_block_can_prune] = chain.GetPruneRange(last_prune); uint64_t nCurrentUsage = CalculateCurrentUsage(); // We don't check to prune until after we've allocated new space for files @@ -519,7 +527,7 @@ bool BlockManager::IsBlockPruned(const CBlockIndex* pblockindex) return (m_have_pruned && !(pblockindex->nStatus & BLOCK_HAVE_DATA) && pblockindex->nTx > 0); } -const CBlockIndex* BlockManager::GetFirstStoredBlock(const CBlockIndex& start_block) +const CBlockIndex* BlockManager::GetFirstStoredBlock(const CBlockIndex& start_block) const { AssertLockHeld(::cs_main); const CBlockIndex* last_block = &start_block; @@ -1075,15 +1083,10 @@ void ThreadImport(ChainstateManager& chainman, std::vector vImportFile // scan for better chains in the block chain database, that are not yet connected in the active best chain - // We can't hold cs_main during ActivateBestChain even though we're accessing - // the chainman unique_ptrs since ABC requires us not to be holding cs_main, so retrieve - // the relevant pointers before the ABC call. - for (Chainstate* chainstate : WITH_LOCK(::cs_main, return chainman.GetAll())) { - BlockValidationState state; - if (!chainstate->ActivateBestChain(state, nullptr)) { - AbortNode(strprintf("Failed to connect best block (%s)", state.ToString())); - return; - } + if (auto result{chainman.ActivateBestChains()}; !result) { + LogPrintf("Failed to connect best block (%s)\n", util::ErrorString(result).original); + StartShutdown(); + return; } if (args.GetBoolArg("-stopafterblockimport", DEFAULT_STOPAFTERBLOCKIMPORT)) { diff --git a/src/node/blockstorage.h b/src/node/blockstorage.h index 5306e59bbf07..ada267e9bb3d 100644 --- a/src/node/blockstorage.h +++ b/src/node/blockstorage.h @@ -139,8 +139,7 @@ class BlockManager void FindFilesToPruneManual( std::set& setFilesToPrune, int nManualPruneHeight, - const Chainstate& chain, - ChainstateManager& chainman); + const Chainstate& chain); /** * Prune block and undo files (blk???.dat and rev???.dat) so that the disk space used is less than a user-defined target. @@ -282,7 +281,7 @@ class BlockManager const CBlockIndex* GetLastCheckpoint(const CCheckpointData& data) EXCLUSIVE_LOCKS_REQUIRED(cs_main); //! Find the first block that is not pruned - const CBlockIndex* GetFirstStoredBlock(const CBlockIndex& start_block LIFETIMEBOUND) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + const CBlockIndex* GetFirstStoredBlock(const CBlockIndex& start_block LIFETIMEBOUND) const EXCLUSIVE_LOCKS_REQUIRED(::cs_main); /** True if any block files have ever been pruned. */ bool m_have_pruned = false; diff --git a/src/node/chainstate.cpp b/src/node/chainstate.cpp index 64d24b9b57dc..c1c4b1bd8ff5 100644 --- a/src/node/chainstate.cpp +++ b/src/node/chainstate.cpp @@ -229,23 +229,24 @@ static ChainstateLoadResult CompleteChainstateInitialization(ChainstateManager& return {ChainstateLoadStatus::FAILURE, _("Error initializing block database")}; } - auto is_coinsview_empty = [&](Chainstate* chainstate) EXCLUSIVE_LOCKS_REQUIRED(::cs_main) { - return options.reindex || options.reindex_chainstate || chainstate->CoinsTip().GetBestBlock().IsNull(); + auto is_coinsview_empty = [&](Chainstate& chainstate) EXCLUSIVE_LOCKS_REQUIRED(::cs_main) { + return options.reindex || options.reindex_chainstate || chainstate.CoinsTip().GetBestBlock().IsNull(); }; assert(chainman.m_total_coinstip_cache > 0); assert(chainman.m_total_coinsdb_cache > 0); - // Conservative value which is arbitrarily chosen, as it will ultimately be changed - // by a call to `chainman.MaybeRebalanceCaches()`. We just need to make sure - // that the sum of the two caches (40%) does not exceed the allowable amount - // during this temporary initialization state. - double init_cache_fraction = 0.2; + // If running with multiple chainstates, limit the cache sizes with a + // discount factor. If discounted the actual cache size will be + // recalculated by `chainman.MaybeRebalanceCaches()`. The discount factor + // is conservatively chosen such that the sum of the caches does not exceed + // the allowable amount during this temporary initialization state. + double init_cache_fraction = chainman.HistoricalChainstate() ? 0.2 : 1.0; // At this point we're either in reindex or we've loaded a useful // block tree into BlockIndex()! - for (Chainstate* chainstate : chainman.GetAll()) { + for (const auto& chainstate : chainman.m_chainstates) { LogPrintf("Initializing chainstate %s\n", chainstate->ToString()); chainstate->InitCoinsDB( @@ -287,7 +288,7 @@ static ChainstateLoadResult CompleteChainstateInitialization(ChainstateManager& return {ChainstateLoadStatus::FAILURE, _("Failed to commit Evo database")}; } - if (!is_coinsview_empty(chainstate)) { + if (!is_coinsview_empty(*chainstate)) { // LoadChainTip initializes the chain based on CoinsTip()'s best block if (!chainstate->LoadChainTip()) { return {ChainstateLoadStatus::FAILURE, _("Error initializing block database")}; @@ -304,7 +305,6 @@ static ChainstateLoadResult CompleteChainstateInitialization(ChainstateManager& if (dmnman->IsMigrationRequired() && !dmnman->MigrateLegacyDiffs(chainman.ActiveChainstate().m_chain.Tip())) { return {ChainstateLoadStatus::FAILURE, _("Failed to upgrade Evo database")}; } - // Now that chainstates are loaded and we're able to flush to // disk, rebalance the coins caches to desired levels based // on the condition of each chainstate. @@ -348,14 +348,25 @@ ChainstateLoadResult LoadChainstate(ChainstateManager& chainman, const CacheSize chainman.m_total_coinsdb_cache = cache_sizes.coins_db; // Load the fully validated chainstate. - chainman.InitializeChainstate(options.mempool, *evodb, chain_helper); + Chainstate& validated_cs{chainman.InitializeChainstate(options.mempool, *evodb, chain_helper)}; // Load a chain created from a UTXO snapshot, if any exist. bilingual_str snapshot_error; - if (!chainman.DetectSnapshotChainstate(options.mempool, snapshot_error)) { + Chainstate* assumeutxo_cs{chainman.LoadAssumeutxoChainstate(options.mempool, snapshot_error)}; + if (!snapshot_error.empty()) { return {ChainstateLoadStatus::FAILURE, snapshot_error}; } + if (assumeutxo_cs && (options.reindex || options.reindex_chainstate)) { + // Reset chainstate target to network tip instead of snapshot block. + validated_cs.SetTargetBlock(nullptr); + LogInfo("[snapshot] deleting snapshot chainstate due to reindexing"); + if (!chainman.DeleteChainstate(*assumeutxo_cs)) { + return {ChainstateLoadStatus::FAILURE_FATAL, Untranslated("Couldn't remove snapshot chainstate.")}; + } + assumeutxo_cs = nullptr; + } + auto [init_status, init_error] = CompleteChainstateInitialization(chainman, cache_sizes, options, *evodb, dmnman, llmq_ctx, chain_helper); if (init_status != ChainstateLoadStatus::SUCCESS) { @@ -378,7 +389,9 @@ ChainstateLoadResult LoadChainstate(ChainstateManager& chainman, const CacheSize // snapshot is actually validated? Because this entails unusual // filesystem operations to move leveldb data directories around, and that seems // too risky to do in the middle of normal runtime. - const auto snapshot_completion = chainman.MaybeCompleteSnapshotValidation(); + auto snapshot_completion{assumeutxo_cs + ? chainman.MaybeValidateSnapshot(validated_cs, *assumeutxo_cs) + : SnapshotCompletionResult::SKIPPED}; if (snapshot_completion == SnapshotCompletionResult::SKIPPED) { // Do nothing; expected case. @@ -392,16 +405,14 @@ ChainstateLoadResult LoadChainstate(ChainstateManager& chainman, const CacheSize chain_helper.reset(); llmq_ctx.reset(); dmnman.reset(); - if (!chainman.ValidatedSnapshotCleanup()) { + if (!chainman.ValidatedSnapshotCleanup(validated_cs, *assumeutxo_cs)) { return {ChainstateLoadStatus::FAILURE_FATAL, Untranslated("Background chainstate cleanup failed unexpectedly.")}; } // Because ValidatedSnapshotCleanup() has torn down chainstates with // ChainstateManager::ResetChainstates(), reinitialize them here without // duplicating the blockindex work above. - assert(chainman.GetAll().empty()); - assert(!chainman.IsSnapshotActive()); - assert(!chainman.IsSnapshotValidated()); + assert(chainman.m_chainstates.empty()); chainman.InitializeChainstate(options.mempool, *evodb, chain_helper); @@ -426,14 +437,14 @@ ChainstateLoadResult LoadChainstate(ChainstateManager& chainman, const CacheSize ChainstateLoadResult VerifyLoadedChainstate(ChainstateManager& chainman, const ChainstateLoadOptions& options, CEvoDB& evodb, std::function notify_bls_state) { - auto is_coinsview_empty = [&](Chainstate* chainstate) EXCLUSIVE_LOCKS_REQUIRED(::cs_main) { - return options.reindex || options.reindex_chainstate || chainstate->CoinsTip().GetBestBlock().IsNull(); + auto is_coinsview_empty = [&](Chainstate& chainstate) EXCLUSIVE_LOCKS_REQUIRED(::cs_main) { + return options.reindex || options.reindex_chainstate || chainstate.CoinsTip().GetBestBlock().IsNull(); }; LOCK(cs_main); - for (Chainstate* chainstate : chainman.GetAll()) { - if (!is_coinsview_empty(chainstate)) { + for (auto& chainstate : chainman.m_chainstates) { + if (!is_coinsview_empty(*chainstate)) { const CBlockIndex* tip = chainstate->m_chain.Tip(); if (tip && tip->nTime > GetTime() + MAX_FUTURE_BLOCK_TIME) { return {ChainstateLoadStatus::FAILURE, _("The block database contains a block which appears to be from the future. " @@ -470,7 +481,7 @@ ChainstateLoadResult VerifyLoadedChainstate(ChainstateManager& chainman, const C // TODO: CEvoDB instance should probably be a part of Chainstate // (for multiple chainstates to actually work in parallel) // and not a global - if (&chainman.ActiveChainstate() == chainstate && !evodb.IsEmpty()) { + if (&chainman.ActiveChainstate() == chainstate.get() && !evodb.IsEmpty()) { // EvoDB processed some blocks earlier but we have no blocks anymore, something is wrong return {ChainstateLoadStatus::FAILURE, _("Error initializing block database")}; } diff --git a/src/node/interfaces.cpp b/src/node/interfaces.cpp index 6539edd6afb2..f5f727c02864 100644 --- a/src/node/interfaces.cpp +++ b/src/node/interfaces.cpp @@ -50,6 +50,7 @@ #include #include #include +#include #include #include #include @@ -99,6 +100,7 @@ using interfaces::MnList; using interfaces::MnListPtr; using interfaces::Node; using interfaces::WalletLoader; +using kernel::ChainstateRole; namespace node { namespace { @@ -1109,7 +1111,7 @@ class NotificationsProxy : public CValidationInterface { m_notifications->transactionRemovedFromMempool(tx, reason); } - void BlockConnected(ChainstateRole role, const std::shared_ptr& block, const CBlockIndex* index) override + void BlockConnected(const ChainstateRole& role, const std::shared_ptr& block, const CBlockIndex* index) override { m_notifications->blockConnected(role, kernel::MakeBlockInfo(index, block.get())); } @@ -1121,7 +1123,7 @@ class NotificationsProxy : public CValidationInterface { m_notifications->updatedBlockTip(); } - void ChainStateFlushed(ChainstateRole role, const CBlockLocator& locator) override + void ChainStateFlushed(const ChainstateRole& role, const CBlockLocator& locator) override { m_notifications->chainStateFlushed(role, locator); } @@ -1447,6 +1449,11 @@ class ChainImpl : public Chain LOCK(::cs_main); return chainman().m_blockman.m_have_pruned; } + std::optional getPruneHeight() override + { + LOCK(chainman().GetMutex()); + return GetPruneHeight(chainman().m_blockman, chainman().ActiveChain()); + } bool isReadyToBroadcast() override { return !node::fImporting && !node::fReindex && !isInitialBlockDownload(); } bool isInitialBlockDownload() override { return chainman().ActiveChainstate().IsInitialBlockDownload(); @@ -1520,7 +1527,8 @@ class ChainImpl : public Chain } bool hasAssumedValidChain() override { - return chainman().IsSnapshotActive(); + LOCK(::cs_main); + return bool{chainman().CurrentChainstate().m_from_snapshot_blockhash}; } NodeContext* context() override { return &m_node; } diff --git a/src/node/utxo_snapshot.cpp b/src/node/utxo_snapshot.cpp index 9a9e4332ea83..67eb15909c81 100644 --- a/src/node/utxo_snapshot.cpp +++ b/src/node/utxo_snapshot.cpp @@ -22,9 +22,7 @@ bool WriteSnapshotBaseBlockhash(Chainstate& snapshot_chainstate) AssertLockHeld(::cs_main); assert(snapshot_chainstate.m_from_snapshot_blockhash); - const std::optional chaindir = snapshot_chainstate.CoinsDB().StoragePath(); - assert(chaindir); // Sanity check that chainstate isn't in-memory. - const fs::path write_to = *chaindir / node::SNAPSHOT_BLOCKHASH_FILENAME; + const fs::path write_to = snapshot_chainstate.StoragePath() / node::SNAPSHOT_BLOCKHASH_FILENAME; FILE* file{fsbridge::fopen(write_to, "wb")}; AutoFile afile{file}; @@ -46,7 +44,7 @@ bool WriteSnapshotBaseBlockhash(Chainstate& snapshot_chainstate) fs::PathToString(write_to)); return false; } - DirectoryCommit(*chaindir); + DirectoryCommit(write_to.parent_path()); return true; } @@ -85,10 +83,10 @@ std::optional ReadSnapshotBaseBlockhash(fs::path chaindir) return base_blockhash; } -std::optional FindSnapshotChainstateDir() +std::optional FindAssumeutxoChainstateDir(const fs::path& data_dir) { fs::path possible_dir = - gArgs.GetDataDirNet() / fs::u8path(strprintf("chainstate%s", SNAPSHOT_CHAINSTATE_SUFFIX)); + data_dir / fs::u8path(strprintf("chainstate%s", SNAPSHOT_CHAINSTATE_SUFFIX)); if (fs::exists(possible_dir)) { return possible_dir; diff --git a/src/node/utxo_snapshot.h b/src/node/utxo_snapshot.h index e900d4ec2b39..6820b718f911 100644 --- a/src/node/utxo_snapshot.h +++ b/src/node/utxo_snapshot.h @@ -6,18 +6,32 @@ #ifndef BITCOIN_NODE_UTXO_SNAPSHOT_H #define BITCOIN_NODE_UTXO_SNAPSHOT_H +#include #include +#include #include #include +#include #include +#include +#include #include +#include + +// UTXO set snapshot magic bytes. +static constexpr std::array SNAPSHOT_MAGIC_BYTES = {'u', 't', 'x', 'o', 0xff}; namespace node { //! Metadata describing a serialized version of a UTXO set from which an -//! assumeutxo Chainstate can be constructed. +//! assumeutxo Chainstate can be constructed. All fields come from an untrusted +//! file and must be validated before use. class SnapshotMetadata { + inline static const uint16_t VERSION{2}; + const std::set m_supported_versions{VERSION}; + std::array m_network_magic; + public: //! The hash of the block that reflects the tip of the chain for the //! UTXO set contained in this snapshot. @@ -27,15 +41,55 @@ class SnapshotMetadata //! during snapshot load to estimate progress of UTXO set reconstruction. uint64_t m_coins_count = 0; - SnapshotMetadata() { } SnapshotMetadata( + const CMessageHeader::MessageStartChars& network_magic) { + std::copy(std::begin(network_magic), std::end(network_magic), m_network_magic.begin()); + } + SnapshotMetadata( + const CMessageHeader::MessageStartChars& network_magic, const uint256& base_blockhash, - uint64_t coins_count, - unsigned int nchaintx) : + uint64_t coins_count) : m_base_blockhash(base_blockhash), - m_coins_count(coins_count) { } + m_coins_count(coins_count) { + std::copy(std::begin(network_magic), std::end(network_magic), m_network_magic.begin()); + } + + template + void Serialize(Stream& s) const + { + s << SNAPSHOT_MAGIC_BYTES; + s << VERSION; + s << m_network_magic; + s << m_base_blockhash; + s << m_coins_count; + } + + template + void Unserialize(Stream& s) + { + std::array snapshot_magic; + s >> snapshot_magic; + if (snapshot_magic != SNAPSHOT_MAGIC_BYTES) { + throw std::ios_base::failure("Invalid UTXO set snapshot magic bytes. Please check if this is indeed a snapshot file or if you are using an outdated snapshot format."); + } + + uint16_t version; + s >> version; + if (!m_supported_versions.count(version)) { + throw std::ios_base::failure(strprintf("Version of snapshot %s does not match any of the supported versions.", version)); + } + + std::array message; + s >> message; + if (message != m_network_magic) { + throw std::ios_base::failure(strprintf( + "The network magic of the snapshot (%s) does not match the network magic of this node (%s).", + HexStr(message), HexStr(m_network_magic))); + } - SERIALIZE_METHODS(SnapshotMetadata, obj) { READWRITE(obj.m_base_blockhash, obj.m_coins_count); } + s >> m_base_blockhash; + s >> m_coins_count; + } }; //! The file in the snapshot chainstate dir which stores the base blockhash. This is @@ -62,7 +116,7 @@ constexpr std::string_view SNAPSHOT_CHAINSTATE_SUFFIX = "_snapshot"; //! Return a path to the snapshot-based chainstate dir, if one exists. -std::optional FindSnapshotChainstateDir(); +std::optional FindAssumeutxoChainstateDir(const fs::path& data_dir); } // namespace node diff --git a/src/rpc/blockchain.cpp b/src/rpc/blockchain.cpp index 7d64c8e083c4..500b64c7a6cb 100644 --- a/src/rpc/blockchain.cpp +++ b/src/rpc/blockchain.cpp @@ -53,6 +53,13 @@ #include #include #include + +std::optional GetPruneHeight(const node::BlockManager& blockman, const CChain& chain) +{ + AssertLockHeld(::cs_main); + if (!blockman.m_have_pruned || !chain.Tip()) return std::nullopt; + return std::max(0, blockman.GetFirstStoredBlock(*chain.Tip())->nHeight - 1); +} #include #include #include @@ -1988,6 +1995,7 @@ void ReconsiderBlock(ChainstateManager& chainman, uint256 block_hash, bool ignor } chainman.ActiveChainstate().ResetBlockFailureFlags(pblockindex, ignore_chainlocks); + chainman.RecalculateBestHeader(); } BlockValidationState state; @@ -3290,7 +3298,7 @@ UniValue WriteUTXOSnapshot( tip->nHeight, tip->GetBlockHash().ToString(), fs::PathToString(path), fs::PathToString(temppath))); - SnapshotMetadata metadata{tip->GetBlockHash(), maybe_stats->coins_count, tip->nChainTx}; + SnapshotMetadata metadata{chainstate.m_chainman.GetParams().MessageStart(), tip->GetBlockHash(), maybe_stats->coins_count}; afile << metadata; @@ -3399,13 +3407,20 @@ static RPCHelpMan loadtxoutset() "Couldn't open file " + fs::PathToString(path) + " for reading."); } - SnapshotMetadata metadata; - afile >> metadata; + SnapshotMetadata metadata{chainman.GetParams().MessageStart()}; + try { + afile >> metadata; + } catch (const std::ios_base::failure& e) { + throw JSONRPCError(RPC_DESERIALIZATION_ERROR, strprintf("Unable to parse metadata: %s", e.what())); + } uint256 base_blockhash = metadata.m_base_blockhash; if (!chainman.GetParams().AssumeutxoForBlockhash(base_blockhash).has_value()) { + const auto available_heights{chainman.GetParams().GetAvailableSnapshotHeights()}; + const std::string heights_formatted{Join(available_heights, ", ", [&](const auto& height) { return ToString(height); })}; throw JSONRPCError(RPC_INTERNAL_ERROR, strprintf("Unable to load UTXO snapshot, " - "assumeutxo block hash in snapshot metadata not recognized (%s)", base_blockhash.ToString())); + "assumeutxo block hash in snapshot metadata not recognized (hash: %s). The following snapshot heights are available: %s.", + base_blockhash.ToString(), heights_formatted)); } int max_secs_to_wait_for_headers = 60 * 10; CBlockIndex* snapshot_start_block = nullptr; @@ -3440,12 +3455,19 @@ static RPCHelpMan loadtxoutset() if (!activation_result) { throw JSONRPCError(RPC_INTERNAL_ERROR, "Unable to load UTXO snapshot: " + util::ErrorString(activation_result).original); } - CBlockIndex* new_tip{WITH_LOCK(::cs_main, return chainman.ActiveTip())}; + + // Because we can't provide historical blocks during tip or background sync. + // Update local services to reflect we are a limited peer until we are fully sync. + node.connman->RemoveLocalServices(NODE_NETWORK); + // Setting the limited state is usually redundant because the node can always + // provide the last 288 blocks, but it doesn't hurt to set it. + node.connman->AddLocalServices(NODE_NETWORK_LIMITED); + CBlockIndex& snapshot_index{*CHECK_NONFATAL(*activation_result)}; UniValue result(UniValue::VOBJ); result.pushKV("coins_loaded", metadata.m_coins_count); - result.pushKV("tip_hash", new_tip->GetBlockHash().ToString()); - result.pushKV("base_height", new_tip->nHeight); + result.pushKV("tip_hash", snapshot_index.GetBlockHash().ToString()); + result.pushKV("base_height", snapshot_index.nHeight); result.pushKV("path", fs::PathToString(path)); return result; }, @@ -3486,7 +3508,7 @@ return RPCHelpMan{ ChainstateManager& chainman = EnsureAnyChainman(request.context); - auto make_chain_data = [&](const Chainstate& cs, bool validated) EXCLUSIVE_LOCKS_REQUIRED(::cs_main) { + auto make_chain_data = [&](const Chainstate& cs) EXCLUSIVE_LOCKS_REQUIRED(::cs_main) { AssertLockHeld(::cs_main); UniValue data(UniValue::VOBJ); if (!cs.m_chain.Tip()) { @@ -3504,17 +3526,16 @@ return RPCHelpMan{ if (cs.m_from_snapshot_blockhash) { data.pushKV("snapshot_blockhash", cs.m_from_snapshot_blockhash->ToString()); } - data.pushKV("validated", validated); + data.pushKV("validated", cs.m_assumeutxo == Assumeutxo::VALIDATED); return data; }; obj.pushKV("headers", chainman.m_best_header ? chainman.m_best_header->nHeight : -1); - - const auto& chainstates = chainman.GetAll(); UniValue obj_chainstates{UniValue::VARR}; - for (Chainstate* cs : chainstates) { - obj_chainstates.push_back(make_chain_data(*cs, !cs->m_from_snapshot_blockhash || chainstates.size() == 1)); + if (const Chainstate * cs{chainman.HistoricalChainstate()}) { + obj_chainstates.push_back(make_chain_data(*cs)); } + obj_chainstates.push_back(make_chain_data(chainman.CurrentChainstate())); obj.pushKV("chainstates", std::move(obj_chainstates)); return obj; } diff --git a/src/rpc/blockchain.h b/src/rpc/blockchain.h index 904d69e8ed8e..c064cd9cb50d 100644 --- a/src/rpc/blockchain.h +++ b/src/rpc/blockchain.h @@ -21,6 +21,7 @@ extern RecursiveMutex cs_main; // NOLINT(readability-redundant-declaration) class CBlock; class CBlockIndex; +class CChain; class Chainstate; class CCoinsView; namespace chainlock { class Chainlocks; } @@ -70,6 +71,9 @@ UniValue CreateUTXOSnapshot( const fs::path& path, const fs::path& tmppath); +//! Return height of highest block that has been pruned, or std::nullopt if no blocks have been pruned. +std::optional GetPruneHeight(const node::BlockManager& blockman, const CChain& chain) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + /** * Calculate statistics about the unspent transaction output set * diff --git a/src/test/coinstatsindex_tests.cpp b/src/test/coinstatsindex_tests.cpp index 4dc492604deb..3c2748c596cc 100644 --- a/src/test/coinstatsindex_tests.cpp +++ b/src/test/coinstatsindex_tests.cpp @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -13,6 +14,8 @@ #include +using kernel::ChainstateRole; + BOOST_AUTO_TEST_SUITE(coinstatsindex_tests) BOOST_FIXTURE_TEST_CASE(coinstatsindex_initial_sync, TestChain100Setup) @@ -103,7 +106,7 @@ BOOST_FIXTURE_TEST_CASE(coinstatsindex_unclean_shutdown, TestChain100Setup) // Send block connected notification, then stop the index without // sending a chainstate flushed notification. Prior to #24138, this // would cause the index to be corrupted and fail to reload. - ValidationInterfaceTest::BlockConnected(ChainstateRole::NORMAL, index, new_block, new_block_index); + ValidationInterfaceTest::BlockConnected(ChainstateRole{}, index, new_block, new_block_index); index.Stop(); } diff --git a/src/test/fuzz/deserialize.cpp b/src/test/fuzz/deserialize.cpp index d34cbb25fea3..118a1679592e 100644 --- a/src/test/fuzz/deserialize.cpp +++ b/src/test/fuzz/deserialize.cpp @@ -309,7 +309,7 @@ FUZZ_TARGET_DESERIALIZE(blocktransactionsrequest_deserialize, { DeserializeFromFuzzingInput(buffer, btr); }) FUZZ_TARGET_DESERIALIZE(snapshotmetadata_deserialize, { - SnapshotMetadata snapshot_metadata; + SnapshotMetadata snapshot_metadata{Params().MessageStart()}; DeserializeFromFuzzingInput(buffer, snapshot_metadata); }) FUZZ_TARGET_DESERIALIZE(uint160_deserialize, { diff --git a/src/test/fuzz/utxo_snapshot.cpp b/src/test/fuzz/utxo_snapshot.cpp index 92d86ccaaa4c..0723e9f35483 100644 --- a/src/test/fuzz/utxo_snapshot.cpp +++ b/src/test/fuzz/utxo_snapshot.cpp @@ -36,7 +36,7 @@ FUZZ_TARGET(utxo_snapshot, .init = initialize_chain) const auto snapshot_path = gArgs.GetDataDirNet() / "fuzzed_snapshot.dat"; - Assert(!chainman.SnapshotBlockhash()); + Assert(!chainman.ActiveChainstate().m_from_snapshot_blockhash); { AutoFile outfile{fsbridge::fopen(snapshot_path, "wb")}; @@ -46,7 +46,7 @@ FUZZ_TARGET(utxo_snapshot, .init = initialize_chain) const auto ActivateFuzzedSnapshot{[&] { AutoFile infile{fsbridge::fopen(snapshot_path, "rb")}; - SnapshotMetadata metadata; + SnapshotMetadata metadata{chainman.GetParams().MessageStart()}; try { infile >> metadata; } catch (const std::ios_base::failure&) { @@ -68,8 +68,6 @@ FUZZ_TARGET(utxo_snapshot, .init = initialize_chain) if (ActivateFuzzedSnapshot()) { LOCK(::cs_main); Assert(!chainman.ActiveChainstate().m_from_snapshot_blockhash->IsNull()); - Assert(*chainman.ActiveChainstate().m_from_snapshot_blockhash == - *chainman.SnapshotBlockhash()); const auto& coinscache{chainman.ActiveChainstate().CoinsTip()}; int64_t chain_tx{}; for (const auto& block : *g_chain) { @@ -82,7 +80,6 @@ FUZZ_TARGET(utxo_snapshot, .init = initialize_chain) Assert(g_chain->size() == coinscache.GetCacheSize()); Assert(chain_tx == chainman.ActiveTip()->nChainTx); } else { - Assert(!chainman.SnapshotBlockhash()); Assert(!chainman.ActiveChainstate().m_from_snapshot_blockhash); } // Snapshot should refuse to load a second time regardless of validity diff --git a/src/test/util/chainstate.h b/src/test/util/chainstate.h index 52869256d70d..8a4cf591916a 100644 --- a/src/test/util/chainstate.h +++ b/src/test/util/chainstate.h @@ -56,7 +56,7 @@ CreateAndActivateUTXOSnapshot( // FILE* infile{fsbridge::fopen(snapshot_path, "rb")}; AutoFile auto_infile{infile}; - node::SnapshotMetadata metadata; + node::SnapshotMetadata metadata{node.chainman->GetParams().MessageStart()}; auto_infile >> metadata; malleation(auto_infile, metadata); @@ -79,7 +79,7 @@ CreateAndActivateUTXOSnapshot( Chainstate& chain = node.chainman->ActiveChainstate(); Assert(chain.LoadGenesisBlock()); // These cache values will be corrected shortly in `MaybeRebalanceCaches`. - chain.InitCoinsDB(1 << 20, true, false, ""); + chain.InitCoinsDB(1 << 20, /*in_memory=*/true, /*should_wipe=*/false); chain.InitCoinsCache(1 << 20); chain.CoinsTip().SetBestBlock(gen_hash); chain.setBlockIndexCandidates.insert(node.chainman->m_blockman.LookupBlockIndex(gen_hash)); diff --git a/src/test/util/validation.cpp b/src/test/util/validation.cpp index fa281f04bd67..22ae74a9a754 100644 --- a/src/test/util/validation.cpp +++ b/src/test/util/validation.cpp @@ -9,6 +9,8 @@ #include #include +using kernel::ChainstateRole; + void TestChainState::ResetIbd() { m_cached_finished_ibd = false; @@ -23,10 +25,10 @@ void TestChainState::JumpOutOfIbd() } void ValidationInterfaceTest::BlockConnected( - ChainstateRole role, - CValidationInterface& obj, - const std::shared_ptr& block, - const CBlockIndex* pindex) + const ChainstateRole& role, + CValidationInterface& obj, + const std::shared_ptr& block, + const CBlockIndex* pindex) { obj.BlockConnected(role, block, pindex); } diff --git a/src/test/util/validation.h b/src/test/util/validation.h index 87a5b72fea7b..acfb76abf583 100644 --- a/src/test/util/validation.h +++ b/src/test/util/validation.h @@ -20,7 +20,7 @@ class ValidationInterfaceTest { public: static void BlockConnected( - ChainstateRole role, + const kernel::ChainstateRole& role, CValidationInterface& obj, const std::shared_ptr& block, const CBlockIndex* pindex); diff --git a/src/test/validation_block_tests.cpp b/src/test/validation_block_tests.cpp index 74bc0faf34aa..dbb64f7a2eac 100644 --- a/src/test/validation_block_tests.cpp +++ b/src/test/validation_block_tests.cpp @@ -21,6 +21,7 @@ #include +using kernel::ChainstateRole; using node::BlockAssembler; namespace validation_block_tests { @@ -45,7 +46,7 @@ struct TestSubscriber final : public CValidationInterface { BOOST_CHECK_EQUAL(m_expected_tip, pindexNew->GetBlockHash()); } - void BlockConnected(ChainstateRole role, const std::shared_ptr& block, const CBlockIndex* pindex) override + void BlockConnected(const ChainstateRole& role, const std::shared_ptr& block, const CBlockIndex* pindex) override { BOOST_CHECK_EQUAL(m_expected_tip, block->hashPrevBlock); BOOST_CHECK_EQUAL(m_expected_tip, pindex->pprev->GetBlockHash()); diff --git a/src/test/validation_chainstate_tests.cpp b/src/test/validation_chainstate_tests.cpp index fa512eb4d361..b10035c5ed20 100644 --- a/src/test/validation_chainstate_tests.cpp +++ b/src/test/validation_chainstate_tests.cpp @@ -36,17 +36,17 @@ class TipEventCounter final : public CValidationInterface int background_block_connected{0}; int background_chainstate_flushed{0}; - void BlockConnected(ChainstateRole role, const std::shared_ptr&, const CBlockIndex*) override + void BlockConnected(const kernel::ChainstateRole& role, const std::shared_ptr&, const CBlockIndex*) override { ++block_connected; - if (role == ChainstateRole::BACKGROUND) ++background_block_connected; + if (role.historical) ++background_block_connected; } void UpdatedBlockTip(const CBlockIndex*, const CBlockIndex*, bool) override { ++updated_tip; } void NotifyMasternodeListChanged(bool, const CDeterministicMNList&, const CDeterministicMNListDiff&) override { ++mn_list_changed; } - void ChainStateFlushed(ChainstateRole role, const CBlockLocator&) override + void ChainStateFlushed(const kernel::ChainstateRole& role, const CBlockLocator&) override { ++chainstate_flushed; - if (role == ChainstateRole::BACKGROUND) ++background_chainstate_flushed; + if (role.historical) ++background_chainstate_flushed; } }; @@ -122,7 +122,7 @@ BOOST_FIXTURE_TEST_CASE(chainstate_update_tip, TestChain100Setup) this, NoMalleation, /*reset_chainstate=*/ true)); // Ensure our active chain is the snapshot chainstate. - BOOST_CHECK(WITH_LOCK(::cs_main, return chainman.IsSnapshotActive())); + BOOST_CHECK(WITH_LOCK(::cs_main, return chainman.CurrentChainstate().m_from_snapshot_blockhash)); curr_tip = ::g_best_block; @@ -134,16 +134,7 @@ BOOST_FIXTURE_TEST_CASE(chainstate_update_tip, TestChain100Setup) curr_tip = ::g_best_block; - BOOST_CHECK_EQUAL(chainman.GetAll().size(), 2); - - Chainstate& background_cs{*[&] { - for (Chainstate* cs : chainman.GetAll()) { - if (cs != &chainman.ActiveChainstate()) { - return cs; - } - } - assert(false); - }()}; + Chainstate& background_cs{*Assert(WITH_LOCK(::cs_main, return chainman.HistoricalChainstate()))}; // Append the first block to the background chain. BlockValidationState state; diff --git a/src/test/validation_chainstatemanager_tests.cpp b/src/test/validation_chainstatemanager_tests.cpp index eb02424776bc..7e8bcb09b196 100644 --- a/src/test/validation_chainstatemanager_tests.cpp +++ b/src/test/validation_chainstatemanager_tests.cpp @@ -70,17 +70,16 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager, TestChain100Setup) CEvoDB& evodb = *m_node.evodb; std::vector chainstates; - BOOST_CHECK(!manager.SnapshotBlockhash().has_value()); + BOOST_CHECK(WITH_LOCK(::cs_main, return !manager.CurrentChainstate().m_from_snapshot_blockhash)); // Create a legacy (IBD) chainstate. // Chainstate& c1 = manager.ActiveChainstate(); chainstates.push_back(&c1); - BOOST_CHECK(!manager.IsSnapshotActive()); + BOOST_CHECK(WITH_LOCK(::cs_main, return !manager.CurrentChainstate().m_from_snapshot_blockhash)); BOOST_CHECK(!manager.IsSnapshotActiveAndUnvalidated()); BOOST_CHECK(llmq::CSigSharesManager::IsQuorumSigningAllowed(manager)); - BOOST_CHECK(WITH_LOCK(::cs_main, return !manager.IsSnapshotValidated())); auto all = manager.GetAll(); BOOST_CHECK_EQUAL_COLLECTIONS(all.begin(), all.end(), chainstates.begin(), chainstates.end()); @@ -94,16 +93,17 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager, TestChain100Setup) auto exp_tip = c1.m_chain.Tip(); BOOST_CHECK_EQUAL(active_tip, exp_tip); - BOOST_CHECK(!manager.SnapshotBlockhash().has_value()); + BOOST_CHECK(WITH_LOCK(::cs_main, return !manager.CurrentChainstate().m_from_snapshot_blockhash)); // Create a snapshot-based chainstate. // const uint256 snapshot_blockhash = active_tip->GetBlockHash(); SeedSnapshotMarker(evodb, snapshot_blockhash); - Chainstate& c2 = WITH_LOCK(::cs_main, return manager.ActivateExistingSnapshot(snapshot_blockhash)); + Chainstate& c2{WITH_LOCK(::cs_main, return manager.AddChainstate(std::make_unique( + nullptr, manager.m_blockman, manager, evodb, m_node.chain_helper, snapshot_blockhash)))}; chainstates.push_back(&c2); - BOOST_CHECK_EQUAL(manager.SnapshotBlockhash().value(), snapshot_blockhash); + BOOST_CHECK_EQUAL(WITH_LOCK(::cs_main, return *manager.CurrentChainstate().m_from_snapshot_blockhash), snapshot_blockhash); c2.InitCoinsDB( /*cache_size_bytes=*/1 << 23, /*in_memory=*/true, /*should_wipe=*/false); { @@ -116,9 +116,8 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager, TestChain100Setup) BlockValidationState _; BOOST_CHECK(c2.ActivateBestChain(_, nullptr)); - BOOST_CHECK_EQUAL(manager.SnapshotBlockhash().value(), snapshot_blockhash); - BOOST_CHECK(manager.IsSnapshotActive()); - BOOST_CHECK(WITH_LOCK(::cs_main, return !manager.IsSnapshotValidated())); + BOOST_CHECK_EQUAL(WITH_LOCK(::cs_main, return *manager.CurrentChainstate().m_from_snapshot_blockhash), snapshot_blockhash); + BOOST_CHECK(WITH_LOCK(::cs_main, return manager.CurrentChainstate().m_assumeutxo == Assumeutxo::UNVALIDATED)); BOOST_CHECK(manager.IsSnapshotActiveAndUnvalidated()); BOOST_CHECK(!llmq::CSigSharesManager::IsQuorumSigningAllowed(manager)); BOOST_CHECK_EQUAL(&c2, &manager.ActiveChainstate()); @@ -153,7 +152,8 @@ BOOST_AUTO_TEST_CASE(snapshot_startup_missing_base_header_is_nonfatal) const uint256 missing_base{GetRandHash()}; SeedSnapshotMarker(*m_node.evodb, missing_base); - Chainstate* snapshot = WITH_LOCK(::cs_main, return &manager.ActivateExistingSnapshot(missing_base)); + Chainstate* snapshot = WITH_LOCK(::cs_main, return &manager.AddChainstate(std::make_unique( + nullptr, manager.m_blockman, manager, *m_node.evodb, m_node.chain_helper, missing_base))); // Startup detection is allowed to precede receipt/loading of the base // header. Accessors and background candidate setup must fail softly. @@ -213,7 +213,8 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_rebalance_caches, TestChain100Setup) // CBlockIndex* snapshot_base{WITH_LOCK(manager.GetMutex(), return manager.ActiveChain()[manager.ActiveChain().Height() / 2])}; SeedSnapshotMarker(evodb, snapshot_base->GetBlockHash()); - Chainstate& c2 = WITH_LOCK(cs_main, return manager.ActivateExistingSnapshot(*snapshot_base->phashBlock)); + Chainstate& c2{WITH_LOCK(::cs_main, return manager.AddChainstate(std::make_unique( + nullptr, manager.m_blockman, manager, evodb, m_node.chain_helper, *snapshot_base->phashBlock)))}; chainstates.push_back(&c2); c2.InitCoinsDB( /*cache_size_bytes=*/1 << 23, /*in_memory=*/true, /*should_wipe=*/false); @@ -253,12 +254,10 @@ struct SnapshotTestSetup : TestChain100Setup { { ChainstateManager& chainman = *Assert(m_node.chainman); - BOOST_CHECK(!chainman.IsSnapshotActive()); - { LOCK(::cs_main); - BOOST_CHECK(!chainman.IsSnapshotValidated()); - BOOST_CHECK(!node::FindSnapshotChainstateDir()); + BOOST_CHECK(!chainman.CurrentChainstate().m_from_snapshot_blockhash); + BOOST_CHECK(!node::FindAssumeutxoChainstateDir(gArgs.GetDataDirNet())); } size_t initial_size; @@ -286,7 +285,6 @@ struct SnapshotTestSetup : TestChain100Setup { // Snapshot should refuse to load at this height. BOOST_REQUIRE(!CreateAndActivateUTXOSnapshot(this)); BOOST_CHECK(!chainman.ActiveChainstate().m_from_snapshot_blockhash); - BOOST_CHECK(!chainman.SnapshotBlockhash()); // Mine 10 more blocks, putting at us height 110 where a valid assumeutxo value can // be found. @@ -308,7 +306,7 @@ struct SnapshotTestSetup : TestChain100Setup { auto_infile >> coin; })); - BOOST_CHECK(!node::FindSnapshotChainstateDir()); + BOOST_CHECK(!node::FindAssumeutxoChainstateDir(gArgs.GetDataDirNet())); BOOST_REQUIRE(!CreateAndActivateUTXOSnapshot( this, [](AutoFile& auto_infile, SnapshotMetadata& metadata) { @@ -332,18 +330,15 @@ struct SnapshotTestSetup : TestChain100Setup { })); BOOST_REQUIRE(CreateAndActivateUTXOSnapshot(this)); - BOOST_CHECK(fs::exists(*node::FindSnapshotChainstateDir())); + BOOST_CHECK(fs::exists(*node::FindAssumeutxoChainstateDir(gArgs.GetDataDirNet()))); // Ensure our active chain is the snapshot chainstate. BOOST_CHECK(!chainman.ActiveChainstate().m_from_snapshot_blockhash->IsNull()); - BOOST_CHECK_EQUAL( - *chainman.ActiveChainstate().m_from_snapshot_blockhash, - *chainman.SnapshotBlockhash()); Chainstate& snapshot_chainstate = chainman.ActiveChainstate(); // To be checked against later when we try loading a subsequent snapshot. - uint256 loaded_snapshot_blockhash{*chainman.SnapshotBlockhash()}; + uint256 loaded_snapshot_blockhash{*Assert(WITH_LOCK(::cs_main, return chainman.CurrentChainstate().m_from_snapshot_blockhash))}; BOOST_CHECK(m_node.evodb->VerifyBestBlock( EvoDbIdentity::SNAPSHOT, loaded_snapshot_blockhash)); @@ -354,12 +349,12 @@ struct SnapshotTestSetup : TestChain100Setup { { LOCK(::cs_main); - fs::path found = *node::FindSnapshotChainstateDir(); + fs::path found = *node::FindAssumeutxoChainstateDir(gArgs.GetDataDirNet()); // Note: WriteSnapshotBaseBlockhash() is implicitly tested above. BOOST_CHECK_EQUAL( *node::ReadSnapshotBaseBlockhash(found), - *chainman.SnapshotBlockhash()); + *Assert(chainman.CurrentChainstate().m_from_snapshot_blockhash)); } const auto& au_data = ::Params().AssumeutxoForHeight(snapshot_height); @@ -482,7 +477,7 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_init_height_200, SnapshotTest const auto assumeutxo = Params().AssumeutxoForHeight(200); BOOST_REQUIRE(assumeutxo); - BOOST_REQUIRE_EQUAL(*chainman.SnapshotBlockhash(), assumeutxo->blockhash); + BOOST_REQUIRE_EQUAL(*Assert(WITH_LOCK(::cs_main, return chainman.CurrentChainstate().m_from_snapshot_blockhash)), assumeutxo->blockhash); ChainstateManager& restarted = this->SimulateNodeRestart(); this->LoadVerifyActivateChainstate(); @@ -621,11 +616,11 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_loadblockindex, TestChain100Setup) const uint256 snapshot_blockhash = assumed_base->GetBlockHash(); SeedSnapshotMarker(*m_node.evodb, snapshot_blockhash); - // Note: cs2's tip is not set when ActivateExistingSnapshot is called. - Chainstate& cs2 = WITH_LOCK(::cs_main, - return chainman.ActivateExistingSnapshot(*assumed_base->phashBlock)); + // Note: cs2's tip is not set when AddChainstate is called. + Chainstate& cs2{WITH_LOCK(::cs_main, return chainman.AddChainstate(std::make_unique( + nullptr, chainman.m_blockman, chainman, *m_node.evodb, m_node.chain_helper, *assumed_base->phashBlock)))}; - // Note: cs2's tip is not set when ActivateExistingSnapshot is called. + // Note: cs2's tip is not set when AddChainstate is called. // Set tip of the fully validated chain to be the validated tip cs1.m_chain.SetTip(*validated_tip); @@ -691,7 +686,8 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_loadblockindex, TestChain100Setup) BOOST_CHECK_EQUAL(cs2.setBlockIndexCandidates.size(), num_indexes - last_assumed_valid_idx + 1); } -//! Ensure that snapshot chainstates initialize properly when found on disk. +//! Ensure that snapshot chainstate can be loaded when found on disk after a +//! restart, and that new blocks can be connected to both chainstates. BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_init, SnapshotTestSetup) { ChainstateManager& chainman = *Assert(m_node.chainman); @@ -699,11 +695,11 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_init, SnapshotTestSetup) this->SetupSnapshot(); - fs::path snapshot_chainstate_dir = *node::FindSnapshotChainstateDir(); + fs::path snapshot_chainstate_dir = *node::FindAssumeutxoChainstateDir(gArgs.GetDataDirNet()); BOOST_CHECK(fs::exists(snapshot_chainstate_dir)); BOOST_CHECK_EQUAL(snapshot_chainstate_dir, gArgs.GetDataDirNet() / "chainstate_snapshot"); - BOOST_CHECK(chainman.IsSnapshotActive()); + BOOST_CHECK(WITH_LOCK(::cs_main, return chainman.CurrentChainstate().m_from_snapshot_blockhash)); const uint256 snapshot_tip_hash = WITH_LOCK(chainman.GetMutex(), return chainman.ActiveTip()->GetBlockHash()); @@ -725,8 +721,7 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_init, SnapshotTestSetup) BOOST_CHECK_EQUAL(bg_chainstate.m_chain.Height(), 109); // Test that simulating a shutdown (resetting ChainstateManager) and then performing - // chainstate reinitializing successfully cleans up the background-validation - // chainstate data, and we end up with a single chainstate that is at tip. + // chainstate reinitializing successfully reloads both chainstates. ChainstateManager& chainman_restarted = this->SimulateNodeRestart(); BOOST_TEST_MESSAGE("Performing Load/Verify/Activate of chainstate"); @@ -734,11 +729,20 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_init, SnapshotTestSetup) // This call reinitializes the chainstates. this->LoadVerifyActivateChainstate(); + std::vector chainstates; { LOCK(chainman_restarted.GetMutex()); - BOOST_CHECK_EQUAL(chainman_restarted.GetAll().size(), 2); - BOOST_CHECK(chainman_restarted.IsSnapshotActive()); - BOOST_CHECK(!chainman_restarted.IsSnapshotValidated()); + chainstates = chainman_restarted.GetAll(); + BOOST_CHECK_EQUAL(chainstates.size(), 2); + // Background chainstate has height of 109 not 110 here due to a quirk + // of the LoadVerifyActivate only calling ActivateBestChain on one + // chainstate. The height would be 110 after a real restart, but it's + // fine for this test which is focused on the snapshot chainstate. + BOOST_CHECK_EQUAL(chainstates[0]->m_chain.Height(), 109); + BOOST_CHECK_EQUAL(chainstates[1]->m_chain.Height(), 210); + + BOOST_CHECK(chainman_restarted.CurrentChainstate().m_from_snapshot_blockhash); + BOOST_CHECK(chainman_restarted.CurrentChainstate().m_assumeutxo == Assumeutxo::UNVALIDATED); BOOST_CHECK_EQUAL(chainman_restarted.ActiveTip()->GetBlockHash(), snapshot_tip_hash); BOOST_CHECK_EQUAL(chainman_restarted.ActiveHeight(), 210); @@ -752,12 +756,10 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_init, SnapshotTestSetup) BOOST_CHECK_EQUAL(chainman_restarted.ActiveHeight(), 220); // Background chainstate should be unaware of new blocks on the snapshot - // chainstate. - for (Chainstate* cs : chainman_restarted.GetAll()) { - if (cs != &chainman_restarted.ActiveChainstate()) { - BOOST_CHECK_EQUAL(cs->m_chain.Height(), 109); - } - } + // chainstate, but the block disconnected above is now reattached. + BOOST_CHECK_EQUAL(chainstates[0]->m_chain.Height(), 110); + BOOST_CHECK_EQUAL(chainstates[1]->m_chain.Height(), 220); + BOOST_CHECK_EQUAL(chainman_restarted.GetAll().size(), 1); } } @@ -944,7 +946,7 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_init_missing_evodb_marker, Sn m_node.mempool.get(), *m_node.evodb, m_node.chain_helper)); bilingual_str error; - BOOST_CHECK(!WITH_LOCK(::cs_main, return restarted.DetectSnapshotChainstate(m_node.mempool.get(), error))); + BOOST_CHECK(!WITH_LOCK(::cs_main, return restarted.LoadAssumeutxoChainstate(m_node.mempool.get(), error))); BOOST_CHECK(error.original.find("Snapshot chainstate EvoDB marker") != std::string::npos); WITH_LOCK(::cs_main, restarted.ResetChainstates()); @@ -959,26 +961,27 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_completion, SnapshotTestSetup ChainstateManager& chainman = *Assert(m_node.chainman); Chainstate& active_cs = chainman.ActiveChainstate(); + Chainstate& validated_cs{*Assert(WITH_LOCK(cs_main, return chainman.HistoricalChainstate()))}; auto tip_cache_before_complete = active_cs.m_coinstip_cache_size_bytes; auto db_cache_before_complete = active_cs.m_coinsdb_cache_size_bytes; SnapshotCompletionResult res; auto mock_shutdown = [](bilingual_str msg) {}; - fs::path snapshot_chainstate_dir = *node::FindSnapshotChainstateDir(); + fs::path snapshot_chainstate_dir = *node::FindAssumeutxoChainstateDir(gArgs.GetDataDirNet()); BOOST_CHECK(fs::exists(snapshot_chainstate_dir)); BOOST_CHECK_EQUAL(snapshot_chainstate_dir, gArgs.GetDataDirNet() / "chainstate_snapshot"); - BOOST_CHECK(chainman.IsSnapshotActive()); + BOOST_CHECK(WITH_LOCK(::cs_main, return chainman.CurrentChainstate().m_from_snapshot_blockhash)); const uint256 snapshot_tip_hash = WITH_LOCK(chainman.GetMutex(), return chainman.ActiveTip()->GetBlockHash()); res = WITH_LOCK(::cs_main, - return chainman.MaybeCompleteSnapshotValidation(mock_shutdown)); + return chainman.MaybeValidateSnapshot(validated_cs, active_cs, mock_shutdown)); BOOST_CHECK_EQUAL(res, SnapshotCompletionResult::SUCCESS); - WITH_LOCK(::cs_main, BOOST_CHECK(chainman.IsSnapshotValidated())); - BOOST_CHECK(chainman.IsSnapshotActive()); + BOOST_CHECK(WITH_LOCK(::cs_main, return chainman.CurrentChainstate().m_assumeutxo == Assumeutxo::VALIDATED)); + BOOST_CHECK(WITH_LOCK(::cs_main, return chainman.CurrentChainstate().m_from_snapshot_blockhash)); // Cache should have been rebalanced and reallocated to the "only" remaining // chainstate. @@ -991,7 +994,7 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_completion, SnapshotTestSetup // Trying completion again should return false. res = WITH_LOCK(::cs_main, - return chainman.MaybeCompleteSnapshotValidation(mock_shutdown)); + return chainman.MaybeValidateSnapshot(validated_cs, active_cs, mock_shutdown)); BOOST_CHECK_EQUAL(res, SnapshotCompletionResult::SKIPPED); // The invalid snapshot path should not have been used. @@ -1025,8 +1028,7 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_completion, SnapshotTestSetup { LOCK(chainman_restarted.GetMutex()); BOOST_CHECK_EQUAL(chainman_restarted.GetAll().size(), 1); - BOOST_CHECK(!chainman_restarted.IsSnapshotActive()); - BOOST_CHECK(!chainman_restarted.IsSnapshotValidated()); + BOOST_CHECK(!chainman_restarted.CurrentChainstate().m_from_snapshot_blockhash); BOOST_CHECK(active_cs2.m_coinstip_cache_size_bytes > tip_cache_before_complete); BOOST_CHECK(active_cs2.m_coinsdb_cache_size_bytes > db_cache_before_complete); @@ -1079,7 +1081,7 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_completion_incorrect_base_mn_ BOOST_REQUIRE(m_node.evodb->CommitRootTransaction(EvoDbIdentity::SNAPSHOT, /*sync=*/true)); const auto result = WITH_LOCK(::cs_main, - return chainman.MaybeCompleteSnapshotValidation([](bilingual_str) {})); + return chainman.MaybeValidateSnapshot(*validation_chainstate, *snapshot_chainstate, [](bilingual_str) {})); mutable_consensus.DIP0003Height = old_dip3_height; BOOST_CHECK_EQUAL(result, SnapshotCompletionResult::EVO_STATE_MISMATCH); BOOST_CHECK_EQUAL(&chainman.ActiveChainstate(), validation_chainstate); @@ -1110,11 +1112,11 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_records_only_required_background_work_ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_cleanup_recovers_first_rename, SnapshotTestSetup) { - this->SetupSnapshot(); + auto [validated_cs, unvalidated_cs] = this->SetupSnapshot(); ChainstateManager& chainman = *Assert(m_node.chainman); const uint256 snapshot_tip = WITH_LOCK(::cs_main, return chainman.ActiveChainstate().CoinsTip().GetBestBlock()); BOOST_REQUIRE_EQUAL(WITH_LOCK(::cs_main, - return chainman.MaybeCompleteSnapshotValidation([](bilingual_str) {})), + return chainman.MaybeValidateSnapshot(*validated_cs, *unvalidated_cs, [](bilingual_str) {})), SnapshotCompletionResult::SUCCESS); this->SimulateNodeRestart(); @@ -1131,11 +1133,11 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_cleanup_recovers_first_rename BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_cleanup_recovers_completed_swap, SnapshotTestSetup) { - this->SetupSnapshot(); + auto [validated_cs, unvalidated_cs] = this->SetupSnapshot(); ChainstateManager& chainman = *Assert(m_node.chainman); const uint256 snapshot_tip = WITH_LOCK(::cs_main, return chainman.ActiveChainstate().CoinsTip().GetBestBlock()); BOOST_REQUIRE_EQUAL(WITH_LOCK(::cs_main, - return chainman.MaybeCompleteSnapshotValidation([](bilingual_str) {})), + return chainman.MaybeValidateSnapshot(*validated_cs, *unvalidated_cs, [](bilingual_str) {})), SnapshotCompletionResult::SUCCESS); this->SimulateNodeRestart(); @@ -1174,11 +1176,11 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_cleanup_recovers_invalid_rena BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_cleanup_recovers_promoted_swap, SnapshotTestSetup) { - this->SetupSnapshot(); + auto [validated_cs, unvalidated_cs] = this->SetupSnapshot(); ChainstateManager& chainman = *Assert(m_node.chainman); const uint256 snapshot_tip = WITH_LOCK(::cs_main, return chainman.ActiveChainstate().CoinsTip().GetBestBlock()); BOOST_REQUIRE_EQUAL(WITH_LOCK(::cs_main, - return chainman.MaybeCompleteSnapshotValidation([](bilingual_str) {})), + return chainman.MaybeValidateSnapshot(*validated_cs, *unvalidated_cs, [](bilingual_str) {})), SnapshotCompletionResult::SUCCESS); this->SimulateNodeRestart(); @@ -1196,7 +1198,7 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_base_is_cached, SnapshotTestS { this->SetupSnapshot(); ChainstateManager& chainman = *Assert(m_node.chainman); - const uint256 base_hash{*chainman.SnapshotBlockhash()}; + const uint256 base_hash{*Assert(WITH_LOCK(::cs_main, return chainman.CurrentChainstate().m_from_snapshot_blockhash))}; { LOCK(::cs_main); @@ -1213,6 +1215,7 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_completion_hash_mismatch, Sna { auto chainstates = this->SetupSnapshot(); Chainstate& validation_chainstate = *std::get<0>(chainstates); + Chainstate& unvalidated_cs = *std::get<1>(chainstates); ChainstateManager& chainman = *Assert(m_node.chainman); SnapshotCompletionResult res; auto mock_shutdown = [](bilingual_str msg) {}; @@ -1234,7 +1237,7 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_completion_hash_mismatch, Sna { ASSERT_DEBUG_LOG("failed to validate the -assumeutxo snapshot state"); res = WITH_LOCK(::cs_main, - return chainman.MaybeCompleteSnapshotValidation(mock_shutdown)); + return chainman.MaybeValidateSnapshot(validation_chainstate, unvalidated_cs, mock_shutdown)); BOOST_CHECK_EQUAL(res, SnapshotCompletionResult::HASH_MISMATCH); } @@ -1263,8 +1266,7 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_snapshot_completion_hash_mismatch, Sna { LOCK(::cs_main); BOOST_CHECK_EQUAL(chainman_restarted.GetAll().size(), 1); - BOOST_CHECK(!chainman_restarted.IsSnapshotActive()); - BOOST_CHECK(!chainman_restarted.IsSnapshotValidated()); + BOOST_CHECK(!chainman_restarted.CurrentChainstate().m_from_snapshot_blockhash); BOOST_CHECK_EQUAL(chainman_restarted.ActiveHeight(), 210); } diff --git a/src/test/validationinterface_tests.cpp b/src/test/validationinterface_tests.cpp index eb97041d1362..11f8c5a53ee9 100644 --- a/src/test/validationinterface_tests.cpp +++ b/src/test/validationinterface_tests.cpp @@ -8,7 +8,6 @@ #include #include #include -#include #include BOOST_FIXTURE_TEST_SUITE(validationinterface_tests, ChainTestingSetup) diff --git a/src/txdb.h b/src/txdb.h index 1b3162796e46..f6d36a52156c 100644 --- a/src/txdb.h +++ b/src/txdb.h @@ -75,9 +75,6 @@ class CCoinsViewDB final : public CCoinsView //! Dynamically alter the underlying leveldb cache size. void ResizeCache(size_t new_cache_size) EXCLUSIVE_LOCKS_REQUIRED(cs_main); - - //! @returns filesystem path to on-disk storage or std::nullopt if in memory. - std::optional StoragePath() { return m_db->StoragePath(); } }; /** Access to the block database (blocks/index/) */ diff --git a/src/validation.cpp b/src/validation.cpp index baa357a7bc7b..976b08ed9c51 100644 --- a/src/validation.cpp +++ b/src/validation.cpp @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -76,6 +77,7 @@ #include #include +using kernel::ChainstateRole; using kernel::LoadMempool; using fsbridge::FopenFn; @@ -1619,6 +1621,7 @@ Chainstate::Chainstate(CTxMemPool* mempool, m_blockman(blockman), m_params(chainman.GetParams()), m_chainman(chainman), + m_assumeutxo(from_snapshot_blockhash ? Assumeutxo::UNVALIDATED : Assumeutxo::VALIDATED), m_from_snapshot_blockhash(from_snapshot_blockhash) {} ::EvoDbIdentity Chainstate::EvoDbIdentity() const @@ -1634,7 +1637,16 @@ std::string Chainstate::EvoDbInconsistencyMessage() return "Found EvoDB inconsistency, you must reindex to continue"; } -const CBlockIndex* Chainstate::SnapshotBase() +fs::path Chainstate::StoragePath() const +{ + fs::path path{gArgs.GetDataDirNet() / "chainstate"}; + if (m_from_snapshot_blockhash) { + path += node::SNAPSHOT_CHAINSTATE_SUFFIX; + } + return path; +} + +const CBlockIndex* Chainstate::SnapshotBase() const { if (!m_from_snapshot_blockhash) return nullptr; // Snapshot detection precedes LoadBlockIndex during startup. A stale or @@ -1644,18 +1656,36 @@ const CBlockIndex* Chainstate::SnapshotBase() return m_cached_snapshot_base; } +const CBlockIndex* Chainstate::TargetBlock() const +{ + if (!m_target_blockhash) return nullptr; + if (!m_cached_target_block) m_cached_target_block = m_chainman.m_blockman.LookupBlockIndex(*m_target_blockhash); + return m_cached_target_block; +} + +void Chainstate::SetTargetBlock(CBlockIndex* block) +{ + if (block) { + m_target_blockhash = block->GetBlockHash(); + } else { + m_target_blockhash.reset(); + } + m_cached_target_block = block; +} + +void Chainstate::SetTargetBlockHash(uint256 block_hash) +{ + m_target_blockhash = block_hash; + m_cached_target_block = nullptr; +} + void Chainstate::InitCoinsDB( size_t cache_size_bytes, bool in_memory, - bool should_wipe, - fs::path leveldb_name) + bool should_wipe) { - if (m_from_snapshot_blockhash) { - leveldb_name += node::SNAPSHOT_CHAINSTATE_SUFFIX; - } - m_coins_views = std::make_unique( - leveldb_name, cache_size_bytes, in_memory, should_wipe); + StoragePath(), cache_size_bytes, in_memory, should_wipe); } void Chainstate::InitCoinsCache(size_t cache_size_bytes) @@ -1719,7 +1749,7 @@ void Chainstate::CheckForkWarningConditions() // Before we get past initial download, we cannot reliably alert about forks // (we assume we don't get stuck on a fork before finishing our initial sync) - if (IsInitialBlockDownload()) { + if (GetRole().historical || IsInitialBlockDownload()) { return; } @@ -1742,7 +1772,7 @@ void Chainstate::InvalidChainFound(CBlockIndex* pindexNew) m_chainman.m_best_invalid = pindexNew; } if (m_chainman.m_best_header != nullptr && m_chainman.m_best_header->GetAncestor(pindexNew->nHeight) == pindexNew) { - m_chainman.m_best_header = m_chain.Tip(); + m_chainman.RecalculateBestHeader(); } LogPrintf("%s: invalid block=%s height=%d log2_work=%f date=%s\n", __func__, @@ -2629,7 +2659,7 @@ bool Chainstate::FlushStateToDisk( m_blockman.FindFilesToPruneManual( setFilesToPrune, std::min(last_prune, nManualPruneHeight), - *this, m_chainman); + *this); } else { LOG_TIME_MILLIS_WITH_CATEGORY("find files to prune", BCLog::BENCHMARK); @@ -3110,13 +3140,11 @@ bool Chainstate::ConnectTip(BlockValidationState& state, CBlockIndex* pindexNew, ::g_stats_client->gauge("blocks.tip.NumTransactions", blockConnecting.vtx.size(), 1.0f); ::g_stats_client->gauge("blocks.tip.SigOps", nSigOps, 1.0f); - // If we are the background validation chainstate, check to see if we are done - // validating the snapshot (i.e. our tip has reached the snapshot's base block). - if (this != &m_chainman.ActiveChainstate()) { - // This call may set `m_disabled`, which is referenced immediately afterwards in - // ActivateBestChain, so that we stop connecting blocks past the snapshot base. - m_chainman.MaybeCompleteSnapshotValidation(); - } + // If this historical chainstate reached its target, validate the current + // assumeutxo chainstate. Completion stores m_target_utxohash so this + // chainstate is no longer selected for historical downloads. + Chainstate& current_cs{m_chainman.CurrentChainstate()}; + m_chainman.MaybeValidateSnapshot(*this, current_cs); connectTrace.BlockConnected(pindexNew, std::move(pthisBlock)); return true; @@ -3349,11 +3377,11 @@ bool Chainstate::ActivateBestChain(BlockValidationState& state, std::shared_ptr< auto start = Now(); - // Belt-and-suspenders check that we aren't attempting to advance the background - // chainstate past the snapshot base block. - if (WITH_LOCK(::cs_main, return m_disabled)) { - LogPrintf("m_disabled is set - this chainstate should not be in operation. " /* Continued */ - "Please report this as a bug. %s\n", PACKAGE_BUGREPORT); + // Belt-and-suspenders check that we aren't attempting to advance the + // chainstate past the target block. + if (WITH_LOCK(::cs_main, return m_target_utxohash)) { + LogPrintf("m_target_utxohash is set - this chainstate should not be in operation. Please report this as a bug. %s\n", + PACKAGE_BUGREPORT); return false; } @@ -3410,12 +3438,8 @@ bool Chainstate::ActivateBestChain(BlockValidationState& state, std::shared_ptr< GetMainSignals().BlockConnected(this->GetRole(), trace.pblock, trace.pindex); } - // This will have been toggled in - // ActivateBestChainStep -> ConnectTip -> MaybeCompleteSnapshotValidation, - // if at all, so we should catch it here. - // - // Break this do-while to ensure we don't advance past the base snapshot. - if (m_disabled) { + // Break this do-while to ensure we don't advance past the target block. + if (ReachedTarget()) { break; } } while (!m_chain.Tip() || (starting_tip && CBlockIndexWorkComparator()(m_chain.Tip(), starting_tip))); @@ -3443,24 +3467,33 @@ bool Chainstate::ActivateBestChain(BlockValidationState& state, std::shared_ptr< // When we reach this point, we switched to a new tip (stored in pindexNewTip). if (nStopAtHeight && pindexNewTip && pindexNewTip->nHeight >= nStopAtHeight) StartShutdown(); - if (exited_ibd) { - // If a background chainstate is in use, we may need to rebalance our - // allocation of caches once a chainstate exits initial block download. - LOCK(::cs_main); - m_chainman.MaybeRebalanceCaches(); - } + bool reached_target; + { + LOCK(m_chainman.GetMutex()); + if (exited_ibd) { + // If a background chainstate is in use, we may need to rebalance our + // allocation of caches once a chainstate exits initial block download. + m_chainman.MaybeRebalanceCaches(); + } - if (WITH_LOCK(::cs_main, return m_disabled)) { - // Background chainstate has reached the snapshot base block, so exit. + // Write changes periodically to disk, after relay. + if (!FlushStateToDisk(state, FlushStateMode::PERIODIC)) { + return false; + } + + reached_target = ReachedTarget(); + } - // Restart indexes to resume indexing for all blocks unique to the snapshot - // chain. This resumes indexing "in order" from where the indexing on the - // background validation chain left off. + if (reached_target) { + // Chainstate has reached the target block, so exit. + // + // Restart indexes so indexes can resync and index new blocks after + // the target block. // // This cannot be done while holding cs_main (within - // MaybeCompleteSnapshotValidation) or a cs_main deadlock will occur. - if (m_chainman.restart_indexes) { - m_chainman.restart_indexes(); + // MaybeValidateSnapshot) or a cs_main deadlock will occur. + if (m_chainman.snapshot_download_completed) { + m_chainman.snapshot_download_completed(); } break; } @@ -3859,24 +3892,32 @@ void Chainstate::TryAddBlockIndexCandidate(CBlockIndex* pindex) if (pindex->nStatus & BLOCK_CONFLICT_CHAINLOCK) { return; } + + // Do not continue building a chainstate that is based on an invalid + // snapshot. This is a belt-and-suspenders type of check because if an + // invalid snapshot is loaded, the node will shut down to force a manual + // intervention. But it is good to handle this case correctly regardless. + if (m_assumeutxo == Assumeutxo::INVALID) { + return; + } // The block only is a candidate for the most-work-chain if it has the same // or more work than our current tip. if (m_chain.Tip() != nullptr && setBlockIndexCandidates.value_comp()(pindex, m_chain.Tip())) { return; } - bool is_active_chainstate = this == &m_chainman.ActiveChainstate(); - if (is_active_chainstate) { - // The active chainstate should always add entries that have more + const CBlockIndex* target_block{TargetBlock()}; + if (m_target_blockhash && !target_block) { + return; + } + if (!target_block) { + // If no specific target block, add all entries that have more // work than the tip. setBlockIndexCandidates.insert(pindex); - } else if (!m_disabled) { - // For the background chainstate, we only consider connecting blocks - // towards the snapshot base (which can't be nullptr or else we'll - // never make progress). - const CBlockIndex* snapshot_base{m_chainman.GetSnapshotBaseBlock()}; - if (!snapshot_base) return; - if (snapshot_base->GetAncestor(pindex->nHeight) == pindex) { + } else { + // If there is a target block, only consider connecting blocks + // towards the target block. + if (target_block->GetAncestor(pindex->nHeight) == pindex) { setBlockIndexCandidates.insert(pindex); } } @@ -3894,7 +3935,7 @@ void ChainstateManager::ReceivedBlockTransactions(const CBlock& block, CBlockInd // nChainTx value is not zero, assert that value is actually correct. auto prev_tx_sum = [](CBlockIndex& block) { return block.nTx + (block.pprev ? block.pprev->nChainTx : 0); }; if (!Assume(pindexNew->nChainTx == 0 || pindexNew->nChainTx == prev_tx_sum(*pindexNew) || - pindexNew == GetSnapshotBaseBlock())) { + std::ranges::any_of(m_chainstates, [&](const auto& cs) EXCLUSIVE_LOCKS_REQUIRED(cs_main) { return cs->SnapshotBase() == pindexNew; }))) { LogWarning("Internal bug detected: block %d has unexpected nChainTx %i that should be %i (%s %s). Please report this issue here: %s\n", pindexNew->nHeight, pindexNew->nChainTx, prev_tx_sum(*pindexNew), PACKAGE_NAME, FormatFullVersion(), PACKAGE_BUGREPORT); pindexNew->nChainTx = 0; @@ -3925,7 +3966,7 @@ void ChainstateManager::ReceivedBlockTransactions(const CBlock& block, CBlockInd } pindex->nChainTx = prev_tx_sum(*pindex); pindex->nSequenceId = nBlockSequenceId++; - for (Chainstate *c : GetAll()) { + for (const auto& c : m_chainstates) { c->TryAddBlockIndexCandidate(pindex); } std::pair::iterator, std::multimap::iterator> range = m_blockman.m_blocks_unlinked.equal_range(pindex); @@ -4521,7 +4562,7 @@ bool ChainstateManager::ProcessNewBlock(const std::shared_ptr& blo return error("%s: ActivateBestChain failed: %s", __func__, state.ToString()); LogPrintf("%s : ACCEPTED\n", __func__); - Chainstate* bg_chain{WITH_LOCK(cs_main, return BackgroundSyncInProgress() ? m_ibd_chainstate.get() : nullptr)}; + Chainstate* bg_chain{WITH_LOCK(cs_main, return HistoricalChainstate())}; BlockValidationState bg_state; if (bg_chain && !bg_chain->ActivateBestChain(bg_state, block)) { return error("%s: [background] ActivateBestChain failed (%s)", __func__, bg_state.ToString()); @@ -4908,7 +4949,7 @@ bool ChainstateManager::LoadBlockIndex() // Load block index from databases bool needs_init = fReindex; if (!fReindex) { - bool ret{m_blockman.LoadBlockIndexDB(SnapshotBlockhash())}; + bool ret{m_blockman.LoadBlockIndexDB(CurrentChainstate().m_from_snapshot_blockhash)}; if (!ret) return false; m_blockman.ScanAndUnlinkAlreadyPrunedFiles(); @@ -4924,11 +4965,11 @@ bool ChainstateManager::LoadBlockIndex() // VALID_TRANSACTIONS (eg if we haven't yet downloaded the block), // so we special-case the snapshot block as a potential candidate // here. - if (pindex == GetSnapshotBaseBlock() || + if (pindex == CurrentChainstate().SnapshotBase() || (pindex->IsValid(BLOCK_VALID_TRANSACTIONS) && (pindex->HaveNumChainTxs() || pindex->pprev == nullptr))) { - for (Chainstate* chainstate : GetAll()) { + for (const auto& chainstate : m_chainstates) { chainstate->TryAddBlockIndexCandidate(pindex); } } @@ -5114,16 +5155,8 @@ void ChainstateManager::LoadExternalBlockFile( // until after all of the block files are loaded. ActivateBestChain can be // called by concurrent network message processing. but, that is not // reliable for the purpose of pruning while importing. - bool activation_failure = false; - for (auto c : GetAll()) { - BlockValidationState state; - if (!c->ActivateBestChain(state, pblock)) { - LogPrint(BCLog::REINDEX, "failed to activate chain (%s)\n", state.ToString()); - activation_failure = true; - break; - } - } - if (activation_failure) { + if (auto result{ActivateBestChains()}; !result) { + LogDebug(BCLog::REINDEX, "%s\n", util::ErrorString(result).original); break; } } @@ -5226,7 +5259,7 @@ void ChainstateManager::CheckBlockIndex() // to earlier blocks that have not been downloaded or validated yet, so // checks for later blocks can assume the earlier blocks were validated and // be stricter, testing for more requirements. - const CBlockIndex* snap_base{GetSnapshotBaseBlock()}; + const CBlockIndex* snap_base{CurrentChainstate().SnapshotBase()}; CBlockIndex *snap_first_missing{}, *snap_first_notx{}, *snap_first_notv{}, *snap_first_nocv{}, *snap_first_nosv{}; auto snap_update_firsts = [&] { if (pindex == snap_base) { @@ -5269,7 +5302,7 @@ void ChainstateManager::CheckBlockIndex() if (pindex->pprev == nullptr) { // Genesis block checks. assert(pindex->GetBlockHash() == GetConsensus().hashGenesisBlock); // Genesis block's hash must match. - for (auto c : GetAll()) { + for (const auto& c : m_chainstates) { if (c->m_chain.Genesis() != nullptr) { assert(pindex == c->m_chain.Genesis()); // The chain's genesis block must be this block. } @@ -5327,7 +5360,7 @@ void ChainstateManager::CheckBlockIndex() } // Chainstate-specific checks on setBlockIndexCandidates - for (auto c : GetAll()) { + for (const auto& c : m_chainstates) { if (c->m_chain.Tip() == nullptr) continue; // Two main factors determine whether pindex is a candidate in // setBlockIndexCandidates: @@ -5367,13 +5400,13 @@ void ChainstateManager::CheckBlockIndex() // is the base of the snapshot, pindex is also a potential // candidate. if (pindexFirstMissing == nullptr || pindex == c->m_chain.Tip() || pindex == c->SnapshotBase()) { - // If this chainstate is the active chainstate, pindex - // must be in setBlockIndexCandidates. Otherwise, this - // chainstate is a background validation chainstate, and - // pindex only needs to be added if it is an ancestor of - // the snapshot that is being validated. - if (c == &ActiveChainstate() || snap_base->GetAncestor(pindex->nHeight) == pindex) { - assert(c->setBlockIndexCandidates.count(pindex)); + // If this chainstate is not a historical chainstate + // targeting a specific block, pindex must be in + // setBlockIndexCandidates. Otherwise, pindex only + // needs to be added if it is an ancestor of the target + // block. + if (!c->TargetBlock() || c->TargetBlock()->GetAncestor(pindex->nHeight) == pindex) { + assert(c->setBlockIndexCandidates.contains(const_cast(pindex))); } } // If some parent is missing, then it could be that this block was in @@ -5412,11 +5445,10 @@ void ChainstateManager::CheckBlockIndex() // tip. // So if this block is itself better than any m_chain.Tip() and it wasn't in // setBlockIndexCandidates, then it must be in m_blocks_unlinked. - for (auto c : GetAll()) { - const bool is_active = c == &ActiveChainstate(); + for (const auto& c : m_chainstates) { if (!CBlockIndexWorkComparator()(pindex, c->m_chain.Tip()) && c->setBlockIndexCandidates.count(pindex) == 0) { if (pindexFirstInvalid == nullptr && pindexFirstConflicing == nullptr) { - if (is_active || snap_base->GetAncestor(pindex->nHeight) == pindex) { + if (!c->TargetBlock() || c->TargetBlock()->GetAncestor(pindex->nHeight) == pindex) { assert(foundInUnlinked); } } @@ -5530,9 +5562,8 @@ double GuessVerificationProgress(const ChainTxData& data, const CBlockIndex *pin if (pindex == nullptr) return 0.0; - if (!Assume(pindex->nChainTx > 0)) { - LogWarning("Internal bug detected: block %d has unset nChainTx (%s %s). Please report this issue here: %s\n", - pindex->nHeight, PACKAGE_NAME, FormatFullVersion(), PACKAGE_BUGREPORT); + if (pindex->nChainTx == 0) { + LogDebug(BCLog::VALIDATION, "Block %d has unset nChainTx. Unable to estimate verification progress.\n", pindex->nHeight); return 0.0; } @@ -5549,22 +5580,13 @@ double GuessVerificationProgress(const ChainTxData& data, const CBlockIndex *pin return std::min(pindex->nChainTx / fTxTotal, 1.0); } -std::optional ChainstateManager::SnapshotBlockhash() const { - LOCK(::cs_main); // for m_active_chainstate access - if (m_active_chainstate && m_active_chainstate->m_from_snapshot_blockhash) { - // If a snapshot chainstate exists, it will always be our active. - return m_active_chainstate->m_from_snapshot_blockhash; - } - return std::nullopt; -} - std::vector ChainstateManager::GetAll() { LOCK(::cs_main); std::vector out; - for (Chainstate* cs : {m_ibd_chainstate.get(), m_snapshot_chainstate.get()}) { - if (this->IsUsable(cs)) out.push_back(cs); + for (const auto& cs : m_chainstates) { + if (cs && cs->m_assumeutxo != Assumeutxo::INVALID && !cs->m_target_utxohash) out.push_back(cs.get()); } return out; @@ -5575,13 +5597,9 @@ Chainstate& ChainstateManager::InitializeChainstate(CTxMemPool* mempool, const std::unique_ptr& chain_helper) { AssertLockHeld(::cs_main); - assert(!m_ibd_chainstate); - assert(!m_active_chainstate); - - m_ibd_chainstate = std::make_unique( - mempool, m_blockman, *this, evoDb, chain_helper); - m_active_chainstate = m_ibd_chainstate.get(); - return *m_active_chainstate; + assert(m_chainstates.empty()); + m_chainstates.emplace_back(std::make_unique(mempool, m_blockman, *this, evoDb, chain_helper)); + return *m_chainstates.back(); } [[nodiscard]] static bool DeleteCoinsDBFromDisk(const fs::path db_path, bool is_snapshot) @@ -5624,26 +5642,31 @@ Chainstate& ChainstateManager::InitializeChainstate(CTxMemPool* mempool, return destroyed && !fs::exists(db_path); } -util::Result ChainstateManager::ActivateSnapshot( +util::Result ChainstateManager::ActivateSnapshot( AutoFile& coins_file, const SnapshotMetadata& metadata, bool in_memory) { uint256 base_blockhash = metadata.m_base_blockhash; - if (this->SnapshotBlockhash()) { - return util::Error{_("Can't activate a snapshot-based chainstate more than once")}; - } + CBlockIndex* snapshot_start_block{nullptr}; { LOCK(::cs_main); + if (this->CurrentChainstate().m_from_snapshot_blockhash) { + return util::Error{_("Can't activate a snapshot-based chainstate more than once")}; + } + + if (this->CurrentChainstate().m_from_snapshot_blockhash) { + return util::Error{Untranslated("Can't activate a snapshot-based chainstate more than once")}; + } if (!GetParams().AssumeutxoForBlockhash(base_blockhash).has_value()) { return util::Error{strprintf(_("assumeutxo block hash in snapshot metadata not recognized (%s)"), base_blockhash.ToString())}; } - CBlockIndex* snapshot_start_block = m_blockman.LookupBlockIndex(base_blockhash); + snapshot_start_block = m_blockman.LookupBlockIndex(base_blockhash); if (!snapshot_start_block) { return util::Error{strprintf(_("The base block header (%s) must appear in the headers chain. Make sure all headers are syncing, and call loadtxoutset again."), base_blockhash.ToString())}; @@ -5658,7 +5681,8 @@ util::Result ChainstateManager::ActivateSnapshot( return util::Error{_("A forked headers-chain with more work than the chain with the snapshot base block header exists. Please proceed to sync without AssumeUtxo.")}; } - if (Assert(m_active_chainstate->GetMempool())->size() > 0) { + auto mempool{CurrentChainstate().GetMempool()}; + if (mempool && mempool->size() > 0) { return util::Error{_("Can't activate a snapshot when mempool not empty.")}; } } @@ -5707,19 +5731,19 @@ util::Result ChainstateManager::ActivateSnapshot( LOCK(::cs_main); snapshot_chainstate->InitCoinsDB( static_cast(current_coinsdb_cache_size * SNAPSHOT_CACHE_PERC), - in_memory, false, "chainstate"); + in_memory, /*should_wipe=*/false); snapshot_chainstate->InitCoinsCache( static_cast(current_coinstip_cache_size * SNAPSHOT_CACHE_PERC)); } - auto cleanup_bad_snapshot = [&](const std::string& reason) EXCLUSIVE_LOCKS_REQUIRED(::cs_main) -> util::Result { + auto cleanup_bad_snapshot = [&](const std::string& reason) EXCLUSIVE_LOCKS_REQUIRED(::cs_main) -> util::Result { LogPrintf("[snapshot] activation failed - %s\n", reason); this->ReleaseSnapshotPruneLock(); this->MaybeRebalanceCaches(); // PopulateAndValidateSnapshot can return (in error) before the leveldb datadir // has been created, so only attempt removal if we got that far. - if (auto snapshot_datadir = node::FindSnapshotChainstateDir()) { + if (auto snapshot_datadir = node::FindAssumeutxoChainstateDir(gArgs.GetDataDirNet())) { // We have to destruct leveldb::DB in order to release the db lock, otherwise // DestroyDB() (in DeleteCoinsDBFromDisk()) will fail. See `leveldb::~DBImpl()`. // Destructing the chainstate (and so resetting the coinsviews object) does this. @@ -5755,27 +5779,18 @@ util::Result ChainstateManager::ActivateSnapshot( } } - assert(!m_snapshot_chainstate); - m_snapshot_chainstate.swap(snapshot_chainstate); - const bool chaintip_loaded = m_snapshot_chainstate->LoadChainTip(); + Chainstate& chainstate{AddChainstate(std::move(snapshot_chainstate))}; + const bool chaintip_loaded{chainstate.LoadChainTip()}; assert(chaintip_loaded); - - // Transfer possession of the mempool to the snapshot chainstate. - // Mempool is empty at this point because we're still in IBD. - Assert(m_active_chainstate->m_mempool->size() == 0); - Assert(!m_snapshot_chainstate->m_mempool); - m_snapshot_chainstate->m_mempool = m_active_chainstate->m_mempool; - m_active_chainstate->m_mempool = nullptr; - m_active_chainstate = m_snapshot_chainstate.get(); - m_snapshot_chainstate->m_evoDb.SetDefaultIdentity(EvoDbIdentity::SNAPSHOT); - m_blockman.m_snapshot_height = this->GetSnapshotBaseHeight(); + m_blockman.m_snapshot_height = Assert(chainstate.SnapshotBase())->nHeight; + chainstate.m_evoDb.SetDefaultIdentity(EvoDbIdentity::SNAPSHOT); LogPrintf("[snapshot] successfully activated snapshot %s\n", base_blockhash.ToString()); LogPrintf("[snapshot] (%.2f MB)\n", - m_snapshot_chainstate->CoinsTip().DynamicMemoryUsage() / (1000 * 1000)); + chainstate.CoinsTip().DynamicMemoryUsage() / (1000 * 1000)); this->MaybeRebalanceCaches(); - return {}; + return snapshot_start_block; } @@ -5789,21 +5804,25 @@ util::Result ChainstateManager::ActivateSnapshot( // Eventually (TODO), we could somehow separate this function's runtime from // maintenance of the active chain, but that will either require // -// (i) setting `m_disabled` immediately and ensuring all chainstate accesses go -// through IsUsable() checks, or +// (i) recording completion immediately and ensuring all chainstate accesses +// respect the explicit assumeutxo state, or // // (ii) giving each chainstate its own lock instead of using cs_main for everything. bool ChainstateManager::HandleSnapshotStateMismatch( const std::string& reason, std::function shutdown_fnc) { LOCK(::cs_main); - if (!m_snapshot_chainstate || m_active_chainstate != m_snapshot_chainstate.get() || - !IsUsable(m_snapshot_chainstate.get()) || !IsUsable(m_ibd_chainstate.get()) || - m_snapshot_chainstate->m_evoDb.HasActiveTransaction()) { + Chainstate& unvalidated_cs{CurrentChainstate()}; + Chainstate* validated_cs{HistoricalChainstate()}; + if (!unvalidated_cs.m_from_snapshot_blockhash || + unvalidated_cs.m_assumeutxo != Assumeutxo::UNVALIDATED || + !validated_cs || validated_cs->m_assumeutxo != Assumeutxo::VALIDATED || + validated_cs->m_target_utxohash || + unvalidated_cs.m_evoDb.HasActiveTransaction()) { return false; } - const int snapshot_tip_height{m_snapshot_chainstate->m_chain.Height()}; + const int snapshot_tip_height{unvalidated_cs.m_chain.Height()}; const int snapshot_base_height{GetSnapshotBaseHeight().value_or(snapshot_tip_height)}; bilingual_str user_error = strprintf(_( "%s failed to validate the -assumeutxo snapshot state. " @@ -5823,23 +5842,22 @@ bool ChainstateManager::HandleSnapshotStateMismatch( LogPrintf("[snapshot] !!! %s\n", user_error.original); LogPrintf("[snapshot] deleting snapshot, reverting to validated chain, and stopping node\n"); - m_ibd_chainstate->ForceFlushStateToDisk(); - m_snapshot_chainstate->ForceFlushStateToDisk(); - if (!m_ibd_chainstate->m_evoDb.CommitRootTransaction(EvoDbIdentity::NORMAL, /*sync=*/true) || - !m_snapshot_chainstate->m_evoDb.CommitRootTransaction(EvoDbIdentity::SNAPSHOT, /*sync=*/true)) { + // Resume ordinary validation if the snapshot chainstate is rejected. + unvalidated_cs.m_assumeutxo = Assumeutxo::INVALID; + validated_cs->SetTargetBlock(nullptr); + validated_cs->ForceFlushStateToDisk(); + unvalidated_cs.ForceFlushStateToDisk(); + if (!validated_cs->m_evoDb.CommitRootTransaction(EvoDbIdentity::NORMAL, /*sync=*/true) || + !unvalidated_cs.m_evoDb.CommitRootTransaction(EvoDbIdentity::SNAPSHOT, /*sync=*/true)) { user_error += Untranslated("\nFailed to sync EvoDB before invalidating the snapshot."); } - m_active_chainstate = m_ibd_chainstate.get(); - m_snapshot_chainstate->m_disabled = true; ReleaseSnapshotPruneLock(); - assert(!IsUsable(m_snapshot_chainstate.get())); - assert(IsUsable(m_ibd_chainstate.get())); - auto rename_result = m_snapshot_chainstate->InvalidateCoinsDBOnDisk(); + auto rename_result = unvalidated_cs.InvalidateCoinsDBOnDisk(); if (!rename_result) { user_error += Untranslated("\n") + util::ErrorString(rename_result); - } else if (!m_ibd_chainstate->m_evoDb.DiscardSnapshotMarkers()) { + } else if (!validated_cs->m_evoDb.DiscardSnapshotMarkers()) { LogPrintf("[snapshot] failed to remove invalid snapshot EvoDB markers\n"); } shutdown_fnc(user_error); @@ -5850,20 +5868,14 @@ bool ChainstateManager::HandleSnapshotStateMismatch( Chainstate& ChainstateManager::ActiveChainstate() const { LOCK(::cs_main); - assert(m_active_chainstate); - return *m_active_chainstate; -} - -bool ChainstateManager::IsSnapshotActive() const -{ - LOCK(::cs_main); - return m_snapshot_chainstate && m_active_chainstate == m_snapshot_chainstate.get(); + return CurrentChainstate(); } bool ChainstateManager::IsSnapshotActiveAndUnvalidated() const { LOCK(::cs_main); - return m_snapshot_chainstate && m_active_chainstate == m_snapshot_chainstate.get() && !IsSnapshotValidated(); + Chainstate& current_cs{CurrentChainstate()}; + return current_cs.m_from_snapshot_blockhash && current_cs.m_assumeutxo == Assumeutxo::UNVALIDATED; } bool ChainstateManager::IsQuorumTypeEnabled(const Consensus::LLMQType llmqType, @@ -5919,34 +5931,30 @@ bool ChainstateManager::IsQuorumTypeEnabled(const Consensus::LLMQType llmqType, void ChainstateManager::MaybeRebalanceCaches() { AssertLockHeld(::cs_main); - bool ibd_usable = this->IsUsable(m_ibd_chainstate.get()); - bool snapshot_usable = this->IsUsable(m_snapshot_chainstate.get()); - assert(ibd_usable || snapshot_usable); - - if (ibd_usable && !snapshot_usable) { - LogPrintf("[snapshot] allocating all cache to the IBD chainstate\n"); - // Allocate everything to the IBD chainstate. - m_ibd_chainstate->ResizeCoinsCaches(m_total_coinstip_cache, m_total_coinsdb_cache); - } - else if (snapshot_usable && !ibd_usable) { + Chainstate& current_cs{CurrentChainstate()}; + Chainstate* historical_cs{HistoricalChainstate()}; + if (!historical_cs && !current_cs.m_from_snapshot_blockhash) { + // Allocate everything to the IBD chainstate. This will always happen + // when we are not using a snapshot. + current_cs.ResizeCoinsCaches(m_total_coinstip_cache, m_total_coinsdb_cache); + } else if (!historical_cs) { // If background validation has completed and snapshot is our active chain... LogPrintf("[snapshot] allocating all cache to the snapshot chainstate\n"); // Allocate everything to the snapshot chainstate. - m_snapshot_chainstate->ResizeCoinsCaches(m_total_coinstip_cache, m_total_coinsdb_cache); - } - else if (ibd_usable && snapshot_usable) { + current_cs.ResizeCoinsCaches(m_total_coinstip_cache, m_total_coinsdb_cache); + } else { // If both chainstates exist, determine who needs more cache based on IBD status. // // Note: shrink caches first so that we don't inadvertently overwhelm available memory. - if (m_snapshot_chainstate->IsInitialBlockDownload()) { - m_ibd_chainstate->ResizeCoinsCaches( + if (current_cs.IsInitialBlockDownload()) { + historical_cs->ResizeCoinsCaches( m_total_coinstip_cache * 0.05, m_total_coinsdb_cache * 0.05); - m_snapshot_chainstate->ResizeCoinsCaches( + current_cs.ResizeCoinsCaches( m_total_coinstip_cache * 0.95, m_total_coinsdb_cache * 0.95); } else { - m_snapshot_chainstate->ResizeCoinsCaches( + current_cs.ResizeCoinsCaches( m_total_coinstip_cache * 0.05, m_total_coinsdb_cache * 0.05); - m_ibd_chainstate->ResizeCoinsCaches( + historical_cs->ResizeCoinsCaches( m_total_coinstip_cache * 0.95, m_total_coinsdb_cache * 0.95); } } @@ -5954,18 +5962,16 @@ void ChainstateManager::MaybeRebalanceCaches() void ChainstateManager::ResetChainstates() { - if (m_active_chainstate) { - m_active_chainstate->m_evoDb.SetDefaultIdentity(EvoDbIdentity::NORMAL); + if (!m_chainstates.empty()) { + CurrentChainstate().m_evoDb.SetDefaultIdentity(EvoDbIdentity::NORMAL); } - m_ibd_chainstate.reset(); - m_snapshot_chainstate.reset(); - m_active_chainstate = nullptr; + m_chainstates.clear(); } void ChainstateManager::ProtectSnapshotBaseFromPruning() { AssertLockHeld(::cs_main); - const CBlockIndex* base{GetSnapshotBaseBlock()}; + const CBlockIndex* base{CurrentChainstate().SnapshotBase()}; if (!base) return; // The generic prune-lock buffer makes this conservative: automatic and @@ -6008,16 +6014,16 @@ bool IsBIP30Unspendable(const CBlockIndex& block_index) DeploymentActiveAfter(pindexPrev, chainman, Consensus::DEPLOYMENT_V24)); } -bool ChainstateManager::DetectSnapshotChainstate(CTxMemPool* mempool, bilingual_str& error) +Chainstate* ChainstateManager::LoadAssumeutxoChainstate(CTxMemPool* mempool, bilingual_str& error) { - assert(!m_snapshot_chainstate); - std::optional path = node::FindSnapshotChainstateDir(); + assert(!CurrentChainstate().m_from_snapshot_blockhash); + std::optional path = node::FindAssumeutxoChainstateDir(gArgs.GetDataDirNet()); if (!path) { - return true; + return nullptr; } std::optional base_blockhash = node::ReadSnapshotBaseBlockhash(*path); if (!base_blockhash) { - return true; + return nullptr; } LogPrintf("[snapshot] detected active snapshot chainstate (%s) - loading\n", fs::PathToString(*path)); @@ -6028,137 +6034,124 @@ bool ChainstateManager::DetectSnapshotChainstate(CTxMemPool* mempool, bilingual_ LogPrintf("[snapshot] snapshot EvoDB marker is missing for base block %s\n", base_blockhash->ToString()); error = _("Snapshot chainstate EvoDB marker is missing. Reindex is required."); - return false; + return nullptr; } Assert(this->ActiveChainstate().m_mempool == mempool); - this->ActivateExistingSnapshot(*base_blockhash); - return true; + auto snapshot_chainstate{std::make_unique( + nullptr, m_blockman, *this, evo_db, this->ActiveChainstate().m_chain_helper, *base_blockhash)}; + LogPrintf("[snapshot] switching active chainstate to %s\n", snapshot_chainstate->ToString()); + return &this->AddChainstate(std::move(snapshot_chainstate)); } -Chainstate& ChainstateManager::ActivateExistingSnapshot(uint256 base_blockhash) +Chainstate& ChainstateManager::AddChainstate(std::unique_ptr chainstate) { - assert(!m_snapshot_chainstate); - CEvoDB& evo_db = this->ActiveChainstate().m_evoDb; - m_snapshot_chainstate = std::make_unique( - /*mempool=*/nullptr, m_blockman, *this, - evo_db, - this->ActiveChainstate().m_chain_helper, - base_blockhash); - LogPrintf("[snapshot] switching active chainstate to %s\n", m_snapshot_chainstate->ToString()); - - Assert(m_active_chainstate->m_mempool); - Assert(m_active_chainstate->m_mempool->size() == 0); - Assert(!m_snapshot_chainstate->m_mempool); - m_snapshot_chainstate->m_mempool = m_active_chainstate->m_mempool; - m_active_chainstate->m_mempool = nullptr; - m_active_chainstate = m_snapshot_chainstate.get(); - evo_db.SetDefaultIdentity(EvoDbIdentity::SNAPSHOT); - return *m_snapshot_chainstate; -} -static fs::path GetSnapshotCoinsDBPath(Chainstate& cs) EXCLUSIVE_LOCKS_REQUIRED(::cs_main) + Chainstate& prev_chainstate{CurrentChainstate()}; + assert(prev_chainstate.m_assumeutxo == Assumeutxo::VALIDATED); + Assert(prev_chainstate.m_mempool); + // Set target block for historical chainstate to snapshot block. + assert(!prev_chainstate.m_target_blockhash); + prev_chainstate.m_target_blockhash = chainstate->m_from_snapshot_blockhash; + m_chainstates.push_back(std::move(chainstate)); + Chainstate& curr_chainstate{CurrentChainstate()}; + assert(&curr_chainstate == m_chainstates.back().get()); + + // Transfer possession of the mempool to the chainstate. + // Mempool is empty at this point because we're still in IBD. + assert(prev_chainstate.m_mempool->size() == 0); + assert(!curr_chainstate.m_mempool); + std::swap(curr_chainstate.m_mempool, prev_chainstate.m_mempool); + curr_chainstate.m_evoDb.SetDefaultIdentity(EvoDbIdentity::SNAPSHOT); + return curr_chainstate; +} +ChainstateRole Chainstate::GetRole() const { - AssertLockHeld(::cs_main); - // Should never be called on a non-snapshot chainstate. - assert(cs.m_from_snapshot_blockhash); - auto storage_path_maybe = cs.CoinsDB().StoragePath(); - // Should never be called with a non-existent storage path. - assert(storage_path_maybe); - return *storage_path_maybe; + return ChainstateRole{ + .validated = m_assumeutxo == Assumeutxo::VALIDATED, + .historical = bool{m_target_blockhash}, + }; } util::Result Chainstate::InvalidateCoinsDBOnDisk() { - fs::path snapshot_datadir = GetSnapshotCoinsDBPath(*this); + // Should never be called on a non-snapshot chainstate. + assert(m_from_snapshot_blockhash); // Coins views no longer usable. m_coins_views.reset(); - auto invalid_path = snapshot_datadir + "_INVALID"; - std::string dbpath = fs::PathToString(snapshot_datadir); - std::string target = fs::PathToString(invalid_path); - LogPrintf("[snapshot] renaming snapshot datadir %s to %s\n", dbpath, target); + const fs::path db_path{StoragePath()}; + const fs::path invalid_path{db_path + "_INVALID"}; + const std::string db_path_str{fs::PathToString(db_path)}; + const std::string invalid_path_str{fs::PathToString(invalid_path)}; + LogPrintf("[snapshot] renaming snapshot datadir %s to %s\n", db_path_str, invalid_path_str); - // The invalid snapshot datadir is simply moved and not deleted because we may + // The invalid storage directory is simply moved and not deleted because we may // want to do forensics later during issue investigation. The user is instructed - // accordingly in MaybeCompleteSnapshotValidation(). + // accordingly in MaybeValidateSnapshot(). try { - fs::rename(snapshot_datadir, invalid_path); - DirectoryCommit(snapshot_datadir.parent_path()); + fs::rename(db_path, invalid_path); + DirectoryCommit(db_path.parent_path()); } catch (const fs::filesystem_error& e) { - auto src_str = fs::PathToString(snapshot_datadir); - auto dest_str = fs::PathToString(invalid_path); - - LogPrintf("%s: error renaming file '%s' -> '%s': %s\n", - __func__, src_str, dest_str, e.what()); + LogPrintf("While invalidating the coins db: Error renaming file '%s' -> '%s': %s\n", + db_path_str, invalid_path_str, e.what()); return util::Error{strprintf(_( "Rename of '%s' -> '%s' failed. " "You should resolve this by manually moving or deleting the invalid " "snapshot directory %s, otherwise you will encounter the same error again " "on the next startup."), - src_str, dest_str, src_str)}; + db_path_str, invalid_path_str, db_path_str)}; } return {}; } -bool ChainstateManager::DeleteSnapshotChainstate() +bool ChainstateManager::DeleteChainstate(Chainstate& chainstate) { AssertLockHeld(::cs_main); - Assert(m_snapshot_chainstate); - Assert(m_ibd_chainstate); - - fs::path snapshot_datadir = Assert(node::FindSnapshotChainstateDir()).value(); - if (!DeleteCoinsDBFromDisk(snapshot_datadir, /*is_snapshot=*/ true)) { - LogPrintf("Deletion of %s failed. Please remove it manually to continue reindexing.\n", - fs::PathToString(snapshot_datadir)); + assert(!chainstate.m_coins_views); + const fs::path db_path{chainstate.StoragePath()}; + if (!DeleteCoinsDBFromDisk(db_path, /*is_snapshot=*/bool{chainstate.m_from_snapshot_blockhash})) { + LogError("Deletion of %s failed. Please remove it manually to continue reindexing.", + fs::PathToString(db_path)); return false; } - m_active_chainstate = m_ibd_chainstate.get(); - m_active_chainstate->m_mempool = m_snapshot_chainstate->m_mempool; - m_snapshot_chainstate.reset(); + std::unique_ptr prev_chainstate{Assert(RemoveChainstate(chainstate))}; + Chainstate& curr_chainstate{CurrentChainstate()}; + assert(prev_chainstate->m_mempool->size() == 0); + assert(!curr_chainstate.m_mempool); + std::swap(curr_chainstate.m_mempool, prev_chainstate->m_mempool); return true; } -const CBlockIndex* ChainstateManager::GetSnapshotBaseBlock() const -{ - return m_active_chainstate ? m_active_chainstate->SnapshotBase() : nullptr; -} - std::optional ChainstateManager::GetSnapshotBaseHeight() const { - const CBlockIndex* base = this->GetSnapshotBaseBlock(); + const CBlockIndex* base = CurrentChainstate().SnapshotBase(); return base ? std::make_optional(base->nHeight) : std::nullopt; } -bool ChainstateManager::ValidatedSnapshotCleanup() +void ChainstateManager::RecalculateBestHeader() { - AssertLockHeld(::cs_main); - auto get_storage_path = [](auto& chainstate) EXCLUSIVE_LOCKS_REQUIRED(::cs_main) -> std::optional { - if (!(chainstate && chainstate->HasCoinsViews())) { - return {}; + AssertLockHeld(cs_main); + m_best_header = ActiveChain().Tip(); + for (auto& [_, index] : m_blockman.m_block_index) { + if (!(index.nStatus & (BLOCK_FAILED_MASK | BLOCK_CONFLICT_CHAINLOCK)) && m_best_header->nChainWork < index.nChainWork) { + m_best_header = &index; } - return chainstate->CoinsDB().StoragePath(); - }; - std::optional ibd_chainstate_path_maybe = get_storage_path(m_ibd_chainstate); - std::optional snapshot_chainstate_path_maybe = get_storage_path(m_snapshot_chainstate); + } +} - if (!this->IsSnapshotValidated()) { +bool ChainstateManager::ValidatedSnapshotCleanup(Chainstate& validated_cs, Chainstate& unvalidated_cs) +{ + AssertLockHeld(::cs_main); + if (unvalidated_cs.m_assumeutxo != Assumeutxo::VALIDATED) { // No need to clean up. return false; } - // If either path doesn't exist, that means at least one of the chainstates - // is in-memory, in which case we can't do on-disk cleanup. You'd better be - // in a unittest! - if (!ibd_chainstate_path_maybe || !snapshot_chainstate_path_maybe) { - LogPrintf("[snapshot] snapshot chainstate cleanup cannot happen with " /* Continued */ - "in-memory chainstates. You are testing, right?\n"); - return false; - } + const fs::path validated_path{validated_cs.StoragePath()}; + const fs::path assumed_valid_path{unvalidated_cs.StoragePath()}; + const fs::path delete_path{validated_path + "_todelete"}; - const auto& snapshot_chainstate_path = *snapshot_chainstate_path_maybe; - const auto& ibd_chainstate_path = *ibd_chainstate_path_maybe; - - const uint256 snapshot_tip = m_snapshot_chainstate->CoinsTip().GetBestBlock(); - CEvoDB& evo_db = m_snapshot_chainstate->m_evoDb; + const uint256 snapshot_tip = unvalidated_cs.CoinsTip().GetBestBlock(); + CEvoDB& evo_db = unvalidated_cs.m_evoDb; // Since we're going to be moving around the underlying leveldb filesystem content // for each chainstate, make sure that the chainstates (and their constituent @@ -6167,14 +6160,10 @@ bool ChainstateManager::ValidatedSnapshotCleanup() // The caller of this method will be responsible for reinitializing chainstates // if they want to continue operation. this->ResetChainstates(); - - // No chainstates should be considered usable. - assert(this->GetAll().size() == 0); + assert(this->m_chainstates.size() == 0); LogPrintf("[snapshot] deleting background chainstate directory (now unnecessary) (%s)\n", - fs::PathToString(ibd_chainstate_path)); - - fs::path tmp_old{ibd_chainstate_path + "_todelete"}; + fs::PathToString(validated_path)); auto rename_failed_abort = []( fs::path p_old, @@ -6189,22 +6178,21 @@ bool ChainstateManager::ValidatedSnapshotCleanup() }; try { - fs::rename(ibd_chainstate_path, tmp_old); - DirectoryCommit(ibd_chainstate_path.parent_path()); + fs::rename(validated_path, delete_path); + DirectoryCommit(validated_path.parent_path()); } catch (const fs::filesystem_error& e) { - rename_failed_abort(ibd_chainstate_path, tmp_old, e); + rename_failed_abort(validated_path, delete_path, e); throw; } - LogPrintf("[snapshot] moving snapshot chainstate (%s) to " /* Continued */ - "default chainstate directory (%s)\n", - fs::PathToString(snapshot_chainstate_path), fs::PathToString(ibd_chainstate_path)); + LogPrintf("[snapshot] moving snapshot chainstate (%s) to default chainstate directory (%s)\n", + fs::PathToString(assumed_valid_path), fs::PathToString(validated_path)); try { - fs::rename(snapshot_chainstate_path, ibd_chainstate_path); - DirectoryCommit(snapshot_chainstate_path.parent_path()); + fs::rename(assumed_valid_path, validated_path); + DirectoryCommit(assumed_valid_path.parent_path()); } catch (const fs::filesystem_error& e) { - rename_failed_abort(snapshot_chainstate_path, ibd_chainstate_path, e); + rename_failed_abort(assumed_valid_path, validated_path, e); throw; } @@ -6215,52 +6203,35 @@ bool ChainstateManager::ValidatedSnapshotCleanup() return false; } - if (!DeleteCoinsDBFromDisk(tmp_old, /*is_snapshot=*/false)) { + if (!DeleteCoinsDBFromDisk(delete_path, /*is_snapshot=*/false)) { // No need to AbortNode because once the unneeded bg chainstate data is // moved, it will not interfere with subsequent initialization. LogPrintf("Deletion of %s failed. Please remove it manually, as the " /* Continued */ "directory is now unnecessary.\n", - fs::PathToString(tmp_old)); + fs::PathToString(delete_path)); } else { LogPrintf("[snapshot] deleted background chainstate directory (%s)\n", - fs::PathToString(ibd_chainstate_path)); + fs::PathToString(validated_path)); } return true; } -ChainstateRole Chainstate::GetRole() const -{ - if (m_chainman.GetAll().size() <= 1) { - return ChainstateRole::NORMAL; - } - return (this != &m_chainman.ActiveChainstate()) ? - ChainstateRole::BACKGROUND : - ChainstateRole::ASSUMEDVALID; -} - -Chainstate& ChainstateManager::GetChainstateForIndexing() +std::pair Chainstate::GetPruneRange(int last_height_can_prune) const { - // We can't always return `m_ibd_chainstate` because after background validation - // has completed, `m_snapshot_chainstate == m_active_chainstate`, but it can be - // indexed. - return (this->GetAll().size() > 1) ? *m_ibd_chainstate : *m_active_chainstate; -} - -std::pair ChainstateManager::GetPruneRange(const Chainstate& chainstate, int last_height_can_prune) -{ - if (chainstate.m_chain.Height() <= 0) { + if (m_chain.Height() <= 0) { return {0, 0}; } int prune_start{0}; - if (this->GetAll().size() > 1 && m_snapshot_chainstate.get() == &chainstate) { - // Leave the blocks in the background IBD chain alone if we're pruning - // the snapshot chain. - prune_start = *Assert(GetSnapshotBaseHeight()) + 1; + if (m_from_snapshot_blockhash && m_assumeutxo != Assumeutxo::VALIDATED) { + // Only prune blocks _after_ the snapshot if this is a snapshot chain + // that has not been fully validated yet. The earlier blocks need to be + // kept to validate the snapshot + prune_start = Assert(SnapshotBase())->nHeight + 1; } int max_prune = std::max( - 0, chainstate.m_chain.Height() - static_cast(MIN_BLOCKS_TO_KEEP)); + 0, m_chain.Height() - static_cast(MIN_BLOCKS_TO_KEEP)); // last block to prune is the lesser of (caller-specified height, MIN_BLOCKS_TO_KEEP from the tip) // @@ -6272,3 +6243,36 @@ std::pair ChainstateManager::GetPruneRange(const Chainstate& chainstat return {prune_start, prune_end}; } + +std::optional> ChainstateManager::GetHistoricalBlockRange() const +{ + const Chainstate* chainstate{HistoricalChainstate()}; + if (!chainstate) return {}; + return std::make_pair(chainstate->m_chain.Tip(), chainstate->TargetBlock()); +} + +util::Result ChainstateManager::ActivateBestChains() +{ + // We can't hold cs_main during ActivateBestChain even though we're accessing + // the chainman unique_ptrs since ABC requires us not to be holding cs_main, so retrieve + // the relevant pointers before the ABC call. + AssertLockNotHeld(cs_main); + std::vector chainstates; + { + LOCK(GetMutex()); + chainstates.reserve(m_chainstates.size()); + for (const auto& chainstate : m_chainstates) { + if (chainstate && chainstate->m_assumeutxo != Assumeutxo::INVALID && !chainstate->m_target_utxohash) { + chainstates.push_back(chainstate.get()); + } + } + } + for (Chainstate* chainstate : chainstates) { + BlockValidationState state; + if (!chainstate->ActivateBestChain(state, nullptr)) { + LOCK(GetMutex()); + return util::Error{Untranslated(strprintf("%s Failed to connect best block (%s)", chainstate->ToString(), state.ToString()))}; + } + } + return {}; +} diff --git a/src/validation.h b/src/validation.h index 78e5890b8135..bc541eb77f4d 100644 --- a/src/validation.h +++ b/src/validation.h @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -463,6 +464,16 @@ enum class CoinsCacheSizeState OK = 0 }; +//! Chainstate assumeutxo validity. +enum class Assumeutxo { + //! Every block in the chain has been validated. + VALIDATED, + //! Blocks after an assumeutxo snapshot have been validated but the snapshot itself has not been validated. + UNVALIDATED, + //! The assumeutxo snapshot failed validation. + INVALID, +}; + /** * Chainstate stores and provides an API to update our local knowledge of the * current best chain. @@ -505,22 +516,11 @@ class Chainstate //! Dash const std::unique_ptr& m_chain_helper; CEvoDB& m_evoDb; - - //! This toggle exists for use when doing background validation for UTXO - //! snapshots. - //! - //! In the expected case, it is set once the background validation chain reaches the - //! same height as the base of the snapshot and its UTXO set is found to hash to - //! the expected assumeutxo value. It signals that we should no longer connect - //! blocks to the background chainstate. When set on the background validation - //! chainstate, it signifies that we have fully validated the snapshot chainstate. - //! - //! In the unlikely case that the snapshot chainstate is found to be invalid, this - //! is set to true on the snapshot chainstate. - bool m_disabled GUARDED_BY(::cs_main) {false}; - //! Cached result of LookupBlockIndex(*m_from_snapshot_blockhash) - const CBlockIndex* m_cached_snapshot_base GUARDED_BY(::cs_main) {nullptr}; + mutable const CBlockIndex* m_cached_snapshot_base GUARDED_BY(::cs_main){nullptr}; + + //! Cached result of LookupBlockIndex(*m_target_blockhash) + mutable const CBlockIndex* m_cached_target_block GUARDED_BY(::cs_main){nullptr}; public: //! Reference to a BlockManager instance which itself is shared across all @@ -548,11 +548,14 @@ class Chainstate std::string EvoDbInconsistencyMessage(); + //! Return path to chainstate leveldb directory. + fs::path StoragePath() const; + //! Return the current role of the chainstate. See `ChainstateManager` //! documentation for a description of the different types of chainstates. //! //! @sa ChainstateRole - ChainstateRole GetRole() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + kernel::ChainstateRole GetRole() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main); /** * Initialize the CoinsViews UTXO set database management data structures. The in-memory @@ -563,8 +566,7 @@ class Chainstate void InitCoinsDB( size_t cache_size_bytes, bool in_memory, - bool should_wipe, - fs::path leveldb_name = "chainstate"); + bool should_wipe); //! Initialize the in-memory coins cache (to be done after the health of the on-disk database //! is verified). @@ -582,6 +584,11 @@ class Chainstate //! @see CChain, CBlockIndex. CChain m_chain; + //! Assumeutxo state indicating whether all blocks in the chain were + //! validated, or if the chainstate is based on an assumeutxo snapshot and + //! the snapshot has not been validated. + Assumeutxo m_assumeutxo GUARDED_BY(::cs_main); + /** * The blockhash which is the base of the snapshot this chainstate was created from. * @@ -590,12 +597,42 @@ class Chainstate const std::optional m_from_snapshot_blockhash; std::optional> m_required_background_mn_list_hashes; + //! Target block for this chainstate. If this is not set, chainstate will + //! target the most-work, valid block. If this is set, ChainstateManager + //! considers this a "historical" chainstate since it will only contain old + //! blocks up to the target block, not newer blocks. + std::optional m_target_blockhash GUARDED_BY(::cs_main); + + //! Hash of the UTXO set at the target block, computed when the chainstate + //! reaches the target block, and null before then. + std::optional m_target_utxohash GUARDED_BY(::cs_main); + /** * The base of the snapshot this chainstate was created from. * * nullptr if this chainstate was not created from a snapshot. */ - const CBlockIndex* SnapshotBase() EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + const CBlockIndex* SnapshotBase() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + + //! Return target block which chainstate tip is expected to reach, if this + //! is a historic chainstate being used to validate a snapshot, or null if + //! chainstate targets the most-work block. + const CBlockIndex* TargetBlock() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + //! Set target block for this chainstate. If null, chainstate will target + //! the most-work valid block. If non-null chainstate will be a historic + //! chainstate and target the specified block. + void SetTargetBlock(CBlockIndex* block) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + //! Set target block for this chainstate using just a block hash. Useful + //! when the block database has not been loaded yet. + void SetTargetBlockHash(uint256 block_hash) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + + //! Return true if chainstate reached target block. + bool ReachedTarget() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main) + { + const CBlockIndex* target_block{TargetBlock()}; + assert(!target_block || target_block->GetAncestor(m_chain.Height()) == m_chain.Tip()); + return target_block && target_block == m_chain.Tip(); + } /** * The set of all CBlockIndex entries that have as much work as our current @@ -644,9 +681,6 @@ class Chainstate //! Destructs all objects related to accessing the UTXO set. void ResetCoinsViews() { m_coins_views.reset(); } - //! Does this chainstate have a UTXO set attached? - bool HasCoinsViews() const { return (bool)m_coins_views; } - //! The cache size of the on-disk coins view. size_t m_coinsdb_cache_size_bytes{0}; @@ -700,9 +734,9 @@ class Chainstate * validationinterface callback. * * Note that if this is called while a snapshot chainstate is active, and if - * it is called on a background chainstate whose tip has reached the base block - * of the snapshot, its execution will take *MINUTES* while it hashes the - * background UTXO set to verify the assumeutxo value the snapshot was activated + * it is called on a validated chainstate whose tip has reached the base + * block of the snapshot, its execution will take *MINUTES* while it hashes + * the UTXO set to verify the assumeutxo value the snapshot was activated * with. `cs_main` will be held during this time. * * @returns true unless a system error occurred @@ -782,6 +816,9 @@ class Chainstate { return m_mempool ? &m_mempool->cs : nullptr; } + //! Return the [start, end] (inclusive) of block heights we can prune. + std::pair GetPruneRange(int last_height_can_prune) const EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + private: bool ActivateBestChainStep(BlockValidationState& state, CBlockIndex* pindexMostWork, const std::shared_ptr& pblock, bool& fInvalidFound, ConnectTrace& connectTrace) EXCLUSIVE_LOCKS_REQUIRED(cs_main, m_mempool->cs); bool ConnectTip(BlockValidationState& state, CBlockIndex* pindexNew, const std::shared_ptr& pblock, ConnectTrace& connectTrace, DisconnectedBlockTransactions& disconnectpool) EXCLUSIVE_LOCKS_REQUIRED(cs_main, m_mempool->cs); @@ -840,14 +877,13 @@ enum class SnapshotCompletionResult { // base block. MISSING_CHAINPARAMS, - // Failed to generate UTXO statistics (to check UTXO set hash) for the background - // chainstate. + // Failed to generate UTXO statistics (to check UTXO set hash) for the + // validated chainstate. STATS_FAILED, - // The UTXO set hash of the background validation chainstate does not match - // the one expected by assumeutxo chainparams. + // The UTXO set hash of the validated chainstate does not match the one + // expected by assumeutxo chainparams. HASH_MISMATCH, - // The blockhash of the current tip of the background validation chainstate does // not match the one expected by the snapshot chainstate. BASE_BLOCKHASH_MISMATCH, @@ -858,69 +894,28 @@ enum class SnapshotCompletionResult { }; /** - * Provides an interface for creating and interacting with one or two - * chainstates: an IBD chainstate generated by downloading blocks, and - * an optional snapshot chainstate loaded from a UTXO snapshot. Managed - * chainstates can be maintained at different heights simultaneously. + * Interface for managing multiple \ref Chainstate objects, where each + * chainstate is associated with chainstate* subdirectory in the data directory + * and contains a database of UTXOs existing at a different point in history. + * (See \ref Chainstate class for more information.) * - * This class provides abstractions that allow the retrieval of the current - * most-work chainstate ("Active") as well as chainstates which may be in - * background use to validate UTXO snapshots. + * Normally there is exactly one Chainstate, which contains the UTXO set of + * chain tip if syncing is completed, or the UTXO set the most recent validated + * block if the initial sync is still in progress. * - * Definitions: - * - * *IBD chainstate*: a chainstate whose current state has been "fully" - * validated by the initial block download process. - * - * *Snapshot chainstate*: a chainstate populated by loading in an - * assumeutxo UTXO snapshot. - * - * *Active chainstate*: the chainstate containing the current most-work - * chain. Consulted by most parts of the system (net_processing, - * wallet) as a reflection of the current chain and UTXO set. - * This may either be an IBD chainstate or a snapshot chainstate. - * - * *Background IBD chainstate*: an IBD chainstate for which the - * IBD process is happening in the background while use of the - * active (snapshot) chainstate allows the rest of the system to function. + * However, if an assumeutxo snapshot is loaded before syncing is completed, + * there will be two chainstates. The original fully validated chainstate will + * continue to exist and download new blocks in the background. But the new + * snapshot which is loaded will become a second chainstate. The second + * chainstate will be used as the chain tip for the wallet and RPCs even though + * it is only assumed to be valid. When the initial chainstate catches up to the + * snapshot height and confirms that the assumeutxo snapshot is actually valid, + * the second chainstate will be marked validated and become the only chainstate + * again. */ class ChainstateManager { private: - //! The chainstate used under normal operation (i.e. "regular" IBD) or, if - //! a snapshot is in use, for background validation. - //! - //! Its contents (including on-disk data) will be deleted *upon shutdown* - //! after background validation of the snapshot has completed. We do not - //! free the chainstate contents immediately after it finishes validation - //! to cautiously avoid a case where some other part of the system is still - //! using this pointer (e.g. net_processing). - //! - //! Once this pointer is set to a corresponding chainstate, it will not - //! be reset until init.cpp:Shutdown(). - //! - //! It is important for the pointer to not be deleted until shutdown, - //! because cs_main is not always held when the pointer is accessed, for - //! example when calling ActivateBestChain, so there's no way you could - //! prevent code from using the pointer while deleting it. - std::unique_ptr m_ibd_chainstate GUARDED_BY(::cs_main); - - //! A chainstate initialized on the basis of a UTXO snapshot. If this is - //! non-null, it is always our active chainstate. - //! - //! Once this pointer is set to a corresponding chainstate, it will not - //! be reset until init.cpp:Shutdown(). - //! - //! It is important for the pointer to not be deleted until shutdown, - //! because cs_main is not always held when the pointer is accessed, for - //! example when calling ActivateBestChain, so there's no way you could - //! prevent code from using the pointer while deleting it. - std::unique_ptr m_snapshot_chainstate GUARDED_BY(::cs_main); - - //! Points to either the ibd or snapshot chainstate; indicates our - //! most-work chain. - Chainstate* m_active_chainstate GUARDED_BY(::cs_main) {nullptr}; - CBlockIndex* m_best_invalid GUARDED_BY(::cs_main){nullptr}; const CChainParams m_chainparams; @@ -947,15 +942,6 @@ class ChainstateManager std::array m_warningcache GUARDED_BY(::cs_main); - //! Return true if a chainstate is considered usable. - //! - //! This is false when a background validation chainstate has completed its - //! validation of an assumed-valid chainstate, or when a snapshot - //! chainstate has been found to be invalid. - bool IsUsable(const Chainstate* const cs) const EXCLUSIVE_LOCKS_REQUIRED(::cs_main) { - return cs && !cs->m_disabled; - } - public: using Options = kernel::ChainstateManagerOpts; @@ -963,9 +949,9 @@ class ChainstateManager : m_chainparams{opts.chainparams}, m_blockman{{opts.chainparams}} {}; - //! Function to restart active indexes; set dynamically to avoid a circular - //! dependency on index/base.cpp. - std::function restart_indexes = std::function{}; + //! Function called when snapshot background validation completes; set + //! dynamically to avoid a circular dependency on index/base.cpp. + std::function snapshot_download_completed = std::function(); const CChainParams& GetParams() const { return m_chainparams; } const Consensus::Params& GetConsensus() const { return m_chainparams.GetConsensus(); } @@ -1057,7 +1043,7 @@ class ChainstateManager const std::unique_ptr& chain_helper) LIFETIMEBOUND EXCLUSIVE_LOCKS_REQUIRED(::cs_main); - //! Get all chainstates currently being used. + //! Dash helper returning usable chainstates for LLMQ and legacy callers. std::vector GetAll(); //! Construct and activate a Chainstate on the basis of UTXO snapshot data. @@ -1069,21 +1055,16 @@ class ChainstateManager //! - Verify that the hash of the resulting coinsdb matches the expected hash //! per assumeutxo chain parameters. //! - Wait for our headers chain to include the base block of the snapshot. - //! - "Fast forward" the tip of the new chainstate to the base of the snapshot, - //! faking nTx* block index data along the way. - //! - Move the new chainstate to `m_snapshot_chainstate` and make it our - //! ChainstateActive(). - [[nodiscard]] util::Result ActivateSnapshot( + //! - "Fast forward" the tip of the new chainstate to the base of the snapshot. + //! - Construct the new Chainstate and add it to m_chainstates. + [[nodiscard]] util::Result ActivateSnapshot( AutoFile& coins_file, const node::SnapshotMetadata& metadata, bool in_memory); - //! Once the background validation chainstate has reached the height which - //! is the base of the UTXO snapshot in use, compare its coins to ensure - //! they match those expected by the snapshot. - //! - //! If the coins match (expected), then mark the validation chainstate for - //! deletion and continue using the snapshot chainstate as active. - //! Otherwise, revert to using the ibd chainstate and shutdown. - SnapshotCompletionResult MaybeCompleteSnapshotValidation( + //! Try to validate an assumeutxo snapshot using a validated historical + //! chainstate targeted at the snapshot block. + SnapshotCompletionResult MaybeValidateSnapshot( + Chainstate& validated_cs, + Chainstate& unvalidated_cs, std::function shutdown_fnc = [](bilingual_str msg) { AbortNode(msg.original, msg); }) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); @@ -1096,25 +1077,58 @@ class ChainstateManager std::function shutdown_fnc = [](bilingual_str msg) { AbortNode(msg.original, msg); }); - //! Returns nullptr if no snapshot has been loaded. - const CBlockIndex* GetSnapshotBaseBlock() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + //! Return current chainstate targeting the most-work, network tip. + Chainstate& CurrentChainstate() const EXCLUSIVE_LOCKS_REQUIRED(GetMutex()) + { + for (auto& cs : m_chainstates) { + if (cs && cs->m_assumeutxo != Assumeutxo::INVALID && !cs->m_target_blockhash) return *cs; + } + abort(); + } - //! The most-work chain. - Chainstate& ActiveChainstate() const; - CChain& ActiveChain() const EXCLUSIVE_LOCKS_REQUIRED(GetMutex()) { return ActiveChainstate().m_chain; } - int ActiveHeight() const EXCLUSIVE_LOCKS_REQUIRED(GetMutex()) { return ActiveChain().Height(); } - CBlockIndex* ActiveTip() const EXCLUSIVE_LOCKS_REQUIRED(GetMutex()) { return ActiveChain().Tip(); } + //! Return historical chainstate targeting a specific block, if any. + Chainstate* HistoricalChainstate() const EXCLUSIVE_LOCKS_REQUIRED(GetMutex()) + { + for (auto& cs : m_chainstates) { + if (cs && cs->m_assumeutxo != Assumeutxo::INVALID && cs->m_target_blockhash && !cs->m_target_utxohash) return cs.get(); + } + return nullptr; + } - //! The state of a background sync (for net processing) - bool BackgroundSyncInProgress() const EXCLUSIVE_LOCKS_REQUIRED(GetMutex()) { - return IsUsable(m_snapshot_chainstate.get()) && IsUsable(m_ibd_chainstate.get()); + //! Return fully validated chainstate that should be used for indexing, to + //! support indexes that need to index blocks in order and can't start from + //! the snapshot block. + Chainstate& ValidatedChainstate() const EXCLUSIVE_LOCKS_REQUIRED(GetMutex()) + { + for (auto* cs : {&CurrentChainstate(), HistoricalChainstate()}) { + if (cs && cs->m_assumeutxo == Assumeutxo::VALIDATED) return *cs; + } + abort(); } - //! The tip of the background sync chain - const CBlockIndex* GetBackgroundSyncTip() const EXCLUSIVE_LOCKS_REQUIRED(GetMutex()) { - return BackgroundSyncInProgress() ? m_ibd_chainstate->m_chain.Tip() : nullptr; + //! Remove a chainstate. + std::unique_ptr RemoveChainstate(Chainstate& chainstate) EXCLUSIVE_LOCKS_REQUIRED(GetMutex()) + { + auto it{std::find_if(m_chainstates.begin(), m_chainstates.end(), [&](auto& cs) { return cs.get() == &chainstate; })}; + if (it != m_chainstates.end()) { + auto ret{std::move(*it)}; + m_chainstates.erase(it); + return ret; + } + return nullptr; } + //! Alternatives to CurrentChainstate() used by older code to query latest + //! chainstate information without locking cs_main. Newer code should avoid + //! querying ChainstateManager and use Chainstate objects directly, or + //! should use CurrentChainstate() instead. + //! @{ + Chainstate& ActiveChainstate() const; + CChain& ActiveChain() const EXCLUSIVE_LOCKS_REQUIRED(GetMutex()) { return ActiveChainstate().m_chain; } + int ActiveHeight() const EXCLUSIVE_LOCKS_REQUIRED(GetMutex()) { return ActiveChain().Height(); } + CBlockIndex* ActiveTip() const EXCLUSIVE_LOCKS_REQUIRED(GetMutex()) { return ActiveChain().Tip(); } + //! @} + node::BlockMap& BlockIndex() EXCLUSIVE_LOCKS_REQUIRED(::cs_main) { AssertLockHeld(::cs_main); @@ -1131,21 +1145,12 @@ class ChainstateManager */ mutable VersionBitsCache m_versionbitscache; - //! @returns true if a snapshot-based chainstate is in use. Also implies - //! that a background validation chainstate is also in use. - bool IsSnapshotActive() const; - - std::optional SnapshotBlockhash() const; - - //! Is there a snapshot in use and has it been fully validated? - bool IsSnapshotValidated() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main) - { - return m_snapshot_chainstate && m_ibd_chainstate && m_ibd_chainstate->m_disabled; - } - //! Whether active-state-dependent masternode duties must remain disabled. bool IsSnapshotActiveAndUnvalidated() const; + //! Height of the active snapshot base, if a snapshot chainstate is current. + std::optional GetSnapshotBaseHeight() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + /** * Import blocks from an external file * @@ -1254,8 +1259,12 @@ class ChainstateManager std::optional optHaveDIP0024Quorums = std::nullopt) const; //! When starting up, search the datadir for a chainstate based on a UTXO - //! snapshot that is in the process of being validated. - bool DetectSnapshotChainstate(CTxMemPool* mempool, bilingual_str& error) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + //! snapshot that is in the process of being validated and load it if found. + //! Return pointer to the Chainstate if it is loaded. + Chainstate* LoadAssumeutxoChainstate(CTxMemPool* mempool, bilingual_str& error) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + + //! Add new chainstate. + Chainstate& AddChainstate(std::unique_ptr chainstate) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); void ResetChainstates() EXCLUSIVE_LOCKS_REQUIRED(::cs_main); @@ -1263,15 +1272,9 @@ class ChainstateManager void ProtectSnapshotBaseFromPruning() EXCLUSIVE_LOCKS_REQUIRED(::cs_main); void ReleaseSnapshotPruneLock() EXCLUSIVE_LOCKS_REQUIRED(::cs_main); - //! Switch the active chainstate to one based on a UTXO snapshot that was loaded - //! previously. - //! Remove the snapshot-based chainstate and all on-disk artifacts. + //! Remove the chainstate and all on-disk artifacts. //! Used when reindex{-chainstate} is called during snapshot use. - [[nodiscard]] bool DeleteSnapshotChainstate() EXCLUSIVE_LOCKS_REQUIRED(::cs_main); - - //! Switch the active chainstate to one based on a UTXO snapshot that was loaded - //! previously. - Chainstate& ActivateExistingSnapshot(uint256 base_blockhash) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + [[nodiscard]] bool DeleteChainstate(Chainstate& chainstate) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); //! If we have validated a snapshot chain during this runtime, copy its //! chainstate directory over to the main `chainstate` location, completing @@ -1282,29 +1285,29 @@ class ChainstateManager //! directories are moved or deleted. //! //! @sa node/chainstate:LoadChainstate() - bool ValidatedSnapshotCleanup() EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + bool ValidatedSnapshotCleanup(Chainstate& validated_cs, Chainstate& unvalidated_cs) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); - //! @returns the chainstate that indexes should consult when ensuring that an - //! index is synced with a chain where we can expect block index entries to have - //! BLOCK_HAVE_DATA beneath the tip. - //! - //! In other words, give us the chainstate for which we can reasonably expect - //! that all blocks beneath the tip have been indexed. In practice this means - //! when using an assumed-valid chainstate based upon a snapshot, return only the - //! fully validated chain. - Chainstate& GetChainstateForIndexing() EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + //! Get range of historical blocks to download. + std::optional> GetHistoricalBlockRange() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main); - //! Return the [start, end] (inclusive) of block heights we can prune. - //! - //! start > end is possible, meaning no blocks can be pruned. - std::pair GetPruneRange( - const Chainstate& chainstate, int last_height_can_prune) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + //! Call ActivateBestChain() on every chainstate. + util::Result ActivateBestChains() LOCKS_EXCLUDED(::cs_main); - //! Return the height of the base block of the snapshot in use, if one exists, else - //! nullopt. - std::optional GetSnapshotBaseHeight() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + //! If, due to invalidation / reconsideration of blocks, the previous + //! best header is no longer valid / guaranteed to be the most-work + //! header in our block-index not known to be invalid, recalculate it. + void RecalculateBestHeader() EXCLUSIVE_LOCKS_REQUIRED(::cs_main); ~ChainstateManager(); + + //! List of chainstates. Note: in general, it is not safe to delete + //! Chainstate objects once they are added to this list because there is no + //! mutex that can be locked to prevent Chainstate pointers from being used + //! while they are deleted. (cs_main doesn't work because it is too narrow + //! and is released in the middle of Chainstate::ActivateBestChain to let + //! notifications be processed. m_chainstate_mutex doesn't work because it + //! is not locked at other times when the chainstate is in use.) + std::vector> m_chainstates GUARDED_BY(::cs_main); }; /** Deployment* info via ChainstateManager */ diff --git a/src/validationinterface.cpp b/src/validationinterface.cpp index eeea283b5011..e8167ccb37c3 100644 --- a/src/validationinterface.cpp +++ b/src/validationinterface.cpp @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -19,6 +20,7 @@ #include std::string RemovalReasonToString(const MemPoolRemovalReason& r) noexcept; +using kernel::ChainstateRole; /** * MainSignalsImpl manages a list of shared_ptr callbacks. @@ -230,7 +232,8 @@ void CMainSignals::TransactionRemovedFromMempool(const CTransactionRef& tx, MemP RemovalReasonToString(reason)); } -void CMainSignals::BlockConnected(ChainstateRole role, const std::shared_ptr &pblock, const CBlockIndex *pindex) { +void CMainSignals::BlockConnected(const ChainstateRole& role, const std::shared_ptr& pblock, const CBlockIndex* pindex) +{ auto event = [role, pblock, pindex, this] { m_internals->Iterate([&](CValidationInterface& callbacks) { callbacks.BlockConnected(role, pblock, pindex); }); }; @@ -248,7 +251,8 @@ void CMainSignals::BlockDisconnected(const std::shared_ptr &pblock pindex->nHeight); } -void CMainSignals::ChainStateFlushed(ChainstateRole role, const CBlockLocator &locator) { +void CMainSignals::ChainStateFlushed(const ChainstateRole& role, const CBlockLocator& locator) +{ auto event = [role, locator, this] { m_internals->Iterate([&](CValidationInterface& callbacks) { callbacks.ChainStateFlushed(role, locator); }); }; diff --git a/src/validationinterface.h b/src/validationinterface.h index 3ab9834f3fab..6c0f81c49b57 100644 --- a/src/validationinterface.h +++ b/src/validationinterface.h @@ -7,6 +7,7 @@ #define BITCOIN_VALIDATIONINTERFACE_H #include +#include #include // CTransaction(Ref) #include @@ -169,7 +170,7 @@ class CValidationInterface { * * Called on a background thread. */ - virtual void BlockConnected(ChainstateRole role, const std::shared_ptr &block, const CBlockIndex *pindex) {} + virtual void BlockConnected(const kernel::ChainstateRole& role, const std::shared_ptr& block, const CBlockIndex* pindex) {} /** * Notifies listeners of a block being disconnected * @@ -200,7 +201,7 @@ class CValidationInterface { * * Called on a background thread. */ - virtual void ChainStateFlushed(ChainstateRole role, const CBlockLocator &locator) {} + virtual void ChainStateFlushed(const kernel::ChainstateRole& role, const CBlockLocator& locator) {} /** * Notifies listeners of a block validation result. * If the provided BlockValidationState IsValid, the provided block @@ -245,7 +246,7 @@ class CMainSignals { void SynchronousUpdatedBlockTip(const CBlockIndex *, const CBlockIndex *, bool fInitialDownload); void TransactionAddedToMempool(const CTransactionRef&, int64_t, uint64_t mempool_sequence); void TransactionRemovedFromMempool(const CTransactionRef&, MemPoolRemovalReason, uint64_t mempool_sequence); - void BlockConnected(ChainstateRole, const std::shared_ptr &, const CBlockIndex *pindex); + void BlockConnected(const kernel::ChainstateRole&, const std::shared_ptr&, const CBlockIndex* pindex); void BlockDisconnected(const std::shared_ptr &, const CBlockIndex* pindex); void NotifyTransactionLock(const CTransactionRef &tx, const std::shared_ptr& islock); void NotifyChainLock(const CBlockIndex* pindex, const std::shared_ptr& clsig, const std::string& id); @@ -254,7 +255,7 @@ class CMainSignals { void NotifyInstantSendDoubleSpendAttempt(const CTransactionRef ¤tTx, const CTransactionRef &previousTx); void NotifyRecoveredSig(const std::shared_ptr &sig, const std::string& id, bool proactive_relay); void NotifyMasternodeListChanged(bool undo, const CDeterministicMNList& oldMNList, const CDeterministicMNListDiff& diff); - void ChainStateFlushed(ChainstateRole, const CBlockLocator &); + void ChainStateFlushed(const kernel::ChainstateRole&, const CBlockLocator&); void BlockChecked(const CBlock&, const BlockValidationState&); void NewPoWValidBlock(const CBlockIndex *, const std::shared_ptr&); }; diff --git a/src/wallet/rpc/backup.cpp b/src/wallet/rpc/backup.cpp index 80397cd37321..f2da2bff65ff 100644 --- a/src/wallet/rpc/backup.cpp +++ b/src/wallet/rpc/backup.cpp @@ -1953,20 +1953,27 @@ RPCHelpMan importdescriptors() { if (scanned_time <= GetImportTimestamp(request, now) || results.at(i).exists("error")) { response.push_back(results.at(i)); } else { + std::string error_msg{strprintf("Rescan failed for descriptor with timestamp %d. There " + "was an error reading a block from time %d, which is after or within %d seconds " + "of key creation, and could contain transactions pertaining to the desc. As a " + "result, transactions and coins using this desc may not appear in the wallet.", + GetImportTimestamp(request, now), scanned_time - TIMESTAMP_WINDOW - 1, TIMESTAMP_WINDOW)}; + if (pwallet->chain().havePruned()) { + error_msg += strprintf(" This error could be caused by pruning or data corruption " + "(see dashd log for details) and could be dealt with by downloading and " + "rescanning the relevant blocks (see -reindex option and rescanblockchain RPC)."); + } else if (pwallet->chain().hasAssumedValidChain()) { + error_msg += strprintf(" This error is likely caused by an in-progress assumeutxo " + "background sync. Check logs or getchainstates RPC for assumeutxo background " + "sync progress and try again later."); + } else { + error_msg += strprintf(" This error could potentially caused by data corruption. If " + "the issue persists you may want to reindex (see -reindex option)."); + } + UniValue result = UniValue(UniValue::VOBJ); result.pushKV("success", UniValue(false)); - result.pushKV( - "error", - JSONRPCError( - RPC_MISC_ERROR, - strprintf("Rescan failed for descriptor with timestamp %d. There was an error reading a " - "block from time %d, which is after or within %d seconds of key creation, and " - "could contain transactions pertaining to the desc. As a result, transactions " - "and coins using this desc may not appear in the wallet. This error could be " - "caused by pruning or data corruption (see dashd log for details) and could " - "be dealt with by downloading and rescanning the relevant blocks (see -reindex " - "option and rescanblockchain RPC).", - GetImportTimestamp(request, now), scanned_time - TIMESTAMP_WINDOW - 1, TIMESTAMP_WINDOW))); + result.pushKV("error", JSONRPCError(RPC_MISC_ERROR, error_msg)); response.push_back(std::move(result)); } } diff --git a/src/wallet/rpc/transactions.cpp b/src/wallet/rpc/transactions.cpp index b5b7de1afd7c..0feb91230bcb 100644 --- a/src/wallet/rpc/transactions.cpp +++ b/src/wallet/rpc/transactions.cpp @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -895,9 +896,15 @@ RPCHelpMan rescanblockchain() } } - // We can't rescan beyond non-pruned blocks, stop and throw an error + // We can't rescan unavailable blocks, stop and throw an error if (!pwallet->chain().hasBlocks(pwallet->GetLastBlockHash(), start_height, stop_height)) { - throw JSONRPCError(RPC_MISC_ERROR, "Can't rescan beyond pruned data. Use RPC call getblockchaininfo to determine your pruned height."); + if (pwallet->chain().havePruned() && pwallet->chain().getPruneHeight() >= start_height) { + throw JSONRPCError(RPC_MISC_ERROR, "Can't rescan beyond pruned data. Use RPC call getblockchaininfo to determine your pruned height."); + } + if (pwallet->chain().hasAssumedValidChain()) { + throw JSONRPCError(RPC_MISC_ERROR, "Failed to rescan unavailable blocks likely due to an in-progress assumeutxo background sync. Check logs or getchainstates RPC for assumeutxo background sync progress and try again later."); + } + throw JSONRPCError(RPC_MISC_ERROR, "Failed to rescan unavailable blocks, potentially caused by data corruption. If the issue persists you may want to reindex (see -reindex option)."); } CHECK_NONFATAL(pwallet->chain().findAncestorByHeight(pwallet->GetLastBlockHash(), start_height, FoundBlock().hash(start_block))); diff --git a/src/wallet/test/fuzz/notifications.cpp b/src/wallet/test/fuzz/notifications.cpp index 43db22fee9c6..6a66723a3c29 100644 --- a/src/wallet/test/fuzz/notifications.cpp +++ b/src/wallet/test/fuzz/notifications.cpp @@ -142,8 +142,8 @@ FUZZ_TARGET(wallet_notifications, .init = initialize_setup) info.height = chain.size(); info.data = █ info.chain_time_max = std::numeric_limits::max(); - a.wallet->blockConnected(ChainstateRole::NORMAL, info); - b.wallet->blockConnected(ChainstateRole::NORMAL, info); + a.wallet->blockConnected(kernel::ChainstateRole{}, info); + b.wallet->blockConnected(kernel::ChainstateRole{}, info); // Store the coins for the next block Coins coins_new; for (const auto& tx : block.vtx) { diff --git a/src/wallet/wallet.cpp b/src/wallet/wallet.cpp index 499677662587..deeb0f680c05 100644 --- a/src/wallet/wallet.cpp +++ b/src/wallet/wallet.cpp @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -57,6 +58,7 @@ #include using interfaces::FoundBlock; +using kernel::ChainstateRole; namespace wallet { static isminetype InputIsMine(const CWallet& wallet, const CTxIn& txin) EXCLUSIVE_LOCKS_REQUIRED(wallet.cs_wallet) @@ -628,11 +630,11 @@ bool CWallet::ChangeWalletPassphrase(const SecureString& strOldWalletPassphrase, return false; } -void CWallet::chainStateFlushed(ChainstateRole role, const CBlockLocator& loc) +void CWallet::chainStateFlushed(const ChainstateRole& role, const CBlockLocator& loc) { // Don't update the best block until the chain is attached so that in case of a shutdown, // the rescan will be restarted at next startup. - if (m_attaching_chain || role == ChainstateRole::BACKGROUND) { + if (m_attaching_chain || role.historical) { return; } WalletBatch batch(GetDatabase()); @@ -1474,9 +1476,9 @@ void CWallet::transactionRemovedFromMempool(const CTransactionRef& tx, MemPoolRe } } -void CWallet::blockConnected(ChainstateRole role, const interfaces::BlockInfo& block) +void CWallet::blockConnected(const ChainstateRole& role, const interfaces::BlockInfo& block) { - if (role == ChainstateRole::BACKGROUND) { + if (role.historical) { return; } assert(block.data); @@ -3235,7 +3237,7 @@ std::shared_ptr CWallet::Create(WalletContext& context, const std::stri } if (chain) { - walletInstance->chainStateFlushed(ChainstateRole::NORMAL, chain->getTipLocator()); + walletInstance->chainStateFlushed(ChainstateRole{}, chain->getTipLocator()); } // Try to create wallet backup right after new wallet was created @@ -3539,7 +3541,7 @@ bool CWallet::AttachChain(const std::shared_ptr& walletInstance, interf } } walletInstance->m_attaching_chain = false; - walletInstance->chainStateFlushed(ChainstateRole::NORMAL, chain.getTipLocator()); + walletInstance->chainStateFlushed(ChainstateRole{}, chain.getTipLocator()); walletInstance->GetDatabase().IncrementUpdateCounter(); } walletInstance->m_attaching_chain = false; diff --git a/src/wallet/wallet.h b/src/wallet/wallet.h index 176197188899..4fac02a90bc9 100644 --- a/src/wallet/wallet.h +++ b/src/wallet/wallet.h @@ -683,7 +683,7 @@ class CWallet final : public WalletStorage, public interfaces::Chain::Notificati CWalletTx* AddToWallet(CTransactionRef tx, const TxState& state, const UpdateWalletTxFn& update_wtx=nullptr, bool fFlushOnClose=true, bool rescanning_old_block = false); bool LoadToWallet(const uint256& hash, const UpdateWalletTxFn& fill_wtx) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); void transactionAddedToMempool(const CTransactionRef& tx, int64_t nAcceptTime) override; - void blockConnected(ChainstateRole role, const interfaces::BlockInfo& block) override; + void blockConnected(const kernel::ChainstateRole& role, const interfaces::BlockInfo& block) override; void blockDisconnected(const interfaces::BlockInfo& block) override; void updatedBlockTip() override; int64_t RescanFromTime(int64_t startTime, const WalletRescanReserver& reserver, bool update); @@ -869,7 +869,7 @@ class CWallet final : public WalletStorage, public interfaces::Chain::Notificati /** should probably be renamed to IsRelevantToMe */ bool IsFromMe(const CTransaction& tx) const; CAmount GetDebit(const CTransaction& tx, const isminefilter& filter) const; - void chainStateFlushed(ChainstateRole role, const CBlockLocator& loc) override; + void chainStateFlushed(const kernel::ChainstateRole& role, const CBlockLocator& loc) override; DBErrors LoadWallet(); void AutoLockMasternodeCollaterals(); diff --git a/src/zmq/zmqnotificationinterface.cpp b/src/zmq/zmqnotificationinterface.cpp index 3ba1651f4296..ee689708b22f 100644 --- a/src/zmq/zmqnotificationinterface.cpp +++ b/src/zmq/zmqnotificationinterface.cpp @@ -5,6 +5,7 @@ #include #include +#include #include #include #include @@ -23,9 +24,9 @@ #include #include -CZMQNotificationInterface::CZMQNotificationInterface() -{ -} +using kernel::ChainstateRole; + +CZMQNotificationInterface::CZMQNotificationInterface() = default; CZMQNotificationInterface::~CZMQNotificationInterface() { @@ -194,9 +195,9 @@ void CZMQNotificationInterface::TransactionRemovedFromMempool(const CTransaction }); } -void CZMQNotificationInterface::BlockConnected(ChainstateRole role, const std::shared_ptr& pblock, const CBlockIndex* pindexConnected) +void CZMQNotificationInterface::BlockConnected(const ChainstateRole& role, const std::shared_ptr& pblock, const CBlockIndex* pindexConnected) { - if (role == ChainstateRole::BACKGROUND) { + if (role.historical) { return; } for (const CTransactionRef& ptx : pblock->vtx) { diff --git a/src/zmq/zmqnotificationinterface.h b/src/zmq/zmqnotificationinterface.h index e18581245364..93cf9da5e8f0 100644 --- a/src/zmq/zmqnotificationinterface.h +++ b/src/zmq/zmqnotificationinterface.h @@ -32,7 +32,7 @@ class CZMQNotificationInterface final : public CValidationInterface // CValidationInterface void TransactionAddedToMempool(const CTransactionRef& tx, int64_t nAcceptTime, uint64_t mempool_sequence) override; void TransactionRemovedFromMempool(const CTransactionRef& tx, MemPoolRemovalReason reason, uint64_t mempool_sequence) override; - void BlockConnected(ChainstateRole role, const std::shared_ptr& pblock, const CBlockIndex* pindexConnected) override; + void BlockConnected(const kernel::ChainstateRole& role, const std::shared_ptr& pblock, const CBlockIndex* pindexConnected) override; void BlockDisconnected(const std::shared_ptr& pblock, const CBlockIndex* pindexDisconnected) override; void UpdatedBlockTip(const CBlockIndex *pindexNew, const CBlockIndex *pindexFork, bool fInitialDownload) override; void NotifyChainLock(const CBlockIndex *pindex, const std::shared_ptr& clsig) override; diff --git a/test/functional/feature_assumeutxo.py b/test/functional/feature_assumeutxo.py index 8649d64d28f9..86ba1ee6d93b 100755 --- a/test/functional/feature_assumeutxo.py +++ b/test/functional/feature_assumeutxo.py @@ -9,6 +9,10 @@ The assumeutxo value generated and used here is committed to in `CRegTestParams::m_assumeutxo_data` in `src/chainparams.cpp`. """ +import contextlib +import time +from shutil import rmtree + from dataclasses import dataclass from decimal import Decimal @@ -16,13 +20,27 @@ create_block, create_coinbase, ) -from test_framework.messages import tx_from_hex +from test_framework.compressor import ( + compress_amount, +) +from test_framework.messages import ( + CBlockHeader, + from_hex, + msg_headers, + tx_from_hex, + ser_varint, + MAX_MONEY, +) +from test_framework.p2p import ( + P2PInterface, +) from test_framework.test_framework import BitcoinTestFramework from test_framework.governance import EXPECTED_STDERR_NO_GOV_PRUNE from test_framework.util import ( assert_equal, assert_raises_rpc_error, sha256sum_file, + try_rpc, ) from test_framework.wallet import ( getnewdestination, @@ -66,38 +84,76 @@ def test_invalid_snapshot_scenarios(self, valid_snapshot_path): def expected_error(log_msg="", rpc_details=""): with node.assert_debug_log([log_msg]): assert_raises_rpc_error(-32603, f"Unable to load UTXO snapshot{rpc_details}", node.loadtxoutset, bad_snapshot_path) + + parsing_error_code = -22 + + self.log.info(" - snapshot file with invalid file magic") + with open(bad_snapshot_path, 'wb') as f: + f.write(b'bad!!' + valid_snapshot_contents[5:]) + assert_raises_rpc_error(parsing_error_code, "Unable to parse metadata: Invalid UTXO set snapshot magic bytes. Please check if this is indeed a snapshot file or if you are using an outdated snapshot format.", node.loadtxoutset, bad_snapshot_path) + + self.log.info(" - legacy post-M5 snapshot file without container framing") + with open(bad_snapshot_path, 'wb') as f: + f.write(valid_snapshot_contents[11:]) + assert_raises_rpc_error(parsing_error_code, "Unable to parse metadata: Invalid UTXO set snapshot magic bytes. Please check if this is indeed a snapshot file or if you are using an outdated snapshot format.", node.loadtxoutset, bad_snapshot_path) + + self.log.info(" - snapshot file with unsupported version") + for version in [0, 1, 3]: + with open(bad_snapshot_path, 'wb') as f: + f.write(valid_snapshot_contents[:5] + version.to_bytes(2, "little") + valid_snapshot_contents[7:]) + assert_raises_rpc_error(parsing_error_code, f"Unable to parse metadata: Version of snapshot {version} does not match any of the supported versions.", node.loadtxoutset, bad_snapshot_path) + + self.log.info(" - snapshot file with mismatching Dash network magic") + for magic in [bytes.fromhex('bf0c6bbd'), bytes.fromhex('cee2caff'), bytes.fromhex('e2caffce'), b'\x00' * 4]: + with open(bad_snapshot_path, 'wb') as f: + f.write(valid_snapshot_contents[:7] + magic + valid_snapshot_contents[11:]) + assert_raises_rpc_error( + parsing_error_code, + f"Unable to parse metadata: The network magic of the snapshot ({magic.hex()}) does not match the network magic of this node (fcc1b7dc).", + node.loadtxoutset, + bad_snapshot_path, + ) + prev_block_hash = self.nodes[0].getblockhash(SNAPSHOT_BASE_HEIGHT - 1) bogus_block_hash = "0" * 64 # Represents any unknown block hash for bad_block_hash in [bogus_block_hash, prev_block_hash]: with open(bad_snapshot_path, 'wb') as f: - # block hash of the snapshot base is stored right at the start (first 32 bytes) - f.write(bytes.fromhex(bad_block_hash)[::-1] + valid_snapshot_contents[32:]) - error_details = f", assumeutxo block hash in snapshot metadata not recognized ({bad_block_hash})" + f.write(valid_snapshot_contents[:11] + bytes.fromhex(bad_block_hash)[::-1] + valid_snapshot_contents[43:]) + error_details = f", assumeutxo block hash in snapshot metadata not recognized (hash: {bad_block_hash}). The following snapshot heights are available: 110, 200, 299." expected_error(rpc_details=error_details) self.log.info(" - snapshot file with wrong number of coins") - valid_num_coins = int.from_bytes(valid_snapshot_contents[32:32 + 8], "little") + valid_num_coins = int.from_bytes(valid_snapshot_contents[43:43 + 8], "little") for off in [-1, +1]: with open(bad_snapshot_path, 'wb') as f: - f.write(valid_snapshot_contents[:32]) + f.write(valid_snapshot_contents[:43]) f.write((valid_num_coins + off).to_bytes(8, "little")) - f.write(valid_snapshot_contents[32 + 8:]) + f.write(valid_snapshot_contents[43 + 8:]) expected_error(log_msg=f"bad evo section marker (or coins left over) after 298 coins" if off == -1 else f"bad snapshot format or truncated snapshot after deserializing 299 coins") self.log.info(" - snapshot file with alternated UTXO data") cases = [ - [b"\xff" * 32, 0, "f7fcf56f2db4e9d1584d7fced850ec5afe26b5c2e01dc62be3dc22c10705f030"], # wrong outpoint hash - [(1).to_bytes(4, "little"), 32, "611366963fd1fe4cfcf3e590502e2436f28f8080c5b9e28b93867028ae8d8eb2"], # wrong outpoint index - [b"\x82", 36, "4a761d666cd80fdb8855946310f249edc702e7c3ab338c0b2305e4d4f033057f"], # wrong coin code VARINT((coinbase ? 1 : 0) | (height << 1)) - [b"\x83", 36, "f81e8792f2e6f6da5843f9250dc39809a1294847e9a371c7c4471f110f260c74"], # another wrong coin code + # (content, offset, wrong_hash, custom_message) + [b"\xff" * 32, 0, "f7fcf56f2db4e9d1584d7fced850ec5afe26b5c2e01dc62be3dc22c10705f030", None], # wrong outpoint hash + [(1).to_bytes(4, "little"), 32, "611366963fd1fe4cfcf3e590502e2436f28f8080c5b9e28b93867028ae8d8eb2", None], # wrong outpoint index + [b"\x82", 36, "4a761d666cd80fdb8855946310f249edc702e7c3ab338c0b2305e4d4f033057f", None], # wrong coin code VARINT((coinbase ? 1 : 0) | (height << 1)) + [b"\x83", 36, "f81e8792f2e6f6da5843f9250dc39809a1294847e9a371c7c4471f110f260c74", None], # another wrong coin code + [ + # compressed txout value + scriptpubkey + ser_varint(compress_amount(MAX_MONEY + 1)) + ser_varint(0), + # outpoint txid + index + two-byte coin height/coinbase code + 32 + 4 + 2, + None, + "bad snapshot data after deserializing 0 coins - bad tx out value" + ], # Amount exceeds MAX_MONEY ] - for content, offset, wrong_hash in cases: + for content, offset, wrong_hash, custom_message in cases: with open(bad_snapshot_path, "wb") as f: - f.write(valid_snapshot_contents[:(32 + 8 + offset)]) + f.write(valid_snapshot_contents[:(5 + 2 + 4 + 32 + 8 + offset)]) f.write(content) - f.write(valid_snapshot_contents[(32 + 8 + offset + len(content)):]) - expected_error(log_msg=f"[snapshot] bad snapshot content hash: expected f6571ed786c40dcbb835b38090eaca87762cf421874461caa779738c7ff602fa, got {wrong_hash}") + f.write(valid_snapshot_contents[(5 + 2 + 4 + 32 + 8 + offset + len(content)):]) + expected_error(log_msg=custom_message or f"[snapshot] bad snapshot content hash: expected f6571ed786c40dcbb835b38090eaca87762cf421874461caa779738c7ff602fa, got {wrong_hash}") def test_invalid_chainstate_scenarios(self, node_index): self.log.info("Test different scenarios of invalid snapshot chainstate in datadir") @@ -219,6 +275,114 @@ def test_snapshot_not_on_most_work_chain(self, dump_output_path): node1.submitheader(main_block1) node1.submitheader(main_block2) + def test_sync_from_assumeutxo_node(self, snapshot): + """ + This test verifies that: + 1. An IBD node can sync headers from an AssumeUTXO node at any time. + 2. IBD nodes do not request historical blocks from AssumeUTXO nodes while they are syncing the background-chain. + 3. The assumeUTXO node dynamically adjusts the network services it offers according to its state. + 4. IBD nodes can fully sync from AssumeUTXO nodes after they finish the background-chain sync. + """ + self.log.info("Testing IBD-sync from assumeUTXO node") + # Node2 starts clean and loads the snapshot. + # Node3 starts clean and seeks to sync-up from snapshot_node. + miner = self.nodes[0] + snapshot_node = self.nodes[2] + ibd_node = self.nodes[3] + + # Start test fresh by cleaning up node directories + for node in (snapshot_node, ibd_node): + self.stop_node(node.index) + rmtree(node.chain_path) + self.start_node(node.index, extra_args=self.extra_args[node.index]) + + # Sync-up headers chain on snapshot_node to load snapshot + headers_provider_conn = snapshot_node.add_p2p_connection(P2PInterface()) + headers_provider_conn.wait_for_getheaders() + msg = msg_headers() + for block_num in range(1, miner.getblockcount()+1): + msg.headers.append(from_hex(CBlockHeader(), miner.getblockheader(miner.getblockhash(block_num), verbose=False))) + headers_provider_conn.send_message(msg) + + # Ensure headers arrived + default_value = {'status': ''} # No status + headers_tip_hash = miner.getbestblockhash() + self.wait_until(lambda: next(filter(lambda x: x['hash'] == headers_tip_hash, snapshot_node.getchaintips()), default_value)['status'] == "headers-only") + snapshot_node.disconnect_p2ps() + + # Load snapshot + snapshot_node.loadtxoutset(snapshot['path']) + + # Connect nodes and verify the ibd_node can sync-up the headers-chain from the snapshot_node + self.connect_nodes(ibd_node.index, snapshot_node.index) + snapshot_block_hash = snapshot['base_hash'] + self.wait_until(lambda: next(filter(lambda x: x['hash'] == snapshot_block_hash, ibd_node.getchaintips()), default_value)['status'] == "headers-only") + + # Once the headers-chain is synced, the ibd_node must avoid requesting historical blocks from the snapshot_node. + # If it does request such blocks, the snapshot_node will ignore requests it cannot fulfill, causing the ibd_node + # to stall. This stall could last for up to 10 min, ultimately resulting in an abrupt disconnection due to the + # ibd_node's perceived unresponsiveness. + time.sleep(3) # Sleep here because we can't detect when a node avoids requesting blocks from other peer. + assert_equal(len(ibd_node.getpeerinfo()[0]['inflight']), 0) + + # Now disconnect nodes and finish background chain sync + self.disconnect_nodes(ibd_node.index, snapshot_node.index) + self.connect_nodes(snapshot_node.index, miner.index) + self.sync_blocks(nodes=(miner, snapshot_node)) + # Check the base snapshot block was stored and ensure node signals full-node service support + self.wait_until(lambda: not try_rpc(-1, "Block not found", snapshot_node.getblock, snapshot_block_hash)) + self.wait_until(lambda: 'NETWORK' in snapshot_node.getnetworkinfo()['localservicesnames']) + + # Now that the snapshot_node is synced, verify the ibd_node can sync from it + self.connect_nodes(snapshot_node.index, ibd_node.index) + assert 'NETWORK' in ibd_node.getpeerinfo()[0]['servicesnames'] + self.sync_blocks(nodes=(ibd_node, snapshot_node)) + + def test_sync_to_most_work_chain_after_background_validation(self): + """ + After background validation completes, node should be able + to download and process blocks from peers without the snapshot block in their chain. + """ + self.log.info("Testing sync to the most-work chain without the snapshot block after background validation") + + forking_node = self.nodes[0] + snapshot_node = self.nodes[2] # Has already completed background validation + + self.log.info("Forking node switches to an alternative chain that forks one block before the snapshot block") + fork_point = SNAPSHOT_BASE_HEIGHT - 1 + forking_node_old_height = forking_node.getblockcount() + forking_node_old_chainwork = int(forking_node.getblockchaininfo()['chainwork'], 16) + forking_node.invalidateblock(forking_node.getblockhash(fork_point + 1)) + + self.log.info("Mine one more block than original chain to make the new chain have most work") + self.generate(forking_node, nblocks=(forking_node_old_height - fork_point) + 1, sync_fun=self.no_op) + assert int(forking_node.getblockchaininfo()['chainwork'], 16) > forking_node_old_chainwork + + self.log.info("Snapshot node should reorg to the most-work chain without the snapshot block") + self.sync_blocks(nodes=(snapshot_node, forking_node)) + + def assert_only_network_limited_service(self, node): + node_services = node.getnetworkinfo()['localservicesnames'] + assert 'NETWORK' not in node_services + assert 'NETWORK_LIMITED' in node_services + + @contextlib.contextmanager + def assert_disk_cleanup(self, node, assumeutxo_used): + """ + Ensure an assumeutxo node is cleaning up the background chainstate + """ + msg = [] + snapshot_path = node.chain_path / "chainstate_snapshot" + if assumeutxo_used: + # Check that the snapshot actually existed before restart + assert snapshot_path.exists() + msg = ["cleaning up unneeded background chainstate"] + + with node.assert_debug_log(msg): + yield + + assert not snapshot_path.exists() + def run_test(self): """ Bring up two (disconnected) nodes, mine some new blocks on the first, @@ -353,12 +517,6 @@ def check_dump_output(output): dump_output5 = n0.dumptxoutset('utxos5.dat', rollback=prev_snap_hash) assert_equal(sha256sum_file(dump_output4['path']), sha256sum_file(dump_output5['path'])) - # TODO: This is a hack to set m_best_header to the correct value after - # dumptxoutset/reconsiderblock. Otherwise the wrong error messages are - # returned in following tests. It can be removed once this bug is - # fixed. See also https://github.com/bitcoin/bitcoin/issues/26245 - self.restart_node(0, ["-reindex"]) - # Ensure n0 is back at the tip assert_equal(n0.getblockchaininfo()["blocks"], FINAL_HEIGHT) @@ -368,6 +526,9 @@ def check_dump_output(output): self.test_snapshot_block_invalidated(dump_output['path']) self.test_snapshot_not_on_most_work_chain(dump_output['path']) + # Prune-node sanity check + assert 'NETWORK' not in n1.getnetworkinfo()['localservicesnames'] + self.log.info(f"Loading snapshot into second node from {dump_output['path']}") # This node's tip is on an ancestor block of the snapshot, which should # be the normal case @@ -375,6 +536,35 @@ def check_dump_output(output): assert_equal(loaded['coins_loaded'], SNAPSHOT_BASE_HEIGHT) assert_equal(loaded['base_height'], SNAPSHOT_BASE_HEIGHT) + self.log.info("Confirm that local services remain unchanged") + # Since n1 is a pruned node, the 'NETWORK' service flag must always be unset. + self.assert_only_network_limited_service(n1) + + self.log.info("Check that UTXO-querying RPCs operate on snapshot chainstate") + snapshot_hash = loaded['tip_hash'] + snapshot_num_coins = loaded['coins_loaded'] + # coinstatsindex might be not caught up yet and is not relevant for this test, so don't use it + utxo_info = n1.gettxoutsetinfo(use_index=False) + assert_equal(utxo_info['txouts'], snapshot_num_coins) + assert_equal(utxo_info['height'], SNAPSHOT_BASE_HEIGHT) + assert_equal(utxo_info['bestblock'], snapshot_hash) + + # find coinbase output at snapshot height on node0 and scan for it on node1, + # where the block is not available, but the snapshot was loaded successfully + coinbase_tx = n0.getblock(snapshot_hash, verbosity=2)['tx'][0] + assert_raises_rpc_error(-1, "Block not found on disk", n1.getblock, snapshot_hash) + coinbase_output_descriptor = coinbase_tx['vout'][0]['scriptPubKey']['desc'] + scan_result = n1.scantxoutset('start', [coinbase_output_descriptor]) + assert_equal(scan_result['success'], True) + assert_equal(scan_result['txouts'], snapshot_num_coins) + assert_equal(scan_result['height'], SNAPSHOT_BASE_HEIGHT) + assert_equal(scan_result['bestblock'], snapshot_hash) + scan_utxos = [(coin['txid'], coin['vout']) for coin in scan_result['unspents']] + assert (coinbase_tx['txid'], 0) in scan_utxos + + txout_result = n1.gettxout(coinbase_tx['txid'], 0) + assert_equal(txout_result['scriptPubKey']['desc'], coinbase_output_descriptor) + def check_tx_counts(final: bool) -> None: """Check nTx and nChainTx intermediate values right after loading the snapshot, and final values after the snapshot is validated.""" @@ -442,6 +632,9 @@ def check_tx_counts(final: bool) -> None: f"-stopatheight={PAUSE_HEIGHT}", *self.extra_args[1]], expected_stderr=EXPECTED_STDERR_NO_GOV_PRUNE) + # Upon restart during snapshot tip sync, the node must remain in 'limited' mode. + self.assert_only_network_limited_service(n1) + # Finally connect the nodes and let them sync. Avoid a race between # connection assertions and -stopatheight tripping. self.connect_nodes(0, 1, wait_for_connect=False) @@ -456,6 +649,9 @@ def check_tx_counts(final: bool) -> None: self.log.info("Restarted node before snapshot validation completed, reloading...") self.restart_node(1, extra_args=self.extra_args[1], expected_stderr=EXPECTED_STDERR_NO_GOV_PRUNE) + # Upon restart, the node must remain in 'limited' mode + self.assert_only_network_limited_service(n1) + # Send snapshot block to n1 out of order. This makes the test less # realistic because normally the snapshot block is one of the last # blocks downloaded, but its useful to test because it triggers more @@ -481,6 +677,10 @@ def check_for_final_height(): self.log.info("Ensuring background validation completes") self.wait_until(lambda: len(n1.getchainstates()['chainstates']) == 1) + # Since n1 is a pruned node, it will not signal NODE_NETWORK after + # completing the background sync. + self.assert_only_network_limited_service(n1) + # Ensure indexes have synced. completed_idx_state = { 'basic block filter index': COMPLETE_IDX, @@ -494,8 +694,9 @@ def check_for_final_height(): for i in (0, 1): n = self.nodes[i] self.log.info(f"Restarting node {i} to ensure (Check|Load)BlockIndex passes") - self.restart_node(i, extra_args=self.extra_args[i], - expected_stderr=EXPECTED_STDERR_NO_GOV_PRUNE if i == 1 else '') + with self.assert_disk_cleanup(n, i == 1): + self.restart_node(i, extra_args=self.extra_args[i], + expected_stderr=EXPECTED_STDERR_NO_GOV_PRUNE if i == 1 else '') assert_equal(n.getblockchaininfo()["blocks"], FINAL_HEIGHT) @@ -512,12 +713,18 @@ def check_for_final_height(): self.log.info("-- Testing all indexes + reindex") assert_equal(n2.getblockcount(), START_HEIGHT) + assert 'NETWORK' in n2.getnetworkinfo()['localservicesnames'] # sanity check self.log.info(f"Loading snapshot into third node from {dump_output['path']}") loaded = n2.loadtxoutset(dump_output['path']) assert_equal(loaded['coins_loaded'], SNAPSHOT_BASE_HEIGHT) assert_equal(loaded['base_height'], SNAPSHOT_BASE_HEIGHT) + # Even though n2 is a full node, it will unset the 'NETWORK' service flag during snapshot loading. + # This indicates other peers that the node will temporarily not provide historical blocks. + self.log.info("Check node2 updated the local services during snapshot load") + self.assert_only_network_limited_service(n2) + for reindex_arg in ['-reindex=1', '-reindex-chainstate=1']: self.log.info(f"Check that restarting with {reindex_arg} will delete the snapshot chainstate") reindex_args = [reindex_arg, *self.extra_args[2]] @@ -553,6 +760,11 @@ def check_for_final_height(): msg = "Unable to load UTXO snapshot: Can't activate a snapshot-based chainstate more than once" assert_raises_rpc_error(-32603, msg, n2.loadtxoutset, dump_output['path']) + # Upon restart, the node must stay in 'limited' mode until the background + # chain sync completes. + self.restart_node(2, extra_args=self.extra_args[2]) + self.assert_only_network_limited_service(n2) + self.connect_nodes(0, 2) self.wait_until(lambda: n2.getchainstates()['chainstates'][-1]['blocks'] == FINAL_HEIGHT) self.sync_blocks(nodes=(n0, n2)) @@ -560,6 +772,9 @@ def check_for_final_height(): self.log.info("Ensuring background validation completes") self.wait_until(lambda: len(n2.getchainstates()['chainstates']) == 1) + # Once background chain sync completes, the full node must start offering historical blocks again. + self.wait_until(lambda: {'NETWORK', 'NETWORK_LIMITED'}.issubset(n2.getnetworkinfo()['localservicesnames'])) + completed_idx_state = { 'basic block filter index': COMPLETE_IDX, 'coinstatsindex': COMPLETE_IDX, @@ -570,7 +785,8 @@ def check_for_final_height(): for i in (0, 2): n = self.nodes[i] self.log.info(f"Restarting node {i} to ensure (Check|Load)BlockIndex passes") - self.restart_node(i, extra_args=self.extra_args[i]) + with self.assert_disk_cleanup(n, i == 2): + self.restart_node(i, extra_args=self.extra_args[i]) assert_equal(n.getblockchaininfo()["blocks"], FINAL_HEIGHT) @@ -597,6 +813,11 @@ def check_for_final_height(): self.test_snapshot_in_a_divergent_chain(dump_output['path']) + # The following test cleans node2 and node3 chain directories. + self.test_sync_from_assumeutxo_node(snapshot=dump_output) + + self.test_sync_to_most_work_chain_after_background_validation() + @dataclass class Block: hash: str diff --git a/test/functional/rpc_dumptxoutset.py b/test/functional/rpc_dumptxoutset.py index d970b4a58e09..5d27cd2485ff 100755 --- a/test/functional/rpc_dumptxoutset.py +++ b/test/functional/rpc_dumptxoutset.py @@ -45,7 +45,7 @@ def run_test(self): # UTXO snapshot hash should be deterministic based on mocked time. assert_equal( sha256sum_file(str(expected_path)).hex(), - '3ee2d4e678f0bcb73e28648434e4b32b5f6ffa600625905ad18fae2a02f42b26') + '1224b65eebf21259930b1c37287ea91dadde2f9e9ae9fe83c4225520e9a7df3e') assert b'DASHEVO\x00' in expected_path.read_bytes() assert_equal( diff --git a/test/functional/test_framework/compressor.py b/test/functional/test_framework/compressor.py new file mode 100644 index 000000000000..1c30d749df55 --- /dev/null +++ b/test/functional/test_framework/compressor.py @@ -0,0 +1,58 @@ +#!/usr/bin/env python3 +# Copyright (c) 2025-present The Bitcoin Core developers +# Distributed under the MIT software license, see the accompanying +# file COPYING or http://www.opensource.org/licenses/mit-license.php. +"""Routines for compressing transaction output amounts and scripts.""" +import unittest + +from .messages import COIN + + +def compress_amount(n): + if n == 0: + return 0 + e = 0 + while ((n % 10) == 0) and (e < 9): + n //= 10 + e += 1 + if e < 9: + d = n % 10 + assert (d >= 1 and d <= 9) + n //= 10 + return 1 + (n*9 + d - 1)*10 + e + else: + return 1 + (n - 1)*10 + 9 + + +def decompress_amount(x): + if x == 0: + return 0 + x -= 1 + e = x % 10 + x //= 10 + n = 0 + if e < 9: + d = (x % 9) + 1 + x //= 9 + n = x * 10 + d + else: + n = x + 1 + while e > 0: + n *= 10 + e -= 1 + return n + + +class TestFrameworkCompressor(unittest.TestCase): + def test_amount_compress_decompress(self): + def check_amount(amount, expected_compressed): + self.assertEqual(compress_amount(amount), expected_compressed) + self.assertEqual(decompress_amount(expected_compressed), amount) + + # test cases from compress_tests.cpp:compress_amounts + check_amount(0, 0x0) + check_amount(1, 0x1) + check_amount(1000000, 0x7) + check_amount(COIN, 0x9) + check_amount(50*COIN, 0x32) + check_amount(21000000*COIN, 0x1406f40) diff --git a/test/functional/test_framework/messages.py b/test/functional/test_framework/messages.py index 5f66dc6af064..eb7ed79efc24 100755 --- a/test/functional/test_framework/messages.py +++ b/test/functional/test_framework/messages.py @@ -116,6 +116,26 @@ def deser_compact_size(f): return nit +def ser_varint(l): + r = b"" + while True: + r = bytes([(l & 0x7f) | (0x80 if len(r) > 0 else 0x00)]) + r + if l <= 0x7f: + return r + l = (l >> 7) - 1 + + +def deser_varint(f): + n = 0 + while True: + dat = f.read(1)[0] + n = (n << 7) | (dat & 0x7f) + if (dat & 0x80) > 0: + n += 1 + else: + return n + + def deser_string(f): nit = deser_compact_size(f) return f.read(nit) @@ -2824,3 +2844,20 @@ def check_addrv2(ip, net): check_addrv2("2bqghnldu6mcug4pikzprwhtjjnsyederctvci6klcwzepnjd46ikjyd.onion", CAddress.NET_TORV3) check_addrv2("255fhcp6ajvftnyo7bwz3an3t4a4brhopm3bamyh2iu5r3gnr2rq.b32.i2p", CAddress.NET_I2P) check_addrv2("fc32:17ea:e415:c3bf:9808:149d:b5a2:c9aa", CAddress.NET_CJDNS) + + def test_varint_encode_decode(self): + def check_varint(num, expected_encoding_hex): + expected_encoding = bytes.fromhex(expected_encoding_hex) + self.assertEqual(ser_varint(num), expected_encoding) + self.assertEqual(deser_varint(BytesIO(expected_encoding)), num) + + # test cases from serialize_tests.cpp:varint_bitpatterns + check_varint(0, "00") + check_varint(0x7f, "7f") + check_varint(0x80, "8000") + check_varint(0x1234, "a334") + check_varint(0xffff, "82fe7f") + check_varint(0x123456, "c7e756") + check_varint(0x80123456, "86ffc7e756") + check_varint(0xffffffff, "8efefefe7f") + check_varint(0xffffffffffffffff, "80fefefefefefefefe7f") diff --git a/test/functional/test_runner.py b/test/functional/test_runner.py index 60f6a0a387bf..c24c454b097c 100755 --- a/test/functional/test_runner.py +++ b/test/functional/test_runner.py @@ -76,6 +76,7 @@ "address", "crypto.bip324_cipher", "blocktools", + "compressor", "crypto.chacha20", "crypto.ellswift", "key", diff --git a/test/functional/wallet_assumeutxo.py b/test/functional/wallet_assumeutxo.py index bb5bd4de34b9..d35d15cceba9 100755 --- a/test/functional/wallet_assumeutxo.py +++ b/test/functional/wallet_assumeutxo.py @@ -4,21 +4,20 @@ # file COPYING or http://www.opensource.org/licenses/mit-license.php. """Test for assumeutxo wallet related behavior. See feature_assumeutxo.py for background. - -## Possible test improvements - -- TODO: test import descriptors while background sync is in progress -- TODO: test loading a wallet (backup) on a pruned node - """ from decimal import Decimal +from test_framework.descriptors import descsum_create +from test_framework.governance import EXPECTED_STDERR_NO_GOV_PRUNE +from test_framework.messages import COIN from test_framework.test_framework import BitcoinTestFramework from test_framework.util import ( assert_equal, + assert_greater_than, assert_raises_rpc_error, ) -from test_framework.wallet import MiniWallet +from test_framework.wallet import address_to_scriptpubkey, MiniWallet +from test_framework.wallet_util import get_generate_key START_HEIGHT = 199 SNAPSHOT_BASE_HEIGHT = 299 @@ -34,29 +33,94 @@ def add_options(self, parser): def set_test_params(self): """Use the pregenerated, deterministic chain up to height 199.""" - self.num_nodes = 2 + self.num_nodes = 4 self.rpc_timeout = 120 self.extra_args = [ [], [], + [], + ["-fastprune", "-prune=1"], ] def setup_network(self): """Start with the nodes disconnected so that one can generate a snapshot including blocks the other hasn't yet seen.""" - self.add_nodes(2) + self.add_nodes(4) self.start_nodes(extra_args=self.extra_args) + def import_descriptor(self, node, wallet_name, key, timestamp): + import_request = [{"desc": descsum_create("pkh(" + key.pubkey + ")"), + "timestamp": timestamp, + "label": "Descriptor import test"}] + wrpc = node.get_wallet_rpc(wallet_name) + return wrpc.importdescriptors(import_request) + + def validate_snapshot_import(self, node, loaded, base_hash): + assert_equal(loaded['coins_loaded'], SNAPSHOT_BASE_HEIGHT) + assert_equal(loaded['base_height'], SNAPSHOT_BASE_HEIGHT) + + normal, snapshot = node.getchainstates()["chainstates"] + assert_equal(normal['blocks'], START_HEIGHT) + assert 'snapshot_blockhash' not in normal + assert_equal(normal['validated'], True) + assert_equal(snapshot['blocks'], SNAPSHOT_BASE_HEIGHT) + assert_equal(snapshot['snapshot_blockhash'], base_hash) + assert_equal(snapshot['validated'], False) + + assert_equal(node.getblockchaininfo()["blocks"], SNAPSHOT_BASE_HEIGHT) + + def complete_background_validation(self, node): + self.connect_nodes(0, node.index) + + # Ensuring snapshot chain syncs to tip + self.wait_until(lambda: node.getchainstates()['chainstates'][-1]['blocks'] == FINAL_HEIGHT) + self.sync_blocks(nodes=(self.nodes[0], node)) + + # Ensuring background validation completes + self.wait_until(lambda: len(node.getchainstates()['chainstates']) == 1) + + def test_backup_during_background_sync_pruned_node(self, n3, dump_output, expected_error_message): + self.log.info("Backup from the snapshot height can be loaded during background sync (pruned node)") + loaded = n3.loadtxoutset(dump_output['path']) + assert_greater_than(n3.pruneblockchain(START_HEIGHT), 0) + self.validate_snapshot_import(n3, loaded, dump_output['base_hash']) + n3.restorewallet("w", "backup_w.dat") + # Balance of w wallet is still 0 because n3 has not synced yet + assert_equal(n3.getbalance(), 0) + + self.log.info("Backup from before the snapshot height can't be loaded during background sync (pruned node)") + assert_raises_rpc_error(-4, expected_error_message, n3.restorewallet, "w2", "backup_w2.dat") + + def test_restore_wallet_pruneheight(self, n3): + self.log.info("Ensuring wallet can't be restored from a backup that was created before the pruneheight (pruned node)") + self.complete_background_validation(n3) + # After background sync, pruneheight is reset to 0, so mine 500 blocks + # and prune the chain again + self.generate(n3, nblocks=500, sync_fun=self.no_op) + assert_equal(n3.pruneblockchain(FINAL_HEIGHT), 298) # 298 is the height of the last block pruned (pruneheight 299) + error_message = "Wallet loading failed. Prune: last wallet synchronisation goes beyond pruned data. You need to -reindex (download the whole blockchain again in case of pruned node)" + # This backup (backup_w2.dat) was created at height 199, so it can't be restored in a node with a pruneheight of 299 + assert_raises_rpc_error(-4, error_message, n3.restorewallet, "w2", "backup_w2.dat") + + self.log.info("Ensuring wallet can be restored from a backup that was created at the pruneheight (pruned node)") + # This backup (backup_w.dat) was created at height 299, so it can be restored in a node with a pruneheight of 299 + n3.restorewallet("w_alt", "backup_w.dat") + # Check balance of w_alt wallet + w_alt = n3.get_wallet_rpc("w_alt") + assert_equal(w_alt.getbalance(), 34) + def run_test(self): """ - Bring up two (disconnected) nodes, mine some new blocks on the first, - and generate a UTXO snapshot. - - Load the snapshot into the second, ensure it syncs to tip and completes - background validation when connected to the first. + Bring up four (disconnected) nodes: + - n0: mine some blocks and create a UTXO snapshot + - n1: load the snapshot and test loading a wallet backup and descriptors during and after background sync + - n2: load the snapshot and check the wallet balance during background sync + - n3: load the snapshot, prune the chain, and test loading a wallet backup during and after background sync """ n0 = self.nodes[0] n1 = self.nodes[1] + n2 = self.nodes[2] + n3 = self.nodes[3] self.mini_wallet = MiniWallet(n0) @@ -90,6 +154,13 @@ def run_test(self): n0.createwallet('w') w = n0.get_wallet_rpc("w") + w_address = w.getnewaddress() + + # Create a second backup before the snapshot height. + n0.createwallet('w2') + w2 = n0.get_wallet_rpc("w2") + w2_address = w2.getnewaddress() + w2.backupwallet("backup_w2.dat") # Generate a series of blocks that `n0` will have in the snapshot, # but that n1 doesn't yet see. In order for the snapshot to activate, @@ -104,14 +175,13 @@ def run_test(self): # make n1 aware of the new header, but don't give it the block. n1.submitheader(newblock) - + n2.submitheader(newblock) + n3.submitheader(newblock) # Ensure everyone is seeing the same headers. for n in self.nodes: assert_equal(n.getblockchaininfo()[ "headers"], SNAPSHOT_BASE_HEIGHT) - w.backupwallet("backup_w.dat") - self.log.info("-- Testing assumeutxo") assert_equal(n0.getblockcount(), SNAPSHOT_BASE_HEIGHT) @@ -121,6 +191,10 @@ def run_test(self): f"Creating a UTXO snapshot at height {SNAPSHOT_BASE_HEIGHT}") dump_output = n0.dumptxoutset('utxos.dat', "latest") + # dumptxoutset flushes chainstate notifications, ensuring Dash's wallet + # best-block locator is durable at the snapshot height before backup. + w.backupwallet("backup_w.dat") + assert_equal( dump_output['txoutset_hash'], 'f6571ed786c40dcbb835b38090eaca87762cf421874461caa779738c7ff602fa') @@ -130,7 +204,13 @@ def run_test(self): # Mine more blocks on top of the snapshot that n1 hasn't yet seen. This # will allow us to test n1's sync-to-tip on top of a snapshot. - self.generate(n0, nblocks=100, sync_fun=self.no_op) + w_skp = address_to_scriptpubkey(w_address) + w2_skp = address_to_scriptpubkey(w2_address) + for i in range(100): + if i % 3 == 0: + self.mini_wallet.send_to(from_node=n0, scriptPubKey=w_skp, amount=COIN) + self.mini_wallet.send_to(from_node=n0, scriptPubKey=w2_skp, amount=10 * COIN) + self.generate(n0, nblocks=1, sync_fun=self.no_op) assert_equal(n0.getblockcount(), FINAL_HEIGHT) assert_equal(n1.getblockcount(), START_HEIGHT) @@ -140,21 +220,33 @@ def run_test(self): self.log.info( f"Loading snapshot into second node from {dump_output['path']}") loaded = n1.loadtxoutset(dump_output['path']) - assert_equal(loaded['coins_loaded'], SNAPSHOT_BASE_HEIGHT) - assert_equal(loaded['base_height'], SNAPSHOT_BASE_HEIGHT) + self.validate_snapshot_import(n1, loaded, dump_output['base_hash']) - normal, snapshot = n1.getchainstates()["chainstates"] - assert_equal(normal['blocks'], START_HEIGHT) - assert_equal(normal.get('snapshot_blockhash'), None) - assert_equal(normal['validated'], True) - assert_equal(snapshot['blocks'], SNAPSHOT_BASE_HEIGHT) - assert_equal(snapshot['snapshot_blockhash'], dump_output['base_hash']) - assert_equal(snapshot['validated'], False) - - assert_equal(n1.getblockchaininfo()["blocks"], SNAPSHOT_BASE_HEIGHT) - - self.log.info("Backup can't be loaded during background sync") - assert_raises_rpc_error(-4, "Wallet loading failed. Error loading wallet. Wallet requires blocks to be downloaded, and software does not currently support loading wallets while blocks are being downloaded out of order when using assumeutxo snapshots. Wallet should be able to load successfully after node sync reaches height 299", n1.restorewallet, "w", "backup_w.dat") + self.log.info("Backup from the snapshot height can be loaded during background sync") + n1.restorewallet("w", "backup_w.dat") + # Balance of w wallet is still 0 because n1 has not synced yet + assert_equal(n1.getbalance(), 0) + + self.log.info("Backup from before the snapshot height can't be loaded during background sync") + expected_error_message = "Wallet loading failed. Error loading wallet. Wallet requires blocks to be downloaded, and software does not currently support loading wallets while blocks are being downloaded out of order when using assumeutxo snapshots. Wallet should be able to load successfully after node sync reaches height 299" + assert_raises_rpc_error(-4, expected_error_message, n1.restorewallet, "w2", "backup_w2.dat") + + self.test_backup_during_background_sync_pruned_node(n3, dump_output, expected_error_message) + + self.log.info("Test loading descriptors during background sync") + wallet_name = "w1" + n1.createwallet(wallet_name, disable_private_keys=True) + key = get_generate_key() + time = n1.getblockchaininfo()['time'] + timestamp = 0 + expected_error_message = f"Rescan failed for descriptor with timestamp {timestamp}. There was an error reading a block from time {time}, which is after or within 7200 seconds of key creation, and could contain transactions pertaining to the desc. As a result, transactions and coins using this desc may not appear in the wallet. This error is likely caused by an in-progress assumeutxo background sync. Check logs or getchainstates RPC for assumeutxo background sync progress and try again later." + result = self.import_descriptor(n1, wallet_name, key, timestamp) + assert_equal(result[0]['error']['code'], -1) + assert_equal(result[0]['error']['message'], expected_error_message) + + self.log.info("Test that rescanning blocks from before the snapshot fails when blocks are not available from the background sync yet") + w1 = n1.get_wallet_rpc(wallet_name) + assert_raises_rpc_error(-1, "Failed to rescan unavailable blocks likely due to an in-progress assumeutxo background sync. Check logs or getchainstates RPC for assumeutxo background sync progress and try again later.", w1.rescanblockchain, 100) PAUSE_HEIGHT = FINAL_HEIGHT - 40 @@ -175,20 +267,29 @@ def run_test(self): self.restart_node(1, extra_args=self.extra_args[1]) # TODO: inspect state of e.g. the wallet before reconnecting - self.connect_nodes(0, 1) + self.complete_background_validation(n1) - self.log.info( - f"Ensuring snapshot chain syncs to tip. ({FINAL_HEIGHT})") - self.wait_until(lambda: n1.getchainstates()[ - 'chainstates'][-1]['blocks'] == FINAL_HEIGHT) - self.sync_blocks(nodes=(n0, n1)) + self.log.info("Ensuring wallet can be restored from a backup created before the snapshot height") + n1.restorewallet("w2", "backup_w2.dat") + assert_equal(n1.getbalance(), 340) - self.log.info("Ensuring background validation completes") - self.wait_until(lambda: len(n1.getchainstates()['chainstates']) == 1) + n1.loadwallet("w") + assert_equal(n1.get_wallet_rpc("w").getbalance(), 34) - self.log.info("Ensuring wallet can be restored from backup") - n1.restorewallet("w", "backup_w.dat") + self.log.info("Check balance of a wallet active during snapshot completion") + n2.restorewallet("w", "backup_w.dat") + n2.loadtxoutset(dump_output['path']) + self.connect_nodes(0, 2) + self.wait_until(lambda: len(n2.getchainstates()['chainstates']) == 1) + assert_equal(n2.getbalance(), 34) + + self.log.info("Ensuring descriptors can be loaded after background sync") + n1.loadwallet(wallet_name) + result = self.import_descriptor(n1, wallet_name, key, timestamp) + assert_equal(result[0]['success'], True) + self.test_restore_wallet_pruneheight(n3) + self.stop_node(3, expected_stderr=EXPECTED_STDERR_NO_GOV_PRUNE) if __name__ == '__main__': AssumeutxoTest().main()