execution, db: bind cache views to state versions and file views

[yperbasis]

Fixes #22463.
Fixes #23028.

This supersedes #23005 while retaining its regression scenarios. It also folds #23047 into the same cache-publication model instead of stacking a second coherence mechanism on top. #23139 describes a possible selective-invalidation optimization for canonical unwinds; it is not required for this correctness fix.

Problem

StateCache and BranchCache are process-global caches of latest state. A transaction may outlive a canonical commit, unwind, or immutable-files publication, so the caches must reject reads and fills from transactions backed by older state.

The previous coherence mechanisms did not identify that complete backing state:

• A transaction that outlives an unwind keeps reading the discarded fork from its own snapshot, and stored-entry invalidation cannot stop it from refilling a dead-fork value afterwards — admission has to reject the reader (execution/cache: stale-fill admission is one-sided — pre-unwind read views refill dead-fork values, and unwound keys bypass the flush cache-apply #22463). The same fill hole exists while an unwind is staged but not yet committed, including when that commit later fails.
• Cache entries do not map one-to-one onto unwind-diff keys: derived address-to-code-hash bindings, branch-cache tiers, and adaptive-pin state outlive the domain writes that produced them, so stored-entry invalidation cannot rely on diff completeness.
• Snapshot downloads can extend the visible domain files without advancing PlainStateVersion and without sending the downloaded values through cache applies. Existing positive or negative entries could remain live after their backing files changed.

Review guide

A useful end-state reading order is:

1. execution/cache/generation_gate.go: generation identity, view revocation, fill admission, commit publication, abort, and files publication.
2. execution/cache/view.go and execution/commitment/branch_cache_view.go: how StateCache and BranchCache expose generation-bound reads, fills, and publication capabilities.
3. db/state/execctx/domain_shared.go: deriving both cache views from a pinned transaction and publishing them after a canonical database commit.
4. db/state/aggregator.go, execution/cache/state_cache.go, and execution/commitment/branch_cache.go: coordinating cache identity with immutable-files visibility and file-provenance coverage.
5. execution/stagedsync/rawdbreset/reset_stages.go and cmd/integration/commands/stages.go: whole-state reset and staged-unwind publication boundaries.
6. The focused regression tests listed below. The remaining production diff is mainly API wiring and removal of the superseded coherence mechanisms.

Cache generation

Both caches now follow one invariant:

A shared cache represents one durable PlainStateVersion over one compatible immutable-files view. A transaction can use the cache only while that exact generation remains published.

A StateCache generation contains the state version and the exclusive values-file ends for accounts, storage, and code. A BranchCache generation contains the state version and the commitment values-file end.

HasExactDomainVisibleEnd decides whether the transaction has an exact read frontier for every relevant domain. The numeric DomainVisibleEnd values are not used as file identity: they describe the combined database-and-history frontier, can remain unchanged when pinned values files change, and require database cursors to resolve. TxNumsInFiles provides the cheap values-file ends that identify the relevant immutable backing. Equal ends remain compatible when physical files are merged or repacked.

Each published generation also has an immutable pointer token. Pointer identity lets publication revoke readers before a new durable generation exists and prevents a previously revoked view from becoming valid if the same numeric identity is published again.

Publication behavior

Boundary	Existing views	Cache entries	Published result
Forward commit	Revoked during commit	Retained; committed updates are applied	New state version
Failed commit	Temporarily revoked	Unchanged	Previous generation restored
Canonical unwind	Revoked	Both caches cleared after commit	Rewound state version
Speculative or local unwind	Detached from the rewound SharedDomains	Shared caches untouched	Existing generation unchanged
Covered files change	Revoked when relevant ends change	Retained	Same state version over new file ends
Uncovered files extension	Revoked	Affected cache cleared	Same state version over new file ends
ResetExec	Revoked	Both caches cleared	Unpublished until canonical execution establishes a generation

SharedDomains.Commit uses the following durable order for both caches:

1. Flush changes into the database transaction and collect cache updates.
2. Prepare adaptive branch-cache changes without mutating the shared cache.
3. Revoke the current cache generations and wait for admitted fills.
4. Commit the database transaction.
5. Apply the collected updates and publish generations with the committed state version.

Long flush and adaptive-planning work happens before revocation. If the database commit fails, publication restores the previous tokens and leaves entries unchanged.

ResetExec is the whole-state replacement path outside SharedDomains.Commit. It revokes and clears both caches before starting the replacement transaction. That transaction wipes the execution tables and advances PlainStateVersion atomically. The caches remain unpublished until canonical execution establishes a new generation, so a pre-reset view cannot match or refill post-reset state.

Why stale data cannot cross a publication

• View construction succeeds only when the transaction's state version and relevant file ends exactly match the published identity.
• A read checks its immutable token before and after the underlying lookup. A publication racing the lookup converts the result into a miss.
• A fill rechecks the token under the admission lock shared with publication. Publication therefore waits for already admitted fills, and an old transaction cannot fill the next generation.
• Cache entries are changed only after a successful database commit. Abort can safely restore the previous token because neither durable state nor cache entries changed.
• A new pointer token is allocated even when numeric identity repeats, so a revoked view cannot become live again accidentally.

Unwind behavior

An unwind detaches that SharedDomains from both shared-cache readers, so the rewound overlay cannot read from or fill either cache. The unwind itself does not clear or revoke the shared caches. If the SharedDomains has canonical publication authority, its later successful Commit revokes existing views and clears both caches as part of publishing the rewound state.

After a canonical unwind commits, both caches are cleared and published at the resulting state version. The full clear is intentional: an unwind diff is not a complete inventory of entries that may have entered either cache from the discarded fork. #23139 evaluates selective retention with change-diff tombstones, but keeps full clearing as the correctness fallback.

Files publication

Aggregator.recalcVisibleFiles is the common publication boundary for downloads, reopen, merge, and dependency recalculation. It reconciles both caches before the new immutable-files bundle becomes visible.

The caches track how far their committed updates cover each relevant domain:

• Files covered by committed updates retain their entries.
• An extension beyond that coverage cannot be proven compatible with the cache publication stream of this process, so the affected cache is cleared.
• Any change of the relevant file ends revokes existing views, even when entries can be retained.
• Publication remains blocked until the new files bundle and its matching cache generation become visible together.

A transaction pinned to the previous files cannot bind to the cache after this boundary. Binding a StateCache after files are already visible performs the same reconciliation. Visibility lowering remains forbidden while the shared caches are attached because file-provenance watermarks only advance.

Intentional behavior and non-goals

• An old transaction misses the shared caches after another transaction or file publication advances the generation. It reads its own database snapshot; it never receives newer cached state.
• Canonical unwind favors a small correctness proof over retaining cache warmth. Both caches are cleared even for a shallow reorg.
• The caches hold one latest generation. Multi-version snapshot caching remains the responsibility of kvcache.
• Physical file replacement with equal exclusive ends does not change identity; the visible latest state remains compatible.

Simplification

The shared generation rule replaces the StateCache read-view epoch and the per-entry unwind epochs, floors, and transaction numbers in both caches. The separate execution/cache/coherence package is removed.

TxNum remains on transient committed updates only to advance file-provenance watermarks. Cached state entries still keep their source Step, which callers need independently of cache coherence.

Performance

The cache-hit path remains lock-free and allocation-free. It adds an atomic pointer comparison before and after the existing lookup. A fill performs an early pointer check and one authoritative check under the admission lock.

Cache-view construction does not resolve numeric domain frontiers. It checks four cheap exactness conditions and four values-file ends from the transaction's pinned files. A transaction different from the SharedDomains base view also reads PlainStateVersion once per constructed getter. No database cursor is opened for cache eligibility.

On darwin/arm64 with an Apple M2 Max, the BenchmarkCacheGetterConstruction median over five 5,000-iteration runs is:

• exactness checks: 0.71 us/op;
• resolving four numeric visible ends: 3.50 us/op.

Avoiding the four LastKey cursors saves about 2.8 us per fresh cache-backed getter in this benchmark and makes construction about 4.9 times faster. Run it with:

go test ./db/state/execctx -run '^$' -bench '^BenchmarkCacheGetterConstruction$' -benchtime=5000x -count=5

Forward commits retain unchanged entries. Canonical unwinds and uncovered files extensions clear their affected caches and pay the re-warm cost.

Two workload-level costs remain to be measured:

• Long-lived parallel-execution workers and builders lose cache reads after another publication advances the generation.
• Canonical unwind clears both caches. Measure StateCache and BranchCache re-warm time and end-to-end FCU latency after shallow and deep reorgs.

Tests

The changes were developed with red-green-refactor regression tests. The main review paths are:

• execution/cache/cache_test.go and execution/cache/files_publication_test.go: generation matching, publication abort, stale read/fill rejection, canonical clear, and files reconciliation.
• execution/commitment/branch_cache_test.go, execution/commitment/branch_cache_absorb_test.go, and execution/commitment/adaptive_pin_test.go: the equivalent BranchCache contract, file provenance, and adaptive-plan publication.
• db/state/execctx/statecache_readfill_test.go, db/state/execctx/statecache_rpc_integration_test.go, and db/state/execctx/branch_cache_flush_test.go: speculative isolation, canonical unwind, bare-Flush memory precedence, old transactions, commit failure, and RPC-visible behavior.
• db/state/aggregator_align_test.go: atomic files visibility, old-files transactions, cache binding, and visibility-lowering protection.
• execution/stagedsync/rawdbreset/reset_stages_test.go and cmd/integration/commands/stages_test.go: whole-state reset and staged-unwind publication.

Together these cover pre-unwind reads and fills, transactions first attached after an unwind or files publication, negative entries, reset, publication abort, adaptive pins, and both process-global caches.

Local verification:

• go test ./execution/cache ./execution/commitment ./db/state/execctx ./execution/exec -count=1
• go test ./db/state -count=1
• go test -race --timeout 20m ./execution/cache ./execution/commitment ./db/state/execctx ./execution/exec ./db/state -count=1
• make lint repeatedly
• make erigon integration

[Copilot]

Pull request overview

This PR refactors execution/cache so StateCache is published and consumed strictly as a single durable-state snapshot keyed by PlainStateVersion (state version), replacing the prior mix of applied-frontier tracking plus per-entry epoch/floor coherence. The goal is to close stale-fill windows around unwinds/commits by making cache usability contingent on an exact state-version match, and by revoking read views before any publication mutates cache contents.

Changes:

• Introduce a version-bound cache.ReadView/publication token model (Publisher/Publication) and remove StateCache per-entry epoch/floor + applied-frontier coherence.
• Rewire canonical vs speculative cache ownership (SetCanonicalStateCache vs SetStateCacheReader) and update execution paths (FCU, SetHead, integration runner) accordingly.
• Update read-ahead warmup and tests/benchmarks to use state-version-bound cache views and new step-based cache APIs.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
execution/vm/contract.go	Update jumpdest cache usage to new Put signature.
execution/execmodule/set_head.go	Switch SetHead path to canonical cache publication API.
execution/execmodule/forkchoice.go	Switch FCU canonical contexts to canonical cache publication API; remove warmup draining.
execution/execmodule/exec_module.go	Remove read-ahead draining helper; use reader-only cache attachment in ValidateChain.
execution/exec/blocks_read_ahead.go	Bind read-ahead warmup fills to PlainStateVersion and domain visibility; fill using kv.Step.
execution/exec/blocks_read_ahead_test.go	Adapt warmup tests to version-bound ReadView and publisher initialization.
execution/cache/view.go	Redefine ReadView as a durable-state-version handle with generation checks around reads/fills.
execution/cache/state_cache.go	Implement generation token, Publisher/Publication, step-based updates, and version-gated View.
execution/cache/generic_cache.go	Remove unwind coherence from GenericCache; store (value, step) for domain caches.
execution/cache/generic_cache_concurrency_test.go	Update concurrency tests for new Put/PutIfAbsent signatures and semantics.
execution/cache/code_cache.go	Remove unwind coherence; make code layers generation-cleared; switch stamping from txNum/epoch to step where needed.
execution/cache/code_cache_concurrency_test.go	Update concurrency tests for new code-cache semantics (clear-fencing only).
execution/cache/code_cache_codehash_test.go	Replace unwind-based tests with clear/publication-based expectations; update APIs.
execution/cache/cache.go	Update Cache interface to step-based API and simplify package-level docs.
execution/cache/cache_test.go	Update StateCache/DomainCache/CodeCache tests to publisher-based publication model and step API.
db/state/execctx/statecache_rpc_integration_test.go	Add/adjust integration tests covering unwinds and RPC views under version-bound cache publication.
db/state/execctx/statecache_readfill_test.go	Rewrite read-fill/unwind tests to new inactive-during-publication behavior and version-bound views.
db/state/execctx/statecache_readfill_bench_test.go	Remove frontier memo benchmark; align benchmark description with generation-check model.
db/state/execctx/flush_storage_cache_test.go	Update storage-cache commit callback test to read via current version-bound cache view.
db/state/execctx/export_test.go	Adjust exported test helpers to use version-bound cache view wiring.
db/state/execctx/domain_visible_end_memo_test.go	Remove now-obsolete visible-end memo concurrency tests.
db/state/execctx/domain_shared.go	Replace frontier memo + applier model with version-bound view selection and canonical Publisher publication pipeline.
db/state/execctx/codehash_routing_test.go	Update derived codehash routing tests to new generation safety (cache-sourced record may seed mapping).
cmd/integration/commands/stages.go	Wire integration stage runner to canonical cache publication API.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 45 out of 45 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 45 out of 45 changed files in this pull request and generated no new comments.

Suppressed comments (2)

db/state/aggregator.go:1931

• When a commitment files publication triggers a BranchCache clear (BeginFilesPublication returns a non-nil backing-change), the AdaptivePinController should be reset as well; otherwise the controller will keep residency/miss state for pins that were just cleared and may make promotion/demotion decisions based on stale cache contents.

	// SharedDomains.Commit acquires cache publication in the same order.
	if domain := a.d[kv.CommitmentDomain]; domain != nil && domain.branchCache != nil {
		if commitmentVisible := visible.d[kv.CommitmentDomain]; commitmentVisible != nil {
			publication.branch = domain.branchCache.BeginFilesPublication(visibleFiles(commitmentVisible.files).EndTxNum())
		}
	}

db/state/aggregator.go:742

• closeDirtyFilesNoReopen resets the commitment BranchCache but does not reset the paired AdaptivePinController. Since AdaptivePinController keeps per-contract residency state, leaving it intact after a BranchCache reset can make the controller treat removed pins as still resident (see AdaptivePinController.Reset doc) and delay re-promotion/extension decisions.

This issue also appears on line 1926 of the same file.

	a.visibilityLoweringForbidden.Store(false)
	if cd := a.d[kv.CommitmentDomain]; cd != nil && cd.branchCache != nil {
		cd.branchCache.Reset()
	}

[Copilot]

Pull request overview

Copilot reviewed 53 out of 53 changed files in this pull request and generated 1 comment.