feat(replay): stake delegations - #1780
Draft
yewman wants to merge 7 commits into
Draft
Conversation
Root+delta model for the per-epoch delegator -> delegation table:
- StakeDelegation (64 B): { voter_pk, stake, activation_epoch,
deactivation_epoch, credits_observed } per delegator. credits_observed
travels with the delegation so partitioned-rewards paths dont need
a second accounts_db read at reward time.
- StakeDelegationsRoot: FixedPubkeyMap<StakeDelegation, 2^20>. Immutable
per-epoch base. Written once at boot (walking every stake account
through accounts_db) and rewritten in place at each epoch boundary
(folding the winning forks delta in). Not read during in-epoch
execution.
- StakeDelegationsDelta: per-fork accumulator of landed mutations,
positional entry array (delegator_pk + delegation + { upsert |
tombstone | unpopulated }). Capped at MAX_STAKE_DELEGATIONS_DELTA_PER_FORK
= 256 entries. Byte-copied at onBlockCreate from parent to child,
same pattern as LiveVoters. tombstone marks accounts closed on the
fork so the fold-into-root path can erase the corresponding key.
ReplayStakes grows two fields:
- stake_delegations_root: StakeDelegationsRoot (~96 MB)
- stake_delegations_delta: [BlockPool.capacity]StakeDelegationsDelta
(~26 MB)
Total ReplayStakes footprint now ~170 MB; well past the point where
splitting the wrapper into standalone top-level regions (per the
provisional-shape note on ReplayStakes) is worthwhile. Left as a
follow-up.
onBlockCreate now also memcpys the delta alongside live_voters,
using the same std.mem.asBytes trick to avoid the Debug-mode
struct-assign stack temporary.
Tests: StakeDelegationsDelta.reset; StakeDelegationsRoot insert /
getPtrConst round-trip on the page allocator (96 MB); onBlockCreate
byte-copies delta and post-clone mutation of the child leaves the
parent untouched.
Follow-ups (separate commits):
- Boot loader: iterate bank_fields.stakes_cache.stake_accounts,
batch-read each stake account through the AccountLookups ring,
decode StakeStateV2::Stake, populate root.
- Pure applyStakeIx primitive(s) that produce delta mutations.
- Fold-into-root + boundary derivation of next-epoch admitted set,
closes the epoch-boundary hard-stop.
Adds a delete primitive on top of the existing insert/getPtr shape. Tombstone slots use the all-0xFF pubkey as sentinel (distinct from the existing all-0x00 empty sentinel); probes skip tombstones without stopping, and inserts reuse the first tombstone seen in the probe run so churn doesnt monotonically grow probe length. Required by the stake-delegations root-advance fold path: closed stake accounts become tombstones on the delegator -> delegation map.
Replaces the per-fork positional StakeDelegationsDelta slab (256-entry
hard cap, 26 KB memcpy per onBlockCreate) with a shared inline arena
of StakeDeltaNodes referenced by per-block chain heads. Adds a
per-block StakeAggregates triple (effective, activating, deactivating)
maintained incrementally by the committer for the StakeHistory sysvar.
Data-structure changes:
- StakeDeltaNode: 104 B extern struct { delegator_pk, delegation,
kind: {upsert, tombstone}, next: Id.Optional }. next doubles as
chain-link when allocated and free-list link when freed (same
pattern as SharedPool.Node, but inline so it embeds into
ReplayStakes without a separate backing region).
- StakeDeltaArena: [MAX_STAKE_DELTA_NODES=131072]StakeDeltaNode +
free_head, all inline. ~13.6 MB. createId/destroyId are O(1)
free-list pop/push. OOM (error.OutOfSpace) is treated as fatal
per proposal §11.3; 131k nodes covers ~433 slots of continuous
peak churn (~40x historical worst).
- StakeAggregates: 24 B extern struct. One per block, byte-copied
parent->child at onBlockCreate.
- ReplayStakes now holds:
stake_delegations_root (unchanged, ~96 MB)
stake_delta_arena (~13.6 MB)
stake_delta_head[1024] (~4 KB)
stake_aggregates[1024] (~24 KB)
Down from ~170 MB to ~158 MB by dropping the [1024]Delta slab.
- onBlockCreate now copies live_voters + stake_aggregates and
resets stake_delta_head[child] to .null. Ancestor deltas are
reached by walking replay.Node.parent (block tree), not by copy.
Design rationale is in .todo/stakes-v2-proposal-v2.md.
No consumer wiring yet — fold-into-root, boundary derivation,
committer plumbing, and snapshot boot loader are follow-up phases.
The epoch-boundary hard-stop still applies.
Two pure committer primitives to push a stake-delegation mutation
onto the current blocks arena chain and update its running
(effective, activating, deactivating) triple.
foldStakeIxUpsert:
- takes delegator_pk + post StakeDelegation + pre/post
aggregate contributions (three-tuple each). Post-minus-pre is
what the block owes to StakeHistory going forward.
- allocs one arena node with kind=upsert, chains it at head of
stake_delta_head[block] so ancestor deltas remain reachable
via the block-tree parent walk.
- applyAggregateDelta uses wrapping u64 arithmetic on each of
the three columns; intermediate values wrap when a single ix
shrinks a delegation, but the cumulative running sum matches
the mathematically-correct total because it equals the sum
of *post* contributions across all deltas.
foldStakeIxTombstone:
- post state is implicitly zero (account no longer exists on
this fork). Aggregate delta is simply -pre_contribution.
- stashes delegation_at_close on the arena node for debug /
audit; the fold-into-root path (phase C) will treat it as
undefined and only read .kind.
Both propagate error.OutOfSpace from the arena; committer path
treats that as fatal per proposal §11.3.
Tests cover:
- upsert on empty block (aggregate + head + node fields)
- two upserts chain newest-first, ancestor reachable via .next
- tombstone subtracts pre-contribution correctly
- shrink case (pre.effective > post.effective) with wrapping
arithmetic produces the expected running total
Not yet called anywhere — the exec tile has no stake program.
The committer call site is stubbed until execution grows one.
Root-advance fold and losing-fork prune, closing the delta-arena lifecycle. Every arena node is now owned by exactly one code path: create by foldStakeIx*, free by applyRootedFold on the winning path or pruneStakeDeltas on losing siblings. applyRootedFold(stakes, block_pool, old_root, new_root): - Walks new_root up the replay.Node.parent chain to old_root exclusive, iterating each block's stake_delta_head chain newest-first. - Deduplicates via ReplayStakes.fold_scratch (~8 MB inline FixedPubkeyMap(void, 1<<18)) — first occurrence per delegator is the effective one; older shadowed nodes are freed silently. - Upserts hit stake_delegations_root.insert (overwriting any existing key); tombstones hit .remove (via the new FixedPubkeyMap.remove primitive from db22ce6). - Clears each visited block's stake_delta_head to .null. - Panics if the parent chain doesnt reach old_root — that violates the caller's descendant invariant. - Rooted aggregates are implicit in stake_aggregates[new_root], which the invariant maintains through fold + prune with no extra work here. pruneStakeDeltas(stakes, block): - Frees every node in block's chain, resets head to .null. - Idempotent (safe on already-.null heads). - Only correct because deltas are per-block-owned; children start with .null head, never a copy of parent's head. ReplayStakes gains: - fold_scratch: FixedPubkeyMap(void, MAX_STAKE_DELTA_NODES * 2) — ~8 MB inline dedup buffer for applyRootedFold. Reset at the start of every fold. Total ReplayStakes footprint now ~166 MB (was ~158 MB). Tests cover: - no-op fold when new_root == old_root - single-hop upsert lands in root, chain drained - 3-hop chain: c > b > a > root, c's stake wins, all heads drained - tombstone on descendant erases pre-seeded rooted row - arena nodes freed back to the pool (LIFO reuse) - pruneStakeDeltas frees a chain, second call is idempotent Callers still stubbed; the exec tile has no vote/stake program. Root advance in services/replay.zig is greenfield.
Applies the review rules: - delete restaters, tighten multi-paragraph docstrings to operative sentences - convert aspirational "once X lands" narrations to TODO(anchor)s - drop "mirrors agave X" prefaces without specific anchors; keep named references only where they anchor a concrete rule - refresh the module docstring and ReplayStakes field docs to reflect the delta arena + aggregates + fold scratch that landed in this PR - clarify the loadFromVersionedEpochStakes `total_stake` rationale in one line instead of six - keep load-bearing comments (memcpy-vs-struct-assign, StakeDeltaNode block-ownership + dual-use `next`, applyRootedFold invariants / cost, fold_scratch sizing) No code semantics changed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WIP