feat(v2:replay): basic epoch voter states - #1770
Draft
yewman wants to merge 12 commits into
Draft
Conversation
Open-addressed, linear-probed, fixed-capacity Pubkey -> V map. Capacity is a comptime power of two; callers size to >= 2x expected occupancy. Pubkey.ZEROES is the empty-slot sentinel. No delete — rebuild via init(). Intended to live inline in extern structs (e.g. EpochVoters), so V must be extern-compatible. Wired into v2/lib/collections.zig; unit tests cover insert/get, missing keys, zero-pubkey rejection, duplicate overwrite, pointer stability across inserts, and random fill at 50% load.
Replay-owned stakes types.
- EpochVoters: extern struct, 2000-entry frozen per-epoch snapshot
of admitted voters (vote_pk, stake, commission_bps) plus a
FixedPubkeyMap<u16, 4096> side index and precomputed total_stake.
Serves the sol_get_epoch_stake syscall (O(1)) and the clock
sysvar timestamp update (linear scan in lockstep with LiveVoters).
- LiveVoter (24 B) / LiveVoters (48 KB): dense per-block live vote
state, positionally aligned with EpochVoters.entries. Kind is
{ unpopulated, update, invalidate }.
- ReplayStakes: single-owner container. epoch_voters + one LiveVoters
per BlockPool slot + boot_epoch. No allocations after init.
Provisional shape: v2 convention is one top-level shared region
per collection (BlockPool style); the wrapper is expected to
dissolve into standalone regions. epoch_voters and live_voters
themselves are durable. boot_epoch is a provisional hard-stop
anchor, deleted once epoch-boundary rotation state lands.
MAX_ALPENGLOW_VOTE_ACCOUNTS = 2000 (SIMD-0357). Map cap = 4096 to
respect FixedPubkeyMap's pow2 / 2x-occupancy invariants.
Tests: EpochVoters init/lookup, LiveVoters.reset.
- EpochVoters.loadFromVersionedEpochStakes(entry, memory_base):
copies { pubkey, stake } into entries, sorts desc, builds the
by_vote_pk index, sums total_stake. commission_bps stays zero —
the trimmed VersionedEpochStakes.VoteAccountEntry only carries
{ pubkey, stake } and no reader consumes commission today.
Errors on over-cap input (MAX_ALPENGLOW_VOTE_ACCOUNTS).
- ReplayStakes.loadFromSnapshot(root_slot, manifest, memory_base):
derives boot_epoch via EpochSchedule.getEpoch, locates the
matching VersionedEpochStakes entry, delegates. Errors if no
entry matches the boot epoch.
- v2/services/replay.zig: allocates a ReplayStakes on the FBA,
init(), calls loadFromSnapshot after bootstrap. Reads root_slot
via a second (non-blocking) getSlotBlocking since bootstrap has
already crossed the release barrier.
- v2/tests/replay/main.zig: fixture manifest gains a minimal
VersionedEpochStakes entry (epoch 0, empty voters) so the loader
succeeds. Downstream reads stay no-ops until fixtures grow real
stakes data.
Two new unit tests cover the sort+index and the over-cap reject.
- ReplayStakes.onBlockCreate(parent, child): explicit @memcpy on the byte view (`std.mem.asBytes`) rather than struct assignment. Struct-assign of a ~48 KB LiveVoters value passes through a stack temporary in Debug mode; the byte-level memcpy stays in-place and keeps the tx-exec call stack healthy. - v2/services/replay.zig: thread ?*ReplayStakes through insertFecSet -> setChildTreeBlockRefs -> setChildBlockRef. Fire onBlockCreate at the two block_pool.create() sites in setChildBlockRef — case (b) slot boundary and case (c) fork/equivocation. Case (a) has no parent BlockRef; case (d) reuses the parent's BlockRef unchanged (same-slot canonical extension keeps writing the same slot). - Production caller in serviceMain passes the real replay_stakes. Bootstrap's synthetic root insertFecSet passes null — root has no parent BlockRef so setChildBlockRef early-outs anyway. Existing test callers also pass null. Test: onBlockCreate mutates the parent slot, clones, mutates the child, asserts parent is untouched.
`solGetEpochStake(epoch_voters, ?*const Pubkey) u64` — pure implementation of SIMD-0133 semantics: - null arg -> epoch_voters.total_stake (total active stake). - non-null -> epoch_voters.stakeOf(pk), which returns 0 on miss (agave parity — not-a-vote-account / non-existent -> 0). VM-side concerns (compute-meter charging, VM memory translation of the pubkey argument) are outside the primitive; the SVM tile will apply them and call this function with a validated ?*const Pubkey. Not yet reachable at runtime — v2 has no SVM interpreter. When the exec tile grows one, register this as the sol_get_epoch_stake handler with the current EpochVoters bound on the invoke context. Test covers null / hit / miss.
`stakeWeightedTimestamp(epoch_voters, live_voters, current_slot, slot_duration_ns, epoch_start, drift) ?i64` — pure implementation of agave's `calculate_stake_weighted_timestamp`: 1. Project each `.update` voter's `last_vote_timestamp` to current_slot via `elapsed_secs = slots * slot_duration_ns / 1e9`. 2. Sort projections asc, walk accumulating stake, return the projection where cumulative first exceeds total / 2 — the stake-weighted median. 3. If epoch_start is provided, bound the median forward / backward by drift.slow_pct / drift.fast_pct of the poh estimate offset since the epoch start. Returns null iff no voter has a live `.update` row with non-zero stake. Also adds: - TimestampDrift extern struct with SIMD-0133 defaults (fast=25%, slow=150%). - EpochStartTimestamp anchor struct. Not yet reachable at runtime — v2 has no per-slot sysvar update path. When one exists it calls this each slot with the current fork's LiveVoters; the returned i64 is written to Clock.unix_timestamp. Scratch buffer: 2000 (estimate, stake) pairs, ~32 KB on the caller's stack. Declared locally in the pure function; safe because we call it top-of-stack, not from a deep tx-exec chain (unlike the onBlockCreate copy that hit the Debug stack-temporary trap). Tests: no-signal returns null; three-voter median crosses at the right stake bucket; slow-drift clamp forward to epoch_start + poh_off + slow_bound.
`foldLandedVote(epoch_voters, live_voters, vote_pk, last_vote_slot, last_vote_timestamp)` — the primitive the committer path invokes per landed vote tx. - Look up the vote pubkey in `by_vote_pk`. Miss: no-op (post- Alpenglow SIMD-0357 caps admitted voters at top-2000; non- admitted votes land at the tx layer but contribute nothing to the timestamp aggregate). - Hit: overwrite the positional row with the new .update state (last_vote_slot, last_vote_timestamp, kind = .update). Not yet reachable at runtime — v2 has no vote-program execution. When the exec tile grows one, the committer path decodes each landed vote ix (accounts + payload) and calls this once with the extracted values. Vote-account deletion (the .invalidate transition) is not modelled. Tests: fresh admitted voter -> .update; second fold overwrites in place; non-admitted voter -> no-op across the whole array.
- ReplayStakes gains `first_slot_of_next_epoch: Slot`, cached at boot from `epoch_schedule.getFirstSlotInEpoch(boot_epoch + 1)`. - ReplayStakes.ensureSlotInBootEpoch(slot) returns error.EpochBoundaryNotYetImplemented for slot >= boundary. Both fields are provisional scaffolding, deleted when boundary derivation lands. - Wired into setChildBlockRef case (b) — the point at which a new BlockRef is allocated for a slot different from the parent. Case (c) is same-slot fork; case (d) is same-slot canonical extension; case (a) has no BlockRef. Only case (b) can cross an epoch. - insertFecSet error set widened to include the new variant. Fixture: v2/tests/replay/main.zig now sets `epoch_schedule = .INIT` instead of leaving it zero-initialized (which gave a degenerate epoch=0 for every slot). Boot epoch for slot 410009999 under EpochSchedule.INIT resolves to 961; the VersionedEpochStakes entry epoch is aligned to the derived boot_epoch so the loader still matches. Test: ensureSlotInBootEpoch passes in-epoch slots (500, 599 for a boot into epoch 5); trips on the boundary (600) and beyond.
Three multi-primitive tests exercising the stakes flow at the module level (no runtime plumbing — that path is covered by the bbt-replay black-box test): - boot -> solGetEpochStake -> foldLandedVote -> stakeWeightedTimestamp: synthesize a VersionedEpochStakes entry with three admitted voters at stakes 100/250/650, load, query the syscall for null/hit/miss, fold three landed votes plus one unadmitted stranger, verify the stake-weighted median crosses at the top-staked voter's timestamp. - fork of depth 3 (root -> A -> B -> C): fold at each hop, memcpy via onBlockCreate, verify each descendant carries all prior state and root remains untouched. - sibling forks (parent -> left, parent -> right with disjoint vote sets): verify left / right / parent stay isolated after the two memcpy branches. Deferred: a real snapshot round-trip (needs a fixture with real stakes data) and syscall conformance vectors (needs a v2 SVM interpreter).
yewman
force-pushed
the
harnew/v2-stakes-mvp
branch
from
July 28, 2026 10:42
1679713 to
4e100f4
Compare
Codecov Report❌ Patch coverage is
... and 2 files with indirect coverage changes 🚀 New features to boost your workflow:
|
…hStakes parse - VoteAccountEntry gains `commission_bps: u16` (+ pad). - VersionedEpochStakes.read peeks the first bytes of each vote-account data blob before discarding the rest, extracts the commission field via voteStateCommissionBps, and stores it on the entry. - voteStateCommissionBps: pure helper. Reads the u32 discriminant and dispatches to the v1_14_11 / v3 layout (u8 percentage at offset 68, scaled by 100) or the v4 layout (u16 inflation_rewards_commission_bps at offset 132, SIMD-0185). Returns 0 for unknown discriminants, uninitialized states, or blobs too short for the encoded variant. - EpochVoters.loadFromVersionedEpochStakes now copies `src.commission_bps` into the row instead of writing 0. Extracting at parse time (one linear read pass over the vote data we would have discarded anyway) is strictly cheaper than a second pass over accounts_db at boot, and keeps the offset knowledge in one place with the rest of the VoteStateVersions layout facts. Tests (in snapshot.zig): v1_14_11 / v3 percentage decode (42 -> 4200 bps, 7 -> 700); v4 bps direct decode (750); malformed and out-of-range inputs return 0 rather than aborting.
Make it explicit in the doc comments that LiveVoter, LiveVoters, foldLandedVote, stakeWeightedTimestamp, and EpochStartTimestamp serve the pre-Alpenglow Clock.unix_timestamp derivation (stake-weighted median of vote timestamps landed as vote txs). Under Alpenglow, individual vote messages leave the block (they travel over the votor / consensus lane instead), so replay never observes them during block execution. Clock.unix_timestamp is written from the block footer's block_producer_time_nanos instead (SIMD-0363; agave Bank::update_clock_from_footer in agave/runtime/src/bank.rs), and none of the LiveVoters machinery is invoked. No functional change; call sites and shape unchanged. The Alpenglow footer-timestamp path is a separate future workstream belonging to the block-execution flow, not to the stakes module.
yewman
force-pushed
the
harnew/v2-stakes-mvp
branch
from
July 30, 2026 12:06
f2f41fa to
1a9651f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WIP