feat: upgrade reth sdk to v2.4.0 - #277
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
| gas_limit: POL_TX_GAS_LIMIT, // this is the env value used in revm for system calls | ||
| gas_price: base_fee.into(), /* gas price is set to the base fee for RPC | ||
| * compatibility reasons */ | ||
| gas_limit, // the block gas limit (36M per the Berachain genesis configurations) | ||
| gas_price: base_fee.into(), /* gas price is set to the base fee for RPC | ||
| * compatibility reasons */ |
There was a problem hiding this comment.
As mentioned on our last sync, I would prefer we try to keep reth 2.0 state compatible so we can roll this out without an EL hard fork. We can consider updating the gaslimit in future more lightweight el hf.
|
@calbera update on this? |
Still testing.
@camembera found an issue with taking snapshots of a v2 node (or rather downloading that snapshot) but its not bera-reth specific. filed an upstream bug here. Next steps are running this live on a devnet and migrating validator nodes from v1 to v2 incrementally and ensure network stays live. Would definitely help to start code review.. I'll open this PR once these tests are done. |
fridrik01
left a comment
fridrik01
left a comment
There was a problem hiding this comment.
Quick review, looks good, will wait for devnet testing and verification for a more detailed review.
There was a problem hiding this comment.
Pull request overview
Upgrades Bera-Reth to Reth SDK v2.4.0 and Storage V2 while adapting consensus, EVM, Engine API, RPC, and operational tooling.
Changes:
- Migrates dependencies and APIs to Reth v2.4.0.
- Adds Storage V2 migration guidance and tests.
- Updates payload, codec, RPC, EVM, CI, and Hive integration.
Reviewed changes
Copilot reviewed 42 out of 43 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
Cargo.toml |
Upgrades dependencies, MSRV, and profiles. |
Cargo.lock |
Locks upgraded dependency graph. |
build.rs |
Migrates vergen APIs. |
Cross.toml |
Documents build-image MSRV. |
Dockerfile |
Pins Rust 1.97 builder. |
Dockerfile.debug |
Pins debug builder image. |
README.md |
Adds BeaconKit and Storage V2 guidance. |
CLAUDE.md |
Refreshes development guidance. |
deny.toml |
Updates advisory exceptions. |
docs/storage-v2.md |
Adds Storage V2 operator guide. |
.github/workflows/sync.yml |
Reports nightly storage settings. |
.github/assets/hive/Dockerfile |
Updates Hive builder image. |
.github/assets/hive/build_simulators.sh |
Updates Hive simulator fixtures. |
.github/assets/hive/run_simulator.sh |
Updates simulator execution behavior. |
.github/assets/hive/parse.py |
Refreshes upstream provenance. |
.github/assets/hive/ignored_tests.yaml |
Marks exclusions for revalidation. |
.github/assets/hive/expected_failures.yaml |
Marks failures for revalidation. |
src/consensus/mod.rs |
Migrates consensus validation APIs. |
src/engine/builder.rs |
Migrates payload-building APIs. |
src/engine/mod.rs |
Updates Engine payload types. |
src/engine/payload.rs |
Moves payload-ID derivation and conversions. |
src/engine/rpc.rs |
Migrates Engine API runtime/provider bounds. |
src/evm/mod.rs |
Adapts revm system calls and gas accounting. |
src/node/evm/builder.rs |
Migrates block-builder interfaces. |
src/node/evm/config.rs |
Updates EVM environment construction. |
src/node/evm/executor.rs |
Migrates executor and gas APIs. |
src/pool/transaction.rs |
Stores Berachain consensus envelopes. |
src/primitives/header.rs |
Migrates header traits and codec tests. |
src/rpc/api.rs |
Migrates RPC type and helper traits. |
src/rpc/mod.rs |
Updates RPC configuration construction. |
src/rpc/receipt.rs |
Rebuilds RPC receipt log metadata. |
src/transaction/mod.rs |
Migrates transaction traits and codec tests. |
src/transaction/pol.rs |
Updates PoL system-call constants/errors. |
src/transaction/txtype.rs |
Migrates decompression errors. |
tests/e2e/mod.rs |
Updates shared payload/runtime setup. |
tests/e2e/storage_v2_test.rs |
Adds Storage V2 lifecycle tests. |
tests/e2e/osaka_engine_api_test.rs |
Migrates payload-triggering test flow. |
tests/e2e/osaka_blob_test.rs |
Updates test runtime setup. |
tests/e2e/prague3_empty_block_test.rs |
Updates test runtime setup. |
tests/e2e/pol_revert_test.rs |
Updates test runtime setup. |
tests/e2e/gas_limit_regression_test.rs |
Updates test runtime setup. |
tests/e2e/deposit_test.rs |
Updates test runtime setup. |
tests/e2e/coinbase_system_state_change_test.rs |
Updates test runtime setup. |
Suppressed comments (2)
src/engine/payload.rs:257
- This conversion consumes only
value.blockand dropsBerachainBuiltPayload::requests. Post-Prague execution data produced through this required conversion consequently has no EIP-7685 requests in its sidecar, even though the built payload stores them. Destructure the built payload and propagate its requests into the V4 sidecar.
impl From<BerachainBuiltPayload> for BerachainExecutionData {
fn from(value: BerachainBuiltPayload) -> Self {
crate::engine::BerachainEngineTypes::block_to_payload(
Arc::unwrap_or_clone(value.block),
None,
src/engine/payload.rs:339
- The PR description says byte-identical payload IDs are pinned by preserved vectors, but these assertions only compare IDs to each other. Any common change to the hash inputs or encoding still passes. Restore fixed expected payload-ID values generated by the pre-upgrade implementation.
// Test via PayloadAttributes::payload_id which calls berachain_payload_id
let id_no_pubkey = attributes_no_pubkey.payload_id(&parent);
let id_with_pubkey = attributes_with_pubkey.payload_id(&parent);
// Critical test: presence of pubkey should affect payload ID
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| init_datadir(dir.path(), Some(false)); | ||
| assert!(storage_settings(dir.path()).contains("storage_v2: false")); | ||
|
|
||
| let out = migrate_v2(dir.path()); | ||
| assert!(out.status.success(), "migrate-v2 failed: {}", String::from_utf8_lossy(&out.stderr)); |
| fn target_gas_limit(&self) -> Option<u64> { | ||
| self.inner.target_gas_limit |
| assert_eq!(len, 490, "encoded length changed"); | ||
| let hex = alloy_primitives::hex::encode(&buf); | ||
| assert!(hex.starts_with("80202008"), "field bitflags changed: {hex}"); | ||
| assert!(hex.ends_with("2a01c9c3806553f1000000000000000000000000000000000000000000000000000000000000000000043b9aca0062657261"), "encoded tail changed: {hex}"); |
| - The experimental JIT EVM (`--jit`) is present in the CLI but unvetted against | ||
| Berachain's custom EVM; do not enable it on validators. |
Upgrade to Reth SDK v2.4.0 (Storage V2)
Reviewer Outline
Suggested review order, from highest to lowest risk.
Cargo.lock(~4,300 lines) can be skimmed — the hand-written diff is roughly 1,700 lines.Area | Code Files | What to scrutinize
src/engine/payload.rs| The sha256 payload-ID derivation moved into PayloadAttributes::payload_id. Must produce byte-identical IDs; check the preserved unit-test vectors.src/primitives/header.rs,src/transaction/mod.rs| Golden byte-vector tests (mod db_format) pin the Compact encoding. Confirm no codec change slipped in — existing datadirs depend on it.src/engine/rpc.rs,src/engine/mod.rs,src/engine/builder.rs| beacon-kit ≥ v1.4.1 treats EL HTTP 4xx as fatal; the custom getPayloadV4P11 / pre-Osaka getPayloadV5 → UnsupportedFork behavior must survive.src/node/evm/*,src/pool/,src/rpc/, remainingsrc/| Trait-signature churn dictated by upstream; verify against reth v2.4.0 patterns rather than line-by-line.Cargo.toml,build.rs,deny.toml| default-features = false on reth — confirm the explicit feature list loses nothing vs v1.11.4.tests/e2e/storage_v2_test.rs,tests/e2e/gas_limit_regression_test.rs,tests/e2e/*| New coverage; check assertions actually pin the behaviors above.README.md,docs/storage-v2.md,.github/,Dockerfiles,CLAUDE.md| Operator guidance accuracy; hive assets flagged for re-tuning.Points worth reviewer attention:
rethdependency is nowdefault-features = falsewith an explicit feature list. Reth v2.4.0's defaults includejit(revmc, requires a system LLVM toolchain) andgmp; the explicit list reproduces the v1.11.4-equivalent set (otlp,otlp-logs,js-tracer,keccak-cache-global,asm-keccak,min-debug-logs) plusreth-revmwithportable.rust-version = "1.95"(reth v2.4.0's MSRV).[profile.dev]gainsdebug = "line-tables-only"+split-debuginfo = "unpacked", matching upstream reth — debug test binaries otherwise exceed multiple GB each.Storage V2
No node wiring was needed: the v2.4.0 launcher defaults handle everything.
--storage.v2flag (which only applies at datadir creation).bera-reth db migrate-v2 --chain <genesis>, confirmed generic over Berachain's custom primitives.docs/storage-v2.mdsays so explicitly.Operator-facing docs:
docs/storage-v2.md(what changed, the three operator situations, migration procedure,--storage.v2semantics, node modes) plus a new "Storage" section inREADME.md.API migration, file by file
src/primitives/header.rs— upstream removedRlpBincode/SerdeBincodeCompat(impls deleted);HeaderMutgainedset_mix_hash/set_extra_data/set_parent_beacon_block_root;Decompressnow returnsreth_codecs::DecompressError;BlockHeadergainedblock_access_list_hash()/slot_number(), bothNone(Berachain headers don't adopt EIP-7928 BAL fields). The on-diskCompactencoding is intentionally untouched, locked by golden testHEADER_V2_4_0_GOLDENplus a pre-Prague1 vector.src/transaction/mod.rs,txtype.rs—SignedTransactionis now blanket-implemented upstream, so the manual impl is deleted; sameDecompressErrorchange; golden vectorPOL_ENVELOPE_V2_4_0_GOLDENlocks the PoL envelope's on-disk format.src/engine/payload.rs— the structural pivot of this PR.PayloadTypeslosttype PayloadBuilderAttributes, soBerachainPayloadBuilderAttributesis deleted and the custom sha256 payload-ID derivation moved into the new requiredPayloadAttributes::payload_id(&self, parent_hash)(src/engine/payload.rs:37). Consensus-critical; the pre-existingberachain_payload_idunit-test vectors are preserved and pass. Also adds the now-requiredFrom<BerachainBuiltPayload> for BerachainExecutionData.src/engine/mod.rs—block_to_payloadgained abal: Option<Bytes>parameter;ExecutionPayloadgainedgas_limit()/slot_number().src/engine/builder.rs—PayloadConfignow carriespayload_id;BuildArgumentsgainedexecution_cache/state_root_handle(unused here);execute_transactionreturnsGasOutput;finish(state_provider, None)takes the new precomputed-state-root argument;mark_invalidtakes owned errors.src/engine/rpc.rs—EngineApi::newtakesreth_tasks::Runtimeinstead of a boxedTaskSpawner. v2.4.0's default engine capabilities flow through; the existingengine_getPayloadV5removal (pre-OsakaUnsupportedFork) is retained because beacon-kit ≥ v1.4.1 treats EL HTTP 4xx as fatal.src/evm/mod.rs— revm 41:Evmgained the requiredcfg_env(); the system-call path usesMainnetHandler::run_system_call/inspect_run_system_call;ExecutionResult::Successnow carriesResultGas(EIP-8037 gas split), and the PoLtransact_rawpath zeroes it viaResultGas::default().src/node/evm/executor.rs,config.rs,builder.rs—commit_transactionreturnsGasOutput;BlockExecutorFactorygainedtype TxExecutionResultand theExecutorGAT with aStateDBbound;BlockEnvgainedslot_num;BuildPendingEnvgained aBlockOverridesparameter.src/consensus/mod.rs—FullConsensus::validate_block_post_executiongained a fourth parameter (block_access_list_hash, forwarded to the inner Ethereum validator);ConsensusError::Otheris nowArc<dyn Error>, so all string sites useConsensusError::msg(...).src/pool/transaction.rs— implements the newPoolTransaction::consensus_ref(); the pool tx now storesRecovered<BerachainTxEnvelope>.src/rpc/api.rs— alloy 2.x orphan rules: thealloy_network::Network/TransactionBuilderimpls are replaced byimpl reth_rpc_convert::RpcTypes for BerachainNetwork(src/rpc/api.rs:101);TaskSpawner→Runtime; addssend_pool_transactionand the fully-defaultedGetBlockAccessList/EthSubscriptionsmarker impls.Tests
169 tests, all passing. New and changed coverage:
tests/e2e/storage_v2_test.rs(new) — five binary-driven tests viaCARGO_BIN_EXE:test_fresh_datadir_defaults_to_storage_v2test_storage_v2_flag_opts_new_datadir_into_v1test_v1_datadir_remains_readabletest_migrate_v2_converts_v1_datadir_in_placetest_migrate_v2_is_idempotentsrc/primitives/header.rsandsrc/transaction/mod.rspin the exact on-disk bytes, so any future codec drift fails loudly.tests/e2e/mod.rs,osaka_engine_api_test.rs— migrated to the v2 e2e harness (BerachainPayloadAttributesgenerator,Runtime::test(),BuildNewPayload/next_attributes()).CI / ops
deny.toml— drops five ignores resolved by the upgrade (hickory-proto ×2, git2 ×2 via vergen-git2 10, core2); adds dev-onlyRUSTSEC-2023-0089(atomic-polyfill via test-fuzz)..github/workflows/sync.yml— beacon-kit genesis/peer URLs pinned to v1.4.2-rc.0; adds adb settingsstep, making the nightly fresh-datadir sync a Storage V2 canary..github/assets/hive/) — re-synced from reth v2.4.0 with the Berachain deltas re-applied;expected_failures.yaml/ignored_tests.yamlheaders note they were last tuned against v1.11.4 and need re-validation once a v2.4.0-based nightly image exists.Cross.toml—cargo-chefpinned to a Rust ≥ 1.95 tag with an MSRV comment.Documentation
README.md— Rust 1.95+ prerequisite; new "Running with BeaconKit" section (v1.4.2 two-terminal flow, JWT/genesis wiring, why--engine.persistence-threshold 0and--engine.memory-block-buffer-target 0); new "Storage" section.docs/storage-v2.md(new) — full operator guide for the V1 → V2 transition.CLAUDE.md— refreshed reference versions (reth v2.4.0, beacon-kit v1.4.2), corrected file paths, Storage V2 note.TODOs
Still needs operator validation outside this environment:
migrate-v2run.scripts/test-block-progression.sh.