feat: Execution layer fast sync by ark0f · Pull Request #5425 · gear-tech/gear

ark0f · 2026-05-14T00:03:29Z

Summary

Replace db-sync protocol with thin wrapper for bitswap protocol

How to test

Run ethexe_service::tests::fast_sync

Notes

db-sync is gone.

Checklist

PR title follows Conventional Commits (type(scope): description)
Single logical change
Tests added or updated (if logic changed)
Docs updated (if needed)

Closes #5435
Closes #4812

semanticdiff-com · 2026-05-14T00:03:32Z

Review changes with

Changed Files

File	Status
ethexe/consensus/src/validator/initial.rs	84% smaller
ethexe/network/src/validator/topic.rs	80% smaller
ethexe/network/src/injected.rs	78% smaller
ethexe/network/src/gossipsub.rs	47% smaller
ethexe/consensus/src/connect/mod.rs	40% smaller
ethexe/observer/src/utils.rs	30% smaller
ethexe/network/src/utils.rs	27% smaller
ethexe/network/src/lib.rs	21% smaller
ethexe/common/src/network.rs	4% smaller
ethexe/service/src/tests/utils/env.rs	4% smaller
ethexe/network/src/validator/discovery.rs	4% smaller
ethexe/cli/src/params/network.rs	1% smaller
Cargo.lock	Unsupported file format
ethexe/ethereum/src/lib.rs	Unsupported file format
ethexe/ethereum/src/router/events.rs	Unsupported file format
ethexe/network/Cargo.toml	Unsupported file format
ethexe/network/src/bitswap.rs	0% smaller
ethexe/network/src/db_sync/mod.rs	0% smaller
ethexe/network/src/db_sync/requests.rs	0% smaller
ethexe/network/src/db_sync/responses.rs	0% smaller
ethexe/network/src/peer_score.rs	0% smaller
ethexe/observer/Cargo.toml	Unsupported file format
ethexe/service/src/fast_sync.rs	Unsupported file format
ethexe/service/src/lib.rs	Unsupported file format

# Conflicts: # Cargo.lock # ethexe/common/src/network.rs # ethexe/consensus/src/connect/mod.rs # ethexe/consensus/src/validator/initial.rs # ethexe/network/src/db_sync/mod.rs # ethexe/network/src/db_sync/requests.rs # ethexe/network/src/db_sync/responses.rs # ethexe/network/src/lib.rs # ethexe/service/src/fast_sync.rs # ethexe/service/src/lib.rs

# Conflicts: # Cargo.lock # ethexe/service/src/lib.rs # utils/gear-workspace-hack/Cargo.toml

…malachite-errors # Conflicts: # ethexe/compute/src/compute.rs # ethexe/malachite/service/src/externalities.rs

gemini-code-assist · 2026-06-16T17:15:39Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request replaces the custom db-sync protocol with a bitswap-based implementation to improve data synchronization efficiency across the network. It introduces a robust fast-sync mechanism, enabling nodes to recover state more quickly by replaying proposals and finalizations. The changes include a new network behaviour, updated storage logic, and comprehensive testing to ensure the new synchronization path is resilient.

Highlights

Protocol Replacement: Replaced the legacy db-sync protocol with a bitswap-based implementation for more efficient data synchronization.
Fast Sync Implementation: Implemented the fast synchronization logic in ethexe/service/src/fast_sync.rs to allow nodes to catch up more effectively.
Network Behaviour: Added a new bitswap network behaviour to handle content fetching, including a Blockstore implementation.
Dependency Updates: Updated Cargo.lock and Cargo.toml files to include necessary dependencies like beetswap, blockstore, cid, and multihash.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request replaces the custom request/response db_sync protocol with a Bitswap-backed content fetching mechanism in ethexe-network and implements the actual fast synchronization logic for MB-driven ethexe nodes in ethexe-service. It also introduces a fast-sync replay filter in Malachite to suppress outbound events during startup replay, updates event builders to support block range queries, and integrates the demo-mul-by-const program for testing. Feedback on these changes highlights a potential network behaviour stall in the Bitswap polling implementation due to an immediate return of Poll::Pending after a ready event, which should be resolved with a loop. Additionally, a redundant block size check was identified in Blockstore::get, and opportunities for improvement were noted in fast_sync.rs to fetch program state hashes concurrently and to avoid panics by propagating decoding errors as Result instead of using .expect().

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 261ed3ee2f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…malachite-errors # Conflicts: # ethexe/malachite/service/src/externalities.rs

Base integration of `gsobol/ethexe/tune-malachite-errors` (Malachite error-handling rework) into `al/ethexe/bitswap`, with bitswap's Malachite-crate changes dropped in favour of tune's. This models the state after `tune → master → al/ethexe/bitswap` and is the base for the separate `gsobol/ethexe/bitswap-malachite` redesign PR. - ethexe/malachite/*, ethexe/compute/* → tune's versions (prerequisite EB gating still lives inside Malachite externalities here). - ethexe/service/* → bitswap's bitswap-network + fast-sync feature set, adapted to tune's Malachite API (MalachiteServiceStarter, receive_new_eb / receive_eb_synced / receive_eb_prepared, Stream<Result<MalachiteEvent>>). bitswap's now-removed replay-filter wiring is dropped. NOTE: fast sync has no replay suppression on this branch yet, so the `fast_sync` integration test is expected to stall here — that is exactly what the `bitswap-malachite` PR fixes by relocating the prerequisite gate into ethexe-compute and adding an app-level replay gate. `cargo fmt`, `cargo clippy` and compilation are clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EgjaE68s3nARWKBCa4Jid1

…chite Replaces the in-Malachite fast-sync replay filter with a layered design, on top of `gsobol/ethexe/al-bitswap` (al/ethexe/bitswap + tune-malachite-errors). A pure app-level "wait for BlockFinalized(target)" gate is impossible against tune's externalities: `try_emit_or_queue` head-of-line-blocks the event queue on an unprepared prerequisite EB, so the target event would never reach the app. So the prerequisite gate is moved out of Malachite: - Malachite externalities no longer queue events behind a prepared-EB prerequisite (try_emit_or_queue / pending_events / drain_pending_events / prerequisite_satisfied removed; receive_eb_prepared gone). process_mb_* emit immediately and strictly in order. - ethexe-compute now owns the "defer an MB until the EB it advances to is prepared" gate (ComputeSubService::deferred + receive_prepared_block), scanning all deferred requests on each BlockPrepared since the prepare sub-service only reports the chain head; defer-at-launch avoids head-of-line blocking ready requests. - The service applies an app-level ReplayGate after fast sync: it suppresses replayed BlockProposals until the committed target MB is reached (by hash for a fresh DB, or via a one-shot ancestry walk when the target was already finalized locally before start), then resumes live consensus. A missing CompactMb mid-walk fails loudly instead of hanging. cargo fmt, cargo clippy and cargo nextest run -p "ethexe-*" all pass (543 tests, incl. the fast_sync integration test). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EgjaE68s3nARWKBCa4Jid1

# Conflicts: # ethexe/malachite/core/src/config.rs # ethexe/malachite/core/src/externalities.rs # ethexe/malachite/core/src/lib.rs # ethexe/malachite/core/src/service.rs # ethexe/malachite/core/src/store.rs # ethexe/malachite/core/tests/multi_validators.rs # ethexe/malachite/service/src/config.rs # ethexe/malachite/service/src/externalities.rs # ethexe/malachite/service/src/lib.rs # ethexe/malachite/service/src/service.rs # ethexe/malachite/service/tests/restart_resilience.rs # ethexe/service/src/lib.rs # ethexe/service/src/tests/utils/env.rs

… into al/ethexe/bitswap # Conflicts: # ethexe/common/src/primitives.rs # ethexe/compute/src/compute.rs # ethexe/compute/src/lib.rs # ethexe/malachite/core/src/app.rs # ethexe/malachite/core/src/config.rs # ethexe/malachite/core/src/lib.rs # ethexe/malachite/core/src/service.rs # ethexe/malachite/core/src/store.rs # ethexe/malachite/core/tests/multi_validators.rs # ethexe/malachite/service/src/config.rs # ethexe/malachite/service/src/externalities.rs # ethexe/malachite/service/src/lib.rs # ethexe/malachite/service/src/service.rs # ethexe/malachite/service/src/starter.rs # ethexe/malachite/service/src/types.rs # ethexe/malachite/service/tests/restart_resilience.rs # ethexe/service/src/fast_sync.rs # ethexe/service/src/lib.rs # ethexe/service/src/tests/utils/env.rs

# Conflicts: # Cargo.lock # ethexe/malachite/core/src/config.rs # ethexe/malachite/core/src/lib.rs # ethexe/malachite/core/src/service.rs # ethexe/malachite/core/tests/multi_validators.rs # ethexe/malachite/service/src/config.rs # ethexe/malachite/service/src/lib.rs # ethexe/malachite/service/src/starter.rs # ethexe/network/src/gossipsub.rs # ethexe/network/src/lib.rs # ethexe/service/src/lib.rs # ethexe/service/src/tests/utils/env.rs

ark0f added 5 commits May 13, 2026 19:38

Initial db-sync removal

2f40389

Merge remote-tracking branch 'origin/master' into al/ethexe/bitswap

02b398f

Initial announces

7b26975

Done announces

cf5582d

Initial fast sync

9313ad7

ark0f added type: refactor Internal improvements without changing behavior scope: vara.eth Vara Ethereum application layer (L2) labels May 14, 2026

ark0f added 4 commits May 14, 2026 03:13

Remove unused deps

7abbff0

Update docs

1b22b0f

Add tests

5a8c584

Map database panics to FatalDatabaseError and add corresponding test.

692b447

breathx changed the title ~~refactor!(ethexe-network): Use Bitswap protocol~~ refactor!(ethexe/network): use Bitswap protocol May 14, 2026

breathx added the ci: full Run full CI matrix (all platforms and profiles) label May 14, 2026

breathx assigned ark0f May 14, 2026

ark0f added 2 commits May 14, 2026 20:20

Merge branch 'master' into al/ethexe/bitswap

349dde9

Merge branch 'master' into al/ethexe/bitswap

769ee11

ark0f removed the ci: full Run full CI matrix (all platforms and profiles) label May 15, 2026

ark0f added 12 commits May 25, 2026 21:42

Initial fast sync

61241cc

Remove mb_sync

0a7c1ec

Initial assert_chain

e908675

Merge remote-tracking branch 'origin/master' into al/ethexe/bitswap

7ed0faa

# Conflicts: # Cargo.lock # ethexe/service/src/lib.rs # utils/gear-workspace-hack/Cargo.toml

make workspace-hack

fcac944

Pending removal of custom_network_bootstrap_addresses

e74c8eb

Manual fixes

201aba0

Simplify request and response in bitswap

599a385

Implement Handle::request_many

67e27c5

Manual fixes

23a5bae

Speed up Malachite sync in tests

25a0366

ark0f and others added 6 commits June 14, 2026 15:53

fix fast sync batch digest invariant

b9039b7

Document Malachite fast sync chain head seed

5a5a08a

Merge remote-tracking branch 'origin/master' into gsobol/ethexe/tune-…

509d326

…malachite-errors # Conflicts: # ethexe/compute/src/compute.rs # ethexe/malachite/service/src/externalities.rs

Remove stale fast-sync delay wait

607bdf6

Fix fast sync computed MB wait

c07d3fe

Merge remote-tracking branch 'origin/master' into al/ethexe/bitswap

261ed3e

ark0f marked this pull request as ready for review June 16, 2026 17:08

gemini-code-assist Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread ethexe/network/src/bitswap.rs Outdated

Comment thread ethexe/network/src/bitswap.rs

Comment thread ethexe/service/src/fast_sync.rs

Comment thread ethexe/service/src/fast_sync.rs

chatgpt-codex-connector Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread ethexe/service/src/fast_sync.rs

Comment thread ethexe/network/src/bitswap.rs

Comment thread ethexe/service/src/fast_sync.rs Outdated

ark0f and others added 3 commits June 16, 2026 20:25

chore: fix CI checks

54b300f

Merge remote-tracking branch 'origin/master' into gsobol/ethexe/tune-…

14cdd65

…malachite-errors # Conflicts: # ethexe/malachite/service/src/externalities.rs

test(ethexe): wait for referenced MB computations

d7e66c1

ark0f force-pushed the al/ethexe/bitswap branch from 7e680da to d7e66c1 Compare June 16, 2026 23:14

ark0f and others added 14 commits June 17, 2026 02:18

Fix docs

0ad87e9

Fix fast sync test race

cc41e9e

Remove duplicate size check in Bitswap

adc4e25

fix(vara.eth): drain bitswap inner events

3e0683a

Remove redundant fast sync compute wait

6d25f41

Skip zero program states

f0afc8e

Bind fast sync batch digest to replay block

040c734

Merge remote-tracking branch 'origin/master' into al/ethexe/bitswap

42e2080

Merge branch 'master' into gsobol/ethexe/tune-malachite-errors

e9ace05

Post-merge fixes

22a694c

grishasobol mentioned this pull request Jun 30, 2026

Replace unsafe on-chain data sync in fast_sync with a canonical approach #4926

Closed

Navigation Menu

Uh oh!

feat: Execution layer fast sync - #5425

feat: Execution layer fast sync#5425
ark0f wants to merge 89 commits into
masterfrom
al/ethexe/bitswap

ark0f commented May 14, 2026 •

edited

Loading

Uh oh!

semanticdiff-com Bot commented May 14, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot commented Jun 16, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

ark0f commented May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How to test

Notes

Checklist

Uh oh!

semanticdiff-com Bot commented May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot commented Jun 16, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ark0f commented May 14, 2026 •

edited

Loading

semanticdiff-com Bot commented May 14, 2026 •

edited

Loading