Skip to content

chore: delete old devnet#666

Open
0xForerunner wants to merge 1 commit into
mainfrom
fr/del-devnet
Open

chore: delete old devnet#666
0xForerunner wants to merge 1 commit into
mainfrom
fr/del-devnet

Conversation

@0xForerunner

Copy link
Copy Markdown
Contributor

No description provided.

@0xOsiris

Copy link
Copy Markdown
Contributor

Before we fully deprecate do we have feature parity:

  • Can I modify the hard fork parameters in the current devnet via a config file e.g. isthmus, holocene, jovian
  • Load Testing, Stress Tests, and E2E Stress Tests are fully hooked up to the new version
  • Bundling / User Op Support
  • Tx Proxy Support

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Compared on the same runner in the same workflow run.

group                                                                                              base                                   pr
-----                                                                                              ----                                   --
flashblock_validation_launch_flashblock_sequence_eth_transfers/stream/4fb_x_125tx                  1.10     30.8±0.31ms        ? ?/sec    1.00     27.9±0.10ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_eth_transfers/stream/4fb_x_250tx                  1.05     58.9±0.16ms        ? ?/sec    1.00     56.0±0.08ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_eth_transfers/stream/4fb_x_50tx                   1.21     13.8±0.17ms        ? ?/sec    1.00     11.4±0.02ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_eth_transfers_with_bal/stream/4fb_x_125tx         1.04     36.4±0.98ms        ? ?/sec    1.00     35.1±0.25ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_eth_transfers_with_bal/stream/4fb_x_250tx         1.02     68.3±0.51ms        ? ?/sec    1.00     66.8±0.25ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_eth_transfers_with_bal/stream/4fb_x_50tx          1.02     16.6±0.36ms        ? ?/sec    1.00     16.3±0.29ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_fib/stream/4fb_x_125tx                            1.00     26.8±0.05ms        ? ?/sec    1.00     26.8±0.02ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_fib/stream/4fb_x_250tx                            1.00     53.1±0.52ms        ? ?/sec    1.00     52.9±0.08ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_fib/stream/4fb_x_50tx                             1.01     11.2±0.12ms        ? ?/sec    1.00     11.1±0.03ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_fib_with_bal/stream/4fb_x_125tx                   1.00     33.6±0.31ms        ? ?/sec    1.03     34.7±0.43ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_fib_with_bal/stream/4fb_x_250tx                   1.00     63.0±0.42ms        ? ?/sec    1.02     64.1±0.31ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_fib_with_bal/stream/4fb_x_50tx                    1.01     15.8±0.36ms        ? ?/sec    1.00     15.6±0.25ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_world_id_like_bn254/stream/4fb_x_10tx             1.00    135.8±0.27ms        ? ?/sec    1.00    135.2±0.19ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_world_id_like_bn254/stream/4fb_x_12tx             1.00    162.1±0.41ms        ? ?/sec    1.00    162.0±0.24ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_world_id_like_bn254/stream/4fb_x_5tx              1.01     68.3±0.13ms        ? ?/sec    1.00     67.9±0.05ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_world_id_like_bn254_with_bal/stream/4fb_x_10tx    1.01     22.3±1.35ms        ? ?/sec    1.00     22.0±1.45ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_world_id_like_bn254_with_bal/stream/4fb_x_12tx    1.00     24.3±1.77ms        ? ?/sec    1.03     25.1±1.72ms        ? ?/sec
flashblock_validation_launch_flashblock_sequence_world_id_like_bn254_with_bal/stream/4fb_x_5tx     1.01     18.9±0.33ms        ? ?/sec    1.00     18.7±0.20ms        ? ?/sec
flashblock_validation_process_flashblock_eth_transfers/txs/1000                                    1.00     48.8±0.09ms        ? ?/sec    1.00     48.8±0.10ms        ? ?/sec
flashblock_validation_process_flashblock_eth_transfers/txs/50                                      1.00      2.6±0.02ms        ? ?/sec    1.00      2.6±0.01ms        ? ?/sec
flashblock_validation_process_flashblock_eth_transfers/txs/500                                     1.00     24.5±0.06ms        ? ?/sec    1.00     24.4±0.06ms        ? ?/sec
flashblock_validation_process_flashblock_eth_transfers_with_bal/txs/1000                           1.02     60.0±1.62ms        ? ?/sec    1.00     58.6±0.17ms        ? ?/sec
flashblock_validation_process_flashblock_eth_transfers_with_bal/txs/50                             1.00      3.8±0.07ms        ? ?/sec    1.03      3.9±0.05ms        ? ?/sec
flashblock_validation_process_flashblock_eth_transfers_with_bal/txs/500                            1.00     29.4±0.24ms        ? ?/sec    1.01     29.8±0.13ms        ? ?/sec
flashblock_validation_process_flashblock_fib/txs/1000                                              1.00     48.1±0.09ms        ? ?/sec    1.00     47.9±0.22ms        ? ?/sec
flashblock_validation_process_flashblock_fib/txs/50                                                1.00      2.5±0.01ms        ? ?/sec    1.00      2.5±0.01ms        ? ?/sec
flashblock_validation_process_flashblock_fib/txs/500                                               1.00     24.1±0.04ms        ? ?/sec    1.00     24.0±0.05ms        ? ?/sec
flashblock_validation_process_flashblock_fib_with_bal/txs/1000                                     1.01     56.2±0.19ms        ? ?/sec    1.00     55.7±0.79ms        ? ?/sec
flashblock_validation_process_flashblock_fib_with_bal/txs/50                                       1.00      3.7±0.03ms        ? ?/sec    1.00      3.7±0.05ms        ? ?/sec
flashblock_validation_process_flashblock_fib_with_bal/txs/500                                      1.00     28.2±0.16ms        ? ?/sec    1.00     28.3±0.25ms        ? ?/sec
flashblock_validation_process_flashblock_world_id_like_bn254/txs/10                                1.00     33.6±0.02ms        ? ?/sec    1.00     33.8±0.09ms        ? ?/sec
flashblock_validation_process_flashblock_world_id_like_bn254/txs/25                                1.00     84.1±0.12ms        ? ?/sec    1.00     84.2±0.15ms        ? ?/sec
flashblock_validation_process_flashblock_world_id_like_bn254/txs/50                                1.00    167.6±0.20ms        ? ?/sec    1.00    168.4±0.44ms        ? ?/sec
flashblock_validation_process_flashblock_world_id_like_bn254_with_bal/txs/10                       1.02      5.7±0.55ms        ? ?/sec    1.00      5.6±0.37ms        ? ?/sec
flashblock_validation_process_flashblock_world_id_like_bn254_with_bal/txs/25                       1.00      8.1±0.08ms        ? ?/sec    1.01      8.2±0.11ms        ? ?/sec
flashblock_validation_process_flashblock_world_id_like_bn254_with_bal/txs/50                       1.00     14.8±0.07ms        ? ?/sec    1.00     14.9±0.18ms        ? ?/sec

Threshold: 10% regression triggers failure

@alessandromazza98

alessandromazza98 commented May 25, 2026

Copy link
Copy Markdown
Contributor

Before we fully deprecate do we have feature parity:

  • Can I modify the hard fork parameters in the current devnet via a config file e.g. isthmus, holocene, jovian
  • Load Testing, Stress Tests, and E2E Stress Tests are fully hooked up to the new version
  • Bundling / User Op Support
  • Tx Proxy Support

I agree on this. In general I think it's not super hard to integrate all these things, but they're not yet integrated. I've tried the following (while devnet was running):

❯ just stress load-test --rpc-url http://127.0.0.1:65182
RUST_LOG="info" cargo run -p xtask -- stress $@
    Finished [`dev` profile [unoptimized + debuginfo]](https://doc.rust-lang.org/cargo/reference/profiles.html#default-profiles) target(s) in 1.46s
     Running `target/debug/xtask stress load-test --rpc-url 'http://127.0.0.1:65182'`
Error: No such file or directory (os error 2)

Location:
    xtask/src/stress/cli/transactions.rs:129:58
error: recipe `stress` failed on line 74 with exit code 1

This shows that current rust local devnet needs to incorporate those features yet.

@alessandromazza98

Copy link
Copy Markdown
Contributor

I agree on this. In general I think it's not super hard to integrate all these things, but they're not yet integrated. I've tried the following (while devnet was running):

❯ just stress load-test --rpc-url http://127.0.0.1:65182
RUST_LOG="info" cargo run -p xtask -- stress $@
    Finished [`dev` profile [unoptimized + debuginfo]](https://doc.rust-lang.org/cargo/reference/profiles.html#default-profiles) target(s) in 1.46s
     Running `target/debug/xtask stress load-test --rpc-url 'http://127.0.0.1:65182'`
Error: No such file or directory (os error 2)

Location:
    xtask/src/stress/cli/transactions.rs:129:58
error: recipe `stress` failed on line 74 with exit code 1

This shows that current rust local devnet needs to incorporate those features yet.

EDIT: I've fixed it now, so now just stress works fine automatically with the new rust local devnet

@alessandromazza98

Copy link
Copy Markdown
Contributor

Before we fully deprecate do we have feature parity:

  • Can I modify the hard fork parameters in the current devnet via a config file e.g. isthmus, holocene, jovian
  • Load Testing, Stress Tests, and E2E Stress Tests are fully hooked up to the new version
  • Bundling / User Op Support
  • Tx Proxy Support

as of now we have feature parity around first and second items in this list.

There is no bundler and tx proxy in the new rust local devnet, therefore the last 2 items in the list are not satisfied. I would say that it's still good enough because the local devnet is useful to check and ensure we are not breaking anything on the node when doing a big change in the code.

For tx proxy / bundlers we do have DEV environment where we can test a lot.

Further, they can be added on follow up PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants