feat(spice): making congestion control work under SPICE. txs backpressure. by ssavenko-near · Pull Request #15960 · near/nearcore

ssavenko-near · 2026-06-23T13:50:20Z

Part 2 (of 3) of making congestion control work under SPICE.

The congestion inputs that gate transaction admission aren't available in the SPICE chunk header.
This PR derives them from the last certified block's executed ChunkExtras and migrates the congestion tx-limit tests to test-loop and enables them under spice.

Production

Add spice_block_congestion_info: per-shard BlockCongestionInfo reconstructed from a block's executed ChunkExtras. Prefers the locally executed ChunkExtra from execution, falling back to the chain-wide
certified result when not tracking the relevant shard.
Chunk producer: gate tx admission on the last certified block's congestion (local gas throttling + filtering txs to congested receiver shards), via PrepareTransactionsBlockContext.congestion_info.
RPC handler: reject transactions to a congested receiver shard (just as in non-spice path).
Factor the DBCol::execution_results read into a shared get_execution_result_from_store. SpiceCoreReader and the new path now share it.

Tests

Migrate integration-tests/.../features/congestion_control.rs → test-loop congestion_control_limits.rs and drop the #[cfg_attr(spice, ignore)] gates: the four local/remote tx-limit, filtering, and RPC-rejection tests now run under spice and non-spice.
Congestion is read from the executed ChunkExtra at the execution head (last_executed(), which trails consensus under spice) so the same assertions hold in both modes.

codecov · 2026-06-23T13:58:01Z

Codecov Report

❌ Patch coverage is 94.64286% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.61%. Comparing base (715e0f9) to head (d80b3da).

Files with missing lines	Patch %	Lines
chain/chain/src/spice/chunk_application.rs	96.66%	0 Missing and 1 partial ⚠️
chain/client/src/chunk_producer.rs	83.33%	1 Missing ⚠️
chain/client/src/rpc_handler.rs	90.90%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #15960   +/-   ##
=======================================
  Coverage   72.60%   72.61%           
=======================================
  Files         952      952           
  Lines      205109   205157   +48     
  Branches   205109   205157   +48     
=======================================
+ Hits       148914   148965   +51     
+ Misses      51218    51217    -1     
+ Partials     4977     4975    -2

Flag	Coverage Δ
pytests-nightly	`1.10% <0.00%> (-0.01%)`	⬇️
unittests	`69.53% <5.35%> (-0.02%)`	⬇️
unittests-nightly	`69.51% <5.35%> (-0.03%)`	⬇️
unittests-spice	`67.00% <94.64%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot

Pull request overview

Ports congestion-control integration tests from the legacy TestEnv framework to test-loop so they can exercise the real (including spice) transaction admission and execution pipeline.

Changes:

Add test-loop versions of congestion-control limit/filter/RPC-rejection tests with a custom runtime config.
Wire spice transaction-admission to derive per-shard congestion from executed ChunkExtras (via certified execution results) in both RPC tx submission and chunk production.
Remove the legacy integration-tests congestion-control module.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
test-loop-tests/src/utils/transactions.rs	Clarifies docs for `run_txs_parallel_on`.
test-loop-tests/src/tests/spice/congestion.rs	Tightens module-level and inline comments for the spice congestion/bandwidth lagging-execution test.
test-loop-tests/src/tests/mod.rs	Registers the new `congestion_control_limits` test module.
test-loop-tests/src/tests/congestion_control.rs	Comment-only clarification in existing congestion-control test.
test-loop-tests/src/tests/congestion_control_limits.rs	New `test-loop` test suite covering tx inclusion limits and RPC shard-congestion rejection.
integration-tests/src/tests/features/mod.rs	Removes the legacy congestion-control feature test module entry.
integration-tests/src/tests/features/congestion_control.rs	Deletes the legacy `TestEnv` congestion-control tests.
chain/client/src/rpc_handler.rs	For spice, derive receiver-shard congestion from executed `ChunkExtra`s (certified) for RPC tx rejection.
chain/client/src/chunk_producer.rs	For spice, supply per-shard congestion info (from executed `ChunkExtra`s) into `PrepareTransactionsBlockContext`.
chain/chain/src/spice/core.rs	Extracts a helper to read certified execution results from `DBCol::execution_results`.
chain/chain/src/spice/chunk_application.rs	Adds `spice_block_congestion_info` helper to build tx-admission congestion info from executed `ChunkExtra`s.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Port the four congestion-control integration tests (transaction inclusion limits, filtering, and RPC rejection) from the TestEnv framework to test-loop, as phase 1 (non-spice parity) of the migration. TestEnv stubs spice's execution actors and so can never exercise spice; test-loop runs the real pipeline. Faithful port: single node tracking all shards, custom RuntimeConfig (zeroed wasm op cost + pinned congestion-control params), synchronous process_tx for pool-fill bursts and the ShardCongested assertion, and per-chunk tx-count measurement via one-block stepping. Tests are ignored under spice for now (congestion gating reads a lagging signal there — phase 2). The integration-tests originals are kept as the parity reference.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

darioush

I think the test-loop could maybe be simplified a bit more in a follow-up.

darioush · 2026-06-25T14:03:44Z

    ) -> Option<Arc<ChunkExecutionResult>> {
-        let key = get_execution_results_key(block_hash, shard_id);
-        self.chain_store.store().caching_get_ser(DBCol::execution_results(), &key)
+        get_execution_result_from_store(&self.chain_store, block_hash, shard_id)


seems this method could be inlined now.

darioush · 2026-06-25T14:36:08Z

+                let mut prev_block_context =
                    PrepareTransactionsBlockContext::new(prev_block, &*self.epoch_manager)?;


nit: I kind of dislike mutating variables just returned from a constructor. I think we could modify new to take the header and congestion info and pass the block's congestion info in the non-spice case.

darioush · 2026-06-25T14:46:50Z

+// time the live parameters change.
+fn set_default_congestion_control(config_store: &RuntimeConfigStore, config: &mut RuntimeConfig) {
+    #[allow(deprecated)]
+    let cc_protocol_version = ProtocolFeature::_DeprecatedCongestionControl.protocol_version();


let's preserve the TODO about updating this.

darioush · 2026-06-25T14:48:41Z

+    node.client().runtime_adapter.get_runtime_config(protocol_version).congestion_control_config
+}
+
+// ---- transaction builders (mirror the integration-test helpers) ----


now that integration test is gone, let's remove references to it.

darioush · 2026-06-25T15:07:28Z

+
+/// Congestion info of the shard's last executed chunk, from its `ChunkExtra` at
+/// `last_executed()` (the execution head, which trails consensus under spice).
+fn head_congestion_info(node: &TestLoopNode, shard_id: ShardId) -> CongestionInfo {


in the old test non-spice would use something like:
head_block.chunks()[shard].congestion_info() → congestion from block H - 1

New test-loop test:
get_chunk_extra(head_hash, shard).congestion_info() → congestion from block H

darioush · 2026-06-25T15:24:43Z

+fn head_height(env: &TestLoopEnv) -> BlockHeight {
+    env.validator().head().height
+}
+
+/// Advance the chain by exactly one block.
+fn produce_one_block(env: &mut TestLoopEnv) {
+    env.validator_runner().run_for_number_of_blocks(1);
+}


nit: I would suggest to inline these simple methods.

ssavenko-near force-pushed the slavas/spice-congestion branch 2 times, most recently from d14022d to 896cf80 Compare June 24, 2026 15:08

ssavenko-near force-pushed the slavas/spice-congestion-p2 branch from 01da9fb to 885e8e1 Compare June 24, 2026 15:14

Base automatically changed from slavas/spice-congestion to master June 24, 2026 15:55

ssavenko-near force-pushed the slavas/spice-congestion-p2 branch from 8cc01be to 27f19a2 Compare June 24, 2026 21:45

ssavenko-near requested a review from Copilot June 24, 2026 21:46

Copilot started reviewing on behalf of ssavenko-near June 24, 2026 21:47 View session

Copilot AI reviewed Jun 24, 2026

View reviewed changes

ssavenko-near changed the title ~~test: migrate congestion-related tests to test-loop~~ spice: making congestion control work under SPICE. txs backpressure. Jun 24, 2026

ssavenko-near force-pushed the slavas/spice-congestion-p2 branch from 27f19a2 to 80ff3a0 Compare June 24, 2026 23:02

ssavenko-near changed the title ~~spice: making congestion control work under SPICE. txs backpressure.~~ feat(spice): making congestion control work under SPICE. txs backpressure. Jun 24, 2026

ssavenko-near requested a review from Copilot June 24, 2026 23:05

Copilot started reviewing on behalf of ssavenko-near June 24, 2026 23:05 View session

Copilot AI reviewed Jun 24, 2026

View reviewed changes

Comment thread chain/client/src/rpc_handler.rs Outdated

ssavenko-near force-pushed the slavas/spice-congestion-p2 branch from 80ff3a0 to 7a86081 Compare June 24, 2026 23:17

ssavenko-near marked this pull request as ready for review June 24, 2026 23:26

ssavenko-near requested a review from a team as a code owner June 24, 2026 23:26

ssavenko-near requested review from Copilot and darioush June 24, 2026 23:26

Copilot started reviewing on behalf of ssavenko-near June 24, 2026 23:27 View session

Copilot AI reviewed Jun 24, 2026

View reviewed changes

Comment thread test-loop-tests/src/tests/congestion_control_limits.rs Outdated

Comment thread chain/chain/src/spice/core.rs

ssavenko-near added 2 commits June 25, 2026 11:46

implement tx backpressure & enable spice tests

d80b3da

ssavenko-near force-pushed the slavas/spice-congestion-p2 branch from 7a86081 to d80b3da Compare June 25, 2026 09:49

ssavenko-near requested a review from Copilot June 25, 2026 09:59

Copilot started reviewing on behalf of ssavenko-near June 25, 2026 09:59 View session

Copilot AI reviewed Jun 25, 2026

View reviewed changes

darioush approved these changes Jun 25, 2026

View reviewed changes

		let mut prev_block_context =
		PrepareTransactionsBlockContext::new(prev_block, &*self.epoch_manager)?;

Uh oh!

Conversation

ssavenko-near commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Production

Tests

Uh oh!

codecov Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

darioush left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

darioush Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

darioush Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

darioush Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

darioush Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

darioush Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

darioush Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ssavenko-near commented Jun 23, 2026 •

edited

Loading

codecov Bot commented Jun 23, 2026 •

edited

Loading

darioush left a comment •

edited

Loading