README.md

Benchmark index

Use this index to find benchmark source and execution requirements. Recorded measurements and conclusions are in the evaluation index; reports may cover only part of the current parameter matrix.

Run

Run from the repository root using the SDK selected by global.json. Benchmarks target .NET 10 and require its runtime. List available methods, then select a class or method with --filter:

dotnet run --project tests/RequestBatcher.Benchmarks/RequestBatcher.Benchmarks.csproj \
  --configuration Release -- --list flat

dotnet run --project tests/RequestBatcher.Benchmarks/RequestBatcher.Benchmarks.csproj \
  --configuration Release -- --filter '*DeduplicationConcurrencyBenchmarks*'

Check the selected class's parameters and job settings before running. Use --artifacts <directory> to choose an output directory.

Pre-commit benchmarks and correctness stress tests

Follow the critical-path validation rules before committing changes to submission, scheduling, completion, lifecycle, or observability hot paths. Select the relevant workloads below. BenchmarkDotNet repeatedly measures bounded waves; the larger response stress test checks correctness under concurrent load. Run both when changing shared submission, scheduling, or response-completion logic.

Changed path	Workloads	Checks / measurements
Request/response correlation and completion	ResponseStressTests, ResponseCompletionBenchmarks	Stress: single, explicit-batch, and mixed submissions, each with and without partition keys; every response value, batch response order, and exactly-once handling. Benchmarks: 256-request waves, success/failure/cancellation and explicit-response success, time and allocation.
Response materialization and assignment	ResponseHotPathBenchmarks, ResponseAssignmentBenchmarks, ResponseFactoryBufferBenchmarks	Typed/static-delegate single and explicit response submissions with synchronous/20 ms handlers; response-factory and ordered-sequence assignment; temporary-buffer allocation versus pooling for value/reference responses. Validate every response and report time/allocation per wave or batch.
Explicit admission, shared completion, backpressure	ExplicitSubmissionBenchmarks, ExplicitCompletionBenchmarks	Mixed/group submissions; capacity waiting, partial cancellation, failure fan-out, and per-ID processing validation.
Queue pulling and dispatch capacity	QueuePullBenchmarks, DispatchCapacityBenchmarks, DispatchLoopOverheadBenchmarks	4,096 requests across partition/concurrency settings with synchronous/20 ms handlers; 1,024-request synchronous single/explicit waves with batch limits 1/32 to expose dispatch overhead.
Tracing or metrics	TracingIoOverheadBenchmarks, MetricsIoOverheadBenchmarks	1,024 requests per wave, single or groups of 32, 20 ms handlers, per-ID exactly-once validation; disabled/sampling or metric-reader configurations. These use request-only handlers.
Named tracing and synchronous dispatch	NamedTracingHotPathBenchmarks	256-request single/explicit waves, no I/O or metrics, disabled/propagation-only/recorded listeners, and normal/suppressed submission flow. Checks every ID, dispatch bounds, span names, flags, completed span counts, and suppression ownership.
Execute metrics completion	ExecuteMetricsTests, DelegateExecuteMetricsTests	Enabled metrics with concurrent single and explicit response submissions; checks response values, collection order, exactly-once handling, and one duration sample per public call. Delegate cases also cover Handler binding, tracing on/off, and cancellation.
SDK sampling and trace links	TracingStressTests	32 waves of 512 requests at 0%/100% sampling, mixed submissions, exactly-once handling and actual-batch links; also mixed-sampling policy tests. No Docker needed for this class.
Request-only delegate forwarding	DelegateRequestOnlyStressTests, DelegateRequestOnlyBenchmarks, DelegateForwardingBenchmarks	Eight producers, single/list/mixed submissions, four 2,048-request waves per case; validates every request's transformation and exactly-once handling. Benchmarks measure full-pipeline and isolated forwarding allocation.
Database paths	PostgreSQL/Redis benchmarks below	Matched direct/batched operations, result or persisted-data validation; require Docker.

Run the response correctness stress test:

dotnet test tests/RequestBatcher.Tests/RequestBatcher.Tests.csproj \
  --configuration Release --filter 'FullyQualifiedName~ResponseStressTests' \
  --logger 'console;verbosity=normal'

It uses eight concurrent producers, batch size 32, concurrency eight, pending capacity 256, and explicit groups alternating between 17 and 129 requests, with smaller final groups. Producers submit their round's operations before awaiting their results, allowing queue capacity to be exceeded by in-flight work. Each of six cases sends three rounds of 10,000 distinct requests through the same pipeline: 180,000 validated responses per target framework. The asynchronous handler assigns responses in reverse entry order; assertions do not assume handler execution order. The test runs in the ordinary unit-test suite on .NET 8 and .NET 10. Its elapsed time is test execution time, not a throughput or latency benchmark result.

Run a relevant performance benchmark, for example:

dotnet run --project tests/RequestBatcher.Benchmarks/RequestBatcher.Benchmarks.csproj \
  --configuration Release -- --filter '*ResponseCompletionBenchmarks*' \
  --artifacts /private/tmp/requestbatcher-response-benchmarks

Run the SDK tracing stress class without starting containers:

dotnet test tests/it/RequestBatcher.Observability.IntegrationTests/RequestBatcher.Observability.IntegrationTests.csproj \
  --configuration Release --filter 'FullyQualifiedName~TracingStressTests'

Keep raw measurements, per-run exports, and logs local. Record concise time/allocation conclusions, workload, runtime, commands, and correctness outcome in the evaluation summaries. For optimization claims, compare the same workload against the baseline with the same environment. Benchmark startup/smoke success alone is insufficient evidence of an improvement.

Batching and handler APIs

These benchmarks run in process without Docker. Some use simulated asynchronous delays.

Source / filter class	Comparison
DeduplicationConcurrencyBenchmarks	Batch size, concurrency, duplicates and submission size; bulk reads, chunked WhenAll and per-key fan-out with simulated I/O.
TaskWhenAllOverheadBenchmarks	WhenAll versus awaiting already-started tasks, with completed and pending task sets.
DelayDownstreamOverheadBenchmarks	Direct and batched calls with simulated 50 ms I/O. Paced arrivals by default; set REQUEST_BATCHER_TRAFFIC=burst for bursts.
QueuePullBenchmarks	One consumer drains 4,096 prefilled BufferQueue records; batch limits 32/128 and 1/16 partitions. See the pull capacity evaluation.
DispatchCapacityBenchmarks	Direct full batches versus public explicit submission, synchronous/20 ms handlers, concurrency 8/32 and 1/16 partitions.
DispatchLoopOverheadBenchmarks	Concurrent singles and one explicit submission of 1,024 distinct requests, synchronous handlers, batch limits 1/32 and concurrency eight. Validates every ID exactly once and dispatch bounds; reports time and allocation per request.
PartitionKeySubmissionBenchmarks	Concurrent single submissions versus one explicit submission under partition-key routing.
PartitionSubmissionBenchmarks	Two independent experiments: filter *SingleSubmissions* or *ExplicitSubmissions*. Each varies Partition 1/4 at BatchSize 64 and MaxConcurrency 4; validates every response, group order, exactly-once handling and concurrency. Records batch formation, query work, time and allocation. See the evaluation.
ExplicitSubmissionBenchmarks	Explicit groups versus mixed single/group submissions.
NamedBatchingBenchmarks	Named versus unnamed pipelines, single/list submissions and singleton/scoped handlers.
DelegateHandlerBenchmarks	Typed versus cached delegate handlers, with and without scoped service injection.
DelegateRequestOnlyBenchmarks	Typed singleton versus no-service delegate; 256-request single/list waves with Yield or 20 ms work and per-ID validation.
DelegateForwardingBenchmarks	Isolated typed/delegate forwarding of an incomplete operation, with exactly-once completion validation.
DelegateServiceInjectionBenchmarks	Typed/delegate service injection across dependency lifetimes and ordinary/keyed services.
BatchScopeBenchmarks	Per-batch scope creation/disposal, root versus scoped singleton resolution, and actual delegate invocation with a singleton. Synchronous isolated work with identity, result, exactly-once processing and disposal checks; no Docker. See the scope cost evaluation.
DelegateReturnTypeBenchmarks	Typed handlers versus delegates returning void, completed Task or completed ValueTask.
DelegateFirstSubmissionBenchmarks	First submission through a fresh typed/delegate pipeline; includes initial delegate binding, excludes provider creation.

Completion and allocation

Source / filter class	Measures
PackageComparisonBenchmarks	Public typed/static-delegate response handlers using factory assignment, with 0 ms and 20 ms simulated downstream delay per handler batch: 256 single submissions, one explicit submission, or 128 singles plus eight groups of 16. Validates response values, explicit order, exactly-once handling, and dispatch bounds. The package comparison runner builds identical isolated .NET 10 workloads against two exact NuGet versions, reverses version order in round two, and records time/allocation per wave and allocation per request.
BatchCompletionBenchmarks	Per-request TCS plus WhenAll versus shared batch completion.
RequestCompletionBenchmarks	Internal pending-request success, failure and cancellation, without the full queue pipeline.
PendingRequestAllocationBenchmarks	One completed pending request, with and without a cancellable caller token.
TracingCompletionBenchmarks	Task, ValueTask, and callback tracing completion bridges, comparing 1,024 pending sources per measured invocation.
ResponseCompletionBenchmarks	Public response success, failure and pre-dispatch cancellation, with request-only and explicit-submission controls.
ResponseHotPathBenchmarks	Matched typed and static-delegate response paths, 256 distinct requests per wave, synchronous/20 ms handlers, and concurrent single/one explicit submission. Every wave checks response values, input order, exactly-once handling, batch size, and concurrency. Reports per wave; use --unrollFactor 1.
ResponseAssignmentBenchmarks	Factory and ordered-sequence response assignment for batches of 32/256. Includes fresh entry construction and per-response validation in both methods; reports per batch.
ResponseFactoryBufferBenchmarks	Experimental pooled temporary buffer versus the production factory overload for int/string responses at batch sizes 1/32/256. Includes identical entry construction and result validation; clears references on pool return. Reports per batch and does not pool arrays returned to callers.
ExplicitCompletionBenchmarks	Public explicit-submission success, failure, partial cancellation, capacity waiting and mixed submissions.
BatchErrorAggregationBenchmarks	Shared completion accounting for success, repeated exception references and distinct exceptions, sequentially and concurrently.

PostgreSQL and Redis

These benchmarks start temporary databases through Testcontainers and require a running Docker daemon. Container and data preparation are outside measurement. Their results are separate from the historical scheduling-probe experiments.

Source / filter class	Comparison
PostgreSqlWriteBenchmarks	Direct INSERTs, single submissions and explicit batch submission.
PostgreSqlPriceQueryBenchmarks	Direct product queries versus batch-local deduplication and bulk reads.
PostgreSqlPriceQueryRateBenchmarks	Paced direct and batched queries across target arrival rates and concurrency limits.
RedisReadBenchmarks	Direct GETs, single submissions and explicit batch submission.
RedisWriteBenchmarks	Direct SETs, single submissions and explicit batch submission.

Paced explicit-submission probe

For paced explicit submissions against the current public API, use MeasurePacedBatchFanout.cs. It is a bounded load probe rather than a BenchmarkDotNet throughput loop: arrivals do not wait for earlier responses. See the paced batch fan-out evaluation for its direct bulk-query baseline, correctness checks, runtime, commands, and latency results.

Isolated scheduling probe

SchedulingProbe uses --scheduling-probe, separately from BenchmarkDotNet. It requires evaluation hooks such as EvaluationOffload, which are absent from production source. The arrival scheduling runner injects these hooks into an isolated source copy before invoking the probe. See the arrival-scheduling evaluation for the recorded experiment.

Observability

The optional RequestBatcher.OpenTelemetry package exposes AddRequestBatcherInstrumentation() for application tracing. Historical benchmark results in this index are not tracing measurements and should not be read as results with the new instrumentation enabled. See the observability design for span lifetime, links, sampling, and performance boundaries.

Source / filter class	Comparison
ObservabilityOverheadBenchmarks	Disabled, metrics-only, and tracing-plus-metrics OpenTelemetry providers around 256 named async pricing requests with a 20 ms handler delay.
TracingIoOverheadBenchmarks	Published versions and source snapshots, with tracing disabled or SDK sampling at 0%, 1%, 10%, and 100%, for single and explicit submissions with 20 ms simulated downstream calls. See the measurement and reproduction guide.
NamedTracingHotPathBenchmarks	Named synchronous single/explicit dispatch with tracing disabled, propagation-only, or recorded and normal/suppressed submission flow. Validates per-ID processing, completed span names/flags/counts, and suppression ownership; reports per 256-request wave.
MetricsIoOverheadBenchmarks	Metrics disabled, SDK aggregation only, and periodic collection every 1,000 ms or 100 ms, with tracing disabled and 20 ms simulated downstream calls. Validates all four histograms and the backlog gauge, including zero after each completed run. See the metrics evaluation.

The corresponding observability overhead evaluation records the measured results and workload limitations.