fix https://github.com/NVIDIA/TensorRT-Edge-LLM/issues/211: advance the Philox offset per sampled token

[filipemartinsubrobotics]

What does this PR do?

Type of change: Bug fix

Overview: Both LLM sampler call sites called topKtopPSamplingFromLogits without a philoxOffset, so it took its declared default of 0 at every decode step. curand_init(seed, batchIdx, offset) is keyed on that offset, so a single uniform was drawn for an entire sequence and sampling collapsed onto the argmax — temperature and top_p selected which deterministic path was taken but never produced a sample. This advances a per-token offset at both sites, reset per sequence through the existing resetForNewSequences hook.

Usage

No API change. Sampling parameters in the input JSON now take effect:

# {"batch_size": 1, "temperature": 0.3, "top_p": 0.8, "max_generate_length": 120, "requests": [...]}
llm_inference --engineDir engines/llm --checkpointDir onnx/llm \
  --inputFile req.json --outputFile out.json

Before: output byte-identical to temperature: 0.0. After: differs from greedy, and still reproducible for a given input.

Measured on an INT4-AWQ 2.44 B Cosmos reasoner, 120 generated tokens, three runs each:

temperature	top_p	before	after
0.0	—	greedy	unchanged, byte-identical
0.3	0.80	identical to greedy	differs, reproducible
1.0	0.95	identical to greedy	differs, reproducible

Before the fix, four configurations x three runs plus the greedy baseline shared one sha256 over 679 characters.

Pull Request Checklist

Pre-commit Checks

• I have installed pre-commit by running pip install pre-commit.
• I have installed the hooks with pre-commit install.
• I have run the hooks manually with pre-commit run --all-files and fixed any reported issues.

(Run scoped to the five changed files: clang-format, codespell, CRLF and license hooks all pass. git-clang-format corrected a lambda brace — the repo sets BeforeLambdaBody: false, which my first draft got wrong.)

Tests

• Tests have been added or updated as needed.
• All tests are passing.

SamplingTest.AdvancingPhiloxOffsetChangesSampledToken in unittests/cpp/sampler/samplingTests.cpp pins the offset contract: distinct offsets must be able to select distinct tokens, and a repeated offset must reproduce.

One caveat, stated rather than buried: this test passes on the unpatched tree. The kernel was never at fault — the bug was two callers — so it documents the invariant a caller must uphold rather than guarding the regression. Catching a non-advancing caller would need a decoder-level test against a real runtime, which I have not written. If you would like that, say so and I will attempt it.

Documentation

• Updated any necessary documentation — none needed; no user-facing behaviour is documented differently. The --help text already said these parameters belong in the input JSON, which is now true in effect as well as in intent.

Compatibility

• The change is backward compatible — greedy decoding is byte-identical, and sampled output remains reproducible for a fixed input.

Additional Information

Fixes #211. My first diagnosis in that issue was wrong and is corrected in a comment there: the parse was never missing.

What identifies this as an omission rather than a design choice is that the TTS talker path already varies the offset deliberately — qwen3OmniTTSRuntime.cpp:4085 passes /*philoxOffset=*/reprefillOffset under a comment about varying it per call.

Out of scope on purpose: six TTS call sites carry the same omission (qwen3OmniTTSRuntime.cpp 2186, 2446, 2558, 2660, 2760, 4326). I have left them alone to keep this to one concern, and cannot test the TTS path. Happy to open a follow-up.

A fixed seed (42) is retained so a given input reproduces; the offset supplies the per-step variation. If you would prefer the seed to be user-settable, that is a separate feature and I have not added it.

Verified on DGX B300 (SM103), CUDA 13.0, TensorRT 10.13.2, Edge-LLM v0.10.1 (e8b2952).