Context
The per-die (per-XCD) dispatch work for the MI300X calibration models a single
kernel's workgroups being dispatched across the 8 XCDs in parallel. To preserve
the ability to run multiple kernels concurrently, the Command Processor's
dispatcher pool is kept as per-kernel queues, and per-die dispatch is modeled
within each kernel's dispatch (8 per-die CU sub-pools + per-die rate gates,
shared across concurrent kernels via the partitioned CU pool).
This leaves the concurrent-kernel dispatch policy implicit. It should be an
explicit, configurable knob rather than a hard-coded behavior.
Ask
Make the concurrent-kernel dispatch policy configurable, e.g.:
- full-overlap (proposed default): every in-flight kernel may spread across
all 8 dies and they contend for shared CU resources (closest to today's
behavior).
- die-partition: concurrent kernels are assigned disjoint subsets of dies
(spatial partitioning across XCDs).
- serialize: one kernel dispatches at a time (drain before the next starts).
Notes / open questions
- Real-hardware behavior here is not well documented publicly; the default
should be labeled a modeling assumption, not a validated HW policy.
- Related knob: the per-workgroup -> die assignment policy (round-robin vs
contiguous vs greedy) is likewise unverified. It does not affect
uniform-grid calibration (e.g. empty_kernel), but it matters for
cache-locality-sensitive and load-imbalanced kernels, so it should also be
configurable.
- In the simple per-kernel-lane model, two concurrent kernels dispatching to the
same die would each independently run that die's rate gate, doubling its
per-die dispatch bandwidth. A faithful model would time-share a single die's
dispatch bandwidth across kernels (matches real per-XCD dispatch hardware).
Filed as a follow-up while implementing per-die dispatch on branch
mi300x/clock-2.1ghz-dispatch (MI300X calibration, PR #277).
Context
The per-die (per-XCD) dispatch work for the MI300X calibration models a single
kernel's workgroups being dispatched across the 8 XCDs in parallel. To preserve
the ability to run multiple kernels concurrently, the Command Processor's
dispatcher pool is kept as per-kernel queues, and per-die dispatch is modeled
within each kernel's dispatch (8 per-die CU sub-pools + per-die rate gates,
shared across concurrent kernels via the partitioned CU pool).
This leaves the concurrent-kernel dispatch policy implicit. It should be an
explicit, configurable knob rather than a hard-coded behavior.
Ask
Make the concurrent-kernel dispatch policy configurable, e.g.:
all 8 dies and they contend for shared CU resources (closest to today's
behavior).
(spatial partitioning across XCDs).
Notes / open questions
should be labeled a modeling assumption, not a validated HW policy.
contiguous vs greedy) is likewise unverified. It does not affect
uniform-grid calibration (e.g. empty_kernel), but it matters for
cache-locality-sensitive and load-imbalanced kernels, so it should also be
configurable.
same die would each independently run that die's rate gate, doubling its
per-die dispatch bandwidth. A faithful model would time-share a single die's
dispatch bandwidth across kernels (matches real per-XCD dispatch hardware).
Filed as a follow-up while implementing per-die dispatch on branch
mi300x/clock-2.1ghz-dispatch(MI300X calibration, PR #277).