Skip to content

Make concurrent-kernel dispatch policy configurable #282

Description

@syifan

Context

The per-die (per-XCD) dispatch work for the MI300X calibration models a single
kernel's workgroups being dispatched across the 8 XCDs in parallel. To preserve
the ability to run multiple kernels concurrently, the Command Processor's
dispatcher pool is kept as per-kernel queues, and per-die dispatch is modeled
within each kernel's dispatch (8 per-die CU sub-pools + per-die rate gates,
shared across concurrent kernels via the partitioned CU pool).

This leaves the concurrent-kernel dispatch policy implicit. It should be an
explicit, configurable knob rather than a hard-coded behavior.

Ask

Make the concurrent-kernel dispatch policy configurable, e.g.:

  • full-overlap (proposed default): every in-flight kernel may spread across
    all 8 dies and they contend for shared CU resources (closest to today's
    behavior).
  • die-partition: concurrent kernels are assigned disjoint subsets of dies
    (spatial partitioning across XCDs).
  • serialize: one kernel dispatches at a time (drain before the next starts).

Notes / open questions

  • Real-hardware behavior here is not well documented publicly; the default
    should be labeled a modeling assumption, not a validated HW policy.
  • Related knob: the per-workgroup -> die assignment policy (round-robin vs
    contiguous vs greedy) is likewise unverified. It does not affect
    uniform-grid calibration (e.g. empty_kernel), but it matters for
    cache-locality-sensitive and load-imbalanced kernels, so it should also be
    configurable.
  • In the simple per-kernel-lane model, two concurrent kernels dispatching to the
    same die would each independently run that die's rate gate, doubling its
    per-die dispatch bandwidth. A faithful model would time-share a single die's
    dispatch bandwidth across kernels (matches real per-XCD dispatch hardware).

Filed as a follow-up while implementing per-die dispatch on branch
mi300x/clock-2.1ghz-dispatch (MI300X calibration, PR #277).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions