Skip to content

perf(shader): compute v_pk_add_f16 and v_pk_mul_f16 with native f16 math - #3531

Merged
adriadam10 merged 2 commits into
boykopovar:mainfrom
karanvora2599:perf/shader-native-packed-f16
Oct 11, 2026
Merged

adriadam10 merged 2 commits into
boykopovar:mainfrom
karanvora2599:perf/shader-native-packed-f16

Conversation

@karanvora2599

@karanvora2599 karanvora2599 commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

What

v_pk_add_f16 and v_pk_mul_f16 now use native f16 math when the device has it. Each half used to be converted to f32, computed in f32 and rounded back to f16. With native f16 the arithmetic is one f16vec2 OpFAdd or OpFMul, decorated NoContraction. The existing NaN, FP16_OVFL and clamp rules still run on the result bits.

Background. NVIDIA GPUs have packed f16 add and multiply instructions (HADD2, HMUL2 from Turing on), and so do Intel Xe and AMD RDNA.

Before / after on an RTX 4060 Laptop (Ada, driver 591.86). SASS instructions per instruction come from APS5_PIPELINE_STATS binary size over a chain of 16 instructions, FLOAT_MODE 0xc0:

Instruction SASS before SASS after Registers before → after
v_pk_add_f16 72.5 22.5 33 → 27
v_pk_mul_f16 72.5 22.5 33 → 27

The other 334 VOP1, VOP2, VOP3, VOP3P and VOPC opcodes in the same sweep compile to the same size. Most of what's left in each instruction is the NaN rule on the result bits.

Runtime on the same GPU, one build: each instruction is compiled once for the device and once for the same target without Float16 (the f32 path main takes), as a dependent chain of 2048 instructions over 16384 groups of 32 threads, FLOAT_MODE 0xc0; median of 15 interleaved dispatches. A v_fma_f32 chain takes 6.7 ms.

Chain f32 path native
v_pk_add_f16 34.6 ms 11.2 ms
v_pk_mul_f16 33.3 ms 11.3 ms

Why the results don't change. Both paths give the correctly rounded f16 result before the NaN, FP16_OVFL and clamp rules run.

  • f32 has 24 significand bits, at least 2 × 11 + 2, so rounding the exact sum or product to f32 and then to f16 gives the same value as rounding it once.
  • Vulkan requires f16 OpFAdd and OpFMul to be correctly rounded, the module declares round to nearest even and preserved denormals for f16, and NoContraction keeps the driver from fusing them with anything.
  • Any NaN a native operation produces is replaced by the NaN the existing rules select from the source bits: quieted in IEEE mode, and the default 0xfe00 for an invalid operation.
  • FP16_OVFL turns an infinite result from finite sources into ±0x7bff, as clampF16Overflow does in f32.
  • Clamp uses clampF16Bits. Clamping commutes with round to nearest because 0 and 1 are representable, and with DX10_CLAMP any NaN clamps to +0 on both paths.

Where it matters. Shaders compiled with packed f16 math. On devices without the three capabilities (Pascal has no shaderFloat16; lavapipe doesn't preserve f16 denormals) the translation is unchanged.

Tested

Windows 11, MinGW-w64 GCC 15.2.0, Release, main at 1c75f4d. RTX 4060 Laptop (driver 591.86), Intel UHD Graphics (Raptor Lake, driver 101.6790) and lavapipe from Mesa 25.2.8 (the version CI installs; Windows build, GALLIUM_OVERRIDE_CPU_CAPS=avx).

  • New agc_driver_packed_float16_native:
    • It compiles one shader twice, with and without Float16, and compares every result bit for bit. The shader covers v_pk_add_f16, v_pk_mul_f16, v_pk_fma_f16 and v_pk_fmac_f16, with op_sel, op_sel_hi, neg_lo, neg_hi and clamp variants; the fma rows check that the fma forms are unaffected.
    • It runs in nine float modes: no FLOAT_MODE, and 0xc0 with every combination of DX10_CLAMP, IEEE_MODE and FP16_OVFL.
    • Inputs are 32 batches of random halves weighted toward zeros, denormals, infinities, quiet and signaling NaNs, values near the f16 maximum and small normals.
    • It also checks that the native build uses f16vec2 add and mul and no f16vec2 Fma, and that the build without Float16 uses no f16vec2 math. It is skipped on a device without native f16.
    • Results: 294,912 halves identical on the RTX 4060 and on the Intel UHD; skipped on lavapipe.
    • Two deliberate bugs make it fail: leaving out the FP16_OVFL fix on the native path (v_pk_mul_f16 gives 0xfc00 where the f32 path gives 0xfbff), and leaving out the bit fixes (v_pk_mul_f16 with clamp gives 0xfc00, the unclamped -inf, where the f32 path gives 0).
  • The f16 and packed tests (23, among them agc_driver_packed_alu, agc_driver_float16_fma_rounding, agc_driver_fp16_overflow, agc_driver_f16_min_max_nan and agc_driver_float16_ternary_clamp) pass on all three devices; the Intel UHD skips the three that need Float64 or interpolation features, lavapipe the native test and agc_driver_interpolation_f16.
  • The shader tests (agc_driver_*, agc_shader_*, agc_spirv_*, vector_*, agc_masked_select, agc_validate_program, agc_denormal*, agc_float16_*; 320 tests):
    • RTX 4060: everything passes except the device-lost agc_driver_flat_loads*, agc_driver_global_loads*, agc_driver_flat_apertures_2, agc_driver_flat_apertures_4 (then 0xc0000409) and agc_driver_pixel_interlock, which fail the same way on main.
    • lavapipe 25.2.8: everything passes or is skipped for missing features, except agc_driver_image_plane_address_on_line_implicit_lod, which fails the same way with main's recompiler.

Limits.

  • I haven't run it on AMD hardware.
  • v_pk_fma_f16, v_pk_fmac_f16, v_pk_min_f16, v_pk_max_f16, the scalar f16 instructions and the f16 transcendentals still compute in f32. Scalar add, sub and mul follow in a separate PR stacked on this one; f16 min/max is a separate PR.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • AI-assisted: yes

@MarcZX10

Copy link
Copy Markdown
Contributor

The conventions and relinker checks stop during checkout, before reaching your code: the logs report fatal: No url found for submodule path '3rdparty/imgui' in .gitmodules on base 1e8fff68. This is the shared main checkout issue tracked by #3511. The PR-specific register-context checks pass; the full Linux and Windows builds are skipped under the repository PR policy.

@karanvora2599

Copy link
Copy Markdown
Contributor Author

I added @jweidenh's scanner fix from #3557's review for this PR's test (f139b34, with them as author): FindF16x2Math read spirv[offset + 1] for every instruction once the f16 vector type was known, so the module's final one-word OpFunctionEnd read one word past the end. agc_driver_packed_float16_native still passes on an RTX 4060 and an Intel UHD (294,912 halves).

Runtime on the RTX 4060 (same build, compiled for the device and for the same target without Float16; chains of 2048 instructions over 16384 groups of 32 threads, median of 15 interleaved dispatches): v_pk_add_f16 34.6 → 11.2 ms and v_pk_mul_f16 33.3 → 11.3 ms, against 6.7 ms for a v_fma_f32 chain.

@adriadam10

Copy link
Copy Markdown
Collaborator

Rebased onto current main (the conventions checkout failure was the 3rdparty/imgui gitlink fixed by #3511).

Checked on RDNA2 hardware (gfx1035, f16 denormals kept): v_pk_add_f16, v_pk_mul_f16, the op_sel/neg add and the clamped mul from the test, in the test's nine float modes (DX10_CLAMP × IEEE_MODE × FP16_OVFL), over 3,072 rows covering every pair of 62 edge halves (±0, subnormals, 0x7bff, infinities, quiet and signaling NaNs with payloads) plus random ones. The native path on RADV matches the hardware bit for bit on all 110,592 results, as does the f32 path. agc_driver_packed_float16_native passes on RADV with the native path active and fails without the change; the shader tests pass. Merging.

@adriadam10
adriadam10 force-pushed the perf/shader-native-packed-f16 branch from f139b34 to 9e2711a Compare October 11, 2026 21:32
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 11, 2026
karanvora2599 and others added 2 commits October 11, 2026 23:43
v_pk_add_f16 and v_pk_mul_f16 computed each half in f32 and rounded it
back to f16, then fixed the NaN, FP16_OVFL and clamp results on the
bits. On a device with native f16 that rounds to nearest even and keeps
f16 denormals (the Float16, RoundingModeRTE and DenormPreserve
capabilities that boykopovar#1252 already checks), the arithmetic is now one
f16vec2 OpFAdd or OpFMul decorated NoContraction, and the same NaN,
FP16_OVFL and clamp rules run on its result bits.

The results don't change. f32 has at least 2 * 11 + 2 significand bits,
so rounding an f16 sum or product to f32 and then to f16 gives the
correctly rounded f16 result. Vulkan requires f16 OpFAdd and OpFMul to
be correctly rounded, the module keeps f16 denormals, and any NaN they
produce is replaced by the NaN the existing rules select.

v_pk_fma_f16 and v_pk_fmac_f16 keep the f32 path: Vulkan gives GLSL Fma
only the precision of OpFMul followed by OpFAdd, so a native f16 Fma
may round twice.

New test agc_driver_packed_float16_native compiles one shader with
packed add, mul, fma and fmac (op_sel, op_sel_hi, neg_lo, neg_hi and
clamp variants) with and without Float16, checks that only add and mul
use f16vec2 math, and compares every result bit for bit on random and
special inputs (zeros, denormals, infinities, NaNs, values near the f16
maximum) in nine float modes.
FindF16x2Math read spirv[offset + 1] for every instruction once the f16 vector type was known, so a final one-word OpFunctionEnd read past the module. Check the opcode and its minimum word count before reading operands.
@adriadam10

Copy link
Copy Markdown
Collaborator

Rebased onto current main (the conventions checkout failure was the 3rdparty/imgui gitlink fixed by #3511).

Checked on RDNA2 hardware (gfx1035, f16 denormals kept): v_pk_add_f16, v_pk_mul_f16, the op_sel/neg add and the clamped mul from the test, in the test's nine float modes (DX10_CLAMP × IEEE_MODE × FP16_OVFL), over 3,072 rows covering every pair of 62 edge halves (±0, subnormals, 0x7bff, infinities, quiet and signaling NaNs with payloads) plus random ones. The native path on RADV matches the hardware bit for bit on all 110,592 results, as does the f32 path. agc_driver_packed_float16_native passes on RADV with the native path active and fails without the change; the shader tests pass. Merging.

@adriadam10
adriadam10 force-pushed the perf/shader-native-packed-f16 branch from 9e2711a to 2ac8c72 Compare October 11, 2026 21:53
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 11, 2026
@adriadam10
adriadam10 merged commit 825cbd4 into boykopovar:main Oct 11, 2026
14 checks passed
@karanvora2599
karanvora2599 deleted the perf/shader-native-packed-f16 branch October 12, 2026 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants