Repository navigation
perf(shader): compute v_pk_add_f16 and v_pk_mul_f16 with native f16 math - #3531
Conversation
|
The conventions and relinker checks stop during checkout, before reaching your code: the logs report |
|
I added @jweidenh's scanner fix from #3557's review for this PR's test (f139b34, with them as author): Runtime on the RTX 4060 (same build, compiled for the device and for the same target without |
|
Rebased onto current Checked on RDNA2 hardware (gfx1035, f16 denormals kept): |
f139b34 to
9e2711a
Compare
v_pk_add_f16 and v_pk_mul_f16 computed each half in f32 and rounded it back to f16, then fixed the NaN, FP16_OVFL and clamp results on the bits. On a device with native f16 that rounds to nearest even and keeps f16 denormals (the Float16, RoundingModeRTE and DenormPreserve capabilities that boykopovar#1252 already checks), the arithmetic is now one f16vec2 OpFAdd or OpFMul decorated NoContraction, and the same NaN, FP16_OVFL and clamp rules run on its result bits. The results don't change. f32 has at least 2 * 11 + 2 significand bits, so rounding an f16 sum or product to f32 and then to f16 gives the correctly rounded f16 result. Vulkan requires f16 OpFAdd and OpFMul to be correctly rounded, the module keeps f16 denormals, and any NaN they produce is replaced by the NaN the existing rules select. v_pk_fma_f16 and v_pk_fmac_f16 keep the f32 path: Vulkan gives GLSL Fma only the precision of OpFMul followed by OpFAdd, so a native f16 Fma may round twice. New test agc_driver_packed_float16_native compiles one shader with packed add, mul, fma and fmac (op_sel, op_sel_hi, neg_lo, neg_hi and clamp variants) with and without Float16, checks that only add and mul use f16vec2 math, and compares every result bit for bit on random and special inputs (zeros, denormals, infinities, NaNs, values near the f16 maximum) in nine float modes.
FindF16x2Math read spirv[offset + 1] for every instruction once the f16 vector type was known, so a final one-word OpFunctionEnd read past the module. Check the opcode and its minimum word count before reading operands.
|
Rebased onto current Checked on RDNA2 hardware (gfx1035, f16 denormals kept): |
9e2711a to
2ac8c72
Compare
What
v_pk_add_f16andv_pk_mul_f16now use native f16 math when the device has it. Each half used to be converted to f32, computed in f32 and rounded back to f16. With native f16 the arithmetic is onef16vec2OpFAddorOpFMul, decoratedNoContraction. The existing NaN,FP16_OVFLand clamp rules still run on the result bits.Background. NVIDIA GPUs have packed f16 add and multiply instructions (
HADD2,HMUL2from Turing on), and so do Intel Xe and AMD RDNA.Float16,RoundingModeRTEandDenormPreservecapabilities.VulkanDeviceadds those only when the device reportsshaderFloat16together with RTE rounding, preserved denormals and preserved signed zero, inf and NaN for f16, the check feat(recompiler): convert f32 to f16 natively when the device rounds 16-bit floats to nearest even #1252 introduced.FLOAT_MODEother than f16 denormals kept with round to nearest even. So the native path covers every mode that packed f16 supports.v_pk_fma_f16andv_pk_fmac_f16keep the f32 path with the round-to-odd sum from fix(shader): round the f16 fused multiply-adds once #2000. Vulkan gives GLSLFmaonly the precision ofOpFMulfollowed byOpFAdd, for f16 as for f32, so a native f16Fmamay round twice (the same reasonv_fma_f64is emulated, feat(shader): implement basic f64 VALU ops #446, and perf(shader): round v_cvt_pknorm and v_cvt_pk_u8 in f32 #2410's review).OpFmaKHRfromVK_KHR_shader_fma(shaderFmaFloat16) would guarantee a fused f16 fma; none of the drivers I can test exposes it yet.Before / after on an RTX 4060 Laptop (Ada, driver 591.86). SASS instructions per instruction come from
APS5_PIPELINE_STATSbinary size over a chain of 16 instructions,FLOAT_MODE0xc0:v_pk_add_f16v_pk_mul_f16The other 334 VOP1, VOP2, VOP3, VOP3P and VOPC opcodes in the same sweep compile to the same size. Most of what's left in each instruction is the NaN rule on the result bits.
Runtime on the same GPU, one build: each instruction is compiled once for the device and once for the same target without
Float16(the f32 pathmaintakes), as a dependent chain of 2048 instructions over 16384 groups of 32 threads,FLOAT_MODE0xc0; median of 15 interleaved dispatches. Av_fma_f32chain takes 6.7 ms.v_pk_add_f16v_pk_mul_f16Why the results don't change. Both paths give the correctly rounded f16 result before the NaN,
FP16_OVFLand clamp rules run.OpFAddandOpFMulto be correctly rounded, the module declares round to nearest even and preserved denormals for f16, andNoContractionkeeps the driver from fusing them with anything.0xfe00for an invalid operation.FP16_OVFLturns an infinite result from finite sources into ±0x7bff, asclampF16Overflowdoes in f32.clampF16Bits. Clamping commutes with round to nearest because 0 and 1 are representable, and withDX10_CLAMPany NaN clamps to +0 on both paths.Where it matters. Shaders compiled with packed f16 math. On devices without the three capabilities (Pascal has no
shaderFloat16; lavapipe doesn't preserve f16 denormals) the translation is unchanged.Tested
Windows 11, MinGW-w64 GCC 15.2.0, Release,
mainat 1c75f4d. RTX 4060 Laptop (driver 591.86), Intel UHD Graphics (Raptor Lake, driver 101.6790) and lavapipe from Mesa 25.2.8 (the version CI installs; Windows build,GALLIUM_OVERRIDE_CPU_CAPS=avx).agc_driver_packed_float16_native:Float16, and compares every result bit for bit. The shader coversv_pk_add_f16,v_pk_mul_f16,v_pk_fma_f16andv_pk_fmac_f16, withop_sel,op_sel_hi,neg_lo,neg_hiand clamp variants; the fma rows check that the fma forms are unaffected.FLOAT_MODE, and 0xc0 with every combination ofDX10_CLAMP,IEEE_MODEandFP16_OVFL.f16vec2add and mul and nof16vec2Fma, and that the build withoutFloat16uses nof16vec2math. It is skipped on a device without native f16.FP16_OVFLfix on the native path (v_pk_mul_f16gives 0xfc00 where the f32 path gives 0xfbff), and leaving out the bit fixes (v_pk_mul_f16with clamp gives 0xfc00, the unclamped -inf, where the f32 path gives 0).agc_driver_packed_alu,agc_driver_float16_fma_rounding,agc_driver_fp16_overflow,agc_driver_f16_min_max_nanandagc_driver_float16_ternary_clamp) pass on all three devices; the Intel UHD skips the three that needFloat64or interpolation features, lavapipe the native test andagc_driver_interpolation_f16.agc_driver_*,agc_shader_*,agc_spirv_*,vector_*,agc_masked_select,agc_validate_program,agc_denormal*,agc_float16_*; 320 tests):agc_driver_flat_loads*,agc_driver_global_loads*,agc_driver_flat_apertures_2,agc_driver_flat_apertures_4(then0xc0000409) andagc_driver_pixel_interlock, which fail the same way onmain.agc_driver_image_plane_address_on_line_implicit_lod, which fails the same way withmain's recompiler.Limits.
v_pk_fma_f16,v_pk_fmac_f16,v_pk_min_f16,v_pk_max_f16, the scalar f16 instructions and the f16 transcendentals still compute in f32. Scalar add, sub and mul follow in a separate PR stacked on this one; f16 min/max is a separate PR.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt.mdfiles or images added to the repository (attach them to this PR)