Repository navigation
fix(shader): flush ds_add_f32 denormals and NaNs as the hardware does - #5492
adriadam10 merged 2 commits into
Conversation
|
Windows test of #5492: Windows 10 (19045), MinGW-w64 GCC 15.2, Release, AI-assisted (Claude Code). |
CnRJay
left a comment
There was a problem hiding this comment.
A good, measured change. Two notes, one about cost and one about where the measurement comes from.
Hoist the data-only work out of the CAS loop. SharedAtomicFAdd32 goes through AtomicUpdate, so the replacement lambda runs on every CAS retry. Three of its parts depend only on data: flush(data), nan(data) and data | quiet. Under contention (the usual case for ds_add_f32: histograms, reductions, light-list counters), every extra instruction between the OpAtomicLoad and the OpAtomicCompareExchange widens the window in which another lane can change the value, so more iterations fail and retry. Driver compilers may hoist this by LICM, but values that feed an atomic loop aren't always treated as invariant. Computing the flushed data, the data-is-NaN flag and the quieted data once before AtomicUpdate, and keeping only the old-dependent selects inside the loop, makes the loop body as short as main's plus the old-value checks.
RDNA1 against RDNA2. The other LDS float atomics in the same debt entry were measured on gfx1036 (RDNA2), which is the family the console's GPU belongs to. This one was measured on gfx1010 (RDNA1). The denormal and round-mode handling of LDS float atomics did change between some GCN/RDNA generations, so the debt entry should say that the gfx10.3 behaviour is assumed from gfx10.1. A rerun of the oracle on any RDNA2 card would settle it.
The test design is good: the expected values are the measured results rather than a model, and the 111 rows that depend on the host preserving denormals are left out.
|
Correction and update to my Windows test above: (1) I wrote that the GPU was shown by a AI-assisted (Claude Code). |
…t) (#481) * fix(recompiler): give vertex-path shaders the NGG group info Ported from upstream PR boykopovar#5081 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(shader): put the compute wave index in TG_SIZE[11:6] Ported from upstream PR boykopovar#5214 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(shader): take the 32_AR export alpha from channel 1 as GFX10 does Ported from upstream PR boykopovar#5110 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(shader): order buffer and image memory at s_barrier Ported from upstream PR boykopovar#5117 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(shader): decode GFX10 scratch offsets as one dword Ported from upstream PR boykopovar#5426 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(shader): accept MIMG sampler tuples ending in VCC Ported from upstream PR boykopovar#4996 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(recompiler): name the attribute of an out-of-range interpolation reference Ported from upstream PR boykopovar#5450 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(shader): decode the high-half result of v_interp_p2_f16 Ported from upstream PR boykopovar#5471. The Vop3InterpDecode unit-test hunk is not ported (that test file is not on main); the execution test InterpolationF16 carries the coverage. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(shader): clear the bits of inactive lanes in VALU carry-out and v_div_scale masks Ported from upstream PR boykopovar#5162 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(shader): apply the f32 output modifier and denormal flush to v_cvt_f32_f64 Ported from upstream PR boykopovar#5159 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(shader): update per-invocation LDS without atomics Ported from upstream PR boykopovar#5160. Conflicts resolved against main: kept main's barrier condition for non-LDS shared atomics; the unit-test hunk for SRT read patching (a different upstream change, not on main) is dropped. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(shader): flush ds_add_f32 denormals and NaNs as the hardware does Ported from upstream PR boykopovar#5492 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(shader): run DS src2 lanes one at a time in lane order Ported from upstream PR boykopovar#5498 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(agc): compare NOT_EQUAL unordered, reject TRUNC_COORD point sampling and round the array layer half to even in color compare emulation Ported from upstream PR boykopovar#907 (follow-up to its predecessor). Hand-applied to main's reworked emitter: NOT_EQUAL uses OpFUnordNotEqual, the array layer is clamp(roundEven(z)), and ResourceMaterializer rejects TRUNC_COORD with point filtering. The upstream execution-test rewrite is not ported (it targets a diverged test file). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * chore(shader): bump ShaderDiskCache FormatVersion to 35 The ported DS fixes add MemoryInfo::flushDenormalResults (now encoded in the plan) and new IR opcodes, so cached payloads from format 34 must not load. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test(shader): run ported upstream execution tests on Metal through a device shim The upstream execution tests for the ported fixes compile unchanged against a Metal stand-in for the Vulkan test device (guest buffer descriptors are mirrored into shared MTLBuffers). Covers boykopovar#5162 (VALU carry-out masks), boykopovar#5159 (v_cvt_f32_f64 result modes), boykopovar#5492 (ds_add_f32) and boykopovar#5498 (DS src2 lane order). The LdsSrc2 and LdsSrc2Wide tests use device-memory LDS, which the Metal bridge rejects, and are not registered. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test(shader): run ported decoder and collector unit tests in the Metal build Registers the upstream tests for boykopovar#4996, boykopovar#5426 and boykopovar#5450 (already on main's tree but not built for Metal) and a high-half decode check for boykopovar#5471. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test(shader): check per-invocation LDS updates without atomics on the Metal recompiler Standalone copy of the boykopovar#5160 verifyFunctionLdsUpdates check. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test(shader): check the TG_SIZE wave index on Metal Buffer-store variant of the boykopovar#5214 execution test. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test(shader): run the s_barrier memory-order execution test on Metal Registers boykopovar#5117's upstream test through the device shim. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test(shader): draw a 32_AR export on Metal Guest-replay draw for boykopovar#5110: alpha comes from export channel 1. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test(shader): draw an NGG vertex shader that reads the group info Guest-replay draw for boykopovar#5081. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test(shader): reject TRUNC_COORD point sampling in the color compare emulation Recompile-level check for boykopovar#907. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test(shader): run the src2 lane-order test without the Metal debug layer Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * docs(shader): port upstream debt entries and add the FormatVersion 35 history line Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
|
This doesn't build on current main: CnRJay's two points are worth taking too: compute the data-only parts once before |
2b9f7aa to
17b5c4c
Compare
|
Rebased on main. |
|
RDNA2 rerun of the measurement, as CnRJay asked: on gfx1031 every row follows the rule in the description. Setup: AMD Radeon RX 6700 XT (gfx1031), Windows 11 26200, Adrenalin 26.8.1 (driver 32.0.21045.5002). Same method as the attached gfx1010 write-up: Operands: 1574 pairs of the classes the write-up lists (24 x 24 specials, 400 random pairs with exponents 0-3, 598 pairs that nearly cancel or are 30 binades apart; the random pairs are mine). Modes: the same 18 (denormal field 0-3 x round mode 0-3 at IEEE 0 and DX10 clamp 1, plus IEEE 1, plus DX10 clamp 0). Each stored value is compared with the six-step rule, with the add done as an exactly rounded f32 add between the flushes.
Rows that differ between denormal modes at round 0 on this set: 0/3 346, 0/1 242, 0/2 71, 1/3 143, 2/3 275. So gfx1010 and gfx1031 agree, and the debt sentence "RDNA2 (gfx10.3, the console's family) was not measured for AI-assisted: yes (Claude Code). |
17b5c4c to
848ce1a
Compare
ds_add_f32 and ds_add_rtn_f32 now follow the measured hardware: both operands are flushed to a signed zero when bit 0 of the f32 denormal field is clear and the result when bit 1 is clear, the sum is rounded to nearest even, a NaN data operand is stored quieted, else a NaN old value is stored quieted, and +inf + -inf stores 0xffc00000. Measured on gfx1010 and, in boykopovar#5492, on gfx1031. The new MemoryInfo field is stored in the shader disk cache (field count 31, minimum record 79 bytes, FormatVersion 34). The parts that depend only on the data operand are computed once, before the CAS loop. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
@thuanquang Thank you for the RDNA2 run. The debt entry now cites it instead of assuming gfx10.3 matches gfx10.1: "thuanquang measured the same rule on RDNA2 (gfx1031, RX 6700 XT) in #5492, with the same operand classes and 18 modes in wave32 and wave64; the PS5's gfx1013 is not measured." Rebased onto AI-assisted (Claude Code). |
ds_add_f32 and ds_add_rtn_f32 now follow the measured hardware: both operands are flushed to a signed zero when bit 0 of the f32 denormal field is clear and the result when bit 1 is clear, the sum is rounded to nearest even, a NaN data operand is stored quieted, else a NaN old value is stored quieted, and +inf + -inf stores 0xffc00000. Measured on gfx1010 and, in boykopovar#5492, on gfx1031. The new MemoryInfo field is stored in the shader disk cache (field count 31, minimum record 79 bytes, FormatVersion 34). The parts that depend only on the data operand are computed once, before the CAS loop. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
|
Checked on gfx1035 with the HW oracle: Added one commit on your branch after rebasing onto
Merging once the full Build is green. |
848ce1a to
3d48459
Compare
What
Before:
ds_add_f32andds_add_rtn_f32were a plain hostOpFAdd. Denormal operands and results were not flushed when the shader'sFLOAT_MODEflushes f32 denormals. That includes 0xc0, the mode most titles use and the mode assumed for a shader without a known float mode. A signalling NaN was stored unquieted, and which NaN was stored, and the NaN from+inf + -inf, were whatever the host produced. The technical debt listedds_add_f32as not measured.After: the add follows the rule measured on hardware:
+inf + -infstores 0xffc00000.MemoryInfogets aflushDenormalResultsflag next toflushDenormals, because input and result flushing come from different bits.Hardware semantics, measured on RDNA1 (gfx1010, AMD Radeon RX 5700 XT) with the
tools/hw-oracletemplate. The kernels were assembled with the Windows driver'samd_comgr_2.dlland launched with its HIP runtime.IEEE_MODEandDX10_CLAMPhave no effect.The measurement write-up is attached.
measurement-results-ds-add-f32-gfx1010.md
Remaining gap, recorded in technical debt: the sum is still the host
OpFAddbetween the flushes. Where the shader keeps f32 denormals, the result also depends on the host preserving them, and the module does not requestDenormPreservefor 32-bit floats. The Windows AMD driver used here flushes them.Tested
agc_driver_lds_add_f32covers 64 lanes in two waves, withds_add_rtn_f32andds_add_f32, in five float modes (denormal field 0-3 with round 0, and field 3 with round 3). The expected values are the RX 5700 XT's measured results, not a model. The test checks the 209 of the 320 rows that do not depend on the host preserving f32 denormals, and it also checks the value returned byds_add_rtn_f32. It passes. Without the change it fails (a signalling NaN is stored unquieted).ctest -R "lds_|shader_memory": everything passes exceptagc_driver_lds_mixed_width_atomics, which also fails on unchangedmain(dacd785) on this machine.python3 tools/check_conventions.py --base origin/mainpasses.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)SpirvMemoryInstructions.cppchanges merge cleanly with this one; whichever merges second rebases that line. feat(shader): translate the 64-bit and f32/f64 min/max DS src2 forms #5474 merges cleanly with this one)