Skip to content

fix(shader): translate f32 conversions in f32 denormal modes 1 and 2 - #10595

Open
TechLevelZero wants to merge 2 commits into
boykopovar:mainfrom
TechLevelZero:fix/f32-conversion-denormal-modes
Open

TechLevelZero wants to merge 2 commits into
boykopovar:mainfrom
TechLevelZero:fix/f32-conversion-denormal-modes

Conversation

@TechLevelZero

Copy link
Copy Markdown

What

A shader whose FLOAT_MODE f32 denormal field is 1 or 2 failed to translate on any f32 conversion ("f32 denormal mode N is not implemented"), and v_cvt_pk_u8_f32 also threw when a directed round mode was set at run time. Measured on hardware, these conversions only flush their f32 inputs, by bit 0 of the field, like the f32 compares:

  • Mode 1 keeps denormal inputs, as mode 3 does.
  • Mode 2 flushes them to a zero of their sign, as mode 0 does.

f32DenormalFlushFor now returns the compares' input flush for v_cvt_i32_f32, v_cvt_u32_f32, v_cvt_rpi_i32_f32, v_cvt_flr_i32_f32, v_frexp_exp_i32_f32, v_cvt_f64_f32, v_cvt_pknorm_i16_f32, v_cvt_pknorm_u16_f32, v_cvt_pk_u8_f32, v_cvt_pkrtz_f16_f32 and v_cvt_f16_f32. v_cvt_pk_u8_f32 is now a Directed32 rounding use: setRounding already writes the tracked round mode into floatMode, which its translation reads. v_div_scale_f32, v_div_fixup_f32 and v_mad_mixlo/mixhi_f16 still throw in modes 1 and 2.

Source: measured on RDNA2, gfx1036 (AMD Ryzen 7 9700X integrated graphics, Fedora 44, Linux 7.2.7, ROCm 7.1.1 HSA runtime), with the hardware oracle's kernel template.

  • v_cvt_pk_u8_f32: 2,148 sources (k/8 for k = -16 to 2,099, the values next to 0.5, 1.5, 2.5, 254.5, 255.5, 256 and -0.5, ±1e10, denormals and minimum normals of both signs, infinities, quiet and signaling NaNs) at IEEE 0 and 1, wave32 and wave64, f32 denormal modes 0 to 3 and f32 round modes 0 to 3, each set once in the kernel descriptor and once at run time by s_round_mode and s_denorm_mode: 128 configurations, all matching one model. The 38 vectors already pinned in VCvtPkU8F32RoundMode (measured on gfx1030) match gfx1036 in all four denormal modes.
  • The other conversions: 148 sources for the one-source ones and 484 operand pairs of 22 values for the two-source ones, at IEEE 0 and 1 and f32 round modes 0 to 3. Mode 1 equals mode 3 and mode 2 equals mode 0 on every row. v_cvt_f64_f32, v_cvt_f16_f32 and v_cvt_pkrtz_f16_f32 were also run in every f16/f64 denormal mode and f16 round mode, with the same result.
  • v_div_scale_f32 (7,968 of 10,648 triples) and v_div_fixup_f32 (836) differ between modes 1 and 3, so they are left throwing. v_mad_mixlo/mixhi_f16 don't exist on gfx1036 and were not measured.
  • Fixed settings: DX10_CLAMP=1, FP16_OVFL=0; f16 denormals kept and f16 round to nearest even unless stated.

Related: #5941, #6009 and #6010 change f32DenormalFlushFor for other instructions (v_mullit_f32, min/max, add/multiply). This PR doesn't touch their case labels.

Tested

Fedora 44 (host) / Ubuntu 26.04 containers, GCC 15, built on main ac28b89bd (the branch rebases onto fa515def8 without conflicts).

  • agc_driver_f32_convert_denormal_modes (new): 32 operand pairs × 12 result words in each f32 denormal mode at IEEE 0 and 1. Modes 0 and 2 must equal the measured flushed values. Mode 3 must equal the measured kept values, or the flushed value on a driver that flushes f32 denormals. Mode 1 must equal mode 3 on the same device.
  • agc_driver_vcvt_pk_u8_f32_round_mode (extended): all four f32 denormal modes, and each round mode set by s_round_mode in the shader as well as in FLOAT_MODE.
  • agc_driver_f32_denormal_flush still refuses modes 1 and 2 for its other instructions.
  • All three pass with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1 and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation on the three GPUs below. The validation layer loaded on each, and there are no VUID- messages.
  • On the A800, the 1,278 tests tools/select_tests.py selects for this change ran on main and on the branch (ctest -V -j6, same environment): no new VUID- messages, and the same failures on both sides apart from agc_driver_store_idle_results and agc_driver_store_idle_results_wake, which failed once on the branch under parallel load and then passed 5 of 5 runs alone on both main and the branch. The 27 failures on both sides are mostly tests that need guest write watching, which the container's seccomp profile blocks (userfaultfd failed (Operation not permitted)).
  • python3 tools/check_conventions.py --base origin/main: 0 errors. The two debt-length warnings are on entries 88 and 128, which were already over 1,000 characters on main.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: NVIDIA A800 80GB PCIe, NVIDIA 610.57.04, RHEL 10.2 (Ubuntu 26.04 container); AMD Radeon RX 9070 XT (RADV GFX1201), Mesa 26.0.8, Fedora 44 Linux 7.2.7 (Ubuntu 26.04 container); AMD Ryzen 7 9700X integrated graphics (RADV RAPHAEL_MENDOCINO, gfx1036), Mesa 26.0.8, Fedora 44 Linux 7.2.7 (Ubuntu 26.04 container)
  • AI-assisted: yes

🤖 Generated with Claude Code

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the translation changes against the surrounding FLOAT_MODE handling and the reported hardware measurements. The f32 conversion set now uses the same denormal input-flush bit as the compare opcodes, while the explicitly unmeasured divide-scale/fixup and f16 MAD-mix opcodes remain rejected. Moving VCvtPkU8F32 to Directed32 is consistent with setRounding tracking the runtime mode. The new GPU test covers signed denormals and normals across the four modes and the extended round-mode test checks descriptor and runtime settings. Hosted conventions, intake and labels pass; the repository workflow did not run GPU/product tests, so the multi-GPU hardware results in the PR description are author-reported and were not reproduced locally.

@TechLevelZero

Copy link
Copy Markdown
Author

One clarification: v_div_scale_f32 and v_div_fixup_f32 were measured. Mode 1 differs from mode 3 on 7,968 and 836 of the operand triples, so they stay rejected until their output flush is modelled. Only v_mad_mixlo/mixhi_f16 are unmeasured (gfx1036 lacks them).

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 docs/dev/debt/shader.md
PR 5371 docs/dev/debt/shader.md
PR 6235 docs/dev/debt/shader.md
PR 6280 docs/dev/debt/shader.md
PR 6412 docs/dev/debt/shader.md
PR 6488 docs/dev/debt/shader.md
PR 6924 docs/dev/debt/shader.md
PR 6959 docs/dev/debt/shader.md
PR 7410 docs/dev/debt/shader.md
PR 7484 docs/dev/debt/shader.md
PR 7684 docs/dev/debt/shader.md
PR 8004 docs/dev/debt/shader.md
PR 8077 docs/dev/debt/shader.md
PR 9335 docs/dev/debt/shader.md
PR 9387 docs/dev/debt/shader.md
PR 9889 core/shader/recompiler/Translation/src/VectorInstructions.cpp
docs/dev/debt/shader.md
PR 10233 docs/dev/debt/shader.md

@github-actions github-actions Bot added shader core/shader docs agc libSceAgc* labels Oct 24, 2026

@imellag imellag left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on a device without shaderFloat64 that flushes f32 denormals in hardware: Apple M4 Pro, macOS 27.0.1, MoltenVK 1.4.2 ([float-controls] f32 denormal flush: hardware), x86-64 Release under Rosetta 2. main does not build on macOS yet, so I applied this PR's code and test changes to #8658's macOS port, ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation, 3 runs:

without this PR with this PR
agc_driver_vcvt_pk_u8_f32_round_mode passes passes
agc_driver_f32_denormal_flush fails (fused lane 8 no float mode v_fma_f32 v14, v4, v5, v6 is 0x00000000, expected 0x00800000, the hardware flush) fails the same way
agc_driver_f32_convert_denormal_modes not present fails before checking anything

The new test stops at ShaderRecompiler::Recompile: 64-bit float instructions need the Float64 capability, which the device lacks: its shader includes v_cvt_f64_f32, and MoltenVK has no shaderFloat64. The f64 tests on main skip in that case (TargetHasCapability(device->Target(), spv::CapabilityFloat64) then VulkanTestSkipped, as in F64Conversions.cpp and DivisionOutputModifiers.cpp).

Skipping the whole test would lose the eleven f32-only conversions on exactly the kind of device CheckKept's "or flushed" branch is written for. As a local experiment only, I replaced v_cvt_f64_f32 with s_nop and left its two columns out of the checks: the test then passes 3 of 3 here, in all four modes at IEEE 0 and 1. So running v_cvt_f64_f32 in a second shader only when the target has CapabilityFloat64 would keep the rest covered on such devices.

From that run, in mode 3 the results whose kept and flushed values differ matched the kept value, except v_cvt_flr_i32_f32 of the five negative denormal inputs, which read 0 (the flushed value) instead of -1. That is the host's flush of the floor's input, which the test accepts as intended, and mode 3's translation is the same as on main.

No was not found but was requested line; the only validation message is VUID-VkDeviceCreateInfo-pProperties-04451 (MoltenVK's portability subset at device creation), with and without this PR.

AI-assisted: yes

TechLevelZero and others added 2 commits October 24, 2026 16:23
The f32 conversions that only flush their inputs follow bit 0 of the
FLOAT_MODE f32 denormal field, as the f32 compares do: mode 1 keeps
denormal inputs like mode 3 and mode 2 flushes them like mode 0.
v_cvt_pk_u8_f32 also follows a round mode set at run time.

Measured on RDNA2 (gfx1036) with the hardware oracle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…denormal test

The conversion test failed before checking anything on a device without
shaderFloat64, as its single shader included v_cvt_f64_f32. That
instruction now runs in a second shader only when the target has the
Float64 capability, so the other f32 conversions are still checked on
such devices.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@TechLevelZero
TechLevelZero force-pushed the fix/f32-conversion-denormal-modes branch from 8c339be to 55ac01b Compare October 24, 2026 15:29
@TechLevelZero

Copy link
Copy Markdown
Author

Thanks for testing on MoltenVK, and for the experiment. I did what you suggested: v_cvt_f64_f32 now runs in a second shader, only when the target has CapabilityFloat64. Its two result words are checked only in that pass, and the other eleven conversions are still checked on every device. The branch is rebased onto current main.

Checked with ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation:

  • NVIDIA A800 (610.57.04, Linux): passes, both shaders, no VUID- messages.
  • AMD RX 9070 XT and Ryzen 7 9700X iGPU (RADV, Mesa 26.0.8, Linux): passes, both shaders, no VUID- messages.
  • The same build on the A800 with the capability check forced to false, to take the no-Float64 path: prints v_cvt_f64_f32 skipped, the device has no shaderFloat64 and passes on the f32 conversions.

The agc_driver_f32_denormal_flush failure on MoltenVK happens with and without this PR, so I've left it out of this one.

@TechLevelZero
TechLevelZero requested a review from imellag October 24, 2026 15:31

@imellag imellag left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that covers it. Rechecked 55ac01bc1e on the same M4 Pro (MoltenVK 1.4.2, no shaderFloat64, f32 denormals flushed in hardware), the two commits applied to #8658's macOS port, same variables, 3 runs:

  • agc_driver_f32_convert_denormal_modes passes every time and prints v_cvt_f64_f32 skipped, the device has no shaderFloat64, so the eleven f32 conversions are now checked on this device too.
  • agc_driver_vcvt_pk_u8_f32_round_mode passes; agc_driver_f32_denormal_flush fails as before, the same with and without this PR, as you said.
  • No was not found but was requested line; the only validation message is still MoltenVK's VUID-VkDeviceCreateInfo-pProperties-04451 at device creation.

AI-assisted: yes

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* docs shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants