Skip to content

fix(shader): correct v_log_f32 near 1.0 when the driver's log2 strays from a series - #14

Closed
SleglJan wants to merge 1 commit into
mainfrom
fix/shader-log2-near-one
Closed

SleglJan wants to merge 1 commit into
mainfrom
fix/shader-log2-near-one

Conversation

@SleglJan

@SleglJan SleglJan commented Oct 9, 2026

Copy link
Copy Markdown
Owner

What

v_log_f32 is lowered to the driver's Log2. Near 1.0 the result tends to zero while the driver's absolute error does not: on an RTX 3080 (615.71.09) v_log_f32 of 1 − 2^-24 is 0xb24b4000 (−4.7e-8) where the hardware returns 0xb3b8aa3c (−8.6e-8), 23,947,836 ULP off, and on 16,384 inputs spaced 2^-15 apart in [0.875, 1.125] it returns the hardware's bits for 1,685 and is up to 29,121 ULP off. RADV (Mesa 26.2.4) selects the native instruction and matches all of them.

Measured on RDNA2 (gfx1036, Ryzen 7950X3D iGPU) with tools/hw-oracle (Linux, CachyOS kernel 7.2.9, hsa-rocr 7.2.4, clang 23.1.1 as the assembler; IEEE 0, f32 denormals flushed, f16 kept, DX10_CLAMP 1, RNE) on 2,368 inputs 1 ± 2^-k (1 + m/64) for k = 1..24 and on the dense sample above: the hardware's log2 is within 1 ULP of the correctly rounded value on every input (equal to it on 56 to 95 % of each exponent's inputs), with no linear rule like the sine's. An eight-term series in d = x − 1 (exact by Sterbenz's lemma for x in [0.5, 2]), evaluated with fused multiply-adds and one multiply by 1/ln 2, is within 2 ULP of the hardware and of the correctly rounded value on all 10,172 inputs within 1/8 of 1.0; six terms reach 8 ULP there, and a band of 1/4 needs ten terms.

The lowering keeps the driver's Log2 and takes the series only for |x − 1| < 1/8 when the two differ by more than 4 ULP (the bit patterns compared as integers): the hardware and the series are both within 2 ULP of the correctly rounded value, so a driver result more than 4 ULP from the series is the one that is off. On RADV the native result is never more than 2 ULP from the series, the gate never fires and every bit stays; taking the series unconditionally would move 3,425 of the 8,190 dense inputs in the band off the hardware's bits by up to 2 ULP. The translation's picks for zero, negative, infinite and NaN inputs and the denormal flush are unchanged (the band lies inside [0.875, 1.125]); v_log_f16 shares the path and was already within 1 ULP on NVIDIA.

Through the recompiler, v_log_f32 results equal to the hardware bits and the largest ULP distance, before → after:

RADV (Mesa 26.2.4, the iGPU) RTX 3080 (615.71.09)
sweep, 32,832 inputs (every exponent with 64 mantissas of both signs, specials) 32,832 → 32,832 24,593, max 23,947,836 → 24,598, max 6, 99.5 % within 1 ULP
dense, 16,384 inputs in [0.875, 1.125] 16,384 → 16,384 1,685, max 29,121 → 5,258, max 9
1 ± 2^-k (1 + m/64), 2,368 inputs 2,368 → 2,368 max 23,947,836 → max 5 inside the band

The RTX 3080's remaining 9 ULP (0x3f900b00, 1.1254) and the sweep's 6 ULP are its own log2 just outside the band. The other seven ops of the transcendental sweep are unchanged.

Data (oracle captures, the hosts before and after input by input, scripts): https://github.com/SleglJan/AnyPS5/tree/audit/oracle-2026-10-08/oracle-audit/log.

Tested

Linux (CachyOS, kernel 7.2.9, GCC 16.2.1), Release, ANYPS5_ENABLE_SPIRV_TOOLS=ON, RTX 3080 and RADV, on this branch (based on main 0fdda7b).

  • New agc_driver_log_near_one: 64 lanes, f32 and f16 in each (1 ± 2^-k for k = 1..24, 1.0, 0.875, 1.125, 0.9, 1.1, 0.5, 2, 3, 10, 0.1, 1e-3, 1e3, the smallest normal, the largest finite, +0 and −1), pinned on the oracle in the titles' mode and run in FLOAT_MODE 0xc0; zero, infinite and NaN results must match the bits, the others are compared in ULPs with a per-lane tolerance of 2 inside the band and 12 outside it (measured maxima: RADV 0 everywhere, NVIDIA 1 inside the band, 9 outside). Written first: on main it passes on RADV and fails on the RTX 3080 at lane 6 (v_log_f32 of 1 + 2^-4 is 0x3db31fca, expected 0x3db31fb8, 18 ULP); with the change it passes, three repeats on each GPU.
  • agc_driver_vop1_float_unary, agc_driver_float16_misc, agc_driver_float16_result_modifiers, agc_driver_formatted_store and agc_driver_typed_heaps pass on both GPUs, three repeats; full ctest: pending.
  • tools/check_conventions.py --base upstream/main and tools/check_test_registration.py pass.
  • Fork CI: pending.

Overlap: boykopovar#2380 rewrites the neighbouring TechnicalDebt entry (the transcendental ops); this PR adds a bullet after it, so whichever merges second needs a one-line rebase. No open PR touches EmitFPLog2.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

https://claude.ai/code/session_012GLfKH7Sqe51wX254k1XsV

@SleglJan
SleglJan force-pushed the fix/shader-log2-near-one branch 2 times, most recently from b1ccda3 to 0b26546 Compare October 10, 2026 16:56
… from a series

The driver's Log2 has an absolute error that does not shrink with the
result, so near 1.0 the relative error explodes: on an RTX 3080, v_log_f32
of 1 - 2^-24 is 23,947,836 ULP off. Measured on RDNA2 (gfx1036), the
hardware's log2 is within 1 ULP of the correctly rounded value, and an
eight-term series in x - 1 is within 2 ULP of both for |x - 1| <= 1/8.

The lowering keeps the driver's Log2 and takes the series only inside that
band when the two differ by more than 4 ULP, so RADV's native result stays
bit-exact and NVIDIA's is within 6 ULP inside the band.
agc_driver_log_near_one pins 64 inputs per type from the oracle with
per-lane tolerances.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012GLfKH7Sqe51wX254k1XsV
@SleglJan
SleglJan force-pushed the fix/shader-log2-near-one branch from 0b26546 to 699ff87 Compare October 10, 2026 17:16
@SleglJan

Copy link
Copy Markdown
Owner Author

Opened upstream as boykopovar#2880; closing here.

@SleglJan SleglJan closed this Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant