Skip to content

AMD Windows: since #4152, 47 AGC tests crash in amdvlk64.dll because library shader modules are destroyed before the background link #4435

Closed

Description

@LeandroLP23

On AMD's Windows driver, 47 of the 431 AGC tests fail on main since #4152. Run one at a time on a Radeon 860M (RDNA 3.5, driver 32.0.13058.8001, driverInfo 24.30.58.08), Windows 11 (26300), WinLibs GCC 15.2.0 r7, Release, with the shader cache emptied first:

build ctest -R '^agc_' -j 1
main e7c70a2df 384/431: 19 SegFaults, 27 non-zero exits, 1 timeout
main e7c70a2df with APS5_NO_PIPELINE_LTO=1 431/431
9bc362351, the merge before #4152, only the 47 that fail on main 47/47

A first-parent bisect over f60923bf5..origin/main, two runs per step, ends at the merge of #4152 (2979ae37a): its parent passes 2/2, it fails 2/2. The crashes are inside the driver, for example:

FATAL: unhandled exception 0xc0000005 on thread 100808 ''
  rip 0x00007ffb11f31120 amdvlk64.dll+0x2cc1120

This is the fault that #3173 fixed for #2755. AMD's driver still reads a library part's shader modules during the background link-time-optimized link that VK_PIPELINE_CREATE_RETAIN_LINK_TIME_OPTIMIZATION_INFO_BIT_EXT requests, after Pipeline has destroyed them. Destroying them is legal Vulkan, so this works around a driver bug. #3173 kept those modules in the device's library store until ClearPipelineLibraries. #4152 reworked LinkPipelineFromLibraries for parts prebuilt at registration and dropped that store (DeviceLibraries::modules and its modules parameter). Pipeline.cpp again destroys _modules right after the parts are created, also on the new prebuild path.

#4363 changes the same function for the depth format but doesn't restore the modules: with it merged the two tests I tried still fail 3/3.

A fix that puts the module store back on the current code, including the prebuild path, follows as a PR: with it all 431 pass.

The 47 tests

agc_driver_alpha_to_coverage agc_driver_conservative_rasterization agc_driver_depth_bias_per_face agc_driver_depth_fast_clear agc_driver_depth_fast_clear_submit agc_driver_draw_cache_reuse agc_driver_draw_offset_indices agc_driver_dual_source_blend agc_driver_exec_host_lanes agc_driver_image_plane_address_on_line_implicit_lod agc_driver_interpolation_f16 agc_driver_mesh agc_driver_msaa_enable_single_sample agc_driver_ngg_vertex_wave_info agc_driver_packed_unorm_color_target agc_driver_pipeline_optimized_readback_color32 agc_driver_pipeline_optimized_readback_color64 agc_driver_pipeline_optimized_readback_d16 agc_driver_pipeline_optimized_readback_d32 agc_driver_pixel_barycentric_emulation agc_driver_pixel_fixed_point_position agc_driver_pixel_helper_lanes agc_driver_pixel_interlock agc_driver_pixel_lds agc_driver_pixel_lds_lane_slots agc_driver_pixel_uint16_export agc_driver_pixel_valid_mask agc_driver_pixel_wave_reduction agc_driver_prepared_exports agc_driver_prepared_shaders agc_driver_prepared_shaders_debug agc_driver_prepared_shaders_indirect agc_driver_prepared_shaders_uncached agc_driver_provoking_vertex agc_driver_recorded_kept_bytes agc_driver_relative_stage_inputs agc_driver_scalar_swappc_fetch agc_driver_srgb8_color_target agc_driver_stencil_ref_export agc_driver_swap_alt_blend agc_driver_tessellation agc_driver_uint8x4_color_target agc_driver_vertex_fetch agc_driver_zero_position_vertex agc_driver_zero_stride_vertex_fetch agc_pipeline_cache agc_pipeline_library

cc @adriadam10 (#4152)

AI-assisted: yes (Claude Code).

Activity

  1. Amoenus commented on Oct 12, 2026

    @Amoenus
    Contributor

    Additional device/current-main reproduction: on Windows 11 Enterprise 26200, AMD Radeon(TM) Graphics (1002:13c0, AMD driver 32.0.21036.18), WinLibs GCC 15.2.0 r7 / Release, unchanged main d592d54e11ecce022abd62635a58bf2d048f3a2c also crashes in amdvlk64.dll in agc_driver_draw_cache_reuse.

    Commands (one test at a time, ANYPS5_GPU="AMD Radeon(TM) Graphics", ANYPS5_NO_SHADER_CACHE=1):

    • Normal main: ctest --test-dir <build> --output-on-failure --parallel 1 --repeat until-fail:20 -R '^agc_driver_draw_cache_reuse$' stopped on its first execution with 0xc0000005, inside amdvlk64.dll.
    • Same main executable/runtime, adding the existing APS5_NO_PIPELINE_LTO=1: the same test passes five consecutive executions.
    • A separate centroid-only candidate also hit the crash after one passing AMD execution. Its unchanged-main comparison reproduces it; that candidate does not change pipeline/library lifetime code. Both full NVIDIA suites passed (964 passed, seven skips, zero failures out of 971), so that is not evidence of an all-device pass.

    This confirms another AMD Windows device/current-main reproduction and the LTO-off control. I have not tested unchanged #4445, bisected this device, or independently proved the shader-module-lifetime cause. No new issue or duplicate implementation is proposed.

    Additional array/color1555 controls

    The same AMD device also failed agc_driver_array_layer_target and agc_driver_color1555_target during #4318's current-main rebase validation. A subsequent control used stock main fixture sources and matching headers, with the native test executable from our centroid checkout and frozen unchanged-main d592d54e runtime DLLs (not another whole-main rebuild):

    Control Original R_X-only array fixture Color1555 fixture
    main runtime, ordinary LTO, five executions 5/5 pass 5/5 crash in amdvlk64.dll
    same main runtime, existing LTO-off control 5/5 pass 5/5 pass
    rebased #4318, existing LTO-off control 5/5 pass, including both R_X/D_X array cases 5/5 pass

    The extended candidate array's normal-LTO failure is not claimed to be reproduced by the original R_X-only fixture. This adds the color1555 reproduction and diagnostic controls; it still does not validate #4445 or prove the module-lifetime cause independently. The full NVIDIA rebase suite passes (965 passed, seven skipped, zero failures /972). No diagnostic switch was added to #4318's implementation.

    Current-main retest after the lifetime fix merged

    On main 2a5eca11 plus our unchanged centroid-only patch, with ordinary LTO and ANYPS5_NO_SHADER_CACHE=1, ten relevant groups pass five times each on both this AMD device and RTX5080. In particular, draw-cache reuse, array-target and color1555 no longer crash in the selected AMD retest. Full NVIDIA suite:967 passed/0 failed/7 skipped of974. The base includes merged #4445 and other changes; this reports current-main behavior, not an isolated cause bisect or a test of unchanged #4445 head. The older failures/control results above remain scoped to their stated base.

    AI-assisted: yes (Codex).

    Follow-up on main 2f4cc89 (2026-10-13)

    The earlier crash/recovery findings above remain scoped to their stated commits. This follow-up is not a reproduction of the original shader-module-lifetime cause and does not ask to reopen this resolved issue.

    Our latest rebase validation exposed a harness error: APS5_NO_GPL is presence-based, so setting it to 0 also disables GPL. Corrected runs leave it unset for GPL on and use 1 for off. Recorder-based helper tests independently select NVIDIA; they are not AMD coverage.

    On the same AMD device/driver, #4318 at b6f1a4a69 passes eleven selected groups five times in the corrected default/GPL-on campaign. Two groups fail: agc_driver_graphics reports the previously recorded host-exclusion fixture signature; the extended agc_driver_array_layer_target reads 0xdeadbeef instead of 0x3e800000 after its draw. Stock main's original R_X-only array fixture is not evidence that the extended D_X fixture's failure is reproduced on main. Full NVIDIA suite: 989 passed, 0 failed, 7 skipped of 996. #4318 was subsequently merged; this AMD residual remains unresolved.

    An earlier frozen D_X runtime reproduced four off-path tiled-render mismatches. Stock production-source/header-matched main runtime controls passed those four cases five times on both devices. Recompiling the driver/compiler from unchanged tracked source clears matched color1555/packed-tail failures in crossed-executable controls. The old DLL and failed logs are retained; the exact artifact discrepancy is unconfirmed.

    For #4451 at 3040cd741, the corrected campaign passes all ten selected groups on AMD with GPL off, but actual GPL on fails dithered alpha coverage, existing undithered alpha coverage and wave32 subgroup. NVIDIA passes both modes and its full suite passes 990/0/7 of 997. Stock main runtime controls of undithered coverage and wave32 pass with GPL on, fail with GPL off, and pass with GPL on plus the existing no-pipeline-LTO diagnostic. This different control pattern does not establish a shared cause. The fresh driver/compiler and full-target rebuild retains these same three GPL-on failures. Each also fails its first execution with the existing no-pipeline-LTO diagnostic; the fresh full NVIDIA suite passes 990/0/7 of 997. No production workaround was added.

    Current evidence and updates are maintained in #4318 and #4451. Any confirmed distinct defect will be checked for duplication before a separate issue is opened. AI-assisted: Codex.

    Same-build and relocation follow-up: stock main production/headers and the original undithered/wave32 fixtures rebuilt in #4451's build directory reproduce the same AMD on/off pattern as that candidate. Moving byte-identical executables reverses the result even with the original libraries and the same current directory. This was not isolated by the earlier different-runtime control. The distinct reproduction, SHA-256 identities and limits are now consolidated in #4768; it does not claim recurrence of this issue's resolved lifetime crash. Candidate sources were restored exactly and rebuilt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions