Skip to content

feat(agc): enable VK_NV_device_diagnostics_config behind APS5_NV_DIAGNOSTICS - #9934

Merged
oneandonlydean merged 1 commit into
boykopovar:mainfrom
Celegans12:feat/agc-nv-diagnostics-config
Oct 24, 2026
Merged

oneandonlydean merged 1 commit into
boykopovar:mainfrom
Celegans12:feat/agc-nv-diagnostics-config

Conversation

@Celegans12

@Celegans12 Celegans12 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

What

On NVIDIA a GPU hang or fault usually surfaces only as VK_ERROR_DEVICE_LOST. In our runs VK_EXT_device_fault (#5453) added no fault address. Nsight Aftermath can write a crash dump with the device state, the faulting engine and the shader. But it does that only for a Vulkan device created with VK_NV_device_diagnostics_config, and our driver never asked for it.

  • APS5_NV_DIAGNOSTICS (presence) creates the device with VK_NV_device_diagnostics_config and turns on shader debug info, resource tracking, automatic checkpoints and shader error reporting. It also enables VK_NV_device_diagnostic_checkpoints where the device supports it; APS5_GPU_CHECKPOINTS still decides whether markers are recorded.
  • Unset, the device is created exactly as before. On a device without the extension the variable only prints a notice, so non-NVIDIA GPUs are unaffected.
  • ChooseDeviceDiagnostics (in DeviceFault.hpp, next to the fault report) makes the choice, so a unit test can check it. ENVIRONMENT.md has a row for the variable.

Overlap: #7920 adds VK_AMD_buffer_marker checkpoints in the same device setup, DeviceFault.hpp, its test and ENVIRONMENT.md. The two touch different lines and merge cleanly in either order; on the combination agc_device_fault passes. checkpoints keeps its meaning there (the NV extension in use), so neither needs the other first.

Why we wanted it: in Ghost of Yōtei a stop that showed up only as a device loss (result 0) looked like a GPU hang. With this on, Aftermath dumped it as Error_DMA_PageFault, a transfer write to address 0, which sent us down a different path.

Tested

  • Windows 11 Pro, MinGW-w64 GCC 15.2.0, -DBUILD_TESTING=ON, now based on main 0db47b2: agc_device_fault, env_reference, conventions_env_reference 3/3 with --repeat until-fail:3. The full ctest ran on the earlier base 1a745ae: 1898/1904, the 6 failures (agc_driver_sample_order*, log_macros*, guest_kqueue) also on main.
  • New Diagnostics case in agc_device_fault: nothing chosen without the variable or device support, otherwise the four flags plus checkpoints when supported. It doesn't build on main.
  • Validation (APS5_NV_DIAGNOSTICS=1, VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation, ANYPS5_NO_SHADER_CACHE=1, ANYPS5_REQUIRE_VULKAN=1, ctest -V), agc_device_fault and the packed color target tests, main 0db47b2 vs this head: 3/3 each, the loader inserted the layer in both runs, no was not found line, no VUID- or warning; the head logs the extension as enabled.
  • Linux (WSL, lavapipe), CI's job on the merge with main 0db47b2: the 5 focused tests pass; the full job before the refresh: 1832/1852, the same 20 failing on main there.
  • Not tested: AMD, Intel, console behaviour.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: NVIDIA GeForce RTX 3080 Laptop GPU, driver 610.62, Windows 11 Pro; lavapipe (Mesa 25.2.8, WSL Ubuntu 24.04)
  • AI-assisted: yes

@MarcZX10

Copy link
Copy Markdown
Contributor

I reviewed the feature-query and device-create chains, the extension enablement, and the helper tests. When the opt-in is absent, this leaves device creation unchanged; when requested, it enables the diagnostics feature/config only when supported and adds diagnostic checkpoints only when that extension is available (preserving the existing APS5_GPU_CHECKPOINTS path). The reported NVIDIA/lavapipe A/B runs and agc_device_fault tests are useful; current hosted eligibility, conventions, intake, labels and report checks pass, while product build jobs are skipped. I did not run GPU tests locally.

I ran git diff --check upstream/main...refs/review/9934 and python3 -I tools/check_conventions.py --base upstream/main --head refs/review/9934; both pass. However, the head is 74 commits behind current main (merge base 1a745ae38ea6a6c38949144ffb09ff962a9c1f1d). One intervening AGC change also modifies the VulkanDevice.cpp feature chain, so please refresh and rerun the focused validation-layer A/B tests on the final head before merge. I found no independent code defect in this pass.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1395 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 1896 docs/dev/ENVIRONMENT.md
PR 2434 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 4040 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 4072 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 4472 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 5021 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 5441 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 5925 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6186 docs/dev/ENVIRONMENT.md
PR 6188 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6202 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6617 docs/dev/ENVIRONMENT.md
PR 6924 docs/dev/ENVIRONMENT.md
PR 6960 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6977 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 7201 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 7275 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 7369 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 7449 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 7457 docs/dev/ENVIRONMENT.md
PR 7484 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 7604 docs/dev/ENVIRONMENT.md
PR 7618 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 7642 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 7814 docs/dev/ENVIRONMENT.md
PR 7922 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 8004 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 8014 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 8077 docs/dev/ENVIRONMENT.md
PR 8246 docs/dev/ENVIRONMENT.md
PR 8484 docs/dev/ENVIRONMENT.md
PR 8542 docs/dev/ENVIRONMENT.md
PR 9098 docs/dev/ENVIRONMENT.md
PR 9348 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9356 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 9387 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 9400 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 9431 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9432 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9508 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 9672 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9673 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9674 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9675 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9676 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 9842 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9891 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9892 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9893 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9895 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 9898 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 10194 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md
PR 10231 docs/dev/ENVIRONMENT.md
PR 10244 docs/dev/ENVIRONMENT.md
PR 10254 docs/dev/ENVIRONMENT.md
PR 10264 docs/dev/ENVIRONMENT.md
PR 10272 docs/dev/ENVIRONMENT.md
PR 10420 core/libs/prx/libSceAgcDriver/Execution/include/DeviceFault.hpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/ENVIRONMENT.md

@MarcZX10

Copy link
Copy Markdown
Contributor

I reviewed exact head 460d79e548179ea49cd6473a70277d7cf56f4396 against current main fcd455870cb9610d929f744719a4d20ffd544499; git diff --check and current-main conventions pass. The feature query/config pNext chain and conditional extension enablement fit the existing device-create chain, with the opt-in absent path leaving device creation unchanged. The helper tests cover request/support combinations, while your RTX 3080 validation-layer A/B is the evidence that exercises actual device creation; it reports 3/3 focused tests on both main and branch with the layer loaded and no VUIDs. The Windows full suite and WSL suite are contributor-reported, including the stated baseline failures. Hosted policy checks pass; product jobs are skipped by path policy. I did not rerun local GPU/C++ tests because the root filesystem is full. No code defect found in this pass.

@MarcZX10

Copy link
Copy Markdown
Contributor

Additional exact-head test evidence: I compiled and ran the submitted DeviceFault.cpp plus DeviceFault.hpp using GCC 12.2 and the repository Vulkan headers; agc_device_fault: passed. A first -Werror build stopped on missing-field-initializer warnings from existing Vulkan struct initialization in the header; the focused run passed when that warning was disabled. This exercises the diagnostics choice helper but not vkCreateDevice. Since the branch was last based on 1a745ae and current main adds mixed-attachment/sample-count feature-chain changes in the same VulkanDevice.cpp constructor, the current-main refresh and paired focused validation-layer run noted earlier remain necessary for the AGC review rule.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update after the later review at 2026-10-22 23:58:57 UTC: I refreshed the exact PR head and reran the current-main convention and whitespace checks. At current upstream main 0db47b263d1019043f7d28fad4e7ee0a1ea5ec6a, git diff --check upstream/main...refs/review/9934 and python3 -I tools/check_conventions.py --base upstream/main --head refs/review/9934 pass. The PR is still based at 1a745ae38ea6a6c38949144ffb09ff962a9c1f1d (74 commits behind), and 7920 remains open, sharing VulkanDevice.cpp and ENVIRONMENT.md; its current implementation adds AMD buffer-marker diagnostics. The substantial intervening VulkanDevice changes also remove/sample-gate several feature-chain members, so the final rebased chain needs another exact review. Your RTX 3080 and lavapipe A/B evidence is valuable, but after refreshing please rerun the focused validation-layer baseline/head tests on the final head and resolve the overlap/checklist (Depends on if there is an agreed ordering). Hosted policy checks pass; product jobs are skipped. I did not run the Vulkan tests locally in this session.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please refresh this AGC change onto current main and resolve its overlap with open PR #7920, which also edits VulkanDevice.cpp and ENVIRONMENT.md to add the AMD buffer-marker diagnostics path. The overlap detector could not compare many shared VulkanDevice versions, so the checklist claim that no open PR implements overlapping functions needs an explicit coordinated order or explanation. After refreshing, rerun the repository-required focused ctest -V A/B on the same GPU for main and the final head with VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation, ANYPS5_NO_SHADER_CACHE=1, and ANYPS5_REQUIRE_VULKAN=1; confirm the layer loaded and compare VUIDs. Existing RTX 3080/lavapipe test evidence is useful but predates current main. The current-main whitespace and conventions checks pass; this request is for the required refreshed shared-chain/overlap review and final-head GPU evidence.

…NOSTICS

A device loss on NVIDIA reports only VK_ERROR_DEVICE_LOST, and
VK_EXT_device_fault often gives no address. Nsight Aftermath can write a
crash dump that names the faulting shader and the kind of fault, but
only when the device was created with VK_NV_device_diagnostics_config.
APS5_NV_DIAGNOSTICS (presence) now creates the device with that
extension (shader debug info, resource tracking, automatic checkpoints,
shader error reporting) and with VK_NV_device_diagnostic_checkpoints
when the device supports them. Unset, or on a device without the
extension, the device is created as before; the latter prints a notice.
ChooseDeviceDiagnostics holds the choice so the test can check it.

AI-assisted: yes
@Celegans12
Celegans12 force-pushed the feat/agc-nv-diagnostics-config branch from 460d79e to c77548f Compare October 23, 2026 06:14
@Celegans12

Copy link
Copy Markdown
Contributor Author

@MarcZX10, I've refreshed this onto current main (0db47b2, head c77548f). The rebase didn't change the code lines. Your mixed-attachment/sample-count change only adds members ahead of byteFeatures, and the diagnostics structs are still put at the front of its chain. I did move the test's Diagnostics case to the top of DeviceFault.cpp and its call to the start of main(), so it no longer sits on the lines #7920 adds.

I reran the validation A/B on main and the final head, RTX 3080 Laptop, driver 610.62, with APS5_NV_DIAGNOSTICS=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation ANYPS5_NO_SHADER_CACHE=1 ANYPS5_REQUIRE_VULKAN=1 ctest -V over agc_device_fault and the two packed color target tests. Both runs are 3/3. VK_LOADER_DEBUG=layer shows the layer inserted in each, there's no "was not found" line, and neither log has a VUID- error or a warning. On the head both device tests also log the extension as enabled. CI's Linux job on the merge with main passes the same tests plus env_reference.

On #7920 (@hugodearaujo's AMD buffer markers): the two merge cleanly in either order, and a local merge of both builds and passes agc_device_fault. checkpoints still means the NV checkpoint extension there, so this PR's !checkpoints check keeps working. I don't think either needs to go first, and I've written that in the body rather than adding a Depends on.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact current-main head c77548f5f109a71894605d9d52f5cb624103b42d. The change is opt-in: absent APS5_NV_DIAGNOSTICS, device creation follows the existing chain; when set, it queries and enables VK_NV_device_diagnostics_config only when supported, adds the diagnostic checkpoint extension only when available, and preserves APS5_GPU_CHECKPOINTS behavior. The refreshed test matrix covers supported/unsupported combinations. The author reports repeated Windows unit/env tests and validation-layer A/B on NVIDIA, plus focused Linux checks on lavapipe; I did not run GPU tests locally. Local diff-check, conventions, and current-main sync pass (1 ahead/0 behind). The author reports #7920’s AMD checkpoint change combines and tests cleanly; this review found no remaining blocking overlap. Product build jobs are skipped by path policy. Approval is for this exact head.

@oneandonlydean oneandonlydean self-assigned this Oct 23, 2026
@oneandonlydean
oneandonlydean merged commit 2250381 into boykopovar:main Oct 24, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants