perf(agc): join a buffer bound from inside a range larger than the import window into the small ranges after it

core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp

@@ -1717,6 +1717,7 @@ ImportRun joinedRun(const GuestAllocations::Lease& lease, std::uint64_t begin, s
     if (run.ranges.empty()) return run;
     if (std::all_of(run.ranges.begin(), run.ranges.end(), [window](const auto& range) { return range->bytes > window; })) return run;
     if (begin == run.base) return run;
+    if (run.ranges.front()->bytes > window && run.bytes - run.ranges.front()->bytes <= window) return run;
     return {};
 }
 

core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp

@@ -632,6 +632,54 @@ void importJoinTests(const Context& context) {
     ::operator delete(block, std::align_val_t{window});
 }
 
+void importJoinLargeFirstTests(const Context& context) {
+    constexpr std::size_t piece = 65536;
+    constexpr std::size_t window = 2 * piece;
+    constexpr std::size_t large = 3 * piece;
+    void* block = ::operator new(large + 4 * piece, std::align_val_t{window});
+    auto* guest = static_cast<std::uint8_t*>(block);
+    std::memset(guest, 0x22, large + 4 * piece);
+    const auto first = reinterpret_cast<std::uintptr_t>(block);
+    auto importing = context;
+    importing.hostImportAlignment = piece;
+    SetHostImportWindow(window);
+    const auto change = [&](std::uint8_t* range, std::size_t bytes, bool add) {
+        auto* mutation = GuestAllocations::GuestAllocationsBegin_nid_postfix();
+        if (add) GuestAllocations::GuestAllocationsAdd_nid_postfix(mutation, range, bytes, true, false, true);
+        else GuestAllocations::GuestAllocationsRemove_nid_postfix(mutation, range);
+        GuestAllocations::GuestAllocationsEnd_nid_postfix(mutation);
+    };
+    const auto bind = [&](std::uintptr_t address, std::size_t bytes) {
+        GuestBufferMemory plain(importing);
+        plain.AddReadable(address, bytes);
+        plain.Upload(false);
+        std::uint32_t adjustment = 0;
+        const auto view = plain.Descriptor(address, bytes, adjustment);
+        const auto copied = plain.CopiedBytes();
+        plain.WriteBack();
+        return std::pair{view.buffer, copied};
+    };
+    const auto end = first + large;
+    change(guest, large, true);
+    change(guest + large, piece, true);
+    change(guest + large + piece, piece, true);
+    change(guest + large + 2 * piece, piece, true);
+    const auto farCopied = bind(end - 16, 16 + 2 * piece + 16).second;
+    Require(farCopied != 0 && HostImportFor(importing, end - 16, 2 * piece + 32) == nullptr, "a buffer bound from inside a range larger than the import window across more than a window of small ranges was joined");
+    change(guest + large + piece, piece, false);
+    change(guest + large + 2 * piece, piece, false);
+    const auto [joinedBuffer, joinedCopied] = bind(end - 16, 32);
+    const auto* joined = HostImportFor(importing, end - 16, 32);
+    Require(joined != nullptr && joined->base == first && joined->bytes == large + piece && joinedBuffer == joined->buffer && joinedCopied == 0, "a buffer bound from inside a range larger than the import window into the small range after it was not imported over both");
+    const auto [again, againCopied] = bind(end - 16, 32);
+    Require(again == joinedBuffer && againCopied == 0, "a later bind across the joined ranges did not use the joined import");
+    change(guest + large, piece, false);
+    change(guest, large, false);
+    Require(HostImportFor(importing, first, 16) == nullptr && HostImportFor(importing, end, 16) == nullptr, "the host imports outlived their ranges");
+    SetHostImportWindow(0);
+    ::operator delete(block, std::align_val_t{window});
+}
+
 class ImportBudgetRanges {
 public:
     static constexpr std::size_t piece = 65536;
@@ -4780,6 +4828,7 @@ void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) {
     spaceWriteMergedPastImportTests(context, access);
     spaceReadOverWritableImportTableTests(context);
     importJoinTests(context);
+    importJoinLargeFirstTests(context);
     importCoverageHintBudgetTests(context, access);
     failedImportJoinProbeBudgetTests(context, access);
     failedJoinWrittenViewTests(context, access);

docs/dev/debt/libSceAgcDriver.md

@@ -251,7 +251,7 @@
 - [HTILE written by other kernels](../../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver) - when a depth target is bound and its HTILE was written since it was last read, the words over its whole pipe-aligned metadata (`HtileMetadataBytes`, the extent `HtileFillCovers` takes) are read, outside the surfaces mutex since the read waits for recorded work: when every one marks the depth (ZMask 0) or the stencil (SMem 0) cleared, the surface takes that fast clear and the words are stored back as expanded (depth only: ZMask 15 over the full Z range; with stencil: ZMask 15, Z range and SMem, SR1 and SR0 3), so only the title's next clear clears it again. PPSA29141 clears its depth every frame with a compute kernel the fill hooks do not recognize (`InvokeHTileClearComputeShader` in its `ClearMRT` pass); without this its depth kept every earlier frame and moving objects left black trails. PPSA25344 (Unreal Engine 5) reads its HTILE after the depth prepass in its Nanite depth export, which takes a tile with ZMask 0 as holding the clear value; without the expanded words the export put Nanite depth over the prepass and the Nanite material pass painted over the meshes in front. The word layouts are Mesa radv's for GFX10. Metadata with some tiles cleared is not modelled, and the compression state draws leave in HTILE is not tracked, so a title reading its HTILE after a draw sees the stored expanded words. A recognized HTILE fill or clear makes the next bind read the words again. A slice other than 0 of a depth array is read at its own HTILE (`DB_HTILE_DATA_BASE` plus the slice times `HtileMetadataBytes`) but the HTILE word layout of slices 1 and up is not modelled (PPSA05684's tile masks need X8 and Y8 to take Z0 and Z1 as well, slices 4 and up unmeasured), which neither the check that every word is cleared nor the store-back of each word in place needs, so such a slice takes the clear like slice 0. A depth-only HTILE whose tiles all have ZMask 0 but different Min/Max Z (per-tile clear values, PPSA05684's cascades) is taken as a whole-surface clear to `DB_DEPTH_CLEAR`, the value the DB's depth test reads for a cleared tile (`HTILE_Z_CLEAR_REG` in Mesa's `src/amd/common/ac_descriptors.h`: the clear word "only clears HiZ", "the DB register contain[s] the full clear values"); the HiZ test before it, which rejects against each tile's own Min/Max Z range (the HTILE definitions in the same file), is not modelled, so a draw that HiZ would reject over a tile cleared with Min/Max Z 0 is tested against `DB_DEPTH_CLEAR` instead. A multisampled depth target, and any target whose HTILE overlaps a resident native depth guard, skips this read and store-back and goes to the native multisample depth owner, which keeps its own refusal for a fast-clear transition.
 - [Native multisampled depth under GPU buffers](../../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp) (libSceAgcDriver): a buffer range a shader may write that overlaps the depth or HTILE range of a resident native multisampled depth is handled at upload like a guest read, before the GPU runs: a surface the host has not written is retired, and a written one goes through guest publication. A buffer bound read-only is not, so a shader reading the depth's memory through it sees the guest bytes from before the surface's draws.
 - [RADV fill-memory kernel](../../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/BufferFill.cpp) (libSceAgcDriver) - the [fill HLE](../../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/BufferFill.cpp) takes RADV's 16-byte `fill_memory` compute kernel (`MatchClampedGlobalFill`: 11 GFX10.3 words, 64x1x1 groups, six user SGPRs with the address in `user[2..3]`, the largest store offset in `user[4]` and the value in `user[5]`; each thread stores 16 bytes at `min(thread * 16, user[4])`) as a fill of `min(threads * 16, user[4] + 16)` bytes. RADV initializes and fast-clears DCC keys and other metadata with it, so the key and metadata tracking sees such a fill where it is in the command stream; run as a dispatch, its global stores name no range, and a key fill was only seen once it had landed, after later writes to the surface. Only a 16-byte aligned address and largest offset are taken. RADV's 4-byte variant (fills below 16 bytes), other Mesa versions' encodings and other compilers still run as dispatches. The kernel words are the ones vkQuake's PS5 RADV build (Mesa 25.2) submitted; they are not compared with the NIR builder's output for other targets.
-- [Host imports](../../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp) (libSceAgcDriver) on Linux skip ranges that are not mapped read-write, which are copied instead. The NVIDIA driver (595.91.07) refuses `VK_EXT_external_memory_host` imports of read-only or executable mappings (and shared mappings of regular files) with `VK_ERROR_OUT_OF_DEVICE_MEMORY`, after which every `vkAllocateMemory` on the device fails the same way. Read-only ranges are never imported, even on drivers that accept them. Shared mappings of regular files are not checked for. On Windows, an alias of adjacent registered ranges is refused if any page is not a shared view; a range made entirely of shared views is aliased run by run, including across sections or nonconsecutive section offsets. Adjacent registered ranges are joined into one import: the largest aligned power-of-two block, from twice the import alignment up to the 8 MiB window, that holds the requested address and that they fill exactly, so a block only grows by doubling as its window fills. A block that would exceed the budget, would replace an import a build holds, or fails (a refused Windows alias, `vkAllocateMemory`) steps down to the next smaller block; a failed block is not tried again until one of its ranges changes. Ranges larger than the window are imported one at a time. A buffer bound across adjacent ranges that are each larger than the window, or bound over adjacent ranges of any size from the first one's start (ending anywhere in the last range it crosses), joins the ranges it crosses into one import when it is bound, and their own imports are retired; a buffer that crosses any other two imports is copied. A heap grown in pieces re-imports its joined run only when a buffer bound from its first piece's start crosses into a piece the joined import does not cover, or when a sparse descriptor heap over it is bound: a heap a shader binds whole while only the pieces from its base are mapped is bound in place from the import of that run, its views ending at the import's end as the copy's do, once the previous bind of that address saw the same last mapped byte (a heap that grows between every two of its binds stays copied; one bound more often than it grows is joined again after each growth) and at most 64 times per joined run base; past that, with a message, it is not joined again: it is bound in place while an existing import covers its run and copied otherwise, until no registered range starts at that base or the device changes. Ranges larger than the window are taken to be registered whole and never grown in pieces: a title that grows a heap in pieces larger than 8 MiB re-imports the joined run each time a bound buffer crosses into a new piece. The 8 MiB window is a choice: unbounded runs were re-imported whole on every growth of a heap mapped in small pieces, and the NVIDIA driver (610.57.04, 615.78.08) then refused imports or lost the device. Not measured on RADV.
+- [Host imports](../../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp) (libSceAgcDriver) on Linux skip ranges that are not mapped read-write, which are copied instead. The NVIDIA driver (595.91.07) refuses `VK_EXT_external_memory_host` imports of read-only or executable mappings (and shared mappings of regular files) with `VK_ERROR_OUT_OF_DEVICE_MEMORY`, after which every `vkAllocateMemory` on the device fails the same way. Read-only ranges are never imported, even on drivers that accept them. Shared mappings of regular files are not checked for. On Windows, an alias of adjacent registered ranges is refused if any page is not a shared view; a range made entirely of shared views is aliased run by run, including across sections or nonconsecutive section offsets. Adjacent registered ranges are joined into one import: the largest aligned power-of-two block, from twice the import alignment up to the 8 MiB window, that holds the requested address and that they fill exactly, so a block only grows by doubling as its window fills. A block that would exceed the budget, would replace an import a build holds, or fails (a refused Windows alias, `vkAllocateMemory`) steps down to the next smaller block; a failed block is not tried again until one of its ranges changes. Ranges larger than the window are imported one at a time. A buffer bound across adjacent ranges that are each larger than the window, or bound over adjacent ranges of any size from the first one's start (ending anywhere in the last range it crosses), or bound from inside a range larger than the window over following ranges that together are no larger than the window, joins the ranges it crosses into one import when it is bound, and their own imports are retired; a buffer that crosses any other two imports is copied. A heap grown in pieces re-imports its joined run only when a buffer bound from its first piece's start crosses into a piece the joined import does not cover, or when a sparse descriptor heap over it is bound: a heap a shader binds whole while only the pieces from its base are mapped is bound in place from the import of that run, its views ending at the import's end as the copy's do, once the previous bind of that address saw the same last mapped byte (a heap that grows between every two of its binds stays copied; one bound more often than it grows is joined again after each growth) and at most 64 times per joined run base; past that, with a message, it is not joined again: it is bound in place while an existing import covers its run and copied otherwise, until no registered range starts at that base or the device changes. Ranges larger than the window are taken to be registered whole and never grown in pieces: a title that grows a heap in pieces larger than 8 MiB re-imports the joined run each time a bound buffer crosses into a new piece. The 8 MiB window is a choice: unbounded runs were re-imported whole on every growth of a heap mapped in small pieces, and the NVIDIA driver (610.57.04, 615.78.08) then refused imports or lost the device. Not measured on RADV.
 - [Cube texture descriptors](../../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestTextureResource.cpp) of two to five slices are only sampled: the [materializer](../../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) rejects loads, stores and dimension queries through them, which on RDNA2 stop at `DEPTH` while the view holds six faces. Out-of-range cube-array sampling is unimplemented; PS5 equivalence is unmeasured. A range of fewer than six slices samples six faces from `BASE_ARRAY` (measured with `tools/hw-oracle` on gfx1036 for two to five slices, and on RX 6600, gfx1032, for equal `BASE_ARRAY` and `DEPTH`); one of six or more keeps every slice up to `DEPTH`. Samplers with `DISABLE_CUBE_WRAP` are created with `VK_SAMPLER_CREATE_NON_SEAMLESS_CUBE_MAP_BIT_EXT`, the flag RADV maps to that bit (Mesa `src/amd/vulkan/radv_sampler.c`, `src/amd/common/ac_descriptors.c`); without `VK_EXT_non_seamless_cube_map` a cube sampled through such a sampler throws.
 - [Array layer views](../../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestTextureResource.cpp) (libSceAgcDriver) - `ArrayLayersServeView` lets a 2D texture at a 2D array image's base address, or a 2D array texture over its first layers, view that image (`Texture::CanCopyFrom`). Mesa-built homebrew (ps5-opengl) writes the view's last slice plus one to `CB_COLOR_ATTRIB3.MIP0_DEPTH`, so a slice-0 render target is a two-layer array image, and the title samples it as a plain 2D texture (SuperTuxKart PPSA69421 drew a black 3D scene and minimap without this). The per-layer layout of a mipmapped array is assumed equal to a standalone surface's; it is not measured on the console. A 2D texture at a later slice's address still gets an image of its own.
 - [Linear 2D texture pitch](../../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestTextureResource.cpp) (libSceAgcDriver) - the PS5's gfx1013 is not measured, and a `DEPTH` pitch on a block-compressed, mipmapped or base-array linear 2D T# throws, as does any pitch other than the default from a PS4 guest. A single-level uncompressed linear 2D T# uses `DEPTH` + 1 as its row pitch in elements, raised to the width and rounded up to 256 bytes, as `image_load` read it on RDNA2 (gfx1036, AMD Windows driver 32.0.21030.31, [hardware oracle](../HW_ORACLE.md)) in 135 kernels over 8_UINT, 16_UINT, 32_UINT and 32_32_UINT, widths 16 to 300 and pitches of 1 to 2048 elements.