perf(agc-driver): join the imports of adjacent ranges of any size that a buffer is bound exactly over

A bound buffer is joined across adjacent registered ranges only when every
one of them is larger than the 8 MiB window. PPSA25344's loading binds 16 MiB
and 4 MiB buffers whose backing was registered in two to six adjacent
pieces with the same access, some of them smaller than the window, so each
use of such a buffer was copied on the CPU: up to 348 GB in the first 300 s
of loading.

A bound buffer that starts at the first crossed range's start and ends at the
last one's end now joins those ranges whatever their size. A heap grown in
pieces matches that only for a binding over exactly its current pieces, so
growth beside a joined import does not re-import it. joinedCovers, used when a
joined import retires, accepts any adjacent run of ranges.

importJoinTests binds exactly over two 64 KiB pieces of a partly filled
window and requires one import of both; a binding across them that starts
inside the first piece stays apart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp

@@ -633,14 +633,11 @@ bool runCovers(const GuestAllocations::Lease& lease, std::uint64_t begin, std::u
     return !run.ranges.empty() && end <= run.base + run.bytes;
 }
 
-ImportRun joinedRun(const GuestAllocations::Lease& lease, std::uint64_t begin, std::uint64_t end) {
-    const auto window = importWindow.load(std::memory_order_relaxed);
+ImportRun crossedRun(const GuestAllocations::Lease& lease, std::uint64_t begin, std::uint64_t end) {
     auto last = leasedIndexContaining(lease, begin);
-    if (window == 0 || last == lease.size()) return {};
+    if (last == lease.size()) return {};
     const auto first = last;
-    for (;;) {
-        if (lease[last]->bytes <= window) return {};
-        if (lease[last]->address + lease[last]->bytes >= end) break;
+    while (lease[last]->address + lease[last]->bytes < end) {
         if (last + 1 == lease.size() || !continuesRun(*lease[last], *lease[last + 1])) return {};
         ++last;
     }
@@ -650,8 +647,18 @@ ImportRun joinedRun(const GuestAllocations::Lease& lease, std::uint64_t begin, s
     return run;
 }
 
+ImportRun joinedRun(const GuestAllocations::Lease& lease, std::uint64_t begin, std::uint64_t end) {
+    const auto window = importWindow.load(std::memory_order_relaxed);
+    if (window == 0) return {};
+    auto run = crossedRun(lease, begin, end);
+    if (run.ranges.empty()) return run;
+    if (std::all_of(run.ranges.begin(), run.ranges.end(), [window](const auto& range) { return range->bytes > window; })) return run;
+    if (begin == run.base && end == run.base + run.bytes) return run;
+    return {};
+}
+
 bool joinedCovers(const GuestAllocations::Lease& lease, std::uint64_t begin, std::uint64_t end) {
-    return !joinedRun(lease, begin, end).ranges.empty();
+    return !crossedRun(lease, begin, end).ranges.empty();
 }
 
 template<typename Entry>

core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp

@@ -262,12 +262,12 @@ void importJoinTests(const Context& context) {
         else GuestAllocations::GuestAllocationsRemove_nid_postfix(mutation, range);
         GuestAllocations::GuestAllocationsEnd_nid_postfix(mutation);
     };
-    const auto bind = [&](std::uintptr_t address) {
+    const auto bind = [&](std::uintptr_t address, std::size_t bytes = 32) {
         GuestBufferMemory plain(importing);
-        plain.AddReadable(address, 32);
+        plain.AddReadable(address, bytes);
         plain.Upload(false);
         std::uint32_t adjustment = 0;
-        const auto view = plain.Descriptor(address, 32, adjustment);
+        const auto view = plain.Descriptor(address, bytes, adjustment);
         const auto copied = plain.CopiedBytes();
         plain.WriteBack();
         return std::pair{view.buffer, copied};
@@ -298,6 +298,10 @@ void importJoinTests(const Context& context) {
     change(heap, piece, true);
     change(heap + piece, piece, true);
     Require(bind(reinterpret_cast<std::uintptr_t>(heap + piece) - 16).second != 0 && HostImportFor(importing, reinterpret_cast<std::uintptr_t>(heap + piece) - 16, 32) == nullptr, "two pieces no larger than the import window were joined");
+    const auto heapBase = reinterpret_cast<std::uintptr_t>(heap);
+    const auto [piecesBuffer, piecesCopied] = bind(heapBase, 2 * piece);
+    const auto* pieces = HostImportFor(importing, heapBase, 2 * piece);
+    Require(pieces != nullptr && pieces->base == heapBase && pieces->bytes == 2 * piece && piecesBuffer == pieces->buffer && piecesCopied == 0, "a buffer bound exactly over two pieces no larger than the import window was not imported as one");
     const auto* kept = HostImportFor(importing, first, 16);
     Require(kept != nullptr && kept->buffer == joinedBuffer && kept->bytes == 2 * large, "a registry change elsewhere re-imported the joined ranges");
     change(guest + 2 * large, large, true);

docs/dev/debt/libSceAgcDriver.md

@@ -75,7 +75,7 @@
 - [Descriptor pool chain](../../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp) (libSceAgcDriver) - each pool holds two storage buffer descriptors more than a set can take, as a workaround: RADV (Mesa 26.0 and later) lets a `FREE_DESCRIPTOR_SET` pool place a set up to 32 bytes past the pool's buffer ([`util_vma_heap_init`](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/vulkan/radv_descriptor_pool.c) is given `bo_size + RADV_POOL_HEAP_OFFSET` as a size), and with a page-multiple pool size those bytes are an unmapped page
 - [CMASK fast clears](../../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver) - each fast-cleared CMASK tile (code 0) is stored as `CB_COLOR_CLEAR_WORD` into its 8x8 texels at the first draw into the target or at its fast-clear eliminate or DCC decompress pass, not as the color block writes the tile, and the CMASK is then stored as expanded (0xF): a texture or copy read of the target after that draw and before the title's eliminate sees the clear value in tiles no draw wrote, where the console keeps their previous bytes. Tile addresses follow the pipe-aligned SW_64KB_Z_X CMASK equation for 16 pipes; other CMASK swizzle modes, a cleared tile of a DCC target and a fast clear of a mipmapped, 3D or array color target throw. A tile with a code other than 0 and 0xF keeps its texels like an expanded tile: PPSA18391 (engine: Clickteam Fusion) fast clears a target whose CMASK it never initialized, so some of its nibbles hold leftover values such as 1 and 10; what the console does with such codes on a single-sample target is unverified on hardware
 - [HTILE depth fast clears](../../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver) - HTILE is not modeled. Only RADV's `clear_htile_mask` compute kernel (17 GFX10 words, 64x1x1 groups) is recognized, and only when it leaves ZMASK at 0 (`user[4]` and `user[5]` low nibbles both 0); the dispatch still runs, and the host depth image is cleared to `DB_DEPTH_CLEAR` the next time the surface with that `DB_HTILE_DATA_BASE` address is bound. The address match is exact, so a clear of a different HTILE range does not hit. HTILE initialization, the stencil-only update, and fast clears made by the SDK's own paths, CP DMA fills or other kernels are not covered and leave the depth image as it was. The kernel words are the ones the PR author saw RADV submit; they are not compared with Mesa's source. A second kernel, the 9-word dword fill PPSA01802 (Kena: Bridge of Spirits) clears its depth HTILE with (64x1x1 groups, one `buffer_store_format_x` per thread through the stride-4 V# in `user[0..3]`, the value read by `s_buffer_load_dword` through the V# in `user[4..7]`), is taken as a fill of that range when the store format is 32_UINT, 32_SINT or 32_FLOAT and the value V# covers the dword (NUM_RECORDS times its stride, or NUM_RECORDS bytes when the stride is 0) ([`MatchMemoryValueFill`](../../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp), then the whole-HTILE and ZMask rules of `NoteDepthMetadataFill`). The value is read from guest memory when the dispatch is decoded, so a value that earlier GPU work in the same submission has not written back yet is not seen. When the [fill HLE](../../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/BufferFill.cpp) takes the dispatch instead (an output descriptor of one 32-bit value per element, a value descriptor that covers the loaded dword, a 16-byte aligned range), the value is read again when the fill is recorded, after the labels queued before it, and the HTILE fill is noted with that value.
-- [Host imports](../../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp) (libSceAgcDriver) on Linux skip ranges that are not mapped read-write, which are copied instead. The NVIDIA driver (595.91.07) refuses `VK_EXT_external_memory_host` imports of read-only or executable mappings (and shared mappings of regular files) with `VK_ERROR_OUT_OF_DEVICE_MEMORY`, after which every `vkAllocateMemory` on the device fails the same way. Read-only ranges are never imported, even on drivers that accept them. Shared mappings of regular files are not checked for. On Windows, an alias of adjacent registered ranges is refused if any page is not a shared view; a range made entirely of shared views is aliased run by run, including across sections or nonconsecutive section offsets. Adjacent registered ranges are joined into one import where they fill a whole 8 MiB aligned window; ranges of a partly filled window, and ranges larger than the window, are imported one at a time. A buffer bound across adjacent ranges that are each larger than the window joins them into one import when it is bound, and their own imports are retired; a buffer that crosses any other two imports is copied. Ranges larger than the window are taken to be registered whole and never grown in pieces: a title that grows a heap in pieces larger than 8 MiB re-imports the joined run each time a bound buffer crosses into a new piece. The 8 MiB window is a choice: unbounded runs were re-imported whole on every growth of a heap mapped in small pieces, and the NVIDIA driver (610.57.04, 615.78.08) then refused imports or lost the device. Not measured on RADV.
+- [Host imports](../../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp) (libSceAgcDriver) on Linux skip ranges that are not mapped read-write, which are copied instead. The NVIDIA driver (595.91.07) refuses `VK_EXT_external_memory_host` imports of read-only or executable mappings (and shared mappings of regular files) with `VK_ERROR_OUT_OF_DEVICE_MEMORY`, after which every `vkAllocateMemory` on the device fails the same way. Read-only ranges are never imported, even on drivers that accept them. Shared mappings of regular files are not checked for. On Windows, an alias of adjacent registered ranges is refused if any page is not a shared view; a range made entirely of shared views is aliased run by run, including across sections or nonconsecutive section offsets. Adjacent registered ranges are joined into one import where they fill a whole 8 MiB aligned window; ranges of a partly filled window, and ranges larger than the window, are imported one at a time. A buffer bound across adjacent ranges that are each larger than the window, or bound exactly over adjacent ranges of any size (from the first one's start to the last one's end), joins them into one import when it is bound, and their own imports are retired; a buffer that crosses any other two imports is copied. A heap grown in pieces re-imports its joined run only when a buffer is bound exactly over all of its pieces. Ranges larger than the window are taken to be registered whole and never grown in pieces: a title that grows a heap in pieces larger than 8 MiB re-imports the joined run each time a bound buffer crosses into a new piece. The 8 MiB window is a choice: unbounded runs were re-imported whole on every growth of a heap mapped in small pieces, and the NVIDIA driver (610.57.04, 615.78.08) then refused imports or lost the device. Not measured on RADV.
 - [Cube texture descriptors](../../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestTextureResource.cpp) with equal `BASE_ARRAY` and `DEPTH` expose six faces starting at `BASE_ARRAY`. Measured with `image_load`, `image_sample` and `image_get_resinfo` on RX 6600 (gfx1032), using 64x64 R32_UINT faces and base slices 0, 5 and 6. Other ranges containing fewer than six faces or incomplete cube arrays still throw: hardware load bounds and filtered-sample addressing differ. Out-of-range cube-array sampling remains unimplemented. Samplers with `DISABLE_CUBE_WRAP` are created with `VK_SAMPLER_CREATE_NON_SEAMLESS_CUBE_MAP_BIT_EXT`, the flag RADV maps to that bit (Mesa `src/amd/vulkan/radv_sampler.c`, `src/amd/common/ac_descriptors.c`); without `VK_EXT_non_seamless_cube_map` a cube texture sampled through such a sampler throws. PS5 equivalence has not been measured on console hardware.
 - [Multisampled CMASK](../../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) - a multisampled color target's CMASK is decoded under CB_COLOR_INFO FAST_CLEAR (bit 13, as for a single-sample target) or COMPRESSION (bit 14). Which bit a multisampled target sets is inferred: its fast clears were seen on GTA V's multisampled targets through their CMASK fills, not in a register dump. A COMPRESSION-only target's zero CMASK is not read as cleared (inferred: zero means cleared only under FAST_CLEAR), so only a fill after its first use clears it. A slice of a multisampled array is its own image at its own address with its own CB_COLOR0_CMASK (inferred from the register layout). Its CMASK is main's whole-metablock size.
 - [Multisampled CMASK state](../../../core/libs/prx/libSceAgcDriver/Graphics/src/MultisampleTarget.cpp) (libSceAgcDriver) - a multisampled target has no guest texels, so its CMASK state comes from fills rather than from a read at each draw as a single-sample target's does. A fill of the whole CMASK with 0 clears the host image at the target's next use, one with 0xffffffff marks it expanded, and any other pattern, or a fill that covers only part of the CMASK, throws at the next use (inferred: main's rule that a mixed CMASK is unmodeled, applied to the one host image). A FAST_CLEAR target starts from its CMASK keys at its first use, and a COMPRESSION-only target starts expanded. Copies by DMA into the CMASK are not seen, and a later use with another CMASK address throws. The clear does not rewrite the CMASK bytes as expanded, so a new target over the same CMASK (a different extent, so a different image) seeds from them again and clears once more.