Color targets with the SW_4KB_S swizzle (CB_COLOR_ATTRIB3 tile mode 5) are laid out in 4 KiB
blocks with the standard element order the texture detiler uses, and are resident through the
same storage images as 64 KiB targets. A CB_COLOR_VIEW selecting one slice (SLICE_START equal to
SLICE_MAX) renders into that slice: the target starts one slice size per slice after the surface,
the slice size being the surface's mip chain as a texture of the same tile mode lays it out. Views
of several slices are still rejected.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
3D textures with more than one mip level were refused. Each slab of
blockDepth slices of a thick (SW_4KB_S / SW_64KB_S) volume now holds the
whole mip chain, smallest level first and level 0 last, as 2D chains are
laid out. A level's depth halves per level, so the texture loops skip the
slices a level does not have. Thin 3D (D/R 64KB_X) chains use the 2D mip
layout per depth slice. Thick mip tails still throw.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every ZPASS_DONE counter dump drained the device so the CPU could add up
the occlusion queries of the finished batches and store the running total.
PPSA21564 dumps around each occlusion-tested pass: about 1.5k drains per
10 s at its title screen, and in a level's early passes four drains a frame
that took 5.6 s of every 10 s of the graphics worker.
The running total now lives in a small buffer the GPU updates in batch
order. A batch still ends its occlusion query at submit; a query that saw
draws is remembered, and its pool is kept alive until it is folded. A dump
ends the open batch's query, copies the remembered results next to the
total, adds them with a one-thread compute shader that also stores the 16
DB counters into the dump's host import, and begins a fresh query for the
rest of the batch. The dump is ordered with the work around it like any
other GPU store, noted as a pending write and stamped. More than 64
remembered results are folded at a submit. Query pools are recycled.
A dump into memory the GPU cannot see, under a pending copied write-back
or over a label the CPU stores at a completion falls back to the drain,
which settles the remembered results on the CPU. Without buffer device
addresses the counting stays on the CPU.
A mesh program took its draw's index count from push constants, so an
indirect mesh-path (NGG) draw needed the CPU to read its record first,
waiting for the GPU to finish everything recorded before. In PPSA21564
that was about 170 waits per 10 s at the title, menu and level (about
1 s), and one per frame in the intro (2.2 s per 10 s).
The two free mesh push dwords now carry the device address of an argument
record (a VkDrawMeshTasksIndirectCommandEXT followed by the index count
and first index). When the address is set, the prologue takes the index
count and the first index from the record (the new MeshArgument IR value,
a load through a physical storage buffer pointer) and offsets its index
reads by the first index; a direct draw leaves the address 0 and draws as
before. Mesh modules declare the physical storage buffer capability for
it, and the driver's SPIR-V validation accepts the extension in mesh
modules.
An indexed mesh draw with one record that patches no user SGPR now stays
on the GPU. A one-thread compute pass (Graphics/shaders/MeshArguments.comp)
turns the record, read in place from its host import, into the draw's
argument record: the groups and instances of vkCmdDrawMeshTasksIndirectEXT
and the index count and first index the mesh program reads, with the CPU
path's rules (count clamped to the index buffer, nothing for an empty
record or past the device's group limits). The index buffer descriptor
covers the whole buffer and the program offsets by the first index. When
the records cannot be read in place, the CPU reads them and fills the
same argument record (ResolveMeshArguments).
PPSA21564's galaxy map renders a 1920x1080 surface as a color target whose
CB names DCC keys at one address and then writes it from a compute shader
whose T# names keys at another. Both key ranges read uncompressed every
frame, but the storage cache remade the surface's image whenever a
descriptor named metadata other than the image's, so the image was
flushed, destroyed, created and uploaded twice a frame (1231 constructions
in a 56 s scene), with the views and draw recipes built on it thrown away
each time.
A descriptor naming other keys now keeps the image when the move is
unobservable: the image's own keys and the named keys both read
uncompressed, and its content was neither uploaded nor filled under a
clear code. That is exactly the content an image made under the named
keys would hold once this one's pending results were stored. Any other
case (a fast clear of either key range, a register clear, an image
holding a clear) still remakes the image as before, so clears of the new
keys reach it and pending results are stored first.
Every proof that stands in for a lookup checks the same rule: the sampled
view hit, the fast revalidation of views and storage elements (storage
elements now record their descriptor's keys), the own-object refresh, and
the draw recipe's resident targets. Named-key scans keep per-image proofs,
so a repeated check costs no scan while the key bytes are unstamped.
agc_dcc_key_sharing drives KeysServeSurface, the rule ServesKeysAt uses,
through a model of the storage cache.
A CB metadata pass (and a resident target's register-clear check) read the
DCC keys with CurrentDccKeys, which waited for the GPU whenever recorded
work wrote the key range. In the intro video of PPSA21564 about 800 passes
per 10 s waited about 1.8 ms each, and every one of those waits was behind
a key store whose result the driver already knew: its own uncompressed key
store, or a title fill of uniform keys recorded on the GPU.
Pending-write notes now carry a serial in recording order, and the key
memo remembers the note of the store it describes. When the newest
pending note over the keys is that store's own note, nothing recorded
after it writes the keys, so they read as the store's keys without a
wait. Any later writer, a range the store does not cover, or a unit
shadow over the keys waits as before.
DB_DEPTH_CONTROL's DEPTH_BOUNDS_ENABLE now runs the Vulkan depth bounds
test with DB_DEPTH_BOUNDS_MIN/MAX, and PA_SU_SC_MODE_CNTL's
POLY_OFFSET_FRONT/BACK_ENABLE enable depth bias from
PA_SU_POLY_OFFSET_FRONT/BACK_SCALE/OFFSET and _CLAMP. Both are dynamic
state, so draws that differ only in their values share a pipeline.
The hardware slope scale is in 1/16 units. Depth bias with units other
than the depth format's (PA_SU_POLY_OFFSET_DB_FMT_CNTL), or that differs
between front and back faces that are both drawn, is refused. The
registers join the draw key. Depth-conditional color writes are still
refused.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The [present] line, printed under APS5_PROFILE_DRAW, takes its GPU busy and
idle gaps from the whole-batch timestamps in each batch's completion record.
Those stamps were only written under APS5_PROFILE_GPU, which also times every
command. Under APS5_PROFILE_DRAW alone every batch finished with zero stamps,
so the line read "GPU busy 0.0 ms, idle gaps 0.0 ms" with no missing records.
Under APS5_PROFILE_DRAW each batch now carries just its two batch stamps in a
two-query pool, reused across batches like the command buffers.
APS5_PROFILE_GPU is unchanged, and nothing is written with profiling off.
agc_driver_recorder_tests (GPU, by hand) checks that three consecutive
batches' completion records carry ordered stamps exactly when batch stamps
are on, and none with profiling off.
The draw resource key holds each read-only buffer's V# base and size and every
ShaderData/FlattenedSrt word, so draws whose buffers move per frame (constant and
vertex rings) never hit a template. The key now leaves those out; a hit checks every
moved read-only buffer (accessible, no pending or shadowed writes over it) and
rebinds it from its new address with the template's push adjustment, and data words
that changed get a fresh data buffer with the template's patches. A template whose
buffers cannot be rebound counts as a miss and is built fresh.
APS5_NO_MOVED_BUFFER_TEMPLATES=1 keys the words as before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A RELEASE_MEM with an interrupt select drained the device: the queue
worker submitted everything recorded and waited for the GPU before it
stored the label and delivered the interrupt. PPSA21564 issues about 300
of them per frame, so the worker and the GPU took turns.
The label is now recorded like any other (on the GPU behind the work, or
as a completion action behind pending write-backs), and the delivery
becomes a completion action of the batch holding the work before the
packet (Recorder::AfterRecordedWork, after the batch's earlier
completions). It is counted with the completion labels, so idle workers
and pollers reap it, and it starts the label flush deadline. With draws
or dispatches recorded since the last submit the batch is submitted at
once. A batch holding only label stores stays open and goes out with the
next work, at the label flush deadline, at a wait, at a flip or at the
end of the submission, like any other pending label. A label the GPU
cannot store, or one that does not decode, still drains.
The interrupt so never arrives before the work ahead of it completed and
its label landed; it can only arrive later, by at most the flush
deadline plus the work recorded into the same batch.
Every vertex attribute copied its own range of guest memory, so a draw
over an interleaved buffer copied it once per attribute (seven or eleven
times for PPSA21564's level meshes). Attributes with the same stride and
input rate whose first elements lie within one record now share one copy
of their union, each bound at its offset into it.
PPSA21564's final composite renders into 4K 10-bit scanout buffers
(COLOR_2_10_10_10 with the alternate swap, A2R10G10B10). No guest texture
format matched that color format, so the target stayed non-resident:
every frame's composite draw ran synchronously (about 11 ms, 143 per 10 s
at the title) and the flip read the frame back through guest memory.
The blue-low 10-bit color target now keeps its bits in the storage image
of its surface's red-low 2-10-10-10 format, as B8G8R8A8 targets map to
R8G8B8A8.
Presenting it: DisplayTexelFormat names a display buffer's texel order as
the guest-memory path reads it (R8G8B8A8 base: red low, else blue low;
the 10-bit variant: A2B10G10R10 or A2R10G10B10). ResidentPresentPath
blits a resident image only when it holds the display's 8-bit texel
format (an exact copy). A 10-bit image (whose blit would round where the
guest path drops the low two bits), the other 8-bit order or an sRGB type
is converted instead: GpuColorTransfer::DetileImage copies the image's
texels raw and runs the guest path's own conversion shader on them, so
both paths present the same pixels.
The image's write-back is unchanged: its results stay pending like any
resident target's and are stored when a reader of the memory flushes
them.
Tests: agc_driver_recorder_tests checks the format decision for both
8-bit and both 10-bit orders, and converts an image in each order,
comparing it with the CPU decode and the GPU detile of the same words in
guest memory.
ColorTargetLayout built its SW_64KB_R_X offset tables in every
constructor, and every draw constructs a layout per color target while
decoding and again while setting the target up. The tables depend only on
the element size; they are now built once per size under std::call_once
and the layout points into them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Take scanned every retained slot of its tier for one with the same size,
usage and memory properties, so the slot bound stayed at 512. With 512
slots GTA V's in-game builds overflowed the small tier: each draw evicted
and recreated buffers, and the submission thread spent most of its time
in the driver's allocation and free calls. The slots are now kept in
lists per key, oldest first: Take pops the newest of its key, eviction
takes the oldest front. The bound is 4096 slots per tier, held to a sixth
of maxMemoryAllocationCount.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every draw copied its index and vertex ranges from guest memory into
fresh buffers and scanned the indices, although the draws of a frame read
the same meshes again: about 10 GiB of vertex copies per 10 s at
PPSA21564's title.
CopyDrawInput now keeps the copy in the recorder's draw snapshot cache
and binds it again while the write tracker proves the range unchanged
since the copy was taken, with the same rules as the storage-buffer
snapshots: UnchangedSince on the collect made right before the copy and
an unchanged allocation registry. As GuestMemory::Read does, the flush
hook runs first, so pending GPU results over the range are stored and
stamped before the check. Unwatched memory is copied every time. An index
copy keeps its highest index; the device limit is still checked for every
draw.
The cache key includes the snapshot's use (storage, vertex, 16-bit or
32-bit index), so each use keeps its own buffer with its own usage. A
vertex copy runs from the buffer's start to its highest fetched element,
so draws over one shared vertex pool ask for many sizes: a vertex
snapshot also serves a shorter read at the same address while that
read's bytes are unchanged, and a new one replaces the shorter ones.
Vertex and index snapshots have their own least recently used pool
(1 GiB, 16384 entries), so storage copies, most of which change every
frame, do not push them out before their next use.
Tests: agc_driver_recorder_tests copies index and vertex ranges of
write-watched memory and requires a reuse (with the stored highest index)
while the bytes are unchanged, a new copy with the current bytes after a
CPU store, a MarkWritten store and a registry mutation, separate copies
per use, no reuse without a recorder, prefix reuse for vertex reads only,
and storage snapshots not evicting vertex snapshots.
A recorded draw binds a CPU copy of each read-only guest buffer it reads
in place, made for every draw. PPSA21564's title-stage draws bind the
same large light-list buffers written by compute passes (up to 16 MiB
each) hundreds of times per frame, so about 4 ms of every draw went into
copying about 30 MiB into freshly allocated host buffers.
The recorder now keeps the snapshots, least recently used first, under a
256 MiB budget and 1024 entries. A later draw of the same range binds the
kept, immutable copy while GuestMemory::UnchangedSince holds for the
write generation collected right before the copy and the guest allocation
registry generation is unchanged. A CPU store (stamped by a collect), a
driver or GPU store (MarkWritten) or a registry mutation makes the next
draw copy again. Unwatched memory has no generation and is copied for
every draw, as before.
A recency list (each entry holding its own position) gives the eviction
victim and moves a used entry to the back in O(1).
Tests: agc_driver_recorder_tests binds a write-watched, host-imported
range twice and requires the same snapshot, then a new copy with the new
bytes after a CPU store, a MarkWritten store and a registry mutation; and
checks the least recently used eviction order at the entry cap, the byte
budget and a CPU store dropping a kept snapshot.
An MRT slot below the highest written one that the draw does not write is a VK_ATTACHMENT_UNUSED
reference with a disabled blend; color targets keep their slot for export mappings, pipeline and
pass keys. DB_DEPTH_VIEW SLICE_START moves the depth and stencil planes by whole slices, so each
slice of a depth array is its own depth surface.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Draws with CB_COLOR_CONTROL MODE ELIMINATE_FAST_CLEAR (2) or DCC_DECOMPRESS
(6) were rejected as "only normal color rendering with copy ROP". On
hardware these full-target blits write the fast-clear value into the
fast-cleared pixels (or store the pixels uncompressed); the shader's output
never lands. In this driver's DCC model (surfaces stored uncompressed,
uniform fast-clear keys make every read return the clear value) that is:
- keys that are a clear code: the target's texels become that value and
its keys read uncompressed. Reads see the same values before and after.
- uncompressed keys, or a target without DCC: nothing changes.
- mixed keys (per-block metadata), a mipmapped DCC target (its keys are not
decoded) or a register clear of a texel over 64 bits: throw.
The texels go into the target's resident image (its refresh already
cleared it to a code's value; the register code is cleared on the GPU with
FillClear) and reach memory through its write-back, or into guest memory
on the CPU when there is no resident image. A resident image made under
other DCC keys is remade under the target's keys by the storage cache
before the pass reads it. DecodeColorMetadataPass
requires the draw to be only that pass: copy ROP, no depth or stencil
work, no multisampling, a viewport and scissor covering each target.
Indirect passes throw. The CB_COLOR decode moves into DecodeColorBuffer,
which also reads CB_COLOR_CLEAR_WORD0/1.
The register clear code (DCC key 0x20) had no value before: its keys were
read as the stored texels. A target under register keys now reads as its
CB_COLOR_CLEAR_WORD texel: a resident target is cleared to it before a draw
renders into it (then its keys read uncompressed), and a non-resident
target's copy starts from it.
Every decision that materializes a clear reads the keys through
CurrentDccKeys, which first waits for the batches up to the newest recorded
writer of the key range. The title's fast-clear fill of the metadata can
still be pending on the GPU when the next draw is decoded; reading the
stale bytes there made a later draw clear the target over the results of
the earlier ones.
The remaining CB modes (resolve, decompress, FMASK decompress) keep the
rejection, which now names the mode.
A shader sampling with MIMG DIM 2D over a T# of type 2D array was
refused: "guest texture dimension disagrees with the shader's declared
image shape (shape 1, dimension 2)". The access carries no slice
coordinate, so the hardware samples slice 0 of the resource view, the
BASE_ARRAY layer, as it does for the storage access of the previous
commit.
A 2D array Texture now also makes a 2D view of that layer
(Texture::FirstLayerView), whether it holds its own snapshot or views the
surface's storage image, and the sampled binding uses it for such an
access. Every other shape/dimension mismatch still throws.
A compute shader addressing an image with MIMG DIM 2D over a T# of type 2D
array was refused ("guest storage texture dimension disagrees with the
shader's declared image shape"). Such an access carries no slice
coordinate, so the hardware reads and writes slice 0 of the resource view,
which is the BASE_ARRAY layer of the surface.
The storage element is now bound through a 2D view of that layer of the
surface's shared storage image (StorageTexture::FirstLayerView, created on
demand per mip and released with the image). Every other shape/dimension
mismatch still throws, and both mismatch messages now name the shape and
the dimension.
The recorder test device sets formatProperties, which storage images need
to pick their Vulkan format.
A guest buffer view off minStorageBufferOffsetAlignment is bound from the
aligned offset below it, and the difference goes into the shader's memory
byte offsets (3dec6ced), but only when those live in the push constants:
a shader whose shader data is a buffer failed with "a guest buffer off the
storage buffer offset alignment in a shader whose shader data is a buffer
is not implemented".
The adjustment now goes into the shader data buffer's byte at
memoryOffsetDword * 4 + element, and a template's data refresh writes the
dispatch's words with the same patches applied.
A buffer fill that starts at a surface's DCC keys and covers at least its key range (titles fill
the padding too) is a key fill. A uniform key fill leaves the image's texels and pending results
in place and records the fill's code: a sampled read through the surface's keys binds the clear
value, a color target through the keys starts from it at its next refresh, and reads that bypass
the keys still see the texels. Key scans return the fill's code until its write lands, and a
write-back of results drawn before the fill keeps the keys holding that code. The immediate clear
of the covered images is opt-in (APS5_KEYS_FILL_CLEAR=1): it fixes titles whose sampled reads
reach the pending image before a refresh, at the cost of reads that bypass the keys (Astro Bot's
title logo, reported by @oneandonlydean).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
T# word1 MIN_LOD (unsigned 4.8, in levels of the whole surface) was
rejected. Sampled views now chain VkImageViewMinLodCreateInfoEXT
(VK_EXT_image_view_min_lod, enabled when the device offers the minLod
feature). Its minLod counts levels of the whole image exactly as MIN_LOD
counts levels of the whole surface, because the image holds every mip and
the view starts at BASE_LEVEL.
- A clamp at or below BASE_LEVEL cannot change which level a sample reads
and is left off; one past the view's last level is bounded by it (the
view selects that level anyway, and Vulkan requires the bound).
- A clamp that binds on a device without the feature throws.
- The texture cache already keys by the eight descriptor words, so views
with different clamps are not shared.
- Storage views address one level: a clamp at or below that level is
accepted, one above it still throws.
GuestMemory::CollectWrites was built on the Windows arena's MEM_WRITE_WATCH
and answered 0 on Linux, so every cached copy of guest memory (textures,
resident render targets, buffers, DCC key proofs) read as changed at every
check: resident targets were stored and re-uploaded at every refresh and
unit shadows stayed off. Astro Bot's intro ran at ~3 fps.
libc gains GuestWriteWatch, the Linux counterpart of GetWriteWatch. Every
guest mapping libkernel makes (MapAligned: fixed, hinted and aligned) is
registered with one userfaultfd in asynchronous write-protect mode
(UFFD_FEATURE_WP_ASYNC: the kernel lifts a page's protection on its first
write without a handler thread, for user and kernel-mode writes alike), and
PAGEMAP_SCAN with PM_SCAN_WP_MATCHING reports the pages written since the
last scan of a range and protects them again in one call. Unmap forgets
the range. Direct memory is mapped from shared backings over its
reservation and a fixed remap replaces the mapping, so both register the
new mapping afresh. The watch is per mapping: a write through one mapping
of shared physical pages is not seen at another, so physical pages mapped
at more than one address leave the watch in every mapping of them.
A registered mapping reads as written until a collect covers it. The watch
keeps that itself instead of relying on how the kernel classifies pages
never faulted in: Linux 6.17 reports a range without page tables as not
written and leaves it unprotected, 7.2 reports it as written. The first
collect of a fresh range reports it written and write-protects it with
UFFDIO_WRITEPROTECT, which puts markers on its unpopulated pages
(UFFD_FEATURE_WP_UNPOPULATED), so from then on only real writes report.
Fallback: the userfaultfd is opened with UFFD_USER_MODE_ONLY where
vm.unprivileged_userfaultfd restricts it, and a probe at startup checks the
mechanism on scratch pages (a populated and an unpopulated page written
after protection report exactly, untouched ones do not). Without it (kernel
older than 6.7, no userfaultfd permission, headers without PAGEMAP_SCAN, or
APS5_NO_WRITE_WATCH=1) the watch reports unavailable and the tracker stays
unwatched exactly as before: CollectWrites answers 0 and callers compare
bytes. A range with a page outside the registered mappings also answers 0,
and a protection or scan that fails reports its remaining pages as written
and answers 0.
The driver's tracker keeps its contract and generations. On Linux its
block stamps live in 4 GiB leaves over the user address space, made on
first stamp, instead of the arena's flat arrays; the collect memo now works
on both platforms. GuestMemory::WriteWatched and Watched replace the arena
queries in UnitShadowEnabled and ShadowDestinationFor, so unit shadows run
on Linux over watched memory and stay off everywhere else. The Windows
arena path is unchanged apart from going through the same stamp helpers.
Host imports: a driver may lift the write protection of imported pages
itself. amdgpu validates the userptr BO behind a VK_EXT_external_memory_host
import with a writable HMM fault on every submission after the pages were
protected again, so PAGEMAP_SCAN reports them as written with no CPU store,
and a resident display image read as written at every present (found by
Pacuka on an RX 6650 XT with RADV: the presented image went black). NVIDIA
pins the pages once at the import and stays exact from then on. A probe at
device setup decides per device: it maps and watches scratch pages, collects
them clean, imports them, collects, submits a GPU copy that reads the import,
waits, and collects again. Pages written after the submission mean imports
resolve write protection: every import from then on takes its range off the
watch (GuestMemory::Unwatch) before it is made, so Watched is false for it,
CollectWrites answers 0 and callers compare bytes; GPU writes to watched
memory keep their MarkWritten stamps. The result is logged once:
"[write-watch] host imports resolve write protection: yes/no". A probe that
cannot run takes the unwatch side. APS5_WRITE_WATCH_IMPORTS=watch|unwatch
overrides the probe (default probe; anything else throws), and
APS5_NO_WRITE_WATCH=1 still turns the watch off. On the RTX 4090 the probe
reads 0 of 4 pages written after the GPU read (4 after the import itself),
so imports stay watched there.
The import itself lifts the protection once where imports stay watched:
on the RTX 4090 the pinning at vkAllocateMemory reports every imported page
as written (4 of 4 in the probe), with no CPU store. Such a report would
read as a CPU write over any GPU results pending in the range and drop
them. So every import of a watched range is made inside an import window
(GuestMemory::ImportWatched), under the tracker lock: the range is
collected right before the import, so earlier CPU writes are stamped as
usual, and again right after it, and what the window reports is stamped
"maybe written". Readers without pending results see it as a change
(UnchangedSince fails, ChangedBlocks reports it), so caches re-upload or
compare. Decisions about pending GPU results read it as untracked:
ChangedBlocks reports BlockMaybeWritten apart from BlockWritten, and the
drop rule, the write-back's keep blocks, the alias checks, the unit
shadow's stale drop and the whole-image stale checks (through the new
WrittenSince, which also answers false for a range off the watch) store
the results instead of dropping them. A real CPU store racing an import
also races the pending GPU writes to the same memory, which has no
defined order on hardware either, so the GPU results winning is
acceptable. From the second collect on the range is protected again and
watched normally. A range the watch does not cover whole, or a collect of
the window that fails, is taken off the watch instead.
An unwatched import stays off the watch for the rest of its mapping: a
remap registers the range afresh, the import kept over it takes it off
again at the next reconcile, and a re-import after an unmap follows the
decision. Unwatch invalidates the collect memo, so a collect memoized
before the import answers 0. A range that leaves the watch while a storage
image holds a generation for it no longer reads as CPU-written:
ChangedBlocks now tells whether the range is watched, and the drop rule,
the write-back's keep blocks and the alias borrow treat an unwatched range
as untracked (stored, never dropped), as a generation of 0 is.
Tests: guest_memory checks the watch through libkernel mappings (a CPU
store, a read, a kernel write through read(), another thread, a memcpy
across pages, an mprotect cycle, a partial munmap and a fixed remap) and
over a fresh 2 MiB-aligned mapping without page tables, checks direct
memory (watched, an alias takes the shared pages off the watch in both
mappings, a fixed direct remap over flexible memory is watched afresh), and
names the failing line and the reported pages. The recorder tests allocate watched
memory on both platforms, so the UnchangedSince, key proof, close race and
unit shadow tests run on Linux, and the storage refresh test gains a
watched variant: results kept on the GPU reach a CPU reader, and a plain
CPU store into one unit with newer results pending is found by the watch
alone. The recorder tests now also run the real import probe, check the
decision with the result injected (unwatch: an import leaves the watch, a
memoized collect, a partly imported range and the unimported neighbour
answer as they should, MarkWritten stamps only watched memory, a remap and
a re-import after an unmap follow the decision; watch: the import stays
watched and stamped), and check that results pending when their memory
leaves the watch are stored rather than dropped. A further test imports the
memory of a storage image with results pending in watch mode: the
results survive the import window and reach guest memory, a cache
generation taken before the import sees a change, the window reads as
maybe written and not as a CPU store, and a CPU store after it reads as
one again. Unit shadows and the
watched storage refresh report themselves as not run when imports are
compared.
The geometry path's merged ES/GS programs (VGT_SHADER_STAGES_EN.GS_EN)
were compiled as mesh-stage programs whose input metadata was never
filled in, so every such draw failed with "mesh shader translation
configuration is not supported".
Recompiler:
- BuildShaderStageInputInfo takes the draw's MeshConfiguration; the mesh
input helpers (primitive size, step and counts) are implemented.
- The prologue emulates one NGG subgroup per workgroup: s2 GS_TG_INFO,
s3 the merged wave info per wave, the GS vertex offsets (ES thread
index times VGT_ESGS_RING_ITEMSIZE, odd strip triangles swapped), the
primitive id, and the ES vertex and instance ids. An indexed draw's
vertex id is read from the index buffer through a raw V# the driver
puts in hidden user words 4..7.
- s_sendmsg GS_ALLOC_REQ becomes MeshAllocate. The epilogue zeroes the
allocation first, clamps it to the output limits, culls primitives no
thread exported and decodes the 9-bit vertex indices.
- The draw parameters live in the last six push dwords. A BDA fault in
a mesh program no longer stops the invocation (as for other programs
with barriers, the fault is recorded and the read returns zero), so
every invocation reaches the subgroup barriers and the epilogue.
- The cache keys cover the mesh configuration; requests serialize its
new esgsItemSize field (version 4).
Driver:
- DecodeShaderStages reads VGT_ESGS_RING_ITEMSIZE (now in the draw key)
and sizes the subgroup for its ES vertices, GS primitives and outputs.
- Stages compile below the mesh draw parameters, which every mesh draw
pushes; the mesh stage always takes part in the push range.
- The index-buffer V# is set per draw (per record on the CPU-resolved
indirect path); the index-buffer alias check spares read-only
elements.
- The workgroup and LocalSize checks count two guest lanes per
invocation on 32-wide subgroups; mesh draws may carry a first vertex
or instance.
agc_driver_mesh_tests (EXCLUDE_FROM_ALL, needs VK_EXT_mesh_shader)
recompiles a hand-assembled merged ES/GS program in the PS5 compiler's
layout and draws indexed and non-indexed triangle lists in one- and
two-wave subgroups and a triangle strip across odd-starting subgroups,
reading the pixels back. agc_shader_replay prints a request's draw and
mesh configuration.
Michon
Dean
Dean Galvin
Alexey
SamG-Coder