A build's sampled image elements often repeat the element just before them
in plan order (the same eight descriptor words). Each still took its own
stage-A record (decode, collect, DCC key read, cache entry) and its own
stage-B lookup.
The lookups run in plan order and nothing runs between two consecutive ones,
so a repeated element's lookup would find the state the previous lookup just
left: the entry it validated or made, current against the same collect. Such
an element now binds the previous element's texture (when the depth-compare
flag matches too, which is part of the cache key), and stage A gives it a
record of the decoded descriptor only. An element repeating an earlier but
not the immediately preceding one still takes its own lookup, as does any
element after a storage image lookup.
A resource build decodes every image element's eight descriptor words,
describes the surface (a mip layout with a vector of levels) and finds the
sampled texture's cache entry by hashing its key, once in stage A and again
in stage B. Most elements repeat a small set of descriptors from build to
build.
Each worker now keeps a small direct-mapped memo by descriptor words of the
decoded resource, its guest size and view component mapping (pure functions
of the words) and of the cache entry the words last found. The entry is
reused only while the sampled texture cache has erased no entry since it
was found (an erase counter bumped by eraseTexture), so the list iterator is
still valid and still holds the same key; anything else takes the hash
lookup.
Every draw into a resident target refreshes the target's storage image
(StorageTexture::Refresh): a pending-registry scan for an alias,
FlushPending over the surface, the collect, the key proof, a
whole-surface UnchangedSince, four vectors and a generation update over
every 64 KiB unit, nearly always finding the image current.
A refresh that finds the image current now keeps a proof: the image's
generation and uploaded keys, its key proof's generation and the pending
registry serial. The next refresh is answered from it when the image's
own state is still that, the surface and its key range are unstamped
since those generations (CollectWrites over both, then one
UnchangedSinceAll), and no other image has results pending over the
surface: the registry serial is unchanged, or the registry is scanned and
holds none (a newly pending image, a flush in progress). The proof is
taken only under GuestMemory::GpuMutex, without a pending alias, with
stable keys and when no other pending image overlaps the surface; any
full refresh drops it. A CPU write, a driver or GPU store, a key store,
another image's pending results or a different image therefore always
refresh in full, and the image's own pending results (a draw into it)
keep the proof.
agc_driver_recorder_tests refreshProofTests: an unchanged image and one
holding its own pending results are proved; a CPU write, a driver store
into the surface, a key store, another image's pending results, a new
image of the surface and a fast clear of the keys each refresh in full.
A stale storage snapshot is revalidated by comparing every tracker block
stamped since it was taken. A title that binds buffers a dispatch writes
in place every frame (the whole declared range stamped, few pages really
changed) makes the graphics worker compare gigabytes per second.
Compares of 1 MiB or more are now split by block across a small pool of
helper threads (hardware threads / 8, from 1 to 4). The compare is memory
bound, so the worker's share roughly halves rather than dividing by the
thread count. A caller that finds the pool busy compares on its own
thread. Results and the patch-in-place rules are unchanged.
agc_driver_graphics_tests parallelCompareTests: per-span results on the
pool and serial paths, and two threads comparing at once.
A stale storage snapshot was compared whole with the guest bytes and, on
any difference, copied whole into a new buffer, although few of the
tracker blocks stamped since the copy usually hold different bytes.
Only the 64 KiB tracker blocks stamped since the snapshot's generation
are compared now. When some differ and the cache holds the only
reference (no recorded draw binds the buffer), those blocks are copied
into it in place; otherwise the range is copied again as before.
agc_driver_recorder_tests drawSnapshotPatchTests: changed bytes in
several blocks and a whole changed block patch the snapshot nothing else
holds, an unchanged one is reused, and one a recorded draw still holds is
copied again with its bytes untouched.
A written V# element bound in place in a host import makes every load a
PCIe round trip. A particle update that loads and stores a 10 MiB pool
(2048x64 threads over 131072 records of 80 bytes), above the 2 MiB written
window, took about 4.8 ms of GPU per frame.
An element the shader also reads (DescriptorBinding::bufferRead), written by
a direct dispatch whose threads times the V# stride cover at least half of
it, now takes device-local staging up to 16 MiB: that dispatch drops to
about 0.06 ms plus 0.9 ms of bulk copies. The coverage test keeps a kernel
that updates about 12k records of a 16 MiB pool in place, where staging all
of it would cost 1.3 ms of copies for a 0.08 ms kernel. Write-only
elements, indirect dispatches and raw (stride 0) buffers keep the 2 MiB
window.
A storage write-back copied the image into a linear buffer and retiled
that buffer; an upload detiled into a linear buffer and copied it into
the image. With storage image access without a format, the retile now
reads each texel from the image through an unsigned integer view of the
element size, and the detile writes it there, so the linear buffer, its
copy and one barrier per move go away. The texels move bit for bit as
before, through the same tiling equations and windows.
This covers the block-unit windows of write-backs and uploads and the
whole-surface direct upload, for 2D, array and cube images.
agc_driver_recorder_tests singlePassTests: a first upload, a write-back
of new texels in every mip and a second upload over the stored memory
leave the images and guest memory byte for byte equal to the copies
through a linear buffer, in plain and write-watched memory.
A recorded draw continued the open render pass only after a draw that
wrote no memory, so every draw with a storage write ended the pass and
paid two barriers. A draw now continues the open pass when its accesses
have no hazard (PassHazards) against those of the draws since the last
barrier. Address-based draws, draws through buffer device addresses and
draws with copied writes count as reading and writing any guest memory.
A draw of other attachments with no hazard ends the pass without a
barrier and begins its own without one, and a hazard break records the
begin barrier alone, towards every graphics stage, instead of the pass
end barrier plus the begin barrier. A pass with a depth attachment or
attachment proxies, or one followed by work after the pass, keeps both
barriers. APS5_FULL_BARRIERS keeps every barrier, as for the other merges.
PassHazards keeps the guest ranges, images and attachments the draws
since the last barrier read and wrote, and answers whether a new draw has
a RAW, WAW or WAR hazard against them. Address-based reads and unknown
writes count as touching any guest memory; attachments of the current
pass are ordered by rasterization and only conflict with other passes or
with sampled and storage uses. Unit tests in agc_pass_hazards.
A draw's storage snapshot was copied again whenever the write tracker
reported a store over its range, although most such stores leave the
bytes as they were: in one title screen about 10k of 14k stale lookups per
10 s held the same bytes (3.5 GiB of copies).
A stale storage snapshot is now compared with the guest bytes, after the
caller's collect of the range, before it is dropped. When they are equal
it takes that collect's generation and is reused; any store after the
collect is stamped newer and fails the next check, as for a fresh copy. A
registry change still drops it without reading, and vertex and index
snapshots are dropped as before.
materializeRegisterClear read the target's DCC keys with CurrentDccKeys on
every draw into a resident target with metadata: a full AllKeysEqual scan
of the key range (52 KiB for a 2432x1368 RGBA8 target), about 440k scans
per 10 s and 7.2% of the queue 0 worker's CPU time in one level.
ProvedCurrentDccKeys answers CurrentDccKeys from a proof kept with the
resident image (DccRangeProof: the key range, the keys and the write
generation collected before the scan), the rule ProvedClearKeys already
applies to a storage image's own keys: the proof holds while the key range
is unstamped since that generation (CollectWrites first, then
UnchangedSince), and a scan becomes the proof only when it read the bytes
(not the pending store memo) and no recorded work still writes them. Every
key writer stamps the range, so a store, a pending result or another key
range always scans again. The proof is used under GuestMemory::GpuMutex
only, and not with APS5_NO_KEY_FAST_PATH.
agc_driver_recorder_tests: an unchanged range is proved, and a key store
stamp, a CPU write, another range or address, and a pending key fill each
force a scan; the fill's result is proved once its batch finished.
Every small host buffer (size classes below 1 MiB: draw and dispatch data,
copied regions, snapshots) had a VkDeviceMemory of its own. When the pool
missed, the worker paid a vkAllocateMemory and vkMapMemory, and the release
thread a vkUnmapMemory and vkFreeMemory, for each one. The frees also stall
the worker's vkQueueSubmit in the driver.
A host-visible buffer of a size class now gets its own VkBuffer bound into a
slot of a shared block (2 MiB, or 8 slots for the large classes) per memory
type, size class and device-address use, mapped once. Releasing it destroys
only the VkBuffer and returns the slot; a block is freed when it empties and
its slab already keeps an empty one. A slot is used only when the buffer's
alignment and the non-coherent atom divide its size class, so the slot
offset is valid for the bind and Invalidate covers exactly the slot. The
pool still retains released buffers by class as before; only what backs
them changed.
A draw whose stages bind a BDA table or a fault buffer builds resources
that need completion work, so noteReusable never keeps them and the shared
resource cache never holds an entry such a draw could hit. Building the
content key and probing the cache for each of them was pure cost. Those
draws now skip both (ShaderResources::NeverReusable) and count as
uncacheable.
The hardware turns anisotropy off under ANISO_OVERRIDE when the T#'s BASE_LEVEL equals its LAST_LEVEL. The view's level count is taken after LAST_LEVEL is clamped to MAX_MIP, so a T# with BASE 0, LAST 3, MAX_MIP 0 counted as single-level here while the hardware keeps anisotropic filtering for it (measured by @adriadam10 on the hardware oracle). The pairing now reads BASE_LEVEL and LAST_LEVEL from word 3 (DescriptorSingleLevel).
The per-draw path gets a test, agc_driver_image_sample_aniso_override: one ANISO_OVERRIDE sampler shared by two images in a dispatch, with single-level, mipmapped, mixed, and LAST_LEVEL > MAX_MIP descriptors, checking which pairings run (with their samples) and which are refused as mixed. With the old view-based check the LAST_LEVEL > MAX_MIP pairing is refused.
applyAnisoOverride reads the APS5_NO_SAMPLER_CACHE switch through the same cached flag as addImageBinding instead of calling getenv per sampler per draw.
AI-assisted (Claude Code).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With ANISO_OVERRIDE the texture unit disables anisotropic filtering when the
sampled view has a single level. Each draw now records, per sampler, whether
its paired images are single-level or mipmapped. A sampler with the bit and an
anisotropic filter whose images are all single-level is bound as the same S#
with point or linear filtering in place of the anisotropic filters, taken from
the sampler cache. One sampler descriptor serves every image paired with it,
so a sampler paired with both kinds in one draw still throws. Samplers of
mipmapped images keep their anisotropy, as on the hardware.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CB_BLENDn_CONTROL factors 15-18 (BLEND_SRC1_COLOR, BLEND_INV_SRC1_COLOR,
BLEND_SRC1_ALPHA, BLEND_INV_SRC1_ALPHA) threw "unsupported blend
factor". They blend MRT0 with the pixel shader's MRT1 export as the
second source, as RADV and radeonsi program it.
DecodeState maps them to the VK_BLEND_FACTOR_SRC1_* factors and accepts
them only with pixel exports to slots 0 and 1, color writes to slot 0
alone, an MRT1 export with MRT0's export format and shader mask, an
ADD, SUBTRACT or REVERSE_SUBTRACT operation and an identity component
order. The blend is draw state, so the specialized module of the
registered pixel shader moves its MRT1 output to location 0 index 1,
which ValidateShaders requires; pipelines need dualSrcBlend, now enabled
when present. The request field dualSourceBlend is in the module key,
the result memo, the draw key and the serialized request (version 15).
Guest samplers with DISABLE_CUBE_WRAP (S# word 0 bit 28) are created with
VK_SAMPLER_CREATE_NON_SEAMLESS_CUBE_MAP_BIT_EXT when the device enables
VK_EXT_non_seamless_cube_map; RADV maps the flag to the same bit. The bit
only changes cube filtering, so without the extension the sampler is still
created and only a cube texture sampled through it throws.
The buffer pool's device tier (detiler scratch buffers and the staging
shadows of written guest buffers) retained at most 512 MiB by default.
A title whose working set of these buffers is larger allocates and
frees device memory continually: PPSA14632 (LEGO Horizon Adventures)
had 70761 misses and 69828 evictions per 10 s with 485 of 512 MiB
retained, each a vkAllocateMemory or vkFreeMemory.
The default budget is now an eighth of the largest device-local heap,
and at least the old 512 MiB, so a 16 GiB card retains up to 2 GiB and
an 8 GiB card 1 GiB. APS5_STAGING_POOL_MIB still sets it.
Only the constructor knows the device, so only the budget comes from the
heap. Whether the tier is in use at all (APS5_STAGING_POOL_MIB unset or
not 0) stays a static check, DeviceTierEnabled, which Capacity, Usage and
tierFor share, so the size classes and the tier stay consistent.
On PPSA14632 with 2 GiB the device tier has 1312 misses and 99
evictions per 10 s (1628 MiB retained), and presents rise from about 34
to about 39 per 10 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FillClearColor refused half and B10G11R11 patterns with a denormal code,
since an implementation may flush them in a clear, so such fills over a
resident image took the CPU path: PPSA01325 fast-clears its 3840x2160
R16G16_SFLOAT target 0x515390000 with a denormal half every frame, a
33 MiB guest write and re-upload (about 85 ms).
ClearKeepsDenormals probes the device once per format: a 1x1 image
cleared with denormal codes (half 0x0001, 0x83ff, 0x0200, 0x8001;
11/10-bit 0x001) and read back. When the bits survive, FillClear takes
the refused pattern again with SmallFloat decoding denormals exactly.
NaNs, float32 denormals and the SNORM minimum stay refused.
agc_driver_recorder_tests clears an R16G16_SFLOAT resident image with a
normal, a denormal and a NaN pattern: the denormal is cleared exactly
when the probe says the device keeps it, and the NaN is refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UTjpbMX7BUFMZZaxJE98zV
A strided vertex fetch whose index is at or past the descriptor's num_records threw and aborted the title. PPSA01325 draws its fullscreen composite as a 4-vertex strip over a 3-record buffer, so the fourth vertex past the records stopped presentation.
Hardware returns (0, 0, 0, 0) for such a fetch with OOB_SELECT 0 and 1, measured on RDNA2 gfx1036 with tools/hw-oracle in #2241. VertexBufferReadSize now spans the whole fetch, and the staging copy reads the record bytes then zero-fills from num_records * stride onward. Each such fetch gets its own copy, outside the shared copies of its sibling attributes, the host-import in-place path and the draw-snapshot cache. OOB_SELECT 2 and 3 keep throwing past num_records, since each needs its own rule. Recorded in docs/dev/TechnicalDebt.md.
The pin waiter now receives the mutation range, drains GPU work and retires every overlapping host import before the unmap callback releases the pages. Deferred import retirement keeps only the device handle and the Vulkan destroy functions, so the static import registry no longer owns a Context or its BufferPool past the device.
A draw that rasterizes both faces with different front and back depth
bias was rejected, since Vulkan has one depth bias per draw and the
cull mode is pipeline state. Such a draw is now recorded twice in the
same pass: front faces with the back faces culled and the front bias,
then back faces with the front faces culled and the back bias. A face
whose offset is disabled gets no bias.
Dean
Dean Galvin
Adrià
Adrià Franch
Scott Schuster
Celegans12
AlxcBTW
alfalfa-star
Raihan Abdul Majid
danzaio
MarcZX100
Sasha Papo
Daniele Giovane
Joey Fisher