The validator takes the geometryShader and sampleRateShading enables from the
context (set where #295 enables them), so a device without them refuses the
draw instead of creating a module that needs features it never enabled.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Storage images of a SINT format threw "storage image descriptor uses
an unsupported format"; only write-only 32_SINT was accepted.
The recompiler now writes 16_SINT, 8_8_SINT, 32_SINT, 16_16_SINT,
8_8_8_8_SINT, 32_32_SINT, 16_16_16_16_SINT and 32_32_32_32_SINT
storage images with the Uint numeric class. 8- and 16-bit components
are saturated (EmitFormatStoreComponent) and 32-bit ones stored
unchanged, as measured on RDNA2 (gfx1035); image_store_pck stores
32-bit components unchanged too. The driver binds these surfaces
through a cached view of the same-size UINT format.
8_SINT, 32_32_32_SINT, the packed SINT formats (11_11_10, 10_11_11,
2_10_10_10, 10_10_10_2), D16 stores, packed stores of 8- and 16-bit
components and atomics other than 32-bit ones on 32_SINT and 64-bit
ones on 32_32_SINT still throw. New test: agc_driver_image_store_sint,
with DST_SEL X,Y,0,1 on the two-channel formats, as with X,Y,Z,W the
hardware writes channel 1 from the W data slot (TechnicalDebt).
NVIDIA 610.62 cannot sample R8G8_SRGB, and lavapipe can sample neither
it nor R8_SRGB, so an 8_8 sRGB T# made an image the device does not
support: creating it crashes the NVIDIA driver and lavapipe loads zeros.
The device's format support is read once. An 8 or 8_8 sRGB texture the
device cannot sample is created and viewed as R8_UNORM or R8G8_UNORM,
and SpirvTarget::srgbDecodeFormats makes the materializer mark the
image, so image loads map every channel through a 256-entry table of
the sRGB EOTF rounded to nearest even at bf16 precision, which is what
RDNA2 (gfx1035) returns; both channels of 8_8 are sRGB, and DST_SEL 0
and 1 map to themselves. Samples and gathers of such an image throw,
and its bindless table entries are unmapped like converted formats.
APS5_SRGB_SHADER_DECODE=1 takes this path on every device. Shader disk
cache entries and keys carry the mark: cache format version 7.
A pixel or vertex shader with a 64-bit image atomic declares Int64Atomics,
Int64ImageEXT and SPV_EXT_shader_image_int64, which graphics shader
validation rejected ("SPIR-V requires unsupported device capability"),
so the draw was dropped. Compute dispatches are not validated this way.
The device records whether it enabled shaderImageInt64Atomics
(imageInt64Atomics in the device state and the graphics Context), and
ValidateShaders accepts the two capabilities and the extension when it
did.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
image_atomic_* with a two-dword data mask (four for cmpswap) is the
64-bit form of an integer image atomic. The decoder rejected it with
"invalid MIMG data mask". PPSA14632 (Unreal Engine) uses
image_atomic_umax this way in a compute shader during its first frames.
Swap, add, sub, signed and unsigned min/max, and, or, xor and cmpswap
now decode with 64-bit data and translate to new ImageAtomic*64 IR
opcodes. An image updated by them gets its own descriptor binding
range, declared as an R64ui image of 64-bit integers, and the module
uses Int64ImageEXT from SPV_EXT_shader_image_int64. The driver enables
shaderImageInt64Atomics when the device offers it and binds an
R64_UINT view of the 32_32 UINT, SINT or FLOAT surface. Without the
feature, recompiling such a shader throws.
64-bit inc, dec and the float atomics still throw, as does any other
access in the same shader to an image it updates with 64-bit atomics.
The new agc_driver_image_atomics64 test runs every operation on 32_32
UINT, SINT and FLOAT surfaces and checks the returned and stored values
against edge cases across the dword boundary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An S# with FILTER_MODE (word0 bits 29-30) set threw in
DecodeSamplerResource. Modes 1 and 2 now map to Vulkan's MIN and MAX
sampler reduction through VK_EXT_sampler_filter_minmax, which the device
enables when it reports both filterMinmax properties; mode 3 throws.
Min/max still throws with anisotropy, a linear mip filter (lavapipe and
NVIDIA disagree there), depth comparison, color-texture compare
emulation, and, in PointFilteredSamplerWord (S# words 0 and 2), when it
filters between texels or mips of an image the recompiler point-samples.
A texture sampled through a linear min/max sampler needs
SAMPLED_IMAGE_FILTER_MINMAX on its view format; the image-sampler pairs
travel in DescriptorBinding::imageSamplers (shader disk cache format 5).
Adds the agc_driver_image_sample_reduction GPU test and extends the
sampler, emulated color compare and disk cache tests.
GFX10 image format 30 (10_11_11_UNORM: X and Y are unorm11 in bits
0-21, Z is unorm10 in bits 22-31, per Mesa's gfx10 tables) has no
Vulkan format, so its sampled and storage descriptors threw at decode.
Format 30 now takes the R32_UINT route of 10_11_11_UINT: the driver
binds the raw dwords as R32_UINT and the shader extracts each field and
converts it with the buffer format helpers (NormalizeFormatComponent on
loads, EmitFormatStoreComponent on stores); DST_SEL 1 reads 1.0f.
Samples, gathers, LOD queries, 16-bit data and loads or stores whose
DST_SEL names the missing W channel throw.
Converted textures are now viewed with an identity component mapping,
since the shader applies DST_SEL itself: a 10_11_11_UINT view whose X
selector was not X lost the packed dword. DCC clear codes 0001 and 1110
of a converted texture throw instead of filling one R32_UINT channel.
An S# with FORCE_UNNORMALIZED threw in DecodeSamplerResource. It is now
bound through a Vulkan sampler with unnormalizedCoordinates when every
use stays where the S# and the host select the same texels.
ShaderInfoCollector records the live uses of each sampler after Apply
and dead-code elimination, and the emitter shares its explicit-LOD
predicate. Populate flags a direct S# with the bit that only explicit-LOD
image_sample without offset, comparison, derivatives or _a uses, paired
only with direct 1D or 2D images without a format conversion; anything
else throws. The driver checks the S# filters, clamps, TRUNC_COORD and
MC_COORD_TRUNC, creates the sampler with nearest mips and a zero LOD
range, and requires a single-level, single-layer 1D or 2D view of mip 0.
The shader disk cache format becomes 5.
sceVideoOutRegisterBuffers2 threw "DCC presentation is not implemented"
for category 1 (DCC-compressed) buffers, which PPSA26344 registers. Such
a buffer now needs DCC metadata and the tiled layout, and dcc_control may
only use the CB_DCC_CONTROL block layout bits 0x10026c.
At a flip the keys are read behind their recorded writers, one byte per
256 bytes as in the driver's DCC model. Uncompressed keys present the
stored texels as before; a uniform clear code presents its color with a
GPU clear of the scaler's source image; a pending storage image is used
when it serves the keys. Mixed or unreadable keys, clear codes the
display format has no DCC encoding for, register-clear keys over a
pending image and a clear color wider than 32 bits throw. Buffers
without DCC take the old path.
The resource materializer refused every atomic image descriptor whose
format was not 32_UINT. RDNA2 image atomics act on the raw dword and
the numeric format only affects addressing, so 32_SINT and 32_FLOAT
surfaces are now accepted with the Uint numeric class and run through
the existing integer atomics and float compare-exchange loops. Other
formats still throw, and the message now names the format.
DescriptorBinding gains imageAtomic, carried through the shader disk
cache (format version 4) and the resource content key. The driver
binds those elements through a cached R32_UINT view of R32_SINT and
R32_SFLOAT storage images.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3D textures in SW_4KB_S or SW_64KB_S whose smaller levels reach the mip
tail were refused ("3D texture mip tails are not implemented"). The
tail now follows addrlib's GFX10 thick layout
(Gfx10Lib::ComputeSurfaceInfoMacroTiled):
- a level enters the tail once its size, rounded up, fits the thick
tail extent (the block with its height halved for 4 KiB blocks and
its width halved for 64 KiB blocks) and at most GetMaxNumMipsInTail
levels remain (5 for 4 KiB, 10 for 64 KiB);
- each slab starts with one tail block, then the other levels from the
smallest to level 0;
- tail level i sits in slot maxMipsInTail - 1 - i, at the element offset
addrlib derives from that slot's byte offset and the 256-byte thick
micro block, at every depth slice.
The tests check element addresses of five volumes (4 KiB and 64 KiB
blocks, 1 to 16 bytes per element, odd sizes, a chain that starts in
the tail) against Mesa addrlib's Addr2ComputeSurfaceAddrFromCoord for
a Navi10 configuration.
PPSA21564 clears each occlusion query slot with a 256-byte WRITE_DATA
before its begin dump, about 8,400 times per frame in one level. The
clear is larger than a deferred label, so it is recorded at once, and
Recorder::NoteLabel entered all 64 of its dwords in the pending-label
table: 64 map inserts, 64 batch dword entries and 64 group entries, then
64 lookups in CloseLabelGroup and 64 erases when the batch is reaped. A
replay of that pattern (21 clears per batch, 20 batches in flight) costs
1.6 us for the inserts, 0.7 us for the close and 1.3 us for the erases
per clear, about 31 ms of the graphics worker per frame there. The new
path costs 0.07 us per clear in the same replay.
The table serves WAIT_REG_MEM values and CPU pollers, which read 4 or 8
bytes. A store larger than 64 bytes (the bound the driver's own label
history already uses) now enters one range instead of its dwords:
- table entries inside the range are removed, as the store replaced
them before: a lookup there finds no value and falls back to the
pending-write path, where it used to find the store's value;
- the range leaves with its batch, like the dword entries;
- PendingLabelIn sees the range, and the write-evidence check, which
syncs a capture read over any pending label dword, now also syncs
over a pending range (Recorder::WideLabelIn).
The store is still a pending write of the open batch (the flush hook
syncs CPU readers), still starts the label flush deadline and is
recorded exactly as before. Stores of 64 bytes or less are unchanged.
Test: agc_driver_recorder_tests largeLabelTests (a 256-byte label enters
no dwords and removes the older entry under it but not one beside it,
its range, pending write and deadline, a later 4-byte label over it,
a 64-byte label entered in full, everything gone after the batch).
Every occlusion counter dump the recorder handles on the GPU ends the
batch's occlusion query, copies the pending results, folds them with a
compute dispatch, then resets and begins a new query. Each of those
commands was resolved through vkGetDeviceProcAddr, a loader name lookup,
on every call: seven lookups per dump on the graphics worker, and a title
dumps the counter twice per occlusion query. The mesh argument and timing
paths resolved their bind, push, dispatch and reset per call too.
They join the entry points the recorder already resolves in its
constructor: vkCmdBeginQuery, vkCmdEndQuery, vkCmdResetQueryPool,
vkCmdCopyQueryPoolResults, vkCmdBindPipeline, vkCmdPushConstants and
vkCmdDispatch. The same functions are called with the same arguments;
APS5_NO_PROC_TABLE=1 still resolves every call.
A GPU-direct storage write-back retiles the image's elements into an
uninitialised device scratch buffer and then copies whole ranges of it
into the import or the import's unit shadow. The retile writes only the
bytes elements occupy, so every other byte of the copied ranges was
stored from uninitialised scratch memory: the padding of the tile blocks
a surface covers only partly (its last block column and row), mip tail
blocks, the bytes between mips and the pitch padding of linear surfaces.
Another surface over the same memory, or the CPU, then read garbage
there instead of what the memory held. Both the block-window path and
the whole-layer path did this.
Before the retile, the bytes of the copied ranges that no element holds
are now copied into the scratch from where they are current (a fresh unit
shadow, else the import), so the write-back stores them unchanged. Whole
tile blocks every byte of which holds an element are left out
(CoveredMipBytes), so only partly covered blocks pay for the extra copy.
Tests: CoveredMipBytes against the XOR address equations of the four
64 KiB XOR modes at every element size; a device test that stores
cleared images over patterned memory (tiled with and without mip chains,
off a 64 KiB boundary, 4 KiB standard, linear; plain and watched memory)
and checks every byte that is not a texel kept its pattern.
Pixel shader color exports are compacted: export MRTi goes to the i-th
MRT slot with a non-zero CB_SHADER_MASK nibble and takes the i-th
SPI_SHADER_COL_FORMAT field. Draws still mapped export i to slot i and
read slot i's format field, so a shader with CB_SHADER_MASK 0xf000f and
SPI_SHADER_COL_FORMAT 0x44 (export 1 to slot 4) was refused with
"color export format 0".
Each color target now carries its export index next to its slot.
Attachment references, blend states, export mappings and the pipeline
and pass keys follow the export index; the color buffer, blend control
and target mask come from the slot. An unwritten export below the
highest written one stays an unused attachment.
Color targets with the SW_4KB_S swizzle (CB_COLOR_ATTRIB3 tile mode 5) are laid out in 4 KiB
blocks with the standard element order the texture detiler uses, and are resident through the
same storage images as 64 KiB targets. A CB_COLOR_VIEW selecting one slice (SLICE_START equal to
SLICE_MAX) renders into that slice: the target starts one slice size per slice after the surface,
the slice size being the surface's mip chain as a texture of the same tile mode lays it out. Views
of several slices are still rejected.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
3D textures with more than one mip level were refused. Each slab of
blockDepth slices of a thick (SW_4KB_S / SW_64KB_S) volume now holds the
whole mip chain, smallest level first and level 0 last, as 2D chains are
laid out. A level's depth halves per level, so the texture loops skip the
slices a level does not have. Thin 3D (D/R 64KB_X) chains use the 2D mip
layout per depth slice. Thick mip tails still throw.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The descriptors of one surface share its storage image through the
surface key, which includes the level count. A view past MAX_MIP
counts the extra levels, so its storage image was a second image of
the same memory, and the two flushed and re-uploaded each other every
frame (PPSA21564 writes level 6 of each 1920x1080 bloom chain after
building levels 1 to 5 through MAX_MIP 5 views).
A surface once viewed past its allocated levels now keeps a single
image holding the extended chain, and its other descriptors look that
image up; the allocated-level image it replaces is evicted with its
results stored. The extra levels lie in the allocation (the decode
proved that), so levels 0 to MAX_MIP keep their layout and the views
of them read the same texels.
PPSA21564 ends each of its two 1920x1080 RGBA16F bloom downsample
chains (MAX_MIP 5) with a compute pass that stores level 6 through a
view of levels 6..6, every frame from the PS logo on. The descriptor
decode threw for such a view.
GFX10 uses MAX_MIP only to place the levels: a level's address comes
from the levels below it, and a level in the mip tail is a fixed slot of
the tail block counted from the first tail level. A level past MAX_MIP
is addressed the same way, so when the chain extended to the view's last
level keeps every allocated level, the tail and the surface size
unchanged, the view reaches that slot inside the surface's allocation.
addrlib (GFX10, Navi10 with 16 pipes) computes the same 0x1640000-byte
layout for 6, 7, 9 and 11 levels of this surface, with level 6 in the
tail at (0, 32). The view then decodes with the extended chain over the
same guest bytes. Views whose extra levels would move the allocated
ones, such as linear chains or chains without a tail, still throw.
Every ZPASS_DONE counter dump drained the device so the CPU could add up
the occlusion queries of the finished batches and store the running total.
PPSA21564 dumps around each occlusion-tested pass: about 1.5k drains per
10 s at its title screen, and in a level's early passes four drains a frame
that took 5.6 s of every 10 s of the graphics worker.
The running total now lives in a small buffer the GPU updates in batch
order. A batch still ends its occlusion query at submit; a query that saw
draws is remembered, and its pool is kept alive until it is folded. A dump
ends the open batch's query, copies the remembered results next to the
total, adds them with a one-thread compute shader that also stores the 16
DB counters into the dump's host import, and begins a fresh query for the
rest of the batch. The dump is ordered with the work around it like any
other GPU store, noted as a pending write and stamped. More than 64
remembered results are folded at a submit. Query pools are recycled.
A dump into memory the GPU cannot see, under a pending copied write-back
or over a label the CPU stores at a completion falls back to the drain,
which settles the remembered results on the CPU. Without buffer device
addresses the counting stays on the CPU.
A mesh program took its draw's index count from push constants, so an
indirect mesh-path (NGG) draw needed the CPU to read its record first,
waiting for the GPU to finish everything recorded before. In PPSA21564
that was about 170 waits per 10 s at the title, menu and level (about
1 s), and one per frame in the intro (2.2 s per 10 s).
The two free mesh push dwords now carry the device address of an argument
record (a VkDrawMeshTasksIndirectCommandEXT followed by the index count
and first index). When the address is set, the prologue takes the index
count and the first index from the record (the new MeshArgument IR value,
a load through a physical storage buffer pointer) and offsets its index
reads by the first index; a direct draw leaves the address 0 and draws as
before. Mesh modules declare the physical storage buffer capability for
it, and the driver's SPIR-V validation accepts the extension in mesh
modules.
An indexed mesh draw with one record that patches no user SGPR now stays
on the GPU. A one-thread compute pass (Graphics/shaders/MeshArguments.comp)
turns the record, read in place from its host import, into the draw's
argument record: the groups and instances of vkCmdDrawMeshTasksIndirectEXT
and the index count and first index the mesh program reads, with the CPU
path's rules (count clamped to the index buffer, nothing for an empty
record or past the device's group limits). The index buffer descriptor
covers the whole buffer and the program offsets by the first index. When
the records cannot be read in place, the CPU reads them and fills the
same argument record (ResolveMeshArguments).
PPSA21564's galaxy map renders a 1920x1080 surface as a color target whose
CB names DCC keys at one address and then writes it from a compute shader
whose T# names keys at another. Both key ranges read uncompressed every
frame, but the storage cache remade the surface's image whenever a
descriptor named metadata other than the image's, so the image was
flushed, destroyed, created and uploaded twice a frame (1231 constructions
in a 56 s scene), with the views and draw recipes built on it thrown away
each time.
A descriptor naming other keys now keeps the image when the move is
unobservable: the image's own keys and the named keys both read
uncompressed, and its content was neither uploaded nor filled under a
clear code. That is exactly the content an image made under the named
keys would hold once this one's pending results were stored. Any other
case (a fast clear of either key range, a register clear, an image
holding a clear) still remakes the image as before, so clears of the new
keys reach it and pending results are stored first.
Every proof that stands in for a lookup checks the same rule: the sampled
view hit, the fast revalidation of views and storage elements (storage
elements now record their descriptor's keys), the own-object refresh, and
the draw recipe's resident targets. Named-key scans keep per-image proofs,
so a repeated check costs no scan while the key bytes are unstamped.
agc_dcc_key_sharing drives KeysServeSurface, the rule ServesKeysAt uses,
through a model of the storage cache.
A CB metadata pass (and a resident target's register-clear check) read the
DCC keys with CurrentDccKeys, which waited for the GPU whenever recorded
work wrote the key range. In the intro video of PPSA21564 about 800 passes
per 10 s waited about 1.8 ms each, and every one of those waits was behind
a key store whose result the driver already knew: its own uncompressed key
store, or a title fill of uniform keys recorded on the GPU.
Pending-write notes now carry a serial in recording order, and the key
memo remembers the note of the store it describes. When the newest
pending note over the keys is that store's own note, nothing recorded
after it writes the keys, so they read as the store's keys without a
wait. Any later writer, a range the store does not cover, or a unit
shadow over the keys waits as before.
DB_DEPTH_CONTROL's DEPTH_BOUNDS_ENABLE now runs the Vulkan depth bounds
test with DB_DEPTH_BOUNDS_MIN/MAX, and PA_SU_SC_MODE_CNTL's
POLY_OFFSET_FRONT/BACK_ENABLE enable depth bias from
PA_SU_POLY_OFFSET_FRONT/BACK_SCALE/OFFSET and _CLAMP. Both are dynamic
state, so draws that differ only in their values share a pipeline.
The hardware slope scale is in 1/16 units. Depth bias with units other
than the depth format's (PA_SU_POLY_OFFSET_DB_FMT_CNTL), or that differs
between front and back faces that are both drawn, is refused. The
registers join the draw key. Depth-conditional color writes are still
refused.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The [present] line, printed under APS5_PROFILE_DRAW, takes its GPU busy and
idle gaps from the whole-batch timestamps in each batch's completion record.
Those stamps were only written under APS5_PROFILE_GPU, which also times every
command. Under APS5_PROFILE_DRAW alone every batch finished with zero stamps,
so the line read "GPU busy 0.0 ms, idle gaps 0.0 ms" with no missing records.
Under APS5_PROFILE_DRAW each batch now carries just its two batch stamps in a
two-query pool, reused across batches like the command buffers.
APS5_PROFILE_GPU is unchanged, and nothing is written with profiling off.
agc_driver_recorder_tests (GPU, by hand) checks that three consecutive
batches' completion records carry ordered stamps exactly when batch stamps
are on, and none with profiling off.
Michon
Adrià
Celegans12
alfalfa-star
Mehran
Dean
Dean Galvin
Nicolas Abosch
Pacuka
Alexey