On GFX10.3 to GFX11.7, ACO lowers a subgroup-scope OpReadClockKHR to
s_getreg_b32 of SHADER_CYCLES, 20 bits zero-extended to 64, so s_memtime
wrapped every 2^20 cycles and went backwards. Detect those families from
the RADV device name and read the device clock there instead.
Fixes#985
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DecodeState and the draw precheck rejected DB_SHADER_CONTROL bit 16,
PRIMITIVE_ORDERED_PIXEL_SHADER (POPS), which runs the ordered section
of overlapping pixels' shaders in primitive order. The bit now reaches
the recompiler as a pixel stage field, and such a pixel shader declares
PixelInterlockOrderedEXT and runs its whole body between
OpBeginInvocationInterlockEXT and OpEndInvocationInterlockEXT; an
invocation that a BDA read fault kills ends the interlock first. The
POPS collision SGPRs read 0, so a title skips its POPS wait and runs
its per-collision loop once. An image_load still reads a sampled
binding, which misses an earlier overlapping primitive's image_store in
the same draw. The device enables VK_EXT_fragment_shader_interlock when
it has fragmentShaderPixelInterlock; without it the recompile throws,
as does a nonzero POPS_OVERLAP_NUM_SAMPLES. The field is part of the
recompile cache key, the draw key and the serialized request (version 10).
A compute program with COMPUTE_PGM_RSRC2.SCRATCH_EN takes its per-lane private size from the AGC
shader header (scratch_size_dw_per_thread) instead of being refused. The size travels in
ShaderComputeStageInfo (cache key, request serialization version 10) to the translator, whose
scratch accesses already lower to a per-invocation Function array of that many dwords.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
NVIDIA 610.62 cannot sample R8G8_SRGB, and lavapipe can sample neither
it nor R8_SRGB, so an 8_8 sRGB T# made an image the device does not
support: creating it crashes the NVIDIA driver and lavapipe loads zeros.
The device's format support is read once. An 8 or 8_8 sRGB texture the
device cannot sample is created and viewed as R8_UNORM or R8G8_UNORM,
and SpirvTarget::srgbDecodeFormats makes the materializer mark the
image, so image loads map every channel through a 256-entry table of
the sRGB EOTF rounded to nearest even at bf16 precision, which is what
RDNA2 (gfx1035) returns; both channels of 8_8 are sRGB, and DST_SEL 0
and 1 map to themselves. Samples and gathers of such an image throw,
and its bindless table entries are unmapped like converted formats.
APS5_SRGB_SHADER_DECODE=1 takes this path on every device. Shader disk
cache entries and keys carry the mark: cache format version 7.
image_atomic_* with a two-dword data mask (four for cmpswap) is the
64-bit form of an integer image atomic. The decoder rejected it with
"invalid MIMG data mask". PPSA14632 (Unreal Engine) uses
image_atomic_umax this way in a compute shader during its first frames.
Swap, add, sub, signed and unsigned min/max, and, or, xor and cmpswap
now decode with 64-bit data and translate to new ImageAtomic*64 IR
opcodes. An image updated by them gets its own descriptor binding
range, declared as an R64ui image of 64-bit integers, and the module
uses Int64ImageEXT from SPV_EXT_shader_image_int64. The driver enables
shaderImageInt64Atomics when the device offers it and binds an
R64_UINT view of the 32_32 UINT, SINT or FLOAT surface. Without the
feature, recompiling such a shader throws.
64-bit inc, dec and the float atomics still throw, as does any other
access in the same shader to an image it updates with 64-bit atomics.
The new agc_driver_image_atomics64 test runs every operation on 32_32
UINT, SINT and FLOAT surfaces and checks the returned and stored values
against edge cases across the dword boundary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An S# with FILTER_MODE (word0 bits 29-30) set threw in
DecodeSamplerResource. Modes 1 and 2 now map to Vulkan's MIN and MAX
sampler reduction through VK_EXT_sampler_filter_minmax, which the device
enables when it reports both filterMinmax properties; mode 3 throws.
Min/max still throws with anisotropy, a linear mip filter (lavapipe and
NVIDIA disagree there), depth comparison, color-texture compare
emulation, and, in PointFilteredSamplerWord (S# words 0 and 2), when it
filters between texels or mips of an image the recompiler point-samples.
A texture sampled through a linear min/max sampler needs
SAMPLED_IMAGE_FILTER_MINMAX on its view format; the image-sampler pairs
travel in DescriptorBinding::imageSamplers (shader disk cache format 5).
Adds the agc_driver_image_sample_reduction GPU test and extends the
sampler, emulated color compare and disk cache tests.
A T# of type 12 (1D array) threw while decoding. It now decodes to a
1D array dimension, checked like a 2D array with a height of one, and
holds depth + 1 layers laid out as a 2D array's. Its Vulkan image is
1D with that many layers, viewed as VK_IMAGE_VIEW_TYPE_1D_ARRAY for
sampling and storage, and the recompiler declares a 1D array shape for
arrayed 1D image resources instead of failing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An S# with FORCE_UNNORMALIZED threw in DecodeSamplerResource. It is now
bound through a Vulkan sampler with unnormalizedCoordinates when every
use stays where the S# and the host select the same texels.
ShaderInfoCollector records the live uses of each sampler after Apply
and dead-code elimination, and the emitter shares its explicit-LOD
predicate. Populate flags a direct S# with the bit that only explicit-LOD
image_sample without offset, comparison, derivatives or _a uses, paired
only with direct 1D or 2D images without a format conversion; anything
else throws. The driver checks the S# filters, clamps, TRUNC_COORD and
MC_COORD_TRUNC, creates the sampler with nearest mips and a zero LOD
range, and requires a single-level, single-layer 1D or 2D view of mip 0.
The shader disk cache format becomes 5.
DecodeState, the draw precheck and DecodePixelStageInfo rejected every
nonzero CONSERVATIVE_Z_EXPORT (DB_SHADER_CONTROL bits 13-14).
EXPORT_LESS_THAN_Z (1) and EXPORT_GREATER_THAN_Z (2) promise that the
exported Z stays on one side of the interpolated Z. The value now
reaches the recompiler as a pixel stage field, and a pixel shader that
writes FragDepth declares DepthLess or DepthGreater next to
DepthReplacing, the SPIR-V modes RADV maps to these values. Vulkan lets
those modes only add a depth test of the interpolated Z before shading,
so a shader that keeps the promise renders the same. The reserved value
3 still throws.
The field is part of the recompile cache key, the draw key and the
serialized request (version 7), and ValidateShaders accepts both modes.
The resource materializer refused every atomic image descriptor whose
format was not 32_UINT. RDNA2 image atomics act on the raw dword and
the numeric format only affects addressing, so 32_SINT and 32_FLOAT
surfaces are now accepted with the Uint numeric class and run through
the existing integer atomics and float compare-exchange loops. Other
formats still throw, and the message now names the format.
DescriptorBinding gains imageAtomic, carried through the shader disk
cache (format version 4) and the resource content key. The driver
binds those elements through a cached R32_UINT view of R32_SINT and
R32_SFLOAT storage images.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A mesh program took its draw's index count from push constants, so an
indirect mesh-path (NGG) draw needed the CPU to read its record first,
waiting for the GPU to finish everything recorded before. In PPSA21564
that was about 170 waits per 10 s at the title, menu and level (about
1 s), and one per frame in the intro (2.2 s per 10 s).
The two free mesh push dwords now carry the device address of an argument
record (a VkDrawMeshTasksIndirectCommandEXT followed by the index count
and first index). When the address is set, the prologue takes the index
count and the first index from the record (the new MeshArgument IR value,
a load through a physical storage buffer pointer) and offsets its index
reads by the first index; a direct draw leaves the address 0 and draws as
before. Mesh modules declare the physical storage buffer capability for
it, and the driver's SPIR-V validation accepts the extension in mesh
modules.
An indexed mesh draw with one record that patches no user SGPR now stays
on the GPU. A one-thread compute pass (Graphics/shaders/MeshArguments.comp)
turns the record, read in place from its host import, into the draw's
argument record: the groups and instances of vkCmdDrawMeshTasksIndirectEXT
and the index count and first index the mesh program reads, with the CPU
path's rules (count clamped to the index buffer, nothing for an empty
record or past the device's group limits). The index buffer descriptor
covers the whole buffer and the program offsets by the first index. When
the records cannot be read in place, the CPU reads them and fills the
same argument record (ResolveMeshArguments).
On devices whose default subgroup is 64 lanes (RADV, AMD on Windows) a
wave32 compute program was compiled for a 64-wide host subgroup, so the two
guest waves of a 64-thread workgroup shared one subgroup: ballots, EXEC,
v_mbcnt and lane reads of the second wave saw the first one's lanes.
With VK_EXT_subgroup_size_control, when compute can require 32 lanes, the
dispatch recompiles wave32 programs for a 32-wide host subgroup
(VulkanDevice::ComputeTarget) and the pipeline requires that size. The
result records the width it was compiled for (hostSubgroupSize, also in the
disk cache).
Co-Authored-By: Claude <noreply@anthropic.com>
image_sample_*_o takes its texel offset from an address VGPR. The emitter
accepted only an immediate there and failed with "requires a constant texel
offset for image sampling" for any other, including an offset the program
moves into the VGPR under EXEC.
An offset that resolves to a constant at its packed address position stays a
ConstOffset. Any other becomes the Offset image operand, which Vulkan allows
on sampling instructions only with VK_KHR_maintenance8's maintenance8 feature
and the ImageGatherExtended capability. The device enables maintenance8 when
available and reports it through SpirvTarget::nonConstantImageOffsets, which
the cache key, the request serialization (version 6) and the SPIRV-Tools
validator options follow. Without it the sample still fails.
agc_shader_replay --maintenance8 recompiles a request for such a device.
Explicit v_interp_p1/p2 interpolation (fragment-shader barycentrics) reads every interpolated
parameter per vertex, and the rect-list shaders refused every per-vertex parameter. Their
tessellation stage emits each corner's parameter value, which is what a per-vertex read of the
generated triangles needs. Only custom parameters (custom interpolation or passthrough), whose
fourth vertex has no value of its own, stay refused; FragmentParameter carries that flag.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
SPI_PS_INPUT_ENA/ADDR with PERSP_CENTROID (bit 2) or LINEAR_CENTROID
(bit 6) was rejected; PPSA21564 binds 0x326 (perspective center and
centroid, linear center, POS_X/Y).
The PS input VGPR layout now follows the hardware rule: SPI_PS_INPUT_ADDR
places every input it names in SPI order (two VGPRs per I/J pair, three
for the pull model, one otherwise), loaded or not, and ENA & ADDR selects
the loaded ones. ShaderPixelStageInfo carries inputAddr (replacing
perspectiveCenterVgpr) plus perspectiveCentroid and linearCentroid; the
recompiler places each loaded input with PixelInputVgpr. This also fixes
the system inputs, which were always written from v0 (psSystemInputBase
was never set), so POS_X overwrote the center I in any draw with
barycentrics enabled.
Draws are rasterized single-sampled, where the centroid is the pixel
center: with fragment barycentrics the centroid pairs carry the center
barycentrics (BaryCoordKHR for perspective; the linear center and
centroid pairs now get BaryCoordNoPerspKHR).
NoPerspective was decided for every parameter by one flag (LINEAR_CENTER
loaded). v_interp_p2 now records whether it reads the J of a linear or a
perspective pair, and a parameter is NoPerspective when it is read
through a linear pair; parameters read through neither keep the old
rule. A parameter read through both kinds of pair throws. Only pixel
stages consult this.
The request serializer moves to version 5 (older requests read back with
their loaded inputs as the layout); the cache key and the driver's draw
key compare and hash the new fields. The unsupported-input error names
ENA and ADDR.
The geometry path's merged ES/GS programs (VGT_SHADER_STAGES_EN.GS_EN)
were compiled as mesh-stage programs whose input metadata was never
filled in, so every such draw failed with "mesh shader translation
configuration is not supported".
Recompiler:
- BuildShaderStageInputInfo takes the draw's MeshConfiguration; the mesh
input helpers (primitive size, step and counts) are implemented.
- The prologue emulates one NGG subgroup per workgroup: s2 GS_TG_INFO,
s3 the merged wave info per wave, the GS vertex offsets (ES thread
index times VGT_ESGS_RING_ITEMSIZE, odd strip triangles swapped), the
primitive id, and the ES vertex and instance ids. An indexed draw's
vertex id is read from the index buffer through a raw V# the driver
puts in hidden user words 4..7.
- s_sendmsg GS_ALLOC_REQ becomes MeshAllocate. The epilogue zeroes the
allocation first, clamps it to the output limits, culls primitives no
thread exported and decodes the 9-bit vertex indices.
- The draw parameters live in the last six push dwords. A BDA fault in
a mesh program no longer stops the invocation (as for other programs
with barriers, the fault is recorded and the read returns zero), so
every invocation reaches the subgroup barriers and the epilogue.
- The cache keys cover the mesh configuration; requests serialize its
new esgsItemSize field (version 4).
Driver:
- DecodeShaderStages reads VGT_ESGS_RING_ITEMSIZE (now in the draw key)
and sizes the subgroup for its ES vertices, GS primitives and outputs.
- Stages compile below the mesh draw parameters, which every mesh draw
pushes; the mesh stage always takes part in the push range.
- The index-buffer V# is set per draw (per record on the CPU-resolved
indirect path); the index-buffer alias check spares read-only
elements.
- The workgroup and LocalSize checks count two guest lanes per
invocation on 32-wide subgroups; mesh draws may carry a first vertex
or instance.
agc_driver_mesh_tests (EXCLUDE_FROM_ALL, needs VK_EXT_mesh_shader)
recompiles a hand-assembled merged ES/GS program in the PS5 compiler's
layout and draws indexed and non-indexed triangle lists in one- and
two-wave subgroups and a triangle strip across odd-starting subgroups,
reading the pixels back. agc_shader_replay prints a request's draw and
mesh configuration.
Only the lines this branch added lose their comments; the commits before
keep them, with the reasoning behind each change, for whoever digs into
that code later.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A view whose offset in its GPU owner is not a multiple of
minStorageBufferOffsetAlignment could not be bound, and an address-based
build (its registered heaps imported whole) put descriptor-bound buffers at
arbitrary offsets inside them, so such dispatches failed.
GuestBufferMemory::Descriptor now binds from the aligned offset below the
view and reports the difference; ShaderResources stores it, per guest buffer
element, into the push constant block at the shader's memory byte offsets
(RecompileResult::memoryOffsetDword), which the shader already adds to every
access. Every push constant path (dispatches, recorded and synchronous draws,
draw recipes) takes the patches; the assembling overload of
Pipeline::PushConstants is gone. A shader whose shader data is a buffer fails
when it would need one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dispatch sized in threads (USE_THREAD_DIMENSIONS) launches a partial last
workgroup on RDNA2, but the host rounds it up to whole groups, so the extra
threads ran: past the end of a node list they read past it and wrote over
live records.
Such dispatches now compile a separate variant (ShaderComputeStageInfo::
partialThreads, keyed only on whether it is set) whose entry EXEC excludes
threads at or past the requested size. The size reaches the shader as three
shader data dwords after the memory offsets (StageInputKind::
DispatchThreadLimit), is mixed into the dispatch cache key and the result
memo hash, and serialized in recompile requests (version 3). Whole-group
dispatches keep their variant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dispatch cache with per-key value variants and data-only hits (flat-SRT leaves the recompiler
proves pure are refreshed into the resource template instead of re-captured), dispatch and draw
recipes replaying a proven prepare, register-keyed draw entries, template data refresh, unit
shadows in device-local slabs, 64 KiB block-unit storage tracking with windowed detile, DCC
clear-key proofs, per-class GPU timing, a CPU copy path with batch read tracking, present slot
ring, rate-limited reaps and lazy completion counting, plan-failure and poisoned source memos,
bindless image tables (runtime-indexed descriptor arrays from a CPU-read table) so the
per-frame compute program that failed its plan now compiles. Worker affinity opt-in, per-draw
logs behind APS5_TRACE_DRAWS, wait-timeout reports include the recorder's view of the label.
Fixes a use-after-free behind BDA-miss dispatch snapshots and the CPU-write stamp of
UnchangedSinceCollected. Intro-video stage: 154 -> ~500 presents per 10 s at 4K.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177gthrWEBBfBu5xJDXgBfo
- s_*_saveexec reads its source before writing the old EXEC into the
destination, so the aliased form (vcc, vcc) computes exec & vcc; the
per-lane exec bit comes from the lane's own bit. Every Bink 2 decoder
kernel uses that form; this was the video corruption.
- buffer atomics add/sub/or/xor with a zero operand become an atomic
load or nothing.
- DescriptorBinding carries bufferWritten/bufferAtomic/imageWritten;
results expose a variantId and share SPIR-V copy-on-write; source
keys use a length plus 64-bit hash; captures can be reused.
- wave-LDS ordering barriers, v_movrels/v_movreld lowering, SMEM with a
VCC base, f16 conversion opcode fix, a per-function SPIRV-Tools pass
list, env-gated IR dumps and probes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177gthrWEBBfBu5xJDXgBfo
Adrià Franch
boykopovar
nisbenz
Celegans12
Michon
Adrià
alfalfa-star
Claude
Dean
Dean Galvin
asimplestray
SamG-Coder
STL1te
devilismyfriend