Route oversized compute LDS through a per-workgroup storage buffer.
Retain native shared memory for programs that fit the device limit.
Carry the spill stride through prepared shader results and dispatches.
Preserve the stride in the full-result codec and its round-trip fixture.
Keep the Linux RecompileResult size guard aligned with the added field.
Exercise interleaved workgroups and the device-limit boundary.
This topic contains only the LDS change and its reviewer-requested repair.
The base includes upstream's separately merged NaN result correction.
CB_BLENDn_CONTROL factors 15-18 (BLEND_SRC1_COLOR, BLEND_INV_SRC1_COLOR,
BLEND_SRC1_ALPHA, BLEND_INV_SRC1_ALPHA) threw "unsupported blend
factor". They blend MRT0 with the pixel shader's MRT1 export as the
second source, as RADV and radeonsi program it.
DecodeState maps them to the VK_BLEND_FACTOR_SRC1_* factors and accepts
them only with pixel exports to slots 0 and 1, color writes to slot 0
alone, an MRT1 export with MRT0's export format and shader mask, an
ADD, SUBTRACT or REVERSE_SUBTRACT operation and an identity component
order. The blend is draw state, so the specialized module of the
registered pixel shader moves its MRT1 output to location 0 index 1,
which ValidateShaders requires; pipelines need dualSrcBlend, now enabled
when present. The request field dualSourceBlend is in the module key,
the result memo, the draw key and the serialized request (version 15).
Without VK_KHR_fragment_shader_barycentric the recompiler dropped
v_interp_p1_f32 and made v_interp_p2_f32 read the attribute Vulkan
interpolated, but never wrote the I/J VGPRs: a pixel shader that read
them for anything else got 0, and v_interp_mov_f32 of P10/P20 (or P0 of
an interpolated input) needed PerVertexKHR and failed pipeline
validation.
The fallback translation now loads the I/J VGPRs and marks v_interp_p1
and v_interp_p2 with witnesses. After the front-end simplification
(SSA, folding, DCE, read-lane and masked-select elimination),
HostInterpolationChecker requires every p2 to read its J, and its p1's
I, from a barycentric of the parameter's kind, every p1 result to reach
only the matching p2 (through a write masked by the p2's EXEC, whose
inactive lanes keep the old register), no parameter read as raw
vertices and no slot shared by different interpolations. It then
lowers the witnesses to the host-interpolated attribute as before,
reruns DCE and masked-select elimination, and accepts the program only
when no barycentric read remains, so reusing the I/J VGPRs after
interpolating keeps host interpolation and the previous SPIR-V. Any other
program is translated with the barycentric path and marked as emulated:
its barycentrics become vec3 inputs (NoPerspective for the linear
weights) and its per-vertex parameters flat vec4[3] inputs, placed in
the free locations by LayoutBarycentricEmulation. The driver then
inserts a generated geometry stage between the vertex and fragment
shaders that copies the position and parameters, writes the one-hot
weights of each vertex and the three vertices' parameter values, so the
rasterizer interpolates the same weights the extension provides.
The stage needs geometryShader and a vertex shader before it;
tessellation, mesh and rect-list pipelines, other primitive types and
interfaces beyond 32 locations or the device's geometry limits throw.
CompiledShaderArtifact carries the emulation (shader disk cache format 21).
Hardware snaps window corners to 1/256 pixel before interpolating
(measured by yahya308 on an RTX 4070 Laptop and a Radeon 780M in #1547);
lavapipe snaps only for coverage (lp_rast.h FIXED_ORDER 8), so the test
accepts either reference per pixel at the original 1e-4 margin.
Sources:
- v_interp_p1_f32/v_interp_p2_f32/v_interp_mov_f32 semantics (P0,
P10, P20, I/J): AMD RDNA2 ISA, "Parameter Interpolation".
- BaryCoordKHR/BaryCoordNoPerspKHR weights and PerVertexKHR vertex order:
SPV_KHR_fragment_shader_barycentric, VK_KHR_fragment_shader_barycentric.
- Geometry shader triangle input order and provoking vertex: Vulkan
specification, "Primitive Topologies" and "Geometry Shading".
- Interpolation decorations decided by the fragment input: Vulkan
specification, "Interpolation Decorations".
- lavapipe interpolation: Mesa src/gallium/drivers/llvmpipe/lp_rast.h
(FIXED_ORDER 8), lp_setup_tri.c (subpixel_snap for coverage only),
lp_state_setup.c (init_args, plane setup from float positions).
Retain an immutable decoded program and structured control flow while
preparing the registration variants for one shader.
Match owned code, stage and the complete SWAPPC inputs before reuse.
Translate and plan each variant separately using its current inputs.
Keep cache keys and persisted artifact layouts unchanged.
Provide a cache-neutral off switch for controlled comparisons.
Cover identity, lifetime, failed replacements and variant equivalence.
CB_COLOR_INFO format 6 (COLOR_10_11_11) with NUMBER_UNORM threw in
DecodeColorFormat: Vulkan has only the B10G11R11 float format. Such a
target now renders into an R32_UINT attachment in the layout of
format-30 images (unorm11 R and G in bits 0-21, unorm10 B in bits
22-31), so later image loads of the surface read the same words.
The target format is draw state, so the packing is a specialization of
the registered pixel shader: every float MRT export also computes the
packed word (EmitFormatStoreComponent) behind specialization constant
ExportPackingBase + target, and the specialized module turns a packed
target's output into a uint4 that ValidateShaders expects. The new
request field targetExportPacking is in the module key, the result memo,
the draw key and the serialized request (version 14). Blending, partial
RGB writes, UNORM16/SNORM16/UINT16 exports, DCC and ROUND_MODE throw.
With f32 denormals flushed, every f32 result is flushed when it is
written and every f32 source when it is read, so a source computed by an
earlier flushing op is flushed twice. A flush returns any value that is
not a denormal unchanged, and the driver can't tell that the first flush
already removed them.
DenormalFlushEliminator replaces a flush whose source can't be a
denormal with the source: a flush, a normal or zero constant, a sign
change, absolute value, clamp or bit cast of such a value, an integer or
f16 converted to f32, and a select or phi of such values.
Capturing a shader threw "null or misaligned address" when the flat
SRT walk read through a pointer loaded from memory that is null or
unmapped, even when the program skips that s_load. Such a read now
yields zero and a record with its pc and address, as does a slot whose
address needs it. The registered artifact checks one flag word per
guarded slot at each ReadConst; the capture appends the flags and the
records to the flattened SRT, and a set flag raises an Unmapped BDA
fault through the fault buffer and stops the invocation as a BDA read
does. Nothing per capture enters a key: poisoned and clean captures
share one variant and specialization. Descriptors, branch conditions,
uniform fills, table bases and user-data pointers that need such a
value still throw, and poisoned results stay out of the dispatch and
draw caches. The shader disk cache format becomes 21.
Decodes SOP1 0x21 s_swappc_b64 (link pair = pc + 4) and resolves its call/return pairing through the existing s_setpc_b64 path: static call regions in GraphBuilder and the vertex-entry fetch-shader call inlined from the parsed fetch-shader plan. s_call_b64 (SOPK 0x16) and s_rfe_b64 (SOP1 0x22) throw at decode. Unmodeled cases throw; see docs/dev/TechnicalDebt.md.
On GFX10.3 to GFX11.7, ACO lowers a subgroup-scope OpReadClockKHR to
s_getreg_b32 of SHADER_CYCLES, 20 bits zero-extended to 64, so s_memtime
wrapped every 2^20 cycles and went backwards. Detect those families from
the RADV device name and read the device clock there instead.
Fixes#985
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A compute program with COMPUTE_PGM_RSRC2.SCRATCH_EN takes its per-lane private size from the AGC
shader header (scratch_size_dw_per_thread) instead of being refused. The size travels in
ShaderComputeStageInfo (cache key, request serialization version 10) to the translator, whose
scratch accesses already lower to a per-invocation Function array of that many dwords.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
NVIDIA 610.62 cannot sample R8G8_SRGB, and lavapipe can sample neither
it nor R8_SRGB, so an 8_8 sRGB T# made an image the device does not
support: creating it crashes the NVIDIA driver and lavapipe loads zeros.
The device's format support is read once. An 8 or 8_8 sRGB texture the
device cannot sample is created and viewed as R8_UNORM or R8G8_UNORM,
and SpirvTarget::srgbDecodeFormats makes the materializer mark the
image, so image loads map every channel through a 256-entry table of
the sRGB EOTF rounded to nearest even at bf16 precision, which is what
RDNA2 (gfx1035) returns; both channels of 8_8 are sRGB, and DST_SEL 0
and 1 map to themselves. Samples and gathers of such an image throw,
and its bindless table entries are unmapped like converted formats.
APS5_SRGB_SHADER_DECODE=1 takes this path on every device. Shader disk
cache entries and keys carry the mark: cache format version 7.
The source entry remembers a resource plan that failed to build, but
not a failed emission: findOrCompileVariant kept only successful
variants, and the first attempt used up the prepared program, so every
request for a program whose emission throws ran PrepareResourceProgram
and the emitter again before the draw was skipped.
compileVariant now reports an exception from SpirvEmitter::Emit (and
from SPIRV-Tools validation when enabled), and the source entry keeps
it per code address, binding layout and resource specialization; the
next request for that variant rethrows it before the disk-cache lookup.
Descriptor population depends on the snapshot and an out-of-memory
error is transient, so neither is kept. Nothing reaches the disk cache,
and APS5_NO_FAILURE_MEMO=1 turns the memo off as for plan failures.
Every VGPR write under EXEC is translated to Select(exec, new, old), so
in SSA each register's previous value stays live until the next write,
even when the guest only used the register as a temporary. In NGG mesh
shaders at 255 registers that keeps dead values alive and the driver
spills them to local memory.
MaskedSelectEliminator (after SSA, folding and the readlane pass)
replaces r = Select(m, new, old) by new when no lane where m is false
can observe r:
- every value derived from r is either a lane-local pure op or a phi,
and their uses are checked the same way;
- or the derived value reaches the true operand of a select, or an
operand of a LogicalAnd, whose condition implies m: m itself, a
LogicalAnd containing m, or a thread bit rebuilt from a ballot of
such a condition (also through ANDs of the mask words);
- anything else (cross-lane ops, ballots, readfirstlane, DPP, swizzle,
permutes, stores, exports, atomics, branch references, implicit-LOD
pixel samples, the false operand of any select) keeps the select.
The select and every value it reaches must be in blocks outside loops,
so each of them runs at most once per invocation and every use sees the
same instance of m. Selects whose condition is the full wave
((0xffffffff >> (lane & 31)) & 1) are replaced in any block.
Exact: in lanes where m is true r equals new, so every derived value is
unchanged there; in lanes where m is false only derived values change,
and each path from them ends in a select or AND that discards those
lanes. The selects are removed latest first and dead code is removed
between rounds, so chains of writes to one register collapse.
On devices whose default subgroup is 64 lanes (RADV, AMD on Windows) a
wave32 compute program was compiled for a 64-wide host subgroup, so the two
guest waves of a 64-thread workgroup shared one subgroup: ballots, EXEC,
v_mbcnt and lane reads of the second wave saw the first one's lanes.
With VK_EXT_subgroup_size_control, when compute can require 32 lanes, the
dispatch recompiles wave32 programs for a 32-wide host subgroup
(VulkanDevice::ComputeTarget) and the pipeline requires that size. The
result records the width it was compiled for (hostSubgroupSize, also in the
disk cache).
Co-Authored-By: Claude <noreply@anthropic.com>
image_sample_*_o takes its texel offset from an address VGPR. The emitter
accepted only an immediate there and failed with "requires a constant texel
offset for image sampling" for any other, including an offset the program
moves into the VGPR under EXEC.
An offset that resolves to a constant at its packed address position stays a
ConstOffset. Any other becomes the Offset image operand, which Vulkan allows
on sampling instructions only with VK_KHR_maintenance8's maintenance8 feature
and the ImageGatherExtended capability. The device enables maintenance8 when
available and reports it through SpirvTarget::nonConstantImageOffsets, which
the cache key, the request serialization (version 6) and the SPIRV-Tools
validator options follow. Without it the sample still fails.
agc_shader_replay --maintenance8 recompiles a request for such a device.
The geometry path's merged ES/GS programs (VGT_SHADER_STAGES_EN.GS_EN)
were compiled as mesh-stage programs whose input metadata was never
filled in, so every such draw failed with "mesh shader translation
configuration is not supported".
Recompiler:
- BuildShaderStageInputInfo takes the draw's MeshConfiguration; the mesh
input helpers (primitive size, step and counts) are implemented.
- The prologue emulates one NGG subgroup per workgroup: s2 GS_TG_INFO,
s3 the merged wave info per wave, the GS vertex offsets (ES thread
index times VGT_ESGS_RING_ITEMSIZE, odd strip triangles swapped), the
primitive id, and the ES vertex and instance ids. An indexed draw's
vertex id is read from the index buffer through a raw V# the driver
puts in hidden user words 4..7.
- s_sendmsg GS_ALLOC_REQ becomes MeshAllocate. The epilogue zeroes the
allocation first, clamps it to the output limits, culls primitives no
thread exported and decodes the 9-bit vertex indices.
- The draw parameters live in the last six push dwords. A BDA fault in
a mesh program no longer stops the invocation (as for other programs
with barriers, the fault is recorded and the read returns zero), so
every invocation reaches the subgroup barriers and the epilogue.
- The cache keys cover the mesh configuration; requests serialize its
new esgsItemSize field (version 4).
Driver:
- DecodeShaderStages reads VGT_ESGS_RING_ITEMSIZE (now in the draw key)
and sizes the subgroup for its ES vertices, GS primitives and outputs.
- Stages compile below the mesh draw parameters, which every mesh draw
pushes; the mesh stage always takes part in the push range.
- The index-buffer V# is set per draw (per record on the CPU-resolved
indirect path); the index-buffer alias check spares read-only
elements.
- The workgroup and LocalSize checks count two guest lanes per
invocation on 32-wide subgroups; mesh draws may carry a first vertex
or instance.
agc_driver_mesh_tests (EXCLUDE_FROM_ALL, needs VK_EXT_mesh_shader)
recompiles a hand-assembled merged ES/GS program in the PS5 compiler's
layout and draws indexed and non-indexed triangle lists in one- and
two-wave subgroups and a triangle strip across odd-starting subgroups,
reading the pixels back. agc_shader_replay prints a request's draw and
mesh configuration.
Every run recompiled every shader: the recompiler's variant cache and
the device's VkPipelineCache only lived in memory, and the first frames
of a title spent hundreds of milliseconds per large program in the
front end, the emitter and the optimizer.
Compiled variants (the SPIR-V plus the compiled shader info and binding
allocation that per-snapshot binding population needs) are now stored
on disk, one file per key, written atomically by a background thread,
and a later run loads them instead of compiling. The key holds every
input the in-memory variant cache matches on: the in-memory recompile
key (stage, code hash, stage registers, user data count, target), every
code word, the host subgroup size, the binding layout, the resource
specialization, and the values of the APS5_* switches the recompiler
sources read (found at build time; switches that only report are left
out). Entries live under a directory per source version, a build-time
hash of the recompiler and AGC driver sources and the build
configuration, and carry their key and checksums: an entry whose
format, version, key, size or checksum does not match, or whose payload
does not decode exactly, is ignored and recompiled. Directories no
build used for two weeks and stale temporary files are removed. Requests
under the debug probe bypass the disk cache.
The device's VkPipelineCache is loaded at creation when the file's
checksum and Vulkan header match the device (vendor, device and
pipelineCacheUUID; the driver version is in the file name), saved every
10 s while it grew, and saved at teardown.
Location: a shader_cache folder beside the running executable (not the
working directory), so the relinked game stays a self-contained folder;
ANYPS5_SHADER_CACHE_DIR overrides it. ANYPS5_NO_SHADER_CACHE=1 turns both caches off; the AGC
tests run with it set so they never write to the user's cache.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ButHENCbXXt8h4EZuWUabJ
Only the lines this branch added lose their comments; the commits before
keep them, with the reasoning behind each change, for whoever digs into
that code later.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The loop guard's report names the shader (its code address when its plan
was built, now the translator's shader hash), and LoopGuardTripped() tells
whether any dispatch reported one.
- APS5_WATCH_MEMORY=<hex address>:<hex bytes> (with APS5_SYNC_DISPATCH=1)
keeps the last 64 contents of a range with the dispatch that left each, and
writes them to watch_<n>_<program>.bin when the loop guard first trips; the
address matches any heap base above bit 36.
- agc_shader_replay --mem prints the guest buffer V#s and which the shader
writes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A view whose offset in its GPU owner is not a multiple of
minStorageBufferOffsetAlignment could not be bound, and an address-based
build (its registered heaps imported whole) put descriptor-bound buffers at
arbitrary offsets inside them, so such dispatches failed.
GuestBufferMemory::Descriptor now binds from the aligned offset below the
view and reports the difference; ShaderResources stores it, per guest buffer
element, into the push constant block at the shader's memory byte offsets
(RecompileResult::memoryOffsetDword), which the shader already adds to every
access. Every push constant path (dispatches, recorded and synchronous draws,
draw recipes) takes the patches; the assembling overload of
Pipeline::PushConstants is gone. A shader whose shader data is a buffer fails
when it would need one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dispatch sized in threads (USE_THREAD_DIMENSIONS) launches a partial last
workgroup on RDNA2, but the host rounds it up to whole groups, so the extra
threads ran: past the end of a node list they read past it and wrote over
live records.
Such dispatches now compile a separate variant (ShaderComputeStageInfo::
partialThreads, keyed only on whether it is set) whose entry EXEC excludes
threads at or past the requested size. The size reaches the shader as three
shader data dwords after the memory offsets (StageInputKind::
DispatchThreadLimit), is mixed into the dispatch cache key and the result
memo hash, and serialized in recompile requests (version 3). Whole-group
dispatches keep their variant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adrià
Celegans12
russhil
Karan Vora
Adrià Franch
ebosido444-ai
boykopovar
nisbenz
Michon
Dean Galvin
Dean
STL1te
SamG-Coder