Shaders with f64 instructions now declare DenormPreserve for 64-bit floats
when the device reports shaderDenormPreserveFloat64. Without it Mesa ANV
flushed the subnormal result of v_fma_f64 to zero.
(cherry picked from commit 7df6c709de994cea0019707d45b6e7b96698fad1)
A pixel shader whose stage info asks for alpha-to-coverage declares SampleMask, starts with every sample covered and, at its MRT0 export, clears the mask when the alpha is below one half or NaN, running with late fragment tests. The shader keeps running, as a zero coverage mask does on the GPU. The flag is part of the static shader identity and of the request serialization, now version 16.
Intersect copied fixed-point sets without allocating each intermediate.
Preserve sorted results and the existing analysis iteration order.
Compact oversized storage only when publishing a changed set.
Keep an allocating control switch outside the cache key.
Compare complete fixed points over empty intersections, loops and DAGs.
Register the same equivalence fixtures for the allocating control.
CB_BLENDn_CONTROL factors 15-18 (BLEND_SRC1_COLOR, BLEND_INV_SRC1_COLOR,
BLEND_SRC1_ALPHA, BLEND_INV_SRC1_ALPHA) threw "unsupported blend
factor". They blend MRT0 with the pixel shader's MRT1 export as the
second source, as RADV and radeonsi program it.
DecodeState maps them to the VK_BLEND_FACTOR_SRC1_* factors and accepts
them only with pixel exports to slots 0 and 1, color writes to slot 0
alone, an MRT1 export with MRT0's export format and shader mask, an
ADD, SUBTRACT or REVERSE_SUBTRACT operation and an identity component
order. The blend is draw state, so the specialized module of the
registered pixel shader moves its MRT1 output to location 0 index 1,
which ValidateShaders requires; pipelines need dualSrcBlend, now enabled
when present. The request field dualSourceBlend is in the module key,
the result memo, the draw key and the serialized request (version 15).
Both dominance analyses start from sorted unique sets and intersect them.
Insert the current block ID with lower_bound instead of sorting again.
Preserve every intermediate set and the existing ID-order iteration.
Keep the generic unsorted helper and all graph rewrite logic unchanged.
Compare both analyses against the original algorithms on six raw CFGs.
Cover finite loops, closed SCCs, unreachable islands and permuted IDs.
A pixel shader whose early discards branch to one small ending block
gave that block several conditional predecessors outside any loop, and
structurizing failed. When the regular pass throws, the structurizer
restarts from the original graph and first gives each conditional
predecessor its own copy of such a return block (within the clone word
floor and a budget of four clones per block).
Uncharted: Legacy of Thieves Collection (PPSA05684) stopped at such a pixel shader with "conditional block 15 has no structured merge".
CB_COLOR_INFO format 6 (COLOR_10_11_11) with NUMBER_UNORM threw in
DecodeColorFormat: Vulkan has only the B10G11R11 float format. Such a
target now renders into an R32_UINT attachment in the layout of
format-30 images (unorm11 R and G in bits 0-21, unorm10 B in bits
22-31), so later image loads of the surface read the same words.
The target format is draw state, so the packing is a specialization of
the registered pixel shader: every float MRT export also computes the
packed word (EmitFormatStoreComponent) behind specialization constant
ExportPackingBase + target, and the specialized module turns a packed
target's output into a uint4 that ValidateShaders expects. The new
request field targetExportPacking is in the module key, the result memo,
the draw key and the serialized request (version 14). Blending, partial
RGB writes, UNORM16/SNORM16/UINT16 exports, DCC and ROUND_MODE throw.
computePostDominators iterates its sets to a fixed point, visiting the
blocks in id order. Post-dominance flows from a block's successors back to
it, and successors mostly have higher ids, so each pass moved the sets back
by about one block: a long program needed about as many passes as it has
blocks, each costing O(blocks^2). recomputeAnalyses runs it after every
split of splitSharedMergeBlocks, up to four times per block.
Visit the blocks in post-order of the CFG (successors first), then any the
entry does not reach. The sets start from every block and only shrink, so
the iteration still converges to the same greatest fixed point, now in a
few passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reject tuple and descriptor overlap, wrong-pair returns, relative SGPR reads, reversed regions and ordinary or conditional fall-through into a callee. Preserve the existing scalar write analysis and require exact return-pair matching.
Seven targeted regressions fail before these guards and pass afterward. Seven focused CMake tests and 15 SPIR-V validations pass; the execution matrix passes 3,200 comparisons and detects a one-bit expected-output mutation.
Reuse the prerequisite call-region analysis for signed relative calls, including callees after s_endpgm. Lower the saved link as shader base plus PC+4 and reject unsupported targets and call shapes.
Add decoder, IR, CFG and Vulkan execution regressions. Seven focused CMake tests pass; five layouts cover 15 dispatches and 3,200 output comparisons on lavapipe. Record ISA sources and native RDNA2/PS5 limits in TechnicalDebt.
Decodes SOP1 0x21 s_swappc_b64 (link pair = pc + 4) and resolves its call/return pairing through the existing s_setpc_b64 path: static call regions in GraphBuilder and the vertex-entry fetch-shader call inlined from the parsed fetch-shader plan. s_call_b64 (SOPK 0x16) and s_rfe_b64 (SOP1 0x22) throw at decode. Unmodeled cases throw; see docs/dev/TechnicalDebt.md.
On GFX10.3 to GFX11.7, ACO lowers a subgroup-scope OpReadClockKHR to
s_getreg_b32 of SHADER_CYCLES, 20 bits zero-extended to 64, so s_memtime
wrapped every 2^20 cycles and went backwards. Detect those families from
the RADV device name and read the device clock there instead.
Fixes#985
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DecodeState and the draw precheck rejected DB_SHADER_CONTROL bit 16,
PRIMITIVE_ORDERED_PIXEL_SHADER (POPS), which runs the ordered section
of overlapping pixels' shaders in primitive order. The bit now reaches
the recompiler as a pixel stage field, and such a pixel shader declares
PixelInterlockOrderedEXT and runs its whole body between
OpBeginInvocationInterlockEXT and OpEndInvocationInterlockEXT; an
invocation that a BDA read fault kills ends the interlock first. The
POPS collision SGPRs read 0, so a title skips its POPS wait and runs
its per-collision loop once. An image_load still reads a sampled
binding, which misses an earlier overlapping primitive's image_store in
the same draw. The device enables VK_EXT_fragment_shader_interlock when
it has fragmentShaderPixelInterlock; without it the recompile throws,
as does a nonzero POPS_OVERLAP_NUM_SAMPLES. The field is part of the
recompile cache key, the draw key and the serialized request (version 10).
toHexString, contains, intersectSorted, addUnique, sortUnique, remapId,
remapIds and rebuildPredecessors were defined with identical bodies in
the anonymous namespaces of two of ControlFlowGraph.cpp, GraphBuilder.cpp
and Structurizer.cpp. They are now declared in ControlFlowHelpers.hpp,
which only these three files include, and defined in ControlFlowGraph.cpp.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A compute program with COMPUTE_PGM_RSRC2.SCRATCH_EN takes its per-lane private size from the AGC
shader header (scratch_size_dw_per_thread) instead of being refused. The size travels in
ShaderComputeStageInfo (cache key, request serialization version 10) to the translator, whose
scratch accesses already lower to a per-invocation Function array of that many dwords.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
NVIDIA 610.62 cannot sample R8G8_SRGB, and lavapipe can sample neither
it nor R8_SRGB, so an 8_8 sRGB T# made an image the device does not
support: creating it crashes the NVIDIA driver and lavapipe loads zeros.
The device's format support is read once. An 8 or 8_8 sRGB texture the
device cannot sample is created and viewed as R8_UNORM or R8G8_UNORM,
and SpirvTarget::srgbDecodeFormats makes the materializer mark the
image, so image loads map every channel through a 256-entry table of
the sRGB EOTF rounded to nearest even at bf16 precision, which is what
RDNA2 (gfx1035) returns; both channels of 8_8 are sRGB, and DST_SEL 0
and 1 map to themselves. Samples and gathers of such an image throw,
and its bindless table entries are unmapped like converted formats.
APS5_SRGB_SHADER_DECODE=1 takes this path on every device. Shader disk
cache entries and keys carry the mark: cache format version 7.
A selection inside a loop whose arm either joins the other target or
continues/breaks (`if (a) { if (b) continue; x; }`) got the nearest
common post-dominator as its merge. When the loop exits after the outer
join, that is the outer selection's merge, so both selections shared
it. splitSharedMergeBlocks then inserted a forwarding block in front of
it, recomputed the same merge, and repeated until the budget ran out:
PPSA14632's compute shader 0x1a80ee0000 stopped with "CFG shared merge
splitting exceeded budget: originalBlocks=92 currentBlocks=460".
findSelectionMerge now picks the joined target when the other arm is a
conditional block with the header as its only predecessor, and every
successor of that arm is the joined target or the loop's continue or
merge block. The shader structures with 13 added blocks and no clones
or route variables.
In the result the outer selection header dominates the loop's continue
target, so the test's verifyStructured counted the continue target as a
selection member that branches outside (back to the loop header).
SPIR-V uses structural dominance, where the loop header's edge to its
continue target keeps it out of the selection, so verifyStructured now
skips the continue construct of an enclosing loop. spirv-val accepts
the new test's SPIR-V for both subgroup sizes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A loop whose exits have no common post-dominator, such as a normal exit
to one s_endpgm tail beside an early-out to a kill tail, got no merge
block, and Structurize threw "loop at block N has no structured
merge/continue". With three or more exits, detectNaturalLoops threw
first, on a lookup of InvalidControlFlowId.
Such a loop now leaves through one new merge block that dispatches on
route variables. They are cleared before the header and set on each
exit edge to a target other than the latch's exit. Inner loops are
routed first; a routing that would change the natural loops throws.
A loop-exit tail branch whose in-body target is the continue block no
longer merges there, which spirv-val rejects ("Header block ... is
contained in the loop construct ..., but its merge block ... is not").
DecodeState, the draw precheck and DecodePixelStageInfo rejected every
nonzero CONSERVATIVE_Z_EXPORT (DB_SHADER_CONTROL bits 13-14).
EXPORT_LESS_THAN_Z (1) and EXPORT_GREATER_THAN_Z (2) promise that the
exported Z stays on one side of the interpolated Z. The value now
reaches the recompiler as a pixel stage field, and a pixel shader that
writes FragDepth declares DepthLess or DepthGreater next to
DepthReplacing, the SPIR-V modes RADV maps to these values. Vulkan lets
those modes only add a depth test of the interpolated Z before shading,
so a shader that keeps the promise renders the same. The reserved value
3 still throws.
The field is part of the recompile cache key, the draw key and the
serialized request (version 7), and ValidateShaders accepts both modes.
SOP1 opcode 33 with SDST null keeps no return address, so it is exactly
s_setpc_b64; the decoder now reads it that way, as KytyPS5 does. A call
with a real destination still throws "unsupported SOP1 opcode 33".
GraphBuilder resolves two more jump shapes. A long branch is
s_getpc_b64 and s_add_u32/s_addc_u32 or s_sub_u32/s_subb_u32
immediates, also through vcc, all in one block. A dword jump table
branches to the table base minus an entry that s_load_dword reads at an
index bounded by s_min_u32 and scaled by 4; the resolver follows
register writes back through straight-line code, and anything else is
still refused. The code table load now also handles one-dword entries.
agc_shader_control_flow checks the decode, the graphs and the refusals;
the new agc_driver_scalar_swappc runs all shapes on the GPU.
MIMG 0xe7 is the node test of image_bvh_intersect_ray with a 64-bit node
pointer: the pointer takes the first two address dwords (low, high), so
the instruction addresses 12 dwords, or 9 with A16.
The node index and address are computed from the whole pointer. A zero
base in the T# is a null BVH only for the 32-bit form: with 64-bit
pointers the T# may cover all of memory from base 0 and the pointer then
carries the node address, so only the T# size bounds the node.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ds_permute_b32 is the forward (scatter) counterpart of ds_bpermute_b32:
every active lane sends its data to lane ((addr + offset) >> 2) & 31 of
its wave. As measured on hardware, the highest sending lane wins, an
active lane nobody writes to gets 0 and inactive lanes neither send nor
receive. It lowers to one shuffle of the data, address and EXEC bit per
source lane, using the same wave base as ds_bpermute_b32.
Co-Authored-By: Claude <noreply@anthropic.com>
A selection whose one arm only returns (kill or s_endpgm) and whose other
arm branches straight to the merge of an enclosing selection had no
structured merge, so the draw was skipped. PPSA21564's BotEye G-buffer
pixel shaders (eight variants) have this shape.
When no other split applies, the edge from such a header to the shared
join is now split into an empty block, which becomes the header's merge.
image_sample_*_o takes its texel offset from an address VGPR. The emitter
accepted only an immediate there and failed with "requires a constant texel
offset for image sampling" for any other, including an offset the program
moves into the VGPR under EXEC.
An offset that resolves to a constant at its packed address position stays a
ConstOffset. Any other becomes the Offset image operand, which Vulkan allows
on sampling instructions only with VK_KHR_maintenance8's maintenance8 feature
and the ImageGatherExtended capability. The device enables maintenance8 when
available and reports it through SpirvTarget::nonConstantImageOffsets, which
the cache key, the request serialization (version 6) and the SPIRV-Tools
validator options follow. Without it the sample still fails.
agc_shader_replay --maintenance8 recompiles a request for such a device.
A conditional inside a loop with one target in the loop body and the
other outside it was taken for a break and emitted without a merge. When
the outside target is not the loop's merge but a tail of blocks that
reaches it, SPIR-V has no construct for that branch: spirv-val rejects
the module ("Selection must be structured") and the driver is free to
compile it into anything.
Such a branch now merges at its in-body target when the tail is private
to it (every tail block is dominated by the tail's first block, stays in
the loop construct and leaves only for the loop's merge), so the tail is
an "if (c) { tail; break; }" the loop already allows. Merges are given
to these after every other construct's, and one already taken leaves the
branch as before.
All 13 PPSA21564 programs whose V#s are read from GPU memory (the
deferred lighting and ray-traced passes) had two such branches in their
traversal loops. With this change none of the 74 dumped title programs
has one, and the 209 modules of a title-to-cinematic run validate apart
from three unrelated image-offset and entry-point errors.
Tests: agc_structurizer_tests structures a loop that exits through one
tail block and a loop with two exit tails, and requires every merge-less
conditional to target its loop's merge or continue.
krishnaraya
Adrià
ThatsLeo
Gustavo Ariel Gamarra Rojas
Chad
Celegans12
hugodearaujo
XairRrR
Dean
EdouardSence
raghav314
Adrià Franch
Sanjaysrinivas
ebosido444-ai
nisbenz
LeandroLP23
Michon
alfalfa-star
ahirm-finarth
DotDebian
Alexey
Dean Galvin
SamG-Coder