v_pk_add_f16 and v_pk_mul_f16 computed each half in f32 and rounded it
back to f16, then fixed the NaN, FP16_OVFL and clamp results on the
bits. On a device with native f16 that rounds to nearest even and keeps
f16 denormals (the Float16, RoundingModeRTE and DenormPreserve
capabilities that #1252 already checks), the arithmetic is now one
f16vec2 OpFAdd or OpFMul decorated NoContraction, and the same NaN,
FP16_OVFL and clamp rules run on its result bits.
The results don't change. f32 has at least 2 * 11 + 2 significand bits,
so rounding an f16 sum or product to f32 and then to f16 gives the
correctly rounded f16 result. Vulkan requires f16 OpFAdd and OpFMul to
be correctly rounded, the module keeps f16 denormals, and any NaN they
produce is replaced by the NaN the existing rules select.
v_pk_fma_f16 and v_pk_fmac_f16 keep the f32 path: Vulkan gives GLSL Fma
only the precision of OpFMul followed by OpFAdd, so a native f16 Fma
may round twice.
New test agc_driver_packed_float16_native compiles one shader with
packed add, mul, fma and fmac (op_sel, op_sel_hi, neg_lo, neg_hi and
clamp variants) with and without Float16, checks that only add and mul
use f16vec2 math, and compares every result bit for bit on random and
special inputs (zeros, denormals, infinities, NaNs, values near the f16
maximum) in nine float modes.
winpthreads initializes a statically initialized rwlock on its first
lock and fails a concurrent first lock with EINVAL, which
std::shared_mutex ignores, so two threads can hold the lock at once.
The source, binding plan and specialized module caches take their
first locks from several threads, so their mutexes now take and
release the lock in their constructor, as guest rwlocks do.
The GuestWriteWatch mutex the technical debt entry also named is in
the Linux-only watch, so it is not affected.
DecodeShaderStages accepts LS, HS and an NGG passthrough ES, reads the
3-bit VGT_TF_PARAM fields, refuses its other bits but DISTRIBUTION_MODE,
and needs HOS levels that keep [1, 63]; 0x286-0x287 join the draw key.
Local and domain programs are prepared with their stage group: its
headers are merged over one set of defaults, and a local program left
unbound by the hull header is prepared at its first draw.
The local program ends at its jump into the hull half.
TessellationLowering lowers LDS and ring addresses that are affine in
the control point and relative index to tessellation interfaces with
strides from the code, and refuses accesses outside the patch, per-patch
writes from other control points and a modified primitive export. Hull
lanes read their own saved exec bit, also of host-subgroup-masked
copies; other masks are refused, gl_InvocationID loads as a scalar and
factors clamp to 63.
Shaders with f64 instructions now declare DenormPreserve for 64-bit floats
when the device reports shaderDenormPreserveFloat64. Without it Mesa ANV
flushed the subnormal result of v_fma_f64 to zero.
(cherry picked from commit 7df6c709de994cea0019707d45b6e7b96698fad1)
Route oversized compute LDS through a per-workgroup storage buffer.
Retain native shared memory for programs that fit the device limit.
Carry the spill stride through prepared shader results and dispatches.
Preserve the stride in the full-result codec and its round-trip fixture.
Keep the Linux RecompileResult size guard aligned with the added field.
Exercise interleaved workgroups and the device-limit boundary.
This topic contains only the LDS change and its reviewer-requested repair.
The base includes upstream's separately merged NaN result correction.
CB_BLENDn_CONTROL factors 15-18 (BLEND_SRC1_COLOR, BLEND_INV_SRC1_COLOR,
BLEND_SRC1_ALPHA, BLEND_INV_SRC1_ALPHA) threw "unsupported blend
factor". They blend MRT0 with the pixel shader's MRT1 export as the
second source, as RADV and radeonsi program it.
DecodeState maps them to the VK_BLEND_FACTOR_SRC1_* factors and accepts
them only with pixel exports to slots 0 and 1, color writes to slot 0
alone, an MRT1 export with MRT0's export format and shader mask, an
ADD, SUBTRACT or REVERSE_SUBTRACT operation and an identity component
order. The blend is draw state, so the specialized module of the
registered pixel shader moves its MRT1 output to location 0 index 1,
which ValidateShaders requires; pipelines need dualSrcBlend, now enabled
when present. The request field dualSourceBlend is in the module key,
the result memo, the draw key and the serialized request (version 15).
Without VK_KHR_fragment_shader_barycentric the recompiler dropped
v_interp_p1_f32 and made v_interp_p2_f32 read the attribute Vulkan
interpolated, but never wrote the I/J VGPRs: a pixel shader that read
them for anything else got 0, and v_interp_mov_f32 of P10/P20 (or P0 of
an interpolated input) needed PerVertexKHR and failed pipeline
validation.
The fallback translation now loads the I/J VGPRs and marks v_interp_p1
and v_interp_p2 with witnesses. After the front-end simplification
(SSA, folding, DCE, read-lane and masked-select elimination),
HostInterpolationChecker requires every p2 to read its J, and its p1's
I, from a barycentric of the parameter's kind, every p1 result to reach
only the matching p2 (through a write masked by the p2's EXEC, whose
inactive lanes keep the old register), no parameter read as raw
vertices and no slot shared by different interpolations. It then
lowers the witnesses to the host-interpolated attribute as before,
reruns DCE and masked-select elimination, and accepts the program only
when no barycentric read remains, so reusing the I/J VGPRs after
interpolating keeps host interpolation and the previous SPIR-V. Any other
program is translated with the barycentric path and marked as emulated:
its barycentrics become vec3 inputs (NoPerspective for the linear
weights) and its per-vertex parameters flat vec4[3] inputs, placed in
the free locations by LayoutBarycentricEmulation. The driver then
inserts a generated geometry stage between the vertex and fragment
shaders that copies the position and parameters, writes the one-hot
weights of each vertex and the three vertices' parameter values, so the
rasterizer interpolates the same weights the extension provides.
The stage needs geometryShader and a vertex shader before it;
tessellation, mesh and rect-list pipelines, other primitive types and
interfaces beyond 32 locations or the device's geometry limits throw.
CompiledShaderArtifact carries the emulation (shader disk cache format 21).
Hardware snaps window corners to 1/256 pixel before interpolating
(measured by yahya308 on an RTX 4070 Laptop and a Radeon 780M in #1547);
lavapipe snaps only for coverage (lp_rast.h FIXED_ORDER 8), so the test
accepts either reference per pixel at the original 1e-4 margin.
Sources:
- v_interp_p1_f32/v_interp_p2_f32/v_interp_mov_f32 semantics (P0,
P10, P20, I/J): AMD RDNA2 ISA, "Parameter Interpolation".
- BaryCoordKHR/BaryCoordNoPerspKHR weights and PerVertexKHR vertex order:
SPV_KHR_fragment_shader_barycentric, VK_KHR_fragment_shader_barycentric.
- Geometry shader triangle input order and provoking vertex: Vulkan
specification, "Primitive Topologies" and "Geometry Shading".
- Interpolation decorations decided by the fragment input: Vulkan
specification, "Interpolation Decorations".
- lavapipe interpolation: Mesa src/gallium/drivers/llvmpipe/lp_rast.h
(FIXED_ORDER 8), lp_setup_tri.c (subpixel_snap for coverage only),
lp_state_setup.c (init_args, plane setup from float positions).
Retain an immutable decoded program and structured control flow while
preparing the registration variants for one shader.
Match owned code, stage and the complete SWAPPC inputs before reuse.
Translate and plan each variant separately using its current inputs.
Keep cache keys and persisted artifact layouts unchanged.
Provide a cache-neutral off switch for controlled comparisons.
Cover identity, lifetime, failed replacements and variant equivalence.
CB_COLOR_INFO format 6 (COLOR_10_11_11) with NUMBER_UNORM threw in
DecodeColorFormat: Vulkan has only the B10G11R11 float format. Such a
target now renders into an R32_UINT attachment in the layout of
format-30 images (unorm11 R and G in bits 0-21, unorm10 B in bits
22-31), so later image loads of the surface read the same words.
The target format is draw state, so the packing is a specialization of
the registered pixel shader: every float MRT export also computes the
packed word (EmitFormatStoreComponent) behind specialization constant
ExportPackingBase + target, and the specialized module turns a packed
target's output into a uint4 that ValidateShaders expects. The new
request field targetExportPacking is in the module key, the result memo,
the draw key and the serialized request (version 14). Blending, partial
RGB writes, UNORM16/SNORM16/UINT16 exports, DCC and ROUND_MODE throw.
With f32 denormals flushed, every f32 result is flushed when it is
written and every f32 source when it is read, so a source computed by an
earlier flushing op is flushed twice. A flush returns any value that is
not a denormal unchanged, and the driver can't tell that the first flush
already removed them.
DenormalFlushEliminator replaces a flush whose source can't be a
denormal with the source: a flush, a normal or zero constant, a sign
change, absolute value, clamp or bit cast of such a value, an integer or
f16 converted to f32, and a select or phi of such values.
Capturing a shader threw "null or misaligned address" when the flat
SRT walk read through a pointer loaded from memory that is null or
unmapped, even when the program skips that s_load. Such a read now
yields zero and a record with its pc and address, as does a slot whose
address needs it. The registered artifact checks one flag word per
guarded slot at each ReadConst; the capture appends the flags and the
records to the flattened SRT, and a set flag raises an Unmapped BDA
fault through the fault buffer and stops the invocation as a BDA read
does. Nothing per capture enters a key: poisoned and clean captures
share one variant and specialization. Descriptors, branch conditions,
uniform fills, table bases and user-data pointers that need such a
value still throw, and poisoned results stay out of the dispatch and
draw caches. The shader disk cache format becomes 21.
Decodes SOP1 0x21 s_swappc_b64 (link pair = pc + 4) and resolves its call/return pairing through the existing s_setpc_b64 path: static call regions in GraphBuilder and the vertex-entry fetch-shader call inlined from the parsed fetch-shader plan. s_call_b64 (SOPK 0x16) and s_rfe_b64 (SOP1 0x22) throw at decode. Unmodeled cases throw; see docs/dev/TechnicalDebt.md.
On GFX10.3 to GFX11.7, ACO lowers a subgroup-scope OpReadClockKHR to
s_getreg_b32 of SHADER_CYCLES, 20 bits zero-extended to 64, so s_memtime
wrapped every 2^20 cycles and went backwards. Detect those families from
the RADV device name and read the device clock there instead.
Fixes#985
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A compute program with COMPUTE_PGM_RSRC2.SCRATCH_EN takes its per-lane private size from the AGC
shader header (scratch_size_dw_per_thread) instead of being refused. The size travels in
ShaderComputeStageInfo (cache key, request serialization version 10) to the translator, whose
scratch accesses already lower to a per-invocation Function array of that many dwords.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adrià Franch
Dean
Karan Vora
Ahir
Celegans12
ThatsLeo
Chad
Adrià
russhil
ebosido444-ai
boykopovar
nisbenz
Michon