Decode the three VOP3 16-bit interpolation instructions (attribute, channel
and high-half select in the src0 field, the I/J VGPR in src1, P0 or the first
step's result in src2) and translate them with a new
GetInterpolationParameterF16 IR value. It reads the selected 16-bit half of
each vertex's parameter through VK_KHR_fragment_shader_barycentric, so P10 and
P20 are vertex differences rounded to f16; both steps are a multiply and an
add, as in the f32 forms.
An input in FP16_INTERP_MODE whose low half is defaulted is still declared at
its slot when its high half is read, and the high half takes its default from
USE_DEFAULT_ATTR1 and DEFAULT_VAL_ATTR1. Without barycentrics, on an input
without FP16_INTERP_MODE or on a pass-through input the instructions throw.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decodes IMAGE_GATHER4H_PCK (MIMG opcode 98) and IMAGE_GATHER8H_PCK
(opcode 99), implements gather4h_pck through translation, the SPIR-V
backend and a GPU execution test, and documents gather8h_pck as a
desktop null-op.
Semantics were measured on gfx1031 hardware with hand-encoded MIMG
words (LLVM rejects the PCK gather mnemonics on every RDNA1/RDNA2
target). The measured rule for gather4h_pck: the four-texel horizontal
window is floor(u*width-0.5)+{-1,0,+1,+2} clamped to the width, the row
is floor(v*height) clamped, the raw bits of the window texels are
concatenated into a little-endian bitstream (texel k at bit offset
k*texelBits), and output register j - the j-th set DMASK bit in
ascending order - receives stream word j, zero past the end of the
stream. Exactly popcount(dmask) consecutive VGPRs are written from vdst
ascending; texels wider than 32 bits write zeros (hardware limit); rows
past the height read zeros; negative coordinates clamp to zero. The
window anchor matches the repo's existing PS5-validated HorizontalOffsets
rule, so this reuses the established convention.
The decoder accepts multi-bit DMASKs for these two opcodes only and
sizes their data span as popcount(dmask) (measured), unlike gather4h
which always writes four components. Translation carries the packed
flag through imageMemoryInfoFromInstruction; the SPIR-V emitter
reassembles the bitstream per lane with per-texel OpImageFetch and
byte assembly. D16 data throws until measured.
gather8h_pck never returns texel data on this hardware: across dmasks
0x1-0xFF including the upper nibble, RGBA8/RGBA16/RGBA32 formats, 1D/2D
and 3D descriptors, normalized and unnormalized sampling, sentinel-
prefilled destination registers confirm it only ever zero-fills
popcount(dmask) registers. It decodes but throws at translation.
The measured semantics and the gather8h_pck null-op are recorded in
docs/dev/TechnicalDebt.md.
Decode MIMG opcode 0x80 on 2D MSAA and 2D MSAA array images with a
single-bit dmask and return four dwords: the selected component of the
fragment named in the address and the three that follow it. Each fragment
is read with its own image fetch, adding the fragment step to the packed
address dword so a16 layouts keep working.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
These make up the precise f64 division macro compilers emit for a / b.
Like the f32 ops, the rules were measured on an RDNA2 GPU (41152 rows,
model matches all of them):
- div_scale decides from the exponent fields only: 2^128 or 2^-128
scaling and VCC for differences >= 768 or <= -768, a denormal or huge
(>= 2045) denominator and a numerator exponent <= 53; a NaN source is
passed through unchanged and a zero input gives 0xfff8000000000000.
- div_fmas scales by 2^128 when the exponent of S2 is >= 1024, else
2^-128, before the single rounding, so it reuses the exact fma
emulation with the scale folded into its final rescale.
- div_fixup follows the ISA: NaN sources are quieted, a NaN quotient
with finite inputs becomes +-inf, exponent differences below -1075
give a signed zero.
Co-Authored-By: Claude <noreply@anthropic.com>
Decode and translate the remaining unary f64 transcendentals and
v_trig_preop_f64.
On RDNA2 hardware v_rcp_f64, v_rsq_f64 and v_sqrt_f64 are approximations
(relative error up to ~2^-24.3 over 8209 inputs; sqrt(1.0) is not even
exact), so they are emitted as exact double operations and the test
compares against the hardware value with a 2^-23 tolerance. Special
cases are exact and selected on bits: NaN inputs are quieted (RADV keeps
signaling NaN on FDiv), negative inputs give the default NaN
0xfff8000000000000, rsq(-0) = -inf, sqrt(-0) = -0.
v_trig_preop_f64 matches the hardware bit for bit on 24627 results plus
a sweep of the table tail: the 2/PI table is 1184 bits (bits below b17
read as zero), shift = 53 * S1[4:0] + max(exp - 1077, 0), +128 scale for
exp >= 1968, omod ignored. The 53-bit window is read from a private u32
table and scaled in two exact steps so only the final multiply rounds.
Stacked on the f64 infrastructure of #446.
Co-Authored-By: Claude <noreply@anthropic.com>
MIMG 0xe7 is the node test of image_bvh_intersect_ray with a 64-bit node
pointer: the pointer takes the first two address dwords (low, high), so
the instruction addresses 12 dwords, or 9 with A16.
The node index and address are computed from the whole pointer. A zero
base in the T# is a null BVH only for the 32-bit form: with 64-bit
pointers the T# may cover all of memory from base 0 and the pointer then
carries the node address, so only the T# size bounds the node.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MUBUF opcode 0x34 subtracts the data register from a buffer dword and
clamps the result to zero, returning the previous value. The form without
glc throws: no GFX10 target is known to provide it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
VOP3P 0x13 threw "VOP3P opcode is not implemented". It computes
src0.lo * src1.lo + src0.hi * src1.hi + src2 like v_dot2c_f32_f16, with
the VOP3P op_sel/op_sel_hi and neg_lo/neg_hi modifiers on the packed f16
sources, neg on the f32 accumulator and clamp. op_sel and neg_hi on the
accumulator throw.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The three MIMG float atomics threw as unknown opcodes. They now run as a
compare-and-swap loop on the 32-bit texel with the bit-level rules the
float buffer atomics already use (NaN, signed zero and sNaN quieting as
measured on RDNA2), so no float atomic feature is needed from the driver.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer_atomic_fcmpswap, fcmpswap_x2, fmin_x2, fmax_x2, inc_x2 and
dec_x2 are compare-and-swap loops on the dword or qword. fcmpswap
compares as floats (-0 equals +0, NaN equals nothing) and fmin/fmax are
IEEE minNum/maxNum with -0 < +0, a quieted result for a signalling NaN
and the other operand for a quiet one, all with integer ops so the
driver's float controls do not matter. The existing 32-bit fmin/fmax
kept the old value whenever a NaN or two zeros were involved; it now
uses the same rule.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer_atomic_inc, buffer_atomic_dec, buffer_atomic_cmpswap_x2 and
buffer_atomic_{add,sub,smin,umin,smax,umax,and,xor}_x2 threw in the decoder.
The 64-bit ones use native 64-bit SPIR-V atomics like the existing
swap_x2/or_x2; inc and dec wrap at the data operand through the existing
compare-exchange helpers.
Co-Authored-By: Claude <noreply@anthropic.com>
Write the raw texel bits: dmask bit i selects texel dword i, taken from the
next data register, and every other dword of the texel is written as 0; the
descriptor swizzle is ignored (measured on RDNA2 hardware). Only formats whose
bits go through the storage view unchanged are accepted (unsigned integer
components, raw 32-bit signed integer and 32-bit float); normalized, 16-bit
float, narrow signed and 96-bit formats throw.
Co-Authored-By: Claude <noreply@anthropic.com>
Return the raw texel bits: dmask bit i selects texel dword i, dwords past the
texel read 0 and the _sgn forms sign-extend texels narrower than 32 bits
(measured on RDNA2 hardware). Only formats whose bits are exactly recoverable
from the sampled view are accepted (unsigned/signed integer, unorm up to 16
bits, 32-bit float components), on a direct sampled image with identity
swizzle; anything else throws.
Co-Authored-By: Claude <noreply@anthropic.com>
These make up the precise f32 division macro compilers emit for a / b. No
reference emulator implements them and the ISA pseudocode disagrees with the
hardware, so the rules were measured on an RDNA2 GPU:
- div_scale decides from the exponent fields only: 2^64 or 2^-64 scaling
and VCC for differences >= 96 or <= -96, a denormal or huge (>= 253)
denominator and a numerator exponent <= 24; a zero input gives 0xffc00000.
- div_fmas scales by 2^64 when the exponent of S2 is >= 128, else 2^-64,
before the single rounding (the ISA says a fixed 2^32).
- div_fixup follows the ISA, and a NaN quotient with finite inputs becomes
+-inf.
s_denorm_mode is accepted and ignored; listed in TechnicalDebt.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer_atomic_fcmpswap, fcmpswap_x2, fmin_x2, fmax_x2, inc_x2 and
dec_x2 are compare-and-swap loops on the dword or qword. fcmpswap
compares as floats (-0 equals +0, NaN equals nothing) and fmin/fmax are
IEEE minNum/maxNum with -0 < +0, a quieted result for a signalling NaN
and the other operand for a quiet one, all with integer ops so the
driver's float controls do not matter. The existing 32-bit fmin/fmax
kept the old value whenever a NaN or two zeros were involved; it now
uses the same rule.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer_atomic_inc, buffer_atomic_dec, buffer_atomic_cmpswap_x2 and
buffer_atomic_{add,sub,smin,umin,smax,umax,and,xor}_x2 threw in the decoder.
The 64-bit ones use native 64-bit SPIR-V atomics like the existing
swap_x2/or_x2; inc and dec wrap at the data operand through the existing
compare-exchange helpers.
Co-Authored-By: Claude <noreply@anthropic.com>
Decode VOP3b 0x177 and translate it next to v_mad_u64_u32: the high half of
the product comes from SMulHi, and the carry-out is the sign of the exact
sum (bit 64 of the sign-extended product plus the sign-extended addend), as
measured on RDNA2 hardware.
Co-Authored-By: Claude <noreply@anthropic.com>
Decode and translate v_add/mul/fma/min/max/ldexp_f64, v_trunc/ceil/floor/
rndne/fract_f64, v_frexp_mant/exp_i32_f64 and the f64 <-> f32/i32/u32
conversions. f64 values stay as their raw bits (U64) in the IR and are
bitcast to double in SPIR-V, which needs the Float64 capability: the
device enables shaderFloat64 when it also preserves signed zero, inf and
NaN for 64-bit floats, and the shader declares SignedZeroInfNanPreserve 64.
Semantics measured on RDNA2 hardware (8209 inputs per op): literals are
the high dword, the 1/(2*pi) inline constant is the exact double,
omod is ignored, clamp maps NaN to +0, min/max quiet a
signaling NaN and drop a quiet one, NaN results are quiet, and the
integer conversions saturate with NaN -> 0. Truncation, rounding and
ldexp are done on the bits so they do not depend on the driver: lavapipe
returned wrong results for f64 RoundEven and Trunc, and GLSL Ldexp is
undefined past +-1024. Fma is NoContraction so RADV keeps it fused.
Co-Authored-By: Claude <noreply@anthropic.com>
s_cmovk_i32 writes the sign-extended 16-bit immediate when SCC is set.
s_version is a no-op marker. s_setreg_imm32_b32 takes its value from a
literal dword and is ignored like s_setreg_b32; both are now listed as
silent stubs in TechnicalDebt.
Co-Authored-By: Claude <noreply@anthropic.com>
Decode VOP3 0x172, 0x173 and 0x175. Each op slides a 4-byte window over the
64-bit src0 and adds the byte SAD against src1 to the matching 16-bit
(pk_u16) or 32-bit (mqsad_u32) field of src2; the mqsad forms skip zero
reference bytes. Sums wrap per field, as measured on RDNA2 hardware.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer_store_format_d16_{x,xy,xyz,xyzw}, buffer_store_format_d16_hi_x
and tbuffer_store_format_d16_{x,xy,xyz,xyzw} take one 16-bit half per
component (two per VGPR; hi_x takes the high half). As on hardware,
float, norm and scaled formats read the half as f16, widen it exactly
to f32 and then encode it like a 32-bit formatted store; uint formats
zero-extend it and sint formats sign-extend it before saturating, and
16-bit float components store it unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer_load_format_d16_{x,xy,xyz,xyzw}, buffer_load_format_d16_hi_x and
tbuffer_load_format_d16_{x,xy,xyz,xyzw} convert each component to 16
bits and pack two per VGPR. The conversions follow the hardware:
unorm/snorm round the exact value to nearest even, scaled integers
round to nearest even and saturate at 65504, 32-bit floats truncate
toward zero (saturating at 65504), and 32-bit integers saturate to 16
bits. All of it is done with integer ops so it does not depend on the
driver's f16 rounding. _x and _hi_x keep the other half, xyz keeps the
upper half of its second register, and a descriptor constant one is
f16 1.0 for non-integer formats.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer/flat/global/scratch_load_{ubyte,sbyte,short}_d16(_hi),
buffer/flat/global/scratch_store_{byte,short}_d16_hi and
ds_read_{u8,i8}_d16(_hi), ds_write_b8_d16_hi threw in the decoder. They use
the same 16-bit destination/source select as the existing ds_read_u16_d16
and ds_write_b16_d16_hi: a load writes one half and keeps the other, a
_d16_hi store takes its data from the high half.
Co-Authored-By: Claude <noreply@anthropic.com>
v_movrelsd moves VGPR[src + M0] to VGPR[dst + M0]. The _2 form and
v_swaprel take the source offset from M0[9:0] and the destination
offset from M0[25:16]; v_swaprel and v_swap exchange the two registers.
They reuse the select-over-registers lowering of v_movrels/v_movreld,
now shared as readRelativeVector/writeRelativeVector.
Co-Authored-By: Claude <noreply@anthropic.com>
krishnaraya
Daan Dehing
Adrià Franch
Petre
Mark Kuebel
Chad
Kikoarl
Dean
DotDebian
Alexey