These make up the precise f64 division macro compilers emit for a / b.
Like the f32 ops, the rules were measured on an RDNA2 GPU (41152 rows,
model matches all of them):
- div_scale decides from the exponent fields only: 2^128 or 2^-128
scaling and VCC for differences >= 768 or <= -768, a denormal or huge
(>= 2045) denominator and a numerator exponent <= 53; a NaN source is
passed through unchanged and a zero input gives 0xfff8000000000000.
- div_fmas scales by 2^128 when the exponent of S2 is >= 1024, else
2^-128, before the single rounding, so it reuses the exact fma
emulation with the scale folded into its final rescale.
- div_fixup follows the ISA: NaN sources are quieted, a NaN quotient
with finite inputs becomes +-inf, exponent differences below -1075
give a signed zero.
Co-Authored-By: Claude <noreply@anthropic.com>
Decode and translate the remaining unary f64 transcendentals and
v_trig_preop_f64.
On RDNA2 hardware v_rcp_f64, v_rsq_f64 and v_sqrt_f64 are approximations
(relative error up to ~2^-24.3 over 8209 inputs; sqrt(1.0) is not even
exact), so they are emitted as exact double operations and the test
compares against the hardware value with a 2^-23 tolerance. Special
cases are exact and selected on bits: NaN inputs are quieted (RADV keeps
signaling NaN on FDiv), negative inputs give the default NaN
0xfff8000000000000, rsq(-0) = -inf, sqrt(-0) = -0.
v_trig_preop_f64 matches the hardware bit for bit on 24627 results plus
a sweep of the table tail: the 2/PI table is 1184 bits (bits below b17
read as zero), shift = 53 * S1[4:0] + max(exp - 1077, 0), +128 scale for
exp >= 1968, omod ignored. The 53-bit window is read from a private u32
table and scaled in two exact steps so only the final multiply rounds.
Stacked on the f64 infrastructure of #446.
Co-Authored-By: Claude <noreply@anthropic.com>
MIMG 0xe7 is the node test of image_bvh_intersect_ray with a 64-bit node
pointer: the pointer takes the first two address dwords (low, high), so
the instruction addresses 12 dwords, or 9 with A16.
The node index and address are computed from the whole pointer. A zero
base in the T# is a null BVH only for the 32-bit form: with 64-bit
pointers the T# may cover all of memory from base 0 and the pointer then
carries the node address, so only the T# size bounds the node.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MUBUF opcode 0x34 subtracts the data register from a buffer dword and
clamps the result to zero, returning the previous value. The form without
glc throws: no GFX10 target is known to provide it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
VOP3P 0x13 threw "VOP3P opcode is not implemented". It computes
src0.lo * src1.lo + src0.hi * src1.hi + src2 like v_dot2c_f32_f16, with
the VOP3P op_sel/op_sel_hi and neg_lo/neg_hi modifiers on the packed f16
sources, neg on the f32 accumulator and clamp. op_sel and neg_hi on the
accumulator throw.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The three MIMG float atomics threw as unknown opcodes. They now run as a
compare-and-swap loop on the 32-bit texel with the bit-level rules the
float buffer atomics already use (NaN, signed zero and sNaN quieting as
measured on RDNA2), so no float atomic feature is needed from the driver.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer_atomic_fcmpswap, fcmpswap_x2, fmin_x2, fmax_x2, inc_x2 and
dec_x2 are compare-and-swap loops on the dword or qword. fcmpswap
compares as floats (-0 equals +0, NaN equals nothing) and fmin/fmax are
IEEE minNum/maxNum with -0 < +0, a quieted result for a signalling NaN
and the other operand for a quiet one, all with integer ops so the
driver's float controls do not matter. The existing 32-bit fmin/fmax
kept the old value whenever a NaN or two zeros were involved; it now
uses the same rule.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer_atomic_inc, buffer_atomic_dec, buffer_atomic_cmpswap_x2 and
buffer_atomic_{add,sub,smin,umin,smax,umax,and,xor}_x2 threw in the decoder.
The 64-bit ones use native 64-bit SPIR-V atomics like the existing
swap_x2/or_x2; inc and dec wrap at the data operand through the existing
compare-exchange helpers.
Co-Authored-By: Claude <noreply@anthropic.com>
Write the raw texel bits: dmask bit i selects texel dword i, taken from the
next data register, and every other dword of the texel is written as 0; the
descriptor swizzle is ignored (measured on RDNA2 hardware). Only formats whose
bits go through the storage view unchanged are accepted (unsigned integer
components, raw 32-bit signed integer and 32-bit float); normalized, 16-bit
float, narrow signed and 96-bit formats throw.
Co-Authored-By: Claude <noreply@anthropic.com>
Return the raw texel bits: dmask bit i selects texel dword i, dwords past the
texel read 0 and the _sgn forms sign-extend texels narrower than 32 bits
(measured on RDNA2 hardware). Only formats whose bits are exactly recoverable
from the sampled view are accepted (unsigned/signed integer, unorm up to 16
bits, 32-bit float components), on a direct sampled image with identity
swizzle; anything else throws.
Co-Authored-By: Claude <noreply@anthropic.com>
These make up the precise f32 division macro compilers emit for a / b. No
reference emulator implements them and the ISA pseudocode disagrees with the
hardware, so the rules were measured on an RDNA2 GPU:
- div_scale decides from the exponent fields only: 2^64 or 2^-64 scaling
and VCC for differences >= 96 or <= -96, a denormal or huge (>= 253)
denominator and a numerator exponent <= 24; a zero input gives 0xffc00000.
- div_fmas scales by 2^64 when the exponent of S2 is >= 128, else 2^-64,
before the single rounding (the ISA says a fixed 2^32).
- div_fixup follows the ISA, and a NaN quotient with finite inputs becomes
+-inf.
s_denorm_mode is accepted and ignored; listed in TechnicalDebt.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer_atomic_fcmpswap, fcmpswap_x2, fmin_x2, fmax_x2, inc_x2 and
dec_x2 are compare-and-swap loops on the dword or qword. fcmpswap
compares as floats (-0 equals +0, NaN equals nothing) and fmin/fmax are
IEEE minNum/maxNum with -0 < +0, a quieted result for a signalling NaN
and the other operand for a quiet one, all with integer ops so the
driver's float controls do not matter. The existing 32-bit fmin/fmax
kept the old value whenever a NaN or two zeros were involved; it now
uses the same rule.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer_atomic_inc, buffer_atomic_dec, buffer_atomic_cmpswap_x2 and
buffer_atomic_{add,sub,smin,umin,smax,umax,and,xor}_x2 threw in the decoder.
The 64-bit ones use native 64-bit SPIR-V atomics like the existing
swap_x2/or_x2; inc and dec wrap at the data operand through the existing
compare-exchange helpers.
Co-Authored-By: Claude <noreply@anthropic.com>
Decode VOP3b 0x177 and translate it next to v_mad_u64_u32: the high half of
the product comes from SMulHi, and the carry-out is the sign of the exact
sum (bit 64 of the sign-extended product plus the sign-extended addend), as
measured on RDNA2 hardware.
Co-Authored-By: Claude <noreply@anthropic.com>
Decode and translate v_add/mul/fma/min/max/ldexp_f64, v_trunc/ceil/floor/
rndne/fract_f64, v_frexp_mant/exp_i32_f64 and the f64 <-> f32/i32/u32
conversions. f64 values stay as their raw bits (U64) in the IR and are
bitcast to double in SPIR-V, which needs the Float64 capability: the
device enables shaderFloat64 when it also preserves signed zero, inf and
NaN for 64-bit floats, and the shader declares SignedZeroInfNanPreserve 64.
Semantics measured on RDNA2 hardware (8209 inputs per op): literals are
the high dword, the 1/(2*pi) inline constant is the exact double,
omod is ignored, clamp maps NaN to +0, min/max quiet a
signaling NaN and drop a quiet one, NaN results are quiet, and the
integer conversions saturate with NaN -> 0. Truncation, rounding and
ldexp are done on the bits so they do not depend on the driver: lavapipe
returned wrong results for f64 RoundEven and Trunc, and GLSL Ldexp is
undefined past +-1024. Fma is NoContraction so RADV keeps it fused.
Co-Authored-By: Claude <noreply@anthropic.com>
s_cmovk_i32 writes the sign-extended 16-bit immediate when SCC is set.
s_version is a no-op marker. s_setreg_imm32_b32 takes its value from a
literal dword and is ignored like s_setreg_b32; both are now listed as
silent stubs in TechnicalDebt.
Co-Authored-By: Claude <noreply@anthropic.com>
Decode VOP3 0x172, 0x173 and 0x175. Each op slides a 4-byte window over the
64-bit src0 and adds the byte SAD against src1 to the matching 16-bit
(pk_u16) or 32-bit (mqsad_u32) field of src2; the mqsad forms skip zero
reference bytes. Sums wrap per field, as measured on RDNA2 hardware.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer_store_format_d16_{x,xy,xyz,xyzw}, buffer_store_format_d16_hi_x
and tbuffer_store_format_d16_{x,xy,xyz,xyzw} take one 16-bit half per
component (two per VGPR; hi_x takes the high half). As on hardware,
float, norm and scaled formats read the half as f16, widen it exactly
to f32 and then encode it like a 32-bit formatted store; uint formats
zero-extend it and sint formats sign-extend it before saturating, and
16-bit float components store it unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer_load_format_d16_{x,xy,xyz,xyzw}, buffer_load_format_d16_hi_x and
tbuffer_load_format_d16_{x,xy,xyz,xyzw} convert each component to 16
bits and pack two per VGPR. The conversions follow the hardware:
unorm/snorm round the exact value to nearest even, scaled integers
round to nearest even and saturate at 65504, 32-bit floats truncate
toward zero (saturating at 65504), and 32-bit integers saturate to 16
bits. All of it is done with integer ops so it does not depend on the
driver's f16 rounding. _x and _hi_x keep the other half, xyz keeps the
upper half of its second register, and a descriptor constant one is
f16 1.0 for non-integer formats.
Co-Authored-By: Claude <noreply@anthropic.com>
buffer/flat/global/scratch_load_{ubyte,sbyte,short}_d16(_hi),
buffer/flat/global/scratch_store_{byte,short}_d16_hi and
ds_read_{u8,i8}_d16(_hi), ds_write_b8_d16_hi threw in the decoder. They use
the same 16-bit destination/source select as the existing ds_read_u16_d16
and ds_write_b16_d16_hi: a load writes one half and keeps the other, a
_d16_hi store takes its data from the high half.
Co-Authored-By: Claude <noreply@anthropic.com>
v_movrelsd moves VGPR[src + M0] to VGPR[dst + M0]. The _2 form and
v_swaprel take the source offset from M0[9:0] and the destination
offset from M0[25:16]; v_swaprel and v_swap exchange the two registers.
They reuse the select-over-registers lowering of v_movrels/v_movreld,
now shared as readRelativeVector/writeRelativeVector.
Co-Authored-By: Claude <noreply@anthropic.com>
Decode VOP3 opcode 0x35f and translate it on the raw half bits, matching
the RDNA2 hardware: NaN numerator or denominator is returned quieted,
0/0 and inf/inf give 0xfe00, a NaN quotient becomes a signed infinity,
and op_sel, abs and neg apply as for the other f16 ternary ops.
Co-Authored-By: Claude <noreply@anthropic.com>
Decode and translate v_dot2_i32_i16, v_dot2_u32_u16, v_dot4_i32_i8,
v_dot4_u32_u8, v_dot8_i32_i4, v_dot8_u32_u4 (VOP3P 0x14-0x19) and
v_dot4c_i32_i8 (VOP2 0x0d).
Clamp, neg and the op_sel combinations whose behaviour is not
established throw.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decode VOP3 0x14d and translate it as the per-byte average of S0 and S1,
rounded up where bit 0 of the matching S2 byte is set and truncated
otherwise.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dean
Adrià Franch
DotDebian
Chad
Alexey