Background registration preparations (#2583) copy the device's
SpirvTarget, whose capability and extension spans point into the
device's shader profile. Driver::stop reset the device without waiting
for the detached pool, so a preparation still queued or running read
freed extension names and agc_driver crashed at exit on RADV.
Driver::stop now closes the pool after joining the queue workers: it
stops queueing and waits until no task is queued or running. A
preparation submitted after that runs on the caller's thread, which
holds its own reference to the device, like the existing fallback for
a pool without threads.
Fixes#2891
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The driver's Log2 has an absolute error that does not shrink with the
result, so near 1.0 the relative error explodes: on an RTX 3080, v_log_f32
of 1 - 2^-24 is 23,947,836 ULP off. Measured on RDNA2 (gfx1036), the
hardware's log2 is within 1 ULP of the correctly rounded value, and an
eight-term series in x - 1 is within 2 ULP of both for |x - 1| <= 1/8.
The lowering keeps the driver's Log2 and takes the series only inside that
band when the two differ by more than 4 ULP, so RADV's native result stays
bit-exact and NVIDIA's is within 6 ULP inside the band.
agc_driver_log_near_one pins 64 inputs per type from the oracle with
per-lane tolerances.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012GLfKH7Sqe51wX254k1XsV
Enable the existing input-only f32 flush policy for division scaling.
RDNA2 measurements pin all three sources and signed zeros; keep the
scaling rules unchanged and retain their separate technical debt.
Co-Authored-By: Codex gpt-6.1-sol <noreply@openai.com>
Keep each registered header and its prepared shader state instead of
replacing every registration for the same code address. Resolve explicit
ABIs by header and programs by type, preserving re-registration order.
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
On RADV the race between ds_add_u64 under the workgroup lock and a
concurrent ds_add_u32 only shows when the lanes get their operands from
per-lane buffer loads: with v_mov_b32 constants or loads from one address
no increment is lost there. The new case has the split-lane kernel's
parity split and loads each lane's (0xffffffff, 0, 1, 0) row from the input
buffer at tid * 16 with buffer_load_dwordx4, so the input buffer now holds
256 rows. With SpirvMemoryInstructions.cpp reverted it fails on RADV and
on the RTX 3080; with the change it passes on both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fnrk5Dazjb2tLooVSpdRQq
Even lanes run ds_add_u64 and odd lanes run ds_add_u32 on the slot's high
dword, one pass per dispatch, as in the reported kernel, and the dispatch
is repeated 32 times from one compiled shader.
With ANYPS5_ENABLE_SPIRV_TOOLS=ON the scalar loop around the ds_add_u64 lock loop made
SPIRV-Tools' block merge pass merge the loop header with its multi-block continue
target, and validation after optimization rejected the module. The 16 iterations are
now emitted inline; the expected sums and the divergent exec masks are unchanged.
The hardware serialises 32-bit and 64-bit LDS atomics on one address; the lock that makes the 64-bit forms atomic did not order 32-bit atomics, which lost increments between the two halves of a 64-bit update (#2297).
Comments are reserved for technical debt (docs/dev/TechnicalDebt.md), so
the ones the previous commit added to the recorder, the snapshot keep and
the tests go.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HZHkb8kUwnHAuBBbHywvJf
A ShaderResources from the resource caches is the same object for every
draw or dispatch that uses it, and Recorder::ReusableDrawSnapshot hands
the next draw the same snapshot buffer, so passing their bytes with each
Keep counted one allocation once per use and submitted the batch before
it held the budget. Recorder::KeepBytes counts an owner's bytes once per
batch, and the dispatch, draw and snapshot keeps use it.
The recorded kept bytes test runs a dispatch and a draw that copy a
guest buffer and two draws that snapshot one on the device, and fails
when any of the three sites stops counting or a reused snapshot counts
twice. The BdaResources check pins the copied region it sets up.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HZHkb8kUwnHAuBBbHywvJf
The registry calls an observer on the mapping thread, after the registry
lock is released, with the ranges a kernel mapping gave GPU access:
direct and flexible maps with it, and fixed remaps with it over a
registered range (a reservation). The mutation looks them up while it
still holds the lock and hands them over weakly, with the registry
generation it ended at, so the observer does not keep a range alive.
The driver imports those ranges on its current device, so an
address-based build finds the import instead of pinning the memory on
the queue that first uses it. A range that gains GPU access through
sceKernelMprotect, and AnyPS5's own libc heap, are still imported on
first use.
The import at map does not reconcile the other imports with the
registry, so its cost does not grow with the number of imports. It skips
a range that is gone, and any range when the imports were reconciled
with a newer registry since; the next lookup imports those as before.
It installs the pin waiter as the other import paths do, so unmapping a
range imported at map retires its import. The live import bytes are kept as a
running total instead of being summed over every import. Once the import
table belongs to the current device the import takes only the table's
lock; the first import on a device (which runs the write-watch probe)
and a device that was just replaced go through the GPU lock. Clearing
the observer waits for calls already running.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured on RDNA2 (gfx1035) in review:
- the shared and private apertures are 4 GB windows (the gfx9+ layout of kfd_flat_memory.c), so a flat
address routes by its whole high dword, 0x80000000 or 0x70000000, not by its top 16 bits;
- a dword or short access whose last byte is past the end of LDS reads 0 or writes nothing as a whole;
multi-dword accesses are checked per dword.
The aperture test gets rows that straddle and pass the end of LDS; the TechnicalDebt line names the
public source of the layout and keeps only the PS5's base values as inferred.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Main's StoreAddressU32x2..x4 (one BDA lookup per wide store, b95842597) bypassed the aperture routing:
a flat_store_dwordx2 into the shared aperture went to BDA at 0x8000000000000000 and LDS kept its old
value. The wide store now stores each component through the aperture like the 32-bit store, and keeps
the single BDA lookup outside the apertures.
The aperture test's V# word 3 is 0x31016fac (OOB_SELECT 3, raw), as in main's other execution tests:
with OOB_SELECT 0 its output buffer read as out of range after #639, which was the failure reported on
four drivers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sub-dword stores and unaligned dword accesses go byte-wise; 32- and 64-bit FLAT atomics
route to LDS (64-bit under the workgroup lock), scratch (read-modify-write) or BDA.
agc_driver_flat_apertures covers them in 121 rows, with mixed-address waves and wave64.
The aperture bases and the unmodelled operands 235-238 are listed as inferred.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A compute FLAT access whose address has the console's shared aperture in its high dword (0x8000xxxx)
reads or writes the workgroup's LDS at its low dword, and one in the private aperture (0x7000xxxx)
the lane's scratch; other addresses stay guest memory through BDA. GTA V's ray traversal keeps its
stack in LDS and spills it to scratch through such addresses; treated as guest memory the pushes were
lost and the traversal never ended (device lost). An execution test round-trips both apertures.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Scalar f16 add/mul/sub ignore FLOAT_MODE denorm16 flush bits on main.
Flush inputs when bit 0 is clear and results when bit 1 is clear at
nearest-even. Multiply checks tininess after 11-bit rounding
(0x387ff000); add/sub use 0x38800000.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Submit the same vertex and pixel draw twice through the driver and
require the second one to hit the draw cache, with graphics pipeline
libraries and with APS5_NO_GPL=1. With pipeline libraries the pixel
stage is cached at its fixed push slot, so a lookup at the stacked
offset misses every draw on its layout. The driver exposes a copy of
its draw-cache counters for the test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A comparison sample with a 2D-array address on a plain 2D color T# takes the emulated comparison, whose runtime modes were only the 2D-array and cube ones. No mode matched, and materialization threw that the image descriptor was incompatible with the static runtime image interface. The emulated comparison now also gets a 2D mode, as the native modes of a 2D-array access already have; the slice is ignored.
The ternary forms share minMaxF16 with v_min_f16 and v_max_f16 and follow
the same rule on gfx1036: a subnormal half input becomes a signed zero when
bit 0 of the f16 denormal field is clear, at IEEE=0 and IEEE=1.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Registration prepares the variants a shader's registered state
predicts; a draw or dispatch whose state needs another one threw
"prepared shader artifact is missing for the requested static ABI".
Titles that link their own libSceAgc build that state themselves and
never call the driver's static ABI resolution, so their draws differ
from the registered prediction. The missing artifact is now prepared at use, logged, and kept
with the snapshot like the deferred and null pixel cases. SourceHandleFor
now checks that the request refers to the registered code, as
InvocationFor already did.
Uncharted: Legacy of Thieves Collection (PPSA05684) still misses on
main 197ffac9 (after the fixed push slots of #2661) for its own pixel
shaders: "prepared shader artifact is missing for the requested static
ABI: address=85899411712 stage=5 wave=64 pushOffset=128
pushCapacity=128".
v_div_fmas_f32 always quieted source NaNs and let a NaN addend
hide the default NaN from an invalid product. Reuse the existing
bit selection helpers, measured on gfx1036 in four float modes.
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
RDNA2 min/max flush half inputs to signed zero when input
preservation is disabled, while output flushing alone keeps them.
Apply this to the measured binary forms and pin the hardware rows.
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Measure the mixed f16/f32 boundary on gfx1036 and pin both factor
slots, half selection, signs and zero f16 addends in the existing test.
Record this measured scope while keeping legacy mad/mac unchanged.
Co-Authored-By: Codex gpt-6.1-sol <noreply@openai.com>
With f32 denormals flushed, RDNA2 flushes a product only when its value
rounded to 24 bits in the current round mode is below 2^-126. The
tiny-product check tested the exact product, so products that round up
to 2^-126 became zero in v_mul_f32, v_mul_legacy_f32, v_mullit_f32 and
zero-addend f32 FMAs. The separately rounded v_mad_legacy_f32 and
v_mac_legacy_f32 keep the exact check, as they are not measured there.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
v_cvt_f16_u16 SDWA only accepted word and dword source selects. Silent
Hill Townfall (PPSA33286) converts the top byte into the low half of a register
and keeps the high half (src0_sel:BYTE_3 dst_sel:WORD_0
dst_unused:UNUSED_PRESERVE), so its shaders failed to recompile. The
generic SDWA source path already zero-extends byte selects, which is the
u16 value the conversion expects, so the rule now accepts every source
select, with whole and partial destinations.
agc_driver_sdwa_cvt_f16_u16 runs four byte selects with dword, WORD_0
and WORD_1 destinations on the GPU and checks the converted halves and
the preserved bits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit d55dfe8b33c6ee418f40a92e49236f8679a6470b)
Adrià Franch
Adrià
Daan Dehing
Jan Šlégl
Alex
Hussein Kadhim
MathiasDevelopes
raghav314
Michon
Chad
JAB-dev
Dean Galvin
XairRrR
Lexroid