APS5_NO_PERF_DISK_PLANS=1 is listed as a neutral switch, so it stays out
of the variant key, and it exists to compare warm starts with and
without stored plans. Nothing tested either property: the cross-process
test ran with the switch unset only.
A fourth child process now sets the switch against the cache the first
process filled. It must run the front end for all 14 fixtures, neither
load nor store a plan, still hit all 14 stored variants, and produce the
same SPIR-V, bindings, push constants, snapshots, read traces and plans
as the first process.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The plan tests compared a decoded plan with the original only through
the codec itself (re-encoding must give the same bytes) and through
materialization. Neither catches a field that EncodePlan and DecodePlan
both leave out: the bytes still agree, and the walk does not read every
field. Leaving userDataBase, the plan's sRGB formats, the value ids or
several MemoryInfo fields (secondaryOffset, dataDwords, dataFormat,
planningOnly, idxen) out of both sides passed every test. The hand-built
plan also never materializes (its uniform values cannot be evaluated),
so its 1000 runtimes all failed with the same error, and no plan held an
immediate wider than 32 bits.
samePlan compares every field of two plans without the codec: the value
graph by storage index (opcode, type, id, flags, register, immediate,
arguments, phi blocks), the block ids, the roots, and the plain data
through the structures' own equality. The round trip now requires it
for the 14 fixtures and the hand-built plan, which sets every MemoryInfo
field, all indirect table fields, a second buffer, the whole uniform
fill and a 64-bit immediate to values other than their defaults. The
second process also builds each fixture's plan with its own front end
and requires it to equal the plan it used, so a stored plan must match
what the front end builds in another process, not only its own
re-encoding.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DecodePlan checked every descriptor source index a plan holds except the
two inside an indirect image table: the heap source and the material
source. A damaged entry could therefore decode with a table that points
past the plan's descriptor sources, and only the materializer's own
bounds checks stood between it and an out-of-range read. The front end
interns both sources before the image source that names them, so a plan
it builds always passes the new check.
agc_shader_disk_cache now requires a plan whose table heap or material
source lies past the sources not to decode.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
getSource built ResourceProgram (PrepareResourceProgram and ExtractPlan)
for the first request of every shader a process saw, before
findOrCompileVariant looked in the disk cache. With a warm cache every
launch still decoded, built the CFG, structurized, translated and ran
SSA, the fold passes, SrtWalker and ResourceTracker for every shader;
only SPIR-V emission was skipped.
getSource now builds the plan key and tries LoadPlan first. On a hit it
sets the plan and leaves the program null: findOrCompileVariant already
rebuilds the program when no stored variant fits the layout. The loaded
plan takes the request's code address as its shader hash, as a plan
built in this process would. On a miss it builds the plan as before and
stores it. A plan the front end refuses is not stored. The disk is not
used when the cache is off, under the debug probe, with APS5_DUMP_IR
(which prints the front-end passes), or with APS5_NO_PERF_DISK_PLANS=1,
a cache-neutral switch for comparisons.
FrontEndBuilds() counts PrepareResourceProgram calls. The new
cross-process test prepares, captures and materializes 14 shader
fixtures in one process and repeats it in a second process sharing the
cache directory: the second process must run the front end 0 times, hit
14 plans and 14 variants, and produce byte-identical SPIR-V, bindings,
push constants, snapshots, read traces and plans. A damaged plan entry
is rejected and rebuilt alone with the same output and stored again; a
changed code word and another switch value miss.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The disk cache keeps compiled variants (SPIR-V, shader info, binding
layout) but not the IrResourcePlan the front end extracts for the
resource walk, so a process can only get a plan by running the whole
front end. Since #471 the plan owns its values (ownPlanValues clones the
graph behind every root into valueStorage and blockStorage), so it can
be written out and rebuilt.
- EncodePlan writes the value graph in storage order, each value as
LEB128 integers: opcode, type, id, flags, register, operand count,
the immediate if it has one, each argument as the zigzag distance to
the value's own index (0 for null) and, for phis, the index of each
predecessor block. The roots (descriptor source dwords, SRT reads,
control-flow conditions, uniform-fill values) are value indices; the
plain data (memory info, indirect image tables, control-flow edges,
materialization sources, guarded, clean and pure flat slots, the
flags, the ShaderInfo through the variant's encoder) follows. A plan
whose roots or arguments point outside its storage is not encoded.
Control-flow conditions can pull most of a program into the plan:
a synthetic 2,038-dword compute shader with a branch every 20 dwords
has 42,775 plan values, which take 518 KB this way and 2.5 MB with
fixed-width fields; its compiled variant entry is 935 KB.
- DecodePlan rebuilds the graph the way ownPlanValues does and rejects
an unknown opcode, an overlong integer, an index past the values or
blocks, a descriptor source, image, buffer or sampler past the
sources, a branch without two successors, an edge past the blocks,
and a uniform fill of more than four words.
- BuildPlanKey is the in-memory source key (RecompileCacheKey::Build and
the host subgroup size), every code word and the switch key, after a
plan entry tag. It leaves out the binding layout, so all layouts of a
shader share one plan.
- EncodePlanEntry/DecodePlanEntry use the variant's header and hashes;
LoadPlan and StorePlan use the same directory and background writer
and count plan hits, misses, writes and load failures on their own.
- Size asserts cover the plan and the structures it is written from.
- FormatVersion 30.
agc_shader_disk_cache now round-trips the plans of 14 in-tree shader
fixtures (SRT reads, guarded and chained pointers, control-flow
conditions, an indirect image table, a loop phi, a fragment shader) and
a hand-built plan with a uniform fill. For each it materializes the
original and the decoded plan over 1000 random user-data and memory
runtimes and requires the same snapshot, poison, null-root count and
read trace, or the same error. Truncated, bit-flipped and
other-format entries are rejected, malformed plans do not decode, and
the key follows the code, the target and the host subgroup size but not
the layout, addresses or user data.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since #1463, every f16 operand read decodes sign, exponent and mantissa
with integer operations, 8 to 12 instructions more per read than
UnpackHalf2x16 on NVIDIA. The decode works around an AMD Windows driver
that folds the mask of a byte-selected half into UnpackHalf2x16.
A new target flag, float16DenormPreserve, is set when the device reports
shaderDenormPreserveFloat16 and denormBehaviorIndependence ALL, the same
rule as float64DenormPreserve. With it, EmitF16BitsToF32 uses
UnpackHalf2x16 and declares DenormPreserve for 16-bit floats, under
which SPV_KHR_float_controls keeps unpacked subnormal halves. NVIDIA
returns the default NaN for every NaN half, so the NaN bits are rebuilt
from the half: its sign at bit 31, its payload at bits 22:13. The
DenormPreserve 16 mode is declared once, shared with the native f32 to
f16 conversion. Without the flag (lavapipe; RADV reports 32_BIT_ONLY)
the integer decode stays. Halves of fields narrower than 16 bits, such
as SDWA byte selects, keep the integer decode on every device.
The flag is serialized (request version 17) and part of the cache key.
New test agc_driver_f16_to_f32_unpack converts all 65,536 halves, with
nonzero high bits, through v_cvt_f32_f16, and byte-selected halves
through v_add_f16 SDWA, with each decode the device supports, and
compares the bits with an exact decode. agc_driver_f16_to_f32_unpack_spirv
checks which decode the SPIR-V uses with and without the flag.
A packed ancillary word that still reaches an export, or any other read
the masked-select pass cannot drop, no longer stops the recompile. After
that pass every remaining use gets the composed word: the primitive class
at bits 0-1, SampleId at bits 8-11 and Layer at bits 16-28, with every
other bit zero. Those lanes do export the ancillary VGPR on the console,
so the word has to be right, not just present.
The primitive class (0 point, 1 line, 2 triangle, 3 rectangle) comes from
the draw. RasterPrimitiveOf maps rect lists to 3, line lists on the vertex
path to 1 and everything else the driver accepts to 2, because its
geometry and tessellation paths only output triangles. It reaches the
shader as pipeline specialization constant 8233, the way dual-source
blending does: a request field that is folded into the specialized
module and is part of the result memo, the draw key and the serialized
request (version 17).
Only shaders that read the class pay for it. ShaderInfo records the read
(disk cache format 28), and only those shaders put the class in their
module key, so a shader that reads just the sample or layer field keeps
one module for every primitive class. The draw sets the class only when
the pixel shader loads the ancillary VGPR, so other draws keep the draw
cache entries they have today.
The VRS rates (bits 2-5) and the eye id (bit 7) are 0 because the driver
supports neither VRS nor multiview. SampleId is 0 because the driver
rejects MSAA and per-sample shading, and Layer is 0 while it ignores
vertex layer exports. TechnicalDebt.md records the layout sources and
these assumptions.
Tests:
- pixel_ancillary builds an export to target 0 whose two channels select
between a packed value and a phi carrying the word. It checks the word
for each primitive class, sample and layer, checks that the three
shapes the masked-select pass keeps now compile with the composed word,
and that a program whose select the pass drops reads no class.
- agc_driver_pixel_ancillary_word exports the word from a draw and reads
back 2, 3 and 1 for triangle, rectangle and line. A shader that reads
only the sample field shares one module across the classes, and the
whole-word shader gets one per class. It also checks RasterPrimitiveOf.
- agc_shader_memory covers the version 17 round trip, and
agc_shader_disk_cache the new flag.
Both ancillary tests fail with the old refusal when the composition is
disabled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Publish Windows cache entries with the POSIX rename operation so an
open delete-sharing reader keeps its original file after replacement.
Close the rename handle before returning and retain the existing
fallback only when the extended operation is unsupported.
Exercise old and new reader contents, deletion lifetime and temporary
file cleanup in the registered disk-cache regression on both platforms.
Multi-level views, and views starting above mip 0, are bound as their base
level only when every use of the image is image_sample_lz through an
unnormalized S# without minimum LOD or LOD bias, the case measured exact on
RDNA2. Selected-LOD, implicit-LOD and offset uses keep throwing.
The recompiler records selected-LOD uses (SamplerUseNonZeroLod) and proves
the case per image (imageUnnormalizedBaseLevel); the shader disk cache
format goes to 27.
Co-Authored-By: Claude <noreply@anthropic.com>
Explicit-LOD offset samples through FORCE_UNNORMALIZED samplers now add
sign-extended texel offsets to f32 coordinates before clamping, using
OpFAdd with NoContraction and a sampler specialization selector.
Without native nonconstant-offset support, retain only runtime image
modes valid for unnormalized sampling and reject normalized samplers.
Preserve constant-swizzle handling, indirect-image rejection and main's
flat-volume and flat-line compatibility rules when selecting those modes.
Serialize image compatibility and folding flags and the sampler folding
flag in cache format 34, with roundtrip, preparation and execution coverage.
Retain main's request format 15 and runtime ABI 11.
Shaders with f64 instructions now declare DenormPreserve for 64-bit floats
when the device reports shaderDenormPreserveFloat64. Without it Mesa ANV
flushed the subnormal result of v_fma_f64 to zero.
(cherry picked from commit 7df6c709de994cea0019707d45b6e7b96698fad1)
DescriptorBinding gains bufferRead next to bufferWritten and bufferAtomic,
filled from the resource tracker's per-buffer read flag and kept in the
shader disk cache (format 24).
Route oversized compute LDS through a per-workgroup storage buffer.
Retain native shared memory for programs that fit the device limit.
Carry the spill stride through prepared shader results and dispatches.
Preserve the stride in the full-result codec and its round-trip fixture.
Keep the Linux RecompileResult size guard aligned with the added field.
Exercise interleaved workgroups and the device-limit boundary.
This topic contains only the LDS change and its reviewer-requested repair.
The base includes upstream's separately merged NaN result correction.
A compute shader in Astro Bot samples an image as 2D while the bound
T# is 1D, and the recompiler refused it. Offer a 1D runtime mode for
sampled images used only by image_sample_lz and mip-less image_load,
which reads the texel the first coordinate selects, as measured on
RDNA2 in #2237.
A 3D image access bound to a 2D T# matched no runtime image mode and
threw "image descriptor is incompatible with the static runtime image
interface" (Sonic Origins, #2023). CouchRiv measured RDNA2 (gfx1036)
with the hardware oracle: image_sample_lz with dim 3D ignores r and
reads the (u,v) texel, and image_load reads the (x,y) texel when z is
0 and returns zero otherwise. Add a 2D mode for 3D accesses whose only
uses are those two forms, and make the load return zero for z != 0.
Every other access keeps throwing.
Fixes#2023
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without VK_KHR_fragment_shader_barycentric the recompiler dropped
v_interp_p1_f32 and made v_interp_p2_f32 read the attribute Vulkan
interpolated, but never wrote the I/J VGPRs: a pixel shader that read
them for anything else got 0, and v_interp_mov_f32 of P10/P20 (or P0 of
an interpolated input) needed PerVertexKHR and failed pipeline
validation.
The fallback translation now loads the I/J VGPRs and marks v_interp_p1
and v_interp_p2 with witnesses. After the front-end simplification
(SSA, folding, DCE, read-lane and masked-select elimination),
HostInterpolationChecker requires every p2 to read its J, and its p1's
I, from a barycentric of the parameter's kind, every p1 result to reach
only the matching p2 (through a write masked by the p2's EXEC, whose
inactive lanes keep the old register), no parameter read as raw
vertices and no slot shared by different interpolations. It then
lowers the witnesses to the host-interpolated attribute as before,
reruns DCE and masked-select elimination, and accepts the program only
when no barycentric read remains, so reusing the I/J VGPRs after
interpolating keeps host interpolation and the previous SPIR-V. Any other
program is translated with the barycentric path and marked as emulated:
its barycentrics become vec3 inputs (NoPerspective for the linear
weights) and its per-vertex parameters flat vec4[3] inputs, placed in
the free locations by LayoutBarycentricEmulation. The driver then
inserts a generated geometry stage between the vertex and fragment
shaders that copies the position and parameters, writes the one-hot
weights of each vertex and the three vertices' parameter values, so the
rasterizer interpolates the same weights the extension provides.
The stage needs geometryShader and a vertex shader before it;
tessellation, mesh and rect-list pipelines, other primitive types and
interfaces beyond 32 locations or the device's geometry limits throw.
CompiledShaderArtifact carries the emulation (shader disk cache format 21).
Hardware snaps window corners to 1/256 pixel before interpolating
(measured by yahya308 on an RTX 4070 Laptop and a Radeon 780M in #1547);
lavapipe snaps only for coverage (lp_rast.h FIXED_ORDER 8), so the test
accepts either reference per pixel at the original 1e-4 margin.
Sources:
- v_interp_p1_f32/v_interp_p2_f32/v_interp_mov_f32 semantics (P0,
P10, P20, I/J): AMD RDNA2 ISA, "Parameter Interpolation".
- BaryCoordKHR/BaryCoordNoPerspKHR weights and PerVertexKHR vertex order:
SPV_KHR_fragment_shader_barycentric, VK_KHR_fragment_shader_barycentric.
- Geometry shader triangle input order and provoking vertex: Vulkan
specification, "Primitive Topologies" and "Geometry Shading".
- Interpolation decorations decided by the fragment input: Vulkan
specification, "Interpolation Decorations".
- lavapipe interpolation: Mesa src/gallium/drivers/llvmpipe/lp_rast.h
(FIXED_ORDER 8), lp_setup_tri.c (subpixel_snap for coverage only),
lp_state_setup.c (init_args, plane setup from float positions).
Capturing a shader threw "null or misaligned address" when the flat
SRT walk read through a pointer loaded from memory that is null or
unmapped, even when the program skips that s_load. Such a read now
yields zero and a record with its pc and address, as does a slot whose
address needs it. The registered artifact checks one flag word per
guarded slot at each ReadConst; the capture appends the flags and the
records to the flattened SRT, and a set flag raises an Unmapped BDA
fault through the fault buffer and stops the invocation as a BDA read
does. Nothing per capture enters a key: poisoned and clean captures
share one variant and specialization. Descriptors, branch conditions,
uniform fills, table bases and user-data pointers that need such a
value still throw, and poisoned results stay out of the dispatch and
draw caches. The shader disk cache format becomes 21.
When one guest sampler is paired both with an image that needs point
filtering (sint, depth-bits or converted format) and with an ordinary
image, ResourceMaterializer::Apply appends a point-filtered copy of the
sampler after the original samplers. The resource snapshot only holds
the original samplers, so DescriptorBindingBuilder::Populate threw
"guest sampler index is out of range" for the copy, both in the sampler
binding and in the unnormalized-coordinate check.
SamplerResource::copyOf records the sampler a copy was made from, and
the guest sampler descriptor of a copy is read from that sampler; the
point-filter rewrite of word 2 still applies to the copy. The field is
part of the disk cache encoding: cache format version 8.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On GFX10.3 to GFX11.7, ACO lowers a subgroup-scope OpReadClockKHR to
s_getreg_b32 of SHADER_CYCLES, 20 bits zero-extended to 64, so s_memtime
wrapped every 2^20 cycles and went backwards. Detect those families from
the RADV device name and read the device clock there instead.
Fixes#985
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NVIDIA 610.62 cannot sample R8G8_SRGB, and lavapipe can sample neither
it nor R8_SRGB, so an 8_8 sRGB T# made an image the device does not
support: creating it crashes the NVIDIA driver and lavapipe loads zeros.
The device's format support is read once. An 8 or 8_8 sRGB texture the
device cannot sample is created and viewed as R8_UNORM or R8G8_UNORM,
and SpirvTarget::srgbDecodeFormats makes the materializer mark the
image, so image loads map every channel through a 256-entry table of
the sRGB EOTF rounded to nearest even at bf16 precision, which is what
RDNA2 (gfx1035) returns; both channels of 8_8 are sRGB, and DST_SEL 0
and 1 map to themselves. Samples and gathers of such an image throw,
and its bindless table entries are unmapped like converted formats.
APS5_SRGB_SHADER_DECODE=1 takes this path on every device. Shader disk
cache entries and keys carry the mark: cache format version 7.
An S# with FILTER_MODE (word0 bits 29-30) set threw in
DecodeSamplerResource. Modes 1 and 2 now map to Vulkan's MIN and MAX
sampler reduction through VK_EXT_sampler_filter_minmax, which the device
enables when it reports both filterMinmax properties; mode 3 throws.
Min/max still throws with anisotropy, a linear mip filter (lavapipe and
NVIDIA disagree there), depth comparison, color-texture compare
emulation, and, in PointFilteredSamplerWord (S# words 0 and 2), when it
filters between texels or mips of an image the recompiler point-samples.
A texture sampled through a linear min/max sampler needs
SAMPLED_IMAGE_FILTER_MINMAX on its view format; the image-sampler pairs
travel in DescriptorBinding::imageSamplers (shader disk cache format 5).
Adds the agc_driver_image_sample_reduction GPU test and extends the
sampler, emulated color compare and disk cache tests.
The source entry remembers a resource plan that failed to build, but
not a failed emission: findOrCompileVariant kept only successful
variants, and the first attempt used up the prepared program, so every
request for a program whose emission throws ran PrepareResourceProgram
and the emitter again before the draw was skipped.
compileVariant now reports an exception from SpirvEmitter::Emit (and
from SPIRV-Tools validation when enabled), and the source entry keeps
it per code address, binding layout and resource specialization; the
next request for that variant rethrows it before the disk-cache lookup.
Descriptor population depends on the snapshot and an out-of-memory
error is transient, so neither is kept. Nothing reaches the disk cache,
and APS5_NO_FAILURE_MEMO=1 turns the memo off as for plan failures.
An S# with FORCE_UNNORMALIZED threw in DecodeSamplerResource. It is now
bound through a Vulkan sampler with unnormalizedCoordinates when every
use stays where the S# and the host select the same texels.
ShaderInfoCollector records the live uses of each sampler after Apply
and dead-code elimination, and the emitter shares its explicit-LOD
predicate. Populate flags a direct S# with the bit that only explicit-LOD
image_sample without offset, comparison, derivatives or _a uses, paired
only with direct 1D or 2D images without a format conversion; anything
else throws. The driver checks the S# filters, clamps, TRUNC_COORD and
MC_COORD_TRUNC, creates the sampler with nearest mips and a zero LOD
range, and requires a single-level, single-layer 1D or 2D view of mip 0.
The shader disk cache format becomes 5.
Niko
Karan Vora
AlxcBTW
Adrià Franch
Celegans12
Adrià
ThatsLeo
krishnaraya
Dean Galvin
Nahuel Genchi
Daan Dehing
russhil
Dean
Petre Buia
Petre
boykopovar
Gjon Gojcevic
nisbenz