A CMASK that was neither all fast-cleared nor all expanded threw, so a
title that fast clears a target and then draws over part of it before the
eliminate could not run past it.
The CMASK is now decoded tile by tile: one nibble per 8x8 tile in
1024x512 metablocks of 4 KiB, addressed with the pipe-aligned
SW_64KB_Z_X CMASK equation for 16 pipes (addrlib
GFX10_CMASK_SW_PATTERN, Gfx10Lib::HwlComputeCmaskAddrFromCoord). Code 0
stores CB_COLOR_CLEAR_WORD into the tile's texels, 0xF keeps them; any
other code, a cleared tile of a DCC target and any other CMASK swizzle
mode throw. Edge tiles are included and the CMASK is then stored as
expanded.
For a resident target whose CMASK is host-imported the clear is a
compute pass recorded in stream order: it reads the nibbles from the
import, stores the clear into the cleared tiles through a UINT view of
the image and fills the CMASK expanded, so nothing waits for pending GPU
writes of the CMASK. Invalid codes it reads set a bit that the next
materialization throws on. Otherwise the CPU reads the CMASK and stores
the cleared tiles into guest memory, or fills the resident image when
every tile is cleared. A CMASK nothing has stored over since it was
expanded is skipped.
The decode requires CB_COLOR_ATTRIB3 FMASK_SW_MODE SW_64KB_Z_X with
CMASK_PIPE_ALIGNED, a 4 KiB aligned writable CMASK and texels of at most
64 bits, and CMASK_BASE_EXT is part of the draw key.
Tests: CMASK sizes and nibble addresses against addrlib; decode
rejections and draw key coverage; CPU eliminate of full, partial,
out-of-range-nibble and DCC decompress cases and invalid codes. The new
agc_driver_cmask_fast_clear GPU test draws over fast-cleared and partly
fast-cleared linear and SW_64KB_R_X targets, eliminates partial, edge
and checkerboard CMASKs through the compute pass and checks that an
invalid code it reads throws.
A PIXEL_PIPE_STAT_DUMP event (EVENT_WRITE 0x39, Recorder::DumpSamples)
ended the render pass the previous draw left open: the dump ended the
batch-long occlusion query, copied its result with
vkCmdCopyQueryPoolResults and ran the fold kernel, none of which Vulkan
allows inside a render pass. VulkanDevice::DumpSamplesOnGpu also flushed
the queued store run over the dumped bytes first, which ended the pass
and recorded the run's barriers as well. A title that clears each query
slot with WRITE_DATA and brackets every bounding draw with a begin and an
end dump gets a render pass, two copy barriers and a label run per draw.
Samples are now counted in segments. A segment is one slot of a
per-batch occlusion query pool (128 slots, reset outside any pass when the
batch's first pass needs one). It begins at the first draw recorded after
a pass begins or after a dump, inside that pass's subpass, and ends at the
next dump or right before vkCmdEndRenderPass, so each query begins and
ends in one subpass. A pass whose pool has no free slot left is not
continued, so a new pool can be reset. The draw path that records a whole
pass of its own prepares a slot before its pass and ends its segment
before ending it.
A dump ends the open segment and is queued in the batch's store run with
the number of segments before it; it no longer ends the pass and no
longer flushes the queued stores. The run is recorded where queued stores
already are: at Submit, or earlier when a later command reads or writes
the queued bytes. Its recording copies the results of the segments the
dumps need (one vkCmdCopyQueryPoolResults per contiguous slot range) and
the dump list, then records the queue in groups: a group's stores, then
one dispatch of the SampleDumps kernel, which adds each dump's segments to
the running total and stores the 16 counters in queue order. A store over
bytes a dump of the current group writes starts a new group, so every
byte ends with the value of its last write in program order
(Recorder::QueuedWriteGroups). At Submit, segments no dump consumed are
folded the same way once 64 are pending.
No reader sees a dump later than before: a CPU reader of the dumped bytes
syncs the batch (the dump still notes them as a pending write), and Submit
records the run before the batch ends; a later label or store is queued
after the dump in the same run; GPU work that reads or writes the queued
bytes flushes the run first, as for queued label stores. No reader sees it
earlier either: the dump is recorded after every draw whose samples it
counts, and its segments end at the dump. Without the GPU sample counter
(no buffer device addresses) the batch-long query is kept as before.
Tests (agc_driver_recorder_tests, GPU, run by hand): QueuedWriteGroups
keeps the program order's bytes for 20,000 random store and dump
sequences; a queued dump does not land before its run is recorded, a
reader of other bytes leaves it queued and a reader of its bytes records
it; clear, dump and store orders over shared bytes; two dumps inside one
render pass keep it open (the next draw continues it) and split its three
draws' samples exactly with precise queries; a reader flushing a dump
inside a pass ends the pass; 160 dumps in one pass past a pool's 128
slots; 70 passes without a dump folded at Submit.
A sampler with FORCE_DEGAMMA (S# word 0 bit 20) threw at decode, so
every dispatch or draw sampling through one was skipped.
DecodeSamplerResource now records the bit, but accepts it only for a
caller that pairs the sampler with its images. The direct sampler
binding does, and RequireDegammaFormat checks each sampled image
against its paired samplers: only already sRGB formats (8, 8_8 and
8_8_8_8 SRGB, BC1, BC2, BC3 and BC7 SRGB) pass, decoded once, as
shadPS4 (PS4) models it. Other textures and unpaired decodes still
throw; gfx10 behaviour is not measured.
The device-local tier of BufferPool only reused a buffer of the exact
size and usage it was asked for. Partial storage-image detiles and
retiles ask for a new size and the other transfer direction almost
every time, so the tier sat at its budget and missed and evicted
dozens of buffers per frame, and the vkFreeMemory calls on the release
thread held up submits and allocations on the recording threads.
Device-tier requests of 1 MiB and more now round up to an eighth of
their octave, the tier's buffers are created for storage and both
transfer directions, and a request takes the smallest retained buffer
from its own class up to twice its size. Host tiers, the budget and
eviction are unchanged. agc_buffer_pool covers the classes, the fit
and its bound, reuse only after release, the budget and the unchanged
lists.
A build served by the cached address space walked every region of the
space for its mirrors, sorted the heap mirrors, compared each heap
mirror's blocks with the write tracker under its own lock and swept the
whole mirror registry. In an offline replay with 37,000 heap mirrors
that took about 10 ms per build, every build.
The space now keeps its writable mirrors, its sorted heap mirrors and
their adjacent runs from when it is published. Each run is still
flushed and collected; a run with no block stamped since the generation
its last complete check collected skips the per-mirror compare, and that
generation is stored only once the build's blocks were copied. The
registry sweep is skipped while every live mirror's range is held by the
cached address space's lease and no mirror was added, replaced or
dropped since.
A draw's guest buffer region inside a host import at an offset off
minStorageBufferOffsetAlignment was copied per build (a new buffer, an
input snapshot, a vkCmdCopyBuffer that ends the open render pass), and
DirectRegions refused such builds as templates.
Descriptor already binds any DWORD-aligned view from the aligned offset
below it and passes the remainder through the shader's memory offset
slot. A draw build that is not address-based and whose every guest buffer
element has such a slot (a push constant byte, or a shader data byte
inside its data buffer) now binds a region whose offset in its import is
DWORD aligned in place, like an aligned one: its read-only views are
snapshotted per draw by PrepareDrawBindings at address - adjustment, the
import serial keys the template, and Revalidate proves it per use.
Dispatches keep the copy (they read in-place inputs at execution, not at
record).
agc_driver_recorder_tests misalignedRegionTests: a misaligned read-only
draw region is read in place, the build is reusable, the push constant
carries the adjustment, and the per-draw snapshot starts at the aligned
offset below the view.
With stride 0, OOB_SELECT 2 puts a fetch out of range only when NUM_RECORDS
is 0, and OOB_SELECT 3 only when the offset is not below NUM_RECORDS (GFX10
table in Mesa ac_descriptors.c). A one-record descriptor therefore returns
its whole element: Unreal's per-draw PrimitiveId stream is such a descriptor,
and refusing it skipped the draw. A wholly out-of-range fetch reads a zero
buffer.
The storage image and sampled texture caches each let go of their least
recently used entries past a fixed 2 GiB, and a sampled view of a
storage image counted that image a second time.
The storage image cache keeps a quarter of the largest device-local
heap, at least the 2 GiB of before (TextureCacheBudget). The sampled
cache gets what VK_EXT_memory_budget leaves on that heap after the
memory in use outside sampled textures and cached storage images, the
storage cache's budget and an eighth of the heap budget, read at most
once a second and at least 2 GiB; without the extension it keeps the
storage cache's rule. A snapshot counts its image allocation, a texture
still bound after leaving the cache counts as sampled memory, and a
change of more than 10% is logged at most every 10 s. A nonzero
APS5_TEXTURE_CACHE_MIB sets both budgets; a view of a cached storage
image counts nothing and leaves the cache when its image does.
The recompiler emits OpCapability Int64Atomics for 64-bit buffer atomics
in any stage, and the device enables shaderBufferInt64Atomics when it
has it, but graphics shader validation only accepted its fixed list of
capabilities, so every draw whose shaders use one was rejected with
"SPIR-V requires unsupported device capability 12". Compute dispatches
are not validated this way and already ran them.
The device records the feature (bufferInt64Atomics in the device state
and the graphics Context) and ValidateShaders accepts Int64Atomics when
it is enabled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SPI_SHADER_COL_FORMAT 7 (UINT16_ABGR) was rejected by DecodeState,
though the recompiler already handles it: a pixel shader's MRT output
under mode 7 is a uvec4, and a compressed export's 16-bit halves are
taken with OpBitFieldUExtract. The format is now accepted when the
target's CB_COLOR_INFO number type is UINT; into any other target it
still throws, as does SINT16_ABGR (8), whose halves would need sign
extension.
The target records the export (ColorTarget::uintExport). Shader
validation then requires a uint4 output at its location instead of a
float4 one, and blending into it is refused (Vulkan has no blending of
integer attachments).
PPSA14632 (LEGO Horizon Adventures) writes a 2848x1600 R8_UINT target
this way; those draws were dropped.
agc_driver_pixel_uint16_export draws compressed MRT0 exports under mode
7 into a linear 16_16_16_16 UINT target, one row per pair of packed
words (0x0000ffff/0xffff0000, 0x00010002/0xffffffff and four distinct
halves), and reads every 16-bit component back under the identity,
STD_REV and ALT_REV export mappings. agc_driver_graphics covers the
decode and the blend refusal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When another live image overlaps the fill, a fill at a live image's dccAddress is a key fill
only while it stays within DccKeyCount (the pipe-aligned key count); with nothing overlapping
the length rule is unchanged. ClassifyFill, NoteKeysFill and ClearByKeysFill share the match.
agc_driver_fill_cover_keys pins Keys, Inside and Several for fills at and beside the keys.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The console's DCC of a single-level 4-byte SW_64KB_R_X surface is
ceil(W/512) * ceil(H/512) * 4096 bytes, 49152 for 1920x1080, but the
driver stored and read one key per 256 surface bytes (34560).
PPSA26344 retiles its render DCC into the displayable copy it
registers (cs_dcc_retile_32bpp); a retile of the stale tail gives the
mixed keys PPSA12544 stops on.
DccKeyCount gives that extent. Color target and storage image key
stores cover it when the DCC is pipe-aligned (CB_COLOR_ATTRIB3 bit 30,
T# word 6 bit 19), whose CB keys reach its last meta block, and throw
where it is not writable; unaligned DCC keeps one key per 256 bytes,
as do texture and color target key reads. DisplayBufferKeys reads the
display DCC over the extent, presents only when every key agrees,
names the first differing key's pixels and checks a pending image
against those keys.
A draw whose PA_SC_CONSERVATIVE_RASTERIZATION_CNTL (context 0x313) was
not the default 0x6000 threw in DecodeState and the draw precheck.
0x6001 sets OVER_RAST_ENABLE and now decodes to Vulkan's overestimation
mode, chained to the pipeline's rasterization state and its key.
The device enables VK_EXT_conservative_rasterization only when its
primitiveOverestimationSize is at most 1/256 pixel and it rasterizes
degenerate triangles; elsewhere the pipeline throws. Overestimation is
accepted only when triangles reach the rasterizer: VGT_GS_OUT_PRIM_TYPE
behind a geometry shader, VGT_TF_PARAM's output topology behind
tessellation, VGT_PRIMITIVE_TYPE otherwise. Other 0x313 values and
centroid interpolation still throw.
Adds the agc_driver_conservative_rasterization GPU test and extends
agc_driver_graphics.
A V# whose base is not a multiple of 4 bytes from the start of its GPU
owner threw in GuestBufferMemory::Descriptor, because the shader added
the offset below the view only in whole dwords.
Descriptor now binds from the offset alignment below the view and
passes the whole remainder. Such a view's range covers the dwords
around it, and its descriptor region starts and ends on dword
boundaries, so a lone view copied to its own buffer binds the same way.
The buffer specialization records the V# base's low two bits. Raw
vector loads through such a V# join two dwords, glc dwordx2 loads
included, stores merge their bytes into both with compare-exchange
loops, and bounds checks include the second dword. s_buffer_load
ignores the low two bits, as SrtEvaluator does. Formatted accesses
whose element alignment (min(element bytes, 4)) the base breaks throw,
as they fault the wave on RDNA2, and so do atomics.
StorageTexture::MarkDirty skips the first write's uncompressed key
store while the image's filledKeys holds a clear code, and only an
upload or clearByKeysFill reset that fill. After a 0001 key fill, a
kernel storing 0000 over the keys and a storage write, the refresh of
write-watched memory read 0000, the keys the image was uploaded under,
so it did not upload again and kept the stale 0001. The write then
left the 0000 keys in memory, and a kernel reading them saw the clear
colour.
StorageTexture::ProvedKeys, the key read of Refresh and the fast
revalidation, now forgets the fill when it reads other keys while the
image has no results pending. An image with pending results keeps it,
since its write-backs treat those results as killed by that fill.
Dean Galvin
Dean
Sasha Papo
Adrià
Zheorgyan
Celegans12
Joey Fisher
boykopovar
krishnaraya
TideOPL
alfalfa-star
Michon