MaskedSelectEliminator walked forward from each select to see whether the
lanes its mask excludes are read. Since fda4aa90 the walks pass explicit-LOD
gathers, size queries and guarded image writes, so in a large pixel shader
most of them cover the rest of the program, and the cost grows with the
square of its size (#3504).
The pass now computes, for every value, the set of masks under which it is
unread, from its uses backwards: a use guarded by a condition contributes the
masks that condition implies, a lane-local use adds what its result is unread
under, and any other use contributes nothing. A select is dropped when its
mask is in its own set. Loops are solved as strongly connected components.
The rules for guards, lane-local uses, loops, whole quad masks, selected
execs, rebuilt mask words and guarded first-lane reads are the ones the walk
applied. A first-lane read passes the masks its read mask implies when its
own result is unread outside that read mask, which is solved once per read in
the same way. The walk's step limit, its 64-level implication depth and its
first-lane nesting depth are gone, so a proof no longer depends on the order
of the questions.
A lane set contained in X is contained in WQM(X). Containment is proven
through BitwiseAnd and through ballots evaluated in the uniform entry
blocks, where every invocation is active. Under a whole quad mode mask the
masked lanes form whole quads, so implicit-LOD pixel samples, which only
read their own quad, do not observe them.
This removes the selects that kept the packed pixel ancillary input alive
after its fields were extracted in GTA V (PPSA04264) pixel shaders.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1W4jdNSKTV7xwuS6xQPJV
Following the ImageRead case, every image operation was checked against
the two lists the masked-lane walk uses: accesses guarded by a trailing exec
operand, and lane-local values.
- ImageWrite carries its exec last like StoreBuffer, and EmitWriteOp stores
only inside an if on it (bindless tables and FMASK refuse the store). It
joins the exec-guarded list.
- A gather with an LOD of its own (gather4_l, gather4_lz) or outside a pixel
shader reads only its lane's address, like the explicit-LOD samples the
walk already follows. A pixel gather without one still keeps the select,
since on the GPU it takes its LOD from the quad.
- ImageQueryDimensions returns the size of its lane's level of the image and
is lane-local.
Left out on purpose:
- ImageQueryLod and implicit-LOD samples read their quad.
- ImageBvhIntersectRay has a trailing exec operand, but its emitter ignores
it and records faults in every lane.
- Image, buffer, shared and address atomics are guarded by their exec in the
emitter too, but the test from 60c65521 pins them as keeping the select,
and no refused shader has needed them so far.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An ImageRead carries a trailing exec argument like the buffer loads, but it
was in neither the read-only-where-active list nor the lane-local list, so a
masked select feeding its address was kept.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Kena: Bridge of Spirits (PPSA01802) refuses ten pixel shaders with
packed pixel ancillary input has an unsupported live use (used by SelectU32 ? ?)
at any visit limit. All ten have 30 to 33 phis on loop back edges (reported
on Linux with an RX 9070 XT).
Two rules stopped the proof at a loop, and the first one fires before the
second is ever asked:
- The walk from the select gave up at the first use inside a loop. A value
derived from the select reaches a loop phi as soon as its register is
written inside the loop, because the masked write becomes
Select(loopExec, new, phi). A loop that leaves exec alone needs nothing
more than this.
- An exec that a loop narrows and carries around its back edge is a phi
inside the loop, and such phis counted as not implying the write's exec.
The walk now follows uses into loops. A phi inside a loop implies the
write's exec when all of its incoming values do, where the phi itself counts
as implying it while its back-edge values are checked. This is sound because
each rule of the proof takes a lane bit of one value from a value computed
earlier: a phi's value on each iteration is the incoming value computed
before the edge was taken, a set lane of an And is set in both operands, a
set lane of a ballot is a lane whose predicate held. If any value in the
proof had a lane set where the write's exec is clear, the earliest such
value would take that lane from an earlier value of the proof that is set
there too, which contradicts it being the earliest. Two conditions keep this
true:
- The write's exec must be computed once, outside every loop. Otherwise a
phi can carry a ballot of it from an earlier iteration, which is not the
value the select used. Masks computed inside a loop keep the old rules.
- A result may only be kept if the assumptions it used held. When a value
that was assumed turns out not to be implied, every result recorded since
it was assumed is dropped. Results that say "not implied" are kept; they
only make the proof refuse more.
A lane that is turned back on inside the loop makes the back-edge exec wider
than the write's exec, so the phi is not implied and a read under it keeps
the select. A lane that is turned back on but never reads the derived value
does not.
New tests cover a loop that carries the write with exec saved, narrowed and
restored (read inside and after the loop), a loop that leaves exec alone,
and the helper-only ancillary select feeding such loops. The refusing tests
each fail when the matching check is removed: a revived lane that reads the
write inside the loop (ignoring back-edge values), a result proven under an
assumption that later fails (keeping it), and a write mask computed inside
an earlier loop (allowing such masks). Two existing tests expected a loop to
keep the select although every read is under the write's own exec; they now
expect it to go, next to new counterparts with a wider exec.
On the Taxi Chaos 2 requests, compute shaders with a loop inside a branch now
lose the branch's masked writes: 0x194000b400 removes 207 selects instead of
36, 0x19002bd200 111 instead of 28, and seven more gain 5 to 43. In
0x16400cf700 the 26 new removals are writes under the branch condition
tid < count; the loop's exec starts as a ballot of that condition, only loses
lanes at the latch, and every store and atomic inside the loop runs under it.
No proof on these requests had to drop a result.
Walks now go on into loops, so the pass takes longer on loop shaders, but it
stays below main. Time in MaskedSelectEliminator per prepare (median of 3,
the pass runs twice per prepare), 54 Taxi Chaos 2 requests, one run:
shader previous commit this commit main 5edae353
0x1ac0413000 334 ms 349 ms 424 ms
0x19002bd200 38 ms 103 ms 112 ms
0x19c016aa00 9 ms 74 ms 16 ms
0x194000b400 13 ms 31 ms 32 ms
all 54 547 ms 805 ms 977 ms
0x19c016aa00 removes nothing more; its walks only end later inside its
loops. The 144 requests of the NVIDIA dump go from 1283 to 1589 ms and gain
removals in the copies of the same shaders. 197 of the 198 requests prepare,
as before.
Related: #2548
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LEGO Horizon Adventures (PPSA14632) still refuses a pixel shader with
packed pixel ancillary input has an unsupported live use (used by SelectU32 ? ?)
although it has the form the previous commit handles: the only whole-word
reader is SelectU32(IEqual32(HelperInvocation, 0), BitFieldUExtract(x, 0, 11),
ancillary), the exec words are Ballot(not helper) & Ballot(true), and there
are no image ops, stores or loop phis. About 10,250 IR values derive from the
word, so the walk from that select stops at its fixed limit of 4096 visited
values and the select is kept. With the limit raised to 8192 the shader
prepares (reported on Linux with an RX 9070 XT).
A walk may now take max(4096, 2 * instructions) steps. A step is a value
visited for the first time or an implication evaluated for the first time.
The walk only visits values derived from the select, at most one per
instruction, so a walk is only cut short when the implications it evaluates
cost as much as the program itself. On the 198 Taxi Chaos 2 requests no walk
needs more than 0.79 * instructions steps. One walk stays linear in the
program, O(instructions + operands): each value is expanded once, and with
the per-walk memo each implication is evaluated once.
The walk is also cheaper, so that the larger limit costs nothing:
- visited values are marked in a table indexed by value id instead of a hash
set, and the table, the work list and the memo are reused across walks;
- blocks outside loops are looked up in a table indexed by block id instead
of a hash set. This lookup ran once per use and was most of the pass's time.
Time in MaskedSelectEliminator per prepare (median of 3, the pass runs twice
per prepare), 54 Taxi Chaos 2 requests:
shader before 40a8a5a7 this commit main 5edae353
0x1ac0413000 689 ms 285 ms 401 ms
0x19002bd200 102 ms 34 ms 100 ms
0x18c07bf000 191 ms 52 ms (refused)
all 54 1294 ms 484 ms 919 ms
0x1ac0413000 and its three copies in the NVIDIA dump now remove 659 selects
instead of 625: their walks no longer stop at 4096. No other request changes
what it removes. 197 of the 198 requests prepare before and after; the other
one fails the same way in both (v_readlane_b32 of lane 63 on a 32-lane host
subgroup).
Related: #2548
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Taxi Chaos 2 (PPSA30279) pixel shader 0x18c07bf000 is refused when it is
registered, and the title stops:
packed pixel ancillary input has an unsupported live use (used by SelectU32 ? ?)
The shader reads the ancillary VGPR (v2) once, with v_bfe_u32 v3, v2, 16, 11,
which #1904 lowers to Layer. The next write to v2, v_rcp_f32 v2, s33, runs
before any kill under the initial exec and becomes
SelectU32(IEqual32(HelperInvocation, 0), rcp, ancillary). The raw word
therefore survives only in helper lanes. The shader has no s_wqm, no DPP,
readlane or swizzle, no stores, and only *_lz samples and gathers. Every
helper-lane value that comes from v2 is dropped by a later exec select, a
compare that is ANDed with exec before its ballot, or an export whose exec
excludes helper lanes.
MaskedSelectEliminator exists to remove selects like this one, but it could
not prove the case. The kill paths save and restore exec, so the later execs
are phis at block merges, and the AND chains the kills build go deeper than
its eight-level limit.
When it checks whether a later exec implies the write's exec, it now:
- accepts a phi when the phi and each incoming value lie outside loops and
every incoming value implies that exec;
- pairs low and high exec words merged by phis in one block, predecessor by
predecessor;
- memoises results for each select and follows up to 64 levels. A cycle
counts as not implied.
Once the select is gone, nothing reads the ancillary word, so no bit of it
is produced. A select whose false lanes can still be observed is kept and
the shader is still refused. That covers a data test that leaves the word in
non-helper lanes that later export, an export under the full wave, and an
unmasked ballot.
Related: #2548
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A register written under EXEC is usually consumed by an LDS or buffer
access or an export under the same EXEC. MaskedSelectEliminator kept
the select in that case, which leaves the NGG mesh shaders where they
were.
The emitter performs buffer and LDS loads and stores (8 to 128 bits,
GPU-selected V# included) and SetAttribute only in lanes whose active
operand is true: stores and exports sit in an if on it, loads return 0
outside it. Every other operand is read only in those lanes. So such an
access whose active operand implies the select's condition reads the
derived value only where it equals the original, and it now ends the
search like a masked select does. The active operand itself, atomics,
append/consume, scratch and address-space accesses and image accesses
still keep the select.
Every VGPR write under EXEC is translated to Select(exec, new, old), so
in SSA each register's previous value stays live until the next write,
even when the guest only used the register as a temporary. In NGG mesh
shaders at 255 registers that keeps dead values alive and the driver
spills them to local memory.
MaskedSelectEliminator (after SSA, folding and the readlane pass)
replaces r = Select(m, new, old) by new when no lane where m is false
can observe r:
- every value derived from r is either a lane-local pure op or a phi,
and their uses are checked the same way;
- or the derived value reaches the true operand of a select, or an
operand of a LogicalAnd, whose condition implies m: m itself, a
LogicalAnd containing m, or a thread bit rebuilt from a ballot of
such a condition (also through ANDs of the mask words);
- anything else (cross-lane ops, ballots, readfirstlane, DPP, swizzle,
permutes, stores, exports, atomics, branch references, implicit-LOD
pixel samples, the false operand of any select) keeps the select.
The select and every value it reaches must be in blocks outside loops,
so each of them runs at most once per invocation and every use sees the
same instance of m. Selects whose condition is the full wave
((0xffffffff >> (lane & 31)) & 1) are replaced in any block.
Exact: in lanes where m is true r equals new, so every derived value is
unchanged there; in lanes where m is false only derived values change,
and each path from them ends in a select or AND that discards those
lanes. The selects are removed latest first and dead code is removed
between rounds, so chains of writes to one register collapse.
Dean Galvin
eljackjoestar-cmd
thibaut
AlxcBTW
xXPBSXx