A thread that blocks on a semaphore between iterations is waiting, like the
condition-variable waits already on the list. A job-wait loop that runs queued
work between timed semaphore waits was otherwise taken for a hang once loading
outlasted the guard window.
The export was a success stub that never wrote its out parameter, so callers
sized buffers from uninitialized memory. The signature is (id, uint64_t* size);
it now answers from the APR file registry like sceKernelAprGetFileStat.
#955 overlaps several open PRs. Resolution:
- GuestPageTracker, GuestBufferCache, ShaderIdentity, SharedBackingViews: upstream's versions
(they carry the same allocation-free invalidation, hashing and lock-free reads as #944/#968).
- RELEASE_MEM: upstream's deferred release flush (supersedes #966).
- CommandStreamQueue.Done: ours (#941 queued frame boundary; upstream only reformatted).
- Flip snapshot pool: ours (locked; snapshots are taken on the command-stream thread and
retired on the render thread), upstream's unlocked pool dropped.
- Gen5SpirvTranslator: our per-register variables and structurer (#923, #975/#976) first,
upstream's guarded in-order blocks as the fallback, then the dispatcher; upstream's
lane-spill slots, bounded move-relative, exec-known-full and PS input sharing kept.
* [Gpu] Astro Bot performance: 14 -> ~60 fps at the title scene
- Defer RELEASE_MEM command-buffer flushes instead of submitting per packet
- Resolve indexed indirect draws and dispatches on the GPU
- Async buffer readbacks on a dedicated transfer/compute queue
- Pool image backings, flip snapshots and tiler resources
- One-frame command-stream run-ahead
- Lock-free backed-range and image page-owner checks on CPU writes
- Scalar value cache with generation-stamped reset; memoized invariant phis
- Shader compiler: structured forward-only CFGs with function-scope
registers, EXEC-full guard elision, bounded V_MOVREL offsets
- Trivial-leaf import dispatch and strchr/strrchr fast paths
- SHARPEMU_GUEST_REFRESH_RATE option
* [Tests] Fix full-suite flaky tests
- Red-zone patcher: when every fixed trampoline candidate is taken, look
for any free host range within relative-jump reach
(PhysicalVirtualMemory.EnumerateFreeHostRanges); the test places its
image in a probed free hole
- Kernel memory tests: probe a free PRT-area address instead of the fixed
0x1026C00000; the reservation-extent test only rejects picks inside
the reservation
- Release checks tolerate another thread reusing the freed range
- Allocation tests measure on a dedicated thread and keep the least
allocating of several rounds (AllocationMeasurement.SteadyState)
* [Gpu] Astro Bot: structured loops, buffer slabs, pad motion
- Shader: emit natural loops as structured SPIR-V loops (OpLoopMerge)
instead of the block dispatcher, and merge SCC-uniform if/else joins
as selects so descriptors picked on a branch stay evaluable per draw.
- Shader: decode V_FFBH_I32 and V_MIN3_F16; read the DS GDS bit from
bit 17 (GFX10) and route GDS append/consume through shared atomics.
- Gpu: sub-allocate small device-local buffers from shared memory slabs.
- Pad: report motion data by default (Astro Bot never imports
scePadSetMotionSensorState), convert acceleration from m/s^2 to G,
and add scePadResetOrientation / scePadSetAngularVelocityDeadbandState.
* [CPU] Stop pinning guest threads to host cores by default
Guest affinity packed Demon's Souls' 13 spinning BPE job workers onto 10 host lanes and put its renderer next to them at equal priority, so the renderer and lock holders waited out whole scheduler quanta. Intro 10.6 -> 44.7 fps, menus 11.6 -> 47.5 fps. SHARPEMU_GUEST_AFFINITY=1 restores pinning.
* [Media] Convert host movie frames to YUV on the decoder thread
The render thread converted every decoded Bink frame from BGRA back to YUV 4:2:0 (6.7 ms per frame). The decoder thread now produces the planes and the presenter uploads them as-is. PS Studios logo 33 -> ~50 fps, attract movie 44 -> 60 fps.
* [Agc] Accept a zero destination in sceAgcDcbWriteData
Demon's Souls emits WRITE_DATA with address 0 into a template and patches it after copying. Returning null made the title patch a relocated null pointer, which wrote a stack-derived value into its label and was then parsed as an unknown NOP marker (GPU FATAL).
* [Loader] Patch short red-zone faulting accesses next to branches
A faulting access shorter than a jump could not be relocated when followed by a branch or a branch target, so a handled fault let Windows overwrite the red zone. Spans may now end with a direct branch (run after the stack pointer is restored) or start up to three instructions earlier. Fixes a Demon's Souls crash after the new-game intro movie.
* [GPU] Astro Bot rendering fixes: compute subgroups, DCC clears, PS inputs, texture MIN_LOD
- Compute: pin non-emulated compute pipelines to a 32-lane subgroup
(VK_EXT_subgroup_size_control) so wave64 groups other than 64
invocations keep lanes 32..63 active on AMD (striped froxel volume).
- Shader translator: 64-bit AND for the wave32 S_*_SAVEEXEC_B64 mask.
- PS inputs: slots that read one VS parameter share one host input;
a smooth/flat slot next to a V_INTERP_MOV slot is rebuilt from the
per-vertex values (black save cards, NaN normals, black props).
- Render targets: drop unwritten color targets of depth passes from both
the attachments and the pixel outputs, keeping locations aligned.
- DCC: materialize pending zero fast clears before shader access so a
later attachment bind does not wipe compute writes.
- Textures: honour the T# MIN_LOD with VK_EXT_image_view_min_lod so
streamed textures never sample unloaded mips (blown-out stadium sky).
- Device: enable VK_KHR_maintenance5 for point lists without PointSize.
* [Audio][GPU] ATRAC9 independent mono streams, BC storage fallback, 32-bit bit counts
- LibAtrac9: decode config header 0x30 streams (PS5 ambisonics), where every
channel is an independent mono stream interleaved frame by frame.
- Images: when the driver refuses storage usage on a block-compressed image,
compute writes go through an uncompressed replacement of the same memory.
- SPIR-V: count 64-bit masks with two 32-bit OpBitCount, which Vulkan requires
without maintenance9.
- Agc: drop the indirect-draw emit debug log; report the draw modifier in the
unsupported indirect draw error.
* [GPU] Partial texture-pool uploads and cheaper per-draw preparation
- A CPU write into a large layered texture (Demon's Souls' 256-layer BC3
pools) re-uploaded the whole image, ~750 ms of GPU detiling in one frame.
Each (mip, layer) piece's guest bytes are hashed at upload; a refresh
uploads only the pieces that changed.
- Device-address preparation touched and checked every registered buffer for
each dispatch. Recency touches now run once per retirement tick, and uploads
only visit buffers in blocks the tracker does not know to be clean; hot
pages stay dirty, so they are still re-uploaded every time.
- Resource materialization cache: when the only changed words are ones the
flattened table alone reads (per-frame inline constants), only the table is
evaluated again.
- A full batch no longer splits an open render pass before 512 draws.
- Lock-free lookups for shader headers, device format properties and the idle
host movie bridge; the SPIR-V dump toggle is read once per program cache.
* Demon soul - fix texture
* Delete runahead tests
* Screcrow fix for nvidia GPU
A RELEASE_MEM that writes a 32-bit label submitted the open batch at once. Without an interrupt
nothing inside the slice waits for the label, and every slice that makes progress already ends
with a submit - including one that stops at a WAIT_REG_MEM on that label - so the label is
written when that batch completes either way.
The per-label submits split Superliminal's frames into ~35 extra command buffers; each ended the
render pass, which stores and reloads every attachment on a tile-based GPU. The title menu went
from 15.6 to 21 fps and Octopath from 13-21 to 18-29 fps, with no hang over 150 s runs.
The image collector runs after every completed guest submission as well as at each flip, and
every run advanced the age it compares against. A title submits dozens of command buffers per
frame, so "unused for 16 ticks" meant a fraction of a frame: every texture sampled once a frame
was deleted and recreated, and Superliminal uploaded 3,591 textures (2.2 GB) every 5 seconds.
Deleting a GPU-written image the cache cannot read back also dropped its contents.
Only a flip now advances the age. Collection after a submission still runs, so memory pressure is
relieved during long stretches without a flip. Superliminal's per-frame uploads went from ~55 to
none, and its intro scene from 12 to 45 fps.
Only forward, nested branches were emitted as structured control flow; every program with a
loop, and every single-block program, went through the loop around a switch on the guest PC,
which keeps all registers live and stops the host compiler from optimizing across blocks.
A block that a later block branches back to - on a wave-uniform condition or with s_branch -
now opens a SPIR-V loop whose body is structured like any range, whose continue evaluates
that branch's condition, and whose iterations share the dispatcher's step guard so a guest
loop that never ends still cannot wedge the GPU queue. A conditional branch from the body to
the block after the latch is a break out of the innermost loop. Any other backward edge, or a
break past an outer loop, keeps the dispatcher as before.
Across eight games, programs left on the dispatcher went from 760 to 104 (Poppy, Silent Hill,
Oregon Trail and Dreaming Sarah to none); every emitted module passes spirv-val and the
frames match. Octopath's 12.5M-invocation instanced draw went from 12.2 to 10.8 ms.
Every mapping change queued behind the GPU worker so it could not race a pending GPU read.
During loading that worker is busy translating shaders, and Demon's Souls' reservations and
batch maps waited seconds for a queue slot (23.6 s over 132 calls, up to 7.1 s each) while
the change itself took about a millisecond.
A change that only places memory where the GPU was never given anything - a kernel-chosen
address, or a fixed range that is empty or reserved and overlaps no registered span - cannot
race a GPU read and has nothing to unregister, so it now runs on the calling thread. Those
paths also skip the unregister request for free and reserved ranges, which are never
registered; waiting on the worker while holding the mapping lock that GPU reads take would
deadlock, so reaching a registered span there throws instead.
The guest context-transfer stub that fiber switches go through keeps the
Windows TEB stack bounds in sync by storing the target stack top and
bottom to gs:[8] and gs:[16]. Unlike the guest-entry and exception stubs,
it did so on every host. On macOS gs addresses the pthread TSD, where
those slots hold errno's address and another key: a fiber switch wrote
its stack bounds over them, and the next failed libc call on the thread
(a timed-out condition wait) crashed reading errno through the pointer.
Demon's Souls died this way within 10-50 s of boot; a hardware
watchpoint on the thread's errno slot stopped in this stub.
The fiber bounds sync now lives next to the other TEB-bound helpers and,
like them, emits nothing off Windows.
The red-zone patcher computes a shifted core for every span: the run of
faultable guest memory accesses that executes with RSP moved below the
red zone. A span over register-only instructions has no such access, so
the core could not be computed and the span was refused as too short.
SHA rewrite sites are exactly that shape (sha256msg1/msg2/rnds2 on
registers), so no SHA instruction was rewritten any more: on hosts
without the SHA extensions every one trapped and was emulated one at a
time. Poppy Playtime logged sha_rewrites=0 with 144 too-short refusals,
hit hundreds of thousands of illegal-instruction traps and stopped making
progress within a minute.
A span with no guest memory access has nothing that must run outside the
shift, and the span builder has already refused any instruction that
touches RSP, so the whole span is the core.
Every runtime module load sets up the import stubs of all loaded modules
again, and each setup built a fresh trampoline and rewrote the stub's 16 bytes
-- under guest threads that may be executing exactly those bytes.
A trampoline only encodes its import index, so the trampolines are now cached
by index and reused, and a stub that already jumps to the trampoline it would
be given is left untouched, with no unprotect/write/restore cycle at all. A
reload of a module whose stubs are already patched therefore writes no guest
code.
The trampoline ABI test builds its backend with GetUninitializedObject, which
skips field initializers, so it has to supply the new cache; both fields now
go through one helper that names a field it cannot find.
The export returns the context memory size through one size_t. For heap
output it also wrote a second qword (an alignment) after it, and titles keep
other locals there: Octopath Traveler II's allocator reads the dword at +12 of
that block as a divisor, so it faulted with #DE during boot. Write only the
size, wherever the output lives.
A draw with no color target, no depth target and no pixel stage was treated as
having no observable effect and dropped. That is wrong when the vertex or
emulated geometry stage stores to a buffer or a storage image: a GPU-culling or
stream-out pass whose only output is the buffer a later indirect draw reads
looks exactly like that, and dropping it leaves the indirect draw reading
stale data.
Such a draw now runs for its stores alone. The check only resolves the shader
programs when there is an export address to resolve, so a genuinely empty draw
still costs nothing.
sceImeKeyboardGetInfo zeroed 0x88 bytes at its out pointer and returned
success. SceImeKeyboardInfo is 0x24 bytes -- six 4-byte members plus
int8_t reserved[12] -- so the call scribbled 0x64 bytes past the caller's
struct, and a title that reads the device type or the status got a zero for
both, which is not a state the library ever reports.
The export now writes exactly sizeof(SceImeKeyboardInfo) and reports an OSK
device in the DISCONNECTED state, which is what the host can honestly claim
since no physical keyboard is bridged. sceImeKeyboardGetResourceId returned
success without touching the caller's array, leaving the storage that
sceImeKeyboardGetInfo is called with uninitialized; it now publishes the
caller's user id with an empty id array. Both use the real error values.
Also adds a test for the existing scePadResetOrientation, which is a
handle-only call with no out parameter, so a bad handle is its only failure.
The three waveform-parse entry points returned NOT_FOUND, and Void Terrarium
calls ParseWaveformData on its audio thread for every sound it loads.
They now parse the container and publish the description SceNgs2WaveformInfo
lays out (0xE8 bytes, with the block table at 0x48). A RIFF/WAVE fmt chunk
resolves to PCM by bit depth, to IEEE float, through the SubFormat GUID for
WAVE_FORMAT_EXTENSIBLE, or to ATRAC9, whose configuration is decoded so that
the audio unit size, samples per unit, units per frame and delay samples
describe the real frame and superframe. A smpl chunk supplies the loop points
and a fact chunk the decoded sample count. The "VAGp" container maps onto the
VAG waveform type at 28 samples per 16-byte HE-VAG unit.
ParseWaveformFile resolves the guest path and reads the header at the given
offset. ParseWaveformUser would have to drive a guest read callback, so it
reports INVALID_WAVEFORM_DATA rather than fabricating a description.
Unreal binds a whole volume as a render target with the last slice set to
its depth, one past the end, and the resolver stopped the session on it.
The color block writes only the slices the volume has, so the view is now
clamped to them. A view that starts past the last slice is still fatal.
Out-parameter handling in libSceAudioOut2 was inferred at runtime: a helper
picked whichever of two argument registers looked like a writable pointer, and
another decided from the host address range whether a pointer was a guest
stack slot. Neither is a signal. The ABI names exactly one out pointer per
argument slot, and guest stacks sit at 0x7FFF_xxxx_xxxx on Windows but
0x6FFF_xxxx_xxxx on macOS, so the heuristic silently changed behaviour per
host. Every export now reads its out pointer from the register the prototype
gives it: ContextQueryMemory from rsi, PortCreate from rdx, PortGetState from
rsi, GetSpeakerInfo and GetSystemState from rdi.
The written sizes are now the real sizeof() of each struct.
SceAudioOut2PortState is 0x40 bytes, not 0x20. SceAudioOut2SpeakerInfo is
0x50, not 0x20, and its contents were invented; it now reports a TV speaker
type with front-left and front-right and the two stereo speaker angles.
sceAudioOut2SpeakerArrayCreate follows its real signature (pHandle,
pVbapParams, pAmbiParams): the handle goes to rdi, the speaker count and the
caller's buffer and size come out of the param block, and the handle is that
buffer when the title supplied one. The previous reading wrote the handle over
the VBAP parameters' speaker-position pointer and left the caller's handle
uninitialized. sceAudioOut2GetSpeakerArrayCoefficients writes
uiNumCoefficients floats to rsi and the ambisonics form writes them to rdx;
both used to zero a fixed 0x400-byte slab, and the ambisonics form could pick
its own count register as the destination address.
Errors are returned in the libSceAudioOut2 error space (0x80268xxx) rather
than as generic codes a title never compares against, and
sceAudioOut2UserGetSupportedAttributes is added.
sceAudioOut2ContextQueryMemory and sceAudioOut2ContextGetQueueLevel are left
exactly as they are, so that the two changes already in flight for them do not
collide with this one; the two address-range predicates survive for those two
callers alone.
sceAgcCreateInterpolantMapping and its version 2 run once per draw. Each one
read every semantic word with its own guest read — the output semantic table
was re-read from the start for every pixel input — and then wrote the 32
SPI_PS_INPUT_CNTL (register, value) pairs as 64 separate four-byte guest
writes, each paying the full guest write path.
Both semantic tables are now read once, the register block is built on the
stack (the identity pairs first, then the computed values over them), and the
whole block goes out in a single guest write. The output table is rented from
the pool when it is larger than a kilobyte. The bit arithmetic is unchanged,
and a new test pins version 1's output to the values the per-word
implementation produced.
Measured on Dreaming Sarah in-game: 8.6 us to 1.7 us per call, which is 5.0 s
down to 1.0 s of the main thread over about two minutes.
A host shader compiler can spend tens of seconds on a single large translated
compute program, and the command stream could not advance while it did, because
the pipeline cache holds its lock across the creation call. Every other queue
that wanted any pipeline waited behind it too.
The pipeline host can now report that a compute pipeline is not ready yet. The
Vulkan presenter answers that way the first time it sees a program: it prepares
everything the command stream owns (descriptor set layout, pipeline layout,
shader module) up front and runs only vkCreateComputePipelines on a worker
thread, so the task touches nothing else. The number of compiles in flight is
bounded, since each one blocks its thread inside the host compiler and a frame
can touch many programs at once. Asking again returns the finished pipeline
without starting a second compile.
The dispatch waits for its pipeline rather than being dropped: a guest that
does not replay a one-shot dispatch deadlocks, because the dispatch can feed a
label a later packet waits on. Asking again from the executor keeps that wait
outside the pipeline cache's lock. Hosts that compile cheaply keep the blocking
behaviour through the interface default, and
SHARPEMU_ASYNC_COMPUTE_PIPELINES=0 compiles inline.
A suspend point inserts GPU commands that drain the graphics pipe. The call
that inserts it does not wait for those commands, and the hardware allows one
suspend point to be in flight, but the queue implemented it by blocking the
calling thread until every accepted submission had been processed.
The frame boundary is now a queued submission of its own, ordered behind the
graphics work it ends, and processing it flushes what the frame recorded. The
caller returns immediately for the first boundary and only waits when a second
one arrives while the first is still pending. Shutdown still drains, because no
slice will run a queued boundary once the queue stops accepting work, and a
cancelled queue drops pending boundaries along with the rest.
Measured on Octopath Traveler II: the call went from 223 ms to 72 ms, and the
guest render thread spent 38 s instead of 107 s of its 170 s inside it.
PredicationZPassWaitOp is 0 for kWaitForQueryResults and 1 for
kDoNotPredicateIfQueryResultsNotReady, so the wait bit was being read
inverted. Every SET_PREDICATION packet that asked the command processor not to
stall drained the whole host queue instead, and the packets that did ask to
wait read the predication label without waiting for the results at all, so
they could act on a stale value.
The drain now happens only for kWaitForQueryResults, and only on the branch
that actually reads the label. A packet tagged
kDoNotPredicateIfQueryResultsNotReady leaves its work unpredicated, which is
what the hardware does while the results are still outstanding, and clearing
predication reads no memory at all so it never drains.
A resource descriptor the host materializes is built by replaying the shader's
scalar operations on the host. The validator that decides whether a value can
be replayed listed only part of what the scalar evaluator implements, so a
descriptor computed through any of the others was rejected even though the
host can compute it exactly.
The list now covers every pure scalar function the evaluator supports: the
remaining integer maximum, absolute value and high multiplies, the signed and
unsigned comparisons, 64-bit equality, the bit-counting and bit-scanning
operations, the quad mask, the integer/float conversions, and the float
arithmetic and comparisons. Anything with side effects or a lane-varying
result is still rejected.
S_CODE_END pads the space after a shader's last instruction. When the code
after the final S_ENDPGM ends in a backward branch, the linear decoder walks
straight into that padding and failed with an unknown SOPP opcode 0x1F, which
stopped the session.
The decoder now names 0x1F and ends the program there, unless a forward branch
still targets code beyond it -- in which case the padding really would be
inside the program and the decode still fails.
Two things were wrong in the SOPK forms, in both the SPIR-V and the MSL
backend.
S_CMPK_*_U32 compares against SIMM16 zero-extended; only the signed compares
sign-extend it. Both translators sign-extended it for every compare, so
"s_cmpk_le_u32 s4, 0xffff" was true for any value at all. NGG geometry
programs use exactly that test on the packed vertex and primitive count, so
they took their no-primitive path and exported a single null primitive, which
left Unreal's volume passes empty. The register S_CMPK names is also only
read, yet user-data liveness recorded it as a definition, so a compare on a
user-data register left that register unloaded; it now counts as a use, as
does the register of the SOPK accumulations.
S_ADDK_I32 is defined as D.i = D.i + signext(SIMM16) with SCC set on signed
overflow, the same SCC rule as S_ADD_I32, which both backends already
implement. Neither wrote SCC for the SOPK form, so a following
S_CBRANCH_SCC0/1 or S_CSELECT read whatever the previous instruction had left
there. S_MOVK_I32 and S_MULK_I32 correctly leave SCC alone.
v_min_f32 and v_max_f32 return the other operand when one of them is NaN, and
v_cvt_u32_f32 and v_cvt_i32_f32 saturate an out-of-range value to the target
type's limit and turn NaN into zero. The GLSL FMin and FMax instructions and
SPIR-V's ConvertFToU and ConvertFToS all leave those cases undefined, so a
host driver is free to hand back the NaN or an arbitrary integer, and Metal's
fast math does return the NaN.
The minimum and maximum now select the other operand when either side is NaN,
and the two conversions clamp at the type's limits and return zero for NaN.
NaN is tested on the value's bits rather than with an ordered comparison, so
fast math cannot assume the test away.
Device tests cover this and the wave-wide maximum that Unreal's motion blur
computes with DPP row_shr, v_permlanex16_b32 and v_readlane_b32 across a
64-lane wave.
The two remaining reserved pixel system inputs were also left reading zero.
SAMPLE_COVERAGE now reads the first element of the SampleMask built-in, and
POS_FIXED_PT packs the integer pixel position taken from FragCoord, x in bits
[15:0] and y in bits [31:16], so no enabled pixel system input is left unset.
kuba
Astell
Berk