The heap was created lazily by pipeline creation and by the shader prewarm,
but a draw binds its bindless slot table before it creates its pipeline. A
title whose first draw samples images, with no pending prewarm work, hit a
null heap in BindFlattenedResourceTable.
The heap depends only on a device capability, so create it with the stores
and drop the lazy calls.
* Implement AGC GS oversubscription descriptor output
* fix(agc): drop the Yotei 0x191/0x192 register workaround
Ghost of Yotei 2.002 submitted two register descriptors whose output
sceAgcGetGsOversubscription left uninitialized, so it wrote user-config 0x191 and
shader 0x192, registers gfx10 does not define. The export now fills the descriptors
(user-config 0x260 and shader 0x81), so those writes no longer happen.
Remove the workaround that accepted and ignored both registers, together with its
logging and the two tests that pinned it. An unknown register in either table is a
fatal error again, which the export's own tests rely on.
Ghost of Yotei 2.0 starts and presents its first frame with this change and the
export from #1083.
---------
Co-authored-by: foufouadi <attar.foued@gmail.com>
* perf(images): download dropped images through the tiler ring
PublishBeforeDrop moves an image's GPU contents into guest memory before an
overlap drops it. Demon's Souls drops a few full-screen targets every frame,
and each tiled download allocated a pooled scratch the size of the image; past
the pool budget the completion callback freed it again. Creating and freeing
those buffers took about 11 % of the render thread.
The download now takes its linear copy from the same ring the uploads use,
with the copy regions moved to the ring offset. A download that needs the
BGRA16 swap keeps its own scratch, so the swap output from the ring cannot
wrap onto it. Ring downloads hold no pooled scratch, so a collection under
pressure no longer has to end the tick to bound them; the pressure test now
checks that instead.
* revert(gpu): remove unified guest buffers and mapped readbacks
#1058 placed guest buffers in device-local, host-visible memory and read
them back through the mapping; its author reverted it the same day in
#1065, and #926 brought it back. The memory-type check also matches a
discrete GPU with resizable BAR, where the CPU reads that memory uncached
across the bus: on an RTX 5070 Demon's Souls' render thread went from
0.036 ms to 0.33 ms per readback and from 11.6 ms to 19.4 ms of work per
frame, and a Ryzen 7 5800X3D dropped to 25-30 FPS.
Guest buffers are device-local again and every readback copies, as before
#1058. SHARPEMU_UNIFIED_BUFFERS and SHARPEMU_MAPPED_READBACK_KB are gone, with
the two mapped-readback tests and the two shared-page tests that only ran on
mapped buffers. Shared page shadows still adopt pages from downloads; pulling
shader bytes from a GPU copy needs a mapping and no longer happens.
* feat: run and pack ZArchive (.zar) game archives
* feat(gui): optionally delete the game folder after packing a .zar
* feat(gui): name a packed .zar after the game version
* feat(gui): explain packing in the .zar dialog
* feat(gui): name a packed .zar by title id and version
* [GUI] French: add missing Library.View strings and polish translation
- Add Library.View.Grid / Library.View.Carousel (were missing in fr.json)
- Translate Bink playback options Host/Guest and make the description match the displayed option names
- Use infinitive form for RenderDoc and crash dump descriptions, like the other options
- Use typographic apostrophes consistently
* [GUI] French: reword the Bink Ignorer description
Ignorer permet de passer les vidéos Bink reads more naturally than Ignorer saute les vidéos Bink.
---------
Co-authored-by: foufouadi <attar.foued@gmail.com>
* [ShaderCompiler] Pass guest LDS size to SPIR-V compilation
* [SpirvTranslator] Synchronize wave64 half masks through LDS exchange
* [Fiber] Avoid formatting unused names and disabled traces
* [Shader] Build materialization cache ranges without temporary copies
* [ShaderCompiler] Intern scalar graphs without string keys or hit allocations
* [Media] Vectorize movie YUV conversion without changing pixels
* [Shader] Reuse scratch storage for resource read recording
* [Tests] Initialize presenter state for GPU binding validation
* [Vulkan] Keep temporary descriptor writes off the managed heap
* [GPU] Reuse bounded descriptor preparation scratch arrays
* [ShaderCompiler] Persist versioned scalar analysis across launches
* [GPU] Remove duplicated image layouts from texture bindings
* [ShaderCompiler] Bind cached graph payloads to their input keys
* [Vulkan] Load content-addressed driver cache shards on demand
* [Tests] Keep presenter allocation fixture independent of diagnostics
* WIP: Yotei runtime descriptors, compile-cost fixes and Codex memory work before PR #1000
Snapshot before merging upstream PR #1000: runtime image descriptors (global table),
format-store and fault-flush compile fixes, runtime buffer strides, stencil plane
fixes, Codex's image memory, D16 and residency work. Tests: all pass except the
two known failures (DescriptorReloadedInsideLoop_SharesOneSampler,
RuntimeFormattedLoad_ReadsItsElementThroughThePageTable).
* [Cpu] Claim guest thread stack and TLS regions atomically
TryMapGuestThreadRegion looked for a free slot and mapped it in two steps, and Map
reuses a region that already exists. Two host threads claiming a nested-callback stack
at the same time therefore got the same stack, and Map zeroed the first one's frames.
In Ghost of Yotei two render threads ran the AGC command-buffer-full callback on that
shared stack at once: one returned through the other's entry stub onto the other
thread's host stack, abandoning managed frames. The runtime then failed fast with
"attempted to call a UnmanagedCallersOnly method from managed code" or an access
violation, one to four minutes after the intro.
The slot search and the mapping now run under one lock shared by the stack and TLS
claims.
* [Gpu] Preserve downloaded buffers and batch resource descriptor reads
* [Vulkan] Forget driver cache identities of destroyed shader modules
The remaining change of the updated #1006 head (e6659681) that the integration branch did not carry.
* [Vk] Measure GPU busy time from per-submission timestamps
Per-draw timestamps are written only with SHARPEMU_PROFILE_GPU_DRAWS=1; by
default each submission gets two timestamps and the overlapping intervals
are merged into a [PERF][GPU_BUSY] line with the busy share and the longest
submission.
* [HLE] Reserve a host core for the render thread
On Windows hosts with 8 to 64 logical processors the render thread gets the
last physical core of the fastest efficiency class at the highest priority,
and guest threads are kept off that core so spinning job workers cannot take
the renderer's SMT sibling. SHARPEMU_RENDER_CORE=0 turns it off.
* [Vk] Spin briefly before blocking on a GPU wait
A GPU wait polls the timeline for SHARPEMU_GPU_WAIT_SPIN_US microseconds
(150 by default) before it sleeps, which saves the wake-up latency when the
work completes soon.
* [GpuMemory] Answer CPU-dirty range queries without the tracker lock
MayHaveCpuDirtyPages reads the per-block summary and the region masks without
taking the region lock; a true answer still goes through the locked path.
Regions also bump a CPU-dirty epoch whenever a page becomes CPU-dirty.
* [Gpu] Skip the device-address sweep while no page became CPU-dirty
PrepareBda remembers the CPU-dirty epoch and the mapping it swept last and
skips the full sweep of mapped memory until either changes.
* [GpuMemory] Scan possibly CPU-dirty blocks 64 at a time
ForEachPossiblyCpuDirtyRange combined a region lookup and a summary read for
every 4 MiB block of mapped memory on each device-address sweep. A bitset of
blocks that have a region and the dirty summary words are now scanned a word
at a time, with the same runs as before.
* [Gpu] Skip syncing a written buffer range that is already GPU-owned
SynchronizeBuffer returns early when a written range lies entirely in the
GPU-modified ranges, uses the lock-free CPU-dirty query for reads, and the
render profile splits buffer sync time into written, texel and upload parts.
* [Gpu] Version the GPU-modified buffer ranges
A written range bumps GpuModifiedVersion only when it adds bytes to the
GPU-modified set, so readers can cache a clean verdict for a page until the
GPU writes something new.
* [Gpu] Track the extent of DCC metadata and answer overlap queries
A color target records the size of its DCC metadata, and the image cache can
tell whether a guest range overlaps registered DCC metadata.
* [Diagnostics] Trace the GPU writers of DCC metadata and of a chosen address
SHARPEMU_TRACE_DCC_WRITERS=1 writes the shader program and a call stack for
GPU buffer writes that overlap DCC metadata; SHARPEMU_TRACE_WRITER_ADDRESS
does the same for writes that cover one guest address.
* [Gpu] Fill DCC metadata on the GPU and in guest memory at once
A fill that targets DCC metadata which no GPU write owns is recorded as a GPU
fill and also written to the guest backing directly, so the range never
becomes GPU-dirty and later CPU reads need no readback. Ranges the GPU still
owns or that overlap an image keep the plain buffer fill.
* [Gpu] Turn AGC fill compute shaders into buffer fills
BoundedFillDetector recognizes the AGC range-fill and pattern-fill compute
programs, including thread-dimension dispatches, and a dispatch of one that
targets DCC metadata becomes a DCC fill instead of a compute dispatch.
Refused conversions are reported by the DCC writer trace.
* [HLE] Resolve backing aliases for readers that hold an access window
SharedBackingViews publishes its view snapshot as an alias identity, and a
reader can enter an alias access window once and resolve guest pages to their
backing alias without the views lock until the snapshot changes.
* [Diagnostics] Trace shader resource reads that wait for GPU-written memory
SHARPEMU_TRACE_GPU_READS=1 writes the shader, stage, address and table phase
of a resource read that has to synchronize GPU-written memory, once per shader
and page.
* [ShaderCompiler] Keep recent materialization variants per draw key
A cache key now holds up to four materializations in most-recently-used order,
so alternating descriptor contents reuse their earlier result instead of
re-materializing. The cache report counts stale and unreadable entries and
table refreshes, and ReadingTable tells a reader when the flattened table is
being evaluated.
* [VideoOut] Read resource words through cached clean pages and the backing alias
TryReadGuestWord and TryReadResidentGuestBytes remember the last 64 pages
found free of GPU-dirty bytes, keyed by the GPU-modified version, and read
them through the backing alias inside the alias access window that each
command stream slice now holds. GPU-dirty checks are byte-precise, and a
resident read refuses unreadable ranges before it asks the page tracker.
* [Gpu] Read GPU-written guest memory back asynchronously and ahead of time
VulkanAsyncReadback keeps eight slots: slot zero serves synchronous reads and
the others carry asynchronous downloads. A guest thread that reads GPU-written
memory lets the render thread start the copy and waits for it on its own
thread, then the render thread applies it if the range is unchanged; a
foreign read keeps the source buffer alive until it completes.
Windows the render thread had to read synchronously become hot. When a batch
writes into a hot window, the window is downloaded right after the batch is
submitted, and the next read uses that copy if it is still current. A pending
write to a hot window ends the open batch early. SHARPEMU_ASYNC_GUEST_READBACK=0
keeps the synchronous path; [PERF][ASYNC_READBACK] reports the counts.
* [ShaderCompiler] Prefetch contiguous resource words in one resident read
Resource materialization read every descriptor and flattened-table word
through its own guarded TryReadGuestWord call, about 2.6 million per five
seconds in Demon's Souls. The raw reads of each plan are grouped once; at run
time their addresses are resolved, sorted and read as contiguous runs through
the resident guest reader, and the values seed the evaluator cache. A run the
GPU may still own, or one out of bounds or through a null base, falls back to
the word reads. The cache recorder records the prefetched words, and
SHARPEMU_RESOURCE_PREFETCH=0 turns the prefetch off.
* [Gpu] Reuse image lookups that found the same backing
FindImage remembers, in a 256-slot table, each request that resolved to an
image with the same backing, keyed by the request bytes and a generation
that every index registration or removal advances. A repeated request
touches the image and returns the remembered result without walking the page
owner index. SHARPEMU_IMAGE_LOOKUP_MEMO=0 turns it off; [PERF][IMAGE_LOOKUP]
reports hits and misses.
* [ShaderCompiler] Look up fused programs without the registry lock
Every shader header lookup and decode asked the fused-program registry under
its lock. The registry now publishes an immutable dictionary that readers
take with a volatile read, and a registration copies it under the lock.
* [ShaderCompiler] Decode fused programs whose halves are far apart
The fused decoder joined the continuation at its address distance from the
entry and refused halves more than 4 GiB apart or in the wrong order, which
Poppy Playtime now hits for a vertex shader (invalid-fused-layout). Such a
continuation is placed right after the entry instead, and each of its
instructions keeps its real offset from the program address in AddressOffset;
S_GETPC_B64 in the scalar graph, the SPIR-V and the MSL translators uses that
offset, and the persisted scalar analysis key includes it.
* [Gpu] Collect an image only once it is old enough in frames
Since images age by frame, clamping the age to the current frame made every
image of the first frame collectable during the first 16 frames, and a
submission that ended no frame could delete a texture the frame still used.
Without memory pressure nothing is collected until the full age has passed;
under pressure the clamp still lets collection relieve memory early.
* [Tests] Cover the deferred global barrier of an attachment-only scope
A global barrier still ends a scope whose draws store to memory. In a scope
whose draws only write attachments it waits for the scope to end, and a later
memory-writing draw ends the scope and records it.
* [Agc] Make the number of suspend points in flight configurable
CommandStreamQueue takes the number of frame boundaries that may wait behind
the graphics commands. The default stays one, so the guest CPU can run one
frame ahead, and SHARPEMU_SUSPEND_POINTS_IN_FLIGHT=0 makes a suspend point
wait until its own boundary is processed, for diagnosis. Tests cover both
modes, including a label that the CPU rewrites after the suspend point.
* [Gpu] Throttle the device-address fault and 3D slice range logs
Little Nightmares wrote 195 thousand "Accessed non-GPU cached memory" lines in
one session and Silent Hill 6800 slice range lines. Both now print the first
occurrences and then only at power-of-two counts, with the count.
* [Docs] Document the new environment variables
Adds SHARPEMU_ASYNC_GUEST_READBACK, SHARPEMU_GPU_WAIT_SPIN_US,
SHARPEMU_IMAGE_LOOKUP_MEMO, SHARPEMU_RESOURCE_PREFETCH,
SHARPEMU_SUSPEND_POINTS_IN_FLIGHT, SHARPEMU_RENDER_CORE,
SHARPEMU_TRACE_DCC_WRITERS, SHARPEMU_TRACE_GPU_READS,
SHARPEMU_TRACE_WRITER_ADDRESS and SHARPEMU_PROFILE_GPU_DRAWS, and corrects the
group counts to the rows the tables hold.
* [GpuCommands] Write the 64-bit label of a compute-done release with an interrupt
RELEASE_MEM with event 0x2F and a 64-bit data selection was accepted only
without an interrupt. It is now handled like the other end-of-pipe events:
the label is written and the interrupt is raised after it.
* [GpuCommands] Let a wait see its own queue's preceding label store
A memory wait that directly follows a 4- or 8-byte store of its queue to the
same address, with only waits or inert NOPs and markers between them, compares
against the stored value when guest memory no longer holds it. Dead Cells
writes a label and waits for it while the CPU already rewrites it for the next
frame. The forwarded value lives only within one slice, and any draw, dispatch
or transfer in between drops it.
* [ShaderCompiler] Key fused programs by the underlying guest memory
Each native guest thread wraps the guest memory in its own tracking wrapper,
so a fused program registered while one thread created the shader was not
found by the render context. The registry now unwraps ICpuMemoryWrapper
before it looks up the per-memory table.
* [Agc] Implement the legacy sceAgcFuseShaderHalves export
The older NID (nApJjpKNBl4) shares the fusing path but keeps the front half's
user data pointer, takes the larger value of the resource allocation fields
of both halves, takes the user SGPR count and the ES component flag from the
front half, and copies the checksum register of the matching stage.
* [ShaderCompiler] Decode S_CMOV_B64 as a conditional select
S_CMOV_B64 is decoded as S_CSELECT_B64 with the old destination pair as the
false source, so it reuses the SCC, register-pair and EXEC handling of the
translators and of the scalar value graph.
* [Gpu] Merge the stencil replacement references of a face
Vulkan has one stencil reference per face for the comparison and every
replace operation. Replacements with the test value and with the operation
value are now merged into that reference over the written bits, and refused
only when they conflict on bits the comparison also reads. A replacement
with an operation value of zero over the written bits becomes a zero
operation, which ASTRO's PLAYROOM uses to clear one stencil bit.
* [Json] Add the sce::Json::Value copy constructor and keep shadow values on copies
The copy constructor (fSb2oQTNrgA) shares the assignment path. Assignment
and reads now carry the value a constructor or setter recorded in the object
heap, so a copy keeps it after the source is destroyed.
* [Font] Implement sceFontTextSourceInit
Fills the 0x60-byte text source with the text range, the cursor, the parser
callback and its object, and the text size, and leaves the memory after it
untouched.
* [AudioIn] Answer sceAudioInGetSilentState without an input backend
No AudioIn port can be opened until an input backend exists, so the export
returns an AudioIn error code (0x80260101) instead of being an unresolved
import.
* [PhtreadSpecificSemanticsTests] Fixes for computed shaders
* [Gpu] Return idle buffer memory and bound retired memory under pressure
- Small-buffer slab chunks track their live blocks; completely idle chunks go back to
Vulkan under memory pressure instead of staying allocated until device teardown.
- Buffers removed from lookup still hold GPU memory until their retirement completes.
The cache now counts them, collects on registered plus retired bytes, and waits for
completion once retired memory passes 256 MiB.
- The tiler's pooled scratch buffers are released under pressure, and image collection
waits for completion when outstanding scratch is over budget.
* fix(shader): preserve wave64 ballot masks on Vulkan
* test(shader): cover subgroup lane bits in wave mask predicates
* [Pad] Give each new touchpad contact its own touch id
The pad assigns a new id (1 to 127, then back to 1) to every finger contact and keeps it
while the finger stays down. SharpEmu reported the finger slot instead, so lifting the
finger and touching elsewhere read as one contact jumping across the pad. Games that tell
strokes apart by the id (Ghost of Yotei's name drawing) never saw a new stroke.
* [Gpu] Stop device-address faults from churning the buffer cache
Shaders reach guest memory through pointers, and a page they touch outside the cache
faults and is registered afterwards. In Ghost of Yotei that path re-created tens of
thousands of buffers every few minutes:
- Faulted ranges outside the GPU mappings were never touched again, so they aged out
while still in use and faulted again. They are now kept and touched each tick, and
forgotten once their guest mapping is gone.
- A fault inside a guest mapping now brings in its aligned 2 MiB window, clipped to the
mapping, instead of one 16 KiB page.
- Faults on memory the guest never mapped (bad pointers) no longer create buffers: the
shader reads zeros there, as on the console. They were 98% of the faults and about
2.6 GiB of junk buffers; the cache now holds ~0.5 GiB in the same scene.
- The per-fault log line is gone; it wrote hundreds of thousands of lines per session.
* [Vulkan] Save and load driver pipeline caches past 2 GiB
Ghost of Yotei's driver cache grows past 2 GiB. Saving went through a .NET array, so
every save failed and new pipelines were lost at the next launch; loading had the same
limit. Both now stream the unchanged file format from and to native memory in chunks,
so existing caches still load.
* [Np] Treat NO_ENTITLEMENT for unowned add-on content as an expected result
sceNpEntitlementAccessGetAddcontEntitlementInfo returns NO_ENTITLEMENT when the user does
not own the add-on, as on the console. The import log reported it as a warning.
* [Vulkan] Log only slow pipeline creations
Every pipeline creation was logged, cache hits included, which wrote thousands of
0.0 ms lines per session. Only creations of 500 ms or more are logged now.
* [Shader] Treat a phi that merges one value as that value when planning table reads
* [Test] Read the formatted-load V# from GPU memory so the load goes through the page table
* [Repo] Add license headers to new files and build pipebench with central package versions
* [Test] Add the missing copyright line to the large dispatcher tests
* [Repo] Remove game-specific RenderDoc capture scripts
* [Repo] Remove the pipebench tool
* [Kernel] Implement sceKernelAioPollRequest
* [ShaderCompiler] Return the entry header address of a registered fused program
An overload of TryGetFusedProgramParts also gives the entry header, so a
caller can register the same fused program on another memory.
* [Vulkan] Keep saving a pipeline cache that outgrows its limit
A driver cache over 256 MiB was refused on every save, so it was never
written again and every launch compiled the same pipelines. Such a cache
is now written as an empty file once, and the next launch starts a fresh
one. A save is skipped when the driver reports the size that was loaded
or last saved, and a failed or deferred save waits for the next interval
instead of running again on every new pipeline.
* [Vulkan] Write only the cache shards that gained pipelines
A shard remembers the driver size it was loaded or saved with and is not
written again while that size holds. Shard saves are logged as one
summary line, and a shard load only when it is slow.
* [Vulkan] Remove the single-file cache once the shard mode takes over
The shard mode never reads the single cache file of the default path, so
it is deleted instead of staying on disk.
* [Gpu] Record compiled compute programs in a shader prewarm list
When the pipeline host gives a list, each compute permutation the game
compiles is appended at once to shader-prewarm.bin: the guest code the
decoder read (once per program), the fused parts, the compute input, the
system registers, the resource specialization and the push data start.
Records carry a checksum, so a cut tail is dropped at the next open.
TryCompilePrewarm rebuilds the compile request from a record through the
same layout and request code as the runtime, so it produces the same
SPIR-V.
* [Vulkan] Prewarm the recorded compute pipelines after a translator or driver change
A first run after a translator change compiled every compute pipeline
one at a time while the render thread waited (Demon's Souls: 282
pipelines, 144 s). When the translator assemblies or the driver differ
from shader-prewarm.stamp, background threads at below-normal priority
translate the listed programs again and create their pipelines, so the
driver has them before the game asks. With the same build and driver the
launch does no prewarm work. The log reports the prewarm result and how
many runtime compute modules it produced.
* [Vulkan] Keep the pipeline cache in memory on Windows
The GPU driver keeps its own shader cache on disk, and the shader prewarm
list rebuilds it after an update, so Windows no longer writes the driver
cache file by default. SHARPEMU_VK_PIPELINE_CACHE=file keeps one; Linux
keeps the file and macOS keeps its shards.
* [Docs] Document the pipeline cache modes and the shader prewarm list
Adds SHARPEMU_SHADER_PREWARM, SHARPEMU_VK_PIPELINE_CACHE_SHARDS and
SHARPEMU_VK_PIPELINE_CACHE_LEGACY_SHARDS, and describes the new cache
file default.
* [DeS] Fixes for page faults and stack bounds issues in the emulator and audio fixes
* [ShaderPrewarm] Hold the game until the prewarm finishes and resume interrupted runs
The prewarm compiled thousands of compute pipelines on background threads
while the now faster boot ran beside it, so logos, Bink movies and the first
scenes raced the compile and dropped frames.
- Hold the guest at its first splash hide, GPU submission or CPU flip until
the prewarm is decided and done; the render thread keeps presenting the
splash. SHARPEMU_SHADER_PREWARM_WAIT=0 opts out.
- Stamp only the shader translators, the compiler type and the driver, so a
Libs-only build no longer triggers a full prewarm.
- Record finished pipelines in shader-prewarm.progress so an interrupted
prewarm resumes with the remaining ones.
* [AudioOut2] Block a blocking push only until a queue slot frees
A blocking sceAudioOut2ContextPush waited only when the host stream hit its
128 KiB cap, so a title rendering faster than real time kept about 680 ms
queued and every sound played that far behind (GTA V: queue=2 context,
queued_ms avg=678). The push now waits until the device holds the cushion
plus QueueDepth-1 grains, the level GetQueueLevel already reports as full;
the host cap stays as the limit for non-blocking bursts. GTA V now holds
about 41 ms with no underruns.
* [ShaderCompiler] Share the device-address formatted load decoder per component count
Every BUFFER_LOAD_FORMAT through a device address inlined the whole page-table
walk, format conversion and bounds check. Shaders with many such loads grew to
several megabytes of SPIR-V; 128 D16 loads added about 4600 words each.
Emit one DontInline function per component count, taking the address, size,
offset, descriptor and caller PC. Fault recording, formats, swizzles, EXEC
and out-of-range values are unchanged.
Offline A/B on three extracted shaders (RTX 5070, driver cache disabled):
18.6 s -> 2.9 s, 12.9 s -> 2.8 s, 15.9 s -> 3.2 s.
* [Gpu] Never copy a linear image readback past its guest range
The readback storage is sized from the guest range, but the copy follows the
linear layout computed for the format, whose padded rows can need more
bytes. A 513x513 R8 image declared as 0x40401 bytes was copied with
768-byte rows (0x60201 bytes): the copy wrote past its slot in the
download ring, and at the end of the ring into freed memory, which lost
the device.
Plan the readback only when the copy fits the guest range (and the
partial-level readback only when it fits its covered bytes), with one
warning naming the image. A linear download that would still exceed its
destination now fails explicitly instead of corrupting GPU memory.
* [Gpu] Draw merged LS/HS tessellation through a native tessellator
The guest hull runs unchanged in a cooperative wave64 compute workgroup and
computes its own control points and tessellation factors. A generated
control stage feeds those factors to Vulkan's tessellator, and the guest
domain shader is translated to a tessellation evaluation stage that reads
the guest control-point ring.
- VGT_TF_PARAM gives the domain, spacing and winding; VGT_LS_HS_CONFIG
gives the patches per group and the control-point counts.
- Factors are clamped by the guest's minimum and maximum levels; zero or
NaN outer factors still cull the patch. Isoline factors are swapped
into Vulkan's order.
- The SPI_SHADER_USER_DATA_ADDR_HS pair is carried separately from the LS
scalars and moved to s0:s1 before the guest code runs.
- Patches are drawn one hull group at a time, because the guest rings
are reused by relative patch id.
- A configuration the bridge cannot express (power-of-two partitioning,
an undefined domain, an empty hull group) skips the draw with one
warning instead of stopping the emulator.
* [VideoOut] Verify a guest page once for clean resource reads
Resource materialization reads a few hundred guest words per draw through
the clean readers, and each read re-checked the whole GPU ownership of its
bytes: GPU-dirty pages and bytes, then the image cache, which built a new
list (twice per word) under its lock. In Ghost of Yotei that was about 550
million reads and 90 GB of allocations per minute in game.
- The image cache answers whether an unsynchronized GPU image covers the
bytes without building the list; the list is built only when one does.
- The clean-page memo, already used by plain reads, now serves clean reads
too. A page is verified clean once (no GPU-dirty bytes or pages, no
GPU-modified image) and trusted until a buffer or an image is written by
the GPU, which advances the buffer cache's or the images' write version.
GPU-dirty pages are still checked on every read.
Measured in game: one materialization went from 0.33 ms to 0.17 ms.
* [Gpu] Keep the guest's face culling on tessellated patches
A patch list stands for legacy rectangle lists, which are never culled, so
every patch-list draw dropped its culling. Tessellated draws use the patch
topology too, and their output rasterizes as ordinary triangles: back
faces were drawn. Only a patch list without tessellation stages is a
rectangle list now.
* [ShaderCompiler] Give the domain shader its patch ID in v8
The ES ABI passes u, v, the patch within its hull group and the patch ID in
v5..v8. v8 was never written. Each native draw covers one hull group, so
the patch ID is the group's first patch plus the native primitive ID.
* [Gpu] Run tessellation hull groups together in their own off-chip slots
The hardware gives each hull threadgroup in flight its own off-chip buffer
and its own run of the factor ring, so groups run together and one draw
consumes them. Every group used slot zero instead, which serialized the
draw into one dispatch, one draw and full barriers per group.
- sceAgcDriverSetHsOffchipParam (VGT_HS_OFFCHIP_PARAM) is kept: buffer
count and granularity. Ghost of Yotei sets 256 buffers of 8K dwords.
- Hull workgroup k runs group k: its first patch, its patch count, s2 at
slot k and s4 at group k's run of the factor ring.
- The domain shader finds its group from the native primitive ID: the
off-chip slot in s4, the patch within the group in v7, the patch ID in
v8.
- One dispatch and one draw cover as many groups as the off-chip buffers
and the factor ring hold. Without the guest's configuration, one group
runs at a time as before.
* [Tests] Restore two upstream tests the branch dropped without a behaviour change
DepthState_PlayroomReplacementClearsOnlyTheWrittenBit and
FormattedUntypedLoad_SpecializesOnTheDescriptorFormat were removed by
earlier branch work although the code still behaves as they expect; both
pass unchanged.
* [Tests] Check the AudioOut2 port volume and reserved bytes again
The layout change dropped the checks of the -1 volume at +4 and the zeroed
bytes at +8, which the export still writes.
* [Repo] Remove the branch's debugging switches
The capture-after-seconds RenderDoc request, the crash-capture directory and
first-chance fault traces, the unsupported shader dump, the AGC allocation
failure traces and the control-flow trace served earlier investigations
only. The configuration switches (direct memory size, image cache budget,
forcing the shader dispatcher) stay.
* [PlayGo] Do not reject chunk IDs the playgo.pgm numbering only assumes
playgo.pgm is read for its chunk count only; the IDs 0..count-1 are
derived, yet they were marked authoritative and any other ID returned
BAD_CHUNK_ID. The list is still reported, but like other metadata that
is not understood, it no longer rejects an ID outside it.
* [Vulkan] Bound the bindless sampler binding by the device's sampler limits
The persistent heap always declared 4096 update-after-bind samplers and
left them out of the pool-wide descriptor budget it split between sampled
and storage images. On a device with fewer samplers per stage, or a tight
update-after-bind total (MoltenVK, some Intel drivers), creating the heap
could fail.
The sampler binding now takes the smallest of the device's per-stage and
per-set sampler limits (at most 4096), and its share comes out of the
total before the images split the rest.
* [Gpu] Copy linear render targets with the guest's own pitch
A linear render target's builder lays it out with the guest's pitch, which
need not be aligned like a texture's: a 513x513 R8 target is 513 bytes per
row. Transfers recomputed the texture's aligned linear layout (768-byte
rows), so its uploads read shifted rows and its readbacks did not fit the
guest range and were skipped.
A single-level, non-volume linear image whose description holds a
consistent slice layout now transfers with that layout. Textures, which
carry no layout of their own, keep the format's linear layout, and the
readback guard still refuses a copy that does not fit.
* [Gpu] Free images and retry once when an image allocation runs out of memory
The image budget comes from the device's reported memory, so an allocation
can still fail when the driver or another process holds more than
expected. That was fatal.
Running out of device or host memory while allocating an image's backing
is now recoverable: the cache retires every image collection allows, waits
for the retired ones to be destroyed, returns unused pool memory and tries
once more. A second failure stays fatal. Tests make an allocation fail
through an internal hook on GpuDeviceInfo.
* [Vulkan] Size descriptor pools for the sets that need them
The branch had raised every descriptor pool to 65536 storage buffers,
262144 sampled images and 8192 storage images so the largest shader's set
would fit, and every pool paid for that on every device.
Pools are back to upstream's ordinary size (8192/8192/1024/1024). When a
set does not fit one, the next pool is sized for four sets of its kind,
and a retired pool is reused only if it can hold the set. MaxImages stays
the planner's bound on distinct image descriptors per shader.
* [Vulkan] Blend with a second source from the MRT1 export
Blend factors 15-18 (SRC1 color and alpha) were mapped to Vulkan's Src1
factors without enabling dualSrcBlend or giving the fragment shader a
second output, so the blend read an undefined value and validation
reported every such pipeline.
When the device supports dualSrcBlend, it is enabled, and a draw whose
target 0 blends with an SRC1 factor (and binds no other target, as on the
hardware) gets a second fragment output fed by the MRT1 export, declared at
location 0 with index 1. The translator accepts that one extra output and
keys the compiled program by it.
* [Tests] Skip device tests without a Vulkan device, as upstream does
The branch had turned the device gates into assertions, so every device test
failed on runners without a GPU. SHARPEMU_TEST_REQUIRE_DEVICE=1 still makes a
missing device a failure.
* [Gpu] Wait for pending image readbacks before uploading over them
A tiled image retired under pressure publishes its contents only when its
readback tick completes. Found again before then, it was rebuilt from the
guest bytes it supersedes. Uploads now wait for the readbacks into their range.
* [Shader] Unpack compressed 16-bit pixel exports by their export format
Compressed exports were always unpacked as half floats. UNORM16, SNORM16,
UINT16 and SINT16 exports (G-buffer normals among them) now follow the
SPI_SHADER_COL_FORMAT of their target, which also keys the compiled program.
* [Agc] Emit direct draws for every draw modifier
sceAgcDcbDrawIndex and sceAgcDcbDrawIndexAuto returned without writing the
draw for any modifier other than 0x40000000, so whole passes vanished
without a trace.
* [Agc] Emit addressed occlusion counter events
sceAgcDcbEventWrite refused every event with an address, so the 0x38/0x39
counter events never reached the command stream. Their results are now
published through the ordered buffer fill path instead of CPU memory only.
* [Gpu] Execute target-less draws whose pixel shader writes memory
A draw without color or depth targets was dropped before compilation even
when its pixel shader stores to buffers or images. Yotei fills its visibility
buffer this way; the dropped pass left most of the scene black.
* [Shader] Keep EXEC guards after V_CMPX with a legacy scalar destination
GFX10 V_CMPX in VOP3 or SDWA form changes EXEC and ignores its scalar
destination. The EXEC-full analysis took it as a write of that destination,
kept EXEC full and elided the guards of the following vector writes.
Ported from voidborn-astra/sharperemu 8499b905.
* [Vulkan] Use unsigned operations for scalar pair bit instructions
Ported from voidborn-astra/sharperemu c0cff157.
* [Gpu] Invalidate image copies for raw buffer writes
Ported from voidborn-astra/sharperemu caa4149d.
* [Gpu] Retain GPU image contents during collection
A GPU-written image whose readback a dirty buffer overlap blocks was cleared
and deleted, losing its contents. It is kept instead; same-page CPU writes are
resolved by hash before deciding.
Ported from voidborn-astra/sharperemu 3b5eb2f3.
* [Shader] Read the selected lane for V_READLANE in graphics stages
Graphics stages took the reading lane's own value instead of the selected
lane. A graphics wave is one group of up to 32 host lanes; the selected lane
of that group is now read with a subgroup shuffle.
* [Tests] Cover Yotei's wave64 march and waterfall loop shapes
A step counter compared into EXEC with lanes leaving through VCC, and a
waterfall over an SDWA compare into an SGPR pair. Both already end every lane
in both halves of the wave.
* [Vulkan] Make depth bounds dynamic pipeline state
The depth bounds were part of the graphics pipeline key. Ghost of Yotei moves
them per draw, so every value created a new pipeline: about 80 per frame and
thousands kept for the session. They are now set with vkCmdSetDepthBounds per
draw; the key bytes they used stay reserved.
* [Shader] Read absent graphics lanes as the reading lane
A host subgroup of a graphics stage can be partly empty, where a PS5 wave
still has dead lanes. DPP, V_READLANE and V_PERMLANE reads of an absent lane
returned undefined values: Ghost of Yotei's wave minimum then never reached its
exit and its pixel shaders ran until the step guard. An absent lane now reads
the calling lane's value, which keeps min/max/and/or reductions exact.
* [Memory] Skip the ownership transition for pages already CPU-dirty
A CPU write to pages that are all CPU-dirty re-ran ChangeState, which scans the
whole 4 MiB region and republishes the same summary, for every managed guest
store. CPU and GPU dirty states are exclusive, so nothing can change there.
* [GPU] Keep the previous frame's images through mid-frame collection
Collection runs before every draw while image memory is under pressure. It
evicted every image not used yet in the current frame, which the rest of the
frame samples again, so Ghost of Yotei recreated and uploaded about 7000
images per frame. Below the critical level the mid-frame pass now leaves
them alone; frame-end collection and allocation-time reclaim are unchanged.
* [GPU] Copy the newer GPU contents between same-layout image aliases
One surface read through two formats of the same layout is two cache
images over the same guest bytes. Before one is used as a texture, storage
image or target, it copies the newest GPU contents of its same-layout
aliases on the GPU, ordered by write sequence, as unified memory shows
them. Depth keeps its reinterpretation path.
* fix(kernel): provide read-only entropy device descriptors
(cherry picked from commit 2ad0eca819ba8456b42177d4e682915d99fb7639)
* fix(net): track epoll descriptor creation and destruction
(cherry picked from commit e1c5658ed5f9579756111c89943b136fb995182b)
* fix(pthread): implement relative timed mutex acquisition
(cherry picked from commit c75f28f489a7fa3f78d6baae533e49c33dc10cb7)
* fix(net): implement level-triggered socket epoll waits
(cherry picked from commit 2f27f0078888007861f71afc5615e5c9a7246532)
* fix(kernel): release timed-out event flag waiters
(cherry picked from commit 4daa8a34e4c0534d3a0ebcc79f7a15597aaed330)
* fix(system-service): report empty event polls
(cherry picked from commit 65fb9aae7a7ee994ce8ab5b70f8fb00df2f7c742)
* fix(videoout): preserve decoded frame presentation order
(cherry picked from commit 66526f0eedc4da4253a6650b5563c257ff5c278d)
* fix(ajm): decode ATRAC9 jobs from split buffers
(cherry picked from commit e8c9925d86eb886d8734913a0ba7c10c125cc209)
* fix(videodec): finish queued frames before decoder deletion
(cherry picked from commit eae19e7c170998af3f4cf865759536fb33886e28)
* fix(net): confirm socket errors before reporting epoll error readiness
(cherry picked from commit 95128f67d446b4f659692584d241bdc85a5a9af7)
* fix(net): format guest Ethernet addresses
(cherry picked from commit 08cca779b5c000b46322ce16eb75b9739f312175)
* fix(net): keep one sceNetEtherNtostr export
Both branches added sceNetEtherNtostr. Keep the version that reports an
unreadable source address as a fault, the errno a guest read fault gives.
* fix(net): distinguish pending accepts from socket hangups
(cherry picked from commit 28ad0c0a9d989ad8f368d471354ede295ce9acf7)
* fix(net): avoid querying payload bytes on listening sockets
(cherry picked from commit 2aac142b2fb58b4b398bb1f2dfdf96cd94a388f1)
* fix(audio): avoid pacing empty nonblocking context pushes
A nonblocking push returns at once. The grain it claims still counts in the
queue level when no host backend is attached; only a blocking push waits
for its slot.
(cherry picked from commit 9b2db6d814f89b44cf46da11c3f8ced474ace063,
adapted to keep the queued grain accounted)
* chore(net): remove an investigation trace call
(network part of 8d0bd2fc; its PlayGo trace only exists in the skipped d342dd1c)
* fix(videodec2): preserve the picture-info size prefix
Without a picture only the ready flag at offset 8 is cleared; the caller's
size prefix and the other fields are preserved. A decoded picture still
fills the whole output info, pitch included.
(cherry picked from commit 050e9c824ad4a1ac2d9a40a6d6dd9ba1f63fd4fb,
adapted to keep the complete output info for decoded pictures)
* fix(videodec): preserve adjacent picture-info fields
Width and height are 32-bit fields with the pitch between them.
(cherry picked from commit 88a79715; the complete output info this branch
writes for a decoded picture already used 32-bit fields, so only the
dimensions helper and its test are added)
* fix(shader): report descriptor accesses that fall back to the null descriptor
A control-dependent image or sampler descriptor with no plan-time source
reads the null descriptor, so its texels read as zero. The plan records
each such access and the host warns once per shader with its PC, so the
loss is visible instead of silent.
* fix(shader): read control-dependent descriptors at run time before the null fallback
A sampled float access whose image or sampler descriptor has no plan-time
source read the null descriptor even when the runtime descriptor table
could serve it, because the null fallback ran before the runtime path was
tried. Such an access now reads its descriptors at run time, as the
hardware reads them from memory; the null fallback stays for accesses the
table cannot serve yet.
* fix(gpu): keep invalid runtime image descriptors on the null texture
A shader reading its descriptors at run time reports every image it does not
find. One with a null base or a format without a host equivalent stopped
the emulator when the host tried to create it. It now stays out of the
table, so the shader keeps reading the null texture, as the hardware does
for such a descriptor.
* fix(gpu): translate equivalent stencil replacement states
(cherry picked from commit a6e7922dbede9548fe5ef8795d29387de98b1bf8)
* fix(gpu): accept GFX10 image resource-level bit
(cherry picked from commit 1cbafc0a52e44592e34c5ae708d98573f6d2e9d6)
* fix(gpu): preserve addressed texture view alignment
(cherry picked from commit 936d873f70922337b7e9557cf4e579f4ed73d471)
* test: keep only the tests that 1cbafc0a and 936d873f add
Resolving their test conflicts kept test blocks from later #1057 commits
that depend on code not merged yet; only each commit's own test remains.
* fix(gpu): read disjoint descriptor bytes through backing aliases
GPU ownership is tracked in bytes but page protection covers whole pages:
CPU-owned bytes on a page that also holds GPU-owned bytes are read through
the backing alias, without bringing the neighbouring GPU bytes back.
(cherry picked from commit 7c8f0b97; this branch's clean word reader, which
first synchronises GPU-owned words, is kept, and the test keeps the
resident-range assertions)
* fix(gpu): read indirect tessellation draw arguments on the CPU
Non-indexed indirect draws now leave their arguments to the GPU (#1036), but the
merged tessellation path counts patches from the vertex count on the CPU: an indirect
tessellation draw arrived with the placeholder count of one and drew no patch.
* fix(gpu): write a replaced stencil plane back before dropping its proxy
A depth buffer whose stencil plane has another size at the same address drops the
other buffer's stencil proxy. When the GPU had written that plane, deleting the
proxy stopped the emulator (Ghost of Tsushima). The plane now reaches guest memory
first, as when a depth buffer changes planes, and stencil write-backs register as
publishing downloads so an upload of the range waits for them.
* feat(shader): read image-load descriptors at run time
An ImageLoad whose descriptor has no plan-time source looks its words up in the
runtime descriptor table instead of reading the null descriptor. The key carries the
numeric class the descriptor format selects, so the shader fetches through the uint,
sint or float heap array the host view matches.
- Only active lanes record misses; an inactive lane may hold anything.
- A missed descriptor over unbacked memory stays out of the table, and an entry whose
memory the title freed leaves it when its image is retired.
- An entry keeps the texture it resolved to until the cache retires its image, so a
draw no longer looks every entry up again; each draw still acquires its views.
- A plan that reads descriptors at run time on a host without the bindless heap is a
rejected plan, not a fatal layout error.
* fix(shader): resolve descriptors from empty scalar buffers
Descriptor words read from an empty scalar buffer are zero whatever the offset, so
the access reads the null descriptor; materialization rechecks that the buffer is
still empty. Ghost of Tsushima's pixel shaders read image descriptors this way.
(cherry picked from commit 5d03a71aacdead6adfbc9f46b5eeb777168e759a; on this branch
an access the runtime descriptor table can read keeps that path, which also serves a
buffer that is not empty, so the proof applies to the other accesses only. Its tests
use a depth-compare sample accordingly, the indirect-image case stays a runtime read,
and the planning-dump fixture adds one to its descriptor as f73663e8 does upstream)
* fix(shader): preserve scalar sources in immediate comparisons
(cherry picked from commit 9f4e325df34d9c075d9625da9994ecccfa9d1ac3)
* fix(shader): count unsigned leading bits in scalar instructions
(cherry picked from commit 09e9e1de7a8de12d92de39d51eed744ff4416fa6)
* fix(shader): bind signed integer storage images
(cherry picked from commit 1c134311d66ea6690a4549dff933017e1e478c5a)
* fix(shader): view only 32-bit signed storage writes as unsigned
A write-only signed storage image went through an unsigned view for every signed
format while no signed storage binding existed. With signed bindings (1c134311),
only the 32-bit format keeps that view, where the bits are the same; a 16-bit signed
image stores through its own signed view again.
* feat(shader): read adjusted-sample descriptors at run time
The IMAGE_SAMPLE_*_A forms carry their adjustment in the sampler's reserved dword-3
bits. Their descriptors without a plan-time source now go through the runtime table
like a plain sample, with those bits masked out of the sampler key so each
adjustment does not register a sampler of its own. A Ghost of Tsushima pixel shader
reads the image of such a sample from a material record that the plan cannot prove.
* fix(shader): qualify single-mode sample interpolants
(cherry picked from commit 3026bb288124f001f828f550755a2e4117c67ec0)
* fix(gpu): preserve sample masks and decode signed locations
(cherry picked from commit 3ab11b99d98482694037365c4e3865839544d891)
* fix(gpu): distinguish attachment sample counts in pipeline keys
(cherry picked from commit 51dd915d42f2ec877df970b66e0f7d2fc57b1300)
* fix(gpu): preserve EQAA depth and overrasterization controls
(cherry picked from commit 1087b9fba3914d3afda21041897c223dda3f324d)
* fix(gpu): retain scan mode control for sample iteration
(cherry picked from commit c8614eae54bc66d18ee60ac594280b9659b1fdd4)
* test(gpu): verify captured mixed-sample register decoding
(cherry picked from commit 943fc08e17f11dbe0995849813900a666781d42c)
* feat(shader): support explicit pixel interpolation samples
(cherry picked from commit fa66d55425a6108760f5dc1f7d6d5e9f62c2b8d9)
* fix(shader): honor eligible early depth testing in Vulkan
(cherry picked from commit 9b34fa404eaca555170e9c5fb18622a907b70179)
* fix(vulkan): preserve pixel sample-mask exports
(cherry picked from commit 5fbe6a3b38c1724c8c66fcbb6eb37aacdde4a981)
* fix(vulkan): preserve pixel depth exports
(cherry picked from commit bad8e123e566cf0a1a835fd5fa8736625ffa9581)
* fix(gpu): separate pixel iteration rate from interpolation inputs
(cherry picked from commit 9e0bdd4cea8105243347772328ec69a0a77e7b11)
* fix(vulkan): interpolate fixed custom samples with explicit offsets
(cherry picked from commit 8e7796636bed4fad5d5c65ce04a35b970fc369be)
* fix(vulkan): gate fragment execution with post-depth sample coverage
(cherry picked from commit 4c981e8edfae91b0a9ecdc336a2813ec841941f6)
* fix(gpu): apply sample exclusion to early fragment execution
(cherry picked from commit 0f5b5a71c54983db73d5d461088f3285cf13641a)
* fix(vulkan): preserve per-sample custom interpolation coordinates
(cherry picked from commit bfb5d7618b7ec994f2197c8f50d8b717f3c6bca6)
* fix(gpu): preserve late depth updates for shader coverage changes
(cherry picked from commit 4f8bcb6d142a7352a58d02328d798b390f244ee5)
* fix(gpu): key shader programs by custom sample positions
(cherry picked from commit dd36fcea9b83fd90ac1aea5af770cfdebbe5f987)
* fix(vulkan): preserve two-sample depth with single-sample color
(cherry picked from commit 042e5c4d19c5b104f0c20b0a38d87cb349d9241d)
* fix(gpu): reject unrepresented two-sample coverage masks
(cherry picked from commit b8fe958ac343b049295392a61c99419115020e63)
* fix(gpu): preserve forced shader depth ordering
(cherry picked from commit 58a4b282c89b2f35d04fe46a3f702c990f5164f0)
* fix(gpu): use late depth for unforced shader depth exports
(cherry picked from commit 29fe184bf1d241af6e8bfa521020d38d2eaba589)
* fix(shader): reject unsupported multisample sampling modules
(cherry picked from commit e77451e273d12e9c11355004654d592235e6aa63)
* fix(gpu): keep runtime descriptors that describe no view out of the table
A T# read at run time holds whatever the lane computed; the hardware reads one that
describes no valid view as the null texture. Ghost of Tsushima samples through such a
descriptor (image type 0, base level past the last level) before choosing another
one. The runtime path now asks for the texture without stopping on it: the T# checks
of the texture builder go through one rejection, fatal on the planned path as before
and a failure for the runtime one, which keeps that descriptor out of the table.
* perf(loader): sample repeated unresolved import warnings
(cherry picked from commit 842c3eea5228f017f26941e2f5ad9f69e678c002)
* perf(shader): read buffer-table descriptors at run time when the access allows it
A descriptor read from a table in a buffer, indexed by a lane value, was planned as
a dense table: materialization takes every record the buffer holds and each draw
finds, refreshes and binds all of them. A Ghost of Tsushima material table made that
about 2000 images per draw, nearly all of the render thread's time. An access the
runtime descriptor table can read now takes that path and binds only the records the
shader reads; a storage, compare or cube access keeps the dense table.
* fix(shader): read sample 0 for samples of a multisample image
Vulkan cannot sample a multisample image, and e77451e2 stopped such a module from
compiling, which stopped the emulator at a Ghost of Tsushima gameplay pixel shader.
A sample instruction on a 2D multisample image now fetches sample 0 of the texel its
normalized coordinate falls in, as shadPS4 lowers it, and the image is bound without
a sampler. A gather or an arrayed multisample sample is still refused.
* fix(gpu): free images and retry once when a buffer allocation runs out of memory
Images already retried after freeing what can go; a buffer allocation that found
every memory type full stopped the emulator (Ghost of Tsushima gameplay). The buffer
allocation now reports the failure, the buffer cache asks the image cache to free its
images and wait for retired ones, and tries once more before stopping.
* fix(vulkan): initialize format buffer scratch for runtime resources
* fix(shader): invalidate stale prewarm resource plans
* chore(sync): carry upstream Unity compatibility fixes
Apply upstream/main's post-revert Unity compatibility changes while preserving the Yotei-specific AudioOut2 state layout. The upstream rendering reversion is intentionally not imported because this branch still depends on those rendering changes.
* fix(shader): bind incompatible-only indirect image tables as typed nulls
An indirect image table whose candidates all have a dimension the MIMG
instruction cannot address (Demon's Souls 0x9D1D8F5B7F200DE7: every
candidate 2D, instruction MSAA) failed with "no typed candidate". Keep
the instruction's declared image type and bind every candidate as a
typed null instead.
* debug(gpu): dump the whole command buffer on an unknown packet
Temporary diagnostic for a tester's Yōtei crash ("packet opcode is
unknown", header 0x00000000). The old dump showed only 24 dwords around
the bad packet, too few to re-parse the stream and find which earlier
packet is misaligned. Dump from the start of the buffer instead.
* perf(gpu): copy CPU-write-hot pages once per memory visibility point
Every draw with a read-only device-address range re-copied the range's
CPU-write-hot pages: they stay writable, so they read as dirty forever.
In Ghost of Yotei that was ~47% of the render thread (thousands of
draws per frame re-copying the same pages, each copy ending the render
pass behind a full barrier).
The guest only guarantees CPU writes are visible at memory visibility
points (a submission slice, a satisfied wait), the rule the PrepareBda
sweep already follows. Each buffer now remembers the range whose hot
pages it copied in the current visibility generation and skips them
there; pages that fault dirty are still copied on every draw. A
command-processor write invalidates only the buffers holding the
written bytes instead of starting a new generation.
SHARPEMU_BDA_RESYNC_HOT_EVERY_DRAW=1 restores the previous behaviour.
[PERF][BDA_SYNC] reports visibility points, processor writes and how
many syncs skipped hot pages.
* perf(vulkan): stop retaining per-draw texture bindings until GPU completion
Ghost of Yotei binds ~300 images per draw, and each one allocated a
TextureResource of ~1.3 KB that stayed referenced until its submission
retired. At under one frame per second every binding outlived gen0, so
nearly every GC was a gen1 collection (299 gen1 vs 35 gen0 in 15 s).
- Drop TextureResource.Resolution: it was written in one place and never
read, and duplicated the ImageRequest (about half the object).
- Keep only the host movie planes, which own a staging buffer recycled
at completion; the other bindings die with the draw.
Measured in game: gen1 collections fell to 9 per 15 s.
* perf(images): widen the image lookup memo to 4096 slots
The direct-mapped memo had 256 slots, while a Yotei draw binds ~300
images and a frame thousands, so slots evicted each other and only 42%
of lookups hit. With 4096 slots (about 2.5 MB) 90% hit.
* fix(agc): allocate command buffer space atomically
Guest threads run HLE exports concurrently. Two exports appending to one
command buffer both read its cursor before either stored it back, so the
second packet overwrote the first. A tester's Yotei DCB showed a
PopMarker NOP written over the first two dwords of a SET_SH_REG 0x228
packet; the stream then hit its trailing zero dword ("packet opcode is
unknown, header 0x00000000") about 20 s into the game.
The cursor update is now atomic per buffer through striped locks; the
lock is released around the guest's buffer-full callback. A one-time
"[AGC][WARN] Two threads append to one command buffer" line reports the
race when the same buffer is contended.
* fix(gpu): stop executing queued GPU work once guest memory is torn down
Closing the window raced the runtime teardown: the runtime detached the
guest GPU memory registry while the presenter's shutdown drain was still
executing queued draws, so the first device-address draw died with
"A device-address program needs the guest GPU memory registry" and the
end-of-run reports were lost. The drain now drops the remaining work
once the registry is gone.
Also report the audio backend a port actually opened instead of a
hardcoded "coreaudio" in the AudioOut output trace.
* fix(runtime): release guest GPU memory only after the presenter's caches
Runtime teardown disposed the guest GPU memory while the presenter was
still shutting down on its own thread: its image and buffer caches still
held page watches, so PageGuard.Dispose failed fast with "Cannot release
the page guard while page state is active". Wait (bounded) until the
presenter detaches its stores, and keep the memory mapped if it does not.
* perf(buffers): stop read-aheads of windows the GPU keeps rewriting
Eager readbacks copied a 512 KiB window after every batch that wrote a
hot window. In Yōtei's loading phase and cinematic the GPU rewrites those
windows before the CPU reads them: of ~2,900 read-aheads per 10 s, 3 were
used and ~2,800 were discarded as stale. Count consecutive wasted
read-aheads per window, skip a window after two, and give every window a
new chance every 256 batches.
* perf(buffers): read only the requested pages from mapped device memory
A guest read of GPU-written memory served from a unified (ReBAR) buffer
copied the whole 512 KiB readback window out of device-local memory,
which the CPU reads uncached across the bus. In Yōtei's loading phase
that memmove was 30% of the render thread. Mapped reads now copy a
16 KiB window around the request; copy-command readbacks keep the large
window. Loading phase, same 10 s window: QueueRelay 2447 -> 947 ms,
frames 205 -> 252.
* diag(gpu): report how tessellated draws split into batches
Adds [PERF][TESSELLATION] to the render profile: draws, instances, patches, batches per draw and the off-chip buffer count. Each batch prepares and binds every stage again, so batches per draw is the multiplier on tessellation cost.
* perf(gpu): run all instances of a tessellated draw in shared batches
DrawTessellation looped over instances on the CPU: every instance took
its own hull dispatch, a full binding preparation of the hull, domain
and pixel stages, and its own draw. Yōtei's cinematic issues up to
~1,900 instances per tessellated draw, ~15,000 iterations per 10 s, and
tessellation was half of the render thread.
Number patches across all instances instead and batch that numbering by
the factor-ring and off-chip capacity, as the hardware distributes the
patches of an instanced draw. A new tessellation data word carries the
patches per instance; each hull thread derives its instance ID (v5), its
patch ID (v0) and its index-buffer vertex, and the domain stage its patch
ID (v8), from the shared number. Zero patches per instance keeps the old
values exactly.
Yōtei: one batch per tessellated draw, cinematic 0.6-0.7 -> 1.1-1.3 FPS.
* perf(memory): let CPU-write-hot pages cool down once per frame
A page the CPU rewrites right after three uploads turns hot: it stays
writable and dirty and every sync copies it whole. Nothing ever cooled a
page down, so a ring the game fills a little each frame ended up hot in
full. In Yōtei's cinematic two 28-32 MiB buffers were copied whole at
every device-address sync, ~60 GB per 10 s.
Once per presented frame, hot pages now lose their heat: the next sync
copies them once more and protects them again, and only pages the CPU
keeps rewriting within a frame turn hot again. SHARPEMU_KEEP_CPU_WRITE_HEAT=1
restores the old behaviour.
Yōtei cinematic: uploads ~60 GB -> ~0.6 GB per 10 s, 1.1 -> 1.5-1.6 FPS.
* diag(images): trace lost GPU image contents and name guest passes
SHARPEMU_TRACE_IMAGE_DROPS=1 reports every time an image loses GPU-written
contents that never reached guest memory: the code site, the image, the
range of a buffer write that caused it and how much of the image it
covers, the image's dirty flags, and the guest render pass decoded from
its push markers (one stack per queue). Evictions report whether they
published the contents first.
A renderdoc.capture file next to the executable now requests a RenderDoc
capture at the next guest flip, for unattended runs.
* fix(images): keep an image being resolved out of collection
ResolveOverlap can allocate a replacement (GrowImage, a depth overlap
copy), and an allocation under memory pressure collects images that
were not used this tick. The image being resolved is the copy source of
that replacement: when it was collected, the copy read a freed slot and
the process failed fast with "The slot is not allocated" (seen twice in
Yōtei with a 4 GB image cache budget). Touch it before resolving.
* fix(images): publish GPU-only image contents before an overlap drops them
Memory keeps its bytes when another image or a buffer write reuses part
of it. When an overlapping image request or a partial buffer write
dropped an image whose GPU contents had never reached guest memory, the
contents were simply lost; a later image over those bytes started from
stale guest memory. Yōtei's cloud tables alias a 2048x2048 RG16F and a
512x512 RGBA16F at one address, and several post-process targets are
partly overwritten by compute buffer writes (7-82 % coverage).
Before such a drop, copy the image into the buffer over its range on the
GPU (the existing image-to-buffer path) and let the buffer own the bytes,
as after any GPU write. Full overwrites still drop directly.
SHARPEMU_IMAGE_DROP_WITHOUT_PUBLISH=1 restores the old behaviour.
* perf(gpu): poll the timeline at most every 0.5 ms in per-draw housekeeping
The completed-submission collection before every draw read the timeline
semaphore twice per draw (vkGetSemaphoreCounterValue), about 12 % of the
command thread in Ghost of Yotei. The per-draw path now polls at most once
per interval and uses the last value otherwise; real waits still read the
timeline at once. SHARPEMU_TIMELINE_POLL_US overrides the interval.
* perf(gpu): look up known samplers without taking the store lock
Every sampler of every draw took the store lock, and the async compute
queue contended for it (3.7 % of the command thread in Monitor slow paths).
Hits now read a concurrent dictionary; only a miss locks to create.
* perf(buffers): read one tracker page per mapped readback
Mapped readbacks copy uncached device-local memory at about 300 MB/s, and
most want a descriptor word or a packet dword. The window drops from 16 KiB
to one 4 KiB tracker page. SHARPEMU_MAPPED_READBACK_KB overrides it.
* perf(images): reuse per-thread level scratch when resolving color targets
Every draw resolved its color targets with two fresh 16-entry arrays that are
only copied into the image description; they were about 10 % of the managed
allocation volume in Ghost of Yotei. One cleared pair per thread replaces them.
* fix(agc): leave the command buffer alone in Push/PopMarker
Ghost of Yotei 1.512 calls the marker exports from a second guest thread on a
DCB that the main thread fills with inline-written packets. Appending a marker
NOP read a stale cursor and overwrote the game's packets: a tester's dump shows
a PopMarker NOP over the first two dwords of a SET_SH_REG, whose stray value
dword then failed as an unknown packet (header 0x00000000). Markers are
tool-only NOPs on PS5, so by default they now return the current cursor
without writing. SHARPEMU_AGC_MARKERS=1 restores the packets for the pass
names that traces decode.
* diag(shader): say why resource materialization failed
Two testers of Ghost of Yotei 1.512 stop on 'could not be materialized ...
reason=Other' for one pixel shader, and Other named no step. Each failure
site now records what failed and the values it saw (the source and dword that
could not be evaluated, the guest address that could not be read, the bad
count, stride or descriptor words); the fatal message appends it as detail=.
Return values and the fatal/reject policy are unchanged.
* fix(agc): preserve marker reservations for the DCB writer
Restore packet reservation and capacity callbacks for markers. Suppress markers from another known guest writer under the allocation gate so they cannot overwrite unpublished inline packets. Scope writer identities to canonical memory across native thread wrappers.
* fix(gpu): preserve partial images before capturing buffer bindings
* fix(audio): report the channel count in AudioOut2 port state
* Revert "debug(gpu): dump the whole command buffer on an unknown packet"
The full dump identified the tester's crash: a PopMarker from a second guest
thread overwrote a SET_SH_REG (fixed by the marker reservation change). Back to
the short dump around the bad packet.
* fix(shader): probe record-aligned material offsets past the indirect image cap
Ghost of Yotei 1.512 stops on 'indirect image 0x494: 72559 unbounded selector
probes exceed 65536' for a 2502-record material table with a 116-byte stride.
Without a proven selector set every multiple of gcd(stride, 2^32) is probed,
because selector * stride + offset wraps modulo 2^32; reaching an offset that
is not a record start needs a selector of at least 2^32 / stride. Tables under
the cap keep the exhaustive enumeration; past it, the record-aligned offsets
are probed and a warning names the image once.
* fix(gpu): track guest pages written through device addresses
Shaders that store to memory through a runtime V# or a flat/global
pointer reach guest memory through the device-address page table, not
through a bound buffer. The buffer cache only learned about GPU writes
from bound writable bindings, so these stores were invisible to it: the
page stayed "clean", the host never downloaded the GPU copy, and the
next CPU upload or eviction silently overwrote the GPU result with stale
guest bytes (or the CPU kept reading stale bytes). Ghost of Yotei's
GPU-driven passes (producer computes filling per-object data and V#
tables) write almost exclusively this way.
The translator now reports the page of every device-address store and
atomic in a second bitmap that follows the fault bitmap in the fault
buffer. Non-pixel invocations merge bits of one bitmap word and publish
it when the word changes or the invocation ends; pixel invocations,
which a discard can end early, publish at once. The fault processor
compacts that bitmap with the same pass as the fault bitmap and, once
the scan completes, GuestBufferCache.NoteDeviceAddressWrites marks the
pages GPU-written exactly as a written binding would: GPU-dirty in the
page tracker, recorded in the GPU-modified ranges, the owning buffer's
write tick bumped, and overlapping image memory copies invalidated.
A page the CPU dirtied before the write is reported cannot be owned by
both sides; it stays CPU-owned and the conflict is logged once per power
of two. On Yotei these conflicts remain (dozens per run, mostly the
producer input pages at 0x50CB...), so mixed CPU/GPU pages can still
lose bytes; resolving them needs a byte-level merge and is left for a
follow-up.
Diagnostics added alongside:
- SHARPEMU_TRACE_COHERENCE=1 checks, every fourth presented frame with
the GPU idle, that pages the tracker calls clean hold the same bytes
in guest memory and in the mapped buffer, and reports each divergence
(SHARPEMU_TRACE_COHERENCE_MB sets the slice checked per pass). With
SHARPEMU_TRACE_GPU_MEMORY_ADDRESS=coherence the first diverging page
is traced. On Yotei the detector went from hundreds of divergences to
zero with this fix.
- Guest buffers are named "guest 0x<address>+0x<size>" when Vulkan debug
labels are enabled, so RenderDoc captures show which guest range a
binding is.
* fix(gpu): merge shader and CPU writes that share a guest page
On the console the CPU and the GPU write one memory, so a 4 KiB page
can hold bytes from both. The page tracker gives each page a single
owner. When a shader stored through a device address into a page the
CPU had dirtied before the store was reported, the page stayed
CPU-owned and the next upload copied the whole guest page, stale
where the shader wrote, over the GPU copy. The guest never saw those
shader bytes either. Ghost of Yotei hits this dozens of times per run
(producer input pages at 0x50CB..., per-draw rings at 0x2002...).
SharedPageShadows keeps, per such page, the bytes both copies last
agreed on and does a three-way merge:
- Uploads of a shadowed page copy only the byte runs the CPU changed
since the shadow; the shader's bytes in the buffer stay.
- When a device-address write is reported on a CPU-dirty shadowed
page (once every upload staged for it has executed), the bytes the
GPU changed since the shadow are written into guest memory where
the CPU left them alone. A byte both changed keeps the CPU value,
which the next upload carries to the GPU.
- Downloads refresh the shadow, since both copies agree afterwards.
A shader-written page gets its shadow when the CPU first touches it:
that access downloads the page, so guest memory matches the GPU copy
and no extra read of uncached mapped memory is needed. Pages only the
GPU uses cost one dictionary entry. A page the CPU dirtied before its
first shader write was reported is shadowed from the mapped GPU copy;
shader bytes an upload already overwrote there are lost once, and the
log says so. At most 16384 pages (64 MiB) are shadowed.
On Yotei: 0 unresolved conflicts (8 one-time losses), 0 coherence
divergences, no throughput loss in an A/B run (about 5.6k vs 6.2k
draws/s over the same cinematic window). The in-game darkening is
unchanged, so these lost writes were not its cause.
* fix(gpu): expose the partial last word of byte-addressed buffers
Storage buffers are declared to shaders as uint[], and Vulkan's runtime
array length counts only whole words. A buffer whose size is not a
multiple of four therefore hid its last one to three bytes: byte,
short and formatted loads there read zero, and stores were dropped.
Ghost of Yotei's occluder producer reads a one-byte-per-instance state
buffer of 95 bytes. The last three instances were the ones to produce,
so the shader saw them as inactive, wrote no matrices, and the depth
prepass drew the object with the previous frame's stale matrices:
exploding vertices that cover the scene with black bands and holes.
Byte-addressed and formatted bindings now get a view rounded up to the
next word. The number of padding bytes travels in the two spare bits of
the runtime stride entry (strides use 14 bits), and the shader keeps
the guest byte bound for those bindings: byte assembly may read the
padded word, but an element is loaded or stored only when all its bytes
are inside the guest range, and whole-word accesses keep the old bound.
The padded view must stay inside the cache buffer, which spans whole
pages, so this never reaches past its VkBuffer.
The device test reads and stores the tail of 41- to 102-byte buffers,
aligned and unaligned, formatted and byte loads, with and without the
zero-out-of-bounds read path, and checks that bytes past the guest
range stay hidden and untouched; 13 of its 17 cases fail without the
padding.
* fix(gpu): only wait for shared page uploads that copy bytes
A shadowed page's merge waits until every upload staged for it has
executed, since until then the GPU copy lacks the staged CPU bytes. The
staged tick was advanced for every upload that visited the page, also
when filtering removed all of its bytes because the CPU changed none.
Such an upload copies nothing, yet merges waited for its tick, and a
page re-staged every frame could keep shader writes out of guest memory
indefinitely.
The tick now advances only for runs that survive filtering and for a
whole-page copy into a new owner. The tests cover an eliminated upload
followed by a merge and a real pending upload that still blocks it.
* fix(gpu): keep shared page shadows when a buffer grows
When a request outgrows a cached buffer, the cache creates a larger one
and copies the old buffer's GPU contents into it. Shadows of shared
pages still named the old buffer as owner, so the next upload into the
replacement treated the page as unknown and copied the whole guest
page: stale where shaders had written and not yet been reported, it
overwrote those bytes in the GPU copy.
The shadows (and the pages waiting for their first CPU access) now
follow the copy to the replacement, and a merge waits for the copy's
tick, since the replacement holds the old bytes only once it executes.
The device test grows a buffer between an unreported shader store and
the next upload and checks that both shader writes and the CPU write
survive.
* diag(gpu): watch guest ranges per frame and force full barriers
Two opt-in diagnostics used to trace Ghost of Yotei's stale occluder
matrices back to the producer that never wrote them:
- SHARPEMU_WATCH_BLOCKS=addr:len[,addr:len...] logs, after every
presented frame, each watched range whose bytes changed in guest
memory or in the GPU copy (both contents, dirty state, submitted and
completed ticks), and every cache event touching a watched range:
device-address write reports, uploads, writable bindings, CPU write
faults, host DMA writes, fills and copies.
- SHARPEMU_DIAG_BARRIER_EVERY_CALL=1 records a full memory barrier
before every draw and dispatch, to rule missing guest barriers in or
out. It is very slow.
Both are off by default and cost one branch when off.
* fix(gpu): read and write whole words that start inside a buffer range
The previous fix padded only byte-addressed and formatted bindings and
kept the floor word count as the bound for whole-word accesses. The
hardware range-checks a buffer access by its first byte (stride 0 or
swizzle off: offset >= num_records is out of range), so a dword load or
store that starts inside a range ending mid-word is performed whole.
A dword-addressed buffer of 95 bytes therefore returned zero for its
last word, bytes 92..94 included, where the console returns the data;
whether the read got zero depended on the driver's robust access
granularity, not on the guest descriptor.
Every runtime-stride view is now padded to whole words, and a word is
in range when it is inside the padded view, which is exactly the
hardware bound. Byte and formatted elements keep the exact byte limit
carried in the runtime stride entry.
The new device test loads dwords from 93- to 96-byte buffers with both
read paths; three of its five cases fail without the change.
* fix(shader): track both words of binary64 vector results
(cherry picked from commit 2e339ac4143061e76bfefb4e065b04eba4647f77)
* fix(shader): lower half-word logical shifts on Metal
(cherry picked from commit fa9776ec2b71a30c6a833562d8e342f5718a612f)
* feat(gpu): pack narrow color channels without changing encoded bytes
(cherry picked from commit e98fe0b4e68cd5e070124c97d583911561f5fd00)
* feat(gpu): transfer narrow sRGB images through expanded backing
(cherry picked from commit 9a5f4e08aaf101ebd68ca18d43efac17ec94112c)
* fix(gpu): retain explicit pitch for linear color target transfers
(cherry picked from commit 88695316e2119b18cb095ae7ad00e1c17ea1f6fb)
* fix(shader): decode explicit-LOD comparison gathers
(cherry picked from commit a629cc3949cba63b52cfbdc0b41199a3fc1cba79)
* fix(gpu): decode narrow sRGB color target outputs
(cherry picked from commit 66aa64a01fc4ad984d53fa248b742481d3b54e61)
* fix(agc): accept the shader register Ghost of Yotei 2.002 writes at 0x192
Ghost of Yotei Complete Edition 2.002 applies a shader register table at
startup that writes 1 to SH offset 0x192 (0x2D92), just past the LS block.
gfx10 documents no register there and no shader state depends on it, but
the table writer had no entry for it and stopped the emulator right after
the splash screen. Ignore it like the other pace and control registers.
* diag(agc): log the values written to the undocumented shader register 0x192
The register is still not applied, but the first value and every change
are logged, so a rendering difference in Ghost of Yotei 2.002 can be
traced back to what the title wrote there.
* perf(shader): look runtime descriptors up through one shared function
Reading buffer-table descriptors at run time (90e35ba8) inlined the hash and
the eight probes of the runtime descriptor table at every image and sampler
access. A Ghost of Yotei pixel shader with 56 such accesses grew from 419 KB
to 2,051 KB of SPIR-V and took about 6.5 s to compile instead of 0.5 s.
The image and sampler lookups are now one DontInline function each, called
with the key and the lane's EXEC state, as the device buffer format decoder
already is; the same shader is 507 KB and still binds only the records it
reads.
* fix(shader): read format bytes in a buffer's last partial dword on the device
A format load through a device-side V# assembled each component byte from
its aligned dword, and that dword read zero unless all four of its bytes were
below num_records. The hardware range-checks the element, so the last one to
three elements of a byte buffer whose size is not a multiple of four read 0
instead of memory.
Ghost of Yotei's culling updater reads each object's 8_SINT visibility state,
skips the store when the new state equals the one it read, and counts the
element. Its 19-byte state buffers read 0 for elements 16..18, so the store
was skipped while the element was counted: counts[] exceeded the instance
slots the producer wrote, and the extra instances drew as black blocks.
The dword is now read when any of its bytes is inside the buffer; the existing
per-component check still zeroes a component past num_records.
* fix(loader): prove jump table bounds and tables over the control-flow graph
The red-zone patcher refused every short faultable site of a function whose
jump tables it could not resolve. The resolver only accepted a lea-only window
between a 32-bit compare and the table load, so Ghost of Tsushima's render
thread kept unprotected accesses in a leaf function that holds a pointer in the
red zone, and a host exception dispatch clobbered it.
Resolve each table by walking the function's control-flow graph backwards from
the load: every path must pass the fallthrough of ja after an unsigned compare
of the selector, reaching the load through register moves and zero-extensions
only, and must define the table register with lea of the same RIP-relative
table. The graph includes the resolved table edges, and resolution repeats until
a round reproduces its own targets. Tsushima's indirect-branch refusals drop
from 55 to 8.
* fix(agc): accept the user-config register Ghost of Yotei 2.002 writes at 0x191
The 2.002 build writes 0 to user-config offset 0x191 from a register table,
right after shader register 0x192. No gfx10-12 register lives there. Accept
it and log value changes, like 0x192.
* fix(images): keep a mapped mip chain whole when a texture starts past mip 0
A sampled texture whose descriptor starts past mip 0 was always made an image
of only the levels from there, for streamed textures whose larger mips are not
mapped. A pass that samples a lower mip of an image the GPU rendered (Demon's
Souls' fog and bloom chains) then got a separate image of the same memory,
filled from stale guest bytes and traded with the render target every frame:
white blocks across the fog and 0.2 fps.
The image cache now keeps the resident-levels form only when the chain is not
fully mapped; otherwise the texture is the whole chain viewed from its base
level, the same image as every other view of that memory.
* fix(np): report the package's real trophies from sceNpTrophy2GetTrophyInfoArray
b5b3d691 treated rdx as the first trophy id and always returned the requested
limit as the count. The call is (context, handle, offset, limit, details[],
data[], *count): offset indexes into the package and the count is how many
trophies exist from there, capped by limit. Hades asks for 50 and then walked
past its own on-stack arrays, crashing in a job thread before the first frame.
Read sce_sys/trophy2/trophy00.ucp (unencrypted container with tropconf.json and
tropmeta_*.json) and fill SceNpTrophy2Details (1312 bytes) and SceNpTrophy2Data
(32 bytes) for the trophies that really exist, all locked. Layouts and the
package format follow KytyPS5's NpTrophy2 implementation.
Co-authored-by: Felipe <54341095+Pipe530@users.noreply.github.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
* chore(audio): mark the YoteiPC-derived ATRAC9 code as GPL-2.0-only
The multichannel block ordering in Atrac9DecodeState follows YoteiPC's at9_api.c,
which its author publishes under GPL-2.0 (no "or later" grant). The file now says
GPL-2.0-only with the source and author named, and the license text is added for REUSE.
LibAtrac9 itself stays MIT.
---------
Co-authored-by: ParantezTech <12572227+par274@users.noreply.github.com>
Co-authored-by: kuba <xnetcat.dev@gmail.com>
Co-authored-by: MathiasYG <06mathias6@gmail.com>
Co-authored-by: voidborn_astra <41128975+voidborn-astra@users.noreply.github.com>
Co-authored-by: Felipe <54341095+Pipe530@users.noreply.github.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
* [SaveData] Read the dialog mode and user data at their offsets in the open parameters
* [SaveData] Report each slot's own block count instead of the fixed maximum
* [Np] Report the offline profile as signed out from sceNpCheckNpReachability
* [Kernel] Keep an oversized reservation at the guest's hint instead of failing
A 768 GB sceKernelReserveVirtualRange with a hint at 0x1000000000 cannot use the
sparse aperture (it starts at 256 GB, and 256 GB plus 768 GB passes the 1008 GB
user limit), so it needed one contiguous host placeholder. When a host
allocation made before the startup reservation sits inside that window, the
call fails with 0x8002000C and the guest stops at boot.
Reserve such a range at the guest's own hint as a guest-only sparse range, as
smaller large reservations already are.
* [Kernel] Sign-extend the int id of user events into the event ident
sceKernelAddUserEvent, sceKernelAddUserEventEdge, sceKernelDeleteUserEvent and
sceKernelTriggerUserEvent take an int id, which the kernel stores in the
uintptr_t ident of the event. A negative id such as -1 therefore reads back from
sceKernelGetEventId as 0xFFFFFFFFFFFFFFFF. The exports used the zero-extended
register (0x00000000FFFFFFFF), so a guest that treats a negative id as a
wake-up event took its normal event path instead and never went back to wait.
Use the sign-extended id in all four exports.
* [Cpu] Prefer LLE for libc FILE-object exports
* [Libc] Add file-lock compatibility exports
* [Kernel] Expose the process argument count and vector
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [AppContent] Report missing add-on content entitlements
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Videodec2] Decode guest access units into pitched NV12 frames
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Network] Add the RUDP context lifecycle
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [SharePlay] Report inactive share-play state
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Np] Complete offline authentication requests as signed out
* [Kernel] Warn when APR cannot resolve a resource path
* [GPU] Skip piece hashing for stencil association proxies
* [VideoOut] Add deduplicated volume texture binding traces
* [SpirvTranslator] Match guest F32 denormal and trigonometric semantics
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Network] Add socket epoll registration and readiness waits
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Network] Close POSIX socket descriptors through sceNetSocketClose
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Network] Add Ethernet address string conversion
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Json] Track the global special-float handler
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Json] Add the special-float format setter
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Pad] Add the n3kSX62fgNo compatibility export
* [Pad] Trace input state changes when requested
* [PlayGo] Keep unknown chunk lists empty
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [PlayGo] Keep chunk metadata parsing in the metadata loader
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [PlayGo] Expose runtime state reset
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [PlayGo] Add install and optional chunk queries
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Loader] Preserve switch targets while patching guest red-zone accesses
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Loader] Report patching progress for large executables
* [SpirvTranslator] Preserve F32 signed zero, infinity and NaN when supported
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [GPU] Use Gen5 render-target tile equations on the CPU and GPU
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [GPU] Decode DCC fixed and register clear values
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [GPU] Track DCC metadata coverage for color targets and volume textures
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [GPU] Materialize DCC slice clears from guest metadata
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Tests] Cover Gen5 tiling, DCC clears and offline authentication
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Tests] Match red-zone coverage to forward-only patching
* [Pad] Track independent standard, special and remote port handles
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [UserService] Add user number and termination exports
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [NP] Expose the game trials flag
* [Kernel] Allow renaming and unlinking open guest files
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Tests] Cover controller ports, user exports, trials and open-file renames
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Kernel] Clear the outputs and report no space when no direct memory span is free
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Kernel] Return POSIX results from _open and keep sceKernelOpen raw
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Kernel] Open /dev/random and /dev/urandom as host RNG descriptors
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Kernel] Read random device descriptors from the host RNG
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Kernel] Close random device descriptors
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Kernel] Report random device descriptors in fstat
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Cpu] Place the guest exception mcontext at the start of the context
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Cpu] Report the flags and Sony stack alias in guest exception contexts
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Cpu] Expose the guest exception context writer to tests
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [NP] Keep Universal Data System handles across repeated initialization
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [NP] Create Universal Data System contexts before explicit initialization
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [NP] Reset Universal Data System state with the runtime
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Pad] Write the device class extended information at its guest size
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Pad] Add scePadDeviceClassParseData
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Tests] Cover random devices, POSIX open, UDS initialization, pad class data and exception contexts
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [SaveData] Write the mount info structure at its 48-byte guest size
* [Stubs] Add sceTextToSpeech2Cancel and sceTextToSpeech2GetSpeechStatus
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Kernel] Add sceKernelSync
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [Loader] Patch red-zone sites from the previous instruction when every indirect branch is resolved
* [Tests] Cover backward red-zone spans, direct memory no-space results and the mount info size
---------
Co-authored-by: Sekai <140012992+SekaixX@users.noreply.github.com>
* [GPU] Sweep CPU writes for device-address programs at memory visibility points
A device-address program may read any mapped page, so before each such
draw or dispatch the buffer cache uploaded every CPU-dirty page whenever
the global CPU-dirty epoch had moved. A guest that streams its per-draw
ring while the command stream is being translated moves the epoch between
almost every pair of draws, and each upload ended the rendering scope and
drained the GPU with an ALL_COMMANDS barrier pair - about 670 times per
frame in Silent Hill: The Short Message's menu.
CPU writes only need to be visible at memory visibility points: when a
submission slice starts, when a wait packet is satisfied, and after a
command-processor memory write (later commands may read those bytes through
device addresses). The sweep now runs at the first device-address program
after such a point, and its uploads share one barrier pair instead of one
per buffer. SHARPEMU_BDA_SWEEP_EVERY_DRAW=1 restores the per-draw sweep.
Menu GPU time drops from about 435 to 290 ms per frame.
(cherry picked from commit fafe92b71271cf750dd89251891bac20fb87f1ff)
* [GPU] Let command-processor DMA writes turn pages CPU-dirty instead of copying
A DMA fill or copy into memory without GPU-written bytes is executed on
the CPU, and the bytes were then pushed into every overlapping cache
buffer through a staging copy and a barrier pair, which ends the
rendering scope (~60 per frame in Silent Hill: The Short Message's menu).
When no GPU-written byte shares the written pages, the pages now simply
turn CPU-dirty: every later GPU reader synchronizes CPU-dirty pages before
reading (bound buffers on obtain, device-address programs at the next
memory visibility point, which the write now raises), so the bytes still
arrive in command order. Pages that hold GPU-written bytes keep the
staging copy. SHARPEMU_DEFER_HOST_WRITES=0 restores the copy.
(cherry picked from commit 3fe21faa3821551fa99b9398064f0652e45d2e84)
* [GPU] Keep hot CPU-written pages writable across device-address sweeps
A page the guest rewrites every frame turns hot after two upload/write
cycles, and the bound-buffer path then serves it from the stream ring so
the page stays writable. The device-address sweep and the range
synchronization still re-protected hot pages after every upload, so the
guest's next write to each ring page took a protection fault - about
5,000 faults per second on Silent Hill: The Short Message's render
thread, each a signal round trip under Rosetta plus page-guard lock
traffic shared with the command worker.
Both sweeps now preserve hot pages: they stay dirty and writable and are
copied again on the next sweep, a few hundred KiB of ring memory per
sweep. Faults drop to a third and the menu goes from 3.8 to 4.2 FPS.
SHARPEMU_BDA_SWEEP_REPROTECT_HOT=1 restores the re-protection.
(cherry picked from commit 65d2b55ddc07644d52e5ad3112cbe323a2421abd)
* [GPU] Refresh writable hot pages at every BDA visibility point
* [VideoOut] Add an internal render resolution scale option
Adds the host-side setting an internal-resolution implementation needs:
--render-scale on the command line, SHARPEMU_RENDER_SCALE for scripted
runs, and a global and per-game choice in the GUI. The value is a host
resolution multiplier of the guest render targets, normalized to a
hundredth in [0.25, 2] so the command line, the GUI and the environment
agree bit for bit; the renderer compares scales when it decides whether
two views of the same guest memory match.
RenderScalePolicy holds the decision rule. It is a pure function of the
guest image description, so every view, alias, copy and descriptor of
one guest range reaches the same answer without consulting the cache.
Only screen-class, single-level, single-sample, uncompressed 2D render
and depth targets scale; block grids, multisample footprints, HTile and
DCC coverage, mip chains, cube and array faces, asset tilings and small
surfaces (shadow maps, lookup tables, eye-adaptation readbacks) keep
their guest resolution. The environment variable wins over the stored
setting so test runs do not depend on the user's configuration.
No renderer consumes the scale yet; a scale of one is the current
behaviour everywhere.
* [GPU] Hold scalable guest images at a host render resolution
The backing of an image the scaling rules accept is created at
ceil(guest * scale) while its ImageDescription keeps the guest geometry,
so the cache, the overlap rules and the page tracker go on reasoning in
guest bytes.
Everything that moves guest bytes now goes through a guest-resolution
twin of the scaled image: an upload fills the twin from the copy regions
the tiler built and blits into the backing, a download blits the backing
into the twin and copies out of it. A copy whose two images disagree on
host resolution runs at guest resolution the same way, so callers keep
describing rectangles in guest pixels; the destination seeds its twin
first because a partial copy is blitted back whole. The multisample
depth conversion takes a guest-sized source for the same reason, and the
resolve region crosses into the destination's resolution instead of
silently resolving the top-left corner.
Two fallbacks keep correctness ahead of the saving. A format that cannot
blit keeps its guest resolution, since the twin could never reach it.
A storage image drops back to guest resolution the first time a compute
shader binds it: the dispatch grid does not shrink with the host image,
so the shader would write it with holes above 1.0 and redundantly below.
The guest range is remembered, so every later image over that memory is
created unscaled too.
* [GPU] Scale the render area, viewport and scissor with the attachments
A resolved target now reports the host extent of its attachment beside
the guest one, and the three places that size rasterization follow it:
the dynamic rendering area, which also bounds every load-op clear; the
guest viewport transform; and the guest scissor, whose edges round
outward so no guest pixel the game asked for loses host coverage.
All attachments of a pass share one rule, so the first bound target
decides the factor.
The clip-disable path needs care because the vertex program bakes the
viewport transform against a reference extent. Dividing that reference
by an upscale keeps the host viewport (reference x scale) inside the
device limit while the baked transform stays scale independent, so the
same program maps guest pixels onto host positions at either resolution.
At a scale of one the reference is the device limit, exactly as before.
* [SpirvTranslator] Translate pixel positions and image coordinates across the render scale
A program expresses its pixel position, its integer image coordinates
and the sizes it queries in guest pixels, but the host may hold the
image behind any of them at another size. Which images are scaled is a
per-draw fact, so the factors travel in shader data rather than in a
permutation: the coordinate factor, its reciprocal, a 64-bit mask of
scaled image resources and the factor the pixel position is divided by.
With that:
- gl_FragCoord is divided by the attachments' factor as it is loaded, so
POS_X/POS_Y, POS_FIXED_PT and everything derived from them - screen
UVs above all - stay in guest pixels.
- An integer image coordinate is mapped onto host texels through the
bound image's factor. Only the two scaled axes move; an array layer or
a volume slice is not a pixel.
- A resource-info query reports the guest size, because the program
derives coordinates and loop bounds from it.
A select against the mask makes an unscaled binding cost one
instruction, and the layout only carries the dwords when the scale is
not one, so nothing changes at native resolution.
* [GPU] Keep one host resolution across the attachments of a pass
A rendering scope has one render area, one viewport and one scissor, so
its attachments have to agree on a host resolution. They disagree when
one of them cannot be scaled at all - a depth target with a stencil
plane next to a scalable colour target, say - and the pass then renders
a fraction of its area. The draw now drops every scaled attachment back
to guest resolution and resolves its targets again; the guest ranges are
remembered, so the decision converges after one draw and cannot
oscillate.
That fallback only works if the executor knows what the cache actually
gave it, which is not what the scaling rules would answer: a demoted
range still looks scalable to them. The resolved targets therefore carry
the image's own factor, and the render area, viewport, scissor and the
depth/stencil copy extent read it from there.
Two rules change with it. DCC only describes guest bytes the host never
stores, so a colour target that carries it can still be scaled; HTile
encodes guest depth blocks the renderer reads back as clear state and
stays native. And a storage image only drops back when the program
actually writes it, since a read-only load is mapped correctly.
Scaling now also requires a backend that installed the format
predicate - the Vulkan device - so the Metal host is unaffected.
* [Docs] Describe internal resolution scaling and its limits
Replaces the note saying arbitrary internal resolution scaling is not
implemented: it is, for screen-class render and depth targets. Records
what the renderer translates (rasterization, shader coordinates, guest
transfers, presentation), the exact rule for what scales, every case
that falls back to guest resolution, the artefacts measured on Silent
Hill, and the per-scope GPU times at 50, 100 and 200 percent.
* [SpirvTranslator] Preserve guest texel offsets at scaled resolutions
* [Agc] Emit addressed draw events for occlusion query counters
* [Gpu] Keep fallback occlusion results coherent and conservatively visible
* [Gpu] Honor disabled color buffers during depth-only draws
* [Gpu] Upload complete unchecked constant vertex attributes
* [Agc] Emit direct draws for every draw modifier
sceAgcDcbDrawIndex and sceAgcDcbDrawIndexAuto returned without writing a
packet unless the modifier was exactly 0x40000000. The modifier names the
shader stage and user registers of the bound pipeline, so any other
pipeline lost its draws without a trace. Unreal bakes its color-grading
LUT through a geometry-stage pipeline, which left the LUT unwritten and
tinted every 3D frame of Poppy Playtime blue.
Both draws now emit their packets for any modifier. Modifiers other than
the vertex-stage default are logged once each.
(cherry picked from commit 8d46a6068b7d1a999662eeac1ee17f9e6d87c2eb)
* [VideoOut] Separate guest display mode from presentation buffer dimensions
* [GPU] Map half-wave shuffle lanes into the host subgroup
* [Docs] Explain display modes and internal resolution limitations
* [GUI] Translate guest display mode settings in all supported languages
A CPU read of GPU-written guest memory recorded a copy at the tail of
the current command buffer and waited for it, so it also waited for
every draw recorded after the writer. In Silent Hill's title menu most
of those reads targeted buffers whose writer had retired long before,
and UE's render thread spent a third of its time in them.
On a device with device-local, host-visible, coherent memory (Apple
GPUs), guest buffers are now allocated there. A readback whose writers
retired copies straight out of the buffer's mapping; one whose writer
is submitted but still running waits for that submission alone and then
copies; the command worker submits a writer that is still recording
and reads after it. A guest-thread wait also marks its window hot, so
the next GPU write there is submitted right after its command.
SHARPEMU_UNIFIED_BUFFERS=0 keeps device-local buffers.
* [CLI] Restart under Rosetta with a 64 MiB gen0 GC budget
Every collection suspends the runtime through FlushProcessWriteBuffers,
which Rosetta 2 services by parking each thread in its runtime
(notify_thread_waiters). A signal that reaches a parked thread trips
Rosetta's "expected saved LR to be in translated code" assertion and
wedges the process; Demon's Souls hit it in about half of its 3-5 minute
runs while collecting almost twice a second. The gen0 budget is only read
from the environment, so the process re-executes itself once with
DOTNET_GCgen0size set (same pid). With 64 MiB, Demon's Souls collects
about once every 6 seconds, gen0 and gen1 only, 0.26 s of pauses in
110 s. A 512 MiB budget made every collection a blocking gen1/gen2 pass
of about a second. SHARPEMU_ROSETTA_GC_BUDGET=0 opts out.
* [VideoOut] Let a buffer wait ignore CPU flips the graphics queue has not reached
sceVideoOutSubmitFlip reserves its flip request, may sleep for pacing,
and only then queues the flip preparation on the graphics queue. A
command buffer submitted before it that starts with
sceAgcDcbWaitUntilSafeForRendering on the same display buffer saw the
reserved request as pending and waited for a preparation queued behind
itself: the graphics queue never advanced and every compute queue waiting
on its labels stalled with it. Demon's Souls hung this way on the boot
splash in about one run in three.
A CPU flip now counts for the wait only once the graphics queue runs its
preparation; GPU flips are ordered from their packet.
(cherry picked from commit 8915df35abad5fe8b15cc6418f1e563785d6e008)
* [Media] Expire a host movie only once its playback stops advancing
Host playback follows the guest's own Bink decode and audio clock, so on
a slow host a movie plays slower than real time. The watchdog fired at
the movie's duration plus a minute regardless, marked the file timed out
and forced EOF on the guest's reads while its decoder was still
mid-stream: Demon's Souls' 171 s attract movie froze on its last
host frame for good. The watchdog now re-arms while frames keep
advancing and expires a movie only after a minute without progress.
(cherry picked from commit daad0f097d0c6369097aeef32b55c9f2d171110a)
* [CLI] Preserve the managed entry point when restarting under Rosetta
* [Gpu] Cover the whole tile plan when a buffer reads back an image
TrySynchronizeBufferFromImage sized the copy from the mip layout, but the
tile plan of a fully fitting image can extend past it; Silent Hill hit a
128x128 slice planned at the exact end of the mip chain and the tiler
aborted the process. Grow the copy to the plan's tiled end when the
buffer holds it, otherwise keep the guest bytes.
* [SpirvTranslator] Read half-selected V_FMA_MIX literals as f16
op_sel_hi marks a V_FMA_MIX operand as f16 for literals too, but literals
were always read as f32 bit patterns. Silent Hill's deferred lighting
weights a luminance with the f16 literals 0x34CD/0x38B8/0x2F0A (0.30, 0.59,
0.11); as f32 denormals they made the luminance zero, the shader clamped it
to 1e-4 and divided by it, and the title-menu scene came out thousands of
times too bright, which auto exposure then turned black.
* [Gpu] Materialize every fixed DCC clear code on a sampled image
A DCC fast clear stays pending until the surface binds as a colour target;
when a shader samples the surface first, only the clear-to-zero code was
materialized. The other fixed codes (alpha 1, rgb 1, rgba 1) carry their
own colour too. Silent Hill fast-clears UE's separate-translucency target
to (0,0,0,1) and only samples it, so the post composite read stale zeros
and the whole title-menu scene came out black.
* [SpirvTranslator] Keep wave64 LDS phase barriers when a half-mask plan applies
The half-mask plan's barriers order its own mask exchanges, not the
program's shared-memory traffic, yet planning a program switched the LDS
phase ordering off. UE's FFT bloom (wave64, 512 threads, LDS butterflies)
then read half-written rows: Silent Hill's convolution output collapsed
into a flat haze (max 56 for a scene peaking above 1500) and the histogram
fed by it never adapted exposure.
* [Gpu] Publish linear GPU-written images to guest memory by default
Unreal reads its eye-adaptation result back on the CPU through a linear
staging texture (FRHIGPUTextureReadback) and derives View.PreExposure from
it. The image cache was built with linear readback disabled, so the CPU
always read zeros, PreExposure stayed 1.0 and the scene rendered without
pre-exposure. FFT bloom's bright-pixel prefilter, which works in
pre-exposed space, then flattened every pixel to its max luma and the
Silent Hill title menu came out nearly black. SHARPEMU_READBACK_LINEAR_IMAGES=0
restores the old behaviour.
* [Tests] Enable MoltenVK portability devices in headless harness
* [SpirvTranslator] Saturate V_CVT_PKRTZ_F16_F32 at 65504 instead of overflowing to infinity
Rounding toward zero never produces infinity from a finite value: past the
f16 range the result is +-65504. We truncated the mantissa and let
packHalf2x16 round, so large values became infinite. Silent Hill's deferred
lighting packs title-menu values past 65504; its height fog then multiplies
scene color by a tiny transmittance, which keeps 65504 finite but leaves an
infinity infinite, a later pass turns it into NaN and the whole 3D menu
background renders black. Subnormal results now also keep whole 2^-24 steps.
A MoltenVK device test covers overflow, truncation, subnormals, NaN and inf.
* [Tests] Wait for the blocked worker before publishing its signal
* [HLE] Answer POSIX host memory queries without the mapping lock
Every HLE import that reads stack arguments queries the guest stack's
protection first. On POSIX the query took the same lock Protect holds
across its mprotect call, and GPU write tracking protects pages
constantly, so each query waited tens of microseconds; Silent Hill: The
Short Message spent up to a second of every five seconds in queries.
Regions are now also published as an immutable snapshot (replaced under
the lock when a region is added or removed), and a query of untracked
memory or of a region without per-page overrides is answered from it
without the lock. Regions with overrides keep the locked path, which now
finds where a protection run ends through an ordered set of overridden
pages instead of visiting every override (write tracking leaves tens of
thousands), and the next region by binary search. Query time drops from
200-1000 ms to 5-50 ms per 5 s.
This is the POSIX counterpart of replacing page-by-page VirtualQuery
scans on Windows.
(cherry picked from commit b6ffe31f28fa93842726b71315c7731f97b5fda8)
* [Cpu] Sleep through nanosleep, resumed on EINTR, in the POSIX Sleep stub
The POSIX stand-in for kernel32!Sleep (which the native sceKernelUsleep
intrinsic calls with milliseconds) converted to microseconds and called
usleep. macOS rejects usleep(>= 1 s) with EINVAL and returns at once, so
every guest sleep of a second or more came back immediately; and a
signal (the runtime's thread activations, for one) ended any sleep early
through an error the stub never looked at.
The stub now builds a timespec and calls nanosleep, and when nanosleep
returns EINTR it resumes with the remaining time. A test exercises the
stub with the Win64 argument register for both a 1.2 s and a 20 ms
sleep.
(cherry picked from commit 43b770f86b5e3aa2332121f1bc2ddf2acc9d2968)
* [HLE] Query POSIX view regions from a lock-free snapshot
Every HLE import first queries the guest stack's protection, and the
view-region query took the same gate the GPU write tracker holds across
its mprotect calls, so each import waited on page-protection traffic
from other threads (the main guest thread spent ~2% of its time in that
wait at Silent Hill: The Short Message's menu).
The region list is now also published as an array snapshot whenever it
changes, and queries read the snapshot without the gate. Per-page
protections already changed in place and were read racily; nothing
changes there.
(cherry picked from commit d7e72cb4974388883696516420b2e9d462778715)
* [Cpu] Skip the scheduler gate when a host thread re-registers the same guest context
scePthreadSelf on a guest thread the scheduler does not own registers
the thread's context on every call, and that took the scheduler-wide
gate each time - tens of thousands of times a second on Silent Hill:
The Short Message's main thread, contending with the dispatcher and
every blocked-thread wake.
Each host thread remembers the handle and context it registered last;
a repeat registration only refreshes the external-handle hint and
returns. New or changed registrations still take the gate.
(cherry picked from commit d1991b2b75261cd1f1ccb4e005b10805f0da24c1)
* [Cpu] Keep each thread's guest continuation stub instead of mapping one per resume
Every resume of a blocked guest thread allocated a 512-byte executable
stub page and a host-RSP slot, and released both when the thread blocked
again: an mmap/munmap pair plus two acquisitions of the host-memory gate
per resume, which the GPU write tracker holds across its own protection
changes. Contended on that gate, resumes were 62% of all lock waiting in
Silent Hill: The Short Message's menu.
Each host thread now keeps a stub page and slot per nesting depth for its
lifetime; a resume reopens the page for writing with mprotect directly
(the page is emulator-private, so it bypasses the tracked host-memory
layer), re-emits the stub and seals it again. Lock waiting across threads
halves (13.6 s to 6.5 s per 30 s).
(cherry picked from commit 2c2293abdd5d19dd1be3d6941dcedf9b482c6420)
* [Kernel] Round host condvar waits up to whole milliseconds
Monitor.Wait truncates its timeout to milliseconds, so a timed wait with
less than 1 ms left became a zero timeout and spun until the deadline.
UE's game thread waits for the render fence in 1 ms slices, which kept a
host core busy for the whole wait and contended the condvar lock with the
signaling threads. Timed waits may wake late, so round up instead.
SH menu 4.29 -> 4.64 fps together with the readback retry change.
(cherry picked from commit 66eaa0095fe594ff8e285cbf31c32bd84274ac85)
* [Tests] Run the POSIX Sleep stub test only in x64 processes
The stub is x86-64 machine code; a native arm64 test host cannot map or run
it, so the macOS arm64 CI runner failed to allocate the stub page.
A vector register write is modelled as select(EXEC, result, old value), so the
old value of every register the shader writes stays live across the write. That
guard is the translator's largest per-instruction cost: dropping it altogether
(wrong results) takes Silent Hill's volumetric-fog compute from 13.43 to 10.64
ms, and the exec-full analysis can only prove 194 of its 875 instructions run
with every lane set.
A run of instructions whose only effect is their vector destination can instead
run inside one if (EXEC). The masked-off lanes skip the whole run, which is what
the hardware does to them anyway, and only the join needs the old values. A run
stops at anything the wave runs even where this lane is masked off: SALU is not
EXEC masked and every invocation keeps its own copy of the scalar registers, a
compare writes VCC or EXEC (GFX10 keeps that destination in the instruction's
control, not in its destination list), a cross-lane read needs its neighbours to
have run, and LDS, memory and the DPP encodings reach outside the register. Six
instructions is where the branch starts paying for itself.
Compute only: the same grouping made the deferred lighting pixel program 28%
slower (11.03 -> 14.11 ms per scope), where a wave already runs with helper
lanes. SHARPEMU_EXEC_RUN=0 turns it off, SHARPEMU_EXEC_RUN_MIN sets the length.
Silent Hill title menu, same build, SHARPEMU_EXEC_RUN=0 vs on:
offline replay min ms fog D5DF9DC46C4E35EA 13.42 -> 12.21 (-9.0%)
FFT 674E5A7163897BC9 16.37..17.38 -> 17.06 (flat)
NGG B482CD3A01A37F0F 0.494 -> 0.492 (unchanged)
in-game ms per dispatch fog D5DF9DC46C4E35EA 16.59 -> 15.34 (-7.5%)
FFT 674E5A7163897BC9 13.81 -> 13.77
2BFB24830D8FCFAA 4.78 -> 4.58 (-4%)
in-game ms per scope 848AFAD7/9D180B8A 11.03 -> 11.36 (unchanged SPIR-V)
F987C1A4/379BE850 1.90 -> 1.72 (unchanged SPIR-V)
spirv-val passes on the fog, FFT and NGG modules; the menu image is unchanged
with Anita present.
* [SpirvTranslator] Load dword-aligned 32-bit formatted buffer components as words
A formatted untyped buffer load decodes the descriptor format at run time,
reading every component byte by byte through the generic layout select
chain. For dword-aligned 32-bit data formats (32, 32_32, 32_32_32,
32_32_32_32) with a raw-bit number format (UINT, SINT, FLOAT) a component is
one buffer word; that branch loads it directly and applies the same swizzle
and bounds rules. The descriptor is uniform, so the branch does not diverge.
UE GPU-skinning vertex shaders fetch bone data this way; Silent Hill: The
Short Message's menu goes from 1.3 to 2.0 FPS.
(cherry picked from commit 20540cd153855964d817daf065de3e243fd547bd)
* [SpirvTranslator] Convert f16 with the native pack/unpack when the host measured it exact
Every packed-f16 (VOP3P) lane widens its operands to f32 and rounds the
result back, and both conversions were the branchless integer sequences in
EmitHalfToFloat / EmitFloatToHalf: about twenty scalar ops each, because
GLSL UnpackHalf2x16 / PackHalf2x16 have no guaranteed rounding or subnormal
behaviour without float-controls execution modes. A device capability bit
does not settle it either - MoltenVK on Apple silicon reports
shaderRoundingModeRTEFloat16 and shaderSignedZeroInfNanPreserveFloat16 yet
not shaderDenormPreserveFloat16 - so only a measurement can.
ShaderCompileRequest.NativeHalfConversionExact, which the host sets only
after measuring the two ext instructions against these sequences on the
running device, swaps both helpers for the single ext instruction the
driver lowers to one hardware convert. It defaults to false, so an
unmeasured device keeps the exact emulation.
Silent Hill f2 volumetric fog dispatch: 30.7 -> 21.9 ms per dispatch in
replay, output unchanged.
(cherry picked from commit 10f76eeac0d4306172e41ecff86b737f6a3c9634)
(cherry picked from commit 7f18cc96ae58ac5fd47da6562c4ba3abe22ccb4a)
(cherry picked from commit 972f658033f8339effe42c5d305af87bcbab93f0)
* [Vk] Probe f16 conversion at device setup and report the verdict
A one-time compute pass at device setup measures the device's GLSL
UnpackHalf2x16 / PackHalf2x16 against the translator's own f16 conversion
before any guest shader is compiled, and only a clean measurement lets the
translator use them (ShaderCompileRequest.NativeHalfConversionExact). The
vectors cover all 65536 f16 bit patterns, every f16 value widened to f32,
the midpoint between adjacent f16 values and that midpoint one f32 ulp
either way, the overflow edge, signed zeroes, infinities, NaNs and f32
subnormals, and each is measured in both directions: 643200 comparisons.
Every finite value, zero and infinity must be bit-exact, sign included. A
NaN only has to stay a NaN, because neither side defines its payload: the
emulation picks sign | 0x7C00 | 0x200 narrowing and carries the f16 payload
widening, while the measured hardware convert canonicalizes to a positive
quiet NaN. On an Apple M4 that is the only disagreement - 2314 of 643200,
every one a negative NaN whose sign the hardware cleared - and f16
subnormals convert exactly despite shaderDenormPreserveFloat16 being false.
The probe never fails device setup: any error leaves the exact emulation in
place and says so on the reported line. SHARPEMU_NATIVE_HALF=0 forces the
emulation, =1 takes native without measuring. The result is one line:
[VK][F16] native half conversion exact=True mismatches=0 tested=643200
The flag reaches the translator through IShaderPipelineHost, so it is
process-constant and cannot mix two translations of one program; the saved
driver pipeline cache keys its entries by the module it was given, and the
Metal host does not probe and keeps the emulation. Offline replays run the
same probe, so a case translates the way the game would.
(cherry picked from commit 6a343683d890b4c6d123c84fdacb41e718ec4c19)
(cherry picked from commit 65ca663aa2bfb1fcc69709c73ab6189b07ead3c1)
(cherry picked from commit 003b1216bd311e5c5fb1d48ca01a66efa3fc51de)
* [SpirvTranslator] Read an unaligned buffer word as a funnel shift of two words
The byte-wise decode of a formatted buffer load read each byte of a component
from its own bounds-checked word load: four loads, four range tests and a byte
assembly for a value that spans at most two words. One funnel shift of those
two words is the same value, including out of range, where a word reads zero on
both paths and so contributes zero bytes. Every sub-dword buffer load goes
through the same helper.
The decode is the branch a formatted load takes when its address is not dword
aligned, so its size also sets the register pressure of the aligned branch next
to it. Silent Hill: The Short Message's volumetric fog dispatch
(0xD5DF9DC46C4E35EA, 8 formatted loads in its light loop) goes from 8.68 to
7.28 ms in offline replay, and the module from 394 to 360 KiB. An experiment
that dropped the byte-wise branch entirely measured 7.35 ms, so the funnel
shift is worth all of what the branch cost.
(cherry picked from commit 0c996ffab38144d795517e68cb2ca9d8d48e8988)
* [SpirvTranslator] Load a guest buffer word unchecked where the device zeroes out-of-range reads
Every guest buffer word is read through a range test, an address clamp and a
zero select: three operations whose only purpose is to produce the zero the
hardware returns past the end of a V#. A device with robustBufferAccess2 that
honours it returns that zero itself, so the three are pure overhead there -
dropping them measured about 12% of Silent Hill's volumetric fog dispatch, 482
checked words in one program.
ShaderCompileRequest.ZeroOutOfBoundsBufferReads turns them off and defaults to
false: an out-of-range read is undefined without the promise, so only a host
that measured the device may set it. It is set from a device probe, never from
a capability bit, because MoltenVK runs on Metal, which has no hardware bounds
checking.
(cherry picked from commit 2ba626a0ac7157f93f348cfe7573ff1092762d8c)
* [Vk] Probe out-of-range buffer reads at device setup and report the verdict
A one-time compute pass binds 1 KiB of a 4 KiB allocation that carries a
sentinel everywhere and reads all 4 KiB back through the descriptor: the words
inside the range must still read their sentinel and the words past it must read
zero. Only a clean measurement lets the translator drop the range test, address
clamp and zero select it puts around every guest buffer word
(ShaderCompileRequest.ZeroOutOfBoundsBufferReads).
The probe only runs where the device enabled robustBufferAccess2, because an
out-of-range read is undefined without it and can fault the device; everywhere
else it reports the reason and the checks stay. On this Apple M4 through
MoltenVK the feature is not supported at all, so nothing changes:
[VK][OOB] out-of-range buffer reads zero=False mismatches=unprobed tested=0
note=robustBufferAccess2 is not enabled on this device
SHARPEMU_ZERO_OOB_READS=0 keeps the checks, =1 drops them without measuring.
The flag reaches the translator through IShaderPipelineHost, so it is
process-constant and cannot mix two translations of one program.
(cherry picked from commit e81d924349042acc1122a66d4c767268bd6ef2a4)
* [SpirvTranslator] Keep pure scalar lane spills in private slots under emulated wave64
A wave64 program run as a 64-thread workgroup on a 32-lane host emulates
v_readlane with an LDS exchange and a workgroup barrier. Compilers use
constant-lane v_writelane/v_readlane pairs on a VGPR as SGPR spill slots;
when that VGPR is written only by constant-lane writelanes and read only by
constant-lane readlanes, each (vgpr, lane) pair is a private scalar, so the
readlane loads the slot instead of exchanging through LDS. This halves the
heaviest compute dispatch in Silent Hill: The Short Message's menu.
(cherry picked from commit 9915b88e68b6b69d6b74ea931a22eb0589c56f1a)
* [SpirvTranslator] Zero private LDS lazily through a written-dword bitmap
Graphics stages model LDS as a per-invocation private array that must
read zero where the invocation has not written. The array carried a zero
initializer, but Metal keeps a dynamically indexed private array in stack
memory, so every invocation cleared all of it: an NGG vertex shader (the
PS5 default, with in-shader culling through LDS) cleared 8 KiB per vertex.
The array is now uninitialized and a zero-initialized bitmap of written
dwords (1/32 of the size) records which slots were addressed; LdsPointer
zeroes a slot the first time it is addressed, so loads, stores and atomics
keep the zero-read semantics. Compute LDS stays workgroup memory.
Silent Hill: The Short Message's menu, dominated by ~45M shadow-pass
vertices per frame, goes from about 2.0 to 3.1 FPS.
(cherry picked from commit 5738b8574397cf2be08cd4e1518b33057c8867b5)
* [SpirvTranslator] Test the lane's mask bit in one register under emulated wave64
VCC and EXEC keep a per-lane boolean that every mask write refreshes, and the
refresh composed the 64-bit mask from its two registers, built a 64-bit lane
bit and masked them. A host without 64-bit integer arithmetic (Apple GPUs
through MoltenVK) emulates each of those with a pair of 32-bit operations. The
bit lives in exactly one of the two registers, so selecting that register and
shifting inside it is the same test in 32-bit arithmetic.
Only the emulated-wave64 path changes; wave32 already tested one register, and
a native wave64 host keeps the existing form, which also clears the bit for
lanes outside the guest wave.
(cherry picked from commit 1b2651047fdf232304142c82168e487244dda4ef)
* [SpirvTranslator] Let s_barrier publish only what the guest waited for
Every compute s_barrier became an OpControlBarrier with AcquireRelease
over uniform, workgroup and image memory, on the assumption that the
guest had drained vector memory first. s_barrier only orders execution;
what it publishes is whatever the preceding s_waitcnt drained. A barrier
that follows an LDS-only wait (lgkmcnt(0)) publishes LDS, and ordering
device memory there costs a cache flush the guest never asked for: UE's
FFT bloom convolution barriers 48 times per group, all LDS-only.
The translator now tracks, per block, whether any wait drained vector
memory - SOPP vmcnt(0), or the GFX10 SOPK vscnt(0)/vmcnt(0) forms, since
stores are counted by vscnt there - and a barrier after an LDS-only wait
carries workgroup memory only. A drained counter stays drained for the
rest of the block, and a barrier with no wait in its block keeps the
full semantics (the wait may be in a block this one cannot be ordered
against).
(cherry picked from commit 1f81df59bc07d05f2baa6283da95f4901dada444)
* [Vk] Run async readbacks on a second general queue family when no dedicated one exists
The async readback queue was only taken from a family without graphics,
so drivers that expose several general families (each its own hardware
queue) kept every guest readback on the main queue and the command
worker. Any other family with transfer support now qualifies after the
dedicated transfer and compute families.
(cherry picked from commit 7b04d6185dfc1711f0fbdf5c75c842a065acd1fa)
* [GPU] Let guest reads of GPU-written memory wait on the main queue off the worker
A guest thread reading memory the GPU wrote hands the download to the
command worker, which recorded the copies, submitted and then waited for
the whole GPU before handing the bytes back - unless a dedicated readback
queue existed, where the download ran asynchronously and only the guest
thread waited. MoltenVK exposes one queue family in-process, so on macOS
every such read idled the worker for tens of milliseconds, several times
per frame in Silent Hill: The Short Message's menu (a quarter of the
worker's time).
Without a readback queue the worker now copies the pieces into a buffer
of their own, submits without waiting and returns a pending download
carrying the submission tick; the guest thread waits for that tick on the
timeline (safe from any thread) and completes the read back on the
worker, as the readback-queue path does. The worker's GPU wait time drops
to zero; SHARPEMU_MAIN_QUEUE_PENDING_READS=0 restores the synchronous
path.
(cherry picked from commit 6c9ad44bcea7f9396a7564621fcbda3841834052)
* [GPU] Re-issue a stale pending readback instead of reading synchronously on the worker
A guest read of GPU-written memory records a copy and waits for its tick
off the queue. When another GPU write lands first, the completion fell
back to a synchronous read on the command worker, which stalled it until
the GPU drained (up to half of SH's readbacks). The guest thread now
re-issues the pending read and waits again; only the fourth attempt falls
back to the synchronous read.
(cherry picked from commit 17bf5bb958ebfd4dd980dbf241cbe9e77fffd938)
The f16 fast path shadowed the whole vector register file with a 512-entry
private array of vec2 and zero-initialised it, even though every access uses a
constant register number. That is the array shape Metal keeps in thread memory
instead of registers - the same trap V_MOVREL's dynamically indexed register
file hit (Octopath's instanced vertex shader, 112 -> 6.4 ms/draw) - and it also
left AMD with an unused 4 KiB .bss global that broke two-stage links. One
private variable per register, declared on first use, removes it.
308 of the 1019 shaders Silent Hill's title menu translates used the array,
including the two hottest pixel programs. The generated Metal for the
volumetric-fog compute D5DF9DC46C4E35EA drops from 282 KB to 269 KB and no
longer declares a 4 KiB thread array. Measured GPU time on the menu is
unchanged within run-to-run noise (fog 18.2/19.9 -> 20.1 ms per dispatch,
pixel 934F121D8F666572 0.048/0.061 -> 0.041 ms per scope), so this is a
code-shape fix that removes the spill risk rather than a measured win.
* [ShaderCompiler] Materialize directly for plans whose cached inputs never repeat
The materialization cache records every guest word a draw reads and keys
the result on the draw's user data. Most Silent Hill draws point their
user data at per-draw constants, so only 8% of lookups hit and every miss
still paid for recording the reads and storing an entry. A plan now
counts its lookups and hits; after 64 lookups with fewer than one hit in
eight it walks its resources directly and probes the cache again on every
64th draw. On the SH title menu the command worker's busy time per draw
fell from ~125 to ~111 us (cache hit rate of the remaining lookups 70%).
* [GPU] Scan a pixel program's interpolated attributes once per decoded program
Every draw scanned all instructions of its pixel program twice, once
through a LINQ chain with Distinct/Order/ToArray, to count interpolated
attributes and the vertex outputs they read. The sorted attribute set
is a property of the decoded program, so it is computed once and kept
beside it. Pixel input resolution on the SH menu dropped from ~14 us to
a few us per draw.
Draws that sample the depth attachment need its read-only layout; every
other draw got the writable layout, even one that only tests depth. A run
of depth-sampling particles between test-only draws flipped the layout on
every draw, and each flip ended the rendering scope. Image state tracking
also recorded a barrier whenever the access changed, even from one read
to another in the same layout.
A draw that neither samples nor writes the attachment now keeps the
read-only layout it is already in, and reads after reads in one layout
record no barrier: the tracked state accumulates every reader, so the next
writer still waits for all of them. Depth-transition scope breaks in
Silent Hill: The Short Message's menu drop from ~1250 to ~820 per 5 s.
(cherry picked from commit 5563417d72fadefb115b300291b1e019b3171466)
Indexed indirect draws already let vkCmdDrawIndexedIndirect read their
counts, but a non-indexed DRAW_INDIRECT read its four arguments on the
CPU. When a compute pass had just written them (UE writes rectangle-list
draw arguments this way every frame), the read downloaded the buffer and
waited for the whole GPU: about one 20-50 ms stall per frame in Silent
Hill: The Short Message's menu.
The interpreter now hands the argument address to the host when it draws
non-indexed indirect draws on the GPU (deferring the instance count read
as the indexed path does), and the executor records vkCmdDrawIndirect.
The guest layout (vertexCount, instanceCount, firstVertex, firstInstance)
is VkDrawIndirectCommand. Rectangle lists use the general rectangle-list
variant, since the count is unknown; fans, polygons, legacy lists and
targetless draws (which may be retained and replayed) still read the
arguments on the CPU. SHARPEMU_CPU_INDIRECT_DRAW=1 keeps the CPU path.
(cherry picked from commit f2b91befb7dd27d65a35546f1e57dc88f04a5c82)
The native memcpy fast path copies with rep movsb straight into guest
memory. Pages the GPU reads are write-protected so CPU writes can be
tracked, so a copy into them takes one protection fault per page, and
under Rosetta each fault is a signal round trip of about 0.2 ms. The
managed memcpy marks the whole destination written before it copies and
takes no faults. Copies of at least 4 KiB now leave the stub for the
managed export; smaller ones keep the native copy, where one import
transition would cost more than the copy. SHARPEMU_MEMCPY_MANAGED_MIN
overrides the threshold.
Demon's Souls, 230 s through the PlayStation Studios intro: 536k
write-tracking faults become 71k, and the intro presents about a fifth
more frames.
* [Vulkan] Enable shader clip and cull distances
* [Vulkan] Track shader clip distance support
* [ShaderCompiler] Add a clip distance support flag to compile requests
* [GPU] Expose host clip distance support to pipelines
* [GPU] Pass clip distance support to shader compilation
* [ShaderCompiler] Reserve a clip plane for position exports
* [ShaderCompiler] Cull primitives with a zero position
* [GPU] Bind vertex buffers with their guest descriptor sizes
* [GPU] Accept B10G11R11 float vertex attributes
* [GPU] Spill oversized image readbacks to a temporary buffer
* [GPU] Carry DCC metadata on compressed textures
* [GPU] Mark sampled DCC clears through the metadata fill path
* [Tests] Cover vertex descriptor bounds and oversized image readbacks
* [GPU] Revert binding vertex buffers with their guest descriptor sizes
* [GPU] Carry DCC metadata only on single-layer 2D textures
* [Tests] Cover DCC metadata on 2D, 3D and array textures
Each shader resource plan is compiled once to a DynamicMethod that evaluates
its resource table graph with plan-local value slots instead of the
interpreter's hashed cache. ASTRO's PLAYROOM goes from about 20 to 60 fps.
Compilation runs on a low-priority "SRT compiler" thread; until a plan is
ready, the interpreter evaluates it, so a new shader no longer stalls the
render thread while its plan is JIT-compiled. A plan whose compilation fails
stays on the interpreter. SHARPEMU_SRT_NATIVE=0 turns the compiled path off;
it is also off when the runtime does not support dynamic code.
* [DeS] Fixes for page faults and stack bounds issues in the emulator and audio fixes
* [ShaderPrewarm] Hold the game until the prewarm finishes and resume interrupted runs
The prewarm compiled thousands of compute pipelines on background threads
while the now faster boot ran beside it, so logos, Bink movies and the first
scenes raced the compile and dropped frames.
- Hold the guest at its first splash hide, GPU submission or CPU flip until
the prewarm is decided and done; the render thread keeps presenting the
splash. SHARPEMU_SHADER_PREWARM_WAIT=0 opts out.
- Stamp only the shader translators, the compiler type and the driver, so a
Libs-only build no longer triggers a full prewarm.
- Record finished pipelines in shader-prewarm.progress so an interrupted
prewarm resumes with the remaining ones.
* [AudioOut2] Block a blocking push only until a queue slot frees
A blocking sceAudioOut2ContextPush waited only when the host stream hit its
128 KiB cap, so a title rendering faster than real time kept about 680 ms
queued and every sound played that far behind (GTA V: queue=2 context,
queued_ms avg=678). The push now waits until the device holds the cushion
plus QueueDepth-1 grains, the level GetQueueLevel already reports as full;
the host cap stays as the limit for non-blocking bursts. GTA V now holds
about 41 ms with no underruns.
* [GPU] Run bounded modulo copies of plain words on the host
Demon's Souls copies its per-frame constant and descriptor tables with a
compute kernel (dst[i] = src[i % N] for i < count). Run on the GPU, the
destinations stay GPU-dirty, and every draw whose resource tables read them
waits for a readback: about 19% of the render thread in the title scene,
for values that never changed.
The kernel is now recognized by its instruction sequence. When both
descriptors are plain 32-bit words of the same format and neither range is
GPU-dirty or GPU image memory, the copy is written to guest memory through
the host DMA path and the dispatch is skipped; anything else still runs on
the GPU. SHARPEMU_HOST_BOUNDED_COPY=0 turns it off.
Title scene with profiling: 7.4-8.6 -> 11.8-15.9 fps.
* [Diagnostics] Summarize synchronized table reads and trace several writer addresses
SHARPEMU_TRACE_GPU_READS=1 now also reports, every five seconds, the guest
words resource planning read through a GPU synchronization: read count, how
often the value changed, and the first and last value. SHARPEMU_TRACE_WRITER_ADDRESS
accepts a comma-separated list.
Upstream e7a4f0a0 started reporting HDR to the guest whenever the display
has HDR enabled. On an HDR Mac, Silent Hill switches its output to HDR,
a guest pipeline then fails to compile in MoltenVK (ushort4 output to a
BGRA8 attachment) and the process crashes inside MoltenVK on every run.
Auto now stays SDR on macOS; --hdr=on still opts in.
* [ShaderCompiler] Fix packed 10 and 11 bit buffer format layouts
* [SpirvTranslator] Pass W as the pixel position input instead of its reciprocal
* [SpirvTranslator] Keep lane spills in per invocation slots for all 64 lanes
* [SpirvTranslator] Match the hardware float conversion and NaN rules
* [SpirvTranslator] Apply VOP3 sign modifiers to conditional mask sources
* [GPU] Download guest DCC metadata before a known target is sampled
* [ShaderCompiler] End bit scan image tables at a sampler loaded from the same base
* [ShaderCompiler] Treat image table candidates with unsupported formats as null
* [Gpu] Recognize AGC's loaded-value buffer fill as an image/metadata clear
Octopath clears its DCC with a shader that s_buffer_load_dwords the fill value
from a one-record buffer and stores it at every thread index; only the variant
with the value in user data was recognized, so the DCC fast clear of UE's
translucency layer was lost and the layer kept stale alpha 0, which erased the
world map under it in the composite.
* [Vk] Bind descriptor sets instead of push descriptors on MoltenVK
MoltenVK returns zero from OpArrayLength for storage buffers bound through
push descriptors, so every bounds-checked buffer load read zero and every
store was dropped. In Poppy Playtime this left UE's motion blur indirect
dispatch arguments unwritten; the GPU then ran a 4294967295x1612x1490
dispatch and hung, and the game aborted on the render thread timeout.
* [ShaderCompiler] Size compute LDS to the dispatch's allocation
Every compute module declared the full 32 KiB of workgroup memory, which on
Apple GPUs leaves one workgroup resident per core. Declare the dispatch's
COMPUTE_PGM_RSRC2.LDS_SIZE (plus the wave64 bridge dwords), rounded to a power
of two so the index mask still bounds accesses. Silent Hill's heaviest per-pixel
pass (LDS_SIZE=4) replays in 11 ms instead of 46 ms.
* [Loader] Guard frame-pointer spill slots that live in the guest red zone
Host signal frames are pushed onto the guest stack, so GuestRedZonePatcher
protects functions that keep live data in the 128-byte red zone. It only
recognized RSP-relative accesses. Clang spills just as often through the frame
pointer: after 'push rbp; mov rbp, rsp' and N callee-saved pushes, RSP sits at
RBP-N and every [rbp-d] with d > N is below RSP. A fault taken there (lazy
commit, write tracking) overwrote those slots, which showed up as garbage
pointers on random threads in Poppy Playtime's level load.
Derive RBP-to-RSP from the standard prologue (leading pushes and the first
'sub rsp, imm') and treat [rbp-d] in (delta, delta + 128] as red-zone use. An
under-estimated delta only widens the guarded window. In Poppy's eboot this
protects 787 functions instead of 485.
* [AvPlayer] Report a loaded source as active before playback starts
sceAvPlayerIsActive answered true only after sceAvPlayerStart. Titles poll it
after adding a source and before starting playback; Superliminal waits for it
to turn true before it calls Start, so New Game never left the loading screen.
Report a player as active from the moment its source is loaded until it is
stopped or reaches its end, and track Stop explicitly so a stopped player
stays inactive.
* [Gpu] Copy the mips of a GFX10 mip tail block written through its own view into the chain
GFX10 packs every mip smaller than half a swizzle block into one block at the
start of the chain. A title can render into that block through a single-level
view exactly the size of the block: Unity's screen-space reflection blur writes
mips 3 and up of its 1024x512 color pyramid through one 128x64 view. The cache
keeps that view as its own image, so the chain's tail mips kept stale data and
sampling them produced rainbow blobs over Superliminal's menu.
When a chain is acquired with data, look for a GPU-written single-level image
at the chain's address with the tail block's extent, format and tile mode, and
copy each tail mip out of it at its placement inside the block. A per-image GPU
write sequence makes the copy happen once per new write.
* [SpirvTranslator] Give each used register its own private variable
The translator kept the scalar and vector register files as two private arrays
(128 and 512 uints) in every module. MoltenVK lowers a private array that size
to per-thread memory, so every register read and write in every invocation went
through memory. Declare one private variable per register the program touches,
on first use; the SPIR-V is otherwise unchanged.
* [ShaderCompiler] Resolve V_MOVREL* with selects instead of a spilled VGPR array
A program with any V_MOVREL* instruction declared its whole VGPR file as a
512-entry private array indexed at run time, which Metal spills to thread
memory, so every register access became a memory access. Keep one variable
per register and resolve relative accesses with a select over the registers
the program names. Octopath's instanced vertex shader (1536 x 8124): 112 ms ->
6.4 ms per draw in replay, output byte-identical.
* [AvPlayer] Start a player on its own when the title registers no event callback
Reporting a loaded source as active before Start regressed Quake, which
waits for its player to go inactive before starting its queued intro
videos. A player left loaded but unstarted stays active forever. Only a
title that registers an event callback can see the ready state and call
sceAvPlayerStart itself, so a player whose title asks for autoStart or
registers no callback now plays by itself once its source is ready,
reaches the end of the stream and reports inactive. Titles that start
playback themselves keep seeing a loaded source as active.
* [Gpu] Recognize AGC's constant buffer fill as an image/metadata clear
Octopath clears the DCC of its 1920x2200 DOF/bokeh accumulation target with a
third fill kernel: v_mov_b32 v1, 0 and buffer_store_format_x v1 at the thread
index, with only the destination descriptor in user data. It was not
recognized, the DCC fast clear was lost, and the additive bokeh pass kept
blending onto the previous frames until the target held thousands and inf,
which showed as noise and banded garbage over every 3D scene.
Program info now carries the constant a program stores when it is only index
arithmetic, one v_mov of a constant and one buffer store; a one-buffer 32-bit
formatted fill over every record with that value becomes a clear.
* [Vk] On macOS, retire a GPU tick only once its submission's fence signals
MoltenVK records Metal command buffers with unretained resource references
and signals a Vulkan timeline semaphore from a Metal event scheduled at the
end of the command buffer. That event can fire before Metal has finished
executing the command buffer, so a host thread that treats the timeline
value as authoritative can free resources the GPU is still reading. Metal
API validation reports this as "destroyed while still required by a command
buffer", and the resulting use-after-free showed up as intermittent GPU
tick stalls.
Every submission that signals the timeline now also signals a VkFence, which
MoltenVK signals from the Metal command buffer's completion handler. The
timeline value a host thread is allowed to act on is clamped to just below
the oldest tick whose fence has not signaled yet, and a timeline wait is
followed by a blocking wait on the last fence covering that tick. Fences are
pooled, tracked before the submit so no reader can observe a signaled tick
without its fence, and reset only once no thread is blocked on them. Fences
are polled at all only when the timeline already claims the tick is done, so
the common path costs one queue peek. The behaviour is macOS-only and can be
turned off with SHARPEMU_FENCE_RETIREMENT=0.
* [Vk] Refuse push descriptors on macOS for every device through one policy
The guard that refuses push descriptors on MoltenVK keyed on the
VK_KHR_portability_subset device extension, but that extension is not
enumerated for every device: the headless device the draw replay and the
device tests create took the raw driver limit, so replays on macOS read
guest storage buffers past their first bytes as zero (MoltenVK reports a
zero OpArrayLength for buffers bound through push descriptors).
The limit now comes from one policy both the presenter and the headless
device use, and macOS, whose only driver is MoltenVK, pushes none
regardless of which extensions are listed.
* [SpirvTranslator] Order emulated wave64 LDS phases in every block
A guest wave64 spans two host subgroups. The guest orders one lane's LDS
write before another lane's read by executing in lockstep, so the translator
puts a workgroup barrier between the halves wherever the wave switches
between writing and reading LDS. It only did so for programs of a single
block, which left every program with branches -- an NGG geometry stage
compute program has seventeen exec-skip branches around its ES to GS LDS
exchange -- reading the other half's LDS words with no ordering at all, so
the result depended on timing.
Every guest branch condition is wave-uniform in this emulation (EXEC and VCC
tests go through the wave mask and SCC is scalar), so every block is entered
by the whole wave and may hold a barrier, exactly as the wave64 ballot
exchanges already do. A block can be reached from either phase, so its first
LDS access always starts with a barrier.
* [Vk] Keep the rendering scope open across barriers and guest cache flushes
Tile-based GPUs pay for every rendering scope: ending one stores the
attachments out of tile memory and beginning the next one loads them back.
The presenter ended the scope for three things that do not actually need it.
An image barrier that repeats an attachment access in the same layout only
matters to work that runs after the scope, because accesses to an attachment
inside one scope are already ordered by rasterization order. Such barriers
are now queued and recorded as the scope ends instead of ending it.
A guest cache flush (a global memory barrier) inside a scope whose draws only
wrote attachments is deferred the same way, and every scope now opens with
that global barrier so anything recorded before the scope stays ordered
before its draws. A draw whose vertex or pixel stage stores to buffers,
storage images or device addresses ends a scope with a deferred flush first,
so a store can never move ahead of it. Finally, the DMA fill and copy entry
points no longer end the scope up front: the buffer cache completes most
transfers in guest memory, and the paths that do record GPU work end the
scope themselves.
SHARPEMU_DEFER_GLOBAL_BARRIERS=0 restores ending the scope at every flush.
* [SpirvTranslator] Take wave32 EXEC/VCC lane state from the low register
In wave32 a lane mask is its low register alone: EXEC_HI and VCC_HI are
ordinary SGPRs. Writing one of them therefore must not recompute the wave
lane state, and the lane state has to be tested against the low dword rather
than against a 64-bit mask composed from the low and high registers.
The lane-bit test that guards this covered the wave64 constant width only, so
it now accepts a mask of either width.
* [AudioOut2] Pace one grain per Advance+Push cycle and report the queue level
sceAudioOut2ContextAdvance and sceAudioOut2ContextPush each submitted the
context's pending PCM and, whenever nothing reached a host device, each slept
a whole grain in the software pacing fallback. A title that calls Advance then
Push once per grain therefore spent two grain periods per grain, so its audio
thread ran at half real time and held up every thread waiting on it.
Advance now claims the next grain on the context's clock and returns without
blocking (or, when its own submit already blocked in the backend, marks the
grain served), and Push waits for the grain Advance claimed; a Push with no
preceding Advance claims one itself. Neither call skips pending PCM. Because
the output holds QueueDepth grains, a grain may be queued as soon as the grain
QueueDepth-1 ahead of it starts playing, so a context can run that far ahead
of real time before a push blocks.
sceAudioOut2ContextGetQueueLevel reported an empty queue with the full depth
available on every call. Titles size their rendering by those two values --
Wwise renders exactly puiAvailableQueues grains per update and pushes them
non-blocking -- so a queue that never filled made it render a full queue every
update and then block inside Push for a grain per push while holding its audio
lock, which starved the game thread. The level now comes from the context's
grain clock, or, once a grain has reached a host device, from that device's
queued milliseconds beyond a short cushion.
* [Vk] Keep one depth attachment layout for draws that do not sample it
The depth attachment layout followed each draw's depth/stencil write mask, so
a draw sequence that toggles depth writes — opaque geometry, then decals and
translucency — changed the layout every few draws, and every layout change
ended the rendering scope.
The read-only attachment layouts are only required when the draw also samples
the attachment. Every other draw now uses the writable layout for the
aspects the format has, and the write mask alone selects the access, so the
layout stays constant and the scope survives. The presenter answers whether a
texture bound for the draw being prepared views the depth attachment's
subresources, defaulting to yes for hosts that do not track it.
* [Audio] Sleep for the SDL queue's drain time instead of polling every ms
Submit's backpressure wait is what paces every audio grain on the SDL
output, and it polled the queued byte count with Thread.Sleep(1), costing
the audio thread several wakeups per grain.
It now sleeps for the time the device needs to drain back under the queue
cap, still bounded by the existing deadline and the maximum wait.
* [CLI] Shut down cooperatively on SIGTERM
SIGTERM (launchd, process managers, kill) used the runtime default and
exited with GPU work in flight. It now takes the same host-interrupt path
as Ctrl+C: the presenter closes and tears Vulkan down before exit.
* [SpirvTranslator] Fold constant integer and boolean instructions
The translator materializes a good deal of translation-time knowledge as
SPIR-V instructions: a specialized descriptor's buffer format, swizzles,
strides, and the conditions that guard host-guaranteed ranges all end up as
arithmetic, comparisons and selections over operands the builder itself just
emitted as constants. That reaches the driver as runtime work, and on Metal
every one of those instructions costs pipeline-creation time.
The module builder now remembers the value of each 32-bit integer and boolean
constant it emits and folds pure instructions over them while it builds:
integer arithmetic, bit operations, shifts, bitfield extraction, the integer
and logical comparisons, the logical connectives, and a selection on a known
condition. Folding happens at emission, so a folded result feeds further
folding without a separate pass. Anything whose operands are not all known
constants is emitted exactly as before.
The buffer format decoder uses this directly: a specialized descriptor makes
the unified format a constant, so it decodes on the host instead of emitting
a module-level lookup table and an indexed read.
The test that covered that path asserted the table's presence and now asserts
its absence.
* [SpirvTranslator] Skip host-guaranteed bounds checks and unused LDS
Two sources of instructions the driver has to compile but can never need.
A read of a planned slot in the host-flattened resource table is always
inside the table the host binds, because the table holds every planned read
and written range. Those loads nevertheless carried a runtime array-length
compare and two selects each; a large pixel shader made about 376 of them.
A slot below the planned reserved word count now loads directly, and only a
slot computed at run time keeps the check.
ds_swizzle and ds_bpermute exchange data between lanes and GDS instructions
address the global data share, so none of them touches LDS. A graphics stage
that uses only those was still given the 8 KiB zero-initialized private LDS
array, which Metal pays for in compile time and in private memory per
invocation. The array is now declared only for instructions that really
address LDS.
* [Cpu] Answer pthread_self and pthread_getspecific in native stubs
The managed import gateway costs a guest-to-host stack switch, a full guest
register pack, a GCHandle dereference and the whole dispatch bookkeeping
before the HLE export even runs. For the two hottest trivial imports that is
three orders of magnitude more work than the answer itself: one UE5 title's
load measured 12.9M scePthreadGetspecific calls (38 s of wall time) and 4.3M
scePthreadSelf calls, at roughly 2.9 us each.
The few values those two imports need are now published into a per-host-thread
native block whose address lives in a native TLS slot, rebound whenever a
thread enters or is restored as a guest thread, and the two imports are
patched to emitted x86-64 stubs that read the block directly. pthread-specific
values live in native per-thread tables so a stub and the managed export read
exactly the same storage and cannot drift apart. The managed path stays the
source of truth for every case a stub declines -- no block published, no guest
thread bound, no value table, a key outside the table -- which it reaches by
jumping to the import's normal trampoline with every register intact,
including the incoming RAX variadic vector count.
The value-table slot follows whichever handle the current thread reports, the
bound guest thread or the host thread's own synthetic handle, so
pthread_getspecific is answered on unbound host threads as well; the primary
execution thread never enters through the guest-thread path and is a major
caller in UE5 titles. The self-handle slot stays guest-only, because
pthread_self on an unbound thread registers that thread's CPU context with the
guest scheduler and must keep reaching the managed export. Each stub also
bumps a per-thread hit counter, reported next to the existing HLE cost
histogram, so a call that misses the stub is visible rather than invisible.
The stubs are gated to macOS x64, where the emitted TLS read matches the
platform's pthread key array, and switched off whenever import or pthread
tracing needs every call to pass through the gateway.
* [CPU] Run libc memcpy/memmove as a native guest-side stub
Guest memcpy went through the managed gateway on every call: two write
tracking notifications, the memory reader lock and two region lookups, about
0.8 us per call under Rosetta even for a copy of a few bytes. Dreaming Sarah
makes roughly 45k of those calls a second.
Guest memory is identity-mapped, and every fault a copy can take -- write
tracking, lazy commit -- is resolved from the fault address alone, so the copy
can run as a forward rep movsb in a fast-path stub. A destination that sits
inside the source, where a forward copy would read bytes it has already
written, and the write-watch diagnostic both keep the managed export.
* [SpirvTranslator] Fill the FRONT_FACE and ANCILLARY pixel inputs
A pixel shader that enables FRONT_FACE or ANCILLARY reads those VGPRs, but the
translator only reserved their compact slots and left them reading zero.
Unreal Engine bakes its colour grading LUT one volume slice per instance and
has the pixel shader read the slice back out of ANCILLARY[28:16], the render
target array index. With that register zero every slice was baked from the
blue = 0 slice, so the tonemapped image lost its blue channel.
ANCILLARY is now the Layer built-in placed at bit 16, and FRONT_FACE the
FrontFacing built-in mapped to 1.0f for a front face and 0 for a back face,
which satisfies both the float greater-than and the integer not-equal tests
compilers emit for it.
* [SpirvTranslator] Fill the SAMPLE_COVERAGE and POS_FIXED_PT pixel inputs
The two remaining reserved pixel system inputs were also left reading zero.
SAMPLE_COVERAGE now reads the first element of the SampleMask built-in, and
POS_FIXED_PT packs the integer pixel position taken from FragCoord, x in bits
[15:0] and y in bits [31:16], so no enabled pixel system input is left unset.
* [CPU] Run the POSIX signal handler on the host stack for guest faults
Guest faults are delivered on the guest stack, and the managed handler ran
there. The runtime only knows the thread's own stack; Windows swaps the TEB
stack bounds while guest code runs, but POSIX has no equivalent. A garbage
collection during a guest fault then misread the thread's frames and
reported garbage object references, and the process failed with "attempted
to call a UnmanagedCallersOnly method from managed code". A small native
trampoline now moves to the host stack below the frames that entered guest
code before it calls the managed handler.
* [VideoOut] Give every swapchain image its own sRGB-encode intermediate
A float guest flip is blitted through an sRGB image before it is copied into
the UNORM swapchain image, and one such intermediate served every frame. Each
frame discards its contents by transitioning it Undefined -> TransferDst
before blitting, so with two frames in flight the next frame's blit raced the
previous frame's copy-out and the presented image came out fully zeroed. The
guest's own display buffers held the correct picture the whole time, so the
window simply went black for long stretches.
Each swapchain image now owns its own intermediate, created and destroyed with
the swapchain, so a frame can only ever discard the intermediate belonging to
the image it is about to present.
On Superliminal the same run went from 60 of 84 presented-swapchain readbacks
black to 10 of 187, all of them from before the menu appears, and the menu no
longer turns black after a couple of minutes.
* [VideoOut] Present a flip only after the tick that copied it
CaptureFlip recorded the display-surface copy into the guest batch, flushed
it, and then read the shared _submitTimeline field for the presentation's
RequiredTick. The presenter thread writes that same field from its own
submission, so the flip could be tagged with an older tick that was already
complete. The render loop then blitted the freshly created (never written)
flip snapshot, which presents as fully zeroed RGBA - a black frame with an
alpha of 0. Take the tick that this flush returned instead.
* [VideoOut] Pool the per-flip display-surface snapshot images
CaptureFlip created a 1920x1080 RGBA16F VkImage with its own VkDeviceMemory
for every flip and destroyed both when the flip retired: around twenty
vkCreateImage plus vkAllocateMemory pairs a second for the same extent, and on
MoltenVK as many Metal texture allocations.
Retired snapshots are now kept in a small per-format, per-extent pool and
handed back instead. A snapshot is fully overwritten by the copy that follows
it and its barrier takes it from Undefined, so neither its previous contents
nor its layout matter. The pool holds at most eight images, is guarded because
the command-stream thread takes snapshots while the render thread retires
them, and is drained at teardown. SHARPEMU_FLIP_SNAPSHOT_POOL=0 turns it off.
* [CPU] Bridge SIGFPE into the guest fault handler so #DE crashes are reported
The POSIX bridge installed handlers for SIGSEGV, SIGBUS, SIGILL, SIGTRAP and
SIGABRT, but not SIGFPE. A guest integer divide by zero therefore took the
runtime default and killed the process with no exception record and no guest
context, so the faulting instruction was invisible.
SIGFPE now routes to the same handler and is reported as an integer divide by
zero. Like SIGILL, it skips the image write tracker, which only ever resolves
data-access faults.
* [Cpu] Resolve GPU-tracked page faults before building the exception context
A CPU access to a page the GPU memory tracker protects faults on every first
touch after an upload or a GPU write, thousands of times a second in a frame.
The POSIX bridge converted each one into a full Win64 CONTEXT (general and
vector registers), then walked the interrupt, image-tracker, lazy-commit and
allocator-hole handlers before the tracker ever saw it.
The tracker needs only the fault address and the access kind, so it now runs
right after the image tracker, straight off the raw signal. An env var
restores the old handler order.
* [CPU] Serialize import stub and lazy TLS site patches
Patching a guest code site unprotects the page it lives on, writes, and
restores the old protection. Two threads patching sites that share a page
therefore race: one restores the page to read-execute while the other is
still writing into it, and the second write faults or is lost. Both happen in
practice -- a runtime module load sets up import stubs while guest threads are
running, and guest threads patch their own lazy TLS access sites as they first
reach them, several of which land on one page.
Every import stub patch and every lazy TLS site patch now runs under a single
gate, so the unprotect/write/restore sequence is never interleaved.
* [SpirvTranslator] Follow guest NaN and saturation rules for float ops
v_min_f32 and v_max_f32 return the other operand when one of them is NaN, and
v_cvt_u32_f32 and v_cvt_i32_f32 saturate an out-of-range value to the target
type's limit and turn NaN into zero. The GLSL FMin and FMax instructions and
SPIR-V's ConvertFToU and ConvertFToS all leave those cases undefined, so a
host driver is free to hand back the NaN or an arbitrary integer, and Metal's
fast math does return the NaN.
The minimum and maximum now select the other operand when either side is NaN,
and the two conversions clamp at the type's limits and return zero for NaN.
NaN is tested on the value's bits rather than with an ordered comparison, so
fast math cannot assume the test away.
Device tests cover this and the wave-wide maximum that Unreal's motion blur
computes with DPP row_shr, v_permlanex16_b32 and v_readlane_b32 across a
64-lane wave.
* [ShaderCompiler] Fix the SOPK immediate width and S_ADDK_I32's SCC
Two things were wrong in the SOPK forms, in both the SPIR-V and the MSL
backend.
S_CMPK_*_U32 compares against SIMM16 zero-extended; only the signed compares
sign-extend it. Both translators sign-extended it for every compare, so
"s_cmpk_le_u32 s4, 0xffff" was true for any value at all. NGG geometry
programs use exactly that test on the packed vertex and primitive count, so
they took their no-primitive path and exported a single null primitive, which
left Unreal's volume passes empty. The register S_CMPK names is also only
read, yet user-data liveness recorded it as a definition, so a compare on a
user-data register left that register unloaded; it now counts as a use, as
does the register of the SOPK accumulations.
S_ADDK_I32 is defined as D.i = D.i + signext(SIMM16) with SCC set on signed
overflow, the same SCC rule as S_ADD_I32, which both backends already
implement. Neither wrote SCC for the SOPK form, so a following
S_CBRANCH_SCC0/1 or S_CSELECT read whatever the previous instruction had left
there. S_MOVK_I32 and S_MULK_I32 correctly leave SCC alone.
* [ShaderCompiler] End the decode at S_CODE_END
S_CODE_END pads the space after a shader's last instruction. When the code
after the final S_ENDPGM ends in a backward branch, the linear decoder walks
straight into that padding and failed with an unknown SOPP opcode 0x1F, which
stopped the session.
The decoder now names 0x1F and ends the program there, unless a forward branch
still targets code beyond it -- in which case the padding really would be
inside the program and the decode still fails.
* [ShaderCompiler] Widen host-evaluable scalar operations for descriptors
A resource descriptor the host materializes is built by replaying the shader's
scalar operations on the host. The validator that decides whether a value can
be replayed listed only part of what the scalar evaluator implements, so a
descriptor computed through any of the others was rejected even though the
host can compute it exactly.
The list now covers every pure scalar function the evaluator supports: the
remaining integer maximum, absolute value and high multiplies, the signed and
unsigned comparisons, 64-bit equality, the bit-counting and bit-scanning
operations, the quad mask, the integer/float conversions, and the float
arithmetic and comparisons. Anything with side effects or a lane-varying
result is still rejected.
* [Gpu] Stop draining the GPU for predication that asked not to wait
PredicationZPassWaitOp is 0 for kWaitForQueryResults and 1 for
kDoNotPredicateIfQueryResultsNotReady, so the wait bit was being read
inverted. Every SET_PREDICATION packet that asked the command processor not to
stall drained the whole host queue instead, and the packets that did ask to
wait read the predication label without waiting for the results at all, so
they could act on a stale value.
The drain now happens only for kWaitForQueryResults, and only on the branch
that actually reads the label. A packet tagged
kDoNotPredicateIfQueryResultsNotReady leaves its work unpredicated, which is
what the hardware does while the results are still outstanding, and clearing
predication reads no memory at all so it never drains.
* [Agc] Queue the suspend point instead of draining the command stream
A suspend point inserts GPU commands that drain the graphics pipe. The call
that inserts it does not wait for those commands, and the hardware allows one
suspend point to be in flight, but the queue implemented it by blocking the
calling thread until every accepted submission had been processed.
The frame boundary is now a queued submission of its own, ordered behind the
graphics work it ends, and processing it flushes what the frame recorded. The
caller returns immediately for the first boundary and only waits when a second
one arrives while the first is still pending. Shutdown still drains, because no
slice will run a queued boundary once the queue stops accepting work, and a
cancelled queue drops pending boundaries along with the rest.
Measured on Octopath Traveler II: the call went from 223 ms to 72 ms, and the
guest render thread spent 38 s instead of 107 s of its 170 s inside it.
* [Cpu] Give guest execution threads a 64 MiB host stack for nested continuations
Every guest call that blocks in an HLE import resumes through a managed
continuation frame, so a guest thread that blocks deep inside its own call
tree nests host frames instead of unwinding them. Demon's Souls' render
thread overflowed the default thread stack within a minute of reaching its
main loop.
Guest execution threads now reserve 64 MiB. The reservation is virtual, so a
generous stack costs nothing until it is touched.
* [Gpu] Report pipeline creations that take longer than 250 ms
A host shader compiler can spend seconds inside vkCreateGraphicsPipelines or
vkCreateComputePipelines for one translated program, and the guest's command
stream cannot advance while it does. There was nothing in the log to say that
had happened, so the stall looked like the game hanging.
Pipeline creation is now timed, and any creation of 250 ms or more is reported
with the stage hashes, the SPIR-V byte count of each module that went into it,
and the running total spent creating pipelines, which is what identifies the
program to look at.
* [Gpu] Compile compute pipelines off the command-stream thread
A host shader compiler can spend tens of seconds on a single large translated
compute program, and the command stream could not advance while it did, because
the pipeline cache holds its lock across the creation call. Every other queue
that wanted any pipeline waited behind it too.
The pipeline host can now report that a compute pipeline is not ready yet. The
Vulkan presenter answers that way the first time it sees a program: it prepares
everything the command stream owns (descriptor set layout, pipeline layout,
shader module) up front and runs only vkCreateComputePipelines on a worker
thread, so the task touches nothing else. The number of compiles in flight is
bounded, since each one blocks its thread inside the host compiler and a frame
can touch many programs at once. Asking again returns the finished pipeline
without starting a second compile.
The dispatch waits for its pipeline rather than being dropped: a guest that
does not replay a one-shot dispatch deadlocks, because the dispatch can feed a
label a later packet waits on. Asking again from the executor keeps that wait
outside the pipeline cache's lock. Hosts that compile cheaply keep the blocking
behaviour through the interface default, and
SHARPEMU_ASYNC_COMPUTE_PIPELINES=0 compiles inline.
* [GpuMemory] Stop allocating on every tracked CPU write and shader identity hash
MarkCpuWrite runs on every guest CPU write into tracked memory — HLE register
patching, memcpy, index buffer setup. Its flush lambda and the region visitor
inside InvalidateRegion were both capturing closures, so every such write
allocated two closure objects and two delegates even when nothing needed
flushing at all.
InvalidateRegion now has a state-passing overload built on a ref-state
VisitRegions, and both call sites use static lambdas. The public
Action-based overload is kept for the callers that are not on a hot path.
ShaderIdentity.Compute is the other one: it allocated a fresh XxHash3 and a
byte[] per code range on every draw of a shader that carries no declared hash.
It now resets a per-thread hasher and reads each range into a pooled buffer.
* [Gpu] Allocate nothing per host DMA fill or copy
A DMA fill the buffer cache completes on the host allocated a 16 KiB array for
every call, even though almost all of them are label-sized, and a host copy
allocated a 64 KiB array per call. WriteHostMemory then scanned every
registered buffer for each write it made.
The fill builds its repeating chunk in a stack buffer sized to the fill, the
copy rents its staging buffer from the shared pool, and WriteHostMemory walks
only the registered buffers the write actually overlaps, which it can do
because the registry is ordered and disjoint.
* [Audio] Fix out-struct sizes and layouts in AudioOut, Ajm and AudioPropagation
sceAudioOutGetPortState wrote 16 bytes and put the volume in a single byte at
+7, which is inside rerouteCounter. SceAudioOutPortState is 0x20 bytes with
volume as an int16 at +4; the export now writes the whole struct with
volume = -1, which is what the library reports for anything but the pad
speaker. Its guest-stack bail is gone for the same reason it is going in
libSceAudioOut2: the call names one out pointer and its host address range is
not a signal.
ClearAjmBatchError zeroed 24 bytes, the naive sum of the field widths.
SceAjmBatchError is 0x20 bytes because its two pointers force 8-byte
alignment, so pJobOriginRa lands at +0x18 and kept stale guest bytes across
every batch start and wait.
sceAudioPropagationSystemQueryMemory wrote a {size, alignment} pair at +0x00
and +0x08, which is not what SceAudioPropagationSystemMemory holds there: that
is the struct descriptor the caller has already initialized. The two size
fields live at +0x18 (CPU) and +0x28 (GPU), and the descriptor is now left
alone.
* [SpirvTranslator] Emit forward, nested control flow as structured selections
Every program with more than one basic block ran as a loop around a switch on
the guest PC. That shape keeps every guest register live across iterations and
stops the host compiler from optimizing across blocks, so on Metal the register
files spill.
A program whose branches only skip forward over properly nested ranges -- an
s_cbranch_* skip, an if/else with an s_branch over the else range, a skip to
the end of the program -- is exactly a nest of SPIR-V selections, and is now
emitted as one. The structuring is attempted first without emitting: if any
branch goes backwards, leaves a range it entered, or targets an instruction
that is not a block boundary, nothing is emitted and the program keeps the PC
dispatcher, so no program loses a shape the dispatcher handled. Setting
SHARPEMU_STRUCTURED_CF=0 forces the dispatcher for every program.
Measured on two replayed programs, with identical output: an instanced vertex
shader 8.1 -> 4.8 ms and a per-pixel compute program 10.7 -> 6.6 ms.
* [Agc] Batch the interpolant mapping's guest reads and register writes
sceAgcCreateInterpolantMapping and its version 2 run once per draw. Each one
read every semantic word with its own guest read — the output semantic table
was re-read from the start for every pixel input — and then wrote the 32
SPI_PS_INPUT_CNTL (register, value) pairs as 64 separate four-byte guest
writes, each paying the full guest write path.
Both semantic tables are now read once, the register block is built on the
stack (the identity pairs first, then the computed values over them), and the
whole block goes out in a single guest write. The output table is rented from
the pool when it is larger than a kilobyte. The bit arithmetic is unchanged,
and a new test pins version 1's output to the values the per-word
implementation produced.
Measured on Dreaming Sarah in-game: 8.6 us to 1.7 us per call, which is 5.0 s
down to 1.0 s of the main thread over about two minutes.
* [AudioOut2] Take every out pointer and struct size from the ABI
Out-parameter handling in libSceAudioOut2 was inferred at runtime: a helper
picked whichever of two argument registers looked like a writable pointer, and
another decided from the host address range whether a pointer was a guest
stack slot. Neither is a signal. The ABI names exactly one out pointer per
argument slot, and guest stacks sit at 0x7FFF_xxxx_xxxx on Windows but
0x6FFF_xxxx_xxxx on macOS, so the heuristic silently changed behaviour per
host. Every export now reads its out pointer from the register the prototype
gives it: ContextQueryMemory from rsi, PortCreate from rdx, PortGetState from
rsi, GetSpeakerInfo and GetSystemState from rdi.
The written sizes are now the real sizeof() of each struct.
SceAudioOut2PortState is 0x40 bytes, not 0x20. SceAudioOut2SpeakerInfo is
0x50, not 0x20, and its contents were invented; it now reports a TV speaker
type with front-left and front-right and the two stereo speaker angles.
sceAudioOut2SpeakerArrayCreate follows its real signature (pHandle,
pVbapParams, pAmbiParams): the handle goes to rdi, the speaker count and the
caller's buffer and size come out of the param block, and the handle is that
buffer when the title supplied one. The previous reading wrote the handle over
the VBAP parameters' speaker-position pointer and left the caller's handle
uninitialized. sceAudioOut2GetSpeakerArrayCoefficients writes
uiNumCoefficients floats to rsi and the ambisonics form writes them to rdx;
both used to zero a fixed 0x400-byte slab, and the ambisonics form could pick
its own count register as the destination address.
Errors are returned in the libSceAudioOut2 error space (0x80268xxx) rather
than as generic codes a title never compares against, and
sceAudioOut2UserGetSupportedAttributes is added.
sceAudioOut2ContextQueryMemory and sceAudioOut2ContextGetQueueLevel are left
exactly as they are, so that the two changes already in flight for them do not
collide with this one; the two address-range predicates survive for those two
callers alone.
* [Kernel] Memoize guest path resolution against the mount configuration
Resolving a guest path to a host path is pure text work plus a reparse-point
check per path component, but a resource streamer calls
sceKernelAprResolveFilepathsToIdsAndFileSizes with the same paths hundreds of
thousands of times over a load, so the work dominates.
Successful resolutions are now memoized. The only inputs that can change an
already-computed answer are the mount table and the host mount roots, so each
entry records the root configuration it was computed under and is ignored if
that has changed, and registering or removing a guest mount clears the cache
outright. A denial is not memoized: it is a containment decision about the
host filesystem's current shape, so it stays live. The cache is bounded, and
its size is tracked in a plain counter because ConcurrentDictionary.Count
takes every bucket lock, which on a path a dozen guest threads resolve at
once costs more than the resolution it guards.
* [Gpu] Route packed pixel color exports to the slots CB_SHADER_MASK enables
Shader compilers drop the MRTs a pixel program does not write, and the
hardware then delivers the Nth color export to the Nth target that
CB_SHADER_MASK enables — the export number and the target slot are not the
same thing. Unreal's base pass is the clear case: it exports MRT0 and MRT1 but
means targets 0 and 4 (SPI_SHADER_COL_FORMAT=0x54, CB_SHADER_MASK=0x000F000F).
Mapping each export to the target of the same number therefore wrote the
fourth G-buffer's data into the first, which showed up as a scene rendered in
world normals. Exports are now paired with the enabled slots in order, the
compiled pixel program keys its outputs by the export that feeds the slot
rather than by the slot, and a bound slot that no export reaches gets a zero
color write mask so it keeps its contents. When the two registers do not pair
up — different counts, or no formats at all — each export keeps going to the
slot of the same number, as before.
* [GPU] Clamp a volume render-target view to the volume depth
Unreal binds a whole volume as a render target with the last slice set to
its depth, one past the end, and the resolver stopped the session on it.
The color block writes only the slices the volume has, so the view is now
clamped to them. A view that starts past the last slice is still fatal.
* [Kernel] Remember APR paths absent from the read-only game image
Looking up a resolved path's size stats the host filesystem, and a path that
is not there costs two stats (a file probe and a directory probe) every single
time it is asked for. A resource streamer asks for far more paths than the
image contains, so misses are the common case and they are repeated.
/app0 is the read-only game image: a path that is not in it now will not
appear later. Absent paths under that root are therefore remembered, which
turns the repeated pair of host stats on what is often a slow external volume
into a dictionary probe. Paths outside /app0 can be created at run time and
are not remembered.
* [Kernel] Verify mount containment per component, not by listing directories
Containment verification stats each component of a resolved path once to reject
reparse points. It was changed to verify a whole parent directory in one
listing instead, on the theory that one listing is cheaper than N stats.
It is not, for a game image: .NET fills the attributes of every entry it
enumerates, so listing a directory stats all of it. A title whose asset
directories hold thousands of entries paid for every file it never opens --
directories of up to 7,390 entries, about 9 s of a load spent in those stats.
Each component is verified with its own stat again, and the per-component
memo that made the listing look attractive stays, so a component is still
verified only once.
* [Ngs2] Implement sceNgs2ParseWaveformData/File/User
The three waveform-parse entry points returned NOT_FOUND, and Void Terrarium
calls ParseWaveformData on its audio thread for every sound it loads.
They now parse the container and publish the description SceNgs2WaveformInfo
lays out (0xE8 bytes, with the block table at 0x48). A RIFF/WAVE fmt chunk
resolves to PCM by bit depth, to IEEE float, through the SubFormat GUID for
WAVE_FORMAT_EXTENSIBLE, or to ATRAC9, whose configuration is decoded so that
the audio unit size, samples per unit, units per frame and delay samples
describe the real frame and superframe. A smpl chunk supplies the loop points
and a fact chunk the decoded sample count. The "VAGp" container maps onto the
VAG waveform type at 28 samples per 16-byte HE-VAG unit.
ParseWaveformFile resolves the guest path and reads the header at the given
offset. ParseWaveformUser would have to drive a guest read callback, so it
reports INVALID_WAVEFORM_DATA rather than fabricating a description.
* [NpWebApi2] Add sceNpWebApi2CheckTimeout
Poppy Playtime calls it every frame, about 6k times a session, and each
call went through the unresolved-import path. No web API request is ever
in flight, so there is nothing to expire and it returns success.
* [Ime] Fill in sceImeKeyboardGetInfo and sceImeKeyboardGetResourceId
sceImeKeyboardGetInfo zeroed 0x88 bytes at its out pointer and returned
success. SceImeKeyboardInfo is 0x24 bytes -- six 4-byte members plus
int8_t reserved[12] -- so the call scribbled 0x64 bytes past the caller's
struct, and a title that reads the device type or the status got a zero for
both, which is not a state the library ever reports.
The export now writes exactly sizeof(SceImeKeyboardInfo) and reports an OSK
device in the DISCONNECTED state, which is what the host can honestly claim
since no physical keyboard is bridged. sceImeKeyboardGetResourceId returned
success without touching the caller's array, leaving the storage that
sceImeKeyboardGetInfo is called with uninitialized; it now publishes the
caller's user id with an empty id array. Both use the real error values.
Also adds a test for the existing scePadResetOrientation, which is a
handle-only call with no out parameter, so a bad handle is its only failure.
* [Gpu] Run an attachment-less draw whose vertex stage stores to memory
A draw with no color target, no depth target and no pixel stage was treated as
having no observable effect and dropped. That is wrong when the vertex or
emulated geometry stage stores to a buffer or a storage image: a GPU-culling or
stream-out pass whose only output is the buffer a later indirect draw reads
looks exactly like that, and dropping it leaves the indirect draw reading
stale data.
Such a draw now runs for its stores alone. The check only resolves the shader
programs when there is an export address to resolve, so a genuinely empty draw
still costs nothing.
* [AudioOut2] Write a single size_t from sceAudioOut2ContextQueryMemory
The export returns the context memory size through one size_t. For heap
output it also wrote a second qword (an alignment) after it, and titles keep
other locals there: Octopath Traveler II's allocator reads the dword at +12 of
that block as a divisor, so it faulted with #DE during boot. Write only the
size, wherever the output lives.
* [CPU] Reuse import trampolines and leave patched stubs alone on module reloads
Every runtime module load sets up the import stubs of all loaded modules
again, and each setup built a fresh trampoline and rewrote the stub's 16 bytes
-- under guest threads that may be executing exactly those bytes.
A trampoline only encodes its import index, so the trampolines are now cached
by index and reused, and a stub that already jumps to the trampoline it would
be given is left untouched, with no unprotect/write/restore cycle at all. A
reload of a module whose stubs are already patched therefore writes no guest
code.
The trampoline ABI test builds its backend with GetUninitializedObject, which
skips field initializers, so it has to supply the new cache; both fields now
go through one helper that names a field it cannot find.
* [Cpu] Read import stack arguments from the thread's own stack without a region query
Every import copies six stack argument slots (twice, with the per-thread
diagnostics). The per-thread readable-range cache is keyed by the host mapping
generation, which every tracker protection change bumps, so on macOS nearly
each slot fell through to a locked host region query: about half of the busy
managed samples at Octopath Traveler II's title were in that query or waiting
for its lock. Addresses inside the calling guest thread's stack, which the
backend maps read-write for the thread's lifetime, are now read directly.
* [HLE] Do not log expected wait timeouts, mutex relocks and closed pad reads
The import gateway logs an error result unless the NID is on a list of calls
whose failure is a normal outcome. Four common ones were missing:
scePthreadCondTimedwait and sceKernelWaitEventFlag time out exactly as the
caller asked, scePthreadMutexLock reports EDEADLK to the owner of an
error-checking mutex (titles use that to build their own recursive locks), and
scePadReadState reports a handle that is not open because titles poll every pad
slot.
Poppy Playtime logs roughly 260k of these lines per session, which buries the
failures that do matter.
* [Kernel] Run one mapping transaction per sceKernelBatchMap
A mapping change has to be handed off to the GPU worker before any lock the
image and buffer reads need, and the handoff waits for the GPU to drain.
sceKernelBatchMap walked its entries calling the same per-mapping entry points
the single-mapping exports use, so every entry in the batch took its own
handoff and its own drain.
The batch now opens one transaction and runs all of its entries inside it. A
thread-local flag marks that a transaction is already open so a nested entry
point runs directly instead of handing off again, and the flag is cleared in a
finally so an entry that throws cannot leave it set.
* [Loader] Rewrite SHA instructions in spans that touch no guest memory
The red-zone patcher computes a shifted core for every span: the run of
faultable guest memory accesses that executes with RSP moved below the
red zone. A span over register-only instructions has no such access, so
the core could not be computed and the span was refused as too short.
SHA rewrite sites are exactly that shape (sha256msg1/msg2/rnds2 on
registers), so no SHA instruction was rewritten any more: on hosts
without the SHA extensions every one trapped and was emulated one at a
time. Poppy Playtime logged sha_rewrites=0 with 144 too-short refusals,
hit hundreds of thousands of illegal-instruction traps and stopped making
progress within a minute.
A span with no guest memory access has nothing that must run outside the
shift, and the span builder has already refused any instruction that
touches RSP, so the whole span is the core.
* [Cpu] Keep fiber switches from writing stack bounds into the macOS TSD
The guest context-transfer stub that fiber switches go through keeps the
Windows TEB stack bounds in sync by storing the target stack top and
bottom to gs:[8] and gs:[16]. Unlike the guest-entry and exception stubs,
it did so on every host. On macOS gs addresses the pthread TSD, where
those slots hold errno's address and another key: a fiber switch wrote
its stack bounds over them, and the next failed libc call on the thread
(a timed-out condition wait) crashed reading errno through the pointer.
Demon's Souls died this way within 10-50 s of boot; a hardware
watchpoint on the thread's errno slot stopped in this stub.
The fiber bounds sync now lives next to the other TEB-bound helpers and,
like them, emits nothing off Windows.
* [Kernel] Take APR file sizes from the mount check's stat instead of listing directories
Resolving an APR path already stats every component to rule out links out of the mount.
The same stat now records the file's size, so size lookup no longer lists the file's whole
directory. Listing stats every entry: Demon's Souls keeps up to 7,390 files per asset
directory on a slow disk, and each first lookup in one paid for all of them.
FileInfo reports a missing component as -1 instead of throwing; it still ends the check.
* [Kernel] Place mappings in untouched address space without waiting for the GPU worker
Every mapping change queued behind the GPU worker so it could not race a pending GPU read.
During loading that worker is busy translating shaders, and Demon's Souls' reservations and
batch maps waited seconds for a queue slot (23.6 s over 132 calls, up to 7.1 s each) while
the change itself took about a millisecond.
A change that only places memory where the GPU was never given anything - a kernel-chosen
address, or a fixed range that is empty or reserved and overlaps no registered span - cannot
race a GPU read and has nothing to unregister, so it now runs on the calling thread. Those
paths also skip the unregister request for free and reserved ranges, which are never
registered; waiting on the worker while holding the mapping lock that GPU reads take would
deadlock, so reaching a registered span there throws instead.
* [HLE] Resolve single-view backing reads and writes without the views lock
Every HLE access to aliased guest memory took one lock shared by all guest threads, and
Demon's Souls' job threads queued on it for 8-byte mutex reads. Readers now binary-search a
copy of the view list; mutations drop the copy under the lock and the next reader rebuilds
it. A reader that races a mutation uses the list from before it, as if it ran first.
* [ShaderCompiler] Find each instruction's block by search and reuse register files
Several passes found a block's instructions by scanning the whole program once per block,
and the value graph did it again on every block visit. Blocks are sorted and contiguous, so
IrControlFlowGraph.BlockOf finds an instruction's block by binary search and each pass
groups the program once.
The value graph also allocated two 256-entry register files per block visit. Large Demon's
Souls compute shaders visit thousands of blocks, and the collections that caused stopped the
renderer for about 2.6 s of its first-frame shader work. Replaced entry and exit states are
now reused; a reused state is reset exactly as the constructor would, at the same point, so
the undefined values it starts with and the instructions they are charged to are unchanged.
The resource graphs and SPIR-V of the game's first 134 programs are identical.
* [Cpu] Deliver guest access faults on macOS as signals, not runtime Mach exceptions
The runtime receives EXC_BAD_ACCESS for its threads through a Mach exception port before
the kernel raises SIGSEGV or SIGBUS. It treats a fault in the page holding RSP, or the page
below, as a stack overflow and aborts - assuming RSP is on a stack it owns. Guest code runs on
guest stacks whose pages fault whenever GPU or image write tracking protects them, so Demon's
Souls died with a bare "Stack overflow." about every other minute of play.
Threads now clear that port the first time they enter guest code. Their access faults reach
the signal handler, which resolves guest faults and chains the rest to the runtime's own
signal handlers, as on Linux. Each tracking fault also skips the Mach round trip, which
roughly doubled Demon's Souls' frame rate (about 2 to 4-5 fps).
* [ShaderCompiler] Solve user-data liveness on bit sets and hash graph nodes by id
The liveness fixpoint that picks a program's user-data registers walked 256 booleans per
block per pass; it now works on four 64-bit words per block with the same equations.
ScalarValue keys many graph dictionaries. It keeps reference equality but hashes its unique
id, which skips the runtime's identity-hash path.
* [SpirvTranslator] Mask a wave32 saveexec result at 64 bits
The low-half mask for s_*_saveexec_b64 on a 32-lane wave used the 32-bit AND helper on the
64-bit exec value, emitting OpBitwiseAnd %uint with a %ulong operand. spirv-val rejects the
module and the driver's behaviour for it is undefined; Octopath has four such programs.
* [SpirvTranslator] Structure loops, early exits and single-block programs
Only forward, nested branches were emitted as structured control flow; every program with a
loop, and every single-block program, went through the loop around a switch on the guest PC,
which keeps all registers live and stops the host compiler from optimizing across blocks.
A block that a later block branches back to - on a wave-uniform condition or with s_branch -
now opens a SPIR-V loop whose body is structured like any range, whose continue evaluates
that branch's condition, and whose iterations share the dispatcher's step guard so a guest
loop that never ends still cannot wedge the GPU queue. A conditional branch from the body to
the block after the latch is a break out of the innermost loop. Any other backward edge, or a
break past an outer loop, keeps the dispatcher as before.
Across eight games, programs left on the dispatcher went from 760 to 104 (Poppy, Silent Hill,
Oregon Trail and Dreaming Sarah to none); every emitted module passes spirv-val and the
frames match. Octopath's 12.5M-invocation instanced draw went from 12.2 to 10.8 ms.
* [Gpu] Age cached images by frame, not by guest submission
The image collector runs after every completed guest submission as well as at each flip, and
every run advanced the age it compares against. A title submits dozens of command buffers per
frame, so "unused for 16 ticks" meant a fraction of a frame: every texture sampled once a frame
was deleted and recreated, and Superliminal uploaded 3,591 textures (2.2 GB) every 5 seconds.
Deleting a GPU-written image the cache cannot read back also dropped its contents.
Only a flip now advances the age. Collection after a submission still runs, so memory pressure is
relieved during long stretches without a flip. Superliminal's per-frame uploads went from ~55 to
none, and its intro scene from 12 to 45 fps.
* [Gpu] Submit on a release-memory label write only when it raises an interrupt
A RELEASE_MEM that writes a 32-bit label submitted the open batch at once. Without an interrupt
nothing inside the slice waits for the label, and every slice that makes progress already ends
with a submit - including one that stops at a WAIT_REG_MEM on that label - so the label is
written when that batch completes either way.
The per-label submits split Superliminal's frames into ~35 extra command buffers; each ended the
render pass, which stores and reloads every attachment on a tile-based GPU. The title menu went
from 15.6 to 21 fps and Octopath from 13-21 to 18-29 fps, with no hang over 150 s runs.
* [VideoOut] Reinterpret the sRGB-encoded frame on a presenter image, not the swapchain
A floating-point display buffer is sRGB-encoded by blitting it into an sRGB image and then
copied byte-for-byte into the UNORM swapchain image. MoltenVK does that copy through a
reinterpreting texture view it caches on the destination; on a presentable image the drawable
behind it changes every frame, and destroying the swapchain then walked those views through a
freed device. Superliminal, whose display buffers are RGBA16F, crashed at every exit (SIGSEGV
in MVKImagePlane::releaseMTLTexture from vkDestroySwapchainKHR) and each crash kept two crash
reporter processes busy for minutes.
The encoded bytes are now copied into a presenter-owned image in the swapchain's format first,
so the swapchain image only ever receives a same-format copy. Frames are unchanged; the exit is
clean.
* [Cpu] Stop guest threads at their next import once the host shuts down
Host shutdown only kept guest threads from starting another slice; a thread already inside
guest code kept calling HLE while the services behind it were torn down. Superliminal's job
threads then got a null command buffer from the GPU library, wrote through it, and the process
died with SIGSEGV on its way out.
After shutdown is requested an import now returns to the host entry stub instead of running,
ending that thread's slice; the scheduler already refuses to enter guest code again.
(cherry picked from commit 9e0e8ec0aa8b0bfaa030e5e97d51dbc0223f93e1)
* [Gpu] Declare every parameter the pixel program reads in the paired vertex program
A vertex program declares outputs for the parameters it exports plus as many locations as the
pixel program has inputs. SPI_PS_INPUT_CNTL can map a pixel input to any parameter number,
though, so a pixel program with two inputs may read parameters 4 and 5. Metal requires every
fragment input to be written by the vertex stage, so the pipeline failed to link ("fragment
inputs user(locn4), user(locn5) ... not written by vertex shader"), the command stream slice
failed and Silent Hill stopped rendering at a scene change.
The required output count now covers the highest location the pixel program reads, resolved
through the same input mapping its translator uses; missing exports are zero-filled as before.
The pipeline creation error also names the vertex and pixel programs.
(cherry picked from commit 80ec9fb53cfae922e458e2dd43d4b1010dbd2701)
* [Kernel] sceKernelAprGetFileSize writes the resolved file size
The export was a success stub that never wrote its out parameter, so callers
sized buffers from uninitialized memory. The signature is (id, uint64_t* size);
it now answers from the APR file registry like sceKernelAprGetFileStat.
* [Cpu] sceKernelWaitSema is an import-loop guard boundary
A thread that blocks on a semaphore between iterations is waiting, like the
condition-variable waits already on the list. A job-wait loop that runs queued
work between timed semaphore waits was otherwise taken for a hang once loading
outlasted the guard window.
* [Fiber] Avoid formatting unused names and disabled traces
* [Shader] Build materialization cache ranges without temporary copies
* [ShaderCompiler] Intern scalar graphs without string keys or hit allocations
* [Media] Vectorize movie YUV conversion without changing pixels
* [Shader] Reuse scratch storage for resource read recording
* [Tests] Initialize presenter state for GPU binding validation
* [Vulkan] Keep temporary descriptor writes off the managed heap
* [GPU] Reuse bounded descriptor preparation scratch arrays
* [ShaderCompiler] Persist versioned scalar analysis across launches
* [GPU] Remove duplicated image layouts from texture bindings
* [ShaderCompiler] Bind cached graph payloads to their input keys
* [Vulkan] Load content-addressed driver cache shards on demand
* [Tests] Keep presenter allocation fixture independent of diagnostics
* [Vulkan] Forget driver cache identities of destroyed shader modules
The remaining change of the updated #1006 head (e6659681) that the integration branch did not carry.
* [Vk] Measure GPU busy time from per-submission timestamps
Per-draw timestamps are written only with SHARPEMU_PROFILE_GPU_DRAWS=1; by
default each submission gets two timestamps and the overlapping intervals
are merged into a [PERF][GPU_BUSY] line with the busy share and the longest
submission.
* [HLE] Reserve a host core for the render thread
On Windows hosts with 8 to 64 logical processors the render thread gets the
last physical core of the fastest efficiency class at the highest priority,
and guest threads are kept off that core so spinning job workers cannot take
the renderer's SMT sibling. SHARPEMU_RENDER_CORE=0 turns it off.
* [Vk] Spin briefly before blocking on a GPU wait
A GPU wait polls the timeline for SHARPEMU_GPU_WAIT_SPIN_US microseconds
(150 by default) before it sleeps, which saves the wake-up latency when the
work completes soon.
* [GpuMemory] Answer CPU-dirty range queries without the tracker lock
MayHaveCpuDirtyPages reads the per-block summary and the region masks without
taking the region lock; a true answer still goes through the locked path.
Regions also bump a CPU-dirty epoch whenever a page becomes CPU-dirty.
* [Gpu] Skip the device-address sweep while no page became CPU-dirty
PrepareBda remembers the CPU-dirty epoch and the mapping it swept last and
skips the full sweep of mapped memory until either changes.
* [GpuMemory] Scan possibly CPU-dirty blocks 64 at a time
ForEachPossiblyCpuDirtyRange combined a region lookup and a summary read for
every 4 MiB block of mapped memory on each device-address sweep. A bitset of
blocks that have a region and the dirty summary words are now scanned a word
at a time, with the same runs as before.
* [Gpu] Skip syncing a written buffer range that is already GPU-owned
SynchronizeBuffer returns early when a written range lies entirely in the
GPU-modified ranges, uses the lock-free CPU-dirty query for reads, and the
render profile splits buffer sync time into written, texel and upload parts.
* [Gpu] Version the GPU-modified buffer ranges
A written range bumps GpuModifiedVersion only when it adds bytes to the
GPU-modified set, so readers can cache a clean verdict for a page until the
GPU writes something new.
* [Gpu] Track the extent of DCC metadata and answer overlap queries
A color target records the size of its DCC metadata, and the image cache can
tell whether a guest range overlaps registered DCC metadata.
* [Diagnostics] Trace the GPU writers of DCC metadata and of a chosen address
SHARPEMU_TRACE_DCC_WRITERS=1 writes the shader program and a call stack for
GPU buffer writes that overlap DCC metadata; SHARPEMU_TRACE_WRITER_ADDRESS
does the same for writes that cover one guest address.
* [Gpu] Fill DCC metadata on the GPU and in guest memory at once
A fill that targets DCC metadata which no GPU write owns is recorded as a GPU
fill and also written to the guest backing directly, so the range never
becomes GPU-dirty and later CPU reads need no readback. Ranges the GPU still
owns or that overlap an image keep the plain buffer fill.
* [Gpu] Turn AGC fill compute shaders into buffer fills
BoundedFillDetector recognizes the AGC range-fill and pattern-fill compute
programs, including thread-dimension dispatches, and a dispatch of one that
targets DCC metadata becomes a DCC fill instead of a compute dispatch.
Refused conversions are reported by the DCC writer trace.
* [HLE] Resolve backing aliases for readers that hold an access window
SharedBackingViews publishes its view snapshot as an alias identity, and a
reader can enter an alias access window once and resolve guest pages to their
backing alias without the views lock until the snapshot changes.
* [Diagnostics] Trace shader resource reads that wait for GPU-written memory
SHARPEMU_TRACE_GPU_READS=1 writes the shader, stage, address and table phase
of a resource read that has to synchronize GPU-written memory, once per shader
and page.
* [ShaderCompiler] Keep recent materialization variants per draw key
A cache key now holds up to four materializations in most-recently-used order,
so alternating descriptor contents reuse their earlier result instead of
re-materializing. The cache report counts stale and unreadable entries and
table refreshes, and ReadingTable tells a reader when the flattened table is
being evaluated.
* [VideoOut] Read resource words through cached clean pages and the backing alias
TryReadGuestWord and TryReadResidentGuestBytes remember the last 64 pages
found free of GPU-dirty bytes, keyed by the GPU-modified version, and read
them through the backing alias inside the alias access window that each
command stream slice now holds. GPU-dirty checks are byte-precise, and a
resident read refuses unreadable ranges before it asks the page tracker.
* [Gpu] Read GPU-written guest memory back asynchronously and ahead of time
VulkanAsyncReadback keeps eight slots: slot zero serves synchronous reads and
the others carry asynchronous downloads. A guest thread that reads GPU-written
memory lets the render thread start the copy and waits for it on its own
thread, then the render thread applies it if the range is unchanged; a
foreign read keeps the source buffer alive until it completes.
Windows the render thread had to read synchronously become hot. When a batch
writes into a hot window, the window is downloaded right after the batch is
submitted, and the next read uses that copy if it is still current. A pending
write to a hot window ends the open batch early. SHARPEMU_ASYNC_GUEST_READBACK=0
keeps the synchronous path; [PERF][ASYNC_READBACK] reports the counts.
* [ShaderCompiler] Prefetch contiguous resource words in one resident read
Resource materialization read every descriptor and flattened-table word
through its own guarded TryReadGuestWord call, about 2.6 million per five
seconds in Demon's Souls. The raw reads of each plan are grouped once; at run
time their addresses are resolved, sorted and read as contiguous runs through
the resident guest reader, and the values seed the evaluator cache. A run the
GPU may still own, or one out of bounds or through a null base, falls back to
the word reads. The cache recorder records the prefetched words, and
SHARPEMU_RESOURCE_PREFETCH=0 turns the prefetch off.
* [Gpu] Reuse image lookups that found the same backing
FindImage remembers, in a 256-slot table, each request that resolved to an
image with the same backing, keyed by the request bytes and a generation
that every index registration or removal advances. A repeated request
touches the image and returns the remembered result without walking the page
owner index. SHARPEMU_IMAGE_LOOKUP_MEMO=0 turns it off; [PERF][IMAGE_LOOKUP]
reports hits and misses.
* [ShaderCompiler] Look up fused programs without the registry lock
Every shader header lookup and decode asked the fused-program registry under
its lock. The registry now publishes an immutable dictionary that readers
take with a volatile read, and a registration copies it under the lock.
* [ShaderCompiler] Decode fused programs whose halves are far apart
The fused decoder joined the continuation at its address distance from the
entry and refused halves more than 4 GiB apart or in the wrong order, which
Poppy Playtime now hits for a vertex shader (invalid-fused-layout). Such a
continuation is placed right after the entry instead, and each of its
instructions keeps its real offset from the program address in AddressOffset;
S_GETPC_B64 in the scalar graph, the SPIR-V and the MSL translators uses that
offset, and the persisted scalar analysis key includes it.
* [Gpu] Collect an image only once it is old enough in frames
Since images age by frame, clamping the age to the current frame made every
image of the first frame collectable during the first 16 frames, and a
submission that ended no frame could delete a texture the frame still used.
Without memory pressure nothing is collected until the full age has passed;
under pressure the clamp still lets collection relieve memory early.
* [Tests] Cover the deferred global barrier of an attachment-only scope
A global barrier still ends a scope whose draws store to memory. In a scope
whose draws only write attachments it waits for the scope to end, and a later
memory-writing draw ends the scope and records it.
* [Agc] Make the number of suspend points in flight configurable
CommandStreamQueue takes the number of frame boundaries that may wait behind
the graphics commands. The default stays one, so the guest CPU can run one
frame ahead, and SHARPEMU_SUSPEND_POINTS_IN_FLIGHT=0 makes a suspend point
wait until its own boundary is processed, for diagnosis. Tests cover both
modes, including a label that the CPU rewrites after the suspend point.
* [Gpu] Throttle the device-address fault and 3D slice range logs
Little Nightmares wrote 195 thousand "Accessed non-GPU cached memory" lines in
one session and Silent Hill 6800 slice range lines. Both now print the first
occurrences and then only at power-of-two counts, with the count.
* [Docs] Document the new environment variables
Adds SHARPEMU_ASYNC_GUEST_READBACK, SHARPEMU_GPU_WAIT_SPIN_US,
SHARPEMU_IMAGE_LOOKUP_MEMO, SHARPEMU_RESOURCE_PREFETCH,
SHARPEMU_SUSPEND_POINTS_IN_FLIGHT, SHARPEMU_RENDER_CORE,
SHARPEMU_TRACE_DCC_WRITERS, SHARPEMU_TRACE_GPU_READS,
SHARPEMU_TRACE_WRITER_ADDRESS and SHARPEMU_PROFILE_GPU_DRAWS, and corrects the
group counts to the rows the tables hold.
* [GpuCommands] Write the 64-bit label of a compute-done release with an interrupt
RELEASE_MEM with event 0x2F and a 64-bit data selection was accepted only
without an interrupt. It is now handled like the other end-of-pipe events:
the label is written and the interrupt is raised after it.
* [GpuCommands] Let a wait see its own queue's preceding label store
A memory wait that directly follows a 4- or 8-byte store of its queue to the
same address, with only waits or inert NOPs and markers between them, compares
against the stored value when guest memory no longer holds it. Dead Cells
writes a label and waits for it while the CPU already rewrites it for the next
frame. The forwarded value lives only within one slice, and any draw, dispatch
or transfer in between drops it.
* [ShaderCompiler] Key fused programs by the underlying guest memory
Each native guest thread wraps the guest memory in its own tracking wrapper,
so a fused program registered while one thread created the shader was not
found by the render context. The registry now unwraps ICpuMemoryWrapper
before it looks up the per-memory table.
* [Agc] Implement the legacy sceAgcFuseShaderHalves export
The older NID (nApJjpKNBl4) shares the fusing path but keeps the front half's
user data pointer, takes the larger value of the resource allocation fields
of both halves, takes the user SGPR count and the ES component flag from the
front half, and copies the checksum register of the matching stage.
* [ShaderCompiler] Decode S_CMOV_B64 as a conditional select
S_CMOV_B64 is decoded as S_CSELECT_B64 with the old destination pair as the
false source, so it reuses the SCC, register-pair and EXEC handling of the
translators and of the scalar value graph.
* [Gpu] Merge the stencil replacement references of a face
Vulkan has one stencil reference per face for the comparison and every
replace operation. Replacements with the test value and with the operation
value are now merged into that reference over the written bits, and refused
only when they conflict on bits the comparison also reads. A replacement
with an operation value of zero over the written bits becomes a zero
operation, which ASTRO's PLAYROOM uses to clear one stencil bit.
* [Json] Add the sce::Json::Value copy constructor and keep shadow values on copies
The copy constructor (fSb2oQTNrgA) shares the assignment path. Assignment
and reads now carry the value a constructor or setter recorded in the object
heap, so a copy keeps it after the source is destroyed.
* [Font] Implement sceFontTextSourceInit
Fills the 0x60-byte text source with the text range, the cursor, the parser
callback and its object, and the text size, and leaves the memory after it
untouched.
* [AudioIn] Answer sceAudioInGetSilentState without an input backend
No AudioIn port can be opened until an input backend exists, so the export
returns an AudioIn error code (0x80260101) instead of being an unresolved
import.
* [PhtreadSpecificSemanticsTests] Fixes for computed shaders
* [ShaderCompiler] Return the entry header address of a registered fused program
An overload of TryGetFusedProgramParts also gives the entry header, so a
caller can register the same fused program on another memory.
* [Vulkan] Keep saving a pipeline cache that outgrows its limit
A driver cache over 256 MiB was refused on every save, so it was never
written again and every launch compiled the same pipelines. Such a cache
is now written as an empty file once, and the next launch starts a fresh
one. A save is skipped when the driver reports the size that was loaded
or last saved, and a failed or deferred save waits for the next interval
instead of running again on every new pipeline.
* [Vulkan] Write only the cache shards that gained pipelines
A shard remembers the driver size it was loaded or saved with and is not
written again while that size holds. Shard saves are logged as one
summary line, and a shard load only when it is slow.
* [Vulkan] Remove the single-file cache once the shard mode takes over
The shard mode never reads the single cache file of the default path, so
it is deleted instead of staying on disk.
* [Gpu] Record compiled compute programs in a shader prewarm list
When the pipeline host gives a list, each compute permutation the game
compiles is appended at once to shader-prewarm.bin: the guest code the
decoder read (once per program), the fused parts, the compute input, the
system registers, the resource specialization and the push data start.
Records carry a checksum, so a cut tail is dropped at the next open.
TryCompilePrewarm rebuilds the compile request from a record through the
same layout and request code as the runtime, so it produces the same
SPIR-V.
* [Vulkan] Prewarm the recorded compute pipelines after a translator or driver change
A first run after a translator change compiled every compute pipeline
one at a time while the render thread waited (Demon's Souls: 282
pipelines, 144 s). When the translator assemblies or the driver differ
from shader-prewarm.stamp, background threads at below-normal priority
translate the listed programs again and create their pipelines, so the
driver has them before the game asks. With the same build and driver the
launch does no prewarm work. The log reports the prewarm result and how
many runtime compute modules it produced.
* [Vulkan] Keep the pipeline cache in memory on Windows
The GPU driver keeps its own shader cache on disk, and the shader prewarm
list rebuilds it after an update, so Windows no longer writes the driver
cache file by default. SHARPEMU_VK_PIPELINE_CACHE=file keeps one; Linux
keeps the file and macOS keeps its shards.
* [Docs] Document the pipeline cache modes and the shader prewarm list
Adds SHARPEMU_SHADER_PREWARM, SHARPEMU_VK_PIPELINE_CACHE_SHARDS and
SHARPEMU_VK_PIPELINE_CACHE_LEGACY_SHARDS, and describes the new cache
file default.
---------
Co-authored-by: kuba <xnetcat.dev@gmail.com>
Foued Attar
Ainesh M.
Mathias
Heazer13
ParantezTech
T
Usif
qwe1qwe
kuba