Keep ownership of a socket descriptor with the guest when connect fails, and route generic close calls through the socket table. Add a deterministic regression test for the failure and close sequence.
Signed-off-by: missatjuhvdk1 <177474143+missatjuhvdk1@users.noreply.github.com>
Co-authored-by: missatjuhvdk1 <177474143+missatjuhvdk1@users.noreply.github.com>
* [Perf] Gate event-flag tracing so it allocates nothing when disabled
TraceEventFlag built its interpolated argument (and, on the wait path,
FormatFrameChain + a new StringBuilder(256) in FormatGuestWaitObject plus
~12 guest-memory reads) on every call, then checked the env var inside the
method — so every sceKernelSetEventFlag/Clear/Poll/Wait paid a string
allocation and an Environment.GetEnvironmentVariable P/Invoke even with
tracing off. Cache the flag once in a static readonly bool and gate every
call site, matching the semaphore/event-queue pattern. Behavior is
unchanged when SHARPEMU_LOG_EVENT_FLAG=1.
* [Perf] Hoist IsNoBlockLeaf classification to import-stub setup
The leaf-dispatch path called IsNoBlockLeafImport(nid) — a ~30-literal
string pattern match — on every leaf import. IsLeaf/NidHash were already
precomputed on ImportStubEntry at stub setup; add IsNoBlockLeaf alongside
them and read the field in the hot path. Behavior unchanged.
* [Perf] Cache trace env-var flags read on hot paths
Two SHARPEMU_LOG_* env vars were read via Environment.GetEnvironmentVariable
(a P/Invoke + transient string) on hot paths: SHARPEMU_LOG_FIBER on every
fiber context transfer, and SHARPEMU_LOG_DIRECT_MEMORY on every direct-memory
op (~8 sites). Cache both once — _logFiber alongside the other backend _log*
flags, and _traceDirectMemory behind the existing ShouldTraceDirectMemory
helper. Behavior unchanged.
* [Perf] Avoid per-iteration thread snapshot in the idle pump loop
PumpUntilGuestThreadsIdle allocated a full GuestThreadState[] snapshot
(via LINQ Values.ToArray()) on every spin just to tally three run-state
booleans. Tally them under the lock with an allocation-free helper, and
only materialize the snapshot inside the gated (default-off) diagnostic
dump. SnapshotGuestThreads now uses Values.CopyTo instead of LINQ.
Behavior unchanged.
* [Perf] De-LINQ GetPixelColorExportMask on the per-draw path
GetPixelColorExportMask ran a Select/OfType/Where/Aggregate chain over all
shader instructions, allocating iterators + closures, and is called per
render target (twice per draw via CreateRenderState and HasPixelColorExport).
Replace with a manual scan producing the identical mask — no allocation, and
it removes an authored-LINQ use the repo bans.
* [Perf] De-LINQ per-draw render-target selection in AgcExports
The bound-render-target selection used Where(...).OrderBy(...).ToArray()
(plus a second Where/ToArray fallback) on every translated draw, allocating
LINQ iterators + closures. Replace with an explicit filter into a pre-sized
list + List.Sort by slot; slots are distinct so this matches the stable
OrderBy. Same result, no per-draw LINQ allocations.
* [Perf] De-LINQ per-draw render-target validation in the Vulkan presenter
SubmitOffscreenTranslatedDraw validated its render targets with
targets.Any(...) twice plus targets.Select(a=>a.Address).Distinct().Count()
(a per-draw HashSet), and broadcast a single blend with
Enumerable.Repeat(...).ToArray(). Targets are <= 8, so replace with manual
scans (invalid-target check; combined dimension-mismatch + pairwise
aliasing check) and Array.Fill. Same results, no per-draw LINQ allocations.
* [Perf] Binary-search VirtualQuery region lookup (SortedList)
_mappedRegions was an unordered Dictionary, so TryFindVirtualQueryRegionLocked
scanned every region for containment/next — O(n) per sceKernelVirtualQuery and
O(n^2) when an allocator walks the address space with the findNext flag. Store
regions in a SortedList keyed by base address (every write already uses the
region's own address as the key) and find the containing/next region with a
binary search over the sorted keys. Also drops a now-redundant Values.OrderBy.
Non-overlapping regions assumed (mmap semantics), so only the floor region can
contain the query. Behavior-preserving; worth spot-checking VirtualQuery-heavy
titles.
* [Perf] Remove stray CLI packages.lock.json committed by mistake
git add -A in an earlier commit swept in a regenerated
src/SharpEmu.CLI/packages.lock.json. main tracks no lock files (central
package management, no RestorePackagesWithLockFile), and REUSE.toml no
longer covers packages.lock.json, so the committed file failed the REUSE
Compliance check. Remove it.
* Boot compatibility fixes for UE titles, GUI toggles, and DeS render/boot work
Checkpoint of the Monster Truck Championship and Demon's Souls boot work.
Each piece is independently useful and verified against the titles.
- playgo: scePlayGoGetLocus now returns BAD_CHUNK_ID for chunk ids outside
the known set, matching real firmware. Titles enumerate chunk ids until
that error; answering OK for every id made the scan wrap the ushort range
and spin forever. Missing-sidecar and no-app0 fallbacks report a
fully-installed single chunk 0 so scePlayGoOpen keeps succeeding.
- kernel: restore the SHARPEMU_WRITABLE_APP0 opt-in. Unpackaged UE dumps
write their Saved tree under /app0 during PS5 component init and treat
the denial as a fatal boot error.
- pad: accept handle 0 as the primary pad across all pad calls. Real
firmware hands out small non-negative handles and some titles read state
with handle 0.
- bthid: env-gated experiment hooks for the Thrustmaster wheel middleware
investigation (fail-only-RegisterCallback modes and a synthetic
enumeration callback with a zeroed event struct). All default off.
- gui: add SHARPEMU_LOG_IO and SHARPEMU_WRITABLE_APP0 toggles to the
Environment tab.
- videoout: per-swapchain-image render-finished semaphores (the shared
semaphore raced the swapchain); whole-mip-chain layout init for offscreen
guest images (sampled binds read mips stuck in Undefined); GPU-resident
texture availability now canonicalizes through the texture format table
and accepts compatibility-class aliases, cutting per-frame CPU texture
re-reads (143 GB -> 55 GB per 300 s in Demon's Souls, 0.2 -> 0.5 fps).
- hle: add sceSystemServiceGetNoticeScreenSkipFlag,
sceSystemServiceGetMainAppTitleId (title id published from the runtime),
and sceNpWebApi2CreateUserContext (refuses so the online layer backs off).
- rtc: SHARPEMU_RTC_PROBE_RANGE diagnostic dumps the code around a busy-wait
caller of sceRtcGetCurrentTick once; costs nothing when unset.
* [ShaderCompiler] Fix VReadlaneB32 scalar destination field
The scalar destination lives in the low vdst byte (bits 0-7); it was read
from bits 8-14, the VOP3B carry-out field readlane does not have, sending
every readlane result to s0. Verified against raw gfx10 encodings and
LLVM's assembler tests (v_readlane_b32 s5, v1, s2 -> low byte 0x05).
* [VideoOut] Survive device loss and flip-order asserts without dying
Two ways a frame could take down the whole presenter:
- Device loss between any two Vulkan calls in a frame unwound the window
thread, and the Dispose-time fence check then threw again, masking the
original error. Catch the loss at the frame boundary, retire
presentations and guest submissions whose fences can never signal, and
keep the window loop pumping so the game (audio, logic) carries on.
- The ordered-flip capture invariant is violated ~100 times per run by
Demon's Souls (PPSA01342); on debug builds the Debug.Assert fail-fasts
the process with nothing in the log. Downgrade it to a once-per-version
warning until the capture/wait ordering is understood.
* Fix Dead Cells shader cache regression
---------
Co-authored-by: StealUrKill <35749471+StealUrKill@users.noreply.github.com>
* [Core] Add POSIX native execution and PS5 SELF support
Extend the native backend, guest TLS, fixed-address memory, and loader paths needed by PS5 titles on Windows, Linux, and macOS. Keep workstation GC so high-core-count hosts do not reserve over fixed guest image bases.
* [HLE] Expand PS5 service and media compatibility
Add the kernel, threading, save-data, networking, audio, video-codec, font, dialog, and service exports required by newer PS5 software. Preserve every SysAbi NID currently registered by main while adding the compatibility surface used by ASTRO BOT.
* [AGC/Vulkan] Extend Gen5 shader and presentation support
Expand PM4 handling, Gen5 shader translation, MRT and packed export support, guest image tracking, depth initialization, texture aliasing, and Vulkan presentation. Add the performance overlay and address-filtered diagnostics used to validate ASTRO BOT with original shaders.
* [Core] Align static TLS reservation across hosts
* [Pad] Align primary user ID with UserService
* [Gpu] Preserve runtime scalar buffers across renderer seam
* [AGC] Restore omitted command helper exports
* [Vulkan] Reuse primary views for promoted MRT targets
* [Vulkan] Preserve scratch storage bindings in compute dispatches
* [SourceGenerators] Add the SysAbi export generator and analyzers (phase 0)
New SharpEmu.SourceGenerators Roslyn component, complete and tested but
consumed by nothing yet — the emulator projects adopt it in the
following commits.
Ps5Nid ports the PS NID derivation (base64 of the byte-reversed first
eight SHA1 bytes of name + fixed suffix) from
scripts/generate_aerolib_binary.py to C#, so what has always been a
manual, out-of-band computation becomes a compile-time capability.
SysAbiExportGenerator emits a per-assembly SysAbiExportRegistry whose
CreateExports(Generation) reproduces ModuleManager's reflection scan
exactly — same generation inheritance and filtering, same method-name
fallback, same libKernel default — with attribute-omitted NIDs derived
algorithmically (equivalent to the runtime catalog lookup, which is
built from the same computation). Parameterless handlers are adapted to
the SysAbiFunction shape; invalid declarations are skipped here because
the analyzer rejects them as build errors, so nothing drops silently.
SysAbiExportAnalyzer turns the runtime failure modes into diagnostics:
SHEM001 duplicate NID (across declared and derived forms), SHEM002
malformed NID, SHEM003 uncallable handler signature, SHEM004 NID
contradicting its export name (the class of drift previously fixed by
hand), SHEM005 unresolvable export, SHEM006 export name unknown to
ps5_names.txt when the catalog is wired as an AdditionalFile, SHEM007
handler not reachable by generated code.
The self-contained test suite drives both in-process against the real
SharpEmu.HLE metadata: known catalog NID pairs pin the algorithm, the
generated registry must itself compile, and each diagnostic has a
triggering fixture. Fittingly, the NID pinning test caught a wrong
pair in its own first draft — the exact mistake SHEM004 exists to stop.
* [SourceGenerators] Adopt the generated export registry in the emulator (phase 1)
SharpEmu.Libs consumes the generator and analyzers, with
scripts/ps5_names.txt wired as the AdditionalFile catalog. The runtime
now registers exports from the compile-time SysAbiExportRegistry
instead of the boot-time reflection scan; RegisterFromAssembly is
retained solely as the arbiter for a parity test that pins the two
tables identical — same NIDs, names, libraries, targets, and handler
methods — across Gen4, Gen5, and combined registration.
First contact between the analyzer and all 715 existing exports
surfaced real drift the old offline checker structurally missed
(scripts/check_sysabi_aerolib.py skipped any NID absent from
aerolib.bin): three exports whose friendly names collide with real
catalog symbols of different NIDs, now suppressed at-site with reasons
pending AGC API confirmation, alongside the established synthetic
Unknown* labels for uncatalogued NIDs, which prompted a rule
refinement — SHEM004 only hard-errors when the export name is a real
catalog symbol, since synthetic labels cannot be validated by hashing
and the NID is authoritative for them. The two allowlisted mismatches
in the python checker no longer trigger anything, and the checker is
deleted: the analyzer subsumes it with the semantic model instead of
regex, and validates every declared pair rather than only
catalog-known NIDs.
* [SourceGenerators] Generate aerolib.bin at build time from ps5_names.txt
The runtime NID -> name catalog is derived data and no longer lives in
the repository: a Framework-only MSBuild task (GenerateAerolibBinaryTask,
sharing the same Ps5Nid implementation the analyzers use) builds it
into the intermediate directory from scripts/ps5_names.txt — now the
single source of truth — and SharpEmu.HLE embeds it from there. The
output is byte-identical to the previously committed binary, verified
with cmp against git history; a new test pins that the embedded catalog
loads and resolves a known symbol both directions.
scripts/generate_aerolib_binary.py is deleted (its algorithm lives in
Ps5Nid, its invocation in the build); the REUSE annotation for the
binary goes with it. MSBuild's Inputs/Outputs check means the ~154k NID
hashes only recompute when the names file actually changes. The task
implements ITask against Microsoft.Build.Framework directly, keeping
the vulnerable-flagged Utilities.Core package out and the analyzer
project's file-IO ban suppressed only inside the task itself.
* [SourceGenerators] Emit typed-signature register thunks (phase 2)
[SysAbiExport] handlers can now be written with real signatures — a
CpuContext followed by up to six int/uint/long/ulong parameters — and
the generator emits the SysV unmarshalling thunk, mapping parameters
positionally to RDI/RSI/RDX/RCX/R8/R9 with the same unchecked-cast
idiom hand-written handlers use. SHEM003 accepts the new shape and
rejects register overflow and non-register-representable types. Both
shapes coexist, so migration is per-handler; sceKernelPollSema,
sceKernelSignalSema, and sceKernelCancelSema migrate as the
demonstration (the last showing raw ulong guest-address passthrough).
The reflection scan cannot represent typed handlers, so it retires
here: RegisterFromAssembly, its signature validation, and
ResolveExportInfo are deleted, and the parity test that pinned the
generated registry to the scan is replaced by content-invariant tests
(duplicate-free, full 715-export surface, catalog identity). Deleting
the scan surfaced a phase-1 latent regression — the pre-JIT warm sweep
enumerated only reflection-scanned assemblies, so the generated
registration path warmed nothing and re-exposed the guest-thread
fail-fast risk; the warm set is now derived from the registered
handler delegates themselves.
* [SourceGenerators] Marshal guest strings declaratively with [GuestCString] (phase 3)
A string parameter on a typed [SysAbiExport] handler, annotated
[GuestCString(maxLength)], now makes the generated thunk read the
null-terminated UTF-8 string from the argument register's guest
address before the handler runs, returning
ORBIS_GEN2_ERROR_MEMORY_FAULT to the guest when the read fails —
the exact prologue nearly every string-taking handler writes by hand.
The attribute lives in SharpEmu.HLE next to SysAbiExportAttribute;
SHEM008 rejects misuse (non-string parameter, non-positive MaxLength)
while a bare string parameter stays a SHEM003 signature error.
_open, open, and sceKernelOpen migrate as the demonstration; they were
chosen because their hand-written prologue faulted on a null pointer
the same way the thunk does (handlers that return INVALID_ARGUMENT for
null pointers, like sceKernelCreateSema, keep the raw shape so guest-
visible semantics stay untouched).
* [SourceGenerators] Apply review findings across the branch
Behavior: the open/_open/sceKernelOpen [GuestCString] demo migration is
reverted — the local compat reader falls back to host memory for paths
in loader-mapped regions that ctx.Memory cannot see, so the generated
thunk would have turned recoverable reads into MEMORY_FAULT. The
marshalling infrastructure stays, proven by generator/analyzer tests;
production migration waits for a handler whose semantics the thunk
reproduces exactly. A comment on the handler records why.
Build robustness: the aerolib target is skipped for design-time builds
(the IDE resolves project references without compiling them, so on a
fresh clone the task assembly does not exist yet), and the task/names
paths are centralized in properties. The generator now emits no
registry for export-free assemblies, so referencing the analyzer can
never mint a colliding SharpEmu.Generated type.
Cleanup and perf: the pragma-suppression sites left mis-indented by the
phase-1 relocation are reformatted and the restores moved after the
method body; the dead ExportsForTesting hook and its InternalsVisibleTo
are deleted; the aerolib task reuses one SHA1 instance across ~150k
names; the analyzer caches the parsed catalog per file snapshot instead
of re-parsing 150k lines every compilation start, shares the attribute
name constant with the generator, and computes the catalog-membership
check once.
* [CI] Run the test suites in the build workflow
The workflow compiled the test projects (they are in SharpEmu.slnx) but
never executed them. A solution-level dotnet test now runs between
build and publish, so any test failure fails the build — including the
AerolibCatalogTests/SysAbiRegistryTests that guard the build-generated
aerolib.bin and the generated export registry. Generation failures of
aerolib.bin itself already fail the build step: the MSBuild task logs
an error event and returns false, and a missing task assembly or
missing embedded output are hard MSBuild errors. The NuGet cache key
now also tracks the test projects' lock files.
* [SourceGenerators] Address review feedback
Multi-diagnostic analyzer tests no longer assume a stable diagnostic
order (analyzer execution is concurrent), and the aerolib task logs
the full exception instead of only its message so build failures keep
the type and stack trace.
* [SourceGenerators] Address second review round
Symbol-name comparisons in the shape rules and analyzer now pin an
explicit SymbolDisplayFormat.FullyQualifiedFormat instead of relying on
the display-format default, and the aerolib task fails loudly on a
symbol name that would overflow the format's ushort length prefix
instead of silently truncating it, with null-safe output-directory
handling made explicit.
* [SourceGenerators] Embed aerolib.bin via a target so design-time builds never reference it
The static EmbeddedResource item referenced the generated file even in
design-time builds, where the generation target is skipped — on a fresh
clone the IDE would try to embed a file that never existed. The item is
now created inside an EmbedAerolibBinary target gated on
DesignTimeBuild, separate from the generation target so an up-to-date
skip of GenerateAerolibBinary cannot drop the item with the rest of its
body, and hooked before AssignTargetPaths since dynamic resource items
added later miss the resource pipeline.
Verified fresh build, incremental rebuild (embedded catalog test both
times), and a simulated design-time compile with no artifacts present.
* [SourceGenerators] Regenerate test lock file after rebase onto main
Rebase fallout: main's package graph shifted under #200, so the
SourceGenerators.Tests lock file is re-evaluated to keep --locked-mode
restore green at the branch tip.
* [Build] Drop NuGet lock files; rely on central package management
Central package management was already in effect (ManagePackageVersionsCentrally
with all versions in Directory.Packages.props and no inline PackageReference
versions), so the per-project packages.lock.json files and the lock-mode
workflow only added maintenance overhead. This removes all eleven lock files,
drops RestorePackagesWithLockFile so restore no longer regenerates them, and
takes --locked-mode off the CI restore steps (re-keying the NuGet cache on the
central props files). Package versions remain centrally pinned in
Directory.Packages.props.
* [macos/linux] Cross-platform host memory, TLS, and ABI layer for POSIX
Introduces the foundation for running SharpEmu on macOS (osx-x64 under
Rosetta 2) and Linux (linux-x64). The CPU backend executes guest x86-64
code natively, so these targets run the whole process as x86-64; this
commit replaces the Windows-only host primitives with platform-dispatched
equivalents so the guest boots and services HLE calls off Windows.
Memory (HostMemory.cs, new): a Win32-semantics facade over
mmap/mprotect/munmap with a shadow region table answering VirtualQuery.
PhysicalVirtualMemory, DirectExecutionBackend, StubManager, and the two
Kernel*CompatExports now go through it instead of kernel32 P/Invokes.
Exact-address requests use MAP_FIXED_NOREPLACE (Linux) / guarded
MAP_FIXED (macOS) so they match Win32 "map there or fail" semantics.
TLS + host helpers (PosixHostStubs.cs, new): pthread-backed TLS and
Win64-ABI-compatible stubs for the kernel32 helpers the backend embeds
into emitted x86-64 code (TlsGetValue, QueryPerformanceCounter,
SwitchToThread, Sleep). A Win64->SysV thunk wraps managed callbacks,
since .NET on POSIX compiles them for the SysV ABI while the emitted
call sites use Win64.
Guest address layout: the 0x7FFx window is Windows-only (dyld shared
cache / Rosetta runtime live there on POSIX), so stack/TLS/stub regions
relocate to 0x6FFx off Windows.
Vectored exception handling is gated off on POSIX for now (guest faults
are not yet recovered) — the signal-based bridge is the next step. Also
adds osx-x64 to the RID list and a Docker-based Linux smoke-test script.
Status: on both macOS (Rosetta) and Linux (amd64), the guest now boots,
runs native x86-64 code, and dispatches HLE imports. macOS stops at a
Rosetta translation-cache issue; Linux runs ~252 imports through C++
static-init before hitting the missing fault handler (SIGSEGV).
* [posix] Bridge the vectored exception handler to sigaction(SIGSEGV/SIGBUS/SIGILL)
Guest faults on macOS/Linux previously terminated the process because the
recovery logic in DirectExecutionBackend.Exceptions.cs was Windows-only.
This adds a POSIX front-end that reuses the existing handler bodies:
- DirectExecutionBackend.PosixSignals.cs installs SA_SIGINFO handlers via
an [UnmanagedCallersOnly] entry, rebuilds the Win64 EXCEPTION_POINTERS /
CONTEXT view from the platform mcontext (Darwin __ss thread state via
the mcontext pointer at ucontext+48, Linux glibc gregs at ucontext+40 --
offsets verified against the headers on both platforms), runs the same
chain as the VEH path (TryRecoverUnresolvedSentinel trap-sentinel
recovery, TryHandleLazyCommittedPage demand paging, VectoredHandler
diagnostics incl. FS/GS TLS-fault detection), and writes register
changes back into the mcontext so sigreturn resumes the repaired guest.
Unrecovered faults chain to the previously installed handler so the
.NET runtime keeps mapping its own faults to managed exceptions.
- The whole recovery path is warmed up with fabricated inputs before the
handlers are installed. This is required under Rosetta 2: the signal
trampoline cannot enter x86 code that has never been executed (and so
never translated) -- a cold handler is silently never invoked and the
faulting instruction retries forever (reproduced and verified in an
isolated .NET test under Rosetta for Linux). It also keeps first-fault
JIT work out of the signal frame.
- Handlers run without SA_ONSTACK: the runtime's alternate stacks are too
small for the diagnostic path, while guest (2MB) and host thread stacks
match where Windows dispatches exceptions anyway.
- The raw reads in the shared fault diagnostics (stack qwords, RBP walk,
code bytes at RIP) now probe the region table on POSIX before touching
memory, since a nested SIGSEGV inside the handler would kill the
process before diagnostics finish. Windows keeps its try/catch reads.
- Escape hatches: SHARPEMU_DISABLE_POSIX_SIGNALS=1 skips installation,
SHARPEMU_DISABLE_RAW_HANDLER=1 disables sentinel recovery (parity with
Windows), SHARPEMU_LOG_POSIX_SIGNALS=1 traces every delivery (first 16
and every 1024th are always traced).
Verified with the test game: Linux (amd64 container) previously died with
SIGSEGV right after import #252; it now recovers/diagnoses signals and the
run proceeds to the real next blocker, an unpatched negative-offset guest
TLS read (fault at TLS base - 0x1708), which gets the full NATIVE
EXCEPTION dump before terminating. macOS is unchanged: the bridge installs
and the game still stops at the known Rosetta translation-cache error at
import 12, which is the next work item.
* [posix] Fix guest memory layout faults: TLS prefix, exact mmap, map search base
Three fixes that take the test game from dying during libc init to running
its full main loop on macOS and Linux:
- Static TLS blocks live below the TCB (FreeBSD amd64 variant II) and
libc.prx reaches past -0x1700, but only a 4KB prefix was mapped below
the TLS base. The prefix is now 64KB on POSIX (Windows keeps 4KB); the
fault was a read at TLS base - 0x1708 during libc init.
- HostMemory exact allocation on macOS used MAP_FIXED, which silently
maps over untracked host memory. The direct-memory allocator's address
scan walked into the .NET runtime's JIT heap and replaced live code,
which under Rosetta 2 surfaced as "no code fragment associated with
the given arm pc". Exact placement now passes the address as a hint
and fails on relocation, like MAP_FIXED_NOREPLACE does on Linux.
- sceKernelMapDirectMemory/MapFlexibleMemory searched for free space
starting at 4GB, which is the Mach-O image base on macOS. The default
search base is 0x20_0000_0000 on POSIX, and TryAllocateAtOrAbove now
asks the kernel for a placement instead of page-stepping through host-
owned address space (Rosetta ignores mmap hints for whole VA windows),
over-allocating when the caller needs more than page alignment.
Windows behavior is unchanged; all divergences are platform-guarded.
* [macos] Video presenter on the main thread, MoltenVK support, window keyboard input
Gets the test game from a headless loop to a playable window on macOS:
- AppKit traps with SIGILL ("NSUpdateCycleInitialize() is called off the
main thread") when GLFW runs on a worker thread. The CLI now moves
emulation onto a worker thread on macOS and parks the real main thread
in HostMainThread.Pump(); the presenter posts its whole window loop
there instead of spawning a thread, and a shutdown handler asks the
render loop to close the window so the pump unwinds on guest exit.
- MoltenVK: enable VK_KHR_portability_enumeration (+ the portability
instance flag) and VK_KHR_portability_subset when advertised, and gate
robustBufferAccess2 on the device actually supporting it (Metal does
not; the old code keyed it off robustImageAccess2 and vkCreateDevice
failed with ErrorFeatureNotPresent).
- Input: pad exports polled user32 GetAsyncKeyState, so POSIX hosts threw
DllNotFoundException per scePadReadState call. The presenter now
attaches the window's keyboard via Silk.NET.Input into HostWindowInput,
and the pad exports map the existing VK-code layout onto it off
Windows. Headless hosts (Linux containers) report a disconnected
keyboard and fall back to neutral pad data silently.
GLFW needs an x86-64 Vulkan loader under Rosetta: place a universal
libMoltenVK.dylib next to SharpEmu named libvulkan.1.dylib (Homebrew's
arm64-only copy cannot load into the x86-64 process) and export
DYLD_LIBRARY_PATH to that directory.
Verified: Dreaming Sarah boots to a MoltenVK-backed 2560x1440 window on
macOS (Apple M4, Rosetta 2), renders the intro, title, and menus, and
keyboard input drives it into gameplay. Linux (amd64 container) runs the
same build headless through millions of imports with no faults. Windows
paths unchanged; arm64 and x64 builds clean.
* [posix] CoreAudio playback, self-contained MoltenVK loading, input/log polish
- Audio: sceAudioOut ports now play through an AudioQueue backend on macOS
(stereo PCM16 with the same 32KB backpressure pacing as the WinMM path).
The WinMM port and the new CoreAudio port share an IHostAudioPort
interface and sample converter; hosts without a backend (Linux
containers) keep the silent fallback.
- MoltenVK: GLFW resolves Vulkan with dlopen("libvulkan.1.dylib"), which
cannot see the app-local universal MoltenVK build, so the presenter now
feeds vkGetInstanceProcAddr straight into glfwInitVulkanLoader (GLFW
3.4) before creating the window. No DYLD_LIBRARY_PATH needed; the CLI
also preloads the dylib for Silk.NET and prints setup hints when it is
missing. scripts/fetch-macos-moltenvk.sh stages the official universal
dylib next to a build.
- The virtual-range allocator's failure trace now names the address and
length instead of "AllocateAt invocation threw".
Investigated and documented (not port defects): the savedata transaction
failure is identical on Linux and macOS (HLE argument-register mapping for
sceSaveDataCreateTransactionResource), and the in-game tile speckling has
no platform-specific code in its path - the one macOS-only delta is that
MoltenVK lacks robustBufferAccess2, so out-of-bounds shader reads return
garbage instead of zeros.
Verified on macOS: window, audio backend, and keyboard input all come up
with zero environment configuration; the game runs to gameplay. Linux
headless run unchanged (silent audio, no faults). Windows paths untouched;
arm64 and x64 builds clean.
* [cpu] Preserve guest registers and flags across patched TLS accesses
The TLS patch handler replaces guest `mov reg, fs:[...]` instructions,
which preserve every other register and the flags - but the handler
loaded the TLS index into ecx and called TlsGetValue (Win64: clobbers
rcx/rdx/r8-r11) with `sub/add rsp` trashing the arithmetic flags. Guest
code that keeps live values or comparison results across a TLS access
computed garbage deterministically. The handler now saves rcx, rdx,
r8-r11, and the flags around the call, keeping the same inner stack
alignment. This applies to the load patches and both store-helper stubs,
on every platform.
Also in this change, from the rendering-artifact investigation:
- The present blit picks linear filtering for any fractional scale
(nearest only for integer upscales): a 3840x2160 guest frame blitted
into a 2560x1440 swapchain with nearest silently dropped every third
row/column.
- ClampViewport no longer trims the guest viewport rectangle to the
render target; trimming changed the guest's scale/offset and skewed
texel addressing. Vulkan permits viewports beyond the framebuffer
(the scissor confines rendering), so only spec bounds are enforced.
- Env-gated diagnostics grown during the investigation: guest texture
dumps (SHARPEMU_TEXTURE_DUMP_DIR), aliased guest-image readback dumps
(SHARPEMU_TRACE_GUEST_IMAGES=alias), small-render-target write movies
(SHARPEMU_TRACE_GUEST_WRITES=small), unattended input injection
(SHARPEMU_AUTO_CROSS=secs,...), viewport nudging
(SHARPEMU_VIEWPORT_EPSILON), chunked-draw toggle
(SHARPEMU_DISABLE_CHUNKED_DRAWS), and rect-list/draw vertex traces.
Known remaining issue (root cause narrowed, not yet fixed): the game's
terrain texture pages are corrupted in guest memory before any GPU work
- the mound's solid-fill 32x32 tiles decode to fully transparent texels
and the grass page has deterministic gaps, byte-identical across runs.
Ruled out: memcpy/memmove/memset/realloc HLE semantics, sampler wrap
modes, texel-boundary rounding, chunked draws, viewport handling. Next
step is auditing the Chowdren asset decode path (custom compressed
images) against the emulator's import surface.
* [linux] ALSA playback backend for sceAudioOut
sceAudioOut ports on Linux now play through libasound instead of the
silent fallback. The PCM device opens in blocking mode with ~170ms of
device buffer (the time-equivalent of the 32KB queue the WinMM and
CoreAudio ports keep), so snd_pcm_writei provides the same backpressure
pacing without a managed queue. Underruns and suspend/resume go through
snd_pcm_recover with one retry per submit; anything else drops the
buffer rather than stalling the guest.
The "default" device routes through PulseAudio/PipeWire on desktops
and straight to hardware on bare ALSA; SHARPEMU_ALSA_DEVICE overrides
it (the null device makes the path testable in containers). A missing
libasound or device fails port creation and lands in the existing
silent fallback.
Verified in an amd64 container: the test game opens the port
(backend=alsa, 48kHz stereo float32) and streams sceAudioOutOutput
through the null device for a full run; without a usable device the
port logs a warning and falls back to silent. Playback on real Linux
audio hardware has not been tested.
* [fixes] Address review feedback: commit bounds, CoreAudio shutdown, dump errors
- HostMemory: a MEM_COMMIT that runs past its reservation now fails like
Win32 instead of committing a prefix and reporting success. All current
callers already clamp their ranges to the region, so this only guards
future callers.
- CoreAudioPort: Dispose wakes a submitter waiting on backpressure and
the wait treats ObjectDisposedException as a timed-out wait, so closing
a port during playback can no longer throw. A failed AudioQueueStart
tears the queue down and fails fast instead of leaving an undrainable
queue that stalls every later submit on its timeout.
- AgcExports: texture dumping catches all write failures (bad path,
permissions), logging a warning instead of crashing when
SHARPEMU_TEXTURE_DUMP_DIR points somewhere unusable.
Verified with the Linux container run: game boots and streams audio with
the stricter commit check, and a dump dir under /proc produces warnings
instead of taking the process down.
* [ci] Build linux-x64 and osx-x64 archives
Adds a build-posix matrix job (ubuntu-latest / macos-latest) mirroring
the Windows build: locked restore, Release build, self-contained CLI
publish, and a tar.gz artifact per RID (tar keeps the executable bit).
The macOS archive also stages the universal MoltenVK dylib via
scripts/fetch-macos-moltenvk.sh so the artifact runs without any manual
Vulkan setup. The release job still only ships the Windows archive.
* [cli] Keep POSIX glfw natives outside the single-file bundle
The KeepGlfwOutsideSingleFile target only matched filenames starting
with 'glfw', which covers Windows (glfw3.dll) but not libglfw.3.dylib /
libglfw.so.3. Those got embedded into the single-file bundle, and
Silk.NET's library loader does not probe the bundle extraction
directory, so a published build died with "Couldn't find a suitable
window platform" (and the glfwInitVulkanLoader wiring, which loads the
library from AppContext.BaseDirectory, could not run either). Keeping
the POSIX names loose next to the executable fixes both, the same way
the Windows build already handled it.
Found by running the CI-built osx-x64 archive: video failed while local
loose-file builds worked. With the fix the published single-file build
opens the MoltenVK window, wires the loader, and reaches gameplay.
* [ci] Publish linux-x64 and osx-x64 release archives
The build-posix artifacts now ship as per-RID GitHub releases on main
pushes and manual dispatches, tagged the same way as the win64 ones
(<rid>-<ref>-<sha>). Archives stay tar.gz so the executable bit
survives extraction.
* [cli] Fail early on non-x86-64 host processes
The CPU backend executes guest x86-64 code natively, so the process
must be x86-64 (win-x64/linux-x64 on x64 hardware, osx-x64 under
Rosetta 2 on Apple Silicon). An arm64 process previously failed deep
inside emulation startup, indistinguishable from MoltenVK, signal
handler, or guest memory problems. CLI mode now checks the process
architecture up front and exits with a message naming the supported
execution model (and the Rosetta install command on macOS). The
GUI-only path stays usable on arm64.
* [video] Log the selected Vulkan device name and API version
The presenter never named the GPU it picked, so a 'no video' report
could not be told apart from a real windowing failure without guessing.
It now logs the device name, type, and API version right after
selection. A software rasterizer (llvmpipe/lavapipe/SwiftShader) shows
up here and typically lacks the device features the translated shaders
need, which is the likely cause when a window opens and presents frames
but nothing draws.
* [video] Steer GLFW to XWayland on Wayland sessions
GLFW's native Wayland backend does not reliably map the Vulkan window
with some drivers (NVIDIA in particular): frames present but the window
never becomes visible, so the game runs with audio and no picture. A
report on an RTX 5080 showed exactly this — all device features present,
frames presenting, but the log had 'libdecor-gtk.so failed to init' and
a 1.4x-scaled window, both Wayland tells.
On a Wayland session that also exposes an X server (DISPLAY set), the
presenter now clears WAYLAND_DISPLAY for its own process before GLFW
initializes, so GLFW selects its dependable X11/XWayland backend.
SHARPEMU_ENABLE_WAYLAND=1 opts back into native Wayland. Headless
(no DISPLAY) and non-Linux hosts are unaffected.
* [video] Force GLFW X11 backend via the platform init hint, log the platform
The previous attempt cleared WAYLAND_DISPLAY to steer GLFW off Wayland,
but a reporter still hit the native-Wayland path (the Wayland-only
libdecor error persisted), so that env trick doesn't switch GLFW.
Use GLFW's supported mechanism instead: glfwInitHint(GLFW_PLATFORM,
GLFW_PLATFORM_X11) before GLFW initializes, called into the same libglfw
GLFW itself loads (the pattern InitializeMacVulkanLoader already uses).
Still gated on a Wayland session with an X server present (DISPLAY set)
so we never force X11 where XWayland can't catch it, and still
overridable with SHARPEMU_ENABLE_WAYLAND=1.
Also logs 'GLFW windowing platform in use: <backend>' after init via
glfwGetPlatform, so a 'no window' report shows X11 vs Wayland outright.
Verified on macOS: the readback correctly reports Cocoa and the
presenter is unaffected (the fix is a no-op off Linux).
* [video] Run the GLFW window on the main thread on Linux too
GLFW requires window creation and event processing on the process main
thread on every platform: initialization, window creation, and
glfwPollEvents are main-thread-only, and X11 in particular has a single
event queue that must be serviced there. A window created and polled on
another thread may never map — which is why the game ran (audio, imports,
even Vulkan present) with no visible window on Linux.
macOS already routed the window loop to the main-thread pump (AppKit
needs it); Windows is fine because it has a per-thread event queue. Linux
was the gap: it spawned a background thread for the presenter. Extend the
existing HostMainThread pattern to Linux — emulation runs on a worker,
the main thread pumps the window work the presenter posts.
Refs GLFW intro guide (thread-safety): init, window creation, and event
processing are restricted to the main thread.
Verified: macOS still boots to its window unchanged; the Linux headless
container runs to millions of imports with no deadlock or regression.
On-screen confirmation on a real Linux desktop is still pending, but this
is the documented root cause for a windowless-but-running Linux session.
* [posix] Skip Win32 native guest workers
* [vulkan] Synchronize offscreen targets before present
* [vulkan] Transition fresh textures from undefined layout
* [vulkan] Report swapchain pixels before source readback
* [vulkan] Emit requested guest image diagnostics
* [agc] Diagnose guest texture fallbacks
* [linux] Keep guest GPU mappings in low address space
* [video] Reduce diagnostic stalls and drain complete frames
* [memory] Harden packed GPU address handling
* [readme] Document Linux and macOS release support
* [posix] Integrate the host platform abstraction
* [posix] Restore guest thread address window
* [video] Run the performance HUD on POSIX hosts
The FPS/CPU/work HUD bailed out unless the host was Windows; only the
per-thread hottest-thread scan actually needs Windows APIs. Keep that
scan Windows-only (POSIX reports 'idle') and let the rest of the HUD
run everywhere — the title is already set from the render thread, which
owns the window on macOS and Linux.
* [posix] Implement native guest worker threads
Guest entry stubs must not run above CLR-managed frames on CLR-created
threads (see the NativeWorker preamble); the PR previously fell back to
the inline calli path on POSIX, which reproduced the documented
'attempted to call a UnmanagedCallersOnly method from managed code'
fail-fast (observed after Dreaming Sarah's menu select) and left the
runtime's suspension machinery walking guest frames.
Provide the missing POSIX half of the worker loop:
- PosixHostStubs grows Win64-convention WaitForSingleObject/SetEvent/
ExitThread stubs backed by dispatch semaphores (macOS) / unnamed POSIX
semaphores (Linux) plus pthread_exit, with EINTR retry in the wait.
- Worker events are creatable/signalable/waitable from managed code too,
so NativeGuestExecutor.Run keeps its handshake (AutoResetEvent stays
on Windows byte-for-byte).
- PosixHostThreading implements CreateNativeThread/WaitForThreadExit/
CloseThreadHandle over pthreads (liveness probed with
pthread_kill(0), then joined).
- RunPrologue/RunEpilogue are routed through the existing Win64->SysV
thunks, so the emitted loop stays identical across platforms.
* [macos] Disable concurrent GC under Rosetta's write-watch hazard
Background GC's write-watch revisit (SoftwareWriteWatch::GetDirty ->
FlushProcessWriteBuffers) calls thread_get_register_pointer_values on
every thread; under Rosetta 2 that Mach call stalls indefinitely on
threads executing translated guest code. The background mark phase then
never finishes and every allocating or Monitor-taking thread wedges
behind it — observed as Dreaming Sarah freezing at the menu/loading
screen with FPS 0 in 5 of 7 runs, dispatcher/watchdog parked in
Monitor.Enter and all BGC threads waiting in t_join.
Non-concurrent GC never takes that path; a 5-minute soak now holds
22-31 fps in-game with zero stalls. Windows and Linux keep concurrent
GC.
* [diag] Periodic guest-thread snapshots with gate-owner tracking
SHARPEMU_PERIODIC_SNAPSHOT_SECONDS=N dumps the stall snapshot every N
seconds even while imports are progressing, for soft stalls where the
game stops advancing but threads keep spinning. The periodic dump never
touches the guest-thread gate (it must keep reporting when the gate is
what's wedged): it reads a lock-free owner record — every gate
acquisition now goes through LockGate(site), which notes site/thread —
and walks the thread table without the lock, tolerating torn reads.
SHARPEMU_PERIODIC_SNAPSHOT_FILE redirects the dump to a side file for
the case where the console itself is wedged (frozen log mirror was one
of the observed failure modes).
* [nuget] Add osx-x64 RID targets to lock files
* [cpu] Back off the guest join poll
TryJoinThread polled the host thread at a fixed 1ms; a game main thread
joining a long-lived worker (Dreaming Sarah parks there for the whole
session) burned ~5% of managed CPU in Join/Sleep syscalls. Ramp the
poll interval to 10ms once the join is clearly long-lived — exit
detection latency for long joins moves from ~1ms to at most 10ms, and
short-lived joins still resolve on the first 1ms polls.
* [nuget] Add linux-x64/win-x64 RID targets to lock files
* [posix] Keep guest stacks clear of the import-stub descent
The import-stub region descends from 0x7000_0000_0000 on the same 16MB
grid as the guest thread windows; moving stacks to 0x6FFF_E000_0000 put
them inside the stub region's 64-module descent range (floor
0x6FFF_C000_0000), silently consuming the top ~32 stack slots on hosts
with many loaded modules. Drop the POSIX stack base to 0x6FFF_A000_0000:
512MB below the stub floor, still 2.5GB above the TLS window. Windows
keeps 0x7FFF_E000_0000 (its bands are ~15TB apart).
* [pad] Read window gamepads on POSIX hosts
XInput and the DualSense hid reader are Windows-only, which left
macOS/Linux with keyboard input only. The presenter's Silk/GLFW input
context already enumerates gamepads on both platforms, so track their
state in HostWindowInput (event-driven on the window thread, snapshot
guarded like the key set) translated to ORBIS conventions: GLFW's Xbox
layout maps A/B/X/Y to Cross/Circle/Square/Triangle, sticks bias from
-1..1 to 0..255 with Y growing down, and triggers rescale from GLFW's
-1..1 resting-at--1 range with digital L2/R2 bits past 25%.
The merge into ReadHostInputState is gated to non-Windows so a physical
pad is never sampled twice through both a native reader and GLFW.
Hotplug is handled via ConnectionChanged; with no pad connected the
path is inert.
Untested against a physical controller (none attached to the dev host);
axis conventions follow the GLFW gamepad-mapping contract.
* [nuget] Refresh lock files after cross-RID restores
* [posix] Adopt the host audio/input seams from main
Main's #192 abstracted audio output and pad/keyboard input behind
IHostAudioOutput/IHostInput; re-express the POSIX backends behind them:
- CoreAudioPort/AlsaAudioPort move to Host/Posix as
PosixCoreAudioStream/PosixAlsaAudioStream implementing
IHostAudioStream. The seam converts to stereo PCM16 before Submit, so
the ports' own conversion (and IHostAudioPort/AudioSampleConverter)
is gone; queueing and backpressure are unchanged.
- PosixHostAudio selects CoreAudio (macOS) / ALSA (Linux) as the
platform's IHostAudioOutput.
- PosixHostInput implements IHostInput over an
IPosixWindowInputSource that HostWindowInput registers when the
presenter attaches the window's GLFW input context: keyboard with
virtual-key translation, the window gamepad snapshot (now in seam
HostGamepadState/HostGamepadButtons terms), and keyboard-connected as
the focus signal. Rumble/lightbar no-op (GLFW has no such API).
- PadExports drops its direct HostWindowInput gamepad merge — pads now
flow through IHostInput.GetGamepadStates like every platform.
- PosixHostThreading.RequestTimerResolution is a documented no-op.
All three RIDs build; SharpEmu.Libs.Tests pass (26/26).
* [nuget] Regenerate GUI lock file for RID-less locked restore
Local cross-RID builds stamped a win-x64 runtimes section into
SharpEmu.GUI's lock file; the project declares no RuntimeIdentifiers,
so CI's 'dotnet restore --locked-mode' failed with NU1004 on every
platform. Regenerated via a plain solution restore (--force-evaluate),
matching what the workflow validates.
* [HLE] Stop allocating on the memcpy/memset and trace hot paths
memcpy/memmove no longer allocate a bounce buffer sized to the whole
copy (large copies previously landed on the LOH); they loop through a
single pooled 256 KB rental, copying high-to-low when the destination
overlaps above the source so memmove semantics survive the chunking.
memset reuses a shared zero chunk for the dominant zero-fill case and
rents/fills only min(length, 16K) bytes for non-zero values instead of
allocating and filling a fresh 16 KB array per call; the map-time
zero-fill loop shares the same zero chunk.
SHARPEMU_LOG_SEMA / SHARPEMU_LOG_VIDEOOUT are now read once into cached
bools and every TraceSemaphore/TraceVideoOut call site is guarded, so
trace messages are no longer interpolated (and the env var no longer
queried) on every semaphore op and every flip with tracing off. Trace
output when the flags are set is unchanged.
* [HLE] Remove per-frame allocations from the vblank/flip/equeue plumbing
The 60 Hz vblank pump no longer allocates per edge: PumpVblanks reuses a
pump-thread-only port list instead of a LINQ Where/ToArray, and
SignalVblank/SubmitFlip snapshot their event registrations into pooled
rentals instead of copying the List on every edge and every flip (the
snapshot must still be taken, since triggers run outside _stateGate and
a per-port reusable buffer would race the pump thread against a guest
thread's first-edge signal).
sceKernelWaitEqueue delivery rents the dequeue buffer from the pool
instead of allocating an array per wait, and event-queue wake keys are
formatted once per handle (cached in a ConcurrentDictionary, dropped on
queue delete) instead of building the string on every enqueue. The
semaphore wake key moves onto KernelSemaphoreState at creation, the
same pattern the pthread mutex state already uses, removing the
per-signal/per-wait formatting. SHARPEMU_LOG_EQUEUE is read once into a
cached bool like the sema/videoout flags.
* [HLE] Read guest C-strings without per-call buffer allocations
CpuContext.TryReadNullTerminatedUtf8 allocated a byte[capacity] and
issued one TryRead per byte for every string-argument import. It now
reads through a stack buffer (pooled above 512 bytes) in 128-byte bulk
chunks, falling back to per-byte reads only when a chunk touches an
unreadable range so a terminator sitting just before unmapped memory
still resolves exactly as before. The chunk bound also keeps the
overread past the terminator smaller than the old loop's worst case is
wide, so no fault can appear where the byte loop succeeded.
TryReadAsciiZ (dlsym/symbol resolution) drops its List<byte> + ToArray
round-trip for the same stack/pooled buffer, keeping the byte-by-byte
TryReadByteCompat reads because their Marshal.ReadByte fallback must
probe exactly up to the terminator. Only the final string is allocated
on either path now.
* [HLE] Replace blocking-wait closures with waiter continuation objects
Every wait that actually parked a guest thread allocated two capturing
lambdas (plus their display classes) for the scheduler's resume/wake
callbacks. RequestCurrentThreadBlock and the backend's blocked-thread
state now carry a single IGuestThreadBlockWaiter instead of the
Func<int>/Func<bool> pair: TryWake keeps the run-under-the-scheduler-
gate contract and Resume still produces the guest's RAX on the woken
thread. The waiter stays attached through the wake transition (the old
code nulled only the wake handler there) and is consumed at resume.
The existing waiter objects absorb the captured state as fields, so a
blocking wait now allocates exactly one object: SemaphoreWaiter,
PthreadMutexWaiter, and EventFlagWaiter implement the interface
directly, and the equeue, cond, and rwlock waits get small waiter
classes replacing their closures. Handler bodies delegate to the same
static methods with the same arguments as before; the untimed event
flag wait's mutable captured result becomes a field on its waiter.
* [HLE] Back pending event queues with a ring deque instead of LinkedList
LinkedList<KernelQueuedEvent> allocated a node object on every
non-coalesced enqueue — one per vblank/flip edge per registered queue,
60+ times a second in steady state. KernelEventDeque is a grow-only
ring buffer over a KernelQueuedEvent[] with the three operations the
queue actually uses (AddLast, RemoveFirst, find-and-update-in-place by
ident/filter), so steady-state enqueue/dequeue allocates nothing and
the coalescing update writes the struct back through an indexer instead
of a node reference. All accesses stay under _eventQueueGate, matching
the LinkedList usage it replaces.
* [HLE] Cap memcpy chunk iterations at the requested size, not the rented length
Address Copilot review: ArrayPool.Rent may return a larger array than
requested, so sizing each iteration by chunk.Length let the copy
granularity depend on pool bucketing internals instead of the intended
256 KB chunking. Behavior was already correct for any chunk size (each
iteration re-reads the source, and the overlap ordering is size-
independent), but the loop now mins against the requested chunkLength,
matching what memset already does.
* [HLE] Skip the flip/vblank snapshot rental when no events are registered
Address Copilot review: SignalVblank and SubmitFlip rented (and
returned) a pooled snapshot even with zero registrations — steady
per-frame pool traffic for games that never register flip events and
only poll flip status. Zero-count signals now skip the rental, the
copy, and the trigger loop entirely, which also retires the
Math.Max(count, 1) minimum-rent guard.
Fills NID gaps hit by PS5 titles during boot, controller setup, and
rendering, and surfaces the common runtime switches in the GUI. All
exports are additive (no behavior change to existing exports) and free of
NID and export-name collisions with upstream.
New export libraries:
- libSceBluetoothHid: Init/RegisterDevice/RegisterCallback success stubs so
titles proceed past Bluetooth controller setup (opt-out via
SHARPEMU_BTHID_UNAVAILABLE=1).
- libSceNpCppWebApi: Common::initialize no-op success; UE5 online titles
abort PS5-component startup on a negative SCE error.
Additions to existing libraries:
- libScePad: scePadOpenExt (shared PadOpenCore, accepts special ports 1/2 and
the ScePadOpenExtParam pointer), scePadClose, scePadGetExtControllerInformation.
- libSceVideoOut: sceVideoOutConfigureOutput, sceVideoOutInitializeOutputOptions.
- libSceAgc: DCB builders sceAgcDcbSetIndexCount, sceAgcDcbJump, DcbSetPredication,
SetPacketPredication (emit valid skippable packets; full draw processing TODO).
- libSceAmpr: measure and write KernelEventQueueOnCompletion pair.
- libKernel: scePthreadGet/Setschedparam, sceKernelChmod (validate and accept;
POSIX permission bits have no host equivalent on Windows).
- libSceNetCtl: sceNetCtlRegisterCallbackV6 (delegates to the v4 callback).
- libSceMouse: sceMouseInit.
- libSceUserService: sceUserServiceGetAgeLevel (adult, skips parental gates).
GUI: new Options Environment tab exposing common SHARPEMU_* switches as
toggles (BTHID_UNAVAILABLE, DISABLE_IMPORT_LOOP_GUARD, VK_VALIDATION,
DUMP_SPIRV, LOG_DIRECT_MEMORY, LOG_NP). Persisted in gui-settings.json and
applied to the emulator process environment at launch; localized with
English fallback.
* [Host] Introduce host platform abstraction with IHostMemory
Add SharpEmu.HLE/Host with IHostPlatform/IHostMemory interfaces, neutral
page-protection/region enums, and a HostPlatform.Current factory that
resolves the Windows backend (or throws PlatformNotSupportedException on
other OSes, matching today's de-facto behavior). WindowsHostMemory wraps
the exact VirtualAlloc/VirtualFree/VirtualProtect/VirtualQuery calls used
across the engine today, with identical MEM_*/PAGE_* constants.
Migrate StubManager as the first consumer: its private kernel32 P/Invokes
and enums are replaced by IHostMemory calls that issue the same two
native operations (RWX commit+reserve of the PLT arena, release on
Dispose). No behavior change.
This is the first step toward supporting non-Windows hosts; subsequent
commits move the remaining direct P/Invokes in Core and Libs behind the
same seam.
* [Host] Route PhysicalVirtualMemory through IHostMemory
Replace the class's private VirtualAlloc/VirtualFree/VirtualProtect/
VirtualQuery P/Invokes with IHostMemory calls. Every site maps 1:1 onto
the exact native call it issued before: MEM_COMMIT|MEM_RESERVE ->
Allocate, MEM_RESERVE -> Reserve, fault-path commits -> Commit, and
MEM_RELEASE -> Free, with identical protection values produced by the
Windows backend.
IHostMemory gains ProtectRaw so the save/restore protection sequences in
TryWriteExclusive and TryTemporarilyProtectForRead round-trip the raw OS
protection word (including modifier bits the neutral enum cannot
represent) exactly as before. Raw PAGE_* constants remain only for the
internal region-classification helpers, which only ever see values this
class itself assigned.
The exact-address free-on-mismatch, lazy reserve-only threshold, prime
loop, and all trace strings are unchanged.
* [Host] Add IGuestAddressSpace and retire the reflection-based allocator lookup
Introduce IGuestAddressSpace in SharpEmu.HLE (fixed-address AllocateAt /
TryAllocateAtOrAbove and guest mprotect via TryProtect) with signatures
copied from PhysicalVirtualMemory, which now implements it. TryProtect
reproduces the read/write/execute decomposition that
KernelMemoryCompatExports.ResolveHostProtection performs, yielding the
same PAGE_* values through the Windows backend.
KernelVirtualRangeAllocator previously located AllocateAt via cached
MethodInfo reflection (because SharpEmu.Libs cannot see Core types) and
walked wrapper memories through an untyped 'Inner' property. Both are
now typed: ICpuMemoryWrapper exposes the decorated memory (implemented
by TrackedCpuMemory, whose Inner property already existed) and the
allocator type-tests for IGuestAddressSpace with the same bounded
unwrap depth. Failure paths keep the exact [LOADER][TRACE] strings.
* [Host] Move Kernel HLE memory exports off direct kernel32 P/Invokes
KernelMemoryCompatExports loses its private VirtualQuery/VirtualProtect/
VirtualAlloc/VirtualFree declarations and MemoryBasicInformation struct:
- Guest mprotect (sceKernelMprotect/sceKernelMtypeprotect) now routes
through IGuestAddressSpace.TryProtect resolved from ctx.Memory. The
orbis read/write/execute decomposition moves into a GuestPageProtection
conversion whose mapping is value-identical to the removed
ResolveHostProtection.
- The guarded libc heap and host-page accessibility checks go through
IHostMemory (same commit+reserve/protect/free sequence; guard-page and
protection-mask checks compare HostRegionInfo.RawProtection against the
same PAGE_* literals as before).
- HostMemory is exposed as a property so merely loading the type never
resolves the platform backend on non-Windows hosts.
KernelRuntimeCompatExports' RDTSC stub allocates its 16-byte RWX page via
IHostMemory.Allocate; the OperatingSystem.IsWindows() gate returning null
is unchanged.
* [Host] Abstract thread, TLS, and symbol primitives in the execution backend
Add IHostThreading (native TLS slots, current-thread id, affinity, raw
thread create/join, diagnostic register capture) and IHostSymbolResolver
(enum-keyed host function addresses baked into emitted stubs), with
Windows implementations wrapping the exact kernel32 calls the backend
made directly before.
DirectExecutionBackend takes an optional IHostPlatform (defaulting to
HostPlatform.Current) and routes every TlsAlloc/TlsFree/TlsSet/GetValue,
GetCurrentThreadId, SetThreadAffinityMask, GetModuleHandle/GetProcAddress
and the suspend+GetThreadContext diagnostic snapshot through it. The
snapshot moves wholesale into WindowsHostThreading (including the Win64
CONTEXT size/flags/offsets, which are Windows-specific by nature) and
returns a neutral HostCapturedRegisters.
NativeGuestExecutor resolves WaitForSingleObject/SetEvent/ExitThread via
the symbol resolver — the same addresses end up in the emitted run loop,
so stub bytes are unchanged — and creates/joins its raw worker thread
through IHostThreading with the same stack-reservation semantics. The
run-loop emitter itself does not move.
Marshal.GetLastWin32Error() in the affinity-failure log still observes
SetThreadAffinityMask's error because the wrapper makes no intervening
SetLastError call.
* [Host] Move fault handling and remaining backend memory ops behind the seam
Add IHostFaultHandling (handler-thunk creation, first-chance handler
install/remove, unhandled-filter set) with WindowsFaultHandling in a new
Cpu/Native/Windows/ folder. The exception-handler trampoline emitter
moves there whole — same pre-filtered NTSTATUS codes, same TEB gs:[8]/
gs:[0x10] stack-limit reads, same host-RSP TLS switch — parameterized
only by (managed callback, TLS slot, TlsGetValue address), which is
exactly what SetupExceptionHandler passed it before. Handler
installation order, the AddVectoredExceptionHandler(first=1) flag, the
SHARPEMU_DISABLE_RAW_HANDLER gate, and all install/teardown log strings
are unchanged.
Every remaining VirtualAlloc/VirtualProtect/VirtualFree/VirtualQuery/
FlushInstructionCache in the backend partials routes through IHostMemory
with 1:1 call mapping (RWX emit -> RX downgrade -> flush for stub
emission, reserve/commit for the PRT aperture and lazy-commit fault
path, raw-protection round-trips via ProtectRaw). HostRegionInfo gains
RawState/RawAllocationProtection so the lazy-commit trace lines and
protection-mask checks keep printing and comparing the exact native
values.
Windows semantics leaked as bare literals become named constants with
identical values: NTSTATUS codes (WindowsFaultCodes) and Win64 CONTEXT
byte offsets (Win64ContextOffsets, with the existing CTX_* constants
aliased to it and handler-local numeric offsets replaced by the names).
* [Host] Resolve the host platform explicitly at the composition root
SharpEmuRuntime.CreateDefault() now resolves HostPlatform.Current once
and passes it explicitly to PhysicalVirtualMemory and (via a new
optional CpuDispatcher parameter) to DirectExecutionBackend, replacing
the implicit default-argument fallbacks. On unsupported OSes boot now
fails at the root with PlatformNotSupportedException and a clear
message instead of on the first native call. A future Linux/macOS
backend plugs in by returning a different IHostPlatform here.
* [Host] Convert the platform backends to source-generated P/Invokes
Replace [DllImport] with [LibraryImport] in the four Windows backend
files added by this branch (WindowsHostMemory, WindowsHostThreading,
WindowsHostSymbolResolver, WindowsFaultHandling). Marshalling stubs are
now generated at compile time instead of JIT-emitted at runtime, which
fits the pre-JIT-everything boot model and keeps the backends
NativeAOT/trimming ready.
Interop stays zero-copy: all signatures are blittable, GetModuleHandleW
now pins the managed string via Utf16 marshalling instead of copying,
and GetProcAddress names marshal through a stack-allocated Utf8 buffer.
Implicit contracts become explicit where LibraryImport requires it:
TlsFree/TlsSetValue gain [MarshalAs(UnmanagedType.Bool)] (the 4-byte
Win32 BOOL DllImport assumed silently), and GetModuleHandle targets the
W entry point directly since LibraryImport never probes suffixes.
The CONTEXT snapshot buffer stays a NativeMemory allocation rather than
stackalloc: CONTEXT requires 16-byte alignment, now documented at the
call site. Native call sequences are unchanged.
* [Host] Address Copilot review: harden failure paths, honor injected platform
- Free the handler thunk page when the RX protection downgrade fails
(the leak predates this branch, but the failure path is boot-fatal so
releasing the page is unobservable).
- TraceThreadMode and the static diagnostics helpers now resolve host
primitives through the backend bound to the current thread, falling
back to HostPlatform.Current only when no run is active (identical on
supported configs, honors injection everywhere a backend exists).
- HostPlatform.Create additionally requires an x64 process so native
Windows ARM64 fails with the promised PlatformNotSupportedException
instead of emitting x86-64 stubs into an ARM64 process.
* Add SaveData transaction and NP UDS layout HLE stubs
Wire Prepare, Commit, and Umount2 for implicit save transactions,
unregister guest mounts on Umount2, and add NP UDS CreateEvent,
DestroyEvent, and EventPropertyObjectSetString for layout-load imports.
* Add NP UDS SetArray and PostEvent layout HLE stubs
Add sceNpUniversalDataSystemEventPropertyObjectSetArray and
sceNpUniversalDataSystemPostEvent for layout-load imports on PPSA02929.
Move KMcEa+rHsIo from libKernel MapMemory mislabel to sceAvPlayerAddSource.
Align WV1GwM32NgY ExportName with sceNpWebApi2PushEventCreateHandle. Behavior unchanged.
Implement six missing libc string/memory search and concatenation
routines in the kernel compat layer. Titles frequently call these
during startup string handling (path parsing, config lookups, format
string assembly), and without them the loader currently falls through
to unresolved-import handling.
The implementations follow the existing byte-at-a-time compat helpers
(TryReadCompat/TryWriteCompat) already used by strcpy/strncpy/memcmp,
matching native semantics: strchr/strrchr scan through and including
the terminator, memchr is bounded strictly by count, strcat/strncat
overwrite the destination terminator and re-terminate, and strstr
returns the haystack pointer for an empty needle. NIDs are the
libSceLibcInternal/libc symbol hashes for each name.
* Expand Aerolib catalog from nids.csv and wire socket/net NID handlers
Load authoritative NID pairs from scripts/nids.csv with ps5_names fallback.
Replace mislabeled kernel zero stubs with socket/connect/bind/getsockname HLE
and sceNet byte-order exports backed by the CSV symbol names.
* Add inet_pton, htons, and bzero kernel compat with CSV NIDs
Wire libc network helpers using authoritative NID names from nids.csv
instead of synthetic Gst* exports used on the crt-loader branch.
* Fix REUSE annotation for scripts/nids.csv
* Drop bundled nids.csv; extend ps5_names and regenerate Aerolib
Remove scripts/nids.csv from the repository and fold csv-only symbol names
into scripts/ps5_names.txt so Aerolib keeps the full catalog via name2nid.
* fix(vfs): defer host cursor commit in getdirentries
Follow-up VFS hardening task applying the same guest-writes-first discipline established in the time subsystem to directory enumeration.
Deferred Commit in KernelGetdirentriesCore:
- Reordered guest output so the 512-byte dirent buffer is written first via TryWriteCompat, basep is updated second (when non-null) via TryWriteUInt64Compat, and directory.NextIndex is advanced only after both guest writes succeed.
- Removed the early basep write at method entry that could mutate guest memory before buffer validation and advance the host cursor before a successful dirent delivery, causing permanent entry loss on MEMORY_FAULT at bufferAddress.
- EOF handling: when currentIndex >= Entries.Length, write basep with the final offset and return 0 without mutating NextIndex, matching FreeBSD getdirentries(2) semantics and preventing infinite retry loops.
KernelGetdents path: basePointerAddress is passed as 0, so the transaction collapses to buffer write then host cursor advance with no basep side effect.
Out of scope: coalesced {id, size} writes in sceKernelAprResolveFilepathsToIdsAndFileSizes; NetCtl connected-state stubs.
Files: KernelMemoryCompatExports.cs
* fix(vfs): resolve-first bulk commit in APR filepath resolution
Refactored sceKernelAprResolveFilepathsToIdsAndFileSizes to stop writing ids and sizes into guest memory one element at a time.
- Removed the uint.MaxValue placeholder write at the start of each loop iteration.
- Path resolution and file size lookup now fill host-side buffers first; on EFAULT or NOT_FOUND the guest ids/sizes arrays are left untouched.
- ids and sizes are packed into contiguous byte buffers and written with one TryWriteCompat call per output array instead of separate TryWriteUInt32Compat / TryWriteUInt64Compat per index.
- AmprFileRegistry.Register is called only after guest writes succeed.
- AmprFileRegistry.ComputeFileId is internal so ids can be computed without registering paths during the resolve loop.
Files: KernelMemoryCompatExports.cs, AmprFileRegistry.cs
Comprehensive refactoring of the system time subsystem to unify clock dispatching, support precise clock extensions, and secure memory boundaries against partial state corruption.
Centralized Clock Dispatch Engine:
- Extracted shared elapsed-tick calculation and clock-routing math into a unified internal static bool ResolveClockTime() dispatch engine under KernelRuntimeCompatExports.cs.
- Moved all clock identifiers from KernelMemoryCompatExports to KernelRuntimeCompatExports as internal const int constants to eliminate cross-file duplication while preserving raw compiler switch-case layout optimizations.
- Added native alias mapping support for CLOCK_REALTIME_PRECISE (9) and CLOCK_MONOTONIC_PRECISE (11).
- Hardened the Orbis sceKernelClockGettime path by routing it through the new dispatcher, resolving a pre-existing logic flaw where any non-zero clock_id incorrectly fell back to monotonic time. Invalid IDs now properly fail with ORBIS_GEN2_ERROR_INVALID_ARGUMENT.
Coalesced Single-Transaction Memory Writes:
- Replaced consecutive isolated 8-byte scalar writes across POSIX clock_gettime, gettimeofday, and Orbis sceKernelClockGettime/sceKernelGettimeofday with safe single-transaction 16-byte stackalloc byte buffer writes via BinaryPrimitives and ctx.Memory.TryWrite. This entirely prevents partial memory state corruption on virtual page boundaries.
- Implemented a single 8-byte coalesced zero-fill transaction for the deprecated/legacy timezone buffer (timezoneAddress != 0), aligning it with standard FreeBSD stub behavior.
- Standardized POSIX failure path routines. Write faults cleanly issue TrySetErrno(ctx, Efault) while safely omitting explicit manual Rax writes, letting the import dispatcher natively sign-extend the return -1 value to 0xFFFFFFFFFFFFFFFF.
Zero-Alloc Host RDTSC Execution Stub:
- Patched CreateRdtscReader() to stream native architecture opcodes out of stack-allocated spans directly into host executable memory zones (VirtualAlloc) via unsafe { Buffer.MemoryCopy(...) }, completely removing the high-frequency .ToArray() runtime allocation overhead on the hot path.
Files: KernelRuntimeCompatExports.cs, KernelMemoryCompatExports.cs
* [cpu] Implement SysV variadic float ABI (xmm0-7 capture, float returns, printf %f)
The import trampoline spilled only xmm0 and never reloaded a return xmm0. The
guest uses the System V AMD64 ABI: variadic float args pass in xmm0..xmm7 and
float/double returns come back in xmm0. As a result variadic float args past
the first were unavailable to HLE handlers, float returns never reached the
guest, and direct printf read %f/%e/%g from GP registers instead of XMM,
printing garbage and desynchronizing every following argument.
- Trampoline: spill xmm0..xmm7 into a 0x80-byte save area below the GP argpack
(r12 stays at the argpack base) and reload the return xmm0 in the epilogue.
- Gateway: read xmm0..7 from the save area into CpuContext and write the
handler's xmm0 back. XMM is caller-saved in SysV, so restoring xmm0 on return
is safe for non-float imports too.
- RegisterPrintfArgumentSource: read float args from xmm0..7 with independent
GP/FP counters and a shared stack-overflow cursor.
Every emitted byte was decoded; a unit test confirms float args read xmm0..7
(not GP) and interleaved "%d %f %d %f" stays synchronized. Build 0/0.
* [cpu] Document the scalar-only leaf-import constraint at its registration site
- IsLeafImport: spell out the no-XMM-args / no-XMM-return invariant the fast
path relies on and what breaks if it is violated; record the 2026-07-11 audit.
- Name every previously uncommented NID in the leaf list (mutex lock/unlock,
usleep, the Ampr/Apr command-buffer block, the unknown AGC packet NID).
- IsNoBlockLeafImport: document that it is a sub-filter of IsLeafImport and
that its five extra entries currently take the full gateway path; fix the
mislabeled K-jXhbt2gn4 comment (pthread_mutex_trylock, not
scePthreadMutexTrylock, which is upoVrzMHFeE).
- Point the DispatchImport call-site note at the audited list.
Comment-only change: the comment-stripped diff is empty and the solution
builds with 0 warnings / 0 errors.
Refactored parts of the time subsystem to improve POSIX/Orbis compliance and secure guest memory boundaries.
**1. POSIX `clock_gettime` updates:**
- Added `CLOCK_REALTIME_FAST` (10) and `CLOCK_MONOTONIC_FAST` (12) support for games using FreeBSD fast clock extensions.
- Fixed `NULL` pointer handling for `timespecAddress == 0`. It now returns `-1` with `EINVAL` (22) instead of `EFAULT` to match Orbis runtime behavior.
- Invalid `clock_id` values now properly fallback to `default` -> `-1` + `EINVAL`.
**2. Memory safety & monotonic tracking:**
- Replaced dual 8-byte scalar writes in both POSIX `clock_gettime` and Orbis `sceKernelClockGettime` with a single 16-byte write via `stackalloc byte[16]` and `BinaryPrimitives`. This prevents partial memory corruption at page boundaries.
- Bad non-NULL guest addresses now fail cleanly as `EFAULT` (POSIX) or `MEMORY_FAULT` (Orbis).
- Extracted core monotonic math into `GetProcessMonotonicTime()` in `KernelRuntimeCompatExports.cs` so both clock paths share the exact same `_processStartCounter` base.
**3. Host RDTSC optimization:**
- Fixed `CreateRdtscReader()` to copy stack-allocated opcode bytes into host `VirtualAlloc` memory via `unsafe { Buffer.MemoryCopy(...) }`. This completely gets rid of the redundant `.ToArray()` allocation on the hot path.
**Out of scope:** `sceKernelGettimeofday` / POSIX `gettimeofday` partial-write hardening; stricter clock validation in `sceKernelClockGettime`.
* [shader-decoder-part1] Implemented a shader decoder for Gen5 shaders, including IR generation, metadata reading, scalar evaluation, and SPIR-V translation. Updated related exports and video output components to support the new shader decoding functionality.
* [shader decoder] correct RDNA2 operands, fixing synchronization problems
* [shader-decoder] RDNA2 decoder improvements
* [shader-decoder] fix RDNA2 shift masking and sprite draws
* [shader-decoder] improve RDNA2 shader decoder to support more instructions and fix some issues with the previous implementation.
Mees van den Kieboom
Zaid Yousef
kuba
Gutemberg Ribeiro
Spooks
Miguel Cruz
StealUrKill
Gutemberg Ribeiro
Mike Saito
j92580498-max
Berk
Foued Attar
PandaCatz
Dawid