* fix(agc): keep the driver's per-thread scratch out of thread_local teardown
With MinGW's emulated TLS, a thread_local with a destructor in a DLL is
destroyed after its storage is gone: when a std::thread ends, winpthread
runs libgcc's emutls key destructor, which frees the thread's thread_local
storage, and only then does the loader's DLL_THREAD_DETACH run the DLL's
tls_atexit list, so the destructors run on freed memory. The queue worker
touches four of them in libSceAgcDriver (the dispatch-ahead map, the
pending waits, the draw-snapshot compare vectors and the image decode
memo), and Driver::stop's join of the worker then corrupts the heap.
Under cdb's checking heap agc_driver_draw_cache_reuse stopped at the
worker's exit with heap corruption in all 127 runs, with graphics
pipeline libraries, with APS5_NO_PIPELINE_LTO=1 and with APS5_NO_GPL=1.
Without it the damage shows intermittently, as exit 0xc0000374 (2 of 143
runs with a cold NVIDIA shader cache) or as a fault in whatever allocates
next, such as NVIDIA's compiler on the optimized-link thread (#2980).
These objects now live in HostThreadLocal, fiber-local storage that owns
them, as the recorder's buffers do since e424b6b7b.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(libs): keep thread_local objects with destructors out of module teardown
The teardown order behind the driver's heap corruption holds in every
module: with MinGW's emulated TLS a thread's thread_local storage is freed
before the module's DLL_THREAD_DETACH runs its thread_local destructors,
so they run on freed objects. PreciseSleepUs's timer (used, among others,
by the video-out vblank thread) then closes a stale handle; libkernel's
directory path, socket waker and UUID random device, libSceRandom's random
device and libScePosixForWebKit's random device and host lookup free or
close stale objects. They move to HostThreadLocal. The other thread_locals
in the modules are trivially destructible or only built for Linux.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(libs): fail when a module registers a thread_local destructor
prx_thread_local_destructors lists the symbols of every built module and,
where MinGW's thread_local destructor support is linked in, disassembles
the module for calls to __cxa_thread_atexit outside libc's registration of
guest destructors. A module that keeps a thread_local with a destructor
fails it with the function that registers one: with main's libSceRandom
it reports sceRandomGetRandomNumber, and with this branch all 122 modules
pass, libc being the only one that links the support.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(agc): keep the image memo behind its pointer
The image decode memo went into HostThreadLocal as the whole array. On
Linux, where HostThreadLocal is a plain thread_local, that put 1024 memo
entries into every thread's TLS block instead of one pointer. It keeps
its unique_ptr as before, now owned by HostThreadLocal, so Linux builds
the same thread_local as main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dean Galvin <dean.galvin@live.com>
mbstowcs, and libSceAmpr's copy of it, counted at most n characters
when the destination was NULL, so the usual sizing call
mbstowcs(NULL, s, 0) returned 0 and mbstowcs(NULL, s, 1) returned 1 for
a three-character string. A caller that sizes its buffer that way got a
buffer too small for the string and no terminator.
POSIX says that with a null pwcs mbstowcs "shall return the length
required to convert the entire array regardless of the value of n", and
FreeBSD does that: lib/libc/locale/mbstowcs.c passes SIZE_T_MAX as the
source limit, and the C locale's _none_mbsnrtowcs (none.c) returns the
distance to the terminating NUL when "dst == NULL".
With a null destination, SonyMbstowcs now returns the length of the
whole string.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Kart PS5 (PPSA99611) imports __divti3 and stops at start-up on main. The guest has no signed 128-bit division; add it alongside __udivti3, dividing the magnitudes and negating the quotient, and throwing on a zero divisor like the unsigned routine.
std::random_device in libstdc++ reads RDSEED on x86 Linux. On an AMD
EPYC 9V45 (Zen 5) CI runner its 32-bit results repeat about 130 times
in 65536 draws, so guest_posix_random fails. libc now exports
HostEntropy_nid_no_patch, which uses getentropy (rand_s on Windows) and
throws when the OS cannot provide entropy. arc4random, arc4random_buf,
sceRandomGetRandomNumber, /dev/random and /dev/urandom reads,
std::_Random_device and sceKernelUuidCreate use it, and the arc4random
and random device TechnicalDebt entries say so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fopen opened its stream with _wfopen, which shares read and write
access but not delete access. While the guest held a stream open,
rename of its file failed with EIO and remove with EBUSY. POSIX rename
and remove work on open files, as on Linux and FreeBSD, and since
#3677 the descriptors sceKernelOpen returns share delete access.
fopen now creates the handle with CreateFileW sharing read, write and
delete, with the access and creation disposition of the mode, and
wraps it with _open_osfhandle and _wfdopen, keeping append mode. A
failed CreateFileW sets errno through the error mapping #3677 added to
libkernel's NativeOpen, which moves to libc as
WindowsOpenError_nid_no_patch so that both use it. freopen still
reopens through _wfreopen; TechnicalDebt records it.
Keep guest creation-mask state in libc behind C-linkage helpers so NID
patching preserves the shared exports. Reject a pinned descriptor whose
file identity differs from the snapshot taken before classification.
Exercise close and reuse from another thread at both acquisition windows
for writes, resize and seek, and create temporary ordinary-file fixtures
so runtime tests work without the source tree.
Constraint: NID exemptions require the literal suffix at the end of the exported symbol
Confidence: high
Scope-risk: narrow
Tested: 18 targeted Linux runtime tests with zero skips; patched_libraries for libc/libkernel/libkernel_sys; standalone ASan/UBSan with warnings as errors; source-prefix-remapped fixtures; Clang analyzer and conventions
Not-tested: Full project build and library set, Windows runtime, console hardware and titles
Directive: Device/inode validation establishes file identity, not open-file-description identity
mingw's fprintf writes an unbuffered stream a character at a time and
holds the stream's lock for the whole line. The APS5_LOG_* lines that
carry a RecompileRequest ("Shader ... could not be prepared after
registration", 15-20 KB each) took one WriteFile per character, and every
other thread writing to stderr waited behind them, guest calls that log
included. Demon's Souls (PPSA01342 v1.005, Windows) stops on a black
screen after two frames on main: a BPE job worker waits for the stderr
lock in sceSaveDataDirNameSearch's trace while the shader preparation
pool prints its dumps, so GameCoreLoop never finishes a frame.
The log macros now format the line into memory and write it with one
fwrite, as Driver::reportSkip does since 8cf2f6580. The UCRT writes an
unbuffered stream's fwrite as one even-length block and the last byte,
so a line takes at most two writes on Windows and one on Linux.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The AMM ranges were reserved at the last 64 GiB of the guest arena and
the reservation threw when that window was not free. On Windows the
stacks of the threads that exist before the arena is reserved (the main
thread and the loader's workers) are placed at random addresses and are
left in the arena as host regions; when one fell in the last 64 GiB, the
first AMM use threw "fixed guest mapping ... overlaps a host region in
the guest arena". Reserve the highest free 64 GiB window instead: the
last 64 GiB when it is free, and the window just below an occupied part
of it otherwise.
The guest fgets handed the size straight to the host. With a size of 0
glibc and the Windows runtime return NULL without setting errno or the
stream error indicator, and a negative size goes to the Windows
invalid-parameter path. FreeBSD's fgets checks n <= 0 before reading:
it sets __SERR on the stream, sets errno to EINVAL and returns NULL,
leaving the buffer and the file position untouched.
fgets_nid_postfix now does the same before calling the host. The
FileStream flag it uses was only set for encoding errors so far and is
renamed from SetEncodingError/encodingError to SetError/guestError; it
already stays set until clearerr like __SERR.
guest_standard_streams checks sizes 0, -1 and INT_MIN (NULL, errno 22,
error indicator set and cleared by clearerr, buffer and position
untouched) and that a following read still returns the line; it fails
without the change.
Fixes#3084
AI-assisted: yes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guest formatter threw on %n$ and *m$, which FreeBSD's vfprintf and
glibc accept. It now reads the arguments a numbered format names into a
table first, with the types their conversions give them, and follows
FreeBSD's rules for unnumbered conversions and unnamed arguments.
FreeBSD's vfprintf takes ' as the thousands-grouping flag. The Windows
formatter threw on it as an unsupported conversion. The guest's setlocale
only accepts the C locale, which has no thousands separator, so the flag
changes nothing: it is dropped before the host's printf sees it, and the
host's locale cannot add separators.
C and FreeBSD's vfprintf print a single 0 when # meets octal zero with a
precision of 0. The Windows formatter passed the conversion to the host's
printf, which printed nothing; it now asks for one digit instead.
GuestMemory.cpp reads PROCMAP_QUERY_VMA_READABLE and PROCMAP_QUERY_VMA_WRITABLE from the query result. The fallback for kernel headers without PROCMAP_QUERY now defines them too, so GuestMemory can use this header instead of its own copy.
FreeBSD fchdir(2) changes the working directory to the directory referenced by a descriptor. The guest imports fchdir from libkernel_sys (PPSA99764, issue #1578) and the module did not export it.
The descriptor's directory is resolved from the path recorded when the directory was opened on Windows, and from the /proc/self/fd link target on Linux. The guest working directory is then set through the libc routine shared with chdir. A descriptor that is not open returns EBADF and one that is not a directory returns ENOTDIR.
AI-assisted: yes
(cherry picked from commit 76329cbb08781e13d757c3ff955acf71f8bd506f)
C17 7.21.6.2p16 returns EOF only if input fails before the first conversion completes, and a suppressed conversion completes. FreeBSD counts it in nconversions (vfscanf.c, releng/11.4, lines 799-805). The host scanf returned EOF there, so sscanf("12", "%*d%d", &v) reported EOF instead of 0. Each suppressed conversion now records its end with a %n and a result of EOF becomes 0 when one was recorded.
PSXS5, SwanStationPS5 and Nuvio stop at start-up on setcontext on main,
and EVO Player and ProsperoStore do on Windows. The three are imported
by the statically linked OpenSSL's ASYNC code.
The three use FreeBSD 11's amd64 ucontext_t (mcontext at 0x10, mc_len
800, uc_link at 0x330, 0x370 bytes), which the titles are built
against: EVO Player's ASYNC_is_capable captures into a 0x370-byte stack
buffer, and async_fibre_makecontext sets uc_link at 0x330 and uc_stack
at 0x338. getcontext saves every register at its call in assembly and
stores the context as FreeBSD's sys_getcontext does: rax and rdx as 0,
the carry flag clear, the FXSAVE image and the signal mask, and nothing
from uc_link on. setcontext checks mc_len and mc_flags as set_mcontext
does, installs the signal mask, masks MXCSR with the host's MXCSR_MASK
and returns through iretq, so every general register and the flags are
restored and nothing is written below the target stack pointer.
makecontext follows FreeBSD 11's amd64 libc.
Windows checks guard-page growth and exception unwinding against the
stack bounds in the TEB. setcontext and longjmp now set them to the
stack they switch to, which makecontext registers and pins writable as
libSceFiber does for fiber stacks. OpenSSL enters a context with
setcontext and then switches with _setjmp and _longjmp.
Use unlocked POSIX EOF/error queries after unlocked byte operations so status synchronization cannot reacquire the stream lock. Keep the verified UCRT status calls on Windows, and cover byte, status, error, and held-lock behavior.
Co-authored-by: MarcZX100 <marczxrr@gmail.com>
The Windows formatter passed %p to the host CRT, which prints 16
zero-padded digits without a prefix. FreeBSD prints 0x followed by the
lowercase hex digits (0x0 for a null pointer), as if by %#lx.
On Linux every call opened /proc/self/maps and scanned it line by line with sscanf. Unity titles call it on each stack walk, and with about 900 mappings God of War Sons of Sparta (PPSA28997) spent most of its main thread there. The covering mapping is now asked of the kernel with PROCMAP_QUERY (Linux 6.11), on one descriptor opened once, as GuestMemory does; older kernels and APS5_NO_PROCMAP_QUERY keep the line scan.
AnyPS5's libc heap registered every block with GPU access, so every
address-based build imported or mirrored each malloc block the title had
made. The console's libc.prx maps its heap with protection 3 (CPU read
and write, no GPU bits), so the GPU never sees it there either.
Heap blocks are now registered without GPU access; reallocating keeps the
block's access. The AvPlayer fallback for a title that passes no texture
allocator stands in for that allocator, which hands out memory the GPU
reads, so it allocates through a new GuestHeapAlignForGpu.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The registry calls an observer on the mapping thread, after the registry
lock is released, with the ranges a kernel mapping gave GPU access:
direct and flexible maps with it, and fixed remaps with it over a
registered range (a reservation). The mutation looks them up while it
still holds the lock and hands them over weakly, with the registry
generation it ended at, so the observer does not keep a range alive.
The driver imports those ranges on its current device, so an
address-based build finds the import instead of pinning the memory on
the queue that first uses it. A range that gains GPU access through
sceKernelMprotect, and AnyPS5's own libc heap, are still imported on
first use.
The import at map does not reconcile the other imports with the
registry, so its cost does not grow with the number of imports. It skips
a range that is gone, and any range when the imports were reconciled
with a newer registry since; the next lookup imports those as before.
It installs the pin waiter as the other import paths do, so unmapping a
range imported at map retires its import. The live import bytes are kept as a
running total instead of being summed over every import. Once the import
table belongs to the current device the import takes only the table's
lock; the first import on a device (which runs the write-watch probe)
and a device that was just replaced go through the GPU lock. Clearing
the observer waits for calls already running.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dean
AlxcBTW
eljackjoestar-cmd
milescoviello
Nixitov
winstrex
Yunus Shaikh
Adrià
bilel
jweidenh
Santhosh Kumar
MarcZX100
fdfawef324
Zurisen
fablevoid
DariusHQ
Gustavo Ariel Gamarra Rojas
Tharun Balaji
Hussein Kadhim
Pouare514
Romans Pokrovskis
Adrià Franch
raghav314
David