libkernel numbered guest sockets from GuestSockets::FirstDescriptor
(0x10000000), so a title that puts a socket in a 1024-bit fd_set writes
fd / 8 bytes past it. The Epic Online Services SDK does this on its
RtcNetworkThread and three titles stop there (#1794):
Palworld PPSA23126 (queeeef, main a11c8f7f7):
FATAL: unhandled exception 0xc0000005 on thread 37164 'RtcNetworkThread'
rip 0x00007ffcdb1ea2be eossdk-ps5-shipping.prx.guest.prx+0xdea2be
rcx 0000000010000005 rsi 0000000000400000 r12 0000000010000005
The Mound PPSA26223 (xXPBSXx, main 0a8b1d32b):
FATAL: unhandled exception 0xc0000005 on thread 'RtcNetworkThread',
rip eossdk-ps5-shipping.prx.guest.prx+0xefc6f4, rcx = 0x10000005,
rsi = 0x400000
Dying Light 2 PPSA02262 (tv7):
FATAL: unhandled exception 0xc0000005 on thread 'RtcNetworkThread'
rcx 0000000010000005 ... rsi 0000000000400000 ... r12 0000000010000005
On FreeBSD 11.4 (8c639945) FD_SETSIZE is 1024 (sys/sys/select.h:58-59)
and sockets take the lowest free entry of the one process descriptor
table that files use (kern_descrip.c:1643-1660 fdalloc,
uipc_syscalls.c kern_socket through falloc).
Every socket number is now a real host descriptor: a zero-access NUL
handle registered with _open_osfhandle on Windows (as directory
descriptors already are) and an F_DUPFD_CLOEXEC duplicate of the host
socket on Linux. The socket table maps that number to the socket, so
"is this a socket" is a table lookup instead of a range check at the 31
former FirstDescriptor comparisons. close releases both the socket entry
and the number, dup and dup2 duplicate the number through the host table
and keep the socket shared, dup2 of a file onto a socket number drops the
socket entry, and sceKernelClose now closes sockets too. sceNetSocket and
sceNetAccept take their ids from the same table instead of a counter from
32, so libkernel and libSceNet numbers never collide.
select polls guest sockets alongside libSceNet sockets, the random device,
stdio and (on Linux) host descriptors; poll throws only for an open
descriptor that is not a guest socket and reports POLLNVAL otherwise.
File calls that used to fail with EBADF on the socket range return
FreeBSD's answer for a socket: lseek ESPIPE (kern_lseek), ftruncate and
getdirentries EINVAL (invfo_truncate, getvnode), flock EOPNOTSUPP;
fstat on a socket throws. pread on a closed descriptor returns EBADF.
title failing on host descriptor numbers; these three titles are that
case, and this change keeps host numbering for files and only moves
sockets into it.
Tests: guest_socket_descriptors (new: socket below 1024, select on a
socket and a file together, close frees the number for a file, dup of a
socket, sceKernelClose), guest_select (socket and pipe together, Linux),
guest_poll, guest_dup, guest_socket_pair, guest_fstatfs,
guest_posix_filesystem and guest_sce_net updated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 20378375b31eb5db922db6e55f45e44a4dae6445)
On Linux a guest stderr redirect also moves the host's: guest
descriptors are host descriptors, and the guest stderr stream wraps the
host stderr FILE. After dup2(fd, 2), or freopen(path, mode, stderr) with
full buffering, the libc terminate report, the host fprintf(stderr)
diagnostics and libstdc++'s terminate message go into the title's pipe
or file, and abort() discards what is still buffered. RetroArch
(PPSA99169) reopens stderr onto app0/trace.txt with full buffering, so
its stop on main left no message anywhere.
Before the first guest redirect of descriptor 2 (libkernel dup2, libc
freopen of a stream on descriptor 2), keep a close-on-exec duplicate of
the host descriptor 2 and point the host stderr at an unbuffered stream
on it. The guest stream keeps its own FILE and follows the redirect.
The libc terminate report writes to the kept descriptor with write()
instead of the stderr stream, and libkernel's sceKernelDebugOutText uses
the same kept descriptor.
Cloudberry Kingdom (PPSA23827, #3177) calls kqueue right after start-up
and stopped on the throwing declarations merged with #3337; Palworld's
Discord module (PPSA23126, #2648) imports kevent.
kqueue reserves a host null-device descriptor in the file descriptor
space and registers it as an event queue that reports each ready event
once per call and moves level-triggered ones behind the others, as
kqueue_scan does. close and sceKernelClose close the queue and drop the
descriptor's events from every queue, dup2 drops them for the target,
and dup/dup2 of a kqueue throw, so a reused descriptor number is never
read as a stale kqueue.
kevent follows FreeBSD 11 kern_event.c (release/11.0.1 34bd3475):
kqueue_kevent for the changelist (filter 0 skipped, EV_SYSFLAGS
cleared, EV_ERROR receipts with the errno in data, EV_RECEIPT, the
early return when nevents is exhausted), kqueue_register for EV_ADD,
EV_DELETE, EV_ENABLE, EBADF, ENOENT and EINVAL, kqueue_scan for the
timeout (EINVAL for a bad timespec, zero polls, above INT32_MAX waits
forever) and filt_timerattach, filt_usertouch for the timer and user
filters. Read and write events reuse the guest socket poll of
sceKernelAddReadEvent/WriteEvent.
Throws: other filters, EV_DISABLE, EV_DISPATCH, EV_FORCEONESHOT,
NOTE_LOWAT, EV_CLEAR on a socket, read/write events on non-socket
descriptors or another kqueue, changes to an existing timer (11.0 and
11.4 differ), a NOTE_NSECONDS fraction above one second and a negative
nevents. Recorded in TechnicalDebt.
Test guest_kqueue covers creation and close, timers, a user event
triggered from another thread, read and write events on a socket pair,
EV_ERROR receipts, the timeout and the error cases.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The three JIT shared memory exports (avvJ3J0H0EY, MR221Mwo0Pc,
YKT49TOLQWs) are absent imports of three homebrew titles that cannot
finish loading without them. Predicted next stops (registry order):
- VLC PPSA85300 after getdtablesize (idx 354):
"FAIL: unresolved ELF import YKT49TOLQWs from libkernel.prx"
(Map idx 452, Create idx 462)
- XPSemu PPSA97358 after tcsendbreak (idx 214):
"FAIL: unresolved ELF import avvJ3J0H0EY from libkernel.prx"
(Create idx 398, Map idx 399)
- RetroArch PPSA99169:
"FAIL: unresolved ELF import avvJ3J0H0EY from libkernel.prx"
(Create 476, Alias 477, Map 478)
XPSemu and VLC (ps5/compat/mmap.c, __wrap_mmap) create an object for
every anonymous PROT_EXEC mapping (QEMU's TCG buffer) and map it with
the same protection, falling back to flexible memory on any non-zero
return. RetroArch's eboot runs a probe: Create("ps5-platform-probe",
0x10000, 7, &fd), Alias(fd, 3, &alias), Map(fd, 5, &address), then logs
"jit create=0x%08x alias=0x%08x map=0x%08x".
Signatures from Orbital cc408367 tools/ida/db_types.json (name, len,
maxProt, fdOut; fd, maxProt, fdOut; fd, prot, startOut), matching
ps4sdk 4df9d001 include/sce/kernel.h, ps5-jar-loader 8034177e
LibKernel.java and the three titles' call sites. SharpProspero 5dfdb4b0
declares the alias as returning the descriptor; the RetroArch call site
passes an out pointer.
The object is a host anonymous file (memfd on Linux, a delete-on-close
temporary file mapped as a PAGE_EXECUTE_READWRITE section on Windows),
the alias a duplicate descriptor, and each map maps the whole object
through the direct memory placement path, so a write through one view
is visible through the others. Bad lengths and protection bits are
EINVAL, protection past the descriptor's maximum is EACCES (FreeBSD 11
vm_map_protect), unknown descriptors are EBADF; close, dup and dup2
keep the descriptor table in step. Unverified details are recorded in
TechnicalDebt.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds tcsendbreak (vXE-dIUIZlE), tcdrain (brGGJWsZauw), tcflush
(FxHW5NuEXsY), tcflow (OG8xRaGIpqk), tcgetpgrp (5OEl1mRhUm0) and
tcsetpgrp (YMuesZTyTl4) next to tcgetattr and tcsetattr in
File/src/Terminal.cpp. The NIDs come from tools/nid_names.py and match
aerolib.csv (zecoxao/sce_symbols 2883963a).
They follow FreeBSD 11.4 lib/libc/gen/termios.c (8c639945), which
implements each one over a terminal ioctl: tcflush returns EINVAL for a
queue other than TCIFLUSH, TCOFLUSH or TCIOFLUSH and tcflow for an
action outside TCOOFF..TCION before the descriptor is used; every other
call fails with the error the merged tcgetattr gives (EBADF for a closed
descriptor, ENOTTY for a file or pipe, EADDRNOTAVAIL or EOPNOTSUPP for a
socket, per soo_ioctl and in_control/in6_control). tcsendbreak fails on
TIOCSBRK before its 0.4 s pause, and tcflow's TCIOFF/TCION fail in
tcgetattr. FreeBSD's pipe_ioctl answers TIOCGPGRP/TIOCSPGRP on a pipe or
FIFO with the SIGIO owner, which is not modelled, so tcgetpgrp and
tcsetpgrp throw on a host pipe above descriptor 2.
Titles (homebrew catalog #1578), predicted stops from registry order:
- PS5MC PPSA99420 imports tcsendbreak (442), tcdrain (443), tcflush
(444), tcflow (445), tcgetpgrp (462) and tcsetpgrp (463); once pause
and setrlimit are in, it stops at
"FAIL: unresolved ELF import vXE-dIUIZlE from libkernel.prx".
- XPSemu PPSA97358 imports tcsendbreak (214), its first uncovered import
after execv, sigwait, setrlimit, mlockall, setuid and getopt:
"FAIL: unresolved ELF import vXE-dIUIZlE from libkernel.prx".
guest_terminal covers each call over stdin, stdout, stderr, a file, both
pipe ends and an AF_INET socket, the EINVAL cases (also on -1), and
EBADF for -1, a closed socket, a closed pipe and a closed file.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Titles check for a plugin with sceKernelStat before loading it through the engine path. The relinker emits only X.prx.guest.prx, so the check failed and PSNCore started with the raw (0, NULL) arguments. Share the dlopen mapping with sceKernelStat, read-only sceKernelOpen and sceKernelCheckReachability when the original name is absent.
dup and dup2 of a Windows directory descriptor threw. FreeBSD's
kern_dup makes both descriptors refer to one open file, so they share
the read position; the duplicate now points at the same DirectoryState,
which getdents and lseek go through, and keeps it after the original
is closed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ProsperoRadio (PPSA99001) stops at 33.7 s on Windows, with #3649
applied so that it gets past poll, on the throw from #3840:
what(): fcntl: record locks are not implemented on Windows, command=12, fd=3
F_GETLK, F_SETLK, F_SETLKW and the F_OGETLK, F_OSETLK and F_OSETLKW
forms now follow FreeBSD 11.4's kern_fcntl and lf_advlockasync
(8c639945): an SEEK_CUR start is resolved against the descriptor offset
(EOVERFLOW past OFF_MAX), F_RDLCK needs read access and F_WRLCK write
access (EBADF), another type is EINVAL, and the range is checked against
the file size for SEEK_END (EINVAL for a negative start or an unknown
whence, EOVERFLOW past OFF_MAX). The guest is one process, and
lf_blocks never lets locks of one owner conflict, so whichever
descriptor or thread asks, F_SETLK and F_SETLKW succeed without waiting
and F_GETLK reports F_UNLCK, as on the console. Nothing is held on the
host: Windows byte-range locks are mandatory and would fail the guest's
own reads and writes through other descriptors. A lock from the end of
a directory throws.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A Windows GUI build started without standard handles has no descriptor 2.
_dup(2) then reached the invalid-parameter handler, which aborts, before the
"cannot keep the host standard error" error could be thrown. It now goes
through WithoutParameterHandler like the other descriptor calls in this file.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Guest descriptors are host descriptors, so a guest that moves standard
error with dup2 also moves the stream sceKernelDebugOutText writes to. A
title that forwards its standard error to the kernel log through a pipe
then reads its own debug output back and logs it again without end. Before
the guest's first dup2 onto descriptor 2, the host's standard error is kept,
and sceKernelDebugOutText writes there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
_wstat64, the FindFirstFileExW listing and std::filesystem::file_size
report the size of a file symbolic link itself (0). stat therefore gave
0 for linked files, and APR resolved them as empty, so Unity treated a
linked globalgamemanagers as empty and never read it.
stat takes the size from the handle it already opens, which follows the
link. The APR listing skips reparse points, and names missing from it
are sized through stat.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fcntl and sceKernelFcntl read a third argument as a 64-bit integer for
every command on a non-socket descriptor. F_GETFD and F_GETFL pass no
third argument, and the commands that take one pass an int or a pointer.
They now read nothing, an int or a pointer, by command. An int read as
a 64-bit value kept whatever the caller left in the upper half of the
register, so F_READAHEAD -1 enabled read-ahead instead of disabling it.
ioctl and _ioctl answered EBADF for every descriptor outside the
libSceNet socket table. Decoded call sites: PS5MC (PPSA99420, FIONBIO x3,
FIONREAD x2, TIOCGWINSZ x3), ProsperoTV (PPSA99003, FIONBIO on a pipe
once poll on pipes is in), XPSemu (PPSA97358, 21 sites).
Following FreeBSD 11.4 kern_ioctl (sys/kern/sys_generic.c:758),
vn_ioctl (sys/kern/vfs_vnops.c:1488), pipe_ioctl (sys/kern/sys_pipe.c:1346)
and randomdev_ioctl (sys/dev/random/randomdev.c:301), release/11.4.0
8c639945: FIONBIO sets or clears O_NONBLOCK through the same status
flags as fcntl F_SETFL (EFAULT for a null argument before EBADF);
FIONREAD reports the size past the offset of a file, the queued bytes of
a pipe's read end, 0 on its write end, ENOTTY on the random device, and
throws on a directory or another character device; termios requests
(group 't', TIOCGWINSZ included) fail with ENOTTY, as tcgetattr already
takes these descriptors as not a terminal; any other request throws
naming the request and descriptor kind. Sockets keep their path
unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
fcntl and sceKernelFcntl answered EBADF (sceKernelFcntl threw) for
every descriptor outside the libSceNet socket table, so files, pipes,
directories and descriptors 0 to 2 could not be queried or changed.
Observed: Yamagi Quake II (PPSA99007) stops with
"Sys_Error: change stdin to blocking Bad file descriptor", from
fcntl(fileno(stdin), F_SETFL, fcntl(0, F_GETFL, 0) & ~FNDELAY)
(yquake2 src/backends/unix/system.c:81-86).
Predicted from the titles' sources and decoded call sites: XPSemu
(PPSA97358, 97 fcntl sites: F_GETFD 12, F_SETFD 20, F_GETFL 8,
F_SETFL 9, F_DUPFD 1, F_DUPFD_CLOEXEC 7, F_GETLK 9) fails in
qemu_init_main_loop, where its pipe2 (pipe plus F_SETFD FD_CLOEXEC) and
glib's g_unix_set_fd_nonblocking (F_GETFL, F_SETFL) build the event
notifier (util/event_notifier-posix.c:44-67); VLC (PPSA85300, 43 sites,
F_RDAHEAD on every file it opens), RetroArch (PPSA99169, 17),
Porpoise (PPSA99764, 13) and Ruffle (9) call it on files and pipes.
The commands follow FreeBSD 11.4 kern_fcntl and kern_dup
(sys/kern/kern_descrip.c:479-799, 812, release/11.4.0 8c639945):
F_GETFD/F_SETFD keep the close-on-exec flag per descriptor, F_GETFL
reports the host access mode with the status flags, F_SETFL changes
only O_NONBLOCK, O_APPEND, O_ASYNC, O_FSYNC, O_DIRECT and FRDAHEAD,
F_DUPFD/F_DUPFD_CLOEXEC give the lowest free descriptor at or above the
start through dup (EINVAL for a negative start), F_DUP2FD goes through
dup2, F_RDAHEAD/F_READAHEAD set FRDAHEAD on files (EBADF on pipes,
ENOTTY elsewhere), and unknown commands fail with EINVAL. Status flags
live in a guest table shared by dup, dup2 and F_DUPFD like an open file
description. O_NONBLOCK switches a pipe (PIPE_NOWAIT on Windows, where an
empty read and a full write now report EAGAIN; the host flag on Linux).
Record locks use the host POSIX locks on Linux and throw on Windows.
Changes the host cannot model throw naming the flag and descriptor kind.
Sockets keep their path unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(agc): keep the driver's per-thread scratch out of thread_local teardown
With MinGW's emulated TLS, a thread_local with a destructor in a DLL is
destroyed after its storage is gone: when a std::thread ends, winpthread
runs libgcc's emutls key destructor, which frees the thread's thread_local
storage, and only then does the loader's DLL_THREAD_DETACH run the DLL's
tls_atexit list, so the destructors run on freed memory. The queue worker
touches four of them in libSceAgcDriver (the dispatch-ahead map, the
pending waits, the draw-snapshot compare vectors and the image decode
memo), and Driver::stop's join of the worker then corrupts the heap.
Under cdb's checking heap agc_driver_draw_cache_reuse stopped at the
worker's exit with heap corruption in all 127 runs, with graphics
pipeline libraries, with APS5_NO_PIPELINE_LTO=1 and with APS5_NO_GPL=1.
Without it the damage shows intermittently, as exit 0xc0000374 (2 of 143
runs with a cold NVIDIA shader cache) or as a fault in whatever allocates
next, such as NVIDIA's compiler on the optimized-link thread (#2980).
These objects now live in HostThreadLocal, fiber-local storage that owns
them, as the recorder's buffers do since e424b6b7b.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(libs): keep thread_local objects with destructors out of module teardown
The teardown order behind the driver's heap corruption holds in every
module: with MinGW's emulated TLS a thread's thread_local storage is freed
before the module's DLL_THREAD_DETACH runs its thread_local destructors,
so they run on freed objects. PreciseSleepUs's timer (used, among others,
by the video-out vblank thread) then closes a stale handle; libkernel's
directory path, socket waker and UUID random device, libSceRandom's random
device and libScePosixForWebKit's random device and host lookup free or
close stale objects. They move to HostThreadLocal. The other thread_locals
in the modules are trivially destructible or only built for Linux.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(libs): fail when a module registers a thread_local destructor
prx_thread_local_destructors lists the symbols of every built module and,
where MinGW's thread_local destructor support is linked in, disassembles
the module for calls to __cxa_thread_atexit outside libc's registration of
guest destructors. A module that keeps a thread_local with a destructor
fails it with the function that registers one: with main's libSceRandom
it reports sceRandomGetRandomNumber, and with this branch all 122 modules
pass, libc being the only one that links the support.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(agc): keep the image memo behind its pointer
The image decode memo went into HostThreadLocal as the whole array. On
Linux, where HostThreadLocal is a plain thread_local, that put 1024 memo
entries into every thread's TLS block instead of one pointer. It keeps
its unique_ptr as before, now owned by HostThreadLocal, so Linux builds
the same thread_local as main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dean Galvin <dean.galvin@live.com>
std::random_device in libstdc++ reads RDSEED on x86 Linux. On an AMD
EPYC 9V45 (Zen 5) CI runner its 32-bit results repeat about 130 times
in 65536 draws, so guest_posix_random fails. libc now exports
HostEntropy_nid_no_patch, which uses getentropy (rand_s on Windows) and
throws when the OS cannot provide entropy. arc4random, arc4random_buf,
sceRandomGetRandomNumber, /dev/random and /dev/urandom reads,
std::_Random_device and sceKernelUuidCreate use it, and the arc4random
and random device TechnicalDebt entries say so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adrià
Dean
MarcZX100
screamm
jagrat Singh
marshmll
David
David
Eli Yarson
LeandroLP23
lorkydey
Ori Shamir
Scott Schuster
Ondra97
winstrex
Santhosh Kumar
Ahir
AlxcBTW
Nithishvelan M
ArjenS-LA
PrintOrb