gmtime, localtime, mktime, strftime and asctime took and returned the
host std::tm. The guest's FreeBSD tm has tm_gmtoff and tm_zone after the
nine calendar fields, so on Windows, where std::tm is 36 bytes, a title
reading those fields read past the thread-local result, and gmtime_s left
them unwritten. Windows gmtime_s also accepts only 1969-12-31 12:00 to
3001-01-19 21:59 UTC, where the guest libc converts any time.
Convert through a GuestTm with the guest layout on both hosts. Compute
the UTC fields from the day count, so every time whose year fits an int
converts, and on Windows take the local offset and zone name from the
CRT. gmtime reports the zone as UTC like the guest libc. localtime and
mktime keep the CRT's range on Windows; recorded in TechnicalDebt.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
On Windows, nanosleep and this_thread::sleep_for/sleep_until go
through winpthreads, which ends a wait on a 15.6 ms GetTickCount64
step. Blocking sceAudioOutOutput and sceAudioOut2ContextPush calls
returned in bursts about every 16 ms, a blocking sceAudio3dPortPush
ran at about 68 % of real time, and sceAudioInInput without a device
woke anywhere from 1 ms to 31 ms apart. They now sleep with
PreciseSleepUs or the new PreciseSleepUntil.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Retain host backing views for emulator writes and completion labels. Validate full spans, preserve write versions and GPU ordering, and keep the fallback for other mappings. Test remaps, aliases, access checks, partial failures, tracking, view lifetime, and completion ordering.
Written with AI assistance.
C17 has no binary prefix: with base 0 or 2, strtol("0b101", &end, base)
reads the subject "0", returns 0 and leaves end at the 'b'. With glibc
2.38 or later, g++ (which defines _GNU_SOURCE) binds std::strtoll,
strtoull, strtoul, strtoimax and strtoumax to the C23 __isoc23_*
functions, which accept 0b/0B: the Linux exports returned 5 and
consumed the whole string. The Windows UCRT already follows C17, so the
same guest call gave different results on the two hosts.
strtol, strtoul, strtoll, strtoull, _Stoull, strtoimax, strtoumax and
_Stoul now return 0 with the end pointer after the '0' when, after
white space and an optional sign, the text starts with 0b or 0B and a
binary digit, in base 0 or 2. Everything else still goes to the host
function. The scanf family's %i has the same glibc behaviour; it is
recorded in TechnicalDebt.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A title that replaces the allocator through its libc parameters also
supplies malloc_stats_fast in that table, and PPSA28997 calls it. It is
now forwarded to the title's function. Without a replacement allocator
that provides it, the call throws.
The repo allows one implementation per export. Move libc's rename into
libc/src/Rename.cpp, compile that file into both libc and libkernel, and
drop the second copy from libkernel's Stdio.cpp. FilesystemError moves to a
header so the shared source can use it.
The test now checks that the symbol it resolves belongs to libkernel (module
handle on Windows, dladdr on Linux) so it fails when the export is removed.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Preserve files across unmount, block new lookups until remount, and validate mount-dependent operations. Keep prior-run storage inaccessible after initialization.
Written with AI assistance.
On macOS the host keeps libc++abi: libc++ binds __cxa_throw two-level,
so moving only the personality into this runtime would leave the two
halves disagreeing on the exception header. The host personalities stay
libc++abi's and libunwind's, and libc.prx serves guest code:
- the guest's libstdc++ std::exception_ptr is modelled directly, since
libc++'s differs, and the Itanium type_info layouts that libc++abi
keeps private are declared for the runtime;
- guest frames are found through their PT_GNU_EH_FRAME, using a
dl_iterate_phdr emulation over the ELF program headers that relinked
images keep in __APS5DATA,__meta (written by relinker --macos, which
comes in a later pull request);
- host frames are found by scanning __TEXT,__eh_frame;
- sceKernelGetModuleInfoForUnwind uses the same emulation and falls back
to dladdr for host images.
The libc++ paths compile only with libc++/libc++abi and the rest only
on __APPLE__, so Linux and Windows builds are unchanged.
exception_c_cleanup throws from host code, which this runtime does not
serve on macOS, so it is not built there.
Guest C built with -fexceptions imports __gcc_personality_v0 for its
cleanup frames, and libc.prx did not export it. The routine becomes
__gcc_personality_v0_nid_postfix, the host __gcc_personality_v0
forwards to it as __gxx_personality_v0 already does, and
CallPersonality recognizes both names.
Opening a volume needs administrator rights, so the volume flush skipped
every volume in a normal user session. The libc, libkernel, save data and
app content file functions now record each host file and directory they
change. On Windows sceKernelSync flushes the recorded paths that changed
since the previous call, plus their changed parent directories, through
handles a normal user can open. Linux still calls sync().
The VA_ARGS parameter list and VarArgsAbi.hpp (RegSaveArea, FillRegSaveArea,
BuildVaList and the Xmm alias in SimdTypes.hpp) were only used by the four
Linux exports that now take their arguments with va_start.
On Linux, direct memory is a memfd mapped MAP_SHARED. amdgpu accepts only
private anonymous memory for a host pointer import (libdrm sets
AMDGPU_GEM_USERPTR_ANONONLY), so every import of direct memory failed with
VK_ERROR_INVALID_EXTERNAL_HANDLE and its ranges were copied.
When a host pointer import fails, the range's memfd slice is turned into a
dma-buf through /dev/udmabuf and imported with VK_EXT_external_memory_dma_buf.
The direct memory backing is sealed against shrinking and growing, as
udmabuf requires. libkernel reports the backing of a guest range through
GuestArenaSharedBacking. Without /dev/udmabuf, or past its size limit, the
range is copied as before. APS5_TRACE_HOST_IMPORT now prints the mappings
behind a failed import on Linux too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Write tracking protects shared guest pages read-only. A push on a fiber
stack in such a page faults, and Windows cannot dispatch it because the
exception frame lands on the same protected stack, so the process was
terminated without a report. Pin fiber contexts writable for their
lifetime; pinned pages are always collected as written.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MAP_FIXED | MAP_NO_OVERWRITE over a range that already holds an
allocation fails with ENOMEM and leaves the mapping in place, as in
fpPS4's TVirtualManager.check_fixed. UFC 6 (PPSA23566) relies on it to
probe for a free address and aborted on the throw.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CpuTopology::Get and Pin did nothing off Windows, so on a hybrid CPU the
queue workers, the draw front end and the presenter ran wherever the
scheduler put them. On a Linux kernel that reports the same capacity for
every CPU, those threads spent part of each frame on the efficiency cores.
Get now reads the process affinity with sched_getaffinity and the core
classes from /sys/devices/cpu_core/cpus and /sys/devices/cpu_atom/cpus, and
Pin restricts the calling thread with pthread_setaffinity_np. A CPU is
hybrid only when both lists exist and are disjoint, so non-hybrid CPUs are
never pinned. On Linux the workers are pinned to the performance cores of
a hybrid CPU by default; APS5_NO_WORKER_AFFINITY=1 leaves them free.
Windows keeps the APS5_WORKER_AFFINITY=1 opt-in, and
APS5_WORKER_AFFINITY_MASK=<hex> still picks the cores on any platform.
PPSA21564 on a hybrid CPU, pinned against unpinned (2 runs each): title
46.7 to 49.0 fps, menu 45.4 to 47.7, intro level 37.2 to 41.3 (space) and
31.7 to 37.6 (ships). Hub descent 6.56 to 7.92 fps (3 runs each).
A store the driver makes with the CPU into watched guest memory (a
GuestMemory::Write, a changed-bytes write-back, a completion action's label
store, the copy HLE's CPU path) faults its pages dirty in the write watch and
stamps the range with MarkWritten. The faults stayed behind: collects are
memoized per packet, so the next walk over those pages came later and stamped
them again, as a CPU write made after every check in between.
PPSA21564 fast-clears its normal G-buffer every frame with a register DCC key
fill on the GPU. While that fill is still pending, the first draw into the
target materializes the clear with a CPU store of the clear texels and a
refresh of the resident image. A dozen draws later a walk found the store's
faults, the target read as changed, and the image was uploaded again from
guest memory, dropping the draws recorded in between. Lighting then sampled
cleared normals and the level's moon and rocks went unlit in about half the
frames.
The driver's stores now consume those faults themselves: a walk before the
store stamps earlier CPU writes as such, a walk after it takes the store's
own faults as a driver store, and the range is stamped as before.
On Windows the arena's range also holds host regions that were already in
use when it was reserved (heaps, thread stacks). They are not write-watched,
so a store into one is stamped as before, without a walk.
A file read into guest memory is a host write: before the I/O it pins the shared pages it
covers, which reopens any page armed for write tracking and keeps write tracking from arming
it again until the read ends, when the pages count as written. A destination the guest cannot
write fails with EFAULT before any I/O; read and pread throw as for any failed read, AIO
reports SCE_KERNEL_ERROR_EFAULT, and APR throws. APR also throws when a read ends before the
requested size.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
MapViewOfFile3 takes an offset that is a multiple of the allocation granularity
(64 KiB), while shared guest pages are 16 KiB. An alias of a range starting at
a 16 KiB page that is not 64 KiB aligned was refused ("the section offset is not
a multiple of the allocation granularity"), so its host import failed. The view
now starts at the granularity boundary below the offset, the alias points past
the lead, and unmapping rounds back to the view's base.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
GuestMemory::CollectWrites was built on the Windows arena's MEM_WRITE_WATCH
and answered 0 on Linux, so every cached copy of guest memory (textures,
resident render targets, buffers, DCC key proofs) read as changed at every
check: resident targets were stored and re-uploaded at every refresh and
unit shadows stayed off. Astro Bot's intro ran at ~3 fps.
libc gains GuestWriteWatch, the Linux counterpart of GetWriteWatch. Every
guest mapping libkernel makes (MapAligned: fixed, hinted and aligned) is
registered with one userfaultfd in asynchronous write-protect mode
(UFFD_FEATURE_WP_ASYNC: the kernel lifts a page's protection on its first
write without a handler thread, for user and kernel-mode writes alike), and
PAGEMAP_SCAN with PM_SCAN_WP_MATCHING reports the pages written since the
last scan of a range and protects them again in one call. Unmap forgets
the range. Direct memory is mapped from shared backings over its
reservation and a fixed remap replaces the mapping, so both register the
new mapping afresh. The watch is per mapping: a write through one mapping
of shared physical pages is not seen at another, so physical pages mapped
at more than one address leave the watch in every mapping of them.
A registered mapping reads as written until a collect covers it. The watch
keeps that itself instead of relying on how the kernel classifies pages
never faulted in: Linux 6.17 reports a range without page tables as not
written and leaves it unprotected, 7.2 reports it as written. The first
collect of a fresh range reports it written and write-protects it with
UFFDIO_WRITEPROTECT, which puts markers on its unpopulated pages
(UFFD_FEATURE_WP_UNPOPULATED), so from then on only real writes report.
Fallback: the userfaultfd is opened with UFFD_USER_MODE_ONLY where
vm.unprivileged_userfaultfd restricts it, and a probe at startup checks the
mechanism on scratch pages (a populated and an unpopulated page written
after protection report exactly, untouched ones do not). Without it (kernel
older than 6.7, no userfaultfd permission, headers without PAGEMAP_SCAN, or
APS5_NO_WRITE_WATCH=1) the watch reports unavailable and the tracker stays
unwatched exactly as before: CollectWrites answers 0 and callers compare
bytes. A range with a page outside the registered mappings also answers 0,
and a protection or scan that fails reports its remaining pages as written
and answers 0.
The driver's tracker keeps its contract and generations. On Linux its
block stamps live in 4 GiB leaves over the user address space, made on
first stamp, instead of the arena's flat arrays; the collect memo now works
on both platforms. GuestMemory::WriteWatched and Watched replace the arena
queries in UnitShadowEnabled and ShadowDestinationFor, so unit shadows run
on Linux over watched memory and stay off everywhere else. The Windows
arena path is unchanged apart from going through the same stamp helpers.
Host imports: a driver may lift the write protection of imported pages
itself. amdgpu validates the userptr BO behind a VK_EXT_external_memory_host
import with a writable HMM fault on every submission after the pages were
protected again, so PAGEMAP_SCAN reports them as written with no CPU store,
and a resident display image read as written at every present (found by
Pacuka on an RX 6650 XT with RADV: the presented image went black). NVIDIA
pins the pages once at the import and stays exact from then on. A probe at
device setup decides per device: it maps and watches scratch pages, collects
them clean, imports them, collects, submits a GPU copy that reads the import,
waits, and collects again. Pages written after the submission mean imports
resolve write protection: every import from then on takes its range off the
watch (GuestMemory::Unwatch) before it is made, so Watched is false for it,
CollectWrites answers 0 and callers compare bytes; GPU writes to watched
memory keep their MarkWritten stamps. The result is logged once:
"[write-watch] host imports resolve write protection: yes/no". A probe that
cannot run takes the unwatch side. APS5_WRITE_WATCH_IMPORTS=watch|unwatch
overrides the probe (default probe; anything else throws), and
APS5_NO_WRITE_WATCH=1 still turns the watch off. On the RTX 4090 the probe
reads 0 of 4 pages written after the GPU read (4 after the import itself),
so imports stay watched there.
The import itself lifts the protection once where imports stay watched:
on the RTX 4090 the pinning at vkAllocateMemory reports every imported page
as written (4 of 4 in the probe), with no CPU store. Such a report would
read as a CPU write over any GPU results pending in the range and drop
them. So every import of a watched range is made inside an import window
(GuestMemory::ImportWatched), under the tracker lock: the range is
collected right before the import, so earlier CPU writes are stamped as
usual, and again right after it, and what the window reports is stamped
"maybe written". Readers without pending results see it as a change
(UnchangedSince fails, ChangedBlocks reports it), so caches re-upload or
compare. Decisions about pending GPU results read it as untracked:
ChangedBlocks reports BlockMaybeWritten apart from BlockWritten, and the
drop rule, the write-back's keep blocks, the alias checks, the unit
shadow's stale drop and the whole-image stale checks (through the new
WrittenSince, which also answers false for a range off the watch) store
the results instead of dropping them. A real CPU store racing an import
also races the pending GPU writes to the same memory, which has no
defined order on hardware either, so the GPU results winning is
acceptable. From the second collect on the range is protected again and
watched normally. A range the watch does not cover whole, or a collect of
the window that fails, is taken off the watch instead.
An unwatched import stays off the watch for the rest of its mapping: a
remap registers the range afresh, the import kept over it takes it off
again at the next reconcile, and a re-import after an unmap follows the
decision. Unwatch invalidates the collect memo, so a collect memoized
before the import answers 0. A range that leaves the watch while a storage
image holds a generation for it no longer reads as CPU-written:
ChangedBlocks now tells whether the range is watched, and the drop rule,
the write-back's keep blocks and the alias borrow treat an unwatched range
as untracked (stored, never dropped), as a generation of 0 is.
Tests: guest_memory checks the watch through libkernel mappings (a CPU
store, a read, a kernel write through read(), another thread, a memcpy
across pages, an mprotect cycle, a partial munmap and a fixed remap) and
over a fresh 2 MiB-aligned mapping without page tables, checks direct
memory (watched, an alias takes the shared pages off the watch in both
mappings, a fixed direct remap over flexible memory is watched afresh), and
names the failing line and the reported pages. The recorder tests allocate watched
memory on both platforms, so the UnchangedSince, key proof, close race and
unit shadow tests run on Linux, and the storage refresh test gains a
watched variant: results kept on the GPU reach a CPU reader, and a plain
CPU store into one unit with newer results pending is found by the watch
alone. The recorder tests now also run the real import probe, check the
decision with the result injected (unwatch: an import leaves the watch, a
memoized collect, a partly imported range and the unimported neighbour
answer as they should, MarkWritten stamps only watched memory, a remap and
a re-import after an unmap follow the decision; watch: the import stays
watched and stamped), and check that results pending when their memory
leaves the watch are stored rather than dropped. A further test imports the
memory of a storage image with results pending in watch mode: the
results survive the import window and reach guest memory, a cache
generation taken before the import sees a change, the window reads as
maybe written and not as a CPU store, and a CPU store after it reads as
one again. Unit shadows and the
watched storage refresh report themselves as not run when imports are
compared.
As on FreeBSD, a munmap range covering more than one registered allocation (or gaps between them)
is unmapped allocation by allocation instead of throwing "unmap crosses allocation boundaries".
GTA V unmaps such ranges in its Story Mode prologue.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
sceSaveDataSetParam dropped the params, and sceSaveDataGetParam and
sceSaveDataDirNameSearch returned zeros, so a title that checks the params of
the save it found took its profile for a new one and rewrote it with defaults
on every boot. The params now live in a <directory>.param sidecar next to the
save directory, written by SetParam (mtime refreshed) and returned by GetParam
and by the directory search, with the directory's write time when no param
was stored yet.
The mount point is /savedata<N>, as on the console: a title read its profile
through /host//_sm/0/Profile when the mount point had another shape, and that
path does not exist.
The module logs the first three calls of each entry point, like the core
module; APS5_SAVEDATA_TRACE lifts the limit.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Only the lines this branch added lose their comments; the commits before
keep them, with the reasoning behind each change, for whoever digs into
that code later.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
libkernel:
- direct memory keeps its contents per physical page across unmap and
remap; a page mapped at two live addresses at once throws
- sceKernelDirectMemoryQuery answers from tracked allocations
- SCE protections read back as set, GPU and AMPR bits included
- APR prefix path resolve, file size and stat by id, positional reads,
fsync/pread/pwrite; directories open as descriptors on Windows
- sceKernelCheckReachability, getpeername, shutdown
- Dinkumware C11 threads (_Mtx_*, _Cnd_*, _Thrd_*) in the title libc
layout; any pthread stack size, rounded at thread creation
libc:
- unaddressed guest mappings go to the system-managed area from
0x2_0000_0000, leaving the application area to fixed mappings
- the process starts in /app0
- guest sscanf on Windows; a null %s prints as (null) as on FreeBSD
- localeconv, _FNan, _FDtest, _FExp, __udivti3, wcsncpy_s, aligned
operator new, _Throw_Cpp_error
Tests: direct memory following physical pages, guest formatting.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adrià
Mohd Faour
popcoder565
TibixDev
Claude
Aryan Sharma
ps54all
ToyuAU
mugurc
Dean
kjhq
Raj
alfalfa-star
SP4C3B4R-8
Chad
Ethan Barr
Adrià Franch
Dean Galvin
boykopovar
Michon
SamG-Coder
asimplestray
STL1te