The AMM ranges were reserved at the last 64 GiB of the guest arena and
the reservation threw when that window was not free. On Windows the
stacks of the threads that exist before the arena is reserved (the main
thread and the loader's workers) are placed at random addresses and are
left in the arena as host regions; when one fell in the last 64 GiB, the
first AMM use threw "fixed guest mapping ... overlaps a host region in
the guest arena". Reserve the highest free 64 GiB window instead: the
last 64 GiB when it is free, and the window just below an occupied part
of it otherwise.
Throw when /proc/self/maps cannot be read or parsed, and when the kernel
places a reservation chunk at another address (MAP_FIXED_NOREPLACE
treated as a hint). Remove the unused PageSize and keep the POSIX-only
headers in the non-Windows block.
The Linux implementation of GuestArenaReset released the range in addition to
resetting the memory, while the Windows implementation only resets the memory and
leaves accounting to the caller. Two call sites already release explicitly after a
reset, so every one of them released the same range twice: sceKernelMunmap through
ArenaForget, and GuestHeap::release through GuestArenaRelease.
The two releases acquire the arena lock separately. Contains is span-based, so the
second release still runs, and Release erases every overlapping entry. A thread that
allocates the range between the two calls therefore has its entry erased, and the arena
hands the same guest address range out twice.
Delete the release from the Linux reset so both platforms restore the documented
contract. Every existing caller already pairs the reset with an explicit release.
The arena constructor was Windows-only, so on Linux Available() was false: the guest heap fell back to the host allocator and direct memory landed wherever the kernel pleased. Titles that require a low virtual address rejected the mapping and re-issued the identical request, which surfaced as a start-up assert inside their own allocator.
Seed the window from the mappings that already exist instead of reserving it up front, because a PROT_NONE reservation of the whole range starves the kernel's own top-down mmap, which picks addresses just below the stack inside that range. Commit, reset and set-protection become portable so the arena owns its used ranges on both platforms, and fixed mappings are checked against the arena's used ranges rather than against EEXIST from a reservation.
The window-placement and no-overwrite checks in guest_memory were compiled only for Windows; they now run on Linux too, and the window check fails without this change.
Fixes#692
macOS has no counterpart to userfaultfd write-protect or
MEM_WRITE_WATCH, so GuestWriteWatch had no backend there and the driver
re-read every guest buffer on each use.
The macOS backend write-protects watched pages. The first write to a
page faults, the SIGBUS/SIGSEGV handler records the page and restores
its protection, and Collect reports the recorded pages and protects
them again. A page is protected before its record is cleared, under the
lock the handler takes, so a write that lands in between is recorded.
The lock hands out turns in order: an unfair one starved writers under
Rosetta.
The protection is real, which adds three entry points:
- GuestWriteWatchBeginHostWrite/EndHostWrite lift it around a system
call that writes guest memory, which would otherwise fail with
EFAULT. GuestArena::HostWrite calls them, as it calls the Windows
equivalent.
- GuestWriteWatchProtectionChanged tells the watch the protection given
to watched pages by mprotect or a new mapping.
- GuestWriteWatchProtection reports that protection, since a watched
page is read-only between a collection and the next write.
Register keeps the recorded protection of a page it already watches:
read from the kernel it would be the watch's own read-only protection,
and the page would never be watched again.
The tests build without the prx libraries and run natively and through
Rosetta.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Retain host backing views for emulator writes and completion labels. Validate full spans, preserve write versions and GPU ordering, and keep the fallback for other mappings. Test remaps, aliases, access checks, partial failures, tracking, view lifetime, and completion ordering.
Written with AI assistance.
On Linux, direct memory is a memfd mapped MAP_SHARED. amdgpu accepts only
private anonymous memory for a host pointer import (libdrm sets
AMDGPU_GEM_USERPTR_ANONONLY), so every import of direct memory failed with
VK_ERROR_INVALID_EXTERNAL_HANDLE and its ranges were copied.
When a host pointer import fails, the range's memfd slice is turned into a
dma-buf through /dev/udmabuf and imported with VK_EXT_external_memory_dma_buf.
The direct memory backing is sealed against shrinking and growing, as
udmabuf requires. libkernel reports the backing of a guest range through
GuestArenaSharedBacking. Without /dev/udmabuf, or past its size limit, the
range is copied as before. APS5_TRACE_HOST_IMPORT now prints the mappings
behind a failed import on Linux too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Write tracking protects shared guest pages read-only. A push on a fiber
stack in such a page faults, and Windows cannot dispatch it because the
exception frame lands on the same protected stack, so the process was
terminated without a report. Pin fiber contexts writable for their
lifetime; pinned pages are always collected as written.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Windows the guest arena ended at 0x72_0000_0000: #231 kept its
448 GiB size after #153 had moved its start down to 0x2_0000_0000.
Titles map fixed addresses up to the application map area end,
0xFC_0000_0000 (SCE_KERNEL_APP_MAP_AREA_END_ADDR on the PS4), and a
direct memory mapping above the old end threw "shared mapping outside
the guest arena".
The arena is now reserved up to 0xFC_0000_0000. The out-of-arena
errors name the address and length, and a fixed mapping over a used
arena range says whether it hit a host region. guest_memory maps
direct memory at the last two pages below the end, reads it through
an alias and checks that a mapping at the end is refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A store the driver makes with the CPU into watched guest memory (a
GuestMemory::Write, a changed-bytes write-back, a completion action's label
store, the copy HLE's CPU path) faults its pages dirty in the write watch and
stamps the range with MarkWritten. The faults stayed behind: collects are
memoized per packet, so the next walk over those pages came later and stamped
them again, as a CPU write made after every check in between.
PPSA21564 fast-clears its normal G-buffer every frame with a register DCC key
fill on the GPU. While that fill is still pending, the first draw into the
target materializes the clear with a CPU store of the clear texels and a
refresh of the resident image. A dozen draws later a walk found the store's
faults, the target read as changed, and the image was uploaded again from
guest memory, dropping the draws recorded in between. Lighting then sampled
cleared normals and the level's moon and rocks went unlit in about half the
frames.
The driver's stores now consume those faults themselves: a walk before the
store stamps earlier CPU writes as such, a walk after it takes the store's
own faults as a driver store, and the range is stamped as before.
On Windows the arena's range also holds host regions that were already in
use when it was reserved (heaps, thread stacks). They are not write-watched,
so a store into one is stamped as before, without a walk.
A file read into guest memory is a host write: before the I/O it pins the shared pages it
covers, which reopens any page armed for write tracking and keeps write tracking from arming
it again until the read ends, when the pages count as written. A destination the guest cannot
write fails with EFAULT before any I/O; read and pread throw as for any failed read, AIO
reports SCE_KERNEL_ERROR_EFAULT, and APR throws. APR also throws when a read ends before the
requested size.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The arena is always [0x200000000, 0x7200000000). Every free, allocation-granularity-aligned
piece of the range is reserved as a placeholder; what lies between the pieces (thread stacks
and other reservations Windows made before libc starts, unaligned slivers) and the
system-reserved range are holes, used ranges a release never frees. A failed query or
reservation of the range throws.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Only the lines this branch added lose their comments; the commits before
keep them, with the reasoning behind each change, for whoever digs into
that code later.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
libkernel:
- direct memory keeps its contents per physical page across unmap and
remap; a page mapped at two live addresses at once throws
- sceKernelDirectMemoryQuery answers from tracked allocations
- SCE protections read back as set, GPU and AMPR bits included
- APR prefix path resolve, file size and stat by id, positional reads,
fsync/pread/pwrite; directories open as descriptors on Windows
- sceKernelCheckReachability, getpeername, shutdown
- Dinkumware C11 threads (_Mtx_*, _Cnd_*, _Thrd_*) in the title libc
layout; any pthread stack size, rounded at thread creation
libc:
- unaddressed guest mappings go to the system-managed area from
0x2_0000_0000, leaving the application area to fixed mappings
- the process starts in /app0
- guest sscanf on Windows; a null %s prints as (null) as on FreeBSD
- localeconv, _FNan, _FDtest, _FExp, __udivti3, wcsncpy_s, aligned
operator new, _Throw_Cpp_error
Tests: direct memory following physical pages, guest formatting.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- libc: reserve guest memory in one arena below the PS5 map limit with
MEM_WRITE_WATCH (GuestArena) and a caching guest heap; the allocation
registry gains a generation counter, an invalidator callback for the
driver's page cache and a pin waiter for leased ranges. This replaces
the fault-based GuestMemoryTracking/GuestMemoryBacking layer, whose
files are removed.
- libkernel: mmap/mprotect/munmap on the arena, flexible memory budget,
real sceKernelVirtualQuery; POSIX pthread layer, rwlocks, timed locks,
TSD, event flags; 0.5 ms scheduler tick and high-resolution sleeps;
TSC/clock/time functions; file APIs return SCE errno codes; the APR
file-streaming API.
- libc: vectored crash reporter with backtraces for fatal exceptions,
SIGABRT, UCRT invalid parameters and process exit; SSE4a EXTRQ/INSERTQ
emulation for Intel hosts; %ls/%lc/%S/%C with the guest's 16-bit
wchar_t; hand-written setjmp/longjmp for the guest jmp_buf; a path
alias table for short save-data mount points; more string, stdio and
math exports.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177gthrWEBBfBu5xJDXgBfo
Adrià
MarcZX100
Zurisen
Dean Galvin
jeffrey12cali
Elijah De Guzman
TibixDev
Adrià Franch
ToyuAU
alfalfa-star
SP4C3B4R-8
Celegans12
boykopovar
Michon
STL1te
devilismyfriend