sceKernelVirtualQuery, sceKernelMunmap, sceKernelReleaseDirectMemory and the
reservation paths walked a copy of the whole allocation registry or of every
direct mapping under one lock on each call. The registry now answers lookups
from its ordered map, applies range replacements as patches instead of
rebuilding the map, and direct mappings carry an index by physical offset so a
release finds only the mappings it covers. Minecraft (PPSA17221) holds about
7,000 direct mappings and spent its world generation in these walks.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
mlockall (x7g7Ebeo8-U, aerolib) follows FreeBSD 11.4's sys_mlockall
(sys/vm/vm_mmap.c:1067-1131, freebsd-src
8c639945a44588f6cfdddfb057bb5aad88735da0) for the flag check: 0 or
bits outside MCL_CURRENT (1) and MCL_FUTURE (2) fail with EINVAL. The
title process is taken to pass priv_check(PRIV_VM_MLOCK), as mlock
already does.
Valid flags throw. MCL_FUTURE would need every later guest and host
mapping to be locked; nothing in the mapping paths does that, and
passing it to the host would make later host allocations fail above
the host RLIMIT_MEMLOCK. MCL_CURRENT would have to lock the whole
process, which on AnyPS5 also holds host stacks, heaps, modules and
driver mappings (sceKernelVirtualQuery reports them as guest memory),
with ENOMEM above a guest RLIMIT_MEMLOCK that is not known (getrlimit
throws for it). mlock pins host pages (VirtualLock, mlock) range by
range and keeps no record, so there is no state to extend. Recorded
in TechnicalDebt.
XPSemu (PPSA97358) calls it only with -overcommit mem-lock=on
(ZiZc3/XPSemu 8eec07b4, os-posix.c:330-355, reached from
system/vl.c:831-835):
int flags = MCL_CURRENT | MCL_FUTURE;
...
ret = mlockall(flags);
if (ret < 0) {
error_report("mlockall: %s", strerror(errno));
}
and exits on failure, so that option ends in the throw; without it
the title is not affected. Predicted stop removed (registry index 201,
not run):
FAIL: unresolved ELF import x7g7Ebeo8-U from libkernel.prx
guest_mlock checks EINVAL for 0, 4, 5 and -1 and the throw for 1, 2
and 3.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 87f4aa331b02d098fe27516809495447ff6a9fa6)
The libc.prx heap that Animal Well (PPSA22520 01.000.011, SDK 9.04)
bundles maps each arena with sceKernelMapFlexibleMemory(&addr, len, 3,
0x8000). It threw "Unsupported memory mapping flags 0x8000" before the
first frame. The flag is ignored as sceKernelMapNamedFlexibleMemoryInternal
already does; other unknown flags still throw.
PPSA30601 (Resonance) stops at the throwing stub. The four int* arguments follow
OpenOrbis' libkernel.h, the Orbital type database and SharpProspero; the values
follow KytyPS5 (2048 entries of 2 MiB per side) without its accounting of used
entries, listed in TechnicalDebt.md. A null pointer returns EFAULT.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Review of #3787 asked for no retries after an error in Jit.cpp.
Linux: memfd_create is called once with MFD_CLOEXEC. MFD_EXEC was
tried first and dropped on EINVAL; the flag only sets the file's execute
permission bits (Linux 6.3 and later), not whether the object maps
PROT_EXEC, which is decided by the mapping protection, as for the direct
memory memfd in DirectMemory.cpp. A one-shot choice from the headers
would fail on older kernels, so the flag is dropped; TechnicalDebt
records it.
Windows: the delete-on-close backing file is named from the process id,
a process-wide atomic counter and the performance counter, so CREATE_NEW
cannot meet an existing name, and any CreateFileW failure throws with
the Win32 error instead of looping up to 64 times on ERROR_FILE_EXISTS.
The descriptor stays a CRT descriptor on the file, so close, dup and
dup2 keep working as before.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The three JIT shared memory exports (avvJ3J0H0EY, MR221Mwo0Pc,
YKT49TOLQWs) are absent imports of three homebrew titles that cannot
finish loading without them. Predicted next stops (registry order):
- VLC PPSA85300 after getdtablesize (idx 354):
"FAIL: unresolved ELF import YKT49TOLQWs from libkernel.prx"
(Map idx 452, Create idx 462)
- XPSemu PPSA97358 after tcsendbreak (idx 214):
"FAIL: unresolved ELF import avvJ3J0H0EY from libkernel.prx"
(Create idx 398, Map idx 399)
- RetroArch PPSA99169:
"FAIL: unresolved ELF import avvJ3J0H0EY from libkernel.prx"
(Create 476, Alias 477, Map 478)
XPSemu and VLC (ps5/compat/mmap.c, __wrap_mmap) create an object for
every anonymous PROT_EXEC mapping (QEMU's TCG buffer) and map it with
the same protection, falling back to flexible memory on any non-zero
return. RetroArch's eboot runs a probe: Create("ps5-platform-probe",
0x10000, 7, &fd), Alias(fd, 3, &alias), Map(fd, 5, &address), then logs
"jit create=0x%08x alias=0x%08x map=0x%08x".
Signatures from Orbital cc408367 tools/ida/db_types.json (name, len,
maxProt, fdOut; fd, maxProt, fdOut; fd, prot, startOut), matching
ps4sdk 4df9d001 include/sce/kernel.h, ps5-jar-loader 8034177e
LibKernel.java and the three titles' call sites. SharpProspero 5dfdb4b0
declares the alias as returning the descriptor; the RetroArch call site
passes an out pointer.
The object is a host anonymous file (memfd on Linux, a delete-on-close
temporary file mapped as a PAGE_EXECUTE_READWRITE section on Windows),
the alias a duplicate descriptor, and each map maps the whole object
through the direct memory placement path, so a write through one view
is visible through the others. Bad lengths and protection bits are
EINVAL, protection past the descriptor's maximum is EACCES (FreeBSD 11
vm_map_protect), unknown descriptors are EBADF; close, dup and dup2
keep the descriptor table in step. Unverified details are recorded in
TechnicalDebt.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Direct memory can already be mapped more than once, so there is no
aliasing to enable and the call returns 0. ProsperoEden (PPSA99008)
imports it and calls it before mapping the same direct memory again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Windows path from #3788 changed the protection of any committed private host region outside the guest registry, including a range crossing a guest stack's lower bound. Ranges inside a live guest thread stack keep the path from #2982.
Pin the descriptor before mmap validation so guest descriptor reuse cannot
redirect the eventual mapping. Apply the canonical guest creation mask when
creating named objects, distinguishing new objects before setting permissions
without changing the host process umask. Add regressions that reproduce both
failures and document the remaining host requirements.
Constraint: Guest umask must not mutate the process-wide host mask
Confidence: high
Scope-risk: narrow
Tested: 17 targeted Linux runtime tests passed with zero skips; conventions and diff checks; Clang analyzer on three changed production translation units
Not-tested: Full project build and suite, Windows runtime, console hardware and titles
Complete the object lifecycle through guest mapping and unlink APIs, and
preserve FreeBSD shared-object size and seek rules across scalar, vector
and asynchronous descriptor writes. Pin descriptors during operations and
coordinate size-sensitive operations across cooperating emulator processes.
Constraint: Linux host support with existing dependencies and allocation tracking
Rejected: Bare descriptor registry | close, duplication and reuse invalidate identity
Rejected: Per-object OFD locks | forked descriptors share the same lock description
Confidence: high
Scope-risk: moderate
Directive: Keep guest resize, truncating open and bounded writes under the same namespace guard
Tested: Ten Linux runtime suites; focused shared-memory ASan/UBSan suite with warnings as errors
Not-tested: Windows build, PS5 hardware and title compatibility
Thread stacks are host-private memory; the call threw instead of setting the native protection.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
munmap and sceKernelMunmap over a range in which no page is mapped
failed: munmap returned EINVAL and sceKernelMunmap let the registry's
exception escape. Both now return 0. The console returns 0 for
sceKernelMunmap of a range it just unmapped and of a never-mapped
address, and EINVAL only for a zero length, as measured by prosper's
console oracle (kernelx.golden.tsv, kx_fmem_unmap_again,
kx_fmem_munmap_unmapped).
The other half of #3196, a range with an unmapped page in the middle, is
already fixed on main by 04ed61957; the test covers both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sceKernelAvailableFlexibleMemorySize and the flexible maps counted only
explicit flexible mappings, so the full 448 MiB was reported while the
title's executable and modules were loaded. A title that derives its code
and data size from the budget minus the available size got an unsigned
underflow.
The segments of the main executable and of every .guest.prx module that
the title's ELF gives permissions now count, rounded to 16 KiB pages, as
shadPS4 maps every non-system module segment as flexible memory. The
relinker's own blocks and the host system libraries do not count. The
charge ends when a module is unloaded.
Refs #2883
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fnrk5Dazjb2tLooVSpdRQq
A stack base need not be page aligned (it is not on Linux), so rounding the
range down first could leave the stack and miss the stack path. The range the
guest asked for is now checked against the guest stack, and the rounded
pages are protected, as FreeBSD rounds the address down.
A thread that exits with part of its stack protected left those pages to the
next thread that reused the stack. Protected stack pages are recorded, and a
thread's stack is made readable and writable again when it is unregistered at
exit.
AI-assisted (Claude Code).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Windows a guest thread runs on its host thread's stack, which is not a
registered guest allocation, so mprotect on any of its pages failed with
EINVAL. A guest that guards the bottom of its own stack (Dolphin's JIT
read-protects 64 KiB there to catch stack overflows) got an error where the
PS5 changes the protection.
A range that lies inside a live guest thread stack, with no GPU access
requested, now takes the native protection directly. Ranges that cross the
stack's bounds are still rejected.
AI-assisted (Claude Code).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The registry calls an observer on the mapping thread, after the registry
lock is released, with the ranges a kernel mapping gave GPU access:
direct and flexible maps with it, and fixed remaps with it over a
registered range (a reservation). The mutation looks them up while it
still holds the lock and hands them over weakly, with the registry
generation it ended at, so the observer does not keep a range alive.
The driver imports those ranges on its current device, so an
address-based build finds the import instead of pinning the memory on
the queue that first uses it. A range that gains GPU access through
sceKernelMprotect, and AnyPS5's own libc heap, are still imported on
first use.
The import at map does not reconcile the other imports with the
registry, so its cost does not grow with the number of imports. It skips
a range that is gone, and any range when the imports were reconciled
with a newer registry since; the next lookup imports those as before.
It installs the pin waiter as the other import paths do, so unmapping a
range imported at map retires its import. The live import bytes are kept as a
running total instead of being summed over every import. Once the import
table belongs to the current device the import takes only the table's
lock; the first import on a device (which runs the write-watch probe)
and a device that was just replaced go through the GPU lock. Clearing
the observer waits for calls already running.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A map without MAP_FIXED whose hint lies wholly inside a range still
reserved by sceKernelReserveVirtualRange is placed at the hint, as a fixed
map into the reservation would be. Hints outside reservations and ranges
that are partly mapped keep their hint behaviour. The reservation is
checked under the registry mutation, so it cannot be released and its
address reused between the check and the map (MarcZX100, #2687).
Fixes#2687
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2cY3SsTYnh4u4ZtEr2Adx
Without MAP_FIXED the address is a hint, as for FreeBSD's mmap: the reservation goes at the hint when that range is free, else at the first free range above it. The hint was dropped, so every hinted reservation landed at the lowest free address.
RetroArch (PPSA99169) reserves its cores' direct-memory heap near 0x2000000000 outside the GPU window (0x200000000-0x300000000) and refuses a range inside it; with every reservation at 0x200400000 the heap was never made, and Dolphin's first aligned allocation returned null.
AI-assisted (Claude Code).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An unmapped range inside the guest arena stays a PROT_NONE host
reservation, so host munlock succeeds on it. Check the guest mapping
state first, as msync does, and return ENOMEM for ranges the guest
has not mapped or reserved.
The arena constructor was Windows-only, so on Linux Available() was false: the guest heap fell back to the host allocator and direct memory landed wherever the kernel pleased. Titles that require a low virtual address rejected the mapping and re-issued the identical request, which surfaced as a start-up assert inside their own allocator.
Seed the window from the mappings that already exist instead of reserving it up front, because a PROT_NONE reservation of the whole range starves the kernel's own top-down mmap, which picks addresses just below the stack inside that range. Commit, reset and set-protection become portable so the arena owns its used ranges on both platforms, and fixed mappings are checked against the arena's used ranges rather than against EEXIST from a reservation.
The window-placement and no-overwrite checks in guest_memory were compiled only for Windows; they now run on Linux too, and the window check fails without this change.
Fixes#692
The registry records whether a range was mapped or protected with GPU
access (0x10/0x20), and an address-based build leaves out ranges without
it, as the console's GPU page tables do. The main image is registered
without GPU access; the title's own mprotect with GPU bits adds it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dean
MarcZX100
Yaash Singhh
Mouad-Scriptz-0
David
Adrià
jagrat Singh
Yunus Shaikh
babyhuey
xXPBSXx
screamm
abhinav29102005
winstrex
Dean Galvin
eljackjoestar-cmd
Adrià Franch
bilel
Claude
fdfawef324
Jan Šlégl
Scott Schuster
raghav314
ps54all
jeffrey12cali