The Windows path from #3788 changed the protection of any committed private host region outside the guest registry, including a range crossing a guest stack's lower bound. Ranges inside a live guest thread stack keep the path from #2982.
Pin the descriptor before mmap validation so guest descriptor reuse cannot
redirect the eventual mapping. Apply the canonical guest creation mask when
creating named objects, distinguishing new objects before setting permissions
without changing the host process umask. Add regressions that reproduce both
failures and document the remaining host requirements.
Constraint: Guest umask must not mutate the process-wide host mask
Confidence: high
Scope-risk: narrow
Tested: 17 targeted Linux runtime tests passed with zero skips; conventions and diff checks; Clang analyzer on three changed production translation units
Not-tested: Full project build and suite, Windows runtime, console hardware and titles
Complete the object lifecycle through guest mapping and unlink APIs, and
preserve FreeBSD shared-object size and seek rules across scalar, vector
and asynchronous descriptor writes. Pin descriptors during operations and
coordinate size-sensitive operations across cooperating emulator processes.
Constraint: Linux host support with existing dependencies and allocation tracking
Rejected: Bare descriptor registry | close, duplication and reuse invalidate identity
Rejected: Per-object OFD locks | forked descriptors share the same lock description
Confidence: high
Scope-risk: moderate
Directive: Keep guest resize, truncating open and bounded writes under the same namespace guard
Tested: Ten Linux runtime suites; focused shared-memory ASan/UBSan suite with warnings as errors
Not-tested: Windows build, PS5 hardware and title compatibility
Thread stacks are host-private memory; the call threw instead of setting the native protection.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
munmap and sceKernelMunmap over a range in which no page is mapped
failed: munmap returned EINVAL and sceKernelMunmap let the registry's
exception escape. Both now return 0. The console returns 0 for
sceKernelMunmap of a range it just unmapped and of a never-mapped
address, and EINVAL only for a zero length, as measured by prosper's
console oracle (kernelx.golden.tsv, kx_fmem_unmap_again,
kx_fmem_munmap_unmapped).
The other half of #3196, a range with an unmapped page in the middle, is
already fixed on main by 04ed61957; the test covers both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sceKernelAvailableFlexibleMemorySize and the flexible maps counted only
explicit flexible mappings, so the full 448 MiB was reported while the
title's executable and modules were loaded. A title that derives its code
and data size from the budget minus the available size got an unsigned
underflow.
The segments of the main executable and of every .guest.prx module that
the title's ELF gives permissions now count, rounded to 16 KiB pages, as
shadPS4 maps every non-system module segment as flexible memory. The
relinker's own blocks and the host system libraries do not count. The
charge ends when a module is unloaded.
Refs #2883
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fnrk5Dazjb2tLooVSpdRQq
A stack base need not be page aligned (it is not on Linux), so rounding the
range down first could leave the stack and miss the stack path. The range the
guest asked for is now checked against the guest stack, and the rounded
pages are protected, as FreeBSD rounds the address down.
A thread that exits with part of its stack protected left those pages to the
next thread that reused the stack. Protected stack pages are recorded, and a
thread's stack is made readable and writable again when it is unregistered at
exit.
AI-assisted (Claude Code).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Windows a guest thread runs on its host thread's stack, which is not a
registered guest allocation, so mprotect on any of its pages failed with
EINVAL. A guest that guards the bottom of its own stack (Dolphin's JIT
read-protects 64 KiB there to catch stack overflows) got an error where the
PS5 changes the protection.
A range that lies inside a live guest thread stack, with no GPU access
requested, now takes the native protection directly. Ranges that cross the
stack's bounds are still rejected.
AI-assisted (Claude Code).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The registry calls an observer on the mapping thread, after the registry
lock is released, with the ranges a kernel mapping gave GPU access:
direct and flexible maps with it, and fixed remaps with it over a
registered range (a reservation). The mutation looks them up while it
still holds the lock and hands them over weakly, with the registry
generation it ended at, so the observer does not keep a range alive.
The driver imports those ranges on its current device, so an
address-based build finds the import instead of pinning the memory on
the queue that first uses it. A range that gains GPU access through
sceKernelMprotect, and AnyPS5's own libc heap, are still imported on
first use.
The import at map does not reconcile the other imports with the
registry, so its cost does not grow with the number of imports. It skips
a range that is gone, and any range when the imports were reconciled
with a newer registry since; the next lookup imports those as before.
It installs the pin waiter as the other import paths do, so unmapping a
range imported at map retires its import. The live import bytes are kept as a
running total instead of being summed over every import. Once the import
table belongs to the current device the import takes only the table's
lock; the first import on a device (which runs the write-watch probe)
and a device that was just replaced go through the GPU lock. Clearing
the observer waits for calls already running.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A map without MAP_FIXED whose hint lies wholly inside a range still
reserved by sceKernelReserveVirtualRange is placed at the hint, as a fixed
map into the reservation would be. Hints outside reservations and ranges
that are partly mapped keep their hint behaviour. The reservation is
checked under the registry mutation, so it cannot be released and its
address reused between the check and the map (MarcZX100, #2687).
Fixes#2687
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2cY3SsTYnh4u4ZtEr2Adx
Without MAP_FIXED the address is a hint, as for FreeBSD's mmap: the reservation goes at the hint when that range is free, else at the first free range above it. The hint was dropped, so every hinted reservation landed at the lowest free address.
RetroArch (PPSA99169) reserves its cores' direct-memory heap near 0x2000000000 outside the GPU window (0x200000000-0x300000000) and refuses a range inside it; with every reservation at 0x200400000 the heap was never made, and Dolphin's first aligned allocation returned null.
AI-assisted (Claude Code).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An unmapped range inside the guest arena stays a PROT_NONE host
reservation, so host munlock succeeds on it. Check the guest mapping
state first, as msync does, and return ENOMEM for ranges the guest
has not mapped or reserved.
The arena constructor was Windows-only, so on Linux Available() was false: the guest heap fell back to the host allocator and direct memory landed wherever the kernel pleased. Titles that require a low virtual address rejected the mapping and re-issued the identical request, which surfaced as a start-up assert inside their own allocator.
Seed the window from the mappings that already exist instead of reserving it up front, because a PROT_NONE reservation of the whole range starves the kernel's own top-down mmap, which picks addresses just below the stack inside that range. Commit, reset and set-protection become portable so the arena owns its used ranges on both platforms, and fixed mappings are checked against the arena's used ranges rather than against EEXIST from a reservation.
The window-placement and no-overwrite checks in guest_memory were compiled only for Windows; they now run on Linux too, and the window check fails without this change.
Fixes#692
The registry records whether a range was mapped or protected with GPU
access (0x10/0x20), and an address-based build leaves out ranges without
it, as the console's GPU page tables do. The main image is registered
without GPU access; the title's own mprotect with GPU bits adds it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Windows, sceKernelMprotect accepted only the executable's image and threw
"Memory protection of a foreign image is not supported" for every other
image. Bundled title modules are relinked into DLLs named <module>.guest.prx,
so a title that reprotects its own module pages failed. Dying Light 2's
bundled libNRD does this at start-up (#1794).
DoMprotect now also accepts images whose file name ends in .guest.prx, and
registers their committed pages like the executable's. Other images are
still refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A zero, unaligned, overflowing or out-of-range length reached the physical pool, where a sub-page length silently freed a whole page and fragmented the block map, and an out-of-range length walked the page table past the backing store. Validate alignment and bounds up front and return EINVAL instead.
An anonymous private mapping with MAP_ALIGNED_SUPER (FreeBSD MAP_ALIGNED(1))
is placed on a 2 MiB superpage boundary. Yamagi Quake II requests it for its
hunk allocations of 2 MiB and more.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every direct memory allocation had a memfd of its own, so each one the
GPU used became its own udmabuf and its own imported buffer object. An
imported buffer object cannot be always valid in the GPU VM: amdgpu
validates every one of them at every command submission. PPSA14632
(LEGO Horizon Adventures) imports about 7000 allocations, 5800 of them
64 KiB, and each vkQueueSubmit took about 2.3 ms (5 s per 10 s, held
under the GPU mutex). A standalone test shows the cost follows the
count of imported dma-buf buffer objects, not their size or the count
of ordinary allocations.
- libkernel: on Linux all direct memory lives in one sparse memfd of
DIRECT_MEMORY_SIZE, at file offset = physical address
(PhysicalBacking keeps the file and its offset; SharedBacking returns
that offset). A range allocated again is zeroed through a temporary
mapping, as the old per-allocation memfd was.
- Driver: an import of shared direct memory that lies inside one
64 MiB chunk of the file binds its own VkBuffer into that chunk's
memory, one udmabuf import per chunk (HostImport::chunk holds it, so
it lives until the last buffer bound to it retires). Ranges that
cross a chunk or do not meet the buffer alignment are imported alone
as before.
On PPSA14632 vkQueueSubmit drops from about 2400 us to 46 us on
average, the GPU idles 36 ms instead of 172 ms per present, and
presents go from about 40 to 80-97 per 10 s.
Released direct memory stays committed in the file (see
TechnicalDebt.md): a live chunk import pins its pages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Retain host backing views for emulator writes and completion labels. Validate full spans, preserve write versions and GPU ordering, and keep the fallback for other mappings. Test remaps, aliases, access checks, partial failures, tracking, view lifetime, and completion ordering.
Written with AI assistance.
Use the current registered range after protection splits and partial unmaps, including FIND_NEXT lookups. Add a regression covering extents, protection and surviving contents.
AI-assisted: yes
msync follows FreeBSD 11's kern_msync and vm_map_sync over the guest's
mappings as sceKernelVirtualQuery reports them: the address is rounded
down to the page, a range that wraps and MS_ASYNC with MS_INVALIDATE are
EINVAL, an unmapped start or page is ENOMEM, and length 0 checks the
mapping that holds the address. The guest has no file mappings, so a
successful call has nothing to write back. MS_INVALIDATE on a mapped
range throws, since FreeBSD's EBUSY for mlocked pages needs lock state
that is not tracked.
EVO-PLAYER PPSA99039 v0.11.0 imports it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sceKernelMunlock and the POSIX munlock unlock the page-rounded range with
FreeBSD's results: 0 for a mapped range whether or not it is locked, EINVAL
for a range that wraps around zero, ENOMEM for an unmapped range. On Linux
the host munlock gives them. On Windows a reserved placeholder counts as
mapped only while the guest reservation or protection registry covers it,
and a region VirtualUnlock refuses as not fully locked is unlocked page by
page, as VirtualUnlock is all-or-nothing. sceKernelMlock shares the range
rounding.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012GLfKH7Sqe51wX254k1XsV
The POSIX form of sceKernelMlock: 0 on success, -1 with errno (EINVAL,
ENOMEM, EPERM, EAGAIN) where the sce form returns the kernel error, as
munmap and mprotect do beside it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Releasing direct memory left the virtual ranges that mapped it registered,
so sceKernelVirtualQuery reported them as flexible memory at offset 0 and
titles walking a range with FIND_NEXT released physical page 0.
Collect every view of the released pages before freeing them and unmap
them through UnmapRegistered, as shadPS4's MemoryManager::Free does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mprotect changes the protection of guest memory through the same path
as sceKernelMprotect and reports failures through errno: EINVAL for a
null or overflowing range or a range the allocation tracker rejects,
ENOMEM when allocation fails. A zero length succeeds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A fixed mapping without no-overwrite unmaps the registered pieces it overlaps before mapping, as it already did when the range lay inside registered allocations.
On Linux, direct memory is a memfd mapped MAP_SHARED. amdgpu accepts only
private anonymous memory for a host pointer import (libdrm sets
AMDGPU_GEM_USERPTR_ANONONLY), so every import of direct memory failed with
VK_ERROR_INVALID_EXTERNAL_HANDLE and its ranges were copied.
When a host pointer import fails, the range's memfd slice is turned into a
dma-buf through /dev/udmabuf and imported with VK_EXT_external_memory_dma_buf.
The direct memory backing is sealed against shrinking and growing, as
udmabuf requires. libkernel reports the backing of a guest range through
GuestArenaSharedBacking. Without /dev/udmabuf, or past its size limit, the
range is copied as before. APS5_TRACE_HOST_IMPORT now prints the mappings
behind a failed import on Linux too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A title maps direct memory with protection 0x3f2, which adds bits
0x100 and 0x200 to the CPU, GPU and AMPR bits, and the mapping threw.
They are the audio coprocessor (ACP) read and write bits, as named in
the KytyPS5 and etaHEN headers. The coprocessor work runs on the host
CPU, so they map to host read and read/write access like the GPU bits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dean
Dean Galvin
Adrià
Adrià Franch
bilel
Claude
xXPBSXx
fdfawef324
Jan Šlégl
Scott Schuster
raghav314
ps54all
jeffrey12cali
Lexroid
Pouare514
Hannah Lyons
alfalfa-star
TibixDev
MarcZX100
Praveen Kumar
hikazey
Andreas
kjhq
Gjon Gojcevic
JAB-dev