Commit Graph

876 Commits

Author SHA1 Message Date
boykopovar b22ede6c93 refactor(cmake): move some libs test definitions to local cmakelists 2026-10-01 12:21:46 +03:00
SamG-Coder 7f834fc415 feat(libc): add C locale conversion and sample runtime exports 2026-10-01 11:58:51 +03:00
boykopovar e57ee5120d
Merge pull request #206 from Praetorian0420/main
Implement network control and socket functionality improvements
2026-10-01 11:41:41 +03:00
boykopovar 33e11c7ea2
Merge pull request #203 from oneandonlydean/fix/libkernel-direct-memory-memfd
fix(libkernel): back direct memory with a memfd on Linux
2026-10-01 11:31:43 +03:00
Praetorian0420 e3eeac4a81 Implement network control and socket functionality improvements
- Enhanced the network control library to check for host addresses and update connection states accordingly.
- Added support for socket operations including connect, listen, accept, send, and recv in the UDP and TCP socket implementations.
- Introduced error handling for various socket operations, ensuring proper feedback for unsupported actions.
- Created comprehensive tests for both TCP and UDP socket functionalities, validating the expected behavior of the network stack.
- Refactored existing code to improve clarity and maintainability, including the removal of unused functions and the addition of necessary error checks.
2026-09-30 23:29:01 -05:00
Dean Galvin d544596283 fix(libkernel): back direct memory with a memfd on Linux
std::tmpfile() places the shared direct memory backing in /tmp, so a
title's direct memory counts against that filesystem's size (a tmpfs is
half of RAM by default): once /tmp fills, touching a backing page raises
SIGBUS. memfd_create gives the same shared memory pages without a
filesystem or its size limit.
2026-10-01 16:06:42 +13:00
Michon 1d806a4688 fix(libkernel): honour MAP_FIXED in sceKernelReserveVirtualRange and sceKernelMemoryPoolReserve
Both ignored the flags and the requested address and reserved wherever the arena
had room, so a title that reserves a fixed range (Marvel's Wolverine passes
0x400010, MAP_FIXED | MAP_NO_COALESCE) and then uses the address it asked for
touched memory it never reserved. A fixed reservation now lands at the address,
after checking the range is free; other reservations are placed as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 00:00:22 +02:00
boykopovar c71be475fc
Merge pull request #187 from MichonGoddijn231849/feat/demons-souls-exports
feat: export the AMPR command buffer waits and writes, sceAgcFuseShaderHalves, sceAgcSetSubmitMode, sceNpSessionSignalingCreateContext and sceNetEpollAbort
2026-09-30 22:03:54 +03:00
boykopovar 95f34bed3d
Merge pull request #171 from asimplestray/feat/kernel-time-signal
feat(kernel): implement Sleep, sigprocmask and pthread finalize
2026-09-30 21:03:48 +03:00
boykopovar c3aed31639
Merge pull request #169 from asimplestray/feat/kernel-file-ops
feat(kernel): implement file metadata ops and address conversion
2026-09-30 21:03:08 +03:00
boykopovar ba1d32f450
Merge pull request #168 from asimplestray/feat/kernel-psem
feat(kernel): implement pthread semaphore lifecycle
2026-09-30 21:01:53 +03:00
boykopovar 6756edb91f
Merge pull request #164 from MichonGoddijn231849/fix/munmap-across-mappings
fix(libkernel): unmap ranges that span several mappings
2026-09-30 20:55:43 +03:00
asimplestray a4e755ccb1 feat(kernel): implement file metadata ops and address conversion 2026-09-30 16:51:37 +00:00
Michon 2b5f2d32d3 feat: export the AMPR command buffer waits and writes, sceAgcFuseShaderHalves, sceAgcSetSubmitMode, sceNpSessionSignalingCreateContext and sceNetEpollAbort
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 18:19:55 +02:00
boykopovar 077d4a487f
Merge pull request #125 from oneandonlydean/feat/pthread-prio-inherit
feat(libkernel): accept priority-inheritance mutexes
2026-09-30 18:11:38 +03:00
Dean Galvin d8c7c0ccdf feat(agc): track guest memory writes on Linux with async userfaultfd write protection
GuestMemory::CollectWrites was built on the Windows arena's MEM_WRITE_WATCH
and answered 0 on Linux, so every cached copy of guest memory (textures,
resident render targets, buffers, DCC key proofs) read as changed at every
check: resident targets were stored and re-uploaded at every refresh and
unit shadows stayed off. Astro Bot's intro ran at ~3 fps.

libc gains GuestWriteWatch, the Linux counterpart of GetWriteWatch. Every
guest mapping libkernel makes (MapAligned: fixed, hinted and aligned) is
registered with one userfaultfd in asynchronous write-protect mode
(UFFD_FEATURE_WP_ASYNC: the kernel lifts a page's protection on its first
write without a handler thread, for user and kernel-mode writes alike), and
PAGEMAP_SCAN with PM_SCAN_WP_MATCHING reports the pages written since the
last scan of a range and protects them again in one call. Unmap forgets
the range. Direct memory is mapped from shared backings over its
reservation and a fixed remap replaces the mapping, so both register the
new mapping afresh. The watch is per mapping: a write through one mapping
of shared physical pages is not seen at another, so physical pages mapped
at more than one address leave the watch in every mapping of them.

A registered mapping reads as written until a collect covers it. The watch
keeps that itself instead of relying on how the kernel classifies pages
never faulted in: Linux 6.17 reports a range without page tables as not
written and leaves it unprotected, 7.2 reports it as written. The first
collect of a fresh range reports it written and write-protects it with
UFFDIO_WRITEPROTECT, which puts markers on its unpopulated pages
(UFFD_FEATURE_WP_UNPOPULATED), so from then on only real writes report.

Fallback: the userfaultfd is opened with UFFD_USER_MODE_ONLY where
vm.unprivileged_userfaultfd restricts it, and a probe at startup checks the
mechanism on scratch pages (a populated and an unpopulated page written
after protection report exactly, untouched ones do not). Without it (kernel
older than 6.7, no userfaultfd permission, headers without PAGEMAP_SCAN, or
APS5_NO_WRITE_WATCH=1) the watch reports unavailable and the tracker stays
unwatched exactly as before: CollectWrites answers 0 and callers compare
bytes. A range with a page outside the registered mappings also answers 0,
and a protection or scan that fails reports its remaining pages as written
and answers 0.

The driver's tracker keeps its contract and generations. On Linux its
block stamps live in 4 GiB leaves over the user address space, made on
first stamp, instead of the arena's flat arrays; the collect memo now works
on both platforms. GuestMemory::WriteWatched and Watched replace the arena
queries in UnitShadowEnabled and ShadowDestinationFor, so unit shadows run
on Linux over watched memory and stay off everywhere else. The Windows
arena path is unchanged apart from going through the same stamp helpers.

Host imports: a driver may lift the write protection of imported pages
itself. amdgpu validates the userptr BO behind a VK_EXT_external_memory_host
import with a writable HMM fault on every submission after the pages were
protected again, so PAGEMAP_SCAN reports them as written with no CPU store,
and a resident display image read as written at every present (found by
Pacuka on an RX 6650 XT with RADV: the presented image went black). NVIDIA
pins the pages once at the import and stays exact from then on. A probe at
device setup decides per device: it maps and watches scratch pages, collects
them clean, imports them, collects, submits a GPU copy that reads the import,
waits, and collects again. Pages written after the submission mean imports
resolve write protection: every import from then on takes its range off the
watch (GuestMemory::Unwatch) before it is made, so Watched is false for it,
CollectWrites answers 0 and callers compare bytes; GPU writes to watched
memory keep their MarkWritten stamps. The result is logged once:
"[write-watch] host imports resolve write protection: yes/no". A probe that
cannot run takes the unwatch side. APS5_WRITE_WATCH_IMPORTS=watch|unwatch
overrides the probe (default probe; anything else throws), and
APS5_NO_WRITE_WATCH=1 still turns the watch off. On the RTX 4090 the probe
reads 0 of 4 pages written after the GPU read (4 after the import itself),
so imports stay watched there.

The import itself lifts the protection once where imports stay watched:
on the RTX 4090 the pinning at vkAllocateMemory reports every imported page
as written (4 of 4 in the probe), with no CPU store. Such a report would
read as a CPU write over any GPU results pending in the range and drop
them. So every import of a watched range is made inside an import window
(GuestMemory::ImportWatched), under the tracker lock: the range is
collected right before the import, so earlier CPU writes are stamped as
usual, and again right after it, and what the window reports is stamped
"maybe written". Readers without pending results see it as a change
(UnchangedSince fails, ChangedBlocks reports it), so caches re-upload or
compare. Decisions about pending GPU results read it as untracked:
ChangedBlocks reports BlockMaybeWritten apart from BlockWritten, and the
drop rule, the write-back's keep blocks, the alias checks, the unit
shadow's stale drop and the whole-image stale checks (through the new
WrittenSince, which also answers false for a range off the watch) store
the results instead of dropping them. A real CPU store racing an import
also races the pending GPU writes to the same memory, which has no
defined order on hardware either, so the GPU results winning is
acceptable. From the second collect on the range is protected again and
watched normally. A range the watch does not cover whole, or a collect of
the window that fails, is taken off the watch instead.

An unwatched import stays off the watch for the rest of its mapping: a
remap registers the range afresh, the import kept over it takes it off
again at the next reconcile, and a re-import after an unmap follows the
decision. Unwatch invalidates the collect memo, so a collect memoized
before the import answers 0. A range that leaves the watch while a storage
image holds a generation for it no longer reads as CPU-written:
ChangedBlocks now tells whether the range is watched, and the drop rule,
the write-back's keep blocks and the alias borrow treat an unwatched range
as untracked (stored, never dropped), as a generation of 0 is.

Tests: guest_memory checks the watch through libkernel mappings (a CPU
store, a read, a kernel write through read(), another thread, a memcpy
across pages, an mprotect cycle, a partial munmap and a fixed remap) and
over a fresh 2 MiB-aligned mapping without page tables, checks direct
memory (watched, an alias takes the shared pages off the watch in both
mappings, a fixed direct remap over flexible memory is watched afresh), and
names the failing line and the reported pages. The recorder tests allocate watched
memory on both platforms, so the UnchangedSince, key proof, close race and
unit shadow tests run on Linux, and the storage refresh test gains a
watched variant: results kept on the GPU reach a CPU reader, and a plain
CPU store into one unit with newer results pending is found by the watch
alone. The recorder tests now also run the real import probe, check the
decision with the result injected (unwatch: an import leaves the watch, a
memoized collect, a partly imported range and the unimported neighbour
answer as they should, MarkWritten stamps only watched memory, a remap and
a re-import after an unmap follow the decision; watch: the import stays
watched and stamped), and check that results pending when their memory
leaves the watch are stored rather than dropped. A further test imports the
memory of a storage image with results pending in watch mode: the
results survive the import window and reach guest memory, a cache
generation taken before the import sees a change, the window reads as
maybe written and not as a CPU store, and a CPU store after it reads as
one again. Unit shadows and the
watched storage refresh report themselves as not run when imports are
compared.
2026-10-01 00:08:08 +13:00
Michon 962fc34e5a fix(libkernel): unmap ranges that span several mappings
As on FreeBSD, a munmap range covering more than one registered allocation (or gaps between them)
is unmapped allocation by allocation instead of throwing "unmap crosses allocation boundaries".
GTA V unmaps such ranges in its Story Mode prologue.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-30 10:54:47 +02:00
STL1te 9cfdbff17c refactor(libc, kernel): drop the libc exports the title takes from its own libc.prx
The C11 thread layer (_Mtx_*, _Cnd_*, _Thrd_*, _Xtime_get_ticks, std::_Pad) and the libc additions
_FNan, _FDtest, _FExp, __udivti3, _Throw_Cpp_error, generic_category, localeconv, wcsncpy_s and the
aligned operator new came from the --skip-sce-module route, where our libc stood in for the title's.
With the title's sce_module loaded, its libc.prx exports every one of these NIDs and the relinker
lists the guest modules ahead of the host prx, so the imports never reach our copies. Checked both
by the NID tables (the executable imports them from libc, the guest libc.prx exports them) and by
making the four std::_Pad functions throw: the title boots, reaches the menu and loads story mode
without calling them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 10:56:03 +03:00
boykopovar f7c53a4481 fix(memory): restore write-watch tracking for shared mappings 2026-09-30 09:59:51 +03:00
boykopovar abb9c85266 fix(kernel): share direct memory mappings and track physical offsets 2026-09-30 09:59:51 +03:00
asimplestray c6d098d487 feat(kernel): implement Sleep, sigprocmask and pthread finalize 2026-09-29 23:40:48 +00:00
asimplestray 064006b617 feat(kernel): implement pthread semaphore lifecycle 2026-09-29 22:13:07 +00:00
STL1te 6b36c050fe style: drop the branch's code comments per CONVENTIONS.md
Only the lines this branch added lose their comments; the commits before
keep them, with the reasoning behind each change, for whoever digs into
that code later.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:25:29 +03:00
STL1te ee1d34230d feat(kernel): sceKernelDebugWriteCppExceptionInfo prints the uncaught C++ exception
A title libc from sce_module calls it from __cxa_throw/__cxa_rethrow when
no handler is found, right before std::terminate, with the thrown object,
0x400 (meaning unknown), the demangled type name and what() (or null).
The console keeps it for the core dump; there are no guest core dumps
here, so it is printed to stderr. It was the only import such a libc
needed beyond the existing libs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:22:23 +03:00
STL1te b7eaabbc1e feat(kernel): share direct memory mapped at two addresses read-only
A title maps physical pages at a new address while the GPU still reads them
through the old one. Every view of such a page gets the same contents and
loses write access while the page is shared; the last view left gets its
protection back. A CPU write to a shared page faults, and views that differ
when one is unmapped (a GPU write through an import) throw, instead of the
views silently diverging.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:22:17 +03:00
STL1te c206f7049a feat(kernel, libc): memory, files and threads for retail title boot
libkernel:
- direct memory keeps its contents per physical page across unmap and
  remap; a page mapped at two live addresses at once throws
- sceKernelDirectMemoryQuery answers from tracked allocations
- SCE protections read back as set, GPU and AMPR bits included
- APR prefix path resolve, file size and stat by id, positional reads,
  fsync/pread/pwrite; directories open as descriptors on Windows
- sceKernelCheckReachability, getpeername, shutdown
- Dinkumware C11 threads (_Mtx_*, _Cnd_*, _Thrd_*) in the title libc
  layout; any pthread stack size, rounded at thread creation

libc:
- unaddressed guest mappings go to the system-managed area from
  0x2_0000_0000, leaving the application area to fixed mappings
- the process starts in /app0
- guest sscanf on Windows; a null %s prints as (null) as on FreeBSD
- localeconv, _FNan, _FDtest, _FExp, __udivti3, wcsncpy_s, aligned
  operator new, _Throw_Cpp_error

Tests: direct memory following physical pages, guest formatting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:22:13 +03:00
boykopovar c0755b9ec8
Merge pull request #102 from asimplestray/feat/kernel-aio
feat(kernel): implement Aio request lifecycle
2026-09-29 11:07:10 +03:00
boykopovar d841b8420a
Merge pull request #97 from SamG-Coder/codex/kernel-notifications
feat(kernel): deliver plain-text notifications to host output
2026-09-29 10:51:52 +03:00
OverkillLabs 55ed2719d5 refactor: apply naming conventions and drop extern-C comment
Renames template parameters to the TKey-style the conventions doc
requires (TValue, TFunction), removes the extern "C" close comment
(not one of the two allowed structural exceptions), and replaces
the magic error/flag numbers left after the previous comment
cleanup with the existing SCE_KERNEL_ERROR_* constants and a named
RTLD_NOW constant so the code documents itself.
2026-09-29 04:20:28 +03:00
OverkillLabs d9f8d87c27 refactor: strip explanatory comments to match repo conventions
Comments in this codebase may only mark technical debt (per
docs/dev/CONVENTIONS.md), plus the #endif and namespace-close
structural exceptions. Removes the descriptive comments added on
this branch across PreciseWait.hpp, FormattingWide.cpp, the
direct-memory pool query/free-run plumbing, and the module loader
wiring, keeping the two allowed structural markers intact.
2026-09-29 04:18:04 +03:00
OverkillLabs 5387aef3b7 Merge remote-tracking branch 'origin/main' into feat/kernel-libc-fixes
# Conflicts:
#	core/libs/prx/libkernel/DirectMemory/DirectMemory.hpp
2026-09-29 04:13:58 +03:00
Dean Galvin 70bbd700e2 feat(libkernel): accept priority-inheritance mutexes
scePthreadMutexattrSetprotocol threw for SCE_PTHREAD_PRIO_INHERIT, so a
title creating its mutexes with priority inheritance stopped at the
first one.

Guest thread priorities are recorded (scePthreadSetprio/Getprio) but
never applied to host scheduling, so there is no priority for a waiter
to lend to the owner: an inheriting mutex locks, unlocks and reports
errors exactly like a PRIO_NONE one here, and the protocol is accepted.
The POSIX pthread_mutexattr_setprotocol wrapper already accepts it.
Priority protection (PRIO_PROTECT, with its ceilings) and unknown
protocols still throw.
2026-09-29 13:22:21 +13:00
SamG-Coder 5e708c1608 feat(kernel): add native pthread barriers and thread helpers 2026-09-29 07:39:01 +10:00
asimplestray c25c543f1c feat(kernel): implement Aio request lifecycle 2026-09-28 20:39:33 +00:00
SamG-Coder d70d7ca268 feat(kernel): deliver plain-text notification requests to host output 2026-09-29 05:14:56 +10:00
OverkillLabs cfe9a458cc kernel: direct-memory pool querying and module loader wiring
- DirectMemory: track each allocated block's type and extent in the
  physical pool (DirectMemoryBlock), splitting the record on a partial
  free, and expose DirectMemoryQueryBlock/DirectMemoryFreeRun. Wires
  sceKernelDirectMemoryQuery to answer with the real allocated range
  and type instead of a hardcoded one-page/type-3 guess, and
  sceKernelAvailableDirectMemorySize to report the actual free run
  instead of the whole remaining aperture. DirectMemory.cpp's mapping
  path already tolerates MAP_FIXED overlap into a registered range and
  already has honor-then-fallback address-hint handling and an
  ignored-flags mask, so it is unchanged here.
- MemoryPoolExport: implement sceKernelMemoryPoolBatch/Commit/Decommit/
  Expand/GetBlockStats/Reserve (previously all dead stubs) using the
  existing DoMprotect/DirectMemoryAlloc/DoReserveVirtual primitives and
  a small pool-state block counter.
- Module: wire sceKernelDlsym/sceKernelLoadStartModule/
  sceKernelStopUnloadModule to the existing host-module loader
  (dlopen_nid_postfix/dlsym_nid_postfix/dlclose_nid_postfix), replacing
  dead stubs. sceKernelGetModuleInfoForUnwind/FromAddr are unrelated
  and left as-is.
2026-09-28 13:18:25 +03:00
boykopovar 8bd030d531
Merge pull request #26 from QDarkRex/fix/kernel-error-codes
fix(kernel): use SCE kernel error codes in equeue, direct memory and mutex
| Constant | Before | After |
|---|---|---|
| `EQUEUE_ERROR_ETIMEDOUT` | `0x80010023` | `0x8002003C` |
| `EQUEUE_ERROR_EBADF` | `0x80010016` | `0x80020009` |
| `EQUEUE_ERROR_ENOENT` | `0x80010011` | `0x80020002` |
| `EQUEUE_ERROR_EINVAL` / `SCE_KERNEL_ERROR_EINVAL` (DirectMemory) | `0x80010005` | `0x80020016` |
| `EQUEUE_ERROR_EFAULT` / `SCE_KERNEL_ERROR_EFAULT` (DirectMemory) | `0x80010009` | `0x8002000E` |
| `SCE_KERNEL_ERROR_EAGAIN` (DirectMemory) | `0x80010002` | `0x80020023` |
| `SCE_KERNEL_ERROR_ENOMEM` (DirectMemory) | `0x80010007` | `0x8002000C` |
| `SCE_KERNEL_ERROR_EACCES` (DirectMemory) | `0x80010004` | `0x8002000D` |
| `SCE_KERNEL_ERROR_EDEADLK` (Mutex.cpp) | `0x80020023` (= EAGAIN) | `0x8002000B` |
2026-09-28 11:18:14 +03:00
Claude 0b560c6af1
Merge fix/audit-2026-09 into refactor/kernel-error-header
Bring in the Linux scePthreadExit fix from #25.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018emyZ4Rk8yoSJvxTFBe4DL
2026-09-28 07:47:20 +00:00
Claude a60256ea4d
fix(kernel): exit POSIX guest threads without forced unwinding
On Linux scePthreadExit called pthread_exit, whose forced unwind runs
the personality routine of every guest frame. Those resolve to
libc.prx's __gxx_personality_v0, which cannot read libgcc's unwind
context and aborts, so guest_thread_self crashed on Linux.

Record a jump target in the thread start routine and longjmp back to it
from scePthreadExit instead. Like _endthreadex on Windows, this skips
the guest frames without unwinding them; the start routine still
releases the thread normally. Threads not created by scePthreadCreate
keep the pthread_exit fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018emyZ4Rk8yoSJvxTFBe4DL
2026-09-28 07:47:12 +00:00
Claude 2aced8e8d9
Merge fix/audit-2026-09 (with upstream main) into refactor/kernel-error-header
Bring in the resolved #25 branch, which already carries upstream main
including boykopovar/AnyPS5#10, and re-apply the KernelErrors.hpp
refactor onto the rewritten files:

- Equeue: drop the local EQUEUE_ERROR_* constants and use the shared
  SCE_KERNEL_ERROR_* names.
- Pthread Attr/Thread: drop local error constants and include
  KernelErrors.hpp.
- Pthread Cond/Mutex: take upstream; they no longer define local
  SCE_KERNEL_ERROR_* constants.
- Semaphore: drop KERNEL_SEMA_ERROR_* in favour of KernelErrors.hpp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018emyZ4Rk8yoSJvxTFBe4DL
2026-09-28 07:40:42 +00:00
Claude a28cfb7585
Merge upstream main into fix/kernel-error-codes
Resolve the conflict with boykopovar/AnyPS5#10 in Pthread/src/Mutex.cpp
by taking upstream: its rewrite already returns EDEADLK as 0x8002000B.
The equeue and direct memory error code fixes are unaffected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018emyZ4Rk8yoSJvxTFBe4DL
2026-09-28 07:36:16 +00:00
Claude 210987745c
Merge upstream main into fix/audit-2026-09
Resolve conflicts with boykopovar/AnyPS5#10 (sce module loading) and
later upstream changes.

- Pthread Cond/Mutex: take upstream. It already releases mutex
  ownership across condition waits, returns ETIMEDOUT 0x8002003C, and
  rejects unlocks from non-owners for every mutex type.
- Pthread Thread: keep upstream's lifecycle, Windows adoption and job
  affinity; re-apply the POSIX fixes on top: reference-counted
  release on join/detach/exit, self-join rejection on both platforms,
  and scePthreadSelf adoption for threads not created by
  scePthreadCreate.
- Semaphore: keep upstream's TimedWait::Condition and wait tracing;
  re-apply sceKernelDeleteSema waking waiters with EACCES.
- Relinker: keep --windows-gui alongside upstream's --skip-sce-module,
  trampoline-aware Patch signature and new Python tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018emyZ4Rk8yoSJvxTFBe4DL
2026-09-28 07:34:25 +00:00
boykopovar 73d014ed83
Merge pull request #56 from oneandonlydean/feat/linux-pread-getdents
feat(libkernel): pread, pwrite, getdents and getdirentries on linux
- **pread/pwrite:** use the existing `NativePread`/`NativePwrite` and don't move the file offset. A null buffer with a non-zero size returns `EFAULT`, a negative offset `EINVAL`, and host failures go through `SceErrorFromErrno`.
- **getdirentries:**
  - reads host `getdents64` records and converts them to the FreeBSD `struct dirent` layout the guest expects (`d_fileno` u32, `d_reclen` u16, `d_type` u8, `d_namlen` u8, NUL-terminated name, record padded to 4 bytes);
  - `*basep` gets the directory position from before the read;
  - when a record doesn't fit in the guest buffer, the descriptor is moved with `lseek` back to that entry's cookie, so the next call resumes there;
  - if not even the first record fits, the call returns `EINVAL` without consuming anything, and a name longer than 255 bytes returns `ENAMETOOLONG`.
- **getdents** is `getdirentries` without `basep`.
- Adds the guest `EFAULT`, `EINVAL` and `ENAMETOOLONG` constants.
2026-09-28 09:38:47 +03:00
Dean Galvin f7e1bdf0b9 feat(libkernel): pread, pwrite, getdents and getdirentries on linux
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ButHENCbXXt8h4EZuWUabJ
2026-09-28 16:40:22 +13:00
Dean Galvin 9754937a5d feat(libkernel): hinted and no-overwrite mappings, virtual range names and usleep
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ButHENCbXXt8h4EZuWUabJ
2026-09-28 14:22:48 +13:00
Dean Galvin 93a214836f fix(libkernel): build sceKernelVirtualQuery on linux
The host-region fallback added with the Demon's Souls work calls VirtualQuery and
includes windows.h unconditionally, so libkernel no longer compiles on Linux.
On Linux, read the containing mapping and its permissions from /proc/self/maps.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ButHENCbXXt8h4EZuWUabJ
2026-09-28 13:32:36 +13:00
devilismyfriend 72336201b2 perf(libkernel): resolve initialised mutexes and conditions without the global lock
Every lock, unlock, signal and wait took the process-wide initialisation
mutex to read the handle, serialising all guest synchronisation. The
handle is now read atomically and the lock is only taken to initialise a
static initializer or to report a destroyed object.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177gthrWEBBfBu5xJDXgBfo
2026-09-27 11:26:59 -07:00
devilismyfriend e764385d51 fix(libkernel): drop the scePthreadOnce stub that shadowed the implementation
The merge brought a NotImplemented scePthreadOnce_nid_postfix next to the
existing scePthreadOnce. The NID patcher gives the NID to the postfixed
symbol, so the stub replaced the working export and the title aborted at
its first once-initialisation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177gthrWEBBfBu5xJDXgBfo
2026-09-27 11:26:58 -07:00
devilismyfriend a503d0b617 Merge remote-tracking branch 'origin/main' into feat/demons-souls-boot
Brings in main up to 9958d26 (homebrew runtime, POSIX wrappers in
libkernel, sce_module guest modules and --skip-sce-module, the --to-intel
general pipeline, libSceRandom, libSceRtc, libScePosixForWebKit,
libRenoirCore, test wiring).

Resolutions where both sides implemented the same thing:
- libkernel pthread: main's structure (lazy static initialisation,
  destroyed markers, MutexOperations/CondOperations, mutex protocol and
  condattr clock) with this branch's TimedWait deadlines and condition
  (precise timed lock/wait on Windows), wait tracing, the adaptive mutex
  type 4, POSIX default ERRORCHECK mutexes, EDEADLK/EPERM results instead
  of throws, and condition clocks honoured for absolute timeouts.
- libc locale: the guest Dinkumware ABI (_Locimp, facet vtables, short
  ctype/case tables, locale ids, stream storage) is kept because guest code
  inlines those layouts; main's C character functions are kept on top.
- libc stdio/strings/math: main's stream functions (guest FILE status
  sync), strerror/perror, remove, formatting, string and math exports win
  over this branch's duplicates.
- libc paths: main's working directory with this branch's path aliases
  resolved on the normalised guest path.
- libkernel file: this branch's sceKernel* results as SCE errors, main's
  POSIX wrappers and both fstat helpers.
- relinker: this branch's stub lowering keeps its trampolines through the
  main executable path; guest modules refuse lowerings that need stubs.
- AGC driver: main's rendering wait is executed by this branch's workers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177gthrWEBBfBu5xJDXgBfo
2026-09-27 10:52:21 -07:00
boykopovar 9cc36a02fe feat(libkernel): add __progname str value 2026-09-27 11:43:40 +03:00
boykopovar f5ea673c10 feat(libkernel): impl posix stdio wrappers, reusing sceKernel file ops 2026-09-27 11:43:40 +03:00
boykopovar 7a925c68e5 fix(libkernel): handle clock_second 13 in clock_gettime and clock_getres 2026-09-27 11:43:39 +03:00
boykopovar 13c648bebb feat(libkernel): impl file, time, process, signal for posix 2026-09-27 11:43:38 +03:00
boykopovar 62c16792ff feat(libkernel): impl pthread condition variables + posix thread attribute queries 2026-09-27 11:43:37 +03:00
boykopovar a709e220f2 feat(libkernel): implement posix mutex locking and timed waits 2026-09-27 11:43:37 +03:00
boykopovar c9cb3cae39 feat(libkernel): implement disabled address sanitizer state + replacement tables 2026-09-27 11:43:36 +03:00
boykopovar 9fd6816356 fix(libkernel): store rtld allocator table separately from application heap 2026-09-27 11:43:35 +03:00
boykopovar dbb56adcf6 feat(libkernel): implement process, cpu, uuid, gpi, gpo, sync funcs 2026-09-27 11:43:34 +03:00
boykopovar 55e1c4a238 feat(libkernel): implement application heap and thread atexit registration 2026-09-27 11:43:33 +03:00
boykopovar 5887c8d6e2 feat(libkernel): add memory segment, backtrace, heap error report definitions 2026-09-27 11:43:32 +03:00
boykopovar 40101471ee feat(libkernel): add new posix, pthread, semaphore, process funcs definitions 2026-09-27 11:43:31 +03:00
boykopovar 0783e369da feat(libkernel): add definitions for standard guest libc imports 2026-09-27 11:43:31 +03:00
QDarkRex 2081071237 refactor(kernel): share SCE kernel error codes through KernelErrors.hpp
Kernel error codes were redeclared per file under three prefixes
(SCE_KERNEL_ERROR_, EQUEUE_ERROR_, KERNEL_SEMA_ERROR_), which let wrong
values go unnoticed. Define them once as 0x80020000 | errno for errno
1..94 and use the SCE_KERNEL_ERROR_ names everywhere. Values are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 13:32:28 +07:00
QDarkRex ac0f8af228 fix(kernel): use SCE kernel error codes in equeue, direct memory and mutex
SCE kernel errors are 0x80020000 plus the FreeBSD errno. The equeue and
direct memory constants used an unrelated 0x8001xxxx range with mismatched
errno values (e.g. ETIMEDOUT was 0x80010023), so a guest checking
sceKernelWaitEqueue for SCE_KERNEL_ERROR_ETIMEDOUT saw an unexpected error.
SCE_KERNEL_ERROR_EDEADLK in Mutex.cpp held the EAGAIN value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 13:27:23 +07:00
QDarkRex 97cae1458e fix(kernel): use SCE kernel error codes in equeue, direct memory and mutex
SCE kernel errors are 0x80020000 plus the FreeBSD errno. The equeue and
direct memory constants used an unrelated 0x8001xxxx range with mismatched
errno values (e.g. ETIMEDOUT was 0x80010023), so a guest checking
sceKernelWaitEqueue for SCE_KERNEL_ERROR_ETIMEDOUT saw an unexpected error.
SCE_KERNEL_ERROR_EDEADLK in Mutex.cpp held the EAGAIN value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 13:22:01 +07:00
QDarkRex be127fd0f1 feat(kernel): implement sceKernelDeleteSema
Waiters are woken with SCE_KERNEL_ERROR_EACCES and the semaphore is freed
once they have left.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 12:38:59 +07:00
QDarkRex 15b749c2ba fix(kernel): return 64-bit offsets from sceKernelLseek
The SCE ABI returns off_t; offsets above INT_MAX threw instead of being returned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 12:38:59 +07:00
QDarkRex 76b7f998f5 fix(kernel): give every thread a pthread handle and keep mutex ownership consistent
- scePthreadSelf returned null on Linux and for threads not created by
  scePthreadCreate (including the main thread); such threads now get a
  lazily adopted, non-joinable handle
- scePthreadExit on Linux skipped FinishThread, losing the join value
- detached threads on Linux were never freed; use the reference count
  already used on Windows
- unlocking a recursive mutex from a non-owner was undefined behaviour;
  return SCE_KERNEL_ERROR_EPERM
- scePthreadCondTimedwait did not restore the mutex owner after waking,
  so another thread locking during the wait made the waiter's unlock
  fail with EPERM and left the mutex locked forever
- SCE_KERNEL_ERROR_ETIMEDOUT in Cond.cpp was 0x80020062 instead of 0x8002003C

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 12:38:59 +07:00
devilismyfriend af3c2024ed feat(runtime): timed waits, sync primitive fixes, heap and RTC fixes, unattended-run switches
libkernel: a TimedWait service for condition variables, semaphores, event flags, rwlocks and
equeues with the scheduler tick; mutex/cond/sem corner cases the title hits; job worker
affinity is opt-in (APS5_JOB_AFFINITY). libc: calloc/realloc heap slot swap, a default
terminate handler with a backtrace, CPU topology helper. libSceRtc export fixes with tests.
libSceVideoOut: APS5_NO_PAD_INPUT ignores host input for unattended measurements,
APS5_FLIP_QUEUE_WAIT keeps a stalled process alive for a debugger. ApplicationHeap and
GuestRtc tests wired into the test build.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177gthrWEBBfBu5xJDXgBfo
2026-09-26 19:00:39 -07:00
Pacuka b98257b75d fix(libc): register the main guest image on Linux
Guest mprotect calls on the executable's own segments failed with
"guest protection or unmap range is not registered" because only the
Windows build registered the main image. Register the PT_LOAD segments of
the main executable page by page, merging the protections of segments that
share a page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 11:15:16 +02:00
Pacuka ca23f10612 fix(kernel): accept AMPR and ACP access bits in memory protections
Retail titles pass protections such as 0xC2 (CPU read/write plus the
0x40 and 0x80 device access bits). The device bits do not affect CPU
access and were rejected as unsupported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 11:15:16 +02:00
Pacuka d15e93681f fix(kernel): map SCE protections to readable host pages
SCE_KERNEL_PROT_CPU_RW (0x02) and CPU_EXEC (0x04) include read access,
but were translated to write-only and execute-only host pages. GPU read
and write bits also need host access because the emulated GPU reads and
writes guest memory from the CPU. Write-only mappings made the AGC driver
reject guest data segments as unreadable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 11:15:16 +02:00
boykopovar e4825fbdc2
Merge pull request #3 from SamG-Coder/main
Expand native PS5 homebrew compatibility and regression coverage
2026-09-26 09:54:21 +03:00
devilismyfriend f3ac61000d feat(runtime): guest arena, kernel thread/sync/file APIs, crash hooks
- libc: reserve guest memory in one arena below the PS5 map limit with
  MEM_WRITE_WATCH (GuestArena) and a caching guest heap; the allocation
  registry gains a generation counter, an invalidator callback for the
  driver's page cache and a pin waiter for leased ranges. This replaces
  the fault-based GuestMemoryTracking/GuestMemoryBacking layer, whose
  files are removed.
- libkernel: mmap/mprotect/munmap on the arena, flexible memory budget,
  real sceKernelVirtualQuery; POSIX pthread layer, rwlocks, timed locks,
  TSD, event flags; 0.5 ms scheduler tick and high-resolution sleeps;
  TSC/clock/time functions; file APIs return SCE errno codes; the APR
  file-streaming API.
- libc: vectored crash reporter with backtraces for fatal exceptions,
  SIGABRT, UCRT invalid parameters and process exit; SSE4a EXTRQ/INSERTQ
  emulation for Intel hosts; %ls/%lc/%S/%C with the guest's 16-bit
  wchar_t; hand-written setjmp/longjmp for the guest jmp_buf; a path
  alias table for short save-data mount points; more string, stdio and
  math exports.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177gthrWEBBfBu5xJDXgBfo
2026-09-25 16:27:49 -07:00
boykopovar 40df5282e6 perf(agc): track guest memory ownership + optimize gpu synchronization 2026-09-25 19:47:18 +03:00
SamG-Coder 322dc24c9e feat(kernel): implement guest once initialization 2026-09-25 07:22:05 +10:00
SamG-Coder 453fec28c5 feat(kernel): add tracked dynamic module loading and NID lookup 2026-09-25 06:48:50 +10:00
SamG-Coder b8c1097db4 feat(kernel): support socket nonblocking flags through fcntl 2026-09-25 06:34:30 +10:00
SamG-Coder 56c7a589db feat(kernel): bridge guest signal registration and delivery 2026-09-25 06:27:11 +10:00
SamG-Coder cbd22dd854 feat(kernel): implement guest system configuration queries 2026-09-25 06:25:16 +10:00
SamG-Coder 9cbefb7e04 feat(kernel): implement tracked UDP sockets and ioctl 2026-09-25 06:23:44 +10:00
SamG-Coder ff725218cb feat(kernel): translate guest IP address conversion APIs 2026-09-25 06:18:32 +10:00
SamG-Coder 1bbdf7daa6 feat(kernel): support private anonymous guest memory mappings 2026-09-25 06:06:31 +10:00
boykopovar 9f2b41b9a1 fix(libc): route guest allocations through application heap 2026-09-24 06:53:12 +03:00
boykopovar 630d918b4a feat(agc): add timing logs for path after recompling 2026-09-23 22:11:41 +03:00
boykopovar 10e4d5e469 perf(shader): make -ools validation and optimization optional 2026-09-23 21:48:51 +03:00
boykopovar 762b91df59 feat(libkernel): app metadata param.json parsing impl 2026-09-23 15:18:48 +03:00
boykopovar cf1953058b feat(libkernel): add app metadata reading module 2026-09-23 15:09:31 +03:00
boykopovar 2df58cf09c fix(kernel): support page-aligned mprotect for loaded images 2026-09-23 12:45:39 +03:00
boykopovar 3028c87491 feat(agc): add bda memory mapping and resource management (wip) 2026-09-21 18:54:06 +03:00
boykopovar 3da76b9b33 refactor(graphics): add new graphics pipeline log + remove redunant ones 2026-09-18 21:03:11 +03:00
boykopovar d084eed394 feat(shader): rdna memory op decoder impl 2026-09-15 07:07:50 +03:00
boykopovar bd0969c3a4 fix(libs): ex from libc prx: unwind for catch fix (mingw) 2026-09-13 19:21:31 +03:00
boykopovar 5d715ee549 refactor(libkernel): move from Export.cpp to separated files 2026-09-13 16:41:04 +03:00
boykopovar 8f2bca5481 refactor(libkernel): merged dup pthread functions from _sys and _web 2026-09-13 16:24:42 +03:00
boykopovar 7c44d13b3a feat(libs): global shared path resolver in libc 2026-09-13 12:45:50 +03:00
boykopovar f66a8fc5c4 feat(libkernel): libkernel file functions impl 2026-09-13 12:07:47 +03:00
boykopovar f19a234f38 feat(libkernel): add gettime getres gettimeofday impl 2026-09-13 11:21:37 +03:00
boykopovar 94ab47a417 feat(libkernel): some semaphore functions impl 2026-09-12 19:27:53 +03:00
boykopovar e38f76a823 refactor(libkernel): move some functions from export.cpp to new semaphore and file 2026-09-12 19:05:39 +03:00
boykopovar a7521f3a7e feat(libkernel): libkernel time functions impl 2026-09-12 16:33:29 +03:00
boykopovar 37c8bafd13 fix(libkernel): fix export non-ps5 functions with extern c 2026-09-12 16:07:14 +03:00
boykopovar 31943e9a62 fix(sceVideoOut): add nid_postdfix to equeue libkernel exports 2026-09-12 11:21:01 +03:00
boykopovar 497821e94f feat(sceVideoOut): video out functions impl (wip) 2026-09-12 11:02:17 +03:00
boykopovar 69b8080ea4 feat(libkernel): add equeue functions impl 2026-09-12 10:40:27 +03:00
boykopovar e9bbde1a6e refactor(libkernel): pthread functions separation 2026-09-11 14:12:44 +03:00
boykopovar 999a723a5f fix(windows): precommit guest thread stacks to prevent guard page skips 2026-09-10 20:25:25 +03:00
boykopovar b765cefa42 fix(libs): fix args sysv abi for types (windows) 2026-09-10 19:54:03 +03:00
boykopovar 36c29c9a66 feat(libs): sysv abi macros for windows exports 2026-09-10 13:30:05 +03:00
boykopovar abc72da741 feat(libs): define vabi macros + add to declarations 2026-09-10 08:07:03 +03:00
boykopovar 7cf8576587 feat(build): all libs link libc + libkernel 2026-09-08 09:35:28 +03:00
boykopovar 6e8f04a65a feat(libkernel): cross platform libkernel build 2026-09-08 09:17:39 +03:00
boykopovar 07d1463ae1 fix(libkernel): replace throwing with return err code for invalid len 2026-09-08 00:16:56 +03:00
boykopovar ce729fb9c7 fix(libkernel): fix random direct memory bug 2026-09-07 20:39:54 +03:00
boykopovar 6c4d6c18b2 fix(libkernel): fix scePthreadCreate sync 2026-09-07 19:13:33 +03:00
boykopovar bc14e4743c fix(relinker): map zero-flag ps5 load segments as rwx 2026-09-07 10:10:00 +03:00
boykopovar 9d4c87b686 fix(nid): fix updates detections for nid_patcher rerun 2026-09-07 09:55:41 +03:00
boykopovar 975e4a72d4 refactor(libkernel): refactir direct memory exports organization 2026-09-07 09:08:59 +03:00
boykopovar e77d724a1b feat(libkernel): add pthread module impl(wip) 2026-09-07 08:50:31 +03:00
boykopovar e79f5ebc58 refactor(libkernel): separate direct memory impl 2026-09-07 08:08:31 +03:00
boykopovar 6ecfc35551 feat(libkernel): libkernel direct memory impl 2026-09-06 16:03:29 +03:00
boykopovar 6123893998 feat(libs): throw not impl ex in every stub 2026-09-06 15:05:04 +03:00
boykopovar 55a120e318 fix(nid): detect linker suffix-merged symbol names postfixes 2026-08-21 14:45:07 +03:00
boykopovar 8627076df5 fix(libs): drop std name collisions via nid postfix + remove custom entry point 2026-08-21 00:26:40 +03:00
boykopovar 47099897ac fix(libs): fix libscelibcinternal crt symbol conflict, stray macro in pthread_once, missing typedefs, unnamed params 2026-08-20 23:58:06 +03:00
boykopovar 59bee8d187 feat(libs): add libs function signatures 2026-08-20 23:18:16 +03:00