FFmpeg's configure runs in the found sh and writes MSYS paths into its
Makefile, which a native MinGW make cannot open (Windows CI: "No rule to
make target '/d/a/.../3rdparty/FFmpeg/Makefile'"). Windows builds keep
the black-picture fallback unless ANYPS5_ENABLE_FFMPEG=ON is set in an
MSYS2 environment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add FFmpeg n9.0.2 as the 3rdparty/FFmpeg submodule (shallow) and build
it with ExternalProject_Add through its own configure and make, instead
of using the system libraries from pkg-config.
Only avcodec, avutil and swscale are built, with the H.264 decoder as
the single component (--disable-everything, --disable-autodetect), as
static PIC archives linked into libSceVideodec2.prx with
--exclude-libs,ALL. Without nasm the x86 assembly is disabled.
ANYPS5_ENABLE_FFMPEG=OFF, or a missing sh/make, keeps the black-picture
fallback.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds libSceVideodec2. Access units (Annex B, or length-prefixed rewritten to
Annex B) are decoded by FFmpeg's H.264 decoder into the title's NV12 frame
buffers (256-byte pitch, chroma after the visible luma rows). Decode hands
out at most one picture per call, Flush drains the reordered ones, Reset
clears the decoder; timestamps and attached data travel with the picture.
FFmpeg is found through pkg-config and is optional: without it the decoders
return black pictures. guest_videodec2_tests checks the pictures of an
embedded stream against reference hashes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lower INSERTQ xmm1, xmm2 (F2 0F 79) out of line with SSE2 only instead of
failing the --to-intel relink. Length and index are read from bits 69:64
and 77:72 of the source; dst[127:64] is zeroed as on hardware.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A title can open AudioOut2 ports of type 0x3 (the controller's speaker)
and 0x6 (its two voice-coil actuators) next to its bed and object ports.
Both were folded into the TV mix, so the haptics came out of the PC
speakers as low thumps and the controller got nothing.
A USB DualSense is also a 4-channel 48 kHz sound card: channels 1-2 feed
the speaker and headset jack, channels 3-4 the left and right actuators.
When SDL lists such a card, a context with these ports opens it and
queues a grain on it with every push: the speaker port on channels 1-2
(a mono port duplicated) and the vibration port's left and right
channels on 3-4, at the port volumes and clamped. Both ports then leave
the main mix. The card's queue gets the main queue's silence cushion when
it runs dry and drops grains that run more than two grains beyond the
main queue, so the controller keeps pace with the TV. An unplugged card
is released (its ports return to the main mix) and looked for again
every 2 s. Without a card nothing changes, and other port types and
channel counts keep their routing.
SDL2's PulseAudio backend labels a 4-channel stream FL FR FC LFE instead
of quad, and the sound server then folds channels 3-4 into the speaker.
Under that backend the card is opened as 5.1 with FC and LFE silent and
the actuators on the back pair, which lands on the card's RL/RR.
The storage image cache keys a surface by memory, extent, layout and
storage format, and the image keeps the DCC key range of the first
descriptor that made it. When a later descriptor names other metadata
(the title reallocated the keys, or the memory held another surface
before), the image kept the old keys: fast clears of the new keys never
reached it, and its write-backs marked keys the surface no longer owns.
A descriptor whose DCC address differs from the cached image's now evicts
that image (its pending results are flushed first) and a new one is made
from memory under the named keys. A sampled view of another image's
pending results is taken only under the same keys. A descriptor without
metadata keeps whatever image the surface has.
The bundled SDL was configured with SDL_HIDAPI OFF, so a DualSense went
through the generic joystick driver of the platform: no touchpad, no
light bar and no DualSense effect reports, which left the light bar,
trigger effect and (on Linux) rumble requests that PadInput already
sends through SDL without effect on the controller. With HIDAPI, SDL's
PS5 driver handles the pad: touchpad, light bar, rumble, trigger effects
and the motion sensors.
A USB pad always uses the driver's enhanced reports.
SDL_HINT_JOYSTICK_HIDAPI_PS5_RUMBLE enables them for a Bluetooth pad as
well, which otherwise stays on the simple report without effects.
pad_sdl checks that the bundled SDL is built with the HIDAPI joystick
driver.
A mesh-stage program reads its draw parameters from the end of the
pipeline's push block (MeshDrawPushOffsetBytes), so its SPIR-V always
declares the push-constant variable. When its user data do not fit the
push range it has no push data of its own and
RecompileResult::pushConstants is empty, and validation refused the draw
with "invalid push constant interface".
Validation now expects the push block of every mesh-stage program and
still requires it to match the recompiler's metadata for the other
stages. agc_driver_mesh_tests draws the triangle list with a geometry
program given no push range for its own data.
The GE adds GE_INDX_OFFSET to every index it fetches. The driver applied
it to auto draws only, so DRAW_INDEX_2 and DRAW_INDEX_OFFSET_2 fetched
from vertex 0 whatever the register held. Both packets now take it as
their base vertex, and like an auto draw they refuse a queue state without
the register (its reset value, 0, is part of the initial state).
GuestMemory::CollectWrites was built on the Windows arena's MEM_WRITE_WATCH
and answered 0 on Linux, so every cached copy of guest memory (textures,
resident render targets, buffers, DCC key proofs) read as changed at every
check: resident targets were stored and re-uploaded at every refresh and
unit shadows stayed off. Astro Bot's intro ran at ~3 fps.
libc gains GuestWriteWatch, the Linux counterpart of GetWriteWatch. Every
guest mapping libkernel makes (MapAligned: fixed, hinted and aligned) is
registered with one userfaultfd in asynchronous write-protect mode
(UFFD_FEATURE_WP_ASYNC: the kernel lifts a page's protection on its first
write without a handler thread, for user and kernel-mode writes alike), and
PAGEMAP_SCAN with PM_SCAN_WP_MATCHING reports the pages written since the
last scan of a range and protects them again in one call. Unmap forgets
the range. Direct memory is mapped from shared backings over its
reservation and a fixed remap replaces the mapping, so both register the
new mapping afresh. The watch is per mapping: a write through one mapping
of shared physical pages is not seen at another, so physical pages mapped
at more than one address leave the watch in every mapping of them.
A registered mapping reads as written until a collect covers it. The watch
keeps that itself instead of relying on how the kernel classifies pages
never faulted in: Linux 6.17 reports a range without page tables as not
written and leaves it unprotected, 7.2 reports it as written. The first
collect of a fresh range reports it written and write-protects it with
UFFDIO_WRITEPROTECT, which puts markers on its unpopulated pages
(UFFD_FEATURE_WP_UNPOPULATED), so from then on only real writes report.
Fallback: the userfaultfd is opened with UFFD_USER_MODE_ONLY where
vm.unprivileged_userfaultfd restricts it, and a probe at startup checks the
mechanism on scratch pages (a populated and an unpopulated page written
after protection report exactly, untouched ones do not). Without it (kernel
older than 6.7, no userfaultfd permission, headers without PAGEMAP_SCAN, or
APS5_NO_WRITE_WATCH=1) the watch reports unavailable and the tracker stays
unwatched exactly as before: CollectWrites answers 0 and callers compare
bytes. A range with a page outside the registered mappings also answers 0,
and a protection or scan that fails reports its remaining pages as written
and answers 0.
The driver's tracker keeps its contract and generations. On Linux its
block stamps live in 4 GiB leaves over the user address space, made on
first stamp, instead of the arena's flat arrays; the collect memo now works
on both platforms. GuestMemory::WriteWatched and Watched replace the arena
queries in UnitShadowEnabled and ShadowDestinationFor, so unit shadows run
on Linux over watched memory and stay off everywhere else. The Windows
arena path is unchanged apart from going through the same stamp helpers.
Host imports: a driver may lift the write protection of imported pages
itself. amdgpu validates the userptr BO behind a VK_EXT_external_memory_host
import with a writable HMM fault on every submission after the pages were
protected again, so PAGEMAP_SCAN reports them as written with no CPU store,
and a resident display image read as written at every present (found by
Pacuka on an RX 6650 XT with RADV: the presented image went black). NVIDIA
pins the pages once at the import and stays exact from then on. A probe at
device setup decides per device: it maps and watches scratch pages, collects
them clean, imports them, collects, submits a GPU copy that reads the import,
waits, and collects again. Pages written after the submission mean imports
resolve write protection: every import from then on takes its range off the
watch (GuestMemory::Unwatch) before it is made, so Watched is false for it,
CollectWrites answers 0 and callers compare bytes; GPU writes to watched
memory keep their MarkWritten stamps. The result is logged once:
"[write-watch] host imports resolve write protection: yes/no". A probe that
cannot run takes the unwatch side. APS5_WRITE_WATCH_IMPORTS=watch|unwatch
overrides the probe (default probe; anything else throws), and
APS5_NO_WRITE_WATCH=1 still turns the watch off. On the RTX 4090 the probe
reads 0 of 4 pages written after the GPU read (4 after the import itself),
so imports stay watched there.
The import itself lifts the protection once where imports stay watched:
on the RTX 4090 the pinning at vkAllocateMemory reports every imported page
as written (4 of 4 in the probe), with no CPU store. Such a report would
read as a CPU write over any GPU results pending in the range and drop
them. So every import of a watched range is made inside an import window
(GuestMemory::ImportWatched), under the tracker lock: the range is
collected right before the import, so earlier CPU writes are stamped as
usual, and again right after it, and what the window reports is stamped
"maybe written". Readers without pending results see it as a change
(UnchangedSince fails, ChangedBlocks reports it), so caches re-upload or
compare. Decisions about pending GPU results read it as untracked:
ChangedBlocks reports BlockMaybeWritten apart from BlockWritten, and the
drop rule, the write-back's keep blocks, the alias checks, the unit
shadow's stale drop and the whole-image stale checks (through the new
WrittenSince, which also answers false for a range off the watch) store
the results instead of dropping them. A real CPU store racing an import
also races the pending GPU writes to the same memory, which has no
defined order on hardware either, so the GPU results winning is
acceptable. From the second collect on the range is protected again and
watched normally. A range the watch does not cover whole, or a collect of
the window that fails, is taken off the watch instead.
An unwatched import stays off the watch for the rest of its mapping: a
remap registers the range afresh, the import kept over it takes it off
again at the next reconcile, and a re-import after an unmap follows the
decision. Unwatch invalidates the collect memo, so a collect memoized
before the import answers 0. A range that leaves the watch while a storage
image holds a generation for it no longer reads as CPU-written:
ChangedBlocks now tells whether the range is watched, and the drop rule,
the write-back's keep blocks and the alias borrow treat an unwatched range
as untracked (stored, never dropped), as a generation of 0 is.
Tests: guest_memory checks the watch through libkernel mappings (a CPU
store, a read, a kernel write through read(), another thread, a memcpy
across pages, an mprotect cycle, a partial munmap and a fixed remap) and
over a fresh 2 MiB-aligned mapping without page tables, checks direct
memory (watched, an alias takes the shared pages off the watch in both
mappings, a fixed direct remap over flexible memory is watched afresh), and
names the failing line and the reported pages. The recorder tests allocate watched
memory on both platforms, so the UnchangedSince, key proof, close race and
unit shadow tests run on Linux, and the storage refresh test gains a
watched variant: results kept on the GPU reach a CPU reader, and a plain
CPU store into one unit with newer results pending is found by the watch
alone. The recorder tests now also run the real import probe, check the
decision with the result injected (unwatch: an import leaves the watch, a
memoized collect, a partly imported range and the unimported neighbour
answer as they should, MarkWritten stamps only watched memory, a remap and
a re-import after an unmap follow the decision; watch: the import stays
watched and stamped), and check that results pending when their memory
leaves the watch are stored rather than dropped. A further test imports the
memory of a storage image with results pending in watch mode: the
results survive the import window and reach guest memory, a cache
generation taken before the import sees a change, the window reads as
maybe written and not as a CPU store, and a CPU store after it reads as
one again. Unit shadows and the
watched storage refresh report themselves as not run when imports are
compared.
The geometry path's merged ES/GS programs (VGT_SHADER_STAGES_EN.GS_EN)
were compiled as mesh-stage programs whose input metadata was never
filled in, so every such draw failed with "mesh shader translation
configuration is not supported".
Recompiler:
- BuildShaderStageInputInfo takes the draw's MeshConfiguration; the mesh
input helpers (primitive size, step and counts) are implemented.
- The prologue emulates one NGG subgroup per workgroup: s2 GS_TG_INFO,
s3 the merged wave info per wave, the GS vertex offsets (ES thread
index times VGT_ESGS_RING_ITEMSIZE, odd strip triangles swapped), the
primitive id, and the ES vertex and instance ids. An indexed draw's
vertex id is read from the index buffer through a raw V# the driver
puts in hidden user words 4..7.
- s_sendmsg GS_ALLOC_REQ becomes MeshAllocate. The epilogue zeroes the
allocation first, clamps it to the output limits, culls primitives no
thread exported and decodes the 9-bit vertex indices.
- The draw parameters live in the last six push dwords. A BDA fault in
a mesh program no longer stops the invocation (as for other programs
with barriers, the fault is recorded and the read returns zero), so
every invocation reaches the subgroup barriers and the epilogue.
- The cache keys cover the mesh configuration; requests serialize its
new esgsItemSize field (version 4).
Driver:
- DecodeShaderStages reads VGT_ESGS_RING_ITEMSIZE (now in the draw key)
and sizes the subgroup for its ES vertices, GS primitives and outputs.
- Stages compile below the mesh draw parameters, which every mesh draw
pushes; the mesh stage always takes part in the push range.
- The index-buffer V# is set per draw (per record on the CPU-resolved
indirect path); the index-buffer alias check spares read-only
elements.
- The workgroup and LocalSize checks count two guest lanes per
invocation on 32-wide subgroups; mesh draws may carry a first vertex
or instance.
agc_driver_mesh_tests (EXCLUDE_FROM_ALL, needs VK_EXT_mesh_shader)
recompiles a hand-assembled merged ES/GS program in the PS5 compiler's
layout and draws indexed and non-indexed triangle lists in one- and
two-wave subgroups and a triangle strip across odd-starting subgroups,
reading the pixels back. agc_shader_replay prints a request's draw and
mesh configuration.
Two paths lost a render target's results whenever CollectWrites answers 0
(no write watching: the Linux guest arena, or memory outside the Windows
arena), where every unit reads as changed at every check.
StorageTexture::Refresh dropped every pending result whenever the
surface's DCC keys read as a clear code, not only when the keys changed to
one. Draws never update the title's keys, so a resident target cleared
under a clear code keeps that code while holding newer draw results, and
the lookup of the next draw sampling the target re-cleared the image.
Astro Bot's intro composite sampled an empty UI layer on Linux. Only new
clear keys drop results now; results kept under the image's own clear code
are stored whole (the store marks every key uncompressed, as writeBack
does) and re-uploaded like any untracked unit.
A unit shadow slab is fresh only at a tracker generation and MarkShadowed
ignores generation 0, so a write-back retiled into a slab over unwatched
memory was never marked, published or read back: its texels were lost and
the next upload re-read the stale import. UnitShadowEnabled now also
requires a write-watched arena, and ShadowDestinationFor offers no slab for
a range outside the arena; those retiles go into the import, as with
APS5_NO_UNIT_SHADOW=1.
agc_driver_recorder_tests gains a GPU test over unwatched memory: a target
cleared under 0000 keys, drawn into, then refreshed keeps the draw's
texels and stores them to guest memory; no shadow slab is offered for it;
new 0001 keys still drop pending results.
Nothing built the graphics test target, so it is now registered with the other AGC tests. It set
APS5_PIN_WAIT_MS through _putenv_s only, which does not exist outside Windows; it now uses setenv
there. The heap mirror tests need a write-watched guest arena and are skipped with a notice
without one, as the recorder tests already do.
Every run recompiled every shader: the recompiler's variant cache and
the device's VkPipelineCache only lived in memory, and the first frames
of a title spent hundreds of milliseconds per large program in the
front end, the emitter and the optimizer.
Compiled variants (the SPIR-V plus the compiled shader info and binding
allocation that per-snapshot binding population needs) are now stored
on disk, one file per key, written atomically by a background thread,
and a later run loads them instead of compiling. The key holds every
input the in-memory variant cache matches on: the in-memory recompile
key (stage, code hash, stage registers, user data count, target), every
code word, the host subgroup size, the binding layout, the resource
specialization, and the values of the APS5_* switches the recompiler
sources read (found at build time; switches that only report are left
out). Entries live under a directory per source version, a build-time
hash of the recompiler and AGC driver sources and the build
configuration, and carry their key and checksums: an entry whose
format, version, key, size or checksum does not match, or whose payload
does not decode exactly, is ignored and recompiled. Directories no
build used for two weeks and stale temporary files are removed. Requests
under the debug probe bypass the disk cache.
The device's VkPipelineCache is loaded at creation when the file's
checksum and Vulkan header match the device (vendor, device and
pipelineCacheUUID; the driver version is in the file name), saved every
10 s while it grew, and saved at teardown.
Location: a shader_cache folder beside the running executable (not the
working directory), so the relinked game stays a self-contained folder;
ANYPS5_SHADER_CACHE_DIR overrides it. ANYPS5_NO_SHADER_CACHE=1 turns both caches off; the AGC
tests run with it set so they never write to the user's cache.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ButHENCbXXt8h4EZuWUabJ
NVIDIA's compiler inlines every call of the BDA lookup functions, each with its binary-search
loop, and then spends minutes compiling shaders with a few hundred guest memory accesses. Marked
DontInline, they compile in seconds. Pipeline creation without the driver cache, NVIDIA RTX PRO
6000 Blackwell:
| Shader | SPIR-V words | Inlined | DontInline |
|--------------|--------------|----------|------------|
| 0x25b1e8600 | 280168 | 104.4 s | 4.8 s |
| 0x7ff6df189d00 | 95698 | 86.3 s | 1.5 s |
| 0x25b27ce00 | 39959 | 6.0 s | 0.45 s |
| 0x25b27e400 | 40640 | 6.6 s | 0.48 s |
| 0x7ff6df2a8000 | 69179 | 5.7 s | 1.1 s |
| 0x7ff6df253400 | 90491 | 4.1 s | 0.74 s |
| 0x7ff6df256700 | 92459 | 4.0 s | 0.69 s |
The menu renders 3311 frames in 120 s against 3325 without the change; the story mode load takes
the same time and GPU dispatch time in both builds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 7615af4d4dd31581f52aa5d88118c73435fd6ae9)
As on FreeBSD, a munmap range covering more than one registered allocation (or gaps between them)
is unmapped allocation by allocation instead of throwing "unmap crosses allocation boundaries".
GTA V unmaps such ranges in its Story Mode prologue.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Starting in app0 when the folder exists made the working directory, and with it the tests, depend
on the layout next to the executable. The title opens its files by absolute /app0 paths, so the
fallback was never used: the route to story mode runs the same without it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The C11 thread layer (_Mtx_*, _Cnd_*, _Thrd_*, _Xtime_get_ticks, std::_Pad) and the libc additions
_FNan, _FDtest, _FExp, __udivti3, _Throw_Cpp_error, generic_category, localeconv, wcsncpy_s and the
aligned operator new came from the --skip-sce-module route, where our libc stood in for the title's.
With the title's sce_module loaded, its libc.prx exports every one of these NIDs and the relinker
lists the guest modules ahead of the host prx, so the imports never reach our copies. Checked both
by the NID tables (the executable imports them from libc, the guest libc.prx exports them) and by
making the four std::_Pad functions throw: the title boots, reaches the menu and loads story mode
without calling them.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
It checks the Windows formatter (the guest va_list must stay untouched).
On Linux the guest variadic path hands the call to glibc, which advances
the va_list, and snprintf_nid_postfix with arguments on the stack
crashes; upstream never registered this test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WinLibs puts its own cmake.exe (4.3, OpenSSL) first in PATH and ships no
CA certificates, so file(DOWNLOAD) of the FFmpeg package failed with
curl error 60. SSL_CERT_FILE points it at the bundle of Git for Windows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The MinGW FFmpeg package needs bcrypt, ole32 and user32; linking them
unconditionally broke libSceAjm.native.prx on Linux. The Linux package's
x86 assembly references its constants PC32, which ld refuses in a shared
object unless the archives' symbols stay local: --exclude-libs,ALL keeps
them out of the prx's dynamic symbols.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sceSaveDataSetParam dropped the params, and sceSaveDataGetParam and
sceSaveDataDirNameSearch returned zeros, so a title that checks the params of
the save it found took its profile for a new one and rewrote it with defaults
on every boot. The params now live in a <directory>.param sidecar next to the
save directory, written by SetParam (mtime refreshed) and returned by GetParam
and by the directory search, with the directory's write time when no param
was stored yet.
The mount point is /savedata<N>, as on the console: a title read its profile
through /host//_sm/0/Profile when the mount point had another shape, and that
path does not exist.
The module logs the first three calls of each entry point, like the core
module; APS5_SAVEDATA_TRACE lifts the limit.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
S_CBRANCH_CDBGSYS, S_CBRANCH_CDBGUSER, S_CBRANCH_CDBGSYS_OR_USER and S_CBRANCH_CDBGSYS_AND_USER
(SOPP 0x17-0x1a) branch only while a debugger sets the conditional debug bits in the wave's
status, which never happens on a retail console, so they decode to one no-op opcode.
mingw's fprintf writes an unbuffered stderr a character at a time: a title skipping many draws
spent seconds per frame printing 17 KB RecompileRequest dumps (GTA V ran at 0.1 fps). Reports are
built into one string and written once; dumps stop after 8 per reason (APS5_SKIP_REQUEST_DUMPS,
-1 for all).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A float inline constant (4.0, 2.0, -4.0...) read by an f16 operation is the half value in the low
16 bits (LLVM getInlineEncodingV216); it was the low half of the f32 bits, 0. op_sel[0..2] now
selects the high half of v_fma/min3/max3/med3_f16 sources. GTA V's CAS sharpen computed
rcp(1 + 4w) as rcp(0) and stored black.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Pacuka
ljkx
Dean Galvin
Qznad
boykopovar
carman24r
STL1te
Michon
STL1te
asimplestray
MartinCajiao