From cefb5a9fc0f3db8f634dfd533810adb15786a3a7 Mon Sep 17 00:00:00 2001 From: TideOPL Date: Wed, 7 Oct 2026 10:15:39 +0100 Subject: [PATCH 001/460] docs(libScePad): record the scePadRead queue gap in technical debt Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libScePad/Export.cpp | 9 ++++----- docs/dev/TechnicalDebt.md | 1 + 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/core/libs/prx/libScePad/Export.cpp b/core/libs/prx/libScePad/Export.cpp index c426e5067..bc9464c19 100644 --- a/core/libs/prx/libScePad/Export.cpp +++ b/core/libs/prx/libScePad/Export.cpp @@ -142,11 +142,10 @@ int APS5_VABI scePadOpen_nid_postfix(int userId, int type, int index, const void int APS5_VABI scePadReadState(int handle, PadData* data); int APS5_VABI scePadRead_nid_postfix(int handle, PadData* data, int num) { - // The title drains the queued states; one current state is reported per call. - if (data == nullptr || num <= 0) APS5_INVALID_ARG_EX; - const int result = scePadReadState(handle, data); - if (result != 0) return result; - return 1; + if (data == nullptr || num <= 0) APS5_INVALID_ARG_EX; + const int result = scePadReadState(handle, data); + if (result != 0) return result; + return 1; } int APS5_VABI scePadReadState(int handle, PadData* data) { diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 57ce5607d..7154cc035 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -233,3 +233,4 @@ Throughout the project, every function at every stage either **does exactly what - [sceFontOpenFontSet](../../core/libs/prx/libSceFont/src/SystemFont.cpp) (libSceFont) opens only the primary file of a system font set. Characters the console takes from the set's other files (Hangul, Thai and Arabic in Japanese and Chinese sets, Arabic in European sets) are reported as unsupported glyphs. The console file names are those of the PS4 firmware; the names of the PS5-only JG2 sets (`0x1A......`) are unknown, so they use the SSTJpPro files. When a console file is missing, an openly licensed substitute (Noto Sans, Noto Sans Mono, Noto Sans Thai, Noto Sans CJK) is loaded instead. It is not the system font: its metrics differ, so text width, line height and wrapping differ from the console. System font faces are loaded by FreeType in host memory, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. - [Font writing lines](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - `sceFontCreateWritingLine`, `sceFontWritingLineWritesOrder`, `sceFontWritingLineRefersRenderStep`, `sceFontWritingLineGetRenderMetrics` and `sceFontWritingLineClear` follow [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/main/src/libs/libFont.cpp): runs are placed left to right, each at the line's advance so far, with no spacing or adjustment. Only horizontal and left-to-right lines, writing attribute 0 and non-null run metrics are modelled; other values, the metrics of an empty line and unknown or null handles throw, since no reference gives their result or error code. `sceFontWritingLineGetOrderingSpace` still throws (KytyPS5's values are placeholders). Lines are allocated on the host, not in the font memory - [sceImeKeyboardGetResourceId](../../core/libs/prx/libSceIme/Export.cpp) (libSceIme) reports that no USB keyboard is connected (`0x80bc0004`, as [shadPS4](https://github.com/shadps4-emu/shadPS4) does) because IME keyboard events are not delivered to the handler, although [libSceKeyboard](../../core/libs/prx/libSceKeyboard/src/keyboard_impl.cpp) exposes the host keyboard +- [scePadRead](../../core/libs/prx/libScePad/Export.cpp) (libScePad) keeps no queue of past states: each call writes the current state to the first entry and returns 1 whatever `num` is, so input that changes and changes back between two calls is not reported From efa5afbfbd9bed76551b81508fe61c5bef6a3976 Mon Sep 17 00:00:00 2001 From: TideOPL Date: Wed, 7 Oct 2026 11:03:30 +0100 Subject: [PATCH 002/460] fix(libkernel): read the process start before the clock in process time sceKernelGetProcessTime and sceKernelGetProcessTimeCounter subtracted GetStartNanos() from GetMonotonicNanos() in one expression. GCC reads the clock first, so on the first call in a process the start was captured after it and the result wrapped to about 2^64. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libkernel/CMakeLists.txt | 10 ++++++++++ core/libs/prx/libkernel/Time/Time.cpp | 6 ++++-- core/libs/tests/GuestProcessTime.cpp | 25 +++++++++++++++++++++++++ 3 files changed, 39 insertions(+), 2 deletions(-) create mode 100644 core/libs/tests/GuestProcessTime.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index a9521483b..475dfb4be 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -90,6 +90,16 @@ if(BUILD_TESTING) set_tests_properties(guest_posix_sem_init guest_posix_sem_post PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_process_time_tests ${LIBS_INCLUDE_DIR}/tests/GuestProcessTime.cpp) + target_include_directories(guest_process_time_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_process_time_tests PRIVATE libkernel libc) + configure_windows_unwind(guest_process_time_tests) + add_test(NAME guest_process_time COMMAND guest_process_time_tests) + add_test(NAME guest_process_time_counter COMMAND guest_process_time_tests counter) + if(WIN32) + set_tests_properties(guest_process_time guest_process_time_counter PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_library(guest_module_fixture SHARED ${LIBS_INCLUDE_DIR}/tests/GuestModuleFixture.cpp) target_include_directories(guest_module_fixture PRIVATE ${LIBS_INCLUDE_DIR}) add_custom_command(TARGET guest_module_fixture POST_BUILD diff --git a/core/libs/prx/libkernel/Time/Time.cpp b/core/libs/prx/libkernel/Time/Time.cpp index 395f85829..521328a18 100644 --- a/core/libs/prx/libkernel/Time/Time.cpp +++ b/core/libs/prx/libkernel/Time/Time.cpp @@ -147,11 +147,13 @@ static const bool g_timerSelfTest = [] { extern "C" { std::uint64_t APS5_VABI sceKernelGetProcessTime() { - return (GetMonotonicNanos() - GetStartNanos()) / 1000ULL; + const std::uint64_t start = GetStartNanos(); + return (GetMonotonicNanos() - start) / 1000ULL; } std::uint64_t APS5_VABI sceKernelGetProcessTimeCounter() { - return GetMonotonicNanos() - GetStartNanos(); + const std::uint64_t start = GetStartNanos(); + return GetMonotonicNanos() - start; } std::uint64_t APS5_VABI sceKernelGetProcessTimeCounterFrequency() { diff --git a/core/libs/tests/GuestProcessTime.cpp b/core/libs/tests/GuestProcessTime.cpp new file mode 100644 index 000000000..cb8a6186e --- /dev/null +++ b/core/libs/tests/GuestProcessTime.cpp @@ -0,0 +1,25 @@ +#include "prx/libkernel/Time/include/Time.hpp" +#include +#include +#include +#include + +static void Require(bool condition, const char* message) { + if (!condition) { + std::fprintf(stderr, "%s\n", message); + std::abort(); + } +} + +// The first call in a process also fixes the process start, so it must not read the clock before it. +int main(int argc, char** argv) { + constexpr std::uint64_t minuteMicros = 60ULL * 1000 * 1000; + if (argc > 1 && std::strcmp(argv[1], "counter") == 0) { + Require(sceKernelGetProcessTimeCounter() < minuteMicros * 1000, "first process time counter wrapped"); + return 0; + } + const std::uint64_t first = sceKernelGetProcessTime(); + Require(first < minuteMicros, "first process time wrapped"); + Require(sceKernelGetProcessTime() >= first, "process time went backwards"); + return 0; +} From efa6cd56f08fa4b4b0b90096355023ae64136743 Mon Sep 17 00:00:00 2001 From: TideOPL Date: Wed, 7 Oct 2026 11:03:30 +0100 Subject: [PATCH 003/460] fix(libScePad): queue input states for scePadRead scePadRead reported only the current state, so input that changed and changed back between two reads was lost. Published states are now kept in a queue of up to 64 and drained oldest first; with none queued the current state is returned as before. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libScePad/Export.cpp | 7 +-- core/libs/prx/libScePad/include/PadState.hpp | 2 + core/libs/prx/libScePad/src/PadState.cpp | 50 +++++++++++++++++--- core/libs/tests/GuestPad.cpp | 17 +++++++ docs/dev/TechnicalDebt.md | 1 - 5 files changed, 64 insertions(+), 13 deletions(-) diff --git a/core/libs/prx/libScePad/Export.cpp b/core/libs/prx/libScePad/Export.cpp index bc9464c19..0d20fed00 100644 --- a/core/libs/prx/libScePad/Export.cpp +++ b/core/libs/prx/libScePad/Export.cpp @@ -139,13 +139,10 @@ int APS5_VABI scePadOpen_nid_postfix(int userId, int type, int index, const void return PAD_HANDLE; } -int APS5_VABI scePadReadState(int handle, PadData* data); - int APS5_VABI scePadRead_nid_postfix(int handle, PadData* data, int num) { + if (handle != PAD_HANDLE) return PAD_ERROR_INVALID_HANDLE; if (data == nullptr || num <= 0) APS5_INVALID_ARG_EX; - const int result = scePadReadState(handle, data); - if (result != 0) return result; - return 1; + return Pad::Read(data, num); } int APS5_VABI scePadReadState(int handle, PadData* data) { diff --git a/core/libs/prx/libScePad/include/PadState.hpp b/core/libs/prx/libScePad/include/PadState.hpp index c23b02c4d..1d2205fa6 100644 --- a/core/libs/prx/libScePad/include/PadState.hpp +++ b/core/libs/prx/libScePad/include/PadState.hpp @@ -51,6 +51,8 @@ struct PadOutputState { namespace Pad { void Initialize(); PadData ReadState(); +// Drains up to `num` queued states, oldest first; with none queued, writes the current state. Returns the count written. +int Read(PadData* data, int num); void SetVibration(std::uint8_t large, std::uint8_t small); void SetVibrationMode(int mode); void SetLightBar(bool valid, std::uint8_t r, std::uint8_t g, std::uint8_t b); diff --git a/core/libs/prx/libScePad/src/PadState.cpp b/core/libs/prx/libScePad/src/PadState.cpp index ea3e212d4..3da6c4f21 100644 --- a/core/libs/prx/libScePad/src/PadState.cpp +++ b/core/libs/prx/libScePad/src/PadState.cpp @@ -4,6 +4,7 @@ #include #include #include +#include #include #include @@ -12,6 +13,14 @@ namespace { PadInputState state; PadOutputState output; std::uint64_t timestamp = 0; + + struct QueuedInput { + PadInputState input; + std::uint64_t timestamp = 0; + }; + // States published since the last scePadRead; the oldest is dropped once 64 are waiting. + constexpr std::size_t kQueueCapacity = 64; + std::deque queue; std::exception_ptr failure; bool initialized = false; @@ -93,15 +102,14 @@ void Pad::Initialize() { if (!initialized) { timestamp = sceKernelGetProcessTime(); lastFuseTime = timestamp; + queue.clear(); } initialized = true; } -PadData Pad::ReadState() { - std::lock_guard lock(stateMutex); - if (failure) std::rethrow_exception(failure); - if (!initialized) throw std::runtime_error("Pad: read before initialization"); - const std::uint64_t now = sceKernelGetProcessTime(); +namespace { +// Converts one input sample; motion fusion integrates up to `now` and touch ids advance per call. +PadData BuildData(const PadInputState& state, std::uint64_t changeTime, std::uint64_t now) { PadData data{}; data.buttons = state.buttons; data.left_stick_x = state.sticks[0]; @@ -123,7 +131,7 @@ PadData Pad::ReadState() { orientation = Quat{}; biasIntegral[0] = biasIntegral[1] = biasIntegral[2] = 0.0f; } - lastFuseTime = now; + lastFuseTime = std::max(lastFuseTime, now); data.orientation_x = orientation.x; data.orientation_y = orientation.y; data.orientation_z = orientation.z; @@ -138,7 +146,7 @@ PadData Pad::ReadState() { data.connected = true; data.connected_count = 1; - data.timestamp = timestamp; + data.timestamp = changeTime; std::uint8_t touchNum = 0; for (int i = 0; i < 2; ++i) { @@ -163,6 +171,32 @@ PadData Pad::ReadState() { data.touch_data_touch_num = touchNum; return data; } +} + +PadData Pad::ReadState() { + std::lock_guard lock(stateMutex); + if (failure) std::rethrow_exception(failure); + if (!initialized) throw std::runtime_error("Pad: read before initialization"); + return BuildData(state, timestamp, sceKernelGetProcessTime()); +} + +int Pad::Read(PadData* data, int num) { + if (data == nullptr || num <= 0) throw std::invalid_argument("Pad: invalid read buffer"); + std::lock_guard lock(stateMutex); + if (failure) std::rethrow_exception(failure); + if (!initialized) throw std::runtime_error("Pad: read before initialization"); + if (queue.empty()) { + data[0] = BuildData(state, timestamp, sceKernelGetProcessTime()); + return 1; + } + int count = 0; + while (count < num && !queue.empty()) { + const QueuedInput& entry = queue.front(); + data[count++] = BuildData(entry.input, entry.timestamp, entry.timestamp); + queue.pop_front(); + } + return count; +} void Pad::SetVibration(std::uint8_t large, std::uint8_t small) { std::lock_guard lock(stateMutex); @@ -280,6 +314,8 @@ extern "C" void PadPublishInput_nid_postfix(const PadInputState& input) { state.touch == input.touch && state.deviceKind == input.deviceKind) return; state = input; timestamp = sceKernelGetProcessTime(); + if (queue.size() == kQueueCapacity) queue.pop_front(); + queue.push_back({input, timestamp}); } extern "C" void PadReportInputFailure_nid_postfix(std::exception_ptr error) { diff --git a/core/libs/tests/GuestPad.cpp b/core/libs/tests/GuestPad.cpp index 866c44ba7..68c23fd6e 100644 --- a/core/libs/tests/GuestPad.cpp +++ b/core/libs/tests/GuestPad.cpp @@ -59,6 +59,22 @@ static void CheckTouchContact() { Pad::ReadState(); } +static void CheckReadQueue(int handle) { + PadData drained[64]{}; + scePadRead_nid_postfix(handle, drained, 64); + PadInputState pressed; + pressed.buttons = static_cast(Pad::PadButton::Cross); + PadPublishInput_nid_postfix(pressed); + PadPublishInput_nid_postfix(PadInputState{}); + PadData data[4]{}; + Require(scePadRead_nid_postfix(handle, data, 4) == 2); + Require((data[0].buttons & pressed.buttons) != 0); + Require((data[1].buttons & pressed.buttons) == 0); + Require(data[0].timestamp <= data[1].timestamp); + Require(scePadRead_nid_postfix(handle, data, 4) == 1); + Require(data[0].buttons == 0); +} + static void CheckReadStateHandle(int handle) { PadData data{}; Require(scePadReadState(0, &data) == PAD_ERROR_INVALID_HANDLE); @@ -81,6 +97,7 @@ int main() { Require(scePadGetHandle(user, 2, 0) == handle); CheckTiltCorrection(handle); CheckTouchContact(); + CheckReadQueue(handle); CheckReadStateHandle(handle); Require(scePadGetHandle(0xff, 16, 0) == handle); Require(scePadGetHandle(user, 16, 0) == noHandle); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 7154cc035..57ce5607d 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -233,4 +233,3 @@ Throughout the project, every function at every stage either **does exactly what - [sceFontOpenFontSet](../../core/libs/prx/libSceFont/src/SystemFont.cpp) (libSceFont) opens only the primary file of a system font set. Characters the console takes from the set's other files (Hangul, Thai and Arabic in Japanese and Chinese sets, Arabic in European sets) are reported as unsupported glyphs. The console file names are those of the PS4 firmware; the names of the PS5-only JG2 sets (`0x1A......`) are unknown, so they use the SSTJpPro files. When a console file is missing, an openly licensed substitute (Noto Sans, Noto Sans Mono, Noto Sans Thai, Noto Sans CJK) is loaded instead. It is not the system font: its metrics differ, so text width, line height and wrapping differ from the console. System font faces are loaded by FreeType in host memory, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. - [Font writing lines](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - `sceFontCreateWritingLine`, `sceFontWritingLineWritesOrder`, `sceFontWritingLineRefersRenderStep`, `sceFontWritingLineGetRenderMetrics` and `sceFontWritingLineClear` follow [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/main/src/libs/libFont.cpp): runs are placed left to right, each at the line's advance so far, with no spacing or adjustment. Only horizontal and left-to-right lines, writing attribute 0 and non-null run metrics are modelled; other values, the metrics of an empty line and unknown or null handles throw, since no reference gives their result or error code. `sceFontWritingLineGetOrderingSpace` still throws (KytyPS5's values are placeholders). Lines are allocated on the host, not in the font memory - [sceImeKeyboardGetResourceId](../../core/libs/prx/libSceIme/Export.cpp) (libSceIme) reports that no USB keyboard is connected (`0x80bc0004`, as [shadPS4](https://github.com/shadps4-emu/shadPS4) does) because IME keyboard events are not delivered to the handler, although [libSceKeyboard](../../core/libs/prx/libSceKeyboard/src/keyboard_impl.cpp) exposes the host keyboard -- [scePadRead](../../core/libs/prx/libScePad/Export.cpp) (libScePad) keeps no queue of past states: each call writes the current state to the first entry and returns 1 whatever `num` is, so input that changes and changes back between two calls is not reported From 71e0f6428f91d8c025631e856e9df332648cccad Mon Sep 17 00:00:00 2001 From: TideOPL Date: Wed, 7 Oct 2026 11:06:09 +0100 Subject: [PATCH 004/460] style(libScePad): remove code comments Per CONVENTIONS, code comments only mark technical debt. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libScePad/include/PadState.hpp | 1 - core/libs/prx/libScePad/src/PadState.cpp | 2 -- core/libs/tests/GuestProcessTime.cpp | 1 - 3 files changed, 4 deletions(-) diff --git a/core/libs/prx/libScePad/include/PadState.hpp b/core/libs/prx/libScePad/include/PadState.hpp index 1d2205fa6..8100e33c5 100644 --- a/core/libs/prx/libScePad/include/PadState.hpp +++ b/core/libs/prx/libScePad/include/PadState.hpp @@ -51,7 +51,6 @@ struct PadOutputState { namespace Pad { void Initialize(); PadData ReadState(); -// Drains up to `num` queued states, oldest first; with none queued, writes the current state. Returns the count written. int Read(PadData* data, int num); void SetVibration(std::uint8_t large, std::uint8_t small); void SetVibrationMode(int mode); diff --git a/core/libs/prx/libScePad/src/PadState.cpp b/core/libs/prx/libScePad/src/PadState.cpp index 3da6c4f21..14ce43eac 100644 --- a/core/libs/prx/libScePad/src/PadState.cpp +++ b/core/libs/prx/libScePad/src/PadState.cpp @@ -18,7 +18,6 @@ namespace { PadInputState input; std::uint64_t timestamp = 0; }; - // States published since the last scePadRead; the oldest is dropped once 64 are waiting. constexpr std::size_t kQueueCapacity = 64; std::deque queue; std::exception_ptr failure; @@ -108,7 +107,6 @@ void Pad::Initialize() { } namespace { -// Converts one input sample; motion fusion integrates up to `now` and touch ids advance per call. PadData BuildData(const PadInputState& state, std::uint64_t changeTime, std::uint64_t now) { PadData data{}; data.buttons = state.buttons; diff --git a/core/libs/tests/GuestProcessTime.cpp b/core/libs/tests/GuestProcessTime.cpp index cb8a6186e..b24cf8249 100644 --- a/core/libs/tests/GuestProcessTime.cpp +++ b/core/libs/tests/GuestProcessTime.cpp @@ -11,7 +11,6 @@ static void Require(bool condition, const char* message) { } } -// The first call in a process also fixes the process start, so it must not read the clock before it. int main(int argc, char** argv) { constexpr std::uint64_t minuteMicros = 60ULL * 1000 * 1000; if (argc > 1 && std::strcmp(argv[1], "counter") == 0) { From 58f596ee894d5d0d326a379032950ff4170c6964 Mon Sep 17 00:00:00 2001 From: Suryansh Prajapati <58465650+drk1rd@users.noreply.github.com> Date: Thu, 8 Oct 2026 13:31:24 +0530 Subject: [PATCH 005/460] fix(shader): convert f16 bits to f32 without UnpackHalf2x16 On the AMD Windows driver (Radeon RDNA2 iGPU, 32.0.21030.2001) UnpackHalf2x16 of a byte-selected half converts the unmasked source register: an SDWA BYTE_0 source of 0xfff0fff0 reads as f16 0xfff0 instead of 0x00f0, so agc_driver_sdwa_float_selectors and agc_driver_sdwa_ldexp_f16 fail there. EmitF16BitsToF32 now decodes sign, exponent and mantissa with integer operations, scaling subnormals by 2^-24, which is exact for every half and leaves the driver nothing to fold into the conversion. --- .../SpirvBackend/src/SpirvModuleEmitter.cpp | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp index 1fb490ff1..518aca2b8 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp @@ -860,11 +860,20 @@ std::uint32_t EmitTrigCycleF32(SpirvEmitterState& state, std::uint32_t src, bool } std::uint32_t EmitF16BitsToF32(SpirvEmitterState& state, std::uint32_t bits) { - const auto unpacked = state.module.AllocateId(); - const auto result = state.module.AllocateId(); - state.module.AddFunction(spv::OpExtInst, TypeF32Vector(state, 2), unpacked, GlslStd450(state), GLSLstd450UnpackHalf2x16, bits); - state.module.AddFunction(spv::OpCompositeExtract, TypeF32(state), result, unpacked, 0u); - return result; + const auto u32 = TypeU32(state); + const auto f32 = TypeF32(state); + const auto op = [&](spv::Op opcode, std::uint32_t lhs, std::uint32_t rhs) { return Binary(state, opcode, u32, lhs, rhs); }; + const auto sign = op(spv::OpShiftLeftLogical, op(spv::OpBitwiseAnd, bits, ConstantU32(state, 0x8000u)), ConstantU32(state, 16u)); + const auto exponent = op(spv::OpBitwiseAnd, op(spv::OpShiftRightLogical, bits, ConstantU32(state, 10u)), ConstantU32(state, 0x1fu)); + const auto mantissa = op(spv::OpBitwiseAnd, bits, ConstantU32(state, 0x3ffu)); + const auto mantissa32 = op(spv::OpShiftLeftLogical, mantissa, ConstantU32(state, 13u)); + const auto normal = op(spv::OpBitwiseOr, op(spv::OpShiftLeftLogical, op(spv::OpIAdd, exponent, ConstantU32(state, 127u - 15u)), ConstantU32(state, 23u)), mantissa32); + const auto special = op(spv::OpBitwiseOr, ConstantU32(state, 0x7f800000u), mantissa32); + const auto subnormal = Unary(state, spv::OpBitcast, u32, Binary(state, spv::OpFMul, f32, Unary(state, spv::OpConvertUToF, f32, mantissa), ConstantF32Value(state, 0x1p-24f))); + const auto boolean = TypeBool(state); + const auto finite = Select(state, u32, Binary(state, spv::OpIEqual, boolean, exponent, ConstantU32(state, 0u)), subnormal, normal); + const auto magnitude = Select(state, u32, Binary(state, spv::OpIEqual, boolean, exponent, ConstantU32(state, 0x1fu)), special, finite); + return Unary(state, spv::OpBitcast, f32, op(spv::OpBitwiseOr, sign, magnitude)); } void EmitProgram(SpirvEmitterState& state) { From 8b28f53a4dcf11cdf0bee7b673f9833e5263e4d2 Mon Sep 17 00:00:00 2001 From: TideOPL Date: Thu, 8 Oct 2026 11:00:22 +0100 Subject: [PATCH 006/460] revert(libkernel): move the process time fix to its own pull request --- core/libs/prx/libkernel/CMakeLists.txt | 10 ---------- core/libs/prx/libkernel/Time/Time.cpp | 6 ++---- core/libs/tests/GuestProcessTime.cpp | 24 ------------------------ 3 files changed, 2 insertions(+), 38 deletions(-) delete mode 100644 core/libs/tests/GuestProcessTime.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index 752f3f7e8..31ad9ae4b 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -84,16 +84,6 @@ if(BUILD_TESTING) set_tests_properties(guest_posix_sem_init guest_posix_sem_post PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() - add_test_executable(guest_process_time_tests ${LIBS_INCLUDE_DIR}/tests/GuestProcessTime.cpp) - target_include_directories(guest_process_time_tests PRIVATE ${LIBS_INCLUDE_DIR}) - target_link_libraries(guest_process_time_tests PRIVATE libkernel libc) - configure_windows_unwind(guest_process_time_tests) - add_test(NAME guest_process_time COMMAND guest_process_time_tests) - add_test(NAME guest_process_time_counter COMMAND guest_process_time_tests counter) - if(WIN32) - set_tests_properties(guest_process_time guest_process_time_counter PROPERTIES ENVIRONMENT_MODIFICATION - "PATH=path_list_prepend:$") - endif() add_library(guest_module_fixture SHARED ${LIBS_INCLUDE_DIR}/tests/GuestModuleFixture.cpp) target_include_directories(guest_module_fixture PRIVATE ${LIBS_INCLUDE_DIR}) add_custom_command(TARGET guest_module_fixture POST_BUILD diff --git a/core/libs/prx/libkernel/Time/Time.cpp b/core/libs/prx/libkernel/Time/Time.cpp index 521328a18..395f85829 100644 --- a/core/libs/prx/libkernel/Time/Time.cpp +++ b/core/libs/prx/libkernel/Time/Time.cpp @@ -147,13 +147,11 @@ static const bool g_timerSelfTest = [] { extern "C" { std::uint64_t APS5_VABI sceKernelGetProcessTime() { - const std::uint64_t start = GetStartNanos(); - return (GetMonotonicNanos() - start) / 1000ULL; + return (GetMonotonicNanos() - GetStartNanos()) / 1000ULL; } std::uint64_t APS5_VABI sceKernelGetProcessTimeCounter() { - const std::uint64_t start = GetStartNanos(); - return GetMonotonicNanos() - start; + return GetMonotonicNanos() - GetStartNanos(); } std::uint64_t APS5_VABI sceKernelGetProcessTimeCounterFrequency() { diff --git a/core/libs/tests/GuestProcessTime.cpp b/core/libs/tests/GuestProcessTime.cpp deleted file mode 100644 index b24cf8249..000000000 --- a/core/libs/tests/GuestProcessTime.cpp +++ /dev/null @@ -1,24 +0,0 @@ -#include "prx/libkernel/Time/include/Time.hpp" -#include -#include -#include -#include - -static void Require(bool condition, const char* message) { - if (!condition) { - std::fprintf(stderr, "%s\n", message); - std::abort(); - } -} - -int main(int argc, char** argv) { - constexpr std::uint64_t minuteMicros = 60ULL * 1000 * 1000; - if (argc > 1 && std::strcmp(argv[1], "counter") == 0) { - Require(sceKernelGetProcessTimeCounter() < minuteMicros * 1000, "first process time counter wrapped"); - return 0; - } - const std::uint64_t first = sceKernelGetProcessTime(); - Require(first < minuteMicros, "first process time wrapped"); - Require(sceKernelGetProcessTime() >= first, "process time went backwards"); - return 0; -} From 39463913c70f0ac7aff7c486662be20510ef69c6 Mon Sep 17 00:00:00 2001 From: TideOPL Date: Thu, 8 Oct 2026 21:56:33 +0100 Subject: [PATCH 007/460] fix(libScePad): fuse queued pad states at the read time Keeping lastFuseTime at its maximum never let it recover from a process time that wrapped on the first call (fixed separately in #1501), so tilt correction stopped and guest_pad failed. Queued states keep their own timestamp; the orientation is fused once at the time of the read, as scePadReadState does. --- core/libs/prx/libScePad/src/PadState.cpp | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libScePad/src/PadState.cpp b/core/libs/prx/libScePad/src/PadState.cpp index 14ce43eac..1bd213078 100644 --- a/core/libs/prx/libScePad/src/PadState.cpp +++ b/core/libs/prx/libScePad/src/PadState.cpp @@ -129,7 +129,7 @@ PadData BuildData(const PadInputState& state, std::uint64_t changeTime, std::uin orientation = Quat{}; biasIntegral[0] = biasIntegral[1] = biasIntegral[2] = 0.0f; } - lastFuseTime = std::max(lastFuseTime, now); + lastFuseTime = now; data.orientation_x = orientation.x; data.orientation_y = orientation.y; data.orientation_z = orientation.z; @@ -183,14 +183,15 @@ int Pad::Read(PadData* data, int num) { std::lock_guard lock(stateMutex); if (failure) std::rethrow_exception(failure); if (!initialized) throw std::runtime_error("Pad: read before initialization"); + const std::uint64_t now = sceKernelGetProcessTime(); if (queue.empty()) { - data[0] = BuildData(state, timestamp, sceKernelGetProcessTime()); + data[0] = BuildData(state, timestamp, now); return 1; } int count = 0; while (count < num && !queue.empty()) { const QueuedInput& entry = queue.front(); - data[count++] = BuildData(entry.input, entry.timestamp, entry.timestamp); + data[count++] = BuildData(entry.input, entry.timestamp, now); queue.pop_front(); } return count; From 1c6e53b9bf454346c9d01b195549f0de37dfaaa8 Mon Sep 17 00:00:00 2001 From: "husari.orozco" Date: Fri, 9 Oct 2026 10:46:35 -0600 Subject: [PATCH 008/460] ci(conventions): exempt cmake/SystemDeps.cmake from the system-dependency rule --- tools/check_conventions.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/check_conventions.py b/tools/check_conventions.py index 78a136d13..204e65eb6 100644 --- a/tools/check_conventions.py +++ b/tools/check_conventions.py @@ -126,6 +126,8 @@ class Check: def build(self): for path, lines in self.files(lambda p: p.name == "CMakeLists.txt" or p.suffix == ".cmake"): + if path == "cmake/SystemDeps.cmake": + continue for number, text in lines: if SYSTEM_DEPENDENCY.search(text.split("#", 1)[0]): self.report("system-dependency", path, number, text.strip()) From b7df8884ff176661561b456cb032696a1bb10620 Mon Sep 17 00:00:00 2001 From: Karan Vora <52519664+karanvora2599@users.noreply.github.com> Date: Fri, 9 Oct 2026 12:44:14 -0400 Subject: [PATCH 009/460] perf(shader): saturate f32 to integer conversions once The translation of v_cvt_i32/u32/flr_i32/rpi_i32_f32 and v_cvt_i16/u16_f16 already maps NaN to 0 and clamps the value into range before ConvertS32F32/ConvertU32F32, and the backend truncated it again and repeated the NaN and range selects on the result. Those IR ops have no other producers, so the backend now emits one OpConvertFToS or OpConvertFToU, and EmitF32ToU32 and EmitTruncF32Value, which had no other users, are removed. The translation drops its FPTrunc32, as the conversions round toward zero, and the selects whose constant the conversion of the clamped value already gives. v_cvt_i32_f32 and v_cvt_u32_f32 read their source without the f32 denormal flush: a denormal truncates to 0 either way. --- .../SpirvBackend/SpirvEmitterInstructions.hpp | 1 - .../src/SpirvAlu/SpirvAluEmitterMath.cpp | 34 ++----------------- .../Translation/src/ConvertInstructions.cpp | 19 +++++------ 3 files changed, 12 insertions(+), 42 deletions(-) diff --git a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterInstructions.hpp b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterInstructions.hpp index a2f5aa8d5..74a9d65e7 100644 --- a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterInstructions.hpp +++ b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterInstructions.hpp @@ -165,7 +165,6 @@ std::uint32_t EmitAndU32(SpirvEmitterState& state, std::uint32_t arg0, std::uint std::uint32_t EmitLogicalAndBool(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1); std::uint32_t EmitLogicalOrBool(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1); std::uint32_t EmitLogicalNotBool(SpirvEmitterState& state, std::uint32_t arg0); -std::uint32_t EmitTruncF32Value(SpirvEmitterState& state, std::uint32_t arg0); std::uint32_t EmitFNegateValue(SpirvEmitterState& state, std::uint32_t arg0); std::uint32_t EmitFAbsValue(SpirvEmitterState& state, std::uint32_t arg0); std::uint32_t EmitPackHalf2x16(SpirvEmitterState& state, std::uint32_t arg0); diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp index ab3a31dfe..4fefca365 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp @@ -140,31 +140,6 @@ std::uint32_t EmitExt(SpirvEmitterState& state, std::uint32_t type, std::uint32_ return result; } -std::uint32_t EmitF32ToU32(SpirvEmitterState& state, std::uint32_t src, bool signedValue) { - const auto truncated = EmitTruncF32Value(state, src); - const auto convertedRaw = state.module.AllocateId(); - if (signedValue) { - const auto convertedSigned = state.module.AllocateId(); - state.module.AddFunction(spv::OpConvertFToS, TypeI32(state), convertedSigned, truncated); - state.module.AddFunction(spv::OpBitcast, TypeU32(state), convertedRaw, convertedSigned); - } else { - state.module.AddFunction(spv::OpConvertFToU, TypeU32(state), convertedRaw, truncated); - } - const auto nan = EmitClassifyF32(state, src).nan; - if (signedValue) { - const auto below = Binary(state, spv::OpFOrdLessThanEqual, TypeBool(state), src, ConstantF32(state, 0xcf000000u)); - const auto above = Binary(state, spv::OpFOrdGreaterThanEqual, TypeBool(state), src, ConstantF32(state, 0x4f000000u)); - const auto high = Select(state, TypeU32(state), above, ConstantU32(state, 0x7fffffffu), convertedRaw); - const auto low = Select(state, TypeU32(state), below, ConstantU32(state, 0x80000000u), high); - return Select(state, TypeU32(state), nan, ConstantU32(state, 0u), low); - } - const auto below = Binary(state, spv::OpFOrdLessThanEqual, TypeBool(state), src, ConstantF32(state, 0u)); - const auto above = Binary(state, spv::OpFOrdGreaterThanEqual, TypeBool(state), src, ConstantF32(state, 0x4f800000u)); - const auto zero = Binary(state, spv::OpLogicalOr, TypeBool(state), nan, below); - const auto high = Select(state, TypeU32(state), above, ConstantU32(state, 0xffffffffu), convertedRaw); - return Select(state, TypeU32(state), zero, ConstantU32(state, 0u), high); -} - std::uint32_t EmitDppWriteCondition(SpirvValueEmitContext& ctx, const IrValue& inst) { auto& state = ctx.state; const auto flags = inst.Flags(); @@ -443,11 +418,12 @@ std::uint32_t EmitConvertF16F32(SpirvEmitterState& state, std::uint32_t arg0) { } std::uint32_t EmitConvertS32F32(SpirvEmitterState& state, std::uint32_t arg0) { - return EmitF32ToU32(state, arg0, true); + const auto converted = Unary(state, spv::OpConvertFToS, TypeI32(state), arg0); + return Unary(state, spv::OpBitcast, TypeU32(state), converted); } std::uint32_t EmitConvertU32F32(SpirvEmitterState& state, std::uint32_t arg0) { - return EmitF32ToU32(state, arg0, false); + return Unary(state, spv::OpConvertFToU, TypeU32(state), arg0); } std::uint32_t EmitConvertF32S32(SpirvEmitterState& state, std::uint32_t arg0) { @@ -1119,10 +1095,6 @@ std::uint32_t EmitLogicalNotBool(SpirvEmitterState& state, std::uint32_t arg0) { return EmitNative(state, arg0); } -std::uint32_t EmitTruncF32Value(SpirvEmitterState& state, std::uint32_t arg0) { - return EmitGlsl(state, arg0); -} - std::uint32_t EmitFNegateValue(SpirvEmitterState& state, std::uint32_t arg0) { return EmitNative(state, arg0); } diff --git a/core/shader/recompiler/Translation/src/ConvertInstructions.cpp b/core/shader/recompiler/Translation/src/ConvertInstructions.cpp index b0a367c0d..590b8d380 100644 --- a/core/shader/recompiler/Translation/src/ConvertInstructions.cpp +++ b/core/shader/recompiler/Translation/src/ConvertInstructions.cpp @@ -18,10 +18,10 @@ IrU32 TranslationContext::convertF32ToU32Saturated(IrF32 value, float upperBound const IrU1 nan(ir.Emit(IrOpcode::FPIsNan32, IrType::U1, {&value.Value()})); const IrU1 low(ir.Emit(IrOpcode::FPOrdLessThanEqual32, IrType::U1, {&value.Value(), &zero.Value()})); const IrU1 high(ir.Emit(IrOpcode::FPOrdGreaterThanEqual32, IrType::U1, {&value.Value(), &ir.ConstantF32(upperBound)})); - const IrF32 truncated(ir.Emit(IrOpcode::FPTrunc32, IrType::F32, {&value.Value()})); - const IrF32 safeLow = selectF32(IrU1(ir.LogicalOr(nan.Value(), low.Value())), zero, truncated); + const IrF32 safeLow = selectF32(IrU1(ir.LogicalOr(nan.Value(), low.Value())), zero, value); const IrF32 safe = selectF32(high, IrF32(ir.ConstantF32(safeUpper)), safeLow); const IrU32 converted(ir.Emit(IrOpcode::ConvertU32F32, IrType::U32, {&safe.Value()})); + if (static_cast(safeUpper) == highResult) return converted; return IrU32(ir.Select(high.Value(), ir.Constant(highResult), converted.Value())); } @@ -29,14 +29,13 @@ IrU32 TranslationContext::convertF32ToI32Saturated(IrF32 value, float lowerBound const IrU1 nan(ir.Emit(IrOpcode::FPIsNan32, IrType::U1, {&value.Value()})); const IrU1 low(ir.Emit(IrOpcode::FPOrdLessThanEqual32, IrType::U1, {&value.Value(), &ir.ConstantF32(lowerBound)})); const IrU1 high(ir.Emit(IrOpcode::FPOrdGreaterThanEqual32, IrType::U1, {&value.Value(), &ir.ConstantF32(upperBound)})); - const IrF32 truncated(ir.Emit(IrOpcode::FPTrunc32, IrType::F32, {&value.Value()})); - const IrF32 safeLow = selectF32(low, IrF32(ir.ConstantF32(lowerBound)), truncated); + const IrF32 safeLow = selectF32(low, IrF32(ir.ConstantF32(lowerBound)), value); const IrF32 safeHigh = selectF32(high, IrF32(ir.ConstantF32(safeUpper)), safeLow); const IrF32 safe = selectF32(nan, IrF32(ir.ConstantF32(0.0f)), safeHigh); - const IrU32 converted(ir.Emit(IrOpcode::ConvertS32F32, IrType::U32, {&safe.Value()})); - const IrU32 clampedHigh(ir.Select(high.Value(), ir.Constant(upperResult), converted.Value())); - const IrU32 clamped(ir.Select(low.Value(), ir.Constant(lowerResult), clampedHigh.Value())); - return IrU32(ir.Select(nan.Value(), ir.Constant(0u), clamped.Value())); + IrU32 converted(ir.Emit(IrOpcode::ConvertS32F32, IrType::U32, {&safe.Value()})); + if (static_cast(static_cast(lowerBound)) != lowerResult) converted = IrU32(ir.Select(low.Value(), ir.Constant(lowerResult), converted.Value())); + if (static_cast(static_cast(safeUpper)) != upperResult) converted = IrU32(ir.Select(high.Value(), ir.Constant(upperResult), converted.Value())); + return converted; } IrU32 TranslationContext::packU16Lanes(IrU32 low, IrU32 high) { @@ -66,13 +65,13 @@ void TranslationContext::vCvtF32I32(const RdnaInstruction& inst) { } void TranslationContext::vCvtU32F32(const RdnaInstruction& inst) { - const IrF32 value(*readOperand(sourceAt(inst, 0u), IrType::F32)); + const IrF32 value(ir.BitCastF32(*readOperand(sourceAt(inst, 0u), IrType::U32))); const IrU32 result = convertF32ToU32Saturated(value, 4294967296.0f, 4294967040.0f, 0xffffffffu); writeOperand(inst.destination, &result.Value()); } void TranslationContext::vCvtI32F32(const RdnaInstruction& inst) { - const IrF32 value(*readOperand(sourceAt(inst, 0u), IrType::F32)); + const IrF32 value(ir.BitCastF32(*readOperand(sourceAt(inst, 0u), IrType::U32))); const IrU32 result = convertF32ToI32Saturated(value, -2147483648.0f, 2147483648.0f, 2147483520.0f, 0x80000000u, 0x7fffffffu); writeOperand(inst.destination, &result.Value()); } From 42692dee11d7a6fe68261c50abba9ce3c71107d5 Mon Sep 17 00:00:00 2001 From: Karan Vora <52519664+karanvora2599@users.noreply.github.com> Date: Fri, 9 Oct 2026 13:38:56 -0400 Subject: [PATCH 010/460] perf(shader): fold the denormal flush into f32 compares against constants When f32 denormals are flushed, v_cmp_*_f32 compares flush(a) with flush(b). With one operand a constant, the flush of the other operand can go into the compare: a denormal and its flushed zero compare the same way with a NaN, an infinity or a normal constant, and comparing flush(x) with a zero is comparing x with +-2^-126 (flush(x) < 0 is x <= -2^-126, flush(x) == 0 is |x| < 2^-126, and so on, the unordered forms alike). emitFloatCompare reads both operands unflushed, finds a constant operand after its abs/neg modifiers, mirrors the predicate when the constant is first, and emits one compare. Two registers, f16 compares and modes that keep f32 denormals are unchanged. New test agc_driver_f32_compare_constant checks all 12 predicates against flush(a) op flush(b). --- .../tests/execution/F32CompareConstant.cpp | 183 ++++++++++++++++++ .../Translation/TranslationContext.hpp | 1 + .../Translation/src/CompareInstructions.cpp | 76 +++++++- 3 files changed, 258 insertions(+), 2 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/F32CompareConstant.cpp diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/F32CompareConstant.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/F32CompareConstant.cpp new file mode 100644 index 000000000..4b16ff31a --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/F32CompareConstant.cpp @@ -0,0 +1,183 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 32; +constexpr std::uint32_t Words = 8; +alignas(256) std::array Input{ + 0x00000000u, 0x80000000u, 0x00000001u, 0x80000001u, 0x007fffffu, 0x807fffffu, 0x00800000u, 0x80800000u, + 0x00800001u, 0x80800001u, 0x3f000000u, 0xbf000000u, 0x3effffffu, 0x3f000001u, 0xc0800000u, 0xc07fffffu, + 0xc0800001u, 0x3f800000u, 0x7f800000u, 0xff800000u, 0x7fc00000u, 0xffc00001u, 0x7f800001u, 0x7f7fffffu, + 0xff7fffffu, 0x00400000u, 0x80400000u, 0x0000ffffu, 0x3e800000u, 0xbe800000u, 0x80000000u, 0x01000000u, +}; +alignas(256) std::array Output{}; + +struct Constant { + std::uint32_t bits; + std::uint32_t inlineSource; +}; + +constexpr std::array Constants{{ + {0x00000000u, 0x80u}, {0x80000000u, 0xffu}, {0x00000001u, 0xffu}, {0x807fffffu, 0xffu}, {0x3f000000u, 0xf0u}, + {0xc0800000u, 0xf7u}, {0x00800000u, 0xffu}, {0x7f800000u, 0xffu}, {0x7fc00000u, 0xffu}, +}}; + +constexpr std::array Predicates{0x01u, 0x02u, 0x03u, 0x04u, 0x05u, 0x06u, 0x09u, 0x0au, 0x0bu, 0x0cu, 0x0du, 0x0eu}; +constexpr std::array PredicateNames{"lt", "eq", "le", "gt", "lg", "ge", "nge", "nlg", "ngt", "nle", "neq", "nlt"}; + +constexpr std::uint32_t Compares = static_cast(Constants.size() * Predicates.size() * 2u); + +std::vector Build(bool registerOperand) { + std::vector code{0x34020082u, 0x34060085u, 0xe0301000u, 0x80000401u, 0xbf8c3f70u, 0x7e2c0281u}; + for (std::uint32_t word = 0; word < Words; ++word) code.push_back(0x7e000280u | ((30u + word) << 17)); + std::uint32_t bit = 0; + for (const auto& constant : Constants) { + if (registerOperand) code.insert(code.end(), {0x7e2a02ffu, constant.bits}); + const std::uint32_t source = registerOperand ? 256u + 21u : constant.inlineSource; + const bool literal = !registerOperand && source == 0xffu; + for (const std::uint32_t predicate : Predicates) { + for (const bool constantFirst : {true, false}) { + if (constantFirst) { + code.push_back(0x7c000000u | (predicate << 17) | (4u << 9) | source); + } else { + code.push_back(0xd4000000u | (predicate << 16) | 106u); + code.push_back((source << 9) | (256u + 4u)); + } + if (literal) code.push_back(constant.bits); + const std::uint32_t accumulator = 30u + bit / 32u; + code.push_back((0x01u << 25) | (20u << 17) | (22u << 9) | 0x80u); + code.push_back(0xd4000000u | (0x36fu << 16) | accumulator); + code.push_back(((256u + accumulator) << 18) | ((0x80u + bit % 32u) << 9) | (256u + 20u)); + ++bit; + } + } + } + for (std::uint32_t word = 0; word < Words; ++word) code.insert(code.end(), {0xe0701000u | (word * 4u), 0x80010003u | ((30u + word) << 8)}); + code.push_back(0xbf810000u); + return code; +} + +std::uint32_t Flush(std::uint32_t bits, bool flush) { + return flush && (bits & 0x7f800000u) == 0u ? bits & 0x80000000u : bits; +} + +bool Compare(std::uint32_t predicate, std::uint32_t lhsBits, std::uint32_t rhsBits) { + const float lhs = std::bit_cast(lhsBits); + const float rhs = std::bit_cast(rhsBits); + switch (predicate) { + case 0x01u: return lhs < rhs; + case 0x02u: return lhs == rhs; + case 0x03u: return lhs <= rhs; + case 0x04u: return lhs > rhs; + case 0x05u: return lhs < rhs || lhs > rhs; + case 0x06u: return lhs >= rhs; + case 0x09u: return !(lhs >= rhs); + case 0x0au: return !(lhs < rhs || lhs > rhs); + case 0x0bu: return !(lhs > rhs); + case 0x0cu: return !(lhs <= rhs); + case 0x0du: return !(lhs == rhs); + case 0x0eu: return !(lhs < rhs); + default: throw std::runtime_error("f32 compare constant: unknown predicate"); + } +} + +std::string Hex(std::uint32_t value) { + char text[16]; + std::snprintf(text, sizeof(text), "0x%08x", value); + return text; +} + +std::array BufferDescriptor(const void* data, std::uint32_t bytes) { + const auto address = reinterpret_cast(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; +} + +void Run(AgcDriver::VulkanDevice& device, const std::optional& floatMode, bool registerOperand) { + Output.fill(0xdeadbeefu); + const auto code = Build(registerOperand); + std::vector userData(8, 0u); + const auto input = BufferDescriptor(Input.data(), static_cast(Input.size() * 4u)); + const auto output = BufferDescriptor(Output.data(), static_cast(Output.size() * 4u)); + std::copy(input.begin(), input.end(), userData.begin()); + std::copy(output.begin(), output.end(), userData.begin() + 4); + const std::span words(code); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(words)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {false, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), words, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + request.context.floatMode = floatMode; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); +} + +void Check(const std::string& mode, bool flush) { + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + const std::uint32_t value = Flush(Input[tid], flush); + std::uint32_t bit = 0; + for (const auto& constant : Constants) { + const std::uint32_t flushedConstant = Flush(constant.bits, flush); + for (std::size_t predicate = 0; predicate < Predicates.size(); ++predicate) { + for (const bool constantFirst : {true, false}) { + const bool expected = constantFirst ? Compare(Predicates[predicate], flushedConstant, value) : Compare(Predicates[predicate], value, flushedConstant); + const bool actual = ((Output[tid * Words + bit / 32u] >> (bit % 32u)) & 1u) != 0u; + Require(actual == expected, "f32 compare constant: " + mode + " lane " + std::to_string(tid) + " v_cmp_" + PredicateNames[predicate] + "_f32 " + + (constantFirst ? Hex(constant.bits) + ", " + Hex(Input[tid]) : Hex(Input[tid]) + ", " + Hex(constant.bits)) + " is " + (actual ? "1" : "0") + ", expected " + (expected ? "1" : "0")); + ++bit; + } + } + } + } +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + static_assert(Compares <= Words * 32u); + struct Mode { + const char* name; + std::optional floatMode; + bool flush; + }; + const std::array modes{{ + {"no float mode", std::nullopt, true}, + {"IEEE=0 f32 denormals flushed", ShaderRecompiler::ShaderFloatMode{0xc0u, true, false, false}, true}, + {"IEEE=1 all denormals flushed", ShaderRecompiler::ShaderFloatMode{0x00u, false, true, false}, true}, + }}; + for (const auto& mode : modes) { + for (const bool registerOperand : {true, false}) { + Run(*device, mode.floatMode, registerOperand); + Check(std::string(mode.name) + (registerOperand ? ", register operand" : ", constant operand"), mode.flush); + } + } + std::puts("f32 compare constant tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp b/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp index bf5a58d69..34e79fefe 100644 --- a/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp +++ b/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp @@ -147,6 +147,7 @@ private: void emitIntegerCompare(const RdnaInstruction& inst, IrOpcode opcode, IrType type, bool scalar, bool cmpx); void emitInteger16Compare(const RdnaInstruction& inst, IrOpcode opcode, bool signedValue, bool cmpx); void emitFloatCompare(const RdnaInstruction& inst, IrOpcode opcode, bool half, bool cmpx, bool swap = false); + IrU1 floatCompareWithConstant(IrOpcode opcode, IrU32 variableBits, std::uint32_t constant); void emitInteger64Order(const RdnaInstruction& inst, bool signedValue, bool swap, bool negate, bool cmpx); void emitFloatOrderedCompare(const RdnaInstruction& inst, bool ordered, bool half, bool cmpx); void emitFloatClassCompare(const RdnaInstruction& inst, bool cmpx); diff --git a/core/shader/recompiler/Translation/src/CompareInstructions.cpp b/core/shader/recompiler/Translation/src/CompareInstructions.cpp index 9a8766729..a96ced998 100644 --- a/core/shader/recompiler/Translation/src/CompareInstructions.cpp +++ b/core/shader/recompiler/Translation/src/CompareInstructions.cpp @@ -1,11 +1,47 @@ #include "Translation/CompareInstructions.hpp" #include "Translation/TranslationContext.hpp" #include +#include +#include #include #include namespace ShaderRecompiler { +namespace { + +std::optional constantBits(const IrValue& value) { + if (value.HasImmediate()) return value.ImmediateU32(); + const IrOpcode opcode = value.Opcode(); + if (opcode != IrOpcode::BitwiseAnd32 && opcode != IrOpcode::BitwiseOr32 && opcode != IrOpcode::BitwiseXor32) return std::nullopt; + const auto lhs = constantBits(*value.Argument(0)); + const auto rhs = constantBits(*value.Argument(1)); + if (!lhs.has_value() || !rhs.has_value()) return std::nullopt; + if (opcode == IrOpcode::BitwiseAnd32) return *lhs & *rhs; + if (opcode == IrOpcode::BitwiseOr32) return *lhs | *rhs; + return *lhs ^ *rhs; +} + +IrOpcode mirroredFloatCompare(IrOpcode opcode) { + switch (opcode) { + case IrOpcode::FPOrdLessThan32: return IrOpcode::FPOrdGreaterThan32; + case IrOpcode::FPUnordLessThan32: return IrOpcode::FPUnordGreaterThan32; + case IrOpcode::FPOrdGreaterThan32: return IrOpcode::FPOrdLessThan32; + case IrOpcode::FPUnordGreaterThan32: return IrOpcode::FPUnordLessThan32; + case IrOpcode::FPOrdLessThanEqual32: return IrOpcode::FPOrdGreaterThanEqual32; + case IrOpcode::FPUnordLessThanEqual32: return IrOpcode::FPUnordGreaterThanEqual32; + case IrOpcode::FPOrdGreaterThanEqual32: return IrOpcode::FPOrdLessThanEqual32; + case IrOpcode::FPUnordGreaterThanEqual32: return IrOpcode::FPUnordLessThanEqual32; + case IrOpcode::FPOrdEqual32: + case IrOpcode::FPUnordEqual32: + case IrOpcode::FPOrdNotEqual32: + case IrOpcode::FPUnordNotEqual32: return opcode; + default: throw std::runtime_error("f32 compare against a constant: unexpected compare opcode"); + } +} + +} + void TranslateCompareInstruction(IrBuilder& builder, const RdnaInstruction& instruction) { throw std::runtime_error("TranslateCompareInstruction not implemented"); } @@ -49,13 +85,49 @@ void TranslationContext::emitFloatCompare(const RdnaInstruction& inst, IrOpcode lhs = &readF16AsF32(sourceAt(inst, 0u)).Value(); rhs = &readF16AsF32(sourceAt(inst, 1u)).Value(); } else { - lhs = readOperand(sourceAt(inst, 0u), IrType::F32); - rhs = readOperand(sourceAt(inst, 1u), IrType::F32); + IrValue* lhsBits = readOperand(sourceAt(inst, 0u), IrType::U32); + IrValue* rhsBits = readOperand(sourceAt(inst, 1u), IrType::U32); + if (swap) std::swap(lhsBits, rhsBits); + const auto lhsConstant = constantBits(*lhsBits); + const auto rhsConstant = constantBits(*rhsBits); + if ((f32DenormalFlush & 1u) != 0u && lhsConstant.has_value() != rhsConstant.has_value()) { + const IrOpcode oriented = lhsConstant.has_value() ? mirroredFloatCompare(opcode) : opcode; + emitCompareResult(inst, floatCompareWithConstant(oriented, IrU32(lhsConstant.has_value() ? *rhsBits : *lhsBits), lhsConstant.has_value() ? *lhsConstant : *rhsConstant), false, cmpx); + return; + } + lhs = &ir.BitCastF32(flushF32Denormal(IrU32(*lhsBits)).Value()); + rhs = &ir.BitCastF32(flushF32Denormal(IrU32(*rhsBits)).Value()); + emitCompareResult(inst, IrU1(ir.Emit(opcode, IrType::U1, {lhs, rhs})), false, cmpx); + return; } if (swap) std::swap(lhs, rhs); emitCompareResult(inst, IrU1(ir.Emit(opcode, IrType::U1, {lhs, rhs})), false, cmpx); } +IrU1 TranslationContext::floatCompareWithConstant(IrOpcode opcode, IrU32 variableBits, std::uint32_t constant) { + IrValue& value = ir.BitCastF32(variableBits.Value()); + const auto compare = [&](IrOpcode compareOpcode, IrValue& lhs, std::uint32_t rhs) { return IrU1(ir.Emit(compareOpcode, IrType::U1, {&lhs, &ir.BitCastF32(ir.Constant(rhs))})); }; + if ((constant & 0x7f800000u) != 0u) return compare(opcode, value, constant); + IrValue& magnitude = ir.BitCastF32(ir.BitwiseAnd(variableBits.Value(), ir.Constant(0x7fffffffu))); + constexpr std::uint32_t MinNormal = 0x00800000u; + constexpr std::uint32_t NegativeMinNormal = 0x80800000u; + switch (opcode) { + case IrOpcode::FPOrdLessThan32: return compare(IrOpcode::FPOrdLessThanEqual32, value, NegativeMinNormal); + case IrOpcode::FPUnordLessThan32: return compare(IrOpcode::FPUnordLessThanEqual32, value, NegativeMinNormal); + case IrOpcode::FPOrdLessThanEqual32: return compare(IrOpcode::FPOrdLessThan32, value, MinNormal); + case IrOpcode::FPUnordLessThanEqual32: return compare(IrOpcode::FPUnordLessThan32, value, MinNormal); + case IrOpcode::FPOrdGreaterThan32: return compare(IrOpcode::FPOrdGreaterThanEqual32, value, MinNormal); + case IrOpcode::FPUnordGreaterThan32: return compare(IrOpcode::FPUnordGreaterThanEqual32, value, MinNormal); + case IrOpcode::FPOrdGreaterThanEqual32: return compare(IrOpcode::FPOrdGreaterThan32, value, NegativeMinNormal); + case IrOpcode::FPUnordGreaterThanEqual32: return compare(IrOpcode::FPUnordGreaterThan32, value, NegativeMinNormal); + case IrOpcode::FPOrdEqual32: return compare(IrOpcode::FPOrdLessThan32, magnitude, MinNormal); + case IrOpcode::FPUnordEqual32: return compare(IrOpcode::FPUnordLessThan32, magnitude, MinNormal); + case IrOpcode::FPOrdNotEqual32: return compare(IrOpcode::FPOrdGreaterThanEqual32, magnitude, MinNormal); + case IrOpcode::FPUnordNotEqual32: return compare(IrOpcode::FPUnordGreaterThanEqual32, magnitude, MinNormal); + default: throw std::runtime_error("f32 compare against a constant: unexpected compare opcode"); + } +} + void TranslationContext::emitInteger64Order(const RdnaInstruction& inst, bool signedValue, bool swap, bool negate, bool cmpx) { IrValue* lhs = readOperand(sourceAt(inst, swap ? 1u : 0u), IrType::U64); IrValue* rhs = readOperand(sourceAt(inst, swap ? 0u : 1u), IrType::U64); From 4a5cd6e40b6defc786ad8f99b621c5eb5e5205c3 Mon Sep 17 00:00:00 2001 From: Archit Dewan <108434204+ArchitDewan@users.noreply.github.com> Date: Fri, 9 Oct 2026 13:53:46 -0400 Subject: [PATCH 011/460] fix(ajm): reject ATRAC9 gradients beyond decoder bounds --- core/libs/prx/libSceAjm.native/CMakeLists.txt | 16 ++++++++- .../prx/libSceAjm.native/PatchLibAtrac9.cmake | 10 ++++++ .../libSceAjm.native/tests/Atrac9Gradient.c | 36 +++++++++++++++++++ 3 files changed, 61 insertions(+), 1 deletion(-) create mode 100644 core/libs/prx/libSceAjm.native/PatchLibAtrac9.cmake create mode 100644 core/libs/prx/libSceAjm.native/tests/Atrac9Gradient.c diff --git a/core/libs/prx/libSceAjm.native/CMakeLists.txt b/core/libs/prx/libSceAjm.native/CMakeLists.txt index aa4e24f83..8e784fc1f 100644 --- a/core/libs/prx/libSceAjm.native/CMakeLists.txt +++ b/core/libs/prx/libSceAjm.native/CMakeLists.txt @@ -1,6 +1,9 @@ enable_language(C) set(LIBATRAC9_DIR ${CMAKE_SOURCE_DIR}/3rdparty/LibAtrac9/C/src) +set(LIBATRAC9_PATCHED_DIR ${CMAKE_CURRENT_BINARY_DIR}/atrac9) +set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS ${LIBATRAC9_DIR}/unpack.c) +include(${CMAKE_CURRENT_SOURCE_DIR}/PatchLibAtrac9.cmake) add_library(atrac9 STATIC EXCLUDE_FROM_ALL ${LIBATRAC9_DIR}/band_extension.c ${LIBATRAC9_DIR}/bit_allocation.c @@ -13,9 +16,10 @@ add_library(atrac9 STATIC EXCLUDE_FROM_ALL ${LIBATRAC9_DIR}/quantization.c ${LIBATRAC9_DIR}/scale_factors.c ${LIBATRAC9_DIR}/tables.c - ${LIBATRAC9_DIR}/unpack.c + ${LIBATRAC9_PATCHED_DIR}/unpack.c ${LIBATRAC9_DIR}/utility.c ) +target_include_directories(atrac9 PRIVATE ${LIBATRAC9_DIR}) set_target_properties(atrac9 PROPERTIES POSITION_INDEPENDENT_CODE ON) add_library(libSceAjm.native SHARED EXCLUDE_FROM_ALL @@ -31,6 +35,16 @@ target_link_libraries(libSceAjm.native PRIVATE atrac9 ffmpeg libc libkernel) configure_windows_unwind(libSceAjm.native) if(BUILD_TESTING) + add_test_executable(atrac9_gradient_tests tests/Atrac9Gradient.c ${LIBATRAC9_DIR}/bit_reader.c) + target_include_directories(atrac9_gradient_tests PRIVATE ${LIBATRAC9_PATCHED_DIR} ${LIBATRAC9_DIR}) + target_compile_options(atrac9_gradient_tests PRIVATE -ffunction-sections -fdata-sections) + if(APPLE) + target_link_options(atrac9_gradient_tests PRIVATE LINKER:-dead_strip) + else() + target_link_options(atrac9_gradient_tests PRIVATE LINKER:--gc-sections) + endif() + add_test(NAME atrac9_gradient COMMAND atrac9_gradient_tests) + add_test_executable(guest_ajm_tests ${LIBS_INCLUDE_DIR}/tests/GuestAjm.cpp) target_include_directories(guest_ajm_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_ajm_tests PRIVATE libSceAjm.native libc) diff --git a/core/libs/prx/libSceAjm.native/PatchLibAtrac9.cmake b/core/libs/prx/libSceAjm.native/PatchLibAtrac9.cmake new file mode 100644 index 000000000..e795b3fba --- /dev/null +++ b/core/libs/prx/libSceAjm.native/PatchLibAtrac9.cmake @@ -0,0 +1,10 @@ +file(READ ${LIBATRAC9_DIR}/unpack.c libatrac9Unpack) +set(libatrac9OldGuard "block->GradientEndUnit < 0 || block->GradientEndUnit >= 48") +set(libatrac9NewGuard "block->GradientEndUnit < 0 || block->GradientEndUnit > 31") +string(FIND "${libatrac9Unpack}" "${libatrac9OldGuard}" libatrac9GuardPosition) +if(libatrac9GuardPosition EQUAL -1) + message(FATAL_ERROR "LibAtrac9 gradient guard has changed; review the local bounds fix") +endif() +string(REPLACE "${libatrac9OldGuard}" "${libatrac9NewGuard}" libatrac9Unpack "${libatrac9Unpack}") +file(MAKE_DIRECTORY ${LIBATRAC9_PATCHED_DIR}) +file(WRITE ${LIBATRAC9_PATCHED_DIR}/unpack.c "${libatrac9Unpack}") diff --git a/core/libs/prx/libSceAjm.native/tests/Atrac9Gradient.c b/core/libs/prx/libSceAjm.native/tests/Atrac9Gradient.c new file mode 100644 index 000000000..5ad7595aa --- /dev/null +++ b/core/libs/prx/libSceAjm.native/tests/Atrac9Gradient.c @@ -0,0 +1,36 @@ +#include "unpack.c" +#include + +static void WriteBits(unsigned char* data, int* position, int value, int width) +{ + for (int bit = width - 1; bit >= 0; --bit) + { + int offset = (*position)++; + data[offset / 8] |= (unsigned char)(((value >> bit) & 1) << (7 - offset % 8)); + } +} + +static At9Status GradientStatus(int endUnit) +{ + unsigned char data[4] = {0}; + int position = 0; + WriteBits(data, &position, 0, 2); + WriteBits(data, &position, 0, 6); + WriteBits(data, &position, endUnit - 1, 6); + WriteBits(data, &position, 0, 5); + WriteBits(data, &position, 0, 5); + WriteBits(data, &position, 0, 4); + + Block block = {0}; + BitReaderCxt reader; + InitBitReaderCxt(&reader, data, sizeof(data)); + return ReadGradientParams(&block, &reader); +} + +int main(void) +{ + if (GradientStatus(31) != ERR_SUCCESS) return EXIT_FAILURE; + if (GradientStatus(32) != ERR_UNPACK_GRAD_END_UNIT_OOB) return EXIT_FAILURE; + if (GradientStatus(47) != ERR_UNPACK_GRAD_END_UNIT_OOB) return EXIT_FAILURE; + return EXIT_SUCCESS; +} From 7142b6e76fdf8af2c25f2ccf9448c74ad18c1ef3 Mon Sep 17 00:00:00 2001 From: MathiasDevelopes <25400788+MathiasDevelopes@users.noreply.github.com> Date: Wed, 7 Oct 2026 21:27:28 +0200 Subject: [PATCH 012/460] feat(recompiler): convert f32 to f16 natively when the device rounds 16-bit floats to nearest even The integer emulation of the round-to-nearest-even float-to-half conversion is exact but costly: a compute shader with about 3600 conversions never finished compiling on NVIDIA. When the device enables shaderFloat16 and guarantees shaderRoundingModeRTEFloat16, shaderDenormPreserveFloat16 and shaderSignedZeroInfNanPreserveFloat16, it advertises Float16, RoundingModeRTE and DenormPreserve, and the recompiler emits OpFConvert under the 16-bit RoundingModeRTE, DenormPreserve and SignedZeroInfNanPreserve execution modes instead. NaNs keep the emulated quiet NaN with the high payload bits. Devices without these guarantees keep the emulation. Graphics validation accepts the RoundingModeRTE, DenormPreserve and SignedZeroInfNanPreserve execution modes for 16-, 32- and 64-bit floats, so vertex and pixel shaders with a native conversion still draw. Co-Authored-By: Claude Opus 5.5 (1M context) --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 13 +++ .../Execution/src/VulkanDevice.cpp | 19 ++++ .../Graphics/src/ShaderValidation.cpp | 6 +- .../tests/Float16Conversion.cpp | 97 +++++++++++++++++++ .../prx/libSceAgcDriver/tests/Graphics.cpp | 22 +++++ .../SpirvBackend/SpirvEmitterState.hpp | 1 + .../src/SpirvAlu/SpirvAluEmitterMath.cpp | 32 ++++++ 7 files changed, 188 insertions(+), 2 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/Float16Conversion.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 1d8e5eacf..d6fc123df 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -316,6 +316,19 @@ if(ANYPS5_ENABLE_SPIRV_TOOLS) configure_windows_unwind(agc_spirv_optimizer_tests) endif() +add_test_executable(agc_float16_conversion_tests EXCLUDE_FROM_ALL tests/Float16Conversion.cpp) +target_link_libraries(agc_float16_conversion_tests PRIVATE shader_recompiler_agc_driver libc) +set_target_properties(agc_float16_conversion_tests PROPERTIES CXX_EXTENSIONS OFF) +configure_windows_unwind(agc_float16_conversion_tests) +if(BUILD_TESTING) + set_target_properties(agc_float16_conversion_tests PROPERTIES EXCLUDE_FROM_ALL OFF) + add_test(NAME agc_float16_conversion COMMAND agc_float16_conversion_tests) + set_tests_properties(agc_float16_conversion PROPERTIES ENVIRONMENT "ANYPS5_NO_SHADER_CACHE=1") + if(WIN32) + set_tests_properties(agc_float16_conversion PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") + endif() +endif() + add_test_executable(agc_dpp_write_tests EXCLUDE_FROM_ALL tests/DppWrite.cpp) target_link_libraries(agc_dpp_write_tests PRIVATE shader_recompiler_agc_driver libc) set_target_properties(agc_dpp_write_tests PROPERTIES CXX_EXTENSIONS OFF) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index 96c366d80..035cc3d09 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -826,6 +826,21 @@ VulkanDevice::VulkanDevice(const PresentationWindow* window) : state(std::make_u deviceExtensions.push_back(VK_KHR_SHADER_FLOAT_CONTROLS_EXTENSION_NAME); state->capabilities.push_back(spv::CapabilitySignedZeroInfNanPreserve); state->spirvExtensions.push_back("SPV_KHR_float_controls"); + VkPhysicalDeviceShaderFloat16Int8FeaturesKHR float16Features{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SHADER_FLOAT16_INT8_FEATURES_KHR}; + if (hasExtension(VK_KHR_SHADER_FLOAT16_INT8_EXTENSION_NAME)) { + VkPhysicalDeviceFeatures2 features{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2, &float16Features}; + state->InstanceFunction("vkGetPhysicalDeviceFeatures2")(selected, &features); + } + const bool float16Rte = float16Features.shaderFloat16 == VK_TRUE && floatControls.shaderRoundingModeRTEFloat16 == VK_TRUE && + floatControls.shaderDenormPreserveFloat16 == VK_TRUE && floatControls.shaderSignedZeroInfNanPreserveFloat16 == VK_TRUE; + float16Features = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SHADER_FLOAT16_INT8_FEATURES_KHR}; + float16Features.shaderFloat16 = VK_TRUE; + if (float16Rte) { + deviceExtensions.push_back(VK_KHR_SHADER_FLOAT16_INT8_EXTENSION_NAME); + state->capabilities.push_back(spv::CapabilityFloat16); + state->capabilities.push_back(spv::CapabilityRoundingModeRTE); + state->capabilities.push_back(spv::CapabilityDenormPreserve); + } deviceExtensions.push_back(VK_KHR_BUFFER_DEVICE_ADDRESS_EXTENSION_NAME); deviceExtensions.push_back(VK_KHR_8BIT_STORAGE_EXTENSION_NAME); state->capabilities.push_back(4448); @@ -1040,6 +1055,10 @@ VulkanDevice::VulkanDevice(const PresentationWindow* window) : state(std::make_u clockFeatures.pNext = byteFeatures.pNext; byteFeatures.pNext = &clockFeatures; } + if (float16Rte) { + float16Features.pNext = byteFeatures.pNext; + byteFeatures.pNext = &float16Features; + } if (bufferInt64Atomics) { atomicInt64Features.pNext = byteFeatures.pNext; byteFeatures.pNext = &atomicInt64Features; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp index 527ef1087..576c6e95b 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp @@ -229,6 +229,8 @@ Module Inspect(const CompiledShader& compiled, const State& state, const VkPhysi capability == spv::CapabilityInt64 || capability == spv::CapabilityInt16 || capability == spv::CapabilityFloat16 || + capability == spv::CapabilityRoundingModeRTE || + capability == spv::CapabilityDenormPreserve || capability == spv::CapabilityFloat64 || capability == spv::CapabilityStorageBuffer8BitAccess || capability == spv::CapabilityPhysicalStorageBufferAddresses || @@ -302,8 +304,8 @@ Module Inspect(const CompiledShader& compiled, const State& state, const VkPhysi for (std::size_t i = 5; i < count; ++i) Require(module.interface.insert(instruction[i]).second, "duplicate SPIR-V interface ID"); break; case spv::OpExecutionMode: - if (count == 4 && instruction[2] == spv::ExecutionModeSignedZeroInfNanPreserve) { - Require(instruction[3] == 32u || instruction[3] == 64u, "SignedZeroInfNanPreserve requires Float32 or Float64"); + if (count == 4 && (instruction[2] == spv::ExecutionModeSignedZeroInfNanPreserve || instruction[2] == spv::ExecutionModeRoundingModeRTE || instruction[2] == spv::ExecutionModeDenormPreserve)) { + Require(instruction[3] == 16u || instruction[3] == 32u || instruction[3] == 64u, "float controls execution modes require Float16, Float32 or Float64"); executionModeTargets.insert(instruction[1]); break; } diff --git a/core/libs/prx/libSceAgcDriver/tests/Float16Conversion.cpp b/core/libs/prx/libSceAgcDriver/tests/Float16Conversion.cpp new file mode 100644 index 000000000..4035a5849 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/Float16Conversion.cpp @@ -0,0 +1,97 @@ +#include "Recompiler.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using namespace ShaderRecompiler; + +void require(bool condition, const std::string& message) { + if (!condition) throw std::runtime_error(message); +} + +constexpr std::array CvtF16F32Store{0x7e021500u, 0xe0700000u, 0x80000100u, 0xbf810000u}; + +std::vector recompile(std::span capabilities) { + const std::array userData{0x10000000u, 0x00100000u, 0x40u, 0x00027facu}; + const std::array extensions{"SPV_KHR_storage_buffer_storage_class"}; + RecompileRequest request{}; + request.shader = {ShaderStage::Compute, 0x20000u, CvtF16F32Store, 0, {}}; + request.context.waveSize = 64; + request.context.userDataBaseRegister = 0; + request.context.userData = userData; + request.context.compute = ShaderComputeStageInfo{{64u, 1u, 1u}, 0u, {false, false, false}, false, 1u}; + request.target.vulkanVersion = 0x00403000u; + request.target.spirvVersion = 0x00010600u; + request.target.subgroupSize = 64; + request.target.bdaAbiVersion = 1; + request.target.supportedCapabilities = capabilities; + request.target.supportedExtensions = extensions; + request.target.maxWorkgroupSize = {1024u, 1024u, 64u}; + request.target.maxWorkgroupInvocations = 1024; + request.target.maxWorkgroupSharedMemoryBytes = 49152; + request.layout = {0, 0, 0, 128}; + request.useCache = false; + auto result = Recompile(request); + require(!result.spirv.empty(), "v_cvt_f16_f32 did not recompile"); + return std::move(result.spirv); +} + +struct Usage { + std::size_t floatConverts = 0; + bool float16Capability = false; + bool roundingModeRte16 = false; + bool denormPreserve16 = false; +}; + +Usage scan(std::span words) { + Usage usage; + for (std::size_t cursor = 5; cursor < words.size();) { + const auto count = words[cursor] >> 16u; + const auto op = static_cast(words[cursor] & 0xffffu); + require(count != 0 && cursor + count <= words.size(), "truncated SPIR-V instruction"); + if (op == spv::OpFConvert) ++usage.floatConverts; + if (op == spv::OpCapability && words[cursor + 1] == spv::CapabilityFloat16) usage.float16Capability = true; + if (op == spv::OpExecutionMode && count == 4 && words[cursor + 3] == 16u) { + if (words[cursor + 2] == spv::ExecutionModeRoundingModeRTE) usage.roundingModeRte16 = true; + if (words[cursor + 2] == spv::ExecutionModeDenormPreserve) usage.denormPreserve16 = true; + } + cursor += count; + } + return usage; +} + +void verifyNative() { + const std::array capabilities{spv::CapabilityShader, spv::CapabilityGroupNonUniform, spv::CapabilityFloat16, spv::CapabilityRoundingModeRTE, spv::CapabilityDenormPreserve}; + const auto usage = scan(recompile(capabilities)); + require(usage.floatConverts != 0 && usage.float16Capability, "a device that rounds 16-bit floats to nearest even did not get OpFConvert to f16"); + require(usage.roundingModeRte16 && usage.denormPreserve16, "native f16 conversion without the 16-bit RoundingModeRTE and DenormPreserve execution modes"); +} + +void verifyEmulated() { + const std::array withoutRte{spv::CapabilityShader, spv::CapabilityGroupNonUniform, spv::CapabilityFloat16}; + const auto usage = scan(recompile(withoutRte)); + require(usage.floatConverts == 0 && !usage.float16Capability && !usage.roundingModeRte16, "a device without RoundingModeRTE for 16-bit floats got native f16 conversion"); +} + +} + +int main() { + try { + verifyNative(); + verifyEmulated(); + } catch (const std::exception& error) { + std::fprintf(stderr, "Float16Conversion: %s\n", error.what()); + return 1; + } + std::puts("Float16Conversion: native and emulated f16 conversion selected by device capabilities"); + return 0; +} diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index a035d3bf6..775731062 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -1836,6 +1836,7 @@ struct ModuleShape { bool layer = false; bool fragDepth = false; std::uint32_t sampleMaskLength = 0; + std::uint32_t floatControlsWidth = 0; }; void emit(std::vector& out, spv::Op op, std::initializer_list operands) { @@ -2022,6 +2023,9 @@ std::vector makeModule(const ModuleShape& shape) { if (shape.fragmentMode) emit(words, spv::OpExecutionMode, {main, *shape.fragmentMode}); if (shape.fragment) emit(words, spv::OpExecutionMode, {main, spv::ExecutionModeOriginUpperLeft}); if (shape.fragDepth) emit(words, spv::OpExecutionMode, {main, spv::ExecutionModeDepthReplacing}); + if (shape.floatControlsWidth != 0) { + for (const auto mode : {spv::ExecutionModeRoundingModeRTE, spv::ExecutionModeDenormPreserve, spv::ExecutionModeSignedZeroInfNanPreserve}) emit(words, spv::OpExecutionMode, {main, mode, shape.floatControlsWidth}); + } words.insert(words.end(), annotations.begin(), annotations.end()); words.insert(words.end(), declarations.begin(), declarations.end()); words.insert(words.end(), function.begin(), function.end()); @@ -2409,6 +2413,23 @@ void ConservativeRasterizationTests() { } } +void floatControlsModeTests() { + AgcDriver::Graphics::State state{}; + state.stages.path = AgcDriver::Graphics::ShaderPath::Vertex; + ShaderRecompiler::RecompileResult vertex; + ShaderRecompiler::RecompileResult pixel; + const std::array shaders{{{ShaderRecompiler::ShaderStage::Vertex, &vertex, 0}, {ShaderRecompiler::ShaderStage::Fragment, &pixel, 0}}}; + const VkPhysicalDeviceSubgroupProperties subgroup{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SUBGROUP_PROPERTIES}; + for (const auto width : {16u, 32u, 64u}) { + vertex.spirv = makeModule({.floatControlsWidth = width}); + pixel.spirv = makeModule({.fragment = true, .floatControlsWidth = width}); + AgcDriver::Graphics::ValidateShaders(shaders, state, subgroup, false); + } + vertex.spirv = makeModule({.floatControlsWidth = 8u}); + pixel.spirv = makeModule({.fragment = true}); + expectFailure([&] { AgcDriver::Graphics::ValidateShaders(shaders, state, subgroup, false); }, "float controls execution modes"); +} + void pixelParameterSlotTests() { using AgcDriver::Graphics::CompiledShader; const VkPhysicalDeviceSubgroupProperties subgroup{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SUBGROUP_PROPERTIES}; @@ -2842,6 +2863,7 @@ int main() { vertexCopyTests(); pixelParameterSlotTests(); rectListTests(); + floatControlsModeTests(); mock = MockVulkan{}; auto bdaContext = mockContext(); bdaContext.bufferDeviceAddress = true; diff --git a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp index a2498d029..ae9102cc9 100644 --- a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp +++ b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp @@ -112,6 +112,7 @@ struct SpirvEmitterState { std::uint32_t bdaAtomicPointerFunction = 0; std::uint32_t bdaNoteWriteFunction = 0; std::uint32_t bdaFaultFunction = 0; + bool nativeF16ModesEmitted = false; std::array, 2> bdaDwordReadFunctions {}; std::uint32_t bdaStopValue = 0; // False for programs with workgroup barriers: faulting BDA accesses then continue (see BdaInvocationsMayStop). diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp index ab3a31dfe..86e5d8e14 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp @@ -1131,7 +1131,39 @@ std::uint32_t EmitFAbsValue(SpirvEmitterState& state, std::uint32_t arg0) { return EmitGlsl(state, arg0); } +static bool NativeF16Rte(const SpirvEmitterState& state) { + const auto supported = [&](spv::Capability capability) { + return std::find(state.supportedCapabilities.begin(), state.supportedCapabilities.end(), static_cast(capability)) != state.supportedCapabilities.end(); + }; + return supported(spv::CapabilityFloat16) && supported(spv::CapabilityRoundingModeRTE) && supported(spv::CapabilityDenormPreserve); +} + +static std::uint32_t EmitNativeF32ToF16BitsRte(SpirvEmitterState& state, std::uint32_t value) { + state.module.EmitCapability(spv::CapabilityFloat16); + state.module.EmitCapability(spv::CapabilityRoundingModeRTE); + state.module.EmitCapability(spv::CapabilityDenormPreserve); + if (!state.nativeF16ModesEmitted) { + state.module.AddExecutionMode(state.mainFunc, spv::ExecutionModeRoundingModeRTE, 16u); + state.module.AddExecutionMode(state.mainFunc, spv::ExecutionModeDenormPreserve, 16u); + state.module.AddExecutionMode(state.mainFunc, spv::ExecutionModeSignedZeroInfNanPreserve, 16u); + state.nativeF16ModesEmitted = true; + } + const auto u32 = TypeU32(state); + const auto f16 = state.module.Type(spv::OpTypeFloat, 16u); + const auto half = Unary(state, spv::OpFConvert, f16, value); + const auto pair = state.module.AllocateId(); + state.module.AddFunction(spv::OpCompositeConstruct, state.module.Type(spv::OpTypeVector, f16, 2u), pair, half, state.module.Constant(spv::OpConstant, f16, 0u)); + const auto bits = Unary(state, spv::OpBitcast, u32, pair); + const auto source = Unary(state, spv::OpBitcast, u32, value); + const auto magnitude = Binary(state, spv::OpBitwiseAnd, u32, source, ConstantU32(state, 0x7fffffffu)); + const auto sign = Binary(state, spv::OpBitwiseAnd, u32, Binary(state, spv::OpShiftRightLogical, u32, source, ConstantU32(state, 16u)), ConstantU32(state, 0x8000u)); + const auto payload = Binary(state, spv::OpShiftRightLogical, u32, Binary(state, spv::OpBitwiseAnd, u32, magnitude, ConstantU32(state, 0x7fffffu)), ConstantU32(state, 13u)); + const auto nan = Binary(state, spv::OpBitwiseOr, u32, sign, Binary(state, spv::OpBitwiseOr, u32, ConstantU32(state, 0x7e00u), payload)); + return Select(state, u32, Binary(state, spv::OpUGreaterThan, TypeBool(state), magnitude, ConstantU32(state, 0x7f800000u)), nan, bits); +} + std::uint32_t EmitF32ToF16BitsRte(SpirvEmitterState& state, std::uint32_t value) { + if (NativeF16Rte(state)) return EmitNativeF32ToF16BitsRte(state, value); const auto u32 = TypeU32(state); const auto boolean = TypeBool(state); const auto constant = [&](std::uint32_t bits) { return ConstantU32(state, bits); }; From 8a578128b2d634ff239a1d2444f3253bacf48bcb Mon Sep 17 00:00:00 2001 From: Karan Vora <52519664+karanvora2599@users.noreply.github.com> Date: Fri, 9 Oct 2026 14:11:02 -0400 Subject: [PATCH 013/460] perf(shader): convert v_cvt_pkrtz_f16_f32 from the native nearest-even conversion With the 16-bit RoundingModeRTE, DenormPreserve and SignedZeroInfNanPreserve guarantees, convert each value to the nearest half with OpFConvert, convert the half back to f32 and, when it is further from zero than the source, step its bits one toward zero. That is the toward-zero half for every finite value, including subnormal results and 65504 for values from 65520 up; infinities and the quiet NaN with the high payload bits are kept. Other devices keep the integer conversion. New test agc_driver_f16_pack_toward_zero checks every half, its f32 neighbours and the midpoints to the next half, with both signs. --- .../tests/execution/F16PackTowardZero.cpp | 117 ++++++++++++++++++ .../src/SpirvAlu/SpirvAluEmitterMath.cpp | 18 ++- 2 files changed, 133 insertions(+), 2 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/F16PackTowardZero.cpp diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/F16PackTowardZero.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/F16PackTowardZero.cpp new file mode 100644 index 000000000..b07d5ca68 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/F16PackTowardZero.cpp @@ -0,0 +1,117 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 32; +constexpr std::uint32_t Capacity = 1u << 18; +alignas(256) std::array Input{}; +alignas(256) std::array Output{}; + +std::uint32_t HalfToFloatBits(std::uint32_t half) { + const std::uint32_t exponent = (half >> 10u) & 0x1fu; + const std::uint32_t mantissa = half & 0x3ffu; + if (exponent != 0u) return ((exponent + 112u) << 23u) | (mantissa << 13u); + if (mantissa == 0u) return 0u; + std::uint32_t shift = 0; + while ((mantissa << shift & 0x400u) == 0u) ++shift; + return ((113u - shift) << 23u) | (((mantissa << shift) & 0x3ffu) << 13u); +} + +std::vector Inputs() { + std::vector inputs{ + 0x00000000u, 0x00000001u, 0x007fffffu, 0x00800000u, 0x33000000u, 0x33000001u, 0x337fffffu, 0x33800000u, + 0x387fffffu, 0x38800000u, 0x477fe000u, 0x477fefffu, 0x477ff000u, 0x477fffffu, 0x47800000u, 0x7f7fffffu, + 0x7f800000u, 0x7fc00000u, 0x7f800001u, 0x7fffe000u, 0x7fbfffffu, 0x7f802000u, + }; + for (std::uint32_t half = 0; half < 0x7c00u; ++half) { + const std::uint32_t exact = HalfToFloatBits(half); + const std::uint32_t next = HalfToFloatBits(half + 1u); + const std::uint32_t middle = exact + (next - exact) / 2u; + for (const std::uint32_t value : {exact, exact + 1u, middle - 1u, middle, middle + 1u}) inputs.push_back(value); + } + while (inputs.size() % Threads != 0u) inputs.push_back(0u); + if (inputs.size() > Capacity) throw std::runtime_error("f16 pack toward zero: too many inputs"); + return inputs; +} + +std::uint32_t TowardZero(std::uint32_t bits) { + if ((bits & 0x7f800000u) == 0u) bits &= 0x80000000u; + const std::uint32_t sign = (bits >> 16u) & 0x8000u; + const std::uint32_t exponent = (bits >> 23u) & 0xffu; + const std::uint32_t mantissa = bits & 0x7fffffu; + if (exponent == 0xffu) return mantissa == 0u ? sign | 0x7c00u : sign | 0x7e00u | (mantissa >> 13u); + if (exponent >= 143u) return sign | 0x7bffu; + if (exponent >= 113u) return sign | ((exponent - 112u) << 10u) | (mantissa >> 13u); + if (exponent >= 103u) return sign | ((mantissa | 0x800000u) >> (126u - exponent)); + return sign; +} + +std::array BufferDescriptor(const void* data, std::uint32_t bytes) { + const auto address = reinterpret_cast(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; +} + +std::string Hex(std::uint32_t value) { + char text[16]; + std::snprintf(text, sizeof(text), "0x%08x", value); + return text; +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + const auto inputs = Inputs(); + std::copy(inputs.begin(), inputs.end(), Input.begin()); + Output.fill(0xdeadbeefu); + const std::vector code{ + 0x34020082u, 0x8f098708u, 0x4a020209u, 0xe0301000u, 0x80000401u, 0xbf8c3f70u, + 0xd52f0005u, 0x40020904u, 0xe0701000u, 0x80010501u, 0xbf810000u, + }; + std::vector userData(8, 0u); + const auto input = BufferDescriptor(Input.data(), static_cast(Input.size() * 4u)); + const auto output = BufferDescriptor(Output.data(), static_cast(Output.size() * 4u)); + std::copy(input.begin(), input.end(), userData.begin()); + std::copy(output.begin(), output.end(), userData.begin() + 4); + const std::span words(code); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(words)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {true, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), words, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device->Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + request.context.floatMode = ShaderRecompiler::ShaderFloatMode{0xc0u, true, false, false}; + const auto result = ShaderRecompiler::Recompile(request); + device->Dispatch(result, static_cast(inputs.size() / Threads), 1, 1, {}, reinterpret_cast(code.data())); + device->WaitIdle(); + for (std::size_t index = 0; index < inputs.size(); ++index) { + const std::uint32_t source = inputs[index]; + const std::uint32_t expected = TowardZero(source) | (TowardZero(source ^ 0x80000000u) << 16u); + Require(Output[index] == expected, "f16 pack toward zero: v_cvt_pkrtz_f16_f32 of " + Hex(source) + ", -" + Hex(source) + " is " + Hex(Output[index]) + ", expected " + Hex(expected)); + } + std::printf("f16 pack toward zero tests passed (%zu inputs)\n", inputs.size()); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp index 86e5d8e14..31472ab81 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp @@ -459,9 +459,23 @@ std::uint32_t EmitCompositeExtractU64(SpirvEmitterState& state, std::uint32_t ar return EmitNative(state, arg0, arg1->ImmediateU32()); } +static bool NativeF16Rte(const SpirvEmitterState& state); + +static std::uint32_t EmitF32ToF16BitsTowardZero(SpirvEmitterState& state, std::uint32_t value) { + if (!NativeF16Rte(state)) return EmitF32ToF16RtzBits(state, value); + const auto u32 = TypeU32(state); + const auto f16 = state.module.Type(spv::OpTypeFloat, 16u); + const auto nearest = EmitF32ToF16BitsRte(state, value); + const auto half = state.module.AllocateId(); + state.module.AddFunction(spv::OpCompositeExtract, f16, half, Unary(state, spv::OpBitcast, state.module.Type(spv::OpTypeVector, f16, 2u), nearest), 0u); + const auto widened = Unary(state, spv::OpFConvert, TypeF32(state), half); + const auto away = Binary(state, spv::OpFOrdGreaterThan, TypeBool(state), EmitFAbsValue(state, widened), EmitFAbsValue(state, value)); + return Select(state, u32, away, Binary(state, spv::OpISub, u32, nearest, ConstantU32(state, 1u)), nearest); +} + std::uint32_t EmitPackFloat2x16Rtz(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1) { - const auto low = EmitF32ToF16RtzBits(state, arg0); - const auto high = Binary(state, spv::OpShiftLeftLogical, TypeU32(state), EmitF32ToF16RtzBits(state, arg1), ConstantU32(state, 16u)); + const auto low = EmitF32ToF16BitsTowardZero(state, arg0); + const auto high = Binary(state, spv::OpShiftLeftLogical, TypeU32(state), EmitF32ToF16BitsTowardZero(state, arg1), ConstantU32(state, 16u)); return Binary(state, spv::OpBitwiseOr, TypeU32(state), low, high); } From 719f003adfc21d4574f8d9edd858e293de9718ac Mon Sep 17 00:00:00 2001 From: Karan Vora <52519664+karanvora2599@users.noreply.github.com> Date: Fri, 9 Oct 2026 15:29:37 -0400 Subject: [PATCH 014/460] perf(shader): compute v_dot2_f32_f16 in f32 outside the special cases v_dot2_f32_f16 rounded a.x * b.x + a.y * b.y + c once, computed with f64 TwoSums and round-to-odd. GeForce GPUs run f64 at 1/32 or 1/64 of the f32 rate, which made it cost about 44 v_fma_f32 on an RTX 4060. The f16 products are exact in f32, and the correctly rounded sum of three f32 values only needs f32: two TwoSums, a round-to-odd sum of their errors and a final add (Boldo and Melquiond, "Emulation of FMA and Correctly Rounded Sums: Proved Algorithms Using Rounding to Odd", IEEE TC 57(4), 2008), with round-to-odd emulated by stepping an even rounded sum one ulp toward its error. With finite inputs, c zero or at least 2^-100 and a result between 2^-100 and 2^127, every value involved is zero or a multiple of 2^-123, so no driver flush can reach it. Other inputs, and zero results, take the f64 lowering in a branch. New test agc_driver_f32_dot2_rounding checks random, cancelling and tie cases against an exact fixed-point sum, and the NaN, infinity and zero rules. --- .../tests/execution/F32Dot2Rounding.cpp | 274 ++++++++++++++++++ .../SpirvBackend/SpirvEmitterHelpers.hpp | 1 + .../SpirvBackend/SpirvEmitterState.hpp | 1 + .../src/SpirvAlu/SpirvAluEmitterFloat64.cpp | 90 +++++- .../SpirvBackend/src/SpirvModuleEmitter.cpp | 1 + 5 files changed, 366 insertions(+), 1 deletion(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/F32Dot2Rounding.cpp diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/F32Dot2Rounding.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/F32Dot2Rounding.cpp new file mode 100644 index 000000000..26ac818ca --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/F32Dot2Rounding.cpp @@ -0,0 +1,274 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 32; +constexpr std::uint32_t Capacity = 1u << 18; +alignas(256) std::array Input{}; +alignas(256) std::array Output{}; + +struct Case { + std::uint32_t a; + std::uint32_t b; + std::uint32_t c; + std::uint32_t expected; +}; + +class Exact { +public: + void Add(std::uint64_t magnitude, std::uint32_t shift, bool negative) { + std::array term{}; + for (std::uint32_t bit = 0; bit < 64u; ++bit) { + if (((magnitude >> bit) & 1u) == 0u) continue; + const std::uint32_t position = bit + shift; + term[position / 32u] |= 1u << (position % 32u); + } + if (negative) { + for (auto& limb : term) limb = ~limb; + Accumulate(term, 1u); + } else { + Accumulate(term, 0u); + } + } + + bool Zero() const { return std::all_of(limbs.begin(), limbs.end(), [](std::uint32_t limb) { return limb == 0u; }); } + + std::optional Rounded() const { + auto magnitude = limbs; + const bool negative = (magnitude[Limbs - 1u] >> 31u) != 0u; + if (negative) { + for (auto& limb : magnitude) limb = ~limb; + std::uint64_t carry = 1u; + for (auto& limb : magnitude) { + const std::uint64_t sum = std::uint64_t{limb} + carry; + limb = static_cast(sum); + carry = sum >> 32u; + } + } + int top = -1; + for (int bit = static_cast(Limbs * 32u) - 1; bit >= 0; --bit) { + if (Bit(magnitude, static_cast(bit))) { + top = bit; + break; + } + } + if (top < 23) return std::nullopt; + std::uint64_t mantissa = 0; + for (int bit = top; bit > top - 24; --bit) mantissa = (mantissa << 1u) | (Bit(magnitude, static_cast(bit)) ? 1u : 0u); + const int guardBit = top - 24; + const bool guard = guardBit >= 0 && Bit(magnitude, static_cast(guardBit)); + bool sticky = false; + for (int bit = guardBit - 1; bit >= 0 && !sticky; --bit) sticky = Bit(magnitude, static_cast(bit)); + int exponent = top - 22; + if (guard && (sticky || (mantissa & 1u) != 0u)) { + if (++mantissa == (1u << 24u)) { + mantissa >>= 1u; + ++exponent; + } + } + if (exponent < 1 || exponent > 254) return std::nullopt; + return (negative ? 0x80000000u : 0u) | (static_cast(exponent) << 23u) | static_cast(mantissa & 0x7fffffu); + } + +private: + static constexpr std::uint32_t Limbs = 10; + + static bool Bit(const std::array& value, std::uint32_t bit) { return ((value[bit / 32u] >> (bit % 32u)) & 1u) != 0u; } + + void Accumulate(const std::array& term, std::uint64_t carry) { + for (std::uint32_t index = 0; index < Limbs; ++index) { + const std::uint64_t sum = std::uint64_t{limbs[index]} + term[index] + carry; + limbs[index] = static_cast(sum); + carry = sum >> 32u; + } + } + + std::array limbs{}; +}; + +struct Half { + std::uint32_t bits; + bool nan; + bool infinite; + bool zero; + bool negative; + std::uint64_t significand; + std::uint32_t scale; +}; + +Half Decode(std::uint32_t packed, std::uint32_t shift) { + const std::uint32_t bits = (packed >> shift) & 0xffffu; + const std::uint32_t magnitude = bits & 0x7fffu; + const std::uint32_t exponent = magnitude >> 10u; + const std::uint32_t fraction = magnitude & 0x3ffu; + Half half{bits, magnitude > 0x7c00u, magnitude == 0x7c00u, magnitude == 0u, (bits & 0x8000u) != 0u, 0u, 0u}; + half.significand = exponent == 0u ? fraction : (fraction | 0x400u); + half.scale = exponent == 0u ? 0u : exponent - 1u; + return half; +} + +std::optional Dot2(std::uint32_t a, std::uint32_t b, std::uint32_t c) { + const Half aLow = Decode(a, 0u); + const Half bLow = Decode(b, 0u); + const Half aHigh = Decode(a, 16u); + const Half bHigh = Decode(b, 16u); + const auto quiet = [](const Half& half) { return (half.negative ? 0x80000000u : 0u) | 0x7fc00000u | ((half.bits & 0x3ffu) << 13u); }; + for (const Half* half : {&aLow, &bLow, &aHigh, &bHigh}) { + if (half->nan) return quiet(*half); + } + const std::uint32_t magnitudeC = c & 0x7fffffffu; + if (magnitudeC > 0x7f800000u) return c | 0x00400000u; + const bool negativeC = (c & 0x80000000u) != 0u; + const bool infiniteC = magnitudeC == 0x7f800000u; + const bool signLow = aLow.negative != bLow.negative; + const bool signHigh = aHigh.negative != bHigh.negative; + const auto invalidProduct = [](const Half& lhs, const Half& rhs) { return (lhs.infinite && rhs.zero) || (rhs.infinite && lhs.zero); }; + const auto infiniteProduct = [](const Half& lhs, const Half& rhs) { return (lhs.infinite && !rhs.zero) || (rhs.infinite && !lhs.zero); }; + const bool infiniteLow = infiniteProduct(aLow, bLow); + const bool infiniteHigh = infiniteProduct(aHigh, bHigh); + const bool invalid = invalidProduct(aLow, bLow) || invalidProduct(aHigh, bHigh) || (infiniteLow && infiniteHigh && signLow != signHigh) || + (infiniteLow && infiniteC && signLow != negativeC) || (infiniteHigh && infiniteC && signHigh != negativeC); + if (invalid) return 0xffc00000u; + if (infiniteLow || infiniteHigh || infiniteC) return ((infiniteLow ? signLow : infiniteHigh ? signHigh : negativeC) ? 0x80000000u : 0u) | 0x7f800000u; + Exact sum; + sum.Add(aLow.significand * bLow.significand, aLow.scale + bLow.scale + 101u, signLow); + sum.Add(aHigh.significand * bHigh.significand, aHigh.scale + bHigh.scale + 101u, signHigh); + if (magnitudeC >= 0x00800000u) sum.Add((c & 0x7fffffu) | 0x800000u, (magnitudeC >> 23u) - 1u, negativeC); + if (sum.Zero()) return (signLow && signHigh && negativeC) ? 0x80000000u : 0u; + const auto rounded = sum.Rounded(); + if (!rounded.has_value() || ((*rounded >> 23u) & 0xffu) < 27u || ((*rounded >> 23u) & 0xffu) >= 254u) return std::nullopt; + return rounded; +} + +std::uint32_t RandomHalf(std::mt19937& random) { + const std::uint32_t exponent = random() % 31u; + return ((random() & 1u) << 15u) | (exponent << 10u) | (random() & 0x3ffu); +} + +std::uint32_t RandomF32(std::mt19937& random, std::uint32_t lowExponent, std::uint32_t highExponent) { + const std::uint32_t exponent = lowExponent + random() % (highExponent - lowExponent + 1u); + return ((random() & 1u) << 31u) | (exponent << 23u) | (random() & 0x7fffffu); +} + +std::uint32_t HalfToF32(std::uint32_t half) { + const Half decoded = Decode(half, 0u); + if (decoded.zero) return decoded.negative ? 0x80000000u : 0u; + std::uint64_t significand = decoded.significand; + std::uint32_t exponent = decoded.scale + 103u; + while ((significand & 0x400u) == 0u) { + significand <<= 1u; + --exponent; + } + return (decoded.negative ? 0x80000000u : 0u) | (exponent << 23u) | static_cast((significand & 0x3ffu) << 13u); +} + +std::vector Cases() { + std::vector cases; + const auto add = [&](std::uint32_t a, std::uint32_t b, std::uint32_t c) { + const auto expected = Dot2(a, b, c); + if (expected.has_value()) cases.push_back({a, b, c, *expected}); + }; + std::mt19937 random(2026u); + for (std::uint32_t index = 0; index < 60000u; ++index) add(RandomHalf(random) | (RandomHalf(random) << 16u), RandomHalf(random) | (RandomHalf(random) << 16u), RandomF32(random, 60u, 170u)); + for (std::uint32_t index = 0; index < 30000u; ++index) add(RandomHalf(random) | (RandomHalf(random) << 16u), RandomHalf(random) | (RandomHalf(random) << 16u), 0u); + for (std::uint32_t index = 0; index < 60000u; ++index) { + const std::uint32_t a = RandomHalf(random) | (RandomHalf(random) << 16u); + const std::uint32_t b = RandomHalf(random) | (RandomHalf(random) << 16u); + const float low = std::bit_cast(HalfToF32(a & 0xffffu)) * std::bit_cast(HalfToF32(b & 0xffffu)); + const float high = std::bit_cast(HalfToF32(a >> 16u)) * std::bit_cast(HalfToF32(b >> 16u)); + const std::uint32_t pick = random() % 4u; + const float base = pick == 0u ? -low : pick == 1u ? -high : -(low + high); + const std::uint32_t nudge = static_cast(static_cast(random() % 9u) - 4); + add(a, b, std::bit_cast(base) + nudge); + } + const std::array powers{0x3c00u, 0x4000u, 0x3800u, 0x0400u, 0x0001u, 0x7bffu, 0x3c01u, 0x3bffu}; + for (const std::uint32_t lowA : powers) for (const std::uint32_t highA : powers) for (const std::uint32_t lowB : powers) for (const std::uint32_t highB : powers) { + for (const std::uint32_t c : {0x00000000u, 0x3f800000u, 0xbf800000u, 0x33800000u, 0x4b800000u, 0x3f800001u, 0xcb7fffffu, 0x0d800000u, 0x00800000u, 0x8c7fffffu}) { + add(lowA | (highA << 16u), lowB | (highB << 16u), c); + add(lowA | (highA << 16u), (lowB ^ 0x8000u) | (highB << 16u), c); + } + } + const std::array specials{0x7c00u, 0xfc00u, 0x7e00u, 0x7c01u, 0xfe55u, 0x0000u, 0x8000u, 0x3c00u, 0xbc00u, 0x0001u}; + for (const std::uint32_t lowA : specials) for (const std::uint32_t highA : specials) for (const std::uint32_t lowB : specials) { + for (const std::uint32_t c : {0x00000000u, 0x80000000u, 0x7f800000u, 0xff800000u, 0x7fc00000u, 0x7f800123u, 0x3f800000u, 0x00000001u}) add(lowA | (highA << 16u), lowB | (0x3c00u << 16u), c); + } + while (cases.size() % Threads != 0u) cases.push_back({0u, 0u, 0x3f800000u, 0x3f800000u}); + if (cases.size() > Capacity) throw std::runtime_error("f32 dot2 rounding: too many cases"); + return cases; +} + +std::array BufferDescriptor(const void* data, std::uint32_t bytes) { + const auto address = reinterpret_cast(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; +} + +std::string Hex(std::uint32_t value) { + char text[16]; + std::snprintf(text, sizeof(text), "0x%08x", value); + return text; +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + const auto cases = Cases(); + for (std::size_t index = 0; index < cases.size(); ++index) { + Input[index * 4u] = cases[index].a; + Input[index * 4u + 1u] = cases[index].b; + Input[index * 4u + 2u] = cases[index].c; + } + Output.fill(0xdeadbeefu); + const std::vector code{ + 0x34020084u, 0x8f098908u, 0x4a020209u, 0xe0381000u, 0x80000401u, 0xbf8c3f70u, + 0xcc134008u, 0x1c1a0b04u, 0x2c060282u, 0xe0701000u, 0x80010803u, 0xbf810000u, + }; + std::vector userData(8, 0u); + const auto input = BufferDescriptor(Input.data(), static_cast(Input.size() * 4u)); + const auto output = BufferDescriptor(Output.data(), static_cast(Output.size() * 4u)); + std::copy(input.begin(), input.end(), userData.begin()); + std::copy(output.begin(), output.end(), userData.begin() + 4); + const std::span words(code); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(words)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {true, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), words, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device->Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + request.context.floatMode = ShaderRecompiler::ShaderFloatMode{0xc0u, true, false, false}; + const auto result = ShaderRecompiler::Recompile(request); + device->Dispatch(result, static_cast(cases.size() / Threads), 1, 1, {}, reinterpret_cast(code.data())); + device->WaitIdle(); + for (std::size_t index = 0; index < cases.size(); ++index) { + const auto& entry = cases[index]; + Require(Output[index] == entry.expected, "f32 dot2 rounding: v_dot2_f32_f16 " + Hex(entry.a) + ", " + Hex(entry.b) + ", " + Hex(entry.c) + " is " + Hex(Output[index]) + ", expected " + Hex(entry.expected)); + } + std::printf("f32 dot2 rounding tests passed (%zu cases)\n", cases.size()); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterHelpers.hpp b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterHelpers.hpp index 07af37ad0..18155e060 100644 --- a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterHelpers.hpp +++ b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterHelpers.hpp @@ -133,6 +133,7 @@ std::uint32_t EmitTrigCycleF32(SpirvEmitterState& state, std::uint32_t src, bool std::uint32_t EmitF16BitsToF32(SpirvEmitterState& state, std::uint32_t bits); void EmitProgram(SpirvEmitterState& state); void DefineGetBdaPointer(SpirvEmitterState& state); +void DefineDot2F32F16Function(SpirvEmitterState& state); void EmitLabel(SpirvEmitterState& state, std::uint32_t label); std::uint32_t Unary(SpirvEmitterState& state, std::uint32_t opcode, std::uint32_t type, std::uint32_t value); std::uint32_t Binary(SpirvEmitterState& state, std::uint32_t opcode, std::uint32_t type, std::uint32_t lhs, std::uint32_t rhs); diff --git a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp index a2498d029..6eb5e53f3 100644 --- a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp +++ b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp @@ -112,6 +112,7 @@ struct SpirvEmitterState { std::uint32_t bdaAtomicPointerFunction = 0; std::uint32_t bdaNoteWriteFunction = 0; std::uint32_t bdaFaultFunction = 0; + std::uint32_t dot2F32F16Function = 0; std::array, 2> bdaDwordReadFunctions {}; std::uint32_t bdaStopValue = 0; // False for programs with workgroup barriers: faulting BDA accesses then continue (see BdaInvocationsMayStop). diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterFloat64.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterFloat64.cpp index 95c2d7450..d6434d2df 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterFloat64.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterFloat64.cpp @@ -534,7 +534,9 @@ std::uint32_t EmitFPTrigPreop64(SpirvEmitterState& state, std::uint32_t arg0, st return FromF64(state, Exact(state, spv::OpFMul, Exact(state, spv::OpFMul, mantissa, scale), ConstantF64(state, 0x1a70000000000000ull))); } -std::uint32_t EmitFPDot2F32F16(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1, std::uint32_t arg2) { +namespace { + +std::uint32_t Dot2F32F16Exact(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1, std::uint32_t arg2) { const auto u32 = TypeU32(state); const auto op = [&](spv::Op opcode, std::uint32_t lhs, std::uint32_t value) { return Binary(state, opcode, u32, lhs, ConstantU32(state, value)); }; const auto test = [&](spv::Op opcode, std::uint32_t lhs, std::uint32_t value) { return Binary(state, opcode, TypeBool(state), lhs, ConstantU32(state, value)); }; @@ -624,6 +626,92 @@ std::uint32_t EmitFPDot2F32F16(SpirvEmitterState& state, std::uint32_t arg0, std return pick(aLow.nan, quietHalf(aLow), result); } + +std::uint32_t ExactF32(SpirvEmitterState& state, std::uint32_t opcode, std::uint32_t lhs, std::uint32_t rhs) { + const auto result = Binary(state, opcode, TypeF32(state), lhs, rhs); + state.module.AddAnnotation(spv::OpDecorate, result, spv::DecorationNoContraction); + return result; +} + +std::array TwoSumF32(SpirvEmitterState& state, std::uint32_t lhs, std::uint32_t rhs) { + const auto sum = ExactF32(state, spv::OpFAdd, lhs, rhs); + const auto rhsPart = ExactF32(state, spv::OpFSub, sum, lhs); + const auto lhsPart = ExactF32(state, spv::OpFSub, sum, rhsPart); + return {sum, ExactF32(state, spv::OpFAdd, ExactF32(state, spv::OpFSub, lhs, lhsPart), ExactF32(state, spv::OpFSub, rhs, rhsPart))}; +} + +std::uint32_t Dot2F32F16Fast(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1, std::uint32_t arg2, std::uint32_t& fast) { + const auto u32 = TypeU32(state); + const auto f32 = TypeF32(state); + const auto boolean = TypeBool(state); + const auto op = [&](spv::Op opcode, std::uint32_t lhs, std::uint32_t value) { return Binary(state, opcode, u32, lhs, ConstantU32(state, value)); }; + const auto test = [&](spv::Op opcode, std::uint32_t lhs, std::uint32_t value) { return Binary(state, opcode, boolean, lhs, ConstantU32(state, value)); }; + const auto all = [&](std::uint32_t lhs, std::uint32_t rhs) { return Binary(state, spv::OpLogicalAnd, boolean, lhs, rhs); }; + const auto half = [&](std::uint32_t packed, std::uint32_t shift, std::uint32_t& finite) { + const auto bits = op(spv::OpBitwiseAnd, op(spv::OpShiftRightLogical, packed, shift), 0xffffu); + const auto magnitude = op(spv::OpBitwiseAnd, bits, 0x7fffu); + const auto sign = op(spv::OpShiftLeftLogical, op(spv::OpBitwiseAnd, bits, 0x8000u), 16u); + const auto normal = op(spv::OpIAdd, op(spv::OpShiftLeftLogical, magnitude, 13u), 112u << 23u); + const auto subnormal = Unary(state, spv::OpBitcast, u32, ExactF32(state, spv::OpFMul, Unary(state, spv::OpConvertUToF, f32, magnitude), ConstantF32(state, 0x33800000u))); + const auto unsignedBits = Select(state, u32, test(spv::OpULessThan, magnitude, 0x400u), subnormal, normal); + finite = all(finite, test(spv::OpULessThan, magnitude, 0x7c00u)); + return Unary(state, spv::OpBitcast, f32, Binary(state, spv::OpBitwiseOr, u32, unsignedBits, sign)); + }; + const auto magnitudeC = op(spv::OpBitwiseAnd, arg2, 0x7fffffffu); + const auto flushedC = test(spv::OpULessThan, magnitudeC, 0x00800000u); + fast = all(test(spv::OpULessThan, magnitudeC, 0x7f800000u), Binary(state, spv::OpLogicalOr, boolean, flushedC, test(spv::OpUGreaterThanEqual, magnitudeC, 0x0d800000u))); + const auto valueC = Unary(state, spv::OpBitcast, f32, Select(state, u32, flushedC, op(spv::OpBitwiseAnd, arg2, 0x80000000u), arg2)); + const auto productLow = ExactF32(state, spv::OpFMul, half(arg0, 0u, fast), half(arg1, 0u, fast)); + const auto productHigh = ExactF32(state, spv::OpFMul, half(arg0, 16u, fast), half(arg1, 16u, fast)); + const auto upper = TwoSumF32(state, productHigh, valueC); + const auto total = TwoSumF32(state, productLow, upper[0]); + const auto errors = TwoSumF32(state, total[1], upper[1]); + const auto nearestBits = Unary(state, spv::OpBitcast, u32, errors[0]); + const auto errorBits = Unary(state, spv::OpBitcast, u32, errors[1]); + const auto inexact = test(spv::OpINotEqual, op(spv::OpBitwiseAnd, errorBits, 0x7fffffffu), 0u); + const auto even = test(spv::OpIEqual, op(spv::OpBitwiseAnd, nearestBits, 1u), 0u); + const auto sameSign = test(spv::OpIEqual, op(spv::OpBitwiseAnd, Binary(state, spv::OpBitwiseXor, u32, nearestBits, errorBits), 0x80000000u), 0u); + const auto odd = Binary(state, spv::OpIAdd, u32, nearestBits, Select(state, u32, sameSign, ConstantU32(state, 1u), ConstantU32(state, 0xffffffffu))); + const auto roundedToOdd = Unary(state, spv::OpBitcast, f32, Select(state, u32, all(inexact, even), odd, nearestBits)); + const auto result = Unary(state, spv::OpBitcast, u32, ExactF32(state, spv::OpFAdd, total[0], roundedToOdd)); + const auto magnitude = op(spv::OpBitwiseAnd, result, 0x7fffffffu); + fast = all(fast, all(test(spv::OpUGreaterThanEqual, magnitude, 0x0d800000u), test(spv::OpULessThan, magnitude, 0x7f000000u))); + return result; +} + +} + +void DefineDot2F32F16Function(SpirvEmitterState& state) { + bool used = false; + for (const IrBlock* block : state.program.BlockOrder()) { + for (const IrValue* inst : block->Instructions()) used = used || inst->Opcode() == IrOpcode::FPDot2F32F16; + } + if (!used) return; + const auto u32 = TypeU32(state); + state.dot2F32F16Function = state.module.AllocateId(); + state.module.AddName(state.dot2F32F16Function, "dot2_f32_f16_exact"); + state.module.AddFunction(spv::OpFunction, u32, state.dot2F32F16Function, spv::FunctionControlDontInlineMask, state.module.Type(spv::OpTypeFunction, u32, u32, u32, u32)); + std::array parameters{}; + for (auto& parameter : parameters) { + parameter = state.module.AllocateId(); + state.module.AddFunction(spv::OpFunctionParameter, u32, parameter); + } + EmitLabel(state, state.module.AllocateId()); + state.module.AddFunction(spv::OpReturnValue, Dot2F32F16Exact(state, parameters[0], parameters[1], parameters[2])); + state.module.AddFunction(spv::OpFunctionEnd); +} + +std::uint32_t EmitFPDot2F32F16(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1, std::uint32_t arg2) { + if (state.dot2F32F16Function == 0u) FailEmit("v_dot2_f32_f16 function was not defined before function emission"); + std::uint32_t fast = 0; + const auto fastResult = Dot2F32F16Fast(state, arg0, arg1, arg2, fast); + return EmitValueOrDefaultIfCondition(state, Unary(state, spv::OpLogicalNot, TypeBool(state), fast), TypeU32(state), fastResult, [&] { + const auto result = state.module.AllocateId(); + state.module.AddFunction(spv::OpFunctionCall, TypeU32(state), result, state.dot2F32F16Function, arg0, arg1, arg2); + return result; + }); +} + std::uint32_t EmitFPInterpolateF32(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1, std::uint32_t arg2, const IrValue* mode) { return Interpolate(state, arg0, arg1, arg2, false, mode->ImmediateU32()); } diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp index 4fb0e39aa..659007e77 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp @@ -918,6 +918,7 @@ void EmitProgram(SpirvEmitterState& state) { state.labels.emplace(block, label); } DefineGetBdaPointer(state); + DefineDot2F32F16Function(state); for (const IrBlock* block : program.BlockOrder()) { const bool needsScratch = std::any_of(block->Instructions().begin(), block->Instructions().end(), [](const IrValue* inst) { return inst->Opcode() == IrOpcode::SwizzleU32; From 7accd92281fedaa043211dda02ffab2a26091ce4 Mon Sep 17 00:00:00 2001 From: hazzardouslysolid <219063691+hazzardouslysolid@users.noreply.github.com> Date: Sat, 10 Oct 2026 04:06:11 +0800 Subject: [PATCH 015/460] fix(libkernel): initialise scaled clock origin before reading time --- core/libs/prx/libkernel/Time/Time.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libkernel/Time/Time.cpp b/core/libs/prx/libkernel/Time/Time.cpp index 035fa4582..0cd57c647 100644 --- a/core/libs/prx/libkernel/Time/Time.cpp +++ b/core/libs/prx/libkernel/Time/Time.cpp @@ -70,9 +70,9 @@ static std::uint64_t ClockOrigin() { } static std::uint64_t GetMonotonicNanos() { - const auto raw = RawMonotonicNanos(); - if (TimeScale() == 1.0) return raw; + if (TimeScale() == 1.0) return RawMonotonicNanos(); const auto origin = ClockOrigin(); + const auto raw = RawMonotonicNanos(); return origin + static_cast(static_cast(raw - origin) * TimeScale()); } From fb1eb20ed41530c5e6d6c42da07b55693c7d588e Mon Sep 17 00:00:00 2001 From: hazzardouslysolid <219063691+hazzardouslysolid@users.noreply.github.com> Date: Sat, 10 Oct 2026 03:54:35 +0800 Subject: [PATCH 016/460] fix(rtc): floor pre-epoch timestamps for timezone lookup --- core/libs/prx/libSceRtc/Export.cpp | 3 ++- core/libs/tests/GuestRtc.cpp | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceRtc/Export.cpp b/core/libs/prx/libSceRtc/Export.cpp index 2e213f70b..e1c13629d 100644 --- a/core/libs/prx/libSceRtc/Export.cpp +++ b/core/libs/prx/libSceRtc/Export.cpp @@ -96,7 +96,8 @@ std::uint64_t currentTick() { } std::int64_t localOffsetSeconds(std::uint64_t utcTick) { - const auto seconds = static_cast((static_cast(utcTick) - static_cast(UNIX_EPOCH_TICK)) / TICKS_PER_SECOND); + const auto elapsed = std::chrono::microseconds{static_cast(utcTick) - static_cast(UNIX_EPOCH_TICK)}; + const auto seconds = static_cast(std::chrono::floor(elapsed).count()); std::tm local{}; #ifdef _WIN32 if (localtime_s(&local, &seconds) != 0) return 0; diff --git a/core/libs/tests/GuestRtc.cpp b/core/libs/tests/GuestRtc.cpp index bf3528b2c..252649a32 100644 --- a/core/libs/tests/GuestRtc.cpp +++ b/core/libs/tests/GuestRtc.cpp @@ -427,4 +427,26 @@ int main() { Require(sceRtcConvertLocalTimeToUtc(&result, &result) == 0 && result.tick == source.tick); } SetTimeZone("UTC0"); + +#ifndef _WIN32 + SetTimeZone("STD0DST,J1/0,J2/0"); + const std::time_t beforeTransition = -1; + const std::time_t atTransition = 0; + std::tm beforeState{}, afterState{}; + Require(localtime_r(&beforeTransition, &beforeState) != nullptr && beforeState.tm_isdst == 0); + Require(localtime_r(&atTransition, &afterState) != nullptr && afterState.tm_isdst == 1); + for (const std::uint64_t distance : {1000000ull, 500000ull, 1ull}) { + source.tick = unixEpochTick - distance; + Require(sceRtcConvertUtcToLocalTime(&source, &result) == 0 && result.tick == source.tick); + Require(sceRtcConvertUtcToLocalTime(&source, &source) == 0 && source.tick == unixEpochTick - distance); + } + for (const std::uint64_t distance : {0ull, 500000ull}) { + source.tick = unixEpochTick + distance; + Require(sceRtcConvertUtcToLocalTime(&source, &result) == 0 && result.tick == source.tick + 3600000000ull); + } + tick.tick = unixEpochTick - 500000ull; + Require(sceRtcFormatRFC3339LocalTime(text, &tick) == 0 && std::strcmp(text, "1969-12-31T23:59:59.50Z") == 0); + Require(sceRtcFormatRFC2822LocalTime(rfc2822, &tick) == 0 && std::strcmp(rfc2822, "Wed, 31 Dec 1969 23:59:59 +0000") == 0); + SetTimeZone("UTC0"); +#endif } From 32eb8b27e3c73e6a120a45b96afad2fa524d72b9 Mon Sep 17 00:00:00 2001 From: validation agent Date: Sat, 10 Oct 2026 09:28:48 +0800 Subject: [PATCH 017/460] test(libkernel): cover the scaled process clock origin --- core/libs/prx/libkernel/CMakeLists.txt | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index 8e5c79987..881af7cca 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -117,6 +117,13 @@ if(BUILD_TESTING) configure_windows_unwind(guest_process_time_tests) add_test(NAME guest_process_time COMMAND guest_process_time_tests) add_test(NAME guest_process_time_counter COMMAND guest_process_time_tests counter) + add_test(NAME guest_process_time_scaled COMMAND guest_process_time_tests) + add_test(NAME guest_process_time_counter_scaled COMMAND guest_process_time_tests counter) + set_tests_properties(guest_process_time_scaled guest_process_time_counter_scaled PROPERTIES ENVIRONMENT "APS5_TIME_SCALE=0.5") + if(WIN32) + set_tests_properties(guest_process_time_scaled guest_process_time_counter_scaled PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() if(WIN32) set_tests_properties(guest_process_time guest_process_time_counter PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") From 7cd185ec79092ae7f0d52b6271e6f2fa5328731d Mon Sep 17 00:00:00 2001 From: LuisBiceps <54749472+LuisBiceps@users.noreply.github.com> Date: Fri, 9 Oct 2026 02:47:53 +0800 Subject: [PATCH 018/460] feat(libc): implement getifaddrs and freeifaddrs List the host's IPv4 and IPv6 addresses in FreeBSD's struct ifaddrs layout, with FreeBSD sockaddrs and interface flags, from getifaddrs on Linux and GetAdaptersAddresses on Windows. The list is one allocation, so freeifaddrs is a single free. --- core/libs/prx/libc/CMakeLists.txt | 13 +- core/libs/prx/libc/src/Interfaces.cpp | 230 ++++++++++++++++++++++++++ core/libs/tests/GuestInterfaces.cpp | 102 ++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 345 insertions(+), 1 deletion(-) create mode 100644 core/libs/prx/libc/src/Interfaces.cpp create mode 100644 core/libs/tests/GuestInterfaces.cpp diff --git a/core/libs/prx/libc/CMakeLists.txt b/core/libs/prx/libc/CMakeLists.txt index 5dc9ab41a..505d6d46a 100644 --- a/core/libs/prx/libc/CMakeLists.txt +++ b/core/libs/prx/libc/CMakeLists.txt @@ -19,6 +19,7 @@ target_sources(libc PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src/GuestAllocations.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/GuestArena.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/GuestWriteWatch.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/src/Interfaces.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/GuestHeap.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/HeapDiagnostics.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/Mspace.cpp @@ -45,7 +46,7 @@ target_include_directories(libc PRIVATE ) if(WIN32) - target_link_libraries(libc PRIVATE advapi32) + target_link_libraries(libc PRIVATE advapi32 ws2_32 iphlpapi) endif() if(BUILD_TESTING) @@ -255,6 +256,16 @@ if(BUILD_TESTING) set_tests_properties(guest_syslog PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_interface_tests ${LIBS_INCLUDE_DIR}/tests/GuestInterfaces.cpp) + target_include_directories(guest_interface_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_interface_tests PRIVATE libc) + configure_windows_unwind(guest_interface_tests) + add_test(NAME guest_interfaces COMMAND guest_interface_tests) + if(WIN32) + target_link_libraries(guest_interface_tests PRIVATE ws2_32 iphlpapi) + set_tests_properties(guest_interfaces PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(mspace_tests ${LIBS_INCLUDE_DIR}/tests/Mspace.cpp) target_include_directories(mspace_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(mspace_tests PRIVATE libc) diff --git a/core/libs/prx/libc/src/Interfaces.cpp b/core/libs/prx/libc/src/Interfaces.cpp new file mode 100644 index 000000000..401f5bfd8 --- /dev/null +++ b/core/libs/prx/libc/src/Interfaces.cpp @@ -0,0 +1,230 @@ +#ifdef _WIN32 +#define WIN32_LEAN_AND_MEAN +#include +#include +#include +#else +#include +#include +#include +#include +#endif +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include +#include +#include + +extern "C" int* APS5_VABI __error_nid_postfix(); + +struct GuestInterfaceAddress { + GuestInterfaceAddress* next; + char* name; + std::uint32_t flags; + void* address; + void* netmask; + void* destination; + void* data; +}; +static_assert(sizeof(GuestInterfaceAddress) == 56); +static_assert(offsetof(GuestInterfaceAddress, flags) == 16); +static_assert(offsetof(GuestInterfaceAddress, address) == 24); +static_assert(offsetof(GuestInterfaceAddress, destination) == 40); +static_assert(offsetof(GuestInterfaceAddress, data) == 48); + +namespace { +constexpr std::uint32_t guestUp = 0x1; +constexpr std::uint32_t guestBroadcast = 0x2; +constexpr std::uint32_t guestLoopback = 0x8; +constexpr std::uint32_t guestPointToPoint = 0x10; +constexpr std::uint32_t guestRunning = 0x40; +constexpr std::uint32_t guestMulticast = 0x8000; +constexpr std::size_t guestNameCapacity = 16; + +struct Entry { + std::string name; + std::uint32_t flags; + std::vector address; + std::vector netmask; + std::vector destination; +}; + +std::vector GuestIpv4(const std::uint8_t* bytes) { + std::vector guest(16, 0); + guest[0] = 16; + guest[1] = 2; + std::memcpy(guest.data() + 4, bytes, 4); + return guest; +} + +std::vector GuestIpv6(const std::uint8_t* bytes, std::uint32_t scope) { + std::vector guest(28, 0); + guest[0] = 28; + guest[1] = 28; + std::memcpy(guest.data() + 8, bytes, 16); + std::memcpy(guest.data() + 24, &scope, 4); + return guest; +} + +std::vector GuestAddress(const sockaddr* address) { + if (address->sa_family == AF_INET) + return GuestIpv4(reinterpret_cast(&reinterpret_cast(address)->sin_addr)); + const auto* v6 = reinterpret_cast(address); + return GuestIpv6(reinterpret_cast(&v6->sin6_addr), v6->sin6_scope_id); +} + +std::string CheckedName(const char* name) { + const std::string checked(name); + if (checked.empty() || checked.size() >= guestNameCapacity) + throw std::runtime_error("getifaddrs: host interface name does not fit the guest's 16-byte limit: " + checked); + return checked; +} + +std::size_t Aligned(std::size_t size) { + return (size + 7) & ~static_cast(7); +} + +#ifdef _WIN32 +std::vector PrefixMask(int family, unsigned prefix) { + const unsigned bits = family == AF_INET ? 32 : 128; + if (prefix > bits) throw std::runtime_error("getifaddrs: host prefix length " + std::to_string(prefix) + " is out of range"); + std::uint8_t mask[16] = {}; + for (unsigned bit = 0; bit < prefix; ++bit) mask[bit / 8] |= static_cast(0x80 >> (bit % 8)); + return family == AF_INET ? GuestIpv4(mask) : GuestIpv6(mask, 0); +} + +std::vector BroadcastAddress(const std::vector& address, const std::vector& netmask) { + std::vector broadcast = address; + for (std::size_t index = 4; index < 8; ++index) + broadcast[index] = static_cast(address[index] | static_cast(~netmask[index])); + return broadcast; +} + +std::uint32_t InterfaceFlags(const IP_ADAPTER_ADDRESSES* adapter) { + std::uint32_t flags = 0; + if (adapter->OperStatus == IfOperStatusUp) flags |= guestUp | guestRunning; + if (adapter->IfType == IF_TYPE_SOFTWARE_LOOPBACK) flags |= guestLoopback; + else if (adapter->IfType == IF_TYPE_PPP || adapter->IfType == IF_TYPE_TUNNEL) flags |= guestPointToPoint; + else flags |= guestBroadcast; + if (!(adapter->Flags & IP_ADAPTER_NO_MULTICAST)) flags |= guestMulticast; + return flags; +} + +std::string InterfaceName(const IP_ADAPTER_ADDRESSES* adapter) { + char name[NDIS_IF_MAX_STRING_SIZE + 1] = {}; + const auto status = ConvertInterfaceLuidToNameA(&adapter->Luid, name, sizeof(name)); + if (status != NO_ERROR) throw std::runtime_error("getifaddrs: ConvertInterfaceLuidToNameA failed with " + std::to_string(status)); + return CheckedName(name); +} + +std::vector HostEntries() { + const ULONG options = GAA_FLAG_SKIP_ANYCAST | GAA_FLAG_SKIP_MULTICAST | GAA_FLAG_SKIP_DNS_SERVER; + ULONG size = 0; + std::vector storage; + ULONG result = GetAdaptersAddresses(AF_UNSPEC, options, nullptr, nullptr, &size); + while (result == ERROR_BUFFER_OVERFLOW) { + storage.resize(size); + result = GetAdaptersAddresses(AF_UNSPEC, options, nullptr, reinterpret_cast(storage.data()), &size); + } + if (result == ERROR_NO_DATA) return {}; + if (result != NO_ERROR) throw std::runtime_error("getifaddrs: GetAdaptersAddresses failed with " + std::to_string(result)); + std::vector entries; + for (auto* adapter = reinterpret_cast(storage.data()); adapter; adapter = adapter->Next) { + std::string name; + for (auto* unicast = adapter->FirstUnicastAddress; unicast; unicast = unicast->Next) { + const sockaddr* address = unicast->Address.lpSockaddr; + if (!address || (address->sa_family != AF_INET && address->sa_family != AF_INET6)) continue; + if (name.empty()) name = InterfaceName(adapter); + Entry entry{name, InterfaceFlags(adapter), GuestAddress(address), PrefixMask(address->sa_family, unicast->OnLinkPrefixLength), {}}; + if (address->sa_family == AF_INET && (entry.flags & guestBroadcast)) + entry.destination = BroadcastAddress(entry.address, entry.netmask); + entries.push_back(std::move(entry)); + } + } + return entries; +} +#else +std::uint32_t InterfaceFlags(unsigned int host) { + constexpr std::pair flags[] = { + {IFF_UP, guestUp}, {IFF_BROADCAST, guestBroadcast}, {IFF_LOOPBACK, guestLoopback}, + {IFF_POINTOPOINT, guestPointToPoint}, {IFF_RUNNING, guestRunning}, {IFF_NOARP, 0x80}, + {IFF_PROMISC, 0x100}, {IFF_ALLMULTI, 0x200}, {IFF_MULTICAST, guestMulticast}, + }; + std::uint32_t guest = 0; + for (const auto& [hostFlag, guestFlag] : flags) + if (host & hostFlag) guest |= guestFlag; + return guest; +} + +std::vector HostEntries() { + ifaddrs* host = nullptr; + if (::getifaddrs(&host) != 0) throw std::runtime_error(std::string("getifaddrs: host enumeration failed: ") + std::strerror(errno)); + std::vector entries; + try { + for (auto* item = host; item; item = item->ifa_next) { + if (!item->ifa_addr || (item->ifa_addr->sa_family != AF_INET && item->ifa_addr->sa_family != AF_INET6)) continue; + if (!item->ifa_netmask) throw std::runtime_error(std::string("getifaddrs: host address without a netmask on ") + item->ifa_name); + Entry entry{CheckedName(item->ifa_name), InterfaceFlags(item->ifa_flags), GuestAddress(item->ifa_addr), GuestAddress(item->ifa_netmask), {}}; + const sockaddr* destination = item->ifa_flags & IFF_POINTOPOINT ? item->ifa_dstaddr : item->ifa_flags & IFF_BROADCAST ? item->ifa_broadaddr : nullptr; + if (destination && destination->sa_family == item->ifa_addr->sa_family) entry.destination = GuestAddress(destination); + entries.push_back(std::move(entry)); + } + } catch (...) { + ::freeifaddrs(host); + throw; + } + ::freeifaddrs(host); + return entries; +} +#endif +} + +extern "C" { + +int APS5_VABI getifaddrs_nid_postfix(GuestInterfaceAddress** list) { + if (!list) throw std::invalid_argument("getifaddrs: null list"); + const std::vector entries = HostEntries(); + if (entries.empty()) { + *list = nullptr; + return 0; + } + std::size_t size = entries.size() * sizeof(GuestInterfaceAddress); + for (const auto& entry : entries) + size += Aligned(entry.name.size() + 1) + Aligned(entry.address.size()) + Aligned(entry.netmask.size()) + Aligned(entry.destination.size()); + auto* block = static_cast(std::malloc(size)); + if (!block) { + *__error_nid_postfix() = 12; + return -1; + } + auto* items = reinterpret_cast(block); + std::size_t offset = entries.size() * sizeof(GuestInterfaceAddress); + const auto place = [&](const void* source, std::size_t length) { + void* target = block + offset; + std::memcpy(target, source, length); + offset += Aligned(length); + return target; + }; + for (std::size_t index = 0; index < entries.size(); ++index) { + const Entry& entry = entries[index]; + GuestInterfaceAddress& item = items[index]; + item.next = index + 1 < entries.size() ? &items[index + 1] : nullptr; + item.name = static_cast(place(entry.name.c_str(), entry.name.size() + 1)); + item.flags = entry.flags; + item.address = place(entry.address.data(), entry.address.size()); + item.netmask = place(entry.netmask.data(), entry.netmask.size()); + item.destination = entry.destination.empty() ? nullptr : place(entry.destination.data(), entry.destination.size()); + item.data = nullptr; + } + *list = items; + return 0; +} + +void APS5_VABI freeifaddrs_nid_postfix(GuestInterfaceAddress* list) { + std::free(list); +} + +} diff --git a/core/libs/tests/GuestInterfaces.cpp b/core/libs/tests/GuestInterfaces.cpp new file mode 100644 index 000000000..05635b9e6 --- /dev/null +++ b/core/libs/tests/GuestInterfaces.cpp @@ -0,0 +1,102 @@ +#ifdef _WIN32 +#define WIN32_LEAN_AND_MEAN +#include +#include +#include +#else +#include +#include +#endif +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include + +struct GuestInterfaceAddress { + GuestInterfaceAddress* next; + char* name; + std::uint32_t flags; + void* address; + void* netmask; + void* destination; + void* data; +}; +static_assert(sizeof(GuestInterfaceAddress) == 56); + +extern "C" { +int APS5_VABI getifaddrs_nid_postfix(GuestInterfaceAddress** list); +void APS5_VABI freeifaddrs_nid_postfix(GuestInterfaceAddress* list); +} + +static void Require(bool value) { if (!value) std::abort(); } + +static std::size_t HostIpv4Count() { + std::size_t count = 0; +#ifdef _WIN32 + const ULONG options = GAA_FLAG_SKIP_ANYCAST | GAA_FLAG_SKIP_MULTICAST | GAA_FLAG_SKIP_DNS_SERVER; + ULONG size = 0; + std::vector storage; + ULONG result = GetAdaptersAddresses(AF_UNSPEC, options, nullptr, nullptr, &size); + while (result == ERROR_BUFFER_OVERFLOW) { + storage.resize(size); + result = GetAdaptersAddresses(AF_UNSPEC, options, nullptr, reinterpret_cast(storage.data()), &size); + } + Require(result == NO_ERROR); + for (auto* adapter = reinterpret_cast(storage.data()); adapter; adapter = adapter->Next) + for (auto* unicast = adapter->FirstUnicastAddress; unicast; unicast = unicast->Next) + if (unicast->Address.lpSockaddr && unicast->Address.lpSockaddr->sa_family == AF_INET) ++count; +#else + ifaddrs* host = nullptr; + Require(getifaddrs(&host) == 0); + for (auto* item = host; item; item = item->ifa_next) + if (item->ifa_addr && item->ifa_addr->sa_family == AF_INET) ++count; + freeifaddrs(host); +#endif + return count; +} + +int main() { + constexpr std::uint32_t knownFlags = 0x1 | 0x2 | 0x8 | 0x10 | 0x40 | 0x80 | 0x100 | 0x200 | 0x8000; + const std::uint8_t loopbackAddress[] = {127, 0, 0, 1}; + const std::uint8_t loopbackMask[] = {255, 0, 0, 0}; + GuestInterfaceAddress* list = nullptr; + Require(getifaddrs_nid_postfix(&list) == 0 && list != nullptr); + std::size_t entries = 0; + std::size_t ipv4 = 0; + bool loopback = false; + for (auto* item = list; item; item = item->next) { + Require(++entries < 100000); + Require(item->name != nullptr); + const std::size_t nameLength = std::strlen(item->name); + Require(nameLength >= 1 && nameLength <= 15); + Require((item->flags & ~knownFlags) == 0); + Require(item->data == nullptr); + const auto* address = static_cast(item->address); + const auto* netmask = static_cast(item->netmask); + Require(address != nullptr && netmask != nullptr); + Require((address[0] == 16 && address[1] == 2) || (address[0] == 28 && address[1] == 28)); + Require(netmask[0] == address[0] && netmask[1] == address[1]); + if (item->destination) { + const auto* destination = static_cast(item->destination); + Require(destination[0] == address[0] && destination[1] == address[1]); + } + if (address[1] != 2) continue; + ++ipv4; + if (std::memcmp(address + 4, loopbackAddress, 4) == 0) + loopback = std::memcmp(netmask + 4, loopbackMask, 4) == 0 && (item->flags & 0x9) == 0x9 && !(item->flags & 0x2); + } + Require(loopback); + Require(ipv4 == HostIpv4Count()); + freeifaddrs_nid_postfix(list); + freeifaddrs_nid_postfix(nullptr); + bool rejected = false; + try { + getifaddrs_nid_postfix(nullptr); + } catch (const std::invalid_argument&) { + rejected = true; + } + Require(rejected); +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 9850a1576..19c53a157 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -438,3 +438,4 @@ Throughout the project, every function at every stage either **does exactly what - [sigaction](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) records the handler, flags and mask of every signal from 1 to 128 and reports them back, following FreeBSD's `sigaction` (`EINVAL` for other signals, and for a handler on `SIGKILL` or `SIGSTOP`); `signal` records its handler with `SA_RESTART`, as FreeBSD's libc does. Only handlers without `SA_SIGINFO` on the signals `signal` already delivers (`SIGINT`, `SIGILL`, `SIGABRT`, `SIGFPE`, `SIGSEGV`, `SIGTERM`) run, the same way `signal` handlers do; their mask and flags are not applied. `SA_SIGINFO` handlers and other signals (`SIGBUS`, `SIGHUP`, ...) are recorded but never called, since that needs FreeBSD's `siginfo_t` and `ucontext_t`; `raise` of an `SA_SIGINFO` handler on a signal `signal` delivers throws instead of terminating, and `raise` of the other signals still returns `EINVAL` - [AJM](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp), [APR](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) and [NGS2](../../core/libs/prx/libSceNgs2.native/src/System.cpp) state is allocated with `new` and never freed on purpose: title threads still call into these libraries while `std::exit` runs static destructors, so ordinary statics would be destroyed under them. Keep them leaked; `guest_ajm_teardown` covers the AJM case on Unix. - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. +- [getifaddrs](../../core/libs/prx/libc/src/Interfaces.cpp) (libc) lists the host's IPv4 and IPv6 addresses with FreeBSD layouts and flags. FreeBSD's per-interface AF_LINK entries and their if_data are not produced, and Windows interface names are the host's NDIS names (for example ethernet_32769), not the console's. From a09b67b142c804b97bd2341609f85de4adcf23ae Mon Sep 17 00:00:00 2001 From: LuisBiceps <54749472+LuisBiceps@users.noreply.github.com> Date: Fri, 9 Oct 2026 03:19:46 +0800 Subject: [PATCH 019/460] feat(libc): report getifaddrs errors through errno and handle Windows edge cases Return -1 with the FreeBSD errno for a null list and for expected host and allocation failures, as getcwd does. On Windows, give an unspecified prefix length a host mask, leave PPP and tunnel adapters without broadcast or point-to-point flags since their peer address is not reported, fall back to if for long NDIS names, and bound the adapter query retries. Compare every entry with the host's own list in the test. --- core/libs/prx/libc/src/Interfaces.cpp | 107 ++++++++++-------- core/libs/tests/GuestInterfaces.cpp | 156 ++++++++++++++++++-------- docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 172 insertions(+), 93 deletions(-) diff --git a/core/libs/prx/libc/src/Interfaces.cpp b/core/libs/prx/libc/src/Interfaces.cpp index 401f5bfd8..7e8aadef3 100644 --- a/core/libs/prx/libc/src/Interfaces.cpp +++ b/core/libs/prx/libc/src/Interfaces.cpp @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -91,7 +92,7 @@ std::size_t Aligned(std::size_t size) { #ifdef _WIN32 std::vector PrefixMask(int family, unsigned prefix) { const unsigned bits = family == AF_INET ? 32 : 128; - if (prefix > bits) throw std::runtime_error("getifaddrs: host prefix length " + std::to_string(prefix) + " is out of range"); + if (prefix > bits) prefix = bits; std::uint8_t mask[16] = {}; for (unsigned bit = 0; bit < prefix; ++bit) mask[bit / 8] |= static_cast(0x80 >> (bit % 8)); return family == AF_INET ? GuestIpv4(mask) : GuestIpv6(mask, 0); @@ -108,8 +109,7 @@ std::uint32_t InterfaceFlags(const IP_ADAPTER_ADDRESSES* adapter) { std::uint32_t flags = 0; if (adapter->OperStatus == IfOperStatusUp) flags |= guestUp | guestRunning; if (adapter->IfType == IF_TYPE_SOFTWARE_LOOPBACK) flags |= guestLoopback; - else if (adapter->IfType == IF_TYPE_PPP || adapter->IfType == IF_TYPE_TUNNEL) flags |= guestPointToPoint; - else flags |= guestBroadcast; + else if (adapter->IfType != IF_TYPE_PPP && adapter->IfType != IF_TYPE_TUNNEL) flags |= guestBroadcast; if (!(adapter->Flags & IP_ADAPTER_NO_MULTICAST)) flags |= guestMulticast; return flags; } @@ -118,21 +118,23 @@ std::string InterfaceName(const IP_ADAPTER_ADDRESSES* adapter) { char name[NDIS_IF_MAX_STRING_SIZE + 1] = {}; const auto status = ConvertInterfaceLuidToNameA(&adapter->Luid, name, sizeof(name)); if (status != NO_ERROR) throw std::runtime_error("getifaddrs: ConvertInterfaceLuidToNameA failed with " + std::to_string(status)); - return CheckedName(name); + if (std::strlen(name) < guestNameCapacity) return CheckedName(name); + return "if" + std::to_string(adapter->IfIndex ? adapter->IfIndex : adapter->Ipv6IfIndex); } -std::vector HostEntries() { +int HostEntries(std::vector& entries) { const ULONG options = GAA_FLAG_SKIP_ANYCAST | GAA_FLAG_SKIP_MULTICAST | GAA_FLAG_SKIP_DNS_SERVER; ULONG size = 0; std::vector storage; ULONG result = GetAdaptersAddresses(AF_UNSPEC, options, nullptr, nullptr, &size); - while (result == ERROR_BUFFER_OVERFLOW) { + for (unsigned calls = 1; result == ERROR_BUFFER_OVERFLOW && calls < 4; ++calls) { storage.resize(size); result = GetAdaptersAddresses(AF_UNSPEC, options, nullptr, reinterpret_cast(storage.data()), &size); } - if (result == ERROR_NO_DATA) return {}; + if (result == ERROR_NO_DATA) return 0; + if (result == ERROR_BUFFER_OVERFLOW) { *__error_nid_postfix() = 55; return -1; } + if (result == ERROR_NOT_ENOUGH_MEMORY) { *__error_nid_postfix() = 12; return -1; } if (result != NO_ERROR) throw std::runtime_error("getifaddrs: GetAdaptersAddresses failed with " + std::to_string(result)); - std::vector entries; for (auto* adapter = reinterpret_cast(storage.data()); adapter; adapter = adapter->Next) { std::string name; for (auto* unicast = adapter->FirstUnicastAddress; unicast; unicast = unicast->Next) { @@ -145,7 +147,7 @@ std::vector HostEntries() { entries.push_back(std::move(entry)); } } - return entries; + return 0; } #else std::uint32_t InterfaceFlags(unsigned int host) { @@ -160,10 +162,18 @@ std::uint32_t InterfaceFlags(unsigned int host) { return guest; } -std::vector HostEntries() { +int HostEntries(std::vector& entries) { ifaddrs* host = nullptr; - if (::getifaddrs(&host) != 0) throw std::runtime_error(std::string("getifaddrs: host enumeration failed: ") + std::strerror(errno)); - std::vector entries; + if (::getifaddrs(&host) != 0) { + const int hostError = errno; + switch (hostError) { + case ENOMEM: *__error_nid_postfix() = 12; return -1; + case ENOBUFS: *__error_nid_postfix() = 55; return -1; + case EMFILE: *__error_nid_postfix() = 24; return -1; + case ENFILE: *__error_nid_postfix() = 23; return -1; + default: throw std::runtime_error(std::string("getifaddrs: host enumeration failed: ") + std::strerror(hostError)); + } + } try { for (auto* item = host; item; item = item->ifa_next) { if (!item->ifa_addr || (item->ifa_addr->sa_family != AF_INET && item->ifa_addr->sa_family != AF_INET6)) continue; @@ -178,7 +188,7 @@ std::vector HostEntries() { throw; } ::freeifaddrs(host); - return entries; + return 0; } #endif } @@ -186,41 +196,44 @@ std::vector HostEntries() { extern "C" { int APS5_VABI getifaddrs_nid_postfix(GuestInterfaceAddress** list) { - if (!list) throw std::invalid_argument("getifaddrs: null list"); - const std::vector entries = HostEntries(); - if (entries.empty()) { - *list = nullptr; + if (!list) { *__error_nid_postfix() = 14; return -1; } + try { + std::vector entries; + if (HostEntries(entries) != 0) return -1; + if (entries.empty()) { + *list = nullptr; + return 0; + } + std::size_t size = entries.size() * sizeof(GuestInterfaceAddress); + for (const auto& entry : entries) + size += Aligned(entry.name.size() + 1) + Aligned(entry.address.size()) + Aligned(entry.netmask.size()) + Aligned(entry.destination.size()); + auto* block = static_cast(std::malloc(size)); + if (!block) { + *__error_nid_postfix() = 12; + return -1; + } + auto* items = reinterpret_cast(block); + std::size_t offset = entries.size() * sizeof(GuestInterfaceAddress); + const auto place = [&](const void* source, std::size_t length) { + void* target = block + offset; + std::memcpy(target, source, length); + offset += Aligned(length); + return target; + }; + for (std::size_t index = 0; index < entries.size(); ++index) { + const Entry& entry = entries[index]; + GuestInterfaceAddress& item = items[index]; + item.next = index + 1 < entries.size() ? &items[index + 1] : nullptr; + item.name = static_cast(place(entry.name.c_str(), entry.name.size() + 1)); + item.flags = entry.flags; + item.address = place(entry.address.data(), entry.address.size()); + item.netmask = place(entry.netmask.data(), entry.netmask.size()); + item.destination = entry.destination.empty() ? nullptr : place(entry.destination.data(), entry.destination.size()); + item.data = nullptr; + } + *list = items; return 0; - } - std::size_t size = entries.size() * sizeof(GuestInterfaceAddress); - for (const auto& entry : entries) - size += Aligned(entry.name.size() + 1) + Aligned(entry.address.size()) + Aligned(entry.netmask.size()) + Aligned(entry.destination.size()); - auto* block = static_cast(std::malloc(size)); - if (!block) { - *__error_nid_postfix() = 12; - return -1; - } - auto* items = reinterpret_cast(block); - std::size_t offset = entries.size() * sizeof(GuestInterfaceAddress); - const auto place = [&](const void* source, std::size_t length) { - void* target = block + offset; - std::memcpy(target, source, length); - offset += Aligned(length); - return target; - }; - for (std::size_t index = 0; index < entries.size(); ++index) { - const Entry& entry = entries[index]; - GuestInterfaceAddress& item = items[index]; - item.next = index + 1 < entries.size() ? &items[index + 1] : nullptr; - item.name = static_cast(place(entry.name.c_str(), entry.name.size() + 1)); - item.flags = entry.flags; - item.address = place(entry.address.data(), entry.address.size()); - item.netmask = place(entry.netmask.data(), entry.netmask.size()); - item.destination = entry.destination.empty() ? nullptr : place(entry.destination.data(), entry.destination.size()); - item.data = nullptr; - } - *list = items; - return 0; + } catch (const std::bad_alloc&) { *__error_nid_postfix() = 12; return -1; } } void APS5_VABI freeifaddrs_nid_postfix(GuestInterfaceAddress* list) { diff --git a/core/libs/tests/GuestInterfaces.cpp b/core/libs/tests/GuestInterfaces.cpp index 05635b9e6..634b1b728 100644 --- a/core/libs/tests/GuestInterfaces.cpp +++ b/core/libs/tests/GuestInterfaces.cpp @@ -12,7 +12,6 @@ #include #include #include -#include #include struct GuestInterfaceAddress { @@ -25,78 +24,145 @@ struct GuestInterfaceAddress { void* data; }; static_assert(sizeof(GuestInterfaceAddress) == 56); +static_assert(offsetof(GuestInterfaceAddress, flags) == 16); +static_assert(offsetof(GuestInterfaceAddress, address) == 24); +static_assert(offsetof(GuestInterfaceAddress, netmask) == 32); +static_assert(offsetof(GuestInterfaceAddress, destination) == 40); +static_assert(offsetof(GuestInterfaceAddress, data) == 48); extern "C" { int APS5_VABI getifaddrs_nid_postfix(GuestInterfaceAddress** list); void APS5_VABI freeifaddrs_nid_postfix(GuestInterfaceAddress* list); +int* APS5_VABI __error_nid_postfix(); } static void Require(bool value) { if (!value) std::abort(); } -static std::size_t HostIpv4Count() { - std::size_t count = 0; +struct HostInterfaceAddress { + int family; + std::uint8_t address[16]; + std::uint8_t netmask[16]; + std::uint32_t scope; +}; + +static HostInterfaceAddress HostAddress(const sockaddr* address) { + HostInterfaceAddress entry{}; + entry.family = address->sa_family; + if (entry.family == AF_INET) { + std::memcpy(entry.address, &reinterpret_cast(address)->sin_addr, 4); + } else { + const auto* v6 = reinterpret_cast(address); + std::memcpy(entry.address, &v6->sin6_addr, 16); + entry.scope = v6->sin6_scope_id; + } + return entry; +} + +static std::vector HostSnapshot() { + std::vector entries; #ifdef _WIN32 const ULONG options = GAA_FLAG_SKIP_ANYCAST | GAA_FLAG_SKIP_MULTICAST | GAA_FLAG_SKIP_DNS_SERVER; ULONG size = 0; std::vector storage; ULONG result = GetAdaptersAddresses(AF_UNSPEC, options, nullptr, nullptr, &size); - while (result == ERROR_BUFFER_OVERFLOW) { + for (unsigned calls = 1; result == ERROR_BUFFER_OVERFLOW && calls < 4; ++calls) { storage.resize(size); result = GetAdaptersAddresses(AF_UNSPEC, options, nullptr, reinterpret_cast(storage.data()), &size); } + if (result == ERROR_NO_DATA) return entries; Require(result == NO_ERROR); for (auto* adapter = reinterpret_cast(storage.data()); adapter; adapter = adapter->Next) - for (auto* unicast = adapter->FirstUnicastAddress; unicast; unicast = unicast->Next) - if (unicast->Address.lpSockaddr && unicast->Address.lpSockaddr->sa_family == AF_INET) ++count; + for (auto* unicast = adapter->FirstUnicastAddress; unicast; unicast = unicast->Next) { + const sockaddr* address = unicast->Address.lpSockaddr; + if (!address || (address->sa_family != AF_INET && address->sa_family != AF_INET6)) continue; + HostInterfaceAddress entry = HostAddress(address); + const unsigned bits = entry.family == AF_INET ? 32 : 128; + unsigned prefix = unicast->OnLinkPrefixLength; + if (prefix > bits) prefix = bits; + for (unsigned bit = 0; bit < prefix; ++bit) entry.netmask[bit / 8] |= static_cast(0x80 >> (bit % 8)); + entries.push_back(entry); + } #else ifaddrs* host = nullptr; - Require(getifaddrs(&host) == 0); - for (auto* item = host; item; item = item->ifa_next) - if (item->ifa_addr && item->ifa_addr->sa_family == AF_INET) ++count; - freeifaddrs(host); + Require(::getifaddrs(&host) == 0); + try { + for (auto* item = host; item; item = item->ifa_next) { + if (!item->ifa_addr || (item->ifa_addr->sa_family != AF_INET && item->ifa_addr->sa_family != AF_INET6)) continue; + Require(item->ifa_netmask != nullptr); + HostInterfaceAddress entry = HostAddress(item->ifa_addr); + if (entry.family == AF_INET) + std::memcpy(entry.netmask, &reinterpret_cast(item->ifa_netmask)->sin_addr, 4); + else + std::memcpy(entry.netmask, &reinterpret_cast(item->ifa_netmask)->sin6_addr, 16); + entries.push_back(entry); + } + } catch (...) { + ::freeifaddrs(host); + throw; + } + ::freeifaddrs(host); #endif - return count; + return entries; } int main() { constexpr std::uint32_t knownFlags = 0x1 | 0x2 | 0x8 | 0x10 | 0x40 | 0x80 | 0x100 | 0x200 | 0x8000; const std::uint8_t loopbackAddress[] = {127, 0, 0, 1}; const std::uint8_t loopbackMask[] = {255, 0, 0, 0}; - GuestInterfaceAddress* list = nullptr; - Require(getifaddrs_nid_postfix(&list) == 0 && list != nullptr); - std::size_t entries = 0; - std::size_t ipv4 = 0; - bool loopback = false; - for (auto* item = list; item; item = item->next) { - Require(++entries < 100000); - Require(item->name != nullptr); - const std::size_t nameLength = std::strlen(item->name); - Require(nameLength >= 1 && nameLength <= 15); - Require((item->flags & ~knownFlags) == 0); - Require(item->data == nullptr); - const auto* address = static_cast(item->address); - const auto* netmask = static_cast(item->netmask); - Require(address != nullptr && netmask != nullptr); - Require((address[0] == 16 && address[1] == 2) || (address[0] == 28 && address[1] == 28)); - Require(netmask[0] == address[0] && netmask[1] == address[1]); - if (item->destination) { - const auto* destination = static_cast(item->destination); - Require(destination[0] == address[0] && destination[1] == address[1]); + bool matched = false; + for (unsigned attempt = 0; attempt < 3 && !matched; ++attempt) { + const std::vector host = HostSnapshot(); + GuestInterfaceAddress* list = nullptr; + Require(getifaddrs_nid_postfix(&list) == 0); + std::size_t entries = 0; + bool equal = true; + bool loopback = false; + for (auto* item = list; item; item = item->next) { + Require(++entries < 100000); + Require(item->name != nullptr); + const std::size_t nameLength = std::strlen(item->name); + Require(nameLength >= 1 && nameLength <= 15); + Require((item->flags & ~knownFlags) == 0); + Require(item->data == nullptr); + const auto* address = static_cast(item->address); + const auto* netmask = static_cast(item->netmask); + Require(address != nullptr && netmask != nullptr); + Require(reinterpret_cast(address) % 8 == 0); + Require(reinterpret_cast(netmask) % 8 == 0); + Require((address[0] == 16 && address[1] == 2) || (address[0] == 28 && address[1] == 28)); + Require(netmask[0] == address[0] && netmask[1] == address[1]); + if (item->destination) { + const auto* destination = static_cast(item->destination); + Require(reinterpret_cast(destination) % 8 == 0); + Require(destination[0] == address[0] && destination[1] == address[1]); + } + if (address[1] == 2 && (item->flags & 0x2)) Require(item->destination != nullptr); + if (item->flags & 0x10) Require(item->destination != nullptr); + if (entries <= host.size()) { + const HostInterfaceAddress& expected = host[entries - 1]; + const bool ipv4 = address[1] == 2; + const std::size_t offset = ipv4 ? 4 : 8; + const std::size_t length = ipv4 ? 4 : 16; + equal = equal && expected.family == (ipv4 ? AF_INET : AF_INET6) && + std::memcmp(address + offset, expected.address, length) == 0 && + std::memcmp(netmask + offset, expected.netmask, length) == 0; + if (!ipv4) { + std::uint32_t scope; + std::memcpy(&scope, address + 24, 4); + equal = equal && scope == expected.scope; + } + } else { + equal = false; + } + if (address[1] == 2 && std::memcmp(address + 4, loopbackAddress, 4) == 0) + loopback = loopback || (std::memcmp(netmask + 4, loopbackMask, 4) == 0 && (item->flags & 0x9) == 0x9 && !(item->flags & 0x2)); } - if (address[1] != 2) continue; - ++ipv4; - if (std::memcmp(address + 4, loopbackAddress, 4) == 0) - loopback = std::memcmp(netmask + 4, loopbackMask, 4) == 0 && (item->flags & 0x9) == 0x9 && !(item->flags & 0x2); + matched = equal && entries == host.size() && loopback; + freeifaddrs_nid_postfix(list); } - Require(loopback); - Require(ipv4 == HostIpv4Count()); - freeifaddrs_nid_postfix(list); + Require(matched); freeifaddrs_nid_postfix(nullptr); - bool rejected = false; - try { - getifaddrs_nid_postfix(nullptr); - } catch (const std::invalid_argument&) { - rejected = true; - } - Require(rejected); + *__error_nid_postfix() = 0; + Require(getifaddrs_nid_postfix(nullptr) == -1); + Require(*__error_nid_postfix() == 14); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 19c53a157..ba94c676f 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -438,4 +438,4 @@ Throughout the project, every function at every stage either **does exactly what - [sigaction](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) records the handler, flags and mask of every signal from 1 to 128 and reports them back, following FreeBSD's `sigaction` (`EINVAL` for other signals, and for a handler on `SIGKILL` or `SIGSTOP`); `signal` records its handler with `SA_RESTART`, as FreeBSD's libc does. Only handlers without `SA_SIGINFO` on the signals `signal` already delivers (`SIGINT`, `SIGILL`, `SIGABRT`, `SIGFPE`, `SIGSEGV`, `SIGTERM`) run, the same way `signal` handlers do; their mask and flags are not applied. `SA_SIGINFO` handlers and other signals (`SIGBUS`, `SIGHUP`, ...) are recorded but never called, since that needs FreeBSD's `siginfo_t` and `ucontext_t`; `raise` of an `SA_SIGINFO` handler on a signal `signal` delivers throws instead of terminating, and `raise` of the other signals still returns `EINVAL` - [AJM](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp), [APR](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) and [NGS2](../../core/libs/prx/libSceNgs2.native/src/System.cpp) state is allocated with `new` and never freed on purpose: title threads still call into these libraries while `std::exit` runs static destructors, so ordinary statics would be destroyed under them. Keep them leaked; `guest_ajm_teardown` covers the AJM case on Unix. - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. -- [getifaddrs](../../core/libs/prx/libc/src/Interfaces.cpp) (libc) lists the host's IPv4 and IPv6 addresses with FreeBSD layouts and flags. FreeBSD's per-interface AF_LINK entries and their if_data are not produced, and Windows interface names are the host's NDIS names (for example ethernet_32769), not the console's. +- [getifaddrs](../../core/libs/prx/libc/src/Interfaces.cpp) (libc) lists the host's IPv4 and IPv6 addresses in FreeBSD's `ifaddrs` and sockaddr layouts. FreeBSD's per-interface AF_LINK entries (MAC, index, MTU) and their `if_data` are not produced, so interfaces without an address are omitted. Names are the host's (Linux `lo`, `eth0`; Windows NDIS names such as `ethernet_32769`, or `if` when one does not fit 15 bytes), not the console's. On Windows the flags are derived from the adapter: PPP and tunnel adapters get neither `IFF_BROADCAST` nor `IFF_POINTOPOINT` because their peer address is not reported, and an address whose prefix length the host reports as unspecified gets a host mask. From 5c0610f334f452e41623fe4ebcea680b2ba482b9 Mon Sep 17 00:00:00 2001 From: LuisBiceps <54749472+LuisBiceps@users.noreply.github.com> Date: Fri, 9 Oct 2026 03:37:55 +0800 Subject: [PATCH 020/460] feat(libc): keep getifaddrs flags consistent with their addresses on Linux Drop IFF_POINTOPOINT from an address without a peer and compute an IPv4 broadcast address the host omits from the netmask, since FreeBSD always provides the address when the flag is set. Check zero port, flowinfo and padding in the test. --- core/libs/prx/libc/src/Interfaces.cpp | 24 +++++++++++++++--------- core/libs/tests/GuestInterfaces.cpp | 6 ++++++ docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 22 insertions(+), 10 deletions(-) diff --git a/core/libs/prx/libc/src/Interfaces.cpp b/core/libs/prx/libc/src/Interfaces.cpp index 7e8aadef3..3391a0705 100644 --- a/core/libs/prx/libc/src/Interfaces.cpp +++ b/core/libs/prx/libc/src/Interfaces.cpp @@ -89,6 +89,13 @@ std::size_t Aligned(std::size_t size) { return (size + 7) & ~static_cast(7); } +std::vector BroadcastAddress(const std::vector& address, const std::vector& netmask) { + std::vector broadcast = address; + for (std::size_t index = 4; index < 8; ++index) + broadcast[index] = static_cast(address[index] | static_cast(~netmask[index])); + return broadcast; +} + #ifdef _WIN32 std::vector PrefixMask(int family, unsigned prefix) { const unsigned bits = family == AF_INET ? 32 : 128; @@ -98,13 +105,6 @@ std::vector PrefixMask(int family, unsigned prefix) { return family == AF_INET ? GuestIpv4(mask) : GuestIpv6(mask, 0); } -std::vector BroadcastAddress(const std::vector& address, const std::vector& netmask) { - std::vector broadcast = address; - for (std::size_t index = 4; index < 8; ++index) - broadcast[index] = static_cast(address[index] | static_cast(~netmask[index])); - return broadcast; -} - std::uint32_t InterfaceFlags(const IP_ADAPTER_ADDRESSES* adapter) { std::uint32_t flags = 0; if (adapter->OperStatus == IfOperStatusUp) flags |= guestUp | guestRunning; @@ -179,8 +179,14 @@ int HostEntries(std::vector& entries) { if (!item->ifa_addr || (item->ifa_addr->sa_family != AF_INET && item->ifa_addr->sa_family != AF_INET6)) continue; if (!item->ifa_netmask) throw std::runtime_error(std::string("getifaddrs: host address without a netmask on ") + item->ifa_name); Entry entry{CheckedName(item->ifa_name), InterfaceFlags(item->ifa_flags), GuestAddress(item->ifa_addr), GuestAddress(item->ifa_netmask), {}}; - const sockaddr* destination = item->ifa_flags & IFF_POINTOPOINT ? item->ifa_dstaddr : item->ifa_flags & IFF_BROADCAST ? item->ifa_broadaddr : nullptr; - if (destination && destination->sa_family == item->ifa_addr->sa_family) entry.destination = GuestAddress(destination); + const int family = item->ifa_addr->sa_family; + if (item->ifa_flags & IFF_POINTOPOINT) { + if (item->ifa_dstaddr && item->ifa_dstaddr->sa_family == family) entry.destination = GuestAddress(item->ifa_dstaddr); + else entry.flags &= ~guestPointToPoint; + } else if (family == AF_INET && (item->ifa_flags & IFF_BROADCAST)) { + entry.destination = item->ifa_broadaddr && item->ifa_broadaddr->sa_family == AF_INET + ? GuestAddress(item->ifa_broadaddr) : BroadcastAddress(entry.address, entry.netmask); + } entries.push_back(std::move(entry)); } } catch (...) { diff --git a/core/libs/tests/GuestInterfaces.cpp b/core/libs/tests/GuestInterfaces.cpp index 634b1b728..eeb24342a 100644 --- a/core/libs/tests/GuestInterfaces.cpp +++ b/core/libs/tests/GuestInterfaces.cpp @@ -131,6 +131,12 @@ int main() { Require(reinterpret_cast(netmask) % 8 == 0); Require((address[0] == 16 && address[1] == 2) || (address[0] == 28 && address[1] == 28)); Require(netmask[0] == address[0] && netmask[1] == address[1]); + Require(address[2] == 0 && address[3] == 0 && netmask[2] == 0 && netmask[3] == 0); + if (address[1] == 2) { + for (std::size_t index = 8; index < 16; ++index) Require(address[index] == 0 && netmask[index] == 0); + } else { + for (std::size_t index = 4; index < 8; ++index) Require(address[index] == 0 && netmask[index] == 0); + } if (item->destination) { const auto* destination = static_cast(item->destination); Require(reinterpret_cast(destination) % 8 == 0); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index ba94c676f..04c942ac6 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -438,4 +438,4 @@ Throughout the project, every function at every stage either **does exactly what - [sigaction](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) records the handler, flags and mask of every signal from 1 to 128 and reports them back, following FreeBSD's `sigaction` (`EINVAL` for other signals, and for a handler on `SIGKILL` or `SIGSTOP`); `signal` records its handler with `SA_RESTART`, as FreeBSD's libc does. Only handlers without `SA_SIGINFO` on the signals `signal` already delivers (`SIGINT`, `SIGILL`, `SIGABRT`, `SIGFPE`, `SIGSEGV`, `SIGTERM`) run, the same way `signal` handlers do; their mask and flags are not applied. `SA_SIGINFO` handlers and other signals (`SIGBUS`, `SIGHUP`, ...) are recorded but never called, since that needs FreeBSD's `siginfo_t` and `ucontext_t`; `raise` of an `SA_SIGINFO` handler on a signal `signal` delivers throws instead of terminating, and `raise` of the other signals still returns `EINVAL` - [AJM](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp), [APR](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) and [NGS2](../../core/libs/prx/libSceNgs2.native/src/System.cpp) state is allocated with `new` and never freed on purpose: title threads still call into these libraries while `std::exit` runs static destructors, so ordinary statics would be destroyed under them. Keep them leaked; `guest_ajm_teardown` covers the AJM case on Unix. - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. -- [getifaddrs](../../core/libs/prx/libc/src/Interfaces.cpp) (libc) lists the host's IPv4 and IPv6 addresses in FreeBSD's `ifaddrs` and sockaddr layouts. FreeBSD's per-interface AF_LINK entries (MAC, index, MTU) and their `if_data` are not produced, so interfaces without an address are omitted. Names are the host's (Linux `lo`, `eth0`; Windows NDIS names such as `ethernet_32769`, or `if` when one does not fit 15 bytes), not the console's. On Windows the flags are derived from the adapter: PPP and tunnel adapters get neither `IFF_BROADCAST` nor `IFF_POINTOPOINT` because their peer address is not reported, and an address whose prefix length the host reports as unspecified gets a host mask. +- [getifaddrs](../../core/libs/prx/libc/src/Interfaces.cpp) (libc) lists the host's IPv4 and IPv6 addresses in FreeBSD's `ifaddrs` and sockaddr layouts. FreeBSD's per-interface AF_LINK entries (MAC, index, MTU) and their `if_data` are not produced, so interfaces without an address are omitted. Names are the host's (Linux `lo`, `eth0`; Windows NDIS names such as `ethernet_32769`, or `if` when one does not fit 15 bytes), not the console's. A Linux point-to-point address without a peer loses `IFF_POINTOPOINT`, and an IPv4 broadcast address the host omits is computed from the netmask. On Windows the flags are only `IFF_UP`, `IFF_RUNNING`, `IFF_LOOPBACK`, `IFF_BROADCAST` and `IFF_MULTICAST`, derived from the adapter: PPP and tunnel adapters get neither `IFF_BROADCAST` nor `IFF_POINTOPOINT` because their peer address is not reported, and an address whose prefix length the host reports as unspecified gets a host mask. From 63a20d672bd3f99d955c478d3c51dac5f5f3845b Mon Sep 17 00:00:00 2001 From: LuisBiceps <54749472+LuisBiceps@users.noreply.github.com> Date: Fri, 9 Oct 2026 20:37:30 +0800 Subject: [PATCH 021/460] refactor(libc): move the getifaddrs test and TechnicalDebt entry away from lines other pull requests change The guest_interfaces test now follows guest_thread_callback_abi instead of guest_errors, and the TechnicalDebt entry follows the libc SSE4a trap emulation entry instead of ending the Functional list, where other open pull requests add theirs. --- core/libs/prx/libc/CMakeLists.txt | 20 ++++++++++---------- docs/dev/TechnicalDebt.md | 2 +- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/core/libs/prx/libc/CMakeLists.txt b/core/libs/prx/libc/CMakeLists.txt index 505d6d46a..3f073418e 100644 --- a/core/libs/prx/libc/CMakeLists.txt +++ b/core/libs/prx/libc/CMakeLists.txt @@ -210,6 +210,16 @@ if(BUILD_TESTING) set_tests_properties(guest_thread_callback_abi PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_interface_tests ${LIBS_INCLUDE_DIR}/tests/GuestInterfaces.cpp) + target_include_directories(guest_interface_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_interface_tests PRIVATE libc) + configure_windows_unwind(guest_interface_tests) + add_test(NAME guest_interfaces COMMAND guest_interface_tests) + if(WIN32) + target_link_libraries(guest_interface_tests PRIVATE ws2_32 iphlpapi) + set_tests_properties(guest_interfaces PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_reallocalign_tests ${LIBS_INCLUDE_DIR}/tests/GuestReallocalign.cpp) target_include_directories(guest_reallocalign_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_reallocalign_tests PRIVATE libc) @@ -256,16 +266,6 @@ if(BUILD_TESTING) set_tests_properties(guest_syslog PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() - add_test_executable(guest_interface_tests ${LIBS_INCLUDE_DIR}/tests/GuestInterfaces.cpp) - target_include_directories(guest_interface_tests PRIVATE ${LIBS_INCLUDE_DIR}) - target_link_libraries(guest_interface_tests PRIVATE libc) - configure_windows_unwind(guest_interface_tests) - add_test(NAME guest_interfaces COMMAND guest_interface_tests) - if(WIN32) - target_link_libraries(guest_interface_tests PRIVATE ws2_32 iphlpapi) - set_tests_properties(guest_interfaces PROPERTIES ENVIRONMENT_MODIFICATION - "PATH=path_list_prepend:$") - endif() add_test_executable(mspace_tests ${LIBS_INCLUDE_DIR}/tests/Mspace.cpp) target_include_directories(mspace_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(mspace_tests PRIVATE libc) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 04c942ac6..3b832b4fb 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -366,6 +366,7 @@ Throughout the project, every function at every stage either **does exactly what - [PA_SC_CONSERVATIVE_RASTERIZATION_CNTL](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) - 0x6001 uses Vulkan overestimation on devices that add at most 1/256 pixel and keep degenerate triangles; the console's uncertainty region and degenerate handling are unmeasured. A value with `OVER_RAST_ENABLE` and `UNDER_RAST_ENABLE` clear is decoded as disabled even when `UNCERTAINTY_REGION_MODE` or the `OUTER_`/`INNER_UNCERTAINTY_EDGERULE_OVERRIDE` bits (19:16) are set, as PPSA01474 writes 0xe6000; that these fields have no effect without over- or underestimation is assumed from their names, not measured. - The Linux placement of the `--to-intel` stubs in the [ELF patcher](../../core/relinker/elfpatcher/src/linux/LinuxElfPatcher.cpp) is covered only by a synthetic test. - The libc [SSE4a trap emulation](../../core/libs/prx/libc/src/specifics/windows/Sse4aEmulation.hpp) on Windows is superseded by `--to-intel` and remains only until the relinked title has been verified without it. +- [getifaddrs](../../core/libs/prx/libc/src/Interfaces.cpp) (libc) lists the host's IPv4 and IPv6 addresses in FreeBSD's `ifaddrs` and sockaddr layouts. FreeBSD's per-interface AF_LINK entries (MAC, index, MTU) and their `if_data` are not produced, so interfaces without an address are omitted. Names are the host's (Linux `lo`, `eth0`; Windows NDIS names such as `ethernet_32769`, or `if` when one does not fit 15 bytes), not the console's. A Linux point-to-point address without a peer loses `IFF_POINTOPOINT`, and an IPv4 broadcast address the host omits is computed from the netmask. On Windows the flags are only `IFF_UP`, `IFF_RUNNING`, `IFF_LOOPBACK`, `IFF_BROADCAST` and `IFF_MULTICAST`, derived from the adapter: PPP and tunnel adapters get neither `IFF_BROADCAST` nor `IFF_POINTOPOINT` because their peer address is not reported, and an address whose prefix length the host reports as unspecified gets a host mask. - [Descriptor pool chain](../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp) (libSceAgcDriver) - each pool holds two storage buffer descriptors more than a set can take, as a workaround: RADV (Mesa 26.0 and later) lets a `FREE_DESCRIPTOR_SET` pool place a set up to 32 bytes past the pool's buffer ([`util_vma_heap_init`](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/vulkan/radv_descriptor_pool.c) is given `bo_size + RADV_POOL_HEAP_OFFSET` as a size), and with a page-multiple pool size those bytes are an unmapped page - [CMASK fast clears](../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver) - each fast-cleared CMASK tile (code 0) is stored as `CB_COLOR_CLEAR_WORD` into its 8x8 texels at the first draw into the target or at its fast-clear eliminate or DCC decompress pass, not as the color block writes the tile, and the CMASK is then stored as expanded (0xF): a texture or copy read of the target after that draw and before the title's eliminate sees the clear value in tiles no draw wrote, where the console keeps their previous bytes. Tile addresses follow the pipe-aligned SW_64KB_Z_X CMASK equation for 16 pipes; other CMASK swizzle modes, codes other than 0 and 0xF, a cleared tile of a DCC target and a fast clear of a mipmapped, 3D or array color target throw. When the clear runs as a compute pass (a resident target whose CMASK is host-imported), an invalid code throws at the next fast-clear materialization, not at the one that read it - [HTILE depth fast clears](../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver) - HTILE is not modeled. Only RADV's `clear_htile_mask` compute kernel (17 GFX10 words, 64x1x1 groups) is recognized, and only when it leaves ZMASK at 0 (`user[4]` and `user[5]` low nibbles both 0); the dispatch still runs, and the host depth image is cleared to `DB_DEPTH_CLEAR` the next time the surface with that `DB_HTILE_DATA_BASE` address is bound. The address match is exact, so a clear of a different HTILE range does not hit. HTILE initialization, the stencil-only update, and fast clears made by the SDK's own paths, CP DMA fills or other kernels are not covered and leave the depth image as it was. The kernel words are the ones the PR author saw RADV submit; they are not compared with Mesa's source. @@ -438,4 +439,3 @@ Throughout the project, every function at every stage either **does exactly what - [sigaction](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) records the handler, flags and mask of every signal from 1 to 128 and reports them back, following FreeBSD's `sigaction` (`EINVAL` for other signals, and for a handler on `SIGKILL` or `SIGSTOP`); `signal` records its handler with `SA_RESTART`, as FreeBSD's libc does. Only handlers without `SA_SIGINFO` on the signals `signal` already delivers (`SIGINT`, `SIGILL`, `SIGABRT`, `SIGFPE`, `SIGSEGV`, `SIGTERM`) run, the same way `signal` handlers do; their mask and flags are not applied. `SA_SIGINFO` handlers and other signals (`SIGBUS`, `SIGHUP`, ...) are recorded but never called, since that needs FreeBSD's `siginfo_t` and `ucontext_t`; `raise` of an `SA_SIGINFO` handler on a signal `signal` delivers throws instead of terminating, and `raise` of the other signals still returns `EINVAL` - [AJM](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp), [APR](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) and [NGS2](../../core/libs/prx/libSceNgs2.native/src/System.cpp) state is allocated with `new` and never freed on purpose: title threads still call into these libraries while `std::exit` runs static destructors, so ordinary statics would be destroyed under them. Keep them leaked; `guest_ajm_teardown` covers the AJM case on Unix. - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. -- [getifaddrs](../../core/libs/prx/libc/src/Interfaces.cpp) (libc) lists the host's IPv4 and IPv6 addresses in FreeBSD's `ifaddrs` and sockaddr layouts. FreeBSD's per-interface AF_LINK entries (MAC, index, MTU) and their `if_data` are not produced, so interfaces without an address are omitted. Names are the host's (Linux `lo`, `eth0`; Windows NDIS names such as `ethernet_32769`, or `if` when one does not fit 15 bytes), not the console's. A Linux point-to-point address without a peer loses `IFF_POINTOPOINT`, and an IPv4 broadcast address the host omits is computed from the netmask. On Windows the flags are only `IFF_UP`, `IFF_RUNNING`, `IFF_LOOPBACK`, `IFF_BROADCAST` and `IFF_MULTICAST`, derived from the adapter: PPP and tunnel adapters get neither `IFF_BROADCAST` nor `IFF_POINTOPOINT` because their peer address is not reported, and an address whose prefix length the host reports as unspecified gets a host mask. From 58d1f5ab18cffe68f9215a1dd891eab21e3caf76 Mon Sep 17 00:00:00 2001 From: Dean Galvin Date: Fri, 9 Oct 2026 10:58:36 +1300 Subject: [PATCH 022/460] fix(recompiler): split descriptor webs whose Phis sit in different blocks A sample whose T# is chosen in one if/else and whose S# is chosen in a later one has descriptor Phis in two blocks. The web walk required every Phi of the tuple to sit in one block, so it gave up and the plan threw "GetImageResource dword 0 is not a valid runtime value". The walk now splits at the Phis of the latest block in program order, and Phis of earlier blocks pass through each edge unchanged until the walk reaches their block. --- .../libSceAgcDriver/tests/ShaderMemory.cpp | 9 +++++++ .../Optimization/src/ResourceTracker.cpp | 27 ++++++++++++++++--- 2 files changed, 33 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp b/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp index bc78344e5..8ecb49655 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp @@ -892,6 +892,15 @@ void verifyDescriptorPhis() { }; require(loopImages.size() == 2u && holdsLoopImage(first) && holdsLoopImage(second), "descriptor Phi: the loop's chained T# Phis were not split into the two SRT T#s"); + const std::vector sequentialCode{0xf4000400u, 0xfa000050u, 0xbf8cc07fu, 0xbf068010u, 0xbf850003u, 0xf40c0200u, 0xfa000000u, 0xbf820002u, 0xf40c0200u, 0xfa000060u, 0xbf068010u, 0xbf850003u, 0xf4080500u, 0xfa000020u, 0xbf820002u, 0xf4080500u, 0xfa000030u, 0xf4080600u, 0xfa000040u, 0xbf8cc07fu, 0xf09c0f08u, 0x00a20000u, 0xbf8c3f70u, 0xe0700000u, 0x80060000u, 0xbf810000u}; + auto sequential = makeRequest(sequentialCode); + const auto sequentialPlan = GetResourcePlan(sequential); + require(sequentialPlan->info.images.size() == 2u && sequentialPlan->info.samplers.size() == 2u, "descriptor Phi: a T# and an S# chosen in separate blocks were not split into their SRT descriptors"); + AgcDriver::ShaderMemory sequentialMemory({}); + const auto sequentialCapture = sequentialMemory.Capture(sequential); + sequential.context.memory = sequentialMemory.Regions(); + static_cast(Recompile(sequential, *sequentialCapture)); + const auto entryWrites = [&](std::initializer_list words) { auto code = loopCode; code.insert(code.begin() + 8, words); diff --git a/core/shader/recompiler/Optimization/src/ResourceTracker.cpp b/core/shader/recompiler/Optimization/src/ResourceTracker.cpp index 3f20951dd..54961f231 100644 --- a/core/shader/recompiler/Optimization/src/ResourceTracker.cpp +++ b/core/shader/recompiler/Optimization/src/ResourceTracker.cpp @@ -766,6 +766,7 @@ private: std::map, IrValue*> selectors; std::vector> phis; std::unordered_map resolved; + std::unordered_map order; }; IrValue* WebSelector(std::vector tuple, DescriptorWeb& web, IrValue& position) { @@ -803,7 +804,27 @@ private: web.arms.push_back(std::move(tuple)); return &m_builder.Constant(static_cast(web.arms.size() - 1u)); } - IrBlock* block = (*phi)->Parent(); + if (web.order.empty()) { + const auto& order = m_program.BlockOrder(); + for (std::size_t index = 0; index < order.size(); index++) { + web.order.emplace(order[index], index); + } + } + IrBlock* block = nullptr; + std::size_t latest = 0; + for (const IrValue* value : tuple) { + if (!value->IsPhi()) { + continue; + } + const auto found = web.order.find(value->Parent()); + if (found == web.order.end()) { + return nullptr; + } + if (block == nullptr || found->second > latest) { + block = value->Parent(); + latest = found->second; + } + } if (block == nullptr || web.selectors.size() == DescriptorWeb::maxStates) { return nullptr; } @@ -812,7 +833,7 @@ private: return nullptr; } for (const IrValue* value : tuple) { - if (value->IsPhi() && (value->Parent() != block || value->PhiBlockCount() != predecessors.size())) { + if (value->IsPhi() && value->Parent() == block && value->PhiBlockCount() != predecessors.size()) { return nullptr; } } @@ -822,7 +843,7 @@ private: for (IrBlock* predecessor : predecessors) { std::vector incoming; for (IrValue* value : tuple) { - IrValue* edge = value->IsPhi() ? PhiIncoming(*value, predecessor) : value; + IrValue* edge = value->IsPhi() && value->Parent() == block ? PhiIncoming(*value, predecessor) : value; if (edge == nullptr) { return nullptr; } From a9c132a6290232565048f49fd55e74903b943b7c Mon Sep 17 00:00:00 2001 From: Joey Fisher Date: Wed, 7 Oct 2026 12:30:00 -0600 Subject: [PATCH 023/460] feat(agc): opt-in raised priority for the queue workers and the presenter --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 4 +- .../Execution/include/Driver.hpp | 1 + .../Execution/include/ThreadPriority.hpp | 10 + .../Execution/src/Driver/EntryPoints.cpp | 5 + .../src/Driver/Queues/Submission.cpp | 9 +- .../Execution/src/ThreadPriority.cpp | 213 ++++++++++++++++++ .../prx/libSceVideoOut/src/VideoOutDriver.cpp | 10 +- docs/dev/BUILD.md | 4 + 8 files changed, 253 insertions(+), 3 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/Execution/include/ThreadPriority.hpp create mode 100644 core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 583e96a21..62792dedd 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -150,6 +150,7 @@ set(agcDriverSources ${CMAKE_CURRENT_SOURCE_DIR}/Execution/src/Driver/Synchronization/WaitMemory.cpp ${CMAKE_CURRENT_SOURCE_DIR}/Execution/src/Driver/Synchronization/WaitStatistics.cpp ${CMAKE_CURRENT_SOURCE_DIR}/Execution/src/WorkerSampler.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/Execution/src/ThreadPriority.cpp ${CMAKE_CURRENT_SOURCE_DIR}/Execution/src/GuestMemory.cpp ${CMAKE_CURRENT_SOURCE_DIR}/Execution/src/ShaderMemory.cpp ${CMAKE_CURRENT_SOURCE_DIR}/Execution/src/BdaFeatures.cpp @@ -229,6 +230,7 @@ target_link_libraries(libSceAgcDriver PRIVATE SDL2-static libc libkernel + ${CMAKE_DL_LIBS} ) set_target_properties(libSceAgcDriver PROPERTIES CXX_EXTENSIONS OFF) @@ -811,7 +813,7 @@ target_include_directories(agc_driver_mesh_tests PRIVATE ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include ${CMAKE_SOURCE_DIR}/3rdparty/SPIRV-Headers/include ) -target_link_libraries(agc_driver_mesh_tests PRIVATE shader_recompiler_agc_driver SDL2-static libc libkernel) +target_link_libraries(agc_driver_mesh_tests PRIVATE shader_recompiler_agc_driver SDL2-static libc libkernel ${CMAKE_DL_LIBS}) set_target_properties(agc_driver_mesh_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_driver_mesh_tests) diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/Driver.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/Driver.hpp index 16ef18fbe..76b1a346f 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/Driver.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/Driver.hpp @@ -12,6 +12,7 @@ void Submit(const Packet* packet, std::uint32_t queue); } extern "C" void AgcDriverWaitIdle_nid_postfix(); +extern "C" void AgcDriverRaiseWorkerThreadPriority_nid_postfix(const char* role); extern "C" void AgcDriverLockVulkanLoader_nid_postfix(); extern "C" void AgcDriverUnlockVulkanLoader_nid_postfix(); extern "C" void AgcDriverShutdown_nid_postfix(); diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/ThreadPriority.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/ThreadPriority.hpp new file mode 100644 index 000000000..723ce9ec5 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/Execution/include/ThreadPriority.hpp @@ -0,0 +1,10 @@ +#ifndef CORE_LIBS_PRX_LIBSCEAGCDRIVER_EXECUTION_INCLUDE_THREADPRIORITY_HPP +#define CORE_LIBS_PRX_LIBSCEAGCDRIVER_EXECUTION_INCLUDE_THREADPRIORITY_HPP + +namespace AgcDriver { + +void RaiseWorkerThreadPriority(const char* role); + +} + +#endif diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/EntryPoints.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/EntryPoints.cpp index 526c7ab37..277f9673d 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/EntryPoints.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/EntryPoints.cpp @@ -1,3 +1,4 @@ +#include "prx/libSceAgcDriver/Execution/include/ThreadPriority.hpp" #include "prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp" #include @@ -55,6 +56,10 @@ void ReportFailure(std::exception_ptr error) { } +extern "C" void AgcDriverRaiseWorkerThreadPriority_nid_postfix(const char* role) { + AgcDriver::RaiseWorkerThreadPriority(role); +} + extern "C" void AgcDriverWaitIdle_nid_postfix() try { AgcDriver::WaitIdle(); } catch (const ProcessShutdown&) { diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp index 3df687bf6..9d17a60fc 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp @@ -5,7 +5,9 @@ #include "prx/libSceAgcDriver/Execution/include/Driver/Synchronization/SynchronizationStatistics.hpp" #include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" #include "prx/libSceAgcDriver/Execution/include/Pm4.hpp" +#include "prx/libSceAgcDriver/Execution/include/ThreadPriority.hpp" #include +#include #include namespace AgcDriver::DriverDetail { @@ -312,7 +314,12 @@ void Driver::enqueue(Submission submission) { for (const auto dword : submission.labelWrites) ++worker.unfinishedWrites[dword]; worker.pending.push_back(std::move(submission)); worker.queued.fetch_add(1, std::memory_order_acq_rel); - if (!worker.thread.joinable()) worker.thread = std::thread([this, queue] { run(queue); }); + if (!worker.thread.joinable()) worker.thread = std::thread([this, queue] { + char role[32]; + std::snprintf(role, sizeof(role), "queue 0x%x worker", queue); + RaiseWorkerThreadPriority(role); + run(queue); + }); } void Driver::noteHeldAtSubmit(Submission& submission, std::size_t cursor) { diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp new file mode 100644 index 000000000..13f7ccbba --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp @@ -0,0 +1,213 @@ +#include "prx/libSceAgcDriver/Execution/include/ThreadPriority.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#ifdef _WIN32 +#include +#else +#include +#include +#include +#include +#include +#endif + +namespace AgcDriver { + +namespace { + +enum class Mode { Off, High, Realtime }; + +Mode RequestedMode() { + static const Mode mode = [] { + const char* text = std::getenv("APS5_THREAD_PRIORITY"); + if (text == nullptr || *text == '\0' || std::strcmp(text, "off") == 0 || std::strcmp(text, "0") == 0) return Mode::Off; + if (std::strcmp(text, "rt") == 0 || std::strcmp(text, "realtime") == 0) return Mode::Realtime; + return Mode::High; + }(); + return mode; +} + +int EnvInt(const char* name, int fallback) { + const char* text = std::getenv(name); + return text != nullptr && *text != '\0' ? std::atoi(text) : fallback; +} + +void LogOnce(const char* role, const std::string& text) { + static std::mutex mutex; + static std::set seen; + std::lock_guard lock(mutex); + if (!seen.insert(std::string(role) + "|" + text).second) return; + std::fprintf(stderr, "[thread-priority] %s: %s\n", role, text.c_str()); +} + +#ifndef _WIN32 + +struct DBusConnection; +struct DBusMessage; +struct DBusError { + const char* name; + const char* message; + unsigned int dummy1 : 1, dummy2 : 1, dummy3 : 1, dummy4 : 1, dummy5 : 1; + void* padding1; +}; +constexpr int DbusBusSystem = 1; +constexpr int DbusTypeInvalid = 0; +constexpr int DbusTypeInt32 = static_cast('i'); +constexpr int DbusTypeUint32 = static_cast('u'); +constexpr int DbusTypeUint64 = static_cast('t'); + +struct Dbus { + void* handle = nullptr; + DBusConnection* connection = nullptr; + int (*threadsInitDefault)() = nullptr; + void (*errorInit)(DBusError*) = nullptr; + void (*errorFree)(DBusError*) = nullptr; + int (*errorIsSet)(const DBusError*) = nullptr; + DBusConnection* (*busGet)(int, DBusError*) = nullptr; + DBusMessage* (*messageNewMethodCall)(const char*, const char*, const char*, const char*) = nullptr; + int (*messageAppendArgs)(DBusMessage*, int, ...) = nullptr; + DBusMessage* (*sendWithReplyAndBlock)(DBusConnection*, DBusMessage*, int, DBusError*) = nullptr; + void (*messageUnref)(DBusMessage*) = nullptr; + std::string failure; +}; + +Dbus& LoadDbus() { + static Dbus dbus = [] { + Dbus loaded; + loaded.handle = dlopen("libdbus-1.so.3", RTLD_NOW | RTLD_LOCAL); + if (loaded.handle == nullptr) { + loaded.failure = "libdbus-1.so.3 is not available"; + return loaded; + } + const auto resolve = [&](auto& function, const char* name) { + function = reinterpret_cast>(dlsym(loaded.handle, name)); + if (function == nullptr && loaded.failure.empty()) loaded.failure = std::string("libdbus lacks ") + name; + }; + resolve(loaded.threadsInitDefault, "dbus_threads_init_default"); + resolve(loaded.errorInit, "dbus_error_init"); + resolve(loaded.errorFree, "dbus_error_free"); + resolve(loaded.errorIsSet, "dbus_error_is_set"); + resolve(loaded.busGet, "dbus_bus_get"); + resolve(loaded.messageNewMethodCall, "dbus_message_new_method_call"); + resolve(loaded.messageAppendArgs, "dbus_message_append_args"); + resolve(loaded.sendWithReplyAndBlock, "dbus_connection_send_with_reply_and_block"); + resolve(loaded.messageUnref, "dbus_message_unref"); + if (!loaded.failure.empty()) return loaded; + loaded.threadsInitDefault(); + DBusError error{}; + loaded.errorInit(&error); + loaded.connection = loaded.busGet(DbusBusSystem, &error); + if (loaded.connection == nullptr) { + loaded.failure = std::string("system bus: ") + (loaded.errorIsSet(&error) && error.message != nullptr ? error.message : "unavailable"); + loaded.errorFree(&error); + } + return loaded; + }(); + return dbus; +} + +bool RtkitCall(const char* method, std::uint64_t tid, int type, const void* value, std::string& failure) { + auto& dbus = LoadDbus(); + if (dbus.connection == nullptr) { + failure = dbus.failure; + return false; + } + static std::mutex mutex; + std::lock_guard lock(mutex); + auto* message = dbus.messageNewMethodCall("org.freedesktop.RealtimeKit1", "/org/freedesktop/RealtimeKit1", "org.freedesktop.RealtimeKit1", method); + if (message == nullptr) { + failure = "dbus message allocation failed"; + return false; + } + const std::uint64_t pid = static_cast(getpid()); + bool ok = false; + if (type == DbusTypeInt32) ok = dbus.messageAppendArgs(message, DbusTypeUint64, &pid, DbusTypeUint64, &tid, DbusTypeInt32, value, DbusTypeInvalid) != 0; + else ok = dbus.messageAppendArgs(message, DbusTypeUint64, &pid, DbusTypeUint64, &tid, DbusTypeUint32, value, DbusTypeInvalid) != 0; + if (!ok) { + dbus.messageUnref(message); + failure = "dbus argument append failed"; + return false; + } + DBusError error{}; + dbus.errorInit(&error); + auto* reply = dbus.sendWithReplyAndBlock(dbus.connection, message, 5000, &error); + dbus.messageUnref(message); + if (reply == nullptr) { + failure = std::string(method) + ": " + (dbus.errorIsSet(&error) && error.message != nullptr ? error.message : "no reply"); + dbus.errorFree(&error); + return false; + } + dbus.messageUnref(reply); + return true; +} + +bool PrepareRealtime(std::string& failure) { + rlimit limit{}; + if (getrlimit(RLIMIT_RTTIME, &limit) != 0) { + failure = "getrlimit(RLIMIT_RTTIME) failed"; + return false; + } + const rlim_t maxUs = static_cast(EnvInt("APS5_THREAD_RTTIME_US", 200000)); + limit.rlim_max = maxUs; + limit.rlim_cur = maxUs / 2; + if (setrlimit(RLIMIT_RTTIME, &limit) != 0) { + failure = "setrlimit(RLIMIT_RTTIME) failed"; + return false; + } + sched_param param{}; + if (sched_getparam(0, ¶m) != 0 || sched_setscheduler(0, sched_getscheduler(0) | SCHED_RESET_ON_FORK, ¶m) != 0) { + failure = "SCHED_RESET_ON_FORK could not be set"; + return false; + } + return true; +} + +void Raise(const char* role) { + const auto tid = static_cast(syscall(SYS_gettid)); + std::string failure; + if (RequestedMode() == Mode::Realtime) { + const std::uint32_t priority = static_cast(std::max(1, std::min(99, EnvInt("APS5_THREAD_RT", 10)))); + if (PrepareRealtime(failure) && RtkitCall("MakeThreadRealtimeWithPID", tid, DbusTypeUint32, &priority, failure)) { + LogOnce(role, "SCHED_FIFO " + std::to_string(priority) + " via rtkit"); + return; + } + LogOnce(role, "realtime refused (" + failure + "), trying a nice level"); + } + const std::int32_t nice = static_cast(std::max(-20, std::min(19, EnvInt("APS5_THREAD_NICE", -10)))); + if (RtkitCall("MakeThreadHighPriorityWithPID", tid, DbusTypeInt32, &nice, failure)) { + LogOnce(role, "nice " + std::to_string(nice) + " via rtkit"); + return; + } + if (setpriority(PRIO_PROCESS, static_cast(tid), nice) == 0) { + LogOnce(role, "nice " + std::to_string(nice) + " (setpriority)"); + return; + } + LogOnce(role, "not raised: " + failure + "; setpriority refused too (RLIMIT_NICE)"); +} + +#else + +void Raise(const char* role) { + const int priority = RequestedMode() == Mode::Realtime ? THREAD_PRIORITY_TIME_CRITICAL : THREAD_PRIORITY_HIGHEST; + if (SetThreadPriority(GetCurrentThread(), priority)) LogOnce(role, priority == THREAD_PRIORITY_TIME_CRITICAL ? "time-critical" : "highest"); + else LogOnce(role, "SetThreadPriority failed"); +} + +#endif + +} + +void RaiseWorkerThreadPriority(const char* role) { + if (RequestedMode() == Mode::Off) return; + Raise(role); +} + +} diff --git a/core/libs/prx/libSceVideoOut/src/VideoOutDriver.cpp b/core/libs/prx/libSceVideoOut/src/VideoOutDriver.cpp index 2b52be2cd..22b137f8e 100644 --- a/core/libs/prx/libSceVideoOut/src/VideoOutDriver.cpp +++ b/core/libs/prx/libSceVideoOut/src/VideoOutDriver.cpp @@ -258,7 +258,15 @@ VideoOutDriver::VideoOutDriver() { AgcDriverWaitIdle_nid_postfix(); std::promise started; auto attached = started.get_future(); - presentThread = std::jthread([this, &started](std::stop_token token) { presentLoop(token, started); }); + presentThread = std::jthread([this, &started](std::stop_token token) { + try { + AgcDriverRaiseWorkerThreadPriority_nid_postfix("presenter"); + } catch (...) { + started.set_exception(std::current_exception()); + return; + } + presentLoop(token, started); + }); attached.get(); vblankThread = std::jthread([this](std::stop_token token) { vblankLoop(token); }); LibcRegisterShutdown_nid_postfix([] { VideoOutDriver::Get().Shutdown(); }); diff --git a/docs/dev/BUILD.md b/docs/dev/BUILD.md index 15edb681c..1bb4cec60 100644 --- a/docs/dev/BUILD.md +++ b/docs/dev/BUILD.md @@ -90,6 +90,10 @@ With `CMAKE_OSX_ARCHITECTURES=x86_64` Apple silicon runs the tests through Roset Set `APS5_PIPELINE_STATS=1` to capture and print driver statistics for each newly created graphics or compute pipeline. This requires `VK_KHR_pipeline_executable_properties` and `pipelineExecutableInfo`; an unsupported device fails with an error. Statistic names and units are driver-specific. Capturing statistics can increase pipeline compilation cost. The setting is disabled by default. +## Driver thread priority + +Set `APS5_THREAD_PRIORITY=high` to raise the AGC driver's queue workers and the video output presenter above the title's threads. On Linux the threads get nice `APS5_THREAD_NICE` (default -10) through rtkit, or through `setpriority` when rtkit is not available; on Windows they get `THREAD_PRIORITY_HIGHEST`. `APS5_THREAD_PRIORITY=rt` asks rtkit for `SCHED_FIFO` priority `APS5_THREAD_RT` (default 10, `RLIMIT_RTTIME` `APS5_THREAD_RTTIME_US`, default 200000) and falls back to the nice level; on Windows it sets `THREAD_PRIORITY_TIME_CRITICAL`. Each thread logs the outcome once as `[thread-priority]`. The setting is disabled by default. + ## Shader recompiler The shader recompilation logic in [core/shader/recompiler](../../core/shader/recompiler) is isolated from the rest of the project and is a pure function of its input data, designed for integration into any other project. The current CMake target also includes cache support and links a supplied runtime target, glslang, and optionally SPIRV-Tools. From 22147dbbe6a2543f42743bb72ec1467873c92efd Mon Sep 17 00:00:00 2001 From: Joey Fisher Date: Thu, 8 Oct 2026 15:46:28 -0600 Subject: [PATCH 024/460] fix(agc): lower the RLIMIT_RTTIME hard limit only when rtkit requires it Read rtkit's MaxRealtimePriority and RTTimeUSecMax before touching the limit, keep a hard limit that already satisfies rtkit, lower the soft limit to at most half of the hard one, and log when the hard limit is lowered. rtkit checks the hard limit, so it cannot be left alone when it is above RTTimeUSecMax; the side effect is listed in TechnicalDebt.md. Log the policy rtkit applied, SCHED_RR by default. --- .../Execution/src/ThreadPriority.cpp | 130 +++++++++++++++--- docs/dev/BUILD.md | 2 +- docs/dev/TechnicalDebt.md | 1 + 3 files changed, 112 insertions(+), 21 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp index 13f7ccbba..6d2f0e2c5 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp @@ -63,6 +63,19 @@ constexpr int DbusTypeInvalid = 0; constexpr int DbusTypeInt32 = static_cast('i'); constexpr int DbusTypeUint32 = static_cast('u'); constexpr int DbusTypeUint64 = static_cast('t'); +constexpr int DbusTypeInt64 = static_cast('x'); +constexpr int DbusTypeString = static_cast('s'); +constexpr int DbusTypeVariant = static_cast('v'); + +struct DBusMessageIter { + void* dummy1; + void* dummy2; + std::uint32_t dummy3; + int dummy4, dummy5, dummy6, dummy7, dummy8, dummy9, dummy10, dummy11; + int pad1; + void* pad2; + void* pad3; +}; struct Dbus { void* handle = nullptr; @@ -76,6 +89,10 @@ struct Dbus { int (*messageAppendArgs)(DBusMessage*, int, ...) = nullptr; DBusMessage* (*sendWithReplyAndBlock)(DBusConnection*, DBusMessage*, int, DBusError*) = nullptr; void (*messageUnref)(DBusMessage*) = nullptr; + int (*iterInit)(DBusMessage*, DBusMessageIter*) = nullptr; + int (*iterGetArgType)(DBusMessageIter*) = nullptr; + void (*iterRecurse)(DBusMessageIter*, DBusMessageIter*) = nullptr; + void (*iterGetBasic)(DBusMessageIter*, void*) = nullptr; std::string failure; }; @@ -100,6 +117,10 @@ Dbus& LoadDbus() { resolve(loaded.messageAppendArgs, "dbus_message_append_args"); resolve(loaded.sendWithReplyAndBlock, "dbus_connection_send_with_reply_and_block"); resolve(loaded.messageUnref, "dbus_message_unref"); + resolve(loaded.iterInit, "dbus_message_iter_init"); + resolve(loaded.iterGetArgType, "dbus_message_iter_get_arg_type"); + resolve(loaded.iterRecurse, "dbus_message_iter_recurse"); + resolve(loaded.iterGetBasic, "dbus_message_iter_get_basic"); if (!loaded.failure.empty()) return loaded; loaded.threadsInitDefault(); DBusError error{}; @@ -114,6 +135,19 @@ Dbus& LoadDbus() { return dbus; } +DBusMessage* RtkitSend(DBusMessage* message, const char* method, std::string& failure) { + auto& dbus = LoadDbus(); + DBusError error{}; + dbus.errorInit(&error); + auto* reply = dbus.sendWithReplyAndBlock(dbus.connection, message, 5000, &error); + dbus.messageUnref(message); + if (reply == nullptr) { + failure = std::string(method) + ": " + (dbus.errorIsSet(&error) && error.message != nullptr ? error.message : "no reply"); + dbus.errorFree(&error); + } + return reply; +} + bool RtkitCall(const char* method, std::uint64_t tid, int type, const void* value, std::string& failure) { auto& dbus = LoadDbus(); if (dbus.connection == nullptr) { @@ -136,30 +170,64 @@ bool RtkitCall(const char* method, std::uint64_t tid, int type, const void* valu failure = "dbus argument append failed"; return false; } - DBusError error{}; - dbus.errorInit(&error); - auto* reply = dbus.sendWithReplyAndBlock(dbus.connection, message, 5000, &error); - dbus.messageUnref(message); - if (reply == nullptr) { - failure = std::string(method) + ": " + (dbus.errorIsSet(&error) && error.message != nullptr ? error.message : "no reply"); - dbus.errorFree(&error); - return false; - } + auto* reply = RtkitSend(message, method, failure); + if (reply == nullptr) return false; dbus.messageUnref(reply); return true; } -bool PrepareRealtime(std::string& failure) { - rlimit limit{}; - if (getrlimit(RLIMIT_RTTIME, &limit) != 0) { - failure = "getrlimit(RLIMIT_RTTIME) failed"; +bool RtkitProperty(const char* name, std::int64_t& value, std::string& failure) { + auto& dbus = LoadDbus(); + if (dbus.connection == nullptr) { + failure = dbus.failure; return false; } - const rlim_t maxUs = static_cast(EnvInt("APS5_THREAD_RTTIME_US", 200000)); - limit.rlim_max = maxUs; - limit.rlim_cur = maxUs / 2; - if (setrlimit(RLIMIT_RTTIME, &limit) != 0) { - failure = "setrlimit(RLIMIT_RTTIME) failed"; + auto* message = dbus.messageNewMethodCall("org.freedesktop.RealtimeKit1", "/org/freedesktop/RealtimeKit1", "org.freedesktop.DBus.Properties", "Get"); + if (message == nullptr) { + failure = "dbus message allocation failed"; + return false; + } + const char* interfaceName = "org.freedesktop.RealtimeKit1"; + if (dbus.messageAppendArgs(message, DbusTypeString, &interfaceName, DbusTypeString, &name, DbusTypeInvalid) == 0) { + dbus.messageUnref(message); + failure = "dbus argument append failed"; + return false; + } + auto* reply = RtkitSend(message, name, failure); + if (reply == nullptr) return false; + DBusMessageIter iter{}; + DBusMessageIter variant{}; + bool ok = false; + if (dbus.iterInit(reply, &iter) != 0 && dbus.iterGetArgType(&iter) == DbusTypeVariant) { + dbus.iterRecurse(&iter, &variant); + const int type = dbus.iterGetArgType(&variant); + if (type == DbusTypeInt32) { + std::int32_t number = 0; + dbus.iterGetBasic(&variant, &number); + value = number; + ok = true; + } else if (type == DbusTypeInt64) { + std::int64_t number = 0; + dbus.iterGetBasic(&variant, &number); + value = number; + ok = true; + } + } + dbus.messageUnref(reply); + if (!ok) failure = std::string(name) + ": unexpected reply type"; + return ok; +} + +bool PrepareRealtime(std::uint32_t priority, const char* role, std::string& failure) { + std::int64_t maxPriority = 0; + std::int64_t maxRttime = 0; + if (!RtkitProperty("MaxRealtimePriority", maxPriority, failure) || !RtkitProperty("RTTimeUSecMax", maxRttime, failure)) return false; + if (static_cast(priority) > maxPriority) { + failure = "priority " + std::to_string(priority) + " is above rtkit's MaxRealtimePriority " + std::to_string(maxPriority); + return false; + } + if (maxRttime <= 0) { + failure = "rtkit reports RTTimeUSecMax " + std::to_string(maxRttime); return false; } sched_param param{}; @@ -167,6 +235,27 @@ bool PrepareRealtime(std::string& failure) { failure = "SCHED_RESET_ON_FORK could not be set"; return false; } + static std::mutex mutex; + std::lock_guard lock(mutex); + rlimit limit{}; + if (getrlimit(RLIMIT_RTTIME, &limit) != 0) { + failure = "getrlimit(RLIMIT_RTTIME) failed"; + return false; + } + const rlim_t requested = static_cast(std::max(1, EnvInt("APS5_THREAD_RTTIME_US", 200000))); + const rlim_t ceiling = std::min(requested, static_cast(maxRttime)); + rlimit wanted = limit; + if (wanted.rlim_max > ceiling) wanted.rlim_max = ceiling; + if (wanted.rlim_cur > wanted.rlim_max / 2) wanted.rlim_cur = wanted.rlim_max / 2; + if (wanted.rlim_cur == limit.rlim_cur && wanted.rlim_max == limit.rlim_max) return true; + if (setrlimit(RLIMIT_RTTIME, &wanted) != 0) { + failure = "setrlimit(RLIMIT_RTTIME) failed"; + return false; + } + if (wanted.rlim_max != limit.rlim_max) { + const auto text = [](rlim_t value) { return value == RLIM_INFINITY ? std::string("unlimited") : std::to_string(value); }; + LogOnce(role, "RLIMIT_RTTIME hard limit lowered from " + text(limit.rlim_max) + " to " + text(wanted.rlim_max) + " us for the process"); + } return true; } @@ -175,8 +264,9 @@ void Raise(const char* role) { std::string failure; if (RequestedMode() == Mode::Realtime) { const std::uint32_t priority = static_cast(std::max(1, std::min(99, EnvInt("APS5_THREAD_RT", 10)))); - if (PrepareRealtime(failure) && RtkitCall("MakeThreadRealtimeWithPID", tid, DbusTypeUint32, &priority, failure)) { - LogOnce(role, "SCHED_FIFO " + std::to_string(priority) + " via rtkit"); + if (PrepareRealtime(priority, role, failure) && RtkitCall("MakeThreadRealtimeWithPID", tid, DbusTypeUint32, &priority, failure)) { + const int policy = sched_getscheduler(0) & ~SCHED_RESET_ON_FORK; + LogOnce(role, std::string(policy == SCHED_FIFO ? "SCHED_FIFO " : "SCHED_RR ") + std::to_string(priority) + " via rtkit"); return; } LogOnce(role, "realtime refused (" + failure + "), trying a nice level"); diff --git a/docs/dev/BUILD.md b/docs/dev/BUILD.md index 1bb4cec60..fca71450f 100644 --- a/docs/dev/BUILD.md +++ b/docs/dev/BUILD.md @@ -92,7 +92,7 @@ Set `APS5_PIPELINE_STATS=1` to capture and print driver statistics for each newl ## Driver thread priority -Set `APS5_THREAD_PRIORITY=high` to raise the AGC driver's queue workers and the video output presenter above the title's threads. On Linux the threads get nice `APS5_THREAD_NICE` (default -10) through rtkit, or through `setpriority` when rtkit is not available; on Windows they get `THREAD_PRIORITY_HIGHEST`. `APS5_THREAD_PRIORITY=rt` asks rtkit for `SCHED_FIFO` priority `APS5_THREAD_RT` (default 10, `RLIMIT_RTTIME` `APS5_THREAD_RTTIME_US`, default 200000) and falls back to the nice level; on Windows it sets `THREAD_PRIORITY_TIME_CRITICAL`. Each thread logs the outcome once as `[thread-priority]`. The setting is disabled by default. +Set `APS5_THREAD_PRIORITY=high` to raise the AGC driver's queue workers and the video output presenter above the title's threads. On Linux the threads get nice `APS5_THREAD_NICE` (default -10) through rtkit, or through `setpriority` when rtkit is not available; on Windows they get `THREAD_PRIORITY_HIGHEST`. `APS5_THREAD_PRIORITY=rt` asks rtkit for real-time priority `APS5_THREAD_RT` (default 10, at most rtkit's `MaxRealtimePriority`) under rtkit's policy, `SCHED_RR` by default, and falls back to the nice level; on Windows it sets `THREAD_PRIORITY_TIME_CRITICAL`. rtkit only grants it when the process's hard `RLIMIT_RTTIME` is at most its `RTTimeUSecMax` (200000 us by default), so a hard limit above `APS5_THREAD_RTTIME_US` (default 200000, capped at `RTTimeUSecMax`) is lowered to it and the soft limit to at most half of it. The hard limit cannot be raised again and stays lowered for the whole process even if rtkit then refuses; see [technical debt](TechnicalDebt.md#functional). Each thread logs the outcome once as `[thread-priority]`. The setting is disabled by default. ## Shader recompiler diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 5a5353560..bf6fe4ed1 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -293,6 +293,7 @@ Throughout the project, every function at every stage either **does exactly what - [Compute LDS beyond the device's shared-memory limit](../../core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp) (libSceAgcDriver) lives in a coherent storage buffer with one slice of `LDS bytes / 4 + 1` dwords per workgroup (the last dword is the atomic lock), shared by all such dispatches and grown in 1 MiB steps up to `maxStorageBufferRange`; a dispatch whose slices do not fit throws. It is slower than shared memory and its cost was not measured. An indirect dispatch of such a program always reads its group counts on the CPU. Its results are pinned against the in-limit semantics in `agc_driver_lds_beyond_device_limit`, not measured on hardware. - [Graphics pipeline libraries](../../core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp) (libSceAgcDriver) are fast-linked only, never relinked with `VK_PIPELINE_CREATE_LINK_TIME_OPTIMIZATION_BIT_EXT`. A vertex stage then keeps the outputs its fragment stage does not read: on an RTX 4090 (NVIDIA 615.71) 13 of 15 PPSA36082 vertex shaders paired with a pixel shader that reads none of them used up to 68 registers instead of 38 and a binary up to 21 % larger, while pairs that read every output compiled to the same code as a whole pipeline (#2242). - [sceAvPlayerGetVideoData](../../core/libs/prx/libSceAvPlayer/src/Source.cpp) (libSceAvPlayer) in sync mode `SCE_AVPLAYER_AV_SYNC_MODE_NONE` skips queued frames that the audio clock has already passed while the title pulls audio, so a title that requests video slower than the film's frame rate keeps the picture with the sound. Whether the console hands out every frame in this mode is unknown. PPSA12544 requests 7-19 frames per second for a 30 fps cinematic +- [`APS5_THREAD_PRIORITY=rt`](../../core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp) (libSceAgcDriver) lowers the process's hard `RLIMIT_RTTIME` when it is above `APS5_THREAD_RTTIME_US` or rtkit's `RTTimeUSecMax`, because [rtkit](https://github.com/heftig/rtkit/blob/c295fa849f52b487be6433e69e08b46251950399/rtkit-daemon.c#L592-L637) (`verify_process_rttime`) reads the hard limit from `/proc//limits` and refuses real-time scheduling when it is higher. An unprivileged process cannot raise the hard limit again, so it stays lowered for the process and its threads even when rtkit then refuses the request (burst limit, user or start-time checks) and the threads fall back to a nice level. rtkit's `MaxRealtimePriority` and `RTTimeUSecMax` are read first, so an unreachable rtkit or a priority above its maximum leaves the limit untouched. The lowered limit only matters for threads that later run under a real-time policy - [libSceHttp2 asynchronous requests](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) never reach the network: `sceHttp2SendRequestAsync` accepts the request, queues a completion whose result is the library's network error and triggers the user event named by the kqueue option (equeue at offset 0, event id at 8, user data at 16, layout inferred from PPSA17221); `sceHttp2WaitAsync` returns that completion. The completion's `event_type` value is unknown and left 0. PPSA17221 completes its request object only after `sceHttp2SendRequestAsync` returns, and its handler for a synchronous failure calls `shared_from_this` before that, so a send must not fail synchronously. A kqueue option or user event that is missing, an extra option, and a wait with no completed operation throw - [system](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) runs no command. It follows FreeBSD 11's `lib/libc/stdlib/system.c` for a sandbox without `/bin/sh`: a null command returns 1 unconditionally, as that version does, and any command returns the wait status of a child that could not exec the shell and called `_exit(127)` (`0x7f00`). Whether the console's application sandbox has no shell, and whether `vfork` succeeds there (FreeBSD returns -1 when it fails), is unverified - [sceKernelWriteThrottlingStatus](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) reports no write throttling: the host SSD has no write budget, so the first qword of the 32-byte status is set to `UINT32_MAX` and the rest is zeroed. PPSA17221 reads only that qword: it multiplies it by 64 KiB and compares the product with byte thresholds up to 0x4AF00000 to pick one of five tiers. When the call fails the title assumes 0x4B000001 bytes, which selects the same top tier as `UINT32_MAX`. What the value measures on hardware (a remaining budget or a rate) and the meaning of the other three qwords are unknown From 73e4afd95162a5ca2870fb43f1dcf270655e6228 Mon Sep 17 00:00:00 2001 From: Joey Fisher Date: Thu, 8 Oct 2026 16:43:33 -0600 Subject: [PATCH 025/460] test(agc): unit-test APS5_THREAD_PRIORITY parsing and the RLIMIT_RTTIME plan --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 13 ++++ .../Execution/include/ThreadPriority.hpp | 24 +++++++ .../Execution/src/ThreadPriority.cpp | 71 ++++++++++--------- .../libSceAgcDriver/tests/ThreadPriority.cpp | 60 ++++++++++++++++ 4 files changed, 136 insertions(+), 32 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/ThreadPriority.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 62792dedd..7a7d077e6 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -321,6 +321,19 @@ if(BUILD_TESTING) endif() endif() +add_test_executable(agc_thread_priority_tests EXCLUDE_FROM_ALL + tests/ThreadPriority.cpp + Execution/src/ThreadPriority.cpp +) +target_include_directories(agc_thread_priority_tests PRIVATE ${LIBS_INCLUDE_DIR}) +target_link_libraries(agc_thread_priority_tests PRIVATE ${CMAKE_DL_LIBS}) +set_target_properties(agc_thread_priority_tests PROPERTIES CXX_EXTENSIONS OFF) +configure_windows_unwind(agc_thread_priority_tests) +if(BUILD_TESTING) + set_target_properties(agc_thread_priority_tests PROPERTIES EXCLUDE_FROM_ALL OFF) + add_test(NAME agc_thread_priority COMMAND agc_thread_priority_tests) +endif() + add_test_executable(agc_structurizer_tests EXCLUDE_FROM_ALL tests/Structurizer.cpp) target_link_libraries(agc_structurizer_tests PRIVATE shader_recompiler_agc_driver libc) set_target_properties(agc_structurizer_tests PROPERTIES CXX_EXTENSIONS OFF) diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/ThreadPriority.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/ThreadPriority.hpp index 723ce9ec5..6a38c4f77 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/ThreadPriority.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/ThreadPriority.hpp @@ -1,8 +1,32 @@ #ifndef CORE_LIBS_PRX_LIBSCEAGCDRIVER_EXECUTION_INCLUDE_THREADPRIORITY_HPP #define CORE_LIBS_PRX_LIBSCEAGCDRIVER_EXECUTION_INCLUDE_THREADPRIORITY_HPP +#include +#include +#include + namespace AgcDriver { +enum class ThreadPriorityMode { Off, High, Realtime }; + +struct RttimeLimit { + std::uint64_t soft = 0; + std::uint64_t hard = 0; + bool operator==(const RttimeLimit&) const = default; +}; + +struct RtkitLimits { + std::int64_t maxPriority = 0; + std::int64_t maxRttimeUs = 0; +}; + +struct RealtimePlan { + std::string refusal; + std::optional limit; +}; + +ThreadPriorityMode ParseThreadPriorityMode(const char* text); +RealtimePlan PlanRealtime(std::uint32_t priority, const std::optional& rtkit, RttimeLimit current, std::uint64_t requestedUs); void RaiseWorkerThreadPriority(const char* role); } diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp index 6d2f0e2c5..cd63fdb5e 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp @@ -23,15 +23,8 @@ namespace AgcDriver { namespace { -enum class Mode { Off, High, Realtime }; - -Mode RequestedMode() { - static const Mode mode = [] { - const char* text = std::getenv("APS5_THREAD_PRIORITY"); - if (text == nullptr || *text == '\0' || std::strcmp(text, "off") == 0 || std::strcmp(text, "0") == 0) return Mode::Off; - if (std::strcmp(text, "rt") == 0 || std::strcmp(text, "realtime") == 0) return Mode::Realtime; - return Mode::High; - }(); +ThreadPriorityMode RequestedMode() { + static const ThreadPriorityMode mode = ParseThreadPriorityMode(std::getenv("APS5_THREAD_PRIORITY")); return mode; } @@ -221,20 +214,9 @@ bool RtkitProperty(const char* name, std::int64_t& value, std::string& failure) bool PrepareRealtime(std::uint32_t priority, const char* role, std::string& failure) { std::int64_t maxPriority = 0; std::int64_t maxRttime = 0; - if (!RtkitProperty("MaxRealtimePriority", maxPriority, failure) || !RtkitProperty("RTTimeUSecMax", maxRttime, failure)) return false; - if (static_cast(priority) > maxPriority) { - failure = "priority " + std::to_string(priority) + " is above rtkit's MaxRealtimePriority " + std::to_string(maxPriority); - return false; - } - if (maxRttime <= 0) { - failure = "rtkit reports RTTimeUSecMax " + std::to_string(maxRttime); - return false; - } - sched_param param{}; - if (sched_getparam(0, ¶m) != 0 || sched_setscheduler(0, sched_getscheduler(0) | SCHED_RESET_ON_FORK, ¶m) != 0) { - failure = "SCHED_RESET_ON_FORK could not be set"; - return false; - } + std::string rtkitFailure; + std::optional rtkit; + if (RtkitProperty("MaxRealtimePriority", maxPriority, rtkitFailure) && RtkitProperty("RTTimeUSecMax", maxRttime, rtkitFailure)) rtkit = RtkitLimits{maxPriority, maxRttime}; static std::mutex mutex; std::lock_guard lock(mutex); rlimit limit{}; @@ -242,12 +224,19 @@ bool PrepareRealtime(std::uint32_t priority, const char* role, std::string& fail failure = "getrlimit(RLIMIT_RTTIME) failed"; return false; } - const rlim_t requested = static_cast(std::max(1, EnvInt("APS5_THREAD_RTTIME_US", 200000))); - const rlim_t ceiling = std::min(requested, static_cast(maxRttime)); - rlimit wanted = limit; - if (wanted.rlim_max > ceiling) wanted.rlim_max = ceiling; - if (wanted.rlim_cur > wanted.rlim_max / 2) wanted.rlim_cur = wanted.rlim_max / 2; - if (wanted.rlim_cur == limit.rlim_cur && wanted.rlim_max == limit.rlim_max) return true; + const auto requested = static_cast(std::max(1, EnvInt("APS5_THREAD_RTTIME_US", 200000))); + const auto plan = PlanRealtime(priority, rtkit, {limit.rlim_cur, limit.rlim_max}, requested); + if (!plan.refusal.empty()) { + failure = rtkit.has_value() ? plan.refusal : rtkitFailure; + return false; + } + sched_param param{}; + if (sched_getparam(0, ¶m) != 0 || sched_setscheduler(0, sched_getscheduler(0) | SCHED_RESET_ON_FORK, ¶m) != 0) { + failure = "SCHED_RESET_ON_FORK could not be set"; + return false; + } + if (!plan.limit.has_value()) return true; + const rlimit wanted{plan.limit->soft, plan.limit->hard}; if (setrlimit(RLIMIT_RTTIME, &wanted) != 0) { failure = "setrlimit(RLIMIT_RTTIME) failed"; return false; @@ -262,7 +251,7 @@ bool PrepareRealtime(std::uint32_t priority, const char* role, std::string& fail void Raise(const char* role) { const auto tid = static_cast(syscall(SYS_gettid)); std::string failure; - if (RequestedMode() == Mode::Realtime) { + if (RequestedMode() == ThreadPriorityMode::Realtime) { const std::uint32_t priority = static_cast(std::max(1, std::min(99, EnvInt("APS5_THREAD_RT", 10)))); if (PrepareRealtime(priority, role, failure) && RtkitCall("MakeThreadRealtimeWithPID", tid, DbusTypeUint32, &priority, failure)) { const int policy = sched_getscheduler(0) & ~SCHED_RESET_ON_FORK; @@ -286,7 +275,7 @@ void Raise(const char* role) { #else void Raise(const char* role) { - const int priority = RequestedMode() == Mode::Realtime ? THREAD_PRIORITY_TIME_CRITICAL : THREAD_PRIORITY_HIGHEST; + const int priority = RequestedMode() == ThreadPriorityMode::Realtime ? THREAD_PRIORITY_TIME_CRITICAL : THREAD_PRIORITY_HIGHEST; if (SetThreadPriority(GetCurrentThread(), priority)) LogOnce(role, priority == THREAD_PRIORITY_TIME_CRITICAL ? "time-critical" : "highest"); else LogOnce(role, "SetThreadPriority failed"); } @@ -295,8 +284,26 @@ void Raise(const char* role) { } +ThreadPriorityMode ParseThreadPriorityMode(const char* text) { + if (text == nullptr || *text == '\0' || std::strcmp(text, "off") == 0 || std::strcmp(text, "0") == 0) return ThreadPriorityMode::Off; + if (std::strcmp(text, "rt") == 0 || std::strcmp(text, "realtime") == 0) return ThreadPriorityMode::Realtime; + return ThreadPriorityMode::High; +} + +RealtimePlan PlanRealtime(std::uint32_t priority, const std::optional& rtkit, RttimeLimit current, std::uint64_t requestedUs) { + if (!rtkit.has_value()) return {"rtkit's limits are unknown", std::nullopt}; + if (static_cast(priority) > rtkit->maxPriority) return {"priority " + std::to_string(priority) + " is above rtkit's MaxRealtimePriority " + std::to_string(rtkit->maxPriority), std::nullopt}; + if (rtkit->maxRttimeUs <= 0) return {"rtkit reports RTTimeUSecMax " + std::to_string(rtkit->maxRttimeUs), std::nullopt}; + const auto ceiling = std::min(requestedUs, static_cast(rtkit->maxRttimeUs)); + auto wanted = current; + if (wanted.hard > ceiling) wanted.hard = ceiling; + if (wanted.soft > wanted.hard / 2) wanted.soft = wanted.hard / 2; + if (wanted == current) return {}; + return {{}, wanted}; +} + void RaiseWorkerThreadPriority(const char* role) { - if (RequestedMode() == Mode::Off) return; + if (RequestedMode() == ThreadPriorityMode::Off) return; Raise(role); } diff --git a/core/libs/prx/libSceAgcDriver/tests/ThreadPriority.cpp b/core/libs/prx/libSceAgcDriver/tests/ThreadPriority.cpp new file mode 100644 index 000000000..c7ea3a7a7 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/ThreadPriority.cpp @@ -0,0 +1,60 @@ +#include "prx/libSceAgcDriver/Execution/include/ThreadPriority.hpp" +#include +#include +#include +#include +#include +#include + +namespace { + +using namespace AgcDriver; + +constexpr std::uint64_t Unlimited = std::numeric_limits::max(); +constexpr RtkitLimits Rtkit{20, 200000}; + +void Require(bool condition, const std::string& message) { + if (!condition) throw std::runtime_error(message); +} + +void ParseTests() { + for (const char* text : {static_cast(nullptr), "", "off", "0"}) Require(ParseThreadPriorityMode(text) == ThreadPriorityMode::Off, std::string("APS5_THREAD_PRIORITY=") + (text == nullptr ? "(unset)" : text) + " must leave priorities alone"); + for (const char* text : {"rt", "realtime"}) Require(ParseThreadPriorityMode(text) == ThreadPriorityMode::Realtime, std::string("APS5_THREAD_PRIORITY=") + text + " must ask for real-time priority"); + for (const char* text : {"high", "1", "on"}) Require(ParseThreadPriorityMode(text) == ThreadPriorityMode::High, std::string("APS5_THREAD_PRIORITY=") + text + " must ask for a nice level"); +} + +void ExpectRefused(const RealtimePlan& plan, const std::string& what) { + Require(!plan.refusal.empty() && !plan.limit.has_value(), what + " must refuse real-time priority without touching RLIMIT_RTTIME"); +} + +void ExpectLimit(const RealtimePlan& plan, RttimeLimit expected, const std::string& what) { + Require(plan.refusal.empty() && plan.limit.has_value(), what + " must set RLIMIT_RTTIME"); + Require(*plan.limit == expected, what + " set RLIMIT_RTTIME to soft " + std::to_string(plan.limit->soft) + " hard " + std::to_string(plan.limit->hard) + ", expected soft " + std::to_string(expected.soft) + " hard " + std::to_string(expected.hard)); +} + +void PlanTests() { + ExpectRefused(PlanRealtime(10, std::nullopt, {Unlimited, Unlimited}, 200000), "an unreachable rtkit"); + ExpectRefused(PlanRealtime(10, std::nullopt, {30000, 150000}, 200000), "an unreachable rtkit"); + ExpectRefused(PlanRealtime(21, Rtkit, {Unlimited, Unlimited}, 200000), "a priority above rtkit's MaxRealtimePriority"); + ExpectRefused(PlanRealtime(10, RtkitLimits{20, 0}, {Unlimited, Unlimited}, 200000), "rtkit without an RTTimeUSecMax"); + ExpectLimit(PlanRealtime(20, Rtkit, {Unlimited, Unlimited}, 200000), {100000, 200000}, "an unlimited hard limit"); + ExpectLimit(PlanRealtime(10, Rtkit, {100000, Unlimited}, 200000), {100000, 200000}, "a lowered soft limit under an unlimited hard limit"); + ExpectLimit(PlanRealtime(10, Rtkit, {Unlimited, Unlimited}, 500000), {100000, 200000}, "APS5_THREAD_RTTIME_US above RTTimeUSecMax"); + ExpectLimit(PlanRealtime(10, Rtkit, {Unlimited, Unlimited}, 100000), {50000, 100000}, "APS5_THREAD_RTTIME_US below RTTimeUSecMax"); + ExpectLimit(PlanRealtime(10, Rtkit, {150000, 150000}, 200000), {75000, 150000}, "a hard limit rtkit accepts with a soft limit above half of it"); + const auto kept = PlanRealtime(10, Rtkit, {30000, 150000}, 200000); + Require(kept.refusal.empty() && !kept.limit.has_value(), "limits rtkit already accepts must be left alone"); +} + +} + +int main() { + try { + ParseTests(); + PlanTests(); + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } + return 0; +} From 7add840f111cc6046cc605fdf2ab2215b2f41bf7 Mon Sep 17 00:00:00 2001 From: Joey Fisher Date: Thu, 8 Oct 2026 19:15:08 -0600 Subject: [PATCH 026/460] fix(agc): link the thread-priority test against libc for its personality routine --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 7a7d077e6..aadc95809 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -326,12 +326,16 @@ add_test_executable(agc_thread_priority_tests EXCLUDE_FROM_ALL Execution/src/ThreadPriority.cpp ) target_include_directories(agc_thread_priority_tests PRIVATE ${LIBS_INCLUDE_DIR}) -target_link_libraries(agc_thread_priority_tests PRIVATE ${CMAKE_DL_LIBS}) +target_link_libraries(agc_thread_priority_tests PRIVATE libc ${CMAKE_DL_LIBS}) set_target_properties(agc_thread_priority_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_thread_priority_tests) if(BUILD_TESTING) set_target_properties(agc_thread_priority_tests PROPERTIES EXCLUDE_FROM_ALL OFF) add_test(NAME agc_thread_priority COMMAND agc_thread_priority_tests) + if(WIN32) + set_tests_properties(agc_thread_priority PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() endif() add_test_executable(agc_structurizer_tests EXCLUDE_FROM_ALL tests/Structurizer.cpp) From 0fe61e41917574b98a448051f1dbaca1032bd157 Mon Sep 17 00:00:00 2001 From: Joey Fisher Date: Fri, 9 Oct 2026 09:10:19 -0600 Subject: [PATCH 027/460] fix(agc): pick one priority path per mode and throw when it is refused --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 6 ++++ .../Execution/src/ThreadPriority.cpp | 28 +++++++++---------- .../libSceAgcDriver/tests/ThreadPriority.cpp | 19 ++++++++++++- docs/dev/BUILD.md | 2 +- docs/dev/TechnicalDebt.md | 2 +- 5 files changed, 39 insertions(+), 18 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index aadc95809..702b93040 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -335,6 +335,12 @@ if(BUILD_TESTING) if(WIN32) set_tests_properties(agc_thread_priority PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") + else() + foreach(mode rt high) + add_test(NAME agc_thread_priority_refused_${mode} COMMAND agc_thread_priority_tests refused) + set_tests_properties(agc_thread_priority_refused_${mode} PROPERTIES ENVIRONMENT + "APS5_THREAD_PRIORITY=${mode};DBUS_SYSTEM_BUS_ADDRESS=unix:path=/nonexistent/aps5-no-rtkit") + endforeach() endif() endif() diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp index cd63fdb5e..007008df7 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp @@ -6,6 +6,7 @@ #include #include #include +#include #include #include @@ -253,31 +254,28 @@ void Raise(const char* role) { std::string failure; if (RequestedMode() == ThreadPriorityMode::Realtime) { const std::uint32_t priority = static_cast(std::max(1, std::min(99, EnvInt("APS5_THREAD_RT", 10)))); - if (PrepareRealtime(priority, role, failure) && RtkitCall("MakeThreadRealtimeWithPID", tid, DbusTypeUint32, &priority, failure)) { - const int policy = sched_getscheduler(0) & ~SCHED_RESET_ON_FORK; - LogOnce(role, std::string(policy == SCHED_FIFO ? "SCHED_FIFO " : "SCHED_RR ") + std::to_string(priority) + " via rtkit"); - return; + if (!PrepareRealtime(priority, role, failure) || !RtkitCall("MakeThreadRealtimeWithPID", tid, DbusTypeUint32, &priority, failure)) { + throw std::runtime_error(std::string("APS5_THREAD_PRIORITY=rt: ") + role + " realtime refused: " + failure); } - LogOnce(role, "realtime refused (" + failure + "), trying a nice level"); + const int policy = sched_getscheduler(0) & ~SCHED_RESET_ON_FORK; + LogOnce(role, std::string(policy == SCHED_FIFO ? "SCHED_FIFO " : "SCHED_RR ") + std::to_string(priority) + " via rtkit"); + return; } const std::int32_t nice = static_cast(std::max(-20, std::min(19, EnvInt("APS5_THREAD_NICE", -10)))); - if (RtkitCall("MakeThreadHighPriorityWithPID", tid, DbusTypeInt32, &nice, failure)) { - LogOnce(role, "nice " + std::to_string(nice) + " via rtkit"); - return; + if (!RtkitCall("MakeThreadHighPriorityWithPID", tid, DbusTypeInt32, &nice, failure)) { + throw std::runtime_error(std::string("APS5_THREAD_PRIORITY=high: ") + role + " nice " + std::to_string(nice) + " refused: " + failure); } - if (setpriority(PRIO_PROCESS, static_cast(tid), nice) == 0) { - LogOnce(role, "nice " + std::to_string(nice) + " (setpriority)"); - return; - } - LogOnce(role, "not raised: " + failure + "; setpriority refused too (RLIMIT_NICE)"); + LogOnce(role, "nice " + std::to_string(nice) + " via rtkit"); } #else void Raise(const char* role) { const int priority = RequestedMode() == ThreadPriorityMode::Realtime ? THREAD_PRIORITY_TIME_CRITICAL : THREAD_PRIORITY_HIGHEST; - if (SetThreadPriority(GetCurrentThread(), priority)) LogOnce(role, priority == THREAD_PRIORITY_TIME_CRITICAL ? "time-critical" : "highest"); - else LogOnce(role, "SetThreadPriority failed"); + if (!SetThreadPriority(GetCurrentThread(), priority)) { + throw std::runtime_error(std::string("APS5_THREAD_PRIORITY: ") + role + " SetThreadPriority failed (error " + std::to_string(GetLastError()) + ")"); + } + LogOnce(role, priority == THREAD_PRIORITY_TIME_CRITICAL ? "time-critical" : "highest"); } #endif diff --git a/core/libs/prx/libSceAgcDriver/tests/ThreadPriority.cpp b/core/libs/prx/libSceAgcDriver/tests/ThreadPriority.cpp index c7ea3a7a7..569852233 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ThreadPriority.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ThreadPriority.cpp @@ -1,5 +1,6 @@ #include "prx/libSceAgcDriver/Execution/include/ThreadPriority.hpp" #include +#include #include #include #include @@ -46,10 +47,26 @@ void PlanTests() { Require(kept.refusal.empty() && !kept.limit.has_value(), "limits rtkit already accepts must be left alone"); } +// Run with APS5_THREAD_PRIORITY=rt or =high and no reachable rtkit: the raise must throw, not settle for less. +void RefusalTest() { + bool threw = false; + try { + RaiseWorkerThreadPriority("test"); + } catch (const std::runtime_error& error) { + threw = true; + std::cout << "refused: " << error.what() << '\n'; + } + Require(threw, "a refused priority request must throw"); } -int main() { +} + +int main(int argc, char** argv) { try { + if (argc > 1 && std::string(argv[1]) == "refused") { + RefusalTest(); + return 0; + } ParseTests(); PlanTests(); } catch (const std::exception& error) { diff --git a/docs/dev/BUILD.md b/docs/dev/BUILD.md index fca71450f..0b06c87ed 100644 --- a/docs/dev/BUILD.md +++ b/docs/dev/BUILD.md @@ -92,7 +92,7 @@ Set `APS5_PIPELINE_STATS=1` to capture and print driver statistics for each newl ## Driver thread priority -Set `APS5_THREAD_PRIORITY=high` to raise the AGC driver's queue workers and the video output presenter above the title's threads. On Linux the threads get nice `APS5_THREAD_NICE` (default -10) through rtkit, or through `setpriority` when rtkit is not available; on Windows they get `THREAD_PRIORITY_HIGHEST`. `APS5_THREAD_PRIORITY=rt` asks rtkit for real-time priority `APS5_THREAD_RT` (default 10, at most rtkit's `MaxRealtimePriority`) under rtkit's policy, `SCHED_RR` by default, and falls back to the nice level; on Windows it sets `THREAD_PRIORITY_TIME_CRITICAL`. rtkit only grants it when the process's hard `RLIMIT_RTTIME` is at most its `RTTimeUSecMax` (200000 us by default), so a hard limit above `APS5_THREAD_RTTIME_US` (default 200000, capped at `RTTimeUSecMax`) is lowered to it and the soft limit to at most half of it. The hard limit cannot be raised again and stays lowered for the whole process even if rtkit then refuses; see [technical debt](TechnicalDebt.md#functional). Each thread logs the outcome once as `[thread-priority]`. The setting is disabled by default. +Set `APS5_THREAD_PRIORITY=high` to raise the AGC driver's queue workers and the video output presenter above the title's threads. On Linux the threads get nice `APS5_THREAD_NICE` (default -10) through rtkit; on Windows they get `THREAD_PRIORITY_HIGHEST`. `APS5_THREAD_PRIORITY=rt` asks rtkit for real-time priority `APS5_THREAD_RT` (default 10, at most rtkit's `MaxRealtimePriority`) under rtkit's policy, `SCHED_RR` by default; on Windows it sets `THREAD_PRIORITY_TIME_CRITICAL`. rtkit only grants it when the process's hard `RLIMIT_RTTIME` is at most its `RTTimeUSecMax` (200000 us by default), so a hard limit above `APS5_THREAD_RTTIME_US` (default 200000, capped at `RTTimeUSecMax`) is lowered to it and the soft limit to at most half of it. The hard limit cannot be raised again and stays lowered for the whole process even if rtkit then refuses; see [technical debt](TechnicalDebt.md#functional). There is no fallback between the modes: when rtkit or `SetThreadPriority` refuses, the thread throws with the refusal. Each raised thread logs its priority once as `[thread-priority]`. The setting is disabled by default. ## Shader recompiler diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index bf6fe4ed1..d3f7e128c 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -293,7 +293,7 @@ Throughout the project, every function at every stage either **does exactly what - [Compute LDS beyond the device's shared-memory limit](../../core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp) (libSceAgcDriver) lives in a coherent storage buffer with one slice of `LDS bytes / 4 + 1` dwords per workgroup (the last dword is the atomic lock), shared by all such dispatches and grown in 1 MiB steps up to `maxStorageBufferRange`; a dispatch whose slices do not fit throws. It is slower than shared memory and its cost was not measured. An indirect dispatch of such a program always reads its group counts on the CPU. Its results are pinned against the in-limit semantics in `agc_driver_lds_beyond_device_limit`, not measured on hardware. - [Graphics pipeline libraries](../../core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp) (libSceAgcDriver) are fast-linked only, never relinked with `VK_PIPELINE_CREATE_LINK_TIME_OPTIMIZATION_BIT_EXT`. A vertex stage then keeps the outputs its fragment stage does not read: on an RTX 4090 (NVIDIA 615.71) 13 of 15 PPSA36082 vertex shaders paired with a pixel shader that reads none of them used up to 68 registers instead of 38 and a binary up to 21 % larger, while pairs that read every output compiled to the same code as a whole pipeline (#2242). - [sceAvPlayerGetVideoData](../../core/libs/prx/libSceAvPlayer/src/Source.cpp) (libSceAvPlayer) in sync mode `SCE_AVPLAYER_AV_SYNC_MODE_NONE` skips queued frames that the audio clock has already passed while the title pulls audio, so a title that requests video slower than the film's frame rate keeps the picture with the sound. Whether the console hands out every frame in this mode is unknown. PPSA12544 requests 7-19 frames per second for a 30 fps cinematic -- [`APS5_THREAD_PRIORITY=rt`](../../core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp) (libSceAgcDriver) lowers the process's hard `RLIMIT_RTTIME` when it is above `APS5_THREAD_RTTIME_US` or rtkit's `RTTimeUSecMax`, because [rtkit](https://github.com/heftig/rtkit/blob/c295fa849f52b487be6433e69e08b46251950399/rtkit-daemon.c#L592-L637) (`verify_process_rttime`) reads the hard limit from `/proc//limits` and refuses real-time scheduling when it is higher. An unprivileged process cannot raise the hard limit again, so it stays lowered for the process and its threads even when rtkit then refuses the request (burst limit, user or start-time checks) and the threads fall back to a nice level. rtkit's `MaxRealtimePriority` and `RTTimeUSecMax` are read first, so an unreachable rtkit or a priority above its maximum leaves the limit untouched. The lowered limit only matters for threads that later run under a real-time policy +- [`APS5_THREAD_PRIORITY=rt`](../../core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp) (libSceAgcDriver) lowers the process's hard `RLIMIT_RTTIME` when it is above `APS5_THREAD_RTTIME_US` or rtkit's `RTTimeUSecMax`, because [rtkit](https://github.com/heftig/rtkit/blob/c295fa849f52b487be6433e69e08b46251950399/rtkit-daemon.c#L592-L637) (`verify_process_rttime`) reads the hard limit from `/proc//limits` and refuses real-time scheduling when it is higher. An unprivileged process cannot raise the hard limit again, so it stays lowered for the process and its threads even when rtkit then refuses the request (burst limit, user or start-time checks) and the thread throws. rtkit's `MaxRealtimePriority` and `RTTimeUSecMax` are read first, so an unreachable rtkit or a priority above its maximum leaves the limit untouched. The lowered limit only matters for threads that later run under a real-time policy - [libSceHttp2 asynchronous requests](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) never reach the network: `sceHttp2SendRequestAsync` accepts the request, queues a completion whose result is the library's network error and triggers the user event named by the kqueue option (equeue at offset 0, event id at 8, user data at 16, layout inferred from PPSA17221); `sceHttp2WaitAsync` returns that completion. The completion's `event_type` value is unknown and left 0. PPSA17221 completes its request object only after `sceHttp2SendRequestAsync` returns, and its handler for a synchronous failure calls `shared_from_this` before that, so a send must not fail synchronously. A kqueue option or user event that is missing, an extra option, and a wait with no completed operation throw - [system](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) runs no command. It follows FreeBSD 11's `lib/libc/stdlib/system.c` for a sandbox without `/bin/sh`: a null command returns 1 unconditionally, as that version does, and any command returns the wait status of a child that could not exec the shell and called `_exit(127)` (`0x7f00`). Whether the console's application sandbox has no shell, and whether `vfork` succeeds there (FreeBSD returns -1 when it fails), is unverified - [sceKernelWriteThrottlingStatus](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) reports no write throttling: the host SSD has no write budget, so the first qword of the 32-byte status is set to `UINT32_MAX` and the rest is zeroed. PPSA17221 reads only that qword: it multiplies it by 64 KiB and compares the product with byte thresholds up to 0x4AF00000 to pick one of five tiers. When the call fails the title assumes 0x4B000001 bytes, which selects the same top tier as `UINT32_MAX`. What the value measures on hardware (a remaining budget or a rate) and the meaning of the other three qwords are unknown From d5a396d92760c40bdbf0b8857e78d18b7e6a00b4 Mon Sep 17 00:00:00 2001 From: Scramblrrrr Date: Fri, 9 Oct 2026 19:50:29 -0300 Subject: [PATCH 028/460] fix(shader): preserve aliased embedded fetch attributes --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 13 ++++++++ .../tests/EmbeddedVertexFetchAliasing.cpp | 31 +++++++++++++++++++ .../Translation/src/EmbeddedVertexFetch.cpp | 5 +-- 3 files changed, 47 insertions(+), 2 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/EmbeddedVertexFetchAliasing.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 583e96a21..ba6719af7 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -1367,3 +1367,16 @@ if(BUILD_TESTING) "PATH=path_list_prepend:$") endif() endif() + +add_test_executable(agc_embedded_vertex_fetch_aliasing_tests EXCLUDE_FROM_ALL tests/EmbeddedVertexFetchAliasing.cpp) +target_link_libraries(agc_embedded_vertex_fetch_aliasing_tests PRIVATE shader_recompiler_agc_driver libc) +set_target_properties(agc_embedded_vertex_fetch_aliasing_tests PROPERTIES CXX_EXTENSIONS OFF) +configure_windows_unwind(agc_embedded_vertex_fetch_aliasing_tests) +if(BUILD_TESTING) + set_target_properties(agc_embedded_vertex_fetch_aliasing_tests PROPERTIES EXCLUDE_FROM_ALL OFF) + add_test(NAME agc_embedded_vertex_fetch_aliasing COMMAND agc_embedded_vertex_fetch_aliasing_tests) + if(WIN32) + set_tests_properties(agc_embedded_vertex_fetch_aliasing PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() +endif() diff --git a/core/libs/prx/libSceAgcDriver/tests/EmbeddedVertexFetchAliasing.cpp b/core/libs/prx/libSceAgcDriver/tests/EmbeddedVertexFetchAliasing.cpp new file mode 100644 index 000000000..3e38216df --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/EmbeddedVertexFetchAliasing.cpp @@ -0,0 +1,31 @@ +#include "RdnaDecoder/RdnaInstructionDecoder.hpp" +#include "Translation/EmbeddedVertexFetch.hpp" +#include +#include +#include +#include + +using namespace ShaderRecompiler; + +int main() { + int failures = 0; + for (const std::uint32_t waveSize : {32u, 64u}) { + for (const std::uint32_t indexRegister : {0u, 1u, 2u, 3u, 16u}) { + const std::array code{ + 0xf4000004u | (indexRegister << 6u), 0xfa00000cu, + 0xf4080005u, indexRegister << 25u, + 0x02140b08u, + 0xe00c2000u, 0x80000b0au, + 0xbf810000u, + }; + const auto decoded = RdnaInstructionDecoder{}.Decode(code); + const auto plan = EmbeddedVertexFetchAnalyzer{}.Analyze(decoded, 0, 2, 0, 16, waveSize); + if (plan.loads.size() != 1 || plan.loads.front().attributeId != 3 || plan.loads.front().componentCount != 4 || + plan.loads.front().programCounter != 20 || plan.loads.front().prologLoads != std::vector{0, 8}) { + std::fprintf(stderr, "wave%u descriptor offset s%u lost its attribute or load provenance\n", waveSize, indexRegister); + ++failures; + } + } + } + return failures == 0 ? 0 : 1; +} diff --git a/core/shader/recompiler/Translation/src/EmbeddedVertexFetch.cpp b/core/shader/recompiler/Translation/src/EmbeddedVertexFetch.cpp index a5ba98424..7bc692922 100644 --- a/core/shader/recompiler/Translation/src/EmbeddedVertexFetch.cpp +++ b/core/shader/recompiler/Translation/src/EmbeddedVertexFetch.cpp @@ -516,12 +516,13 @@ EmbeddedFetchPlan EmbeddedVertexFetchAnalyzer::Analyze(const RdnaProgram& progra dst.loadPcs.push_back(inst.programCounter); } } else if (isScalarOperand(inst.source1) && scalarSlot(inst.source1) < sgprs.size() && sgprs[scalarSlot(inst.source1)].kind == SgprValueKind::Attribute && (inst.memoryOffset & 0x3u) == 0u) { + const auto attribute = sgprs[scalarSlot(inst.source1)]; for (std::uint32_t i = 0u; i < decodedDstSize(inst) && slot + i < sgprs.size(); i++) { auto& dst = sgprs[slot + i]; dst = SgprValue{}; dst.kind = SgprValueKind::Buffer; - dst.attributeId = sgprs[scalarSlot(inst.source1)].attributeId; - dst.loadPcs = sgprs[scalarSlot(inst.source1)].loadPcs; + dst.attributeId = attribute.attributeId; + dst.loadPcs = attribute.loadPcs; dst.loadPcs.push_back(inst.programCounter); } } else { From 2fb7a77acc1cd24f08f89ec6708add6770279a94 Mon Sep 17 00:00:00 2001 From: Dean Galvin Date: Sat, 10 Oct 2026 23:45:38 +1300 Subject: [PATCH 029/460] feat(shader): note which guest buffer elements a shader loads from DescriptorBinding gains bufferRead next to bufferWritten and bufferAtomic, filled from the resource tracker's per-buffer read flag and kept in the shader disk cache (format 24). --- core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp | 5 +++-- .../recompiler/Optimization/src/DescriptorBindingBuilder.cpp | 1 + core/shader/recompiler/Recompiler.hpp | 1 + core/shader/recompiler/ShaderDiskCache.cpp | 4 +++- core/shader/recompiler/ShaderDiskCache.hpp | 2 +- 5 files changed, 9 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp b/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp index 94fa9b9a3..0aeb9425e 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp @@ -41,7 +41,7 @@ void setEnvironment(const char* name, const std::string& value) { } bool sameBinding(const DescriptorBinding& left, const DescriptorBinding& right) { - return left.kind == right.kind && left.role == right.role && left.descriptorSet == right.descriptorSet && left.binding == right.binding && left.count == right.count && left.guestDescriptor == right.guestDescriptor && left.readOnly == right.readOnly && left.imageShape == right.imageShape && left.samplerDepthCompare == right.samplerDepthCompare && left.imageWritten == right.imageWritten && left.imageDepthCompare == right.imageDepthCompare && left.imageAtomic == right.imageAtomic && left.bufferAtomic == right.bufferAtomic && left.bufferWritten == right.bufferWritten && left.samplerUnnormalized == right.samplerUnnormalized && left.imageUnnormalized == right.imageUnnormalized && left.imageSamplers == right.imageSamplers; + return left.kind == right.kind && left.role == right.role && left.descriptorSet == right.descriptorSet && left.binding == right.binding && left.count == right.count && left.guestDescriptor == right.guestDescriptor && left.readOnly == right.readOnly && left.imageShape == right.imageShape && left.samplerDepthCompare == right.samplerDepthCompare && left.imageWritten == right.imageWritten && left.imageDepthCompare == right.imageDepthCompare && left.imageAtomic == right.imageAtomic && left.bufferAtomic == right.bufferAtomic && left.bufferWritten == right.bufferWritten && left.bufferRead == right.bufferRead && left.samplerUnnormalized == right.samplerUnnormalized && left.imageUnnormalized == right.imageUnnormalized && left.imageSamplers == right.imageSamplers; } bool sameBindings(const std::vector& left, const std::vector& right) { @@ -121,6 +121,7 @@ DescriptorBinding sampleBinding(std::uint32_t seed) { binding.samplerUnnormalized = {false, true, true}; binding.imageUnnormalized = {true, false, seed % 2 == 0}; binding.imageSamplers = {0x5u, 0u, 0x80000000u}; + binding.bufferRead = {true, false, false, true, true, false}; return binding; } @@ -755,7 +756,7 @@ void verifyBindingPlanSelection() { BindingAllocationResult full; builder.Populate(full, compiled, plan, 0u, snapshot, {}); require(full.bindings.size() == 4u, "binding plan lost a descriptor before module selection"); - require(full.bindings[0].bufferAtomic == std::vector{true, false} && full.bindings[0].bufferWritten == std::vector{true, false}, "binding plan lost buffer access metadata"); + require(full.bindings[0].bufferAtomic == std::vector{true, false} && full.bindings[0].bufferWritten == std::vector{true, false} && full.bindings[0].bufferRead == std::vector{false, true}, "binding plan lost buffer access metadata"); const std::array liveBindings{full.bindings[0].binding, full.bindings[0].binding, full.bindings[3].binding}; const auto selected = builder.Select(plan, liveBindings); snapshot.userData.clear(); diff --git a/core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp b/core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp index 0c06248de..7455a1384 100644 --- a/core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp +++ b/core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp @@ -371,6 +371,7 @@ DescriptorBindingPlan DescriptorBindingBuilder::Prepare(const IrBindingLayout& l const auto& buffer = info.buffers.at(resource); physical.bufferAtomic.push_back(buffer.atomic); physical.bufferWritten.push_back(buffer.written || buffer.atomic); + physical.bufferRead.push_back(buffer.read); if (buffer.written || buffer.atomic) ++entry.writtenBuffers; else ++entry.readOnlyBuffers; } diff --git a/core/shader/recompiler/Recompiler.hpp b/core/shader/recompiler/Recompiler.hpp index a8a342f56..098509efb 100644 --- a/core/shader/recompiler/Recompiler.hpp +++ b/core/shader/recompiler/Recompiler.hpp @@ -362,6 +362,7 @@ struct DescriptorBinding { std::vector samplerUnnormalized; std::vector imageUnnormalized; std::vector imageSamplers; + std::vector bufferRead; }; struct VertexAttribute { diff --git a/core/shader/recompiler/ShaderDiskCache.cpp b/core/shader/recompiler/ShaderDiskCache.cpp index cb69e754d..8a5d0d497 100644 --- a/core/shader/recompiler/ShaderDiskCache.cpp +++ b/core/shader/recompiler/ShaderDiskCache.cpp @@ -67,7 +67,7 @@ namespace ShaderRecompiler::ShaderDiskCache { static_assert(sizeof(CompiledShaderArtifact) == 192, "CompiledShaderArtifact changed: update the artifact encoder"); static_assert(sizeof(ShaderInvocation) == 112, "ShaderInvocation changed: update the invocation encoder"); static_assert(sizeof(RecompileResult) == 312, "RecompileResult changed: update EncodeResult and DecodeResult"); -static_assert(sizeof(DescriptorBinding) == 448, "DescriptorBinding changed: update the binding encoder"); +static_assert(sizeof(DescriptorBinding) == 488, "DescriptorBinding changed: update the binding encoder"); static_assert(sizeof(VertexAttribute) == 32, "VertexAttribute changed: update the attribute encoder"); static_assert(sizeof(VertexInput) == 16, "VertexInput changed: update the vertex input encoder"); static_assert(sizeof(FragmentParameter) == 12, "FragmentParameter changed: update the parameter encoder"); @@ -257,6 +257,7 @@ void encodeBinding(Writer& writer, const DescriptorBinding& binding) { writer.Flags(binding.samplerUnnormalized); writer.Flags(binding.imageUnnormalized); writer.Values(std::span(binding.imageSamplers)); + writer.Flags(binding.bufferRead); } void decodeBinding(Reader& reader, DescriptorBinding& binding) { @@ -280,6 +281,7 @@ void decodeBinding(Reader& reader, DescriptorBinding& binding) { reader.Flags(binding.samplerUnnormalized); reader.Flags(binding.imageUnnormalized); reader.Values(binding.imageSamplers); + reader.Flags(binding.bufferRead); } void encodeArtifact(Writer& writer, const CompiledShaderArtifact& result) { diff --git a/core/shader/recompiler/ShaderDiskCache.hpp b/core/shader/recompiler/ShaderDiskCache.hpp index aa0651861..6111e47c1 100644 --- a/core/shader/recompiler/ShaderDiskCache.hpp +++ b/core/shader/recompiler/ShaderDiskCache.hpp @@ -12,7 +12,7 @@ namespace ShaderRecompiler::ShaderDiskCache { -inline constexpr std::uint32_t FormatVersion = 23; +inline constexpr std::uint32_t FormatVersion = 24; enum class LoadStatus { Loaded, From 2688b76463dd7669f875d24b436b2a8cd9a32345 Mon Sep 17 00:00:00 2001 From: Dean Galvin Date: Sat, 10 Oct 2026 23:46:03 +1300 Subject: [PATCH 030/460] perf(agc): stage read and written buffer elements a dispatch sweeps A written V# element bound in place in a host import makes every load a PCIe round trip. A particle update that loads and stores a 10 MiB pool (2048x64 threads over 131072 records of 80 bytes), above the 2 MiB written window, took about 4.8 ms of GPU per frame. An element the shader also reads (DescriptorBinding::bufferRead), written by a direct dispatch whose threads times the V# stride cover at least half of it, now takes device-local staging up to 16 MiB: that dispatch drops to about 0.06 ms plus 0.9 ms of bulk copies. The coverage test keeps a kernel that updates about 12k records of a 16 MiB pool in place, where staging all of it would cost 1.3 ms of copies for a 0.08 ms kernel. Write-only elements, indirect dispatches and raw (stride 0) buffers keep the 2 MiB window. --- .../Execution/include/VulkanDevice.hpp | 2 +- .../src/Driver/Dispatch/Dispatch.cpp | 3 +- .../Execution/src/VulkanDevice.cpp | 8 +- .../Graphics/include/GuestBufferMemory.hpp | 5 +- .../Graphics/include/ShaderResources.hpp | 5 +- .../Graphics/src/GuestBufferMemory.cpp | 14 ++-- .../Graphics/src/ShaderResources.cpp | 10 ++- .../prx/libSceAgcDriver/tests/Recorder.cpp | 74 +++++++++++++++++++ 8 files changed, 103 insertions(+), 18 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp index 4144b30fa..1cd29e02f 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp @@ -219,7 +219,7 @@ public: // records. Null when nothing is prepared: the resource cache may serve the dispatch (its // Revalidate stays under the mutex), or APS5_LOCKED_BUILD=1 keeps the whole build under it as // before. `shader` and `snapshots` must outlive the dispatch. - std::shared_ptr PrepareDispatch(const ShaderRecompiler::RecompileResult& shader, std::span snapshots); + std::shared_ptr PrepareDispatch(const ShaderRecompiler::RecompileResult& shader, std::span snapshots, std::uint64_t dispatchThreads = 0); // APS5_PROFILE_DRAW: the parts of a PrepareDispatch in milliseconds, in the order key, find, // precollect, presync, stage A (the driver's 'prepare:' rows). static std::span PreparePhaseMs(const PreparedDispatch& prepared); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp index d95362e25..cb71f04dd 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp @@ -321,6 +321,7 @@ void Driver::dispatch(QueueState& queue, std::span packet, groups[axis] = (groups[axis] + threads - 1) / threads; } } + const std::uint64_t dispatchThreads = indirectArguments != 0 ? 0 : std::uint64_t{groups[0]} * groups[1] * groups[2] * compute.numThreads[0] * compute.numThreads[1] * compute.numThreads[2]; static const bool traceIo = std::getenv("APS5_TRACE_DISPATCH_IO") != nullptr; if (traceIo) { @@ -363,7 +364,7 @@ void Driver::dispatch(QueueState& queue, std::span packet, std::shared_ptr prepared; if (recipeHit == nullptr || VulkanDevice::VerifyRecipes()) { try { - prepared = localDevice->PrepareDispatch(compiled, snapshots); + prepared = localDevice->PrepareDispatch(compiled, snapshots, dispatchThreads); } catch (const std::exception& error) { if (rethrow(error)) return; } diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index 7cfd5f2b5..39f46cf57 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -2815,6 +2815,7 @@ struct PreparedDispatch { // APS5_PROFILE_DRAW: stage A's time, added to the [dispatch] phase totals by the dispatch, and // the prepare's parts for the driver's 'prepare:' rows (PreparePhase order). double prepareMs = 0; + std::uint64_t dispatchThreads = 0; enum PreparePhase : std::size_t { PrepareKey, PrepareFind, PreparePrecollect, PreparePresync, PrepareStageA, PreparePhaseCount }; std::array phaseMs{}; // The stage-A pre-sync (see PrepareDispatch): the newest recorder serial waited for without the @@ -3258,7 +3259,7 @@ std::uint64_t VulkanDevice::presync(std::span VulkanDevice::PrepareDispatch(const ShaderRecompiler::RecompileResult& shader, std::span snapshots) { +std::shared_ptr VulkanDevice::PrepareDispatch(const ShaderRecompiler::RecompileResult& shader, std::span snapshots, std::uint64_t dispatchThreads) { ShaderRecompiler::RuntimeAbi::RequireVersion(shader.runtimeAbiVersion); // APS5_LOCKED_BUILD=1: the whole build under the mutex, as before the split. static const bool lockedBuild = std::getenv("APS5_LOCKED_BUILD") != nullptr; @@ -3268,6 +3269,7 @@ std::shared_ptr VulkanDevice::PrepareDispatch(const ShaderReco const Graphics::CompiledShader compute{ShaderRecompiler::ShaderStage::Compute, &shader, 0}; const auto context = graphicsContext(); auto prepared = std::make_shared(); + prepared->dispatchThreads = dispatchThreads; // A cached build (the map locks itself) is revalidated under the mutex by the dispatch, which // looks it up again by the key made here; the object stays local, only its surfaces matter // below. Anything inserted between here and the dispatch is simply replaced by this build. @@ -3287,7 +3289,7 @@ std::shared_ptr VulkanDevice::PrepareDispatch(const ShaderReco phase(PreparedDispatch::PrepareFind); } if (cached == nullptr) { - prepared->resources = std::make_shared(context, compute, snapshots, true); + prepared->resources = std::make_shared(context, compute, snapshots, true, dispatchThreads); phase(PreparedDispatch::PrepareStageA); } if (cached != nullptr && CachedPrecollect()) { @@ -3698,7 +3700,7 @@ VulkanDevice::IndirectOutcome VulkanDevice::dispatch(const ShaderRecompiler::Rec } if (resources == nullptr) { const auto buildStart = profile ? std::chrono::steady_clock::now() : std::chrono::steady_clock::time_point{}; - resources = std::make_shared(context, shaders[0], snapshots); + resources = std::make_shared(context, shaders[0], snapshots, false, prepared != nullptr ? prepared->dispatchThreads : 0); if (profile) timer.add(PhaseResourcesFullBuild, std::chrono::duration(std::chrono::steady_clock::now() - buildStart).count()); if (cacheable) { ++d.cacheMisses; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp index 7e77f95f4..11f7df91e 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp @@ -183,7 +183,7 @@ public: // (DescriptorBinding::bufferWritten): the CPU never has to wait for it. `atomic` marks an // element the shader updates atomically (DescriptorBinding::bufferAtomic), staged in device // memory whatever its size (see AllowDeviceStaging). - void AddWritable(std::uint64_t address, std::size_t bytes, bool atomic = false); + void AddWritable(std::uint64_t address, std::size_t bytes, bool atomic = false, bool swept = false); void AddReadable(std::uint64_t address, std::size_t bytes); // Device-local staging of written and atomic elements inside host imports (see // GuestBufferMemory.cpp): allowed only for a build whose every use records its work and then @@ -294,6 +294,7 @@ private: bool copiedBack = false; // An element the shader updates atomically lies inside (AddWritable's `atomic`). bool atomic = false; + bool swept = false; // The gpuCopy buffer is a device-local staging shadow (see stagingEligible): no host // mapping, so nothing is ever stored from it by the CPU, and the region is taken even when // the import could bind it in place. @@ -327,7 +328,7 @@ private: // The region of a descriptor-bound range (AddWritable/AddReadable), committed pages only. A // range inside a base region of the space adds nothing: the region serves it, as today's merge // of the two did. - void addDescriptorRegion(std::uint64_t address, std::size_t bytes, bool atomic); + void addDescriptorRegion(std::uint64_t address, std::size_t bytes, bool atomic, bool swept); // Gives a region a buffer of its own with its bytes (guest memory for host-backed and writable // ranges, plus the write-back's reference copy for a range a descriptor writes; else its snapshot). void copyRegion(Region& region, bool addressable); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp index db2b505ec..41fb292a5 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp @@ -109,7 +109,7 @@ public: // then no other member may be used. `compute` and `snapshots` must outlive Complete(). An // address-based shader (BDA tables) builds entirely in Complete(): its lease acquisition // reconciles imports and refreshes mirrors, which needs the lock. - ShaderResources(const Context& context, const CompiledShader& compute, std::span snapshots, bool deferred); + ShaderResources(const Context& context, const CompiledShader& compute, std::span snapshots, bool deferred, std::uint64_t dispatchThreads = 0); void Complete(); bool Completed() const { return completed; } ~ShaderResources(); @@ -304,7 +304,7 @@ private: // `written` is the element's DescriptorBinding::bufferWritten: a read-only element binds the // same way but is left out of the write set (no write-back, no pending-write note). `atomic` // is its bufferAtomic (see GuestBufferMemory::AddWritable). - std::size_t addGuestBuffer(std::span words, const ColorTarget* target, std::uint64_t indexAddress, std::size_t indexBytes, bool written, bool atomic); + std::size_t addGuestBuffer(std::span words, const ColorTarget* target, std::uint64_t indexAddress, std::size_t indexBytes, bool written, bool atomic, bool read); std::size_t addDataBuffer(std::span words); // Stage A: the layout entry of an image binding (samplers are taken at once, the sampler cache // locks itself); stage B looks the sampled textures and storage images up (resolveImageBinding). @@ -422,6 +422,7 @@ private: // notes (counted in MarkGpuWrites for the [buffers] line). std::size_t readOnlyBuffers = 0; bool drawBuild = false; + std::uint64_t dispatchThreads = 0; std::vector> textures; std::vector textureFirstLayer; std::vector> storageTextures; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp index 7cc25556b..a1baa95dd 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp @@ -1997,7 +1997,7 @@ const GuestBufferMemory::Region* GuestBufferMemory::owner(std::uint64_t address) return own; } -void GuestBufferMemory::addDescriptorRegion(std::uint64_t address, std::size_t bytes, bool atomic) { +void GuestBufferMemory::addDescriptorRegion(std::uint64_t address, std::size_t bytes, bool atomic, bool swept) { validate(address, bytes); const auto begin = address & ~std::uint64_t{3}; const auto end = begin == address ? address + bytes : (address + bytes + 3) & ~std::uint64_t{3}; @@ -2015,6 +2015,7 @@ void GuestBufferMemory::addDescriptorRegion(std::uint64_t address, std::size_t b // the shader stores to them; what is written back is decided by Writes() alone. Region region{begin, end, true, {}, nullptr}; region.atomic = atomic; + region.swept = swept; auto committed = GuestMemory::DescribeCommitted(begin, static_cast(end - begin)); if (!committed.whole) { // A GPU heap bound whole while the guest commits its pages on demand, or a descriptor left @@ -2025,8 +2026,8 @@ void GuestBufferMemory::addDescriptorRegion(std::uint64_t address, std::size_t b regionsSorted = false; } -void GuestBufferMemory::AddWritable(std::uint64_t address, std::size_t bytes, bool atomic) { - addDescriptorRegion(address, bytes, atomic); +void GuestBufferMemory::AddWritable(std::uint64_t address, std::size_t bytes, bool atomic, bool swept) { + addDescriptorRegion(address, bytes, atomic, swept); writes.emplace_back(address, address + bytes); } @@ -2034,7 +2035,7 @@ void GuestBufferMemory::AddReadable(std::uint64_t address, std::size_t bytes) { // Not in `writes`: no reference copy (copyRegion), no write-back, no pending-write note, no // direct-write mark, and UploadPrepare copies it without the device lock. A written descriptor // overlapping the range still covers it through its own Writes() entry. - addDescriptorRegion(address, bytes, false); + addDescriptorRegion(address, bytes, false, false); } void GuestBufferMemory::AddSnapshot(const GuestMemorySnapshot& snapshot) { @@ -2157,6 +2158,8 @@ VkBufferUsageFlags gpuCopyUsage(bool addressable) { // APS5_WRITTEN_SHADOW_MIN_KIB / APS5_WRITTEN_SHADOW_MAX_KIB (16 / 2048) bound the written window // (a kernel streaming once through a large buffer, the engine's memcpy kernel over 4-8 MiB video // frames, would only gain the two copies), APS5_ATOMIC_STAGE_MAX_KIB (1024) the atomic one. +constexpr std::uint64_t SweptShadowMax = std::uint64_t{16} << 20u; + bool writtenShadowEnabled() { static const bool disabled = std::getenv("APS5_NO_WRITTEN_SHADOW") != nullptr; return !disabled; @@ -2297,7 +2300,7 @@ bool GuestBufferMemory::stagingEligible(const Region& region, bool addressable) const auto bytes = region.end - region.begin; if (!WritesOverlap(region.begin, static_cast(bytes))) return false; if (region.atomic && atomicStagingEnabled() && bytes <= atomicStageMax()) return true; - return writtenShadowEnabled() && bytes >= writtenShadowMin() && bytes <= writtenShadowMax(); + return writtenShadowEnabled() && bytes >= writtenShadowMin() && bytes <= (region.swept ? std::max(writtenShadowMax(), SweptShadowMax) : writtenShadowMax()); } void GuestBufferMemory::UploadPrepare(bool addressable) { @@ -2338,6 +2341,7 @@ void GuestBufferMemory::UploadPrepare(bool addressable) { mergeBacked(previous, region); if (region.end > previous.end) previous.direct = nullptr; previous.atomic = previous.atomic || region.atomic; + previous.swept = previous.swept || region.swept; // A range starting before the mirror (only possible after the swap) keeps its prefix; the // earlier merged region ends at or before it, so the merged list stays sorted. previous.begin = std::min(previous.begin, region.begin); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index e480bc288..a05fd4a3a 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -1003,7 +1003,7 @@ bool UsesAddressTables(const CompiledShader& compute) { } -ShaderResources::ShaderResources(const Context& context, const CompiledShader& compute, std::span snapshots, bool deferred) : context(context), guestMemory(context), deferredCompute(compute), deferredSnapshots(snapshots) { +ShaderResources::ShaderResources(const Context& context, const CompiledShader& compute, std::span snapshots, bool deferred, std::uint64_t dispatchThreads) : context(context), guestMemory(context), dispatchThreads(dispatchThreads), deferredCompute(compute), deferredSnapshots(snapshots) { Require(compute.stage == ShaderRecompiler::ShaderStage::Compute, "compute resources require a compute shader"); // Every use of a compute build is a recorded dispatch that calls MarkGpuWrites, which staged // buffers need (a synchronous draw's use would not). @@ -1268,7 +1268,8 @@ void ShaderResources::buildPrepare(std::span shaders, cons // vector) counts as written, like imageWritten below. const bool written = element >= binding.bufferWritten.size() || binding.bufferWritten[element]; const bool atomic = element < binding.bufferAtomic.size() && binding.bufferAtomic[element]; - const auto index = addGuestBuffer(std::span(binding.guestDescriptor).subspan(static_cast(element) * 4, 4), target, indexAddress, indexBytes, written, atomic); + const bool read = element < binding.bufferRead.size() && binding.bufferRead[element]; + const auto index = addGuestBuffer(std::span(binding.guestDescriptor).subspan(static_cast(element) * 4, 4), target, indexAddress, indexBytes, written, atomic, read); const auto& push = shader.program->pushConstants; if (!push.empty()) { const auto position = shader.program->memoryOffsetDword * 4u + element; @@ -2590,7 +2591,7 @@ DescriptorCache::Stats DescriptorCache::Counters() const { return stats; } -std::size_t ShaderResources::addGuestBuffer(std::span words, const ColorTarget* target, std::uint64_t indexAddress, std::size_t indexBytes, bool written, bool atomic) { +std::size_t ShaderResources::addGuestBuffer(std::span words, const ColorTarget* target, std::uint64_t indexAddress, std::size_t indexBytes, bool written, bool atomic, bool read) { Require(words.size() == 4, "buffer descriptor must contain four DWORDs"); Require((words[1] & 0x40000000u) == 0, "buffer descriptor has reserved bits set"); const ShaderRecompiler::ShaderBufferResource descriptor{{words[0], words[1], words[2], words[3]}}; @@ -2609,7 +2610,8 @@ std::size_t ShaderResources::addGuestBuffer(std::span words static const bool allWritten = std::getenv("APS5_ALL_BUFFERS_WRITTEN") != nullptr; Require(!written || !overlap(address, size, indexAddress, indexBytes), "writable shader buffer aliases the index buffer"); written = written || allWritten; - if (written) guestMemory.AddWritable(address, size, atomic); + const bool swept = read && dispatchThreads != 0 && descriptor.Stride() != 0 && dispatchThreads * descriptor.Stride() * 2u >= byteSize; + if (written) guestMemory.AddWritable(address, size, atomic, swept); else { guestMemory.AddReadable(address, size); ++readOnlyBuffers; diff --git a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp index 34dc7009f..8ff1b26c0 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp @@ -1144,6 +1144,79 @@ void resourceReadTests(const Device& device, Recorder& recorder) { HostImportFor(context, address, bytes); } +void readWrittenStagingTests(const Device& device, Recorder& recorder) { + const auto& context = device.GetContext(); + if (context.hostImportAlignment == 0) { + std::cout << "host imports unavailable: read and written staging not tested\n"; + return; + } + constexpr std::size_t bytes = 8u << 20u; +#ifdef _WIN32 + void* block = VirtualAlloc(nullptr, bytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE); +#else + void* block = std::aligned_alloc(65536, bytes); +#endif + Require(block != nullptr, "cannot allocate the test block"); + std::memset(block, 0x11, bytes); + const auto address = reinterpret_cast(block); + { + GuestAllocations::Mutation mutation; + mutation.Add(block, bytes, true, true, true); + } + if (HostImportFor(context, address, bytes) == nullptr) { + std::cout << "host import of the test block refused: read and written staging not tested\n"; + return; + } + const auto boundInPlace = [&](std::uint64_t element, std::size_t elementBytes, bool read) { + GuestBufferMemory memory(context); + memory.AllowDeviceStaging(); + memory.AddWritable(element, elementBytes, false, read); + memory.Upload(false); + const auto reads = memory.InPlaceReads(); + const bool direct = reads.size() == 1 && reads[0].first == element && reads[0].second == element + elementBytes; + Require(direct || reads.empty(), "an element was bound neither in place nor staged"); + memory.RecordCopyBacks(recorder); + recorder.Sync(); + return direct; + }; + Require(!boundInPlace(address + 4096, 1u << 20u, false), "a written 1 MiB element was not staged"); + Require(boundInPlace(address + 4096, 4u << 20u, false), "a written-only 4 MiB element was staged past the written window"); + Require(!boundInPlace(address + 4096, 4u << 20u, true), "a read and written 4 MiB element was bound in place"); + const auto buildInPlace = [&](std::uint64_t dispatchThreads, bool read) { + const auto element = address + 4096; + constexpr std::uint32_t stride = 16; + constexpr std::uint32_t records = (4u << 20u) / stride; + ShaderRecompiler::RecompileResult program; + ShaderRecompiler::DescriptorBinding binding; + binding.kind = ShaderRecompiler::DescriptorKind::StorageBuffer; + binding.role = ShaderRecompiler::DescriptorRole::GuestBuffers; + binding.descriptorSet = 0; + binding.binding = 0; + binding.count = 1; + binding.guestDescriptor = {static_cast(element), (static_cast(element >> 32u) & 0xffffu) | (stride << 16u), records, 0x31000000u}; + binding.bufferWritten = {true}; + binding.bufferRead = {read}; + program.bindings.push_back(binding); + const CompiledShader compute{ShaderRecompiler::ShaderStage::Compute, &program, 0}; + ShaderResources resources(context, compute, {}, false, dispatchThreads); + const auto reads = resources.InPlaceReads(); + const bool direct = std::any_of(reads.begin(), reads.end(), [&](const auto& range) { return range.first <= element && range.second >= element + records * stride; }); + resources.MarkGpuWrites(recorder); + recorder.Sync(); + return direct; + }; + Require(!buildInPlace((4u << 20u) / 16u, true), "a dispatch sweeping a read and written 4 MiB element bound it in place"); + Require(buildInPlace(1024, true), "a dispatch touching 16 KiB of a read and written 4 MiB element staged all of it"); + Require(buildInPlace(0, true), "a dispatch of unknown size staged a read and written 4 MiB element"); + Require(buildInPlace((4u << 20u) / 16u, false), "a dispatch sweeping a written-only 4 MiB element staged it"); + Require(std::all_of(static_cast(block), static_cast(block) + bytes, [](std::byte value) { return value == std::byte{0x11}; }), "a staged copy-back changed the guest bytes"); + { + GuestAllocations::Mutation mutation; + mutation.Remove(block); + } + HostImportFor(context, address, bytes); +} + void misalignedRegionTests(const Device& device, Recorder& recorder) { const auto& context = device.GetContext(); const auto alignment = context.limits.minStorageBufferOffsetAlignment; @@ -3887,6 +3960,7 @@ int main(int argc, char** argv) { closeRaceTests(device, recorder); keyProofTests(device, recorder); resourceReadTests(device, recorder); + readWrittenStagingTests(device, recorder); misalignedSnapshotTests(device, recorder); misalignedRegionTests(device, recorder); drawSnapshotReuseTests(device, recorder); From b7e32b3648f901ffb365b5c93233fdefc8793cc6 Mon Sep 17 00:00:00 2001 From: Dextroid17 <217507631+Dextroid17@users.noreply.github.com> Date: Thu, 8 Oct 2026 21:22:39 -0400 Subject: [PATCH 031/460] fix(shader): decode the scalar aperture sources 235-238 instead of failing the shader --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 8 ++ core/libs/tests/ApertureSources.cpp | 89 +++++++++++++++++++ .../include/RdnaDecoder/RdnaInstruction.hpp | 6 +- .../src/RdnaInstructionDecoder.cpp | 8 ++ .../RdnaDecoder/src/RdnaMemoryOpDecoder.cpp | 4 + .../src/TranslationContext/OperandAccess.cpp | 12 +++ docs/dev/TechnicalDebt.md | 1 + 7 files changed, 127 insertions(+), 1 deletion(-) create mode 100644 core/libs/tests/ApertureSources.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 583e96a21..4d1478a8a 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -1225,6 +1225,14 @@ if(BUILD_TESTING) set_tests_properties(scalar_scratch_load PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(aperture_sources_tests ${LIBS_INCLUDE_DIR}/tests/ApertureSources.cpp) + target_link_libraries(aperture_sources_tests PRIVATE shader_recompiler_agc_driver libc) + configure_windows_unwind(aperture_sources_tests) + add_test(NAME aperture_sources COMMAND aperture_sources_tests) + if(WIN32) + set_tests_properties(aperture_sources PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(scalar_call_tests ${LIBS_INCLUDE_DIR}/tests/ScalarCall.cpp) target_link_libraries(scalar_call_tests PRIVATE shader_recompiler_agc_driver libc) configure_windows_unwind(scalar_call_tests) diff --git a/core/libs/tests/ApertureSources.cpp b/core/libs/tests/ApertureSources.cpp new file mode 100644 index 000000000..f7e05eda1 --- /dev/null +++ b/core/libs/tests/ApertureSources.cpp @@ -0,0 +1,89 @@ +#include "IntermediateRepresentation/IrProgram.hpp" +#include "RdnaDecoder/RdnaInstructionDecoder.hpp" +#include "RdnaDecoder/RdnaScalarOpDecoder.hpp" +#include "Translation/TranslationContext.hpp" +#include +#include +#include +#include + +using namespace ShaderRecompiler; + +namespace { + +struct ApertureSource { + std::uint32_t code; + RdnaOperandKind kind; + const char* name; +}; + +const std::array ApertureSources{{ + {235u, RdnaOperandKind::SrcSharedBase, "src_shared_base"}, + {236u, RdnaOperandKind::SrcSharedLimit, "src_shared_limit"}, + {237u, RdnaOperandKind::SrcPrivateBase, "src_private_base"}, + {238u, RdnaOperandKind::SrcPrivateLimit, "src_private_limit"}, +}}; + +void Require(bool value, const std::string& message) { + if (!value) throw std::runtime_error(message); +} + +// The refusal must name the source it refused: a bare "not modelled" cannot be told from a decode gap. +template +void ExpectRefusal(const char* source, Fn translate) { + try { + translate(); + } catch (const std::exception& error) { + if (std::string(error.what()).find(source) == std::string::npos) { + throw std::runtime_error(std::string("the refusal does not name ") + source + ": " + error.what()); + } + return; + } + throw std::runtime_error(std::string("reading ") + source + " was accepted"); +} + +// s_mov_b32 s0, : SOP1, format 0x7d, opcode 0x03, the source code in the low byte. +RdnaInstruction ScalarMove(std::uint32_t source) { + const std::array words{0x80000000u | (0x7du << 23u) | (0x03u << 8u) | source, 0u}; + return DecodeRdnaSop1(0u, words, 0u); +} + +void Translate(const RdnaInstruction& instruction) { + IrProgram program; + auto& block = program.CreateBlock(); + program.SetEntryBlock(block); + program.BlockOrder() = {&block}; + program.Metadata().blockInfo.resize(1); + TranslationContext context(program, block, 256); + context.TranslateInstruction(instruction); +} + +void CheckDecode() { + for (const auto& source : ApertureSources) { + const auto operand = DecodeRdnaScalarSource(source.code, 0u); + Require(operand.kind == source.kind, + "scalar source " + std::to_string(source.code) + " decoded to the wrong operand kind"); + Require(RdnaOperandToString(operand) == source.name, + "scalar source " + std::to_string(source.code) + " is named " + RdnaOperandToString(operand)); + } +} + +// Translating the move reads the source as a value, which is the path a title that carries the +// aperture into an address does not take and one that reads it does. +void CheckReadRefuses() { + for (const auto& source : ApertureSources) { + const RdnaInstruction move = ScalarMove(source.code); + Require(move.op == RdnaOpcode::SMovB32, "the scalar move decoded to another opcode"); + Require(move.source0.kind == source.kind, "the scalar move decodes source0 as the wrong operand kind"); + ExpectRefusal(source.name, [&] { Translate(move); }); + } +} + +} + +int main() { + CheckDecode(); + CheckReadRefuses(); + std::puts("aperture sources tests passed"); + return 0; +} diff --git a/core/shader/recompiler/RdnaDecoder/include/RdnaDecoder/RdnaInstruction.hpp b/core/shader/recompiler/RdnaDecoder/include/RdnaDecoder/RdnaInstruction.hpp index f038fdc0f..85dc6da28 100644 --- a/core/shader/recompiler/RdnaDecoder/include/RdnaDecoder/RdnaInstruction.hpp +++ b/core/shader/recompiler/RdnaDecoder/include/RdnaDecoder/RdnaInstruction.hpp @@ -53,7 +53,11 @@ enum class RdnaOperandKind { VccZ, ExecZ, M0, - PopsExitingWaveId + PopsExitingWaveId, + SrcSharedBase, + SrcSharedLimit, + SrcPrivateBase, + SrcPrivateLimit }; enum class RdnaImageDimension : std::uint32_t { diff --git a/core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp b/core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp index f03944673..fc5e2b9d6 100644 --- a/core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp +++ b/core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp @@ -217,6 +217,10 @@ RdnaOperand DecodeRdnaScalarSource(std::uint32_t code, std::uint32_t programCoun case 125u: operand.kind = RdnaOperandKind::Null; return operand; case 126u: operand.kind = RdnaOperandKind::ExecLo; return operand; case 127u: operand.kind = RdnaOperandKind::ExecHi; return operand; + case 235u: operand.kind = RdnaOperandKind::SrcSharedBase; return operand; + case 236u: operand.kind = RdnaOperandKind::SrcSharedLimit; return operand; + case 237u: operand.kind = RdnaOperandKind::SrcPrivateBase; return operand; + case 238u: operand.kind = RdnaOperandKind::SrcPrivateLimit; return operand; case 239u: operand.kind = RdnaOperandKind::PopsExitingWaveId; return operand; case 248u: operand.kind = RdnaOperandKind::FloatInlineConstant; @@ -315,6 +319,10 @@ std::string RdnaOperandToString(const RdnaOperand& operand) { case RdnaOperandKind::ExecZ: return "execz"; case RdnaOperandKind::M0: return "m0"; case RdnaOperandKind::PopsExitingWaveId: return "pops_exiting_wave_id"; + case RdnaOperandKind::SrcSharedBase: return "src_shared_base"; + case RdnaOperandKind::SrcSharedLimit: return "src_shared_limit"; + case RdnaOperandKind::SrcPrivateBase: return "src_private_base"; + case RdnaOperandKind::SrcPrivateLimit: return "src_private_limit"; } throw std::invalid_argument("unsupported operand kind for string conversion"); diff --git a/core/shader/recompiler/RdnaDecoder/src/RdnaMemoryOpDecoder.cpp b/core/shader/recompiler/RdnaDecoder/src/RdnaMemoryOpDecoder.cpp index f794274c6..77ea8f032 100644 --- a/core/shader/recompiler/RdnaDecoder/src/RdnaMemoryOpDecoder.cpp +++ b/core/shader/recompiler/RdnaDecoder/src/RdnaMemoryOpDecoder.cpp @@ -436,6 +436,10 @@ RdnaOperand scalarSource(std::uint32_t code) { case 125u: operand.kind = RdnaOperandKind::Null; return operand; case 126u: operand.kind = RdnaOperandKind::ExecLo; return operand; case 127u: operand.kind = RdnaOperandKind::ExecHi; return operand; + case 235u: operand.kind = RdnaOperandKind::SrcSharedBase; return operand; + case 236u: operand.kind = RdnaOperandKind::SrcSharedLimit; return operand; + case 237u: operand.kind = RdnaOperandKind::SrcPrivateBase; return operand; + case 238u: operand.kind = RdnaOperandKind::SrcPrivateLimit; return operand; case 239u: operand.kind = RdnaOperandKind::PopsExitingWaveId; return operand; case 248u: operand.kind = RdnaOperandKind::FloatInlineConstant; operand.value = floatBits(0.15915494309189535f); return operand; case 251u: operand.kind = RdnaOperandKind::VccZ; return operand; diff --git a/core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp b/core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp index 63d6d727b..2ddafd996 100644 --- a/core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp +++ b/core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp @@ -1,7 +1,9 @@ +#include "RdnaDecoder/RdnaInstructionDecoder.hpp" #include "Translation/TranslationContext.hpp" #include #include #include +#include namespace ShaderRecompiler { @@ -418,6 +420,11 @@ IrU32 TranslationContext::readScalarCode(std::uint32_t code) { const std::array mask = ballotMask(IrU1(ir.GetExec())); return mask[code - 126u]; } + case 235u: + case 236u: + case 237u: + case 238u: + throw std::runtime_error("TranslationContext::readScalarCode: the scalar aperture source " + RdnaOperandToString(DecodeRdnaScalarSource(code, 0u)) + " is not modelled"); default: return IrU32(ir.Constant(0u)); } } @@ -447,6 +454,11 @@ IrU32 TranslationContext::readRawU32(const RdnaOperand& operand) { const IrU32 zero(ir.Constant(0u)); return IrU32(ir.Select(ir.IEqual(mask.Value(), zero.Value()), ir.Constant(1u), zero.Value())); } + case RdnaOperandKind::SrcSharedBase: + case RdnaOperandKind::SrcSharedLimit: + case RdnaOperandKind::SrcPrivateBase: + case RdnaOperandKind::SrcPrivateLimit: + throw std::runtime_error("TranslationContext::readRawU32: the scalar aperture source " + RdnaOperandToString(operand) + " is not modelled"); default: throw std::runtime_error("TranslationContext::readRawU32 invalid decoded operand used as a raw U32 source"); } } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 5a5353560..851a0a415 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -448,6 +448,7 @@ Throughout the project, every function at every stage either **does exactly what - [Font writing lines](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - `sceFontCreateWritingLine`, `sceFontWritingLineWritesOrder`, `sceFontWritingLineRefersRenderStep`, `sceFontWritingLineGetRenderMetrics` and `sceFontWritingLineClear` follow [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/main/src/libs/libFont.cpp): runs are placed left to right, each at the line's advance so far, with no spacing or adjustment. Only horizontal and left-to-right lines, writing attribute 0 and non-null run metrics are modelled; other values, the metrics of an empty line and unknown or null handles throw, since no reference gives their result or error code. `sceFontWritingLineGetOrderingSpace` still throws (KytyPS5's values are placeholders). Lines are allocated on the host, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. - [s_swappc_b64](../../core/shader/recompiler/Translation/src/ControlFlowInstructions.cpp) writes the link pair with the same model as `s_getpc_b64`: the guest return address `pc + 4` as a byte offset over `GetShaderBase`. Any other use of the link pair (arithmetic, compare, hash, store, or a second `s_swappc_b64` sharing it) throws; constant-offset folding over the link is not modeled. The special encoding `s_swappc_b64` with SDST 125 still decodes as `s_setpc_b64`, which matches the hardware: measured on RDNA2 (gfx1036) in wave32 and wave64, it branches to the source pair and leaves a marker SGPR pair untouched, and with an SGPR destination the link is `PC + 4`. - The [s_swappc_b64 fetch-shader call at vertex/local/tessellation/mesh entry](../../core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp) assumes the called fetch-shader blob ends in `s_setpc_b64` to the link pair (the GNM fetch-shader shape shadPS4's `EmitFetch` copies); the blob is not part of the shader program, so its return closes the pairing by assumption and any in-program reader of the link pair throws. Attribute fetches are inlined from the request's parsed fetch-shader plan through the same `GetAttribute` path as the embedded fetch analyzer, honoring the buffer resource `dst_sel`; an attribute writing more than four registers, or an entry call without a parsed plan, throws. The vertex/instance offset accumulator inside the fetch blob is not replayed for external blobs. +- The scalar aperture sources [src_shared_base, src_shared_limit, src_private_base, src_private_limit](../../core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp) (core/shader/recompiler) decode to their own operand kinds (scalar source codes 235 to 238, named as in LLVM's `SIRegisterInfo.td`), but their value is not modelled: reading one refuses with the source it refused named, because the wave's shared and private aperture bases and limits are unknown. Both the scalar-code read and the raw-operand read refuse; a shader that only carries the value into an address still prepares, one that reads it does not - [VertexBufferReadSize](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) and the [vertex staging copy](../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver) - a vertex fetch whose index is at or past the descriptor's `num_records` no longer throws for `OOB_SELECT` 0 and 1. The copy still spans the whole fetch, but the bytes from `num_records * stride` onward read as zeros, so the out-of-range vertex is `(0, 0, 0, 0)`. Measured with `buffer_load_format_xyzw` (`idxen`) through the [hardware oracle](HW_ORACLE.md) on RDNA2 gfx1036 for PPSA01325's word 3 `0x0004dfac` (`OOB_SELECT` 0) in #2241: indices 3, 4 and 5 over 3 records each return `00000000 00000000 00000000 00000000`, W included, unlike Vulkan `robustBufferAccess`, which returns `(0, 0, 0, 1)`. The zeros come from the record check, not from memory: stored values placed past the records are ignored, and a `num_records` 6 control returns them. `RESOURCE_LEVEL` 1, `OOB_SELECT` 1 and FORMAT 74 `32_32_32_FLOAT` give the same zeros, in wave32 and wave64; gfx1013 (the console) is not measured. `OOB_SELECT` 2 returns the stored values past the records and `OOB_SELECT` 3 compares `num_records` against the byte offset, both measured on the same setup in #2241; the two keep throwing past `num_records` here, since each needs its own rule. Zero-stride descriptors (`stride` 0) keep their separate byte-count check and are not covered here. Each out-of-range fetch is copied alone and outside the draw-snapshot cache, which is neither read nor written for it: a reused snapshot would carry another draw's bytes into the zero region, and a stored zero-padded copy would hand zeros to a later in-range draw at the same address. The valid prefix is still flushed for GPU writes before it is read. - [sceImeKeyboardGetResourceId](../../core/libs/prx/libSceIme/Export.cpp) (libSceIme) reports that no USB keyboard is connected (`0x80bc0004`, as [shadPS4](https://github.com/shadps4-emu/shadPS4) does) because IME keyboard events are not delivered to the handler, although [libSceKeyboard](../../core/libs/prx/libSceKeyboard/src/keyboard_impl.cpp) exposes the host keyboard - [Player review dialog](../../core/libs/prx/libSceCdlgPlayerReview/Export.cpp) (libSceCdlgPlayerReview) shows nothing: `scePlayerReviewDialogOpen` finishes at once and the result reports that the user canceled, as `sceWebBrowserDialogOpen` does From 4f9ddffce0ebaa5f9b96afce7cfc970dfabd77b4 Mon Sep 17 00:00:00 2001 From: alfalfa-star Date: Sat, 10 Oct 2026 13:05:34 +0200 Subject: [PATCH 032/460] perf(agc): read a fast-cleared surface's guest bytes only when the import is compared #2096 keeps a copy of a storage image's guest bytes so that Refresh can compare memory the write watch does not cover. The host-import upload path takes that copy only when the range is unwatched, but the DCC fast-clear path took it on every upload. Reading the whole surface goes through the flush hook, so each 64 KiB block of a surface that a unit shadow covers recorded a publish and waited for the GPU. The clear path now resolves the surface's host import first (the import can unwatch the range, as a compared host-pointer import does) and takes the copy only when the range is then unwatched, like the import path. Write-watched memory, such as direct memory behind a dma-buf import, skips it; a range that is unwatched later re-uploads instead of comparing, as before #2096. agc_driver_dcc_keys_first_write checks that a fast-cleared upload keeps the copy exactly when its range is unwatched at that point. PPSA14632 (LEGO Horizon Adventures) on Linux, RX 9070 XT: the flush hook waited ~60,000 times per 10 s for these publishes (6.4 s), now ~3,500, and presents rise from ~24 to ~100 per 10 s. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp | 1 + core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp | 4 +++- .../prx/libSceAgcDriver/tests/execution/DccKeysFirstWrite.cpp | 2 ++ 3 files changed, 6 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp index 2fa7b67e5..a9c0e59bf 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp @@ -288,6 +288,7 @@ public: // GuestMemory::GpuMutex only, as Refresh is; never from a build's stage A. DccKeys UploadedKeys() const { return uploadedKeys; } DccKeys FilledKeys() const { return filledKeys; } + bool GuestSnapshotValid() const { return originalValid; } DccKeyProof& KeyProof() const { return keyProof; } DccKeys ProvedKeys() const; bool ServesKeysAt(std::uint64_t dccAddress) const; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp index 71720b8b5..5a3f5eac4 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp @@ -1686,7 +1686,9 @@ void StorageTexture::upload(const std::vector* layers) { // The clear is recorded into the open batch like a direct upload (the image kept by it): a // batch of its own submitted the recorder's work first and waited for all of it, 25-40 ms // under the GPU mutex at the movie stage. APS5_NO_RECORDED_CLEAR=1 waits as before. - captureGuestBytes(nullptr); + static_cast(HostImportFor(context, descriptor.baseAddress, static_cast(guestBytes))); + if (GuestMemory::Watched(descriptor.baseAddress, static_cast(guestBytes))) originalValid = false; + else captureGuestBytes(nullptr); forgetBorrowed(0, trackedLayers); stampLayers(false); static const bool recordClear = std::getenv("APS5_NO_RECORDED_CLEAR") == nullptr; diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/DccKeysFirstWrite.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/DccKeysFirstWrite.cpp index 47547e67b..20e68d438 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/DccKeysFirstWrite.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/DccKeysFirstWrite.cpp @@ -209,6 +209,8 @@ void Run(AgcDriver::VulkanDevice& device, std::uint8_t* block) { Require(image != nullptr && image->Descriptor().dccAddress == AddressOf(keys), "the written surface has no pending image under its keys"); Require(AgcDriver::Graphics::StorageImageServesKeys(*image, AddressOf(copied)), "the written image does not serve the copied keys"); Require(image->UploadedKeys() == DccKeys::Uncompressed && image->FilledKeys() == DccKeys::Uncompressed, "the written image still holds the clear it was uploaded under"); + const bool stillWatched = AgcDriver::GuestMemory::Watched(surface, SurfaceBytes); + Require(image->GuestSnapshotValid() != stillWatched, stillWatched ? "a fast-cleared upload of write-watched memory read its guest bytes" : "a fast-cleared upload of untracked memory kept no guest bytes to compare"); const auto pendingVersion = image->Version(); for (int repeat = 0; repeat < 3; ++repeat) { Require(image->Refresh(), "an unchanged pending image was uploaded again"); From 8839f6a1ba2cc32e14c98a3fd7f0289bcf84cc4a Mon Sep 17 00:00:00 2001 From: Tharun Balaji Date: Fri, 9 Oct 2026 15:06:23 +0530 Subject: [PATCH 033/460] feat(libkernel): implement sceKernelFdatasync --- core/libs/prx/libkernel/File/src/Stdio.cpp | 8 ++++++++ core/libs/tests/GuestFilesystem.cpp | 3 +++ 2 files changed, 11 insertions(+) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index 711b9f388..cad87ab6f 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -463,6 +463,14 @@ int APS5_VABI sceKernelFsync(int fd) { #endif } +int APS5_VABI sceKernelFdatasync(int fd) { +#ifdef _WIN32 + return ::_commit(fd); +#else + return ::fdatasync(fd); +#endif +} + int APS5_VABI sceKernelWriteThrottlingStatus(std::uint64_t* status) { if (status == nullptr) throw std::invalid_argument("sceKernelWriteThrottlingStatus: status is null"); status[0] = std::numeric_limits::max(); diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index bd0f598b7..d55e41e93 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -21,6 +21,7 @@ int APS5_VABI futimes_nid_postfix(int, const KernelTimeval*); int APS5_VABI socket_nid_postfix(int, int, int); int APS5_VABI sceKernelFsync(int); int APS5_VABI fdatasync_nid_postfix(int); +int APS5_VABI sceKernelFdatasync(int); int APS5_VABI sceKernelWriteThrottlingStatus(std::uint64_t*); int APS5_VABI sceKernelFtruncate(int, long long); int APS5_VABI sceKernelTruncate_nid_postfix(const char*, long long); @@ -140,6 +141,7 @@ int main() { #endif Require(descriptor >= 0 && sceKernelFsync(descriptor) == 0); Require(fdatasync_nid_postfix(descriptor) == 0); + Require(sceKernelFdatasync(descriptor) == 0); const auto ownerWrite = [&] { return (std::filesystem::status(sized).permissions() & std::filesystem::perms::owner_write) != std::filesystem::perms::none; }; @@ -163,6 +165,7 @@ int main() { Require(fchmod_nid_postfix(descriptor, 0600) == -1 && *__error_nid_postfix() == 9); Require(futimes_nid_postfix(descriptor, nullptr) == -1 && *__error_nid_postfix() == 9); Require(fdatasync_nid_postfix(descriptor) == -1 && *__error_nid_postfix() == 9); + Require(sceKernelFdatasync(descriptor) == -1); #endif const int socket = socket_nid_postfix(2, 2, 0); Require(socket >= 0); From 36bc9214cf05b91f8c52b72a118142a34a7b5b4b Mon Sep 17 00:00:00 2001 From: Tharun Balaji Date: Sat, 10 Oct 2026 13:42:48 +0530 Subject: [PATCH 034/460] fix(libkernel): return SCE errors from sceKernelFdatasync --- core/libs/prx/libkernel/File/src/Stdio.cpp | 5 +++-- core/libs/tests/GuestFilesystem.cpp | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index cad87ab6f..983528a93 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -465,10 +465,11 @@ int APS5_VABI sceKernelFsync(int fd) { int APS5_VABI sceKernelFdatasync(int fd) { #ifdef _WIN32 - return ::_commit(fd); + if (::_commit(fd) != 0) return SceErrorFromErrno(errno); #else - return ::fdatasync(fd); + if (::fdatasync(fd) != 0) return SceErrorFromErrno(errno); #endif + return 0; } int APS5_VABI sceKernelWriteThrottlingStatus(std::uint64_t* status) { diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index d55e41e93..86c8c5b22 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -165,7 +165,7 @@ int main() { Require(fchmod_nid_postfix(descriptor, 0600) == -1 && *__error_nid_postfix() == 9); Require(futimes_nid_postfix(descriptor, nullptr) == -1 && *__error_nid_postfix() == 9); Require(fdatasync_nid_postfix(descriptor) == -1 && *__error_nid_postfix() == 9); - Require(sceKernelFdatasync(descriptor) == -1); + Require(sceKernelFdatasync(descriptor) == static_cast(0x80020009u)); #endif const int socket = socket_nid_postfix(2, 2, 0); Require(socket >= 0); From fb34636f47664e7102506914070d853fd70da654 Mon Sep 17 00:00:00 2001 From: Renan Pires Date: Fri, 9 Oct 2026 16:54:11 -0300 Subject: [PATCH 035/460] feat(libkernel): implement dup and dup2 --- core/libs/prx/libkernel/CMakeLists.txt | 10 ++ core/libs/prx/libkernel/File/src/Stdio.cpp | 54 ++++++++ .../Socket/include/SocketRuntime.hpp | 2 + core/libs/prx/libkernel/Socket/src/Udp.cpp | 30 ++++ core/libs/tests/GuestDup.cpp | 129 ++++++++++++++++++ 5 files changed, 225 insertions(+) create mode 100644 core/libs/tests/GuestDup.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index 6938ffc9a..f20dee8df 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -685,6 +685,16 @@ if(BUILD_TESTING) set_tests_properties(guest_socket_pair PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_dup_tests ${LIBS_INCLUDE_DIR}/tests/GuestDup.cpp) + target_include_directories(guest_dup_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_dup_tests PRIVATE libkernel libc) + configure_windows_unwind(guest_dup_tests) + add_test(NAME guest_dup COMMAND guest_dup_tests) + set_tests_properties(guest_dup PROPERTIES TIMEOUT 20) + if(WIN32) + set_tests_properties(guest_dup PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_elf_phdr_match_tests ${LIBS_INCLUDE_DIR}/tests/GuestElfPhdrMatch.cpp) target_include_directories(guest_elf_phdr_match_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_elf_phdr_match_tests PRIVATE libkernel libc) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index 711b9f388..e3cab493c 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -257,6 +257,21 @@ static int PathError(const char* path) { return *path == '\0' ? GUEST_ENOENT : 0; } +#ifdef _WIN32 +extern "C" _invalid_parameter_handler _set_thread_local_invalid_parameter_handler(_invalid_parameter_handler); +static void IgnoreDescriptorParameter(const wchar_t*, const wchar_t*, const wchar_t*, unsigned int, std::uintptr_t) {} +template static int WithoutParameterHandler(TCall call) { + const auto previous = _set_thread_local_invalid_parameter_handler(IgnoreDescriptorParameter); + const int result = call(); + _set_thread_local_invalid_parameter_handler(previous); + return result; +} +static void RejectDirectoryDuplicate(int descriptor, const char* function) { + if (File::DirectoryDescriptorPath(descriptor)) + throw std::runtime_error(std::string(function) + ": duplicating a directory descriptor is not supported on Windows"); +} +#endif + extern "C" { int APS5_VABI chmod_nid_postfix(const char* path, int mode) { @@ -286,6 +301,45 @@ int APS5_VABI _close_nid_postfix(int descriptor) { return close_nid_postfix(descriptor); } +int APS5_VABI dup_nid_postfix(int d) { + if (d >= GuestSockets::FirstDescriptor) return GuestSockets::Duplicate(d); + if (d < 0) return PosixFailure(GUEST_EBADF); +#ifdef _WIN32 + RejectDirectoryDuplicate(d, __func__); + const int duplicate = WithoutParameterHandler([d] { return ::_dup(d); }); +#else + const int duplicate = ::dup(d); +#endif + if (duplicate < 0) return PosixFailure(SceErrorFromErrno(errno) & 0xffff); + if (duplicate >= GuestSockets::FirstDescriptor) + throw std::runtime_error(std::string(__func__) + ": host descriptor reached the guest socket range"); + return duplicate; +} + +int APS5_VABI dup2_nid_postfix(int from, int to) { + if (from < 0 || to < 0) return PosixFailure(GUEST_EBADF); + const bool socketFrom = from >= GuestSockets::FirstDescriptor; + const bool socketTo = to >= GuestSockets::FirstDescriptor; + if (socketFrom && socketTo) return GuestSockets::DuplicateTo(from, to); + if (socketFrom) { + if (!GuestSockets::IsOpen(from)) return PosixFailure(GUEST_EBADF); + throw std::runtime_error(std::string(__func__) + ": moving a guest socket onto host descriptor " + std::to_string(to) + " is not supported"); + } + if (socketTo) + throw std::runtime_error(std::string(__func__) + ": moving host descriptor " + std::to_string(from) + " into the guest socket range is not supported"); +#ifdef _WIN32 + if (WithoutParameterHandler([from] { return ::_get_osfhandle(from) == -1 ? -1 : 0; }) != 0) return PosixFailure(GUEST_EBADF); + if (from == to) return to; + RejectDirectoryDuplicate(from, __func__); + if (WithoutParameterHandler([from, to] { return ::_dup2(from, to); }) != 0) return PosixFailure(SceErrorFromErrno(errno) & 0xffff); + File::ForgetDirectoryDescriptor(to); + return to; +#else + const int result = ::dup2(from, to); + return result < 0 ? PosixFailure(SceErrorFromErrno(errno) & 0xffff) : result; +#endif +} + int APS5_VABI flock_nid_postfix(int d, int operation) { const int type = operation & 8 ? 8 : operation & 2 ? 2 : operation & 1 ? 1 : 0; if (type == 0) return PosixFailure(GUEST_EBADF); diff --git a/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp b/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp index 88599c60f..db775297c 100644 --- a/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp +++ b/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp @@ -3,6 +3,8 @@ #include namespace GuestSockets { constexpr int FirstDescriptor = 0x10000000; +int Duplicate(int descriptor); +int DuplicateTo(int descriptor, int target); int Close(int descriptor); bool IsOpen(int descriptor); int Family(int descriptor); diff --git a/core/libs/prx/libkernel/Socket/src/Udp.cpp b/core/libs/prx/libkernel/Socket/src/Udp.cpp index 6468a1b17..e87ecd8f5 100644 --- a/core/libs/prx/libkernel/Socket/src/Udp.cpp +++ b/core/libs/prx/libkernel/Socket/src/Udp.cpp @@ -219,6 +219,36 @@ std::shared_ptr Find(int descriptor) { } } +int GuestSockets::Duplicate(int descriptor) { + try { + std::lock_guard lock(socketsMutex); + const auto found = sockets.find(descriptor); + if (found == sockets.end()) return Fail(9); + if (nextDescriptor == INT_MAX) return Fail(24); + const int duplicate = nextDescriptor; + sockets.emplace(duplicate, found->second); + ++nextDescriptor; + return duplicate; + } catch (const std::bad_alloc&) { + return Fail(12); + } +} + +int GuestSockets::DuplicateTo(int descriptor, int target) { + try { + std::lock_guard lock(socketsMutex); + const auto found = sockets.find(descriptor); + if (found == sockets.end()) return Fail(9); + if (target == descriptor) return target; + if (target == INT_MAX) return Fail(9); + sockets.insert_or_assign(target, found->second); + if (target >= nextDescriptor) nextDescriptor = target + 1; + return target; + } catch (const std::bad_alloc&) { + return Fail(12); + } +} + namespace { constexpr int GuestUnix = 1; constexpr int GuestCloseOnExec = 0x10000000; diff --git a/core/libs/tests/GuestDup.cpp b/core/libs/tests/GuestDup.cpp new file mode 100644 index 000000000..e78909848 --- /dev/null +++ b/core/libs/tests/GuestDup.cpp @@ -0,0 +1,129 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include + +extern "C" { +int APS5_VABI dup_nid_postfix(int); +int APS5_VABI dup2_nid_postfix(int, int); +int APS5_VABI pipe_nid_postfix(int*); +int APS5_VABI close_nid_postfix(int); +std::int64_t APS5_VABI read_nid_postfix(int, void*, std::uint64_t); +std::int64_t APS5_VABI write_nid_postfix(int, const char*, std::int64_t); +int APS5_VABI socketpair_nid_postfix(int, int, int, int*); +std::int64_t APS5_VABI send_nid_postfix(int, const void*, std::uint64_t, int); +std::int64_t APS5_VABI recv_nid_postfix(int, void*, std::uint64_t, int); +int* APS5_VABI __error_nid_postfix(); +} + +static void Require(bool value) { if (!value) std::abort(); } + +constexpr int GuestEbadf = 9; +constexpr int Unix = 1; +constexpr int Stream = 1; +constexpr int FirstSocket = 0x10000000; + +static void RequireFailure(int result, int error) { + Require(result == -1); + Require(*__error_nid_postfix() == error); +} + +static void RequirePipeCarries(int writer, int reader, const char* text) { + const auto length = static_cast(std::strlen(text)); + Require(write_nid_postfix(writer, text, length) == length); + char buffer[16] = {}; + Require(read_nid_postfix(reader, buffer, sizeof(buffer)) == length); + Require(std::memcmp(buffer, text, static_cast(length)) == 0); +} + +static void RequireSocketCarries(int writer, int reader, const char* text) { + const auto length = std::strlen(text); + Require(send_nid_postfix(writer, text, length, 0) == static_cast(length)); + char buffer[16] = {}; + Require(recv_nid_postfix(reader, buffer, sizeof(buffer), 0) == static_cast(length)); + Require(std::memcmp(buffer, text, length) == 0); +} + +template static bool Throws(TCall call) { + try { + call(); + } catch (const std::runtime_error&) { + return true; + } + return false; +} + +static void DupOutlivesOriginal() { + int pipe[2]; + Require(pipe_nid_postfix(pipe) == 0); + const int duplicate = dup_nid_postfix(pipe[1]); + Require(duplicate >= 0 && duplicate < FirstSocket && duplicate != pipe[1]); + Require(close_nid_postfix(pipe[1]) == 0); + RequirePipeCarries(duplicate, pipe[0], "dup"); + Require(close_nid_postfix(duplicate) == 0); + char end = 0; + Require(read_nid_postfix(pipe[0], &end, 1) == 0); + Require(close_nid_postfix(pipe[0]) == 0); +} + +static void Dup2ReplacesTarget() { + int first[2]; + int second[2]; + Require(pipe_nid_postfix(first) == 0); + Require(pipe_nid_postfix(second) == 0); + Require(dup2_nid_postfix(first[1], second[1]) == second[1]); + RequirePipeCarries(second[1], first[0], "dup2"); + char end = 0; + Require(read_nid_postfix(second[0], &end, 1) == 0); + Require(dup2_nid_postfix(first[1], first[1]) == first[1]); + for (const int descriptor : {first[0], first[1], second[0], second[1]}) Require(close_nid_postfix(descriptor) == 0); +} + +static void BadDescriptors() { + int pipe[2]; + Require(pipe_nid_postfix(pipe) == 0); + const int closed = pipe[1]; + Require(close_nid_postfix(closed) == 0); + RequireFailure(dup_nid_postfix(-1), GuestEbadf); + RequireFailure(dup_nid_postfix(closed), GuestEbadf); + RequireFailure(dup2_nid_postfix(closed, pipe[0]), GuestEbadf); + RequireFailure(dup2_nid_postfix(closed, closed), GuestEbadf); + RequireFailure(dup2_nid_postfix(pipe[0], -1), GuestEbadf); + RequireFailure(dup2_nid_postfix(-1, pipe[0]), GuestEbadf); + RequireFailure(dup_nid_postfix(FirstSocket + 0x0ffffff0), GuestEbadf); + RequireFailure(dup2_nid_postfix(FirstSocket + 0x0ffffff0, FirstSocket + 0x0ffffff1), GuestEbadf); + Require(close_nid_postfix(pipe[0]) == 0); +} + +static void SocketDuplicates() { + int pair[2]; + Require(socketpair_nid_postfix(Unix, Stream, 0, pair) == 0); + const int duplicate = dup_nid_postfix(pair[0]); + Require(duplicate >= FirstSocket && duplicate != pair[0] && duplicate != pair[1]); + Require(close_nid_postfix(pair[0]) == 0); + RequireSocketCarries(duplicate, pair[1], "socket"); + const int target = duplicate + 0x1000; + Require(dup2_nid_postfix(duplicate, target) == target); + Require(close_nid_postfix(duplicate) == 0); + RequireSocketCarries(target, pair[1], "moved"); + Require(dup2_nid_postfix(target, target) == target); + int next[2]; + Require(socketpair_nid_postfix(Unix, Stream, 0, next) == 0); + Require(next[0] > target && next[1] > target); + RequireSocketCarries(target, pair[1], "kept"); + Require(dup2_nid_postfix(next[0], target) == target); + RequireSocketCarries(target, next[1], "over"); + Require(Throws([&] { dup2_nid_postfix(target, 1); })); + int pipe[2]; + Require(pipe_nid_postfix(pipe) == 0); + Require(Throws([&] { dup2_nid_postfix(pipe[0], target); })); + for (const int descriptor : {target, pair[1], next[0], next[1], pipe[0], pipe[1]}) Require(close_nid_postfix(descriptor) == 0); +} + +int main() { + DupOutlivesOriginal(); + Dup2ReplacesTarget(); + BadDescriptors(); + SocketDuplicates(); +} From cb34140cfb585fea66ea633e5fd90f9f489664df Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sat, 10 Oct 2026 00:44:20 +0200 Subject: [PATCH 036/460] docs(technical-debt): record dup and dup2 limits --- docs/dev/TechnicalDebt.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 5a5353560..9d9e553fc 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -323,6 +323,7 @@ Throughout the project, every function at every stage either **does exactly what - [sigaltstack](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) records each thread's alternate signal stack and reports it back, following FreeBSD's `kern_sigaltstack` (`EINVAL` for flags other than `SS_DISABLE`, `ENOMEM` below `MINSIGSTKSZ` 2048), but handlers never run on it, so `SS_ONSTACK` and `EPERM` are never reported - [libSceUlt queues](../../core/libs/prx/libSceUlt/Export.cpp) use preallocated host storage instead of the supplied guest work area; its existing size formula and the 512-byte object layouts remain unverified. Waiting-queue pool thread limits and waiter priority ordering are not implemented. Finalization wakes blocked queue callers with `ULT_ERROR_STATE`. - [Thread cancellation](../../core/libs/prx/libkernel/Pthread/src/Thread.cpp) (libkernel) follows FreeBSD libthr (states and types 0/1 and 0/2, `PTHREAD_CANCELED` 1, the `_pthread_cleanup_info` node layout), not measured on the console. A request is acted on only in `scePthreadTestcancel`, the condition variable, `sem` and `scePthreadSem` waits, `scePthreadJoin` and the sleeps, and their POSIX names; file and socket calls are not cancellation points. A sleep is not interrupted: the thread exits when it ends. The asynchronous type acts like the deferred one, except at the two set calls. +- [dup and dup2](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) throw for the combinations they do not model: `dup2` between a guest socket and a host descriptor in either direction, and on Windows duplicating a directory descriptor. Guest sockets share one socket object between the duplicates and the descriptor numbers come from the guest socket counter, not the lowest free number as in FreeBSD `kern_dup`. The errno values for descriptor exhaustion are not verified against the console. - [MIMG `tfe` and `lwe`](../../core/shader/recompiler/RdnaDecoder/src/RdnaImageOpDecoder.cpp) never write the status VGPR after the data. On hardware it is written only when a texel fetch hits an unmapped page of a partially resident texture, or, for `lwe` on sample/gather, when the LOD is below the T# `MIN_LOD_WARN`; texture memory is always resident here, the LOD warning (texture-streaming feedback, like the T# `MIN_LOD_WARN` itself) is never reported, and `lwe` on `image_get_lod` throws. - [Packed horizontal gathers](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_gather4h_pck`, MIMG opcode 98): the four window texels are read raw and concatenated into a little-endian bitstream, with DMASK bit i writing stream word i to the next destination register; measured on RDNA2 (gfx1031) at every edge: the anchor is floor(x-0.5) with elements at -1, 0, +1 and +2 clamped horizontally into the width, the row is floor(y), negative rows clamp to row 0, rows at or past the height return zeros, sampler addressing modes do not affect the window, and formats wider than 32 bits per texel write zeros in every register. The emitter fetches each window texel at LOD 0 and packs its bits like `image_load_pck`, so the sampler's filter, addressing and LOD state are not used; formats whose bits are not recoverable from the view and D16 data throw. The anchor and row multiply the normalized coordinate by the image size without contraction. The stream rule was measured for R8_UNORM, RG8_UNORM and RGBA8; only RGBA8 has an execution test. `image_gather8h_pck` (opcode 99) never returns texel data on this hardware - across dmasks 0x1 to 0xFF including the upper nibble, RGBA8, RGBA16 and RGBA32 formats, 1D, 2D and 3D descriptors and normalized and unnormalized sampling it only zero-fills popcount(dmask) registers (sentinel-prefilled destinations) - so it decodes but throws at translation; its eight-texel semantics remain unmeasured. - [waitpid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) returns -1 with `ECHILD` (`EINVAL` for unknown options): nothing here creates a child process; child processes would need a real `waitpid`. From df131b9c390ccc51acac3cb0c9a504a0819fa384 Mon Sep 17 00:00:00 2001 From: virp-mta Date: Sat, 10 Oct 2026 15:54:21 +0400 Subject: [PATCH 037/460] fix(libkernel): maintain random device and file lock invariants in dup and dup2 --- .../libkernel/File/include/RandomDevice.hpp | 1 + .../prx/libkernel/File/src/RandomDevice.cpp | 5 +++ core/libs/prx/libkernel/File/src/Stdio.cpp | 14 +++++- core/libs/tests/GuestDup.cpp | 45 +++++++++++++++++++ 4 files changed, 64 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libkernel/File/include/RandomDevice.hpp b/core/libs/prx/libkernel/File/include/RandomDevice.hpp index fbe4bd17b..f637b52be 100644 --- a/core/libs/prx/libkernel/File/include/RandomDevice.hpp +++ b/core/libs/prx/libkernel/File/include/RandomDevice.hpp @@ -10,6 +10,7 @@ bool IsRandomDevicePath(std::string_view path); int OpenRandomDevice(); bool IsRandomDevice(int fd); bool ReadRandomDevice(int fd, void* buf, std::size_t nbytes); +void RememberRandomDevice(int fd); void ForgetRandomDevice(int fd); } diff --git a/core/libs/prx/libkernel/File/src/RandomDevice.cpp b/core/libs/prx/libkernel/File/src/RandomDevice.cpp index 3d68811b6..1edaddd69 100644 --- a/core/libs/prx/libkernel/File/src/RandomDevice.cpp +++ b/core/libs/prx/libkernel/File/src/RandomDevice.cpp @@ -56,6 +56,11 @@ bool ReadRandomDevice(int fd, void* buf, std::size_t nbytes) { return true; } +void RememberRandomDevice(int fd) { + std::lock_guard lock(g_mutex); + g_descriptors.insert(fd); +} + void ForgetRandomDevice(int fd) { std::lock_guard lock(g_mutex); g_descriptors.erase(fd); diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index e3cab493c..f2b80e263 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -313,6 +313,7 @@ int APS5_VABI dup_nid_postfix(int d) { if (duplicate < 0) return PosixFailure(SceErrorFromErrno(errno) & 0xffff); if (duplicate >= GuestSockets::FirstDescriptor) throw std::runtime_error(std::string(__func__) + ": host descriptor reached the guest socket range"); + if (File::IsRandomDevice(d)) File::RememberRandomDevice(duplicate); return duplicate; } @@ -327,16 +328,27 @@ int APS5_VABI dup2_nid_postfix(int from, int to) { } if (socketTo) throw std::runtime_error(std::string(__func__) + ": moving host descriptor " + std::to_string(from) + " into the guest socket range is not supported"); + const bool sourceIsRandom = File::IsRandomDevice(from); #ifdef _WIN32 if (WithoutParameterHandler([from] { return ::_get_osfhandle(from) == -1 ? -1 : 0; }) != 0) return PosixFailure(GUEST_EBADF); if (from == to) return to; RejectDirectoryDuplicate(from, __func__); if (WithoutParameterHandler([from, to] { return ::_dup2(from, to); }) != 0) return PosixFailure(SceErrorFromErrno(errno) & 0xffff); File::ForgetDirectoryDescriptor(to); + File::ForgetFileLock(to); + File::ForgetRandomDevice(to); + if (sourceIsRandom) File::RememberRandomDevice(to); return to; #else + if (from == to) { + if (::fcntl(from, F_GETFL) < 0) return PosixFailure(GUEST_EBADF); + return to; + } const int result = ::dup2(from, to); - return result < 0 ? PosixFailure(SceErrorFromErrno(errno) & 0xffff) : result; + if (result < 0) return PosixFailure(SceErrorFromErrno(errno) & 0xffff); + File::ForgetRandomDevice(to); + if (sourceIsRandom) File::RememberRandomDevice(to); + return result; #endif } diff --git a/core/libs/tests/GuestDup.cpp b/core/libs/tests/GuestDup.cpp index e78909848..1e3683f92 100644 --- a/core/libs/tests/GuestDup.cpp +++ b/core/libs/tests/GuestDup.cpp @@ -14,6 +14,7 @@ std::int64_t APS5_VABI write_nid_postfix(int, const char*, std::int64_t); int APS5_VABI socketpair_nid_postfix(int, int, int, int*); std::int64_t APS5_VABI send_nid_postfix(int, const void*, std::uint64_t, int); std::int64_t APS5_VABI recv_nid_postfix(int, void*, std::uint64_t, int); +int APS5_VABI open_nid_postfix(const char*, int, int); int* APS5_VABI __error_nid_postfix(); } @@ -121,9 +122,53 @@ static void SocketDuplicates() { for (const int descriptor : {target, pair[1], next[0], next[1], pipe[0], pipe[1]}) Require(close_nid_postfix(descriptor) == 0); } +static void RandomDeviceDuplicates() { + const int random = open_nid_postfix("/dev/urandom", 0, 0); + Require(random >= 0); + const int dupRandom = dup_nid_postfix(random); + Require(dupRandom >= 0 && dupRandom != random); + unsigned char randomBuf1[16] = {}; + unsigned char randomBuf2[16] = {}; + Require(read_nid_postfix(random, randomBuf1, sizeof(randomBuf1)) == sizeof(randomBuf1)); + Require(read_nid_postfix(dupRandom, randomBuf2, sizeof(randomBuf2)) == sizeof(randomBuf2)); + Require(std::memcmp(randomBuf1, randomBuf2, sizeof(randomBuf1)) != 0); + + Require(dup2_nid_postfix(random, random) == random); + Require(read_nid_postfix(random, randomBuf1, sizeof(randomBuf1)) == sizeof(randomBuf1)); + + int pipe[2]; + Require(pipe_nid_postfix(pipe) == 0); + Require(write_nid_postfix(pipe[1], "hello", 5) == 5); + Require(dup2_nid_postfix(pipe[0], random) == random); + char textBuf[8] = {}; + Require(read_nid_postfix(random, textBuf, 5) == 5); + Require(std::memcmp(textBuf, "hello", 5) == 0); + + const int targetHost = pipe[0]; + Require(dup2_nid_postfix(dupRandom, targetHost) == targetHost); + unsigned char randomBuf3[16] = {}; + Require(read_nid_postfix(targetHost, randomBuf3, sizeof(randomBuf3)) == sizeof(randomBuf3)); + Require(std::memcmp(randomBuf3, randomBuf2, sizeof(randomBuf3)) != 0); + + const int secondRandom = open_nid_postfix("/dev/random", 0, 0); + Require(secondRandom >= 0); + Require(dup2_nid_postfix(dupRandom, secondRandom) == secondRandom); + unsigned char randomBuf4[16] = {}; + Require(read_nid_postfix(secondRandom, randomBuf4, sizeof(randomBuf4)) == sizeof(randomBuf4)); + Require(std::memcmp(randomBuf4, randomBuf3, sizeof(randomBuf4)) != 0); + + const int invalidFd = FirstSocket - 100; + RequireFailure(dup2_nid_postfix(invalidFd, secondRandom), GuestEbadf); + unsigned char randomBuf5[16] = {}; + Require(read_nid_postfix(secondRandom, randomBuf5, sizeof(randomBuf5)) == sizeof(randomBuf5)); + + for (const int descriptor : {random, dupRandom, pipe[1], secondRandom}) Require(close_nid_postfix(descriptor) == 0); +} + int main() { DupOutlivesOriginal(); Dup2ReplacesTarget(); BadDescriptors(); SocketDuplicates(); + RandomDeviceDuplicates(); } From 8c06af0ad9502d2dd85ef7fa61f59f1a75c01260 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Fri, 9 Oct 2026 23:11:00 -0500 Subject: [PATCH 038/460] fix(agc): honour primitive restart in mesh-path triangle strips A geometry (NGG) strip draw on the mesh path assigned every primitive the next index positions without looking for the restart index: a triangle spanning a restart fetched vertex 0xFFFF and was drawn to wherever that read landed, and the strip parity never reset, so after an odd-length strip every triangle's winding flipped and face culling dropped it. For an indexed strip draw with restart enabled and the restart index present, the draw now records each position's strip start (one past the last restart at or before it) and passes the table after the mesh arguments, flagged in draw word 3. The mesh prologue reads the start of its primitive's last vertex: a primitive whose start lies past its first vertex spans a restart and is made degenerate, and the others take their parity from their strip's start (or the draw's first index). The restart vertex itself fetches vertex 0. The table is built from the index bytes the draw already reads (its copy, or the guest memory an in-place import reads), next to #1968's restart-aware highest-index scan. The new Mesh.cpp test draws quads as four-index strips each followed by a restart (16- and 32-bit indices, and an indirect draw starting mid-buffer) with front faces culled: every quad must render whole and the gaps between them stay clear. Without the table the first check fails ("a triangle crossed the restart after quad 0"). AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- .../Graphics/include/VertexInput.hpp | 25 +++++++++ .../prx/libSceAgcDriver/Graphics/src/Draw.cpp | 44 ++++++++++++--- core/libs/prx/libSceAgcDriver/tests/Mesh.cpp | 55 ++++++++++++++++++- .../IntermediateRepresentation/IrOpcode.hpp | 1 + .../src/IrOpcode.cpp | 1 + core/shader/recompiler/Recompiler.hpp | 3 + .../SpirvBackend/SpirvEmitterInstructions.hpp | 1 + .../SpirvBackend/src/SpirvFlowEmitter.cpp | 1 + .../SpirvBackend/src/SpirvModuleEmitter.cpp | 46 +++++++++++++--- .../Translation/src/InstructionTranslator.cpp | 22 ++++++-- 10 files changed, 177 insertions(+), 22 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp index d3b182c7a..0bc1a7c21 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp @@ -246,6 +246,31 @@ inline std::optional HighestDrawIndex(std::span return highest; } +inline std::vector MeshRestartTable(std::span indices, std::uint32_t indexSize) { + Require(indexSize == 2 || indexSize == 4, "unsupported index size"); + const auto restartIndex = indexSize == 2 ? 0xffffu : 0xffffffffu; + std::vector starts(indices.size() / indexSize); + std::uint32_t stripStart = 0; + bool restarted = false; + for (std::size_t position = 0; position < starts.size(); ++position) { + std::uint32_t index = 0; + if (indexSize == 2) { + std::uint16_t value = 0; + std::memcpy(&value, indices.data() + position * 2u, sizeof(value)); + index = value; + } else { + std::memcpy(&index, indices.data() + position * 4u, sizeof(index)); + } + if (index == restartIndex) { + restarted = true; + stripStart = static_cast(position) + 1u; + } + starts[position] = stripStart; + } + if (!restarted) starts.clear(); + return starts; +} + inline std::vector SoloZeroPaddedFetchIndices(const std::vector& fetches, const std::vector& fetchValid) { Require(fetchValid.size() == fetches.size(), "fetch validity does not match the fetch count"); std::vector solo; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp index 4d238de0c..edca5809a 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp @@ -961,6 +961,7 @@ struct DrawInputs { std::set fragmentOutputs; VkPipelineStageFlags shaderStages = 0; std::uint32_t meshGroups = 0; + std::vector restartTable; }; std::shared_ptr zeroVertexBuffer(const Context& context, const ShaderRecompiler::VertexAttribute& attribute) { @@ -1062,6 +1063,10 @@ DrawInputs prepareDrawInputs(const Context& context, const State& state, const P Require(*highest <= context.limits.maxDrawIndexedIndexValue, "index exceeds the device's indexed draw limit"); Require(!fanGeometry || !state.primitiveRestart || *highest != (draw.indexSize == 2 ? 0xffffu : 0xffffffffu), "primitive restart in a triangle fan geometry draw is unsupported"); inputs.maxIndex = *highest; + if (skipRestart && state.stages.mesh && state.stages.mesh->inputPrimitive == 6) { + const auto indexSpan = import != nullptr ? std::span(reinterpret_cast(draw.indexAddress), static_cast(indexBytes)) : copy.buffer->Bytes().first(static_cast(indexBytes)); + inputs.restartTable = MeshRestartTable(indexSpan, draw.indexSize); + } if (import != nullptr) { inputs.indexHandle = import->buffer; inputs.indexOffset = draw.indexAddress - import->base; @@ -1363,10 +1368,30 @@ bool recordIndirectArguments(const Context& context, VkCommandBuffer commands, R return true; } -std::shared_ptr recordMeshArguments(const Context& context, VkCommandBuffer commands, Recorder* recorder, bool recorded, const State& state, const Pm4::DrawParameters& draw, const IndirectRecord& indirect, const std::function& countBarrier) { +std::shared_ptr meshArgumentBuffer(const Context& context, std::span restartTable) { + const auto tableBytes = restartTable.empty() ? 0u : sizeof(std::uint32_t) * (restartTable.size() + 1u); + auto arguments = std::make_shared(context, ShaderRecompiler::MeshArgumentBytes + tableBytes, VK_BUFFER_USAGE_INDIRECT_BUFFER_BIT | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT); + if (!restartTable.empty()) { + Require(restartTable.size() <= std::numeric_limits::max(), "mesh restart table is too large"); + const auto length = static_cast(restartTable.size()); + auto* bytes = arguments->Bytes().data() + ShaderRecompiler::MeshArgumentBytes; + std::memcpy(bytes, &length, sizeof(length)); + std::memcpy(bytes + sizeof(length), restartTable.data(), restartTable.size_bytes()); + } + return arguments; +} + +std::shared_ptr directMeshArguments(const Context& context, const Pm4::DrawParameters& draw, std::span restartTable) { + auto arguments = meshArgumentBuffer(context, restartTable); + const MeshArguments resolved{0u, 0u, 0u, draw.indexCount, 0u}; + std::memcpy(arguments->Bytes().data(), &resolved, sizeof(resolved)); + return arguments; +} + +std::shared_ptr recordMeshArguments(const Context& context, VkCommandBuffer commands, Recorder* recorder, bool recorded, const State& state, const Pm4::DrawParameters& draw, const IndirectRecord& indirect, const std::function& countBarrier, std::span restartTable) { const auto* args = indirect.args; const auto rules = MeshArgumentRulesFor(context, *state.stages.mesh, draw.indexCount); - auto arguments = std::make_shared(context, ShaderRecompiler::MeshArgumentBytes, VK_BUFFER_USAGE_INDIRECT_BUFFER_BIT | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT); + auto arguments = meshArgumentBuffer(context, restartTable); if (indirect.path == IndirectDrawPath::Gpu && recorder != nullptr && indirect.argumentImport->address != 0) { const std::array words{rules.indexCount, rules.inputSize, rules.step, rules.primitivesPerGroup, rules.maxGroups, rules.maxInstances, rules.maxTotal}; if (recorder->RecordMeshArguments(commands, indirect.argumentImport->address + (args->arguments - indirect.argumentImport->base), arguments->DeviceAddress(), words)) { @@ -1481,7 +1506,7 @@ struct RecordedDraw { VkShaderStageFlags pushStages = 0; }; -void pushDrawConstants(const Pipeline& pipeline, VkCommandBuffer commands, const State& state, const Pm4::DrawParameters& draw, std::span shaders, const ShaderResources& resources, const std::array* bytes, VkShaderStageFlags stages, VkDeviceAddress meshArguments = 0) { +void pushDrawConstants(const Pipeline& pipeline, VkCommandBuffer commands, const State& state, const Pm4::DrawParameters& draw, std::span shaders, const ShaderResources& resources, const std::array* bytes, VkShaderStageFlags stages, VkDeviceAddress meshArguments = 0, bool restartTable = false) { auto block = bytes != nullptr ? *bytes : AssemblePushConstants(shaders); if (bytes == nullptr) { resources.PatchPushConstants(block); @@ -1493,7 +1518,7 @@ void pushDrawConstants(const Pipeline& pipeline, VkCommandBuffer commands, const } const auto firstVertex = draw.indirect ? draw.indirect->vertexConstant : draw.firstVertex; const auto firstInstance = draw.indirect ? draw.indirect->instanceConstant : draw.firstInstance; - const std::array words{draw.indexCount, firstVertex, firstInstance, draw.indexed ? draw.indexSize : 0u, static_cast(meshArguments), static_cast(meshArguments >> 32u)}; + const std::array words{draw.indexCount, firstVertex, firstInstance, draw.indexed ? draw.indexSize | (restartTable ? ShaderRecompiler::MeshIndexRestartTable : 0u) : 0u, static_cast(meshArguments), static_cast(meshArguments >> 32u)}; static_assert(ShaderRecompiler::MeshDrawPushOffsetBytes + ShaderRecompiler::MeshDrawPushBytes == PipelinePushSlotBytes); std::memcpy(block.data() + ShaderRecompiler::MeshDrawPushOffsetBytes, words.data(), sizeof(words)); pipeline.PushConstants(commands, stages | VK_SHADER_STAGE_MESH_BIT_EXT, block); @@ -1651,6 +1676,7 @@ void recordDraw(const Context& context, const State& state, const Pm4::DrawParam VkBuffer argumentBuffer = VK_NULL_HANDLE; VkDeviceSize argumentOffset = 0; bool rewritten = false; + if (!meshIndirect && !inputs.restartTable.empty()) meshArguments = directMeshArguments(context, draw, inputs.restartTable); if (continued) { record.pipeline->Continue(commands, state); } else { @@ -1665,7 +1691,7 @@ void recordDraw(const Context& context, const State& state, const Pm4::DrawParam } APS5_LOG_CHARS_OUT_DEBUG("Upload barrier recorded"); if (meshIndirect) { - meshArguments = recordMeshArguments(context, commands, recorder, true, state, draw, *record.indirect, countBarrier); + meshArguments = recordMeshArguments(context, commands, recorder, true, state, draw, *record.indirect, countBarrier, inputs.restartTable); argumentBuffer = meshArguments->Handle(); } else if (gpuIndirect) { rewritten = recordIndirectArguments(context, commands, recorder, true, *record.indirect, scratch, argumentBuffer, argumentOffset, countBarrier); @@ -1683,7 +1709,7 @@ void recordDraw(const Context& context, const State& state, const Pm4::DrawParam resources.Bind(commands, VK_PIPELINE_BIND_POINT_GRAPHICS, record.pipeline->Layout()); } APS5_LOG_CHARS_OUT_DEBUG("Resources bound"); - pushDrawConstants(*record.pipeline, commands, state, draw, shaders, resources, record.pushBytes, record.pushStages, meshArguments != nullptr ? meshArguments->DeviceAddress() : 0); + pushDrawConstants(*record.pipeline, commands, state, draw, shaders, resources, record.pushBytes, record.pushStages, meshArguments != nullptr ? meshArguments->DeviceAddress() : 0, !inputs.restartTable.empty()); APS5_LOG_CHARS_OUT_DEBUG("Push constants recorded"); recordDrawCommands(context, commands, state, draw, inputs, record.indirect, argumentBuffer, argumentOffset); if (record.pipeline->SplitsFaces()) { @@ -2117,8 +2143,10 @@ void Draw(const Context& context, const State& state, const Pm4::DrawParameters& APS5_LOG_CHARS_OUT_DEBUG("Upload barrier recorded"); std::shared_ptr meshArguments; if (state.stages.mesh && args != nullptr) { - meshArguments = recordMeshArguments(context, commands, recorder, recorded, state, draw, indirect, countBarrier); + meshArguments = recordMeshArguments(context, commands, recorder, recorded, state, draw, indirect, countBarrier, inputs.restartTable); argumentBuffer = meshArguments->Handle(); + } else if (!inputs.restartTable.empty()) { + meshArguments = directMeshArguments(context, draw, inputs.restartTable); } else if (gpuIndirect) { rewritten = recordIndirectArguments(context, commands, recorder, recorded, indirect, scratch, argumentBuffer, argumentOffset, countBarrier); } @@ -2153,7 +2181,7 @@ void Draw(const Context& context, const State& state, const Pm4::DrawParameters& APS5_LOG_CHARS_OUT_DEBUG("Pipeline Begin OK"); resources->Bind(commands, VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline->Layout()); APS5_LOG_CHARS_OUT_DEBUG("Resources bound"); - pushDrawConstants(*pipeline, commands, state, draw, shaders, *resources, nullptr, 0, meshArguments != nullptr ? meshArguments->DeviceAddress() : 0); + pushDrawConstants(*pipeline, commands, state, draw, shaders, *resources, nullptr, 0, meshArguments != nullptr ? meshArguments->DeviceAddress() : 0, !inputs.restartTable.empty()); APS5_LOG_CHARS_OUT_DEBUG("Push constants recorded"); recordDrawCommands(context, commands, state, draw, inputs, args != nullptr ? &indirect : nullptr, argumentBuffer, argumentOffset); if (pipeline->SplitsFaces()) { diff --git a/core/libs/prx/libSceAgcDriver/tests/Mesh.cpp b/core/libs/prx/libSceAgcDriver/tests/Mesh.cpp index 9cc2ceaa9..c92bd7fad 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Mesh.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Mesh.cpp @@ -95,6 +95,9 @@ alignas(256) std::array Fan{}; alignas(256) std::array FanScrambled{}; alignas(256) std::array FanIndices{}; alignas(256) std::array FanRestart{}; +alignas(256) std::array Quads{}; +alignas(256) std::array QuadStrips{}; +alignas(256) std::array QuadStrips32{}; std::array VertexBufferDescriptor(const void* vertices, std::uint32_t count) { const auto address = reinterpret_cast(vertices); @@ -127,6 +130,7 @@ struct MeshDraw { std::uint32_t geometryPushBytes = ShaderRecompiler::MeshDrawPushOffsetBytes; bool restart = false; std::uint32_t waveSize = 64u; + VkCullModeFlags cull = VK_CULL_MODE_NONE; }; void DrawMesh(AgcDriver::VulkanDevice& device, const MeshDraw& setup) { @@ -178,7 +182,7 @@ void DrawMesh(AgcDriver::VulkanDevice& device, const MeshDraw& setup) { state.viewport = {0, static_cast(Height), static_cast(Width), -static_cast(Height), 0, 1}; state.negativeOneToOne = false; state.scissor = {{0, 0}, {Width, Height}}; - state.cullMode = VK_CULL_MODE_NONE; + state.cullMode = setup.cull; state.frontFace = VK_FRONT_FACE_COUNTER_CLOCKWISE; state.blend.colorWriteMask = 15; state.blends = {state.blend}; @@ -205,6 +209,24 @@ void CheckFan(const char* what) { Require(PixelIs(0, Background) && PixelIs(Pixels.size() - 4u, Background), std::string(what) + ": the corners changed"); } +std::size_t QuadPixel(std::uint32_t quad, float across) { + const auto x = static_cast((static_cast(quad) + across * 0.8f) * Width / Columns); + return (static_cast(Height * 3u / 4u) * Width + x) * 4u; +} + +void CheckQuads(const char* what, std::uint32_t first, std::uint32_t count) { + for (std::uint32_t quad = 0; quad < Columns; ++quad) { + const bool drawn = quad >= first && quad < first + count; + for (const float across : {0.25f, 0.75f}) { + const auto offset = QuadPixel(quad, across); + Require(PixelIs(offset, drawn ? TriangleColor(quad) : Background), std::string(what) + ": quad " + std::to_string(quad) + (across < 0.5f ? " left" : " right") + " half pixel " + PixelText(offset)); + } + const auto gap = QuadPixel(quad, 1.12f); + Require(PixelIs(gap, Background), std::string(what) + ": a triangle crossed the restart after quad " + std::to_string(quad) + ", pixel " + PixelText(gap)); + } + Require(PixelIs((static_cast(Height / 4u) * Width + Width / 2u) * 4u, Background), std::string(what) + ": the upper half changed"); +} + void CheckTriangles(const char* what) { for (std::uint32_t triangle = 0; triangle < Triangles; ++triangle) { const auto offset = CellPixel(triangle); @@ -249,6 +271,22 @@ int main() { FanRestart[k] = FanIndices[k]; } FanRestart[FanRim] = 0xffffu; + for (std::uint32_t quad = 0; quad < Columns; ++quad) { + const auto color = TriangleColor(quad); + const std::array rgba{color[0] / 255.0f, color[1] / 255.0f, color[2] / 255.0f, 1.0f}; + const float left = -1.0f + 2.0f * static_cast(quad) / Columns; + const float right = left + 1.6f / Columns; + Quads[4 * quad] = {{left, -1.0f, 0.5f, 1.0f}, rgba}; + Quads[4 * quad + 1] = {{left, 0.0f, 0.5f, 1.0f}, rgba}; + Quads[4 * quad + 2] = {{right, -1.0f, 0.5f, 1.0f}, rgba}; + Quads[4 * quad + 3] = {{right, 0.0f, 0.5f, 1.0f}, rgba}; + for (std::uint32_t k = 0; k < 4; ++k) { + QuadStrips[5 * quad + k] = static_cast(4 * quad + k); + QuadStrips32[5 * quad + k] = 4 * quad + k; + } + QuadStrips[5 * quad + 4] = 0xffffu; + QuadStrips32[5 * quad + 4] = 0xffffffffu; + } const auto testDevice = OpenVulkanTestDevice(); if (!testDevice) return VulkanTestSkipped; @@ -332,6 +370,21 @@ int main() { } Require(PixelIs((static_cast(Height / 4u) * Width + Width / 2u) * 4u, Background), "triangle strip: the upper half changed"); + const auto quadBuffer = VertexBufferDescriptor(Quads.data(), static_cast(Quads.size())); + ClearPixels(); + DrawMesh(device, {strip, {reinterpret_cast(QuadStrips.data()), static_cast(QuadStrips.size()), 2, 1, 0, true}, quadBuffer, ShaderRecompiler::MeshDrawPushOffsetBytes, true, 64u, VK_CULL_MODE_FRONT_BIT}); + CheckQuads("16-bit triangle strips with restarts", 0, Columns); + ClearPixels(); + DrawMesh(device, {strip, {reinterpret_cast(QuadStrips32.data()), static_cast(QuadStrips32.size()), 4, 1, 0, true}, quadBuffer, ShaderRecompiler::MeshDrawPushOffsetBytes, true, 64u, VK_CULL_MODE_FRONT_BIT}); + CheckQuads("32-bit triangle strips with restarts", 0, Columns); + const std::array quadWords{15u, 1u, 5u, 0u, 0u}; + std::copy(quadWords.begin(), quadWords.end(), record); + AgcDriver::Pm4::DrawParameters quadDraw{reinterpret_cast(QuadStrips.data()), static_cast(QuadStrips.size()), 2, 1, 0, true}; + quadDraw.indirect = AgcDriver::Pm4::DrawParameters::IndirectDraw{reinterpret_cast(record), 0x25u, 20u, 20u, 1u, false, 0u, 0x280u, 0x280u, 0x280u, false, 0u}; + ClearPixels(); + DrawMesh(device, {strip, quadDraw, quadBuffer, ShaderRecompiler::MeshDrawPushOffsetBytes, true, 64u, VK_CULL_MODE_FRONT_BIT}); + CheckQuads("indirect triangle strips with restarts", 1, 3); + const ShaderRecompiler::MeshConfiguration fan{5u, 3u, 5u, 9u, 3u, 64u, 1024u, 0u, 4u}; ClearPixels(); DrawMesh(device, {fan, {0, static_cast(Fan.size()), 0, 1, 0, false}, VertexBufferDescriptor(Fan.data(), static_cast(Fan.size()))}); diff --git a/core/shader/recompiler/IntermediateRepresentation/include/IntermediateRepresentation/IrOpcode.hpp b/core/shader/recompiler/IntermediateRepresentation/include/IntermediateRepresentation/IrOpcode.hpp index f08b2da82..e580b1118 100644 --- a/core/shader/recompiler/IntermediateRepresentation/include/IntermediateRepresentation/IrOpcode.hpp +++ b/core/shader/recompiler/IntermediateRepresentation/include/IntermediateRepresentation/IrOpcode.hpp @@ -79,6 +79,7 @@ enum class IrOpcode : std::uint16_t { RealtimeClock, MeshDrawParameter, MeshArgument, + MeshRestartStart, MeshAllocate, TessellationBase, GetTessellationAttribute, diff --git a/core/shader/recompiler/IntermediateRepresentation/src/IrOpcode.cpp b/core/shader/recompiler/IntermediateRepresentation/src/IrOpcode.cpp index ec6ddd79e..7abf9208a 100644 --- a/core/shader/recompiler/IntermediateRepresentation/src/IrOpcode.cpp +++ b/core/shader/recompiler/IntermediateRepresentation/src/IrOpcode.cpp @@ -120,6 +120,7 @@ constexpr std::array(IrOpcode::Count)> Meta makeMeta("RealtimeClock", U64), makeMeta("MeshDrawParameter", U32, U32), makeMeta("MeshArgument", U32, U32), + makeMeta("MeshRestartStart", U32, U32, U1), makeMeta("MeshAllocate", Void, U32), makeMeta("TessellationBase", U32, U32), makeMeta("GetTessellationAttribute", U32, U32, U32, U1), diff --git a/core/shader/recompiler/Recompiler.hpp b/core/shader/recompiler/Recompiler.hpp index e22a50162..ab6def974 100644 --- a/core/shader/recompiler/Recompiler.hpp +++ b/core/shader/recompiler/Recompiler.hpp @@ -278,6 +278,9 @@ inline constexpr std::uint32_t MeshArgumentAddressDword = 4; inline constexpr std::uint32_t MeshArgumentIndexCountDword = 3; inline constexpr std::uint32_t MeshArgumentFirstIndexDword = 4; inline constexpr std::uint32_t MeshArgumentBytes = 20; +inline constexpr std::uint32_t MeshIndexRestartTable = 0x100; +inline constexpr std::uint32_t MeshRestartLengthDword = MeshArgumentBytes / 4; +inline constexpr std::uint32_t MeshRestartTableDword = MeshRestartLengthDword + 1; inline constexpr std::uint32_t MeshIndexBufferUserWord = 4; inline constexpr std::uint32_t WorkgroupMemoryDescriptorSet = 1; diff --git a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterInstructions.hpp b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterInstructions.hpp index 8e275b607..3513c1a8f 100644 --- a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterInstructions.hpp +++ b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterInstructions.hpp @@ -62,6 +62,7 @@ void EmitVoid(SpirvValueEmitContext& context); void EmitBarrier(SpirvEmitterState& state); std::uint32_t EmitMeshDrawParameter(SpirvValueEmitContext& ctx, const IrValue& inst); std::uint32_t EmitMeshArgument(SpirvValueEmitContext& ctx, const IrValue& inst); +std::uint32_t EmitMeshRestartStart(SpirvValueEmitContext& ctx, const IrValue& inst); std::uint32_t EmitGetTessellationAttribute(SpirvValueEmitContext& ctx, const IrValue& inst); void EmitSetTessellationAttribute(SpirvValueEmitContext& ctx, const IrValue& inst); std::uint32_t EmitGetUserData(SpirvEmitterState& state, ScalarReg reg); diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp index d4e8eaa77..e851f59c3 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp @@ -294,6 +294,7 @@ void EmitDirectInstruction(SpirvValueEmitContext& ctx, const IrValue& inst) { case IrOpcode::RealtimeClock: return Invoke(EmitRealtimeClock, ctx, inst); case IrOpcode::MeshDrawParameter: return Invoke(EmitMeshDrawParameter, ctx, inst); case IrOpcode::MeshArgument: return Invoke(EmitMeshArgument, ctx, inst); + case IrOpcode::MeshRestartStart: return Invoke(EmitMeshRestartStart, ctx, inst); case IrOpcode::MeshAllocate: return Invoke(EmitMeshAllocate, ctx, inst); case IrOpcode::TessellationBase: return Invoke(EmitTessellationBase, ctx, inst); case IrOpcode::GetTessellationAttribute: return Invoke(EmitGetTessellationAttribute, ctx, inst); diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp index baf755e18..07f3ca948 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp @@ -1053,12 +1053,9 @@ std::uint32_t EmitMeshDrawParameter(SpirvValueEmitContext& ctx, const IrValue& i return result; } -std::uint32_t EmitMeshArgument(SpirvValueEmitContext& ctx, const IrValue& inst) { - auto& state = ctx.state; - const auto index = inst.Argument(0)->ImmediateU32(); - if (state.program.Resources().stage != IrShaderStage::Mesh || index >= MeshArgumentBytes / 4u) { - ctx.Fail(inst, "invalid mesh argument"); - } +namespace { + +std::uint32_t MeshArgumentAddress(SpirvEmitterState& state) { const auto push = [&](std::uint32_t dword) { const auto pointer = state.module.AllocateId(); state.module.AddFunction(spv::OpAccessChain, TypePushConstantElementPointer(state), pointer, state.pushConstantVariable, ConstantU32(state, 0u), ConstantU32(state, MeshDrawPushOffsetBytes / 4u + dword)); @@ -1068,7 +1065,42 @@ std::uint32_t EmitMeshArgument(SpirvValueEmitContext& ctx, const IrValue& inst) }; const auto low = push(MeshArgumentAddressDword); const auto high = push(MeshArgumentAddressDword + 1u); - const auto address = Binary(state, spv::OpBitwiseOr, TypeScalarU64(state), low, Binary(state, spv::OpShiftLeftLogical, TypeScalarU64(state), high, ConstantU32(state, 32u))); + return Binary(state, spv::OpBitwiseOr, TypeScalarU64(state), low, Binary(state, spv::OpShiftLeftLogical, TypeScalarU64(state), high, ConstantU32(state, 32u))); +} + +std::uint32_t LoadMeshArgumentWord(SpirvEmitterState& state, std::uint32_t address, std::uint32_t dword) { + const auto element = Binary(state, spv::OpIAdd, TypeScalarU64(state), address, Binary(state, spv::OpShiftLeftLogical, TypeScalarU64(state), Unary(state, spv::OpUConvert, TypeScalarU64(state), dword), ConstantU32(state, 2u))); + const auto pointer = state.module.AllocateId(); + state.module.AddFunction(spv::OpConvertUToPtr, TypePointer(state, spv::StorageClassPhysicalStorageBuffer, TypeU32(state)), pointer, element); + const auto loaded = state.module.AllocateId(); + state.module.AddFunction(spv::OpLoad, TypeU32(state), loaded, pointer, spv::MemoryAccessAlignedMask, 4u); + return loaded; +} + +} + +std::uint32_t EmitMeshRestartStart(SpirvValueEmitContext& ctx, const IrValue& inst) { + auto& state = ctx.state; + if (state.program.Resources().stage != IrShaderStage::Mesh) { + ctx.Fail(inst, "mesh restart table read outside a mesh shader"); + } + return EmitValueOrZeroIfCondition(state, ctx.Arg(inst, 1), [&]() { + const auto address = MeshArgumentAddress(state); + const auto length = LoadMeshArgumentWord(state, address, ConstantU32(state, MeshRestartLengthDword)); + const auto last = Binary(state, spv::OpISub, TypeU32(state), length, ConstantU32(state, 1u)); + const auto position = state.module.AllocateId(); + state.module.AddFunction(spv::OpExtInst, TypeU32(state), position, GlslStd450(state), GLSLstd450UMin, ctx.Arg(inst, 0), last); + return LoadMeshArgumentWord(state, address, Binary(state, spv::OpIAdd, TypeU32(state), position, ConstantU32(state, MeshRestartTableDword))); + }); +} + +std::uint32_t EmitMeshArgument(SpirvValueEmitContext& ctx, const IrValue& inst) { + auto& state = ctx.state; + const auto index = inst.Argument(0)->ImmediateU32(); + if (state.program.Resources().stage != IrShaderStage::Mesh || index >= MeshArgumentBytes / 4u) { + ctx.Fail(inst, "invalid mesh argument"); + } + const auto address = MeshArgumentAddress(state); const auto present = Binary(state, spv::OpINotEqual, TypeBool(state), address, BdaConstant(state, 0u)); const auto before = state.currentLabel; const auto loadLabel = state.module.AllocateId(); diff --git a/core/shader/recompiler/Translation/src/InstructionTranslator.cpp b/core/shader/recompiler/Translation/src/InstructionTranslator.cpp index b764b46af..f6b44497a 100644 --- a/core/shader/recompiler/Translation/src/InstructionTranslator.cpp +++ b/core/shader/recompiler/Translation/src/InstructionTranslator.cpp @@ -262,12 +262,20 @@ void emitEntryPrologue(IrProgram& program, IrBlock& entryBlock, const TranslateO IrValue& primitiveCount = minimum(subtractSaturate(primitives, waveBase), u32(options.waveSize)); IrValue& waveInfo = entryIr.BitwiseOr(entryIr.ShiftLeftLogical(wave, u32(24u)), u32((totalThreads / options.waveSize) << 28u)); entryIr.SetScalarReg(static_cast(3), entryIr.BitwiseOr(waveInfo, entryIr.BitwiseOr(entryIr.ShiftLeftLogical(primitiveCount, u32(8u)), vertexCount))); - IrValue& parity = mesh.inputPrimitive == kTriStripPrimitiveType ? entryIr.BitwiseAnd(entryIr.IAdd(firstPrimitive, local), u32(1u)) : u32(0u); + const bool strip = mesh.inputPrimitive == kTriStripPrimitiveType; + IrValue& indexWord = draw(3u); + IrValue& stripPosition = entryIr.IAdd(entryIr.IAdd(firstPrimitive, local), firstIndex); + IrValue& restartTable = entryIr.INotEqual(entryIr.BitwiseAnd(indexWord, u32(MeshIndexRestartTable)), u32(0u)); + IrValue& stripStart = strip ? entryIr.Emit(IrOpcode::MeshRestartStart, IrOpcodeType(IrOpcode::MeshRestartStart), {&entryIr.IAdd(stripPosition, u32(2u)), &restartTable}) : u32(0u); + IrValue& parity = strip ? entryIr.BitwiseAnd(entryIr.ISub(stripPosition, entryIr.Emit(IrOpcode::UMax32, IrOpcodeType(IrOpcode::UMax32), {&stripStart, &firstIndex})), u32(1u)) : u32(0u); IrValue& vertex = entryIr.IMul(local, step); IrValue& item = u32(mesh.esgsItemSize); - IrValue& first = fan ? entryIr.IMul(entryIr.IAdd(vertex, u32(1u)), item) : entryIr.IMul(entryIr.IAdd(vertex, parity), item); - IrValue& second = fan ? entryIr.IMul(entryIr.IAdd(vertex, u32(2u)), item) : size >= 2u ? entryIr.IMul(entryIr.ISub(entryIr.IAdd(vertex, u32(1u)), parity), item) : u32(0u); - IrValue& third = fan ? u32(0u) : size == 3u ? entryIr.IMul(entryIr.IAdd(vertex, u32(2u)), item) : u32(0u); + const auto unlessRestarted = [&](IrValue& offset) -> IrValue& { + return strip ? entryIr.Select(entryIr.ULessThan(stripPosition, stripStart), entryIr.IMul(vertex, item), offset) : offset; + }; + IrValue& first = fan ? entryIr.IMul(entryIr.IAdd(vertex, u32(1u)), item) : unlessRestarted(entryIr.IMul(entryIr.IAdd(vertex, parity), item)); + IrValue& second = fan ? entryIr.IMul(entryIr.IAdd(vertex, u32(2u)), item) : size >= 2u ? unlessRestarted(entryIr.IMul(entryIr.ISub(entryIr.IAdd(vertex, u32(1u)), parity), item)) : u32(0u); + IrValue& third = fan ? u32(0u) : size == 3u ? unlessRestarted(entryIr.IMul(entryIr.IAdd(vertex, u32(2u)), item)) : u32(0u); entryIr.SetVectorReg(static_cast(0), entryIr.BitwiseOr(entryIr.BitwiseAnd(first, u32(0xffffu)), entryIr.ShiftLeftLogical(second, u32(16u)))); entryIr.SetVectorReg(static_cast(1), entryIr.BitwiseAnd(third, u32(0xffffu))); entryIr.SetVectorReg(static_cast(2), entryIr.IAdd(firstPrimitive, local)); @@ -277,7 +285,7 @@ void emitEntryPrologue(IrProgram& program, IrBlock& entryBlock, const TranslateO throw std::runtime_error("mesh shader translation requires the merged program's eight hidden user words"); } IrValue& inputVertex = fan ? entryIr.Select(entryIr.IEqual(local, u32(0u)), u32(0u), entryIr.IAdd(firstVertex, local)) : entryIr.IAdd(firstVertex, local); - IrValue& indexBytes = draw(3u); + IrValue& indexBytes = entryIr.BitwiseAnd(indexWord, u32(0xffu)); IrValue& indexed = entryIr.INotEqual(indexBytes, u32(0u)); IrValue& byteOffset = entryIr.IMul(entryIr.IAdd(inputVertex, firstIndex), indexBytes); IrValue& indexResource = entryIr.Emit(IrOpcode::GetBufferResource, IrOpcodeType(IrOpcode::GetBufferResource), {&entryIr.GetUserData(static_cast(4)), &entryIr.GetUserData(static_cast(5)), &entryIr.GetUserData(static_cast(6)), &entryIr.GetUserData(static_cast(7))}); @@ -288,7 +296,9 @@ void emitEntryPrologue(IrProgram& program, IrBlock& entryBlock, const TranslateO for (std::uint32_t reg = 4u; reg < 8u; reg++) { entryIr.SetScalarReg(static_cast(reg), u32(0u)); } - entryIr.SetVectorReg(static_cast(5), entryIr.IAdd(draw(1u), entryIr.Select(indexed, index, inputVertex))); + IrValue& restartIndex = entryIr.Select(entryIr.IEqual(indexBytes, u32(2u)), u32(0xffffu), u32(0xffffffffu)); + IrValue& fetchedIndex = entryIr.Select(entryIr.LogicalAnd(restartTable, entryIr.IEqual(index, restartIndex)), u32(0u), index); + entryIr.SetVectorReg(static_cast(5), entryIr.IAdd(draw(1u), entryIr.Select(indexed, fetchedIndex, inputVertex))); entryIr.SetVectorReg(static_cast(6), u32(0u)); entryIr.SetVectorReg(static_cast(7), u32(0u)); entryIr.SetVectorReg(static_cast(8), entryIr.IAdd(draw(2u), builtin(StageInputKind::WorkgroupId, 1u))); From 58e871cf2a6a3ce438a4c045a5044cb49b9b9873 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Fri, 9 Oct 2026 19:02:58 -0500 Subject: [PATCH 039/460] fix(libkernel): place sceKernelReserveVirtualRange at its address hint Without MAP_FIXED the address is a hint, as for FreeBSD's mmap: the reservation goes at the hint when that range is free, else at the first free range above it. The hint was dropped, so every hinted reservation landed at the lowest free address. RetroArch (PPSA99169) reserves its cores' direct-memory heap near 0x2000000000 outside the GPU window (0x200000000-0x300000000) and refuses a range inside it; with every reservation at 0x200400000 the heap was never made, and Dolphin's first aligned allocation returned null. AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- .../prx/libkernel/DirectMemory/DirectMemory.cpp | 2 +- core/libs/tests/GuestMemory.cpp | 17 +++++++++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp index ff3933405..aa5ef902d 100644 --- a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp +++ b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp @@ -918,7 +918,7 @@ int DoReserveVirtual(void** addr, size_t len, int flags, size_t alignment) { } if (fixed) mutation.RequireAvailable(*addr, len); constexpr int GuestMapNoCoalesce = 0x400000; - void* mapped = MapAligned(fixed ? *addr : nullptr, len, PROT_NONE, fixed ? GuestMapFixedFlag | (flags & GuestMapNoCoalesce) : 0, alignment); + void* mapped = MapAligned(*addr, len, PROT_NONE, fixed ? GuestMapFixedFlag | (flags & GuestMapNoCoalesce) : 0, alignment); try { mutation.Add(mapped, len, false, false, false); } catch (...) { diff --git a/core/libs/tests/GuestMemory.cpp b/core/libs/tests/GuestMemory.cpp index caed0eb13..707d2a374 100644 --- a/core/libs/tests/GuestMemory.cpp +++ b/core/libs/tests/GuestMemory.cpp @@ -466,6 +466,22 @@ static void CheckGpuAccessFollowsProtection() { Require(sceKernelMunmap(flexible, page) == 0); } +static void CheckHintedVirtualReservation() { + constexpr std::size_t page = 0x4000; + void* probe = nullptr; + Require(sceKernelReserveVirtualRange(&probe, page * 8, 0, 0) == 0); + Require(sceKernelMunmap(probe, page * 8) == 0); + void* const hint = static_cast(probe) + page * 2; + void* placed = hint; + Require(sceKernelReserveVirtualRange(&placed, page * 2, 0, 0) == 0); + Require(placed == hint); + void* above = hint; + Require(sceKernelReserveVirtualRange(&above, page * 2, 0, 0) == 0); + Require(above > hint); + Require(sceKernelMunmap(above, page * 2) == 0); + Require(sceKernelMunmap(placed, page * 2) == 0); +} + static void CheckFixedVirtualReservation() { constexpr std::size_t page = 0x4000; void* probe = nullptr; @@ -1526,6 +1542,7 @@ int main() { CheckFixedMappingReplacesPartialOverlap(); CheckDirectMemoryGpuProtBits(); CheckGpuAccessFollowsProtection(); + CheckHintedVirtualReservation(); CheckFixedVirtualReservation(); CheckReservedRangeIsNotCommitted(); CheckNoOverwriteRefusesLiveMapping(); From f2b4bf3b5b8c6ec385bc74c2db47c7f78bc9aaac Mon Sep 17 00:00:00 2001 From: Get-fix <100056769+Get-fix@users.noreply.github.com> Date: Sat, 10 Oct 2026 16:12:11 +0300 Subject: [PATCH 040/460] fix(audio): export the sceAudioOutEx bitstream port functions Jelly5 (PPSA99505) imports sceAudioOutExGetMonitorInfo (PhnNN0UEV6Y), sceAudioOutExConfigureOutput (VcE+gXSwFXI), sceAudioOutExOpen (6X6dp+07h4U) and sceAudioOutExClose (0TfjSulCV2A) from libSceAudioOut and fails to load because no library exports them. It calls them only for its optional HDMI bitstream output, and their signatures are unconfirmed, so they are exported without an implementation and throw if they are called, like sceSaveDataCancel. --- core/libs/prx/libSceAudioOut/CMakeLists.txt | 10 ++++++ core/libs/prx/libSceAudioOut/src/AudioOut.cpp | 20 +++++++++++ .../prx/libSceAudioOut/tests/AudioOutEx.cpp | 34 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 65 insertions(+) create mode 100644 core/libs/prx/libSceAudioOut/tests/AudioOutEx.cpp diff --git a/core/libs/prx/libSceAudioOut/CMakeLists.txt b/core/libs/prx/libSceAudioOut/CMakeLists.txt index d42ddf36a..dfd3f5ea7 100644 --- a/core/libs/prx/libSceAudioOut/CMakeLists.txt +++ b/core/libs/prx/libSceAudioOut/CMakeLists.txt @@ -47,6 +47,16 @@ if(BUILD_TESTING) "PATH=path_list_prepend:$") endif() + add_test_executable(audio_out_ex_tests tests/AudioOutEx.cpp) + target_include_directories(audio_out_ex_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(audio_out_ex_tests PRIVATE libSceAudioOut libc) + configure_windows_unwind(audio_out_ex_tests) + add_test(NAME audio_out_ex COMMAND audio_out_ex_tests) + if(WIN32) + set_tests_properties(audio_out_ex PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() + add_test_executable(audio_out2_latency_tests tests/AudioOut2Latency.cpp) target_include_directories(audio_out2_latency_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(audio_out2_latency_tests PRIVATE libSceAudioOut libc) diff --git a/core/libs/prx/libSceAudioOut/src/AudioOut.cpp b/core/libs/prx/libSceAudioOut/src/AudioOut.cpp index 5dcdd2821..959d3419e 100644 --- a/core/libs/prx/libSceAudioOut/src/AudioOut.cpp +++ b/core/libs/prx/libSceAudioOut/src/AudioOut.cpp @@ -515,4 +515,24 @@ int APS5_VABI sceAudioOutSetMixLevelPadSpk(int handle, int mixLevel) { return 0; } +int APS5_VABI sceAudioOutExClose() { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceAudioOutExConfigureOutput() { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceAudioOutExGetMonitorInfo() { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceAudioOutExOpen() { + NotImplemented_nid_no_patch(__func__); + return 0; +} + } diff --git a/core/libs/prx/libSceAudioOut/tests/AudioOutEx.cpp b/core/libs/prx/libSceAudioOut/tests/AudioOutEx.cpp new file mode 100644 index 000000000..8cc1266ea --- /dev/null +++ b/core/libs/prx/libSceAudioOut/tests/AudioOutEx.cpp @@ -0,0 +1,34 @@ +#include "prx/libc/include/general/VabiMacros.hpp" + +#include +#include +#include + +extern "C" { +int APS5_VABI sceAudioOutExClose(); +int APS5_VABI sceAudioOutExConfigureOutput(); +int APS5_VABI sceAudioOutExGetMonitorInfo(); +int APS5_VABI sceAudioOutExOpen(); +} + +using ExFunction = int (APS5_VABI *)(); + +static bool ReportsNotImplemented(ExFunction function, const char* name) { + try { + function(); + } catch (const std::runtime_error& error) { + if (std::string(error.what()) == std::string(name) + " not implemented") return true; + std::fprintf(stderr, "%s threw: %s\n", name, error.what()); + return false; + } + std::fprintf(stderr, "%s returned instead of reporting that it is not implemented\n", name); + return false; +} + +int main() { + bool passed = ReportsNotImplemented(sceAudioOutExClose, "sceAudioOutExClose"); + passed = ReportsNotImplemented(sceAudioOutExConfigureOutput, "sceAudioOutExConfigureOutput") && passed; + passed = ReportsNotImplemented(sceAudioOutExGetMonitorInfo, "sceAudioOutExGetMonitorInfo") && passed; + passed = ReportsNotImplemented(sceAudioOutExOpen, "sceAudioOutExOpen") && passed; + return passed ? 0 : 1; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index c320ae0a1..4464a8b8d 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -103,6 +103,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceSaveDataTransferringMountPs4, sceSaveDataDirNameSearchPs4](../../core/libs/prx/libSceSaveData.native/Export.cpp) (libSceSaveData.native) - arguments assumed to be those of `sceSaveDataTransferringMount` and `sceSaveDataDirNameSearch`: KytyPS5 binds both NIDs to the PS5 functions, and [prosper](https://github.com/mattias800/prosper) captured the same result layout on a title for the search. There is no PS4 save area, so the mount returns `NOT_FOUND` and the search reports zero hits - [sceCoredumpWriteUserString](../../core/libs/prx/libkernel/System/src/Coredump.cpp) and [sceSaveDataConvert, sceSaveDataGetConvertProgress](../../core/libs/prx/libSceSaveData.native/Export.cpp) - imported by PPSA16064, not called during its start-up and added without an implementation; unknown signatures - [sceSaveDataCancel](../../core/libs/prx/libSceSaveData.native/Export.cpp) (libSceSaveData.native) - imported by PPSA23126, which fails to load without it; added without an implementation because its signature is unknown. Every save data call here finishes before it returns, so nothing is ever in flight to cancel, but what the console returns then is unknown +- [sceAudioOutExClose, sceAudioOutExConfigureOutput, sceAudioOutExGetMonitorInfo, sceAudioOutExOpen](../../core/libs/prx/libSceAudioOut/src/AudioOut.cpp) (libSceAudioOut) - imported by PPSA99505, which fails to load without them; added without an implementation and declared without parameters. PPSA99505 calls them only for its HDMI bitstream output, an option that is off by default, to read the formats the TV or receiver lists, switch the HDMI output mode and open a port for IEC 61937 bursts ([Jelly5](https://github.com/02dnot/Jelly5/blob/552b866d89b2a58b27f7f7b0484dc743ce235526/app/src/jelly5_bitstream.cpp#L110-L179)); [EVO-PLAYER-PS5](https://github.com/sainsaji/EVO-PLAYER-PS5/blob/da0c0451333bab82e79fffd2de4718e917f9dcb0/projects/evoplayer/core/src/audio/evo_pt.cpp) also calls `sceAudioOutExOpen`, `sceAudioOutExConfigureOutput` and `sceAudioOutExClose` for HDMI bitstream output. The two declare the arguments of `sceAudioOutExConfigureOutput` differently and neither is confirmed on a console. Audio reaches the host device through SDL as PCM, so there is no encoded HDMI output to switch to - [sce::Json::Parser::parse](../../core/libs/prx/libSceJson2/Export.cpp) (libSceJson2) - a number that overflows a double (e.g. `1e309`) throws: whether the console rejects the document or stores an infinite real is unknown - [sceUserServiceGetNpAccountId](../../core/libs/prx/libSceUserService/Export.cpp) (libSceUserService) - the account id for a user with no linked PSN account is assumed to be 0 with success; libSceNpManager's `sceNpGetAccountIdA` answers the same question with `SCE_NP_ERROR_SIGNED_OUT` - [sceUserServiceGetUserColor](../../core/libs/prx/libSceUserService/Export.cpp) (libSceUserService) - the default profile colour is assumed to be blue (0) From b7dec57a84771328770f3054a28d8d83ce2ae6fe Mon Sep 17 00:00:00 2001 From: Yuvaraj <119717169+Yuva1024@users.noreply.github.com> Date: Sat, 10 Oct 2026 19:25:28 +0530 Subject: [PATCH 041/460] test(avplayer): stop the unsynced test racing the player's auto-start TestUnsyncedVideoKeepsUpWithAudio built its player without an event callback, which enables auto-start (autoStartEnabled = init.auto_start || callback.event_callback == nullptr). The controller thread then enables the streams and starts playback shortly after the source is added, racing the test's own sceAvPlayerEnableStream. When the controller won, Source::EnableStream saw started == true and the test failed with "enable video failed". The test now passes an Events object, as the other tests that drive the player by hand do. Co-Authored-By: Claude Sonnet 5.5 --- core/libs/tests/GuestAvPlayer.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/core/libs/tests/GuestAvPlayer.cpp b/core/libs/tests/GuestAvPlayer.cpp index 2fb32a5db..14206efd2 100644 --- a/core/libs/tests/GuestAvPlayer.cpp +++ b/core/libs/tests/GuestAvPlayer.cpp @@ -1035,7 +1035,8 @@ void TestPs5ExtendedInitLayout() { } void TestUnsyncedVideoKeepsUpWithAudio() { - AvPlayerInitData init = InitData(nullptr); + Events events; + AvPlayerInitData init = InitData(&events); auto* player = sceAvPlayerInit(&init); Check(player != nullptr, "init failed"); Check(sceAvPlayerSetAvSyncMode(player, 1) == 0, "sync mode rejected"); From 14c2eca3b0dce4de9e05c207f32b54eb07b39f60 Mon Sep 17 00:00:00 2001 From: alfalfa-star Date: Wed, 7 Oct 2026 17:35:45 +0200 Subject: [PATCH 042/460] perf(agc): sync only the source for DMA_DATA copies into host-only memory A DMA_DATA copy larger than the store limit runs as a transfer between host imports. When the destination is not registered guest memory (the GPU can neither read nor write it, so no import exists), the transfer was refused and the packet drained the device: every batch recorded so far was submitted and waited for, then the CPU copied. Such a copy only has to see the source's results. When CopyBuffer refused for want of an import (path 2) and nothing can still store over the destination (VulkanDevice::StoresPendingOver: a recorded write, a label, a copied write-back, a storage image's pending write-back or a unit shadow), the packet now flushes storage results pending over the source and waits for the batches that write it (Recorder::SyncThrough), then copies on the CPU without a drain. Anything else still drains as before. The [sync] line counts these copies. agc_driver_tests copies a buffer the GPU just wrote into host memory, once directly and once behind a label into the destination, and checks the bytes and that the copy did not drain. PPSA14632 (LEGO Horizon Adventures) copies 256 KiB of GPU results into host memory this way, about 50 times per 10 s: device drains drop from about 50 to none per 10 s. Co-Authored-By: Claude Opus 5.5 --- .../SynchronizationStatistics.hpp | 2 +- .../Execution/include/VulkanDevice.hpp | 1 + .../src/Driver/Packets/PacketMemory.cpp | 8 +++++ .../SynchronizationStatistics.cpp | 4 +-- .../Execution/src/VulkanDevice.cpp | 6 ++++ .../libs/prx/libSceAgcDriver/tests/Submit.cpp | 31 +++++++++++++++++++ 6 files changed, 49 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Synchronization/SynchronizationStatistics.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Synchronization/SynchronizationStatistics.hpp index 48d832499..80cbe89d1 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Synchronization/SynchronizationStatistics.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Synchronization/SynchronizationStatistics.hpp @@ -30,7 +30,7 @@ enum TrySite { TryLabel = 0, TryFlush, TryReap, TryPoll, TryIdle, TryCapture, Tr extern std::atomic triesFailed[TrySites]; -extern std::atomic storesOnGpu, storesBehindCompletions, storesOnCpu, storesDrained; +extern std::atomic storesOnGpu, storesBehindCompletions, storesOnCpu, storesDrained, copiesToHostMemory; extern std::map drainCounts; diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp index d35c1e2ef..272ef9b13 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp @@ -78,6 +78,7 @@ public: std::optional PendingLabel(std::uint64_t address, std::size_t bytes, std::uint64_t afterStamp, Graphics::Recorder::LabelRefusal* refusal = nullptr) const; bool OpenWriteOverlaps(std::uint64_t address, std::size_t bytes) const; bool RecordedWritesSettled(std::uint64_t address, std::size_t bytes) const; + bool StoresPendingOver(std::uint64_t address, std::size_t bytes) const; // Fills [address, address + bytes) of host-imported guest memory with a repeating 16-byte pattern, // recorded behind the open batch; false when the range is not imported (the caller stores it). // Recorded GPU stores over the range are ordered before the fill by its barrier; finished diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Packets/PacketMemory.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Packets/PacketMemory.cpp index 15273ae87..1e381a778 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Packets/PacketMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Packets/PacketMemory.cpp @@ -5,6 +5,8 @@ #include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" #include "prx/libSceAgcDriver/Execution/include/Pm4.hpp" #include "prx/libSceAgcDriver/Eq/include/Event.hpp" +#include "prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Recorder.hpp" #include #include #include @@ -143,6 +145,12 @@ bool Driver::preparePacketMemory(const Submission& submission, QueueState& queue if (outcome.path == 1 || outcome.path == 3) { wroteOnGpu = true; drained = false; + } else if (outcome.path == 2 && !Graphics::RegisteredReadableCovers(copy->destination, copy->bytes) && !localDevice->StoresPendingOver(copy->destination, copy->bytes)) { + Graphics::StorageTexture::FlushPending(copy->source, copy->bytes, nullptr, "buffer copy source", Graphics::PublishScope::Whole); + if (auto* recorder = Graphics::Recorder::Active()) recorder->SyncThrough(copy->source, copy->bytes); + ++copiesToHostMemory; + orderedAlready = true; + drained = false; } } } diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/SynchronizationStatistics.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/SynchronizationStatistics.cpp index ca531b0cb..06031abbb 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/SynchronizationStatistics.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/SynchronizationStatistics.cpp @@ -21,7 +21,7 @@ std::atomic packetLockRecordUs{0}, packetLockSubmitUs{0}; std::atomic triesFailed[TrySites] = {}; -std::atomic storesOnGpu{0}, storesBehindCompletions{0}, storesOnCpu{0}, storesDrained{0}; +std::atomic storesOnGpu{0}, storesBehindCompletions{0}, storesOnCpu{0}, storesDrained{0}, copiesToHostMemory{0}; std::map drainCounts; @@ -39,7 +39,7 @@ SubmissionCosts& submissionCosts(std::uint32_t queue) { void reportSync() { std::string report; for (const auto& [code, count] : drainCounts) report += " " + (code == 0xffffu ? std::string("flip") : Pm4::Name(code << 8u)) + "=" + std::to_string(count); - AgcDriver::ProfilePrint_nid_no_patch("[sync] %llu device drains by packet:%s (%llu waited without the GPU mutex); %llu labels written on the GPU, %llu deferred behind completions, %llu no-op, fallbacks: idle %llu, completions %llu, not imported %llu, undecodable %llu; %llu labels queued per worker, recorded in %llu groups, %llu recorded at once; stores: %llu recorded on the GPU, %llu behind completions, %llu on the CPU (idle), %llu synced (drained)\n", static_cast(drainTotal), report.c_str(), static_cast(unlockedDrains.load()), static_cast(gpuLabels.load()), static_cast(completionLabels.load()), static_cast(noOpLabels.load()), static_cast(labelFallbacks[1].load()), static_cast(labelFallbacks[2].load()), static_cast(labelFallbacks[3].load()), static_cast(labelFallbacks[4].load()), static_cast(queuedLabels.load()), static_cast(labelGroups.load()), static_cast(immediateLabels.load()), static_cast(storesOnGpu.load()), static_cast(storesBehindCompletions.load()), static_cast(storesOnCpu.load()), static_cast(storesDrained.load())); + AgcDriver::ProfilePrint_nid_no_patch("[sync] %llu device drains by packet:%s (%llu waited without the GPU mutex); %llu labels written on the GPU, %llu deferred behind completions, %llu no-op, fallbacks: idle %llu, completions %llu, not imported %llu, undecodable %llu; %llu labels queued per worker, recorded in %llu groups, %llu recorded at once; stores: %llu recorded on the GPU, %llu behind completions, %llu on the CPU (idle), %llu synced (drained); %llu copies into host memory synced on their source\n", static_cast(drainTotal), report.c_str(), static_cast(unlockedDrains.load()), static_cast(gpuLabels.load()), static_cast(completionLabels.load()), static_cast(noOpLabels.load()), static_cast(labelFallbacks[1].load()), static_cast(labelFallbacks[2].load()), static_cast(labelFallbacks[3].load()), static_cast(labelFallbacks[4].load()), static_cast(queuedLabels.load()), static_cast(labelGroups.load()), static_cast(immediateLabels.load()), static_cast(storesOnGpu.load()), static_cast(storesBehindCompletions.load()), static_cast(storesOnCpu.load()), static_cast(storesDrained.load()), static_cast(copiesToHostMemory.load())); } void countLabelOutcome(int reason) { diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index be76518fd..3d3bb3b77 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -1465,6 +1465,12 @@ bool VulkanDevice::RecordedWritesSettled(std::uint64_t address, std::size_t byte return Graphics::Recorder::PendingCompletionLabels() == 0 || !recorder.CompletionLabelIn(address, bytes); } +bool VulkanDevice::StoresPendingOver(std::uint64_t address, std::size_t bytes) const { + const std::array, 1> range{{{address, address + bytes}}}; + if (Graphics::StorageTexture::AnyPendingOverlaps(range) || Graphics::AnyShadowedOverlaps(range) || state->CopiedWriterOverlaps(address, bytes)) return true; + return state->recorder && (state->recorder->PendingWriteOverlaps(address, bytes) || state->recorder->PendingLabelIn(address, bytes)); +} + int VulkanDevice::WriteLabelOnGpu(std::uint64_t address, std::span bytes, std::uint64_t stamp, std::uint32_t queue, bool reapFirst) { if (!state->recorder || bytes.empty() || bytes.size() > 65536 || address % 4 != 0 || bytes.size() % 4 != 0) return 4; auto& recorder = *state->recorder; diff --git a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp index 3c9db0b0c..cd6f89778 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp @@ -2,6 +2,8 @@ #include "execution/VulkanTestDevice.hpp" #include "prx/libSceAgcDriver/Execution/include/QueueState.hpp" #include "prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp" +#include "prx/libSceAgcDriver/Execution/include/Driver/Synchronization/SynchronizationStatistics.hpp" +#include "prx/libc/include/GuestHeap.hpp" #include "prx/libc/include/Shutdown.hpp" #include "prx/libSceAgcDriver/Submit/include/Dcb.hpp" #include "prx/libSceAgcDriver/Submit/include/Acb.hpp" @@ -440,6 +442,34 @@ void testMultiSubmissions() { } } +std::array memoryCopy(const void* destination, const void* source, std::uint32_t bytes) { + const auto from = reinterpret_cast(source), to = reinterpret_cast(destination); + return {0xc0055000, 0x60000000, static_cast(from), static_cast(static_cast(from) >> 32u), static_cast(to), static_cast(static_cast(to) >> 32u), bytes}; +} + +void testCopyIntoHostMemory() { + constexpr std::size_t words = 32768; + constexpr auto bytes = static_cast(words * 4); + auto* produced = static_cast(GuestHeap::GuestHeapAlign_nid_postfix(65536, bytes)); + auto* source = static_cast(GuestHeap::GuestHeapAlign_nid_postfix(65536, bytes)); + std::vector destination(words); + for (const bool label : {false, true}) { + for (std::size_t i = 0; i < words; ++i) { + produced[i] = 0xa5000000u | static_cast(i); + source[i] = 0x5b000000u | static_cast(i); + } + std::fill(destination.begin(), destination.end(), 0u); + const auto copiesBefore = AgcDriver::DriverDetail::copiesToHostMemory.load(); + if (label) submit(0, commands(memoryCopy(source, produced, bytes), writeData(destination.data(), 0x1abe1u), memoryCopy(destination.data(), source, bytes))); + else submit(0, commands(memoryCopy(source, produced, bytes), memoryCopy(destination.data(), source, bytes))); + AgcDriverWaitIdle_nid_postfix(); + check(std::equal(destination.begin(), destination.end(), produced), label ? "a copy into host memory behind a label into its destination lost bytes" : "a copy into host memory did not see the GPU copy into its source"); + check(AgcDriver::DriverDetail::copiesToHostMemory.load() == copiesBefore + 1, "a copy into host memory drained instead of syncing its source"); + } + GuestHeap::GuestHeapFree_nid_postfix(source); + GuestHeap::GuestHeapFree_nid_postfix(produced); +} + void testShaderHeaderAlignment() { alignas(256) static const std::array code{0xbf810000}; struct Header { @@ -643,6 +673,7 @@ int main() { testMultiSubmissions(); testShaderHeaderAlignment(); testHeaderWithoutProgramAddress(); + testCopyIntoHostMemory(); testRegisteredFloatMode(); testWorkerFailure(); check(expectFailure([] { LibcRunShutdown_nid_postfix(); }).find("required shader register") != std::string::npos, "shutdown lost worker failure"); From 48693920aa5008db3c993ddd523cf30aee2b38ba Mon Sep 17 00:00:00 2001 From: Celegans12 <53881228+Celegans12@users.noreply.github.com> Date: Sat, 10 Oct 2026 16:08:00 +0200 Subject: [PATCH 043/460] perf(recompiler): reuse dominance intersection storage Intersect copied fixed-point sets without allocating each intermediate. Preserve sorted results and the existing analysis iteration order. Compact oversized storage only when publishing a changed set. Keep an allocating control switch outside the cache key. Compare complete fixed points over empty intersections, loops and DAGs. Register the same equivalence fixtures for the allocating control. --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 5 ++ .../libSceAgcDriver/tests/Structurizer.cpp | 31 +++++++++++++ .../ControlFlow/src/Structurizer.cpp | 46 ++++++++++++++++--- core/shader/recompiler/ShaderDiskCache.cpp | 1 + 4 files changed, 77 insertions(+), 6 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index d24c90ab0..b3205d739 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -921,6 +921,11 @@ if(BUILD_TESTING) "PATH=path_list_prepend:$") endif() endforeach() + add_test(NAME agc_structurizer_intersection_off COMMAND agc_structurizer_tests) + set_tests_properties(agc_structurizer_intersection_off PROPERTIES ENVIRONMENT "ANYPS5_NO_SHADER_CACHE=1;APS5_NO_PERF_DOM_INTERSECTION=1" WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}") + if(WIN32) + set_tests_properties(agc_structurizer_intersection_off PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") + endif() set_tests_properties(agc_driver_provoking_vertex PROPERTIES TIMEOUT 30) set_tests_properties(agc_driver_flat_store PROPERTIES RUN_SERIAL TRUE) set_tests_properties(agc_driver_storage_cache_budget PROPERTIES TIMEOUT 30) diff --git a/core/libs/prx/libSceAgcDriver/tests/Structurizer.cpp b/core/libs/prx/libSceAgcDriver/tests/Structurizer.cpp index d6dfb7861..9fdc01513 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Structurizer.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Structurizer.cpp @@ -276,6 +276,37 @@ int main() { auto reordered = makeGraph({{1}, {}, {2}, {0, 4}, {1}}); reordered.entryBlock = 3; requireOriginalDominanceSets(std::move(reordered), "a nonzero entry with permuted block IDs"); + requireOriginalDominanceSets(makeGraph({{1, 2}, {}, {}}), "disjoint exits with an empty post-dominator intersection"); + requireOriginalDominanceSets(makeGraph({{2}, {2}, {}}), "separate roots with an empty dominator intersection"); + requireOriginalDominanceSets(makeGraph({{1}, {1, 2}, {}}), "a self-loop with an exit"); + std::uint32_t randomState = 0x7b19a5c3u; + const auto randomWord = [&] { randomState = randomState * 1664525u + 1013904223u; return randomState; }; + for (std::uint32_t trial = 0; trial < 24u; ++trial) { + const auto count = 8u + trial % 9u; + std::vector ids(count); + for (std::uint32_t id = 0; id < count; ++id) ids[id] = id; + for (auto size = count; size > 1u; --size) std::swap(ids[size - 1u], ids[randomWord() % size]); + if (ids.front() == 0u) std::swap(ids.front(), ids.back()); + std::vector> successors(count); + for (std::uint32_t position = 0; position + 1u < count; ++position) { + const auto remaining = count - position - 1u; + const auto degree = position == 0u ? 2u : position == 1u ? 1u : std::min(randomWord() % 3u, remaining); + if (degree == 0u) continue; + auto& edges = successors[ids[position]]; + const auto first = position + 1u + randomWord() % remaining; + edges.push_back(ids[first]); + if (degree == 2u) { + auto second = position + 1u + randomWord() % (remaining - 1u); + if (second >= first) ++second; + edges.push_back(ids[second]); + } + std::sort(edges.begin(), edges.end()); + } + auto dag = makeGraph(successors); + dag.entryBlock = ids.front(); + const auto name = "deterministic DAG " + std::to_string(trial); + requireOriginalDominanceSets(std::move(dag), name.c_str()); + } } catch (const std::exception& error) { std::fprintf(stderr, "%s\n", error.what()); return 1; diff --git a/core/shader/recompiler/ControlFlow/src/Structurizer.cpp b/core/shader/recompiler/ControlFlow/src/Structurizer.cpp index d86026724..df7ac3f8b 100644 --- a/core/shader/recompiler/ControlFlow/src/Structurizer.cpp +++ b/core/shader/recompiler/ControlFlow/src/Structurizer.cpp @@ -1,6 +1,7 @@ #include "ControlFlow/Structurizer.hpp" #include "ControlFlow/ControlFlowHelpers.hpp" #include +#include #include #include #include @@ -20,6 +21,39 @@ std::vector allBlockIds(std::uint32_t count) { return ids; } +bool inPlaceIntersections() { + static const bool enabled = std::getenv("APS5_NO_PERF_DOM_INTERSECTION") == nullptr; + return enabled; +} + +void intersectAnalysisSet(std::vector& values, const std::vector& other) { + if (!inPlaceIntersections()) { + values = intersectSorted(values, other); + return; + } + std::size_t left = 0; + std::size_t right = 0; + std::size_t count = 0; + while (left < values.size() && right < other.size()) { + if (values[left] < other[right]) { + ++left; + } else if (other[right] < values[left]) { + ++right; + } else { + values[count++] = values[left++]; + ++right; + } + } + values.resize(count); +} + +void replaceAnalysisSet(std::vector& destination, std::vector&& values) { + if (inPlaceIntersections() && values.capacity() - values.size() > values.size()) { + std::vector(values).swap(values); + } + destination = std::move(values); +} + void addSortedUnique(std::vector& values, std::uint32_t value) { const auto position = std::lower_bound(values.begin(), values.end(), value); if (position == values.end() || *position != value) { @@ -351,12 +385,12 @@ std::uint32_t mergeBesideReturns(const ControlFlowGraph& graph, const BasicBlock } auto next = postDominators[kept[block.id].front()]; for (std::size_t i = 1; i < kept[block.id].size(); ++i) { - next = intersectSorted(next, postDominators[kept[block.id][i]]); + intersectAnalysisSet(next, postDominators[kept[block.id][i]]); } addUnique(next, block.id); sortUnique(next); if (next != postDominators[block.id]) { - postDominators[block.id] = std::move(next); + replaceAnalysisSet(postDominators[block.id], std::move(next)); changed = true; } } @@ -1340,13 +1374,13 @@ void Structurizer::computeDominatorTree(ControlFlowGraph& graph) const { } else { next = graph.blocks[block.predecessors.front()].dominators; for (std::size_t i = 1; i < block.predecessors.size(); ++i) { - next = intersectSorted(next, graph.blocks[block.predecessors[i]].dominators); + intersectAnalysisSet(next, graph.blocks[block.predecessors[i]].dominators); } addSortedUnique(next, block.id); } if (next != block.dominators) { - block.dominators = std::move(next); + replaceAnalysisSet(block.dominators, std::move(next)); changed = true; } } @@ -1421,13 +1455,13 @@ void Structurizer::computePostDominators(ControlFlowGraph& graph) const { } else { next = graph.blocks[block.successors.front()].postDominators; for (std::size_t i = 1; i < block.successors.size(); ++i) { - next = intersectSorted(next, graph.blocks[block.successors[i]].postDominators); + intersectAnalysisSet(next, graph.blocks[block.successors[i]].postDominators); } addSortedUnique(next, block.id); } if (next != block.postDominators) { - block.postDominators = std::move(next); + replaceAnalysisSet(block.postDominators, std::move(next)); changed = true; } } diff --git a/core/shader/recompiler/ShaderDiskCache.cpp b/core/shader/recompiler/ShaderDiskCache.cpp index cb69e754d..43d5d9ba2 100644 --- a/core/shader/recompiler/ShaderDiskCache.cpp +++ b/core/shader/recompiler/ShaderDiskCache.cpp @@ -660,6 +660,7 @@ constexpr std::string_view NeutralSwitches[] = { "APS5_NO_CODE_HASH_KEY", "APS5_NO_FAILURE_MEMO", "APS5_NO_PERF_FRONTEND_PAIR", + "APS5_NO_PERF_DOM_INTERSECTION", "APS5_NO_RESULT_MEMO", }; From aa9ee1b0ad9759aaf42c5244f93418345a848b1b Mon Sep 17 00:00:00 2001 From: TideOPL Date: Fri, 9 Oct 2026 11:33:08 +0100 Subject: [PATCH 044/460] fix(agc): give pooled buffers a VkBuffer of exactly the requested size Buffer and DeviceBuffer created their VkBuffer at the pool's size class, so a robust read past the requested size stayed in bounds and returned the bytes an earlier user of the pooled memory left there instead of an out-of-range result. Each VkBuffer now has the requested size, created with the pool's usage, over memory of its class (or its slab slot); one taken from the pool for a different size is recreated over the same memory at the same offset. agc_buffer_pool follows a pooled buffer by its memory, so its class and reuse cases hold with the exact-size buffers. --- .../Graphics/include/BufferPool.hpp | 2 +- .../Graphics/include/Resources.hpp | 1 - .../Graphics/src/Resources.cpp | 67 +++- .../prx/libSceAgcDriver/tests/BufferPool.cpp | 58 +-- .../tests/execution/PooledBufferRange.cpp | 343 ++++++++++++++++++ 5 files changed, 437 insertions(+), 34 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/PooledBufferRange.cpp diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/BufferPool.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/BufferPool.hpp index 454890253..b2b4d0978 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/BufferPool.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/BufferPool.hpp @@ -18,12 +18,12 @@ struct BufferAllocation { void* mapping; VkDeviceAddress address; VkDeviceSize allocationBytes; - // Size the VkBuffer was created with (see BufferPool::Capacity), not the size a user asked for. std::size_t bytes; VkBufferUsageFlags usage; VkMemoryPropertyFlags properties = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT; VkDeviceSize offset = 0; bool slab = false; + std::size_t bufferBytes = 0; }; struct SlabSlot { diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/Resources.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/Resources.hpp index 3c288acfb..8830f781d 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/Resources.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/Resources.hpp @@ -29,7 +29,6 @@ private: VkDeviceMemory memory = VK_NULL_HANDLE; void* mapping = nullptr; std::size_t size; - // The VkBuffer's own size, `size` rounded up to its pool class (see BufferPool::Capacity). std::size_t capacity; // Fully made (or taken from the pool), so release returns it to the pool instead of destroying // the handles a failed construction left. diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Resources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Resources.cpp index 361f9d540..70026869a 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Resources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Resources.cpp @@ -3,10 +3,40 @@ #include "prx/libSceAgcDriver/Graphics/include/Recorder.hpp" #include "prx/libSceAgcDriver/Execution/include/PerformanceTimer.hpp" #include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" +#include #include namespace AgcDriver::Graphics { +namespace { + +VkDeviceSize ClassAllocationBytes(const VkMemoryRequirements& requirements, std::size_t capacity) { + const auto alignment = std::max(requirements.alignment, 1); + return std::max(requirements.size, (static_cast(capacity) + alignment - 1) / alignment * alignment); +} + +VkBuffer BindExactBuffer(const Context& context, std::size_t size, VkBufferUsageFlags usage, VkDeviceMemory memory, VkDeviceSize offset, VkDeviceSize availableBytes) { + VkBufferCreateInfo info{VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO}; + info.size = size; + info.usage = usage; + info.sharingMode = VK_SHARING_MODE_EXCLUSIVE; + VkBuffer buffer = VK_NULL_HANDLE; + Check(context.Function("vkCreateBuffer")(context.device, &info, nullptr, &buffer), "vkCreateBuffer pooled"); + try { + VkMemoryRequirements requirements{}; + context.Function("vkGetBufferMemoryRequirements")(context.device, buffer, &requirements); + Require(requirements.size <= availableBytes, "pooled buffer memory is smaller than the requested buffer"); + Require(requirements.alignment == 0 || offset % requirements.alignment == 0, "pooled buffer offset breaks the requested buffer's alignment"); + Check(context.Function("vkBindBufferMemory")(context.device, buffer, memory, offset), "vkBindBufferMemory pooled"); + } catch (...) { + context.Function("vkDestroyBuffer")(context.device, buffer, nullptr); + throw; + } + return buffer; +} + +} + Buffer::Buffer(const Context& context, std::size_t size, VkBufferUsageFlags usage, VkMemoryPropertyFlags properties) : context(context), size(size), capacity(BufferPool::Capacity(size, properties)), usage(BufferPool::Usage(usage, properties)), properties(properties) { Require(size != 0, "zero-sized GPU buffer"); const bool addressable = (usage & VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT) != 0; @@ -21,12 +51,24 @@ Buffer::Buffer(const Context& context, std::size_t size, VkBufferUsageFlags usag capacity = allocation->bytes; offset = allocation->offset; slab = allocation->slab; + if (allocation->bufferBytes != size) { + context.Function("vkDestroyBuffer")(context.device, buffer, nullptr); + buffer = VK_NULL_HANDLE; + deviceAddress = 0; + try { + buffer = BindExactBuffer(context, size, this->usage, memory, offset, allocationBytes); + initializeAddress(usage); + } catch (...) { + release(); + throw; + } + } ready = true; return; } try { VkBufferCreateInfo info{VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO}; - info.size = capacity; + info.size = size; info.usage = this->usage; info.sharingMode = VK_SHARING_MODE_EXCLUSIVE; Check(context.Function("vkCreateBuffer")(context.device, &info, nullptr, &buffer), "vkCreateBuffer"); @@ -35,8 +77,8 @@ Buffer::Buffer(const Context& context, std::size_t size, VkBufferUsageFlags usag VkMemoryAllocateInfo allocation{VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO}; const VkMemoryAllocateFlagsInfo flags{VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_FLAGS_INFO, nullptr, VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT, 0}; if (addressable) allocation.pNext = &flags; - allocation.allocationSize = requirements.size; - allocationBytes = requirements.size; + allocation.allocationSize = ClassAllocationBytes(requirements, capacity); + allocationBytes = allocation.allocationSize; // The CPU reads most of these buffers back (write-back, diffs), which is very slow from // write-combined memory, so the default host properties prefer cached host memory. constexpr VkMemoryPropertyFlags hostDefault = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT; @@ -78,7 +120,7 @@ Buffer::~Buffer() { void Buffer::release() noexcept { if (ready && cache) { - cache->Put({buffer, memory, mapping, deviceAddress, allocationBytes, capacity, usage, properties, offset, slab}); + cache->Put({buffer, memory, mapping, deviceAddress, allocationBytes, capacity, usage, properties, offset, slab, size}); return; } if (slab) { @@ -117,19 +159,28 @@ DeviceBuffer::DeviceBuffer(const Context& context, std::size_t size, VkBufferUsa memory = allocation->memory; allocationBytes = allocation->allocationBytes; capacity = allocation->bytes; + if (allocation->bufferBytes == size) return; + context.Function("vkDestroyBuffer")(context.device, buffer, nullptr); + buffer = VK_NULL_HANDLE; + try { + buffer = BindExactBuffer(context, size, this->usage, memory, 0, allocationBytes); + } catch (...) { + release(); + throw; + } return; } try { VkBufferCreateInfo info{VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO}; - info.size = capacity; + info.size = size; info.usage = this->usage; info.sharingMode = VK_SHARING_MODE_EXCLUSIVE; Check(context.Function("vkCreateBuffer")(context.device, &info, nullptr, &buffer), "vkCreateBuffer device"); VkMemoryRequirements requirements{}; context.Function("vkGetBufferMemoryRequirements")(context.device, buffer, &requirements); VkMemoryAllocateInfo allocation{VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO}; - allocation.allocationSize = requirements.size; - allocationBytes = requirements.size; + allocation.allocationSize = ClassAllocationBytes(requirements, capacity); + allocationBytes = allocation.allocationSize; allocation.memoryTypeIndex = context.MemoryType(requirements.memoryTypeBits, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT); Check(context.Function("vkAllocateMemory")(context.device, &allocation, nullptr, &memory), "vkAllocateMemory device buffer"); Check(context.Function("vkBindBufferMemory")(context.device, buffer, memory, 0), "vkBindBufferMemory device"); @@ -145,7 +196,7 @@ DeviceBuffer::~DeviceBuffer() { void DeviceBuffer::release() noexcept { if (buffer && memory && cache) { - cache->Put({buffer, memory, nullptr, 0, allocationBytes, capacity, usage, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT}); + cache->Put({buffer, memory, nullptr, 0, allocationBytes, capacity, usage, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT, 0, false, size}); return; } if (buffer) context.Function("vkDestroyBuffer")(context.device, buffer, nullptr); diff --git a/core/libs/prx/libSceAgcDriver/tests/BufferPool.cpp b/core/libs/prx/libSceAgcDriver/tests/BufferPool.cpp index 8c8cb0ec7..e1379137b 100644 --- a/core/libs/prx/libSceAgcDriver/tests/BufferPool.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/BufferPool.cpp @@ -105,6 +105,14 @@ PFN_vkVoidFunction VKAPI_CALL mockProc(VkDevice, const char* name) { return it == table.end() ? nullptr : it->second; } +VkDeviceMemory MemoryOf(VkBuffer buffer) { + return mock.bound.at(buffer).first; +} + +VkDeviceSize ClassOf(VkBuffer buffer) { + return mock.memoryBytes.at(MemoryOf(buffer)); +} + Context mockContext() { Context context{}; context.deviceProc = mockProc; @@ -119,44 +127,46 @@ Context mockContext() { void SizeClassesAndDirections() { mock = MockDevice{}; auto context = mockContext(); - VkBuffer first = VK_NULL_HANDLE; + VkDeviceMemory first = VK_NULL_HANDLE; { DeviceBuffer upload(context, 10 * MiB + 4096, VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT); - first = upload.Handle(); - Expect(mock.sizes.at(first) == 11 * MiB, "a 10 MiB + 4 KiB device buffer was created with " + std::to_string(mock.sizes.at(first)) + " bytes, not its 11 MiB class"); + first = MemoryOf(upload.Handle()); + Expect(mock.sizes.at(upload.Handle()) == 10 * MiB + 4096, "a 10 MiB + 4 KiB device buffer was created with " + std::to_string(mock.sizes.at(upload.Handle())) + " bytes, not the requested size"); + Expect(ClassOf(upload.Handle()) == 11 * MiB, "a 10 MiB + 4 KiB device buffer has " + std::to_string(ClassOf(upload.Handle())) + " bytes of memory, not its 11 MiB class"); constexpr VkBufferUsageFlags all = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT; - Expect((mock.usages.at(first) & all) == all, "a device buffer was created without storage and both transfer directions"); + Expect((mock.usages.at(upload.Handle()) & all) == all, "a device buffer was created without storage and both transfer directions"); Expect(upload.Size() == 10 * MiB + 4096, "a device buffer reports its class instead of the requested size"); } const auto made = mock.allocations; DeviceBuffer writeBack(context, 10 * MiB + 512 * 1024, VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT); - Expect(writeBack.Handle() == first && mock.allocations == made, "a retained 11 MiB device buffer did not serve a write-back of another size and direction in its class"); + Expect(MemoryOf(writeBack.Handle()) == first && mock.allocations == made, "a retained 11 MiB device buffer did not serve a write-back of another size and direction in its class"); Expect(writeBack.Size() == 10 * MiB + 512 * 1024, "a reused device buffer reports the retained size"); + Expect(mock.sizes.at(writeBack.Handle()) == 10 * MiB + 512 * 1024, "a reused device buffer kept the VkBuffer of the earlier size"); } void LargerClass() { mock = MockDevice{}; auto context = mockContext(); constexpr auto usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT; - VkBuffer twelve = VK_NULL_HANDLE; + VkDeviceMemory twelve = VK_NULL_HANDLE; { DeviceBuffer large(context, 12 * MiB, usage); - twelve = large.Handle(); + twelve = MemoryOf(large.Handle()); } { DeviceBuffer smaller(context, 7 * MiB, usage); - Expect(smaller.Handle() == twelve, "a 7 MiB request did not take the retained 12 MiB buffer"); + Expect(MemoryOf(smaller.Handle()) == twelve, "a 7 MiB request did not take the retained 12 MiB buffer"); } const auto made = mock.allocations; { DeviceBuffer tooSmall(context, 5 * MiB, usage); - Expect(tooSmall.Handle() != twelve && mock.allocations == made + 1, "a 5 MiB request took a retained buffer of more than twice its size"); + Expect(MemoryOf(tooSmall.Handle()) != twelve && mock.allocations == made + 1, "a 5 MiB request took a retained buffer of more than twice its size"); Expect(mock.sizes.at(tooSmall.Handle()) == 5 * MiB, "a 5 MiB device buffer was not created at its own class"); } DeviceBuffer again(context, 12 * MiB, usage); - Expect(again.Handle() == twelve, "a buffer served to a smaller request did not return to its own 12 MiB class"); + Expect(MemoryOf(again.Handle()) == twelve, "a buffer served to a smaller request did not return to its own 12 MiB class"); DeviceBuffer best(context, 4 * MiB + 1, usage); - Expect(best.Handle() != twelve && mock.sizes.at(best.Handle()) == 5 * MiB, "a 4 MiB + 1 request did not take the smallest retained class that fits (5 MiB)"); + Expect(MemoryOf(best.Handle()) != twelve && ClassOf(best.Handle()) == 5 * MiB, "a 4 MiB + 1 request did not take the smallest retained class that fits (5 MiB)"); } void KeptUntilFence() { @@ -187,28 +197,28 @@ void Budget() { constexpr auto usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT; std::vector> live; for (int i = 0; i < 12; ++i) live.push_back(std::make_unique(context, (48 + static_cast(i) * 3) * MiB, usage)); - const auto first = live.front()->Handle(); + const auto first = MemoryOf(live.front()->Handle()); VkDeviceSize madeBytes = 0; - for (const auto& buffer : live) madeBytes += mock.sizes.at(buffer->Handle()); + for (const auto& buffer : live) madeBytes += ClassOf(buffer->Handle()); Expect(madeBytes > 512 * MiB, "the budget case does not exceed the budget"); for (auto& buffer : live) buffer.reset(); Expect(mock.frees != 0 && mock.liveBytes <= 512 * MiB, "the device tier retains " + std::to_string(mock.liveBytes / MiB) + " MiB after " + std::to_string(mock.frees) + " evictions, over its 512 MiB budget"); DeviceBuffer probe(context, 48 * MiB, usage); - Expect(probe.Handle() != first, "the least recently used device buffer survived the eviction"); + Expect(MemoryOf(probe.Handle()) != first, "the least recently used device buffer survived the eviction"); } void AddressAndHostUnchanged() { mock = MockDevice{}; auto context = mockContext(); - VkBuffer scratch = VK_NULL_HANDLE; + VkDeviceMemory scratch = VK_NULL_HANDLE; { DeviceBuffer buffer(context, 2 * MiB, VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT); - scratch = buffer.Handle(); + scratch = MemoryOf(buffer.Handle()); } Buffer addressed(context, 2 * MiB, VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT); - Expect(addressed.Handle() != scratch && addressed.DeviceAddress() != 0, "an addressable device-local buffer took a scratch buffer without a device address"); + Expect(MemoryOf(addressed.Handle()) != scratch && addressed.DeviceAddress() != 0, "an addressable device-local buffer took a scratch buffer without a device address"); Buffer shadow(context, 2 * MiB - 256, VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT); - Expect(shadow.Handle() == scratch, "a device-local staging buffer did not share the scratch buffers' class"); + Expect(MemoryOf(shadow.Handle()) == scratch, "a device-local staging buffer did not share the scratch buffers' class"); VkBuffer host = VK_NULL_HANDLE; { Buffer copy(context, 10 * MiB + 4096, VK_BUFFER_USAGE_TRANSFER_SRC_BIT, HostVisible); @@ -226,19 +236,19 @@ void SmallDeviceClasses() { mock = MockDevice{}; auto context = mockContext(); constexpr auto usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT; - VkBuffer mebibyte = VK_NULL_HANDLE; - VkBuffer half = VK_NULL_HANDLE; + VkDeviceMemory mebibyte = VK_NULL_HANDLE; + VkDeviceMemory half = VK_NULL_HANDLE; { DeviceBuffer one(context, MiB, usage); - mebibyte = one.Handle(); + mebibyte = MemoryOf(one.Handle()); } { DeviceBuffer smaller(context, 300 * 1024, usage); - Expect(smaller.Handle() != mebibyte && mock.sizes.at(smaller.Handle()) == 512 * 1024, "a 300 KiB request took a 1 MiB buffer or skipped its power-of-two class"); - half = smaller.Handle(); + Expect(MemoryOf(smaller.Handle()) != mebibyte && ClassOf(smaller.Handle()) == 512 * 1024, "a 300 KiB request took a 1 MiB buffer or skipped its power-of-two class"); + half = MemoryOf(smaller.Handle()); } DeviceBuffer again(context, 260 * 1024, usage); - Expect(again.Handle() == half, "a 260 KiB request did not take the retained 512 KiB buffer"); + Expect(MemoryOf(again.Handle()) == half, "a 260 KiB request did not take the retained 512 KiB buffer"); constexpr auto indirect = VK_BUFFER_USAGE_INDIRECT_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT; VkBuffer tiny = VK_NULL_HANDLE; { diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/PooledBufferRange.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/PooledBufferRange.cpp new file mode 100644 index 000000000..42ac67075 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/PooledBufferRange.cpp @@ -0,0 +1,343 @@ +#include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" +#include "prx/libSceAgcDriver/Graphics/include/BufferPool.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Resources.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::BufferPool; +using AgcDriver::Graphics::Check; +using AgcDriver::Graphics::Context; +using AgcDriver::Graphics::Require; + +constexpr std::uint32_t Stale = 0xababababu; +constexpr std::uint32_t Fresh = 0x11111111u; +constexpr VkMemoryPropertyFlags HostMemory = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT; +constexpr std::size_t RequestedBytes = 100; +constexpr std::uint32_t PastEnd = 28; + +PFN_vkGetDeviceProcAddr RealDeviceProc = nullptr; +PFN_vkCreateBuffer RealCreateBuffer = nullptr; + +std::vector& CreatedSizes() { + static std::vector sizes; + return sizes; +} + +VKAPI_ATTR VkResult VKAPI_CALL CountedCreateBuffer(VkDevice device, const VkBufferCreateInfo* info, const VkAllocationCallbacks* allocator, VkBuffer* buffer) { + CreatedSizes().push_back(info->size); + return RealCreateBuffer(device, info, allocator, buffer); +} + +VKAPI_ATTR PFN_vkVoidFunction VKAPI_CALL CountingDeviceProc(VkDevice device, const char* name) { + if (std::strcmp(name, "vkCreateBuffer") == 0) return reinterpret_cast(&CountedCreateBuffer); + return RealDeviceProc(device, name); +} + +class Device { +public: + Device() { +#ifdef _WIN32 + library = SDL_LoadObject("vulkan-1.dll"); +#else + library = SDL_LoadObject("libvulkan.so.1"); +#endif + Require(library != nullptr, "cannot load Vulkan"); + try { + instanceProc = reinterpret_cast(SDL_LoadFunction(library, "vkGetInstanceProcAddr")); + Require(instanceProc != nullptr, "missing Vulkan instance resolver"); + VkApplicationInfo application{VK_STRUCTURE_TYPE_APPLICATION_INFO}; + application.apiVersion = VK_API_VERSION_1_1; + VkInstanceCreateInfo info{VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO}; + info.pApplicationInfo = &application; + Check(function("vkCreateInstance")(&info, nullptr, &instance), "vkCreateInstance"); + std::uint32_t count = 0; + const auto enumerate = function("vkEnumeratePhysicalDevices"); + Check(enumerate(instance, &count, nullptr), "vkEnumeratePhysicalDevices"); + Require(count != 0, "no Vulkan device"); + std::vector devices(count); + Check(enumerate(instance, &count, devices.data()), "vkEnumeratePhysicalDevices"); + const auto properties = function("vkGetPhysicalDeviceProperties"); + context.physical = devices.front(); + for (const auto candidate : devices) { + VkPhysicalDeviceProperties described{}; + properties(candidate, &described); + if (described.deviceType == VK_PHYSICAL_DEVICE_TYPE_DISCRETE_GPU) { + context.physical = candidate; + break; + } + } + VkPhysicalDeviceProperties selected{}; + properties(context.physical, &selected); + context.limits = selected.limits; + std::printf("device %s\n", selected.deviceName); + VkPhysicalDeviceFeatures supported{}; + function("vkGetPhysicalDeviceFeatures")(context.physical, &supported); + Require(supported.robustBufferAccess == VK_TRUE, "the device lacks robustBufferAccess"); + const auto queues = function("vkGetPhysicalDeviceQueueFamilyProperties"); + queues(context.physical, &count, nullptr); + std::vector families(count); + queues(context.physical, &count, families.data()); + std::uint32_t family = 0; + while (family < count && (families[family].queueFlags & VK_QUEUE_COMPUTE_BIT) == 0) ++family; + Require(family < count, "no Vulkan compute queue"); + const float priority = 1; + VkDeviceQueueCreateInfo queue{VK_STRUCTURE_TYPE_DEVICE_QUEUE_CREATE_INFO}; + queue.queueFamilyIndex = family; + queue.queueCount = 1; + queue.pQueuePriorities = &priority; + VkPhysicalDeviceFeatures features{}; + features.robustBufferAccess = VK_TRUE; + VkDeviceCreateInfo device{VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO}; + device.queueCreateInfoCount = 1; + device.pQueueCreateInfos = &queue; + device.pEnabledFeatures = &features; + Check(function("vkCreateDevice")(context.physical, &device, nullptr, &context.device), "vkCreateDevice"); + RealDeviceProc = function("vkGetDeviceProcAddr"); + RealCreateBuffer = reinterpret_cast(RealDeviceProc(context.device, "vkCreateBuffer")); + Require(RealCreateBuffer != nullptr, "missing vkCreateBuffer"); + context.deviceProc = &CountingDeviceProc; + function("vkGetPhysicalDeviceMemoryProperties")(context.physical, &context.memory); + context.formatProperties = function("vkGetPhysicalDeviceFormatProperties"); + context.Function("vkGetDeviceQueue")(context.device, family, 0, &context.queue); + VkCommandPoolCreateInfo pool{VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO}; + pool.queueFamilyIndex = family; + Check(context.Function("vkCreateCommandPool")(context.device, &pool, nullptr, &context.pool), "vkCreateCommandPool"); + } catch (...) { + release(); + throw; + } + } + + ~Device() { release(); } + Device(const Device&) = delete; + Device& operator=(const Device&) = delete; + const Context& GetContext() const { return context; } + +private: + template + TFunction function(const char* name) const { + const auto result = reinterpret_cast(instanceProc(instance, name)); + Require(result != nullptr, name); + return result; + } + + void release() noexcept { + if (context.pool != VK_NULL_HANDLE) context.Function("vkDestroyCommandPool")(context.device, context.pool, nullptr); + context.bufferPool.reset(); + if (context.device != VK_NULL_HANDLE) function("vkDestroyDevice")(context.device, nullptr); + if (instance != VK_NULL_HANDLE) function("vkDestroyInstance")(instance, nullptr); + if (library != nullptr) SDL_UnloadObject(library); + } + + void* library = nullptr; + PFN_vkGetInstanceProcAddr instanceProc = nullptr; + VkInstance instance = VK_NULL_HANDLE; + Context context{}; +}; + +void Emit(std::vector& words, std::uint32_t opcode, std::initializer_list operands) { + words.push_back(static_cast((operands.size() + 1u) << 16u) | opcode); + words.insert(words.end(), operands); +} + +std::vector CopyPastEndModule() { + enum : std::uint32_t { Void = 1, FunctionType, Uint, Words, Block, BlockPointer, Source, Destination, UintPointer, Zero, Index, Main, Label, SourcePointer, Value, DestinationPointer, Bound }; + constexpr std::uint32_t storageBuffer = 12; + std::vector words{0x07230203u, 0x00010300u, 0u, Bound, 0u}; + Emit(words, 17, {1}); + Emit(words, 14, {0, 1}); + Emit(words, 15, {5, Main, 0x6e69616du, 0u}); + Emit(words, 16, {Main, 17, 1, 1, 1}); + Emit(words, 71, {Words, 6, 4}); + Emit(words, 71, {Block, 2}); + Emit(words, 72, {Block, 0, 35, 0}); + Emit(words, 71, {Source, 34, 0}); + Emit(words, 71, {Source, 33, 0}); + Emit(words, 71, {Destination, 34, 0}); + Emit(words, 71, {Destination, 33, 1}); + Emit(words, 19, {Void}); + Emit(words, 33, {FunctionType, Void}); + Emit(words, 21, {Uint, 32, 0}); + Emit(words, 29, {Words, Uint}); + Emit(words, 30, {Block, Words}); + Emit(words, 32, {BlockPointer, storageBuffer, Block}); + Emit(words, 59, {BlockPointer, Source, storageBuffer}); + Emit(words, 59, {BlockPointer, Destination, storageBuffer}); + Emit(words, 32, {UintPointer, storageBuffer, Uint}); + Emit(words, 43, {Uint, Zero, 0}); + Emit(words, 43, {Uint, Index, PastEnd}); + Emit(words, 54, {Void, Main, 0, FunctionType}); + Emit(words, 248, {Label}); + Emit(words, 65, {UintPointer, SourcePointer, Source, Zero, Index}); + Emit(words, 61, {Uint, Value, SourcePointer}); + Emit(words, 65, {UintPointer, DestinationPointer, Destination, Zero, Zero}); + Emit(words, 62, {DestinationPointer, Value}); + Emit(words, 253, {}); + Emit(words, 56, {}); + return words; +} + +std::uint32_t ReadPastEnd(const Context& context, AgcDriver::Graphics::Buffer& source, AgcDriver::Graphics::Buffer& destination) { + const std::array bindings{{{0, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT, nullptr}, {1, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT, nullptr}}}; + VkDescriptorSetLayoutCreateInfo setLayoutInfo{VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO}; + setLayoutInfo.bindingCount = static_cast(bindings.size()); + setLayoutInfo.pBindings = bindings.data(); + VkDescriptorSetLayout setLayout = VK_NULL_HANDLE; + Check(context.Function("vkCreateDescriptorSetLayout")(context.device, &setLayoutInfo, nullptr, &setLayout), "vkCreateDescriptorSetLayout"); + const VkDescriptorPoolSize poolSize{VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 2}; + VkDescriptorPoolCreateInfo poolInfo{VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO}; + poolInfo.maxSets = 1; + poolInfo.poolSizeCount = 1; + poolInfo.pPoolSizes = &poolSize; + VkDescriptorPool descriptorPool = VK_NULL_HANDLE; + Check(context.Function("vkCreateDescriptorPool")(context.device, &poolInfo, nullptr, &descriptorPool), "vkCreateDescriptorPool"); + VkDescriptorSetAllocateInfo allocateInfo{VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO}; + allocateInfo.descriptorPool = descriptorPool; + allocateInfo.descriptorSetCount = 1; + allocateInfo.pSetLayouts = &setLayout; + VkDescriptorSet set = VK_NULL_HANDLE; + Check(context.Function("vkAllocateDescriptorSets")(context.device, &allocateInfo, &set), "vkAllocateDescriptorSets"); + const std::array infos{{{source.Handle(), 0, VK_WHOLE_SIZE}, {destination.Handle(), 0, VK_WHOLE_SIZE}}}; + std::array writes{}; + for (std::uint32_t binding = 0; binding < writes.size(); ++binding) { + writes[binding] = {VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET}; + writes[binding].dstSet = set; + writes[binding].dstBinding = binding; + writes[binding].descriptorCount = 1; + writes[binding].descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER; + writes[binding].pBufferInfo = &infos[binding]; + } + context.Function("vkUpdateDescriptorSets")(context.device, static_cast(writes.size()), writes.data(), 0, nullptr); + + const auto module = CopyPastEndModule(); + VkShaderModuleCreateInfo moduleInfo{VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO}; + moduleInfo.codeSize = module.size() * sizeof(std::uint32_t); + moduleInfo.pCode = module.data(); + VkShaderModule shader = VK_NULL_HANDLE; + Check(context.Function("vkCreateShaderModule")(context.device, &moduleInfo, nullptr, &shader), "vkCreateShaderModule"); + VkPipelineLayoutCreateInfo layoutInfo{VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO}; + layoutInfo.setLayoutCount = 1; + layoutInfo.pSetLayouts = &setLayout; + VkPipelineLayout pipelineLayout = VK_NULL_HANDLE; + Check(context.Function("vkCreatePipelineLayout")(context.device, &layoutInfo, nullptr, &pipelineLayout), "vkCreatePipelineLayout"); + VkComputePipelineCreateInfo pipelineInfo{VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO}; + pipelineInfo.stage = {VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO}; + pipelineInfo.stage.stage = VK_SHADER_STAGE_COMPUTE_BIT; + pipelineInfo.stage.module = shader; + pipelineInfo.stage.pName = "main"; + pipelineInfo.layout = pipelineLayout; + VkPipeline pipeline = VK_NULL_HANDLE; + Check(context.Function("vkCreateComputePipelines")(context.device, VK_NULL_HANDLE, 1, &pipelineInfo, nullptr, &pipeline), "vkCreateComputePipelines"); + { + std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex()); + AgcDriver::Graphics::CommandBatch batch(context); + context.Function("vkCmdBindPipeline")(batch.Handle(), VK_PIPELINE_BIND_POINT_COMPUTE, pipeline); + context.Function("vkCmdBindDescriptorSets")(batch.Handle(), VK_PIPELINE_BIND_POINT_COMPUTE, pipelineLayout, 0, 1, &set, 0, nullptr); + context.Function("vkCmdDispatch")(batch.Handle(), 1, 1, 1); + AgcDriver::Graphics::RecordMemoryBarrier(context, batch.Handle(), VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, VK_PIPELINE_STAGE_HOST_BIT, VK_ACCESS_SHADER_WRITE_BIT, VK_ACCESS_HOST_READ_BIT); + batch.SubmitAndWait(); + } + context.Function("vkDestroyPipeline")(context.device, pipeline, nullptr); + context.Function("vkDestroyPipelineLayout")(context.device, pipelineLayout, nullptr); + context.Function("vkDestroyShaderModule")(context.device, shader, nullptr); + context.Function("vkDestroyDescriptorPool")(context.device, descriptorPool, nullptr); + context.Function("vkDestroyDescriptorSetLayout")(context.device, setLayout, nullptr); + destination.Invalidate(); + std::uint32_t value = 0; + std::memcpy(&value, destination.Bytes().data(), sizeof(value)); + return value; +} + +void RobustReadPastEnd(const Context& context) { + constexpr VkBufferUsageFlags usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT; + const auto classBytes = BufferPool::Capacity(RequestedBytes, HostMemory); + Require(classBytes > (PastEnd + 1) * sizeof(std::uint32_t), "the requested buffer's size class does not reach the word read past its end"); + { + AgcDriver::Graphics::Buffer earlier(context, classBytes, usage); + const auto bytes = earlier.Bytes(); + for (std::size_t offset = 0; offset + sizeof(Stale) <= bytes.size(); offset += sizeof(Stale)) std::memcpy(bytes.data() + offset, &Stale, sizeof(Stale)); + } + AgcDriver::Graphics::Buffer source(context, RequestedBytes, usage); + const auto bytes = source.Bytes(); + Require(bytes.size() == RequestedBytes, "the pooled buffer does not span the requested bytes"); + for (std::size_t offset = 0; offset + sizeof(Fresh) <= bytes.size(); offset += sizeof(Fresh)) std::memcpy(bytes.data() + offset, &Fresh, sizeof(Fresh)); + AgcDriver::Graphics::Buffer destination(context, 4, VK_BUFFER_USAGE_STORAGE_BUFFER_BIT); + std::memset(destination.Bytes().data(), 0, destination.Bytes().size()); + const auto value = ReadPastEnd(context, source, destination); + char text[160]; + std::snprintf(text, sizeof(text), "a robust read of word %u of a %zu-byte pooled buffer returned 0x%08x, the bytes a %zu-byte earlier user left in its memory", PastEnd, RequestedBytes, value, classBytes); + Require(value != Stale, text); + Require(value == 0 || value == Fresh, "a robust read past the buffer returned neither zero nor a value inside it"); +} + +void ExpectCreated(std::initializer_list sizes, const std::string& what) { + const std::vector expected(sizes); + std::string got; + for (const auto size : CreatedSizes()) got += " " + std::to_string(size); + Require(CreatedSizes() == expected, what + ": VkBuffer sizes created:" + (got.empty() ? std::string(" none") : got)); + CreatedSizes().clear(); +} + +void ExactSizes(const Context& context) { + constexpr VkBufferUsageFlags usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT; + const auto hostClass = BufferPool::Capacity(100, HostMemory); + const auto deviceClass = BufferPool::Capacity(3000, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT); + Require(BufferPool::Capacity(80, HostMemory) == hostClass && hostClass != 80, "80 and 100 bytes are not one host size class"); + Require(BufferPool::Capacity(2500, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT) == deviceClass && deviceClass != 2500, "2500 and 3000 bytes are not one device size class"); + CreatedSizes().clear(); + { + AgcDriver::Graphics::Buffer fresh(context, 100, usage); + ExpectCreated({100}, "a fresh 100-byte buffer"); + } + { + AgcDriver::Graphics::Buffer pooled(context, 80, usage); + ExpectCreated({80}, "an 80-byte buffer taken from the " + std::to_string(hostClass) + "-byte class"); + } + { + AgcDriver::Graphics::Buffer again(context, 80, usage); + ExpectCreated({}, "an 80-byte buffer reusing an 80-byte one"); + } + { + AgcDriver::Graphics::DeviceBuffer fresh(context, 3000, usage); + ExpectCreated({3000}, "a fresh 3000-byte device buffer"); + } + { + AgcDriver::Graphics::DeviceBuffer pooled(context, 2500, usage); + ExpectCreated({2500}, "a 2500-byte device buffer taken from the " + std::to_string(deviceClass) + "-byte class"); + } +} + +} + +int main() { + try { + std::unique_ptr device; + try { + device = std::make_unique(); + } catch (const std::exception& error) { + if (std::getenv("ANYPS5_REQUIRE_VULKAN") != nullptr) throw; + std::printf("skipped, no usable Vulkan device: %s\n", error.what()); + return VulkanTestSkipped; + } + RobustReadPastEnd(device->GetContext()); + ExactSizes(device->GetContext()); + std::puts("pooled buffer range tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} From a79bf5a7276ddb7213494e9a66e288be3bcf9d6a Mon Sep 17 00:00:00 2001 From: tototomate123 Date: Thu, 8 Oct 2026 16:47:53 +0200 Subject: [PATCH 045/460] fix(libkernel): return guest errors from chmod calls --- core/libs/prx/libkernel/File/src/Stdio.cpp | 15 +++++++-------- core/libs/tests/GuestFilesystem.cpp | 5 +++++ 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index 711b9f388..9a676157a 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -260,13 +260,9 @@ static int PathError(const char* path) { extern "C" { int APS5_VABI chmod_nid_postfix(const char* path, int mode) { - if (path == nullptr) { - APS5_INVALID_ARG_EX; - } - auto native = ResolvePath_nid_no_patch(path); - if (NativeChmod(native, mode) != 0) { - throw std::runtime_error(std::string(__func__) + ": chmod failed for " + native.string() + ", errno=" + std::to_string(errno)); - } + if (const int error = PathError(path)) return PosixFailure(error); + const auto native = ResolvePath_nid_no_patch(path); + if (NativeChmod(native, mode & 07777) != 0) return PosixResult(SceErrorFromErrno(errno)); return 0; } @@ -733,7 +729,10 @@ int APS5_VABI rmdir_nid_postfix(const char* path) { extern "C" { int APS5_VABI sceKernelChmod_nid_postfix(const char* path, std::uint16_t mode) { - return chmod_nid_postfix(path, mode); + if (!path) return SceErrorFromErrno(GUEST_EFAULT); + const auto native = ResolvePath_nid_no_patch(path); + if (NativeChmod(native, mode & 07777) != 0) return SceErrorFromErrno(errno); + return 0; } int APS5_VABI sceKernelFchmod(int d, std::uint16_t mode) { diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index bd0f598b7..fb1cec538 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -17,6 +17,7 @@ int APS5_VABI rename_nid_postfix(const char*, const char*); int APS5_VABI sceKernelChmod_nid_postfix(const char*, unsigned short); int APS5_VABI sceKernelFchmod(int, unsigned short); int APS5_VABI fchmod_nid_postfix(int, int); +int APS5_VABI chmod_nid_postfix(const char*, int); int APS5_VABI futimes_nid_postfix(int, const KernelTimeval*); int APS5_VABI socket_nid_postfix(int, int, int); int APS5_VABI sceKernelFsync(int); @@ -54,6 +55,10 @@ static void Check(bool value, int line) { } #define Require(value) Check((value), __LINE__) int main() { + Require(chmod_nid_postfix(nullptr, 0600) == -1 && *__error_nid_postfix() == 14); + Require(sceKernelChmod_nid_postfix(nullptr, 0600) == static_cast(0x8002000eu)); + Require(chmod_nid_postfix("anyps5_missing_chmod_path", 0600) == -1 && *__error_nid_postfix() == 2); + Require(sceKernelChmod_nid_postfix("anyps5_missing_chmod_path", 0600) == static_cast(0x80020002u)); Require(sceKernelDebugOutText(-1, "text") == static_cast(0x80020016u)); std::uint64_t throttling[4] = {1, 2, 3, 4}; Require(sceKernelWriteThrottlingStatus(throttling) == 0); From 7ea423f78bd5f8b72eca4dd7c30a26ea1ffe993f Mon Sep 17 00:00:00 2001 From: tototomate123 Date: Thu, 8 Oct 2026 16:47:53 +0200 Subject: [PATCH 046/460] fix(relinker): map executable section addresses through load segments --- .../src/general/SectionHeaderTableBuilder.cpp | 25 ++++++++++++++++--- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/core/relinker/elfpatcher/src/general/SectionHeaderTableBuilder.cpp b/core/relinker/elfpatcher/src/general/SectionHeaderTableBuilder.cpp index c403a058e..e9ee612df 100644 --- a/core/relinker/elfpatcher/src/general/SectionHeaderTableBuilder.cpp +++ b/core/relinker/elfpatcher/src/general/SectionHeaderTableBuilder.cpp @@ -1,5 +1,7 @@ #include #include +#include +#include namespace Elfpatcher { @@ -38,6 +40,21 @@ void SectionHeaderTableBuilder::WriteTable( std::vector& buf, const SectionHeaderTableRequest& request ) const { + const auto phOffset = Io::ReadU64(buf, kEhdrPhOffOffset); + const auto phCount = Io::ReadU16(buf, kEhdrPhNumOffset); + const auto phStride = Io::ReadU16(buf, kEhdrPhEntSizeOffset); + const auto mappedAddress = [&](std::uint64_t offset) { + for (std::uint16_t index = 0; index < phCount; ++index) { + const auto header = phOffset + index * phStride; + if (Io::ReadU32(buf, header) != PT_LOAD) continue; + const auto start = Io::ReadU64(buf, header + 8); + const auto size = Io::ReadU64(buf, header + 32); + if (offset >= start && offset - start < size) + return Io::ReadU64(buf, header + 16) + offset - start; + } + throw std::runtime_error("Allocated output section is outside load segments"); + }; + const auto shStrTabOff = static_cast(buf.size()); constexpr std::uint64_t shStrTabSize = sizeof(kShStrTabBlob); for (const char i : kShStrTabBlob) @@ -48,10 +65,10 @@ void SectionHeaderTableBuilder::WriteTable( _writeSectionHeader(buf, shOff + kShdrEntrySize * 0, 0, SHT_NULL, 0, 0, 0, 0, 0, 0, 0, 0); _writeSectionHeader(buf, shOff + kShdrEntrySize * 1, kShStrTabNameOffset, SHT_STRTAB, 0, shStrTabOff, shStrTabOff, shStrTabSize, 0, 0, kNoSpecialAlignment, 0); - _writeSectionHeader(buf, shOff + kShdrEntrySize * 2, kDynStrNameOffset, SHT_STRTAB, SHF_ALLOC, request.DynStrOffset, request.DynStrOffset, request.DynStrSize, 0, 0, kNoSpecialAlignment, 0); - _writeSectionHeader(buf, shOff + kShdrEntrySize * 3, kDynSymNameOffset, SHT_DYNSYM, SHF_ALLOC, request.DynSymOffset, request.DynSymOffset, request.DynSymSize, kDynSymLinkToStrTab, kDynSymInfoFirstGlobal, kDynSymAlign, kSymEntrySize); - _writeSectionHeader(buf, shOff + kShdrEntrySize * 4, kDynamicNameOffset, SHT_DYNAMIC, SHF_ALLOC | SHF_WRITE, request.DynamicSegmentOffset, request.DynamicSegmentOffset, request.DynamicSegmentSize, kDynamicLinkToStrTab, 0, kDynSymAlign, kDynEntrySize); - _writeSectionHeader(buf, shOff + kShdrEntrySize * 5, kTextNameOffset, SHT_PROGBITS, SHF_ALLOC | SHF_EXECINSTR, request.StubOffset, request.StubOffset, request.StubSize, 0, 0, kNoSpecialAlignment, 0); + _writeSectionHeader(buf, shOff + kShdrEntrySize * 2, kDynStrNameOffset, SHT_STRTAB, SHF_ALLOC, mappedAddress(request.DynStrOffset), request.DynStrOffset, request.DynStrSize, 0, 0, kNoSpecialAlignment, 0); + _writeSectionHeader(buf, shOff + kShdrEntrySize * 3, kDynSymNameOffset, SHT_DYNSYM, SHF_ALLOC, mappedAddress(request.DynSymOffset), request.DynSymOffset, request.DynSymSize, kDynSymLinkToStrTab, kDynSymInfoFirstGlobal, kDynSymAlign, kSymEntrySize); + _writeSectionHeader(buf, shOff + kShdrEntrySize * 4, kDynamicNameOffset, SHT_DYNAMIC, SHF_ALLOC | SHF_WRITE, mappedAddress(request.DynamicSegmentOffset), request.DynamicSegmentOffset, request.DynamicSegmentSize, kDynamicLinkToStrTab, 0, kDynSymAlign, kDynEntrySize); + _writeSectionHeader(buf, shOff + kShdrEntrySize * 5, kTextNameOffset, SHT_PROGBITS, SHF_ALLOC | SHF_EXECINSTR, mappedAddress(request.StubOffset), request.StubOffset, request.StubSize, 0, 0, kNoSpecialAlignment, 0); _byteWriter->WriteU64(buf, kEhdrShOffOffset, shOff); _byteWriter->WriteU16(buf, kEhdrShEntSizeOffset, static_cast(kShdrEntrySize)); From 9107be460ff426c47fa7beb69f021dc02b7a5c7e Mon Sep 17 00:00:00 2001 From: tototomate123 Date: Thu, 8 Oct 2026 16:47:53 +0200 Subject: [PATCH 047/460] fix(libkernel): implement kernel futimes with POSIX error translation --- core/libs/prx/libkernel/File/src/Stdio.cpp | 12 ++++++++---- core/libs/tests/GuestFilesystem.cpp | 5 ++++- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index 9a676157a..070d23d03 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -774,17 +774,21 @@ int APS5_VABI utimes_nid_postfix(const char* path, const KernelTimeval* times) { return PosixResult(sceKernelUtimes_nid_postfix(path, times)); } -int APS5_VABI futimes_nid_postfix(int d, const KernelTimeval* times) { - if (d >= GuestSockets::FirstDescriptor) return PosixFailure(GuestSockets::IsOpen(d) ? GUEST_EINVAL : GUEST_EBADF); +int APS5_VABI sceKernelFutimes(int d, const KernelTimeval* times) { + if (d >= GuestSockets::FirstDescriptor) return SceErrorFromErrno(GuestSockets::IsOpen(d) ? GUEST_EINVAL : GUEST_EBADF); if (times != nullptr) { for (int i = 0; i < 2; ++i) { - if (times[i].tv_usec < 0 || times[i].tv_usec >= 1000000) return PosixFailure(GUEST_EINVAL); + if (times[i].tv_usec < 0 || times[i].tv_usec >= 1000000) return SceErrorFromErrno(GUEST_EINVAL); } } - if (NativeFutimes(d, times) != 0) return PosixResult(SceErrorFromErrno(errno)); + if (NativeFutimes(d, times) != 0) return SceErrorFromErrno(errno); return 0; } +int APS5_VABI futimes_nid_postfix(int d, const KernelTimeval* times) { + return PosixResult(sceKernelFutimes(d, times)); +} + int APS5_VABI fsync_nid_postfix(int fd) { if (sceKernelFsync(fd) != 0) return PosixFailure(errno); return 0; diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index fb1cec538..018c2c10d 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -19,6 +19,7 @@ int APS5_VABI sceKernelFchmod(int, unsigned short); int APS5_VABI fchmod_nid_postfix(int, int); int APS5_VABI chmod_nid_postfix(const char*, int); int APS5_VABI futimes_nid_postfix(int, const KernelTimeval*); +int APS5_VABI sceKernelFutimes(int, const KernelTimeval*); int APS5_VABI socket_nid_postfix(int, int, int); int APS5_VABI sceKernelFsync(int); int APS5_VABI fdatasync_nid_postfix(int); @@ -153,11 +154,12 @@ int main() { Require(sceKernelFtruncate(descriptor, 3) == 0); FileStat times{}; const KernelTimeval past[2]{{1000000000, 0}, {1000000000, 500000}}; - Require(futimes_nid_postfix(descriptor, past) == 0); + Require(sceKernelFutimes(descriptor, past) == 0); Require(stat_nid_postfix(sized.string().c_str(), ×) == 0 && times.st_mtim.tv_sec == 1000000000); Require(futimes_nid_postfix(descriptor, nullptr) == 0); Require(stat_nid_postfix(sized.string().c_str(), ×) == 0 && times.st_mtim.tv_sec > 1000000000); const KernelTimeval overflow[2]{{0, 0}, {0, 1000000}}; + Require(sceKernelFutimes(descriptor, overflow) == static_cast(0x80020016u)); Require(futimes_nid_postfix(descriptor, overflow) == -1 && *__error_nid_postfix() == 22); const KernelTimeval negative[2]{{0, -1}, {0, 0}}; Require(futimes_nid_postfix(descriptor, negative) == -1 && *__error_nid_postfix() == 22); @@ -166,6 +168,7 @@ int main() { #ifndef _WIN32 Require(sceKernelFchmod(descriptor, 0600) == static_cast(0x80020009u)); Require(fchmod_nid_postfix(descriptor, 0600) == -1 && *__error_nid_postfix() == 9); + Require(sceKernelFutimes(descriptor, nullptr) == static_cast(0x80020009u)); Require(futimes_nid_postfix(descriptor, nullptr) == -1 && *__error_nid_postfix() == 9); Require(fdatasync_nid_postfix(descriptor) == -1 && *__error_nid_postfix() == 9); #endif From 0a229a43dca70d9d4b4b367ffeb79bad81ea00ba Mon Sep 17 00:00:00 2001 From: tototomate123 Date: Thu, 8 Oct 2026 16:48:25 +0200 Subject: [PATCH 048/460] fix(shader): evaluate signed minimum in runtime SRT expressions --- .../libSceAgcDriver/tests/ShaderMemory.cpp | 24 +++++++++++++++++++ .../src/SrtWalker/SrtEvaluator.cpp | 6 +++++ .../SrtWalker/SrtInstructionPredicates.cpp | 1 + 3 files changed, 31 insertions(+) diff --git a/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp b/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp index bc78344e5..8db39381c 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp @@ -7,6 +7,7 @@ #include "Optimization/ShaderStageInputInfo.hpp" #include "Optimization/SrtWalker/SrtDescriptorEvaluation.hpp" #include "Optimization/SrtWalker/SrtEvaluator.hpp" +#include "Optimization/SrtWalker/SrtInstructionPredicates.hpp" #include "Optimization/SrtWalker/SrtFlatSlotClasses.hpp" #include "SpirvBackend/SpirvAnalysis.hpp" #if ANYPS5_ENABLE_SPIRV_TOOLS @@ -360,6 +361,28 @@ void verifyRegisterSources() { require(!EquivalentValue(plan, &samplerRegister, &firstVector), "different register types were merged"); } +void verifySignedMinimum() { + using namespace ShaderRecompiler; + IrResourcePlan plan; + IrValue reg(IrOpcode::Void, IrType::ScalarReg, 0); + reg.SetRegister({RegisterBank::Scalar, 0}); + IrValue input(IrOpcode::GetUserData, IrType::U32, 1); + input.AddArgument(®); + IrValue limit(IrOpcode::Void, IrType::U32, 2); + limit.SetImmediateU32(8); + IrValue minimum(IrOpcode::SMin32, IrType::U32, 3); + minimum.AddArgument(&input); + minimum.AddArgument(&limit); + require(Detail::IsRuntimeUniformOp(IrOpcode::SMin32), "signed minimum must be a runtime uniform operation"); + for (const auto& [value, expected] : std::array, 6>{{{0, 0}, {4, 4}, {20, 8}, {0x7fffffffu, 8}, {0xffffffffu, 0xffffffffu}, {0x80000000u, 0x80000000u}}}) { + std::array data{value}; + SrtRuntime runtime{data}; + Detail::Evaluator evaluator(plan, runtime); + std::uint32_t result = 0; + require(evaluator.Evaluate(&minimum, result) && result == expected, "SRT signed minimum used unsigned ordering or failed evaluation"); + } +} + void verifyEvaluatedValues() { using namespace ShaderRecompiler; std::vector> values; @@ -2568,6 +2591,7 @@ int main(int argc, char** argv) { verifyEvaluatedValues(); verifyFrontendPair(); verifySignedSrtComparison(); + verifySignedMinimum(); verifyPureFlatSlots(); verifyBindlessTable(); verifyDescriptorPhis(); diff --git a/core/shader/recompiler/Optimization/src/SrtWalker/SrtEvaluator.cpp b/core/shader/recompiler/Optimization/src/SrtWalker/SrtEvaluator.cpp index 76c6ceb53..d658afb33 100644 --- a/core/shader/recompiler/Optimization/src/SrtWalker/SrtEvaluator.cpp +++ b/core/shader/recompiler/Optimization/src/SrtWalker/SrtEvaluator.cpp @@ -256,6 +256,12 @@ bool Evaluator::EvaluateInst(IrValue& inst, std::uint64_t& result) { return true; } return false; + case IrOpcode::SMin32: + if (binary()) { + result = static_cast(std::min(std::bit_cast(static_cast(a)), std::bit_cast(static_cast(b)))); + return true; + } + return false; case IrOpcode::UMin32: if (binary()) { result = std::min(static_cast(a), static_cast(b)); diff --git a/core/shader/recompiler/Optimization/src/SrtWalker/SrtInstructionPredicates.cpp b/core/shader/recompiler/Optimization/src/SrtWalker/SrtInstructionPredicates.cpp index 274524be7..0f3a3a6af 100644 --- a/core/shader/recompiler/Optimization/src/SrtWalker/SrtInstructionPredicates.cpp +++ b/core/shader/recompiler/Optimization/src/SrtWalker/SrtInstructionPredicates.cpp @@ -70,6 +70,7 @@ bool IsRuntimeUniformOp(IrOpcode opcode) { case IrOpcode::ISub64: case IrOpcode::IMul32: case IrOpcode::IMul64: + case IrOpcode::SMin32: case IrOpcode::UMin32: case IrOpcode::ShiftLeftLogical32: case IrOpcode::ShiftLeftLogical64: From ded8b4df2373bb5cc66c97fd4a22eb224c5a80d2 Mon Sep 17 00:00:00 2001 From: tototomate123 Date: Sat, 10 Oct 2026 16:57:31 +0200 Subject: [PATCH 049/460] fix(relinker): preserve executable exports and dynamic lookup tags --- core/relinker/CMakeLists.txt | 2 + .../elfpatcher/general/ElfConstants.hpp | 2 + .../elfpatcher/src/linux/LinuxElfPatcher.cpp | 12 +++++ .../src/pipeline/RelinkerPipeline.cpp | 22 ++++++++ .../tests/test_linux_executable_exports.py | 53 +++++++++++++++++++ 5 files changed, 91 insertions(+) create mode 100644 core/relinker/relinker/tests/test_linux_executable_exports.py diff --git a/core/relinker/CMakeLists.txt b/core/relinker/CMakeLists.txt index 288dddef6..5514279eb 100644 --- a/core/relinker/CMakeLists.txt +++ b/core/relinker/CMakeLists.txt @@ -243,6 +243,8 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_guest_runpath.py $) add_test(NAME executable_segments COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_executable_segments.py $) + add_test(NAME linux_executable_exports COMMAND ${Python3_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_linux_executable_exports.py $) add_test(NAME linux_entry_argv COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_linux_entry_argv.py $) add_test(NAME linux_program_header_order COMMAND ${Python3_EXECUTABLE} diff --git a/core/relinker/elfpatcher/include/elfpatcher/general/ElfConstants.hpp b/core/relinker/elfpatcher/include/elfpatcher/general/ElfConstants.hpp index 60da5e79f..615c7c774 100644 --- a/core/relinker/elfpatcher/include/elfpatcher/general/ElfConstants.hpp +++ b/core/relinker/elfpatcher/include/elfpatcher/general/ElfConstants.hpp @@ -76,10 +76,12 @@ inline constexpr std::uint32_t PT_OS_RELRO = 0x61000010; inline constexpr std::uint32_t PT_LOOS = 0x61000000; inline constexpr std::uint32_t PT_HIOS = 0x6fffffff; +inline constexpr std::int64_t DT_DEBUG = 21; inline constexpr std::int64_t DT_NULL = 0; inline constexpr std::int64_t DT_NEEDED = 1; inline constexpr std::int64_t DT_PLTRELSZ = 2; inline constexpr std::int64_t DT_PLTGOT = 3; +inline constexpr std::int64_t DT_HASH = 4; inline constexpr std::int64_t DT_STRTAB = 5; inline constexpr std::int64_t DT_SYMTAB = 6; inline constexpr std::int64_t DT_RELA = 7; diff --git a/core/relinker/elfpatcher/src/linux/LinuxElfPatcher.cpp b/core/relinker/elfpatcher/src/linux/LinuxElfPatcher.cpp index 24746a4a1..1db4b2741 100644 --- a/core/relinker/elfpatcher/src/linux/LinuxElfPatcher.cpp +++ b/core/relinker/elfpatcher/src/linux/LinuxElfPatcher.cpp @@ -4,6 +4,7 @@ #include #include #include +#include #include #include #include @@ -112,6 +113,15 @@ std::vector LinuxElfPatcher::Patch( buf.push_back(b); alignBuf(buf, kDynSymAlignment); + const auto hashOff = static_cast(buf.size()); + const auto symbolCount = static_cast(dynSection.DynSymData.size() / kSymEntrySize); + Io::AppendU32(buf, 1); + Io::AppendU32(buf, symbolCount); + Io::AppendU32(buf, symbolCount > 1 ? 1 : 0); + for (std::uint32_t index = 0; index < symbolCount; ++index) + Io::AppendU32(buf, index != 0 && index + 1 < symbolCount ? index + 1 : 0); + alignBuf(buf, kDynSymAlignment); + const auto relaOff = static_cast(buf.size()); for (std::uint8_t b : dynSection.RelaData) buf.push_back(b); @@ -138,6 +148,8 @@ std::vector LinuxElfPatcher::Patch( _appendDynEntry(dynSegBuf, DT_STRSZ, dynStrSize); _appendDynEntry(dynSegBuf, DT_SYMTAB, vaddrOfExtraBlockOffset(dynSymOff)); _appendDynEntry(dynSegBuf, DT_SYMENT, kSymEntrySize); + _appendDynEntry(dynSegBuf, DT_HASH, vaddrOfExtraBlockOffset(hashOff)); + _appendDynEntry(dynSegBuf, DT_DEBUG, 0); if (!dynSection.RelaData.empty()) { _appendDynEntry(dynSegBuf, DT_RELA, vaddrOfExtraBlockOffset(relaOff)); _appendDynEntry(dynSegBuf, DT_RELASZ, dynSection.RelaData.size()); diff --git a/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp b/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp index d631eea28..162d25956 100644 --- a/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp +++ b/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp @@ -294,6 +294,28 @@ RelinkResult RelinkerPipeline::Relink(const std::vector& sourceElf } auto dynSection = _dynamicSectionBuilder->BuildDynamicSection(dynamicRefs, neededLibraries, dynJmpRelOffset, pltCount); + if (hasTag(DT_OS_SYMTABSZ)) { + const auto symbolBytes = getTagValue(DT_OS_SYMTABSZ); + if (symbolBytes % symEntSize != 0 || dynSymTabOffset > raw.size() || symbolBytes > raw.size() - dynSymTabOffset) + throw RelinkerException("Invalid executable dynamic symbol table size"); + for (ByteCount off = symEntSize; off < symbolBytes; off += symEntSize) { + const auto* symbol = raw.data() + dynSymTabOffset + off; + std::uint16_t section = 0; + std::memcpy(§ion, symbol + 6, sizeof(section)); + if (section == 0 || (symbol[4] >> 4) == 0 || (symbol[5] & 3) == 1 || (symbol[5] & 3) == 2) continue; + std::uint32_t sourceName = 0; + std::memcpy(&sourceName, symbol, sizeof(sourceName)); + auto name = readCStr(sourceName); + name.resize(name.find('#') == std::string::npos ? name.size() : name.find('#')); + const auto nameOffset = static_cast(dynSection.DynStrData.size()); + dynSection.DynStrData.insert(dynSection.DynStrData.end(), name.begin(), name.end()); + dynSection.DynStrData.push_back(0); + const auto position = dynSection.DynSymData.size(); + dynSection.DynSymData.insert(dynSection.DynSymData.end(), symbol, symbol + symEntSize); + std::memcpy(dynSection.DynSymData.data() + position, &nameOffset, sizeof(nameOffset)); + } + } + auto appendRela = [&](std::vector& buf, std::uint64_t offset, std::uint64_t info, std::int64_t addend) { std::size_t pos = buf.size(); buf.resize(pos + 24); diff --git a/core/relinker/relinker/tests/test_linux_executable_exports.py b/core/relinker/relinker/tests/test_linux_executable_exports.py new file mode 100644 index 000000000..fc878aefc --- /dev/null +++ b/core/relinker/relinker/tests/test_linux_executable_exports.py @@ -0,0 +1,53 @@ +import os +from pathlib import Path +import platform +import struct +import subprocess +import sys +import tempfile +from test_linux_load_alignment import fixture + + +def main(): + if sys.platform != "linux" or platform.machine() != "x86_64": + return + relinker = Path(sys.argv[1]).resolve() + with tempfile.TemporaryDirectory(prefix="anyps5-main-export-") as directory: + work = Path(directory) + probe = work / "probe.c" + probe.write_text( + "#include \n" + "#include \n" + "void check(void) {\n" + " int (*function)(void) = dlsym(RTLD_DEFAULT, \"FixtureExport\");\n" + " _exit(function && function() == 42 ? 42 : 1);\n" + "}\n") + library = work / "probe.so" + subprocess.run([os.environ.get("CC", "cc"), "-shared", "-fPIC", str(probe), "-o", str(library), "-ldl", "-Wl,-init,check"], check=True) + image = fixture() + strings = b"\0FixtureExport#A#B\0".ljust(32, b"\0") + str(library).encode() + b"\0" + image[0x4600:0x4600 + len(strings)] = strings + image[0x4210:0x4216] = bytes.fromhex("b82a000000c3") + struct.pack_into(" Date: Sat, 10 Oct 2026 16:46:16 +0200 Subject: [PATCH 050/460] perf(agc): take a storage image's generation baseline only when its guest bytes are settled An upload of a storage image up to 1 MiB reads the surface's guest bytes into a baseline (#1599), which a later write-back compares against for blocks the CPU stamped. When a unit shadow, another image's pending results or a recorded GPU store covers the surface, that read goes through the flush hook: the shadow is published and the worker waits for the GPU, although the upload itself runs on the GPU from the import. The baseline is now taken only when nothing of that kind overlaps the surface. Otherwise the image has none, and its write-back keeps the CPU's bytes for stamped blocks, the rule before #1599. TechnicalDebt's entry names the condition. agc_storage_baseline_settled creates a storage image over a watched host import with a recorded store pending over part of it and checks that the upload did not submit the open batch; without the change the baseline read syncs on that store. agc_unchanged_cpu_stamps (#1599) still passes. PPSA14632 (LEGO Horizon Adventures) refreshes such images about 870 times per 10 s from queue 0 dispatches: the hook's recorded-store syncs go from about 3400 waiting 4.0 s to about 630 waiting 0.3 s per 10 s, and a direct dispatch's device call from 356 to 107 us. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 6 ++ .../Graphics/include/Texture.hpp | 1 + .../libSceAgcDriver/Graphics/src/Texture.cpp | 9 +- .../prx/libSceAgcDriver/tests/Recorder.cpp | 85 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 2 +- 5 files changed, 101 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index d24c90ab0..7e4600b07 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -699,6 +699,12 @@ if(BUILD_TESTING) set_tests_properties(agc_unchanged_cpu_stamps PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test(NAME agc_storage_baseline_settled COMMAND agc_driver_recorder_tests --settled-baseline-only) + set_tests_properties(agc_storage_baseline_settled PROPERTIES SKIP_RETURN_CODE 77) + if(WIN32) + set_tests_properties(agc_storage_baseline_settled PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test(NAME agc_host_import_unmap COMMAND agc_driver_recorder_tests --host-import-unmap-only) set_tests_properties(agc_host_import_unmap PROPERTIES SKIP_RETURN_CODE 77) if(WIN32) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp index 0003ac1e1..d5f747c29 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp @@ -317,6 +317,7 @@ private: void writeBack(std::uint64_t address, std::size_t bytes); void writeBackLayers(const std::vector& layers); bool unchangedSinceBaseline(std::uint64_t from, std::uint64_t to) const; + bool guestBytesSettled() const; // Tracked units as 64 KiB write-stamp blocks (`blockUnits`: a thin tiled surface at a 64 KiB // aligned base; APS5_NO_BLOCK_TRACKING=1 tracks array layers as above instead): a fill of one // layer, a CPU write or another image's store then costs the blocks it touched, moved through diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp index dc6f3e0f7..2c236e3ce 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp @@ -1754,7 +1754,7 @@ void StorageTexture::upload(const std::vector* layers) { const auto now = GuestMemory::CollectWrites(descriptor.baseAddress, static_cast(guestBytes)); const auto stampLayers = [&](bool selectedOnly) { const bool whole = !selectedOnly || layers == nullptr; - if (guestBytes > GenerationBaselineBytes) generationBaseline.clear(); + if (guestBytes > GenerationBaselineBytes || !guestBytesSettled()) generationBaseline.clear(); else if (whole) { generationBaseline.resize(static_cast(guestBytes)); GuestMemory::ReadCommitted(descriptor.baseAddress, generationBaseline); @@ -3745,6 +3745,13 @@ void StorageTexture::writeBack(std::uint64_t address, std::size_t bytes) { writeBackLayers(layers); } +bool StorageTexture::guestBytesSettled() const { + const std::array, 1> range{{{descriptor.baseAddress, descriptor.baseAddress + guestBytes}}}; + if (AnyShadowedOverlaps(range) || AnyPendingOverlaps(range)) return false; + const auto* recorder = Recorder::Active(); + return recorder == nullptr || !recorder->PendingWriteOverlaps(descriptor.baseAddress, static_cast(guestBytes)); +} + bool StorageTexture::unchangedSinceBaseline(std::uint64_t from, std::uint64_t to) const { const auto& baseline = originalValid ? original : generationBaseline; if (baseline.size() != guestBytes) return false; diff --git a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp index 8103a58f0..cb596ac95 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp @@ -2980,6 +2980,86 @@ void unchangedCpuStampTests(const Device& device, Recorder& recorder) { std::cout << "unchanged CPU stamps: ok\n"; } +void settledBaselineTests(const Device& device, Recorder& recorder) { + const auto& base = device.GetContext(); +#ifdef _WIN32 + std::cout << "generation baseline over a pending store: Linux write watch only\n"; + return; +#endif + if (base.hostImportAlignment == 0) { + std::cout << "host imports unavailable: generation baseline over a pending store not tested\n"; + return; + } + constexpr std::uint32_t side = 256; + constexpr std::uint64_t offset = 0x2000; + constexpr std::size_t bytes = 6 * 65536; + const auto decided = PrepareImportWatch(base); + SetImportWatch(base, ImportWatch::Watch); + struct Restore { + const Context& context; + ImportWatch decided; + ~Restore() { SetImportWatch(context, decided); } + } restore{base, decided}; + for (const bool pending : {false, true}) { + void* block = AllocateWatched(bytes, 65536); + if (block == nullptr) { + std::cout << "no write watching: generation baseline over a pending store not tested\n"; + return; + } + std::memset(block, 0x5a, bytes); + const auto blockAddress = reinterpret_cast(block); + const auto address = blockAddress + offset; + { + GuestAllocations::Mutation mutation; + mutation.Add(block, bytes, true, true, true); + } + struct Unregister { + const Context& context; + void* block; + std::uint64_t address; + std::size_t bytes; + ~Unregister() { + { + GuestAllocations::Mutation mutation; + mutation.Remove(block); + } + HostImportFor(context, address, bytes); + ReleaseWatched(block, bytes); + } + } unregister{base, block, blockAddress, bytes}; + AgcDriver::GuestMemory::CollectWritesUncached(blockAddress, bytes); + TextureDetiler detiler(base); + auto context = base; + context.detiler = &detiler; + GuestTextureResource resource{}; + resource.baseAddress = address; + resource.width = side; + resource.height = side; + resource.mipCount = 1; + resource.tileMode = TextureTileMode::kLinear; + resource.dimension = TextureDimension::k2D; + resource.format = 56; + resource.dstSelX = 4; + resource.dstSelY = 5; + resource.dstSelZ = 6; + resource.dstSelW = 7; + const auto surfaceBytes = static_cast(DescribeSurface(resource).guestBytes); + Require(HostImportFor(context, address, surfaceBytes) != nullptr && AgcDriver::GuestMemory::Watched(address, surfaceBytes), "(b) the surface is not a watched host import"); + static_cast(recorder.Commands()); + if (pending) recorder.NotePendingWrite(address + 4096, 4096); + const auto before = recorder.Submissions(); + { + auto image = std::make_shared(context, detiler, resource, 0); + recorder.Keep(image); + if (pending) Require(recorder.Submissions() == before, "(b) an upload over a pending recorded store submitted the open batch to read its generation baseline"); + } + recorder.Submit(); + device.WaitQueue(); + recorder.Sync(); + } + std::cout << "generation baseline over a pending store: ok\n"; +} + void importWindowTests(const Device& device, Recorder& recorder) { using namespace AgcDriver::GuestMemory; const auto& base = device.GetContext(); @@ -4369,6 +4449,10 @@ int main(int argc, char** argv) { unchangedCpuStampTests(device, recorder); return 0; } + if (argc == 2 && std::string_view(argv[1]) == "--settled-baseline-only") { + settledBaselineTests(device, recorder); + return 0; + } if (argc == 2 && std::string_view(argv[1]) == "--completion-labels-only") { completionCountTests(device, recorder); afterRecordedWorkTests(device, recorder); @@ -4421,6 +4505,7 @@ int main(int argc, char** argv) { singlePassTests(device, recorder, false); singlePassTests(device, recorder, true); unchangedCpuStampTests(device, recorder); + settledBaselineTests(device, recorder); importWatchTests(device); staleGenerationTests(device, recorder); importWindowTests(device, recorder); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..0ba0dedfb 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -400,7 +400,7 @@ Throughout the project, every function at every stage either **does exactly what - [Host imports](../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp) (libSceAgcDriver) on Linux skip ranges that are not mapped read-write, which are copied instead. The NVIDIA driver (595.91.07) refuses `VK_EXT_external_memory_host` imports of read-only or executable mappings (and shared mappings of regular files) with `VK_ERROR_OUT_OF_DEVICE_MEMORY`, after which every `vkAllocateMemory` on the device fails the same way. Read-only ranges are never imported, even on drivers that accept them. Shared mappings of regular files are not checked for. - [Cube texture descriptors](../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestTextureResource.cpp) with equal `BASE_ARRAY` and `DEPTH` expose six faces starting at `BASE_ARRAY`. Measured with `image_load`, `image_sample` and `image_get_resinfo` on RX 6600 (gfx1032), using 64x64 R32_UINT faces and base slices 0, 5 and 6. Other ranges containing fewer than six faces or incomplete cube arrays still throw: hardware load bounds and filtered-sample addressing differ. Out-of-range cube-array sampling remains unimplemented. Samplers with `DISABLE_CUBE_WRAP` are created with `VK_SAMPLER_CREATE_NON_SEAMLESS_CUBE_MAP_BIT_EXT`, the flag RADV maps to that bit (Mesa `src/amd/vulkan/radv_sampler.c`, `src/amd/common/ac_descriptors.c`); without `VK_EXT_non_seamless_cube_map` a cube texture sampled through such a sampler throws. PS5 equivalence has not been measured on console hardware. - [DCC display buffers](../../core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp) are presented only when all keys of the console's DCC extent agree on uncompressed or a fast-clear code, padding keys included; other keys, and register-clear keys over a pending image, throw. -- [Storage image write-back](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) stores the GPU results over a CPU-stamped range whose guest bytes equal the copy taken when the image's layers got their generation (images up to 1 MiB; larger ones keep the previous rule: the CPU's bytes win). A CPU store that rewrites the baseline value after the GPU changed it is not told apart from no store, and the GPU result wins there. The 1 MiB limit is a choice, not a console value. +- [Storage image write-back](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) stores the GPU results over a CPU-stamped range whose guest bytes equal the copy taken when the image's layers got their generation (images up to 1 MiB whose guest bytes were settled at that point, with no unit shadow, pending image results or recorded GPU store over them; other images keep the previous rule: the CPU's bytes win). A CPU store that rewrites the baseline value after the GPU changed it is not told apart from no store, and the GPU result wins there. The 1 MiB limit is a choice, not a console value. - [DCC key stores](../../core/libs/prx/libSceAgcDriver/Graphics/src/DccMetadata.cpp) cover the console's DCC extent only for pipe-aligned DCC of single-level, single-layer 4-byte SW_64KB_R_X surfaces; other surfaces, and every texture and color target key read, cover one byte per 256 bytes. - [DCC key fills over other images](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) - a uniform fill at a live image's `dccAddress`, at least one key per 256 bytes long, is a key fill of that image even when other live images overlap it, but only while it is no longer than the image's key extent (`DccKeyCount`, the pipe-aligned count); a longer one is classified by its overlaps as before. The bound is inferred, not measured on a title: a longer fill also writes the bytes past the keys, which the overlapping image may own. Whether the title's bloom-target key fills stay within the extent was not checked (`APS5_TRACE_DCC_KEYS` prints the length of the fills classified as keys, not of the refused ones). - [8_SRGB color targets](../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver) on a device that cannot attach and blend R8_SRGB render into an RGBA8_SRGB copy of the target, so the sRGB encode, decode and blend rounding are the device's; the console's are not measured. From 127e028825e0cc53770511819bc4715814876304 Mon Sep 17 00:00:00 2001 From: alfalfa-star Date: Sat, 10 Oct 2026 16:46:16 +0200 Subject: [PATCH 051/460] perf(agc): list a dispatch or draw as a copied writer only while it has a CPU write-back A recorded dispatch or draw whose use needs completion work goes into the copied-writer lists (VulkanDevice's copiedWriters, DrawCopiedWriters). Fills, buffer copies and indirect argument reads treat every write of a listed object as a store the CPU makes when its batch is reaped, and wait for that batch. Completion work is also needed for the BDA fault check alone, and the written sub-ranges of GPU copies are copied back by the GPU (RecordCopyBacks) before the listing. Such an object has no CPU write-back (HasCopiedWrites is false), and its stores are ordered by the barriers like direct writes, yet it was listed and waited for. Both lists now take an object only when HasCopiedWrites holds after MarkGpuWrites. The completion action still runs for the fault check. agc_driver_copied_writer_copy records a dispatch that reads through global (BDA) loads and stores through a bound buffer into registered memory, then copies that output on the GPU (VulkanDevice::CopyBuffer) before the batch ran. It checks that the copy is a transfer between the host imports without a sync, and the copied bytes. Without the change the copy waits for the dispatch's batch. PPSA14632 (LEGO Horizon Adventures): every one of the DMA_DATA copy waits counted over 75 s came from such writers. Queue 0's GPU-mutex hold for copies drops from 3.5 s to 30 ms per 10 s, and presents rise from about 102 to 109 per 10 s. Co-Authored-By: Claude Opus 5.5 --- .../Execution/src/VulkanDevice.cpp | 2 +- .../prx/libSceAgcDriver/Graphics/src/Draw.cpp | 2 +- .../tests/execution/CopiedWriterCopy.cpp | 135 ++++++++++++++++++ 3 files changed, 137 insertions(+), 2 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/CopiedWriterCopy.cpp diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index be76518fd..90e0bdc2b 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -3561,7 +3561,7 @@ void VulkanDevice::recordDispatch(RecordedDispatch& record) { kept->WriteBackBuffers(); }); // Listed after the registration: a throw there leaves nothing that would pin the CPU path forever. - writers->push_back(kept); + if (resources.HasCopiedWrites()) writers->push_back(kept); } recordStep(PhaseRecordCompletion); } diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp index fb6a8680e..4dfc54bfa 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp @@ -1442,7 +1442,7 @@ void keepRecordedDraw(Recorder& recorder, const std::shared_ptr resources->MarkGpuWrites(recorder); // The write-back (fault check, copied buffers) runs when the batch completed; a fault is // reported by the recorder ("deferred write-back failed") instead of thrown out of the draw. - if (listed) { + if (listed && resources->HasCopiedWrites()) { // As VulkanDevice::dispatch lists its copied writers: delisted before the write-back // (one that fails must not keep indirect dispatches on the CPU), listed after the // registration (a throw there leaves nothing behind). diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/CopiedWriterCopy.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/CopiedWriterCopy.cpp new file mode 100644 index 000000000..c4b7506b9 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/CopiedWriterCopy.cpp @@ -0,0 +1,135 @@ +#include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "prx/libc/include/GuestAllocations.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#ifdef _WIN32 +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include +#endif +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 32; +constexpr std::uint32_t OutputBytes = Threads * 16u * 4u; +constexpr std::size_t BlockBytes = 65536; +constexpr std::uint8_t Fill = 0xcd; + +alignas(256) constexpr std::array LoadsCode{ + 0x34020084, 0x34040086, 0xdc388000, 0x04080001, 0xbf8c3f70, 0x7e1402ff, 0x5a5a0000, 0x7e1602ff, + 0x5a5a0001, 0x7e1802ff, 0x5a5a0002, 0x7e1a02ff, 0x5a5a0003, 0x7e1c02ff, 0x5a5a0004, 0x7e1e02ff, + 0x5a5a0005, 0x7e2002ff, 0x5a5a0006, 0x7e2202ff, 0x5a5a0007, 0x7e2402ff, 0x5a5a0008, 0x7e2602ff, + 0x5a5a0009, 0x7e2802ff, 0x5a5a000a, 0x7e2a02ff, 0x5a5a000b, 0x7e2c02ff, 0x5a5a000c, 0x7e2e02ff, + 0x5a5a000d, 0x7e3002ff, 0x5a5a000e, 0x7e3202ff, 0x5a5a000f, 0x4a4008ff, 0x00000a00, 0x4a420eff, + 0x00000200, 0xd70f6a22, 0x02024008, 0x7e460209, 0x50464680, 0x4a4c0aff, 0x00000200, 0xd70f6a24, + 0x02024c08, 0x7e4a0209, 0x504a4a80, 0x36500c81, 0x7daa5080, 0xdc208800, 0x0a080020, 0xdc2487ff, + 0x0b080020, 0xdc288ffd, 0x0c080020, 0xdc2c8001, 0x0d7d0022, 0xdc308fff, 0x0e080020, 0xdc308000, + 0x18080021, 0xdc208800, 0x197d0022, 0xbf8c0070, 0xbefe04c1, 0xe0781000, 0x80010a02, 0xe0781010, + 0x80010e02, 0xe0781020, 0x80011202, 0xe0781030, 0x80011602, 0xbf810000 +}; + +class GuestBlock { +public: + GuestBlock() { +#ifdef _WIN32 + block = static_cast(VirtualAlloc(nullptr, BlockBytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); +#else + block = static_cast(std::aligned_alloc(BlockBytes, BlockBytes)); +#endif + Require(block != nullptr, "copied writer: cannot allocate a guest block"); + std::memset(block, Fill, BlockBytes); + GuestAllocations::Mutation().Add(block, BlockBytes, true, true, true); + } + + ~GuestBlock() { + GuestAllocations::Mutation().Remove(block); +#ifdef _WIN32 + VirtualFree(block, 0, MEM_RELEASE); +#else + std::free(block); +#endif + } + + GuestBlock(const GuestBlock&) = delete; + GuestBlock& operator=(const GuestBlock&) = delete; + + std::uint8_t* Data() const { return block; } + std::uint64_t Address() const { return static_cast(reinterpret_cast(block)); } + +private: + std::uint8_t* block = nullptr; +}; + +void RecordDispatch(AgcDriver::VulkanDevice& device, const GuestBlock& input, const GuestBlock& output) { + std::vector userData(10, 0u); + const auto address = output.Address(); + const std::array descriptor{static_cast(address), static_cast((address >> 32u) & 0xffffu), OutputBytes, 0x31016facu}; + std::copy(descriptor.begin(), descriptor.end(), userData.begin() + 4); + userData[8] = static_cast(input.Address()); + userData[9] = static_cast(input.Address() >> 32u); + const std::span code(LoadsCode); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0, {false, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + const GuestBlock input; + std::memset(input.Data(), 0, BlockBytes); + const GuestBlock output; + const GuestBlock destination; + RecordDispatch(*device, input, output); + AgcDriver::VulkanDevice::CopyOutcome outcome{}; + { + std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex()); + outcome = device->CopyBuffer(destination.Address(), output.Address(), OutputBytes, 0, std::numeric_limits::max(), 0, 0, 0, [](std::span, std::uint64_t) {}); + } + if (outcome.path == 2) { + device->WaitIdle(); + std::puts("skipped, the device imports no host memory"); + return VulkanTestSkipped; + } + Require(outcome.path == 1, "copied writer: the copy did not run as a transfer between the host imports (path " + std::to_string(outcome.path) + ")"); + Require(!outcome.synced, "copied writer: a copy of what an address-based dispatch stored through a bound buffer waited for its batch, although no CPU write-back follows it"); + device->WaitIdle(); + Require(std::any_of(output.Data(), output.Data() + OutputBytes, [](std::uint8_t byte) { return byte != Fill; }), "copied writer: the dispatch stored nothing"); + for (std::uint32_t offset = 0; offset < OutputBytes; ++offset) { + Require(destination.Data()[offset] == output.Data()[offset], "copied writer: copied byte " + std::to_string(offset) + " is " + std::to_string(destination.Data()[offset]) + ", the dispatch stored " + std::to_string(output.Data()[offset])); + } + std::puts("copied writer copy tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} From 57be03a4c1cb52f7ee3ca0bec3b4b6874cea1a60 Mon Sep 17 00:00:00 2001 From: Tharun Balaji Date: Sat, 10 Oct 2026 20:42:36 +0530 Subject: [PATCH 052/460] fix: guard sceKernelFdatasync against guest socket descriptors Add the guest socket descriptor guard to sceKernelFdatasync before any native syscall, matching the existing sceKernelFchmod pattern. This prevents high guest socket descriptors (e.g. 0x10000000) from reaching the Windows CRT _commit, which would trigger the invalid-parameter handler, and ensures the correct SCE error is returned on all platforms. Refactor fdatasync_nid_postfix to delegate to sceKernelFdatasync via PosixResult, removing the duplicated socket guard and native syscall. Extend GuestFilesystem tests to cover: - sceKernelFdatasync on a valid guest socket returns SCE EINVAL (0x80020016) - sceKernelFdatasync on a closed guest socket returns SCE EBADF (0x80020009) - sceKernelFdatasync on a high invalid descriptor (0x7fffffff) returns SCE EBADF - closed-ordinary-descriptor tests now run on Windows (verified _commit on a closed descriptor returns EBADF without provoking the CRT handler) --- core/libs/prx/libkernel/File/src/Stdio.cpp | 15 +++++---------- core/libs/tests/GuestFilesystem.cpp | 5 +++-- 2 files changed, 8 insertions(+), 12 deletions(-) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index 983528a93..f31ffaa8a 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -464,12 +464,13 @@ int APS5_VABI sceKernelFsync(int fd) { } int APS5_VABI sceKernelFdatasync(int fd) { + if (fd >= GuestSockets::FirstDescriptor) return SceErrorFromErrno(GuestSockets::IsOpen(fd) ? GUEST_EINVAL : GUEST_EBADF); #ifdef _WIN32 - if (::_commit(fd) != 0) return SceErrorFromErrno(errno); + if (::_commit(fd) != 0) return SceErrorFromErrno(errno); #else - if (::fdatasync(fd) != 0) return SceErrorFromErrno(errno); + if (::fdatasync(fd) != 0) return SceErrorFromErrno(errno); #endif - return 0; + return 0; } int APS5_VABI sceKernelWriteThrottlingStatus(std::uint64_t* status) { @@ -801,13 +802,7 @@ int APS5_VABI fsync_nid_postfix(int fd) { } int APS5_VABI fdatasync_nid_postfix(int fd) { - if (fd >= GuestSockets::FirstDescriptor) return PosixFailure(GuestSockets::IsOpen(fd) ? GUEST_EINVAL : GUEST_EBADF); -#ifdef _WIN32 - if (::_commit(fd) != 0) return PosixResult(SceErrorFromErrno(errno)); -#else - if (::fdatasync(fd) != 0) return PosixResult(SceErrorFromErrno(errno)); -#endif - return 0; + return PosixResult(sceKernelFdatasync(fd)); } } diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index 86c8c5b22..93551d5f6 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -160,23 +160,24 @@ int main() { Require(futimes_nid_postfix(descriptor, negative) == -1 && *__error_nid_postfix() == 22); Require(std::fclose(native) == 0); Require(std::filesystem::file_size(sized) == 3); -#ifndef _WIN32 Require(sceKernelFchmod(descriptor, 0600) == static_cast(0x80020009u)); Require(fchmod_nid_postfix(descriptor, 0600) == -1 && *__error_nid_postfix() == 9); Require(futimes_nid_postfix(descriptor, nullptr) == -1 && *__error_nid_postfix() == 9); Require(fdatasync_nid_postfix(descriptor) == -1 && *__error_nid_postfix() == 9); Require(sceKernelFdatasync(descriptor) == static_cast(0x80020009u)); -#endif const int socket = socket_nid_postfix(2, 2, 0); Require(socket >= 0); Require(sceKernelFchmod(socket, 0600) == static_cast(0x80020016u)); Require(fchmod_nid_postfix(socket, 0600) == -1 && *__error_nid_postfix() == 22); Require(futimes_nid_postfix(socket, nullptr) == -1 && *__error_nid_postfix() == 22); Require(fdatasync_nid_postfix(socket) == -1 && *__error_nid_postfix() == 22); + Require(sceKernelFdatasync(socket) == static_cast(0x80020016u)); Require(close_nid_postfix(socket) == 0); Require(fchmod_nid_postfix(socket, 0600) == -1 && *__error_nid_postfix() == 9); Require(futimes_nid_postfix(socket, nullptr) == -1 && *__error_nid_postfix() == 9); Require(fdatasync_nid_postfix(socket) == -1 && *__error_nid_postfix() == 9); + Require(sceKernelFdatasync(socket) == static_cast(0x80020009u)); + Require(sceKernelFdatasync(0x7fffffff) == static_cast(0x80020009u)); Require(remove_nid_postfix(sized.string().c_str()) == 0); const auto present = root / "present.txt"; const auto presentName = present.string(); From 22cb7094ae0b677694dcd42cb84c60a5251c28d2 Mon Sep 17 00:00:00 2001 From: krishnaraya Date: Fri, 9 Oct 2026 13:08:05 -0400 Subject: [PATCH 053/460] fix(agc): sample 5_6_5, 1_5_5_5 and 4_4_4_4 textures in AMD's channel order Co-Authored-By: Claude Opus 5.5 (1M context) --- .../Graphics/include/TextureFormat.hpp | 1 + .../Graphics/src/ShaderResources.cpp | 3 +- .../Graphics/src/TextureFormat.cpp | 13 ++ .../libSceAgcDriver/tests/TextureFormat.cpp | 25 +++ .../tests/execution/ImagePacked16Load.cpp | 151 ++++++++++++++++++ 5 files changed, 192 insertions(+), 1 deletion(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/ImagePacked16Load.cpp diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/TextureFormat.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/TextureFormat.hpp index 47552e4dc..fc7be9ec7 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/TextureFormat.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/TextureFormat.hpp @@ -13,6 +13,7 @@ namespace AgcDriver::Graphics { struct Context; VkFormat ResolveTextureFormat(std::uint32_t guestFormat); +VkComponentSwizzle TextureComponentChannel(std::uint32_t guestFormat, VkComponentSwizzle component); std::uint32_t SrgbDecodeFormats(PFN_vkGetPhysicalDeviceFormatProperties formatProperties, VkPhysicalDevice physical); VkFormat SampledTextureFormat(const Context& context, std::uint32_t guestFormat); bool IsSrgbTextureFormat(std::uint32_t guestFormat); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index b5d4296a3..11274823a 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -54,7 +54,8 @@ VkComponentSwizzle ComponentSwizzleFor(std::uint8_t dstSel) { VkComponentMapping ViewComponents(const GuestTextureResource& resource) { if (IsConvertedTextureFormat(resource.format)) return {VK_COMPONENT_SWIZZLE_R, VK_COMPONENT_SWIZZLE_G, VK_COMPONENT_SWIZZLE_B, VK_COMPONENT_SWIZZLE_A}; - return {ComponentSwizzleFor(resource.dstSelX), ComponentSwizzleFor(resource.dstSelY), ComponentSwizzleFor(resource.dstSelZ), ComponentSwizzleFor(resource.dstSelW)}; + const auto channel = [&](std::uint8_t dstSel) { return TextureComponentChannel(resource.format, ComponentSwizzleFor(dstSel)); }; + return {channel(resource.dstSelX), channel(resource.dstSelY), channel(resource.dstSelZ), channel(resource.dstSelW)}; } // Sampled textures are reused across draws and dispatches while their guest bytes are unchanged; a diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp index 96eeaf9c2..524412d9f 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp @@ -154,6 +154,19 @@ std::optional FindGuestTextureFormat(VkFormat format, std::uint32 return std::nullopt; } +VkComponentSwizzle TextureComponentChannel(std::uint32_t guestFormat, VkComponentSwizzle component) { + if (component < VK_COMPONENT_SWIZZLE_R || component > VK_COMPONENT_SWIZZLE_A) return component; + const auto index = static_cast(component - VK_COMPONENT_SWIZZLE_R); + constexpr std::array bgra{VK_COMPONENT_SWIZZLE_B, VK_COMPONENT_SWIZZLE_G, VK_COMPONENT_SWIZZLE_R, VK_COMPONENT_SWIZZLE_A}; + constexpr std::array abgr{VK_COMPONENT_SWIZZLE_A, VK_COMPONENT_SWIZZLE_B, VK_COMPONENT_SWIZZLE_G, VK_COMPONENT_SWIZZLE_R}; + switch (guestFormat) { + case 133: + case 134: return bgra[index]; + case 136: return abgr[index]; + default: return component; + } +} + std::uint32_t BytesPerElement(std::uint32_t guestFormat) { return findFormatEntry(guestFormat).bytesPerElement; } diff --git a/core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp b/core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp index d04cf6500..185741b41 100644 --- a/core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp @@ -5,8 +5,10 @@ #include "RdnaDecoder/RdnaDescriptorFormat.hpp" #include #include +#include #include #include +#include namespace { @@ -55,6 +57,28 @@ void reject(TAction action, std::string_view reason) { throw std::runtime_error(std::string("expected texture format rejection: ") + std::string(reason)); } +int channelLowBit(VkFormat format, VkComponentSwizzle channel) { + const auto index = static_cast(channel - VK_COMPONENT_SWIZZLE_R); + switch (format) { + case VK_FORMAT_R5G6B5_UNORM_PACK16: return std::array{11, 5, 0, -1}[index]; + case VK_FORMAT_A1R5G5B5_UNORM_PACK16: return std::array{10, 5, 0, 15}[index]; + case VK_FORMAT_R4G4B4A4_UNORM_PACK16: return std::array{12, 8, 4, 0}[index]; + default: throw std::runtime_error("no channel layout for packed format " + std::to_string(format)); + } +} + +void packedChannelTests() { + constexpr std::array>, 3> packed{{{133u, {0, 5, 11, -1}}, {134u, {0, 5, 10, 15}}, {136u, {0, 4, 8, 12}}}}; + constexpr std::array components{VK_COMPONENT_SWIZZLE_R, VK_COMPONENT_SWIZZLE_G, VK_COMPONENT_SWIZZLE_B, VK_COMPONENT_SWIZZLE_A}; + for (const auto& [format, lowBits] : packed) { + for (std::size_t component = 0; component < components.size(); ++component) { + const auto channel = TextureComponentChannel(format, components[component]); + Require(channelLowBit(ResolveTextureFormat(format), channel) == lowBits[component], "format " + std::to_string(format) + " component " + std::to_string(component) + " is not read from bit " + std::to_string(lowBits[component])); + } + } + Require(TextureComponentChannel(133, VK_COMPONENT_SWIZZLE_ONE) == VK_COMPONENT_SWIZZLE_ONE && TextureComponentChannel(56, VK_COMPONENT_SWIZZLE_R) == VK_COMPONENT_SWIZZLE_R, "constant selectors and other formats must keep their channels"); +} + void convertedDccClearTests() { alignas(64) std::array keys{}; GuestTextureResource resource{}; @@ -76,6 +100,7 @@ void convertedDccClearTests() { void RunTextureFormatTests() { srgbDecodeTests(); + packedChannelTests(); Require(ResolveTextureFormat(1) == VK_FORMAT_R8_UNORM, "format 1 must resolve to R8_UNORM"); Require(ResolveTextureFormat(2) == VK_FORMAT_R8_SNORM, "format 2 must resolve to R8_SNORM"); Require(BytesPerElement(1) == 1u, "format 1 must be one byte wide"); diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/ImagePacked16Load.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/ImagePacked16Load.cpp new file mode 100644 index 000000000..67ad0e45c --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/ImagePacked16Load.cpp @@ -0,0 +1,151 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Texture.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 32; +constexpr std::uint32_t Width = 256; +constexpr std::uint32_t Groups = Width / Threads; +constexpr std::uint32_t Type2D = 9; +constexpr std::uint32_t SwizzleXYZW = 0xfacu; +constexpr std::uint32_t SwizzleZYXW = 0xf2eu; +constexpr std::uint32_t One = 0x3f800000u; +constexpr std::array PointSampler{0x00000092u, 0x00fff000u, 0u, 0u}; + +struct Packed { + std::uint32_t format; + std::array masks; + const char* name; +}; + +constexpr std::array Formats{{ + {133u, {0x001fu, 0x07e0u, 0xf800u, 0x0000u}, "5_6_5_UNORM"}, + {134u, {0x001fu, 0x03e0u, 0x7c00u, 0x8000u}, "1_5_5_5_UNORM"}, + {136u, {0x000fu, 0x00f0u, 0x0f00u, 0xf000u}, "4_4_4_4_UNORM"}, +}}; + +alignas(256) std::array Output{}; +alignas(256) std::array Texels{}; + +alignas(256) constexpr std::array LoadXyzw{ + 0x7e080218u, 0x343c0885u, 0x4a3c3d00u, 0x34063c84u, 0x7e3e0280u, 0x7e140280u, 0x7e160280u, 0x7e180280u, + 0x7e1a0280u, 0xf0001f08u, 0x00010a1eu, 0xbf8c3f70u, 0xe0781000u, 0x80000a03u, 0xbf810000u, +}; + +alignas(256) constexpr std::array SampleLz{ + 0x34060084u, 0x7e280280u, 0x7e2a0280u, 0xf09c0f08u, 0x00610a14u, 0xbf8c3f70u, 0xe0781000u, 0x80000a03u, + 0xbf810000u, +}; + +std::string Hex(std::uint32_t value) { + char text[16]; + std::snprintf(text, sizeof(text), "0x%08x", value); + return text; +} + +std::uint32_t Pattern(std::uint32_t x) { + return (x + 1u) & 15u; +} + +std::array TextureDescriptor(std::uint32_t format, std::uint32_t swizzle) { + const auto address = static_cast(reinterpret_cast(Texels.data())); + return { + static_cast(address >> 8u), + static_cast((address >> 40u) & 0xffu) | (format << 20u) | (((Width - 1u) & 3u) << 30u), + (Width - 1u) >> 2u, + swizzle | (Type2D << 28u), + 0u, + 0u, + 0u, + 0u, + }; +} + +void Fill(const Packed& format) { + for (std::uint32_t x = 0; x < Width; ++x) { + std::uint16_t texel = 0; + for (std::uint32_t component = 0; component < 4u; ++component) { + if (((Pattern(x) >> component) & 1u) != 0u) texel = static_cast(texel | format.masks[component]); + } + Texels[x] = texel; + } +} + +void Run(AgcDriver::VulkanDevice& device, std::span code, std::uint32_t format, std::uint32_t swizzle, std::uint32_t groups) { + Output.fill(0xdeadbeefu); + std::vector userData(24, 0u); + const auto outputAddress = reinterpret_cast(Output.data()); + const std::array output{static_cast(outputAddress), static_cast((outputAddress >> 32u) & 0xffffu), static_cast(Output.size() * 4u), 0x31016facu}; + const auto texture = TextureDescriptor(format, swizzle); + std::copy(output.begin(), output.end(), userData.begin()); + std::copy(texture.begin(), texture.end(), userData.begin() + 4); + std::copy(PointSampler.begin(), PointSampler.end(), userData.begin() + 12); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {true, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, groups, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); +} + +void Verify(const Packed& format, std::uint32_t swizzle, std::uint32_t texels, bool sampled, const std::string& what) { + for (std::uint32_t index = 0; index < texels; ++index) { + const auto x = sampled ? 0u : index; + std::array channels{}; + for (std::uint32_t component = 0; component < 4u; ++component) { + channels[component] = format.masks[component] == 0u || ((Pattern(x) >> component) & 1u) != 0u ? One : 0u; + } + for (std::uint32_t component = 0; component < 4u; ++component) { + const auto selector = (swizzle >> (component * 3u)) & 7u; + const auto expected = selector == 0u ? 0u : selector == 1u ? One : channels[selector - 4u]; + const auto actual = Output[index * 4u + component]; + Require(actual == expected, what + ": " + (sampled ? "thread " : "texel ") + std::to_string(index) + " component " + std::to_string(component) + " is " + Hex(actual) + ", expected " + Hex(expected)); + } + } +} + +void Check(AgcDriver::VulkanDevice& device, const Packed& format, std::uint32_t swizzle, const std::string& swizzleName) { + Fill(format); + Run(device, LoadXyzw, format.format, swizzle, Groups); + Verify(format, swizzle, Width, false, std::string(format.name) + " image_load " + swizzleName); + Run(device, SampleLz, format.format, swizzle, 1u); + Verify(format, swizzle, Threads, true, std::string(format.name) + " image_sample_lz " + swizzleName); +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + for (const auto& format : Formats) { + Check(*device, format, SwizzleXYZW, "X Y Z W"); + Check(*device, format, SwizzleZYXW, "Z Y X W"); + } + std::puts("image packed 16-bit load tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} From 1feb89cd5679e727644328a214c51a6cbbcaa98a Mon Sep 17 00:00:00 2001 From: krishnaraya Date: Sat, 10 Oct 2026 11:22:07 -0400 Subject: [PATCH 054/460] fix(agc): read X for a DST_SEL W of 5_6_5 textures as measured on gfx1035 Co-Authored-By: Claude Opus 5.5 (1M context) --- core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp | 3 ++- core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp | 2 +- .../prx/libSceAgcDriver/tests/execution/ImagePacked16Load.cpp | 3 ++- docs/dev/TechnicalDebt.md | 1 + 4 files changed, 6 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp index 524412d9f..6926b8df7 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp @@ -157,10 +157,11 @@ std::optional FindGuestTextureFormat(VkFormat format, std::uint32 VkComponentSwizzle TextureComponentChannel(std::uint32_t guestFormat, VkComponentSwizzle component) { if (component < VK_COMPONENT_SWIZZLE_R || component > VK_COMPONENT_SWIZZLE_A) return component; const auto index = static_cast(component - VK_COMPONENT_SWIZZLE_R); + constexpr std::array bgrb{VK_COMPONENT_SWIZZLE_B, VK_COMPONENT_SWIZZLE_G, VK_COMPONENT_SWIZZLE_R, VK_COMPONENT_SWIZZLE_B}; constexpr std::array bgra{VK_COMPONENT_SWIZZLE_B, VK_COMPONENT_SWIZZLE_G, VK_COMPONENT_SWIZZLE_R, VK_COMPONENT_SWIZZLE_A}; constexpr std::array abgr{VK_COMPONENT_SWIZZLE_A, VK_COMPONENT_SWIZZLE_B, VK_COMPONENT_SWIZZLE_G, VK_COMPONENT_SWIZZLE_R}; switch (guestFormat) { - case 133: + case 133: return bgrb[index]; case 134: return bgra[index]; case 136: return abgr[index]; default: return component; diff --git a/core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp b/core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp index 185741b41..00a5f561a 100644 --- a/core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp @@ -68,7 +68,7 @@ int channelLowBit(VkFormat format, VkComponentSwizzle channel) { } void packedChannelTests() { - constexpr std::array>, 3> packed{{{133u, {0, 5, 11, -1}}, {134u, {0, 5, 10, 15}}, {136u, {0, 4, 8, 12}}}}; + constexpr std::array>, 3> packed{{{133u, {0, 5, 11, 0}}, {134u, {0, 5, 10, 15}}, {136u, {0, 4, 8, 12}}}}; constexpr std::array components{VK_COMPONENT_SWIZZLE_R, VK_COMPONENT_SWIZZLE_G, VK_COMPONENT_SWIZZLE_B, VK_COMPONENT_SWIZZLE_A}; for (const auto& [format, lowBits] : packed) { for (std::size_t component = 0; component < components.size(); ++component) { diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/ImagePacked16Load.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/ImagePacked16Load.cpp index 67ad0e45c..7d7053efa 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/ImagePacked16Load.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/ImagePacked16Load.cpp @@ -113,8 +113,9 @@ void Verify(const Packed& format, std::uint32_t swizzle, std::uint32_t texels, b const auto x = sampled ? 0u : index; std::array channels{}; for (std::uint32_t component = 0; component < 4u; ++component) { - channels[component] = format.masks[component] == 0u || ((Pattern(x) >> component) & 1u) != 0u ? One : 0u; + channels[component] = format.masks[component] != 0u && ((Pattern(x) >> component) & 1u) != 0u ? One : 0u; } + if (format.masks[3] == 0u) channels[3] = channels[0]; for (std::uint32_t component = 0; component < 4u; ++component) { const auto selector = (swizzle >> (component * 3u)) & 7u; const auto expected = selector == 0u ? 0u : selector == 1u ? One : channels[selector - 4u]; diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..143bd5d34 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -423,6 +423,7 @@ Throughout the project, every function at every stage either **does exactly what - [Negative color-texture comparison wrap indices](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (libSceAgcDriver) use explicit `OpSRem` normalization to implement modulo; the negative-offset point-wrap regression fails with direct `OpSMod` and passes with normalization on Mesa 22.3.6 llvmpipe (LLVM 15.0.6), but negative-index wrap has not been measured on PS5 hardware. - [libScePlayerInvitationDialog](../../core/libs/prx/libScePlayerInvitationDialog/libScePlayerInvitationDialog.cpp) simulates dialog completion without displaying UI or sending invitations, so `scePlayerInvitationDialogGetResult` reports `USER_CANCELED`; its parameter ABI remains unverified. The 40-byte result layout (error code, result, 32 reserved bytes) is taken from [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Dialog/PlayerInvitationDialog.cs) and the error codes from the other common dialogs, not confirmed on a title. - [Converted unorm images](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (10_11_11_UNORM, read and written through an R32_UINT view): samples, gathers, LOD queries, 16-bit data and a DST_SEL of the missing W channel throw. +- [Packed 16-bit textures](../../core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp) (libSceAgcDriver) - `5_6_5`, `1_5_5_5` and `4_4_4_4` are viewed as R5G6B5, A1R5G5B5 and R4G4B4A4 with a swizzle that reads X from the low bits, and a DST_SEL of the missing W of `5_6_5` reads X. Measured on RDNA2 (gfx1035), not on the console: through `image_load` and `image_sample_lz`, X, Y, Z (and W of `1_5_5_5` and `4_4_4_4`) come from the low, middle and high fields, and W of `5_6_5` returns X (the low 5 bits), not 1. - [Converted unorm image loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) divide by 2^n - 1 with the buffer format path's `OpFDiv`, which Vulkan allows to be up to 2.5 ULP off, so some drivers are not bit-exact. - [64-bit LDS atomics](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (`ds_*_u64`, `ds_*_b64`, `ds_*_f64`) read and write the two dwords under a workgroup lock stored after the guest LDS, as LDS is a 32-bit array without 64-bit atomics. They are atomic with respect to each other only: 32-bit accesses to the same dwords issued concurrently are not ordered by the lock. 64-bit GDS atomics throw. On the hardware the LDS serialises 32-bit and 64-bit atomics on the same address (256 lanes mixing `ds_add_u32` and `ds_add_u64` on one slot sum exactly), and under the lock the 32-bit adds can be lost: in 20 runs per host and wave size, RADV lost 32 to 96 of the 128 in every run and the RTX 3080 16 to 80 in 18 of 20 wave32 runs and 16 in 3 of 20 wave64 runs, always a multiple of 16. A 64-bit atomic at an address that is 4-byte but not 8-byte aligned faults the wave on gfx1036, as on gfx1035. - [LDS float atomics](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (`ds_min/max/cmpst_f32`, `ds_min/max/cmpst_f64` and their `_rtn` forms) follow the hardware measured on RDNA2 (gfx1036) over 14 x 14 operand sets (zeros, ones, infinities, denormals, quiet and signalling NaNs of both signs and with payloads) in IEEE 0 and 1 with every value 0-3 of the type's denormal field: both operands are flushed to a signed zero when bit 0 of the field is clear (0 and 2; 1 and 3 keep them; the f32 field for the f32 forms, the f16/f64 field for the f64 forms; a shader without a known float mode flushes f32 and keeps f64), then a signalling-NaN data operand is stored quieted, else a signalling-NaN old value is stored quieted, else a quiet-NaN old value is replaced by the data, else a quiet-NaN data operand loses, else the ordered compare with -0 below +0 decides, and the flushed value is what is written, also when the old value wins. `cmpst` compares the flushed bits (both zeros equal, NaN never equal) and leaves memory unchanged when they differ. The returned value is always the old value as it was in memory. `IEEE_MODE` has no effect. The same bit rule serves the buffer float atomics (`AtomicFloatBits`). Not measured: a denormal or signalling-NaN `data1` of `cmpst` (stored as given), and `ds_add_f32`, which keeps the host's `OpFAdd`. From f4b6e1011d5fe96af758da27220b46dbca288cdf Mon Sep 17 00:00:00 2001 From: boykopovar Date: Sat, 10 Oct 2026 18:37:56 +0300 Subject: [PATCH 055/460] ci(pr): set min intake pr count to 10 --- .github/workflows/labels.yml | 2 +- .github/workflows/pr-intake.yml | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/labels.yml b/.github/workflows/labels.yml index 7ac7a0c10..703f7ecee 100644 --- a/.github/workflows/labels.yml +++ b/.github/workflows/labels.yml @@ -24,7 +24,7 @@ jobs: force_label() { gh label create "$1" --color "$2" --description "$3" --force; } force_label 'needs rebase' b60205 'Conflicts with main, rebase needed' - force_label 'intake paused' d93f0b 'Closed while PR intake is limited to authors with at least 20 PRs merged into main' + force_label 'intake paused' d93f0b 'Closed while PR intake is limited to authors with at least 10 PRs merged into main' force_label ci 5319e7 '.github' force_label agc 1d76db 'libSceAgc*' force_label shader 1d76db 'core/shader' diff --git a/.github/workflows/pr-intake.yml b/.github/workflows/pr-intake.yml index a632bd61c..c45ffe04c 100644 --- a/.github/workflows/pr-intake.yml +++ b/.github/workflows/pr-intake.yml @@ -27,7 +27,7 @@ jobs: if (pr.state !== 'open' || pr.base.ref !== 'main') { return; } - const minimumMergedPRs = 20; + const minimumMergedPRs = 10; const { data: mergedPRs } = await github.rest.search.issuesAndPullRequests({ q: `repo:${owner}/${repo} is:pr is:merged base:main author:${pr.user.login}`, per_page: 1 @@ -48,7 +48,7 @@ jobs: try { await github.rest.issues.createLabel({ owner, repo, name, color: 'd93f0b', - description: 'Closed while PR intake is limited to authors with at least 20 PRs merged into main' + description: 'Closed while PR intake is limited to authors with at least 10 PRs merged into main' }); } catch (createError) { if (createError.status !== 422) { From 5d4e0f2b4326ab3c5b53661d31a2aa0834a45c08 Mon Sep 17 00:00:00 2001 From: ThatsLeo Date: Sat, 10 Oct 2026 15:58:23 +0200 Subject: [PATCH 056/460] fix(agc): size the pipeline cache fixture to the device's vertex input limits Mesa ANV reports 29 vertex attributes; Vulkan guarantees 16. --- core/libs/prx/libSceAgcDriver/tests/Recorder.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp index 33cc36f95..556b99267 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp @@ -4211,8 +4211,8 @@ void pipelineCacheTests(const Device& device, bool benchmark) { state.cullMode = VK_CULL_MODE_BACK_BIT; Require(lookup(input) != first, "pipeline cache ignored changed raster state"); state.cullMode = 0; - for (const auto attributes : {0u, 8u, 32u}) { - Require(attributes <= context.limits.maxVertexInputBindings && attributes <= context.limits.maxVertexInputAttributes, "pipeline cache fixture exceeds vertex input limits"); + const auto largest = std::min({32u, context.limits.maxVertexInputBindings, context.limits.maxVertexInputAttributes}); + for (const auto attributes : {0u, 8u, largest}) { input.bindings.clear(); input.attributes.clear(); for (std::uint32_t i = 0; i < attributes; ++i) { From 7eba9f5b160a73be0b3a8bf38d11af74600579f6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jan=20=C5=A0l=C3=A9gl?= Date: Sat, 10 Oct 2026 14:49:50 +0200 Subject: [PATCH 057/460] perf(recompiler): index memory users once per indirect image planning pass MemoryIndexBelongsTo walked every instruction of the program and asked BufferAccessOf, AddressOpcodeInfoOf and ImageOpcodeInfoOf about each one. TryMakeTableImage calls it for each of a table handle's eight descriptor dwords and once more for a material key, and PlanIndirectImages calls TryMakeTableImage for every image instruction whose handle is not planned yet, so each such instruction could cost up to nine walks of the whole program. PlanIndirectImages now indexes the instructions that access memory by their memory index once, before it plans, and MemoryIndexBelongsTo looks the index up while that pass runs. The planning pass adds, removes and changes no instruction, so the index stays valid until the pass drops it at its end. Outside the pass, where the function has no caller, it walks the program as before. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Fnrk5Dazjb2tLooVSpdRQq --- .../Optimization/src/ResourceTracker.cpp | 29 +++++++++++++++++-- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/core/shader/recompiler/Optimization/src/ResourceTracker.cpp b/core/shader/recompiler/Optimization/src/ResourceTracker.cpp index 3f20951dd..f83921280 100644 --- a/core/shader/recompiler/Optimization/src/ResourceTracker.cpp +++ b/core/shader/recompiler/Optimization/src/ResourceTracker.cpp @@ -274,11 +274,29 @@ private: return memory.kind == ResourceKind::ScalarBuffer && memory.dataBits == 32u && memory.dataDwords == 1u ? &memory : nullptr; } - bool MemoryIndexBelongsTo(std::uint32_t index, const IrValue& owner) const { + static bool AccessesMemory(IrOpcode op) { + return BufferAccessOf(op) != BufferAccess::None || AddressOpcodeInfoOf(op).access != AddressAccess::None || ImageOpcodeInfoOf(op).access != ImageAccess::None; + } + + void IndexMemoryUsers() { for (const auto& block : m_program.Blocks()) { for (const IrValue* inst : block->Instructions()) { - const auto op = inst->Opcode(); - if ((BufferAccessOf(op) == BufferAccess::None && AddressOpcodeInfoOf(op).access == AddressAccess::None && ImageOpcodeInfoOf(op).access == ImageAccess::None) || inst == &owner) { + if (AccessesMemory(inst->Opcode())) { + m_memoryUsers[inst->Flags().index].push_back(inst); + } + } + } + m_memoryUsersIndexed = true; + } + + bool MemoryIndexBelongsTo(std::uint32_t index, const IrValue& owner) const { + if (m_memoryUsersIndexed) { + const auto users = m_memoryUsers.find(index); + return users == m_memoryUsers.end() || std::ranges::all_of(users->second, [&](const IrValue* user) { return user == &owner; }); + } + for (const auto& block : m_program.Blocks()) { + for (const IrValue* inst : block->Instructions()) { + if (!AccessesMemory(inst->Opcode()) || inst == &owner) { continue; } if (inst->Flags().index == index) { @@ -898,6 +916,7 @@ private: if (!enabled) { return; } + IndexMemoryUsers(); for (auto& block : m_program.Blocks()) { for (IrValue* inst : block->Instructions()) { const auto imageInfo = ImageOpcodeInfoOf(inst->Opcode()); @@ -920,6 +939,8 @@ private: } } } + m_memoryUsersIndexed = false; + m_memoryUsers.clear(); } void GetHandle(IrValue* value, IrOpcode expected, std::uint32_t width, IrValue*& handle, std::uint32_t& source, bool sampler = false, bool sampleAdjust = false) { @@ -1263,6 +1284,8 @@ private: // APS5_TRACE_BDA: address accesses seen by Collect (the first bdaTraceLimit are printed). unsigned m_bdaTraces = 0; std::vector m_indirectImages; + std::unordered_map> m_memoryUsers; + bool m_memoryUsersIndexed = false; struct EdgeSelectorEntry { IrBlock* block = nullptr; std::vector edges; From 33184a1b335b7fc643725c5afe4b7debe27fe2f3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jan=20=C5=A0l=C3=A9gl?= Date: Fri, 9 Oct 2026 21:52:38 +0200 Subject: [PATCH 058/460] feat(libSceSsl): implement sceSslGetCaList, sceSslFreeCaList and sceSslUnloadCert PPSA11386 imports sceSslGetCaList and sceSslFreeCaList and PPSA15508 imports sceSslUnloadCert; on main all three are missing and the loaders of both titles stop on them. No root certificate bundle is shipped and no loaded certificate is kept, so an SSL context holds no certificates: sceSslGetCaList clears the list and returns NOT_FOUND, as sceSslGetCaCerts does; sceSslFreeCaList accepts only that cleared list; sceSslUnloadCert has nothing to drop and is assumed to return 0. A null list returns the invalid-argument code that sceSslGetCaCerts and KytyPS5 return. An unknown context and a list sceSslGetCaList did not return throw, as their codes are not confirmed. Signatures and the SceSslCaList layout follow SharpProspero and shadPS4. guest_ssl_tests now links libc, as guest_http2_tests does, so an exception thrown from libSceSsl unwinds with the same unwinder that resumes it. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceSsl/CMakeLists.txt | 3 +- core/libs/prx/libSceSsl/Export.cpp | 31 ++++++++++++++++ core/libs/tests/GuestSsl.cpp | 49 ++++++++++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 83 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceSsl/CMakeLists.txt b/core/libs/prx/libSceSsl/CMakeLists.txt index b6cd45507..d66b9716e 100644 --- a/core/libs/prx/libSceSsl/CMakeLists.txt +++ b/core/libs/prx/libSceSsl/CMakeLists.txt @@ -5,7 +5,8 @@ configure_windows_unwind(libSceSsl) if(BUILD_TESTING) add_test_executable(guest_ssl_tests ${LIBS_INCLUDE_DIR}/tests/GuestSsl.cpp) target_include_directories(guest_ssl_tests PRIVATE ${LIBS_INCLUDE_DIR}) - target_link_libraries(guest_ssl_tests PRIVATE libSceSsl) + target_link_libraries(guest_ssl_tests PRIVATE libSceSsl libc) + configure_windows_unwind(guest_ssl_tests) add_test(NAME guest_ssl COMMAND guest_ssl_tests) if(WIN32) set_tests_properties(guest_ssl PROPERTIES ENVIRONMENT_MODIFICATION diff --git a/core/libs/prx/libSceSsl/Export.cpp b/core/libs/prx/libSceSsl/Export.cpp index cd9b00548..a1623ff2a 100644 --- a/core/libs/prx/libSceSsl/Export.cpp +++ b/core/libs/prx/libSceSsl/Export.cpp @@ -6,6 +6,7 @@ #include #include #include +#include // No network is emulated: contexts, templates and requests can be created, but any request // that would touch the network fails with the library's network error. @@ -37,6 +38,17 @@ struct SslCaCerts { void* pool; }; +struct SslCaList { + void** certs; + int num; +}; +static_assert(sizeof(SslCaList) == 16); + +void RequireContext(const char* function, int sslCtxId) { + std::lock_guard lock(g_poolsMutex); + if (!g_pools.contains(sslCtxId)) throw std::invalid_argument(std::string(function) + ": unknown context"); +} + } extern "C" { @@ -55,6 +67,20 @@ int APS5_VABI sceSslGetCaCerts(int ssl_ctx_id, void* ca_certs) { return ERROR_NOT_FOUND; } +int APS5_VABI sceSslGetCaList(int sslCtxId, SslCaList* caList) { + if (!caList) return ERROR_INVALID_ARG; + RequireContext(__func__, sslCtxId); + *caList = {}; + return ERROR_NOT_FOUND; +} + +int APS5_VABI sceSslFreeCaList(int sslCtxId, SslCaList* caList) { + if (!caList) return ERROR_INVALID_ARG; + RequireContext(__func__, sslCtxId); + if (caList->certs || caList->num) throw std::invalid_argument("sceSslFreeCaList: list not returned by sceSslGetCaList"); + return 0; +} + int APS5_VABI sceSslInit_nid_postfix(uint64_t pool_size) { const int id = g_nextHandle.fetch_add(1, std::memory_order_relaxed); std::lock_guard lock(g_poolsMutex); @@ -83,6 +109,11 @@ int APS5_VABI sceSslLoadCert() { return 0; } +int APS5_VABI sceSslUnloadCert(int sslCtxId) { + RequireContext(__func__, sslCtxId); + return 0; +} + int APS5_VABI sceSslGetMemoryPoolStats(int ssl_ctx_id, SslMemoryPoolStats* stats) { if (stats == nullptr) APS5_INVALID_ARG_EX; std::lock_guard lock(g_poolsMutex); diff --git a/core/libs/tests/GuestSsl.cpp b/core/libs/tests/GuestSsl.cpp index 327e6b0b5..a8bed9258 100644 --- a/core/libs/tests/GuestSsl.cpp +++ b/core/libs/tests/GuestSsl.cpp @@ -2,11 +2,16 @@ #include #include #include +#include extern "C" { int APS5_VABI sceSslInit_nid_postfix(std::size_t); +int APS5_VABI sceSslTerm_nid_postfix(int); int APS5_VABI sceSslGetCaCerts(int, void*); int APS5_VABI sceSslFreeCaCerts(int, void*); +int APS5_VABI sceSslGetCaList(int, void*); +int APS5_VABI sceSslFreeCaList(int, void*); +int APS5_VABI sceSslUnloadCert(int); } struct SslMemoryPoolStats { @@ -20,12 +25,27 @@ extern "C" int APS5_VABI sceSslGetMemoryPoolStats(int, SslMemoryPoolStats*); static void Require(bool value) { if (!value) std::abort(); } +template +static bool Throws(TCall call) { + try { + call(); + } catch (const std::exception&) { + return true; + } + return false; +} + struct SslCaCerts { void* certs; std::size_t num; void* pool; }; +struct SslCaList { + void** certs; + int num; +}; + int main() { constexpr int notFound = static_cast(0x8095F004); constexpr int invalidArg = static_cast(0x8095177A); @@ -46,4 +66,33 @@ int main() { certs = {&marker, 3, &marker}; Require(sceSslFreeCaCerts(context, &certs) == 0); Require(certs.certs == nullptr && certs.num == 0 && certs.pool == nullptr); + + Require(sceSslGetCaList(context, nullptr) == invalidArg); + Require(sceSslFreeCaList(context, nullptr) == invalidArg); + + void* handle = ▮ + SslCaList list{&handle, 3}; + Require(sceSslGetCaList(context, &list) == notFound); + Require(list.certs == nullptr && list.num == 0); + Require(sceSslFreeCaList(context, &list) == 0); + Require(list.certs == nullptr && list.num == 0); + + SslCaList foreign{&handle, 1}; + Require(Throws([&] { sceSslFreeCaList(context, &foreign); })); + foreign = {nullptr, 1}; + Require(Throws([&] { sceSslFreeCaList(context, &foreign); })); + foreign = {&handle, 0}; + Require(Throws([&] { sceSslFreeCaList(context, &foreign); })); + + Require(sceSslUnloadCert(context) == 0); + Require(sceSslGetCaList(context, &list) == notFound); + Require(list.certs == nullptr && list.num == 0); + + Require(sceSslTerm_nid_postfix(context) == 0); + Require(sceSslGetCaList(context, nullptr) == invalidArg); + Require(sceSslFreeCaList(context, nullptr) == invalidArg); + list = {&handle, 3}; + Require(Throws([&] { sceSslGetCaList(context, &list); })); + Require(Throws([&] { sceSslFreeCaList(context, &list); })); + Require(Throws([&] { sceSslUnloadCert(context); })); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..5952a0498 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -185,6 +185,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceKernelGetOperationMode](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - the signature `(int* mode, int* submode)` comes from PPSA12544, which logs both values after the call; the values are unknown and 0 is reported for both, as in the [prosper](https://github.com/mattias800/prosper/pull/4153) reimplementation. Null outputs throw - Font, Http2, Net, Ssl, SystemService and libkernel exports imported by PPSA12544 and added without an implementation have unknown signatures - [sceHttp2GetMemoryPoolStats, sceSslGetMemoryPoolStats, sceNetGetMemoryPoolStats](../../core/libs/prx/libSceNet/Export.cpp) (libSceHttp2, libSceSsl, libSceNet) - signatures `(int id, stats*)` and the statistics layouts (pool size, peak and current use, and for Http2 and Ssl a 32-bit reserved field) follow the PS4 SDK's `SceHttp2MemoryPoolStats`, `SceSslMemoryPoolStats` and `SceNetMemoryPoolStats`, not confirmed for the PS5. The pool size is the one given at init or pool creation; nothing is allocated from these pools, so the peak and current use are 0. Net returns `EBADF` for an unknown pool and `EINVAL` for a null output, as its other pool calls do; for Http2 and Ssl the error codes are unknown, so both cases throw +- [sceSslGetCaList, sceSslFreeCaList, sceSslUnloadCert](../../core/libs/prx/libSceSsl/Export.cpp) (libSceSsl) - signatures and the 16-byte `SceSslCaList` layout (an array of certificate handles and a 32-bit count) follow [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/5dfdb4b0b166e83dc9ae275001afb695f12c7174/src/SharpProspero/Interop/Net/Http.cs) and shadPS4's `OrbisSslCaList`, not confirmed for the PS5. On the console the list holds the certificate authorities the SSL service has; [shadPS4](https://github.com/shadps4-emu/shadPS4/pull/4104) returns a placeholder certificate from `sceSslGetCaCerts` because Rise of the Tomb Raider fails its initialization checks without one. No root certificate bundle is shipped and no loaded certificate is kept, so here a context holds none: `sceSslGetCaList` clears the list and returns `NOT_FOUND`, as `sceSslGetCaCerts` does, `sceSslFreeCaList` accepts only that cleared list, and `sceSslUnloadCert` (imported by PPSA15508) has nothing to drop and is assumed to return 0. A null list returns `0x8095177A`, which `sceSslGetCaCerts` and [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/59183dbfa81ed7efdff6069177fed50de0552725/src/libs/network.cpp) return for a null argument (shadPS4, PS4, returns `0x8095F007`). An unknown context throws (KytyPS5 returns `SSL_ERROR_INVALID_ID`, `0x8095F006`, from `sceSslGetCaCerts`), and so does a list `sceSslGetCaList` did not return - [sceHttp2SetResolveRetry](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) - signature `(int id, int32_t retry)` from libSceHttp's `sceHttpSetResolveRetry`; the value is accepted and ignored like the other Http2 settings, since no request reaches the network - [sceHttp2WebSocketCreateRequest, sceHttp2WebSocketCloseAsync, sceHttp2WebSocketSendTextMessageAsync, sceHttp2WebSocketSendDataMessageAsync](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) - imported by PPSA16064 and not called during its start-up; unknown signatures - [libSceVrSetupDialog](../../core/libs/prx/libSceVrSetupDialog/Export.cpp), and the Share, NpTrophy2, NpEntitlementAccess and WebBrowserDialog exports imported by PPSA12544's Unity plugins and added without an implementation, have unknown signatures From 875ff3f8d98e479268f84eec406c614529c4e436 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jan=20=C5=A0l=C3=A9gl?= Date: Fri, 9 Oct 2026 21:53:47 +0200 Subject: [PATCH 059/460] feat(libSceNet): declare sceNetShowNetstat PPSA11386 imports it; on main it is one of the title's three missing imports, with sceSslGetCaList and sceSslFreeCaList, and the title does not call it during its start-up. The signature is unknown, so it is declared without parameters and throws. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceNet/Export.cpp | 5 +++++ docs/dev/TechnicalDebt.md | 1 + 2 files changed, 6 insertions(+) diff --git a/core/libs/prx/libSceNet/Export.cpp b/core/libs/prx/libSceNet/Export.cpp index 5e9d23160..2bd2c3eb7 100644 --- a/core/libs/prx/libSceNet/Export.cpp +++ b/core/libs/prx/libSceNet/Export.cpp @@ -374,6 +374,11 @@ int* APS5_VABI sceNetErrnoLoc(void) { return errno_slot(); } +int APS5_VABI sceNetShowNetstat(void) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + int PollSockets(KernelSocketPoll::Entry* entries, int count, int timeoutMilliseconds) { std::vector> natives(static_cast(count)); { diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..e22f0f9ff 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -188,6 +188,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceHttp2SetResolveRetry](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) - signature `(int id, int32_t retry)` from libSceHttp's `sceHttpSetResolveRetry`; the value is accepted and ignored like the other Http2 settings, since no request reaches the network - [sceHttp2WebSocketCreateRequest, sceHttp2WebSocketCloseAsync, sceHttp2WebSocketSendTextMessageAsync, sceHttp2WebSocketSendDataMessageAsync](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) - imported by PPSA16064 and not called during its start-up; unknown signatures - [libSceVrSetupDialog](../../core/libs/prx/libSceVrSetupDialog/Export.cpp), and the Share, NpTrophy2, NpEntitlementAccess and WebBrowserDialog exports imported by PPSA12544's Unity plugins and added without an implementation, have unknown signatures +- [sceNetShowNetstat](../../core/libs/prx/libSceNet/Export.cpp) (libSceNet) - imported by PPSA11386 and not called during its start-up; unknown signature, declared without parameters - [sceAjmDecWVorbisCreateHeaderPacket, sceAjmDecWVorbisCreateSetupPacket](../../core/libs/prx/libSceAjm.native/Export.cpp) (libSceAjmWVorbis, implemented in libSceAjm.native) - unknown signatures - Acm, GameLiveStreaming, Net, NpEntitlementAccess, NpManager, NpWebApi2, VoiceQoS, WebBrowserDialog and libkernel exports imported by PPSA14632 and added without an implementation are declared without parameters - [libSceAudioPropagation](../../core/libs/prx/libSceAudioPropagation/Export.cpp) (libSceAudioPropagation) - signatures and struct descriptors recovered from PPSA21567's calls (#481), not from documentation. The system options layout is unknown (only checked for non-null); the fourth argument of `sceAudioPropagationSourceCalculateAudioPaths` is unknown; `sceAudioPropagationSourceGetAudioPath` is assumed to output an 8-byte path handle. The meaning of the `RenderInfo` word at +0x28 is unknown (PPSA21567 and PPSA21564 pass 2); only 2 is accepted. Both titles pass one `RenderInfo` per call: a count of 0 throws, and larger counts are handled element by element without proof. No error code is known, so invalid input throws instead of returning one From c2a159bab7d8ce4672ae29ee471eceb4234036c2 Mon Sep 17 00:00:00 2001 From: ThatsLeo Date: Sat, 10 Oct 2026 15:57:34 +0200 Subject: [PATCH 060/460] fix(build): link the Linux library tests as PIE The Linux loader takes the unnamed main image for guest code, as the relinker's ET_DYN output is. Fedora's GCC does not default to PIE, so the test executables were ET_EXEC and three loader tests failed. --- core/libs/CMakeLists.txt | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/core/libs/CMakeLists.txt b/core/libs/CMakeLists.txt index 32eb46fa5..91c9262aa 100644 --- a/core/libs/CMakeLists.txt +++ b/core/libs/CMakeLists.txt @@ -3,6 +3,12 @@ if(NOT WIN32) string(STRIP "${CMAKE_EXE_LINKER_FLAGS}" CMAKE_EXE_LINKER_FLAGS) endif() +if(CMAKE_SYSTEM_NAME STREQUAL "Linux") + include(CheckPIESupported) + check_pie_supported(LANGUAGES C CXX) + set(CMAKE_POSITION_INDEPENDENT_CODE ON) +endif() + add_executable(nid_patcher ${CMAKE_CURRENT_SOURCE_DIR}/nid/src/Sha1.cpp ${CMAKE_CURRENT_SOURCE_DIR}/nid/src/NidCompute.cpp From 3779ffa3757ea2461734898d48cdeb397671de52 Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Sat, 10 Oct 2026 04:15:23 +0900 Subject: [PATCH 061/460] fix(libkernel): accept a trailing slash in sceKernelMkdir sceKernelMkdir checked that the parent of the resolved path exists, but for "dir/" std::filesystem::path reports "dir" itself as the parent, so the call failed with ENOENT and created nothing, while FreeBSD's mkdir (and the POSIX mkdir export) accept it. Strip trailing separators first, and report ENOTDIR when the parent exists but is not a directory. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 5f74ddba8b346e6c552cf02554d47382708206f3) --- core/libs/prx/libkernel/File/src/Stdio.cpp | 7 +++++-- core/libs/tests/GuestKernelSync.cpp | 4 ++++ 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index 711b9f388..5db08e781 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -544,10 +544,13 @@ int APS5_VABI getdents_nid_postfix(int fd, char* buf, int nbytes) { int APS5_VABI sceKernelMkdir(const char* path, uint16_t mode) { (void)mode; if (path == nullptr) throw std::invalid_argument("sceKernelMkdir: path is null"); - const auto native = ResolvePath_nid_no_patch(path); + auto native = ResolvePath_nid_no_patch(path); + while (!native.has_filename() && native.has_relative_path()) native = native.parent_path(); std::error_code error; if (std::filesystem::exists(native, error)) return SceErrorFromErrno(GUEST_EEXIST); - if (!std::filesystem::exists(native.parent_path(), error)) return SceErrorFromErrno(GUEST_ENOENT); + const auto parent = std::filesystem::status(native.parent_path(), error); + if (!std::filesystem::exists(parent)) return SceErrorFromErrno(GUEST_ENOENT); + if (!std::filesystem::is_directory(parent)) return SceErrorFromErrno(GUEST_ENOTDIR); if (!std::filesystem::create_directory(native, error)) return SceErrorFromErrno(error.value() ? error.value() : GUEST_EIO); RecordWrittenPath_nid_no_patch(native); return 0; diff --git a/core/libs/tests/GuestKernelSync.cpp b/core/libs/tests/GuestKernelSync.cpp index 526218a99..9fc11185f 100644 --- a/core/libs/tests/GuestKernelSync.cpp +++ b/core/libs/tests/GuestKernelSync.cpp @@ -40,6 +40,10 @@ int main() { Require(sceKernelMkdir("kernel_sync_probe", 0777) == 0); Require(Recorded("kernel_sync_probe")); std::ofstream("kernel_sync_probe/existing.bin") << "existing"; + Require(sceKernelMkdir("kernel_sync_probe/nested/", 0777) == 0); + Require(std::filesystem::is_directory("kernel_sync_probe/nested") && Recorded("kernel_sync_probe/nested")); + Require(sceKernelMkdir("kernel_sync_probe/nested/", 0777) == static_cast(0x80020011u)); + Require(sceKernelMkdir("kernel_sync_probe/existing.bin/child", 0777) == static_cast(0x80020014u)); const int reader = sceKernelOpen("kernel_sync_probe/existing.bin", 0x0, 0); Require(reader >= 0 && sceKernelClose(reader) == 0); void* stream = fopen_nid_postfix("kernel_sync_probe/existing.bin", "r"); From 5837210c9fabb80d4ca7ddae6f4de1cbfc975d6a Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 12:53:58 +0000 Subject: [PATCH 062/460] fix(libkernel): preserve mkdir error handling for trailing paths --- core/libs/prx/libkernel/File/src/Stdio.cpp | 6 +++++- core/libs/tests/GuestKernelSync.cpp | 1 + 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index 5db08e781..fe4a412ba 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -547,8 +547,12 @@ int APS5_VABI sceKernelMkdir(const char* path, uint16_t mode) { auto native = ResolvePath_nid_no_patch(path); while (!native.has_filename() && native.has_relative_path()) native = native.parent_path(); std::error_code error; - if (std::filesystem::exists(native, error)) return SceErrorFromErrno(GUEST_EEXIST); + const auto status = std::filesystem::status(native, error); + if (error && error != std::errc::no_such_file_or_directory) return SceErrorFromErrno(error.value()); + if (std::filesystem::exists(status)) return SceErrorFromErrno(GUEST_EEXIST); + error.clear(); const auto parent = std::filesystem::status(native.parent_path(), error); + if (error) return SceErrorFromErrno(error.value()); if (!std::filesystem::exists(parent)) return SceErrorFromErrno(GUEST_ENOENT); if (!std::filesystem::is_directory(parent)) return SceErrorFromErrno(GUEST_ENOTDIR); if (!std::filesystem::create_directory(native, error)) return SceErrorFromErrno(error.value() ? error.value() : GUEST_EIO); diff --git a/core/libs/tests/GuestKernelSync.cpp b/core/libs/tests/GuestKernelSync.cpp index 9fc11185f..0bf3bb5fd 100644 --- a/core/libs/tests/GuestKernelSync.cpp +++ b/core/libs/tests/GuestKernelSync.cpp @@ -43,6 +43,7 @@ int main() { Require(sceKernelMkdir("kernel_sync_probe/nested/", 0777) == 0); Require(std::filesystem::is_directory("kernel_sync_probe/nested") && Recorded("kernel_sync_probe/nested")); Require(sceKernelMkdir("kernel_sync_probe/nested/", 0777) == static_cast(0x80020011u)); + Require(sceKernelMkdir("kernel_sync_probe/./", 0777) == static_cast(0x80020011u)); Require(sceKernelMkdir("kernel_sync_probe/existing.bin/child", 0777) == static_cast(0x80020014u)); const int reader = sceKernelOpen("kernel_sync_probe/existing.bin", 0x0, 0); Require(reader >= 0 && sceKernelClose(reader) == 0); From 734fe53f8219f356992ff2e49f27823ef2144142 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 14:50:50 +0000 Subject: [PATCH 063/460] test(libkernel): cover mkdir at guest root --- core/libs/tests/GuestKernelSync.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/core/libs/tests/GuestKernelSync.cpp b/core/libs/tests/GuestKernelSync.cpp index 0bf3bb5fd..4bfbdefd0 100644 --- a/core/libs/tests/GuestKernelSync.cpp +++ b/core/libs/tests/GuestKernelSync.cpp @@ -37,6 +37,7 @@ static bool Recorded(const char* guest) { } int main() { std::filesystem::remove_all("kernel_sync_probe"); + Require(sceKernelMkdir("/", 0777) == static_cast(0x80020011u)); Require(sceKernelMkdir("kernel_sync_probe", 0777) == 0); Require(Recorded("kernel_sync_probe")); std::ofstream("kernel_sync_probe/existing.bin") << "existing"; From 046680da48eac6dba5d5994679758ef77f0b1815 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 14:55:26 +0000 Subject: [PATCH 064/460] fix(libkernel): reject empty mkdir paths --- core/libs/prx/libkernel/File/src/Stdio.cpp | 1 + core/libs/tests/GuestKernelSync.cpp | 1 + 2 files changed, 2 insertions(+) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index fe4a412ba..17436b46d 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -544,6 +544,7 @@ int APS5_VABI getdents_nid_postfix(int fd, char* buf, int nbytes) { int APS5_VABI sceKernelMkdir(const char* path, uint16_t mode) { (void)mode; if (path == nullptr) throw std::invalid_argument("sceKernelMkdir: path is null"); + if (!*path) return SceErrorFromErrno(GUEST_ENOENT); auto native = ResolvePath_nid_no_patch(path); while (!native.has_filename() && native.has_relative_path()) native = native.parent_path(); std::error_code error; diff --git a/core/libs/tests/GuestKernelSync.cpp b/core/libs/tests/GuestKernelSync.cpp index 4bfbdefd0..5214ff65c 100644 --- a/core/libs/tests/GuestKernelSync.cpp +++ b/core/libs/tests/GuestKernelSync.cpp @@ -37,6 +37,7 @@ static bool Recorded(const char* guest) { } int main() { std::filesystem::remove_all("kernel_sync_probe"); + Require(sceKernelMkdir("", 0777) == static_cast(0x80020002u)); Require(sceKernelMkdir("/", 0777) == static_cast(0x80020011u)); Require(sceKernelMkdir("kernel_sync_probe", 0777) == 0); Require(Recorded("kernel_sync_probe")); From fccfbee72b345129789ec51c6b0adcc49f933b2b Mon Sep 17 00:00:00 2001 From: AlxcBTW <187763517+AlxcBTW@users.noreply.github.com> Date: Sat, 10 Oct 2026 14:33:50 +0200 Subject: [PATCH 065/460] fix(relinker): convert Windows guest TLS reads after zero alignment padding The Windows TLS rewriter only patches FS-relative instructions that CodeInstructionCollector decodes. Code that no symbol, unwind range or root reaches is covered by a linear sweep that starts after int3 padding and stops at the first instruction beginning with two zero bytes, so zero-filled bytes are not read as code. The .NET NativeAOT homebrew titles pad between functions with zero bytes instead of int3 and have unwind entries for only part of their code. In Prospero Vibrate (PPSA99920) the sweep ended at the padding after the first function. Functions reached only through switch jump tables, which the control-flow graph does not follow, or through pointers it cannot resolve kept 105 of 670 FS loads unpatched: 84 `mov reg, fs:[0x28]` stack-guard reads (65 into rax, 19 into rcx) and 21 `mov rax, fs:[0]` thread-pointer reads. The title faulted reading address 0x28 within a second of start. The sweep now treats a zero run that follows a jump, call, return or trap and ends at the next 16-byte boundary as alignment padding, and continues at that boundary. A call counts because a function can end in a call to a noreturn function. Zero fill after straight-line code, or a whole zero 16-byte block, still ends the sweep as before. All 670 FS loads in PPSA99920 are now patched, as are the 102 of 685 in Prospero Explorer (PPSA99109) and the 103 of 718 in Prospero Multi Tools (PPSA99110). New tls_function_coverage padding cases: loads behind zero padding after a return, a call and a jump, with 13, 2, 1 and 5 zero bytes, are patched; a load behind zero fill after a nop is left unchanged. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 0306c677784b6c4272990ce65f6daf9fb2dbc3ca) --- .../src/analysis/CodeInstructionCollector.cpp | 9 +++++++++ .../tests/test_tls_function_coverage.py | 18 ++++++++++++++++++ 2 files changed, 27 insertions(+) diff --git a/core/relinker/relinker/src/analysis/CodeInstructionCollector.cpp b/core/relinker/relinker/src/analysis/CodeInstructionCollector.cpp index e3a6f2201..495395106 100644 --- a/core/relinker/relinker/src/analysis/CodeInstructionCollector.cpp +++ b/core/relinker/relinker/src/analysis/CodeInstructionCollector.cpp @@ -229,7 +229,15 @@ std::set CodeInstructionCollector::Collect(const std::ve while (address < size && !paddedStart(address)) address += 16; return address; }; + bool afterTransfer = false; for (std::size_t position = nextPaddedStart(0); position < size;) { + const auto aligned = (position + 15) & ~std::size_t{15}; + const bool zeroPadding = afterTransfer && aligned != position && aligned <= size && std::all_of(text + position, text + aligned, [](const std::uint8_t value) { return value == 0; }); + afterTransfer = false; + if (zeroPadding) { + position = aligned; + continue; + } if (position + 1 < size && text[position] == 0 && text[position + 1] == 0) { position = nextPaddedStart(position); continue; @@ -253,6 +261,7 @@ std::set CodeInstructionCollector::Collect(const std::ve } const auto opcode = position + info.OpcodeOffset; if (info.SegmentPrefix == 0x64 && opcode + 2 < next && (text[opcode + 1] & 0xc7) == 0x04 && text[opcode + 2] == 0x25) addRoot(header.MappedAddress + position); + afterTransfer = info.FlowKind != Codegen::ControlFlowKind::Sequential && info.FlowKind != Codegen::ControlFlowKind::ConditionalBranch; position = next; } } diff --git a/core/relinker/relinker/tests/test_tls_function_coverage.py b/core/relinker/relinker/tests/test_tls_function_coverage.py index cd2bb8148..5abf3e195 100644 --- a/core/relinker/relinker/tests/test_tls_function_coverage.py +++ b/core/relinker/relinker/tests/test_tls_function_coverage.py @@ -619,6 +619,24 @@ def main(): zero_filled[0x1910:0x1910 + len(TLS_LOAD)] = TLS_LOAD pe = convert_padded("zero-filled-after-padding", zero_filled) assert pe_bytes_at(pe, 0x11910, len(TLS_LOAD)) == TLS_LOAD, "zero-filled-after-padding" + stack_guard = bytes.fromhex("64 48 8b 04 25 28 00 00 00") + zero_aligned = make_image("register", "unwind") + zero_aligned[0x18f0:0x1900] = b"\xcc" * 16 + zero_aligned[0x1900:0x1910] = bytes.fromhex("31 c0 c3") + bytes(13) + zero_aligned[0x1910:0x1920] = stack_guard + b"\xe8" + struct.pack(" Date: Sat, 10 Oct 2026 14:41:37 +0300 Subject: [PATCH 066/460] fix(libkernel): accept a zero count in sceKernelSignalSema sceKernelSignalSema threw invalid_argument for a count of 0. Marvel's Wolverine (PPSA03671) signals with a count computed as min(a, b) - c, which can be 0, and ignores the result, so the throw ended the title a few minutes in. A zero count now adds no tokens and returns SCE_OK, as in shadPS4 and Kyty. A null semaphore and a negative count still throw. guest_kernel_errors signals 0 on a full and on an empty semaphore and checks that the count is unchanged. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit 7191d34c4d79622ba71087f0da2b75eb81ad2c23) --- core/libs/prx/libkernel/Semaphore/src/Semaphore.cpp | 2 +- core/libs/tests/GuestKernelErrors.cpp | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libkernel/Semaphore/src/Semaphore.cpp b/core/libs/prx/libkernel/Semaphore/src/Semaphore.cpp index 46490e9a7..8210cdb99 100644 --- a/core/libs/prx/libkernel/Semaphore/src/Semaphore.cpp +++ b/core/libs/prx/libkernel/Semaphore/src/Semaphore.cpp @@ -36,7 +36,7 @@ int APS5_VABI sceKernelPollSema(KernelSema sem, int need) { } int APS5_VABI sceKernelSignalSema(KernelSema sem, int count) { - if (sem == nullptr || count <= 0) { + if (sem == nullptr || count < 0) { APS5_INVALID_ARG_EX; } diff --git a/core/libs/tests/GuestKernelErrors.cpp b/core/libs/tests/GuestKernelErrors.cpp index ddb8828a5..5c14f7fca 100644 --- a/core/libs/tests/GuestKernelErrors.cpp +++ b/core/libs/tests/GuestKernelErrors.cpp @@ -62,7 +62,10 @@ int main() { Require(sceKernelCreateSema(&semaphore, "limit", 1, 1, 3, nullptr) == SCE_OK); Require(sceKernelSignalSema(semaphore, 3) == SCE_KERNEL_ERROR_EINVAL); Require(sceKernelSignalSema(semaphore, 2) == SCE_OK); + Require(sceKernelSignalSema(semaphore, 0) == SCE_OK); Require(sceKernelPollSema(semaphore, 3) == SCE_OK); + Require(sceKernelSignalSema(semaphore, 0) == SCE_OK); + Require(sceKernelPollSema(semaphore, 1) == SCE_KERNEL_ERROR_EBUSY); Require(sceKernelDeleteSema(semaphore) == SCE_OK); KernelEqueue eq = 0; From d686f7057c9b1afdb074269acabcc38f88f7fe51 Mon Sep 17 00:00:00 2001 From: Get-fix <100056769+Get-fix@users.noreply.github.com> Date: Sat, 10 Oct 2026 19:12:49 +0300 Subject: [PATCH 067/460] fix(libkernel): export execvp VLC (PPSA85300) imports execvp (0NJDcQ2B8vE) from libkernel and fails to load because no library exports it. Replacing the process image is not supported, so it throws if called, like sceSystemServiceLoadExec does for executable replacement. VLC itself cannot reach the call: its fork and vfork fail with ENOSYS, so the child that would call execvp is never created. --- core/libs/prx/libkernel/CMakeLists.txt | 9 ++++++++ .../libs/prx/libkernel/System/src/Process.cpp | 7 ++++++ core/libs/tests/GuestExecvp.cpp | 23 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 40 insertions(+) create mode 100644 core/libs/tests/GuestExecvp.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index 6938ffc9a..ee16d7fb8 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -578,6 +578,15 @@ if(BUILD_TESTING) set_tests_properties(guest_waitpid PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_execvp_tests ${LIBS_INCLUDE_DIR}/tests/GuestExecvp.cpp) + target_include_directories(guest_execvp_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_execvp_tests PRIVATE libkernel libc) + configure_windows_unwind(guest_execvp_tests) + add_test(NAME guest_execvp COMMAND guest_execvp_tests) + if(WIN32) + set_tests_properties(guest_execvp PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_kernel_error_tests ${LIBS_INCLUDE_DIR}/tests/GuestKernelErrors.cpp) target_include_directories(guest_kernel_error_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_kernel_error_tests PRIVATE libkernel libc) diff --git a/core/libs/prx/libkernel/System/src/Process.cpp b/core/libs/prx/libkernel/System/src/Process.cpp index 4df88ebd6..6852d719b 100644 --- a/core/libs/prx/libkernel/System/src/Process.cpp +++ b/core/libs/prx/libkernel/System/src/Process.cpp @@ -237,6 +237,13 @@ int APS5_VABI waitpid_nid_postfix(int pid, int* status, int options) { return -1; } +int APS5_VABI execvp_nid_postfix(const char* file, char* const* arguments) { + (void)file; + (void)arguments; + NotImplemented_nid_no_patch("execvp: executable replacement"); + return -1; +} + int APS5_VABI sceKernelGetCurrentCpu(void) { #ifdef _WIN32 PROCESSOR_NUMBER processor{}; diff --git a/core/libs/tests/GuestExecvp.cpp b/core/libs/tests/GuestExecvp.cpp new file mode 100644 index 000000000..ebf28b63a --- /dev/null +++ b/core/libs/tests/GuestExecvp.cpp @@ -0,0 +1,23 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +extern "C" int APS5_VABI execvp_nid_postfix(const char*, char* const*); +static void Require(bool value) { if (!value) std::abort(); } +static bool ReportsNotImplemented(const char* file, char* const* arguments) { + try { + execvp_nid_postfix(file, arguments); + } catch (const std::runtime_error& error) { + return std::string(error.what()) == "execvp: executable replacement not implemented"; + } + return false; +} +int main() { + char shell[] = "sh"; + char option[] = "-c"; + char command[] = "exit 0"; + char* arguments[] = {shell, option, command, nullptr}; + Require(ReportsNotImplemented("sh", arguments)); + Require(ReportsNotImplemented("/bin/sh", arguments)); + Require(ReportsNotImplemented(nullptr, nullptr)); +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..7b6edfe4e 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -327,6 +327,7 @@ Throughout the project, every function at every stage either **does exactly what - [MIMG `tfe` and `lwe`](../../core/shader/recompiler/RdnaDecoder/src/RdnaImageOpDecoder.cpp) never write the status VGPR after the data. On hardware it is written only when a texel fetch hits an unmapped page of a partially resident texture, or, for `lwe` on sample/gather, when the LOD is below the T# `MIN_LOD_WARN`; texture memory is always resident here, the LOD warning (texture-streaming feedback, like the T# `MIN_LOD_WARN` itself) is never reported, and `lwe` on `image_get_lod` throws. - [Packed horizontal gathers](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_gather4h_pck`, MIMG opcode 98): the four window texels are read raw and concatenated into a little-endian bitstream, with DMASK bit i writing stream word i to the next destination register; measured on RDNA2 (gfx1031) at every edge: the anchor is floor(x-0.5) with elements at -1, 0, +1 and +2 clamped horizontally into the width, the row is floor(y), negative rows clamp to row 0, rows at or past the height return zeros, sampler addressing modes do not affect the window, and formats wider than 32 bits per texel write zeros in every register. The emitter fetches each window texel at LOD 0 and packs its bits like `image_load_pck`, so the sampler's filter, addressing and LOD state are not used; formats whose bits are not recoverable from the view and D16 data throw. The anchor and row multiply the normalized coordinate by the image size without contraction. The stream rule was measured for R8_UNORM, RG8_UNORM and RGBA8; only RGBA8 has an execution test. `image_gather8h_pck` (opcode 99) never returns texel data on this hardware - across dmasks 0x1 to 0xFF including the upper nibble, RGBA8, RGBA16 and RGBA32 formats, 1D, 2D and 3D descriptors and normalized and unnormalized sampling it only zero-fills popcount(dmask) registers (sentinel-prefilled destinations) - so it decodes but throws at translation; its eight-texel semantics remain unmeasured. - [waitpid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) returns -1 with `ECHILD` (`EINVAL` for unknown options): nothing here creates a child process; child processes would need a real `waitpid`. +- [execvp](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) throws: replacing the process image is not supported. PPSA85300 imports it and fails to load without it; its `fork` and `vfork` fail with `ENOSYS` ([libc_missing.c](https://github.com/ZiZc3/VLC/blob/f3253020a64368153233ede29183014c0b6c8979/ps5/compat/libc_missing.c#L290-L292)), so the child process that would call it is never created. - [MIMG BY2/BY4 loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_load[_mip]_by2/by4`, opcodes 0x42, 0x43, 0x4a and 0x4b) were measured on RDNA2 (gfx1035) with the [hardware oracle](HW_ORACLE.md) for 2D images with identity swizzle. They read N texels (BY2: 2, BY4: 4) from `x & ~(N - 1)` on row y, texel by texel and component by component within each texel, with each texel converted like `image_load`. Every register is zero when the mip level is past the last level, `x + N - 1` is outside the level's width (unaligned x, unsigned), or y is outside its height. Data only comes back when the N texels fit in one dword: R8 and R16 (DMASK 0x3) or RG8 (0xf) for BY2, and R8 (0xf) for BY4. With 32-bit texels (R32, RG32, RG16, RGBA8) the hardware returns zeros. Other formats, masks, dimensions, swizzles, D16/A16 and indirect descriptors throw. The stores (0x52, 0x53, 0x5a and 0x5b) decode but throw at translation. Measured on hardware, they write from the unaligned x up to the end of its dword, which is not implemented. - [2D stores to 1D storage images](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp): an `image_store` with DIM 2D through a 1D T# writes texel x and ignores y, as measured on RDNA2 (gfx1035) for y from 0 to 4 and 0xffffffff, single- and four-component formats and a 1x1 8_8_8_8_UNORM T#: the T# type decides the addressing, DIM only how many address VGPRs are read. Storage loads, atomics and `image_store_mip` through such a pair still throw - [MIMG PCK2/PCK4 loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_load[_mip]_pck2/pck4`, opcodes 0x70, 0x71, 0x73 and 0x74) were measured on RDNA2 (gfx1035) with the [hardware oracle](HW_ORACLE.md) for 2D images with identity swizzle. With data mask 0x1, they write one dword: the raw bits of the N texels (PCK2: 2, PCK4: 4) from `x & ~(N - 1)` on row y, concatenated (element k at bit `k * texel bits`) and zero-extended. The dword is zero when the mip level is past the last level, `x + N - 1` is outside the level's width (unaligned x, unsigned), or y is outside its height. Data only comes back when N texels fit in 32 bits: R8, R16 or RG8 for PCK2, R8 for PCK4. Wider texels (R32, RG16, RGBA8) and masks other than 0x1 write zeros on hardware. Those masks throw at decode, and formats outside UNORM/UINT/SINT R8, R16 and RG8 throw. #1314's earlier measurement saw only zeros because it used masks 0x3 and 0xf. The stores (0x76, 0x77, 0x79 and 0x7a) decode but throw at translation. They are not measured, and opcode 0x76 on 32_32_32_32_FLOAT hung the GPU (#1314). From 7fe298bfab990cd83a8bfc56e82976a23ce755f0 Mon Sep 17 00:00:00 2001 From: MathiasDevelopes <25400788+MathiasDevelopes@users.noreply.github.com> Date: Sat, 10 Oct 2026 18:15:59 +0200 Subject: [PATCH 068/460] fix(libkernel): return ENOENT from sceKernelStat for unconvertible paths A guest path whose bytes the host cannot convert to a native path made ResolvePath throw std::filesystem::filesystem_error, which aborted the process. No file can have such a name on the host, so stat reports ENOENT, as for any other missing file. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libkernel/File/src/Open.cpp | 7 ++++++- core/libs/tests/GuestFilesystem.cpp | 2 ++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libkernel/File/src/Open.cpp b/core/libs/prx/libkernel/File/src/Open.cpp index 329c2a55d..b45fa864e 100644 --- a/core/libs/prx/libkernel/File/src/Open.cpp +++ b/core/libs/prx/libkernel/File/src/Open.cpp @@ -199,7 +199,12 @@ int APS5_VABI sceKernelStat(const char* path, FileStat* sb) { if (sb == nullptr) { throw std::invalid_argument(std::string(__func__) + ": sb is null"); } - const auto native = ResolvePath_nid_no_patch(path); + std::filesystem::path native; + try { + native = ResolvePath_nid_no_patch(path); + } catch (const std::filesystem::filesystem_error&) { + return SceErrorFromErrno(2); + } std::error_code error; constexpr int GuestEnotdir = 20; const auto status = std::filesystem::status(native, error); diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index bd0f598b7..1265d59fc 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -187,6 +187,8 @@ int main() { Require(stat_nid_postfix("", &status) == -1 && *__error_nid_postfix() == 2); Require(stat_nid_postfix((presentName + "/").c_str(), &status) == -1 && *__error_nid_postfix() == 20); Require(sceKernelStat((presentName + "/").c_str(), &status) == static_cast(0x80020014u)); + Require(sceKernelStat("\xff\xfe", &status) == static_cast(0x80020002u)); + Require(stat_nid_postfix("\xff\xfe", &status) == -1 && *__error_nid_postfix() == 2); FileStat dirStatus{}; Require(stat_nid_postfix((rootName + "/").c_str(), &dirStatus) == 0 && (dirStatus.st_mode & 0170000) == 0040000); Require(stat_nid_postfix(nullptr, &status) == -1 && *__error_nid_postfix() == 14); From ef5221a1e6c7b9ae686320c9e3be8689cedd0b05 Mon Sep 17 00:00:00 2001 From: Lexroid Date: Sat, 10 Oct 2026 00:41:50 +0300 Subject: [PATCH 069/460] fix(recompiler): accept byte source selects on v_cvt_f16_u16 SDWA v_cvt_f16_u16 SDWA only accepted word and dword source selects. Silent Hill Townfall (PPSA33286) converts the top byte into the low half of a register and keeps the high half (src0_sel:BYTE_3 dst_sel:WORD_0 dst_unused:UNUSED_PRESERVE), so its shaders failed to recompile. The generic SDWA source path already zero-extends byte selects, which is the u16 value the conversion expects, so the rule now accepts every source select, with whole and partial destinations. agc_driver_sdwa_cvt_f16_u16 runs four byte selects with dword, WORD_0 and WORD_1 destinations on the GPU and checks the converted halves and the preserved bits. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit d55dfe8b33c6ee418f40a92e49236f8679a6470b) --- .../tests/execution/SdwaCvtF16U16.cpp | 109 ++++++++++++++++++ .../RdnaDecoder/src/RdnaVectorOpDecoder.cpp | 2 +- 2 files changed, 110 insertions(+), 1 deletion(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/SdwaCvtF16U16.cpp diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/SdwaCvtF16U16.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/SdwaCvtF16U16.cpp new file mode 100644 index 000000000..6bd183717 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/SdwaCvtF16U16.cpp @@ -0,0 +1,109 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 64; +constexpr std::uint32_t Inputs = 4; +constexpr std::uint32_t Results = 4; +constexpr std::uint32_t Garbage = 0xabcd1234u; +alignas(256) std::array Input{}; +alignas(256) std::array Output{}; + +alignas(256) constexpr std::array SdwaCode{ + 0x34020082, 0x34060082, 0xe0302000, 0x80000401, + 0x7e1402ff, 0xabcd1234, 0x7e1602ff, 0xabcd1234, 0x7e1802ff, 0xabcd1234, 0x7e1a02ff, 0xabcd1234, + 0xbf8c3f70, 0x7e14a0f9, 0x00031404, 0x7e16a0f9, 0x00000604, 0x7e18a0f9, 0x00011504, 0x7e1aa0f9, 0x00020604, + 0xe0702000, 0x80010a03, 0xe0702004, 0x80010b03, 0xe0702008, 0x80010c03, 0xe070200c, 0x80010d03, + 0xbf810000, +}; + +std::uint32_t HalfBits(std::uint32_t value) { + if (value == 0u) return 0u; + int exponent = 0; + const float mantissa = std::frexp(static_cast(value), &exponent); + const auto fraction = static_cast(std::ldexp(mantissa, 11)); + return (static_cast(exponent + 14) << 10u) | (fraction & 0x3ffu); +} + +void FillInput() { + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + Input[tid * Inputs] = tid | ((tid + 64u) << 8u) | ((255u - tid) << 16u) | (((tid * 3u + 128u) & 0xffu) << 24u); + } +} + +std::string Hex(std::uint32_t value) { + char text[16]; + std::snprintf(text, sizeof(text), "0x%08x", value); + return text; +} + +std::array BufferDescriptor(const void* data, std::uint32_t count) { + const auto address = reinterpret_cast(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu) | (4u << 16u), count, 0x11016facu}; +} + +void Run(AgcDriver::VulkanDevice& device) { + Output.fill(0xdeadbeefu); + std::vector userData(8, 0u); + const auto input = BufferDescriptor(Input.data(), static_cast(Input.size())); + const auto output = BufferDescriptor(Output.data(), static_cast(Output.size())); + std::copy(input.begin(), input.end(), userData.begin()); + std::copy(output.begin(), output.end(), userData.begin() + 4); + const std::span code(SdwaCode); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0, {false, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); +} + +void Check() { + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + const auto source = Input[tid * Inputs]; + const auto* out = &Output[tid * Results]; + const auto half = [&](std::uint32_t byte) { return HalfBits((source >> (byte * 8u)) & 0xffu); }; + const auto where = [&](std::uint32_t result) { return "sdwa cvt f16 u16: thread " + std::to_string(tid) + " result " + std::to_string(result) + " is " + Hex(out[result]); }; + const auto preserved = (Garbage & 0xffff0000u) | half(3u); + Require(out[0] == preserved, where(0) + ", expected " + Hex(preserved) + ": src0_sel:BYTE_3 dst_sel:WORD_0 dst_unused:UNUSED_PRESERVE must convert the top byte and keep the high half"); + Require(out[1] == half(0u), where(1) + ", expected " + Hex(half(0u)) + ": src0_sel:BYTE_0 dst_sel:DWORD must convert the low byte"); + const auto high = (half(1u) << 16u) | (Garbage & 0xffffu); + Require(out[2] == high, where(2) + ", expected " + Hex(high) + ": src0_sel:BYTE_1 dst_sel:WORD_1 dst_unused:UNUSED_PRESERVE must keep the low half"); + Require(out[3] == half(2u), where(3) + ", expected " + Hex(half(2u)) + ": src0_sel:BYTE_2 dst_sel:DWORD must convert the third byte"); + } +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + FillInput(); + Run(*device); + Check(); + std::puts("sdwa cvt f16 u16 tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/RdnaDecoder/src/RdnaVectorOpDecoder.cpp b/core/shader/recompiler/RdnaDecoder/src/RdnaVectorOpDecoder.cpp index 521e629c2..cd3a2bba3 100644 --- a/core/shader/recompiler/RdnaDecoder/src/RdnaVectorOpDecoder.cpp +++ b/core/shader/recompiler/RdnaDecoder/src/RdnaVectorOpDecoder.cpp @@ -1126,7 +1126,7 @@ constexpr Vop1SdwaRule vop1SdwaRules[] = { {RdnaOpcode::VBfrevB32, sdwaSelAll(), 0, 0, false}, {RdnaOpcode::VCvtF32F16, sdwaSelWords() | sdwaSelFull(), 0, 0, true}, {RdnaOpcode::VCvtF16F32, sdwaSelAll(), sdwaSelWords(), sdwaSelAll(), true}, - {RdnaOpcode::VCvtF16U16, sdwaSelWords() | sdwaSelFull(), sdwaSelWords(), sdwaSelWords() | sdwaSelFull(), false}, + {RdnaOpcode::VCvtF16U16, sdwaSelAll(), sdwaSelWords(), sdwaSelAll(), false}, {RdnaOpcode::VCvtU16F16, sdwaSelWords() | sdwaSelFull(), sdwaSelWords(), sdwaSelWords() | sdwaSelFull(), true}, {RdnaOpcode::VCvtF16I16, sdwaSelWords() | sdwaSelFull(), sdwaSelWords(), sdwaSelWords() | sdwaSelFull(), false}, {RdnaOpcode::VCvtI16F16, sdwaSelWords() | sdwaSelFull(), sdwaSelWords(), sdwaSelWords() | sdwaSelFull(), true}, From a68dbbe11234847cfbb5375cf653b8d154365100 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Fri, 9 Oct 2026 02:04:47 -0400 Subject: [PATCH 070/460] fix(shader): detect f32 tininess after rounding when flushing products With f32 denormals flushed, RDNA2 flushes a product only when its value rounded to 24 bits in the current round mode is below 2^-126. The tiny-product check tested the exact product, so products that round up to 2^-126 became zero in v_mul_f32, v_mul_legacy_f32, v_mullit_f32 and zero-addend f32 FMAs. The separately rounded v_mad_legacy_f32 and v_mac_legacy_f32 keep the exact check, as they are not measured there. Co-Authored-By: Claude Opus 5.5 (1M context) --- .../tests/execution/F32DenormalFlush.cpp | 170 +++++++++++++++++- .../Translation/TranslationContext.hpp | 2 +- .../Translation/src/FloatInstructions.cpp | 2 +- .../src/TranslationContext/OperandAccess.cpp | 24 ++- docs/dev/TechnicalDebt.md | 2 +- 5 files changed, 187 insertions(+), 13 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/F32DenormalFlush.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/F32DenormalFlush.cpp index 63a393720..3c61af0cb 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/F32DenormalFlush.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/F32DenormalFlush.cpp @@ -22,6 +22,7 @@ using ShaderRecompiler::ShaderStage; constexpr std::uint32_t Threads = 32; constexpr std::uint32_t Inputs = 4; constexpr std::uint32_t Results = 19; +constexpr std::uint32_t TinyResults = 16; alignas(256) std::array Input{}; alignas(256) std::array Output{}; @@ -124,9 +125,133 @@ constexpr const char* Names[19] = { "v_mullit_f32 v28, v4, v5, 1.0" }; -void Fill(std::uint32_t tid, std::uint32_t* words) { - std::copy(std::begin(Rows[tid]), std::end(Rows[tid]), words); -} +alignas(256) constexpr std::array TinyCode{ + 0x34020084u, 0x34040086u, 0xe0381000u, 0x80000401u, 0xbf8c3f70u, 0x10140b04u, 0x0e160b04u, 0xd54b000cu, + 0x041a0b04u, 0x7e1a0306u, 0x561a0b04u, 0x581c0d04u, 0x3f7ffffeu, 0x5a1e0b04u, 0x80000000u, 0xcc200010u, + 0x041a0b04u, 0xd5080011u, 0x22020b04u, 0xd5400012u, 0x041a0b04u, 0x7e260306u, 0x0c260b04u, 0xe0781000u, + 0x80010a02u, 0xe0781010u, 0x80010e02u, 0xe0741020u, 0x80011202u, 0xbf810000u, +}; + +constexpr std::uint32_t TinyRows[32][4] = { + {0x00800001u, 0x3f7ffffeu, 0x00000000u, 0x00000000u}, + {0x80800001u, 0x3f7ffffeu, 0x80000000u, 0x00000000u}, + {0x3f7ffffeu, 0x00800001u, 0x00000000u, 0x00000000u}, + {0x008003e6u, 0x3f7ff834u, 0x00000000u, 0x00000000u}, + {0x00f80000u, 0x3f042108u, 0x80000000u, 0x00000000u}, + {0x80f80000u, 0x3f042108u, 0x00000000u, 0x00000000u}, + {0x00800f95u, 0x3f7fe0d9u, 0x00000000u, 0x00000000u}, + {0x00800001u, 0x3f7ffffdu, 0x00000000u, 0x00000000u}, + {0x00ffffffu, 0x3f000000u, 0x00000000u, 0x00000000u}, + {0x008005a8u, 0x3f7ff4b0u, 0x00000000u, 0x00000000u}, + {0x00f7ffffu, 0x3f042108u, 0x00000000u, 0x00000000u}, + {0x00ffffffu, 0x3effffffu, 0x00000000u, 0x00000000u}, + {0x008005a9u, 0x3f7ff4aeu, 0x00000000u, 0x00000000u}, + {0x00800001u, 0x3f7ffffeu, 0x00000001u, 0x00000000u}, + {0x007fffffu, 0x3f800001u, 0x00000000u, 0x00000000u}, + {0x00800000u, 0x3f7fffffu, 0x00000000u, 0x00000000u}, + {0x80800001u, 0x3f7ffffeu, 0x80000000u, 0x00000000u}, + {0x00800001u, 0x3f7ffffeu, 0x00000000u, 0x00000000u}, + {0xbf7ffffeu, 0x00800001u, 0x80000000u, 0x00000000u}, + {0x808003e6u, 0x3f7ff834u, 0x80000000u, 0x00000000u}, + {0x80f80000u, 0x3f042108u, 0x00000000u, 0x00000000u}, + {0x00f80000u, 0x3f042108u, 0x80000000u, 0x00000000u}, + {0x80800f95u, 0x3f7fe0d9u, 0x80000000u, 0x00000000u}, + {0x80800001u, 0x3f7ffffdu, 0x80000000u, 0x00000000u}, + {0x80ffffffu, 0x3f000000u, 0x80000000u, 0x00000000u}, + {0x808005a8u, 0x3f7ff4b0u, 0x80000000u, 0x00000000u}, + {0x80f7ffffu, 0x3f042108u, 0x80000000u, 0x00000000u}, + {0x80ffffffu, 0x3effffffu, 0x80000000u, 0x00000000u}, + {0x808005a9u, 0x3f7ff4aeu, 0x80000000u, 0x00000000u}, + {0x80800001u, 0x3f7ffffeu, 0x80000001u, 0x00000000u}, + {0x807fffffu, 0x3f800001u, 0x80000000u, 0x00000000u}, + {0x80800000u, 0x3f7fffffu, 0x80000000u, 0x00000000u} +}; +constexpr std::uint32_t TinyExpected[32][8] = { + {0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x80800000u}, + {0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x00800000u}, + {0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x3f7ffffcu, 0x00800000u, 0x00800000u, 0x80800000u}, + {0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x008003e5u, 0x00800000u, 0x00800000u, 0x80800000u}, + {0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x00f7fffeu, 0x00800000u, 0x00800000u, 0x80800000u}, + {0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x80f7fffeu, 0x80800000u, 0x80800000u, 0x00800000u}, + {0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00800f94u, 0x00000000u, 0x00000000u, 0x80000000u}, + {0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00800000u, 0x00000000u, 0x00000000u, 0x80000000u}, + {0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00fffffdu, 0x00000000u, 0x00000000u, 0x80000000u}, + {0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x008005a7u, 0x00800000u, 0x00800000u, 0x80800000u}, + {0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00f7fffdu, 0x00000000u, 0x00000000u, 0x80000000u}, + {0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00fffffdu, 0x00000000u, 0x00000000u, 0x80000000u}, + {0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x008005a8u, 0x00000000u, 0x00000000u, 0x80000000u}, + {0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x80800000u}, + {0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x80000000u}, + {0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x80000000u}, + {0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x00800000u}, + {0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x80800000u}, + {0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0xbf7ffffcu, 0x80800000u, 0x80800000u, 0x00800000u}, + {0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x808003e5u, 0x80800000u, 0x80800000u, 0x00800000u}, + {0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x80f7fffeu, 0x80800000u, 0x80800000u, 0x00800000u}, + {0x00800000u, 0x00800000u, 0x00800000u, 0x00800000u, 0x00f7fffeu, 0x00800000u, 0x00800000u, 0x80800000u}, + {0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80800f94u, 0x80000000u, 0x80000000u, 0x00000000u}, + {0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80800000u, 0x80000000u, 0x80000000u, 0x00000000u}, + {0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80fffffdu, 0x80000000u, 0x80000000u, 0x00000000u}, + {0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x808005a7u, 0x80800000u, 0x80800000u, 0x00800000u}, + {0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80f7fffdu, 0x80000000u, 0x80000000u, 0x00000000u}, + {0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80fffffdu, 0x80000000u, 0x80000000u, 0x00000000u}, + {0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x808005a8u, 0x80000000u, 0x80000000u, 0x00000000u}, + {0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x80800000u, 0x00800000u}, + {0x80000000u, 0x00000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x00000000u}, + {0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x00000000u} +}; +constexpr std::uint32_t TinyDirected[3][32][2] = { + { + {0x00800000u, 0x80000000u}, {0x80000000u, 0x00800000u}, {0x00800000u, 0x80000000u}, {0x00800000u, 0x80000000u}, + {0x00800000u, 0x80000000u}, {0x80000000u, 0x00800000u}, {0x00800000u, 0x80000000u}, {0x00000000u, 0x80000000u}, + {0x00000000u, 0x80000000u}, {0x00800000u, 0x80000000u}, {0x00000000u, 0x80000000u}, {0x00000000u, 0x80000000u}, + {0x00800000u, 0x80000000u}, {0x00800000u, 0x80000000u}, {0x00000000u, 0x80000000u}, {0x00000000u, 0x80000000u}, + {0x80000000u, 0x00800000u}, {0x00800000u, 0x80000000u}, {0x80000000u, 0x00800000u}, {0x80000000u, 0x00800000u}, + {0x80000000u, 0x00800000u}, {0x00800000u, 0x80000000u}, {0x80000000u, 0x00800000u}, {0x80000000u, 0x00000000u}, + {0x80000000u, 0x00000000u}, {0x80000000u, 0x00800000u}, {0x80000000u, 0x00000000u}, {0x80000000u, 0x00000000u}, + {0x80000000u, 0x00800000u}, {0x80000000u, 0x00800000u}, {0x80000000u, 0x00000000u}, {0x80000000u, 0x00000000u} + }, + { + {0x00000000u, 0x80800000u}, {0x80800000u, 0x00000000u}, {0x00000000u, 0x80800000u}, {0x00000000u, 0x80800000u}, + {0x00000000u, 0x80800000u}, {0x80800000u, 0x00000000u}, {0x00000000u, 0x80800000u}, {0x00000000u, 0x80000000u}, + {0x00000000u, 0x80000000u}, {0x00000000u, 0x80800000u}, {0x00000000u, 0x80000000u}, {0x00000000u, 0x80000000u}, + {0x00000000u, 0x80800000u}, {0x00000000u, 0x80800000u}, {0x00000000u, 0x80000000u}, {0x00000000u, 0x80000000u}, + {0x80800000u, 0x00000000u}, {0x00000000u, 0x80800000u}, {0x80800000u, 0x00000000u}, {0x80800000u, 0x00000000u}, + {0x80800000u, 0x00000000u}, {0x00000000u, 0x80800000u}, {0x80800000u, 0x00000000u}, {0x80000000u, 0x00000000u}, + {0x80000000u, 0x00000000u}, {0x80800000u, 0x00000000u}, {0x80000000u, 0x00000000u}, {0x80000000u, 0x00000000u}, + {0x80800000u, 0x00000000u}, {0x80800000u, 0x00000000u}, {0x80000000u, 0x00000000u}, {0x80000000u, 0x00000000u} + }, + { + {0x00000000u, 0x80000000u}, {0x80000000u, 0x00000000u}, {0x00000000u, 0x80000000u}, {0x00000000u, 0x80000000u}, + {0x00000000u, 0x80000000u}, {0x80000000u, 0x00000000u}, {0x00000000u, 0x80000000u}, {0x00000000u, 0x80000000u}, + {0x00000000u, 0x80000000u}, {0x00000000u, 0x80000000u}, {0x00000000u, 0x80000000u}, {0x00000000u, 0x80000000u}, + {0x00000000u, 0x80000000u}, {0x00000000u, 0x80000000u}, {0x00000000u, 0x80000000u}, {0x00000000u, 0x80000000u}, + {0x80000000u, 0x00000000u}, {0x00000000u, 0x80000000u}, {0x80000000u, 0x00000000u}, {0x80000000u, 0x00000000u}, + {0x80000000u, 0x00000000u}, {0x00000000u, 0x80000000u}, {0x80000000u, 0x00000000u}, {0x80000000u, 0x00000000u}, + {0x80000000u, 0x00000000u}, {0x80000000u, 0x00000000u}, {0x80000000u, 0x00000000u}, {0x80000000u, 0x00000000u}, + {0x80000000u, 0x00000000u}, {0x80000000u, 0x00000000u}, {0x80000000u, 0x00000000u}, {0x80000000u, 0x00000000u} + } +}; +constexpr std::uint32_t TinyLegacyExpected[32] = { + 0x00000000u, 0x80000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, + 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, + 0x80000000u, 0x00000000u, 0x80000000u, 0x80000000u, 0x00000000u, 0x00000000u, 0x80000000u, 0x80000000u, + 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x00000000u, 0x80000000u +}; +constexpr const char* TinyLegacyNames[2] = { + "v_mad_legacy_f32 v18, v4, v5, v6", + "v_mac_legacy_f32 v19, v4, v5" +}; +constexpr const char* TinyNames[8] = { + "v_mul_f32 v10, v4, v5", + "v_mul_legacy_f32 v11, v4, v5", + "v_fma_f32 v12, v4, v5, v6", + "v_fmac_f32 v13, v4, v5", + "v_fmamk_f32 v14, v4, 0x3f7ffffe, v6", + "v_fmaak_f32 v15, v4, v5, 0x80000000", + "v_fma_mix_f32 v16, v4, v5, v6", + "v_mul_f32 v17, -v4, v5" +}; std::array BufferDescriptor(const void* data, std::uint32_t bytes) { const auto address = reinterpret_cast(data); @@ -147,15 +272,14 @@ bool IsNan(std::uint32_t value) { return (value & 0x7fffffffu) > 0x7f800000u; } -void Run(AgcDriver::VulkanDevice& device, const std::optional& floatMode) { - for (std::uint32_t tid = 0; tid < Threads; ++tid) Fill(tid, &Input[tid * Inputs]); +void Run(AgcDriver::VulkanDevice& device, const std::optional& floatMode, std::span code = Code, const std::uint32_t (*rows)[Inputs] = Rows) { + for (std::uint32_t tid = 0; tid < Threads; ++tid) std::copy(std::begin(rows[tid]), std::end(rows[tid]), &Input[tid * Inputs]); Output.fill(0xdeadbeefu); std::vector userData(8, 0u); const auto input = BufferDescriptor(Input.data(), static_cast(Input.size() * 4u)); const auto output = BufferDescriptor(Output.data(), static_cast(Output.size() * 4u)); std::copy(input.begin(), input.end(), userData.begin()); std::copy(output.begin(), output.end(), userData.begin() + 4); - const std::span code(Code); const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {false, false, false}, false, 1}; ShaderRecompiler::RecompileRequest request{ @@ -185,6 +309,26 @@ void Check(const char* mode) { } } +void CheckTiny(const char* mode) { + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + for (std::uint32_t i = 0; i < 8; ++i) Expect(tid, Output[tid * TinyResults + i], TinyExpected[tid][i], (std::string(mode) + " " + TinyNames[i]).c_str()); + } +} + +void CheckTinyLegacy(const std::string& mode) { + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + for (std::uint32_t i = 0; i < 2; ++i) Expect(tid, Output[tid * TinyResults + 8u + i], TinyLegacyExpected[tid], (mode + " " + TinyLegacyNames[i]).c_str()); + } +} + +void CheckTinyDirected(std::uint32_t round) { + const std::string mode = "round mode " + std::to_string(round) + " "; + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + Expect(tid, Output[tid * TinyResults], TinyDirected[round - 1u][tid][0], (mode + TinyNames[0]).c_str()); + Expect(tid, Output[tid * TinyResults + 7u], TinyDirected[round - 1u][tid][1], (mode + TinyNames[7]).c_str()); + } +} + } int main() { @@ -197,6 +341,20 @@ int main() { Check("IEEE=0 f32 denormals flushed"); Run(*device, ShaderRecompiler::ShaderFloatMode{0x00u, false, true, false}); Check("IEEE=1 all denormals flushed"); + Run(*device, std::nullopt, TinyCode, TinyRows); + CheckTinyLegacy("no float mode"); + CheckTiny("no float mode"); + Run(*device, ShaderRecompiler::ShaderFloatMode{0xc0u, true, false, false}, TinyCode, TinyRows); + CheckTinyLegacy("IEEE=0 f32 denormals flushed"); + CheckTiny("IEEE=0 f32 denormals flushed"); + Run(*device, ShaderRecompiler::ShaderFloatMode{0x00u, false, true, false}, TinyCode, TinyRows); + CheckTinyLegacy("IEEE=1 all denormals flushed"); + CheckTiny("IEEE=1 all denormals flushed"); + for (std::uint32_t round = 1u; round < 4u; ++round) { + Run(*device, ShaderRecompiler::ShaderFloatMode{0xc0u | round, true, false, false}, TinyCode, TinyRows); + CheckTinyLegacy("round mode " + std::to_string(round)); + CheckTinyDirected(round); + } for (const std::uint32_t mode : {0xd0u, 0xe0u}) { bool refused = false; try { diff --git a/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp b/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp index e6340cfdb..8125bc758 100644 --- a/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp +++ b/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp @@ -61,7 +61,7 @@ private: void writeF16(const RdnaOperand& operand, IrF32 value, std::initializer_list sources); IrU32 readU32(const RdnaOperand& operand); IrU32 flushF32Denormal(IrU32 bits); - IrF32 flushTinyProduct(IrValue* lhs, IrValue* rhs, IrValue* product, IrValue* addend = nullptr); + IrF32 flushTinyProduct(IrValue* lhs, IrValue* rhs, IrValue* product, IrValue* addend = nullptr, bool afterRounding = true); IrU32 quietNan32(IrU32 bits); IrU32 quietNan16(IrU32 bits); IrValue* nanResultF32(std::initializer_list sources, IrValue* result, IrValue* invalidProduct = nullptr); diff --git a/core/shader/recompiler/Translation/src/FloatInstructions.cpp b/core/shader/recompiler/Translation/src/FloatInstructions.cpp index b85979973..a072c4d85 100644 --- a/core/shader/recompiler/Translation/src/FloatInstructions.cpp +++ b/core/shader/recompiler/Translation/src/FloatInstructions.cpp @@ -651,7 +651,7 @@ bool TranslationContext::vFmaLegacyF32(const RdnaInstruction& inst, bool accumul const auto factor = [&](IrValue* value) { return &ir.Emit(IrOpcode::SelectF32, IrType::F32, {&zero.Value(), &ir.ConstantF32(0.0f), value}); }; IrValue* lhsFactor = factor(lhs); IrValue* rhsFactor = factor(rhs); - IrValue* product = &flushTinyProduct(lhsFactor, rhsFactor, &ir.Emit(IrOpcode::FPMul32, IrType::F32, {lhsFactor, rhsFactor})).Value(); + IrValue* product = &flushTinyProduct(lhsFactor, rhsFactor, &ir.Emit(IrOpcode::FPMul32, IrType::F32, {lhsFactor, rhsFactor}), nullptr, false).Value(); const IrU1 zeroSum(ir.LogicalAnd(isZero(product).Value(), isZero(addend).Value())); IrValue& zeroSumBits = ir.BitwiseAnd(ir.BitwiseAnd(ir.BitCastU32(*product), ir.BitCastU32(*addend)), ir.Constant(0x80000000u)); IrValue* sum = &ir.Emit(IrOpcode::SelectF32, IrType::F32, {&zeroSum.Value(), &ir.BitCastF32(zeroSumBits), &ir.Emit(IrOpcode::FPAdd32, IrType::F32, {product, addend})}); diff --git a/core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp b/core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp index 63d6d727b..da7adbcc6 100644 --- a/core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp +++ b/core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp @@ -153,7 +153,7 @@ IrU32 TranslationContext::flushF32Denormal(IrU32 bits) { return IrU32(ir.Select(denormal.Value(), ir.BitwiseAnd(bits.Value(), ir.Constant(0x80000000u)), bits.Value())); } -IrF32 TranslationContext::flushTinyProduct(IrValue* lhs, IrValue* rhs, IrValue* product, IrValue* addend) { +IrF32 TranslationContext::flushTinyProduct(IrValue* lhs, IrValue* rhs, IrValue* product, IrValue* addend, bool afterRounding) { if ((f32DenormalFlush & 2u) == 0u || keepsProductOutOfTinyRange(*lhs) || keepsProductOutOfTinyRange(*rhs)) return IrF32(*product); IrValue& lhsBits = ir.BitCastU32(*lhs); IrValue& rhsBits = ir.BitCastU32(*rhs); @@ -162,11 +162,27 @@ IrF32 TranslationContext::flushTinyProduct(IrValue* lhs, IrValue* rhs, IrValue* IrValue& lhsExponent = exponent(lhsBits); IrValue& rhsExponent = exponent(rhsBits); const auto finiteNonZero = [&](IrValue& value) -> IrValue& { return ir.LogicalAnd(ir.INotEqual(value, ir.Constant(0u)), ir.INotEqual(value, ir.Constant(0xffu))); }; - IrValue& carry = hasUnitSignificand(*lhs) || hasUnitSignificand(*rhs) ? ir.Constant(0u) : ir.Select(ir.UGreaterThan(ir.Emit(IrOpcode::UMulHi, IrType::U32, {&significand(lhsBits), &significand(rhsBits)}), ir.Constant(0x7fffu)), ir.Constant(1u), ir.Constant(0u)); - IrValue* tiny = &ir.LogicalAnd(ir.LogicalAnd(finiteNonZero(lhsExponent), finiteNonZero(rhsExponent)), ir.ULessThan(ir.IAdd(ir.IAdd(lhsExponent, rhsExponent), carry), ir.Constant(128u))); + const std::uint32_t roundMode = floatMode.has_value() ? floatMode->floatMode & 3u : 0u; + IrValue* carry = &ir.Constant(0u); + IrValue* roundsUp = nullptr; + if (!hasUnitSignificand(*lhs) && !hasUnitSignificand(*rhs)) { + IrValue& lhsSignificand = significand(lhsBits); + IrValue& rhsSignificand = significand(rhsBits); + IrValue& high = ir.Emit(IrOpcode::UMulHi, IrType::U32, {&lhsSignificand, &rhsSignificand}); + carry = &ir.Select(ir.UGreaterThan(high, ir.Constant(0x7fffu)), ir.Constant(1u), ir.Constant(0u)); + if (afterRounding && roundMode != 3u) roundsUp = &ir.LogicalAnd(ir.IEqual(high, ir.Constant(0x7fffu)), ir.UGreaterThan(ir.IMul(lhsSignificand, rhsSignificand), ir.Constant(roundMode == 0u ? 0xffbfffffu : 0xff800000u))); + } + IrValue& exponentSum = ir.IAdd(ir.IAdd(lhsExponent, rhsExponent), *carry); + IrValue* tiny = &ir.LogicalAnd(ir.LogicalAnd(finiteNonZero(lhsExponent), finiteNonZero(rhsExponent)), ir.ULessThan(exponentSum, ir.Constant(128u))); if (addend != nullptr) tiny = &ir.LogicalAnd(*tiny, ir.IEqual(ir.BitwiseAnd(ir.BitCastU32(*addend), ir.Constant(0x7fffffffu)), ir.Constant(0u))); IrValue& sign = ir.BitwiseAnd(ir.BitwiseXor(lhsBits, rhsBits), ir.Constant(0x80000000u)); - return IrF32(ir.Emit(IrOpcode::SelectF32, IrType::F32, {tiny, &ir.BitCastF32(sign), product})); + IrValue* rounded = &sign; + if (roundsUp != nullptr) { + roundsUp = &ir.LogicalAnd(ir.IEqual(exponentSum, ir.Constant(127u)), *roundsUp); + if (roundMode != 0u) roundsUp = &ir.LogicalAnd(*roundsUp, ir.IEqual(sign, ir.Constant(roundMode == 1u ? 0u : 0x80000000u))); + rounded = &ir.Select(*roundsUp, ir.BitwiseOr(sign, ir.Constant(0x00800000u)), sign); + } + return IrF32(ir.Emit(IrOpcode::SelectF32, IrType::F32, {tiny, &ir.BitCastF32(*rounded), product})); } IrValue* TranslationContext::readOperand(const RdnaOperand& operand, IrType type) { diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..f146a6de8 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -376,7 +376,7 @@ Throughout the project, every function at every stage either **does exactly what - [v_fma_mixlo_f16 and v_fma_mixhi_f16](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) round the exact product-sum once to f16 with the f64 arithmetic of v_interp_p2_f16, so they need the Float64 capability. On RDNA2 (gfx1036) they follow the `FLOAT_MODE` f16 round and denormal fields, not the f32 round field: -1.5 * 2^-24 + 2^-126 from f32 sources (0xbfc00000, 0x33800000, 0x00800000) gives 0x8001 with f16 denormals kept and round to nearest even, 0x8000 with f16 denormals flushed and 0x8002 with f16 round toward -infinity. Only f16 denormals kept with round to nearest even is translated; another f16 mode throws, as for packed f16. `FP16_OVFL` set is not measured. - [Signaling NaN inputs](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) of the f32 and f16 unary instructions (`v_sqrt/rcp/rsq/log/exp/sin/cos/fract/floor/ceil/trunc/rndne/rcp_iflag/frexp_mant`), `v_ldexp_f32/f16`, `v_frexp_mant_f16`, `v_div_fixup_f32/f16` and `v_mullit_f32` are returned quieted only with IEEE=1 and unchanged with IEEE=0, measured on RDNA2 (gfx1035) in both modes, and so are those of the f64 forms (`v_sqrt/rcp/rsq/fract/floor/ceil/trunc/rndne/frexp_mant/ldexp/div_fixup_f64`) and `v_cvt_f64_f32`, measured on RDNA2 (gfx1036) in both modes; shaders without a known float mode follow IEEE=0. `v_pack_b32_f16` quiets each signaling NaN half by the same rule, measured on RDNA2 (gfx1036) in both modes. A signaling NaN result of `v_cubesc/cubetc/cubema_f32` is quieted by the same rule and keeps the sign the instruction gives it, measured on RDNA2 (gfx1036) in both modes. `v_add/sub/subrev/mul/fma/fmac/fmamk/fmaak/mul_legacy_f32` and `v_fma_mix_f32` with f32 sources choose their NaN result on the bits, measured over 1000 operand triples in both modes: the first NaN source in operand order after its modifiers (the subtrahend negated, as the hardware adds its negation), quieted only with IEEE=1, or the default NaN 0xffc00000 for an invalid operation; in the fma forms an invalid product (inf * 0) gives the default NaN even when src2 is a NaN, and in the legacy forms a zero factor makes the product 0 even when the other factor is a NaN (a NaN src2 still wins). `v_mad_legacy_f32` and `v_mac_legacy_f32` follow the same rule with their separately rounded product, as measured on gfx1010 (#1281, #1290): a NaN source quieted only with IEEE=1, inf - inf giving 0xffc00000, the zero rule hiding a NaN factor and a NaN src2 winning. The f16 and f64 arithmetic, `v_fma_mix_f32` with f16 sources and the separately rounded mad/mac forms (not measurable on RDNA2) follow the same rule on hardware but their NaN bits are left to the driver. `v_cvt_f16_f32`, `v_cvt_f32_f64`, `v_dot2_f32_f16`, `v_fma_mixlo_f16` and `v_fma_mixhi_f16` always quiet and `v_div_scale_f32` never does: the f32 signaling NaN 0x7f812345 as src0 or src2 gives 0x7e09 from mixlo and mixhi at IEEE=0 and IEEE=1 on RDNA2 (gfx1036), where `v_fma_mix_f32` gives 0x7f812345 at IEEE=0 and 0x7fc12345 at IEEE=1. The payload of an f16 NaN source of mixlo and mixhi is left to the driver's f16 to f32 conversion, which NVIDIA replaces. - [Packed f16 NaN results](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) (`v_pk_add/mul/fma/min/max_f16`) follow the hardware measured on RDNA2 (gfx1035) at IEEE=0 and IEEE=1 with f16 round-to-nearest-even and f16 denormals kept: with IEEE=1 a signaling NaN input is quieted and min/max return the other operand only for a quiet NaN; with IEEE=0 NaN inputs pass through unquieted and min/max ignore signaling. In `v_pk_fma_f16` an invalid product (inf * 0) gives the default NaN even when src2 is a NaN. A shader without a known float mode is translated as IEEE=0, the mode titles use; a mode with another f16 rounding or that flushes f16 denormals throws. -- [f32 denormals](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follow the shader's `FLOAT_MODE` f32 denormal field (bits 5:4) when it flushes them (0, and shaders without a known float mode, as in the 0xc0 most titles use), measured on RDNA2 (gfx1035) over 1444 rows per instruction at IEEE=0 and IEEE=1. `v_add/sub/subrev/mul/mul_legacy/mullit/fma/fmac/fmamk/fmaak/min/max/min3/max3/med3/fract/floor/ceil/frexp_mant/sin/cos_f32` flush their f32 inputs and their result to a zero of the same sign (`v_mullit_f32` measured on RDNA2 (gfx1036) over 6907 rows per mode at IEEE=0 and IEEE=1, so a positive denormal src2 counts as zero); the f32 compares, `v_cvt_i32/u32/rpi_i32/flr_i32_f32`, `v_frexp_exp_i32_f32`, `v_cvt_f64_f32`, `v_cvt_f16_f32`, `v_cvt_pkrtz_f16_f32`, `v_cvt_pknorm_i16/u16_f32` and `v_cvt_pk_u8_f32` flush their f32 inputs. `v_div_fixup_f32` flushes its src1 and src2 and keeps a denormal src0 and result, measured on RDNA2 (gfx1036) over 6859 rows per mode at IEEE=0 and IEEE=1. A product is checked for tininess on its exact value: one below 2^-126 gives zero even when it rounds to 2^-126, computed exactly from the significands for `v_mul_f32`, `v_mul_legacy_f32`, `v_mullit_f32`, and for `v_fma_f32`, `v_fmac_f32`, `v_fmamk_f32`, `v_fmaak_f32` and `v_fma_mix_f32` when the addend is zero; with a nonzero addend the fused result is kept, so an exact result just below 2^-126 that rounds to it is not flushed. The hardware detects tininess after rounding to 24 bits with an unbounded exponent, measured on RDNA2 (gfx1036) for `v_mul_f32` and `v_mullit_f32` at IEEE=0 and IEEE=1: `0x3f7ffffe * 0x00800001` (2^-126 * (1 - 2^-46)) gives `0x00800000` there and zero here, while `0x3f7fffff * 0x00800000` (2^-126 * (1 - 2^-24)) gives zero on both. `v_fma_mix_f32` was measured with f32 sources only; its f16 sources are not flushed. `v_fma_mixlo_f16` and `v_fma_mixhi_f16` flush their f32 sources too and keep a product below 2^-126, measured on RDNA2 (gfx1036). `v_mad_legacy_f32` and `v_mac_legacy_f32` flush their inputs, their product and their result in every mode, see its entry. Denormal modes 1 and 2 throw for the other instructions. Not covered, and not measurable on RDNA2 (no such opcodes) or not measured: `v_mac_f32`, `v_mad_f32`, `v_madak_f32` and `v_madmk_f32` (rounded separately), `v_dot2_f32_f16`, `v_div_scale_f32` and `v_div_fmas_f32` (on hardware `v_div_fmas_f32` flushes only its result), f32 interpolation and `ds_add_f32` keep the previous lowering; `ds_min/max/cmpst_f32` flush both operands by the field, see the LDS float atomics entry. With f32 denormals kept (3) the results depend on the driver's default denormal behaviour, as no `DenormPreserve` execution mode is requested, and `v_ldexp_f32` flushes in every mode. +- [f32 denormals](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follow the shader's `FLOAT_MODE` f32 denormal field (bits 5:4) when it flushes them (0, and shaders without a known float mode, as in the 0xc0 most titles use), measured on RDNA2 (gfx1035) over 1444 rows per instruction at IEEE=0 and IEEE=1. `v_add/sub/subrev/mul/mul_legacy/mullit/fma/fmac/fmamk/fmaak/min/max/min3/max3/med3/fract/floor/ceil/frexp_mant/sin/cos_f32` flush their f32 inputs and their result to a zero of the same sign (`v_mullit_f32` measured on RDNA2 (gfx1036) over 6907 rows per mode at IEEE=0 and IEEE=1, so a positive denormal src2 counts as zero); the f32 compares, `v_cvt_i32/u32/rpi_i32/flr_i32_f32`, `v_frexp_exp_i32_f32`, `v_cvt_f64_f32`, `v_cvt_f16_f32`, `v_cvt_pkrtz_f16_f32`, `v_cvt_pknorm_i16/u16_f32` and `v_cvt_pk_u8_f32` flush their f32 inputs. `v_div_fixup_f32` flushes its src1 and src2 and keeps a denormal src0 and result, measured on RDNA2 (gfx1036) over 6859 rows per mode at IEEE=0 and IEEE=1. Tininess is detected after rounding, measured on RDNA2 (gfx1036) for `v_mul_f32` and `v_mullit_f32` at IEEE=0 and IEEE=1: a product is flushed when its value rounded to 24 bits with an unbounded exponent is below 2^-126, so `0x3f7ffffe * 0x00800001` (2^-126 * (1 - 2^-46)) gives `0x00800000` and `0x3f7fffff * 0x00800000` (2^-126 * (1 - 2^-24)) gives zero. The check rounds in the f32 round mode of `FLOAT_MODE` (bits 1:0), measured in all four modes, while the products themselves are still rounded to nearest even. This is computed exactly from the significands for `v_mul_f32`, `v_mul_legacy_f32`, `v_mullit_f32`, and for `v_fma_f32`, `v_fmac_f32`, `v_fmamk_f32`, `v_fmaak_f32` and `v_fma_mix_f32` when the addend is zero; with a nonzero addend the fused result is kept, so an exact result just below 2^-126 that rounds to it is not flushed. `v_fma_mix_f32` was measured with f32 sources only; its f16 sources are not flushed. `v_fma_mixlo_f16` and `v_fma_mixhi_f16` flush their f32 sources too and keep a product below 2^-126, measured on RDNA2 (gfx1036). `v_mad_legacy_f32` and `v_mac_legacy_f32` flush their inputs, their product and their result in every mode, see its entry; their product is still checked for tininess on its exact value, so one below 2^-126 gives zero even when it rounds to 2^-126, as the console's separately rounded encoding is not measured at this boundary. Denormal modes 1 and 2 throw for the other instructions. Not covered, and not measurable on RDNA2 (no such opcodes) or not measured: `v_mac_f32`, `v_mad_f32`, `v_madak_f32` and `v_madmk_f32` (rounded separately), `v_dot2_f32_f16`, `v_div_scale_f32` and `v_div_fmas_f32` (on hardware `v_div_fmas_f32` flushes only its result), f32 interpolation and `ds_add_f32` keep the previous lowering; `ds_min/max/cmpst_f32` flush both operands by the field, see the LDS float atomics entry. With f32 denormals kept (3) the results depend on the driver's default denormal behaviour, as no `DenormPreserve` execution mode is requested, and `v_ldexp_f32` flushes in every mode. - The executable file that [relinker](../../core/relinker/elfpatcher/src/windows/WindowsPeWriter.cpp) generates opens the console when launched, which is inconvenient for playability. - [Overlay output bus](../../core/libs/prx/libSceVideoOut/src/VideoOutDriver.cpp) (libSceVideoOut) - flips on a port opened with `SCE_VIDEO_OUT_BUS_TYPE_OVERLAY` complete and signal their events like main-bus flips, but their buffers are not shown: the overlay is not composited over the main output. Presenting them as whole frames made the window alternate between the two buses - [PA_CL_CLIP_CNTL](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) - `DX_LINEAR_ATTR_CLIP_ENA` (bit 24) clear is drawn like bit 24 set, with Vulkan's clipping of `NoPerspective` outputs; what a clear bit changes on the console is undocumented. From d59efa478efc270ff297d8370048fa35706187d0 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 11:10:09 -0400 Subject: [PATCH 071/460] test(shader): pin mixed-source tininess after rounding Measure the mixed f16/f32 boundary on gfx1036 and pin both factor slots, half selection, signs and zero f16 addends in the existing test. Record this measured scope while keeping legacy mad/mac unchanged. Co-Authored-By: Codex gpt-6.1-sol --- .../tests/execution/F32DenormalFlush.cpp | 144 ++++++++++++++++++ docs/dev/TechnicalDebt.md | 2 +- 2 files changed, 145 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/F32DenormalFlush.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/F32DenormalFlush.cpp index 3c61af0cb..b089628a9 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/F32DenormalFlush.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/F32DenormalFlush.cpp @@ -253,6 +253,121 @@ constexpr const char* TinyNames[8] = { "v_mul_f32 v17, -v4, v5" }; +alignas(256) constexpr std::array MixedTinyCode{ + 0x34020084u, 0x34040086u, 0xe0381000u, 0x80000401u, 0xbf8c3f70u, 0xcc20000au, 0x141a0b04u, 0xcc20000bu, + 0x0c1a0905u, 0xcc20100cu, 0x141a0b04u, 0xcc20080du, 0x0c1a0905u, 0xcc20400eu, 0x141e0b04u, 0xcc20400fu, + 0x0c1e0905u, 0x7e1002ffu, 0x3f7ffffeu, 0xcc204010u, 0x041e1104u, 0xcc206011u, 0x041e1104u, 0xe0781000u, + 0x80010a02u, 0xe0781010u, 0x80010e02u, 0xbf810000u, +}; + +constexpr std::uint32_t MixedTinyRows[Threads][Inputs] = { + {0x00801002u, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x80801002u, 0x00003bffu, 0x80000000u, 0x80008000u}, + {0x00801000u, 0x3bff3bffu, 0x00000000u, 0x00000000u}, + {0x00801001u, 0x3bff3bffu, 0x00000000u, 0x00000000u}, + {0x00801002u, 0x3bff3bffu, 0x00000000u, 0x00000000u}, + {0x00801003u, 0x3bff3bffu, 0x00000000u, 0x00000000u}, + {0x80801000u, 0x3bff3bffu, 0x80000000u, 0x80008000u}, + {0x80801001u, 0x3bff3bffu, 0x80000000u, 0x80008000u}, + {0x80801002u, 0x3bff3bffu, 0x80000000u, 0x80008000u}, + {0x80801003u, 0x3bff3bffu, 0x80000000u, 0x80008000u}, + {0x00801000u, 0xbbffbbffu, 0x80000000u, 0x80008000u}, + {0x00801001u, 0xbbffbbffu, 0x80000000u, 0x80008000u}, + {0x00801002u, 0xbbffbbffu, 0x80000000u, 0x80008000u}, + {0x00801003u, 0xbbffbbffu, 0x80000000u, 0x80008000u}, + {0x00800001u, 0x3bff3bffu, 0x00000000u, 0x00000000u}, + {0x80800001u, 0x3bff3bffu, 0x80000000u, 0x80008000u}, +}; +constexpr std::uint32_t MixedTinyExpected[16][3] = { + {0x00800000u, 0x00000000u, 0x00801001u}, + {0x80800000u, 0x80000000u, 0x80801001u}, + {0x00000000u, 0x00000000u, 0x00800fffu}, + {0x00000000u, 0x00000000u, 0x00801000u}, + {0x00800000u, 0x00800000u, 0x00801001u}, + {0x00800001u, 0x00800001u, 0x00801002u}, + {0x80000000u, 0x80000000u, 0x80800fffu}, + {0x80000000u, 0x80000000u, 0x80801000u}, + {0x80800000u, 0x80800000u, 0x80801001u}, + {0x80800001u, 0x80800001u, 0x80801002u}, + {0x80000000u, 0x80000000u, 0x00800fffu}, + {0x80000000u, 0x80000000u, 0x00801000u}, + {0x80800000u, 0x80800000u, 0x00801001u}, + {0x80800001u, 0x80800001u, 0x00801002u}, + {0x00000000u, 0x00000000u, 0x00800000u}, + {0x80000000u, 0x80000000u, 0x80800000u}, +}; +constexpr std::uint32_t MixedTinyDirected[3][16][2] = { + { + {0x00800000u, 0x00000000u}, + {0x80000000u, 0x80000000u}, + {0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000000u}, + {0x00800000u, 0x00800000u}, + {0x00800001u, 0x00800001u}, + {0x80000000u, 0x80000000u}, + {0x80000000u, 0x80000000u}, + {0x80000000u, 0x80000000u}, + {0x80800000u, 0x80800000u}, + {0x80000000u, 0x80000000u}, + {0x80000000u, 0x80000000u}, + {0x80000000u, 0x80000000u}, + {0x80800000u, 0x80800000u}, + {0x00000000u, 0x00000000u}, + {0x80000000u, 0x80000000u}, + }, + { + {0x00000000u, 0x00000000u}, + {0x80800000u, 0x80000000u}, + {0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000000u}, + {0x00800000u, 0x00800000u}, + {0x80000000u, 0x80000000u}, + {0x80000000u, 0x80000000u}, + {0x80800000u, 0x80800000u}, + {0x80800001u, 0x80800001u}, + {0x80000000u, 0x80000000u}, + {0x80000000u, 0x80000000u}, + {0x80800000u, 0x80800000u}, + {0x80800001u, 0x80800001u}, + {0x00000000u, 0x00000000u}, + {0x80000000u, 0x80000000u}, + }, + { + {0x00000000u, 0x00000000u}, + {0x80000000u, 0x80000000u}, + {0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000000u}, + {0x00800000u, 0x00800000u}, + {0x80000000u, 0x80000000u}, + {0x80000000u, 0x80000000u}, + {0x80000000u, 0x80000000u}, + {0x80800000u, 0x80800000u}, + {0x80000000u, 0x80000000u}, + {0x80000000u, 0x80000000u}, + {0x80000000u, 0x80000000u}, + {0x80800000u, 0x80800000u}, + {0x00000000u, 0x00000000u}, + {0x80000000u, 0x80000000u}, + }, +}; +constexpr std::uint32_t MixedAddendDirected[3][2] = { + {0x00800000u, 0x80000000u}, + {0x00000000u, 0x80800000u}, + {0x00000000u, 0x80000000u}, +}; +constexpr const char* MixedTinyNames[8] = { + "v_fma_mix_f32 v10, v4, v5, v6 op_sel_hi:[0,1,0]", + "v_fma_mix_f32 v11, v5, v4, v6 op_sel_hi:[1,0,0]", + "v_fma_mix_f32 v12, v4, v5, v6 op_sel:[0,1,0] op_sel_hi:[0,1,0]", + "v_fma_mix_f32 v13, v5, v4, v6 op_sel:[1,0,0] op_sel_hi:[1,0,0]", + "v_fma_mix_f32 v14, v4, v5, v7 op_sel_hi:[0,1,1]", + "v_fma_mix_f32 v15, v5, v4, v7 op_sel_hi:[1,0,1]", + "v_fma_mix_f32 v16, v4, v8, v7 op_sel_hi:[0,0,1]", + "v_fma_mix_f32 v17, v4, v8, v7 op_sel:[0,0,1] op_sel_hi:[0,0,1]", +}; + std::array BufferDescriptor(const void* data, std::uint32_t bytes) { const auto address = reinterpret_cast(data); return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; @@ -315,6 +430,25 @@ void CheckTiny(const char* mode) { } } +void CheckMixedTiny(const std::string& mode, std::uint32_t round = 0u) { + for (std::uint32_t tid = 0; tid < std::size(MixedTinyExpected); ++tid) { + for (std::uint32_t i = 0; i < std::size(MixedTinyNames); ++i) { + const bool highFactor = i == 2u || i == 3u; + std::uint32_t expected = MixedTinyExpected[tid][i >= 6u ? 2u : highFactor ? 1u : 0u]; + if (round != 0u) { + if (i >= 6u) { + if (tid < 14u) continue; + expected = MixedAddendDirected[round - 1u][tid - 14u]; + } else { + if ((MixedTinyRows[tid][0] & 0x7fffffffu) == 0x00801003u) continue; + expected = MixedTinyDirected[round - 1u][tid][highFactor ? 1u : 0u]; + } + } + Expect(tid, Output[tid * TinyResults + i], expected, (mode + " " + MixedTinyNames[i]).c_str()); + } + } +} + void CheckTinyLegacy(const std::string& mode) { for (std::uint32_t tid = 0; tid < Threads; ++tid) { for (std::uint32_t i = 0; i < 2; ++i) Expect(tid, Output[tid * TinyResults + 8u + i], TinyLegacyExpected[tid], (mode + " " + TinyLegacyNames[i]).c_str()); @@ -341,6 +475,16 @@ int main() { Check("IEEE=0 f32 denormals flushed"); Run(*device, ShaderRecompiler::ShaderFloatMode{0x00u, false, true, false}); Check("IEEE=1 all denormals flushed"); + Run(*device, std::nullopt, MixedTinyCode, MixedTinyRows); + CheckMixedTiny("no float mode"); + Run(*device, ShaderRecompiler::ShaderFloatMode{0xc0u, true, false, false}, MixedTinyCode, MixedTinyRows); + CheckMixedTiny("IEEE=0 mixed sources"); + Run(*device, ShaderRecompiler::ShaderFloatMode{0x00u, false, true, false}, MixedTinyCode, MixedTinyRows); + CheckMixedTiny("IEEE=1 mixed sources"); + for (std::uint32_t round = 1u; round < 4u; ++round) { + Run(*device, ShaderRecompiler::ShaderFloatMode{0xc0u | round, true, false, false}, MixedTinyCode, MixedTinyRows); + CheckMixedTiny("mixed sources round mode " + std::to_string(round), round); + } Run(*device, std::nullopt, TinyCode, TinyRows); CheckTinyLegacy("no float mode"); CheckTiny("no float mode"); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index f146a6de8..482cca797 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -376,7 +376,7 @@ Throughout the project, every function at every stage either **does exactly what - [v_fma_mixlo_f16 and v_fma_mixhi_f16](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) round the exact product-sum once to f16 with the f64 arithmetic of v_interp_p2_f16, so they need the Float64 capability. On RDNA2 (gfx1036) they follow the `FLOAT_MODE` f16 round and denormal fields, not the f32 round field: -1.5 * 2^-24 + 2^-126 from f32 sources (0xbfc00000, 0x33800000, 0x00800000) gives 0x8001 with f16 denormals kept and round to nearest even, 0x8000 with f16 denormals flushed and 0x8002 with f16 round toward -infinity. Only f16 denormals kept with round to nearest even is translated; another f16 mode throws, as for packed f16. `FP16_OVFL` set is not measured. - [Signaling NaN inputs](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) of the f32 and f16 unary instructions (`v_sqrt/rcp/rsq/log/exp/sin/cos/fract/floor/ceil/trunc/rndne/rcp_iflag/frexp_mant`), `v_ldexp_f32/f16`, `v_frexp_mant_f16`, `v_div_fixup_f32/f16` and `v_mullit_f32` are returned quieted only with IEEE=1 and unchanged with IEEE=0, measured on RDNA2 (gfx1035) in both modes, and so are those of the f64 forms (`v_sqrt/rcp/rsq/fract/floor/ceil/trunc/rndne/frexp_mant/ldexp/div_fixup_f64`) and `v_cvt_f64_f32`, measured on RDNA2 (gfx1036) in both modes; shaders without a known float mode follow IEEE=0. `v_pack_b32_f16` quiets each signaling NaN half by the same rule, measured on RDNA2 (gfx1036) in both modes. A signaling NaN result of `v_cubesc/cubetc/cubema_f32` is quieted by the same rule and keeps the sign the instruction gives it, measured on RDNA2 (gfx1036) in both modes. `v_add/sub/subrev/mul/fma/fmac/fmamk/fmaak/mul_legacy_f32` and `v_fma_mix_f32` with f32 sources choose their NaN result on the bits, measured over 1000 operand triples in both modes: the first NaN source in operand order after its modifiers (the subtrahend negated, as the hardware adds its negation), quieted only with IEEE=1, or the default NaN 0xffc00000 for an invalid operation; in the fma forms an invalid product (inf * 0) gives the default NaN even when src2 is a NaN, and in the legacy forms a zero factor makes the product 0 even when the other factor is a NaN (a NaN src2 still wins). `v_mad_legacy_f32` and `v_mac_legacy_f32` follow the same rule with their separately rounded product, as measured on gfx1010 (#1281, #1290): a NaN source quieted only with IEEE=1, inf - inf giving 0xffc00000, the zero rule hiding a NaN factor and a NaN src2 winning. The f16 and f64 arithmetic, `v_fma_mix_f32` with f16 sources and the separately rounded mad/mac forms (not measurable on RDNA2) follow the same rule on hardware but their NaN bits are left to the driver. `v_cvt_f16_f32`, `v_cvt_f32_f64`, `v_dot2_f32_f16`, `v_fma_mixlo_f16` and `v_fma_mixhi_f16` always quiet and `v_div_scale_f32` never does: the f32 signaling NaN 0x7f812345 as src0 or src2 gives 0x7e09 from mixlo and mixhi at IEEE=0 and IEEE=1 on RDNA2 (gfx1036), where `v_fma_mix_f32` gives 0x7f812345 at IEEE=0 and 0x7fc12345 at IEEE=1. The payload of an f16 NaN source of mixlo and mixhi is left to the driver's f16 to f32 conversion, which NVIDIA replaces. - [Packed f16 NaN results](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) (`v_pk_add/mul/fma/min/max_f16`) follow the hardware measured on RDNA2 (gfx1035) at IEEE=0 and IEEE=1 with f16 round-to-nearest-even and f16 denormals kept: with IEEE=1 a signaling NaN input is quieted and min/max return the other operand only for a quiet NaN; with IEEE=0 NaN inputs pass through unquieted and min/max ignore signaling. In `v_pk_fma_f16` an invalid product (inf * 0) gives the default NaN even when src2 is a NaN. A shader without a known float mode is translated as IEEE=0, the mode titles use; a mode with another f16 rounding or that flushes f16 denormals throws. -- [f32 denormals](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follow the shader's `FLOAT_MODE` f32 denormal field (bits 5:4) when it flushes them (0, and shaders without a known float mode, as in the 0xc0 most titles use), measured on RDNA2 (gfx1035) over 1444 rows per instruction at IEEE=0 and IEEE=1. `v_add/sub/subrev/mul/mul_legacy/mullit/fma/fmac/fmamk/fmaak/min/max/min3/max3/med3/fract/floor/ceil/frexp_mant/sin/cos_f32` flush their f32 inputs and their result to a zero of the same sign (`v_mullit_f32` measured on RDNA2 (gfx1036) over 6907 rows per mode at IEEE=0 and IEEE=1, so a positive denormal src2 counts as zero); the f32 compares, `v_cvt_i32/u32/rpi_i32/flr_i32_f32`, `v_frexp_exp_i32_f32`, `v_cvt_f64_f32`, `v_cvt_f16_f32`, `v_cvt_pkrtz_f16_f32`, `v_cvt_pknorm_i16/u16_f32` and `v_cvt_pk_u8_f32` flush their f32 inputs. `v_div_fixup_f32` flushes its src1 and src2 and keeps a denormal src0 and result, measured on RDNA2 (gfx1036) over 6859 rows per mode at IEEE=0 and IEEE=1. Tininess is detected after rounding, measured on RDNA2 (gfx1036) for `v_mul_f32` and `v_mullit_f32` at IEEE=0 and IEEE=1: a product is flushed when its value rounded to 24 bits with an unbounded exponent is below 2^-126, so `0x3f7ffffe * 0x00800001` (2^-126 * (1 - 2^-46)) gives `0x00800000` and `0x3f7fffff * 0x00800000` (2^-126 * (1 - 2^-24)) gives zero. The check rounds in the f32 round mode of `FLOAT_MODE` (bits 1:0), measured in all four modes, while the products themselves are still rounded to nearest even. This is computed exactly from the significands for `v_mul_f32`, `v_mul_legacy_f32`, `v_mullit_f32`, and for `v_fma_f32`, `v_fmac_f32`, `v_fmamk_f32`, `v_fmaak_f32` and `v_fma_mix_f32` when the addend is zero; with a nonzero addend the fused result is kept, so an exact result just below 2^-126 that rounds to it is not flushed. `v_fma_mix_f32` was measured with f32 sources only; its f16 sources are not flushed. `v_fma_mixlo_f16` and `v_fma_mixhi_f16` flush their f32 sources too and keep a product below 2^-126, measured on RDNA2 (gfx1036). `v_mad_legacy_f32` and `v_mac_legacy_f32` flush their inputs, their product and their result in every mode, see its entry; their product is still checked for tininess on its exact value, so one below 2^-126 gives zero even when it rounds to 2^-126, as the console's separately rounded encoding is not measured at this boundary. Denormal modes 1 and 2 throw for the other instructions. Not covered, and not measurable on RDNA2 (no such opcodes) or not measured: `v_mac_f32`, `v_mad_f32`, `v_madak_f32` and `v_madmk_f32` (rounded separately), `v_dot2_f32_f16`, `v_div_scale_f32` and `v_div_fmas_f32` (on hardware `v_div_fmas_f32` flushes only its result), f32 interpolation and `ds_add_f32` keep the previous lowering; `ds_min/max/cmpst_f32` flush both operands by the field, see the LDS float atomics entry. With f32 denormals kept (3) the results depend on the driver's default denormal behaviour, as no `DenormPreserve` execution mode is requested, and `v_ldexp_f32` flushes in every mode. +- [f32 denormals](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follow the shader's `FLOAT_MODE` f32 denormal field (bits 5:4) when it flushes them (0, and shaders without a known float mode, as in the 0xc0 most titles use), measured on RDNA2 (gfx1035) over 1444 rows per instruction at IEEE=0 and IEEE=1. `v_add/sub/subrev/mul/mul_legacy/mullit/fma/fmac/fmamk/fmaak/min/max/min3/max3/med3/fract/floor/ceil/frexp_mant/sin/cos_f32` flush their f32 inputs and their result to a zero of the same sign (`v_mullit_f32` measured on RDNA2 (gfx1036) over 6907 rows per mode at IEEE=0 and IEEE=1, so a positive denormal src2 counts as zero); the f32 compares, `v_cvt_i32/u32/rpi_i32/flr_i32_f32`, `v_frexp_exp_i32_f32`, `v_cvt_f64_f32`, `v_cvt_f16_f32`, `v_cvt_pkrtz_f16_f32`, `v_cvt_pknorm_i16/u16_f32` and `v_cvt_pk_u8_f32` flush their f32 inputs. `v_div_fixup_f32` flushes its src1 and src2 and keeps a denormal src0 and result, measured on RDNA2 (gfx1036) over 6859 rows per mode at IEEE=0 and IEEE=1. Tininess is detected after rounding, measured on RDNA2 (gfx1036) for `v_mul_f32` and `v_mullit_f32` at IEEE=0 and IEEE=1: a product is flushed when its value rounded to 24 bits with an unbounded exponent is below 2^-126, so `0x3f7ffffe * 0x00800001` (2^-126 * (1 - 2^-46)) gives `0x00800000` and `0x3f7fffff * 0x00800000` (2^-126 * (1 - 2^-24)) gives zero. The check rounds in the f32 round mode of `FLOAT_MODE` (bits 1:0), measured in all four modes, while the products themselves are still rounded to nearest even. This is computed exactly from the significands for `v_mul_f32`, `v_mul_legacy_f32`, `v_mullit_f32`, and for `v_fma_f32`, `v_fmac_f32`, `v_fmamk_f32`, `v_fmaak_f32` and `v_fma_mix_f32` when the addend is zero; with a nonzero addend the fused result is kept, so an exact result just below 2^-126 that rounds to it is not flushed. `v_fma_mix_f32` follows this rule with mixed f16/f32 sources too, measured on RDNA2 (gfx1036) with f16 in either multiplicand slot, low and high half selection, and an f32 or f16 zero addend: f32 `0x00801002` times f16 `0x3bff` gives `0x00800000` to nearest at IEEE=0 and IEEE=1, while the neighbours `0x00801001` and `0x00801003` give zero and `0x00800001`. The tiny-product boundary was also measured for both signs in all four f32 round modes at IEEE=0; f16 inputs retain their previous conversion and denormal handling. `v_fma_mixlo_f16` and `v_fma_mixhi_f16` flush their f32 sources too and keep a product below 2^-126, measured on RDNA2 (gfx1036). `v_mad_legacy_f32` and `v_mac_legacy_f32` flush their inputs, their product and their result in every mode, see its entry; their product is still checked for tininess on its exact value, so one below 2^-126 gives zero even when it rounds to 2^-126, as the console's separately rounded encoding is not measured at this boundary. Denormal modes 1 and 2 throw for the other instructions. Not covered, and not measurable on RDNA2 (no such opcodes) or not measured: `v_mac_f32`, `v_mad_f32`, `v_madak_f32` and `v_madmk_f32` (rounded separately), `v_dot2_f32_f16`, `v_div_scale_f32` and `v_div_fmas_f32` (on hardware `v_div_fmas_f32` flushes only its result), f32 interpolation and `ds_add_f32` keep the previous lowering; `ds_min/max/cmpst_f32` flush both operands by the field, see the LDS float atomics entry. With f32 denormals kept (3) the results depend on the driver's default denormal behaviour, as no `DenormPreserve` execution mode is requested, and `v_ldexp_f32` flushes in every mode. - The executable file that [relinker](../../core/relinker/elfpatcher/src/windows/WindowsPeWriter.cpp) generates opens the console when launched, which is inconvenient for playability. - [Overlay output bus](../../core/libs/prx/libSceVideoOut/src/VideoOutDriver.cpp) (libSceVideoOut) - flips on a port opened with `SCE_VIDEO_OUT_BUS_TYPE_OVERLAY` complete and signal their events like main-bus flips, but their buffers are not shown: the overlay is not composited over the main output. Presenting them as whole frames made the window alternate between the two buses - [PA_CL_CLIP_CNTL](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) - `DX_LINEAR_ATTR_CLIP_ENA` (bit 24) clear is drawn like bit 24 set, with Vulkan's clipping of `NoPerspective` outputs; what a clear bit changes on the console is undocumented. From e8209307e9705f62b2b4a2ee2d9cc7b4d3cf5a88 Mon Sep 17 00:00:00 2001 From: MathiasDevelopes <25400788+MathiasDevelopes@users.noreply.github.com> Date: Sat, 10 Oct 2026 18:24:54 +0200 Subject: [PATCH 072/460] feat(libSceAjm): implement sceAjmBatchJobDecodeSplit and sceAjmBatchJobSetResampleParametersEx sceAjmBatchJobDecodeSplit queues a split run job with the stream sideband, as sceAjmBatchJobDecode does for a single buffer pair. sceAjmBatchJobSetResampleParametersEx with a zero ratio change per sample sets the ratio as sceAjmBatchJobSetResampleParameters does; a non-zero change throws, since its ramp is unknown. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceAjm.native/Export.cpp | 23 -------- core/libs/prx/libSceAjm.native/src/Ajm.cpp | 9 +++ core/libs/tests/GuestAjm.cpp | 64 ++++++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 74 insertions(+), 23 deletions(-) diff --git a/core/libs/prx/libSceAjm.native/Export.cpp b/core/libs/prx/libSceAjm.native/Export.cpp index 8f5edd6e9..e6c8fcbd7 100644 --- a/core/libs/prx/libSceAjm.native/Export.cpp +++ b/core/libs/prx/libSceAjm.native/Export.cpp @@ -14,18 +14,6 @@ static void AjmStub(const char* name) { extern "C" { -int APS5_VABI sceAjmBatchJobDecodeSplit(AjmBatchInfo* info, uint32_t instance, const AjmBuffer* input_buffers, size_t input_buffers_num, const AjmBuffer* output_buffers, size_t output_buffers_num, void* result) { - (void)info; - (void)instance; - (void)input_buffers; - (void)input_buffers_num; - (void)output_buffers; - (void)output_buffers_num; - (void)result; - AjmStub(__func__); - return 0; -} - int APS5_VABI sceAjmBatchJobEncode(AjmBatchInfo* info, uint32_t instance, const void* pcm_input, size_t pcm_input_size, void* bitstream_output, size_t bitstream_output_size, void* result) { (void)info; (void)instance; @@ -38,17 +26,6 @@ int APS5_VABI sceAjmBatchJobEncode(AjmBatchInfo* info, uint32_t instance, const return 0; } -int APS5_VABI sceAjmBatchJobSetResampleParametersEx(AjmBatchInfo* info, uint32_t instance, float ratio_start, float ratio_change_per_sample, uint32_t flags, void* result) { - (void)info; - (void)instance; - (void)ratio_start; - (void)ratio_change_per_sample; - (void)flags; - (void)result; - AjmStub(__func__); - return 0; -} - const char* APS5_VABI sceAjmStrError(int error) { (void)error; AjmStub(__func__); diff --git a/core/libs/prx/libSceAjm.native/src/Ajm.cpp b/core/libs/prx/libSceAjm.native/src/Ajm.cpp index 89dc47bc0..6f207a53e 100644 --- a/core/libs/prx/libSceAjm.native/src/Ajm.cpp +++ b/core/libs/prx/libSceAjm.native/src/Ajm.cpp @@ -1340,6 +1340,11 @@ int APS5_VABI sceAjmBatchJobSetResampleParameters(AjmBatchInfo* info, uint32_t i return Append(info, header, nullptr, nullptr); } +int APS5_VABI sceAjmBatchJobSetResampleParametersEx(AjmBatchInfo* info, uint32_t instance, float ratio_start, float ratio_change_per_sample, uint32_t flags, void* result) { + if (ratio_change_per_sample != 0.0f) throw std::runtime_error("sceAjmBatchJobSetResampleParametersEx: a ratio change per sample is unsupported"); + return sceAjmBatchJobSetResampleParameters(info, instance, ratio_start, flags, result); +} + int APS5_VABI sceAjmBatchJobGetResampleInfo(AjmBatchInfo* info, uint32_t instance, void* result) { return Append(info, MakeHeader(JobKind::GetResampleInfo, instance, result, sizeof(SidebandResult) + sizeof(SidebandResampleInfo)), nullptr, nullptr); } @@ -1348,6 +1353,10 @@ int APS5_VABI sceAjmBatchJobDecode(AjmBatchInfo* info, uint32_t instance, const return sceAjmBatchJobRun(info, instance, SIDEBAND_STREAM, bitstream_input, bitstream_input_size, pcm_output, pcm_output_size, result, sizeof(SidebandResult) + sizeof(SidebandStream)); } +int APS5_VABI sceAjmBatchJobDecodeSplit(AjmBatchInfo* info, uint32_t instance, const AjmBuffer* input_buffers, size_t input_buffers_num, const AjmBuffer* output_buffers, size_t output_buffers_num, void* result) { + return sceAjmBatchJobRunSplit(info, instance, SIDEBAND_STREAM, input_buffers, input_buffers_num, output_buffers, output_buffers_num, result, sizeof(SidebandResult) + sizeof(SidebandStream)); +} + int APS5_VABI sceAjmBatchJobDecodeSingle(AjmBatchInfo* info, uint32_t instance, const void* bitstream_input, size_t bitstream_input_size, void* pcm_output, size_t pcm_output_size, void* result) { return sceAjmBatchJobRun(info, instance, SIDEBAND_STREAM, bitstream_input, bitstream_input_size, pcm_output, pcm_output_size, result, sizeof(SidebandResult) + sizeof(SidebandStream)); } diff --git a/core/libs/tests/GuestAjm.cpp b/core/libs/tests/GuestAjm.cpp index c8ab3c3fa..f11cb7c10 100644 --- a/core/libs/tests/GuestAjm.cpp +++ b/core/libs/tests/GuestAjm.cpp @@ -32,6 +32,8 @@ int APS5_VABI sceAjmBatchCancel(std::uint32_t, std::uint32_t); int APS5_VABI sceAjmBatchJobClearContext(AjmBatchInfo*, std::uint32_t, void*); int APS5_VABI sceAjmBatchJobSetResampleParameters(AjmBatchInfo*, std::uint32_t, float, std::uint32_t, void*); int APS5_VABI sceAjmBatchJobGetResampleInfo(AjmBatchInfo*, std::uint32_t, void*); +int APS5_VABI sceAjmBatchJobSetResampleParametersEx(AjmBatchInfo*, std::uint32_t, float, float, std::uint32_t, void*); +int APS5_VABI sceAjmBatchJobDecodeSplit(AjmBatchInfo*, std::uint32_t, const AjmBuffer*, std::size_t, const AjmBuffer*, std::size_t, void*); } static void Require(bool value) { if (!value) std::abort(); } @@ -367,6 +369,28 @@ void TestDecodeSingle(std::uint32_t context) { Require(sceAjmInstanceDestroy(context, instance) == 0); } +void TestDecodeSplit(std::uint32_t context) { + std::uint32_t whole = 0; + std::uint32_t split = 0; + Require(sceAjmInstanceCreate(context, 0, 0, &whole) == 0 && sceAjmInstanceCreate(context, 0, 0, &split) == 0); + std::vector reference(1152); + DecodeSideband expected{}; + RunDecode(context, whole, MP3_MONO, sizeof(MP3_MONO), reference.data(), reference.size() * sizeof(std::int16_t), expected); + std::vector first(512); + std::vector second(640); + const AjmBuffer input{const_cast(MP3_MONO), sizeof(MP3_MONO)}; + const AjmBuffer outputs[2] = {{first.data(), first.size() * sizeof(std::int16_t)}, {second.data(), second.size() * sizeof(std::int16_t)}}; + std::vector batch(0x100); + AjmBatchInfo info{}; + DecodeSideband sideband{}; + Require(sceAjmBatchInitialize(batch.data(), batch.size(), &info) == 0); + Require(sceAjmBatchJobDecodeSplit(&info, split, &input, 1, outputs, 2, &sideband) == 0); + Submit(context, info); + Require(expected.result == 0 && sideband.result == 0 && sideband.inputConsumed == expected.inputConsumed && sideband.outputWritten == expected.outputWritten && sideband.totalDecodedSamples == expected.totalDecodedSamples); + Require(std::equal(first.begin(), first.end(), reference.begin()) && std::equal(second.begin(), second.end(), reference.begin() + 512)); + Require(sceAjmInstanceDestroy(context, whole) == 0 && sceAjmInstanceDestroy(context, split) == 0); +} + void TestGaplessDecode(std::uint32_t context) { std::uint32_t instance = 0; Require(sceAjmInstanceCreate(context, 0, 0, &instance) == 0); @@ -921,6 +945,44 @@ void TestResampleMp3(std::uint32_t context) { for (const std::uint32_t instance : {plain, faster, slower}) Require(sceAjmInstanceDestroy(context, instance) == 0); } +void TestResampleParametersEx(std::uint32_t context) { + std::uint32_t plain = 0; + std::uint32_t extended = 0; + Require(sceAjmInstanceCreate(context, 0, 0, &plain) == 0 && sceAjmInstanceCreate(context, 0, 0, &extended) == 0); + std::vector plainPcm(512); + std::vector extendedPcm(512); + for (const bool ex : {false, true}) { + std::vector batch(4096); + AjmBatchInfo info{}; + std::int64_t setResult[2] = {-1, -1}; + DecodeSideband sideband{}; + auto& pcm = ex ? extendedPcm : plainPcm; + const auto instance = ex ? extended : plain; + Require(sceAjmBatchInitialize(batch.data(), batch.size(), &info) == 0); + if (ex) Require(sceAjmBatchJobSetResampleParametersEx(&info, instance, 2.0f, 0.0f, 1, setResult) == 0); + else Require(sceAjmBatchJobSetResampleParameters(&info, instance, 2.0f, 1, setResult) == 0); + Require(sceAjmBatchJobDecode(&info, instance, MP3_MONO, sizeof(MP3_MONO), pcm.data(), pcm.size() * sizeof(std::int16_t), &sideband) == 0); + Submit(context, info); + Require(setResult[0] == 0 && sideband.result == 0); + } + Require(extendedPcm == plainPcm); + Require(GetResampleInfo(context, extended).ratio == 2.0f && GetResampleInfo(context, extended).samples == GetResampleInfo(context, plain).samples); + + std::vector batch(4096); + AjmBatchInfo info{}; + std::int64_t result[2] = {-1, -1}; + Require(sceAjmBatchInitialize(batch.data(), batch.size(), &info) == 0); + for (const float bad : {0.0f, -1.0f, std::nanf(""), INFINITY}) Require(sceAjmBatchJobSetResampleParametersEx(&info, extended, bad, 0.0f, 1, result) == static_cast(0x80930005)); + bool rampThrows = false; + try { + static_cast(sceAjmBatchJobSetResampleParametersEx(&info, extended, 1.0f, 0.001f, 1, result)); + } catch (const std::runtime_error&) { + rampThrows = true; + } + Require(rampThrows && info.offset == 0); + Require(sceAjmInstanceDestroy(context, plain) == 0 && sceAjmInstanceDestroy(context, extended) == 0); +} + const std::uint8_t AT9_MONO_SILENT_SUPERFRAME[] = { 0x00, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, @@ -1042,6 +1104,7 @@ int main() { TestMp3(context); TestOpus(context); TestDecodeSingle(context); + TestDecodeSplit(context); TestGaplessDecode(context); TestCodecInfo(context); TestGetInfo(context); @@ -1053,6 +1116,7 @@ int main() { TestControlOpus(context); TestResampleOpus(context); TestResampleMp3(context); + TestResampleParametersEx(context); TestResampleAt9(context); TestAt9GaplessSegments(context); TestAt9RunDecodesWholeInput(context); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..13be47c92 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -207,6 +207,7 @@ Throughout the project, every function at every stage either **does exactly what - [NGS2 port matrix](../../core/libs/prx/libSceNgs2.native/src/Render.cpp) (libSceNgs2.native) - a port without a matrix, or with a selected matrix whose levels were never set, mixes through a default channel map: mono reaches both stereo channels at -3 dB and the 5.1 and 7.1 centre, stereo folds to mono at -3 dB per side, and wider sources fold down with 0.5 and -3 dB gains. The map and the equivalence of an unset matrix with no matrix were measured from the PS4 firmware 11.00 `libSceNgs2.sprx`, not yet from the PS5 library. That library only accepts submixer and mastering voices with 1, 2, 6 or 8 channels, so the map into 3, 4, 5 or 7 channels throws. A matrix set with fewer levels than the destination needs still mixes only the rows it has - [sceAjmBatchJobGetCodecInfo, sceAjmBatchJobGetGaplessDecode](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp) (libSceAjm.native) - the `RUN_GET_CODEC_INFO` flag (0x800), the ATRAC9 codec info layout and its next frame size are taken from the [shadPS4](https://github.com/shadps4-emu/shadPS4) and [RPCSX](https://github.com/RPCSX/rpcsx) reimplementations, not confirmed on a PS5 title; a run job without buffers returns result 0, as in shadPS4. The gapless sideband reports the total and skip counts the title set and does not count them down: RPCSX does the same, shadPS4 counts them down and BryKytyPS5 returns its current counts - [AJM Opus decoder](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp) (libSceAjm.native, codec 24) - the initialize parameter layout (`u32 channels, u32 sample rate, u32`, seen as `2, 48000, 0`) and the little-endian `u16` byte count before each packet come from one title (The Smurfs Dreams); only 48000 Hz and a zero third word are accepted +- [sceAjmBatchJobSetResampleParametersEx](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp) (libSceAjm.native) - the `(info, instance, float ratio_start, float ratio_change_per_sample, flags, result)` signature is inferred from the name and from `sceAjmBatchJobSetResampleParameters`; with a zero change per sample it sets the ratio as `sceAjmBatchJobSetResampleParameters` does (Watch Dogs: Legion, PPSA01500, calls it so). How a non-zero change ramps the ratio is unknown and throws - [sceAjmBatchJobControl](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp) (libSceAjm.native) - RESET (bit 13), INITIALIZE (bit 14) and the sideband input order (gapless decode, then initialize parameters) come from [shadPS4](https://github.com/shadps4-emu/shadPS4) (PS4); other flags throw - [sceAjmBatchJobControl](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp) (libSceAjm.native) - RESET is assumed to clear the context as sceAjmBatchJobClearContext does - [sceAjmBatchJobControl](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp) (libSceAjm.native) - initialize parameters are 8 bytes for ATRAC9, 12 for Opus and none for MP3, as one title passes them; other codecs and sizes, and an output other than the 8-byte result, throw From e203c4322110176903b1dc2e9ef5f72355c67abc Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Fri, 9 Oct 2026 20:46:39 -0400 Subject: [PATCH 073/460] feat(libSceHttp): accept sceHttpSetRecvBlockSize --- core/libs/prx/libSceHttp/Export.cpp | 6 ++++++ core/libs/tests/GuestHttp.cpp | 2 ++ docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceHttp/Export.cpp b/core/libs/prx/libSceHttp/Export.cpp index 22c0d2e5e..c1ab8550f 100644 --- a/core/libs/prx/libSceHttp/Export.cpp +++ b/core/libs/prx/libSceHttp/Export.cpp @@ -188,6 +188,12 @@ int APS5_VABI sceHttpSetRecvTimeOut(int id, uint32_t usec) { return 0; } +int APS5_VABI sceHttpSetRecvBlockSize(int id, uint32_t block_size) { + (void)id; + (void)block_size; + return 0; +} + int APS5_VABI sceHttpSetResponseHeaderMaxSize(int id, uint64_t header_size) { (void)id; (void)header_size; diff --git a/core/libs/tests/GuestHttp.cpp b/core/libs/tests/GuestHttp.cpp index 2faab2957..1a5dcc2d3 100644 --- a/core/libs/tests/GuestHttp.cpp +++ b/core/libs/tests/GuestHttp.cpp @@ -22,6 +22,7 @@ int APS5_VABI sceHttpsEnableOption(int, std::uint32_t); int APS5_VABI sceHttpsLoadCert(int, int, void*, void*, void*); int APS5_VABI sceHttpGetLastErrno(int, int*); int APS5_VABI sceHttpSetResponseHeaderMaxSize(int, std::uint64_t); +int APS5_VABI sceHttpSetRecvBlockSize(int, std::uint32_t); int APS5_VABI sceHttpRedirectCacheFlush(int); int APS5_VABI sceHttpsUnloadCert(int); int APS5_VABI sceHttpsSetSslVersion(int, int); @@ -209,6 +210,7 @@ int main() { Require(sceHttpsLoadCert(1, 0, nullptr, nullptr, nullptr) == 0); Require(sceHttpsUnloadCert(1) == 0); Require(sceHttpSetResponseHeaderMaxSize(1, 8192) == 0); + Require(sceHttpSetRecvBlockSize(1, 0x4000) == 0); Require(sceHttpRedirectCacheFlush(1) == 0); int authUserArg = 0; Require(sceHttpSetAuthInfoCallback(1, AuthInfo, &authUserArg) == 0); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..53289764a 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -36,7 +36,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceAvPlayerSetTrickSpeed](../../core/libs/prx/libSceAvPlayer/src/Source.cpp) (libSceAvPlayer) with a negative speed runs the clock backwards but delivers no frames; when a forward speed is set again, playback resumes from the rewound time. - [sceRudpActivate](../../core/libs/prx/libSceRudp/Export.cpp) (libSceRudp) - there is no reliable-UDP stack on the host to activate, so it returns success without doing anything, as [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/rudp/rudp.cpp) does for its own `sceRudpActivate`. PPSA01325 calls it with no arguments during start-up; what the console activates, and whether it reports an error before `sceRudpInit`, is unverified - [ulobjmgr](../../core/libs/prx/ulobjmgr/Export.cpp) registers no object: `_sceUlobjmgrRegisterObject` always hands out id 0 and `_sceUlobjmgrUnregisterObject` releases nothing, as shadPS4 does -- [libSceHttp](../../core/libs/prx/libSceHttp/Export.cpp) - no request reaches the network, so `sceHttpSetResponseHeaderMaxSize` has no response header to limit and `sceHttpRedirectCacheFlush` no redirect to forget; `sceHttpsUnloadCert` returns success like `sceHttpsLoadCert`, which keeps no certificate; `sceHttpSetRedirectCallback` and `sceHttpSetCookieRecvCallback` accept callbacks that are never called, `sceHttpsSetSslVersion` has no TLS connection to configure and `sceHttpsGetSslError` reports error 0 with detail 0; `sceHttpSetAuthInfoCallback` accepts a callback that is never called, as no server asks for authentication +- [libSceHttp](../../core/libs/prx/libSceHttp/Export.cpp) - no request reaches the network, so `sceHttpSetResponseHeaderMaxSize` has no response header to limit, `sceHttpSetRecvBlockSize` (`(int id, u32 blockSize)` as in [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.h)) no response body to read in blocks and `sceHttpRedirectCacheFlush` no redirect to forget; `sceHttpsUnloadCert` returns success like `sceHttpsLoadCert`, which keeps no certificate; `sceHttpSetRedirectCallback` and `sceHttpSetCookieRecvCallback` accept callbacks that are never called, `sceHttpsSetSslVersion` has no TLS connection to configure and `sceHttpsGetSslError` reports error 0 with detail 0; `sceHttpSetAuthInfoCallback` accepts a callback that is never called, as no server asks for authentication - [sceFontGlyphDefineAttribute](../../core/libs/prx/libSceFont/src/Render.cpp) (libSceFont) checks the glyph and ignores the attribute and its value, whose meaning is unknown. PPSA01325 sets attribute 0x11 to 0 on a glyph from `sceFontGenerateCharGlyph` that it deletes unused, and draws the character with `sceFontRenderCharGlyphImageHorizontal` ### Unknown function info From d2c160be7606c25fea808e3a1d07dbd639c1a075 Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Fri, 9 Oct 2026 00:08:37 -0400 Subject: [PATCH 074/460] feat(libkernel): implement cpuset_getaffinity for the calling thread --- .../prx/libkernel/Pthread/Posix/Thread.cpp | 22 ++++++++++++++++ core/libs/tests/GuestAvailableCpumask.cpp | 25 +++++++++++++++++++ 2 files changed, 47 insertions(+) diff --git a/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp b/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp index 226925ada..b525f205b 100644 --- a/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp +++ b/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp @@ -30,6 +30,28 @@ int APS5_VABI scePthreadGetprio(Pthread thread, int* prio); static constexpr int GUEST_SCHED_FIFO = 1; extern "C" { +int* APS5_VABI __error_nid_postfix(); +int APS5_VABI scePthreadGetaffinity(Pthread thread, KernelCpumask* mask); + +int APS5_VABI cpuset_getaffinity_nid_postfix(int level, int which, std::int64_t id, std::size_t size, void* mask) { + constexpr int LevelWhich = 3, WhichThread = 1; + constexpr int GuestEfault = 14, GuestErange = 34; + constexpr std::size_t MaximumSize = 256 / 8; + if (size < sizeof(KernelCpumask) || size > MaximumSize) { + *__error_nid_postfix() = GuestErange; + return -1; + } + if (level != LevelWhich || which != WhichThread || id != -1) NotImplemented_nid_no_patch("cpuset_getaffinity other than the calling thread"); + if (!mask) { + *__error_nid_postfix() = GuestEfault; + return -1; + } + KernelCpumask affinity = 0; + scePthreadGetaffinity(scePthreadSelf(), &affinity); + std::memset(mask, 0, size); + std::memcpy(mask, &affinity, sizeof(affinity)); + return 0; +} int APS5_VABI pthread_create_nid_postfix(Pthread* thread, const PthreadAttr* attr, PthreadEntry entry, void* arg) { return PosixThread::ToErrno(scePthreadCreate(thread, attr, entry, arg, nullptr)); diff --git a/core/libs/tests/GuestAvailableCpumask.cpp b/core/libs/tests/GuestAvailableCpumask.cpp index 7cc506661..171d43459 100644 --- a/core/libs/tests/GuestAvailableCpumask.cpp +++ b/core/libs/tests/GuestAvailableCpumask.cpp @@ -1,4 +1,6 @@ #include "SceTypes.hpp" +#include +#include #include #include @@ -11,6 +13,8 @@ int APS5_VABI scePthreadSetaffinity(Pthread thread, KernelCpumask mask); int APS5_VABI scePthreadAttrInit(PthreadAttr* attr); int APS5_VABI scePthreadAttrDestroy(PthreadAttr* attr); int APS5_VABI scePthreadAttrGetaffinity(const PthreadAttr* attr, KernelCpumask* mask); +int APS5_VABI cpuset_getaffinity_nid_postfix(int level, int which, std::int64_t id, std::size_t size, void* mask); +int* APS5_VABI __error_nid_postfix(); } static constexpr int SCE_OK = 0; @@ -23,6 +27,20 @@ static void* APS5_VABI Worker(void* arg) { return nullptr; } +static void* APS5_VABI ReportCpuset(void* arg) { + auto* result = static_cast(arg); + std::uint64_t set[4] = {~0ull, ~0ull, ~0ull, ~0ull}; + result[0] = cpuset_getaffinity_nid_postfix(3, 1, -1, 8, set); + result[1] = set[0]; + result[2] = set[1]; + result[3] = cpuset_getaffinity_nid_postfix(3, 1, -1, sizeof(set), set); + result[4] = set[0] | (set[1] != 0 || set[2] != 0 || set[3] != 0 ? 1ull << 63 : 0); + result[5] = cpuset_getaffinity_nid_postfix(3, 1, -1, 4, set) == -1 && *__error_nid_postfix() == 34; + result[6] = cpuset_getaffinity_nid_postfix(3, 1, -1, 33, set) == -1 && *__error_nid_postfix() == 34; + result[7] = cpuset_getaffinity_nid_postfix(3, 1, -1, 8, nullptr) == -1 && *__error_nid_postfix() == 14; + return nullptr; +} + static void* APS5_VABI ReportMask(void* arg) { *static_cast(arg) = sceKernelGetAvailableCpumask(); return nullptr; @@ -61,5 +79,12 @@ int main() { Require(scePthreadCreate(&thread, &attr, ReportMask, &fromThread, nullptr) == SCE_OK); Require(scePthreadJoin(thread, nullptr) == SCE_OK); Require(fromThread == available); + + std::uint64_t cpuset[8] = {}; + Require(scePthreadCreate(&thread, &attr, ReportCpuset, cpuset, nullptr) == SCE_OK); + Require(scePthreadJoin(thread, nullptr) == SCE_OK); + Require(cpuset[0] == 0 && cpuset[1] == available && cpuset[2] == ~0ull); + Require(cpuset[3] == 0 && cpuset[4] == available); + Require(cpuset[5] == 1 && cpuset[6] == 1 && cpuset[7] == 1); Require(scePthreadAttrDestroy(&attr) == SCE_OK); } From 18590d1f798bd13959574eb99d395710f0f4264c Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 12:40:39 -0400 Subject: [PATCH 075/460] feat(libs): provide the missing G2P dialog Close export Eager binding fails when a caller imports sceG2PDialogClose. Add the missing throwing stub alongside the existing G2P exports and record the assumed Close signature in TechnicalDebt. Co-Authored-By: GPT-6.1 Sol --- core/libs/prx/libSceCdlgG2P/Export.cpp | 5 +++++ docs/dev/TechnicalDebt.md | 1 + 2 files changed, 6 insertions(+) diff --git a/core/libs/prx/libSceCdlgG2P/Export.cpp b/core/libs/prx/libSceCdlgG2P/Export.cpp index 5b46c1222..7ad726f79 100644 --- a/core/libs/prx/libSceCdlgG2P/Export.cpp +++ b/core/libs/prx/libSceCdlgG2P/Export.cpp @@ -18,6 +18,11 @@ int APS5_VABI sceG2PDialogOpen(void) { return 0; } +int APS5_VABI sceG2PDialogClose(void) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + int APS5_VABI sceG2PDialogGetStatus(void) { NotImplemented_nid_no_patch(__func__); return 0; diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..945ca2651 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -179,6 +179,7 @@ Throughout the project, every function at every stage either **does exactly what - AudioIn and NpSessionSignaling exports added without an implementation have assumed signatures - [sceNpSessionSignalingGetConnectionFromPeerAddress](../../core/libs/prx/libSceNpSessionSignaling/Export.cpp) (libSceNpSessionSignaling) - imported by PPSA01289 and added without an implementation; unknown signature - [libSceCdlgG2P](../../core/libs/prx/libSceCdlgG2P/Export.cpp) - `sceG2PDialogInitialize`, `Terminate`, `Open`, `GetStatus`, `UpdateStatus` and `GetResult`, imported by PPSA01289, are added without an implementation and declared without parameters +- [sceG2PDialogClose](../../core/libs/prx/libSceCdlgG2P/Export.cpp) (libSceCdlgG2P) - throwing stub; the name comes from the aerolib NID list, and the signature `int(void)` is assumed from `sceLoginDialogClose`, not verified for G2P - [vieBRwlh1Lw](../../core/libs/prx/libSceAgc/Unimplemented.cpp) (libSceAgc) - unknown name, signature - [scePadIsRemoteController](../../core/libs/prx/libScePad/Export.cpp) (libScePad) - the name hashes to NID `fCWdlnmB1Ks`; the signature `(int handle, bool* remote)` comes from PPSA16064, which zeroes a byte, passes its address after `scePadGetControllerInformation` and ignores the result. The error codes follow the other pad calls. Controllers are local devices, so the answer is always false - [scePadSetAngularVelocityBiasCorrectionState](../../core/libs/prx/libScePad/Export.cpp) (libScePad) - inferred signature `(int handle, bool enabled)` follows `scePadSetTiltCorrectionState` and `scePadSetAngularVelocityDeadbandState`; an invalid handle returns `PAD_ERROR_INVALID_HANDLE`, disabling is a no-op, and enabling throws because bias correction is not modelled From 5ac10820ea011514bb1eab9d398fd6f0364a8f14 Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Thu, 8 Oct 2026 11:24:12 -0400 Subject: [PATCH 076/460] feat(libScePosixForWebKit): implement arc4random_buf --- core/libs/CMakeLists.txt | 10 +++++ core/libs/prx/libScePosixForWebKit/Export.cpp | 13 +++++++ core/libs/tests/GuestPosixRandom.cpp | 37 +++++++++++++++++++ 3 files changed, 60 insertions(+) create mode 100644 core/libs/tests/GuestPosixRandom.cpp diff --git a/core/libs/CMakeLists.txt b/core/libs/CMakeLists.txt index 32eb46fa5..bae33c894 100644 --- a/core/libs/CMakeLists.txt +++ b/core/libs/CMakeLists.txt @@ -238,4 +238,14 @@ if(BUILD_TESTING) set_tests_properties(savedata_native_write_replacement PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + + add_test_executable(guest_posix_random_tests tests/GuestPosixRandom.cpp) + target_include_directories(guest_posix_random_tests PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}) + target_link_libraries(guest_posix_random_tests PRIVATE libScePosixForWebKit) + configure_windows_unwind(guest_posix_random_tests) + add_test(NAME guest_posix_random COMMAND guest_posix_random_tests) + if(WIN32) + set_tests_properties(guest_posix_random PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() endif() diff --git a/core/libs/prx/libScePosixForWebKit/Export.cpp b/core/libs/prx/libScePosixForWebKit/Export.cpp index 71f7b26f6..509ea37a9 100644 --- a/core/libs/prx/libScePosixForWebKit/Export.cpp +++ b/core/libs/prx/libScePosixForWebKit/Export.cpp @@ -1,5 +1,9 @@ #include "prx/libc/include/general/VabiMacros.hpp" +#include #include +#include +#include +#include extern "C" { @@ -19,4 +23,13 @@ char* APS5_VABI strcasestr_nid_postfix(const char* text, const char* needle) { return nullptr; } +void APS5_VABI arc4random_buf_nid_postfix(void* buffer, std::size_t size) { + static thread_local std::random_device device; + auto* bytes = static_cast(buffer); + for (std::size_t offset = 0; offset < size; offset += sizeof(unsigned int)) { + const unsigned int value = device(); + std::memcpy(bytes + offset, &value, std::min(sizeof(value), size - offset)); + } +} + } diff --git a/core/libs/tests/GuestPosixRandom.cpp b/core/libs/tests/GuestPosixRandom.cpp new file mode 100644 index 000000000..43526c711 --- /dev/null +++ b/core/libs/tests/GuestPosixRandom.cpp @@ -0,0 +1,37 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include + +extern "C" void APS5_VABI arc4random_buf_nid_postfix(void*, std::size_t); + +void Require(bool condition, std::source_location location = std::source_location::current()) { + if (!condition) { + std::fprintf(stderr, "Posix random check failed at line %u\n", location.line()); + std::abort(); + } +} + +int main() { + std::array first{}; + std::array second{}; + first.fill(0xa5); + arc4random_buf_nid_postfix(first.data(), 0); + Require(std::all_of(first.begin(), first.end(), [](unsigned char value) { return value == 0xa5; })); + for (const std::size_t size : {1u, 3u, 5u, 4098u}) { + first.fill(0xa5); + arc4random_buf_nid_postfix(first.data(), size); + Require(std::all_of(first.begin() + static_cast(size), first.end(), + [](unsigned char value) { return value == 0xa5; })); + } + arc4random_buf_nid_postfix(first.data(), first.size()); + arc4random_buf_nid_postfix(second.data(), second.size()); + Require(first != second); + std::array counts{}; + for (const auto value : first) ++counts[value]; + Require(*std::max_element(counts.begin(), counts.end()) < 64); + return 0; +} From 2a675598f2fc8cd320fdbe86ddb4bdaa7c9c3828 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 12:57:01 -0400 Subject: [PATCH 077/460] feat(libSceNpManager): add the missing request timeout export Validate timeout arguments and track live requests so unknown or deleted ids return REQUEST_NOT_FOUND. This supplies the remaining absent load import alongside the dialog Close export; timeouts follow the existing offline network model and are not stored. Co-Authored-By: GPT-6.1-sol --- core/libs/prx/libSceNpManager/Export.cpp | 35 +++++++++++++++++++++-- core/libs/tests/GuestNpManager.cpp | 36 ++++++++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 69 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libSceNpManager/Export.cpp b/core/libs/prx/libSceNpManager/Export.cpp index e086b9ebf..d60ff5a7d 100644 --- a/core/libs/prx/libSceNpManager/Export.cpp +++ b/core/libs/prx/libSceNpManager/Export.cpp @@ -4,6 +4,7 @@ #include "prx/libc/include/General.hpp" #include #include +#include // PSN is not emulated: the user is reported as signed out and online queries fail. static constexpr int SCE_NP_ERROR_INVALID_ARGUMENT = static_cast(0x80550003); @@ -11,6 +12,7 @@ static constexpr int SCE_NP_ERROR_SIGNED_OUT = static_cast(0x80550006); static constexpr int SCE_NP_ERROR_USER_NOT_FOUND = static_cast(0x80550007); static constexpr int SCE_NP_ERROR_CALLBACK_ALREADY_REGISTERED = static_cast(0x80550008); static constexpr int SCE_NP_ERROR_CALLBACK_NOT_REGISTERED = static_cast(0x80550009); +static constexpr int SCE_NP_ERROR_REQUEST_NOT_FOUND = static_cast(0x80550014); static constexpr uint32_t NP_STATE_SIGNED_OUT = 1; static constexpr int NP_POLL_ASYNC_FINISHED = 0; static constexpr uint32_t NP_REACHABILITY_STATE_UNAVAILABLE = 0; @@ -21,6 +23,15 @@ namespace { std::mutex reachabilityMutex; void* reachabilityCallback = nullptr; void* reachabilityUserdata = nullptr; +std::mutex requestMutex; +std::unordered_set liveRequests; + +int AddRequest() { + const int reqId = g_nextRequest.fetch_add(1, std::memory_order_relaxed); + std::lock_guard lock(requestMutex); + liveRequests.insert(reqId); + return reqId; +} } extern "C" { @@ -55,15 +66,16 @@ int APS5_VABI sceNpCheckPremium(int req_id, const NpCheckPremiumParameter* param int APS5_VABI sceNpCreateAsyncRequest(const NpCreateAsyncRequestParameter* param) { (void)param; - return g_nextRequest.fetch_add(1, std::memory_order_relaxed); + return AddRequest(); } int APS5_VABI sceNpCreateRequest(void) { - return g_nextRequest.fetch_add(1, std::memory_order_relaxed); + return AddRequest(); } int APS5_VABI sceNpDeleteRequest(int req_id) { - (void)req_id; + std::lock_guard lock(requestMutex); + liveRequests.erase(req_id); return 0; } @@ -178,6 +190,23 @@ int APS5_VABI sceNpSetNpTitleId(const NpTitleId* title_id, const NpTitleSecret* return 0; } +int APS5_VABI sceNpSetTimeout(int reqId, int32_t resolveRetry, uint32_t resolveTimeout, uint32_t connTimeout, + uint32_t sendTimeout, uint32_t recvTimeout) { + constexpr uint32_t minResolveTimeout = 1000000; + constexpr uint32_t minTransferTimeout = 10000000; + auto belowMinimum = [](uint32_t timeout, uint32_t minimum) { return timeout != 0 && timeout < minimum; }; + if (reqId <= 0 || resolveRetry < 0) return SCE_NP_ERROR_INVALID_ARGUMENT; + if (resolveRetry == 0 && resolveTimeout == 0 && connTimeout == 0 && sendTimeout == 0 && recvTimeout == 0) { + return SCE_NP_ERROR_INVALID_ARGUMENT; + } + if (belowMinimum(resolveTimeout, minResolveTimeout) || belowMinimum(connTimeout, minTransferTimeout) || + belowMinimum(sendTimeout, minTransferTimeout) || belowMinimum(recvTimeout, minTransferTimeout)) { + return SCE_NP_ERROR_INVALID_ARGUMENT; + } + std::lock_guard lock(requestMutex); + return liveRequests.count(reqId) ? 0 : SCE_NP_ERROR_REQUEST_NOT_FOUND; +} + int APS5_VABI sceNpUnregisterStateCallback(void) { return 0; } diff --git a/core/libs/tests/GuestNpManager.cpp b/core/libs/tests/GuestNpManager.cpp index 106a1e05b..eaa8a77d4 100644 --- a/core/libs/tests/GuestNpManager.cpp +++ b/core/libs/tests/GuestNpManager.cpp @@ -9,10 +9,19 @@ extern "C" { int APS5_VABI sceNpGetNpId(int user_id, NpId* np_id); } +extern "C" { +int APS5_VABI sceNpCreateRequest(void); +int APS5_VABI sceNpCreateAsyncRequest(const NpCreateAsyncRequestParameter* param); +int APS5_VABI sceNpDeleteRequest(int reqId); +int APS5_VABI sceNpSetTimeout(int reqId, std::int32_t resolveRetry, std::uint32_t resolveTimeout, + std::uint32_t connTimeout, std::uint32_t sendTimeout, std::uint32_t recvTimeout); +} + namespace { constexpr int InvalidArgument = static_cast(0x80550003u); constexpr int SignedOut = static_cast(0x80550006u); +constexpr int RequestNotFound = static_cast(0x80550014u); void Require(bool condition, const char* message) { if (!condition) { @@ -21,9 +30,36 @@ void Require(bool condition, const char* message) { } } +void CheckRequestTimeouts() { + Require(sceNpSetTimeout(0x7fffffff, 2, 0, 0, 0, 0) == RequestNotFound, "never created request"); + Require(sceNpSetTimeout(0x7fffffff, 0, 0, 0, 0, 0) == InvalidArgument, "arguments checked before the request"); + const int requestId = sceNpCreateRequest(); + Require(requestId > 0, "sceNpCreateRequest must return a positive request id"); + Require(sceNpSetTimeout(requestId, 2, 1000000, 10000000, 10000000, 10000000) == 0, "minimum timeouts"); + Require(sceNpSetTimeout(requestId, 0, 0, 0, 0, 30000000) == 0, "only the receive timeout"); + Require(sceNpSetTimeout(requestId, 3, 0, 0, 0, 0) == 0, "only the resolve retries"); + Require(sceNpSetTimeout(requestId, 0, 0, 0, 0, 0) == InvalidArgument, "every value left at its default"); + Require(sceNpSetTimeout(0, 2, 0, 0, 0, 0) == InvalidArgument, "request id 0"); + Require(sceNpSetTimeout(-1, 2, 0, 0, 0, 0) == InvalidArgument, "negative request id"); + Require(sceNpSetTimeout(requestId, -1, 0, 0, 0, 30000000) == InvalidArgument, "negative resolve retries"); + Require(sceNpSetTimeout(requestId, 0, 999999, 0, 0, 0) == InvalidArgument, "resolve timeout below 1 s"); + Require(sceNpSetTimeout(requestId, 0, 0, 9999999, 0, 0) == InvalidArgument, "connect timeout below 10 s"); + Require(sceNpSetTimeout(requestId, 0, 0, 0, 9999999, 0) == InvalidArgument, "send timeout below 10 s"); + Require(sceNpSetTimeout(requestId, 0, 0, 0, 0, 1) == InvalidArgument, "receive timeout below 10 s"); + Require(sceNpDeleteRequest(requestId) == 0, "sceNpDeleteRequest failed"); + Require(sceNpSetTimeout(requestId, 2, 0, 0, 0, 0) == RequestNotFound, "deleted request"); + NpCreateAsyncRequestParameter param{}; + const int asyncId = sceNpCreateAsyncRequest(¶m); + Require(asyncId > 0, "sceNpCreateAsyncRequest must return a positive request id"); + Require(sceNpSetTimeout(asyncId, 2, 0, 0, 0, 0) == 0, "async request"); + Require(sceNpDeleteRequest(asyncId) == 0, "sceNpDeleteRequest failed on the async request"); + Require(sceNpSetTimeout(asyncId, 2, 0, 0, 0, 0) == RequestNotFound, "deleted async request"); +} + } int main() { + CheckRequestTimeouts(); NpId npId{}; std::memset(&npId, 0x5a, sizeof(npId)); NpId untouched{}; diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 945ca2651..4d1d895ea 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -220,6 +220,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceHttpUriUnescape](../../core/libs/prx/libSceHttp/src/Uri.cpp) (libSceHttp) - signature follows [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Net/Http.cs); percent decoding, invalid sequences and error codes follow [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.cpp), not confirmed on a PS5 title or hardware. Size includes the final NUL; malformed escapes and `+` remain unchanged, and decoding runs once. - [sceHttpUriSweepPath](../../core/libs/prx/libSceHttp/src/Uri.cpp) (libSceHttp) - behaviour follows the [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.cpp) reimplementation of the PS4 library and its tests (including the PS4 SDK example), not confirmed on a PS5 title or hardware. It is not RFC 3986 5.2.4: a path that doesn't start with `/` is copied unchanged and a trailing `.` or `..` without a slash is kept. Unlike shadPS4, nothing at or past `srcSize - 1` is read, and `srcSize == 1` writes only the terminator - [sceHttpSetResponseHeaderMaxSize, sceHttpRedirectCacheFlush, sceHttpsUnloadCert](../../core/libs/prx/libSceHttp/Export.cpp) (libSceHttp) - signatures follow [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.cpp) (PS4); no PS5 reference has them +- [sceNpSetTimeout](../../core/libs/prx/libSceNpManager/Export.cpp) (libSceNpManager) - the signature follows [orbital](https://github.com/AlexAltea/orbital/blob/b321fcdbd700cc02da1cb1ad25412781d5ec73c6/tools/ida/db_types.json), [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/0fe263a4760dfbfa973366890061749b4af0de97/src/core/libraries/np/np_manager.cpp) and [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/5dfdb4b0b166e83dc9ae275001afb695f12c7174/src/SharpProspero/Interop/Np/NpAccount.cs); the rejected values (all five 0, negative retries, a resolve timeout under 1 s or another timeout under 10 s, 0 meaning the default) are shadPS4's, not checked on a console. An id that was never created or was deleted returns `REQUEST_NOT_FOUND`, as in shadPS4; there is no network, so the timeouts are not stored - [sceHttpSetRedirectCallback, sceHttpSetCookieRecvCallback, sceHttpsSetSslVersion, sceHttpsGetSslError](../../core/libs/prx/libSceHttp/Export.cpp) (libSceHttp) - signatures follow [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.cpp) (PS4), which only logs and returns 0; a null `errNum` or `detail` in `sceHttpsGetSslError` is assumed to return `SCE_HTTP_ERROR_INVALID_VALUE` like `sceHttpGetLastErrno` - [sceHttpSetAuthInfoCallback](../../core/libs/prx/libSceHttp/Export.cpp) (libSceHttp) - signature and callback type follow [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.h) (PS4); no PS5 reference has them. The `id` is not checked, as in the other libSceHttp setters - [libSceHttp2 cookie boxes](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) - `sceHttp2CreateCookieBox` is taken to take the library context id first, as in the [prosper](https://github.com/mattias800/prosper) reimplementation; further arguments are unknown and ignored. `sceHttp2SetRequestNoContentLength` is assumed to take only the request id. Cookies are never stored, so `sceHttp2CookieFlush` has nothing to discard From 510787170ed6e2ea30cad1bc4714f7a7675ba92c Mon Sep 17 00:00:00 2001 From: Abdulrahman Elkordy <196347165+Silverash-i8@users.noreply.github.com> Date: Sat, 10 Oct 2026 17:23:17 +0300 Subject: [PATCH 078/460] fix(agc): link optimized pipelines with the library store's layout The background link-time-optimized link used the Pipeline's own layout, which ~Pipeline destroys right after marking the slot released, and ClearCachedPipelines destroys the cached pipelines before ClearPipelineLibraries waits for pending links. A link already past the released check then created its pipeline with a destroyed VkPipelineLayout. On Intel's Windows driver (Arrow Lake iGPU, 32.0.101.8991) agc_driver_vertex_fetch, agc_driver_pixel_interlock and agc_driver_mesh faulted inside igc-default64.dll every run, and agc_driver_prepared_exports and agc_driver_pixel_barycentric_emulation intermittently; all passed with APS5_NO_PIPELINE_LTO=1. The library store's layout, made from the same create info and used for the libraries, lives until ClearPipelineLibraries has waited for the pending links. (cherry picked from commit 758eeb9fb255d88c41945b299dc3ec5ac1578331) --- .../prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp index e1db0d226..5e142cbe4 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp @@ -199,15 +199,15 @@ VkPipeline LinkPipelineFromLibraries(const Context& context, const VkGraphicsPip *optimized = slot; const auto createPipelines = context.Function("vkCreateGraphicsPipelines"); const auto cache = context.pipelineCache; - const auto pipelineLayout = info.layout; - enqueue(store, context.device, [slot, createPipelines, cache, pipelineLayout, libraries, &store, devicePointer = &device] { + const auto libraryLayout = pipelineLayout->second; + enqueue(store, context.device, [slot, createPipelines, cache, libraryLayout, libraries, &store, devicePointer = &device] { if (slot->released.load()) return; VkPipelineLibraryCreateInfoKHR linked{VK_STRUCTURE_TYPE_PIPELINE_LIBRARY_CREATE_INFO_KHR}; linked.libraryCount = static_cast(libraries.size()); linked.pLibraries = libraries.data(); VkGraphicsPipelineCreateInfo create{VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO, &linked}; create.flags = VK_PIPELINE_CREATE_LINK_TIME_OPTIMIZATION_BIT_EXT; - create.layout = pipelineLayout; + create.layout = libraryLayout; VkPipeline result = VK_NULL_HANDLE; if (createPipelines(slot->device, cache, 1, &create, nullptr, &result) != VK_SUCCESS) return; slot->handle.store(result); From 2d0cf268047854697faa952dcad2647033923e9b Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 13:07:31 -0400 Subject: [PATCH 079/460] fix(shader): flush f16 min/max denormal inputs RDNA2 min/max flush half inputs to signed zero when input preservation is disabled, while output flushing alone keeps them. Apply this to the measured binary forms and pin the hardware rows. Co-Authored-By: GPT-6.1 Sol --- .../tests/execution/F16MinMaxNan.cpp | 213 +++++++++++++++++- .../Translation/src/FloatInstructions.cpp | 6 + docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 216 insertions(+), 5 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/F16MinMaxNan.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/F16MinMaxNan.cpp index 364b2c0da..d252078c9 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/F16MinMaxNan.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/F16MinMaxNan.cpp @@ -30,6 +30,20 @@ alignas(256) constexpr std::array Code{ 0x42020b04u, 0xe0781000u, 0x80010a03u, 0xe0781010u, 0x80010e03u, 0xbf810000u, }; +alignas(256) constexpr std::array SdwaLowCode{ + 0x34020084u, 0x34060086u, 0xe0381000u, 0x80000401u, 0xbf8c3f70u, 0x7e140280u, 0x7e160280u, 0x7e180280u, + 0x7e1a0280u, 0x7e1c0280u, 0x7e1e0280u, 0x7e200280u, 0x7e220280u, 0x74140af9u, 0x04041404u, 0x72160af9u, + 0x04041404u, 0x741e0af9u, 0x24141404u, 0x72200af9u, 0x14241404u, 0xe0781000u, 0x80010a03u, 0xe0781010u, + 0x80010e03u, 0xbf810000u, +}; + +alignas(256) constexpr std::array SdwaHighCode{ + 0x34020084u, 0x34060086u, 0xe0381000u, 0x80000401u, 0xbf8c3f70u, 0x7e140280u, 0x7e160280u, 0x7e180280u, + 0x7e1a0280u, 0x7e1c0280u, 0x7e1e0280u, 0x7e200280u, 0x7e220280u, 0x74140af9u, 0x05051404u, 0x72160af9u, + 0x05051404u, 0x741e0af9u, 0x25151404u, 0x72200af9u, 0x15251404u, 0xe0781000u, 0x80010a03u, 0xe0781010u, + 0x80010e03u, 0xbf810000u, +}; + constexpr std::uint32_t Rows[128][4] = { {0x12340000u, 0x56780000u, 0x9abcc000u, 0x00000000u}, {0x12340000u, 0x56788000u, 0x9abc0001u, 0x00000000u}, @@ -420,9 +434,181 @@ constexpr std::uint32_t ExpectedIeee[128][7] = { {0x00007e00u, 0x00007e00u, 0x00003c00u, 0x00003c00u, 0x00003c00u, 0x0000fe00u, 0x00007e00u}, {0x00008001u, 0x00000001u, 0x00007e01u, 0x00007e01u, 0x00007e01u, 0x00008001u, 0x00000001u} }; +constexpr std::uint32_t DenormalRows[32][4] = { + {0x12343c01u, 0x567803ffu, 0x00000000u, 0x00000000u}, + {0x1234bc01u, 0x567803ffu, 0x00000000u, 0x00000000u}, + {0x12340001u, 0x56788001u, 0x00000000u, 0x00000000u}, + {0x123403ffu, 0x567883ffu, 0x00000000u, 0x00000000u}, + {0x12340001u, 0x56780000u, 0x00000000u, 0x00000000u}, + {0x12348001u, 0x56788000u, 0x00000000u, 0x00000000u}, + {0x123403ffu, 0x56780400u, 0x00000000u, 0x00000000u}, + {0x123483ffu, 0x56788400u, 0x00000000u, 0x00000000u}, + {0x12340001u, 0x56787e00u, 0x00000000u, 0x00000000u}, + {0x12348001u, 0x5678fe55u, 0x00000000u, 0x00000000u}, + {0x123403ffu, 0x56787c01u, 0x00000000u, 0x00000000u}, + {0x123483ffu, 0x5678fd23u, 0x00000000u, 0x00000000u}, + {0x12340001u, 0x56787c00u, 0x00000000u, 0x00000000u}, + {0x12348001u, 0x5678fc00u, 0x00000000u, 0x00000000u}, + {0x123403ffu, 0x56780001u, 0x00000000u, 0x00000000u}, + {0x123483ffu, 0x56788001u, 0x00000000u, 0x00000000u}, + {0x123403ffu, 0x56783c01u, 0x00000000u, 0x00000000u}, + {0x123403ffu, 0x5678bc01u, 0x00000000u, 0x00000000u}, + {0x12348001u, 0x56780001u, 0x00000000u, 0x00000000u}, + {0x123483ffu, 0x567803ffu, 0x00000000u, 0x00000000u}, + {0x12340000u, 0x56780001u, 0x00000000u, 0x00000000u}, + {0x12348000u, 0x56788001u, 0x00000000u, 0x00000000u}, + {0x12340400u, 0x567803ffu, 0x00000000u, 0x00000000u}, + {0x12348400u, 0x567883ffu, 0x00000000u, 0x00000000u}, + {0x12347e00u, 0x56780001u, 0x00000000u, 0x00000000u}, + {0x1234fe55u, 0x56788001u, 0x00000000u, 0x00000000u}, + {0x12347c01u, 0x567803ffu, 0x00000000u, 0x00000000u}, + {0x1234fd23u, 0x567883ffu, 0x00000000u, 0x00000000u}, + {0x12347c00u, 0x56780001u, 0x00000000u, 0x00000000u}, + {0x1234fc00u, 0x56788001u, 0x00000000u, 0x00000000u}, + {0x12340001u, 0x567803ffu, 0x00000000u, 0x00000000u}, + {0x12348001u, 0x567883ffu, 0x00000000u, 0x00000000u}, +}; +constexpr std::uint32_t DenormalFlushed[32][4] = { + {0x00000000u, 0x00003c01u, 0x0000bc01u, 0x00003c01u}, + {0x0000bc01u, 0x00000000u, 0x00000000u, 0x00003c01u}, + {0x00008000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00008000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00000000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000400u, 0x00008000u, 0x00000000u}, + {0x00008400u, 0x00008000u, 0x00000000u, 0x00000400u}, + {0x00000000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00007c00u, 0x00008000u, 0x00000000u}, + {0x0000fc00u, 0x00008000u, 0x00000000u, 0x00007c00u}, + {0x00000000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00003c01u, 0x00008000u, 0x00000000u}, + {0x0000bc01u, 0x00000000u, 0x00008000u, 0x00003c01u}, + {0x00008000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x00008000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000400u, 0x00008400u, 0x00000400u}, + {0x00008400u, 0x00008000u, 0x00000000u, 0x00000400u}, + {0x00000000u, 0x00000000u, 0x00000000u, 0x00008000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000000u, 0x00000000u, 0x00008000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00007c00u, 0x0000fc00u, 0x00007c00u}, + {0x0000fc00u, 0x00008000u, 0x00000000u, 0x00007c00u}, + {0x00000000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, +}; +constexpr std::uint32_t DenormalKept[32][4] = { + {0x000003ffu, 0x00003c01u, 0x0000bc01u, 0x00003c01u}, + {0x0000bc01u, 0x000003ffu, 0x000003ffu, 0x00003c01u}, + {0x00008001u, 0x00000001u, 0x00008001u, 0x00000001u}, + {0x000083ffu, 0x000003ffu, 0x000083ffu, 0x000003ffu}, + {0x00000000u, 0x00000001u, 0x00008001u, 0x00000001u}, + {0x00008001u, 0x00008000u, 0x00000000u, 0x00000001u}, + {0x000003ffu, 0x00000400u, 0x000083ffu, 0x000003ffu}, + {0x00008400u, 0x000083ffu, 0x000003ffu, 0x00000400u}, + {0x00000001u, 0x00000001u, 0x00008001u, 0x00000001u}, + {0x00008001u, 0x00008001u, 0x00000001u, 0x00000001u}, + {0x000003ffu, 0x000003ffu, 0x000083ffu, 0x000003ffu}, + {0x000083ffu, 0x000083ffu, 0x000003ffu, 0x000003ffu}, + {0x00000001u, 0x00007c00u, 0x00008001u, 0x00000001u}, + {0x0000fc00u, 0x00008001u, 0x00000001u, 0x00007c00u}, + {0x00000001u, 0x000003ffu, 0x000083ffu, 0x000003ffu}, + {0x000083ffu, 0x00008001u, 0x00000001u, 0x000003ffu}, + {0x000003ffu, 0x00003c01u, 0x000083ffu, 0x000003ffu}, + {0x0000bc01u, 0x000003ffu, 0x000083ffu, 0x00003c01u}, + {0x00008001u, 0x00000001u, 0x00000001u, 0x00000001u}, + {0x000083ffu, 0x000003ffu, 0x000003ffu, 0x000003ffu}, + {0x00000000u, 0x00000001u, 0x00008000u, 0x00000000u}, + {0x00008001u, 0x00008000u, 0x00000000u, 0x00000001u}, + {0x000003ffu, 0x00000400u, 0x00008400u, 0x00000400u}, + {0x00008400u, 0x000083ffu, 0x000003ffu, 0x00000400u}, + {0x00000001u, 0x00000001u, 0x00000001u, 0x00008001u}, + {0x00008001u, 0x00008001u, 0x00000001u, 0x00000001u}, + {0x000003ffu, 0x000003ffu, 0x000003ffu, 0x000083ffu}, + {0x000083ffu, 0x000083ffu, 0x000003ffu, 0x000003ffu}, + {0x00000001u, 0x00007c00u, 0x0000fc00u, 0x00007c00u}, + {0x0000fc00u, 0x00008001u, 0x00000001u, 0x00007c00u}, + {0x00000001u, 0x000003ffu, 0x00008001u, 0x00000001u}, + {0x000083ffu, 0x00008001u, 0x00000001u, 0x000003ffu}, +}; +constexpr std::uint32_t DenormalFlushedIeee[32][4] = { + {0x00000000u, 0x00003c01u, 0x0000bc01u, 0x00003c01u}, + {0x0000bc01u, 0x00000000u, 0x00000000u, 0x00003c01u}, + {0x00008000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00008000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00000000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000400u, 0x00008000u, 0x00000000u}, + {0x00008400u, 0x00008000u, 0x00000000u, 0x00000400u}, + {0x00000000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, + {0x00007e01u, 0x00007e01u, 0x00007e01u, 0x0000fe01u}, + {0x0000ff23u, 0x0000ff23u, 0x00007f23u, 0x00007f23u}, + {0x00000000u, 0x00007c00u, 0x00008000u, 0x00000000u}, + {0x0000fc00u, 0x00008000u, 0x00000000u, 0x00007c00u}, + {0x00000000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00003c01u, 0x00008000u, 0x00000000u}, + {0x0000bc01u, 0x00000000u, 0x00008000u, 0x00003c01u}, + {0x00008000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x00008000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000400u, 0x00008400u, 0x00000400u}, + {0x00008400u, 0x00008000u, 0x00000000u, 0x00000400u}, + {0x00000000u, 0x00000000u, 0x00000000u, 0x00008000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, + {0x00007e01u, 0x00007e01u, 0x0000fe01u, 0x00007e01u}, + {0x0000ff23u, 0x0000ff23u, 0x00007f23u, 0x00007f23u}, + {0x00000000u, 0x00007c00u, 0x0000fc00u, 0x00007c00u}, + {0x0000fc00u, 0x00008000u, 0x00000000u, 0x00007c00u}, + {0x00000000u, 0x00000000u, 0x00008000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00000000u, 0x00000000u}, +}; +constexpr std::uint32_t DenormalKeptIeee[32][4] = { + {0x000003ffu, 0x00003c01u, 0x0000bc01u, 0x00003c01u}, + {0x0000bc01u, 0x000003ffu, 0x000003ffu, 0x00003c01u}, + {0x00008001u, 0x00000001u, 0x00008001u, 0x00000001u}, + {0x000083ffu, 0x000003ffu, 0x000083ffu, 0x000003ffu}, + {0x00000000u, 0x00000001u, 0x00008001u, 0x00000001u}, + {0x00008001u, 0x00008000u, 0x00000000u, 0x00000001u}, + {0x000003ffu, 0x00000400u, 0x000083ffu, 0x000003ffu}, + {0x00008400u, 0x000083ffu, 0x000003ffu, 0x00000400u}, + {0x00000001u, 0x00000001u, 0x00008001u, 0x00000001u}, + {0x00008001u, 0x00008001u, 0x00000001u, 0x00000001u}, + {0x00007e01u, 0x00007e01u, 0x00007e01u, 0x0000fe01u}, + {0x0000ff23u, 0x0000ff23u, 0x00007f23u, 0x00007f23u}, + {0x00000001u, 0x00007c00u, 0x00008001u, 0x00000001u}, + {0x0000fc00u, 0x00008001u, 0x00000001u, 0x00007c00u}, + {0x00000001u, 0x000003ffu, 0x000083ffu, 0x000003ffu}, + {0x000083ffu, 0x00008001u, 0x00000001u, 0x000003ffu}, + {0x000003ffu, 0x00003c01u, 0x000083ffu, 0x000003ffu}, + {0x0000bc01u, 0x000003ffu, 0x000083ffu, 0x00003c01u}, + {0x00008001u, 0x00000001u, 0x00000001u, 0x00000001u}, + {0x000083ffu, 0x000003ffu, 0x000003ffu, 0x000003ffu}, + {0x00000000u, 0x00000001u, 0x00008000u, 0x00000000u}, + {0x00008001u, 0x00008000u, 0x00000000u, 0x00000001u}, + {0x000003ffu, 0x00000400u, 0x00008400u, 0x00000400u}, + {0x00008400u, 0x000083ffu, 0x000003ffu, 0x00000400u}, + {0x00000001u, 0x00000001u, 0x00000001u, 0x00008001u}, + {0x00008001u, 0x00008001u, 0x00000001u, 0x00000001u}, + {0x00007e01u, 0x00007e01u, 0x0000fe01u, 0x00007e01u}, + {0x0000ff23u, 0x0000ff23u, 0x00007f23u, 0x00007f23u}, + {0x00000001u, 0x00007c00u, 0x0000fc00u, 0x00007c00u}, + {0x0000fc00u, 0x00008001u, 0x00000001u, 0x00007c00u}, + {0x00000001u, 0x000003ffu, 0x00008001u, 0x00000001u}, + {0x000083ffu, 0x00008001u, 0x00000001u, 0x000003ffu}, +}; constexpr std::uint32_t Chunks = 4; std::uint32_t Chunk = 0; bool Ieee = false; +std::uint32_t Denorm16 = 4u; +std::uint32_t SourceForm = 0u; constexpr const char* Names[7] = { "v_min_f16 v10, v4, v5", "v_max_f16 v11, v4, v5", @@ -434,7 +620,12 @@ constexpr const char* Names[7] = { }; void Fill(std::uint32_t tid, std::uint32_t* words) { - std::copy(std::begin(Rows[Chunk * Threads + tid]), std::end(Rows[Chunk * Threads + tid]), words); + const auto& row = Denorm16 < 4u ? DenormalRows[tid] : Rows[Chunk * Threads + tid]; + std::copy(std::begin(row), std::end(row), words); + if (SourceForm == 2u) { + words[0] = (words[0] << 16u) | 0x1234u; + words[1] = (words[1] << 16u) | 0x5678u; + } } std::array BufferDescriptor(const void* data, std::uint32_t bytes) { @@ -449,7 +640,7 @@ std::string Hex(std::uint32_t value) { } void Expect(std::uint32_t tid, std::uint32_t actual, std::uint32_t expected, const char* name) { - Require(actual == expected, std::string(Ieee ? "f16 min max nan, IEEE mode: row " : "f16 min max nan: row ") + std::to_string(tid) + " " + name + " is " + Hex(actual) + ", expected " + Hex(expected)); + Require(actual == expected, std::string(Ieee ? "f16 min max nan, IEEE mode: row " : "f16 min max nan: row ") + std::to_string(tid) + ", denorm16 " + std::to_string(Denorm16) + ", source form " + std::to_string(SourceForm) + " " + name + " is " + Hex(actual) + ", expected " + Hex(expected)); } void Run(AgcDriver::VulkanDevice& device) { @@ -460,7 +651,7 @@ void Run(AgcDriver::VulkanDevice& device) { const auto output = BufferDescriptor(Output.data(), static_cast(Output.size() * 4u)); std::copy(input.begin(), input.end(), userData.begin()); std::copy(output.begin(), output.end(), userData.begin() + 4); - const std::span code(Code); + const std::span code = SourceForm == 1u ? std::span(SdwaLowCode) : (SourceForm == 2u ? std::span(SdwaHighCode) : std::span(Code)); const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {false, false, false}, false, 1}; ShaderRecompiler::RecompileRequest request{ @@ -470,7 +661,8 @@ void Run(AgcDriver::VulkanDevice& device) { {0, 0, 0, 128} }; request.useCache = false; - if (Ieee) request.context.floatMode = ShaderRecompiler::ShaderFloatMode{0xf0u, true, true, false}; + if (Denorm16 < 4u) request.context.floatMode = ShaderRecompiler::ShaderFloatMode{Denorm16 << 6u, true, Ieee, false}; + else if (Ieee) request.context.floatMode = ShaderRecompiler::ShaderFloatMode{0xf0u, true, true, false}; const auto result = ShaderRecompiler::Recompile(request); device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); device.WaitIdle(); @@ -480,6 +672,12 @@ void Check() { for (std::uint32_t tid = 0; tid < Threads; ++tid) { const std::uint32_t* in = &Input[tid * Inputs]; const std::uint32_t* out = &Output[tid * Results]; + if (Denorm16 < 4u) { + const auto& expected = Ieee ? ((Denorm16 & 1u) != 0u ? DenormalKeptIeee : DenormalFlushedIeee) : ((Denorm16 & 1u) != 0u ? DenormalKept : DenormalFlushed); + constexpr std::array indices{0u, 1u, 5u, 6u}; + for (std::uint32_t i = 0u; i < indices.size(); ++i) Expect(tid, out[indices[i]], expected[tid][i], Names[indices[i]]); + continue; + } for (std::uint32_t i = 0; i < 7; ++i) { Expect(Chunk * Threads + tid, out[i], (Ieee ? ExpectedIeee : Expected)[Chunk * Threads + tid][i], Names[i]); } @@ -498,6 +696,13 @@ int main() { Run(*device); Check(); } + for (SourceForm = 0u; SourceForm < 3u; ++SourceForm) { + for (Denorm16 = 0u; Denorm16 < 4u; ++Denorm16) { + Run(*device); + Check(); + } + } + SourceForm = 0u; } std::puts("f16 min max nan tests passed"); return 0; diff --git a/core/shader/recompiler/Translation/src/FloatInstructions.cpp b/core/shader/recompiler/Translation/src/FloatInstructions.cpp index b85979973..67caada40 100644 --- a/core/shader/recompiler/Translation/src/FloatInstructions.cpp +++ b/core/shader/recompiler/Translation/src/FloatInstructions.cpp @@ -615,6 +615,12 @@ bool TranslationContext::minMaxF16(const RdnaInstruction& inst, IrOpcode opcode) for (std::uint32_t index = 0u; index < (ternary ? 3u : 2u); ++index) { const RdnaOperand& operand = sourceAt(inst, index); args[index] = Half{&readF16AsF32(operand).Value(), readF16Bits(operand)}; + if (!ternary && floatMode.has_value() && (floatMode->floatMode & 0x40u) == 0u) { + const IrU1 tiny(ir.ULessThan(ir.BitwiseAnd(args[index].bits.Value(), ir.Constant(0x7fffu)), ir.Constant(0x0400u))); + const IrU32 sign(ir.BitwiseAnd(args[index].bits.Value(), ir.Constant(0x8000u))); + args[index].bits = IrU32(ir.Select(tiny.Value(), sign.Value(), args[index].bits.Value())); + args[index].value = &ir.Emit(IrOpcode::SelectF32, IrType::F32, {&tiny.Value(), &ir.BitCastF32(ir.ShiftLeftLogical(sign.Value(), ir.Constant(16u))), args[index].value}); + } } const auto pair = [&](IrOpcode pairOpcode, const Half& lhs, const Half& rhs) { IrValue& value = ir.Emit(pairOpcode, IrType::F32, {lhs.value, rhs.value}); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..2f649a96e 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -368,7 +368,7 @@ Throughout the project, every function at every stage either **does exactly what - A [COND_INDIRECT_BUFFER](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp) (`sceAgcCbBranch`) runs only with the always-pass compare function; a comparison or a predicated packet throws. Modes 1 and 2 run the first buffer, assumed to be the if-then and if-then-else modes of AMD's PM4 packet. [sceAgcCbBranch](../../core/libs/prx/libSceAgc/Cb/src/Branch.cpp) does not check the compare address of an always-pass branch or the address of an empty buffer, as PPSA14632 passes an uninitialised compare address and a null else buffer of 0 dwords. - A [label or store](../../core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp) (`RELEASE_MEM`, `WRITE_DATA`, `DMA_DATA`) into guest memory the device has no host import of waits for all recorded work and is then written by the CPU, so the queue worker stalls once per such label. The console's CP stores it in stream order without that stall. - [SET_*_REG_INDIRECT](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp) (libSceAgcDriver) reads its register list when the command buffer is submitted, with the packets. The console's graphics PFP reads it when it reaches the packet, so a list written after the submission (by the CPU, or by GPU work ahead of the packet behind PFP_SYNC_ME) is read here before that write. -- The [shader float mode](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp) (`FLOAT_MODE`, `DX10_CLAMP`, `IEEE_MODE` and `FP16_OVFL` of `COMPUTE_PGM_RSRC1` and `SPI_SHADER_PGM_RSRC1_PS/GS/HS`, bit positions from Mesa's `gfx10.json` and `gfx103.json`) is decoded into `GuestContext::floatMode` and is part of the shader cache keys and the serialized request (#981). `IEEE_MODE` is read by `v_min_f32`, `v_max_f32`, `v_min3_f32`, `v_max3_f32` and `v_med3_f32`, their f16 forms, `v_min_f64` and `v_max_f64`, `IEEE_MODE` and the `FLOAT_MODE` output denormal bits by the output modifier, `DX10_CLAMP` by the clamp modifier, `FP16_OVFL` by every instruction with an f16 result (see its entry), and the whole mode by the 16-bit interpolation instructions (see their entries). With IEEE=1 a signaling NaN operand returns that NaN quieted (the first operand checked first, min3/max3 as two nested pairs, med3 as min3 when any operand is NaN), measured on RDNA2 (gfx1035) over every combination of ±0, ±1, ±inf, denormals, quiet and signaling NaNs with f32 denormals kept and flushed. IEEE=0 and unknown modes keep the previous f32 lowering, which matches IEEE=0. The f16 forms choose their NaN results on the bits: a NaN operand returns the other operand unchanged (src0 when both are NaN), and with IEEE=1 a signaling NaN operand returns that NaN quieted, in the same order as f32. `v_min_f64` and `v_max_f64` quiet a signaling NaN operand the same way with IEEE=1; with IEEE=0 they return src0 unchanged when src1 is NaN, otherwise src1 when src0 is NaN. Both were measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1 with f32 denormals kept and flushed and f16/f64 denormals kept; RDNA2 has no f64 min3, max3 or med3. Unknown modes follow IEEE=0. With f16/f64 denormals flushed the hardware also flushes their denormal inputs (`v_min_f16(0x0001, 0x8001)` gives 0x8000), which these lowerings do not. NaN quieting in f16 arithmetic (the unary instructions and the f32 arithmetic follow it, see below) and `FLOAT_MODE` (f32 denormals flushed before the min/max compare in Astro's Playroom's 0xc0) still follow a single mode, as the entries below record. The mode comes from the registered shader header's `sh_registers`, the same source at registration and at draw/dispatch so the prepared artifact keys match. A title that rewrites RSRC1 at run time, or a raw compute program with no header, compiles with that mode or none. +- The [shader float mode](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp) (`FLOAT_MODE`, `DX10_CLAMP`, `IEEE_MODE` and `FP16_OVFL` of `COMPUTE_PGM_RSRC1` and `SPI_SHADER_PGM_RSRC1_PS/GS/HS`, bit positions from Mesa's `gfx10.json` and `gfx103.json`) is decoded into `GuestContext::floatMode` and is part of the shader cache keys and the serialized request (#981). `IEEE_MODE` is read by `v_min_f32`, `v_max_f32`, `v_min3_f32`, `v_max3_f32` and `v_med3_f32`, their f16 forms, `v_min_f64` and `v_max_f64`, `IEEE_MODE` and the `FLOAT_MODE` output denormal bits by the output modifier, `DX10_CLAMP` by the clamp modifier, `FP16_OVFL` by every instruction with an f16 result (see its entry), and the whole mode by the 16-bit interpolation instructions (see their entries). With IEEE=1 a signaling NaN operand returns that NaN quieted (the first operand checked first, min3/max3 as two nested pairs, med3 as min3 when any operand is NaN), measured on RDNA2 (gfx1035) over every combination of ±0, ±1, ±inf, denormals, quiet and signaling NaNs with f32 denormals kept and flushed. IEEE=0 and unknown modes keep the previous f32 lowering, which matches IEEE=0. The f16 forms choose their NaN results on the bits: a NaN operand returns the other operand unchanged (src0 when both are NaN), and with IEEE=1 a signaling NaN operand returns that NaN quieted, in the same order as f32. `v_min_f64` and `v_max_f64` quiet a signaling NaN operand the same way with IEEE=1; with IEEE=0 they return src0 unchanged when src1 is NaN, otherwise src1 when src0 is NaN. Both were measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1 with f32 denormals kept and flushed and f16/f64 denormals kept; RDNA2 has no f64 min3, max3 or med3. Unknown modes follow IEEE=0. `v_min_f16` and `v_max_f16` flush each denormal input to its signed zero when bit 0 of the f16/f64 denormal field is clear (0 and 2); fields 1 and 3 preserve inputs, even when output flushing is enabled. Measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1, including signed zeros, NaNs and neg/abs modifiers. Unknown modes keep f16 denormals. With f16/f64 denormals flushed the hardware also flushes denormal inputs of the f16 ternary forms and f64 min/max, which those lowerings still do not. NaN quieting in f16 arithmetic (the unary instructions and the f32 arithmetic follow it, see below) and `FLOAT_MODE` (f32 denormals flushed before the min/max compare in Astro's Playroom's 0xc0) still follow a single mode, as the entries below record. The mode comes from the registered shader header's `sh_registers`, the same source at registration and at draw/dispatch so the prepared artifact keys match. A title that rewrites RSRC1 at run time, or a raw compute program with no header, compiles with that mode or none. - A [WAIT_REG_MEM](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/WaitMemory.cpp) (libSceAgcDriver) that a compute queue passes on another queue's recorded label store, before the GPU has executed it, is landed before the queue's next dispatch, draw, COND_EXEC or predicated packet reads guest memory: the worker polls until memory satisfies the wait or no unfinished batch writes the label. The graphics queue (queue 0) is exempt, as landing its waits costs PPSA01341 about a quarter of its frame rate; a title that fills memory a graphics command buffer reads after it submits the buffer behind such a wait reads it early there. `APS5_LAND_WAITS_ALL_QUEUES=1` lands the graphics queue's waits too. - The [dispatches that follow a compute queue's wait](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/DispatchAhead.cpp) (libSceAgcDriver), up to the next wait that memory does not satisfy, a packet that writes guest memory other than a label, or a conditional packet, are resolved (memory captured, shader recompiled) when the wait passes, as the CP reads them right after it; the graphics queue is exempt (`APS5_GROUP_CAPTURE_ALL_QUEUES=1` includes it). The device still reads guest memory when the dispatch is issued; when that memory no longer matches the capture, the dispatch is resolved again from current memory. - [v_mullit_f32](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) follows the hardware measured with the MODE register at IEEE=1 and denormals kept (a signaling NaN src0 is quieted only with IEEE=1, as measured at IEEE=0 too; a positive denormal src2 counts as positive); with f32 denormals flushed it follows the f32 denormals entry below, and denormal modes 1 and 2 throw. From 9a3e55b8e8d0148e96eef5df6e821313d72fe0ef Mon Sep 17 00:00:00 2001 From: Renan Pires Date: Fri, 9 Oct 2026 21:41:10 -0300 Subject: [PATCH 080/460] feat(libkernel): implement _fstatfs --- core/libs/prx/libkernel/CMakeLists.txt | 10 ++ core/libs/prx/libkernel/File/src/Stdio.cpp | 110 +++++++++++++++++ core/libs/tests/GuestFstatfs.cpp | 132 +++++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 253 insertions(+) create mode 100644 core/libs/tests/GuestFstatfs.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index 6938ffc9a..429cff789 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -782,6 +782,16 @@ if(BUILD_TESTING) set_tests_properties(guest_file_vector PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_fstatfs_tests ${LIBS_INCLUDE_DIR}/tests/GuestFstatfs.cpp) + target_include_directories(guest_fstatfs_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_fstatfs_tests PRIVATE libkernel libc) + configure_windows_unwind(guest_fstatfs_tests) + add_test(NAME guest_fstatfs COMMAND guest_fstatfs_tests) + set_tests_properties(guest_fstatfs PROPERTIES TIMEOUT 15) + if(WIN32) + set_tests_properties(guest_fstatfs PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_sandbox_random_word_tests ${LIBS_INCLUDE_DIR}/tests/GuestSandboxRandomWord.cpp) target_include_directories(guest_sandbox_random_word_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_sandbox_random_word_tests PRIVATE libkernel libc) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index 711b9f388..2b1b97320 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -30,6 +30,41 @@ struct KernelIovec { static constexpr int KERNEL_IOV_MAX = 1024; +struct KernelStatfs { + std::uint32_t f_version; + std::uint32_t f_type; + std::uint64_t f_flags; + std::uint64_t f_bsize; + std::uint64_t f_iosize; + std::uint64_t f_blocks; + std::uint64_t f_bfree; + std::int64_t f_bavail; + std::uint64_t f_files; + std::int64_t f_ffree; + std::uint64_t f_syncwrites; + std::uint64_t f_asyncwrites; + std::uint64_t f_syncreads; + std::uint64_t f_asyncreads; + std::uint64_t f_spare[10]; + std::uint32_t f_namemax; + std::uint32_t f_owner; + std::int32_t f_fsid[2]; + char f_charspare[80]; + char f_fstypename[16]; + char f_mntfromname[88]; + char f_mntonname[88]; +}; +static_assert(sizeof(KernelStatfs) == 472); +static_assert(offsetof(KernelStatfs, f_bavail) == 48); +static_assert(offsetof(KernelStatfs, f_namemax) == 184); +static_assert(offsetof(KernelStatfs, f_fstypename) == 280); +static_assert(offsetof(KernelStatfs, f_mntonname) == 384); + +static constexpr std::uint32_t KERNEL_STATFS_VERSION = 0x20030518; +static constexpr std::uint64_t KERNEL_MNT_RDONLY = 0x1; +static constexpr std::uint64_t KERNEL_MNT_NOEXEC = 0x4; +static constexpr std::uint64_t KERNEL_MNT_NOSUID = 0x8; + #ifdef _WIN32 #include #include @@ -237,6 +272,68 @@ static int PosixResult(int result) { return result < 0 ? PosixFailure(result & 0xffff) : result; } +#ifdef _WIN32 +static int NativeFstatfs(int descriptor, KernelStatfs* fs) { + if (!File::DirectoryDescriptorPath(descriptor)) { + const auto handle = reinterpret_cast(::_get_osfhandle(descriptor)); + if (handle == INVALID_HANDLE_VALUE) return GUEST_EBADF; + const DWORD type = ::GetFileType(handle); + if (type == FILE_TYPE_PIPE) return GUEST_EINVAL; + if (type != FILE_TYPE_DISK) + throw std::runtime_error("_fstatfs: descriptor " + std::to_string(descriptor) + " is not a file on a Windows volume"); + } + const auto path = NativeDescriptorPath(descriptor); + if (!path) throw std::runtime_error("_fstatfs: no path for descriptor " + std::to_string(descriptor) + ", errno=" + std::to_string(errno)); + std::wstring root(path->native().size() + 2, L'\0'); + if (!::GetVolumePathNameW(path->c_str(), root.data(), static_cast(root.size()))) + throw std::runtime_error("_fstatfs: GetVolumePathNameW failed for " + path->string() + ", error=" + std::to_string(::GetLastError())); + DWORD sectorsPerCluster = 0, bytesPerSector = 0, freeClusters = 0, totalClusters = 0; + ULARGE_INTEGER available{}, total{}, totalFree{}; + DWORD nameMax = 0, volumeFlags = 0; + if (!::GetDiskFreeSpaceW(root.c_str(), §orsPerCluster, &bytesPerSector, &freeClusters, &totalClusters) || + !::GetDiskFreeSpaceExW(root.c_str(), &available, &total, &totalFree) || + !::GetVolumeInformationW(root.c_str(), nullptr, 0, nullptr, &nameMax, &volumeFlags, nullptr, 0)) + throw std::runtime_error("_fstatfs: volume query failed for " + path->string() + ", error=" + std::to_string(::GetLastError())); + const std::uint64_t cluster = static_cast(sectorsPerCluster) * bytesPerSector; + if (cluster == 0) throw std::runtime_error("_fstatfs: volume of " + path->string() + " reports no cluster size"); + fs->f_flags = (volumeFlags & FILE_READ_ONLY_VOLUME) ? KERNEL_MNT_RDONLY : 0; + fs->f_bsize = cluster; + fs->f_iosize = cluster; + fs->f_blocks = total.QuadPart / cluster; + fs->f_bfree = totalFree.QuadPart / cluster; + fs->f_bavail = static_cast(available.QuadPart / cluster); + fs->f_namemax = nameMax; + return 0; +} +#else +#include +#include +static int NativeFstatfs(int descriptor, KernelStatfs* fs) { + constexpr long PipeFilesystem = 0x50495045; + struct stat file{}; + if (::fstat(descriptor, &file) != 0) return SceErrorFromErrno(errno) & 0xffff; + if (S_ISSOCK(file.st_mode)) return GUEST_EINVAL; + if (S_ISFIFO(file.st_mode)) { + struct statfs kind{}; + if (::fstatfs(descriptor, &kind) == 0 && static_cast(kind.f_type) == PipeFilesystem) return GUEST_EINVAL; + } + struct statvfs host{}; + if (::fstatvfs(descriptor, &host) != 0) return SceErrorFromErrno(errno) & 0xffff; + if (host.f_flag & ST_RDONLY) fs->f_flags |= KERNEL_MNT_RDONLY; + if (host.f_flag & ST_NOSUID) fs->f_flags |= KERNEL_MNT_NOSUID; + if (host.f_flag & ST_NOEXEC) fs->f_flags |= KERNEL_MNT_NOEXEC; + fs->f_bsize = host.f_frsize != 0 ? host.f_frsize : host.f_bsize; + fs->f_iosize = host.f_bsize; + fs->f_blocks = host.f_blocks; + fs->f_bfree = host.f_bfree; + fs->f_bavail = static_cast(host.f_bavail); + fs->f_files = host.f_files; + fs->f_ffree = static_cast(host.f_favail); + fs->f_namemax = static_cast(host.f_namemax); + return 0; +} +#endif + extern "C" int APS5_VABI pipe_nid_postfix(int* descriptors) { if (!descriptors) return PosixFailure(GUEST_EFAULT); const GuestArena::HostWrite destination(descriptors, 2 * sizeof(int)); @@ -303,6 +400,19 @@ int APS5_VABI flock_nid_postfix(int d, int operation) { return 0; } +int APS5_VABI _fstatfs_nid_postfix(int d, KernelStatfs* buf) { + if (d >= GuestSockets::FirstDescriptor) return PosixFailure(GuestSockets::IsOpen(d) ? GUEST_EINVAL : GUEST_EBADF); + if (d < 0) return PosixFailure(GUEST_EBADF); + KernelStatfs result{}; + result.f_version = KERNEL_STATFS_VERSION; + if (const int error = NativeFstatfs(d, &result)) return PosixFailure(error); + if (buf == nullptr) return PosixFailure(GUEST_EFAULT); + const GuestArena::HostWrite destination(buf, sizeof(result)); + if (!destination.Open()) return PosixFailure(GUEST_EFAULT); + std::memcpy(buf, &result, sizeof(result)); + return 0; +} + int64_t APS5_VABI fstat_nid_disambig1_nid_postfix(int d, FileStat* sb) { if (sb == nullptr) { APS5_INVALID_ARG_EX; diff --git a/core/libs/tests/GuestFstatfs.cpp b/core/libs/tests/GuestFstatfs.cpp new file mode 100644 index 000000000..cc06e5e21 --- /dev/null +++ b/core/libs/tests/GuestFstatfs.cpp @@ -0,0 +1,132 @@ +#include "prx/libc/include/General.hpp" +#include "prx/libkernel/File/include/FileFlags.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include + +struct GuestStatfs { + std::uint32_t f_version; + std::uint32_t f_type; + std::uint64_t f_flags; + std::uint64_t f_bsize; + std::uint64_t f_iosize; + std::uint64_t f_blocks; + std::uint64_t f_bfree; + std::int64_t f_bavail; + std::uint64_t f_files; + std::int64_t f_ffree; + std::uint64_t f_syncwrites; + std::uint64_t f_asyncwrites; + std::uint64_t f_syncreads; + std::uint64_t f_asyncreads; + std::uint64_t f_spare[10]; + std::uint32_t f_namemax; + std::uint32_t f_owner; + std::int32_t f_fsid[2]; + char f_charspare[80]; + char f_fstypename[16]; + char f_mntfromname[88]; + char f_mntonname[88]; +}; +static_assert(sizeof(GuestStatfs) == 472); + +extern "C" { +int APS5_VABI _fstatfs_nid_postfix(int, GuestStatfs*); +int APS5_VABI sceKernelOpen(const char*, int, std::uint16_t); +int APS5_VABI close_nid_postfix(int); +int APS5_VABI pipe_nid_postfix(int*); +int APS5_VABI socketpair_nid_postfix(int, int, int, int*); +int* APS5_VABI __error_nid_postfix(); +} + +static void Check(bool value, int line) { + if (!value) { + std::fprintf(stderr, "fstatfs check failed at line %d\n", line); + std::abort(); + } +} +#define Require(value) Check((value), __LINE__) + +static constexpr std::uint32_t StatfsVersion = 0x20030518; +static constexpr int GuestEbadf = 9; +static constexpr int GuestEfault = 14; +static constexpr int GuestEinval = 22; +static constexpr int FirstSocket = 0x10000000; + +static void RequireFailure(int descriptor, int error) { + GuestStatfs fs; + std::memset(&fs, 0x5a, sizeof(fs)); + *__error_nid_postfix() = 0; + Require(_fstatfs_nid_postfix(descriptor, &fs) == -1); + Require(*__error_nid_postfix() == error); + for (std::size_t i = 0; i < sizeof(fs); ++i) Require(reinterpret_cast(&fs)[i] == 0x5a); +} + +static void RequireVolume(int descriptor, const std::filesystem::path& path) { + GuestStatfs fs; + std::memset(&fs, 0x5a, sizeof(fs)); + Require(_fstatfs_nid_postfix(descriptor, &fs) == 0); + const auto space = std::filesystem::space(path); + Require(fs.f_version == StatfsVersion); + Require(fs.f_bsize > 0 && fs.f_iosize > 0); + Require(fs.f_blocks > 0 && fs.f_bfree <= fs.f_blocks); + Require(fs.f_bavail >= 0 && static_cast(fs.f_bavail) <= fs.f_bfree); + Require(fs.f_blocks * fs.f_bsize <= space.capacity && space.capacity - fs.f_blocks * fs.f_bsize < fs.f_bsize); + Require((fs.f_flags & 0x1) == 0); + Require(fs.f_namemax >= 255); + Require(fs.f_fstypename[0] == '\0' && fs.f_mntfromname[0] == '\0' && fs.f_mntonname[0] == '\0'); +} + +int main() { + const auto root = std::filesystem::temp_directory_path() / + ("anyps5-fstatfs-" + std::to_string(std::chrono::steady_clock::now().time_since_epoch().count())); + Require(std::filesystem::create_directory(root)); + const auto filePath = root / "file.txt"; + { + std::ofstream file(filePath, std::ios::binary); + file << "fstatfs"; + Require(static_cast(file)); + } + + AddPathAlias_nid_no_patch("/anyps5-fstatfs-test", root.string().c_str()); + const int file = sceKernelOpen("/anyps5-fstatfs-test/file.txt", SCE_KERNEL_O_RDONLY, 0); + Require(file >= 0); + RequireVolume(file, filePath); + *__error_nid_postfix() = 0; + Require(_fstatfs_nid_postfix(file, nullptr) == -1); + Require(*__error_nid_postfix() == GuestEfault); + + const int directory = sceKernelOpen("/anyps5-fstatfs-test", SCE_KERNEL_O_RDONLY | SCE_KERNEL_O_DIRECTORY, 0); + Require(directory >= 0); + RequireVolume(directory, root); + Require(close_nid_postfix(directory) == 0); + + int pipe[2]; + Require(pipe_nid_postfix(pipe) == 0); + RequireFailure(pipe[0], GuestEinval); + RequireFailure(pipe[1], GuestEinval); + Require(close_nid_postfix(pipe[0]) == 0); + Require(close_nid_postfix(pipe[1]) == 0); + + int sockets[2]; + Require(socketpair_nid_postfix(1, 1, 0, sockets) == 0); + Require(sockets[0] >= FirstSocket); + RequireFailure(sockets[0], GuestEinval); + Require(close_nid_postfix(sockets[0]) == 0); + Require(close_nid_postfix(sockets[1]) == 0); + RequireFailure(sockets[0], GuestEbadf); + + RequireFailure(-1, GuestEbadf); + Require(close_nid_postfix(file) == 0); +#ifndef _WIN32 + RequireFailure(file, GuestEbadf); +#endif + + std::filesystem::remove_all(root); +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..de1344b5c 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -469,3 +469,4 @@ Throughout the project, every function at every stage either **does exactly what - [kill](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) follows FreeBSD's `kern_kill` and `killpg1` for the only guest process: `EINVAL` for signals outside 0 to 128, signal 0 only checks that the target exists, and its own pid, 0 and minus its own pid (the guest is taken to lead its own process group) signal it through `raise`, so they share `raise`'s limits above. Any other pid, and -1, which skips the caller, report `ESRCH`, so `EPERM` is never reported - [AJM](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp), [APR](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) and [NGS2](../../core/libs/prx/libSceNgs2.native/src/System.cpp) state is allocated with `new` and never freed on purpose: title threads still call into these libraries while `std::exit` runs static destructors, so ordinary statics would be destroyed under them. Keep them leaked; `guest_ajm_teardown` covers the AJM case on Unix. - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. +- [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields From 1ab577f8179c643465d012dd4b9a8e345d82262c Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 13:08:02 -0400 Subject: [PATCH 081/460] fix(shader): follow IEEE_MODE in v_div_fmas_f32 NaN selection v_div_fmas_f32 always quieted source NaNs and let a NaN addend hide the default NaN from an invalid product. Reuse the existing bit selection helpers, measured on gfx1036 in four float modes. Co-Authored-By: GPT-6.1 Sol --- .../tests/execution/Division.cpp | 54 +++++++++++++++++-- .../Translation/src/FloatInstructions.cpp | 8 +-- 2 files changed, 52 insertions(+), 10 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/Division.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/Division.cpp index 09059269b..ae9a497a7 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/Division.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/Division.cpp @@ -7,6 +7,7 @@ #include #include #include +#include #include #include #include @@ -574,6 +575,29 @@ constexpr DivisionVector DivisionVectors[] = { {0x2210592fu, 0x0ac8fdd3u, 0x56b7dabeu}, }; +struct FmasNanVector { + std::uint32_t s0, s1, s2, noIeee, ieee; +}; + +constexpr FmasNanVector FmasNanVectors[] = { + {0xff812345u, 0x3f800000u, 0x3f800000u, 0xff812345u, 0xffc12345u}, + {0x7f812345u, 0x3f800000u, 0x3f800000u, 0x7f812345u, 0x7fc12345u}, + {0x3f800000u, 0xff812345u, 0x3f800000u, 0xff812345u, 0xffc12345u}, + {0x3f800000u, 0x3f800000u, 0xff812345u, 0xff812345u, 0xffc12345u}, + {0x7fc12345u, 0xff812345u, 0xff812345u, 0x7fc12345u, 0x7fc12345u}, + {0xff812345u, 0x7fc12345u, 0x7f812345u, 0xff812345u, 0xffc12345u}, + {0x3f800000u, 0x7fc12345u, 0xff812345u, 0x7fc12345u, 0x7fc12345u}, + {0x3f800000u, 0x3f800000u, 0x7fc12345u, 0x7fc12345u, 0x7fc12345u}, + {0x7f800000u, 0x00000000u, 0x7fc12345u, 0xffc00000u, 0xffc00000u}, + {0xff800000u, 0x80000000u, 0xff812345u, 0xffc00000u, 0xffc00000u}, + {0x00000000u, 0xff800000u, 0x7f812345u, 0xffc00000u, 0xffc00000u}, + {0x80000000u, 0x7f800000u, 0xffc12345u, 0xffc00000u, 0xffc00000u}, + {0x7f800000u, 0x3f800000u, 0xff800000u, 0xffc00000u, 0xffc00000u}, + {0x3f800000u, 0x7f7fffffu, 0xff7fffffu, 0x00000000u, 0x00000000u}, + {0x3f800000u, 0xf1800000u, 0x71800000u, 0x00000000u, 0x00000000u}, + {0x7f800000u, 0x3f800000u, 0x3f800000u, 0x7f800000u, 0x7f800000u}, +}; + std::array BufferDescriptor(const void* data, std::uint32_t count) { const auto address = reinterpret_cast(data); return {static_cast(address), static_cast((address >> 32u) & 0xffffu) | (4u << 16u), count, 0x11016facu}; @@ -581,7 +605,8 @@ std::array BufferDescriptor(const void* data, std::uint32_t co class Shader { public: - Shader(AgcDriver::VulkanDevice& device, std::span code) : device(device), code(code) { + Shader(AgcDriver::VulkanDevice& device, std::span code, + const std::optional& floatMode = ShaderRecompiler::ShaderFloatMode{0xf0u, true, true, false}) : device(device), code(code) { std::vector userData(8, 0u); const auto input = BufferDescriptor(Input.data(), static_cast(Input.size())); const auto output = BufferDescriptor(Output.data(), static_cast(Output.size())); @@ -596,7 +621,7 @@ public: {0, 0, 0, 128} }; request.useCache = false; - request.context.floatMode = ShaderRecompiler::ShaderFloatMode{0xf0u, true, true, false}; + request.context.floatMode = floatMode; result = ShaderRecompiler::Recompile(request); } @@ -651,6 +676,24 @@ void CheckHelpers(AgcDriver::VulkanDevice& device, bool fused) { } } +void CheckFmasNan(AgcDriver::VulkanDevice& device, const std::optional& floatMode) { + Shader shader(device, HelperCode, floatMode); + Input.fill(0u); + for (std::uint32_t lane = 0; lane < std::size(FmasNanVectors) * 2u; ++lane) { + const auto& vector = FmasNanVectors[lane / 2u]; + std::copy_n(std::array{vector.s0, vector.s1, vector.s2, lane & 1u}.begin(), 4, Input.begin() + lane * Stride); + } + shader.Run(); + const bool ieee = floatMode.has_value() && floatMode->ieeeMode; + for (std::uint32_t lane = 0; lane < std::size(FmasNanVectors) * 2u; ++lane) { + const auto& vector = FmasNanVectors[lane / 2u]; + const auto actual = Output[lane * Stride + 2u]; + const auto expected = ieee ? vector.ieee : vector.noIeee; + const std::string mode = floatMode.has_value() ? Hex(floatMode->floatMode) + " IEEE=" + std::to_string(ieee) : "no float mode"; + Require(actual == expected, "division fmas nan: lane " + std::to_string(lane) + " " + mode + " vcc " + std::to_string(lane & 1u) + ": v_div_fmas_f32 is " + Hex(actual) + ", expected " + Hex(expected)); + } +} + void CheckDivision(AgcDriver::VulkanDevice& device) { Shader shader(device, DivisionCode); constexpr std::uint32_t count = sizeof(DivisionVectors) / sizeof(DivisionVectors[0]); @@ -677,10 +720,15 @@ int main() { if (!device) return VulkanTestSkipped; const bool fused = HostFusesFma(*device); CheckHelpers(*device, fused); + CheckFmasNan(*device, std::nullopt); + CheckFmasNan(*device, ShaderRecompiler::ShaderFloatMode{0xf0u, true, true, false}); + CheckFmasNan(*device, ShaderRecompiler::ShaderFloatMode{0xc0u, true, false, false}); + CheckFmasNan(*device, ShaderRecompiler::ShaderFloatMode{0xf0u, false, false, false}); + CheckFmasNan(*device, ShaderRecompiler::ShaderFloatMode{0xc0u, false, true, false}); if (fused) { CheckDivision(*device); } else { - std::puts("the device splits fma into a multiply and an add, so v_div_fmas_f32 and the division macro are not checked"); + std::puts("the device splits fma into a multiply and an add, so only the directed v_div_fmas_f32 rows are checked and the division macro is not checked"); } std::puts("division tests passed"); return 0; diff --git a/core/shader/recompiler/Translation/src/FloatInstructions.cpp b/core/shader/recompiler/Translation/src/FloatInstructions.cpp index b85979973..28da1b27b 100644 --- a/core/shader/recompiler/Translation/src/FloatInstructions.cpp +++ b/core/shader/recompiler/Translation/src/FloatInstructions.cpp @@ -485,14 +485,8 @@ bool TranslationContext::vDivFmasF32(const RdnaInstruction& inst) { &ir.Emit(IrOpcode::FPMul32, IrType::F32, {addend, &power})}); IrValue& afterwards = ir.Emit(IrOpcode::FPMul32, IrType::F32, {&plain, &power}); IrValue& scaledResult = ir.Emit(IrOpcode::SelectF32, IrType::F32, {&rescale.Value(), &rescaled, &afterwards}); - const auto isNan = [&](IrValue& value) { return IrU1(ir.UGreaterThan(ir.BitwiseAnd(ir.BitCastU32(value), ir.Constant(0x7fffffffu)), ir.Constant(0x7f800000u))); }; - const auto quiet = [&](IrValue& value) { return IrU32(ir.BitwiseOr(ir.BitCastU32(value), ir.Constant(0x400000u))); }; IrValue& computed = ir.Emit(IrOpcode::SelectF32, IrType::F32, {&scale.Value(), &scaledResult, &plain}); - IrU32 result(ir.Select(isNan(computed).Value(), ir.Constant(0xffc00000u), ir.BitCastU32(computed))); - result = IrU32(ir.Select(isNan(*addend).Value(), quiet(*addend).Value(), result.Value())); - result = IrU32(ir.Select(isNan(*rhs).Value(), quiet(*rhs).Value(), result.Value())); - result = IrU32(ir.Select(isNan(*lhs).Value(), quiet(*lhs).Value(), result.Value())); - writeOperand(inst.destination, &ir.BitCastF32(result.Value())); + writeOperand(inst.destination, nanResultF32({lhs, rhs, addend}, &computed, &invalidProductF32(lhs, rhs))); return true; } From e5a81a3ca9a517f1115b909fb4ebb8fb5a92545a Mon Sep 17 00:00:00 2001 From: arcwiser Date: Sat, 10 Oct 2026 12:11:09 +1100 Subject: [PATCH 082/460] test(videoout): isolate application fixtures (cherry picked from commit 7e982a521f2ba5fafabcba45c78cb23918fa9007) --- core/libs/prx/libSceVideoOut/tests/Flip.cpp | 6 +-- .../tests/GuestVideoOutResolutionStatus.cpp | 2 + .../libs/tests/GuestVideoOutWqhdDetection.cpp | 10 +--- core/libs/tests/VideoOutTestEnvironment.hpp | 53 +++++++++++++++++++ 4 files changed, 59 insertions(+), 12 deletions(-) create mode 100644 core/libs/tests/VideoOutTestEnvironment.hpp diff --git a/core/libs/prx/libSceVideoOut/tests/Flip.cpp b/core/libs/prx/libSceVideoOut/tests/Flip.cpp index fa4a07c6f..02ff20088 100644 --- a/core/libs/prx/libSceVideoOut/tests/Flip.cpp +++ b/core/libs/prx/libSceVideoOut/tests/Flip.cpp @@ -7,6 +7,7 @@ #include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" #include "prx/libSceAgcDriver/Submit/include/Dcb.hpp" #include "prx/libc/include/Shutdown.hpp" +#include "tests/VideoOutTestEnvironment.hpp" #include #include #include @@ -14,8 +15,6 @@ #include #include #include -#include -#include #include #include #include @@ -657,9 +656,8 @@ void testOneDevice() { } int main(int argc, char** argv) { + VideoOutTestEnvironment environment; try { - std::filesystem::create_directories("app0/sce_sys"); - std::ofstream("app0/sce_sys/param.json", std::ios::binary) << R"({"titleId":"PPSA00000","localizedParameters":{"en-US":{"titleName":"Example"}},"downloadDataSize":0})"; if (argc == 2 && std::string(argv[1]) == "decode") testDecode(); else if (argc == 2 && std::string(argv[1]) == "controls") testControls(); else if (argc == 2 && std::string(argv[1]) == "present") testPresentation(); diff --git a/core/libs/tests/GuestVideoOutResolutionStatus.cpp b/core/libs/tests/GuestVideoOutResolutionStatus.cpp index 0e81b766c..5294d7c29 100644 --- a/core/libs/tests/GuestVideoOutResolutionStatus.cpp +++ b/core/libs/tests/GuestVideoOutResolutionStatus.cpp @@ -1,5 +1,6 @@ #include "SceTypes.hpp" #include "prx/libc/include/Shutdown.hpp" +#include "tests/VideoOutTestEnvironment.hpp" #include #include #include @@ -28,6 +29,7 @@ static bool Rejects(int handle, VideoOutResolutionStatus* status) { } int main() { + VideoOutTestEnvironment environment; int handle = 0; try { handle = sceVideoOutOpen(SYSTEM_USER, MAIN_BUS, 0, nullptr); diff --git a/core/libs/tests/GuestVideoOutWqhdDetection.cpp b/core/libs/tests/GuestVideoOutWqhdDetection.cpp index 0cb8a54ec..c0e68b0c4 100644 --- a/core/libs/tests/GuestVideoOutWqhdDetection.cpp +++ b/core/libs/tests/GuestVideoOutWqhdDetection.cpp @@ -1,10 +1,9 @@ #include "SceTypes.hpp" #include "prx/libc/include/Shutdown.hpp" +#include "tests/VideoOutTestEnvironment.hpp" #include #include #include -#include -#include #include extern "C" { @@ -34,12 +33,7 @@ static bool SameStatus(const VideoOutOutputStatus& a, const VideoOutOutputStatus } int main() { - std::filesystem::create_directories("app0/sce_sys"); - { - std::ofstream param("app0/sce_sys/param.json", std::ios::binary); - param << R"({"titleId":"PPSA00000","localizedParameters":{"en-US":{"titleName":"Example"}},"downloadDataSize":0})"; - Require(static_cast(param)); - } + VideoOutTestEnvironment environment; int handle = 0; try { handle = sceVideoOutOpen(SYSTEM_USER, MAIN_BUS, 0, nullptr); diff --git a/core/libs/tests/VideoOutTestEnvironment.hpp b/core/libs/tests/VideoOutTestEnvironment.hpp new file mode 100644 index 000000000..786cf5e32 --- /dev/null +++ b/core/libs/tests/VideoOutTestEnvironment.hpp @@ -0,0 +1,53 @@ +#pragma once + +#include +#include +#include +#include +#include +#include +#include +#include + +class VideoOutTestEnvironment final { + std::filesystem::path previous; + std::filesystem::path root; + +public: + VideoOutTestEnvironment() : previous(std::filesystem::current_path()) { + const auto seed = static_cast(std::chrono::steady_clock::now().time_since_epoch().count()) ^ + (static_cast(std::random_device{}()) << 32u) ^ std::random_device{}(); + for (std::uint32_t attempt = 0; attempt < 64; ++attempt) { + const auto candidate = std::filesystem::temp_directory_path() / + ("anyps5-video-out-" + std::to_string(seed + attempt)); + std::error_code error; + if (std::filesystem::create_directory(candidate, error)) { + root = candidate; + break; + } + if (error) throw std::runtime_error("Cannot create VideoOut test directory: " + error.message()); + } + if (root.empty()) throw std::runtime_error("Cannot allocate a unique VideoOut test directory"); + try { + std::filesystem::current_path(root); + std::filesystem::create_directories("app0/sce_sys"); + std::ofstream param("app0/sce_sys/param.json", std::ios::binary); + param << R"({"titleId":"PPSA00000","localizedParameters":{"en-US":{"titleName":"Example"}},"downloadDataSize":0})"; + if (!param) throw std::runtime_error("Cannot write VideoOut param.json fixture"); + } catch (...) { + std::error_code error; + std::filesystem::current_path(previous, error); + std::filesystem::remove_all(root, error); + throw; + } + } + + ~VideoOutTestEnvironment() { + std::error_code error; + std::filesystem::current_path(previous, error); + std::filesystem::remove_all(root, error); + } + + VideoOutTestEnvironment(const VideoOutTestEnvironment&) = delete; + VideoOutTestEnvironment& operator=(const VideoOutTestEnvironment&) = delete; +}; From b74774f713349932e431bd1df5e8282e99dd923a Mon Sep 17 00:00:00 2001 From: aplatogg Date: Sat, 10 Oct 2026 19:51:19 +0700 Subject: [PATCH 083/460] fix(relinker): support non-ASCII input paths on Windows The MinGW CRT delivers argv to main as ANSI code page bytes, and every narrow string became a filesystem path through the same code page, so a game in a directory with characters outside the code page failed to open (#2692). Read the command line as Unicode (wmain via -municode), keep paths as UTF-8 strings, convert at the filesystem boundaries through Io::NativePath and Io::Utf8Path, and run the --autorun output through CreateProcessW. (cherry picked from commit 30a5181f8a71b177cf2dc03334e39998938b3472) --- core/relinker/CMakeLists.txt | 14 +++- core/relinker/cli/src/Autorun.cpp | 43 ++++++++-- core/relinker/io/include/io/NativePath.hpp | 14 ++++ core/relinker/io/src/FileReader.cpp | 6 +- core/relinker/io/src/FileWriter.cpp | 5 +- core/relinker/io/src/NativePath.cpp | 46 ++++++++++ core/relinker/main.cpp | 83 +++++++++++++++---- .../relinker/src/guest/GuestModuleBuilder.cpp | 3 +- .../relinker/tests/test_input_nonascii.py | 33 ++++++++ 9 files changed, 216 insertions(+), 31 deletions(-) create mode 100644 core/relinker/io/include/io/NativePath.hpp create mode 100644 core/relinker/io/src/NativePath.cpp create mode 100644 core/relinker/relinker/tests/test_input_nonascii.py diff --git a/core/relinker/CMakeLists.txt b/core/relinker/CMakeLists.txt index 288dddef6..d6fed3b85 100644 --- a/core/relinker/CMakeLists.txt +++ b/core/relinker/CMakeLists.txt @@ -33,6 +33,7 @@ add_executable(relinker ${CMAKE_CURRENT_SOURCE_DIR}/io/src/BufferUtils.cpp ${CMAKE_CURRENT_SOURCE_DIR}/io/src/FileReader.cpp ${CMAKE_CURRENT_SOURCE_DIR}/io/src/FileWriter.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/io/src/NativePath.cpp ${CMAKE_CURRENT_SOURCE_DIR}/codegen/src/Amd64OnlyConverter.cpp ${CMAKE_CURRENT_SOURCE_DIR}/codegen/src/InstructionScanner.cpp ${CMAKE_CURRENT_SOURCE_DIR}/codegen/src/x86/X64InstructionDecoder.cpp @@ -77,6 +78,11 @@ target_include_directories(relinker PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/relinker/include ) +if(WIN32) + target_compile_options(relinker PRIVATE -municode) + target_link_options(relinker PRIVATE -municode) +endif() + if(WIN32) add_test_executable(windows_dependency_diagnostics_tests EXCLUDE_FROM_ALL ${CMAKE_CURRENT_SOURCE_DIR}/elfpatcher/tests/WindowsDependencyDiagnosticsTests.cpp @@ -130,8 +136,9 @@ endif() if(BUILD_TESTING) add_test_executable(autorun_exit_tests ${CMAKE_CURRENT_SOURCE_DIR}/cli/tests/AutorunExitTests.cpp - ${CMAKE_CURRENT_SOURCE_DIR}/cli/src/Autorun.cpp) - target_include_directories(autorun_exit_tests PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/cli/include) + ${CMAKE_CURRENT_SOURCE_DIR}/cli/src/Autorun.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/io/src/NativePath.cpp) + target_include_directories(autorun_exit_tests PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/cli/include ${CMAKE_CURRENT_SOURCE_DIR}/io/include ${CMAKE_CURRENT_SOURCE_DIR}/domain/include) add_test(NAME autorun_exit_status COMMAND autorun_exit_tests) set_tests_properties(autorun_exit_status PROPERTIES TIMEOUT 20) add_test_executable(cli_args_tests @@ -152,6 +159,7 @@ if(BUILD_TESTING) add_test_executable(file_writer_tests ${CMAKE_CURRENT_SOURCE_DIR}/io/tests/FileWriterTests.cpp ${CMAKE_CURRENT_SOURCE_DIR}/io/src/FileWriter.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/io/src/NativePath.cpp ) target_include_directories(file_writer_tests PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/domain/include @@ -300,6 +308,8 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_input_preservation.py $) add_test(NAME input_file COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_input_file.py $) + add_test(NAME input_nonascii COMMAND ${Python3_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_input_nonascii.py $) add_test(NAME string_table_bounds COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_string_table_bounds.py $) add_test(NAME call_registry COMMAND ${Python3_EXECUTABLE} diff --git a/core/relinker/cli/src/Autorun.cpp b/core/relinker/cli/src/Autorun.cpp index 0660289e4..7030a03da 100644 --- a/core/relinker/cli/src/Autorun.cpp +++ b/core/relinker/cli/src/Autorun.cpp @@ -1,23 +1,55 @@ #include +#include +#include #include #include #include #include #include +#ifdef _WIN32 +#include +#endif + namespace Cli { int Autorun(const std::string& absPath, bool toWindows) { if (!toWindows) { - std::filesystem::permissions(absPath, + std::filesystem::permissions(Io::NativePath(absPath), std::filesystem::perms::owner_exec | std::filesystem::perms::group_exec | std::filesystem::perms::others_exec, std::filesystem::perm_options::add); } + int rawCode = 0; + int exitCode = 0; + #ifdef _WIN32 - const std::string cmd = "\"" + absPath + "\""; + std::wstring command = L"\"" + Io::NativePath(absPath).wstring() + L"\""; + STARTUPINFOW startup{}; + startup.cb = sizeof(startup); + PROCESS_INFORMATION process{}; + if (!CreateProcessW(nullptr, command.data(), nullptr, nullptr, FALSE, 0, nullptr, nullptr, &startup, &process)) + throw Domain::RelinkerException("Cannot run the output executable: " + absPath); + if (WaitForSingleObject(process.hProcess, INFINITE) != WAIT_OBJECT_0) { + const unsigned long waitError = GetLastError(); + CloseHandle(process.hThread); + CloseHandle(process.hProcess); + throw Domain::RelinkerException("Cannot wait for the output executable: " + absPath + " (error " + std::to_string(waitError) + ")"); + } + unsigned long rawExitCode = 0; + if (!GetExitCodeProcess(process.hProcess, &rawExitCode)) { + const unsigned long exitError = GetLastError(); + CloseHandle(process.hThread); + CloseHandle(process.hProcess); + throw Domain::RelinkerException("Cannot read the output executable exit code: " + absPath + " (error " + std::to_string(exitError) + ")"); + } + CloseHandle(process.hThread); + CloseHandle(process.hProcess); + rawCode = static_cast(rawExitCode); + exitCode = rawCode; + std::cout << "\nExit code: " << rawCode << '\n'; #else std::string cmd = "'"; for (const char character : absPath) { @@ -25,13 +57,8 @@ int Autorun(const std::string& absPath, bool toWindows) { else cmd += character; } cmd += '\''; -#endif - const int rawCode = std::system(cmd.c_str()); + rawCode = std::system(cmd.c_str()); - int exitCode = rawCode; -#ifdef _WIN32 - std::cout << "\nExit code: " << rawCode << '\n'; -#else if (rawCode != -1) { const int signal = rawCode & 0x7F; exitCode = signal ? 128 + signal : (rawCode >> 8) & 0xFF; diff --git a/core/relinker/io/include/io/NativePath.hpp b/core/relinker/io/include/io/NativePath.hpp new file mode 100644 index 000000000..77ac098a1 --- /dev/null +++ b/core/relinker/io/include/io/NativePath.hpp @@ -0,0 +1,14 @@ +#ifndef IO_NATIVEPATH_HPP +#define IO_NATIVEPATH_HPP + +#include +#include + +namespace Io { + +std::filesystem::path NativePath(const std::string& path); +std::string Utf8Path(const std::filesystem::path& path); + +} + +#endif diff --git a/core/relinker/io/src/FileReader.cpp b/core/relinker/io/src/FileReader.cpp index 935676b6d..228903ab4 100644 --- a/core/relinker/io/src/FileReader.cpp +++ b/core/relinker/io/src/FileReader.cpp @@ -1,4 +1,5 @@ #include +#include #include #include #include @@ -6,10 +7,11 @@ namespace Io { std::vector FileReader::Read(const std::string& path) { + const auto native = NativePath(path); std::error_code error; - if (!std::filesystem::is_regular_file(path, error)) + if (!std::filesystem::is_regular_file(native, error)) throw Domain::RelinkerException("Cannot open file: " + path); - std::ifstream f(path, std::ios::binary | std::ios::ate); + std::ifstream f(native, std::ios::binary | std::ios::ate); if (!f) throw Domain::RelinkerException("Cannot open file: " + path); const std::streamsize size = f.tellg(); diff --git a/core/relinker/io/src/FileWriter.cpp b/core/relinker/io/src/FileWriter.cpp index a8c65fd63..e80448f50 100644 --- a/core/relinker/io/src/FileWriter.cpp +++ b/core/relinker/io/src/FileWriter.cpp @@ -1,11 +1,12 @@ #include +#include #include #include namespace Io { void FileWriter::Write(const std::string& path, const std::vector& data) { - std::ofstream f(path, std::ios::binary); + std::ofstream f(NativePath(path), std::ios::binary); if (!f) throw Domain::RelinkerException("Cannot open output file: " + path); f.write(reinterpret_cast(data.data()), static_cast(data.size())); @@ -15,7 +16,7 @@ void FileWriter::Write(const std::string& path, const std::vector& } void FileWriter::Write(const std::string& path, const std::string& content) { - std::ofstream f(path); + std::ofstream f(NativePath(path)); if (!f) throw Domain::RelinkerException("Cannot open output file: " + path); f << content; diff --git a/core/relinker/io/src/NativePath.cpp b/core/relinker/io/src/NativePath.cpp new file mode 100644 index 000000000..911b1026f --- /dev/null +++ b/core/relinker/io/src/NativePath.cpp @@ -0,0 +1,46 @@ +#include +#include + +#ifdef _WIN32 +#include +#include +#endif + +namespace Io { + +#ifdef _WIN32 + +std::filesystem::path NativePath(const std::string& path) { + if (path.empty()) return {}; + const int wideSize = MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, path.data(), static_cast(path.size()), nullptr, 0); + if (wideSize <= 0) throw Domain::RelinkerException("Path is not valid UTF-8: " + path); + std::wstring wide(static_cast(wideSize), L'\0'); + if (MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, path.data(), static_cast(path.size()), wide.data(), wideSize) != wideSize) + throw Domain::RelinkerException("Path is not valid UTF-8: " + path); + return std::filesystem::path(std::move(wide)); +} + +std::string Utf8Path(const std::filesystem::path& path) { + const std::wstring wide = path.wstring(); + if (wide.empty()) return {}; + const int utf8Size = WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, wide.data(), static_cast(wide.size()), nullptr, 0, nullptr, nullptr); + if (utf8Size <= 0) throw Domain::RelinkerException("Path cannot be encoded as UTF-8"); + std::string utf8(static_cast(utf8Size), '\0'); + if (WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, wide.data(), static_cast(wide.size()), utf8.data(), utf8Size, nullptr, nullptr) != utf8Size) + throw Domain::RelinkerException("Path cannot be encoded as UTF-8"); + return utf8; +} + +#else + +std::filesystem::path NativePath(const std::string& path) { + return std::filesystem::path(path); +} + +std::string Utf8Path(const std::filesystem::path& path) { + return path.string(); +} + +#endif + +} diff --git a/core/relinker/main.cpp b/core/relinker/main.cpp index 89564d51b..5bb9f2777 100644 --- a/core/relinker/main.cpp +++ b/core/relinker/main.cpp @@ -1,7 +1,7 @@ #include #include #include -#include +#include #include #include #include @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -22,13 +23,21 @@ #include #include #include +#include #include #include #include #include #include -int main(const int argc, char* argv[]) { +#ifdef _WIN32 +#include +#include +#endif + +namespace { + +int Run(const int argc, char* argv[]) { Cli::Args args; try { args = Cli::ParseArgs(argc, argv); @@ -43,16 +52,16 @@ int main(const int argc, char* argv[]) { } try { - auto extension = std::filesystem::path(args.outputPath).extension().string(); + auto extension = Io::Utf8Path(Io::NativePath(args.outputPath).extension()); for (auto& character : extension) if (character >= 'A' && character <= 'Z') character = static_cast(character + ('a' - 'A')); if (!args.toWindows && !args.toMacos && extension == ".exe") std::cerr << "WARNING: Output filename ends with .exe, but --windows was not specified. The output will be a Linux ELF executable.\n"; Io::FileReader fileReader; Io::FileWriter fileWriter; auto sourceBytes = fileReader.Read(args.inputPath); - if (std::filesystem::exists(args.outputPath) && std::filesystem::equivalent(args.inputPath, args.outputPath)) + if (std::filesystem::exists(Io::NativePath(args.outputPath)) && std::filesystem::equivalent(Io::NativePath(args.inputPath), Io::NativePath(args.outputPath))) throw Domain::RelinkerException("Executable output would overwrite the input executable"); - const std::string absPath = std::filesystem::absolute(args.outputPath).string(); + const std::string absPath = Io::Utf8Path(std::filesystem::absolute(Io::NativePath(args.outputPath))); std::vector trampolines; if (args.toIntel) { @@ -99,17 +108,17 @@ int main(const int argc, char* argv[]) { std::vector guestArtifacts; if (!args.skipSceModule) { - guestArtifacts = Relinker::GuestModuleBuilder().Build(args.inputPath, absPath, result.DynamicSection, args.toWindows, args.toMacos, args.toIntel, *syscallScanner, args.lazyBinding, args.runPath, args.excludedSceModules, args.sceModulePath); + guestArtifacts = Relinker::GuestModuleBuilder().Build(Io::NativePath(args.inputPath), Io::NativePath(absPath), result.DynamicSection, args.toWindows, args.toMacos, args.toIntel, *syscallScanner, args.lazyBinding, args.runPath, args.excludedSceModules, Io::NativePath(args.sceModulePath)); } if (args.writeRegistry) { - const std::filesystem::path outFsPath(absPath); - const std::string registryPath = (outFsPath.parent_path() / (outFsPath.stem().string() + ".registry.json")).string(); + const std::filesystem::path outFsPath = Io::NativePath(absPath); + const std::string registryPath = Io::Utf8Path(outFsPath.parent_path() / Io::NativePath(Io::Utf8Path(outFsPath.stem()) + ".registry.json")); const auto registryWriter = std::make_shared(); fileWriter.Write(registryPath, registryWriter->WriteCallRegistry(result.RegistryEntries)); for (const auto& artifact : guestArtifacts) { - const auto modulePath = outFsPath.parent_path() / (outFsPath.stem().string() + "." + artifact.Path.filename().string() + ".registry.json"); - fileWriter.Write(modulePath.string(), registryWriter->WriteModuleImports(artifact.Imports)); + const auto modulePath = outFsPath.parent_path() / Io::NativePath(Io::Utf8Path(outFsPath.stem()) + "." + Io::Utf8Path(artifact.Path.filename()) + ".registry.json"); + fileWriter.Write(Io::Utf8Path(modulePath), registryWriter->WriteModuleImports(artifact.Imports)); } } @@ -119,7 +128,7 @@ int main(const int argc, char* argv[]) { if (args.toMacos) { patcher = std::make_shared(); } else if (args.toWindows) { - patcher = std::make_shared(args.windowsGui, std::filesystem::path(args.inputPath).parent_path() / "sce_sys" / "icon0.png"); + patcher = std::make_shared(args.windowsGui, Io::NativePath(args.inputPath).parent_path() / "sce_sys" / "icon0.png"); } else { patcher = std::make_shared( std::make_shared(), @@ -141,16 +150,19 @@ int main(const int argc, char* argv[]) { } for (const auto& artifact : guestArtifacts) { std::filesystem::create_directories(artifact.Path.parent_path()); - fileWriter.Write(artifact.Path.string(), artifact.Bytes); - std::cout << "Guest module: " << artifact.Path.string() << '\n'; + fileWriter.Write(Io::Utf8Path(artifact.Path), artifact.Bytes); + std::cout << "Guest module: " << Io::Utf8Path(artifact.Path) << '\n'; } fileWriter.Write(absPath, executableBytes); std::cout << "External prx references: " << result.RegistryEntries.size() << "\nOutput file: " << absPath << '\n'; std::cout << "Expected runtime layout (relative to the output executable):\n" - << std::filesystem::path(absPath).filename().string() << "\n" + << Io::Utf8Path(Io::NativePath(absPath).filename()) << "\n" << "libs/\n *.prx\napp0/\n \n"; - for (const auto& artifact : guestArtifacts) - std::cout << " " << artifact.Path.lexically_relative(std::filesystem::path(absPath).parent_path() / "app0").generic_string() << '\n'; + for (const auto& artifact : guestArtifacts) { + std::string relative = Io::Utf8Path(artifact.Path.lexically_relative(Io::NativePath(absPath).parent_path() / "app0")); + std::replace(relative.begin(), relative.end(), '\\', '/'); + std::cout << " " << relative << '\n'; + } std::cout << "Game resources and system libraries must be placed in this layout separately.\n"; if (args.runPath != "$ORIGIN/libs") std::cout << "Custom library search path (--rpath): " << args.runPath << '\n'; @@ -174,3 +186,42 @@ int main(const int argc, char* argv[]) { return 0; } + +#ifdef _WIN32 + +std::string Utf8Argument(const wchar_t* argument) { + const int size = WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, argument, static_cast(wcslen(argument)), nullptr, 0, nullptr, nullptr); + if (size <= 0) throw std::runtime_error("Cannot encode the command line as UTF-8"); + std::string utf8(static_cast(size), '\0'); + if (WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, argument, static_cast(wcslen(argument)), utf8.data(), size, nullptr, nullptr) != size) + throw std::runtime_error("Cannot encode the command line as UTF-8"); + return utf8; +} + +} + +int wmain(const int argc, wchar_t* argv[]) { + try { + std::vector arguments; + arguments.reserve(static_cast(argc)); + for (int index = 0; index < argc; ++index) arguments.push_back(Utf8Argument(argv[index])); + std::vector pointers; + pointers.reserve(arguments.size() + 1); + for (auto& argument : arguments) pointers.push_back(argument.data()); + pointers.push_back(nullptr); + return Run(static_cast(arguments.size()), pointers.data()); + } catch (const std::exception& error) { + std::cerr << "FAIL: " << error.what() << "\n"; + return 1; + } +} + +#else + +} + +int main(const int argc, char* argv[]) { + return Run(argc, argv); +} + +#endif diff --git a/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp b/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp index 42dbc7403..9d5043867 100644 --- a/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp +++ b/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp @@ -2,6 +2,7 @@ #include #include #include +#include #include #include #include @@ -119,7 +120,7 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path if (std::filesystem::exists(outputPath)) { for (const auto& path : paths) if (std::filesystem::equivalent(path, outputPath)) - throw Domain::RelinkerException("Executable output would overwrite an input module: " + outputPath.string()); + throw Domain::RelinkerException("Executable output would overwrite an input module: " + Io::Utf8Path(outputPath)); } std::map> exports; std::map> sharedExports; diff --git a/core/relinker/relinker/tests/test_input_nonascii.py b/core/relinker/relinker/tests/test_input_nonascii.py new file mode 100644 index 000000000..6d9598878 --- /dev/null +++ b/core/relinker/relinker/tests/test_input_nonascii.py @@ -0,0 +1,33 @@ +"""Convert inputs whose directory and file names hold characters outside the local code page.""" + +from pathlib import Path +import subprocess +import sys +import tempfile + +from test_optional_plt import fixture + + +def main(): + relinker = Path(sys.argv[1]).resolve() + names = ["проба", "ゲーム", "próba m²"] + with tempfile.TemporaryDirectory(prefix="anyps5-nonascii-") as directory: + work = Path(directory) / "игры" + work.mkdir() + for name in names: + source = work / (name + ".elf") + source.write_bytes(bytes(fixture())) + output = work / (name + ".exe") + registry = work / (name + ".registry.json") + result = subprocess.run([str(relinker), "--skip-sce-module", "--registry", "--windows", str(source), str(output)], + capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=20) + assert result.returncode == 0, (name, result) + assert output.read_bytes()[:2] == b"MZ" and registry.exists(), (name, result) + output.unlink() + registry.unlink() + source.unlink() + print("Non-ASCII input path tests passed") + + +if __name__ == "__main__": + main() From 8cbebea6b7733e3eb7267d27796a2e3bc5d6a198 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 13:11:41 +0000 Subject: [PATCH 084/460] fix(relinker): preserve unicode module paths --- core/relinker/CMakeLists.txt | 9 +++- core/relinker/cli/src/CliArgs.cpp | 3 +- core/relinker/main.cpp | 3 +- .../relinker/src/guest/GuestModuleBuilder.cpp | 44 +++++++++---------- 4 files changed, 33 insertions(+), 26 deletions(-) diff --git a/core/relinker/CMakeLists.txt b/core/relinker/CMakeLists.txt index d6fed3b85..3de337d78 100644 --- a/core/relinker/CMakeLists.txt +++ b/core/relinker/CMakeLists.txt @@ -143,8 +143,13 @@ if(BUILD_TESTING) set_tests_properties(autorun_exit_status PROPERTIES TIMEOUT 20) add_test_executable(cli_args_tests ${CMAKE_CURRENT_SOURCE_DIR}/cli/tests/CliArgsTests.cpp - ${CMAKE_CURRENT_SOURCE_DIR}/cli/src/CliArgs.cpp) - target_include_directories(cli_args_tests PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/cli/include) + ${CMAKE_CURRENT_SOURCE_DIR}/cli/src/CliArgs.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/io/src/NativePath.cpp) + target_include_directories(cli_args_tests PRIVATE + ${CMAKE_CURRENT_SOURCE_DIR}/cli/include + ${CMAKE_CURRENT_SOURCE_DIR}/io/include + ${CMAKE_CURRENT_SOURCE_DIR}/domain/include + ) add_test(NAME cli_args COMMAND cli_args_tests) add_test_executable(buffer_bounds_tests ${CMAKE_CURRENT_SOURCE_DIR}/io/tests/BufferBoundsTests.cpp diff --git a/core/relinker/cli/src/CliArgs.cpp b/core/relinker/cli/src/CliArgs.cpp index 8fcda4b31..1a0f1e00d 100644 --- a/core/relinker/cli/src/CliArgs.cpp +++ b/core/relinker/cli/src/CliArgs.cpp @@ -1,4 +1,5 @@ #include +#include #include #include #include @@ -87,7 +88,7 @@ Args ParseArgs(int argc, char* argv[]) { throw std::runtime_error(Usage()); - if (!sceModulePathSpecified && !args.skipSceModule && !args.inputPath.empty()) args.sceModulePath = std::filesystem::absolute(args.inputPath).parent_path().string(); + if (!sceModulePathSpecified && !args.skipSceModule && !args.inputPath.empty()) args.sceModulePath = Io::Utf8Path(std::filesystem::absolute(Io::NativePath(args.inputPath)).parent_path()); return args; } diff --git a/core/relinker/main.cpp b/core/relinker/main.cpp index 5bb9f2777..949dc190c 100644 --- a/core/relinker/main.cpp +++ b/core/relinker/main.cpp @@ -204,7 +204,8 @@ int wmain(const int argc, wchar_t* argv[]) { try { std::vector arguments; arguments.reserve(static_cast(argc)); - for (int index = 0; index < argc; ++index) arguments.push_back(Utf8Argument(argv[index])); + arguments.emplace_back(); + for (int index = 1; index < argc; ++index) arguments.push_back(Utf8Argument(argv[index])); std::vector pointers; pointers.reserve(arguments.size() + 1); for (auto& argument : arguments) pointers.push_back(argument.data()); diff --git a/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp b/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp index 9d5043867..4f8bb12bd 100644 --- a/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp +++ b/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp @@ -52,9 +52,9 @@ std::string ModuleStem(std::string name, const bool windows) { std::vector GuestModuleBuilder::Build(const std::filesystem::path& inputPath, const std::filesystem::path& outputPath, Domain::SysVDynamicSection& dynamic, const bool windows, const bool macos, const bool toIntel, ISyscallScanner& syscallScanner, const bool lazyBinding, const std::string& runPath, const std::set& excludedModules, const std::filesystem::path& sceModulePath) const { const auto root = std::filesystem::absolute(sceModulePath).lexically_normal(); - if (!std::filesystem::exists(root)) throw Domain::RelinkerException("Guest module parent directory does not exist: " + root.string()); - if (!std::filesystem::is_directory(root)) throw Domain::RelinkerException("Guest module parent path is not a directory: " + root.string()); - const auto missingModuleMessage = std::string(GuestModulePattern) + " was not found in: " + root.string() + ". Use --skip-sce-module to disable guest module processing."; + if (!std::filesystem::exists(root)) throw Domain::RelinkerException("Guest module parent directory does not exist: " + Io::Utf8Path(root)); + if (!std::filesystem::is_directory(root)) throw Domain::RelinkerException("Guest module parent path is not a directory: " + Io::Utf8Path(root)); + const auto missingModuleMessage = std::string(GuestModulePattern) + " was not found in: " + Io::Utf8Path(root) + ". Use --skip-sce-module to disable guest module processing."; const auto singular = root / "sce_module"; const auto plural = root / "sce_modules"; const auto prx = root / "prx"; @@ -70,22 +70,22 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path std::set unmatchedExclusions = excludedModules; const auto isElf = [](const std::filesystem::path& path) { std::ifstream stream(path, std::ios::binary); - if (!stream) throw Domain::RelinkerException("Cannot read guest candidate: " + path.string()); + if (!stream) throw Domain::RelinkerException("Cannot read guest candidate: " + Io::Utf8Path(path)); char magic[4]{}; stream.read(magic, 4); - if (stream.bad()) throw Domain::RelinkerException("Cannot read guest candidate magic: " + path.string()); + if (stream.bad()) throw Domain::RelinkerException("Cannot read guest candidate magic: " + Io::Utf8Path(path)); if (stream.gcount() != 4) return false; const auto byte = [&](const std::size_t index) { return static_cast(magic[index]); }; const bool self = (byte(0) == 0x4f && byte(1) == 0x15 && byte(2) == 0x3d && byte(3) == 0x1d) || (byte(0) == 0x54 && byte(1) == 0x14 && byte(2) == 0xf5 && byte(3) == 0xee); - if (self) throw Domain::RelinkerException("Guest module is a SELF container, not an ELF: " + path.string()); + if (self) throw Domain::RelinkerException("Guest module is a SELF container, not an ELF: " + Io::Utf8Path(path)); return byte(0) == 0x7f && magic[1] == 'E' && magic[2] == 'L' && magic[3] == 'F'; }; for (const auto& directory : directories) { - if (!std::filesystem::is_directory(directory)) throw Domain::RelinkerException("Guest module path is not a directory: " + directory.string()); + if (!std::filesystem::is_directory(directory)) throw Domain::RelinkerException("Guest module path is not a directory: " + Io::Utf8Path(directory)); for (const auto& entry : std::filesystem::directory_iterator(directory)) { - if (entry.path().filename().string().ends_with(GuestModuleSuffix)) continue; - if (excludedModules.contains(entry.path().filename().string())) { - unmatchedExclusions.erase(entry.path().filename().string()); + if (Io::Utf8Path(entry.path().filename()).ends_with(GuestModuleSuffix)) continue; + if (excludedModules.contains(Io::Utf8Path(entry.path().filename()))) { + unmatchedExclusions.erase(Io::Utf8Path(entry.path().filename())); continue; } if (!entry.is_regular_file()) continue; @@ -97,9 +97,9 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path std::map neededAliases; for (const auto& name : neededNames) { if (excludedModules.contains(name)) continue; - if (std::any_of(paths.begin(), paths.end(), [&](const auto& path) { return path.filename().string() == name || (windows && FoldFilename(path.filename().string()) == FoldFilename(name)); })) continue; + if (std::any_of(paths.begin(), paths.end(), [&](const auto& path) { return Io::Utf8Path(path.filename()) == name || (windows && FoldFilename(Io::Utf8Path(path.filename())) == FoldFilename(name)); })) continue; const auto stem = ModuleStem(name, windows); - const auto alias = stem.empty() ? paths.end() : std::find_if(paths.begin(), paths.end(), [&](const auto& path) { return ModuleStem(path.filename().string(), windows) == stem; }); + const auto alias = stem.empty() ? paths.end() : std::find_if(paths.begin(), paths.end(), [&](const auto& path) { return ModuleStem(Io::Utf8Path(path.filename()), windows) == stem; }); if (alias != paths.end()) neededAliases.emplace(name, *alias); else missingNeeded.insert(name); } @@ -110,7 +110,7 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path return name == identity || name == identity + ".prx" || name == identity + ".sprx" || name == identity + ".suprx"; }); }; - for (const auto& path : paths) discovered.emplace(path, GuestImageReader().Read(path, reader.Read(path.string()))); + for (const auto& path : paths) discovered.emplace(path, GuestImageReader().Read(path, reader.Read(Io::Utf8Path(path)))); if (!unmatchedExclusions.empty()) throw Domain::RelinkerException("Excluded guest module file not found: " + *unmatchedExclusions.begin()); std::sort(paths.begin(), paths.end()); paths.erase(std::unique(paths.begin(), paths.end()), paths.end()); @@ -140,7 +140,7 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path if (symbol.Section == 0 || symbol.Section == AbsoluteSection || (symbol.Info >> 4) == 0 || symbol.Visibility == 1 || symbol.Visibility == 2) continue; auto& providers = exports[symbol.Name]; const bool repeated = std::find(providers.begin(), providers.end(), images.size()) != providers.end(); - if (!windows && repeated) throw Domain::RelinkerException("Duplicate guest export after stripping #: " + symbol.Name + " in " + path.string() + " and " + path.string()); + if (!windows && repeated) throw Domain::RelinkerException("Duplicate guest export after stripping #: " + symbol.Name + " in " + Io::Utf8Path(path) + " and " + Io::Utf8Path(images[providers.front()].SourcePath)); if (!repeated) providers.push_back(images.size()); if (!windows && providers.size() > 1) sharedExports[symbol.Name].insert(providers.begin(), providers.end()); } @@ -160,12 +160,12 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path std::map guestNames; std::map windowsGuestFiles; for (std::size_t index = 0; index < images.size(); ++index) { - for (const auto& name : {images[index].SourcePath.filename().string(), images[index].Soname}) { + for (const auto& name : {Io::Utf8Path(images[index].SourcePath.filename()), images[index].Soname}) { if (name.empty()) continue; const auto [found, inserted] = guestNames.emplace(name, index); if (!inserted && found->second != index) throw Domain::RelinkerException("Ambiguous guest dependency name: " + name); } - if (windows) windowsGuestFiles.emplace(FoldFilename(images[index].SourcePath.filename().string()), index); + if (windows) windowsGuestFiles.emplace(FoldFilename(Io::Utf8Path(images[index].SourcePath.filename())), index); } for (const auto& [name, path] : neededAliases) { if (guestNames.contains(name)) continue; @@ -198,7 +198,7 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path const auto shared = sharedExports.find(name); if (shared == sharedExports.end()) return; std::string providers; - for (const auto provider : shared->second) providers += " " + images[provider].SourcePath.string(); + for (const auto provider : shared->second) providers += " " + Io::Utf8Path(images[provider].SourcePath); throw Domain::RelinkerException("Ambiguous guest import " + name + " in " + importer + ": exported by" + providers); }; const auto rename = [](std::vector& symbols, std::vector& strings, std::size_t index, const std::string& name) { @@ -215,7 +215,7 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path const auto end = std::find(start, dynamic.DynStrData.end(), 0); if (end == dynamic.DynStrData.end()) throw Domain::RelinkerException("Unterminated executable symbol name"); const std::string name(start, end); - rejectSharedImport(name.substr(0, name.find('#')), inputPath.string()); + rejectSharedImport(name.substr(0, name.find('#')), Io::Utf8Path(inputPath)); if (!windows && exports.contains(name)) rename(dynamic.DynSymData, dynamic.DynStrData, offset / 24, name); } std::vector> dependencies(images.size()); @@ -233,7 +233,7 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path for (std::size_t symbolIndex = 0; symbolIndex < images[index].Symbols.size(); ++symbolIndex) { const auto& symbol = images[index].Symbols[symbolIndex]; if (symbol.Section != 0 || symbol.Name.empty()) continue; - rejectSharedImport(symbol.Name, images[index].SourcePath.string()); + rejectSharedImport(symbol.Name, Io::Utf8Path(images[index].SourcePath)); const auto found = exports.find(symbol.Name); std::vector providers; if (found != exports.end()) { @@ -264,7 +264,7 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path std::vector order; std::vector states(images.size()); const std::function visit = [&](std::size_t index) { - if (states[index] == 1) throw Domain::RelinkerException("Cyclic guest initialization dependency: " + images[index].SourcePath.string()); + if (states[index] == 1) throw Domain::RelinkerException("Cyclic guest initialization dependency: " + Io::Utf8Path(images[index].SourcePath)); if (states[index] == 2) return; states[index] = 1; for (const auto dependency : dependencies[index]) visit(dependency); @@ -299,12 +299,12 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path const auto destination = outputDirectory / relativeDirectory; const auto target = destination / image.OutputName; if (target.lexically_normal() == std::filesystem::absolute(outputPath).lexically_normal()) throw Domain::RelinkerException("Guest output collides with the executable output"); - for (const auto& source : paths) if (std::filesystem::exists(target) && std::filesystem::equivalent(source, target)) throw Domain::RelinkerException("Guest output would overwrite an input module: " + target.string()); + for (const auto& source : paths) if (std::filesystem::exists(target) && std::filesystem::equivalent(source, target)) throw Domain::RelinkerException("Guest output would overwrite an input module: " + Io::Utf8Path(target)); if (std::filesystem::exists(target) && std::filesystem::equivalent(inputPath, target)) throw Domain::RelinkerException("Guest output would overwrite the input executable"); Domain::GuestRuntime runtime; runtime.UsePlatformTlsResolver = image.UsePlatformTlsResolver; runtime.Path = relativeDirectory + "/" + image.OutputName; - runtime.Names = {image.SourcePath.filename().string(), image.Soname}; + runtime.Names = {Io::Utf8Path(image.SourcePath.filename()), image.Soname}; std::vector output; if (windows) output = Elfpatcher::GuestModuleWriter().WriteWindows(image, runtime); else { From fe81332b08662e65cabf7e7e4711aac62103833d Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 14:39:23 +0000 Subject: [PATCH 085/460] test(relinker): cover unicode guest module paths --- core/relinker/relinker/tests/test_windows_import_modules.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/core/relinker/relinker/tests/test_windows_import_modules.py b/core/relinker/relinker/tests/test_windows_import_modules.py index 83e83ff56..3bc090877 100644 --- a/core/relinker/relinker/tests/test_windows_import_modules.py +++ b/core/relinker/relinker/tests/test_windows_import_modules.py @@ -221,6 +221,10 @@ def main(): run = subprocess.run([str(output)], capture_output=True, text=True, timeout=30) assert run.returncode == (11 if owner == 'a.prx' else 22), (run.returncode, run.stdout, run.stderr) + result, output = convert('unicode-ü日', 'a.prx', guest_owner='b.prx') + assert result.returncode == 0, (result.stdout, result.stderr) + assert output.is_file() and (output.parent / 'app0' / 'prx' / 'a.prx.guest.prx').is_file() + check_internal_guest_libc(convert, work, relinker) for filename, module_name in [('foo.native.prx', 'foo_native'), From c5ffebe15aebc564330a1b9ce2fa84a9efa19cf2 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 17:28:07 +0000 Subject: [PATCH 086/460] fix(relinker): preserve empty Windows arguments --- core/relinker/main.cpp | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/core/relinker/main.cpp b/core/relinker/main.cpp index 949dc190c..87bb581ac 100644 --- a/core/relinker/main.cpp +++ b/core/relinker/main.cpp @@ -190,10 +190,12 @@ int Run(const int argc, char* argv[]) { #ifdef _WIN32 std::string Utf8Argument(const wchar_t* argument) { - const int size = WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, argument, static_cast(wcslen(argument)), nullptr, 0, nullptr, nullptr); + const std::size_t argumentSize = std::wcslen(argument); + if (argumentSize == 0) return {}; + const int size = WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, argument, static_cast(argumentSize), nullptr, 0, nullptr, nullptr); if (size <= 0) throw std::runtime_error("Cannot encode the command line as UTF-8"); std::string utf8(static_cast(size), '\0'); - if (WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, argument, static_cast(wcslen(argument)), utf8.data(), size, nullptr, nullptr) != size) + if (WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, argument, static_cast(argumentSize), utf8.data(), size, nullptr, nullptr) != size) throw std::runtime_error("Cannot encode the command line as UTF-8"); return utf8; } From b7d5f18c4187db2df52101dff2608c7a4488c3e9 Mon Sep 17 00:00:00 2001 From: ctj01 Date: Sat, 10 Oct 2026 08:37:13 -0500 Subject: [PATCH 087/460] fix(relinker): drop the input PT_PHDR and PT_INTERP from Linux output The Linux patcher writes its own PT_PHDR and PT_INTERP but also copied the input's. glibc takes the load bias from the last PT_PHDR, which was the input's entry with the pre-relink address, so such an executable crashed at start. Both copies also took program header slots, so an input with PT_PHDR and PT_INTERP and no spare slots failed to relink. --- core/relinker/CMakeLists.txt | 2 + .../elfpatcher/src/general/SegmentFilter.cpp | 2 +- .../tests/test_linux_input_program_headers.py | 104 ++++++++++++++++++ 3 files changed, 107 insertions(+), 1 deletion(-) create mode 100644 core/relinker/relinker/tests/test_linux_input_program_headers.py diff --git a/core/relinker/CMakeLists.txt b/core/relinker/CMakeLists.txt index 288dddef6..bf34dc97b 100644 --- a/core/relinker/CMakeLists.txt +++ b/core/relinker/CMakeLists.txt @@ -247,6 +247,8 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_linux_entry_argv.py $) add_test(NAME linux_program_header_order COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_linux_program_header_order.py $) + add_test(NAME linux_input_program_headers COMMAND ${Python3_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_linux_input_program_headers.py $) add_test(NAME windows_gui COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_windows_gui.py $) add_test(NAME windows_working_directory COMMAND ${Python3_EXECUTABLE} diff --git a/core/relinker/elfpatcher/src/general/SegmentFilter.cpp b/core/relinker/elfpatcher/src/general/SegmentFilter.cpp index ff8f24e7f..f387b4e7c 100644 --- a/core/relinker/elfpatcher/src/general/SegmentFilter.cpp +++ b/core/relinker/elfpatcher/src/general/SegmentFilter.cpp @@ -14,7 +14,7 @@ bool SegmentFilter::ShouldSkip(const Domain::ProgramHeader& ph) const { if (ph.Type == PT_OS_PROCPARAM) return false; if (ph.Type == PT_GNU_EH_FRAME) return false; if (_isSceSpecificSegment(ph.Type)) return true; - if (ph.Type == PT_DYNAMIC) return true; + if (ph.Type == PT_DYNAMIC || ph.Type == PT_PHDR || ph.Type == PT_INTERP) return true; if (ph.Type == PT_NOTE && ph.MappedAddress == 0 && ph.FileSize > 0) return true; return false; } diff --git a/core/relinker/relinker/tests/test_linux_input_program_headers.py b/core/relinker/relinker/tests/test_linux_input_program_headers.py new file mode 100644 index 000000000..45c1f5251 --- /dev/null +++ b/core/relinker/relinker/tests/test_linux_input_program_headers.py @@ -0,0 +1,104 @@ +"""Check that a Linux relink replaces the input's PT_PHDR and PT_INTERP with its own instead of copying them.""" + +from pathlib import Path +import platform +import signal +import struct +import subprocess +import sys +import tempfile + +PT_LOAD = 1 +PT_DYNAMIC = 2 +PT_INTERP = 3 +PT_PHDR = 6 +PT_SCE_VERSION = 0x6FFFFF01 +HOST_INTERP = b"/lib64/ld-linux-x86-64.so.2\0" +INPUT_INTERP = b"/libexec/ld-elf.so.1\0" +TAGS = [(5, 0x2600), (10, 1), (6, 0x2620), (11, 24), (7, 0x2700), (8, 0), (9, 24), (0, 0)] + + +def fixture(phdr, interp, slots): + image = bytearray(0x3000) + image[:16] = b"\x7fELF\x02\x01\x01" + bytes(9) + headers = [] + if phdr: + headers.append((PT_PHDR, 4, 0x40, 0x40, slots * 56, slots * 56, 8)) + if interp: + headers.append((PT_INTERP, 4, 0x300, 0x300, len(INPUT_INTERP), len(INPUT_INTERP), 1)) + headers += [(PT_LOAD, 4, 0x0000, 0x0000, 0x1000, 0x1000, 0x1000), + (PT_LOAD, 5, 0x1000, 0x1000, 0x1000, 0x1000, 0x1000), + (PT_LOAD, 6, 0x2000, 0x2000, 0x1000, 0x1000, 0x1000), + (PT_DYNAMIC, 6, 0x2400, 0x2400, len(TAGS) * 16, len(TAGS) * 16, 8)] + assert len(headers) <= slots, (len(headers), slots) + headers += [(PT_SCE_VERSION, 0, 0, 0, 0, 0, 1)] * (slots - len(headers)) + struct.pack_into(" Date: Sat, 10 Oct 2026 12:42:01 -0500 Subject: [PATCH 088/460] feat(psml): declare scePsmlMfsrCreateContext1300 and scePsmlMfsrGetDispatchMfsrPacket1300 PPSA28416 imports both from libScePsml; the names hash to NIDs kLF7WtQ1-Dg and DDVUyPyYHTA. --- core/libs/prx/libScePsml_debug/Export.cpp | 10 ++++++++++ docs/dev/TechnicalDebt.md | 1 + 2 files changed, 11 insertions(+) diff --git a/core/libs/prx/libScePsml_debug/Export.cpp b/core/libs/prx/libScePsml_debug/Export.cpp index d85faa0ae..e7326689c 100644 --- a/core/libs/prx/libScePsml_debug/Export.cpp +++ b/core/libs/prx/libScePsml_debug/Export.cpp @@ -30,6 +30,16 @@ std::int32_t APS5_VABI scePsmlMfsrGetDispatchMfsrPacket1100(void* context, void* return SCE_PSML_ERROR_NOT_INITIALIZED; } +int APS5_VABI scePsmlMfsrCreateContext1300() { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI scePsmlMfsrGetDispatchMfsrPacket1300() { + NotImplemented_nid_no_patch(__func__); + return 0; +} + int APS5_VABI scePsmlMfsrGetSharedResourcesInitRequirement() { NotImplemented_nid_no_patch(__func__); return 0; diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..a89380b7b 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -173,6 +173,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceAudioOut2PortSetAttributes](../../core/libs/prx/libSceAudioOut/src/AudioOut2Port.cpp) (libSceAudioOut) - each PCM buffer set (attribute 0) is taken as one grain: buffers set between two pushes are queued and played one per grain, at most 16 queued. A grain with no new buffer repeats the last one only when the port has always been given that same buffer (rewritten in place); a port whose buffers move on plays silence instead. Inferred from PPSA12544, whose object port (type 6) is given one grain of a 64-grain ring per set, often two or three between pushes and then none for a few; the console behaviour for several sets per push and for a push without a new buffer is not documented - [scePsmlMfsrGetContextBufferRequirement1100, scePsmlMfsrCreateContext1100, scePsmlMfsrGetDispatchMfsrPacket1100](../../core/libs/prx/libScePsml_debug/Export.cpp) (libScePsml_debug) - argument layouts unknown; only 0x8A810001 (not initialized) is returned - [scePsmlMfsrGetDispatchMfsrPacket1000](../../core/libs/prx/libScePsml_debug/Export.cpp) (libScePsml) - unknown signature, imported by PPSA25344; declared without parameters. libScePsml and libScePsml_debug are built from the same source +- [scePsmlMfsrCreateContext1300, scePsmlMfsrGetDispatchMfsrPacket1300](../../core/libs/prx/libScePsml_debug/Export.cpp) (libScePsml) - unknown signatures, imported by PPSA28416 next to `scePsmlMfsrGetContextBufferRequirement1100`; declared without parameters. The names are not in the aerolib NID list; they hash to the imported NIDs `kLF7WtQ1-Dg` and `DDVUyPyYHTA` (`tools/nid_names.py`) - [scePngEncEncode](../../core/libs/prx/libScePngEnc/Export.cpp) (libScePngEnc) - whether rows may stay unfiltered when `filter_type` names a subset of the filters is unknown; as in shadPS4, they may only for 0 and for the all-filters mask - [sceJpegEncEncode](../../core/libs/prx/libSceJpegEnc/Export.cpp) (libSceJpegEnc) - `restart_interval` follows SharpProspero's field doc (0 none, -1 every row of blocks, positive a block count); a positive value is taken as the JPEG DRI interval in MCUs and -1 as one restart per MCU row, which no reference confirms. Other negative values throw - [sceVoiceQoSInit](../../core/libs/prx/libSceVoiceQoS/Export.cpp) (libSceVoiceQoS) - `0x804E0905` for a second initialization comes from [prosper](https://github.com/mattias800/prosper)'s reading of the module and was not measured: on a PS5 a valid call blocks inside a homebrew payload, so it never returns there. The app type and the block are not used, as no endpoint is implemented From a73b4e3f6d174e616fb49876f864e777fddd70ec Mon Sep 17 00:00:00 2001 From: Chad Date: Sat, 10 Oct 2026 12:43:28 -0500 Subject: [PATCH 089/460] feat(audioout): declare sceAudioOut2EnableChat Imported by party-ps5-shipping.prx bundled with PPSA28416. --- core/libs/prx/libSceAudioOut/src/AudioOut2System.cpp | 5 +++++ docs/dev/TechnicalDebt.md | 1 + 2 files changed, 6 insertions(+) diff --git a/core/libs/prx/libSceAudioOut/src/AudioOut2System.cpp b/core/libs/prx/libSceAudioOut/src/AudioOut2System.cpp index 67d66b44b..dff796528 100644 --- a/core/libs/prx/libSceAudioOut/src/AudioOut2System.cpp +++ b/core/libs/prx/libSceAudioOut/src/AudioOut2System.cpp @@ -34,4 +34,9 @@ int APS5_VABI sceAudioOut2UserGetSupportedAttributes(void) { return 0; } +int APS5_VABI sceAudioOut2EnableChat(void) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..002d498eb 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -66,6 +66,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceAgcDcbSetCfRegisterDirect, sceAgcDcbSetCfRegisterRangeDirect](../../core/libs/prx/libSceAgc/DcbState/src/Registers.cpp) (libSceAgc) - signatures follow [OpenAGC](https://github.com/OpenAGC/OpenAGC/blob/main/src/game_compat.c) and [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Agc/SceAgc.cs), which agree; the SET_CONFIG_REG (0x68) packet is OpenAGC's. The driver does not execute SET_CONFIG_REG, so a submitted packet throws - [scePadOpenExt](../../core/libs/prx/libScePad/Export.cpp) (libScePad) - only special ports (type 2) are handled, and they return `DEVICE_NOT_CONNECTED` (0x80920007) because the host has no special controller; whether the console opens a handle and reports the device as disconnected later is unknown. PPSA01317 passes a 16-byte param that names the device by vendor and product id (`b7 0e 04 0e ...` for 0x0eb7:0x0e04) and skips the user on any negative result. Other port types throw - PPSA23566 imports declared without parameters, signatures unknown: [sceAudioOut2UserGetSupportedAttributes](../../core/libs/prx/libSceAudioOut/src/AudioOut2System.cpp), [sceConvertKeycodeGetCharacterFromKeyboardData](../../core/libs/prx/libSceConvertKeycode/Export.cpp), [sce::Json::MemAllocator::notifyError](../../core/libs/prx/libSceJson2/Export.cpp), [sceNetResolverAbort](../../core/libs/prx/libSceNet/Export.cpp), [sceNpCommerceDialogOpen2](../../core/libs/prx/libSceNpCommerce/Export.cpp), [sceNpEntitlementAccessGetPftFlag](../../core/libs/prx/libSceNpEntitlementAccess/Export.cpp), [sceProprietaryVoiceChatHelper{Initialize,Terminate,SetVoiceChatState,GetVoiceChatUsageState}](../../core/libs/prx/libSceProprietaryVoiceChatHelperPrx/Export.cpp), [sceVoice{GetResourceInfo,ResetPort,EnableChat,DisableChat}](../../core/libs/prx/libSceVoice/Export.cpp). Names from the shadPS4 aerolib NID list +- PPSA28416 imports declared without parameters, signatures unknown: [sceAudioOut2EnableChat](../../core/libs/prx/libSceAudioOut/src/AudioOut2System.cpp), imported by its bundled `party-ps5-shipping.prx`. Name from the shadPS4 aerolib NID list - [sceVoiceChatRequestCreateVoiceChatGroup, sceVoiceChatRequestJoinVoiceChatGroup, sceVoiceChatRequestLeaveVoiceChatGroup, sceVoiceChatRequestDeleteVoiceChatGroup](../../core/libs/prx/libSceVoiceChat/Export.cpp) (libSceVoiceChat) - assumed to take a request id and two pointers like the other `sceVoiceChatRequest*` calls; offline, every argument is ignored and `SIGNED_OUT` is returned, as for those - [sceAvPlayerAddSource](../../core/libs/prx/libSceAvPlayer/src/Source.cpp) (libSceAvPlayer) - requests up to 4 additional decode-ahead video framebuffers from the memory replacement beyond `num_output_video_framebuffers` when memory is available; the console's internal buffering behaviour is unknown - [sceTextToSpeech2Initialize](../../core/libs/prx/libSceTextToSpeech2/Export.cpp) (libSceTextToSpeech2) - the console's error for an unavailable speech engine is unknown; it returns the kernel's `EOPNOTSUPP` (0x8002002D). PPSA01317 passes a param that starts with 0x2000000 and only tests the result for non-zero; the other exports it imports are declared without parameters From df415ff1fd695025731e699e20997704612ebeba Mon Sep 17 00:00:00 2001 From: LeandroLP23 Date: Fri, 9 Oct 2026 09:51:49 +0200 Subject: [PATCH 090/460] feat(libkernel): implement pthread_sigmask Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libkernel/System/src/Signals.cpp | 5 +++++ core/libs/tests/GuestSignals.cpp | 11 +++++++++++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 17 insertions(+) diff --git a/core/libs/prx/libkernel/System/src/Signals.cpp b/core/libs/prx/libkernel/System/src/Signals.cpp index f1ee8327e..93dfb9fad 100644 --- a/core/libs/prx/libkernel/System/src/Signals.cpp +++ b/core/libs/prx/libkernel/System/src/Signals.cpp @@ -162,6 +162,11 @@ int APS5_VABI sigprocmask_nid_postfix(int how, const void* set, void* previousSe return _sigprocmask_nid_postfix(how, static_cast(set), static_cast(previousSet)); } + +int APS5_VABI pthread_sigmask_nid_postfix(int how, const GuestSignalSet* set, GuestSignalSet* previousSet) { + if (set != nullptr && (how < 1 || how > 3)) return 22; + return _sigprocmask_nid_postfix(how, set, previousSet); +} } extern "C" { diff --git a/core/libs/tests/GuestSignals.cpp b/core/libs/tests/GuestSignals.cpp index dc5bf1a96..b90cb37c9 100644 --- a/core/libs/tests/GuestSignals.cpp +++ b/core/libs/tests/GuestSignals.cpp @@ -10,6 +10,7 @@ Handler APS5_VABI signal_nid_postfix(int, Handler); int APS5_VABI raise_nid_postfix(int); int APS5_VABI sigaction_nid_postfix(int, const void*, void*); int APS5_VABI sigprocmask_nid_postfix(int, const void*, void*); +int APS5_VABI pthread_sigmask_nid_postfix(int, const void*, void*); int* APS5_VABI __error_nid_postfix(); int APS5_VABI _is_signal_return_nid_postfix(std::uint64_t); } @@ -59,6 +60,16 @@ int main() { Require(sigprocmask_nid_postfix(2, &blocked, nullptr) == 0); Require(sigprocmask_nid_postfix(1, nullptr, &previous) == 0); Require(previous.bits[0] == 0); + *__error_nid_postfix() = 0; + Require(pthread_sigmask_nid_postfix(1, &blocked, nullptr) == 0); + Require(pthread_sigmask_nid_postfix(0, &blocked, &previous) == 22); + Require(pthread_sigmask_nid_postfix(4, &blocked, &previous) == 22); + Require(*__error_nid_postfix() == 0); + Require(pthread_sigmask_nid_postfix(0, nullptr, &previous) == 0); + Require(previous.bits[0] == 0x20); + Require(sigprocmask_nid_postfix(1, nullptr, &previous) == 0 && previous.bits[0] == 0x20); + Require(pthread_sigmask_nid_postfix(2, &blocked, &previous) == 0 && previous.bits[0] == 0x20); + Require(pthread_sigmask_nid_postfix(1, nullptr, &previous) == 0 && previous.bits[0] == 0); const auto callback = reinterpret_cast(Callback); const auto infoCallback = reinterpret_cast(InfoCallback); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..515a9f2f5 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -319,6 +319,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceKernelRaiseException](../../core/libs/prx/libkernel/System/src/Exception.cpp) delivers on Windows only and throws on Linux - [sceKernelRaiseException](../../core/libs/prx/libkernel/System/src/Exception.cpp) builds the handler's `ucontext` with FreeBSD's amd64 layout (16-byte sigmask, `mcontext` at 0x40, `mc_rsp` at 0xf8); the console's layout is not verified beyond the fields PS5Util reads - [sceKernelRaiseException](../../core/libs/prx/libkernel/System/src/Exception.cpp) does not consult the target thread's signal mask +- [pthread_sigmask](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) shares the process-wide mask of `sigprocmask` and `_sigprocmask`, which keeps only the first 32 signals; FreeBSD keeps a mask per thread - [sceKernelRaiseException](../../core/libs/prx/libkernel/System/src/Exception.cpp) reaches a thread that waits in an alertable `TimedWait` wait through an APC. A thread blocked anywhere else, including all waits under `APS5_COARSE_TIMED_WAITS` and host waits inside the emulator, is redirected and runs the handler only when that wait returns - [sceKernelRaiseException](../../core/libs/prx/libkernel/System/src/Exception.cpp) on Windows runs the handler wherever the target thread was interrupted. It waits only while the thread is inside winpthreads or restoring a context through `NtContinue`, and throws if that lasts 1 s, so a handler that calls into other host code the thread was interrupted in, such as the C runtime's heap or a lock of the emulator, can deadlock on it or reenter it - [sigaltstack](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) records each thread's alternate signal stack and reports it back, following FreeBSD's `kern_sigaltstack` (`EINVAL` for flags other than `SS_DISABLE`, `ENOMEM` below `MINSIGSTKSZ` 2048), but handlers never run on it, so `SS_ONSTACK` and `EPERM` are never reported From 68a1aea611f107f798d75bd742552a0521a27f5a Mon Sep 17 00:00:00 2001 From: LeandroLP23 Date: Sat, 10 Oct 2026 19:48:06 +0200 Subject: [PATCH 091/460] docs(libkernel): record that environ is not libc's environment Co-Authored-By: Claude Opus 5.5 --- docs/dev/TechnicalDebt.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..ed02e32ee 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -156,6 +156,7 @@ Throughout the project, every function at every stage either **does exactly what - [std_execute_once](../../core/libs/prx/libc/Export.cpp) (libc) - assumed to be the plain-named twin of `std::_Execute_once` (`_ZSt13_Execute_onceRSt9once_flagPFiPvS1_PS1_ES1_`, the callee of Dinkumware `call_once`): it delegates there and gets the [thread.once.callonce] semantics already verified for that symbol (one invocation per flag, concurrent callers block until it completes, and a failing or throwing invocation leaves the flag unready so the next caller runs it). The name equivalence is not confirmed on a title, and the callback convention (called with (null, arg, null), non-zero return meaning success) is the twin's, unverified for this symbol - [tcgetattr, tcsetattr](../../core/libs/prx/libkernel/File/src/Terminal.cpp) (libkernel) - every file and pipe descriptor is taken as not a terminal (`ENOTTY`), and a socket gets `EADDRNOTAVAIL` for `AF_INET` and `EOPNOTSUPP` otherwise, as FreeBSD's `in_control` and `pru_control_notsupp` answer a termios ioctl; what descriptors 0 to 2 of a title are on the console, and whether any of them is a terminal there, is unknown - [__progname](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - unknown data export +- [environ](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - data export pointing to a fixed empty environment so titles that import it load; it is not the environment libc's `getenv` reads, so it lists none of those variables and `setenv`/`putenv` changes do not appear in it - [seteuid, setegid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - the guest process is assumed to run as root, with real, effective and saved user and group ids 0. As in FreeBSD's `sys_seteuid` and `sys_setegid`, id 0 is then accepted without a privilege check and changes no credential, so 0 is returned; any other id would change the effective id, which is not modelled, and throws. The ids and privileges a title has on a console are unverified - [pthread_barrierattr_setpshared](../../core/libs/prx/libkernel/Pthread/Posix/Barrier.cpp) (libkernel) - PTHREAD_PROCESS_SHARED is accepted on the assumption that the console follows FreeBSD 11.0 libthr, which accepts it (9.0 rejects it with EINVAL); unverified on hardware. Guest code runs in one process, so a shared barrier behaves as a private one - [pthread_getname_np](../../core/libs/prx/libkernel/Pthread/Posix/Thread.cpp) (libkernel) - a thread that has already been joined returns `ESRCH` (3) on a PS5 (firmware 13.60), but its object is freed after the join here, so the pointer cannot be told apart from a live thread and its name is read as for a live one From 9fc0af0b10f95757a70a2138b8f2243e5e08cd53 Mon Sep 17 00:00:00 2001 From: Yassine Date: Sat, 10 Oct 2026 15:16:42 +0000 Subject: [PATCH 092/460] fix(avplayer): do not let auto start restart playback the title started sceAvPlayerAddSource queues an auto start for a player created with auto start or without an event callback. When the title enabled its streams and called sceAvPlayerStart before the controller thread ran it, the auto start called Start again on the playing player, which stopped it, released every video buffer, including the frame the title held, and started the film over. guest_avplayer occasionally caught this as "video frame is not a texture allocation"; right after the test's own start, 8 texture buffers were released in 15 of 15 runs. The auto start now starts the player only while it is still ready, and checks that under the player lock together with the start. TestAutoStartKeepsStartedPlayback starts a callback-less player itself and checks that no texture buffer is released afterwards and that the frame it holds stays allocated. It fails on every run without the fix. --- .../prx/libSceAvPlayer/include/AvPlayer.hpp | 2 ++ core/libs/prx/libSceAvPlayer/src/Player.cpp | 15 ++++++++++- core/libs/tests/GuestAvPlayer.cpp | 27 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 44 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAvPlayer/include/AvPlayer.hpp b/core/libs/prx/libSceAvPlayer/include/AvPlayer.hpp index 0eeb944fb..c401749ba 100644 --- a/core/libs/prx/libSceAvPlayer/include/AvPlayer.hpp +++ b/core/libs/prx/libSceAvPlayer/include/AvPlayer.hpp @@ -130,6 +130,8 @@ private: void deliver(const Event& event); void autoStart(); void checkEndOfFile(); + bool readyLocked() const; + int startLocked(); int stopLocked(); AvPlayerFileReplacement file{}; diff --git a/core/libs/prx/libSceAvPlayer/src/Player.cpp b/core/libs/prx/libSceAvPlayer/src/Player.cpp index 1e265d395..bdf69bfc3 100644 --- a/core/libs/prx/libSceAvPlayer/src/Player.cpp +++ b/core/libs/prx/libSceAvPlayer/src/Player.cpp @@ -136,7 +136,15 @@ void Player::deliver(const Event& event) { callback.event_callback(callback.object_ptr, event.id, 0, event.id == EventWarningId ? &warning : nullptr); } +bool Player::readyLocked() const { + return source && state == State::Ready; +} + void Player::autoStart() { + { + std::lock_guard lock(mutex); + if (!readyLocked()) return; + } const int count = StreamCount(); if (count <= 0) { Stop(); @@ -155,7 +163,8 @@ void Player::autoStart() { } if (video != -1) EnableStream(static_cast(video)); if (audio != -1) EnableStream(static_cast(audio)); - Start(); + std::lock_guard lock(mutex); + if (readyLocked()) startLocked(); } void Player::checkEndOfFile() { @@ -246,6 +255,10 @@ int Player::ChangeStream(std::uint32_t from, std::uint32_t to) { int Player::Start() { std::lock_guard lock(mutex); + return startLocked(); +} + +int Player::startLocked() { if (!source) return SCE_AVPLAYER_ERROR_OPERATION_FAILED; if (state != State::Ready && state != State::Stop && stopLocked() != SCE_OK) return SCE_AVPLAYER_ERROR_OPERATION_FAILED; const int result = source->Start(); diff --git a/core/libs/tests/GuestAvPlayer.cpp b/core/libs/tests/GuestAvPlayer.cpp index 2fb32a5db..1d50a1497 100644 --- a/core/libs/tests/GuestAvPlayer.cpp +++ b/core/libs/tests/GuestAvPlayer.cpp @@ -1034,6 +1034,32 @@ void TestPs5ExtendedInitLayout() { run(true); } +void TestAutoStartKeepsStartedPlayback() { + bool startedByTitle = false; + for (int attempt = 0; attempt < 10 && !startedByTitle; ++attempt) { + AvPlayerInitData init = InitData(nullptr); + auto* player = sceAvPlayerInit(&init); + Check(player != nullptr, "init failed"); + Check(sceAvPlayerAddSource(player, "/app0/avplayer.mp4") == 0, "add source failed"); + startedByTitle = sceAvPlayerEnableStream(player, VideoStream) == 0 && sceAvPlayerEnableStream(player, EnglishAudioStream) == 0; + if (startedByTitle) { + const auto releasedTextures = [] { + std::lock_guard lock(allocations.mutex); + return allocations.textures - ActiveTextureCountLocked(); + }; + Check(sceAvPlayerStart(player) == 0, "start failed"); + const int released = releasedTextures(); + AvPlayerFrameInfoEx frame{}; + Check(WaitFor([&] { return sceAvPlayerGetVideoDataEx(player, &frame) != 0; }), "no video after the title's start"); + std::this_thread::sleep_for(std::chrono::milliseconds(100)); + Check(releasedTextures() == released, "auto-start restarted playback the title had started"); + Check(IsAllocation(frame.p_data, true), "auto-start released the frame the title holds"); + } + Check(sceAvPlayerClose(player) == 0, "close failed"); + } + Check(startedByTitle, "auto-start won every race against the title's own start"); +} + void TestUnsyncedVideoKeepsUpWithAudio() { AvPlayerInitData init = InitData(nullptr); auto* player = sceAvPlayerInit(&init); @@ -1075,6 +1101,7 @@ int main() { TestFileReplacementAutoStart(); TestHandedOutFramesStayIntact(); TestPs5ExtendedInitLayout(); + TestAutoStartKeepsStartedPlayback(); TestUnsyncedVideoKeepsUpWithAudio(); std::puts("AvPlayer tests passed"); return 0; diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..559981ba2 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -198,6 +198,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceAvPlayerStartEx](../../core/libs/prx/libSceAvPlayer/Export.cpp) (libSceAvPlayer) - start info layout unknown; it is ignored and playback starts as with `sceAvPlayerStart` - [sceAvPlayerInit / sceAvPlayerInitEx](../../core/libs/prx/libSceAvPlayer/src/Player.cpp) (libSceAvPlayer) - behaviour without a memory replacement unknown; frame and sample buffers then come from the guest heap - [sceAvPlayerInitEx](../../core/libs/prx/libSceAvPlayer/Export.cpp) (libSceAvPlayer) - the PS5 init data layout (`this_size` 0x230: auto start at 0x74, nine 0x30-byte thread parameter blocks from 0x78, output video frame buffer count at 0x228) follows [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/main/src/libs/avPlayer.cpp), [prosper](https://github.com/mattias800/prosper/blob/main/prosper/src/hle/video/avplayer.cpp), [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Media/AvPlayer.cs) and [sharpemu](https://github.com/sharpemu/sharpemu/blob/main/src/SharpEmu.Libs/AvPlayer/AvPlayerExports.cs), which agree on it. PPSA25063 fills the same fields, writes auto start 0 and starts playback with `sceAvPlayerStart`; a title that sets auto start has not been seen. A smaller `this_size` is read with the PS4 offsets. The thread parameters are ignored +- [AvPlayer auto start](../../core/libs/prx/libSceAvPlayer/src/Player.cpp) (libSceAvPlayer) - the auto start that `sceAvPlayerAddSource` queues for a player with auto start or without an event callback starts the player only while it is still ready: a title that starts the player itself before the auto start runs keeps its playback and the frames it holds. Not confirmed on a title. - [SceAvPlayerVideoEx](../../core/libs/SceTypes.hpp) (libSceAvPlayer) - frame rate field and encoding unknown; it is left zero in frame and stream info - [sceRazorCpuFlushOccurred](../../core/libs/prx/libSceRazorCpu/Export.cpp) (libSceRazorCpu) - signature and return values (1 = flush since the last call, 0 = none) from [SharpProspero](https://github.com/SvenGDK/SharpProspero); without a capture nothing is flushed, so it returns 0 and writes 0 cycles to the optional out pointer (whether the real library writes it when nothing was flushed is unknown) - [sceRazorCpuPushMarkerStatic, sceRazorCpuPopMarker](../../core/libs/prx/libSceRazorCpu/Export.cpp) (libSceRazorCpu) - signatures from PPSA26344's calls; (name, color, flags) assumed for the push; both do nothing without a capture From 70fe1087123be81a409c99727a8dbdaa76c06c8a Mon Sep 17 00:00:00 2001 From: XairRrR Date: Fri, 9 Oct 2026 15:43:23 +0200 Subject: [PATCH 093/460] fix(recompiler): fold extracts into a vector shuffle only between vectors The specialization pass turned OpCompositeConstruct of extracts from one composite into OpVectorShuffle without checking the types. When the source is a struct (the {uint, uint} pairs of extended multiplies and phis of them), the result is invalid SPIR-V ("The type of Vector 1 must be a vector type"), which the NVIDIA driver (615.71.09) crashes on in vkCreateGraphicsPipelines instead of rejecting. The fold now requires both the source and the result to be vector types; a test builds the struct and vector cases by hand. Uncharted: Legacy of Thieves Collection (PPSA05684) crashed in the driver's SPIR-V compiler right after loading its first level. --- .../prx/libSceAgcDriver/tests/SpirvModule.cpp | 40 +++++++++++++++++++ .../SpirvBackend/src/SpirvSpecialization.cpp | 4 +- 2 files changed, 43 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/SpirvModule.cpp b/core/libs/prx/libSceAgcDriver/tests/SpirvModule.cpp index bfd9fd061..98353c139 100644 --- a/core/libs/prx/libSceAgcDriver/tests/SpirvModule.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/SpirvModule.cpp @@ -1,4 +1,5 @@ #include "SpirvBackend/SpirvModule.hpp" +#include "SpirvBackend/SpirvSpecialization.hpp" #include #include #include @@ -184,7 +185,46 @@ void testEmittedWordsAreStable() { } +void testStructExtractsAreNotShuffled() { + const auto op = [](std::uint32_t count, spv::Op code) { return (count << 16u) | static_cast(code); }; + for (const bool vectorSource : {false, true}) { + const std::vector words{ + spv::MagicNumber, 0x00010300u, 0u, 20u, 0u, + op(2, spv::OpCapability), spv::CapabilityShader, + op(3, spv::OpMemoryModel), spv::AddressingModelLogical, spv::MemoryModelGLSL450, + op(6, spv::OpEntryPoint), spv::ExecutionModelGLCompute, 10u, 0x6e69616du, 0u, 9u, + op(6, spv::OpExecutionMode), 10u, spv::ExecutionModeLocalSize, 1u, 1u, 1u, + op(2, spv::OpTypeVoid), 1u, + op(4, spv::OpTypeInt), 2u, 32u, 0u, + vectorSource ? op(4, spv::OpTypeVector) : op(4, spv::OpTypeStruct), 3u, 2u, 2u, + op(4, spv::OpTypeVector), 4u, 2u, 4u, + op(4, spv::OpTypePointer), 5u, spv::StorageClassPrivate, 4u, + op(3, spv::OpTypeFunction), 6u, 1u, + op(4, spv::OpConstant), 2u, 7u, 1u, + op(4, spv::OpConstant), 2u, 8u, 2u, + op(4, spv::OpVariable), 5u, 9u, spv::StorageClassPrivate, + op(5, spv::OpFunction), 1u, 10u, 0u, 6u, + op(2, spv::OpLabel), 11u, + op(5, spv::OpCompositeConstruct), 3u, 12u, 7u, 8u, + op(5, spv::OpCompositeExtract), 2u, 13u, 12u, 0u, + op(5, spv::OpCompositeExtract), 2u, 14u, 12u, 1u, + op(7, spv::OpCompositeConstruct), 4u, 15u, 13u, 14u, 13u, 13u, + op(3, spv::OpStore), 9u, 15u, + op(1, spv::OpReturn), + op(1, spv::OpFunctionEnd), + }; + const auto specialized = ShaderRecompiler::SpecializeSpirv(words); + bool shuffled = false; + for (std::size_t cursor = 5; cursor < specialized.size(); cursor += specialized[cursor] >> 16u) { + if ((specialized[cursor] >> 16u) == 0u) break; + if ((specialized[cursor] & 0xffffu) == spv::OpVectorShuffle) shuffled = true; + } + check(shuffled == vectorSource, vectorSource ? "extracts of one vector were not folded into a shuffle" : "extracts of a struct were folded into a vector shuffle"); + } +} + int main() { + testStructExtractsAreNotShuffled(); testRepeatedTypesShareOneId(); testOperandShapesDoNotCollide(); testOperandKindsAgree(); diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvSpecialization.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvSpecialization.cpp index 2a62cbcde..188cae8f5 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvSpecialization.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvSpecialization.cpp @@ -242,6 +242,7 @@ public: if (hasResult && hasType) resultTypes.emplace(instruction.at(2), instruction.at(1)); if (op == spv::OpTypeBool) types.emplace(instruction.at(1), ScalarType{1u, true}); if (op == spv::OpTypeInt) types.emplace(instruction.at(1), ScalarType{instruction.at(2), false}); + if (op == spv::OpTypeVector) vectorTypes.insert(instruction.at(1)); if (op == spv::OpConstant && types.contains(instruction.at(1)) && types.at(instruction[1]).width <= 32u) values.emplace(instruction.at(2), instruction.at(3)); if (op == spv::OpConstantTrue || op == spv::OpConstantFalse) values.emplace(instruction.at(2), op == spv::OpConstantTrue ? 1u : 0u); cursor += count; @@ -349,7 +350,7 @@ private: if (op == spv::OpCompositeExtract && instruction.size() == 5u) extracts.emplace(instruction[2], std::pair{instruction[3], instruction[4]}); if (op == spv::OpCompositeConstruct && instruction.size() == 7u) { const auto first = extracts.find(instruction[3]); - bool shuffle = first != extracts.end(); + bool shuffle = first != extracts.end() && vectorTypes.contains(instruction[1]) && resultTypes.contains(first->second.first) && vectorTypes.contains(resultTypes.at(first->second.first)); for (std::size_t index = 4; shuffle && index < instruction.size(); ++index) { const auto found = extracts.find(instruction[index]); shuffle = found != extracts.end() && found->second.first == first->second.first; @@ -614,6 +615,7 @@ private: std::map types; std::map values; std::map resultTypes; + std::set vectorTypes; std::set removed; }; From 048c9ff013f8c73d75805a64f0be0d9b332e6d4e Mon Sep 17 00:00:00 2001 From: XairRrR Date: Fri, 9 Oct 2026 19:40:08 +0200 Subject: [PATCH 094/460] feat(agc-driver): render to COLOR_8_8 targets with SWAP_ALT CB_COLOR_INFO COMP_SWAP 1 (SWAP_ALT) on a two-channel format stores the x and w exports in its two channels (Mesa si_translate_colorswap, "X__Y"). COLOR_8_8 UNORM and SNORM targets with it bind as R8G8 with export mapping 0xec; blending into them still throws, since Vulkan would apply alpha factors to the wrong channel. Uncharted: Legacy of Thieves Collection (PPSA05684) stopped in its first level with "unsupported color format 3 number type 0 component swap 1". --- core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp | 11 +++++++---- core/libs/prx/libSceAgcDriver/tests/Graphics.cpp | 8 ++++++++ 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp index 60d0c2dad..f29320922 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp @@ -398,11 +398,13 @@ DecodedColorFormat DecodeColorFormat(std::uint32_t format, std::uint32_t number, if (number == floating) return single(VK_FORMAT_R16_SFLOAT, 2); if (number == uint) return single(VK_FORMAT_R16_UINT, 2); return fail(); - case 3: - if (swap != 0) return fail(); - if (number == unorm) return {VK_FORMAT_R8G8_UNORM, 2}; - if (number == snorm) return {VK_FORMAT_R8G8_SNORM, 2}; + case 3: { + if (swap > 1) return fail(); + const auto mapping = static_cast(alternate ? 0xecu : 0xe4u); + if (number == unorm) return {VK_FORMAT_R8G8_UNORM, 2, mapping}; + if (number == snorm) return {VK_FORMAT_R8G8_SNORM, 2, mapping}; return fail(); + } case 4: if (number == floating) return single(VK_FORMAT_R32_SFLOAT, 4); if (number == uint) return single(VK_FORMAT_R32_UINT, 4); @@ -725,6 +727,7 @@ State DecodeState(const QueueState& queue) { state.blendEnable = (blend >> 30u) & 1u; if (state.blendEnable && (mapping == 0x1bu || mapping == 0x93u)) throw std::runtime_error("AGC graphics: blending into a color target with a reversed component order is not implemented"); Require(!state.blendEnable || !color.uintExport, "blending into an unsigned integer target is unsupported"); + if (state.blendEnable && mapping == 0xecu) throw std::runtime_error("AGC graphics: blending into an 8_8 color target with the alternate component order is not implemented"); if (state.blendEnable) { Require((read(cx, 0x31c + slot * 0xfu) & 0x10000u) == 0, "blend bypass conflicts with enabled blending"); state.srcColorBlendFactor = blendFactor(blend & 0x1fu); diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index ee6645aad..70d0b899b 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -761,6 +761,14 @@ void TuningFieldTests() { } void ReversedComponentOrderTests() { + { + auto queue = makeState(); + queue.context[0x31c] = (queue.context[0x31c] & ~((0x1fu << 2u) | (7u << 8u) | (3u << 11u))) | (3u << 2u) | (1u << 11u); + const auto state = AgcDriver::Graphics::DecodeState(queue); + Require(state.colors.size() == 1 && state.colors[0].format == VK_FORMAT_R8G8_UNORM && state.colors[0].componentMapping == 0xecu, "an 8_8 target with SWAP_ALT did not store the x and w exports in its two channels"); + queue.context[0x1e0] = 0x40010001u; + expectFailure([&] { AgcDriver::Graphics::DecodeState(queue); }, "alternate component order"); + } for (const auto& [swap, mapping] : {std::pair{2u, 0x1bu}, std::pair{3u, 0x93u}}) { auto queue = makeState(); queue.context[0x31c] = (queue.context[0x31c] & ~(3u << 11u)) | (swap << 11u); From c7cd20e82f59ed00e6abfa61721354cb66c6e5aa Mon Sep 17 00:00:00 2001 From: XairRrR Date: Fri, 9 Oct 2026 12:32:01 +0200 Subject: [PATCH 095/460] feat(libSceVoice): track the port mute flag sceVoiceSetMuteFlag was NotImplemented with an unknown signature and sceVoiceGetMuteFlag was not exported. Signatures follow shadPS4's voice.h: a port id and a bool, and for sceVoiceGetMuteFlag a port id and a bool pointer. The flag is stored per port like the volume; an unknown port throws and a null output pointer is an invalid argument. Uncharted: Legacy of Thieves Collection (PPSA05684) imports sceVoiceGetMuteFlag (Pc4z1QjForU), so on main it stops at startup with "undefined symbol: Pc4z1QjForU". --- core/libs/prx/libSceVoice/Export.cpp | 15 +++++++++++++-- docs/dev/TechnicalDebt.md | 1 - 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libSceVoice/Export.cpp b/core/libs/prx/libSceVoice/Export.cpp index 898511c88..ba55f55dd 100644 --- a/core/libs/prx/libSceVoice/Export.cpp +++ b/core/libs/prx/libSceVoice/Export.cpp @@ -24,6 +24,7 @@ struct Port { std::int32_t type; float volume; std::uint32_t bitrate; + bool muted = false; }; struct Voice { @@ -204,8 +205,18 @@ int APS5_VABI sceVoiceWriteToIPort(uint32_t input_port_id, const void* data, uin return 0; } -int APS5_VABI sceVoiceSetMuteFlag() { - NotImplemented_nid_no_patch(__func__); +int APS5_VABI sceVoiceSetMuteFlag(uint32_t port_id, bool muted) { + auto& voice = State(); + std::lock_guard lock(voice.mutex); + RequirePort(voice, port_id, __func__).muted = muted; + return 0; +} + +int APS5_VABI sceVoiceGetMuteFlag(uint32_t port_id, bool* muted) { + if (muted == nullptr) APS5_INVALID_ARG_EX; + auto& voice = State(); + std::lock_guard lock(voice.mutex); + *muted = RequirePort(voice, port_id, __func__).muted; return 0; } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..2bc086001 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -234,7 +234,6 @@ Throughout the project, every function at every stage either **does exactly what - [sceNgs2VoiceGetPortInfo](../../core/libs/prx/libSceNgs2.native/src/Voice.cpp) (libSceNgs2.native) - layout from shadPS4; the delay and destination input are zero because ports have neither yet - [sceNgs2ParseWaveformFile](../../core/libs/prx/libSceNgs2.native/src/Atrac9.cpp) (libSceNgs2.native) - the signature `(path, offset, info)` is taken from the call site of PPSA21402, which sets only three argument registers; the file is read from `offset` to its end, the data and block offsets are taken to be relative to the start of the file, not of `offset`, and an unreadable file throws. The audio unit fields of a 16-bit PCM waveform are assumed to be one sample frame per unit and per frame; `INVALID_WAVEFORM_DATA` for a `data` chunk that runs past the buffer or ends inside a frame is assumed, and looped PCM and other PCM formats throw - [sceSaveDataInitialize3](../../core/libs/prx/libSceSaveData.native/Export.cpp) (libSceSaveData.native) - a repeated initialize succeeds: PPSA12544's executable and its Unity SaveData plugin both initialize, and the plugin fails on any error. Initializations are counted and `sceSaveDataTerminate` ends the session at the last one; how the console pairs them is unknown -- [sceVoiceSetMuteFlag](../../core/libs/prx/libSceVoice/Export.cpp) (libSceVoice) - unknown signature - [sceTextToSpeech2GetSystemStatus](../../core/libs/prx/libSceTextToSpeech2/Export.cpp) (libSceTextToSpeech2) - unknown signature - [sceCoredumpAttachUserMemoryFile](../../core/libs/prx/libkernel/System/src/Coredump.cpp) (libkernel) - unknown signature - [sceCoredumpAttachMemoryRegion](../../core/libs/prx/libkernel/System/src/Coredump.cpp) (libkernel) - unknown signature From 252955a18553be70482c561a0f11bb911e24e786 Mon Sep 17 00:00:00 2001 From: XairRrR Date: Fri, 9 Oct 2026 17:27:32 +0200 Subject: [PATCH 096/460] fix(agc-driver): name the view and owner bounds when a guest buffer view exceeds its GPU owner --- .../libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp index 8b206eb5d..488984ccf 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp @@ -2863,7 +2863,12 @@ VkDescriptorBufferInfo GuestBufferMemory::Descriptor(std::uint64_t address, std: const auto* found = owner(address); Require(found != nullptr, "guest buffer has no GPU owner"); const auto& region = *found; - Require(address >= region.begin && address + bytes <= region.end && (region.buffer != nullptr || region.direct != nullptr || region.mirror != nullptr), "guest buffer view exceeds its GPU owner"); + const auto describe = [&](const char* what) { + char text[256]; + std::snprintf(text, sizeof(text), "guest buffer view 0x%llx+0x%llx %s its GPU owner 0x%llx-0x%llx (buffer %d, direct %d, mirror %d)", static_cast(address), static_cast(bytes), what, static_cast(region.begin), static_cast(region.end), region.buffer != nullptr, region.direct != nullptr, region.mirror != nullptr); + return std::string(text); + }; + if (!(address >= region.begin && address + bytes <= region.end && (region.buffer != nullptr || region.direct != nullptr || region.mirror != nullptr))) Require(false, describe("exceeds")); const auto base = region.direct != nullptr ? region.direct->base : region.mirror != nullptr ? region.mirror->base : region.begin; const auto offset = address - base; Require(context.limits.minStorageBufferOffsetAlignment != 0, "no storage buffer offset alignment"); @@ -2871,7 +2876,7 @@ VkDescriptorBufferInfo GuestBufferMemory::Descriptor(std::uint64_t address, std: adjustment = static_cast(offset % std::max(context.limits.minStorageBufferOffsetAlignment, 4)); const auto range = ViewBytes(bytes, adjustment); const auto end = region.direct != nullptr ? region.direct->base + region.direct->bytes : region.end; - Require(address - adjustment + range <= end, "guest buffer view exceeds its GPU owner"); + if (address - adjustment + range > end) Require(false, describe("rounded up to its view range exceeds")); Require(range <= context.limits.maxStorageBufferRange, "guest buffer view exceeds descriptor range limit"); const auto handle = region.direct != nullptr ? region.direct->buffer : region.mirror != nullptr ? region.mirror->buffer->Handle() : region.buffer->Handle(); return {handle, offset - adjustment, range}; From 4adca33251b0622571034549c2cc701c7cdf6f88 Mon Sep 17 00:00:00 2001 From: XairRrR Date: Fri, 9 Oct 2026 14:45:32 +0200 Subject: [PATCH 097/460] fix(recompiler): describe the descriptor and modes when no runtime image mode matches --- .../recompiler/Optimization/src/ResourceMaterializer.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp b/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp index f1e3a5545..cf7514e63 100644 --- a/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp +++ b/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp @@ -779,7 +779,14 @@ std::uint32_t ResourceMaterializer::RuntimeImageMode(const ImageResource& image, if (mode.numericClass == decoded.numericClass && mode.dimension == decoded.dimension && mode.conversionFormat == decoded.conversionFormat && mode.packedFormat == decoded.packedFormat && mode.cube == decoded.cube && mode.depthBits == decoded.depthBits && mode.depthUnorm16 == decoded.depthUnorm16 && mode.srgbDecode == decoded.srgbDecode) return index; } if (image.dimension == RdnaImageDimension::Dim1D && decoded.dimension != RdnaImageDimension::Dim1D) throw std::runtime_error("image address has too few coordinate components"); - throw std::runtime_error("image descriptor is incompatible with the static runtime image interface"); + const auto describe = [](IrTextureNumericClass numeric, RdnaImageDimension dimension, IrBufferFormat conversion, IrBufferFormat packed, bool cube, bool depthBits, bool depthUnorm16, bool srgb) { + return "{class " + std::to_string(static_cast(numeric)) + ", dim " + std::to_string(static_cast(dimension)) + ", conversion " + std::to_string(static_cast(conversion)) + ", packed " + std::to_string(static_cast(packed)) + ", cube " + std::to_string(cube) + ", depthBits " + std::to_string(depthBits) + ", unorm16 " + std::to_string(depthUnorm16) + ", srgb " + std::to_string(srgb) + "}"; + }; + std::string message = "image descriptor is incompatible with the static runtime image interface: descriptor format " + std::to_string(static_cast(format)) + " decodes to " + describe(decoded.numericClass, decoded.dimension, decoded.conversionFormat, decoded.packedFormat, decoded.cube, decoded.depthBits, decoded.depthUnorm16, decoded.srgbDecode) + (emulated ? " (emulated compare)" : "") + "; modes:"; + for (const auto& mode : modes) message += " " + describe(mode.numericClass, mode.dimension, mode.conversionFormat, mode.packedFormat, mode.cube, mode.depthBits, mode.depthUnorm16, mode.srgbDecode); + char words[96]; + std::snprintf(words, sizeof(words), "; T# %08x %08x %08x %08x %08x %08x %08x %08x", descriptor.dwords[0], descriptor.dwords[1], descriptor.dwords[2], descriptor.dwords[3], descriptor.dwords[4], descriptor.dwords[5], descriptor.dwords[6], descriptor.dwords[7]); + throw std::runtime_error(message + words); } std::uint32_t ResourceMaterializer::EmulatedCompareState(const ShaderInfo& info, const ResourceSnapshot& snapshot, std::uint32_t index) { From e1dbb8fa17520debadd8e988ad2cb57fa6819fd6 Mon Sep 17 00:00:00 2001 From: XairRrR Date: Fri, 9 Oct 2026 18:12:50 +0200 Subject: [PATCH 098/460] fix(agc-driver): list both stages' interfaces when graphics interfaces disagree --- .../prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp index e880bddaf..c5d9168d1 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp @@ -573,10 +573,16 @@ std::set ValidateShaders(std::span shaders, for (const auto& [location, signature] : current.inputs) { const auto output = previous.outputs.find(location); if (output == previous.outputs.end() || output->second != signature) { + std::string exports; + for (const auto value : shaders[i - 1].program->parameterExports) exports += " " + std::to_string(value); + std::string parameters; + for (const auto& parameter : shaders[i].program->fragmentParameters) parameters += " " + std::to_string(parameter.location) + "<-" + std::to_string(parameter.sourceLocation); + std::string outputs; + for (const auto& [index, kind] : previous.outputs) outputs += " " + std::to_string(index) + ":" + kind; Require(false, "graphics interfaces disagree at location " + std::to_string(location) + " (stage " + std::to_string(i - 1) + " output " + (output == previous.outputs.end() ? "missing" : output->second) + - ", stage " + std::to_string(i) + " input " + signature + ")"); + ", stage " + std::to_string(i) + " input " + signature + "; previous outputs:" + outputs + "; parameter exports:" + exports + "; fragment parameters:" + parameters + ")"); } } } From 1232fd6cdf7ef183e40da5e6358220eed9f7807e Mon Sep 17 00:00:00 2001 From: XairRrR Date: Sat, 10 Oct 2026 13:33:14 +0200 Subject: [PATCH 099/460] feat(agc-driver): prepare shader variants registration did not predict Registration prepares the variants a shader's registered state predicts; a draw or dispatch whose state needs another one threw "prepared shader artifact is missing for the requested static ABI". Titles that link their own libSceAgc build that state themselves and never call the driver's static ABI resolution, so their draws differ from the registered prediction. The missing artifact is now prepared at use, logged, and kept with the snapshot like the deferred and null pixel cases. SourceHandleFor now checks that the request refers to the registered code, as InvocationFor already did. Uncharted: Legacy of Thieves Collection (PPSA05684) still misses on main 197ffac9 (after the fixed push slots of #2661) for its own pixel shaders: "prepared shader artifact is missing for the requested static ABI: address=85899411712 stage=5 wave=64 pushOffset=128 pushCapacity=128". --- .../src/Driver/Shaders/ShaderRegistry.cpp | 52 +++++++------------ .../tests/execution/PreparedGraphics.cpp | 8 ++- .../tests/execution/PreparedShaders.cpp | 43 +++++++++++---- docs/dev/ARCHITECTURE.md | 1 + 4 files changed, 60 insertions(+), 44 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp index 52a46d03b..af6348610 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp @@ -227,20 +227,24 @@ void PublishRegisteredShader(std::shared_ptr& registry, const st namespace { -bool PreparedAtUse(const ShaderSnapshot& snapshot, const ShaderRecompiler::RecompileRequest& request) { +void ReportPreparedAtUse(const ShaderSnapshot& snapshot, const ShaderRecompiler::RecompileRequest& request) { if (snapshot.codeAddress == NullPixelProgramAddress() && request.shader.stage == ShaderRecompiler::ShaderStage::Fragment && request.context.waveSize == 32u) { APS5_LOG_ERR("The null pixel program has no wave%u artifact for this draw; preparing it at draw", request.context.waveSize); - return true; + return; + } + if (!snapshot.header.empty()) { + if (!snapshot.prepared->deferred) APS5_LOG_ERR("Shader 0x%llx stage %u has no artifact prepared at registration for the state of this draw or dispatch; preparing it at use", static_cast(request.shader.codeAddress), static_cast(request.shader.stage)); + return; } - if (!snapshot.header.empty()) return snapshot.prepared->deferred; if (snapshot.type != 0 || request.shader.stage != ShaderRecompiler::ShaderStage::Compute) throw std::runtime_error("AGC driver: unregistered program is not a compute shader"); APS5_LOG_ERR("Compute shader 0x%llx was not registered; preparing its artifact at dispatch", static_cast(snapshot.codeAddress)); - return true; } } std::shared_ptr SourceHandleFor(const ShaderSnapshot& snapshot, std::size_t codeOffset, const ShaderRecompiler::RecompileRequest& request) { + require(codeOffset < snapshot.code.size(), "prepared shader code offset is outside the snapshot"); + require(std::ranges::equal(request.shader.code, std::span(snapshot.code).subspan(codeOffset)), "prepared shader request does not refer to registered code"); struct PreparedKeyStorage {}; auto& key = HostThreadLocal, PreparedKeyStorage>(); ShaderRecompiler::BuildPreparedShaderKey(request, key); @@ -249,18 +253,10 @@ std::shared_ptr SourceHandleFor(const Shad for (const auto& entry : snapshot.prepared->entries) { if (entry.codeOffset == codeOffset && ShaderRecompiler::MatchesPreparedShader(request, *entry.handle, key)) return entry.handle; } - if (PreparedAtUse(snapshot, request)) { - auto handle = PrepareShaderWithDiagnostics(request); - snapshot.prepared->entries.push_back({codeOffset, handle}); - return handle; - } - std::string layouts; - for (const auto& entry : snapshot.prepared->entries) { - if (entry.codeOffset != codeOffset || entry.handle == nullptr || entry.handle->artifact == nullptr) continue; - const auto& layout = entry.handle->artifact->layout; - layouts += " [" + std::to_string(layout.pushConstantOffsetBytes) + "," + std::to_string(layout.pushConstantSizeBytes) + "]"; - } - throw std::runtime_error("AGC driver: prepared shader artifact is missing for the requested static ABI: address=" + std::to_string(request.shader.codeAddress) + " stage=" + std::to_string(static_cast(request.shader.stage)) + " wave=" + std::to_string(request.context.waveSize) + " pushOffset=" + std::to_string(request.layout.pushConstantOffsetBytes) + " pushCapacity=" + std::to_string(request.layout.pushConstantSizeBytes) + " preparedLayouts=" + layouts); + ReportPreparedAtUse(snapshot, request); + auto handle = PrepareShaderWithDiagnostics(request); + snapshot.prepared->entries.push_back({codeOffset, handle}); + return handle; } ShaderRecompiler::RectListShaders PreparedRectangle(const ShaderSnapshot& snapshot, std::uint64_t vertexId, std::uint64_t fragmentId) { @@ -300,22 +296,14 @@ ShaderRecompiler::PreparedShaderInvocation InvocationFor(const ShaderSnapshot& s invocationRequest.shader.code = ShaderRecompiler::GetPreparedCode(*entry.handle); if (auto invocation = ShaderRecompiler::PreparedShaderInvocation::TryCreate(invocationRequest, entry.handle, key)) return std::move(*invocation); } - if (PreparedAtUse(snapshot, request)) { - auto handle = PrepareShaderWithDiagnostics(request); - invocationRequest = request; - invocationRequest.shader.code = ShaderRecompiler::GetPreparedCode(*handle); - auto invocation = ShaderRecompiler::PreparedShaderInvocation::TryCreate(invocationRequest, handle, key); - if (!invocation.has_value()) throw std::runtime_error("AGC driver: raw compute artifact does not match its invocation"); - snapshot.prepared->entries.push_back({codeOffset, std::move(handle)}); - return std::move(*invocation); - } - std::string layouts; - for (const auto& entry : snapshot.prepared->entries) { - if (entry.codeOffset != codeOffset || entry.handle == nullptr || entry.handle->artifact == nullptr) continue; - const auto& layout = entry.handle->artifact->layout; - layouts += " [" + std::to_string(layout.pushConstantOffsetBytes) + "," + std::to_string(layout.pushConstantSizeBytes) + "]"; - } - throw std::runtime_error("AGC driver: prepared shader artifact is missing for the requested static ABI: address=" + std::to_string(request.shader.codeAddress) + " stage=" + std::to_string(static_cast(request.shader.stage)) + " wave=" + std::to_string(request.context.waveSize) + " pushOffset=" + std::to_string(request.layout.pushConstantOffsetBytes) + " pushCapacity=" + std::to_string(request.layout.pushConstantSizeBytes) + " preparedLayouts=" + layouts); + ReportPreparedAtUse(snapshot, request); + auto handle = PrepareShaderWithDiagnostics(request); + invocationRequest = request; + invocationRequest.shader.code = ShaderRecompiler::GetPreparedCode(*handle); + auto invocation = ShaderRecompiler::PreparedShaderInvocation::TryCreate(invocationRequest, handle, key); + if (!invocation.has_value()) throw std::runtime_error("AGC driver: an artifact prepared at use does not match its invocation"); + snapshot.prepared->entries.push_back({codeOffset, std::move(handle)}); + return std::move(*invocation); } std::optional RegisteredFloatMode(const ShaderSnapshot& snapshot) { if (snapshot.registeredState == nullptr) return std::nullopt; diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedGraphics.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedGraphics.cpp index eb1a511a1..db2a20d9e 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedGraphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedGraphics.cpp @@ -231,13 +231,17 @@ void Check(AgcDriver::VulkanDevice& device, AgcDriver::Graphics::ShaderPath path request.layout.pushConstantSizeBytes -= 4; static_cast(InvocationFor(*program.snapshot, program.codeOffset, request)); request.layout.pushConstantOffsetBytes = 2; - Reject([&] { static_cast(InvocationFor(*program.snapshot, program.codeOffset, request)); }, "artifact is missing"); + Reject([&] { static_cast(InvocationFor(*program.snapshot, program.codeOffset, request)); }, "not dword-aligned"); request.layout = {0, 0, 0, mesh ? MeshDrawPushOffsetBytes : 128u}; if (pixel) request.context.pixel->interpolatorSettings[0] ^= 1u; else if (tessellation) ++request.graphics->tessellation->outputControlPoints; else if (mesh) ++request.graphics->mesh->maxVertices; else ++request.context.userDataBaseRegister; - Reject([&] { static_cast(InvocationFor(*program.snapshot, program.codeOffset, request)); }, "artifact is missing"); + const auto predicted = program.snapshot->prepared->entries.size(); + static_cast(InvocationFor(*program.snapshot, program.codeOffset, request)); + Require(program.snapshot->prepared->entries.size() == predicted + 1, "a draw state that registration did not predict was not prepared"); + static_cast(InvocationFor(*program.snapshot, program.codeOffset, request)); + Require(program.snapshot->prepared->entries.size() == predicted + 1, "an artifact prepared at a draw was not reused"); } ShaderRegistry invalidRegistry; DrawDecode invalid{}; diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp index 695192b8d..1a60e4153 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp @@ -19,6 +19,8 @@ #include #include #include +#include +#include #include #include #include @@ -92,12 +94,18 @@ void Run(AgcDriver::VulkanDevice& device) { const ShaderRecompiler::ShaderComputeStageInfo compute{{1, 1, 1}, 0, {false, false, false}, false, 1, {}}; ShaderRecompiler::RecompileRequest request{{ShaderRecompiler::ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, {32, 0, users, compute, {}, {}, {}}, device.ComputeTarget(32), {0, 0, 0, 128}}; RunUnregistered(device, request); + { + AgcDriver::DriverDetail::ShaderSnapshot unpredicted{request.shader.codeAddress, 0, 0, {code.begin(), code.end()}, {}}; + unpredicted.header.resize(sizeof(Shader)); + const auto prepared = AgcDriver::DriverDetail::SourceHandleFor(unpredicted, 0, request); + Require(unpredicted.prepared->entries.size() == 1 && unpredicted.prepared->entries.front().handle == prepared, "a registered shader with no matching artifact was not prepared"); + Require(AgcDriver::DriverDetail::SourceHandleFor(unpredicted, 0, request) == prepared && unpredicted.prepared->entries.size() == 1, "an artifact prepared at a dispatch was not reused"); + } AgcDriver::DriverDetail::ShaderSnapshot snapshot{request.shader.codeAddress, 0, 0, {code.begin(), code.end()}, {}}; snapshot.header.resize(sizeof(Shader)); - ExpectFailure([&] { static_cast(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request)); }, "artifact is missing"); const auto handle = ShaderRecompiler::PrepareShader(request); snapshot.prepared->entries.push_back({0, handle}); - ExpectFailure([&] { static_cast(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 1, request)); }, "artifact is missing"); + ExpectFailure([&] { static_cast(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 1, request)); }, "outside the snapshot"); const auto& artifact = ShaderRecompiler::GetPreparedArtifact(*handle); for (std::uint32_t value = 0; value < 8; ++value) { users.fill(value); @@ -118,7 +126,10 @@ void Run(AgcDriver::VulkanDevice& device) { ExpectFailure([&] { static_cast(AgcDriver::DriverDetail::InvocationFor(snapshot, 0, request)); }, "does not refer to registered code"); ExpectFailure([&] { static_cast(AgcDriver::DriverDetail::InvocationFor(snapshot, 1, registeredRequest)); }, "outside the snapshot"); registeredRequest.context.compute->numThreads[0] = 2; - ExpectFailure([&] { static_cast(AgcDriver::DriverDetail::InvocationFor(snapshot, 0, registeredRequest)); }, "artifact is missing"); + static_cast(AgcDriver::DriverDetail::InvocationFor(snapshot, 0, registeredRequest)); + Require(snapshot.prepared->entries.size() == 2, "a dispatch state that registration did not predict was not prepared"); + static_cast(AgcDriver::DriverDetail::InvocationFor(snapshot, 0, registeredRequest)); + Require(snapshot.prepared->entries.size() == 2, "an artifact prepared at a dispatch was not reused"); ShaderRecompiler::SrtRuntime preparedRuntime{}; preparedRuntime.userData = users; const auto preparedCapture = invocation.Capture(preparedRuntime); @@ -144,12 +155,13 @@ void Run(AgcDriver::VulkanDevice& device) { replacement->Dispatch(*replacementResult, 1, 1, 1); replacement->WaitIdle(); replacementRequest.target.nonConstantImageOffsets = !replacementRequest.target.nonConstantImageOffsets; - ExpectFailure([&] { static_cast(AgcDriver::DriverDetail::InvocationFor(snapshot, 0, replacementRequest)); }, "artifact is missing"); + static_cast(AgcDriver::DriverDetail::InvocationFor(snapshot, 0, replacementRequest)); + Require(snapshot.prepared->entries.size() == 3, "a target that registration did not prepare for was not prepared"); ShaderRecompiler::SrtRuntime runtime{}; runtime.userData = users; const auto capture = ShaderRecompiler::CaptureResources(request, runtime, *handle); request.context.compute->numThreads[0] = 2; - ExpectFailure([&] { static_cast(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request)); }, "artifact is missing"); + Require(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request) == snapshot.prepared->entries[1].handle, "a dispatch did not find the artifact prepared for its state"); ExpectFailure([&] { static_cast(ShaderRecompiler::CaptureResources(request, runtime, *handle)); }, "does not match the static ABI"); ExpectFailure([&] { static_cast(ShaderRecompiler::MaterializeShader(request, *capture, *handle)); }, "does not match the static ABI"); const auto otherHandle = ShaderRecompiler::PrepareShader(request); @@ -160,10 +172,10 @@ void Run(AgcDriver::VulkanDevice& device) { request.layout.pushConstantSizeBytes = 124; Require(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request) == handle, "compatible push constant capacity discarded the prepared artifact"); request.layout.pushConstantSizeBytes = 126; - ExpectFailure([&] { static_cast(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request)); }, "artifact is missing"); + ExpectFailure([&] { static_cast(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request)); }, "not dword-aligned"); request.layout.pushConstantSizeBytes = 128; code[0] = 0xffffffffu; - ExpectFailure([&] { static_cast(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request)); }, "artifact is missing"); + ExpectFailure([&] { static_cast(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request)); }, "does not refer to registered code"); } void FailureCapture(AgcDriver::VulkanDevice& device) { @@ -434,9 +446,20 @@ void Registration(bool indirect) { commands.insert(commands.end(), {0xc0033700u, 0x00100200u, static_cast(destinationAddress), static_cast(destinationAddress >> 32u), 7}); packet = Packet{commands.data(), static_cast(commands.size()), 0, {}}; sceAgcDriverSubmitAcb(0x20, &packet); - ExpectFailure([] { AgcDriverWaitIdle_nid_postfix(); }, "artifact is missing"); - ExpectFailure([] { AgcDriverWaitIdle_nid_postfix(); }, "artifact is missing"); - ExpectFailure([] { AgcDriverShutdown_nid_postfix(); }, "artifact is missing"); + AgcDriverWaitIdle_nid_postfix(); + const auto deadline = std::chrono::steady_clock::now() + std::chrono::seconds(10); + while (std::atomic_ref(destination).load() != 7 && std::chrono::steady_clock::now() < deadline) std::this_thread::yield(); + Require(destination == 7, "a dispatch whose state registration did not predict did not run"); + destination = 0; + alignas(256) static const std::array invalid{0xffffffffu}; + const auto invalidAddress = reinterpret_cast(invalid.data()); + commands[2] = static_cast(invalidAddress >> 8u); + commands[5] = static_cast(invalidAddress >> 40u); + packet = Packet{commands.data(), static_cast(commands.size()), 0, {}}; + sceAgcDriverSubmitAcb(0x20, &packet); + ExpectFailure([] { AgcDriverWaitIdle_nid_postfix(); }, ""); + ExpectFailure([] { AgcDriverWaitIdle_nid_postfix(); }, ""); + ExpectFailure([] { AgcDriverShutdown_nid_postfix(); }, ""); Require(destination == 0, "failed dispatch executed a subsequent memory write"); } diff --git a/docs/dev/ARCHITECTURE.md b/docs/dev/ARCHITECTURE.md index 3713b767c..4e3165c6a 100644 --- a/docs/dev/ARCHITECTURE.md +++ b/docs/dev/ARCHITECTURE.md @@ -70,6 +70,7 @@ flowchart LR - A compute shader whose LDS plus its lock dword exceeds the host's `maxComputeSharedMemorySize` is not given a `Workgroup` array. `Recompile` sets `ShaderInfo::sharedMemoryBytes` for it and the SPIR-V backend addresses the LDS in a coherent storage buffer at descriptor set 1 (`WorkgroupMemoryDescriptorSet`), with a slice of `RecompileResult::workgroupMemoryDwords` per workgroup, indexed by `WorkgroupID` and `NumWorkGroups`. `VulkanDevice` keeps one such buffer, sized from the dispatch dimensions, and binds it after the resource set; programs that fit keep using shared memory. - `Driver::RegisterShader` splits shader preparation in two. `PlanRegistered` runs on the registering guest thread: it validates the registers and entry point, decodes the code and builds the recompile request, so invalid registrations still fail there. The RDNA to IR translation and resource plan (`PrepareShader`) run on a background pool of `hardware_concurrency() / 2` detached threads (at least 2, 64 MiB stacks). - The snapshot is published as pending. `SourceHandleFor`, `InvocationFor` and `ShaderPreparationTransaction::Edit` / `Read` wait for the pending job before reading `PreparedShaders`, and rethrow its failure. `APS5_SYNC_SHADER_PREPARE=1` prepares on the registering thread instead. +- A draw or dispatch whose state matches no artifact prepared at registration (a title that builds its static ABI state itself, without `sceAgcDriverResolve*Abi`) gets one prepared at use by `SourceHandleFor` or `InvocationFor`, logged once per artifact and kept in the snapshot for the next use. - A draw's `Graphics::Pipeline` comes from `CachedPipeline`. When the device has `VK_EXT_graphics_pipeline_library` with fast linking, `VK_EXT_extended_dynamic_state` and `VK_KHR_dynamic_rendering`, draws render with `vkCmdBeginRenderingKHR` instead of render passes and the pipeline is linked from four libraries (vertex input, pre-rasterization shaders, fragment shader, fragment output) kept per device in [`PipelineLibrary.cpp`](../../core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp). A library built for one pipeline is reused by the next: another blend, vertex layout or target format builds only that part (the shader libraries name no attachment format), and cull mode, front face, depth and stencil state are dynamic, so changing them only links. The fast link is what the draw waits for; a background thread then links the same libraries with link-time optimization and the pipeline binds that one from the next draw on. Push constants then use a 256-byte block with a 128-byte slot per side: the pre-rasterization stages stack from byte 0, the pixel shader always starts at byte 128 (`StagePushOffset`), so its library does not depend on how much push data its vertex shader uses. Pipelines with per-face depth bias, rect lists, geometry shaders or a stage without a variant id are created whole, still with dynamic rendering; devices without the extensions (or with `APS5_NO_GPL=1`) keep render passes and whole pipelines. ### Shader MODE register access From a1827af12a69ea3f1bdcc402da82a8194befe143 Mon Sep 17 00:00:00 2001 From: AlxcBTW <187763517+AlxcBTW@users.noreply.github.com> Date: Sat, 10 Oct 2026 20:03:01 +0200 Subject: [PATCH 100/460] fix(agc): treat other-shaped views of a depth surface's stencil plane as reused memory Unity reuses the memory of transient render targets: in FPV Simulator (PPSA10698) an RGBA8 1920x1080 color texture is sampled at the address that a D32S8 surface's stencil plane occupied earlier. The lookup found the stencil plane and threw "sampling the stencil plane ... is not implemented", which ended the presentation. 9d65003c made depth-plane views in a format that cannot hold the plane fall through to the ordinary texture path. A stencil-plane view whose texels are not one byte wide cannot hold the 8-bit stencil plane either, so it now falls through the same way. AI-assisted: yes Co-Authored-By: Claude Opus 5.5 --- .../prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp | 2 ++ .../libs/prx/libSceAgcDriver/tests/DepthSurfaceReuse.cpp | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp index f7ace9d03..0b8982df3 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp @@ -213,6 +213,8 @@ std::shared_ptr DepthSurfaceTexture(const Context& context, std::span= 4 && ShaderRecompiler::DepthBitsTextureWidth(words[1], words[3]) == (d16 ? 16u : 32u); if (ResolveTextureFormat(resource.format) != (d16 ? VK_FORMAT_R16_UNORM : VK_FORMAT_R32_SFLOAT) && !depthBits) return nullptr; + } else if (BytesPerElement(resource.format) != 1u) { + return nullptr; } return (*found)->Sampled(words, resource, components); } diff --git a/core/libs/prx/libSceAgcDriver/tests/DepthSurfaceReuse.cpp b/core/libs/prx/libSceAgcDriver/tests/DepthSurfaceReuse.cpp index b91361cea..63ddb1ae1 100644 --- a/core/libs/prx/libSceAgcDriver/tests/DepthSurfaceReuse.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/DepthSurfaceReuse.cpp @@ -12,11 +12,13 @@ namespace { using namespace AgcDriver::Graphics; +constexpr std::uint32_t FormatR8Uint = 5; constexpr std::uint32_t FormatR16Unorm = 7; constexpr std::uint32_t FormatR16Uint = 11; constexpr std::uint32_t FormatR32Uint = 20; constexpr std::uint32_t FormatR32Float = 22; constexpr std::uint32_t FormatR11G11B10Float = 36; +constexpr std::uint32_t FormatR8G8B8A8Unorm = 56; constexpr std::uint32_t TileDepth64KB = 0x18; constexpr std::uint32_t TileRenderTarget64KB = 0x1b; constexpr std::uint32_t Type2D = 9; @@ -163,8 +165,11 @@ void RunDepthSurfaceReuseTests() { context.limits.maxFramebufferHeight = 16384; constexpr std::uint64_t depth32 = 0x40000000; constexpr std::uint64_t depth16 = 0x50000000; + constexpr std::uint64_t depthStencil = 0x60000000; + constexpr std::uint64_t stencil = 0x60800000; DepthSurfaceView(context, {depth32, 0, {384, 384}, VK_FORMAT_D32_SFLOAT, 1.0f, 0}); DepthSurfaceView(context, {depth16, 0, {256, 256}, VK_FORMAT_D16_UNORM, 1.0f, 0}); + DepthSurfaceView(context, {depthStencil, stencil, {320, 180}, VK_FORMAT_D32_SFLOAT_S8_UINT, 1.0f, 0}); Require(lookup(context, {depth32, FormatR32Float, 384, 384}) != nullptr, "an R32F view of a D32 surface's own extent must sample its depth plane"); Require(lookup(context, {depth16, FormatR16Unorm, 256, 256}) != nullptr, "an R16 view of a D16 surface's own extent must sample its depth plane"); @@ -177,5 +182,9 @@ void RunDepthSurfaceReuseTests() { Require(lookup(context, {depth32, FormatR32Uint, 384, 384, TileRenderTarget64KB}) == nullptr, "an R32 uint view without a depth layout must be read as reused memory"); Require(lookup(context, {depth16, FormatR32Float, 256, 256}) == nullptr, "an R32F view of a D16 surface's own extent must be read as reused memory"); + Require(lookup(context, {stencil, FormatR8Uint, 320, 180}) != nullptr, "an 8-bit view of a stencil plane's own extent must sample the stencil plane"); + Require(lookup(context, {stencil, FormatR8G8B8A8Unorm, 320, 180, TileRenderTarget64KB}) == nullptr, "an RGBA8 view of a stencil plane's own extent must be read as reused memory"); + Require(lookup(context, {stencil, FormatR32Float, 320, 180}) == nullptr, "an R32F view of a stencil plane's own extent must be read as reused memory"); + ClearDepthSurfaces(context.device); } From 3444ebf5cdde062aaae30c7fe31cdb517d3ac32b Mon Sep 17 00:00:00 2001 From: ps54all Date: Sun, 11 Oct 2026 07:10:29 +1300 Subject: [PATCH 101/460] fix(libkernel): map non-fixed hints inside a reserved range in place A map without MAP_FIXED whose hint lies wholly inside a range still reserved by sceKernelReserveVirtualRange is placed at the hint, as a fixed map into the reservation would be. Hints outside reservations and ranges that are partly mapped keep their hint behaviour. The reservation is checked under the registry mutation, so it cannot be released and its address reused between the check and the map (MarcZX100, #2687). Fixes #2687 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A2cY3SsTYnh4u4ZtEr2Adx --- .../libkernel/DirectMemory/DirectMemory.cpp | 7 ++++ core/libs/tests/GuestMemory.cpp | 32 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 40 insertions(+) diff --git a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp index ff3933405..85a6b373e 100644 --- a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp +++ b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp @@ -762,6 +762,11 @@ bool FixedNoOverwriteConflict(const GuestAllocations::Mutation& mutation, void* return mutation.Overlaps(addr, len) && !Reserved(addr, len); } +int PlaceHintInReservation(void* addr, size_t len, int flags) { + if (addr == nullptr || (flags & GuestMapFixedFlag) != 0 || !Reserved(addr, len)) return flags; + return flags | GuestMapFixedFlag; +} + int DoMapDirect(void** addr, size_t len, int prot, int flags, int64_t physStart, size_t alignment) { ValidateOutput(addr); if (len == 0 || (len & (PS5_PAGE_SIZE - 1)) != 0) return SCE_KERNEL_ERROR_EINVAL; @@ -769,6 +774,7 @@ int DoMapDirect(void** addr, size_t len, int prot, int flags, int64_t physStart, return SCE_KERNEL_ERROR_EINVAL; } GuestAllocations::Mutation mutation; + flags = PlaceHintInReservation(*addr, len, flags); if (FixedNoOverwriteConflict(mutation, *addr, len, flags)) return SCE_KERNEL_ERROR_ENOMEM; if (RemapFixedIntoRegistered(mutation, *addr, len, prot, flags, physStart)) { EraseReservations(*addr, len); @@ -798,6 +804,7 @@ int DoMapAnon(void** addr, size_t len, int prot, int flags, size_t alignment) { ValidateOutput(addr); if (len == 0 || (len & (PS5_PAGE_SIZE - 1)) != 0) return SCE_KERNEL_ERROR_EINVAL; GuestAllocations::Mutation mutation; + flags = PlaceHintInReservation(*addr, len, flags); if (FixedNoOverwriteConflict(mutation, *addr, len, flags)) return SCE_KERNEL_ERROR_ENOMEM; if (RemapFixedIntoRegistered(mutation, *addr, len, prot, flags)) { EraseReservations(*addr, len); diff --git a/core/libs/tests/GuestMemory.cpp b/core/libs/tests/GuestMemory.cpp index caed0eb13..8ce757509 100644 --- a/core/libs/tests/GuestMemory.cpp +++ b/core/libs/tests/GuestMemory.cpp @@ -514,6 +514,37 @@ static void CheckFixedVirtualReservation() { Require(sceKernelMunmap(pooled, page * 2) == 0); } +static void CheckHintInsideReservation() { + constexpr std::size_t page = 0x4000; + void* reserved = nullptr; + Require(sceKernelReserveVirtualRange(&reserved, page * 4, 0, 0) == 0); + auto* base = static_cast(reserved); + std::int64_t phys = 0; + Require(sceKernelAllocateDirectMemory(0, 0x7fffffffffll, page * 3, 0, 0, &phys) == 0); + void* direct = base + page; + Require(sceKernelMapDirectMemory(&direct, page, 3, 0, phys, 0) == 0); + Require(direct == base + page); + static_cast(direct)[0] = 21; + void* flexible = base + page * 2; + Require(sceKernelMapFlexibleMemory(&flexible, page, 3, 0) == 0); + Require(flexible == base + page * 2); + static_cast(flexible)[0] = 22; + VirtualQueryInfo info{}; + Require(sceKernelVirtualQuery(direct, 0, &info, sizeof(info)) == 0); + Require(info.is_direct && info.offset == static_cast(phys)); + Require(sceKernelVirtualQuery(flexible, 0, &info, sizeof(info)) == 0); + Require(info.is_flexible); + Require(sceKernelVirtualQuery(base + page * 3, 0, &info, sizeof(info)) == 0); + Require(!info.is_committed && info.protection == 0); + void* overlapping = base + page; + Require(sceKernelMapDirectMemory(&overlapping, page * 2, 3, 0, phys + page, 0) == 0); + Require(overlapping != base + page); + Require(static_cast(direct)[0] == 21 && static_cast(flexible)[0] == 22); + Require(sceKernelMunmap(overlapping, page * 2) == 0); + Require(sceKernelMunmap(reserved, page * 4) == 0); + Require(sceKernelReleaseDirectMemory(phys, page * 3) == 0); +} + static void CheckReservedRangeIsNotCommitted() { constexpr std::size_t page = 0x4000; void* reserved = nullptr; @@ -1528,6 +1559,7 @@ int main() { CheckGpuAccessFollowsProtection(); CheckFixedVirtualReservation(); CheckReservedRangeIsNotCommitted(); + CheckHintInsideReservation(); CheckNoOverwriteRefusesLiveMapping(); CheckMlock(); CheckSharedDirectMemoryLifecycle(); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..12dc5caac 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -146,6 +146,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceKernelGetCurrentCpu](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - guest affinities are not applied to host threads, so the host CPU index is mapped into the calling thread's affinity clipped to `sceKernelGetAvailableCpumask` (the available mask when they do not overlap): host CPU n reports the (n modulo the CPU count)-th CPU of that mask, so a thread pinned to one CPU reports that CPU, as a FreeBSD `cpuset` guarantees on the console. Which CPU the console reports for a thread allowed on several is not known; shadPS4 returns 0 - [sceKernelMapNamedFlexibleMemoryInternal](../../core/libs/prx/libkernel/DirectMemory/Export.cpp) (libkernel) - flag 0x8000 is accepted and ignored; its meaning is not established. KytyPS5 (`GUEST_MAP_UNKNOWN_8000`) and sharpemu accept it without reading it. Every known caller is the SDK 9 `libc.prx` heap (Astro Bot PPSA21564, PPSA22520), which passes it with read/write protection and uses the mapping as an older `libc.prx` uses `sceKernelMapNamedFlexibleMemory` with flags 0 - [sceKernelMtypeprotect](../../core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp) (libkernel) - sets the memory type of the direct mappings in the page-rounded range and of the direct memory behind them, as shadPS4 and prosper do (KytyPS5 ignores the type); the type is not validated, flexible pages in the range only get the protection, and a later map of the retyped direct memory reports the type it was allocated with. `sceKernelBatchMap2` operation 4 takes the same path with the entry type +- [Map hints inside a reservation](../../core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp) (libkernel) - `sceKernelMapDirectMemory` and the flexible and anonymous maps place a mapping without `MAP_FIXED` at its hint address when the whole range lies in a range still reserved by `sceKernelReserveVirtualRange`, as if `MAP_FIXED` were set. Taken from PPSA01474, which reserves 64 GiB and then maps its heaps inside it with hints and no `MAP_FIXED`, and uses the hint addresses afterwards; not measured on the console. Hints outside a reservation, or over a range that is partly mapped, stay hints - [Guest arena](../../core/libs/prx/libc/src/GuestArena.cpp) (libc) - the application map area is assumed to end at 0xFC_0000_0000, as `SCE_KERNEL_APP_MAP_AREA_END_ADDR` does on the PS4. On Windows, host allocations made in the arena before libc loads stay, and a fixed guest mapping over one throws - [sceLibcInternalBacktraceForGame](../../core/libs/prx/libc/src/HeapDiagnostics.cpp) (libSceLibcInternal, implemented in libc) - unknown signature - [sceLibcInternalHeapErrorReportForGame](../../core/libs/prx/libc/src/HeapDiagnostics.cpp) (libSceLibcInternal, implemented in libc) - unknown signature From 9cf85e94cbcc1a08de0f2e51bad75c84e44ad794 Mon Sep 17 00:00:00 2001 From: Chad Date: Sat, 10 Oct 2026 13:11:10 -0500 Subject: [PATCH 102/460] fix(recompiler): accept SDWA byte sources on v_cvt_f16_u16 and v_cvt_f16_i16 PPSA28416 stopped at v_cvt_f16_u16_sdwa dst_sel:WORD_0 dst_unused:UNUSED_PRESERVE src0_sel:BYTE_3. The integer source extraction already implements byte selects with src0_sext. --- .../tests/execution/SdwaDestination.cpp | 10 ++++++++-- .../recompiler/RdnaDecoder/src/RdnaVectorOpDecoder.cpp | 4 ++-- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/SdwaDestination.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/SdwaDestination.cpp index 7a27e485a..c1cf037a1 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/SdwaDestination.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/SdwaDestination.cpp @@ -22,7 +22,7 @@ constexpr std::uint32_t Garbage = 0xabcd1234u; alignas(256) std::array Input{}; alignas(256) std::array Output{}; -alignas(256) constexpr std::array SdwaCode{ +alignas(256) constexpr std::array SdwaCode{ 0x34020082, 0x34060084, 0xe0302000, 0x80000401, 0xe0302004, 0x80000501, 0xe0302008, 0x80000601, 0x7e1402ff, 0xabcd1234, 0x7e1602ff, 0xabcd1234, 0x7e1802ff, 0xabcd1234, 0x7e1a02ff, 0xabcd1234, 0x7e1c02ff, 0xabcd1234, 0x7e1e02ff, 0xabcd1234, 0x7e2002ff, 0xabcd1234, 0x7e2202ff, 0xabcd1234, @@ -30,9 +30,10 @@ alignas(256) constexpr std::array SdwaCode{ 0x7e1814f9, 0x00061505, 0x7e1a14f9, 0x00060604, 0x7e1c14f9, 0x00060605, 0xd746000d, 0x0435210e, 0x641e0cf9, 0x05040606, 0x7e2002f9, 0x00061004, 0x7e2202f9, 0x00061204, 0x7e2402f9, 0x00060104, 0x7e2602ff, 0xabcd1234, 0x7e266ef9, 0x00061404, + 0x7e2802ff, 0xabcd1234, 0x7e28a0f9, 0x00031404, 0x7e2a02ff, 0xabcd1234, 0x7e2aa2f9, 0x000b1404, 0xe0702000, 0x80010a03, 0xe0702004, 0x80010b03, 0xe0702008, 0x80010c03, 0xe070200c, 0x80010d03, 0xe0702010, 0x80010f03, 0xe0702014, 0x80011003, 0xe0702018, 0x80011103, - 0xe070201c, 0x80011203, 0xe0702020, 0x80011303, 0xbf810000, + 0xe070201c, 0x80011203, 0xe0702020, 0x80011303, 0xe0702024, 0x80011403, 0xe0702028, 0x80011503, 0xbf810000, }; float Quarter(std::uint32_t index) { @@ -109,6 +110,11 @@ void Check() { Require(out[7] == (low << 8u), where(7) + ", expected " + Hex(low << 8u) + ": v_mov_b32_sdwa dst_sel:BYTE_1 dst_unused:UNUSED_PAD must zero the other bytes"); const auto inverted = ~in[0] & 0xffffu; Require(out[8] == ((Garbage & 0xffff0000u) | inverted), where(8) + ", expected " + Hex((Garbage & 0xffff0000u) | inverted) + ": v_not_b32_sdwa dst_sel:WORD_0 dst_unused:UNUSED_PRESERVE must keep the high half"); + const auto top = in[0] >> 24u; + const auto unsignedHalf = HalfBits(static_cast(top)); + Require(out[9] == ((Garbage & 0xffff0000u) | unsignedHalf), where(9) + ", expected " + Hex((Garbage & 0xffff0000u) | unsignedHalf) + ": v_cvt_f16_u16_sdwa dst_sel:WORD_0 dst_unused:UNUSED_PRESERVE src0_sel:BYTE_3 must convert the zero-extended top byte"); + const auto signedHalf = HalfBits(static_cast(static_cast(top))); + Require(out[10] == ((Garbage & 0xffff0000u) | signedHalf), where(10) + ", expected " + Hex((Garbage & 0xffff0000u) | signedHalf) + ": v_cvt_f16_i16_sdwa dst_sel:WORD_0 dst_unused:UNUSED_PRESERVE src0_sel:BYTE_3 src0_sext must convert the sign-extended top byte"); } } diff --git a/core/shader/recompiler/RdnaDecoder/src/RdnaVectorOpDecoder.cpp b/core/shader/recompiler/RdnaDecoder/src/RdnaVectorOpDecoder.cpp index 521e629c2..0e6078d53 100644 --- a/core/shader/recompiler/RdnaDecoder/src/RdnaVectorOpDecoder.cpp +++ b/core/shader/recompiler/RdnaDecoder/src/RdnaVectorOpDecoder.cpp @@ -1126,9 +1126,9 @@ constexpr Vop1SdwaRule vop1SdwaRules[] = { {RdnaOpcode::VBfrevB32, sdwaSelAll(), 0, 0, false}, {RdnaOpcode::VCvtF32F16, sdwaSelWords() | sdwaSelFull(), 0, 0, true}, {RdnaOpcode::VCvtF16F32, sdwaSelAll(), sdwaSelWords(), sdwaSelAll(), true}, - {RdnaOpcode::VCvtF16U16, sdwaSelWords() | sdwaSelFull(), sdwaSelWords(), sdwaSelWords() | sdwaSelFull(), false}, + {RdnaOpcode::VCvtF16U16, sdwaSelAll(), sdwaSelWords(), sdwaSelAll(), false}, {RdnaOpcode::VCvtU16F16, sdwaSelWords() | sdwaSelFull(), sdwaSelWords(), sdwaSelWords() | sdwaSelFull(), true}, - {RdnaOpcode::VCvtF16I16, sdwaSelWords() | sdwaSelFull(), sdwaSelWords(), sdwaSelWords() | sdwaSelFull(), false}, + {RdnaOpcode::VCvtF16I16, sdwaSelAll(), sdwaSelWords(), sdwaSelAll(), false}, {RdnaOpcode::VCvtI16F16, sdwaSelWords() | sdwaSelFull(), sdwaSelWords(), sdwaSelWords() | sdwaSelFull(), true}, {RdnaOpcode::VRcpF16, sdwaSelWords() | sdwaSelFull(), sdwaSelWords(), sdwaSelWords() | sdwaSelFull(), true}, {RdnaOpcode::VSqrtF16, sdwaSelWords() | sdwaSelFull(), sdwaSelWords(), sdwaSelWords() | sdwaSelFull(), true}, From 08728f95b98a5473735f01e2d84c78da3da9a82f Mon Sep 17 00:00:00 2001 From: ps54all Date: Sat, 10 Oct 2026 12:41:40 +0200 Subject: [PATCH 103/460] feat(libkernel): implement pthread_mutexattr_setprioceiling and getprioceiling The mutex attribute now keeps its protocol and priority ceiling the way FreeBSD's libthr does: the ceiling can be set and read only under PTHREAD_PRIO_PROTECT (EINVAL otherwise), and setting a protocol resets it to the highest SCHED_RR priority. Mutexes still do not apply priority protection; that is listed as a silent stub. Co-Authored-By: Claude Opus 5.5 (cherry picked from commit d10c024f8fdbcdfced0187550f272333264f07df) --- core/libs/prx/libkernel/CMakeLists.txt | 10 +++ .../prx/libkernel/Pthread/Posix/Mutex.cpp | 20 ++++++ .../prx/libkernel/Pthread/include/Pthread.hpp | 2 + core/libs/tests/GuestPthreadMutexattr.cpp | 62 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 5 files changed, 95 insertions(+) create mode 100644 core/libs/tests/GuestPthreadMutexattr.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index 6938ffc9a..322841b5d 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -293,6 +293,16 @@ if(BUILD_TESTING) set_tests_properties(guest_pthread_attr PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_pthread_mutexattr_tests ${LIBS_INCLUDE_DIR}/tests/GuestPthreadMutexattr.cpp) + target_include_directories(guest_pthread_mutexattr_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_pthread_mutexattr_tests PRIVATE libkernel libc) + configure_windows_unwind(guest_pthread_mutexattr_tests) + add_test(NAME guest_pthread_mutexattr COMMAND guest_pthread_mutexattr_tests) + set_tests_properties(guest_pthread_mutexattr PROPERTIES TIMEOUT 20) + if(WIN32) + set_tests_properties(guest_pthread_mutexattr PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_pthread_start_tests ${LIBS_INCLUDE_DIR}/tests/PthreadStart.cpp) target_include_directories(guest_pthread_start_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_pthread_start_tests PRIVATE libkernel libc) diff --git a/core/libs/prx/libkernel/Pthread/Posix/Mutex.cpp b/core/libs/prx/libkernel/Pthread/Posix/Mutex.cpp index b34390ef7..a51401a86 100644 --- a/core/libs/prx/libkernel/Pthread/Posix/Mutex.cpp +++ b/core/libs/prx/libkernel/Pthread/Posix/Mutex.cpp @@ -15,6 +15,7 @@ namespace { constexpr int POSIX_EINVAL = 22; constexpr int POSIX_PRIO_PROTECT = 2; +constexpr int POSIX_SCHED_RR = 3; constexpr std::uintptr_t POSIX_ADAPTIVE_MUTEX_INITIALIZER = 1; bool _isStaticInitializer(PthreadMutex mutex) { @@ -37,6 +38,8 @@ void _initializeStatic(PthreadMutex* mutex, const char* funcName) { extern "C" { +int APS5_VABI sched_get_priority_max_nid_postfix(int policy); + int APS5_VABI pthread_mutex_destroy_nid_postfix(PthreadMutex* mutex) { if (!mutex) throw std::runtime_error("pthread_mutex_destroy: null mutex"); if (_isStaticInitializer(*mutex)) { @@ -91,6 +94,23 @@ int APS5_VABI pthread_mutexattr_init_nid_postfix(PthreadMutexattr* attr) { int APS5_VABI pthread_mutexattr_setprotocol_nid_postfix(PthreadMutexattr* attr, int protocol) { if (!attr || !*attr) throw std::runtime_error("pthread_mutexattr_setprotocol: null attr"); if (protocol < 0 || protocol > POSIX_PRIO_PROTECT) return POSIX_EINVAL; + (*attr)->protocol = protocol; + (*attr)->ceiling = sched_get_priority_max_nid_postfix(POSIX_SCHED_RR); + return 0; +} + +int APS5_VABI pthread_mutexattr_setprioceiling_nid_postfix(PthreadMutexattr* attr, int prioceiling) { + if (!attr || !*attr) throw std::runtime_error("pthread_mutexattr_setprioceiling: null attr"); + if ((*attr)->protocol != POSIX_PRIO_PROTECT) return POSIX_EINVAL; + (*attr)->ceiling = prioceiling; + return 0; +} + +int APS5_VABI pthread_mutexattr_getprioceiling_nid_postfix(const PthreadMutexattr* attr, int* prioceiling) { + if (!attr || !*attr) throw std::runtime_error("pthread_mutexattr_getprioceiling: null attr"); + if (!prioceiling) throw std::runtime_error("pthread_mutexattr_getprioceiling: null prioceiling"); + if ((*attr)->protocol != POSIX_PRIO_PROTECT) return POSIX_EINVAL; + *prioceiling = (*attr)->ceiling; return 0; } diff --git a/core/libs/prx/libkernel/Pthread/include/Pthread.hpp b/core/libs/prx/libkernel/Pthread/include/Pthread.hpp index 11e6eea38..6e69de708 100644 --- a/core/libs/prx/libkernel/Pthread/include/Pthread.hpp +++ b/core/libs/prx/libkernel/Pthread/include/Pthread.hpp @@ -28,6 +28,8 @@ enum class MutexType : std::uint32_t { struct PthreadMutexattrPrivate { MutexType type; + int protocol = 0; + int ceiling = 0; }; struct PthreadMutexPrivate { diff --git a/core/libs/tests/GuestPthreadMutexattr.cpp b/core/libs/tests/GuestPthreadMutexattr.cpp new file mode 100644 index 000000000..1725c4a38 --- /dev/null +++ b/core/libs/tests/GuestPthreadMutexattr.cpp @@ -0,0 +1,62 @@ +#include "SceTypes.hpp" +#include +#include + +extern "C" { +int APS5_VABI pthread_mutexattr_init_nid_postfix(PthreadMutexattr* attr); +int APS5_VABI pthread_mutexattr_destroy_nid_postfix(PthreadMutexattr* attr); +int APS5_VABI pthread_mutexattr_setprotocol_nid_postfix(PthreadMutexattr* attr, int protocol); +int APS5_VABI pthread_mutexattr_setprioceiling_nid_postfix(PthreadMutexattr* attr, int prioceiling); +int APS5_VABI pthread_mutexattr_getprioceiling_nid_postfix(const PthreadMutexattr* attr, int* prioceiling); +int APS5_VABI pthread_mutex_init_nid_postfix(PthreadMutex* mutex, const PthreadMutexattr* attr); +int APS5_VABI pthread_mutex_lock_nid_postfix(PthreadMutex* mutex); +int APS5_VABI pthread_mutex_unlock_nid_postfix(PthreadMutex* mutex); +int APS5_VABI pthread_mutex_destroy_nid_postfix(PthreadMutex* mutex); +int APS5_VABI sched_get_priority_max_nid_postfix(int policy); +} + +static constexpr int GUEST_EINVAL = 22; +static constexpr int PRIO_NONE = 0; +static constexpr int PRIO_INHERIT = 1; +static constexpr int PRIO_PROTECT = 2; +static constexpr int SCHED_RR = 3; + +static void Require(bool value) { if (!value) std::abort(); } + +template +static bool Throws(Call call) { + try { + call(); + } catch (const std::exception&) { + return true; + } + return false; +} + +int main() { + PthreadMutexattr attr = nullptr; + Require(pthread_mutexattr_init_nid_postfix(&attr) == 0); + int ceiling = -1; + Require(pthread_mutexattr_getprioceiling_nid_postfix(&attr, &ceiling) == GUEST_EINVAL && ceiling == -1); + Require(pthread_mutexattr_setprioceiling_nid_postfix(&attr, 300) == GUEST_EINVAL); + Require(pthread_mutexattr_setprotocol_nid_postfix(&attr, PRIO_INHERIT) == 0); + Require(pthread_mutexattr_setprioceiling_nid_postfix(&attr, 300) == GUEST_EINVAL); + Require(pthread_mutexattr_setprotocol_nid_postfix(&attr, PRIO_PROTECT) == 0); + Require(pthread_mutexattr_getprioceiling_nid_postfix(&attr, &ceiling) == 0 && ceiling == sched_get_priority_max_nid_postfix(SCHED_RR)); + Require(pthread_mutexattr_setprioceiling_nid_postfix(&attr, 300) == 0); + Require(pthread_mutexattr_getprioceiling_nid_postfix(&attr, &ceiling) == 0 && ceiling == 300); + PthreadMutex mutex = nullptr; + Require(pthread_mutex_init_nid_postfix(&mutex, &attr) == 0); + Require(pthread_mutex_lock_nid_postfix(&mutex) == 0); + Require(pthread_mutex_unlock_nid_postfix(&mutex) == 0); + Require(pthread_mutex_destroy_nid_postfix(&mutex) == 0); + Require(pthread_mutexattr_setprotocol_nid_postfix(&attr, PRIO_PROTECT) == 0); + Require(pthread_mutexattr_getprioceiling_nid_postfix(&attr, &ceiling) == 0 && ceiling == sched_get_priority_max_nid_postfix(SCHED_RR)); + Require(pthread_mutexattr_setprotocol_nid_postfix(&attr, PRIO_NONE) == 0); + Require(pthread_mutexattr_getprioceiling_nid_postfix(&attr, &ceiling) == GUEST_EINVAL); + Require(Throws([&] { pthread_mutexattr_getprioceiling_nid_postfix(&attr, nullptr); })); + Require(pthread_mutexattr_destroy_nid_postfix(&attr) == 0); + Require(Throws([&] { pthread_mutexattr_setprioceiling_nid_postfix(&attr, 300); })); + Require(Throws([&] { pthread_mutexattr_getprioceiling_nid_postfix(&attr, &ceiling); })); + return 0; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..ad9639220 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -18,6 +18,7 @@ Throughout the project, every function at every stage either **does exactly what - [libSceErrorDialog](../../core/libs/prx/libSceErrorDialog/Export.cpp) - the error dialog runs the state machine but shows nothing; the error code passed to `sceErrorDialogOpen` is only logged - [libSceLoginDialog](../../core/libs/prx/libSceLoginDialog/Export.cpp) - the login dialog runs the common-dialog state machine but shows nothing and reports the user as having cancelled - [libSceHmd](../../core/libs/prx/libSceHmd/Export.cpp) implements only the disconnected-headset path: initialization succeeds, device queries report `NotDetected`, and opening a device returns `DeviceDisconnected`. Headset support, tracking, rendering and additional HMD exports are not implemented; SDK-level ABI compatibility and in-game behaviour remain unverified. +- [pthread_mutexattr_setprotocol, pthread_mutexattr_setprioceiling, pthread_mutexattr_getprioceiling](../../core/libs/prx/libkernel/Pthread/Posix/Mutex.cpp) (libkernel) - the protocol and the priority ceiling are stored and reported as in FreeBSD's [libthr](https://github.com/freebsd/freebsd-src/blob/main/lib/libthr/thread/thr_mutexattr.c) (the ceiling is set or read only under `PTHREAD_PRIO_PROTECT`, and setting a protocol resets it to the highest `SCHED_RR` priority), but mutexes do not apply them: a priority-inherit or priority-protect mutex locks like a normal one and does not change its owner's priority - [libSceHmd2](../../core/libs/prx/libSceHmd2/Export.cpp) has no PS VR2 runtime: `sceHmd2Initialize` fails, and every other export it has (and every [libSceVrTracker2](../../core/libs/prx/libSceVrTracker2/Export.cpp) export) throws. PPSA01317 skips its whole VR set-up when initialization fails - [libSceMbus](../../core/libs/prx/libSceMbus/Export.cpp) implements only the no-device path: initialization and termination succeed, no device events are reported and device queries return no entries - [libSceAudioPropagation](../../core/libs/prx/libSceAudioPropagation/Export.cpp) implements only the no-acoustic-propagation path: the system keeps a 16-byte header in the title's CPU memory (no GPU memory), rooms, portals, sources and materials are tracked handles, attributes are validated and have no effect, no rays are requested and sources report no audio paths. Ray results, path calculation and audio paths are accepted only when empty and throw otherwise; `sceAudioPropagationSourceRender` writes silence (zeroes) to each source's output buffer for the given size, without a dry signal. Objects still alive when their system is destroyed become unusable, and their later destroy or unregister is a no-op: PPSA21567 releases its reference-counted room, portal and source owners just before `sceAudioPropagationSystemDestroy`, so the order for every object is not proven. Occlusion, reflections and reverb from level geometry are absent; in-game behaviour remains unverified. From 5fcac960e8bbe7b7c24e57b3b0a1649adfee116b Mon Sep 17 00:00:00 2001 From: jweidenh <38249793+jweidenh@users.noreply.github.com> Date: Fri, 9 Oct 2026 06:49:04 -0700 Subject: [PATCH 104/460] fix(tools): honor dependency declarations without a colon --- CMakeLists.txt | 2 + tools/pr_overlap.py | 3 +- tools/tests/test_pr_overlap_dependencies.py | 78 +++++++++++++++++++++ 3 files changed, 82 insertions(+), 1 deletion(-) create mode 100644 tools/tests/test_pr_overlap_dependencies.py diff --git a/CMakeLists.txt b/CMakeLists.txt index f71c2a4de..ff5546acc 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -39,6 +39,8 @@ if(BUILD_TESTING) if(Python3_Interpreter_FOUND) add_test(NAME pr_overlap_encoding COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/tools/tests/test_pr_overlap_encoding.py) + add_test(NAME pr_overlap_dependencies COMMAND ${Python3_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/tools/tests/test_pr_overlap_dependencies.py) add_test(NAME pr_overlap_tools COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/tools/tests/test_pr_overlap.py) add_test(NAME progress_report COMMAND ${Python3_EXECUTABLE} diff --git a/tools/pr_overlap.py b/tools/pr_overlap.py index a2b60a362..40a5efb9d 100644 --- a/tools/pr_overlap.py +++ b/tools/pr_overlap.py @@ -10,6 +10,7 @@ from urllib.parse import urlencode PARENT = 'if .parent then "\\(.parent.owner.login)/\\(.parent.name)" else .nameWithOwner end' EXPORT = re.compile(r"^\+.*\bAPS5_VABI\s+(\w+)\s*\(") DEPENDS = re.compile(r'Depends on:([^\r\n]*(?:\r?\n[ \t]*[-*][ \t]*#\d+[^\r\n]*)*)') +PLAIN_DEPENDS = re.compile(r'Depends on[ \t]+(#\d+\b(?:(?:[ \t]*,[ \t]*(?:and[ \t]+)?|[ \t]+and[ \t]+)#\d+\b)*)') HUNK = re.compile(r"^@@ -(\d+)(?:,(\d+))?", re.M) MARKER = "" @@ -31,7 +32,7 @@ def open_prs(base): out = gh("api", "--paginate", "--slurp", f"repos/{REPO}/pulls?{query}") prs = {} for pr in chain.from_iterable(json.loads(out)): - depends = DEPENDS.search(pr["body"] or "") + depends = DEPENDS.search(pr["body"] or "") or PLAIN_DEPENDS.search(pr["body"] or "") prs[pr["number"]] = { "ref": f"refs/pr/{pr['number']}", "sha": pr["head"]["sha"], diff --git a/tools/tests/test_pr_overlap_dependencies.py b/tools/tests/test_pr_overlap_dependencies.py new file mode 100644 index 000000000..791221261 --- /dev/null +++ b/tools/tests/test_pr_overlap_dependencies.py @@ -0,0 +1,78 @@ +import json +from pathlib import Path +import sys +import unittest +from unittest.mock import patch + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +import pr_overlap + + +def pull_requests(body): + response = [{"number": number, "user": {"login": "contributor"}, "head": {"sha": str(number) * 40}, + "body": body if number == 1 else "Depends on: none"} for number in (1, 2)] + with patch.object(pr_overlap, "REPO", "owner/repo", create=True), \ + patch.object(pr_overlap, "gh", return_value=json.dumps([response])): + return pr_overlap.open_prs("main") + + +class DependencyTests(unittest.TestCase): + def test_documented_form_without_colon(self): + for body in ("Depends on #2.", "Depends on #2, #3.", "Depends on\t#2 and #3.", + "Depends on #2, and #3.", "**Depends on #2.**", "- [x] Depends on #2"): + with self.subTest(body=body): + expected = {2, 3} if "#3" in body else {2} + self.assertEqual(pull_requests(body)[1]["depends"], expected) + + def test_colon_forms_are_preserved(self): + for body in ("Depends on:#2, #3", "- [x] Depends on: #2, #3", "Depends on:\n- #2\n- #3", + "- [x] Depends on:\r\n * #2\r\n * #3"): + with self.subTest(body=body): + self.assertEqual(pull_requests(body)[1]["depends"], {2, 3}) + + def test_later_references_are_not_dependencies(self): + for newline in ("\n", "\r\n"): + for declaration in ("Depends on #2.", "Depends on: #2"): + body = newline.join((declaration, "- #3", "", "Related work: #4.", "- #5")) + with self.subTest(body=body): + expected = {2, 3} if ":" in declaration else {2} + self.assertEqual(pull_requests(body)[1]["depends"], expected) + + def test_no_dependencies(self): + for body in (None, "", "Depends on: none", "Depends on: ", + "Related to #2.", "Depends on compiler support; see #2.", + "Depends on\n#2", "Depends on #2suffix"): + with self.subTest(body=body): + self.assertEqual(pull_requests(body)[1]["depends"], set()) + + def test_plain_declaration_stops_before_unrelated_prose(self): + for body in ("Depends on #2. Related #3 covers something else.", + "Depends on #2; related #3 covers something else.", + "Depends on #2 and compiler support from #3.", + "Depends on #2, related work is in #3.", + "**Depends on #2**. Refs #3."): + with self.subTest(body=body): + self.assertEqual(pull_requests(body)[1]["depends"], {2}) + + def test_checklist_declaration_keeps_precedence(self): + for declaration, expected in (("#3", {3}), ("none", set())): + body = "Previously: Depends on #2.\n\n- [x] Depends on: " + declaration + with self.subTest(body=body): + self.assertEqual(pull_requests(body)[1]["depends"], expected) + + def test_documented_dependency_suppresses_overlap(self): + prs = pull_requests("Depends on #2.") + for pr in prs.values(): + pr["added"] = {"shared.cpp"} + self.assertIsNone(pr_overlap.overlaps(prs, 1, 2)) + self.assertIsNone(pr_overlap.overlaps(prs, 2, 1)) + + def test_unrelated_reference_keeps_overlap(self): + prs = pull_requests("Related to #2.") + for pr in prs.values(): + pr["added"] = {"shared.cpp"} + self.assertEqual(pr_overlap.overlaps(prs, 1, 2), {"added": {"shared.cpp"}}) + + +if __name__ == "__main__": + unittest.main() From f45d8933f212540eb447499bcfd4b31a6da38dbc Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Sat, 10 Oct 2026 17:04:23 +0900 Subject: [PATCH 105/460] fix(libkernel): unlink read-only files on Windows FreeBSD's unlink ignores the file's own permission bits, so it succeeds on a file the guest made read-only with chmod(0400). On Windows the guest chmod sets the read-only attribute and _wunlink then failed with EACCES, which sceKernelUnlink and unlink reported. When the path is a regular file without write permission, make it writable before the single _wunlink and restore the read-only attribute if the unlink fails, as remove() does since #2342. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libkernel/File/src/Open.cpp | 10 +++++++++- core/libs/tests/GuestFilesystem.cpp | 10 ++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libkernel/File/src/Open.cpp b/core/libs/prx/libkernel/File/src/Open.cpp index 329c2a55d..537fafb5f 100644 --- a/core/libs/prx/libkernel/File/src/Open.cpp +++ b/core/libs/prx/libkernel/File/src/Open.cpp @@ -46,7 +46,15 @@ static int NativeClose(int fd) { return result; } static int NativeUnlink(const std::filesystem::path& p) { - return ::_wunlink(p.wstring().c_str()); + const auto path = p.wstring(); + struct _stat64 status{}; + const bool unlocked = ::_wstat64(path.c_str(), &status) == 0 && (status.st_mode & _S_IFMT) == _S_IFREG && + !(status.st_mode & _S_IWRITE) && ::_wchmod(path.c_str(), _S_IREAD | _S_IWRITE) == 0; + if (::_wunlink(path.c_str()) == 0) return 0; + const int error = errno; + if (unlocked) ::_wchmod(path.c_str(), _S_IREAD); + errno = error; + return -1; } static int MapFlags(int sceFlags) { int f = 0; diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index bd0f598b7..397867fb3 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -298,6 +298,16 @@ int main() { Require(sceKernelRmdir(rootName.c_str()) == static_cast(0x80020042u)); Require(rmdir_nid_postfix(presentName.c_str()) == -1 && *__error_nid_postfix() == 20); Require(rmdir_nid_postfix(missingName.c_str()) == -1 && *__error_nid_postfix() == 2); + const auto lockedFile = [&](const char* name) { + const auto locked = root / name; + { std::ofstream stream(locked); stream << "removable"; } + Require(sceKernelChmod_nid_postfix(locked.string().c_str(), 0400) == 0); + return locked; + }; + const auto lockedPosix = lockedFile("locked-posix"); + Require(unlink_nid_postfix(lockedPosix.string().c_str()) == 0 && !std::filesystem::exists(lockedPosix)); + const auto lockedKernel = lockedFile("locked-kernel"); + Require(sceKernelUnlink(lockedKernel.string().c_str()) == 0 && !std::filesystem::exists(lockedKernel)); Require(rmdir_nid_postfix("") == -1 && *__error_nid_postfix() == 2); Require(rmdir_nid_postfix(nullptr) == -1 && *__error_nid_postfix() == 14); Require(unlink_nid_postfix(missingName.c_str()) == -1 && *__error_nid_postfix() == 2); From 5abc86f3d55417bd69efce87dca06186ee7049a5 Mon Sep 17 00:00:00 2001 From: JAB-dev <68183649+JAB-dev@users.noreply.github.com> Date: Sat, 10 Oct 2026 20:11:36 +0200 Subject: [PATCH 106/460] fix(agc): look up cached draws at the fixed pixel push slot With graphics pipeline libraries a draw places the pixel shader's push constants at the fixed fragment slot (Graphics::StagePushOffset) and the draw cache records that offset, but lookupDraw still expected the stacked offset after the vertex shader's push bytes. Every pixel stage then missed as a layout mismatch, so no cached draw or draw recipe was reused and each draw was captured and materialized again. Co-Authored-By: Claude Opus 5.5 --- .../prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawLookup.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawLookup.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawLookup.cpp index 0df4edd20..090b9b10d 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawLookup.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawLookup.cpp @@ -89,6 +89,7 @@ void Driver::lookupDraw(const Submission& submission, const std::shared_ptrGraphicsPipelineLibraries()); const auto& variants = entry->stages[i]; auto outcome = EntryOutcome::Differing; bool anyLayout = false; From c3b8aa3264c74e2ada9a254359daddb2f80ee3d0 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 14:19:14 -0400 Subject: [PATCH 107/460] fix(shader): flush f16 min3/max3/med3 denormal inputs The ternary forms share minMaxF16 with v_min_f16 and v_max_f16 and follow the same rule on gfx1036: a subnormal half input becomes a signed zero when bit 0 of the f16 denormal field is clear, at IEEE=0 and IEEE=1. Co-Authored-By: Claude Opus 5.5 (1M context) --- .../tests/execution/F16MinMaxNan.cpp | 204 +++++++++++++++--- .../Translation/src/FloatInstructions.cpp | 2 +- docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 174 insertions(+), 34 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/F16MinMaxNan.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/F16MinMaxNan.cpp index d252078c9..9afb2e1a3 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/F16MinMaxNan.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/F16MinMaxNan.cpp @@ -435,38 +435,38 @@ constexpr std::uint32_t ExpectedIeee[128][7] = { {0x00008001u, 0x00000001u, 0x00007e01u, 0x00007e01u, 0x00007e01u, 0x00008001u, 0x00000001u} }; constexpr std::uint32_t DenormalRows[32][4] = { - {0x12343c01u, 0x567803ffu, 0x00000000u, 0x00000000u}, - {0x1234bc01u, 0x567803ffu, 0x00000000u, 0x00000000u}, - {0x12340001u, 0x56788001u, 0x00000000u, 0x00000000u}, - {0x123403ffu, 0x567883ffu, 0x00000000u, 0x00000000u}, - {0x12340001u, 0x56780000u, 0x00000000u, 0x00000000u}, - {0x12348001u, 0x56788000u, 0x00000000u, 0x00000000u}, - {0x123403ffu, 0x56780400u, 0x00000000u, 0x00000000u}, - {0x123483ffu, 0x56788400u, 0x00000000u, 0x00000000u}, - {0x12340001u, 0x56787e00u, 0x00000000u, 0x00000000u}, - {0x12348001u, 0x5678fe55u, 0x00000000u, 0x00000000u}, - {0x123403ffu, 0x56787c01u, 0x00000000u, 0x00000000u}, - {0x123483ffu, 0x5678fd23u, 0x00000000u, 0x00000000u}, - {0x12340001u, 0x56787c00u, 0x00000000u, 0x00000000u}, - {0x12348001u, 0x5678fc00u, 0x00000000u, 0x00000000u}, - {0x123403ffu, 0x56780001u, 0x00000000u, 0x00000000u}, - {0x123483ffu, 0x56788001u, 0x00000000u, 0x00000000u}, - {0x123403ffu, 0x56783c01u, 0x00000000u, 0x00000000u}, - {0x123403ffu, 0x5678bc01u, 0x00000000u, 0x00000000u}, - {0x12348001u, 0x56780001u, 0x00000000u, 0x00000000u}, - {0x123483ffu, 0x567803ffu, 0x00000000u, 0x00000000u}, - {0x12340000u, 0x56780001u, 0x00000000u, 0x00000000u}, - {0x12348000u, 0x56788001u, 0x00000000u, 0x00000000u}, - {0x12340400u, 0x567803ffu, 0x00000000u, 0x00000000u}, - {0x12348400u, 0x567883ffu, 0x00000000u, 0x00000000u}, - {0x12347e00u, 0x56780001u, 0x00000000u, 0x00000000u}, - {0x1234fe55u, 0x56788001u, 0x00000000u, 0x00000000u}, - {0x12347c01u, 0x567803ffu, 0x00000000u, 0x00000000u}, - {0x1234fd23u, 0x567883ffu, 0x00000000u, 0x00000000u}, - {0x12347c00u, 0x56780001u, 0x00000000u, 0x00000000u}, - {0x1234fc00u, 0x56788001u, 0x00000000u, 0x00000000u}, - {0x12340001u, 0x567803ffu, 0x00000000u, 0x00000000u}, - {0x12348001u, 0x567883ffu, 0x00000000u, 0x00000000u}, + {0x12343c01u, 0x567803ffu, 0x9abc0400u, 0x00000000u}, + {0x1234bc01u, 0x567803ffu, 0x9abcbc00u, 0x00000000u}, + {0x12340001u, 0x56788001u, 0x9abc03ffu, 0x00000000u}, + {0x123403ffu, 0x567883ffu, 0x9abc83ffu, 0x00000000u}, + {0x12340001u, 0x56780000u, 0x9abc8001u, 0x00000000u}, + {0x12348001u, 0x56788000u, 0x9abc0001u, 0x00000000u}, + {0x123403ffu, 0x56780400u, 0x9abc7e55u, 0x00000000u}, + {0x123483ffu, 0x56788400u, 0x9abcfd23u, 0x00000000u}, + {0x12340001u, 0x56787e00u, 0x9abc0400u, 0x00000000u}, + {0x12348001u, 0x5678fe55u, 0x9abc8400u, 0x00000000u}, + {0x123403ffu, 0x56787c01u, 0x9abc8001u, 0x00000000u}, + {0x123483ffu, 0x5678fd23u, 0x9abc0001u, 0x00000000u}, + {0x12340001u, 0x56787c00u, 0x9abc7c01u, 0x00000000u}, + {0x12348001u, 0x5678fc00u, 0x9abcfe55u, 0x00000000u}, + {0x123403ffu, 0x56780001u, 0x9abc0200u, 0x00000000u}, + {0x123483ffu, 0x56788001u, 0x9abc8200u, 0x00000000u}, + {0x123403ffu, 0x56783c01u, 0x9abc03ffu, 0x00000000u}, + {0x123403ffu, 0x5678bc01u, 0x9abc83ffu, 0x00000000u}, + {0x12348001u, 0x56780001u, 0x9abc7c01u, 0x00000000u}, + {0x123483ffu, 0x567803ffu, 0x9abc7e55u, 0x00000000u}, + {0x12340000u, 0x56780001u, 0x9abc0400u, 0x00000000u}, + {0x12348000u, 0x56788001u, 0x9abc8400u, 0x00000000u}, + {0x12340400u, 0x567803ffu, 0x9abc0001u, 0x00000000u}, + {0x12348400u, 0x567883ffu, 0x9abc8001u, 0x00000000u}, + {0x12347e00u, 0x56780001u, 0x9abc7e00u, 0x00000000u}, + {0x1234fe55u, 0x56788001u, 0x9abc7c01u, 0x00000000u}, + {0x12347c01u, 0x567803ffu, 0x9abc0001u, 0x00000000u}, + {0x1234fd23u, 0x567883ffu, 0x9abc8001u, 0x00000000u}, + {0x12347c00u, 0x56780001u, 0x9abc03ffu, 0x00000000u}, + {0x1234fc00u, 0x56788001u, 0x9abc83ffu, 0x00000000u}, + {0x12340001u, 0x567803ffu, 0x9abc7c00u, 0x00000000u}, + {0x12348001u, 0x567883ffu, 0x9abcfc00u, 0x00000000u}, }; constexpr std::uint32_t DenormalFlushed[32][4] = { {0x00000000u, 0x00003c01u, 0x0000bc01u, 0x00003c01u}, @@ -604,6 +604,142 @@ constexpr std::uint32_t DenormalKeptIeee[32][4] = { {0x00000001u, 0x000003ffu, 0x00008001u, 0x00000001u}, {0x000083ffu, 0x00008001u, 0x00000001u, 0x000003ffu}, }; +constexpr std::uint32_t TernaryDenormalFlushed[32][3] = { + {0x00000000u, 0x00003c01u, 0x00000400u}, + {0x0000bc01u, 0x00000000u, 0x0000bc00u}, + {0x00008000u, 0x00000000u, 0x00000000u}, + {0x00008000u, 0x00000000u, 0x00008000u}, + {0x00008000u, 0x00000000u, 0x00000000u}, + {0x00008000u, 0x00000000u, 0x00008000u}, + {0x00000000u, 0x00000400u, 0x00000000u}, + {0x00008400u, 0x00008000u, 0x00008400u}, + {0x00000000u, 0x00000400u, 0x00000000u}, + {0x00008400u, 0x00008000u, 0x00008400u}, + {0x00008000u, 0x00000000u, 0x00008000u}, + {0x00008000u, 0x00000000u, 0x00008000u}, + {0x00000000u, 0x00007c00u, 0x00000000u}, + {0x0000fc00u, 0x00008000u, 0x0000fc00u}, + {0x00000000u, 0x00000000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00008000u}, + {0x00000000u, 0x00003c01u, 0x00000000u}, + {0x0000bc01u, 0x00000000u, 0x00008000u}, + {0x00008000u, 0x00000000u, 0x00008000u}, + {0x00008000u, 0x00000000u, 0x00008000u}, + {0x00000000u, 0x00000400u, 0x00000000u}, + {0x00008400u, 0x00008000u, 0x00008000u}, + {0x00000000u, 0x00000400u, 0x00000000u}, + {0x00008400u, 0x00008000u, 0x00008000u}, + {0x00000000u, 0x00000000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00008000u}, + {0x00000000u, 0x00000000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00008000u}, + {0x00000000u, 0x00007c00u, 0x00000000u}, + {0x0000fc00u, 0x00008000u, 0x00008000u}, + {0x00000000u, 0x00007c00u, 0x00000000u}, + {0x0000fc00u, 0x00008000u, 0x00008000u}, +}; +constexpr std::uint32_t TernaryDenormalKept[32][3] = { + {0x000003ffu, 0x00003c01u, 0x00000400u}, + {0x0000bc01u, 0x000003ffu, 0x0000bc00u}, + {0x00008001u, 0x000003ffu, 0x00000001u}, + {0x000083ffu, 0x000003ffu, 0x000083ffu}, + {0x00008001u, 0x00000001u, 0x00000000u}, + {0x00008001u, 0x00000001u, 0x00008000u}, + {0x000003ffu, 0x00000400u, 0x000003ffu}, + {0x00008400u, 0x000083ffu, 0x00008400u}, + {0x00000001u, 0x00000400u, 0x00000001u}, + {0x00008400u, 0x00008001u, 0x00008400u}, + {0x00008001u, 0x000003ffu, 0x00008001u}, + {0x000083ffu, 0x00000001u, 0x000083ffu}, + {0x00000001u, 0x00007c00u, 0x00000001u}, + {0x0000fc00u, 0x00008001u, 0x0000fc00u}, + {0x00000001u, 0x000003ffu, 0x00000200u}, + {0x000083ffu, 0x00008001u, 0x00008200u}, + {0x000003ffu, 0x00003c01u, 0x000003ffu}, + {0x0000bc01u, 0x000003ffu, 0x000083ffu}, + {0x00008001u, 0x00000001u, 0x00008001u}, + {0x000083ffu, 0x000003ffu, 0x000083ffu}, + {0x00000000u, 0x00000400u, 0x00000001u}, + {0x00008400u, 0x00008000u, 0x00008001u}, + {0x00000001u, 0x00000400u, 0x000003ffu}, + {0x00008400u, 0x00008001u, 0x000083ffu}, + {0x00000001u, 0x00000001u, 0x00000001u}, + {0x00008001u, 0x00008001u, 0x00008001u}, + {0x00000001u, 0x000003ffu, 0x00000001u}, + {0x000083ffu, 0x00008001u, 0x000083ffu}, + {0x00000001u, 0x00007c00u, 0x000003ffu}, + {0x0000fc00u, 0x00008001u, 0x000083ffu}, + {0x00000001u, 0x00007c00u, 0x000003ffu}, + {0x0000fc00u, 0x00008001u, 0x000083ffu}, +}; +constexpr std::uint32_t TernaryDenormalFlushedIeee[32][3] = { + {0x00000000u, 0x00003c01u, 0x00000400u}, + {0x0000bc01u, 0x00000000u, 0x0000bc00u}, + {0x00008000u, 0x00000000u, 0x00000000u}, + {0x00008000u, 0x00000000u, 0x00008000u}, + {0x00008000u, 0x00000000u, 0x00000000u}, + {0x00008000u, 0x00000000u, 0x00008000u}, + {0x00000000u, 0x00000400u, 0x00000000u}, + {0x0000ff23u, 0x0000ff23u, 0x0000ff23u}, + {0x00000000u, 0x00000400u, 0x00000000u}, + {0x00008400u, 0x00008000u, 0x00008400u}, + {0x00008000u, 0x00008000u, 0x00008000u}, + {0x00000000u, 0x00000000u, 0x00000000u}, + {0x00007e01u, 0x00007e01u, 0x00007e01u}, + {0x0000fc00u, 0x00008000u, 0x0000fc00u}, + {0x00000000u, 0x00000000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00008000u}, + {0x00000000u, 0x00003c01u, 0x00000000u}, + {0x0000bc01u, 0x00000000u, 0x00008000u}, + {0x00007e01u, 0x00007e01u, 0x00007e01u}, + {0x00008000u, 0x00000000u, 0x00008000u}, + {0x00000000u, 0x00000400u, 0x00000000u}, + {0x00008400u, 0x00008000u, 0x00008000u}, + {0x00000000u, 0x00000400u, 0x00000000u}, + {0x00008400u, 0x00008000u, 0x00008000u}, + {0x00000000u, 0x00000000u, 0x00000000u}, + {0x00007e01u, 0x00007e01u, 0x00007e01u}, + {0x00000000u, 0x00000000u, 0x00000000u}, + {0x00008000u, 0x00008000u, 0x00008000u}, + {0x00000000u, 0x00007c00u, 0x00000000u}, + {0x0000fc00u, 0x00008000u, 0x00008000u}, + {0x00000000u, 0x00007c00u, 0x00000000u}, + {0x0000fc00u, 0x00008000u, 0x00008000u}, +}; +constexpr std::uint32_t TernaryDenormalKeptIeee[32][3] = { + {0x000003ffu, 0x00003c01u, 0x00000400u}, + {0x0000bc01u, 0x000003ffu, 0x0000bc00u}, + {0x00008001u, 0x000003ffu, 0x00000001u}, + {0x000083ffu, 0x000003ffu, 0x000083ffu}, + {0x00008001u, 0x00000001u, 0x00000000u}, + {0x00008001u, 0x00000001u, 0x00008000u}, + {0x000003ffu, 0x00000400u, 0x000003ffu}, + {0x0000ff23u, 0x0000ff23u, 0x0000ff23u}, + {0x00000001u, 0x00000400u, 0x00000001u}, + {0x00008400u, 0x00008001u, 0x00008400u}, + {0x00008001u, 0x00008001u, 0x00008001u}, + {0x00000001u, 0x00000001u, 0x00000001u}, + {0x00007e01u, 0x00007e01u, 0x00007e01u}, + {0x0000fc00u, 0x00008001u, 0x0000fc00u}, + {0x00000001u, 0x000003ffu, 0x00000200u}, + {0x000083ffu, 0x00008001u, 0x00008200u}, + {0x000003ffu, 0x00003c01u, 0x000003ffu}, + {0x0000bc01u, 0x000003ffu, 0x000083ffu}, + {0x00007e01u, 0x00007e01u, 0x00007e01u}, + {0x000083ffu, 0x000003ffu, 0x000083ffu}, + {0x00000000u, 0x00000400u, 0x00000001u}, + {0x00008400u, 0x00008000u, 0x00008001u}, + {0x00000001u, 0x00000400u, 0x000003ffu}, + {0x00008400u, 0x00008001u, 0x000083ffu}, + {0x00000001u, 0x00000001u, 0x00000001u}, + {0x00007e01u, 0x00007e01u, 0x00007e01u}, + {0x00000001u, 0x00000001u, 0x00000001u}, + {0x00008001u, 0x00008001u, 0x00008001u}, + {0x00000001u, 0x00007c00u, 0x000003ffu}, + {0x0000fc00u, 0x00008001u, 0x000083ffu}, + {0x00000001u, 0x00007c00u, 0x000003ffu}, + {0x0000fc00u, 0x00008001u, 0x000083ffu}, +}; constexpr std::uint32_t Chunks = 4; std::uint32_t Chunk = 0; bool Ieee = false; @@ -676,6 +812,10 @@ void Check() { const auto& expected = Ieee ? ((Denorm16 & 1u) != 0u ? DenormalKeptIeee : DenormalFlushedIeee) : ((Denorm16 & 1u) != 0u ? DenormalKept : DenormalFlushed); constexpr std::array indices{0u, 1u, 5u, 6u}; for (std::uint32_t i = 0u; i < indices.size(); ++i) Expect(tid, out[indices[i]], expected[tid][i], Names[indices[i]]); + if (SourceForm == 0u) { + const auto& ternaryExpected = Ieee ? ((Denorm16 & 1u) != 0u ? TernaryDenormalKeptIeee : TernaryDenormalFlushedIeee) : ((Denorm16 & 1u) != 0u ? TernaryDenormalKept : TernaryDenormalFlushed); + for (std::uint32_t i = 0u; i < 3u; ++i) Expect(tid, out[i + 2u], ternaryExpected[tid][i], Names[i + 2u]); + } continue; } for (std::uint32_t i = 0; i < 7; ++i) { diff --git a/core/shader/recompiler/Translation/src/FloatInstructions.cpp b/core/shader/recompiler/Translation/src/FloatInstructions.cpp index 67caada40..054bebc39 100644 --- a/core/shader/recompiler/Translation/src/FloatInstructions.cpp +++ b/core/shader/recompiler/Translation/src/FloatInstructions.cpp @@ -615,7 +615,7 @@ bool TranslationContext::minMaxF16(const RdnaInstruction& inst, IrOpcode opcode) for (std::uint32_t index = 0u; index < (ternary ? 3u : 2u); ++index) { const RdnaOperand& operand = sourceAt(inst, index); args[index] = Half{&readF16AsF32(operand).Value(), readF16Bits(operand)}; - if (!ternary && floatMode.has_value() && (floatMode->floatMode & 0x40u) == 0u) { + if (floatMode.has_value() && (floatMode->floatMode & 0x40u) == 0u) { const IrU1 tiny(ir.ULessThan(ir.BitwiseAnd(args[index].bits.Value(), ir.Constant(0x7fffu)), ir.Constant(0x0400u))); const IrU32 sign(ir.BitwiseAnd(args[index].bits.Value(), ir.Constant(0x8000u))); args[index].bits = IrU32(ir.Select(tiny.Value(), sign.Value(), args[index].bits.Value())); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 2f649a96e..df3f6db0c 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -368,7 +368,7 @@ Throughout the project, every function at every stage either **does exactly what - A [COND_INDIRECT_BUFFER](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp) (`sceAgcCbBranch`) runs only with the always-pass compare function; a comparison or a predicated packet throws. Modes 1 and 2 run the first buffer, assumed to be the if-then and if-then-else modes of AMD's PM4 packet. [sceAgcCbBranch](../../core/libs/prx/libSceAgc/Cb/src/Branch.cpp) does not check the compare address of an always-pass branch or the address of an empty buffer, as PPSA14632 passes an uninitialised compare address and a null else buffer of 0 dwords. - A [label or store](../../core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp) (`RELEASE_MEM`, `WRITE_DATA`, `DMA_DATA`) into guest memory the device has no host import of waits for all recorded work and is then written by the CPU, so the queue worker stalls once per such label. The console's CP stores it in stream order without that stall. - [SET_*_REG_INDIRECT](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp) (libSceAgcDriver) reads its register list when the command buffer is submitted, with the packets. The console's graphics PFP reads it when it reaches the packet, so a list written after the submission (by the CPU, or by GPU work ahead of the packet behind PFP_SYNC_ME) is read here before that write. -- The [shader float mode](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp) (`FLOAT_MODE`, `DX10_CLAMP`, `IEEE_MODE` and `FP16_OVFL` of `COMPUTE_PGM_RSRC1` and `SPI_SHADER_PGM_RSRC1_PS/GS/HS`, bit positions from Mesa's `gfx10.json` and `gfx103.json`) is decoded into `GuestContext::floatMode` and is part of the shader cache keys and the serialized request (#981). `IEEE_MODE` is read by `v_min_f32`, `v_max_f32`, `v_min3_f32`, `v_max3_f32` and `v_med3_f32`, their f16 forms, `v_min_f64` and `v_max_f64`, `IEEE_MODE` and the `FLOAT_MODE` output denormal bits by the output modifier, `DX10_CLAMP` by the clamp modifier, `FP16_OVFL` by every instruction with an f16 result (see its entry), and the whole mode by the 16-bit interpolation instructions (see their entries). With IEEE=1 a signaling NaN operand returns that NaN quieted (the first operand checked first, min3/max3 as two nested pairs, med3 as min3 when any operand is NaN), measured on RDNA2 (gfx1035) over every combination of ±0, ±1, ±inf, denormals, quiet and signaling NaNs with f32 denormals kept and flushed. IEEE=0 and unknown modes keep the previous f32 lowering, which matches IEEE=0. The f16 forms choose their NaN results on the bits: a NaN operand returns the other operand unchanged (src0 when both are NaN), and with IEEE=1 a signaling NaN operand returns that NaN quieted, in the same order as f32. `v_min_f64` and `v_max_f64` quiet a signaling NaN operand the same way with IEEE=1; with IEEE=0 they return src0 unchanged when src1 is NaN, otherwise src1 when src0 is NaN. Both were measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1 with f32 denormals kept and flushed and f16/f64 denormals kept; RDNA2 has no f64 min3, max3 or med3. Unknown modes follow IEEE=0. `v_min_f16` and `v_max_f16` flush each denormal input to its signed zero when bit 0 of the f16/f64 denormal field is clear (0 and 2); fields 1 and 3 preserve inputs, even when output flushing is enabled. Measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1, including signed zeros, NaNs and neg/abs modifiers. Unknown modes keep f16 denormals. With f16/f64 denormals flushed the hardware also flushes denormal inputs of the f16 ternary forms and f64 min/max, which those lowerings still do not. NaN quieting in f16 arithmetic (the unary instructions and the f32 arithmetic follow it, see below) and `FLOAT_MODE` (f32 denormals flushed before the min/max compare in Astro's Playroom's 0xc0) still follow a single mode, as the entries below record. The mode comes from the registered shader header's `sh_registers`, the same source at registration and at draw/dispatch so the prepared artifact keys match. A title that rewrites RSRC1 at run time, or a raw compute program with no header, compiles with that mode or none. +- The [shader float mode](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp) (`FLOAT_MODE`, `DX10_CLAMP`, `IEEE_MODE` and `FP16_OVFL` of `COMPUTE_PGM_RSRC1` and `SPI_SHADER_PGM_RSRC1_PS/GS/HS`, bit positions from Mesa's `gfx10.json` and `gfx103.json`) is decoded into `GuestContext::floatMode` and is part of the shader cache keys and the serialized request (#981). `IEEE_MODE` is read by `v_min_f32`, `v_max_f32`, `v_min3_f32`, `v_max3_f32` and `v_med3_f32`, their f16 forms, `v_min_f64` and `v_max_f64`, `IEEE_MODE` and the `FLOAT_MODE` output denormal bits by the output modifier, `DX10_CLAMP` by the clamp modifier, `FP16_OVFL` by every instruction with an f16 result (see its entry), and the whole mode by the 16-bit interpolation instructions (see their entries). With IEEE=1 a signaling NaN operand returns that NaN quieted (the first operand checked first, min3/max3 as two nested pairs, med3 as min3 when any operand is NaN), measured on RDNA2 (gfx1035) over every combination of ±0, ±1, ±inf, denormals, quiet and signaling NaNs with f32 denormals kept and flushed. IEEE=0 and unknown modes keep the previous f32 lowering, which matches IEEE=0. The f16 forms choose their NaN results on the bits: a NaN operand returns the other operand unchanged (src0 when both are NaN), and with IEEE=1 a signaling NaN operand returns that NaN quieted, in the same order as f32. `v_min_f64` and `v_max_f64` quiet a signaling NaN operand the same way with IEEE=1; with IEEE=0 they return src0 unchanged when src1 is NaN, otherwise src1 when src0 is NaN. Both were measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1 with f32 denormals kept and flushed and f16/f64 denormals kept; RDNA2 has no f64 min3, max3 or med3. Unknown modes follow IEEE=0. `v_min_f16` and `v_max_f16` flush each denormal input to its signed zero when bit 0 of the f16/f64 denormal field is clear (0 and 2); fields 1 and 3 preserve inputs, even when output flushing is enabled. Measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1, including signed zeros, NaNs and neg/abs modifiers. Unknown modes keep f16 denormals. With f16/f64 denormals flushed the hardware also flushes denormal inputs of f64 min/max, which those lowerings still do not. NaN quieting in f16 arithmetic (the unary instructions and the f32 arithmetic follow it, see below) and `FLOAT_MODE` (f32 denormals flushed before the min/max compare in Astro's Playroom's 0xc0) still follow a single mode, as the entries below record. The mode comes from the registered shader header's `sh_registers`, the same source at registration and at draw/dispatch so the prepared artifact keys match. A title that rewrites RSRC1 at run time, or a raw compute program with no header, compiles with that mode or none. - A [WAIT_REG_MEM](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/WaitMemory.cpp) (libSceAgcDriver) that a compute queue passes on another queue's recorded label store, before the GPU has executed it, is landed before the queue's next dispatch, draw, COND_EXEC or predicated packet reads guest memory: the worker polls until memory satisfies the wait or no unfinished batch writes the label. The graphics queue (queue 0) is exempt, as landing its waits costs PPSA01341 about a quarter of its frame rate; a title that fills memory a graphics command buffer reads after it submits the buffer behind such a wait reads it early there. `APS5_LAND_WAITS_ALL_QUEUES=1` lands the graphics queue's waits too. - The [dispatches that follow a compute queue's wait](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/DispatchAhead.cpp) (libSceAgcDriver), up to the next wait that memory does not satisfy, a packet that writes guest memory other than a label, or a conditional packet, are resolved (memory captured, shader recompiled) when the wait passes, as the CP reads them right after it; the graphics queue is exempt (`APS5_GROUP_CAPTURE_ALL_QUEUES=1` includes it). The device still reads guest memory when the dispatch is issued; when that memory no longer matches the capture, the dispatch is resolved again from current memory. - [v_mullit_f32](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) follows the hardware measured with the MODE register at IEEE=1 and denormals kept (a signaling NaN src0 is quieted only with IEEE=1, as measured at IEEE=0 too; a positive denormal src2 counts as positive); with f32 denormals flushed it follows the f32 denormals entry below, and denormal modes 1 and 2 throw. From f01f95d1f92f70929b6c70074f397724428ec5a0 Mon Sep 17 00:00:00 2001 From: Joey Fisher Date: Sat, 10 Oct 2026 12:23:12 -0600 Subject: [PATCH 108/460] fix(agc): report a refused queue worker priority as a driver failure and reject unknown APS5_THREAD_PRIORITY values --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 5 +++++ .../Execution/src/Driver/Queues/Submission.cpp | 7 ++++++- .../Execution/src/ThreadPriority.cpp | 7 ++++--- core/libs/prx/libSceAgcDriver/tests/Pm4.cpp | 14 ++++++++++++++ .../prx/libSceAgcDriver/tests/ThreadPriority.cpp | 16 ++++++++++++---- docs/dev/BUILD.md | 2 +- 6 files changed, 42 insertions(+), 9 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 702b93040..279347170 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -936,6 +936,11 @@ if(BUILD_TESTING) set_tests_properties(agc_driver_pm4_failure PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + if(NOT WIN32) + add_test(NAME agc_driver_pm4_priority_refused COMMAND agc_driver_pm4_tests priority-refused) + set_tests_properties(agc_driver_pm4_priority_refused PROPERTIES ENVIRONMENT + "ANYPS5_NO_SHADER_CACHE=1;APS5_THREAD_PRIORITY=high;DBUS_SYSTEM_BUS_ADDRESS=unix:path=/nonexistent/aps5-no-rtkit" SKIP_RETURN_CODE 77) + endif() add_test(NAME agc_driver_flip_compute COMMAND agc_driver_flip_tests compute) set_tests_properties(agc_driver_flip_compute PROPERTIES ENVIRONMENT "ANYPS5_NO_SHADER_CACHE=1" SKIP_RETURN_CODE 77) if(WIN32) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp index 9d17a60fc..83d3b48ac 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp @@ -317,7 +317,12 @@ void Driver::enqueue(Submission submission) { if (!worker.thread.joinable()) worker.thread = std::thread([this, queue] { char role[32]; std::snprintf(role, sizeof(role), "queue 0x%x worker", queue); - RaiseWorkerThreadPriority(role); + try { + RaiseWorkerThreadPriority(role); + } catch (...) { + ReportFailure(std::current_exception()); + return; + } run(queue); }); } diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp index 007008df7..2db74fc43 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp @@ -283,9 +283,10 @@ void Raise(const char* role) { } ThreadPriorityMode ParseThreadPriorityMode(const char* text) { - if (text == nullptr || *text == '\0' || std::strcmp(text, "off") == 0 || std::strcmp(text, "0") == 0) return ThreadPriorityMode::Off; - if (std::strcmp(text, "rt") == 0 || std::strcmp(text, "realtime") == 0) return ThreadPriorityMode::Realtime; - return ThreadPriorityMode::High; + if (text == nullptr || *text == '\0' || std::strcmp(text, "off") == 0) return ThreadPriorityMode::Off; + if (std::strcmp(text, "high") == 0) return ThreadPriorityMode::High; + if (std::strcmp(text, "rt") == 0) return ThreadPriorityMode::Realtime; + throw std::invalid_argument(std::string("APS5_THREAD_PRIORITY=") + text + " is not off, high or rt"); } RealtimePlan PlanRealtime(std::uint32_t priority, const std::optional& rtkit, RttimeLimit current, std::uint64_t requestedUs) { diff --git a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp index 90d0e7486..e6ef42a26 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp @@ -1015,6 +1015,15 @@ void testAsyncMemoryFailure() { expectFailure([] { LibcRunShutdown_nid_postfix(); }, "guest"); } +void testRefusedWorkerPriority() { + alignas(8) static std::uint32_t result = 0; + auto words = writeWord(result, 91); + submitWords(words); + expectFailure([] { AgcDriverWaitIdle_nid_postfix(); }, "refused"); + check(result == 0, "a queue worker whose priority was refused executed its submission"); + expectFailure([] { LibcRunShutdown_nid_postfix(); }, "refused"); +} + } void testUnwrittenUserData() { @@ -1032,6 +1041,11 @@ int main(int argc, char** argv) { std::puts("PM4 asynchronous memory failure propagated to idle, suspend, submit and shutdown"); return 0; } + if (argc == 2 && std::string(argv[1]) == "priority-refused") { + testRefusedWorkerPriority(); + std::puts("PM4 refused queue worker priority propagated to idle and shutdown"); + return 0; + } testCatalog(); testNopPad(); testWriteChangedKeepsUntouchedBytes(); diff --git a/core/libs/prx/libSceAgcDriver/tests/ThreadPriority.cpp b/core/libs/prx/libSceAgcDriver/tests/ThreadPriority.cpp index 569852233..9c43f7e6c 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ThreadPriority.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ThreadPriority.cpp @@ -19,9 +19,18 @@ void Require(bool condition, const std::string& message) { } void ParseTests() { - for (const char* text : {static_cast(nullptr), "", "off", "0"}) Require(ParseThreadPriorityMode(text) == ThreadPriorityMode::Off, std::string("APS5_THREAD_PRIORITY=") + (text == nullptr ? "(unset)" : text) + " must leave priorities alone"); - for (const char* text : {"rt", "realtime"}) Require(ParseThreadPriorityMode(text) == ThreadPriorityMode::Realtime, std::string("APS5_THREAD_PRIORITY=") + text + " must ask for real-time priority"); - for (const char* text : {"high", "1", "on"}) Require(ParseThreadPriorityMode(text) == ThreadPriorityMode::High, std::string("APS5_THREAD_PRIORITY=") + text + " must ask for a nice level"); + for (const char* text : {static_cast(nullptr), "", "off"}) Require(ParseThreadPriorityMode(text) == ThreadPriorityMode::Off, std::string("APS5_THREAD_PRIORITY=") + (text == nullptr ? "(unset)" : text) + " must leave priorities alone"); + Require(ParseThreadPriorityMode("rt") == ThreadPriorityMode::Realtime, "APS5_THREAD_PRIORITY=rt must ask for real-time priority"); + Require(ParseThreadPriorityMode("high") == ThreadPriorityMode::High, "APS5_THREAD_PRIORITY=high must ask for a nice level"); + for (const char* text : {"hgih", "HIGH", "realtime", "on", "1", "0", " high"}) { + bool threw = false; + try { + static_cast(ParseThreadPriorityMode(text)); + } catch (const std::invalid_argument&) { + threw = true; + } + Require(threw, std::string("APS5_THREAD_PRIORITY=") + text + " must be rejected"); + } } void ExpectRefused(const RealtimePlan& plan, const std::string& what) { @@ -47,7 +56,6 @@ void PlanTests() { Require(kept.refusal.empty() && !kept.limit.has_value(), "limits rtkit already accepts must be left alone"); } -// Run with APS5_THREAD_PRIORITY=rt or =high and no reachable rtkit: the raise must throw, not settle for less. void RefusalTest() { bool threw = false; try { diff --git a/docs/dev/BUILD.md b/docs/dev/BUILD.md index 0b06c87ed..38264ca7c 100644 --- a/docs/dev/BUILD.md +++ b/docs/dev/BUILD.md @@ -92,7 +92,7 @@ Set `APS5_PIPELINE_STATS=1` to capture and print driver statistics for each newl ## Driver thread priority -Set `APS5_THREAD_PRIORITY=high` to raise the AGC driver's queue workers and the video output presenter above the title's threads. On Linux the threads get nice `APS5_THREAD_NICE` (default -10) through rtkit; on Windows they get `THREAD_PRIORITY_HIGHEST`. `APS5_THREAD_PRIORITY=rt` asks rtkit for real-time priority `APS5_THREAD_RT` (default 10, at most rtkit's `MaxRealtimePriority`) under rtkit's policy, `SCHED_RR` by default; on Windows it sets `THREAD_PRIORITY_TIME_CRITICAL`. rtkit only grants it when the process's hard `RLIMIT_RTTIME` is at most its `RTTimeUSecMax` (200000 us by default), so a hard limit above `APS5_THREAD_RTTIME_US` (default 200000, capped at `RTTimeUSecMax`) is lowered to it and the soft limit to at most half of it. The hard limit cannot be raised again and stays lowered for the whole process even if rtkit then refuses; see [technical debt](TechnicalDebt.md#functional). There is no fallback between the modes: when rtkit or `SetThreadPriority` refuses, the thread throws with the refusal. Each raised thread logs its priority once as `[thread-priority]`. The setting is disabled by default. +Set `APS5_THREAD_PRIORITY=high` to raise the AGC driver's queue workers and the video output presenter above the title's threads. On Linux the threads get nice `APS5_THREAD_NICE` (default -10) through rtkit; on Windows they get `THREAD_PRIORITY_HIGHEST`. `APS5_THREAD_PRIORITY=rt` asks rtkit for real-time priority `APS5_THREAD_RT` (default 10, at most rtkit's `MaxRealtimePriority`) under rtkit's policy, `SCHED_RR` by default; on Windows it sets `THREAD_PRIORITY_TIME_CRITICAL`. rtkit only grants it when the process's hard `RLIMIT_RTTIME` is at most its `RTTimeUSecMax` (200000 us by default), so a hard limit above `APS5_THREAD_RTTIME_US` (default 200000, capped at `RTTimeUSecMax`) is lowered to it and the soft limit to at most half of it. The hard limit cannot be raised again and stays lowered for the whole process even if rtkit then refuses; see [technical debt](TechnicalDebt.md#functional). There is no fallback between the modes: when rtkit or `SetThreadPriority` refuses, a queue worker reports the refusal as a driver failure, which the next submission, wait or shutdown rethrows, and the presenter's start throws it. Each raised thread logs its priority once as `[thread-priority]`. The setting is `off` by default; a value other than `off`, `high` or `rt` is reported the same way as a refusal. ## Shader recompiler From c96100aff193179273d9ef8ebcc13f400a61c232 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Thu, 8 Oct 2026 00:24:53 +0000 Subject: [PATCH 109/460] fix(libc): map directory symlink loop errors (cherry picked from commit d5e348c7b6846b2cd2cb5d1ecad9ad3493462047) --- core/libs/prx/libc/src/Directory.cpp | 3 +++ core/libs/tests/GuestDirectory.cpp | 7 +++++++ 2 files changed, 10 insertions(+) diff --git a/core/libs/prx/libc/src/Directory.cpp b/core/libs/prx/libc/src/Directory.cpp index b6fff287b..c3eefced4 100644 --- a/core/libs/prx/libc/src/Directory.cpp +++ b/core/libs/prx/libc/src/Directory.cpp @@ -17,6 +17,9 @@ int DirectoryError(int error) { case ENFILE: return 23; case ENOMEM: return 12; case ENAMETOOLONG: return 63; +#ifdef ELOOP + case ELOOP: return 62; +#endif default: return 5; } } diff --git a/core/libs/tests/GuestDirectory.cpp b/core/libs/tests/GuestDirectory.cpp index 4ada7880c..d628a0e72 100644 --- a/core/libs/tests/GuestDirectory.cpp +++ b/core/libs/tests/GuestDirectory.cpp @@ -42,6 +42,13 @@ int main() { Require(closedir_nid_postfix(directory) == 0); Require(opendir_nid_postfix((root / "missing").string().c_str()) == nullptr); Require(*__error_nid_postfix() == 2); +#ifndef _WIN32 + const auto symlinkLoop = root / "symlink-loop"; + std::filesystem::create_symlink("symlink-loop", symlinkLoop); + Require(opendir_nid_postfix(symlinkLoop.string().c_str()) == nullptr); + Require(*__error_nid_postfix() == 62); + Require(std::filesystem::remove(symlinkLoop)); +#endif Require(opendir_nid_postfix((root / "sample.txt").string().c_str()) == nullptr); Require(*__error_nid_postfix() == 20); Require(opendir_nid_postfix("") == nullptr && *__error_nid_postfix() == 2); From 0272f99ad35e5839b17db88504ddd179cecd5ce5 Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Sat, 10 Oct 2026 13:00:00 +0900 Subject: [PATCH 110/460] feat(libkernel): implement siglongjmp PokeMMO Prospero (PPSA98001) stops at siglongjmp (pebqbE5ws8s) from libkernel. The title links its own FreeBSD sigsetjmp, so siglongjmp follows FreeBSD's amd64 __siglongjmp and its sigjmp_buf layout: when the save-mask flag at offset 88 is set it restores the mask saved at 72 through _sigprocmask, then reloads rbx, rsp, rbp, r12-r15 and the x87 control word and returns to the saved address with val, or 1 for 0. Co-Authored-By: Claude Opus 5.5 --- .../prx/libkernel/System/src/SignalSets.cpp | 41 ++++++++++ core/libs/tests/GuestSignalSets.cpp | 74 +++++++++++++++++++ 2 files changed, 115 insertions(+) diff --git a/core/libs/prx/libkernel/System/src/SignalSets.cpp b/core/libs/prx/libkernel/System/src/SignalSets.cpp index f2831d86c..8345aad33 100644 --- a/core/libs/prx/libkernel/System/src/SignalSets.cpp +++ b/core/libs/prx/libkernel/System/src/SignalSets.cpp @@ -60,3 +60,44 @@ int APS5_VABI sigismember_nid_postfix(const GuestSignalSet* set, int signal) { return (set->bits[WordIndex(signal)] & Bit(signal)) != 0 ? 1 : 0; } } +#ifdef _WIN32 +#define APS5_SIGNAL_ASM_FUNCTION(name) ".globl " name "\n.def " name "; .scl 2; .type 32; .endef\n" name ":\n" +#define APS5_SIGNAL_ASM_CALL(name) " call " name "\n" +#else +#define APS5_SIGNAL_ASM_FUNCTION(name) ".globl " name "\n.type " name ", @function\n" name ":\n" +#define APS5_SIGNAL_ASM_CALL(name) " call " name "@PLT\n" +#endif + +asm(".text\n" + APS5_SIGNAL_ASM_FUNCTION("siglongjmp_nid_postfix") + " cmpl $0, 88(%rdi)\n" + " jz 2f\n" + " movq %rdi, %rdx\n" + " pushq %rdi\n" + " pushq %rsi\n" + " movl $3, %edi\n" + " leaq 72(%rdx), %rsi\n" + " xorl %edx, %edx\n" + " subq $8, %rsp\n" + APS5_SIGNAL_ASM_CALL("_sigprocmask_nid_postfix") + " addq $8, %rsp\n" + " popq %rsi\n" + " popq %rdi\n" + "2:\n" + " movq %rdi, %rdx\n" + " movl %esi, %eax\n" + " movq 0(%rdx), %rcx\n" + " movq 8(%rdx), %rbx\n" + " movq 16(%rdx), %rsp\n" + " movq 24(%rdx), %rbp\n" + " movq 32(%rdx), %r12\n" + " movq 40(%rdx), %r13\n" + " movq 48(%rdx), %r14\n" + " movq 56(%rdx), %r15\n" + " fldcw 64(%rdx)\n" + " testl %eax, %eax\n" + " jnz 1f\n" + " incl %eax\n" + "1:\n" + " movq %rcx, 0(%rsp)\n" + " ret\n"); diff --git a/core/libs/tests/GuestSignalSets.cpp b/core/libs/tests/GuestSignalSets.cpp index f9df0f153..018783bf4 100644 --- a/core/libs/tests/GuestSignalSets.cpp +++ b/core/libs/tests/GuestSignalSets.cpp @@ -15,15 +15,89 @@ int APS5_VABI sigaddset_nid_postfix(GuestSignalSet*, int); int APS5_VABI sigdelset_nid_postfix(GuestSignalSet*, int); int APS5_VABI sigismember_nid_postfix(const GuestSignalSet*, int); int* APS5_VABI __error_nid_postfix(); +int APS5_VABI _sigprocmask_nid_postfix(int, const GuestSignalSet*, GuestSignalSet*); +int APS5_VABI TitleSigsetjmp(std::uint64_t* buffer, int saveMask); +[[noreturn]] void APS5_VABI siglongjmp_nid_postfix(std::uint64_t* buffer, int value); } +#ifdef _WIN32 +#define TEST_ASM_FUNCTION(name) ".globl " name "\n.def " name "; .scl 2; .type 32; .endef\n" name ":\n" +#define TEST_ASM_CALL(name) " call " name "\n" +#else +#define TEST_ASM_FUNCTION(name) ".globl " name "\n.type " name ", @function\n" name ":\n" +#define TEST_ASM_CALL(name) " call " name "@PLT\n" +#endif + +asm(".text\n" + TEST_ASM_FUNCTION("TitleSigsetjmp") + " movl %esi, 88(%rdi)\n" + " testl %esi, %esi\n" + " jz 2f\n" + " pushq %rdi\n" + " movq %rdi, %rcx\n" + " movq $1, %rdi\n" + " movq $0, %rsi\n" + " leaq 72(%rcx), %rdx\n" + TEST_ASM_CALL("_sigprocmask_nid_postfix") + " popq %rdi\n" + "2:\n" + " movq %rdi, %rcx\n" + " movq 0(%rsp), %rdx\n" + " movq %rdx, 0(%rcx)\n" + " movq %rbx, 8(%rcx)\n" + " movq %rsp, 16(%rcx)\n" + " movq %rbp, 24(%rcx)\n" + " movq %r12, 32(%rcx)\n" + " movq %r13, 40(%rcx)\n" + " movq %r14, 48(%rcx)\n" + " movq %r15, 56(%rcx)\n" + " fnstcw 64(%rcx)\n" + " xorq %rax, %rax\n" + " ret\n"); + static void Require(bool value) { if (!value) std::abort(); } static bool Equals(const GuestSignalSet& set, std::uint32_t w0, std::uint32_t w1, std::uint32_t w2, std::uint32_t w3) { return set.bits[0] == w0 && set.bits[1] == w1 && set.bits[2] == w2 && set.bits[3] == w3; } +static std::uint32_t BlockedMask() { + GuestSignalSet current{}; + Require(_sigprocmask_nid_postfix(1, nullptr, ¤t) == 0); + return current.bits[0]; +} + +static void SetBlockedMask(std::uint32_t mask) { + const GuestSignalSet set{{mask, 0, 0, 0}}; + Require(_sigprocmask_nid_postfix(3, &set, nullptr) == 0); +} + +static int JumpBack(int saveMask, int value, std::uint32_t maskAtSet, std::uint32_t maskAtJump) { + std::uint64_t buffer[12]{}; + volatile int jumped = 0; + SetBlockedMask(maskAtSet); + const int result = TitleSigsetjmp(buffer, saveMask); + if (jumped == 0) { + Require(result == 0); + jumped = 1; + SetBlockedMask(maskAtJump); + siglongjmp_nid_postfix(buffer, value); + } + return result; +} + +static void CheckSiglongjmp() { + Require(JumpBack(1, 0, 0x5u, 0x30u) == 1); + Require(BlockedMask() == 0x5u); + Require(JumpBack(1, 7, 0x9u, 0u) == 7); + Require(BlockedMask() == 0x9u); + Require(JumpBack(0, -3, 0x5u, 0x30u) == -3); + Require(BlockedMask() == 0x30u); + SetBlockedMask(0); +} + int main() { + CheckSiglongjmp(); static constexpr int Untouched = 5; GuestSignalSet set{{0x12345678u, 0x9abcdef0u, 0xffffffffu, 1u}}; *__error_nid_postfix() = Untouched; From b0690903bbec833824755c74217072e1d992c48f Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 13:16:33 +0000 Subject: [PATCH 111/460] fix(libc): match FreeBSD rename errors on Windows --- core/libs/prx/libc/src/Rename.cpp | 68 +++++++++++++++++++++++++++-- core/libs/tests/GuestFilesystem.cpp | 46 +++++++++++++++++++ 2 files changed, 111 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libc/src/Rename.cpp b/core/libs/prx/libc/src/Rename.cpp index 3e5c5658c..b5bb04527 100644 --- a/core/libs/prx/libc/src/Rename.cpp +++ b/core/libs/prx/libc/src/Rename.cpp @@ -1,17 +1,79 @@ #include "prx/libc/include/FilesystemError.hpp" #include "prx/libc/include/General.hpp" #include +#include #include +static int RenamePath_nid_no_patch(const std::filesystem::path& source, const std::filesystem::path& destination) { + constexpr int GuestEnoent = 2; + constexpr int GuestEio = 5; + constexpr int GuestEnotdir = 20; + constexpr int GuestEisdir = 21; + constexpr int GuestEinval = 22; + constexpr int GuestEnotempty = 66; + namespace fs = std::filesystem; + std::error_code error; + const auto sourceStatus = fs::symlink_status(source, error); + if (error && error != std::errc::no_such_file_or_directory) return FilesystemError(error); + if (sourceStatus.type() == fs::file_type::not_found) return GuestEnoent; + + const auto parent = destination.parent_path().empty() ? fs::path(".") : destination.parent_path(); + error.clear(); + const auto parentStatus = fs::status(parent, error); + if (error) return FilesystemError(error); + if (!fs::exists(parentStatus)) return GuestEnoent; + if (!fs::is_directory(parentStatus)) return GuestEnotdir; + + error.clear(); + const auto destinationStatus = fs::symlink_status(destination, error); + if (error && error != std::errc::no_such_file_or_directory) return FilesystemError(error); + const bool destinationExists = destinationStatus.type() != fs::file_type::not_found; + const bool sourceIsDirectory = fs::is_directory(sourceStatus); + const bool destinationIsDirectory = destinationExists && fs::is_directory(destinationStatus); + bool sameFile = false; + if (destinationExists) { + error.clear(); + sameFile = fs::equivalent(source, destination, error); + if (error == std::errc::no_such_file_or_directory) error.clear(); + if (error) return FilesystemError(error); + if (!sameFile && sourceIsDirectory && !destinationIsDirectory) return GuestEnotdir; + if (!sameFile && !sourceIsDirectory && destinationIsDirectory) return GuestEisdir; + } + if (sourceIsDirectory && !sameFile) { + error.clear(); + const auto canonicalSource = fs::weakly_canonical(source, error); + if (error) return FilesystemError(error); + const auto canonicalDestination = fs::weakly_canonical(destination, error); + if (error) return FilesystemError(error); + const auto relative = canonicalDestination.lexically_relative(canonicalSource); + if (!relative.empty() && *relative.begin() != "..") return GuestEinval; + } + if (destinationExists && !sameFile && destinationIsDirectory) { + error.clear(); + if (!fs::is_empty(destination, error)) { + return error ? FilesystemError(error) : GuestEnotempty; + } + } +#ifdef _WIN32 + if (!sameFile && sourceIsDirectory && destinationIsDirectory) { + error.clear(); + if (!fs::remove(destination, error)) return error ? FilesystemError(error) : GuestEio; + } +#endif + + error.clear(); + fs::rename(source, destination, error); + return error ? FilesystemError(error) : 0; +} + extern "C" int APS5_VABI rename_nid_postfix(const char* from, const char* to) { if (!from || !to) { errno = 14; return -1; } if (!*from || !*to) { errno = 2; return -1; } try { const auto source = ResolvePath_nid_no_patch(from); const auto destination = ResolvePath_nid_no_patch(to); - std::error_code error; - std::filesystem::rename(source, destination, error); - if (error) { errno = FilesystemError(error); return -1; } + const int error = RenamePath_nid_no_patch(source, destination); + if (error) { errno = error; return -1; } RecordWrittenPath_nid_no_patch(source); RecordWrittenPath_nid_no_patch(destination); return 0; diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index 945bcdcf2..9243e74c8 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -120,6 +120,52 @@ int main() { Require(rename_nid_postfix(file.string().c_str(), renamed.string().c_str()) == -1); Require(*__error_nid_postfix() == 2); Require(rename_nid_postfix(renamed.string().c_str(), file.string().c_str()) == 0); + { + const auto area = root / "libc_rename"; + const auto sourceFile = area / "source.txt"; + const auto targetFile = area / "target.txt"; + const auto sourceDirectory = area / "source_directory"; + const auto emptyDirectory = area / "empty_directory"; + const auto fullDirectory = area / "full_directory"; + Require(std::filesystem::create_directories(area)); + { std::ofstream stream(sourceFile); stream << "source"; } + { std::ofstream stream(targetFile); stream << "target"; } + Require(std::filesystem::create_directories(sourceDirectory / "child")); + Require(std::filesystem::create_directories(emptyDirectory)); + Require(std::filesystem::create_directories(fullDirectory)); + { std::ofstream stream(fullDirectory / "entry"); stream << "entry"; } + const auto name = [](const std::filesystem::path& path) { return path.string(); }; + const auto missing = name(area / "missing"); + const auto absentParent = name(area / "absent" / "target"); + const auto fileParent = name(sourceFile / "target"); + Require(rename_nid_postfix(missing.c_str(), name(targetFile).c_str()) == -1 && *__error_nid_postfix() == 2); + Require(rename_nid_postfix(name(sourceFile).c_str(), absentParent.c_str()) == -1 && *__error_nid_postfix() == 2); + Require(rename_nid_postfix(name(sourceFile).c_str(), fileParent.c_str()) == -1 && *__error_nid_postfix() == 20); + Require(rename_nid_postfix(name(sourceFile).c_str(), name(fullDirectory).c_str()) == -1 && *__error_nid_postfix() == 21); + Require(rename_nid_postfix(name(sourceDirectory).c_str(), name(targetFile).c_str()) == -1 && *__error_nid_postfix() == 20); + Require(rename_nid_postfix(name(sourceDirectory).c_str(), name(fullDirectory).c_str()) == -1 && *__error_nid_postfix() == 66); + const auto childDestination = name(sourceDirectory / "child" / "moved"); + Require(rename_nid_postfix(name(sourceDirectory).c_str(), childDestination.c_str()) == -1 && *__error_nid_postfix() == 22); + const auto childDirectory = name(sourceDirectory / "child"); + Require(rename_nid_postfix(name(sourceDirectory).c_str(), childDirectory.c_str()) == -1 && *__error_nid_postfix() == 22); + Require(std::filesystem::is_regular_file(sourceFile) && std::filesystem::is_regular_file(fullDirectory / "entry")); + Require(rename_nid_postfix(name(sourceFile).c_str(), name(targetFile).c_str()) == 0); + { std::ifstream stream(targetFile); std::string contents; std::getline(stream, contents); Require(contents == "source"); } + Require(!std::filesystem::exists(sourceFile)); + Require(rename_nid_postfix(name(targetFile).c_str(), name(targetFile).c_str()) == 0); + Require(rename_nid_postfix(name(sourceDirectory).c_str(), name(emptyDirectory).c_str()) == 0); + Require(!std::filesystem::exists(sourceDirectory) && std::filesystem::is_directory(emptyDirectory / "child")); +#ifndef _WIN32 + const auto dangling = area / "dangling"; + std::filesystem::create_symlink(area / "nonexistent", dangling); + Require(std::filesystem::is_symlink(dangling)); + const auto replacement = area / "replacement"; + { std::ofstream stream(replacement); stream << "replacement"; } + Require(rename_nid_postfix(name(replacement).c_str(), name(dangling).c_str()) == 0); + Require(!std::filesystem::is_symlink(dangling) && std::filesystem::is_regular_file(dangling)); +#endif + std::filesystem::remove_all(area); + } Require(remove_nid_postfix(file.string().c_str()) == 0); Require(!std::filesystem::exists(file)); Require(remove_nid_postfix(file.string().c_str()) == -1 && *__error_nid_postfix() == 2); From e7311656c94cb0c6048e3e1e5684ea6b8a60568e Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Fri, 9 Oct 2026 19:39:57 +0900 Subject: [PATCH 112/460] feat(libc): implement __isinf Export the double version next to __isnan and __isfinite, returning 1 for either infinity like __isinff. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libc/src/Math.cpp | 1 + core/libs/tests/GuestMath.cpp | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/core/libs/prx/libc/src/Math.cpp b/core/libs/prx/libc/src/Math.cpp index 2828d8ed4..624dce8e6 100644 --- a/core/libs/prx/libc/src/Math.cpp +++ b/core/libs/prx/libc/src/Math.cpp @@ -84,6 +84,7 @@ float APS5_VABI cbrtf_nid_postfix(float x) { return std::cbrt(x); } float APS5_VABI remainderf_nid_postfix(float x, float y) { return std::remainder(x, y); } int APS5_VABI __isfinite_nid_postfix(double x) { return std::isfinite(x) ? 1 : 0; } int APS5_VABI __isnan_nid_postfix(double x) { return std::isnan(x) ? 1 : 0; } +int APS5_VABI __isinf_nid_postfix(double x) { return std::isinf(x) ? 1 : 0; } int APS5_VABI __signbit_nid_postfix(double x) { return std::signbit(x) ? 1 : 0; } double APS5_VABI modf_nid_postfix(double x, double* integral) { return std::modf(x, integral); } diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index ee0b4d20f..3dc657e92 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -49,6 +49,7 @@ int APS5_VABI __isfinitef_nid_postfix(float); int APS5_VABI __isnormal_nid_postfix(double); int APS5_VABI __isnormalf_nid_postfix(float); int APS5_VABI __isinff_nid_postfix(float); +int APS5_VABI __isinf_nid_postfix(double); std::lldiv_t APS5_VABI lldiv_nid_postfix(long long, long long); std::lldiv_t APS5_VABI ldiv_nid_postfix(std::int64_t, std::int64_t); } @@ -254,6 +255,9 @@ int main() { const auto nan = std::numeric_limits::quiet_NaN(); Require(__isinff_nid_postfix(infinity) == 1 && __isinff_nid_postfix(-infinity) == 1); Require(__isinff_nid_postfix(nan) == 0 && __isinff_nid_postfix(1.f) == 0); + Require(__isinf_nid_postfix(std::numeric_limits::infinity()) == 1 && __isinf_nid_postfix(-std::numeric_limits::infinity()) == 1); + Require(__isinf_nid_postfix(std::numeric_limits::quiet_NaN()) == 0 && __isinf_nid_postfix(std::numeric_limits::max()) == 0); + Require(__isinf_nid_postfix(0.) == 0 && __isinf_nid_postfix(1e308 * 10) == 1); Require(__isfinitef_nid_postfix(0.f) == 1 && __isfinitef_nid_postfix(infinity) == 0); Require(__isfinitef_nid_postfix(nan) == 0); Require(__isnormalf_nid_postfix(1.f) == 1 && __isnormalf_nid_postfix(0.f) == 0); From 909158503d94251fb28cffec2b7b71c05a1704ee Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Fri, 9 Oct 2026 19:49:06 +0900 Subject: [PATCH 113/460] feat(libc): implement atol and atoll Both parse like strtol and strtoll in base 10. The guest long is 64 bits wide on every host, so atol uses strtoll rather than the host atol, whose long is 32 bits on Windows. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libc/src/Strings.cpp | 8 ++++++++ core/libs/tests/GuestMath.cpp | 7 +++++++ 2 files changed, 15 insertions(+) diff --git a/core/libs/prx/libc/src/Strings.cpp b/core/libs/prx/libc/src/Strings.cpp index f60b7ea24..87dfb4dc0 100644 --- a/core/libs/prx/libc/src/Strings.cpp +++ b/core/libs/prx/libc/src/Strings.cpp @@ -163,6 +163,14 @@ int APS5_VABI atoi_nid_postfix(const char* str) { return std::atoi(str); } +std::int64_t APS5_VABI atol_nid_postfix(const char* str) { + return std::strtoll(str, nullptr, 10); +} + +long long APS5_VABI atoll_nid_postfix(const char* str) { + return std::strtoll(str, nullptr, 10); +} + std::div_t APS5_VABI div_nid_postfix(int numerator, int denominator) { return std::div(numerator, denominator); } diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index ee0b4d20f..031e9fadd 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -20,6 +20,8 @@ unsigned long long APS5_VABI strtoull_nid_postfix(const char*, char**, int); std::uintmax_t APS5_VABI strtoumax_nid_postfix(const char*, char**, int); unsigned long long APS5_VABI _Stoull_nid_postfix(const char*, char**, int); std::uint64_t APS5_VABI _Stoul_nid_postfix(const char*, char**, int); +std::int64_t APS5_VABI atol_nid_postfix(const char*); +long long APS5_VABI atoll_nid_postfix(const char*); int* APS5_VABI __error_nid_postfix(); struct LibcFloatConstant { std::uint32_t bits[4]; }; extern LibcFloatConstant _FInf_nid_postfix; @@ -161,6 +163,11 @@ static void CheckIntegerConversions() { Require(_Stoull_nid_postfix(text, &end, base) == 0 && end == text + 3); Require(_Stoul_nid_postfix(text, &end, base) == 0 && end == text + 3); } + Require(atol_nid_postfix(" \t-4294967296tail") == -INT64_C(4294967296)); + Require(atol_nid_postfix("9223372036854775807") == INT64_MAX && atol_nid_postfix("+12") == 12); + Require(atol_nid_postfix("0x10") == 0 && atol_nid_postfix("010") == 10 && atol_nid_postfix("") == 0); + Require(atoll_nid_postfix("-9223372036854775808") == INT64_MIN && atoll_nid_postfix("4294967297x") == 4294967297LL); + Require(atoll_nid_postfix(" -0012") == -12 && atoll_nid_postfix("z1") == 0); *__error_nid_postfix() = 13; Require(strtol_nid_postfix("-4294967296", nullptr, 10) == -INT64_C(4294967296)); Require(*__error_nid_postfix() == 13); From 3c431989438c3cc30d81b27316eb492629ef036a Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Fri, 9 Oct 2026 20:09:33 +0900 Subject: [PATCH 114/460] feat(libc): implement llroundf Export it next to lroundf and llround, rounding halfway cases away from zero into the 64-bit guest long long. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libc/src/Math.cpp | 1 + core/libs/tests/GuestMath.cpp | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/core/libs/prx/libc/src/Math.cpp b/core/libs/prx/libc/src/Math.cpp index 2828d8ed4..504d281bf 100644 --- a/core/libs/prx/libc/src/Math.cpp +++ b/core/libs/prx/libc/src/Math.cpp @@ -37,6 +37,7 @@ float APS5_VABI frexpf_nid_postfix(float x, int* exponent) { return std::frexp(x std::int64_t APS5_VABI lround_nid_postfix(double x) { return std::llround(x); } std::int64_t APS5_VABI lroundf_nid_postfix(float x) { return std::llround(x); } std::int64_t APS5_VABI llround_nid_postfix(double x) { return std::llround(x); } +std::int64_t APS5_VABI llroundf_nid_postfix(float x) { return std::llround(x); } int APS5_VABI __isfinitef_nid_postfix(float x) { return std::isfinite(x) ? 1 : 0; } int APS5_VABI __isnormal_nid_postfix(double x) { return std::isnormal(x) ? 1 : 0; } int APS5_VABI __isnormalf_nid_postfix(float x) { return std::isnormal(x) ? 1 : 0; } diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index ee0b4d20f..efbec065e 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -45,6 +45,7 @@ std::int64_t APS5_VABI lround_nid_postfix(double); std::div_t APS5_VABI div_nid_postfix(int, int); std::int64_t APS5_VABI lroundf_nid_postfix(float); std::int64_t APS5_VABI llround_nid_postfix(double); +std::int64_t APS5_VABI llroundf_nid_postfix(float); int APS5_VABI __isfinitef_nid_postfix(float); int APS5_VABI __isnormal_nid_postfix(double); int APS5_VABI __isnormalf_nid_postfix(float); @@ -250,6 +251,9 @@ int main() { Require(lroundf_nid_postfix(4294967296.f) == INT64_C(4294967296)); Require(lroundf_nid_postfix(2.5f) == 3); Require(llround_nid_postfix(-4294967296.5) == -INT64_C(4294967297)); + Require(llroundf_nid_postfix(2.5f) == 3 && llroundf_nid_postfix(-2.5f) == -3 && llroundf_nid_postfix(-0.4f) == 0); + Require(llroundf_nid_postfix(8589934592.f) == INT64_C(8589934592) && llroundf_nid_postfix(-0x1p62f) == -(INT64_C(1) << 62)); + Require(llroundf_nid_postfix(0.49999997f) == 0 && llroundf_nid_postfix(16777215.f) == 16777215); const auto infinity = std::numeric_limits::infinity(); const auto nan = std::numeric_limits::quiet_NaN(); Require(__isinff_nid_postfix(infinity) == 1 && __isinff_nid_postfix(-infinity) == 1); From fc828e19e76c572e2601759b75fa4645715b10d1 Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Fri, 9 Oct 2026 20:11:24 +0900 Subject: [PATCH 115/460] feat(libc): implement nextafterf Export it next to remainderf, stepping to the adjacent float in the direction of the second argument. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libc/src/Math.cpp | 1 + core/libs/tests/GuestMath.cpp | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/core/libs/prx/libc/src/Math.cpp b/core/libs/prx/libc/src/Math.cpp index 2828d8ed4..e37fed52b 100644 --- a/core/libs/prx/libc/src/Math.cpp +++ b/core/libs/prx/libc/src/Math.cpp @@ -82,6 +82,7 @@ float APS5_VABI roundf_nid_postfix(float x) { return std::round(x); } double APS5_VABI round_nid_postfix(double x) { return std::round(x); } float APS5_VABI cbrtf_nid_postfix(float x) { return std::cbrt(x); } float APS5_VABI remainderf_nid_postfix(float x, float y) { return std::remainder(x, y); } +float APS5_VABI nextafterf_nid_postfix(float x, float y) { return std::nextafter(x, y); } int APS5_VABI __isfinite_nid_postfix(double x) { return std::isfinite(x) ? 1 : 0; } int APS5_VABI __isnan_nid_postfix(double x) { return std::isnan(x) ? 1 : 0; } int APS5_VABI __signbit_nid_postfix(double x) { return std::signbit(x) ? 1 : 0; } diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index ee0b4d20f..d15346a84 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -39,6 +39,7 @@ double APS5_VABI exp2_nid_postfix(double); double APS5_VABI ldexp_nid_postfix(double, int); double APS5_VABI scalbn_nid_postfix(double, int); float APS5_VABI scalbnf_nid_postfix(float, int); +float APS5_VABI nextafterf_nid_postfix(float, float); double APS5_VABI frexp_nid_postfix(double, int*); float APS5_VABI frexpf_nid_postfix(float, int*); std::int64_t APS5_VABI lround_nid_postfix(double); @@ -242,6 +243,13 @@ int main() { Require(ldexp_nid_postfix(0.75, 4) == 12.); Require(scalbn_nid_postfix(0.75, -2) == 0.1875); Require(scalbnf_nid_postfix(0.75f, 4) == 12.f); + Require(nextafterf_nid_postfix(1.f, 2.f) == 0x1.000002p0f && nextafterf_nid_postfix(1.f, 0.f) == 0x1.fffffep-1f); + Require(nextafterf_nid_postfix(0.f, 1.f) == std::numeric_limits::denorm_min()); + Require(nextafterf_nid_postfix(0.f, -1.f) == -std::numeric_limits::denorm_min()); + Require(nextafterf_nid_postfix(std::numeric_limits::denorm_min(), -1.f) == 0.f); + Require(nextafterf_nid_postfix(1.f, 1.f) == 1.f && !std::signbit(nextafterf_nid_postfix(-0.f, 0.f))); + Require(nextafterf_nid_postfix(std::numeric_limits::max(), std::numeric_limits::infinity()) == std::numeric_limits::infinity()); + Require(std::isnan(nextafterf_nid_postfix(std::numeric_limits::quiet_NaN(), 1.f)) && std::isnan(nextafterf_nid_postfix(1.f, std::numeric_limits::quiet_NaN()))); int exponent = 0; Require(frexp_nid_postfix(12., &exponent) == 0.75 && exponent == 4); Require(frexpf_nid_postfix(-12.f, &exponent) == -0.75f && exponent == 4); From 70d490315845028f5321e9c81b133170823de5f6 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 18:28:46 +0000 Subject: [PATCH 116/460] fix(hw-oracle): batch wave64 probes into 512-row dispatches --- tools/hw-oracle/hw_oracle.py | 5 +++-- tools/hw-oracle/test_hw_oracle.py | 19 +++++++++++++++++++ 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/tools/hw-oracle/hw_oracle.py b/tools/hw-oracle/hw_oracle.py index 6cb1c4738..fbecb317e 100644 --- a/tools/hw-oracle/hw_oracle.py +++ b/tools/hw-oracle/hw_oracle.py @@ -89,8 +89,9 @@ def run(body, rows, extra=b"", wave64=False, coarse=False, *, ieee, denorm32, de work = Path(tmp) code = assemble(body, work, wave64, ieee=ieee, denorm32=denorm32, denorm16=denorm16, dx10_clamp=dx10_clamp, round32=round32, round16=round16, fp16_overflow=fp16_overflow, lds=lds) - for start in range(0, len(padded), ROWS_PER_DISPATCH): - chunk = padded[start:start + ROWS_PER_DISPATCH] + rows_per_dispatch = ROWS_PER_DISPATCH // 2 if wave64 else ROWS_PER_DISPATCH + for start in range(0, len(padded), rows_per_dispatch): + chunk = padded[start:start + rows_per_dispatch] (work / "in.bin").write_bytes(b"".join(struct.pack("<4I", *r) for r in chunk) + extra) subprocess.run([oracle(), str(code), str(len(chunk)), str(work / "in.bin"), str(work / "out.bin"), "16"], check=True, env=env) data = (work / "out.bin").read_bytes() diff --git a/tools/hw-oracle/test_hw_oracle.py b/tools/hw-oracle/test_hw_oracle.py index 0b8ad0ea9..606517706 100644 --- a/tools/hw-oracle/test_hw_oracle.py +++ b/tools/hw-oracle/test_hw_oracle.py @@ -123,6 +123,25 @@ class FloatModeTests(unittest.TestCase): class GroupSegmentTests(unittest.TestCase): + def test_wave64_dispatches_at_most_512_rows(self): + rows = [(index, 0, 0, 0) for index in range(1025)] + for wave64, expected in ((False, [1024, 32]), (True, [512, 512, 64])): + with self.subTest(wave64=wave64), tempfile.TemporaryDirectory() as tmp: + dispatch_sizes = [] + + def dispatch(command, *, check, env): + self.assertTrue(check) + dispatch_sizes.append(int(command[2])) + Path(command[4]).write_bytes(bytes(int(command[2]) * 64)) + + with patch.object(hw_oracle, "assemble", return_value=Path(tmp) / "kernel.co"), \ + patch.object(hw_oracle, "oracle", return_value="oracle"), \ + patch.object(hw_oracle.subprocess, "run", side_effect=dispatch): + result = hw_oracle.run("s_nop 0", rows, wave64=wave64, **MODES) + + self.assertEqual(dispatch_sizes, expected) + self.assertEqual(len(result), len(rows)) + def test_cli_forwards_lds(self): with tempfile.TemporaryDirectory() as tmp: body = Path(tmp) / "body.s" From 028b5c0e1f815e1d58ee49d26bc62dfc96246f12 Mon Sep 17 00:00:00 2001 From: ThatsLeo Date: Sat, 10 Oct 2026 18:15:01 +0200 Subject: [PATCH 117/460] fix(agc): drain the device only when the swapchain is recreated Resize drained the device before reading the surface extent, so a drawable size that never matches the surface, as with fractional scaling, drained it on every present. (cherry picked from commit b4aae246766a108f78cd9ad29d7c874ea036d481) --- core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index 65f70f840..51c9352d8 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -1984,7 +1984,6 @@ void VulkanDevice::Resize(std::uint32_t width, std::uint32_t height) { return; } if (state->extent.width == width && state->extent.height == height) return; - WaitIdle(); VkSurfaceCapabilitiesKHR surface{}; check(state->InstanceFunction("vkGetPhysicalDeviceSurfaceCapabilitiesKHR")(state->physical, state->surface, &surface), "vkGetPhysicalDeviceSurfaceCapabilitiesKHR resize"); // The window's drawable size and the surface's extent can disagree for a moment while the window @@ -1993,11 +1992,13 @@ void VulkanDevice::Resize(std::uint32_t width, std::uint32_t height) { width = surface.currentExtent.width; height = surface.currentExtent.height; if (width == 0 || height == 0) { + WaitIdle(); state->extent = {0, 0}; return; } if (state->extent.width == width && state->extent.height == height) return; } + WaitIdle(); require(width >= surface.minImageExtent.width && width <= surface.maxImageExtent.width && height >= surface.minImageExtent.height && height <= surface.maxImageExtent.height, "unsupported resized output extent"); require((surface.supportedUsageFlags & VK_IMAGE_USAGE_TRANSFER_DST_BIT) != 0 && (surface.supportedCompositeAlpha & VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR) != 0 && (surface.supportedTransforms & VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR) != 0, "resized surface capabilities are unsupported"); VkSwapchainCreateInfoKHR create{VK_STRUCTURE_TYPE_SWAPCHAIN_CREATE_INFO_KHR}; From 27353e16b7bddf935fe021dd2506108955554ed7 Mon Sep 17 00:00:00 2001 From: Dean Galvin Date: Sun, 11 Oct 2026 07:32:16 +1300 Subject: [PATCH 118/460] fix(shader): give an emulated comparison of a 2D-array access a plain 2D mode A comparison sample with a 2D-array address on a plain 2D color T# takes the emulated comparison, whose runtime modes were only the 2D-array and cube ones. No mode matched, and materialization threw that the image descriptor was incompatible with the static runtime image interface. The emulated comparison now also gets a 2D mode, as the native modes of a 2D-array access already have; the slice is ignored. --- .../tests/execution/EmulatedColorCompare.cpp | 10 ++++++++-- .../Optimization/src/ResourceMaterializer.cpp | 3 +++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/EmulatedColorCompare.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/EmulatedColorCompare.cpp index c20841723..3e4ca1aac 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/EmulatedColorCompare.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/EmulatedColorCompare.cpp @@ -56,6 +56,11 @@ alignas(256) constexpr std::array Code{ 0x34160082, 0xe0701000, 0x8001040b, 0xbf810000, 0xbf810000, 0xbf810000, }; +alignas(256) constexpr std::array ArrayCode{ + 0x1614008c, 0xe03c1000, 0x8000010a, 0xbf8c3f70, 0xf0bc0128, 0x00820401, 0xbf8c3f70, + 0x34160082, 0xe0701000, 0x8001040b, 0xbf810000, 0xbf810000, 0xbf810000, +}; + alignas(256) constexpr std::array CubeCode{ 0x16140090, 0xe0381000, 0x8000010a, 0xbf8c3f70, 0xf0bc0118, 0x00820401, 0xbf8c3f70, 0x34160082, 0xe0701000, 0x8001040b, 0xbf810000, 0xbf810000, 0xbf810000, @@ -218,9 +223,9 @@ ShaderRecompiler::RecompileResult Compile(AgcDriver::VulkanDevice& device, std:: return ShaderRecompiler::Recompile(request); } -void Run(AgcDriver::VulkanDevice& device, const Sampler& sampler, const char* name, bool offsets = false, bool useCache = false) { +void Run(AgcDriver::VulkanDevice& device, const Sampler& sampler, const char* name, bool offsets = false, bool useCache = false, bool array = false) { Output.fill(-1.0f); - const std::span code = offsets ? std::span(OffsetCode) : std::span(Code); + const std::span code = offsets ? std::span(OffsetCode) : array ? std::span(ArrayCode) : std::span(Code); const auto result = Compile(device, Format8888UNorm, sampler, code, useCache); for (const auto& binding : result.bindings) { Require(binding.role != ShaderRecompiler::DescriptorRole::GuestSamplers, "emulated comparison retained a sampler binding"); @@ -292,6 +297,7 @@ int main() { Run(*device, {ClampEdge, FilterPoint}, "point, clamp to edge"); Run(*device, {ClampWrap, FilterPoint}, "point, wrap"); Run(*device, {ClampWrap, FilterPoint}, "point, wrap with negative offsets", true); + Run(*device, {ClampEdge, FilterBilinear}, "bilinear, 2D array instruction on a 2D texture", false, false, true); Run(*device, {ClampEdge, FilterBilinear}, "bilinear, clamp to edge"); Run(*device, {ClampWrap, FilterBilinear}, "bilinear, wrap"); Run(*device, {ClampBorder, FilterPoint, BorderWhite}, "point, white border"); diff --git a/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp b/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp index f1e3a5545..e091aadf5 100644 --- a/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp +++ b/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp @@ -711,6 +711,9 @@ std::vector ResourceMaterializer::RuntimeImageModes(const ImageRe mode.shaderSwizzle = ShaderImageIdentitySwizzle; modes.push_back(mode); if (image.dimension == RdnaImageDimension::Dim2DArray) { + auto plain = mode; + plain.dimension = RdnaImageDimension::Dim2D; + modes.push_back(plain); mode.cube = true; modes.push_back(mode); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index f50ebe544..ba1d276ed 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -355,6 +355,7 @@ Throughout the project, every function at every stage either **does exactly what - A value read through a null or unmapped pointer still throws at record time if a [branch condition, uniform fill or table base](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp) needs it, or if the pointer comes from user data, even if the console skips the read. - A [V#, T# or S#](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp) loaded through a null or unmapped pointer read from memory is zero when the work is recorded, and the capture throws if no flat SRT slot records that read. - [2D accesses through a 1D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) read the texel the 1D addressing selects and ignore the second coordinate, as simonbbby measured on RDNA2 (gfx1036) in #2237 for `image_sample_lz` and mip-less `image_load` with point filtering and wrap, clamp and border addressing; the PS5's gfx1013 is not measured. Other 2D accesses through a 1D T# (storage, depth, packed, BY loads, mips, derivatives, comparison) still throw. +- [Emulated comparison of a 2D-array access through a 2D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) samples the 2D image and ignores the slice, as the native modes of a 2D-array access through a 2D T# already do. This follows the T# type deciding the addressing; it is not measured on hardware for comparison samples - On a host subgroup narrower than 32 lanes, [`threadBit`](../../core/shader/recompiler/Translation/src/TranslationContext/ExecMask.cpp) reads an EXEC written as a value (`s_mov_b32`/`s_mov_b64`) at bit (subgroup invocation id) of EXEC_LO, so lanes at or above the subgroup size read a lower lane's bit. - [EXEC read as a value](../../core/shader/recompiler/Translation/src/TranslationContext/ExecMask.cpp) (`s_mov_b64 sN, exec`, the `s_*_saveexec` copy, ALU sources) holds only lanes with a host invocation; the console's also holds the wave's other lanes, e.g. 32-63 of a wave64 on 32 host lanes. - [`v_readlane_b32`](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) and fetch-inactive `v_permlane16_b32`/`v_permlanex16_b32` read undefined data from a lane that the host subgroup does not hold or whose host invocation is inactive; the console reads the lane's VGPR. From 696b93a107f55f8ff3d07a5e0ec1d5333363e46e Mon Sep 17 00:00:00 2001 From: Dean Galvin Date: Sun, 11 Oct 2026 07:32:30 +1300 Subject: [PATCH 119/460] fix(shader): drop s_barrier in vertex programs that share nothing between waves An NGG passthrough program on the vertex path whose s_barrier only orders wave 0's GS_ALLOC_REQ produced a workgroup OpControlBarrier, which the Vertex execution model does not allow; SPIR-V validation rejected the module at the program's first draw. In a Vertex-stage program the barrier now emits nothing when the program has no LDS or GDS access and no memory write or atomic, so no wave can observe another. With any of those it throws. --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 14 ++++ .../libSceAgcDriver/tests/VertexBarrier.cpp | 65 +++++++++++++++++++ .../SpirvBackend/SpirvEmitterInstructions.hpp | 2 +- .../SpirvBackend/src/SpirvFlowEmitter.cpp | 18 ++++- docs/dev/TechnicalDebt.md | 1 + 5 files changed, 98 insertions(+), 2 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/VertexBarrier.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index d24c90ab0..0b3761880 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -1365,6 +1365,20 @@ if(BUILD_TESTING) endif() endif() +add_test_executable(agc_vertex_barrier_tests EXCLUDE_FROM_ALL tests/VertexBarrier.cpp) +target_link_libraries(agc_vertex_barrier_tests PRIVATE shader_recompiler_agc_driver libc) +set_target_properties(agc_vertex_barrier_tests PROPERTIES CXX_EXTENSIONS OFF) +configure_windows_unwind(agc_vertex_barrier_tests) +if(BUILD_TESTING) + set_target_properties(agc_vertex_barrier_tests PROPERTIES EXCLUDE_FROM_ALL OFF) + add_test(NAME agc_vertex_barrier COMMAND agc_vertex_barrier_tests) + set_tests_properties(agc_vertex_barrier PROPERTIES ENVIRONMENT "ANYPS5_NO_SHADER_CACHE=1") + if(WIN32) + set_tests_properties(agc_vertex_barrier PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() +endif() + if(BUILD_TESTING) add_test_executable(image_by_tests ${LIBS_INCLUDE_DIR}/tests/ImageBy.cpp) target_link_libraries(image_by_tests PRIVATE shader_recompiler_agc_driver libc) diff --git a/core/libs/prx/libSceAgcDriver/tests/VertexBarrier.cpp b/core/libs/prx/libSceAgcDriver/tests/VertexBarrier.cpp new file mode 100644 index 000000000..8d591fe05 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/VertexBarrier.cpp @@ -0,0 +1,65 @@ +#include "Recompiler.hpp" +#include +#include +#include +#include +#include +#include +#include + +namespace { + +constexpr std::array BarrierExport{0xbf8a0000u, 0xf80008cfu, 0u, 0xbf810000u}; +constexpr std::array BarrierLdsExport{0xd8340000u, 0u, 0xbf8a0000u, 0xf80008cfu, 0u, 0xbf810000u}; + +ShaderRecompiler::RecompileRequest vertexRequest(std::span code) { + using namespace ShaderRecompiler; + RecompileRequest request{}; + request.shader = {ShaderStage::Vertex, 0x10000u, code, 0, {}}; + request.context.waveSize = 32; + request.context.vertex = ShaderVertexStageInfo{}; + request.target.vulkanVersion = 0x00401000u; + request.target.spirvVersion = 0x00010300u; + request.target.subgroupSize = 32; + request.layout.pushConstantSizeBytes = 128; + request.useCache = false; + return request; +} + +bool hasControlBarrier(const ShaderRecompiler::RecompileResult& result) { + const auto& words = result.spirv.Words(); + for (std::size_t at = 5; at < words.size() && (words[at] >> 16u) != 0; at += words[at] >> 16u) { + if (static_cast(words[at] & 0xffffu) == spv::OpControlBarrier) return true; + } + return false; +} + +void verifyBarrierWithoutMemory() { + const auto result = ShaderRecompiler::Recompile(vertexRequest(BarrierExport)); + if (hasControlBarrier(result)) throw std::runtime_error("a vertex program's s_barrier became a workgroup OpControlBarrier, which the Vertex execution model does not allow"); +} + +void verifyBarrierWithLds() { + try { + (void)ShaderRecompiler::Recompile(vertexRequest(BarrierLdsExport)); + } catch (const std::exception& error) { + if (std::string(error.what()).find("s_barrier in a vertex program that accesses LDS or GDS or writes memory is not implemented") != std::string::npos) return; + throw; + } + throw std::runtime_error("a vertex program with s_barrier and an LDS write compiled"); +} + +} // namespace + +int main() { + try { + verifyBarrierWithoutMemory(); + verifyBarrierWithLds(); + std::cout << "vertex barrier tests passed\n"; + return 0; + } catch (const std::exception& error) { + const std::string message(error.what()); + std::cerr << message.substr(0, message.find("RecompileRequest:")) << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterInstructions.hpp b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterInstructions.hpp index 8e275b607..972fe2c94 100644 --- a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterInstructions.hpp +++ b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterInstructions.hpp @@ -59,7 +59,7 @@ std::uint32_t EmitFPSin(SpirvEmitterState& state, std::uint32_t arg0); std::uint32_t EmitFPCos(SpirvEmitterState& state, std::uint32_t arg0); std::uint32_t EmitIdentity(SpirvValueEmitContext& ctx, std::uint32_t value); void EmitVoid(SpirvValueEmitContext& context); -void EmitBarrier(SpirvEmitterState& state); +void EmitBarrier(SpirvValueEmitContext& ctx, const IrValue& inst); std::uint32_t EmitMeshDrawParameter(SpirvValueEmitContext& ctx, const IrValue& inst); std::uint32_t EmitMeshArgument(SpirvValueEmitContext& ctx, const IrValue& inst); std::uint32_t EmitGetTessellationAttribute(SpirvValueEmitContext& ctx, const IrValue& inst); diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp index d4e8eaa77..be1cc8642 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp @@ -776,6 +776,13 @@ void PatchStructuredPhis(SpirvValueEmitContext& ctx, StructuredFunctionState& fu } } +bool CommunicatesThroughMemory(IrOpcode opcode) { + const auto buffer = BufferAccessOf(opcode); + const auto address = AddressOpcodeInfoOf(opcode).access; + const auto image = ImageOpcodeInfoOf(opcode).access; + return SharedAccessOf(opcode) != SharedAccess::None || buffer == BufferAccess::Write || buffer == BufferAccess::Atomic || address == AddressAccess::Write || address == AddressAccess::Atomic || image == ImageAccess::Write || image == ImageAccess::Atomic; +} + } void EmitControlFlow(SpirvModule& module, const IrProgram& program) { @@ -829,7 +836,16 @@ void EmitControlFlow(SpirvValueEmitContext& context, StructuredFunctionState& fu void EmitVoid(SpirvValueEmitContext&) { } -void EmitBarrier(SpirvEmitterState& state) { +void EmitBarrier(SpirvValueEmitContext& ctx, const IrValue& inst) { + auto& state = ctx.state; + if (state.program.Resources().stage == IrShaderStage::Vertex) { + for (const auto* block : state.program.BlockOrder()) { + for (const auto* instruction : block->Instructions()) { + if (CommunicatesThroughMemory(instruction->Opcode())) ctx.Fail(inst, "s_barrier in a vertex program that accesses LDS or GDS or writes memory is not implemented"); + } + } + return; + } const bool tessellation = state.program.Resources().stage == IrShaderStage::TessellationControl; const auto memoryScope = tessellation ? spv::ScopeInvocation : spv::ScopeWorkgroup; const std::uint32_t semantics = tessellation ? static_cast(spv::MemorySemanticsMaskNone) : spv::MemorySemanticsAcquireReleaseMask | LdsMemorySemantics(state); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index f50ebe544..54236ff26 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -117,6 +117,7 @@ Throughout the project, every function at every stage either **does exactly what - [SET_PREDICATION](../../core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp) (AGC driver) - boolean predication follows the AMD PM4 encoding [Mesa](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/common/sid.h) uses: operation 3 reads 64 bits and 4 reads 32 bits, and bit 8 runs predicated packets when the value is non-zero (clear: when it is zero); the bit 12 hint is ignored. The address must have its low 4 bits clear, as `sceAgcDcbSetPredication` writes it. Query predication (Z-pass, primitive count), a predicated COND_EXEC, predicated command buffer chains and flips in predicated command buffers throw - [sceAgcGetGsOversubscription](../../core/libs/prx/libSceAgc/Shader/src/GsOversubscription.cpp) (libSceAgc) - signature and occupancy formula (base and expanded vertex/export capacities, budget shift, `GE_PC_ALLOC` / `SPI_SHADER_PGM_RSRC4_GS` encoding) follow KytyPS5 only, not confirmed on hardware or a title; a negative or NaN factor that gives a negative target, a missing GS context register or a zero `GE_MAX_OUTPUT_PER_SUBGROUP` throws - [Vertex entry merged wave info](../../core/shader/recompiler/Translation/src/InstructionTranslator.cpp) (recompiler) - `s3` of every vertex-path shader is set to `waveSize | waveSize << 8` (vertex and primitive count of a full wave, wave index 0). Layout taken from the prologue Mesa's ACO emits for NGG vertex shaders (`s_bfe_u64 exec` from `s3`), not measured on hardware. The driver gives every graphics program user data from `s8`, so the condition does not tell NGG from legacy vertex shaders: they just never read `s3`. A draw whose last wave is partial still reports a full wave. +- [s_barrier in vertex-path programs](../../core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp) (recompiler) - emits no instruction when the program has no LDS or GDS access and no memory write or atomic, so no wave of the subgroup can observe another; the `GS_ALLOC_REQ` such a barrier orders on NGG passthrough programs is already dropped on the vertex path. The Vertex execution model has no workgroup barrier. Programs with LDS, GDS or memory writes throw. Not measured on hardware - [Zero-stride vertex fetch range](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - for a stride-0 V# with OOB_SELECT 2 the whole fetch reads memory unless NUM_RECORDS is 0; with OOB_SELECT 3 it reads zero when NUM_RECORDS is 0 and throws when the fetch only partly fits. Rules taken from the GFX10 OOB_SELECT table in Mesa [ac_descriptors.c](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/common/ac_descriptors.c), not measured on hardware. Whether OOB_SELECT 3 checks each dword of a multi-dword fetch separately is unknown. OOB_SELECT 0 and 1 keep the structured record checks. - [sceAgcCbCondWrite](../../core/libs/prx/libSceAgc/Cb/src/Branch.cpp) (libSceAgc) - argument order and the 9-dword COND_WRITE layout follow the clean-room [OpenAGC](https://github.com/OpenAGC/OpenAGC/blob/main/src/game_compat.c) and [sharpemu](https://github.com/sharpemu/sharpemu/blob/main/src/SharpEmu.Libs/Agc/AgcExports.CommandPackets.cs), which agree on them; SharpProspero lists another order. The write space is accepted only as 1 (memory), the one value both encode the same way, and other values throw. - [sceAgcGetIsTrinityMode](../../core/libs/prx/libSceAgc/Misc/src/Platform.cpp) (libSceAgc) - signature from the only PPSA26344 call (a pointer to a one-byte flag, result ignored); a null pointer throws because its error code is unknown From 60148c0e9607f6a00b9deba81893d3efbc317ff1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jan=20=C5=A0l=C3=A9gl?= Date: Fri, 9 Oct 2026 21:57:16 +0200 Subject: [PATCH 120/460] fix(libSceSharePlay): accept a null heap in sceSharePlayInitialize PPSA11386 calls sceSharePlayInitialize(NULL, 0x1800) at start-up and stops on any non-zero result, so the console accepts a null heap. The library keeps no state and does not use the heap; a zero size still throws, as its error is unknown. libSceSharePlay gets its own CMakeLists for the test. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceSharePlay/CMakeLists.txt | 15 ++++++++++ core/libs/prx/libSceSharePlay/Export.cpp | 5 ++-- core/libs/tests/GuestSharePlay.cpp | 31 ++++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 50 insertions(+), 2 deletions(-) create mode 100644 core/libs/prx/libSceSharePlay/CMakeLists.txt create mode 100644 core/libs/tests/GuestSharePlay.cpp diff --git a/core/libs/prx/libSceSharePlay/CMakeLists.txt b/core/libs/prx/libSceSharePlay/CMakeLists.txt new file mode 100644 index 000000000..1560813da --- /dev/null +++ b/core/libs/prx/libSceSharePlay/CMakeLists.txt @@ -0,0 +1,15 @@ +add_library(libSceSharePlay SHARED EXCLUDE_FROM_ALL Export.cpp) +target_include_directories(libSceSharePlay PRIVATE ${LIBS_INCLUDE_DIR}) +configure_windows_unwind(libSceSharePlay) + +if(BUILD_TESTING) + add_test_executable(guest_share_play_tests ${LIBS_INCLUDE_DIR}/tests/GuestSharePlay.cpp) + target_include_directories(guest_share_play_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_share_play_tests PRIVATE libSceSharePlay libc) + configure_windows_unwind(guest_share_play_tests) + add_test(NAME guest_share_play COMMAND guest_share_play_tests) + if(WIN32) + set_tests_properties(guest_share_play PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() +endif() diff --git a/core/libs/prx/libSceSharePlay/Export.cpp b/core/libs/prx/libSceSharePlay/Export.cpp index d0289f4bb..7db889918 100644 --- a/core/libs/prx/libSceSharePlay/Export.cpp +++ b/core/libs/prx/libSceSharePlay/Export.cpp @@ -5,8 +5,9 @@ extern "C" { -int APS5_VABI sceSharePlayInitialize(void* heap, size_t heap_size) { - if (heap == nullptr || heap_size == 0) APS5_INVALID_ARG_EX; +int APS5_VABI sceSharePlayInitialize(void* heap, size_t heapSize) { + (void)heap; + if (heapSize == 0) APS5_INVALID_ARG_EX; return 0; } diff --git a/core/libs/tests/GuestSharePlay.cpp b/core/libs/tests/GuestSharePlay.cpp new file mode 100644 index 000000000..5acffe557 --- /dev/null +++ b/core/libs/tests/GuestSharePlay.cpp @@ -0,0 +1,31 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include + +extern "C" { +int APS5_VABI sceSharePlayInitialize(void*, std::size_t); +int APS5_VABI sceSharePlayTerminate(void); +} + +static void Require(bool value) { if (!value) std::abort(); } + +template +static bool Throws(TCall call) { + try { + call(); + } catch (const std::exception&) { + return true; + } + return false; +} + +int main() { + alignas(16) static unsigned char heap[0x1800]; + Require(sceSharePlayInitialize(nullptr, sizeof(heap)) == 0); + Require(sceSharePlayTerminate() == 0); + Require(sceSharePlayInitialize(heap, sizeof(heap)) == 0); + Require(sceSharePlayTerminate() == 0); + Require(Throws([&] { sceSharePlayInitialize(heap, 0); })); + Require(Throws([&] { sceSharePlayInitialize(nullptr, 0); })); +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 6237c108c..f534da6ba 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -270,6 +270,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceShareGetCurrentStatus](../../core/libs/prx/libSceShare/Export.cpp) (libSceShare) - validation (feature flag non-zero, status non-null) and the all-zero 16-byte status taken from the [KytyPS5](https://github.com/KytyPS5/KytyPS5) reimplementation, not confirmed on a PS5 title; the meaning of the recording status values is unknown - [sceShareGetRunningStatus](../../core/libs/prx/libSceShare/Export.cpp) (libSceShare) - only the first 32-bit word of the status is known: PPSA01317 treats bits 0x3c of it as a running share feature and gives the call a 40-byte buffer. The host never records or broadcasts, so it stores 0 in that word and leaves the rest untouched; a null status returns `INVALID_PARAM` by analogy with `sceShareGetCurrentStatus` - [sceShareSetContentParamForApplicationTitle](../../core/libs/prx/libSceShare/Export.cpp) (libSceShare) - imported by PPSA12544's Unity plugins; the signature follows [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/main/src/libs/libShare.cpp), not confirmed on a PS5 title. The application title is only checked for null, as KytyPS5 does, since nothing is captured +- [sceSharePlayInitialize](../../core/libs/prx/libSceSharePlay/Export.cpp) (libSceSharePlay) - the library keeps no state, so the heap is not used. A null heap is accepted: PPSA11386 passes a null heap and 0x1800 bytes and stops on any non-zero result. The error for a zero size is unknown, so it throws - [sceAgcDcbSetZPassPredicationEnableGetSize, sceAgcDcbSetPredicationDisableGetSize, sceAgcDcbSetBoolPredicationEnableGetSize](../../core/libs/prx/libSceAgc/DcbState/src/Predication.cpp) (libSceAgc) - the builders are inline in the SDK and not exported; the size is assumed to be that of the SET_PREDICATION packet `sceAgcDcbSetPredication` writes (16 bytes) - [Resource registration](../../core/libs/prx/libSceAgcDriver/Resource/src/Registration.cpp) (libSceAgcDriver) - registration always fails with `RESOURCE_REGISTRATION_UNAVAILABLE`, so the owner and resource queries, `sceAgcDriverSetResourceUserData`, `sceAgcDriverFindResourcesPublic` and `sceAgcDriverUnregisterAllResourcesForOwner` return the same error without reading their arguments. Their parameter lists follow the libSceGnmDriver counterparts and are unverified. `sceAgcDriverGetResourceRegistrationMaxNameLength` writes 0xfc and returns 0, as [prosper](https://github.com/mattias800/prosper) does from a title's registration configuration; the `uint32_t*` parameter matches the SDK wrapper `sce::Agc::ResourceRegistration::getMaxNameLength(unsigned int*)`. sharpemu writes 256 and KytyPS5 returns `RESOURCE_REGISTRATION_UNAVAILABLE`, so the value is unverified; a null output throws - [sceAgcDriverRegisterMultipleResources](../../core/libs/prx/libSceAgcDriver/Resource/src/Registration.cpp) uses the existing unavailable-registration state and leaves caller memory untouched. Its four arguments (uint32 handle output, uint32 owner, descriptor pointer, uint32 count) follow the original PPSA34502 / 01.010.000 codzm call. The caller builds 40-byte descriptors and explicitly accepts 0x8A6C9018 with an invalid output handle. Enabled registration and console argument-validation precedence are unverified. From 54b30588fd189f26ebf24d348cf7d4c66407758a Mon Sep 17 00:00:00 2001 From: Chad Date: Sat, 10 Oct 2026 13:31:53 -0500 Subject: [PATCH 121/460] fix(recompiler): merge a loop selection whose arm leaves the iteration where its arms rejoin --- .../prx/libSceAgcDriver/tests/ControlFlow.cpp | 33 ++++++++++ .../ControlFlow/src/Structurizer.cpp | 61 +++++++++++++++++++ 2 files changed, 94 insertions(+) diff --git a/core/libs/prx/libSceAgcDriver/tests/ControlFlow.cpp b/core/libs/prx/libSceAgcDriver/tests/ControlFlow.cpp index 14e020f13..9a3d90099 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ControlFlow.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ControlFlow.cpp @@ -371,6 +371,39 @@ latch: buffer_store_dword v1, off, s[0:3], 0 s_endpgm)", Store({0xbe880380u, 0x7e020280u, 0x7d880008u, 0xbf860004u, 0x7d880088u, 0xbf870003u, 0x4a020281u, 0xbf820002u, 0x4a020282u, 0x4a020283u, 0x80089008u, 0xbf0ac008u, 0xbf85fff5u}), Split::Clone}, + {"loop selection arms that leave the iteration rejoin before a join the enclosing selection shares", R"( + v_mov_b32 v1, 0 + s_mov_b32 s8, 0 +loop: + s_cmp_eq_u32 s8, s2 + s_cbranch_scc1 join + v_cmp_gt_u32 vcc, 8, v0 + s_cbranch_vccz arm + v_add_nc_u32 v1, 1, v1 + v_cmp_gt_u32 vcc, 16, v0 + s_cbranch_vccz away +arm: + v_add_nc_u32 v1, 2, v1 + v_cmp_gt_u32 vcc, 4, v0 + s_cbranch_vccz aside +join: + v_add_nc_u32 v1, 3, v1 + s_cmp_lt_u32 s8, 4 + s_cbranch_scc0 done + s_branch latch +away: + v_add_nc_u32 v1, 4, v1 + s_branch latch +aside: + v_add_nc_u32 v1, 5, v1 +latch: + s_add_u32 s8, s8, 1 + s_branch loop +done: + buffer_store_dword v1, off, s[0:3], 0 + s_endpgm)", + Store({0x7e020280u, 0xbe880380u, 0xbf060208u, 0xbf850008u, 0x7d880088u, 0xbf860003u, 0x4a020281u, 0x7d880090u, 0xbf860007u, 0x4a020282u, 0x7d880084u, + 0xbf860006u, 0x4a020283u, 0xbf0a8408u, 0xbf840006u, 0xbf820003u, 0x4a020284u, 0xbf820001u, 0x4a020285u, 0x80088108u, 0xbf82ffedu}), Split::None}, {"continue beside the inner merge of a nested selection in a loop", R"( s_mov_b32 s8, 0 v_mov_b32 v1, 0 diff --git a/core/shader/recompiler/ControlFlow/src/Structurizer.cpp b/core/shader/recompiler/ControlFlow/src/Structurizer.cpp index d86026724..4d26ed9ed 100644 --- a/core/shader/recompiler/ControlFlow/src/Structurizer.cpp +++ b/core/shader/recompiler/ControlFlow/src/Structurizer.cpp @@ -392,6 +392,59 @@ std::uint32_t mergeBesideReturns(const ControlFlowGraph& graph, const BasicBlock return *merge; } +std::uint32_t iterationSelectionMerge(const ControlFlowGraph& graph, const NaturalLoop& loop, const BasicBlock& header, std::uint32_t join) { + const auto count = static_cast(graph.blocks.size()); + const auto escapes = [&](std::uint32_t blockId) { return blockId == loop.continueBlock || blockId == loop.mergeBlock; }; + std::vector joins(count, false); + std::vector pending = {join}; + joins[join] = true; + while (!pending.empty()) { + const auto blockId = pending.back(); + pending.pop_back(); + for (const auto predecessor : graph.FindBlock(blockId).predecessors) { + if (!joins[predecessor] && !escapes(predecessor)) { + joins[predecessor] = true; + pending.push_back(predecessor); + } + } + } + const auto trueTarget = header.terminator.trueBlock; + const auto falseTarget = header.terminator.falseBlock; + if (!joins[trueTarget] || !joins[falseTarget]) { + return InvalidControlFlowId; + } + std::vector> postDominators(count, allBlockIds(count)); + postDominators[join] = {join}; + for (bool changed = true; changed;) { + changed = false; + for (const auto& block : graph.blocks) { + if (!joins[block.id] || block.id == join) { + continue; + } + auto next = allBlockIds(count); + for (const auto successor : block.successors) { + if (joins[successor]) { + next = intersectSorted(next, postDominators[successor]); + } + } + addUnique(next, block.id); + sortUnique(next); + if (next != postDominators[block.id]) { + postDominators[block.id] = std::move(next); + changed = true; + } + } + } + const auto common = intersectSorted(postDominators[trueTarget], postDominators[falseTarget]); + const auto merge = std::find_if(common.begin(), common.end(), [&](std::uint32_t candidate) { + return std::all_of(common.begin(), common.end(), [&](std::uint32_t other) { return contains(postDominators[candidate], other); }); + }); + if (merge == common.end() || *merge == join || *merge == header.id || !graph.Dominates(header.id, *merge) || !isInsideLoopConstruct(graph, loop, *merge)) { + return InvalidControlFlowId; + } + return *merge; +} + std::uint32_t findSelectionMerge(const ControlFlowGraph& graph, const BasicBlock& block) { const auto globalMerge = graph.FindNearestCommonPostDominator(block.terminator.trueBlock, block.terminator.falseBlock); const auto* loop = findInnermostContainingLoop(graph, block.id); @@ -449,6 +502,14 @@ std::uint32_t findSelectionMerge(const ControlFlowGraph& graph, const BasicBlock if (trueJoins != falseJoins) { return trueJoins ? trueTarget : falseTarget; } + const auto escapesAround = [&](std::uint32_t arm) { + return canReachBefore(graph, arm, loop->continueBlock, globalMerge) || canReachBefore(graph, arm, loop->mergeBlock, globalMerge); + }; + if (trueJoins && !graph.Dominates(block.id, globalMerge) && (escapesAround(trueTarget) || escapesAround(falseTarget))) { + if (const auto merge = iterationSelectionMerge(graph, *loop, block, globalMerge); merge != InvalidControlFlowId) { + return merge; + } + } } return globalMerge; } From 71040b6df52cd30bddf8f18722d724e94ffcd281 Mon Sep 17 00:00:00 2001 From: QRMOE Date: Fri, 9 Oct 2026 07:14:12 +0200 Subject: [PATCH 122/460] feat(libSceLibcInternal): implement guest utime AI-assisted: yes. Preserves the guest 64-bit time ABI and forwards through the existing utimes path. Registers guest_utime. --- core/libs/prx/libSceLibcInternal/CMakeLists.txt | 12 +++++++++++- core/libs/prx/libSceLibcInternal/FileTime.cpp | 17 +++++++++++++++++ 2 files changed, 28 insertions(+), 1 deletion(-) create mode 100644 core/libs/prx/libSceLibcInternal/FileTime.cpp diff --git a/core/libs/prx/libSceLibcInternal/CMakeLists.txt b/core/libs/prx/libSceLibcInternal/CMakeLists.txt index 2c0060756..9ece01010 100644 --- a/core/libs/prx/libSceLibcInternal/CMakeLists.txt +++ b/core/libs/prx/libSceLibcInternal/CMakeLists.txt @@ -1,9 +1,19 @@ -add_library(libSceLibcInternal SHARED EXCLUDE_FROM_ALL Export.cpp) +add_library(libSceLibcInternal SHARED EXCLUDE_FROM_ALL Export.cpp FileTime.cpp) target_include_directories(libSceLibcInternal PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(libSceLibcInternal PRIVATE ${CMAKE_DL_LIBS}) configure_windows_unwind(libSceLibcInternal) if(BUILD_TESTING) + add_test_executable(guest_utime_tests ${LIBS_INCLUDE_DIR}/tests/GuestUtime.cpp) + target_include_directories(guest_utime_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_utime_tests PRIVATE libSceLibcInternal libc) + configure_windows_unwind(guest_utime_tests) + add_test(NAME guest_utime COMMAND guest_utime_tests) + set_tests_properties(guest_utime PROPERTIES TIMEOUT 20) + if(WIN32) + set_tests_properties(guest_utime PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_thread_atexit_tests ${LIBS_INCLUDE_DIR}/tests/GuestThreadAtexit.cpp) target_include_directories(guest_thread_atexit_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_thread_atexit_tests PRIVATE libSceLibcInternal libkernel libc) diff --git a/core/libs/prx/libSceLibcInternal/FileTime.cpp b/core/libs/prx/libSceLibcInternal/FileTime.cpp new file mode 100644 index 000000000..8269bfc29 --- /dev/null +++ b/core/libs/prx/libSceLibcInternal/FileTime.cpp @@ -0,0 +1,17 @@ +#include "SceTypes.hpp" +#include "prx/libc/include/general/VabiMacros.hpp" +#include + +struct LibcUtimbuf { + std::int64_t actime; + std::int64_t modtime; +}; +static_assert(sizeof(LibcUtimbuf) == 16); + +extern "C" int APS5_VABI utimes_nid_postfix(const char*, const KernelTimeval*); + +extern "C" int APS5_VABI utime_nid_postfix(const char* path, const LibcUtimbuf* times) { + if (times == nullptr) return utimes_nid_postfix(path, nullptr); + const KernelTimeval values[2] = {{times->actime, 0}, {times->modtime, 0}}; + return utimes_nid_postfix(path, values); +} From 858e9350aa4e3c5a3ef99e46f9dd332f7d3a3ec8 Mon Sep 17 00:00:00 2001 From: QRMOE Date: Fri, 9 Oct 2026 07:14:35 +0200 Subject: [PATCH 123/460] test(libSceLibcInternal): verify guest utime with native timestamps AI-assisted: yes. Checks independent native file times, null/current times, guest errno and rejected paths. --- core/libs/tests/GuestUtime.cpp | 89 ++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) create mode 100644 core/libs/tests/GuestUtime.cpp diff --git a/core/libs/tests/GuestUtime.cpp b/core/libs/tests/GuestUtime.cpp new file mode 100644 index 000000000..27df8f2cb --- /dev/null +++ b/core/libs/tests/GuestUtime.cpp @@ -0,0 +1,89 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include +#include +#ifdef _WIN32 +#include +#else +#include +#endif + +struct LibcUtimbuf { + std::int64_t actime; + std::int64_t modtime; +}; +static_assert(sizeof(LibcUtimbuf) == 16); + +extern "C" { +int APS5_VABI utime_nid_postfix(const char*, const LibcUtimbuf*); +int* APS5_VABI __error_nid_postfix(); +} + +static void Require(bool condition, int line) { + if (!condition) { + std::fprintf(stderr, "Utime check failed at line %d\n", line); + std::abort(); + } +} +#define Check(value) Require((value), __LINE__) + +static LibcUtimbuf ReadTimes(const std::filesystem::path& path, bool wholeSeconds = false) { +#ifdef _WIN32 + const HANDLE file = CreateFileW(path.c_str(), FILE_READ_ATTRIBUTES, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, nullptr, OPEN_EXISTING, 0, nullptr); + Check(file != INVALID_HANDLE_VALUE); + FILETIME access{}, modified{}; + Check(GetFileTime(file, nullptr, &access, &modified)); + Check(CloseHandle(file)); + const auto decode = [wholeSeconds](FILETIME time) { + const auto ticks = (static_cast(time.dwHighDateTime) << 32) | time.dwLowDateTime; + if (wholeSeconds) Check(ticks % 10000000 == 0); + return static_cast(ticks / 10000000) - 11644473600LL; + }; + return {decode(access), decode(modified)}; +#else + struct stat info{}; + Check(::stat(path.c_str(), &info) == 0); + if (wholeSeconds) { +#ifdef __APPLE__ + Check(info.st_atimespec.tv_nsec == 0 && info.st_mtimespec.tv_nsec == 0); +#else + Check(info.st_atim.tv_nsec == 0 && info.st_mtim.tv_nsec == 0); +#endif + } + return {info.st_atime, info.st_mtime}; +#endif +} + +int main() { + const auto root = std::filesystem::path("anyps5-utime-test-" + + std::to_string(std::chrono::steady_clock::now().time_since_epoch().count())); + Check(std::filesystem::create_directory(root)); + const auto file = root / "data.bin"; + { std::ofstream stream(file); stream << "timestamps"; Check(stream.good()); } + const auto path = file.string(); + const LibcUtimbuf times{1000000000LL, 1000000123LL}; + *__error_nid_postfix() = 71; + Check(utime_nid_postfix(path.c_str(), ×) == 0); + Check(*__error_nid_postfix() == 71); + const auto stored = ReadTimes(file, true); + Check(stored.actime == times.actime && stored.modtime == times.modtime); + const auto missing = (root / "missing.bin").string(); + Check(utime_nid_postfix(missing.c_str(), ×) == -1 && *__error_nid_postfix() == 2); + Check(utime_nid_postfix("", ×) == -1 && *__error_nid_postfix() == 2); + Check(utime_nid_postfix(nullptr, ×) == -1 && *__error_nid_postfix() == 14); + Check(utime_nid_postfix(nullptr, nullptr) == -1 && *__error_nid_postfix() == 14); + const auto unchanged = ReadTimes(file, true); + Check(unchanged.actime == times.actime && unchanged.modtime == times.modtime); + const auto before = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now()); + Check(utime_nid_postfix(path.c_str(), nullptr) == 0); + const auto after = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now()); + const auto current = ReadTimes(file); + Check(current.actime >= before - 1 && current.actime <= after + 1); + Check(current.modtime >= before - 1 && current.modtime <= after + 1); + Check(std::filesystem::remove_all(root) == 2); +} From da26c7e86e64e604b6c155e236be2daf79158c9d Mon Sep 17 00:00:00 2001 From: QRMOE Date: Fri, 9 Oct 2026 07:30:16 +0200 Subject: [PATCH 124/460] fix(libSceLibcInternal): preserve errno after successful utime AI-assisted: yes. Restore guest errno only on successful utimes forwarding and retain delegated failure errors. Register the controlled-backend regression test. Windows library build and focused tests pass; Linux rerun pending. --- core/libs/prx/libSceLibcInternal/CMakeLists.txt | 5 +++++ core/libs/prx/libSceLibcInternal/FileTime.cpp | 10 +++++++--- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libSceLibcInternal/CMakeLists.txt b/core/libs/prx/libSceLibcInternal/CMakeLists.txt index 9ece01010..da7eb464b 100644 --- a/core/libs/prx/libSceLibcInternal/CMakeLists.txt +++ b/core/libs/prx/libSceLibcInternal/CMakeLists.txt @@ -4,6 +4,11 @@ target_link_libraries(libSceLibcInternal PRIVATE ${CMAKE_DL_LIBS}) configure_windows_unwind(libSceLibcInternal) if(BUILD_TESTING) + add_test_executable(guest_utime_forwarding_tests ${LIBS_INCLUDE_DIR}/tests/GuestUtimeForwarding.cpp FileTime.cpp) + target_include_directories(guest_utime_forwarding_tests PRIVATE ${LIBS_INCLUDE_DIR}) + configure_windows_unwind(guest_utime_forwarding_tests) + add_test(NAME guest_utime_forwarding COMMAND guest_utime_forwarding_tests) + set_tests_properties(guest_utime_forwarding PROPERTIES TIMEOUT 20) add_test_executable(guest_utime_tests ${LIBS_INCLUDE_DIR}/tests/GuestUtime.cpp) target_include_directories(guest_utime_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_utime_tests PRIVATE libSceLibcInternal libc) diff --git a/core/libs/prx/libSceLibcInternal/FileTime.cpp b/core/libs/prx/libSceLibcInternal/FileTime.cpp index 8269bfc29..a3bbfe8ed 100644 --- a/core/libs/prx/libSceLibcInternal/FileTime.cpp +++ b/core/libs/prx/libSceLibcInternal/FileTime.cpp @@ -9,9 +9,13 @@ struct LibcUtimbuf { static_assert(sizeof(LibcUtimbuf) == 16); extern "C" int APS5_VABI utimes_nid_postfix(const char*, const KernelTimeval*); +extern "C" int* APS5_VABI __error_nid_postfix(); extern "C" int APS5_VABI utime_nid_postfix(const char* path, const LibcUtimbuf* times) { - if (times == nullptr) return utimes_nid_postfix(path, nullptr); - const KernelTimeval values[2] = {{times->actime, 0}, {times->modtime, 0}}; - return utimes_nid_postfix(path, values); + int* const error = __error_nid_postfix(); + const int saved = *error; + const KernelTimeval values[2] = {{times ? times->actime : 0, 0}, {times ? times->modtime : 0, 0}}; + const int result = utimes_nid_postfix(path, times ? values : nullptr); + if (result == 0) *error = saved; + return result; } From 71ff6079cc2f91a5ccf8de08b6ab3f8ff66e5a69 Mon Sep 17 00:00:00 2001 From: QRMOE Date: Fri, 9 Oct 2026 07:30:44 +0200 Subject: [PATCH 125/460] test(libSceLibcInternal): regress successful utime errno changes AI-assisted: yes. Add a deterministic backend that overwrites errno on success; verify restoration on success and retained failure errors. Extend native tests for current-time errno preservation. The new regression fails on original ca87abb and passes with this fix. --- core/libs/tests/GuestUtime.cpp | 6 +++ core/libs/tests/GuestUtimeForwarding.cpp | 65 ++++++++++++++++++++++++ 2 files changed, 71 insertions(+) create mode 100644 core/libs/tests/GuestUtimeForwarding.cpp diff --git a/core/libs/tests/GuestUtime.cpp b/core/libs/tests/GuestUtime.cpp index 27df8f2cb..884cff4e4 100644 --- a/core/libs/tests/GuestUtime.cpp +++ b/core/libs/tests/GuestUtime.cpp @@ -73,14 +73,20 @@ int main() { const auto stored = ReadTimes(file, true); Check(stored.actime == times.actime && stored.modtime == times.modtime); const auto missing = (root / "missing.bin").string(); + *__error_nid_postfix() = 71; Check(utime_nid_postfix(missing.c_str(), ×) == -1 && *__error_nid_postfix() == 2); + *__error_nid_postfix() = 71; Check(utime_nid_postfix("", ×) == -1 && *__error_nid_postfix() == 2); + *__error_nid_postfix() = 71; Check(utime_nid_postfix(nullptr, ×) == -1 && *__error_nid_postfix() == 14); + *__error_nid_postfix() = 71; Check(utime_nid_postfix(nullptr, nullptr) == -1 && *__error_nid_postfix() == 14); const auto unchanged = ReadTimes(file, true); Check(unchanged.actime == times.actime && unchanged.modtime == times.modtime); const auto before = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now()); + *__error_nid_postfix() = 71; Check(utime_nid_postfix(path.c_str(), nullptr) == 0); + Check(*__error_nid_postfix() == 71); const auto after = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now()); const auto current = ReadTimes(file); Check(current.actime >= before - 1 && current.actime <= after + 1); diff --git a/core/libs/tests/GuestUtimeForwarding.cpp b/core/libs/tests/GuestUtimeForwarding.cpp new file mode 100644 index 000000000..870d83d40 --- /dev/null +++ b/core/libs/tests/GuestUtimeForwarding.cpp @@ -0,0 +1,65 @@ +#include "SceTypes.hpp" +#include +#include +#include + +struct LibcUtimbuf { + std::int64_t actime; + std::int64_t modtime; +}; + +extern "C" int APS5_VABI utime_nid_postfix(const char*, const LibcUtimbuf*); + +namespace { +int guestError; +int backendResult; +int backendError; +int calls; +const char* expectedPath; +const LibcUtimbuf* expectedTimes; + +void Require(bool condition, int line) { + if (!condition) { + std::fprintf(stderr, "Utime forwarding check failed at line %d\n", line); + std::abort(); + } +} +} + +#define Check(value) Require((value), __LINE__) + +extern "C" int* APS5_VABI __error_nid_postfix() { + return &guestError; +} + +extern "C" int APS5_VABI utimes_nid_postfix(const char* path, const KernelTimeval* times) { + ++calls; + Check(path == expectedPath); + Check((times == nullptr) == (expectedTimes == nullptr)); + if (expectedTimes) { + Check(times[0].tv_sec == expectedTimes->actime && times[0].tv_usec == 0); + Check(times[1].tv_sec == expectedTimes->modtime && times[1].tv_usec == 0); + } + guestError = backendError; + return backendResult; +} + +int main() { + const char path[] = "/app0/timestamps"; + const LibcUtimbuf times{0x100000001LL, -123}; + for (const LibcUtimbuf* value : {×, static_cast(nullptr)}) { + for (const char* name : {path, static_cast(nullptr)}) { + expectedPath = name; + expectedTimes = value; + for (int result : {0, -1}) { + backendResult = result; + backendError = result == 0 ? 22 : 14; + guestError = 71; + const int previousCalls = calls; + Check(utime_nid_postfix(name, value) == result); + Check(calls == previousCalls + 1); + Check(guestError == (result == 0 ? 71 : 14)); + } + } + } +} From 2ed75efdcc2331eb106a46402e874e4040b1534f Mon Sep 17 00:00:00 2001 From: QRMOE Date: Fri, 9 Oct 2026 07:10:10 +0200 Subject: [PATCH 126/460] feat(libc): implement guest gethostname AI-assisted: yes. Implements the reported homebrew import with FreeBSD buffer behavior and the host hostname. --- core/libs/prx/libc/src/Hostname.cpp | 53 +++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 core/libs/prx/libc/src/Hostname.cpp diff --git a/core/libs/prx/libc/src/Hostname.cpp b/core/libs/prx/libc/src/Hostname.cpp new file mode 100644 index 000000000..a3414e793 --- /dev/null +++ b/core/libs/prx/libc/src/Hostname.cpp @@ -0,0 +1,53 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include +#include +#ifdef _WIN32 +#include +#else +#include +#endif + +namespace { + +std::string Hostname() { +#ifdef _WIN32 + DWORD size = 0; + if (!GetComputerNameExA(ComputerNameDnsHostname, nullptr, &size) && GetLastError() != ERROR_MORE_DATA) + throw std::system_error(static_cast(GetLastError()), std::system_category()); + std::vector buffer(size); + if (!GetComputerNameExA(ComputerNameDnsHostname, buffer.data(), &size)) + throw std::system_error(static_cast(GetLastError()), std::system_category()); + return {buffer.data(), size}; +#else + struct utsname host{}; + if (::uname(&host) != 0) throw std::system_error(errno, std::generic_category()); + return host.nodename; +#endif +} + +} + +extern "C" int APS5_VABI gethostname_nid_postfix(char* name, std::size_t length) { + const int savedError = errno; + try { + const auto host = Hostname(); + if (name == nullptr) { errno = savedError; return 0; } + const auto count = std::min(length, host.size() + 1); + if (count != 0) std::memcpy(name, host.c_str(), count); + if (length <= host.size()) { errno = 63; return -1; } + errno = savedError; + return 0; + } catch (const std::bad_alloc&) { + errno = 12; + return -1; + } catch (const std::system_error& error) { + const auto condition = error.code().default_error_condition(); + errno = condition == std::errc::permission_denied ? 13 : 5; + return -1; + } +} From fb1b5244155c6a9e7c91f12f3693828583094329 Mon Sep 17 00:00:00 2001 From: QRMOE Date: Fri, 9 Oct 2026 07:10:25 +0200 Subject: [PATCH 127/460] build(libc): register guest hostname implementation and test AI-assisted: yes. Builds the implementation and registers guest_hostname with a 20-second timeout. --- core/libs/prx/libc/CMakeLists.txt | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/core/libs/prx/libc/CMakeLists.txt b/core/libs/prx/libc/CMakeLists.txt index 596c12d2b..fe54af6b8 100644 --- a/core/libs/prx/libc/CMakeLists.txt +++ b/core/libs/prx/libc/CMakeLists.txt @@ -31,6 +31,7 @@ target_sources(libc PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src/RuntimeSupport.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/LocaleSupport.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/Process.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/src/Hostname.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/Environment.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/Directory.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/Filesystem.cpp @@ -50,6 +51,16 @@ if(WIN32) endif() if(BUILD_TESTING) + add_test_executable(guest_hostname_tests ${LIBS_INCLUDE_DIR}/tests/GuestHostname.cpp) + target_include_directories(guest_hostname_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_hostname_tests PRIVATE libc) + configure_windows_unwind(guest_hostname_tests) + add_test(NAME guest_hostname COMMAND guest_hostname_tests) + set_tests_properties(guest_hostname PROPERTIES TIMEOUT 20) + if(WIN32) + set_tests_properties(guest_hostname PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_time_conversion_tests ${LIBS_INCLUDE_DIR}/tests/GuestTimeConversion.cpp) target_include_directories(guest_time_conversion_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_time_conversion_tests PRIVATE libc) From 55075d1302e6854a4a16a0839fb1a46a3107061d Mon Sep 17 00:00:00 2001 From: QRMOE Date: Fri, 9 Oct 2026 07:10:57 +0200 Subject: [PATCH 128/460] test(libc): cover hostname bounds and guest errno AI-assisted: yes. Validates native hostname, every boundary capacity, buffer guards, truncation, null queries and thread-local errno. --- core/libs/tests/GuestHostname.cpp | 62 +++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 core/libs/tests/GuestHostname.cpp diff --git a/core/libs/tests/GuestHostname.cpp b/core/libs/tests/GuestHostname.cpp new file mode 100644 index 000000000..bc76129ab --- /dev/null +++ b/core/libs/tests/GuestHostname.cpp @@ -0,0 +1,62 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include +#include +#ifdef _WIN32 +#include +#else +#include +#endif + +extern "C" { +int APS5_VABI gethostname_nid_postfix(char*, std::size_t); +int* APS5_VABI __error_nid_postfix(); +} + +static void Require(bool condition, int line) { + if (!condition) { + std::fprintf(stderr, "Hostname check failed at line %d\n", line); + std::abort(); + } +} +#define Check(value) Require((value), __LINE__) + +int main() { + std::array native{}; +#ifdef _WIN32 + DWORD size = native.size(); + Check(GetComputerNameExA(ComputerNameDnsHostname, native.data(), &size)); +#else + Check(::gethostname(native.data(), native.size()) == 0); +#endif + const std::string expected = native.data(); + Check(!expected.empty()); + for (std::size_t length = 0; length <= expected.size() + 2; ++length) { + std::vector buffer(length + 2, '#'); + *__error_nid_postfix() = 71; + const int result = gethostname_nid_postfix(buffer.data() + 1, length); + Check(buffer.front() == '#' && buffer.back() == '#'); + if (length <= expected.size()) { + Check(result == -1 && *__error_nid_postfix() == 63); + Check(std::memcmp(buffer.data() + 1, expected.data(), length) == 0); + } else { + Check(result == 0 && *__error_nid_postfix() == 71); + Check(std::strcmp(buffer.data() + 1, expected.c_str()) == 0); + for (std::size_t index = expected.size() + 2; index < buffer.size(); ++index) Check(buffer[index] == '#'); + } + } + *__error_nid_postfix() = 22; + Check(gethostname_nid_postfix(nullptr, 0) == 0 && *__error_nid_postfix() == 22); + Check(gethostname_nid_postfix(nullptr, 1024) == 0 && *__error_nid_postfix() == 22); + std::thread worker([] { + char byte = '#'; + *__error_nid_postfix() = 0; + Check(gethostname_nid_postfix(&byte, 0) == -1 && *__error_nid_postfix() == 63 && byte == '#'); + }); + worker.join(); + Check(*__error_nid_postfix() == 22); +} From 8f32b6eca023c28cfdf1ee30d718f4b2445141f8 Mon Sep 17 00:00:00 2001 From: QRMOE Date: Fri, 9 Oct 2026 13:28:49 +0200 Subject: [PATCH 129/460] build(libc): avoid descriptor PR test registration overlap AI-assisted: yes. Moves hostname test registration to avoid #2170 overlap. The full current-main build and focused Windows tests pass; implementation and test source are unchanged. --- core/libs/prx/libc/CMakeLists.txt | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/core/libs/prx/libc/CMakeLists.txt b/core/libs/prx/libc/CMakeLists.txt index fe54af6b8..f7fb8428e 100644 --- a/core/libs/prx/libc/CMakeLists.txt +++ b/core/libs/prx/libc/CMakeLists.txt @@ -51,16 +51,6 @@ if(WIN32) endif() if(BUILD_TESTING) - add_test_executable(guest_hostname_tests ${LIBS_INCLUDE_DIR}/tests/GuestHostname.cpp) - target_include_directories(guest_hostname_tests PRIVATE ${LIBS_INCLUDE_DIR}) - target_link_libraries(guest_hostname_tests PRIVATE libc) - configure_windows_unwind(guest_hostname_tests) - add_test(NAME guest_hostname COMMAND guest_hostname_tests) - set_tests_properties(guest_hostname PROPERTIES TIMEOUT 20) - if(WIN32) - set_tests_properties(guest_hostname PROPERTIES ENVIRONMENT_MODIFICATION - "PATH=path_list_prepend:$") - endif() add_test_executable(guest_time_conversion_tests ${LIBS_INCLUDE_DIR}/tests/GuestTimeConversion.cpp) target_include_directories(guest_time_conversion_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_time_conversion_tests PRIVATE libc) @@ -212,6 +202,16 @@ if(BUILD_TESTING) set_tests_properties(guest_environment PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_hostname_tests ${LIBS_INCLUDE_DIR}/tests/GuestHostname.cpp) + target_include_directories(guest_hostname_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_hostname_tests PRIVATE libc) + configure_windows_unwind(guest_hostname_tests) + add_test(NAME guest_hostname COMMAND guest_hostname_tests) + set_tests_properties(guest_hostname PROPERTIES TIMEOUT 20) + if(WIN32) + set_tests_properties(guest_hostname PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_lifecycle_tests ${LIBS_INCLUDE_DIR}/tests/GuestLifecycle.cpp) target_include_directories(guest_lifecycle_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_lifecycle_tests PRIVATE libc) From 01072bfe9700f964ad70c7c576f36863cb4e238b Mon Sep 17 00:00:00 2001 From: QRMOE Date: Sat, 10 Oct 2026 16:32:37 +0200 Subject: [PATCH 130/460] test(libc): require a stable guest hostname AI-assisted: yes. Verify AnyPS5 rather than the native PC hostname. Cover buffer capacities, guard bytes, maximum size_t capacity, null queries, success errno and thread-local isolation. The old implementation fails this regression; the final fixed tree passes the complete Windows build and five repeated runs each of guest_hostname and guest_errors on main 5edae3536. Linux and title tests of this new revision are pending. --- core/libs/tests/GuestHostname.cpp | 32 +++++++++++++++---------------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/core/libs/tests/GuestHostname.cpp b/core/libs/tests/GuestHostname.cpp index bc76129ab..36e18845d 100644 --- a/core/libs/tests/GuestHostname.cpp +++ b/core/libs/tests/GuestHostname.cpp @@ -3,14 +3,10 @@ #include #include #include -#include +#include +#include #include #include -#ifdef _WIN32 -#include -#else -#include -#endif extern "C" { int APS5_VABI gethostname_nid_postfix(char*, std::size_t); @@ -26,15 +22,7 @@ static void Require(bool condition, int line) { #define Check(value) Require((value), __LINE__) int main() { - std::array native{}; -#ifdef _WIN32 - DWORD size = native.size(); - Check(GetComputerNameExA(ComputerNameDnsHostname, native.data(), &size)); -#else - Check(::gethostname(native.data(), native.size()) == 0); -#endif - const std::string expected = native.data(); - Check(!expected.empty()); + constexpr std::string_view expected = "AnyPS5"; for (std::size_t length = 0; length <= expected.size() + 2; ++length) { std::vector buffer(length + 2, '#'); *__error_nid_postfix() = 71; @@ -45,17 +33,27 @@ int main() { Check(std::memcmp(buffer.data() + 1, expected.data(), length) == 0); } else { Check(result == 0 && *__error_nid_postfix() == 71); - Check(std::strcmp(buffer.data() + 1, expected.c_str()) == 0); + Check(std::strcmp(buffer.data() + 1, expected.data()) == 0); for (std::size_t index = expected.size() + 2; index < buffer.size(); ++index) Check(buffer[index] == '#'); } } + std::array oversized; + oversized.fill('#'); + *__error_nid_postfix() = 71; + Check(gethostname_nid_postfix(oversized.data() + 1, std::numeric_limits::max()) == 0); + Check(*__error_nid_postfix() == 71); + Check(std::strcmp(oversized.data() + 1, expected.data()) == 0); + Check(oversized.front() == '#' && oversized.back() == '#'); *__error_nid_postfix() = 22; Check(gethostname_nid_postfix(nullptr, 0) == 0 && *__error_nid_postfix() == 22); Check(gethostname_nid_postfix(nullptr, 1024) == 0 && *__error_nid_postfix() == 22); - std::thread worker([] { + std::thread worker([expected] { char byte = '#'; *__error_nid_postfix() = 0; Check(gethostname_nid_postfix(&byte, 0) == -1 && *__error_nid_postfix() == 63 && byte == '#'); + std::array name{}; + Check(gethostname_nid_postfix(name.data(), name.size()) == 0); + Check(std::strcmp(name.data(), expected.data()) == 0 && *__error_nid_postfix() == 63); }); worker.join(); Check(*__error_nid_postfix() == 22); From 3eb52310b64b552f882cb9e3932cab78d391c72f Mon Sep 17 00:00:00 2001 From: QRMOE Date: Sat, 10 Oct 2026 16:33:14 +0200 Subject: [PATCH 131/460] fix(libc): keep the guest hostname independent of the host AI-assisted: yes. Return AnyPS5 without reading the native machine name, hostname environment or network configuration. Preserve FreeBSD bounded-copy behavior, guest ENAMETOOLONG, null queries and success errno. The final tree passes a complete Windows Release build and guest_hostname/guest_errors repeated five times each on main 5edae3536. No new Linux or title test is claimed. --- core/libs/prx/libc/src/Hostname.cpp | 53 +++++------------------------ 1 file changed, 8 insertions(+), 45 deletions(-) diff --git a/core/libs/prx/libc/src/Hostname.cpp b/core/libs/prx/libc/src/Hostname.cpp index a3414e793..7b205e365 100644 --- a/core/libs/prx/libc/src/Hostname.cpp +++ b/core/libs/prx/libc/src/Hostname.cpp @@ -1,53 +1,16 @@ #include "prx/libc/include/general/VabiMacros.hpp" #include #include +#include #include -#include -#include -#include -#include -#ifdef _WIN32 -#include -#else -#include -#endif - -namespace { - -std::string Hostname() { -#ifdef _WIN32 - DWORD size = 0; - if (!GetComputerNameExA(ComputerNameDnsHostname, nullptr, &size) && GetLastError() != ERROR_MORE_DATA) - throw std::system_error(static_cast(GetLastError()), std::system_category()); - std::vector buffer(size); - if (!GetComputerNameExA(ComputerNameDnsHostname, buffer.data(), &size)) - throw std::system_error(static_cast(GetLastError()), std::system_category()); - return {buffer.data(), size}; -#else - struct utsname host{}; - if (::uname(&host) != 0) throw std::system_error(errno, std::generic_category()); - return host.nodename; -#endif -} - -} extern "C" int APS5_VABI gethostname_nid_postfix(char* name, std::size_t length) { + constexpr char hostname[] = "AnyPS5"; + if (name == nullptr) return 0; const int savedError = errno; - try { - const auto host = Hostname(); - if (name == nullptr) { errno = savedError; return 0; } - const auto count = std::min(length, host.size() + 1); - if (count != 0) std::memcpy(name, host.c_str(), count); - if (length <= host.size()) { errno = 63; return -1; } - errno = savedError; - return 0; - } catch (const std::bad_alloc&) { - errno = 12; - return -1; - } catch (const std::system_error& error) { - const auto condition = error.code().default_error_condition(); - errno = condition == std::errc::permission_denied ? 13 : 5; - return -1; - } + const auto count = std::min(length, sizeof(hostname)); + if (count != 0) std::memcpy(name, hostname, count); + if (length < sizeof(hostname)) { errno = 63; return -1; } + errno = savedError; + return 0; } From f153c28b52f8a3ad659118d3c12722c1f46a17de Mon Sep 17 00:00:00 2001 From: QRMOE Date: Sat, 10 Oct 2026 16:33:38 +0200 Subject: [PATCH 132/460] docs(libc): record the fixed guest hostname policy AI-assisted: yes. Record that gethostname exposes AnyPS5 rather than the native machine name. Console hostname configuration, sysctl integration and name resolution remain unverified and unimplemented. This one-line note is placed beside an unchanged libc compatibility entry so the branch merges cleanly into tested main 5edae3536. --- docs/dev/TechnicalDebt.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index f50ebe544..4d6bda1a5 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -400,6 +400,7 @@ Throughout the project, every function at every stage either **does exactly what - The Linux placement of the `--to-intel` stubs in the [ELF patcher](../../core/relinker/elfpatcher/src/linux/LinuxElfPatcher.cpp) is covered only by a synthetic test. - The libc [SSE4a trap emulation](../../core/libs/prx/libc/src/specifics/windows/Sse4aEmulation.hpp) on Windows is superseded by `--to-intel` and remains only until the relinked title has been verified without it. - [getifaddrs](../../core/libs/prx/libc/src/Interfaces.cpp) (libc) lists the host's IPv4 and IPv6 addresses in FreeBSD's `ifaddrs` and sockaddr layouts. FreeBSD's per-interface AF_LINK entries (MAC, index, MTU) and their `if_data` are not produced, so interfaces without an address are omitted. Names are the host's (Linux `lo`, `eth0`; Windows NDIS names such as `ethernet_32769`, or `if` when one does not fit 15 bytes), not the console's. A Linux point-to-point address without a peer loses `IFF_POINTOPOINT`, and an IPv4 broadcast address the host omits is computed from the netmask. On Windows the flags are only `IFF_UP`, `IFF_RUNNING`, `IFF_LOOPBACK`, `IFF_BROADCAST` and `IFF_MULTICAST`, derived from the adapter: PPP and tunnel adapters get neither `IFF_BROADCAST` nor `IFF_POINTOPOINT` because their peer address is not reported, and an address whose prefix length the host reports as unspecified gets a host mask. +- [gethostname](../../core/libs/prx/libc/src/Hostname.cpp) (libc) reports the fixed guest name `AnyPS5` without querying the host machine name. Buffer copying, null queries and `ENAMETOOLONG` follow [FreeBSD gethostname](https://github.com/freebsd/freebsd-src/blob/main/lib/libc/gen/gethostname.c); the console's configurable hostname and its integration with `sysctl` and name resolution are unverified and not implemented. - [Descriptor pool chain](../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp) (libSceAgcDriver) - each pool holds two storage buffer descriptors more than a set can take, as a workaround: RADV (Mesa 26.0 and later) lets a `FREE_DESCRIPTOR_SET` pool place a set up to 32 bytes past the pool's buffer ([`util_vma_heap_init`](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/vulkan/radv_descriptor_pool.c) is given `bo_size + RADV_POOL_HEAP_OFFSET` as a size), and with a page-multiple pool size those bytes are an unmapped page - [CMASK fast clears](../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver) - each fast-cleared CMASK tile (code 0) is stored as `CB_COLOR_CLEAR_WORD` into its 8x8 texels at the first draw into the target or at its fast-clear eliminate or DCC decompress pass, not as the color block writes the tile, and the CMASK is then stored as expanded (0xF): a texture or copy read of the target after that draw and before the title's eliminate sees the clear value in tiles no draw wrote, where the console keeps their previous bytes. Tile addresses follow the pipe-aligned SW_64KB_Z_X CMASK equation for 16 pipes; other CMASK swizzle modes, codes other than 0 and 0xF, a cleared tile of a DCC target and a fast clear of a mipmapped, 3D or array color target throw. When the clear runs as a compute pass (a resident target whose CMASK is host-imported), an invalid code throws at the next fast-clear materialization, not at the one that read it - [HTILE depth fast clears](../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver) - HTILE is not modeled. Only RADV's `clear_htile_mask` compute kernel (17 GFX10 words, 64x1x1 groups) is recognized, and only when it leaves ZMASK at 0 (`user[4]` and `user[5]` low nibbles both 0); the dispatch still runs, and the host depth image is cleared to `DB_DEPTH_CLEAR` the next time the surface with that `DB_HTILE_DATA_BASE` address is bound. The address match is exact, so a clear of a different HTILE range does not hit. HTILE initialization, the stencil-only update, and fast clears made by the SDK's own paths, CP DMA fills or other kernels are not covered and leave the depth image as it was. The kernel words are the ones the PR author saw RADV submit; they are not compared with Mesa's source. From 5a43c9b0b99936177e9a57c2552e94804699fbff Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Fri, 9 Oct 2026 00:13:21 -0400 Subject: [PATCH 133/460] feat(libkernel): implement pthread_rwlockattr_init and pthread_rwlockattr_destroy --- core/libs/prx/libkernel/Pthread/Posix/Rwlock.cpp | 8 ++++++++ core/libs/tests/GuestPthreadRwlock.cpp | 14 ++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/core/libs/prx/libkernel/Pthread/Posix/Rwlock.cpp b/core/libs/prx/libkernel/Pthread/Posix/Rwlock.cpp index 9a79b5ecb..b64d1359b 100644 --- a/core/libs/prx/libkernel/Pthread/Posix/Rwlock.cpp +++ b/core/libs/prx/libkernel/Pthread/Posix/Rwlock.cpp @@ -46,6 +46,14 @@ int APS5_VABI pthread_rwlock_init_nid_postfix(PthreadRwlock* rwlock, const Pthre return toPosix(scePthreadRwlockInit(rwlock, attr, nullptr)); } +int APS5_VABI pthread_rwlockattr_destroy_nid_postfix(PthreadRwlockattr* attr) { + return toPosix(scePthreadRwlockattrDestroy(attr)); +} + +int APS5_VABI pthread_rwlockattr_init_nid_postfix(PthreadRwlockattr* attr) { + return toPosix(scePthreadRwlockattrInit(attr)); +} + int APS5_VABI pthread_rwlock_rdlock_nid_postfix(PthreadRwlock* rwlock) { initializeStatic(rwlock, __func__); return toPosix(scePthreadRwlockRdlock(rwlock)); diff --git a/core/libs/tests/GuestPthreadRwlock.cpp b/core/libs/tests/GuestPthreadRwlock.cpp index 00f15cf97..d19319b86 100644 --- a/core/libs/tests/GuestPthreadRwlock.cpp +++ b/core/libs/tests/GuestPthreadRwlock.cpp @@ -11,6 +11,9 @@ int APS5_VABI scePthreadCreate(Pthread* thread, const PthreadAttr* attr, Pthread int APS5_VABI scePthreadJoin(Pthread thread, void** retval); int APS5_VABI clock_gettime_nid_postfix(int clockId, KernelTimespec* tp); int APS5_VABI pthread_rwlock_destroy_nid_postfix(PthreadRwlock* rwlock); +int APS5_VABI pthread_rwlock_init_nid_postfix(PthreadRwlock* rwlock, const PthreadRwlockattr* attr); +int APS5_VABI pthread_rwlockattr_init_nid_postfix(PthreadRwlockattr* attr); +int APS5_VABI pthread_rwlockattr_destroy_nid_postfix(PthreadRwlockattr* attr); int APS5_VABI pthread_rwlock_rdlock_nid_postfix(PthreadRwlock* rwlock); int APS5_VABI pthread_rwlock_wrlock_nid_postfix(PthreadRwlock* rwlock); int APS5_VABI pthread_rwlock_unlock_nid_postfix(PthreadRwlock* rwlock); @@ -124,6 +127,17 @@ int main() { Require(pthread_rwlock_destroy_nid_postfix(&lock) == 0); } + PthreadRwlockattr attr = nullptr; + Require(pthread_rwlockattr_init_nid_postfix(&attr) == 0); + Require(attr != nullptr); + PthreadRwlock withAttr = nullptr; + Require(pthread_rwlock_init_nid_postfix(&withAttr, &attr) == 0); + Require(pthread_rwlock_wrlock_nid_postfix(&withAttr) == 0); + Require(pthread_rwlock_tryrdlock_nid_postfix(&withAttr) == GUEST_EBUSY); + Require(pthread_rwlock_unlock_nid_postfix(&withAttr) == 0); + Require(pthread_rwlock_destroy_nid_postfix(&withAttr) == 0); + Require(pthread_rwlockattr_destroy_nid_postfix(&attr) == 0); + PthreadRwlock rwlock = nullptr; Require(pthread_rwlock_trywrlock_nid_postfix(&rwlock) == 0); Require(pthread_rwlock_tryrdlock_nid_postfix(&rwlock) == GUEST_EBUSY); From 8da88ad3ac2d179d7b5628ef14eac1f1dee26f79 Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Sat, 10 Oct 2026 13:38:04 -0400 Subject: [PATCH 134/460] feat(libSceAudioOut): export the sceAudioOutSys HDMI output functions --- core/libs/prx/libSceAudioOut/src/AudioOut.cpp | 20 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 2 files changed, 21 insertions(+) diff --git a/core/libs/prx/libSceAudioOut/src/AudioOut.cpp b/core/libs/prx/libSceAudioOut/src/AudioOut.cpp index 5dcdd2821..1f34de0c8 100644 --- a/core/libs/prx/libSceAudioOut/src/AudioOut.cpp +++ b/core/libs/prx/libSceAudioOut/src/AudioOut.cpp @@ -496,6 +496,26 @@ int APS5_VABI sceAudioOutGetPortState(int handle, AudioOutPortState* state) { return 0; } +int APS5_VABI sceAudioOutSysClose() { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceAudioOutSysConfigureOutput() { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceAudioOutSysGetHdmiMonitorInfo() { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceAudioOutSysOpen() { + NotImplemented_nid_no_patch(__func__); + return 0; +} + int APS5_VABI sceAudioOutSetMixLevelPadSpk(int handle, int mixLevel) { std::lock_guard lock(g_mutex); Port* port = getPort(handle); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 4656d0fbb..c485fb3f1 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -167,6 +167,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceAudioOutSetMixLevelPadSpk](../../core/libs/prx/libSceAudioOut/src/AudioOut.cpp) (libSceAudioOut) - effect of a negative mix level unknown; only 0 to 32768 (0 dB) is accepted - [sceAudioOutSetMixLevelPadSpk](../../core/libs/prx/libSceAudioOut/src/AudioOut.cpp) (libSceAudioOut) - no source applies the level (shadPS4 only stores it); it is assumed to be a linear gain on the port's own output, and the -9 dB default (11626) is applied to every pad speaker port, including those of titles that never call the function - [sceAudioOutGetLastOutputTime](../../core/libs/prx/libSceAudioOut/src/AudioOut.cpp) (libSceAudioOut) - signature and behaviour taken from the [shadPS4](https://github.com/shadps4-emu/shadPS4) and [fpPS4](https://github.com/red-prig/fpPS4) reimplementations of the PS4 library, not confirmed on a PS5 title: the time is the process time at which `sceAudioOutOutput` or `sceAudioOutOutputs` accepted the port's last block, not the time the block reached the device +- [sceAudioOutSysClose, sceAudioOutSysConfigureOutput, sceAudioOutSysGetHdmiMonitorInfo, sceAudioOutSysOpen](../../core/libs/prx/libSceAudioOut/src/AudioOut.cpp) (libSceAudioOut) - imported by PPSA99051 and PPSA99997, which fail to load without them; added without an implementation and declared without parameters. [EVO-PLAYER-PS5](https://github.com/sainsaji/EVO-PLAYER-PS5/blob/da0c0451333bab82e79fffd2de4718e917f9dcb0/projects/evoplayer/core/src/audio/evo_pt.cpp#L166-L240), whose audio code both titles share, calls `sceAudioOutSysGetHdmiMonitorInfo` to read the formats the TV or receiver lists ([bitstream.cpp](https://github.com/sainsaji/EVO-PLAYER-PS5/blob/da0c0451333bab82e79fffd2de4718e917f9dcb0/projects/evoplayer/core/src/audio/bitstream.cpp#L71)) and `sceAudioOutSysOpen`, `sceAudioOutSysConfigureOutput` and `sceAudioOutSysClose` only to open a port for HDMI bitstream passthrough. Its declared arguments are not confirmed on a console. Audio reaches the host device through SDL as PCM, so there is no HDMI sink to describe or switch - [sceAudioOut2PortCreate](../../core/libs/prx/libSceAudioOut/src/AudioOut2Port.cpp) (libSceAudioOut) - MAIN (type 0) format validation follows PS5 firmware 13.60 measurements: bits 8-11 accept 1, 2, 8 or 12, bits 0-6 accept 0 or 1, bit 7 requires 8 channels, and bits 12-31 do not affect validation. The sample types are still read as float and int16; bit 7's layout, the PCM effect of bits 12-31 and the port `flags` are unverified. Other port types retain unverified format validation, including 6 channels and rejection of bits 12-31 - [sceAudioOut2PortCreate](../../core/libs/prx/libSceAudioOut/src/AudioOut2Port.cpp) (libSceAudioOut) - 12 channels are assumed to be L R C LFE Ls Rs Lb Rb Ltf Rtf Ltb Rtb as in KytyPS5's 12-to-8 fold, and 6 and 8 channels to be its first 6 and 8; no title or SDK header confirms the order - [sceAudioOut2PortCreate](../../core/libs/prx/libSceAudioOut/src/AudioOut2Port.cpp) (libSceAudioOut) - the stereo fold is assumed, not the console's downmix: C at -3 dB into both sides, Ls/Rs, Lb/Rb and Ltf/Rtf at -3 dB and Ltb/Rtb at -6 dB into their own side, LFE dropped From bddcc71ea92349dbaad647a4af91f3230d65a924 Mon Sep 17 00:00:00 2001 From: JAB-dev <68183649+JAB-dev@users.noreply.github.com> Date: Sat, 10 Oct 2026 21:01:34 +0200 Subject: [PATCH 135/460] test(agc): require a repeated draw to reuse the cached draw Submit the same vertex and pixel draw twice through the driver and require the second one to hit the draw cache, with graphics pipeline libraries and with APS5_NO_GPL=1. With pipeline libraries the pixel stage is cached at its fixed push slot, so a lookup at the stacked offset misses every draw on its layout. The driver exposes a copy of its draw-cache counters for the test. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 6 + .../Execution/include/Driver/Driver.hpp | 1 + .../src/Driver/Draw/DrawCacheStatistics.cpp | 5 + .../tests/execution/DrawCacheReuse.cpp | 124 ++++++++++++++++++ 4 files changed, 136 insertions(+) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/DrawCacheReuse.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index d24c90ab0..09a510358 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -942,6 +942,12 @@ if(BUILD_TESTING) set_tests_properties(agc_driver_flip_compute PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test(NAME agc_driver_draw_cache_reuse_off COMMAND agc_driver_draw_cache_reuse_tests) + set_tests_properties(agc_driver_draw_cache_reuse_off PROPERTIES ENVIRONMENT "ANYPS5_NO_SHADER_CACHE=1;APS5_NO_GPL=1" SKIP_RETURN_CODE 77) + if(WIN32) + set_tests_properties(agc_driver_draw_cache_reuse_off PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() foreach(agcTest IN ITEMS agc_spirv_narrow_constant_store_tests agc_spirv_optimizer_tests agc_driver_constant_store_tests) if(NOT TARGET ${agcTest}) diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp index dd91c8cc1..becdafd93 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp @@ -63,6 +63,7 @@ public: static void FoldDrawOffsets(const ShaderRecompiler::RecompileResult& result, const DrawProgram& program, Pm4::DrawParameters& parameters); static std::optional ClassifyIndirectDraw(const ShaderRecompiler::RecompileResult& result, const Graphics::State& graphics, const DrawProgram& frontProgram, const std::shared_ptr& localDevice, Pm4::DrawParameters& drawParameters, bool traceIndirect); static std::string dumpRequest(std::uint64_t address, const ShaderRecompiler::RecompileRequest& request); + DrawEntryCounters DrawCacheCounters(); private: friend class SampledReadScope; diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawCacheStatistics.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawCacheStatistics.cpp index 9d82d8550..94d63e829 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawCacheStatistics.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawCacheStatistics.cpp @@ -18,4 +18,9 @@ void Driver::reportDrawCache(DrawEntryCounters& counters) { counters = DrawEntryCounters{}; } +DrawEntryCounters Driver::DrawCacheCounters() { + std::lock_guard cacheLock(drawCacheMutex); + return drawEntryCounters; +} + } diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/DrawCacheReuse.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/DrawCacheReuse.cpp new file mode 100644 index 000000000..93c8237c2 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/DrawCacheReuse.cpp @@ -0,0 +1,124 @@ +#include "prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp" +#include "prx/libSceAgcDriver/Submit/include/Dcb.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using AgcDriver::DriverDetail::DrawMiss; + +alignas(256) constexpr std::array VertexCode{0x7e000208u, 0x7e020280u, 0x7e040280u, 0x7e0602f2u, 0xf80008cfu, 0x03020100u, 0xbf810000u, 0xbf810000u}; +alignas(256) constexpr std::array PixelCode{0x7e000200u, 0x7e020280u, 0xf800180fu, 0x00010100u, 0xbf810000u, 0xbf810000u, 0xbf810000u, 0xbf810000u}; +alignas(256) std::array ColorMemory{}; + +struct Header { + Shader shader{}; + ShaderUserData userData{}; +}; + +alignas(8) Header VertexHeader{}; +alignas(8) Header PixelHeader{}; + +const Shader* Registered(Header& header, const std::uint32_t* code, std::size_t bytes, std::uint8_t type) { + header.shader.file_header = 0x34333231u; + header.shader.version = 0x18u; + header.shader.header_size = sizeof(Header); + header.shader.shader_size = static_cast(bytes); + header.shader.code = code; + header.shader.type = type; + header.shader.user_data = &header.userData; + AgcDriverRegisterShader_nid_postfix(&header.shader); + return &header.shader; +} + +void Append(std::vector& commands, std::uint32_t opcode, std::initializer_list payload) { + commands.push_back(0xc0000000u | (static_cast(payload.size() - 1) << 16u) | (opcode << 8u)); + commands.insert(commands.end(), payload); +} + +std::vector DrawTwice() { + const auto color = reinterpret_cast(ColorMemory.data()); + const auto vertex = reinterpret_cast(VertexCode.data()); + const auto pixel = reinterpret_cast(PixelCode.data()); + const std::vector> context{ + {0x2d5, 0x2000}, + {0x1b6, 0}, {0x207, 0}, {0x200, 0}, {0x203, 0x800}, + {0x2dc, 0xaa00}, {0x2f8, 0}, {0x292, 2}, {0x293, 0}, + {0x80, 0}, {0x8d, 0}, {0x83, 0xffff}, {0x8c, 0xa}, + {0x2f9, 0x2d}, {0x313, 0x6000}, {0x30e, 0xffffffff}, {0x30f, 0xffffffff}, + {0x206, 0x43f}, {0x204, 0x80000}, {0x205, 0x240}, + {0x8e, 0xf}, {0x8f, 0xf}, {0x202, 0xcc0010}, + {0x1c4, 0}, {0x1c5, 9}, {0x1c3, 4}, {0x31c, 0x28028}, + {0x31b, 0}, {0x31d, 0}, {0x3b0, (63u << 14u) | 3u}, + {0x3b8, 0x9000000}, {0x1e0, 0}, + {0xc, 0}, {0xd, 0x40040}, + {0x81, 0x80000000}, {0x82, 0x40040}, + {0x90, 0x80000000}, {0x91, 0x40040}, + {0x94, 0x80000000}, {0x95, 0x40040}, + {0x318, static_cast(color >> 8u)}, {0x390, static_cast(color >> 40u)}, + {0x10f, std::bit_cast(32.0f)}, {0x110, std::bit_cast(32.0f)}, + {0x111, std::bit_cast(-2.0f)}, {0x112, std::bit_cast(2.0f)}, + {0x113, std::bit_cast(1.0f)}, {0x114, 0}, + {0xb4, 0}, {0xb5, std::bit_cast(1.0f)}, + }; + const std::vector> shader{ + {0xc8, static_cast(vertex >> 8u)}, {0xc9, static_cast(vertex >> 40u)}, {0x8b, 1u << 1u}, {0x8c, 0}, + {0x008, static_cast(pixel >> 8u)}, {0x009, static_cast(pixel >> 40u)}, {0x00b, 1u << 1u}, {0x00c, std::bit_cast(1.0f)}, + }; + std::vector commands; + Append(commands, 0x79, {0x242, 4}); + for (const auto& [offset, value] : context) Append(commands, 0x69, {offset, value}); + for (const auto& [offset, value] : shader) Append(commands, 0x76, {offset, value}); + Append(commands, 0x2d, {3, 2}); + Append(commands, 0x2d, {3, 2}); + return commands; +} + +void SecondDrawHitsTheCache(bool libraries) { + Registered(VertexHeader, VertexCode.data(), sizeof(VertexCode), 2); + Registered(PixelHeader, PixelCode.data(), sizeof(PixelCode), 1); + auto commands = DrawTwice(); + Packet packet{commands.data(), static_cast(commands.size()), 0, {}}; + Require(sceAgcDriverSubmitDcb(&packet) == 0, "the draw submission failed"); + AgcDriverWaitIdle_nid_postfix(); + const auto counters = AgcDriver::DriverDetail::Driver::Get().DrawCacheCounters(); + const auto mode = std::string(libraries ? "with" : "without") + " graphics pipeline libraries"; + const auto layout = counters.misses[static_cast(DrawMiss::Layout)]; + Require(counters.lookups == 2 && counters.absent == 1, mode + ": the two draws made " + std::to_string(counters.lookups) + " draw-cache lookups (" + std::to_string(counters.absent) + " without an entry), expected 2 lookups with 1 absent"); + Require(layout == 0, mode + ": the repeated draw missed the draw cache on its push offset (" + std::to_string(layout) + " layout misses)"); + Require(counters.hits == 1, mode + ": the repeated draw did not reuse the cached draw (" + std::to_string(counters.hits) + " hits)"); +} + +} + +int main() { + try { + bool libraries = false; + { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + libraries = device->GraphicsPipelineLibraries(); + } + if (!libraries && std::getenv("APS5_NO_GPL") == nullptr) return VulkanTestSkipped; + SecondDrawHitsTheCache(libraries); + AgcDriverShutdown_nid_postfix(); + std::cout << "A repeated draw reused the cached draw " << (libraries ? "with" : "without") << " graphics pipeline libraries\n"; + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + AgcDriverShutdown_nid_postfix(); + return 1; + } +} From 02941b8e3f719fbcacd672f5c48074c3de5ebb07 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 19:24:18 +0000 Subject: [PATCH 136/460] fix(tools): detect non-void stub wrappers --- tools/progress.py | 2 +- tools/tests/test_progress.py | 11 +++++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/tools/progress.py b/tools/progress.py index 505c684d7..1b32f8db1 100644 --- a/tools/progress.py +++ b/tools/progress.py @@ -15,7 +15,7 @@ DEFINITION = re.compile(r"\bAPS5_VABI\s+(\w+)\s*\([^;{]*\)\s*(?:noexcept\s*)?(?: STUB = "NotImplemented_nid_no_patch" CMAKE_SHARED = re.compile(r'^\s*include\(\s*"?\$\{CMAKE_CURRENT_SOURCE_DIR\}/\.\./([\w.-]+)/[\w.-]+\.cmake"?\s*\)', re.M) SOURCE_SHARED = re.compile(r'^\s*#\s*include\s*"prx/([\w.-]+)/[\w./-]+\.cpp"', re.M) -STUB_WRAPPER = re.compile(r"\bstatic\s+(?:\[\[noreturn\]\]\s+)?void\s+(\w+)\s*\([^;{]*\)\s*\{") +STUB_WRAPPER = re.compile(r"\bstatic\s+(?:\[\[noreturn\]\]\s+)?[\w:<>,\s*&]+?\s+(\w+)\s*\([^;{]*\)\s*\{") FLAT_SEGMENTS = ("GLOBAL_", "SCRATCH_") OPCODE_SENTINELS = {"Invalid", "Count", "Unknown", "Unsupported"} OPCODE_ALIASES = { diff --git a/tools/tests/test_progress.py b/tools/tests/test_progress.py index ddefe767e..dfff23f69 100644 --- a/tools/tests/test_progress.py +++ b/tools/tests/test_progress.py @@ -9,6 +9,17 @@ import progress class ProgressTests(unittest.TestCase): + def test_nonvoid_stub_wrappers_keep_callers_unimplemented(self): + with tempfile.TemporaryDirectory() as directory: + library = Path(directory) / "libSceExample" + library.mkdir() + (library / "Export.cpp").write_text( + "static int SupportFormat() { NotImplemented_nid_no_patch(__func__); return 0; }\n" + "int APS5_VABI sceExampleSupportFormat() { return SupportFormat(); }\n") + group = progress.scan_library(library) + self.assertEqual(group["done_names"], []) + self.assertEqual(group["todo_names"], ["sceExampleSupportFormat"]) + def test_test_sources_do_not_change_library_progress(self): with tempfile.TemporaryDirectory() as directory: root = Path(directory) / "tests" / "prx" From fb380440d76dcc04fba4a697384ebdf0fb21483c Mon Sep 17 00:00:00 2001 From: Get-fix <100056769+Get-fix@users.noreply.github.com> Date: Sat, 10 Oct 2026 16:44:26 +0300 Subject: [PATCH 137/460] feat(libSceSystemService): select the console language with ANYPS5_LANGUAGE sceSystemServiceParamGetInt always reported English (United States) as the console language, so a title that follows the console's language, such as Jelly5 (PPSA99505) with its Arabic text, can only show English. ANYPS5_LANGUAGE now selects a console language number from 0 to 30 (21 is Arabic). Unset or empty keeps English (United States); any other value throws at the language query instead of being ignored. (cherry picked from commit 9bcf196daa01b07a22be876a7f1143149503e054) --- .../prx/libSceSystemService/CMakeLists.txt | 9 +++ core/libs/prx/libSceSystemService/Export.cpp | 25 +++++++- .../prx/libSceSystemService/SystemService.hpp | 3 + .../libs/tests/GuestSystemServiceLanguage.cpp | 60 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + docs/user/USAGE.md | 4 ++ 6 files changed, 101 insertions(+), 1 deletion(-) create mode 100644 core/libs/tests/GuestSystemServiceLanguage.cpp diff --git a/core/libs/prx/libSceSystemService/CMakeLists.txt b/core/libs/prx/libSceSystemService/CMakeLists.txt index e90856e4e..b99e85275 100644 --- a/core/libs/prx/libSceSystemService/CMakeLists.txt +++ b/core/libs/prx/libSceSystemService/CMakeLists.txt @@ -29,6 +29,15 @@ if(BUILD_TESTING) set_tests_properties(guest_system_service PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_system_service_language_tests ${LIBS_INCLUDE_DIR}/tests/GuestSystemServiceLanguage.cpp) + target_include_directories(guest_system_service_language_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_system_service_language_tests PRIVATE libSceSystemService libc) + configure_windows_unwind(guest_system_service_language_tests) + add_test(NAME guest_system_service_language COMMAND guest_system_service_language_tests) + if(WIN32) + set_tests_properties(guest_system_service_language PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_system_kill_app_tests ${LIBS_INCLUDE_DIR}/tests/GuestSystemKillApp.cpp) target_include_directories(guest_system_kill_app_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_system_kill_app_tests PRIVATE libSceSystemService libc) diff --git a/core/libs/prx/libSceSystemService/Export.cpp b/core/libs/prx/libSceSystemService/Export.cpp index ab607a86e..a5519684e 100644 --- a/core/libs/prx/libSceSystemService/Export.cpp +++ b/core/libs/prx/libSceSystemService/Export.cpp @@ -2,12 +2,35 @@ #include #include #include +#include +#include +#include +#include +#include #include "prx/libc/include/Shutdown.hpp" #include "SceTypes.hpp" #include "prx/libc/include/General.hpp" #include "prx/libkernel/KernelErrors.hpp" #include "prx/libSceSystemService/SystemService.hpp" +namespace { + +int ConsoleLanguage() { + const char* configured = std::getenv("ANYPS5_LANGUAGE"); + if (configured == nullptr || configured[0] == '\0') return SYSTEM_SERVICE_PARAM_LANG_ENGLISH_US; + const std::string_view text(configured); + int language = -1; + const auto [end, error] = std::from_chars(text.data(), text.data() + text.size(), language); + if (error != std::errc{} || end != text.data() + text.size() || language < SYSTEM_SERVICE_PARAM_LANG_JAPANESE || language > SYSTEM_SERVICE_PARAM_LANG_UKRAINIAN) { + throw std::runtime_error("sceSystemServiceParamGetInt: ANYPS5_LANGUAGE must be a console language number from " + + std::to_string(SYSTEM_SERVICE_PARAM_LANG_JAPANESE) + " to " + std::to_string(SYSTEM_SERVICE_PARAM_LANG_UKRAINIAN) + + ", not \"" + std::string(text) + "\""); + } + return language; +} + +} + extern "C" { int APS5_VABI sceSystemServiceLoadExec(const char* path, const char* const* arguments) { @@ -69,7 +92,7 @@ int APS5_VABI sceSystemServiceParamGetInt(int paramId, int* value) { return SYSTEM_SERVICE_ERROR_PARAMETER; } switch (paramId) { - case SYSTEM_SERVICE_PARAM_ID_LANG: *value = SYSTEM_SERVICE_PARAM_LANG_ENGLISH_US; break; + case SYSTEM_SERVICE_PARAM_ID_LANG: *value = ConsoleLanguage(); break; case SYSTEM_SERVICE_PARAM_ID_DATE_FORMAT: *value = SYSTEM_SERVICE_PARAM_DATE_FORMAT_DDMMYYYY; break; case SYSTEM_SERVICE_PARAM_ID_TIME_FORMAT: *value = SYSTEM_SERVICE_PARAM_TIME_FORMAT_24HOUR; break; case SYSTEM_SERVICE_PARAM_ID_TIME_ZONE: *value = 0; break; diff --git a/core/libs/prx/libSceSystemService/SystemService.hpp b/core/libs/prx/libSceSystemService/SystemService.hpp index 109d1c269..01f6f34f2 100644 --- a/core/libs/prx/libSceSystemService/SystemService.hpp +++ b/core/libs/prx/libSceSystemService/SystemService.hpp @@ -21,7 +21,10 @@ constexpr int SYSTEM_SERVICE_PARAM_ID_ENTER_BUTTON_ASSIGN = 1000; constexpr std::size_t SYSTEM_SERVICE_MAX_SYSTEM_NAME_LENGTH = 65; +constexpr int SYSTEM_SERVICE_PARAM_LANG_JAPANESE = 0; constexpr int SYSTEM_SERVICE_PARAM_LANG_ENGLISH_US = 1; +constexpr int SYSTEM_SERVICE_PARAM_LANG_ARABIC = 21; +constexpr int SYSTEM_SERVICE_PARAM_LANG_UKRAINIAN = 30; constexpr int SYSTEM_SERVICE_PARAM_DATE_FORMAT_DDMMYYYY = 1; constexpr int SYSTEM_SERVICE_PARAM_TIME_FORMAT_24HOUR = 1; constexpr int SYSTEM_SERVICE_PARAM_GAME_PARENTAL_OFF = 0; diff --git a/core/libs/tests/GuestSystemServiceLanguage.cpp b/core/libs/tests/GuestSystemServiceLanguage.cpp new file mode 100644 index 000000000..4b0e67f4d --- /dev/null +++ b/core/libs/tests/GuestSystemServiceLanguage.cpp @@ -0,0 +1,60 @@ +#include "SceTypes.hpp" +#include "prx/libSceSystemService/SystemService.hpp" +#include +#include +#include + +extern "C" int APS5_VABI sceSystemServiceParamGetInt(int paramId, int* value); + +namespace { + +void Require(bool value) { if (!value) std::abort(); } + +void SetLanguage(const char* value) { +#ifdef _WIN32 + Require(_putenv_s("ANYPS5_LANGUAGE", value != nullptr ? value : "") == 0); +#else + Require((value != nullptr ? ::setenv("ANYPS5_LANGUAGE", value, 1) : ::unsetenv("ANYPS5_LANGUAGE")) == 0); +#endif +} + +int Language() { + int value = -1; + Require(sceSystemServiceParamGetInt(SYSTEM_SERVICE_PARAM_ID_LANG, &value) == SYSTEM_SERVICE_OK); + return value; +} + +bool Rejected(const char* value) { + SetLanguage(value); + int language = -1; + try { + static_cast(sceSystemServiceParamGetInt(SYSTEM_SERVICE_PARAM_ID_LANG, &language)); + } catch (const std::runtime_error& error) { + return language == -1 && std::string(error.what()).find("ANYPS5_LANGUAGE") != std::string::npos; + } + return false; +} + +} + +int main() { + SetLanguage(nullptr); + Require(Language() == SYSTEM_SERVICE_PARAM_LANG_ENGLISH_US); + SetLanguage(""); + Require(Language() == SYSTEM_SERVICE_PARAM_LANG_ENGLISH_US); + SetLanguage("21"); + Require(Language() == SYSTEM_SERVICE_PARAM_LANG_ARABIC); + SetLanguage("0"); + Require(Language() == SYSTEM_SERVICE_PARAM_LANG_JAPANESE); + SetLanguage("30"); + Require(Language() == SYSTEM_SERVICE_PARAM_LANG_UKRAINIAN); + for (const char* value : {"31", "-1", "+21", " 21", "21 ", "21a", "0x15", "arabic", "4294967317"}) { + Require(Rejected(value)); + } + + SetLanguage("arabic"); + Require(sceSystemServiceParamGetInt(SYSTEM_SERVICE_PARAM_ID_LANG, nullptr) == SYSTEM_SERVICE_ERROR_PARAMETER); + int enterButton = -1; + Require(sceSystemServiceParamGetInt(SYSTEM_SERVICE_PARAM_ID_ENTER_BUTTON_ASSIGN, &enterButton) == SYSTEM_SERVICE_OK); + Require(enterButton == SYSTEM_SERVICE_PARAM_ENTER_BUTTON_CROSS); +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 1f9822255..cae7020a2 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -236,6 +236,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceUserServiceGetAgeLevel](../../core/libs/prx/libSceUserService/Export.cpp) (libSceUserService) - the meaning of the value is unknown; 0 is reported, as for the unset game presets. PPSA12544 (Unity) only stores it for its user profile - [sceGameUpdateGetAddcontLatestVersion](../../core/libs/prx/libSceGameUpdate/Export.cpp) (libSceGameUpdate) - a null `entitlement_label` throws: KytyPS5 accepts it, prosper returns `0x80412803`. Bytes after `found` are zeroed as in KytyPS5; prosper leaves them untouched - [sceSystemServiceParamGetString](../../core/libs/prx/libSceSystemService/Export.cpp) (libSceSystemService) - the system name (parameter 6) is the fixed `PS5`, not the console's name; other parameters and buffers of 1 to 64 bytes throw +- [sceSystemServiceParamGetInt](../../core/libs/prx/libSceSystemService/Export.cpp) (libSceSystemService) - console language numbers 0 to 29 follow the PS4 SDK enum in [OpenOrbis](https://github.com/OpenOrbis/OpenOrbis-PS4-Toolchain/blob/06fcf7fd108fa080035cdd3259ed22eb07e0508c/include/orbis/_types/sys_service.h#L54-L83) (21 Arabic). 30 (Ukrainian) is not in it and follows PPSA99505 ([Jelly5](https://github.com/02dnot/Jelly5/blob/552b866d89b2a58b27f7f7b0484dc743ce235526/app/src/app/i18n.cpp#L29-L60)) and PPSA99002 ([ProsperoLight](https://github.com/blackbearreloaded/ProsperoLight/blob/b708bbde3457bef1073a9ab6959957cb691e8e2f/include/i18n.hpp#L252-L309)), whose numbers agree with the enum. Not confirmed on a console, and neither title starts on `main` yet. Without `ANYPS5_LANGUAGE` the language is English (United States); the date format, time format, time zone, summer time and enter button stay fixed - [sceNgs2SystemSetSampleRate, sceNgs2SystemSetUserData, sceNgs2SystemGetUserData, sceNgs2SystemLock, sceNgs2SystemUnlock](../../core/libs/prx/libSceNgs2.native/src/System.cpp) (libSceNgs2.native) - shadPS4 only checks the system handle (`INVALID_SYSTEM_HANDLE`) and SharpProspero names what each call does; neither gives the other limits. A sample rate of 0 and a null user data pointer throw, any other rate is accepted, and changing the rate while a sampler filter is enabled throws because the filter coefficients were computed for the old rate. Lock and unlock take the same lock as `sceNgs2RackLock` - [sceNgs2RackGetInfo](../../core/libs/prx/libSceNgs2.native/src/Rack.cpp) (libSceNgs2.native) - field order from [shadPS4](https://github.com/shadps4-emu/shadPS4)'s `OrbisNgs2RackInfo` with the 64-byte name of the other info structures (216 bytes); `type` is assumed to be the rack id divided by 0x1000, `min_grain_samples` 64 (the system minimum) and `state_flags` 1; `uid` comes from a per-process rack counter, `render_count` is the owning system's render count and `active_voice_count` counts voices that are not empty (paused and stopped voices included). The last process ratio and tick and `active_channel_work_count` are left zero - [sceNgs2VoiceQueryInfo](../../core/libs/prx/libSceNgs2.native/src/Voice.cpp) (libSceNgs2.native) - only info id 0x4001 is known: PPSA22520 passes an 8-byte buffer and branches on the first word being 1 or 2, so it is assumed to be the voice channel count (0 before the voice is set up); the second word is zeroed and other ids throw diff --git a/docs/user/USAGE.md b/docs/user/USAGE.md index 713119e86..f1afb0e60 100644 --- a/docs/user/USAGE.md +++ b/docs/user/USAGE.md @@ -105,6 +105,10 @@ Windows PowerShell: Games that open the console's system font sets (`sceFontOpenFontSet`) need font files in an `anyps5-fonts/` directory beside the output executable; set `ANYPS5_SYSTEM_FONTS` to use another directory. Files dumped from the console are used under their own names (`SST-Roman.otf`, `SST-Bold.otf`, `SSTJpPro-Regular.otf`, ...). Without them, these openly licensed substitutes are used when present: `NotoSans-{Light,Regular,Medium,Bold}.ttf` and `NotoSans-{LightItalic,Italic,MediumItalic,BoldItalic}.ttf` (Latin and Vietnamese), `NotoSansMono-{Light,Regular,Medium,Bold}.ttf` (typewriter), `NotoSansThai-{Light,Regular,Medium,Bold}.ttf` (Thai) and `NotoSansCJK-{Light,Regular,Medium,Bold}.ttc` (Japanese and Chinese). Without either, opening a system font set fails and the game shows no text in those fonts. +### Console language + +Games ask for the console's language with `sceSystemServiceParamGetInt` and show their text in it when they have it. It is English (United States) unless `ANYPS5_LANGUAGE` is set to another console language number: 0 Japanese, 1 English (United States), 2 French, 3 Spanish, 4 German, 5 Italian, 6 Dutch, 7 Portuguese (Portugal), 8 Russian, 9 Korean, 10 Chinese (Traditional), 11 Chinese (Simplified), 12 Finnish, 13 Swedish, 14 Danish, 15 Norwegian, 16 Polish, 17 Portuguese (Brazil), 18 English (United Kingdom), 19 Turkish, 20 Spanish (Latin America), 21 Arabic, 22 French (Canada), 23 Czech, 24 Hungarian, 25 Greek, 26 Romanian, 27 Thai, 28 Vietnamese, 29 Indonesian, 30 Ukrainian. For example, `ANYPS5_LANGUAGE=21` selects Arabic. An empty value is the same as an unset one; any other value stops the game at its first language query with an error that names the variable. + ### GPU selection The game runs on the first Vulkan 1.1 device with graphics and compute queues and swapchain presentation, preferring a discrete GPU over an integrated one. Set `ANYPS5_GPU` to a part of a device name, compared without regard to case, to run on another device; the names are printed at start-up in the `Physical device candidate` lines. When no usable device contains the text, the start fails and the error lists the device names. From 982f7d7817be2e06af99e7518238d80d1551b9de Mon Sep 17 00:00:00 2001 From: Sanjaysrinivas Date: Thu, 8 Oct 2026 19:54:58 +0200 Subject: [PATCH 138/460] test(libkernel): cover cancellation recovery Refs #1668. Complements #1141 with resource ownership and lifecycle regressions. --- core/libs/prx/libkernel/CMakeLists.txt | 12 + .../libs/tests/GuestPthreadCancelRecovery.cpp | 280 ++++++++++++++++++ docs/dev/ARCHITECTURE.md | 12 + docs/dev/TechnicalDebt.md | 1 + 4 files changed, 305 insertions(+) create mode 100644 core/libs/tests/GuestPthreadCancelRecovery.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index e8f964d97..d4f2ae88f 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -393,6 +393,18 @@ if(BUILD_TESTING) set_tests_properties(guest_pthread_cancel PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_pthread_cancel_recovery_tests ${LIBS_INCLUDE_DIR}/tests/GuestPthreadCancelRecovery.cpp) + target_include_directories(guest_pthread_cancel_recovery_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_pthread_cancel_recovery_tests PRIVATE libkernel libc) + configure_windows_unwind(guest_pthread_cancel_recovery_tests) + add_test(NAME guest_pthread_cancel_recovery COMMAND guest_pthread_cancel_recovery_tests) + add_test(NAME guest_pthread_cancel_recovery_coarse COMMAND guest_pthread_cancel_recovery_tests) + set_tests_properties(guest_pthread_cancel_recovery guest_pthread_cancel_recovery_coarse PROPERTIES TIMEOUT 20) + set_tests_properties(guest_pthread_cancel_recovery_coarse PROPERTIES ENVIRONMENT "APS5_COARSE_TIMED_WAITS=1") + if(WIN32) + set_tests_properties(guest_pthread_cancel_recovery guest_pthread_cancel_recovery_coarse PROPERTIES + ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") + endif() add_test_executable(guest_semaphore_tests ${LIBS_INCLUDE_DIR}/tests/GuestSemaphore.cpp) target_include_directories(guest_semaphore_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_semaphore_tests PRIVATE libkernel libc) diff --git a/core/libs/tests/GuestPthreadCancelRecovery.cpp b/core/libs/tests/GuestPthreadCancelRecovery.cpp new file mode 100644 index 000000000..59e6056c8 --- /dev/null +++ b/core/libs/tests/GuestPthreadCancelRecovery.cpp @@ -0,0 +1,280 @@ +#include "SceTypes.hpp" +#include +#include +#include +#include +#include +#include + +extern "C" { +int APS5_VABI scePthreadCreate(Pthread* thread, const PthreadAttr* attr, PthreadEntry entry, void* arg, const char* name); +int APS5_VABI scePthreadJoin(Pthread thread, void** retval); +int APS5_VABI scePthreadCancel(Pthread thread); +int APS5_VABI scePthreadSetcancelstate(int state, int* oldState); +int APS5_VABI scePthreadSetcanceltype(int type, int* oldType); +void APS5_VABI scePthreadTestcancel(); +int APS5_VABI scePthreadKeyCreate(PthreadKey* key, pthread_key_destructor_func_t destructor); +int APS5_VABI scePthreadKeyDelete(PthreadKey key); +int APS5_VABI scePthreadSetspecific(PthreadKey key, void* value); +int APS5_VABI scePthreadMutexInit(PthreadMutex* mutex, const PthreadMutexattr* attr, const char* name); +int APS5_VABI scePthreadMutexDestroy(PthreadMutex* mutex); +int APS5_VABI scePthreadMutexLock(PthreadMutex* mutex); +int APS5_VABI scePthreadMutexUnlock(PthreadMutex* mutex); +int APS5_VABI scePthreadCondInit(PthreadCond* cond, const PthreadCondattr* attr, const char* name); +int APS5_VABI scePthreadCondDestroy(PthreadCond* cond); +int APS5_VABI scePthreadCondSignal(PthreadCond* cond); +int APS5_VABI scePthreadCondTimedwait(PthreadCond* cond, PthreadMutex* mutex, KernelUseconds usec); +int APS5_VABI scePthreadSemInit(PthreadSem* sem, int flag, unsigned int value, const char* name); +int APS5_VABI scePthreadSemDestroy(PthreadSem* sem); +int APS5_VABI scePthreadSemWait(PthreadSem* sem); +int APS5_VABI scePthreadSemTimedwait(PthreadSem* sem, KernelUseconds usec); +int APS5_VABI scePthreadSemGetvalue(PthreadSem* sem, int* value); +int APS5_VABI sem_init_nid_postfix(void* sem, int pshared, unsigned int value); +int APS5_VABI sem_destroy_nid_postfix(void* sem); +int APS5_VABI sem_wait_nid_postfix(void* sem); +int APS5_VABI sem_reltimedwait_np_nid_postfix(void* sem, std::uint32_t usec); +int APS5_VABI sem_getvalue_nid_postfix(void* sem, int* value); +int APS5_VABI sceKernelUsleep_nid_postfix(KernelUseconds microseconds); +void APS5_VABI __pthread_cleanup_push_imp_nid_postfix(void (APS5_VABI* routine)(void*), void* argument, void* info); +void APS5_VABI __pthread_cleanup_pop_imp_nid_postfix(int execute); +} + +static constexpr int CANCEL_ENABLE = 0; +static constexpr int CANCEL_DISABLE = 1; +static constexpr int CANCEL_DEFERRED = 0; +static constexpr int KERNEL_EINVAL = static_cast(0x80020016); +static constexpr KernelUseconds WAIT_MICROSECONDS = 60000000; +static void* const CANCELED = reinterpret_cast(std::uintptr_t{1}); + +static void Require(bool value, std::source_location location = std::source_location::current()) { + if (value) return; + std::fprintf(stderr, "%s:%u: requirement failed\n", location.file_name(), location.line()); + std::abort(); +} + +static PthreadSem sem = nullptr; +static void* posixSem = nullptr; +static PthreadMutex mutex = nullptr; +static PthreadCond cond = nullptr; +static PthreadKey key = -1; +static Pthread target = nullptr; +static std::atomic ready{false}, release{false}, targetRelease{false}; +static std::atomic events{0}; +static std::array order{}; +static bool usePosix = false, useTimed = false, disabled = false; +static bool conditionReleased = false, returned = false; +static int stage = 0, normalResult = 0, sentinel = 0; +static void* joinResult = &sentinel; + +static void Signal(std::atomic& flag) { + flag.store(true); + flag.notify_all(); +} + +static void Gate() { + Signal(ready); + release.wait(false); +} + +static Pthread Start(PthreadEntry entry) { + ready = false; + release = false; + returned = false; + events = 0; + order = {}; + Pthread thread = nullptr; + Require(scePthreadCreate(&thread, nullptr, entry, nullptr, nullptr) == 0); + ready.wait(false); + return thread; +} + +static void Join(Pthread thread, void* expected) { + void* result = &sentinel; + Require(scePthreadJoin(thread, &result) == 0); + Require(result == expected); +} + +static void APS5_VABI Record(void* value) { + const auto index = events.fetch_add(1); + Require(index < static_cast(order.size())); + order[index] = static_cast(reinterpret_cast(value)); +} + +static void APS5_VABI CleanupWithPoints(void*) { + Record(reinterpret_cast(std::uintptr_t{2})); + scePthreadTestcancel(); + Require(sceKernelUsleep_nid_postfix(1) == 0); + Record(reinterpret_cast(std::uintptr_t{3})); +} + +static void APS5_VABI DestroySpecific(void* value) { + Require(value == &normalResult); + Record(reinterpret_cast(std::uintptr_t{4})); +} + +static void WaitForToken() { + if (usePosix) { + Require((useTimed ? sem_reltimedwait_np_nid_postfix(&posixSem, WAIT_MICROSECONDS) + : sem_wait_nid_postfix(&posixSem)) == 0); + } else { + Require((useTimed ? scePthreadSemTimedwait(&sem, WAIT_MICROSECONDS) : scePthreadSemWait(&sem)) == 0); + } +} + +static int TokenCount() { + int count = -1; + Require((usePosix ? sem_getvalue_nid_postfix(&posixSem, &count) : scePthreadSemGetvalue(&sem, &count)) == 0); + return count; +} + +static void* APS5_VABI PendingToken(void*) { + std::uintptr_t first[8], second[8]; + Require(scePthreadSetspecific(key, &normalResult) == 0); + __pthread_cleanup_push_imp_nid_postfix(Record, reinterpret_cast(std::uintptr_t{1}), first); + __pthread_cleanup_push_imp_nid_postfix(CleanupWithPoints, nullptr, second); + Gate(); + WaitForToken(); + returned = true; + return nullptr; +} + +static void CheckDefaults() { + int old = -1; + Require(scePthreadSetcancelstate(CANCEL_ENABLE, &old) == 0 && old == CANCEL_ENABLE); + Require(scePthreadSetcanceltype(CANCEL_DEFERRED, &old) == 0 && old == CANCEL_DEFERRED); + scePthreadTestcancel(); +} + +static void* APS5_VABI ConsumeToken(void*) { + CheckDefaults(); + WaitForToken(); + return &normalResult; +} + +static void PendingTokenRecovery() { + Require((usePosix ? sem_init_nid_postfix(&posixSem, 0, 1) : scePthreadSemInit(&sem, 0, 1, nullptr)) == 0); + const auto thread = Start(PendingToken); + Require(scePthreadCancel(thread) == 0); + Require(scePthreadCancel(thread) == 0); + Require(events == 0 && TokenCount() == 1); + Signal(release); + Join(thread, CANCELED); + Require(!returned && events == 4); + Require((order == std::array{2, 3, 1, 4})); + Require(TokenCount() == 1); + Pthread consumer = nullptr; + Require(scePthreadCreate(&consumer, nullptr, ConsumeToken, nullptr, nullptr) == 0); + Join(consumer, &normalResult); + Require(TokenCount() == 0); + Require((usePosix ? sem_destroy_nid_postfix(&posixSem) : scePthreadSemDestroy(&sem)) == 0); +} + +static void APS5_VABI Unlock(void*) { + Require(scePthreadMutexUnlock(&mutex) == 0); + Record(reinterpret_cast(std::uintptr_t{1})); +} + +static void* APS5_VABI ConditionWaiter(void*) { + CheckDefaults(); + if (disabled) Require(scePthreadSetcancelstate(CANCEL_DISABLE, nullptr) == 0); + Require(scePthreadMutexLock(&mutex) == 0); + std::uintptr_t info[8]; + __pthread_cleanup_push_imp_nid_postfix(Unlock, nullptr, info); + Signal(ready); + while (!conditionReleased) Require(scePthreadCondTimedwait(&cond, &mutex, WAIT_MICROSECONDS) == 0); + Require(scePthreadMutexUnlock(&mutex) == 0); + __pthread_cleanup_pop_imp_nid_postfix(0); + if (disabled) { + stage = 1; + scePthreadTestcancel(); + int old = -1; + Require(scePthreadSetcancelstate(CANCEL_ENABLE, &old) == 0 && old == CANCEL_DISABLE); + stage = 2; + scePthreadTestcancel(); + } + returned = true; + return &normalResult; +} + +static void ConditionRecovery() { + Require(scePthreadMutexInit(&mutex, nullptr, nullptr) == 0); + Require(scePthreadCondInit(&cond, nullptr, nullptr) == 0); + conditionReleased = false; + stage = 0; + const auto thread = Start(ConditionWaiter); + Require(scePthreadMutexLock(&mutex) == 0); + Require(scePthreadCancel(thread) == 0); + Require(events == 0); + if (disabled) { + conditionReleased = true; + Require(scePthreadCondSignal(&cond) == 0); + } + Require(scePthreadMutexUnlock(&mutex) == 0); + Join(thread, CANCELED); + Require(!returned); + Require(disabled ? stage == 2 && events == 0 : events == 1 && order[0] == 1); + disabled = false; + conditionReleased = false; + const auto next = Start(ConditionWaiter); + Require(scePthreadMutexLock(&mutex) == 0); + conditionReleased = true; + Require(scePthreadCondSignal(&cond) == 0); + Require(scePthreadMutexUnlock(&mutex) == 0); + Join(next, &normalResult); + Require(returned && events == 0); + Require(scePthreadCondDestroy(&cond) == 0); + Require(scePthreadMutexDestroy(&mutex) == 0); +} + +static void* APS5_VABI HeldTarget(void*) { + CheckDefaults(); + targetRelease.wait(false); + return &normalResult; +} + +static void* APS5_VABI PendingJoin(void*) { + std::uintptr_t info[8]; + __pthread_cleanup_push_imp_nid_postfix(Record, reinterpret_cast(std::uintptr_t{1}), info); + Gate(); + Require(scePthreadJoin(target, &joinResult) == 0); + returned = true; + return nullptr; +} + +static void JoinRecovery() { + targetRelease = false; + joinResult = &sentinel; + Require(scePthreadCreate(&target, nullptr, HeldTarget, nullptr, nullptr) == 0); + const auto joiner = Start(PendingJoin); + Require(scePthreadCancel(joiner) == 0); + Signal(release); + Join(joiner, CANCELED); + Require(!returned && joinResult == &sentinel && events == 1 && order[0] == 1); + Signal(targetRelease); + Join(target, &normalResult); + Require(scePthreadCreate(&target, nullptr, HeldTarget, nullptr, nullptr) == 0); + Join(target, &normalResult); +} + +int main() { + int old = 17; + Require(scePthreadSetcancelstate(CANCEL_DISABLE, nullptr) == 0); + Require(scePthreadSetcancelstate(7, &old) == KERNEL_EINVAL && old == 17); + Require(scePthreadSetcancelstate(CANCEL_ENABLE, &old) == 0 && old == CANCEL_DISABLE); + old = 17; + Require(scePthreadSetcanceltype(1, &old) == KERNEL_EINVAL && old == 17); + Require(scePthreadSetcanceltype(CANCEL_DEFERRED, &old) == 0 && old == CANCEL_DEFERRED); + Require(scePthreadKeyCreate(&key, reinterpret_cast(DestroySpecific)) == 0); + for (bool posix : {false, true}) { + usePosix = posix; + for (bool timed : {false, true}) { + useTimed = timed; + PendingTokenRecovery(); + } + } + Require(scePthreadKeyDelete(key) == 0); + ConditionRecovery(); + disabled = true; + ConditionRecovery(); + JoinRecovery(); +} diff --git a/docs/dev/ARCHITECTURE.md b/docs/dev/ARCHITECTURE.md index 3713b767c..42746ab07 100644 --- a/docs/dev/ARCHITECTURE.md +++ b/docs/dev/ARCHITECTURE.md @@ -45,6 +45,18 @@ flowchart LR - [`core/relinker/main.cpp`](../../core/relinker/main.cpp) runs the steps in this order. `--to-intel` is optional, see [USAGE.md](../user/USAGE.md). - Each library in [`core/libs/prx`](../../core/libs/prx) builds as a shared library. After the build, `nid_patcher` ([`core/libs/nid`](../../core/libs/nid)) renames every export to its NID, computed from the function name. `APS5_EXPORT("", func)` sets the NID directly when the name is unknown. +## Thread cancellation + +[`Thread.cpp`](../../core/libs/prx/libkernel/Pthread/src/Thread.cpp) records cancellation requests on the target guest thread. New threads start with cancellation enabled and deferred. A disabled request stays pending; enabling deferred cancellation returns before the next cancellation point acts on it. Repeated requests do not add cleanup invocations. + +[`Cancel.hpp`](../../core/libs/prx/libkernel/Pthread/include/Cancel.hpp) connects condition, semaphore and join waits to the request. A canceled condition waiter reacquires its guest mutex before cleanup. Semaphore and join waits release their internal locks before exiting. A request pending on entry to a semaphore wait is acted on before consuming an available token; a canceled join leaves its target joinable and its result output untouched. + +Cancellation exits through `scePthreadExit` with result pointer `1`. It runs cleanup handlers in reverse push order before thread-specific destructors; cancellation points inside cleanup must not start another exit. The host thread then terminates through the existing Windows or Linux lifecycle, and join waits for that termination. Host thread termination is not used to inject cancellation into a running guest frame. + +The reference contract and constants come from FreeBSD libthr, pinned at [`5ed7eb0d`](https://github.com/freebsd/freebsd-src/blob/5ed7eb0d97ba4436218e810f61bd059acba984c2/lib/libthr/thread/thr_cancel.c), with cleanup and exit ordering in [`thr_exit.c`](https://github.com/freebsd/freebsd-src/blob/5ed7eb0d97ba4436218e810f61bd059acba984c2/lib/libthr/thread/thr_exit.c#L196-L259). These are reference semantics, not console measurements. The supported cancellation points and incomplete asynchronous behavior are listed in [TechnicalDebt](TechnicalDebt.md). + +[`GuestPthreadCancelRecovery.cpp`](../../core/libs/tests/GuestPthreadCancelRecovery.cpp) checks pending requests before ordinary and timed semaphore acquisition, cleanup reentrancy and destructor ordering, enabled and disabled timed condition waits, canceled join output, and subsequent resource and thread reuse. Host atomic gates establish pending requests; acquiring the condition mutex after the worker releases it establishes wait entry without a scheduling sleep. Both `guest_pthread_cancel_recovery` and `guest_pthread_cancel_recovery_coarse` retain checks in Release and have a 20-second timeout, shorter than the 60-second guest waits. The latter sets `APS5_COARSE_TIMED_WAITS=1` to exercise the Windows fallback; Linux uses that condition-variable path by default. + ## Graphics ```mermaid diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 1f9822255..2efd6c461 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -333,6 +333,7 @@ Throughout the project, every function at every stage either **does exactly what - [libSceUlt queues](../../core/libs/prx/libSceUlt/Export.cpp) use preallocated host storage instead of the supplied guest work area; its existing size formula and the 512-byte object layouts remain unverified. Waiting-queue pool thread limits and waiter priority ordering are not implemented. Finalization wakes blocked queue callers with `ULT_ERROR_STATE`. - [Thread cancellation](../../core/libs/prx/libkernel/Pthread/src/Thread.cpp) (libkernel) follows FreeBSD libthr (states and types 0/1 and 0/2, `PTHREAD_CANCELED` 1, the `_pthread_cleanup_info` node layout), not measured on the console. A request is acted on only in `scePthreadTestcancel`, the condition variable, `sem` and `scePthreadSem` waits, `scePthreadJoin` and the sleeps, and their POSIX names; file and socket calls are not cancellation points. A sleep is not interrupted: the thread exits when it ends. The asynchronous type acts like the deferred one, except at the two set calls. - [dup and dup2](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) throw for the combinations they do not model: `dup2` between a guest socket and a host descriptor in either direction, and on Windows duplicating a directory descriptor. Guest sockets share one socket object between the duplicates and the descriptor numbers come from the guest socket counter, not the lowest free number as in FreeBSD `kern_dup`. The errno values for descriptor exhaustion are not verified against the console. +- [Cancellation coverage](ARCHITECTURE.md#thread-cancellation) checks deferred request handling and recovery, not full asynchronous support or console ABI compatibility. Cancellation during file/socket I/O, sleep interruption, cancellation versus exit/detach, condition signal conservation under cancellation, and safe rejection of non-null invalid or reclaimed thread handles remain unverified. No console title or homebrew establishes the cleanup NIDs or buffer layout. Accepted asynchronous mode currently has no diagnostic for its deferred behavior. - [MIMG `tfe` and `lwe`](../../core/shader/recompiler/RdnaDecoder/src/RdnaImageOpDecoder.cpp) never write the status VGPR after the data. On hardware it is written only when a texel fetch hits an unmapped page of a partially resident texture, or, for `lwe` on sample/gather, when the LOD is below the T# `MIN_LOD_WARN`; texture memory is always resident here, the LOD warning (texture-streaming feedback, like the T# `MIN_LOD_WARN` itself) is never reported, and `lwe` on `image_get_lod` throws. - [Packed horizontal gathers](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_gather4h_pck`, MIMG opcode 98): the four window texels are read raw and concatenated into a little-endian bitstream, with DMASK bit i writing stream word i to the next destination register; measured on RDNA2 (gfx1031) at every edge: the anchor is floor(x-0.5) with elements at -1, 0, +1 and +2 clamped horizontally into the width, the row is floor(y), negative rows clamp to row 0, rows at or past the height return zeros, sampler addressing modes do not affect the window, and formats wider than 32 bits per texel write zeros in every register. The emitter fetches each window texel at LOD 0 and packs its bits like `image_load_pck`, so the sampler's filter, addressing and LOD state are not used; formats whose bits are not recoverable from the view and D16 data throw. The anchor and row multiply the normalized coordinate by the image size without contraction. The stream rule was measured for R8_UNORM, RG8_UNORM and RGBA8; only RGBA8 has an execution test. `image_gather8h_pck` (opcode 99) never returns texel data on this hardware - across dmasks 0x1 to 0xFF including the upper nibble, RGBA8, RGBA16 and RGBA32 formats, 1D, 2D and 3D descriptors and normalized and unnormalized sampling it only zero-fills popcount(dmask) registers (sentinel-prefilled destinations) - so it decodes but throws at translation; its eight-texel semantics remain unmeasured. - [waitpid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) returns -1 with `ECHILD` (`EINVAL` for unknown options): nothing here creates a child process; child processes would need a real `waitpid`. From c0610950d795fc8996e1506a84eb1cba465e2141 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 19:37:13 +0000 Subject: [PATCH 139/460] fix(agc): ignore unchanged imported texture blocks --- core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp index dc6f3e0f7..81ef0329e 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp @@ -1408,6 +1408,13 @@ bool StorageTexture::Refresh() { stampedBlocks.assign(generations.size(), 0); cpuBlocks.assign(generations.size(), 0); tracked = GuestMemory::ChangedBlocks(descriptor.baseAddress, static_cast(guestBytes), generations, stampedBlocks, cpuBlocks); + for (std::uint32_t unit = 0; tracked && blockUnits && unit < trackedLayers; ++unit) { + if (stampedBlocks[unit] != GuestMemory::BlockMaybeWritten) continue; + std::array block{}; + if (!compareUntracked(layerBegin(unit), static_cast(layerBytes(unit)), block) || block[0] != GuestMemory::BlockUnchanged) continue; + stampedBlocks[unit] = GuestMemory::BlockUnchanged; + cpuBlocks[unit] = 0; + } compared = !tracked && compareUntracked(descriptor.baseAddress, static_cast(guestBytes), stampedBlocks, true); if (compared) cpuBlocks = stampedBlocks; }; From 72f623197a3d50545a9c407cc873e93b5c1bc6f5 Mon Sep 17 00:00:00 2001 From: MrMha <114929652+MrMha@users.noreply.github.com> Date: Sat, 10 Oct 2026 23:39:36 +0400 Subject: [PATCH 140/460] fix(agc): keep polling unsatisfied memory waits after diagnostics Keep WAIT_REG_MEM diagnostic timeouts from completing unsatisfied waits. Register the existing gated-dispatch capture test with a 1ms diagnostic timeout and document the missing-producer limitation. --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 8 ++++++++ .../Execution/src/Driver/Synchronization/WaitMemory.cpp | 3 +-- docs/dev/TechnicalDebt.md | 1 + 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index d24c90ab0..3af48a852 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -921,6 +921,14 @@ if(BUILD_TESTING) "PATH=path_list_prepend:$") endif() endforeach() + add_test(NAME agc_driver_gated_dispatch_capture_short_timeout COMMAND agc_driver_gated_dispatch_capture_tests) + set_tests_properties(agc_driver_gated_dispatch_capture_short_timeout PROPERTIES + ENVIRONMENT "ANYPS5_NO_SHADER_CACHE=1;APS5_GPU_WAIT_TIMEOUT_MS=1;APS5_NO_PIPELINE_LTO=1" + SKIP_RETURN_CODE 77 TIMEOUT 30 WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}") + if(WIN32) + set_tests_properties(agc_driver_gated_dispatch_capture_short_timeout PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() set_tests_properties(agc_driver_provoking_vertex PROPERTIES TIMEOUT 30) set_tests_properties(agc_driver_flat_store PROPERTIES RUN_SERIAL TRUE) set_tests_properties(agc_driver_storage_cache_budget PROPERTIES TIMEOUT 30) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/WaitMemory.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/WaitMemory.cpp index 25e72727c..1475997dd 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/WaitMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/WaitMemory.cpp @@ -366,7 +366,7 @@ void Driver::waitMemory(std::span packet, std::uint32_t que static std::uint64_t timeouts = 0; const std::lock_guard lock(reportedMutex); if (++timeouts % 20 == 0) std::fprintf(stderr, "[gpu] %llu GPU waits have timed out\n", static_cast(timeouts)); - if (!reported.insert(packet[2] | (static_cast(packet[3]) << 32u)).second) return; + if (!reported.insert(packet[2] | (static_cast(packet[3]) << 32u)).second) continue; } const bool wide = ((packet[0] >> 8u) & 0xffu) == 0x93u; const std::uint64_t reference = wide ? packet[4] | (static_cast(packet[5]) << 32u) : packet[4]; @@ -392,7 +392,6 @@ void Driver::waitMemory(std::span packet, std::uint32_t que } context.Print(stderr, "[gpu] preceding packet %s\n"); } - if (warned) return; } ++outcomes.polled; } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 1f9822255..a69c7b788 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -371,6 +371,7 @@ Throughout the project, every function at every stage either **does exactly what - [v_fma_f64 and v_div_fmas_f64](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterFloat64.cpp) with a subnormal operand or result round once, as IEEE 754 fusedMultiplyAdd does with f64 denormals kept, measured on RDNA2 (gfx1035) at IEEE=0 and IEEE=1, `v_div_fmas_f64` with VCC clear and set. The f16/f64 denormal field of `FLOAT_MODE` (bits 7:6) is not read: when it flushes, the hardware flushes the operands and the result of `v_fma_f64` (tininess before rounding) and only the result of `v_div_fmas_f64`, while these keep them. - [SINT storage images](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) of 8- and 16-bit components saturate each stored 32-bit value to the component range, as adriadam10 measured for image stores on RDNA2 (gfx1035) in #944 (448 rows match); the PS5's gfx1013 is not measured. - [`StoreTexel`](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) writes channel 1 of a two-channel format from the data slot whose DST_SEL is Y, for SINT and UINT alike. With DST_SEL X,Y,Z,W, gfx1035 writes it from the W slot, as adriadam10 measured in #944. +- [WAIT_REG_MEM](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/WaitMemory.cpp) reports a diagnostic after `APS5_GPU_WAIT_TIMEOUT_MS` without treating the timeout as completion. An unsatisfied wait keeps polling and checking driver failures; a missing producer can therefore leave its queue waiting indefinitely. Existing recorded-label shortcuts remain in use. - [COND_EXEC](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Packets/PacketExecution.cpp) reads its condition after the earlier packets of its queue complete. Without PFP_SYNC_ME the console's graphics PFP reads it ahead of the ME, so a title that races the read sees a later value here. - A [flip inside a COND_EXEC range](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Packets/CommandValidation.cpp) throws: the flip is reserved with the video output when the buffer is submitted, and how the console withdraws a skipped one is unknown. - A [COND_EXEC range](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp) must hold whole packets and end inside its command buffer. A chained INDIRECT_BUFFER or a REWIND inside a range throws, as what the CP does with the skipped dwords then is unknown. From 19803c0086298cfbc3653d8846dc7676a7c88f4a Mon Sep 17 00:00:00 2001 From: Doston Abdullayev Date: Sat, 10 Oct 2026 14:50:38 +0500 Subject: [PATCH 141/460] fix(relinker): start Windows executables from directories outside the ANSI code page The Windows entry stub read its path with GetModuleFileNameA, which turns every character the ANSI code page cannot represent into '?', and then used it with SetCurrentDirectoryA and LoadLibraryExA. A converted executable in such a directory (Japanese on a Western-locale system, an emoji, a script without an ANSI code page) stopped at start-up with ERROR_INVALID_NAME. Read the path with GetModuleFileNameW and enter the directory and load the libraries with SetCurrentDirectoryW and LoadLibraryExW, with library paths stored as UTF-16. Messages and the --windows-diagnostics walker still get the paths in the ANSI code page, as recorded in TechnicalDebt. Fixes #2640 Co-Authored-By: Claude Opus 5.5 --- .../src/windows/WindowsEntryStubBuilder.cpp | 103 +++++++++++++----- .../src/windows/WindowsImportBuilder.cpp | 2 +- .../tests/test_windows_working_directory.py | 1 + docs/dev/TechnicalDebt.md | 1 + 4 files changed, 81 insertions(+), 26 deletions(-) diff --git a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp index b696e76a4..e4720a92e 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp @@ -31,6 +31,35 @@ std::string normalizeRunPath(std::string path) { return path; } +std::u16string toUtf16(const std::string& text) { + static constexpr std::uint32_t minimum[] = {0, 0, 0x80, 0x800, 0x10000}; + std::u16string result; + for (std::size_t index = 0; index < text.size();) { + const auto lead = static_cast(text[index]); + const std::size_t length = lead < 0x80 ? 1 : (lead & 0xe0) == 0xc0 ? 2 : (lead & 0xf0) == 0xe0 ? 3 : (lead & 0xf8) == 0xf0 ? 4 : 0; + if (length == 0 || text.size() - index < length) + throw Domain::RelinkerException("Windows library path is not valid UTF-8: " + text); + std::uint32_t code = length == 1 ? lead : lead & (0x7fu >> length); + for (std::size_t next = 1; next < length; ++next) { + const auto continuation = static_cast(text[index + next]); + if ((continuation & 0xc0) != 0x80) + throw Domain::RelinkerException("Windows library path is not valid UTF-8: " + text); + code = code << 6 | (continuation & 0x3f); + } + if (code < minimum[length] || code > 0x10ffff || (code >= 0xd800 && code < 0xe000)) + throw Domain::RelinkerException("Windows library path is not valid UTF-8: " + text); + if (code >= 0x10000) { + code -= 0x10000; + result.push_back(static_cast(0xd800 + (code >> 10))); + result.push_back(static_cast(0xdc00 + (code & 0x3ff))); + } else { + result.push_back(static_cast(code)); + } + index += length; + } + return result; +} + } WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, const std::uint32_t entryRva, const WindowsImports& nativeImports, const std::vector& libraries, const std::vector& imports, const std::string& runPath, const bool lazyBinding, const bool dependencyDiagnostics, const std::vector& guestModules) const { @@ -54,9 +83,17 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con Io::AppendString(data, value); return rva; }; + const auto addWideString = [&](const std::u16string& value) { + Io::AlignBuffer(data, 2); + const auto rva = CheckedRva(dataRva + data.size()); + for (const auto unit : value) + Io::AppendU16(data, unit); + Io::AppendU16(data, 0); + return rva; + }; - const auto programPath = reserve(PathCapacity); - const auto modulePath = reserve(PathCapacity); + const auto programPath = reserve(PathCapacity * 2); + const auto modulePath = reserve(PathCapacity * 2); const auto argumentBlock = reserve(8); const auto argumentCount = reserve(4); const auto wideArguments = reserve(8); @@ -72,14 +109,15 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con data.insert(data.end(), {1, 10, 6, 0, 10, 0xb2, 6, 0xc0, 4, 0x70, 3, 0x60, 2, 0x50, 1, 0x30}); std::vector libraryPaths; - std::vector libraryNames; + std::vector libraryUnits; for (std::size_t index = 0; index < libraries.size(); ++index) { auto name = index < guestModules.size() ? libraries[index] : path + libraries[index]; std::replace(name.begin(), name.end(), '/', '\\'); - if (name.size() + 1 >= PathCapacity) + const auto wideName = toUtf16(name); + if (wideName.size() + 1 >= PathCapacity) throw Domain::RelinkerException("Windows library path exceeds the startup buffer: " + name); - libraryPaths.push_back(addString(name)); - libraryNames.push_back(std::move(name)); + libraryPaths.push_back(addWideString(wideName)); + libraryUnits.push_back(wideName.size()); } std::vector symbolNames; @@ -138,6 +176,7 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con std::vector resolvedPaths; for (std::size_t index = 0; index < libraries.size(); ++index) resolvedPaths.push_back(reserve(PathCapacity)); + const std::uint32_t programAnsiPath = dependencyDiagnostics ? reserve(PathCapacity) : 0; const auto diagnosticsOffset = data.size(); std::vector errors = {"FAIL: cannot obtain executable path\n", "FAIL: executable or library path is too long\n", "FAIL: executable path has no directory\n"}; @@ -248,12 +287,25 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con code.PatchBranch(success, code.GetRva()); }; + const auto writeAnsi = [&](const std::uint32_t destination) { + code.Emit({0x31, 0xc9, 0x31, 0xd2, 0x41, 0xb9, 0xff, 0xff, 0xff, 0xff}); + code.Rip({0x48, 0x8d, 0x05}, destination); + code.Emit({0x48, 0x89, 0x44, 0x24, 0x20, 0x48, 0xc7, 0x44, 0x24, 0x28}); + code.U32(PathCapacity); + code.Emit({0x48, 0xc7, 0x44, 0x24, 0x30, 0, 0, 0, 0, 0x48, 0xc7, 0x44, 0x24, 0x38, 0, 0, 0, 0}); + call("WideCharToMultiByte"); + code.Emit({0x85, 0xc0}); + const auto converted = code.Branch({0x0f, 0x85}); + fail(1, 0xc0000106u); + code.PatchBranch(converted, code.GetRva()); + }; + code.Emit({0x53, 0x55, 0x56, 0x57, 0x41, 0x54, 0x48, 0x83, 0xec, 0x60}); code.Emit({0x31, 0xc9}); code.Rip({0x48, 0x8d, 0x15}, programPath); code.Emit({0x41, 0xb8}); code.U32(PathCapacity); - call("GetModuleFileNameA"); + call("GetModuleFileNameW"); requireNonzero(0, 0xc000000du); code.Emit({0x3d}); code.U32(PathCapacity); @@ -263,53 +315,54 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con code.Emit({0x89, 0xc1}); code.Rip({0x48, 0x8d, 0x35}, programPath); code.Rip({0x48, 0x8d, 0x3d}, modulePath); - code.Emit({0xfc, 0xf3, 0xa4, 0x49, 0x89, 0xfc}); + code.Emit({0xfc, 0x66, 0xf3, 0xa5, 0x49, 0x89, 0xfc}); code.Rip({0x48, 0x8d, 0x1d}, modulePath); const auto findSeparator = code.GetRva(); code.Emit({0x49, 0x39, 0xdc}); const auto hasDirectory = code.Branch({0x0f, 0x85}); fail(2, 0xc000000du); code.PatchBranch(hasDirectory, code.GetRva()); - code.Emit({0x49, 0xff, 0xcc, 0x41, 0x80, 0x3c, 0x24, 0x5c}); + code.Emit({0x49, 0x83, 0xec, 0x02, 0x66, 0x41, 0x83, 0x3c, 0x24, 0x5c}); code.Rip({0x0f, 0x85}, findSeparator); - code.Emit({0x49, 0xff, 0xc4}); - code.Emit({0x41, 0xc6, 0x04, 0x24, 0x00}); + code.Emit({0x49, 0x83, 0xc4, 0x02}); + code.Emit({0x66, 0x41, 0xc7, 0x04, 0x24, 0x00, 0x00}); code.Rip({0x48, 0x8d, 0x0d}, modulePath); - call("SetCurrentDirectoryA"); + call("SetCurrentDirectoryW"); requireNonzero(directoryError, 0xc000000du); + if (dependencyDiagnostics) { + code.Rip({0x4c, 0x8d, 0x05}, programPath); + writeAnsi(programAnsiPath); + } for (std::size_t index = 0; index < libraries.size(); ++index) { if (absolutePath && index >= guestModules.size()) { code.Rip({0x48, 0x8d, 0x0d}, libraryPaths[index]); } else { + const auto units = CheckedRva(libraryUnits[index] + 1); code.Emit({0x4c, 0x89, 0xe0, 0x48, 0x29, 0xd8, 0x48, 0x05}); - code.U32(CheckedRva(libraryNames[index].size() + 1)); + code.U32(units * 2); code.Emit({0x48, 0x3d}); - code.U32(PathCapacity); + code.U32(PathCapacity * 2); const auto fits = code.Branch({0x0f, 0x86}); fail(1, 0xc0000106u); code.PatchBranch(fits, code.GetRva()); code.Emit({0x4c, 0x89, 0xe7}); code.Rip({0x48, 0x8d, 0x35}, libraryPaths[index]); code.Emit({0xb9}); - code.U32(CheckedRva(libraryNames[index].size() + 1)); - code.Emit({0xf3, 0xa4}); + code.U32(units); + code.Emit({0x66, 0xf3, 0xa5}); code.Rip({0x48, 0x8d, 0x0d}, modulePath); } - code.Emit({0x48, 0x89, 0xce}); - call("lstrlenA"); - code.Emit({0x8d, 0x48, 1}); - code.Rip({0x48, 0x8d, 0x3d}, resolvedPaths[index]); - code.Emit({0xf3, 0xa4}); + code.Emit({0x48, 0x89, 0xce, 0x49, 0x89, 0xf0}); + writeAnsi(resolvedPaths[index]); if (dependencyDiagnostics) { writeString(loading); writeString(resolvedPaths[index]); } - code.Rip({0x48, 0x8d, 0x0d}, resolvedPaths[index]); - code.Emit({0x31, 0xd2, 0x41, 0xb8, 0, 0x11, 0, 0}); - call("LoadLibraryExA"); + code.Emit({0x48, 0x89, 0xf1, 0x31, 0xd2, 0x41, 0xb8, 0, 0x11, 0, 0}); + call("LoadLibraryExW"); code.Emit({0x48, 0x85, 0xc0}); const auto loadSucceeded = code.Branch({0x0f, 0x85}); captureLastError(); @@ -320,7 +373,7 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con writeLastError(); if (dependencyDiagnostics) { code.Rip({0x48, 0x8d, 0x0d}, resolvedPaths[index]); - code.Rip({0x48, 0x8d, 0x15}, programPath); + code.Rip({0x48, 0x8d, 0x15}, programAnsiPath); dependencyCalls.push_back(code.Branch({0xe8})); } raise(0xc0000135u); diff --git a/core/relinker/elfpatcher/src/windows/WindowsImportBuilder.cpp b/core/relinker/elfpatcher/src/windows/WindowsImportBuilder.cpp index 070970f7d..6f5b0fb32 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsImportBuilder.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsImportBuilder.cpp @@ -5,7 +5,7 @@ namespace Elfpatcher::Windows { WindowsImports WindowsImportBuilder::Build(const std::uint32_t sectionRva) const { - const std::vector names = {"ExitProcess", "FormatMessageA", "FreeLibrary", "GetCommandLineW", "GetFileAttributesA", "GetLastError", "GetModuleFileNameA", "GetModuleHandleA", "GetProcAddress", "GetStdHandle", "GetSystemDirectoryA", "LoadLibraryExA", "LocalFree", "RaiseException", "SetCurrentDirectoryA", "VirtualAlloc", "WideCharToMultiByte", "WriteFile", "lstrcatA", "lstrcmpA", "lstrcmpiA", "lstrcpyA", "lstrlenA"}; + const std::vector names = {"ExitProcess", "FormatMessageA", "FreeLibrary", "GetCommandLineW", "GetFileAttributesA", "GetLastError", "GetModuleFileNameA", "GetModuleFileNameW", "GetModuleHandleA", "GetProcAddress", "GetStdHandle", "GetSystemDirectoryA", "LoadLibraryExA", "LoadLibraryExW", "LocalFree", "RaiseException", "SetCurrentDirectoryW", "VirtualAlloc", "WideCharToMultiByte", "WriteFile", "lstrcatA", "lstrcmpA", "lstrcmpiA", "lstrcpyA", "lstrlenA"}; WindowsImports result{{".idata", sectionRva, SectionRead | SectionWrite | 0x40u, std::vector(40)}, {sectionRva, 40}, {}, {}}; auto& bytes = result.Section.Data; const auto lookupOffset = bytes.size(); diff --git a/core/relinker/relinker/tests/test_windows_working_directory.py b/core/relinker/relinker/tests/test_windows_working_directory.py index 426acb896..340afd953 100644 --- a/core/relinker/relinker/tests/test_windows_working_directory.py +++ b/core/relinker/relinker/tests/test_windows_working_directory.py @@ -103,6 +103,7 @@ def main(): work = Path(directory) check(relinker, work, 'console', []) check(relinker, work, 'gui', ['--windows-gui']) + check(relinker, work, 'unicode ü日', []) print('Windows working directory integration tests passed') diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 1f9822255..30a3b9ab7 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -387,6 +387,7 @@ Throughout the project, every function at every stage either **does exactly what - [Packed f16 NaN results](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) (`v_pk_add/mul/fma/min/max_f16`) follow the hardware measured on RDNA2 (gfx1035) at IEEE=0 and IEEE=1 with f16 round-to-nearest-even and f16 denormals kept: with IEEE=1 a signaling NaN input is quieted and min/max return the other operand only for a quiet NaN; with IEEE=0 NaN inputs pass through unquieted and min/max ignore signaling. In `v_pk_fma_f16` an invalid product (inf * 0) gives the default NaN even when src2 is a NaN. A shader without a known float mode is translated as IEEE=0, the mode titles use; a mode with another f16 rounding or that flushes f16 denormals throws. - [f32 denormals](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follow the shader's `FLOAT_MODE` f32 denormal field (bits 5:4) when it flushes them (0, and shaders without a known float mode, as in the 0xc0 most titles use), measured on RDNA2 (gfx1035) over 1444 rows per instruction at IEEE=0 and IEEE=1. `v_add/sub/subrev/mul/mul_legacy/mullit/fma/fmac/fmamk/fmaak/min/max/min3/max3/med3/fract/floor/ceil/frexp_mant/sin/cos_f32` flush their f32 inputs and their result to a zero of the same sign (`v_mullit_f32` measured on RDNA2 (gfx1036) over 6907 rows per mode at IEEE=0 and IEEE=1, so a positive denormal src2 counts as zero); the f32 compares, `v_cvt_i32/u32/rpi_i32/flr_i32_f32`, `v_frexp_exp_i32_f32`, `v_cvt_f64_f32`, `v_cvt_f16_f32`, `v_cvt_pkrtz_f16_f32`, `v_cvt_pknorm_i16/u16_f32` and `v_cvt_pk_u8_f32` flush their f32 inputs. `v_div_fixup_f32` flushes its src1 and src2 and keeps a denormal src0 and result, measured on RDNA2 (gfx1036) over 6859 rows per mode at IEEE=0 and IEEE=1. A product is checked for tininess on its exact value: one below 2^-126 gives zero even when it rounds to 2^-126, computed exactly from the significands for `v_mul_f32`, `v_mul_legacy_f32`, `v_mullit_f32`, and for `v_fma_f32`, `v_fmac_f32`, `v_fmamk_f32`, `v_fmaak_f32` and `v_fma_mix_f32` when the addend is zero; with a nonzero addend the fused result is kept, so an exact result just below 2^-126 that rounds to it is not flushed. The hardware detects tininess after rounding to 24 bits with an unbounded exponent, measured on RDNA2 (gfx1036) for `v_mul_f32` and `v_mullit_f32` at IEEE=0 and IEEE=1: `0x3f7ffffe * 0x00800001` (2^-126 * (1 - 2^-46)) gives `0x00800000` there and zero here, while `0x3f7fffff * 0x00800000` (2^-126 * (1 - 2^-24)) gives zero on both. `v_fma_mix_f32` was measured with f32 sources only; its f16 sources are not flushed. `v_fma_mixlo_f16` and `v_fma_mixhi_f16` flush their f32 sources too and keep a product below 2^-126, measured on RDNA2 (gfx1036). `v_mad_legacy_f32` and `v_mac_legacy_f32` flush their inputs, their product and their result in every mode, see its entry. Denormal modes 1 and 2 throw for the other instructions. Not covered, and not measurable on RDNA2 (no such opcodes) or not measured: `v_mac_f32`, `v_mad_f32`, `v_madak_f32` and `v_madmk_f32` (rounded separately), `v_dot2_f32_f16`, `v_div_scale_f32` and `v_div_fmas_f32` (on hardware `v_div_fmas_f32` flushes only its result), f32 interpolation and `ds_add_f32` keep the previous lowering; `ds_min/max/cmpst_f32` flush both operands by the field, see the LDS float atomics entry. With f32 denormals kept (3) the results depend on the driver's default denormal behaviour, as no `DenormPreserve` execution mode is requested, and `v_ldexp_f32` flushes in every mode. - The executable file that [relinker](../../core/relinker/elfpatcher/src/windows/WindowsPeWriter.cpp) generates opens the console when launched, which is inconvenient for playability. +- The Windows [entry stub](../../core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp) (relinker) finds, enters and loads from its directory through the wide API, but prints library paths, and passes them to the `--windows-diagnostics` [dependency walker](../../core/relinker/elfpatcher/src/windows/WindowsDependencyStubBuilder.cpp), in the ANSI code page. The walker uses the ANSI API, so under a directory the code page cannot represent the paths it prints and checks contain `?`. - [Overlay output bus](../../core/libs/prx/libSceVideoOut/src/VideoOutDriver.cpp) (libSceVideoOut) - flips on a port opened with `SCE_VIDEO_OUT_BUS_TYPE_OVERLAY` complete and signal their events like main-bus flips, but their buffers are not shown: the overlay is not composited over the main output. Presenting them as whole frames made the window alternate between the two buses - [PA_CL_CLIP_CNTL](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) - `DX_LINEAR_ATTR_CLIP_ENA` (bit 24) clear is drawn like bit 24 set, with Vulkan's clipping of `NoPerspective` outputs; what a clear bit changes on the console is undocumented. - [10_11_11 unorm color targets](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) render into R32_UINT attachments: the pixel shader packs each export, rounded to nearest even like format-30 image stores, which is assumed to match the CB's conversion. From 28b1f1578f1a0a3e2cdbbddae65d613c802c1eeb Mon Sep 17 00:00:00 2001 From: Doston Abdullayev Date: Sat, 10 Oct 2026 15:27:10 +0500 Subject: [PATCH 142/460] fix(relinker): walk Windows dependency diagnostics through the wide API The --windows-diagnostics walker looked up modules and directories with GetModuleHandleA, LoadLibraryExA, GetFileAttributesA, GetModuleFileNameA and GetSystemDirectoryA, so under a directory outside the ANSI code page it failed with "dependency diagnostics Windows API failed" instead of naming the missing dependency. Keep the walker's paths in UTF-8 and convert them at the Windows API calls, which now use the wide functions. The entry stub passes it, and prints, its paths as UTF-8 instead of the ANSI code page. The dependency diagnostics test now builds its fixtures under a non-ASCII directory. Co-Authored-By: Claude Opus 5.5 --- .../windows/WindowsDependencyStubBuilder.cpp | 62 ++++++++++++++++--- .../src/windows/WindowsEntryStubBuilder.cpp | 14 +++-- .../src/windows/WindowsImportBuilder.cpp | 2 +- .../WindowsDependencyDiagnosticsTests.cpp | 11 ++-- docs/dev/TechnicalDebt.md | 1 - 5 files changed, 70 insertions(+), 20 deletions(-) diff --git a/core/relinker/elfpatcher/src/windows/WindowsDependencyStubBuilder.cpp b/core/relinker/elfpatcher/src/windows/WindowsDependencyStubBuilder.cpp index a6e4c952d..5b1dda8c7 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsDependencyStubBuilder.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsDependencyStubBuilder.cpp @@ -23,6 +23,9 @@ WindowsDependencyStubBuilder::WindowsDependencyStubBuilder(PeSection& data) { storage.emplace(name, CheckedRva(data.Rva + data.Data.size())); data.Data.resize(data.Data.size() + 32); } + Io::AlignBuffer(data.Data, 2); + storage.emplace("wide", CheckedRva(data.Rva + data.Data.size())); + data.Data.resize(data.Data.size() + PathCapacity * 2); const std::map messages = { {"invalid", "FAIL: invalid or unsupported PE dependency metadata\n"}, {"limit", "FAIL: dependency diagnostics capacity exceeded\n"}, @@ -105,6 +108,37 @@ WindowsDependencyStub WindowsDependencyStubBuilder::Build(WindowsStubEmitter& co a.Jump("io", 0x84); a.End(); + a.Begin("wide"); + a.Mov(R8, Cx); + a.Value(Cx, 65001); + a.Value(Dx, 0); + a.Value(R9, 0xffffffffffffffffu); + a.Data(Ax, "wide"); + a.Store(Sp, 32, Ax); + a.Value(Ax, PathCapacity); + a.Store(Sp, 40, Ax); + a.Api("MultiByteToWideChar"); + a.Test(Ax); + a.Jump("limit", 0x84); + a.Data(Ax, "wide"); + a.End(); + + a.Begin("narrow"); + a.Store(Sp, 32, Cx); + a.Value(Cx, 65001); + a.Value(Dx, 0); + a.Data(R8, "wide"); + a.Value(R9, 0xffffffffffffffffu); + a.Value(Ax, PathCapacity); + a.Store(Sp, 40, Ax); + a.Value(Ax, 0); + a.Store(Sp, 48, Ax); + a.Store(Sp, 56, Ax); + a.Api("WideCharToMultiByte"); + a.Test(Ax); + a.Jump("limit", 0x84); + a.End(); + a.Begin("directory"); a.Mov(Bx, Cx); a.Call("copy"); @@ -201,14 +235,18 @@ WindowsDependencyStub WindowsDependencyStubBuilder::Build(WindowsStubEmitter& co a.Mov(Dx, Si); a.Call("copy"); a.Address(Cx, Bx, 64); - a.Api("GetModuleHandleA"); + a.Call("wide"); + a.Mov(Cx, Ax); + a.Api("GetModuleHandleW"); a.Store(Bx, 8, Ax); a.Test(Ax); a.Jump("registerMapped", 0x85); a.Address(Cx, Bx, 64); + a.Call("wide"); + a.Mov(Cx, Ax); a.Value(Dx, 0); a.Value(R8, 0x20); - a.Api("LoadLibraryExA"); + a.Api("LoadLibraryExW"); a.Test(Ax); a.Jump("io", 0x84); a.Mask(Ax, 0xfffffffcu); @@ -263,7 +301,9 @@ WindowsDependencyStub WindowsDependencyStubBuilder::Build(WindowsStubEmitter& co a.Mov(R8, Bx); a.Call("join"); a.Data(Cx, "scratch"); - a.Api("GetFileAttributesA"); + a.Call("wide"); + a.Mov(Cx, Ax); + a.Api("GetFileAttributesW"); a.Value(Dx, 0xffffffffu); a.Compare(Ax, Dx); a.Jump(std::string("resolveNext") + directory, 0x84); @@ -295,13 +335,15 @@ WindowsDependencyStub WindowsDependencyStubBuilder::Build(WindowsStubEmitter& co a.Jump("io", 0x84); a.Mark("resolveLoaded"); a.Mov(Cx, Ax); - a.Data(Dx, "scratch"); + a.Data(Dx, "wide"); a.Value(R8, PathCapacity); - a.Api("GetModuleFileNameA"); + a.Api("GetModuleFileNameW"); a.Test(Ax); a.Jump("io", 0x84); a.CompareValue(Ax, PathCapacity); a.Jump("limit", 0x83); + a.Data(Cx, "scratch"); + a.Call("narrow"); a.Mark("resolvePath"); a.Data(Cx, "scratch"); a.Call("register"); @@ -623,13 +665,19 @@ WindowsDependencyStub WindowsDependencyStubBuilder::Build(WindowsStubEmitter& co a.Data(Cx, "executableDirectory"); a.Mov(Dx, Si); a.Call("directory"); - a.Data(Cx, "systemDirectory"); + a.Data(Cx, "wide"); a.Value(Dx, PathCapacity - 1); - a.Api("GetSystemDirectoryA"); + a.Api("GetSystemDirectoryW"); a.Test(Ax); a.Jump("io", 0x84); a.CompareValue(Ax, PathCapacity - 1); a.Jump("limit", 0x83); + a.Data(Cx, "systemDirectory"); + a.Call("narrow"); + a.Data(Cx, "systemDirectory"); + a.Api("lstrlenA"); + a.CompareValue(Ax, PathCapacity - 1); + a.Jump("limit", 0x83); a.Data(Dx, "systemDirectory"); a.Add(Dx, Ax); a.Value(Ax, '\\'); diff --git a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp index e4720a92e..d254e6d94 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp @@ -176,7 +176,7 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con std::vector resolvedPaths; for (std::size_t index = 0; index < libraries.size(); ++index) resolvedPaths.push_back(reserve(PathCapacity)); - const std::uint32_t programAnsiPath = dependencyDiagnostics ? reserve(PathCapacity) : 0; + const std::uint32_t programText = dependencyDiagnostics ? reserve(PathCapacity) : 0; const auto diagnosticsOffset = data.size(); std::vector errors = {"FAIL: cannot obtain executable path\n", "FAIL: executable or library path is too long\n", "FAIL: executable path has no directory\n"}; @@ -287,8 +287,10 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con code.PatchBranch(success, code.GetRva()); }; - const auto writeAnsi = [&](const std::uint32_t destination) { - code.Emit({0x31, 0xc9, 0x31, 0xd2, 0x41, 0xb9, 0xff, 0xff, 0xff, 0xff}); + const auto writeUtf8 = [&](const std::uint32_t destination) { + code.Emit({0xb9}); + code.U32(65001); + code.Emit({0x31, 0xd2, 0x41, 0xb9, 0xff, 0xff, 0xff, 0xff}); code.Rip({0x48, 0x8d, 0x05}, destination); code.Emit({0x48, 0x89, 0x44, 0x24, 0x20, 0x48, 0xc7, 0x44, 0x24, 0x28}); code.U32(PathCapacity); @@ -331,7 +333,7 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con requireNonzero(directoryError, 0xc000000du); if (dependencyDiagnostics) { code.Rip({0x4c, 0x8d, 0x05}, programPath); - writeAnsi(programAnsiPath); + writeUtf8(programText); } for (std::size_t index = 0; index < libraries.size(); ++index) { @@ -354,7 +356,7 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con code.Rip({0x48, 0x8d, 0x0d}, modulePath); } code.Emit({0x48, 0x89, 0xce, 0x49, 0x89, 0xf0}); - writeAnsi(resolvedPaths[index]); + writeUtf8(resolvedPaths[index]); if (dependencyDiagnostics) { writeString(loading); @@ -373,7 +375,7 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con writeLastError(); if (dependencyDiagnostics) { code.Rip({0x48, 0x8d, 0x0d}, resolvedPaths[index]); - code.Rip({0x48, 0x8d, 0x15}, programAnsiPath); + code.Rip({0x48, 0x8d, 0x15}, programText); dependencyCalls.push_back(code.Branch({0xe8})); } raise(0xc0000135u); diff --git a/core/relinker/elfpatcher/src/windows/WindowsImportBuilder.cpp b/core/relinker/elfpatcher/src/windows/WindowsImportBuilder.cpp index 6f5b0fb32..30cb39ecd 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsImportBuilder.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsImportBuilder.cpp @@ -5,7 +5,7 @@ namespace Elfpatcher::Windows { WindowsImports WindowsImportBuilder::Build(const std::uint32_t sectionRva) const { - const std::vector names = {"ExitProcess", "FormatMessageA", "FreeLibrary", "GetCommandLineW", "GetFileAttributesA", "GetLastError", "GetModuleFileNameA", "GetModuleFileNameW", "GetModuleHandleA", "GetProcAddress", "GetStdHandle", "GetSystemDirectoryA", "LoadLibraryExA", "LoadLibraryExW", "LocalFree", "RaiseException", "SetCurrentDirectoryW", "VirtualAlloc", "WideCharToMultiByte", "WriteFile", "lstrcatA", "lstrcmpA", "lstrcmpiA", "lstrcpyA", "lstrlenA"}; + const std::vector names = {"ExitProcess", "FormatMessageA", "FreeLibrary", "GetCommandLineW", "GetFileAttributesW", "GetLastError", "GetModuleFileNameW", "GetModuleHandleA", "GetModuleHandleW", "GetProcAddress", "GetStdHandle", "GetSystemDirectoryW", "LoadLibraryExA", "LoadLibraryExW", "LocalFree", "MultiByteToWideChar", "RaiseException", "SetCurrentDirectoryW", "VirtualAlloc", "WideCharToMultiByte", "WriteFile", "lstrcatA", "lstrcmpA", "lstrcmpiA", "lstrcpyA", "lstrlenA"}; WindowsImports result{{".idata", sectionRva, SectionRead | SectionWrite | 0x40u, std::vector(40)}, {sectionRva, 40}, {}, {}}; auto& bytes = result.Section.Data; const auto lookupOffset = bytes.size(); diff --git a/core/relinker/elfpatcher/tests/WindowsDependencyDiagnosticsTests.cpp b/core/relinker/elfpatcher/tests/WindowsDependencyDiagnosticsTests.cpp index a5951091c..c1c649a8e 100644 --- a/core/relinker/elfpatcher/tests/WindowsDependencyDiagnosticsTests.cpp +++ b/core/relinker/elfpatcher/tests/WindowsDependencyDiagnosticsTests.cpp @@ -21,8 +21,9 @@ using namespace Elfpatcher::Windows; Fs::path createFixtureDirectory(const Fs::path& parent) { std::random_device random; for (int attempt = 0; attempt < 100; ++attempt) { - const auto directory = parent / ("windows-diagnostic-fixtures-" + - std::to_string(GetCurrentProcessId()) + "-" + std::to_string(random())); + auto name = Fs::path(u8"windows-diagnostic-fixtures-ü日-"); + name += std::to_string(GetCurrentProcessId()) + "-" + std::to_string(random()); + const auto directory = parent / name; if (Fs::create_directory(directory)) return directory; } @@ -172,15 +173,15 @@ void expectDiagnostic(const Fs::path& runner, const std::vector& ex HANDLE output = nullptr; if (!CreatePipe(&input, &output, &security, 0) || !SetHandleInformation(input, HANDLE_FLAG_INHERIT, 0)) throw std::runtime_error("Cannot create diagnostic test pipe"); - STARTUPINFOA startup{}; + STARTUPINFOW startup{}; startup.cb = sizeof(startup); startup.dwFlags = STARTF_USESTDHANDLES; startup.hStdOutput = output; startup.hStdError = output; startup.hStdInput = GetStdHandle(STD_INPUT_HANDLE); PROCESS_INFORMATION process{}; - const auto filename = runner.string(); - if (!CreateProcessA(filename.c_str(), nullptr, nullptr, nullptr, TRUE, CREATE_NO_WINDOW, nullptr, runner.parent_path().string().c_str(), &startup, &process)) + const auto filename = runner.wstring(); + if (!CreateProcessW(filename.c_str(), nullptr, nullptr, nullptr, TRUE, CREATE_NO_WINDOW, nullptr, runner.parent_path().wstring().c_str(), &startup, &process)) throw std::runtime_error("Cannot start diagnostic test: " + std::to_string(GetLastError())); CloseHandle(output); if (WaitForSingleObject(process.hProcess, 20000) != WAIT_OBJECT_0) { diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 30a3b9ab7..1f9822255 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -387,7 +387,6 @@ Throughout the project, every function at every stage either **does exactly what - [Packed f16 NaN results](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) (`v_pk_add/mul/fma/min/max_f16`) follow the hardware measured on RDNA2 (gfx1035) at IEEE=0 and IEEE=1 with f16 round-to-nearest-even and f16 denormals kept: with IEEE=1 a signaling NaN input is quieted and min/max return the other operand only for a quiet NaN; with IEEE=0 NaN inputs pass through unquieted and min/max ignore signaling. In `v_pk_fma_f16` an invalid product (inf * 0) gives the default NaN even when src2 is a NaN. A shader without a known float mode is translated as IEEE=0, the mode titles use; a mode with another f16 rounding or that flushes f16 denormals throws. - [f32 denormals](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follow the shader's `FLOAT_MODE` f32 denormal field (bits 5:4) when it flushes them (0, and shaders without a known float mode, as in the 0xc0 most titles use), measured on RDNA2 (gfx1035) over 1444 rows per instruction at IEEE=0 and IEEE=1. `v_add/sub/subrev/mul/mul_legacy/mullit/fma/fmac/fmamk/fmaak/min/max/min3/max3/med3/fract/floor/ceil/frexp_mant/sin/cos_f32` flush their f32 inputs and their result to a zero of the same sign (`v_mullit_f32` measured on RDNA2 (gfx1036) over 6907 rows per mode at IEEE=0 and IEEE=1, so a positive denormal src2 counts as zero); the f32 compares, `v_cvt_i32/u32/rpi_i32/flr_i32_f32`, `v_frexp_exp_i32_f32`, `v_cvt_f64_f32`, `v_cvt_f16_f32`, `v_cvt_pkrtz_f16_f32`, `v_cvt_pknorm_i16/u16_f32` and `v_cvt_pk_u8_f32` flush their f32 inputs. `v_div_fixup_f32` flushes its src1 and src2 and keeps a denormal src0 and result, measured on RDNA2 (gfx1036) over 6859 rows per mode at IEEE=0 and IEEE=1. A product is checked for tininess on its exact value: one below 2^-126 gives zero even when it rounds to 2^-126, computed exactly from the significands for `v_mul_f32`, `v_mul_legacy_f32`, `v_mullit_f32`, and for `v_fma_f32`, `v_fmac_f32`, `v_fmamk_f32`, `v_fmaak_f32` and `v_fma_mix_f32` when the addend is zero; with a nonzero addend the fused result is kept, so an exact result just below 2^-126 that rounds to it is not flushed. The hardware detects tininess after rounding to 24 bits with an unbounded exponent, measured on RDNA2 (gfx1036) for `v_mul_f32` and `v_mullit_f32` at IEEE=0 and IEEE=1: `0x3f7ffffe * 0x00800001` (2^-126 * (1 - 2^-46)) gives `0x00800000` there and zero here, while `0x3f7fffff * 0x00800000` (2^-126 * (1 - 2^-24)) gives zero on both. `v_fma_mix_f32` was measured with f32 sources only; its f16 sources are not flushed. `v_fma_mixlo_f16` and `v_fma_mixhi_f16` flush their f32 sources too and keep a product below 2^-126, measured on RDNA2 (gfx1036). `v_mad_legacy_f32` and `v_mac_legacy_f32` flush their inputs, their product and their result in every mode, see its entry. Denormal modes 1 and 2 throw for the other instructions. Not covered, and not measurable on RDNA2 (no such opcodes) or not measured: `v_mac_f32`, `v_mad_f32`, `v_madak_f32` and `v_madmk_f32` (rounded separately), `v_dot2_f32_f16`, `v_div_scale_f32` and `v_div_fmas_f32` (on hardware `v_div_fmas_f32` flushes only its result), f32 interpolation and `ds_add_f32` keep the previous lowering; `ds_min/max/cmpst_f32` flush both operands by the field, see the LDS float atomics entry. With f32 denormals kept (3) the results depend on the driver's default denormal behaviour, as no `DenormPreserve` execution mode is requested, and `v_ldexp_f32` flushes in every mode. - The executable file that [relinker](../../core/relinker/elfpatcher/src/windows/WindowsPeWriter.cpp) generates opens the console when launched, which is inconvenient for playability. -- The Windows [entry stub](../../core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp) (relinker) finds, enters and loads from its directory through the wide API, but prints library paths, and passes them to the `--windows-diagnostics` [dependency walker](../../core/relinker/elfpatcher/src/windows/WindowsDependencyStubBuilder.cpp), in the ANSI code page. The walker uses the ANSI API, so under a directory the code page cannot represent the paths it prints and checks contain `?`. - [Overlay output bus](../../core/libs/prx/libSceVideoOut/src/VideoOutDriver.cpp) (libSceVideoOut) - flips on a port opened with `SCE_VIDEO_OUT_BUS_TYPE_OVERLAY` complete and signal their events like main-bus flips, but their buffers are not shown: the overlay is not composited over the main output. Presenting them as whole frames made the window alternate between the two buses - [PA_CL_CLIP_CNTL](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) - `DX_LINEAR_ATTR_CLIP_ENA` (bit 24) clear is drawn like bit 24 set, with Vulkan's clipping of `NoPerspective` outputs; what a clear bit changes on the console is undocumented. - [10_11_11 unorm color targets](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) render into R32_UINT attachments: the pixel shader packs each export, rounded to nearest even like format-30 image stores, which is assumed to match the CB's conversion. From b1f0f1bff7fd26fb81d227c22f486436f86e243c Mon Sep 17 00:00:00 2001 From: Doston Abdullayev Date: Sat, 10 Oct 2026 15:32:46 +0500 Subject: [PATCH 143/460] test(relinker): convert the non-ASCII working directory case in an ASCII directory The relinker reads its arguments as narrow strings, so on a Windows system whose ANSI code page cannot represent the directory it cannot open the input, and the windows_working_directory case failed before reaching the executable. Convert the case under an ASCII name and rename the directory before starting the executable. Co-Authored-By: Claude Opus 5.5 --- .../relinker/tests/test_windows_working_directory.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/core/relinker/relinker/tests/test_windows_working_directory.py b/core/relinker/relinker/tests/test_windows_working_directory.py index 340afd953..c3cc6bf96 100644 --- a/core/relinker/relinker/tests/test_windows_working_directory.py +++ b/core/relinker/relinker/tests/test_windows_working_directory.py @@ -62,7 +62,7 @@ def spinning_provider(): return image -def check(relinker, work, name, extra): +def check(relinker, work, name, extra, renamed=None): case = work / name / 'app' (case / 'prx').mkdir(parents=True) (case / 'prx' / 'a.prx').write_bytes(spinning_provider()) @@ -74,6 +74,11 @@ def check(relinker, work, name, extra): assert result.returncode == 0, (result.stdout, result.stderr) if os.name != 'nt': return + if renamed is not None: + (work / name).rename(work / renamed) + name = renamed + case = work / name / 'app' + output = case / 'output.exe' elsewhere = work / name / 'elsewhere' elsewhere.mkdir() process = subprocess.Popen([str(output)], cwd=elsewhere, stdin=subprocess.DEVNULL, @@ -103,7 +108,7 @@ def main(): work = Path(directory) check(relinker, work, 'console', []) check(relinker, work, 'gui', ['--windows-gui']) - check(relinker, work, 'unicode ü日', []) + check(relinker, work, 'unicode', [], 'unicode ü日') print('Windows working directory integration tests passed') From 2f3f1a9c416ed41b0188dc8c0844b97862ff6c44 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 15:26:20 -0400 Subject: [PATCH 144/460] fix(shader): flush scalar f16 add/mul/sub denormals to signed zero Scalar f16 add/mul/sub ignore FLOAT_MODE denorm16 flush bits on main. Flush inputs when bit 0 is clear and results when bit 1 is clear at nearest-even. Multiply checks tininess after 11-bit rounding (0x387ff000); add/sub use 0x38800000. Co-Authored-By: Claude Opus 5.5 Co-authored-by: Cursor --- .../tests/execution/Float16Rounding.cpp | 160 +++++++++++++++++- .../Translation/src/FloatInstructions.cpp | 18 +- docs/dev/TechnicalDebt.md | 1 + 3 files changed, 171 insertions(+), 8 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/Float16Rounding.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/Float16Rounding.cpp index ec1c19913..6bc111aa6 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/Float16Rounding.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/Float16Rounding.cpp @@ -292,6 +292,145 @@ constexpr Vector Vectors[] = { {0x498f4a8bu, 0x6749af47u, 0xc69e8021u, {0xabcd7c00u, 0xabcdaf47u, 0xabcd0004u}}, }; +constexpr Vector DenormalVectors[4][32] = { + { + {0u, 0x00000401u, 0x000003ffu, {0u, 0xabcd0401u, 0xabcd0000u}}, + {0u, 0x00000400u, 0x00008001u, {0u, 0xabcd0400u, 0xabcd8000u}}, + {0u, 0x00000001u, 0x00000001u, {0u, 0xabcd0000u, 0xabcd0000u}}, + {0u, 0x000003ffu, 0x00000001u, {0u, 0xabcd0000u, 0xabcd0000u}}, + {0u, 0x00008001u, 0x00008001u, {0u, 0xabcd8000u, 0xabcd0000u}}, + {0u, 0x000003ffu, 0x00008001u, {0u, 0xabcd0000u, 0xabcd8000u}}, + {0u, 0x00000400u, 0x00008400u, {0u, 0xabcd0000u, 0xabcd8000u}}, + {0u, 0x00008400u, 0x00000001u, {0u, 0xabcd8400u, 0xabcd8000u}}, + {0u, 0x00003c00u, 0x00000001u, {0u, 0xabcd3c00u, 0xabcd0000u}}, + {0u, 0x0000bc00u, 0x00000001u, {0u, 0xabcdbc00u, 0xabcd8000u}}, + {0u, 0x00003c00u, 0x00008001u, {0u, 0xabcd3c00u, 0xabcd8000u}}, + {0u, 0x00003bffu, 0x00000400u, {0u, 0xabcd3bffu, 0xabcd0000u}}, + {0u, 0x00003c01u, 0x000003ffu, {0u, 0xabcd3c01u, 0xabcd0000u}}, + {0u, 0x0000bc01u, 0x000003ffu, {0u, 0xabcdbc01u, 0xabcd8000u}}, + {0u, 0x00003bffu, 0x00008400u, {0u, 0xabcd3bffu, 0xabcd8000u}}, + {0u, 0x00003800u, 0x00000400u, {0u, 0xabcd3800u, 0xabcd0000u}}, + {0u, 0x0000b800u, 0x00000400u, {0u, 0xabcdb800u, 0xabcd8000u}}, + {0u, 0x00003800u, 0x00000401u, {0u, 0xabcd3800u, 0xabcd0000u}}, + {0u, 0x00003800u, 0x000003ffu, {0u, 0xabcd3800u, 0xabcd0000u}}, + {0u, 0x00000400u, 0x00000400u, {0u, 0xabcd0800u, 0xabcd0000u}}, + {0u, 0x00003c00u, 0x00000400u, {0u, 0xabcd3c00u, 0xabcd0400u}}, + {0u, 0x00007c00u, 0x00000001u, {0u, 0xabcd7c00u, 0xabcdfe00u}}, + {0u, 0x0000fc00u, 0x00008001u, {0u, 0xabcdfc00u, 0xabcdfe00u}}, + {0u, 0x00000001u, 0x00007c00u, {0u, 0xabcd7c00u, 0xabcdfe00u}}, + {0u, 0x00000000u, 0x00008000u, {0u, 0xabcd0000u, 0xabcd8000u}}, + {0u, 0x00008000u, 0x00008000u, {0u, 0xabcd8000u, 0xabcd0000u}}, + {0u, 0x00003c01u, 0x0000bc00u, {0u, 0xabcd1400u, 0xabcdbc01u}}, + {0u, 0x00007bffu, 0x00003c01u, {0u, 0xabcd7bffu, 0xabcd7c00u}}, + {0u, 0x00007e55u, 0x00000001u, {0u, 0xabcd7e55u, 0xabcd7e55u}}, + {0u, 0x00000001u, 0x0000fe01u, {0u, 0xabcdfe01u, 0xabcdfe01u}}, + {0u, 0x00007c01u, 0x000003ffu, {0u, 0xabcd7e01u, 0xabcd7e01u}}, + {0u, 0x000003ffu, 0x0000fd23u, {0u, 0xabcdff23u, 0xabcdff23u}}, + }, + { + {0u, 0x00000401u, 0x000003ffu, {0u, 0xabcd0800u, 0xabcd0000u}}, + {0u, 0x00000400u, 0x00008001u, {0u, 0xabcd0000u, 0xabcd8000u}}, + {0u, 0x00000001u, 0x00000001u, {0u, 0xabcd0000u, 0xabcd0000u}}, + {0u, 0x000003ffu, 0x00000001u, {0u, 0xabcd0400u, 0xabcd0000u}}, + {0u, 0x00008001u, 0x00008001u, {0u, 0xabcd8000u, 0xabcd0000u}}, + {0u, 0x000003ffu, 0x00008001u, {0u, 0xabcd0000u, 0xabcd8000u}}, + {0u, 0x00000400u, 0x00008400u, {0u, 0xabcd0000u, 0xabcd8000u}}, + {0u, 0x00008400u, 0x00000001u, {0u, 0xabcd8000u, 0xabcd8000u}}, + {0u, 0x00003c00u, 0x00000001u, {0u, 0xabcd3c00u, 0xabcd0000u}}, + {0u, 0x0000bc00u, 0x00000001u, {0u, 0xabcdbc00u, 0xabcd8000u}}, + {0u, 0x00003c00u, 0x00008001u, {0u, 0xabcd3c00u, 0xabcd8000u}}, + {0u, 0x00003bffu, 0x00000400u, {0u, 0xabcd3bffu, 0xabcd0000u}}, + {0u, 0x00003c01u, 0x000003ffu, {0u, 0xabcd3c01u, 0xabcd0400u}}, + {0u, 0x0000bc01u, 0x000003ffu, {0u, 0xabcdbc01u, 0xabcd8400u}}, + {0u, 0x00003bffu, 0x00008400u, {0u, 0xabcd3bffu, 0xabcd8000u}}, + {0u, 0x00003800u, 0x00000400u, {0u, 0xabcd3800u, 0xabcd0000u}}, + {0u, 0x0000b800u, 0x00000400u, {0u, 0xabcdb800u, 0xabcd8000u}}, + {0u, 0x00003800u, 0x00000401u, {0u, 0xabcd3800u, 0xabcd0000u}}, + {0u, 0x00003800u, 0x000003ffu, {0u, 0xabcd3800u, 0xabcd0000u}}, + {0u, 0x00000400u, 0x00000400u, {0u, 0xabcd0800u, 0xabcd0000u}}, + {0u, 0x00003c00u, 0x00000400u, {0u, 0xabcd3c00u, 0xabcd0400u}}, + {0u, 0x00007c00u, 0x00000001u, {0u, 0xabcd7c00u, 0xabcd7c00u}}, + {0u, 0x0000fc00u, 0x00008001u, {0u, 0xabcdfc00u, 0xabcd7c00u}}, + {0u, 0x00000001u, 0x00007c00u, {0u, 0xabcd7c00u, 0xabcd7c00u}}, + {0u, 0x00000000u, 0x00008000u, {0u, 0xabcd0000u, 0xabcd8000u}}, + {0u, 0x00008000u, 0x00008000u, {0u, 0xabcd8000u, 0xabcd0000u}}, + {0u, 0x00003c01u, 0x0000bc00u, {0u, 0xabcd1400u, 0xabcdbc01u}}, + {0u, 0x00007bffu, 0x00003c01u, {0u, 0xabcd7bffu, 0xabcd7c00u}}, + {0u, 0x00007e55u, 0x00000001u, {0u, 0xabcd7e55u, 0xabcd7e55u}}, + {0u, 0x00000001u, 0x0000fe01u, {0u, 0xabcdfe01u, 0xabcdfe01u}}, + {0u, 0x00007c01u, 0x000003ffu, {0u, 0xabcd7e01u, 0xabcd7e01u}}, + {0u, 0x000003ffu, 0x0000fd23u, {0u, 0xabcdff23u, 0xabcdff23u}}, + }, + { + {0u, 0x00000401u, 0x000003ffu, {0u, 0xabcd0401u, 0xabcd0000u}}, + {0u, 0x00000400u, 0x00008001u, {0u, 0xabcd0400u, 0xabcd8000u}}, + {0u, 0x00000001u, 0x00000001u, {0u, 0xabcd0000u, 0xabcd0000u}}, + {0u, 0x000003ffu, 0x00000001u, {0u, 0xabcd0000u, 0xabcd0000u}}, + {0u, 0x00008001u, 0x00008001u, {0u, 0xabcd8000u, 0xabcd0000u}}, + {0u, 0x000003ffu, 0x00008001u, {0u, 0xabcd0000u, 0xabcd8000u}}, + {0u, 0x00000400u, 0x00008400u, {0u, 0xabcd0000u, 0xabcd8000u}}, + {0u, 0x00008400u, 0x00000001u, {0u, 0xabcd8400u, 0xabcd8000u}}, + {0u, 0x00003c00u, 0x00000001u, {0u, 0xabcd3c00u, 0xabcd0000u}}, + {0u, 0x0000bc00u, 0x00000001u, {0u, 0xabcdbc00u, 0xabcd8000u}}, + {0u, 0x00003c00u, 0x00008001u, {0u, 0xabcd3c00u, 0xabcd8000u}}, + {0u, 0x00003bffu, 0x00000400u, {0u, 0xabcd3bffu, 0xabcd0400u}}, + {0u, 0x00003c01u, 0x000003ffu, {0u, 0xabcd3c01u, 0xabcd0000u}}, + {0u, 0x0000bc01u, 0x000003ffu, {0u, 0xabcdbc01u, 0xabcd8000u}}, + {0u, 0x00003bffu, 0x00008400u, {0u, 0xabcd3bffu, 0xabcd8400u}}, + {0u, 0x00003800u, 0x00000400u, {0u, 0xabcd3800u, 0xabcd0200u}}, + {0u, 0x0000b800u, 0x00000400u, {0u, 0xabcdb800u, 0xabcd8200u}}, + {0u, 0x00003800u, 0x00000401u, {0u, 0xabcd3800u, 0xabcd0200u}}, + {0u, 0x00003800u, 0x000003ffu, {0u, 0xabcd3800u, 0xabcd0000u}}, + {0u, 0x00000400u, 0x00000400u, {0u, 0xabcd0800u, 0xabcd0000u}}, + {0u, 0x00003c00u, 0x00000400u, {0u, 0xabcd3c00u, 0xabcd0400u}}, + {0u, 0x00007c00u, 0x00000001u, {0u, 0xabcd7c00u, 0xabcdfe00u}}, + {0u, 0x0000fc00u, 0x00008001u, {0u, 0xabcdfc00u, 0xabcdfe00u}}, + {0u, 0x00000001u, 0x00007c00u, {0u, 0xabcd7c00u, 0xabcdfe00u}}, + {0u, 0x00000000u, 0x00008000u, {0u, 0xabcd0000u, 0xabcd8000u}}, + {0u, 0x00008000u, 0x00008000u, {0u, 0xabcd8000u, 0xabcd0000u}}, + {0u, 0x00003c01u, 0x0000bc00u, {0u, 0xabcd1400u, 0xabcdbc01u}}, + {0u, 0x00007bffu, 0x00003c01u, {0u, 0xabcd7bffu, 0xabcd7c00u}}, + {0u, 0x00007e55u, 0x00000001u, {0u, 0xabcd7e55u, 0xabcd7e55u}}, + {0u, 0x00000001u, 0x0000fe01u, {0u, 0xabcdfe01u, 0xabcdfe01u}}, + {0u, 0x00007c01u, 0x000003ffu, {0u, 0xabcd7e01u, 0xabcd7e01u}}, + {0u, 0x000003ffu, 0x0000fd23u, {0u, 0xabcdff23u, 0xabcdff23u}}, + }, + { + {0u, 0x00000401u, 0x000003ffu, {0u, 0xabcd0800u, 0xabcd0000u}}, + {0u, 0x00000400u, 0x00008001u, {0u, 0xabcd03ffu, 0xabcd8000u}}, + {0u, 0x00000001u, 0x00000001u, {0u, 0xabcd0002u, 0xabcd0000u}}, + {0u, 0x000003ffu, 0x00000001u, {0u, 0xabcd0400u, 0xabcd0000u}}, + {0u, 0x00008001u, 0x00008001u, {0u, 0xabcd8002u, 0xabcd0000u}}, + {0u, 0x000003ffu, 0x00008001u, {0u, 0xabcd03feu, 0xabcd8000u}}, + {0u, 0x00000400u, 0x00008400u, {0u, 0xabcd0000u, 0xabcd8000u}}, + {0u, 0x00008400u, 0x00000001u, {0u, 0xabcd83ffu, 0xabcd8000u}}, + {0u, 0x00003c00u, 0x00000001u, {0u, 0xabcd3c00u, 0xabcd0001u}}, + {0u, 0x0000bc00u, 0x00000001u, {0u, 0xabcdbc00u, 0xabcd8001u}}, + {0u, 0x00003c00u, 0x00008001u, {0u, 0xabcd3c00u, 0xabcd8001u}}, + {0u, 0x00003bffu, 0x00000400u, {0u, 0xabcd3bffu, 0xabcd0400u}}, + {0u, 0x00003c01u, 0x000003ffu, {0u, 0xabcd3c01u, 0xabcd0400u}}, + {0u, 0x0000bc01u, 0x000003ffu, {0u, 0xabcdbc01u, 0xabcd8400u}}, + {0u, 0x00003bffu, 0x00008400u, {0u, 0xabcd3bffu, 0xabcd8400u}}, + {0u, 0x00003800u, 0x00000400u, {0u, 0xabcd3800u, 0xabcd0200u}}, + {0u, 0x0000b800u, 0x00000400u, {0u, 0xabcdb800u, 0xabcd8200u}}, + {0u, 0x00003800u, 0x00000401u, {0u, 0xabcd3800u, 0xabcd0200u}}, + {0u, 0x00003800u, 0x000003ffu, {0u, 0xabcd3800u, 0xabcd0200u}}, + {0u, 0x00000400u, 0x00000400u, {0u, 0xabcd0800u, 0xabcd0000u}}, + {0u, 0x00003c00u, 0x00000400u, {0u, 0xabcd3c00u, 0xabcd0400u}}, + {0u, 0x00007c00u, 0x00000001u, {0u, 0xabcd7c00u, 0xabcd7c00u}}, + {0u, 0x0000fc00u, 0x00008001u, {0u, 0xabcdfc00u, 0xabcd7c00u}}, + {0u, 0x00000001u, 0x00007c00u, {0u, 0xabcd7c00u, 0xabcd7c00u}}, + {0u, 0x00000000u, 0x00008000u, {0u, 0xabcd0000u, 0xabcd8000u}}, + {0u, 0x00008000u, 0x00008000u, {0u, 0xabcd8000u, 0xabcd0000u}}, + {0u, 0x00003c01u, 0x0000bc00u, {0u, 0xabcd1400u, 0xabcdbc01u}}, + {0u, 0x00007bffu, 0x00003c01u, {0u, 0xabcd7bffu, 0xabcd7c00u}}, + {0u, 0x00007e55u, 0x00000001u, {0u, 0xabcd7e55u, 0xabcd7e55u}}, + {0u, 0x00000001u, 0x0000fe01u, {0u, 0xabcdfe01u, 0xabcdfe01u}}, + {0u, 0x00007c01u, 0x000003ffu, {0u, 0xabcd7e01u, 0xabcd7e01u}}, + {0u, 0x000003ffu, 0x0000fd23u, {0u, 0xabcdff23u, 0xabcdff23u}}, + } +}; + constexpr std::array Names{"v_cvt_f16_f32", "v_add_f16", "v_mul_f16"}; std::array BufferDescriptor(const void* data, std::uint32_t count) { @@ -313,10 +452,10 @@ bool Matches(std::uint32_t actual, std::uint32_t expected) { return actual == expected || ((actual >> 16u) == (expected >> 16u) && IsNan16(actual) && IsNan16(expected)); } -void Run(AgcDriver::VulkanDevice& device, std::uint32_t first, std::uint32_t count) { +void Run(AgcDriver::VulkanDevice& device, std::uint32_t first, std::uint32_t count, std::span vectors = Vectors, std::uint32_t denorm16 = 4u, bool ieee = true, bool overflow = false) { Input.fill(0u); for (std::uint32_t lane = 0; lane < count; ++lane) { - const auto& vector = Vectors[first + lane]; + const auto& vector = vectors[first + lane]; Input[lane * Inputs] = vector.a; Input[lane * Inputs + 1] = vector.b; Input[lane * Inputs + 2] = vector.c; @@ -336,16 +475,17 @@ void Run(AgcDriver::VulkanDevice& device, std::uint32_t first, std::uint32_t cou device.Target(), {0, 0, 0, 128} }; + if (denorm16 < 4u) request.context.floatMode = ShaderRecompiler::ShaderFloatMode{denorm16 << 6u, true, ieee, overflow}; request.useCache = false; const auto result = ShaderRecompiler::Recompile(request); device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); device.WaitIdle(); } -void Check(std::uint32_t first, std::uint32_t count) { +void Check(std::uint32_t first, std::uint32_t count, std::span vectors = Vectors, std::uint32_t firstColumn = 0u) { for (std::uint32_t lane = 0; lane < count; ++lane) { - const auto& vector = Vectors[first + lane]; - for (std::uint32_t column = 0; column < Names.size(); ++column) { + const auto& vector = vectors[first + lane]; + for (std::uint32_t column = firstColumn; column < Names.size(); ++column) { const auto expected = vector.expected[column]; const auto actual = Output[lane * Results + column]; Require(Matches(actual, expected), std::string("f16 rounding: vector ") + std::to_string(first + lane) + " (" + Hex(vector.a) + ", " + Hex(vector.b) + ", " + Hex(vector.c) + ") " + Names[column] + " is " + Hex(actual) + ", expected " + Hex(expected)); @@ -365,6 +505,16 @@ int main() { Run(*device, first, count); Check(first, count); } + for (bool ieee : {false, true}) { + for (bool overflow : {false, true}) { + for (std::uint32_t denorm16 = 0u; denorm16 < 4u; ++denorm16) { + auto vectors = std::to_array(DenormalVectors[denorm16]); + if (overflow) vectors[27].expected[2] = 0xabcd7bffu; + Run(*device, 0u, 32u, vectors, denorm16, ieee, overflow); + Check(0u, 32u, vectors, 1u); + } + } + } std::puts("f16 rounding tests passed"); return 0; } catch (const std::exception& error) { diff --git a/core/shader/recompiler/Translation/src/FloatInstructions.cpp b/core/shader/recompiler/Translation/src/FloatInstructions.cpp index b85979973..e32e9a6e2 100644 --- a/core/shader/recompiler/Translation/src/FloatInstructions.cpp +++ b/core/shader/recompiler/Translation/src/FloatInstructions.cpp @@ -211,9 +211,21 @@ bool TranslationContext::vDivFixupF16(const RdnaInstruction& inst) { } bool TranslationContext::float16Binary(const RdnaInstruction& inst, IrOpcode opcode, bool reverse) { - const IrF32 lhs = readF16AsF32(sourceAt(inst, reverse ? 1u : 0u)); - const IrF32 rhs = readF16AsF32(sourceAt(inst, reverse ? 0u : 1u)); - writeF16(inst.destination, IrF32(ir.Emit(opcode, IrType::F32, {&lhs.Value(), &rhs.Value()})), {&lhs.Value(), &rhs.Value()}); + const bool measured = (opcode == IrOpcode::FPAdd32 || opcode == IrOpcode::FPSub32 || opcode == IrOpcode::FPMul32) && floatMode.has_value() && (floatMode->floatMode & 0x0cu) == 0u; + const auto flush = [&](IrF32 value, std::uint32_t threshold) { + const IrU32 bits(ir.BitCastU32(value.Value())); + const IrU1 tiny(ir.ULessThan(ir.BitwiseAnd(bits.Value(), ir.Constant(0x7fffffffu)), ir.Constant(threshold))); + return IrF32(ir.BitCastF32(ir.Select(tiny.Value(), ir.BitwiseAnd(bits.Value(), ir.Constant(0x80000000u)), bits.Value()))); + }; + const auto read = [&](const RdnaOperand& operand) { + const IrF32 value = readF16AsF32(operand); + return measured && (floatMode->floatMode & 0x40u) == 0u ? flush(value, 0x38800000u) : value; + }; + const IrF32 lhs = read(sourceAt(inst, reverse ? 1u : 0u)); + const IrF32 rhs = read(sourceAt(inst, reverse ? 0u : 1u)); + IrF32 result(ir.Emit(opcode, IrType::F32, {&lhs.Value(), &rhs.Value()})); + if (measured && (floatMode->floatMode & 0x80u) == 0u) result = flush(result, opcode == IrOpcode::FPMul32 ? 0x387ff000u : 0x38800000u); + writeF16(inst.destination, result, {&lhs.Value(), &rhs.Value()}); return true; } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 1f9822255..7af580ea2 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -384,6 +384,7 @@ Throughout the project, every function at every stage either **does exactly what - [v_mad_legacy_f32 and v_mac_legacy_f32](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) (VOP3 0x140, VOP2 0x06) round the product to f32 before the add, give a +0 product for a zero factor, and flush their f32 denormal sources, their product and their result to a zero of the same sign in every f32 denormal mode, an exception to the `FLOAT_MODE` model in [f32DenormalFlushFor](../../core/shader/recompiler/Translation/src/VectorInstructions.cpp). Measured on gfx1010 (RX 5700 XT) with `tools/hw-oracle/template.s`, assembled by the AMD Windows driver's `amd_comgr_2.dll` and launched through its `amdhip64_6.dll` instead of `oracle.c` and ROCr, over the 32 rows of the five `FmaLegacy.cpp` and the four `MacLegacy.cpp` encodings: identical in wave32 and wave64 at IEEE 0 and 1 with f32 denormal modes 0 to 3, except that the two signaling NaN rows are quieted only with IEEE=1 (#1281, #1290). A sum of two zeros takes its sign on the bits (+0 unless both are -0), as lavapipe returned -0 for +0 + -0. The PS5's gfx1013 is not measured; LLVM decodes the encoding as `v_mad_legacy_f32` there, as on gfx1010. RDNA2 differs: on gfx1031 (RX 6700 XT, measured by pandather in #1281) the encoding is `v_fma_legacy_f32` and fused, with the same zero rule and denormal-source flush, so an RDNA2 measurement is not evidence for the console. - [v_fma_mixlo_f16 and v_fma_mixhi_f16](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) round the exact product-sum once to f16 with the f64 arithmetic of v_interp_p2_f16, so they need the Float64 capability. On RDNA2 (gfx1036) they follow the `FLOAT_MODE` f16 round and denormal fields, not the f32 round field: -1.5 * 2^-24 + 2^-126 from f32 sources (0xbfc00000, 0x33800000, 0x00800000) gives 0x8001 with f16 denormals kept and round to nearest even, 0x8000 with f16 denormals flushed and 0x8002 with f16 round toward -infinity. Only f16 denormals kept with round to nearest even is translated; another f16 mode throws, as for packed f16. `FP16_OVFL` set is not measured. - [Signaling NaN inputs](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) of the f32 and f16 unary instructions (`v_sqrt/rcp/rsq/log/exp/sin/cos/fract/floor/ceil/trunc/rndne/rcp_iflag/frexp_mant`), `v_ldexp_f32/f16`, `v_frexp_mant_f16`, `v_div_fixup_f32/f16` and `v_mullit_f32` are returned quieted only with IEEE=1 and unchanged with IEEE=0, measured on RDNA2 (gfx1035) in both modes, and so are those of the f64 forms (`v_sqrt/rcp/rsq/fract/floor/ceil/trunc/rndne/frexp_mant/ldexp/div_fixup_f64`) and `v_cvt_f64_f32`, measured on RDNA2 (gfx1036) in both modes; shaders without a known float mode follow IEEE=0. `v_pack_b32_f16` quiets each signaling NaN half by the same rule, measured on RDNA2 (gfx1036) in both modes. A signaling NaN result of `v_cubesc/cubetc/cubema_f32` is quieted by the same rule and keeps the sign the instruction gives it, measured on RDNA2 (gfx1036) in both modes. `v_add/sub/subrev/mul/fma/fmac/fmamk/fmaak/mul_legacy_f32` and `v_fma_mix_f32` with f32 sources choose their NaN result on the bits, measured over 1000 operand triples in both modes: the first NaN source in operand order after its modifiers (the subtrahend negated, as the hardware adds its negation), quieted only with IEEE=1, or the default NaN 0xffc00000 for an invalid operation; in the fma forms an invalid product (inf * 0) gives the default NaN even when src2 is a NaN, and in the legacy forms a zero factor makes the product 0 even when the other factor is a NaN (a NaN src2 still wins). `v_mad_legacy_f32` and `v_mac_legacy_f32` follow the same rule with their separately rounded product, as measured on gfx1010 (#1281, #1290): a NaN source quieted only with IEEE=1, inf - inf giving 0xffc00000, the zero rule hiding a NaN factor and a NaN src2 winning. The f16 and f64 arithmetic, `v_fma_mix_f32` with f16 sources and the separately rounded mad/mac forms (not measurable on RDNA2) follow the same rule on hardware but their NaN bits are left to the driver. `v_cvt_f16_f32`, `v_cvt_f32_f64`, `v_dot2_f32_f16`, `v_fma_mixlo_f16` and `v_fma_mixhi_f16` always quiet and `v_div_scale_f32` never does: the f32 signaling NaN 0x7f812345 as src0 or src2 gives 0x7e09 from mixlo and mixhi at IEEE=0 and IEEE=1 on RDNA2 (gfx1036), where `v_fma_mix_f32` gives 0x7f812345 at IEEE=0 and 0x7fc12345 at IEEE=1. The payload of an f16 NaN source of mixlo and mixhi is left to the driver's f16 to f32 conversion, which NVIDIA replaces. +- [Scalar f16 add, sub, subrev and mul denormals](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) follow the `FLOAT_MODE` f16 denormal field at nearest-even: bit 0 preserves inputs and bit 1 preserves the result; otherwise a denormal is flushed to its signed zero. Measured on RDNA2 (gfx1036) with IEEE=0/1 and FP16_OVFL=0/1, over 32 operand pairs in all four denormal modes, including signed zeros, denormals, minimum normals, infinities and NaNs. Multiply detects tininess after rounding to 11 significant bits with an unbounded exponent, before gradual half packing: `0x3bff * 0x0400` flushes, while `0x3c01 * 0x03ff` rounds to `0x0400` and is kept. Directed rounding and ternary arithmetic keep their previous denormal lowering. Shaders without a known float mode keep the previous lowering. Scalar `v_sub_f16`/`v_subrev_f16` share `float16Binary` with add/mul and were measured on gfx1036 over the same operand set as add (1156 non-NaN-input rows per mode); NaN result bits still depend on the driver as recorded above. - [Packed f16 NaN results](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) (`v_pk_add/mul/fma/min/max_f16`) follow the hardware measured on RDNA2 (gfx1035) at IEEE=0 and IEEE=1 with f16 round-to-nearest-even and f16 denormals kept: with IEEE=1 a signaling NaN input is quieted and min/max return the other operand only for a quiet NaN; with IEEE=0 NaN inputs pass through unquieted and min/max ignore signaling. In `v_pk_fma_f16` an invalid product (inf * 0) gives the default NaN even when src2 is a NaN. A shader without a known float mode is translated as IEEE=0, the mode titles use; a mode with another f16 rounding or that flushes f16 denormals throws. - [f32 denormals](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follow the shader's `FLOAT_MODE` f32 denormal field (bits 5:4) when it flushes them (0, and shaders without a known float mode, as in the 0xc0 most titles use), measured on RDNA2 (gfx1035) over 1444 rows per instruction at IEEE=0 and IEEE=1. `v_add/sub/subrev/mul/mul_legacy/mullit/fma/fmac/fmamk/fmaak/min/max/min3/max3/med3/fract/floor/ceil/frexp_mant/sin/cos_f32` flush their f32 inputs and their result to a zero of the same sign (`v_mullit_f32` measured on RDNA2 (gfx1036) over 6907 rows per mode at IEEE=0 and IEEE=1, so a positive denormal src2 counts as zero); the f32 compares, `v_cvt_i32/u32/rpi_i32/flr_i32_f32`, `v_frexp_exp_i32_f32`, `v_cvt_f64_f32`, `v_cvt_f16_f32`, `v_cvt_pkrtz_f16_f32`, `v_cvt_pknorm_i16/u16_f32` and `v_cvt_pk_u8_f32` flush their f32 inputs. `v_div_fixup_f32` flushes its src1 and src2 and keeps a denormal src0 and result, measured on RDNA2 (gfx1036) over 6859 rows per mode at IEEE=0 and IEEE=1. A product is checked for tininess on its exact value: one below 2^-126 gives zero even when it rounds to 2^-126, computed exactly from the significands for `v_mul_f32`, `v_mul_legacy_f32`, `v_mullit_f32`, and for `v_fma_f32`, `v_fmac_f32`, `v_fmamk_f32`, `v_fmaak_f32` and `v_fma_mix_f32` when the addend is zero; with a nonzero addend the fused result is kept, so an exact result just below 2^-126 that rounds to it is not flushed. The hardware detects tininess after rounding to 24 bits with an unbounded exponent, measured on RDNA2 (gfx1036) for `v_mul_f32` and `v_mullit_f32` at IEEE=0 and IEEE=1: `0x3f7ffffe * 0x00800001` (2^-126 * (1 - 2^-46)) gives `0x00800000` there and zero here, while `0x3f7fffff * 0x00800000` (2^-126 * (1 - 2^-24)) gives zero on both. `v_fma_mix_f32` was measured with f32 sources only; its f16 sources are not flushed. `v_fma_mixlo_f16` and `v_fma_mixhi_f16` flush their f32 sources too and keep a product below 2^-126, measured on RDNA2 (gfx1036). `v_mad_legacy_f32` and `v_mac_legacy_f32` flush their inputs, their product and their result in every mode, see its entry. Denormal modes 1 and 2 throw for the other instructions. Not covered, and not measurable on RDNA2 (no such opcodes) or not measured: `v_mac_f32`, `v_mad_f32`, `v_madak_f32` and `v_madmk_f32` (rounded separately), `v_dot2_f32_f16`, `v_div_scale_f32` and `v_div_fmas_f32` (on hardware `v_div_fmas_f32` flushes only its result), f32 interpolation and `ds_add_f32` keep the previous lowering; `ds_min/max/cmpst_f32` flush both operands by the field, see the LDS float atomics entry. With f32 denormals kept (3) the results depend on the driver's default denormal behaviour, as no `DenormPreserve` execution mode is requested, and `v_ldexp_f32` flushes in every mode. - The executable file that [relinker](../../core/relinker/elfpatcher/src/windows/WindowsPeWriter.cpp) generates opens the console when launched, which is inconvenient for playability. From 92b20608e8d073a9e02c00fb47634df3fa082b58 Mon Sep 17 00:00:00 2001 From: mugurc Date: Sat, 10 Oct 2026 19:53:49 +0300 Subject: [PATCH 145/460] feat(relinker): convert for Rosetta with --to-rosetta Rosetta runs AVX2, FMA, BMI1/2, F16C and MOVBE, but, unlike an Intel host, it has neither RDSEED, CLWB nor RDPID. --to-rosetta converts everything --to-intel does and lowers these three as well: - RDSEED becomes RDRAND (the /6 form of the same opcode), which also returns a random value and sets CF. - CLWB only writes a cache line back, so it becomes a NOP of the same length. - RDPID becomes a load of 0 through an optional stub: Rosetta's TSC_AUX is a constant, not a processor index, so every thread reports processor 0. A site with no movable instruction after it for the jump is kept. The flag implies --to-intel and conflicts with --windows. It is independent of --macos: it only changes what the conversion produces, so --macos alone leaves these instructions as they are, as on an Intel Mac, and a title for Rosetta is relinked with --macos --to-rosetta. Intel output is unchanged: the target is a parameter of the converter, Intel by default. --- core/relinker/CMakeLists.txt | 2 + core/relinker/cli/include/Cli.hpp | 1 + core/relinker/cli/src/CliArgs.cpp | 7 ++- .../include/codegen/IAmd64OnlyConverter.hpp | 2 +- .../x86/Amd64OnlySubstitutionTable.hpp | 3 + .../x86/Amd64OnlySubstitutionTypes.hpp | 5 ++ .../codegen/x86/DecodedInstruction.hpp | 3 + .../x86/IAmd64OnlyInstructionMatcher.hpp | 2 +- .../codegen/src/Amd64OnlyConverter.cpp | 8 ++- .../src/x86/Amd64OnlyInstructionMatcher.cpp | 45 ++++++++++++-- .../codegen/src/x86/DecodedInstruction.cpp | 45 ++++++++++++++ .../codegen/tests/Amd64OnlyConverterTests.cpp | 52 ++++++++++++++++ core/relinker/main.cpp | 4 +- .../include/relinker/guest/GuestImage.hpp | 2 +- .../relinker/src/guest/GuestModuleBuilder.cpp | 4 +- .../relinker/tests/test_to_rosetta.py | 59 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + docs/user/USAGE.md | 3 +- 18 files changed, 232 insertions(+), 16 deletions(-) create mode 100644 core/relinker/relinker/tests/test_to_rosetta.py diff --git a/core/relinker/CMakeLists.txt b/core/relinker/CMakeLists.txt index 3de337d78..4bc292b5f 100644 --- a/core/relinker/CMakeLists.txt +++ b/core/relinker/CMakeLists.txt @@ -330,6 +330,8 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_entry_point_bounds.py $) add_test(NAME init_array_roots COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_init_array_roots.py $) + add_test(NAME to_rosetta COMMAND ${Python3_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_to_rosetta.py $) else() message(STATUS "Python3 interpreter not found; skipping Python relinker tests") endif() diff --git a/core/relinker/cli/include/Cli.hpp b/core/relinker/cli/include/Cli.hpp index 9e44a1d9a..38776589d 100644 --- a/core/relinker/cli/include/Cli.hpp +++ b/core/relinker/cli/include/Cli.hpp @@ -12,6 +12,7 @@ struct Args { bool skipSyscallCheck = false; bool skipSceModule = false; bool toIntel = false; + bool toRosetta = false; bool writeRegistry = false; bool toWindows = false; bool toMacos = false; diff --git a/core/relinker/cli/src/CliArgs.cpp b/core/relinker/cli/src/CliArgs.cpp index 1a0f1e00d..f2c6a95a6 100644 --- a/core/relinker/cli/src/CliArgs.cpp +++ b/core/relinker/cli/src/CliArgs.cpp @@ -8,7 +8,7 @@ namespace Cli { const char* Usage() { - return "Usage: relinker [--help] [--windows | --macos] [--windows-diagnostics] [--windows-gui] [--skip-syscall-check] [--skip-sce-module | --sce-module-path ] [--exclude-sce-module ]... [--to-intel] [unused-filter=0|1|2] [--registry] [--rpath ] [--lazy-binding] [--autorun] \n" + return "Usage: relinker [--help] [--windows | --macos] [--windows-diagnostics] [--windows-gui] [--skip-syscall-check] [--skip-sce-module | --sce-module-path ] [--exclude-sce-module ]... [--to-intel] [--to-rosetta] [unused-filter=0|1|2] [--registry] [--rpath ] [--lazy-binding] [--autorun] \n" "Example: relinker input.elf output.elf"; } @@ -36,6 +36,9 @@ Args ParseArgs(int argc, char* argv[]) { args.excludedSceModules.insert(argv[++i]); } else if (arg == "--to-intel") { args.toIntel = true; + } else if (arg == "--to-rosetta") { + args.toRosetta = true; + args.toIntel = true; } else if (arg.rfind("unused-filter=", 0) == 0) { const std::string value = arg.substr(14); if (unusedFilterSpecified || value.size() != 1 || value[0] < '0' || value[0] > '2') @@ -78,6 +81,8 @@ Args ParseArgs(int argc, char* argv[]) { if (args.toWindows && args.toMacos) throw std::runtime_error("--windows conflicts with --macos"); + if (args.toRosetta && args.toWindows) + throw std::runtime_error("--to-rosetta conflicts with --windows"); if (args.windowsDiagnostics && !args.toWindows) throw std::runtime_error("--windows-diagnostics requires --windows"); diff --git a/core/relinker/codegen/include/codegen/IAmd64OnlyConverter.hpp b/core/relinker/codegen/include/codegen/IAmd64OnlyConverter.hpp index 1bc42caf8..ca82119d7 100644 --- a/core/relinker/codegen/include/codegen/IAmd64OnlyConverter.hpp +++ b/core/relinker/codegen/include/codegen/IAmd64OnlyConverter.hpp @@ -18,7 +18,7 @@ public: ) const = 0; }; -std::unique_ptr MakeAmd64OnlyConverter(); +std::unique_ptr MakeAmd64OnlyConverter(Amd64OnlyTarget target = Amd64OnlyTarget::Intel); } diff --git a/core/relinker/codegen/include/codegen/x86/Amd64OnlySubstitutionTable.hpp b/core/relinker/codegen/include/codegen/x86/Amd64OnlySubstitutionTable.hpp index 8a8918671..b7ee6f081 100644 --- a/core/relinker/codegen/include/codegen/x86/Amd64OnlySubstitutionTable.hpp +++ b/core/relinker/codegen/include/codegen/x86/Amd64OnlySubstitutionTable.hpp @@ -38,6 +38,9 @@ inline constexpr Entry kMwaitx = {"MWAITX", nullptr, 0}; inline constexpr Entry kClzero = {"CLZERO", nullptr, 0}; inline constexpr Entry kRdpru = {"RDPRU", nullptr, 0}; inline constexpr Entry kMcommit = {"MCOMMIT", nullptr, 0}; +inline constexpr Entry kRdseed = {"RDSEED", nullptr, 0}; +inline constexpr Entry kRdpid = {"RDPID", nullptr, 0}; +inline constexpr Entry kClwb = {"CLWB", nullptr, 0}; inline constexpr Entry kExtrq = {"EXTRQ", nullptr, 0}; inline constexpr Entry kInsertq = {"INSERTQ", nullptr, 0}; inline constexpr Entry kExtrqRegisterForm = {"EXTRQ register form", nullptr, 0}; diff --git a/core/relinker/codegen/include/codegen/x86/Amd64OnlySubstitutionTypes.hpp b/core/relinker/codegen/include/codegen/x86/Amd64OnlySubstitutionTypes.hpp index 2b835fd94..bcb5df816 100644 --- a/core/relinker/codegen/include/codegen/x86/Amd64OnlySubstitutionTypes.hpp +++ b/core/relinker/codegen/include/codegen/x86/Amd64OnlySubstitutionTypes.hpp @@ -8,6 +8,11 @@ namespace Codegen { +enum class Amd64OnlyTarget : std::uint8_t { + Intel, + Rosetta +}; + enum class Amd64OnlyLowering : std::uint8_t { InPlace, Trampoline, diff --git a/core/relinker/codegen/include/codegen/x86/DecodedInstruction.hpp b/core/relinker/codegen/include/codegen/x86/DecodedInstruction.hpp index dc85cbcd4..798128e36 100644 --- a/core/relinker/codegen/include/codegen/x86/DecodedInstruction.hpp +++ b/core/relinker/codegen/include/codegen/x86/DecodedInstruction.hpp @@ -23,6 +23,9 @@ struct DecodedInstruction { [[nodiscard]] bool IsMcommit() const; [[nodiscard]] bool IsMovntss() const; [[nodiscard]] bool IsMovntsd() const; + [[nodiscard]] bool IsRdseed() const; + [[nodiscard]] bool IsRdpid() const; + [[nodiscard]] bool IsClwb() const; private: [[nodiscard]] std::size_t _skipPrefixesAndRex(bool* outHasOperandSizePrefix, bool* outHasRepnePrefix, bool* outHasRepPrefix) const; diff --git a/core/relinker/codegen/include/codegen/x86/IAmd64OnlyInstructionMatcher.hpp b/core/relinker/codegen/include/codegen/x86/IAmd64OnlyInstructionMatcher.hpp index 65d6f04c1..df17eb175 100644 --- a/core/relinker/codegen/include/codegen/x86/IAmd64OnlyInstructionMatcher.hpp +++ b/core/relinker/codegen/include/codegen/x86/IAmd64OnlyInstructionMatcher.hpp @@ -25,7 +25,7 @@ public: ) const = 0; }; -std::unique_ptr MakeAmd64OnlyInstructionMatcher(); +std::unique_ptr MakeAmd64OnlyInstructionMatcher(Amd64OnlyTarget target = Amd64OnlyTarget::Intel); } diff --git a/core/relinker/codegen/src/Amd64OnlyConverter.cpp b/core/relinker/codegen/src/Amd64OnlyConverter.cpp index 409b62e87..201630464 100644 --- a/core/relinker/codegen/src/Amd64OnlyConverter.cpp +++ b/core/relinker/codegen/src/Amd64OnlyConverter.cpp @@ -26,6 +26,8 @@ auto _atFileOffset(const Domain::FileByteOffset base, const TOperation& operatio class Amd64OnlyConverter : public IAmd64OnlyConverter { public: + explicit Amd64OnlyConverter(const Amd64OnlyTarget target) : _matcher(MakeAmd64OnlyInstructionMatcher(target)) {} + [[nodiscard]] ConvertResult Convert( std::vector fileBytes, const std::vector& codeSegments @@ -38,7 +40,7 @@ private: Amd64OnlyMatch Substitution; }; - std::unique_ptr _matcher = MakeAmd64OnlyInstructionMatcher(); + std::unique_ptr _matcher; std::unique_ptr _scanner = MakeInstructionScanner(); void _convertSegment( @@ -205,8 +207,8 @@ ConvertResult Amd64OnlyConverter::Convert( } -std::unique_ptr MakeAmd64OnlyConverter() { - return std::make_unique(); +std::unique_ptr MakeAmd64OnlyConverter(const Amd64OnlyTarget target) { + return std::make_unique(target); } } diff --git a/core/relinker/codegen/src/x86/Amd64OnlyInstructionMatcher.cpp b/core/relinker/codegen/src/x86/Amd64OnlyInstructionMatcher.cpp index caff05f46..35b1e0cc8 100644 --- a/core/relinker/codegen/src/x86/Amd64OnlyInstructionMatcher.cpp +++ b/core/relinker/codegen/src/x86/Amd64OnlyInstructionMatcher.cpp @@ -107,6 +107,19 @@ bool _isClzeroOpcode(const DecodedInstruction& instr) { return pos + 2 < instr.Length && instr.Data[pos] == X64OpcodeConstants::TwoByteOpcodeEscape && instr.Data[pos + 1] == X64OpcodeConstants::TwoByteGrp7 && instr.Data[pos + 2] == 0xFC; } +std::vector _rdpidReplacement(const DecodedInstruction& instr) { + using namespace X64OpcodeConstants; + const auto opcode = instr.OpcodeOffset(); + const auto rex = opcode > 0 && instr.Data[opcode - 1] >= RexMin && instr.Data[opcode - 1] <= RexMax ? instr.Data[opcode - 1] : 0; + const auto reg = static_cast((instr.Data[opcode + 2] & ModRmRmMask) | ((rex & 1) << 3)); + std::vector bytes; + if (reg >= 8) + bytes.push_back(0x41); + bytes.push_back(static_cast(0xB8 + (reg & 7))); + bytes.insert(bytes.end(), 4, 0); + return bytes; +} + bool _validWait(const DecodedInstruction& instr) { const auto opcode = instr.Data + instr.OpcodeOffset(); return instr.Length <= kMaxInstructionLength && std::find(instr.Data, opcode, X64OpcodeConstants::PrefixLock) == opcode; @@ -114,6 +127,8 @@ bool _validWait(const DecodedInstruction& instr) { class Amd64OnlyInstructionMatcher : public IAmd64OnlyInstructionMatcher { public: + explicit Amd64OnlyInstructionMatcher(const Amd64OnlyTarget target) : _target(target) {} + [[nodiscard]] std::optional Match( const std::uint8_t* data, std::size_t length, @@ -126,6 +141,7 @@ public: ) const override; private: + Amd64OnlyTarget _target; Sse4aLowering _lowering; Sha256Lowering _sha256Lowering; Sha1Lowering _sha1Lowering; @@ -137,6 +153,7 @@ private: [[nodiscard]] Amd64OnlyMatch _matchSha256(const DecodedInstruction& instr, std::span trailing) const; [[nodiscard]] Amd64OnlyMatch _matchSha1(const DecodedInstruction& instr, std::span trailing) const; [[nodiscard]] Amd64OnlyMatch _matchClzero(const DecodedInstruction& instr, std::span trailing) const; + [[nodiscard]] std::optional _matchRosetta(const DecodedInstruction& instr, std::span trailing) const; }; Amd64OnlyMatch Amd64OnlyInstructionMatcher::_matchMovnts(const DecodedInstruction& instr, const Entry& entry) const { @@ -177,6 +194,21 @@ Amd64OnlyMatch Amd64OnlyInstructionMatcher::_matchClzero(const DecodedInstructio return _trampoline(kClzero.Name, instr.Length, _outOfLine(instr.Length, trailing, [&](StubBodyBuilder& body) { _clzeroLowering.EmitOutOfLine(body, operands); })); } +std::optional Amd64OnlyInstructionMatcher::_matchRosetta(const DecodedInstruction& instr, std::span trailing) const { + if (_target != Amd64OnlyTarget::Rosetta) + return std::nullopt; + if (instr.IsRdseed()) { + std::vector replacement(instr.Data, instr.Data + instr.Length); + replacement[instr.OpcodeOffset() + 2] &= static_cast(~(1u << X64OpcodeConstants::ModRmRegShift)); + return Amd64OnlyMatch{kRdseed.Name, instr.Length, Amd64OnlyLowering::InPlace, std::move(replacement), {}, 0}; + } + if (instr.IsClwb()) + return _inPlace(kClwb, instr.Length, {}); + if (instr.IsRdpid()) + return _trampoline(kRdpid.Name, instr.Length, _outOfLine(instr.Length, trailing, [&](StubBodyBuilder& body) { body.Raw(_rdpidReplacement(instr)); }), true); + return std::nullopt; +} + std::optional Amd64OnlyInstructionMatcher::MatchSequence( std::span> instructions, std::span trailing @@ -211,13 +243,18 @@ std::optional Amd64OnlyInstructionMatcher::MatchSequence( if (name == nullptr) name = _reciprocalEntry(*reciprocal).Name; _reciprocalLowering.EmitOutOfLine(body, *reciprocal); + } else if (_target == Amd64OnlyTarget::Rosetta && instr.IsRdpid()) { + if (name == nullptr) + name = kRdpid.Name; + body.Raw(_rdpidReplacement(instr)); } else { return std::nullopt; } body.Advance(instr.Length); } _moveTrailing(body, trailing); - const bool optional = DecodeVexReciprocal(instructions.front().data(), instructions.front().size()).has_value(); + const DecodedInstruction first{instructions.front().data(), instructions.front().size()}; + const bool optional = DecodeVexReciprocal(first.Data, first.Length).has_value() || (_target == Amd64OnlyTarget::Rosetta && first.IsRdpid()); return _trampoline(name, instructions.front().size(), body.Finish(), optional); } @@ -268,13 +305,13 @@ std::optional Amd64OnlyInstructionMatcher::Match( if (instr.IsMcommit()) return _unsupported(kMcommit, length); - return std::nullopt; + return _matchRosetta(instr, trailing); } } -std::unique_ptr MakeAmd64OnlyInstructionMatcher() { - return std::make_unique(); +std::unique_ptr MakeAmd64OnlyInstructionMatcher(const Amd64OnlyTarget target) { + return std::make_unique(target); } } diff --git a/core/relinker/codegen/src/x86/DecodedInstruction.cpp b/core/relinker/codegen/src/x86/DecodedInstruction.cpp index 0c91e49ac..3bdb906ef 100644 --- a/core/relinker/codegen/src/x86/DecodedInstruction.cpp +++ b/core/relinker/codegen/src/x86/DecodedInstruction.cpp @@ -262,6 +262,51 @@ bool DecodedInstruction::IsMcommit() const { return Data[pos] == TwoByteOpcodeEscape && Data[pos + 1] == TwoByteGrp7 && Data[pos + 2] == 0xFA; } +bool DecodedInstruction::IsRdseed() const { + bool operandSizeOverride = false; + bool repnePrefix = false; + bool repPrefix = false; + const std::size_t pos = _skipPrefixesAndRex(&operandSizeOverride, &repnePrefix, &repPrefix); + + if (repnePrefix || repPrefix || pos + 2 >= Length) { + return false; + } + + const auto modrm = Data[pos + 2]; + return Data[pos] == TwoByteOpcodeEscape && Data[pos + 1] == TwoByteGrp9 && + ((modrm >> ModRmModShift) & ModRmModMask) == ModRmModRegister && ((modrm >> ModRmRegShift) & ModRmRegMask) == 7; +} + +bool DecodedInstruction::IsRdpid() const { + bool operandSizeOverride = false; + bool repnePrefix = false; + bool repPrefix = false; + const std::size_t pos = _skipPrefixesAndRex(&operandSizeOverride, &repnePrefix, &repPrefix); + + if (operandSizeOverride || repnePrefix || !repPrefix || pos + 2 >= Length) { + return false; + } + + const auto modrm = Data[pos + 2]; + return Data[pos] == TwoByteOpcodeEscape && Data[pos + 1] == TwoByteGrp9 && + ((modrm >> ModRmModShift) & ModRmModMask) == ModRmModRegister && ((modrm >> ModRmRegShift) & ModRmRegMask) == 7; +} + +bool DecodedInstruction::IsClwb() const { + bool operandSizeOverride = false; + bool repnePrefix = false; + bool repPrefix = false; + const std::size_t pos = _skipPrefixesAndRex(&operandSizeOverride, &repnePrefix, &repPrefix); + + if (!operandSizeOverride || repnePrefix || repPrefix || pos + 2 >= Length) { + return false; + } + + const auto modrm = Data[pos + 2]; + return Data[pos] == TwoByteOpcodeEscape && Data[pos + 1] == TwoByteGrp15 && + ((modrm >> ModRmModShift) & ModRmModMask) != ModRmModRegister && ((modrm >> ModRmRegShift) & ModRmRegMask) == 6; +} + bool DecodedInstruction::IsMovntss() const { bool operandSizeOverride = false; bool repnePrefix = false; diff --git a/core/relinker/codegen/tests/Amd64OnlyConverterTests.cpp b/core/relinker/codegen/tests/Amd64OnlyConverterTests.cpp index 236b8e9cd..dba57fd9f 100644 --- a/core/relinker/codegen/tests/Amd64OnlyConverterTests.cpp +++ b/core/relinker/codegen/tests/Amd64OnlyConverterTests.cpp @@ -649,6 +649,56 @@ void converterClzero() { require(failureOffset([&] { (void)converter->Convert(segmentRelative, {segmentHeader(text.size())}); }, "FS-relative CLZERO was accepted") == 0x20A, "CLZERO operand failure does not carry the file offset"); } +void rosettaSubstitutions() { + using Codegen::Amd64OnlyLowering; + const auto intel = Codegen::MakeAmd64OnlyInstructionMatcher(); + const auto rosetta = Codegen::MakeAmd64OnlyInstructionMatcher(Codegen::Amd64OnlyTarget::Rosetta); + const auto match = [](const auto& matcher, const Bytes& bytes) { return matcher->Match(bytes.data(), bytes.size()); }; + for (const auto& bytes : {Bytes{0x0F, 0xC7, 0xF8}, Bytes{0xF3, 0x0F, 0xC7, 0xF8}, Bytes{0x66, 0x0F, 0xAE, 0x30}}) + require(!match(intel, bytes).has_value(), "RDSEED, RDPID or CLWB was converted for an Intel host"); + const auto rdseed = match(rosetta, {0x48, 0x0F, 0xC7, 0xF9}); + require(rdseed && rdseed->Lowering == Amd64OnlyLowering::InPlace && rdseed->ReplacementBytes == Bytes{0x48, 0x0F, 0xC7, 0xF1} && rdseed->InstructionName == "RDSEED", "RDSEED was not rewritten to RDRAND"); + const auto rdseed16 = match(rosetta, {0x66, 0x41, 0x0F, 0xC7, 0xFA}); + require(rdseed16 && rdseed16->ReplacementBytes == Bytes{0x66, 0x41, 0x0F, 0xC7, 0xF2}, "16-bit RDSEED was not rewritten to RDRAND"); + require(!match(rosetta, {0x0F, 0xC7, 0x38}).has_value(), "VMPTRST (the memory form of 0F C7 /7) was taken for RDSEED"); + const auto clwb = match(rosetta, {0x66, 0x0F, 0xAE, 0x74, 0x24, 0x08}); + require(clwb && clwb->Lowering == Amd64OnlyLowering::InPlace && clwb->ReplacementBytes == Bytes{0x66, 0x0F, 0x1F, 0x44, 0x00, 0x00} && clwb->InstructionName == "CLWB", "CLWB was not replaced by a NOP"); + require(!match(rosetta, {0x66, 0x0F, 0xAE, 0xF1}).has_value(), "TPAUSE (the register form of 66 0F AE /6) was taken for CLWB"); + require(!match(rosetta, {0x66, 0x0F, 0xAE, 0x38}).has_value(), "CLFLUSHOPT was taken for CLWB"); + const auto rdpid = match(rosetta, {0xF3, 0x41, 0x0F, 0xC7, 0xFB}); + require(rdpid && rdpid->Lowering == Amd64OnlyLowering::Trampoline && rdpid->Optional && rdpid->InstructionName == "RDPID", "RDPID was not lowered through an optional stub"); + require(rdpid->ReturnBranchOffset == 6 && Bytes(rdpid->StubBody.begin(), rdpid->StubBody.begin() + 7) == Bytes{0x41, 0xBB, 0, 0, 0, 0, 0xE9}, "RDPID r11 stub does not load 0 before returning"); +} + +void converterRosetta() { + const auto converter = Codegen::MakeAmd64OnlyConverter(Codegen::Amd64OnlyTarget::Rosetta); + Bytes file(0x300, 0xCC); + const Bytes text = { + 0x0F, 0xC7, 0xF8, + 0xF3, 0x0F, 0xC7, 0xF9, + 0x48, 0x83, 0xC1, 0x01, + 0x66, 0x0F, 0xAE, 0x37, + 0xC3}; + std::copy(text.begin(), text.end(), file.begin() + 0x200); + const auto result = converter->Convert(file, {segmentHeader(text.size())}); + require(result.ReplacedCount == 2 && result.Trampolines.size() == 1 && result.KeptCount == 0 && result.Reports.size() == 3, "RDSEED, RDPID and CLWB were not converted for Rosetta"); + const auto& site = result.Trampolines[0]; + require(site.Offset == 0x203 && site.Length == 8 && result.Reports[1].InstructionName == "RDPID", "RDPID did not absorb the following instruction"); + const Bytes add = {0x48, 0x83, 0xC1, 0x01}; + require(Bytes(site.Body.begin(), site.Body.begin() + 5) == Bytes{0xB9, 0, 0, 0, 0} && Bytes(site.Body.begin() + static_cast(site.ReturnBranchOffset - add.size()), site.Body.begin() + static_cast(site.ReturnBranchOffset)) == add, "RDPID stub does not load 0 and then run the absorbed instruction"); + auto expected = file; + expected[0x202] = 0xF0; + const Bytes nop = {0x0F, 0x1F, 0x40, 0x00}; + std::copy(nop.begin(), nop.end(), expected.begin() + 0x20B); + require(result.Bytes == expected, "RDSEED or CLWB was replaced with the wrong bytes"); + auto beforeReturn = file; + beforeReturn[0x207] = 0xC3; + const auto kept = converter->Convert(beforeReturn, {segmentHeader(text.size())}); + require(kept.KeptCount == 1 && kept.Trampolines.empty() && kept.ReplacedCount == 2, "RDPID without room for a jump was not kept"); + const auto intel = Codegen::MakeAmd64OnlyConverter()->Convert(file, {segmentHeader(text.size())}); + require(intel.Reports.empty() && intel.Bytes == file, "The Intel target converted RDSEED, RDPID or CLWB"); +} + void reciprocalOperands() { const auto vex2 = Codegen::DecodeVexReciprocal(Bytes{0xC5, 0xF8, 0x52, 0xD5}.data(), 4); require(vex2 && vex2->Operation == Codegen::ReciprocalOperation::ReciprocalSquareRoot && vex2->Destination == 2 && vex2->Source == 5, "VEX2 VRSQRTPS was not decoded"); @@ -1340,6 +1390,8 @@ int main() { converterSha1(); converterMonitorWait(); converterClzero(); + rosettaSubstitutions(); + converterRosetta(); converterReciprocal(); converterStrayRex(); rewriterStrayRex(); diff --git a/core/relinker/main.cpp b/core/relinker/main.cpp index 87bb581ac..922290659 100644 --- a/core/relinker/main.cpp +++ b/core/relinker/main.cpp @@ -66,7 +66,7 @@ int Run(const int argc, char* argv[]) { std::vector trampolines; if (args.toIntel) { const auto codeSegments = Relinker::ElfReader(sourceBytes).ReadCodeSegments(); - auto converted = Codegen::MakeAmd64OnlyConverter()->Convert(std::move(sourceBytes), codeSegments); + auto converted = Codegen::MakeAmd64OnlyConverter(args.toRosetta ? Codegen::Amd64OnlyTarget::Rosetta : Codegen::Amd64OnlyTarget::Intel)->Convert(std::move(sourceBytes), codeSegments); sourceBytes = std::move(converted.Bytes); trampolines = std::move(converted.Trampolines); std::map stubsByName; @@ -108,7 +108,7 @@ int Run(const int argc, char* argv[]) { std::vector guestArtifacts; if (!args.skipSceModule) { - guestArtifacts = Relinker::GuestModuleBuilder().Build(Io::NativePath(args.inputPath), Io::NativePath(absPath), result.DynamicSection, args.toWindows, args.toMacos, args.toIntel, *syscallScanner, args.lazyBinding, args.runPath, args.excludedSceModules, Io::NativePath(args.sceModulePath)); + guestArtifacts = Relinker::GuestModuleBuilder().Build(Io::NativePath(args.inputPath), Io::NativePath(absPath), result.DynamicSection, args.toWindows, args.toMacos, args.toIntel, args.toRosetta, *syscallScanner, args.lazyBinding, args.runPath, args.excludedSceModules, Io::NativePath(args.sceModulePath)); } if (args.writeRegistry) { diff --git a/core/relinker/relinker/include/relinker/guest/GuestImage.hpp b/core/relinker/relinker/include/relinker/guest/GuestImage.hpp index e80610e72..8ba1296d6 100644 --- a/core/relinker/relinker/include/relinker/guest/GuestImage.hpp +++ b/core/relinker/relinker/include/relinker/guest/GuestImage.hpp @@ -58,7 +58,7 @@ struct GuestArtifact { class GuestModuleBuilder { public: - std::vector Build(const std::filesystem::path& inputPath, const std::filesystem::path& outputPath, Domain::SysVDynamicSection& dynamic, bool windows, bool macos, bool toIntel, ISyscallScanner& syscallScanner, bool lazyBinding, const std::string& runPath, const std::set& excludedModules, const std::filesystem::path& sceModulePath) const; + std::vector Build(const std::filesystem::path& inputPath, const std::filesystem::path& outputPath, Domain::SysVDynamicSection& dynamic, bool windows, bool macos, bool toIntel, bool toRosetta, ISyscallScanner& syscallScanner, bool lazyBinding, const std::string& runPath, const std::set& excludedModules, const std::filesystem::path& sceModulePath) const; }; } diff --git a/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp b/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp index 4f8bb12bd..e4ad42a72 100644 --- a/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp +++ b/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp @@ -50,7 +50,7 @@ std::string ModuleStem(std::string name, const bool windows) { } -std::vector GuestModuleBuilder::Build(const std::filesystem::path& inputPath, const std::filesystem::path& outputPath, Domain::SysVDynamicSection& dynamic, const bool windows, const bool macos, const bool toIntel, ISyscallScanner& syscallScanner, const bool lazyBinding, const std::string& runPath, const std::set& excludedModules, const std::filesystem::path& sceModulePath) const { +std::vector GuestModuleBuilder::Build(const std::filesystem::path& inputPath, const std::filesystem::path& outputPath, Domain::SysVDynamicSection& dynamic, const bool windows, const bool macos, const bool toIntel, const bool toRosetta, ISyscallScanner& syscallScanner, const bool lazyBinding, const std::string& runPath, const std::set& excludedModules, const std::filesystem::path& sceModulePath) const { const auto root = std::filesystem::absolute(sceModulePath).lexically_normal(); if (!std::filesystem::exists(root)) throw Domain::RelinkerException("Guest module parent directory does not exist: " + Io::Utf8Path(root)); if (!std::filesystem::is_directory(root)) throw Domain::RelinkerException("Guest module parent path is not a directory: " + Io::Utf8Path(root)); @@ -147,7 +147,7 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path std::vector codeHeaders; for (const auto& header : image.Headers) if (header.Type == 1 && (header.Flags & 1) != 0) codeHeaders.push_back(header); if (toIntel) { - auto converted = Codegen::MakeAmd64OnlyConverter()->Convert(std::move(image.Bytes), codeHeaders); + auto converted = Codegen::MakeAmd64OnlyConverter(toRosetta ? Codegen::Amd64OnlyTarget::Rosetta : Codegen::Amd64OnlyTarget::Intel)->Convert(std::move(image.Bytes), codeHeaders); image.Trampolines = std::move(converted.Trampolines); image.Bytes = std::move(converted.Bytes); } diff --git a/core/relinker/relinker/tests/test_to_rosetta.py b/core/relinker/relinker/tests/test_to_rosetta.py new file mode 100644 index 000000000..5d35e0186 --- /dev/null +++ b/core/relinker/relinker/tests/test_to_rosetta.py @@ -0,0 +1,59 @@ +"""Check that --to-rosetta lowers what Rosetta lacks, and that --to-intel and a plain relink leave it alone.""" + +from pathlib import Path +import struct +import subprocess +import sys +import tempfile + +from test_linux_load_alignment import fixture + +RDSEED = bytes.fromhex("0fc7f8") +RDRAND = bytes.fromhex("0fc7f0") +CLWB = bytes.fromhex("660fae37") +NOP4 = bytes.fromhex("0f1f4000") +SITES = RDSEED + CLWB + b"\xc3" +LOWERED = RDRAND + NOP4 + b"\xc3" + + +def executable(): + image = fixture() + struct.pack_into(" Date: Sat, 10 Oct 2026 14:39:53 -0500 Subject: [PATCH 146/460] fix(recompiler): read SRT words through a null user-data pointer as zero --- .../prx/libSceAgcDriver/tests/ShaderMemory.cpp | 14 +++++++++++--- .../IrMetadata/ResourcePlan.hpp | 1 + .../include/Optimization/SrtWalker.hpp | 2 +- .../SrtWalker/SrtDescriptorEvaluation.hpp | 2 +- .../Optimization/SrtWalker/SrtEvaluator.hpp | 2 ++ .../Optimization/src/ResourceMaterializer.cpp | 2 +- .../src/SrtWalker/SrtDescriptorEvaluation.cpp | 11 ++++++++++- .../Optimization/src/SrtWalker/SrtEvaluator.cpp | 6 ++++++ .../Optimization/src/SrtWalker/SrtWalker.cpp | 4 ++-- core/shader/recompiler/Recompiler.cpp | 2 +- docs/dev/TechnicalDebt.md | 3 ++- 11 files changed, 38 insertions(+), 11 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp b/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp index bc78344e5..232db3af5 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp @@ -2529,7 +2529,12 @@ void verifyGuardedNullPointers() { } NestedRequest rootless(branchCode, nullptr); AgcDriver::ShaderMemory rootlessMemory({}); - expectFailure([&] { static_cast(rootlessMemory.Capture(rootless.request)); }, "null or misaligned address", "guarded pointer: a null user-data pointer was accepted"); + const auto rootlessCapture = rootlessMemory.Capture(rootless.request); + require(rootlessCapture->snapshot.nullRootReads != 0u && GuardRecords(*rootlessCapture), "guarded pointer: reads through a null user-data pointer were not counted"); + require(std::all_of(rootlessCapture->snapshot.buffers.begin(), rootlessCapture->snapshot.buffers.end(), [](const DescriptorValue& value) { return std::all_of(value.dwords.begin(), value.dwords.end(), [](std::uint32_t word) { return word == 0u; }); }), "guarded pointer: a V# read through a null user-data pointer is not zero"); + rootless.request.context.memory = rootlessMemory.Regions(); + const auto rootlessResult = Recompile(rootless.request, *rootlessCapture); + require(rootlessResult->poisonedSrtReads >= rootlessCapture->snapshot.nullRootReads, "guarded pointer: a capture through a null user-data pointer is cacheable"); point(0u); #ifdef _WIN32 VirtualFree(reserved, 0, MEM_RELEASE); @@ -2788,8 +2793,11 @@ int main(int argc, char** argv) { request.context.memory = {}; const std::array nullUserData{}; request.context.userData = nullUserData; - AgcDriver::ShaderMemory invalid({}); - expectFailure([&] { invalid.Capture(request); }, "null or misaligned address", "null user-data pointer was accepted"); + AgcDriver::ShaderMemory nullRoot({}); + const auto nullRootCapture = nullRoot.Capture(request); + require(nullRootCapture->snapshot.nullRootReads != 0u, "reads through a null user-data pointer were not counted"); + request.context.memory = nullRoot.Regions(); + require(Recompile(request, *nullRootCapture)->poisonedSrtReads >= nullRootCapture->snapshot.nullRootReads, "a capture through a null user-data pointer is cacheable"); std::cout << "Shader memory capture, strict validation and deterministic replay passed\n"; return 0; } catch (const std::exception& error) { diff --git a/core/shader/recompiler/IntermediateRepresentation/include/IntermediateRepresentation/IrMetadata/ResourcePlan.hpp b/core/shader/recompiler/IntermediateRepresentation/include/IntermediateRepresentation/IrMetadata/ResourcePlan.hpp index 3f1b27f6d..cb9565fb4 100644 --- a/core/shader/recompiler/IntermediateRepresentation/include/IntermediateRepresentation/IrMetadata/ResourcePlan.hpp +++ b/core/shader/recompiler/IntermediateRepresentation/include/IntermediateRepresentation/IrMetadata/ResourcePlan.hpp @@ -42,6 +42,7 @@ struct ResourceSnapshot { std::vector userData; UniformFill uniformFill; std::vector srtPoison; + std::uint32_t nullRootReads = 0; }; struct UniformFillPlan { diff --git a/core/shader/recompiler/Optimization/include/Optimization/SrtWalker.hpp b/core/shader/recompiler/Optimization/include/Optimization/SrtWalker.hpp index 16cdedac5..4b80539b0 100644 --- a/core/shader/recompiler/Optimization/include/Optimization/SrtWalker.hpp +++ b/core/shader/recompiler/Optimization/include/Optimization/SrtWalker.hpp @@ -46,7 +46,7 @@ public: void EvaluateUniformValues(const IrResourcePlan& program, std::span values, const SrtRuntime& runtime, std::span results) const; void EvaluateDescriptorSource(const IrResourcePlan& program, std::uint32_t source, const SrtRuntime& runtime, DescriptorValue& result) const; void EvaluateDescriptorSources(const IrResourcePlan& program, std::span sources, const SrtRuntime& runtime, std::vector& results) const; - void EvaluateRuntimeSources(const IrResourcePlan& program, std::span sources, const SrtRuntime& runtime, std::vector& results, std::vector& flat, std::span cleanFlatSlots, std::vector& activeSources, std::vector* poison = nullptr) const; + void EvaluateRuntimeSources(const IrResourcePlan& program, std::span sources, const SrtRuntime& runtime, std::vector& results, std::vector& flat, std::span cleanFlatSlots, std::vector& activeSources, std::vector* poison = nullptr, std::uint32_t* nullRootReads = nullptr) const; void Walk(const IrResourcePlan& program, const SrtRuntime& runtime, std::vector& flat) const; }; diff --git a/core/shader/recompiler/Optimization/include/Optimization/SrtWalker/SrtDescriptorEvaluation.hpp b/core/shader/recompiler/Optimization/include/Optimization/SrtWalker/SrtDescriptorEvaluation.hpp index eb9c568ad..2b881a416 100644 --- a/core/shader/recompiler/Optimization/include/Optimization/SrtWalker/SrtDescriptorEvaluation.hpp +++ b/core/shader/recompiler/Optimization/include/Optimization/SrtWalker/SrtDescriptorEvaluation.hpp @@ -15,7 +15,7 @@ namespace ShaderRecompiler::Detail { // Why the last EvaluateRuntimeSourcesImpl call on this thread returned false. const std::string& RuntimeSourceFailureReason(); -bool EvaluateRuntimeSourcesImpl(const IrResourcePlan& program, std::span sources, const SrtRuntime& runtime, std::vector& results, std::vector& flat, bool evaluateFlat, std::span cleanFlatSlots, std::vector& activeSources, std::vector* poison = nullptr); +bool EvaluateRuntimeSourcesImpl(const IrResourcePlan& program, std::span sources, const SrtRuntime& runtime, std::vector& results, std::vector& flat, bool evaluateFlat, std::span cleanFlatSlots, std::vector& activeSources, std::vector* poison = nullptr, std::uint32_t* nullRootReads = nullptr); } diff --git a/core/shader/recompiler/Optimization/include/Optimization/SrtWalker/SrtEvaluator.hpp b/core/shader/recompiler/Optimization/include/Optimization/SrtWalker/SrtEvaluator.hpp index 269bfceb4..8332618a6 100644 --- a/core/shader/recompiler/Optimization/include/Optimization/SrtWalker/SrtEvaluator.hpp +++ b/core/shader/recompiler/Optimization/include/Optimization/SrtWalker/SrtEvaluator.hpp @@ -78,6 +78,7 @@ public: bool Evaluate(IrValue* value, std::uint32_t& result); bool EvaluateWide(IrValue* raw, std::uint64_t& result); void ReportInaccessibleReads(InaccessibleRead* sink) { _inaccessible = sink; } + void ReportNullRootReads(std::uint32_t* sink) { _nullRoots = sink; } private: static float Float32(std::uint64_t bits); @@ -95,6 +96,7 @@ private: Evaluator* _cleanEvaluator = nullptr; IrValue* _activeMask = nullptr; InaccessibleRead* _inaccessible = nullptr; + std::uint32_t* _nullRoots = nullptr; EvaluatedValues _cache; std::vector _visiting; }; diff --git a/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp b/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp index f1e3a5545..5eed221c8 100644 --- a/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp +++ b/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp @@ -444,7 +444,7 @@ void materializeSnapshot(const IrResourcePlan& plan, const SrtRuntime& runtime, std::vector values; std::vector activeSources; - walker.EvaluateRuntimeSources(plan, plan.materializationSources, runtime, values, snapshot.flattenedSrt, plan.cleanFlatSlots, activeSources, &snapshot.srtPoison); + walker.EvaluateRuntimeSources(plan, plan.materializationSources, runtime, values, snapshot.flattenedSrt, plan.cleanFlatSlots, activeSources, &snapshot.srtPoison, &snapshot.nullRootReads); std::size_t cursor = 0; if (values.size() < plan.info.buffers.size()) { diff --git a/core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp b/core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp index 752c49c96..bc89e71e9 100644 --- a/core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp +++ b/core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp @@ -45,7 +45,7 @@ const DescriptorSource* Source(const IrResourcePlan& program, std::uint32_t sour } -bool EvaluateRuntimeSourcesImpl(const IrResourcePlan& program, std::span sources, const SrtRuntime& runtime, std::vector& results, std::vector& flat, bool evaluateFlat, std::span cleanFlatSlots, std::vector& activeSources, std::vector* poison) { +bool EvaluateRuntimeSourcesImpl(const IrResourcePlan& program, std::span sources, const SrtRuntime& runtime, std::vector& results, std::vector& flat, bool evaluateFlat, std::span cleanFlatSlots, std::vector& activeSources, std::vector* poison, std::uint32_t* nullRootReads) { failureReason().clear(); static const bool debug = std::getenv("APS5_SRT_DEBUG") != nullptr; if (debug) { @@ -94,9 +94,14 @@ bool EvaluateRuntimeSourcesImpl(const IrResourcePlan& program, std::span> zeroedSources; std::vector evaluated; @@ -176,6 +181,10 @@ bool EvaluateRuntimeSourcesImpl(const IrResourcePlan& program, std::span sources, const SrtRuntime& runtime, std::vector& results, std::vector& flat, std::span cleanFlatSlots, std::vector& activeSources, std::vector* poison) const { - if (!Detail::EvaluateRuntimeSourcesImpl(program, sources, runtime, results, flat, true, cleanFlatSlots, activeSources, poison)) { +void SrtWalker::EvaluateRuntimeSources(const IrResourcePlan& program, std::span sources, const SrtRuntime& runtime, std::vector& results, std::vector& flat, std::span cleanFlatSlots, std::vector& activeSources, std::vector* poison, std::uint32_t* nullRootReads) const { + if (!Detail::EvaluateRuntimeSourcesImpl(program, sources, runtime, results, flat, true, cleanFlatSlots, activeSources, poison, nullRootReads)) { throw std::runtime_error("SrtWalker::EvaluateRuntimeSources failed to evaluate runtime sources: " + Detail::RuntimeSourceFailureReason()); } } diff --git a/core/shader/recompiler/Recompiler.cpp b/core/shader/recompiler/Recompiler.cpp index 430b18975..bbaece613 100644 --- a/core/shader/recompiler/Recompiler.cpp +++ b/core/shader/recompiler/Recompiler.cpp @@ -769,7 +769,7 @@ RecompileResult materializeResult(const CompiledVariant& variant, const Recompil result.workgroupMemoryDwords = WorkgroupMemoryStrideDwords(variant.info.info); result.bindings = std::move(bindings.bindings); result.pushConstants = std::move(bindings.pushConstants); - result.poisonedSrtReads = static_cast(snapshot.srtPoison.size()); + result.poisonedSrtReads = static_cast(snapshot.srtPoison.size()) + snapshot.nullRootReads; return result; } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index cfd755962..04bc76e56 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -356,7 +356,8 @@ Throughout the project, every function at every stage either **does exactly what - [Shader MODE register access](../../core/shader/recompiler/Translation/src/ScalarInstructions.cpp) reads the initial rounding fields from `GuestContext::floatMode`; missing metadata retains the legacy round-to-nearest-even assumption. Immediate rounding writes are accepted only when they preserve that state. State-changing or SGPR-sourced MODE writes throw until runtime mode tracking and affected arithmetic are implemented. Non-rounding register reads still throw; `s_setreg_b32` writes to other hardware registers are still ignored. Tests follow the public RDNA2/LLVM contract, not console measurements. - A [flat SRT read](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp) through a null or unmapped pointer loaded from memory raises a [BDA fault](../../core/libs/prx/libSceAgcDriver/Graphics/src/BdaResources.cpp) only if the program executes it, then stops or reads zero like a BDA read. - Repeated `s_load`s of one [flat SRT](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp) dword through a null or unmapped pointer report one of their pcs, and an `s_load` whose inaccessible dwords are all unused, or only address repeated loads, reports no fault of its own. -- A value read through a null or unmapped pointer still throws at record time if a [branch condition, uniform fill or table base](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp) needs it, or if the pointer comes from user data, even if the console skips the read. +- A value read through a null or unmapped pointer still throws at record time if a [branch condition, uniform fill or table base](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp) needs it, or if the pointer comes from user data and is not null, even if the console skips the read. +- A [V# or flat SRT word](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtEvaluator.cpp) read through a null pointer taken from user data is zero and counts as a [poisoned SRT read](../../core/shader/recompiler/Recompiler.cpp), so a dispatch that reads it is not cached. The console faults if the program executes the read; PPSA28416 branches around it with `s_cbranch_execz`. The fault is not modeled. - A [V#, T# or S#](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp) loaded through a null or unmapped pointer read from memory is zero when the work is recorded, and the capture throws if no flat SRT slot records that read. - [2D accesses through a 1D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) read the texel the 1D addressing selects and ignore the second coordinate, as simonbbby measured on RDNA2 (gfx1036) in #2237 for `image_sample_lz` and mip-less `image_load` with point filtering and wrap, clamp and border addressing; the PS5's gfx1013 is not measured. Other 2D accesses through a 1D T# (storage, depth, packed, BY loads, mips, derivatives, comparison) still throw. - On a host subgroup narrower than 32 lanes, [`threadBit`](../../core/shader/recompiler/Translation/src/TranslationContext/ExecMask.cpp) reads an EXEC written as a value (`s_mov_b32`/`s_mov_b64`) at bit (subgroup invocation id) of EXEC_LO, so lanes at or above the subgroup size read a lower lane's bit. From 9cb2bf8de507c64a158cfbdc5a9c416c3509884a Mon Sep 17 00:00:00 2001 From: Chad Date: Sat, 10 Oct 2026 14:58:18 -0500 Subject: [PATCH 147/460] fix(agc): stop the dispatch look-ahead before indirect arguments an earlier dispatch may write --- .../src/Driver/Dispatch/DispatchAhead.cpp | 23 +++++ .../IndirectArgumentsWrittenAhead.cpp | 99 +++++++++++++++++++ 2 files changed, 122 insertions(+) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/IndirectArgumentsWrittenAhead.cpp diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/DispatchAhead.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/DispatchAhead.cpp index 4867fef41..60d370b08 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/DispatchAhead.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/DispatchAhead.cpp @@ -82,6 +82,10 @@ void NoteLabel(std::vector& labels, std::uint64_t address, std::span regions, const std::vector& labels) { for (const auto& label : labels) { for (const auto& region : regions) { @@ -104,6 +108,7 @@ void Driver::resolveGroupAhead(const Submission& submission, const QueueState& l QueueState scratch = live; std::vector labels; std::vector> writes; + bool unknownWrites = false; for (const auto& label : deferredLabels().labels) NoteLabel(labels, label.address, std::span(label.bytes).first(label.size)); struct Resolving { std::size_t offset; @@ -130,6 +135,21 @@ void Driver::resolveGroupAhead(const Submission& submission, const QueueState& l if (!Pm4::WaitSatisfiedUnchecked(packet) && !WrittenByRun(packet, labels)) break; } else if (opcode == 0x15 || opcode == 0x16) { if (resolved.contains(cursor)) break; + if (opcode == 0x16) { + std::uint64_t arguments = 0; + try { + arguments = Pm4::DispatchArgumentAddress(packet, scratch); + } catch (const std::exception&) { + break; + } + const auto argumentsEnd = arguments + 3u * sizeof(std::uint32_t); + const bool written = std::any_of(writes.begin(), writes.end(), [&](const auto& range) { return range.first < argumentsEnd && arguments < range.second; }) + || std::any_of(labels.begin(), labels.end(), [&](const RunLabel& label) { return label.address < argumentsEnd && arguments < label.address + label.size; }); + if (unknownWrites || written) { + ++stats.writtenBefore; + break; + } + } currentPacketOffset() = cursor; try { if (opcode == 0x15) dispatch(scratch, packet, submission); @@ -150,6 +170,9 @@ void Driver::resolveGroupAhead(const Submission& submission, const QueueState& l break; } if (found->second.compiledResult != nullptr) appendWrittenRanges(*found->second.compiledResult, writes); + if (found->second.compiledResult == nullptr || WritesThroughPointers(*found->second.compiledResult)) unknownWrites = true; + } else { + unknownWrites = true; } } else if (opcode == 0x37 || opcode == 0x49) { const auto label = Pm4::DecodeLabelWrite(packet); diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/IndirectArgumentsWrittenAhead.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/IndirectArgumentsWrittenAhead.cpp new file mode 100644 index 000000000..ad4c1f5a4 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/IndirectArgumentsWrittenAhead.cpp @@ -0,0 +1,99 @@ +#include "VulkanTestDevice.hpp" +#include "prx/libSceAgcDriver/Execution/include/Driver.hpp" +#include "prx/libSceAgcDriver/Submit/include/Acb.hpp" +#include +#include +#include +#include +#include +#include +#include + +namespace { + +void Require(bool condition, const std::string& message) { + if (!condition) throw std::runtime_error(message); +} + +alignas(256) std::array WriterCode{0xf4080200u, 0xfa000000u, 0xbf8cc07fu, 0x7e000202u, 0x7e020203u, 0x7e040204u, 0xe07c0000u, 0x80020000u, 0xbf810000u}; +alignas(256) std::array CounterCode{0xf4080200u, 0xfa000000u, 0xbf8cc07fu, 0x7e000281u, 0xe0c80000u, 0x80020000u, 0xbf810000u}; + +alignas(256) std::array Arguments{1, 1, 1, 0}; +alignas(256) std::array ArgumentsDescriptor{}; +alignas(256) std::array Counter{}; +alignas(256) std::array CounterDescriptor{}; +alignas(256) volatile std::uint32_t Gate = 1; +alignas(256) volatile std::uint32_t Done = 0; + +std::uint32_t Low(const volatile void* pointer) { + return static_cast(reinterpret_cast(pointer)); +} + +std::uint32_t High(const volatile void* pointer) { + return static_cast(static_cast(reinterpret_cast(pointer)) >> 32u); +} + +std::array Descriptor(const void* pointer, std::uint32_t bytes) { + return {Low(pointer), High(pointer) & 0xffffu, bytes, 0x31016facu}; +} + +void SetShaderRegister(std::vector& words, std::uint32_t offset, std::uint32_t value) { + words.insert(words.end(), {0xc0017600u, offset, value}); +} + +void Program(std::vector& words, const void* code, std::initializer_list userData) { + const auto address = static_cast(reinterpret_cast(code)); + SetShaderRegister(words, 0x20c, static_cast(address >> 8u)); + SetShaderRegister(words, 0x20d, static_cast(address >> 40u) & 0xffu); + for (const std::uint32_t offset : {0x207u, 0x208u, 0x209u}) SetShaderRegister(words, offset, 1); + SetShaderRegister(words, 0x212, 0); + SetShaderRegister(words, 0x213, static_cast(userData.size()) << 1u); + std::uint32_t offset = 0x240; + for (const auto value : userData) SetShaderRegister(words, offset++, value); +} + +} + +int main() { + try { + auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + device.reset(); + ArgumentsDescriptor = Descriptor(Arguments.data(), sizeof(Arguments)); + CounterDescriptor = Descriptor(Counter.data(), sizeof(Counter)); + std::vector warmUp; + Program(warmUp, CounterCode.data(), {Low(CounterDescriptor.data()), High(CounterDescriptor.data())}); + warmUp.insert(warmUp.end(), {0xc0031500u, 1, 1, 1, 0x8041u}); + warmUp.insert(warmUp.end(), {0xc0033700u, 0x00100200u, Low(&Done), High(&Done), 2u}); + Packet first{warmUp.data(), static_cast(warmUp.size()), 0, {}}; + Require(sceAgcDriverSubmitAcb(0x20, &first) == 0, "warm-up submit failed"); + AgcDriverWaitIdle_nid_postfix(); + Require(Done == 2 && Counter[0] == 1, "the warm-up dispatch did not run"); + Counter[0] = 0; + std::vector words; + words.insert(words.end(), {0xc0053c00u, 0x13u, Low(&Gate), High(&Gate), 1u, 0xffffffffu, 0x19u}); + Program(words, WriterCode.data(), {Low(ArgumentsDescriptor.data()), High(ArgumentsDescriptor.data()), 2u, 3u, 4u}); + words.insert(words.end(), {0xc0031500u, 1, 1, 1, 0x8041u}); + Program(words, CounterCode.data(), {Low(CounterDescriptor.data()), High(CounterDescriptor.data())}); + words.insert(words.end(), {0xc0021102u, 1u, Low(Arguments.data()), High(Arguments.data())}); + words.insert(words.end(), {0xc0011600u, 0u, 0x8061u}); + words.insert(words.end(), {0xc0033700u, 0x00100200u, Low(&Done), High(&Done), 1u}); + Packet packet{words.data(), static_cast(words.size()), 0, {}}; + Require(sceAgcDriverSubmitAcb(0x20, &packet) == 0, "compute submit failed"); + AgcDriverWaitIdle_nid_postfix(); + Require(Done == 1, "the submission did not finish"); + Require(Arguments[0] == 2 && Arguments[1] == 3 && Arguments[2] == 4, "the first dispatch did not write the indirect arguments"); + Require(Counter[0] == 24, "an indirect dispatch whose arguments the dispatch before it writes ran " + std::to_string(Counter[0]) + " groups, expected 24"); + std::cout << "indirect arguments written ahead test passed\n"; + AgcDriverShutdown_nid_postfix(); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + try { + AgcDriverShutdown_nid_postfix(); + } catch (const std::exception& shutdownError) { + std::cerr << shutdownError.what() << '\n'; + } + return 1; + } +} From 7a3c38c0058c305cce9ad7934fd2bc960afd5661 Mon Sep 17 00:00:00 2001 From: Chad Date: Sat, 10 Oct 2026 14:39:54 -0500 Subject: [PATCH 148/460] fix(agc): name the group counts of a dispatch past the device limits --- .../Execution/src/VulkanDevice.cpp | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index 65f70f840..182cfd64d 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -68,6 +68,12 @@ void require(bool condition, const char* reason) { if (!condition) throw std::runtime_error(std::string("Vulkan presentation: ") + reason); } +std::string workgroupLimitError(std::uint32_t x, std::uint32_t y, std::uint32_t z, const std::uint32_t* limit) { + char text[160]; + std::snprintf(text, sizeof(text), "Vulkan dispatch: workgroup count %ux%ux%u exceeds device limits %ux%ux%u", x, y, z, limit[0], limit[1], limit[2]); + return text; +} + // The ShaderResources content cache dispatches share with recorded draws: Graphics::ResourceCache, // one process-wide instance (see SharedResourceCache) that the State references so this file keeps // its Find/Insert/Remove/Clear calls; the device clears it at teardown before its descriptor caches @@ -3438,7 +3444,11 @@ void VulkanDevice::decideIndirect(RecordedDispatch& record, IndirectOutcome& out record.arguments = 0; timer.indirect = false; std::snprintf(groupsText, 40, "%ux%ux%u", record.x, record.y, record.z); - if (record.x > limit[0] || record.y > limit[1] || record.z > limit[2]) throw std::runtime_error("Vulkan dispatch: workgroup count exceeds device limits"); + if (record.x > limit[0] || record.y > limit[1] || record.z > limit[2]) { + char where[96]; + std::snprintf(where, sizeof(where), " (indirect arguments at 0x%llx read by the CPU, reason %d)", static_cast(arguments), outcome.cpuReason); + throw std::runtime_error(workgroupLimitError(record.x, record.y, record.z, limit) + where); + } } void VulkanDevice::recordDispatch(RecordedDispatch& record) { @@ -3589,7 +3599,7 @@ VulkanDevice::IndirectOutcome VulkanDevice::dispatch(const ShaderRecompiler::Rec const auto context = graphicsContext(); const auto* limit = state->properties.limits.maxComputeWorkGroupCount; if (arguments == 0 && (x > limit[0] || y > limit[1] || z > limit[2])) { - throw std::runtime_error("Vulkan dispatch: workgroup count exceeds device limits"); + throw std::runtime_error(workgroupLimitError(x, y, z, limit)); } // Pipelines are shared by dispatches of one compiled variant; descriptor set layouts built from the // same bindings are compatible, so the pipeline layout of the first dispatch serves them all. @@ -3876,7 +3886,7 @@ RecipeOutcome VulkanDevice::DispatchRecipe(const ShaderRecompiler::RecompileResu const auto context = graphicsContext(); const auto* limit = state->properties.limits.maxComputeWorkGroupCount; if (arguments == 0 && (x > limit[0] || y > limit[1] || z > limit[2])) { - throw std::runtime_error("Vulkan dispatch: workgroup count exceeds device limits"); + throw std::runtime_error(workgroupLimitError(x, y, z, limit)); } if (recipe.needsCompletion || recipe.holdsLease || !hit->resources->Reusable()) throw std::runtime_error("Vulkan dispatch: recipe over a non-reusable template"); // Without the refresh the template holds the words it was keyed with: a data-only hit takes From cbf86dfecfc92977d10343b934eccd7f7318f731 Mon Sep 17 00:00:00 2001 From: Chad Date: Sat, 10 Oct 2026 15:00:12 -0500 Subject: [PATCH 149/460] fix(agc): encode a one-dword NOP as the header-only form --- core/libs/prx/libSceAgc/Command/src/Packet.cpp | 2 +- core/libs/prx/libSceAgc/tests/Commands.cpp | 17 +++++++++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgc/Command/src/Packet.cpp b/core/libs/prx/libSceAgc/Command/src/Packet.cpp index 4a4e9e318..4bf5a844f 100644 --- a/core/libs/prx/libSceAgc/Command/src/Packet.cpp +++ b/core/libs/prx/libSceAgc/Command/src/Packet.cpp @@ -85,7 +85,7 @@ void ValidatePacket(const std::uint32_t* packet, std::uint32_t opcode, std::uint } std::uint32_t* WriteNop(CommandBuffer* buffer, std::uint32_t count, const char* function) { - const auto header = Header(0x10u, count); + const auto header = count == 1 ? 0xffff1000u : Header(0x10u, count); auto* packet = Allocate(buffer, count, function); packet[0] = header; std::fill_n(packet + 1, count - 1u, 0u); diff --git a/core/libs/prx/libSceAgc/tests/Commands.cpp b/core/libs/prx/libSceAgc/tests/Commands.cpp index 58b9f1026..1f02cc134 100644 --- a/core/libs/prx/libSceAgc/tests/Commands.cpp +++ b/core/libs/prx/libSceAgc/tests/Commands.cpp @@ -34,6 +34,7 @@ extern "C" std::uint32_t* APS5_VABI sceAgcAcbPushMarker(CommandBuffer* buf, cons extern "C" std::uint32_t* APS5_VABI sceAgcAcbPopMarker(CommandBuffer* buf); extern "C" std::uint32_t* APS5_VABI sceAgcAcbSetMarker(CommandBuffer* buf, const char* str, std::uint32_t color); extern "C" std::uint32_t* APS5_VABI sceAgcDcbSetIndexBuffer(CommandBuffer* buf, std::uint64_t indexAddress); +extern "C" std::uint32_t* APS5_VABI sceAgcSetNop(CommandBuffer* buf, std::uint32_t sizeDw); namespace { @@ -109,6 +110,21 @@ void testPackets() { check(noDown.cursor_up == scratch.data() + 6, "allocation past the top of a buffer without a down cursor advanced its cursor"); } +void testNop() { + Storage storage; + storage.words.fill(0xdeadbeefu); + auto* single = sceAgcSetNop(&storage.buffer, 1); + auto* pair = sceAgcSetNop(&storage.buffer, 2); + auto* triple = sceAgcSetNop(&storage.buffer, 3); + const std::array expected{0xffff1000u, 0xc0001000u, 0, 0xc0011000u, 0, 0, 0xdeadbeefu}; + check(single == storage.words.data() && pair == single + 1 && triple == pair + 2, "incorrect NOP placement"); + check(std::equal(expected.begin(), expected.end(), storage.words.begin()), "incorrect NOP packet"); + check(storage.buffer.cursor_up == storage.words.data() + 6, "incorrect NOP cursor advance"); + expectFailure([&] { sceAgcSetNop(&storage.buffer, 0); }); + expectFailure([&] { sceAgcSetNop(&storage.buffer, 0x4002u); }); + check(storage.buffer.cursor_up == storage.words.data() + 6, "invalid NOP advanced the cursor"); +} + void testClearState() { Storage storage; storage.words.fill(0xdeadbeefu); @@ -404,6 +420,7 @@ int main(int argc, char** argv) { return 0; } testPackets(); + testNop(); testClearState(); testIndexedIndirectDraws(); testMarkers(); From a3437abd11c4f2080ee9b12c2f3983375ac6673d Mon Sep 17 00:00:00 2001 From: Abdulrahman Elkordy <196347165+Silverash-i8@users.noreply.github.com> Date: Sat, 10 Oct 2026 19:00:22 +0300 Subject: [PATCH 150/460] fix(agc): require 32-lane subgroups for wave32 mesh programs The mesh stage was compiled at the driver's default subgroup width, while the translated mesh program lays wave64 lanes out on, and ballots over, a 32-lane host subgroup. Intel's Windows driver (Arrow Lake, 32.0.101.8991) reports subgroupSize 32 but prefers 16 for mesh shaders, and every agc_driver_mesh draw drew no triangle there. The mesh stage now gets VkPipelineShaderStageRequiredSubgroupSizeCreateInfo{32} when the program's hostSubgroupSize is 32 and the device lists the mesh stage in requiredSubgroupSizeStages, as compute pipelines already do. --- .../prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp | 8 ++++++-- .../libs/prx/libSceAgcDriver/Graphics/include/Context.hpp | 1 + core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp | 2 ++ 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index 65f70f840..7fb542ec1 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -248,6 +248,7 @@ struct VulkanDevice::State { // VK_KHR_timeline_semaphore enabled: the recorder's unlocked waits are available. bool timelineSemaphores = false; bool computeWave32 = false; + bool meshWave32 = false; std::uint32_t maxComputeSubgroupSize = 0; // Indirect draw features enabled (see Graphics::Context). bool drawIndirectFirstInstance = false; @@ -1235,9 +1236,11 @@ VulkanDevice::VulkanDevice(const PresentationWindow* window) : state(std::make_u state->InstanceFunction("vkGetPhysicalDeviceProperties2")(selected, &sizeProperties); state->computeWave32 = subgroupSizeFeatures.subgroupSizeControl == VK_TRUE && subgroupSize.minSubgroupSize <= 32u && subgroupSize.maxSubgroupSize >= 32u && (subgroupSize.requiredSubgroupSizeStages & VK_SHADER_STAGE_COMPUTE_BIT) != 0; + state->meshWave32 = state->meshShader && subgroupSizeFeatures.subgroupSizeControl == VK_TRUE && subgroupSize.minSubgroupSize <= 32u && subgroupSize.maxSubgroupSize >= 32u && + (subgroupSize.requiredSubgroupSizeStages & VK_SHADER_STAGE_MESH_BIT_EXT) != 0; state->maxComputeSubgroupSize = subgroupSize.maxSubgroupSize; } - if (state->computeWave32) { + if (state->computeWave32 || state->meshWave32) { subgroupSizeFeatures = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SUBGROUP_SIZE_CONTROL_FEATURES_EXT}; subgroupSizeFeatures.subgroupSizeControl = VK_TRUE; deviceExtensions.push_back(VK_EXT_SUBGROUP_SIZE_CONTROL_EXTENSION_NAME); @@ -1245,7 +1248,7 @@ VulkanDevice::VulkanDevice(const PresentationWindow* window) : state(std::make_u deviceInfo.ppEnabledExtensionNames = deviceExtensions.data(); subgroupSizeFeatures.pNext = byteFeatures.pNext; byteFeatures.pNext = &subgroupSizeFeatures; - APS5_LOG_OUT("Compute wave32 programs run on subgroups of %u", 32u); + APS5_LOG_OUT("Wave32 programs run on subgroups of %u: compute %d, mesh %d", 32u, state->computeWave32, state->meshWave32); } timelineFeatures = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_TIMELINE_SEMAPHORE_FEATURES_KHR}; timelineFeatures.timelineSemaphore = VK_TRUE; @@ -2716,6 +2719,7 @@ Graphics::Context VulkanDevice::buildContext() const { context.samplerFilterMinmax = state->samplerFilterMinmax; context.nonSeamlessCubeMap = state->nonSeamlessCubeMap; context.conservativeRasterization = state->conservativeRasterization; + context.meshWave32 = state->meshWave32; context.provokingVertexLast = state->provokingVertexLast; context.graphicsPipelineLibrary = state->graphicsPipelineLibrary; context.provokingVertexModePerPipeline = state->provokingVertexModePerPipeline; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/Context.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/Context.hpp index a948e08fb..83a15159e 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/Context.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/Context.hpp @@ -142,6 +142,7 @@ struct Context { bool samplerFilterMinmax = false; bool nonSeamlessCubeMap = false; bool conservativeRasterization = false; + bool meshWave32 = false; VkBuffer emptyBuffer = VK_NULL_HANDLE; // The device's list of recorded dispatches whose copied written buffers await a CPU write-back // (VulkanDevice's State::copiedWriters; the draw counterpart is DrawCopiedWriters): an indirect diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp index 2ab89e7d4..ae8f7b15c 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp @@ -212,6 +212,7 @@ Pipeline::Pipeline(const Context& context, const State& state, const VertexInput std::vector stages(shaders.size()); std::vector specializations; specializations.reserve(shaders.size()); + const VkPipelineShaderStageRequiredSubgroupSizeCreateInfoEXT meshSubgroup{VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_REQUIRED_SUBGROUP_SIZE_CREATE_INFO_EXT, nullptr, 32u}; for (std::uint32_t i = 0; i < shaders.size(); ++i) { const auto& shader = *shaders[i].program; VkShaderModuleCreateInfo module{VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO}; @@ -223,6 +224,7 @@ Pipeline::Pipeline(const Context& context, const State& state, const VertexInput stages[i].stage = stage; stages[i].module = _modules[i]; stages[i].pName = "main"; + if (shaders[i].stage == ShaderRecompiler::ShaderStage::Mesh && context.meshWave32 && shader.hostSubgroupSize == 32u) stages[i].pNext = &meshSubgroup; specializations.emplace_back(shader); stages[i].pSpecializationInfo = specializations.back().Info(); } From 402bc1cdfc4bb7b5166d71f755e3091e3ef88043 Mon Sep 17 00:00:00 2001 From: Chad Date: Sat, 10 Oct 2026 15:05:43 -0500 Subject: [PATCH 151/460] feat(agc): implement sceAgcBranchPatchSetThenTarget_0300 and SetElseTarget_0300 --- core/libs/prx/libSceAgc/Patch/src/Branch.cpp | 16 ++++++++++ core/libs/prx/libSceAgc/Unimplemented.cpp | 10 ------- core/libs/prx/libSceAgc/tests/BranchPatch.cpp | 29 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 2 +- 4 files changed, 46 insertions(+), 11 deletions(-) diff --git a/core/libs/prx/libSceAgc/Patch/src/Branch.cpp b/core/libs/prx/libSceAgc/Patch/src/Branch.cpp index b2f791615..1be190798 100644 --- a/core/libs/prx/libSceAgc/Patch/src/Branch.cpp +++ b/core/libs/prx/libSceAgc/Patch/src/Branch.cpp @@ -19,6 +19,12 @@ void PatchBranchTarget(std::uint32_t* cmd, std::uint32_t field, const volatile s cmd[field + 2u] = (cmd[field + 2u] & ~0xfffffu) | sizeInDwords; } +void PatchBranchTarget(std::uint32_t* cmd, std::uint32_t field, std::uint8_t cachePolicy, const volatile std::uint32_t* target, std::uint32_t sizeInDwords, const char* function) { + Agc::Command::CheckBits(cachePolicy, 3, function); + PatchBranchTarget(cmd, field, target, sizeInDwords, function); + cmd[field + 2u] = (cmd[field + 2u] & ~(3u << 28u)) | (static_cast(cachePolicy) << 28u); +} + } extern "C" { @@ -42,4 +48,14 @@ int APS5_VABI sceAgcBranchPatchSetElseTarget(std::uint32_t* cmd, const volatile return 0; } +int APS5_VABI sceAgcBranchPatchSetThenTarget_0300(std::uint32_t* cmd, std::uint8_t cachePolicy, const volatile std::uint32_t* target, std::uint32_t sizeInDwords) { + PatchBranchTarget(cmd, 8, cachePolicy, target, sizeInDwords, __func__); + return 0; +} + +int APS5_VABI sceAgcBranchPatchSetElseTarget_0300(std::uint32_t* cmd, std::uint8_t cachePolicy, const volatile std::uint32_t* target, std::uint32_t sizeInDwords) { + PatchBranchTarget(cmd, 11, cachePolicy, target, sizeInDwords, __func__); + return 0; +} + } diff --git a/core/libs/prx/libSceAgc/Unimplemented.cpp b/core/libs/prx/libSceAgc/Unimplemented.cpp index 4841b8d7a..71b993705 100644 --- a/core/libs/prx/libSceAgc/Unimplemented.cpp +++ b/core/libs/prx/libSceAgc/Unimplemented.cpp @@ -80,11 +80,6 @@ int APS5_VABI sceAgcCbMemsetExclusive() { return 0; } -int APS5_VABI sceAgcBranchPatchSetThenTarget_0300() { - NotImplemented_nid_no_patch(__func__); - return 0; -} - int APS5_VABI sceAgcGetGsPrimPayload() { NotImplemented_nid_no_patch(__func__); return 0; @@ -100,11 +95,6 @@ int APS5_VABI sceAgcGetShaderInstrumentation() { return 0; } -int APS5_VABI sceAgcBranchPatchSetElseTarget_0300() { - NotImplemented_nid_no_patch(__func__); - return 0; -} - APS5_EXPORT("vieBRwlh1Lw", sceAgcUnknown_vieBRwlh1Lw); int APS5_VABI sceAgcUnknown_vieBRwlh1Lw(void) { NotImplemented_nid_no_patch(__func__); diff --git a/core/libs/prx/libSceAgc/tests/BranchPatch.cpp b/core/libs/prx/libSceAgc/tests/BranchPatch.cpp index 2382676ee..58afe9b1c 100644 --- a/core/libs/prx/libSceAgc/tests/BranchPatch.cpp +++ b/core/libs/prx/libSceAgc/tests/BranchPatch.cpp @@ -13,6 +13,8 @@ std::uint32_t APS5_VABI sceAgcCbBranchGetSize(); int APS5_VABI sceAgcBranchPatchSetCompareAddress(std::uint32_t*, const volatile std::uint64_t*); int APS5_VABI sceAgcBranchPatchSetThenTarget(std::uint32_t*, const volatile std::uint32_t*, std::uint32_t); int APS5_VABI sceAgcBranchPatchSetElseTarget(std::uint32_t*, const volatile std::uint32_t*, std::uint32_t); +int APS5_VABI sceAgcBranchPatchSetThenTarget_0300(std::uint32_t*, std::uint8_t, const volatile std::uint32_t*, std::uint32_t); +int APS5_VABI sceAgcBranchPatchSetElseTarget_0300(std::uint32_t*, std::uint8_t, const volatile std::uint32_t*, std::uint32_t); } namespace { @@ -107,6 +109,31 @@ void testTarget(TSetter setter, std::size_t field, std::uint32_t cachePolicyBits check(storage.words == expected, name); } +template +void testTargetWithCachePolicy(TSetter setter, std::size_t field, const char* name) { + Storage storage; + auto expected = storage.words; + expected[field] = 0x89abcdecu; + expected[field + 1] = 0x4567u; + expected[field + 2] = 0x30000004u; + check(setter(storage.packet, 3, targetAt(0x0000456789abcdecull), 4) == 0, name); + check(storage.words == expected, name); + expected[field] = 0x1000u; + expected[field + 1] = 0; + expected[field + 2] = 0xfffffu; + check(setter(storage.packet, 0, targetAt(0x1000u), 0xfffffu) == 0, name); + check(storage.words == expected, name); + expectFailure([&] { setter(storage.packet, 4, targetAt(0x2000u), 1); }); + expectFailure([&] { setter(storage.packet, 1, nullptr, 1); }); + expectFailure([&] { setter(storage.packet, 1, targetAt(0x1002u), 1); }); + expectFailure([&] { setter(storage.packet, 1, targetAt(0x2000u), 0x100000u); }); + expectFailure([&] { setter(nullptr, 1, targetAt(0x2000u), 1); }); + storage.packet[0] = 0xc00b3f00u; + expected[0] = 0xc00b3f00u; + expectFailure([&] { setter(storage.packet, 1, targetAt(0x2000u), 1); }); + check(storage.words == expected, name); +} + void testMalformedPackets() { for (const auto header : {0xc0023f00u, 0xc00b3f00u, 0xc00c3f01u, 0xc00c2200u}) { Storage storage; @@ -128,6 +155,8 @@ int main() { testCompareAddress(); testTarget(sceAgcBranchPatchSetThenTarget, 8, 0x10000000u, "then target setter wrote the wrong words"); testTarget(sceAgcBranchPatchSetElseTarget, 11, 0x20000000u, "else target setter wrote the wrong words"); + testTargetWithCachePolicy(sceAgcBranchPatchSetThenTarget_0300, 8, "3.00 then target setter wrote the wrong words"); + testTargetWithCachePolicy(sceAgcBranchPatchSetElseTarget_0300, 11, "3.00 else target setter wrote the wrong words"); testMalformedPackets(); std::puts("AGC branch patch tests passed"); return 0; diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index cfd755962..cd296e87a 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -123,7 +123,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceAgcGetIsTrinityMode](../../core/libs/prx/libSceAgc/Misc/src/Platform.cpp) (libSceAgc) - signature from the only PPSA26344 call (a pointer to a one-byte flag, result ignored); a null pointer throws because its error code is unknown - [zARR5aCmkoY](../../core/libs/prx/libSceAgc/DcbFlow/src/Control.cpp) (libSceAgc) - unknown name, signature - [sceAgcDcbContextStateOpGetSize](../../core/libs/prx/libSceAgc/DcbState/src/ContextState.cpp) (libSceAgc) - sizes from KytyPS5 only (5, 27, 27 and 32 dwords), equal to what `sceAgcDcbContextStateOp_0100` writes; KytyPS5 returns 0 for an operation above 3, here it throws -- [sceAgcBranchPatchSetThenTarget_0300, sceAgcBranchPatchSetElseTarget_0300](../../core/libs/prx/libSceAgc/Unimplemented.cpp) (libSceAgc) - names from their NIDs (`7Wa3aeJgeVU`, `rP5xLdOf26k`), signatures unknown +- [sceAgcBranchPatchSetThenTarget_0300, sceAgcBranchPatchSetElseTarget_0300](../../core/libs/prx/libSceAgc/Patch/src/Branch.cpp) (libSceAgc) - names from their NIDs (`7Wa3aeJgeVU`, `rP5xLdOf26k`); the `(packet, cachePolicy, target, sizeInDwords)` signature is inferred from PPSA28416's calls (a branch packet, 0, an aligned target, 4), with the policy stored where `sceAgcCbBranch` puts it - [sceAgcUpdateInterpolantMapping_0100](../../core/libs/prx/libSceAgc/Misc/src/ShaderFusion.cpp) (libSceAgc) - name from its NID (`k0E7vkgqAuE`); the `(regs, vs, ps)` arguments are the declared ones, behaviour unknown - [sceAgcWriteDataPatchSetDst, sceAgcWriteDataPatchSetCachePolicy and their Async forms](../../core/libs/prx/libSceAgc/Patch/src/WaitRegMem.cpp) (libSceAgc) - the signatures `(packet, uint8 value)` are inferred from the other patch functions, as no public declaration was found; the value is taken in the encoding of the `dst` and `cachePolicy` arguments of `sceAgcDcbWriteData` (graphics) and `sceAgcAcbWriteData` (Async, compute), and the patched packet is the one those calls would write. `sceAgcAsyncWriteDataPatchSetAddressOrOffset` is assumed to patch the address like `sceAgcWriteDataPatchSetAddressOrOffset`, as the compute WRITE_DATA packet keeps it in the same dwords - [dbOlWdppb4o](../../core/libs/prx/libSceAgc/Shader/src/InterpolantMapping.cpp) (libSceAgc) - unknown name; same arguments as `sceAgcCreateInterpolantMapping_0100`; the `SPI_PS_INPUT_CNTL` bits for `is_f16` 2 inputs follow KytyPS5, not confirmed on hardware From 7f2a88262a87d78e34af93d8e662a627090fb804 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 18:07:49 +0000 Subject: [PATCH 152/460] fix(libkernel): detect Windows reparse points in lstat --- .../prx/libkernel/File/src/NativeStat.cpp | 5 ++-- core/libs/tests/GuestFilesystem.cpp | 30 +++++++++++++++++++ 2 files changed, 33 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libkernel/File/src/NativeStat.cpp b/core/libs/prx/libkernel/File/src/NativeStat.cpp index 1c0ada2a6..5454db47d 100644 --- a/core/libs/prx/libkernel/File/src/NativeStat.cpp +++ b/core/libs/prx/libkernel/File/src/NativeStat.cpp @@ -44,8 +44,9 @@ static int DoFstat(int fd, NativeStat* st) { return ReadHandleInfo(reinterpret_cast(::_get_osfhandle(fd)), st); } static int DoLstat(const std::filesystem::path& p, NativeStat* st) { - std::error_code error; - if (std::filesystem::is_symlink(std::filesystem::symlink_status(p, error))) NotImplemented_nid_no_patch("lstat of a Windows symbolic link"); + const auto attributes = GetFileAttributesW(p.c_str()); + if (attributes != INVALID_FILE_ATTRIBUTES && (attributes & FILE_ATTRIBUTE_REPARSE_POINT) != 0) + NotImplemented_nid_no_patch("lstat of a Windows symbolic link"); return DoStat(p, st); } #else diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index c42534124..21b489394 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -1,13 +1,16 @@ #include "SceTypes.hpp" #include "prx/libc/include/general/VabiMacros.hpp" +#include #include #include #include #include #include #include +#include #ifdef _WIN32 #include +#include #else #include #endif @@ -263,6 +266,33 @@ int main() { Require(stat_nid_postfix(dangling.string().c_str(), &status) == -1 && *__error_nid_postfix() == 2); Require(lstat_nid_postfix((root / "present.txt" / "child").string().c_str(), &linkStatus) == -1 && *__error_nid_postfix() == 20); Require(unlink_nid_postfix(link.string().c_str()) == 0 && unlink_nid_postfix(dangling.string().c_str()) == 0); +#else + const auto targetDirectory = root / "target-directory"; + Require(std::filesystem::create_directory(targetDirectory)); + const auto fileLink = root / "file-link"; + const auto directoryLink = root / "directory-link"; + Require(CreateSymbolicLinkW(fileLink.c_str(), present.c_str(), SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); + Require(CreateSymbolicLinkW(directoryLink.c_str(), targetDirectory.c_str(), + SYMBOLIC_LINK_FLAG_DIRECTORY | SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); + const auto checkUnsupportedLink = [](const std::filesystem::path& link) { + FileStat untouched; + std::memset(&untouched, 0x5a, sizeof(untouched)); + constexpr int sentinelError = 123; + *__error_nid_postfix() = sentinelError; + bool rejected = false; + try { + lstat_nid_postfix(link.string().c_str(), &untouched); + } catch (const std::runtime_error& error) { + rejected = std::strcmp(error.what(), "lstat of a Windows symbolic link not implemented") == 0; + } + const bool errorUnchanged = *__error_nid_postfix() == sentinelError; + const auto* bytes = reinterpret_cast(&untouched); + const bool statusUnchanged = std::all_of(bytes, bytes + sizeof(untouched), [](unsigned char value) { return value == 0x5a; }); + return rejected && errorUnchanged && statusUnchanged; + }; + Require(checkUnsupportedLink(fileLink)); + Require(checkUnsupportedLink(directoryLink)); + Require(std::filesystem::remove(fileLink) && std::filesystem::remove(directoryLink)); #endif FileStat identity{}; FileStat byDescriptor{}; From 314c372285d67d84a2a4c5e2443b78872fb69fb4 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 18:08:27 +0000 Subject: [PATCH 153/460] test(libkernel): cover dangling Windows lstat links --- core/libs/tests/GuestFilesystem.cpp | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index 21b489394..e53c45718 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -271,9 +271,11 @@ int main() { Require(std::filesystem::create_directory(targetDirectory)); const auto fileLink = root / "file-link"; const auto directoryLink = root / "directory-link"; + const auto danglingLink = root / "dangling-link"; Require(CreateSymbolicLinkW(fileLink.c_str(), present.c_str(), SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); Require(CreateSymbolicLinkW(directoryLink.c_str(), targetDirectory.c_str(), SYMBOLIC_LINK_FLAG_DIRECTORY | SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); + Require(CreateSymbolicLinkW(danglingLink.c_str(), missingName.c_str(), SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); const auto checkUnsupportedLink = [](const std::filesystem::path& link) { FileStat untouched; std::memset(&untouched, 0x5a, sizeof(untouched)); @@ -292,7 +294,8 @@ int main() { }; Require(checkUnsupportedLink(fileLink)); Require(checkUnsupportedLink(directoryLink)); - Require(std::filesystem::remove(fileLink) && std::filesystem::remove(directoryLink)); + Require(checkUnsupportedLink(danglingLink)); + Require(std::filesystem::remove(fileLink) && std::filesystem::remove(directoryLink) && std::filesystem::remove(danglingLink)); #endif FileStat identity{}; FileStat byDescriptor{}; From a22206bfbdd70978a72291262be759d369e4156b Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 20:10:19 +0000 Subject: [PATCH 154/460] test(libkernel): fix Windows lstat fixture cleanup --- core/libs/tests/GuestFilesystem.cpp | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index e53c45718..a13c14585 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -230,7 +230,8 @@ int main() { Require(remove_nid_postfix(sized.string().c_str()) == 0); const auto present = root / "present.txt"; const auto presentName = present.string(); - const auto missingName = (root / "missing.txt").string(); + const auto missing = root / "missing.txt"; + const auto missingName = missing.string(); const auto rootName = root.string(); { std::ofstream stream(present); stream << "posix"; } FileStat status{}; @@ -275,7 +276,7 @@ int main() { Require(CreateSymbolicLinkW(fileLink.c_str(), present.c_str(), SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); Require(CreateSymbolicLinkW(directoryLink.c_str(), targetDirectory.c_str(), SYMBOLIC_LINK_FLAG_DIRECTORY | SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); - Require(CreateSymbolicLinkW(danglingLink.c_str(), missingName.c_str(), SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); + Require(CreateSymbolicLinkW(danglingLink.c_str(), missing.c_str(), SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); const auto checkUnsupportedLink = [](const std::filesystem::path& link) { FileStat untouched; std::memset(&untouched, 0x5a, sizeof(untouched)); @@ -295,7 +296,8 @@ int main() { Require(checkUnsupportedLink(fileLink)); Require(checkUnsupportedLink(directoryLink)); Require(checkUnsupportedLink(danglingLink)); - Require(std::filesystem::remove(fileLink) && std::filesystem::remove(directoryLink) && std::filesystem::remove(danglingLink)); + Require(std::filesystem::remove(fileLink) && std::filesystem::remove(directoryLink) && + std::filesystem::remove(danglingLink) && std::filesystem::remove(targetDirectory)); #endif FileStat identity{}; FileStat byDescriptor{}; From 5d7fc5a342fe24835a680bee162f4732a071b8f2 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 16:29:45 +0000 Subject: [PATCH 155/460] test(libc): keep macOS write watch stress bounded --- core/libs/tests/WriteWatch.cpp | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/core/libs/tests/WriteWatch.cpp b/core/libs/tests/WriteWatch.cpp index 84b3029c5..da24203a3 100644 --- a/core/libs/tests/WriteWatch.cpp +++ b/core/libs/tests/WriteWatch.cpp @@ -144,7 +144,7 @@ void CheckProtection() { void CheckThreads() { constexpr std::size_t Threads = 4; constexpr std::size_t PagesPerThread = 64; - constexpr int Rounds = 200; + constexpr int Rounds = 50; Region region(Threads * PagesPerThread); GuestWriteWatchRegister_nid_postfix(region.base, region.bytes); Require(region.Collect() == All(Threads * PagesPerThread), "a new range is not reported in full"); @@ -164,7 +164,10 @@ void CheckThreads() { for (const auto index : region.Collect()) seen[index] = region.base[index * region.page]; }; start.store(true); - while (finished.load() != Threads) take(); + while (finished.load() != Threads) { + take(); + std::this_thread::yield(); + } for (auto& writer : writers) writer.join(); take(); for (std::size_t index = 0; index < seen.size(); ++index) From 0c2d85238b1a16b87d1c54973b48e3f6f90f2cda Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Sat, 10 Oct 2026 16:20:54 -0400 Subject: [PATCH 156/460] feat(libkernel): implement preadv and pwritev --- core/libs/prx/libkernel/File/src/Stdio.cpp | 10 ++++++++ core/libs/tests/GuestFileVector.cpp | 27 ++++++++++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index e51329185..b7f3f1143 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -885,6 +885,16 @@ int64_t APS5_VABI sceKernelPwritev(int d, const KernelIovec* iov, int iovcnt, in #endif +int64_t APS5_VABI preadv_nid_postfix(int d, const KernelIovec* iov, int iovcnt, int64_t offset) { + const auto result = sceKernelPreadv(d, iov, iovcnt, offset); + return result < 0 ? PosixFailure(static_cast(result) & 0xffff) : result; +} + +int64_t APS5_VABI pwritev_nid_postfix(int d, const KernelIovec* iov, int iovcnt, int64_t offset) { + const auto result = sceKernelPwritev(d, iov, iovcnt, offset); + return result < 0 ? PosixFailure(static_cast(result) & 0xffff) : result; +} + int APS5_VABI sceKernelRename(const char* from, const char* to) { if (from == nullptr || to == nullptr) throw std::invalid_argument("sceKernelRename: path is null"); const auto source = ResolvePath_nid_no_patch(from); diff --git a/core/libs/tests/GuestFileVector.cpp b/core/libs/tests/GuestFileVector.cpp index b73f03fb0..a94009208 100644 --- a/core/libs/tests/GuestFileVector.cpp +++ b/core/libs/tests/GuestFileVector.cpp @@ -36,6 +36,9 @@ std::int64_t APS5_VABI sceKernelReadv(int, const GuestIovec*, int); std::int64_t APS5_VABI sceKernelWritev(int, const GuestIovec*, int); std::int64_t APS5_VABI sceKernelPreadv(int, const GuestIovec*, int, std::int64_t); std::int64_t APS5_VABI sceKernelPwritev(int, const GuestIovec*, int, std::int64_t); +std::int64_t APS5_VABI preadv_nid_postfix(int, const GuestIovec*, int, std::int64_t); +std::int64_t APS5_VABI pwritev_nid_postfix(int, const GuestIovec*, int, std::int64_t); +int* APS5_VABI __error_nid_postfix(); std::int64_t APS5_VABI sceKernelPread(int, void*, std::size_t, std::int64_t); std::int64_t APS5_VABI sceKernelPwrite(int, const void*, std::size_t, std::int64_t); int APS5_VABI sceKernelFtruncate(int, std::int64_t); @@ -123,6 +126,30 @@ int main() { Require(sceKernelPwritev(file, writes, 2, -1) == ErrorEinval); Require(Contents(path) == "01ABCDE7ABCDE"); + Require(sceKernelLseek(file, 4, 0) == 4); + Require(preadv_nid_postfix(file, reads, 2, 1) == 7); + Require(std::memcmp(first, "1AB", 3) == 0 && std::memcmp(second, "CDE7", 4) == 0); + Require(sceKernelLseek(file, 0, 1) == 4); + Require(preadv_nid_postfix(file, reads, 2, 13) == 0); + Require(preadv_nid_postfix(file, reads, 2, -1) == -1 && *__error_nid_postfix() == 22); + Require(preadv_nid_postfix(file, reads, -1, 0) == -1 && *__error_nid_postfix() == 22); + Require(preadv_nid_postfix(file, nullptr, 1, 0) == -1 && *__error_nid_postfix() == 14); + Require(pwritev_nid_postfix(file, writes, 2, 8) == 5); + Require(sceKernelLseek(file, 0, 1) == 4); + Require(Contents(path) == "01ABCDE7ABCDE"); + char x[] = "X"; + char y[] = "Y"; + GuestIovec marks[2] = {{x, 1}, {y, 1}}; + Require(pwritev_nid_postfix(file, marks, 2, 0) == 2); + Require(Contents(path) == "XYABCDE7ABCDE"); + char zero[] = "0"; + char one[] = "1"; + GuestIovec restore[2] = {{zero, 1}, {one, 1}}; + Require(pwritev_nid_postfix(file, restore, 2, 0) == 2); + Require(sceKernelLseek(file, 0, 1) == 4); + Require(pwritev_nid_postfix(file, writes, 2, -1) == -1 && *__error_nid_postfix() == 22); + Require(pwritev_nid_postfix(file, nullptr, 1, 0) == -1 && *__error_nid_postfix() == 14); + char byte = 0; Require(sceKernelPread(file, &byte, 1, 2) == 1 && byte == 'A'); Require(sceKernelPread(file, nullptr, 0, 0) == 0); From 6f1ef27953930d086eeeafc928af77b644a46745 Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Sat, 10 Oct 2026 16:25:16 -0400 Subject: [PATCH 157/460] feat(libSceAudioIn): export sceAudioInInit --- core/libs/prx/libSceAudioIn/Export.cpp | 4 ++++ core/libs/tests/GuestAudioIn.cpp | 2 ++ 2 files changed, 6 insertions(+) diff --git a/core/libs/prx/libSceAudioIn/Export.cpp b/core/libs/prx/libSceAudioIn/Export.cpp index 4ba03302f..50e917ee5 100644 --- a/core/libs/prx/libSceAudioIn/Export.cpp +++ b/core/libs/prx/libSceAudioIn/Export.cpp @@ -90,6 +90,10 @@ std::size_t capture(SDL_AudioDeviceID device, std::uint8_t* dest, std::size_t by extern "C" { +int APS5_VABI sceAudioInInit() { + return 0; +} + int APS5_VABI sceAudioInGetSilentState(int handle) { std::lock_guard lock(g_mutex); const Port* port = find(handle); diff --git a/core/libs/tests/GuestAudioIn.cpp b/core/libs/tests/GuestAudioIn.cpp index 8786835e8..f3b3d3ff4 100644 --- a/core/libs/tests/GuestAudioIn.cpp +++ b/core/libs/tests/GuestAudioIn.cpp @@ -10,6 +10,7 @@ #include extern "C" { +int APS5_VABI sceAudioInInit(); int APS5_VABI sceAudioInOpen(int, std::uint32_t, std::uint32_t, std::uint32_t, std::uint32_t, std::uint32_t); int APS5_VABI sceAudioInInput(int, void*); int APS5_VABI sceAudioInGetSilentState(int); @@ -98,6 +99,7 @@ void TestNoDevice() { int main(int argc, char** argv) { Require(argc == 2); + Require(sceAudioInInit() == 0); if (std::strcmp(argv[1], "capture") == 0) TestCapture(); else TestNoDevice(); } From 951526662f3b3ac50319353922b59fa3f4764264 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 16:02:39 -0400 Subject: [PATCH 158/460] fix(shader): honor directed f16 rounding in f16 conversions and ldexp Directed FLOAT_MODE modes for v_cvt_f16_f32, v_cvt_f16_i16, v_cvt_f16_u16 and v_ldexp_f16 were ignored, so results stayed round-to-nearest-even. Hardware uses FLOAT_MODE bits 3:2 (round16); the directed path runs only when f16 denormals are preserved. Co-Authored-By: Claude Opus 5.5 Co-authored-by: Cursor --- .../tests/execution/Float16Rounding.cpp | 238 ++++++++++++++++++ .../Translation/TranslationContext.hpp | 3 + .../Translation/src/ConvertInstructions.cpp | 4 +- .../Translation/src/FloatInstructions.cpp | 3 +- .../src/TranslationContext/SubwordPacking.cpp | 43 ++++ docs/dev/TechnicalDebt.md | 1 + 6 files changed, 288 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/Float16Rounding.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/Float16Rounding.cpp index ec1c19913..bd249656b 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/Float16Rounding.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/Float16Rounding.cpp @@ -292,6 +292,203 @@ constexpr Vector Vectors[] = { {0x498f4a8bu, 0x6749af47u, 0xc69e8021u, {0xabcd7c00u, 0xabcdaf47u, 0xabcd0004u}}, }; +alignas(256) constexpr std::array DirectedCode{ + 0x34020084u, 0x34040086u, 0xe0381000u, 0x80000401u, 0xbf8c3f70u, 0x7e140280u, 0x7e160280u, 0x7e180280u, + 0x7e1a0280u, 0x7e1c0280u, 0x7e1e0280u, 0x7e200280u, 0x7e220280u, 0x7e240280u, 0x7e260280u, 0x7e280280u, + 0x7e2a0280u, 0x7e2c0280u, 0x7e2e0280u, 0x7e300280u, 0x7e320280u, 0x7e1402ffu, 0x5a5a5a5au, 0x7e1602ffu, + 0x5a5a5a5au, 0x7e1802ffu, 0x5a5a5a5au, 0x7e1a02ffu, 0x5a5a5a5au, 0x7e141504u, 0x7e16a305u, 0x7e18a105u, + 0x761a0f06u, 0xe0781000u, 0x80010a02u, 0xe0781010u, 0x80010e02u, 0xe0781020u, 0x80011202u, 0xe0781030u, + 0x80011602u, 0xbf810000u, +}; + +struct DirectedVector { + std::array inputs; + std::array, 2>, 3> expected; +}; + +constexpr DirectedVector DirectedVectors[] = { + {{0x00000000u, 0x5a5a0000u, 0xa5a50000u, 0x00000000u}, {{ + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}}}, + }}}, + {{0x80000000u, 0x5a5a0001u, 0xa5a58000u, 0x00000001u}, {{ + {{{0x5a5a8000u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5a8000u}, {0x5a5a8000u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5a8000u}}}, + {{{0x5a5a8000u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5a8000u}, {0x5a5a8000u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5a8000u}}}, + {{{0x5a5a8000u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5a8000u}, {0x5a5a8000u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5a8000u}}}, + }}}, + {{0x3f800000u, 0x5a5a8001u, 0xa5a53c00u, 0xffffffe8u}, {{ + {{{0x5a5a3c00u, 0x5a5af7ffu, 0x5a5a7801u, 0x5a5a0001u}, {0x5a5a3c00u, 0x5a5af7ffu, 0x5a5a7801u, 0x5a5a0001u}}}, + {{{0x5a5a3c00u, 0x5a5af800u, 0x5a5a7800u, 0x5a5a0001u}, {0x5a5a3c00u, 0x5a5af800u, 0x5a5a7800u, 0x5a5a0001u}}}, + {{{0x5a5a3c00u, 0x5a5af7ffu, 0x5a5a7800u, 0x5a5a0001u}, {0x5a5a3c00u, 0x5a5af7ffu, 0x5a5a7800u, 0x5a5a0001u}}}, + }}}, + {{0xbf800000u, 0x5a5affffu, 0xa5a5bc00u, 0x00000040u}, {{ + {{{0x5a5abc00u, 0x5a5abc00u, 0x5a5a7c00u, 0x5a5afbffu}, {0x5a5abc00u, 0x5a5abc00u, 0x5a5a7bffu, 0x5a5afbffu}}}, + {{{0x5a5abc00u, 0x5a5abc00u, 0x5a5a7bffu, 0x5a5afc00u}, {0x5a5abc00u, 0x5a5abc00u, 0x5a5a7bffu, 0x5a5afbffu}}}, + {{{0x5a5abc00u, 0x5a5abc00u, 0x5a5a7bffu, 0x5a5afbffu}, {0x5a5abc00u, 0x5a5abc00u, 0x5a5a7bffu, 0x5a5afbffu}}}, + }}}, + {{0x3f800001u, 0x5a5a0000u, 0xa5a53c01u, 0xffffffc0u}, {{ + {{{0x5a5a3c01u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0001u}, {0x5a5a3c01u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0001u}}}, + {{{0x5a5a3c00u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}, {0x5a5a3c00u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}}}, + {{{0x5a5a3c00u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}, {0x5a5a3c00u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}}}, + }}}, + {{0xbf800001u, 0x5a5a0001u, 0xa5a5bc01u, 0x00000041u}, {{ + {{{0x5a5abc00u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5afbffu}, {0x5a5abc00u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5afbffu}}}, + {{{0x5a5abc01u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5afc00u}, {0x5a5abc01u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5afbffu}}}, + {{{0x5a5abc00u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5afbffu}, {0x5a5abc00u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5afbffu}}}, + }}}, + {{0x3f801000u, 0x5a5a07ffu, 0xa5a57bffu, 0xffffffbfu}, {{ + {{{0x5a5a3c01u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a0001u}, {0x5a5a3c01u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a0001u}}}, + {{{0x5a5a3c00u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a0000u}, {0x5a5a3c00u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a0000u}}}, + {{{0x5a5a3c00u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a0000u}, {0x5a5a3c00u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a0000u}}}, + }}}, + {{0xbf801000u, 0x5a5a0800u, 0xa5a5fbffu, 0x00007fffu}, {{ + {{{0x5a5abc00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5afbffu}, {0x5a5abc00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5afbffu}}}, + {{{0x5a5abc01u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5afc00u}, {0x5a5abc01u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5afbffu}}}, + {{{0x5a5abc00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5afbffu}, {0x5a5abc00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5afbffu}}}, + }}}, + {{0x3f801fffu, 0x5a5a0801u, 0xa5a57c00u, 0xffff8000u}, {{ + {{{0x5a5a3c01u, 0x5a5a6801u, 0x5a5a6801u, 0x5a5a7c00u}, {0x5a5a3c01u, 0x5a5a6801u, 0x5a5a6801u, 0x5a5a7c00u}}}, + {{{0x5a5a3c00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5a7c00u}, {0x5a5a3c00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5a7c00u}}}, + {{{0x5a5a3c00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5a7c00u}, {0x5a5a3c00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5a7c00u}}}, + }}}, + {{0xb3000000u, 0x5a5a7fffu, 0xa5a57e55u, 0x00000001u}, {{ + {{{0x5a5a8000u, 0x5a5a7800u, 0x5a5a7800u, 0x5a5a7e55u}, {0x5a5a8000u, 0x5a5a7800u, 0x5a5a7800u, 0x5a5a7e55u}}}, + {{{0x5a5a8001u, 0x5a5a77ffu, 0x5a5a77ffu, 0x5a5a7e55u}, {0x5a5a8001u, 0x5a5a77ffu, 0x5a5a77ffu, 0x5a5a7e55u}}}, + {{{0x5a5a8000u, 0x5a5a77ffu, 0x5a5a77ffu, 0x5a5a7e55u}, {0x5a5a8000u, 0x5a5a77ffu, 0x5a5a77ffu, 0x5a5a7e55u}}}, + }}}, + {{0x33000001u, 0x5a5a8000u, 0xa5a5fe55u, 0xffffffffu}, {{ + {{{0x5a5a0001u, 0x5a5af800u, 0x5a5a7800u, 0x5a5afe55u}, {0x5a5a0001u, 0x5a5af800u, 0x5a5a7800u, 0x5a5afe55u}}}, + {{{0x5a5a0000u, 0x5a5af800u, 0x5a5a7800u, 0x5a5afe55u}, {0x5a5a0000u, 0x5a5af800u, 0x5a5a7800u, 0x5a5afe55u}}}, + {{{0x5a5a0000u, 0x5a5af800u, 0x5a5a7800u, 0x5a5afe55u}, {0x5a5a0000u, 0x5a5af800u, 0x5a5a7800u, 0x5a5afe55u}}}, + }}}, + {{0x33800000u, 0x5a5a8001u, 0xa5a57c01u, 0x0000000au}, {{ + {{{0x5a5a0001u, 0x5a5af7ffu, 0x5a5a7801u, 0x5a5a7e01u}, {0x5a5a0001u, 0x5a5af7ffu, 0x5a5a7801u, 0x5a5a7e01u}}}, + {{{0x5a5a0001u, 0x5a5af800u, 0x5a5a7800u, 0x5a5a7e01u}, {0x5a5a0001u, 0x5a5af800u, 0x5a5a7800u, 0x5a5a7e01u}}}, + {{{0x5a5a0001u, 0x5a5af7ffu, 0x5a5a7800u, 0x5a5a7e01u}, {0x5a5a0001u, 0x5a5af7ffu, 0x5a5a7800u, 0x5a5a7e01u}}}, + }}}, + {{0xb37fffffu, 0x5a5a0000u, 0xa5a50000u, 0x00000010u}, {{ + {{{0x5a5a8000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}, {0x5a5a8000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}}}, + {{{0x5a5a8001u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}, {0x5a5a8001u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}}}, + {{{0x5a5a8000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}, {0x5a5a8000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u}}}, + }}}, + {{0x387fffffu, 0x5a5a0001u, 0xa5a58000u, 0xfffffff0u}, {{ + {{{0x5a5a0400u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5a8000u}, {0x5a5a0400u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5a8000u}}}, + {{{0x5a5a03ffu, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5a8000u}, {0x5a5a03ffu, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5a8000u}}}, + {{{0x5a5a03ffu, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5a8000u}, {0x5a5a03ffu, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5a8000u}}}, + }}}, + {{0x477fe000u, 0x5a5a0801u, 0xa5a503ffu, 0x00000040u}, {{ + {{{0x5a5a7bffu, 0x5a5a6801u, 0x5a5a6801u, 0x5a5a7c00u}, {0x5a5a7bffu, 0x5a5a6801u, 0x5a5a6801u, 0x5a5a7bffu}}}, + {{{0x5a5a7bffu, 0x5a5a6800u, 0x5a5a6800u, 0x5a5a7bffu}, {0x5a5a7bffu, 0x5a5a6800u, 0x5a5a6800u, 0x5a5a7bffu}}}, + {{{0x5a5a7bffu, 0x5a5a6800u, 0x5a5a6800u, 0x5a5a7bffu}, {0x5a5a7bffu, 0x5a5a6800u, 0x5a5a6800u, 0x5a5a7bffu}}}, + }}}, + {{0x477fe001u, 0x5a5a0803u, 0xa5a583ffu, 0xffffffc0u}, {{ + {{{0x5a5a7c00u, 0x5a5a6802u, 0x5a5a6802u, 0x5a5a8000u}, {0x5a5a7bffu, 0x5a5a6802u, 0x5a5a6802u, 0x5a5a8000u}}}, + {{{0x5a5a7bffu, 0x5a5a6801u, 0x5a5a6801u, 0x5a5a8001u}, {0x5a5a7bffu, 0x5a5a6801u, 0x5a5a6801u, 0x5a5a8001u}}}, + {{{0x5a5a7bffu, 0x5a5a6801u, 0x5a5a6801u, 0x5a5a8000u}, {0x5a5a7bffu, 0x5a5a6801u, 0x5a5a6801u, 0x5a5a8000u}}}, + }}}, + {{0xc77fe001u, 0x5a5a7fffu, 0xa5a50400u, 0x00000041u}, {{ + {{{0x5a5afbffu, 0x5a5a7800u, 0x5a5a7800u, 0x5a5a7c00u}, {0x5a5afbffu, 0x5a5a7800u, 0x5a5a7800u, 0x5a5a7bffu}}}, + {{{0x5a5afc00u, 0x5a5a77ffu, 0x5a5a77ffu, 0x5a5a7bffu}, {0x5a5afbffu, 0x5a5a77ffu, 0x5a5a77ffu, 0x5a5a7bffu}}}, + {{{0x5a5afbffu, 0x5a5a77ffu, 0x5a5a77ffu, 0x5a5a7bffu}, {0x5a5afbffu, 0x5a5a77ffu, 0x5a5a77ffu, 0x5a5a7bffu}}}, + }}}, + {{0x477ff000u, 0x5a5a8000u, 0xa5a58400u, 0xffffffbfu}, {{ + {{{0x5a5a7c00u, 0x5a5af800u, 0x5a5a7800u, 0x5a5a8000u}, {0x5a5a7bffu, 0x5a5af800u, 0x5a5a7800u, 0x5a5a8000u}}}, + {{{0x5a5a7bffu, 0x5a5af800u, 0x5a5a7800u, 0x5a5a8001u}, {0x5a5a7bffu, 0x5a5af800u, 0x5a5a7800u, 0x5a5a8001u}}}, + {{{0x5a5a7bffu, 0x5a5af800u, 0x5a5a7800u, 0x5a5a8000u}, {0x5a5a7bffu, 0x5a5af800u, 0x5a5a7800u, 0x5a5a8000u}}}, + }}}, + {{0x47800000u, 0x5a5a8001u, 0xa5a53c00u, 0x00007fffu}, {{ + {{{0x5a5a7c00u, 0x5a5af7ffu, 0x5a5a7801u, 0x5a5a7c00u}, {0x5a5a7bffu, 0x5a5af7ffu, 0x5a5a7801u, 0x5a5a7bffu}}}, + {{{0x5a5a7bffu, 0x5a5af800u, 0x5a5a7800u, 0x5a5a7bffu}, {0x5a5a7bffu, 0x5a5af800u, 0x5a5a7800u, 0x5a5a7bffu}}}, + {{{0x5a5a7bffu, 0x5a5af7ffu, 0x5a5a7800u, 0x5a5a7bffu}, {0x5a5a7bffu, 0x5a5af7ffu, 0x5a5a7800u, 0x5a5a7bffu}}}, + }}}, + {{0xc7800000u, 0x5a5affffu, 0xa5a5bc00u, 0xffff8000u}, {{ + {{{0x5a5afbffu, 0x5a5abc00u, 0x5a5a7c00u, 0x5a5a8000u}, {0x5a5afbffu, 0x5a5abc00u, 0x5a5a7bffu, 0x5a5a8000u}}}, + {{{0x5a5afc00u, 0x5a5abc00u, 0x5a5a7bffu, 0x5a5a8001u}, {0x5a5afbffu, 0x5a5abc00u, 0x5a5a7bffu, 0x5a5a8001u}}}, + {{{0x5a5afbffu, 0x5a5abc00u, 0x5a5a7bffu, 0x5a5a8000u}, {0x5a5afbffu, 0x5a5abc00u, 0x5a5a7bffu, 0x5a5a8000u}}}, + }}}, + {{0x7f7fffffu, 0x5a5a0000u, 0xa5a53c01u, 0x00000000u}, {{ + {{{0x5a5a7c00u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a3c01u}, {0x5a5a7bffu, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a3c01u}}}, + {{{0x5a5a7bffu, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a3c01u}, {0x5a5a7bffu, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a3c01u}}}, + {{{0x5a5a7bffu, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a3c01u}, {0x5a5a7bffu, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a3c01u}}}, + }}}, + {{0xff7fffffu, 0x5a5a0001u, 0xa5a5bc01u, 0x00000001u}, {{ + {{{0x5a5afbffu, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5ac001u}, {0x5a5afbffu, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5ac001u}}}, + {{{0x5a5afc00u, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5ac001u}, {0x5a5afbffu, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5ac001u}}}, + {{{0x5a5afbffu, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5ac001u}, {0x5a5afbffu, 0x5a5a3c00u, 0x5a5a3c00u, 0x5a5ac001u}}}, + }}}, + {{0x7f800000u, 0x5a5a07ffu, 0xa5a57bffu, 0xffffffffu}, {{ + {{{0x5a5a7c00u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a77ffu}, {0x5a5a7c00u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a77ffu}}}, + {{{0x5a5a7c00u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a77ffu}, {0x5a5a7c00u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a77ffu}}}, + {{{0x5a5a7c00u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a77ffu}, {0x5a5a7c00u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a77ffu}}}, + }}}, + {{0xff800000u, 0x5a5a0800u, 0xa5a5fbffu, 0x0000000au}, {{ + {{{0x5a5afc00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5afbffu}, {0x5a5afc00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5afbffu}}}, + {{{0x5a5afc00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5afc00u}, {0x5a5afc00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5afbffu}}}, + {{{0x5a5afc00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5afbffu}, {0x5a5afc00u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5afbffu}}}, + }}}, + {{0x7fc12345u, 0x5a5a0801u, 0xa5a57c00u, 0xfffffff6u}, {{ + {{{0x5a5a7e09u, 0x5a5a6801u, 0x5a5a6801u, 0x5a5a7c00u}, {0x5a5a7e09u, 0x5a5a6801u, 0x5a5a6801u, 0x5a5a7c00u}}}, + {{{0x5a5a7e09u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5a7c00u}, {0x5a5a7e09u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5a7c00u}}}, + {{{0x5a5a7e09u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5a7c00u}, {0x5a5a7e09u, 0x5a5a6800u, 0x5a5a6800u, 0x5a5a7c00u}}}, + }}}, + {{0xffc12345u, 0x5a5a0803u, 0xa5a5fc00u, 0x00000010u}, {{ + {{{0x5a5afe09u, 0x5a5a6802u, 0x5a5a6802u, 0x5a5afc00u}, {0x5a5afe09u, 0x5a5a6802u, 0x5a5a6802u, 0x5a5afc00u}}}, + {{{0x5a5afe09u, 0x5a5a6801u, 0x5a5a6801u, 0x5a5afc00u}, {0x5a5afe09u, 0x5a5a6801u, 0x5a5a6801u, 0x5a5afc00u}}}, + {{{0x5a5afe09u, 0x5a5a6801u, 0x5a5a6801u, 0x5a5afc00u}, {0x5a5afe09u, 0x5a5a6801u, 0x5a5a6801u, 0x5a5afc00u}}}, + }}}, + {{0x7f812345u, 0x5a5a7fffu, 0xa5a57e55u, 0xfffffff0u}, {{ + {{{0x5a5a7e09u, 0x5a5a7800u, 0x5a5a7800u, 0x5a5a7e55u}, {0x5a5a7e09u, 0x5a5a7800u, 0x5a5a7800u, 0x5a5a7e55u}}}, + {{{0x5a5a7e09u, 0x5a5a77ffu, 0x5a5a77ffu, 0x5a5a7e55u}, {0x5a5a7e09u, 0x5a5a77ffu, 0x5a5a77ffu, 0x5a5a7e55u}}}, + {{{0x5a5a7e09u, 0x5a5a77ffu, 0x5a5a77ffu, 0x5a5a7e55u}, {0x5a5a7e09u, 0x5a5a77ffu, 0x5a5a77ffu, 0x5a5a7e55u}}}, + }}}, + {{0xff812345u, 0x5a5a8000u, 0xa5a5fe55u, 0x00000018u}, {{ + {{{0x5a5afe09u, 0x5a5af800u, 0x5a5a7800u, 0x5a5afe55u}, {0x5a5afe09u, 0x5a5af800u, 0x5a5a7800u, 0x5a5afe55u}}}, + {{{0x5a5afe09u, 0x5a5af800u, 0x5a5a7800u, 0x5a5afe55u}, {0x5a5afe09u, 0x5a5af800u, 0x5a5a7800u, 0x5a5afe55u}}}, + {{{0x5a5afe09u, 0x5a5af800u, 0x5a5a7800u, 0x5a5afe55u}, {0x5a5afe09u, 0x5a5af800u, 0x5a5a7800u, 0x5a5afe55u}}}, + }}}, + {{0x00000000u, 0x00000000u, 0xa5a53c01u, 0x00000010u}, {{ + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a7c00u}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a7bffu}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a7bffu}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a7bffu}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a7bffu}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a7bffu}}}, + }}}, + {{0x00000000u, 0x00000000u, 0xa5a5bc01u, 0x00000010u}, {{ + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5afbffu}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5afbffu}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5afc00u}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5afbffu}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5afbffu}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5afbffu}}}, + }}}, + {{0x00000001u, 0x5a5a07ffu, 0xa5a50001u, 0xffffffffu}, {{ + {{{0x5a5a0001u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a0001u}, {0x5a5a0001u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a0001u}}}, + {{{0x5a5a0000u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a0000u}, {0x5a5a0000u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a0000u}}}, + {{{0x5a5a0000u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a0000u}, {0x5a5a0000u, 0x5a5a67ffu, 0x5a5a67ffu, 0x5a5a0000u}}}, + }}}, + {{0x00000000u, 0x00000000u, 0xa5a58001u, 0xffffffffu}, {{ + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a8000u}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a8000u}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a8001u}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a8001u}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a8000u}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a8000u}}}, + }}}, + {{0x00000000u, 0x00000000u, 0xa5a503ffu, 0x00000000u}, {{ + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a03ffu}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a03ffu}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a03ffu}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a03ffu}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a03ffu}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a03ffu}}}, + }}}, + {{0x00000000u, 0x00000000u, 0xa5a583ffu, 0x00000000u}, {{ + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a83ffu}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a83ffu}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a83ffu}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a83ffu}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a83ffu}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a83ffu}}}, + }}}, + {{0x00000000u, 0x00000000u, 0xa5a57bffu, 0x00007fffu}, {{ + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a7c00u}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a7bffu}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a7bffu}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a7bffu}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a7bffu}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a7bffu}}}, + }}}, + {{0x00000000u, 0x00000000u, 0xa5a5fbffu, 0xffff8000u}, {{ + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a8000u}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a8000u}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a8001u}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a8001u}}}, + {{{0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a8000u}, {0x5a5a0000u, 0x5a5a0000u, 0x5a5a0000u, 0x5a5a8000u}}}, + }}}, +}; + constexpr std::array Names{"v_cvt_f16_f32", "v_add_f16", "v_mul_f16"}; std::array BufferDescriptor(const void* data, std::uint32_t count) { @@ -342,6 +539,41 @@ void Run(AgcDriver::VulkanDevice& device, std::uint32_t first, std::uint32_t cou device.WaitIdle(); } +void RunDirected(AgcDriver::VulkanDevice& device, std::uint32_t rounding, std::uint32_t overflow, std::uint32_t round32) { + Input.fill(0u); + for (std::uint32_t lane = 0; lane < std::size(DirectedVectors); ++lane) { + std::copy(DirectedVectors[lane].inputs.begin(), DirectedVectors[lane].inputs.end(), Input.begin() + lane * Inputs); + } + Output.fill(0xdeadbeefu); + std::vector userData(8, 0u); + const auto input = BufferDescriptor(Input.data(), static_cast(Input.size())); + const auto output = BufferDescriptor(Output.data(), static_cast(Output.size())); + std::copy(input.begin(), input.end(), userData.begin()); + std::copy(output.begin(), output.end(), userData.begin() + 4); + const std::span code(DirectedCode); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0, {false, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.context.floatMode = ShaderRecompiler::ShaderFloatMode{0xf0u | (rounding << 2u) | round32, true, true, overflow != 0u}; + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); + for (std::uint32_t lane = 0; lane < std::size(DirectedVectors); ++lane) { + for (std::uint32_t column = 0; column < 4u; ++column) { + const auto expected = DirectedVectors[lane].expected[rounding - 1u][overflow][column]; + const auto actual = Output[lane * Results + column]; + const bool matches = column == 0u ? Matches(actual, expected) : actual == expected; + Require(matches, "directed f16 rounding: mode " + std::to_string(rounding) + " overflow " + std::to_string(overflow) + " lane " + std::to_string(lane) + " column " + std::to_string(column) + " is " + Hex(actual) + ", expected " + Hex(expected)); + } + } +} + void Check(std::uint32_t first, std::uint32_t count) { for (std::uint32_t lane = 0; lane < count; ++lane) { const auto& vector = Vectors[first + lane]; @@ -365,6 +597,12 @@ int main() { Run(*device, first, count); Check(first, count); } + for (std::uint32_t rounding = 1u; rounding < 4u; ++rounding) { + for (std::uint32_t overflow = 0u; overflow < 2u; ++overflow) { + RunDirected(*device, rounding, overflow, 0u); + RunDirected(*device, rounding, overflow, 3u); + } + } std::puts("f16 rounding tests passed"); return 0; } catch (const std::exception& error) { diff --git a/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp b/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp index e6340cfdb..4d7e56668 100644 --- a/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp +++ b/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp @@ -59,6 +59,9 @@ private: IrU32 packHalf2x16(IrF32 low, IrF32 high); void write16Bits(const RdnaOperand& operand, IrU32 value); void writeF16(const RdnaOperand& operand, IrF32 value, std::initializer_list sources); + bool directedF16Rounding() const; + IrU32 convertF32ToF16Bits(IrF32 value, std::initializer_list sources); + void writeF16Conversion(const RdnaOperand& operand, IrF32 value, std::initializer_list sources); IrU32 readU32(const RdnaOperand& operand); IrU32 flushF32Denormal(IrU32 bits); IrF32 flushTinyProduct(IrValue* lhs, IrValue* rhs, IrValue* product, IrValue* addend = nullptr); diff --git a/core/shader/recompiler/Translation/src/ConvertInstructions.cpp b/core/shader/recompiler/Translation/src/ConvertInstructions.cpp index 91241ac31..863a33b0f 100644 --- a/core/shader/recompiler/Translation/src/ConvertInstructions.cpp +++ b/core/shader/recompiler/Translation/src/ConvertInstructions.cpp @@ -79,7 +79,7 @@ void TranslationContext::vCvtI32F32(const RdnaInstruction& inst) { void TranslationContext::vCvtF16F32(const RdnaInstruction& inst) { const IrF32 value(*readOperand(sourceAt(inst, 0u), IrType::F32)); - writeF16(inst.destination, value, {&value.Value()}); + writeF16Conversion(inst.destination, value, {&value.Value()}); } void TranslationContext::vCvtF32F16(const RdnaInstruction& inst) { @@ -91,7 +91,7 @@ void TranslationContext::vCvtF1616(const RdnaInstruction& inst, bool signedValue const IrU32 source = readU16AsU32(sourceAt(inst, 0u), signedValue); const IrOpcode opcode = signedValue ? IrOpcode::ConvertF32S32 : IrOpcode::ConvertF32U32; const IrF32 value(ir.Emit(opcode, IrType::F32, {&source.Value()})); - writeF16(inst.destination, value, {}); + writeF16Conversion(inst.destination, value, {}); } void TranslationContext::vCvt16F16(const RdnaInstruction& inst, bool signedValue) { diff --git a/core/shader/recompiler/Translation/src/FloatInstructions.cpp b/core/shader/recompiler/Translation/src/FloatInstructions.cpp index b85979973..6fd8cb926 100644 --- a/core/shader/recompiler/Translation/src/FloatInstructions.cpp +++ b/core/shader/recompiler/Translation/src/FloatInstructions.cpp @@ -320,8 +320,7 @@ bool TranslationContext::vLdexpF16(const RdnaInstruction& inst) { const IrU32 exponent(ir.Emit(IrOpcode::BitFieldSExtract, IrType::U32, {&exponentBits.Value(), &ir.Constant(0u), &ir.Constant(16u)})); const IrU32 clamped(ir.Emit(IrOpcode::SMax32, IrType::U32, {&ir.Emit(IrOpcode::SMin32, IrType::U32, {&exponent.Value(), &ir.Constant(64u)}), &ir.Constant(static_cast(-64))})); IrValue& power = ir.BitCastF32(ir.ShiftLeftLogical(ir.IAdd(clamped.Value(), ir.Constant(127u)), ir.Constant(23u))); - const IrF16 scaled(ir.Emit(IrOpcode::ConvertF16F32, IrType::F16, {&clampF16Overflow(IrF32(ir.Emit(IrOpcode::FPMul32, IrType::F32, {&value.Value(), &power})), {&value.Value()}).Value()})); - const IrU32 result(ir.Emit(IrOpcode::ConvertU32U16, IrType::U32, {&ir.Emit(IrOpcode::BitCastU16F16, IrType::U16, {&scaled.Value()})})); + const IrU32 result = convertF32ToF16Bits(IrF32(ir.Emit(IrOpcode::FPMul32, IrType::F32, {&value.Value(), &power})), {&value.Value()}); const IrU1 nan(ir.UGreaterThan(ir.BitwiseAnd(bits.Value(), ir.Constant(0x7fffu)), ir.Constant(0x7c00u))); write16Bits(inst.destination, clampF16Bits(inst.destination, IrU32(ir.Select(nan.Value(), quietNan16(bits).Value(), result.Value())))); return true; diff --git a/core/shader/recompiler/Translation/src/TranslationContext/SubwordPacking.cpp b/core/shader/recompiler/Translation/src/TranslationContext/SubwordPacking.cpp index 69f5f4549..e4e3de5c6 100644 --- a/core/shader/recompiler/Translation/src/TranslationContext/SubwordPacking.cpp +++ b/core/shader/recompiler/Translation/src/TranslationContext/SubwordPacking.cpp @@ -156,6 +156,49 @@ void TranslationContext::write16Bits(const RdnaOperand& operand, IrU32 value) { writeRawU32(operand, IrU32(ir.BitwiseOr(cleared.Value(), masked.Value()))); } +bool TranslationContext::directedF16Rounding() const { + return floatMode.has_value() && ((floatMode->floatMode >> 2u) & 3u) != 0u && ((floatMode->floatMode >> 6u) & 3u) == 3u; +} + +IrU32 TranslationContext::convertF32ToF16Bits(IrF32 value, std::initializer_list sources) { + const IrF16 half(ir.Emit(IrOpcode::ConvertF16F32, IrType::F16, {&clampF16Overflow(value, sources).Value()})); + const IrU32 native(ir.Emit(IrOpcode::ConvertU32U16, IrType::U32, {&ir.Emit(IrOpcode::BitCastU16F16, IrType::U16, {&half.Value()})})); + if (!directedF16Rounding()) { + return native; + } + const std::uint32_t rounding = (floatMode->floatMode >> 2u) & 3u; + const IrU32 bits(ir.BitCastU32(value.Value())); + const IrU32 sign(ir.BitwiseAnd(ir.ShiftRightLogical(bits.Value(), ir.Constant(16u)), ir.Constant(0x8000u))); + const IrU32 magnitude(ir.BitwiseAnd(bits.Value(), ir.Constant(0x7fffffffu))); + const IrU32 exponent(ir.ShiftRightLogical(magnitude.Value(), ir.Constant(23u))); + const IrU1 normal(ir.UGreaterThan(exponent.Value(), ir.Constant(112u))); + const IrU32 significand(ir.BitwiseOr(ir.BitwiseAnd(magnitude.Value(), ir.Constant(0x7fffffu)), ir.Constant(0x800000u))); + const IrU32 shift(ir.Emit(IrOpcode::UMin32, IrType::U32, {&ir.ISub(ir.Constant(126u), exponent.Value()), &ir.Constant(24u)})); + const IrU32 subnormal(ir.ShiftRightLogical(significand.Value(), shift.Value())); + const IrU32 truncated(ir.Select(normal.Value(), ir.ISub(ir.ShiftRightLogical(magnitude.Value(), ir.Constant(13u)), ir.Constant(0x1c000u)), subnormal.Value())); + const IrU1 tiny(ir.ULessThan(exponent.Value(), ir.Constant(103u))); + const IrU32 discarded(ir.Select(normal.Value(), ir.BitwiseAnd(magnitude.Value(), ir.Constant(0x1fffu)), + ir.Select(tiny.Value(), magnitude.Value(), ir.ISub(significand.Value(), ir.ShiftLeftLogical(subnormal.Value(), shift.Value()))))); + const IrU1 away(rounding == 1u ? ir.IEqual(sign.Value(), ir.Constant(0u)) : rounding == 2u ? ir.INotEqual(sign.Value(), ir.Constant(0u)) : ir.ConstantBool(false)); + const IrU1 increment(ir.LogicalAnd(away.Value(), ir.INotEqual(discarded.Value(), ir.Constant(0u)))); + const IrU32 rounded(ir.IAdd(truncated.Value(), ir.Select(increment.Value(), ir.Constant(1u), ir.Constant(0u)))); + const IrU32 overflow(fp16Overflow() ? ir.Constant(0x7bffu) : ir.Select(away.Value(), ir.Constant(0x7c00u), ir.Constant(0x7bffu))); + IrU32 result(ir.Select(ir.UGreaterThan(exponent.Value(), ir.Constant(142u)), overflow.Value(), rounded.Value())); + if (fp16Overflow()) { + result = IrU32(ir.Select(ir.IEqual(result.Value(), ir.Constant(0x7c00u)), ir.Constant(0x7bffu), result.Value())); + } + result = IrU32(ir.BitwiseOr(sign.Value(), result.Value())); + return IrU32(ir.Select(ir.ULessThan(magnitude.Value(), ir.Constant(0x7f800000u)), result.Value(), native.Value())); +} + +void TranslationContext::writeF16Conversion(const RdnaOperand& operand, IrF32 value, std::initializer_list sources) { + if (!directedF16Rounding()) { + writeF16(operand, value, sources); + return; + } + write16Bits(operand, convertF32ToF16Bits(applyF16ResultModifiers(operand, value), sources)); +} + void TranslationContext::writeF16(const RdnaOperand& operand, IrF32 value, std::initializer_list sources) { const IrF16 half(ir.Emit(IrOpcode::ConvertF16F32, IrType::F16, {&applyF16ResultModifiers(operand, clampF16Overflow(value, sources)).Value()})); const IrU16 bits(ir.Emit(IrOpcode::BitCastU16F16, IrType::U16, {&half.Value()})); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index f47df2886..e2afb335a 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -436,6 +436,7 @@ Throughout the project, every function at every stage either **does exactly what - The [output modifier](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) (`mul:2`, `mul:4`, `div:2`) follows the shader's float mode as measured on RDNA2 (gfx1035): the hardware applies it only with IEEE mode off and output denormals of the result's precision flushed (bit 1 of the `FLOAT_MODE` denormal field clear), and ignores it otherwise. On f32 results it is applied as the hardware does with f32 denormals flushed; a shader without a known float mode is translated with IEEE mode off and f32 denormals flushed, the mode titles use. On f16 and f64 results it is ignored, and it throws in the modes where the hardware applies it. - The [clamp modifier](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follows `DX10_CLAMP`, measured on RDNA2 (gfx1035) for f32, f16, packed f16 and f64 results at IEEE=0 and IEEE=1: with `DX10_CLAMP`=1 (and in shaders without a known float mode) a NaN result clamps to +0, including the quieted NaN that `v_pk_min_f16` and `v_pk_max_f16` return for a signaling NaN operand with IEEE=1 (measured on RDNA2 gfx1036 at both `IEEE_MODE` and both `DX10_CLAMP` settings); with `DX10_CLAMP`=0 a NaN result is written unchanged and every other value clamps to [0, 1] as before. f64 results follow the same rule on hardware, but a clamped f64 result with `DX10_CLAMP`=0 throws until it has an execution test. With `DX10_CLAMP`=0 the NaN's bits are those the translation already produces for the instruction without clamp, so where those are left to the driver (f32 and scalar f16 arithmetic) the payload is the driver's, not the hardware's input propagation. `v_dot2_f32_f16` and the integer clamps are unchanged. - [FP16_OVFL](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follows the shader's float mode as measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1 with f16 denormals kept and flushed: with `FP16_OVFL`=1 an f16 result that rounds to ±infinity is written as ±0x7bff unless a source operand is infinite, so `v_rcp_f16`, `v_rsq_f16` and `v_log_f16` of ±0 and `v_exp_f16` of 16 or more are clamped while an infinite input passes through, and `v_div_fixup_f16` keeps the infinity it writes for a zero denominator or an infinite numerator and clamps any other infinite quotient, a NaN one included. `v_cvt_pkrtz_f16_f32` rounds toward zero and never overflows. The sources are checked as the f32 values the translation computes with; the rule is applied to `v_fma_mix*_f16` with f32 sources and to `v_interp_p2_f16` without a measurement of infinite f32 sources or parameters. Round modes other than nearest even are not modelled, so the overflow threshold is the nearest-even one (65520). +- [Directed f16 rounding](../../core/shader/recompiler/Translation/src/TranslationContext/SubwordPacking.cpp) for `v_cvt_f16_f32`, `v_cvt_f16_i16`, `v_cvt_f16_u16` and `v_ldexp_f16` follows `FLOAT_MODE` bits 3:2 only when f16 denormals are preserved (bits 7:6 = 3), measured on RDNA2 (gfx1036). With f16 denormals flushed (bits 7:6 = 0), or flushed on one side (modes 1 and 2, not measured), the hardware also flushes the f16 result and the directed path does not model that, so those shaders keep round-to-nearest-even. The `clamp` modifier with directed rounding is not measured. - [Comparison sampling of a color texture](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_sample_c`/`image_sample_c_lz` on a texture that is not R32 float or R16 unorm) is emulated in the shader, since Vulkan compares only depth formats: the red channel is fetched, compared and, for bilinear filtering, blended (compare, then filter). Only 2D and 2D array views, float/unorm/snorm formats, wrap, clamp-to-edge and clamp-to-border addressing (border red 0 for the black borders, 1 for opaque white) and point or bilinear filtering at one level are implemented, with texel offsets added to the texel indices; LOD biases and clamps are accepted, as they cannot select another level of a single-level sample. Half-border addressing, the border color table, gradients, gathers, other dimensions, anisotropy and mip filtering throw. Under bilinear filtering, taps past a clamp-to-border edge compare the border value and blend with the texel taps, so results near the edge depend on the filter weights. AMD documentation does not say how a color format's reference is handled: it is clamped to [0, 1] for unorm and [-1, 1] for snorm, as Vulkan does for unorm depth, and not clamped for float. Bilinear weights are full float, not the hardware's fixed-point subtexel weights - [Negative color-texture comparison wrap indices](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (libSceAgcDriver) use explicit `OpSRem` normalization to implement modulo; the negative-offset point-wrap regression fails with direct `OpSMod` and passes with normalization on Mesa 22.3.6 llvmpipe (LLVM 15.0.6), but negative-index wrap has not been measured on PS5 hardware. - [libScePlayerInvitationDialog](../../core/libs/prx/libScePlayerInvitationDialog/libScePlayerInvitationDialog.cpp) simulates dialog completion without displaying UI or sending invitations, so `scePlayerInvitationDialogGetResult` reports `USER_CANCELED`; its parameter ABI remains unverified. The 40-byte result layout (error code, result, 32 reserved bytes) is taken from [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Dialog/PlayerInvitationDialog.cs) and the error codes from the other common dialogs, not confirmed on a title. From 5830d8e446d151ec1d947e803d4439e6931cca31 Mon Sep 17 00:00:00 2001 From: Hussein Kadhim Date: Sat, 10 Oct 2026 15:17:42 +0300 Subject: [PATCH 159/460] fix(libc): use FreeBSD errlst message texts for strerror and strerror_r Twenty-five of the ninety-seven errno messages differed from FreeBSD releng/11.4 lib/libc/gen/errlst.c (for example errno 0, 30, 47, 62, 70 and 84). (cherry picked from commit ce14c9ad7dc639335ff72b60122bae2bd0fee302) --- core/libs/prx/libc/src/Errors.cpp | 50 ++++++++++++++++++------------- core/libs/tests/GuestErrors.cpp | 38 ++++++++++++++++++++++- 2 files changed, 66 insertions(+), 22 deletions(-) diff --git a/core/libs/prx/libc/src/Errors.cpp b/core/libs/prx/libc/src/Errors.cpp index 963191cf6..c8553460a 100644 --- a/core/libs/prx/libc/src/Errors.cpp +++ b/core/libs/prx/libc/src/Errors.cpp @@ -7,31 +7,39 @@ extern "C" int* APS5_VABI __error_nid_postfix(); namespace { // Guest FreeBSD errno numbering, not the host CRT's numbering. constexpr const char* messages[] = { - "No error", "Operation not permitted", "No such file or directory", "No such process", - "Interrupted system call", "Input/output error", "Device not configured", "Argument list too long", - "Exec format error", "Bad file descriptor", "No child processes", "Resource deadlock avoided", - "Cannot allocate memory", "Permission denied", "Bad address", "Block device required", - "Device busy", "File exists", "Cross-device link", "Operation not supported by device", - "Not a directory", "Is a directory", "Invalid argument", "Too many open files in system", - "Too many open files", "Inappropriate ioctl for device", "Text file busy", "File too large", - "No space left on device", "Illegal seek", "Read-only filesystem", "Too many links", "Broken pipe", + "No error: 0", "Operation not permitted", "No such file or directory", + "No such process", "Interrupted system call", "Input/output error", + "Device not configured", "Argument list too long", "Exec format error", + "Bad file descriptor", "No child processes", "Resource deadlock avoided", + "Cannot allocate memory", "Permission denied", "Bad address", + "Block device required", "Device busy", "File exists", + "Cross-device link", "Operation not supported by device", "Not a directory", + "Is a directory", "Invalid argument", "Too many open files in system", + "Too many open files", "Inappropriate ioctl for device", "Text file busy", + "File too large", "No space left on device", "Illegal seek", + "Read-only file system", "Too many links", "Broken pipe", "Numerical argument out of domain", "Result too large", "Resource temporarily unavailable", "Operation now in progress", "Operation already in progress", "Socket operation on non-socket", "Destination address required", "Message too long", "Protocol wrong type for socket", "Protocol not available", "Protocol not supported", "Socket type not supported", - "Operation not supported", "Protocol family not supported", "Address family not supported", - "Address already in use", "Cannot assign requested address", "Network is down", "Network is unreachable", - "Network connection reset", "Connection aborted", "Connection reset by peer", "No buffer space available", - "Socket is already connected", "Socket is not connected", "Socket is shut down", "Too many references", - "Operation timed out", "Connection refused", "Too many symbolic links", "File name too long", - "Host is down", "No route to host", "Directory not empty", "Too many processes", "Too many users", - "Disk quota exceeded", "Stale file handle", "Remote path has too many levels", "Invalid RPC structure", - "RPC version mismatch", "RPC program unavailable", "Program version mismatch", "Invalid RPC procedure", - "No locks available", "Function not implemented", "Inappropriate file type", "Authentication error", - "Authentication required", "Identifier removed", "No message of requested type", "Value too large", - "Operation canceled", "Invalid byte sequence", "Attribute not found", "Programming error", - "Bad message", "Multihop attempted", "Link severed", "Protocol error", "Insufficient capabilities", - "Operation not permitted in capability mode", "State not recoverable", "Previous owner died" + "Operation not supported", "Protocol family not supported", "Address family not supported by protocol family", + "Address already in use", "Can't assign requested address", "Network is down", + "Network is unreachable", "Network dropped connection on reset", "Software caused connection abort", + "Connection reset by peer", "No buffer space available", "Socket is already connected", + "Socket is not connected", "Can't send after socket shutdown", "Too many references: can't splice", + "Operation timed out", "Connection refused", "Too many levels of symbolic links", + "File name too long", "Host is down", "No route to host", + "Directory not empty", "Too many processes", "Too many users", + "Disc quota exceeded", "Stale NFS file handle", "Too many levels of remote in path", + "RPC struct is bad", "RPC version wrong", "RPC prog. not avail", + "Program version wrong", "Bad procedure for program", "No locks available", + "Function not implemented", "Inappropriate file type or format", "Authentication error", + "Need authenticator", "Identifier removed", "No message of desired type", + "Value too large to be stored in data type", "Operation canceled", "Illegal byte sequence", + "Attribute not found", "Programming error", "Bad message", + "Multihop attempted", "Link has been severed", "Protocol error", + "Capabilities insufficient", "Not permitted in capability mode", "State not recoverable", + "Previous owner died" }; static_assert(sizeof(messages) / sizeof(messages[0]) == 97); bool Known(int error) { return error >= 0 && error < 97; } diff --git a/core/libs/tests/GuestErrors.cpp b/core/libs/tests/GuestErrors.cpp index 52160d562..2679cb928 100644 --- a/core/libs/tests/GuestErrors.cpp +++ b/core/libs/tests/GuestErrors.cpp @@ -9,6 +9,41 @@ int APS5_VABI strerror_r_nid_postfix(int, char*, std::size_t); int* APS5_VABI __error_nid_postfix(); } static void Require(bool value) { if (!value) std::abort(); } +static const char* const freebsd[97] = { + "No error: 0", "Operation not permitted", "No such file or directory", + "No such process", "Interrupted system call", "Input/output error", + "Device not configured", "Argument list too long", "Exec format error", + "Bad file descriptor", "No child processes", "Resource deadlock avoided", + "Cannot allocate memory", "Permission denied", "Bad address", + "Block device required", "Device busy", "File exists", + "Cross-device link", "Operation not supported by device", "Not a directory", + "Is a directory", "Invalid argument", "Too many open files in system", + "Too many open files", "Inappropriate ioctl for device", "Text file busy", + "File too large", "No space left on device", "Illegal seek", + "Read-only file system", "Too many links", "Broken pipe", + "Numerical argument out of domain", "Result too large", "Resource temporarily unavailable", + "Operation now in progress", "Operation already in progress", "Socket operation on non-socket", + "Destination address required", "Message too long", "Protocol wrong type for socket", + "Protocol not available", "Protocol not supported", "Socket type not supported", + "Operation not supported", "Protocol family not supported", "Address family not supported by protocol family", + "Address already in use", "Can't assign requested address", "Network is down", + "Network is unreachable", "Network dropped connection on reset", "Software caused connection abort", + "Connection reset by peer", "No buffer space available", "Socket is already connected", + "Socket is not connected", "Can't send after socket shutdown", "Too many references: can't splice", + "Operation timed out", "Connection refused", "Too many levels of symbolic links", + "File name too long", "Host is down", "No route to host", + "Directory not empty", "Too many processes", "Too many users", + "Disc quota exceeded", "Stale NFS file handle", "Too many levels of remote in path", + "RPC struct is bad", "RPC version wrong", "RPC prog. not avail", + "Program version wrong", "Bad procedure for program", "No locks available", + "Function not implemented", "Inappropriate file type or format", "Authentication error", + "Need authenticator", "Identifier removed", "No message of desired type", + "Value too large to be stored in data type", "Operation canceled", "Illegal byte sequence", + "Attribute not found", "Programming error", "Bad message", + "Multihop attempted", "Link has been severed", "Protocol error", + "Capabilities insufficient", "Not permitted in capability mode", "State not recoverable", + "Previous owner died" +}; int main() { *__error_nid_postfix() = 13; Require(std::strcmp(strerror_nid_postfix(35), "Resource temporarily unavailable") == 0); @@ -25,7 +60,8 @@ int main() { char buffer[128]; for (int error = 0; error <= 96; ++error) { Require(strerror_r_nid_postfix(error, buffer, sizeof(buffer)) == 0); - Require(buffer[0] && std::strstr(buffer, "Unknown") == nullptr); + Require(std::strcmp(buffer, freebsd[error]) == 0); + Require(std::strcmp(strerror_nid_postfix(error), freebsd[error]) == 0); } Require(strerror_r_nid_postfix(-1, buffer, sizeof(buffer)) == 22); Require(std::strstr(buffer, "-1") != nullptr); From 57da6e662e78a403d714c58086e2f9462ce641d1 Mon Sep 17 00:00:00 2001 From: Hussein Kadhim Date: Sat, 10 Oct 2026 15:24:09 +0300 Subject: [PATCH 160/460] test(libc): check the exact unknown-error texts (cherry picked from commit bbd7e79cda352794b86dea85b6db77ab08ad4cda) --- core/libs/tests/GuestErrors.cpp | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/core/libs/tests/GuestErrors.cpp b/core/libs/tests/GuestErrors.cpp index 2679cb928..8429b8150 100644 --- a/core/libs/tests/GuestErrors.cpp +++ b/core/libs/tests/GuestErrors.cpp @@ -1,4 +1,5 @@ #include "prx/libc/include/general/VabiMacros.hpp" +#include #include #include #include @@ -64,8 +65,21 @@ int main() { Require(std::strcmp(strerror_nid_postfix(error), freebsd[error]) == 0); } Require(strerror_r_nid_postfix(-1, buffer, sizeof(buffer)) == 22); - Require(std::strstr(buffer, "-1") != nullptr); + Require(std::strcmp(buffer, "Unknown error: -1") == 0); Require(*__error_nid_postfix() == 13); + struct UnknownError { + int error; + const char* text; + }; + for (const UnknownError& unknown : {UnknownError{97, "Unknown error: 97"}, UnknownError{INT_MAX, "Unknown error: 2147483647"}, + UnknownError{INT_MIN, "Unknown error: -2147483648"}}) { + Require(strerror_r_nid_postfix(unknown.error, buffer, sizeof(buffer)) == 22); + Require(std::strcmp(buffer, unknown.text) == 0); + Require(*__error_nid_postfix() == 13); + Require(std::strcmp(strerror_nid_postfix(unknown.error), unknown.text) == 0); + Require(*__error_nid_postfix() == 22); + *__error_nid_postfix() = 13; + } char sentinel[] = "xyz"; Require(strerror_r_nid_postfix(22, sentinel, 1) == 34); Require(sentinel[0] == 0 && sentinel[1] == 'y'); From ee0b249e4e3623c03246a07409f5f7599c70224f Mon Sep 17 00:00:00 2001 From: David Date: Sat, 10 Oct 2026 23:39:27 +0400 Subject: [PATCH 161/460] fix(libkernel): report ENOTDIR from open with O_DIRECTORY on a file on Windows (cherry picked from commit b2e9f59959f3eef10db3f68886ee9d9ec1d50ea4) --- core/libs/prx/libkernel/File/src/Open.cpp | 7 +++++++ core/libs/tests/GuestFilesystem.cpp | 7 +++++++ 2 files changed, 14 insertions(+) diff --git a/core/libs/prx/libkernel/File/src/Open.cpp b/core/libs/prx/libkernel/File/src/Open.cpp index 0e39397fc..59aeb2b45 100644 --- a/core/libs/prx/libkernel/File/src/Open.cpp +++ b/core/libs/prx/libkernel/File/src/Open.cpp @@ -124,6 +124,13 @@ int APS5_VABI sceKernelOpen(const char* path, int flags, std::uint16_t mode) { return fd < 0 ? SceErrorFromErrno(errno) : fd; } auto native = ResolvePath_nid_no_patch(path); +#ifdef _WIN32 + if ((flags & (SCE_KERNEL_O_DIRECTORY | SCE_KERNEL_O_CREAT)) == SCE_KERNEL_O_DIRECTORY) { + std::error_code error; + const auto status = std::filesystem::status(native, error); + if (std::filesystem::exists(status) && !std::filesystem::is_directory(status)) return SceErrorFromErrno(ENOTDIR); + } +#endif int fd = NativeOpen(native, MapFlags(flags), mode); #ifdef _WIN32 if (fd < 0 && errno != ENOENT) { diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index c42534124..995bd6cd0 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -342,6 +342,13 @@ int main() { Require(open_nid_postfix(rootName.c_str(), 0x0001, 0) == -1 && *__error_nid_postfix() == 21); const int directoryDescriptor = open_nid_postfix(rootName.c_str(), 0, 0); Require(directoryDescriptor >= 0 && close_nid_postfix(directoryDescriptor) == 0); + const int flaggedDirectory = sceKernelOpen(rootName.c_str(), 0x20000, 0); + Require(flaggedDirectory >= 0 && sceKernelClose(flaggedDirectory) == 0); + Require(open_nid_postfix(presentName.c_str(), 0x20000, 0) == -1 && *__error_nid_postfix() == 20); + Require(sceKernelOpen(presentName.c_str(), 0x20000, 0) == static_cast(0x80020014u)); + Require(sceKernelOpen(presentName.c_str(), 0x20402, 0) == static_cast(0x80020014u)); + Require(std::filesystem::file_size(present) == 5); + Require(sceKernelOpen(missingName.c_str(), 0x20000, 0) == static_cast(0x80020002u)); Require(open_nid_postfix("", 0, 0) == -1 && *__error_nid_postfix() == 2); Require(_open_nid_postfix("", 0) == -1 && *__error_nid_postfix() == 2); Require(open_nid_postfix(nullptr, 0, 0) == -1 && *__error_nid_postfix() == 14); From 43ff4e926a84aa040ed187f69932b8c64a9812f6 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 16:26:30 -0400 Subject: [PATCH 162/460] fix(shader): honor directed round32 in integer to f32 conversions v_cvt_f32_i32 and v_cvt_f32_u32 used the host Convert which always rounds to nearest even. When FLOAT_MODE bits 1:0 are directed, build the f32 bits from the integer magnitude so discarded low bits follow round toward +inf, -inf or zero, as measured on RDNA2 (gfx1036). Co-Authored-By: Composer --- .../tests/execution/IntegerToF32Rounding.cpp | 130 ++++++++++++++++++ .../Translation/TranslationContext.hpp | 4 + .../Translation/src/ConvertInstructions.cpp | 45 +++++- 3 files changed, 177 insertions(+), 2 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/IntegerToF32Rounding.cpp diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/IntegerToF32Rounding.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/IntegerToF32Rounding.cpp new file mode 100644 index 000000000..3f6fd3985 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/IntegerToF32Rounding.cpp @@ -0,0 +1,130 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 32; +constexpr std::uint32_t Inputs = 4; +constexpr std::uint32_t Results = 16; +alignas(256) std::array Input{}; +alignas(256) std::array Output{}; + +alignas(256) constexpr std::array Code{ + 0x34020084u, 0x34040086u, 0xe0381000u, 0x80000401u, 0xbf8c3f70u, 0x7e1402ffu, 0x5a5a5a5au, 0x7e1602ffu, + 0x5a5a5a5au, 0x7e140b04u, 0x7e160d05u, 0xe0781000u, 0x80010a02u, 0xbf810000u, +}; + +constexpr std::uint32_t Values[32] = { + 0x00000000u, 0x00000001u, 0x00000002u, 0x00000003u, 0x00000005u, 0x00000008u, 0x0000000fu, 0x00000010u, + 0x00000018u, 0x0000001fu, 0x00000020u, 0x00000021u, 0x0000003fu, 0x00007fffu, 0x00008000u, 0x0000ffffu, + 0x00010000u, 0x007fffffu, 0x00800000u, 0x00ffffffu, 0x01000000u, 0x12345678u, 0x7fffffffu, 0x80000000u, + 0x80000001u, 0xfffffffeu, 0xffffffffu, 0xdeadbeefu, 0x3f800000u, 0x00ff00ffu, 0xffff8000u, 0x0000807fu, +}; + +constexpr std::array, 32> Expected[3] = { + {{ + {{0x00000000u, 0x00000000u}}, {{0x3f800000u, 0x3f800000u}}, {{0x40000000u, 0x40000000u}}, {{0x40400000u, 0x40400000u}}, + {{0x40a00000u, 0x40a00000u}}, {{0x41000000u, 0x41000000u}}, {{0x41700000u, 0x41700000u}}, {{0x41800000u, 0x41800000u}}, + {{0x41c00000u, 0x41c00000u}}, {{0x41f80000u, 0x41f80000u}}, {{0x42000000u, 0x42000000u}}, {{0x42040000u, 0x42040000u}}, + {{0x427c0000u, 0x427c0000u}}, {{0x46fffe00u, 0x46fffe00u}}, {{0x47000000u, 0x47000000u}}, {{0x477fff00u, 0x477fff00u}}, + {{0x47800000u, 0x47800000u}}, {{0x4afffffeu, 0x4afffffeu}}, {{0x4b000000u, 0x4b000000u}}, {{0x4b7fffffu, 0x4b7fffffu}}, + {{0x4b800000u, 0x4b800000u}}, {{0x4d91a2b4u, 0x4d91a2b4u}}, {{0x4f000000u, 0x4f000000u}}, {{0xcf000000u, 0x4f000000u}}, + {{0xceffffffu, 0x4f000001u}}, {{0xc0000000u, 0x4f800000u}}, {{0xbf800000u, 0x4f800000u}}, {{0xce054904u, 0x4f5eadbfu}}, + {{0x4e7e0000u, 0x4e7e0000u}}, {{0x4b7f00ffu, 0x4b7f00ffu}}, {{0xc7000000u, 0x4f7fff80u}}, {{0x47007f00u, 0x47007f00u}}, + }}, + {{ + {{0x00000000u, 0x00000000u}}, {{0x3f800000u, 0x3f800000u}}, {{0x40000000u, 0x40000000u}}, {{0x40400000u, 0x40400000u}}, + {{0x40a00000u, 0x40a00000u}}, {{0x41000000u, 0x41000000u}}, {{0x41700000u, 0x41700000u}}, {{0x41800000u, 0x41800000u}}, + {{0x41c00000u, 0x41c00000u}}, {{0x41f80000u, 0x41f80000u}}, {{0x42000000u, 0x42000000u}}, {{0x42040000u, 0x42040000u}}, + {{0x427c0000u, 0x427c0000u}}, {{0x46fffe00u, 0x46fffe00u}}, {{0x47000000u, 0x47000000u}}, {{0x477fff00u, 0x477fff00u}}, + {{0x47800000u, 0x47800000u}}, {{0x4afffffeu, 0x4afffffeu}}, {{0x4b000000u, 0x4b000000u}}, {{0x4b7fffffu, 0x4b7fffffu}}, + {{0x4b800000u, 0x4b800000u}}, {{0x4d91a2b3u, 0x4d91a2b3u}}, {{0x4effffffu, 0x4effffffu}}, {{0xcf000000u, 0x4f000000u}}, + {{0xcf000000u, 0x4f000000u}}, {{0xc0000000u, 0x4f7fffffu}}, {{0xbf800000u, 0x4f7fffffu}}, {{0xce054905u, 0x4f5eadbeu}}, + {{0x4e7e0000u, 0x4e7e0000u}}, {{0x4b7f00ffu, 0x4b7f00ffu}}, {{0xc7000000u, 0x4f7fff80u}}, {{0x47007f00u, 0x47007f00u}}, + }}, + {{ + {{0x00000000u, 0x00000000u}}, {{0x3f800000u, 0x3f800000u}}, {{0x40000000u, 0x40000000u}}, {{0x40400000u, 0x40400000u}}, + {{0x40a00000u, 0x40a00000u}}, {{0x41000000u, 0x41000000u}}, {{0x41700000u, 0x41700000u}}, {{0x41800000u, 0x41800000u}}, + {{0x41c00000u, 0x41c00000u}}, {{0x41f80000u, 0x41f80000u}}, {{0x42000000u, 0x42000000u}}, {{0x42040000u, 0x42040000u}}, + {{0x427c0000u, 0x427c0000u}}, {{0x46fffe00u, 0x46fffe00u}}, {{0x47000000u, 0x47000000u}}, {{0x477fff00u, 0x477fff00u}}, + {{0x47800000u, 0x47800000u}}, {{0x4afffffeu, 0x4afffffeu}}, {{0x4b000000u, 0x4b000000u}}, {{0x4b7fffffu, 0x4b7fffffu}}, + {{0x4b800000u, 0x4b800000u}}, {{0x4d91a2b3u, 0x4d91a2b3u}}, {{0x4effffffu, 0x4effffffu}}, {{0xcf000000u, 0x4f000000u}}, + {{0xceffffffu, 0x4f000000u}}, {{0xc0000000u, 0x4f7fffffu}}, {{0xbf800000u, 0x4f7fffffu}}, {{0xce054904u, 0x4f5eadbeu}}, + {{0x4e7e0000u, 0x4e7e0000u}}, {{0x4b7f00ffu, 0x4b7f00ffu}}, {{0xc7000000u, 0x4f7fff80u}}, {{0x47007f00u, 0x47007f00u}}, + }}, +}; + +std::array BufferDescriptor(const void* data, std::uint32_t words) { + const auto address = reinterpret_cast(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu), words * 4u, 0x31016facu}; +} + +std::string Hex(std::uint32_t value) { + char text[16]; + std::snprintf(text, sizeof(text), "0x%08x", value); + return text; +} + +void Run(AgcDriver::VulkanDevice& device, std::uint32_t rounding) { + Input.fill(0u); + for (std::uint32_t lane = 0; lane < Threads; ++lane) { + Input[lane * Inputs] = Values[lane]; + Input[lane * Inputs + 1] = Values[lane]; + } + Output.fill(0xdeadbeefu); + std::vector userData(8, 0u); + const auto input = BufferDescriptor(Input.data(), static_cast(Input.size())); + const auto output = BufferDescriptor(Output.data(), static_cast(Output.size())); + std::copy(input.begin(), input.end(), userData.begin()); + std::copy(output.begin(), output.end(), userData.begin() + 4); + const std::span code(Code); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0, {false, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.context.floatMode = ShaderRecompiler::ShaderFloatMode{0xf0u | (rounding << 2u) | rounding, true, true, false}; + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); + for (std::uint32_t lane = 0; lane < Threads; ++lane) { + for (std::uint32_t column = 0; column < 2u; ++column) { + const auto expected = Expected[rounding - 1u][lane][column]; + const auto actual = Output[lane * Results + column]; + Require(actual == expected, "integer to f32 rounding: mode " + std::to_string(rounding) + " lane " + std::to_string(lane) + " column " + std::to_string(column) + " is " + Hex(actual) + ", expected " + Hex(expected)); + } + } +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + for (std::uint32_t rounding = 1u; rounding < 4u; ++rounding) { + Run(*device, rounding); + } + std::puts("integer to f32 rounding tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp b/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp index e6340cfdb..cf724b0ba 100644 --- a/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp +++ b/core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp @@ -145,6 +145,10 @@ private: IrU32 convertF32ToU32Saturated(IrF32 value, float upperBound, float safeUpper, std::uint32_t highResult); IrU32 convertF32ToI32Saturated(IrF32 value, float lowerBound, float upperBound, float safeUpper, std::uint32_t lowerResult, std::uint32_t upperResult); IrU32 convertFlooredF32ToI32(IrF32 source, IrF32 floored); + bool directedF32Rounding() const; + IrF32 convertMagnitudeToF32Directed(IrU32 magnitude, IrU32 signBits, std::uint32_t rounding, IrF32 native); + IrF32 convertU32ToF32(IrU32 source); + IrF32 convertS32ToF32(IrU32 source); IrU32 packU16Lanes(IrU32 low, IrU32 high); void emitCompareResult(const RdnaInstruction& inst, IrU1 value, bool scalar, bool cmpx); void emitCompareConstant(const RdnaInstruction& inst, bool value, bool scalar, bool cmpx); diff --git a/core/shader/recompiler/Translation/src/ConvertInstructions.cpp b/core/shader/recompiler/Translation/src/ConvertInstructions.cpp index 91241ac31..b452dcc6d 100644 --- a/core/shader/recompiler/Translation/src/ConvertInstructions.cpp +++ b/core/shader/recompiler/Translation/src/ConvertInstructions.cpp @@ -53,15 +53,56 @@ void TranslationContext::vCvtF32Ubyte(const RdnaInstruction& inst, std::uint32_t writeOperand(inst.destination, &result.Value()); } +bool TranslationContext::directedF32Rounding() const { + return floatMode.has_value() && (floatMode->floatMode & 3u) != 0u; +} + +IrF32 TranslationContext::convertMagnitudeToF32Directed(IrU32 magnitude, IrU32 signBits, std::uint32_t rounding, IrF32 native) { + const IrU32 msb(ir.Emit(IrOpcode::FindUMsb32, IrType::U32, {&magnitude.Value()})); + const IrU1 inexact(ir.UGreaterThan(msb.Value(), ir.Constant(23u))); + const IrU32 shift(ir.ISub(msb.Value(), ir.Constant(23u))); + const IrU32 top24(ir.ShiftRightLogical(magnitude.Value(), shift.Value())); + const IrU32 discarded(ir.BitwiseAnd(magnitude.Value(), ir.ISub(ir.ShiftLeftLogical(ir.Constant(1u), shift.Value()), ir.Constant(1u)))); + const IrU1 positive(ir.IEqual(signBits.Value(), ir.Constant(0u))); + const IrU1 away(rounding == 1u ? positive : rounding == 2u ? IrU1(ir.LogicalNot(positive.Value())) : IrU1(ir.ConstantBool(false))); + const IrU1 increment(ir.LogicalAnd(away.Value(), ir.INotEqual(discarded.Value(), ir.Constant(0u)))); + const IrU32 rounded(ir.IAdd(top24.Value(), ir.Select(increment.Value(), ir.Constant(1u), ir.Constant(0u)))); + const IrU1 overflow(ir.Emit(IrOpcode::UGreaterThanEqual32, IrType::U1, {&rounded.Value(), &ir.Constant(0x1000000u)})); + const IrU32 finalMsb(ir.Select(overflow.Value(), ir.IAdd(msb.Value(), ir.Constant(1u)), msb.Value())); + const IrU32 finalTop(ir.Select(overflow.Value(), ir.Constant(0x800000u), rounded.Value())); + const IrU32 bits(ir.BitwiseOr(signBits.Value(), ir.BitwiseOr(ir.ShiftLeftLogical(ir.IAdd(finalMsb.Value(), ir.Constant(127u)), ir.Constant(23u)), ir.BitwiseAnd(finalTop.Value(), ir.Constant(0x7fffffu))))); + const IrF32 directed(ir.BitCastF32(bits.Value())); + const IrU1 useNative(ir.LogicalOr(ir.IEqual(magnitude.Value(), ir.Constant(0u)), ir.LogicalNot(inexact.Value()))); + return selectF32(useNative, native, directed); +} + +IrF32 TranslationContext::convertU32ToF32(IrU32 source) { + const IrF32 native(ir.Emit(IrOpcode::ConvertF32U32, IrType::F32, {&source.Value()})); + if (!directedF32Rounding()) { + return native; + } + return convertMagnitudeToF32Directed(source, IrU32(ir.Constant(0u)), floatMode->floatMode & 3u, native); +} + +IrF32 TranslationContext::convertS32ToF32(IrU32 source) { + const IrF32 native(ir.Emit(IrOpcode::ConvertF32S32, IrType::F32, {&source.Value()})); + if (!directedF32Rounding()) { + return native; + } + const IrU32 signBits(ir.BitwiseAnd(source.Value(), ir.Constant(0x80000000u))); + const IrU32 magnitude(ir.Emit(IrOpcode::IAbs32, IrType::U32, {&source.Value()})); + return convertMagnitudeToF32Directed(magnitude, signBits, floatMode->floatMode & 3u, native); +} + void TranslationContext::vCvtF32U32(const RdnaInstruction& inst) { const IrU32 source = readU32(sourceAt(inst, 0u)); - const IrF32 result(ir.Emit(IrOpcode::ConvertF32U32, IrType::F32, {&source.Value()})); + const IrF32 result = convertU32ToF32(source); writeOperand(inst.destination, &result.Value()); } void TranslationContext::vCvtF32I32(const RdnaInstruction& inst) { const IrU32 source = readU32(sourceAt(inst, 0u)); - const IrF32 result(ir.Emit(IrOpcode::ConvertF32S32, IrType::F32, {&source.Value()})); + const IrF32 result = convertS32ToF32(source); writeOperand(inst.destination, &result.Value()); } From ce98a61a83dede6d110a8418cb7e8016b080e21a Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Wed, 7 Oct 2026 23:12:55 -0500 Subject: [PATCH 163/460] feat(libkernel): implement pthread_set_name_np pthread_set_name_np renames through scePthreadRename; a null name clears it, as in FreeBSD. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libkernel/Pthread/Posix/Thread.cpp | 4 ++++ core/libs/tests/GuestThreadSelf.cpp | 10 ++++++++++ 2 files changed, 14 insertions(+) diff --git a/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp b/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp index b525f205b..78af2d4fe 100644 --- a/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp +++ b/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp @@ -102,6 +102,10 @@ int APS5_VABI pthread_getname_np_nid_postfix(Pthread thread, char* name) { return PosixThread::ToErrno(scePthreadGetname(thread, name)); } +void APS5_VABI pthread_set_name_np_nid_postfix(Pthread thread, const char* name) { + scePthreadRename(thread, name != nullptr ? name : ""); +} + Pthread APS5_VABI pthread_self_nid_postfix(void) { return scePthreadSelf(); } diff --git a/core/libs/tests/GuestThreadSelf.cpp b/core/libs/tests/GuestThreadSelf.cpp index 16c542ab2..6447d7ee9 100644 --- a/core/libs/tests/GuestThreadSelf.cpp +++ b/core/libs/tests/GuestThreadSelf.cpp @@ -14,6 +14,8 @@ void APS5_VABI scePthreadTestcancel(); void APS5_VABI pthread_testcancel_nid_postfix(void); int APS5_VABI scePthreadSetcancelstate(int state, int* old_state); int APS5_VABI scePthreadSetcanceltype(int type, int* old_type); +void APS5_VABI pthread_set_name_np_nid_postfix(Pthread thread, const char* name); +int APS5_VABI scePthreadGetname(Pthread thread, char* name); int APS5_VABI scePthreadMutexattrInit(PthreadMutexattr* attr); int APS5_VABI scePthreadMutexattrDestroy(PthreadMutexattr* attr); int APS5_VABI scePthreadMutexattrSettype(PthreadMutexattr* attr, int type); @@ -130,4 +132,12 @@ int main() { Require(scePthreadMutexUnlock(&mutex) == SCE_OK); Require(scePthreadMutexDestroy(&mutex) == SCE_OK); + + char name[32] = {'x'}; + pthread_set_name_np_nid_postfix(mainSelf, "q2-main"); + Require(scePthreadGetname(mainSelf, name) == SCE_OK && std::strcmp(name, "q2-main") == 0); + pthread_set_name_np_nid_postfix(mainSelf, nullptr); + Require(scePthreadGetname(mainSelf, name) == SCE_OK && name[0] == '\0'); + pthread_set_name_np_nid_postfix(nullptr, "ignored"); + } From c5c34ad9e0f40f7658b94d8a5bfedece2ad6d370 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 19:08:56 +0000 Subject: [PATCH 164/460] test(shader): use structural dominance in CFG checks (cherry picked from commit c6d9f9cbed1aea7dcc66b494dd24c8ed2920b09a) --- .../prx/libSceAgcDriver/tests/ControlFlow.cpp | 106 ++++++++++++++++-- 1 file changed, 97 insertions(+), 9 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/ControlFlow.cpp b/core/libs/prx/libSceAgcDriver/tests/ControlFlow.cpp index 14e020f13..10e1de641 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ControlFlow.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ControlFlow.cpp @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -43,24 +44,73 @@ void require(bool condition, const std::string& message) { if (!condition) throw std::runtime_error(message); } -std::vector constructBlocks(const ControlFlowGraph& graph, std::uint32_t header, std::uint32_t exclude) { +class StructuralDominators { +public: + explicit StructuralDominators(const ControlFlowGraph& graph) : dominators(graph.blocks.size()) { + std::vector> predecessors(graph.blocks.size()); + for (const auto& block : graph.blocks) { + for (const auto successor : block.successors) predecessors[successor].push_back(block.id); + const auto& terminator = block.terminator; + if (terminator.mergeBlock != InvalidControlFlowId) predecessors[terminator.mergeBlock].push_back(block.id); + if (terminator.loopHeader && terminator.continueBlock != InvalidControlFlowId) predecessors[terminator.continueBlock].push_back(block.id); + } + + std::vector all; + all.reserve(graph.blocks.size()); + for (const auto& block : graph.blocks) all.push_back(block.id); + for (const auto& block : graph.blocks) dominators[block.id] = block.id == graph.entryBlock ? std::vector{block.id} : all; + + bool changed = true; + while (changed) { + changed = false; + for (const auto& block : graph.blocks) { + if (block.id == graph.entryBlock) continue; + auto next = predecessors[block.id].empty() ? std::vector{} : dominators[predecessors[block.id].front()]; + for (std::size_t i = 1; i < predecessors[block.id].size(); ++i) { + std::vector intersection; + const auto& other = dominators[predecessors[block.id][i]]; + std::set_intersection(next.begin(), next.end(), other.begin(), other.end(), std::back_inserter(intersection)); + next = std::move(intersection); + } + if (!std::binary_search(next.begin(), next.end(), block.id)) { + next.insert(std::lower_bound(next.begin(), next.end(), block.id), block.id); + } + if (next != dominators[block.id]) { + dominators[block.id] = std::move(next); + changed = true; + } + } + } + } + + bool Dominates(std::uint32_t dominator, std::uint32_t block) const { + const auto& values = dominators.at(block); + return std::binary_search(values.begin(), values.end(), dominator); + } + +private: + std::vector> dominators; +}; + +std::vector constructBlocks(const ControlFlowGraph& graph, const StructuralDominators& dominators, std::uint32_t header, std::uint32_t exclude) { std::vector blocks; for (const auto& block : graph.blocks) { - if (graph.Dominates(header, block.id) && (exclude == InvalidControlFlowId || !graph.Dominates(exclude, block.id))) { + if (dominators.Dominates(header, block.id) && (exclude == InvalidControlFlowId || !dominators.Dominates(exclude, block.id))) { blocks.push_back(block.id); } } return blocks; } -bool inEnclosingContinueConstruct(const ControlFlowGraph& graph, std::uint32_t header, std::uint32_t member) { +bool inEnclosingContinueConstruct(const ControlFlowGraph& graph, const StructuralDominators& dominators, std::uint32_t header, std::uint32_t member) { return std::any_of(graph.blocks.begin(), graph.blocks.end(), [&](const BasicBlock& loop) { const auto continueBlock = loop.terminator.continueBlock; - return loop.terminator.loopHeader && graph.Dominates(loop.id, header) && !graph.Dominates(continueBlock, header) && graph.Dominates(continueBlock, member); + return loop.terminator.loopHeader && dominators.Dominates(loop.id, header) && !dominators.Dominates(continueBlock, header) && dominators.Dominates(continueBlock, member); }); } void verifyStructured(const std::string& prefix, const ControlFlowGraph& graph) { + const StructuralDominators dominators(graph); std::map mergeOwners; std::vector> loopExits; for (const auto& block : graph.blocks) { @@ -72,10 +122,10 @@ void verifyStructured(const std::string& prefix, const ControlFlowGraph& graph) for (const auto& block : graph.blocks) { const auto& terminator = block.terminator; if (terminator.loopHeader || terminator.mergeBlock == InvalidControlFlowId) continue; - for (const auto member : constructBlocks(graph, block.id, terminator.mergeBlock)) { - if (inEnclosingContinueConstruct(graph, block.id, member)) continue; + for (const auto member : constructBlocks(graph, dominators, block.id, terminator.mergeBlock)) { + if (inEnclosingContinueConstruct(graph, dominators, block.id, member)) continue; for (const auto successor : graph.FindBlock(member).successors) { - if (successor == terminator.mergeBlock || (graph.Dominates(block.id, successor) && !graph.Dominates(terminator.mergeBlock, successor))) continue; + if (successor == terminator.mergeBlock || (dominators.Dominates(block.id, successor) && !dominators.Dominates(terminator.mergeBlock, successor))) continue; const bool loopExit = std::any_of(loopExits.begin(), loopExits.end(), [&](const auto& exits) { return successor == exits.first || successor == exits.second; }); require(loopExit, prefix + "block " + std::to_string(member) + " of the selection at block " + std::to_string(block.id) + " branches to block " + std::to_string(successor) + " outside the construct"); } @@ -84,14 +134,46 @@ void verifyStructured(const std::string& prefix, const ControlFlowGraph& graph) for (const auto& loop : graph.blocks) { if (!loop.terminator.loopHeader) continue; const auto continueBlock = loop.terminator.continueBlock; - for (const auto member : constructBlocks(graph, loop.id, loop.terminator.mergeBlock)) { + for (const auto member : constructBlocks(graph, dominators, loop.id, loop.terminator.mergeBlock)) { const auto& terminator = graph.FindBlock(member).terminator; - if (terminator.loopHeader || graph.Dominates(continueBlock, member)) continue; + if (terminator.loopHeader || dominators.Dominates(continueBlock, member)) continue; require(terminator.mergeBlock != continueBlock, prefix + "the selection at block " + std::to_string(member) + " in the loop at block " + std::to_string(loop.id) + " merges at the loop's continue block " + std::to_string(continueBlock)); } } } +void verifyLoopMergeDoesNotEnterNestedSelection() { + const std::vector> successors{{1, 4}, {2}, {5, 6}, {4}, {}, {3}, {3, 7}, {1}}; + ControlFlowGraph graph; + graph.entryBlock = 0; + for (std::uint32_t id = 0; id < successors.size(); ++id) { + BasicBlock block; + block.id = id; + block.successors = successors[id]; + if (successors[id].empty()) { + block.terminator.kind = TerminatorKind::Return; + } else if (successors[id].size() == 1) { + block.terminator.kind = TerminatorKind::Branch; + block.terminator.trueBlock = successors[id][0]; + } else { + block.terminator.kind = TerminatorKind::ConditionalBranch; + block.terminator.trueBlock = successors[id][0]; + block.terminator.falseBlock = successors[id][1]; + } + graph.blocks.push_back(std::move(block)); + } + graph.blocks[0].terminator.mergeBlock = 4; + graph.blocks[1].terminator.loopHeader = true; + graph.blocks[1].terminator.mergeBlock = 3; + graph.blocks[1].terminator.continueBlock = 7; + graph.blocks[2].terminator.mergeBlock = 6; + for (const auto& block : graph.blocks) { + for (const auto successor : block.successors) graph.blocks[successor].predecessors.push_back(block.id); + } + + verifyStructured("loop merge structural dominance: ", graph); +} + std::uint32_t followEmptyBlocks(const std::string& prefix, const ControlFlowGraph& graph, std::uint32_t blockId, RouteState& routes) { for (std::size_t steps = 0; steps <= graph.blocks.size(); ++steps) { const auto& block = graph.FindBlock(blockId); @@ -303,6 +385,12 @@ void verifyNullSwappc() { } int main(int argc, char** argv) { + try { + verifyLoopMergeDoesNotEnterNestedSelection(); + } catch (const std::exception& error) { + std::fprintf(stderr, "%s\n", error.what()); + return 1; + } if (argc > 1) { int failures = 0; for (int i = 1; i < argc; ++i) { From 71ced938938aeaf8bda52b3a26c24e5b5ecf004c Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 19:39:47 +0000 Subject: [PATCH 165/460] test(shader): exercise the structurizer dominance regression (cherry picked from commit 03ed5aecb241d0347b38690c49c7c8d0db9ff417) --- .../prx/libSceAgcDriver/tests/ControlFlow.cpp | 31 +++++++++++-------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/ControlFlow.cpp b/core/libs/prx/libSceAgcDriver/tests/ControlFlow.cpp index 10e1de641..d13a89301 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ControlFlow.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ControlFlow.cpp @@ -142,35 +142,40 @@ void verifyStructured(const std::string& prefix, const ControlFlowGraph& graph) } } -void verifyLoopMergeDoesNotEnterNestedSelection() { - const std::vector> successors{{1, 4}, {2}, {5, 6}, {4}, {}, {3}, {3, 7}, {1}}; +ControlFlowGraph makeGraph(const std::vector>& successors) { ControlFlowGraph graph; graph.entryBlock = 0; for (std::uint32_t id = 0; id < successors.size(); ++id) { BasicBlock block; block.id = id; + block.startProgramCounter = id * 8; + block.endProgramCounter = id * 8 + 8; + block.instructionBegin = id * 2; + block.instructionEnd = id * 2 + 2; block.successors = successors[id]; + auto& terminator = block.terminator; if (successors[id].empty()) { - block.terminator.kind = TerminatorKind::Return; + terminator.kind = TerminatorKind::Return; } else if (successors[id].size() == 1) { - block.terminator.kind = TerminatorKind::Branch; - block.terminator.trueBlock = successors[id][0]; + terminator.kind = TerminatorKind::Branch; + terminator.trueBlock = successors[id][0]; } else { - block.terminator.kind = TerminatorKind::ConditionalBranch; - block.terminator.trueBlock = successors[id][0]; - block.terminator.falseBlock = successors[id][1]; + terminator.kind = TerminatorKind::ConditionalBranch; + terminator.condition = BranchCondition::SccNonZero; + terminator.trueBlock = successors[id][0]; + terminator.falseBlock = successors[id][1]; } graph.blocks.push_back(std::move(block)); } - graph.blocks[0].terminator.mergeBlock = 4; - graph.blocks[1].terminator.loopHeader = true; - graph.blocks[1].terminator.mergeBlock = 3; - graph.blocks[1].terminator.continueBlock = 7; - graph.blocks[2].terminator.mergeBlock = 6; for (const auto& block : graph.blocks) { for (const auto successor : block.successors) graph.blocks[successor].predecessors.push_back(block.id); } + return graph; +} +void verifyLoopMergeDoesNotEnterNestedSelection() { + auto graph = makeGraph({{1, 5}, {2, 3}, {5}, {5, 4}, {1}, {}}); + Structurizer{}.Structurize(graph); verifyStructured("loop merge structural dominance: ", graph); } From b69fba6a15c49ae0efd3e4bea672f152877d57e6 Mon Sep 17 00:00:00 2001 From: David Date: Sun, 11 Oct 2026 01:03:30 +0400 Subject: [PATCH 166/460] fix(libkernel): remove read-only directories with rmdir on Windows (cherry picked from commit da6889627a417df433802a45a8fcca58a6cb904f) --- core/libs/prx/libkernel/File/src/Stdio.cpp | 15 ++++++++++++++- core/libs/tests/GuestFilesystem.cpp | 10 ++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index e51329185..69b765f7a 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -75,6 +75,15 @@ static constexpr std::uint64_t KERNEL_MNT_NOSUID = 0x8; static int NativeRmdir(const std::filesystem::path& path) { return ::_wrmdir(path.wstring().c_str()); } +static bool RemoveEmptyDirectory(const std::filesystem::path& path) { + struct _stat64 status{}; + const bool unlocked = ::_wstat64(path.c_str(), &status) == 0 && !(status.st_mode & _S_IWRITE) && + ::_wchmod(path.c_str(), _S_IREAD | _S_IWRITE) == 0; + std::error_code error; + if (std::filesystem::remove(path, error)) return true; + if (unlocked) ::_wchmod(path.c_str(), _S_IREAD); + return false; +} static int NativeMkdir(const std::filesystem::path& path, std::uint16_t mode) { (void)mode; return ::_wmkdir(path.wstring().c_str()); @@ -205,6 +214,10 @@ static std::int64_t NativePwrite(int descriptor, const void* buf, std::size_t nb static int NativeRmdir(const std::filesystem::path& path) { return ::rmdir(path.c_str()); } +static bool RemoveEmptyDirectory(const std::filesystem::path& path) { + std::error_code error; + return std::filesystem::remove(path, error); +} static int NativeMkdir(const std::filesystem::path& path, std::uint16_t mode) { return ::mkdir(path.c_str(), static_cast(mode)); } @@ -904,7 +917,7 @@ int APS5_VABI sceKernelRmdir(const char* path) { std::error_code error; if (!std::filesystem::is_directory(native, error)) return SceErrorFromErrno(std::filesystem::exists(native, error) ? GUEST_ENOTDIR : GUEST_ENOENT); if (!std::filesystem::is_empty(native, error)) return SceErrorFromErrno(GUEST_ENOTEMPTY); - if (!std::filesystem::remove(native, error)) return SceErrorFromErrno(GUEST_EIO); + if (!RemoveEmptyDirectory(native)) return SceErrorFromErrno(GUEST_EIO); RecordWrittenPath_nid_no_patch(native); return 0; } diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index c42534124..4b8cc4c15 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -360,6 +360,16 @@ int main() { Require(unlink_nid_postfix(lockedPosix.string().c_str()) == 0 && !std::filesystem::exists(lockedPosix)); const auto lockedKernel = lockedFile("locked-kernel"); Require(sceKernelUnlink(lockedKernel.string().c_str()) == 0 && !std::filesystem::exists(lockedKernel)); + const auto lockedDirectory = [&](const char* name) { + const auto locked = root / name; + Require(mkdir_nid_postfix(locked.string().c_str(), 0700) == 0); + Require(sceKernelChmod_nid_postfix(locked.string().c_str(), 0555) == 0); + return locked; + }; + const auto lockedDirectoryPosix = lockedDirectory("locked-directory-posix"); + Require(rmdir_nid_postfix(lockedDirectoryPosix.string().c_str()) == 0 && !std::filesystem::exists(lockedDirectoryPosix)); + const auto lockedDirectoryKernel = lockedDirectory("locked-directory-kernel"); + Require(sceKernelRmdir(lockedDirectoryKernel.string().c_str()) == 0 && !std::filesystem::exists(lockedDirectoryKernel)); Require(rmdir_nid_postfix("") == -1 && *__error_nid_postfix() == 2); Require(rmdir_nid_postfix(nullptr) == -1 && *__error_nid_postfix() == 14); Require(unlink_nid_postfix(missingName.c_str()) == -1 && *__error_nid_postfix() == 2); From 63cce467e79328703488d04440d18f03db9e6e91 Mon Sep 17 00:00:00 2001 From: David Date: Sat, 10 Oct 2026 23:37:00 +0200 Subject: [PATCH 167/460] fix(libc): stop strtol at a 0x prefix without hex digits on Windows Co-Authored-By: Claude Fable 5.1 --- core/libs/prx/libc/include/General.hpp | 2 +- core/libs/prx/libc/src/RuntimeSupport.cpp | 6 ++--- core/libs/prx/libc/src/Strings.cpp | 29 +++++++++++++---------- core/libs/tests/GuestMath.cpp | 5 ++++ 4 files changed, 26 insertions(+), 16 deletions(-) diff --git a/core/libs/prx/libc/include/General.hpp b/core/libs/prx/libc/include/General.hpp index 7dc00888d..20139efa3 100644 --- a/core/libs/prx/libc/include/General.hpp +++ b/core/libs/prx/libc/include/General.hpp @@ -11,7 +11,7 @@ extern "C" void NotImplemented_nid_no_patch(const char* funcName); extern "C" void CxaFinalize_nid_no_patch(void* dsoHandle); -extern "C" bool StopAtBinaryPrefix_nid_no_patch(const char* str, char** endptr, int base); +extern "C" bool StopAtPrefixLetter_nid_no_patch(const char* str, char** endptr, int base); extern "C" std::filesystem::path ResolvePath_nid_no_patch(const char* path); // Guest path aliases: a guest prefix such as "/savedata0" (a save-data mount point) resolves to a host diff --git a/core/libs/prx/libc/src/RuntimeSupport.cpp b/core/libs/prx/libc/src/RuntimeSupport.cpp index 4943567c8..532938b9b 100644 --- a/core/libs/prx/libc/src/RuntimeSupport.cpp +++ b/core/libs/prx/libc/src/RuntimeSupport.cpp @@ -95,12 +95,12 @@ unsigned int APS5_VABI _Atomic_load_4_nid_postfix(volatile unsigned int* target, } std::intmax_t APS5_VABI strtoimax_nid_postfix(const char* str, char** endptr, int base) { - if (StopAtBinaryPrefix_nid_no_patch(str, endptr, base)) return 0; + if (StopAtPrefixLetter_nid_no_patch(str, endptr, base)) return 0; return std::strtoimax(str, endptr, base); } std::uintmax_t APS5_VABI strtoumax_nid_postfix(const char* str, char** endptr, int base) { - if (StopAtBinaryPrefix_nid_no_patch(str, endptr, base)) return 0; + if (StopAtPrefixLetter_nid_no_patch(str, endptr, base)) return 0; return std::strtoumax(str, endptr, base); } @@ -134,7 +134,7 @@ void APS5_VABI _Unlockfilelock_nid_postfix(FileStream* stream) { } std::uint64_t APS5_VABI _Stoul_nid_postfix(const char* str, char** endptr, int base) { - if (StopAtBinaryPrefix_nid_no_patch(str, endptr, base)) return 0; + if (StopAtPrefixLetter_nid_no_patch(str, endptr, base)) return 0; return std::strtoull(str, endptr, base); } diff --git a/core/libs/prx/libc/src/Strings.cpp b/core/libs/prx/libc/src/Strings.cpp index f60b7ea24..6acfd452e 100644 --- a/core/libs/prx/libc/src/Strings.cpp +++ b/core/libs/prx/libc/src/Strings.cpp @@ -119,32 +119,37 @@ size_t APS5_VABI strlcpy_nid_postfix(char* dest, const char* src, size_t size) { return srcLen; } -bool StopAtBinaryPrefix_nid_no_patch(const char* str, char** endptr, int base) { - if (base != 0 && base != 2) return false; +bool StopAtPrefixLetter_nid_no_patch(const char* str, char** endptr, int base) { + if (base != 0 && base != 2 && base != 16) return false; while (*str == ' ' || (*str >= '\t' && *str <= '\r')) ++str; if (*str == '+' || *str == '-') ++str; - if (str[0] != '0' || (str[1] != 'b' && str[1] != 'B') || (str[2] != '0' && str[2] != '1')) return false; + if (str[0] != '0' || str[1] == '\0') return false; + const char next = str[2]; + const bool binary = base != 16 && (str[1] == 'b' || str[1] == 'B') && (next == '0' || next == '1'); + const bool hexDigit = (next >= '0' && next <= '9') || (next >= 'a' && next <= 'f') || (next >= 'A' && next <= 'F'); + const bool bareHex = base != 2 && (str[1] == 'x' || str[1] == 'X') && !hexDigit; + if (!binary && !bareHex) return false; if (endptr) *endptr = const_cast(str + 1); return true; } std::int64_t APS5_VABI strtol_nid_postfix(const char* str, char** endptr, int base) { - if (StopAtBinaryPrefix_nid_no_patch(str, endptr, base)) return 0; + if (StopAtPrefixLetter_nid_no_patch(str, endptr, base)) return 0; return std::strtoll(str, endptr, base); } std::uint64_t APS5_VABI strtoul_nid_postfix(const char* str, char** endptr, int base) { - if (StopAtBinaryPrefix_nid_no_patch(str, endptr, base)) return 0; + if (StopAtPrefixLetter_nid_no_patch(str, endptr, base)) return 0; return std::strtoull(str, endptr, base); } long long APS5_VABI strtoll_nid_postfix(const char* str, char** endptr, int base) { - if (StopAtBinaryPrefix_nid_no_patch(str, endptr, base)) return 0; + if (StopAtPrefixLetter_nid_no_patch(str, endptr, base)) return 0; return std::strtoll(str, endptr, base); } unsigned long long APS5_VABI strtoull_nid_postfix(const char* str, char** endptr, int base) { - if (StopAtBinaryPrefix_nid_no_patch(str, endptr, base)) return 0; + if (StopAtPrefixLetter_nid_no_patch(str, endptr, base)) return 0; return std::strtoull(str, endptr, base); } @@ -329,7 +334,7 @@ char* APS5_VABI strnstr_nid_postfix(const char* haystack, const char* needle, si } unsigned long long APS5_VABI _Stoull_nid_postfix(const char* str, char** endptr, int base) { - if (StopAtBinaryPrefix_nid_no_patch(str, endptr, base)) return 0; + if (StopAtPrefixLetter_nid_no_patch(str, endptr, base)) return 0; return std::strtoull(str, endptr, base); } @@ -423,19 +428,19 @@ long double APS5_VABI wcstold_nid_postfix(const char16_t* str, char16_t** endptr } long long APS5_VABI wcstol_nid_postfix(const char16_t* str, char16_t** endptr, int base) { - return ParseAsciiPrefix(str, endptr, [base](const char* text, char** end) { return StopAtBinaryPrefix_nid_no_patch(text, end, base) ? 0LL : std::strtoll(text, end, base); }); + return ParseAsciiPrefix(str, endptr, [base](const char* text, char** end) { return StopAtPrefixLetter_nid_no_patch(text, end, base) ? 0LL : std::strtoll(text, end, base); }); } long long APS5_VABI wcstoll_nid_postfix(const char16_t* str, char16_t** endptr, int base) { - return ParseAsciiPrefix(str, endptr, [base](const char* text, char** end) { return StopAtBinaryPrefix_nid_no_patch(text, end, base) ? 0LL : std::strtoll(text, end, base); }); + return ParseAsciiPrefix(str, endptr, [base](const char* text, char** end) { return StopAtPrefixLetter_nid_no_patch(text, end, base) ? 0LL : std::strtoll(text, end, base); }); } unsigned long long APS5_VABI wcstoul_nid_postfix(const char16_t* str, char16_t** endptr, int base) { - return ParseAsciiPrefix(str, endptr, [base](const char* text, char** end) { return StopAtBinaryPrefix_nid_no_patch(text, end, base) ? 0ULL : std::strtoull(text, end, base); }); + return ParseAsciiPrefix(str, endptr, [base](const char* text, char** end) { return StopAtPrefixLetter_nid_no_patch(text, end, base) ? 0ULL : std::strtoull(text, end, base); }); } unsigned long long APS5_VABI wcstoull_nid_postfix(const char16_t* str, char16_t** endptr, int base) { - return ParseAsciiPrefix(str, endptr, [base](const char* text, char** end) { return StopAtBinaryPrefix_nid_no_patch(text, end, base) ? 0ULL : std::strtoull(text, end, base); }); + return ParseAsciiPrefix(str, endptr, [base](const char* text, char** end) { return StopAtPrefixLetter_nid_no_patch(text, end, base) ? 0ULL : std::strtoull(text, end, base); }); } int APS5_VABI wcscoll_nid_postfix(const char16_t* first, const char16_t* second) { diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index a31ea085f..939b1b136 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -96,6 +96,9 @@ static void CheckIntegerConversions() { {" -0B1!", 0, 0, 3, 0}, {"0b2", 0, 0, 1, 0}, {"0b101", 16, 0xb101, 5, 0}, + {" -0xz!", 0, 0, 3, 0}, + {"0xg", 16, 0, 1, 0}, + {"0x", 16, 0, 1, 0}, {"z!", 36, 35, 1, 0}, {"", 10, 0, 0, 0}, {" \t+!", 10, 0, 0, 0}, @@ -141,6 +144,8 @@ static void CheckIntegerConversions() { {"0B11", 2, 0, 1, 0}, {" +0b1!", 2, 0, 3, 0}, {"0b101", 16, 0xb101, 5, 0}, + {" +0Xg!", 0, 0, 3, 0}, + {"0x!", 16, 0, 1, 0}, {"z!", 36, 35, 1, 0}, {"", 10, 0, 0, 0}, {" \t-!", 10, 0, 0, 0}, From 51fca3cec3f3bf4018b46a2e2599666c770c01bb Mon Sep 17 00:00:00 2001 From: David Date: Sat, 10 Oct 2026 23:37:33 +0200 Subject: [PATCH 168/460] feat(libc): implement _WStoul, _Iswctype and swscanf_s Co-Authored-By: Claude Fable 5.1 --- core/libs/prx/libc/CMakeLists.txt | 10 + core/libs/prx/libc/Export.cpp | 15 - core/libs/prx/libc/src/RuneLocale.cpp | 13 + core/libs/prx/libc/src/ScanningWide.cpp | 408 ++++++++++++++++++++++++ core/libs/prx/libc/src/Strings.cpp | 5 + core/libs/tests/GuestWideConversion.cpp | 163 ++++++++++ docs/dev/TechnicalDebt.md | 1 + 7 files changed, 600 insertions(+), 15 deletions(-) create mode 100644 core/libs/prx/libc/src/ScanningWide.cpp create mode 100644 core/libs/tests/GuestWideConversion.cpp diff --git a/core/libs/prx/libc/CMakeLists.txt b/core/libs/prx/libc/CMakeLists.txt index 596c12d2b..c4796335d 100644 --- a/core/libs/prx/libc/CMakeLists.txt +++ b/core/libs/prx/libc/CMakeLists.txt @@ -28,6 +28,7 @@ target_sources(libc PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src/JumpBuffer.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/Formatting.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/FormattingWide.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/src/ScanningWide.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/RuntimeSupport.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/LocaleSupport.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/Process.cpp @@ -117,6 +118,15 @@ if(BUILD_TESTING) set_tests_properties(guest_wide_strings PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_wide_conversion_tests ${LIBS_INCLUDE_DIR}/tests/GuestWideConversion.cpp) + target_include_directories(guest_wide_conversion_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_wide_conversion_tests PRIVATE libc) + configure_windows_unwind(guest_wide_conversion_tests) + add_test(NAME guest_wide_conversion COMMAND guest_wide_conversion_tests) + if(WIN32) + set_tests_properties(guest_wide_conversion PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_character_tests ${LIBS_INCLUDE_DIR}/tests/GuestCharacter.cpp) target_include_directories(guest_character_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_character_tests PRIVATE libc) diff --git a/core/libs/prx/libc/Export.cpp b/core/libs/prx/libc/Export.cpp index fb67f24bb..df1244784 100644 --- a/core/libs/prx/libc/Export.cpp +++ b/core/libs/prx/libc/Export.cpp @@ -56,22 +56,12 @@ std::uint64_t APS5_VABI _Cnd_destroy_nid_postfix() { return 0; } -std::uint64_t APS5_VABI _Iswctype_nid_postfix() { - NotImplemented_nid_no_patch(__func__); - return 0; -} - APS5_EXPORT("H+8UBOwfScI", libcCyberUnknown08); std::uint64_t APS5_VABI libcCyberUnknown08(void) { NotImplemented_nid_no_patch("H+8UBOwfScI"); return 0; } -std::uint64_t APS5_VABI _WStoul_nid_postfix() { - NotImplemented_nid_no_patch(__func__); - return 0; -} - std::uint64_t APS5_VABI _Cnd_init_nid_postfix() { NotImplemented_nid_no_patch(__func__); return 0; @@ -101,9 +91,4 @@ std::uint64_t APS5_VABI _Cnd_wait_nid_postfix() { NotImplemented_nid_no_patch(__func__); return 0; } - -int APS5_VABI swscanf_s_nid_postfix(void) { - NotImplemented_nid_no_patch(__func__); - return 0; -} } diff --git a/core/libs/prx/libc/src/RuneLocale.cpp b/core/libs/prx/libc/src/RuneLocale.cpp index cba8bd792..787200a28 100644 --- a/core/libs/prx/libc/src/RuneLocale.cpp +++ b/core/libs/prx/libc/src/RuneLocale.cpp @@ -2,6 +2,9 @@ #include #include #include +#include +#include +#include namespace GuestRune { @@ -107,6 +110,16 @@ int APS5_VABI iswupper_nid_postfix(std::uint32_t c) { return GuestRune::Is(c, Gu int APS5_VABI iswxdigit_nid_postfix(std::uint32_t c) { return GuestRune::Is(c, GuestRune::Hex); } int APS5_VABI iswctype_nid_postfix(std::uint32_t c, std::uint64_t mask) { return GuestRune::Is(c, mask); } +int APS5_VABI _Iswctype_nid_postfix(std::uint32_t c, std::uint64_t type) { + static constexpr std::uint64_t masks[] = { + GuestRune::Alpha | GuestRune::Number, GuestRune::Alpha, GuestRune::Control, GuestRune::Digit, + GuestRune::Graph, GuestRune::Lower, GuestRune::Print, GuestRune::Punct, + GuestRune::Space, GuestRune::Upper, GuestRune::Hex, GuestRune::Blank, + }; + if (type == 0 || type > std::size(masks)) throw std::invalid_argument("_Iswctype: unknown class " + std::to_string(type)); + return GuestRune::Is(c, masks[type - 1]); +} + std::uint64_t APS5_VABI wctype_nid_postfix(const char* property) { static constexpr struct { const char* name; diff --git a/core/libs/prx/libc/src/ScanningWide.cpp b/core/libs/prx/libc/src/ScanningWide.cpp new file mode 100644 index 000000000..04917fdeb --- /dev/null +++ b/core/libs/prx/libc/src/ScanningWide.cpp @@ -0,0 +1,408 @@ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "SceTypes.hpp" +#include "prx/libc/include/General.hpp" +#include "prx/libc/include/WindowsFormatting.hpp" + +extern "C" int APS5_VABI iswspace_nid_postfix(std::uint32_t c); +extern "C" std::size_t APS5_VABI wcrtomb_nid_postfix(char* destination, std::uint16_t value, void* state); + +namespace { + +enum class Fit { Invalid, Prefix, Valid }; + +enum class Outcome { Done, Matching, Input, Violation }; + +struct Directive { + bool suppress = false; + std::size_t width = 0; + std::string length; + char16_t conversion = 0; + const char16_t* setBegin = nullptr; + const char16_t* setEnd = nullptr; + bool setExcluded = false; +}; + +bool IsSpace(char16_t character) { + return iswspace_nid_postfix(character) != 0; +} + +int DigitValue(char character) { + if (character >= '0' && character <= '9') return character - '0'; + if (character >= 'a' && character <= 'z') return character - 'a' + 10; + if (character >= 'A' && character <= 'Z') return character - 'A' + 10; + return 36; +} + +bool IsDecimal(char character) { + return character >= '0' && character <= '9'; +} + +bool IsHex(char character) { + return DigitValue(character) < 16; +} + +char Lower(char character) { + return character >= 'A' && character <= 'Z' ? static_cast(character - 'A' + 'a') : character; +} + +bool StartsWord(std::string_view text, std::string_view word) { + if (text.size() > word.size()) return false; + for (std::size_t index = 0; index < text.size(); ++index) { + if (Lower(text[index]) != word[index]) return false; + } + return true; +} + +std::string_view Unsigned(std::string_view text) { + if (!text.empty() && (text[0] == '+' || text[0] == '-')) text.remove_prefix(1); + return text; +} + +Fit IntegerFit(std::string_view text, int base) { + text = Unsigned(text); + if (text.empty()) return Fit::Prefix; + int radix = base; + std::size_t index = 0; + if ((base == 0 || base == 16) && text[0] == '0' && text.size() >= 2 && (text[1] == 'x' || text[1] == 'X')) { + radix = 16; + index = 2; + if (index == text.size()) return Fit::Prefix; + } else if (base == 0) { + radix = text[0] == '0' ? 8 : 10; + } + for (; index < text.size(); ++index) { + if (DigitValue(text[index]) >= radix) return Fit::Invalid; + } + return Fit::Valid; +} + +Fit SpecialFloatFit(std::string_view text) { + if (StartsWord(text, "infinity")) return text.size() == 3 || text.size() == 8 ? Fit::Valid : Fit::Prefix; + if (StartsWord(text, "nan")) return text.size() == 3 ? Fit::Valid : Fit::Prefix; + if (text.size() < 4 || !StartsWord(text.substr(0, 3), "nan") || text[3] != '(') return Fit::Invalid; + for (std::size_t index = 4; index < text.size(); ++index) { + const char character = text[index]; + if (character == ')') return index + 1 == text.size() ? Fit::Valid : Fit::Invalid; + if (DigitValue(character) == 36 && character != '_') return Fit::Invalid; + } + return Fit::Prefix; +} + +Fit FloatFit(std::string_view text) { + text = Unsigned(text); + if (text.empty()) return Fit::Prefix; + if (Lower(text[0]) == 'i' || Lower(text[0]) == 'n') return SpecialFloatFit(text); + const bool hex = text.size() >= 2 && text[0] == '0' && (text[1] == 'x' || text[1] == 'X'); + const auto digit = [hex](char character) { return hex ? IsHex(character) : IsDecimal(character); }; + std::size_t index = hex ? 2 : 0; + std::size_t digits = 0; + for (; index < text.size() && digit(text[index]); ++index) ++digits; + if (index < text.size() && text[index] == '.') { + for (++index; index < text.size() && digit(text[index]); ++index) ++digits; + } + if (index == text.size()) return digits != 0 ? Fit::Valid : Fit::Prefix; + if (digits == 0) return Fit::Invalid; + if (Lower(text[index]) != (hex ? 'p' : 'e')) return Fit::Invalid; + ++index; + if (index < text.size() && (text[index] == '+' || text[index] == '-')) ++index; + if (index == text.size()) return Fit::Prefix; + for (; index < text.size(); ++index) { + if (!IsDecimal(text[index])) return Fit::Invalid; + } + return Fit::Valid; +} + +std::size_t IntegerSize(const std::string& length) { + if (length.empty()) return sizeof(std::int32_t); + if (length == "hh") return sizeof(std::int8_t); + if (length == "h") return sizeof(std::int16_t); + if (length == "l" || length == "ll" || length == "j" || length == "z" || length == "t") return sizeof(std::int64_t); + throw std::invalid_argument("swscanf_s: invalid integer length modifier " + length); +} + +class WideScanner { + const char16_t* const start; + const char16_t* input; + const char16_t* format; + LibcDetail::FormatArguments args; + int assigned = 0; + bool converted = false; + +public: + WideScanner(const char16_t* buffer, const char16_t* formatText, const void* source) + : start(buffer), input(buffer), format(formatText), args(source) {} + + int Run() { + while (*format != 0) { + Outcome outcome = Outcome::Done; + if (IsSpace(*format)) { + while (IsSpace(*format)) ++format; + SkipSpace(); + } else if (*format != u'%') { + outcome = Literal(*format++); + } else { + ++format; + outcome = Conversion(); + } + if (outcome == Outcome::Matching) return assigned; + if (outcome == Outcome::Input) return converted ? assigned : EOF; + if (outcome == Outcome::Violation) return EOF; + } + return assigned; + } + +private: + void SkipSpace() { + while (*input != 0 && IsSpace(*input)) ++input; + } + + Outcome Literal(char16_t expected) { + if (*input == 0) return Outcome::Input; + if (*input != expected) return Outcome::Matching; + ++input; + return Outcome::Done; + } + + Directive Parse() { + Directive directive; + if (*format == u'*') { + directive.suppress = true; + ++format; + } + const char16_t* digits = format; + for (; *format >= u'0' && *format <= u'9'; ++format) { + const std::size_t digit = static_cast(*format - u'0'); + const std::size_t limit = (std::numeric_limits::max() - digit) / 10; + directive.width = directive.width > limit ? std::numeric_limits::max() : directive.width * 10 + digit; + } + if (format != digits && directive.width == 0) throw std::invalid_argument("swscanf_s: zero field width"); + if (*format == u'h' || *format == u'l') { + directive.length += static_cast(*format++); + if (*format == directive.length[0]) directive.length += static_cast(*format++); + } else if (*format == u'j' || *format == u'z' || *format == u't' || *format == u'L') { + directive.length += static_cast(*format++); + } + directive.conversion = *format; + if (directive.conversion == 0) throw std::invalid_argument("swscanf_s: incomplete conversion specification"); + ++format; + if (directive.conversion == u'[') { + directive.setExcluded = *format == u'^'; + if (directive.setExcluded) ++format; + directive.setBegin = format; + if (*format == u']') ++format; + while (*format != 0 && *format != u']') ++format; + if (*format == 0) throw std::invalid_argument("swscanf_s: unterminated scanset"); + directive.setEnd = format++; + for (const char16_t* member = directive.setBegin + 1; member + 1 < directive.setEnd; ++member) { + if (*member == u'-') throw std::invalid_argument("swscanf_s: scanset range is implementation-defined"); + } + } + return directive; + } + + Outcome Conversion() { + if (*format == u'%') { + ++format; + SkipSpace(); + return Literal(u'%'); + } + const Directive directive = Parse(); + switch (directive.conversion) { + case u'd': return Integer(directive, 10, true); + case u'i': return Integer(directive, 0, true); + case u'o': return Integer(directive, 8, false); + case u'u': return Integer(directive, 10, false); + case u'x': + case u'X': return Integer(directive, 16, false); + case u'a': case u'A': case u'e': case u'E': + case u'f': case u'F': case u'g': case u'G': return Float(directive); + case u's': + case u'c': + case u'[': return Text(directive); + case u'n': return Count(directive); + default: throw std::invalid_argument("swscanf_s: unsupported conversion " + std::to_string(directive.conversion)); + } + } + + template + std::string Field(std::size_t width, TFit fit) const { + std::string field; + const std::size_t limit = width == 0 ? std::numeric_limits::max() : width; + while (field.size() < limit) { + const char16_t next = input[field.size()]; + if (next == 0 || next >= 0x80) break; + field.push_back(static_cast(next)); + if (fit(field) == Fit::Invalid) { + field.pop_back(); + break; + } + } + return field; + } + + Outcome Integer(const Directive& directive, int base, bool isSigned) { + const std::size_t size = IntegerSize(directive.length); + void* target = nullptr; + if (!directive.suppress && (target = args.Next()) == nullptr) return Outcome::Violation; + SkipSpace(); + if (*input == 0) return Outcome::Input; + const std::string field = Field(directive.width, [base](std::string_view text) { return IntegerFit(text, base); }); + if (IntegerFit(field, base) != Fit::Valid) return Outcome::Matching; + input += field.size(); + if (target != nullptr) { + const std::uint64_t value = isSigned ? static_cast(std::strtoll(field.c_str(), nullptr, base)) : std::strtoull(field.c_str(), nullptr, base); + std::memcpy(target, &value, size); + ++assigned; + } + converted = true; + return Outcome::Done; + } + + Outcome Float(const Directive& directive) { + static_assert(sizeof(long double) == 16 && std::numeric_limits::digits == 64); + if (!directive.length.empty() && directive.length != "l" && directive.length != "L") + throw std::invalid_argument("swscanf_s: invalid floating length modifier " + directive.length); + void* target = nullptr; + if (!directive.suppress && (target = args.Next()) == nullptr) return Outcome::Violation; + SkipSpace(); + if (*input == 0) return Outcome::Input; + const std::string field = Field(directive.width, FloatFit); + if (FloatFit(field) != Fit::Valid) return Outcome::Matching; + input += field.size(); + if (target != nullptr) { + if (directive.length == "L") { + const long double value = std::strtold(field.c_str(), nullptr); + std::memcpy(target, &value, sizeof(value)); + } else if (directive.length == "l") { + const double value = std::strtod(field.c_str(), nullptr); + std::memcpy(target, &value, sizeof(value)); + } else { + const float value = std::strtof(field.c_str(), nullptr); + std::memcpy(target, &value, sizeof(value)); + } + ++assigned; + } + converted = true; + return Outcome::Done; + } + + static bool InSet(const Directive& directive, char16_t character) { + bool found = false; + for (const char16_t* member = directive.setBegin; member != directive.setEnd && !found; ++member) found = *member == character; + return found != directive.setExcluded; + } + + static Outcome Truncated(void* target, std::size_t capacity, bool wide) { + if (capacity != 0) { + if (wide) static_cast(target)[0] = 0; + else static_cast(target)[0] = 0; + } + return Outcome::Matching; + } + + Outcome Text(const Directive& directive) { + if (!directive.length.empty() && directive.length != "l") + throw std::invalid_argument("swscanf_s: invalid string length modifier " + directive.length); + void* target = nullptr; + std::size_t capacity = 0; + if (!directive.suppress) { + target = args.Next(); + capacity = args.Next(); + if (target == nullptr) return Outcome::Violation; + } + if (directive.conversion == u's') SkipSpace(); + if (*input == 0) return Outcome::Input; + std::u16string item; + if (directive.conversion == u'c') { + const std::size_t count = directive.width == 0 ? 1 : directive.width; + while (item.size() < count && input[item.size()] != 0) item.push_back(input[item.size()]); + if (item.size() < count) return Outcome::Matching; + } else { + const std::size_t limit = directive.width == 0 ? std::numeric_limits::max() : directive.width; + while (item.size() < limit && input[item.size()] != 0) { + const char16_t next = input[item.size()]; + if (directive.conversion == u's' ? IsSpace(next) : !InSet(directive, next)) break; + item.push_back(next); + } + if (item.empty()) return Outcome::Matching; + } + input += item.size(); + if (target == nullptr) { + converted = true; + return Outcome::Done; + } + const std::size_t terminator = directive.conversion == u'c' ? 0 : 1; + if (directive.length == "l") { + if (item.size() + terminator > capacity) return Truncated(target, capacity, true); + std::memcpy(target, item.data(), item.size() * sizeof(char16_t)); + if (terminator != 0) static_cast(target)[item.size()] = 0; + } else { + std::string bytes; + for (const char16_t unit : item) { + char byte[8]; + const std::size_t written = wcrtomb_nid_postfix(byte, unit, nullptr); + if (written == static_cast(-1)) throw std::invalid_argument("swscanf_s: wide character has no single-byte form"); + bytes.append(byte, written); + } + if (bytes.size() + terminator > capacity) return Truncated(target, capacity, false); + std::memcpy(target, bytes.data(), bytes.size()); + if (terminator != 0) static_cast(target)[bytes.size()] = 0; + } + ++assigned; + converted = true; + return Outcome::Done; + } + + Outcome Count(const Directive& directive) { + if (directive.suppress || directive.width != 0) throw std::invalid_argument("swscanf_s: %n with suppression or width"); + const std::size_t size = IntegerSize(directive.length); + void* target = args.Next(); + if (target == nullptr) return Outcome::Violation; + const std::int64_t count = input - start; + std::memcpy(target, &count, size); + return Outcome::Done; + } +}; + +} + +extern "C" { + +int APS5_VABI swscanf_s_nid_postfix(const char16_t* buffer, const char16_t* format, ...) { + if (buffer == nullptr || format == nullptr) return EOF; +#ifdef _WIN32 + __builtin_sysv_va_list args; + __builtin_sysv_va_start(args, format); +#else + std::va_list args; + va_start(args, format); +#endif + try { + const int result = WideScanner(buffer, format, args).Run(); +#ifdef _WIN32 + __builtin_sysv_va_end(args); +#else + va_end(args); +#endif + return result; + } catch (...) { +#ifdef _WIN32 + __builtin_sysv_va_end(args); +#else + va_end(args); +#endif + throw; + } +} + +} diff --git a/core/libs/prx/libc/src/Strings.cpp b/core/libs/prx/libc/src/Strings.cpp index 6acfd452e..08397f02a 100644 --- a/core/libs/prx/libc/src/Strings.cpp +++ b/core/libs/prx/libc/src/Strings.cpp @@ -443,6 +443,11 @@ unsigned long long APS5_VABI wcstoull_nid_postfix(const char16_t* str, char16_t* return ParseAsciiPrefix(str, endptr, [base](const char* text, char** end) { return StopAtPrefixLetter_nid_no_patch(text, end, base) ? 0ULL : std::strtoull(text, end, base); }); } +std::uint64_t APS5_VABI _WStoul_nid_postfix(const char16_t* str, char16_t** endptr, int base) { + if (base < 0 || base == 1 || base > 36) throw std::invalid_argument("_WStoul: unsupported base " + std::to_string(base)); + return wcstoul_nid_postfix(str, endptr, base); +} + int APS5_VABI wcscoll_nid_postfix(const char16_t* first, const char16_t* second) { return wcscmp_nid_postfix(first, second); } diff --git a/core/libs/tests/GuestWideConversion.cpp b/core/libs/tests/GuestWideConversion.cpp new file mode 100644 index 000000000..fa5b81cac --- /dev/null +++ b/core/libs/tests/GuestWideConversion.cpp @@ -0,0 +1,163 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include +#include +#include + +extern "C" { +std::uint64_t APS5_VABI _WStoul_nid_postfix(const char16_t*, char16_t**, int); +int APS5_VABI _Iswctype_nid_postfix(std::uint32_t, std::uint64_t); +int APS5_VABI swscanf_s_nid_postfix(const char16_t*, const char16_t*, ...); +int APS5_VABI iswalnum_nid_postfix(std::uint32_t); +int APS5_VABI iswalpha_nid_postfix(std::uint32_t); +int APS5_VABI iswcntrl_nid_postfix(std::uint32_t); +int APS5_VABI iswdigit_nid_postfix(std::uint32_t); +int APS5_VABI iswgraph_nid_postfix(std::uint32_t); +int APS5_VABI iswlower_nid_postfix(std::uint32_t); +int APS5_VABI iswprint_nid_postfix(std::uint32_t); +int APS5_VABI iswpunct_nid_postfix(std::uint32_t); +int APS5_VABI iswspace_nid_postfix(std::uint32_t); +int APS5_VABI iswupper_nid_postfix(std::uint32_t); +int APS5_VABI iswxdigit_nid_postfix(std::uint32_t); +int APS5_VABI iswblank_nid_postfix(std::uint32_t); +} + +static int failures = 0; + +static void Require(bool condition, const char* message) { + if (!condition) { + std::fprintf(stderr, "%s\n", message); + ++failures; + } +} + +template static bool Throws(TCall call) { + try { + call(); + } catch (const std::exception&) { + return true; + } + return false; +} + +static void CheckStoul(const char16_t* text, int base, std::uint64_t value, std::size_t consumed, int error, const char* message) { + char16_t* end = nullptr; + errno = 0; + const std::uint64_t result = _WStoul_nid_postfix(text, &end, base); + const int saved = errno; + if (result != value || end != text + consumed || saved != error) { + std::fprintf(stderr, "_WStoul %s: value %llu, consumed %lld, errno %d\n", message, + static_cast(result), static_cast(end - text), saved); + ++failures; + } +} + +static void CheckStoul() { + constexpr std::uint64_t max = std::numeric_limits::max(); + CheckStoul(u" 4294967296", 10, 4294967296ULL, 12, 0, "keeps 64 bits"); + CheckStoul(u"18446744073709551615", 10, max, 20, 0, "maximum"); + CheckStoul(u"18446744073709551616", 10, max, 20, ERANGE, "overflow"); + CheckStoul(u"-1", 10, max, 2, 0, "negation"); + CheckStoul(u"\t+7xyz", 10, 7, 3, 0, "stops at a non-digit"); + CheckStoul(u"0x1F", 16, 31, 4, 0, "hex prefix with base 16"); + CheckStoul(u"0x1f", 0, 31, 4, 0, "hex prefix with base 0"); + CheckStoul(u"017", 0, 15, 3, 0, "octal with base 0"); + CheckStoul(u"z", 36, 35, 1, 0, "base 36"); + CheckStoul(u"0b1", 0, 0, 1, 0, "no binary prefix"); + CheckStoul(u"0xg", 16, 0, 1, 0, "prefix without digits"); + CheckStoul(u"12é3", 10, 12, 2, 0, "stops at a non-ASCII unit"); + CheckStoul(u"　5", 10, 0, 0, 0, "non-ASCII space is not skipped"); + CheckStoul(u"", 10, 0, 0, 0, "empty string"); + Require(_WStoul_nid_postfix(u"9", nullptr, 10) == 9, "_WStoul accepts a null end pointer"); + Require(Throws([] { _WStoul_nid_postfix(u"1", nullptr, 1); }), "_WStoul throws for base 1"); + Require(Throws([] { _WStoul_nid_postfix(u"1", nullptr, 37); }), "_WStoul throws for base 37"); +} + +static void CheckClassification() { + int (APS5_VABI* const classes[])(std::uint32_t) = { + iswalnum_nid_postfix, iswalpha_nid_postfix, iswcntrl_nid_postfix, iswdigit_nid_postfix, + iswgraph_nid_postfix, iswlower_nid_postfix, iswprint_nid_postfix, iswpunct_nid_postfix, + iswspace_nid_postfix, iswupper_nid_postfix, iswxdigit_nid_postfix, iswblank_nid_postfix, + }; + for (std::uint64_t type = 1; type <= 12; ++type) { + for (std::uint32_t c = 0; c < 0x300; ++c) { + if ((_Iswctype_nid_postfix(c, type) != 0) != (classes[type - 1](c) != 0)) { + std::fprintf(stderr, "_Iswctype(%#x, %llu) differs from its isw function\n", c, static_cast(type)); + ++failures; + } + } + Require(_Iswctype_nid_postfix(0xffffffffu, type) == 0, "_Iswctype is false for WEOF"); + } + Require(_Iswctype_nid_postfix(u'A', 2) != 0 && _Iswctype_nid_postfix(u'1', 2) == 0, "_Iswctype alpha"); + Require(_Iswctype_nid_postfix(u'7', 4) != 0 && _Iswctype_nid_postfix(u'f', 11) != 0, "_Iswctype digit and xdigit"); + Require(_Iswctype_nid_postfix(u'\t', 12) != 0 && _Iswctype_nid_postfix(u'\n', 12) == 0, "_Iswctype blank"); + Require(_Iswctype_nid_postfix(u'\n', 9) != 0 && _Iswctype_nid_postfix(u' ', 5) == 0, "_Iswctype space and graph"); + Require(_Iswctype_nid_postfix(u'a', 6) != 0 && _Iswctype_nid_postfix(u'a', 10) == 0, "_Iswctype lower and upper"); + Require(_Iswctype_nid_postfix(0x7f, 3) != 0 && _Iswctype_nid_postfix(u'!', 8) != 0, "_Iswctype cntrl and punct"); + Require(_Iswctype_nid_postfix(0xe9, 2) == 0 && _Iswctype_nid_postfix(0x3042, 1) == 0, "_Iswctype C locale has no non-ASCII letters"); + Require(Throws([] { _Iswctype_nid_postfix(u'a', 0); }), "_Iswctype throws for class 0"); + Require(Throws([] { _Iswctype_nid_postfix(u'a', 13); }), "_Iswctype throws for class 13"); + Require(Throws([] { _Iswctype_nid_postfix(u'a', 0x100); }), "_Iswctype throws for a rune mask"); +} + +static void CheckScan() { + int first = 0; + int second = 0; + Require(swscanf_s_nid_postfix(u" 42 -7", u"%d%d", &first, &second) == 2 && first == 42 && second == -7, "%d"); + Require(swscanf_s_nid_postfix(u"x=0x1f", u"x=%i", &first) == 1 && first == 31, "%i with a hex prefix"); + Require(swscanf_s_nid_postfix(u"1 2", u"%*d%d", &first) == 1 && first == 2, "suppressed %d"); + Require(swscanf_s_nid_postfix(u"12345", u"%3d%d", &first, &second) == 2 && first == 123 && second == 45, "%d width"); + std::int64_t wide = 0; + Require(swscanf_s_nid_postfix(u"5000000000", u"%ld", &wide) == 1 && wide == 5000000000LL, "%ld is 64-bit"); + short half = 0; + Require(swscanf_s_nid_postfix(u"-2", u"%hd", &half) == 1 && half == -2, "%hd"); + unsigned value = 0; + Require(swscanf_s_nid_postfix(u"0xg", u"%x", &value) == 0, "%x prefix without digits is a matching failure"); + + char16_t text[6] = {u'?'}; + Require(swscanf_s_nid_postfix(u"hello world", u"%ls", text, std::size_t{6}) == 1 && std::u16string(text) == u"hello", "%ls fits"); + char16_t small[5] = {u'?', u'?', u'?', u'?', u'?'}; + Require(swscanf_s_nid_postfix(u"hello", u"%ls", small, std::size_t{5}) == 0 && small[0] == 0, "%ls too small"); + Require(swscanf_s_nid_postfix(u"7 hello", u"%d%ls", &first, small, std::size_t{5}) == 1 && first == 7, "%ls too small after a conversion"); + char narrow[4] = {}; + Require(swscanf_s_nid_postfix(u"abc def", u"%s", narrow, std::size_t{4}) == 1 && std::strcmp(narrow, "abc") == 0, "%s"); + char16_t pair[3] = {u'?', u'?', u'?'}; + Require(swscanf_s_nid_postfix(u"xyz", u"%2lc", pair, std::size_t{2}) == 1 && pair[0] == u'x' && pair[1] == u'y' && pair[2] == u'?', "%2lc"); + Require(swscanf_s_nid_postfix(u"xyz", u"%2lc", pair, std::size_t{1}) == 0, "%2lc too small"); + Require(swscanf_s_nid_postfix(u"x", u"%2lc", pair, std::size_t{2}) == 0, "%2lc short input"); + Require(swscanf_s_nid_postfix(u"cab12", u"%l[abc]%d", text, std::size_t{6}, &first) == 2 && std::u16string(text) == u"cab" && first == 12, "%l["); + Require(swscanf_s_nid_postfix(u"été!", u"%l[^!]", text, std::size_t{6}) == 1 && std::u16string(text) == u"été", "%l[^"); + std::int64_t count = -1; + Require(swscanf_s_nid_postfix(u" 12 ab", u"%d %ln", &first, &count) == 1 && count == 4, "%ln"); + + double real = 0; + Require(swscanf_s_nid_postfix(u"1.5e3", u"%lf", &real) == 1 && real == 1500.0, "%lf"); + Require(swscanf_s_nid_postfix(u"1e+x", u"%lf", &real) == 0, "%lf incomplete exponent is a matching failure"); + long double extended = 0; + Require(swscanf_s_nid_postfix(u"2.25", u"%Lf", &extended) == 1 && extended == 2.25L, "%Lf"); + float single = 0; + Require(swscanf_s_nid_postfix(u"-inf", u"%f", &single) == 1 && single == -std::numeric_limits::infinity(), "%f infinity"); + + Require(swscanf_s_nid_postfix(u"a", u"b%d", &first) == 0, "literal mismatch is a matching failure"); + Require(swscanf_s_nid_postfix(u"", u"%d", &first) == -1, "empty input is an input failure"); + Require(swscanf_s_nid_postfix(u" ", u"%ls", text, std::size_t{6}) == -1, "white space only is an input failure"); + Require(swscanf_s_nid_postfix(u"5", u"%d%d", &first, &second) == 1, "input failure after a conversion"); + Require(swscanf_s_nid_postfix(u"5", u"%d", static_cast(nullptr)) == -1, "null %d argument"); + Require(swscanf_s_nid_postfix(u"5 abc", u"%d%ls", &first, static_cast(nullptr), std::size_t{4}) == -1, "null %ls argument"); + Require(swscanf_s_nid_postfix(nullptr, u"%d", &first) == -1, "null buffer"); + Require(swscanf_s_nid_postfix(u"5", nullptr) == -1, "null format"); + Require(Throws([&] { swscanf_s_nid_postfix(u"abc", u"%l[a-c]", text, std::size_t{6}); }), "scanset range throws"); + Require(Throws([&] { swscanf_s_nid_postfix(u"5", u"%Ld", &wide); }), "%Ld throws"); + Require(Throws([&] { swscanf_s_nid_postfix(u"Ā", u"%s", narrow, std::size_t{4}); }), "unconvertible narrow %s throws"); +} + +int main() { + CheckStoul(); + CheckClassification(); + CheckScan(); + return failures == 0 ? 0 : 1; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index f47df2886..aaafab4a2 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -487,3 +487,4 @@ Throughout the project, every function at every stage either **does exactly what - [AJM](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp), [APR](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) and [NGS2](../../core/libs/prx/libSceNgs2.native/src/System.cpp) state is allocated with `new` and never freed on purpose: title threads still call into these libraries while `std::exit` runs static destructors, so ordinary statics would be destroyed under them. Keep them leaked; `guest_ajm_teardown` covers the AJM case on Unix. - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields +- [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert From 92c9296ae398f5b0ea24e9f60580473f7fe411e3 Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Fri, 9 Oct 2026 20:43:55 -0400 Subject: [PATCH 169/460] feat(libc): implement flockfile and funlockfile (cherry picked from commit d32eaf181aadf79efc4199b22591f1f5f228d00c) --- core/libs/prx/libc/src/RuntimeSupport.cpp | 8 ++++++++ core/libs/tests/GuestStandardStreams.cpp | 22 ++++++++++++++++++++++ 2 files changed, 30 insertions(+) diff --git a/core/libs/prx/libc/src/RuntimeSupport.cpp b/core/libs/prx/libc/src/RuntimeSupport.cpp index 4943567c8..28e8ade22 100644 --- a/core/libs/prx/libc/src/RuntimeSupport.cpp +++ b/core/libs/prx/libc/src/RuntimeSupport.cpp @@ -133,6 +133,14 @@ void APS5_VABI _Unlockfilelock_nid_postfix(FileStream* stream) { #endif } +void APS5_VABI flockfile_nid_postfix(FileStream* stream) { + _Lockfilelock_nid_postfix(stream); +} + +void APS5_VABI funlockfile_nid_postfix(FileStream* stream) { + _Unlockfilelock_nid_postfix(stream); +} + std::uint64_t APS5_VABI _Stoul_nid_postfix(const char* str, char** endptr, int base) { if (StopAtBinaryPrefix_nid_no_patch(str, endptr, base)) return 0; return std::strtoull(str, endptr, base); diff --git a/core/libs/tests/GuestStandardStreams.cpp b/core/libs/tests/GuestStandardStreams.cpp index 2351dc61f..66fda23d6 100644 --- a/core/libs/tests/GuestStandardStreams.cpp +++ b/core/libs/tests/GuestStandardStreams.cpp @@ -9,6 +9,8 @@ #include #include #include +#include +#include #ifndef _WIN32 #include #endif @@ -47,6 +49,8 @@ FileStream* APS5_VABI fdopen_nid_postfix(int, const char*); int APS5_VABI fclose_nid_postfix(FileStream*); int APS5_VABI _Getmbcurmax_nid_postfix(); int APS5_VABI ___mb_cur_max_nid_postfix(); +void APS5_VABI flockfile_nid_postfix(FileStream*); +void APS5_VABI funlockfile_nid_postfix(FileStream*); } static void Require(bool value) { if (!value) std::abort(); } static int APS5_VABI WriteFormatted(FileStream* stream, const char* format, ...) { @@ -311,5 +315,23 @@ int main() { Require(freopen_nid_postfix(filename.c_str(), "rb", &failed) == nullptr); Require(*__error_nid_postfix() == 2); Require(failed.GuestState().flags == 0 && failed.GuestState().descriptor == -1); + FileStream locked(std::tmpfile()); + flockfile_nid_postfix(&locked); + flockfile_nid_postfix(&locked); + std::atomic lockedWrite = false; + std::thread writer([&] { + Require(fputc_nid_postfix('B', &locked) == 'B'); + lockedWrite = true; + }); + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + Require(!lockedWrite && fputc_nid_postfix('A', &locked) == 'A'); + funlockfile_nid_postfix(&locked); + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + Require(!lockedWrite); + funlockfile_nid_postfix(&locked); + writer.join(); + Require(lockedWrite && fseeko_nid_postfix(&locked, 0, SEEK_SET) == 0); + Require(fgetc_nid_postfix(&locked) == 'A' && fgetc_nid_postfix(&locked) == 'B'); + locked.Close(); return CheckBinaryModes() ? 0 : 1; } From e251de849d7b9d3bf51ff49053ed9ebd6a4ad7c7 Mon Sep 17 00:00:00 2001 From: pushkarreddyy <212133856+pushkarreddyy@users.noreply.github.com> Date: Sun, 11 Oct 2026 02:14:37 +0530 Subject: [PATCH 170/460] feat(libc): implement vsnprintf_s and vsscanf_s (cherry picked from commit 4ca866ea3b989da6947ad110f4d6c7f4efbe27c8) --- core/libs/prx/libc/Export.cpp | 13 ------------- core/libs/prx/libc/src/Formatting.cpp | 23 +++++++++++++++++++++++ core/libs/tests/Formatting.cpp | 26 ++++++++++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 50 insertions(+), 13 deletions(-) diff --git a/core/libs/prx/libc/Export.cpp b/core/libs/prx/libc/Export.cpp index fb67f24bb..c3820cd78 100644 --- a/core/libs/prx/libc/Export.cpp +++ b/core/libs/prx/libc/Export.cpp @@ -25,19 +25,6 @@ int APS5_VABI _ZSt14_Atomic_assertPKcS0__nid_postfix() { return 0; } -// Live Cyberpunk 2077 import used in an fopen/fseek/ftell-like file-size -// idiom as (handle, 0, 2); returning 0 reports success. -int APS5_VABI vsnprintf_s_nid_postfix() { - NotImplemented_nid_no_patch(__func__); - return 0; -} - -// Live Cyberpunk 2077 import used as (handle, 0, 0) in the same file-size -// idiom; returning 0 reports success. -int APS5_VABI vsscanf_s_nid_postfix() { - NotImplemented_nid_no_patch(__func__); - return 0; -} APS5_EXPORT("Ye20uNnlglA", libcCyberUnknown02); diff --git a/core/libs/prx/libc/src/Formatting.cpp b/core/libs/prx/libc/src/Formatting.cpp index 37a59d7d2..b59bbb2c8 100644 --- a/core/libs/prx/libc/src/Formatting.cpp +++ b/core/libs/prx/libc/src/Formatting.cpp @@ -408,6 +408,15 @@ int APS5_VABI vsnprintf_nid_postfix(char* str, size_t size, const char* format, #ifdef _WIN32 +int APS5_VABI vsscanf_s_nid_postfix(const char* buffer, const char* format, VaList* args) { + LibcDetail::FormatArguments arguments(args); + return ScanGuest(buffer, format, true, [&] { return arguments.Next(); }, [&] { return arguments.Next(); }); +} + +int APS5_VABI vsnprintf_s_nid_postfix(char* buffer, size_t size, const char* format, VaList* args) { + return LibcDetail::FormatWindows(buffer, size, format, args); +} + int APS5_VABI snprintf_s_nid_postfix(char* buffer, size_t size, const char* format, ...) { __builtin_sysv_va_list args; __builtin_sysv_va_start(args, format); @@ -426,6 +435,20 @@ int APS5_VABI printf_s_nid_postfix(const char* format, ...) { #else +int APS5_VABI vsscanf_s_nid_postfix(const char* buffer, const char* format, VaList* args) { + LibcDetail::FormatArguments arguments(args); + return ScanGuest(buffer, format, true, [&] { return arguments.Next(); }, [&] { return arguments.Next(); }); +} + +int APS5_VABI vsnprintf_s_nid_postfix(char* buffer, size_t size, const char* format, VaList* args) { + if (HasGuestWideFormat(format)) return LibcDetail::FormatWindows(buffer, size, format, args); + std::va_list copy; + va_copy(copy, *reinterpret_cast(args)); + const int result = std::vsnprintf(buffer, size, format, copy); + va_end(copy); + return result; +} + int APS5_VABI vsprintf_s_nid_postfix(char* buffer, size_t size, const char* format, VaList* args) { if (HasGuestWideFormat(format)) return LibcDetail::FormatWindows(buffer, size, format, args); std::va_list copy; diff --git a/core/libs/tests/Formatting.cpp b/core/libs/tests/Formatting.cpp index 796def8e3..51e0cdf99 100644 --- a/core/libs/tests/Formatting.cpp +++ b/core/libs/tests/Formatting.cpp @@ -11,6 +11,8 @@ int APS5_VABI printf_nid_postfix(const char*, ...); int APS5_VABI libc_printf_nid_postfix(const char*, ...); int APS5_VABI sscanf_nid_postfix(const char*, const char*, ...); int APS5_VABI vsnprintf_nid_postfix(char*, size_t, const char*, VaList*); +int APS5_VABI vsnprintf_s_nid_postfix(char*, size_t, const char*, VaList*); +int APS5_VABI vsscanf_s_nid_postfix(const char*, const char*, VaList*); int APS5_VABI vprintf_nid_postfix(const char*, VaList*); } @@ -30,6 +32,26 @@ static int APS5_VABI FormatList(char* buffer, size_t size, const char* format, . return result; } +static int APS5_VABI FormatListS(char* buffer, size_t size, const char* format, ...) { + __builtin_sysv_va_list args; + __builtin_sysv_va_start(args, format); + VaList list; + std::memcpy(&list, args, sizeof(list)); + const int result = vsnprintf_s_nid_postfix(buffer, size, format, &list); + __builtin_sysv_va_end(args); + return result; +} + +static int APS5_VABI ScanListS(const char* buffer, const char* format, ...) { + __builtin_sysv_va_list args; + __builtin_sysv_va_start(args, format); + VaList list; + std::memcpy(&list, args, sizeof(list)); + const int result = vsscanf_s_nid_postfix(buffer, format, &list); + __builtin_sysv_va_end(args); + return result; +} + static int APS5_VABI PrintList(const char* format, ...) { __builtin_sysv_va_list args; __builtin_sysv_va_start(args, format); @@ -137,6 +159,10 @@ __attribute__((noinline)) static void APS5_VABI RunChecks() { Require(libc_printf_nid_postfix("libc_printf: %s\n", "OK") == 16); Require(libc_printf_nid_postfix("%d %d %d %d %d %d %d\n", 1, 22, 333, 4444, 55555, 666666, 7777777) == 35); Require(PrintList("vprintf: %d\n", 42) == 12); + Require(FormatListS(buffer, sizeof(buffer), "%d-%s", 42, "x") == 4 && std::strcmp(buffer, "42-x") == 0); + int scanNum = 0; + char scanWord[8] = {}; + Require(ScanListS("123 test", "%d %s", &scanNum, scanWord, 8u) == 2 && scanNum == 123 && std::strcmp(scanWord, "test") == 0); std::puts("Formatting checks passed: 10000 iterations"); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index bece55309..b7b8c1fdb 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -253,6 +253,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceKernelAddTimerEvent](../../core/libs/prx/libkernel/Equeue/Equeue.cpp) (libkernel) - `usec` is taken as the period in microseconds; re-adding an existing timer keeps its pending expiration count and restarts the period from the time of the call, where FreeBSD 12+ `filt_timertouch` clears pending expirations on `EV_ADD`; `usec == 0` fires on every wait. None of this is confirmed on a PS5 title - [sceRtcFormatRFC2822, sceRtcFormatRFC2822LocalTime, sceRtcFormatRFC3339LocalTime](../../core/libs/prx/libSceRtc/Export.cpp) (libSceRtc) - only shadPS4 (PS4) has bodies. A null tick returns `INVALID_POINTER` like the existing `sceRtcFormatRFC3339`, where shadPS4 formats the current time instead; offsets outside ±1439 minutes return `INVALID_VALUE` like `sceRtcFormatRFC3339`; the local-time variants use the host's offset at the given instant (shadPS4 uses `sceKernelGettimezone`, the current offset) - [snwprintf_s](../../core/libs/prx/libc/src/FormattingWide.cpp) (libc) - follows C11 K.3.9.1.3 with `RSIZE_MAX` assumed to be `SIZE_MAX >> 1`. A runtime-constraint violation only returns a negative value: `set_constraint_handler_s` is not exported and the console's default handler is unknown. An invalid multibyte `%s` argument is copied byte by byte instead of being reported as an encoding error, as in `vswprintf` +- [vsnprintf_s and vsscanf_s](../../core/libs/prx/libc/src/Formatting.cpp) (libc) - runtime constraint violations in C11 bounds-checking interfaces return negative/EOF values without invoking a constraint handler - [libSceUlt ulthreads](../../core/libs/prx/libSceUlt/Export.cpp) - `_sceUltUlthreadRuntimeOptParamInitialize`, `_sceUltUlthreadRuntimeCreate` and `_sceUltUlthreadCreate` are assumed to take the same arguments as the existing exports without the underscore, including the trailing build version. `sceUltUlthreadTryJoin` is assumed to return `ULT_ERROR_BUSY` while the ulthread runs, and `sceUltUlthreadRuntimeDestroy` to return `ULT_ERROR_BUSY` while an ulthread created on the runtime is not joined. Neither error has been confirmed on PS5 hardware - [libSceUlt queues](../../core/libs/prx/libSceUlt/Export.cpp) - the added `sceUltQueuePop`, `sceUltQueueTryPush`, `sceUltQueueDestroy` and `sceUltQueueDataResourcePoolDestroy` signatures are inferred from the existing push/pop and object APIs. Queue destruction is assumed to reject blocked callers with `ULT_ERROR_BUSY` and otherwise discard queued data; pool exhaustion returns `ULT_ERROR_AGAIN`. These rules, alignment checks and shared slot limits have not been confirmed on PS5 hardware or a title. Non-null queue and data-pool option parameters throw because their layouts are unknown. - [AC1FtjqMCL0](../../core/libs/prx/libkernel/Unimplemented.cpp) (libkernel) - unknown name, signature From 4d537755e98995f2c12501fe9ba22ee10f4baa12 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 21:04:48 +0000 Subject: [PATCH 171/460] test(libc): cover vsscanf_s capacity limits --- core/libs/tests/Formatting.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/core/libs/tests/Formatting.cpp b/core/libs/tests/Formatting.cpp index 51e0cdf99..0b2846884 100644 --- a/core/libs/tests/Formatting.cpp +++ b/core/libs/tests/Formatting.cpp @@ -163,6 +163,8 @@ __attribute__((noinline)) static void APS5_VABI RunChecks() { int scanNum = 0; char scanWord[8] = {}; Require(ScanListS("123 test", "%d %s", &scanNum, scanWord, 8u) == 2 && scanNum == 123 && std::strcmp(scanWord, "test") == 0); + char tooSmall[4] = {'x', 'x', 'x', 'x'}; + Require(ScanListS("abcdef", "%s", tooSmall, 4u) == 0 && tooSmall[0] == '\0'); std::puts("Formatting checks passed: 10000 iterations"); } From db2743e3779b3cc7a22e4957f12f30d2429b2ed7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sat, 10 Oct 2026 23:52:18 +0200 Subject: [PATCH 172/460] docs(technical-debt): describe the vsnprintf_s/vsscanf_s constraint behaviour as implemented --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index b7b8c1fdb..c29b187b4 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -253,7 +253,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceKernelAddTimerEvent](../../core/libs/prx/libkernel/Equeue/Equeue.cpp) (libkernel) - `usec` is taken as the period in microseconds; re-adding an existing timer keeps its pending expiration count and restarts the period from the time of the call, where FreeBSD 12+ `filt_timertouch` clears pending expirations on `EV_ADD`; `usec == 0` fires on every wait. None of this is confirmed on a PS5 title - [sceRtcFormatRFC2822, sceRtcFormatRFC2822LocalTime, sceRtcFormatRFC3339LocalTime](../../core/libs/prx/libSceRtc/Export.cpp) (libSceRtc) - only shadPS4 (PS4) has bodies. A null tick returns `INVALID_POINTER` like the existing `sceRtcFormatRFC3339`, where shadPS4 formats the current time instead; offsets outside ±1439 minutes return `INVALID_VALUE` like `sceRtcFormatRFC3339`; the local-time variants use the host's offset at the given instant (shadPS4 uses `sceKernelGettimezone`, the current offset) - [snwprintf_s](../../core/libs/prx/libc/src/FormattingWide.cpp) (libc) - follows C11 K.3.9.1.3 with `RSIZE_MAX` assumed to be `SIZE_MAX >> 1`. A runtime-constraint violation only returns a negative value: `set_constraint_handler_s` is not exported and the console's default handler is unknown. An invalid multibyte `%s` argument is copied byte by byte instead of being reported as an encoding error, as in `vswprintf` -- [vsnprintf_s and vsscanf_s](../../core/libs/prx/libc/src/Formatting.cpp) (libc) - runtime constraint violations in C11 bounds-checking interfaces return negative/EOF values without invoking a constraint handler +- [vsnprintf_s and vsscanf_s](../../core/libs/prx/libc/src/Formatting.cpp) (libc) - `vsnprintf_s` forwards to `vsnprintf` semantics: the C11 K.3.7.1.5 runtime constraints (null buffer, `size == 0`, `%n`, null `%s`) are not checked, so `size == 0` returns the would-be length instead of a negative value, and no constraint handler is invoked. `vsscanf_s` returns the count of items converted before an undersized `%s`/`%c`/`%[` destination (0 when it is the first), not EOF. Neither is confirmed on a PS5 title - [libSceUlt ulthreads](../../core/libs/prx/libSceUlt/Export.cpp) - `_sceUltUlthreadRuntimeOptParamInitialize`, `_sceUltUlthreadRuntimeCreate` and `_sceUltUlthreadCreate` are assumed to take the same arguments as the existing exports without the underscore, including the trailing build version. `sceUltUlthreadTryJoin` is assumed to return `ULT_ERROR_BUSY` while the ulthread runs, and `sceUltUlthreadRuntimeDestroy` to return `ULT_ERROR_BUSY` while an ulthread created on the runtime is not joined. Neither error has been confirmed on PS5 hardware - [libSceUlt queues](../../core/libs/prx/libSceUlt/Export.cpp) - the added `sceUltQueuePop`, `sceUltQueueTryPush`, `sceUltQueueDestroy` and `sceUltQueueDataResourcePoolDestroy` signatures are inferred from the existing push/pop and object APIs. Queue destruction is assumed to reject blocked callers with `ULT_ERROR_BUSY` and otherwise discard queued data; pool exhaustion returns `ULT_ERROR_AGAIN`. These rules, alignment checks and shared slot limits have not been confirmed on PS5 hardware or a title. Non-null queue and data-pool option parameters throw because their layouts are unknown. - [AC1FtjqMCL0](../../core/libs/prx/libkernel/Unimplemented.cpp) (libkernel) - unknown name, signature From 97ae394037e6a2dcde8827390a102a425c53cbdc Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 21:58:20 +0000 Subject: [PATCH 173/460] fix(test): create isolated video out fixture --- core/libs/tests/GuestVideoOutOutputMode.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/core/libs/tests/GuestVideoOutOutputMode.cpp b/core/libs/tests/GuestVideoOutOutputMode.cpp index dd8dbb31d..01b657eac 100644 --- a/core/libs/tests/GuestVideoOutOutputMode.cpp +++ b/core/libs/tests/GuestVideoOutOutputMode.cpp @@ -1,5 +1,6 @@ #include "SceTypes.hpp" #include "prx/libc/include/Shutdown.hpp" +#include "tests/VideoOutTestEnvironment.hpp" #include #include #include @@ -26,6 +27,7 @@ static constexpr int UNKNOWN_OUTPUT_MODE = static_cast(0x8029001E); static void Require(bool value) { if (!value) std::abort(); } int main() { + VideoOutTestEnvironment environment; std::array raw{}; raw.fill(0xAA); VideoOutOutputOptions options{}; From b446119c7628c5c05914983b1daec3dcb13283b1 Mon Sep 17 00:00:00 2001 From: David Date: Sat, 10 Oct 2026 23:27:12 +0200 Subject: [PATCH 174/460] feat(libSceAmpr): implement the Dinkumware thread, sleep and spin lock exports Co-Authored-By: Claude Fable 5.1 --- core/libs/prx/libSceAmpr/CMakeLists.txt | 12 +- core/libs/prx/libSceAmpr/Export.cpp | 82 ++++++++-- core/libs/tests/GuestAmprThreads.cpp | 206 ++++++++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 285 insertions(+), 16 deletions(-) create mode 100644 core/libs/tests/GuestAmprThreads.cpp diff --git a/core/libs/prx/libSceAmpr/CMakeLists.txt b/core/libs/prx/libSceAmpr/CMakeLists.txt index bf963496e..09a966fea 100644 --- a/core/libs/prx/libSceAmpr/CMakeLists.txt +++ b/core/libs/prx/libSceAmpr/CMakeLists.txt @@ -1,6 +1,6 @@ add_library(libSceAmpr SHARED EXCLUDE_FROM_ALL Export.cpp) target_include_directories(libSceAmpr PRIVATE ${LIBS_INCLUDE_DIR}) -target_link_libraries(libSceAmpr PRIVATE libkernel) +target_link_libraries(libSceAmpr PRIVATE libkernel libc) configure_windows_unwind(libSceAmpr) if(BUILD_TESTING) @@ -13,4 +13,14 @@ if(BUILD_TESTING) set_tests_properties(guest_ampr_markers PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + + add_test_executable(guest_ampr_threads_tests ${LIBS_INCLUDE_DIR}/tests/GuestAmprThreads.cpp) + target_include_directories(guest_ampr_threads_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_ampr_threads_tests PRIVATE libSceAmpr libkernel libc) + configure_windows_unwind(guest_ampr_threads_tests) + add_test(NAME guest_ampr_threads COMMAND guest_ampr_threads_tests) + if(WIN32) + set_tests_properties(guest_ampr_threads PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() endif() diff --git a/core/libs/prx/libSceAmpr/Export.cpp b/core/libs/prx/libSceAmpr/Export.cpp index 89561ff3f..68703624c 100644 --- a/core/libs/prx/libSceAmpr/Export.cpp +++ b/core/libs/prx/libSceAmpr/Export.cpp @@ -3,8 +3,14 @@ #include "SceTypes.hpp" #include "prx/libc/include/General.hpp" #include "prx/libkernel/Apr/include/AprCommandBuffer.hpp" +#include "prx/libkernel/Time/include/Time.hpp" +#include +#include +#include #include +#include #include +#include static constexpr int SCE_AMPR_ERROR_BUFFER_FULL = 0x8002001C; static constexpr int SCE_KERNEL_ERROR_EINVAL = 0x80020016; @@ -210,6 +216,40 @@ static int RecordWriteAddressFromCounterPair(Apr::CommandBufferObject* buffer, v return AppendCommand(buffer, Apr::Opcode::WriteAddressFromCounterPair, Apr::WriteAddressFromCounterCommand{{}, reinterpret_cast(address), counter, counter + 1u}); } +struct Xtime { + std::int64_t sec; + std::int64_t nsec; +}; + +static constexpr int ThrdSuccess = 0; +static constexpr int ThrdError = 4; +static constexpr int GuestClockRealtime = 0; +static constexpr std::int64_t NanosPerSecond = 1000000000; +static constexpr std::int64_t LongestSleepSeconds = 86400; + +static std::atomic_flag sharedPtrSpinLock; + +extern "C" { +int APS5_VABI pthread_join_nid_postfix(Pthread thread, void** value); +int APS5_VABI wcsrtombs_s_nid_postfix(std::size_t* result, char* destination, std::size_t capacity, const std::uint16_t** source, std::size_t limit, void* state); +} + +static KernelTimespec XtimeDeadline(const Xtime& time) { + constexpr auto largest = std::numeric_limits::max(); + constexpr auto smallest = std::numeric_limits::min(); + const auto carry = time.nsec / NanosPerSecond; + auto nanos = time.nsec % NanosPerSecond; + auto seconds = time.sec; + if ((carry > 0 && seconds > largest - carry) || (carry < 0 && seconds < smallest - carry)) throw std::overflow_error("_Thrd_sleep: xtime seconds overflow"); + seconds += carry; + if (nanos < 0) { + if (seconds == smallest) throw std::overflow_error("_Thrd_sleep: xtime seconds overflow"); + nanos += NanosPerSecond; + --seconds; + } + return {seconds, nanos}; +} + extern "C" { int APS5_VABI sceAmprCommandBufferWriteAddressOnCompletion(Apr::CommandBufferObject* buffer, volatile std::uint64_t* address, std::uint64_t value) { @@ -812,9 +852,11 @@ int APS5_VABI _ZSt16_Throw_Cpp_errori_nid_postfix(void) { return 0; } -int APS5_VABI _Thrd_join_nid_postfix(void) { - NotImplemented_nid_no_patch(__func__); - return 0; +int APS5_VABI _Thrd_join_nid_postfix(Pthread thread, int* code) { + void* result = nullptr; + if (pthread_join_nid_postfix(thread, &result) != 0) return ThrdError; + if (code) *code = static_cast(reinterpret_cast(result)); + return ThrdSuccess; } int APS5_VABI _ZNSt4_Pad8_ReleaseEv_nid_postfix(void) { @@ -852,9 +894,8 @@ int APS5_VABI _Cnd_signal_nid_postfix(void) { return 0; } -int APS5_VABI _Unlock_shared_ptr_spin_lock_nid_postfix(void) { - NotImplemented_nid_no_patch(__func__); - return 0; +void APS5_VABI _Unlock_shared_ptr_spin_lock_nid_postfix(void) { + sharedPtrSpinLock.clear(std::memory_order_release); } int APS5_VABI _Cnd_init_with_name_nid_postfix(void) { @@ -892,19 +933,30 @@ int APS5_VABI _ZNSt8time_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE7_Get return 0; } -int APS5_VABI _Lock_shared_ptr_spin_lock_nid_postfix(void) { - NotImplemented_nid_no_patch(__func__); - return 0; +void APS5_VABI _Lock_shared_ptr_spin_lock_nid_postfix(void) { + while (sharedPtrSpinLock.test_and_set(std::memory_order_acquire)) std::this_thread::yield(); } -int APS5_VABI _Thrd_sleep_nid_postfix(void) { - NotImplemented_nid_no_patch(__func__); - return 0; +void APS5_VABI _Thrd_sleep_nid_postfix(const Xtime* target) { + if (!target) APS5_INVALID_ARG_EX; + const auto deadline = XtimeDeadline(*target); + for (;;) { + KernelTimespec now{}; + if (clock_gettime_nid_postfix(GuestClockRealtime, &now) != 0) throw std::runtime_error("_Thrd_sleep: reading the realtime clock failed"); + if (now.tv_sec > deadline.tv_sec || (now.tv_sec == deadline.tv_sec && now.tv_nsec >= deadline.tv_nsec)) return; + KernelTimespec remaining{deadline.tv_sec - now.tv_sec, deadline.tv_nsec - now.tv_nsec}; + if (remaining.tv_nsec < 0) { + remaining.tv_nsec += NanosPerSecond; + --remaining.tv_sec; + } + remaining.tv_sec = std::min(remaining.tv_sec, LongestSleepSeconds); + if (nanosleep_nid_postfix(&remaining, nullptr) != 0) throw std::runtime_error("_Thrd_sleep: nanosleep failed"); + } } -int APS5_VABI wcstombs_s_nid_postfix(void) { - NotImplemented_nid_no_patch(__func__); - return 0; +int APS5_VABI wcstombs_s_nid_postfix(std::size_t* result, char* destination, std::size_t capacity, const std::uint16_t* source, std::size_t limit) { + std::array state{}; + return wcsrtombs_s_nid_postfix(result, destination, capacity, &source, limit, state.data()); } int APS5_VABI __cxa_call_unexpected_nid_postfix(void) { diff --git a/core/libs/tests/GuestAmprThreads.cpp b/core/libs/tests/GuestAmprThreads.cpp new file mode 100644 index 000000000..98be7860b --- /dev/null +++ b/core/libs/tests/GuestAmprThreads.cpp @@ -0,0 +1,206 @@ +#include "SceTypes.hpp" +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct GuestXtime { + std::int64_t sec; + std::int64_t nsec; +}; + +extern "C" { +int APS5_VABI _Thrd_join_nid_postfix(Pthread thread, int* code); +void APS5_VABI _Thrd_sleep_nid_postfix(const GuestXtime* target); +void APS5_VABI _Lock_shared_ptr_spin_lock_nid_postfix(void); +void APS5_VABI _Unlock_shared_ptr_spin_lock_nid_postfix(void); +int APS5_VABI wcstombs_s_nid_postfix(std::size_t* result, char* destination, std::size_t capacity, const std::uint16_t* source, std::size_t limit); +int APS5_VABI scePthreadCreate(Pthread* thread, const PthreadAttr* attr, PthreadEntry entry, void* arg, const char* name); +int APS5_VABI pthread_detach_nid_postfix(Pthread thread); +int APS5_VABI clock_gettime_nid_postfix(int clockId, KernelTimespec* tp); +} + +static constexpr int ThrdSuccess = 0; +static constexpr int ThrdError = 4; +static constexpr int GuestEinval = 22; +static constexpr int GuestErange = 34; +static constexpr int GuestEilseq = 86; +static constexpr std::int64_t NanosPerSecond = 1000000000; +static constexpr auto Failed = static_cast(-1); + +#define Require(value) RequireAt((value), __LINE__) + +static void RequireAt(bool value, int line) { + if (value) return; + std::fprintf(stderr, "requirement failed at line %d\n", line); + std::abort(); +} + +static std::int64_t RealtimeNanos() { + KernelTimespec now{}; + Require(clock_gettime_nid_postfix(0, &now) == 0); + return now.tv_sec * NanosPerSecond + now.tv_nsec; +} + +static GuestXtime XtimeAfter(std::int64_t nanos) { + const auto target = RealtimeNanos() + nanos; + return {target / NanosPerSecond, target % NanosPerSecond}; +} + +static void* APS5_VABI ReturnArgument(void* arg) { + return arg; +} + +static void* APS5_VABI WaitForRelease(void* arg) { + auto* release = static_cast*>(arg); + while (!release->load()) std::this_thread::yield(); + return nullptr; +} + +struct SpinContext { + std::atomic inside{0}; + std::atomic overlapped{false}; + int counter = 0; +}; + +static constexpr int SpinThreads = 4; +static constexpr int SpinIterations = 20000; + +static void* APS5_VABI SpinWorker(void* arg) { + auto& context = *static_cast(arg); + for (int iteration = 0; iteration < SpinIterations; ++iteration) { + _Lock_shared_ptr_spin_lock_nid_postfix(); + if (context.inside.fetch_add(1) != 0) context.overlapped = true; + ++context.counter; + context.inside.fetch_sub(1); + _Unlock_shared_ptr_spin_lock_nid_postfix(); + } + return nullptr; +} + +static Pthread Start(PthreadEntry entry, void* arg) { + Pthread thread = nullptr; + Require(scePthreadCreate(&thread, nullptr, entry, arg, nullptr) == 0); + return thread; +} + +static void TestThreadJoin() { + int code = 0; + Require(_Thrd_join_nid_postfix(Start(ReturnArgument, reinterpret_cast(std::intptr_t{42})), &code) == ThrdSuccess); + Require(code == 42); + code = 0; + Require(_Thrd_join_nid_postfix(Start(ReturnArgument, reinterpret_cast(std::intptr_t{-7})), &code) == ThrdSuccess); + Require(code == -7); + code = 5; + Require(_Thrd_join_nid_postfix(Start(ReturnArgument, reinterpret_cast(std::intptr_t{0x100000003})), &code) == ThrdSuccess); + Require(code == 3); + Require(_Thrd_join_nid_postfix(Start(ReturnArgument, nullptr), nullptr) == ThrdSuccess); + + static std::atomic release{false}; + const auto detached = Start(WaitForRelease, &release); + Require(pthread_detach_nid_postfix(detached) == 0); + code = 9; + Require(_Thrd_join_nid_postfix(detached, &code) == ThrdError); + Require(code == 9); + release = true; +} + +static void TestThreadSleep() { + const auto target = XtimeAfter(50000000); + _Thrd_sleep_nid_postfix(&target); + Require(RealtimeNanos() >= target.sec * NanosPerSecond + target.nsec); + + const auto past = XtimeAfter(-NanosPerSecond); + const auto before = RealtimeNanos(); + _Thrd_sleep_nid_postfix(&past); + Require(RealtimeNanos() - before < NanosPerSecond / 2); + + const auto carried = XtimeAfter(30000000); + const GuestXtime unnormalized{carried.sec - 2, carried.nsec + 2 * NanosPerSecond}; + _Thrd_sleep_nid_postfix(&unnormalized); + Require(RealtimeNanos() >= carried.sec * NanosPerSecond + carried.nsec); + + const auto borrowed = XtimeAfter(20000000); + const GuestXtime negative{borrowed.sec + 1, borrowed.nsec - NanosPerSecond}; + _Thrd_sleep_nid_postfix(&negative); + Require(RealtimeNanos() >= borrowed.sec * NanosPerSecond + borrowed.nsec); + + bool rejected = false; + try { + _Thrd_sleep_nid_postfix(nullptr); + } catch (const std::invalid_argument&) { + rejected = true; + } + Require(rejected); +} + +static void TestSpinLock() { + SpinContext context; + std::array threads{}; + for (auto& thread : threads) thread = Start(SpinWorker, &context); + for (auto thread : threads) Require(_Thrd_join_nid_postfix(thread, nullptr) == ThrdSuccess); + Require(!context.overlapped); + Require(context.counter == SpinThreads * SpinIterations); + _Lock_shared_ptr_spin_lock_nid_postfix(); + _Unlock_shared_ptr_spin_lock_nid_postfix(); +} + +static void TestWcstombs() { + const std::uint16_t text[] = {u'a', u'b', u'c', 0}; + const std::uint16_t wide[] = {u'a', 0x100, 0}; + std::size_t result = 0; + char buffer[8]; + + std::memset(buffer, 'x', sizeof(buffer)); + Require(wcstombs_s_nid_postfix(&result, buffer, sizeof(buffer), text, sizeof(buffer)) == 0); + Require(result == 3 && std::strcmp(buffer, "abc") == 0); + + std::memset(buffer, 'x', sizeof(buffer)); + Require(wcstombs_s_nid_postfix(&result, buffer, sizeof(buffer), text, 2) == 0); + Require(result == 2 && std::strcmp(buffer, "ab") == 0); + + result = 0; + Require(wcstombs_s_nid_postfix(&result, nullptr, 0, text, 0) == 0); + Require(result == 3); + + std::memset(buffer, 'x', sizeof(buffer)); + Require(wcstombs_s_nid_postfix(&result, buffer, 3, text, 3) == GuestErange); + Require(result == Failed && buffer[0] == '\0'); + + result = 0; + Require(wcstombs_s_nid_postfix(&result, nullptr, 4, text, 4) == GuestEinval); + Require(result == Failed); + + std::memset(buffer, 'x', sizeof(buffer)); + Require(wcstombs_s_nid_postfix(&result, buffer, sizeof(buffer), nullptr, sizeof(buffer)) == GuestEinval); + Require(result == Failed && buffer[0] == '\0'); + + std::memset(buffer, 'x', sizeof(buffer)); + Require(wcstombs_s_nid_postfix(nullptr, buffer, sizeof(buffer), text, sizeof(buffer)) == GuestEinval); + Require(buffer[0] == '\0'); + + std::memset(buffer, 'x', sizeof(buffer)); + Require(wcstombs_s_nid_postfix(&result, buffer, 0, text, 4) == GuestEinval); + Require(result == Failed && buffer[0] == 'x'); + + std::memset(buffer, 'x', sizeof(buffer)); + Require(wcstombs_s_nid_postfix(&result, buffer, sizeof(buffer), wide, sizeof(buffer)) == GuestEilseq); + Require(result == Failed && buffer[0] == 'a' && buffer[1] == '\0'); + + Require(wcstombs_s_nid_postfix(&result, nullptr, 0, wide, 0) == GuestEilseq); + Require(result == Failed); +} + +int main() { + TestThreadJoin(); + TestThreadSleep(); + TestSpinLock(); + TestWcstombs(); + return 0; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index bece55309..c6bd59906 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -487,3 +487,4 @@ Throughout the project, every function at every stage either **does exactly what - [AJM](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp), [APR](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) and [NGS2](../../core/libs/prx/libSceNgs2.native/src/System.cpp) state is allocated with `new` and never freed on purpose: title threads still call into these libraries while `std::exit` runs static destructors, so ordinary statics would be destroyed under them. Keep them leaked; `guest_ajm_teardown` covers the AJM case on Unix. - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields +- [Dinkumware thread exports](../../core/libs/prx/libSceAmpr/Export.cpp) (libSceAmpr) follow the Dinkumware threads library shipped with [Visual C++ 2013](https://github.com/adityaramesh/concurrency_runtime/tree/master/VS2013_SP2/crt/src/thr) (`cthread.c`, `xtime.c`) and its `atomic.c`, [microsoft/STL `cthread.cpp`](https://github.com/microsoft/STL/blob/main/stl/src/cthread.cpp) and C11 Annex K, without console verification. `_Thrd_t` is taken to be the guest `pthread_t`, as the `std::_Pad::_Launch(pthread**)` export suggests: `_Thrd_join` returns `_Thrd_error` (4) when `pthread_join` fails and stores the thread's return value truncated to `int`. `_Thrd_sleep` reads a 16-byte xtime (64-bit seconds and nanoseconds) on the guest `CLOCK_REALTIME`, normalizes the nanoseconds and sleeps through `nanosleep`, so it is a cancellation point; a null xtime or one whose normalization overflows throws. The shared_ptr spin lock is one process-wide flag that yields while it spins. `wcstombs_s` is `wcsrtombs_s` with a fresh conversion state, so `*retval` excludes the terminating null as in Annex K, not as in the Microsoft CRT, and `_TRUNCATE` has no meaning From fddcb00a40c47678c29b77dec6f6aa5be88d6c05 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 00:01:46 +0200 Subject: [PATCH 175/460] fix(libSceAmpr): give _Thrd_sleep the console signature and move its TechnicalDebt line The title's libc.prx _Thrd_sleep takes a second xtime* (remaining) and returns int. The TechnicalDebt entry no longer sits at the end of its section. --- core/libs/prx/libSceAmpr/Export.cpp | 4 ++-- core/libs/tests/GuestAmprThreads.cpp | 12 ++++++------ docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/core/libs/prx/libSceAmpr/Export.cpp b/core/libs/prx/libSceAmpr/Export.cpp index 68703624c..22af4667a 100644 --- a/core/libs/prx/libSceAmpr/Export.cpp +++ b/core/libs/prx/libSceAmpr/Export.cpp @@ -937,13 +937,13 @@ void APS5_VABI _Lock_shared_ptr_spin_lock_nid_postfix(void) { while (sharedPtrSpinLock.test_and_set(std::memory_order_acquire)) std::this_thread::yield(); } -void APS5_VABI _Thrd_sleep_nid_postfix(const Xtime* target) { +int APS5_VABI _Thrd_sleep_nid_postfix(const Xtime* target, Xtime*) { if (!target) APS5_INVALID_ARG_EX; const auto deadline = XtimeDeadline(*target); for (;;) { KernelTimespec now{}; if (clock_gettime_nid_postfix(GuestClockRealtime, &now) != 0) throw std::runtime_error("_Thrd_sleep: reading the realtime clock failed"); - if (now.tv_sec > deadline.tv_sec || (now.tv_sec == deadline.tv_sec && now.tv_nsec >= deadline.tv_nsec)) return; + if (now.tv_sec > deadline.tv_sec || (now.tv_sec == deadline.tv_sec && now.tv_nsec >= deadline.tv_nsec)) return 0; KernelTimespec remaining{deadline.tv_sec - now.tv_sec, deadline.tv_nsec - now.tv_nsec}; if (remaining.tv_nsec < 0) { remaining.tv_nsec += NanosPerSecond; diff --git a/core/libs/tests/GuestAmprThreads.cpp b/core/libs/tests/GuestAmprThreads.cpp index 98be7860b..81ea82946 100644 --- a/core/libs/tests/GuestAmprThreads.cpp +++ b/core/libs/tests/GuestAmprThreads.cpp @@ -17,7 +17,7 @@ struct GuestXtime { extern "C" { int APS5_VABI _Thrd_join_nid_postfix(Pthread thread, int* code); -void APS5_VABI _Thrd_sleep_nid_postfix(const GuestXtime* target); +int APS5_VABI _Thrd_sleep_nid_postfix(const GuestXtime* target, GuestXtime* remaining); void APS5_VABI _Lock_shared_ptr_spin_lock_nid_postfix(void); void APS5_VABI _Unlock_shared_ptr_spin_lock_nid_postfix(void); int APS5_VABI wcstombs_s_nid_postfix(std::size_t* result, char* destination, std::size_t capacity, const std::uint16_t* source, std::size_t limit); @@ -113,27 +113,27 @@ static void TestThreadJoin() { static void TestThreadSleep() { const auto target = XtimeAfter(50000000); - _Thrd_sleep_nid_postfix(&target); + Require(_Thrd_sleep_nid_postfix(&target, nullptr) == 0); Require(RealtimeNanos() >= target.sec * NanosPerSecond + target.nsec); const auto past = XtimeAfter(-NanosPerSecond); const auto before = RealtimeNanos(); - _Thrd_sleep_nid_postfix(&past); + Require(_Thrd_sleep_nid_postfix(&past, nullptr) == 0); Require(RealtimeNanos() - before < NanosPerSecond / 2); const auto carried = XtimeAfter(30000000); const GuestXtime unnormalized{carried.sec - 2, carried.nsec + 2 * NanosPerSecond}; - _Thrd_sleep_nid_postfix(&unnormalized); + Require(_Thrd_sleep_nid_postfix(&unnormalized, nullptr) == 0); Require(RealtimeNanos() >= carried.sec * NanosPerSecond + carried.nsec); const auto borrowed = XtimeAfter(20000000); const GuestXtime negative{borrowed.sec + 1, borrowed.nsec - NanosPerSecond}; - _Thrd_sleep_nid_postfix(&negative); + Require(_Thrd_sleep_nid_postfix(&negative, nullptr) == 0); Require(RealtimeNanos() >= borrowed.sec * NanosPerSecond + borrowed.nsec); bool rejected = false; try { - _Thrd_sleep_nid_postfix(nullptr); + _Thrd_sleep_nid_postfix(nullptr, nullptr); } catch (const std::invalid_argument&) { rejected = true; } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index c6bd59906..c2bc3e1f4 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -299,6 +299,7 @@ Throughout the project, every function at every stage either **does exactly what - [libSceOpusDec](../../core/libs/prx/libSceOpusDec/Export.cpp) uses FFmpeg for 48000 Hz mono/stereo signed-16 decoding. The six signatures, 640-byte opaque state allocation and byte-count return follow [public PS5 decoder research](https://github.com/blackbearreloaded/ps5-hardware-audio-decoding/blob/26be5a12f92b2cdc67f8a9e082fd69ae0e9a35a2/docs/DECODING.md#direct-opus-abi) and [ProsperoRadio](https://github.com/blackbearreloaded/ProsperoRadio/blob/2c4029c3a8f43e9e1dfef98374d94d99b14d745d/src/opus_decoder.cpp). Console PCM equivalence and error codes are unverified. Other sample rates, packet loss concealment, invalid inputs and termination with live decoders throw. Context identifiers and host decoder state are private registries; the firmware's opaque state layout and error recovery are unknown. - [mkstemp and isatty](../../core/libs/prx/libScePosixForWebKit/Filesystem.cpp) (libScePosixForWebKit) follow FreeBSD's [temporary-file API](https://github.com/freebsd/freebsd-src/blob/stable/9/lib/libc/stdio/mktemp.c) and [terminal check](https://github.com/freebsd/freebsd-src/blob/stable/9/lib/libc/gen/isatty.c), without console verification. `mkstemp` replaces all trailing `X` characters, accepts a fixed name, and creates files through the guest path resolver with exclusive read/write access and mode 0600. Windows file permissions follow the existing kernel file implementation. `isatty` tests Windows console handles rather than all character devices; files, pipes and guest sockets return `ENOTTY`, closed descriptors return `EBADF`. +- [Dinkumware thread exports](../../core/libs/prx/libSceAmpr/Export.cpp) (libSceAmpr) follow the Dinkumware threads library shipped with [Visual C++ 2013](https://github.com/adityaramesh/concurrency_runtime/tree/master/VS2013_SP2/crt/src/thr) (`cthread.c`, `xtime.c`) and its `atomic.c`, [microsoft/STL `cthread.cpp`](https://github.com/microsoft/STL/blob/main/stl/src/cthread.cpp) and C11 Annex K, without console verification. `_Thrd_t` is taken to be the guest `pthread_t`, as the `std::_Pad::_Launch(pthread**)` export suggests: `_Thrd_join` returns `_Thrd_error` (4) when `pthread_join` fails and stores the thread's return value truncated to `int`. `_Thrd_sleep` reads a 16-byte xtime (64-bit seconds and nanoseconds) on the guest `CLOCK_REALTIME`, normalizes the nanoseconds and sleeps through `nanosleep`, so it is a cancellation point; a null xtime or one whose normalization overflows throws. The console's `_Thrd_sleep` (disassembled from a title's `libc.prx`) also takes an optional `xtime*` of remaining time and returns `int`; here the sleep is never cut short, so it returns 0 and leaves the remaining time untouched. `_Thrd_join` (32-bit store of the result, 0 or 4) and the spin lock (byte exchange loop, no yield) match that disassembly. The shared_ptr spin lock is one process-wide flag that yields while it spins. `wcstombs_s` is `wcsrtombs_s` with a fresh conversion state, so `*retval` excludes the terminating null as in Annex K, not as in the Microsoft CRT, and `_TRUNCATE` has no meaning - [libSceZlib](../../core/libs/prx/libSceZlib/Export.cpp) completes RFC1950 decompression during submission and supports one outstanding request, null-timeout waits and one result retrieval. Timed waits, multiple outstanding requests, repeated retrieval and finalization with an unread result throw. The 64-bit initialization length and 64 KiB output limit with 2 KiB destination alignment follow [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/5dfdb4b0b166e83dc9ae275001afb695f12c7174/src/SharpProspero/Interop/Compression/Zlib.cs); the PS4 initialization length is 32 bits. shadPS4 also requires the output length to be a multiple of 2 KiB and returns `0x81120016` for invalid arguments; this implementation accepts any length up to 64 KiB (SharpProspero documents only the pointer alignment) and throws on invalid arguments. Scratch memory is not used, and its firmware size/alignment requirements are unknown. The initialization errors follow SharpProspero; malformed/checksum/truncated streams report `0x811200FF` and insufficient output `0x8112001C` as in [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/0fe263a4760dfbfa973366890061749b4af0de97/src/core/libraries/zlib/zlib.cpp). PS5 error precedence, scheduling, completion selection, result consumption and cancellation are unverified. The payload SDK's FreeBSD kernel `z_stream` and host errno are not this library's ABI. - [Windows file timestamps](../../core/libs/prx/libkernel/File/include/WindowsFileTime.hpp) use the host filesystem's timestamp range and resolution. `utimes` and `futimes` reject timestamps outside nonnegative signed 64-bit Windows file-time ticks with `EINVAL`, including exactly 1601-01-01 00:00:00 UTC, whose zero value means "leave unchanged" to `SetFileTime`. Guest directory descriptors still resolve through their recorded path. Filesystems with coarser timestamp resolution cannot retain every guest microsecond - [Windows file identity](../../core/libs/prx/libkernel/File/src/NativeStat.cpp) (libkernel) - `stat` and `fstat` report the volume serial number as `st_dev` and the 64-bit file index folded into the guest's 32-bit `st_ino` (never 0), so two different files can in principle share an `st_ino`; the fold is not taken from a reference. `st_nlink` is clamped to 65535, the width of the guest field. The call keeps the `_wstat64` values when `GetFileInformationByHandle` fails @@ -487,4 +488,3 @@ Throughout the project, every function at every stage either **does exactly what - [AJM](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp), [APR](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) and [NGS2](../../core/libs/prx/libSceNgs2.native/src/System.cpp) state is allocated with `new` and never freed on purpose: title threads still call into these libraries while `std::exit` runs static destructors, so ordinary statics would be destroyed under them. Keep them leaked; `guest_ajm_teardown` covers the AJM case on Unix. - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields -- [Dinkumware thread exports](../../core/libs/prx/libSceAmpr/Export.cpp) (libSceAmpr) follow the Dinkumware threads library shipped with [Visual C++ 2013](https://github.com/adityaramesh/concurrency_runtime/tree/master/VS2013_SP2/crt/src/thr) (`cthread.c`, `xtime.c`) and its `atomic.c`, [microsoft/STL `cthread.cpp`](https://github.com/microsoft/STL/blob/main/stl/src/cthread.cpp) and C11 Annex K, without console verification. `_Thrd_t` is taken to be the guest `pthread_t`, as the `std::_Pad::_Launch(pthread**)` export suggests: `_Thrd_join` returns `_Thrd_error` (4) when `pthread_join` fails and stores the thread's return value truncated to `int`. `_Thrd_sleep` reads a 16-byte xtime (64-bit seconds and nanoseconds) on the guest `CLOCK_REALTIME`, normalizes the nanoseconds and sleeps through `nanosleep`, so it is a cancellation point; a null xtime or one whose normalization overflows throws. The shared_ptr spin lock is one process-wide flag that yields while it spins. `wcstombs_s` is `wcsrtombs_s` with a fresh conversion state, so `*retval` excludes the terminating null as in Annex K, not as in the Microsoft CRT, and `_TRUNCATE` has no meaning From 9902f137af55c1bd51a0ab83467d82a51cccfd4a Mon Sep 17 00:00:00 2001 From: David Date: Sat, 10 Oct 2026 23:08:59 +0200 Subject: [PATCH 176/460] feat(libkernel): implement sceKernelFcntl Co-Authored-By: Claude Fable 5.1 (cherry picked from commit f0c649704dbf0b84077c29ea914e0e4799714fe1) --- core/libs/prx/libkernel/CMakeLists.txt | 10 ++++ core/libs/prx/libkernel/File/include/File.hpp | 3 ++ core/libs/prx/libkernel/File/src/Open.cpp | 29 ++++++++-- core/libs/prx/libkernel/File/src/Stdio.cpp | 7 +++ core/libs/tests/GuestKernelFcntl.cpp | 53 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 6 files changed, 100 insertions(+), 3 deletions(-) create mode 100644 core/libs/tests/GuestKernelFcntl.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index adaefe617..fa9dbc417 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -812,6 +812,16 @@ if(BUILD_TESTING) set_tests_properties(guest_fstatfs PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_kernel_fcntl_tests ${LIBS_INCLUDE_DIR}/tests/GuestKernelFcntl.cpp) + target_include_directories(guest_kernel_fcntl_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_kernel_fcntl_tests PRIVATE libkernel libc) + configure_windows_unwind(guest_kernel_fcntl_tests) + add_test(NAME guest_kernel_fcntl COMMAND guest_kernel_fcntl_tests) + set_tests_properties(guest_kernel_fcntl PROPERTIES TIMEOUT 15) + if(WIN32) + set_tests_properties(guest_kernel_fcntl PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_sandbox_random_word_tests ${LIBS_INCLUDE_DIR}/tests/GuestSandboxRandomWord.cpp) target_include_directories(guest_sandbox_random_word_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_sandbox_random_word_tests PRIVATE libkernel libc) diff --git a/core/libs/prx/libkernel/File/include/File.hpp b/core/libs/prx/libkernel/File/include/File.hpp index 5b067ab77..f9a2924e9 100644 --- a/core/libs/prx/libkernel/File/include/File.hpp +++ b/core/libs/prx/libkernel/File/include/File.hpp @@ -16,10 +16,13 @@ std::int64_t APS5_VABI sceKernelWrite(int d, const void* buf, std::size_t nbytes std::int64_t APS5_VABI sceKernelLseek(int d, std::int64_t offset, int whence); int APS5_VABI sceKernelStat(const char* path, FileStat* sb); int APS5_VABI sceKernelUnlink(const char* path); +int APS5_VABI sceKernelFcntl(int d, int command, ...); #ifdef _WIN32 std::int64_t NativePositioned_nid_no_patch(int descriptor, void* buf, std::size_t nbytes, std::int64_t offset, bool write); #endif } +bool DescriptorIsOpen_nid_no_patch(int descriptor); + #endif diff --git a/core/libs/prx/libkernel/File/src/Open.cpp b/core/libs/prx/libkernel/File/src/Open.cpp index 0e39397fc..d46432745 100644 --- a/core/libs/prx/libkernel/File/src/Open.cpp +++ b/core/libs/prx/libkernel/File/src/Open.cpp @@ -11,6 +11,7 @@ #include "SceTypes.hpp" #include +#include #include #include #include @@ -107,6 +108,8 @@ static int MapFlags(int sceFlags) { #endif extern "C" int* APS5_VABI __error_nid_postfix(); +extern "C" int APS5_VABI fcntl_nid_postfix(int descriptor, int command, ...); +static constexpr int GuestSetFlags = 4; static int SceErrorFromErrno(int error) { constexpr int GuestEio = 5; @@ -241,9 +244,29 @@ int APS5_VABI sceKernelUnlink(const char* path) { return 0; } -int APS5_VABI sceKernelFcntl() { - NotImplemented_nid_no_patch(__func__); - return 0; +int APS5_VABI sceKernelFcntl(int d, int command, ...) { + if (d < GuestSockets::FirstDescriptor) { + if (!DescriptorIsOpen_nid_no_patch(d)) return SCE_KERNEL_ERROR_EBADF; + throw std::runtime_error(std::string(__func__) + ": file descriptors are not implemented, fd=" + std::to_string(d)); + } + int result; + if (command == GuestSetFlags) { +#ifdef _WIN32 + __builtin_sysv_va_list arguments; + __builtin_sysv_va_start(arguments, command); + const int flags = __builtin_va_arg(arguments, int); + __builtin_sysv_va_end(arguments); +#else + std::va_list arguments; + va_start(arguments, command); + const int flags = va_arg(arguments, int); + va_end(arguments); +#endif + result = fcntl_nid_postfix(d, command, flags); + } else { + result = fcntl_nid_postfix(d, command); + } + return result < 0 ? SceKernelError(*__error_nid_postfix()) : result; } } diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index b7f3f1143..ce71bec2a 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -363,10 +363,17 @@ template static int WithoutParameterHandler(TCall call) { _set_thread_local_invalid_parameter_handler(previous); return result; } +bool DescriptorIsOpen_nid_no_patch(int descriptor) { + return descriptor >= 0 && WithoutParameterHandler([descriptor] { return ::_get_osfhandle(descriptor) == -1 ? -1 : 0; }) == 0; +} static void RejectDirectoryDuplicate(int descriptor, const char* function) { if (File::DirectoryDescriptorPath(descriptor)) throw std::runtime_error(std::string(function) + ": duplicating a directory descriptor is not supported on Windows"); } +#else +bool DescriptorIsOpen_nid_no_patch(int descriptor) { + return descriptor >= 0 && ::fcntl(descriptor, F_GETFD) >= 0; +} #endif extern "C" { diff --git a/core/libs/tests/GuestKernelFcntl.cpp b/core/libs/tests/GuestKernelFcntl.cpp new file mode 100644 index 000000000..0c80bb893 --- /dev/null +++ b/core/libs/tests/GuestKernelFcntl.cpp @@ -0,0 +1,53 @@ +#include "prx/libc/include/General.hpp" +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include +extern "C" { +int APS5_VABI socket_nid_postfix(int, int, int); +int APS5_VABI close_nid_postfix(int); +int APS5_VABI sceKernelFcntl(int, int, ...); +int APS5_VABI sceKernelOpen(const char*, int, std::uint16_t); +int APS5_VABI sceKernelClose(int); +} +static void Require(bool value) { if (!value) std::abort(); } +static constexpr int GetFlags = 3; +static constexpr int SetFlags = 4; +static constexpr int ReadWrite = 2; +static constexpr int NonBlocking = 4; +static constexpr int SceEbadf = static_cast(0x80020009u); +static constexpr int SceEinval = static_cast(0x80020016u); +static constexpr int SceEnotsup = static_cast(0x8002002du); +int main() { + const int socket = socket_nid_postfix(2, 2, 0); + Require(socket >= 0); + Require(sceKernelFcntl(socket, GetFlags) == ReadWrite); + Require(sceKernelFcntl(socket, SetFlags, ReadWrite | NonBlocking) == 0); + Require(sceKernelFcntl(socket, GetFlags) == (ReadWrite | NonBlocking)); + Require(sceKernelFcntl(socket, SetFlags, 0) == 0); + Require(sceKernelFcntl(socket, GetFlags) == ReadWrite); + Require(sceKernelFcntl(socket, SetFlags, 0x8000) == SceEnotsup); + Require(sceKernelFcntl(socket, GetFlags) == ReadWrite); + Require(sceKernelFcntl(socket, 5) == SceEinval); + Require(sceKernelFcntl(socket + 4096, GetFlags) == SceEbadf); + Require(close_nid_postfix(socket) == 0); + Require(sceKernelFcntl(socket, GetFlags) == SceEbadf); + Require(sceKernelFcntl(-1, GetFlags) == SceEbadf); + Require(sceKernelFcntl(12345, GetFlags) == SceEbadf); + const int file = sceKernelOpen("kernel_fcntl_probe.bin", 0x202, 0644); + Require(file >= 0); + bool rejected = false; + try { + sceKernelFcntl(file, GetFlags); + } catch (const std::runtime_error& error) { + rejected = std::string(error.what()).rfind("sceKernelFcntl: file descriptors are not implemented", 0) == 0; + } + Require(rejected); + Require(sceKernelClose(file) == 0); + Require(sceKernelFcntl(file, GetFlags) == SceEbadf); + std::remove("kernel_fcntl_probe.bin"); + return 0; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 90a95b787..56c7d5ea8 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -486,6 +486,7 @@ Throughout the project, every function at every stage either **does exactly what - [syslog](../../core/libs/prx/libc/src/Syslog.cpp) (libc) writes each message to `stderr` as `[syslog:] `, with a newline added when missing, instead of sending it to the console's system log. Priority and facility handling follow FreeBSD's `vsyslog`: `%m` expands to the guest errno's message, unknown bits are reported (as priority 0x23, since FreeBSD passes `LOG_ERR|LOG_CONS|LOG_PERROR|LOG_PID` as the priority) and masked, a missing facility becomes `LOG_USER`, and errno is preserved. `openlog`, `setlogmask` and `closelog` are not implemented, so the identity, options and mask they set are not applied - [sigaction](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) records the handler, flags and mask of every signal from 1 to 128 and reports them back, following FreeBSD's `sigaction` (`EINVAL` for other signals, and for a handler on `SIGKILL` or `SIGSTOP`); `signal` records its handler with `SA_RESTART`, as FreeBSD's libc does. Only handlers without `SA_SIGINFO` on the signals `signal` already delivers (`SIGINT`, `SIGILL`, `SIGABRT`, `SIGFPE`, `SIGSEGV`, `SIGTERM`) run, the same way `signal` handlers do; their mask and flags are not applied. `SA_SIGINFO` handlers and other signals (`SIGBUS`, `SIGHUP`, ...) are recorded but never called, since that needs FreeBSD's `siginfo_t` and `ucontext_t`; `raise` of an `SA_SIGINFO` handler on a signal `signal` delivers throws instead of terminating, and `raise` of the other signals still returns `EINVAL` - [kill](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) follows FreeBSD's `kern_kill` and `killpg1` for the only guest process: `EINVAL` for signals outside 0 to 128, signal 0 only checks that the target exists, and its own pid, 0 and minus its own pid (the guest is taken to lead its own process group) signal it through `raise`, so they share `raise`'s limits above. Any other pid, and -1, which skips the caller, report `ESRCH`, so `EPERM` is never reported +- [sceKernelFcntl](../../core/libs/prx/libkernel/File/src/Open.cpp) (libkernel) handles socket descriptors through the POSIX `fcntl` export (`F_GETFL`, `F_SETFL` with `O_NONBLOCK`) and returns its guest errno as `SCE_KERNEL_ERROR_*`, the form the other `sceKernel` file exports use; the command semantics follow FreeBSD's [fcntl](https://github.com/freebsd/freebsd-src/blob/stable/11/sys/kern/kern_descrip.c), and a closed descriptor returns `SCE_KERNEL_ERROR_EBADF`. File descriptors throw: their open flags are not tracked, and `F_DUPFD`, `F_GETFD` and `F_SETFD` are unsupported on all descriptors. The return values and error codes are unverified on a console - [AJM](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp), [APR](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) and [NGS2](../../core/libs/prx/libSceNgs2.native/src/System.cpp) state is allocated with `new` and never freed on purpose: title threads still call into these libraries while `std::exit` runs static destructors, so ordinary statics would be destroyed under them. Keep them leaked; `guest_ajm_teardown` covers the AJM case on Unix. - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields From 6a1301f3294d7fcf96bdf43d8719ce3805d0f73b Mon Sep 17 00:00:00 2001 From: Michon <231849@buas.nl> Date: Sun, 4 Oct 2026 19:11:06 +0200 Subject: [PATCH 177/460] feat(recompiler): FLAT addresses in the shared and private apertures reach LDS and scratch A compute FLAT access whose address has the console's shared aperture in its high dword (0x8000xxxx) reads or writes the workgroup's LDS at its low dword, and one in the private aperture (0x7000xxxx) the lane's scratch; other addresses stay guest memory through BDA. GTA V's ray traversal keeps its stack in LDS and spills it to scratch through such addresses; treated as guest memory the pushes were lost and the traversal never ended (device lost). An execution test round-trips both apertures. Co-Authored-By: Claude Opus 5.5 (1M context) --- .../tests/execution/FlatApertures.cpp | 82 ++++++++++++++++++ .../SpirvBackend/src/SpirvAnalysis.cpp | 3 + .../SpirvMemory/SpirvMemoryInstructions.cpp | 86 ++++++++++++++++++- 3 files changed, 170 insertions(+), 1 deletion(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp new file mode 100644 index 000000000..adb356602 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp @@ -0,0 +1,82 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include + + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 32; +alignas(256) std::array Output{}; + +alignas(256) constexpr std::array Code{ + 0x34020082, 0x7e0402ff, 0x80000000, 0x4a0600ff, 0x00001000, 0x7e080288, 0x7e0a02ff, 0x70000000, + 0x4a0c00ff, 0x00002000, 0xdc700000, 0x007d0301, 0xdc700000, 0x007d0604, 0xbf8c0070, 0xdc300000, + 0x077d0001, 0xdc300000, 0x087d0004, 0xbf8c0070, 0xe0701000, 0x80000701, 0xe0701080, 0x80000801, + 0xbf810000, 0xbf810000, 0xbf810000, 0xbf810000, 0xbf810000, 0xbf810000, +}; + +std::array BufferDescriptor(const void* data, std::uint32_t bytes) { + const auto address = reinterpret_cast(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x01016facu}; +} + +std::string Hex(std::uint32_t value) { + char text[16]; + std::snprintf(text, sizeof(text), "0x%08x", value); + return text; +} + +void Run(AgcDriver::VulkanDevice& device) { + Output.fill(0xdeadbeefu); + std::vector userData(4, 0u); + const auto output = BufferDescriptor(Output.data(), static_cast(Output.size() * 4u)); + std::copy(output.begin(), output.end(), userData.begin()); + const std::span code(Code); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 256u, {false, false, false}, false, 1}; + compute.scratchDwords = 4u; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); +} + +void Check() { + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + Require(Output[tid] == tid + 0x1000u, "flat apertures: lane " + std::to_string(tid) + " read " + Hex(Output[tid]) + " back through the shared aperture, expected " + Hex(tid + 0x1000u)); + Require(Output[Threads + tid] == tid + 0x2000u, "flat apertures: lane " + std::to_string(tid) + " read " + Hex(Output[Threads + tid]) + " back through the private aperture, expected " + Hex(tid + 0x2000u)); + } +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + Run(*device); + Check(); + std::puts("flat aperture tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvAnalysis.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvAnalysis.cpp index 91b74f5ec..b49a08668 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvAnalysis.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvAnalysis.cpp @@ -325,6 +325,9 @@ SpirvRequirements AnalyzeProgramRequirements(const IrProgram& program) { } requirements.functionScratch = true; } + if (program.Resources().memoryInfo.at(memoryIndex).kind == ResourceKind::Flat && program.Resources().stage == IrShaderStage::Compute && program.Info().scratchDwords != 0u) { + requirements.functionScratch = true; + } } if (BufferAccessOf(inst->Opcode()) != BufferAccess::None) { const auto memoryIndex = inst->Flags().index; diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp index 7faaadbe1..7599ba75b 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp @@ -336,11 +336,68 @@ const MemoryInfo& SharedMemory(SpirvValueEmitContext& ctx, const IrValue& inst) return mem; } +constexpr std::uint32_t SharedApertureTop = 0x8000u; +constexpr std::uint32_t PrivateApertureTop = 0x7000u; + +bool RoutesApertures(const SpirvEmitterState& state, const MemoryInfo& mem) { + return mem.kind == ResourceKind::Flat && state.program.Resources().stage == IrShaderStage::Compute; +} + +struct ApertureAddress { + std::uint32_t low = 0; + std::uint32_t shared = 0; + std::uint32_t priv = 0; +}; + +ApertureAddress SplitAperture(SpirvEmitterState& state, std::uint32_t address) { + const auto u32 = TypeU32(state); + const auto high = Unary(state, spv::OpUConvert, u32, Binary(state, spv::OpShiftRightLogical, TypeScalarU64(state), address, BdaConstant(state, 32u))); + const auto top = Binary(state, spv::OpShiftRightLogical, u32, high, ConstantU32(state, 16u)); + return {Unary(state, spv::OpUConvert, u32, address), Binary(state, spv::OpIEqual, TypeBool(state), top, ConstantU32(state, SharedApertureTop)), Binary(state, spv::OpIEqual, TypeBool(state), top, ConstantU32(state, PrivateApertureTop))}; +} + +std::uint32_t LoadApertureElement(SpirvValueEmitContext& ctx, ResourceKind kind, std::uint32_t byteOffset, std::uint32_t bits) { + auto& state = ctx.state; + if (kind == ResourceKind::Scratch && state.program.Info().scratchDwords == 0u) return ConstantU32(state, 0u); + MemoryInfo storage{}; + storage.kind = kind; + const auto resource = PrepareMemoryResourceAccess(state, storage); + const auto index = Binary(state, spv::OpShiftRightLogical, TypeU32(state), byteOffset, ConstantU32(state, 2u)); + return EmitValueOrZeroIfCondition(state, EmitMemoryElementInBounds(state, resource, index), [&] { + return bits == 32u ? LoadWordInBounds(ctx, resource, index) : LoadSubwordInBounds(ctx, resource, byteOffset, index, bits, false); + }); +} + +void StoreApertureWord(SpirvValueEmitContext& ctx, ResourceKind kind, std::uint32_t byteOffset, std::uint32_t data) { + auto& state = ctx.state; + if (kind == ResourceKind::Scratch && state.program.Info().scratchDwords == 0u) return; + MemoryInfo storage{}; + storage.kind = kind; + const auto resource = PrepareMemoryResourceAccess(state, storage); + const auto index = Binary(state, spv::OpShiftRightLogical, TypeU32(state), byteOffset, ConstantU32(state, 2u)); + EmitIfCondition(state, EmitMemoryElementInBounds(state, resource, index), [&] { StoreWordInBounds(ctx, resource, index, data); }); +} + +template +std::uint32_t RouteFlatLoad(SpirvEmitterState& state, std::uint32_t address, std::uint32_t type, TAperture&& aperture, TGlobal&& global) { + const auto split = SplitAperture(state, address); + return EmitValueIfElse(state, split.shared, type, [&] { return aperture(ResourceKind::Lds, split.low); }, [&] { + return EmitValueIfElse(state, split.priv, type, [&] { return aperture(ResourceKind::Scratch, split.low); }, global); + }); +} + void LoadAddress(SpirvValueEmitContext& ctx, const IrValue& inst, std::uint32_t bits) { const auto& mem = ctx.Memory(inst); if (bits == 32u && mem.planningOnly) { return; } + if (RoutesApertures(ctx.state, mem)) { + ctx.Define(inst, EmitValueOrZeroIfCondition(ctx.state, ActiveArgument(ctx, inst), [&] { + const auto address = GuestAddress(ctx, inst, mem); + return RouteFlatLoad(ctx.state, address, TypeU32(ctx.state), [&](ResourceKind kind, std::uint32_t low) { return LoadApertureElement(ctx, kind, low, bits); }, [&] { return EmitBdaRead(ctx, inst, address, bits); }); + })); + return; + } switch (mem.kind) { case ResourceKind::Scratch: ctx.Define(inst, bits == 32u ? LoadWord(ctx, inst, mem) : LoadSubword(ctx, inst, mem, bits)); @@ -367,6 +424,22 @@ void LoadAddressWide(SpirvValueEmitContext& ctx, const IrValue& inst, std::uint3 if (mem.planningOnly) { return; } + if (RoutesApertures(ctx.state, mem)) { + auto& state = ctx.state; + const auto type = TypeU32Composite(state, components); + ctx.Define(inst, EmitValueOrDefaultIfCondition(state, ActiveArgument(ctx, inst), type, ConstantU32CompositeZero(state, components), [&] { + const auto base = GuestAddressBase(ctx, inst, mem); + const auto address = AddBdaImmediate(ctx, inst, base, static_cast(mem.offset)); + return RouteFlatLoad(state, address, type, [&](ResourceKind kind, std::uint32_t low) { + std::array values{}; + for (std::uint32_t component = 0; component < components; component++) { + values[component] = LoadApertureElement(ctx, kind, Binary(state, spv::OpIAdd, TypeU32(state), low, ConstantU32(state, component * 4u)), 32u); + } + return ConstructU32Composite(state, components, values); + }, [&] { return ConstructU32Composite(state, components, EmitBdaDwordReads(ctx, inst, base, mem.offset, components)); }); + })); + return; + } switch (mem.kind) { case ResourceKind::Flat: case ResourceKind::Global: @@ -390,7 +463,18 @@ void StoreAddress(SpirvValueEmitContext& ctx, const IrValue& inst, std::uint32_t case ResourceKind::Flat: case ResourceKind::Global: EmitIfCondition(ctx.state, ActiveArgument(ctx, inst), [&]() { - EmitBdaStore(ctx, inst, GuestAddress(ctx, inst, mem), ctx.Arg(inst, inst.ArgumentCount() - 2u), bits); + auto& state = ctx.state; + const auto address = GuestAddress(ctx, inst, mem); + const auto data = ctx.Arg(inst, inst.ArgumentCount() - 2u); + if (bits != 32u || !RoutesApertures(state, mem)) { + EmitBdaStore(ctx, inst, address, data, bits); + return; + } + const auto split = SplitAperture(state, address); + EmitIfCondition(state, split.shared, [&] { StoreApertureWord(ctx, ResourceKind::Lds, split.low, data); }); + EmitIfCondition(state, split.priv, [&] { StoreApertureWord(ctx, ResourceKind::Scratch, split.low, data); }); + const auto global = Unary(state, spv::OpLogicalNot, TypeBool(state), Binary(state, spv::OpLogicalOr, TypeBool(state), split.shared, split.priv)); + EmitIfCondition(state, global, [&] { EmitBdaStore(ctx, inst, address, data, bits); }); }); return; default: From c35f29b6be811571b84ebe0fe9910b23e29b5a03 Mon Sep 17 00:00:00 2001 From: Michon <231849@buas.nl> Date: Thu, 8 Oct 2026 18:08:17 +0200 Subject: [PATCH 178/460] fix(recompiler): every FLAT access kind routes the shared and private apertures; table test Sub-dword stores and unaligned dword accesses go byte-wise; 32- and 64-bit FLAT atomics route to LDS (64-bit under the workgroup lock), scratch (read-modify-write) or BDA. agc_driver_flat_apertures covers them in 121 rows, with mixed-address waves and wave64. The aperture bases and the unmodelled operands 235-238 are listed as inferred. Co-Authored-By: Claude Opus 5.5 (1M context) --- .../tests/execution/FlatApertures.cpp | 468 ++++++++++++++++-- .../SpirvBackend/src/SpirvAnalysis.cpp | 4 + .../SpirvMemory/SpirvMemoryInstructions.cpp | 259 +++++++++- docs/dev/TechnicalDebt.md | 3 + 4 files changed, 695 insertions(+), 39 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp index adb356602..d73947d4d 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp @@ -1,13 +1,27 @@ #include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" #include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "prx/libc/include/GuestAllocations.hpp" #include "Recompiler.hpp" #include "VulkanTestDevice.hpp" +#ifdef _WIN32 +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include +#endif #include #include #include #include +#include +#include +#include #include +#include +#include +#include #include +#include #include @@ -15,15 +29,69 @@ namespace { using AgcDriver::Graphics::Require; using ShaderRecompiler::ShaderStage; +using Words = std::vector; -constexpr std::uint32_t Threads = 32; -alignas(256) std::array Output{}; +constexpr std::uint32_t SharedHigh = 0x80000000u; +constexpr std::uint32_t PrivateHigh = 0x70000000u; +constexpr std::uint32_t LdsDwords = 256u; +constexpr std::uint32_t ScratchDwords = 4u; +constexpr std::size_t GuestBytes = 65536u; +constexpr std::uint32_t GuestStoreSlot = 0x200u; +constexpr std::uint32_t GuestAtomicSlot = 0x800u; +constexpr std::uint32_t WaitCnt0 = 0xbf8c0070u; +constexpr std::uint32_t EndProgram = 0xbf810000u; +constexpr std::uint32_t ShiftLaneBy5 = 0x34020085u; +constexpr std::uint32_t StoreByteOp = 0x18u; +constexpr std::uint32_t StoreShortOp = 0x1au; +constexpr std::uint32_t StoreDwordOp = 0x1cu; +constexpr std::uint32_t StoreDwordx2Op = 0x1du; +constexpr std::uint32_t LoadUbyteOp = 0x08u; +constexpr std::uint32_t LoadUshortOp = 0x0au; +constexpr std::uint32_t LoadDwordOp = 0x0cu; +constexpr std::uint32_t LoadDwordx2Op = 0x0du; -alignas(256) constexpr std::array Code{ - 0x34020082, 0x7e0402ff, 0x80000000, 0x4a0600ff, 0x00001000, 0x7e080288, 0x7e0a02ff, 0x70000000, - 0x4a0c00ff, 0x00002000, 0xdc700000, 0x007d0301, 0xdc700000, 0x007d0604, 0xbf8c0070, 0xdc300000, - 0x077d0001, 0xdc300000, 0x087d0004, 0xbf8c0070, 0xe0701000, 0x80000701, 0xe0701080, 0x80000801, - 0xbf810000, 0xbf810000, 0xbf810000, 0xbf810000, 0xbf810000, 0xbf810000, +enum class Target : std::uint8_t { Shared, Private, Guest, Mixed }; +enum class AtomicOp : std::uint8_t { Swap, Cmpswap, Add, Sub, Smin, Umin, Smax, Umax, And, Or, Xor, Inc, Dec, Fcmpswap, Fmin, Fmax }; + +constexpr std::size_t Atomic32Ops = 13; +constexpr std::array AtomicOpcodes32{0x30u, 0x31u, 0x32u, 0x33u, 0x35u, 0x36u, 0x37u, 0x38u, 0x39u, 0x3au, 0x3bu, 0x3cu, 0x3du}; +constexpr std::array AtomicOpcodes64{0x50u, 0x51u, 0x52u, 0x53u, 0x55u, 0x56u, 0x57u, 0x58u, 0x59u, 0x5au, 0x5bu, 0x5cu, 0x5du, 0x5eu, 0x5fu, 0x60u}; +constexpr std::array TargetNames{"shared", "private", "guest", "mixed"}; +constexpr std::array OpNames{"swap", "cmpswap", "add", "sub", "smin", "umin", "smax", "umax", "and", "or", "xor", "inc", "dec", "fcmpswap", "fmin", "fmax"}; + +struct Row { + Target target; + bool atomic; + AtomicOp op; + std::uint32_t width; + std::uint32_t offset; + std::uint32_t waveSize; +}; + +class GuestBlock { +public: + GuestBlock() { +#ifdef _WIN32 + block = static_cast(VirtualAlloc(nullptr, GuestBytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); +#else + block = static_cast(std::aligned_alloc(GuestBytes, GuestBytes)); +#endif + Require(block != nullptr, "flat apertures: cannot allocate the guest block"); + std::memset(block, 0, GuestBytes); + GuestAllocations::Mutation().Add(block, GuestBytes, true, true); + } + + ~GuestBlock() { + GuestAllocations::Mutation().Remove(block); + } + + GuestBlock(const GuestBlock&) = delete; + GuestBlock& operator=(const GuestBlock&) = delete; + + std::uintptr_t Address() const { return reinterpret_cast(block); } + +private: + std::uint8_t* block = nullptr; }; std::array BufferDescriptor(const void* data, std::uint32_t bytes) { @@ -37,32 +105,368 @@ std::string Hex(std::uint32_t value) { return text; } -void Run(AgcDriver::VulkanDevice& device) { - Output.fill(0xdeadbeefu); - std::vector userData(4, 0u); - const auto output = BufferDescriptor(Output.data(), static_cast(Output.size() * 4u)); - std::copy(output.begin(), output.end(), userData.begin()); - const std::span code(Code); - const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; - ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 256u, {false, false, false}, false, 1}; - compute.scratchDwords = 4u; - ShaderRecompiler::RecompileRequest request{ +std::string Hex64(std::uint64_t value) { + char text[24]; + std::snprintf(text, sizeof(text), "0x%016llx", static_cast(value)); + return text; +} + +std::uint32_t FlatWord0(std::uint32_t op, bool glc, std::uint32_t offset = 0u) { + return 0xdc000000u | (op << 18u) | (glc ? 0x10000u : 0u) | offset; +} + +std::uint32_t FlatWord1(std::uint32_t vdst, std::uint32_t data, std::uint32_t address) { + return (vdst << 24u) | (0x7du << 16u) | (data << 8u) | address; +} + +void EmitFlat(Words& code, std::uint32_t op, bool glc, std::uint32_t vdst, std::uint32_t data, std::uint32_t address) { + code.push_back(FlatWord0(op, glc)); + code.push_back(FlatWord1(vdst, data, address)); +} + +void EmitTableLoad(Words& code, std::uint32_t vdst, std::uint32_t offset) { + code.push_back(0xe0381000u | offset); + code.push_back(0x80010001u | (vdst << 8u)); +} + +void EmitOutputStore(Words& code, std::uint32_t vdata, std::uint32_t offset) { + code.push_back(0xe0701000u | offset); + code.push_back(0x80000001u | (vdata << 8u)); +} + +std::uint32_t StoreOp(std::uint32_t width) { + return width == 8u ? StoreByteOp : (width == 16u ? StoreShortOp : StoreDwordOp); +} + +std::uint32_t LoadOp(std::uint32_t width) { + return width == 8u ? LoadUbyteOp : (width == 16u ? LoadUshortOp : LoadDwordOp); +} + +Words Kernel32(const Row& row) { + Words code{ShiftLaneBy5}; + EmitTableLoad(code, 2u, 0u); + EmitTableLoad(code, 6u, 16u); + code.push_back(WaitCnt0); + if (row.atomic) { + EmitFlat(code, StoreDwordOp, false, 0u, 6u, 2u); + code.push_back(WaitCnt0); + EmitFlat(code, AtomicOpcodes32[static_cast(row.op)], true, 10u, 4u, 2u); + code.push_back(WaitCnt0); + EmitFlat(code, LoadDwordOp, false, 11u, 0u, 2u); + code.push_back(WaitCnt0); + EmitOutputStore(code, 10u, 0u); + EmitOutputStore(code, 11u, 4u); + } else { + EmitFlat(code, StoreOp(row.width), false, 0u, 4u, 2u); + code.push_back(WaitCnt0); + EmitFlat(code, LoadOp(row.width), false, 10u, 0u, 2u); + EmitFlat(code, LoadUbyteOp, false, 11u, 0u, 2u); + code.push_back(FlatWord0(LoadUbyteOp, false, row.width / 8u - 1u)); + code.push_back(FlatWord1(12u, 0u, 2u)); + code.push_back(WaitCnt0); + EmitOutputStore(code, 10u, 0u); + EmitOutputStore(code, 11u, 4u); + EmitOutputStore(code, 12u, 8u); + } + code.push_back(EndProgram); + return code; +} + +Words Kernel64(const Row& row) { + Words code{ShiftLaneBy5}; + EmitTableLoad(code, 2u, 0u); + EmitTableLoad(code, 6u, 16u); + code.push_back(WaitCnt0); + EmitFlat(code, StoreDwordx2Op, false, 0u, 8u, 2u); + code.push_back(WaitCnt0); + EmitFlat(code, AtomicOpcodes64[static_cast(row.op)], true, 10u, 4u, 2u); + code.push_back(WaitCnt0); + EmitFlat(code, LoadDwordx2Op, false, 12u, 0u, 2u); + code.push_back(WaitCnt0); + EmitOutputStore(code, 10u, 0u); + EmitOutputStore(code, 11u, 4u); + EmitOutputStore(code, 12u, 8u); + EmitOutputStore(code, 13u, 12u); + code.push_back(EndProgram); + return code; +} + +Words Kernel(const Row& row) { + return row.atomic && row.width == 64u ? Kernel64(row) : Kernel32(row); +} + +std::uint32_t OperandA(std::uint32_t lane) { + return 0x9e3779b1u * (lane + 1u) ^ 0x5a5a0f0fu; +} + +std::uint32_t InitValue(std::uint32_t lane) { + return (lane % 2u) != 0u ? (0x80000000u | (lane * 0x101u)) : (lane * 0x35u + 7u); +} + +std::uint32_t CmpValue(std::uint32_t lane) { + return (lane % 2u) != 0u ? InitValue(lane) + 1u : InitValue(lane); +} + +std::uint32_t Apply(AtomicOp op, std::uint32_t init, std::uint32_t a, std::uint32_t b) { + const auto signedInit = static_cast(init); + const auto signedA = static_cast(a); + switch (op) { + case AtomicOp::Swap: return a; + case AtomicOp::Cmpswap: return init == b ? a : init; + case AtomicOp::Add: return init + a; + case AtomicOp::Sub: return init - a; + case AtomicOp::Smin: return signedInit < signedA ? init : a; + case AtomicOp::Umin: return init < a ? init : a; + case AtomicOp::Smax: return signedInit > signedA ? init : a; + case AtomicOp::Umax: return init > a ? init : a; + case AtomicOp::And: return init & a; + case AtomicOp::Or: return init | a; + case AtomicOp::Xor: return init ^ a; + case AtomicOp::Inc: return init >= a ? 0u : init + 1u; + case AtomicOp::Dec: return (init == 0u || init > a) ? a : init - 1u; + default: break; + } + return init; +} + +std::uint64_t Pack(std::uint32_t high, std::uint32_t low) { + return (static_cast(high) << 32u) | low; +} + +std::uint64_t Bits(double value) { + std::uint64_t bits = 0; + std::memcpy(&bits, &value, sizeof(bits)); + return bits; +} + +bool IsFloatOp(AtomicOp op) { + return op == AtomicOp::Fcmpswap || op == AtomicOp::Fmin || op == AtomicOp::Fmax; +} + +std::uint64_t IntInit64(std::uint32_t lane) { + return Pack((lane % 2u) != 0u ? (0x80000000u | (lane * 0x1357u)) : (lane * 0x0a0bu + 0x11u), InitValue(lane)); +} + +std::uint64_t IntA64(std::uint32_t lane) { + return Pack(0x85ebca6bu * (lane + 1u) ^ 0xc2b2ae35u, OperandA(lane)); +} + +std::uint64_t IntCmp64(std::uint32_t lane) { + return (lane % 2u) != 0u ? IntInit64(lane) + 1u : IntInit64(lane); +} + +double FloatInit(std::uint32_t lane) { + return (lane % 2u) == 0u ? static_cast(lane + 1u) * 3.25 : -static_cast(lane + 1u) * 7.5; +} + +double FloatA(std::uint32_t lane) { + return (lane % 2u) == 0u ? -static_cast(lane + 2u) * 1.5 : static_cast(lane + 3u) * 2.75 - 2.0; +} + +double FloatCmp(std::uint32_t lane) { + return (lane % 2u) == 0u ? FloatInit(lane) : FloatInit(lane) + 1.0; +} + +std::uint64_t InitBits64(AtomicOp op, std::uint32_t lane) { + return IsFloatOp(op) ? Bits(FloatInit(lane)) : IntInit64(lane); +} + +std::uint64_t OperandBits64(AtomicOp op, std::uint32_t lane) { + return IsFloatOp(op) ? Bits(FloatA(lane)) : IntA64(lane); +} + +std::uint64_t CmpBits64(AtomicOp op, std::uint32_t lane) { + return IsFloatOp(op) ? Bits(FloatCmp(lane)) : IntCmp64(lane); +} + +std::uint64_t Apply64(AtomicOp op, std::uint64_t init, std::uint64_t a, std::uint64_t cmp) { + const auto signedInit = static_cast(init); + const auto signedA = static_cast(a); + switch (op) { + case AtomicOp::Swap: return a; + case AtomicOp::Cmpswap: return init == cmp ? a : init; + case AtomicOp::Add: return init + a; + case AtomicOp::Sub: return init - a; + case AtomicOp::Smin: return signedInit < signedA ? init : a; + case AtomicOp::Umin: return init < a ? init : a; + case AtomicOp::Smax: return signedInit > signedA ? init : a; + case AtomicOp::Umax: return init > a ? init : a; + case AtomicOp::And: return init & a; + case AtomicOp::Or: return init | a; + case AtomicOp::Xor: return init ^ a; + case AtomicOp::Inc: return init >= a ? 0u : init + 1u; + case AtomicOp::Dec: return (init == 0u || init > a) ? a : init - 1u; + default: break; + } + return init; +} + +std::uint64_t Expected64(AtomicOp op, std::uint32_t lane) { + if (IsFloatOp(op)) { + const auto init = FloatInit(lane); + const auto a = FloatA(lane); + switch (op) { + case AtomicOp::Fcmpswap: return Bits(init == FloatCmp(lane) ? a : init); + case AtomicOp::Fmin: return Bits(std::min(init, a)); + case AtomicOp::Fmax: return Bits(std::max(init, a)); + default: break; + } + } + return Apply64(op, IntInit64(lane), IntA64(lane), IntCmp64(lane)); +} + +std::uint64_t LaneAddress(const Row& row, std::uint32_t lane, std::uint64_t guestBase) { + Target target = row.target; + if (target == Target::Mixed) { + target = lane % 3u == 0u ? Target::Shared : (lane % 3u == 1u ? Target::Private : Target::Guest); + } + const std::uint32_t privateBase = row.width == 64u ? 8u : 4u; + switch (target) { + case Target::Shared: return (static_cast(SharedHigh) << 32u) | (8u * lane + row.offset); + case Target::Private: return (static_cast(PrivateHigh) << 32u) | (privateBase + row.offset); + case Target::Guest: return guestBase + 8u * lane + row.offset; + default: break; + } + throw std::runtime_error("flat apertures: no address for the row target"); +} + +std::string Describe(const Row& row) { + std::string access; + if (row.atomic) { + access = std::string(row.width == 64u ? "flat_atomic64_" : "flat_atomic_") + OpNames[static_cast(row.op)]; + } else { + access = "flat_store" + std::to_string(row.width); + } + return access + " " + TargetNames[static_cast(row.target)] + " offset " + std::to_string(row.offset) + " wave" + std::to_string(row.waveSize); +} + +void Check(const Row& row, const std::vector& output) { + const std::string name = Describe(row); + for (std::uint32_t lane = 0; lane < row.waveSize; ++lane) { + const auto a = OperandA(lane); + if (row.atomic) { + const auto init = InitValue(lane); + const auto old = output[lane * 8u]; + Require(old == init, name + ": lane " + std::to_string(lane) + " returned " + Hex(old) + ", expected " + Hex(init)); + const auto expected = Apply(row.op, init, a, CmpValue(lane)); + const auto left = output[lane * 8u + 1u]; + Require(left == expected, name + ": lane " + std::to_string(lane) + " left " + Hex(left) + ", expected " + Hex(expected)); + } else { + const auto expected = row.width == 8u ? (a & 0xffu) : (row.width == 16u ? (a & 0xffffu) : a); + const auto read = output[lane * 8u]; + Require(read == expected, name + ": lane " + std::to_string(lane) + " read " + Hex(read) + " back, expected " + Hex(expected)); + const auto first = output[lane * 8u + 1u]; + Require(first == (a & 0xffu), name + ": lane " + std::to_string(lane) + " has first byte " + Hex(first) + " in memory, expected " + Hex(a & 0xffu)); + const auto last = output[lane * 8u + 2u]; + const auto lastExpected = (a >> (8u * (row.width / 8u - 1u))) & 0xffu; + Require(last == lastExpected, name + ": lane " + std::to_string(lane) + " has last byte " + Hex(last) + " in memory, expected " + Hex(lastExpected)); + } + } +} + +void Check64(const Row& row, const std::vector& output) { + const std::string name = Describe(row); + for (std::uint32_t lane = 0; lane < row.waveSize; ++lane) { + const auto old = Pack(output[lane * 8u + 1u], output[lane * 8u]); + const auto init = InitBits64(row.op, lane); + Require(old == init, name + ": lane " + std::to_string(lane) + " returned " + Hex64(old) + ", expected " + Hex64(init)); + const auto left = Pack(output[lane * 8u + 3u], output[lane * 8u + 2u]); + const auto expected = Expected64(row.op, lane); + Require(left == expected, name + ": lane " + std::to_string(lane) + " left " + Hex64(left) + ", expected " + Hex64(expected)); + } +} + +ShaderRecompiler::RecompileRequest MakeRequest(AgcDriver::VulkanDevice& device, std::span code, const std::array& memory, std::vector& userData, const ShaderRecompiler::ShaderComputeStageInfo& compute, std::uint32_t waveSize) { + return ShaderRecompiler::RecompileRequest{ {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, - {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + {waveSize, 0, userData, compute, std::nullopt, std::nullopt, memory}, device.Target(), {0, 0, 0, 128} }; - request.useCache = false; - const auto result = ShaderRecompiler::Recompile(request); - device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); - device.WaitIdle(); } -void Check() { - for (std::uint32_t tid = 0; tid < Threads; ++tid) { - Require(Output[tid] == tid + 0x1000u, "flat apertures: lane " + std::to_string(tid) + " read " + Hex(Output[tid]) + " back through the shared aperture, expected " + Hex(tid + 0x1000u)); - Require(Output[Threads + tid] == tid + 0x2000u, "flat apertures: lane " + std::to_string(tid) + " read " + Hex(Output[Threads + tid]) + " back through the private aperture, expected " + Hex(tid + 0x2000u)); +void Run(AgcDriver::VulkanDevice& device, const Row& row, const GuestBlock& guest) { + const auto lanes = row.waveSize; + const auto guestSlot = row.atomic ? GuestAtomicSlot : GuestStoreSlot; + const auto guestBase = static_cast(guest.Address()) + guestSlot; + const bool wideAtomic = row.atomic && row.width == 64u; + std::vector table(lanes * 8u, 0u); + for (std::uint32_t lane = 0; lane < lanes; ++lane) { + const auto address = LaneAddress(row, lane, guestBase); + table[lane * 8u] = static_cast(address); + table[lane * 8u + 1u] = static_cast(address >> 32u); + if (wideAtomic) { + const auto operand = OperandBits64(row.op, lane); + const auto cmp = CmpBits64(row.op, lane); + const auto init = InitBits64(row.op, lane); + table[lane * 8u + 2u] = static_cast(operand); + table[lane * 8u + 3u] = static_cast(operand >> 32u); + table[lane * 8u + 4u] = static_cast(cmp); + table[lane * 8u + 5u] = static_cast(cmp >> 32u); + table[lane * 8u + 6u] = static_cast(init); + table[lane * 8u + 7u] = static_cast(init >> 32u); + } else { + table[lane * 8u + 2u] = OperandA(lane); + table[lane * 8u + 3u] = row.atomic ? CmpValue(lane) : 0u; + table[lane * 8u + 4u] = row.atomic ? InitValue(lane) : 0u; + } } + std::vector output(lanes * 8u, 0xdeadbeefu); + std::vector userData(8, 0u); + const auto outputDescriptor = BufferDescriptor(output.data(), static_cast(output.size() * 4u)); + const auto tableDescriptor = BufferDescriptor(table.data(), static_cast(table.size() * 4u)); + std::copy(outputDescriptor.begin(), outputDescriptor.end(), userData.begin()); + std::copy(tableDescriptor.begin(), tableDescriptor.end(), userData.begin() + 4); + const auto code = Kernel(row); + const std::span codeSpan(code); + const std::array memory{{{reinterpret_cast(codeSpan.data()), std::as_bytes(codeSpan)}}}; + ShaderRecompiler::ShaderComputeStageInfo compute{{lanes, 1, 1}, LdsDwords, {false, false, false}, false, 1}; + compute.scratchDwords = ScratchDwords; + auto request = MakeRequest(device, codeSpan, memory, userData, compute, row.waveSize); + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(codeSpan.data())); + device.WaitIdle(); + if (wideAtomic) { + Check64(row, output); + } else { + Check(row, output); + } +} + +std::vector Rows() { + std::vector rows; + const std::array single{Target::Shared, Target::Private, Target::Guest}; + const std::array, 7> stores{{{8u, 0u}, {8u, 3u}, {16u, 0u}, {16u, 1u}, {16u, 3u}, {32u, 0u}, {32u, 2u}}}; + for (const Target target : single) { + for (const auto [width, offset] : stores) { + rows.push_back(Row{target, false, AtomicOp::Swap, width, offset, 32u}); + } + } + rows.push_back(Row{Target::Mixed, false, AtomicOp::Swap, 16u, 3u, 32u}); + rows.push_back(Row{Target::Mixed, false, AtomicOp::Swap, 32u, 2u, 32u}); + for (const Target target : single) { + for (std::uint32_t op = 0; op < Atomic32Ops; ++op) { + rows.push_back(Row{target, true, static_cast(op), 32u, 0u, 32u}); + } + } + for (const Target target : single) { + for (std::uint32_t op = 0; op < OpNames.size(); ++op) { + rows.push_back(Row{target, true, static_cast(op), 64u, 0u, 32u}); + } + } + rows.push_back(Row{Target::Mixed, true, AtomicOp::Add, 32u, 0u, 32u}); + rows.push_back(Row{Target::Mixed, true, AtomicOp::Cmpswap, 32u, 0u, 32u}); + rows.push_back(Row{Target::Mixed, true, AtomicOp::Add, 64u, 0u, 32u}); + rows.push_back(Row{Target::Mixed, true, AtomicOp::Cmpswap, 64u, 0u, 32u}); + rows.push_back(Row{Target::Mixed, true, AtomicOp::Fmax, 64u, 0u, 32u}); + rows.push_back(Row{Target::Mixed, false, AtomicOp::Swap, 16u, 3u, 64u}); + rows.push_back(Row{Target::Mixed, false, AtomicOp::Swap, 32u, 2u, 64u}); + rows.push_back(Row{Target::Mixed, true, AtomicOp::Add, 32u, 0u, 64u}); + rows.push_back(Row{Target::Shared, true, AtomicOp::Cmpswap, 32u, 0u, 64u}); + rows.push_back(Row{Target::Mixed, true, AtomicOp::Add, 64u, 0u, 64u}); + rows.push_back(Row{Target::Shared, true, AtomicOp::Fmin, 64u, 0u, 64u}); + return rows; } } @@ -71,8 +475,16 @@ int main() { try { const auto device = OpenVulkanTestDevice(); if (!device) return VulkanTestSkipped; - Run(*device); - Check(); + const bool wave64 = device->Target().subgroupSize >= 32u; + GuestBlock guest; + std::size_t checked = 0; + for (const Row& row : Rows()) { + if (row.waveSize == 64u && !wave64) continue; + Run(*device, row, guest); + ++checked; + } + if (!wave64) std::printf("skipped the wave64 rows, subgroup size %u cannot hold a wave64 in two lanes\n", device->Target().subgroupSize); + std::printf("flat aperture rows checked: %zu\n", checked); std::puts("flat aperture tests passed"); return 0; } catch (const std::exception& error) { diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvAnalysis.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvAnalysis.cpp index b49a08668..8c6d97555 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvAnalysis.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvAnalysis.cpp @@ -328,6 +328,10 @@ SpirvRequirements AnalyzeProgramRequirements(const IrProgram& program) { if (program.Resources().memoryInfo.at(memoryIndex).kind == ResourceKind::Flat && program.Resources().stage == IrShaderStage::Compute && program.Info().scratchDwords != 0u) { requirements.functionScratch = true; } + if (program.Resources().memoryInfo.at(memoryIndex).kind == ResourceKind::Flat && program.Resources().stage == IrShaderStage::Compute && addressAccess == AddressAccess::Atomic && inst->Type() == IrType::U64) { + requirements.ldsLock = true; + requirements.subgroupBallot = true; + } } if (BufferAccessOf(inst->Opcode()) != BufferAccess::None) { const auto memoryIndex = inst->Flags().index; diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp index 7599ba75b..3c96d5066 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp @@ -356,7 +356,17 @@ ApertureAddress SplitAperture(SpirvEmitterState& state, std::uint32_t address) { return {Unary(state, spv::OpUConvert, u32, address), Binary(state, spv::OpIEqual, TypeBool(state), top, ConstantU32(state, SharedApertureTop)), Binary(state, spv::OpIEqual, TypeBool(state), top, ConstantU32(state, PrivateApertureTop))}; } -std::uint32_t LoadApertureElement(SpirvValueEmitContext& ctx, ResourceKind kind, std::uint32_t byteOffset, std::uint32_t bits) { +std::uint32_t ApertureByteInBounds(SpirvEmitterState& state, const MemoryResourceAccess& resource, std::uint32_t byteOffset, std::uint32_t offset, std::uint32_t index) { + const auto noWrap = Binary(state, spv::OpULessThanEqual, TypeBool(state), byteOffset, ConstantU32(state, 0xffffffffu - offset)); + return AndCondition(state, noWrap, EmitMemoryElementInBounds(state, resource, index)); +} + +std::uint32_t ApertureContained(SpirvEmitterState& state, std::uint32_t byteOffset, std::uint32_t bits) { + const auto inDword = Binary(state, spv::OpBitwiseAnd, TypeU32(state), byteOffset, ConstantU32(state, 3u)); + return Binary(state, spv::OpULessThanEqual, TypeBool(state), EmitAddU32(state, inDword, ConstantU32(state, bits / 8u)), ConstantU32(state, 4u)); +} + +std::uint32_t LoadApertureDword(SpirvValueEmitContext& ctx, ResourceKind kind, std::uint32_t byteOffset, std::uint32_t bits) { auto& state = ctx.state; if (kind == ResourceKind::Scratch && state.program.Info().scratchDwords == 0u) return ConstantU32(state, 0u); MemoryInfo storage{}; @@ -368,18 +378,78 @@ std::uint32_t LoadApertureElement(SpirvValueEmitContext& ctx, ResourceKind kind, }); } -void StoreApertureWord(SpirvValueEmitContext& ctx, ResourceKind kind, std::uint32_t byteOffset, std::uint32_t data) { +std::uint32_t LoadApertureByte(SpirvValueEmitContext& ctx, ResourceKind kind, std::uint32_t byteOffset, std::uint32_t offset) { + auto& state = ctx.state; + if (kind == ResourceKind::Scratch && state.program.Info().scratchDwords == 0u) return ConstantU32(state, 0u); + MemoryInfo storage{}; + storage.kind = kind; + const auto resource = PrepareMemoryResourceAccess(state, storage); + const auto address = EmitAddU32(state, byteOffset, ConstantU32(state, offset)); + const auto index = Binary(state, spv::OpShiftRightLogical, TypeU32(state), address, ConstantU32(state, 2u)); + return EmitValueOrZeroIfCondition(state, ApertureByteInBounds(state, resource, byteOffset, offset, index), [&] { + return LoadSubwordInBounds(ctx, resource, address, index, 8u, false); + }); +} + +std::uint32_t LoadApertureElement(SpirvValueEmitContext& ctx, ResourceKind kind, std::uint32_t byteOffset, std::uint32_t bits) { + auto& state = ctx.state; + if (bits == 8u) return LoadApertureByte(ctx, kind, byteOffset, 0u); + return EmitValueIfElse(state, ApertureContained(state, byteOffset, bits), TypeU32(state), [&] { + return LoadApertureDword(ctx, kind, byteOffset, bits); + }, [&] { + std::uint32_t value = ConstantU32(state, 0u); + for (std::uint32_t byte = 0; byte < bits / 8u; ++byte) { + const auto part = Binary(state, spv::OpShiftLeftLogical, TypeU32(state), LoadApertureByte(ctx, kind, byteOffset, byte), ConstantU32(state, byte * 8u)); + value = Binary(state, spv::OpBitwiseOr, TypeU32(state), value, part); + } + return value; + }); +} + +void StoreApertureByte(SpirvValueEmitContext& ctx, ResourceKind kind, std::uint32_t byteOffset, std::uint32_t offset, std::uint32_t data) { + auto& state = ctx.state; + if (kind == ResourceKind::Scratch && state.program.Info().scratchDwords == 0u) return; + MemoryInfo storage{}; + storage.kind = kind; + const auto resource = PrepareMemoryResourceAccess(state, storage); + const auto address = EmitAddU32(state, byteOffset, ConstantU32(state, offset)); + const auto index = Binary(state, spv::OpShiftRightLogical, TypeU32(state), address, ConstantU32(state, 2u)); + EmitIfCondition(state, ApertureByteInBounds(state, resource, byteOffset, offset, index), [&] { StoreSubwordInBounds(ctx, storage, resource, address, index, 8u, data); }); +} + +void StoreApertureDword(SpirvValueEmitContext& ctx, ResourceKind kind, std::uint32_t byteOffset, std::uint32_t bits, std::uint32_t data) { auto& state = ctx.state; if (kind == ResourceKind::Scratch && state.program.Info().scratchDwords == 0u) return; MemoryInfo storage{}; storage.kind = kind; const auto resource = PrepareMemoryResourceAccess(state, storage); const auto index = Binary(state, spv::OpShiftRightLogical, TypeU32(state), byteOffset, ConstantU32(state, 2u)); - EmitIfCondition(state, EmitMemoryElementInBounds(state, resource, index), [&] { StoreWordInBounds(ctx, resource, index, data); }); + EmitIfCondition(state, EmitMemoryElementInBounds(state, resource, index), [&] { + if (bits == 32u) { + StoreWordInBounds(ctx, resource, index, data); + } else { + StoreSubwordInBounds(ctx, storage, resource, byteOffset, index, bits, data); + } + }); +} + +void StoreApertureElement(SpirvValueEmitContext& ctx, ResourceKind kind, std::uint32_t byteOffset, std::uint32_t bits, std::uint32_t data) { + auto& state = ctx.state; + if (bits == 8u) { + StoreApertureByte(ctx, kind, byteOffset, 0u, data); + return; + } + const auto contained = ApertureContained(state, byteOffset, bits); + EmitIfCondition(state, contained, [&] { StoreApertureDword(ctx, kind, byteOffset, bits, data); }); + EmitIfCondition(state, Unary(state, spv::OpLogicalNot, TypeBool(state), contained), [&] { + for (std::uint32_t byte = 0; byte < bits / 8u; ++byte) { + StoreApertureByte(ctx, kind, byteOffset, byte, Binary(state, spv::OpShiftRightLogical, TypeU32(state), data, ConstantU32(state, byte * 8u))); + } + }); } template -std::uint32_t RouteFlatLoad(SpirvEmitterState& state, std::uint32_t address, std::uint32_t type, TAperture&& aperture, TGlobal&& global) { +std::uint32_t RouteFlatAccess(SpirvEmitterState& state, std::uint32_t address, std::uint32_t type, TAperture&& aperture, TGlobal&& global) { const auto split = SplitAperture(state, address); return EmitValueIfElse(state, split.shared, type, [&] { return aperture(ResourceKind::Lds, split.low); }, [&] { return EmitValueIfElse(state, split.priv, type, [&] { return aperture(ResourceKind::Scratch, split.low); }, global); @@ -394,7 +464,7 @@ void LoadAddress(SpirvValueEmitContext& ctx, const IrValue& inst, std::uint32_t if (RoutesApertures(ctx.state, mem)) { ctx.Define(inst, EmitValueOrZeroIfCondition(ctx.state, ActiveArgument(ctx, inst), [&] { const auto address = GuestAddress(ctx, inst, mem); - return RouteFlatLoad(ctx.state, address, TypeU32(ctx.state), [&](ResourceKind kind, std::uint32_t low) { return LoadApertureElement(ctx, kind, low, bits); }, [&] { return EmitBdaRead(ctx, inst, address, bits); }); + return RouteFlatAccess(ctx.state, address, TypeU32(ctx.state), [&](ResourceKind kind, std::uint32_t low) { return LoadApertureElement(ctx, kind, low, bits); }, [&] { return EmitBdaRead(ctx, inst, address, bits); }); })); return; } @@ -430,7 +500,7 @@ void LoadAddressWide(SpirvValueEmitContext& ctx, const IrValue& inst, std::uint3 ctx.Define(inst, EmitValueOrDefaultIfCondition(state, ActiveArgument(ctx, inst), type, ConstantU32CompositeZero(state, components), [&] { const auto base = GuestAddressBase(ctx, inst, mem); const auto address = AddBdaImmediate(ctx, inst, base, static_cast(mem.offset)); - return RouteFlatLoad(state, address, type, [&](ResourceKind kind, std::uint32_t low) { + return RouteFlatAccess(state, address, type, [&](ResourceKind kind, std::uint32_t low) { std::array values{}; for (std::uint32_t component = 0; component < components; component++) { values[component] = LoadApertureElement(ctx, kind, Binary(state, spv::OpIAdd, TypeU32(state), low, ConstantU32(state, component * 4u)), 32u); @@ -466,13 +536,13 @@ void StoreAddress(SpirvValueEmitContext& ctx, const IrValue& inst, std::uint32_t auto& state = ctx.state; const auto address = GuestAddress(ctx, inst, mem); const auto data = ctx.Arg(inst, inst.ArgumentCount() - 2u); - if (bits != 32u || !RoutesApertures(state, mem)) { + if (!RoutesApertures(state, mem)) { EmitBdaStore(ctx, inst, address, data, bits); return; } const auto split = SplitAperture(state, address); - EmitIfCondition(state, split.shared, [&] { StoreApertureWord(ctx, ResourceKind::Lds, split.low, data); }); - EmitIfCondition(state, split.priv, [&] { StoreApertureWord(ctx, ResourceKind::Scratch, split.low, data); }); + EmitIfCondition(state, split.shared, [&] { StoreApertureElement(ctx, ResourceKind::Lds, split.low, bits, data); }); + EmitIfCondition(state, split.priv, [&] { StoreApertureElement(ctx, ResourceKind::Scratch, split.low, bits, data); }); const auto global = Unary(state, spv::OpLogicalNot, TypeBool(state), Binary(state, spv::OpLogicalOr, TypeBool(state), split.shared, split.priv)); EmitIfCondition(state, global, [&] { EmitBdaStore(ctx, inst, address, data, bits); }); }); @@ -818,6 +888,163 @@ std::uint32_t AddressAtomicOpcode(IrOpcode opcode) { } } +std::uint32_t FlatAtomicNext32(SpirvEmitterState& state, IrOpcode opcode, std::uint32_t old, std::uint32_t value, std::uint32_t comparator) { + const auto u32 = TypeU32(state); + const auto winner = [&](spv::Op compare) { + return Select(state, u32, Binary(state, compare, TypeBool(state), old, value), old, value); + }; + switch (opcode) { + case IrOpcode::AddressAtomicSwap32: + return value; + case IrOpcode::AddressAtomicCmpSwap32: + return Select(state, u32, Binary(state, spv::OpIEqual, TypeBool(state), old, comparator), value, old); + case IrOpcode::AddressAtomicIAdd32: + return Binary(state, spv::OpIAdd, u32, old, value); + case IrOpcode::AddressAtomicISub32: + return Binary(state, spv::OpISub, u32, old, value); + case IrOpcode::AddressAtomicSMin32: + return winner(spv::OpSLessThan); + case IrOpcode::AddressAtomicUMin32: + return winner(spv::OpULessThan); + case IrOpcode::AddressAtomicSMax32: + return winner(spv::OpSGreaterThan); + case IrOpcode::AddressAtomicUMax32: + return winner(spv::OpUGreaterThan); + case IrOpcode::AddressAtomicAnd32: + return Binary(state, spv::OpBitwiseAnd, u32, old, value); + case IrOpcode::AddressAtomicOr32: + return Binary(state, spv::OpBitwiseOr, u32, old, value); + case IrOpcode::AddressAtomicXor32: + return Binary(state, spv::OpBitwiseXor, u32, old, value); + case IrOpcode::AddressAtomicInc32: + return AtomicIncrement(state, old, value); + case IrOpcode::AddressAtomicDec32: + return AtomicDecrement(state, old, value); + case IrOpcode::AddressAtomicUSubSat32: + return AtomicUSubSat(state, old, value); + case IrOpcode::AddressAtomicFMin32: + return AtomicFloatBits{state, false}.minMax(old, value, false); + case IrOpcode::AddressAtomicFMax32: + return AtomicFloatBits{state, false}.minMax(old, value, true); + case IrOpcode::AddressAtomicFCmpSwap32: + return AtomicFloatBits{state, false}.compareSwap(old, comparator, value); + default: + throw std::runtime_error("FlatAtomicNext32: opcode has no aperture lowering"); + } +} + +std::uint32_t ApertureAtomic32(SpirvValueEmitContext& ctx, const IrValue& inst, ResourceKind kind, std::uint32_t address, std::uint32_t byteOffset, std::uint32_t value, std::uint32_t comparator) { + auto& state = ctx.state; + const auto instruction = ConstantU32(state, inst.Flags().pc); + const auto unaligned = Binary(state, spv::OpINotEqual, TypeBool(state), Binary(state, spv::OpBitwiseAnd, TypeU32(state), byteOffset, ConstantU32(state, 3u)), ConstantU32(state, 0u)); + EmitIfCondition(state, unaligned, [&] { RecordBdaFault(state, address, ConstantU32(state, 4u), instruction, BdaAbi::FaultReason::Unaligned); }); + if (kind == ResourceKind::Scratch && state.program.Info().scratchDwords == 0u) return ConstantU32(state, 0u); + MemoryInfo storage{}; + storage.kind = kind; + const auto resource = PrepareMemoryResourceAccess(state, storage); + const auto index = Binary(state, spv::OpShiftRightLogical, TypeU32(state), byteOffset, ConstantU32(state, 2u)); + const auto aligned = Unary(state, spv::OpLogicalNot, TypeBool(state), unaligned); + const auto opcode = inst.Opcode(); + return EmitValueOrZeroIfCondition(state, aligned, [&] { + return EmitValueOrZeroIfCondition(state, EmitMemoryElementInBounds(state, resource, index), [&] { + const auto pointer = EmitMemoryElementPointer(state, resource, index); + if (kind == ResourceKind::Lds) { + return AtomicUpdate(state, pointer, kind, [&](std::uint32_t current) { return FlatAtomicNext32(state, opcode, current, value, comparator); }); + } + const auto old = state.module.AllocateId(); + state.module.AddFunction(spv::OpLoad, TypeU32(state), old, pointer); + state.module.AddFunction(spv::OpStore, pointer, FlatAtomicNext32(state, opcode, old, value, comparator)); + return old; + }); + }); +} + +std::uint32_t FlatAtomicNext64(SpirvEmitterState& state, IrOpcode opcode, std::uint32_t old, std::uint32_t value, std::uint32_t comparator) { + const auto u64 = TypeScalarU64(state); + const AtomicFloatBits bits{state, true}; + const auto winner = [&](spv::Op compare) { + return Select(state, u64, Binary(state, compare, TypeBool(state), old, value), old, value); + }; + switch (opcode) { + case IrOpcode::AddressAtomicSwap64: + return value; + case IrOpcode::AddressAtomicCmpSwap64: + return Select(state, u64, Binary(state, spv::OpIEqual, TypeBool(state), old, comparator), value, old); + case IrOpcode::AddressAtomicIAdd64: + return Binary(state, spv::OpIAdd, u64, old, value); + case IrOpcode::AddressAtomicISub64: + return Binary(state, spv::OpISub, u64, old, value); + case IrOpcode::AddressAtomicSMin64: + return winner(spv::OpSLessThan); + case IrOpcode::AddressAtomicUMin64: + return winner(spv::OpULessThan); + case IrOpcode::AddressAtomicSMax64: + return winner(spv::OpSGreaterThan); + case IrOpcode::AddressAtomicUMax64: + return winner(spv::OpUGreaterThan); + case IrOpcode::AddressAtomicAnd64: + return Binary(state, spv::OpBitwiseAnd, u64, old, value); + case IrOpcode::AddressAtomicOr64: + return Binary(state, spv::OpBitwiseOr, u64, old, value); + case IrOpcode::AddressAtomicXor64: + return Binary(state, spv::OpBitwiseXor, u64, old, value); + case IrOpcode::AddressAtomicInc64: + return bits.increment(old, value); + case IrOpcode::AddressAtomicDec64: + return bits.decrement(old, value); + case IrOpcode::AddressAtomicFMin64: + return bits.minMax(old, value, false); + case IrOpcode::AddressAtomicFMax64: + return bits.minMax(old, value, true); + case IrOpcode::AddressAtomicFCmpSwap64: + return bits.compareSwap(old, comparator, value); + default: + throw std::runtime_error("FlatAtomicNext64: opcode has no aperture lowering"); + } +} + +template +std::uint32_t LockedLdsUpdate(SpirvEmitterState& state, TUpdate&& update); + +std::uint32_t ApertureAtomic64(SpirvValueEmitContext& ctx, const IrValue& inst, ResourceKind kind, std::uint32_t address, std::uint32_t byteOffset, std::uint32_t value, std::uint32_t comparator) { + auto& state = ctx.state; + const auto instruction = ConstantU32(state, inst.Flags().pc); + const auto unaligned = Binary(state, spv::OpINotEqual, TypeBool(state), Binary(state, spv::OpBitwiseAnd, TypeU32(state), byteOffset, ConstantU32(state, 7u)), ConstantU32(state, 0u)); + EmitIfCondition(state, unaligned, [&] { RecordBdaFault(state, address, ConstantU32(state, 8u), instruction, BdaAbi::FaultReason::Unaligned); }); + if (kind == ResourceKind::Scratch && state.program.Info().scratchDwords == 0u) return ConstantU64(state, 0u); + if (kind == ResourceKind::Lds && !state.requirements.ldsLock) ctx.Fail(inst, "64-bit LDS atomic reached through a flat address without the workgroup lock"); + MemoryInfo storage{}; + storage.kind = kind; + const auto resource = PrepareMemoryResourceAccess(state, storage); + const auto low = Binary(state, spv::OpShiftRightLogical, TypeU32(state), byteOffset, ConstantU32(state, 2u)); + const auto high = EmitAddU32(state, low, ConstantU32(state, 1u)); + const auto aligned = Unary(state, spv::OpLogicalNot, TypeBool(state), unaligned); + const auto opcode = inst.Opcode(); + return EmitValueOrDefaultIfCondition(state, aligned, TypeU64(state), ConstantU64(state, 0u), [&] { + return EmitValueOrDefaultIfCondition(state, EmitMemoryElementInBounds(state, resource, high), TypeU64(state), ConstantU64(state, 0u), [&] { + const auto update = [&]() { + const auto lowPointer = EmitMemoryElementPointer(state, resource, low); + const auto highPointer = EmitMemoryElementPointer(state, resource, high); + const auto lowValue = state.module.AllocateId(); + const auto highValue = state.module.AllocateId(); + state.module.AddFunction(spv::OpLoad, TypeU32(state), lowValue, lowPointer); + state.module.AddFunction(spv::OpLoad, TypeU32(state), highValue, highPointer); + const auto old = state.module.AllocateId(); + state.module.AddFunction(spv::OpCompositeConstruct, TypeU64(state), old, lowValue, highValue); + const auto next = Unary(state, spv::OpBitcast, TypeU64(state), FlatAtomicNext64(state, opcode, Unary(state, spv::OpBitcast, TypeScalarU64(state), old), value, comparator)); + const auto nextLow = state.module.AllocateId(); + const auto nextHigh = state.module.AllocateId(); + state.module.AddFunction(spv::OpCompositeExtract, TypeU32(state), nextLow, next, 0u); + state.module.AddFunction(spv::OpCompositeExtract, TypeU32(state), nextHigh, next, 1u); + state.module.AddFunction(spv::OpStore, lowPointer, nextLow); + state.module.AddFunction(spv::OpStore, highPointer, nextHigh); + return old; + }; + return kind == ResourceKind::Lds ? LockedLdsUpdate(state, update) : update(); + }); + }); +} + std::uint32_t AddressAtomic(SpirvValueEmitContext& ctx, const IrValue& inst) { auto& state = ctx.state; const auto& mem = ctx.Memory(inst); @@ -831,7 +1058,7 @@ std::uint32_t AddressAtomic(SpirvValueEmitContext& ctx, const IrValue& inst) { const auto scalarType = wide ? TypeScalarU64(state) : TypeU32(state); const auto scalar = [&](std::uint32_t value) { return wide ? Unary(state, spv::OpBitcast, scalarType, value) : value; }; const auto value = scalar(ctx.Arg(inst, 3)); - const auto old = EmitBdaAtomic(ctx, inst, GuestAddress(ctx, inst, mem), wide ? 8u : 4u, [&](std::uint32_t pointer) { + const auto operation = [&](std::uint32_t pointer) { if (inst.Opcode() == IrOpcode::AddressAtomicInc32 || inst.Opcode() == IrOpcode::AddressAtomicDec32) { const bool increment = inst.Opcode() == IrOpcode::AddressAtomicInc32; return AtomicUpdate(state, pointer, mem.kind, [&](std::uint32_t current) { @@ -873,7 +1100,17 @@ std::uint32_t AddressAtomic(SpirvValueEmitContext& ctx, const IrValue& inst) { } EmitDeviceAtomicMemoryBarrier(state); return result; - }); + }; + if (RoutesApertures(state, mem)) { + const auto address = GuestAddress(ctx, inst, mem); + const bool hasComparator = inst.Opcode() == IrOpcode::AddressAtomicCmpSwap32 || inst.Opcode() == IrOpcode::AddressAtomicFCmpSwap32 || inst.Opcode() == IrOpcode::AddressAtomicCmpSwap64 || inst.Opcode() == IrOpcode::AddressAtomicFCmpSwap64; + const auto comparator = hasComparator ? scalar(ctx.Arg(inst, 4)) : 0u; + if (wide) { + return RouteFlatAccess(state, address, type, [&](ResourceKind kind, std::uint32_t low) { return ApertureAtomic64(ctx, inst, kind, address, low, value, comparator); }, [&] { return Unary(state, spv::OpBitcast, type, EmitBdaAtomic(ctx, inst, address, 8u, operation)); }); + } + return RouteFlatAccess(state, address, TypeU32(state), [&](ResourceKind kind, std::uint32_t low) { return ApertureAtomic32(ctx, inst, kind, address, low, value, comparator); }, [&] { return EmitBdaAtomic(ctx, inst, address, 4u, operation); }); + } + const auto old = EmitBdaAtomic(ctx, inst, GuestAddress(ctx, inst, mem), wide ? 8u : 4u, operation); return wide ? Unary(state, spv::OpBitcast, type, old) : old; }); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 90a95b787..224256cbf 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -367,6 +367,9 @@ Throughout the project, every function at every stage either **does exactly what - A [V#, T# or S#](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp) loaded through a null or unmapped pointer read from memory is zero when the work is recorded, and the capture throws if no flat SRT slot records that read. - [2D accesses through a 1D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) read the texel the 1D addressing selects and ignore the second coordinate, as simonbbby measured on RDNA2 (gfx1036) in #2237 for `image_sample_lz` and mip-less `image_load` with point filtering and wrap, clamp and border addressing; the PS5's gfx1013 is not measured. Other 2D accesses through a 1D T# (storage, depth, packed, BY loads, mips, derivatives, comparison) still throw. - [Emulated comparison of a 2D-array access through a 2D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) samples the 2D image and ignores the slice, as the native modes of a 2D-array access through a 2D T# already do. This follows the T# type deciding the addressing; it is not measured on hardware for comparison samples +- [Scalar aperture base sources](../../core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp) (core/shader/recompiler) - `src_shared_base`, `src_shared_limit`, `src_private_base` and `src_private_limit` (scalar source codes 235 to 238) are not modelled: decoding throws `unsupported scalar source operand code` with the code and the program counter, so a shader that reads one does not compile. Operand 235 (`src_shared_base`) was reported on [#1182](https://github.com/boykopovar/AnyPS5/pull/1182) from PPSA26344; the flat aperture routing does not read the base, so a shader that builds its own flat addresses from it stops at decode. +- [Flat aperture bases](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (core/shader/recompiler) - the shared base 0x8000 and private base 0x7000 (the top 16 bits of a flat address, `SharedApertureTop` and `PrivateApertureTop`) are inferred from GTA V's traversal addresses. They are not measured on a console and not taken from SH_MEM_BASES. No public source found confirms them: the AMD KFD driver snapshots found (`kfd_device_queue_manager_cik.c`, `kfd_flat_memory.c`, CIK and VI era) place LDS and scratch by one top address nybble as 4 GB windows with scratch directly after LDS, which does not give a 0x7000 private base. The gfx10 and PS5 layout is unknown. +- [Flat accesses outside the aperture routing](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (core/shader/recompiler) - only compute stages route flat accesses, so graphics-stage flat accesses to the shared or private aperture still go to BDA. In compute, a 64-bit flat atomic whose address is in the shared aperture takes the workgroup lock (`ldsLock`) for its two-dword update, and every such atomic requests that lock, so a shader with one pays for it even on global addresses (the path is exercised by `agc_driver_global_atomics_float`). Inferred, not measured on LDS or scratch: an aperture access that is not inside one dword is done byte by byte, an unaligned aperture atomic records the BDA unaligned fault, and LDS or scratch accesses past their end are dropped or read zero. - On a host subgroup narrower than 32 lanes, [`threadBit`](../../core/shader/recompiler/Translation/src/TranslationContext/ExecMask.cpp) reads an EXEC written as a value (`s_mov_b32`/`s_mov_b64`) at bit (subgroup invocation id) of EXEC_LO, so lanes at or above the subgroup size read a lower lane's bit. - [EXEC read as a value](../../core/shader/recompiler/Translation/src/TranslationContext/ExecMask.cpp) (`s_mov_b64 sN, exec`, the `s_*_saveexec` copy, ALU sources) holds only lanes with a host invocation; the console's also holds the wave's other lanes, e.g. 32-63 of a wave64 on 32 host lanes. - [`v_readlane_b32`](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) and fetch-inactive `v_permlane16_b32`/`v_permlanex16_b32` read undefined data from a lane that the host subgroup does not hold or whose host invocation is inactive; the console reads the lane's VGPR. From 41144bea53191434e655211b6950c0bf2e34f6e0 Mon Sep 17 00:00:00 2001 From: Michon <231849@buas.nl> Date: Sat, 10 Oct 2026 10:16:15 +0200 Subject: [PATCH 179/460] fix(recompiler): multi-dword FLAT stores route the shared and private apertures Main's StoreAddressU32x2..x4 (one BDA lookup per wide store, b95842597) bypassed the aperture routing: a flat_store_dwordx2 into the shared aperture went to BDA at 0x8000000000000000 and LDS kept its old value. The wide store now stores each component through the aperture like the 32-bit store, and keeps the single BDA lookup outside the apertures. The aperture test's V# word 3 is 0x31016fac (OOB_SELECT 3, raw), as in main's other execution tests: with OOB_SELECT 0 its output buffer read as out of range after #639, which was the failure reported on four drivers. Co-Authored-By: Claude Opus 5.5 (1M context) --- .../tests/execution/FlatApertures.cpp | 2 +- .../SpirvMemory/SpirvMemoryInstructions.cpp | 20 ++++++++++++++++++- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp index d73947d4d..b33d913ee 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp @@ -96,7 +96,7 @@ private: std::array BufferDescriptor(const void* data, std::uint32_t bytes) { const auto address = reinterpret_cast(data); - return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x01016facu}; + return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; } std::string Hex(std::uint32_t value) { diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp index 3c96d5066..d2b7dd85f 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp @@ -1440,7 +1440,25 @@ void StoreAddressWide(SpirvValueEmitContext& ctx, const IrValue& inst, std::uint const auto& mem = ctx.Memory(inst); if (mem.kind != ResourceKind::Flat && mem.kind != ResourceKind::Global) ctx.Fail(inst, "must write a physical address resource"); EmitIfCondition(ctx.state, ActiveArgument(ctx, inst), [&]() { - EmitBdaDwordWrites(ctx, inst, GuestAddressBase(ctx, inst, mem), mem.offset, components, ctx.Arg(inst, inst.ArgumentCount() - 2u)); + auto& state = ctx.state; + const auto base = GuestAddressBase(ctx, inst, mem); + const auto data = ctx.Arg(inst, inst.ArgumentCount() - 2u); + if (!RoutesApertures(state, mem)) { + EmitBdaDwordWrites(ctx, inst, base, mem.offset, components, data); + return; + } + const auto split = SplitAperture(state, AddBdaImmediate(ctx, inst, base, static_cast(mem.offset))); + const auto storeAperture = [&](ResourceKind kind) { + for (std::uint32_t component = 0; component < components; component++) { + const auto value = state.module.AllocateId(); + state.module.AddFunction(spv::OpCompositeExtract, TypeU32(state), value, data, component); + StoreApertureElement(ctx, kind, Binary(state, spv::OpIAdd, TypeU32(state), split.low, ConstantU32(state, component * 4u)), 32u, value); + } + }; + EmitIfCondition(state, split.shared, [&] { storeAperture(ResourceKind::Lds); }); + EmitIfCondition(state, split.priv, [&] { storeAperture(ResourceKind::Scratch); }); + const auto global = Unary(state, spv::OpLogicalNot, TypeBool(state), Binary(state, spv::OpLogicalOr, TypeBool(state), split.shared, split.priv)); + EmitIfCondition(state, global, [&] { EmitBdaDwordWrites(ctx, inst, base, mem.offset, components, data); }); }); } From f4f05d2e6a99879d987b86d6540a796fb3b9e676 Mon Sep 17 00:00:00 2001 From: Michon <231849@buas.nl> Date: Sat, 10 Oct 2026 11:58:49 +0200 Subject: [PATCH 180/460] fix(recompiler): flat apertures are 4 GB windows; an aperture access past the end is dropped whole Measured on RDNA2 (gfx1035) in review: - the shared and private apertures are 4 GB windows (the gfx9+ layout of kfd_flat_memory.c), so a flat address routes by its whole high dword, 0x80000000 or 0x70000000, not by its top 16 bits; - a dword or short access whose last byte is past the end of LDS reads 0 or writes nothing as a whole; multi-dword accesses are checked per dword. The aperture test gets rows that straddle and pass the end of LDS; the TechnicalDebt line names the public source of the layout and keeps only the PS5's base values as inferred. Co-Authored-By: Claude Opus 5.5 (1M context) --- .../tests/execution/FlatApertures.cpp | 9 +++- .../SpirvMemory/SpirvMemoryInstructions.cpp | 54 ++++++++++++------- docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 44 insertions(+), 21 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp index b33d913ee..ef398d101 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp @@ -352,8 +352,12 @@ void Check(const Row& row, const std::vector& output) { const auto left = output[lane * 8u + 1u]; Require(left == expected, name + ": lane " + std::to_string(lane) + " left " + Hex(left) + ", expected " + Hex(expected)); } else { - const auto expected = row.width == 8u ? (a & 0xffu) : (row.width == 16u ? (a & 0xffffu) : a); const auto read = output[lane * 8u]; + if (row.target == Target::Shared && 8u * lane + row.offset + row.width / 8u > LdsDwords * 4u) { + Require(read == 0u, name + ": lane " + std::to_string(lane) + " read " + Hex(read) + " back from an access past the end of LDS, expected 0 (the store is dropped whole)"); + continue; + } + const auto expected = row.width == 8u ? (a & 0xffu) : (row.width == 16u ? (a & 0xffffu) : a); Require(read == expected, name + ": lane " + std::to_string(lane) + " read " + Hex(read) + " back, expected " + Hex(expected)); const auto first = output[lane * 8u + 1u]; Require(first == (a & 0xffu), name + ": lane " + std::to_string(lane) + " has first byte " + Hex(first) + " in memory, expected " + Hex(a & 0xffu)); @@ -445,6 +449,9 @@ std::vector Rows() { } rows.push_back(Row{Target::Mixed, false, AtomicOp::Swap, 16u, 3u, 32u}); rows.push_back(Row{Target::Mixed, false, AtomicOp::Swap, 32u, 2u, 32u}); + for (const auto [width, offset] : std::array, 5>{{{32u, 773u}, {32u, 774u}, {32u, 776u}, {16u, 775u}, {8u, 775u}}}) { + rows.push_back(Row{Target::Shared, false, AtomicOp::Swap, width, offset, 32u}); + } for (const Target target : single) { for (std::uint32_t op = 0; op < Atomic32Ops; ++op) { rows.push_back(Row{target, true, static_cast(op), 32u, 0u, 32u}); diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp index d2b7dd85f..94b336e13 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp @@ -336,8 +336,8 @@ const MemoryInfo& SharedMemory(SpirvValueEmitContext& ctx, const IrValue& inst) return mem; } -constexpr std::uint32_t SharedApertureTop = 0x8000u; -constexpr std::uint32_t PrivateApertureTop = 0x7000u; +constexpr std::uint32_t SharedApertureHigh = 0x80000000u; +constexpr std::uint32_t PrivateApertureHigh = 0x70000000u; bool RoutesApertures(const SpirvEmitterState& state, const MemoryInfo& mem) { return mem.kind == ResourceKind::Flat && state.program.Resources().stage == IrShaderStage::Compute; @@ -352,8 +352,7 @@ struct ApertureAddress { ApertureAddress SplitAperture(SpirvEmitterState& state, std::uint32_t address) { const auto u32 = TypeU32(state); const auto high = Unary(state, spv::OpUConvert, u32, Binary(state, spv::OpShiftRightLogical, TypeScalarU64(state), address, BdaConstant(state, 32u))); - const auto top = Binary(state, spv::OpShiftRightLogical, u32, high, ConstantU32(state, 16u)); - return {Unary(state, spv::OpUConvert, u32, address), Binary(state, spv::OpIEqual, TypeBool(state), top, ConstantU32(state, SharedApertureTop)), Binary(state, spv::OpIEqual, TypeBool(state), top, ConstantU32(state, PrivateApertureTop))}; + return {Unary(state, spv::OpUConvert, u32, address), Binary(state, spv::OpIEqual, TypeBool(state), high, ConstantU32(state, SharedApertureHigh)), Binary(state, spv::OpIEqual, TypeBool(state), high, ConstantU32(state, PrivateApertureHigh))}; } std::uint32_t ApertureByteInBounds(SpirvEmitterState& state, const MemoryResourceAccess& resource, std::uint32_t byteOffset, std::uint32_t offset, std::uint32_t index) { @@ -391,18 +390,32 @@ std::uint32_t LoadApertureByte(SpirvValueEmitContext& ctx, ResourceKind kind, st }); } +std::uint32_t ApertureElementInBounds(SpirvValueEmitContext& ctx, ResourceKind kind, std::uint32_t byteOffset, std::uint32_t bits) { + auto& state = ctx.state; + MemoryInfo storage{}; + storage.kind = kind; + const auto resource = PrepareMemoryResourceAccess(state, storage); + const auto last = bits / 8u - 1u; + const auto address = EmitAddU32(state, byteOffset, ConstantU32(state, last)); + const auto index = Binary(state, spv::OpShiftRightLogical, TypeU32(state), address, ConstantU32(state, 2u)); + return ApertureByteInBounds(state, resource, byteOffset, last, index); +} + std::uint32_t LoadApertureElement(SpirvValueEmitContext& ctx, ResourceKind kind, std::uint32_t byteOffset, std::uint32_t bits) { auto& state = ctx.state; if (bits == 8u) return LoadApertureByte(ctx, kind, byteOffset, 0u); - return EmitValueIfElse(state, ApertureContained(state, byteOffset, bits), TypeU32(state), [&] { - return LoadApertureDword(ctx, kind, byteOffset, bits); - }, [&] { - std::uint32_t value = ConstantU32(state, 0u); - for (std::uint32_t byte = 0; byte < bits / 8u; ++byte) { - const auto part = Binary(state, spv::OpShiftLeftLogical, TypeU32(state), LoadApertureByte(ctx, kind, byteOffset, byte), ConstantU32(state, byte * 8u)); - value = Binary(state, spv::OpBitwiseOr, TypeU32(state), value, part); - } - return value; + if (kind == ResourceKind::Scratch && state.program.Info().scratchDwords == 0u) return ConstantU32(state, 0u); + return EmitValueOrZeroIfCondition(state, ApertureElementInBounds(ctx, kind, byteOffset, bits), [&] { + return EmitValueIfElse(state, ApertureContained(state, byteOffset, bits), TypeU32(state), [&] { + return LoadApertureDword(ctx, kind, byteOffset, bits); + }, [&] { + std::uint32_t value = ConstantU32(state, 0u); + for (std::uint32_t byte = 0; byte < bits / 8u; ++byte) { + const auto part = Binary(state, spv::OpShiftLeftLogical, TypeU32(state), LoadApertureByte(ctx, kind, byteOffset, byte), ConstantU32(state, byte * 8u)); + value = Binary(state, spv::OpBitwiseOr, TypeU32(state), value, part); + } + return value; + }); }); } @@ -439,12 +452,15 @@ void StoreApertureElement(SpirvValueEmitContext& ctx, ResourceKind kind, std::ui StoreApertureByte(ctx, kind, byteOffset, 0u, data); return; } - const auto contained = ApertureContained(state, byteOffset, bits); - EmitIfCondition(state, contained, [&] { StoreApertureDword(ctx, kind, byteOffset, bits, data); }); - EmitIfCondition(state, Unary(state, spv::OpLogicalNot, TypeBool(state), contained), [&] { - for (std::uint32_t byte = 0; byte < bits / 8u; ++byte) { - StoreApertureByte(ctx, kind, byteOffset, byte, Binary(state, spv::OpShiftRightLogical, TypeU32(state), data, ConstantU32(state, byte * 8u))); - } + if (kind == ResourceKind::Scratch && state.program.Info().scratchDwords == 0u) return; + EmitIfCondition(state, ApertureElementInBounds(ctx, kind, byteOffset, bits), [&] { + const auto contained = ApertureContained(state, byteOffset, bits); + EmitIfCondition(state, contained, [&] { StoreApertureDword(ctx, kind, byteOffset, bits, data); }); + EmitIfCondition(state, Unary(state, spv::OpLogicalNot, TypeBool(state), contained), [&] { + for (std::uint32_t byte = 0; byte < bits / 8u; ++byte) { + StoreApertureByte(ctx, kind, byteOffset, byte, Binary(state, spv::OpShiftRightLogical, TypeU32(state), data, ConstantU32(state, byte * 8u))); + } + }); }); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 224256cbf..e7b475b0f 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -368,7 +368,7 @@ Throughout the project, every function at every stage either **does exactly what - [2D accesses through a 1D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) read the texel the 1D addressing selects and ignore the second coordinate, as simonbbby measured on RDNA2 (gfx1036) in #2237 for `image_sample_lz` and mip-less `image_load` with point filtering and wrap, clamp and border addressing; the PS5's gfx1013 is not measured. Other 2D accesses through a 1D T# (storage, depth, packed, BY loads, mips, derivatives, comparison) still throw. - [Emulated comparison of a 2D-array access through a 2D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) samples the 2D image and ignores the slice, as the native modes of a 2D-array access through a 2D T# already do. This follows the T# type deciding the addressing; it is not measured on hardware for comparison samples - [Scalar aperture base sources](../../core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp) (core/shader/recompiler) - `src_shared_base`, `src_shared_limit`, `src_private_base` and `src_private_limit` (scalar source codes 235 to 238) are not modelled: decoding throws `unsupported scalar source operand code` with the code and the program counter, so a shader that reads one does not compile. Operand 235 (`src_shared_base`) was reported on [#1182](https://github.com/boykopovar/AnyPS5/pull/1182) from PPSA26344; the flat aperture routing does not read the base, so a shader that builds its own flat addresses from it stops at decode. -- [Flat aperture bases](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (core/shader/recompiler) - the shared base 0x8000 and private base 0x7000 (the top 16 bits of a flat address, `SharedApertureTop` and `PrivateApertureTop`) are inferred from GTA V's traversal addresses. They are not measured on a console and not taken from SH_MEM_BASES. No public source found confirms them: the AMD KFD driver snapshots found (`kfd_device_queue_manager_cik.c`, `kfd_flat_memory.c`, CIK and VI era) place LDS and scratch by one top address nybble as 4 GB windows with scratch directly after LDS, which does not give a 0x7000 private base. The gfx10 and PS5 layout is unknown. +- [Flat aperture bases](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (core/shader/recompiler) - the layout follows the gfx9+ one in the AMD KFD driver's `kfd_flat_memory.c` (`MAKE_LDS_APP_BASE_V9`, `MAKE_LDS_APP_LIMIT`): each aperture is a 4 GB window whose base is the high dword, so a flat address is routed by its whole high dword, as measured on RDNA2 (gfx1035: `src_shared_base` 0x00010000_00000000, `src_shared_limit` 0x00010000_ffffffff). The PS5's base values, high dwords 0x80000000 (shared, `SharedApertureHigh`) and 0x70000000 (private, `PrivateApertureHigh`), are inferred from GTA V's traversal addresses, not measured on a console or read from SH_MEM_BASES - [Flat accesses outside the aperture routing](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (core/shader/recompiler) - only compute stages route flat accesses, so graphics-stage flat accesses to the shared or private aperture still go to BDA. In compute, a 64-bit flat atomic whose address is in the shared aperture takes the workgroup lock (`ldsLock`) for its two-dword update, and every such atomic requests that lock, so a shader with one pays for it even on global addresses (the path is exercised by `agc_driver_global_atomics_float`). Inferred, not measured on LDS or scratch: an aperture access that is not inside one dword is done byte by byte, an unaligned aperture atomic records the BDA unaligned fault, and LDS or scratch accesses past their end are dropped or read zero. - On a host subgroup narrower than 32 lanes, [`threadBit`](../../core/shader/recompiler/Translation/src/TranslationContext/ExecMask.cpp) reads an EXEC written as a value (`s_mov_b32`/`s_mov_b64`) at bit (subgroup invocation id) of EXEC_LO, so lanes at or above the subgroup size read a lower lane's bit. - [EXEC read as a value](../../core/shader/recompiler/Translation/src/TranslationContext/ExecMask.cpp) (`s_mov_b64 sN, exec`, the `s_*_saveexec` copy, ALU sources) holds only lanes with a host invocation; the console's also holds the wave's other lanes, e.g. 32-63 of a wave64 on 32 host lanes. From df25847fca395f718e29b81d6418fad19aba0f91 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 00:06:53 +0200 Subject: [PATCH 181/460] fix(test): pass the gpu argument to GuestAllocations::Add in FlatApertures --- core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp index ef398d101..810f6a16d 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp @@ -78,7 +78,7 @@ public: #endif Require(block != nullptr, "flat apertures: cannot allocate the guest block"); std::memset(block, 0, GuestBytes); - GuestAllocations::Mutation().Add(block, GuestBytes, true, true); + GuestAllocations::Mutation().Add(block, GuestBytes, true, true, true); } ~GuestBlock() { From 373670294e7344cd095135fb92ffbd95c4e8fdfc Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Sun, 11 Oct 2026 07:10:03 +0900 Subject: [PATCH 182/460] fix(opus): export the libSceOpusCeltDec module ProsperoRadio (PPSA99001) imports six sceOpusCeltDec functions from libSceOpusCeltDec and fails to load because no such module exists. It opens this decoder only as a fallback when libSceOpusDec reports a retryable error for a CELT packet, which libSceOpusDec here never returns, so the module is exported without an implementation and each function throws if it is called, as for sceSaveDataCancel. Co-Authored-By: Claude Opus 5.5 --- .../libs/prx/libSceOpusCeltDec/CMakeLists.txt | 16 +++++++ core/libs/prx/libSceOpusCeltDec/Export.cpp | 36 +++++++++++++++ core/libs/tests/GuestOpusCeltDec.cpp | 44 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 97 insertions(+) create mode 100644 core/libs/prx/libSceOpusCeltDec/CMakeLists.txt create mode 100644 core/libs/prx/libSceOpusCeltDec/Export.cpp create mode 100644 core/libs/tests/GuestOpusCeltDec.cpp diff --git a/core/libs/prx/libSceOpusCeltDec/CMakeLists.txt b/core/libs/prx/libSceOpusCeltDec/CMakeLists.txt new file mode 100644 index 000000000..25870e93b --- /dev/null +++ b/core/libs/prx/libSceOpusCeltDec/CMakeLists.txt @@ -0,0 +1,16 @@ +add_library(libSceOpusCeltDec SHARED EXCLUDE_FROM_ALL ${CMAKE_CURRENT_SOURCE_DIR}/Export.cpp) +target_include_directories(libSceOpusCeltDec PRIVATE ${LIBS_INCLUDE_DIR}) +target_link_libraries(libSceOpusCeltDec PRIVATE libc) +configure_windows_unwind(libSceOpusCeltDec) + +if(BUILD_TESTING) + add_test_executable(guest_opus_celt_dec_tests ${LIBS_INCLUDE_DIR}/tests/GuestOpusCeltDec.cpp) + target_include_directories(guest_opus_celt_dec_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_opus_celt_dec_tests PRIVATE libSceOpusCeltDec libc) + configure_windows_unwind(guest_opus_celt_dec_tests) + add_test(NAME guest_opus_celt_dec COMMAND guest_opus_celt_dec_tests) + if(WIN32) + set_tests_properties(guest_opus_celt_dec PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() +endif() diff --git a/core/libs/prx/libSceOpusCeltDec/Export.cpp b/core/libs/prx/libSceOpusCeltDec/Export.cpp new file mode 100644 index 000000000..b70e7fe12 --- /dev/null +++ b/core/libs/prx/libSceOpusCeltDec/Export.cpp @@ -0,0 +1,36 @@ +#include "prx/libc/include/General.hpp" +#include + +extern "C" { + +int APS5_VABI sceOpusCeltDecInitialize(std::uint32_t*) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceOpusCeltDecTerminate(std::uint32_t*) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceOpusCeltDecGetSize(int) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceOpusCeltDecCreateEx(std::uint32_t*, void*, int, int) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceOpusCeltDecDecode(void*, const std::uint8_t*, int, std::int16_t*, int) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceOpusCeltDecDestroy(void*) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +} diff --git a/core/libs/tests/GuestOpusCeltDec.cpp b/core/libs/tests/GuestOpusCeltDec.cpp new file mode 100644 index 000000000..c95c1542c --- /dev/null +++ b/core/libs/tests/GuestOpusCeltDec.cpp @@ -0,0 +1,44 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include + +extern "C" { +int APS5_VABI sceOpusCeltDecInitialize(std::uint32_t*); +int APS5_VABI sceOpusCeltDecTerminate(std::uint32_t*); +int APS5_VABI sceOpusCeltDecGetSize(int); +int APS5_VABI sceOpusCeltDecCreateEx(std::uint32_t*, void*, int, int); +int APS5_VABI sceOpusCeltDecDecode(void*, const std::uint8_t*, int, std::int16_t*, int); +int APS5_VABI sceOpusCeltDecDestroy(void*); +} + +static int failures = 0; + +static void RequireNotImplemented(const char* name, const std::function& call) { + try { + call(); + } catch (const std::runtime_error& error) { + if (std::string(error.what()) == std::string(name) + " not implemented") return; + std::fprintf(stderr, "%s threw: %s\n", name, error.what()); + ++failures; + return; + } + std::fprintf(stderr, "%s returned instead of reporting that it is not implemented\n", name); + ++failures; +} + +int main() { + std::uint32_t context = 0; + std::uint8_t state[64] = {}; + std::int16_t pcm[16] = {}; + const std::uint8_t packet[] = {0xFC, 0x00}; + RequireNotImplemented("sceOpusCeltDecInitialize", [&] { sceOpusCeltDecInitialize(&context); }); + RequireNotImplemented("sceOpusCeltDecTerminate", [&] { sceOpusCeltDecTerminate(&context); }); + RequireNotImplemented("sceOpusCeltDecGetSize", [] { sceOpusCeltDecGetSize(2); }); + RequireNotImplemented("sceOpusCeltDecCreateEx", [&] { sceOpusCeltDecCreateEx(&context, state, 48000, 2); }); + RequireNotImplemented("sceOpusCeltDecDecode", [&] { sceOpusCeltDecDecode(state, packet, sizeof(packet), pcm, sizeof(pcm)); }); + RequireNotImplemented("sceOpusCeltDecDestroy", [&] { sceOpusCeltDecDestroy(state); }); + return failures == 0 ? 0 : 1; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index fab6c9813..d515db127 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -107,6 +107,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceCoredumpWriteUserString](../../core/libs/prx/libkernel/System/src/Coredump.cpp) and [sceSaveDataConvert, sceSaveDataGetConvertProgress](../../core/libs/prx/libSceSaveData.native/Export.cpp) - imported by PPSA16064, not called during its start-up and added without an implementation; unknown signatures - [sceSaveDataCancel](../../core/libs/prx/libSceSaveData.native/Export.cpp) (libSceSaveData.native) - imported by PPSA23126, which fails to load without it; added without an implementation because its signature is unknown. Every save data call here finishes before it returns, so nothing is ever in flight to cancel, but what the console returns then is unknown - [sceAudioOutExClose, sceAudioOutExConfigureOutput, sceAudioOutExGetMonitorInfo, sceAudioOutExOpen](../../core/libs/prx/libSceAudioOut/src/AudioOut.cpp) (libSceAudioOut) - imported by PPSA99505, which fails to load without them; added without an implementation and declared without parameters. PPSA99505 calls them only for its HDMI bitstream output, an option that is off by default, to read the formats the TV or receiver lists, switch the HDMI output mode and open a port for IEC 61937 bursts ([Jelly5](https://github.com/02dnot/Jelly5/blob/552b866d89b2a58b27f7f7b0484dc743ce235526/app/src/jelly5_bitstream.cpp#L110-L179)); [EVO-PLAYER-PS5](https://github.com/sainsaji/EVO-PLAYER-PS5/blob/da0c0451333bab82e79fffd2de4718e917f9dcb0/projects/evoplayer/core/src/audio/evo_pt.cpp) also calls `sceAudioOutExOpen`, `sceAudioOutExConfigureOutput` and `sceAudioOutExClose` for HDMI bitstream output. The two declare the arguments of `sceAudioOutExConfigureOutput` differently and neither is confirmed on a console. Audio reaches the host device through SDL as PCM, so there is no encoded HDMI output to switch to +- [sceOpusCeltDecCreateEx, sceOpusCeltDecDecode, sceOpusCeltDecDestroy, sceOpusCeltDecGetSize, sceOpusCeltDecInitialize, sceOpusCeltDecTerminate](../../core/libs/prx/libSceOpusCeltDec/Export.cpp) (libSceOpusCeltDec) - imported by PPSA99001, which fails to load without the module; added without an implementation. Their signatures match libSceOpusDec's in [ProsperoRadio](https://github.com/blackbearreloaded/ProsperoRadio/blob/2c4029c3a8f43e9e1dfef98374d94d99b14d745d/src/opus_decoder.cpp). PPSA99001 opens this decoder only as a fallback when libSceOpusDec reports a retryable error for a CELT packet, which [libSceOpusDec](../../core/libs/prx/libSceOpusDec/Export.cpp) here never returns, so the fallback is not reached - [sce::Json::Parser::parse](../../core/libs/prx/libSceJson2/Export.cpp) (libSceJson2) - a number that overflows a double (e.g. `1e309`) throws: whether the console rejects the document or stores an infinite real is unknown - [sceUserServiceGetNpAccountId](../../core/libs/prx/libSceUserService/Export.cpp) (libSceUserService) - the account id for a user with no linked PSN account is assumed to be 0 with success; libSceNpManager's `sceNpGetAccountIdA` answers the same question with `SCE_NP_ERROR_SIGNED_OUT` - [sceUserServiceGetUserColor](../../core/libs/prx/libSceUserService/Export.cpp) (libSceUserService) - the default profile colour is assumed to be blue (0) From 97e81c42249cfc7dfb7b2afd8b5ada492dd3035e Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Sat, 10 Oct 2026 16:26:58 -0400 Subject: [PATCH 183/460] feat(libkernel): implement sync --- core/libs/prx/libkernel/System/src/Process.cpp | 4 ++++ core/libs/tests/GuestKernelSync.cpp | 7 +++++++ 2 files changed, 11 insertions(+) diff --git a/core/libs/prx/libkernel/System/src/Process.cpp b/core/libs/prx/libkernel/System/src/Process.cpp index 4df88ebd6..99ed06653 100644 --- a/core/libs/prx/libkernel/System/src/Process.cpp +++ b/core/libs/prx/libkernel/System/src/Process.cpp @@ -293,6 +293,10 @@ void APS5_VABI sceKernelSync(void) { SyncWrittenPaths_nid_no_patch(); } +void APS5_VABI sync_nid_postfix(void) { + sceKernelSync(); +} + int APS5_VABI sched_get_priority_max_nid_postfix(int policy) { validateSchedulingPolicy(policy); return 256; diff --git a/core/libs/tests/GuestKernelSync.cpp b/core/libs/tests/GuestKernelSync.cpp index 526218a99..6cf1cb15c 100644 --- a/core/libs/tests/GuestKernelSync.cpp +++ b/core/libs/tests/GuestKernelSync.cpp @@ -21,6 +21,7 @@ int APS5_VABI sceKernelRename(const char*, const char*); int APS5_VABI sceKernelUnlink(const char*); int APS5_VABI sceKernelChmod_nid_postfix(const char*, std::uint16_t); void APS5_VABI sceKernelSync(); +void APS5_VABI sync_nid_postfix(); void* APS5_VABI fopen_nid_postfix(const char*, const char*); int APS5_VABI fclose_nid_postfix(void*); } @@ -71,5 +72,11 @@ int main() { CloseHandle(exclusive); #endif sceKernelSync(); + const int synced = sceKernelOpen("kernel_sync_probe/synced.bin", 0x1 | 0x200 | 0x400, 0644); + Require(synced >= 0 && sceKernelWrite(synced, "data", 4) == 4 && sceKernelClose(synced) == 0); + Require(Recorded("kernel_sync_probe/synced.bin")); + Require(sceKernelUnlink("kernel_sync_probe/synced.bin") == 0); + sync_nid_postfix(); + Require(!Recorded("kernel_sync_probe/synced.bin")); std::filesystem::remove_all("kernel_sync_probe"); } From 5f4a28c2bcc48c2e87bc944d68b806b9752ad15e Mon Sep 17 00:00:00 2001 From: "john.maia" Date: Thu, 8 Oct 2026 11:05:52 +0100 Subject: [PATCH 184/460] fix(relinker): keep Linux imports declared from host modules off guest exports On Linux every executable or guest-module import whose NID a guest module exports was renamed to the guest-suffixed name, whatever module the import declares. A title that bundles a module exporting system NIDs then had those imports bound to it: the libc.prx loader shim of ps5-native-app-boilerplate exports placeholder functions under the libSceLibcInternal NIDs, so DOOM PPSA99666's snprintf, vsnprintf, memcpy and malloc (declared from libSceLibcInternal) returned 0 and the title never found its WAD (#1497). An import now binds to a guest export only when the module it declares is a guest module, as on Windows. Imports without import-module metadata keep the previous behaviour, and a title's own libc.prx imported from libc still binds to it. Co-Authored-By: Claude Opus 5.5 --- .../relinker/src/guest/GuestModuleBuilder.cpp | 15 ++++++- .../relinker/tests/test_guest_symbol_names.py | 45 ++++++++++++++----- 2 files changed, 48 insertions(+), 12 deletions(-) diff --git a/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp b/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp index 4f8bb12bd..f885f10ed 100644 --- a/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp +++ b/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp @@ -206,7 +206,17 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path Io::WriteU32(symbols, index * 24, static_cast(strings.size())); Io::AppendString(strings, name + GuestSymbolSuffix); }; + const auto declaresHost = [&](const std::string& library) { return !windows && !library.empty() && findGuest(library) == guestNames.end(); }; if (dynamic.DynSymData.size() % 24 != 0) throw Domain::RelinkerException("Invalid executable symbol table"); + std::vector executableModules(dynamic.DynSymData.size() / 24); + for (const auto* table : {&dynamic.RelaData, &dynamic.RelaPltData}) { + if (table->size() % 24 != 0) throw Domain::RelinkerException("Invalid executable relocation table"); + for (std::size_t offset = 0; offset < table->size(); offset += 24) { + const auto symbol = Io::ReadU64(*table, offset + 8) >> 32; + const auto module = dynamic.ImportModules.find(Io::ReadU64(*table, offset)); + if (symbol != 0 && symbol < executableModules.size() && module != dynamic.ImportModules.end()) executableModules[symbol] = module->second; + } + } for (std::size_t offset = 0; offset < dynamic.DynSymData.size(); offset += 24) { if (Io::ReadU16(dynamic.DynSymData, offset + 6) != 0) continue; const auto nameOffset = Io::ReadU32(dynamic.DynSymData, offset); @@ -215,6 +225,7 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path const auto end = std::find(start, dynamic.DynStrData.end(), 0); if (end == dynamic.DynStrData.end()) throw Domain::RelinkerException("Unterminated executable symbol name"); const std::string name(start, end); + if (declaresHost(executableModules[offset / 24])) continue; rejectSharedImport(name.substr(0, name.find('#')), Io::Utf8Path(inputPath)); if (!windows && exports.contains(name)) rename(dynamic.DynSymData, dynamic.DynStrData, offset / 24, name); } @@ -232,7 +243,7 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path } for (std::size_t symbolIndex = 0; symbolIndex < images[index].Symbols.size(); ++symbolIndex) { const auto& symbol = images[index].Symbols[symbolIndex]; - if (symbol.Section != 0 || symbol.Name.empty()) continue; + if (symbol.Section != 0 || symbol.Name.empty() || declaresHost(symbol.Library)) continue; rejectSharedImport(symbol.Name, Io::Utf8Path(images[index].SourcePath)); const auto found = exports.find(symbol.Name); std::vector providers; @@ -257,7 +268,7 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path for (std::size_t index = 1; index < image.Symbols.size(); ++index) { const auto& symbol = image.Symbols[index]; const bool exported = symbol.Section != 0 && (symbol.Info >> 4) != 0 && symbol.Visibility != 1 && symbol.Visibility != 2; - if ((symbol.Section == 0 || exported) && exports.contains(symbol.Name)) rename(image.Dynamic.DynSymData, image.Dynamic.DynStrData, index, symbol.Name); + if (((symbol.Section == 0 && !declaresHost(symbol.Library)) || exported) && exports.contains(symbol.Name)) rename(image.Dynamic.DynSymData, image.Dynamic.DynStrData, index, symbol.Name); } } } diff --git a/core/relinker/relinker/tests/test_guest_symbol_names.py b/core/relinker/relinker/tests/test_guest_symbol_names.py index a99180830..dc01e84e3 100644 --- a/core/relinker/relinker/tests/test_guest_symbol_names.py +++ b/core/relinker/relinker/tests/test_guest_symbol_names.py @@ -5,6 +5,7 @@ import sys import tempfile from test_guest_intel_trampolines import PLAIN_SITE, elf_loads, guest_fixture, main_fixture +from test_windows_import_modules import consumer, provider DYNAMIC = 0x4800 STRINGS = 0x4A00 @@ -36,15 +37,18 @@ def guest_module(symbols): return image -def importing_executable(names): +def importing_executable(names, modules=()): image = main_fixture() - strings, offsets = string_table(names) + needed = [module + ".prx" for module in modules] + strings, offsets = string_table([*names, *needed, *modules]) image[STRINGS:STRINGS + len(strings)] = strings - for index, offset in enumerate(offsets, 1): + for index, offset in enumerate(offsets[:len(names)], 1): struct.pack_into(" Date: Sun, 11 Oct 2026 00:14:26 +0200 Subject: [PATCH 185/460] test(relinker): check the declared-host import by membership, not as the module's only import --- core/relinker/relinker/tests/test_guest_symbol_names.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/relinker/relinker/tests/test_guest_symbol_names.py b/core/relinker/relinker/tests/test_guest_symbol_names.py index dc01e84e3..e37ca64ca 100644 --- a/core/relinker/relinker/tests/test_guest_symbol_names.py +++ b/core/relinker/relinker/tests/test_guest_symbol_names.py @@ -172,7 +172,7 @@ def main(): exporter = module_symbols((guests / "b.prx.guest.prx").read_bytes()) assert exporter == ["shared#guest"], exporter importer = module_symbols((guests / "c.prx.guest.prx").read_bytes()) - assert importer == ["shared"], importer + assert "shared" in importer and "shared#guest" not in importer, importer print("Guest symbol name tests passed") From 9968fda961997107a68d4a442013bd0f847e82c8 Mon Sep 17 00:00:00 2001 From: Josip Parlov Date: Sun, 11 Oct 2026 00:15:48 +0200 Subject: [PATCH 186/460] docs(npmanager): name ZVZ++68nNI8 sceNpGetSandboxInfo (throwing stub), imported by PPSA28416 --- core/libs/prx/libSceNpManager/Export.cpp | 6 ++++++ docs/dev/TechnicalDebt.md | 1 + 2 files changed, 7 insertions(+) diff --git a/core/libs/prx/libSceNpManager/Export.cpp b/core/libs/prx/libSceNpManager/Export.cpp index d60ff5a7d..a384c2202 100644 --- a/core/libs/prx/libSceNpManager/Export.cpp +++ b/core/libs/prx/libSceNpManager/Export.cpp @@ -243,4 +243,10 @@ int APS5_VABI sceNpGetUserIdByAccountId() { return 0; } +int APS5_VABI sceNpGetSandboxInfo(void* sandbox_info) { + (void)sandbox_info; + NotImplemented_nid_no_patch(__func__); + return 0; +} + } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index bece55309..6c07d6e0d 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -42,6 +42,7 @@ Throughout the project, every function at every stage either **does exactly what ### Unknown function info +- [sceNpGetSandboxInfo](../../core/libs/prx/libSceNpManager/Export.cpp) (libSceNpManager, NID `ZVZ++68nNI8`) - the name hashes to the imported NID (found with nidhunt, confirmed with `tools/nid_names.py`). STAR WARS: Galactic Racer (PPSA28416) imports it; its OnlineSubsystemPS5 calls it with one pointer to a 0x50-byte stack buffer and reads the buffer as a string (discussions/3086). The returned string, the exact parameter struct and console errors are unknown, so every call throws without reading or writing caller memory. A working implementation belongs with that title's bring-up. - [sceVideoOutGetOutputStatusWithVrr](../../core/libs/prx/libSceVideoOut/Unimplemented.cpp) (libSceVideoOut, NID `gWT7X8H0bYs`) - the name hashes to the imported NID. PPSA26344 version 02.002.000 passes a VideoOut handle and an output pointer at three call sites. It compares a 32-bit value at byte 4 with 2, tests bit 0x10 at byte 16, and, when set, compares low 16-bit values at bytes 48, 50 and 52 with 48, 60 and 120. The full output layout, size, field meanings and console errors are unknown. Every call throws without reading or writing caller memory. - [getuid, geteuid, getgid, getegid and issetugid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) report the root identity (0) and a process that never changed credentials (`issetugid` 0), as a jailbroken process has; the host user and group are not exposed to the guest. The values a title sees on a console are unverified - [Packed pixel ancillary word](../../core/shader/recompiler/Optimization/src/ConstantFolder.cpp) (shader recompiler) - the sample index at bits 8-11 and the render target index at bits 16-28 follow #295, and their equality with the SampleId and Layer builtins is inferred, not measured. A value passed through a select, move or phi is rebuilt from those two fields with bits 0-7, 12-15 and 29-31 zero. That is exact only because every read of the word must be an extract inside one modelled field; any other read throws. The other bits are unmodelled and their RDNA 2 layout is unknown: the RDNA 3.5 ISA is reported to give prim type at bits 0-1, VRS rates at bits 2-5 and eye id at bit 7, which was not checked against RDNA 2 From 2d206e31bb8986099885d4487d69847b840dfdb0 Mon Sep 17 00:00:00 2001 From: Renan Pires Date: Sat, 10 Oct 2026 18:42:21 -0300 Subject: [PATCH 187/460] feat(libScePad): export scePadVrControllerReadState --- core/libs/prx/libScePad/Export.cpp | 7 +++++++ docs/dev/TechnicalDebt.md | 1 + 2 files changed, 8 insertions(+) diff --git a/core/libs/prx/libScePad/Export.cpp b/core/libs/prx/libScePad/Export.cpp index 7dc22890e..b5a281ddd 100644 --- a/core/libs/prx/libScePad/Export.cpp +++ b/core/libs/prx/libScePad/Export.cpp @@ -260,6 +260,13 @@ int APS5_VABI scePadVrControllerRead() { return 0; } +int APS5_VABI scePadVrControllerReadState(int handle, void* state) { + (void)handle; + (void)state; + NotImplemented_nid_no_patch(__func__); + return 0; +} + int APS5_VABI scePadIsRemoteController(int handle, bool* remote) { if (handle != PAD_HANDLE) return PAD_ERROR_INVALID_HANDLE; if (remote == nullptr) return PAD_ERROR_INVALID_ARG; diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 47e19bea1..2b251a143 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -66,6 +66,7 @@ Throughout the project, every function at every stage either **does exactly what - [scePsmlMfsr2CreateContext, scePsmlMfsr2CreateSharedResources, scePsmlMfsr2GetContextInitRequirement, scePsmlMfsr2GetDispatchPackets, scePsmlMfsr2GetDispatchPacketsSizeInDwords, scePsmlMfsr2GetSharedResourcesInitRequirement, scePsmlMfsr2Init, scePsmlMfsr2ReleaseContext, scePsmlMfsr2ReleaseSharedResources](../../core/libs/prx/libScePsml_debug/Export.cpp) (libScePsml_mfsr2) - unknown signatures, imported by PPSA23566; declared without parameters - [sceAgcDcbSetCfRegisterDirect, sceAgcDcbSetCfRegisterRangeDirect](../../core/libs/prx/libSceAgc/DcbState/src/Registers.cpp) (libSceAgc) - signatures follow [OpenAGC](https://github.com/OpenAGC/OpenAGC/blob/main/src/game_compat.c) and [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Agc/SceAgc.cs), which agree; the SET_CONFIG_REG (0x68) packet is OpenAGC's. The driver does not execute SET_CONFIG_REG, so a submitted packet throws - [scePadOpenExt](../../core/libs/prx/libScePad/Export.cpp) (libScePad) - only special ports (type 2) are handled, and they return `DEVICE_NOT_CONNECTED` (0x80920007) because the host has no special controller; whether the console opens a handle and reports the device as disconnected later is unknown. PPSA01317 passes a 16-byte param that names the device by vendor and product id (`b7 0e 04 0e ...` for 0x0eb7:0x0e04) and skips the user on any negative result. Other port types throw +- [scePadVrControllerReadState](../../core/libs/prx/libScePad/Export.cpp) (libScePad, NID `iA-DdobUen8`) - the name hashes to the imported NID; the `(int handle, void* state)` signature follows [PS5VR](https://github.com/theghostonline/PS5VR/blob/0e408909ce7a39891a2e06eee3438d07a09e11e3/app/src/app_vr.c#L326-L369) (PPSA99177), which passes a PS VR2 Sense controller handle from `scePadOpen` type 3 or 4 and a 0x68-byte state it reads as buttons at 0, stick bytes at 4 and a connected byte at 0x40. The title calls it only after `sceHmd2Initialize` succeeds, which it does not here, so every call throws; the state layout is otherwise unknown - PPSA23566 imports declared without parameters, signatures unknown: [sceAudioOut2UserGetSupportedAttributes](../../core/libs/prx/libSceAudioOut/src/AudioOut2System.cpp), [sceConvertKeycodeGetCharacterFromKeyboardData](../../core/libs/prx/libSceConvertKeycode/Export.cpp), [sce::Json::MemAllocator::notifyError](../../core/libs/prx/libSceJson2/Export.cpp), [sceNetResolverAbort](../../core/libs/prx/libSceNet/Export.cpp), [sceNpCommerceDialogOpen2](../../core/libs/prx/libSceNpCommerce/Export.cpp), [sceNpEntitlementAccessGetPftFlag](../../core/libs/prx/libSceNpEntitlementAccess/Export.cpp), [sceProprietaryVoiceChatHelper{Initialize,Terminate,SetVoiceChatState,GetVoiceChatUsageState}](../../core/libs/prx/libSceProprietaryVoiceChatHelperPrx/Export.cpp), [sceVoice{GetResourceInfo,ResetPort,EnableChat,DisableChat}](../../core/libs/prx/libSceVoice/Export.cpp). Names from the shadPS4 aerolib NID list - PPSA28416 imports declared without parameters, signatures unknown: [sceAudioOut2EnableChat](../../core/libs/prx/libSceAudioOut/src/AudioOut2System.cpp), imported by its bundled `party-ps5-shipping.prx`. Name from the shadPS4 aerolib NID list - [sceVoiceChatRequestCreateVoiceChatGroup, sceVoiceChatRequestJoinVoiceChatGroup, sceVoiceChatRequestLeaveVoiceChatGroup, sceVoiceChatRequestDeleteVoiceChatGroup](../../core/libs/prx/libSceVoiceChat/Export.cpp) (libSceVoiceChat) - assumed to take a request id and two pointers like the other `sceVoiceChatRequest*` calls; offline, every argument is ignored and `SIGNED_OUT` is returned, as for those From 2828fdc8c6f21d57814c9d5a3b462983f5f9ab97 Mon Sep 17 00:00:00 2001 From: Joey Fisher Date: Wed, 7 Oct 2026 11:17:26 -0600 Subject: [PATCH 188/460] perf(agc): prove sampled depth surfaces in the fast resource proof --- .../Graphics/include/DepthSurface.hpp | 1 + .../Graphics/include/ShaderResources.hpp | 3 ++- .../Graphics/src/DepthSurface.cpp | 13 +++++++++++++ .../Graphics/src/ShaderResources.cpp | 18 +++++++++++++++--- 4 files changed, 31 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp index 04fef30c9..092f31fcf 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp @@ -18,6 +18,7 @@ void ClearDepthSurfaces(VkDevice device); bool DepthSurfaceAt(std::uint64_t address); std::uint64_t HtileDepthClearAddress(std::span code, std::span userData, const std::array& numThreads); void NoteHtileDepthClear(std::uint64_t htileAddress); +bool DepthSurfaceHolds(const Context& context, std::uint64_t address, const Texture* texture); std::shared_ptr DepthSurfaceTexture(const Context& context, std::span words, const GuestTextureResource& resource, VkComponentMapping components); } diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp index 53b5650c6..4b49f8f9f 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp @@ -213,7 +213,7 @@ public: bool DataWordsDiffer(const CompiledShader& shader) const; // Why the fast proof of a Revalidate left the object to the full walk (the [rescache] // revalidate line's reasons); Count: it did not. - enum class FastFail : std::size_t { NoRecord, Collect, Pending, Evicted, Changed, Keys, ClearedView, StorageKeys, Count }; + enum class FastFail : std::size_t { NoRecord, Collect, Pending, Evicted, Changed, Keys, ClearedView, StorageKeys, Depth, Count }; // Why a Pending failure was left to the full walk instead of the own-object refresh (T1, see // refreshOwnObjects); Count: it was not. enum class OwnRefreshFallback : std::size_t { Disabled, Snapshot, Keys, ForeignView, SurfaceKey, NotImported, Uncached, Rerun, Count }; @@ -371,6 +371,7 @@ private: std::uint64_t collected = 0; // Storage-sourced textures: the image the view follows (kept alive by the texture). const StorageTexture* source = nullptr; + bool depth = false; bool valid = false; }; void captureValidation(); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp index f7ace9d03..951b177e6 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp @@ -120,6 +120,10 @@ public: return texture; } + bool Holds(const Texture* texture) const { + return std::any_of(textures.begin(), textures.end(), [&](const auto& entry) { return entry.second.get() == texture; }); + } + const Context context; const DepthTarget target; VkImage image = VK_NULL_HANDLE; @@ -217,6 +221,15 @@ std::shared_ptr DepthSurfaceTexture(const Context& context, std::spanSampled(words, resource, components); } +bool DepthSurfaceHolds(const Context& context, std::uint64_t address, const Texture* texture) { + std::lock_guard lock(surfacesMutex()); + const auto& list = surfaces(); + const auto found = std::find_if(list.rbegin(), list.rend(), [&](const auto& surface) { + return surface->context.device == context.device && (surface->target.address == address || (surface->target.stencilAddress != 0 && surface->target.stencilAddress == address)); + }); + return found != list.rend() && (*found)->Holds(texture); +} + bool DepthSurfaceAt(std::uint64_t address) { std::lock_guard lock(surfacesMutex()); return std::any_of(surfaces().begin(), surfaces().end(), [&](const auto& surface) { return surface->target.address == address || surface->target.stencilAddress == address; }); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index 8afcbb6c1..5fbe2ec60 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -358,6 +358,7 @@ struct LookupRecord { DccKeys keys; std::uint64_t generation; const StorageTexture* source; + bool depth = false; }; thread_local std::vector* lookupLogSlot = nullptr; @@ -503,7 +504,10 @@ std::uint64_t sampledBudget(const Context& context, TextureCache& cache) { std::shared_ptr cachedTexture(const Context& context, std::span words, const GuestTextureResource& resource, VkComponentMapping components, std::uint64_t guestBytes = 0, bool depthCompare = false) { CaptureTrace::Log("sampled-lookup address=%llx width=%u height=%u dcc=%llx", static_cast(resource.baseAddress), resource.width, resource.height, static_cast(resource.dccAddress)); if (auto* recorder = Recorder::Active(); recorder != nullptr) recorder->BoundKeptBytes(); - if (auto depth = DepthSurfaceTexture(context, words, resource, components)) return depth; + if (auto depth = DepthSurfaceTexture(context, words, resource, components)) { + logLookup({depth.get(), resource, 0, DccKeys::Uncompressed, 0, nullptr, true}); + return depth; + } const auto depthBitsWidth = words.size() >= 4 ? ShaderRecompiler::DepthBitsTextureWidth(words[1], words[3]) : 0u; if (depthBitsWidth == 32u) { char text[160]; @@ -1672,7 +1676,7 @@ namespace { // of the reused ones, fast = every image proved current from stamps, full = the lookups were repeated. // Why the fast path left an object to the full walk (the "fast-fail by reason" counts). using FastFail = ShaderResources::FastFail; -constexpr std::array(FastFail::Count)> FastFailNames{"no record", "collect", "pending image", "evicted image", "memory changed", "keys", "cleared view", "storage keys"}; +constexpr std::array(FastFail::Count)> FastFailNames{"no record", "collect", "pending image", "evicted image", "memory changed", "keys", "cleared view", "storage keys", "depth surface"}; using OwnRefreshFallback = ShaderResources::OwnRefreshFallback; constexpr std::array(OwnRefreshFallback::Count)> OwnRefreshFallbackNames{"disabled", "snapshot texture", "cleared view", "foreign view", "surface key", "not imported", "uncached", "re-run failed"}; @@ -1824,7 +1828,7 @@ void ShaderResources::captureValidation() { }; validatedTextures.assign(textures.size(), {}); for (std::size_t i = 0; i < textures.size(); ++i) { - if (const auto* found = record(textures[i].get())) validatedTextures[i] = {found->resource, found->bytes, found->keys, found->generation, 0, found->source, true}; + if (const auto* found = record(textures[i].get())) validatedTextures[i] = {found->resource, found->bytes, found->keys, found->generation, 0, found->source, found->depth, true}; } lookupLog().clear(); } @@ -1880,6 +1884,10 @@ bool ShaderResources::fastRevalidate(std::uint64_t serialBefore, std::span(surface.bytes); surface.collected = GuestMemory::CollectWrites(address, bytes); @@ -1998,6 +2006,10 @@ bool ShaderResources::fastRevalidateEach() { for (std::size_t i = 0; i < textures.size(); ++i) { auto& surface = validatedTextures[i]; if (!surface.valid) return false; + if (surface.depth) { + if (!DepthSurfaceHolds(context, surface.resource.baseAddress, textures[i].get())) return false; + continue; + } const auto address = surface.resource.baseAddress; const auto bytes = static_cast(surface.bytes); surface.collected = GuestMemory::CollectWrites(address, bytes); From cdf303ea4bbc0f5e8c424f41f67c0eda8b6f2eb3 Mon Sep 17 00:00:00 2001 From: Joey Fisher Date: Wed, 7 Oct 2026 23:15:24 -0600 Subject: [PATCH 189/460] test(agc): prove a set sampling a depth surface on the fast path --- .../prx/libSceAgcDriver/tests/Recorder.cpp | 87 +++++++++++++++++++ 1 file changed, 87 insertions(+) diff --git a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp index 6950dc169..b155300fa 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp @@ -6,6 +6,7 @@ #include "prx/libSceAgcDriver/Graphics/include/Recorder.hpp" #include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" #include "prx/libSceAgcDriver/Graphics/include/DccMetadata.hpp" +#include "prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp" #include "prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp" #include "prx/libSceAgcDriver/Graphics/include/Resources.hpp" #include "prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp" @@ -3989,6 +3990,91 @@ void dataRefreshTests(const Device& device, Recorder& recorder) { recorder.Sync(); } +void depthSurfaceProofTests(const Device& device, Recorder& recorder) { + const auto& base = device.GetContext(); + if (!device.NullDescriptors()) { + std::cout << "nullDescriptor unavailable: depth surface fast proof not tested\n"; + return; + } + constexpr std::uint32_t side = 64; + constexpr std::size_t bytes = 65536; +#ifdef _WIN32 + void* block = VirtualAlloc(nullptr, bytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE); +#else + void* block = std::aligned_alloc(65536, bytes); +#endif + Require(block != nullptr, "cannot allocate the depth surface block"); + std::memset(block, 0, bytes); + const auto address = reinterpret_cast(block); + { + GuestAllocations::Mutation mutation; + mutation.Add(block, bytes, true, true, true); + } + struct Unregister { + const Context& context; + void* block; + ~Unregister() { + ClearDepthSurfaces(context.device); + ClearCachedTextures(context.device); + { + GuestAllocations::Mutation mutation; + mutation.Remove(block); + } +#ifdef _WIN32 + VirtualFree(block, 0, MEM_RELEASE); +#else + std::free(block); +#endif + } + } unregister{base, block}; + TextureDetiler detiler(base); + auto context = base; + context.detiler = &detiler; + context.nullDescriptors = true; + TextureCache cache(context); + context.textureCache = &cache; + const DepthTarget target{address, 0, {side, side}, VK_FORMAT_D32_SFLOAT, 1.0f, 0}; + Require(DepthSurfaceView(context, target) != VK_NULL_HANDLE, "the depth surface has no view"); + ShaderRecompiler::RecompileResult program; + ShaderRecompiler::DescriptorBinding binding; + binding.kind = ShaderRecompiler::DescriptorKind::SampledImage; + binding.role = ShaderRecompiler::DescriptorRole::GuestImages; + binding.descriptorSet = 0; + binding.binding = ShaderRecompiler::RuntimeAbi::FirstImageBinding; + binding.count = 1; + binding.imageShape = ShaderRecompiler::DescriptorImageShape::Image2D; + binding.imageSamplers = {0}; + binding.guestDescriptor = { + static_cast(address >> 8u), + static_cast((address >> 40u) & 0xffu) | (22u << 20u) | (((side - 1u) & 3u) << 30u), + ((side - 1u) >> 2u) | ((side - 1u) << 14u), + 0xfacu | (9u << 28u), + 0u, + 0u, + 0u, + 0u, + }; + program.bindings.push_back(binding); + const CompiledShader compute{ShaderRecompiler::ShaderStage::Compute, &program, 0}; + bool reusable = false; + std::array proved{}; + std::array reports{}; + { + ShaderResources resources(context, compute); + reusable = resources.Reusable(); + for (std::size_t use = 0; reusable && use < reports.size(); ++use) proved[use] = resources.Revalidate(compute, &reports[use]); + recorder.Submit(); + device.WaitQueue(); + recorder.Sync(); + } + Require(reusable, "a set sampling only a depth surface is not reusable"); + for (std::size_t use = 0; use < reports.size(); ++use) { + const auto name = std::to_string(use + 2); + Require(proved[use], "a set sampling an unchanged depth surface failed its proof on use " + name); + Require(reports[use].path == ShaderResources::ProofPath::Fast, "a set sampling an unchanged depth surface left the fast proof for the full walk on use " + name); + } +} + } class SampleProgram { @@ -4672,6 +4758,7 @@ int main(int argc, char** argv) { refreshProofTests(device, recorder); resourceReadTests(device, recorder); readWrittenStagingTests(device, recorder); + depthSurfaceProofTests(device, recorder); misalignedSnapshotTests(device, recorder); misalignedRegionTests(device, recorder); drawSnapshotReuseTests(device, recorder); From a8c4f86b4682fdb68085f34559c717e4f3d134f5 Mon Sep 17 00:00:00 2001 From: Joey Fisher Date: Thu, 8 Oct 2026 08:51:32 -0600 Subject: [PATCH 190/460] fix(agc): limit the depth surface fast proof to plain 2D lookups Array and layered depth views go back to the full walk, so a lookup that refreshes array slices is not skipped by the fast proof. --- .../libSceAgcDriver/Graphics/include/DepthSurface.hpp | 2 +- .../prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp | 4 +++- .../libSceAgcDriver/Graphics/src/ShaderResources.cpp | 4 ++-- core/libs/prx/libSceAgcDriver/tests/Recorder.cpp | 10 ++++++++++ 4 files changed, 16 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp index 092f31fcf..e2ab1bf30 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp @@ -18,7 +18,7 @@ void ClearDepthSurfaces(VkDevice device); bool DepthSurfaceAt(std::uint64_t address); std::uint64_t HtileDepthClearAddress(std::span code, std::span userData, const std::array& numThreads); void NoteHtileDepthClear(std::uint64_t htileAddress); -bool DepthSurfaceHolds(const Context& context, std::uint64_t address, const Texture* texture); +bool DepthSurfaceHolds(const Context& context, const GuestTextureResource& resource, const Texture* texture); std::shared_ptr DepthSurfaceTexture(const Context& context, std::span words, const GuestTextureResource& resource, VkComponentMapping components); } diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp index 951b177e6..9981bf384 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp @@ -221,7 +221,9 @@ std::shared_ptr DepthSurfaceTexture(const Context& context, std::spanSampled(words, resource, components); } -bool DepthSurfaceHolds(const Context& context, std::uint64_t address, const Texture* texture) { +bool DepthSurfaceHolds(const Context& context, const GuestTextureResource& resource, const Texture* texture) { + if (resource.dimension != TextureDimension::k2D || resource.baseArray != 0) return false; + const auto address = resource.baseAddress; std::lock_guard lock(surfacesMutex()); const auto& list = surfaces(); const auto found = std::find_if(list.rbegin(), list.rend(), [&](const auto& surface) { diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index 5fbe2ec60..cd5dcbfda 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -1885,7 +1885,7 @@ bool ShaderResources::fastRevalidate(std::uint64_t serialBefore, std::span Date: Sat, 10 Oct 2026 08:08:14 +0530 Subject: [PATCH 191/460] fix(agc): import GPU-mapped guest memory when it is mapped The registry calls an observer on the mapping thread, after the registry lock is released, with the ranges a kernel mapping gave GPU access: direct and flexible maps with it, and fixed remaps with it over a registered range (a reservation). The mutation looks them up while it still holds the lock and hands them over weakly, with the registry generation it ended at, so the observer does not keep a range alive. The driver imports those ranges on its current device, so an address-based build finds the import instead of pinning the memory on the queue that first uses it. A range that gains GPU access through sceKernelMprotect, and AnyPS5's own libc heap, are still imported on first use. The import at map does not reconcile the other imports with the registry, so its cost does not grow with the number of imports. It skips a range that is gone, and any range when the imports were reconciled with a newer registry since; the next lookup imports those as before. It installs the pin waiter as the other import paths do, so unmapping a range imported at map retires its import. The live import bytes are kept as a running total instead of being summed over every import. Once the import table belongs to the current device the import takes only the table's lock; the first import on a device (which runs the write-watch probe) and a device that was just replaced go through the GPU lock. Clearing the observer waits for calls already running. Co-Authored-By: Claude Opus 5.5 --- .../Execution/include/VulkanDevice.hpp | 2 + .../Execution/src/Driver/Lifecycle.cpp | 13 ++ .../Execution/src/VulkanDevice.cpp | 5 + .../Graphics/include/GuestBufferMemory.hpp | 1 + .../Graphics/src/GuestBufferMemory.cpp | 38 +++- .../tests/execution/HostImportAtMap.cpp | 182 ++++++++++++++++++ .../prx/libc/include/GuestAllocations.hpp | 4 + core/libs/prx/libc/src/GuestAllocations.cpp | 36 +++- .../libkernel/DirectMemory/DirectMemory.cpp | 3 + core/libs/tests/GuestMemory.cpp | 69 +++++++ 10 files changed, 347 insertions(+), 6 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/HostImportAtMap.cpp diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp index e7a9727f4..aedce62e3 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp @@ -9,6 +9,7 @@ #include "prx/libSceAgcDriver/Execution/include/Recipe.hpp" #include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" #include "prx/libSceAgcDriver/Graphics/include/Recorder.hpp" +#include "prx/libc/include/GuestAllocations.hpp" #include #include #include @@ -47,6 +48,7 @@ public: // three-step form below so the GPU wait happens without the mutex. void WaitIdle(); void PrepareForReplacement(); + bool ImportGuestMemory(const GuestAllocations::Mapped& ranges, std::uint64_t generation, bool adoptDevice); // Sends recorded work to the GPU without waiting for it. With `reapFirst` it first retires batches // that already finished, so the in-flight list stays short (APS5_NO_OPPORTUNISTIC_REAP=1 skips // that). A reap runs completion actions, and a write-back can wait for a later batch under the diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Lifecycle.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Lifecycle.cpp index ebe789113..ac5927eed 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Lifecycle.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Lifecycle.cpp @@ -4,6 +4,8 @@ #include "prx/libSceAgcDriver/Execution/include/WorkerSampler.hpp" #include "prx/libSceAgcDriver/Execution/include/ProfileOutput.hpp" #include "prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp" +#include "prx/libc/include/GuestAllocations.hpp" +#include namespace AgcDriver::DriverDetail { @@ -13,6 +15,7 @@ Driver& Driver::Get() { } Driver::~Driver() { + GuestAllocations::GuestAllocationsSetGpuMapObserver_nid_postfix(nullptr); stop(); } @@ -57,6 +60,16 @@ Driver::Driver() { stop(); throw; } + GuestAllocations::GuestAllocationsSetGpuMapObserver_nid_postfix([](const GuestAllocations::Mapped& ranges, std::uint64_t generation) { + try { + const auto current = Driver::Get().device.Load(); + if (current == nullptr || current->ImportGuestMemory(ranges, generation, false)) return; + std::lock_guard lock(GuestMemory::GpuMutex()); + if (Driver::Get().device.Load() == current) current->ImportGuestMemory(ranges, generation, true); + } catch (const std::exception& error) { + std::fprintf(stderr, "[gpu] import of GPU memory at its mapping failed: %s\n", error.what()); + } + }); } void Driver::WaitIdle() { diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index aa1135187..4baa35044 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -1381,6 +1381,11 @@ VulkanDevice::VulkanDevice(const PresentationWindow* window) : state(std::make_u VulkanDevice::~VulkanDevice() = default; +bool VulkanDevice::ImportGuestMemory(const GuestAllocations::Mapped& ranges, std::uint64_t generation, bool adoptDevice) { + if (adoptDevice) GuestMemory::AssertGpuLockHeld("VulkanDevice::ImportGuestMemory"); + return Graphics::ImportMappedRanges(graphicsContext(), ranges, generation, adoptDevice); +} + void VulkanDevice::PrepareForReplacement() { GuestMemory::AssertGpuLockHeld("VulkanDevice::PrepareForReplacement"); require(state->recorder != nullptr && Graphics::Recorder::Active() == state->recorder.get(), "device replacement requires its active recorder"); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp index 7aa407690..62c3833f6 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp @@ -55,6 +55,7 @@ void SetImportWatch(const Context& context, ImportWatch watch); // (alignment and budget permitting), or null. Bytes at `address` are at `address - import->base` in // the import's buffer. const HostImport* HostImportFor(const Context& context, std::uint64_t address, std::size_t bytes); +bool ImportMappedRanges(const Context& context, const GuestAllocations::Mapped& ranges, std::uint64_t generation, bool adoptDevice); // Whether an existing import covers [address, address + bytes), without reconciling the imports // with the registry or making one (HostImportFor may take a registry lease): a hint for choices // made outside the device lock (a sampled texture's path, a dispatch's pre-sync); the path taken diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp index 8b206eb5d..1c9a8125f 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp @@ -143,6 +143,7 @@ struct HostImports { PFN_vkDestroyBuffer destroyBuffer = nullptr; PFN_vkFreeMemory freeMemory = nullptr; std::map imports; + std::uint64_t liveBytes = 0; std::set failed; // Registry generation the imports were last reconciled with. std::uint64_t refreshedGeneration = 0; @@ -200,6 +201,7 @@ void retireImport(VkDevice device, HostImports& state, std::map(state.device, state.destroyBuffer, state.freeMemory, it->second); if (auto* recorder = Recorder::Active(); recorder != nullptr && !recorder->Idle()) recorder->Keep(std::move(holder)); ++state.epoch; + state.liveBytes -= it->second.bytes; state.imports.erase(it); } @@ -537,8 +539,7 @@ const HostImport* importAllocation(const Context& context, HostImports& state, s const char* value = std::getenv("APS5_HOST_IMPORT_MIB"); return (value ? std::strtoull(value, nullptr, 10) : 6144ull) << 20u; }(); - std::uint64_t live = 0; - for (const auto& [address, existing] : state.imports) live += existing.bytes; + const auto live = state.liveBytes; if (live + bytes > budget) { // A refused import turns every later use of the range into CPU copies, so say so. static std::uint64_t refused = 0, refusedBytes = 0; @@ -635,9 +636,8 @@ const HostImport* importAllocation(const Context& context, HostImports& state, s static std::uint64_t importedBytes = 0; importedBytes += bytes; static const bool trace = std::getenv("APS5_TRACE_HOST_IMPORT") != nullptr; - std::uint64_t liveBytes = bytes; - for (const auto& [address, existing] : state.imports) liveBytes += existing.bytes; - if (trace) std::fprintf(stderr, "[gpu] host import of 0x%llx+0x%llx ok (%zu live, %.1f MiB live, %.1f MiB ever)\n", static_cast(base), static_cast(bytes), state.imports.size() + 1, liveBytes / 1048576.0, importedBytes / 1048576.0); + state.liveBytes += bytes; + if (trace) std::fprintf(stderr, "[gpu] host import of 0x%llx+0x%llx ok (%zu live, %.1f MiB live, %.1f MiB ever)\n", static_cast(base), static_cast(bytes), state.imports.size() + 1, state.liveBytes / 1048576.0, importedBytes / 1048576.0); return &state.imports.emplace(base, entry).first->second; } @@ -669,6 +669,7 @@ void refreshImports(const Context& context, HostImports& state, const GuestAlloc #endif } state.imports.clear(); + state.liveBytes = 0; state.failed.clear(); state.device = context.device; state.destroyBuffer = context.Function("vkDestroyBuffer"); @@ -1492,6 +1493,7 @@ void ClearHostImports(VkDevice device) { #endif } state.imports.clear(); + state.liveBytes = 0; state.failed.clear(); state.device = VK_NULL_HANDLE; state.refreshedGeneration = 0; @@ -1682,6 +1684,32 @@ const HostImport* HostImportFor(const Context& context, std::uint64_t address, s return nullptr; } +bool ImportMappedRanges(const Context& context, const GuestAllocations::Mapped& ranges, std::uint64_t generation, bool adoptDevice) { + if (context.hostImportAlignment == 0) return true; + ensurePinWaiter(); + auto& state = Imports(); + std::lock_guard lock(state.mutex); + if (!adoptDevice && (state.device != context.device || state.watchDevice != context.device)) return false; + const bool adopting = state.device != context.device; + GuestAllocations::Lease current; + if (adopting) { + current = GuestAllocations::GuestAllocationsAcquire_nid_postfix(); + refreshImports(context, state, current); + } else if (state.refreshedGeneration > generation) { + return true; + } + for (const auto& mapped : ranges) { + const auto range = mapped.lock(); + if (range == nullptr || (adopting && leasedRangeOwner(current, range->address) != range)) continue; + try { + if (importAllocation(context, state, range->address, range->bytes, GuestAllocations::Lease{range}) == nullptr && GuestAllocations::GuestAllocationsGeneration_nid_postfix() != generation) state.failed.erase(range->address); + } catch (const std::exception& error) { + std::fprintf(stderr, "[gpu] import of GPU memory mapped at 0x%llx+0x%zx failed: %s\n", static_cast(range->address), range->bytes, error.what()); + } + } + return true; +} + bool RegisteredReadableCovers(std::uint64_t address, std::size_t bytes) { if (bytes == 0 || bytes > std::numeric_limits::max() - address) return false; const auto lease = GuestAllocations::GuestAllocationsAcquire_nid_postfix(); diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/HostImportAtMap.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/HostImportAtMap.cpp new file mode 100644 index 000000000..c337c09d8 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/HostImportAtMap.cpp @@ -0,0 +1,182 @@ +#include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp" +#include "prx/libc/include/GuestAllocations.hpp" +#include "VulkanTestDevice.hpp" +#ifdef _WIN32 +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include +#endif +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; + +constexpr std::size_t BlockBytes = 65536; +constexpr std::size_t AllocationBytes = BlockBytes * 4; + +class GuestAllocation { +public: + GuestAllocation() { +#ifdef _WIN32 + memory = static_cast(VirtualAlloc(nullptr, AllocationBytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); +#else + memory = static_cast(std::aligned_alloc(BlockBytes, AllocationBytes)); +#endif + Require(memory != nullptr, "host import at map: cannot allocate the guest memory"); + std::memset(memory, 0, AllocationBytes); + Register(); + } + + ~GuestAllocation() { + if (registered) Unregister(); +#ifdef _WIN32 + VirtualFree(memory, 0, MEM_RELEASE); +#else + std::free(memory); +#endif + } + + GuestAllocation(const GuestAllocation&) = delete; + GuestAllocation& operator=(const GuestAllocation&) = delete; + + std::uint64_t Address() const { return static_cast(reinterpret_cast(memory)); } + + void Register() { + GuestAllocations::Mutation().Add(memory, AllocationBytes, true, true, true); + registered = true; + } + + void Unregister() { + GuestAllocations::Mutation().Remove(memory); + registered = false; + } + + GuestAllocations::Mapped Ranges() const { + GuestAllocations::Mapped ranges; + for (const auto& range : GuestAllocations::GuestAllocationsAcquire_nid_postfix()) { + if (range->address == Address()) ranges.push_back(range); + } + Require(ranges.size() == 1, "host import at map: the allocation is not registered"); + return ranges; + } + +private: + std::uint8_t* memory = nullptr; + bool registered = false; +}; + +void Reconcile(AgcDriver::VulkanDevice& device, const GuestAllocation& probe) { + std::lock_guard lock(AgcDriver::GuestMemory::GpuMutex()); + const std::array pattern{5u, 6u, 7u, 8u}; + Require(device.FillBuffer(probe.Address(), BlockBytes, pattern), "host import at map: the probe fill was not imported"); + device.WaitIdle(); +} + +void Import(AgcDriver::VulkanDevice& device, const GuestAllocations::Mapped& ranges, std::uint64_t generation = GuestAllocations::GuestAllocationsGeneration_nid_postfix()) { + if (device.ImportGuestMemory(ranges, generation, false)) return; + std::lock_guard lock(AgcDriver::GuestMemory::GpuMutex()); + Require(device.ImportGuestMemory(ranges, generation, true), "host import at map: the import under the GPU lock refused"); +} + +bool Imported(VkDevice device, std::uint64_t address, std::size_t bytes) { + AgcDriver::Graphics::Context probe{}; + probe.device = device; + probe.hostImportAlignment = 1; + return AgcDriver::Graphics::HostImportCovers(probe, address, bytes); +} + +} + +int main() { + try { +#ifdef _WIN32 + _putenv_s("APS5_PIN_WAIT_MS", "2000"); +#else + setenv("APS5_PIN_WAIT_MS", "2000", 1); +#endif + auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + GuestAllocation first; + Import(*device, first.Ranges()); + const bool importedFirst = Imported(device->Device(), first.Address(), AllocationBytes); + first.Unregister(); + Require(!Imported(device->Device(), first.Address(), AllocationBytes), "host import at map: unmapping an allocation imported before any lookup kept its import"); + GuestAllocation probe; + bool filled = false; + { + std::lock_guard lock(AgcDriver::GuestMemory::GpuMutex()); + const std::array pattern{1u, 2u, 3u, 4u}; + filled = device->FillBuffer(probe.Address(), BlockBytes, pattern); + device->WaitIdle(); + } + if (!filled) { + std::printf("skipped, the device does not import guest memory\n"); + return VulkanTestSkipped; + } + Require(importedFirst, "host import at map: a mapping before any lookup was not imported"); + GuestAllocation guest; + const auto handle = device->Device(); + Require(!Imported(handle, guest.Address(), AllocationBytes), "host import at map: the allocation was imported before anything asked for it"); + Import(*device, guest.Ranges()); + Require(Imported(handle, guest.Address(), AllocationBytes), "host import at map: the mapped allocation was not imported"); + + GuestAllocation stale; + Import(*device, stale.Ranges()); + stale.Unregister(); + GuestAllocation mapped; + Import(*device, mapped.Ranges()); + Require(Imported(handle, mapped.Address(), AllocationBytes), "host import at map: a mapping made after another changed was not imported"); + + GuestAllocation remapped; + Import(*device, remapped.Ranges()); + remapped.Unregister(); + remapped.Register(); + Import(*device, remapped.Ranges()); + Require(Imported(handle, remapped.Address(), AllocationBytes), "host import at map: a range mapped again was not imported"); + GuestAllocation gone; + const auto goneRanges = gone.Ranges(); + gone.Unregister(); + Import(*device, goneRanges); + Require(!Imported(handle, gone.Address(), AllocationBytes), "host import at map: a mapping unregistered before its import was imported"); + gone.Register(); + + GuestAllocation late; + const auto lateRanges = late.Ranges(); + const auto lateAt = GuestAllocations::GuestAllocationsGeneration_nid_postfix(); + GuestAllocation newer; + Reconcile(*device, probe); + Import(*device, lateRanges, lateAt); + Require(!Imported(handle, late.Address(), AllocationBytes), "host import at map: an import older than the last reconcile was made"); + + GuestAllocation contended; + std::atomic held{false}; + std::atomic release{false}; + std::thread holder([&] { + std::lock_guard lock(AgcDriver::GuestMemory::GpuMutex()); + held = true; + while (!release) std::this_thread::yield(); + }); + while (!held) std::this_thread::yield(); + const bool unlocked = device->ImportGuestMemory(contended.Ranges(), GuestAllocations::GuestAllocationsGeneration_nid_postfix(), false); + release = true; + holder.join(); + Require(unlocked && Imported(handle, contended.Address(), AllocationBytes), "host import at map: a mapping on a device that already imports needed the GPU lock"); + std::puts("host import at map tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/libs/prx/libc/include/GuestAllocations.hpp b/core/libs/prx/libc/include/GuestAllocations.hpp index 90c911034..2640ee4ee 100644 --- a/core/libs/prx/libc/include/GuestAllocations.hpp +++ b/core/libs/prx/libc/include/GuestAllocations.hpp @@ -22,6 +22,7 @@ struct Range { }; using Lease = std::vector>; +using Mapped = std::vector>; extern "C" { void* GuestAllocationsBegin_nid_postfix(); @@ -54,6 +55,8 @@ void GuestAllocationsInvalidate_nid_postfix(std::uintptr_t address, std::size_t // nothing to wait for counts against the same bound as a plain spin). Without one the mutation spins // until the lease is dropped by another thread. void GuestAllocationsSetPinWaiter_nid_postfix(bool (*callback)(std::uintptr_t address, std::size_t bytes)); +void GuestAllocationsSetGpuMapObserver_nid_postfix(void (*callback)(const Mapped& ranges, std::uint64_t generation)); +void GuestAllocationsNoteGpuMapping_nid_postfix(void* mutation, const void* pointer, std::size_t bytes); } class Mutation { @@ -76,6 +79,7 @@ public: void Remove(const void* pointer) { GuestAllocationsRemove_nid_postfix(handle, pointer); } void Unmap(const void* pointer, std::size_t bytes, const std::function& apply) { GuestAllocationsUnmap_nid_postfix(handle, pointer, bytes, apply); } void Protect(const void* pointer, std::size_t bytes, bool readable, bool writable, bool gpu, const std::function& apply) { GuestAllocationsProtect_nid_postfix(handle, pointer, bytes, readable, writable, gpu, apply); } + void NoteGpuMapping(const void* pointer, std::size_t bytes) { GuestAllocationsNoteGpuMapping_nid_postfix(handle, pointer, bytes); } private: void* handle; diff --git a/core/libs/prx/libc/src/GuestAllocations.cpp b/core/libs/prx/libc/src/GuestAllocations.cpp index 8d77fa23d..60a2a8e09 100644 --- a/core/libs/prx/libc/src/GuestAllocations.cpp +++ b/core/libs/prx/libc/src/GuestAllocations.cpp @@ -1,4 +1,5 @@ #include "prx/libc/include/GuestAllocations.hpp" +#include #include #include #include @@ -45,6 +46,8 @@ Registry& registry() { std::atomic generation{1}; std::atomic invalidator{nullptr}; std::atomic pinWaiter{nullptr}; +std::atomic gpuMapObserver{nullptr}; +std::atomic gpuMapObserverCalls{0}; std::chrono::milliseconds pinWait() { static const std::chrono::milliseconds value{[] { @@ -64,6 +67,7 @@ void require(bool condition, const char* reason) { struct MutationState { std::unique_lock lock; std::vector> changed; + std::vector> gpuMapped; }; void recordChange(void* mutation, const void* pointer, std::size_t bytes) { @@ -110,11 +114,32 @@ void* GuestAllocationsBegin_nid_postfix() { void GuestAllocationsEnd_nid_postfix(void* mutation) noexcept { auto* state = static_cast(mutation); - generation.fetch_add(1, std::memory_order_release); + const auto ended = generation.fetch_add(1, std::memory_order_release) + 1; if (const auto callback = invalidator.load(std::memory_order_acquire)) { for (const auto& [address, bytes] : state->changed) callback(address, bytes); } + if (!state->gpuMapped.empty()) gpuMapObserverCalls.fetch_add(1); + const auto observer = state->gpuMapped.empty() ? nullptr : gpuMapObserver.load(); + Lease mapped; + if (observer != nullptr) { + const auto& ranges = registry().ranges; + for (const auto& [address, bytes] : state->gpuMapped) { + auto it = ranges.upper_bound(address); + if (it != ranges.begin()) --it; + for (; it != ranges.end() && it->first < address + bytes; ++it) { + const auto& range = it->second; + if (range->gpu && range->readable && range->bytes != 0 && range->address + range->bytes > address) mapped.push_back(range); + } + } + std::sort(mapped.begin(), mapped.end(), [](const auto& left, const auto& right) { return left->address < right->address; }); + mapped.erase(std::unique(mapped.begin(), mapped.end()), mapped.end()); + } + const Mapped weak(mapped.begin(), mapped.end()); + mapped.clear(); + const bool noted = !state->gpuMapped.empty(); delete state; + if (!weak.empty()) observer(weak, ended); + if (noted) gpuMapObserverCalls.fetch_sub(1); } std::uint64_t GuestAllocationsGeneration_nid_postfix() { @@ -130,6 +155,15 @@ void GuestAllocationsInvalidate_nid_postfix(std::uintptr_t address, std::size_t if (const auto callback = invalidator.load(std::memory_order_acquire)) callback(address, bytes); } +void GuestAllocationsSetGpuMapObserver_nid_postfix(void (*callback)(const Mapped& ranges, std::uint64_t generation)) { + gpuMapObserver.store(callback); + while (gpuMapObserverCalls.load() != 0) std::this_thread::yield(); +} + +void GuestAllocationsNoteGpuMapping_nid_postfix(void* mutation, const void* pointer, std::size_t bytes) { + if (mutation != nullptr && bytes != 0) static_cast(mutation)->gpuMapped.emplace_back(reinterpret_cast(pointer), bytes); +} + void GuestAllocationsSetPinWaiter_nid_postfix(bool (*callback)(std::uintptr_t, std::size_t)) { pinWaiter.store(callback, std::memory_order_release); } diff --git a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp index 4d083c3df..8fd7a4170 100644 --- a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp +++ b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp @@ -511,6 +511,7 @@ bool RemapFixedIntoRegistered(GuestAllocations::Mutation& mutation, void* addr, #endif } }); + if ((prot & GuestProtGpuReadWrite) != 0) mutation.NoteGpuMapping(addr, len); return true; } @@ -788,6 +789,7 @@ int DoMapDirect(void** addr, size_t len, int prot, int flags, int64_t physStart, try { AddMapping(reinterpret_cast(mapped), len, static_cast(physStart), LinuxProtFromSce(prot)); mutation.Add(mapped, len, (prot & 3) != 0, (prot & 2) != 0, (prot & GuestProtGpuReadWrite) != 0); + if ((prot & GuestProtGpuReadWrite) != 0) mutation.NoteGpuMapping(mapped, len); } catch (...) { EraseMappings(reinterpret_cast(mapped), reinterpret_cast(mapped) + len); Unmap(mapped, len); @@ -815,6 +817,7 @@ int DoMapAnon(void** addr, size_t len, int prot, int flags, size_t alignment) { void* mapped = MapAligned(*addr, len, LinuxProtFromSce(prot), flags, alignment); try { mutation.Add(mapped, len, (prot & 3) != 0, (prot & 2) != 0, (prot & GuestProtGpuReadWrite) != 0); + if ((prot & GuestProtGpuReadWrite) != 0) mutation.NoteGpuMapping(mapped, len); } catch (...) { Unmap(mapped, len); throw; diff --git a/core/libs/tests/GuestMemory.cpp b/core/libs/tests/GuestMemory.cpp index 74fd69a54..1e7d99326 100644 --- a/core/libs/tests/GuestMemory.cpp +++ b/core/libs/tests/GuestMemory.cpp @@ -466,6 +466,73 @@ static void CheckGpuAccessFollowsProtection() { Require(sceKernelMunmap(flexible, page) == 0); } +static std::vector> gpuMapped; + +static std::atomic observerEntered{false}; +static std::atomic observerDone{false}; + +static void CheckClearingTheObserverWaitsForItsCalls() { + GuestAllocations::GuestAllocationsSetGpuMapObserver_nid_postfix([](const GuestAllocations::Mapped&, std::uint64_t) { + observerEntered = true; + std::this_thread::sleep_for(std::chrono::milliseconds(100)); + observerDone = true; + }); + constexpr std::size_t page = 0x4000; + std::int64_t phys = 0; + Require(sceKernelAllocateDirectMemory(0, 0x7fffffffffll, page, 0, 0, &phys) == 0); + void* gpu = nullptr; + std::thread mapper([&] { Require(sceKernelMapDirectMemory(&gpu, page, 0x32, 0, phys, 0) == 0); }); + while (!observerEntered) std::this_thread::yield(); + GuestAllocations::GuestAllocationsSetGpuMapObserver_nid_postfix(nullptr); + Require(observerDone); + mapper.join(); + Require(sceKernelMunmap(gpu, page) == 0); + Require(sceKernelReleaseDirectMemory(phys, page) == 0); +} + +static void CheckGpuMapsAreObserved() { + GuestAllocations::GuestAllocationsSetGpuMapObserver_nid_postfix([](const GuestAllocations::Mapped& mapped, std::uint64_t) { + const auto lease = GuestAllocations::GuestAllocationsAcquire_nid_postfix(); + for (const auto& weak : mapped) { + const auto range = weak.lock(); + if (range == nullptr) continue; + const bool registered = std::any_of(lease.begin(), lease.end(), [&](const auto& current) { return current == range; }); + gpuMapped.emplace_back(registered && range->gpu ? range->address : 0, range->bytes); + } + }); + constexpr std::size_t page = 0x4000; + void* heap = GuestHeap::GuestHeapAlign_nid_postfix(page, page); + Require(heap != nullptr && gpuMapped.empty()); + GuestHeap::GuestHeapFree_nid_postfix(heap); + std::int64_t phys = 0; + Require(sceKernelAllocateDirectMemory(0, 0x7fffffffffll, page * 2, 0, 0, &phys) == 0); + void* cpu = nullptr; + Require(sceKernelMapDirectMemory(&cpu, page, 3, 0, phys, 0) == 0); + Require(gpuMapped.empty()); + void* gpu = nullptr; + Require(sceKernelMapDirectMemory(&gpu, page, 0x32, 0, phys + static_cast(page), 0) == 0); + Require(gpuMapped.size() == 1 && gpuMapped.front().first == reinterpret_cast(gpu) && gpuMapped.front().second == page); + Require(sceKernelMprotect(cpu, page, 0x13) == 0); + Require(gpuMapped.size() == 1); + std::int64_t remapPhys = 0; + Require(sceKernelAllocateDirectMemory(0, 0x7fffffffffll, page * 2, 0, 0, &remapPhys) == 0); + void* reserved = nullptr; + Require(sceKernelReserveVirtualRange(&reserved, page * 2, 0, 0) == 0); + void* cpuRemap = reserved; + Require(sceKernelMapDirectMemory(&cpuRemap, page, 3, 0x10, remapPhys, 0) == 0 && cpuRemap == reserved); + Require(gpuMapped.size() == 1); + void* gpuRemap = static_cast(reserved) + page; + void* const gpuTarget = gpuRemap; + Require(sceKernelMapDirectMemory(&gpuRemap, page, 0x32, 0x10, remapPhys + static_cast(page), 0) == 0 && gpuRemap == gpuTarget); + Require(gpuMapped.size() == 2 && gpuMapped.back().first == reinterpret_cast(gpuTarget) && gpuMapped.back().second == page); + GuestAllocations::GuestAllocationsSetGpuMapObserver_nid_postfix(nullptr); + Require(sceKernelMunmap(reserved, page * 2) == 0); + Require(sceKernelReleaseDirectMemory(remapPhys, page * 2) == 0); + Require(sceKernelMunmap(cpu, page) == 0); + Require(sceKernelMunmap(gpu, page) == 0); + Require(sceKernelReleaseDirectMemory(phys, page * 2) == 0); +} + static void CheckHintedVirtualReservation() { constexpr std::size_t page = 0x4000; void* probe = nullptr; @@ -1573,6 +1640,8 @@ int main() { CheckFixedMappingReplacesPartialOverlap(); CheckDirectMemoryGpuProtBits(); CheckGpuAccessFollowsProtection(); + CheckGpuMapsAreObserved(); + CheckClearingTheObserverWaitsForItsCalls(); CheckHintedVirtualReservation(); CheckFixedVirtualReservation(); CheckReservedRangeIsNotCommitted(); From f7a9629e68c9f8b2cd16daa1e14d9c38b4244170 Mon Sep 17 00:00:00 2001 From: StarMrGamer Date: Fri, 9 Oct 2026 19:32:00 +0800 Subject: [PATCH 192/460] fix(relinker): report the syscall number of a forbidden syscall instruction The syscall check names only the code offset, so a title that links a FreeBSD libc (the ps5-payload-dev SDK statically links one into every homebrew binary) stops with no indication of which call it reached. Those stubs load the number with a constant before the instruction (mov rax, 9 for link), so decode the nearest constant load into rax and add it to the message: Forbidden syscall instruction at code offset 0xa: syscall number 9 The number is reported only when it can be established. The constant load must be the instruction directly before the syscall, or one mov r10, rcx (the SDK's first-argument move, which does not write rax) in front of it; any other instruction stops the inference instead of returning a stale value, so mov rax, 9; add rax, 1; syscall reports no number rather than 9. Only the three invocation opcodes report a number: sysret is a return and keeps the current message, as does a stub that loads rax from a register, like the SDK's generic syscall() wrapper. The check still throws; only the text changes. Tested: relinker-only build, full ctest on Linux x86-64 (45/45). The C++ syscall_scanner test covers the opcode matrix and the review regression; the Python test_syscall_number covers the same contract through the harness the relinker lane already runs on Ubuntu, Windows and macOS, where the C++ fixture helpers (sys/mman.h, posix_spawn) do not build. AI-assisted: yes --- core/relinker/CMakeLists.txt | 7 ++ .../relinker/src/analysis/SyscallScanner.cpp | 115 +++++++++++++++--- .../relinker/tests/SyscallScannerTests.cpp | 107 ++++++++++++++++ .../relinker/tests/test_syscall_number.py | 66 ++++++++++ 4 files changed, 278 insertions(+), 17 deletions(-) create mode 100644 core/relinker/relinker/tests/SyscallScannerTests.cpp create mode 100644 core/relinker/relinker/tests/test_syscall_number.py diff --git a/core/relinker/CMakeLists.txt b/core/relinker/CMakeLists.txt index 4bc292b5f..28c495675 100644 --- a/core/relinker/CMakeLists.txt +++ b/core/relinker/CMakeLists.txt @@ -332,6 +332,8 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_init_array_roots.py $) add_test(NAME to_rosetta COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_to_rosetta.py $) + add_test(NAME syscall_number COMMAND ${Python3_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_syscall_number.py $) else() message(STATUS "Python3 interpreter not found; skipping Python relinker tests") endif() @@ -393,6 +395,11 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/RelaBoundsTests.cpp ) add_test(NAME rela_bounds COMMAND rela_bounds_tests $) + + add_test_executable(syscall_scanner_tests + ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/SyscallScannerTests.cpp + ) + add_test(NAME syscall_scanner COMMAND syscall_scanner_tests $) add_test_executable(guest_intel_trampolines_tests ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/GuestIntelTrampolinesTests.cpp ) diff --git a/core/relinker/relinker/src/analysis/SyscallScanner.cpp b/core/relinker/relinker/src/analysis/SyscallScanner.cpp index 8476e026f..028dce700 100644 --- a/core/relinker/relinker/src/analysis/SyscallScanner.cpp +++ b/core/relinker/relinker/src/analysis/SyscallScanner.cpp @@ -1,7 +1,10 @@ #include #include #include +#include +#include #include +#include #include namespace Relinker { @@ -14,6 +17,82 @@ static constexpr std::uint8_t SYSENTER_BYTE0 = 0x0F; static constexpr std::uint8_t SYSENTER_BYTE1 = 0x34; static constexpr std::uint8_t SYSRET_BYTE0 = 0x0F; static constexpr std::uint8_t SYSRET_BYTE1 = 0x07; +static constexpr std::size_t SYSCALL_NUMBER_LOOKBACK = 8; + +enum class ForbiddenInstruction { + None, + Invocation, + Return, +}; + +static ForbiddenInstruction ClassifyForbiddenInstruction( + const std::vector& codeSection, + const std::size_t opcodeOffset, + const std::size_t limit) { + if (opcodeOffset + 1 >= limit) + return ForbiddenInstruction::None; + + const std::uint8_t byte0 = codeSection[opcodeOffset]; + const std::uint8_t byte1 = codeSection[opcodeOffset + 1]; + + const bool isSyscall = (byte0 == SYSCALL_BYTE0 && byte1 == SYSCALL_BYTE1); + const bool isInt80 = (byte0 == INT80_BYTE0 && byte1 == INT80_BYTE1); + const bool isSysenter = (byte0 == SYSENTER_BYTE0 && byte1 == SYSENTER_BYTE1); + const bool isSysret = (byte0 == SYSRET_BYTE0 && byte1 == SYSRET_BYTE1); + + if (isSyscall || isInt80 || isSysenter) + return ForbiddenInstruction::Invocation; + + if (isSysret) + return ForbiddenInstruction::Return; + + return ForbiddenInstruction::None; +} + +static std::optional ConstantRaxLoad(const std::uint8_t* bytes, const std::size_t length) { + if (length == 7 && bytes[0] == 0x48 && bytes[1] == 0xC7 && bytes[2] == 0xC0) { + std::uint32_t value = 0; + std::memcpy(&value, bytes + 3, sizeof(value)); + return value; + } + + if (length == 5 && bytes[0] == 0xB8) { + std::uint32_t value = 0; + std::memcpy(&value, bytes + 1, sizeof(value)); + return value; + } + + return std::nullopt; +} + +static constexpr std::uint8_t MOV_R10_FROM_RCX[] = {0x49, 0x89, 0xCA}; + +static std::optional ConstantSyscallNumber( + const std::vector& codeSection, + const FileByteOffset codeSectionOffset, + const std::vector& matches, + const std::size_t syscallIndex) { + for (std::size_t step = 1; step <= SYSCALL_NUMBER_LOOKBACK && step <= syscallIndex; ++step) { + const auto& match = matches[syscallIndex - step]; + const std::size_t start = static_cast(match.Offset - codeSectionOffset); + + if (start >= codeSection.size() || match.Length > codeSection.size() - start) + return std::nullopt; + + const std::uint8_t* bytes = codeSection.data() + start; + + if (const auto value = ConstantRaxLoad(bytes, match.Length)) + return value; + + const bool movesFirstSyscallArgument = match.Length == sizeof(MOV_R10_FROM_RCX) + && std::memcmp(bytes, MOV_R10_FROM_RCX, sizeof(MOV_R10_FROM_RCX)) == 0; + + if (!movesFirstSyscallArgument) + return std::nullopt; + } + + return std::nullopt; +} class SyscallScanner : public ISyscallScanner { public: @@ -27,29 +106,31 @@ void SyscallScanner::ScanCodeSectionForSyscalls( const std::vector& codeSection, const FileByteOffset codeSectionOffset, const FileByteOffset codeSectionSize) { + const std::size_t limit = std::min(codeSection.size(), static_cast(codeSectionSize)); + const auto scanner = Codegen::MakeInstructionScanner(); const auto matches = scanner->ScanCodeSection(codeSection, codeSectionOffset, codeSectionSize); - for (const auto& [Offset, Length] : matches) { - const std::size_t i = Offset - codeSectionOffset; - const auto opcodeOffset = Codegen::DecodedInstruction{codeSection.data() + i, Length}.OpcodeOffset(); + for (std::size_t index = 0; index < matches.size(); ++index) { + const auto& match = matches[index]; + const std::size_t start = static_cast(match.Offset - codeSectionOffset); + const auto opcodeOffset = Codegen::DecodedInstruction{codeSection.data() + start, match.Length}.OpcodeOffset(); + const std::size_t opcode = start + opcodeOffset; + const std::size_t spanLimit = std::min(limit, start + static_cast(match.Length)); - if (opcodeOffset + 1 < Length) { - const std::uint8_t b0 = codeSection[i + opcodeOffset]; - const std::uint8_t b1 = codeSection[i + opcodeOffset + 1]; + const auto kind = ClassifyForbiddenInstruction(codeSection, opcode, spanLimit); - const bool isSyscall = (b0 == SYSCALL_BYTE0 && b1 == SYSCALL_BYTE1); - const bool isInt80 = (b0 == INT80_BYTE0 && b1 == INT80_BYTE1); - const bool isSysenter = (b0 == SYSENTER_BYTE0 && b1 == SYSENTER_BYTE1); - const bool isSysret = (b0 == SYSRET_BYTE0 && b1 == SYSRET_BYTE1); + if (kind == ForbiddenInstruction::None) + continue; - if (isSyscall || isInt80 || isSysenter || isSysret) { - const FileByteOffset instrOffset = codeSectionOffset + i; - std::ostringstream msg; - msg << "Forbidden syscall instruction at code offset 0x" << std::hex << instrOffset; - throw RelinkerException(msg.str(), instrOffset); - } - } + std::ostringstream msg; + msg << "Forbidden syscall instruction at code offset 0x" << std::hex << match.Offset; + + if (kind == ForbiddenInstruction::Invocation) + if (const auto number = ConstantSyscallNumber(codeSection, codeSectionOffset, matches, index)) + msg << ": syscall number " << std::dec << *number; + + throw RelinkerException(msg.str(), match.Offset); } } diff --git a/core/relinker/relinker/tests/SyscallScannerTests.cpp b/core/relinker/relinker/tests/SyscallScannerTests.cpp new file mode 100644 index 000000000..9c37f0446 --- /dev/null +++ b/core/relinker/relinker/tests/SyscallScannerTests.cpp @@ -0,0 +1,107 @@ +#include "ElfFixture.hpp" +#include "RelinkerProcess.hpp" +#include +#include +#include +#include + +namespace { + +using namespace RelinkerTests; + +void require(const bool condition, const std::string& message) { + if (!condition) throw std::runtime_error(message); +} + +const Bytes kConstantStub = { + 0x48, 0xC7, 0xC0, 0x09, 0x00, 0x00, 0x00, + 0x49, 0x89, 0xCA, + 0x0F, 0x05, + 0xC3, +}; + +const Bytes kUnsignedStub = { + 0xB8, 0x39, 0x00, 0x00, 0x00, + 0x0F, 0x05, + 0xC3, +}; + +const Bytes kRegisterStub = { + 0x48, 0x89, 0xF8, + 0x0F, 0x05, + 0xC3, +}; + +const Bytes kInt80Stub = { + 0xB8, 0x0B, 0x00, 0x00, 0x00, + 0xCD, 0x80, + 0xC3, +}; + +const Bytes kSysenterStub = { + 0x0F, 0x34, + 0xC3, +}; + +const Bytes kSysretStub = { + 0x48, 0xC7, 0xC0, 0x09, 0x00, 0x00, 0x00, + 0x0F, 0x07, + 0xC3, +}; + +const Bytes kClobberedStub = { + 0x48, 0xC7, 0xC0, 0x09, 0x00, 0x00, 0x00, + 0x48, 0x83, 0xC0, 0x01, + 0x0F, 0x05, + 0xC3, +}; + +std::string RunScanner(const std::string& binary, const Bytes& code) { + const TempDirectory directory; + const auto input = directory.Path() / "input.elf"; + const auto output = directory.Path() / "output.elf"; + WriteFile(input, MakeExecutable(code)); + const auto run = RunRelinker(binary, {"--skip-sce-module", input.string(), output.string()}, directory.Path() / "relinker.log"); + require(run.ExitCode != 0, "Relinker accepted a forbidden syscall instruction:\n" + run.Output); + return run.Output; +} + +void RequireNumber(const std::string& binary, const Bytes& code, const std::string& expected, const std::string& description) { + const auto output = RunScanner(binary, code); + require(output.find("Forbidden syscall instruction at code offset 0x") != std::string::npos, + "Relinker did not report the " + description + " as forbidden:\n" + output); + require(output.find(expected) != std::string::npos, + "Relinker did not report " + expected + " for the " + description + ":\n" + output); +} + +void RequireNoNumber(const std::string& binary, const Bytes& code, const std::string& description) { + const auto output = RunScanner(binary, code); + require(output.find("Forbidden syscall instruction at code offset 0x") != std::string::npos, + "Relinker did not report the " + description + " as forbidden:\n" + output); + require(output.find("syscall number") == std::string::npos, + "Relinker reported a syscall number for the " + description + ":\n" + output); +} + +} + +int main(const int argc, char** argv) { + try { + require(argc == 2, "usage: syscall_scanner_tests "); + const std::string binary = argv[1]; + + RequireNumber(binary, kConstantStub, "syscall number 9", "64-bit constant stub"); + RequireNumber(binary, kUnsignedStub, "syscall number 57", "32-bit constant stub"); + RequireNumber(binary, kInt80Stub, "syscall number 11", "int 0x80 stub"); + + RequireNoNumber(binary, kRegisterStub, "stub that loads the number from a register"); + RequireNoNumber(binary, kSysenterStub, "sysenter stub"); + RequireNoNumber(binary, kSysretStub, "sysret stub with a constant load in front of it"); + RequireNoNumber(binary, kClobberedStub, "stub whose intervening instruction writes rax"); + + std::cout << "syscall scanner tests passed\n"; + return 0; + } catch (const std::exception& error) { + std::cerr << "syscall_scanner_tests: " << error.what() << '\n'; + return 1; + } +} diff --git a/core/relinker/relinker/tests/test_syscall_number.py b/core/relinker/relinker/tests/test_syscall_number.py new file mode 100644 index 000000000..77d7d5f17 --- /dev/null +++ b/core/relinker/relinker/tests/test_syscall_number.py @@ -0,0 +1,66 @@ +"""Name the FreeBSD syscall a forbidden syscall stub reaches, when that can be established.""" + +from pathlib import Path +import struct +import subprocess +import sys +import tempfile + +CONSTANT = bytes.fromhex("48c7c009000000") + bytes.fromhex("4989ca") + b"\x0f\x05\xc3" +UNSIGNED = bytes.fromhex("b839000000") + b"\x0f\x05\xc3" +INT80 = bytes.fromhex("b80b000000") + b"\xcd\x80\xc3" +REGISTER = bytes.fromhex("4889f8") + b"\x0f\x05\xc3" +SYSENTER = b"\x0f\x34\xc3" +SYSRET = bytes.fromhex("48c7c009000000") + b"\x0f\x07\xc3" +CLOBBERED = bytes.fromhex("48c7c009000000") + bytes.fromhex("4883c001") + b"\x0f\x05\xc3" + + +def fixture(code): + """An executable whose first bytes are code, so the instruction scan starts on it.""" + image = bytearray(0x8000) + image[:16] = b"\x7fELF\x02\x01\x01" + bytes(9) + struct.pack_into(" Date: Sat, 10 Oct 2026 16:37:22 +0800 Subject: [PATCH 193/460] fix(relinker): resolve the syscall number through prefixed instructions The lookback classifies preceding instructions at their decoded opcode offset, the same place the forbidden-instruction check looks since #1935, so a legacy-prefixed mov rax, imm or mov r10, rcx is recognized. The REX byte stays part of the patterns, so mov r8 and mov r8d cannot masquerade as rax loads. Anything unrecognized still stops the inference. Adds prefixed fixtures to both test suites: a prefixed constant stub that must report syscall number 9, and a prefixed clobbered stub that must report no number. AI-assisted: yes. The submitting author is responsible for every changed line. --- .../relinker/src/analysis/SyscallScanner.cpp | 20 +++++++++++-------- .../relinker/tests/SyscallScannerTests.cpp | 17 ++++++++++++++++ .../relinker/tests/test_syscall_number.py | 8 ++++++-- 3 files changed, 35 insertions(+), 10 deletions(-) diff --git a/core/relinker/relinker/src/analysis/SyscallScanner.cpp b/core/relinker/relinker/src/analysis/SyscallScanner.cpp index 028dce700..45fa513c8 100644 --- a/core/relinker/relinker/src/analysis/SyscallScanner.cpp +++ b/core/relinker/relinker/src/analysis/SyscallScanner.cpp @@ -50,23 +50,25 @@ static ForbiddenInstruction ClassifyForbiddenInstruction( } static std::optional ConstantRaxLoad(const std::uint8_t* bytes, const std::size_t length) { - if (length == 7 && bytes[0] == 0x48 && bytes[1] == 0xC7 && bytes[2] == 0xC0) { + const std::size_t opcode = Codegen::DecodedInstruction{bytes, length}.OpcodeOffset(); + const bool hasRex = opcode > 0 && (bytes[opcode - 1] & 0xF0) == 0x40; + const std::uint8_t rex = hasRex ? bytes[opcode - 1] : 0; + + if (hasRex && rex == 0x48 && length - opcode == 6 && bytes[opcode] == 0xC7 && bytes[opcode + 1] == 0xC0) { std::uint32_t value = 0; - std::memcpy(&value, bytes + 3, sizeof(value)); + std::memcpy(&value, bytes + opcode + 2, sizeof(value)); return value; } - if (length == 5 && bytes[0] == 0xB8) { + if (!hasRex && length - opcode == 5 && bytes[opcode] == 0xB8) { std::uint32_t value = 0; - std::memcpy(&value, bytes + 1, sizeof(value)); + std::memcpy(&value, bytes + opcode + 1, sizeof(value)); return value; } return std::nullopt; } -static constexpr std::uint8_t MOV_R10_FROM_RCX[] = {0x49, 0x89, 0xCA}; - static std::optional ConstantSyscallNumber( const std::vector& codeSection, const FileByteOffset codeSectionOffset, @@ -84,8 +86,10 @@ static std::optional ConstantSyscallNumber( if (const auto value = ConstantRaxLoad(bytes, match.Length)) return value; - const bool movesFirstSyscallArgument = match.Length == sizeof(MOV_R10_FROM_RCX) - && std::memcmp(bytes, MOV_R10_FROM_RCX, sizeof(MOV_R10_FROM_RCX)) == 0; + const std::size_t opcode = Codegen::DecodedInstruction{bytes, match.Length}.OpcodeOffset(); + const bool hasRex = opcode > 0 && (bytes[opcode - 1] & 0xF0) == 0x40; + const bool movesFirstSyscallArgument = hasRex && bytes[opcode - 1] == 0x49 + && match.Length - opcode == 2 && bytes[opcode] == 0x89 && bytes[opcode + 1] == 0xCA; if (!movesFirstSyscallArgument) return std::nullopt; diff --git a/core/relinker/relinker/tests/SyscallScannerTests.cpp b/core/relinker/relinker/tests/SyscallScannerTests.cpp index 9c37f0446..3b7846a8e 100644 --- a/core/relinker/relinker/tests/SyscallScannerTests.cpp +++ b/core/relinker/relinker/tests/SyscallScannerTests.cpp @@ -56,6 +56,20 @@ const Bytes kClobberedStub = { 0xC3, }; +const Bytes kPrefixedConstantStub = { + 0x66, 0x48, 0xC7, 0xC0, 0x09, 0x00, 0x00, 0x00, + 0xF2, 0x49, 0x89, 0xCA, + 0xF3, 0x0F, 0x05, + 0xC3, +}; + +const Bytes kPrefixedClobberedStub = { + 0x66, 0x48, 0xC7, 0xC0, 0x09, 0x00, 0x00, 0x00, + 0x66, 0x48, 0x83, 0xC0, 0x01, + 0xF3, 0x0F, 0x05, + 0xC3, +}; + std::string RunScanner(const std::string& binary, const Bytes& code) { const TempDirectory directory; const auto input = directory.Path() / "input.elf"; @@ -98,6 +112,9 @@ int main(const int argc, char** argv) { RequireNoNumber(binary, kSysretStub, "sysret stub with a constant load in front of it"); RequireNoNumber(binary, kClobberedStub, "stub whose intervening instruction writes rax"); + RequireNumber(binary, kPrefixedConstantStub, "syscall number 9", "prefixed constant stub"); + RequireNoNumber(binary, kPrefixedClobberedStub, "prefixed stub whose intervening instruction writes rax"); + std::cout << "syscall scanner tests passed\n"; return 0; } catch (const std::exception& error) { diff --git a/core/relinker/relinker/tests/test_syscall_number.py b/core/relinker/relinker/tests/test_syscall_number.py index 77d7d5f17..d9f47031d 100644 --- a/core/relinker/relinker/tests/test_syscall_number.py +++ b/core/relinker/relinker/tests/test_syscall_number.py @@ -13,6 +13,8 @@ REGISTER = bytes.fromhex("4889f8") + b"\x0f\x05\xc3" SYSENTER = b"\x0f\x34\xc3" SYSRET = bytes.fromhex("48c7c009000000") + b"\x0f\x07\xc3" CLOBBERED = bytes.fromhex("48c7c009000000") + bytes.fromhex("4883c001") + b"\x0f\x05\xc3" +PREFIXED_CONSTANT = bytes.fromhex("6648c7c009000000") + bytes.fromhex("f24989ca") + bytes.fromhex("f30f05") + b"\xc3" +PREFIXED_CLOBBERED = bytes.fromhex("6648c7c009000000") + bytes.fromhex("664883c001") + bytes.fromhex("f30f05") + b"\xc3" def fixture(code): @@ -50,12 +52,14 @@ def main(): for name, code, expected in (("constant", CONSTANT, "syscall number 9"), ("unsigned", UNSIGNED, "syscall number 57"), - ("int80", INT80, "syscall number 11")): + ("int80", INT80, "syscall number 11"), + ("prefixed-constant", PREFIXED_CONSTANT, "syscall number 9")): if expected not in reject(name, code): raise AssertionError((name, "did not report " + expected)) for name, code in (("register", REGISTER), ("sysenter", SYSENTER), - ("sysret", SYSRET), ("clobbered", CLOBBERED)): + ("sysret", SYSRET), ("clobbered", CLOBBERED), + ("prefixed-clobbered", PREFIXED_CLOBBERED)): stderr = reject(name, code) if "syscall number" in stderr: raise AssertionError((name, "reported a number it cannot establish", stderr)) From 00fafd437999f68d46d6b0b5ad6bd51717b0f5f8 Mon Sep 17 00:00:00 2001 From: MathiasDevelopes <25400788+MathiasDevelopes@users.noreply.github.com> Date: Fri, 9 Oct 2026 16:27:24 +0200 Subject: [PATCH 194/460] fix(agc): count copied guest buffers and draw snapshots in the batch byte budget The recorder bounds the bytes a batch keeps alive (KeptBytesBudget) and waits for older batches past twice that. A recorded dispatch or draw kept its ShaderResources, and a draw its snapshot buffers, with a size of 0, so the host-visible copies of guest buffers they own never counted. A title that binds a 124 MiB writable guest buffer in every draw then held about ninety copies, 11 GB, until the GPU caught up, and aborted with VK_ERROR_OUT_OF_HOST_MEMORY from vkAllocateMemory. GuestBufferMemory reports the bytes of its copied regions, and the dispatch and draw keeps and the draw snapshots pass their bytes to Recorder::Keep, so the existing bound applies to them. Co-Authored-By: Claude Sonnet 5.5 --- core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp | 2 +- .../libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp | 5 +++++ .../prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp | 1 + core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp | 2 +- .../prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp | 4 +++- core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp | 2 ++ 6 files changed, 13 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index aa1135187..ed683ed90 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -3485,7 +3485,7 @@ void VulkanDevice::recordDispatch(RecordedDispatch& record) { const auto commands = recorder.Commands(&covered); recordStep(PhaseRecordCommands); recorder.Keep(record.objects); - recorder.Keep(record.resources); + recorder.Keep(record.resources, record.resources != nullptr ? record.resources->CopiedBytes() : 0u); recordStep(PhaseRecordKeeps); if (record.dataRefresh != RecordedDispatch::DataRefresh::None) { // The template's data buffers take this dispatch's words: a transfer write the pre-dispatch diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp index 7aa407690..cea14f48c 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp @@ -243,6 +243,11 @@ public: // Whether registered allocations are pinned until write-back (address-based shaders): by this // build's own lease, or by the cached address space it holds. bool HoldsLease() const { return !lease.empty() || space != nullptr; } + std::size_t CopiedBytes() const { + std::size_t bytes = 0; + for (const auto& region : regions) if (region.buffer != nullptr) bytes += static_cast(region.end - region.begin); + return bytes; + } // Every uploaded region as [begin, end) when all of them are served by host imports, in place or // through a device-local staging copy of the import (nothing was copied through the CPU, so the // upload can serve a later identical build), else nothing. diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp index 53b5650c6..3539c7019 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp @@ -149,6 +149,7 @@ public: // Whether a written buffer was copied (its results reach guest memory by the CPU write-back). bool HasCopiedWrites() const { return guestMemory.HasCopiedWrites(); } bool HoldsLease() const { return guestMemory.HoldsLease(); } + std::size_t CopiedBytes() const { return guestMemory.CopiedBytes(); } bool WritesOverlap(std::uint64_t address, std::size_t bytes) const { return guestMemory.WritesOverlap(address, bytes); } // Whether a region the recorded work reads in place through a host import overlaps the range. bool ReadsOverlap(std::uint64_t address, std::size_t bytes) const; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp index fb6a8680e..2af257b28 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp @@ -1432,7 +1432,7 @@ void keepRecordedDraw(Recorder& recorder, const std::shared_ptr kept->scratch = std::move(scratch); kept->targets = std::move(targets); const auto& residents = kept->targets; - recorder.Keep(kept); + recorder.Keep(kept, kept->resources != nullptr ? kept->resources->CopiedBytes() : 0u); if (checkRecords) recorder.OnComplete(std::move(checkRecords)); // Counted for the [address-sync] line: a lease released by the completion (the pin waiter // finishes the recorder up to the open batch, which holds the kept resources and gets the diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index 8afcbb6c1..1033234fb 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -3368,7 +3368,9 @@ std::shared_ptr ShaderResources::PrepareDrawBindi writes.push_back(write); } update(context.device, static_cast(writes.size()), writes.data(), 0, nullptr); - recorder.Keep(result); + std::size_t snapshotBytes = 0; + for (const auto& snapshot : result->snapshots) snapshotBytes += snapshot.buffer->Bytes().size(); + recorder.Keep(result, snapshotBytes); return result; } diff --git a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp index fdf13a5c5..79403d4cf 100644 --- a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp @@ -576,7 +576,9 @@ void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { GuestBufferMemory memory(context); memory.AddWritable(address, sizeof(guest)); memory.AddWritable(address + 16, 16); + Require(memory.CopiedBytes() == 0, "a region counted as copied before its upload"); memory.Upload(true); + Require(memory.CopiedBytes() == 0 || memory.CopiedBytes() == sizeof(guest), "copied bytes do not match the one region copied"); std::uint32_t adjustment = 0; const auto first = memory.Descriptor(address, sizeof(guest), adjustment); Require(adjustment == 0, "a view at its owner's start is bound off it"); From 4940de5436a449ad6685ca25bd9b821a7b5419d1 Mon Sep 17 00:00:00 2001 From: MathiasDevelopes <25400788+MathiasDevelopes@users.noreply.github.com> Date: Sat, 10 Oct 2026 09:44:30 +0000 Subject: [PATCH 195/460] fix(agc): count memory shared by a batch's keeps once and test the three keep sites A ShaderResources from the resource caches is the same object for every draw or dispatch that uses it, and Recorder::ReusableDrawSnapshot hands the next draw the same snapshot buffer, so passing their bytes with each Keep counted one allocation once per use and submitted the batch before it held the budget. Recorder::KeepBytes counts an owner's bytes once per batch, and the dispatch, draw and snapshot keeps use it. The recorded kept bytes test runs a dispatch and a draw that copy a guest buffer and two draws that snapshot one on the device, and fails when any of the three sites stops counting or a reused snapshot counts twice. The BdaResources check pins the copied region it sets up. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01HZHkb8kUwnHAuBBbHywvJf --- .../Execution/src/VulkanDevice.cpp | 3 +- .../Graphics/include/Recorder.hpp | 10 + .../prx/libSceAgcDriver/Graphics/src/Draw.cpp | 3 +- .../libSceAgcDriver/Graphics/src/Recorder.cpp | 6 + .../Graphics/src/ShaderResources.cpp | 6 +- .../libSceAgcDriver/tests/BdaResources.cpp | 4 +- .../tests/RecorderKeptBytes.cpp | 23 ++ .../tests/execution/RecordedKeptBytes.cpp | 319 ++++++++++++++++++ 8 files changed, 368 insertions(+), 6 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index ed683ed90..f2a14508f 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -3485,7 +3485,8 @@ void VulkanDevice::recordDispatch(RecordedDispatch& record) { const auto commands = recorder.Commands(&covered); recordStep(PhaseRecordCommands); recorder.Keep(record.objects); - recorder.Keep(record.resources, record.resources != nullptr ? record.resources->CopiedBytes() : 0u); + recorder.Keep(record.resources); + if (record.resources != nullptr) recorder.KeepBytes(record.resources.get(), record.resources->CopiedBytes()); recordStep(PhaseRecordKeeps); if (record.dataRefresh != RecordedDispatch::DataRefresh::None) { // The template's data buffers take this dispatch's words: a transfer write the pre-dispatch diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp index d8888787d..490d2479e 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp @@ -17,6 +17,7 @@ #include #include #include +#include #include #include @@ -85,6 +86,13 @@ public: // Whether recorded work still has completion actions (write-backs the CPU must see) to run. bool HasCompletions() const; void Keep(std::shared_ptr object, std::size_t bytes = 0); + // Counts `bytes` held by `owner` in the open batch's kept bytes once per batch: memory several + // keeps of one batch share (a cached ShaderResources kept by every draw or dispatch using it, a + // reused draw snapshot) counts once. The caller keeps `owner` alive in the same batch, so its + // address names one object for the batch's lifetime. + void KeepBytes(const void* owner, std::size_t bytes); + // The kept bytes of the open batch (0 with none open), for tests. + std::size_t OpenKeptBytes() const { return open != nullptr ? open->keptBytes : 0; } static constexpr std::size_t KeptBytesBudget = std::size_t{512} << 20u; void BoundKeptBytes(); std::size_t InFlightKeptBytes() const { return inFlightKeptBytes; } @@ -511,6 +519,8 @@ private: VkFence fence = VK_NULL_HANDLE; std::vector> kept; std::size_t keptBytes = 0; + // The owners KeepBytes counted in this batch. + std::unordered_set keptOwners; std::vector> completions; std::vector> writes; std::vector writeNotes; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp index 2af257b28..6b808d864 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp @@ -1432,7 +1432,8 @@ void keepRecordedDraw(Recorder& recorder, const std::shared_ptr kept->scratch = std::move(scratch); kept->targets = std::move(targets); const auto& residents = kept->targets; - recorder.Keep(kept, kept->resources != nullptr ? kept->resources->CopiedBytes() : 0u); + recorder.Keep(kept); + if (resources != nullptr) recorder.KeepBytes(resources.get(), resources->CopiedBytes()); if (checkRecords) recorder.OnComplete(std::move(checkRecords)); // Counted for the [address-sync] line: a lease released by the completion (the pin waiter // finishes the recorder up to the open batch, which holds the kept resources and gets the diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Recorder.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Recorder.cpp index 5297bd42f..fea9b921a 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Recorder.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Recorder.cpp @@ -2452,6 +2452,12 @@ void Recorder::Keep(std::shared_ptr object, std::size_t bytes) { open->keptBytes += bytes; } +void Recorder::KeepBytes(const void* owner, std::size_t bytes) { + if (owner == nullptr || bytes == 0) return; + ensureOpen(); + if (open->keptOwners.insert(owner).second) open->keptBytes += bytes; +} + void Recorder::BoundKeptBytes() { if (!GuestMemory::GpuMutex().HeldByThisThread()) return; if (open != nullptr && open->keptBytes >= KeptBytesBudget) Submit(); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index 1033234fb..f5b41011e 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -3368,9 +3368,9 @@ std::shared_ptr ShaderResources::PrepareDrawBindi writes.push_back(write); } update(context.device, static_cast(writes.size()), writes.data(), 0, nullptr); - std::size_t snapshotBytes = 0; - for (const auto& snapshot : result->snapshots) snapshotBytes += snapshot.buffer->Bytes().size(); - recorder.Keep(result, snapshotBytes); + recorder.Keep(result); + // A reused snapshot (Recorder::ReusableDrawSnapshot) is one buffer however many draws bind it. + for (const auto& snapshot : result->snapshots) recorder.KeepBytes(snapshot.buffer.get(), snapshot.buffer->Bytes().size()); return result; } diff --git a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp index 79403d4cf..2ad77ce62 100644 --- a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp @@ -578,7 +578,9 @@ void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { memory.AddWritable(address + 16, 16); Require(memory.CopiedBytes() == 0, "a region counted as copied before its upload"); memory.Upload(true); - Require(memory.CopiedBytes() == 0 || memory.CopiedBytes() == sizeof(guest), "copied bytes do not match the one region copied"); + // `guest` is stack memory no allocation registers, so no host import serves it: its two views merge + // into one copied region. + Require(memory.CopiedBytes() == sizeof(guest), "copied bytes " + std::to_string(memory.CopiedBytes()) + " do not match the one region copied"); std::uint32_t adjustment = 0; const auto first = memory.Descriptor(address, sizeof(guest), adjustment); Require(adjustment == 0, "a view at its owner's start is bound off it"); diff --git a/core/libs/prx/libSceAgcDriver/tests/RecorderKeptBytes.cpp b/core/libs/prx/libSceAgcDriver/tests/RecorderKeptBytes.cpp index a8d342563..4997d6be4 100644 --- a/core/libs/prx/libSceAgcDriver/tests/RecorderKeptBytes.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/RecorderKeptBytes.cpp @@ -194,6 +194,28 @@ void CountFollowsEveryPath() { Expect(recorder.Reap() && recorder.InFlightKeptBytes() == 0, "a reaped batch still counts " + std::to_string(recorder.InFlightKeptBytes()) + " kept bytes in flight"); } +void SharedOwnerCountsOncePerBatch() { + mock = MockDevice{}; + std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex()); + Recorder recorder(mockContext()); + auto shared = std::make_shared(0); + for (int keep = 0; keep < 3; ++keep) { + recorder.Keep(shared); + recorder.KeepBytes(shared.get(), Budget / 2); + recorder.BoundKeptBytes(); + } + Expect(recorder.Recording() && recorder.OpenKeptBytes() == Budget / 2 && mock.submits == 0, "three keeps of one owner's half budget counted " + std::to_string(recorder.OpenKeptBytes()) + " kept bytes, not half the budget once"); + auto other = std::make_shared(1); + recorder.Keep(other); + recorder.KeepBytes(other.get(), Budget / 2); + recorder.BoundKeptBytes(); + Expect(!recorder.Recording() && recorder.Submissions() == 1 && recorder.InFlightKeptBytes() == Budget, "a second owner's half budget did not complete the batch's budget"); + recorder.Keep(shared); + recorder.KeepBytes(shared.get(), Budget / 2); + Expect(recorder.OpenKeptBytes() == Budget / 2, "the next batch did not count an owner the submitted batch counted"); + recorder.Sync(); +} + } int main() { @@ -202,6 +224,7 @@ int main() { InFlightWithinTwiceTheBudget(); SyncedWithinTwiceTheBudget(); CountFollowsEveryPath(); + SharedOwnerCountsOncePerBatch(); } catch (const std::exception& error) { std::fprintf(stderr, "%s\n", error.what()); return 1; diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp new file mode 100644 index 000000000..b82cc95b2 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp @@ -0,0 +1,319 @@ +#include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/ColorTargetLayout.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Recorder.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Texture.hpp" +#include "prx/libc/include/GuestAllocations.hpp" +#include "prx/libc/include/GuestArena.hpp" +#include "prx/libc/include/GuestWriteWatch.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#ifdef _WIN32 +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include +#else +#include +#endif +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// The bytes a recorded dispatch or draw holds alive reach the recorder's batch budget +// (Recorder::KeptBytesBudget): the copies of guest buffers its ShaderResources own (kept by +// VulkanDevice::recordDispatch and Draw.cpp's keepRecordedDraw) and a draw's snapshots of buffers it +// reads in place (ShaderResources::PrepareDrawBindings). A snapshot reused by the next draw of the +// batch is one buffer and counts once. + +namespace { + +using AgcDriver::Graphics::ColorTargetLayout; +using AgcDriver::Graphics::ColorTileMode; +using AgcDriver::Graphics::Recorder; +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 32; +constexpr std::size_t BufferBytes = 65536; +constexpr std::uint32_t Width = 256; +constexpr std::uint32_t Height = 128; + +// v1 = v0 << 3; v[2:3] = 0; buffer_store_dwordx2 v[2:3], v1, s[0:3] offen +alignas(256) constexpr std::array StoreCode{ + 0x34020083, 0x7e040280, 0x7e060280, 0xe0741000, 0x80000201, 0xbf810000, +}; + +// buffer_load_format_xyzw v[0:3], v5, s[0:3] idxen; exp pos0 v[0:3] done; s_endpgm +alignas(256) constexpr std::array VertexCode{ + 0xe0382000, 0x80000005, 0xbf8c3f70, 0xf80008cf, 0x03020100, 0xbf810000, +}; + +// v7 = 1.0; exp mrt0 v7 v7 v7 v7 done vm; s_endpgm +alignas(256) constexpr std::array PixelCode{ + 0x7e0e02f2, 0xf800180f, 0x07070707, 0xbf810000, +}; + +std::uint64_t AddressOf(const void* data) { + return static_cast(reinterpret_cast(data)); +} + +std::array RawBuffer(const void* data, std::uint32_t bytes) { + const auto address = AddressOf(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; +} + +std::array VertexBuffer(const void* data, std::uint32_t stride, std::uint32_t count) { + const auto address = AddressOf(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu) | (stride << 16u), count, 0x01016facu}; +} + +// Guest memory, registered (served in place by a host import where the device has them, and +// write-watched where the host can watch it, as draw snapshots need to be reused) or plain (every +// use copies it). +class GuestBuffer { +public: + explicit GuestBuffer(bool registered, std::size_t bytes = BufferBytes) : registered(registered), bytes(bytes), watched(registered && AgcDriver::GuestMemory::WriteWatched()) { +#ifdef _WIN32 + if (watched) { + block = static_cast(GuestArena::GuestArenaAllocate_nid_postfix(bytes, 65536)); + if (block != nullptr) GuestArena::GuestArenaCommit_nid_postfix(block, bytes, PAGE_READWRITE, bytes); + } else { + block = static_cast(VirtualAlloc(nullptr, bytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); + } +#else + if (watched) { + constexpr std::uintptr_t alignment = 65536; + void* mapped = mmap(nullptr, bytes + alignment, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + Require(mapped != MAP_FAILED, "recorded kept bytes: cannot map a guest buffer"); + const auto begin = reinterpret_cast(mapped); + const auto aligned = (begin + alignment - 1) & ~(alignment - 1); + if (aligned != begin) munmap(mapped, aligned - begin); + if (aligned + bytes != begin + bytes + alignment) munmap(reinterpret_cast(aligned + bytes), begin + alignment - aligned); + block = reinterpret_cast(aligned); + GuestWriteWatch::GuestWriteWatchRegister_nid_postfix(block, bytes); + } else { + block = static_cast(std::aligned_alloc(65536, bytes)); + } +#endif + Require(block != nullptr, "recorded kept bytes: cannot allocate a guest buffer"); + std::memset(block, 0, bytes); + if (registered) GuestAllocations::Mutation().Add(block, bytes, true, true); + } + + ~GuestBuffer() { + if (registered) { + AgcDriver::Graphics::StorageTexture::FlushPending(AddressOf(block), bytes, nullptr, "test release"); + GuestAllocations::Mutation().Remove(block); + } +#ifdef _WIN32 + if (watched) { + GuestArena::GuestArenaReset_nid_postfix(block, bytes); + GuestArena::GuestArenaRelease_nid_postfix(block, bytes); + } else { + VirtualFree(block, 0, MEM_RELEASE); + } +#else + if (watched) { + munmap(block, bytes); + GuestWriteWatch::GuestWriteWatchUnregister_nid_postfix(block, bytes); + } else { + std::free(block); + } +#endif + } + + GuestBuffer(const GuestBuffer&) = delete; + GuestBuffer& operator=(const GuestBuffer&) = delete; + + std::uint8_t* Data() { return block; } + +private: + bool registered; + std::size_t bytes; + bool watched; + std::uint8_t* block = nullptr; +}; + +Recorder& ActiveRecorder() { + auto* recorder = Recorder::Active(); + Require(recorder != nullptr, "recorded kept bytes: no active recorder"); + return *recorder; +} + +std::string Bytes(std::size_t bytes) { + return std::to_string(bytes) + " bytes"; +} + +void Dispatch(AgcDriver::VulkanDevice& device, std::uint8_t* target) { + std::vector userData(16, 0u); + const auto descriptor = RawBuffer(target, static_cast(BufferBytes)); + std::copy(descriptor.begin(), descriptor.end(), userData.begin()); + const std::span code(StoreCode); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {false, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); +} + +const ColorTargetLayout TargetLayout(Width, Height, ColorTileMode::RenderTarget, 4); + +constexpr std::array, 3> Triangle{{ + {-1.0f, -1.0f, 0.5f, 1.0f}, {3.0f, -1.0f, 0.5f, 1.0f}, {-1.0f, 3.0f, 0.5f, 1.0f} +}}; + +// Vertex records for Draw: the triangle, then zeros up to BufferBytes. +void FillVertices(GuestBuffer& vertices) { + std::memcpy(vertices.Data(), Triangle.data(), sizeof(Triangle)); +} + +// A full-target triangle into a resident (tiled) color target, so the draw is recorded, whose +// vertex stage reads its positions from `vertices` (BufferBytes of records, so the bound range is +// the whole buffer). +void Draw(AgcDriver::VulkanDevice& device, GuestBuffer& pixels, GuestBuffer& vertices) { + const auto target = device.Target(); + std::vector vertexUserData(4, 0u); + const auto vertexBuffer = VertexBuffer(vertices.Data(), 16u, static_cast(BufferBytes / 16u)); + std::copy(vertexBuffer.begin(), vertexBuffer.end(), vertexUserData.begin()); + const std::array vertexMemory{{{reinterpret_cast(VertexCode.data()), std::as_bytes(std::span(VertexCode))}}}; + ShaderRecompiler::RecompileRequest vertex{ + {ShaderStage::Vertex, reinterpret_cast(VertexCode.data()), VertexCode, 0, {}}, + {64, 0, vertexUserData, std::nullopt, std::nullopt, ShaderRecompiler::ShaderVertexStageInfo{}, vertexMemory}, + target, + {0, 0, 0, 64} + }; + vertex.useCache = false; + const auto vertexResult = ShaderRecompiler::Recompile(vertex); + const auto vertexPush = static_cast(vertexResult.pushConstants.size()); + + ShaderRecompiler::ShaderPixelStageInfo pixel{}; + pixel.inputAddr = ShaderRecompiler::PixelInputBit(ShaderRecompiler::PixelInput::PositionX) | ShaderRecompiler::PixelInputBit(ShaderRecompiler::PixelInput::PositionY); + pixel.posX = true; + pixel.posY = true; + pixel.targetOutputMode[0] = 9; + pixel.targetExportMapping.fill(0xe4u); + std::vector pixelUserData(4, 0u); + const std::array pixelMemory{{{reinterpret_cast(PixelCode.data()), std::as_bytes(std::span(PixelCode))}}}; + ShaderRecompiler::RecompileRequest fragment{ + {ShaderStage::Fragment, reinterpret_cast(PixelCode.data()), PixelCode, 0, {}}, + {64, 0, pixelUserData, std::nullopt, pixel, std::nullopt, pixelMemory}, + target, + {0, 0, vertexPush, 128 - vertexPush} + }; + fragment.useCache = false; + const auto pixelResult = ShaderRecompiler::Recompile(fragment); + const std::array shaders{{ + {ShaderStage::Vertex, &vertexResult, 0}, + {ShaderStage::Fragment, &pixelResult, vertexPush} + }}; + + AgcDriver::Graphics::State state{}; + state.stages = {AgcDriver::Graphics::ShaderPath::Vertex, 0u, 64u, 64u, std::nullopt, std::nullopt}; + state.color = {AddressOf(pixels.Data()), {Width, Height}, VK_FORMAT_R8G8B8A8_UNORM, TargetLayout.Bytes(), 0xe4u, ColorTileMode::RenderTarget}; + state.colors = {state.color}; + state.hasColorTarget = true; + state.renderExtent = {Width, Height}; + state.topology = VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST; + state.viewport = {0, static_cast(Height), static_cast(Width), -static_cast(Height), 0, 1}; + state.negativeOneToOne = false; + state.scissor = {{0, 0}, {Width, Height}}; + state.cullMode = VK_CULL_MODE_NONE; + state.frontFace = VK_FRONT_FACE_COUNTER_CLOCKWISE; + state.blend.colorWriteMask = 15; + state.blends = {state.blend}; + state.blendConstants = {}; + const AgcDriver::Pm4::DrawParameters draw{0, static_cast(Triangle.size()), 0, 1, 0, false}; + device.Draw(state, draw, shaders); +} + +// VulkanDevice::recordDispatch: the dispatch's copy of the buffer it writes. +void CopiedDispatch(AgcDriver::VulkanDevice& device) { + GuestBuffer target(false); + device.WaitIdle(); + Dispatch(device, target.Data()); + const auto& recorder = ActiveRecorder(); + Require(recorder.Recording(), "recorded kept bytes: the dispatch was not left in the open batch"); + const auto kept = recorder.OpenKeptBytes(); + Require(kept >= BufferBytes, "a dispatch writing a copied " + Bytes(BufferBytes) + " buffer left " + Bytes(kept) + " in its batch's kept bytes"); + device.WaitIdle(); +} + +// The target's first draw makes it resident (its upload counts in that batch's kept bytes), so the +// draws measured after it keep only their own bytes. +void ResidentTarget(AgcDriver::VulkanDevice& device, GuestBuffer& pixels) { + GuestBuffer vertices(false); + FillVertices(vertices); + Draw(device, pixels, vertices); + device.WaitIdle(); +} + +// keepRecordedDraw: the draw's copy of the buffer it reads. +void CopiedDraw(AgcDriver::VulkanDevice& device, GuestBuffer& pixels) { + GuestBuffer vertices(false); + FillVertices(vertices); + device.WaitIdle(); + Draw(device, pixels, vertices); + const auto& recorder = ActiveRecorder(); + Require(recorder.Recording(), "recorded kept bytes: the draw was not left in the open batch"); + const auto kept = recorder.OpenKeptBytes(); + Require(kept >= BufferBytes, "a draw reading a copied " + Bytes(BufferBytes) + " buffer left " + Bytes(kept) + " in its batch's kept bytes"); + device.WaitIdle(); +} + +// PrepareDrawBindings: the draw's snapshot of the buffer it reads in place, counted once for a +// second draw of the batch that reuses it. +bool SnapshotDraws(AgcDriver::VulkanDevice& device, GuestBuffer& pixels) { + GuestBuffer vertices(true); + FillVertices(vertices); + device.WaitIdle(); + Draw(device, pixels, vertices); + auto& recorder = ActiveRecorder(); + Require(recorder.Recording(), "recorded kept bytes: the snapshot draw was not left in the open batch"); + if (recorder.ReusableDrawSnapshot(AddressOf(vertices.Data()), BufferBytes) == nullptr) { + std::puts("the draw took no reusable snapshot (no host import, or the buffer is not write-watched): draw snapshots are not tested"); + device.WaitIdle(); + return false; + } + const auto first = recorder.OpenKeptBytes(); + Require(first >= BufferBytes, "a draw snapshotting a " + Bytes(BufferBytes) + " buffer left " + Bytes(first) + " in its batch's kept bytes"); + Draw(device, pixels, vertices); + Require(recorder.Recording(), "recorded kept bytes: the second snapshot draw was not left in the open batch"); + const auto second = recorder.OpenKeptBytes(); + Require(second == first, "a second draw reusing the batch's snapshot raised its kept bytes from " + Bytes(first) + " to " + Bytes(second)); + device.WaitIdle(); + return true; +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + CopiedDispatch(*device); + GuestBuffer pixels(true, TargetLayout.Bytes()); + ResidentTarget(*device, pixels); + CopiedDraw(*device, pixels); + const bool snapshots = SnapshotDraws(*device, pixels); + std::printf("recorded kept bytes tests passed%s\n", snapshots ? "" : " (draw snapshots not tested)"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} From 078b6d886fab994a5756fec08bf286a38ff4fd9f Mon Sep 17 00:00:00 2001 From: MathiasDevelopes <25400788+MathiasDevelopes@users.noreply.github.com> Date: Sat, 10 Oct 2026 09:52:47 +0000 Subject: [PATCH 196/460] style(agc): drop the comments the conventions check rejects Comments are reserved for technical debt (docs/dev/TechnicalDebt.md), so the ones the previous commit added to the recorder, the snapshot keep and the tests go. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01HZHkb8kUwnHAuBBbHywvJf --- .../Graphics/include/Recorder.hpp | 6 ----- .../Graphics/src/ShaderResources.cpp | 1 - .../libSceAgcDriver/tests/BdaResources.cpp | 2 -- .../tests/execution/RecordedKeptBytes.cpp | 22 ------------------- 4 files changed, 31 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp index 490d2479e..e59a93738 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp @@ -86,12 +86,7 @@ public: // Whether recorded work still has completion actions (write-backs the CPU must see) to run. bool HasCompletions() const; void Keep(std::shared_ptr object, std::size_t bytes = 0); - // Counts `bytes` held by `owner` in the open batch's kept bytes once per batch: memory several - // keeps of one batch share (a cached ShaderResources kept by every draw or dispatch using it, a - // reused draw snapshot) counts once. The caller keeps `owner` alive in the same batch, so its - // address names one object for the batch's lifetime. void KeepBytes(const void* owner, std::size_t bytes); - // The kept bytes of the open batch (0 with none open), for tests. std::size_t OpenKeptBytes() const { return open != nullptr ? open->keptBytes : 0; } static constexpr std::size_t KeptBytesBudget = std::size_t{512} << 20u; void BoundKeptBytes(); @@ -519,7 +514,6 @@ private: VkFence fence = VK_NULL_HANDLE; std::vector> kept; std::size_t keptBytes = 0; - // The owners KeepBytes counted in this batch. std::unordered_set keptOwners; std::vector> completions; std::vector> writes; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index f5b41011e..159c31a31 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -3369,7 +3369,6 @@ std::shared_ptr ShaderResources::PrepareDrawBindi } update(context.device, static_cast(writes.size()), writes.data(), 0, nullptr); recorder.Keep(result); - // A reused snapshot (Recorder::ReusableDrawSnapshot) is one buffer however many draws bind it. for (const auto& snapshot : result->snapshots) recorder.KeepBytes(snapshot.buffer.get(), snapshot.buffer->Bytes().size()); return result; } diff --git a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp index 2ad77ce62..037991ce5 100644 --- a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp @@ -578,8 +578,6 @@ void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { memory.AddWritable(address + 16, 16); Require(memory.CopiedBytes() == 0, "a region counted as copied before its upload"); memory.Upload(true); - // `guest` is stack memory no allocation registers, so no host import serves it: its two views merge - // into one copied region. Require(memory.CopiedBytes() == sizeof(guest), "copied bytes " + std::to_string(memory.CopiedBytes()) + " do not match the one region copied"); std::uint32_t adjustment = 0; const auto first = memory.Descriptor(address, sizeof(guest), adjustment); diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp index b82cc95b2..d2f66285a 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp @@ -28,12 +28,6 @@ #include #include -// The bytes a recorded dispatch or draw holds alive reach the recorder's batch budget -// (Recorder::KeptBytesBudget): the copies of guest buffers its ShaderResources own (kept by -// VulkanDevice::recordDispatch and Draw.cpp's keepRecordedDraw) and a draw's snapshots of buffers it -// reads in place (ShaderResources::PrepareDrawBindings). A snapshot reused by the next draw of the -// batch is one buffer and counts once. - namespace { using AgcDriver::Graphics::ColorTargetLayout; @@ -47,17 +41,14 @@ constexpr std::size_t BufferBytes = 65536; constexpr std::uint32_t Width = 256; constexpr std::uint32_t Height = 128; -// v1 = v0 << 3; v[2:3] = 0; buffer_store_dwordx2 v[2:3], v1, s[0:3] offen alignas(256) constexpr std::array StoreCode{ 0x34020083, 0x7e040280, 0x7e060280, 0xe0741000, 0x80000201, 0xbf810000, }; -// buffer_load_format_xyzw v[0:3], v5, s[0:3] idxen; exp pos0 v[0:3] done; s_endpgm alignas(256) constexpr std::array VertexCode{ 0xe0382000, 0x80000005, 0xbf8c3f70, 0xf80008cf, 0x03020100, 0xbf810000, }; -// v7 = 1.0; exp mrt0 v7 v7 v7 v7 done vm; s_endpgm alignas(256) constexpr std::array PixelCode{ 0x7e0e02f2, 0xf800180f, 0x07070707, 0xbf810000, }; @@ -76,9 +67,6 @@ std::array VertexBuffer(const void* data, std::uint32_t stride return {static_cast(address), static_cast((address >> 32u) & 0xffffu) | (stride << 16u), count, 0x01016facu}; } -// Guest memory, registered (served in place by a host import where the device has them, and -// write-watched where the host can watch it, as draw snapshots need to be reused) or plain (every -// use copies it). class GuestBuffer { public: explicit GuestBuffer(bool registered, std::size_t bytes = BufferBytes) : registered(registered), bytes(bytes), watched(registered && AgcDriver::GuestMemory::WriteWatched()) { @@ -177,14 +165,10 @@ constexpr std::array, 3> Triangle{{ {-1.0f, -1.0f, 0.5f, 1.0f}, {3.0f, -1.0f, 0.5f, 1.0f}, {-1.0f, 3.0f, 0.5f, 1.0f} }}; -// Vertex records for Draw: the triangle, then zeros up to BufferBytes. void FillVertices(GuestBuffer& vertices) { std::memcpy(vertices.Data(), Triangle.data(), sizeof(Triangle)); } -// A full-target triangle into a resident (tiled) color target, so the draw is recorded, whose -// vertex stage reads its positions from `vertices` (BufferBytes of records, so the bound range is -// the whole buffer). void Draw(AgcDriver::VulkanDevice& device, GuestBuffer& pixels, GuestBuffer& vertices) { const auto target = device.Target(); std::vector vertexUserData(4, 0u); @@ -241,7 +225,6 @@ void Draw(AgcDriver::VulkanDevice& device, GuestBuffer& pixels, GuestBuffer& ver device.Draw(state, draw, shaders); } -// VulkanDevice::recordDispatch: the dispatch's copy of the buffer it writes. void CopiedDispatch(AgcDriver::VulkanDevice& device) { GuestBuffer target(false); device.WaitIdle(); @@ -253,8 +236,6 @@ void CopiedDispatch(AgcDriver::VulkanDevice& device) { device.WaitIdle(); } -// The target's first draw makes it resident (its upload counts in that batch's kept bytes), so the -// draws measured after it keep only their own bytes. void ResidentTarget(AgcDriver::VulkanDevice& device, GuestBuffer& pixels) { GuestBuffer vertices(false); FillVertices(vertices); @@ -262,7 +243,6 @@ void ResidentTarget(AgcDriver::VulkanDevice& device, GuestBuffer& pixels) { device.WaitIdle(); } -// keepRecordedDraw: the draw's copy of the buffer it reads. void CopiedDraw(AgcDriver::VulkanDevice& device, GuestBuffer& pixels) { GuestBuffer vertices(false); FillVertices(vertices); @@ -275,8 +255,6 @@ void CopiedDraw(AgcDriver::VulkanDevice& device, GuestBuffer& pixels) { device.WaitIdle(); } -// PrepareDrawBindings: the draw's snapshot of the buffer it reads in place, counted once for a -// second draw of the batch that reuses it. bool SnapshotDraws(AgcDriver::VulkanDevice& device, GuestBuffer& pixels) { GuestBuffer vertices(true); FillVertices(vertices); From d731f5034e102049461f87b4a8d8ce4f679e6fb7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 00:33:12 +0200 Subject: [PATCH 197/460] test(agc): pass the gpu argument to GuestAllocations::Mutation().Add Co-authored-by: orishamir Co-Authored-By: Claude --- .../prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp index d2f66285a..11b98c82d 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp @@ -94,7 +94,7 @@ public: #endif Require(block != nullptr, "recorded kept bytes: cannot allocate a guest buffer"); std::memset(block, 0, bytes); - if (registered) GuestAllocations::Mutation().Add(block, bytes, true, true); + if (registered) GuestAllocations::Mutation().Add(block, bytes, true, true, true); } ~GuestBuffer() { From cfd61a5977d987d4aff1d46a38b5b308b12942e4 Mon Sep 17 00:00:00 2001 From: simonbbby Date: Fri, 9 Oct 2026 10:07:34 -0400 Subject: [PATCH 198/460] fix(relinker): treat address-taken code as reachable in unused-filter=1 The CFG used by unused-filter=1 followed only direct branches, so code reached through lea, relative-relocated pointers or jump tables was unreachable and its imports were dropped. It now roots RIP-relative lea targets, relative relocation targets and relative jump tables, and counts any RIP-relative reference to a GOT slot as an access. Address-taken targets are walked after the direct control flow, and a target that does not decode as an instruction is not code. --- core/relinker/CMakeLists.txt | 2 + .../UnusedNidFilter/IControlFlowGraph.hpp | 10 ++ .../IRelativeRelocationIndex.hpp | 1 + .../UnusedNidFilter/StrictReachability.hpp | 2 + .../src/analysis/CodeInstructionCollector.cpp | 5 + .../relinker/src/analysis/UnusedNidFilter.cpp | 10 +- .../UnusedNidFilter/ControlFlowGraph.cpp | 74 +++++++++++- .../UnusedNidFilter/GotAccessIndex.cpp | 19 --- .../RelativeRelocationIndex.cpp | 7 ++ .../UnusedNidFilter/StrictReachability.cpp | 26 +++-- .../tests/test_unused_filter_indirect.py | 108 ++++++++++++++++++ docs/user/USAGE.md | 2 +- 12 files changed, 231 insertions(+), 35 deletions(-) create mode 100644 core/relinker/relinker/tests/test_unused_filter_indirect.py diff --git a/core/relinker/CMakeLists.txt b/core/relinker/CMakeLists.txt index 28c495675..913c56162 100644 --- a/core/relinker/CMakeLists.txt +++ b/core/relinker/CMakeLists.txt @@ -334,6 +334,8 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_to_rosetta.py $) add_test(NAME syscall_number COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_syscall_number.py $) + add_test(NAME unused_filter_indirect COMMAND ${Python3_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_unused_filter_indirect.py $) else() message(STATUS "Python3 interpreter not found; skipping Python relinker tests") endif() diff --git a/core/relinker/relinker/include/relinker/analysis/UnusedNidFilter/IControlFlowGraph.hpp b/core/relinker/relinker/include/relinker/analysis/UnusedNidFilter/IControlFlowGraph.hpp index a4290599b..7b1add380 100644 --- a/core/relinker/relinker/include/relinker/analysis/UnusedNidFilter/IControlFlowGraph.hpp +++ b/core/relinker/relinker/include/relinker/analysis/UnusedNidFilter/IControlFlowGraph.hpp @@ -3,6 +3,7 @@ #include #include +#include #include #include #include @@ -45,6 +46,15 @@ std::unique_ptr BuildControlFlowGraph( bool followCodeAddresses = false ); +std::unique_ptr BuildAddressTakenControlFlowGraph( + const std::vector& text, + VirtualAddress textVaddr, + VirtualAddress entryVaddr, + const std::vector& extraEntries, + const IRelativeRelocationIndex& relativeRelocations, + const std::vector& data +); + } #endif diff --git a/core/relinker/relinker/include/relinker/analysis/UnusedNidFilter/IRelativeRelocationIndex.hpp b/core/relinker/relinker/include/relinker/analysis/UnusedNidFilter/IRelativeRelocationIndex.hpp index 5a3a44742..3c5023559 100644 --- a/core/relinker/relinker/include/relinker/analysis/UnusedNidFilter/IRelativeRelocationIndex.hpp +++ b/core/relinker/relinker/include/relinker/analysis/UnusedNidFilter/IRelativeRelocationIndex.hpp @@ -13,6 +13,7 @@ class IRelativeRelocationIndex { public: virtual ~IRelativeRelocationIndex() = default; virtual std::optional TargetOfSlot(VirtualAddress slotVaddr) const = 0; + virtual std::vector Targets() const = 0; }; std::unique_ptr BuildRelativeRelocationIndex( diff --git a/core/relinker/relinker/include/relinker/analysis/UnusedNidFilter/StrictReachability.hpp b/core/relinker/relinker/include/relinker/analysis/UnusedNidFilter/StrictReachability.hpp index 5c0d663c7..2261cfee8 100644 --- a/core/relinker/relinker/include/relinker/analysis/UnusedNidFilter/StrictReachability.hpp +++ b/core/relinker/relinker/include/relinker/analysis/UnusedNidFilter/StrictReachability.hpp @@ -39,6 +39,8 @@ struct StrictReachabilityResult { StrictReachabilityResult AnalyzeStrictReachability(const StrictReachabilityInput& input); +std::vector ReadRelativeTableTargets(const std::vector& data, VirtualAddress base, VirtualAddress textVaddr, std::size_t textSize); + } #endif diff --git a/core/relinker/relinker/src/analysis/CodeInstructionCollector.cpp b/core/relinker/relinker/src/analysis/CodeInstructionCollector.cpp index 495395106..540ef359a 100644 --- a/core/relinker/relinker/src/analysis/CodeInstructionCollector.cpp +++ b/core/relinker/relinker/src/analysis/CodeInstructionCollector.cpp @@ -25,6 +25,11 @@ public: if (found == Values.end()) return std::nullopt; return found->second; } + std::vector Targets() const override { + std::vector result; + for (const auto& [slot, target] : Values) result.push_back(target); + return result; + } }; } diff --git a/core/relinker/relinker/src/analysis/UnusedNidFilter.cpp b/core/relinker/relinker/src/analysis/UnusedNidFilter.cpp index e3396f536..1426881a9 100644 --- a/core/relinker/relinker/src/analysis/UnusedNidFilter.cpp +++ b/core/relinker/relinker/src/analysis/UnusedNidFilter.cpp @@ -3,6 +3,7 @@ #include #include #include +#include namespace Relinker { @@ -25,7 +26,14 @@ public: VirtualAddress primary = entries[0]; std::vector extra(entries.begin() + 1, entries.end()); - auto cfg = UnusedNidFilter::BuildControlFlowGraph(textSection, textVAddr, primary, extra, *relativeRelocations); + const ElfReader reader(elfBytes); + std::vector data; + for (const auto& segment : reader.ReadProgramHeaders()) { + if (segment.Type != 1 || (segment.Flags & 1) != 0 || (segment.Flags & 6) == 0 || segment.FileSize == 0) continue; + data.push_back({segment.MappedAddress, reader.ReadSegment(segment)}); + } + + auto cfg = UnusedNidFilter::BuildAddressTakenControlFlowGraph(textSection, textVAddr, primary, extra, *relativeRelocations, data); auto index = UnusedNidFilter::BuildGotAccessIndex(*cfg, textSection, textVAddr); std::vector result; diff --git a/core/relinker/relinker/src/analysis/UnusedNidFilter/ControlFlowGraph.cpp b/core/relinker/relinker/src/analysis/UnusedNidFilter/ControlFlowGraph.cpp index 203ae2b08..e4301940e 100644 --- a/core/relinker/relinker/src/analysis/UnusedNidFilter/ControlFlowGraph.cpp +++ b/core/relinker/relinker/src/analysis/UnusedNidFilter/ControlFlowGraph.cpp @@ -1,5 +1,6 @@ #include #include +#include #include #include #include @@ -26,29 +27,50 @@ private: const std::unordered_set* _borrowed = nullptr; }; -std::unique_ptr BuildControlFlowGraph( +namespace { + +void buildGraph( std::span text, VirtualAddress textVaddr, VirtualAddress entryVaddr, const std::vector& extraEntries, const IRelativeRelocationIndex& relativeRelocations, + const std::vector* data, bool followCodeAddresses, std::unordered_set& reachable ) { const Codegen::X64InstructionDecoder decoder; + std::unordered_set visited; std::queue worklist; + std::vector addressTaken; + bool speculative = false; auto enqueue = [&](VirtualAddress va) { if (reachable.count(va) > 0) return; if (va < textVaddr || va >= textVaddr + static_cast(text.size())) return; - reachable.insert(va); + if (!visited.insert(va).second) return; worklist.push(va); }; + auto takeAddress = [&](VirtualAddress va) { + if (data == nullptr) return; + if (speculative) + enqueue(va); + else + addressTaken.push_back(va); + }; + enqueue(entryVaddr); for (VirtualAddress va : extraEntries) enqueue(va); + for (VirtualAddress va : relativeRelocations.Targets()) takeAddress(va); + + while (!worklist.empty() || !speculative) { + if (worklist.empty()) { + speculative = true; + for (VirtualAddress va : addressTaken) enqueue(va); + continue; + } - while (!worklist.empty()) { VirtualAddress va = worklist.front(); worklist.pop(); @@ -59,7 +81,17 @@ std::unique_ptr BuildControlFlowGraph( std::size_t available = text.size() - bufOff; if (available == 0) throw RelinkerException("CFG: zero available bytes at target", va); - Codegen::DecodedInstructionInfo info = decoder.DecodeInstruction(text.data() + bufOff, available); + Codegen::DecodedInstructionInfo info; + if (speculative) { + try { + info = decoder.DecodeInstruction(text.data() + bufOff, available); + } catch (const Codegen::CodegenException&) { + continue; + } + } else { + info = decoder.DecodeInstruction(text.data() + bufOff, available); + } + reachable.insert(va); VirtualAddress nextVaddr = va + static_cast(info.Length); @@ -69,6 +101,14 @@ std::unique_ptr BuildControlFlowGraph( enqueue(static_cast(static_cast(nextVaddr) + displacement)); } + if (data != nullptr && info.HasRipRelativeDisp && !info.IsTwoByteOpcode && info.Opcode == 0x8D) { + std::int32_t displacement = 0; + std::memcpy(&displacement, text.data() + bufOff + info.RipRelativeDispOffset, 4); + const auto base = static_cast(static_cast(nextVaddr) + displacement); + takeAddress(base); + for (VirtualAddress target : ReadRelativeTableTargets(*data, base, textVaddr, text.size())) takeAddress(target); + } + switch (info.FlowKind) { using enum Codegen::ControlFlowKind; case Sequential: @@ -117,7 +157,20 @@ std::unique_ptr BuildControlFlowGraph( break; } } +} +} + +std::unique_ptr BuildControlFlowGraph( + std::span text, + VirtualAddress textVaddr, + VirtualAddress entryVaddr, + const std::vector& extraEntries, + const IRelativeRelocationIndex& relativeRelocations, + bool followCodeAddresses, + std::unordered_set& reachable +) { + buildGraph(text, textVaddr, entryVaddr, extraEntries, relativeRelocations, nullptr, followCodeAddresses, reachable); return std::make_unique(&reachable); } @@ -145,4 +198,17 @@ std::unique_ptr BuildControlFlowGraph( return BuildControlFlowGraph(std::span(text.data(), text.size()), textVaddr, entryVaddr, extraEntries, relativeRelocations, followCodeAddresses); } +std::unique_ptr BuildAddressTakenControlFlowGraph( + const std::vector& text, + VirtualAddress textVaddr, + VirtualAddress entryVaddr, + const std::vector& extraEntries, + const IRelativeRelocationIndex& relativeRelocations, + const std::vector& data +) { + std::unordered_set reachable; + buildGraph(std::span(text.data(), text.size()), textVaddr, entryVaddr, extraEntries, relativeRelocations, &data, false, reachable); + return std::make_unique(std::move(reachable)); +} + } diff --git a/core/relinker/relinker/src/analysis/UnusedNidFilter/GotAccessIndex.cpp b/core/relinker/relinker/src/analysis/UnusedNidFilter/GotAccessIndex.cpp index 9ca3e74e7..cf9f0b94d 100644 --- a/core/relinker/relinker/src/analysis/UnusedNidFilter/GotAccessIndex.cpp +++ b/core/relinker/relinker/src/analysis/UnusedNidFilter/GotAccessIndex.cpp @@ -1,28 +1,10 @@ #include #include -#include #include #include namespace Relinker::UnusedNidFilter { -namespace { - -constexpr std::uint8_t OneByteMovLoadRm8 = 0x8A; -constexpr std::uint8_t OneByteMovLoadRm = 0x8B; -constexpr std::uint8_t Grp5RegCallIndirect = 2; -constexpr std::uint8_t Grp5RegJmpIndirect = 4; - -bool ReadsMemoryOperandAsPointer(const Codegen::DecodedInstructionInfo& info) { - if (info.IsTwoByteOpcode) return false; - if (info.Opcode == OneByteMovLoadRm8 || info.Opcode == OneByteMovLoadRm) return true; - if (info.Opcode == Codegen::X64OpcodeConstants::OneByteGrp5Rm) - return info.ModRmRegField == Grp5RegCallIndirect || info.ModRmRegField == Grp5RegJmpIndirect; - return false; -} - -} - class GotAccessIndex : public IGotAccessIndex { public: explicit GotAccessIndex(std::unordered_set accessed) @@ -53,7 +35,6 @@ std::unique_ptr BuildGotAccessIndex( Codegen::DecodedInstructionInfo info = decoder.DecodeInstruction(text.data() + bufOff, available); if (!info.HasRipRelativeDisp) continue; - if (!ReadsMemoryOperandAsPointer(info)) continue; VirtualAddress nextVaddr = va + static_cast(info.Length); std::int32_t disp = 0; diff --git a/core/relinker/relinker/src/analysis/UnusedNidFilter/RelativeRelocationIndex.cpp b/core/relinker/relinker/src/analysis/UnusedNidFilter/RelativeRelocationIndex.cpp index 4e050df8c..9a4d99657 100644 --- a/core/relinker/relinker/src/analysis/UnusedNidFilter/RelativeRelocationIndex.cpp +++ b/core/relinker/relinker/src/analysis/UnusedNidFilter/RelativeRelocationIndex.cpp @@ -102,6 +102,13 @@ public: return it->second; } + std::vector Targets() const override { + std::vector result; + result.reserve(_targets.size()); + for (const auto& entry : _targets) result.push_back(entry.second); + return result; + } + private: std::unordered_map _targets; }; diff --git a/core/relinker/relinker/src/analysis/UnusedNidFilter/StrictReachability.cpp b/core/relinker/relinker/src/analysis/UnusedNidFilter/StrictReachability.cpp index 60dcf67b8..71fd7342e 100644 --- a/core/relinker/relinker/src/analysis/UnusedNidFilter/StrictReachability.cpp +++ b/core/relinker/relinker/src/analysis/UnusedNidFilter/StrictReachability.cpp @@ -201,16 +201,7 @@ private: void collectRelativeTables() { for (const auto base : tableBases) { - for (const auto& data : input.Data) { - if (base < data.Address || base - data.Address >= data.Bytes.size()) continue; - for (auto offset = static_cast(base - data.Address); offset + 4 <= data.Bytes.size(); offset += 4) { - std::int32_t displacement; - std::memcpy(&displacement, data.Bytes.data() + offset, sizeof(displacement)); - const auto target = base + static_cast(static_cast(displacement)); - if (!isCode(target)) break; - addAddressTaken(target); - } - } + for (const auto target : ReadRelativeTableTargets(input.Data, base, input.TextVaddr, input.Text.size())) addAddressTaken(target); } } }; @@ -221,4 +212,19 @@ StrictReachabilityResult AnalyzeStrictReachability(const StrictReachabilityInput return Analyzer(input).Run(); } +std::vector ReadRelativeTableTargets(const std::vector& data, VirtualAddress base, VirtualAddress textVaddr, std::size_t textSize) { + std::vector targets; + for (const auto& region : data) { + if (base < region.Address || base - region.Address >= region.Bytes.size()) continue; + for (auto offset = static_cast(base - region.Address); offset + 4 <= region.Bytes.size(); offset += 4) { + std::int32_t displacement; + std::memcpy(&displacement, region.Bytes.data() + offset, sizeof(displacement)); + const auto target = base + static_cast(static_cast(displacement)); + if (target < textVaddr || target - textVaddr >= textSize) break; + targets.push_back(target); + } + } + return targets; +} + } diff --git a/core/relinker/relinker/tests/test_unused_filter_indirect.py b/core/relinker/relinker/tests/test_unused_filter_indirect.py new file mode 100644 index 000000000..f3fd080ab --- /dev/null +++ b/core/relinker/relinker/tests/test_unused_filter_indirect.py @@ -0,0 +1,108 @@ +import json +import struct +import subprocess +import sys +import tempfile +from pathlib import Path + +CODE = 0x1000 +SLOT = 0x800 +POINTER = 0x810 +TABLE = 0x820 +FUNCTION = CODE + 0x40 +SECOND_FUNCTION = CODE + 0x60 + + +def rel(next_address, target): + return struct.pack(" {kept};" in result.stdout, (name, result.stdout) + return kept == 1 + + +def cases(): + lea_call = b"\x48\x8d\x05" + rel(CODE + 7, FUNCTION) + b"\xff\xd0\xc3" + yield "direct-call", True, build(at_offset([(0, b"\xe8" + rel(CODE + 5, FUNCTION) + b"\xc3"), + (0x40, load_slot(FUNCTION))])) + yield "lea-call-register", True, build(at_offset([(0, lea_call), (0x40, load_slot(FUNCTION))])) + vtable_call = b"\x48\x8b\x05" + rel(CODE + 7, POINTER) + b"\xff\xd0\xc3" + yield "relative-relocation-pointer", True, build( + at_offset([(0, vtable_call), (0x40, load_slot(FUNCTION))]), relative=[(POINTER, FUNCTION)]) + jump = (b"\x48\x8d\x0d" + rel(CODE + 7, TABLE) + b"\x48\x63\x04\x91\x48\x01\xc8\xff\xe0") + table = struct.pack(".registry.json` beside the output executable, and `..registry.json` for each converted bundled module. | | `--rpath ` | Set the system library search path. Quote `$ORIGIN` to prevent shell expansion, for example `--rpath '$ORIGIN/libs'` in Bash or PowerShell. Linux guest modules require an absolute path or a path beginning with `$ORIGIN`. Windows requires a nonempty ASCII path and supports `$ORIGIN` as the executable directory. | From 502dae1ef376f933ba20206318cdba36d6ef0641 Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Sat, 10 Oct 2026 18:37:10 -0400 Subject: [PATCH 199/460] feat(libc): implement tzset --- core/libs/prx/libc/src/Time.cpp | 8 ++++++++ core/libs/tests/GuestTimeConversion.cpp | 10 ++++++++++ 2 files changed, 18 insertions(+) diff --git a/core/libs/prx/libc/src/Time.cpp b/core/libs/prx/libc/src/Time.cpp index f46add6e9..fc6609b9a 100644 --- a/core/libs/prx/libc/src/Time.cpp +++ b/core/libs/prx/libc/src/Time.cpp @@ -102,6 +102,14 @@ extern "C" { GuestTm* APS5_VABI localtime_s_nid_postfix(const int64_t* timer, GuestTm* result); GuestTm* APS5_VABI gmtime_s_nid_postfix(const int64_t* timer, GuestTm* result); +void APS5_VABI tzset_nid_postfix(void) { +#ifdef _WIN32 + _tzset(); +#else + ::tzset(); +#endif +} + int64_t APS5_VABI libc_time_nid_postfix(int64_t* timer) { std::time_t t = std::time(nullptr); if (timer != nullptr) *timer = static_cast(t); diff --git a/core/libs/tests/GuestTimeConversion.cpp b/core/libs/tests/GuestTimeConversion.cpp index 6cd199d70..4f83dbd38 100644 --- a/core/libs/tests/GuestTimeConversion.cpp +++ b/core/libs/tests/GuestTimeConversion.cpp @@ -20,6 +20,7 @@ GuestTm* APS5_VABI gmtime_s_nid_postfix(const std::int64_t*, GuestTm*); GuestTm* APS5_VABI localtime_s_nid_postfix(const std::int64_t*, GuestTm*); std::int64_t APS5_VABI mktime_nid_postfix(GuestTm*); std::size_t APS5_VABI strftime_nid_postfix(char*, std::size_t, const char*, const GuestTm*); +void APS5_VABI tzset_nid_postfix(void); } using Converter = GuestTm* (APS5_VABI *)(const std::int64_t*); @@ -137,4 +138,13 @@ int main() { CheckConcurrent(gmtime_nid_postfix, gmtime_s_nid_postfix, "Concurrent gmtime returned another thread's date"); CheckConcurrent(libc_localtime_nid_postfix, localtime_s_nid_postfix, "Concurrent libc_localtime returned another thread's date"); CheckConcurrent(localtime_nid_postfix, localtime_s_nid_postfix, "Concurrent localtime returned another thread's date"); +#ifdef _WIN32 + _putenv_s("TZ", "UTC-5"); +#else + setenv("TZ", "UTC-5", 1); +#endif + tzset_nid_postfix(); + GuestTm shifted{}; + Require(localtime_s_nid_postfix(&epoch, &shifted) == &shifted && shifted.tm_hour == 5 && shifted.tm_gmtoff == 18000, + "tzset did not pick up the new TZ"); } From 2d9b6aebb7558d6e3d412d8d28c833b8e4edee21 Mon Sep 17 00:00:00 2001 From: PhialsBasement <155454343+PhialsBasement@users.noreply.github.com> Date: Sat, 10 Oct 2026 23:19:10 +1100 Subject: [PATCH 200/460] feat(libSceNgs2.native): render silent waveform blocks with flag 0x10 A waveform block that declares samples with no data threw as not fitting them. PPSA04867 adds such a block with flags 0x11 (data_size 0, num_samples 16385) and terminates the process on the throw, followed by a fully specified block under the same flags, which takes the ordinary path. Accept flag 0x10, named SILENCE here: a dataless block under it renders num_samples frames of silence and finishes like a complete block, reporting its bytes through decoded_data_size. A dataless block without the flag keeps throwing. No public reference names the flag; the assumption is recorded in TechnicalDebt. Co-Authored-By: Claude Code --- .../libSceNgs2.native/include/Ngs2Types.hpp | 1 + .../libs/prx/libSceNgs2.native/src/Render.cpp | 1 + core/libs/prx/libSceNgs2.native/src/Voice.cpp | 7 ++++- .../libSceNgs2.native/tests/Ngs2Stream.cpp | 29 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 5 files changed, 38 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceNgs2.native/include/Ngs2Types.hpp b/core/libs/prx/libSceNgs2.native/include/Ngs2Types.hpp index 531b51834..c79d3ad7c 100644 --- a/core/libs/prx/libSceNgs2.native/include/Ngs2Types.hpp +++ b/core/libs/prx/libSceNgs2.native/include/Ngs2Types.hpp @@ -66,6 +66,7 @@ static constexpr std::uint32_t SCE_NGS2_CUSTOM_MODULE_ID_USER_FX2 = 0x1f; static constexpr std::uint32_t SCE_NGS2_WAVEFORM_BLOCKS_FLAG_CONTINUE = 1; static constexpr std::uint32_t SCE_NGS2_WAVEFORM_BLOCKS_FLAG_APPEND = 2; static constexpr std::uint32_t SCE_NGS2_WAVEFORM_BLOCKS_FLAG_RESET = 4; +static constexpr std::uint32_t SCE_NGS2_WAVEFORM_BLOCKS_FLAG_SILENCE = 0x10; static constexpr std::uint32_t SCE_NGS2_VOICE_CALLBACK_FLAG_BLOCK_END = 1; static constexpr std::uint32_t SCE_NGS2_VOICE_CALLBACK_FLAG_BLOCK_REPEAT = 2; diff --git a/core/libs/prx/libSceNgs2.native/src/Render.cpp b/core/libs/prx/libSceNgs2.native/src/Render.cpp index 5c39dd53d..c93902179 100644 --- a/core/libs/prx/libSceNgs2.native/src/Render.cpp +++ b/core/libs/prx/libSceNgs2.native/src/Render.cpp @@ -16,6 +16,7 @@ static constexpr std::uint64_t PHASE_FRACTION = 0xffffffff; static const std::uint8_t* FrameAt(const Ngs2Voice& voice, const Ngs2Block& block, std::uint32_t frame) { const std::size_t frameBytes = voice.channels * sizeof(std::int16_t); + if (block.data == nullptr) return nullptr; if (block.streaming) { for (const auto& piece : block.pieces) { if (frame >= piece.firstFrame && frame - piece.firstFrame < piece.frames) return piece.data + (frame - piece.firstFrame) * frameBytes; diff --git a/core/libs/prx/libSceNgs2.native/src/Voice.cpp b/core/libs/prx/libSceNgs2.native/src/Voice.cpp index 233011108..5a192a12a 100644 --- a/core/libs/prx/libSceNgs2.native/src/Voice.cpp +++ b/core/libs/prx/libSceNgs2.native/src/Voice.cpp @@ -199,7 +199,7 @@ static void AppendStreamData(Ngs2Voice& voice, const Ngs2SamplerVoiceWaveformBlo } static void AddWaveformBlocks(Ngs2Voice& voice, const Ngs2SamplerVoiceWaveformBlocksParam& param) { - constexpr std::uint32_t knownFlags = SCE_NGS2_WAVEFORM_BLOCKS_FLAG_CONTINUE | SCE_NGS2_WAVEFORM_BLOCKS_FLAG_APPEND | SCE_NGS2_WAVEFORM_BLOCKS_FLAG_RESET; + constexpr std::uint32_t knownFlags = SCE_NGS2_WAVEFORM_BLOCKS_FLAG_CONTINUE | SCE_NGS2_WAVEFORM_BLOCKS_FLAG_APPEND | SCE_NGS2_WAVEFORM_BLOCKS_FLAG_RESET | SCE_NGS2_WAVEFORM_BLOCKS_FLAG_SILENCE; if ((param.flags & ~knownFlags) != 0) throw std::runtime_error("NGS2: waveform block flags " + Ngs2Hex(param.flags) + " are not implemented"); if (voice.channels == 0 || (param.num_blocks != 0 && (param.blocks == nullptr || param.data == nullptr))) APS5_INVALID_ARG_EX; const bool reset = (param.flags & SCE_NGS2_WAVEFORM_BLOCKS_FLAG_RESET) != 0; @@ -218,9 +218,14 @@ static void AddWaveformBlocks(Ngs2Voice& voice, const Ngs2SamplerVoiceWaveformBl return; } voice.acceptsBlocks = (param.flags & SCE_NGS2_WAVEFORM_BLOCKS_FLAG_CONTINUE) != 0; + const bool allowsSilence = (param.flags & SCE_NGS2_WAVEFORM_BLOCKS_FLAG_SILENCE) != 0; for (std::uint32_t i = 0; i < param.num_blocks; i++) { const auto& block = param.blocks[i]; if (block.num_samples == 0 && block.data_size == 0) continue; + if (allowsSilence && block.data_size == 0) { + voice.blocks.push_back({nullptr, block}); + continue; + } const std::uint64_t bytes = voice.waveformType == SCE_NGS2_WAVEFORM_TYPE_ATRAC9 ? Ngs2Atrac9BlockBytes(voice, block) : (static_cast(block.num_skip_samples) + block.num_samples) * frameBytes; const bool streaming = block.num_samples != 0 && bytes > block.data_size && voice.waveformType == SCE_NGS2_WAVEFORM_TYPE_PCM_I16L && diff --git a/core/libs/prx/libSceNgs2.native/tests/Ngs2Stream.cpp b/core/libs/prx/libSceNgs2.native/tests/Ngs2Stream.cpp index 4b983d3e7..94994dcbe 100644 --- a/core/libs/prx/libSceNgs2.native/tests/Ngs2Stream.cpp +++ b/core/libs/prx/libSceNgs2.native/tests/Ngs2Stream.cpp @@ -128,9 +128,38 @@ static void TestStreamRejections() { Require(sceNgs2SystemDestroy(system, nullptr) == SCE_NGS2_OK); } +static void TestSilenceBlock() { + const auto system = CreateSystem(); + const auto master = Mastering(system, 1); + const auto rack = CreateRack(system, SCE_NGS2_RACK_ID_SAMPLER); + const auto voice = StreamVoice(rack, 0, master); + + AddBlock(voice, nullptr, 0, Grain, SCE_NGS2_WAVEFORM_BLOCKS_FLAG_SILENCE); + Event(voice, SCE_NGS2_VOICE_EVENT_PLAY); + auto out = RenderI16(system); + for (std::uint32_t i = 0; i < Grain; i++) Require(out[i] == 0); + Require(Flags(voice) == 0); + Ngs2SamplerVoiceState state{}; + Require(sceNgs2VoiceGetState(voice, &state.voice_state, sizeof(state)) == SCE_NGS2_OK); + Require(state.num_decoded_samples == Grain && state.decoded_data_size == Grain * 2); + + const auto withData = StreamVoice(rack, 1, master); + const std::vector pcm(Grain, 1234); + AddBlock(withData, pcm.data(), pcm.size() * 2, Grain, SCE_NGS2_WAVEFORM_BLOCKS_FLAG_SILENCE); + Event(withData, SCE_NGS2_VOICE_EVENT_PLAY); + out = RenderI16(system); + for (std::uint32_t i = 0; i < Grain; i++) Require(out[i] == 1234); + Require(Flags(withData) == 0); + + const auto dataless = StreamVoice(rack, 2, master); + Require(Throws([&] { AddBlock(dataless, nullptr, 0, Grain, 0); })); + Require(sceNgs2SystemDestroy(system, nullptr) == SCE_NGS2_OK); +} + int main() { TestStreamEndsAtDeclaredSamples(); TestStreamClosedBeforeDeclaredSamples(); TestStreamRejections(); + TestSilenceBlock(); return 0; } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index aa109ae29..2bdba83cb 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -203,6 +203,7 @@ Throughout the project, every function at every stage either **does exactly what - [libSceAudioPropagation](../../core/libs/prx/libSceAudioPropagation/Export.cpp) (libSceAudioPropagation) - signatures and struct descriptors recovered from PPSA21567's calls (#481), not from documentation. The system options layout is unknown (only checked for non-null); the fourth argument of `sceAudioPropagationSourceCalculateAudioPaths` is unknown; `sceAudioPropagationSourceGetAudioPath` is assumed to output an 8-byte path handle. The meaning of the `RenderInfo` word at +0x28 is unknown (PPSA21567 and PPSA21564 pass 2); only 2 is accepted. Both titles pass one `RenderInfo` per call: a count of 0 throws, and larger counts are handled element by element without proof. No error code is known, so invalid input throws instead of returning one - [NGS2 mastering voice gain](../../core/libs/prx/libSceNgs2.native/src/Voice.cpp) (libSceNgs2.native) - the param id 0x30000004 is taken from PPSA22520 and the layout from [shadPS4](https://github.com/shadps4-emu/shadPS4)'s `OrbisNgs2MasteringVoiceGainParam`; no reference gives the units or range. The levels are assumed to be linear factors applied when the mastering voice is mixed into the render buffer, so the voice's own samples stay unscaled; the LFE level applies to channel 3 of 6- and 8-channel voices, assuming the L R C LFE order of the AudioOut2 entries above, and the full-band level to the rest. Negative levels are accepted, and a new setup resets both levels to 1 - [NGS2 stereo mastering voice into a surround buffer](../../core/libs/prx/libSceNgs2.native/src/Render.cpp) (libSceNgs2.native) - PPSA21402 renders a 2-channel mastering voice into a 6- or 8-channel render buffer, which the mixing stage rejected. The console's conversion is unknown: the left and right channels are assumed to go to the first two buffer channels (front left and right) and the remaining channels stay silent, with no centre, LFE or surround content. Any other channel count mismatch still throws +- [NGS2 silent waveform blocks](../../core/libs/prx/libSceNgs2.native/src/Voice.cpp) (libSceNgs2.native): flag `0x10` (named `SILENCE` here; no public reference names it) lets a waveform block declare samples without data. PPSA04867 adds one block with flags `0x11`, `data_size` 0 and `num_samples` 16385, then another with flags `0x11` and a fully specified 0x800-byte, 1024-sample mono block, which takes the ordinary path. A dataless block under the flag outputs `num_samples` frames of silence, reports them through `decoded_data_size`, and finishes like a complete block; a dataless block without the flag still throws as not fitting its data. The flag's real name, its interaction with ATRAC9 and repeats, and the console's `waveform_data` for silent blocks are unverified on hardware - [NGS2 streaming PCM waveforms](../../core/libs/prx/libSceNgs2.native/src/Voice.cpp) (libSceNgs2.native) - PPSA21402 streams its music by adding one block whose `num_samples` is the whole track and whose `data_size` is a 16 KiB first piece, with `CONTINUE`, and then adds each following piece with flag `0x2` (called `APPEND` here) and the same `num_samples`. No reference describes flag `0x2`. A 16-bit PCM block without skipped samples or repeats whose `num_samples` exceed its data is assumed to start such a stream; flag `0x2` is assumed to append the block's data to the last streaming block, ignoring its `num_samples`; the voice is assumed to output silence without advancing while it has played all appended data and `CONTINUE` is set; and the waveform is assumed to end at `num_samples` or, once `CONTINUE` is cleared, at the end of the appended data. `sceNgs2VoiceGetState` reports the bytes played as `decoded_data_size`, which the title uses to release its queued data, and the read position as `waveform_data`. Appending to a block that is not a stream, or other formats, skipped samples, repeats and partial frames, throw - [NGS2 matrix levels](../../core/libs/prx/libSceNgs2.native/src/Voice.cpp) (libSceNgs2.native) - levels are clamped to -4..4 and NaN is kept, as the PS4 firmware 11.00 `libSceNgs2.sprx` stores them; not yet confirmed on the PS5 library. That library rejects a level count of 0 and keeps the matrix; the PS5 error code is unknown, so a count of 0 throws. `sceNgs2VoiceRunCommands` is not in that library, so its matrix levels command is assumed to clamp the same way - [sceAvPlayerStartEx](../../core/libs/prx/libSceAvPlayer/Export.cpp) (libSceAvPlayer) - start info layout unknown; it is ignored and playback starts as with `sceAvPlayerStart` From 271a8f291ee5ed50c2375b72ab9f75ea83bdd494 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 00:37:58 +0200 Subject: [PATCH 201/460] fix(libSceNgs2.native): throw for a dataless silence-flag block on an ATRAC9 voice GatherSuperframe finishes a block with no data at once, so such a block would not play num_samples of silence (requested by @oneandonlydean). --- core/libs/prx/libSceNgs2.native/src/Voice.cpp | 1 + .../prx/libSceNgs2.native/tests/Ngs2Atrac9.cpp | 16 ++++++++++++++++ 2 files changed, 17 insertions(+) diff --git a/core/libs/prx/libSceNgs2.native/src/Voice.cpp b/core/libs/prx/libSceNgs2.native/src/Voice.cpp index 5a192a12a..008c1cd26 100644 --- a/core/libs/prx/libSceNgs2.native/src/Voice.cpp +++ b/core/libs/prx/libSceNgs2.native/src/Voice.cpp @@ -223,6 +223,7 @@ static void AddWaveformBlocks(Ngs2Voice& voice, const Ngs2SamplerVoiceWaveformBl const auto& block = param.blocks[i]; if (block.num_samples == 0 && block.data_size == 0) continue; if (allowsSilence && block.data_size == 0) { + if (voice.waveformType == SCE_NGS2_WAVEFORM_TYPE_ATRAC9) throw std::runtime_error("NGS2: a dataless waveform block under the silence flag is not implemented for ATRAC9 voices"); voice.blocks.push_back({nullptr, block}); continue; } diff --git a/core/libs/prx/libSceNgs2.native/tests/Ngs2Atrac9.cpp b/core/libs/prx/libSceNgs2.native/tests/Ngs2Atrac9.cpp index a6e94c2a9..b9278520a 100644 --- a/core/libs/prx/libSceNgs2.native/tests/Ngs2Atrac9.cpp +++ b/core/libs/prx/libSceNgs2.native/tests/Ngs2Atrac9.cpp @@ -481,6 +481,21 @@ static void TestAppendWithoutWaveform() { Require(sceNgs2SystemDestroy(system, nullptr) == SCE_NGS2_OK); } +static void TestSilenceFlagRejected() { + const auto system = CreateSystem(); + const auto voice = Voice(CreateRack(system, SCE_NGS2_RACK_ID_SAMPLER)); + Control(voice, SCE_NGS2_SAMPLER_VOICE_PARAM_SETUP, Ngs2SamplerVoiceSetupParam{{}, {SCE_NGS2_WAVEFORM_TYPE_ATRAC9, 1, 48000, Config, 0, 0}}); + const Ngs2WaveformBlock block{0, 0, 0, 0, SuperframeSamples, 0, 0}; + bool rejected = false; + try { + Control(voice, SCE_NGS2_SAMPLER_VOICE_PARAM_ADD_WAVEFORM_BLOCKS, Ngs2SamplerVoiceWaveformBlocksParam{{}, Superframe, 0x10, 1, &block}); + } catch (const std::runtime_error& error) { + rejected = std::strstr(error.what(), "silence flag") != nullptr; + } + Require(rejected); + Require(sceNgs2SystemDestroy(system, nullptr) == SCE_NGS2_OK); +} + int main() { const auto reference = Reference(); TestSkipAndBlockEnd(reference); @@ -495,5 +510,6 @@ int main() { TestStarvationAndTruncation(reference); TestAppendBoundaries(reference); TestAppendWithoutWaveform(); + TestSilenceFlagRejected(); return 0; } From 2e19053f631aa608ec09dcfd94e750e3809367e4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jan=20=C5=A0l=C3=A9gl?= Date: Fri, 9 Oct 2026 22:58:53 +0200 Subject: [PATCH 202/460] feat(agc): present 16-bit float display buffers PPSA11386 registers 3840x2160 tiled display buffers in pixel format 0xC001000622000000, R16G16B16A16 float in KytyPS5 and SharpProspero (0xC001000600000000 is the blue-first variant), and stops on "VideoOut: unsupported display pixel format". The buffer is eight bytes per texel; its tiled layout is assumed to be the 8-byte SW_64KB_R_X equation applied to full coordinates, as addrlib applies it. The values are taken as linear light and encoded with the sRGB curve for the 8-bit output. The encoding compares the raw half-float bits against two tables of 255 thresholds (color and alpha) that the CPU decode and the color transfer shader share, so both give the same bytes on any driver; negative and NaN values give 0. Guest-memory detile, the resident image path and the CPU decode all handle both formats; DCC metadata on a float buffer throws. Neither the layout nor the transfer is verified against console frames. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Fnrk5Dazjb2tLooVSpdRQq --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 2 +- .../Execution/include/DisplayBuffer.hpp | 1 + .../Execution/include/DisplayFormat.hpp | 2 + .../Execution/src/DisplayBuffer.cpp | 50 ++++++- .../Execution/src/DisplayFormat.cpp | 14 +- .../Execution/src/VulkanDevice.cpp | 6 +- .../Graphics/include/GpuColorTransfer.hpp | 14 +- .../Graphics/include/HalfFloatScanout.hpp | 60 +++++++++ .../Graphics/shaders/ColorTransfer.comp | 25 +++- .../Graphics/src/GpuColorTransfer.cpp | 63 +++++---- .../tests/ColorTransferTests.cpp | 124 ++++++++++++++++++ .../libSceAgcDriver/tests/ResidentPresent.cpp | 24 +++- core/libs/tests/LinearDisplayBuffer.cpp | 78 +++++++++++ docs/dev/TechnicalDebt.md | 1 + 14 files changed, 420 insertions(+), 44 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/Graphics/include/HalfFloatScanout.hpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index fd3eb4310..8af81bd00 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -1118,7 +1118,7 @@ if(BUILD_TESTING) "PATH=path_list_prepend:$") endif() add_test_executable(linear_display_buffer_tests ${LIBS_INCLUDE_DIR}/tests/LinearDisplayBuffer.cpp) - target_include_directories(linear_display_buffer_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_include_directories(linear_display_buffer_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include) target_link_libraries(linear_display_buffer_tests PRIVATE libSceAgcDriver libc) configure_windows_unwind(linear_display_buffer_tests) add_test(NAME linear_display_buffer COMMAND linear_display_buffer_tests) diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/DisplayBuffer.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/DisplayBuffer.hpp index 24a7be45b..e01a9ff0f 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/DisplayBuffer.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/DisplayBuffer.hpp @@ -25,6 +25,7 @@ struct DisplayBuffer { }; std::size_t DisplayBufferSize(const DisplayBuffer& buffer); +std::size_t DisplayBufferOffset(const DisplayBuffer& buffer, std::uint32_t x, std::uint32_t y); std::vector DecodeDisplayBuffer(const DisplayBuffer& buffer, std::span source); std::vector ReadDisplayBuffer(const DisplayBuffer& buffer); std::array DisplayBufferClearPixel(const DisplayBuffer& buffer, Graphics::DccKeys keys); diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/DisplayFormat.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/DisplayFormat.hpp index fc18cff57..899822c38 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/DisplayFormat.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/DisplayFormat.hpp @@ -10,6 +10,8 @@ namespace AgcDriver { VkFormat DisplayTexelFormat(std::uint64_t pixelFormat); bool DisplayTenBit(std::uint64_t pixelFormat); bool DisplayRedLow(std::uint64_t pixelFormat); +bool DisplayHalfFloat(std::uint64_t pixelFormat); +std::uint32_t DisplayTexelBytes(std::uint64_t pixelFormat); enum class ResidentPresent : std::uint8_t { None, Blit, Convert }; ResidentPresent ResidentPresentPath(VkFormat storage, std::uint64_t pixelFormat, bool blitSource); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/DisplayBuffer.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/DisplayBuffer.cpp index 8bc53af7d..d59fe232b 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/DisplayBuffer.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/DisplayBuffer.cpp @@ -3,6 +3,7 @@ #include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" #include "prx/libSceAgcDriver/Execution/include/PerformanceTimer.hpp" #include "prx/libSceAgcDriver/Graphics/include/DccMetadata.hpp" +#include "prx/libSceAgcDriver/Graphics/include/HalfFloatScanout.hpp" #include #include #include @@ -33,6 +34,30 @@ std::uint32_t tileOffset(std::uint32_t x, std::uint32_t y) { return ((y << 4u) & 0x0070u) ^ ((y << 5u) & 0x0f00u) ^ ((y << 9u) & 0x1000u) ^ ((y << 8u) & 0x4000u) ^ ((x << 2u) & 0x000cu) ^ ((x << 5u) & 0x0380u) ^ ((x << 4u) & 0x0400u) ^ ((x << 6u) & 0x0800u) ^ ((x << 9u) & 0xa000u); } +std::uint32_t tileOffset64(std::uint32_t x, std::uint32_t y) { + return ((x << 3u) & 0x0008u) ^ ((x << 4u) & 0x0460u) ^ ((x << 5u) & 0x0300u) ^ ((x << 6u) & 0x0800u) ^ ((x << 9u) & 0x8000u) ^ ((x << 10u) & 0x2000u) ^ ((y << 4u) & 0x0010u) ^ ((y << 6u) & 0x0080u) ^ ((y << 5u) & 0x0f00u) ^ ((y << 10u) & 0x5000u); +} + +std::size_t texelOffset(const DisplayBuffer& buffer, std::uint32_t x, std::uint32_t y, std::uint32_t texelBytes) { + if (buffer.tilingMode == 1) { + const auto pitch = buffer.pitchInPixel == 0 ? buffer.width : buffer.pitchInPixel; + return (static_cast(y) * pitch + x) * texelBytes; + } + const auto blocksPerRow = (buffer.width + 127u) / 128u; + const auto blockHeight = 65536u / 128u / texelBytes; + const auto block = static_cast(y / blockHeight) * blocksPerRow + x / 128u; + return block * 65536u + (texelBytes == 8u ? tileOffset64(x, y) : tileOffset(x, y)); +} + +void decodeHalfFloatTexel(const std::byte* texel, std::byte* pixel, bool redLow) { + std::array halves{}; + std::memcpy(halves.data(), texel, sizeof(halves)); + pixel[0] = static_cast(Graphics::HalfFloatScanoutCode(halves[redLow ? 2 : 0], false)); + pixel[1] = static_cast(Graphics::HalfFloatScanoutCode(halves[1], false)); + pixel[2] = static_cast(Graphics::HalfFloatScanoutCode(halves[redLow ? 0 : 2], false)); + pixel[3] = static_cast(Graphics::HalfFloatScanoutCode(halves[3], true)); +} + void decodeTexel(const std::byte* texel, std::byte* pixel, bool rgba, bool tenBit) { if (tenBit) { std::uint32_t word = 0; @@ -90,11 +115,13 @@ void traceDisplayKeys(const DisplayBuffer& buffer, const std::uint8_t* keys, std std::size_t DisplayBufferSize(const DisplayBuffer& buffer) { require(buffer.width != 0 && buffer.height != 0 && buffer.width <= 16384 && buffer.height <= 16384, "VideoOut: invalid display buffer dimensions"); - if (baseFormat(buffer.pixelFormat) != PixelFormatB8G8R8A8 && baseFormat(buffer.pixelFormat) != PixelFormatR8G8B8A8) { + const bool halfFloat = DisplayHalfFloat(buffer.pixelFormat); + if (!halfFloat && baseFormat(buffer.pixelFormat) != PixelFormatB8G8R8A8 && baseFormat(buffer.pixelFormat) != PixelFormatR8G8B8A8) { char message[96]; std::snprintf(message, sizeof(message), "VideoOut: unsupported display pixel format 0x%016llx", static_cast(buffer.pixelFormat)); throw std::runtime_error(message); } + require(!halfFloat || buffer.dccAddress == 0, "VideoOut: DCC metadata on a 16-bit float display buffer is not implemented"); require(buffer.address != 0 && (buffer.address & 65535u) == 0, "VideoOut: display buffer requires 64 KiB alignment"); require(buffer.tilingMode <= 1, "VideoOut: unsupported display tiling mode"); require(buffer.tilingMode != 0 || buffer.pitchInPixel == 0, "VideoOut: tiled display pitch is unsupported"); @@ -102,16 +129,33 @@ std::size_t DisplayBufferSize(const DisplayBuffer& buffer) { require(buffer.dccAddress != 0 || buffer.dccClearColor == 0, "VideoOut: a DCC clear color needs DCC metadata"); const auto pitch = buffer.pitchInPixel == 0 ? buffer.width : buffer.pitchInPixel; require(pitch >= buffer.width && pitch <= 16384, "VideoOut: invalid linear display pitch"); - const auto size = buffer.tilingMode == 1 ? static_cast(pitch) * buffer.height * 4u - : static_cast((buffer.width + 127u) / 128u) * ((buffer.height + 127u) / 128u) * 65536u; + const auto texelBytes = DisplayTexelBytes(buffer.pixelFormat); + const auto blockHeight = 65536u / 128u / texelBytes; + const auto size = buffer.tilingMode == 1 ? static_cast(pitch) * buffer.height * texelBytes + : static_cast((buffer.width + 127u) / 128u) * ((buffer.height + blockHeight - 1u) / blockHeight) * 65536u; require(size <= std::numeric_limits::max() && size <= std::numeric_limits::max() - buffer.address, "VideoOut: display buffer range overflow"); return static_cast(size); } +std::size_t DisplayBufferOffset(const DisplayBuffer& buffer, std::uint32_t x, std::uint32_t y) { + static_cast(DisplayBufferSize(buffer)); + require(x < buffer.width && y < buffer.height, "VideoOut: display buffer coordinate out of range"); + return texelOffset(buffer, x, y, DisplayTexelBytes(buffer.pixelFormat)); +} + std::vector DecodeDisplayBuffer(const DisplayBuffer& buffer, std::span source) { PerformanceTimer timing("DisplayBuffer.Decode"); require(source.size() == DisplayBufferSize(buffer), "VideoOut: invalid display buffer size"); std::vector pixels(static_cast(buffer.width) * buffer.height * 4); + if (DisplayHalfFloat(buffer.pixelFormat)) { + const bool redLow = DisplayRedLow(buffer.pixelFormat); + timing.Mark("validate_allocate"); + for (std::uint32_t y = 0; y < buffer.height; ++y) { + for (std::uint32_t x = 0; x < buffer.width; ++x) decodeHalfFloatTexel(source.data() + texelOffset(buffer, x, y, 8u), pixels.data() + (static_cast(y) * buffer.width + x) * 4, redLow); + } + timing.Mark("detile_convert"); + return pixels; + } const auto blocksPerRow = (buffer.width + 127u) / 128u; const bool rgba = baseFormat(buffer.pixelFormat) == PixelFormatR8G8B8A8; // Bit 56 marks the 10-bit variant: A2B10G10R10 with red in the low bits (the scanout draws render diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/DisplayFormat.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/DisplayFormat.cpp index 7a013fdbe..7ca977ac5 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/DisplayFormat.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/DisplayFormat.cpp @@ -5,6 +5,8 @@ namespace { constexpr std::uint64_t PixelFormatUnormBit = 0x0100000000000000ull; constexpr std::uint64_t PixelFormatR8G8B8A8 = 0x8000000022000000ull; +constexpr std::uint64_t PixelFormatR16G16B16A16Float = 0xc001000622000000ull; +constexpr std::uint64_t PixelFormatB16G16R16A16Float = 0xc001000600000000ull; } @@ -13,16 +15,26 @@ bool DisplayTenBit(std::uint64_t pixelFormat) { } bool DisplayRedLow(std::uint64_t pixelFormat) { - return (pixelFormat & ~PixelFormatUnormBit) == PixelFormatR8G8B8A8; + return (pixelFormat & ~PixelFormatUnormBit) == PixelFormatR8G8B8A8 || pixelFormat == PixelFormatR16G16B16A16Float; +} + +bool DisplayHalfFloat(std::uint64_t pixelFormat) { + return pixelFormat == PixelFormatR16G16B16A16Float || pixelFormat == PixelFormatB16G16R16A16Float; +} + +std::uint32_t DisplayTexelBytes(std::uint64_t pixelFormat) { + return DisplayHalfFloat(pixelFormat) ? 8u : 4u; } VkFormat DisplayTexelFormat(std::uint64_t pixelFormat) { + if (DisplayHalfFloat(pixelFormat)) return VK_FORMAT_R16G16B16A16_SFLOAT; const bool redLow = DisplayRedLow(pixelFormat); if (DisplayTenBit(pixelFormat)) return redLow ? VK_FORMAT_A2B10G10R10_UNORM_PACK32 : VK_FORMAT_A2R10G10B10_UNORM_PACK32; return redLow ? VK_FORMAT_R8G8B8A8_UNORM : VK_FORMAT_B8G8R8A8_UNORM; } ResidentPresent ResidentPresentPath(VkFormat storage, std::uint64_t pixelFormat, bool blitSource) { + if (DisplayHalfFloat(pixelFormat)) return storage == VK_FORMAT_R16G16B16A16_SFLOAT ? ResidentPresent::Convert : ResidentPresent::None; const bool tenBit = DisplayTenBit(pixelFormat); if (!tenBit && blitSource && storage == DisplayTexelFormat(pixelFormat)) return ResidentPresent::Blit; switch (storage) { diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index aa1135187..791974434 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -2342,7 +2342,7 @@ bool VulkanDevice::present(std::uint32_t width, std::uint32_t height, bool opaqu require(!slot.inFlight, "presentation slot is still in flight"); if (display != nullptr && resident == nullptr) { static_cast(DisplayBufferSize(*display)); - state->colorTransfer->Upload(display->address, width, height, Graphics::ColorTileMode::RenderTarget); + state->colorTransfer->Upload(display->address, width, height, Graphics::ColorTileMode::RenderTarget, DisplayTexelBytes(display->pixelFormat)); } if (!pixels.empty()) state->Upload(pixels); // The frame's recorded work (and a refresh of the resident image) must reach the queue before the @@ -2411,12 +2411,12 @@ bool VulkanDevice::present(std::uint32_t width, std::uint32_t height, bool opaqu pipelineBarrier(commands, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, 0, nullptr, 0, nullptr, 1, &residentBarrier); Graphics::Recorder::CountBarriers(CommandClass::PresentBlit); if (residentConvert) { - state->colorTransfer->DetileImage(commands, resident->Image(), VK_IMAGE_LAYOUT_GENERAL, width, height, Graphics::ColorTileMode::RenderTarget, DisplayRedLow(display->pixelFormat), DisplayTenBit(display->pixelFormat)); + state->colorTransfer->DetileImage(commands, resident->Image(), VK_IMAGE_LAYOUT_GENERAL, width, height, Graphics::ColorTileMode::RenderTarget, DisplayRedLow(display->pixelFormat), DisplayTenBit(display->pixelFormat), DisplayHalfFloat(display->pixelFormat)); state->scaler->RecordUpload(commands, state->colorTransfer->LinearBuffer()); } } else { if (display != nullptr) { - state->colorTransfer->Detile(commands, DisplayRedLow(display->pixelFormat), DisplayTenBit(display->pixelFormat)); + state->colorTransfer->Detile(commands, DisplayRedLow(display->pixelFormat), DisplayTenBit(display->pixelFormat), DisplayHalfFloat(display->pixelFormat)); } if (uniform != nullptr) state->scaler->RecordClear(commands, *uniform); else state->scaler->RecordUpload(commands, display != nullptr ? state->colorTransfer->LinearBuffer() : state->uploadBuffer); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/GpuColorTransfer.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/GpuColorTransfer.hpp index 3d1aaf2ba..246cc3ba9 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/GpuColorTransfer.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/GpuColorTransfer.hpp @@ -12,18 +12,18 @@ public: ~GpuColorTransfer(); GpuColorTransfer(const GpuColorTransfer&) = delete; GpuColorTransfer& operator=(const GpuColorTransfer&) = delete; - void Upload(std::uint64_t address, std::uint32_t width, std::uint32_t height, ColorTileMode mode); - void Detile(VkCommandBuffer commands, bool swapRedBlue = false, bool tenBit = false); - void DetileImage(VkCommandBuffer commands, VkImage image, VkImageLayout layout, std::uint32_t width, std::uint32_t height, ColorTileMode mode, bool swapRedBlue, bool tenBit); + void Upload(std::uint64_t address, std::uint32_t width, std::uint32_t height, ColorTileMode mode, std::uint32_t elementBytes = 4); + void Detile(VkCommandBuffer commands, bool swapRedBlue = false, bool tenBit = false, bool halfFloat = false); + void DetileImage(VkCommandBuffer commands, VkImage image, VkImageLayout layout, std::uint32_t width, std::uint32_t height, ColorTileMode mode, bool swapRedBlue, bool tenBit, bool halfFloat = false); void Tile(VkCommandBuffer commands); void WriteBack(std::uint64_t address); VkBuffer LinearBuffer() const; RenderTarget& Target(const ColorTarget& color, bool blending); private: - void prepare(std::uint32_t width, std::uint32_t height, ColorTileMode mode); - void convert(VkCommandBuffer commands, bool toTiled, bool swapRedBlue, bool tenBit = false); - void dispatch(VkCommandBuffer commands, bool toTiled, bool swapRedBlue, bool tenBit, bool tiledSource); + void prepare(std::uint32_t width, std::uint32_t height, ColorTileMode mode, std::uint32_t elementBytes); + void convert(VkCommandBuffer commands, bool toTiled, bool swapRedBlue, bool tenBit = false, bool halfFloat = false); + void dispatch(VkCommandBuffer commands, bool toTiled, bool swapRedBlue, bool tenBit, bool tiledSource, bool halfFloat); void release() noexcept; Context context; VkDescriptorSetLayout descriptorLayout = VK_NULL_HANDLE; @@ -35,9 +35,11 @@ private: std::unique_ptr tiled; std::unique_ptr tiledDevice; std::unique_ptr linear; + std::unique_ptr thresholds; std::uint32_t width = 0; std::uint32_t height = 0; ColorTileMode mode = ColorTileMode::Linear; + std::uint32_t elementBytes = 4; std::unique_ptr target; VkExtent2D targetExtent{}; VkFormat targetFormat = VK_FORMAT_UNDEFINED; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/HalfFloatScanout.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/HalfFloatScanout.hpp new file mode 100644 index 000000000..8a586a948 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/HalfFloatScanout.hpp @@ -0,0 +1,60 @@ +#ifndef CORE_LIBS_PRX_LIBSCEAGCDRIVER_GRAPHICS_INCLUDE_HALFFLOATSCANOUT_HPP +#define CORE_LIBS_PRX_LIBSCEAGCDRIVER_GRAPHICS_INCLUDE_HALFFLOATSCANOUT_HPP + +#include +#include +#include + +namespace AgcDriver::Graphics { + +inline constexpr std::uint32_t HalfFloatScanoutCodes = 255; +inline constexpr std::uint32_t HalfFloatPositiveInfinity = 0x7c00u; + +inline const std::array& HalfFloatScanoutThresholds() { + static const auto thresholds = [] { + std::array table{}; + const auto finiteValue = [](std::uint32_t bits) { + const auto exponent = static_cast(bits >> 10u); + const auto mantissa = static_cast(bits & 0x3ffu); + return exponent == 0 ? std::ldexp(mantissa, -24) : std::ldexp(1024.0 + mantissa, exponent - 25); + }; + const auto firstAtLeast = [&](double value) { + std::uint32_t low = 0; + std::uint32_t high = HalfFloatPositiveInfinity; + while (low < high) { + const auto middle = (low + high) / 2u; + if (finiteValue(middle) >= value) high = middle; + else low = middle + 1u; + } + return low; + }; + for (std::uint32_t code = 1; code <= HalfFloatScanoutCodes; ++code) { + const double encoded = (code - 0.5) / 255.0; + table[code - 1u] = firstAtLeast(encoded <= 0.04045 ? encoded / 12.92 : std::pow((encoded + 0.055) / 1.055, 2.4)); + table[HalfFloatScanoutCodes + code - 1u] = firstAtLeast(encoded); + } + return table; + }(); + return thresholds; +} + +inline std::uint8_t HalfFloatScanoutCode(std::uint16_t bits, bool alpha) { + static std::array, 2> codes{}; + static const bool filled = [] { + const auto& thresholds = HalfFloatScanoutThresholds(); + for (std::uint32_t kind = 0; kind < 2u; ++kind) { + for (std::uint32_t half = 0; half <= HalfFloatPositiveInfinity; ++half) { + std::uint32_t code = 0; + while (code < HalfFloatScanoutCodes && thresholds[kind * HalfFloatScanoutCodes + code] <= half) ++code; + codes[kind][half] = static_cast(code); + } + } + return true; + }(); + static_cast(filled); + return codes[alpha ? 1u : 0u][bits]; +} + +} + +#endif diff --git a/core/libs/prx/libSceAgcDriver/Graphics/shaders/ColorTransfer.comp b/core/libs/prx/libSceAgcDriver/Graphics/shaders/ColorTransfer.comp index d9ae43c66..c494b7aee 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/shaders/ColorTransfer.comp +++ b/core/libs/prx/libSceAgcDriver/Graphics/shaders/ColorTransfer.comp @@ -3,6 +3,7 @@ layout(local_size_x = 8, local_size_y = 8) in; layout(set = 0, binding = 0, std430) buffer TiledPixels { uint tiled[]; }; layout(set = 0, binding = 1, std430) buffer LinearPixels { uint linear[]; }; +layout(set = 0, binding = 2, std430) readonly buffer ScanoutThresholds { uint thresholds[]; }; layout(push_constant) uniform Parameters { uint width; uint height; @@ -10,6 +11,15 @@ layout(push_constant) uniform Parameters { uint mode; } parameters; +uint scanoutCode(uint bits, uint base) { + if (bits > 0x7c00u) return 0u; + uint code = 0u; + for (uint step = 128u; step != 0u; step >>= 1u) { + if (code + step <= 255u && thresholds[base + code + step - 1u] <= bits) code += step; + } + return code; +} + void main() { uint x = gl_GlobalInvocationID.x; uint y = gl_GlobalInvocationID.y; @@ -21,7 +31,20 @@ void main() { ^ ((x << 2u) & 0x000cu) ^ ((x << 5u) & 0x0380u) ^ ((x << 4u) & 0x0400u) ^ ((x << 6u) & 0x0800u) ^ ((x << 9u) & 0xa000u); tiledIndex = ((y / 128u) * parameters.blocksPerRow + x / 128u) * 16384u + offset / 4u; } - uint pixel = (parameters.mode & 1u) != 0u ? linear[linearIndex] : tiled[tiledIndex]; + uint pixel; + if ((parameters.mode & 16u) != 0u) { + uint texel = linearIndex * 2u; + if ((parameters.mode & 4u) != 0u) { + uint offset = ((x << 3u) & 0x0008u) ^ ((x << 4u) & 0x0460u) ^ ((x << 5u) & 0x0300u) ^ ((x << 6u) & 0x0800u) ^ ((x << 9u) & 0x8000u) ^ ((x << 10u) & 0x2000u) + ^ ((y << 4u) & 0x0010u) ^ ((y << 6u) & 0x0080u) ^ ((y << 5u) & 0x0f00u) ^ ((y << 10u) & 0x5000u); + texel = (((y / 64u) * parameters.blocksPerRow + x / 128u) * 65536u + offset) / 4u; + } + uint first = tiled[texel]; + uint second = tiled[texel + 1u]; + pixel = scanoutCode(first & 0xffffu, 0u) | (scanoutCode(first >> 16u, 0u) << 8u) | (scanoutCode(second & 0xffffu, 0u) << 16u) | (scanoutCode(second >> 16u, 255u) << 24u); + } else { + pixel = (parameters.mode & 1u) != 0u ? linear[linearIndex] : tiled[tiledIndex]; + } // Mode 8: A2B10G10R10 source (red in the low bits) presented as 8-bit RGBA. if ((parameters.mode & 8u) != 0u) pixel = ((pixel >> 2u) & 0xffu) | (((pixel >> 12u) & 0xffu) << 8u) | (((pixel >> 22u) & 0xffu) << 16u) | 0xff000000u; if ((parameters.mode & 2u) != 0u) pixel = (pixel & 0xff00ff00u) | ((pixel & 0xffu) << 16u) | ((pixel >> 16u) & 0xffu); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/GpuColorTransfer.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/GpuColorTransfer.cpp index 346ba590a..4aec57b1d 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/GpuColorTransfer.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/GpuColorTransfer.cpp @@ -1,14 +1,16 @@ #include "prx/libSceAgcDriver/Graphics/include/GpuColorTransfer.hpp" #include "prx/libSceAgcDriver/Graphics/shaders/ColorTransfer_spv.h" #include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" +#include "prx/libSceAgcDriver/Graphics/include/HalfFloatScanout.hpp" #include +#include namespace AgcDriver::Graphics { GpuColorTransfer::GpuColorTransfer(const Context& context) : context(context) { VkShaderModule module = VK_NULL_HANDLE; try { - std::array bindings{}; + std::array bindings{}; for (std::uint32_t i = 0; i < bindings.size(); ++i) { bindings[i] = {i, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT, nullptr}; } @@ -36,7 +38,7 @@ GpuColorTransfer::GpuColorTransfer(const Context& context) : context(context) { Check(context.Function("vkCreateComputePipelines")(context.device, context.pipelineCache, 1, &pipelineInfo, nullptr, &pipeline), "vkCreateComputePipelines color transfer"); context.Function("vkDestroyShaderModule")(context.device, module, nullptr); module = VK_NULL_HANDLE; - const VkDescriptorPoolSize size{VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 2}; + const VkDescriptorPoolSize size{VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 3}; VkDescriptorPoolCreateInfo poolInfo{VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO}; poolInfo.maxSets = 1; poolInfo.poolSizeCount = 1; @@ -47,6 +49,17 @@ GpuColorTransfer::GpuColorTransfer(const Context& context) : context(context) { allocation.descriptorSetCount = 1; allocation.pSetLayouts = &descriptorLayout; Check(context.Function("vkAllocateDescriptorSets")(context.device, &allocation, &descriptorSet), "vkAllocateDescriptorSets color transfer"); + const auto& table = HalfFloatScanoutThresholds(); + thresholds = std::make_unique(context, sizeof(table), VK_BUFFER_USAGE_STORAGE_BUFFER_BIT); + std::memcpy(thresholds->Bytes().data(), table.data(), sizeof(table)); + const VkDescriptorBufferInfo thresholdInfo{thresholds->Handle(), 0, sizeof(table)}; + VkWriteDescriptorSet thresholdWrite{VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET}; + thresholdWrite.dstSet = descriptorSet; + thresholdWrite.dstBinding = 2; + thresholdWrite.descriptorCount = 1; + thresholdWrite.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER; + thresholdWrite.pBufferInfo = &thresholdInfo; + context.Function("vkUpdateDescriptorSets")(context.device, 1, &thresholdWrite, 0, nullptr); } catch (...) { if (module) context.Function("vkDestroyShaderModule")(context.device, module, nullptr); release(); @@ -65,16 +78,18 @@ void GpuColorTransfer::release() noexcept { if (descriptorLayout) context.Function("vkDestroyDescriptorSetLayout")(context.device, descriptorLayout, nullptr); } -void GpuColorTransfer::prepare(std::uint32_t newWidth, std::uint32_t newHeight, ColorTileMode newMode) { - const ColorTargetLayout layout(newWidth, newHeight, newMode); - Require(layout.Bytes() <= context.limits.maxStorageBufferRange && layout.LinearBytes() <= context.limits.maxStorageBufferRange, "color transfer exceeds storage buffer limits"); +void GpuColorTransfer::prepare(std::uint32_t newWidth, std::uint32_t newHeight, ColorTileMode newMode, std::uint32_t newElementBytes) { + Require(newElementBytes == 4u || (newElementBytes == 8u && newMode == ColorTileMode::RenderTarget), "color transfer supports 8-byte texels only in the render target layout"); + const ColorTargetLayout layout(newWidth, newHeight, newMode, newElementBytes); + const auto linearBytes = static_cast(newWidth) * newHeight * 4u; + Require(layout.Bytes() <= context.limits.maxStorageBufferRange && linearBytes <= context.limits.maxStorageBufferRange, "color transfer exceeds storage buffer limits"); Require((newWidth + 7u) / 8u <= context.limits.maxComputeWorkGroupCount[0] && (newHeight + 7u) / 8u <= context.limits.maxComputeWorkGroupCount[1], "color transfer exceeds workgroup limits"); - if (tiled && width == newWidth && height == newHeight && mode == newMode) return; + if (tiled && width == newWidth && height == newHeight && mode == newMode && elementBytes == newElementBytes) return; constexpr VkBufferUsageFlags copies = VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT; auto newTiled = std::make_unique(context, layout.Bytes(), copies); auto newTiledDevice = std::make_unique(context, layout.Bytes(), copies | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT); - auto newLinear = std::make_unique(context, layout.LinearBytes(), copies | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT); - const std::array buffers{{{newTiledDevice->Handle(), 0, layout.Bytes()}, {newLinear->Handle(), 0, layout.LinearBytes()}}}; + auto newLinear = std::make_unique(context, linearBytes, copies | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT); + const std::array buffers{{{newTiledDevice->Handle(), 0, layout.Bytes()}, {newLinear->Handle(), 0, linearBytes}}}; std::array writes{}; for (std::uint32_t i = 0; i < writes.size(); ++i) { writes[i].sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET; @@ -91,33 +106,36 @@ void GpuColorTransfer::prepare(std::uint32_t newWidth, std::uint32_t newHeight, width = newWidth; height = newHeight; mode = newMode; + elementBytes = newElementBytes; } -void GpuColorTransfer::Upload(std::uint64_t address, std::uint32_t newWidth, std::uint32_t newHeight, ColorTileMode newMode) { - prepare(newWidth, newHeight, newMode); - const ColorTargetLayout layout(width, height, mode); +void GpuColorTransfer::Upload(std::uint64_t address, std::uint32_t newWidth, std::uint32_t newHeight, ColorTileMode newMode, std::uint32_t newElementBytes) { + prepare(newWidth, newHeight, newMode, newElementBytes); + const ColorTargetLayout layout(width, height, mode, elementBytes); GuestMemory::Read(address, tiled->Bytes(), layout.Alignment()); } -void GpuColorTransfer::convert(VkCommandBuffer commands, bool toTiled, bool swapRedBlue, bool tenBit) { +void GpuColorTransfer::convert(VkCommandBuffer commands, bool toTiled, bool swapRedBlue, bool tenBit, bool halfFloat) { Require(commands != VK_NULL_HANDLE && tiled && tiledDevice && linear, "color transfer is not prepared"); + Require(halfFloat == (elementBytes == 8u), "color transfer source does not match the prepared texel size"); + Require(!toTiled || !halfFloat, "color transfer cannot write 16-bit float texels back to the guest"); // The guest bytes move to video memory with one DMA copy; the shader's scattered accesses stay local. RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_HOST_BIT | VK_PIPELINE_STAGE_TRANSFER_BIT | VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_ACCESS_HOST_WRITE_BIT | VK_ACCESS_TRANSFER_WRITE_BIT | VK_ACCESS_TRANSFER_READ_BIT | VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_SHADER_WRITE_BIT, VK_ACCESS_TRANSFER_READ_BIT | VK_ACCESS_TRANSFER_WRITE_BIT); CopyBuffer(context, commands, tiled->Handle(), 0, tiledDevice->Handle(), 0, tiledDevice->Size()); - dispatch(commands, toTiled, swapRedBlue, tenBit, mode == ColorTileMode::RenderTarget); + dispatch(commands, toTiled, swapRedBlue, tenBit, mode == ColorTileMode::RenderTarget, halfFloat); if (toTiled) { CopyBuffer(context, commands, tiledDevice->Handle(), 0, tiled->Handle(), 0, tiledDevice->Size()); RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_HOST_BIT, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_HOST_READ_BIT); } } -void GpuColorTransfer::dispatch(VkCommandBuffer commands, bool toTiled, bool swapRedBlue, bool tenBit, bool tiledSource) { +void GpuColorTransfer::dispatch(VkCommandBuffer commands, bool toTiled, bool swapRedBlue, bool tenBit, bool tiledSource, bool halfFloat) { VkMemoryBarrier before{VK_STRUCTURE_TYPE_MEMORY_BARRIER}; before.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT | VK_ACCESS_TRANSFER_READ_BIT | VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_SHADER_WRITE_BIT; before.dstAccessMask = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_SHADER_WRITE_BIT; const auto barrier = context.Function("vkCmdPipelineBarrier"); barrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT | VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, 0, 1, &before, 0, nullptr, 0, nullptr); - const std::array push{width, height, (width + 127u) / 128u, (toTiled ? 1u : 0u) | (swapRedBlue ? 2u : 0u) | (tiledSource ? 4u : 0u) | (tenBit ? 8u : 0u)}; + const std::array push{width, height, (width + 127u) / 128u, (toTiled ? 1u : 0u) | (swapRedBlue ? 2u : 0u) | (tiledSource ? 4u : 0u) | (tenBit ? 8u : 0u) | (halfFloat ? 16u : 0u)}; context.Function("vkCmdBindPipeline")(commands, VK_PIPELINE_BIND_POINT_COMPUTE, pipeline); context.Function("vkCmdBindDescriptorSets")(commands, VK_PIPELINE_BIND_POINT_COMPUTE, pipelineLayout, 0, 1, &descriptorSet, 0, nullptr); context.Function("vkCmdPushConstants")(commands, pipelineLayout, VK_SHADER_STAGE_COMPUTE_BIT, 0, sizeof(push), push.data()); @@ -128,20 +146,20 @@ void GpuColorTransfer::dispatch(VkCommandBuffer commands, bool toTiled, bool swa barrier(commands, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, 1, &after, 0, nullptr, 0, nullptr); } -void GpuColorTransfer::DetileImage(VkCommandBuffer commands, VkImage image, VkImageLayout layout, std::uint32_t newWidth, std::uint32_t newHeight, ColorTileMode newMode, bool swapRedBlue, bool tenBit) { +void GpuColorTransfer::DetileImage(VkCommandBuffer commands, VkImage image, VkImageLayout layout, std::uint32_t newWidth, std::uint32_t newHeight, ColorTileMode newMode, bool swapRedBlue, bool tenBit, bool halfFloat) { Require(commands != VK_NULL_HANDLE && image != VK_NULL_HANDLE, "color transfer image source is unavailable"); - prepare(newWidth, newHeight, newMode); - Require(tiledDevice->Size() >= static_cast(width) * height * 4u, "color transfer buffer is smaller than the image"); + prepare(newWidth, newHeight, newMode, halfFloat ? 8u : 4u); + Require(tiledDevice->Size() >= static_cast(width) * height * elementBytes, "color transfer buffer is smaller than the image"); RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_TRANSFER_BIT | VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_ACCESS_TRANSFER_WRITE_BIT | VK_ACCESS_TRANSFER_READ_BIT | VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_SHADER_WRITE_BIT, VK_ACCESS_TRANSFER_WRITE_BIT); VkBufferImageCopy copy{}; copy.imageSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 0, 1}; copy.imageExtent = {width, height, 1}; context.Function("vkCmdCopyImageToBuffer")(commands, image, layout, tiledDevice->Handle(), 1, ©); - dispatch(commands, false, swapRedBlue, tenBit, false); + dispatch(commands, false, swapRedBlue, tenBit, false, halfFloat); } -void GpuColorTransfer::Detile(VkCommandBuffer commands, bool swapRedBlue, bool tenBit) { - convert(commands, false, swapRedBlue, tenBit); +void GpuColorTransfer::Detile(VkCommandBuffer commands, bool swapRedBlue, bool tenBit, bool halfFloat) { + convert(commands, false, swapRedBlue, tenBit, halfFloat); } void GpuColorTransfer::Tile(VkCommandBuffer commands) { @@ -150,7 +168,8 @@ void GpuColorTransfer::Tile(VkCommandBuffer commands) { void GpuColorTransfer::WriteBack(std::uint64_t address) { Require(tiled != nullptr, "color transfer is not prepared for writeback"); - const ColorTargetLayout layout(width, height, mode); + Require(elementBytes == 4u, "color transfer cannot write 16-bit float texels back to the guest"); + const ColorTargetLayout layout(width, height, mode, elementBytes); GuestMemory::Write(address, tiled->Bytes(), layout.Alignment()); } diff --git a/core/libs/prx/libSceAgcDriver/tests/ColorTransferTests.cpp b/core/libs/prx/libSceAgcDriver/tests/ColorTransferTests.cpp index 6a032c776..4554351ae 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ColorTransferTests.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ColorTransferTests.cpp @@ -1,6 +1,9 @@ #include "ColorTransferTests.hpp" #include "prx/libSceAgcDriver/Graphics/include/GpuColorTransfer.hpp" +#include "prx/libSceAgcDriver/Graphics/include/HalfFloatScanout.hpp" +#include "prx/libSceAgcDriver/Graphics/include/TextureSwizzleEquations.hpp" #include +#include #include #include #include @@ -53,6 +56,125 @@ void checkConversion(const Context& context, std::uint32_t width, std::uint32_t Require(std::equal(expectedTiled.begin(), expectedTiled.end(), guest.begin()), "GPU tile changed pixels or surface padding"); } +std::size_t renderTargetOffset64(std::uint32_t blocksPerRow, std::uint32_t x, std::uint32_t y) { + const auto* equation = FindTextureSwizzleEquation(27u, 8u); + Require(equation != nullptr, "no 8-byte SW_64KB_R_X equation"); + std::size_t offset = 0; + for (std::uint32_t bit = 0; bit < 16u; ++bit) { + const auto mask = equation->bits[bit]; + offset |= static_cast(std::popcount((x & mask & 0xfffu) ^ ((y << 12u) & mask & 0xfff000u)) & 1) << bit; + } + return (static_cast(y / 64u) * blocksPerRow + x / 128u) * 65536u + offset; +} + +std::uint32_t deviceLocalType(const Context& context, std::uint32_t bits) { + for (std::uint32_t i = 0; i < context.memory.memoryTypeCount; ++i) { + if ((bits & (1u << i)) != 0 && (context.memory.memoryTypes[i].propertyFlags & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT) != 0) return i; + } + Require(false, "no device-local memory type for the test image"); + return 0; +} + +void checkHalfFloatDetile(const Context& context, std::uint32_t width, std::uint32_t height, bool redLow) { + const ColorTargetLayout layout(width, height, ColorTileMode::RenderTarget, 8); + std::vector storage(layout.Bytes() + layout.Alignment()); + void* aligned = storage.data(); + auto available = storage.size(); + Require(std::align(layout.Alignment(), layout.Bytes(), aligned, available) != nullptr, "test surface alignment failed"); + std::span guest(static_cast(aligned), layout.Bytes()); + std::fill(guest.begin(), guest.end(), std::byte{0xa5}); + std::vector texels(static_cast(width) * height * 8); + std::vector expected(static_cast(width) * height * 4); + std::uint32_t seed = redLow ? 0x2545f491u : 0x9e3779b9u; + for (std::uint32_t y = 0; y < height; ++y) { + for (std::uint32_t x = 0; x < width; ++x) { + std::array halves{}; + for (auto& half : halves) { + seed = seed * 1664525u + 1013904223u; + half = static_cast(seed >> 16u); + } + const auto index = static_cast(y) * width + x; + std::memcpy(texels.data() + index * 8, halves.data(), sizeof(halves)); + std::memcpy(guest.data() + renderTargetOffset64(layout.BlocksPerRow(), x, y), halves.data(), sizeof(halves)); + auto* pixel = expected.data() + index * 4; + pixel[0] = static_cast(HalfFloatScanoutCode(halves[redLow ? 2 : 0], false)); + pixel[1] = static_cast(HalfFloatScanoutCode(halves[1], false)); + pixel[2] = static_cast(HalfFloatScanoutCode(halves[redLow ? 0 : 2], false)); + pixel[3] = static_cast(HalfFloatScanoutCode(halves[3], true)); + } + } + VkImageCreateInfo imageInfo{VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO}; + imageInfo.imageType = VK_IMAGE_TYPE_2D; + imageInfo.format = VK_FORMAT_R16G16B16A16_SFLOAT; + imageInfo.extent = {width, height, 1}; + imageInfo.mipLevels = 1; + imageInfo.arrayLayers = 1; + imageInfo.samples = VK_SAMPLE_COUNT_1_BIT; + imageInfo.tiling = VK_IMAGE_TILING_OPTIMAL; + imageInfo.usage = VK_IMAGE_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT; + imageInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED; + VkImage image = VK_NULL_HANDLE; + VkDeviceMemory memory = VK_NULL_HANDLE; + Check(context.Function("vkCreateImage")(context.device, &imageInfo, nullptr, &image), "vkCreateImage half-float detile test"); + struct Release { + const Context& context; + VkImage image; + VkDeviceMemory& memory; + ~Release() { + context.Function("vkDestroyImage")(context.device, image, nullptr); + if (memory) context.Function("vkFreeMemory")(context.device, memory, nullptr); + } + } release{context, image, memory}; + VkMemoryRequirements requirements{}; + context.Function("vkGetImageMemoryRequirements")(context.device, image, &requirements); + VkMemoryAllocateInfo allocation{VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO}; + allocation.allocationSize = requirements.size; + allocation.memoryTypeIndex = deviceLocalType(context, requirements.memoryTypeBits); + Check(context.Function("vkAllocateMemory")(context.device, &allocation, nullptr, &memory), "vkAllocateMemory half-float detile test"); + Check(context.Function("vkBindImageMemory")(context.device, image, memory, 0), "vkBindImageMemory half-float detile test"); + Buffer upload(context, texels.size(), VK_BUFFER_USAGE_TRANSFER_SRC_BIT); + std::memcpy(upload.Bytes().data(), texels.data(), texels.size()); + Buffer fromGuest(context, expected.size(), VK_BUFFER_USAGE_TRANSFER_DST_BIT); + Buffer fromImage(context, expected.size(), VK_BUFFER_USAGE_TRANSFER_DST_BIT); + GpuColorTransfer transfer(context); + const auto barrier = context.Function("vkCmdPipelineBarrier"); + const auto readback = [&](VkCommandBuffer commands, Buffer& destination) { + const VkBufferCopy copy{0, 0, expected.size()}; + context.Function("vkCmdCopyBuffer")(commands, transfer.LinearBuffer(), destination.Handle(), 1, ©); + RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT | VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT | VK_PIPELINE_STAGE_HOST_BIT, VK_ACCESS_TRANSFER_READ_BIT | VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_TRANSFER_WRITE_BIT | VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_HOST_READ_BIT); + }; + transfer.Upload(reinterpret_cast(aligned), width, height, ColorTileMode::RenderTarget, 8); + { + CommandBatch batch(context); + const auto commands = batch.Handle(); + VkImageMemoryBarrier toTransfer{VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER}; + toTransfer.dstAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT; + toTransfer.oldLayout = VK_IMAGE_LAYOUT_UNDEFINED; + toTransfer.newLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL; + toTransfer.srcQueueFamilyIndex = toTransfer.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED; + toTransfer.image = image; + toTransfer.subresourceRange = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1}; + barrier(commands, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, 0, nullptr, 0, nullptr, 1, &toTransfer); + VkBufferImageCopy region{}; + region.imageSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 0, 1}; + region.imageExtent = {width, height, 1}; + context.Function("vkCmdCopyBufferToImage")(commands, upload.Handle(), image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, ®ion); + auto toGeneral = toTransfer; + toGeneral.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT; + toGeneral.dstAccessMask = VK_ACCESS_TRANSFER_READ_BIT; + toGeneral.oldLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL; + toGeneral.newLayout = VK_IMAGE_LAYOUT_GENERAL; + barrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, 0, nullptr, 0, nullptr, 1, &toGeneral); + transfer.Detile(commands, redLow, false, true); + readback(commands, fromGuest); + transfer.DetileImage(commands, image, VK_IMAGE_LAYOUT_GENERAL, width, height, ColorTileMode::RenderTarget, redLow, false, true); + readback(commands, fromImage); + batch.SubmitAndWait(); + } + Require(std::equal(expected.begin(), expected.end(), fromGuest.Bytes().begin()), "GPU 16-bit float detile from guest memory differs from the reference"); + Require(std::equal(expected.begin(), expected.end(), fromImage.Bytes().begin()), "GPU 16-bit float detile from an image differs from the reference"); +} + void checkBufferReuse(const Context& context) { constexpr auto usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT; VkBuffer handle = VK_NULL_HANDLE; @@ -115,4 +237,6 @@ void RunColorTransferTests(const AgcDriver::Graphics::Context& context) { checkConversion(context, 130, 129, AgcDriver::Graphics::ColorTileMode::RenderTarget, false); checkConversion(context, 257, 17, AgcDriver::Graphics::ColorTileMode::RenderTarget, true); checkConversion(context, 192, 13, AgcDriver::Graphics::ColorTileMode::Linear, false); + checkHalfFloatDetile(context, 300, 130, true); + checkHalfFloatDetile(context, 129, 70, false); } diff --git a/core/libs/prx/libSceAgcDriver/tests/ResidentPresent.cpp b/core/libs/prx/libSceAgcDriver/tests/ResidentPresent.cpp index b535571d2..9efc3e926 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ResidentPresent.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ResidentPresent.cpp @@ -24,6 +24,7 @@ using AgcDriver::ResidentPresent; constexpr std::uint64_t Bgra8 = 0x8000000000000000ull; constexpr std::uint64_t Rgba8 = 0x8000000022000000ull; constexpr std::uint64_t TenBit = 0x0100000000000000ull; +constexpr std::uint64_t Rgba16Float = 0xc001000622000000ull; void decisionTests() { using AgcDriver::DisplayTexelFormat; @@ -55,19 +56,26 @@ std::uint32_t memoryType(const Context& context, std::uint32_t bits, VkMemoryPro void conversionTest(const Context& context, std::uint64_t pixelFormat, VkFormat format) { constexpr std::uint32_t width = 200; constexpr std::uint32_t height = 130; - std::vector words(static_cast(width) * height * 4); + const auto texelBytes = AgcDriver::DisplayTexelBytes(pixelFormat); + std::vector words(static_cast(width) * height * texelBytes); std::uint32_t seed = 0x9e3779b9u ^ static_cast(format); for (std::size_t i = 0; i < words.size(); ++i) { seed = seed * 1664525u + 1013904223u; words[i] = static_cast(seed >> 24u); } - const ColorTargetLayout layout(width, height, ColorTileMode::RenderTarget); + const ColorTargetLayout layout(width, height, ColorTileMode::RenderTarget, texelBytes); std::vector storage(layout.Bytes() + 65536); auto* aligned = reinterpret_cast((reinterpret_cast(storage.data()) + 65535u) & ~std::uintptr_t{65535}); std::span guest(aligned, layout.Bytes()); - layout.Tile(words, guest); const DisplayBuffer display{reinterpret_cast(aligned), pixelFormat, width, height}; Require(AgcDriver::DisplayBufferSize(display) == guest.size(), "test display buffer size differs from the color layout"); + if (texelBytes == 4u) { + layout.Tile(words, guest); + } else { + for (std::uint32_t y = 0; y < height; ++y) { + for (std::uint32_t x = 0; x < width; ++x) std::memcpy(guest.data() + AgcDriver::DisplayBufferOffset(display, x, y), words.data() + (static_cast(y) * width + x) * texelBytes, texelBytes); + } + } const auto expected = AgcDriver::DecodeDisplayBuffer(display, guest); VkImageCreateInfo imageInfo{VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO}; @@ -107,13 +115,14 @@ void conversionTest(const Context& context, std::uint64_t pixelFormat, VkFormat GpuColorTransfer transfer(context); const bool redLow = AgcDriver::DisplayRedLow(pixelFormat); const bool tenBit = AgcDriver::DisplayTenBit(pixelFormat); + const bool halfFloat = AgcDriver::DisplayHalfFloat(pixelFormat); const auto barrier = context.Function("vkCmdPipelineBarrier"); const auto readback = [&](VkCommandBuffer commands, Buffer& destination) { const VkBufferCopy copy{0, 0, expected.size()}; context.Function("vkCmdCopyBuffer")(commands, transfer.LinearBuffer(), destination.Handle(), 1, ©); RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT | VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT | VK_PIPELINE_STAGE_HOST_BIT, VK_ACCESS_TRANSFER_READ_BIT | VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_TRANSFER_WRITE_BIT | VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_HOST_READ_BIT); }; - transfer.Upload(display.address, width, height, ColorTileMode::RenderTarget); + transfer.Upload(display.address, width, height, ColorTileMode::RenderTarget, texelBytes); { CommandBatch batch(context); const auto commands = batch.Handle(); @@ -135,13 +144,13 @@ void conversionTest(const Context& context, std::uint64_t pixelFormat, VkFormat toGeneral.oldLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL; toGeneral.newLayout = VK_IMAGE_LAYOUT_GENERAL; barrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, 0, nullptr, 0, nullptr, 1, &toGeneral); - transfer.Detile(commands, redLow, tenBit); + transfer.Detile(commands, redLow, tenBit, halfFloat); readback(commands, fromGuest); - transfer.DetileImage(commands, image, VK_IMAGE_LAYOUT_GENERAL, width, height, ColorTileMode::RenderTarget, redLow, tenBit); + transfer.DetileImage(commands, image, VK_IMAGE_LAYOUT_GENERAL, width, height, ColorTileMode::RenderTarget, redLow, tenBit, halfFloat); readback(commands, fromImage); batch.SubmitAndWait(); } - const auto name = std::string(tenBit ? "10-bit " : "8-bit ") + (redLow ? "red-low" : "blue-low") + " display from VkFormat " + std::to_string(static_cast(format)); + const auto name = std::string(halfFloat ? "16-bit float " : tenBit ? "10-bit " : "8-bit ") + (redLow ? "red-low" : "blue-low") + " display from VkFormat " + std::to_string(static_cast(format)); Require(std::equal(expected.begin(), expected.end(), fromGuest.Bytes().begin()), "the GPU detile differs from the CPU decode: " + name); Require(std::equal(expected.begin(), expected.end(), fromImage.Bytes().begin()), "the resident image presents other pixels than guest memory: " + name); } @@ -155,5 +164,6 @@ void RunResidentPresentTests(const Context& context) { conversionTest(context, Bgra8 | TenBit, VK_FORMAT_A2R10G10B10_UNORM_PACK32); conversionTest(context, Bgra8, VK_FORMAT_R8G8B8A8_UNORM); conversionTest(context, Rgba8, VK_FORMAT_B8G8R8A8_UNORM); + conversionTest(context, Rgba16Float, VK_FORMAT_R16G16B16A16_SFLOAT); std::cout << "resident present format and conversion tests passed\n"; } diff --git a/core/libs/tests/LinearDisplayBuffer.cpp b/core/libs/tests/LinearDisplayBuffer.cpp index ad10b3f42..d34623bd0 100644 --- a/core/libs/tests/LinearDisplayBuffer.cpp +++ b/core/libs/tests/LinearDisplayBuffer.cpp @@ -1,7 +1,15 @@ #include "prx/libSceAgcDriver/Execution/include/DisplayBuffer.hpp" +#include "prx/libSceAgcDriver/Execution/include/DisplayFormat.hpp" +#include "prx/libSceAgcDriver/Graphics/include/HalfFloatScanout.hpp" +#include "prx/libSceAgcDriver/Graphics/include/TextureSwizzleEquations.hpp" +#include #include +#include +#include +#include #include #include +#include static void Require(bool condition) { if (!condition) std::abort(); } template static void Reject(TAction action) { @@ -10,6 +18,74 @@ template static void Reject(TAction action) { Require(rejected); } +static std::size_t EquationOffset(std::uint32_t elementBytes, std::uint32_t blocksPerRow, std::uint32_t blockHeight, std::uint32_t x, std::uint32_t y) { + const auto* equation = AgcDriver::Graphics::FindTextureSwizzleEquation(27u, elementBytes); + Require(equation != nullptr); + std::size_t offset = 0; + for (std::uint32_t bit = 0; bit < 16u; ++bit) { + const auto mask = equation->bits[bit]; + offset |= static_cast(std::popcount((x & mask & 0xfffu) ^ ((y << 12u) & mask & 0xfff000u)) & 1) << bit; + } + const auto blockWidth = 65536u / elementBytes / blockHeight; + return (static_cast(y / blockHeight) * blocksPerRow + x / blockWidth) * 65536u + offset; +} + +static void HalfFloatCodes() { + for (std::uint32_t bits = 0; bits <= 0xffffu; ++bits) { + const auto exponent = static_cast((bits >> 10u) & 0x1fu); + const auto mantissa = static_cast(bits & 0x3ffu); + const double magnitude = exponent == 0 ? std::ldexp(mantissa, -24) : exponent == 31 ? (mantissa == 0 ? INFINITY : NAN) : std::ldexp(1024.0 + mantissa, exponent - 25); + const double value = (bits & 0x8000u) != 0u ? -magnitude : magnitude; + unsigned color = 0; + unsigned alpha = 0; + if (value > 0) { + const double clamped = std::min(value, 1.0); + const double encoded = clamped <= 0.0031308 ? 12.92 * clamped : 1.055 * std::pow(clamped, 1.0 / 2.4) - 0.055; + color = static_cast(std::floor(encoded * 255.0 + 0.5)); + alpha = static_cast(std::floor(clamped * 255.0 + 0.5)); + } + Require(AgcDriver::Graphics::HalfFloatScanoutCode(static_cast(bits), false) == color); + Require(AgcDriver::Graphics::HalfFloatScanoutCode(static_cast(bits), true) == alpha); + } +} + +static void HalfFloatDisplay() { + constexpr std::uint64_t rgba16Float = 0xc001000622000000ull; + const std::array halves{0x3c00, 0x3800, 0x0000, 0x3c00, 0xbc00, 0x4000, 0x7e00, 0x3800, 0x1234, 0x1234, 0x1234, 0x1234, + 0x7c00, 0x3400, 0x1419, 0x0000, 0x0001, 0x3555, 0x3c00, 0x3c00, 0x1234, 0x1234, 0x1234, 0x1234}; + std::array input{}; + std::memcpy(input.data(), halves.data(), input.size()); + AgcDriver::DisplayBuffer buffer{65536, rgba16Float, 2, 2, 1, 3}; + Require(AgcDriver::DisplayBufferSize(buffer) == input.size()); + const auto pixels = AgcDriver::DecodeDisplayBuffer(buffer, input); + const std::array expected{0, 188, 255, 255, 0, 255, 0, 128, 3, 137, 255, 0, 255, 156, 0, 255}; + Require(pixels.size() == expected.size()); + for (unsigned i = 0; i < pixels.size(); ++i) Require(std::to_integer(pixels[i]) == expected[i]); + Require(AgcDriver::DisplayBufferOffset(buffer, 1, 1) == 32); + buffer.pixelFormat = 0xc001000600000000ull; + const auto swapped = AgcDriver::DecodeDisplayBuffer(buffer, input); + const std::array blueFirst{255, 188, 0, 255, 0, 255, 0, 128, 255, 137, 3, 0, 0, 156, 255, 255}; + for (unsigned i = 0; i < swapped.size(); ++i) Require(std::to_integer(swapped[i]) == blueFirst[i]); + Require(AgcDriver::DisplayHalfFloat(buffer.pixelFormat) && !AgcDriver::DisplayRedLow(buffer.pixelFormat) && AgcDriver::DisplayTexelFormat(buffer.pixelFormat) == VK_FORMAT_R16G16B16A16_SFLOAT); + Require(AgcDriver::DisplayTexelFormat(rgba16Float) == VK_FORMAT_R16G16B16A16_SFLOAT && AgcDriver::DisplayTexelBytes(rgba16Float) == 8 && AgcDriver::DisplayRedLow(rgba16Float) && !AgcDriver::DisplayTenBit(rgba16Float)); + Require(AgcDriver::ResidentPresentPath(VK_FORMAT_R16G16B16A16_SFLOAT, rgba16Float, true) == AgcDriver::ResidentPresent::Convert); + Require(AgcDriver::ResidentPresentPath(VK_FORMAT_R8G8B8A8_UNORM, rgba16Float, true) == AgcDriver::ResidentPresent::None); + Require(AgcDriver::ResidentPresentPath(VK_FORMAT_R16G16B16A16_SFLOAT, 0x8000000022000000ull, true) == AgcDriver::ResidentPresent::None); + buffer = {65536, rgba16Float, 3840, 2160}; + Require(AgcDriver::DisplayBufferSize(buffer) == 30u * 34u * 65536u); + buffer.dccAddress = 0x20000; + Reject([&] { AgcDriver::DisplayBufferSize(buffer); }); + buffer.dccAddress = 0; + buffer.pixelFormat = 0xc001000700000000ull; + Reject([&] { AgcDriver::DisplayBufferSize(buffer); }); + for (const auto [pixelFormat, elementBytes, blockHeight] : {std::tuple{rgba16Float, 8u, 64u}, std::tuple{std::uint64_t{0x8000000000000000ull}, 4u, 128u}}) { + buffer = {65536, pixelFormat, 300, 200}; + for (std::uint32_t y = 0; y < buffer.height; ++y) { + for (std::uint32_t x = 0; x < buffer.width; ++x) Require(AgcDriver::DisplayBufferOffset(buffer, x, y) == EquationOffset(elementBytes, 3, blockHeight, x, y)); + } + } +} + int main() { AgcDriver::DisplayBuffer buffer{65536, 0x8000000000000000ull, 2, 2, 1, 3}; const std::array input{1,2,3,4, 5,6,7,8, 99,99,99,99, @@ -32,4 +108,6 @@ int main() { Reject([&] { AgcDriver::DisplayBufferSize(buffer); }); buffer.tilingMode = 0; Require(AgcDriver::DisplayBufferSize(buffer) == 65536); + HalfFloatCodes(); + HalfFloatDisplay(); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index fab6c9813..1cf59f727 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -421,6 +421,7 @@ Throughout the project, every function at every stage either **does exactly what - [Cube texture descriptors](../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestTextureResource.cpp) with equal `BASE_ARRAY` and `DEPTH` expose six faces starting at `BASE_ARRAY`. Measured with `image_load`, `image_sample` and `image_get_resinfo` on RX 6600 (gfx1032), using 64x64 R32_UINT faces and base slices 0, 5 and 6. Other ranges containing fewer than six faces or incomplete cube arrays still throw: hardware load bounds and filtered-sample addressing differ. Out-of-range cube-array sampling remains unimplemented. Samplers with `DISABLE_CUBE_WRAP` are created with `VK_SAMPLER_CREATE_NON_SEAMLESS_CUBE_MAP_BIT_EXT`, the flag RADV maps to that bit (Mesa `src/amd/vulkan/radv_sampler.c`, `src/amd/common/ac_descriptors.c`); without `VK_EXT_non_seamless_cube_map` a cube texture sampled through such a sampler throws. PS5 equivalence has not been measured on console hardware. - [DCC display buffers](../../core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp) are presented only when all keys of the console's DCC extent agree on uncompressed or a fast-clear code, padding keys included; other keys, and register-clear keys over a pending image, throw. - [Storage image write-back](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) stores the GPU results over a CPU-stamped range whose guest bytes equal the copy taken when the image's layers got their generation (images up to 1 MiB whose guest bytes were settled at that point, with no unit shadow, pending image results or recorded GPU store over them; other images keep the previous rule: the CPU's bytes win). A CPU store that rewrites the baseline value after the GPU changed it is not told apart from no store, and the GPU result wins there. The 1 MiB limit is a choice, not a console value. +- [16-bit float display buffers](../../core/libs/prx/libSceAgcDriver/Execution/src/DisplayBuffer.cpp) (libSceAgcDriver, libSceVideoOut) - pixel formats `0xC001000622000000` and `0xC001000600000000` are taken as R16G16B16A16 and B16G16R16A16 float, eight bytes per texel, as [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/59183dbfa81ed7efdff6069177fed50de0552725/src/graphics/host_gpu/renderer/image/imageInfo.h) maps them and [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/5dfdb4b0b166e83dc9ae275001afb695f12c7174/src/SharpProspero/Interop/VideoOut/VideoOutTypes.cs) names them (`Rgba16Float`, `Bgra16Float`); PPSA11386 registers 3840x2160 tiled buffers of the first. The tiled layout is assumed to be the 8-byte SW_64KB_R_X equation of `TextureSwizzleEquations.hpp` applied to full coordinates, as addrlib applies it, by analogy with the 32-bit scanout; `ColorTargetLayout` applies the equations to in-block coordinates, which for 8-byte texels differs in every odd 64-row block row (address bit 11 takes y bit 6) and for 16-byte texels also differs, so a float display buffer written through `WriteColorTarget` and presented from guest memory would show those rows misplaced. The values are assumed to be linear light and are encoded with the sRGB curve for the 8-bit output. Bits 32-39 of these formats are `0x06`, as in the 10-bit formats SharpProspero lists without sRGB transfer, while its sRGB formats have `0x00`, its 10-bit PQ formats `0x04` (with `0x07` in bits 40-47) and its float BT.2100 PQ formats `0x07`: they show only that these formats are not the sRGB ones. NaN and negative values give 0, values of 1 and above give 255, alpha is linear. Neither the layout nor the transfer is verified against console or title frames, and HDR output is not modelled. DCC metadata on a float buffer throws. - [DCC key stores](../../core/libs/prx/libSceAgcDriver/Graphics/src/DccMetadata.cpp) cover the console's DCC extent only for pipe-aligned DCC of single-level, single-layer 4-byte SW_64KB_R_X surfaces; other surfaces, and every texture and color target key read, cover one byte per 256 bytes. - [DCC key fills over other images](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) - a uniform fill at a live image's `dccAddress`, at least one key per 256 bytes long, is a key fill of that image even when other live images overlap it, but only while it is no longer than the image's key extent (`DccKeyCount`, the pipe-aligned count); a longer one is classified by its overlaps as before. The bound is inferred, not measured on a title: a longer fill also writes the bytes past the keys, which the overlapping image may own. Whether the title's bloom-target key fills stay within the extent was not checked (`APS5_TRACE_DCC_KEYS` prints the length of the fills classified as keys, not of the refused ones). - [8_SRGB color targets](../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver) on a device that cannot attach and blend R8_SRGB render into an RGBA8_SRGB copy of the target, so the sRGB encode, decode and blend rounding are the device's; the console's are not measured. From 2cd63a7ec21a3efffbb6183010f7c8beb74e83b5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 00:38:22 +0200 Subject: [PATCH 203/460] test(libSceNgs2.native): pass a non-null data pointer for dataless blocks AddWaveformBlocks rejects a null data pointer before reaching the silence path, so TestSilenceBlock aborted and its dataless rejection passed for the wrong reason. --- core/libs/prx/libSceNgs2.native/tests/Ngs2Stream.cpp | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceNgs2.native/tests/Ngs2Stream.cpp b/core/libs/prx/libSceNgs2.native/tests/Ngs2Stream.cpp index 94994dcbe..af1e0b06e 100644 --- a/core/libs/prx/libSceNgs2.native/tests/Ngs2Stream.cpp +++ b/core/libs/prx/libSceNgs2.native/tests/Ngs2Stream.cpp @@ -128,13 +128,15 @@ static void TestStreamRejections() { Require(sceNgs2SystemDestroy(system, nullptr) == SCE_NGS2_OK); } +static const std::int16_t Unused = 0; + static void TestSilenceBlock() { const auto system = CreateSystem(); const auto master = Mastering(system, 1); const auto rack = CreateRack(system, SCE_NGS2_RACK_ID_SAMPLER); const auto voice = StreamVoice(rack, 0, master); - AddBlock(voice, nullptr, 0, Grain, SCE_NGS2_WAVEFORM_BLOCKS_FLAG_SILENCE); + AddBlock(voice, &Unused, 0, Grain, SCE_NGS2_WAVEFORM_BLOCKS_FLAG_SILENCE); Event(voice, SCE_NGS2_VOICE_EVENT_PLAY); auto out = RenderI16(system); for (std::uint32_t i = 0; i < Grain; i++) Require(out[i] == 0); @@ -152,7 +154,7 @@ static void TestSilenceBlock() { Require(Flags(withData) == 0); const auto dataless = StreamVoice(rack, 2, master); - Require(Throws([&] { AddBlock(dataless, nullptr, 0, Grain, 0); })); + Require(Throws([&] { AddBlock(dataless, &Unused, 0, Grain, 0); })); Require(sceNgs2SystemDestroy(system, nullptr) == SCE_NGS2_OK); } From 7ea5898f106e8d020c8af89e8a4ce7a49a71ec10 Mon Sep 17 00:00:00 2001 From: Chad Date: Sat, 10 Oct 2026 15:04:10 -0500 Subject: [PATCH 204/460] fix(agc): ignore the count passed to the indirect register GetSize functions --- core/libs/prx/libSceAgc/DcbState/src/Registers.cpp | 6 +++--- core/libs/prx/libSceAgc/tests/RegisterCommands.cpp | 4 ++-- docs/dev/TechnicalDebt.md | 1 + 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/core/libs/prx/libSceAgc/DcbState/src/Registers.cpp b/core/libs/prx/libSceAgc/DcbState/src/Registers.cpp index f9b0b4c99..0949b2c36 100644 --- a/core/libs/prx/libSceAgc/DcbState/src/Registers.cpp +++ b/core/libs/prx/libSceAgc/DcbState/src/Registers.cpp @@ -29,7 +29,7 @@ std::uint32_t* APS5_VABI sceAgcDcbSetCxRegistersIndirect(CommandBuffer* buf, con } std::uint32_t APS5_VABI sceAgcDcbSetCxRegistersIndirectGetSize(std::uint32_t numRegs) { - Agc::Command::CheckBits(numRegs, 0x3fffu, __func__); + static_cast(numRegs); return 20; } @@ -46,7 +46,7 @@ std::uint32_t* APS5_VABI sceAgcDcbSetShRegistersIndirect(CommandBuffer* buf, con } std::uint32_t APS5_VABI sceAgcDcbSetShRegistersIndirectGetSize(std::uint32_t numRegs) { - Agc::Command::CheckBits(numRegs, 0x3fffu, __func__); + static_cast(numRegs); return 20; } @@ -63,7 +63,7 @@ std::uint32_t* APS5_VABI sceAgcDcbSetUcRegistersIndirect(CommandBuffer* buf, con } std::uint32_t APS5_VABI sceAgcDcbSetUcRegistersIndirectGetSize(std::uint32_t numRegs) { - Agc::Command::CheckBits(numRegs, 0x3fffu, __func__); + static_cast(numRegs); return 20; } diff --git a/core/libs/prx/libSceAgc/tests/RegisterCommands.cpp b/core/libs/prx/libSceAgc/tests/RegisterCommands.cpp index 65343c02d..a4ac99250 100644 --- a/core/libs/prx/libSceAgc/tests/RegisterCommands.cpp +++ b/core/libs/prx/libSceAgc/tests/RegisterCommands.cpp @@ -116,8 +116,8 @@ void testIndirect() { check(storage.words == expected, "count setter modified unrelated packet data"); } const auto before = storage.words; - for (const auto count : {0x4000u, 0xffffffffu}) { - expectFailure([&] { sizes[i](count); }); + for (const auto count : {0x4000u, 0xffffffffu, 0xf5f5039cu}) { + check(sizes[i](count) == 20, "indirect size depends on the count"); expectFailure([&] { setters[i](packet, count); }); } expectFailure([&] { setters[i](nullptr, 0); }); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 2bdba83cb..c3fec1bd8 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -127,6 +127,7 @@ Throughout the project, every function at every stage either **does exactly what - [zARR5aCmkoY](../../core/libs/prx/libSceAgc/DcbFlow/src/Control.cpp) (libSceAgc) - unknown name, signature - [sceAgcDcbContextStateOpGetSize](../../core/libs/prx/libSceAgc/DcbState/src/ContextState.cpp) (libSceAgc) - sizes from KytyPS5 only (5, 27, 27 and 32 dwords), equal to what `sceAgcDcbContextStateOp_0100` writes; KytyPS5 returns 0 for an operation above 3, here it throws - [sceAgcBranchPatchSetThenTarget_0300, sceAgcBranchPatchSetElseTarget_0300](../../core/libs/prx/libSceAgc/Patch/src/Branch.cpp) (libSceAgc) - names from their NIDs (`7Wa3aeJgeVU`, `rP5xLdOf26k`); the `(packet, cachePolicy, target, sizeInDwords)` signature is inferred from PPSA28416's calls (a branch packet, 0, an aligned target, 4), with the policy stored where `sceAgcCbBranch` puts it +- [sceAgcDcbSet{Cx,Sh,Uc}RegistersIndirectGetSize](../../core/libs/prx/libSceAgc/DcbState/src/Registers.cpp) (libSceAgc) - the count argument is ignored: PPSA28416 calls them with leftover register contents (0xf5f5039c), so the real functions likely take none; the size is 20 bytes either way - [sceAgcUpdateInterpolantMapping_0100](../../core/libs/prx/libSceAgc/Misc/src/ShaderFusion.cpp) (libSceAgc) - name from its NID (`k0E7vkgqAuE`); the `(regs, vs, ps)` arguments are the declared ones, behaviour unknown - [sceAgcWriteDataPatchSetDst, sceAgcWriteDataPatchSetCachePolicy and their Async forms](../../core/libs/prx/libSceAgc/Patch/src/WaitRegMem.cpp) (libSceAgc) - the signatures `(packet, uint8 value)` are inferred from the other patch functions, as no public declaration was found; the value is taken in the encoding of the `dst` and `cachePolicy` arguments of `sceAgcDcbWriteData` (graphics) and `sceAgcAcbWriteData` (Async, compute), and the patched packet is the one those calls would write. `sceAgcAsyncWriteDataPatchSetAddressOrOffset` is assumed to patch the address like `sceAgcWriteDataPatchSetAddressOrOffset`, as the compute WRITE_DATA packet keeps it in the same dwords - [dbOlWdppb4o](../../core/libs/prx/libSceAgc/Shader/src/InterpolantMapping.cpp) (libSceAgc) - unknown name; same arguments as `sceAgcCreateInterpolantMapping_0100`; the `SPI_PS_INPUT_CNTL` bits for `is_f16` 2 inputs follow KytyPS5, not confirmed on hardware From 9277d067059fe04971a4ac55caafaf4bcc852b7f Mon Sep 17 00:00:00 2001 From: David Date: Sat, 10 Oct 2026 23:24:29 +0200 Subject: [PATCH 206/460] feat(libc): implement the Dinkumware mutex and condition exports Co-Authored-By: Claude Fable 5.1 --- core/libs/prx/libc/CMakeLists.txt | 11 ++ core/libs/prx/libc/Export.cpp | 32 ------ core/libs/prx/libc/src/Threads.cpp | 158 ++++++++++++++++++++++++++ core/libs/tests/GuestLibcThreads.cpp | 163 +++++++++++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 5 files changed, 333 insertions(+), 32 deletions(-) create mode 100644 core/libs/prx/libc/src/Threads.cpp create mode 100644 core/libs/tests/GuestLibcThreads.cpp diff --git a/core/libs/prx/libc/CMakeLists.txt b/core/libs/prx/libc/CMakeLists.txt index c53a9df99..7e732496a 100644 --- a/core/libs/prx/libc/CMakeLists.txt +++ b/core/libs/prx/libc/CMakeLists.txt @@ -40,6 +40,7 @@ target_sources(libc PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src/Errors.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/Syslog.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/CxxAbiSupport.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/src/Threads.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/exception/Exports.cpp ) @@ -166,6 +167,16 @@ if(BUILD_TESTING) set_tests_properties(guest_std_execute_once PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_libc_threads_tests ${LIBS_INCLUDE_DIR}/tests/GuestLibcThreads.cpp) + target_include_directories(guest_libc_threads_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_libc_threads_tests PRIVATE libc) + configure_windows_unwind(guest_libc_threads_tests) + add_test(NAME guest_libc_threads COMMAND guest_libc_threads_tests) + set_tests_properties(guest_libc_threads PROPERTIES TIMEOUT 20) + if(WIN32) + set_tests_properties(guest_libc_threads PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_fiopen_tests ${LIBS_INCLUDE_DIR}/tests/GuestFiopen.cpp) target_include_directories(guest_fiopen_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_fiopen_tests PRIVATE libc) diff --git a/core/libs/prx/libc/Export.cpp b/core/libs/prx/libc/Export.cpp index e7805c777..cad6c13b0 100644 --- a/core/libs/prx/libc/Export.cpp +++ b/core/libs/prx/libc/Export.cpp @@ -33,15 +33,7 @@ std::uint64_t APS5_VABI libcCyberUnknown02(void) { return 0; } -std::uint64_t APS5_VABI _Mtx_destroy_nid_postfix() { - NotImplemented_nid_no_patch(__func__); - return 0; -} -std::uint64_t APS5_VABI _Cnd_destroy_nid_postfix() { - NotImplemented_nid_no_patch(__func__); - return 0; -} APS5_EXPORT("H+8UBOwfScI", libcCyberUnknown08); std::uint64_t APS5_VABI libcCyberUnknown08(void) { @@ -49,33 +41,9 @@ std::uint64_t APS5_VABI libcCyberUnknown08(void) { return 0; } -std::uint64_t APS5_VABI _Cnd_init_nid_postfix() { - NotImplemented_nid_no_patch(__func__); - return 0; -} -std::uint64_t APS5_VABI _Cnd_broadcast_nid_postfix() { - NotImplemented_nid_no_patch(__func__); - return 0; -} -std::uint64_t APS5_VABI _Mtx_init_nid_postfix() { - NotImplemented_nid_no_patch(__func__); - return 0; -} -std::uint64_t APS5_VABI _Mtx_unlock_nid_postfix() { - NotImplemented_nid_no_patch(__func__); - return 0; -} -std::uint64_t APS5_VABI _Mtx_lock_nid_postfix() { - NotImplemented_nid_no_patch(__func__); - return 0; -} -std::uint64_t APS5_VABI _Cnd_wait_nid_postfix() { - NotImplemented_nid_no_patch(__func__); - return 0; -} } diff --git a/core/libs/prx/libc/src/Threads.cpp b/core/libs/prx/libc/src/Threads.cpp new file mode 100644 index 000000000..e60b0e1a8 --- /dev/null +++ b/core/libs/prx/libc/src/Threads.cpp @@ -0,0 +1,158 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +constexpr int threadSuccess = 0; +constexpr int threadNomem = 1; +constexpr int threadBusy = 3; + +constexpr int mutexPlain = 0x01; +constexpr int mutexTry = 0x02; +constexpr int mutexTimed = 0x04; +constexpr int mutexRecursive = 0x100; + +constexpr std::uint64_t mutexTag = 0x5854'4D5F'4D54'5844ull; +constexpr std::uint64_t conditionTag = 0x5854'4E43'5F44'4E43ull; + +struct DinkumwareMutex { + std::uint64_t tag = mutexTag; + int type = 0; + int count = 0; + std::atomic owner{}; + std::mutex native; +}; + +struct DinkumwareCondition { + std::uint64_t tag = conditionTag; + std::condition_variable native; +}; + +DinkumwareMutex* ResolveMutex(void* handle, const char* function) { + auto* mutex = static_cast(handle); + if (!mutex) + throw std::invalid_argument(std::string(function) + ": null mutex handle"); + if (mutex->tag != mutexTag) + throw std::invalid_argument(std::string(function) + ": not a mutex created by _Mtx_init"); + return mutex; +} + +DinkumwareCondition* ResolveCondition(void* handle, const char* function) { + auto* condition = static_cast(handle); + if (!condition) + throw std::invalid_argument(std::string(function) + ": null condition handle"); + if (condition->tag != conditionTag) + throw std::invalid_argument(std::string(function) + ": not a condition created by _Cnd_init"); + return condition; +} + +bool IsOwnedByCaller(const DinkumwareMutex* mutex) { + return mutex->count > 0 && mutex->owner.load(std::memory_order_acquire) == std::this_thread::get_id(); +} + +} + +extern "C" { + +int APS5_VABI _Mtx_init_nid_postfix(void** handle, int type) { + if (!handle) + throw std::invalid_argument("_Mtx_init: null handle pointer"); + if ((type & ~(mutexPlain | mutexTry | mutexTimed | mutexRecursive)) != 0 || (type & (mutexPlain | mutexTry | mutexTimed)) == 0) + throw std::invalid_argument("_Mtx_init: unsupported mutex type " + std::to_string(type)); + *handle = nullptr; + auto* mutex = new (std::nothrow) DinkumwareMutex(); + if (!mutex) + return threadNomem; + mutex->type = type; + *handle = mutex; + return threadSuccess; +} + +void APS5_VABI _Mtx_destroy_nid_postfix(void* handle) { + if (!handle) + return; + auto* mutex = ResolveMutex(handle, "_Mtx_destroy"); + if (mutex->count != 0) + throw std::logic_error("_Mtx_destroy: mutex destroyed while locked"); + mutex->tag = 0; + delete mutex; +} + +int APS5_VABI _Mtx_lock_nid_postfix(void* handle) { + auto* mutex = ResolveMutex(handle, "_Mtx_lock"); + const auto self = std::this_thread::get_id(); + if (mutex->owner.load(std::memory_order_acquire) == self) { + if ((mutex->type & ~mutexRecursive) != mutexPlain && (mutex->type & mutexRecursive) == 0) + return threadBusy; + if (mutex->count == std::numeric_limits::max()) + throw std::overflow_error("_Mtx_lock: recursive lock count overflow"); + ++mutex->count; + return threadSuccess; + } + mutex->native.lock(); + mutex->owner.store(self, std::memory_order_release); + mutex->count = 1; + return threadSuccess; +} + +int APS5_VABI _Mtx_unlock_nid_postfix(void* handle) { + auto* mutex = ResolveMutex(handle, "_Mtx_unlock"); + if (!IsOwnedByCaller(mutex)) + throw std::logic_error("_Mtx_unlock: mutex is not owned by the calling thread"); + if (--mutex->count == 0) { + mutex->owner.store(std::thread::id{}, std::memory_order_release); + mutex->native.unlock(); + } + return threadSuccess; +} + +int APS5_VABI _Cnd_init_nid_postfix(void** handle) { + if (!handle) + throw std::invalid_argument("_Cnd_init: null handle pointer"); + *handle = nullptr; + auto* condition = new (std::nothrow) DinkumwareCondition(); + if (!condition) + return threadNomem; + *handle = condition; + return threadSuccess; +} + +void APS5_VABI _Cnd_destroy_nid_postfix(void* handle) { + if (!handle) + return; + auto* condition = ResolveCondition(handle, "_Cnd_destroy"); + condition->tag = 0; + delete condition; +} + +int APS5_VABI _Cnd_wait_nid_postfix(void* conditionHandle, void* mutexHandle) { + auto* condition = ResolveCondition(conditionHandle, "_Cnd_wait"); + auto* mutex = ResolveMutex(mutexHandle, "_Cnd_wait"); + if (!IsOwnedByCaller(mutex)) + throw std::logic_error("_Cnd_wait: mutex is not owned by the calling thread"); + if (mutex->count != 1) + throw std::logic_error("_Cnd_wait: recursive mutex is locked more than once"); + mutex->count = 0; + mutex->owner.store(std::thread::id{}, std::memory_order_release); + std::unique_lock lock(mutex->native, std::adopt_lock); + condition->native.wait(lock); + lock.release(); + mutex->owner.store(std::this_thread::get_id(), std::memory_order_release); + mutex->count = 1; + return threadSuccess; +} + +int APS5_VABI _Cnd_broadcast_nid_postfix(void* handle) { + ResolveCondition(handle, "_Cnd_broadcast")->native.notify_all(); + return threadSuccess; +} + +} diff --git a/core/libs/tests/GuestLibcThreads.cpp b/core/libs/tests/GuestLibcThreads.cpp new file mode 100644 index 000000000..85b4c7185 --- /dev/null +++ b/core/libs/tests/GuestLibcThreads.cpp @@ -0,0 +1,163 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include +#include + +extern "C" { +int APS5_VABI _Mtx_init_nid_postfix(void** handle, int type); +void APS5_VABI _Mtx_destroy_nid_postfix(void* handle); +int APS5_VABI _Mtx_lock_nid_postfix(void* handle); +int APS5_VABI _Mtx_unlock_nid_postfix(void* handle); +int APS5_VABI _Cnd_init_nid_postfix(void** handle); +void APS5_VABI _Cnd_destroy_nid_postfix(void* handle); +int APS5_VABI _Cnd_wait_nid_postfix(void* condition, void* mutex); +int APS5_VABI _Cnd_broadcast_nid_postfix(void* handle); +} + +static void Require(bool value) { if (!value) std::abort(); } + +template +static void RequireThrows(TCall call) { + try { + call(); + } catch (const TException&) { + return; + } + std::abort(); +} + +constexpr int success = 0; +constexpr int busy = 3; +constexpr int plain = 0x01; +constexpr int tryable = 0x02; +constexpr int timed = 0x04; +constexpr int recursive = 0x100; + +static void* CreateMutex(int type) { + void* mutex = reinterpret_cast(std::uintptr_t{1}); + Require(_Mtx_init_nid_postfix(&mutex, type) == success && mutex != nullptr); + return mutex; +} + +static void MutexBasics() { + void* mutex = CreateMutex(tryable); + Require(_Mtx_lock_nid_postfix(mutex) == success); + Require(_Mtx_lock_nid_postfix(mutex) == busy); + RequireThrows([&] { _Mtx_destroy_nid_postfix(mutex); }); + std::thread([&] { RequireThrows([&] { _Mtx_unlock_nid_postfix(mutex); }); }).join(); + Require(_Mtx_unlock_nid_postfix(mutex) == success); + RequireThrows([&] { _Mtx_unlock_nid_postfix(mutex); }); + RequireThrows([&] { _Mtx_lock_nid_postfix(&mutex); }); + _Mtx_destroy_nid_postfix(mutex); + _Mtx_destroy_nid_postfix(nullptr); + + void* plainMutex = CreateMutex(plain); + Require(_Mtx_lock_nid_postfix(plainMutex) == success && _Mtx_lock_nid_postfix(plainMutex) == success); + Require(_Mtx_unlock_nid_postfix(plainMutex) == success && _Mtx_unlock_nid_postfix(plainMutex) == success); + _Mtx_destroy_nid_postfix(plainMutex); + + void* timedMutex = CreateMutex(timed | tryable); + Require(_Mtx_lock_nid_postfix(timedMutex) == success && _Mtx_lock_nid_postfix(timedMutex) == busy); + Require(_Mtx_unlock_nid_postfix(timedMutex) == success); + _Mtx_destroy_nid_postfix(timedMutex); + + void* untouched = reinterpret_cast(std::uintptr_t{1}); + RequireThrows([&] { _Mtx_init_nid_postfix(&untouched, 0); }); + RequireThrows([&] { _Mtx_init_nid_postfix(&untouched, recursive); }); + RequireThrows([&] { _Mtx_init_nid_postfix(&untouched, tryable | 0x8); }); + Require(untouched == reinterpret_cast(std::uintptr_t{1})); + RequireThrows([] { _Mtx_init_nid_postfix(nullptr, tryable); }); + RequireThrows([] { _Mtx_lock_nid_postfix(nullptr); }); + RequireThrows([] { _Mtx_unlock_nid_postfix(nullptr); }); +} + +static void RecursiveMutex() { + void* mutex = CreateMutex(tryable | recursive); + for (int i = 0; i < 3; ++i) Require(_Mtx_lock_nid_postfix(mutex) == success); + std::atomic acquired{false}; + std::thread other([&] { + Require(_Mtx_lock_nid_postfix(mutex) == success); + acquired = true; + Require(_Mtx_unlock_nid_postfix(mutex) == success); + }); + for (int i = 0; i < 2; ++i) Require(_Mtx_unlock_nid_postfix(mutex) == success); + std::this_thread::sleep_for(std::chrono::milliseconds(20)); + Require(!acquired); + Require(_Mtx_unlock_nid_postfix(mutex) == success); + other.join(); + Require(acquired); + _Mtx_destroy_nid_postfix(mutex); +} + +static void MutualExclusion() { + void* mutex = CreateMutex(tryable); + long counter = 0; + std::array workers; + for (auto& worker : workers) worker = std::thread([&] { + for (int i = 0; i < 20000; ++i) { + Require(_Mtx_lock_nid_postfix(mutex) == success); + ++counter; + Require(_Mtx_unlock_nid_postfix(mutex) == success); + } + }); + for (auto& worker : workers) worker.join(); + Require(counter == 80000); + _Mtx_destroy_nid_postfix(mutex); +} + +static void ConditionBroadcast() { + void* condition = reinterpret_cast(std::uintptr_t{1}); + Require(_Cnd_init_nid_postfix(&condition) == success && condition != nullptr); + void* mutex = CreateMutex(tryable); + bool ready = false; + int waiting = 0, woken = 0; + std::array waiters; + for (auto& waiter : waiters) waiter = std::thread([&] { + Require(_Mtx_lock_nid_postfix(mutex) == success); + ++waiting; + while (!ready) Require(_Cnd_wait_nid_postfix(condition, mutex) == success); + ++woken; + Require(_Mtx_lock_nid_postfix(mutex) == busy); + Require(_Mtx_unlock_nid_postfix(mutex) == success); + }); + for (;;) { + Require(_Mtx_lock_nid_postfix(mutex) == success); + const bool all = waiting == 2; + Require(_Mtx_unlock_nid_postfix(mutex) == success); + if (all) break; + std::this_thread::yield(); + } + Require(_Mtx_lock_nid_postfix(mutex) == success); + ready = true; + Require(_Cnd_broadcast_nid_postfix(condition) == success); + Require(_Mtx_unlock_nid_postfix(mutex) == success); + for (auto& waiter : waiters) waiter.join(); + Require(woken == 2); + Require(_Cnd_broadcast_nid_postfix(condition) == success); + + RequireThrows([&] { _Cnd_wait_nid_postfix(condition, mutex); }); + void* recursiveMutex = CreateMutex(tryable | recursive); + Require(_Mtx_lock_nid_postfix(recursiveMutex) == success && _Mtx_lock_nid_postfix(recursiveMutex) == success); + RequireThrows([&] { _Cnd_wait_nid_postfix(condition, recursiveMutex); }); + Require(_Mtx_unlock_nid_postfix(recursiveMutex) == success && _Mtx_unlock_nid_postfix(recursiveMutex) == success); + _Mtx_destroy_nid_postfix(recursiveMutex); + + RequireThrows([&] { _Cnd_wait_nid_postfix(mutex, mutex); }); + RequireThrows([&] { _Cnd_broadcast_nid_postfix(&condition); }); + RequireThrows([] { _Cnd_broadcast_nid_postfix(nullptr); }); + RequireThrows([] { _Cnd_init_nid_postfix(nullptr); }); + _Cnd_destroy_nid_postfix(condition); + _Cnd_destroy_nid_postfix(nullptr); + _Mtx_destroy_nid_postfix(mutex); +} + +int main() { + MutexBasics(); + RecursiveMutex(); + MutualExclusion(); + ConditionBroadcast(); +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 2bdba83cb..259d197c0 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -159,6 +159,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceImeGetPanelPositionAndForm](../../core/libs/prx/libSceIme/Export.cpp) (libSceIme) - the output is taken to be the `PositionAndForm` that `sceImeDialogGetPanelPositionAndForm` fills; it is never written, since no panel can be open. That a closed panel is reported before a null output is assumed by analogy with `sceImeSetCaret`, not confirmed on a PS5 title - [_sceLibcInternalForceTlsDestructor](../../core/libs/prx/libSceLibcInternal/Export.cpp) (libSceLibcInternal) - behaviour follows `sceLibcForceTlsDestructor(int)` in a title's `libc.prx` (disassembly): the calling thread's destructors whose dso belongs to the module run newest first, only if the dso's module is loaded and the dso points to itself, in up to 4 passes. The return type is unknown (that function returns nothing), 0 is returned. The module of each dso is looked up when the destructors are forced, not when they are registered as `libc.prx` does, and the lookup is Linux only, so on Windows the call throws when the thread has destructors - [std_execute_once](../../core/libs/prx/libc/Export.cpp) (libc) - assumed to be the plain-named twin of `std::_Execute_once` (`_ZSt13_Execute_onceRSt9once_flagPFiPvS1_PS1_ES1_`, the callee of Dinkumware `call_once`): it delegates there and gets the [thread.once.callonce] semantics already verified for that symbol (one invocation per flag, concurrent callers block until it completes, and a failing or throwing invocation leaves the flag unready so the next caller runs it). The name equivalence is not confirmed on a title, and the callback convention (called with (null, arg, null), non-zero return meaning success) is the twin's, unverified for this symbol +- [Dinkumware mutexes and conditions](../../core/libs/prx/libc/src/Threads.cpp) (libc) - `_Mtx_init`, `_Mtx_destroy`, `_Mtx_lock`, `_Mtx_unlock`, `_Cnd_init`, `_Cnd_destroy`, `_Cnd_wait` and `_Cnd_broadcast` follow microsoft/STL [`xthreads.h`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/inc/xthreads.h), [`mutex.cpp`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/src/mutex.cpp) and [`cond.cpp`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/src/cond.cpp), not the console's Dinkumware build. Settled by those files: `_Mtx_init` and `_Cnd_init` store null and then a heap handle through their pointer argument and return `_Thrd_nomem` (1) when allocation fails; destroy, lock, unlock, wait and broadcast take the handle by value; destroying a null handle does nothing; the other functions return `_Thrd_success` (0); an owner that locks a plain mutex (type `_Mtx_plain`, with or without `_Mtx_recursive`) or a recursive mutex again counts the lock, and an owner that locks any other non-recursive mutex gets `_Thrd_busy` (3). The handle points to a host object, so guest code that reads the mutex or condition fields itself instead of calling these functions breaks. Unverified on a console: whether its `_Mtx_lock`, `_Mtx_unlock`, `_Mtx_destroy`, `_Cnd_wait` and `_Cnd_destroy` take the handle by value like microsoft/STL or a pointer to it (a handle not made by `_Mtx_init` or `_Cnd_init` throws, so the pointer form fails loudly), and how the console build treats an owner relocking a plain or `_Mtx_try` mutex. Cases the sources leave to debug asserts or undefined behaviour throw: a null handle pointer for `_Mtx_init` or `_Cnd_init`, a null handle for the other non-destroy functions, a type that is not a combination of `_Mtx_plain`, `_Mtx_try`, `_Mtx_timed` and `_Mtx_recursive` with at least one of the first three, destroying a locked mutex, unlocking or waiting with a mutex the caller does not own, and waiting with a recursive mutex locked more than once (MSVC releases it entirely there while keeping the remaining count) - [tcgetattr, tcsetattr](../../core/libs/prx/libkernel/File/src/Terminal.cpp) (libkernel) - every file and pipe descriptor is taken as not a terminal (`ENOTTY`), and a socket gets `EADDRNOTAVAIL` for `AF_INET` and `EOPNOTSUPP` otherwise, as FreeBSD's `in_control` and `pru_control_notsupp` answer a termios ioctl; what descriptors 0 to 2 of a title are on the console, and whether any of them is a terminal there, is unknown - [__progname](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - unknown data export - [environ](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - data export pointing to a fixed empty environment so titles that import it load; it is not the environment libc's `getenv` reads, so it lists none of those variables and `setenv`/`putenv` changes do not appear in it From 697e3bf797f1ef446add36f1ddf49077451fc46f Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 22:00:01 +0000 Subject: [PATCH 207/460] fix(libc): match console Dinkumware handle ABI --- core/libs/prx/libc/src/Threads.cpp | 47 ++++++++---- core/libs/tests/GuestLibcThreads.cpp | 106 ++++++++++++++------------- docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 91 insertions(+), 64 deletions(-) diff --git a/core/libs/prx/libc/src/Threads.cpp b/core/libs/prx/libc/src/Threads.cpp index e60b0e1a8..6649f931d 100644 --- a/core/libs/prx/libc/src/Threads.cpp +++ b/core/libs/prx/libc/src/Threads.cpp @@ -7,6 +7,7 @@ #include #include #include +#include #include namespace { @@ -14,6 +15,7 @@ namespace { constexpr int threadSuccess = 0; constexpr int threadNomem = 1; constexpr int threadBusy = 3; +constexpr int threadError = 4; constexpr int mutexPlain = 0x01; constexpr int mutexTry = 0x02; @@ -36,8 +38,10 @@ struct DinkumwareCondition { std::condition_variable native; }; -DinkumwareMutex* ResolveMutex(void* handle, const char* function) { - auto* mutex = static_cast(handle); +DinkumwareMutex* ResolveMutex(void** handle, const char* function) { + if (!handle) + throw std::invalid_argument(std::string(function) + ": null mutex handle pointer"); + auto* mutex = static_cast(*handle); if (!mutex) throw std::invalid_argument(std::string(function) + ": null mutex handle"); if (mutex->tag != mutexTag) @@ -45,8 +49,10 @@ DinkumwareMutex* ResolveMutex(void* handle, const char* function) { return mutex; } -DinkumwareCondition* ResolveCondition(void* handle, const char* function) { - auto* condition = static_cast(handle); +DinkumwareCondition* ResolveCondition(void** handle, const char* function) { + if (!handle) + throw std::invalid_argument(std::string(function) + ": null condition handle pointer"); + auto* condition = static_cast(*handle); if (!condition) throw std::invalid_argument(std::string(function) + ": null condition handle"); if (condition->tag != conditionTag) @@ -55,7 +61,7 @@ DinkumwareCondition* ResolveCondition(void* handle, const char* function) { } bool IsOwnedByCaller(const DinkumwareMutex* mutex) { - return mutex->count > 0 && mutex->owner.load(std::memory_order_acquire) == std::this_thread::get_id(); + return mutex->owner.load(std::memory_order_acquire) == std::this_thread::get_id() && mutex->count > 0; } } @@ -76,8 +82,10 @@ int APS5_VABI _Mtx_init_nid_postfix(void** handle, int type) { return threadSuccess; } -void APS5_VABI _Mtx_destroy_nid_postfix(void* handle) { +void APS5_VABI _Mtx_destroy_nid_postfix(void** handle) { if (!handle) + throw std::invalid_argument("_Mtx_destroy: null mutex handle pointer"); + if (!*handle) return; auto* mutex = ResolveMutex(handle, "_Mtx_destroy"); if (mutex->count != 0) @@ -86,7 +94,7 @@ void APS5_VABI _Mtx_destroy_nid_postfix(void* handle) { delete mutex; } -int APS5_VABI _Mtx_lock_nid_postfix(void* handle) { +int APS5_VABI _Mtx_lock_nid_postfix(void** handle) { auto* mutex = ResolveMutex(handle, "_Mtx_lock"); const auto self = std::this_thread::get_id(); if (mutex->owner.load(std::memory_order_acquire) == self) { @@ -97,13 +105,17 @@ int APS5_VABI _Mtx_lock_nid_postfix(void* handle) { ++mutex->count; return threadSuccess; } - mutex->native.lock(); + try { + mutex->native.lock(); + } catch (const std::system_error&) { + return threadError; + } mutex->owner.store(self, std::memory_order_release); mutex->count = 1; return threadSuccess; } -int APS5_VABI _Mtx_unlock_nid_postfix(void* handle) { +int APS5_VABI _Mtx_unlock_nid_postfix(void** handle) { auto* mutex = ResolveMutex(handle, "_Mtx_unlock"); if (!IsOwnedByCaller(mutex)) throw std::logic_error("_Mtx_unlock: mutex is not owned by the calling thread"); @@ -125,15 +137,17 @@ int APS5_VABI _Cnd_init_nid_postfix(void** handle) { return threadSuccess; } -void APS5_VABI _Cnd_destroy_nid_postfix(void* handle) { +void APS5_VABI _Cnd_destroy_nid_postfix(void** handle) { if (!handle) + throw std::invalid_argument("_Cnd_destroy: null condition handle pointer"); + if (!*handle) return; auto* condition = ResolveCondition(handle, "_Cnd_destroy"); condition->tag = 0; delete condition; } -int APS5_VABI _Cnd_wait_nid_postfix(void* conditionHandle, void* mutexHandle) { +int APS5_VABI _Cnd_wait_nid_postfix(void** conditionHandle, void** mutexHandle) { auto* condition = ResolveCondition(conditionHandle, "_Cnd_wait"); auto* mutex = ResolveMutex(mutexHandle, "_Cnd_wait"); if (!IsOwnedByCaller(mutex)) @@ -143,14 +157,21 @@ int APS5_VABI _Cnd_wait_nid_postfix(void* conditionHandle, void* mutexHandle) { mutex->count = 0; mutex->owner.store(std::thread::id{}, std::memory_order_release); std::unique_lock lock(mutex->native, std::adopt_lock); - condition->native.wait(lock); + try { + condition->native.wait(lock); + } catch (const std::system_error&) { + lock.release(); + mutex->owner.store(std::this_thread::get_id(), std::memory_order_release); + mutex->count = 1; + return threadError; + } lock.release(); mutex->owner.store(std::this_thread::get_id(), std::memory_order_release); mutex->count = 1; return threadSuccess; } -int APS5_VABI _Cnd_broadcast_nid_postfix(void* handle) { +int APS5_VABI _Cnd_broadcast_nid_postfix(void** handle) { ResolveCondition(handle, "_Cnd_broadcast")->native.notify_all(); return threadSuccess; } diff --git a/core/libs/tests/GuestLibcThreads.cpp b/core/libs/tests/GuestLibcThreads.cpp index 85b4c7185..1f4438ff1 100644 --- a/core/libs/tests/GuestLibcThreads.cpp +++ b/core/libs/tests/GuestLibcThreads.cpp @@ -9,13 +9,13 @@ extern "C" { int APS5_VABI _Mtx_init_nid_postfix(void** handle, int type); -void APS5_VABI _Mtx_destroy_nid_postfix(void* handle); -int APS5_VABI _Mtx_lock_nid_postfix(void* handle); -int APS5_VABI _Mtx_unlock_nid_postfix(void* handle); +void APS5_VABI _Mtx_destroy_nid_postfix(void** handle); +int APS5_VABI _Mtx_lock_nid_postfix(void** handle); +int APS5_VABI _Mtx_unlock_nid_postfix(void** handle); int APS5_VABI _Cnd_init_nid_postfix(void** handle); -void APS5_VABI _Cnd_destroy_nid_postfix(void* handle); -int APS5_VABI _Cnd_wait_nid_postfix(void* condition, void* mutex); -int APS5_VABI _Cnd_broadcast_nid_postfix(void* handle); +void APS5_VABI _Cnd_destroy_nid_postfix(void** handle); +int APS5_VABI _Cnd_wait_nid_postfix(void** condition, void** mutex); +int APS5_VABI _Cnd_broadcast_nid_postfix(void** handle); } static void Require(bool value) { if (!value) std::abort(); } @@ -45,25 +45,28 @@ static void* CreateMutex(int type) { static void MutexBasics() { void* mutex = CreateMutex(tryable); - Require(_Mtx_lock_nid_postfix(mutex) == success); - Require(_Mtx_lock_nid_postfix(mutex) == busy); - RequireThrows([&] { _Mtx_destroy_nid_postfix(mutex); }); - std::thread([&] { RequireThrows([&] { _Mtx_unlock_nid_postfix(mutex); }); }).join(); - Require(_Mtx_unlock_nid_postfix(mutex) == success); - RequireThrows([&] { _Mtx_unlock_nid_postfix(mutex); }); - RequireThrows([&] { _Mtx_lock_nid_postfix(&mutex); }); - _Mtx_destroy_nid_postfix(mutex); - _Mtx_destroy_nid_postfix(nullptr); + Require(_Mtx_lock_nid_postfix(&mutex) == success); + Require(_Mtx_lock_nid_postfix(&mutex) == busy); + RequireThrows([&] { _Mtx_destroy_nid_postfix(&mutex); }); + std::thread([&] { RequireThrows([&] { _Mtx_unlock_nid_postfix(&mutex); }); }).join(); + Require(_Mtx_unlock_nid_postfix(&mutex) == success); + RequireThrows([&] { _Mtx_unlock_nid_postfix(&mutex); }); + void* invalidMutex = nullptr; + RequireThrows([&] { _Mtx_lock_nid_postfix(&invalidMutex); }); + _Mtx_destroy_nid_postfix(&mutex); + void* emptyMutex = nullptr; + _Mtx_destroy_nid_postfix(&emptyMutex); + RequireThrows([] { _Mtx_destroy_nid_postfix(nullptr); }); void* plainMutex = CreateMutex(plain); - Require(_Mtx_lock_nid_postfix(plainMutex) == success && _Mtx_lock_nid_postfix(plainMutex) == success); - Require(_Mtx_unlock_nid_postfix(plainMutex) == success && _Mtx_unlock_nid_postfix(plainMutex) == success); - _Mtx_destroy_nid_postfix(plainMutex); + Require(_Mtx_lock_nid_postfix(&plainMutex) == success && _Mtx_lock_nid_postfix(&plainMutex) == success); + Require(_Mtx_unlock_nid_postfix(&plainMutex) == success && _Mtx_unlock_nid_postfix(&plainMutex) == success); + _Mtx_destroy_nid_postfix(&plainMutex); void* timedMutex = CreateMutex(timed | tryable); - Require(_Mtx_lock_nid_postfix(timedMutex) == success && _Mtx_lock_nid_postfix(timedMutex) == busy); - Require(_Mtx_unlock_nid_postfix(timedMutex) == success); - _Mtx_destroy_nid_postfix(timedMutex); + Require(_Mtx_lock_nid_postfix(&timedMutex) == success && _Mtx_lock_nid_postfix(&timedMutex) == busy); + Require(_Mtx_unlock_nid_postfix(&timedMutex) == success); + _Mtx_destroy_nid_postfix(&timedMutex); void* untouched = reinterpret_cast(std::uintptr_t{1}); RequireThrows([&] { _Mtx_init_nid_postfix(&untouched, 0); }); @@ -77,20 +80,20 @@ static void MutexBasics() { static void RecursiveMutex() { void* mutex = CreateMutex(tryable | recursive); - for (int i = 0; i < 3; ++i) Require(_Mtx_lock_nid_postfix(mutex) == success); + for (int i = 0; i < 3; ++i) Require(_Mtx_lock_nid_postfix(&mutex) == success); std::atomic acquired{false}; std::thread other([&] { - Require(_Mtx_lock_nid_postfix(mutex) == success); + Require(_Mtx_lock_nid_postfix(&mutex) == success); acquired = true; - Require(_Mtx_unlock_nid_postfix(mutex) == success); + Require(_Mtx_unlock_nid_postfix(&mutex) == success); }); - for (int i = 0; i < 2; ++i) Require(_Mtx_unlock_nid_postfix(mutex) == success); + for (int i = 0; i < 2; ++i) Require(_Mtx_unlock_nid_postfix(&mutex) == success); std::this_thread::sleep_for(std::chrono::milliseconds(20)); Require(!acquired); - Require(_Mtx_unlock_nid_postfix(mutex) == success); + Require(_Mtx_unlock_nid_postfix(&mutex) == success); other.join(); Require(acquired); - _Mtx_destroy_nid_postfix(mutex); + _Mtx_destroy_nid_postfix(&mutex); } static void MutualExclusion() { @@ -99,14 +102,14 @@ static void MutualExclusion() { std::array workers; for (auto& worker : workers) worker = std::thread([&] { for (int i = 0; i < 20000; ++i) { - Require(_Mtx_lock_nid_postfix(mutex) == success); + Require(_Mtx_lock_nid_postfix(&mutex) == success); ++counter; - Require(_Mtx_unlock_nid_postfix(mutex) == success); + Require(_Mtx_unlock_nid_postfix(&mutex) == success); } }); for (auto& worker : workers) worker.join(); Require(counter == 80000); - _Mtx_destroy_nid_postfix(mutex); + _Mtx_destroy_nid_postfix(&mutex); } static void ConditionBroadcast() { @@ -117,42 +120,45 @@ static void ConditionBroadcast() { int waiting = 0, woken = 0; std::array waiters; for (auto& waiter : waiters) waiter = std::thread([&] { - Require(_Mtx_lock_nid_postfix(mutex) == success); + Require(_Mtx_lock_nid_postfix(&mutex) == success); ++waiting; - while (!ready) Require(_Cnd_wait_nid_postfix(condition, mutex) == success); + while (!ready) Require(_Cnd_wait_nid_postfix(&condition, &mutex) == success); ++woken; - Require(_Mtx_lock_nid_postfix(mutex) == busy); - Require(_Mtx_unlock_nid_postfix(mutex) == success); + Require(_Mtx_lock_nid_postfix(&mutex) == busy); + Require(_Mtx_unlock_nid_postfix(&mutex) == success); }); for (;;) { - Require(_Mtx_lock_nid_postfix(mutex) == success); + Require(_Mtx_lock_nid_postfix(&mutex) == success); const bool all = waiting == 2; - Require(_Mtx_unlock_nid_postfix(mutex) == success); + Require(_Mtx_unlock_nid_postfix(&mutex) == success); if (all) break; std::this_thread::yield(); } - Require(_Mtx_lock_nid_postfix(mutex) == success); + Require(_Mtx_lock_nid_postfix(&mutex) == success); ready = true; - Require(_Cnd_broadcast_nid_postfix(condition) == success); - Require(_Mtx_unlock_nid_postfix(mutex) == success); + Require(_Cnd_broadcast_nid_postfix(&condition) == success); + Require(_Mtx_unlock_nid_postfix(&mutex) == success); for (auto& waiter : waiters) waiter.join(); Require(woken == 2); - Require(_Cnd_broadcast_nid_postfix(condition) == success); + Require(_Cnd_broadcast_nid_postfix(&condition) == success); - RequireThrows([&] { _Cnd_wait_nid_postfix(condition, mutex); }); + RequireThrows([&] { _Cnd_wait_nid_postfix(&condition, &mutex); }); void* recursiveMutex = CreateMutex(tryable | recursive); - Require(_Mtx_lock_nid_postfix(recursiveMutex) == success && _Mtx_lock_nid_postfix(recursiveMutex) == success); - RequireThrows([&] { _Cnd_wait_nid_postfix(condition, recursiveMutex); }); - Require(_Mtx_unlock_nid_postfix(recursiveMutex) == success && _Mtx_unlock_nid_postfix(recursiveMutex) == success); - _Mtx_destroy_nid_postfix(recursiveMutex); + Require(_Mtx_lock_nid_postfix(&recursiveMutex) == success && _Mtx_lock_nid_postfix(&recursiveMutex) == success); + RequireThrows([&] { _Cnd_wait_nid_postfix(&condition, &recursiveMutex); }); + Require(_Mtx_unlock_nid_postfix(&recursiveMutex) == success && _Mtx_unlock_nid_postfix(&recursiveMutex) == success); + _Mtx_destroy_nid_postfix(&recursiveMutex); - RequireThrows([&] { _Cnd_wait_nid_postfix(mutex, mutex); }); - RequireThrows([&] { _Cnd_broadcast_nid_postfix(&condition); }); + RequireThrows([&] { _Cnd_wait_nid_postfix(&mutex, &mutex); }); + void* invalidCondition = nullptr; + RequireThrows([&] { _Cnd_broadcast_nid_postfix(&invalidCondition); }); RequireThrows([] { _Cnd_broadcast_nid_postfix(nullptr); }); RequireThrows([] { _Cnd_init_nid_postfix(nullptr); }); - _Cnd_destroy_nid_postfix(condition); - _Cnd_destroy_nid_postfix(nullptr); - _Mtx_destroy_nid_postfix(mutex); + _Cnd_destroy_nid_postfix(&condition); + void* emptyCondition = nullptr; + _Cnd_destroy_nid_postfix(&emptyCondition); + RequireThrows([] { _Cnd_destroy_nid_postfix(nullptr); }); + _Mtx_destroy_nid_postfix(&mutex); } int main() { diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 259d197c0..f55cc7fdd 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -159,7 +159,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceImeGetPanelPositionAndForm](../../core/libs/prx/libSceIme/Export.cpp) (libSceIme) - the output is taken to be the `PositionAndForm` that `sceImeDialogGetPanelPositionAndForm` fills; it is never written, since no panel can be open. That a closed panel is reported before a null output is assumed by analogy with `sceImeSetCaret`, not confirmed on a PS5 title - [_sceLibcInternalForceTlsDestructor](../../core/libs/prx/libSceLibcInternal/Export.cpp) (libSceLibcInternal) - behaviour follows `sceLibcForceTlsDestructor(int)` in a title's `libc.prx` (disassembly): the calling thread's destructors whose dso belongs to the module run newest first, only if the dso's module is loaded and the dso points to itself, in up to 4 passes. The return type is unknown (that function returns nothing), 0 is returned. The module of each dso is looked up when the destructors are forced, not when they are registered as `libc.prx` does, and the lookup is Linux only, so on Windows the call throws when the thread has destructors - [std_execute_once](../../core/libs/prx/libc/Export.cpp) (libc) - assumed to be the plain-named twin of `std::_Execute_once` (`_ZSt13_Execute_onceRSt9once_flagPFiPvS1_PS1_ES1_`, the callee of Dinkumware `call_once`): it delegates there and gets the [thread.once.callonce] semantics already verified for that symbol (one invocation per flag, concurrent callers block until it completes, and a failing or throwing invocation leaves the flag unready so the next caller runs it). The name equivalence is not confirmed on a title, and the callback convention (called with (null, arg, null), non-zero return meaning success) is the twin's, unverified for this symbol -- [Dinkumware mutexes and conditions](../../core/libs/prx/libc/src/Threads.cpp) (libc) - `_Mtx_init`, `_Mtx_destroy`, `_Mtx_lock`, `_Mtx_unlock`, `_Cnd_init`, `_Cnd_destroy`, `_Cnd_wait` and `_Cnd_broadcast` follow microsoft/STL [`xthreads.h`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/inc/xthreads.h), [`mutex.cpp`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/src/mutex.cpp) and [`cond.cpp`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/src/cond.cpp), not the console's Dinkumware build. Settled by those files: `_Mtx_init` and `_Cnd_init` store null and then a heap handle through their pointer argument and return `_Thrd_nomem` (1) when allocation fails; destroy, lock, unlock, wait and broadcast take the handle by value; destroying a null handle does nothing; the other functions return `_Thrd_success` (0); an owner that locks a plain mutex (type `_Mtx_plain`, with or without `_Mtx_recursive`) or a recursive mutex again counts the lock, and an owner that locks any other non-recursive mutex gets `_Thrd_busy` (3). The handle points to a host object, so guest code that reads the mutex or condition fields itself instead of calling these functions breaks. Unverified on a console: whether its `_Mtx_lock`, `_Mtx_unlock`, `_Mtx_destroy`, `_Cnd_wait` and `_Cnd_destroy` take the handle by value like microsoft/STL or a pointer to it (a handle not made by `_Mtx_init` or `_Cnd_init` throws, so the pointer form fails loudly), and how the console build treats an owner relocking a plain or `_Mtx_try` mutex. Cases the sources leave to debug asserts or undefined behaviour throw: a null handle pointer for `_Mtx_init` or `_Cnd_init`, a null handle for the other non-destroy functions, a type that is not a combination of `_Mtx_plain`, `_Mtx_try`, `_Mtx_timed` and `_Mtx_recursive` with at least one of the first three, destroying a locked mutex, unlocking or waiting with a mutex the caller does not own, and waiting with a recursive mutex locked more than once (MSVC releases it entirely there while keeping the remaining count) +- [Dinkumware mutexes and conditions](../../core/libs/prx/libc/src/Threads.cpp) (libc) use pointer-to-handle arguments for lock, unlock, destroy, wait and broadcast, matching the console aerolib signatures and `_Mtx_lock` disassembly reported in [#3079](https://github.com/boykopovar/AnyPS5/pull/3079). The implementation follows microsoft/STL [`xthreads.h`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/inc/xthreads.h), [`mutex.cpp`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/src/mutex.cpp) and [`cond.cpp`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/src/cond.cpp) for object lifetime and synchronization behavior; the console build is still unverified on hardware. The reported `_Mtx_lock` maps success to `_Thrd_success` (0), `0x8002000b` to `_Thrd_busy` (3), and other failures to `_Thrd_error` (4). How the console build treats an owner relocking a plain or `_Mtx_try` mutex remains unverified - [tcgetattr, tcsetattr](../../core/libs/prx/libkernel/File/src/Terminal.cpp) (libkernel) - every file and pipe descriptor is taken as not a terminal (`ENOTTY`), and a socket gets `EADDRNOTAVAIL` for `AF_INET` and `EOPNOTSUPP` otherwise, as FreeBSD's `in_control` and `pru_control_notsupp` answer a termios ioctl; what descriptors 0 to 2 of a title are on the console, and whether any of them is a terminal there, is unknown - [__progname](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - unknown data export - [environ](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - data export pointing to a fixed empty environment so titles that import it load; it is not the environment libc's `getenv` reads, so it lists none of those variables and `setenv`/`putenv` changes do not appear in it From 64889ae192db80a1c5ff0c9d97dc5be9c1eeaa3a Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 22:01:45 +0000 Subject: [PATCH 208/460] fix(libc): map mutex deadlocks to busy --- core/libs/prx/libc/src/Threads.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libc/src/Threads.cpp b/core/libs/prx/libc/src/Threads.cpp index 6649f931d..17d071a98 100644 --- a/core/libs/prx/libc/src/Threads.cpp +++ b/core/libs/prx/libc/src/Threads.cpp @@ -107,8 +107,8 @@ int APS5_VABI _Mtx_lock_nid_postfix(void** handle) { } try { mutex->native.lock(); - } catch (const std::system_error&) { - return threadError; + } catch (const std::system_error& error) { + return error.code() == std::errc::resource_deadlock_would_occur ? threadBusy : threadError; } mutex->owner.store(self, std::memory_order_release); mutex->count = 1; From 583b3de28fc3538661bf24e25414443f1e7ae56a Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 22:02:52 +0000 Subject: [PATCH 209/460] fix(libc): return thread errors for invalid operations --- core/libs/prx/libc/src/Threads.cpp | 6 +++--- core/libs/tests/GuestLibcThreads.cpp | 9 +++++---- docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 9 insertions(+), 8 deletions(-) diff --git a/core/libs/prx/libc/src/Threads.cpp b/core/libs/prx/libc/src/Threads.cpp index 17d071a98..c6ea5ceea 100644 --- a/core/libs/prx/libc/src/Threads.cpp +++ b/core/libs/prx/libc/src/Threads.cpp @@ -118,7 +118,7 @@ int APS5_VABI _Mtx_lock_nid_postfix(void** handle) { int APS5_VABI _Mtx_unlock_nid_postfix(void** handle) { auto* mutex = ResolveMutex(handle, "_Mtx_unlock"); if (!IsOwnedByCaller(mutex)) - throw std::logic_error("_Mtx_unlock: mutex is not owned by the calling thread"); + return threadError; if (--mutex->count == 0) { mutex->owner.store(std::thread::id{}, std::memory_order_release); mutex->native.unlock(); @@ -151,9 +151,9 @@ int APS5_VABI _Cnd_wait_nid_postfix(void** conditionHandle, void** mutexHandle) auto* condition = ResolveCondition(conditionHandle, "_Cnd_wait"); auto* mutex = ResolveMutex(mutexHandle, "_Cnd_wait"); if (!IsOwnedByCaller(mutex)) - throw std::logic_error("_Cnd_wait: mutex is not owned by the calling thread"); + return threadError; if (mutex->count != 1) - throw std::logic_error("_Cnd_wait: recursive mutex is locked more than once"); + return threadError; mutex->count = 0; mutex->owner.store(std::thread::id{}, std::memory_order_release); std::unique_lock lock(mutex->native, std::adopt_lock); diff --git a/core/libs/tests/GuestLibcThreads.cpp b/core/libs/tests/GuestLibcThreads.cpp index 1f4438ff1..5d4f815d8 100644 --- a/core/libs/tests/GuestLibcThreads.cpp +++ b/core/libs/tests/GuestLibcThreads.cpp @@ -31,6 +31,7 @@ static void RequireThrows(TCall call) { } constexpr int success = 0; +constexpr int error = 4; constexpr int busy = 3; constexpr int plain = 0x01; constexpr int tryable = 0x02; @@ -48,9 +49,9 @@ static void MutexBasics() { Require(_Mtx_lock_nid_postfix(&mutex) == success); Require(_Mtx_lock_nid_postfix(&mutex) == busy); RequireThrows([&] { _Mtx_destroy_nid_postfix(&mutex); }); - std::thread([&] { RequireThrows([&] { _Mtx_unlock_nid_postfix(&mutex); }); }).join(); + std::thread([&] { Require(_Mtx_unlock_nid_postfix(&mutex) == error); }).join(); Require(_Mtx_unlock_nid_postfix(&mutex) == success); - RequireThrows([&] { _Mtx_unlock_nid_postfix(&mutex); }); + Require(_Mtx_unlock_nid_postfix(&mutex) == error); void* invalidMutex = nullptr; RequireThrows([&] { _Mtx_lock_nid_postfix(&invalidMutex); }); _Mtx_destroy_nid_postfix(&mutex); @@ -142,10 +143,10 @@ static void ConditionBroadcast() { Require(woken == 2); Require(_Cnd_broadcast_nid_postfix(&condition) == success); - RequireThrows([&] { _Cnd_wait_nid_postfix(&condition, &mutex); }); + Require(_Cnd_wait_nid_postfix(&condition, &mutex) == error); void* recursiveMutex = CreateMutex(tryable | recursive); Require(_Mtx_lock_nid_postfix(&recursiveMutex) == success && _Mtx_lock_nid_postfix(&recursiveMutex) == success); - RequireThrows([&] { _Cnd_wait_nid_postfix(&condition, &recursiveMutex); }); + Require(_Cnd_wait_nid_postfix(&condition, &recursiveMutex) == error); Require(_Mtx_unlock_nid_postfix(&recursiveMutex) == success && _Mtx_unlock_nid_postfix(&recursiveMutex) == success); _Mtx_destroy_nid_postfix(&recursiveMutex); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index f55cc7fdd..918149eaf 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -159,7 +159,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceImeGetPanelPositionAndForm](../../core/libs/prx/libSceIme/Export.cpp) (libSceIme) - the output is taken to be the `PositionAndForm` that `sceImeDialogGetPanelPositionAndForm` fills; it is never written, since no panel can be open. That a closed panel is reported before a null output is assumed by analogy with `sceImeSetCaret`, not confirmed on a PS5 title - [_sceLibcInternalForceTlsDestructor](../../core/libs/prx/libSceLibcInternal/Export.cpp) (libSceLibcInternal) - behaviour follows `sceLibcForceTlsDestructor(int)` in a title's `libc.prx` (disassembly): the calling thread's destructors whose dso belongs to the module run newest first, only if the dso's module is loaded and the dso points to itself, in up to 4 passes. The return type is unknown (that function returns nothing), 0 is returned. The module of each dso is looked up when the destructors are forced, not when they are registered as `libc.prx` does, and the lookup is Linux only, so on Windows the call throws when the thread has destructors - [std_execute_once](../../core/libs/prx/libc/Export.cpp) (libc) - assumed to be the plain-named twin of `std::_Execute_once` (`_ZSt13_Execute_onceRSt9once_flagPFiPvS1_PS1_ES1_`, the callee of Dinkumware `call_once`): it delegates there and gets the [thread.once.callonce] semantics already verified for that symbol (one invocation per flag, concurrent callers block until it completes, and a failing or throwing invocation leaves the flag unready so the next caller runs it). The name equivalence is not confirmed on a title, and the callback convention (called with (null, arg, null), non-zero return meaning success) is the twin's, unverified for this symbol -- [Dinkumware mutexes and conditions](../../core/libs/prx/libc/src/Threads.cpp) (libc) use pointer-to-handle arguments for lock, unlock, destroy, wait and broadcast, matching the console aerolib signatures and `_Mtx_lock` disassembly reported in [#3079](https://github.com/boykopovar/AnyPS5/pull/3079). The implementation follows microsoft/STL [`xthreads.h`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/inc/xthreads.h), [`mutex.cpp`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/src/mutex.cpp) and [`cond.cpp`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/src/cond.cpp) for object lifetime and synchronization behavior; the console build is still unverified on hardware. The reported `_Mtx_lock` maps success to `_Thrd_success` (0), `0x8002000b` to `_Thrd_busy` (3), and other failures to `_Thrd_error` (4). How the console build treats an owner relocking a plain or `_Mtx_try` mutex remains unverified +- [Dinkumware mutexes and conditions](../../core/libs/prx/libc/src/Threads.cpp) (libc) use pointer-to-handle arguments for lock, unlock, destroy, wait and broadcast, matching the console aerolib signatures and `_Mtx_lock` disassembly reported in [#3079](https://github.com/boykopovar/AnyPS5/pull/3079). The implementation follows microsoft/STL [`xthreads.h`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/inc/xthreads.h), [`mutex.cpp`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/src/mutex.cpp) and [`cond.cpp`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/src/cond.cpp) for object lifetime and synchronization behavior; the console build is still unverified on hardware. The reported `_Mtx_lock` maps success to `_Thrd_success` (0), `0x8002000b` to `_Thrd_busy` (3), and other failures to `_Thrd_error` (4). This implementation returns `_Thrd_error` (4) for unowned unlocks, invalid waits and host `system_error`s; `_Mtx_destroy` has no status result and throws if the mutex is locked. The console's mappings for those failure paths remain unverified. How the console build treats an owner relocking a plain or `_Mtx_try` mutex remains unverified - [tcgetattr, tcsetattr](../../core/libs/prx/libkernel/File/src/Terminal.cpp) (libkernel) - every file and pipe descriptor is taken as not a terminal (`ENOTTY`), and a socket gets `EADDRNOTAVAIL` for `AF_INET` and `EOPNOTSUPP` otherwise, as FreeBSD's `in_control` and `pru_control_notsupp` answer a termios ioctl; what descriptors 0 to 2 of a title are on the console, and whether any of them is a terminal there, is unknown - [__progname](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - unknown data export - [environ](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - data export pointing to a fixed empty environment so titles that import it load; it is not the environment libc's `getenv` reads, so it lists none of those variables and `setenv`/`putenv` changes do not appear in it From f4e393a4e49c81ab33c928e5274381c350b39608 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 22:36:24 +0000 Subject: [PATCH 210/460] fix(libc): clear destroyed thread handles --- core/libs/prx/libc/src/Threads.cpp | 2 ++ core/libs/tests/GuestLibcThreads.cpp | 4 ++++ 2 files changed, 6 insertions(+) diff --git a/core/libs/prx/libc/src/Threads.cpp b/core/libs/prx/libc/src/Threads.cpp index c6ea5ceea..4de0f5f8c 100644 --- a/core/libs/prx/libc/src/Threads.cpp +++ b/core/libs/prx/libc/src/Threads.cpp @@ -92,6 +92,7 @@ void APS5_VABI _Mtx_destroy_nid_postfix(void** handle) { throw std::logic_error("_Mtx_destroy: mutex destroyed while locked"); mutex->tag = 0; delete mutex; + *handle = nullptr; } int APS5_VABI _Mtx_lock_nid_postfix(void** handle) { @@ -145,6 +146,7 @@ void APS5_VABI _Cnd_destroy_nid_postfix(void** handle) { auto* condition = ResolveCondition(handle, "_Cnd_destroy"); condition->tag = 0; delete condition; + *handle = nullptr; } int APS5_VABI _Cnd_wait_nid_postfix(void** conditionHandle, void** mutexHandle) { diff --git a/core/libs/tests/GuestLibcThreads.cpp b/core/libs/tests/GuestLibcThreads.cpp index 5d4f815d8..37eba92a8 100644 --- a/core/libs/tests/GuestLibcThreads.cpp +++ b/core/libs/tests/GuestLibcThreads.cpp @@ -55,6 +55,8 @@ static void MutexBasics() { void* invalidMutex = nullptr; RequireThrows([&] { _Mtx_lock_nid_postfix(&invalidMutex); }); _Mtx_destroy_nid_postfix(&mutex); + Require(mutex == nullptr); + RequireThrows([&] { _Mtx_lock_nid_postfix(&mutex); }); void* emptyMutex = nullptr; _Mtx_destroy_nid_postfix(&emptyMutex); RequireThrows([] { _Mtx_destroy_nid_postfix(nullptr); }); @@ -156,6 +158,8 @@ static void ConditionBroadcast() { RequireThrows([] { _Cnd_broadcast_nid_postfix(nullptr); }); RequireThrows([] { _Cnd_init_nid_postfix(nullptr); }); _Cnd_destroy_nid_postfix(&condition); + Require(condition == nullptr); + RequireThrows([&] { _Cnd_broadcast_nid_postfix(&condition); }); void* emptyCondition = nullptr; _Cnd_destroy_nid_postfix(&emptyCondition); RequireThrows([] { _Cnd_destroy_nid_postfix(nullptr); }); From b603d19bc4c3a2b929cfb77377dae7c904b9958c Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 22:46:46 +0000 Subject: [PATCH 211/460] fix(libc): match console mutex failure behavior --- core/libs/prx/libc/Export.cpp | 9 -------- core/libs/prx/libc/src/Threads.cpp | 15 +++++-------- core/libs/tests/GuestLibcThreads.cpp | 33 ++++++++++++++++++---------- docs/dev/TechnicalDebt.md | 2 +- 4 files changed, 27 insertions(+), 32 deletions(-) diff --git a/core/libs/prx/libc/Export.cpp b/core/libs/prx/libc/Export.cpp index cad6c13b0..cf240955b 100644 --- a/core/libs/prx/libc/Export.cpp +++ b/core/libs/prx/libc/Export.cpp @@ -25,25 +25,16 @@ int APS5_VABI _ZSt14_Atomic_assertPKcS0__nid_postfix() { return 0; } - - APS5_EXPORT("Ye20uNnlglA", libcCyberUnknown02); std::uint64_t APS5_VABI libcCyberUnknown02(void) { NotImplemented_nid_no_patch("Ye20uNnlglA"); return 0; } - - APS5_EXPORT("H+8UBOwfScI", libcCyberUnknown08); std::uint64_t APS5_VABI libcCyberUnknown08(void) { NotImplemented_nid_no_patch("H+8UBOwfScI"); return 0; } - - - - - } diff --git a/core/libs/prx/libc/src/Threads.cpp b/core/libs/prx/libc/src/Threads.cpp index 4de0f5f8c..5399fd351 100644 --- a/core/libs/prx/libc/src/Threads.cpp +++ b/core/libs/prx/libc/src/Threads.cpp @@ -17,9 +17,6 @@ constexpr int threadNomem = 1; constexpr int threadBusy = 3; constexpr int threadError = 4; -constexpr int mutexPlain = 0x01; -constexpr int mutexTry = 0x02; -constexpr int mutexTimed = 0x04; constexpr int mutexRecursive = 0x100; constexpr std::uint64_t mutexTag = 0x5854'4D5F'4D54'5844ull; @@ -71,8 +68,6 @@ extern "C" { int APS5_VABI _Mtx_init_nid_postfix(void** handle, int type) { if (!handle) throw std::invalid_argument("_Mtx_init: null handle pointer"); - if ((type & ~(mutexPlain | mutexTry | mutexTimed | mutexRecursive)) != 0 || (type & (mutexPlain | mutexTry | mutexTimed)) == 0) - throw std::invalid_argument("_Mtx_init: unsupported mutex type " + std::to_string(type)); *handle = nullptr; auto* mutex = new (std::nothrow) DinkumwareMutex(); if (!mutex) @@ -99,7 +94,7 @@ int APS5_VABI _Mtx_lock_nid_postfix(void** handle) { auto* mutex = ResolveMutex(handle, "_Mtx_lock"); const auto self = std::this_thread::get_id(); if (mutex->owner.load(std::memory_order_acquire) == self) { - if ((mutex->type & ~mutexRecursive) != mutexPlain && (mutex->type & mutexRecursive) == 0) + if ((mutex->type & mutexRecursive) == 0) return threadBusy; if (mutex->count == std::numeric_limits::max()) throw std::overflow_error("_Mtx_lock: recursive lock count overflow"); @@ -119,7 +114,7 @@ int APS5_VABI _Mtx_lock_nid_postfix(void** handle) { int APS5_VABI _Mtx_unlock_nid_postfix(void** handle) { auto* mutex = ResolveMutex(handle, "_Mtx_unlock"); if (!IsOwnedByCaller(mutex)) - return threadError; + return threadSuccess; if (--mutex->count == 0) { mutex->owner.store(std::thread::id{}, std::memory_order_release); mutex->native.unlock(); @@ -153,9 +148,9 @@ int APS5_VABI _Cnd_wait_nid_postfix(void** conditionHandle, void** mutexHandle) auto* condition = ResolveCondition(conditionHandle, "_Cnd_wait"); auto* mutex = ResolveMutex(mutexHandle, "_Cnd_wait"); if (!IsOwnedByCaller(mutex)) - return threadError; + return threadSuccess; if (mutex->count != 1) - return threadError; + return threadSuccess; mutex->count = 0; mutex->owner.store(std::thread::id{}, std::memory_order_release); std::unique_lock lock(mutex->native, std::adopt_lock); @@ -165,7 +160,7 @@ int APS5_VABI _Cnd_wait_nid_postfix(void** conditionHandle, void** mutexHandle) lock.release(); mutex->owner.store(std::this_thread::get_id(), std::memory_order_release); mutex->count = 1; - return threadError; + throw; } lock.release(); mutex->owner.store(std::this_thread::get_id(), std::memory_order_release); diff --git a/core/libs/tests/GuestLibcThreads.cpp b/core/libs/tests/GuestLibcThreads.cpp index 37eba92a8..c0516c685 100644 --- a/core/libs/tests/GuestLibcThreads.cpp +++ b/core/libs/tests/GuestLibcThreads.cpp @@ -31,7 +31,6 @@ static void RequireThrows(TCall call) { } constexpr int success = 0; -constexpr int error = 4; constexpr int busy = 3; constexpr int plain = 0x01; constexpr int tryable = 0x02; @@ -49,11 +48,16 @@ static void MutexBasics() { Require(_Mtx_lock_nid_postfix(&mutex) == success); Require(_Mtx_lock_nid_postfix(&mutex) == busy); RequireThrows([&] { _Mtx_destroy_nid_postfix(&mutex); }); - std::thread([&] { Require(_Mtx_unlock_nid_postfix(&mutex) == error); }).join(); + std::thread([&] { Require(_Mtx_unlock_nid_postfix(&mutex) == success); }).join(); + Require(_Mtx_unlock_nid_postfix(&mutex) == success); Require(_Mtx_unlock_nid_postfix(&mutex) == success); - Require(_Mtx_unlock_nid_postfix(&mutex) == error); void* invalidMutex = nullptr; RequireThrows([&] { _Mtx_lock_nid_postfix(&invalidMutex); }); + void* foreignCondition = nullptr; + Require(_Cnd_init_nid_postfix(&foreignCondition) == success && foreignCondition != nullptr); + void* foreignMutex = foreignCondition; + RequireThrows([&] { _Mtx_lock_nid_postfix(&foreignMutex); }); + _Cnd_destroy_nid_postfix(&foreignCondition); _Mtx_destroy_nid_postfix(&mutex); Require(mutex == nullptr); RequireThrows([&] { _Mtx_lock_nid_postfix(&mutex); }); @@ -62,8 +66,8 @@ static void MutexBasics() { RequireThrows([] { _Mtx_destroy_nid_postfix(nullptr); }); void* plainMutex = CreateMutex(plain); - Require(_Mtx_lock_nid_postfix(&plainMutex) == success && _Mtx_lock_nid_postfix(&plainMutex) == success); - Require(_Mtx_unlock_nid_postfix(&plainMutex) == success && _Mtx_unlock_nid_postfix(&plainMutex) == success); + Require(_Mtx_lock_nid_postfix(&plainMutex) == success && _Mtx_lock_nid_postfix(&plainMutex) == busy); + Require(_Mtx_unlock_nid_postfix(&plainMutex) == success); _Mtx_destroy_nid_postfix(&plainMutex); void* timedMutex = CreateMutex(timed | tryable); @@ -71,11 +75,16 @@ static void MutexBasics() { Require(_Mtx_unlock_nid_postfix(&timedMutex) == success); _Mtx_destroy_nid_postfix(&timedMutex); - void* untouched = reinterpret_cast(std::uintptr_t{1}); - RequireThrows([&] { _Mtx_init_nid_postfix(&untouched, 0); }); - RequireThrows([&] { _Mtx_init_nid_postfix(&untouched, recursive); }); - RequireThrows([&] { _Mtx_init_nid_postfix(&untouched, tryable | 0x8); }); - Require(untouched == reinterpret_cast(std::uintptr_t{1})); + void* defaultMutex = nullptr; + Require(_Mtx_init_nid_postfix(&defaultMutex, 0) == success && defaultMutex != nullptr); + Require(_Mtx_lock_nid_postfix(&defaultMutex) == success && _Mtx_lock_nid_postfix(&defaultMutex) == busy); + Require(_Mtx_unlock_nid_postfix(&defaultMutex) == success); + _Mtx_destroy_nid_postfix(&defaultMutex); + void* unknownBitsMutex = nullptr; + Require(_Mtx_init_nid_postfix(&unknownBitsMutex, tryable | 0x8) == success && unknownBitsMutex != nullptr); + Require(_Mtx_lock_nid_postfix(&unknownBitsMutex) == success && _Mtx_lock_nid_postfix(&unknownBitsMutex) == busy); + Require(_Mtx_unlock_nid_postfix(&unknownBitsMutex) == success); + _Mtx_destroy_nid_postfix(&unknownBitsMutex); RequireThrows([] { _Mtx_init_nid_postfix(nullptr, tryable); }); RequireThrows([] { _Mtx_lock_nid_postfix(nullptr); }); RequireThrows([] { _Mtx_unlock_nid_postfix(nullptr); }); @@ -145,10 +154,10 @@ static void ConditionBroadcast() { Require(woken == 2); Require(_Cnd_broadcast_nid_postfix(&condition) == success); - Require(_Cnd_wait_nid_postfix(&condition, &mutex) == error); + Require(_Cnd_wait_nid_postfix(&condition, &mutex) == success); void* recursiveMutex = CreateMutex(tryable | recursive); Require(_Mtx_lock_nid_postfix(&recursiveMutex) == success && _Mtx_lock_nid_postfix(&recursiveMutex) == success); - Require(_Cnd_wait_nid_postfix(&condition, &recursiveMutex) == error); + Require(_Cnd_wait_nid_postfix(&condition, &recursiveMutex) == success); Require(_Mtx_unlock_nid_postfix(&recursiveMutex) == success && _Mtx_unlock_nid_postfix(&recursiveMutex) == success); _Mtx_destroy_nid_postfix(&recursiveMutex); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 918149eaf..b40dff379 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -159,7 +159,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceImeGetPanelPositionAndForm](../../core/libs/prx/libSceIme/Export.cpp) (libSceIme) - the output is taken to be the `PositionAndForm` that `sceImeDialogGetPanelPositionAndForm` fills; it is never written, since no panel can be open. That a closed panel is reported before a null output is assumed by analogy with `sceImeSetCaret`, not confirmed on a PS5 title - [_sceLibcInternalForceTlsDestructor](../../core/libs/prx/libSceLibcInternal/Export.cpp) (libSceLibcInternal) - behaviour follows `sceLibcForceTlsDestructor(int)` in a title's `libc.prx` (disassembly): the calling thread's destructors whose dso belongs to the module run newest first, only if the dso's module is loaded and the dso points to itself, in up to 4 passes. The return type is unknown (that function returns nothing), 0 is returned. The module of each dso is looked up when the destructors are forced, not when they are registered as `libc.prx` does, and the lookup is Linux only, so on Windows the call throws when the thread has destructors - [std_execute_once](../../core/libs/prx/libc/Export.cpp) (libc) - assumed to be the plain-named twin of `std::_Execute_once` (`_ZSt13_Execute_onceRSt9once_flagPFiPvS1_PS1_ES1_`, the callee of Dinkumware `call_once`): it delegates there and gets the [thread.once.callonce] semantics already verified for that symbol (one invocation per flag, concurrent callers block until it completes, and a failing or throwing invocation leaves the flag unready so the next caller runs it). The name equivalence is not confirmed on a title, and the callback convention (called with (null, arg, null), non-zero return meaning success) is the twin's, unverified for this symbol -- [Dinkumware mutexes and conditions](../../core/libs/prx/libc/src/Threads.cpp) (libc) use pointer-to-handle arguments for lock, unlock, destroy, wait and broadcast, matching the console aerolib signatures and `_Mtx_lock` disassembly reported in [#3079](https://github.com/boykopovar/AnyPS5/pull/3079). The implementation follows microsoft/STL [`xthreads.h`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/inc/xthreads.h), [`mutex.cpp`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/src/mutex.cpp) and [`cond.cpp`](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/src/cond.cpp) for object lifetime and synchronization behavior; the console build is still unverified on hardware. The reported `_Mtx_lock` maps success to `_Thrd_success` (0), `0x8002000b` to `_Thrd_busy` (3), and other failures to `_Thrd_error` (4). This implementation returns `_Thrd_error` (4) for unowned unlocks, invalid waits and host `system_error`s; `_Mtx_destroy` has no status result and throws if the mutex is locked. The console's mappings for those failure paths remain unverified. How the console build treats an owner relocking a plain or `_Mtx_try` mutex remains unverified +- [Dinkumware mutexes and conditions](../../core/libs/prx/libc/src/Threads.cpp) (libc) use pointer-to-handle arguments, matching the console aerolib signatures reported in [#3079](https://github.com/boykopovar/AnyPS5/pull/3079). The title's `_Mtx_lock` maps `0` to `_Thrd_success` (0), `0x8002000b` to `_Thrd_busy` (3), and other failures to `_Thrd_error` (4); its internal mutex init recognizes recursive flag `0x100`, while every other type uses default attributes. The observed relock failure for a plain mutex suggests those defaults are error-checking, but that has not been confirmed on hardware. The title's `_Mtx_unlock` and `_Cnd_wait` return success regardless of their internal result. The host implementation treats invalid ownership preconditions as no-ops, and throws for invalid handles, host wait errors and destroying a locked mutex as debug guards; those cases are not confirmed against the console. - [tcgetattr, tcsetattr](../../core/libs/prx/libkernel/File/src/Terminal.cpp) (libkernel) - every file and pipe descriptor is taken as not a terminal (`ENOTTY`), and a socket gets `EADDRNOTAVAIL` for `AF_INET` and `EOPNOTSUPP` otherwise, as FreeBSD's `in_control` and `pru_control_notsupp` answer a termios ioctl; what descriptors 0 to 2 of a title are on the console, and whether any of them is a terminal there, is unknown - [__progname](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - unknown data export - [environ](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - data export pointing to a fixed empty environment so titles that import it load; it is not the environment libc's `getenv` reads, so it lists none of those variables and `setenv`/`putenv` changes do not appear in it From 8ebf892849efea66a8c1cb90510ed30fe4799b7d Mon Sep 17 00:00:00 2001 From: Abdulrahman Elkordy <196347165+Silverash-i8@users.noreply.github.com> Date: Sat, 10 Oct 2026 20:36:48 +0300 Subject: [PATCH 212/460] fix(relinker): start on Windows without a missing library nothing imports from The Windows startup stub stopped at the first DT_NEEDED library it could not load, even when no import is searched in that library. Titles list libraries they import nothing from (Road of Death needs libSceXml.prx, Resident Evil 4 needs Party.prx), so they stopped with GetLastError 126 before running. A host library that no import's search list contains is now skipped with a message when it is not found; its handle stays null and is never read. Missing libraries that something imports from, other load errors and guest modules still stop the start. --- .../src/windows/WindowsEntryStubBuilder.cpp | 20 ++++++++++++++++++- .../tests/test_windows_import_modules.py | 13 ++++++++++++ 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp index b696e76a4..f203aca31 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp @@ -89,6 +89,7 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con std::vector searchTables; std::vector searchCounts; std::vector platformTlsResolvers; + std::vector imported(libraries.size()); for (const auto& import : imports) { std::vector order; for (std::size_t index = 0; index < libraries.size(); ++index) { @@ -119,7 +120,10 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con })); searchTables.push_back(CheckedRva(dataRva + data.size())); searchCounts.push_back(CheckedRva(order.size())); - for (const auto index : order) Io::AppendU32(data, index); + for (const auto index : order) { + Io::AppendU32(data, index); + imported[index] = true; + } } const auto lastError = reserve(4); @@ -129,6 +133,7 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con const auto loaded = addString(" -> OK\n"); const auto loadFailed = addString(" -> FAILED\n"); const auto failedModule = addString("Failed to load module: "); + const auto skippedModule = addString(" -> not found; nothing is imported from it, continuing without it\n"); const auto errorPrefix = addString("GetLastError: "); const auto messageSeparator = addString(" - "); const auto newline = addString("\n"); @@ -313,6 +318,12 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con code.Emit({0x48, 0x85, 0xc0}); const auto loadSucceeded = code.Branch({0x0f, 0x85}); captureLastError(); + std::optional unusedMissing; + if (index >= guestModules.size() && !imported[index]) { + code.Rip({0x8b, 0x05}, lastError); + code.Emit({0x83, 0xf8, 0x7e}); + unusedMissing = code.Branch({0x0f, 0x84}); + } writeString(loadFailed, true); writeString(failedModule, true); writeString(resolvedPaths[index], true); @@ -330,6 +341,13 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con if (dependencyDiagnostics) { writeString(loaded); } + if (unusedMissing) { + const auto next = code.Branch({0xe9}); + code.PatchBranch(*unusedMissing, code.GetRva()); + if (!dependencyDiagnostics) writeString(resolvedPaths[index], true); + writeString(skippedModule, true); + code.PatchBranch(next, code.GetRva()); + } } std::vector unresolvedBranches; diff --git a/core/relinker/relinker/tests/test_windows_import_modules.py b/core/relinker/relinker/tests/test_windows_import_modules.py index 3bc090877..c2a73999d 100644 --- a/core/relinker/relinker/tests/test_windows_import_modules.py +++ b/core/relinker/relinker/tests/test_windows_import_modules.py @@ -330,6 +330,19 @@ def main(): run = subprocess.run([str(output)], capture_output=True, text=True, timeout=30) assert run.returncode != 0 and 'Failed to load module:' in run.stderr and 'missing.prx' in run.stderr, (run.returncode, run.stderr) + case = work / 'missing-unused-dependency' + modules = case / 'prx' + modules.mkdir(parents=True) + (modules / 'a.prx').write_bytes(provider(11)) + source = case / 'input.elf' + source.write_bytes(executable('a.prx', extra_dependencies=('unused.prx',))) + output = case / 'output.exe' + result = subprocess.run([str(relinker), '--windows', str(source), str(output)], capture_output=True, text=True, timeout=30) + assert result.returncode == 0, (result.stdout, result.stderr) + if os.name == 'nt': + run = subprocess.run([str(output)], capture_output=True, text=True, timeout=30) + assert run.returncode == 11 and 'unused.prx -> not found; nothing is imported from it' in run.stderr, (run.returncode, run.stdout, run.stderr) + print('Windows module-scoped import tests passed') From 87ca3a2cf1a9f0bb14ad7263d22b783c731469a1 Mon Sep 17 00:00:00 2001 From: Abdulrahman Elkordy <196347165+Silverash-i8@users.noreply.github.com> Date: Sun, 11 Oct 2026 00:15:18 +0300 Subject: [PATCH 213/460] fix(relinker): count unnamed imports only against the library they declare An import whose module matches no needed library has no library name, and the Windows startup stub counted it against every library, so a missing library with nothing imported from it was still never skipped. Every such import in the executables of Record of Lodoss War, Resident Evil 4, PRAGMATA and Returnal declares a module that exists under another file name (libSceJson for libSceJson2.prx, for example), and none declares libSceXml or Party. The relinker now keeps the declared module names of those imports, for the executable and each guest module, and the stub counts a host library as imported from when its name up to the first dot matches one of them, ignoring case. A module declared as libGuest still keeps a missing libGuest.xyz required, as guest_module_identity checks. Imports without a module ID declare nothing and count against no library. The rule and the title data are from xXPBSXx. --- .../domain/include/domain/GuestRuntime.hpp | 1 + .../domain/include/domain/ImportModule.hpp | 10 ++++++-- core/relinker/domain/include/domain/Types.hpp | 2 ++ .../windows/WindowsEntryStubBuilder.hpp | 3 ++- .../src/windows/WindowsEntryStubBuilder.cpp | 21 ++++++++++++++-- .../src/windows/WindowsGuestModuleWriter.cpp | 1 + .../src/windows/WindowsPePatcher.cpp | 2 +- .../relinker/src/guest/GuestImageReader.cpp | 3 +++ .../src/pipeline/RelinkerPipeline.cpp | 4 +++ .../tests/test_windows_import_modules.py | 25 ++++++++++--------- 10 files changed, 54 insertions(+), 18 deletions(-) diff --git a/core/relinker/domain/include/domain/GuestRuntime.hpp b/core/relinker/domain/include/domain/GuestRuntime.hpp index 2c45abb92..6ce00f5a1 100644 --- a/core/relinker/domain/include/domain/GuestRuntime.hpp +++ b/core/relinker/domain/include/domain/GuestRuntime.hpp @@ -24,6 +24,7 @@ struct GuestRuntime { std::uint32_t FiniRva = 0; bool UsePlatformTlsResolver = true; std::vector Names; + std::vector UnmatchedImportModules; }; } diff --git a/core/relinker/domain/include/domain/ImportModule.hpp b/core/relinker/domain/include/domain/ImportModule.hpp index a2bd98e33..ecc49bd39 100644 --- a/core/relinker/domain/include/domain/ImportModule.hpp +++ b/core/relinker/domain/include/domain/ImportModule.hpp @@ -17,8 +17,7 @@ inline std::string ImportModuleName(std::string name) { return name; } -inline std::string ImportModule(const std::string& symbol, const std::map& modules, - std::span dependencies = {}) { +inline std::string DeclaredImportModule(const std::string& symbol, const std::map& modules) { if (modules.empty()) return {}; const auto first = symbol.find('#'); if (first == std::string::npos) return {}; @@ -38,6 +37,13 @@ inline std::string ImportModule(const std::string& symbol, const std::mapsecond; if (name.empty() || name.find_first_of("/\\:$\r\n") != std::string::npos) throw RelinkerException("Invalid import module name: " + name); + return name; +} + +inline std::string ImportModule(const std::string& symbol, const std::map& modules, + std::span dependencies = {}) { + auto name = DeclaredImportModule(symbol, modules); + if (name.empty()) return {}; if (!name.ends_with(".prx") && !name.ends_with(".sprx") && !name.ends_with(".suprx")) name += ".prx"; if (std::find(dependencies.begin(), dependencies.end(), name) != dependencies.end()) return name; std::string matched; diff --git a/core/relinker/domain/include/domain/Types.hpp b/core/relinker/domain/include/domain/Types.hpp index e49cc0347..35ec25747 100644 --- a/core/relinker/domain/include/domain/Types.hpp +++ b/core/relinker/domain/include/domain/Types.hpp @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -96,6 +97,7 @@ struct SysVDynamicSection { std::vector RelaPltData; std::vector GuestModules; std::map ImportModules; + std::set UnmatchedImportModules; }; struct CallRegistryEntry { diff --git a/core/relinker/elfpatcher/include/elfpatcher/windows/WindowsEntryStubBuilder.hpp b/core/relinker/elfpatcher/include/elfpatcher/windows/WindowsEntryStubBuilder.hpp index a7a1225c1..94f47c6a3 100644 --- a/core/relinker/elfpatcher/include/elfpatcher/windows/WindowsEntryStubBuilder.hpp +++ b/core/relinker/elfpatcher/include/elfpatcher/windows/WindowsEntryStubBuilder.hpp @@ -2,6 +2,7 @@ #define ELFPATCHER_WINDOWS_ENTRYSTUBBUILDER_HPP #include +#include namespace Elfpatcher::Windows { @@ -19,7 +20,7 @@ struct WindowsEntryStub { class WindowsEntryStubBuilder { public: - WindowsEntryStub Build(std::uint32_t dataRva, std::uint32_t entryRva, const WindowsImports& nativeImports, const std::vector& libraries, const std::vector& imports, const std::string& runPath, bool lazyBinding, bool dependencyDiagnostics, const std::vector& guestModules = {}) const; + WindowsEntryStub Build(std::uint32_t dataRva, std::uint32_t entryRva, const WindowsImports& nativeImports, const std::vector& libraries, const std::vector& imports, const std::string& runPath, bool lazyBinding, bool dependencyDiagnostics, const std::vector& guestModules = {}, const std::set& unmatchedImportModules = {}) const; }; } diff --git a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp index f203aca31..74dbb30ee 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp @@ -4,6 +4,8 @@ #include #include #include +#include +#include #include #include @@ -33,7 +35,7 @@ std::string normalizeRunPath(std::string path) { } -WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, const std::uint32_t entryRva, const WindowsImports& nativeImports, const std::vector& libraries, const std::vector& imports, const std::string& runPath, const bool lazyBinding, const bool dependencyDiagnostics, const std::vector& guestModules) const { +WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, const std::uint32_t entryRva, const WindowsImports& nativeImports, const std::vector& libraries, const std::vector& imports, const std::string& runPath, const bool lazyBinding, const bool dependencyDiagnostics, const std::vector& guestModules, const std::set& unmatchedImportModules) const { if (!imports.empty() && libraries.empty()) throw Domain::RelinkerException("ELF imports have no DT_NEEDED libraries"); const auto path = normalizeRunPath(runPath); @@ -122,10 +124,25 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con searchCounts.push_back(CheckedRva(order.size())); for (const auto index : order) { Io::AppendU32(data, index); - imported[index] = true; + if (!import.Library.empty()) imported[index] = true; } } + const auto stem = [](const std::string& name) { + auto result = std::filesystem::path(name).filename().string(); + result.resize(std::min(result.size(), result.find('.'))); + std::transform(result.begin(), result.end(), result.begin(), [](unsigned char character) { return static_cast(std::tolower(character)); }); + return result; + }; + std::set unmatched; + for (const auto& module : unmatchedImportModules) unmatched.insert(stem(module)); + for (const auto& guest : guestModules) { + for (const auto& module : guest.UnmatchedImportModules) unmatched.insert(stem(module)); + } + for (std::size_t index = guestModules.size(); index < libraries.size(); ++index) { + if (unmatched.contains(stem(libraries[index]))) imported[index] = true; + } + const auto lastError = reserve(4); const auto errorDigits = reserve(11); const auto errorMessage = reserve(ErrorMessageCapacity); diff --git a/core/relinker/elfpatcher/src/windows/WindowsGuestModuleWriter.cpp b/core/relinker/elfpatcher/src/windows/WindowsGuestModuleWriter.cpp index 2dc27774d..d3a913de2 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsGuestModuleWriter.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsGuestModuleWriter.cpp @@ -173,6 +173,7 @@ std::vector GuestModuleWriter::WriteWindows(const Relinker::GuestI for (const auto slot : guest.InitArray) runtime.InitArrayRvas.push_back(image.GetRva(slot, 8)); for (const auto slot : guest.FiniArray) runtime.FiniArrayRvas.push_back(image.GetRva(slot, 8)); runtime.InitRva = guest.Init == 0 ? 0 : image.GetRva(guest.Init); + runtime.UnmatchedImportModules.assign(guest.Dynamic.UnmatchedImportModules.begin(), guest.Dynamic.UnmatchedImportModules.end()); runtime.FiniRva = guest.Fini == 0 ? 0 : image.GetRva(guest.Fini); auto result = WindowsPeWriter().Write(sections, entry, directories); const auto characteristics = Io::ReadU16(result, 0x80 + 22); diff --git a/core/relinker/elfpatcher/src/windows/WindowsPePatcher.cpp b/core/relinker/elfpatcher/src/windows/WindowsPePatcher.cpp index 322fbd46c..3ac43727d 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsPePatcher.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsPePatcher.cpp @@ -88,7 +88,7 @@ std::vector WindowsPePatcher::Patch(const std::vector& sourceElf pltCount = compacted.SlotCount; } auto dynSection = _dynamicSectionBuilder->BuildDynamicSection(dynamicRefs, neededLibraries, dynJmpRelOffset, pltCount); + for (const auto& ref : dynamicRefs) { + if (!ref.Library.empty()) continue; + if (auto declared = Domain::DeclaredImportModule(ref.Nid, importModules); !declared.empty()) dynSection.UnmatchedImportModules.insert(std::move(declared)); + } auto appendRela = [&](std::vector& buf, std::uint64_t offset, std::uint64_t info, std::int64_t addend) { std::size_t pos = buf.size(); diff --git a/core/relinker/relinker/tests/test_windows_import_modules.py b/core/relinker/relinker/tests/test_windows_import_modules.py index c2a73999d..cd96d1276 100644 --- a/core/relinker/relinker/tests/test_windows_import_modules.py +++ b/core/relinker/relinker/tests/test_windows_import_modules.py @@ -330,18 +330,19 @@ def main(): run = subprocess.run([str(output)], capture_output=True, text=True, timeout=30) assert run.returncode != 0 and 'Failed to load module:' in run.stderr and 'missing.prx' in run.stderr, (run.returncode, run.stderr) - case = work / 'missing-unused-dependency' - modules = case / 'prx' - modules.mkdir(parents=True) - (modules / 'a.prx').write_bytes(provider(11)) - source = case / 'input.elf' - source.write_bytes(executable('a.prx', extra_dependencies=('unused.prx',))) - output = case / 'output.exe' - result = subprocess.run([str(relinker), '--windows', str(source), str(output)], capture_output=True, text=True, timeout=30) - assert result.returncode == 0, (result.stdout, result.stderr) - if os.name == 'nt': - run = subprocess.run([str(output)], capture_output=True, text=True, timeout=30) - assert run.returncode == 11 and 'unused.prx -> not found; nothing is imported from it' in run.stderr, (run.returncode, run.stdout, run.stderr) + for index, (symbol, module_name) in enumerate([('shared#A#B', None), ('shared', None), ('shared#A#B', 'libSceJson')]): + case = work / ('missing-unused-dependency-' + str(index)) + modules = case / 'prx' + modules.mkdir(parents=True) + (modules / 'a.prx').write_bytes(provider(11)) + source = case / 'input.elf' + source.write_bytes(executable('a.prx', symbol, module_name, extra_dependencies=('unused.prx',))) + output = case / 'output.exe' + result = subprocess.run([str(relinker), '--windows', str(source), str(output)], capture_output=True, text=True, timeout=30) + assert result.returncode == 0, (symbol, module_name, result.stdout, result.stderr) + if os.name == 'nt': + run = subprocess.run([str(output)], capture_output=True, text=True, timeout=30) + assert run.returncode == 11 and 'unused.prx -> not found; nothing is imported from it' in run.stderr, (symbol, module_name, run.returncode, run.stdout, run.stderr) print('Windows module-scoped import tests passed') From d90dff97a815e7a655115ee09d57dfb375ce2fdd Mon Sep 17 00:00:00 2001 From: fma-rc <188743521+fma-rc@users.noreply.github.com> Date: Sun, 11 Oct 2026 00:50:49 +0200 Subject: [PATCH 214/460] fix(libc): keep EINVAL from strerror_r for an unknown error with a short buffer --- core/libs/prx/libc/src/Errors.cpp | 2 +- core/libs/tests/GuestErrors.cpp | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libc/src/Errors.cpp b/core/libs/prx/libc/src/Errors.cpp index 963191cf6..5523e1bf7 100644 --- a/core/libs/prx/libc/src/Errors.cpp +++ b/core/libs/prx/libc/src/Errors.cpp @@ -54,7 +54,7 @@ int APS5_VABI strerror_r_nid_postfix(int error, char* buffer, std::size_t length else std::snprintf(temporary, sizeof(temporary), "Unknown error: %d", error); const auto required = std::strlen(temporary) + 1; int result = Known(error) ? 0 : 22; - if (!buffer || length < required) result = 34; + if (Known(error) && (!buffer || length < required)) result = 34; if (buffer && length) { const auto count = required <= length ? required - 1 : length - 1; std::memcpy(buffer, temporary, count); diff --git a/core/libs/tests/GuestErrors.cpp b/core/libs/tests/GuestErrors.cpp index 52160d562..fd2c8a422 100644 --- a/core/libs/tests/GuestErrors.cpp +++ b/core/libs/tests/GuestErrors.cpp @@ -36,4 +36,7 @@ int main() { sentinel[0] = 'x'; Require(strerror_r_nid_postfix(22, sentinel, 0) == 34 && sentinel[0] == 'x'); Require(strerror_r_nid_postfix(22, nullptr, 0) == 34); + char truncated[4]; + Require(strerror_r_nid_postfix(97, truncated, sizeof(truncated)) == 22 && std::strcmp(truncated, "Unk") == 0); + Require(strerror_r_nid_postfix(-1, truncated, sizeof(truncated)) == 22 && std::strcmp(truncated, "Unk") == 0); } From 23c91d948e776bdd85e3744106ad8ad02a0055b5 Mon Sep 17 00:00:00 2001 From: "john.maia" Date: Fri, 9 Oct 2026 23:51:22 +0100 Subject: [PATCH 215/460] perf(agc): store write-back changes sixteen bytes at a time GuestMemory::WriteChanged stores only the bytes the GPU changed, so that CPU writes to the bytes around them survive. It found them a byte at a time inside every changed 256-byte block and stored each run with memcpy. Where changed and unchanged bytes interleave (the low bytes of floats), every run is a few bytes long, and that loop took 43% of the GPU submission thread in Disney Epic Mickey: Rebrushed's first level (0.9-1.0 ms per MiB). Each 16-byte chunk is now compared with SSE2. A chunk with no change is skipped, a chunk that changed entirely is stored whole, and a mixed one is stored with MASKMOVDQU, which writes only the bytes selected by the mask. An SFENCE after the loop orders those non-temporal stores before the stores that follow. Bytes the GPU did not change are still never written. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0134RWCyiyydJBbfKz8g9Bwe --- .../Execution/src/GuestMemory.cpp | 37 ++++++++++--------- core/libs/prx/libSceAgcDriver/tests/Pm4.cpp | 23 ++++++++++++ 2 files changed, 43 insertions(+), 17 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp index 6f0bc11d2..5029a9422 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp @@ -10,6 +10,8 @@ #include #include #include +#include +#include #include #include #include @@ -1875,28 +1877,29 @@ void WriteChanged(std::uint64_t address, std::span current, std const TimedAccess timed(CounterChangedWrite, current.size()); auto* destination = reinterpret_cast(address); CheckRange(destination, current.size(), 1, true); - constexpr std::size_t block = 256; const auto size = current.size(); - const auto differs = [&](std::size_t at) { - const auto length = std::min(block, size - at); - return std::memcmp(current.data() + at, original.data() + at, length) != 0; - }; // Stamped like a GPU write: a collect memoized for this packet would not see the page fault. storeOwn(address, size, [&] { std::size_t firstChanged = size; std::size_t lastChanged = 0; - for (std::size_t at = 0; at < size; at += block) { - if (!differs(at)) continue; - const auto blockEnd = std::min(at + block, size); - for (std::size_t run = at; run < blockEnd;) { - if (current[run] == original[run]) { ++run; continue; } - auto runEnd = run + 1; - while (runEnd < blockEnd && current[runEnd] != original[runEnd]) ++runEnd; - std::memcpy(destination + run, current.data() + run, runEnd - run); - firstChanged = std::min(firstChanged, run); - lastChanged = std::max(lastChanged, runEnd); - run = runEnd; - } + std::size_t at = 0; + for (; at + 16 <= size; at += 16) { + const auto bytes = _mm_loadu_si128(reinterpret_cast(current.data() + at)); + const auto same = _mm_cmpeq_epi8(bytes, _mm_loadu_si128(reinterpret_cast(original.data() + at))); + const auto equal = static_cast(_mm_movemask_epi8(same)); + if (equal == 0xffffu) continue; + if (equal == 0) _mm_storeu_si128(reinterpret_cast<__m128i*>(destination + at), bytes); + else _mm_maskmoveu_si128(bytes, _mm_xor_si128(same, _mm_set1_epi8(-1)), reinterpret_cast(destination + at)); + const auto changed = ~equal & 0xffffu; + firstChanged = std::min(firstChanged, at + static_cast(std::countr_zero(changed))); + lastChanged = std::max(lastChanged, at + 32 - static_cast(std::countl_zero(changed))); + } + _mm_sfence(); + for (; at < size; ++at) { + if (current[at] == original[at]) continue; + destination[at] = current[at]; + firstChanged = std::min(firstChanged, at); + lastChanged = std::max(lastChanged, at + 1); } return firstChanged < lastChanged ? std::pair{address + firstChanged, address + lastChanged} : std::pair{0, 0}; }); diff --git a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp index e6ef42a26..63c7ca3cf 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp @@ -537,6 +537,28 @@ void testWriteChangedKeepsUntouchedBytes() { check(guest[3] == 7 && guest[100] == 0x55, "write-back rolled back a byte the GPU did not change"); } +void testWriteChangedStoresExactlyTheChangedRuns() { + alignas(256) static std::uint8_t guest[1024 + 13]; + std::vector original(sizeof(guest)), current(sizeof(guest)); + std::uint32_t seed = 12345; + const auto next = [&] { return seed = seed * 1103515245u + 12345u, seed >> 16u; }; + for (std::size_t i = 0; i < sizeof(guest);) { + const auto length = 1 + next() % 40; + const bool changed = next() % 2 == 0; + for (std::size_t j = i; j < std::min(sizeof(guest), i + length); ++j) { + original[j] = static_cast(next()); + current[j] = changed && next() % 9 != 0 ? static_cast(static_cast(original[j]) ^ (1u + next() % 255u)) : original[j]; + } + i += length; + } + for (std::size_t i = 0; i < sizeof(guest); ++i) guest[i] = static_cast(0xa0u ^ i); + AgcDriver::GuestMemory::WriteChanged(reinterpret_cast(guest), current, original); + for (std::size_t i = 0; i < sizeof(guest); ++i) { + const auto expected = current[i] != original[i] ? static_cast(current[i]) : static_cast(0xa0u ^ i); + check(guest[i] == expected, "write-back stored a byte the GPU did not change or missed one it did"); + } +} + void testEventWrite() { for (const auto eventType : {0x07u, 0x0fu, 0x10u}) { AgcDriver::Pm4::Validate(makePacket(0x46, {0x400u | eventType}), 0); @@ -1049,6 +1071,7 @@ int main(int argc, char** argv) { testCatalog(); testNopPad(); testWriteChangedKeepsUntouchedBytes(); + testWriteChangedStoresExactlyTheChangedRuns(); testRegisters(); testRegisterFile(); testContextAndBases(); From 34e4aaf1339fcf50437f3aa9021065516f73ced5 Mon Sep 17 00:00:00 2001 From: Hussein Kadhim Date: Sat, 10 Oct 2026 13:19:50 +0300 Subject: [PATCH 216/460] fix(shader): LDS 32-bit atomics take the workgroup lock of the program's 64-bit LDS atomics The hardware serialises 32-bit and 64-bit LDS atomics on one address; the lock that makes the 64-bit forms atomic did not order 32-bit atomics, which lost increments between the two halves of a 64-bit update (#2297). --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 13 ++ .../tests/SpirvLdsAtomicLock.cpp | 130 +++++++++++++++++ .../tests/execution/LdsMixedWidthAtomics.cpp | 137 ++++++++++++++++++ .../SpirvMemory/SpirvMemoryInstructions.cpp | 28 ++-- docs/dev/TechnicalDebt.md | 2 +- 5 files changed, 298 insertions(+), 12 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/SpirvLdsAtomicLock.cpp create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index ee8273d8a..409f407d0 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -431,6 +431,19 @@ if(BUILD_TESTING) endif() endif() +add_test_executable(agc_spirv_lds_atomic_lock_tests EXCLUDE_FROM_ALL tests/SpirvLdsAtomicLock.cpp) +target_link_libraries(agc_spirv_lds_atomic_lock_tests PRIVATE shader_recompiler_agc_driver libc) +set_target_properties(agc_spirv_lds_atomic_lock_tests PROPERTIES CXX_EXTENSIONS OFF) +configure_windows_unwind(agc_spirv_lds_atomic_lock_tests) +if(BUILD_TESTING) + set_target_properties(agc_spirv_lds_atomic_lock_tests PROPERTIES EXCLUDE_FROM_ALL OFF) + add_test(NAME agc_spirv_lds_atomic_lock COMMAND agc_spirv_lds_atomic_lock_tests) + set_tests_properties(agc_spirv_lds_atomic_lock PROPERTIES ENVIRONMENT "ANYPS5_NO_SHADER_CACHE=1") + if(WIN32) + set_tests_properties(agc_spirv_lds_atomic_lock PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") + endif() +endif() + add_test_executable(agc_dpp_write_tests EXCLUDE_FROM_ALL tests/DppWrite.cpp) target_link_libraries(agc_dpp_write_tests PRIVATE shader_recompiler_agc_driver libc) set_target_properties(agc_dpp_write_tests PROPERTIES CXX_EXTENSIONS OFF) diff --git a/core/libs/prx/libSceAgcDriver/tests/SpirvLdsAtomicLock.cpp b/core/libs/prx/libSceAgcDriver/tests/SpirvLdsAtomicLock.cpp new file mode 100644 index 000000000..7f25bf955 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/SpirvLdsAtomicLock.cpp @@ -0,0 +1,130 @@ +#include "Recompiler.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using namespace ShaderRecompiler; + +constexpr std::uint32_t DsAddU64 = 0xd9000000u; +constexpr std::uint32_t DsAddU32Hi = 0xd8000004u; +constexpr std::uint32_t DsIncU32Hi = 0xd80c0004u; +constexpr std::uint32_t DsOperands = 0x00000201u; +constexpr std::uint32_t DsOperand32 = 0x00000401u; + +struct Usage { + std::size_t elects = 0; + std::size_t exchanges = 0; + std::size_t releases = 0; + std::size_t adds = 0; + bool addBetweenLock = false; +}; + +void Require(bool condition, const std::string& message) { + if (!condition) throw std::runtime_error(message); +} + +std::vector RecompileProgram(std::span code) { + const std::array capabilities{spv::CapabilityShader, spv::CapabilityGroupNonUniform, spv::CapabilityGroupNonUniformBallot, spv::CapabilityGroupNonUniformShuffle}; + const std::array userData{}; + const std::array extensions{}; + RecompileRequest request{}; + request.shader = {ShaderStage::Compute, 0x20000u, code, 0, {}}; + request.context.waveSize = 32; + request.context.userDataBaseRegister = 0; + request.context.userData = userData; + request.context.compute = ShaderComputeStageInfo{{64u, 1u, 1u}, 4u, {false, false, false}, false, 1u}; + request.target.vulkanVersion = 0x00403000u; + request.target.spirvVersion = 0x00010600u; + request.target.subgroupSize = 32; + request.target.bdaAbiVersion = 1; + request.target.supportedCapabilities = capabilities; + request.target.supportedExtensions = extensions; + request.target.maxWorkgroupSize = {1024u, 1024u, 64u}; + request.target.maxWorkgroupInvocations = 1024; + request.target.maxWorkgroupSharedMemoryBytes = 49152; + request.layout = {0, 0, 0, 128}; + request.useCache = false; + auto result = Recompile(request); + Require(!result.spirv.empty(), "the LDS atomic program did not recompile"); + return std::vector(result.spirv.Words().begin(), result.spirv.Words().end()); +} + +Usage Scan(std::span words) { + Usage usage; + bool locked = false; + for (std::size_t cursor = 5; cursor < words.size();) { + const auto count = words[cursor] >> spv::WordCountShift; + const auto op = static_cast(words[cursor] & spv::OpCodeMask); + Require(count != 0 && cursor + count <= words.size(), "truncated SPIR-V instruction"); + if (op == spv::OpGroupNonUniformElect) ++usage.elects; + if (op == spv::OpAtomicCompareExchange) { + ++usage.exchanges; + locked = true; + } + if (op == spv::OpAtomicStore) { + ++usage.releases; + locked = false; + } + if (op == spv::OpAtomicIAdd) { + ++usage.adds; + if (locked) usage.addBetweenLock = true; + } + cursor += count; + } + return usage; +} + +Usage Compile(std::initializer_list body) { + std::vector code{0x7e020280u, 0x7e0402c1u, 0x7e060280u, 0x7e080281u}; + code.insert(code.end(), body); + code.insert(code.end(), {0xbf8cc07fu, 0xbf810000u}); + return Scan(RecompileProgram(code)); +} + +void Check64BitOnlyLocks() { + const auto usage = Compile({DsAddU64, DsOperands}); + Require(usage.elects == 1 && usage.exchanges == 1 && usage.releases == 1, "a ds_add_u64 is not serialised by one elected lane taking the workgroup lock"); +} + +void Check32BitTakesLockWith64Bit() { + const auto usage = Compile({DsAddU64, DsOperands, DsAddU32Hi, DsOperand32}); + Require(usage.elects == 2 && usage.exchanges == 2 && usage.releases == 2, "ds_add_u32 in a program with ds_add_u64 does not take the workgroup lock: elect " + std::to_string(usage.elects) + ", lock acquire " + std::to_string(usage.exchanges) + ", lock release " + std::to_string(usage.releases) + ", expected 2 each"); + Require(usage.adds == 1 && usage.addBetweenLock, "the 32-bit OpAtomicIAdd is not between the lock acquire and the lock release"); +} + +void CheckOtherUpdatePathTakesLock() { + const auto usage = Compile({DsAddU64, DsOperands, DsIncU32Hi, DsOperand32}); + Require(usage.elects == 2 && usage.exchanges >= 2 && usage.releases == 2, "ds_inc_u32 in a program with ds_add_u64 does not take the workgroup lock: elect " + std::to_string(usage.elects) + ", lock release " + std::to_string(usage.releases) + ", expected 2 each"); +} + +void Check32BitOnlyIsUnchanged() { + const auto add = Compile({DsAddU32Hi, DsOperand32}); + Require(add.elects == 0 && add.exchanges == 0 && add.releases == 0 && add.adds == 1, "a program with only ds_add_u32 got a workgroup lock: elect " + std::to_string(add.elects) + ", lock acquire " + std::to_string(add.exchanges) + ", lock release " + std::to_string(add.releases) + ", native add " + std::to_string(add.adds)); + const auto inc = Compile({DsIncU32Hi, DsOperand32}); + Require(inc.elects == 0 && inc.releases == 0, "a program with only ds_inc_u32 got a workgroup lock: elect " + std::to_string(inc.elects) + ", lock release " + std::to_string(inc.releases)); +} + +} + +int main() { + try { + Check64BitOnlyLocks(); + Check32BitTakesLockWith64Bit(); + CheckOtherUpdatePathTakesLock(); + Check32BitOnlyIsUnchanged(); + } catch (const std::exception& error) { + std::fprintf(stderr, "SpirvLdsAtomicLock: %s\n", error.what()); + return 1; + } + std::puts("SpirvLdsAtomicLock: 32-bit LDS atomics take the 64-bit workgroup lock only in programs with a 64-bit LDS atomic"); + return 0; +} diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp new file mode 100644 index 000000000..ca554eabd --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp @@ -0,0 +1,137 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 256; +constexpr std::uint32_t Iterations = 16; +constexpr std::uint32_t LdsDwords = 4; +alignas(256) std::array Output{}; +alignas(256) std::array Unused{}; + +alignas(256) constexpr auto UniformCode = std::to_array({ + 0x34100083, + 0x7e020280, + 0x7e0402c1, + 0x7e060280, + 0x7e080281, + 0x7e0c0280, + 0x7e0e0280, + 0xd9340000, 0x00000601, + 0xbf8cc07f, + 0xbf8a0000, + 0xbe800390 | (8u << 16u), + 0xd9000000, 0x00000201, + 0xd8000004, 0x00000401, + 0xbf8cc07f, + 0x80888108, + 0xbf078008, + 0xbf85fff8, + 0xbf8cc07f, + 0xbf8a0000, + 0xd9d80000, 0x0a000001, + 0xbf8cc07f, + 0xe0741000, 0x80010a08, + 0xbf810000, +}); + +alignas(256) constexpr auto DivergentCode = std::to_array({ + 0x34100083, + 0x7e020280, + 0x7e0402c1, + 0x7e060280, + 0x7e080281, + 0x7e0c0280, + 0x7e0e0280, + 0xd9340000, 0x00000601, + 0xbf8cc07f, + 0xbf8a0000, + 0xbe89037e, + 0xbe800390 | (8u << 16u), + 0xbefe03ff, 0x55555555, + 0xd9000000, 0x00000201, + 0xbefe03ff, 0xaaaaaaaa, + 0xd8000004, 0x00000401, + 0xbf8cc07f, + 0x80888108, + 0xbf078008, + 0xbf85fff4, + 0xbefe0309, + 0xbf8cc07f, + 0xbf8a0000, + 0xd9d80000, 0x0a000001, + 0xbf8cc07f, + 0xe0741000, 0x80010a08, + 0xbf810000, +}); + +std::array BufferDescriptor(const void* data, std::uint32_t bytes) { + const auto address = reinterpret_cast(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; +} + +std::string Hex(std::uint64_t value) { + char text[24]; + std::snprintf(text, sizeof(text), "0x%016llx", static_cast(value)); + return text; +} + +void Run(AgcDriver::VulkanDevice& device, std::span code) { + Output.fill(0xdeadbeefu); + std::vector userData(8, 0u); + const auto input = BufferDescriptor(Unused.data(), static_cast(Unused.size() * 4u)); + const auto output = BufferDescriptor(Output.data(), static_cast(Output.size() * 4u)); + std::copy(input.begin(), input.end(), userData.begin()); + std::copy(output.begin(), output.end(), userData.begin() + 4); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, LdsDwords, {false, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); +} + +void Check(const char* name, std::uint64_t operations) { + const std::uint64_t expected = operations * 0xffffffffull + (operations << 32u); + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + const std::uint64_t actual = Output[tid * 2u] | (static_cast(Output[tid * 2u + 1u]) << 32u); + Require(actual == expected, std::string("lds mixed width atomics ") + name + ": lane " + std::to_string(tid) + " read " + Hex(actual) + ", expected " + Hex(expected) + " after " + std::to_string(operations) + " ds_add_u64 and ds_add_u32 on one slot"); + } +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + Run(*device, UniformCode); + Check("uniform", static_cast(Threads) * Iterations); + Run(*device, DivergentCode); + Check("divergent", static_cast(Threads) / 2u * Iterations); + std::puts("lds mixed width atomics tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp index 94b336e13..b26ab160b 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp @@ -642,6 +642,15 @@ std::uint32_t EmitAtomicOperation(SpirvValueEmitContext& ctx, const IrValue& ins return old; } +template +std::uint32_t LockedLdsUpdate(SpirvEmitterState& state, std::uint32_t type, std::uint32_t zero, TUpdate&& update); + +template +std::uint32_t EmitLdsAtomic32(SpirvEmitterState& state, const MemoryInfo& mem, TOperation&& operation) { + if (mem.kind != ResourceKind::Lds || !state.requirements.ldsLock) return operation(); + return LockedLdsUpdate(state, TypeU32(state), ConstantU32(state, 0u), operation); +} + template std::uint32_t EmitAtomicAccess(SpirvValueEmitContext& ctx, const IrValue& inst, const MemoryInfo& mem, TOperation&& operation) { if (mem.kind == ResourceKind::Buffer) return EmitRuntimeBufferAtomic(ctx, inst, 32u, ActiveArgument(ctx, inst), operation); @@ -649,7 +658,7 @@ std::uint32_t EmitAtomicAccess(SpirvValueEmitContext& ctx, const IrValue& inst, return EmitValueOrZeroIfCondition(state, ActiveArgument(ctx, inst), [&]() { const auto access = PrepareMemoryElement(ctx, mem, DwordIndex(ctx, inst, mem)); return EmitValueOrZeroIfCondition(state, EmitMemoryElementInBounds(state, access.resource, access.index), [&]() { - return operation(EmitMemoryElementPointer(state, access.resource, access.index)); + return EmitLdsAtomic32(state, mem, [&]() { return operation(EmitMemoryElementPointer(state, access.resource, access.index)); }); }); }); } @@ -750,7 +759,7 @@ std::uint32_t Atomic32(SpirvValueEmitContext& ctx, const IrValue& inst, const Me if (mem.kind == ResourceKind::Buffer) return EmitRuntimeBufferAtomic(ctx, inst, 32u, active, apply); return EmitValueOrZeroIfCondition(state, active, [&] { const auto access = PrepareMemoryElement(ctx, mem, DwordIndex(ctx, inst, mem)); - return EmitValueOrZeroIfCondition(state, EmitMemoryElementInBounds(state, access.resource, access.index), [&] { return apply(EmitMemoryElementPointer(state, access.resource, access.index)); }); + return EmitValueOrZeroIfCondition(state, EmitMemoryElementInBounds(state, access.resource, access.index), [&] { return EmitLdsAtomic32(state, mem, [&] { return apply(EmitMemoryElementPointer(state, access.resource, access.index)); }); }); }); } @@ -1019,9 +1028,6 @@ std::uint32_t FlatAtomicNext64(SpirvEmitterState& state, IrOpcode opcode, std::u } } -template -std::uint32_t LockedLdsUpdate(SpirvEmitterState& state, TUpdate&& update); - std::uint32_t ApertureAtomic64(SpirvValueEmitContext& ctx, const IrValue& inst, ResourceKind kind, std::uint32_t address, std::uint32_t byteOffset, std::uint32_t value, std::uint32_t comparator) { auto& state = ctx.state; const auto instruction = ConstantU32(state, inst.Flags().pc); @@ -1056,7 +1062,7 @@ std::uint32_t ApertureAtomic64(SpirvValueEmitContext& ctx, const IrValue& inst, state.module.AddFunction(spv::OpStore, highPointer, nextHigh); return old; }; - return kind == ResourceKind::Lds ? LockedLdsUpdate(state, update) : update(); + return kind == ResourceKind::Lds ? LockedLdsUpdate(state, TypeU64(state), ConstantU64(state, 0u), update) : update(); }); }); } @@ -1179,7 +1185,7 @@ std::uint32_t AppendConsume(SpirvValueEmitContext& ctx, const IrValue& inst, boo } template -std::uint32_t LockedLdsUpdate(SpirvEmitterState& state, TUpdate&& update) { +std::uint32_t LockedLdsUpdate(SpirvEmitterState& state, std::uint32_t type, std::uint32_t zero, TUpdate&& update) { const auto lock = EmitLdsLockPointer(state); const auto entry = state.currentLabel; const auto header = state.module.AllocateId(); @@ -1200,7 +1206,7 @@ std::uint32_t LockedLdsUpdate(SpirvEmitterState& state, TUpdate&& update) { state.module.AddFunction(spv::OpBranch, header); EmitLabel(state, header); state.module.AddFunction(spv::OpPhi, TypeBool(state), done, ConstantBool(state, false), entry, doneNext, cont); - state.module.AddFunction(spv::OpPhi, TypeU64(state), result, ConstantU64(state, 0u), entry, resultNext, cont); + state.module.AddFunction(spv::OpPhi, type, result, zero, entry, resultNext, cont); state.module.AddFunction(spv::OpLoopMerge, merge, cont, spv::LoopControlMaskNone); state.module.AddFunction(spv::OpBranch, body); EmitLabel(state, body); @@ -1230,11 +1236,11 @@ std::uint32_t LockedLdsUpdate(SpirvEmitterState& state, TUpdate&& update) { const auto served = state.module.AllocateId(); const auto servedResult = state.module.AllocateId(); state.module.AddFunction(spv::OpPhi, TypeBool(state), served, ConstantBool(state, true), criticalExit, ConstantBool(state, false), pending); - state.module.AddFunction(spv::OpPhi, TypeU64(state), servedResult, old, criticalExit, result, pending); + state.module.AddFunction(spv::OpPhi, type, servedResult, old, criticalExit, result, pending); state.module.AddFunction(spv::OpBranch, pendingMerge); EmitLabel(state, pendingMerge); state.module.AddFunction(spv::OpPhi, TypeBool(state), doneNext, served, servedMerge, done, body); - state.module.AddFunction(spv::OpPhi, TypeU64(state), resultNext, servedResult, servedMerge, result, body); + state.module.AddFunction(spv::OpPhi, type, resultNext, servedResult, servedMerge, result, body); const auto ballot = state.module.AllocateId(); state.module.AddFunction(spv::OpGroupNonUniformBallot, TypeU32Vector(state, 4u), ballot, ConstantU32(state, spv::ScopeSubgroup), Unary(state, spv::OpLogicalNot, TypeBool(state), doneNext)); std::uint32_t remaining = ConstantU32(state, 0u); @@ -1278,7 +1284,7 @@ std::uint32_t SharedAtomic64(SpirvValueEmitContext& ctx, const IrValue& inst, TR state.module.AddFunction(spv::OpStore, highPointer, nextHigh); return old; }; - return state.requirements.ldsLock ? LockedLdsUpdate(state, update) : update(); + return state.requirements.ldsLock ? LockedLdsUpdate(state, TypeU64(state), ConstantU64(state, 0u), update) : update(); }); }); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index dfc24f7e1..1e7aab2a6 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -454,7 +454,7 @@ Throughout the project, every function at every stage either **does exactly what - [Converted unorm images](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (10_11_11_UNORM, read and written through an R32_UINT view): samples, gathers, LOD queries, 16-bit data and a DST_SEL of the missing W channel throw. - [Packed 16-bit textures](../../core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp) (libSceAgcDriver) - `5_6_5`, `1_5_5_5` and `4_4_4_4` are viewed as R5G6B5, A1R5G5B5 and R4G4B4A4 with a swizzle that reads X from the low bits, and a DST_SEL of the missing W of `5_6_5` reads X. Measured on RDNA2 (gfx1035), not on the console: through `image_load` and `image_sample_lz`, X, Y, Z (and W of `1_5_5_5` and `4_4_4_4`) come from the low, middle and high fields, and W of `5_6_5` returns X (the low 5 bits), not 1. - [Converted unorm image loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) divide by 2^n - 1 with the buffer format path's `OpFDiv`, which Vulkan allows to be up to 2.5 ULP off, so some drivers are not bit-exact. -- [64-bit LDS atomics](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (`ds_*_u64`, `ds_*_b64`, `ds_*_f64`) read and write the two dwords under a workgroup lock stored after the guest LDS, as LDS is a 32-bit array without 64-bit atomics. They are atomic with respect to each other only: 32-bit accesses to the same dwords issued concurrently are not ordered by the lock. 64-bit GDS atomics throw. On the hardware the LDS serialises 32-bit and 64-bit atomics on the same address (256 lanes mixing `ds_add_u32` and `ds_add_u64` on one slot sum exactly), and under the lock the 32-bit adds can be lost: in 20 runs per host and wave size, RADV lost 32 to 96 of the 128 in every run and the RTX 3080 16 to 80 in 18 of 20 wave32 runs and 16 in 3 of 20 wave64 runs, always a multiple of 16. A 64-bit atomic at an address that is 4-byte but not 8-byte aligned faults the wave on gfx1036, as on gfx1035. +- [64-bit LDS atomics](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (`ds_*_u64`, `ds_*_b64`, `ds_*_f64`) read and write the two dwords under a workgroup lock stored after the guest LDS, as LDS is a 32-bit array without 64-bit atomics. On RDNA2 (gfx1036) the LDS serialises a 32-bit and a 64-bit atomic on the same address: 128 lanes of `ds_add_u32` and 128 lanes of `ds_add_u64` on one slot sum exactly. Only that pair was measured. Every 32-bit LDS atomic of a program that contains a 64-bit LDS atomic therefore takes the same lock (the other 32-bit LDS atomics, `ds_min/max`, `ds_cmpst`, the float forms, `ds_inc/dec`, `ds_mskor` and `ds_wrxchg`, are extrapolated from `ds_add_u32` and were not measured against a 64-bit atomic); a program without a 64-bit LDS atomic keeps its native 32-bit atomics. `ds_condxchg32_rtn_b64` is two `ds_mskor` halves, each under the lock but not atomic as a pair, so a 64-bit atomic between the two halves is not excluded. Sub-dword writes (`ds_write_b8`, `ds_write_b16` and their `_d16_hi` forms) and plain `ds_read_*`, `ds_write_*`, `ds_append_u32` and `ds_consume_u32` are not taken under the lock, so a plain access racing a 64-bit atomic on the same dwords can see or overwrite one half of it; whether the hardware orders them was not measured. The lock serialises the workgroup's LDS atomics of such a program; the throughput cost was not measured. `VK_KHR_workgroup_memory_explicit_layout` with `shaderSharedInt64Atomics` would make both widths native and is not used. A 64-bit atomic at an address that is 4-byte but not 8-byte aligned faults the wave on gfx1036, as on gfx1035. - [LDS float atomics](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (`ds_min/max/cmpst_f32`, `ds_min/max/cmpst_f64` and their `_rtn` forms) follow the hardware measured on RDNA2 (gfx1036) over 14 x 14 operand sets (zeros, ones, infinities, denormals, quiet and signalling NaNs of both signs and with payloads) in IEEE 0 and 1 with every value 0-3 of the type's denormal field: both operands are flushed to a signed zero when bit 0 of the field is clear (0 and 2; 1 and 3 keep them; the f32 field for the f32 forms, the f16/f64 field for the f64 forms; a shader without a known float mode flushes f32 and keeps f64), then a signalling-NaN data operand is stored quieted, else a signalling-NaN old value is stored quieted, else a quiet-NaN old value is replaced by the data, else a quiet-NaN data operand loses, else the ordered compare with -0 below +0 decides, and the flushed value is what is written, also when the old value wins. `cmpst` compares the flushed bits (both zeros equal, NaN never equal) and leaves memory unchanged when they differ. The returned value is always the old value as it was in memory. `IEEE_MODE` has no effect. The same bit rule serves the buffer float atomics (`AtomicFloatBits`). Not measured: a denormal or signalling-NaN `data1` of `cmpst` (stored as given), and `ds_add_f32`, which keeps the host's `OpFAdd`. - [V# bases off a DWORD boundary](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (shader recompiler) - raw accesses use the exact bytes and `s_buffer_load` reads `(base & ~3) + (offset & ~3)`, as measured on gfx1035 (Linux, alignment mode UNALIGNED; the console's mode is unknown). Formatted accesses off their element's alignment and atomics throw. - [32-bit bitfield masks](../../core/shader/recompiler/Translation/src/IntegerInstructions.cpp) (`S_BFM_B32`, `V_BFM_B32`) use the truncating 32-bit shift specified by the [RDNA 1.0 ISA](https://www.amd.com/content/dam/amd/en/documents/radeon-tech-docs/instruction-set-architectures/rdna-shader-instruction-set-architecture.pdf), sections 12.1 (opcode 36) and 12.12 (opcode 867). Width and offset use their low five bits. This matches gfx1035 (Linux) for every width/offset pair, with and without upper operand bits set, and both leave SCC unchanged; not measured on the console. From cffe4bf0706b3a2956b4efddf4fa03ae97654b53 Mon Sep 17 00:00:00 2001 From: Hussein Kadhim Date: Sat, 10 Oct 2026 13:41:17 +0300 Subject: [PATCH 217/460] docs(technical-debt): state the measured LDS atomic overlap as the slot's high dword --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 1e7aab2a6..83d36628b 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -454,7 +454,7 @@ Throughout the project, every function at every stage either **does exactly what - [Converted unorm images](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (10_11_11_UNORM, read and written through an R32_UINT view): samples, gathers, LOD queries, 16-bit data and a DST_SEL of the missing W channel throw. - [Packed 16-bit textures](../../core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp) (libSceAgcDriver) - `5_6_5`, `1_5_5_5` and `4_4_4_4` are viewed as R5G6B5, A1R5G5B5 and R4G4B4A4 with a swizzle that reads X from the low bits, and a DST_SEL of the missing W of `5_6_5` reads X. Measured on RDNA2 (gfx1035), not on the console: through `image_load` and `image_sample_lz`, X, Y, Z (and W of `1_5_5_5` and `4_4_4_4`) come from the low, middle and high fields, and W of `5_6_5` returns X (the low 5 bits), not 1. - [Converted unorm image loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) divide by 2^n - 1 with the buffer format path's `OpFDiv`, which Vulkan allows to be up to 2.5 ULP off, so some drivers are not bit-exact. -- [64-bit LDS atomics](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (`ds_*_u64`, `ds_*_b64`, `ds_*_f64`) read and write the two dwords under a workgroup lock stored after the guest LDS, as LDS is a 32-bit array without 64-bit atomics. On RDNA2 (gfx1036) the LDS serialises a 32-bit and a 64-bit atomic on the same address: 128 lanes of `ds_add_u32` and 128 lanes of `ds_add_u64` on one slot sum exactly. Only that pair was measured. Every 32-bit LDS atomic of a program that contains a 64-bit LDS atomic therefore takes the same lock (the other 32-bit LDS atomics, `ds_min/max`, `ds_cmpst`, the float forms, `ds_inc/dec`, `ds_mskor` and `ds_wrxchg`, are extrapolated from `ds_add_u32` and were not measured against a 64-bit atomic); a program without a 64-bit LDS atomic keeps its native 32-bit atomics. `ds_condxchg32_rtn_b64` is two `ds_mskor` halves, each under the lock but not atomic as a pair, so a 64-bit atomic between the two halves is not excluded. Sub-dword writes (`ds_write_b8`, `ds_write_b16` and their `_d16_hi` forms) and plain `ds_read_*`, `ds_write_*`, `ds_append_u32` and `ds_consume_u32` are not taken under the lock, so a plain access racing a 64-bit atomic on the same dwords can see or overwrite one half of it; whether the hardware orders them was not measured. The lock serialises the workgroup's LDS atomics of such a program; the throughput cost was not measured. `VK_KHR_workgroup_memory_explicit_layout` with `shaderSharedInt64Atomics` would make both widths native and is not used. A 64-bit atomic at an address that is 4-byte but not 8-byte aligned faults the wave on gfx1036, as on gfx1035. +- [64-bit LDS atomics](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (`ds_*_u64`, `ds_*_b64`, `ds_*_f64`) read and write the two dwords under a workgroup lock stored after the guest LDS, as LDS is a 32-bit array without 64-bit atomics. On RDNA2 (gfx1036) the LDS serialises a 32-bit atomic on a slot's high dword and a 64-bit atomic on that slot: 128 lanes of `ds_add_u32` and 128 lanes of `ds_add_u64` on one slot sum exactly. Only that pair was measured. Every 32-bit LDS atomic of a program that contains a 64-bit LDS atomic therefore takes the same lock (the other 32-bit LDS atomics, `ds_min/max`, `ds_cmpst`, the float forms, `ds_inc/dec`, `ds_mskor` and `ds_wrxchg`, are extrapolated from `ds_add_u32` and were not measured against a 64-bit atomic); a program without a 64-bit LDS atomic keeps its native 32-bit atomics. `ds_condxchg32_rtn_b64` is two `ds_mskor` halves, each under the lock but not atomic as a pair, so a 64-bit atomic between the two halves is not excluded. Sub-dword writes (`ds_write_b8`, `ds_write_b16` and their `_d16_hi` forms) and plain `ds_read_*`, `ds_write_*`, `ds_append_u32` and `ds_consume_u32` are not taken under the lock, so a plain access racing a 64-bit atomic on the same dwords can see or overwrite one half of it; whether the hardware orders them was not measured. The lock serialises the workgroup's LDS atomics of such a program; the throughput cost was not measured. `VK_KHR_workgroup_memory_explicit_layout` with `shaderSharedInt64Atomics` would make both widths native and is not used. A 64-bit atomic at an address that is 4-byte but not 8-byte aligned faults the wave on gfx1036, as on gfx1035. - [LDS float atomics](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (`ds_min/max/cmpst_f32`, `ds_min/max/cmpst_f64` and their `_rtn` forms) follow the hardware measured on RDNA2 (gfx1036) over 14 x 14 operand sets (zeros, ones, infinities, denormals, quiet and signalling NaNs of both signs and with payloads) in IEEE 0 and 1 with every value 0-3 of the type's denormal field: both operands are flushed to a signed zero when bit 0 of the field is clear (0 and 2; 1 and 3 keep them; the f32 field for the f32 forms, the f16/f64 field for the f64 forms; a shader without a known float mode flushes f32 and keeps f64), then a signalling-NaN data operand is stored quieted, else a signalling-NaN old value is stored quieted, else a quiet-NaN old value is replaced by the data, else a quiet-NaN data operand loses, else the ordered compare with -0 below +0 decides, and the flushed value is what is written, also when the old value wins. `cmpst` compares the flushed bits (both zeros equal, NaN never equal) and leaves memory unchanged when they differ. The returned value is always the old value as it was in memory. `IEEE_MODE` has no effect. The same bit rule serves the buffer float atomics (`AtomicFloatBits`). Not measured: a denormal or signalling-NaN `data1` of `cmpst` (stored as given), and `ds_add_f32`, which keeps the host's `OpFAdd`. - [V# bases off a DWORD boundary](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (shader recompiler) - raw accesses use the exact bytes and `s_buffer_load` reads `(base & ~3) + (offset & ~3)`, as measured on gfx1035 (Linux, alignment mode UNALIGNED; the console's mode is unknown). Formatted accesses off their element's alignment and atomics throw. - [32-bit bitfield masks](../../core/shader/recompiler/Translation/src/IntegerInstructions.cpp) (`S_BFM_B32`, `V_BFM_B32`) use the truncating 32-bit shift specified by the [RDNA 1.0 ISA](https://www.amd.com/content/dam/amd/en/documents/radeon-tech-docs/instruction-set-architectures/rdna-shader-instruction-set-architecture.pdf), sections 12.1 (opcode 36) and 12.12 (opcode 867). Width and offset use their low five bits. This matches gfx1035 (Linux) for every width/offset pair, with and without upper operand bits set, and both leave SCC unchanged; not measured on the console. From 22f02606e3eceb50316c9c67d3fa2cb70a30be4e Mon Sep 17 00:00:00 2001 From: Hussein Kadhim Date: Sat, 10 Oct 2026 15:08:20 +0300 Subject: [PATCH 218/460] test(agc-driver): unroll the LDS mixed-width atomic kernels so they validate with SPIR-V Tools enabled With ANYPS5_ENABLE_SPIRV_TOOLS=ON the scalar loop around the ds_add_u64 lock loop made SPIRV-Tools' block merge pass merge the loop header with its multi-block continue target, and validation after optimization rejected the module. The 16 iterations are now emitted inline; the expected sums and the divergent exec masks are unchanged. --- .../tests/execution/LdsMixedWidthAtomics.cpp | 122 ++++++++++-------- 1 file changed, 68 insertions(+), 54 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp index ca554eabd..78c2d0650 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp @@ -22,61 +22,75 @@ constexpr std::uint32_t LdsDwords = 4; alignas(256) std::array Output{}; alignas(256) std::array Unused{}; -alignas(256) constexpr auto UniformCode = std::to_array({ - 0x34100083, - 0x7e020280, - 0x7e0402c1, - 0x7e060280, - 0x7e080281, - 0x7e0c0280, - 0x7e0e0280, - 0xd9340000, 0x00000601, - 0xbf8cc07f, - 0xbf8a0000, - 0xbe800390 | (8u << 16u), - 0xd9000000, 0x00000201, - 0xd8000004, 0x00000401, - 0xbf8cc07f, - 0x80888108, - 0xbf078008, - 0xbf85fff8, - 0xbf8cc07f, - 0xbf8a0000, - 0xd9d80000, 0x0a000001, - 0xbf8cc07f, - 0xe0741000, 0x80010a08, - 0xbf810000, -}); +template +constexpr auto Unrolled(const std::array& head, const std::array& body, const std::array& tail) { + std::array code{}; + std::size_t next = 0; + for (const auto word : head) code[next++] = word; + for (std::uint32_t iteration = 0; iteration < Iterations; ++iteration) { + for (const auto word : body) code[next++] = word; + } + for (const auto word : tail) code[next++] = word; + return code; +} -alignas(256) constexpr auto DivergentCode = std::to_array({ - 0x34100083, - 0x7e020280, - 0x7e0402c1, - 0x7e060280, - 0x7e080281, - 0x7e0c0280, - 0x7e0e0280, - 0xd9340000, 0x00000601, - 0xbf8cc07f, - 0xbf8a0000, - 0xbe89037e, - 0xbe800390 | (8u << 16u), - 0xbefe03ff, 0x55555555, - 0xd9000000, 0x00000201, - 0xbefe03ff, 0xaaaaaaaa, - 0xd8000004, 0x00000401, - 0xbf8cc07f, - 0x80888108, - 0xbf078008, - 0xbf85fff4, - 0xbefe0309, - 0xbf8cc07f, - 0xbf8a0000, - 0xd9d80000, 0x0a000001, - 0xbf8cc07f, - 0xe0741000, 0x80010a08, - 0xbf810000, -}); +alignas(256) constexpr auto UniformCode = Unrolled( + std::to_array({ + 0x34100083, + 0x7e020280, + 0x7e0402c1, + 0x7e060280, + 0x7e080281, + 0x7e0c0280, + 0x7e0e0280, + 0xd9340000, 0x00000601, + 0xbf8cc07f, + 0xbf8a0000, + }), + std::to_array({ + 0xd9000000, 0x00000201, + 0xd8000004, 0x00000401, + 0xbf8cc07f, + }), + std::to_array({ + 0xbf8cc07f, + 0xbf8a0000, + 0xd9d80000, 0x0a000001, + 0xbf8cc07f, + 0xe0741000, 0x80010a08, + 0xbf810000, + })); + +alignas(256) constexpr auto DivergentCode = Unrolled( + std::to_array({ + 0x34100083, + 0x7e020280, + 0x7e0402c1, + 0x7e060280, + 0x7e080281, + 0x7e0c0280, + 0x7e0e0280, + 0xd9340000, 0x00000601, + 0xbf8cc07f, + 0xbf8a0000, + 0xbe89037e, + }), + std::to_array({ + 0xbefe03ff, 0x55555555, + 0xd9000000, 0x00000201, + 0xbefe03ff, 0xaaaaaaaa, + 0xd8000004, 0x00000401, + 0xbf8cc07f, + }), + std::to_array({ + 0xbefe0309, + 0xbf8cc07f, + 0xbf8a0000, + 0xd9d80000, 0x0a000001, + 0xbf8cc07f, + 0xe0741000, 0x80010a08, + 0xbf810000, + })); std::array BufferDescriptor(const void* data, std::uint32_t bytes) { const auto address = reinterpret_cast(data); From 5253c499dc905392e1d46a52fffc946545a00741 Mon Sep 17 00:00:00 2001 From: Hussein Kadhim Date: Sat, 10 Oct 2026 16:06:22 +0300 Subject: [PATCH 219/460] test(agc-driver): add the issue's split-lane kernel to the LDS mixed-width atomic test Even lanes run ds_add_u64 and odd lanes run ds_add_u32 on the slot's high dword, one pass per dispatch, as in the reported kernel, and the dispatch is repeated 32 times from one compiled shader. --- .../tests/execution/LdsMixedWidthAtomics.cpp | 75 ++++++++++++++----- 1 file changed, 55 insertions(+), 20 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp index 78c2d0650..a13bc66e2 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp @@ -18,23 +18,24 @@ using ShaderRecompiler::ShaderStage; constexpr std::uint32_t Threads = 256; constexpr std::uint32_t Iterations = 16; +constexpr std::uint32_t Repeats = 32; constexpr std::uint32_t LdsDwords = 4; alignas(256) std::array Output{}; alignas(256) std::array Unused{}; -template +template constexpr auto Unrolled(const std::array& head, const std::array& body, const std::array& tail) { - std::array code{}; + std::array code{}; std::size_t next = 0; for (const auto word : head) code[next++] = word; - for (std::uint32_t iteration = 0; iteration < Iterations; ++iteration) { + for (std::uint32_t iteration = 0; iteration < Count; ++iteration) { for (const auto word : body) code[next++] = word; } for (const auto word : tail) code[next++] = word; return code; } -alignas(256) constexpr auto UniformCode = Unrolled( +alignas(256) constexpr auto UniformCode = Unrolled( std::to_array({ 0x34100083, 0x7e020280, @@ -61,7 +62,7 @@ alignas(256) constexpr auto UniformCode = Unrolled( 0xbf810000, })); -alignas(256) constexpr auto DivergentCode = Unrolled( +alignas(256) constexpr auto DivergentCode = Unrolled( std::to_array({ 0x34100083, 0x7e020280, @@ -92,6 +93,38 @@ alignas(256) constexpr auto DivergentCode = Unrolled( 0xbf810000, })); +alignas(256) constexpr auto ParityCode = Unrolled<1>( + std::to_array({ + 0x34100083, + 0x7e020280, + 0x7e0402c1, + 0x7e060280, + 0x7e080281, + 0x7e0c0280, + 0x7e0e0280, + 0xd9340000, 0x00000601, + 0xbf8cc07f, + 0xbf8a0000, + 0xbe89037e, + }), + std::to_array({ + 0xbefe03ff, 0x55555555, + 0xd9000000, 0x00000201, + 0xbf8cc07f, + 0xbefe03ff, 0xaaaaaaaa, + 0xd8000004, 0x00000401, + 0xbf8cc07f, + }), + std::to_array({ + 0xbefe0309, + 0xbf8cc07f, + 0xbf8a0000, + 0xd9d80000, 0x0a000001, + 0xbf8cc07f, + 0xe0741000, 0x80010a08, + 0xbf810000, + })); + std::array BufferDescriptor(const void* data, std::uint32_t bytes) { const auto address = reinterpret_cast(data); return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; @@ -103,8 +136,15 @@ std::string Hex(std::uint64_t value) { return text; } -void Run(AgcDriver::VulkanDevice& device, std::span code) { - Output.fill(0xdeadbeefu); +void Check(const char* name, std::uint64_t operations, std::uint32_t run) { + const std::uint64_t expected = operations * 0xffffffffull + (operations << 32u); + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + const std::uint64_t actual = Output[tid * 2u] | (static_cast(Output[tid * 2u + 1u]) << 32u); + Require(actual == expected, std::string("lds mixed width atomics ") + name + " run " + std::to_string(run) + ": lane " + std::to_string(tid) + " read " + Hex(actual) + ", expected " + Hex(expected) + " after " + std::to_string(operations) + " ds_add_u64 and " + std::to_string(operations) + " ds_add_u32 on one slot"); + } +} + +void Run(AgcDriver::VulkanDevice& device, std::span code, const char* name, std::uint64_t operations, std::uint32_t repeats) { std::vector userData(8, 0u); const auto input = BufferDescriptor(Unused.data(), static_cast(Unused.size() * 4u)); const auto output = BufferDescriptor(Output.data(), static_cast(Output.size() * 4u)); @@ -120,15 +160,11 @@ void Run(AgcDriver::VulkanDevice& device, std::span code) { }; request.useCache = false; const auto result = ShaderRecompiler::Recompile(request); - device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); - device.WaitIdle(); -} - -void Check(const char* name, std::uint64_t operations) { - const std::uint64_t expected = operations * 0xffffffffull + (operations << 32u); - for (std::uint32_t tid = 0; tid < Threads; ++tid) { - const std::uint64_t actual = Output[tid * 2u] | (static_cast(Output[tid * 2u + 1u]) << 32u); - Require(actual == expected, std::string("lds mixed width atomics ") + name + ": lane " + std::to_string(tid) + " read " + Hex(actual) + ", expected " + Hex(expected) + " after " + std::to_string(operations) + " ds_add_u64 and ds_add_u32 on one slot"); + for (std::uint32_t run = 0; run < repeats; ++run) { + Output.fill(0xdeadbeefu); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); + Check(name, operations, run); } } @@ -138,10 +174,9 @@ int main() { try { const auto device = OpenVulkanTestDevice(); if (!device) return VulkanTestSkipped; - Run(*device, UniformCode); - Check("uniform", static_cast(Threads) * Iterations); - Run(*device, DivergentCode); - Check("divergent", static_cast(Threads) / 2u * Iterations); + Run(*device, UniformCode, "uniform", static_cast(Threads) * Iterations, 1); + Run(*device, DivergentCode, "divergent", static_cast(Threads) / 2u * Iterations, 1); + Run(*device, ParityCode, "split lanes", Threads / 2u, Repeats); std::puts("lds mixed width atomics tests passed"); return 0; } catch (const std::exception& error) { From 60301adc6dfbcb2a58efb6cd993fdd5a3087f80f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jan=20=C5=A0l=C3=A9gl?= Date: Sat, 10 Oct 2026 15:30:31 +0200 Subject: [PATCH 220/460] test(agc-driver): add a split-lane case with per-lane operands to the LDS mixed-width atomic test On RADV the race between ds_add_u64 under the workgroup lock and a concurrent ds_add_u32 only shows when the lanes get their operands from per-lane buffer loads: with v_mov_b32 constants or loads from one address no increment is lost there. The new case has the split-lane kernel's parity split and loads each lane's (0xffffffff, 0, 1, 0) row from the input buffer at tid * 16 with buffer_load_dwordx4, so the input buffer now holds 256 rows. With SpirvMemoryInstructions.cpp reverted it fails on RADV and on the RTX 3080; with the change it passes on both. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Fnrk5Dazjb2tLooVSpdRQq --- .../tests/execution/LdsMixedWidthAtomics.cpp | 43 ++++++++++++++++++- 1 file changed, 41 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp index a13bc66e2..6528e8af9 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/LdsMixedWidthAtomics.cpp @@ -21,7 +21,14 @@ constexpr std::uint32_t Iterations = 16; constexpr std::uint32_t Repeats = 32; constexpr std::uint32_t LdsDwords = 4; alignas(256) std::array Output{}; -alignas(256) std::array Unused{}; +alignas(256) std::array Input = [] { + std::array rows{}; + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + rows[tid * 4u] = 0xffffffffu; + rows[tid * 4u + 2u] = 1u; + } + return rows; +}(); template constexpr auto Unrolled(const std::array& head, const std::array& body, const std::array& tail) { @@ -125,6 +132,37 @@ alignas(256) constexpr auto ParityCode = Unrolled<1>( 0xbf810000, })); +alignas(256) constexpr auto LoadedParityCode = Unrolled<1>( + std::to_array({ + 0x34100083, + 0x340a0084, + 0x7e020280, + 0xe0381000, 0x80000c05, + 0x7e0c0280, + 0x7e0e0280, + 0xd9340000, 0x00000601, + 0xbf8c0070, + 0xbf8a0000, + 0xbe89037e, + }), + std::to_array({ + 0xbefe03ff, 0x55555555, + 0xd9000000, 0x00000c01, + 0xbf8cc07f, + 0xbefe03ff, 0xaaaaaaaa, + 0xd8000004, 0x00000e01, + 0xbf8cc07f, + }), + std::to_array({ + 0xbefe0309, + 0xbf8cc07f, + 0xbf8a0000, + 0xd9d80000, 0x0a000001, + 0xbf8cc07f, + 0xe0741000, 0x80010a08, + 0xbf810000, + })); + std::array BufferDescriptor(const void* data, std::uint32_t bytes) { const auto address = reinterpret_cast(data); return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; @@ -146,7 +184,7 @@ void Check(const char* name, std::uint64_t operations, std::uint32_t run) { void Run(AgcDriver::VulkanDevice& device, std::span code, const char* name, std::uint64_t operations, std::uint32_t repeats) { std::vector userData(8, 0u); - const auto input = BufferDescriptor(Unused.data(), static_cast(Unused.size() * 4u)); + const auto input = BufferDescriptor(Input.data(), static_cast(Input.size() * 4u)); const auto output = BufferDescriptor(Output.data(), static_cast(Output.size() * 4u)); std::copy(input.begin(), input.end(), userData.begin()); std::copy(output.begin(), output.end(), userData.begin() + 4); @@ -177,6 +215,7 @@ int main() { Run(*device, UniformCode, "uniform", static_cast(Threads) * Iterations, 1); Run(*device, DivergentCode, "divergent", static_cast(Threads) / 2u * Iterations, 1); Run(*device, ParityCode, "split lanes", Threads / 2u, Repeats); + Run(*device, LoadedParityCode, "split lanes with per-lane operands", Threads / 2u, Repeats); std::puts("lds mixed width atomics tests passed"); return 0; } catch (const std::exception& error) { From c9e1fa0ce19bc12aa4d5c0317734991651402f9c Mon Sep 17 00:00:00 2001 From: Hussein Kadhim Date: Sun, 11 Oct 2026 01:43:29 +0300 Subject: [PATCH 221/460] fix(shader): 32-bit flat atomics in the LDS aperture take the workgroup lock A flat_atomic_* whose address lands in the LDS aperture updated the dword without the lock, so it could lose updates against a ds_add_u64 or a 64-bit flat atomic on the same dwords. It takes the lock like the 32-bit ds atomics when the program has a 64-bit LDS atomic; programs without one emit the same SPIR-V as before. --- .../tests/SpirvLdsAtomicLock.cpp | 18 ++++++++++++++++-- .../SpirvMemory/SpirvMemoryInstructions.cpp | 3 ++- docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 19 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/SpirvLdsAtomicLock.cpp b/core/libs/prx/libSceAgcDriver/tests/SpirvLdsAtomicLock.cpp index 7f25bf955..977d01d44 100644 --- a/core/libs/prx/libSceAgcDriver/tests/SpirvLdsAtomicLock.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/SpirvLdsAtomicLock.cpp @@ -19,6 +19,8 @@ constexpr std::uint32_t DsAddU32Hi = 0xd8000004u; constexpr std::uint32_t DsIncU32Hi = 0xd80c0004u; constexpr std::uint32_t DsOperands = 0x00000201u; constexpr std::uint32_t DsOperand32 = 0x00000401u; +constexpr std::uint32_t FlatAtomicAddU32 = 0xdcc90000u; +constexpr std::uint32_t FlatAtomicOperands = 0x0a7d0402u; struct Usage { std::size_t elects = 0; @@ -33,9 +35,9 @@ void Require(bool condition, const std::string& message) { } std::vector RecompileProgram(std::span code) { - const std::array capabilities{spv::CapabilityShader, spv::CapabilityGroupNonUniform, spv::CapabilityGroupNonUniformBallot, spv::CapabilityGroupNonUniformShuffle}; + const std::array capabilities{spv::CapabilityShader, spv::CapabilityGroupNonUniform, spv::CapabilityGroupNonUniformBallot, spv::CapabilityGroupNonUniformShuffle, spv::CapabilityInt64, spv::CapabilityPhysicalStorageBufferAddresses, spv::CapabilityStorageBuffer8BitAccess}; const std::array userData{}; - const std::array extensions{}; + const std::array extensions{"SPV_KHR_physical_storage_buffer", "SPV_KHR_8bit_storage"}; RecompileRequest request{}; request.shader = {ShaderStage::Compute, 0x20000u, code, 0, {}}; request.context.waveSize = 32; @@ -106,6 +108,16 @@ void CheckOtherUpdatePathTakesLock() { Require(usage.elects == 2 && usage.exchanges >= 2 && usage.releases == 2, "ds_inc_u32 in a program with ds_add_u64 does not take the workgroup lock: elect " + std::to_string(usage.elects) + ", lock release " + std::to_string(usage.releases) + ", expected 2 each"); } +void CheckFlatAtomicTakesLockWith64Bit() { + const auto usage = Compile({DsAddU64, DsOperands, FlatAtomicAddU32, FlatAtomicOperands}); + Require(usage.elects == 2, "a 32-bit flat atomic into the LDS aperture in a program with ds_add_u64 does not take the workgroup lock: elect " + std::to_string(usage.elects) + ", expected 2"); +} + +void CheckFlatAtomicOnlyIsUnchanged() { + const auto usage = Compile({FlatAtomicAddU32, FlatAtomicOperands}); + Require(usage.elects == 0, "a program with only a 32-bit flat atomic got a workgroup lock: elect " + std::to_string(usage.elects)); +} + void Check32BitOnlyIsUnchanged() { const auto add = Compile({DsAddU32Hi, DsOperand32}); Require(add.elects == 0 && add.exchanges == 0 && add.releases == 0 && add.adds == 1, "a program with only ds_add_u32 got a workgroup lock: elect " + std::to_string(add.elects) + ", lock acquire " + std::to_string(add.exchanges) + ", lock release " + std::to_string(add.releases) + ", native add " + std::to_string(add.adds)); @@ -120,6 +132,8 @@ int main() { Check64BitOnlyLocks(); Check32BitTakesLockWith64Bit(); CheckOtherUpdatePathTakesLock(); + CheckFlatAtomicTakesLockWith64Bit(); + CheckFlatAtomicOnlyIsUnchanged(); Check32BitOnlyIsUnchanged(); } catch (const std::exception& error) { std::fprintf(stderr, "SpirvLdsAtomicLock: %s\n", error.what()); diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp index b26ab160b..2515e574c 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp @@ -974,7 +974,8 @@ std::uint32_t ApertureAtomic32(SpirvValueEmitContext& ctx, const IrValue& inst, return EmitValueOrZeroIfCondition(state, EmitMemoryElementInBounds(state, resource, index), [&] { const auto pointer = EmitMemoryElementPointer(state, resource, index); if (kind == ResourceKind::Lds) { - return AtomicUpdate(state, pointer, kind, [&](std::uint32_t current) { return FlatAtomicNext32(state, opcode, current, value, comparator); }); + const auto update = [&]() { return AtomicUpdate(state, pointer, kind, [&](std::uint32_t current) { return FlatAtomicNext32(state, opcode, current, value, comparator); }); }; + return state.requirements.ldsLock ? LockedLdsUpdate(state, TypeU32(state), ConstantU32(state, 0u), update) : update(); } const auto old = state.module.AllocateId(); state.module.AddFunction(spv::OpLoad, TypeU32(state), old, pointer); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 83d36628b..cce0df2ac 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -454,7 +454,7 @@ Throughout the project, every function at every stage either **does exactly what - [Converted unorm images](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (10_11_11_UNORM, read and written through an R32_UINT view): samples, gathers, LOD queries, 16-bit data and a DST_SEL of the missing W channel throw. - [Packed 16-bit textures](../../core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp) (libSceAgcDriver) - `5_6_5`, `1_5_5_5` and `4_4_4_4` are viewed as R5G6B5, A1R5G5B5 and R4G4B4A4 with a swizzle that reads X from the low bits, and a DST_SEL of the missing W of `5_6_5` reads X. Measured on RDNA2 (gfx1035), not on the console: through `image_load` and `image_sample_lz`, X, Y, Z (and W of `1_5_5_5` and `4_4_4_4`) come from the low, middle and high fields, and W of `5_6_5` returns X (the low 5 bits), not 1. - [Converted unorm image loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) divide by 2^n - 1 with the buffer format path's `OpFDiv`, which Vulkan allows to be up to 2.5 ULP off, so some drivers are not bit-exact. -- [64-bit LDS atomics](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (`ds_*_u64`, `ds_*_b64`, `ds_*_f64`) read and write the two dwords under a workgroup lock stored after the guest LDS, as LDS is a 32-bit array without 64-bit atomics. On RDNA2 (gfx1036) the LDS serialises a 32-bit atomic on a slot's high dword and a 64-bit atomic on that slot: 128 lanes of `ds_add_u32` and 128 lanes of `ds_add_u64` on one slot sum exactly. Only that pair was measured. Every 32-bit LDS atomic of a program that contains a 64-bit LDS atomic therefore takes the same lock (the other 32-bit LDS atomics, `ds_min/max`, `ds_cmpst`, the float forms, `ds_inc/dec`, `ds_mskor` and `ds_wrxchg`, are extrapolated from `ds_add_u32` and were not measured against a 64-bit atomic); a program without a 64-bit LDS atomic keeps its native 32-bit atomics. `ds_condxchg32_rtn_b64` is two `ds_mskor` halves, each under the lock but not atomic as a pair, so a 64-bit atomic between the two halves is not excluded. Sub-dword writes (`ds_write_b8`, `ds_write_b16` and their `_d16_hi` forms) and plain `ds_read_*`, `ds_write_*`, `ds_append_u32` and `ds_consume_u32` are not taken under the lock, so a plain access racing a 64-bit atomic on the same dwords can see or overwrite one half of it; whether the hardware orders them was not measured. The lock serialises the workgroup's LDS atomics of such a program; the throughput cost was not measured. `VK_KHR_workgroup_memory_explicit_layout` with `shaderSharedInt64Atomics` would make both widths native and is not used. A 64-bit atomic at an address that is 4-byte but not 8-byte aligned faults the wave on gfx1036, as on gfx1035. +- [64-bit LDS atomics](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (`ds_*_u64`, `ds_*_b64`, `ds_*_f64`) read and write the two dwords under a workgroup lock stored after the guest LDS, as LDS is a 32-bit array without 64-bit atomics. On RDNA2 (gfx1036) the LDS serialises a 32-bit atomic on a slot's high dword and a 64-bit atomic on that slot: 128 lanes of `ds_add_u32` and 128 lanes of `ds_add_u64` on one slot sum exactly. Only that pair was measured. Every 32-bit LDS atomic of a program that contains a 64-bit LDS atomic therefore takes the same lock, including a 32-bit `flat_atomic_*` whose address lands in the LDS aperture (a 64-bit one takes it already); the flat forms were not measured against a 64-bit atomic either (the other 32-bit LDS atomics, `ds_min/max`, `ds_cmpst`, the float forms, `ds_inc/dec`, `ds_mskor` and `ds_wrxchg`, are extrapolated from `ds_add_u32` and were not measured against a 64-bit atomic); a program without a 64-bit LDS atomic keeps its native 32-bit atomics. `ds_condxchg32_rtn_b64` is two `ds_mskor` halves, each under the lock but not atomic as a pair, so a 64-bit atomic between the two halves is not excluded. Sub-dword writes (`ds_write_b8`, `ds_write_b16` and their `_d16_hi` forms) and plain `ds_read_*`, `ds_write_*`, `ds_append_u32` and `ds_consume_u32` are not taken under the lock, so a plain access racing a 64-bit atomic on the same dwords can see or overwrite one half of it; whether the hardware orders them was not measured. The lock serialises the workgroup's LDS atomics of such a program; the throughput cost was not measured. `VK_KHR_workgroup_memory_explicit_layout` with `shaderSharedInt64Atomics` would make both widths native and is not used. A 64-bit atomic at an address that is 4-byte but not 8-byte aligned faults the wave on gfx1036, as on gfx1035. - [LDS float atomics](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (`ds_min/max/cmpst_f32`, `ds_min/max/cmpst_f64` and their `_rtn` forms) follow the hardware measured on RDNA2 (gfx1036) over 14 x 14 operand sets (zeros, ones, infinities, denormals, quiet and signalling NaNs of both signs and with payloads) in IEEE 0 and 1 with every value 0-3 of the type's denormal field: both operands are flushed to a signed zero when bit 0 of the field is clear (0 and 2; 1 and 3 keep them; the f32 field for the f32 forms, the f16/f64 field for the f64 forms; a shader without a known float mode flushes f32 and keeps f64), then a signalling-NaN data operand is stored quieted, else a signalling-NaN old value is stored quieted, else a quiet-NaN old value is replaced by the data, else a quiet-NaN data operand loses, else the ordered compare with -0 below +0 decides, and the flushed value is what is written, also when the old value wins. `cmpst` compares the flushed bits (both zeros equal, NaN never equal) and leaves memory unchanged when they differ. The returned value is always the old value as it was in memory. `IEEE_MODE` has no effect. The same bit rule serves the buffer float atomics (`AtomicFloatBits`). Not measured: a denormal or signalling-NaN `data1` of `cmpst` (stored as given), and `ds_add_f32`, which keeps the host's `OpFAdd`. - [V# bases off a DWORD boundary](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (shader recompiler) - raw accesses use the exact bytes and `s_buffer_load` reads `(base & ~3) + (offset & ~3)`, as measured on gfx1035 (Linux, alignment mode UNALIGNED; the console's mode is unknown). Formatted accesses off their element's alignment and atomics throw. - [32-bit bitfield masks](../../core/shader/recompiler/Translation/src/IntegerInstructions.cpp) (`S_BFM_B32`, `V_BFM_B32`) use the truncating 32-bit shift specified by the [RDNA 1.0 ISA](https://www.amd.com/content/dam/amd/en/documents/radeon-tech-docs/instruction-set-architectures/rdna-shader-instruction-set-architecture.pdf), sections 12.1 (opcode 36) and 12.12 (opcode 867). Width and offset use their low five bits. This matches gfx1035 (Linux) for every width/offset pair, with and without upper operand bits set, and both leave SCC unchanged; not measured on the console. From 2a92eaa35192f2624e984244e9b1e25c27cd632e Mon Sep 17 00:00:00 2001 From: Abdulrahman Elkordy <196347165+Silverash-i8@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:18:11 +0300 Subject: [PATCH 222/460] feat(libs): run the G2P and VR setup dialogs as common dialogs without UI libSceCdlgG2P and libSceVrSetupDialog threw from every export. They now run the common-dialog state machine libSceLoginDialog uses: Initialize, Open, the status calls, GetResult, Close and Terminate follow it, the dialog finishes at Open without showing anything, and GetResult reports that the user cancelled, writing only the leading int32 result. The signatures are assumed from the other common dialogs, as TechnicalDebt records. 13 exports move from stubs to implemented. Claude-Session: https://claude.ai/code/session_012SLQbhAiexxW6TtjYToW8a (cherry picked from commit 07992adf80bae9b2d7da4b751cd21c040fa1c5b8) --- core/libs/prx/libSceCdlgG2P/CMakeLists.txt | 14 ++++ core/libs/prx/libSceCdlgG2P/Export.cpp | 80 +++++++++++++------ .../prx/libSceVrSetupDialog/CMakeLists.txt | 14 ++++ core/libs/prx/libSceVrSetupDialog/Export.cpp | 72 ++++++++++++----- core/libs/tests/GuestG2PDialog.cpp | 54 +++++++++++++ core/libs/tests/GuestVrSetupDialog.cpp | 52 ++++++++++++ docs/dev/TechnicalDebt.md | 6 +- 7 files changed, 242 insertions(+), 50 deletions(-) create mode 100644 core/libs/prx/libSceCdlgG2P/CMakeLists.txt create mode 100644 core/libs/prx/libSceVrSetupDialog/CMakeLists.txt create mode 100644 core/libs/tests/GuestG2PDialog.cpp create mode 100644 core/libs/tests/GuestVrSetupDialog.cpp diff --git a/core/libs/prx/libSceCdlgG2P/CMakeLists.txt b/core/libs/prx/libSceCdlgG2P/CMakeLists.txt new file mode 100644 index 000000000..9809109f6 --- /dev/null +++ b/core/libs/prx/libSceCdlgG2P/CMakeLists.txt @@ -0,0 +1,14 @@ +add_library(libSceCdlgG2P SHARED EXCLUDE_FROM_ALL Export.cpp) +target_include_directories(libSceCdlgG2P PRIVATE ${LIBS_INCLUDE_DIR}) +configure_windows_unwind(libSceCdlgG2P) + +if(BUILD_TESTING) + add_test_executable(guest_g2p_dialog_tests ${LIBS_INCLUDE_DIR}/tests/GuestG2PDialog.cpp) + target_include_directories(guest_g2p_dialog_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_g2p_dialog_tests PRIVATE libSceCdlgG2P) + add_test(NAME guest_g2p_dialog COMMAND guest_g2p_dialog_tests) + if(WIN32) + set_tests_properties(guest_g2p_dialog PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() +endif() diff --git a/core/libs/prx/libSceCdlgG2P/Export.cpp b/core/libs/prx/libSceCdlgG2P/Export.cpp index 7ad726f79..d772b2900 100644 --- a/core/libs/prx/libSceCdlgG2P/Export.cpp +++ b/core/libs/prx/libSceCdlgG2P/Export.cpp @@ -1,40 +1,70 @@ +#include +#include +#include +#include #include "SceTypes.hpp" #include "prx/libc/include/General.hpp" +namespace { + +constexpr int COMMON_DIALOG_STATUS_NONE = 0; +constexpr int COMMON_DIALOG_STATUS_INITIALIZED = 1; +constexpr int COMMON_DIALOG_STATUS_RUNNING = 2; +constexpr int COMMON_DIALOG_STATUS_FINISHED = 3; +constexpr int COMMON_DIALOG_ERROR_NOT_INITIALIZED = static_cast(0x80B80003u); +constexpr int COMMON_DIALOG_ERROR_ALREADY_INITIALIZED = static_cast(0x80B80004u); +constexpr int COMMON_DIALOG_ERROR_NOT_FINISHED = static_cast(0x80B80005u); +constexpr int COMMON_DIALOG_ERROR_BUSY = static_cast(0x80B80007u); +constexpr int COMMON_DIALOG_ERROR_ARG_NULL = static_cast(0x80B8000Du); +constexpr int COMMON_DIALOG_RESULT_USER_CANCELED = 1; + +std::atomic g_status{COMMON_DIALOG_STATUS_NONE}; + +} + extern "C" { int APS5_VABI sceG2PDialogInitialize(void) { - NotImplemented_nid_no_patch(__func__); - return 0; + int expected = COMMON_DIALOG_STATUS_NONE; + if (!g_status.compare_exchange_strong(expected, COMMON_DIALOG_STATUS_INITIALIZED)) return COMMON_DIALOG_ERROR_ALREADY_INITIALIZED; + return 0; } -int APS5_VABI sceG2PDialogTerminate(void) { - NotImplemented_nid_no_patch(__func__); - return 0; -} - -int APS5_VABI sceG2PDialogOpen(void) { - NotImplemented_nid_no_patch(__func__); - return 0; -} - -int APS5_VABI sceG2PDialogClose(void) { - NotImplemented_nid_no_patch(__func__); - return 0; -} - -int APS5_VABI sceG2PDialogGetStatus(void) { - NotImplemented_nid_no_patch(__func__); - return 0; +int APS5_VABI sceG2PDialogOpen(const void* param) { + const int status = g_status.load(); + if (status == COMMON_DIALOG_STATUS_NONE) return COMMON_DIALOG_ERROR_NOT_INITIALIZED; + if (status == COMMON_DIALOG_STATUS_RUNNING) return COMMON_DIALOG_ERROR_BUSY; + if (param == nullptr) return COMMON_DIALOG_ERROR_ARG_NULL; + g_status = COMMON_DIALOG_STATUS_FINISHED; + return 0; } int APS5_VABI sceG2PDialogUpdateStatus(void) { - NotImplemented_nid_no_patch(__func__); - return 0; + return g_status.load(); } -int APS5_VABI sceG2PDialogGetResult(void) { - NotImplemented_nid_no_patch(__func__); - return 0; +int APS5_VABI sceG2PDialogGetStatus(void) { + return g_status.load(); } + +int APS5_VABI sceG2PDialogGetResult(void* result) { + const int status = g_status.load(); + if (status == COMMON_DIALOG_STATUS_NONE) return COMMON_DIALOG_ERROR_NOT_INITIALIZED; + if (result == nullptr) return COMMON_DIALOG_ERROR_ARG_NULL; + if (status != COMMON_DIALOG_STATUS_FINISHED) return COMMON_DIALOG_ERROR_NOT_FINISHED; + const std::int32_t canceled = COMMON_DIALOG_RESULT_USER_CANCELED; + std::memcpy(result, &canceled, sizeof(canceled)); + return 0; +} + +int APS5_VABI sceG2PDialogClose(void) { + if (g_status.load() == COMMON_DIALOG_STATUS_NONE) return COMMON_DIALOG_ERROR_NOT_INITIALIZED; + return 0; +} + +int APS5_VABI sceG2PDialogTerminate(void) { + if (g_status.exchange(COMMON_DIALOG_STATUS_NONE) == COMMON_DIALOG_STATUS_NONE) return COMMON_DIALOG_ERROR_NOT_INITIALIZED; + return 0; +} + } diff --git a/core/libs/prx/libSceVrSetupDialog/CMakeLists.txt b/core/libs/prx/libSceVrSetupDialog/CMakeLists.txt new file mode 100644 index 000000000..1f0ea1700 --- /dev/null +++ b/core/libs/prx/libSceVrSetupDialog/CMakeLists.txt @@ -0,0 +1,14 @@ +add_library(libSceVrSetupDialog SHARED EXCLUDE_FROM_ALL Export.cpp) +target_include_directories(libSceVrSetupDialog PRIVATE ${LIBS_INCLUDE_DIR}) +configure_windows_unwind(libSceVrSetupDialog) + +if(BUILD_TESTING) + add_test_executable(guest_vr_setup_dialog_tests ${LIBS_INCLUDE_DIR}/tests/GuestVrSetupDialog.cpp) + target_include_directories(guest_vr_setup_dialog_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_vr_setup_dialog_tests PRIVATE libSceVrSetupDialog) + add_test(NAME guest_vr_setup_dialog COMMAND guest_vr_setup_dialog_tests) + if(WIN32) + set_tests_properties(guest_vr_setup_dialog PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() +endif() diff --git a/core/libs/prx/libSceVrSetupDialog/Export.cpp b/core/libs/prx/libSceVrSetupDialog/Export.cpp index 5c9a5da6a..128cbcd16 100644 --- a/core/libs/prx/libSceVrSetupDialog/Export.cpp +++ b/core/libs/prx/libSceVrSetupDialog/Export.cpp @@ -1,38 +1,66 @@ +#include #include #include +#include #include "SceTypes.hpp" #include "prx/libc/include/General.hpp" +namespace { + +constexpr int COMMON_DIALOG_STATUS_NONE = 0; +constexpr int COMMON_DIALOG_STATUS_INITIALIZED = 1; +constexpr int COMMON_DIALOG_STATUS_RUNNING = 2; +constexpr int COMMON_DIALOG_STATUS_FINISHED = 3; +constexpr int COMMON_DIALOG_ERROR_NOT_INITIALIZED = static_cast(0x80B80003u); +constexpr int COMMON_DIALOG_ERROR_ALREADY_INITIALIZED = static_cast(0x80B80004u); +constexpr int COMMON_DIALOG_ERROR_NOT_FINISHED = static_cast(0x80B80005u); +constexpr int COMMON_DIALOG_ERROR_BUSY = static_cast(0x80B80007u); +constexpr int COMMON_DIALOG_ERROR_ARG_NULL = static_cast(0x80B8000Du); +constexpr int COMMON_DIALOG_RESULT_USER_CANCELED = 1; + +std::atomic g_status{COMMON_DIALOG_STATUS_NONE}; + +} + extern "C" { -int APS5_VABI sceVrSetupDialogClose(void) { - NotImplemented_nid_no_patch(__func__); - return 0; -} - -int APS5_VABI sceVrSetupDialogGetResult(void) { - NotImplemented_nid_no_patch(__func__); - return 0; -} - int APS5_VABI sceVrSetupDialogInitialize(void) { - NotImplemented_nid_no_patch(__func__); - return 0; + int expected = COMMON_DIALOG_STATUS_NONE; + if (!g_status.compare_exchange_strong(expected, COMMON_DIALOG_STATUS_INITIALIZED)) return COMMON_DIALOG_ERROR_ALREADY_INITIALIZED; + return 0; } -int APS5_VABI sceVrSetupDialogOpen(void) { - NotImplemented_nid_no_patch(__func__); - return 0; -} - -int APS5_VABI sceVrSetupDialogTerminate(void) { - NotImplemented_nid_no_patch(__func__); - return 0; +int APS5_VABI sceVrSetupDialogOpen(const void* param) { + const int status = g_status.load(); + if (status == COMMON_DIALOG_STATUS_NONE) return COMMON_DIALOG_ERROR_NOT_INITIALIZED; + if (status == COMMON_DIALOG_STATUS_RUNNING) return COMMON_DIALOG_ERROR_BUSY; + if (param == nullptr) return COMMON_DIALOG_ERROR_ARG_NULL; + g_status = COMMON_DIALOG_STATUS_FINISHED; + return 0; } int APS5_VABI sceVrSetupDialogUpdateStatus(void) { - NotImplemented_nid_no_patch(__func__); - return 0; + return g_status.load(); +} + +int APS5_VABI sceVrSetupDialogGetResult(void* result) { + const int status = g_status.load(); + if (status == COMMON_DIALOG_STATUS_NONE) return COMMON_DIALOG_ERROR_NOT_INITIALIZED; + if (result == nullptr) return COMMON_DIALOG_ERROR_ARG_NULL; + if (status != COMMON_DIALOG_STATUS_FINISHED) return COMMON_DIALOG_ERROR_NOT_FINISHED; + const std::int32_t canceled = COMMON_DIALOG_RESULT_USER_CANCELED; + std::memcpy(result, &canceled, sizeof(canceled)); + return 0; +} + +int APS5_VABI sceVrSetupDialogClose(void) { + if (g_status.load() == COMMON_DIALOG_STATUS_NONE) return COMMON_DIALOG_ERROR_NOT_INITIALIZED; + return 0; +} + +int APS5_VABI sceVrSetupDialogTerminate(void) { + if (g_status.exchange(COMMON_DIALOG_STATUS_NONE) == COMMON_DIALOG_STATUS_NONE) return COMMON_DIALOG_ERROR_NOT_INITIALIZED; + return 0; } } diff --git a/core/libs/tests/GuestG2PDialog.cpp b/core/libs/tests/GuestG2PDialog.cpp new file mode 100644 index 000000000..d1666fd28 --- /dev/null +++ b/core/libs/tests/GuestG2PDialog.cpp @@ -0,0 +1,54 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include + +extern "C" { +int APS5_VABI sceG2PDialogInitialize(void); +int APS5_VABI sceG2PDialogOpen(const void* param); +int APS5_VABI sceG2PDialogUpdateStatus(void); +int APS5_VABI sceG2PDialogGetStatus(void); +int APS5_VABI sceG2PDialogGetResult(void* result); +int APS5_VABI sceG2PDialogClose(void); +int APS5_VABI sceG2PDialogTerminate(void); +} + +namespace { + +void Require(bool value) { if (!value) std::abort(); } + +constexpr int kStatusFinished = 3; +constexpr int kErrNotInitialized = static_cast(0x80B80003); +constexpr int kErrAlreadyInitialized = static_cast(0x80B80004); +constexpr int kErrNotFinished = static_cast(0x80B80005); +constexpr int kErrArgNull = static_cast(0x80B8000D); +constexpr int kResultUserCanceled = 1; + +} + +int main() { + std::int64_t param = 0; + std::int32_t result[2] = {-1, -1}; + + Require(sceG2PDialogTerminate() == kErrNotInitialized); + Require(sceG2PDialogOpen(¶m) == kErrNotInitialized); + Require(sceG2PDialogGetResult(result) == kErrNotInitialized); + + Require(sceG2PDialogInitialize() == 0); + Require(sceG2PDialogInitialize() == kErrAlreadyInitialized); + + Require(sceG2PDialogOpen(nullptr) == kErrArgNull); + Require(sceG2PDialogGetResult(result) == kErrNotFinished); + + Require(sceG2PDialogOpen(¶m) == 0); + Require(sceG2PDialogGetStatus() == kStatusFinished); + Require(sceG2PDialogUpdateStatus() == kStatusFinished); + + Require(sceG2PDialogGetResult(nullptr) == kErrArgNull); + Require(sceG2PDialogGetResult(result) == 0); + Require(result[0] == kResultUserCanceled); + Require(result[1] == -1); + + Require(sceG2PDialogClose() == 0); + Require(sceG2PDialogTerminate() == 0); + Require(sceG2PDialogTerminate() == kErrNotInitialized); +} diff --git a/core/libs/tests/GuestVrSetupDialog.cpp b/core/libs/tests/GuestVrSetupDialog.cpp new file mode 100644 index 000000000..a3afc479f --- /dev/null +++ b/core/libs/tests/GuestVrSetupDialog.cpp @@ -0,0 +1,52 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include + +extern "C" { +int APS5_VABI sceVrSetupDialogInitialize(void); +int APS5_VABI sceVrSetupDialogOpen(const void* param); +int APS5_VABI sceVrSetupDialogUpdateStatus(void); +int APS5_VABI sceVrSetupDialogGetResult(void* result); +int APS5_VABI sceVrSetupDialogClose(void); +int APS5_VABI sceVrSetupDialogTerminate(void); +} + +namespace { + +void Require(bool value) { if (!value) std::abort(); } + +constexpr int kStatusFinished = 3; +constexpr int kErrNotInitialized = static_cast(0x80B80003); +constexpr int kErrAlreadyInitialized = static_cast(0x80B80004); +constexpr int kErrNotFinished = static_cast(0x80B80005); +constexpr int kErrArgNull = static_cast(0x80B8000D); +constexpr int kResultUserCanceled = 1; + +} + +int main() { + std::int64_t param = 0; + std::int32_t result[2] = {-1, -1}; + + Require(sceVrSetupDialogTerminate() == kErrNotInitialized); + Require(sceVrSetupDialogOpen(¶m) == kErrNotInitialized); + Require(sceVrSetupDialogGetResult(result) == kErrNotInitialized); + + Require(sceVrSetupDialogInitialize() == 0); + Require(sceVrSetupDialogInitialize() == kErrAlreadyInitialized); + + Require(sceVrSetupDialogOpen(nullptr) == kErrArgNull); + Require(sceVrSetupDialogGetResult(result) == kErrNotFinished); + + Require(sceVrSetupDialogOpen(¶m) == 0); + Require(sceVrSetupDialogUpdateStatus() == kStatusFinished); + + Require(sceVrSetupDialogGetResult(nullptr) == kErrArgNull); + Require(sceVrSetupDialogGetResult(result) == 0); + Require(result[0] == kResultUserCanceled); + Require(result[1] == -1); + + Require(sceVrSetupDialogClose() == 0); + Require(sceVrSetupDialogTerminate() == 0); + Require(sceVrSetupDialogTerminate() == kErrNotInitialized); +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index dfc24f7e1..99146692a 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -17,6 +17,8 @@ Throughout the project, every function at every stage either **does exactly what - [libSceCommonDialog](../../core/libs/prx/libSceCommonDialog/Export.cpp) - [libSceErrorDialog](../../core/libs/prx/libSceErrorDialog/Export.cpp) - the error dialog runs the state machine but shows nothing; the error code passed to `sceErrorDialogOpen` is only logged - [libSceLoginDialog](../../core/libs/prx/libSceLoginDialog/Export.cpp) - the login dialog runs the common-dialog state machine but shows nothing and reports the user as having cancelled +- [libSceCdlgG2P](../../core/libs/prx/libSceCdlgG2P/Export.cpp) - the G2P dialog runs the common-dialog state machine but shows nothing and reports the user as having cancelled. The signatures (`Open` and `GetResult` take a pointer, the others none) and the leading `int32` result are assumed from the other common dialogs, not verified for G2P; PPSA01289 imports it +- [libSceVrSetupDialog](../../core/libs/prx/libSceVrSetupDialog/Export.cpp) - the VR setup dialog runs the common-dialog state machine but shows nothing and reports the user as having cancelled, as no headset is supported (see libSceHmd2). The signatures are assumed from the other common dialogs; PPSA12544 imports it - [libSceHmd](../../core/libs/prx/libSceHmd/Export.cpp) implements only the disconnected-headset path: initialization succeeds, device queries report `NotDetected`, and opening a device returns `DeviceDisconnected`. Headset support, tracking, rendering and additional HMD exports are not implemented; SDK-level ABI compatibility and in-game behaviour remain unverified. - [pthread_mutexattr_setprotocol, pthread_mutexattr_setprioceiling, pthread_mutexattr_getprioceiling](../../core/libs/prx/libkernel/Pthread/Posix/Mutex.cpp) (libkernel) - the protocol and the priority ceiling are stored and reported as in FreeBSD's [libthr](https://github.com/freebsd/freebsd-src/blob/main/lib/libthr/thread/thr_mutexattr.c) (the ceiling is set or read only under `PTHREAD_PRIO_PROTECT`, and setting a protocol resets it to the highest `SCHED_RR` priority), but mutexes do not apply them: a priority-inherit or priority-protect mutex locks like a normal one and does not change its owner's priority - [libSceHmd2](../../core/libs/prx/libSceHmd2/Export.cpp) has no PS VR2 runtime: `sceHmd2Initialize` fails, and every other export it has (and every [libSceVrTracker2](../../core/libs/prx/libSceVrTracker2/Export.cpp) export) throws. PPSA01317 skips its whole VR set-up when initialization fails @@ -186,8 +188,6 @@ Throughout the project, every function at every stage either **does exactly what - [sceVoiceQoSInit](../../core/libs/prx/libSceVoiceQoS/Export.cpp) (libSceVoiceQoS) - `0x804E0905` for a second initialization comes from [prosper](https://github.com/mattias800/prosper)'s reading of the module and was not measured: on a PS5 a valid call blocks inside a homebrew payload, so it never returns there. The app type and the block are not used, as no endpoint is implemented - AudioIn and NpSessionSignaling exports added without an implementation have assumed signatures - [sceNpSessionSignalingGetConnectionFromPeerAddress](../../core/libs/prx/libSceNpSessionSignaling/Export.cpp) (libSceNpSessionSignaling) - imported by PPSA01289 and added without an implementation; unknown signature -- [libSceCdlgG2P](../../core/libs/prx/libSceCdlgG2P/Export.cpp) - `sceG2PDialogInitialize`, `Terminate`, `Open`, `GetStatus`, `UpdateStatus` and `GetResult`, imported by PPSA01289, are added without an implementation and declared without parameters -- [sceG2PDialogClose](../../core/libs/prx/libSceCdlgG2P/Export.cpp) (libSceCdlgG2P) - throwing stub; the name comes from the aerolib NID list, and the signature `int(void)` is assumed from `sceLoginDialogClose`, not verified for G2P - [vieBRwlh1Lw](../../core/libs/prx/libSceAgc/Unimplemented.cpp) (libSceAgc) - unknown name, signature - [scePadIsRemoteController](../../core/libs/prx/libScePad/Export.cpp) (libScePad) - the name hashes to NID `fCWdlnmB1Ks`; the signature `(int handle, bool* remote)` comes from PPSA16064, which zeroes a byte, passes its address after `scePadGetControllerInformation` and ignores the result. The error codes follow the other pad calls. Controllers are local devices, so the answer is always false - [scePadSetAngularVelocityBiasCorrectionState](../../core/libs/prx/libScePad/Export.cpp) (libScePad) - inferred signature `(int handle, bool enabled)` follows `scePadSetTiltCorrectionState` and `scePadSetAngularVelocityDeadbandState`; an invalid handle returns `PAD_ERROR_INVALID_HANDLE`, disabling is a no-op, and enabling throws because bias correction is not modelled @@ -197,7 +197,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceSslGetCaList, sceSslFreeCaList, sceSslUnloadCert](../../core/libs/prx/libSceSsl/Export.cpp) (libSceSsl) - signatures and the 16-byte `SceSslCaList` layout (an array of certificate handles and a 32-bit count) follow [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/5dfdb4b0b166e83dc9ae275001afb695f12c7174/src/SharpProspero/Interop/Net/Http.cs) and shadPS4's `OrbisSslCaList`, not confirmed for the PS5. On the console the list holds the certificate authorities the SSL service has; [shadPS4](https://github.com/shadps4-emu/shadPS4/pull/4104) returns a placeholder certificate from `sceSslGetCaCerts` because Rise of the Tomb Raider fails its initialization checks without one. No root certificate bundle is shipped and no loaded certificate is kept, so here a context holds none: `sceSslGetCaList` clears the list and returns `NOT_FOUND`, as `sceSslGetCaCerts` does, `sceSslFreeCaList` accepts only that cleared list, and `sceSslUnloadCert` (imported by PPSA15508) has nothing to drop and is assumed to return 0. A null list returns `0x8095177A`, which `sceSslGetCaCerts` and [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/59183dbfa81ed7efdff6069177fed50de0552725/src/libs/network.cpp) return for a null argument (shadPS4, PS4, returns `0x8095F007`). An unknown context throws (KytyPS5 returns `SSL_ERROR_INVALID_ID`, `0x8095F006`, from `sceSslGetCaCerts`), and so does a list `sceSslGetCaList` did not return - [sceHttp2SetResolveRetry](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) - signature `(int id, int32_t retry)` from libSceHttp's `sceHttpSetResolveRetry`; the value is accepted and ignored like the other Http2 settings, since no request reaches the network - [sceHttp2WebSocketCreateRequest, sceHttp2WebSocketCloseAsync, sceHttp2WebSocketSendTextMessageAsync, sceHttp2WebSocketSendDataMessageAsync](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) - imported by PPSA16064 and not called during its start-up; unknown signatures -- [libSceVrSetupDialog](../../core/libs/prx/libSceVrSetupDialog/Export.cpp), and the Share, NpTrophy2, NpEntitlementAccess and WebBrowserDialog exports imported by PPSA12544's Unity plugins and added without an implementation, have unknown signatures +- The Share, NpTrophy2, NpEntitlementAccess and WebBrowserDialog exports imported by PPSA12544's Unity plugins and added without an implementation have unknown signatures - [sceNetShowNetstat](../../core/libs/prx/libSceNet/Export.cpp) (libSceNet) - imported by PPSA11386 and not called during its start-up; unknown signature, declared without parameters - [sceAjmDecWVorbisCreateHeaderPacket, sceAjmDecWVorbisCreateSetupPacket](../../core/libs/prx/libSceAjm.native/Export.cpp) (libSceAjmWVorbis, implemented in libSceAjm.native) - unknown signatures - Acm, GameLiveStreaming, Net, NpEntitlementAccess, NpManager, NpWebApi2, VoiceQoS, WebBrowserDialog and libkernel exports imported by PPSA14632 and added without an implementation are declared without parameters From 9a13dc8c787e9f2fb02c9157b4e91abdf623cfc0 Mon Sep 17 00:00:00 2001 From: Get-fix <100056769+Get-fix@users.noreply.github.com> Date: Sat, 10 Oct 2026 20:56:09 +0300 Subject: [PATCH 223/460] fix(relinker): stop code scans at the PLT in executables with PT_SCE_DYNLIBDATA A PS4 executable (CUSA00900) keeps 37 MiB of read-only data in its executable segment after the code. The syscall scan and the call-site resolution decoded the whole segment linearly, reached a float table and failed with "Unsupported 0F 79 opcode without SSE4A prefix". In executables with a PT_SCE_DYNLIBDATA segment they now stop at the end of the PLT that follows the last function of the unwind table. The PLT is found by searching forward from that function for its first entry, which pushes the GOT entry at PLTGOT + 8 and jumps through PLTGOT + 16, since code without unwind entries can sit in between (680 bytes of start-up code in an OpenOrbis-built sample). When the executable has PLT relocations and no such entry follows, the relinker fails instead of guessing. Executables without the segment are scanned as before. (cherry picked from commit a21e8fe4c0e6c03189d559e992c0f037c076700d) --- core/relinker/CMakeLists.txt | 2 + .../src/pipeline/RelinkerPipeline.cpp | 40 +++++++ .../relinker/tests/test_sce_code_extent.py | 104 ++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 147 insertions(+) create mode 100644 core/relinker/relinker/tests/test_sce_code_extent.py diff --git a/core/relinker/CMakeLists.txt b/core/relinker/CMakeLists.txt index 913c56162..91937268b 100644 --- a/core/relinker/CMakeLists.txt +++ b/core/relinker/CMakeLists.txt @@ -317,6 +317,8 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_input_nonascii.py $) add_test(NAME string_table_bounds COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_string_table_bounds.py $) + add_test(NAME sce_code_extent COMMAND ${Python3_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_sce_code_extent.py $) add_test(NAME call_registry COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_call_registry.py $) add_test(NAME jmprel_table_bounds COMMAND ${Python3_EXECUTABLE} diff --git a/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp b/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp index d631eea28..0849ddac4 100644 --- a/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp +++ b/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp @@ -2,6 +2,9 @@ #include #include #include +#include +#include +#include #include #include #include @@ -236,6 +239,43 @@ RelinkResult RelinkerPipeline::Relink(const std::vector& sourceElf for (const auto& ref : nidRefs) _validationPolicy->ValidateRelocationTypeSupported(ref.RelocationTypeValue, ref.RelocationTableOffset); + if (std::any_of(programHeaders.begin(), programHeaders.end(), [](const ProgramHeader& ph) { return ph.Type == PT_SCE_DYNLIBDATA; })) { + const auto functions = UnusedNidFilter::ReadExceptionFunctions(raw, programHeaders, {}, {}); + for (auto& [segment, segmentVAddr] : executableSegments) { + VirtualAddress codeEnd = 0; + for (const auto& function : functions) + if (function.Begin >= segmentVAddr && function.Begin - segmentVAddr < segment.size()) + codeEnd = std::max(codeEnd, function.End); + if (codeEnd == 0) + continue; + if (codeEnd - segmentVAddr > segment.size()) + throw RelinkerException("Unwind function exceeds executable segment", codeEnd); + if (hasPltRelocations && dynJmpRelSize != 0) { + const auto target = [&](const std::size_t at) { + std::int32_t displacement = 0; + std::memcpy(&displacement, segment.data() + at + 2, sizeof(displacement)); + return segmentVAddr + at + 6 + static_cast(static_cast(displacement)); + }; + const std::size_t lastFunctionEnd = codeEnd - segmentVAddr; + std::optional plt; + for (std::size_t at = lastFunctionEnd; segment.size() >= 16 && at <= segment.size() - 16; ++at) { + if (segment[at] == 0xff && segment[at + 1] == 0x35 && segment[at + 6] == 0xff && segment[at + 7] == 0x25 + && target(at) == gotVAddr + 8 && target(at + 6) == gotVAddr + 16) { + plt = at; + break; + } + } + if (!plt) + throw RelinkerException("PLT not found after the last unwind function", codeEnd); + const ByteCount pltSize = 16 * (1 + dynJmpRelSize / relaEntSize); + if (pltSize > segment.size() - *plt) + throw RelinkerException("PLT exceeds executable segment", segmentVAddr + *plt); + codeEnd = segmentVAddr + *plt + pltSize; + } + segment.resize(codeEnd - segmentVAddr); + } + } + for (const auto& [segment, segmentVAddr] : executableSegments) _syscallScanner->ScanCodeSectionForSyscalls(segment, segmentVAddr, segment.size()); diff --git a/core/relinker/relinker/tests/test_sce_code_extent.py b/core/relinker/relinker/tests/test_sce_code_extent.py new file mode 100644 index 000000000..8abcbc6e3 --- /dev/null +++ b/core/relinker/relinker/tests/test_sce_code_extent.py @@ -0,0 +1,104 @@ +from pathlib import Path +import json +import struct +import subprocess +import sys +import tempfile + +PT_LOAD = 1 +PT_DYNAMIC = 2 +PT_GNU_EH_FRAME = 0x6474E550 +PT_SCE_DYNLIBDATA = 0x61000000 +PT_SCE_VERSION = 0x6FFFFF01 +FUNCTION = 0x200 +FUNCTION_SIZE = 0x10 +GOT = 0xA00 +EH_FRAME_HEADER = 0x800 +EH_FRAME = 0x840 + + +def fixture(sce_block=True, plt_push_offset=8, gap=0): + plt = FUNCTION + FUNCTION_SIZE + gap + data = plt + 0x20 + image = bytearray(0x1000) + image[:16] = b"\x7fELF\x02\x01\x01" + bytes(9) + struct.pack_into(" Date: Sun, 11 Oct 2026 00:51:04 +0200 Subject: [PATCH 224/460] fix(relinker): place the Windows thread pointer after the TLS size rounded to its own alignment The converted thread pointer was set to the TLS block plus the size rounded to max(p_align, 16), while the guest reads its first variable at tp minus the size rounded to p_align. For a PT_TLS with memsz 0x21 and p_align 8 (Resident Evil 4) that is 0x30 against 0x28, so the first variable was read from the second slot. The thread pointer now uses the size rounded to p_align; the block layout is unchanged. WIP: the test for this is still missing. Co-Authored-By: Claude Sonnet 5.5 --- core/relinker/elfpatcher/src/windows/WindowsTlsBuilder.cpp | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/core/relinker/elfpatcher/src/windows/WindowsTlsBuilder.cpp b/core/relinker/elfpatcher/src/windows/WindowsTlsBuilder.cpp index 54414f33e..df1ce0fea 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsTlsBuilder.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsTlsBuilder.cpp @@ -180,13 +180,15 @@ PeDirectory WindowsTlsBuilder::Build(const std::vector& source, co return {}; const auto alignment = std::max(tls->Alignment, 16); const auto blockSize = CheckedRva((tls->MemorySize + alignment - 1) & ~(alignment - 1)); + const auto threadPointerAlignment = std::max(tls->Alignment, 1); + const auto threadPointerOffset = CheckedRva((tls->MemorySize + threadPointerAlignment - 1) & ~(threadPointerAlignment - 1)); const auto templateOffset = CheckedRva((64 + alignment - 1) & ~(alignment - 1)); constexpr std::uint32_t threadControlBlockSize = 0x30; constexpr std::int64_t stackGuardDisplacement = 0x28; for (const auto& access : accesses) { if (access.StoreImmediate || access.Displacement == 0) continue; const auto displacement = static_cast(std::bit_cast(access.Displacement)); - if (displacement != stackGuardDisplacement && (displacement > 0 || displacement < -static_cast(blockSize))) + if (displacement != stackGuardDisplacement && (displacement > 0 || displacement < -static_cast(threadPointerOffset))) throw Domain::RelinkerException("Windows guest TLS load displacement " + std::to_string(displacement) + " is not in the thread TLS block, 0 or the stack guard at 0x28", access.FileOffset); } PeSection data{".gtls", nextRva, SectionRead | SectionWrite | 0x40u, std::vector(templateOffset + blockSize + threadControlBlockSize)}; @@ -201,7 +203,7 @@ PeDirectory WindowsTlsBuilder::Build(const std::vector& source, co const auto loadPointer = [&] { code.Rip({0x8b, 0x0d}, indexRva); code.Emit({0x65, 0x48, 0x8b, 0x04, 0x25, 0x58, 0, 0, 0, 0x48, 0x8b, 0x04, 0xc8, 0x48, 0x8d, 0x80}); - code.U32(blockSize); + code.U32(threadPointerOffset); }; code.Emit({0x83, 0xfa, 1}); From 272b9d1338dc0c3579feeb6cd6930b2dca087693 Mon Sep 17 00:00:00 2001 From: xXPBSXx <20614131+xXPBSXx@users.noreply.github.com> Date: Sun, 11 Oct 2026 00:55:46 +0200 Subject: [PATCH 225/460] test(relinker): check Windows TLS thread pointer offset Co-Authored-By: Claude Sonnet 5.5 --- core/relinker/CMakeLists.txt | 2 + .../tests/test_windows_tls_thread_pointer.py | 50 +++++++++++++++++++ 2 files changed, 52 insertions(+) create mode 100644 core/relinker/relinker/tests/test_windows_tls_thread_pointer.py diff --git a/core/relinker/CMakeLists.txt b/core/relinker/CMakeLists.txt index 913c56162..3c95d2e6c 100644 --- a/core/relinker/CMakeLists.txt +++ b/core/relinker/CMakeLists.txt @@ -266,6 +266,8 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_windows_working_directory.py $) add_test(NAME windows_address_space COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_windows_address_space.py $) + add_test(NAME windows_tls_thread_pointer COMMAND ${Python3_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_windows_tls_thread_pointer.py $) add_test(NAME windows_bss COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_windows_bss.py $) add_test(NAME windows_icon COMMAND ${Python3_EXECUTABLE} diff --git a/core/relinker/relinker/tests/test_windows_tls_thread_pointer.py b/core/relinker/relinker/tests/test_windows_tls_thread_pointer.py new file mode 100644 index 000000000..41de44b6b --- /dev/null +++ b/core/relinker/relinker/tests/test_windows_tls_thread_pointer.py @@ -0,0 +1,50 @@ +"""Check that the converted Windows thread pointer sits after the TLS size rounded to the PT_TLS alignment.""" + +import os +from pathlib import Path +import struct +import subprocess +import sys +import tempfile + +from test_optional_plt import fixture + +FIRST_VARIABLE = 0x2a +SECOND_VARIABLE = 0x63 +TLS_OFFSET = 0x980 +MEMORY_SIZE = 0x21 +ALIGNMENT = 8 +ROUNDED_SIZE = (MEMORY_SIZE + ALIGNMENT - 1) // ALIGNMENT * ALIGNMENT + + +def guest_with_tls(): + image = fixture() + struct.pack_into(" Date: Sun, 11 Oct 2026 00:58:28 +0200 Subject: [PATCH 226/460] fix(libkernel): report the largest free direct memory block from sceKernelAvailableDirectMemorySize The call returned the first free gap in the range instead of the largest free block. Co-Authored-By: Claude Sonnet 5.5 --- .../libkernel/DirectMemory/DirectMemory.hpp | 1 + .../prx/libkernel/DirectMemory/Export.cpp | 11 +++---- .../prx/libkernel/DirectMemory/MemoryPool.cpp | 32 +++++++++++++++++++ core/libs/tests/GuestMemory.cpp | 32 +++++++++++++++++++ 4 files changed, 70 insertions(+), 6 deletions(-) diff --git a/core/libs/prx/libkernel/DirectMemory/DirectMemory.hpp b/core/libs/prx/libkernel/DirectMemory/DirectMemory.hpp index 988291463..591dabb17 100644 --- a/core/libs/prx/libkernel/DirectMemory/DirectMemory.hpp +++ b/core/libs/prx/libkernel/DirectMemory/DirectMemory.hpp @@ -26,6 +26,7 @@ std::vector DirectMemoryViews(int64_t start, size_t len); void UnmapDirectMemoryViews(const std::vector& views); bool DirectMemoryFind(int64_t offset, bool findNext, int64_t* start, int64_t* end, int* memoryType); void DirectMemoryRetype(int64_t start, size_t len, int memoryType); +bool DirectMemoryLargestFree(int64_t searchStart, int64_t searchEnd, size_t alignment, int64_t* startOut, size_t* sizeOut); size_t DirectMemoryFreeRun(uint64_t offset, uint64_t limit); int DoMapDirect(void** addr, size_t len, int prot, int flags, int64_t physStart, size_t alignment); int DoMapAnon(void** addr, size_t len, int prot, int flags, size_t alignment = PS5_PAGE_SIZE); diff --git a/core/libs/prx/libkernel/DirectMemory/Export.cpp b/core/libs/prx/libkernel/DirectMemory/Export.cpp index bee6dd534..91c0a1f6b 100644 --- a/core/libs/prx/libkernel/DirectMemory/Export.cpp +++ b/core/libs/prx/libkernel/DirectMemory/Export.cpp @@ -133,12 +133,11 @@ int APS5_VABI sceKernelAllocateMainDirectMemory(size_t len, size_t alignment, in int APS5_VABI sceKernelAvailableDirectMemorySize(int64_t search_start, int64_t search_end, size_t alignment, int64_t* phys_addr_out, size_t* size_out) { if (!phys_addr_out || !size_out) return SCE_KERNEL_ERROR_EINVAL; - int64_t tmpPhys = 0; - int ret = DirectMemoryAlloc(search_start, search_end, PS5_PAGE_SIZE, alignment, -1, &tmpPhys); - if (ret != 0) { *phys_addr_out = 0; *size_out = 0; return ret; } - DirectMemoryFree(tmpPhys, PS5_PAGE_SIZE); - *phys_addr_out = tmpPhys; - *size_out = DirectMemoryFreeRun(static_cast(tmpPhys), static_cast(search_end)); + int64_t start = 0; + size_t size = 0; + if (!DirectMemoryLargestFree(search_start, search_end, alignment, &start, &size)) { *phys_addr_out = 0; *size_out = 0; return SCE_KERNEL_ERROR_EAGAIN; } + *phys_addr_out = start; + *size_out = size; return 0; } diff --git a/core/libs/prx/libkernel/DirectMemory/MemoryPool.cpp b/core/libs/prx/libkernel/DirectMemory/MemoryPool.cpp index 645098b71..d7c6c65d7 100644 --- a/core/libs/prx/libkernel/DirectMemory/MemoryPool.cpp +++ b/core/libs/prx/libkernel/DirectMemory/MemoryPool.cpp @@ -72,6 +72,34 @@ struct PhysicalMemoryPool { } } + bool LargestFree(int64_t searchStart, int64_t searchEnd, size_t alignment, int64_t* startOut, size_t* sizeOut) { + std::lock_guard lock(_mutex); + if (searchStart < 0 || searchEnd <= searchStart) return false; + const uint64_t align = (alignment == 0) ? PS5_PAGE_SIZE : alignment; + const uint64_t limit = std::min(static_cast(searchEnd), DIRECT_MEMORY_SIZE); + uint64_t bestStart = 0; + uint64_t bestSize = 0; + uint64_t cur = (static_cast(searchStart) + PS5_PAGE_SIZE - 1) & ~static_cast(PS5_PAGE_SIZE - 1); + while (cur + PS5_PAGE_SIZE <= limit) { + if (!_isFree(cur, PS5_PAGE_SIZE)) { + cur += PS5_PAGE_SIZE; + continue; + } + uint64_t runEnd = cur; + while (runEnd + PS5_PAGE_SIZE <= limit && _isFree(runEnd, PS5_PAGE_SIZE)) runEnd += PS5_PAGE_SIZE; + const uint64_t alignedStart = (cur + align - 1) & ~(align - 1); + if (alignedStart < runEnd && runEnd - alignedStart > bestSize) { + bestStart = alignedStart; + bestSize = runEnd - alignedStart; + } + cur = runEnd; + } + if (bestSize == 0) return false; + *startOut = static_cast(bestStart); + *sizeOut = static_cast(bestSize); + return true; + } + size_t FreeRun(uint64_t offset, uint64_t limit) { std::lock_guard lock(_mutex); uint64_t cur = offset & ~static_cast(PS5_PAGE_SIZE - 1); @@ -145,6 +173,10 @@ void DirectMemoryRetype(int64_t start, size_t len, int memoryType) { PhysicalMemoryPool::Instance().Retype(static_cast(start), len, memoryType); } +bool DirectMemoryLargestFree(int64_t searchStart, int64_t searchEnd, size_t alignment, int64_t* startOut, size_t* sizeOut) { + return PhysicalMemoryPool::Instance().LargestFree(searchStart, searchEnd, alignment, startOut, sizeOut); +} + size_t DirectMemoryFreeRun(uint64_t offset, uint64_t limit) { return PhysicalMemoryPool::Instance().FreeRun(offset, limit); } diff --git a/core/libs/tests/GuestMemory.cpp b/core/libs/tests/GuestMemory.cpp index 1e7d99326..e32ffcbb4 100644 --- a/core/libs/tests/GuestMemory.cpp +++ b/core/libs/tests/GuestMemory.cpp @@ -59,6 +59,7 @@ int APS5_VABI sceKernelClearVirtualRangeName(const void*, std::uint64_t); int APS5_VABI sceKernelAllocateDirectMemory(std::int64_t, std::int64_t, std::size_t, std::size_t, int, std::int64_t*); int APS5_VABI sceKernelMapDirectMemory(void**, std::size_t, int, int, std::int64_t, std::size_t); int APS5_VABI sceKernelReleaseDirectMemory(std::int64_t, std::size_t); +int APS5_VABI sceKernelAvailableDirectMemorySize(std::int64_t, std::int64_t, std::size_t, std::int64_t*, std::size_t*); int APS5_VABI sceKernelCheckedReleaseDirectMemory(std::int64_t, std::size_t); int APS5_VABI sceKernelReserveVirtualRange(void**, std::size_t, int, std::size_t); int APS5_VABI sceKernelMemoryPoolReserve(void*, std::size_t, std::size_t, int, void**); @@ -1621,10 +1622,41 @@ static void CheckVirtualQueryPartialMunmap() { Require(munmap_nid_postfix(memory + page * 2, page) == 0); } + +static void CheckAvailableDirectMemoryIsLargestFreeBlock() { + constexpr std::int64_t window = 0x4000000; + const std::int64_t top = static_cast(DIRECT_MEMORY_SIZE) - window; + std::int64_t first = -1; + std::int64_t gap = -1; + std::int64_t last = -1; + Require(sceKernelAllocateDirectMemory(top, top + window, 0x200000, 0x200000, 0, &first) == 0 && first == top); + Require(sceKernelAllocateDirectMemory(top, top + window, 0x400000, 0x200000, 0, &gap) == 0 && gap == top + 0x200000); + Require(sceKernelAllocateDirectMemory(top, top + window, 0x200000, 0x200000, 0, &last) == 0 && last == top + 0x600000); + Require(sceKernelReleaseDirectMemory(gap, 0x400000) == 0); + + std::int64_t start = -1; + std::size_t size = 0; + Require(sceKernelAvailableDirectMemorySize(top, top + window, 0, &start, &size) == 0); + Require(start == top + 0x800000 && size == static_cast(window - 0x800000)); + Require(sceKernelAvailableDirectMemorySize(top, top + 0x800000, 0, &start, &size) == 0); + Require(start == top + 0x200000 && size == 0x400000); + Require(sceKernelAvailableDirectMemorySize(top, top + window, 0x1000000, &start, &size) == 0); + Require(start == top + 0x1000000 && size == static_cast(window - 0x1000000)); + Require(sceKernelAvailableDirectMemorySize(top, top + 0x200000, 0, &start, &size) == SCE_KERNEL_ERROR_EAGAIN); + Require(start == 0 && size == 0); + Require(sceKernelAvailableDirectMemorySize(top, top + window, 0, nullptr, &size) == SCE_KERNEL_ERROR_EINVAL); + + Require(sceKernelReleaseDirectMemory(first, 0x200000) == 0); + Require(sceKernelReleaseDirectMemory(last, 0x200000) == 0); + Require(sceKernelAvailableDirectMemorySize(top, top + window, 0, &start, &size) == 0); + Require(start == top && size == static_cast(window)); +} + int main() { #ifdef _WIN32 _putenv_s("APS5_PIN_WAIT_MS", "1000"); #endif + CheckAvailableDirectMemoryIsLargestFreeBlock(); CheckVirtualQuerySplitFlexibleRanges(); CheckVirtualQueryForNonReadableGuestRange(); CheckReleaseFlexibleMemory(); From 383bffd8c7d6571e7045cd52508e53653b3d0312 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sat, 10 Oct 2026 08:12:06 -0500 Subject: [PATCH 227/460] feat(libc): implement alarm RetroArch (PPSA99169) stops at alarm on main: `FAIL: unresolved ELF import DQXJraCc1rA from libSceLibcInternal.prx`. alarm(seconds) arms one process-wide real-time alarm and returns the seconds left on the previous one, rounded up like FreeBSD's alarm.c (any partial second counts); alarm(0) cancels it. When it expires, SIGALRM's disposition as recorded by sigaction or signal applies: ignored, nothing happens; default, the process terminates as on FreeBSD (exit status 128 + 14); a guest handler throws, because asynchronous delivery to a guest handler is not implemented. The timer lives in libkernel next to the signal dispositions (GuestAlarm_nid_no_patch); libSceLibcInternal, which the titles import alarm from, forwards to it. AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceLibcInternal/Export.cpp | 6 ++ .../libs/prx/libkernel/System/src/Signals.cpp | 63 +++++++++++++++++++ core/libs/tests/GuestSignals.cpp | 14 +++++ 3 files changed, 83 insertions(+) diff --git a/core/libs/prx/libSceLibcInternal/Export.cpp b/core/libs/prx/libSceLibcInternal/Export.cpp index f9425b744..89afdbc73 100644 --- a/core/libs/prx/libSceLibcInternal/Export.cpp +++ b/core/libs/prx/libSceLibcInternal/Export.cpp @@ -130,4 +130,10 @@ int APS5_VABI _sceLibcInternalForceTlsDestructor_nid_postfix(KernelModule handle return 0; } +unsigned int APS5_VABI GuestAlarm_nid_no_patch(unsigned int seconds); + +unsigned int APS5_VABI alarm_nid_postfix(unsigned int seconds) { + return GuestAlarm_nid_no_patch(seconds); +} + } diff --git a/core/libs/prx/libkernel/System/src/Signals.cpp b/core/libs/prx/libkernel/System/src/Signals.cpp index 93dfb9fad..38de618a2 100644 --- a/core/libs/prx/libkernel/System/src/Signals.cpp +++ b/core/libs/prx/libkernel/System/src/Signals.cpp @@ -1,11 +1,17 @@ #include "prx/libc/include/General.hpp" #include "prx/libc/include/general/VabiMacros.hpp" #include +#include +#include #include #include #include +#include +#include #include +#include #include +#include extern "C" int* APS5_VABI __error_nid_postfix(); extern "C" int APS5_VABI getpid_nid_postfix(void); @@ -74,6 +80,63 @@ bool Install(int guest, const GuestSigaction& action) { } } +namespace { +constexpr int GuestSigalrm = 14; +std::mutex alarmLock; +std::condition_variable alarmChanged; +std::optional alarmDeadline; +bool alarmThreadStarted = false; + +void ExpireAlarm() { + std::uintptr_t handler = 0; + { + std::lock_guard lock(registration); + handler = dispositions[GuestSigalrm].handler; + } + if (handler == 1) return; + if (handler == 0) { + std::fprintf(stderr, "SIGALRM: the alarm expired with the default disposition; the process terminates\n"); + std::fflush(stderr); + std::_Exit(128 + GuestSigalrm); + } + throw std::runtime_error("alarm: delivering SIGALRM to a guest handler is not implemented"); +} + +void RunAlarms() { + std::unique_lock lock(alarmLock); + for (;;) { + if (!alarmDeadline) { + alarmChanged.wait(lock); + continue; + } + const auto deadline = *alarmDeadline; + if (alarmChanged.wait_until(lock, deadline) != std::cv_status::timeout || alarmDeadline != deadline) continue; + alarmDeadline.reset(); + lock.unlock(); + ExpireAlarm(); + lock.lock(); + } +} +} + +extern "C" unsigned int APS5_VABI GuestAlarm_nid_no_patch(unsigned int seconds) { + std::lock_guard lock(alarmLock); + const auto now = std::chrono::steady_clock::now(); + unsigned int remaining = 0; + if (alarmDeadline && *alarmDeadline > now) { + const auto micros = std::chrono::duration_cast(*alarmDeadline - now).count(); + remaining = static_cast((micros + 999999) / 1000000); + } + if (seconds == 0) alarmDeadline.reset(); + else alarmDeadline = now + std::chrono::seconds(seconds); + if (!alarmThreadStarted) { + std::thread(RunAlarms).detach(); + alarmThreadStarted = true; + } + alarmChanged.notify_all(); + return remaining; +} + struct GuestStack { void* sp; std::size_t size; diff --git a/core/libs/tests/GuestSignals.cpp b/core/libs/tests/GuestSignals.cpp index b90cb37c9..a1453fc98 100644 --- a/core/libs/tests/GuestSignals.cpp +++ b/core/libs/tests/GuestSignals.cpp @@ -4,6 +4,8 @@ #include #include #include +#include +#include using Handler = void (APS5_VABI *)(int); extern "C" { Handler APS5_VABI signal_nid_postfix(int, Handler); @@ -13,6 +15,7 @@ int APS5_VABI sigprocmask_nid_postfix(int, const void*, void*); int APS5_VABI pthread_sigmask_nid_postfix(int, const void*, void*); int* APS5_VABI __error_nid_postfix(); int APS5_VABI _is_signal_return_nid_postfix(std::uint64_t); +unsigned int APS5_VABI GuestAlarm_nid_no_patch(unsigned int); } struct GuestSignalSet { std::uint32_t bits[4]; @@ -128,4 +131,15 @@ int main() { Require(sigprocmask_nid_postfix(1, &urgMask, nullptr) == 0); Require(raise_nid_postfix(15) == 0 && received == 15); Require(sigprocmask_nid_postfix(2, &urgMask, nullptr) == 0); + + Require(GuestAlarm_nid_no_patch(0) == 0); + Require(GuestAlarm_nid_no_patch(5) == 0); + Require(GuestAlarm_nid_no_patch(3) == 5); + Require(GuestAlarm_nid_no_patch(0) == 3); + Require(GuestAlarm_nid_no_patch(0) == 0); + const GuestSigaction ignoreAlarm{1, 0, {}}; + Require(sigaction_nid_postfix(14, &ignoreAlarm, nullptr) == 0); + Require(GuestAlarm_nid_no_patch(1) == 0); + std::this_thread::sleep_for(std::chrono::milliseconds(1500)); + Require(GuestAlarm_nid_no_patch(0) == 0); } From f5618d912467c60381d8e77f85324b7249e27002 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 16:51:15 +0000 Subject: [PATCH 228/460] fix(agc): release retired imports while waiting for pins --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 4 +++- .../Graphics/include/Recorder.hpp | 1 + .../Graphics/src/GuestBufferMemory.cpp | 2 ++ .../libSceAgcDriver/Graphics/src/Recorder.cpp | 4 ++++ .../libs/prx/libSceAgcDriver/tests/Recorder.cpp | 17 ++++++++++------- 5 files changed, 20 insertions(+), 8 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index ee8273d8a..ba7f31cbc 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -732,8 +732,10 @@ if(BUILD_TESTING) endif() add_test(NAME agc_host_import_unmap COMMAND agc_driver_recorder_tests --host-import-unmap-only) set_tests_properties(agc_host_import_unmap PROPERTIES SKIP_RETURN_CODE 77) + add_test(NAME agc_unit_shadow COMMAND agc_driver_recorder_tests --unit-shadow-only) + set_tests_properties(agc_unit_shadow PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 30 ENVIRONMENT "APS5_PIN_WAIT_MS=3000") if(WIN32) - set_tests_properties(agc_host_import_unmap PROPERTIES ENVIRONMENT_MODIFICATION + set_tests_properties(agc_host_import_unmap agc_unit_shadow PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() endif() diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp index e59a93738..1f042f2ca 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp @@ -88,6 +88,7 @@ public: void Keep(std::shared_ptr object, std::size_t bytes = 0); void KeepBytes(const void* owner, std::size_t bytes); std::size_t OpenKeptBytes() const { return open != nullptr ? open->keptBytes : 0; } + void FlushDeferredReleases(); static constexpr std::size_t KeptBytesBudget = std::size_t{512} << 20u; void BoundKeptBytes(); std::size_t InFlightKeptBytes() const { return inFlightKeptBytes; } diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp index 1c9a8125f..fb826c431 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp @@ -1283,6 +1283,7 @@ bool WaitForLeasesAndImports(std::uintptr_t address, std::size_t bytes) noexcept synced = true; drained = true; } + if (recorder != nullptr) recorder->FlushDeferredReleases(); const auto end = static_cast(address) + bytes; auto& imports = Imports(); const auto lease = GuestAllocations::GuestAllocationsAcquire_nid_postfix(); @@ -1306,6 +1307,7 @@ bool WaitForLeasesAndImports(std::uintptr_t address, std::size_t bytes) noexcept synced = true; drained = true; } + if (retiredImport && recorder != nullptr) recorder->FlushDeferredReleases(); } catch (const std::exception& error) { std::fprintf(stderr, "[gpu] lease wait failed: %s\n", error.what()); } diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Recorder.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Recorder.cpp index fea9b921a..c788418b9 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Recorder.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Recorder.cpp @@ -3227,6 +3227,10 @@ void Recorder::Sync() { activeSyncSite = previousSite; } +void Recorder::FlushDeferredReleases() { + ReleaseDeferredKeeps(); +} + void Recorder::SyncThrough(std::uint64_t address, std::size_t bytes, bool waitUnlocked) { if (bytes == 0) return; const auto end = address + bytes; diff --git a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp index 6950dc169..be461dbe9 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp @@ -2009,10 +2009,6 @@ void unitShadowTests(const Device& device, Recorder& recorder) { retile(*high, unit128, unit, 0x77, false); Require(AnyShadowedOverlaps(unit128, unit), "the second slab's unit is not shadowed"); high.reset(); - // Retire through the registry: the block is re-registered as its first five units, so the next - // lookup reconciles and retires the import; the fresh units still inside a registered range - // are published into its (kept) buffer, the rest (memory the title took back) are dropped. - // Under the one-slab budget unit 5's slab evicts the second one first (unit 128 published). retile(*ShadowDestinationFor(context, *import, unit5, unit5 + unit), unit5, unit, 0x88, false); retile(*ShadowDestinationFor(context, *import, address + 2 * unit, unit3), address + 2 * unit, unit, 0x99, false); Require(AnyShadowedOverlaps(unit5, unit) && AnyShadowedOverlaps(address + 2 * unit, unit), "units 2 and 5 are not shadowed before the retire"); @@ -2025,9 +2021,8 @@ void unitShadowTests(const Device& device, Recorder& recorder) { Require(!AnyShadowedOverlaps(address, bytes), "the retired import's shadow survived"); recorder.Sync(); Require(words[2 * unit] == 0x99 && words[3 * unit - 1] == 0x99, "the retire did not publish the unit still registered"); - Require(words[5 * unit] == 0x11 && words[5 * unit + unit - 1] == 0x11, "the retire published a unit whose memory is no longer registered"); - if (budgetMiB >= 16) Require(words[127 * unit] == 0x11 && words[128 * unit] == 0x11, "the retire published the second slab's units outside the registration"); - else Require(words[128 * unit] == 0x77, "the eviction before the retire did not publish unit 128"); + Require(words[5 * unit] == 0x88 && words[5 * unit + unit - 1] == 0x88, "the retire did not publish a unit covered by the old registration"); + Require(words[127 * unit] == 0x66 && words[128 * unit] == 0x77, "the retire did not publish the second slab's units covered by the old registration"); } void storageRefreshTests(const Device& device, Recorder& recorder, bool watched) { @@ -4656,6 +4651,14 @@ int main(int argc, char** argv) { hostImportUnmapTests(device, recorder); return 0; } + if (argc == 2 && std::string_view(argv[1]) == "--unit-shadow-only") { + if (!AgcDriver::Graphics::UnitShadowEnabled() || device.GetContext().hostImportAlignment == 0) { + std::cout << "skipped, unit shadows or host imports are unavailable\n"; + return 77; + } + unitShadowTests(device, recorder); + return 0; + } readTrackingTests(device, recorder); writeSettledTests(device, recorder); completionCountTests(device, recorder); From 5ea89c7d4c38eb99b9c5e0b46a5fbdfb718fe5ba Mon Sep 17 00:00:00 2001 From: Abdulrahman Elkordy <196347165+Silverash-i8@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:38:32 +0300 Subject: [PATCH 229/460] feat(font): keep a handle's script languages and typographic features sceFontSetScriptLanguage, sceFontGetScriptLanguage, sceFontSetTypographicDesign and sceFontGetTypographicDesign threw. With the signatures SharpProspero declares, a font handle now keeps the language set for each script and the feature set for each typographic design, under the handle's lock, and returns them, with 0 for one never set. They do not change shaping or rendering yet; TechnicalDebt records the assumptions. Claude-Session: https://claude.ai/code/session_012SLQbhAiexxW6TtjYToW8a (cherry picked from commit c4dcbe4403dc635fde2a823dec311a939816cefc) --- .../prx/libSceFont/include/FontInternal.hpp | 3 ++ core/libs/prx/libSceFont/src/Style.cpp | 40 +++++++++++++++++++ .../libs/prx/libSceFont/src/Unimplemented.cpp | 20 ---------- core/libs/tests/GuestFont.cpp | 15 +++++++ docs/dev/TechnicalDebt.md | 1 + 5 files changed, 59 insertions(+), 20 deletions(-) diff --git a/core/libs/prx/libSceFont/include/FontInternal.hpp b/core/libs/prx/libSceFont/include/FontInternal.hpp index b65648a0d..b2a742c29 100644 --- a/core/libs/prx/libSceFont/include/FontInternal.hpp +++ b/core/libs/prx/libSceFont/include/FontInternal.hpp @@ -7,6 +7,7 @@ #include #include #include +#include #include #include #include @@ -28,6 +29,8 @@ struct FontState { FT_Face face = nullptr; float scaleW = 16.0f; float scaleH = 16.0f; + std::map scriptLanguages; + std::map typographicFeatures; FontState() = default; FontState(const FontState&) = delete; diff --git a/core/libs/prx/libSceFont/src/Style.cpp b/core/libs/prx/libSceFont/src/Style.cpp index e5503b886..67e7abe13 100644 --- a/core/libs/prx/libSceFont/src/Style.cpp +++ b/core/libs/prx/libSceFont/src/Style.cpp @@ -51,6 +51,30 @@ int ReadFontStyle(FontHandle fontHandle, bool checkMagic, Getter&& getter) { return rc; } +template +int AccessFontSettings(FontHandle fontHandle, Access&& access) { + return ReadFontStyle(fontHandle, true, [&](FontHandleNative*) { + FontState* state = TryGetState(fontHandle); + return state ? access(*state) : SCE_FONT_ERROR_INVALID_FONT_HANDLE; + }); +} + +int SetFontSetting(FontHandle fontHandle, std::map FontState::*table, int key, int value) { + return AccessFontSettings(fontHandle, [&](FontState& state) { + (state.*table)[key] = value; + return SCE_FONT_OK; + }); +} + +int GetFontSetting(FontHandle fontHandle, std::map FontState::*table, int key, int* value) { + if (!value) return SCE_FONT_ERROR_INVALID_PARAMETER; + return AccessFontSettings(fontHandle, [&](FontState& state) { + const auto found = (state.*table).find(key); + *value = found == (state.*table).end() ? 0 : found->second; + return SCE_FONT_OK; + }); +} + void ResetScaleOutputs(float* w, float* h) { if (w) *w = 0.0f; if (h) *h = 0.0f; @@ -77,6 +101,22 @@ float ClampWeightDelta(float scale) { extern "C" { +int APS5_VABI sceFontSetScriptLanguage(FontHandle fontHandle, int fontScript, int fontLanguage) { + return SetFontSetting(fontHandle, &FontState::scriptLanguages, fontScript, fontLanguage); +} + +int APS5_VABI sceFontGetScriptLanguage(FontHandle fontHandle, int fontScript, int* fontLanguage) { + return GetFontSetting(fontHandle, &FontState::scriptLanguages, fontScript, fontLanguage); +} + +int APS5_VABI sceFontSetTypographicDesign(FontHandle fontHandle, int typographic, int feature) { + return SetFontSetting(fontHandle, &FontState::typographicFeatures, typographic, feature); +} + +int APS5_VABI sceFontGetTypographicDesign(FontHandle fontHandle, int typographic, int* feature) { + return GetFontSetting(fontHandle, &FontState::typographicFeatures, typographic, feature); +} + int APS5_VABI sceFontSetScalePixel(FontHandle fontHandle, float w, float h) { return UpdateFontStyle(fontHandle, [&](FontHandleNative* font) { const int changed = StyleStateSetScalePixel(&font->style, w, h); diff --git a/core/libs/prx/libSceFont/src/Unimplemented.cpp b/core/libs/prx/libSceFont/src/Unimplemented.cpp index 87c3bbcd5..53b3e2ec9 100644 --- a/core/libs/prx/libSceFont/src/Unimplemented.cpp +++ b/core/libs/prx/libSceFont/src/Unimplemented.cpp @@ -88,16 +88,6 @@ int APS5_VABI sceFontGetGlyphExpandBufferState() { return 0; } -int APS5_VABI sceFontGetScriptLanguage() { - NotImplemented_nid_no_patch(__func__); - return 0; -} - -int APS5_VABI sceFontGetTypographicDesign() { - NotImplemented_nid_no_patch(__func__); - return 0; -} - int APS5_VABI sceFontGlyphGetAttribute() { NotImplemented_nid_no_patch(__func__); return 0; @@ -368,16 +358,6 @@ int APS5_VABI sceFontGraphicsUpdateShapeFillPlot() { return 0; } -int APS5_VABI sceFontSetScriptLanguage() { - NotImplemented_nid_no_patch(__func__); - return 0; -} - -int APS5_VABI sceFontSetTypographicDesign() { - NotImplemented_nid_no_patch(__func__); - return 0; -} - int APS5_VABI sceFontSupportGlyphs() { NotImplemented_nid_no_patch(__func__); return 0; diff --git a/core/libs/tests/GuestFont.cpp b/core/libs/tests/GuestFont.cpp index eb30bd2a2..f87745e78 100644 --- a/core/libs/tests/GuestFont.cpp +++ b/core/libs/tests/GuestFont.cpp @@ -26,6 +26,10 @@ int APS5_VABI sceFontSupportExternalFonts(FontLibrary, std::uint32_t, std::uint3 int APS5_VABI sceFontOpenFontSet(FontLibrary, std::uint32_t, std::uint32_t, const FontOpenDetail*, FontHandle*); int APS5_VABI sceFontOpenFontMemory(FontLibrary, const void*, std::uint32_t, const FontOpenDetail*, FontHandle*); int APS5_VABI sceFontCloseFont(FontHandle); +int APS5_VABI sceFontSetScriptLanguage(FontHandle, int, int); +int APS5_VABI sceFontGetScriptLanguage(FontHandle, int, int*); +int APS5_VABI sceFontSetTypographicDesign(FontHandle, int, int); +int APS5_VABI sceFontGetTypographicDesign(FontHandle, int, int*); int APS5_VABI sceFontSetResolutionDpi(FontHandle, std::uint32_t, std::uint32_t); int APS5_VABI sceFontGetResolutionDpi(FontHandle, std::uint32_t*, std::uint32_t*); int APS5_VABI sceFontSetScalePixel(FontHandle, float, float); @@ -203,6 +207,17 @@ int main() { const std::vector fontData = EmptyGlyphFont(); Require(sceFontOpenFontMemory(library, fontData.data(), static_cast(fontData.size()), nullptr, &font) == SCE_FONT_OK && font != nullptr); + int setting = -1; + Require(sceFontGetScriptLanguage(font, 3, &setting) == SCE_FONT_OK && setting == 0); + Require(sceFontSetScriptLanguage(font, 3, 7) == SCE_FONT_OK && sceFontSetScriptLanguage(font, 4, 9) == SCE_FONT_OK); + Require(sceFontGetScriptLanguage(font, 3, &setting) == SCE_FONT_OK && setting == 7); + Require(sceFontGetScriptLanguage(font, 4, &setting) == SCE_FONT_OK && setting == 9); + Require(sceFontGetScriptLanguage(font, 3, nullptr) == SCE_FONT_ERROR_INVALID_PARAMETER); + Require(sceFontSetScriptLanguage(nullptr, 3, 7) == SCE_FONT_ERROR_INVALID_FONT_HANDLE); + Require(sceFontSetTypographicDesign(font, 1, 2) == SCE_FONT_OK); + Require(sceFontGetTypographicDesign(font, 1, &setting) == SCE_FONT_OK && setting == 2); + Require(sceFontGetTypographicDesign(font, 5, &setting) == SCE_FONT_OK && setting == 0); + Require(sceFontGetTypographicDesign(nullptr, 1, &setting) == SCE_FONT_ERROR_INVALID_FONT_HANDLE); std::uint32_t hDpi = 1; std::uint32_t vDpi = 1; Require(sceFontGetResolutionDpi(font, &hDpi, &vDpi) == SCE_FONT_OK && hDpi == 72 && vDpi == 72); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 73ac7832e..5c391be32 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -39,6 +39,7 @@ Throughout the project, every function at every stage either **does exactly what - [ulobjmgr](../../core/libs/prx/ulobjmgr/Export.cpp) registers no object: `_sceUlobjmgrRegisterObject` always hands out id 0 and `_sceUlobjmgrUnregisterObject` releases nothing, as shadPS4 does - [libSceHttp](../../core/libs/prx/libSceHttp/Export.cpp) - no request reaches the network, so `sceHttpSetResponseHeaderMaxSize` has no response header to limit, `sceHttpSetRecvBlockSize` (`(int id, u32 blockSize)` as in [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.h)) no response body to read in blocks and `sceHttpRedirectCacheFlush` no redirect to forget; `sceHttpsUnloadCert` returns success like `sceHttpsLoadCert`, which keeps no certificate; `sceHttpSetRedirectCallback` and `sceHttpSetCookieRecvCallback` accept callbacks that are never called, `sceHttpsSetSslVersion` has no TLS connection to configure and `sceHttpsGetSslError` reports error 0 with detail 0; `sceHttpSetAuthInfoCallback` accepts a callback that is never called, as no server asks for authentication - [sceFontGlyphDefineAttribute](../../core/libs/prx/libSceFont/src/Render.cpp) (libSceFont) checks the glyph and ignores the attribute and its value, whose meaning is unknown. PPSA01325 sets attribute 0x11 to 0 on a glyph from `sceFontGenerateCharGlyph` that it deletes unused, and draws the character with `sceFontRenderCharGlyphImageHorizontal` +- [sceFontSetScriptLanguage, sceFontGetScriptLanguage, sceFontSetTypographicDesign, sceFontGetTypographicDesign](../../core/libs/prx/libSceFont/src/Style.cpp) (libSceFont) - signatures follow [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.cs), not confirmed for the PS5. The handle keeps the language set for each script and the feature set for each typographic design and returns them, with 0 for one never set; the console default and its valid ranges are unknown, so any value is accepted. Neither setting changes shaping or rendering yet ### Unknown function info From 1d298ad8b4351f9e1f3dbfb91de5ebcbd8d563ec Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 01:03:58 +0200 Subject: [PATCH 230/460] docs(debt): list the font script/typographic setters under Unknown function info Their signatures are inferred from SharpProspero only, so the line belongs with the other unverified signatures and not at the end of Silent stubs. Co-Authored-By: Claude --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 5c391be32..e5e83ab69 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -39,12 +39,12 @@ Throughout the project, every function at every stage either **does exactly what - [ulobjmgr](../../core/libs/prx/ulobjmgr/Export.cpp) registers no object: `_sceUlobjmgrRegisterObject` always hands out id 0 and `_sceUlobjmgrUnregisterObject` releases nothing, as shadPS4 does - [libSceHttp](../../core/libs/prx/libSceHttp/Export.cpp) - no request reaches the network, so `sceHttpSetResponseHeaderMaxSize` has no response header to limit, `sceHttpSetRecvBlockSize` (`(int id, u32 blockSize)` as in [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.h)) no response body to read in blocks and `sceHttpRedirectCacheFlush` no redirect to forget; `sceHttpsUnloadCert` returns success like `sceHttpsLoadCert`, which keeps no certificate; `sceHttpSetRedirectCallback` and `sceHttpSetCookieRecvCallback` accept callbacks that are never called, `sceHttpsSetSslVersion` has no TLS connection to configure and `sceHttpsGetSslError` reports error 0 with detail 0; `sceHttpSetAuthInfoCallback` accepts a callback that is never called, as no server asks for authentication - [sceFontGlyphDefineAttribute](../../core/libs/prx/libSceFont/src/Render.cpp) (libSceFont) checks the glyph and ignores the attribute and its value, whose meaning is unknown. PPSA01325 sets attribute 0x11 to 0 on a glyph from `sceFontGenerateCharGlyph` that it deletes unused, and draws the character with `sceFontRenderCharGlyphImageHorizontal` -- [sceFontSetScriptLanguage, sceFontGetScriptLanguage, sceFontSetTypographicDesign, sceFontGetTypographicDesign](../../core/libs/prx/libSceFont/src/Style.cpp) (libSceFont) - signatures follow [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.cs), not confirmed for the PS5. The handle keeps the language set for each script and the feature set for each typographic design and returns them, with 0 for one never set; the console default and its valid ranges are unknown, so any value is accepted. Neither setting changes shaping or rendering yet ### Unknown function info - [sceVideoOutGetOutputStatusWithVrr](../../core/libs/prx/libSceVideoOut/Unimplemented.cpp) (libSceVideoOut, NID `gWT7X8H0bYs`) - the name hashes to the imported NID. PPSA26344 version 02.002.000 passes a VideoOut handle and an output pointer at three call sites. It compares a 32-bit value at byte 4 with 2, tests bit 0x10 at byte 16, and, when set, compares low 16-bit values at bytes 48, 50 and 52 with 48, 60 and 120. The full output layout, size, field meanings and console errors are unknown. Every call throws without reading or writing caller memory. - [getuid, geteuid, getgid, getegid and issetugid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) report the root identity (0) and a process that never changed credentials (`issetugid` 0), as a jailbroken process has; the host user and group are not exposed to the guest. The values a title sees on a console are unverified +- [sceFontSetScriptLanguage, sceFontGetScriptLanguage, sceFontSetTypographicDesign, sceFontGetTypographicDesign](../../core/libs/prx/libSceFont/src/Style.cpp) (libSceFont) - signatures follow [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.cs), not confirmed for the PS5. The handle keeps the language set for each script and the feature set for each typographic design and returns them, with 0 for one never set; the console default and its valid ranges are unknown, so any value is accepted. Neither setting changes shaping or rendering yet - [Packed pixel ancillary word](../../core/shader/recompiler/Optimization/src/ConstantFolder.cpp) (shader recompiler) - the sample index at bits 8-11 and the render target index at bits 16-28 follow #295, and their equality with the SampleId and Layer builtins is inferred, not measured. A value passed through a select, move or phi is rebuilt from those two fields with bits 0-7, 12-15 and 29-31 zero. That is exact only because every read of the word must be an extract inside one modelled field; any other read throws. The other bits are unmodelled and their RDNA 2 layout is unknown: the RDNA 3.5 ISA is reported to give prim type at bits 0-1, VRS rates at bits 2-5 and eye id at bit 7, which was not checked against RDNA 2 - [sceAudioPropagationSourceSetAudioPath](../../core/libs/prx/libSceAudioPropagation/Export.cpp) (libSceAudioPropagation, NID `tKSmk2JsMAA`) - `(uint64_t path, const void* data, float gain)` follows [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/5dfdb4b0b166e83dc9ae275001afb695f12c7174/src/SharpProspero/Interop/Audio/AudioPropagation.cs) and SAROS's call, which passes a handle from `sceAudioPropagationSourceGetAudioPath`, path data and gain 1.0. Path updates are unsupported and every call throws without reading or writing caller memory. The no-propagation model yields no path handles; path allocation, ownership, rendering and console error codes remain unknown. - [Color comparison descriptor forms](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp): emulated color comparison supports UNORM, SNORM and float formats for 2D, 2D-array and cube views. sRGB formats, anisotropic filters and bilinear cube comparisons without `DISABLE_CUBE_WRAP` are refused; cube-edge sampling behavior and console coverage on PS5's gfx1013 remain unverified. From 70d7aa58e3a30f4f360d0233feb46fbed5c360a9 Mon Sep 17 00:00:00 2001 From: XairRrR Date: Sun, 11 Oct 2026 01:04:12 +0200 Subject: [PATCH 231/460] perf(agc-driver): import adjacent registered ranges as one host-import span A host import covered exactly one registered range, so a buffer crossing two ranges had no import and was copied on the CPU at every use, under the GPU mutex. Titles that map a GPU heap piece by piece (Uncharted maps 1 MiB direct-memory pieces) hit this for every buffer larger than a piece. An import now covers the run of address-adjacent registered ranges that are readable with the same write and GPU access. Runs never overlap, so imports do not either; an import is retired when any range of its run changes, and a refused run is remembered by its base. Callers already address imports relative to their base. --- .../Graphics/include/GuestBufferMemory.hpp | 1 + .../Graphics/src/GuestBufferMemory.cpp | 90 ++++++++++++++++--- .../libSceAgcDriver/tests/BdaResources.cpp | 38 +++++++- 3 files changed, 112 insertions(+), 17 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp index f4585fcd6..f22566c1c 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp @@ -28,6 +28,7 @@ struct HostImport { VkDeviceAddress address; void* alias = nullptr; std::shared_ptr range {}; + std::vector> ranges {}; // Identity for the life of this import (see HostImportSerial); 0 until first asked for. std::uint64_t serial = 0; bool unwatched = false; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp index 1c9a8125f..b19fdecd0 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp @@ -192,12 +192,13 @@ struct RetiredImport { // so the objects live until the batch open now completed; with no batches in flight all GPU work that // used it was synchronous and it is destroyed at once. const GuestAllocations::Range* containingRange(const GuestAllocations::Lease& lease, std::uint64_t begin, std::uint64_t end); +bool runCovers(const GuestAllocations::Lease& lease, std::uint64_t begin, std::uint64_t end); void retireImport(VkDevice device, HostImports& state, std::map::iterator it, const GuestAllocations::Lease& lease) { // Results shadowed for the import reach its (old) buffer first where the memory is still a // readable registered range (the import retires because its registration vanished or changed // size); the holder below outlives the batch that copies them. - RetireShadow(device, it->second, [&lease](std::uint64_t begin, std::uint64_t end) { return containingRange(lease, begin, end) != nullptr; }); + RetireShadow(device, it->second, [&lease](std::uint64_t begin, std::uint64_t end) { return runCovers(lease, begin, end); }); auto holder = std::make_shared(state.device, state.destroyBuffer, state.freeMemory, it->second); if (auto* recorder = Recorder::Active(); recorder != nullptr && !recorder->Idle()) recorder->Keep(std::move(holder)); ++state.epoch; @@ -520,16 +521,74 @@ std::shared_ptr leasedRangeOwner(const GuestAlloc return found != lease.end() && (*found)->address == base ? *found : nullptr; } -bool sameRange(const HostImport& entry, const GuestAllocations::Lease& lease) { - const auto current = leasedRangeOwner(lease, entry.base); - return current != nullptr && !entry.range.owner_before(current) && !current.owner_before(entry.range); +struct ImportRun { + std::uint64_t base = 0; + std::uint64_t bytes = 0; + std::vector> ranges; +}; + +bool continuesRun(const GuestAllocations::Range& before, const GuestAllocations::Range& after) { + return before.address + before.bytes == after.address && before.readable && after.readable && before.writable == after.writable && before.gpu == after.gpu; } -const HostImport* importAllocation(const Context& context, HostImports& state, std::uint64_t base, std::uint64_t bytes, const GuestAllocations::Lease& lease) { +std::size_t leasedIndexContaining(const GuestAllocations::Lease& lease, std::uint64_t address) { + const auto found = std::upper_bound(lease.begin(), lease.end(), address, [](std::uint64_t value, const auto& range) { return value < range->address; }); + if (found == lease.begin()) return lease.size(); + const auto index = static_cast(std::prev(found) - lease.begin()); + const auto& range = *lease[index]; + return range.readable && address < range.address + range.bytes ? index : lease.size(); +} + +ImportRun importRun(const GuestAllocations::Lease& lease, std::uint64_t address) { + const auto index = leasedIndexContaining(lease, address); + if (index == lease.size()) return {}; + auto first = index; + while (first > 0 && continuesRun(*lease[first - 1], *lease[first])) --first; + auto last = index; + while (last + 1 < lease.size() && continuesRun(*lease[last], *lease[last + 1])) ++last; + ImportRun run{lease[first]->address, lease[last]->address + lease[last]->bytes - lease[first]->address, {}}; + run.ranges.assign(lease.begin() + static_cast(first), lease.begin() + static_cast(last) + 1); + return run; +} + +bool runCovers(const GuestAllocations::Lease& lease, std::uint64_t begin, std::uint64_t end) { + const auto run = importRun(lease, begin); + return !run.ranges.empty() && end <= run.base + run.bytes; +} + +bool sameRun(const HostImport& entry, const ImportRun& run) { + if (run.base != entry.base || run.bytes != entry.bytes || run.ranges.size() != entry.ranges.size()) return false; + for (std::size_t index = 0; index < run.ranges.size(); ++index) { + if (entry.ranges[index].owner_before(run.ranges[index]) || run.ranges[index].owner_before(entry.ranges[index])) return false; + } + return true; +} + +bool sameRange(const HostImport& entry, const GuestAllocations::Lease& lease) { + return sameRun(entry, importRun(lease, entry.base)); +} + +const HostImport* findImport(HostImports& state, std::uint64_t begin, std::uint64_t end); +bool importsStale(const Context& context, const HostImports& state); + +const HostImport* importAllocation(const Context& context, HostImports& state, std::uint64_t address, const GuestAllocations::Lease& lease) { + if (!importsStale(context, state)) { + if (const auto* current = findImport(state, address, address + 1)) return current; + } + auto run = importRun(lease, address); + if (run.ranges.empty()) return nullptr; + const auto base = run.base; + const auto bytes = run.bytes; if (const auto found = state.imports.find(base); found != state.imports.end()) { - if (found->second.bytes == bytes && sameRange(found->second, lease)) return &found->second; + if (sameRun(found->second, run)) return &found->second; retireImport(context.device, state, found, lease); } + for (auto it = state.imports.lower_bound(base); it != state.imports.end() && it->first < base + bytes;) { + const auto next = std::next(it); + retireImport(context.device, state, it, lease); + it = next; + } + if (auto it = state.imports.lower_bound(base); it != state.imports.begin() && std::prev(it)->second.base + std::prev(it)->second.bytes > base) retireImport(context.device, state, std::prev(it), lease); const auto alignment = context.hostImportAlignment; if (alignment == 0 || base % alignment != 0 || bytes % alignment != 0 || state.failed.contains(base)) return nullptr; // Pinned imports count against the driver's system memory budget; past it ordinary host @@ -553,7 +612,8 @@ const HostImport* importAllocation(const Context& context, HostImports& state, s return nullptr; } HostImport entry{base, bytes, VK_NULL_HANDLE, VK_NULL_HANDLE, 0}; - entry.range = leasedRangeOwner(lease, base); + entry.range = run.ranges.front(); + entry.ranges = std::move(run.ranges); #ifdef _WIN32 // Drivers pin imported pages, so every page must be committed and accessible. bool writable = true; @@ -684,7 +744,7 @@ void refreshImports(const Context& context, HostImports& state, const GuestAlloc state.refreshedGeneration = generation; for (auto it = state.imports.begin(); it != state.imports.end();) { const auto* range = leasedRangeAt(lease, it->first); - if (range != nullptr && range->bytes == it->second.bytes && sameRange(it->second, lease)) { + if (range != nullptr && sameRange(it->second, lease)) { if (it->second.unwatched) GuestMemory::Unwatch(it->first, it->second.bytes); ++it; continue; @@ -1680,8 +1740,9 @@ const HostImport* HostImportFor(const Context& context, std::uint64_t address, s } const auto lease = GuestAllocations::GuestAllocationsAcquire_nid_postfix(); refreshImports(context, state, lease); - if (const auto* range = containingRange(lease, address, address + bytes)) return importAllocation(context, state, range->address, range->bytes, lease); - return nullptr; + if (!runCovers(lease, address, address + bytes)) return nullptr; + const auto* entry = importAllocation(context, state, address, lease); + return entry != nullptr && findImport(state, address, address + bytes) == entry ? entry : nullptr; } bool ImportMappedRanges(const Context& context, const GuestAllocations::Mapped& ranges, std::uint64_t generation, bool adoptDevice) { @@ -1698,11 +1759,12 @@ bool ImportMappedRanges(const Context& context, const GuestAllocations::Mapped& } else if (state.refreshedGeneration > generation) { return true; } + if (!adopting) current = GuestAllocations::GuestAllocationsAcquire_nid_postfix(); for (const auto& mapped : ranges) { const auto range = mapped.lock(); - if (range == nullptr || (adopting && leasedRangeOwner(current, range->address) != range)) continue; + if (range == nullptr || leasedRangeOwner(current, range->address) != range) continue; try { - if (importAllocation(context, state, range->address, range->bytes, GuestAllocations::Lease{range}) == nullptr && GuestAllocations::GuestAllocationsGeneration_nid_postfix() != generation) state.failed.erase(range->address); + if (importAllocation(context, state, range->address, current) == nullptr && GuestAllocations::GuestAllocationsGeneration_nid_postfix() != generation) state.failed.erase(importRun(current, range->address).base); } catch (const std::exception& error) { std::fprintf(stderr, "[gpu] import of GPU memory mapped at 0x%llx+0x%zx failed: %s\n", static_cast(range->address), range->bytes, error.what()); } @@ -1845,7 +1907,7 @@ void GuestBufferMemory::AcquireRegistered() { if (importable) { auto& state = Imports(); std::lock_guard lock(state.mutex); - entry = importAllocation(context, state, range->address, range->bytes, lease); + entry = importAllocation(context, state, range->address, lease); } lap(timing.importsUs, at); if (entry != nullptr) { @@ -2572,7 +2634,7 @@ void GuestBufferMemory::UploadFinish(bool addressable) { region.direct = nullptr; if (entry == nullptr) entry = findImport(state, region.begin, region.end); if (entry == nullptr) { - if (const auto* range = containingRange(importRanges(), region.begin, region.end)) entry = importAllocation(context, state, range->address, range->bytes, importRanges()); + if (runCovers(importRanges(), region.begin, region.end)) entry = importAllocation(context, state, region.begin, importRanges()); } // A staged region (see stagingEligible) is copied out of the import even when aligned; // without a recorder to record the copies it binds in place like any other. diff --git a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp index 037991ce5..7591d6b96 100644 --- a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp @@ -71,15 +71,16 @@ void importCrossingTests(const Context& context, const BdaTestAccess& access) { const auto crossing = leased.Descriptor(second - 16, 32, adjustment); const auto crossingBytes = access.bytes(crossing.buffer); Require(crossing.offset + crossing.range <= crossingBytes.size(), "a view crossing into the next registered range is bound past its host import"); - Require(crossingBytes[crossing.offset + adjustment + 15] == std::byte{0x11} && crossingBytes[crossing.offset + adjustment + 16] == std::byte{0x22}, "a view crossing into the next registered range misses its bytes"); + Require(secondImport.base == first && secondImport.bytes == 2 * half, "adjacent registered ranges with the same access were not imported as one span"); + Require(crossing.buffer == secondImport.buffer && crossing.offset + adjustment == second - 16 - secondImport.base, "a view crossing into the next registered range is not bound in place in the spanning import"); const auto inside = leased.Descriptor(second + 64, 16, adjustment); const auto insideBytes = access.bytes(inside.buffer); - Require(inside.offset + inside.range <= insideBytes.size() && (inside.buffer == secondImport.buffer ? inside.offset + adjustment == 64 : insideBytes[inside.offset + adjustment] == std::byte{0x22}), "a view in the second registered range is bound past its buffer or misses its bytes"); + Require(inside.offset + inside.range <= insideBytes.size() && (inside.buffer == secondImport.buffer ? inside.offset + adjustment == second + 64 - secondImport.base : insideBytes[inside.offset + adjustment] == std::byte{0x22}), "a view in the second registered range is bound past its buffer or misses its bytes"); for (const auto& range : leased.AddressRanges()) { if (range.end <= first || range.begin >= second + half) continue; const auto mapped = access.addressBytes(range.deviceAddress); Require(mapped.size() >= range.end - range.begin, "a BDA range runs past the end of its host import"); - if (range.begin <= second && second < range.end && range.deviceAddress != secondImport.address) Require(mapped[second - range.begin] == std::byte{0x22}, "the BDA range over the second registered range misses its bytes"); + if (range.begin <= second && second < range.end && (range.deviceAddress < secondImport.address || range.deviceAddress >= secondImport.address + secondImport.bytes)) Require(mapped[second - range.begin] == std::byte{0x22}, "the BDA range over the second registered range misses its bytes"); } leased.WriteBack(); } @@ -88,6 +89,36 @@ void importCrossingTests(const Context& context, const BdaTestAccess& access) { ::operator delete(block, std::align_val_t{half}); } +void importSpanTests(const Context& context) { + constexpr std::size_t piece = 65536; + void* block = ::operator new(3 * piece, std::align_val_t{piece}); + auto* guest = static_cast(block); + const auto first = reinterpret_cast(block); + auto importing = context; + importing.hostImportAlignment = piece; + const auto change = [&](std::uint8_t* range, bool add, bool writable) { + auto* mutation = GuestAllocations::GuestAllocationsBegin_nid_postfix(); + if (add) GuestAllocations::GuestAllocationsAdd_nid_postfix(mutation, range, piece, true, writable, true); + else GuestAllocations::GuestAllocationsRemove_nid_postfix(mutation, range); + GuestAllocations::GuestAllocationsEnd_nid_postfix(mutation); + }; + change(guest, true, false); + change(guest + piece, true, false); + change(guest + 2 * piece, true, true); + const auto* span = HostImportFor(importing, first + piece - 16, 32); + Require(span != nullptr && span->base == first && span->bytes == 2 * piece, "adjacent read-only ranges were not imported as one span"); + const auto* writable = HostImportFor(importing, first + 2 * piece, piece); + Require(writable != nullptr && writable->base == first + 2 * piece && writable->bytes == piece, "a writable range joined the read-only span"); + Require(HostImportFor(importing, first + 2 * piece - 16, 32) == nullptr, "a view crossing ranges of different access was imported"); + change(guest + piece, false, false); + const auto* alone = HostImportFor(importing, first, piece); + Require(alone != nullptr && alone->base == first && alone->bytes == piece, "the span outlived the removal of one of its ranges"); + Require(HostImportFor(importing, first + piece, 16) == nullptr, "the removed range is still imported"); + change(guest, false, false); + change(guest + 2 * piece, false, false); + ::operator delete(block, std::align_val_t{piece}); +} + ShaderRecompiler::DescriptorBinding binding(Role role, std::uint32_t slot) { return {ShaderRecompiler::DescriptorKind::StorageBuffer, role, 0, slot, 1, {}, false}; } @@ -726,6 +757,7 @@ void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { if (dropped.enabled) Require(dropped.waiterDrops == after.waiterDrops + 1 && LeaseCounters().cacheDrops == dropped.waiterDrops, "the free did not drop the cached address space"); } importCrossingTests(context, access); + importSpanTests(context); heapMirrorTests(context, access); importedHeapMirrorTests(context, access); importWatchTests(); From e5ae8dfcb07c44522a860c2dd5b8200f3d1483dd Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sat, 10 Oct 2026 18:04:35 -0500 Subject: [PATCH 232/460] fix(libkernel): keep the alarm state alive until the process exits The detached alarm thread waits on a condition variable that was a namespace-scope static. At exit its destructor ran with the thread still waiting, and glibc's pthread_cond_destroy blocks then, so a process that called alarm never exited on Linux. The mutex, condition variable, deadline and started flag now live in one structure allocated once and never destroyed. AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- .../libs/prx/libkernel/System/src/Signals.cpp | 45 +++++++++++-------- 1 file changed, 27 insertions(+), 18 deletions(-) diff --git a/core/libs/prx/libkernel/System/src/Signals.cpp b/core/libs/prx/libkernel/System/src/Signals.cpp index 38de618a2..8650e8f7f 100644 --- a/core/libs/prx/libkernel/System/src/Signals.cpp +++ b/core/libs/prx/libkernel/System/src/Signals.cpp @@ -82,10 +82,17 @@ bool Install(int guest, const GuestSigaction& action) { namespace { constexpr int GuestSigalrm = 14; -std::mutex alarmLock; -std::condition_variable alarmChanged; -std::optional alarmDeadline; -bool alarmThreadStarted = false; +struct AlarmState { + std::mutex lock; + std::condition_variable changed; + std::optional deadline; + bool threadStarted = false; +}; + +AlarmState& Alarms() { + static auto* state = new AlarmState(); + return *state; +} void ExpireAlarm() { std::uintptr_t handler = 0; @@ -103,15 +110,16 @@ void ExpireAlarm() { } void RunAlarms() { - std::unique_lock lock(alarmLock); + auto& alarms = Alarms(); + std::unique_lock lock(alarms.lock); for (;;) { - if (!alarmDeadline) { - alarmChanged.wait(lock); + if (!alarms.deadline) { + alarms.changed.wait(lock); continue; } - const auto deadline = *alarmDeadline; - if (alarmChanged.wait_until(lock, deadline) != std::cv_status::timeout || alarmDeadline != deadline) continue; - alarmDeadline.reset(); + const auto deadline = *alarms.deadline; + if (alarms.changed.wait_until(lock, deadline) != std::cv_status::timeout || alarms.deadline != deadline) continue; + alarms.deadline.reset(); lock.unlock(); ExpireAlarm(); lock.lock(); @@ -120,20 +128,21 @@ void RunAlarms() { } extern "C" unsigned int APS5_VABI GuestAlarm_nid_no_patch(unsigned int seconds) { - std::lock_guard lock(alarmLock); + auto& alarms = Alarms(); + std::lock_guard lock(alarms.lock); const auto now = std::chrono::steady_clock::now(); unsigned int remaining = 0; - if (alarmDeadline && *alarmDeadline > now) { - const auto micros = std::chrono::duration_cast(*alarmDeadline - now).count(); + if (alarms.deadline && *alarms.deadline > now) { + const auto micros = std::chrono::duration_cast(*alarms.deadline - now).count(); remaining = static_cast((micros + 999999) / 1000000); } - if (seconds == 0) alarmDeadline.reset(); - else alarmDeadline = now + std::chrono::seconds(seconds); - if (!alarmThreadStarted) { + if (seconds == 0) alarms.deadline.reset(); + else alarms.deadline = now + std::chrono::seconds(seconds); + if (!alarms.threadStarted) { std::thread(RunAlarms).detach(); - alarmThreadStarted = true; + alarms.threadStarted = true; } - alarmChanged.notify_all(); + alarms.changed.notify_all(); return remaining; } From fb45627bb79562de6721862bbb76ac7fe3284f79 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 12:17:08 -0400 Subject: [PATCH 233/460] fix(agc): retain registered headers sharing shader code Keep each registered header and its prepared shader state instead of replacing every registration for the same code address. Resolve explicit ABIs by header and programs by type, preserving re-registration order. Co-Authored-By: GPT-6.1-Sol --- .../include/Driver/Shaders/ShaderRegistry.hpp | 4 +- .../src/Driver/Dispatch/Dispatch.cpp | 7 +- .../Execution/src/Driver/Draw/DrawDecode.cpp | 18 ++--- .../src/Driver/Draw/DrawRegisterKey.cpp | 4 +- .../src/Driver/Shaders/ShaderRegistry.cpp | 73 +++++++++++++++---- .../tests/execution/PreparedGraphics.cpp | 6 +- .../tests/execution/PreparedPublication.cpp | 56 ++++++++++++-- .../tests/execution/PreparedShaders.cpp | 63 ++++++++++++++++ 8 files changed, 185 insertions(+), 46 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp index 3ce4f820d..ac19eec56 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -27,7 +28,7 @@ class VulkanDevice; namespace AgcDriver::DriverDetail { struct ShaderSnapshot; -using ShaderRegistry = std::map>; +using ShaderRegistry = std::map>>; struct PreparedShaderState { struct Entry { @@ -89,6 +90,7 @@ std::shared_ptr PrepareShaderWithDiagnosti std::uint64_t NullPixelProgramAddress(); ShaderSnapshot PrepareNullPixelProgram(const VulkanDevice& device); std::optional RegisteredFloatMode(const ShaderSnapshot& snapshot); +std::shared_ptr RegisteredProgram(const ShaderRegistry& registry, std::uint64_t address, std::initializer_list types); void PublishRegisteredShader(std::shared_ptr& registry, const std::shared_ptr& snapshot); void ResolvePreparedGraphics(const ShaderSnapshot& front, const std::shared_ptr& fragment, std::uint32_t primitiveType, const ShaderRecompiler::SpirvTarget& target); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp index cb71f04dd..dcfcc848f 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp @@ -16,12 +16,7 @@ namespace AgcDriver::DriverDetail { void Driver::dispatch(QueueState& queue, std::span packet, const Submission& submission, std::uint64_t indirectArguments) { PerformanceTimer timing("Driver.Dispatch"); const auto address = (static_cast(readRegister(queue.shader, 0x20c)) << 8u) | (static_cast(readRegister(queue.shader, 0x20d) & 0xffu) << 40u); - auto it = submission.shaders->upper_bound(address); - std::shared_ptr registeredShader; - if (it != submission.shaders->begin()) { - --it; - if (address - it->second->codeAddress < it->second->code.size() * sizeof(std::uint32_t)) registeredShader = it->second; - } + auto registeredShader = RegisteredProgram(*submission.shaders, address, {0}); if (!registeredShader) registeredShader = ReadRawComputeShader(address); const auto& snapshot = *registeredShader; require(snapshot.type == 0, "compute program refers to a non-compute shader"); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp index a34853b83..f4444a808 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp @@ -28,13 +28,10 @@ void DecodeGraphicsPrograms(DrawDecode& decoded, const QueueState& queue, const const auto prepare = [&](std::uint64_t address, std::uint8_t type, Stage stage, std::uint32_t rsrc2, std::uint32_t userDataBase) { const bool nullPixel = stage == Stage::Fragment && (address == 0 || pixelSkipped); if (nullPixel) address = NullPixelProgramAddress(); - auto it = registry.upper_bound(address); - require(it != registry.begin(), "graphics program does not belong to a registered shader"); - --it; - const auto& snapshot = *it->second; - require(address - snapshot.codeAddress < snapshot.code.size() * sizeof(std::uint32_t), "graphics program is outside registered shader code"); + const auto registered = RegisteredProgram(registry, address, {type}); + require(registered != nullptr, "graphics program does not belong to a compatible registered shader"); + const auto& snapshot = *registered; require((address - snapshot.codeAddress) % sizeof(std::uint32_t) == 0, "graphics entry point is not dword aligned"); - require(snapshot.type == type, "graphics program refers to an incompatible shader binary type"); if (!nullPixel) Graphics::NoteRegisterRead(Graphics::RegisterBank::Shader, rsrc2); const auto resources = nullPixel ? 0u : ReadGraphicsRegister(queue.shader, rsrc2); const auto userCount = ((resources >> 1u) & 0x1fu) | (((resources >> 27u) & 1u) << 5u); @@ -46,7 +43,7 @@ void DecodeGraphicsPrograms(DrawDecode& decoded, const QueueState& queue, const 8, {}, {{{snapshot.codeAddress, std::as_bytes(std::span(snapshot.code))}, {snapshot.headerAddress, snapshot.header}}}, - it->second, + registered, codeOffset }; for (std::uint32_t i = 0; i < userCount; ++i) { @@ -89,10 +86,9 @@ void DecodeGraphicsPrograms(DrawDecode& decoded, const QueueState& queue, const append(0x0c8, 2, Stage::TessellationEvaluation, 0x08b, 0x08c, Role::Domain); } else if (graphics.stages.path == Graphics::ShaderPath::Geometry) { const auto frontAddress = programAddress(0xc8); - auto snapshot = registry.upper_bound(frontAddress); - require(snapshot != registry.begin(), "geometry front program is not registered"); - --snapshot; - const auto type = snapshot->second->type; + const auto snapshot = RegisteredProgram(registry, frontAddress, {2, 4}); + require(snapshot != nullptr, "geometry front program is not registered"); + const auto type = snapshot->type; require(type == 2 || type == 4, "invalid geometry front binary type"); append(0xc8, type, Stage::Mesh, 0x8b, 0x8c, Role::Main); initializeMerged(programs.back(), 0x82, type == 4); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawRegisterKey.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawRegisterKey.cpp index 36481c410..1759f417e 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawRegisterKey.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawRegisterKey.cpp @@ -50,8 +50,8 @@ std::uint64_t Driver::drawRegisterKey(const QueueState& queue, const ShaderRegis continue; } --it; - mix(reinterpret_cast(it->second.get())); - mix(address - it->second->codeAddress); + for (const auto& snapshot : it->second) mix(reinterpret_cast(snapshot.get())); + mix(address - it->first); } return key; } diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp index 52a46d03b..29db99a99 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp @@ -207,20 +207,57 @@ bool SameHeader(const ShaderSnapshot& snapshot, const Shader* shader) { return std::memcmp(&used, ®istered, offsetof(Shader, num_sh_registers) + sizeof(Shader::num_sh_registers)) == 0; } +std::shared_ptr RegisteredHeader(const ShaderRegistry& registry, const Shader* shader) { + const auto found = registry.find(reinterpret_cast(const_cast(shader->code))); + if (found == registry.end()) return {}; + for (const auto& snapshot : found->second) { + if (snapshot->headerAddress == reinterpret_cast(shader)) return snapshot; + } + for (const auto& snapshot : found->second) { + if (SameHeader(*snapshot, shader)) return snapshot; + } + return {}; +} + +std::shared_ptr RegisteredProgram(const ShaderRegistry& registry, std::uint64_t address, std::initializer_list types) { + auto found = registry.upper_bound(address); + if (found == registry.begin()) return {}; + --found; + std::shared_ptr result; + bool registered = false; + for (const auto& snapshot : found->second) { + if (address - snapshot->codeAddress >= snapshot->code.size() * sizeof(std::uint32_t)) continue; + registered = true; + if (std::ranges::find(types, snapshot->type) == types.end()) continue; + require(result == nullptr || result->type == snapshot->type, "registered program has ambiguous shader binary types"); + result = snapshot; + } + require(!registered || result != nullptr, "registered program refers to an incompatible shader binary type"); + return result; +} + void PublishRegisteredShader(std::shared_ptr& registry, const std::shared_ptr& snapshot) { ShaderPreparationTransaction transaction; + auto published = snapshot; if (registry != nullptr) { const auto found = registry->find(snapshot->codeAddress); if (found != registry->end()) { - const auto& current = *found->second; - if (current.headerAddress == snapshot->headerAddress && current.type == snapshot->type && current.code == snapshot->code && current.header == snapshot->header) { - transaction.Commit(); - return; + for (const auto& current : found->second) { + if (current->headerAddress == snapshot->headerAddress && current->type == snapshot->type && current->code == snapshot->code && current->header == snapshot->header) { + if (current == found->second.back()) { + transaction.Commit(); + return; + } + published = current; + break; + } } } } auto next = registry != nullptr ? std::make_shared(*registry) : std::make_shared(); - next->insert_or_assign(snapshot->codeAddress, snapshot); + auto& headers = (*next)[snapshot->codeAddress]; + std::erase_if(headers, [&](const auto& current) { return current->headerAddress == snapshot->headerAddress || current->code != snapshot->code; }); + headers.push_back(published); registry = std::move(next); transaction.Commit(); } @@ -681,9 +718,10 @@ void Driver::ResolveGraphicsStagesAbi(std::span stages, std GuestMemory::CheckRange(shader, sizeof(Shader), alignof(Shader)); const auto address = reinterpret_cast(const_cast(shader->code)); require(registry != nullptr && registry->contains(address), "graphics ABI refers to an unregistered shader"); - if (owner == nullptr) owner = registry->at(address); - const auto& snapshot = *registry->at(address); - require(SameHeader(snapshot, shader), "graphics ABI refers to a replaced shader header"); + const auto selected = RegisteredHeader(*registry, shader); + require(selected != nullptr, "graphics ABI refers to a replaced shader header"); + if (owner == nullptr) owner = selected; + const auto& snapshot = *selected; require(snapshot.registeredState != nullptr, "registered shader state is missing"); const auto& registered = *snapshot.registeredState; for (const auto& [offset, value] : registered.shader) state.shader.insert_or_assign(offset, value); @@ -747,8 +785,8 @@ void Driver::ResolveShaderAbi(const Shader* shader, std::span(const_cast(shader->code)); require(shaders != nullptr && shaders->contains(address), "static ABI refers to an unregistered shader"); - snapshot = shaders->at(address); - require(SameHeader(*snapshot, shader), "static ABI refers to a replaced shader header"); + snapshot = RegisteredHeader(*shaders, shader); + require(snapshot != nullptr, "static ABI refers to a replaced shader header"); } require(snapshot->registeredState != nullptr, "registered shader state is missing"); QueueState state{}; @@ -809,8 +847,8 @@ void Driver::ResolveGraphicsAbi(const Shader* vertex, const Shader* pixel, std:: const auto lookup = [&](const Shader* shader) { const auto address = reinterpret_cast(const_cast(shader->code)); require(shaders != nullptr && shaders->contains(address), "rectangle ABI refers to an unregistered shader"); - const auto snapshot = shaders->at(address); - require(SameHeader(*snapshot, shader), "rectangle ABI refers to a replaced shader header"); + const auto snapshot = RegisteredHeader(*shaders, shader); + require(snapshot != nullptr, "rectangle ABI refers to a replaced shader header"); return snapshot; }; front = lookup(vertex); @@ -926,10 +964,13 @@ void Driver::RegisterShader(const Shader* shader) { if (shaders != nullptr) { const auto found = shaders->find(snapshot.codeAddress); if (found != shaders->end()) { - const auto& current = *found->second; - if (current.headerAddress == snapshot.headerAddress && current.type == snapshot.type && current.code == snapshot.code && current.header == snapshot.header) { - transaction.Commit(); - return; + for (const auto& current : found->second) { + if (current->headerAddress == snapshot.headerAddress && current->type == snapshot.type && current->code == snapshot.code && current->header == snapshot.header) { + const auto identical = current; + PublishRegisteredShader(shaders, identical); + transaction.Commit(); + return; + } } } } diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedGraphics.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedGraphics.cpp index eb1a511a1..e77a4d2e2 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedGraphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedGraphics.cpp @@ -80,7 +80,7 @@ void Check(AgcDriver::VulkanDevice& device, AgcDriver::Graphics::ShaderPath path fragment.snapshot->code.resize(64); fragment.snapshot->code.insert(fragment.snapshot->code.end(), pixelCode.begin(), pixelCode.end()); ShaderRegistry registry; - for (const auto* fixture : {&front, &back, &domain, &fragment}) registry.emplace(fixture->snapshot->codeAddress, fixture->snapshot); + for (const auto* fixture : {&front, &back, &domain, &fragment}) registry.emplace(fixture->snapshot->codeAddress, std::vector>{fixture->snapshot}); AgcDriver::QueueState queue{}; queue.context[0x8e] = 0xfu; queue.context[0x8f] = 0xfu; @@ -256,8 +256,8 @@ void NullPixelMatchesRegistration(AgcDriver::VulkanDevice& device) { const auto registered = null->prepared->entries; Require(!registered.empty(), "the null pixel program was not prepared at registration"); ShaderRegistry registry; - registry.emplace(front.snapshot->codeAddress, front.snapshot); - registry.emplace(null->codeAddress, null); + registry.emplace(front.snapshot->codeAddress, std::vector>{front.snapshot}); + registry.emplace(null->codeAddress, std::vector>{null}); AgcDriver::QueueState queue{}; queue.context[0x8e] = 0xfu; queue.context[0x8f] = 0xfu; diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedPublication.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedPublication.cpp index 99f9c45a3..5c5260c34 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedPublication.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedPublication.cpp @@ -1,4 +1,5 @@ #include "VulkanTestDevice.hpp" +#include "SceShaders.hpp" #include "prx/libSceAgcDriver/Execution/include/ShaderPreparation.hpp" #include "prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp" #include "Optimization/ResourceProgram.hpp" @@ -6,6 +7,7 @@ #include #include #include +#include #include #include @@ -106,6 +108,45 @@ void Transactions() { Require(!partial && front.prepared->entries.size() == 200 && pixel->prepared->entries.size() == 200, "concurrent group publication was partial or lost an update"); } +void HeaderSelection() { + const auto makeSnapshot = [](std::uint64_t headerAddress, std::uint8_t type) { + auto snapshot = std::make_shared(); + snapshot->codeAddress = 0x10000; + snapshot->headerAddress = headerAddress; + snapshot->type = type; + snapshot->code = {0xbf810000u}; + Shader header{}; + header.code = reinterpret_cast(snapshot->codeAddress); + header.type = type; + snapshot->header.resize(sizeof(Shader)); + std::memcpy(snapshot->header.data(), &header, sizeof(Shader)); + return snapshot; + }; + const auto vertex = makeSnapshot(0x20000, 2); + const auto compute = makeSnapshot(0x30000, 0); + std::shared_ptr registry; + PublishRegisteredShader(registry, vertex); + PublishRegisteredShader(registry, compute); + Require(RegisteredProgram(*registry, 0x10000, {2}) == vertex && RegisteredProgram(*registry, 0x10000, {0}) == compute, "shared code selected another registered type"); + Reject([&] { static_cast(RegisteredProgram(*registry, 0x10000, {1})); }); + Require(RegisteredProgram(*registry, 0x10100, {2}) == nullptr, "an out-of-range entry selected registered code"); + const auto copy = makeSnapshot(0x40000, 2); + PublishRegisteredShader(registry, copy); + Require(RegisteredProgram(*registry, 0x10000, {2}) == copy, "equivalent registered headers became ambiguous"); + PublishRegisteredShader(registry, makeSnapshot(vertex->headerAddress, 2)); + Require(RegisteredProgram(*registry, 0x10000, {2}) == vertex && registry->at(0x10000).size() == 3, "identical earlier registration did not restore its prepared snapshot as current"); + const auto replacement = makeSnapshot(copy->headerAddress, 2); + Shader changed; + std::memcpy(&changed, replacement->header.data(), sizeof(Shader)); + changed.target = 1; + std::memcpy(replacement->header.data(), &changed, sizeof(Shader)); + PublishRegisteredShader(registry, replacement); + Require(RegisteredProgram(*registry, 0x10000, {2}) == replacement, "same-type registration did not keep the latest program metadata"); + const auto geometry = makeSnapshot(0x60000, 4); + PublishRegisteredShader(registry, geometry); + Reject([&] { static_cast(RegisteredProgram(*registry, 0x10000, {2, 4})); }); +} + void Registration(AgcDriver::VulkanDevice& device) { const std::array code{0xbf810000u}; RecompileRequest request{{ShaderStage::Compute, 0x10000, code, 0, {}}, {32, 0, {}, ShaderComputeStageInfo{{1, 1, 1}, 0, {false, false, false}, false, 0, {}}, {}, {}, {}}, device.ComputeTarget(32), {0, 0, 0, 128}}; @@ -130,7 +171,7 @@ void Registration(AgcDriver::VulkanDevice& device) { for (unsigned iteration = 0; iteration < 40; ++iteration) { ShaderPreparationTransaction transaction; PublishRegisteredShader(registry, makeSnapshot()); - const auto current = registry->at(0x10000); + const auto current = registry->at(0x10000).back(); Require(current == original && current->prepared->rectangleRequested, "identical registration replaced a prepared snapshot"); auto& entries = transaction.Edit(*current).entries; entries.push_back({iteration + 1, handle}); @@ -160,11 +201,11 @@ void Registration(AgcDriver::VulkanDevice& device) { transaction.Commit(); } replacementWriter.get(); - Require(registry->at(0x10000) == replacement && submission->at(0x10000) == original, "replacement modified an in-flight registry"); + Require(registry->at(0x10000).back() == replacement && submission->at(0x10000).back() == original, "replacement modified an in-flight registry"); Require(original->prepared->entries.size() == 162 && replacement->prepared->entries.empty(), "stale preparation was published into a replacement snapshot"); original.reset(); - request.shader.code = submission->at(0x10000)->code; - const auto invocation = InvocationFor(*submission->at(0x10000), 0, request); + request.shader.code = submission->at(0x10000).back()->code; + const auto invocation = InvocationFor(*submission->at(0x10000).back(), 0, request); const auto capture = invocation.Capture({}); const auto result = invocation.Materialize(*capture); Require(result->variantId == GetPreparedArtifact(*handle).variantId, "old submission lost its prepared artifact"); @@ -172,19 +213,19 @@ void Registration(AgcDriver::VulkanDevice& device) { device.WaitIdle(); Require(!old.expired(), "in-flight submission released its snapshot early"); submission.reset(); - Require(old.expired(), "completed submission retained an obsolete snapshot"); + Require(!old.expired() && registry->at(0x10000).size() == 2, "registration discarded an earlier header for shared code"); Require(invocation.Materialize(*capture)->spirv.data() == result->spirv.data(), "prepared handle did not retain its artifact"); auto changed = makeSnapshot(); changed->headerAddress = replacement->headerAddress; changed->code[0] ^= 1; PublishRegisteredShader(registry, changed); - Require(registry->at(0x10000) == changed, "changed shader code was treated as identical"); + Require(registry->at(0x10000).back() == changed && registry->at(0x10000).size() == 1 && old.expired(), "changed shader code retained obsolete headers"); auto metadata = makeSnapshot(); metadata->headerAddress = changed->headerAddress; metadata->code = changed->code; metadata->header.push_back(std::byte{1}); PublishRegisteredShader(registry, metadata); - Require(registry->at(0x10000) == metadata, "changed shader metadata was treated as identical"); + Require(registry->at(0x10000).back() == metadata, "changed shader metadata was treated as identical"); } } @@ -192,6 +233,7 @@ void Registration(AgcDriver::VulkanDevice& device) { int main() { try { Transactions(); + HeaderSelection(); auto device = OpenVulkanTestDevice(); if (!device) return VulkanTestSkipped; Registration(*device); diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp index 695192b8d..7d01e74a7 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp @@ -6,6 +6,7 @@ #include "prx/libSceAgcDriver/Execution/include/Driver.hpp" #include "prx/libSceAgcDriver/Execution/include/ShaderPreparation.hpp" #include "prx/libSceAgcDriver/Submit/include/Acb.hpp" +#include "prx/libSceAgcDriver/Submit/include/Dcb.hpp" #include #include #include @@ -334,6 +335,67 @@ void UnsupportedTypeRegistration() { AgcDriverRegisterShader_nid_postfix(&header.shader); } +void SharedCodeHeaders() { + alignas(256) static const std::array code{0x7e0002ffu, 0, 0x7e0202ffu, 0, 0x7e0402ffu, 0, 0x7e0602ffu, 0x3f800000u, 0xf80008cfu, 0x03020100u, 0xbf810000u}; + struct Header { + Shader shader{}; + std::array registers{}; + ShaderSpecialRegs specials{}; + ShaderUserData users{}; + std::array context{{{0x2d5, 0x2000}}}; + } vertex, mesh; + const auto address = reinterpret_cast(code.data()); + for (auto* header : {&vertex, &mesh}) { + header->shader.file_header = 0x34333231u; + header->shader.version = 0x18; + header->shader.header_size = sizeof(Header); + header->shader.shader_size = sizeof(code); + header->shader.code = code.data(); + header->shader.sh_registers = header->registers.data(); + header->shader.num_sh_registers = header->registers.size(); + header->shader.specials = &header->specials; + header->shader.user_data = &header->users; + header->specials.dispatch_modifier = 0x8000; + } + vertex.shader.cx_registers = vertex.context.data(); + vertex.shader.num_cx_registers = vertex.context.size(); + vertex.shader.type = 2; + vertex.registers = {{{0xc8, static_cast(address >> 8u)}, {0xc9, static_cast(address >> 40u)}, {0x8b, 0}, {0x8a, 0}, {0xca, 0}, {0xcb, 0}, {0xcc, 0}}}; + mesh.shader.type = 4; + mesh.registers = vertex.registers; + const std::array primitive{{{0x242, 4}}}; + AgcDriverRegisterShader_nid_postfix(&vertex.shader); + AgcDriverRegisterShader_nid_postfix(&mesh.shader); + AgcDriverResolveShaderAbi_nid_postfix(&vertex.shader, {}, primitive); + AgcDriverRegisterShader_nid_postfix(&mesh.shader); + Shader copy = vertex.shader; + copy.user_data = nullptr; + AgcDriverResolveShaderAbi_nid_postfix(©, {}, primitive); + copy.target ^= 1u; + ExpectFailure([&] { AgcDriverResolveShaderAbi_nid_postfix(©, {}, primitive); }, "replaced shader header"); + std::vector commands; + for (const auto reg : vertex.registers) commands.insert(commands.end(), {0xc0017600u, reg.offset, reg.value}); + for (const auto reg : vertex.context) commands.insert(commands.end(), {0xc0016900u, reg.offset, reg.value}); + commands.insert(commands.end(), {0xc0017600u, 0x8, 0, 0xc0016900u, 0xd, 0x00100010u, 0xc0016900u, 0x1c3, 4, 0xc0017900u, 0x242, 4, 0xc0012d00u, 3, 2}); + Packet packet{commands.data(), static_cast(commands.size()), 0, {}}; + for (unsigned iteration = 0; iteration < 2; ++iteration) { + Require(sceAgcDriverSubmitDcb(&packet) == 0, "shared-code draw submission failed"); + AgcDriverWaitIdle_nid_postfix(); + } + mesh.shader.type = 2; + mesh.shader.cx_registers = mesh.context.data(); + mesh.shader.num_cx_registers = mesh.context.size(); + AgcDriverRegisterShader_nid_postfix(&mesh.shader); + AgcDriverResolveShaderAbi_nid_postfix(&mesh.shader, {}, primitive); + AgcDriverResolveShaderAbi_nid_postfix(&vertex.shader, {}, primitive); + Require(sceAgcDriverSubmitDcb(&packet) == 0, "same-type registered header draw failed"); + AgcDriverWaitIdle_nid_postfix(); + AgcDriverRegisterShader_nid_postfix(&vertex.shader); + AgcDriverResolveShaderAbi_nid_postfix(&vertex.shader, {}, primitive); + Require(sceAgcDriverSubmitDcb(&packet) == 0, "re-registered earlier header draw failed"); + AgcDriverWaitIdle_nid_postfix(); +} + void RegistrationWithoutSpecials() { alignas(256) std::array code{0xbf810000u}; struct Header { @@ -521,6 +583,7 @@ int main(int argc, char** argv) { Require(argc != 2 || std::string_view(argv[1]) != "--fail-before-registration", "injected failure before registration"); PrepareMultisampledStorage(*device); device.reset(); + SharedCodeHeaders(); RegistrationWithoutSpecials(); Registration(argc == 2); std::cout << "prepared shader and transactional registration tests passed\n"; From 946158a503dcdda2815aed51f264c2b47399408f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 01:04:50 +0200 Subject: [PATCH 234/460] docs(architecture): describe the per-code header registry Co-Authored-By: Claude --- docs/dev/ARCHITECTURE.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/ARCHITECTURE.md b/docs/dev/ARCHITECTURE.md index 42746ab07..51bde1220 100644 --- a/docs/dev/ARCHITECTURE.md +++ b/docs/dev/ARCHITECTURE.md @@ -81,6 +81,7 @@ flowchart LR - `ShaderRecompiler::Recompile` keeps compiled variants in memory, and `ShaderDiskCache` stores them on disk so later runs reuse them. - A compute shader whose LDS plus its lock dword exceeds the host's `maxComputeSharedMemorySize` is not given a `Workgroup` array. `Recompile` sets `ShaderInfo::sharedMemoryBytes` for it and the SPIR-V backend addresses the LDS in a coherent storage buffer at descriptor set 1 (`WorkgroupMemoryDescriptorSet`), with a slice of `RecompileResult::workgroupMemoryDwords` per workgroup, indexed by `WorkgroupID` and `NumWorkGroups`. `VulkanDevice` keeps one such buffer, sized from the dispatch dimensions, and binds it after the resource set; programs that fit keep using shared memory. - `Driver::RegisterShader` splits shader preparation in two. `PlanRegistered` runs on the registering guest thread: it validates the registers and entry point, decodes the code and builds the recompile request, so invalid registrations still fail there. The RDNA to IR translation and resource plan (`PrepareShader`) run on a background pool of `hardware_concurrency() / 2` detached threads (at least 2, 64 MiB stacks). +- The shader registry maps a code address to every header registered for that code, in registration order. Static, graphics and rectangle ABI resolution pick the header at the bound address (or one equal to it, `SameHeader`); draws and dispatches pick the program by the stage's binary type (`RegisteredProgram`), and a front entry matching both type 2 and type 4 throws. Within a type the last registration wins; re-registering an identical earlier header makes its snapshot current again without preparing it again, and changed code drops the older headers. - The snapshot is published as pending. `SourceHandleFor`, `InvocationFor` and `ShaderPreparationTransaction::Edit` / `Read` wait for the pending job before reading `PreparedShaders`, and rethrow its failure. `APS5_SYNC_SHADER_PREPARE=1` prepares on the registering thread instead. - A draw's `Graphics::Pipeline` comes from `CachedPipeline`. When the device has `VK_EXT_graphics_pipeline_library` with fast linking, `VK_EXT_extended_dynamic_state` and `VK_KHR_dynamic_rendering`, draws render with `vkCmdBeginRenderingKHR` instead of render passes and the pipeline is linked from four libraries (vertex input, pre-rasterization shaders, fragment shader, fragment output) kept per device in [`PipelineLibrary.cpp`](../../core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp). A library built for one pipeline is reused by the next: another blend, vertex layout or target format builds only that part (the shader libraries name no attachment format), and cull mode, front face, depth and stencil state are dynamic, so changing them only links. The fast link is what the draw waits for; a background thread then links the same libraries with link-time optimization and the pipeline binds that one from the next draw on. Push constants then use a 256-byte block with a 128-byte slot per side: the pre-rasterization stages stack from byte 0, the pixel shader always starts at byte 128 (`StagePushOffset`), so its library does not depend on how much push data its vertex shader uses. Pipelines with per-face depth bias, rect lists, geometry shaders or a stage without a variant id are created whole, still with dynamic rendering; devices without the extensions (or with `APS5_NO_GPL=1`) keep render passes and whole pipelines. From e1f168a24d15ae194ec8f0149bba405b70afdae1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jan=20=C5=A0l=C3=A9gl?= Date: Thu, 8 Oct 2026 18:30:23 +0200 Subject: [PATCH 235/460] feat(libc): implement realpath Resolves a guest path the way FreeBSD 11's realpath does: a relative path starts from the guest working directory, "." and empty components are skipped, ".." drops the last component and never climbs above "/", every other component must exist, and one that is not a directory with anything after it is ENOTDIR. Backslashes separate components as in the rest of the path layer. The result is the guest path; the host is consulted only for existence and type, so host symlinks are followed but not expanded, and ".." is lexical, as in the rest of the path layer. A null buffer is PATH_MAX bytes from the application heap, a null path is EINVAL, an empty one ENOENT, and 1024 bytes or more in the input or the result is ENAMETOOLONG. On failure the caller's buffer holds the path that failed. Co-Authored-By: Claude Fable 5.1 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Fnrk5Dazjb2tLooVSpdRQq --- core/libs/prx/libc/CMakeLists.txt | 11 +++ core/libs/prx/libc/src/RealPath.cpp | 145 ++++++++++++++++++++++++++++ core/libs/tests/GuestRealpath.cpp | 106 ++++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 263 insertions(+) create mode 100644 core/libs/prx/libc/src/RealPath.cpp create mode 100644 core/libs/tests/GuestRealpath.cpp diff --git a/core/libs/prx/libc/CMakeLists.txt b/core/libs/prx/libc/CMakeLists.txt index 7e732496a..9538fce00 100644 --- a/core/libs/prx/libc/CMakeLists.txt +++ b/core/libs/prx/libc/CMakeLists.txt @@ -39,6 +39,7 @@ target_sources(libc PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src/Rename.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/Errors.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/Syslog.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/src/RealPath.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/CxxAbiSupport.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/Threads.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/exception/Exports.cpp @@ -196,6 +197,16 @@ if(BUILD_TESTING) set_tests_properties(guest_working_directory PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_realpath_tests ${LIBS_INCLUDE_DIR}/tests/GuestRealpath.cpp) + target_include_directories(guest_realpath_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_realpath_tests PRIVATE libc) + configure_windows_unwind(guest_realpath_tests) + add_test(NAME guest_realpath COMMAND guest_realpath_tests) + set_tests_properties(guest_realpath PROPERTIES TIMEOUT 20) + if(WIN32) + set_tests_properties(guest_realpath PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_path_case_tests ${LIBS_INCLUDE_DIR}/tests/GuestPathCase.cpp) target_include_directories(guest_path_case_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_path_case_tests PRIVATE libc libkernel) diff --git a/core/libs/prx/libc/src/RealPath.cpp b/core/libs/prx/libc/src/RealPath.cpp new file mode 100644 index 000000000..10a0193b9 --- /dev/null +++ b/core/libs/prx/libc/src/RealPath.cpp @@ -0,0 +1,145 @@ +#include "prx/libc/include/ApplicationHeap.hpp" +#include "prx/libc/include/General.hpp" +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include +#include +#include + +extern "C" char* APS5_VABI getcwd_nid_postfix(char* buffer, std::size_t size); + +namespace { +constexpr std::size_t PathMax = 1024; +constexpr std::string_view Separators = "/\\"; +constexpr int NoEntry = 2; +constexpr int InputOutput = 5; +constexpr int NoMemory = 12; +constexpr int PermissionDenied = 13; +constexpr int NotDirectory = 20; +constexpr int InvalidArgument = 22; +constexpr int TooManyLinks = 62; +constexpr int NameTooLong = 63; + +int StatusError(const std::error_code& error) { + if (error == std::errc::no_such_file_or_directory) return NoEntry; + if (error == std::errc::not_a_directory) return NotDirectory; + if (error == std::errc::permission_denied) return PermissionDenied; + if (error == std::errc::filename_too_long) return NameTooLong; + if (error == std::errc::too_many_symbolic_link_levels) return TooManyLinks; + if (error == std::errc::not_enough_memory) return NoMemory; + return InputOutput; +} + +void LeaveTrouble(char* resolved, const std::string& trouble) { + if (resolved == nullptr) return; + const auto length = trouble.size() < PathMax ? trouble.size() : PathMax - 1; + std::memcpy(resolved, trouble.data(), length); + resolved[length] = '\0'; +} + +void DropLastComponent(std::string& result) { + if (result.size() <= 1) return; + const auto slash = result.rfind('/'); + result.erase(slash == 0 ? 1 : slash); +} + +bool IsSeparator(char character) { return Separators.find(character) != std::string_view::npos; } + +bool ResolveGuest(const char* path, std::string& result, int& error) { + std::string_view left(path); + if (IsSeparator(left.front())) { + result = "/"; + left.remove_prefix(1); + } else { + char current[PathMax]; + if (getcwd_nid_postfix(current, sizeof(current)) == nullptr) { + result = "."; + error = errno; + return false; + } + result = current; + } + if (left.size() >= PathMax || result.size() >= PathMax) { + error = NameTooLong; + return false; + } + while (!left.empty()) { + const auto separator = left.find_first_of(Separators); + const auto token = left.substr(0, separator); + const bool moreFollows = separator != std::string_view::npos; + left = moreFollows ? left.substr(separator + 1) : std::string_view(); + if (result.size() > 1 && result.size() + 1 >= PathMax) { + error = NameTooLong; + return false; + } + if (token.empty() || token == ".") continue; + if (token == "..") { + DropLastComponent(result); + continue; + } + std::string candidate = result; + if (candidate.size() > 1) candidate += '/'; + candidate += token; + result = candidate; + if (candidate.size() >= PathMax) { + error = NameTooLong; + return false; + } + std::error_code code; + const auto status = std::filesystem::status(ResolvePath_nid_no_patch(candidate.c_str()), code); + if (code && code != std::errc::no_such_file_or_directory && code != std::errc::not_a_directory) { + error = StatusError(code); + return false; + } + if (!std::filesystem::exists(status)) { + error = NoEntry; + return false; + } + if (moreFollows && !std::filesystem::is_directory(status)) { + error = NotDirectory; + return false; + } + } + return true; +} +} + +extern "C" char* APS5_VABI realpath_nid_postfix(const char* path, char* resolved) { + if (path == nullptr) { + errno = InvalidArgument; + return nullptr; + } + if (*path == '\0') { + errno = NoEntry; + return nullptr; + } + std::string result; + int error = 0; + try { + if (!ResolveGuest(path, result, error)) { + LeaveTrouble(resolved, result); + errno = error; + return nullptr; + } + if (resolved == nullptr) { + resolved = static_cast(ApplicationHeapAllocate_nid_no_patch(PathMax)); + if (resolved == nullptr) { + errno = NoMemory; + return nullptr; + } + } + std::memcpy(resolved, result.c_str(), result.size() + 1); + return resolved; + } catch (const std::bad_alloc&) { + errno = NoMemory; + return nullptr; + } catch (const std::filesystem::filesystem_error& failure) { + LeaveTrouble(resolved, result); + errno = StatusError(failure.code()); + return nullptr; + } +} diff --git a/core/libs/tests/GuestRealpath.cpp b/core/libs/tests/GuestRealpath.cpp new file mode 100644 index 000000000..faae73050 --- /dev/null +++ b/core/libs/tests/GuestRealpath.cpp @@ -0,0 +1,106 @@ +#include "prx/libc/include/ApplicationHeap.hpp" +#include "prx/libc/include/General.hpp" +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include + +extern "C" { +char* APS5_VABI realpath_nid_postfix(const char*, char*); +int APS5_VABI chdir_nid_postfix(const char*); +void APS5_VABI free_nid_postfix(void*); +int* APS5_VABI __error_nid_postfix(); +} + +static void Require(bool value) { if (!value) std::abort(); } + +static bool Resolves(const std::string& path, const std::string& expected) { + char out[1024]; + std::memset(out, 'x', sizeof(out)); + return realpath_nid_postfix(path.c_str(), out) == out && expected == out; +} + +static bool Fails(const char* path, int error, const std::string& trouble = {}) { + char out[1024]; + std::memset(out, 'x', sizeof(out)); + *__error_nid_postfix() = 0; + if (realpath_nid_postfix(path, out) != nullptr || *__error_nid_postfix() != error) return false; + return trouble.empty() || trouble == out; +} + +int main() { + std::array api{}; + ApplicationHeapRegister_nid_no_patch(api.data()); + const auto host = std::filesystem::canonical(std::filesystem::current_path()); + const auto name = "anyps5-realpath-test-" + std::to_string(std::chrono::steady_clock::now().time_since_epoch().count()); + const auto directory = host / name; + Require(std::filesystem::create_directories(directory / "sub")); + { std::ofstream file(directory / "file.txt"); file << "file"; } + { std::ofstream file(directory / "sub" / "inner.txt"); file << "inner"; } + const std::string guest = "/" + name; + + Require(Resolves("/", "/")); + Require(Resolves(guest, guest)); + Require(Resolves(guest + "/", guest)); + Require(Resolves(guest + "/sub/../file.txt", guest + "/file.txt")); + Require(Resolves(guest + "/./sub//inner.txt", guest + "/sub/inner.txt")); + Require(Resolves("/../.." + guest + "/sub/", guest + "/sub")); + Require(Resolves(guest + "/sub/..", guest)); + Require(Resolves("/..", "/")); + Require(Resolves(guest + "\\sub\\..\\file.txt", guest + "/file.txt")); + Require(Resolves("\\" + name + "\\sub\\", guest + "/sub")); + + std::string upper = guest; + for (auto& character : upper) character = static_cast(std::toupper(static_cast(character))); + Require(Resolves(upper + "/file.txt", upper + "/file.txt")); + + Require(chdir_nid_postfix(guest.c_str()) == 0); + Require(Resolves("sub/inner.txt", guest + "/sub/inner.txt")); + Require(Resolves("../" + name + "/file.txt", guest + "/file.txt")); + Require(Resolves(".", guest)); + Require(Resolves("sub/./../sub", guest + "/sub")); + Require(Resolves("..", "/")); + Require(chdir_nid_postfix("/") == 0); + + AddPathAlias_nid_no_patch("realpath-alias", directory.string().c_str()); + Require(Resolves("/realpath-alias/sub/../file.txt", "/realpath-alias/file.txt")); + Require(Fails("/realpath-alias/missing", 2, "/realpath-alias/missing")); + RemovePathAlias_nid_no_patch("realpath-alias"); + BlockPathAlias_nid_no_patch("realpath-blocked"); + Require(Fails("/realpath-blocked/x", 2, "/realpath-blocked")); + + char* allocated = realpath_nid_postfix((guest + "/file.txt").c_str(), nullptr); + Require(allocated != nullptr && guest + "/file.txt" == allocated); + free_nid_postfix(allocated); + + Require(Fails(nullptr, 22)); + Require(Fails("", 2)); + Require(Fails((guest + "/missing").c_str(), 2, guest + "/missing")); + Require(Fails((guest + "/missing/../file.txt").c_str(), 2, guest + "/missing")); + Require(Fails((guest + "/file.txt/").c_str(), 20, guest + "/file.txt")); + Require(Fails((guest + "/file.txt/x").c_str(), 20, guest + "/file.txt")); + Require(Fails((guest + "/file.txt/.").c_str(), 20)); + Require(Fails((guest + "/file.txt/..").c_str(), 20)); + + const std::string overflowing = "/" + std::string(1023, 'a'); + Require(Fails(overflowing.c_str(), 63, overflowing.substr(0, 1023))); + const std::string longInput = guest + "/sub/" + std::string(1024, '.') + "/inner.txt"; + Require(Fails(longInput.c_str(), 63)); + std::string boundary = guest; + while (boundary.size() + 8 <= 1023) boundary += "/missing"; + boundary += "/" + std::string(1023 - boundary.size() - 1, 'm'); + Require(boundary.size() == 1023); + Require(Fails(boundary.c_str(), 2, guest + "/missing")); + + *__error_nid_postfix() = 0; + Require(realpath_nid_postfix((guest + "/missing").c_str(), nullptr) == nullptr && *__error_nid_postfix() == 2); + + std::filesystem::remove_all(directory); + return 0; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 73ac7832e..c8bfcb48c 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -317,6 +317,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceKernelWriteThrottlingStatus](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) reports no write throttling: the host SSD has no write budget, so the first qword of the 32-byte status is set to `UINT32_MAX` and the rest is zeroed. PPSA17221 reads only that qword: it multiplies it by 64 KiB and compares the product with byte thresholds up to 0x4AF00000 to pick one of five tiers. When the call fails the title assumes 0x4B000001 bytes, which selects the same top tier as `UINT32_MAX`. What the value measures on hardware (a remaining budget or a rate) and the meaning of the other three qwords are unknown - [/dev/random and /dev/urandom](../../core/libs/prx/libkernel/File/src/RandomDevice.cpp) (libkernel) opened for reading return a descriptor backed by the host null device whose reads are filled from `std::random_device`; both paths behave the same, never block, and positioned reads ignore the offset (FreeBSD `random(4)`). Only the exact paths are recognised, a `dup`ed descriptor reads as an empty file, `fstat`, `lseek`, `mmap` and `select` see the null device, and opening them for writing throws - [Wide-character conversions](../../core/libs/prx/libc/src/Multibyte.cpp) (libc) - The wide `wcsto*` functions parse the ASCII prefix of the guest string, as the C locale accepts only ASCII white space and digits. [`wcsrtombs`](../../core/libs/prx/libc/src/Multibyte.cpp) uses the guest's single-byte C locale through `wcrtomb`; other encodings and shift states are not supported +- [realpath](../../core/libs/prx/libc/src/RealPath.cpp) (libc) follows FreeBSD 11's [`realpath.c`](https://github.com/freebsd/freebsd-src/blob/stable/11/lib/libc/stdlib/realpath.c) except for symlinks. The console's guest filesystem has none, so a host symlink under the guest root is followed for existence and type but not expanded, and `..` drops the previous component lexically, as `Resolve` does for every guest path. Through a host link `linkdir -> sub/deep`, `linkdir/../x` gives `x` beside the link, not beside its target. - [SPIRV-Tools optimization](../../core/shader/recompiler/SpirvBackend/src/SpirvOptimizer.cpp) skips CCP and Simplification for modules with storage-only 8- or 16-bit types because the pinned folder produces illegal narrow constants. This also disables unrelated constant folding; the restricted full pipeline mirrors the pinned performance recipe. - [SRT signed comparison](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtEvaluator.cpp) evaluates `SLessThan32` as signed 32-bit less-than, matching the existing SPIR-V backend and [SPIR-V `OpSLessThan`](https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html#OpSLessThan). Signed boundary descriptor expressions are tested on the host; native RDNA2/PS5 execution and title behavior remain unverified. - On macOS the [write watch](../../core/libs/prx/libc/src/GuestWriteWatch.cpp) write-protects watched pages and records the first write to each from a SIGBUS/SIGSEGV handler, because macOS has no counterpart to `userfaultfd` write-protect or `MEM_WRITE_WATCH`. The protection is real, so: From cf637814ebaffb7d31529c0a6157b9e0dc6b3009 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 13:06:26 -0400 Subject: [PATCH 236/460] fix(shader): flush v_div_scale_f32 input denormals Enable the existing input-only f32 flush policy for division scaling. RDNA2 measurements pin all three sources and signed zeros; keep the scaling rules unchanged and retain their separate technical debt. Co-Authored-By: Codex gpt-6.1-sol --- .../tests/execution/Division.cpp | 82 +++++++++++++++++++ .../Translation/src/VectorInstructions.cpp | 1 + docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 84 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/Division.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/Division.cpp index ae9a497a7..9cbfb1997 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/Division.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/Division.cpp @@ -9,6 +9,7 @@ #include #include #include +#include #include #include @@ -41,6 +42,51 @@ alignas(256) constexpr std::array FmaCode{ 0xd54b000a, 0x041a0b04, 0xe0702000, 0x80010a01, 0xbf810000, }; +alignas(256) constexpr std::array ScaleCode{ + 0x34020082u, 0xe0302000u, 0x80000401u, 0xe0302004u, 0x80000501u, 0xe0302008u, 0x80000601u, 0xe030200cu, + 0x80000701u, 0xbf8c3f70u, 0xd56d140au, 0x041a0b04u, 0xd501000bu, 0x00510280u, 0xe0702000u, 0x80010a01u, + 0xe0702004u, 0x80010b01u, 0xbf810000u, +}; + +struct ScaleVector { + std::uint32_t s0, s1, s2, flushed, kept, vcc; +}; + +constexpr ScaleVector ScaleVectors[] = { + {0x00000001u, 0x3f800000u, 0x3f800000u, 0x00000000u, 0x00000001u, 0x00000000u}, + {0x3f800000u, 0x00000001u, 0x3f800000u, 0xffc00000u, 0x3f800000u, 0x00000001u}, + {0x3f800000u, 0x3f800000u, 0x00000001u, 0xffc00000u, 0x3f800000u, 0x00000001u}, + {0x80000001u, 0x3f800000u, 0x3f800000u, 0x80000000u, 0x80000001u, 0x00000000u}, + {0x3f800000u, 0x80000001u, 0x3f800000u, 0xffc00000u, 0x3f800000u, 0x00000001u}, + {0x3f800000u, 0x3f800000u, 0x80000001u, 0xffc00000u, 0x3f800000u, 0x00000001u}, + {0x007fffffu, 0x3f800000u, 0x3f800000u, 0x00000000u, 0x007fffffu, 0x00000000u}, + {0x3f800000u, 0x007fffffu, 0x3f800000u, 0xffc00000u, 0x3f800000u, 0x00000001u}, + {0x3f800000u, 0x3f800000u, 0x007fffffu, 0xffc00000u, 0x3f800000u, 0x00000001u}, + {0x807fffffu, 0x3f800000u, 0x3f800000u, 0x80000000u, 0x807fffffu, 0x00000000u}, + {0x3f800000u, 0x807fffffu, 0x3f800000u, 0xffc00000u, 0x3f800000u, 0x00000001u}, + {0x3f800000u, 0x3f800000u, 0x807fffffu, 0xffc00000u, 0x3f800000u, 0x00000001u}, + {0x3f800000u, 0x00000001u, 0x00000001u, 0xffc00000u, 0x5f800000u, 0x00000000u}, + {0xbf800000u, 0x80000001u, 0x80000001u, 0xffc00000u, 0xdf800000u, 0x00000000u}, + {0x3f800000u, 0x00000001u, 0x80000001u, 0xffc00000u, 0x5f800000u, 0x00000000u}, + {0x00000000u, 0x3f800000u, 0x3f800000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x80000000u, 0x3f800000u, 0x3f800000u, 0x80000000u, 0x80000000u, 0x00000000u}, + {0x3f800000u, 0x3f800000u, 0x3f800000u, 0x3f800000u, 0x3f800000u, 0x00000000u}, + {0xbfc00000u, 0x3f800000u, 0x3f800000u, 0xbfc00000u, 0xbfc00000u, 0x00000000u}, + {0x00800000u, 0x3f800000u, 0x3f800000u, 0x00800000u, 0x00800000u, 0x00000000u}, + {0x80800000u, 0x3f800000u, 0x3f800000u, 0x80800000u, 0x80800000u, 0x00000000u}, + {0x7f800000u, 0x3f800000u, 0x3f800000u, 0x7f800000u, 0x7f800000u, 0x00000000u}, + {0xff800000u, 0x3f800000u, 0x3f800000u, 0xff800000u, 0xff800000u, 0x00000000u}, + {0x7fc12345u, 0x3f800000u, 0x3f800000u, 0x7fc12345u, 0x7fc12345u, 0x00000000u}, + {0xff812345u, 0x3f800000u, 0x3f800000u, 0xff812345u, 0xff812345u, 0x00000000u}, + {0x3f800000u, 0x00000000u, 0x3f800000u, 0xffc00000u, 0xffc00000u, 0x00000001u}, + {0x3f800000u, 0x3f800000u, 0x00000000u, 0xffc00000u, 0xffc00000u, 0x00000001u}, + {0x3f800000u, 0x80000000u, 0x3f800000u, 0xffc00000u, 0xffc00000u, 0x00000001u}, + {0x3f800000u, 0x3f800000u, 0x80000000u, 0xffc00000u, 0xffc00000u, 0x00000001u}, + {0x3f800000u, 0x00800000u, 0x3f800000u, 0x3f800000u, 0x3f800000u, 0x00000001u}, + {0x3f800000u, 0x3f800000u, 0x00800000u, 0x3f800000u, 0x3f800000u, 0x00000001u}, + {0x00800000u, 0x00800000u, 0x3f800000u, 0x20800000u, 0x20800000u, 0x00000001u}, +}; + struct HelperVector { std::uint32_t s0, s1, s2, vcc; std::uint32_t scale, scaleVcc, fmas, fixup; @@ -653,6 +699,41 @@ bool HostFusesFma(AgcDriver::VulkanDevice& device) { return Output[0] == 0x2f8e8ae0u; } +void CheckScaleDenormals(AgcDriver::VulkanDevice& device) { + const std::array, 5> modes{{ + std::nullopt, + ShaderRecompiler::ShaderFloatMode{0xc0u, true, false, false}, + ShaderRecompiler::ShaderFloatMode{0xc0u, true, true, false}, + ShaderRecompiler::ShaderFloatMode{0xf0u, true, false, false}, + ShaderRecompiler::ShaderFloatMode{0xf0u, true, true, false}, + }}; + for (const auto& mode : modes) { + Shader shader(device, ScaleCode, mode); + Input.fill(0u); + for (std::uint32_t lane = 0; lane < std::size(ScaleVectors); ++lane) { + const auto& vector = ScaleVectors[lane]; + std::copy_n(std::array{vector.s0, vector.s1, vector.s2, 0u}.begin(), 4, Input.begin() + lane * Stride); + } + shader.Run(); + for (std::uint32_t lane = 0; lane < std::size(ScaleVectors); ++lane) { + const auto& vector = ScaleVectors[lane]; + const auto* out = &Output[lane * Stride]; + const auto expected = mode.has_value() && ((mode->floatMode >> 4u) & 3u) == 3u ? vector.kept : vector.flushed; + const std::string where = "division scale denormals: lane " + std::to_string(lane) + " mode " + (mode.has_value() ? Hex(mode->floatMode) : "unknown"); + Require(out[0] == expected && out[1] == vector.vcc, where + ": v_div_scale_f32 is " + Hex(out[0]) + " vcc " + std::to_string(out[1]) + ", expected " + Hex(expected) + " vcc " + std::to_string(vector.vcc)); + } + } + for (const std::uint32_t mode : {0xd0u, 0xe0u}) { + bool refused = false; + try { + Shader shader(device, ScaleCode, ShaderRecompiler::ShaderFloatMode{mode, true, false, false}); + } catch (const std::runtime_error& error) { + refused = std::string(error.what()).find("f32 denormal mode") != std::string::npos; + } + Require(refused, "division scale denormals: FLOAT_MODE " + Hex(mode) + " was not refused"); + } +} + void CheckHelpers(AgcDriver::VulkanDevice& device, bool fused) { Shader shader(device, HelperCode); constexpr std::uint32_t count = sizeof(HelperVectors) / sizeof(HelperVectors[0]); @@ -718,6 +799,7 @@ int main() { try { const auto device = OpenVulkanTestDevice(); if (!device) return VulkanTestSkipped; + CheckScaleDenormals(*device); const bool fused = HostFusesFma(*device); CheckHelpers(*device, fused); CheckFmasNan(*device, std::nullopt); diff --git a/core/shader/recompiler/Translation/src/VectorInstructions.cpp b/core/shader/recompiler/Translation/src/VectorInstructions.cpp index 56c9a588d..645b98534 100644 --- a/core/shader/recompiler/Translation/src/VectorInstructions.cpp +++ b/core/shader/recompiler/Translation/src/VectorInstructions.cpp @@ -86,6 +86,7 @@ std::uint32_t TranslationContext::f32DenormalFlushFor(const RdnaInstruction& ins case RdnaOpcode::VCvtPkU8F32: case RdnaOpcode::VCvtPkrtzF16F32: case RdnaOpcode::VCvtF16F32: + case RdnaOpcode::VDivScaleF32: case RdnaOpcode::VDivFixupF32: case RdnaOpcode::VMadMixloF16: case RdnaOpcode::VMadMixhiF16: diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 8d1af8630..554521df7 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -402,7 +402,7 @@ Throughout the project, every function at every stage either **does exactly what - [v_fma_mixlo_f16 and v_fma_mixhi_f16](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) round the exact product-sum once to f16 with the f64 arithmetic of v_interp_p2_f16, so they need the Float64 capability. On RDNA2 (gfx1036) they follow the `FLOAT_MODE` f16 round and denormal fields, not the f32 round field: -1.5 * 2^-24 + 2^-126 from f32 sources (0xbfc00000, 0x33800000, 0x00800000) gives 0x8001 with f16 denormals kept and round to nearest even, 0x8000 with f16 denormals flushed and 0x8002 with f16 round toward -infinity. Only f16 denormals kept with round to nearest even is translated; another f16 mode throws, as for packed f16. `FP16_OVFL` set is not measured. - [Signaling NaN inputs](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) of the f32 and f16 unary instructions (`v_sqrt/rcp/rsq/log/exp/sin/cos/fract/floor/ceil/trunc/rndne/rcp_iflag/frexp_mant`), `v_ldexp_f32/f16`, `v_frexp_mant_f16`, `v_div_fixup_f32/f16` and `v_mullit_f32` are returned quieted only with IEEE=1 and unchanged with IEEE=0, measured on RDNA2 (gfx1035) in both modes, and so are those of the f64 forms (`v_sqrt/rcp/rsq/fract/floor/ceil/trunc/rndne/frexp_mant/ldexp/div_fixup_f64`) and `v_cvt_f64_f32`, measured on RDNA2 (gfx1036) in both modes; shaders without a known float mode follow IEEE=0. `v_pack_b32_f16` quiets each signaling NaN half by the same rule, measured on RDNA2 (gfx1036) in both modes. A signaling NaN result of `v_cubesc/cubetc/cubema_f32` is quieted by the same rule and keeps the sign the instruction gives it, measured on RDNA2 (gfx1036) in both modes. `v_add/sub/subrev/mul/fma/fmac/fmamk/fmaak/mul_legacy_f32` and `v_fma_mix_f32` with f32 sources choose their NaN result on the bits, measured over 1000 operand triples in both modes: the first NaN source in operand order after its modifiers (the subtrahend negated, as the hardware adds its negation), quieted only with IEEE=1, or the default NaN 0xffc00000 for an invalid operation; in the fma forms an invalid product (inf * 0) gives the default NaN even when src2 is a NaN, and in the legacy forms a zero factor makes the product 0 even when the other factor is a NaN (a NaN src2 still wins). `v_mad_legacy_f32` and `v_mac_legacy_f32` follow the same rule with their separately rounded product, as measured on gfx1010 (#1281, #1290): a NaN source quieted only with IEEE=1, inf - inf giving 0xffc00000, the zero rule hiding a NaN factor and a NaN src2 winning. The f16 and f64 arithmetic, `v_fma_mix_f32` with f16 sources and the separately rounded mad/mac forms (not measurable on RDNA2) follow the same rule on hardware but their NaN bits are left to the driver. `v_cvt_f16_f32`, `v_cvt_f32_f64`, `v_dot2_f32_f16`, `v_fma_mixlo_f16` and `v_fma_mixhi_f16` always quiet and `v_div_scale_f32` never does: the f32 signaling NaN 0x7f812345 as src0 or src2 gives 0x7e09 from mixlo and mixhi at IEEE=0 and IEEE=1 on RDNA2 (gfx1036), where `v_fma_mix_f32` gives 0x7f812345 at IEEE=0 and 0x7fc12345 at IEEE=1. The payload of an f16 NaN source of mixlo and mixhi is left to the driver's f16 to f32 conversion, which NVIDIA replaces. - [Packed f16 NaN results](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) (`v_pk_add/mul/fma/min/max_f16`) follow the hardware measured on RDNA2 (gfx1035) at IEEE=0 and IEEE=1 with f16 round-to-nearest-even and f16 denormals kept: with IEEE=1 a signaling NaN input is quieted and min/max return the other operand only for a quiet NaN; with IEEE=0 NaN inputs pass through unquieted and min/max ignore signaling. In `v_pk_fma_f16` an invalid product (inf * 0) gives the default NaN even when src2 is a NaN. A shader without a known float mode is translated as IEEE=0, the mode titles use; a mode with another f16 rounding or that flushes f16 denormals throws. -- [f32 denormals](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follow the shader's `FLOAT_MODE` f32 denormal field (bits 5:4) when it flushes them (0, and shaders without a known float mode, as in the 0xc0 most titles use), measured on RDNA2 (gfx1035) over 1444 rows per instruction at IEEE=0 and IEEE=1. `v_add/sub/subrev/mul/mul_legacy/mullit/fma/fmac/fmamk/fmaak/min/max/min3/max3/med3/fract/floor/ceil/frexp_mant/sin/cos_f32` flush their f32 inputs and their result to a zero of the same sign (`v_mullit_f32` measured on RDNA2 (gfx1036) over 6907 rows per mode at IEEE=0 and IEEE=1, so a positive denormal src2 counts as zero); the f32 compares, `v_cvt_i32/u32/rpi_i32/flr_i32_f32`, `v_frexp_exp_i32_f32`, `v_cvt_f64_f32`, `v_cvt_f16_f32`, `v_cvt_pkrtz_f16_f32`, `v_cvt_pknorm_i16/u16_f32` and `v_cvt_pk_u8_f32` flush their f32 inputs. `v_div_fixup_f32` flushes its src1 and src2 and keeps a denormal src0 and result, measured on RDNA2 (gfx1036) over 6859 rows per mode at IEEE=0 and IEEE=1. Tininess is detected after rounding, measured on RDNA2 (gfx1036) for `v_mul_f32` and `v_mullit_f32` at IEEE=0 and IEEE=1: a product is flushed when its value rounded to 24 bits with an unbounded exponent is below 2^-126, so `0x3f7ffffe * 0x00800001` (2^-126 * (1 - 2^-46)) gives `0x00800000` and `0x3f7fffff * 0x00800000` (2^-126 * (1 - 2^-24)) gives zero. The check rounds in the f32 round mode of `FLOAT_MODE` (bits 1:0), measured in all four modes, while the products themselves are still rounded to nearest even. This is computed exactly from the significands for `v_mul_f32`, `v_mul_legacy_f32`, `v_mullit_f32`, and for `v_fma_f32`, `v_fmac_f32`, `v_fmamk_f32`, `v_fmaak_f32` and `v_fma_mix_f32` when the addend is zero; with a nonzero addend the fused result is kept, so an exact result just below 2^-126 that rounds to it is not flushed. `v_fma_mix_f32` follows this rule with mixed f16/f32 sources too, measured on RDNA2 (gfx1036) with f16 in either multiplicand slot, low and high half selection, and an f32 or f16 zero addend: f32 `0x00801002` times f16 `0x3bff` gives `0x00800000` to nearest at IEEE=0 and IEEE=1, while the neighbours `0x00801001` and `0x00801003` give zero and `0x00800001`. The tiny-product boundary was also measured for both signs in all four f32 round modes at IEEE=0; f16 inputs retain their previous conversion and denormal handling. `v_fma_mixlo_f16` and `v_fma_mixhi_f16` flush their f32 sources too and keep a product below 2^-126, measured on RDNA2 (gfx1036). `v_mad_legacy_f32` and `v_mac_legacy_f32` flush their inputs, their product and their result in every mode, see its entry; their product is still checked for tininess on its exact value, so one below 2^-126 gives zero even when it rounds to 2^-126, as the console's separately rounded encoding is not measured at this boundary. Denormal modes 1 and 2 throw for the other instructions. Not covered, and not measurable on RDNA2 (no such opcodes) or not measured: `v_mac_f32`, `v_mad_f32`, `v_madak_f32` and `v_madmk_f32` (rounded separately), `v_dot2_f32_f16`, `v_div_scale_f32` and `v_div_fmas_f32` (on hardware `v_div_fmas_f32` flushes only its result), f32 interpolation and `ds_add_f32` keep the previous lowering; `ds_min/max/cmpst_f32` flush both operands by the field, see the LDS float atomics entry. With f32 denormals kept (3) the results depend on the driver's default denormal behaviour, as no `DenormPreserve` execution mode is requested, and `v_ldexp_f32` flushes in every mode. +- [f32 denormals](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follow the shader's `FLOAT_MODE` f32 denormal field (bits 5:4) when it flushes them (0, and shaders without a known float mode, as in the 0xc0 most titles use), measured on RDNA2 (gfx1035) over 1444 rows per instruction at IEEE=0 and IEEE=1. `v_add/sub/subrev/mul/mul_legacy/mullit/fma/fmac/fmamk/fmaak/min/max/min3/max3/med3/fract/floor/ceil/frexp_mant/sin/cos_f32` flush their f32 inputs and their result to a zero of the same sign (`v_mullit_f32` measured on RDNA2 (gfx1036) over 6907 rows per mode at IEEE=0 and IEEE=1, so a positive denormal src2 counts as zero); the f32 compares, `v_cvt_i32/u32/rpi_i32/flr_i32_f32`, `v_frexp_exp_i32_f32`, `v_cvt_f64_f32`, `v_cvt_f16_f32`, `v_cvt_pkrtz_f16_f32`, `v_cvt_pknorm_i16/u16_f32` and `v_cvt_pk_u8_f32` flush their f32 inputs. `v_div_scale_f32` flushes all three inputs to signed zeros, measured on RDNA2 (gfx1036) over 2744 operand triples per mode at IEEE=0 and IEEE=1 with f32 denormals flushed and kept; its scale-down behaviour and host denormal preservation remain incomplete. `v_div_fixup_f32` flushes its src1 and src2 and keeps a denormal src0 and result, measured on RDNA2 (gfx1036) over 6859 rows per mode at IEEE=0 and IEEE=1. Tininess is detected after rounding, measured on RDNA2 (gfx1036) for `v_mul_f32` and `v_mullit_f32` at IEEE=0 and IEEE=1: a product is flushed when its value rounded to 24 bits with an unbounded exponent is below 2^-126, so `0x3f7ffffe * 0x00800001` (2^-126 * (1 - 2^-46)) gives `0x00800000` and `0x3f7fffff * 0x00800000` (2^-126 * (1 - 2^-24)) gives zero. The check rounds in the f32 round mode of `FLOAT_MODE` (bits 1:0), measured in all four modes, while the products themselves are still rounded to nearest even. This is computed exactly from the significands for `v_mul_f32`, `v_mul_legacy_f32`, `v_mullit_f32`, and for `v_fma_f32`, `v_fmac_f32`, `v_fmamk_f32`, `v_fmaak_f32` and `v_fma_mix_f32` when the addend is zero; with a nonzero addend the fused result is kept, so an exact result just below 2^-126 that rounds to it is not flushed. `v_fma_mix_f32` follows this rule with mixed f16/f32 sources too, measured on RDNA2 (gfx1036) with f16 in either multiplicand slot, low and high half selection, and an f32 or f16 zero addend: f32 `0x00801002` times f16 `0x3bff` gives `0x00800000` to nearest at IEEE=0 and IEEE=1, while the neighbours `0x00801001` and `0x00801003` give zero and `0x00800001`. The tiny-product boundary was also measured for both signs in all four f32 round modes at IEEE=0; f16 inputs retain their previous conversion and denormal handling. `v_fma_mixlo_f16` and `v_fma_mixhi_f16` flush their f32 sources too and keep a product below 2^-126, measured on RDNA2 (gfx1036). `v_mad_legacy_f32` and `v_mac_legacy_f32` flush their inputs, their product and their result in every mode, see its entry; their product is still checked for tininess on its exact value, so one below 2^-126 gives zero even when it rounds to 2^-126, as the console's separately rounded encoding is not measured at this boundary. Denormal modes 1 and 2 throw for the other instructions. Not covered, and not measurable on RDNA2 (no such opcodes) or not measured: `v_mac_f32`, `v_mad_f32`, `v_madak_f32` and `v_madmk_f32` (rounded separately), `v_dot2_f32_f16` and `v_div_fmas_f32` (on hardware `v_div_fmas_f32` flushes only its result), f32 interpolation and `ds_add_f32` keep the previous lowering; `ds_min/max/cmpst_f32` flush both operands by the field, see the LDS float atomics entry. With f32 denormals kept (3) the results depend on the driver's default denormal behaviour, as no `DenormPreserve` execution mode is requested, and `v_ldexp_f32` flushes in every mode. - The executable file that [relinker](../../core/relinker/elfpatcher/src/windows/WindowsPeWriter.cpp) generates opens the console when launched, which is inconvenient for playability. - [Overlay output bus](../../core/libs/prx/libSceVideoOut/src/VideoOutDriver.cpp) (libSceVideoOut) - flips on a port opened with `SCE_VIDEO_OUT_BUS_TYPE_OVERLAY` complete and signal their events like main-bus flips, but their buffers are not shown: the overlay is not composited over the main output. Presenting them as whole frames made the window alternate between the two buses - [PA_CL_CLIP_CNTL](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) - `DX_LINEAR_ATTR_CLIP_ENA` (bit 24) clear is drawn like bit 24 set, with Vulkan's clipping of `NoPerspective` outputs; what a clear bit changes on the console is undocumented. From 890169e094558c60f967aa1900c1a02a4e926f2b Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sun, 11 Oct 2026 01:16:53 +0200 Subject: [PATCH 237/460] feat(libc): implement memrchr Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libc/CMakeLists.txt | 9 +++++++++ core/libs/prx/libc/src/Strings.cpp | 10 ++++++++++ core/libs/tests/GuestMemrchr.cpp | 17 +++++++++++++++++ 3 files changed, 36 insertions(+) create mode 100644 core/libs/tests/GuestMemrchr.cpp diff --git a/core/libs/prx/libc/CMakeLists.txt b/core/libs/prx/libc/CMakeLists.txt index 7e732496a..f77c34248 100644 --- a/core/libs/prx/libc/CMakeLists.txt +++ b/core/libs/prx/libc/CMakeLists.txt @@ -233,6 +233,15 @@ if(BUILD_TESTING) set_tests_properties(guest_hostname PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_memrchr_tests ${LIBS_INCLUDE_DIR}/tests/GuestMemrchr.cpp) + target_include_directories(guest_memrchr_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_memrchr_tests PRIVATE libc) + configure_windows_unwind(guest_memrchr_tests) + add_test(NAME guest_memrchr COMMAND guest_memrchr_tests) + if(WIN32) + set_tests_properties(guest_memrchr PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_lifecycle_tests ${LIBS_INCLUDE_DIR}/tests/GuestLifecycle.cpp) target_include_directories(guest_lifecycle_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_lifecycle_tests PRIVATE libc) diff --git a/core/libs/prx/libc/src/Strings.cpp b/core/libs/prx/libc/src/Strings.cpp index 4399cbe08..a0ddafb24 100644 --- a/core/libs/prx/libc/src/Strings.cpp +++ b/core/libs/prx/libc/src/Strings.cpp @@ -73,6 +73,16 @@ const void* APS5_VABI memchr_nid_postfix(const void* s, int c, size_t n) { return std::memchr(s, c, n); } +void* APS5_VABI memrchr_nid_postfix(const void* s, int c, size_t n) { + const auto* bytes = static_cast(s); + const auto value = static_cast(c); + while (n != 0) { + --n; + if (bytes[n] == value) return const_cast(bytes + n); + } + return nullptr; +} + int APS5_VABI strcmp_nid_postfix(const char* s1, const char* s2) { return std::strcmp(s1, s2); } diff --git a/core/libs/tests/GuestMemrchr.cpp b/core/libs/tests/GuestMemrchr.cpp new file mode 100644 index 000000000..8715e30bf --- /dev/null +++ b/core/libs/tests/GuestMemrchr.cpp @@ -0,0 +1,17 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include + +extern "C" { +void* APS5_VABI memrchr_nid_postfix(const void*, int, std::size_t); +} + +int main() { + const unsigned char bytes[] = {0xff, 0x01, 0xff, 0x00}; + const auto* found = static_cast(memrchr_nid_postfix(bytes, 0xff, sizeof(bytes))); + if (found != bytes + 2) std::abort(); + found = static_cast(memrchr_nid_postfix(bytes, 0x1ff, sizeof(bytes))); + if (found != bytes + 2) std::abort(); + found = static_cast(memrchr_nid_postfix(bytes, 0xff, 2)); + if (found != bytes) std::abort(); + if (memrchr_nid_postfix(bytes, 0x02, sizeof(bytes)) != nullptr) std::abort(); +} From 580ac084783fec08e6bf98a22e7bd91327183b7c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jan=20=C5=A0l=C3=A9gl?= Date: Fri, 9 Oct 2026 19:17:52 +0200 Subject: [PATCH 238/460] fix(shader): correct v_log_f32 near 1.0 when the driver's log2 strays from a series The driver's Log2 has an absolute error that does not shrink with the result, so near 1.0 the relative error explodes: on an RTX 3080, v_log_f32 of 1 - 2^-24 is 23,947,836 ULP off. Measured on RDNA2 (gfx1036), the hardware's log2 is within 1 ULP of the correctly rounded value, and an eight-term series in x - 1 is within 2 ULP of both for |x - 1| <= 1/8. The lowering keeps the driver's Log2 and takes the series only inside that band when the two differ by more than 4 ULP, so RADV's native result stays bit-exact and NVIDIA's is within 6 ULP inside the band. agc_driver_log_near_one pins 64 inputs per type from the oracle with per-lane tolerances. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_012GLfKH7Sqe51wX254k1XsV --- .../tests/execution/LogNearOne.cpp | 252 ++++++++++++++++++ .../src/SpirvAlu/SpirvAluEmitterMath.cpp | 20 +- docs/dev/TechnicalDebt.md | 3 +- 3 files changed, 273 insertions(+), 2 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/LogNearOne.cpp diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/LogNearOne.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/LogNearOne.cpp new file mode 100644 index 000000000..882142632 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/LogNearOne.cpp @@ -0,0 +1,252 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 64; +constexpr std::uint32_t Inputs = 4; +constexpr std::uint32_t Results = 4; +constexpr std::uint32_t Columns = 2; +alignas(256) std::array Input{}; +alignas(256) std::array Output{}; + +alignas(256) constexpr std::array Code{ + 0x34020084u, 0xe0381000u, 0x80000401u, 0xbf8c3f70u, 0x7e140280u, 0x7e160280u, 0x7e180280u, 0x7e1a0280u, + 0x7e144f04u, 0x7e16af05u, 0xe0781000u, 0x80010a01u, 0xbf810000u, +}; + +constexpr std::uint32_t Rows[Threads][4] = { + {0x3fc00000u, 0x00003e00u, 0x00000000u, 0x00000000u}, + {0x3f000000u, 0x00003800u, 0x00000000u, 0x00000000u}, + {0x3fa00000u, 0x00003d00u, 0x00000000u, 0x00000000u}, + {0x3f400000u, 0x00003a00u, 0x00000000u, 0x00000000u}, + {0x3f900000u, 0x00003c80u, 0x00000000u, 0x00000000u}, + {0x3f600000u, 0x00003b00u, 0x00000000u, 0x00000000u}, + {0x3f880000u, 0x00003c40u, 0x00000000u, 0x00000000u}, + {0x3f700000u, 0x00003b80u, 0x00000000u, 0x00000000u}, + {0x3f840000u, 0x00003c20u, 0x00000000u, 0x00000000u}, + {0x3f780000u, 0x00003bc0u, 0x00000000u, 0x00000000u}, + {0x3f820000u, 0x00003c10u, 0x00000000u, 0x00000000u}, + {0x3f7c0000u, 0x00003be0u, 0x00000000u, 0x00000000u}, + {0x3f810000u, 0x00003c08u, 0x00000000u, 0x00000000u}, + {0x3f7e0000u, 0x00003bf0u, 0x00000000u, 0x00000000u}, + {0x3f808000u, 0x00003c04u, 0x00000000u, 0x00000000u}, + {0x3f7f0000u, 0x00003bf8u, 0x00000000u, 0x00000000u}, + {0x3f804000u, 0x00003c02u, 0x00000000u, 0x00000000u}, + {0x3f7f8000u, 0x00003bfcu, 0x00000000u, 0x00000000u}, + {0x3f802000u, 0x00003c01u, 0x00000000u, 0x00000000u}, + {0x3f7fc000u, 0x00003bfeu, 0x00000000u, 0x00000000u}, + {0x3f801000u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7fe000u, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800800u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7ff000u, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800400u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7ff800u, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800200u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7ffc00u, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800100u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7ffe00u, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800080u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7fff00u, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800040u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7fff80u, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800020u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7fffc0u, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800010u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7fffe0u, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800008u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7ffff0u, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800004u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7ffff8u, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800002u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7ffffcu, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800001u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7ffffeu, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800000u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f7fffffu, 0x00003bffu, 0x00000000u, 0x00000000u}, + {0x3f800000u, 0x00003c00u, 0x00000000u, 0x00000000u}, + {0x3f600000u, 0x00003b00u, 0x00000000u, 0x00000000u}, + {0x3f900000u, 0x00003c80u, 0x00000000u, 0x00000000u}, + {0x3f666666u, 0x00003b33u, 0x00000000u, 0x00000000u}, + {0x3f8ccccdu, 0x00003c66u, 0x00000000u, 0x00000000u}, + {0x3f000000u, 0x00003800u, 0x00000000u, 0x00000000u}, + {0x40000000u, 0x00004000u, 0x00000000u, 0x00000000u}, + {0x40400000u, 0x00004200u, 0x00000000u, 0x00000000u}, + {0x41200000u, 0x00004900u, 0x00000000u, 0x00000000u}, + {0x3dcccccdu, 0x00002e66u, 0x00000000u, 0x00000000u}, + {0x3a83126fu, 0x00001419u, 0x00000000u, 0x00000000u}, + {0x447a0000u, 0x000063d0u, 0x00000000u, 0x00000000u}, + {0x00800000u, 0x00000400u, 0x00000000u, 0x00000000u}, + {0x7f7fffffu, 0x00007bffu, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0xbf800000u, 0x0000bc00u, 0x00000000u, 0x00000000u} +}; + +constexpr std::uint32_t Expected[Threads][Columns] = { + {0x3f15c01au, 0x000038aeu}, + {0xbf800000u, 0x0000bc00u}, + {0x3ea4d3c2u, 0x00003527u}, + {0xbed47fccu, 0x0000b6a4u}, + {0x3e2e00d2u, 0x00003170u}, + {0xbe4544c0u, 0x0000b22au}, + {0x3db31fb8u, 0x00002d99u}, + {0xbdbeb025u, 0x0000adf6u}, + {0x3d35d69cu, 0x000029afu}, + {0xbd3b9ca6u, 0x0000a9ddu}, + {0x3cb73cb4u, 0x000025bau}, + {0xbcba1f74u, 0x0000a5d1u}, + {0x3c37f286u, 0x000021c0u}, + {0xbc3963ddu, 0x0000a1cbu}, + {0x3bb84e24u, 0x00001dc2u}, + {0xbbb906ceu, 0x00009dc8u}, + {0x3b387c20u, 0x000019c4u}, + {0xbb38d875u, 0x000099c7u}, + {0x3ab8932au, 0x000015c5u}, + {0xbab8c155u, 0x000095c6u}, + {0x3a389eb2u, 0x00000000u}, + {0xba38b5c7u, 0x000091c6u}, + {0x39b8a476u, 0x00000000u}, + {0xb9b8b001u, 0x000091c6u}, + {0x3938a759u, 0x00000000u}, + {0xb938ad1eu, 0x000091c6u}, + {0x38b8a8cau, 0x00000000u}, + {0xb8b8abadu, 0x000091c6u}, + {0x3838a983u, 0x00000000u}, + {0xb838aaf4u, 0x000091c6u}, + {0x37b8a9dfu, 0x00000000u}, + {0xb7b8aa98u, 0x000091c6u}, + {0x3738aa0du, 0x00000000u}, + {0xb738aa6au, 0x000091c6u}, + {0x36b8aa24u, 0x00000000u}, + {0xb6b8aa53u, 0x000091c6u}, + {0x3638aa30u, 0x00000000u}, + {0xb638aa47u, 0x000091c6u}, + {0x35b8aa36u, 0x00000000u}, + {0xb5b8aa41u, 0x000091c6u}, + {0x3538aa39u, 0x00000000u}, + {0xb538aa3eu, 0x000091c6u}, + {0x34b8aa3au, 0x00000000u}, + {0xb4b8aa3du, 0x000091c6u}, + {0x3438aa3bu, 0x00000000u}, + {0xb438aa3cu, 0x000091c6u}, + {0x00000000u, 0x00000000u}, + {0xb3b8aa3cu, 0x000091c6u}, + {0x00000000u, 0x00000000u}, + {0xbe4544c0u, 0x0000b22au}, + {0x3e2e00d2u, 0x00003170u}, + {0xbe1ba6b5u, 0x0000b0deu}, + {0x3e0ccdbbu, 0x00003062u}, + {0xbf800000u, 0x0000bc00u}, + {0x3f800000u, 0x00003c00u}, + {0x3fcae00du, 0x00003e57u}, + {0x40549a78u, 0x000042a5u}, + {0xc0549a79u, 0x0000c2a5u}, + {0xc11f73dbu, 0x0000c8fcu}, + {0x411f73dau, 0x000048fcu}, + {0xc2fc0000u, 0x0000cb00u}, + {0x42ffffffu, 0x00004c00u}, + {0xff800000u, 0x0000fc00u}, + {0xffc00000u, 0x0000fe00u} +}; + +constexpr const char* Names[Columns] = {"v_log_f32", "v_log_f16"}; +constexpr std::uint32_t Tolerance[Threads] = {12u, 12u, 12u, 12u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 6u, 12u, 12u, 12u, 12u, 12u, 12u, 12u, 2u, 2u, 2u, 2u}; + +std::array BufferDescriptor(const void* data, std::uint32_t bytes) { + const auto address = reinterpret_cast(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; +} + +std::string Hex(std::uint32_t value) { + char text[16]; + std::snprintf(text, sizeof(text), "0x%08x", value); + return text; +} + +std::uint32_t Ordered(std::uint32_t bits, std::uint32_t width) { + const std::uint32_t sign = 1u << (width - 1u); + return (bits & sign) != 0u ? sign - (bits & ~sign) : bits | sign; +} + +std::uint32_t Distance(std::uint32_t actual, std::uint32_t expected, std::uint32_t width) { + const std::uint32_t a = Ordered(actual, width); + const std::uint32_t e = Ordered(expected, width); + return a > e ? a - e : e - a; +} + +bool IsPinned(std::uint32_t bits, std::uint32_t width) { + const std::uint32_t magnitude = width == 16u ? bits & 0x7fffu : bits & 0x7fffffffu; + const std::uint32_t infinity = width == 16u ? 0x7c00u : 0x7f800000u; + return magnitude == 0u || magnitude >= infinity; +} + +void Run(AgcDriver::VulkanDevice& device) { + for (std::uint32_t tid = 0; tid < Threads; ++tid) std::copy(std::begin(Rows[tid]), std::end(Rows[tid]), &Input[tid * Inputs]); + Output.fill(0xdeadbeefu); + std::vector userData(8, 0u); + const auto input = BufferDescriptor(Input.data(), static_cast(Input.size() * 4u)); + const auto output = BufferDescriptor(Output.data(), static_cast(Output.size() * 4u)); + std::copy(input.begin(), input.end(), userData.begin()); + std::copy(output.begin(), output.end(), userData.begin() + 4); + const std::span code(Code); + const std::array memory{{reinterpret_cast(code.data()), std::as_bytes(code)}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {false, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + request.context.floatMode = ShaderRecompiler::ShaderFloatMode{0xc0u, true, false, false}; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); +} + +void Check() { + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + const std::uint32_t* out = &Output[tid * Results]; + for (std::uint32_t column = 0; column < Columns; ++column) { + const std::uint32_t width = column == 0u ? 32u : 16u; + const std::uint32_t mask = width == 16u ? 0xffffu : 0xffffffffu; + const std::uint32_t actual = out[column] & mask; + const std::uint32_t expected = Expected[tid][column] & mask; + const std::string name = "log near one: lane " + std::to_string(tid) + " " + Names[column] + " of " + Hex(Rows[tid][column]) + " is " + Hex(actual) + ", expected " + Hex(expected); + if (IsPinned(expected, width)) { + Require(actual == expected, name); + continue; + } + Require(Distance(actual, expected, width) <= Tolerance[tid], name + " (tolerance " + std::to_string(Tolerance[tid]) + " ulp)"); + } + } +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + Run(*device); + Check(); + std::puts("log near one tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp index 6a78209dc..2e3525069 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp @@ -734,8 +734,26 @@ std::uint32_t EmitFPExp2(SpirvEmitterState& state, std::uint32_t arg0) { return EmitExt(state, TypeF32(state), GLSLstd450Exp2, {EmitFlushF32DenormToSignedZero(state, arg0)}); } +std::uint32_t Log2SeriesNearOne(SpirvEmitterState& state, std::uint32_t difference) { + auto polynomial = ConstantF32(state, 0xbe000000u); + for (const std::uint32_t coefficient : {0x3e124925u, 0xbe2aaaabu, 0x3e4ccccdu, 0xbe800000u, 0x3eaaaaabu, 0xbf000000u, 0x3f800000u}) { + polynomial = Exact(state, EmitExt(state, TypeF32(state), GLSLstd450Fma, {polynomial, difference, ConstantF32(state, coefficient)})); + } + const auto product = Exact(state, Binary(state, spv::OpFMul, TypeF32(state), difference, polynomial)); + return Exact(state, Binary(state, spv::OpFMul, TypeF32(state), product, ConstantF32(state, 0x3fb8aa3bu))); +} + std::uint32_t EmitFPLog2(SpirvEmitterState& state, std::uint32_t arg0) { - return EmitExt(state, TypeF32(state), GLSLstd450Log2, {EmitFlushF32DenormToSignedZero(state, arg0)}); + const auto source = EmitFlushF32DenormToSignedZero(state, arg0); + const auto value = EmitExt(state, TypeF32(state), GLSLstd450Log2, {source}); + const auto difference = Exact(state, Binary(state, spv::OpFSub, TypeF32(state), source, ConstantF32(state, 0x3f800000u))); + const auto series = Log2SeriesNearOne(state, difference); + const auto valueBits = Unary(state, spv::OpBitcast, TypeU32(state), value); + const auto seriesBits = Unary(state, spv::OpBitcast, TypeU32(state), series); + const auto distance = EmitExt(state, TypeU32(state), GLSLstd450UMin, {Binary(state, spv::OpISub, TypeU32(state), valueBits, seriesBits), Binary(state, spv::OpISub, TypeU32(state), seriesBits, valueBits)}); + const auto near = Binary(state, spv::OpFOrdLessThanEqual, TypeBool(state), EmitExt(state, TypeF32(state), GLSLstd450FAbs, {difference}), ConstantF32(state, 0x3e000000u)); + const auto far = Binary(state, spv::OpUGreaterThan, TypeBool(state), distance, ConstantU32(state, 4u)); + return Select(state, TypeF32(state), Binary(state, spv::OpLogicalAnd, TypeBool(state), near, far), series, value); } std::uint32_t TrigLinearNearZero(SpirvEmitterState& state, std::uint32_t value, std::uint32_t distance, std::uint32_t linearConstant) { diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 6f7560b6c..675b02a4a 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -410,7 +410,8 @@ Throughout the project, every function at every stage either **does exactly what - [PA_CL_CLIP_CNTL](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) - `DX_LINEAR_ATTR_CLIP_ENA` (bit 24) clear is drawn like bit 24 set, with Vulkan's clipping of `NoPerspective` outputs; what a clear bit changes on the console is undocumented. - [10_11_11 unorm color targets](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) render into R32_UINT attachments: the pixel shader packs each export, rounded to nearest even like format-30 image stores, which is assumed to match the CB's conversion. - [PA_SU_SC_MODE_CNTL.PROVOKING_VTX_LAST](../../core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp) (libSceAgcDriver) uses Vulkan's last-vertex mode for native primitives. Bit 19 follows [opengnm's `flatshade_first` mapping](https://github.com/blackbearreloaded/ps5-opengl/blob/db752da4c89380af228497b5b6852163a4e61069/src/gallium/ps5/ps5_screen.c#L2244); vertex selection follows [Vulkan](https://docs.vulkan.org/spec/latest/chapters/drawing.html#drawing-primitive-topologies), without console measurements. Last mode throws without `VK_EXT_provoking_vertex`, for generated rectangle, mesh or tessellation primitives, explicit per-vertex flat inputs, and triangle strips with per-vertex inputs, whose [ordering depends on the device](https://docs.vulkan.org/refpages/latest/refpages/source/VkPhysicalDeviceFragmentShaderBarycentricPropertiesKHR.html). -- [v_rcp, v_rsq, v_sqrt, v_log, v_exp, v_sin and v_cos](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) (f32 and f16) use the driver's approximation for finite results, not the console's: the low bits can differ. Only NaN, invalid, zero and infinite results and the exact sin/cos zeros are chosen on the bits. Measured on RDNA2 (gfx1036) over every f16 input and 32,832 f32 inputs: RADV returns the hardware's bits for every input of rcp, rsq, sqrt and log and all but one of exp (f16 0x11c5, 1 ULP); NVIDIA (615.71.09) stays within 2 ULP except `v_log_f32` near 1.0, where its log2 is up to 24 million ULP off. [v_sin and v_cos](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) take the driver's sine or cosine of the radians of the exact cycle, except within 2^-12 of a turn from a zero of the function, where the result is the linear term of the exact distance to the zero: the hardware's constant 6.2831826 (0x40c90fd5) at the integers, which the hardware applies exactly to inputs below 2^-12 of a turn, and the f32 2π at the half and quarter turns, where the hardware is within 7 ULP of it. RADV returns the hardware's bits for every f16 input and for 32,422 sine and 32,486 cosine results of the 32,832 f32 inputs; the rest are 1 ULP off, except up to 14 ULP on either side of non-zero integers and up to 15 ULP beyond ±256 cycles, where the hardware's own reduction differs from the exact fraction, and up to 6 ULP inside the half- and quarter-turn bands. NVIDIA's sine is within 1 ULP on 93% of the f32 inputs and up to 1,166 ULP off just outside the bands, its cosine within 163 ULP. A denormal f32 input with f32 denormals kept gives the hardware the input times 6.2831826; both hosts flush it. +- [v_rcp, v_rsq, v_sqrt, v_log, v_exp, v_sin and v_cos](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) (f32 and f16) use the driver's approximation for finite results, not the console's: the low bits can differ. Only NaN, invalid, zero and infinite results and the exact sin/cos zeros are chosen on the bits. Measured on RDNA2 (gfx1036) over every f16 input and 32,832 f32 inputs: RADV returns the hardware's bits for every input of rcp, rsq, sqrt and log and all but one of exp (f16 0x11c5, 1 ULP); NVIDIA (615.71.09) stays within 2 ULP except `v_log_f32` near 1.0 (next entry). [v_sin and v_cos](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) take the driver's sine or cosine of the radians of the exact cycle, except within 2^-12 of a turn from a zero of the function, where the result is the linear term of the exact distance to the zero: the hardware's constant 6.2831826 (0x40c90fd5) at the integers, which the hardware applies exactly to inputs below 2^-12 of a turn, and the f32 2π at the half and quarter turns, where the hardware is within 7 ULP of it. RADV returns the hardware's bits for every f16 input and for 32,422 sine and 32,486 cosine results of the 32,832 f32 inputs; the rest are 1 ULP off, except up to 14 ULP on either side of non-zero integers and up to 15 ULP beyond ±256 cycles, where the hardware's own reduction differs from the exact fraction, and up to 6 ULP inside the half- and quarter-turn bands. NVIDIA's sine is within 1 ULP on 93% of the f32 inputs and up to 1,166 ULP off just outside the bands, its cosine within 163 ULP. A denormal f32 input with f32 denormals kept gives the hardware the input times 6.2831826; both hosts flush it. +- [v_log_f32](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) keeps the driver's log2 unless, for inputs within 1/8 of 1.0, it is more than 4 ULP from an eight-term series in x - 1, which then replaces it. Measured on RDNA2 (gfx1036) on 9,280 inputs within 1/8 of 1.0 (other than 1.0), the hardware is within 1 ULP of the correctly rounded log2 and the series within 2 ULP of both, so a kept driver value is within 6 ULP of the hardware; RADV (Mesa 26.2.4) returns the hardware's bits on all of them and the gate never fires; NVIDIA (615.71.09) was up to 23,947,836 ULP off at 1 - 2^-24 and is within 6 ULP inside the band after the change; outside it its own log2 stays up to 9 ULP off (measured up to 1.375). - [Dual-source blending](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) takes the MRT1 export as the second source of the MRT0 blend, as RADV and radeonsi program it; what the CB does with other export or target layouts is unknown, so they throw. - `--to-intel` does not lower RDPRU/MCOMMIT; the [matcher](../../core/relinker/codegen/src/x86/Amd64OnlyInstructionMatcher.cpp) fails the relink instead. - [Ordered not-equal compares](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) (shader recompiler) - `FPOrdNotEqual32` is emitted as `OpFUnordNotEqual` with both operands checked for NaN on their bits instead of `OpFOrdNotEqual`; the two agree for every input. The AMD Windows driver 32.0.13058.8001 (Radeon 860M, RDNA 3.5) returns 1 for a NaN operand when any float form of the compare feeds a ballot on a 32-wide subgroup (#825) From 447f3a5080a7869a75d3575ef82092162383e2a3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jan=20=C5=A0l=C3=A9gl?= Date: Sun, 11 Oct 2026 01:25:46 +0200 Subject: [PATCH 239/460] docs: note the unconditional log2 series and its f16 path v_log_f16 is lowered through the same f32 log2, so the entry names both and states the f16 effect measured over every f16 input (one NVIDIA result moves onto the hardware's bits, none on RADV). The series is chosen with OpSelect, so its cost applies to every v_log. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Fnrk5Dazjb2tLooVSpdRQq --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 675b02a4a..7d40d7db6 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -411,7 +411,7 @@ Throughout the project, every function at every stage either **does exactly what - [10_11_11 unorm color targets](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) render into R32_UINT attachments: the pixel shader packs each export, rounded to nearest even like format-30 image stores, which is assumed to match the CB's conversion. - [PA_SU_SC_MODE_CNTL.PROVOKING_VTX_LAST](../../core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp) (libSceAgcDriver) uses Vulkan's last-vertex mode for native primitives. Bit 19 follows [opengnm's `flatshade_first` mapping](https://github.com/blackbearreloaded/ps5-opengl/blob/db752da4c89380af228497b5b6852163a4e61069/src/gallium/ps5/ps5_screen.c#L2244); vertex selection follows [Vulkan](https://docs.vulkan.org/spec/latest/chapters/drawing.html#drawing-primitive-topologies), without console measurements. Last mode throws without `VK_EXT_provoking_vertex`, for generated rectangle, mesh or tessellation primitives, explicit per-vertex flat inputs, and triangle strips with per-vertex inputs, whose [ordering depends on the device](https://docs.vulkan.org/refpages/latest/refpages/source/VkPhysicalDeviceFragmentShaderBarycentricPropertiesKHR.html). - [v_rcp, v_rsq, v_sqrt, v_log, v_exp, v_sin and v_cos](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) (f32 and f16) use the driver's approximation for finite results, not the console's: the low bits can differ. Only NaN, invalid, zero and infinite results and the exact sin/cos zeros are chosen on the bits. Measured on RDNA2 (gfx1036) over every f16 input and 32,832 f32 inputs: RADV returns the hardware's bits for every input of rcp, rsq, sqrt and log and all but one of exp (f16 0x11c5, 1 ULP); NVIDIA (615.71.09) stays within 2 ULP except `v_log_f32` near 1.0 (next entry). [v_sin and v_cos](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) take the driver's sine or cosine of the radians of the exact cycle, except within 2^-12 of a turn from a zero of the function, where the result is the linear term of the exact distance to the zero: the hardware's constant 6.2831826 (0x40c90fd5) at the integers, which the hardware applies exactly to inputs below 2^-12 of a turn, and the f32 2π at the half and quarter turns, where the hardware is within 7 ULP of it. RADV returns the hardware's bits for every f16 input and for 32,422 sine and 32,486 cosine results of the 32,832 f32 inputs; the rest are 1 ULP off, except up to 14 ULP on either side of non-zero integers and up to 15 ULP beyond ±256 cycles, where the hardware's own reduction differs from the exact fraction, and up to 6 ULP inside the half- and quarter-turn bands. NVIDIA's sine is within 1 ULP on 93% of the f32 inputs and up to 1,166 ULP off just outside the bands, its cosine within 163 ULP. A denormal f32 input with f32 denormals kept gives the hardware the input times 6.2831826; both hosts flush it. -- [v_log_f32](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) keeps the driver's log2 unless, for inputs within 1/8 of 1.0, it is more than 4 ULP from an eight-term series in x - 1, which then replaces it. Measured on RDNA2 (gfx1036) on 9,280 inputs within 1/8 of 1.0 (other than 1.0), the hardware is within 1 ULP of the correctly rounded log2 and the series within 2 ULP of both, so a kept driver value is within 6 ULP of the hardware; RADV (Mesa 26.2.4) returns the hardware's bits on all of them and the gate never fires; NVIDIA (615.71.09) was up to 23,947,836 ULP off at 1 - 2^-24 and is within 6 ULP inside the band after the change; outside it its own log2 stays up to 9 ULP off (measured up to 1.375). +- [v_log_f32 and v_log_f16](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) keep the driver's log2 unless, for inputs within 1/8 of 1.0, it is more than 4 ULP from an eight-term series in x - 1, which then replaces it. `v_log_f16` goes through the same f32 lowering and is rounded to half. The series is evaluated for every `v_log` and chosen with `OpSelect` rather than a branch, so its seven fused multiply-adds and two multiplies are an unconditional cost. Measured on RDNA2 (gfx1036) on 9,280 f32 inputs within 1/8 of 1.0 (other than 1.0), the hardware is within 1 ULP of the correctly rounded log2 and the series within 2 ULP of both, so a kept driver value is within 6 ULP of the hardware; RADV (Mesa 26.2.4) returns the hardware's bits on all of them and the gate never fires; NVIDIA (615.71.09) was up to 23,947,836 ULP off at 1 - 2^-24 and is within 6 ULP inside the band after the change; outside it its own log2 stays up to 9 ULP off (measured up to 1.375). Over every f16 input, the change moves one `v_log_f16` result on NVIDIA (0x3bed, 1 ULP, onto the hardware's bits) and none on RADV. - [Dual-source blending](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) takes the MRT1 export as the second source of the MRT0 blend, as RADV and radeonsi program it; what the CB does with other export or target layouts is unknown, so they throw. - `--to-intel` does not lower RDPRU/MCOMMIT; the [matcher](../../core/relinker/codegen/src/x86/Amd64OnlyInstructionMatcher.cpp) fails the relink instead. - [Ordered not-equal compares](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) (shader recompiler) - `FPOrdNotEqual32` is emitted as `OpFUnordNotEqual` with both operands checked for NaN on their bits instead of `OpFOrdNotEqual`; the two agree for every input. The AMD Windows driver 32.0.13058.8001 (Radeon 860M, RDNA 3.5) returns 1 for a NaN operand when any float form of the compare feeds a ballot on a 32-wide subgroup (#825) From 3983f16e37297a90c39c206d6d2051a7fd789acf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 01:33:54 +0200 Subject: [PATCH 240/460] docs(technical-debt): v_log near 1.0 bound from the exhaustive band sweep Co-Authored-By: Claude --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 7d40d7db6..2d7035957 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -411,7 +411,7 @@ Throughout the project, every function at every stage either **does exactly what - [10_11_11 unorm color targets](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) render into R32_UINT attachments: the pixel shader packs each export, rounded to nearest even like format-30 image stores, which is assumed to match the CB's conversion. - [PA_SU_SC_MODE_CNTL.PROVOKING_VTX_LAST](../../core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp) (libSceAgcDriver) uses Vulkan's last-vertex mode for native primitives. Bit 19 follows [opengnm's `flatshade_first` mapping](https://github.com/blackbearreloaded/ps5-opengl/blob/db752da4c89380af228497b5b6852163a4e61069/src/gallium/ps5/ps5_screen.c#L2244); vertex selection follows [Vulkan](https://docs.vulkan.org/spec/latest/chapters/drawing.html#drawing-primitive-topologies), without console measurements. Last mode throws without `VK_EXT_provoking_vertex`, for generated rectangle, mesh or tessellation primitives, explicit per-vertex flat inputs, and triangle strips with per-vertex inputs, whose [ordering depends on the device](https://docs.vulkan.org/refpages/latest/refpages/source/VkPhysicalDeviceFragmentShaderBarycentricPropertiesKHR.html). - [v_rcp, v_rsq, v_sqrt, v_log, v_exp, v_sin and v_cos](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) (f32 and f16) use the driver's approximation for finite results, not the console's: the low bits can differ. Only NaN, invalid, zero and infinite results and the exact sin/cos zeros are chosen on the bits. Measured on RDNA2 (gfx1036) over every f16 input and 32,832 f32 inputs: RADV returns the hardware's bits for every input of rcp, rsq, sqrt and log and all but one of exp (f16 0x11c5, 1 ULP); NVIDIA (615.71.09) stays within 2 ULP except `v_log_f32` near 1.0 (next entry). [v_sin and v_cos](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) take the driver's sine or cosine of the radians of the exact cycle, except within 2^-12 of a turn from a zero of the function, where the result is the linear term of the exact distance to the zero: the hardware's constant 6.2831826 (0x40c90fd5) at the integers, which the hardware applies exactly to inputs below 2^-12 of a turn, and the f32 2π at the half and quarter turns, where the hardware is within 7 ULP of it. RADV returns the hardware's bits for every f16 input and for 32,422 sine and 32,486 cosine results of the 32,832 f32 inputs; the rest are 1 ULP off, except up to 14 ULP on either side of non-zero integers and up to 15 ULP beyond ±256 cycles, where the hardware's own reduction differs from the exact fraction, and up to 6 ULP inside the half- and quarter-turn bands. NVIDIA's sine is within 1 ULP on 93% of the f32 inputs and up to 1,166 ULP off just outside the bands, its cosine within 163 ULP. A denormal f32 input with f32 denormals kept gives the hardware the input times 6.2831826; both hosts flush it. -- [v_log_f32 and v_log_f16](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) keep the driver's log2 unless, for inputs within 1/8 of 1.0, it is more than 4 ULP from an eight-term series in x - 1, which then replaces it. `v_log_f16` goes through the same f32 lowering and is rounded to half. The series is evaluated for every `v_log` and chosen with `OpSelect` rather than a branch, so its seven fused multiply-adds and two multiplies are an unconditional cost. Measured on RDNA2 (gfx1036) on 9,280 f32 inputs within 1/8 of 1.0 (other than 1.0), the hardware is within 1 ULP of the correctly rounded log2 and the series within 2 ULP of both, so a kept driver value is within 6 ULP of the hardware; RADV (Mesa 26.2.4) returns the hardware's bits on all of them and the gate never fires; NVIDIA (615.71.09) was up to 23,947,836 ULP off at 1 - 2^-24 and is within 6 ULP inside the band after the change; outside it its own log2 stays up to 9 ULP off (measured up to 1.375). Over every f16 input, the change moves one `v_log_f16` result on NVIDIA (0x3bed, 1 ULP, onto the hardware's bits) and none on RADV. +- [v_log_f32 and v_log_f16](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) keep the driver's log2 unless, for inputs within 1/8 of 1.0, it is more than 4 ULP from an eight-term series in x - 1, which then replaces it. `v_log_f16` goes through the same f32 lowering and is rounded to half. The series is evaluated for every `v_log` and chosen with `OpSelect` rather than a branch, so its seven fused multiply-adds and two multiplies are an unconditional cost. Measured on RDNA2 over all 3,145,728 f32 inputs within 1/8 of 1.0 (gfx1035, IEEE 0, f32 denormals flushed), the hardware is within 1 ULP of the correctly rounded log2 and the series within 2 ULP of it and within 3 ULP of the hardware (2 except at 0x3f752e47, 0x3f7baab0 and 0x3f7bc4fb), so a kept driver value is within 7 ULP of the hardware and a driver that returns the hardware's bits never triggers the gate; RADV (Mesa 26.2.4) returns the hardware's bits on the 9,280 such inputs measured on gfx1036; NVIDIA (615.71.09) was up to 23,947,836 ULP off at 1 - 2^-24 and is within 6 ULP inside the band after the change; outside it its own log2 stays up to 9 ULP off (measured up to 1.375). Over every f16 input, the change moves one `v_log_f16` result on NVIDIA (0x3bed, 1 ULP, onto the hardware's bits) and none on RADV. - [Dual-source blending](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) takes the MRT1 export as the second source of the MRT0 blend, as RADV and radeonsi program it; what the CB does with other export or target layouts is unknown, so they throw. - `--to-intel` does not lower RDPRU/MCOMMIT; the [matcher](../../core/relinker/codegen/src/x86/Amd64OnlyInstructionMatcher.cpp) fails the relink instead. - [Ordered not-equal compares](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) (shader recompiler) - `FPOrdNotEqual32` is emitted as `OpFUnordNotEqual` with both operands checked for NaN on their bits instead of `OpFOrdNotEqual`; the two agree for every input. The AMD Windows driver 32.0.13058.8001 (Radeon 860M, RDNA 3.5) returns 1 for a NaN operand when any float form of the compare feeds a ballot on a 32-wide subgroup (#825) From 2dd453dae0804e58c9673bf26280f7a3cea11fb6 Mon Sep 17 00:00:00 2001 From: Daan Dehing Date: Sat, 10 Oct 2026 18:07:35 +0000 Subject: [PATCH 241/460] fix(agc): drain the registration preparation pool before the device is reset Background registration preparations (#2583) copy the device's SpirvTarget, whose capability and extension spans point into the device's shader profile. Driver::stop reset the device without waiting for the detached pool, so a preparation still queued or running read freed extension names and agc_driver crashed at exit on RADV. Driver::stop now closes the pool after joining the queue workers: it stops queueing and waits until no task is queued or running. A preparation submitted after that runs on the caller's thread, which holds its own reference to the device, like the existing fallback for a pool without threads. Fixes #2891 Co-Authored-By: Claude Opus 5.5 --- .../include/Driver/Shaders/ShaderRegistry.hpp | 3 + .../Execution/src/Driver/Lifecycle.cpp | 2 + .../src/Driver/Shaders/ShaderRegistry.cpp | 41 ++++++++-- .../execution/RegistrationPrepareShutdown.cpp | 76 +++++++++++++++++++ 4 files changed, 114 insertions(+), 8 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/RegistrationPrepareShutdown.cpp diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp index ac19eec56..25fd58e2d 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -92,6 +93,8 @@ ShaderSnapshot PrepareNullPixelProgram(const VulkanDevice& device); std::optional RegisteredFloatMode(const ShaderSnapshot& snapshot); std::shared_ptr RegisteredProgram(const ShaderRegistry& registry, std::uint64_t address, std::initializer_list types); void PublishRegisteredShader(std::shared_ptr& registry, const std::shared_ptr& snapshot); +void SubmitRegistrationPreparation(std::function task); +void CloseRegistrationPreparation(); void ResolvePreparedGraphics(const ShaderSnapshot& front, const std::shared_ptr& fragment, std::uint32_t primitiveType, const ShaderRecompiler::SpirvTarget& target); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Lifecycle.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Lifecycle.cpp index ac5927eed..1dbe1cfd1 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Lifecycle.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Lifecycle.cpp @@ -1,5 +1,6 @@ #include "prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp" #include "prx/libSceAgcDriver/Execution/include/Driver/Diagnostics.hpp" +#include "prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp" #include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" #include "prx/libSceAgcDriver/Execution/include/WorkerSampler.hpp" #include "prx/libSceAgcDriver/Execution/include/ProfileOutput.hpp" @@ -44,6 +45,7 @@ void Driver::stop() { if (worker.thread.joinable()) worker.thread.join(); } StopWorkerSampler(); + CloseRegistrationPreparation(); std::lock_guard gpuLock(GuestMemory::GpuMutex()); device.Reset(); replacedDevices.clear(); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp index 1ae54ed52..d01847729 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp @@ -551,15 +551,22 @@ public: } void Submit(std::function task) { - if (threads == 0) { - task(); - return; - } { - std::lock_guard lock(mutex); - tasks.push_back(std::move(task)); + std::unique_lock lock(mutex); + if (threads != 0 && !closed) { + tasks.push_back(std::move(task)); + lock.unlock(); + ready.notify_one(); + return; + } } - ready.notify_one(); + task(); + } + + void Close() { + std::unique_lock lock(mutex); + closed = true; + idle.wait(lock, [&] { return tasks.empty() && running == 0; }); } private: @@ -587,16 +594,26 @@ private: pool.ready.wait(lock, [&] { return !pool.tasks.empty(); }); task = std::move(pool.tasks.front()); pool.tasks.pop_front(); + ++pool.running; } task(); + task = nullptr; + { + std::lock_guard lock(pool.mutex); + --pool.running; + if (pool.tasks.empty() && pool.running == 0) pool.idle.notify_all(); + } } return nullptr; } std::mutex mutex; std::condition_variable ready; + std::condition_variable idle; std::deque> tasks; unsigned threads = 0; + unsigned running = 0; + bool closed = false; }; bool AsyncRegistrationPrepare() { @@ -605,7 +622,7 @@ bool AsyncRegistrationPrepare() { } void PrepareRegisteredInBackground(std::shared_ptr snapshot, std::shared_ptr plan) { - RegistrationPreparePool::Instance().Submit([snapshot = std::move(snapshot), plan = std::move(plan)] { + SubmitRegistrationPreparation([snapshot = std::move(snapshot), plan = std::move(plan)] { PerformanceContext timingContext(FrameTiming::Preparation()); std::vector entries; std::exception_ptr failure; @@ -631,6 +648,14 @@ void PrepareRegisteredInBackground(std::shared_ptr snapsho } +void SubmitRegistrationPreparation(std::function task) { + RegistrationPreparePool::Instance().Submit(std::move(task)); +} + +void CloseRegistrationPreparation() { + RegistrationPreparePool::Instance().Close(); +} + std::vector PrepareGraphicsStages(const DrawDecode& decoded, const ShaderRecompiler::SpirvTarget& target) { ShaderPreparationTransaction transaction; require(decoded.programs.size() == decoded.roles.size(), "graphics ABI program roles are incomplete"); diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/RegistrationPrepareShutdown.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/RegistrationPrepareShutdown.cpp new file mode 100644 index 000000000..94f3c4606 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/RegistrationPrepareShutdown.cpp @@ -0,0 +1,76 @@ +#include "prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp" +#include "prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp" +#include "prx/libc/include/Shutdown.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +void Require(bool condition, const std::string& message) { + if (!condition) throw std::runtime_error("registration prepare shutdown: " + message); +} + +void CheckShutdownWaits() { + std::mutex mutex; + std::condition_variable changed; + bool started = false; + bool released = false; + std::atomic finished{false}; + std::atomic returned{false}; + std::atomic finishedAtReturn{false}; + AgcDriver::DriverDetail::Driver::Get(); + AgcDriver::DriverDetail::SubmitRegistrationPreparation([&] { + std::unique_lock lock(mutex); + started = true; + changed.notify_all(); + changed.wait(lock, [&] { return released; }); + finished.store(true); + }); + { + std::unique_lock lock(mutex); + changed.wait(lock, [&] { return started; }); + } + std::thread shutdown([&] { + LibcRunShutdown_nid_postfix(); + finishedAtReturn.store(finished.load()); + returned.store(true); + }); + std::this_thread::sleep_for(std::chrono::milliseconds(200)); + const bool returnedEarly = returned.load(); + { + std::lock_guard lock(mutex); + released = true; + } + changed.notify_all(); + shutdown.join(); + Require(!returnedEarly, "shutdown returned while a registration preparation was still running"); + Require(finishedAtReturn.load(), "shutdown returned before the running registration preparation finished"); +} + +void CheckLateSubmissionRunsInline() { + const auto caller = std::this_thread::get_id(); + std::thread::id ran; + AgcDriver::DriverDetail::SubmitRegistrationPreparation([&] { ran = std::this_thread::get_id(); }); + Require(ran == caller, "a registration preparation submitted after shutdown did not run on the caller's thread"); +} + +} + +int main() { + try { + CheckShutdownWaits(); + CheckLateSubmissionRunsInline(); + std::puts("registration prepare shutdown tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} From 6825e0237225989fed3817591fcbaa7cdcede236 Mon Sep 17 00:00:00 2001 From: xXPBSXx <20614131+xXPBSXx@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:35:53 +0200 Subject: [PATCH 242/460] fix(agc): ignore the window offset enable bit of the raster mode The window offset register is already required to be zero, so the enable bit has no effect on a draw that gets this far. Co-Authored-By: Claude Sonnet 5.5 --- core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp | 2 +- core/libs/prx/libSceAgcDriver/tests/Graphics.cpp | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp index 60d0c2dad..f51814337 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp @@ -615,7 +615,7 @@ State DecodeState(const QueueState& queue) { // explicitly; KEEP_TOGETHER_ENABLE (bit 24) only affects primitive distribution across the chip. const bool pointsOrLines = result.topology == VK_PRIMITIVE_TOPOLOGY_POINT_LIST || result.topology == VK_PRIMITIVE_TOPOLOGY_LINE_LIST; const auto lineOffset = (raster & 0x1800u) != 0 && !pointsOrLines ? 0x2000u : 0u; - const auto rasterMode = raster & ~0x7u & ~(1u << 24u) & ~0x1800u & ~lineOffset & ~(1u << 19u); + const auto rasterMode = raster & ~0x7u & ~(1u << 24u) & ~0x1800u & ~lineOffset & ~(1u << 19u) & ~(1u << 16u); if (rasterMode != 0 && rasterMode != 0x240u && rasterMode != 0x248u) throw std::runtime_error("AGC graphics: " + zeroMessage(0x205, raster, "polygon mode, depth bias or nonstandard rasterization")); result.cullMode = ((raster & 1u) != 0 ? VK_CULL_MODE_FRONT_BIT : 0u) | ((raster & 2u) != 0 ? VK_CULL_MODE_BACK_BIT : 0u); if (result.rectList) result.cullMode = VK_CULL_MODE_NONE; diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index ee6645aad..17670e911 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -1111,6 +1111,12 @@ void DepthBoundsBiasTests() { queue.context[0x205] = 0x00003a46u; state = AgcDriver::Graphics::DecodeState(queue); Require(state.depthBias && state.depthBiasConstant == 4.0f && state.cullMode == VK_CULL_MODE_BACK_BIT, "a triangle draw with the point and line offset enable lost its depth bias"); + queue.context[0x205] = 0x00013a46u; + state = AgcDriver::Graphics::DecodeState(queue); + Require(state.depthBias && state.depthBiasConstant == 4.0f && state.cullMode == VK_CULL_MODE_BACK_BIT, "the window offset enable bit changed the raster decode while the window offset is zero"); + queue.context[0x80] = 1; + expectFailure([&] { AgcDriver::Graphics::DecodeState(queue); }, "window offset"); + queue.context[0x80] = 0; queue.userConfig[0x242] = 2; expectFailure([&] { AgcDriver::Graphics::DecodeState(queue); }, "polygon mode, depth bias or nonstandard rasterization"); queue.userConfig[0x242] = 4; From b352406f704743e79ab4a9aaf220592e2395779e Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sat, 10 Oct 2026 12:57:10 -0500 Subject: [PATCH 243/460] fix(agc): bound a strided buffer's rounded-up last record by its allocation RADV rounds a strided buffer descriptor's record count up, so stride * records can run up to one stride past the end of the buffer's allocation. The guest buffer region then no longer fits inside the allocation's host import and is copied on the CPU, whole, for every draw that binds it. When a strided descriptor ends less than one stride past the end of the readable registered allocation that contains its base, the range is bound up to the allocation's end. RetroArch's Dolphin core binds a 48 MiB stream buffer with stride 20 (records 0x266667, 12 bytes past the allocation) on every draw. With this change the buffer is bound in place from its import. AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- .../Graphics/include/GuestBufferMemory.hpp | 1 + .../Graphics/src/GuestBufferMemory.cpp | 7 ++++ .../Graphics/src/ShaderResources.cpp | 11 +++++- .../libSceAgcDriver/tests/BdaResources.cpp | 37 +++++++++++++++++++ 4 files changed, 55 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp index f4585fcd6..f180e1e9b 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp @@ -64,6 +64,7 @@ bool HostImportCovers(const Context& context, std::uint64_t address, std::size_t // Whether a readable registered allocation contains [address, address + bytes) right now (one // registry lease): a storage image whose memory was freed or re-registered has nothing to store to. bool RegisteredReadableCovers(std::uint64_t address, std::size_t bytes); +std::uint64_t RegisteredReadableEnd(std::uint64_t address); // A persistent device copy of one registered range of the main guest image (which cannot be host // imported); see GuestBufferMemory.cpp. diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp index f5ef0b995..1f6091d71 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp @@ -1718,6 +1718,13 @@ bool RegisteredReadableCovers(std::uint64_t address, std::size_t bytes) { return containingRange(lease, address, address + bytes) != nullptr; } +std::uint64_t RegisteredReadableEnd(std::uint64_t address) { + if (address == std::numeric_limits::max()) return 0; + const auto lease = GuestAllocations::GuestAllocationsAcquire_nid_postfix(); + const auto* range = containingRange(lease, address, address + 1); + return range != nullptr ? range->address + range->bytes : 0; +} + bool HostImportCovers(const Context& context, std::uint64_t address, std::size_t bytes) { if (context.hostImportAlignment == 0 || bytes == 0 || bytes > std::numeric_limits::max() - address) return false; auto& state = Imports(); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index f5164d688..9ac9ff9bd 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -2673,7 +2673,16 @@ std::size_t ShaderResources::addGuestBuffer(std::span words } Require(byteSize <= context.limits.maxStorageBufferRange, "shader buffer exceeds descriptor range limit"); Require(byteSize <= std::numeric_limits::max(), "shader buffer size exceeds host address space"); - const auto size = static_cast(byteSize); + auto size = static_cast(byteSize); + if (const std::uint64_t stride = descriptor.Stride(); stride != 0) { + constexpr std::uint64_t GuestPageBytes = 0x4000; + const auto end = address + size; + const auto overhang = end % GuestPageBytes; + if (overhang != 0 && overhang < stride) { + const auto allocationEnd = RegisteredReadableEnd(address); + if (allocationEnd > address && allocationEnd < end && end - allocationEnd < stride) size = static_cast(allocationEnd - address); + } + } Require(target == nullptr || !overlap(address, size, target->address, target->bytes), "shader buffer aliases the render target"); // APS5_ALL_BUFFERS_WRITTEN=1: every element is noted as written, as before bufferWritten existed. static const bool allWritten = std::getenv("APS5_ALL_BUFFERS_WRITTEN") != nullptr; diff --git a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp index 037991ce5..68917d1de 100644 --- a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp @@ -569,6 +569,42 @@ void gpuMappingTests(const Context& context) { ::operator delete(gpu, std::align_val_t{bytes}); } +void stridedOverhangTests(const Context& context, const BdaTestAccess& access) { + constexpr std::size_t allocation = 0x8000; + constexpr std::uint32_t stride = 20; + constexpr std::uint32_t records = (allocation + stride - 1) / stride; + auto* block = static_cast(::operator new(allocation * 2, std::align_val_t{allocation})); + std::memset(block, 0x5a, allocation * 2); + { + GuestAllocations::Mutation mutation; + mutation.Add(block, allocation, true, false, true); + } + const auto address = reinterpret_cast(block); + ShaderRecompiler::RecompileResult shader; + auto buffer = binding(Role::GuestBuffers, 6); + buffer.guestDescriptor = {static_cast(address), (static_cast(address >> 32) & 0xffffu) | (stride << 16u), records, 0x31000000u}; + buffer.bufferWritten = {false}; + shader.bindings = {buffer}; + CompiledShader compiled{ShaderRecompiler::ShaderStage::Compute, &shader, 0}; + { + ShaderResources resources(context, compiled); + Require(access.descriptor(6).range == allocation, "a strided buffer's rounded-up last record was bound past its allocation"); + resources.WriteBack(); + } + buffer.guestDescriptor[2] = records - 1; + shader.bindings = {buffer}; + { + ShaderResources resources(context, compiled); + Require(access.descriptor(6).range == static_cast(records - 1) * stride, "a strided buffer inside its allocation was shortened"); + resources.WriteBack(); + } + { + GuestAllocations::Mutation mutation; + mutation.Remove(block); + } + ::operator delete(block, std::align_val_t{allocation}); +} + void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { alignas(64) std::array guest{}; guest[0] = 123; @@ -731,6 +767,7 @@ void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { importWatchTests(); importedFreshTests(context); gpuMappingTests(context); + stridedOverhangTests(context, access); Require(AddressCopyOverflow({{0x1000, 0x3000, 0x2000, "uncommitted pages"}}, 0x2000).empty(), "copies within the limit were refused"); const auto copies = AddressCopyOverflow({{0x1000, 0x2000, 0x1000, "not mirrored"}, {0x10000, 0x30000, 0x18000, "uncommitted pages"}}, 0x2000); Require(!copies.empty() && copies.find("0x10000+0x20000 (0.1 MiB committed, uncommitted pages)") < copies.find("0x1000+0x1000"), "the copy limit does not name the largest copy first"); From dc48c610509a1b45bc153f8cd7c52f2c059c4817 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sat, 10 Oct 2026 17:59:31 -0500 Subject: [PATCH 244/460] test(agc): a strided buffer's rounded-up last record stays within its own allocation When another readable allocation starts right after the buffer's allocation, the bound range still ends at the first allocation: the bytes past it are outside the buffer the descriptor's records describe, and binding into the neighbour would send the range back to a copy on every draw. AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- .../libs/prx/libSceAgcDriver/tests/BdaResources.cpp | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp index 68917d1de..fa8240bd6 100644 --- a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp @@ -591,6 +591,19 @@ void stridedOverhangTests(const Context& context, const BdaTestAccess& access) { Require(access.descriptor(6).range == allocation, "a strided buffer's rounded-up last record was bound past its allocation"); resources.WriteBack(); } + { + GuestAllocations::Mutation mutation; + mutation.Add(block + allocation, allocation, true, false, true); + } + { + ShaderResources resources(context, compiled); + Require(access.descriptor(6).range == allocation, "a strided buffer's rounded-up last record was bound into the next allocation"); + resources.WriteBack(); + } + { + GuestAllocations::Mutation mutation; + mutation.Remove(block + allocation); + } buffer.guestDescriptor[2] = records - 1; shader.bindings = {buffer}; { From 1e1554c768225ec4b6e928c59a5c23d7165d0fdc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 01:37:01 +0200 Subject: [PATCH 245/460] docs(technical-debt): record the strided buffer overhang clamp --- docs/dev/TechnicalDebt.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index b2cbe8e0b..9cfc29192 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -124,6 +124,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceAgcGetGsOversubscription](../../core/libs/prx/libSceAgc/Shader/src/GsOversubscription.cpp) (libSceAgc) - signature and occupancy formula (base and expanded vertex/export capacities, budget shift, `GE_PC_ALLOC` / `SPI_SHADER_PGM_RSRC4_GS` encoding) follow KytyPS5 only, not confirmed on hardware or a title; a negative or NaN factor that gives a negative target, a missing GS context register or a zero `GE_MAX_OUTPUT_PER_SUBGROUP` throws - [Vertex entry merged wave info](../../core/shader/recompiler/Translation/src/InstructionTranslator.cpp) (recompiler) - `s3` of every vertex-path shader is set to `waveSize | waveSize << 8` (vertex and primitive count of a full wave, wave index 0). Layout taken from the prologue Mesa's ACO emits for NGG vertex shaders (`s_bfe_u64 exec` from `s3`), not measured on hardware. The driver gives every graphics program user data from `s8`, so the condition does not tell NGG from legacy vertex shaders: they just never read `s3`. A draw whose last wave is partial still reports a full wave. - [s_barrier in vertex-path programs](../../core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp) (recompiler) - emits no instruction when the program has no LDS or GDS access and no memory write or atomic, so no wave of the subgroup can observe another; the `GS_ALLOC_REQ` such a barrier orders on NGG passthrough programs is already dropped on the vertex path. The Vertex execution model has no workgroup barrier. Programs with LDS, GDS or memory writes throw. Not measured on hardware +- [Strided buffer overhang](../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp) (libSceAgcDriver) - a strided buffer descriptor that ends less than one stride past the end of the readable registered allocation holding its base is bound only up to that allocation's end, even when another readable allocation starts right there. This assumes the bytes past the end are the rounded-up `num_records` of the last record (RADV rounds it up), which no fetch inside the buffer reads; a guest that really keeps a vertex split across two adjacent allocations would read zeros for the cut tail. Taken from the RetroArch Dolphin core case in #2912, not measured on hardware. - [Zero-stride vertex fetch range](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - for a stride-0 V# with OOB_SELECT 2 the whole fetch reads memory unless NUM_RECORDS is 0; with OOB_SELECT 3 it reads zero when NUM_RECORDS is 0 and throws when the fetch only partly fits. Rules taken from the GFX10 OOB_SELECT table in Mesa [ac_descriptors.c](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/common/ac_descriptors.c), not measured on hardware. Whether OOB_SELECT 3 checks each dword of a multi-dword fetch separately is unknown. OOB_SELECT 0 and 1 keep the structured record checks. - [sceAgcCbCondWrite](../../core/libs/prx/libSceAgc/Cb/src/Branch.cpp) (libSceAgc) - argument order and the 9-dword COND_WRITE layout follow the clean-room [OpenAGC](https://github.com/OpenAGC/OpenAGC/blob/main/src/game_compat.c) and [sharpemu](https://github.com/sharpemu/sharpemu/blob/main/src/SharpEmu.Libs/Agc/AgcExports.CommandPackets.cs), which agree on them; SharpProspero lists another order. The write space is accepted only as 1 (memory), the one value both encode the same way, and other values throw. - [sceAgcGetIsTrinityMode](../../core/libs/prx/libSceAgc/Misc/src/Platform.cpp) (libSceAgc) - signature from the only PPSA26344 call (a pointer to a one-byte flag, result ignored); a null pointer throws because its error code is unknown From 718eb9cc3e3fdf19c1eae6b970bc9cea543980d7 Mon Sep 17 00:00:00 2001 From: Zayd Zaari <110066668+zaydzaari@users.noreply.github.com> Date: Sat, 10 Oct 2026 12:12:34 +0000 Subject: [PATCH 246/460] fix(libkernel): return EBUSY when destroying locked rwlock --- .../prx/libkernel/Pthread/include/Pthread.hpp | 1 + .../libs/prx/libkernel/Pthread/src/Rwlock.cpp | 16 ++- core/libs/tests/GuestPthreadRwlock.cpp | 135 ++++++++++++++++++ 3 files changed, 149 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libkernel/Pthread/include/Pthread.hpp b/core/libs/prx/libkernel/Pthread/include/Pthread.hpp index 6e69de708..9e6b73c73 100644 --- a/core/libs/prx/libkernel/Pthread/include/Pthread.hpp +++ b/core/libs/prx/libkernel/Pthread/include/Pthread.hpp @@ -55,6 +55,7 @@ struct PthreadRwlockPrivate { } std::shared_timed_mutex _lock; std::atomic _writer; + std::atomic _readers{0}; }; struct PthreadCondattrPrivate { diff --git a/core/libs/prx/libkernel/Pthread/src/Rwlock.cpp b/core/libs/prx/libkernel/Pthread/src/Rwlock.cpp index 6d8850f0b..7689b9c76 100644 --- a/core/libs/prx/libkernel/Pthread/src/Rwlock.cpp +++ b/core/libs/prx/libkernel/Pthread/src/Rwlock.cpp @@ -24,7 +24,11 @@ static bool OwnsWrite(const PthreadRwlockPrivate* lock) { extern "C" { int APS5_VABI scePthreadRwlockDestroy(PthreadRwlock* rwlock) { - delete RequireRwlock(rwlock, __func__); + auto* lock = RequireRwlock(rwlock, __func__); + if (lock->_writer.load(std::memory_order_acquire) != std::thread::id{} || + lock->_readers.load(std::memory_order_acquire) != 0) + return SCE_KERNEL_ERROR_EBUSY; + delete lock; *rwlock = nullptr; return SCE_OK; } @@ -47,20 +51,25 @@ int APS5_VABI scePthreadRwlockRdlock(PthreadRwlock* rwlock) { #else lock->_lock.lock_shared(); #endif + lock->_readers.fetch_add(1, std::memory_order_release); return SCE_OK; } int APS5_VABI scePthreadRwlockTryrdlock(PthreadRwlock* rwlock) { auto* lock = RequireRwlock(rwlock, __func__); if (OwnsWrite(lock)) return SCE_KERNEL_ERROR_EBUSY; - return lock->_lock.try_lock_shared() ? SCE_OK : SCE_KERNEL_ERROR_EBUSY; + if (!lock->_lock.try_lock_shared()) return SCE_KERNEL_ERROR_EBUSY; + lock->_readers.fetch_add(1, std::memory_order_release); + return SCE_OK; } int APS5_VABI scePthreadRwlockTimedrdlock(PthreadRwlock* rwlock, KernelUseconds usec) { auto* lock = RequireRwlock(rwlock, __func__); if (OwnsWrite(lock)) return SCE_KERNEL_ERROR_EDEADLK; const bool locked = TimedWait::AcquireUntil(TimedWait::DeadlineNanos(usec), [&] { return lock->_lock.try_lock_shared(); }, [&](std::uint64_t micros) { return lock->_lock.try_lock_shared_for(std::chrono::microseconds(micros)); }); - return locked ? SCE_OK : SCE_KERNEL_ERROR_ETIMEDOUT; + if (!locked) return SCE_KERNEL_ERROR_ETIMEDOUT; + lock->_readers.fetch_add(1, std::memory_order_release); + return SCE_OK; } int APS5_VABI scePthreadRwlockWrlock(PthreadRwlock* rwlock) { @@ -97,6 +106,7 @@ int APS5_VABI scePthreadRwlockUnlock(PthreadRwlock* rwlock) { lock->_writer.store(std::thread::id{}, std::memory_order_release); lock->_lock.unlock(); } else { + lock->_readers.fetch_sub(1, std::memory_order_release); lock->_lock.unlock_shared(); } return SCE_OK; diff --git a/core/libs/tests/GuestPthreadRwlock.cpp b/core/libs/tests/GuestPthreadRwlock.cpp index d19319b86..ffae6559a 100644 --- a/core/libs/tests/GuestPthreadRwlock.cpp +++ b/core/libs/tests/GuestPthreadRwlock.cpp @@ -21,11 +21,14 @@ int APS5_VABI pthread_rwlock_tryrdlock_nid_postfix(PthreadRwlock* rwlock); int APS5_VABI pthread_rwlock_trywrlock_nid_postfix(PthreadRwlock* rwlock); int APS5_VABI pthread_rwlock_timedrdlock_nid_postfix(PthreadRwlock* rwlock, const KernelTimespec* abstime); int APS5_VABI pthread_rwlock_timedwrlock_nid_postfix(PthreadRwlock* rwlock, const KernelTimespec* abstime); +int APS5_VABI scePthreadRwlockInit(PthreadRwlock* rwlock, const PthreadRwlockattr* attr, const char* name); +int APS5_VABI scePthreadRwlockDestroy(PthreadRwlock* rwlock); } using TimedLock = int (APS5_VABI *)(PthreadRwlock*, const KernelTimespec*); static constexpr int SCE_OK = 0; +static constexpr int SCE_KERNEL_ERROR_EBUSY = 0x80020010; static constexpr int GUEST_EDEADLK = 11; static constexpr int GUEST_EBUSY = 16; static constexpr int GUEST_EINVAL = 22; @@ -113,6 +116,80 @@ static void CheckConcurrentFirstWrlockExcludes() { Require(overlaps == 0); } +struct HandoffState { + PthreadRwlock rwlock = nullptr; + std::atomic firstHeld{false}; + std::atomic firstRelease{false}; + std::atomic secondAcquired{false}; + std::atomic secondRelease{false}; + Pthread firstThread = nullptr; + Pthread secondThread = nullptr; +}; + +static void* APS5_VABI HandoffFirst(void* arg) { + auto& state = *static_cast(arg); + Require(pthread_rwlock_wrlock_nid_postfix(&state.rwlock) == 0); + state.firstHeld.store(true); + while (!state.firstRelease.load()) std::this_thread::yield(); + Require(pthread_rwlock_unlock_nid_postfix(&state.rwlock) == 0); + return nullptr; +} + +static void* APS5_VABI HandoffSecond(void* arg) { + auto& state = *static_cast(arg); + while (!state.firstHeld.load()) std::this_thread::yield(); + Require(pthread_rwlock_wrlock_nid_postfix(&state.rwlock) == 0); + state.secondAcquired.store(true); + while (!state.secondRelease.load()) std::this_thread::yield(); + Require(pthread_rwlock_unlock_nid_postfix(&state.rwlock) == 0); + return nullptr; +} + +static void CheckWriterHandoff() { + HandoffState state; + Require(scePthreadCreate(&state.firstThread, nullptr, HandoffFirst, &state, nullptr) == SCE_OK); + Require(scePthreadCreate(&state.secondThread, nullptr, HandoffSecond, &state, nullptr) == SCE_OK); + while (!state.firstHeld.load()) std::this_thread::yield(); + Require(pthread_rwlock_destroy_nid_postfix(&state.rwlock) == GUEST_EBUSY); + Require(scePthreadRwlockDestroy(&state.rwlock) == SCE_KERNEL_ERROR_EBUSY); + Require(state.rwlock != nullptr); + state.firstRelease.store(true); + Require(scePthreadJoin(state.firstThread, nullptr) == SCE_OK); + while (!state.secondAcquired.load()) std::this_thread::yield(); + Require(pthread_rwlock_destroy_nid_postfix(&state.rwlock) == GUEST_EBUSY); + Require(scePthreadRwlockDestroy(&state.rwlock) == SCE_KERNEL_ERROR_EBUSY); + Require(state.rwlock != nullptr); + state.secondRelease.store(true); + Require(scePthreadJoin(state.secondThread, nullptr) == SCE_OK); + Require(pthread_rwlock_destroy_nid_postfix(&state.rwlock) == 0); + Require(state.rwlock == nullptr); +} + +static void CheckFailedLockAttempts() { + PthreadRwlock rwlock = nullptr; + Require(pthread_rwlock_wrlock_nid_postfix(&rwlock) == 0); + const KernelTimespec timeout = After(10); + Require(pthread_rwlock_timedrdlock_nid_postfix(&rwlock, &timeout) == GUEST_EDEADLK); + Require(pthread_rwlock_tryrdlock_nid_postfix(&rwlock) == GUEST_EBUSY); + Require(pthread_rwlock_destroy_nid_postfix(&rwlock) == GUEST_EBUSY); + Require(scePthreadRwlockDestroy(&rwlock) == SCE_KERNEL_ERROR_EBUSY); + Require(pthread_rwlock_unlock_nid_postfix(&rwlock) == 0); + Require(pthread_rwlock_destroy_nid_postfix(&rwlock) == 0); + Require(rwlock == nullptr); + + Holder reader{&rwlock, false}; + Start(reader); + const KernelTimespec past = After(-1000); + Require(pthread_rwlock_timedwrlock_nid_postfix(&rwlock, &past) == GUEST_ETIMEDOUT); + Require(pthread_rwlock_trywrlock_nid_postfix(&rwlock) == GUEST_EBUSY); + Require(pthread_rwlock_destroy_nid_postfix(&rwlock) == GUEST_EBUSY); + Require(scePthreadRwlockDestroy(&rwlock) == SCE_KERNEL_ERROR_EBUSY); + reader.release.store(true); + Require(scePthreadJoin(reader.thread, nullptr) == SCE_OK); + Require(pthread_rwlock_destroy_nid_postfix(&rwlock) == 0); + Require(rwlock == nullptr); +} + int main() { CheckConcurrentFirstWrlockExcludes(); const KernelTimespec invalid{0, NANOS_PER_SECOND}; @@ -154,6 +231,8 @@ int main() { Require(pthread_rwlock_timedrdlock_nid_postfix(&rwlock, &invalid) == 0); Require(pthread_rwlock_unlock_nid_postfix(&rwlock) == 0); Require(pthread_rwlock_trywrlock_nid_postfix(&rwlock) == GUEST_EBUSY); + Require(pthread_rwlock_destroy_nid_postfix(&rwlock) == GUEST_EBUSY); + Require(scePthreadRwlockDestroy(&rwlock) == SCE_KERNEL_ERROR_EBUSY); ExpectTimeout(pthread_rwlock_timedwrlock_nid_postfix, &rwlock); deadline = After(5000); reader.release.store(true); @@ -165,6 +244,8 @@ int main() { Start(writer); Require(pthread_rwlock_tryrdlock_nid_postfix(&rwlock) == GUEST_EBUSY); Require(pthread_rwlock_trywrlock_nid_postfix(&rwlock) == GUEST_EBUSY); + Require(pthread_rwlock_destroy_nid_postfix(&rwlock) == GUEST_EBUSY); + Require(scePthreadRwlockDestroy(&rwlock) == SCE_KERNEL_ERROR_EBUSY); ExpectTimeout(pthread_rwlock_timedrdlock_nid_postfix, &rwlock); ExpectTimeout(pthread_rwlock_timedwrlock_nid_postfix, &rwlock); deadline = After(5000); @@ -178,4 +259,58 @@ int main() { catch (const std::runtime_error&) { rejected = true; } Require(rejected); Require(pthread_rwlock_destroy_nid_postfix(&rwlock) == 0); + Require(rwlock == nullptr); + + PthreadRwlock unheld = nullptr; + Require(scePthreadRwlockInit(&unheld, nullptr, nullptr) == SCE_OK); + Require(unheld != nullptr); + Require(pthread_rwlock_destroy_nid_postfix(&unheld) == 0); + Require(unheld == nullptr); + + PthreadRwlock rd = nullptr; + Require(pthread_rwlock_rdlock_nid_postfix(&rd) == 0); + Require(pthread_rwlock_destroy_nid_postfix(&rd) == GUEST_EBUSY); + Require(scePthreadRwlockDestroy(&rd) == SCE_KERNEL_ERROR_EBUSY); + Require(rd != nullptr); + Require(pthread_rwlock_unlock_nid_postfix(&rd) == 0); + Require(pthread_rwlock_tryrdlock_nid_postfix(&rd) == 0); + Require(pthread_rwlock_unlock_nid_postfix(&rd) == 0); + Require(pthread_rwlock_destroy_nid_postfix(&rd) == 0); + Require(rd == nullptr); + + PthreadRwlock wr = nullptr; + Require(pthread_rwlock_wrlock_nid_postfix(&wr) == 0); + Require(pthread_rwlock_destroy_nid_postfix(&wr) == GUEST_EBUSY); + Require(scePthreadRwlockDestroy(&wr) == SCE_KERNEL_ERROR_EBUSY); + Require(wr != nullptr); + Require(pthread_rwlock_unlock_nid_postfix(&wr) == 0); + Require(pthread_rwlock_trywrlock_nid_postfix(&wr) == 0); + Require(pthread_rwlock_unlock_nid_postfix(&wr) == 0); + Require(pthread_rwlock_destroy_nid_postfix(&wr) == 0); + Require(wr == nullptr); + + PthreadRwlock remoteRd = nullptr; + Holder bgReader{&remoteRd, false}; + Start(bgReader); + Require(pthread_rwlock_destroy_nid_postfix(&remoteRd) == GUEST_EBUSY); + Require(scePthreadRwlockDestroy(&remoteRd) == SCE_KERNEL_ERROR_EBUSY); + Require(remoteRd != nullptr); + bgReader.release.store(true); + Require(scePthreadJoin(bgReader.thread, nullptr) == SCE_OK); + Require(pthread_rwlock_destroy_nid_postfix(&remoteRd) == 0); + Require(remoteRd == nullptr); + + PthreadRwlock remoteWr = nullptr; + Holder bgWriter{&remoteWr, true}; + Start(bgWriter); + Require(pthread_rwlock_destroy_nid_postfix(&remoteWr) == GUEST_EBUSY); + Require(scePthreadRwlockDestroy(&remoteWr) == SCE_KERNEL_ERROR_EBUSY); + Require(remoteWr != nullptr); + bgWriter.release.store(true); + Require(scePthreadJoin(bgWriter.thread, nullptr) == SCE_OK); + Require(pthread_rwlock_destroy_nid_postfix(&remoteWr) == 0); + Require(remoteWr == nullptr); + + CheckWriterHandoff(); + CheckFailedLockAttempts(); } From c9d1dbe6eda1d0841ba0223371d694cf6cda629e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 01:37:32 +0200 Subject: [PATCH 247/460] docs(debt): record rwlock destroy EBUSY choice --- docs/dev/TechnicalDebt.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index b2cbe8e0b..3f8915130 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -147,6 +147,7 @@ Throughout the project, every function at every stage either **does exactly what - [Memory protection bits 0x100 and 0x200](../../core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp) (libkernel) - taken as audio coprocessor (ACP) read and write, as in the [KytyPS5](https://github.com/KytyPS5/KytyPS5) and [etaHEN](https://github.com/etaHEN/etaHEN) headers; the coprocessor work runs on the host CPU, so they grant host read and read/write access like the GPU bits - [sceKernelSyncOnAddressWait](../../core/libs/prx/libkernel/SyncOnAddress/src/SyncOnAddress.cpp) (libkernel) - the only known caller passes a null timeout and a name string as the fourth argument; the timeout is assumed to point to microseconds like the other kernel waits, and the name is ignored - [sceKernelSyncOnAddressWait8 / sceKernelSyncOnAddressWait16 / sceKernelSyncOnAddressWait32 / sceKernelSyncOnAddressWait64](../../core/libs/prx/libkernel/SyncOnAddress/src/SyncOnAddress.cpp) (libkernel) - libc's `__std_atomic_wait_direct_8/16/32/64` pass a microsecond timeout or null and ignore the result; the by-value comparand, return values and the 2- and 8-byte alignment of Wait16 and Wait64 are assumed +- [scePthreadRwlockDestroy, pthread_rwlock_destroy](../../core/libs/prx/libkernel/Pthread/src/Rwlock.cpp) (libkernel) - a rwlock held for reading or writing returns `EBUSY` and keeps its handle, as [KytyPS5](https://github.com/KytyPS5/KytyPS5) does; FreeBSD's [libthr](https://github.com/freebsd/freebsd-src/blob/main/lib/libthr/thread/thr_rwlock.c) and shadPS4 free it without checking, and the console's answer is unverified. Destroying a rwlock while another thread is acquiring it is not guarded - [scePthreadCondWait / scePthreadCondTimedwait](../../core/libs/prx/libkernel/Pthread/src/Cond.cpp) (libkernel) - a recursive mutex locked several times is released completely for the wait and relocked to the same depth afterwards, as FreeBSD libthr does (`_mutex_cv_unlock`, `_mutex_cv_lock`); unverified on hardware - [sceKernelAioSubmitReadCommandsMultiple / sceKernelAioSubmitWriteCommandsMultiple / sceKernelAioWaitRequests / sceKernelAioPollRequests / sceKernelAioCancelRequest / sceKernelAioCancelRequests / sceKernelAioDeleteRequests](../../core/libs/prx/libkernel/Aio/src/Aio.cpp) (libkernel) - behaviour from shadPS4 (one id per request for the Multiple submits, cancel turns any request aborted and id 0 reports processing, wait mode 2 returns once one request completed, poll copies every state without waiting); the 128 requests or ids per batch limit is from SharpProspero, with an unknown error, so larger batches throw; wait modes other than 1 (and) and 2 (or) throw; null pointers, negative counts and invalid ids return the codes the single-request siblings use, checked before any state is written - [sceKernelGetAvailableCpumask](../../core/libs/prx/libkernel/Pthread/src/Thread.cpp) (libkernel) - returns the default thread affinity, 0x1FFF (CPUs 0 to 12), as KytyPS5 does. Titles pin threads within it (PPSA02664 passes 0x1FFB, Hades II PPSA36082 0x3 and 0x3F); whether CPU 13 is available to a title is not confirmed From b89b7f953cf383cf3fd41fb2aa432cb5c8892f62 Mon Sep 17 00:00:00 2001 From: Chad Date: Sat, 10 Oct 2026 13:02:16 -0500 Subject: [PATCH 248/460] fix(agc): expand comp-to-single DCC keys in a DCC decompress pass Key 0x10 (GFX9_DCC_CLEAR_SINGLE in Mesa) marks a 256-byte block whose texels all equal its first texel. A CB DCC decompress pass over uniform comp-to-single keys now writes that texel over its block and stores uncompressed keys; a fast-clear eliminate over them throws. --- .../Graphics/include/DccMetadata.hpp | 2 +- .../Graphics/src/DccMetadata.cpp | 2 ++ .../prx/libSceAgcDriver/Graphics/src/Draw.cpp | 25 +++++++++++++++++++ .../prx/libSceAgcDriver/tests/Graphics.cpp | 22 ++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 5 files changed, 51 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/DccMetadata.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/DccMetadata.hpp index 849923f41..87256bd95 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/DccMetadata.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/DccMetadata.hpp @@ -19,7 +19,7 @@ struct Context; // uncompressed here, so the keys that matter are the fast-clear codes a title writes into the metadata // (the surface then reads as a constant whatever its texels hold) and "uncompressed", which the driver // stores after it writes a surface so later reads see the texels. -enum class DccKeys { Uncompressed, Clear0000, Clear0001, Clear1110, Clear1111, ClearRegister, Mixed, Unreadable }; +enum class DccKeys { Uncompressed, Clear0000, Clear0001, Clear1110, Clear1111, ClearRegister, ClearSingle, Mixed, Unreadable }; const char* DccKeysName(DccKeys keys); std::size_t DccKeyBytes(std::uint64_t surfaceBytes); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/DccMetadata.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/DccMetadata.cpp index 69b14ec8f..5b9a4b6b7 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/DccMetadata.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/DccMetadata.cpp @@ -51,6 +51,7 @@ DccKeys ByteKeys(std::uint8_t key) { case 0x80: return DccKeys::Clear1110; case 0xc0: return DccKeys::Clear1111; case 0x20: return DccKeys::ClearRegister; + case 0x10: return DccKeys::ClearSingle; case 0xff: return DccKeys::Uncompressed; default: return DccKeys::Mixed; } @@ -384,6 +385,7 @@ const char* DccKeysName(DccKeys keys) { case DccKeys::Clear1110: return "1110"; case DccKeys::Clear1111: return "1111"; case DccKeys::ClearRegister: return "register"; + case DccKeys::ClearSingle: return "single"; case DccKeys::Mixed: return "mixed"; case DccKeys::Unreadable: return "unreadable"; } diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp index 67126cb91..89ff3385f 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp @@ -132,6 +132,25 @@ void storeClearTexels(const Context& context, const ColorTarget& color, const st MarkDccUncompressed(context, color.dccAddress, color.bytes, colorKeyCount(color, color.bytes)); } +void storeSingleTexels(const Context& context, const ColorTarget& color) { + Require(color.tileMode != ColorTileMode::Linear, "comp-to-single DCC keys of a linear color target"); + const ColorTargetLayout surface(color.extent.width, color.extent.height, color.tileMode, color.elementBytes, color.pipeBankXor); + Require(surface.Bytes() == color.bytes, "comp-to-single DCC keys over a color target whose layout differs from its surface"); + StorageTexture::FlushPending(color.address, color.bytes, nullptr, "comp-to-single expansion"); + std::vector texels(color.bytes); + GuestMemory::Read(color.address, texels, surface.Alignment()); + constexpr std::size_t BlockBytes = 256; + for (std::uint32_t y = 0; y < color.extent.height; ++y) { + for (std::uint32_t x = 0; x < color.extent.width; ++x) { + const auto offset = surface.Offset(x, y); + const auto first = offset / BlockBytes * BlockBytes; + if (offset != first) std::memcpy(texels.data() + offset, texels.data() + first, color.elementBytes); + } + } + GuestMemory::Write(color.address, texels, surface.Alignment()); + MarkDccUncompressed(context, color.dccAddress, color.bytes, colorKeyCount(color, color.bytes)); +} + void materializeRegisterClear(const Context& context, const ColorTarget& color, StorageTexture& resident) { if (color.dccAddress == 0 || resident.Descriptor().dccAddress != color.dccAddress) return; if (ProvedCurrentDccKeys(color.dccAddress, color.bytes, resident.TargetKeyProof()) != DccKeys::ClearRegister) return; @@ -2459,6 +2478,12 @@ void RunColorMetadataPass(const Context& context, const ColorMetadataPass& pass) if (color.dccAddress == 0) continue; auto keys = CurrentDccKeys(color.dccAddress, color.bytes); if (keys == DccKeys::Uncompressed) continue; + if (keys == DccKeys::ClearSingle) { + Require(pass.mode == ColorMetadataPass::Mode::DccDecompress, "CB fast-clear eliminate over comp-to-single DCC keys (whether it expands them is not modeled)"); + storeSingleTexels(context, color); + if (const auto resident = metadataPassResident(context, color)) resident->Refresh(); + continue; + } Require(IsDccClear(keys), std::string("CB metadata pass over DCC keys that are ") + DccKeysName(keys) + " (per-block metadata is not modeled)"); const auto texel = clearTexel(color, keys); const auto resident = metadataPassResident(context, color); diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index 70d0b899b..cf7a795c0 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -1282,6 +1282,28 @@ void metadataPassTests() { for (std::size_t offset = 0; offset < tiledBytes; offset += 4) filled = filled && std::memcmp(tiledTexels + offset, &texel, 4) == 0; Require(filled && stored == expected && std::all_of(tiledKeys, tiledKeys + expected, [](std::uint8_t value) { return value == 0xff; }), std::string("a ") + code + " fast clear eliminate of a 128x128 SW_64KB_R_X target with " + alignment + " DCC stored uncompressed keys over " + std::to_string(stored) + " of its " + std::to_string(keyCount) + " DCC key bytes, expected " + std::to_string(expected)); } + std::vector original(tiledBytes / 4); + for (std::size_t index = 0; index < original.size(); ++index) original[index] = 0x01000000u + static_cast(index); + const auto resetSingle = [&] { + std::memcpy(tiledTexels, original.data(), tiledBytes); + std::memset(tiledKeys, 0x10, keyCount); + }; + resetSingle(); + expectFailure([&] { AgcDriver::Graphics::RunColorMetadataPass(context, *tiledPass); }, "fast-clear eliminate over comp-to-single"); + Require(std::memcmp(tiledTexels, original.data(), tiledBytes) == 0 && std::all_of(tiledKeys, tiledKeys + keyCount, [](std::uint8_t value) { return value == 0x10; }), "a refused fast-clear eliminate over comp-to-single keys changed the " + alignment + " target"); + resetSingle(); + AgcDriver::Graphics::RunColorMetadataPass(context, {AgcDriver::Graphics::ColorMetadataPass::Mode::DccDecompress, tiledPass->targets}); + const AgcDriver::Graphics::ColorTargetLayout layout(tiledSide, tiledSide, AgcDriver::Graphics::ColorTileMode::RenderTarget, 4); + bool expanded = true; + for (std::uint32_t y = 0; y < tiledSide && expanded; ++y) { + for (std::uint32_t x = 0; x < tiledSide && expanded; ++x) { + std::uint32_t texel = 0; + std::memcpy(&texel, tiledTexels + layout.Offset(x, y), 4); + expanded = texel == original[layout.Offset(x & ~7u, y & ~7u) / 4]; + } + } + const auto stored = static_cast(std::count(tiledKeys, tiledKeys + keyCount, std::uint8_t{0xff})); + Require(expanded && stored == expected, "a DCC decompress of comp-to-single keys over a 128x128 SW_64KB_R_X target with " + alignment + " DCC did not give every 8x8 block its first texel (" + std::to_string(stored) + " of " + std::to_string(keyCount) + " keys uncompressed, expected " + std::to_string(expected) + ")"); } } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index b2cbe8e0b..069a5c1f8 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -433,6 +433,7 @@ Throughout the project, every function at every stage either **does exactly what - [Storage image write-back](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) stores the GPU results over a CPU-stamped range whose guest bytes equal the copy taken when the image's layers got their generation (images up to 1 MiB whose guest bytes were settled at that point, with no unit shadow, pending image results or recorded GPU store over them; other images keep the previous rule: the CPU's bytes win). A CPU store that rewrites the baseline value after the GPU changed it is not told apart from no store, and the GPU result wins there. The 1 MiB limit is a choice, not a console value. - [DCC key stores](../../core/libs/prx/libSceAgcDriver/Graphics/src/DccMetadata.cpp) cover the console's DCC extent only for pipe-aligned DCC of single-level, single-layer 4-byte SW_64KB_R_X surfaces; other surfaces, and every texture and color target key read, cover one byte per 256 bytes. - [DCC key fills over other images](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) - a uniform fill at a live image's `dccAddress`, at least one key per 256 bytes long, is a key fill of that image even when other live images overlap it, but only while it is no longer than the image's key extent (`DccKeyCount`, the pipe-aligned count); a longer one is classified by its overlaps as before. The bound is inferred, not measured on a title: a longer fill also writes the bytes past the keys, which the overlapping image may own. Whether the title's bloom-target key fills stay within the extent was not checked (`APS5_TRACE_DCC_KEYS` prints the length of the fills classified as keys, not of the refused ones). +- [Comp-to-single DCC keys](../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver) - key 0x10 (`GFX9_DCC_CLEAR_SINGLE` in Mesa `ac_descriptors.h`) is expanded only by a CB DCC decompress pass over uniform keys of a non-linear target: every texel takes the element at the start of its 256-byte block, the top-left texel of the block where radv's `build_clear_dcc_comp_to_single_shader` stores the clear color. A fast-clear eliminate over such keys and mixed keys throw; textures and color targets that read them use their stored texels. Not verified on hardware - [8_SRGB color targets](../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver) on a device that cannot attach and blend R8_SRGB render into an RGBA8_SRGB copy of the target, so the sRGB encode, decode and blend rounding are the device's; the console's are not measured. - [8_SRGB color targets](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) on such a device throw unless they are single-mip 2D surfaces with a resident image; linear targets have none. - [COPY_DATA from the GPU clock](../../core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp) (libSceAgcDriver, `SRC_SEL` 9) writes the host steady clock in 10 ns ticks when the packet runs, the same counter as the `RELEASE_MEM` timestamp, not the console's GPU clock. The `COPY_DATA` cache policy fields are accepted and have no effect. From 9678b46d8b1773a3b78af3a5f2e8198f1d9abdcf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 01:38:32 +0200 Subject: [PATCH 249/460] docs(technical-debt): state the key range comp-to-single uniformity is judged over --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 069a5c1f8..f132fa5db 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -433,7 +433,7 @@ Throughout the project, every function at every stage either **does exactly what - [Storage image write-back](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) stores the GPU results over a CPU-stamped range whose guest bytes equal the copy taken when the image's layers got their generation (images up to 1 MiB whose guest bytes were settled at that point, with no unit shadow, pending image results or recorded GPU store over them; other images keep the previous rule: the CPU's bytes win). A CPU store that rewrites the baseline value after the GPU changed it is not told apart from no store, and the GPU result wins there. The 1 MiB limit is a choice, not a console value. - [DCC key stores](../../core/libs/prx/libSceAgcDriver/Graphics/src/DccMetadata.cpp) cover the console's DCC extent only for pipe-aligned DCC of single-level, single-layer 4-byte SW_64KB_R_X surfaces; other surfaces, and every texture and color target key read, cover one byte per 256 bytes. - [DCC key fills over other images](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) - a uniform fill at a live image's `dccAddress`, at least one key per 256 bytes long, is a key fill of that image even when other live images overlap it, but only while it is no longer than the image's key extent (`DccKeyCount`, the pipe-aligned count); a longer one is classified by its overlaps as before. The bound is inferred, not measured on a title: a longer fill also writes the bytes past the keys, which the overlapping image may own. Whether the title's bloom-target key fills stay within the extent was not checked (`APS5_TRACE_DCC_KEYS` prints the length of the fills classified as keys, not of the refused ones). -- [Comp-to-single DCC keys](../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver) - key 0x10 (`GFX9_DCC_CLEAR_SINGLE` in Mesa `ac_descriptors.h`) is expanded only by a CB DCC decompress pass over uniform keys of a non-linear target: every texel takes the element at the start of its 256-byte block, the top-left texel of the block where radv's `build_clear_dcc_comp_to_single_shader` stores the clear color. A fast-clear eliminate over such keys and mixed keys throw; textures and color targets that read them use their stored texels. Not verified on hardware +- [Comp-to-single DCC keys](../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver) - key 0x10 (`GFX9_DCC_CLEAR_SINGLE` in Mesa `ac_descriptors.h`) is expanded only by a CB DCC decompress pass over uniform keys of a non-linear target: every texel takes the element at the start of its 256-byte block, the top-left texel of the block where radv's `build_clear_dcc_comp_to_single_shader` stores the clear color. A fast-clear eliminate over such keys and mixed keys throw; textures and color targets that read them use their stored texels. "Uniform" is judged over the key bytes the driver reads, one per 256 surface bytes (see DCC key stores above), so a pipe-aligned surface whose keys differ past that range is expanded as if all were comp-to-single, as for the other clear codes. Not verified on hardware - [8_SRGB color targets](../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver) on a device that cannot attach and blend R8_SRGB render into an RGBA8_SRGB copy of the target, so the sRGB encode, decode and blend rounding are the device's; the console's are not measured. - [8_SRGB color targets](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) on such a device throw unless they are single-mip 2D surfaces with a resident image; linear targets have none. - [COPY_DATA from the GPU clock](../../core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp) (libSceAgcDriver, `SRC_SEL` 9) writes the host steady clock in 10 ns ticks when the packet runs, the same counter as the `RELEASE_MEM` timestamp, not the console's GPU clock. The `COPY_DATA` cache policy fields are accepted and have no effect. From 1d7f8bd3654eac69abca46ca3d6c67f08150c86b Mon Sep 17 00:00:00 2001 From: poonasor Date: Sat, 10 Oct 2026 19:16:57 -0400 Subject: [PATCH 250/460] fix(agc): keep a shader library part's modules until the part is destroyed Pipeline destroyed its shader modules right after LinkPipelineFromLibraries returned, while the pre-rasterization and fragment shader library parts built from them stay cached and are relinked later with VK_PIPELINE_CREATE_LINK_TIME_OPTIMIZATION_BIT_EXT. On AMD's Windows driver (RX 6700 XT, Adrenalin 25.10.45.05) that link faults inside amdvlk64.dll at varying offsets, reading freed heap memory, in 22 agc tests on a cold driver cache; all of them pass with APS5_NO_PIPELINE_LTO=1. Running the optimized link synchronously or without a pipeline cache did not help, keeping the modules alive did. When the optimized link is enabled, the library store takes over the modules of each newly built shader part and destroys them in ClearPipelineLibraries after the pending links have finished. Modules of a pipeline whose parts were all cached are destroyed as before. Co-Authored-By: Claude Opus 5.5 --- .../Graphics/include/PipelineLibrary.hpp | 2 +- .../libSceAgcDriver/Graphics/src/Pipeline.cpp | 2 +- .../Graphics/src/PipelineLibrary.cpp | 17 +++++++++++++++-- 3 files changed, 17 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/PipelineLibrary.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/PipelineLibrary.hpp index 62c7e9359..f480042a3 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/PipelineLibrary.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/PipelineLibrary.hpp @@ -31,7 +31,7 @@ struct OptimizedPipeline { void Release() noexcept; }; -VkPipeline LinkPipelineFromLibraries(const Context& context, const VkGraphicsPipelineCreateInfo& info, const VkPipelineRenderingCreateInfoKHR& rendering, const VkPipelineLayoutCreateInfo& layout, const PipelineLibraryKeys& keys, std::shared_ptr* optimized = nullptr); +VkPipeline LinkPipelineFromLibraries(const Context& context, const VkGraphicsPipelineCreateInfo& info, const VkPipelineRenderingCreateInfoKHR& rendering, const VkPipelineLayoutCreateInfo& layout, const PipelineLibraryKeys& keys, std::shared_ptr* optimized = nullptr, std::span modules = {}); void WaitForOptimizedPipelines(VkDevice device); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp index 2ab89e7d4..5bd89e287 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp @@ -369,7 +369,7 @@ Pipeline::Pipeline(const Context& context, const State& state, const VertexInput pipelineInfo.renderPass = renderPass; if (dynamicRendering) pipelineInfo.pNext = &rendering; timing.Mark("modules_and_state"); - if (libraries) pipeline = LinkPipelineFromLibraries(context, pipelineInfo, rendering, layoutInfo, keys, &optimized); + if (libraries) pipeline = LinkPipelineFromLibraries(context, pipelineInfo, rendering, layoutInfo, keys, &optimized, _modules); else Check(context.Function("vkCreateGraphicsPipelines")(context.device, context.pipelineCache, 1, &pipelineInfo, nullptr, &pipeline), "vkCreateGraphicsPipelines"); if (state.depthBiasPerFace) { raster.cullMode = VK_CULL_MODE_FRONT_BIT; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp index 5e142cbe4..263880966 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp @@ -37,8 +37,10 @@ using Key = std::vector; struct DeviceLibraries { PFN_vkDestroyPipeline destroyPipeline = nullptr; PFN_vkDestroyPipelineLayout destroyLayout = nullptr; + PFN_vkDestroyShaderModule destroyModule = nullptr; std::map layouts; std::array, 4> parts; + std::vector modules; PipelineLibraryCounters counters; }; @@ -99,7 +101,7 @@ std::span PipelineLibraryDynamicStates() { return dynamicStates; } -VkPipeline LinkPipelineFromLibraries(const Context& context, const VkGraphicsPipelineCreateInfo& info, const VkPipelineRenderingCreateInfoKHR& rendering, const VkPipelineLayoutCreateInfo& layout, const PipelineLibraryKeys& keys, std::shared_ptr* optimized) { +VkPipeline LinkPipelineFromLibraries(const Context& context, const VkGraphicsPipelineCreateInfo& info, const VkPipelineRenderingCreateInfoKHR& rendering, const VkPipelineLayoutCreateInfo& layout, const PipelineLibraryKeys& keys, std::shared_ptr* optimized, std::span modules) { Require(info.renderPass == VK_NULL_HANDLE && info.pDynamicState != nullptr, "pipeline library parts need dynamic rendering and the library dynamic states"); auto& store = Libraries(); std::lock_guard lock(store.mutex); @@ -107,6 +109,7 @@ VkPipeline LinkPipelineFromLibraries(const Context& context, const VkGraphicsPip if (device.destroyPipeline == nullptr) { device.destroyPipeline = context.Function("vkDestroyPipeline"); device.destroyLayout = context.Function("vkDestroyPipelineLayout"); + device.destroyModule = context.Function("vkDestroyShaderModule"); } auto pipelineLayout = device.layouts.find(keys.layout); if (pipelineLayout == device.layouts.end()) { @@ -135,6 +138,7 @@ VkPipeline LinkPipelineFromLibraries(const Context& context, const VkGraphicsPip const std::array contextKeys{&none, &keys.layout, &keys.layout, &keys.renderPass}; const VkPipelineRenderingCreateInfoKHR shaderRendering{VK_STRUCTURE_TYPE_PIPELINE_RENDERING_CREATE_INFO_KHR}; VkPipelineDepthStencilStateCreateInfo dynamicDepth{VK_STRUCTURE_TYPE_PIPELINE_DEPTH_STENCIL_STATE_CREATE_INFO}; + static const bool optimize = std::getenv("APS5_NO_PIPELINE_LTO") == nullptr; std::vector libraries; for (std::size_t part = 0; part < flags.size(); ++part) { if (part == 0 && mesh) continue; @@ -179,6 +183,15 @@ VkPipeline LinkPipelineFromLibraries(const Context& context, const VkGraphicsPip VkPipeline handle = VK_NULL_HANDLE; Check(context.Function("vkCreateGraphicsPipelines")(context.device, context.pipelineCache, 1, &create, nullptr, &handle), "vkCreateGraphicsPipelines library"); cache.emplace(key, handle); + if (optimize) { + for (std::uint32_t stage = 0; stage < create.stageCount; ++stage) { + for (auto& module : modules) { + if (module == VK_NULL_HANDLE || module != create.pStages[stage].module) continue; + device.modules.push_back(module); + module = VK_NULL_HANDLE; + } + } + } ++device.counters.built[part]; libraries.push_back(handle); } @@ -191,7 +204,6 @@ VkPipeline LinkPipelineFromLibraries(const Context& context, const VkGraphicsPip VkPipeline pipeline = VK_NULL_HANDLE; Check(context.Function("vkCreateGraphicsPipelines")(context.device, context.pipelineCache, 1, &create, nullptr, &pipeline), "vkCreateGraphicsPipelines link"); ++device.counters.linked; - static const bool optimize = std::getenv("APS5_NO_PIPELINE_LTO") == nullptr; if (optimize && optimized != nullptr) { auto slot = std::make_shared(); slot->device = context.device; @@ -242,6 +254,7 @@ void ClearPipelineLibraries(VkDevice device) { for (auto& part : libraries.parts) { for (const auto& [key, handle] : part) libraries.destroyPipeline(device, handle, nullptr); } + for (const auto module : libraries.modules) libraries.destroyModule(device, module, nullptr); for (const auto& [key, handle] : libraries.layouts) libraries.destroyLayout(device, handle, nullptr); store.devices.erase(found); } From c447b289ca37a4286241dd43383131c8b34aea9c Mon Sep 17 00:00:00 2001 From: overlordxrz-source Date: Sat, 10 Oct 2026 23:42:50 +0000 Subject: [PATCH 251/460] fix(shader): read 2D-array T#s through 3D image accesses Since #2133, a 3D image_sample_lz or image_load bound to a 2D-array T# decodes as 2D and matches the new 2D mode, so it reads layer 0 for any r and returns zero for z != 0. adriadam10 measured on RDNA2 (gfx1035) that dim 3D on a 2D-array T# behaves exactly like dim 2D-array, with r or z as the layer. Decode a 2D-array T# bound to a 3D access as 2D-array and add a 2D-array mode next to the 2D one. The same measurement covers image_load_mip on both T# types and image_get_resinfo, which returns (w, h, 1, levels) on a 2D T# and (w, h, layers, levels) on a 2D-array T#. Allow those accesses and return 1 as the third resinfo component through the 2D mode. image_get_lod is not measured and keeps throwing. --- .../ImageVolumeAddressPlanarTexture.cpp | 209 ++++++++++++++++++ .../Optimization/src/ResourceMaterializer.cpp | 10 +- .../Optimization/src/ResourceTracker.cpp | 5 +- .../SpirvBackend/src/SpirvImageEmitter.cpp | 3 + docs/dev/TechnicalDebt.md | 2 +- 5 files changed, 222 insertions(+), 7 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/ImageVolumeAddressPlanarTexture.cpp diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/ImageVolumeAddressPlanarTexture.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/ImageVolumeAddressPlanarTexture.cpp new file mode 100644 index 000000000..3e311f204 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/ImageVolumeAddressPlanarTexture.cpp @@ -0,0 +1,209 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "prx/libSceAgcDriver/Graphics/include/TextureTiling.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 32; +constexpr std::uint32_t Words = 32; +constexpr std::uint32_t Size = 16; +constexpr std::uint32_t Layers = 2; +constexpr std::uint32_t Format8888UNorm = 56; +constexpr std::uint32_t Type2D = 9; +constexpr std::uint32_t Type2DArray = 13; +constexpr std::uint32_t SampleResult = 8; +constexpr std::uint32_t LoadResult = 12; +constexpr std::uint32_t LoadMipResult = 16; +constexpr std::uint32_t ResinfoResult = 20; +alignas(256) std::array Buffer{}; +alignas(256) std::array Texels{}; + +alignas(256) constexpr std::array Code{ + 0x34020087, 0xe0301000, 0x80000201, 0xe0301004, 0x80000301, 0xe0301008, 0x80000401, 0xe030100c, + 0x80000501, 0xe0301010, 0x80000601, 0xe0301014, 0x80000701, 0xe0301018, 0x80000801, 0xe030101c, + 0x80000901, 0xbf8c3f70, 0xf09c0f10, 0x00610c02, 0xf0001f10, 0x00011005, 0xf0041f10, 0x00011405, + 0xf0381f10, 0x00011809, 0xbf8c3f70, 0xe0701020, 0x80000c01, 0xe0701024, 0x80000d01, 0xe0701028, + 0x80000e01, 0xe070102c, 0x80000f01, 0xe0701030, 0x80001001, 0xe0701034, 0x80001101, 0xe0701038, + 0x80001201, 0xe070103c, 0x80001301, 0xe0701040, 0x80001401, 0xe0701044, 0x80001501, 0xe0701048, + 0x80001601, 0xe070104c, 0x80001701, 0xe0701050, 0x80001801, 0xe0701054, 0x80001901, 0xe0701058, + 0x80001a01, 0xe070105c, 0x80001b01, 0xbf810000, +}; + +alignas(256) constexpr std::array LodCode{ + 0x34020087, 0xe0301000, 0x80000201, 0xe0301004, 0x80000301, 0xe0301008, 0x80000401, 0xbf8c3f70, + 0xf09c0f10, 0x00610c02, 0xf1800310, 0x00611002, 0xbf8c3f70, 0xe0701020, 0x80000c01, 0xe0701040, + 0x80001001, 0xbf810000, +}; + +struct Lane { + std::uint32_t x; + std::uint32_t y; + float r; + std::uint32_t z; + std::uint32_t sampledLayer; + bool loadInside; +}; + +constexpr std::array PlanarR{0.0f, 0.5f, 1.0f, 1.5f, -0.25f, 3.0f}; +constexpr std::array PlanarZ{0u, 1u, 0u, 4u, 0u, 0xffffffffu, 0u, 2u}; +constexpr std::array ArrayR{0.0f, 1.0f, 3.0f, -0.25f, 0.75f, 1.2f}; +constexpr std::array ArrayLayer{0u, 1u, 1u, 0u, 1u, 1u}; + +Lane LaneOf(std::uint32_t tid, bool array) { + const auto x = (tid * 7u + 3u) % Size; + const auto y = (tid * 5u + 1u) % Size; + if (array) return {x, y, ArrayR[tid % ArrayR.size()], tid % Layers, ArrayLayer[tid % ArrayLayer.size()], true}; + const auto z = PlanarZ[tid % PlanarZ.size()]; + return {x, y, PlanarR[tid % PlanarR.size()], z, 0u, z == 0u}; +} + +std::uint32_t Bits(float value) { + return std::bit_cast(value); +} + +std::uint8_t LayerMarker(std::uint32_t layer) { + return static_cast(layer * 100u + 7u); +} + +void FillInput(bool array) { + Buffer.fill(0xdeadbeefu); + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + const auto lane = LaneOf(tid, array); + const std::array words{Bits((static_cast(lane.x) + 0.5f) / static_cast(Size)), Bits((static_cast(lane.y) + 0.5f) / static_cast(Size)), Bits(lane.r), lane.x, lane.y, lane.z, 0u, 0u}; + std::copy(words.begin(), words.end(), &Buffer[tid * Words]); + } +} + +std::array BufferDescriptor(const void* data, std::uint32_t bytes) { + const auto address = reinterpret_cast(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; +} + +std::array TextureDescriptor(bool array) { + const auto address = static_cast(reinterpret_cast(Texels.data())); + return { + static_cast(address >> 8u), + static_cast((address >> 40u) & 0xffu) | (Format8888UNorm << 20u) | (((Size - 1u) & 3u) << 30u), + ((Size - 1u) >> 2u) | ((Size - 1u) << 14u), + 0xfacu | ((array ? Type2DArray : Type2D) << 28u), + array ? Layers - 1u : 0u, + 0u, + 0u, + 0u, + }; +} + +void FillTexture(bool array) { + const auto geometry = AgcDriver::Graphics::DescribeSurface(AgcDriver::Graphics::DecodeTextureResource(TextureDescriptor(array))); + Require(geometry.guestBytes <= Texels.size(), "volume address on a planar texture: the texture does not fit the texel storage"); + const auto& mip = geometry.mips.at(0); + Texels.fill(0xeeu); + for (std::uint32_t layer = 0; layer < (array ? Layers : 1u); ++layer) { + for (std::uint32_t y = 0; y < Size; ++y) { + for (std::uint32_t x = 0; x < Size; ++x) { + auto* texel = &Texels[geometry.GuestLayerOffset(layer) + mip.tiledOffset + static_cast(y) * mip.pitchBytes + x * 4u]; + texel[0] = LayerMarker(layer); + texel[1] = static_cast(x); + texel[2] = static_cast(y); + texel[3] = 255u; + } + } + } +} + +std::array SamplerDescriptor() { + return {0u, 0xfffu << 12u, 1u << 26u, 0u}; +} + +void Run(AgcDriver::VulkanDevice& device, std::span code, bool array) { + std::vector userData(16, 0u); + const auto buffer = BufferDescriptor(Buffer.data(), static_cast(Buffer.size() * 4u)); + const auto texture = TextureDescriptor(array); + const auto sampler = SamplerDescriptor(); + std::copy(buffer.begin(), buffer.end(), userData.begin()); + std::copy(texture.begin(), texture.end(), userData.begin() + 4); + std::copy(sampler.begin(), sampler.end(), userData.begin() + 12); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {false, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); +} + +void ExpectTexel(std::uint32_t tid, std::uint32_t result, const std::array& expected, const std::string& name) { + for (std::uint32_t component = 0; component < 4u; ++component) { + const float value = std::bit_cast(Buffer[tid * Words + result + component]) * 255.0f; + Require(std::lround(value) == static_cast(expected[component]), name + ": thread " + std::to_string(tid) + " component " + std::to_string(component) + " is " + std::to_string(value) + ", expected " + std::to_string(expected[component])); + } +} + +void Check(bool array) { + const std::string texture = array ? "a 2D-array texture" : "a 2D texture"; + const std::array resinfo{Size, Size, array ? Layers : 1u, 1u}; + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + const auto lane = LaneOf(tid, array); + const std::array sampled{LayerMarker(lane.sampledLayer), lane.x, lane.y, 255u}; + const std::array loaded = lane.loadInside ? std::array{LayerMarker(lane.z), lane.x, lane.y, 255u} : std::array{0u, 0u, 0u, 0u}; + const auto z = std::to_string(static_cast(lane.z)); + ExpectTexel(tid, SampleResult, sampled, "image_sample_lz 3d on " + texture + " with r " + std::to_string(lane.r)); + ExpectTexel(tid, LoadResult, loaded, "image_load 3d on " + texture + " with z " + z); + ExpectTexel(tid, LoadMipResult, loaded, "image_load_mip 3d on " + texture + " with z " + z); + for (std::uint32_t component = 0; component < 4u; ++component) { + const auto value = Buffer[tid * Words + ResinfoResult + component]; + Require(value == resinfo[component], "image_get_resinfo 3d on " + texture + ": thread " + std::to_string(tid) + " component " + std::to_string(component) + " is " + std::to_string(value) + ", expected " + std::to_string(resinfo[component])); + } + } +} + +void RequireLodRefused(AgcDriver::VulkanDevice& device, bool array) { + std::string refusal; + try { + Run(device, LodCode, array); + } catch (const std::exception& error) { + refusal = error.what(); + } + Require(refusal.find("incompatible with the static runtime image interface") != std::string::npos, std::string("image_get_lod 3d on a ") + (array ? "2D-array" : "2D") + " texture was not refused: " + refusal); +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + for (const bool array : {false, true}) { + FillInput(array); + FillTexture(array); + Run(*device, Code, array); + Check(array); + RequireLodRefused(*device, array); + } + std::puts("volume address on planar texture tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp b/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp index f7f91586f..070aaef87 100644 --- a/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp +++ b/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp @@ -91,7 +91,7 @@ RdnaImageDimension descriptorDimension(const DescriptorValue& descriptor, RdnaIm case ImageType::Cube: return RdnaImageDimension::Dim2DArray; case ImageType::Color2DArray: - return wantArray ? RdnaImageDimension::Dim2DArray : RdnaImageDimension::Dim2D; + return wantArray || requested == RdnaImageDimension::Dim3D ? RdnaImageDimension::Dim2DArray : RdnaImageDimension::Dim2D; case ImageType::Color2DMsaaArray: return wantArray ? RdnaImageDimension::Dim2DMsaaArray : RdnaImageDimension::Dim2DMsaa; case ImageType::Color2D: @@ -632,9 +632,11 @@ std::vector ResourceMaterializer::RuntimeImageModes(const ImageRe modes.push_back(volume); } if (image.dimension == RdnaImageDimension::Dim3D && image.flatVolumeCompatible && !storage && !depth && conversion == IrBufferFormat::Invalid && packed == IrBufferFormat::Invalid) { - auto plane = mode; - plane.dimension = RdnaImageDimension::Dim2D; - modes.push_back(plane); + for (const auto dimension : {RdnaImageDimension::Dim2D, RdnaImageDimension::Dim2DArray}) { + auto plane = mode; + plane.dimension = dimension; + modes.push_back(plane); + } } if (image.dimension == RdnaImageDimension::Dim2D && storage && image.written && !image.read && !image.atomic && image.mipMode != ImageMipMode::DynamicStorage && !depth && packed == IrBufferFormat::Invalid && image.byElements == 0u) { auto line = mode; diff --git a/core/shader/recompiler/Optimization/src/ResourceTracker.cpp b/core/shader/recompiler/Optimization/src/ResourceTracker.cpp index 20be15285..414d6af97 100644 --- a/core/shader/recompiler/Optimization/src/ResourceTracker.cpp +++ b/core/shader/recompiler/Optimization/src/ResourceTracker.cpp @@ -1104,9 +1104,10 @@ private: image.fmaskCompatible = image.fmaskCompatible && op == IrOpcode::ImageRead && memory.dataBits == 32u; image.depthBitsCompatible = image.depthBitsCompatible && memory.dataBits == 32u; image.constantSwizzleCompatible = image.constantSwizzleCompatible && (op == IrOpcode::ImageSampleRaw || op == IrOpcode::ImageGatherRaw); - const bool flatVolumeLoad = op == IrOpcode::ImageRead && !memory.imageHasMip && memory.imageSampleFlags == 0u; + const bool flatVolumeLoad = op == IrOpcode::ImageRead && memory.imageSampleFlags == 0u; const bool flatVolumeSample = op == IrOpcode::ImageSampleRaw && memory.imageSampleFlags == RdnaImageSampleFlagLevelZero; - image.flatVolumeCompatible = image.flatVolumeCompatible && (flatVolumeLoad || flatVolumeSample) && !memory.imagePacked && memory.imageByElements == 0u; + const bool flatVolumeQuery = op == IrOpcode::ImageQueryDimensions; + image.flatVolumeCompatible = image.flatVolumeCompatible && (flatVolumeLoad || flatVolumeSample || flatVolumeQuery) && !memory.imagePacked && memory.imageByElements == 0u; const bool flatLineLoad = op == IrOpcode::ImageRead && !memory.imageHasMip && memory.imageSampleFlags == 0u; const bool flatLineSample = op == IrOpcode::ImageSampleRaw && memory.imageSampleFlags == RdnaImageSampleFlagLevelZero; image.flatLineCompatible = image.flatLineCompatible && (flatLineLoad || flatLineSample) && !memory.imagePacked && memory.imageByElements == 0u; diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp index 9ca9a0dd1..3d65d9ea1 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp @@ -434,6 +434,9 @@ std::uint32_t QueryDimensions(SpirvValueEmitContext& ctx, const ImageEmitAccess& state.module.AddFunction(spv::OpCompositeExtract, TypeU32(state), result[index], size, index); } } + if (access.mem.imageDimension == RdnaImageDimension::Dim3D && dimension == RdnaImageDimension::Dim2D) { + result[2] = ConstantU32(state, 1); + } if (info.multisampled == 0u) { result[3] = state.module.AllocateId(); state.module.AddFunction(spv::OpImageQueryLevels, TypeU32(state), result[3], image); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 337ef24ed..f7d0c3788 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -359,7 +359,7 @@ Throughout the project, every function at every stage either **does exactly what - [MIMG BY2/BY4 loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_load[_mip]_by2/by4`, opcodes 0x42, 0x43, 0x4a and 0x4b) were measured on RDNA2 (gfx1035) with the [hardware oracle](HW_ORACLE.md) for 2D images with identity swizzle. They read N texels (BY2: 2, BY4: 4) from `x & ~(N - 1)` on row y, texel by texel and component by component within each texel, with each texel converted like `image_load`. Every register is zero when the mip level is past the last level, `x + N - 1` is outside the level's width (unaligned x, unsigned), or y is outside its height. Data only comes back when the N texels fit in one dword: R8 and R16 (DMASK 0x3) or RG8 (0xf) for BY2, and R8 (0xf) for BY4. With 32-bit texels (R32, RG32, RG16, RGBA8) the hardware returns zeros. Other formats, masks, dimensions, swizzles, D16/A16 and indirect descriptors throw. The stores (0x52, 0x53, 0x5a and 0x5b) decode but throw at translation. Measured on hardware, they write from the unaligned x up to the end of its dword, which is not implemented. - [2D stores to 1D storage images](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp): an `image_store` with DIM 2D through a 1D T# writes texel x and ignores y, as measured on RDNA2 (gfx1035) for y from 0 to 4 and 0xffffffff, single- and four-component formats and a 1x1 8_8_8_8_UNORM T#: the T# type decides the addressing, DIM only how many address VGPRs are read. Storage loads, atomics and `image_store_mip` through such a pair still throw - [MIMG PCK2/PCK4 loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_load[_mip]_pck2/pck4`, opcodes 0x70, 0x71, 0x73 and 0x74) were measured on RDNA2 (gfx1035) with the [hardware oracle](HW_ORACLE.md) for 2D images with identity swizzle. With data mask 0x1, they write one dword: the raw bits of the N texels (PCK2: 2, PCK4: 4) from `x & ~(N - 1)` on row y, concatenated (element k at bit `k * texel bits`) and zero-extended. The dword is zero when the mip level is past the last level, `x + N - 1` is outside the level's width (unaligned x, unsigned), or y is outside its height. Data only comes back when N texels fit in 32 bits: R8, R16 or RG8 for PCK2, R8 for PCK4. Wider texels (R32, RG16, RGBA8) and masks other than 0x1 write zeros on hardware. Those masks throw at decode, and formats outside UNORM/UINT/SINT R8, R16 and RG8 throw. #1314's earlier measurement saw only zeros because it used masks 0x3 and 0xf. The stores (0x76, 0x77, 0x79 and 0x7a) decode but throw at translation. They are not measured, and opcode 0x76 on 32_32_32_32_FLOAT hung the GPU (#1314). -- [3D image accesses bound to a 2D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) follow CouchRiv's [hardware oracle](HW_ORACLE.md) measurement on RDNA2 (gfx1036, #2023), not on the PS5's gfx1013: `image_sample_lz` ignores r and reads the (u,v) texel, and `image_load` without a mip reads the (x,y) texel when z is 0 and returns zero otherwise. Every other access of such an image, including samples with an LOD, bias, derivatives, offset or comparison, gathers, queries and stores, keeps throwing because it is not measured. +- [3D image accesses bound to a 2D or 2D-array T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) follow CouchRiv's [hardware oracle](HW_ORACLE.md) measurement on RDNA2 (gfx1036, #2023) and adriadam10's (gfx1035, #2140), not the PS5's gfx1013. On a 2D T#, `image_sample_lz` ignores r and reads the (u,v) texel, `image_load` and `image_load_mip` read the (x,y) texel when z is 0 and return zero otherwise, and `image_get_resinfo` returns (width, height, 1, levels). On a 2D-array T#, these accesses behave like dim 2D-array: r or z is the layer, and `image_get_resinfo` returns the layer count as its third component. Every other access of such an image, including samples with an LOD, bias, derivatives, offset or comparison, gathers, `image_get_lod` and stores, keeps throwing because it is not measured. - [Unnormalized S# coordinates](../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestSamplerResource.cpp) (libSceAgcDriver) - `FORCE_UNNORMALIZED` uses a Vulkan `unnormalizedCoordinates` sampler, so the S# LOD range, LOD bias and mip filter are not applied, which is exact only on a one-level view. - [Unnormalized S#s](../../core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp) are accepted only when direct and used only by explicit-LOD `image_sample` without offset, comparison, derivatives or `_a`, on a single-level, single-layer 1D or 2D view of mip 0, even when the image is also sampled through a normalized S#. - An [unnormalized S#](../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestSamplerResource.cpp) (libSceAgcDriver) throws with unequal minification and magnification filters, anisotropy, `TRUNC_COORD`, `MC_COORD_TRUNC` or X/Y clamping other than last-texel or border. From 90217350227d1b7769db4d77a45a7bf42e63f5d5 Mon Sep 17 00:00:00 2001 From: XairRrR Date: Sat, 10 Oct 2026 13:34:24 +0200 Subject: [PATCH 252/460] feat(libSceVideoOut): accept VRR peg and unpeg on a fixed-rate output sceVideoOutVrrPegToFixedRate and sceVideoOutVrrUnpegFromFixedRate were NotImplemented without arguments. Both take the output handle; the peg also takes two further arguments whose meaning is unknown. VRR is not modelled and the output always runs at a fixed rate, so both validate the handle and return 0; non-zero peg arguments throw. Recorded in TechnicalDebt. Uncharted: Legacy of Thieves Collection (PPSA05684) stops on main with "sceVideoOutVrrUnpegFromFixedRate not implemented"; it passes its output handle to both (and zeros to the peg) and ignores the results. --- core/libs/prx/libSceVideoOut/src/Output.cpp | 15 +++++++++++---- docs/dev/TechnicalDebt.md | 1 + 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceVideoOut/src/Output.cpp b/core/libs/prx/libSceVideoOut/src/Output.cpp index 412ab1bf2..bf4b18327 100644 --- a/core/libs/prx/libSceVideoOut/src/Output.cpp +++ b/core/libs/prx/libSceVideoOut/src/Output.cpp @@ -301,15 +301,22 @@ int APS5_VABI sceVideoOutAdjustColor(int handle, const VideoOutColorSettings* se LibcAwaitExit_nid_postfix(); } -int APS5_VABI sceVideoOutVrrUnpegFromFixedRate() try { - NotImplemented_nid_no_patch(__func__); +int APS5_VABI sceVideoOutVrrUnpegFromFixedRate(int handle) try { + if (VideoOutDriver::Get().GetConfig(handle) == nullptr) { + throw std::runtime_error(std::string(__func__) + ": VIDEO_OUT_ERROR_INVALID_HANDLE"); + } return 0; } catch (const ProcessShutdown&) { LibcAwaitExit_nid_postfix(); } -int APS5_VABI sceVideoOutVrrPegToFixedRate() try { - NotImplemented_nid_no_patch(__func__); +int APS5_VABI sceVideoOutVrrPegToFixedRate(int handle, int32_t arg1, int32_t arg2) try { + if (VideoOutDriver::Get().GetConfig(handle) == nullptr) { + throw std::runtime_error(std::string(__func__) + ": VIDEO_OUT_ERROR_INVALID_HANDLE"); + } + if (arg1 != 0 || arg2 != 0) { + throw std::runtime_error(std::string(__func__) + ": unknown arguments " + std::to_string(arg1) + ", " + std::to_string(arg2)); + } return 0; } catch (const ProcessShutdown&) { LibcAwaitExit_nid_postfix(); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 6f7560b6c..8977323f2 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -31,6 +31,7 @@ Throughout the project, every function at every stage either **does exactly what - [libSceBluetoothHid](../../core/libs/prx/libSceBluetoothHid/Export.cpp) implements only the no-device path: initialization and callback and device registration succeed, the callback is never called because no device connects, and calls that need a connected device throw - [sceVideoOutOpen](../../core/libs/prx/libSceVideoOut/src/Output.cpp) validates the priority and CPU affinity that the open param requests for the VideoOut service thread but does not apply them: the port's present and vblank threads are host threads, and guest priorities and affinities do not reach host scheduling - [sceVideoOutAddVrrStatusFlagsPrivilege](../../core/libs/prx/libSceVideoOut/src/Output.cpp) returns 0 and does nothing: the host output has no VRR state to report. The name comes from an assert message in a title that requires `SCE_VIDEO_OUT_OK`; the NID hashes to it, the arguments are unknown (it is called without any) +- [sceVideoOutVrrPegToFixedRate, sceVideoOutVrrUnpegFromFixedRate](../../core/libs/prx/libSceVideoOut/src/Output.cpp) (libSceVideoOut) - PPSA05684 passes the handle to both, two further zero arguments to `sceVideoOutVrrPegToFixedRate`, and ignores the results. The meaning of those two arguments is unknown, so only zeros are accepted. VRR is not modelled and the output always runs at a fixed rate, so both calls only validate the handle and return 0 - [libSceAudio3d](../../core/libs/prx/libSceAudio3d/Export.cpp) - the port produces no sound: objects can be reserved and `sceAudio3dObjectSetAttributes` accepts their attributes, but the library has no bed exports or mixing to give them audio, the attributes have no effect and `sceAudio3dPortPush` only paces the queue at `granularity` samples per frame at 48 kHz. `sceAudio3dPortSetAttribute` accepts the late reverb level and the downmix spread attributes and ignores them. `sceAudio3dObjectSetAttributes` validates the port, the reserved object and the attribute array (an object id that was never reserved returns the invalid-object error, while shadPS4 returns success for it and the error only for a released object; neither is measured on a console), treating the reset-state attribute as valid with a null value and requiring a value for the others; the attribute identifiers and the value of each are not interpreted. `sceAudio3dPortFlush` drops the frames queued by `sceAudio3dPortAdvance` instead of mixing and submitting them to an output, as there is no output - [libSceTextToSpeech2](../../core/libs/prx/libSceTextToSpeech2/Export.cpp) has no speech engine: `sceTextToSpeech2Initialize` fails and the other exports throw. PPSA01317 skips its narration set-up when initialization fails - [libSceAvPlayer](../../core/libs/prx/libSceAvPlayer/Export.cpp): `sceAvPlayerSetLogCallback` accepts a callback that is never called, as the player produces no log messages, and `sceAvPlayerSetAvailableBandwidth` has no effect, as it governs HLS sources, which `sceAvPlayerAddSource` does not implement. From a2d82815b392909355a2a30d1a4d24b57a6d045f Mon Sep 17 00:00:00 2001 From: ThatsLeo Date: Sat, 10 Oct 2026 16:02:27 +0200 Subject: [PATCH 253/460] fix(shader): take the LDS lock with one attempt per loop iteration The elected lane's inner spin loop never ended on Mesa ANV with the LDS in a storage buffer. It now tries once and the outer loop elects again; the update is unchanged. (cherry picked from commit e894f9092326a849ebe1ce46ff0ed3b25fbb7e9b) --- .../SpirvMemory/SpirvMemoryInstructions.cpp | 27 ++++++++++--------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp index 2515e574c..70296b7f3 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp @@ -1192,9 +1192,8 @@ std::uint32_t LockedLdsUpdate(SpirvEmitterState& state, std::uint32_t type, std: const auto header = state.module.AllocateId(); const auto body = state.module.AllocateId(); const auto pending = state.module.AllocateId(); - const auto spinHeader = state.module.AllocateId(); - const auto spinBody = state.module.AllocateId(); - const auto spinCont = state.module.AllocateId(); + const auto attempt = state.module.AllocateId(); + const auto attemptMerge = state.module.AllocateId(); const auto critical = state.module.AllocateId(); const auto servedMerge = state.module.AllocateId(); const auto pendingMerge = state.module.AllocateId(); @@ -1217,27 +1216,29 @@ std::uint32_t LockedLdsUpdate(SpirvEmitterState& state, std::uint32_t type, std: const auto elected = state.module.AllocateId(); state.module.AddFunction(spv::OpGroupNonUniformElect, TypeBool(state), elected, ConstantU32(state, spv::ScopeSubgroup)); state.module.AddFunction(spv::OpSelectionMerge, servedMerge, spv::SelectionControlMaskNone); - state.module.AddFunction(spv::OpBranchConditional, elected, spinHeader, servedMerge); - EmitLabel(state, spinHeader); - state.module.AddFunction(spv::OpLoopMerge, critical, spinCont, spv::LoopControlMaskNone); - state.module.AddFunction(spv::OpBranch, spinBody); - EmitLabel(state, spinBody); + state.module.AddFunction(spv::OpBranchConditional, elected, attempt, servedMerge); + EmitLabel(state, attempt); const auto previous = state.module.AllocateId(); state.module.AddFunction(spv::OpAtomicCompareExchange, TypeU32(state), previous, lock, ConstantU32(state, spv::ScopeWorkgroup), ConstantU32(state, spv::MemorySemanticsAcquireMask | LdsMemorySemantics(state)), ConstantU32(state, spv::MemorySemanticsMaskNone), ConstantU32(state, 1u), ConstantU32(state, 0u)); const auto acquired = Binary(state, spv::OpIEqual, TypeBool(state), previous, ConstantU32(state, 0u)); - state.module.AddFunction(spv::OpBranchConditional, acquired, critical, spinCont); - EmitLabel(state, spinCont); - state.module.AddFunction(spv::OpBranch, spinHeader); + state.module.AddFunction(spv::OpSelectionMerge, attemptMerge, spv::SelectionControlMaskNone); + state.module.AddFunction(spv::OpBranchConditional, acquired, critical, attemptMerge); EmitLabel(state, critical); const auto old = update(); state.module.AddFunction(spv::OpAtomicStore, lock, ConstantU32(state, spv::ScopeWorkgroup), ConstantU32(state, spv::MemorySemanticsReleaseMask | LdsMemorySemantics(state)), ConstantU32(state, 0u)); const auto criticalExit = state.currentLabel; + state.module.AddFunction(spv::OpBranch, attemptMerge); + EmitLabel(state, attemptMerge); + const auto attempted = state.module.AllocateId(); + const auto attemptedResult = state.module.AllocateId(); + state.module.AddFunction(spv::OpPhi, TypeBool(state), attempted, ConstantBool(state, true), criticalExit, ConstantBool(state, false), attempt); + state.module.AddFunction(spv::OpPhi, type, attemptedResult, old, criticalExit, result, attempt); state.module.AddFunction(spv::OpBranch, servedMerge); EmitLabel(state, servedMerge); const auto served = state.module.AllocateId(); const auto servedResult = state.module.AllocateId(); - state.module.AddFunction(spv::OpPhi, TypeBool(state), served, ConstantBool(state, true), criticalExit, ConstantBool(state, false), pending); - state.module.AddFunction(spv::OpPhi, type, servedResult, old, criticalExit, result, pending); + state.module.AddFunction(spv::OpPhi, TypeBool(state), served, attempted, attemptMerge, ConstantBool(state, false), pending); + state.module.AddFunction(spv::OpPhi, type, servedResult, attemptedResult, attemptMerge, result, pending); state.module.AddFunction(spv::OpBranch, pendingMerge); EmitLabel(state, pendingMerge); state.module.AddFunction(spv::OpPhi, TypeBool(state), doneNext, served, servedMerge, done, body); From 709c12e133bdf98a8077981910b16720fdc85f76 Mon Sep 17 00:00:00 2001 From: Hussein Kadhim Date: Sat, 10 Oct 2026 15:22:35 +0300 Subject: [PATCH 254/460] fix(libkernel): return EPERM or EINVAL when a condition wait gets a mutex the caller does not hold pthread_cond_wait and pthread_cond_timedwait, and their sce counterparts, threw a host exception when the mutex was unlocked, locked by another thread, or not yet initialized, and dereferenced the adaptive static initializer. FreeBSD returns EPERM for these (EINVAL for a destroyed mutex) before touching the condition. (cherry picked from commit a78bb8a855ca3b277f128c6a9e52ff3949b9cb80) --- core/libs/prx/libkernel/CMakeLists.txt | 10 ++ core/libs/prx/libkernel/Pthread/src/Cond.cpp | 28 +++-- core/libs/tests/GuestCondUnownedMutex.cpp | 106 +++++++++++++++++++ 3 files changed, 137 insertions(+), 7 deletions(-) create mode 100644 core/libs/tests/GuestCondUnownedMutex.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index 4961439df..32ac770fc 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -343,6 +343,16 @@ if(BUILD_TESTING) set_tests_properties(guest_cond_recursive_mutex PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_cond_unowned_mutex_tests ${LIBS_INCLUDE_DIR}/tests/GuestCondUnownedMutex.cpp) + target_include_directories(guest_cond_unowned_mutex_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_cond_unowned_mutex_tests PRIVATE libkernel libc) + configure_windows_unwind(guest_cond_unowned_mutex_tests) + add_test(NAME guest_cond_unowned_mutex COMMAND guest_cond_unowned_mutex_tests) + set_tests_properties(guest_cond_unowned_mutex PROPERTIES TIMEOUT 20) + if(WIN32) + set_tests_properties(guest_cond_unowned_mutex PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_cpu_time_clocks_tests ${LIBS_INCLUDE_DIR}/tests/GuestCpuTimeClocks.cpp) target_include_directories(guest_cpu_time_clocks_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_cpu_time_clocks_tests PRIVATE libkernel libc) diff --git a/core/libs/prx/libkernel/Pthread/src/Cond.cpp b/core/libs/prx/libkernel/Pthread/src/Cond.cpp index 66887e6a7..4cddf4d5b 100644 --- a/core/libs/prx/libkernel/Pthread/src/Cond.cpp +++ b/core/libs/prx/libkernel/Pthread/src/Cond.cpp @@ -3,6 +3,7 @@ #include "prx/libkernel/Pthread/include/Cond.hpp" #include "prx/libkernel/Pthread/include/Cancel.hpp" #include "prx/libkernel/Pthread/Posix/Common.hpp" +#include "prx/libkernel/KernelErrors.hpp" #include "prx/libkernel/Time/include/Time.hpp" #include "prx/libkernel/Time/include/TimedWait.hpp" #include @@ -22,6 +23,14 @@ PthreadCond destroyedCond() { return reinterpret_cast(std::uintptr_t{2}); } +PthreadMutex destroyedMutex() { + return reinterpret_cast(std::uintptr_t{2}); +} + +PthreadMutex adaptiveInitializer() { + return reinterpret_cast(std::uintptr_t{1}); +} + PthreadCond resolveCond(PthreadCond* cond) { if (!cond) throw std::invalid_argument("Condition variable pointer is null"); @@ -39,18 +48,23 @@ PthreadCond resolveCond(PthreadCond* cond) { return created; } -PthreadMutex lockedMutex(PthreadMutex* mutex) { - if (!mutex || !*mutex) +int mutexOwnership(PthreadMutex* mutex, PthreadMutex* owned) { + if (!mutex) throw std::invalid_argument("Mutex pointer is null"); - auto* current = *mutex; - if (current->_owner.load(std::memory_order_acquire) != std::this_thread::get_id()) - throw std::runtime_error("Condition wait mutex is not owned by the current thread"); - return current; + const auto current = std::atomic_ref(*mutex).load(std::memory_order_acquire); + if (current == destroyedMutex()) + return SCE_KERNEL_ERROR_EINVAL; + if (!current || current == adaptiveInitializer() || current->_owner.load(std::memory_order_acquire) != std::this_thread::get_id()) + return SCE_KERNEL_ERROR_EPERM; + *owned = current; + return 0; } int waitUntil(PthreadCond* cond, PthreadMutex* mutex, std::optional deadlineNanos, const void* caller) { auto* c = resolveCond(cond); - auto* m = lockedMutex(mutex); + PthreadMutex m = nullptr; + if (const int error = mutexOwnership(mutex, &m)) + return error; bool timedOut = false; const auto waitStart = std::chrono::steady_clock::now(); struct Trace { diff --git a/core/libs/tests/GuestCondUnownedMutex.cpp b/core/libs/tests/GuestCondUnownedMutex.cpp new file mode 100644 index 000000000..f68fe1739 --- /dev/null +++ b/core/libs/tests/GuestCondUnownedMutex.cpp @@ -0,0 +1,106 @@ +#include "SceTypes.hpp" +#include +#include +#include +#include + +extern "C" { +int APS5_VABI scePthreadCreate(Pthread* thread, const PthreadAttr* attr, PthreadEntry entry, void* arg, const char* name); +int APS5_VABI scePthreadJoin(Pthread thread, void** retval); +int APS5_VABI scePthreadMutexInit(PthreadMutex* mutex, const PthreadMutexattr* attr, const char* name); +int APS5_VABI scePthreadMutexDestroy(PthreadMutex* mutex); +int APS5_VABI scePthreadMutexLock(PthreadMutex* mutex); +int APS5_VABI scePthreadMutexUnlock(PthreadMutex* mutex); +int APS5_VABI scePthreadCondInit(PthreadCond* cond, const PthreadCondattr* attr, const char* name); +int APS5_VABI scePthreadCondDestroy(PthreadCond* cond); +int APS5_VABI scePthreadCondWait(PthreadCond* cond, PthreadMutex* mutex); +int APS5_VABI scePthreadCondTimedwait(PthreadCond* cond, PthreadMutex* mutex, unsigned int usec); +int APS5_VABI pthread_cond_wait_nid_postfix(PthreadCond* cond, PthreadMutex* mutex); +int APS5_VABI pthread_cond_timedwait_nid_postfix(PthreadCond* cond, PthreadMutex* mutex, const KernelTimespec* abstime); +int APS5_VABI clock_gettime_nid_postfix(int clockId, KernelTimespec* tp); +} + +static constexpr int SCE_OK = 0; +static constexpr int SCE_KERNEL_ERROR_EPERM = static_cast(0x80020001); +static constexpr int SCE_KERNEL_ERROR_EINVAL = static_cast(0x80020016); +static constexpr int POSIX_EPERM = 1; +static constexpr int POSIX_EINVAL = 22; +static constexpr std::uintptr_t ADAPTIVE_INITIALIZER = 1; + +static void Require(bool value) { if (!value) std::abort(); } + +static int PosixTimed(PthreadCond* cond, PthreadMutex* mutex) { + KernelTimespec deadline{}; + Require(clock_gettime_nid_postfix(0, &deadline) == 0); + deadline.tv_sec += 60; + return pthread_cond_timedwait_nid_postfix(cond, mutex, &deadline); +} + +struct Wait { + int (*run)(PthreadCond*, PthreadMutex*); + int notOwned; + int destroyed; +}; + +static int SceWait(PthreadCond* cond, PthreadMutex* mutex) { return scePthreadCondWait(cond, mutex); } +static int SceTimed(PthreadCond* cond, PthreadMutex* mutex) { return scePthreadCondTimedwait(cond, mutex, 60000000); } +static int PosixWait(PthreadCond* cond, PthreadMutex* mutex) { return pthread_cond_wait_nid_postfix(cond, mutex); } + +struct Holder { + PthreadMutex mutex = nullptr; + std::atomic locked{false}; + std::atomic release{false}; +}; + +static void* APS5_VABI Hold(void* arg) { + auto& holder = *static_cast(arg); + Require(scePthreadMutexLock(&holder.mutex) == SCE_OK); + holder.locked.store(true); + while (!holder.release.load()) + std::this_thread::yield(); + Require(scePthreadMutexUnlock(&holder.mutex) == SCE_OK); + return nullptr; +} + +static void Check(const Wait& wait) { + PthreadCond cond = nullptr; + Require(scePthreadCondInit(&cond, nullptr, nullptr) == SCE_OK); + + PthreadMutex unlocked = nullptr; + Require(scePthreadMutexInit(&unlocked, nullptr, nullptr) == SCE_OK); + Require(wait.run(&cond, &unlocked) == wait.notOwned); + Require(scePthreadMutexLock(&unlocked) == SCE_OK); + Require(scePthreadMutexUnlock(&unlocked) == SCE_OK); + Require(scePthreadMutexDestroy(&unlocked) == SCE_OK); + + PthreadMutex uninitialized = nullptr; + Require(wait.run(&cond, &uninitialized) == wait.notOwned); + + PthreadMutex adaptive = reinterpret_cast(ADAPTIVE_INITIALIZER); + Require(wait.run(&cond, &adaptive) == wait.notOwned); + + PthreadMutex destroyed = nullptr; + Require(scePthreadMutexInit(&destroyed, nullptr, nullptr) == SCE_OK); + Require(scePthreadMutexDestroy(&destroyed) == SCE_OK); + Require(wait.run(&cond, &destroyed) == wait.destroyed); + + Holder holder; + Require(scePthreadMutexInit(&holder.mutex, nullptr, nullptr) == SCE_OK); + Pthread thread = nullptr; + Require(scePthreadCreate(&thread, nullptr, Hold, &holder, nullptr) == SCE_OK); + while (!holder.locked.load()) + std::this_thread::yield(); + Require(wait.run(&cond, &holder.mutex) == wait.notOwned); + holder.release.store(true); + Require(scePthreadJoin(thread, nullptr) == SCE_OK); + Require(scePthreadMutexDestroy(&holder.mutex) == SCE_OK); + + Require(scePthreadCondDestroy(&cond) == SCE_OK); +} + +int main() { + Check({SceWait, SCE_KERNEL_ERROR_EPERM, SCE_KERNEL_ERROR_EINVAL}); + Check({SceTimed, SCE_KERNEL_ERROR_EPERM, SCE_KERNEL_ERROR_EINVAL}); + Check({PosixWait, POSIX_EPERM, POSIX_EINVAL}); + Check({PosixTimed, POSIX_EPERM, POSIX_EINVAL}); +} From 324be50c64bdcbce423f00333cd4971988b02d6b Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 22:57:48 +0000 Subject: [PATCH 255/460] docs(libkernel): record condition wait errors --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 8c103b8c8..21a722572 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -150,7 +150,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceKernelSyncOnAddressWait](../../core/libs/prx/libkernel/SyncOnAddress/src/SyncOnAddress.cpp) (libkernel) - the only known caller passes a null timeout and a name string as the fourth argument; the timeout is assumed to point to microseconds like the other kernel waits, and the name is ignored - [sceKernelSyncOnAddressWait8 / sceKernelSyncOnAddressWait16 / sceKernelSyncOnAddressWait32 / sceKernelSyncOnAddressWait64](../../core/libs/prx/libkernel/SyncOnAddress/src/SyncOnAddress.cpp) (libkernel) - libc's `__std_atomic_wait_direct_8/16/32/64` pass a microsecond timeout or null and ignore the result; the by-value comparand, return values and the 2- and 8-byte alignment of Wait16 and Wait64 are assumed - [scePthreadRwlockDestroy, pthread_rwlock_destroy](../../core/libs/prx/libkernel/Pthread/src/Rwlock.cpp) (libkernel) - a rwlock held for reading or writing returns `EBUSY` and keeps its handle, as [KytyPS5](https://github.com/KytyPS5/KytyPS5) does; FreeBSD's [libthr](https://github.com/freebsd/freebsd-src/blob/main/lib/libthr/thread/thr_rwlock.c) and shadPS4 free it without checking, and the console's answer is unverified. Destroying a rwlock while another thread is acquiring it is not guarded -- [scePthreadCondWait / scePthreadCondTimedwait](../../core/libs/prx/libkernel/Pthread/src/Cond.cpp) (libkernel) - a recursive mutex locked several times is released completely for the wait and relocked to the same depth afterwards, as FreeBSD libthr does (`_mutex_cv_unlock`, `_mutex_cv_lock`); unverified on hardware +- [scePthreadCondWait / scePthreadCondTimedwait](../../core/libs/prx/libkernel/Pthread/src/Cond.cpp) (libkernel) - before waiting, FreeBSD libthr's [`cond_wait_common`](https://github.com/freebsd/freebsd-src/blob/main/lib/libthr/thread/thr_cond.c#L1992-L2029) checks ownership through [`_mutex_owned`](https://github.com/freebsd/freebsd-src/blob/main/lib/libthr/thread/thr_mutex.c#L1166-L1178): an unowned mutex returns `EPERM`, while its destroyed marker returns `EINVAL`. An untouched adaptive static initializer (slot value 1) therefore returns `EPERM`; the test covers this, an uninitialized slot, a mutex owned by another thread, and the destroyed marker for both timed and untimed sce and POSIX waits. A recursive mutex locked several times is released completely for the wait and relocked to the same depth afterwards, as FreeBSD libthr does (`_mutex_cv_unlock`, `_mutex_cv_lock`). These FreeBSD-derived behaviors remain unverified on console hardware - [sceKernelAioSubmitReadCommandsMultiple / sceKernelAioSubmitWriteCommandsMultiple / sceKernelAioWaitRequests / sceKernelAioPollRequests / sceKernelAioCancelRequest / sceKernelAioCancelRequests / sceKernelAioDeleteRequests](../../core/libs/prx/libkernel/Aio/src/Aio.cpp) (libkernel) - behaviour from shadPS4 (one id per request for the Multiple submits, cancel turns any request aborted and id 0 reports processing, wait mode 2 returns once one request completed, poll copies every state without waiting); the 128 requests or ids per batch limit is from SharpProspero, with an unknown error, so larger batches throw; wait modes other than 1 (and) and 2 (or) throw; null pointers, negative counts and invalid ids return the codes the single-request siblings use, checked before any state is written - [sceKernelGetAvailableCpumask](../../core/libs/prx/libkernel/Pthread/src/Thread.cpp) (libkernel) - returns the default thread affinity, 0x1FFF (CPUs 0 to 12), as KytyPS5 does. Titles pin threads within it (PPSA02664 passes 0x1FFB, Hades II PPSA36082 0x3 and 0x3F); whether CPU 13 is available to a title is not confirmed - [sceKernelGetCurrentCpu](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - guest affinities are not applied to host threads, so the host CPU index is mapped into the calling thread's affinity clipped to `sceKernelGetAvailableCpumask` (the available mask when they do not overlap): host CPU n reports the (n modulo the CPU count)-th CPU of that mask, so a thread pinned to one CPU reports that CPU, as a FreeBSD `cpuset` guarantees on the console. Which CPU the console reports for a thread allowed on several is not known; shadPS4 returns 0 From 9b7535c6a4305800812dbf12ec92b1dd31414f94 Mon Sep 17 00:00:00 2001 From: kjhq Date: Wed, 7 Oct 2026 10:38:09 +0000 Subject: [PATCH 256/460] feat(libkernel): implement read and write events for guest sockets sceKernelAddReadEvent/DeleteReadEvent and sceKernelAddWriteEvent/ DeleteWriteEvent register EVFILT_READ and EVFILT_WRITE events on guest sockets. The equeue polls the host socket for readiness on each wait and reports the readable byte count or the send buffer size in data, with EV_EOF on hang-up or error. --- core/libs/prx/libkernel/CMakeLists.txt | 10 ++ core/libs/prx/libkernel/Equeue/Equeue.cpp | 79 ++++++++++++- core/libs/prx/libkernel/Equeue/Equeue.hpp | 6 + .../Socket/include/SocketRuntime.hpp | 1 + core/libs/prx/libkernel/Socket/src/Udp.cpp | 33 ++++++ core/libs/tests/GuestSocketEvents.cpp | 105 ++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 7 files changed, 232 insertions(+), 3 deletions(-) create mode 100644 core/libs/tests/GuestSocketEvents.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index 4961439df..6f5ccdd85 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -726,6 +726,16 @@ if(BUILD_TESTING) set_tests_properties(guest_dup PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_socket_event_tests ${LIBS_INCLUDE_DIR}/tests/GuestSocketEvents.cpp) + target_include_directories(guest_socket_event_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_socket_event_tests PRIVATE libkernel libc) + configure_windows_unwind(guest_socket_event_tests) + add_test(NAME guest_socket_events COMMAND guest_socket_event_tests) + set_tests_properties(guest_socket_events PROPERTIES TIMEOUT 20) + if(WIN32) + set_tests_properties(guest_socket_events PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_elf_phdr_match_tests ${LIBS_INCLUDE_DIR}/tests/GuestElfPhdrMatch.cpp) target_include_directories(guest_elf_phdr_match_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_elf_phdr_match_tests PRIVATE libkernel libc) diff --git a/core/libs/prx/libkernel/Equeue/Equeue.cpp b/core/libs/prx/libkernel/Equeue/Equeue.cpp index b8d41b762..1cd163ec0 100644 --- a/core/libs/prx/libkernel/Equeue/Equeue.cpp +++ b/core/libs/prx/libkernel/Equeue/Equeue.cpp @@ -1,5 +1,7 @@ #include "Equeue.hpp" #include "prx/libkernel/Time/include/Time.hpp" +#include "prx/libc/include/General.hpp" +#include "prx/libkernel/Socket/include/SocketRuntime.hpp" #include #include @@ -11,6 +13,7 @@ static std::unordered_map g_equeues; static std::mutex g_equeueMutex; static uint64_t g_nextEqueue = 1; +static constexpr std::uint64_t PollIntervalNs = 1000000; extern "C" { @@ -75,12 +78,24 @@ bool KernelEqueuePrivate::NextTimerWaitMicros(uint64_t nowNs, uint32_t* out) con return true; } +bool KernelEqueuePrivate::PollEvents() { + bool polled = false; + for (auto& ev : m_events) { + if (ev.filter.pollFunc != nullptr) { + ev.triggered = ev.filter.pollFunc(&ev); + polled = true; + } + } + return polled; +} + int KernelEqueuePrivate::GetTriggeredEvents(KernelEvent* ev, int num) { std::unique_lock lock(m_mutex); if (m_closed) { return SCE_KERNEL_ERROR_EBADF; } TriggerExpiredTimers(MonotonicNs()); + PollEvents(); int ret = 0; for (auto it = m_events.begin(); it != m_events.end();) { auto& e = *it; @@ -123,6 +138,7 @@ int KernelEqueuePrivate::WaitForEvents(KernelEvent* ev, int num, uint32_t micros const std::uint64_t deadline = TimedWait::DeadlineNanos(micros); for (;;) { TriggerExpiredTimers(MonotonicNs()); + const bool polling = PollEvents(); int ret = 0; for (auto it = m_events.begin(); it != m_events.end() && ret < num;) { auto& e = *it; @@ -159,8 +175,14 @@ int KernelEqueuePrivate::WaitForEvents(KernelEvent* ev, int num, uint32_t micros } if (micros == 0) { uint32_t timerWait = 0; - if (NextTimerWaitMicros(MonotonicNs(), &timerWait)) { - m_cond.WaitUntil(lock, TimedWait::NowNanos() + static_cast(timerWait) * 1000ULL); + const bool hasTimer = NextTimerWaitMicros(MonotonicNs(), &timerWait); + if (hasTimer || polling) { + const std::uint64_t now = TimedWait::NowNanos(); + std::uint64_t wake = hasTimer ? now + static_cast(timerWait) * 1000ULL : now + PollIntervalNs; + if (polling) { + wake = std::min(wake, now + PollIntervalNs); + } + m_cond.WaitUntil(lock, wake); } else { m_cond.Wait(lock); } @@ -172,7 +194,11 @@ int KernelEqueuePrivate::WaitForEvents(KernelEvent* ev, int num, uint32_t micros return 0; } const std::uint64_t timerDeadline = now + static_cast(timerWait) * 1000ULL; - m_cond.WaitUntil(lock, hasTimer ? std::min(deadline, timerDeadline) : deadline); + std::uint64_t wake = hasTimer ? std::min(deadline, timerDeadline) : deadline; + if (polling) { + wake = std::min(wake, now + PollIntervalNs); + } + m_cond.WaitUntil(lock, wake); } } } @@ -481,4 +507,51 @@ int APS5_VABI sceKernelDeleteAmprSystemEvent(KernelEqueue eq, int id) { return sceKernelDeleteAmprEvent(eq, id); } +static int AddDescriptorEvent(KernelEqueue eq, int fd, std::size_t size, void* udata, int16_t filter) { + if (fd < GuestSockets::FirstDescriptor) { + NotImplemented_nid_no_patch(filter == EVFILT_READ ? "sceKernelAddReadEvent: non-socket descriptor" : "sceKernelAddWriteEvent: non-socket descriptor"); + } + if (size > 1) { + NotImplemented_nid_no_patch(filter == EVFILT_READ ? "sceKernelAddReadEvent: low watermark" : "sceKernelAddWriteEvent: low watermark"); + } + if (!GuestSockets::IsOpen(fd)) { + return SCE_KERNEL_ERROR_EBADF; + } + KernelEqueueEvent event{}; + event.event.ident = static_cast(fd); + event.event.filter = filter; + event.event.flags = EV_ADD; + event.event.udata = udata; + event.filter.pollFunc = [](KernelEqueueEvent* e) { + std::int64_t data = 0; + bool eof = false; + if (!GuestSockets::Ready(static_cast(e->event.ident), e->event.filter == EVFILT_WRITE, &data, &eof)) { + return false; + } + e->event.data = static_cast(data); + e->event.flags = static_cast(eof ? (e->event.flags | EV_EOF) : (e->event.flags & ~EV_EOF)); + return true; + }; + event.filter.resetFunc = [](KernelEqueueEvent* e) { + e->triggered = false; + }; + return EqueueAddEvent_nid_postfix(eq, event); +} + +int APS5_VABI sceKernelAddReadEvent(KernelEqueue eq, int fd, std::size_t size, void* udata) { + return AddDescriptorEvent(eq, fd, size, udata, EVFILT_READ); +} + +int APS5_VABI sceKernelDeleteReadEvent(KernelEqueue eq, int fd) { + return EqueueDeleteEvent_nid_postfix(eq, static_cast(fd), EVFILT_READ); +} + +int APS5_VABI sceKernelAddWriteEvent(KernelEqueue eq, int fd, std::size_t size, void* udata) { + return AddDescriptorEvent(eq, fd, size, udata, EVFILT_WRITE); +} + +int APS5_VABI sceKernelDeleteWriteEvent(KernelEqueue eq, int fd) { + return EqueueDeleteEvent_nid_postfix(eq, static_cast(fd), EVFILT_WRITE); +} + } diff --git a/core/libs/prx/libkernel/Equeue/Equeue.hpp b/core/libs/prx/libkernel/Equeue/Equeue.hpp index dd3a47706..dd7dea008 100644 --- a/core/libs/prx/libkernel/Equeue/Equeue.hpp +++ b/core/libs/prx/libkernel/Equeue/Equeue.hpp @@ -14,6 +14,8 @@ #include "prx/libkernel/KernelErrors.hpp" #include "prx/libkernel/Time/include/TimedWait.hpp" +static constexpr int16_t EVFILT_READ = -1; +static constexpr int16_t EVFILT_WRITE = -2; static constexpr int16_t EVFILT_TIMER = -7; static constexpr int16_t EVFILT_USER = -11; static constexpr int16_t EVFILT_VIDEO_OUT = -13; @@ -24,6 +26,7 @@ static constexpr uint16_t EV_ADD = 0x0001; static constexpr uint16_t EV_ONESHOT = 0x0010; static constexpr uint16_t EV_CLEAR = 0x0020; static constexpr uint16_t EV_ERROR = 0x4000; +static constexpr uint16_t EV_EOF = 0x8000; static constexpr int EQUEUE_OK = 0; @@ -32,6 +35,7 @@ struct KernelEqueueEvent; using EqueueTriggerFunc = void (*)(KernelEqueueEvent* event, void* triggerData); using EqueueResetFunc = void (*)(KernelEqueueEvent* event); using EqueueDeleteFunc = void (*)(KernelEqueue eq, KernelEqueueEvent* event); +using EqueuePollFunc = bool (*)(KernelEqueueEvent* event); struct KernelFilter { void* data = nullptr; @@ -39,6 +43,7 @@ struct KernelFilter { EqueueTriggerFunc triggerFunc = nullptr; EqueueResetFunc resetFunc = nullptr; EqueueDeleteFunc deleteEventFunc = nullptr; + EqueuePollFunc pollFunc = nullptr; }; struct KernelEqueueEvent { @@ -72,6 +77,7 @@ public: private: void TriggerExpiredTimers(uint64_t nowNs); bool NextTimerWaitMicros(uint64_t nowNs, uint32_t* out) const; + bool PollEvents(); std::list m_events; std::mutex m_mutex; diff --git a/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp b/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp index db775297c..9a8640add 100644 --- a/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp +++ b/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp @@ -10,6 +10,7 @@ bool IsOpen(int descriptor); int Family(int descriptor); std::int64_t Read(int descriptor, void* buffer, std::size_t length); std::int64_t Write(int descriptor, const void* buffer, std::size_t length); +bool Ready(int descriptor, bool write, std::int64_t* data, bool* eof); } extern "C" bool GuestSocketIsOpen_nid_no_patch(int descriptor); diff --git a/core/libs/prx/libkernel/Socket/src/Udp.cpp b/core/libs/prx/libkernel/Socket/src/Udp.cpp index e87ecd8f5..8cb6d12cc 100644 --- a/core/libs/prx/libkernel/Socket/src/Udp.cpp +++ b/core/libs/prx/libkernel/Socket/src/Udp.cpp @@ -349,6 +349,39 @@ int UnnamedAddress(void* address, std::uint32_t* length) { } } +bool GuestSockets::Ready(int descriptor, bool write, std::int64_t* data, bool* eof) { + std::shared_ptr socket; + { + std::lock_guard lock(socketsMutex); + const auto found = sockets.find(descriptor); + if (found == sockets.end()) return false; + socket = found->second; + } +#ifdef _WIN32 + WSAPOLLFD entry{socket->value, static_cast(write ? POLLWRNORM : POLLRDNORM), 0}; + if (WSAPoll(&entry, 1, 0) <= 0) return false; +#else + pollfd entry{socket->value, static_cast(write ? POLLOUT : POLLIN), 0}; + if (::poll(&entry, 1, 0) <= 0) return false; +#endif + *eof = (entry.revents & (POLLHUP | POLLERR)) != 0; + *data = 0; + if (write) { + int size = 0; + socklen_t length = sizeof(size); + if (getsockopt(socket->value, SOL_SOCKET, SO_SNDBUF, reinterpret_cast(&size), &length) == 0) *data = size; + } else { +#ifdef _WIN32 + unsigned long available = 0; + if (ioctlsocket(socket->value, FIONREAD, &available) == 0) *data = static_cast(available); +#else + int available = 0; + if (::ioctl(socket->value, FIONREAD, &available) == 0) *data = available; +#endif + } + return true; +} + extern "C" { int APS5_VABI fcntl_nid_postfix(int descriptor, int command, ...) { const auto socket = Lookup(descriptor); diff --git a/core/libs/tests/GuestSocketEvents.cpp b/core/libs/tests/GuestSocketEvents.cpp new file mode 100644 index 000000000..011577219 --- /dev/null +++ b/core/libs/tests/GuestSocketEvents.cpp @@ -0,0 +1,105 @@ +#include "SceTypes.hpp" +#include +#include +#include +#include + +extern "C" { +int APS5_VABI socket_nid_postfix(int, int, int); +int APS5_VABI bind_nid_postfix(int, const void*, std::uint32_t); +int APS5_VABI listen_nid_postfix(int, int); +int APS5_VABI getsockname_nid_postfix(int, void*, std::uint32_t*); +int APS5_VABI connect_nid_postfix(int, const void*, std::uint32_t); +int APS5_VABI accept_nid_postfix(int, void*, std::uint32_t*); +std::int64_t APS5_VABI send_nid_postfix(int, const void*, std::uint64_t, int); +std::int64_t APS5_VABI recv_nid_postfix(int, void*, std::uint64_t, int); +int APS5_VABI close_nid_postfix(int); +int APS5_VABI sceKernelCreateEqueue(KernelEqueue* eq, const char* name); +int APS5_VABI sceKernelDeleteEqueue(KernelEqueue eq); +int APS5_VABI sceKernelWaitEqueue(KernelEqueue eq, KernelEvent* ev, int num, int* out, const KernelUseconds* timo); +int APS5_VABI sceKernelAddReadEvent(KernelEqueue eq, int fd, std::size_t size, void* udata); +int APS5_VABI sceKernelDeleteReadEvent(KernelEqueue eq, int fd); +int APS5_VABI sceKernelAddWriteEvent(KernelEqueue eq, int fd, std::size_t size, void* udata); +int APS5_VABI sceKernelDeleteWriteEvent(KernelEqueue eq, int fd); +} + +static constexpr int SCE_OK = 0; +static constexpr int SCE_KERNEL_ERROR_ENOENT = static_cast(0x80020002); +static constexpr int SCE_KERNEL_ERROR_EBADF = static_cast(0x80020009); +static constexpr int SCE_KERNEL_ERROR_ETIMEDOUT = static_cast(0x8002003c); +static constexpr int EVFILT_READ = -1; +static constexpr int EVFILT_WRITE = -2; +static constexpr std::uint16_t EV_EOF = 0x8000; + +static void Require(bool value) { if (!value) std::abort(); } + +static bool Rejects(int (APS5_VABI *add)(KernelEqueue, int, std::size_t, void*), KernelEqueue eq, int fd, std::size_t size) { + try { + add(eq, fd, size, nullptr); + } catch (const std::exception&) { + return true; + } + return false; +} + +int main() { + const int listener = socket_nid_postfix(2, 1, 6); + Require(listener >= 0); + std::array address{16, 2, 0, 0, 127, 0, 0, 1}; + Require(bind_nid_postfix(listener, address.data(), address.size()) == 0); + Require(listen_nid_postfix(listener, 4) == 0); + std::uint32_t addressSize = address.size(); + Require(getsockname_nid_postfix(listener, address.data(), &addressSize) == 0); + + KernelEqueue eq = 0; + Require(sceKernelCreateEqueue(&eq, "sockets") == SCE_OK); + const KernelUseconds shortTimeout = 20000; + const KernelUseconds timeout = 1000000; + int marker = 0; + int count = 0; + std::array events{}; + + Require(sceKernelAddReadEvent(eq, listener, 1, &marker) == SCE_OK); + Require(sceKernelWaitEqueue(eq, events.data(), events.size(), &count, &shortTimeout) == SCE_KERNEL_ERROR_ETIMEDOUT); + Require(count == 0); + + const int client = socket_nid_postfix(2, 1, 0); + Require(client >= 0); + Require(connect_nid_postfix(client, address.data(), address.size()) == 0); + Require(sceKernelWaitEqueue(eq, events.data(), events.size(), &count, &timeout) == SCE_OK); + Require(count == 1); + Require(events[0].ident == static_cast(listener)); + Require(events[0].filter == EVFILT_READ); + Require(events[0].udata == &marker); + const int accepted = accept_nid_postfix(listener, nullptr, nullptr); + Require(accepted >= 0); + Require(sceKernelDeleteReadEvent(eq, listener) == SCE_OK); + Require(sceKernelDeleteReadEvent(eq, listener) == SCE_KERNEL_ERROR_ENOENT); + + Require(sceKernelAddWriteEvent(eq, client, 0, nullptr) == SCE_OK); + Require(sceKernelWaitEqueue(eq, events.data(), events.size(), &count, &timeout) == SCE_OK); + Require(count == 1 && events[0].filter == EVFILT_WRITE && events[0].data > 0); + Require(sceKernelDeleteWriteEvent(eq, client) == SCE_OK); + + Require(sceKernelAddReadEvent(eq, accepted, 1, nullptr) == SCE_OK); + const char message[] = "ready"; + Require(send_nid_postfix(client, message, sizeof(message), 0) == sizeof(message)); + Require(sceKernelWaitEqueue(eq, events.data(), events.size(), &count, &timeout) == SCE_OK); + Require(count == 1 && events[0].ident == static_cast(accepted)); + Require(events[0].data == sizeof(message) && (events[0].flags & EV_EOF) == 0); + char received[sizeof(message)]{}; + Require(recv_nid_postfix(accepted, received, sizeof(received), 0) == sizeof(received)); + Require(sceKernelWaitEqueue(eq, events.data(), events.size(), &count, &shortTimeout) == SCE_KERNEL_ERROR_ETIMEDOUT); + + Require(close_nid_postfix(client) == 0); + Require(sceKernelWaitEqueue(eq, events.data(), events.size(), &count, &timeout) == SCE_OK); + Require(count == 1 && events[0].data == 0); + + Require(sceKernelAddReadEvent(eq, client, 1, nullptr) == SCE_KERNEL_ERROR_EBADF); + Require(Rejects(sceKernelAddReadEvent, eq, 0, 1)); + Require(Rejects(sceKernelAddWriteEvent, eq, accepted, 64)); + + Require(close_nid_postfix(accepted) == 0); + Require(close_nid_postfix(listener) == 0); + Require(sceKernelDeleteEqueue(eq) == SCE_OK); +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 8c103b8c8..2fe26defd 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -332,6 +332,7 @@ Throughout the project, every function at every stage either **does exactly what - [Pending storage-image results](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) are stored when something reads their memory, not when the GPU work that wrote them completes. A title that waits for that work, frees the image and reuses its memory can have the new data overwritten by the late store (#1970). `APS5_STORE_AT_FLIP=1` stores every pending result at each flip; storing them at the title's GPU waits (labels, fences) is unimplemented. - Windows host-pointer imports of the guest mapping itself are compared instead of write-watched: a driver can mark imported pages dirty after the synchronous import window. Shared memory imported through a separate writable alias retains protection-based tracking on the guest views; driver access to the import alias does not disarm those views. [GuestMemory](../../core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp) excludes every overlapping 64 KiB tracker block while its backing is retained. A full block replaced by fresh private backing regains tracking and gets a new generation; protection changes, partial replacements and shared remappings do not restore it. This preserves CPU edits through byte comparisons but disables snapshot reuse and unit shadows for excluded blocks; restoring tracking on import retirement without replacing the backing and measuring the comparison cost remain unimplemented. - [shm_open](../../core/libs/prx/libkernel/File/src/SharedMemory.cpp) (libkernel) creates only anonymous objects (`SHM_ANON`), each backed by a host anonymous file: a `memfd` on Linux, a delete-on-close temporary file on Windows. The flag checks follow FreeBSD 11's `kern_shm_open`; the mode is ignored. Named objects throw. The descriptor cannot be mapped yet: `mmap` accepts only private anonymous mappings and fails a descriptor mapping with `EOPNOTSUPP`, so the known caller (the bundled Mesa in PPSA59322, which truncates the object and maps it `MAP_SHARED`) gets that failure +- [sceKernelAddReadEvent and sceKernelAddWriteEvent](../../core/libs/prx/libkernel/Equeue/Equeue.cpp) (libkernel) poll the host socket instead of being woken by it: while such an event is registered, every wait on its equeue checks the socket at least once per millisecond, so readiness is reported up to 1 ms late and an idle wait wakes the thread 1000 times a second. Only guest sockets are supported; other descriptors and a low watermark above 1 throw - [sceNetSendmsg and sceNetRecvmsg](../../core/libs/prx/libSceNet/Export.cpp) (libSceNet) copy the scatter/gather list through one host buffer and accept only the flags `sceNetSend`/`sceNetRecv` accept. Sending control data throws; a receive never returns control data (`msg_controllen` 0), since no socket option that queues it is supported - [Additional content](../../core/libs/prx/libSceAppContent/Export.cpp) (libSceAppContent) is not installed or mounted: `sceAppContentAddcontMount` answers `NOT_FOUND` for every entitlement label and `sceAppContentAddcontUnmount` for every mount point, as no add-content mount is ever handed out; `sceAppContentGetAddcontInfo` answers `DRM_NO_ENTITLEMENT` for every label and `sceAppContentGetAddcontInfoList` reports 0 entries - [sceKernelPread, sceKernelPwrite, sceKernelPreadv, sceKernelPwritev](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) on Windows transfer at the given offset with an overlapped `ReadFile`/`WriteFile` and restore the descriptor's file pointer afterwards, so a concurrent `read`/`write` on the same descriptor can see the moved pointer in between, unlike on the console. Handles that are not disk files return `ESPIPE`. `readv` on a pipe or console handle stops after the first iovec read, a short read the console is assumed to allow; each iovec is transferred separately, so a failure after the first transfer returns the bytes already done From 0abe0b832babdf1a642431798004b7e21c2132f4 Mon Sep 17 00:00:00 2001 From: kjhq Date: Wed, 7 Oct 2026 12:13:39 +0000 Subject: [PATCH 257/460] feat(libkernel): wake socket read and write events instead of polling A wait on an equeue that holds socket events now blocks in poll/WSAPoll on those sockets plus a per-thread loopback wake socket, instead of rechecking every millisecond. Adding, triggering or deleting an event and closing the equeue wake the blocked waiters. Closing a socket removes its read and write events, as on FreeBSD. A read event also reports EV_EOF once the peer has shut down its sending side. --- core/libs/prx/libkernel/Equeue/Equeue.cpp | 69 +++++++++-- core/libs/prx/libkernel/Equeue/Equeue.hpp | 6 + .../Socket/include/SocketRuntime.hpp | 8 ++ core/libs/prx/libkernel/Socket/src/Udp.cpp | 108 +++++++++++++++++- core/libs/tests/GuestSocketEvents.cpp | 38 +++++- docs/dev/TechnicalDebt.md | 2 +- 6 files changed, 216 insertions(+), 15 deletions(-) diff --git a/core/libs/prx/libkernel/Equeue/Equeue.cpp b/core/libs/prx/libkernel/Equeue/Equeue.cpp index 1cd163ec0..b3b130ba0 100644 --- a/core/libs/prx/libkernel/Equeue/Equeue.cpp +++ b/core/libs/prx/libkernel/Equeue/Equeue.cpp @@ -13,7 +13,6 @@ static std::unordered_map g_equeues; static std::mutex g_equeueMutex; static uint64_t g_nextEqueue = 1; -static constexpr std::uint64_t PollIntervalNs = 1000000; extern "C" { @@ -45,6 +44,7 @@ void KernelEqueuePrivate::Close() { } m_events.clear(); m_cond.NotifyAll(); + WakePollers(); } void KernelEqueuePrivate::TriggerExpiredTimers(uint64_t nowNs) { @@ -89,6 +89,30 @@ bool KernelEqueuePrivate::PollEvents() { return polled; } +void KernelEqueuePrivate::WakePollers() { + for (const auto waker : m_pollers) { + GuestSockets::Wake(waker); + } +} + +void KernelEqueuePrivate::WaitForDescriptors(std::unique_lock& lock, std::uint64_t deadlineNanos) { + std::vector interests; + for (const auto& e : m_events) { + if (e.filter.pollFunc != nullptr) { + interests.push_back({static_cast(e.event.ident), e.event.filter == EVFILT_WRITE}); + } + } + const auto waker = GuestSockets::CurrentWaker(); + m_pollers.push_back(waker); + lock.unlock(); + const bool waited = GuestSockets::WaitAny(interests, deadlineNanos); + lock.lock(); + m_pollers.erase(std::find(m_pollers.begin(), m_pollers.end(), waker)); + if (!waited) { + m_cond.WaitUntil(lock, deadlineNanos); + } +} + int KernelEqueuePrivate::GetTriggeredEvents(KernelEvent* ev, int num) { std::unique_lock lock(m_mutex); if (m_closed) { @@ -176,12 +200,10 @@ int KernelEqueuePrivate::WaitForEvents(KernelEvent* ev, int num, uint32_t micros if (micros == 0) { uint32_t timerWait = 0; const bool hasTimer = NextTimerWaitMicros(MonotonicNs(), &timerWait); - if (hasTimer || polling) { - const std::uint64_t now = TimedWait::NowNanos(); - std::uint64_t wake = hasTimer ? now + static_cast(timerWait) * 1000ULL : now + PollIntervalNs; - if (polling) { - wake = std::min(wake, now + PollIntervalNs); - } + const std::uint64_t wake = hasTimer ? TimedWait::NowNanos() + static_cast(timerWait) * 1000ULL : 0; + if (polling) { + WaitForDescriptors(lock, wake); + } else if (hasTimer) { m_cond.WaitUntil(lock, wake); } else { m_cond.Wait(lock); @@ -194,11 +216,12 @@ int KernelEqueuePrivate::WaitForEvents(KernelEvent* ev, int num, uint32_t micros return 0; } const std::uint64_t timerDeadline = now + static_cast(timerWait) * 1000ULL; - std::uint64_t wake = hasTimer ? std::min(deadline, timerDeadline) : deadline; + const std::uint64_t wake = hasTimer ? std::min(deadline, timerDeadline) : deadline; if (polling) { - wake = std::min(wake, now + PollIntervalNs); + WaitForDescriptors(lock, wake); + } else { + m_cond.WaitUntil(lock, wake); } - m_cond.WaitUntil(lock, wake); } } } @@ -227,6 +250,7 @@ int KernelEqueuePrivate::AddEvent(const KernelEqueueEvent& event) { m_events.push_back(event); } m_cond.NotifyOne(); + WakePollers(); return EQUEUE_OK; } @@ -249,6 +273,7 @@ int KernelEqueuePrivate::TriggerEvent(uintptr_t ident, int16_t filter, void* tri it->triggered = true; } m_cond.NotifyOne(); + WakePollers(); return EQUEUE_OK; } @@ -270,9 +295,20 @@ int KernelEqueuePrivate::DeleteEvent(uintptr_t ident, int16_t filter) { it->filter.deleteEventFunc(m_handle, &*it); } m_events.erase(it); + WakePollers(); return EQUEUE_OK; } +void KernelEqueuePrivate::RemoveDescriptorEvents(uintptr_t ident) { + std::unique_lock lock(m_mutex); + const auto removed = std::erase_if(m_events, [ident](const auto& e) { + return e.event.ident == ident && (e.event.filter == EVFILT_READ || e.event.filter == EVFILT_WRITE); + }); + if (removed != 0) { + WakePollers(); + } +} + KernelEqueueRef EqueuePin_nid_postfix(KernelEqueue eq) { if (eq == 0) { return {}; @@ -555,3 +591,16 @@ int APS5_VABI sceKernelDeleteWriteEvent(KernelEqueue eq, int fd) { } } + +void EqueueDescriptorClosed(int descriptor) { + std::vector owners; + { + std::unique_lock lock(g_equeueMutex); + for (const auto& [handle, owner] : g_equeues) { + owners.push_back(owner); + } + } + for (const auto& owner : owners) { + owner->RemoveDescriptorEvents(static_cast(descriptor)); + } +} diff --git a/core/libs/prx/libkernel/Equeue/Equeue.hpp b/core/libs/prx/libkernel/Equeue/Equeue.hpp index dd7dea008..c63f505fb 100644 --- a/core/libs/prx/libkernel/Equeue/Equeue.hpp +++ b/core/libs/prx/libkernel/Equeue/Equeue.hpp @@ -8,6 +8,7 @@ #include #include #include +#include #include "SceTypes.hpp" #include "prx/libc/include/general/VabiMacros.hpp" @@ -71,6 +72,7 @@ public: int DeleteEvent(uintptr_t ident, int16_t filter); int GetTriggeredEvents(KernelEvent* ev, int num); int WaitForEvents(KernelEvent* ev, int num, uint32_t micros); + void RemoveDescriptorEvents(uintptr_t ident); void Close(); static uint64_t MonotonicNs(); @@ -78,8 +80,11 @@ private: void TriggerExpiredTimers(uint64_t nowNs); bool NextTimerWaitMicros(uint64_t nowNs, uint32_t* out) const; bool PollEvents(); + void WakePollers(); + void WaitForDescriptors(std::unique_lock& lock, std::uint64_t deadlineNanos); std::list m_events; + std::vector m_pollers; std::mutex m_mutex; TimedWait::Condition m_cond; std::string m_name; @@ -88,6 +93,7 @@ private: }; using KernelEqueueRef = std::shared_ptr; +void EqueueDescriptorClosed(int descriptor); extern "C" { KernelEqueueRef EqueuePin_nid_postfix(KernelEqueue eq); diff --git a/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp b/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp index 9a8640add..0703293dc 100644 --- a/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp +++ b/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp @@ -1,6 +1,7 @@ #pragma once #include #include +#include namespace GuestSockets { constexpr int FirstDescriptor = 0x10000000; int Duplicate(int descriptor); @@ -11,6 +12,13 @@ int Family(int descriptor); std::int64_t Read(int descriptor, void* buffer, std::size_t length); std::int64_t Write(int descriptor, const void* buffer, std::size_t length); bool Ready(int descriptor, bool write, std::int64_t* data, bool* eof); +struct Interest { + int descriptor; + bool write; +}; +std::uintptr_t CurrentWaker(); +void Wake(std::uintptr_t waker); +bool WaitAny(const std::vector& interests, std::uint64_t deadlineNanos); } extern "C" bool GuestSocketIsOpen_nid_no_patch(int descriptor); diff --git a/core/libs/prx/libkernel/Socket/src/Udp.cpp b/core/libs/prx/libkernel/Socket/src/Udp.cpp index 8cb6d12cc..9ff5d7b33 100644 --- a/core/libs/prx/libkernel/Socket/src/Udp.cpp +++ b/core/libs/prx/libkernel/Socket/src/Udp.cpp @@ -5,6 +5,8 @@ #else #include #include +#include +#include #include #include #include @@ -13,6 +15,8 @@ #include "prx/libc/include/GuestArena.hpp" #include "prx/libc/include/general/VabiMacros.hpp" #include "prx/libkernel/Socket/include/SocketRuntime.hpp" +#include "prx/libkernel/Equeue/Equeue.hpp" +#include "prx/libkernel/Time/include/TimedWait.hpp" #include #include #include @@ -109,6 +113,43 @@ struct Socket { #endif } }; +struct Waker { + NativeSocket value = Invalid; + Waker() { + value = ::socket(AF_INET, SOCK_DGRAM, 0); + sockaddr_in address{}; + address.sin_family = AF_INET; + address.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + socklen_t size = sizeof(address); +#ifdef _WIN32 + u_long nonblocking = 1; + const bool configured = value != Invalid && ioctlsocket(value, FIONBIO, &nonblocking) == 0; +#else + const bool configured = value != Invalid && ::fcntl(value, F_SETFL, ::fcntl(value, F_GETFL) | O_NONBLOCK) == 0; +#endif + if (!configured || bind(value, reinterpret_cast(&address), size) != 0 || + getsockname(value, reinterpret_cast(&address), &size) != 0 || + connect(value, reinterpret_cast(&address), size) != 0) { + const int error = NativeError(); + Close(); + throw std::runtime_error("equeue waker: loopback socket setup failed, guest errno " + std::to_string(error)); + } + } + ~Waker() { Close(); } + void Close() { + if (value == Invalid) return; +#ifdef _WIN32 + closesocket(value); +#else + ::close(value); +#endif + value = Invalid; + } +}; +Waker& ThreadWaker() { + thread_local Waker waker; + return waker; +} std::mutex socketsMutex; std::map> sockets; int nextDescriptor = GuestSockets::FirstDescriptor; @@ -173,8 +214,68 @@ void GuestAddress(const sockaddr_storage& native, void* output, std::uint32_t* l } int GuestSockets::Close(int descriptor) { - std::lock_guard lock(socketsMutex); - return sockets.erase(descriptor) ? 0 : Fail(9); + { + std::lock_guard lock(socketsMutex); + if (!sockets.erase(descriptor)) return Fail(9); + } + EqueueDescriptorClosed(descriptor); + return 0; +} + +std::uintptr_t GuestSockets::CurrentWaker() { + return static_cast(ThreadWaker().value); +} + +void GuestSockets::Wake(std::uintptr_t waker) { + const char signal = 0; + if (send(static_cast(waker), &signal, 1, 0) < 0 && NativeError() != 35) + throw std::runtime_error("equeue waker: send failed, guest errno " + std::to_string(NativeError())); +} + +bool GuestSockets::WaitAny(const std::vector& interests, std::uint64_t deadlineNanos) { + const NativeSocket waker = ThreadWaker().value; + std::vector> held; +#ifdef _WIN32 + std::vector entries; + constexpr SHORT ReadMask = POLLRDNORM; + constexpr SHORT WriteMask = POLLWRNORM; +#else + std::vector entries; + constexpr short ReadMask = POLLIN; + constexpr short WriteMask = POLLOUT; +#endif + { + std::lock_guard lock(socketsMutex); + for (const auto& interest : interests) { + const auto found = sockets.find(interest.descriptor); + if (found == sockets.end()) continue; + held.push_back(found->second); + entries.push_back({found->second->value, interest.write ? WriteMask : ReadMask, 0}); + } + } + entries.push_back({waker, ReadMask, 0}); + std::uint64_t remaining = 0; + if (deadlineNanos != 0) { + const std::uint64_t now = TimedWait::NowNanos(); + if (now >= deadlineNanos) return true; + remaining = deadlineNanos - now; + } +#ifdef _WIN32 + INT timeout = -1; + if (deadlineNanos != 0) { + if (remaining < 1000000) return false; + timeout = static_cast(std::min(remaining / 1000000, INT_MAX)); + } + if (WSAPoll(entries.data(), static_cast(entries.size()), timeout) == SOCKET_ERROR) + throw std::runtime_error("equeue wait: WSAPoll failed, guest errno " + std::to_string(NativeError())); +#else + timespec timeout{static_cast(remaining / 1000000000), static_cast(remaining % 1000000000)}; + if (::ppoll(entries.data(), entries.size(), deadlineNanos != 0 ? &timeout : nullptr, nullptr) < 0 && errno != EINTR) + throw std::runtime_error("equeue wait: ppoll failed, guest errno " + std::to_string(NativeError())); +#endif + char drained[16]; + while (recv(waker, drained, sizeof(drained), 0) > 0) {} + return true; } bool GuestSockets::IsOpen(int descriptor) { @@ -361,8 +462,9 @@ bool GuestSockets::Ready(int descriptor, bool write, std::int64_t* data, bool* e WSAPOLLFD entry{socket->value, static_cast(write ? POLLWRNORM : POLLRDNORM), 0}; if (WSAPoll(&entry, 1, 0) <= 0) return false; #else - pollfd entry{socket->value, static_cast(write ? POLLOUT : POLLIN), 0}; + pollfd entry{socket->value, static_cast(write ? POLLOUT : (POLLIN | POLLRDHUP)), 0}; if (::poll(&entry, 1, 0) <= 0) return false; + if (!write && (entry.revents & POLLRDHUP)) entry.revents |= POLLHUP; #endif *eof = (entry.revents & (POLLHUP | POLLERR)) != 0; *data = 0; diff --git a/core/libs/tests/GuestSocketEvents.cpp b/core/libs/tests/GuestSocketEvents.cpp index 011577219..94bf0656d 100644 --- a/core/libs/tests/GuestSocketEvents.cpp +++ b/core/libs/tests/GuestSocketEvents.cpp @@ -1,8 +1,10 @@ #include "SceTypes.hpp" #include +#include #include #include #include +#include extern "C" { int APS5_VABI socket_nid_postfix(int, int, int); @@ -93,7 +95,41 @@ int main() { Require(close_nid_postfix(client) == 0); Require(sceKernelWaitEqueue(eq, events.data(), events.size(), &count, &timeout) == SCE_OK); - Require(count == 1 && events[0].data == 0); + Require(count == 1 && events[0].data == 0 && (events[0].flags & EV_EOF) != 0); + + KernelEqueue blocking = 0; + Require(sceKernelCreateEqueue(&blocking, "blocking") == SCE_OK); + const int writer = socket_nid_postfix(2, 1, 0); + Require(writer >= 0 && connect_nid_postfix(writer, address.data(), address.size()) == 0); + const int reader = accept_nid_postfix(listener, nullptr, nullptr); + Require(reader >= 0); + Require(sceKernelAddReadEvent(blocking, reader, 1, nullptr) == SCE_OK); + std::thread sender([&] { + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + Require(send_nid_postfix(writer, message, sizeof(message), 0) == sizeof(message)); + }); + const auto start = std::chrono::steady_clock::now(); + Require(sceKernelWaitEqueue(blocking, events.data(), events.size(), &count, nullptr) == SCE_OK); + sender.join(); + Require(std::chrono::steady_clock::now() - start >= std::chrono::milliseconds(40)); + Require(count == 1 && events[0].ident == static_cast(reader) && events[0].data == sizeof(message)); + Require(recv_nid_postfix(reader, received, sizeof(received), 0) == sizeof(received)); + + std::thread waiter([&] { + std::array woken{}; + int wokenCount = 0; + Require(sceKernelWaitEqueue(blocking, woken.data(), woken.size(), &wokenCount, nullptr) == SCE_OK); + Require(wokenCount == 1 && woken[0].filter == EVFILT_WRITE && woken[0].ident == static_cast(writer)); + }); + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + Require(sceKernelAddWriteEvent(blocking, writer, 0, nullptr) == SCE_OK); + waiter.join(); + Require(sceKernelDeleteWriteEvent(blocking, writer) == SCE_OK); + + Require(close_nid_postfix(reader) == 0); + Require(sceKernelDeleteReadEvent(blocking, reader) == SCE_KERNEL_ERROR_ENOENT); + Require(close_nid_postfix(writer) == 0); + Require(sceKernelDeleteEqueue(blocking) == SCE_OK); Require(sceKernelAddReadEvent(eq, client, 1, nullptr) == SCE_KERNEL_ERROR_EBADF); Require(Rejects(sceKernelAddReadEvent, eq, 0, 1)); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 2fe26defd..f06d91026 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -332,7 +332,7 @@ Throughout the project, every function at every stage either **does exactly what - [Pending storage-image results](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) are stored when something reads their memory, not when the GPU work that wrote them completes. A title that waits for that work, frees the image and reuses its memory can have the new data overwritten by the late store (#1970). `APS5_STORE_AT_FLIP=1` stores every pending result at each flip; storing them at the title's GPU waits (labels, fences) is unimplemented. - Windows host-pointer imports of the guest mapping itself are compared instead of write-watched: a driver can mark imported pages dirty after the synchronous import window. Shared memory imported through a separate writable alias retains protection-based tracking on the guest views; driver access to the import alias does not disarm those views. [GuestMemory](../../core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp) excludes every overlapping 64 KiB tracker block while its backing is retained. A full block replaced by fresh private backing regains tracking and gets a new generation; protection changes, partial replacements and shared remappings do not restore it. This preserves CPU edits through byte comparisons but disables snapshot reuse and unit shadows for excluded blocks; restoring tracking on import retirement without replacing the backing and measuring the comparison cost remain unimplemented. - [shm_open](../../core/libs/prx/libkernel/File/src/SharedMemory.cpp) (libkernel) creates only anonymous objects (`SHM_ANON`), each backed by a host anonymous file: a `memfd` on Linux, a delete-on-close temporary file on Windows. The flag checks follow FreeBSD 11's `kern_shm_open`; the mode is ignored. Named objects throw. The descriptor cannot be mapped yet: `mmap` accepts only private anonymous mappings and fails a descriptor mapping with `EOPNOTSUPP`, so the known caller (the bundled Mesa in PPSA59322, which truncates the object and maps it `MAP_SHARED`) gets that failure -- [sceKernelAddReadEvent and sceKernelAddWriteEvent](../../core/libs/prx/libkernel/Equeue/Equeue.cpp) (libkernel) poll the host socket instead of being woken by it: while such an event is registered, every wait on its equeue checks the socket at least once per millisecond, so readiness is reported up to 1 ms late and an idle wait wakes the thread 1000 times a second. Only guest sockets are supported; other descriptors and a low watermark above 1 throw +- [sceKernelAddReadEvent and sceKernelAddWriteEvent](../../core/libs/prx/libkernel/Equeue/Equeue.cpp) (libkernel): a write event reports the socket's send buffer size (`SO_SNDBUF`) in `data`, not the free space left in it, as Windows has no query for that space. Only guest sockets are supported; other descriptors and a low watermark above 1 throw - [sceNetSendmsg and sceNetRecvmsg](../../core/libs/prx/libSceNet/Export.cpp) (libSceNet) copy the scatter/gather list through one host buffer and accept only the flags `sceNetSend`/`sceNetRecv` accept. Sending control data throws; a receive never returns control data (`msg_controllen` 0), since no socket option that queues it is supported - [Additional content](../../core/libs/prx/libSceAppContent/Export.cpp) (libSceAppContent) is not installed or mounted: `sceAppContentAddcontMount` answers `NOT_FOUND` for every entitlement label and `sceAppContentAddcontUnmount` for every mount point, as no add-content mount is ever handed out; `sceAppContentGetAddcontInfo` answers `DRM_NO_ENTITLEMENT` for every label and `sceAppContentGetAddcontInfoList` reports 0 entries - [sceKernelPread, sceKernelPwrite, sceKernelPreadv, sceKernelPwritev](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) on Windows transfer at the given offset with an overlapped `ReadFile`/`WriteFile` and restore the descriptor's file pointer afterwards, so a concurrent `read`/`write` on the same descriptor can see the moved pointer in between, unlike on the console. Handles that are not disk files return `ESPIPE`. `readv` on a pipe or console handle stops after the first iovec read, a short read the console is assumed to allow; each iovec is transferred separately, so a failure after the first transfer returns the bytes already done From 4e7511dc32331ab1f19bf4076ac6d2a9c02e1070 Mon Sep 17 00:00:00 2001 From: AlxcBTW <187763517+AlxcBTW@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:50:12 +0200 Subject: [PATCH 258/460] perf(shader): check f32 product tininess in one multiply to nearest even #2839 decides f32 product tininess after rounding, in all four f32 round modes, from the exponents and a UMulHi and IMul of the significands, and builds the flushed value from them. To nearest even, the mode of shaders without a known float mode and of the 0xc0 most titles use, one multiply decides it: one factor times 2^64, which is exact below 2^64, times the other factor rounds the exact product once to 24 bits in the normal range, and 0 < |that| < 2^-62 is tininess after rounding. A product that rounds up to 2^-126 is then not tiny and keeps the emitted product, which is 2^-126 there, so nothing has to be built from the significands. The flush of v_mul_f32 v5, v6, v7 takes 12 IR instructions instead of 34, counted as the instructions the flush select needs beyond the product. vector_constant_factor_tininess pins 12 to nearest and 36 for the directed path of the same instruction, and checks that the directed modes and v_mad_legacy_f32 (before rounding) keep #2839's check. Its round-to-nearest branches are removed as unreachable. The f32 denormals entry says which rounding the check inherits: the multiply is a plain OpFMul like the product, with no f32 rounding mode requested, so it relies on the driver rounding f32 to nearest even, as the products do. F32ConstantFactorTininess names the row a check before rounding flushes, 0x3f7ffffe * 0x00800001, which keeps the emitted 0x00800000. Against #2839 to nearest even, a model of both checks agrees on 1,074,977,546 operand pairs (every 24-bit significand at both rounding thresholds, random pairs, specials) with each other and with an exact reference. On an RTX 5070 the outputs of #2839's own tininess and mixed-source shaders are bit-identical with either lowering over 165,298 and 164,008 generated rows in all six float modes tested. Co-Authored-By: Claude Opus 5.5 --- .../execution/F32ConstantFactorTininess.cpp | 11 ++++ .../tests/VectorConstantFactorTininess.cpp | 66 +++++++++++++++---- .../src/TranslationContext/OperandAccess.cpp | 19 ++++-- docs/dev/TechnicalDebt.md | 2 +- 4 files changed, 81 insertions(+), 17 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/F32ConstantFactorTininess.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/F32ConstantFactorTininess.cpp index c2a7dae1f..4cc7734df 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/F32ConstantFactorTininess.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/F32ConstantFactorTininess.cpp @@ -76,6 +76,8 @@ constexpr std::array TinyProducts{{ {31u, 6u, 0x00000000u}, }}; +constexpr Pinned TinyOnlyBeforeRounding{2u, 2u, 0x00800000u}; + std::array BufferDescriptor(const void* data, std::uint32_t bytes) { const auto address = reinterpret_cast(data); return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; @@ -139,6 +141,12 @@ void CheckTinyProducts(const char* mode) { } } +void CheckTininessAfterRounding(const char* mode) { + const Pinned& pinned = TinyOnlyBeforeRounding; + const std::uint32_t actual = Output[pinned.tid * Results + pinned.pair * 2u]; + Require(actual == pinned.value, std::string("f32 tininess after rounding: lane ") + std::to_string(pinned.tid) + " " + mode + " " + Names[pinned.pair] + " is " + Hex(actual) + ", expected " + Hex(pinned.value) + ": the exact product 0x3f7ffffe * 0x00800001 is below 2^-126 and rounds to it, and RDNA2 (gfx1036) keeps it (#2682), where a check before rounding flushes it"); +} + } int main() { @@ -148,12 +156,15 @@ int main() { Run(*device, std::nullopt); Check("no float mode"); CheckTinyProducts("no float mode"); + CheckTininessAfterRounding("no float mode"); Run(*device, ShaderRecompiler::ShaderFloatMode{0xc0u, true, false, false}); Check("IEEE=0 f32 denormals flushed"); CheckTinyProducts("IEEE=0 f32 denormals flushed"); + CheckTininessAfterRounding("IEEE=0 f32 denormals flushed"); Run(*device, ShaderRecompiler::ShaderFloatMode{0x00u, false, true, false}); Check("IEEE=1 all denormals flushed"); CheckTinyProducts("IEEE=1 all denormals flushed"); + CheckTininessAfterRounding("IEEE=1 all denormals flushed"); Run(*device, ShaderRecompiler::ShaderFloatMode{0xf0u, true, false, false}); Check("f32 denormals kept"); std::puts("f32 constant factor tests passed"); diff --git a/core/libs/tests/VectorConstantFactorTininess.cpp b/core/libs/tests/VectorConstantFactorTininess.cpp index 27edb18e2..f75cf2777 100644 --- a/core/libs/tests/VectorConstantFactorTininess.cpp +++ b/core/libs/tests/VectorConstantFactorTininess.cpp @@ -1,10 +1,13 @@ #include "Translation/TranslationContext.hpp" #include "RdnaDecoder/RdnaVectorOpDecoder.hpp" +#include +#include #include #include #include #include #include +#include #include using namespace ShaderRecompiler; @@ -13,6 +16,7 @@ namespace { enum class Tininess { None, + Rounded, ExponentsOnly, Full, }; @@ -37,17 +41,51 @@ void Check(const std::string& name, const std::vector& words, Rdn IrProgram program; const auto& block = Translate(program, name, words, opcode, floatMode); std::uint32_t multiplyHigh = 0u; - std::uint32_t tinyCompares = 0u; + std::uint32_t exponentCompares = 0u; + std::uint32_t roundedCompares = 0u; for (const auto* value : block.Instructions()) { if (value->Opcode() == IrOpcode::UMulHi) ++multiplyHigh; - if (value->Opcode() == IrOpcode::ULessThan32 && value->Argument(1)->HasImmediate() && value->Argument(1)->ImmediateU32() == 128u) ++tinyCompares; + if (value->Opcode() != IrOpcode::ULessThan32 || !value->Argument(1)->HasImmediate()) continue; + if (value->Argument(1)->ImmediateU32() == 128u) ++exponentCompares; + if (value->Argument(1)->ImmediateU32() == 0x207fffffu) ++roundedCompares; } - const std::uint32_t expectedCompares = expected == Tininess::None ? 0u : 1u; + const std::uint32_t expectedExponentCompares = expected == Tininess::ExponentsOnly || expected == Tininess::Full ? 1u : 0u; + const std::uint32_t expectedRoundedCompares = expected == Tininess::Rounded ? 1u : 0u; const std::uint32_t expectedMultiplyHigh = expected == Tininess::Full ? 1u : 0u; - Require(tinyCompares == expectedCompares, name + " emits " + std::to_string(tinyCompares) + " tiny product checks, expected " + std::to_string(expectedCompares)); + Require(exponentCompares == expectedExponentCompares, name + " emits " + std::to_string(exponentCompares) + " exponent tiny product checks, expected " + std::to_string(expectedExponentCompares)); + Require(roundedCompares == expectedRoundedCompares, name + " emits " + std::to_string(roundedCompares) + " rounded tiny product checks, expected " + std::to_string(expectedRoundedCompares)); Require(multiplyHigh == expectedMultiplyHigh, name + " emits " + std::to_string(multiplyHigh) + " significand products, expected " + std::to_string(expectedMultiplyHigh)); } +std::unordered_set Cone(const IrValue* root, const std::unordered_set& instructions) { + std::unordered_set cone; + std::vector pending{root}; + while (!pending.empty()) { + const IrValue* value = pending.back(); + pending.pop_back(); + if (!instructions.contains(value) || !cone.insert(value).second) continue; + for (const IrValue* argument : value->Arguments()) pending.push_back(argument); + } + return cone; +} + +void CheckFlushSize(const std::string& name, const std::vector& words, RdnaOpcode opcode, std::uint32_t floatMode, std::size_t expected) { + IrProgram program; + const auto& block = Translate(program, name, words, opcode, floatMode); + const std::unordered_set instructions(block.Instructions().begin(), block.Instructions().end()); + for (const auto* value : block.Instructions()) { + if (value->Opcode() != IrOpcode::SelectF32) continue; + const IrOpcode product = value->Argument(2)->Opcode(); + if (product != IrOpcode::FPMul32 && product != IrOpcode::FPFma32) continue; + const auto flush = Cone(value, instructions); + const auto kept = Cone(value->Argument(2), instructions); + const auto size = static_cast(std::count_if(flush.begin(), flush.end(), [&](const IrValue* instruction) { return !kept.contains(instruction); })); + Require(size == expected, name + " flushes a tiny product with " + std::to_string(size) + " instructions, expected " + std::to_string(expected)); + return; + } + Require(false, name + " emits no tiny product flush"); +} + void CheckFoldedSource(const std::string& name, const std::vector& words, RdnaOpcode opcode, IrOpcode product, std::uint32_t index, std::uint32_t expected) { IrProgram program; const auto& block = Translate(program, name, words, opcode, 0xc0u); @@ -65,24 +103,30 @@ void CheckFoldedSource(const std::string& name, const std::vector int main() { try { - Check("v_mul_f32 v5, v6, v7", {0x100a0f06u}, RdnaOpcode::VMulF32, Tininess::Full); + Check("v_mul_f32 v5, v6, v7", {0x100a0f06u}, RdnaOpcode::VMulF32, Tininess::Rounded); Check("v_mul_f32 v5, 2.0, v7", {0x100a0ef4u}, RdnaOpcode::VMulF32, Tininess::None); Check("v_mul_f32 v5, -4.0, v7", {0x100a0ef7u}, RdnaOpcode::VMulF32, Tininess::None); Check("v_mul_f32 v5, 1.0, v7", {0x100a0ef2u}, RdnaOpcode::VMulF32, Tininess::None); Check("v_mul_f32 v5, 0x7f800000, v7", {0x100a0effu, 0x7f800000u}, RdnaOpcode::VMulF32, Tininess::None); Check("v_mul_f32 v5, 0x00400000, v7", {0x100a0effu, 0x00400000u}, RdnaOpcode::VMulF32, Tininess::None); Check("v_mul_f32 v5, 1, v7", {0x100a0e81u}, RdnaOpcode::VMulF32, Tininess::None); - Check("v_mul_f32 v5, 0.5, v7", {0x100a0ef0u}, RdnaOpcode::VMulF32, Tininess::ExponentsOnly); - Check("v_mul_f32 v5, 0x3e800000, v7", {0x100a0effu, 0x3e800000u}, RdnaOpcode::VMulF32, Tininess::ExponentsOnly); - Check("v_mul_f32 v5, 0x3f7fffff, v7", {0x100a0effu, 0x3f7fffffu}, RdnaOpcode::VMulF32, Tininess::Full); - Check("v_mul_f32 v5, 0.15915494, v7", {0x100a0ef8u}, RdnaOpcode::VMulF32, Tininess::Full); + Check("v_mul_f32 v5, 0.5, v7", {0x100a0ef0u}, RdnaOpcode::VMulF32, Tininess::Rounded); + Check("v_mul_f32 v5, 0x3e800000, v7", {0x100a0effu, 0x3e800000u}, RdnaOpcode::VMulF32, Tininess::Rounded); + Check("v_mul_f32 v5, 0x3f7fffff, v7", {0x100a0effu, 0x3f7fffffu}, RdnaOpcode::VMulF32, Tininess::Rounded); + Check("v_mul_f32 v5, 0.15915494, v7", {0x100a0ef8u}, RdnaOpcode::VMulF32, Tininess::Rounded); Check("v_mul_legacy_f32 v5, 4.0, v7", {0x0e0a0ef6u}, RdnaOpcode::VMulLegacyF32, Tininess::None); Check("v_fmamk_f32 v5, v6, 0x40400000, v7", {0x580a0f06u, 0x40400000u}, RdnaOpcode::VMadmkF32, Tininess::None); - Check("v_fmaak_f32 v5, v6, v7, 0x40400000", {0x5a0a0f06u, 0x40400000u}, RdnaOpcode::VMadakF32, Tininess::Full); + Check("v_fmaak_f32 v5, v6, v7, 0x40400000", {0x5a0a0f06u, 0x40400000u}, RdnaOpcode::VMadakF32, Tininess::Rounded); Check("v_fma_f32 v5, v6, -|2.0|, v7", {0xd54b0205u, 0x441de906u}, RdnaOpcode::VFmaF32, Tininess::None); - Check("v_fma_f32 v5, v6, -|0.5|, v7", {0xd54b0205u, 0x441de106u}, RdnaOpcode::VFmaF32, Tininess::ExponentsOnly); + Check("v_fma_f32 v5, v6, -|0.5|, v7", {0xd54b0205u, 0x441de106u}, RdnaOpcode::VFmaF32, Tininess::Rounded); + Check("v_mul_f32 v5, v6, v7 rounding toward +infinity", {0x100a0f06u}, RdnaOpcode::VMulF32, Tininess::Full, 0xc1u); + Check("v_mul_f32 v5, 0.5, v7 rounding toward zero", {0x100a0ef0u}, RdnaOpcode::VMulF32, Tininess::ExponentsOnly, 0xc3u); + Check("v_fma_f32 v5, v6, -|0.5|, v7 rounding toward -infinity", {0xd54b0205u, 0x441de106u}, RdnaOpcode::VFmaF32, Tininess::ExponentsOnly, 0xc2u); + Check("v_mad_legacy_f32 v5, v6, v7, v8", {0xd5400005u, 0x04220f06u}, RdnaOpcode::VMadLegacyF32, Tininess::Full); Check("v_mul_f32 v5, 2.0, v7 with f32 denormals kept", {0x100a0ef4u}, RdnaOpcode::VMulF32, Tininess::None, 0xf0u); Check("v_mul_f32 v5, v6, v7 with f32 denormals kept", {0x100a0f06u}, RdnaOpcode::VMulF32, Tininess::None, 0xf0u); + CheckFlushSize("v_mul_f32 v5, v6, v7", {0x100a0f06u}, RdnaOpcode::VMulF32, 0xc0u, 12u); + CheckFlushSize("v_mul_f32 v5, v6, v7 rounding toward +infinity", {0x100a0f06u}, RdnaOpcode::VMulF32, 0xc1u, 36u); CheckFoldedSource("v_fma_f32 v5, v6, -|2.0|, v7", {0xd54b0205u, 0x441de906u}, RdnaOpcode::VFmaF32, IrOpcode::FPFma32, 1u, 0xc0000000u); CheckFoldedSource("v_mul_f32 v5, 0x00400000, v7", {0x100a0effu, 0x00400000u}, RdnaOpcode::VMulF32, IrOpcode::FPMul32, 0u, 0x00000000u); CheckFoldedSource("v_mul_f32 v5, 0x80400000, v7", {0x100a0effu, 0x80400000u}, RdnaOpcode::VMulF32, IrOpcode::FPMul32, 0u, 0x80000000u); diff --git a/core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp b/core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp index e8c98a17a..27b6e20f8 100644 --- a/core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp +++ b/core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp @@ -159,12 +159,22 @@ IrF32 TranslationContext::flushTinyProduct(IrValue* lhs, IrValue* rhs, IrValue* if ((f32DenormalFlush & 2u) == 0u || keepsProductOutOfTinyRange(*lhs) || keepsProductOutOfTinyRange(*rhs)) return IrF32(*product); IrValue& lhsBits = ir.BitCastU32(*lhs); IrValue& rhsBits = ir.BitCastU32(*rhs); + const std::uint32_t roundMode = floatMode.has_value() ? floatMode->floatMode & 3u : 0u; + const auto zeroAddend = [&](IrValue* tiny) -> IrValue* { + if (addend == nullptr) return tiny; + return &ir.LogicalAnd(*tiny, ir.IEqual(ir.BitwiseAnd(ir.BitCastU32(*addend), ir.Constant(0x7fffffffu)), ir.Constant(0u))); + }; + if (afterRounding && roundMode == 0u) { + IrValue& scaled = ir.BitwiseAnd(ir.BitCastU32(ir.Emit(IrOpcode::FPMul32, IrType::F32, {&ir.Emit(IrOpcode::FPMul32, IrType::F32, {lhs, &ir.ConstantF32(0x1p64f)}), rhs})), ir.Constant(0x7fffffffu)); + IrValue* tiny = zeroAddend(&ir.ULessThan(ir.ISub(scaled, ir.Constant(1u)), ir.Constant(0x207fffffu))); + IrValue& sign = ir.BitwiseAnd(ir.BitwiseXor(lhsBits, rhsBits), ir.Constant(0x80000000u)); + return IrF32(ir.Emit(IrOpcode::SelectF32, IrType::F32, {tiny, &ir.BitCastF32(sign), product})); + } const auto exponent = [&](IrValue& bits) -> IrValue& { return ir.Emit(IrOpcode::BitFieldUExtract, IrType::U32, {&bits, &ir.Constant(23u), &ir.Constant(8u)}); }; const auto significand = [&](IrValue& bits) -> IrValue& { return ir.BitwiseOr(ir.BitwiseAnd(bits, ir.Constant(0x007fffffu)), ir.Constant(0x00800000u)); }; IrValue& lhsExponent = exponent(lhsBits); IrValue& rhsExponent = exponent(rhsBits); const auto finiteNonZero = [&](IrValue& value) -> IrValue& { return ir.LogicalAnd(ir.INotEqual(value, ir.Constant(0u)), ir.INotEqual(value, ir.Constant(0xffu))); }; - const std::uint32_t roundMode = floatMode.has_value() ? floatMode->floatMode & 3u : 0u; IrValue* carry = &ir.Constant(0u); IrValue* roundsUp = nullptr; if (!hasUnitSignificand(*lhs) && !hasUnitSignificand(*rhs)) { @@ -172,16 +182,15 @@ IrF32 TranslationContext::flushTinyProduct(IrValue* lhs, IrValue* rhs, IrValue* IrValue& rhsSignificand = significand(rhsBits); IrValue& high = ir.Emit(IrOpcode::UMulHi, IrType::U32, {&lhsSignificand, &rhsSignificand}); carry = &ir.Select(ir.UGreaterThan(high, ir.Constant(0x7fffu)), ir.Constant(1u), ir.Constant(0u)); - if (afterRounding && roundMode != 3u) roundsUp = &ir.LogicalAnd(ir.IEqual(high, ir.Constant(0x7fffu)), ir.UGreaterThan(ir.IMul(lhsSignificand, rhsSignificand), ir.Constant(roundMode == 0u ? 0xffbfffffu : 0xff800000u))); + if (afterRounding && roundMode != 3u) roundsUp = &ir.LogicalAnd(ir.IEqual(high, ir.Constant(0x7fffu)), ir.UGreaterThan(ir.IMul(lhsSignificand, rhsSignificand), ir.Constant(0xff800000u))); } IrValue& exponentSum = ir.IAdd(ir.IAdd(lhsExponent, rhsExponent), *carry); - IrValue* tiny = &ir.LogicalAnd(ir.LogicalAnd(finiteNonZero(lhsExponent), finiteNonZero(rhsExponent)), ir.ULessThan(exponentSum, ir.Constant(128u))); - if (addend != nullptr) tiny = &ir.LogicalAnd(*tiny, ir.IEqual(ir.BitwiseAnd(ir.BitCastU32(*addend), ir.Constant(0x7fffffffu)), ir.Constant(0u))); + IrValue* tiny = zeroAddend(&ir.LogicalAnd(ir.LogicalAnd(finiteNonZero(lhsExponent), finiteNonZero(rhsExponent)), ir.ULessThan(exponentSum, ir.Constant(128u)))); IrValue& sign = ir.BitwiseAnd(ir.BitwiseXor(lhsBits, rhsBits), ir.Constant(0x80000000u)); IrValue* rounded = &sign; if (roundsUp != nullptr) { roundsUp = &ir.LogicalAnd(ir.IEqual(exponentSum, ir.Constant(127u)), *roundsUp); - if (roundMode != 0u) roundsUp = &ir.LogicalAnd(*roundsUp, ir.IEqual(sign, ir.Constant(roundMode == 1u ? 0u : 0x80000000u))); + roundsUp = &ir.LogicalAnd(*roundsUp, ir.IEqual(sign, ir.Constant(roundMode == 1u ? 0u : 0x80000000u))); rounded = &ir.Select(*roundsUp, ir.BitwiseOr(sign, ir.Constant(0x00800000u)), sign); } return IrF32(ir.Emit(IrOpcode::SelectF32, IrType::F32, {tiny, &ir.BitCastF32(*rounded), product})); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index b2cbe8e0b..782d88a35 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -404,7 +404,7 @@ Throughout the project, every function at every stage either **does exactly what - [v_fma_mixlo_f16 and v_fma_mixhi_f16](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) round the exact product-sum once to f16 with the f64 arithmetic of v_interp_p2_f16, so they need the Float64 capability. On RDNA2 (gfx1036) they follow the `FLOAT_MODE` f16 round and denormal fields, not the f32 round field: -1.5 * 2^-24 + 2^-126 from f32 sources (0xbfc00000, 0x33800000, 0x00800000) gives 0x8001 with f16 denormals kept and round to nearest even, 0x8000 with f16 denormals flushed and 0x8002 with f16 round toward -infinity. Only f16 denormals kept with round to nearest even is translated; another f16 mode throws, as for packed f16. `FP16_OVFL` set is not measured. - [Signaling NaN inputs](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) of the f32 and f16 unary instructions (`v_sqrt/rcp/rsq/log/exp/sin/cos/fract/floor/ceil/trunc/rndne/rcp_iflag/frexp_mant`), `v_ldexp_f32/f16`, `v_frexp_mant_f16`, `v_div_fixup_f32/f16` and `v_mullit_f32` are returned quieted only with IEEE=1 and unchanged with IEEE=0, measured on RDNA2 (gfx1035) in both modes, and so are those of the f64 forms (`v_sqrt/rcp/rsq/fract/floor/ceil/trunc/rndne/frexp_mant/ldexp/div_fixup_f64`) and `v_cvt_f64_f32`, measured on RDNA2 (gfx1036) in both modes; shaders without a known float mode follow IEEE=0. `v_pack_b32_f16` quiets each signaling NaN half by the same rule, measured on RDNA2 (gfx1036) in both modes. A signaling NaN result of `v_cubesc/cubetc/cubema_f32` is quieted by the same rule and keeps the sign the instruction gives it, measured on RDNA2 (gfx1036) in both modes. `v_add/sub/subrev/mul/fma/fmac/fmamk/fmaak/mul_legacy_f32` and `v_fma_mix_f32` with f32 sources choose their NaN result on the bits, measured over 1000 operand triples in both modes: the first NaN source in operand order after its modifiers (the subtrahend negated, as the hardware adds its negation), quieted only with IEEE=1, or the default NaN 0xffc00000 for an invalid operation; in the fma forms an invalid product (inf * 0) gives the default NaN even when src2 is a NaN, and in the legacy forms a zero factor makes the product 0 even when the other factor is a NaN (a NaN src2 still wins). `v_mad_legacy_f32` and `v_mac_legacy_f32` follow the same rule with their separately rounded product, as measured on gfx1010 (#1281, #1290): a NaN source quieted only with IEEE=1, inf - inf giving 0xffc00000, the zero rule hiding a NaN factor and a NaN src2 winning. The f16 and f64 arithmetic, `v_fma_mix_f32` with f16 sources and the separately rounded mad/mac forms (not measurable on RDNA2) follow the same rule on hardware but their NaN bits are left to the driver. `v_cvt_f16_f32`, `v_cvt_f32_f64`, `v_dot2_f32_f16`, `v_fma_mixlo_f16` and `v_fma_mixhi_f16` always quiet and `v_div_scale_f32` never does: the f32 signaling NaN 0x7f812345 as src0 or src2 gives 0x7e09 from mixlo and mixhi at IEEE=0 and IEEE=1 on RDNA2 (gfx1036), where `v_fma_mix_f32` gives 0x7f812345 at IEEE=0 and 0x7fc12345 at IEEE=1. The payload of an f16 NaN source of mixlo and mixhi is left to the driver's f16 to f32 conversion, which NVIDIA replaces. - [Packed f16 NaN results](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) (`v_pk_add/mul/fma/min/max_f16`) follow the hardware measured on RDNA2 (gfx1035) at IEEE=0 and IEEE=1 with f16 round-to-nearest-even and f16 denormals kept: with IEEE=1 a signaling NaN input is quieted and min/max return the other operand only for a quiet NaN; with IEEE=0 NaN inputs pass through unquieted and min/max ignore signaling. In `v_pk_fma_f16` an invalid product (inf * 0) gives the default NaN even when src2 is a NaN. A shader without a known float mode is translated as IEEE=0, the mode titles use; a mode with another f16 rounding or that flushes f16 denormals throws. -- [f32 denormals](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follow the shader's `FLOAT_MODE` f32 denormal field (bits 5:4) when it flushes them (0, and shaders without a known float mode, as in the 0xc0 most titles use), measured on RDNA2 (gfx1035) over 1444 rows per instruction at IEEE=0 and IEEE=1. `v_add/sub/subrev/mul/mul_legacy/mullit/fma/fmac/fmamk/fmaak/min/max/min3/max3/med3/fract/floor/ceil/frexp_mant/sin/cos_f32` flush their f32 inputs and their result to a zero of the same sign (`v_mullit_f32` measured on RDNA2 (gfx1036) over 6907 rows per mode at IEEE=0 and IEEE=1, so a positive denormal src2 counts as zero); the f32 compares, `v_cvt_i32/u32/rpi_i32/flr_i32_f32`, `v_frexp_exp_i32_f32`, `v_cvt_f64_f32`, `v_cvt_f16_f32`, `v_cvt_pkrtz_f16_f32`, `v_cvt_pknorm_i16/u16_f32` and `v_cvt_pk_u8_f32` flush their f32 inputs. `v_div_scale_f32` flushes all three inputs to signed zeros, measured on RDNA2 (gfx1036) over 2744 operand triples per mode at IEEE=0 and IEEE=1 with f32 denormals flushed and kept; its scale-down behaviour and host denormal preservation remain incomplete. `v_div_fixup_f32` flushes its src1 and src2 and keeps a denormal src0 and result, measured on RDNA2 (gfx1036) over 6859 rows per mode at IEEE=0 and IEEE=1. Tininess is detected after rounding, measured on RDNA2 (gfx1036) for `v_mul_f32` and `v_mullit_f32` at IEEE=0 and IEEE=1: a product is flushed when its value rounded to 24 bits with an unbounded exponent is below 2^-126, so `0x3f7ffffe * 0x00800001` (2^-126 * (1 - 2^-46)) gives `0x00800000` and `0x3f7fffff * 0x00800000` (2^-126 * (1 - 2^-24)) gives zero. The check rounds in the f32 round mode of `FLOAT_MODE` (bits 1:0), measured in all four modes, while the products themselves are still rounded to nearest even. This is computed exactly from the significands for `v_mul_f32`, `v_mul_legacy_f32`, `v_mullit_f32`, and for `v_fma_f32`, `v_fmac_f32`, `v_fmamk_f32`, `v_fmaak_f32` and `v_fma_mix_f32` when the addend is zero; with a nonzero addend the fused result is kept, so an exact result just below 2^-126 that rounds to it is not flushed. `v_fma_mix_f32` follows this rule with mixed f16/f32 sources too, measured on RDNA2 (gfx1036) with f16 in either multiplicand slot, low and high half selection, and an f32 or f16 zero addend: f32 `0x00801002` times f16 `0x3bff` gives `0x00800000` to nearest at IEEE=0 and IEEE=1, while the neighbours `0x00801001` and `0x00801003` give zero and `0x00800001`. The tiny-product boundary was also measured for both signs in all four f32 round modes at IEEE=0; f16 inputs retain their previous conversion and denormal handling. `v_fma_mixlo_f16` and `v_fma_mixhi_f16` flush their f32 sources too and keep a product below 2^-126, measured on RDNA2 (gfx1036). `v_mad_legacy_f32` and `v_mac_legacy_f32` flush their inputs, their product and their result in every mode, see its entry; their product is still checked for tininess on its exact value, so one below 2^-126 gives zero even when it rounds to 2^-126, as the console's separately rounded encoding is not measured at this boundary. Denormal modes 1 and 2 throw for the other instructions. Not covered, and not measurable on RDNA2 (no such opcodes) or not measured: `v_mac_f32`, `v_mad_f32`, `v_madak_f32` and `v_madmk_f32` (rounded separately), `v_dot2_f32_f16` and `v_div_fmas_f32` (on hardware `v_div_fmas_f32` flushes only its result), f32 interpolation and `ds_add_f32` keep the previous lowering; `ds_min/max/cmpst_f32` flush both operands by the field, see the LDS float atomics entry. With f32 denormals kept (3) the results depend on the driver's default denormal behaviour, as no `DenormPreserve` execution mode is requested, and `v_ldexp_f32` flushes in every mode. +- [f32 denormals](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follow the shader's `FLOAT_MODE` f32 denormal field (bits 5:4) when it flushes them (0, and shaders without a known float mode, as in the 0xc0 most titles use), measured on RDNA2 (gfx1035) over 1444 rows per instruction at IEEE=0 and IEEE=1. `v_add/sub/subrev/mul/mul_legacy/mullit/fma/fmac/fmamk/fmaak/min/max/min3/max3/med3/fract/floor/ceil/frexp_mant/sin/cos_f32` flush their f32 inputs and their result to a zero of the same sign (`v_mullit_f32` measured on RDNA2 (gfx1036) over 6907 rows per mode at IEEE=0 and IEEE=1, so a positive denormal src2 counts as zero); the f32 compares, `v_cvt_i32/u32/rpi_i32/flr_i32_f32`, `v_frexp_exp_i32_f32`, `v_cvt_f64_f32`, `v_cvt_f16_f32`, `v_cvt_pkrtz_f16_f32`, `v_cvt_pknorm_i16/u16_f32` and `v_cvt_pk_u8_f32` flush their f32 inputs. `v_div_scale_f32` flushes all three inputs to signed zeros, measured on RDNA2 (gfx1036) over 2744 operand triples per mode at IEEE=0 and IEEE=1 with f32 denormals flushed and kept; its scale-down behaviour and host denormal preservation remain incomplete. `v_div_fixup_f32` flushes its src1 and src2 and keeps a denormal src0 and result, measured on RDNA2 (gfx1036) over 6859 rows per mode at IEEE=0 and IEEE=1. Tininess is detected after rounding, measured on RDNA2 (gfx1036) for `v_mul_f32` and `v_mullit_f32` at IEEE=0 and IEEE=1: a product is flushed when its value rounded to 24 bits with an unbounded exponent is below 2^-126, so `0x3f7ffffe * 0x00800001` (2^-126 * (1 - 2^-46)) gives `0x00800000` and `0x3f7fffff * 0x00800000` (2^-126 * (1 - 2^-24)) gives zero. The check rounds in the f32 round mode of `FLOAT_MODE` (bits 1:0), measured in all four modes, while the products themselves are still rounded to nearest even. This applies to `v_mul_f32`, `v_mul_legacy_f32`, `v_mullit_f32`, and to `v_fma_f32`, `v_fmac_f32`, `v_fmamk_f32`, `v_fmaak_f32` and `v_fma_mix_f32` when the addend is zero; with a nonzero addend the fused result is kept, so an exact result just below 2^-126 that rounds to it is not flushed. In the three directed modes the check is computed exactly from the significands. To nearest even (0, the mode of shaders without a known float mode) one factor is multiplied by 2^64, which is exact below 2^64, and the product with the other factor is compared with 2^-62. That multiply is a plain `OpFMul` like the product, and no f32 rounding mode is requested in the SPIR-V, so the check relies on the driver rounding f32 to nearest even by default, as the products do, and a product that rounds up to 2^-126 is the emitted product. `v_fma_mix_f32` follows this rule with mixed f16/f32 sources too, measured on RDNA2 (gfx1036) with f16 in either multiplicand slot, low and high half selection, and an f32 or f16 zero addend: f32 `0x00801002` times f16 `0x3bff` gives `0x00800000` to nearest at IEEE=0 and IEEE=1, while the neighbours `0x00801001` and `0x00801003` give zero and `0x00800001`. The tiny-product boundary was also measured for both signs in all four f32 round modes at IEEE=0; f16 inputs retain their previous conversion and denormal handling. `v_fma_mixlo_f16` and `v_fma_mixhi_f16` flush their f32 sources too and keep a product below 2^-126, measured on RDNA2 (gfx1036). `v_mad_legacy_f32` and `v_mac_legacy_f32` flush their inputs, their product and their result in every mode, see its entry; their product is still checked for tininess on its exact value, so one below 2^-126 gives zero even when it rounds to 2^-126, as the console's separately rounded encoding is not measured at this boundary. Denormal modes 1 and 2 throw for the other instructions. Not covered, and not measurable on RDNA2 (no such opcodes) or not measured: `v_mac_f32`, `v_mad_f32`, `v_madak_f32` and `v_madmk_f32` (rounded separately), `v_dot2_f32_f16` and `v_div_fmas_f32` (on hardware `v_div_fmas_f32` flushes only its result), f32 interpolation and `ds_add_f32` keep the previous lowering; `ds_min/max/cmpst_f32` flush both operands by the field, see the LDS float atomics entry. With f32 denormals kept (3) the results depend on the driver's default denormal behaviour, as no `DenormPreserve` execution mode is requested, and `v_ldexp_f32` flushes in every mode. - The executable file that [relinker](../../core/relinker/elfpatcher/src/windows/WindowsPeWriter.cpp) generates opens the console when launched, which is inconvenient for playability. - [Overlay output bus](../../core/libs/prx/libSceVideoOut/src/VideoOutDriver.cpp) (libSceVideoOut) - flips on a port opened with `SCE_VIDEO_OUT_BUS_TYPE_OVERLAY` complete and signal their events like main-bus flips, but their buffers are not shown: the overlay is not composited over the main output. Presenting them as whole frames made the window alternate between the two buses - [PA_CL_CLIP_CNTL](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) - `DX_LINEAR_ATTR_CLIP_ENA` (bit 24) clear is drawn like bit 24 set, with Vulkan's clipping of `NoPerspective` outputs; what a clear bit changes on the console is undocumented. From da642e807b6a2c2c31d9dda9614df11902f43693 Mon Sep 17 00:00:00 2001 From: XairRrR Date: Sun, 11 Oct 2026 01:55:16 +0200 Subject: [PATCH 259/460] fix(libSceVideoOut): take 64-bit VRR peg arguments and test peg and unpeg --- core/libs/prx/libSceVideoOut/include/Output.hpp | 2 ++ core/libs/prx/libSceVideoOut/src/Output.cpp | 3 ++- core/libs/prx/libSceVideoOut/tests/Flip.cpp | 8 ++++++++ 3 files changed, 12 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceVideoOut/include/Output.hpp b/core/libs/prx/libSceVideoOut/include/Output.hpp index 4bac79d39..a188ee600 100644 --- a/core/libs/prx/libSceVideoOut/include/Output.hpp +++ b/core/libs/prx/libSceVideoOut/include/Output.hpp @@ -26,6 +26,8 @@ int APS5_VABI sceVideoOutLatencyControlWaitBeforeInput(int handle); int APS5_VABI sceVideoOutLatencyMeasureSetStartPoint(int handle, uint32_t point); int APS5_VABI sceVideoOutColorSettingsSetGamma(VideoOutColorSettings* settings, float gamma); int APS5_VABI sceVideoOutAdjustColor(int handle, const VideoOutColorSettings* settings); +int APS5_VABI sceVideoOutVrrPegToFixedRate(int handle, uint64_t arg1, uint64_t arg2); +int APS5_VABI sceVideoOutVrrUnpegFromFixedRate(int handle); } diff --git a/core/libs/prx/libSceVideoOut/src/Output.cpp b/core/libs/prx/libSceVideoOut/src/Output.cpp index bf4b18327..067e01908 100644 --- a/core/libs/prx/libSceVideoOut/src/Output.cpp +++ b/core/libs/prx/libSceVideoOut/src/Output.cpp @@ -8,6 +8,7 @@ #include #include "SceTypes.hpp" +#include "prx/libSceVideoOut/include/Output.hpp" #include "prx/libc/include/General.hpp" #include "prx/libSceVideoOut/include/VideoOutDriver.hpp" @@ -310,7 +311,7 @@ int APS5_VABI sceVideoOutVrrUnpegFromFixedRate(int handle) try { LibcAwaitExit_nid_postfix(); } -int APS5_VABI sceVideoOutVrrPegToFixedRate(int handle, int32_t arg1, int32_t arg2) try { +int APS5_VABI sceVideoOutVrrPegToFixedRate(int handle, uint64_t arg1, uint64_t arg2) try { if (VideoOutDriver::Get().GetConfig(handle) == nullptr) { throw std::runtime_error(std::string(__func__) + ": VIDEO_OUT_ERROR_INVALID_HANDLE"); } diff --git a/core/libs/prx/libSceVideoOut/tests/Flip.cpp b/core/libs/prx/libSceVideoOut/tests/Flip.cpp index 02ff20088..c883e240c 100644 --- a/core/libs/prx/libSceVideoOut/tests/Flip.cpp +++ b/core/libs/prx/libSceVideoOut/tests/Flip.cpp @@ -340,7 +340,15 @@ void testControls() { vrrStatus.ident = VIDEO_OUT_EVENT_VRR_STATUS; vrrStatus.filter = EVFILT_VIDEO_OUT; check(sceVideoOutGetEventId(&vrrStatus) == VIDEO_OUT_EVENT_VRR_STATUS, "VRR status event id rejected"); + check(sceVideoOutVrrPegToFixedRate(handle, 0, 0) == 0, "VRR peg on an open handle failed"); + check(sceVideoOutVrrUnpegFromFixedRate(handle) == 0, "VRR unpeg on an open handle failed"); + expectFailure([&] { sceVideoOutVrrPegToFixedRate(handle, 1, 0); }); + expectFailure([&] { sceVideoOutVrrPegToFixedRate(handle, 0, 1); }); + expectFailure([&] { sceVideoOutVrrPegToFixedRate(handle, 0x100000000ull, 0); }); + expectFailure([&] { sceVideoOutVrrPegToFixedRate(handle, 0, 0x100000000ull); }); sceVideoOutClose(handle); + expectFailure([&] { sceVideoOutVrrPegToFixedRate(handle, 0, 0); }); + expectFailure([&] { sceVideoOutVrrUnpegFromFixedRate(handle); }); check(owner->GetTriggeredEvents(&event, 1) == 0, "closed port retained pending events"); check(sceKernelDeleteEqueue(queue) == 0, "event queue deletion failed"); LibcRunShutdown_nid_postfix(); From c332bdb31901d3644323b7ada38c4e284c20cfbe Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Sun, 11 Oct 2026 08:56:03 +0900 Subject: [PATCH 260/460] test(libkernel): skip the Windows lstat link checks without the symlink privilege guest_filesystem creates symbolic links for the Windows lstat checks. Without administrator rights or Developer Mode, CreateSymbolicLinkW fails with ERROR_PRIVILEGE_NOT_HELD even with SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE, and the test aborted there (reported as a stale guest errno). Skip those checks with a note in that case; any other failure to create the first link still fails the test. Co-Authored-By: Claude Opus 5.5 --- core/libs/tests/GuestFilesystem.cpp | 62 +++++++++++++++-------------- 1 file changed, 33 insertions(+), 29 deletions(-) diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index d84df7903..fa3aa8176 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -268,36 +268,40 @@ int main() { Require(lstat_nid_postfix((root / "present.txt" / "child").string().c_str(), &linkStatus) == -1 && *__error_nid_postfix() == 20); Require(unlink_nid_postfix(link.string().c_str()) == 0 && unlink_nid_postfix(dangling.string().c_str()) == 0); #else - const auto targetDirectory = root / "target-directory"; - Require(std::filesystem::create_directory(targetDirectory)); const auto fileLink = root / "file-link"; - const auto directoryLink = root / "directory-link"; - const auto danglingLink = root / "dangling-link"; - Require(CreateSymbolicLinkW(fileLink.c_str(), present.c_str(), SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); - Require(CreateSymbolicLinkW(directoryLink.c_str(), targetDirectory.c_str(), - SYMBOLIC_LINK_FLAG_DIRECTORY | SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); - Require(CreateSymbolicLinkW(danglingLink.c_str(), missing.c_str(), SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); - const auto checkUnsupportedLink = [](const std::filesystem::path& link) { - FileStat untouched; - std::memset(&untouched, 0x5a, sizeof(untouched)); - constexpr int sentinelError = 123; - *__error_nid_postfix() = sentinelError; - bool rejected = false; - try { - lstat_nid_postfix(link.string().c_str(), &untouched); - } catch (const std::runtime_error& error) { - rejected = std::strcmp(error.what(), "lstat of a Windows symbolic link not implemented") == 0; - } - const bool errorUnchanged = *__error_nid_postfix() == sentinelError; - const auto* bytes = reinterpret_cast(&untouched); - const bool statusUnchanged = std::all_of(bytes, bytes + sizeof(untouched), [](unsigned char value) { return value == 0x5a; }); - return rejected && errorUnchanged && statusUnchanged; - }; - Require(checkUnsupportedLink(fileLink)); - Require(checkUnsupportedLink(directoryLink)); - Require(checkUnsupportedLink(danglingLink)); - Require(std::filesystem::remove(fileLink) && std::filesystem::remove(directoryLink) && - std::filesystem::remove(danglingLink) && std::filesystem::remove(targetDirectory)); + if (CreateSymbolicLinkW(fileLink.c_str(), present.c_str(), SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) == 0) { + Require(GetLastError() == ERROR_PRIVILEGE_NOT_HELD); + std::fprintf(stderr, "Windows lstat link checks skipped: creating symbolic links needs Developer Mode or administrator rights\n"); + } else { + const auto targetDirectory = root / "target-directory"; + Require(std::filesystem::create_directory(targetDirectory)); + const auto directoryLink = root / "directory-link"; + const auto danglingLink = root / "dangling-link"; + Require(CreateSymbolicLinkW(directoryLink.c_str(), targetDirectory.c_str(), + SYMBOLIC_LINK_FLAG_DIRECTORY | SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); + Require(CreateSymbolicLinkW(danglingLink.c_str(), missing.c_str(), SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE) != 0); + const auto checkUnsupportedLink = [](const std::filesystem::path& link) { + FileStat untouched; + std::memset(&untouched, 0x5a, sizeof(untouched)); + constexpr int sentinelError = 123; + *__error_nid_postfix() = sentinelError; + bool rejected = false; + try { + lstat_nid_postfix(link.string().c_str(), &untouched); + } catch (const std::runtime_error& error) { + rejected = std::strcmp(error.what(), "lstat of a Windows symbolic link not implemented") == 0; + } + const bool errorUnchanged = *__error_nid_postfix() == sentinelError; + const auto* bytes = reinterpret_cast(&untouched); + const bool statusUnchanged = std::all_of(bytes, bytes + sizeof(untouched), [](unsigned char value) { return value == 0x5a; }); + return rejected && errorUnchanged && statusUnchanged; + }; + Require(checkUnsupportedLink(fileLink)); + Require(checkUnsupportedLink(directoryLink)); + Require(checkUnsupportedLink(danglingLink)); + Require(std::filesystem::remove(fileLink) && std::filesystem::remove(directoryLink) && + std::filesystem::remove(danglingLink) && std::filesystem::remove(targetDirectory)); + } #endif FileStat identity{}; FileStat byDescriptor{}; From 9207f7245c93a278f8c20e06175b1575404236d2 Mon Sep 17 00:00:00 2001 From: TibixDev Date: Thu, 8 Oct 2026 10:42:29 +0300 Subject: [PATCH 261/460] fix(agc): keep CMASK clears resident without host imports Use the upstream CMASK implementation and remove the import requirement from image clears. Propagate storage-image creation failures. Test repeated clears, copied write-back, and failure without changing guest data. AI-assisted: yes. --- .../prx/libSceAgcDriver/Graphics/src/Draw.cpp | 7 +- .../libSceAgcDriver/Graphics/src/Texture.cpp | 4 - .../prx/libSceAgcDriver/tests/Recorder.cpp | 82 +++++++++++++++++++ 3 files changed, 83 insertions(+), 10 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp index 89ff3385f..a36593d10 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp @@ -2458,12 +2458,7 @@ namespace { std::shared_ptr metadataPassResident(const Context& context, const ColorTarget& color) { if (color.tileMode == ColorTileMode::Linear || context.detiler == nullptr) return nullptr; - std::shared_ptr resident; - try { - resident = CachedStorageSurface(context, SurfaceForTarget(color)); - } catch (const std::exception&) { - return nullptr; - } + const auto resident = CachedStorageSurface(context, SurfaceForTarget(color)); return resident != nullptr && resident->GuestBytes() == color.bytes ? resident : nullptr; } diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp index e5c3349d0..2e4b692a9 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp @@ -3314,10 +3314,6 @@ bool StorageTexture::FillClear(std::span pattern, std::u return false; } } - if (HostImportFor(context, descriptor.baseAddress, static_cast(guestBytes)) == nullptr) { - refusal = "not imported"; - return false; - } auto* recorder = Recorder::Active(); if (recorder == nullptr) { refusal = "no recorder"; diff --git a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp index df5310f6d..4f429c240 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp @@ -3697,6 +3697,82 @@ void sampleDumpTests(const Device& device, Recorder& recorder) { inPassSampleDumpTests(recorder, context, cover, words, address, target); } +void cmaskPassTests(const Device& device, Recorder& recorder) { + namespace GuestMemory = AgcDriver::GuestMemory; + auto context = device.GetContext(); + context.hostImportAlignment = 0; + TextureDetiler detiler(context); + context.detiler = &detiler; + constexpr std::size_t surfaceBytes = 256 * 256 * 4; + constexpr std::size_t bytes = surfaceBytes + 65536; +#ifdef _WIN32 + void* block = VirtualAlloc(nullptr, bytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE); +#else + void* block = std::aligned_alloc(65536, bytes); +#endif + Require(block != nullptr, "cannot allocate the CMASK pass block"); + const auto address = reinterpret_cast(block); + { + GuestAllocations::Mutation mutation; + mutation.Add(block, bytes, true, true); + } + struct Release { + const Context& context; + void* block; + ~Release() { + ClearCachedTextures(context.device); + { + GuestAllocations::Mutation mutation; + mutation.Remove(block); + } +#ifdef _WIN32 + VirtualFree(block, 0, MEM_RELEASE); +#else + std::free(block); +#endif + } + } release{context, block}; + Require(HostImportFor(context, address, bytes) == nullptr, "CMASK copy-path test imported its memory"); + std::memset(block, 0x55, surfaceBytes); + std::memset(static_cast(block) + surfaceBytes, 0, 4096); + ColorTarget color{}; + color.address = address; + color.extent = {256, 256}; + color.format = VK_FORMAT_R8G8B8A8_UNORM; + color.bytes = surfaceBytes; + color.componentMapping = 0xe4u; + color.tileMode = ColorTileMode::RenderTarget; + color.cmaskAddress = address + surfaceBytes; + color.cmaskBytes = CmaskBytes(color.extent.width, color.extent.height); + color.clearWords = {0x80402010u, 0}; + const auto run = [&] { RunColorMetadataPass(context, {ColorMetadataPass::Mode::EliminateFastClear, {color}}); }; + auto broken = context; + broken.deviceProc = nullptr; + bool refused = false; + try { + RunColorMetadataPass(broken, {ColorMetadataPass::Mode::EliminateFastClear, {color}}); + } catch (const std::runtime_error& error) { + refused = std::string_view(error.what()).find("missing Vulkan device function resolver") != std::string_view::npos; + if (!refused) throw; + } + Require(refused, "CMASK storage creation failure was silently ignored"); + Require(*static_cast(block) == 0x55 && ReadDccKeys(color.cmaskAddress, color.cmaskBytes * 256u) == DccKeys::Clear0000, "failed CMASK storage creation changed guest memory"); + run(); + Require(StorageTexture::FindPending(address, surfaceBytes) != nullptr && *static_cast(block) == 0x55, "CMASK clear without host imports did not stay on the GPU"); + Require(ReadDccKeys(color.cmaskAddress, color.cmaskBytes * 256u) == DccKeys::Uncompressed, "resident CMASK clear left unresolved metadata"); + color.clearWords[0] = 0x12345678; + run(); + std::vector stored(surfaceBytes / 4); + GuestMemory::Read(address, std::as_writable_bytes(std::span(stored))); + Require(std::ranges::all_of(stored, [](auto word) { return word == 0x80402010u; }), "expanded CMASK changed a pending clear or copy write-back lost it"); + const std::vector clearKeys(4096); + GuestMemory::Write(color.cmaskAddress, clearKeys); + run(); + GuestMemory::Read(address, std::as_writable_bytes(std::span(stored))); + Require(std::ranges::all_of(stored, [](auto word) { return word == 0x12345678u; }), "a second resident CMASK clear retained the first frame"); + recorder.Sync(); +} + void metadataPassTests(const Device& device, Recorder& recorder) { const auto& base = device.GetContext(); if (base.hostImportAlignment == 0) { @@ -4755,6 +4831,11 @@ int main(int argc, char** argv) { unitShadowTests(device, recorder); return 0; } + if (argc == 2 && std::string_view(argv[1]) == "--cmask-only") { + cmaskPassTests(device, recorder); + std::cout << "CMASK copied clear, write-back and failure tests passed\n"; + return 0; + } readTrackingTests(device, recorder); writeSettledTests(device, recorder); completionCountTests(device, recorder); @@ -4806,6 +4887,7 @@ int main(int argc, char** argv) { singleCubeTests(device, recorder); atomicViewTests(device, recorder); metadataPassTests(device, recorder); + cmaskPassTests(device, recorder); pendingKeyStoreTests(device, recorder); sampleDumpTests(device, recorder); gpu.unlock(); From 6b9f0cd01a8891d039dbba53e00a551fae42090c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 01:58:31 +0200 Subject: [PATCH 262/460] test(agc): pass the gpu flag to GuestAllocations::Mutation::Add in the CMASK pass test --- core/libs/prx/libSceAgcDriver/tests/Recorder.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp index 4f429c240..cc3c4bff4 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp @@ -3714,7 +3714,7 @@ void cmaskPassTests(const Device& device, Recorder& recorder) { const auto address = reinterpret_cast(block); { GuestAllocations::Mutation mutation; - mutation.Add(block, bytes, true, true); + mutation.Add(block, bytes, true, true, true); } struct Release { const Context& context; From 767b8f83a2aed4dc1f74f4febc8eba48a2874874 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Thu, 8 Oct 2026 10:25:45 -0500 Subject: [PATCH 263/460] feat(agc): accept graphics programs without an AGC shader header Co-Authored-By: Claude Opus 5.5 --- .../include/Driver/Shaders/ShaderRegistry.hpp | 3 +- .../src/Driver/Dispatch/Dispatch.cpp | 2 +- .../Execution/src/Driver/Draw/Draw.cpp | 4 ++- .../Execution/src/Driver/Draw/DrawDecode.cpp | 13 ++++---- .../src/Driver/Shaders/ShaderRegistry.cpp | 9 ++++-- .../Graphics/src/ShaderInputState.cpp | 1 + .../libs/prx/libSceAgcDriver/tests/Submit.cpp | 32 +++++++++---------- 7 files changed, 36 insertions(+), 28 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp index 25fd58e2d..3ddcc2398 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp @@ -84,7 +84,8 @@ struct ShaderSnapshot { }; -std::shared_ptr ReadRawComputeShader(std::uint64_t address); +std::shared_ptr ReadRawShader(std::uint64_t address); +std::shared_ptr ProgramSnapshot(const ShaderRegistry& shaders, std::uint64_t address, std::initializer_list types); std::shared_ptr PrepareShaderWithDiagnostics(const ShaderRecompiler::RecompileRequest& request, ShaderRecompiler::ShaderPreparationContext* preparation = nullptr); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp index dcfcc848f..19d60397f 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp @@ -17,7 +17,7 @@ void Driver::dispatch(QueueState& queue, std::span packet, PerformanceTimer timing("Driver.Dispatch"); const auto address = (static_cast(readRegister(queue.shader, 0x20c)) << 8u) | (static_cast(readRegister(queue.shader, 0x20d) & 0xffu) << 40u); auto registeredShader = RegisteredProgram(*submission.shaders, address, {0}); - if (!registeredShader) registeredShader = ReadRawComputeShader(address); + if (!registeredShader) registeredShader = ReadRawShader(address); const auto& snapshot = *registeredShader; require(snapshot.type == 0, "compute program refers to a non-compute shader"); const auto userCount = (readRegister(queue.shader, 0x213) >> 1u) & 0x1fu; diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp index 6ab1a5cb5..26bc065d8 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp @@ -123,7 +123,9 @@ DrawVerdict Driver::draw(QueueState& queue, std::span packe std::vector linked; for (std::size_t i = 0; i < programs.size(); ++i) { const auto& program = programs[i]; - memory.insert(memory.end(), program.memory.begin(), program.memory.end()); + for (const auto& region : program.memory) { + if (!region.bytes.empty()) memory.push_back(region); + } linked.push_back({roles[i], program.binary, program.userDataBase, program.firstUserSgpr, program.userData}); } timing.Mark("prepare"); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp index f4444a808..60d97fb2f 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp @@ -28,10 +28,10 @@ void DecodeGraphicsPrograms(DrawDecode& decoded, const QueueState& queue, const const auto prepare = [&](std::uint64_t address, std::uint8_t type, Stage stage, std::uint32_t rsrc2, std::uint32_t userDataBase) { const bool nullPixel = stage == Stage::Fragment && (address == 0 || pixelSkipped); if (nullPixel) address = NullPixelProgramAddress(); - const auto registered = RegisteredProgram(registry, address, {type}); - require(registered != nullptr, "graphics program does not belong to a compatible registered shader"); - const auto& snapshot = *registered; + const auto program = ProgramSnapshot(registry, address, {type}); + const auto& snapshot = *program; require((address - snapshot.codeAddress) % sizeof(std::uint32_t) == 0, "graphics entry point is not dword aligned"); + require(snapshot.header.empty() || snapshot.type == type, "graphics program refers to an incompatible shader binary type"); if (!nullPixel) Graphics::NoteRegisterRead(Graphics::RegisterBank::Shader, rsrc2); const auto resources = nullPixel ? 0u : ReadGraphicsRegister(queue.shader, rsrc2); const auto userCount = ((resources >> 1u) & 0x1fu) | (((resources >> 27u) & 1u) << 5u); @@ -43,7 +43,7 @@ void DecodeGraphicsPrograms(DrawDecode& decoded, const QueueState& queue, const 8, {}, {{{snapshot.codeAddress, std::as_bytes(std::span(snapshot.code))}, {snapshot.headerAddress, snapshot.header}}}, - registered, + program, codeOffset }; for (std::uint32_t i = 0; i < userCount; ++i) { @@ -86,9 +86,8 @@ void DecodeGraphicsPrograms(DrawDecode& decoded, const QueueState& queue, const append(0x0c8, 2, Stage::TessellationEvaluation, 0x08b, 0x08c, Role::Domain); } else if (graphics.stages.path == Graphics::ShaderPath::Geometry) { const auto frontAddress = programAddress(0xc8); - const auto snapshot = RegisteredProgram(registry, frontAddress, {2, 4}); - require(snapshot != nullptr, "geometry front program is not registered"); - const auto type = snapshot->type; + const auto front = ProgramSnapshot(registry, frontAddress, {2, 4}); + const auto type = front->header.empty() ? std::uint8_t{2} : front->type; require(type == 2 || type == 4, "invalid geometry front binary type"); append(0xc8, type, Stage::Mesh, 0x8b, 0x8c, Role::Main); initializeMerged(programs.back(), 0x82, type == 4); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp index d01847729..47554a4e2 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp @@ -37,7 +37,7 @@ std::shared_ptr PrepareShaderWithDiagnosti } } -std::shared_ptr ReadRawComputeShader(std::uint64_t address) { +std::shared_ptr ReadRawShader(std::uint64_t address) { GuestMemory::CheckRange(reinterpret_cast(address), sizeof(std::uint32_t), 256); static std::mutex cacheMutex; static std::list> cache; @@ -100,7 +100,12 @@ std::shared_ptr ReadRawComputeShader(std::uint64_t address if (snapshot.code.size() == available) break; } } - throw std::runtime_error("AGC driver: raw compute program has no reachable end within mapped code or the size limit"); + throw std::runtime_error("AGC driver: raw shader program has no reachable end within mapped code or the size limit"); +} + +std::shared_ptr ProgramSnapshot(const ShaderRegistry& shaders, std::uint64_t address, std::initializer_list types) { + if (auto registered = RegisteredProgram(shaders, address, types)) return registered; + return ReadRawShader(address); } namespace { diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderInputState.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderInputState.cpp index 1939acc1e..3927f11a6 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderInputState.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderInputState.cpp @@ -163,6 +163,7 @@ ShaderRecompiler::ShaderPixelStageInfo DecodePixelStageInfo(const Registers& con } ShaderRecompiler::ShaderVertexStageInfo DecodeVertexStageInfo(std::span header, std::uint64_t headerAddress, std::span userData, std::vector* reads, bool staticAbi) { + if (header.empty()) return {}; if (header.size() < sizeof(Shader)) throw std::runtime_error("AGC graphics: shader header is smaller than the fixed AGC header"); Shader shader; std::memcpy(&shader, header.data(), sizeof(Shader)); diff --git a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp index cd6f89778..329b4e42b 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp @@ -593,20 +593,20 @@ int main() { rawCode[1] = 0xbf810000; rawCode[2] = 0xbf810000; const auto rawAddress = reinterpret_cast(rawCode.data()); - const auto literal = AgcDriver::DriverDetail::ReadRawComputeShader(rawAddress); + const auto literal = AgcDriver::DriverDetail::ReadRawShader(rawAddress); check(literal->code.size() == 3 && literal->header.empty(), "raw compute stopped at an instruction literal"); - check(AgcDriver::DriverDetail::ReadRawComputeShader(rawAddress) == literal, "unchanged raw compute code lost its snapshot identity"); + check(AgcDriver::DriverDetail::ReadRawShader(rawAddress) == literal, "unchanged raw compute code lost its snapshot identity"); rawCode[0] = 0xbf820002; rawCode[1] = 0xbf810000; rawCode[2] = 0xbf800000; rawCode[3] = 0xbf810000; - const auto branched = AgcDriver::DriverDetail::ReadRawComputeShader(rawAddress); + const auto branched = AgcDriver::DriverDetail::ReadRawShader(rawAddress); check(branched->code.size() == 4 && literal->code[0] == 0xbe8003ff, "raw compute lost branch targets or modified an earlier snapshot"); check(branched != literal, "changed raw compute code reused a stale snapshot"); std::array, 8> concurrent; std::vector readers; for (auto& snapshot : concurrent) readers.emplace_back([&snapshot, rawAddress] { - snapshot = AgcDriver::DriverDetail::ReadRawComputeShader(rawAddress); + snapshot = AgcDriver::DriverDetail::ReadRawShader(rawAddress); }); for (auto& reader : readers) reader.join(); for (const auto& snapshot : concurrent) check(snapshot == branched, "concurrent raw compute readers lost snapshot reuse"); @@ -615,23 +615,23 @@ int main() { const auto firstAddress = reinterpret_cast(programs.front().data()); readers.clear(); for (auto& snapshot : concurrent) readers.emplace_back([&snapshot, firstAddress] { - snapshot = AgcDriver::DriverDetail::ReadRawComputeShader(firstAddress); + snapshot = AgcDriver::DriverDetail::ReadRawShader(firstAddress); }); for (auto& reader : readers) reader.join(); for (const auto& snapshot : concurrent) check(snapshot == concurrent.front(), "concurrent raw compute misses duplicated snapshots"); - const auto evicted = AgcDriver::DriverDetail::ReadRawComputeShader(firstAddress); + const auto evicted = AgcDriver::DriverDetail::ReadRawShader(firstAddress); for (std::size_t i = 1; i < programs.size(); ++i) - AgcDriver::DriverDetail::ReadRawComputeShader(reinterpret_cast(programs[i].data())); - check(AgcDriver::DriverDetail::ReadRawComputeShader(firstAddress) != evicted && evicted->code[0] == 0xbf810000, + AgcDriver::DriverDetail::ReadRawShader(reinterpret_cast(programs[i].data())); + check(AgcDriver::DriverDetail::ReadRawShader(firstAddress) != evicted && evicted->code[0] == 0xbf810000, "raw compute cache eviction lost snapshot lifetime or exceeded its entry limit"); - check(!expectFailure([&] { AgcDriver::DriverDetail::ReadRawComputeShader(rawAddress + 4); }).empty(), "raw compute accepted a misaligned entry"); - check(!expectFailure([] { AgcDriver::DriverDetail::ReadRawComputeShader(0); }).empty(), "raw compute accepted an unmapped entry"); + check(!expectFailure([&] { AgcDriver::DriverDetail::ReadRawShader(rawAddress + 4); }).empty(), "raw compute accepted a misaligned entry"); + check(!expectFailure([] { AgcDriver::DriverDetail::ReadRawShader(0); }).empty(), "raw compute accepted an unmapped entry"); alignas(256) std::array straddling{}; straddling.fill(0xbf800000); straddling[63] = 0xf4000000; straddling[64] = 0xfa000000; straddling[65] = 0xbf810000; - const auto straddled = AgcDriver::DriverDetail::ReadRawComputeShader(reinterpret_cast(straddling.data())); + const auto straddled = AgcDriver::DriverDetail::ReadRawShader(reinterpret_cast(straddling.data())); check(straddled->code.size() == 66 && straddled->code[64] == 0xfa000000, "raw compute stopped at a memory instruction across its first read window"); #ifdef _WIN32 auto* mapping = static_cast(VirtualAlloc(nullptr, 8192, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); @@ -641,18 +641,18 @@ int main() { auto* boundedCode = mapping + 1024 - 64; std::fill_n(boundedCode, 64, 0xbf800000u); const auto boundedAddress = reinterpret_cast(boundedCode); - const auto unterminated = expectFailure([&] { AgcDriver::DriverDetail::ReadRawComputeShader(boundedAddress); }); + const auto unterminated = expectFailure([&] { AgcDriver::DriverDetail::ReadRawShader(boundedAddress); }); boundedCode[63] = 0xbf810000; - const auto bounded = AgcDriver::DriverDetail::ReadRawComputeShader(boundedAddress); + const auto bounded = AgcDriver::DriverDetail::ReadRawShader(boundedAddress); check(VirtualProtect(mapping + 1024, 4096, PAGE_READWRITE, &protection) != 0, "cannot extend raw compute mapping"); boundedCode[63] = 0xbf800000; boundedCode[64] = 0xbf810000; - const auto extended = AgcDriver::DriverDetail::ReadRawComputeShader(boundedAddress); + const auto extended = AgcDriver::DriverDetail::ReadRawShader(boundedAddress); check(extended != bounded && extended->code.size() == 65, "raw compute reused code before its end changed"); check(VirtualProtect(mapping + 1024, 4096, PAGE_NOACCESS, &protection) != 0, "cannot revoke cached raw compute tail"); - check(!expectFailure([&] { AgcDriver::DriverDetail::ReadRawComputeShader(boundedAddress); }).empty(), "raw compute reused an inaccessible cached tail"); + check(!expectFailure([&] { AgcDriver::DriverDetail::ReadRawShader(boundedAddress); }).empty(), "raw compute reused an inaccessible cached tail"); check(VirtualProtect(mapping, 4096, PAGE_NOACCESS, &protection) != 0, "cannot revoke cached raw compute code"); - check(!expectFailure([&] { AgcDriver::DriverDetail::ReadRawComputeShader(boundedAddress); }).empty(), "raw compute reused inaccessible cached code"); + check(!expectFailure([&] { AgcDriver::DriverDetail::ReadRawShader(boundedAddress); }).empty(), "raw compute reused inaccessible cached code"); check(VirtualFree(mapping, 0, MEM_RELEASE) != 0, "cannot release raw compute boundary test"); check(!unterminated.empty() && bounded->code.size() == 64, "raw compute crossed inaccessible memory or missed its last instruction"); #endif From dddd9f29c22d64a1bc27757f8e874a773eaab001 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Thu, 8 Oct 2026 10:49:55 -0500 Subject: [PATCH 264/460] test(agc): cover raw program snapshots and headerless vertex decode Co-Authored-By: Claude Opus 5.5 --- .../prx/libSceAgcDriver/tests/Graphics.cpp | 9 +++++++ .../libs/prx/libSceAgcDriver/tests/Submit.cpp | 24 +++++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index cf7a795c0..7deea9485 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -613,6 +613,14 @@ void ColorPipeBankXorTests() { } } +void HeaderlessVertexStageTests() { + const std::array userData{1, 2, 3, 4}; + const auto info = AgcDriver::Graphics::DecodeVertexStageInfo({}, 0, userData); + Require(!info.fetchEmbedded && info.resourcesNum == 0, "a vertex program without an AGC header received an embedded fetch table"); + const std::array truncated{}; + expectFailure([&] { static_cast(AgcDriver::Graphics::DecodeVertexStageInfo(truncated, 0x1000, userData)); }, "smaller than the fixed AGC header"); +} + void PixelInputLayoutTests() { using ShaderRecompiler::PixelInput; using ShaderRecompiler::PixelInputVgpr; @@ -4038,6 +4046,7 @@ int main() { TuningFieldTests(); PixelInputLayoutTests(); ColorPipeBankXorTests(); + HeaderlessVertexStageTests(); ComputeScratchTests(); shaderUserDataTailPaddingTests(); InitialContextTests(); diff --git a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp index 329b4e42b..4671c0087 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp @@ -565,6 +565,29 @@ void testRegisteredFloatMode() { } } +void testProgramSnapshots() { + alignas(256) static std::array registered{}; + registered.fill(0xbf800000); + registered[0] = 0xbf810000; + alignas(256) static std::array raw{}; + raw.fill(0xbf800000); + raw[1] = 0xbf810000; + const auto registeredAddress = reinterpret_cast(registered.data()); + const auto rawAddress = reinterpret_cast(raw.data()); + AgcDriver::DriverDetail::ShaderRegistry shaders; + AgcDriver::DriverDetail::ShaderSnapshot snapshot{registeredAddress, 0x1000, 2, {}, {}}; + snapshot.code.assign(registered.begin(), registered.end()); + snapshot.header.resize(sizeof(Shader)); + const auto entry = std::make_shared(std::move(snapshot)); + shaders.emplace(registeredAddress, entry); + check(AgcDriver::DriverDetail::ProgramSnapshot(shaders, registeredAddress) == entry, "a registered program was not resolved to its registration"); + check(AgcDriver::DriverDetail::ProgramSnapshot(shaders, registeredAddress + 4 * 8) == entry, "an entry inside registered code was not resolved to its registration"); + const auto unregistered = AgcDriver::DriverDetail::ProgramSnapshot(shaders, rawAddress); + check(unregistered != entry && unregistered->codeAddress == rawAddress && unregistered->header.empty() && unregistered->code.size() == 2, "an unregistered program was not read as raw code"); + check(AgcDriver::DriverDetail::ProgramSnapshot({}, rawAddress) == unregistered, "an empty registry did not fall back to raw code"); + check(!expectFailure([] { static_cast(AgcDriver::DriverDetail::ProgramSnapshot({}, 0)); }).empty(), "an unmapped unregistered program was accepted"); +} + void testWorkerFailure() { std::array words{0xc0031500, 1, 1, 1, 0x41}; Packet packet{words.data(), static_cast(words.size()), 0, {}}; @@ -656,6 +679,7 @@ int main() { check(VirtualFree(mapping, 0, MEM_RELEASE) != 0, "cannot release raw compute boundary test"); check(!unterminated.empty() && bounded->code.size() == 64, "raw compute crossed inaccessible memory or missed its last instruction"); #endif + testProgramSnapshots(); testEvents(); testValidation(); testClearState(); From 7373ee1311aa9c73906081ffe999a5fa4224dad8 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Thu, 8 Oct 2026 14:53:43 -0500 Subject: [PATCH 265/460] feat(agc): prepare unregistered graphics programs at first use Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 2 + .../src/Driver/Shaders/ShaderRegistry.cpp | 8 +- .../PreparedUnregisteredGraphics.cpp | 95 +++++++++++++++++++ 3 files changed, 103 insertions(+), 2 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/PreparedUnregisteredGraphics.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 866503983..bea3eaaba 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -934,6 +934,8 @@ target_include_directories(agc_driver_prepared_publication_tests PRIVATE $) target_link_libraries(agc_driver_interpolation_modes_tests PRIVATE shader_recompiler_agc_driver) +target_include_directories(agc_driver_prepared_unregistered_graphics_tests PRIVATE $) + target_include_directories(agc_driver_prepared_shaders_tests PRIVATE $ ) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp index 47554a4e2..8e6e210c8 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp @@ -278,8 +278,12 @@ void ReportPreparedAtUse(const ShaderSnapshot& snapshot, const ShaderRecompiler: if (!snapshot.prepared->deferred) APS5_LOG_ERR("Shader 0x%llx stage %u has no artifact prepared at registration for the state of this draw or dispatch; preparing it at use", static_cast(request.shader.codeAddress), static_cast(request.shader.stage)); return; } - if (snapshot.type != 0 || request.shader.stage != ShaderRecompiler::ShaderStage::Compute) throw std::runtime_error("AGC driver: unregistered program is not a compute shader"); - APS5_LOG_ERR("Compute shader 0x%llx was not registered; preparing its artifact at dispatch", static_cast(snapshot.codeAddress)); + if (request.shader.stage == ShaderRecompiler::ShaderStage::Compute) { + if (snapshot.type != 0) throw std::runtime_error("AGC driver: unregistered program is not a compute shader"); + APS5_LOG_ERR("Compute shader 0x%llx was not registered; preparing its artifact at dispatch", static_cast(snapshot.codeAddress)); + return; + } + APS5_LOG_ERR("Graphics shader 0x%llx was not registered; preparing its artifact at first use", static_cast(snapshot.codeAddress)); } } diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedUnregisteredGraphics.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedUnregisteredGraphics.cpp new file mode 100644 index 000000000..c51f19029 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedUnregisteredGraphics.cpp @@ -0,0 +1,95 @@ +#include "VulkanTestDevice.hpp" +#include "prx/libSceAgcDriver/Execution/include/ShaderPreparation.hpp" +#include "SceShaders.hpp" +#include "prx/libSceAgcDriver/Execution/include/Driver/Draw/DrawCache.hpp" +#include "prx/libSceAgcDriver/Graphics/include/ShaderInputState.hpp" +#include "Optimization/ResourceProgram.hpp" +#include +#include +#include +#include +#include +#include + +namespace { + +void Require(bool condition, const char* message) { + if (!condition) throw std::runtime_error(message); +} + +alignas(256) constexpr std::array VertexCode{0xbf810000u}; +alignas(256) constexpr std::array PixelCode{0xc8020002u, 0xc8060102u, 0xc80a0202u, 0xc80e0302u, 0xf800180fu, 0x03020100u, 0xbf810000u}; + +void Bind(AgcDriver::QueueState& queue, const void* code, std::uint32_t program, std::uint32_t resources) { + const auto address = reinterpret_cast(code); + queue.shader[program] = static_cast(address >> 8u); + queue.shader[program + 1u] = static_cast(address >> 40u); + queue.shader[resources] = 16u << 1u; +} + +void Check(AgcDriver::VulkanDevice& device) { + using namespace AgcDriver::DriverDetail; + using namespace ShaderRecompiler; + AgcDriver::QueueState queue{}; + queue.context[0x8e] = 0xfu; + queue.context[0x8f] = 0xfu; + Bind(queue, VertexCode.data(), 0xc8u, 0x8bu); + Bind(queue, PixelCode.data(), 0x8u, 0xbu); + DrawDecode draw{}; + draw.state.stages.path = AgcDriver::Graphics::ShaderPath::Vertex; + draw.state.stages.vertexWaveSize = 32u; + draw.state.stages.fragmentWaveSize = 32u; + draw.pixel.wave32 = true; + draw.pixel.interpolatorCount = 2; + draw.pixel.interpolatorSettings[0] = 0x403u; + draw.pixel.interpolatorSettings[1] = 0x220u; + draw.pixel.targetOutputMode[0] = 9; + draw.pixel.targetExportMapping.fill(0xe4u); + const ShaderRegistry registry; + DecodeGraphicsPrograms(draw, queue, registry, false, true); + Require(draw.programs.size() == 2 && draw.roles.back() == ProgramRole::Fragment, "unregistered graphics programs did not decode as a vertex and a fragment stage"); + for (const auto& program : draw.programs) Require(program.snapshot != nullptr && program.snapshot->header.empty() && program.snapshot->prepared->entries.empty(), "an unregistered graphics program was not read as raw code"); + const auto target = device.Target(); + std::vector linked; + std::vector memory; + for (std::size_t index = 0; index < draw.programs.size(); ++index) { + const auto& program = draw.programs[index]; + linked.push_back({draw.roles[index], program.binary, program.userDataBase, program.firstUserSgpr, program.userData}); + memory.insert(memory.end(), program.memory.begin(), program.memory.end()); + } + std::array variants{}; + for (std::size_t index = 0; index < draw.programs.size(); ++index) { + const auto& program = draw.programs[index]; + const bool pixel = program.binary.stage == ShaderStage::Fragment; + std::optional vertex; + if (!pixel) vertex = AgcDriver::Graphics::DecodeVertexStageInfo(program.binary.header, program.binary.headerAddress, program.userData, nullptr, true); + RecompileRequest request{program.binary, {32u, program.firstUserSgpr, program.userData, {}, pixel ? std::optional(draw.pixel) : std::nullopt, vertex, memory}, target, {0, 0, 0, 128u}, GraphicsCompileContext{program.firstUserSgpr, linked, std::nullopt, std::nullopt, {0, 3, 4, 1}}}; + static_cast(InvocationFor(*program.snapshot, program.codeOffset, request)); + Require(program.snapshot->prepared->entries.size() == 1, "an unregistered graphics program was not prepared at first use"); + variants[index] = GetPreparedArtifact(*program.snapshot->prepared->entries.front().handle).variantId; + static_cast(InvocationFor(*program.snapshot, program.codeOffset, request)); + Require(program.snapshot->prepared->entries.size() == 1, "a prepared unregistered graphics program was prepared again"); + request.layout.pushConstantOffsetBytes = 4; + request.layout.pushConstantSizeBytes -= 4; + static_cast(InvocationFor(*program.snapshot, program.codeOffset, request)); + Require(program.snapshot->prepared->entries.size() == 1, "a relocated push-constant block prepared an unregistered graphics program again"); + } + ResolvePreparedGraphics(*draw.programs[0].snapshot, draw.programs[1].snapshot, 17, target); + const auto rectangle = PreparedRectangle(*draw.programs[0].snapshot, variants[0], variants[1]); + Require(!rectangle.control.spirv.empty() && !rectangle.evaluation.spirv.empty(), "unregistered graphics programs did not prepare rectangle shaders"); +} + +} + +int main() { + try { + auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + Check(*device); + std::cout << "prepared unregistered graphics tests passed\n"; + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} From d1d1681bae0a1b97e5c7f88e17824a60b9d0a73e Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Thu, 8 Oct 2026 16:04:29 -0500 Subject: [PATCH 266/460] fix(agc): drop a cached draw whose unregistered program was rewritten The register draw key identifies an unregistered program only by its address, and a key hit reuses the entry's decode without reading the program again. A different program uploaded to the same address with the same registers would run the old compiled shader. A hit now checks the decode's unregistered programs against guest memory and drops the entry when one changed. Co-Authored-By: Claude Opus 5.5 --- .../Execution/include/Driver/Draw/DrawCache.hpp | 2 ++ .../Execution/include/Driver/Driver.hpp | 1 + .../Execution/src/Driver/Draw/Draw.cpp | 7 +++++++ .../Execution/src/Driver/Draw/DrawCache.cpp | 17 +++++++++++++++++ core/libs/prx/libSceAgcDriver/tests/Submit.cpp | 13 +++++++++++++ 5 files changed, 40 insertions(+) diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Draw/DrawCache.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Draw/DrawCache.hpp index 16d7a286e..06d954d3b 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Draw/DrawCache.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Draw/DrawCache.hpp @@ -81,6 +81,8 @@ struct StageCapture { std::uint32_t pushOffset = 0; }; +bool DecodeProgramsCurrent(const DrawDecode& decode, const ShaderRegistry& registry); + } #endif diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp index becdafd93..cc387b5e7 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp @@ -114,6 +114,7 @@ private: static std::size_t drawCacheEntries(); void accountDrawVariant(const DispatchVariant& variant, bool added); void insertDrawEntry(std::uint64_t key, std::vector>& fresh, std::shared_ptr decode); + void eraseDrawEntry(std::uint64_t key, const std::shared_ptr& expected); std::shared_ptr findDrawRecipe(std::uint64_t key, const std::vector>& stages); void attachDrawRecipe(std::uint64_t key, const std::vector>& stages, std::shared_ptr recipe); void reportDrawCache(DrawEntryCounters& counters); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp index 26bc065d8..fe033b378 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp @@ -81,6 +81,13 @@ DrawVerdict Driver::draw(QueueState& queue, std::span packe ++drawEntryCounters.absent; } } + if (decode != nullptr && !DecodeProgramsCurrent(*decode, *submission.shaders)) { + std::lock_guard cacheLock(drawCacheMutex); + eraseDrawEntry(drawKey, entry); + ++drawEntryCounters.absent; + entry = nullptr; + decode = nullptr; + } phaseTiming.Phase(DrawRowKeyLookupValidate); resolveDrawDecode(queue, submission, decode, registerKey, drawKey, profile); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawCache.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawCache.cpp index bcd1a44fa..5287b7126 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawCache.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawCache.cpp @@ -1,5 +1,6 @@ #include "prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp" #include "prx/libSceAgcDriver/Execution/include/Driver/Draw/DrawCache.hpp" +#include #include namespace AgcDriver::DriverDetail { @@ -107,6 +108,22 @@ void Driver::insertDrawEntry(std::uint64_t key, std::vector& expected) { + const auto found = drawCache.find(key); + if (found == drawCache.end() || found->second != expected) return; + for (const auto& variants : found->second->stages) { + for (const auto& variant : variants) accountDrawVariant(*variant, false); + } + drawOrder.erase(found->second->order); + drawCache.erase(found); +} + +bool DecodeProgramsCurrent(const DrawDecode& decode, const ShaderRegistry& registry) { + return std::all_of(decode.programs.begin(), decode.programs.end(), [&](const DrawProgram& program) { + return !program.snapshot->header.empty() || ProgramSnapshot(registry, program.snapshot->codeAddress + program.codeOffset * sizeof(std::uint32_t)) == program.snapshot; + }); +} + std::shared_ptr Driver::findDrawRecipe(std::uint64_t key, const std::vector>& stages) { std::shared_ptr> records; { diff --git a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp index 4671c0087..555de2555 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp @@ -1,6 +1,7 @@ #include "prx/libSceAgcDriver/Execution/include/Driver.hpp" #include "execution/VulkanTestDevice.hpp" #include "prx/libSceAgcDriver/Execution/include/QueueState.hpp" +#include "prx/libSceAgcDriver/Execution/include/Driver/Draw/DrawCache.hpp" #include "prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp" #include "prx/libSceAgcDriver/Execution/include/Driver/Synchronization/SynchronizationStatistics.hpp" #include "prx/libc/include/GuestHeap.hpp" @@ -586,6 +587,18 @@ void testProgramSnapshots() { check(unregistered != entry && unregistered->codeAddress == rawAddress && unregistered->header.empty() && unregistered->code.size() == 2, "an unregistered program was not read as raw code"); check(AgcDriver::DriverDetail::ProgramSnapshot({}, rawAddress) == unregistered, "an empty registry did not fall back to raw code"); check(!expectFailure([] { static_cast(AgcDriver::DriverDetail::ProgramSnapshot({}, 0)); }).empty(), "an unmapped unregistered program was accepted"); + AgcDriver::DriverDetail::DrawDecode decode{}; + decode.programs.resize(2); + decode.programs[0].snapshot = unregistered; + decode.programs[1].snapshot = entry; + decode.programs[1].codeOffset = 8; + check(AgcDriver::DriverDetail::DecodeProgramsCurrent(decode, shaders), "an unchanged decode was not current"); + registered[8] = 0xbf810000; + check(AgcDriver::DriverDetail::DecodeProgramsCurrent(decode, shaders), "a rewritten registered program made a decode stale"); + raw[0] = 0xbf810000; + check(!AgcDriver::DriverDetail::DecodeProgramsCurrent(decode, shaders), "a rewritten unregistered program left its decode current"); + decode.programs[0].snapshot = AgcDriver::DriverDetail::ProgramSnapshot(shaders, rawAddress); + check(AgcDriver::DriverDetail::DecodeProgramsCurrent(decode, shaders), "a decode with the rewritten program was not current"); } void testWorkerFailure() { From 4bf608565fc90f6e2d422529ed066475544c57f6 Mon Sep 17 00:00:00 2001 From: xXPBSXx <20614131+xXPBSXx@users.noreply.github.com> Date: Sun, 11 Oct 2026 00:51:07 +0200 Subject: [PATCH 267/460] fix(libSceSystemGesture): report no touch events for the primitive and recognizer calls WIP: the TechnicalDebt entry for the silently successful calls is still missing. Co-Authored-By: Claude Sonnet 5.5 --- core/libs/CMakeLists.txt | 10 +++ core/libs/prx/libSceSystemGesture/Export.cpp | 87 +++++++++----------- core/libs/tests/GuestSystemGesture.cpp | 72 ++++++++++++++++ 3 files changed, 122 insertions(+), 47 deletions(-) create mode 100644 core/libs/tests/GuestSystemGesture.cpp diff --git a/core/libs/CMakeLists.txt b/core/libs/CMakeLists.txt index 49fa7bcb3..cb240d7dd 100644 --- a/core/libs/CMakeLists.txt +++ b/core/libs/CMakeLists.txt @@ -173,6 +173,16 @@ if(BUILD_TESTING) "PATH=path_list_prepend:$") endif() + add_test_executable(guest_system_gesture_tests tests/GuestSystemGesture.cpp) + target_include_directories(guest_system_gesture_tests PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}) + target_link_libraries(guest_system_gesture_tests PRIVATE libSceSystemGesture libc) + configure_windows_unwind(guest_system_gesture_tests) + add_test(NAME guest_system_gesture COMMAND guest_system_gesture_tests) + if(WIN32) + set_tests_properties(guest_system_gesture PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() + if(UNIX AND (CMAKE_SYSTEM_PROCESSOR MATCHES "^(x86_64|AMD64)$" OR CMAKE_OSX_ARCHITECTURES STREQUAL "x86_64")) add_subdirectory(tests/register_context) endif() diff --git a/core/libs/prx/libSceSystemGesture/Export.cpp b/core/libs/prx/libSceSystemGesture/Export.cpp index 61f241f15..9c71e1031 100644 --- a/core/libs/prx/libSceSystemGesture/Export.cpp +++ b/core/libs/prx/libSceSystemGesture/Export.cpp @@ -12,10 +12,9 @@ static constexpr int SCE_SYSTEM_GESTURE_ERROR_INDEX_OUT_OF_ARRAY = static_cast +#include +#include + +extern "C" { +std::int32_t APS5_VABI sceSystemGestureOpen(std::int32_t, const void*); +int APS5_VABI sceSystemGestureGetPrimitiveTouchEvents(std::int32_t, void*, std::uint32_t, std::uint32_t*); +int APS5_VABI sceSystemGestureGetPrimitiveTouchEventsCount(std::int32_t); +int APS5_VABI sceSystemGestureGetTouchEvents(std::int32_t, const void*, void*, std::uint32_t, std::uint32_t*); +int APS5_VABI sceSystemGestureGetPrimitiveTouchEventByIndex(std::int32_t, std::uint32_t, void*); +int APS5_VABI sceSystemGestureAppendTouchRecognizer(std::int32_t, void*); +int APS5_VABI sceSystemGestureRemoveTouchRecognizer(std::int32_t, void*); +int APS5_VABI sceSystemGestureResetTouchRecognizer(std::int32_t, void*); +int APS5_VABI sceSystemGestureUpdateTouchRecognizer(std::int32_t, void*); +int APS5_VABI sceSystemGestureUpdateTouchRecognizerRectangle(std::int32_t, void*, const void*); +int APS5_VABI sceSystemGestureResetPrimitiveTouchRecognizer(std::int32_t); +int APS5_VABI sceSystemGestureUpdateAllTouchRecognizer(std::int32_t); +int APS5_VABI sceSystemGestureUpdatePrimitiveTouchRecognizer(std::int32_t, const void*); +} + +static void Require(bool value) { if (!value) std::abort(); } + +int main() { + constexpr int invalidArgument = static_cast(0x80D10002); + constexpr int invalidHandle = static_cast(0x80D10003); + const std::int32_t handle = sceSystemGestureOpen(0, nullptr); + + std::array buffer; + buffer.fill(0x5a); + const auto original = buffer; + std::uint32_t count = 7; + Require(sceSystemGestureGetPrimitiveTouchEvents(handle, buffer.data(), 4, &count) == 0); + Require(count == 0); + Require(buffer == original); + + count = 7; + Require(sceSystemGestureGetPrimitiveTouchEvents(handle, buffer.data(), 4, nullptr) == invalidArgument); + Require(sceSystemGestureGetPrimitiveTouchEvents(handle + 1, buffer.data(), 4, &count) == invalidHandle); + Require(count == 7); + + Require(sceSystemGestureGetPrimitiveTouchEventsCount(handle) == 0); + Require(sceSystemGestureGetPrimitiveTouchEventsCount(handle + 1) == invalidHandle); + + constexpr int indexOutOfArray = static_cast(0x80D10005); + int recognizer = 0; + int rectangle = 0; + count = 7; + Require(sceSystemGestureGetTouchEvents(handle, &recognizer, buffer.data(), 4, &count) == 0 && count == 0); + Require(sceSystemGestureGetTouchEvents(handle, &recognizer, buffer.data(), 4, nullptr) == invalidArgument); + Require(sceSystemGestureGetTouchEvents(handle + 1, &recognizer, buffer.data(), 4, &count) == invalidHandle); + Require(sceSystemGestureGetPrimitiveTouchEventByIndex(handle, 0, buffer.data()) == indexOutOfArray); + Require(sceSystemGestureGetPrimitiveTouchEventByIndex(handle + 1, 0, buffer.data()) == invalidHandle); + Require(buffer == original); + + Require(sceSystemGestureAppendTouchRecognizer(handle, &recognizer) == 0); + Require(sceSystemGestureRemoveTouchRecognizer(handle, &recognizer) == 0); + Require(sceSystemGestureResetTouchRecognizer(handle, &recognizer) == 0); + Require(sceSystemGestureUpdateTouchRecognizer(handle, &recognizer) == 0); + Require(sceSystemGestureUpdateTouchRecognizerRectangle(handle, &recognizer, &rectangle) == 0); + Require(sceSystemGestureResetPrimitiveTouchRecognizer(handle) == 0); + Require(sceSystemGestureUpdateAllTouchRecognizer(handle) == 0); + Require(sceSystemGestureUpdatePrimitiveTouchRecognizer(handle, nullptr) == 0); + Require(sceSystemGestureAppendTouchRecognizer(handle, nullptr) == invalidArgument); + Require(sceSystemGestureUpdateTouchRecognizerRectangle(handle, &recognizer, nullptr) == invalidArgument); + Require(sceSystemGestureAppendTouchRecognizer(handle + 1, &recognizer) == invalidHandle); + Require(sceSystemGestureResetPrimitiveTouchRecognizer(handle + 1) == invalidHandle); + Require(sceSystemGestureUpdateAllTouchRecognizer(handle + 1) == invalidHandle); + Require(sceSystemGestureUpdatePrimitiveTouchRecognizer(handle + 1, nullptr) == invalidHandle); + return 0; +} From 6e2ea0a071292f2ae51912acb86fbc9cc49fe9b7 Mon Sep 17 00:00:00 2001 From: xXPBSXx <20614131+xXPBSXx@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:00:09 +0200 Subject: [PATCH 268/460] docs(technical-debt): list the system gesture no-touch behavior Co-Authored-By: Claude Sonnet 5.5 --- docs/dev/TechnicalDebt.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 21a722572..f826f0d33 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -513,3 +513,4 @@ Throughout the project, every function at every stage either **does exactly what - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert +- [System gesture recognizers](../../core/libs/prx/libSceSystemGesture/Export.cpp) (libSceSystemGesture) - `sceSystemGestureGetPrimitiveTouchEvents`, `GetPrimitiveTouchEventsCount`, `GetTouchEvents`, `GetPrimitiveTouchEventByIndex` and the recognizer append, remove, reset and update calls report that no touch input exists and keep no recognizer state, so a title never receives a gesture event. Handle and pointer arguments are still checked. From 894ef6db0949a2a4ee86026ef1babbfd39a8de6e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 02:04:04 +0200 Subject: [PATCH 269/460] test(libSceSystemGesture): cover the handle and null checks of every recognizer call Moves the no-touch line to "Silent stubs" and records the error-code disagreement with KytyPS5. Co-Authored-By: Claude --- core/libs/tests/GuestSystemGesture.cpp | 8 ++++++++ docs/dev/TechnicalDebt.md | 2 +- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/core/libs/tests/GuestSystemGesture.cpp b/core/libs/tests/GuestSystemGesture.cpp index 26fdd9468..037b9e885 100644 --- a/core/libs/tests/GuestSystemGesture.cpp +++ b/core/libs/tests/GuestSystemGesture.cpp @@ -65,6 +65,14 @@ int main() { Require(sceSystemGestureAppendTouchRecognizer(handle, nullptr) == invalidArgument); Require(sceSystemGestureUpdateTouchRecognizerRectangle(handle, &recognizer, nullptr) == invalidArgument); Require(sceSystemGestureAppendTouchRecognizer(handle + 1, &recognizer) == invalidHandle); + Require(sceSystemGestureRemoveTouchRecognizer(handle, nullptr) == invalidArgument); + Require(sceSystemGestureResetTouchRecognizer(handle, nullptr) == invalidArgument); + Require(sceSystemGestureUpdateTouchRecognizer(handle, nullptr) == invalidArgument); + Require(sceSystemGestureUpdateTouchRecognizerRectangle(handle, nullptr, &rectangle) == invalidArgument); + Require(sceSystemGestureRemoveTouchRecognizer(handle + 1, &recognizer) == invalidHandle); + Require(sceSystemGestureResetTouchRecognizer(handle + 1, &recognizer) == invalidHandle); + Require(sceSystemGestureUpdateTouchRecognizer(handle + 1, &recognizer) == invalidHandle); + Require(sceSystemGestureUpdateTouchRecognizerRectangle(handle + 1, &recognizer, &rectangle) == invalidHandle); Require(sceSystemGestureResetPrimitiveTouchRecognizer(handle + 1) == invalidHandle); Require(sceSystemGestureUpdateAllTouchRecognizer(handle + 1) == invalidHandle); Require(sceSystemGestureUpdatePrimitiveTouchRecognizer(handle + 1, nullptr) == invalidHandle); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index f826f0d33..89a676fd4 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -29,6 +29,7 @@ Throughout the project, every function at every stage either **does exactly what - [libSceSystemService](../../core/libs/prx/libSceSystemService/Export.cpp) - `sceSystemServicePowerTick`, `sceSystemServiceReportAbnormalTermination`, `sceSystemServiceDisableMusicPlayer`, `sceSystemServiceReenableMusicPlayer`, `sceSystemServiceDisableMediaPlay` and `sceSystemServiceReenableMediaPlay` return success and do nothing: the host has no auto power-off timer to reset, no system crash reporter to notify, no system music player to pause or resume and no system media playback to disable. PPSA01521 calls `sceSystemServiceDisableMediaPlay` during start-up, and its wrappers for both pass no arguments - [sceSystemServiceLaunchWebBrowser](../../core/libs/prx/libSceSystemService/Export.cpp) (libSceSystemService) - AnyPS5 launches no web browser, so the call fails and nothing is shown. RommPS PPSA76677 then tells the player to scan its QR code instead - [libSceBluetoothHid](../../core/libs/prx/libSceBluetoothHid/Export.cpp) implements only the no-device path: initialization and callback and device registration succeed, the callback is never called because no device connects, and calls that need a connected device throw +- [System gesture recognizers](../../core/libs/prx/libSceSystemGesture/Export.cpp) (libSceSystemGesture) - `sceSystemGestureGetPrimitiveTouchEvents`, `GetPrimitiveTouchEventsCount`, `GetTouchEvents`, `GetPrimitiveTouchEventByIndex` and the recognizer append, remove, reset and update calls report that no touch input exists and keep no recognizer state, so a title never receives a gesture event. Handle and pointer arguments are still checked. Error codes follow the 0x80D1 values main already uses in this file; KytyPS5 uses 0x8089 (INVALID_ARGUMENT 0x80890001, INVALID_HANDLE 0x80890006), and which one the console returns is not verified - [sceVideoOutOpen](../../core/libs/prx/libSceVideoOut/src/Output.cpp) validates the priority and CPU affinity that the open param requests for the VideoOut service thread but does not apply them: the port's present and vblank threads are host threads, and guest priorities and affinities do not reach host scheduling - [sceVideoOutAddVrrStatusFlagsPrivilege](../../core/libs/prx/libSceVideoOut/src/Output.cpp) returns 0 and does nothing: the host output has no VRR state to report. The name comes from an assert message in a title that requires `SCE_VIDEO_OUT_OK`; the NID hashes to it, the arguments are unknown (it is called without any) - [libSceAudio3d](../../core/libs/prx/libSceAudio3d/Export.cpp) - the port produces no sound: objects can be reserved and `sceAudio3dObjectSetAttributes` accepts their attributes, but the library has no bed exports or mixing to give them audio, the attributes have no effect and `sceAudio3dPortPush` only paces the queue at `granularity` samples per frame at 48 kHz. `sceAudio3dPortSetAttribute` accepts the late reverb level and the downmix spread attributes and ignores them. `sceAudio3dObjectSetAttributes` validates the port, the reserved object and the attribute array (an object id that was never reserved returns the invalid-object error, while shadPS4 returns success for it and the error only for a released object; neither is measured on a console), treating the reset-state attribute as valid with a null value and requiring a value for the others; the attribute identifiers and the value of each are not interpreted. `sceAudio3dPortFlush` drops the frames queued by `sceAudio3dPortAdvance` instead of mixing and submitting them to an output, as there is no output @@ -513,4 +514,3 @@ Throughout the project, every function at every stage either **does exactly what - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert -- [System gesture recognizers](../../core/libs/prx/libSceSystemGesture/Export.cpp) (libSceSystemGesture) - `sceSystemGestureGetPrimitiveTouchEvents`, `GetPrimitiveTouchEventsCount`, `GetTouchEvents`, `GetPrimitiveTouchEventByIndex` and the recognizer append, remove, reset and update calls report that no touch input exists and keep no recognizer state, so a title never receives a gesture event. Handle and pointer arguments are still checked. From 064f5ca8251f117709900a3f694f31168f7daa82 Mon Sep 17 00:00:00 2001 From: David Date: Sun, 11 Oct 2026 01:02:57 +0200 Subject: [PATCH 270/460] feat(libkernel): implement pthread_attr_setsolosched_np The POSIX alias takes (attr, int), returns EINVAL (22) for a null or uninitialized attribute like the other pthread_attr setters, and stores the value through scePthreadAttrSetsolosched, returning its result in errno form. The signature and the stored int follow the SCE twin in KytyPS5 and sharpemu; PS5PCEM implements the alias the same way. The value is not applied to scheduling, recorded in TechnicalDebt. Returnal (PPSA01285 v03.000.005) stops on it after eager binding (#2872, #1155): what(): pthread_attr_setsolosched_np_nid_postfix not implemented LEGO Horizon Adventures (PPSA14632, #734) and Palworld (PPSA23126, #2598) import it. Co-Authored-By: Claude Fable 5.1 --- core/libs/prx/libkernel/Pthread/Posix/Attr.cpp | 7 ++++--- core/libs/tests/GuestPthreadAttr.cpp | 18 ++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 23 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libkernel/Pthread/Posix/Attr.cpp b/core/libs/prx/libkernel/Pthread/Posix/Attr.cpp index a05f78402..a1b80b004 100644 --- a/core/libs/prx/libkernel/Pthread/Posix/Attr.cpp +++ b/core/libs/prx/libkernel/Pthread/Posix/Attr.cpp @@ -10,6 +10,7 @@ int APS5_VABI scePthreadAttrInit(PthreadAttr* attr); int APS5_VABI scePthreadAttrDestroy(PthreadAttr* attr); int APS5_VABI scePthreadAttrSetstacksize(PthreadAttr* attr, std::size_t stacksize); int APS5_VABI scePthreadAttrSetstack(PthreadAttr* attr, void* addr, std::size_t size); +int APS5_VABI scePthreadAttrSetsolosched(PthreadAttr* attr, int solosched); } static bool Valid(const PthreadAttr* attr) { @@ -116,9 +117,9 @@ int APS5_VABI pthread_attr_setstack_nid_postfix(PthreadAttr* attr, void* stack_a return PosixThread::ToErrno(scePthreadAttrSetstack(attr, stack_addr, stack_size)); } -int APS5_VABI pthread_attr_setsolosched_np_nid_postfix() { - NotImplemented_nid_no_patch(__func__); - return 0; +int APS5_VABI pthread_attr_setsolosched_np_nid_postfix(PthreadAttr* attr, int solosched) { + if (!Valid(attr)) return PosixThread::GUEST_EINVAL; + return PosixThread::ToErrno(scePthreadAttrSetsolosched(attr, solosched)); } } diff --git a/core/libs/tests/GuestPthreadAttr.cpp b/core/libs/tests/GuestPthreadAttr.cpp index 8588e2ae1..ee24e0000 100644 --- a/core/libs/tests/GuestPthreadAttr.cpp +++ b/core/libs/tests/GuestPthreadAttr.cpp @@ -19,7 +19,9 @@ int APS5_VABI scePthreadAttrGetschedparam(const PthreadAttr* attr, KernelSchedPa int APS5_VABI scePthreadAttrGetaffinity(const PthreadAttr* attr, KernelCpumask* mask); int APS5_VABI scePthreadAttrGetstacksize(const PthreadAttr* attr, std::size_t* stackSize); int APS5_VABI scePthreadAttrGetdetachstate(const PthreadAttr* attr, int* state); +int APS5_VABI scePthreadAttrGetsolosched(const PthreadAttr* attr, int* solosched); int APS5_VABI pthread_attr_setstacksize_nid_postfix(PthreadAttr* attr, std::size_t stackSize); +int APS5_VABI pthread_attr_setsolosched_np_nid_postfix(PthreadAttr* attr, int solosched); } static constexpr int SCE_OK = 0; @@ -76,6 +78,21 @@ static void CheckStackSizeLimit() { Require(scePthreadAttrDestroy(&attr) == SCE_OK); } +static void CheckSoloSched() { + PthreadAttr attr = nullptr; + Require(pthread_attr_setsolosched_np_nid_postfix(nullptr, 1) == GUEST_EINVAL); + Require(pthread_attr_setsolosched_np_nid_postfix(&attr, 1) == GUEST_EINVAL); + Require(scePthreadAttrInit(&attr) == SCE_OK); + int soloSched = -1; + Require(scePthreadAttrGetsolosched(&attr, &soloSched) == SCE_OK && soloSched == 0); + Require(pthread_attr_setsolosched_np_nid_postfix(&attr, 1) == 0); + Require(scePthreadAttrGetsolosched(&attr, &soloSched) == SCE_OK && soloSched == 1); + Require(pthread_attr_setsolosched_np_nid_postfix(&attr, 0) == 0); + Require(scePthreadAttrGetsolosched(&attr, &soloSched) == SCE_OK && soloSched == 0); + Require(scePthreadAttrDestroy(&attr) == SCE_OK); + Require(pthread_attr_setsolosched_np_nid_postfix(&attr, 1) == GUEST_EINVAL); +} + static void* APS5_VABI Worker(void* arg) { static_cast*>(arg)->get(); return nullptr; @@ -83,6 +100,7 @@ static void* APS5_VABI Worker(void* arg) { int main() { CheckStackSizeLimit(); + CheckSoloSched(); PthreadAttr attr = nullptr; Require(scePthreadAttrInit(&attr) == SCE_OK); Require(scePthreadAttrSetinheritsched(&attr, EXPLICIT_SCHED) == SCE_OK); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 337ef24ed..133ed999f 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -514,3 +514,4 @@ Throughout the project, every function at every stage either **does exactly what - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert +- [pthread_attr_setsolosched_np](../../core/libs/prx/libkernel/Pthread/Posix/Attr.cpp) (libkernel) takes `(attr, int)` and stores the value through `scePthreadAttrSetsolosched`, returning `EINVAL` (22) for a null or uninitialized attribute like the other POSIX attribute setters. Only the name is public (aerolib `2+pVfgiEd7A`); the signature and behaviour follow the SCE twin, which [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/947627d10d4ef4b68ba3c0189aab6c368e70a8d2/src/kernel/pthread.cpp#L2188-L2199) and [sharpemu](https://github.com/sharpemu/sharpemu/blob/a2983692930564815a77f81e74c151268f5b935f/src/SharpEmu.Libs/Kernel/KernelPthreadExtendedCompatExports.cs#L891-L913) store as an `int` without a range check, and [PS5PCEM](https://github.com/iStark/PS5PCEM/blob/71de7681f06c5db59a1120a3bb554c86780c3f5f/src/hle/libs/kernel_threading.zig#L1396-L1398) implements this alias over it. Any value is accepted and threads created from the attribute are not scheduled differently; what solo scheduling does on the console and which values it rejects are unverified. Returnal (PPSA01285 v03.000.005) stopped on it with `what(): pthread_attr_setsolosched_np_nid_postfix not implemented` (#2872, #1155); LEGO Horizon Adventures (PPSA14632, #734) and Palworld (PPSA23126, #2598) import it From 8d90c1f190bc42435d4525dd8c595fbe1bd41a14 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 02:11:41 +0200 Subject: [PATCH 271/460] docs(technical-debt): list pthread_attr_setsolosched_np under unknown function info Co-Authored-By: Claude --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 133ed999f..5ac053cba 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -154,6 +154,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceKernelAioSubmitReadCommandsMultiple / sceKernelAioSubmitWriteCommandsMultiple / sceKernelAioWaitRequests / sceKernelAioPollRequests / sceKernelAioCancelRequest / sceKernelAioCancelRequests / sceKernelAioDeleteRequests](../../core/libs/prx/libkernel/Aio/src/Aio.cpp) (libkernel) - behaviour from shadPS4 (one id per request for the Multiple submits, cancel turns any request aborted and id 0 reports processing, wait mode 2 returns once one request completed, poll copies every state without waiting); the 128 requests or ids per batch limit is from SharpProspero, with an unknown error, so larger batches throw; wait modes other than 1 (and) and 2 (or) throw; null pointers, negative counts and invalid ids return the codes the single-request siblings use, checked before any state is written - [sceKernelGetAvailableCpumask](../../core/libs/prx/libkernel/Pthread/src/Thread.cpp) (libkernel) - returns the default thread affinity, 0x1FFF (CPUs 0 to 12), as KytyPS5 does. Titles pin threads within it (PPSA02664 passes 0x1FFB, Hades II PPSA36082 0x3 and 0x3F); whether CPU 13 is available to a title is not confirmed - [sceKernelGetCurrentCpu](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - guest affinities are not applied to host threads, so the host CPU index is mapped into the calling thread's affinity clipped to `sceKernelGetAvailableCpumask` (the available mask when they do not overlap): host CPU n reports the (n modulo the CPU count)-th CPU of that mask, so a thread pinned to one CPU reports that CPU, as a FreeBSD `cpuset` guarantees on the console. Which CPU the console reports for a thread allowed on several is not known; shadPS4 returns 0 +- [pthread_attr_setsolosched_np](../../core/libs/prx/libkernel/Pthread/Posix/Attr.cpp) (libkernel) takes `(attr, int)` and stores the value through `scePthreadAttrSetsolosched`, returning `EINVAL` (22) for a null or uninitialized attribute like the other POSIX attribute setters. Only the name is public (aerolib `2+pVfgiEd7A`); the signature and behaviour follow the SCE twin, which [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/947627d10d4ef4b68ba3c0189aab6c368e70a8d2/src/kernel/pthread.cpp#L2188-L2199) and [sharpemu](https://github.com/sharpemu/sharpemu/blob/a2983692930564815a77f81e74c151268f5b935f/src/SharpEmu.Libs/Kernel/KernelPthreadExtendedCompatExports.cs#L891-L913) store as an `int` without a range check, and [PS5PCEM](https://github.com/iStark/PS5PCEM/blob/71de7681f06c5db59a1120a3bb554c86780c3f5f/src/hle/libs/kernel_threading.zig#L1396-L1398) implements this alias over it. Any value is accepted and threads created from the attribute are not scheduled differently; what solo scheduling does on the console and which values it rejects are unverified. Returnal (PPSA01285 v03.000.005) stopped on it with `what(): pthread_attr_setsolosched_np_nid_postfix not implemented` (#2872, #1155); LEGO Horizon Adventures (PPSA14632, #734) and Palworld (PPSA23126, #2598) import it - [sceKernelMapNamedFlexibleMemoryInternal](../../core/libs/prx/libkernel/DirectMemory/Export.cpp) (libkernel) - flag 0x8000 is accepted and ignored; its meaning is not established. KytyPS5 (`GUEST_MAP_UNKNOWN_8000`) and sharpemu accept it without reading it. Every known caller is the SDK 9 `libc.prx` heap (Astro Bot PPSA21564, PPSA22520), which passes it with read/write protection and uses the mapping as an older `libc.prx` uses `sceKernelMapNamedFlexibleMemory` with flags 0 - [sceKernelMtypeprotect](../../core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp) (libkernel) - sets the memory type of the direct mappings in the page-rounded range and of the direct memory behind them, as shadPS4 and prosper do (KytyPS5 ignores the type); the type is not validated, flexible pages in the range only get the protection, and a later map of the retyped direct memory reports the type it was allocated with. `sceKernelBatchMap2` operation 4 takes the same path with the entry type - [Map hints inside a reservation](../../core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp) (libkernel) - `sceKernelMapDirectMemory` and the flexible and anonymous maps place a mapping without `MAP_FIXED` at its hint address when the whole range lies in a range still reserved by `sceKernelReserveVirtualRange`, as if `MAP_FIXED` were set. Taken from PPSA01474, which reserves 64 GiB and then maps its heaps inside it with hints and no `MAP_FIXED`, and uses the hint addresses afterwards; not measured on the console. Hints outside a reservation, or over a range that is partly mapped, stay hints @@ -514,4 +515,3 @@ Throughout the project, every function at every stage either **does exactly what - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert -- [pthread_attr_setsolosched_np](../../core/libs/prx/libkernel/Pthread/Posix/Attr.cpp) (libkernel) takes `(attr, int)` and stores the value through `scePthreadAttrSetsolosched`, returning `EINVAL` (22) for a null or uninitialized attribute like the other POSIX attribute setters. Only the name is public (aerolib `2+pVfgiEd7A`); the signature and behaviour follow the SCE twin, which [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/947627d10d4ef4b68ba3c0189aab6c368e70a8d2/src/kernel/pthread.cpp#L2188-L2199) and [sharpemu](https://github.com/sharpemu/sharpemu/blob/a2983692930564815a77f81e74c151268f5b935f/src/SharpEmu.Libs/Kernel/KernelPthreadExtendedCompatExports.cs#L891-L913) store as an `int` without a range check, and [PS5PCEM](https://github.com/iStark/PS5PCEM/blob/71de7681f06c5db59a1120a3bb554c86780c3f5f/src/hle/libs/kernel_threading.zig#L1396-L1398) implements this alias over it. Any value is accepted and threads created from the attribute are not scheduled differently; what solo scheduling does on the console and which values it rejects are unverified. Returnal (PPSA01285 v03.000.005) stopped on it with `what(): pthread_attr_setsolosched_np_nid_postfix not implemented` (#2872, #1155); LEGO Horizon Adventures (PPSA14632, #734) and Palworld (PPSA23126, #2598) import it From 39fdc48b5f2ec587f11f706371ea3330ba1bbd37 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sat, 10 Oct 2026 19:12:14 -0500 Subject: [PATCH 272/460] fix(agc): read unregistered graphics programs only while no shader is registered An unregistered graphics program is read raw only while the shader registry is empty; with shaders registered it throws, as before. A draw that uses a raw program is noted, and RegisterShader throws once that has happened, since a title that registers shaders after drawing unregistered programs is not understood. The draw cache revalidates a raw program by reading it raw. AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- .../include/Driver/Shaders/ShaderRegistry.hpp | 2 ++ .../Execution/src/Driver/Draw/DrawCache.cpp | 4 ++-- .../Execution/src/Driver/Draw/DrawDecode.cpp | 1 + .../src/Driver/Shaders/ShaderRegistry.cpp | 15 +++++++++++++++ core/libs/prx/libSceAgcDriver/tests/Submit.cpp | 17 +++++++++-------- .../tests/execution/PreparedGraphics.cpp | 1 + .../execution/PreparedUnregisteredGraphics.cpp | 9 +++++++++ 7 files changed, 39 insertions(+), 10 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp index 3ddcc2398..297686f5e 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp @@ -86,6 +86,8 @@ struct ShaderSnapshot { std::shared_ptr ReadRawShader(std::uint64_t address); std::shared_ptr ProgramSnapshot(const ShaderRegistry& shaders, std::uint64_t address, std::initializer_list types); +void NoteRawGraphicsDraw(); +bool RawGraphicsDrawn(); std::shared_ptr PrepareShaderWithDiagnostics(const ShaderRecompiler::RecompileRequest& request, ShaderRecompiler::ShaderPreparationContext* preparation = nullptr); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawCache.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawCache.cpp index 5287b7126..7844679fd 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawCache.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawCache.cpp @@ -118,9 +118,9 @@ void Driver::eraseDrawEntry(std::uint64_t key, const std::shared_ptr& drawCache.erase(found); } -bool DecodeProgramsCurrent(const DrawDecode& decode, const ShaderRegistry& registry) { +bool DecodeProgramsCurrent(const DrawDecode& decode, const ShaderRegistry&) { return std::all_of(decode.programs.begin(), decode.programs.end(), [&](const DrawProgram& program) { - return !program.snapshot->header.empty() || ProgramSnapshot(registry, program.snapshot->codeAddress + program.codeOffset * sizeof(std::uint32_t)) == program.snapshot; + return !program.snapshot->header.empty() || ReadRawShader(program.snapshot->codeAddress + program.codeOffset * sizeof(std::uint32_t)) == program.snapshot; }); } diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp index 60d97fb2f..be0f1794d 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp @@ -32,6 +32,7 @@ void DecodeGraphicsPrograms(DrawDecode& decoded, const QueueState& queue, const const auto& snapshot = *program; require((address - snapshot.codeAddress) % sizeof(std::uint32_t) == 0, "graphics entry point is not dword aligned"); require(snapshot.header.empty() || snapshot.type == type, "graphics program refers to an incompatible shader binary type"); + if (!nullPixel && snapshot.header.empty()) NoteRawGraphicsDraw(); if (!nullPixel) Graphics::NoteRegisterRead(Graphics::RegisterBank::Shader, rsrc2); const auto resources = nullPixel ? 0u : ReadGraphicsRegister(queue.shader, rsrc2); const auto userCount = ((resources >> 1u) & 0x1fu) | (((resources >> 27u) & 1u) << 5u); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp index 8e6e210c8..75fb76947 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp @@ -11,6 +11,7 @@ #include "CompiledVariant.hpp" #include "Optimization/ResourceProgram.hpp" #include +#include #include #include #include @@ -105,9 +106,22 @@ std::shared_ptr ReadRawShader(std::uint64_t address) { std::shared_ptr ProgramSnapshot(const ShaderRegistry& shaders, std::uint64_t address, std::initializer_list types) { if (auto registered = RegisteredProgram(shaders, address, types)) return registered; + require(shaders.empty(), "graphics program does not belong to a compatible registered shader"); return ReadRawShader(address); } +namespace { +std::atomic rawGraphicsDrawn{false}; +} + +void NoteRawGraphicsDraw() { + rawGraphicsDrawn.store(true, std::memory_order_relaxed); +} + +bool RawGraphicsDrawn() { + return rawGraphicsDrawn.load(std::memory_order_relaxed); +} + namespace { void AwaitRegisteredPreparation(PreparedShaders& prepared, std::unique_lock& lock) { @@ -965,6 +979,7 @@ void Driver::RegisterShader(const Shader* shader) { PerformanceTimer timing("Shader.Register"); ShaderPreparationTransaction transaction; CheckFailure(); + require(!RawGraphicsDrawn(), "a shader was registered after graphics programs were drawn without registration"); GuestMemory::CheckRange(shader, sizeof(Shader), 1); Shader fields; std::memcpy(&fields, static_cast(shader), sizeof(Shader)); diff --git a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp index 555de2555..4137e06b3 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp @@ -580,13 +580,14 @@ void testProgramSnapshots() { snapshot.code.assign(registered.begin(), registered.end()); snapshot.header.resize(sizeof(Shader)); const auto entry = std::make_shared(std::move(snapshot)); - shaders.emplace(registeredAddress, entry); - check(AgcDriver::DriverDetail::ProgramSnapshot(shaders, registeredAddress) == entry, "a registered program was not resolved to its registration"); - check(AgcDriver::DriverDetail::ProgramSnapshot(shaders, registeredAddress + 4 * 8) == entry, "an entry inside registered code was not resolved to its registration"); - const auto unregistered = AgcDriver::DriverDetail::ProgramSnapshot(shaders, rawAddress); - check(unregistered != entry && unregistered->codeAddress == rawAddress && unregistered->header.empty() && unregistered->code.size() == 2, "an unregistered program was not read as raw code"); - check(AgcDriver::DriverDetail::ProgramSnapshot({}, rawAddress) == unregistered, "an empty registry did not fall back to raw code"); - check(!expectFailure([] { static_cast(AgcDriver::DriverDetail::ProgramSnapshot({}, 0)); }).empty(), "an unmapped unregistered program was accepted"); + shaders[registeredAddress] = {entry}; + check(AgcDriver::DriverDetail::ProgramSnapshot(shaders, registeredAddress, {2}) == entry, "a registered program was not resolved to its registration"); + check(AgcDriver::DriverDetail::ProgramSnapshot(shaders, registeredAddress + 4 * 8, {2}) == entry, "an entry inside registered code was not resolved to its registration"); + check(!expectFailure([&] { static_cast(AgcDriver::DriverDetail::ProgramSnapshot(shaders, rawAddress, {2})); }).empty(), "an unregistered program was read raw while shaders are registered"); + const auto unregistered = AgcDriver::DriverDetail::ProgramSnapshot({}, rawAddress, {2}); + check(unregistered != entry && unregistered->codeAddress == rawAddress && unregistered->header.empty() && unregistered->code.size() == 2, "an empty registry did not read the program as raw code"); + check(AgcDriver::DriverDetail::ProgramSnapshot({}, rawAddress, {2}) == unregistered, "unchanged raw code lost its snapshot identity"); + check(!expectFailure([] { static_cast(AgcDriver::DriverDetail::ProgramSnapshot({}, 0, {2})); }).empty(), "an unmapped unregistered program was accepted"); AgcDriver::DriverDetail::DrawDecode decode{}; decode.programs.resize(2); decode.programs[0].snapshot = unregistered; @@ -597,7 +598,7 @@ void testProgramSnapshots() { check(AgcDriver::DriverDetail::DecodeProgramsCurrent(decode, shaders), "a rewritten registered program made a decode stale"); raw[0] = 0xbf810000; check(!AgcDriver::DriverDetail::DecodeProgramsCurrent(decode, shaders), "a rewritten unregistered program left its decode current"); - decode.programs[0].snapshot = AgcDriver::DriverDetail::ProgramSnapshot(shaders, rawAddress); + decode.programs[0].snapshot = AgcDriver::DriverDetail::ProgramSnapshot({}, rawAddress, {2}); check(AgcDriver::DriverDetail::DecodeProgramsCurrent(decode, shaders), "a decode with the rewritten program was not current"); } diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedGraphics.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedGraphics.cpp index 9f9a0ef90..8a6ac8b97 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedGraphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedGraphics.cpp @@ -244,6 +244,7 @@ void Check(AgcDriver::VulkanDevice& device, AgcDriver::Graphics::ShaderPath path Require(program.snapshot->prepared->entries.size() == predicted + 1, "an artifact prepared at a draw was not reused"); } ShaderRegistry invalidRegistry; + invalidRegistry[1] = {std::make_shared(ShaderSnapshot{1, 0, 2, {}, {}})}; DrawDecode invalid{}; invalid.state = prepared.state; Reject([&] { DecodeGraphicsPrograms(invalid, queue, invalidRegistry, true, true); }, "registered"); diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedUnregisteredGraphics.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedUnregisteredGraphics.cpp index c51f19029..191546739 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedUnregisteredGraphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedUnregisteredGraphics.cpp @@ -2,6 +2,7 @@ #include "prx/libSceAgcDriver/Execution/include/ShaderPreparation.hpp" #include "SceShaders.hpp" #include "prx/libSceAgcDriver/Execution/include/Driver/Draw/DrawCache.hpp" +#include "prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp" #include "prx/libSceAgcDriver/Graphics/include/ShaderInputState.hpp" #include "Optimization/ResourceProgram.hpp" #include @@ -49,6 +50,14 @@ void Check(AgcDriver::VulkanDevice& device) { DecodeGraphicsPrograms(draw, queue, registry, false, true); Require(draw.programs.size() == 2 && draw.roles.back() == ProgramRole::Fragment, "unregistered graphics programs did not decode as a vertex and a fragment stage"); for (const auto& program : draw.programs) Require(program.snapshot != nullptr && program.snapshot->header.empty() && program.snapshot->prepared->entries.empty(), "an unregistered graphics program was not read as raw code"); + Require(RawGraphicsDrawn(), "a draw of unregistered graphics programs was not noted"); + bool refused = false; + try { + Driver::Get().RegisterShader(nullptr); + } catch (const std::exception& error) { + refused = std::string(error.what()).find("registered after graphics programs were drawn") != std::string::npos; + } + Require(refused, "a shader registration after unregistered graphics draws was accepted"); const auto target = device.Target(); std::vector linked; std::vector memory; From 132715e4e2e517f6259c7d6b22c4182253199773 Mon Sep 17 00:00:00 2001 From: gookie <36005381+gookie-dev@users.noreply.github.com> Date: Sun, 11 Oct 2026 02:29:13 +0200 Subject: [PATCH 273/460] perf(libSceNgs2.native): parse waveform files without reading their samples sceNgs2ParseWaveformFile read everything from the offset to the end of the file into a zero-filled buffer and parsed that, although parsing uses only the RIFF chunk headers and the fmt, fact and smpl payloads. A waveform at an offset in a sound bank cost a read of the rest of the bank on each call. ReadChunks now takes the bytes through an accessor. Waveform data passes its buffer; a waveform file seeks to and reads the headers and those payloads alone, with the same checks against the length from the offset to the end of the file. The format dispatch both use moves to ParseWaveform. ngs2_parse adds a bank of two waveforms in one file, the second with a chunk after its samples, and parses each at its offset. PPSA21402 (Balatro) parses its five 33 MB music tracks at start: one call took 8.3 to 26.7 ms and allocated 33 MB, and now takes 0.01 to 0.04 ms with the same result. --- .../libs/prx/libSceNgs2.native/src/Atrac9.cpp | 56 ++++++++++++------- .../prx/libSceNgs2.native/tests/Ngs2Parse.cpp | 27 +++++++++ 2 files changed, 62 insertions(+), 21 deletions(-) diff --git a/core/libs/prx/libSceNgs2.native/src/Atrac9.cpp b/core/libs/prx/libSceNgs2.native/src/Atrac9.cpp index 240c0a7a9..b508ac56a 100644 --- a/core/libs/prx/libSceNgs2.native/src/Atrac9.cpp +++ b/core/libs/prx/libSceNgs2.native/src/Atrac9.cpp @@ -217,14 +217,16 @@ struct RiffChunks { std::size_t dataSize = 0; }; -static int ReadChunks(const std::uint8_t* bytes, std::size_t size, RiffChunks& chunks) { +template +static int ReadChunks(std::size_t size, TRead&& read, RiffChunks& chunks) { if (size < 12) return SCE_NGS2_ERROR_INVALID_WAVEFORM_DATA; - if (std::memcmp(bytes, "RIFF", 4) != 0 || std::memcmp(bytes + 8, "WAVE", 4) != 0) return SCE_NGS2_ERROR_UNKNOWN_WAVEFORM_FORMAT; - const std::size_t riffEnd = 8 + static_cast(ReadLe32(bytes + 4)); + const auto* header = read(0, 12); + if (std::memcmp(header, "RIFF", 4) != 0 || std::memcmp(header + 8, "WAVE", 4) != 0) return SCE_NGS2_ERROR_UNKNOWN_WAVEFORM_FORMAT; + const std::size_t riffEnd = 8 + static_cast(ReadLe32(header + 4)); if (riffEnd < 12) return SCE_NGS2_ERROR_INVALID_WAVEFORM_DATA; const std::size_t end = std::min(size, riffEnd); for (std::size_t offset = 12; offset + 8 <= end;) { - const auto* chunk = bytes + offset; + const auto* chunk = read(offset, 8); const std::size_t payload = offset + 8; const std::size_t chunkSize = ReadLe32(chunk + 4); if (chunkSize > riffEnd - payload) return SCE_NGS2_ERROR_INVALID_WAVEFORM_DATA; @@ -236,13 +238,13 @@ static int ReadChunks(const std::uint8_t* bytes, std::size_t size, RiffChunks& c } if (chunkSize > end - payload) return SCE_NGS2_ERROR_INVALID_WAVEFORM_DATA; if (std::memcmp(chunk, "fmt ", 4) == 0) { - chunks.format = bytes + payload; + chunks.format = read(payload, chunkSize); chunks.formatSize = chunkSize; } else if (std::memcmp(chunk, "fact", 4) == 0) { - chunks.fact = bytes + payload; + chunks.fact = read(payload, chunkSize); chunks.factSize = chunkSize; } else if (std::memcmp(chunk, "smpl", 4) == 0) { - chunks.sampler = bytes + payload; + chunks.sampler = read(payload, chunkSize); chunks.samplerSize = chunkSize; } offset = payload + chunkSize + (chunkSize & 1); @@ -349,6 +351,17 @@ static int ParsePcm16(const RiffChunks& chunks, std::size_t size, Ngs2WaveformIn return SCE_NGS2_OK; } +static int ParseWaveform(const RiffChunks& chunks, std::size_t size, Ngs2WaveformInfo& info) { + if (chunks.formatSize < 2) return SCE_NGS2_ERROR_INVALID_WAVEFORM_FORMAT; + const std::uint16_t tag = ReadLe16(chunks.format); + if (tag == WAVE_FORMAT_EXTENSIBLE && chunks.formatSize >= FMT_GUID_OFFSET + sizeof(ATRAC9_GUID) && + std::memcmp(chunks.format + FMT_GUID_OFFSET, ATRAC9_GUID, sizeof(ATRAC9_GUID)) == 0) { + return ParseAtrac9(chunks, info); + } + if (tag == WAVE_FORMAT_PCM) return ParsePcm16(chunks, size, info); + throw std::runtime_error("NGS2: parsing waveform format tag " + Ngs2Hex(tag) + " is not implemented"); +} + #pragma GCC visibility push(default) extern "C" { @@ -357,17 +370,11 @@ int APS5_VABI sceNgs2ParseWaveformData(const void* data, size_t data_size, Ngs2W if (info == nullptr) return SCE_NGS2_ERROR_INVALID_OUT_ADDRESS; *info = {}; if (data == nullptr) return SCE_NGS2_ERROR_INVALID_WAVEFORM_DATA; + const auto* bytes = static_cast(data); RiffChunks chunks; - const int result = ReadChunks(static_cast(data), data_size, chunks); + const int result = ReadChunks(data_size, [bytes](std::size_t at, std::size_t) { return bytes + at; }, chunks); if (result != SCE_NGS2_OK) return result; - if (chunks.formatSize < 2) return SCE_NGS2_ERROR_INVALID_WAVEFORM_FORMAT; - const std::uint16_t tag = ReadLe16(chunks.format); - if (tag == WAVE_FORMAT_EXTENSIBLE && chunks.formatSize >= FMT_GUID_OFFSET + sizeof(ATRAC9_GUID) && - std::memcmp(chunks.format + FMT_GUID_OFFSET, ATRAC9_GUID, sizeof(ATRAC9_GUID)) == 0) { - return ParseAtrac9(chunks, *info); - } - if (tag == WAVE_FORMAT_PCM) return ParsePcm16(chunks, data_size, *info); - throw std::runtime_error("NGS2: parsing waveform format tag " + Ngs2Hex(tag) + " is not implemented"); + return ParseWaveform(chunks, data_size, *info); } int APS5_VABI sceNgs2ParseWaveformFile(const char* path, uint32_t offset, Ngs2WaveformInfo* info) { @@ -379,11 +386,18 @@ int APS5_VABI sceNgs2ParseWaveformFile(const char* path, uint32_t offset, Ngs2Wa if (!file) throw std::runtime_error(std::string(__func__) + ": cannot open " + host.string()); const auto fileSize = static_cast(file.tellg()); if (offset > fileSize) return SCE_NGS2_ERROR_INVALID_WAVEFORM_DATA; - const std::uint64_t length = fileSize - offset; - std::vector bytes(static_cast(length)); - file.seekg(offset); - if (!file.read(reinterpret_cast(bytes.data()), static_cast(bytes.size()))) throw std::runtime_error(std::string(__func__) + ": read failed for " + host.string()); - const int result = sceNgs2ParseWaveformData(bytes.data(), bytes.size(), info); + const auto length = static_cast(fileSize - offset); + const std::string function = __func__; + std::vector> pieces; + const auto readBytes = [&](std::size_t at, std::size_t count) { + auto& piece = pieces.emplace_back(std::max(count, 1)); + file.seekg(static_cast(offset + at)); + if (!file.read(reinterpret_cast(piece.data()), static_cast(count))) throw std::runtime_error(function + ": read failed for " + host.string()); + return static_cast(piece.data()); + }; + RiffChunks chunks; + int result = ReadChunks(length, readBytes, chunks); + if (result == SCE_NGS2_OK) result = ParseWaveform(chunks, length, *info); if (result != SCE_NGS2_OK) return result; info->data_offset += offset; for (std::uint32_t i = 0; i < info->num_blocks; i++) info->block[i].data_offset += offset; diff --git a/core/libs/prx/libSceNgs2.native/tests/Ngs2Parse.cpp b/core/libs/prx/libSceNgs2.native/tests/Ngs2Parse.cpp index 396881d46..a2545f52e 100644 --- a/core/libs/prx/libSceNgs2.native/tests/Ngs2Parse.cpp +++ b/core/libs/prx/libSceNgs2.native/tests/Ngs2Parse.cpp @@ -87,8 +87,35 @@ static void TestParseFile() { std::filesystem::remove(path); } +static void TestParseFileBank() { + const auto first = PcmFile(2, 48000, 16, 300); + auto second = PcmFile(1, 32000, 16, 50); + PutTag(second, "LIST"); + Put32(second, 4); + PutTag(second, "INFO"); + const std::uint32_t riffSize = static_cast(second.size() - 8); + std::memcpy(second.data() + 4, &riffSize, sizeof(riffSize)); + const char* const guestPath = "/aps5_ngs2_parse_bank.bin"; + const auto path = ResolvePath_nid_no_patch(guestPath); + { + std::ofstream out(path, std::ios::binary); + out.write(reinterpret_cast(first.data()), static_cast(first.size())); + out.write(reinterpret_cast(second.data()), static_cast(second.size())); + } + Ngs2WaveformInfo info{}; + Require(sceNgs2ParseWaveformFile(guestPath, 0, &info) == SCE_NGS2_OK); + Require(info.format.num_channels == 2 && info.format.sample_rate == 48000 && info.num_samples == 300); + Require(info.data_offset == 44 && info.data_size == 1200 && info.block[0].data_offset == 44); + const auto secondOffset = static_cast(first.size()); + Require(sceNgs2ParseWaveformFile(guestPath, secondOffset, &info) == SCE_NGS2_OK); + Require(info.format.num_channels == 1 && info.format.sample_rate == 32000 && info.num_samples == 50); + Require(info.data_offset == secondOffset + 44 && info.data_size == 100 && info.block[0].data_offset == secondOffset + 44); + std::filesystem::remove(path); +} + int main() { TestParsePcm(); TestParseFile(); + TestParseFileBank(); return 0; } From 23a8b8b3d0e63d50dde834e0f7d8839201172046 Mon Sep 17 00:00:00 2001 From: gookie <36005381+gookie-dev@users.noreply.github.com> Date: Sun, 11 Oct 2026 02:33:14 +0200 Subject: [PATCH 274/460] perf(agc): build the initial context registers once InitialContextRegisters built the clear-state context registers on every call: about 200 emplace calls from its list and loops, with the register vectors growing several times on the way to offset 0x3b8. Each QueueState starts from it, so it ran for every CLEAR_STATE, and ResolveShaderAbi and ResolveGraphicsStagesAbi construct a QueueState on each call, which sceAgcLinkShaders, sceAgcCreatePrimState and sceAgcCreateInterpolantMapping make while a title draws. The registers are now built once into a table that is never destroyed, so a queue cleared while the process exits still finds it, and each call returns a copy of the table. A call takes 71 ns instead of 746 ns, and constructing a QueueState 885 ns instead of 1378 ns (200000 calls each, GCC 13 -O2). On the title screen of PPSA21402 (Balatro) the two functions took 44 to 47 of about 2300 CPU samples per 10 s and no longer appear in the profile. --- .../prx/libSceAgcDriver/Execution/include/QueueState.hpp | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/QueueState.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/QueueState.hpp index d05c1881a..3671c3010 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/QueueState.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/QueueState.hpp @@ -139,7 +139,7 @@ private: std::size_t entries = 0; }; -inline Registers InitialContextRegisters() { +inline Registers BuildInitialContextRegisters() { Registers result{ {0x200, 0}, {0x201, 0}, {0x202, 0xcc0010}, {0x203, 0}, {0x204, 0}, {0x205, 0}, {0x206, 1087}, {0x207, 0}, @@ -171,6 +171,11 @@ inline Registers InitialContextRegisters() { return result; } +inline Registers InitialContextRegisters() { + static const Registers& initial = *new Registers(BuildInitialContextRegisters()); + return initial; +} + struct Predication { std::uint64_t address = 0; std::uint32_t operation = 0; From 1191ed3eaac646cc54a89ccdf4fdce524363ac66 Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Fri, 9 Oct 2026 19:28:57 +0900 Subject: [PATCH 275/460] feat(libc): implement the hyperbolic functions Export sinh, cosh, coshf, asinh, asinhf, acosh, acoshf and atanh, forwarding to the C++ standard library like tanh and tanhf. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libc/src/Math.cpp | 8 ++++++++ core/libs/tests/GuestMath.cpp | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/core/libs/prx/libc/src/Math.cpp b/core/libs/prx/libc/src/Math.cpp index 8beda4557..01aa2770e 100644 --- a/core/libs/prx/libc/src/Math.cpp +++ b/core/libs/prx/libc/src/Math.cpp @@ -62,6 +62,14 @@ void APS5_VABI sincosf_nid_postfix(float x, float* sinp, float* cosp) { double APS5_VABI sin_nid_postfix(double x) { return std::sin(x); } double APS5_VABI cos_nid_postfix(double x) { return std::cos(x); } +double APS5_VABI sinh_nid_postfix(double x) { return std::sinh(x); } +double APS5_VABI cosh_nid_postfix(double x) { return std::cosh(x); } +float APS5_VABI coshf_nid_postfix(float x) { return std::cosh(x); } +double APS5_VABI asinh_nid_postfix(double x) { return std::asinh(x); } +float APS5_VABI asinhf_nid_postfix(float x) { return std::asinh(x); } +double APS5_VABI acosh_nid_postfix(double x) { return std::acosh(x); } +float APS5_VABI acoshf_nid_postfix(float x) { return std::acosh(x); } +double APS5_VABI atanh_nid_postfix(double x) { return std::atanh(x); } void APS5_VABI sincos_nid_postfix(double x, double* sinp, double* cosp) { *sinp = std::sin(x); diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index b0a96e78b..bf4554e70 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -46,6 +46,14 @@ double APS5_VABI frexp_nid_postfix(double, int*); float APS5_VABI frexpf_nid_postfix(float, int*); std::int64_t APS5_VABI lround_nid_postfix(double); std::div_t APS5_VABI div_nid_postfix(int, int); +double APS5_VABI sinh_nid_postfix(double); +double APS5_VABI cosh_nid_postfix(double); +float APS5_VABI coshf_nid_postfix(float); +double APS5_VABI asinh_nid_postfix(double); +float APS5_VABI asinhf_nid_postfix(float); +double APS5_VABI acosh_nid_postfix(double); +float APS5_VABI acoshf_nid_postfix(float); +double APS5_VABI atanh_nid_postfix(double); std::int64_t APS5_VABI lroundf_nid_postfix(float); std::int64_t APS5_VABI llround_nid_postfix(double); std::int64_t APS5_VABI llroundf_nid_postfix(float); @@ -188,6 +196,30 @@ static void CheckIntegerConversions() { *__error_nid_postfix() = 0; } +static bool Near(double value, double expected, double tolerance) { return std::abs(value - expected) <= tolerance * std::abs(expected); } + +static void CheckHyperbolic() { + const double infinity = std::numeric_limits::infinity(); + Require(sinh_nid_postfix(0.) == 0. && std::signbit(sinh_nid_postfix(-0.))); + Require(cosh_nid_postfix(0.) == 1. && coshf_nid_postfix(0.f) == 1.f); + Require(asinh_nid_postfix(0.) == 0. && std::signbit(asinh_nid_postfix(-0.)) && std::signbit(asinhf_nid_postfix(-0.f))); + Require(acosh_nid_postfix(1.) == 0. && acoshf_nid_postfix(1.f) == 0.f); + Require(atanh_nid_postfix(0.) == 0. && atanh_nid_postfix(1.) == infinity && atanh_nid_postfix(-1.) == -infinity); + Require(std::isnan(acosh_nid_postfix(0.5)) && std::isnan(acoshf_nid_postfix(0.5f)) && std::isnan(atanh_nid_postfix(2.))); + Require(Near(sinh_nid_postfix(1.), 1.1752011936438014, 1e-15)); + Require(Near(cosh_nid_postfix(1.), 1.5430806348152437, 1e-15)); + Require(Near(coshf_nid_postfix(1.f), 1.5430806348152437, 1e-6)); + Require(Near(asinh_nid_postfix(1.), 0.881373587019543, 1e-15)); + Require(Near(asinhf_nid_postfix(1.f), 0.881373587019543, 1e-6)); + Require(Near(acosh_nid_postfix(2.), 1.3169578969248166, 1e-15)); + Require(Near(acoshf_nid_postfix(2.f), 1.3169578969248166, 1e-6)); + Require(Near(atanh_nid_postfix(0.5), 0.5493061443340549, 1e-15)); + Require(sinh_nid_postfix(1e-300) == 1e-300 && asinh_nid_postfix(1e-300) == 1e-300 && atanh_nid_postfix(1e-300) == 1e-300); + Require(std::isfinite(cosh_nid_postfix(710.4)) && std::isfinite(sinh_nid_postfix(-710.4)) && sinh_nid_postfix(-710.4) < 0.); + Require(Near(asinh_nid_postfix(-1e300), -691.4686750787736, 1e-15)); + Require(Near(acosh_nid_postfix(1e300), 691.4686750787736, 1e-15)); +} + static void CheckFloatClassification() { Require(_FInf_nid_postfix.bits[0] == 0x7f800000u && _FNan_nid_postfix.bits[0] == 0x7fc00000u); for (int word = 1; word < 4; ++word) Require(_FInf_nid_postfix.bits[word] == 0 && _FNan_nid_postfix.bits[word] == 0); @@ -296,4 +328,5 @@ int main() { Require(negativeDenominator.quot == -3 && negativeDenominator.rem == 1); const auto minimum = div_nid_postfix(std::numeric_limits::min(), 10); Require(minimum.quot == -214748364 && minimum.rem == -8); + CheckHyperbolic(); } From 4c4e789a33d62c938cc49de4d990fb7cac29acd1 Mon Sep 17 00:00:00 2001 From: hugodearaujo Date: Sat, 10 Oct 2026 21:40:27 -0300 Subject: [PATCH 276/460] fix(libSceRudp): match the console's init and status checks Measured on a PS5 (firmware 13.60): - sceRudpInit rejects a null pool or a size of 0 or less with INVALID_ARGUMENT, and a pool under 0xC58 bytes or not 8-byte aligned with MEMORY (0x80770007). - sceRudpGetStatus rejects a null pointer or a byte count of 0 or above 0xF8 with INVALID_ARGUMENT and reports the live context count at 0x64. Co-Authored-By: Claude Opus 5.5 --- core/libs/CMakeLists.txt | 10 ++++ core/libs/prx/libSceRudp/Export.cpp | 22 ++++++-- core/libs/tests/GuestOfflineImports.cpp | 5 +- core/libs/tests/GuestRudpStatus.cpp | 73 +++++++++++++++++++++++++ docs/dev/TechnicalDebt.md | 2 +- 5 files changed, 105 insertions(+), 7 deletions(-) create mode 100644 core/libs/tests/GuestRudpStatus.cpp diff --git a/core/libs/CMakeLists.txt b/core/libs/CMakeLists.txt index cb240d7dd..9e733181b 100644 --- a/core/libs/CMakeLists.txt +++ b/core/libs/CMakeLists.txt @@ -173,6 +173,16 @@ if(BUILD_TESTING) "PATH=path_list_prepend:$") endif() + add_test_executable(guest_rudp_status_tests tests/GuestRudpStatus.cpp) + target_include_directories(guest_rudp_status_tests PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}) + target_link_libraries(guest_rudp_status_tests PRIVATE libSceRudp libc) + configure_windows_unwind(guest_rudp_status_tests) + add_test(NAME guest_rudp_status COMMAND guest_rudp_status_tests) + if(WIN32) + set_tests_properties(guest_rudp_status PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() + add_test_executable(guest_system_gesture_tests tests/GuestSystemGesture.cpp) target_include_directories(guest_system_gesture_tests PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}) target_link_libraries(guest_system_gesture_tests PRIVATE libSceSystemGesture libc) diff --git a/core/libs/prx/libSceRudp/Export.cpp b/core/libs/prx/libSceRudp/Export.cpp index efe18931f..3531d0e17 100644 --- a/core/libs/prx/libSceRudp/Export.cpp +++ b/core/libs/prx/libSceRudp/Export.cpp @@ -16,9 +16,15 @@ constexpr int RUDP_ERROR_NOT_INITIALIZED = static_cast(0x80770001u); constexpr int RUDP_ERROR_ALREADY_INITIALIZED = static_cast(0x80770002u); constexpr int RUDP_ERROR_INVALID_CONTEXT_ID = static_cast(0x80770003u); constexpr int RUDP_ERROR_INVALID_ARGUMENT = static_cast(0x80770004u); +constexpr int RUDP_ERROR_MEMORY = static_cast(0x80770007u); constexpr int RUDP_ERROR_CONN_RESET = static_cast(0x80770009u); constexpr int RUDP_ERROR_CONN_REFUSED = static_cast(0x8077000Au); +constexpr int RUDP_POOL_MIN_SIZE = 0xC58; +constexpr std::uintptr_t RUDP_POOL_ALIGNMENT = 8; +constexpr std::size_t RUDP_STATUS_SIZE = 0xF8; +constexpr std::size_t RUDP_STATUS_CURRENT_CONTEXTS = 0x64; + using GuestRudpEventHandler = void(APS5_VABI*)(int, int, int, void*); std::mutex g_mutex; @@ -36,12 +42,16 @@ bool valid_ctx(int ctx) { extern "C" { int APS5_VABI sceRudpInit_nid_postfix(void* mem_pool, int mem_pool_size) { - (void)mem_pool; - (void)mem_pool_size; std::lock_guard lk(g_mutex); if (g_inited) { return RUDP_ERROR_ALREADY_INITIALIZED; } + if (mem_pool == nullptr || mem_pool_size <= 0) { + return RUDP_ERROR_INVALID_ARGUMENT; + } + if (mem_pool_size < RUDP_POOL_MIN_SIZE || reinterpret_cast(mem_pool) % RUDP_POOL_ALIGNMENT != 0) { + return RUDP_ERROR_MEMORY; + } g_inited = true; return 0; } @@ -55,9 +65,13 @@ int APS5_VABI sceRudpGetStatus(void* status, std::size_t size) { if (!g_inited) { return RUDP_ERROR_NOT_INITIALIZED; } - if (status != nullptr && size != 0) { - std::memset(status, 0, size); // state 0 = idle, no connections + if (status == nullptr || size == 0 || size > RUDP_STATUS_SIZE) { + return RUDP_ERROR_INVALID_ARGUMENT; } + unsigned char report[RUDP_STATUS_SIZE] = {}; + const auto contexts = static_cast(g_ctx.size()); + std::memcpy(report + RUDP_STATUS_CURRENT_CONTEXTS, &contexts, sizeof(contexts)); + std::memcpy(status, report, size); return 0; } diff --git a/core/libs/tests/GuestOfflineImports.cpp b/core/libs/tests/GuestOfflineImports.cpp index 536bf001d..853a4bf1c 100644 --- a/core/libs/tests/GuestOfflineImports.cpp +++ b/core/libs/tests/GuestOfflineImports.cpp @@ -75,11 +75,12 @@ int main() { constexpr int rudpNotInitialized = static_cast(0x80770001); Require(sceRudpGetStatus(status.data(), status.size()) == rudpNotInitialized); Require(status == originalStatus); - Require(sceRudpInit_nid_postfix(nullptr, 0) == 0); + alignas(8) static unsigned char rudpPool[0xC58]; + Require(sceRudpInit_nid_postfix(rudpPool, sizeof(rudpPool)) == 0); Require(sceRudpActivate() == 0); Require(sceRudpGetStatus(status.data(), status.size()) == 0); for (unsigned char byte : status) Require(byte == 0); - Require(sceRudpGetStatus(nullptr, 0) == 0); + Require(sceRudpGetStatus(nullptr, 0) == static_cast(0x80770004)); Require(sceRudpTerminate() == 0); return 0; } diff --git a/core/libs/tests/GuestRudpStatus.cpp b/core/libs/tests/GuestRudpStatus.cpp new file mode 100644 index 000000000..3a6e229e3 --- /dev/null +++ b/core/libs/tests/GuestRudpStatus.cpp @@ -0,0 +1,73 @@ +#include "SceTypes.hpp" + +#include +#include +#include +#include +#include + +extern "C" { +using GuestRudpEventHandler = void(APS5_VABI*)(int, int, int, void*); +int APS5_VABI sceRudpInit_nid_postfix(void*, int); +int APS5_VABI sceRudpGetStatus(void*, std::size_t); +int APS5_VABI sceRudpCreateContext(GuestRudpEventHandler, void*, int*); +} + +static void Require(bool value) { if (!value) std::abort(); } + +static void APS5_VABI Handler(int, int, int, void*) {} + +int main() { + constexpr int notInitialized = static_cast(0x80770001); + constexpr int alreadyInitialized = static_cast(0x80770002); + constexpr int invalidArgument = static_cast(0x80770004); + constexpr int memory = static_cast(0x80770007); + constexpr std::size_t statusSize = 0xF8; + constexpr std::size_t currentContexts = 0x64; + alignas(16) static unsigned char pool[0x10000]; + + std::array status; + status.fill(0x5a); + const auto untouched = status; + Require(sceRudpGetStatus(status.data(), statusSize) == notInitialized); + Require(sceRudpGetStatus(nullptr, 0) == notInitialized); + Require(sceRudpGetStatus(status.data(), statusSize + 1) == notInitialized); + Require(status == untouched); + + Require(sceRudpInit_nid_postfix(nullptr, sizeof(pool)) == invalidArgument); + Require(sceRudpInit_nid_postfix(pool, 0) == invalidArgument); + Require(sceRudpInit_nid_postfix(pool, -1) == invalidArgument); + Require(sceRudpInit_nid_postfix(pool, 0xC57) == memory); + for (std::size_t offset : {1u, 2u, 4u}) Require(sceRudpInit_nid_postfix(pool + offset, 0x1000) == memory); + Require(sceRudpInit_nid_postfix(pool + 8, 0xC58) == 0); + Require(sceRudpInit_nid_postfix(nullptr, 0) == alreadyInitialized); + + Require(sceRudpGetStatus(nullptr, statusSize) == invalidArgument); + Require(sceRudpGetStatus(nullptr, 0) == invalidArgument); + Require(sceRudpGetStatus(status.data(), 0) == invalidArgument); + Require(sceRudpGetStatus(status.data(), statusSize + 1) == invalidArgument); + Require(status == untouched); + + Require(sceRudpGetStatus(status.data(), statusSize) == 0); + for (std::size_t i = 0; i < statusSize; ++i) Require(status[i] == 0); + for (std::size_t i = statusSize; i < status.size(); ++i) Require(status[i] == 0x5a); + + int first = -1; + int second = -1; + Require(sceRudpCreateContext(Handler, nullptr, &first) == 0); + Require(sceRudpCreateContext(Handler, nullptr, &second) == 0); + status.fill(0x5a); + Require(sceRudpGetStatus(status.data(), statusSize) == 0); + std::uint32_t contexts = 0; + std::memcpy(&contexts, status.data() + currentContexts, sizeof(contexts)); + Require(contexts == 2); + for (std::size_t i = 0; i < statusSize; ++i) { + if (i < currentContexts || i >= currentContexts + sizeof(contexts)) Require(status[i] == 0); + } + + status.fill(0x5a); + Require(sceRudpGetStatus(status.data(), 1) == 0); + Require(status[0] == 0); + for (std::size_t i = 1; i < status.size(); ++i) Require(status[i] == 0x5a); + return 0; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 74a13e50a..22c42a9b8 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -53,7 +53,7 @@ Throughout the project, every function at every stage either **does exactly what - [Color comparison descriptor forms](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp): emulated color comparison supports UNORM, SNORM and float formats for 2D, 2D-array and cube views. sRGB formats, anisotropic filters and bilinear cube comparisons without `DISABLE_CUBE_WRAP` are refused; cube-edge sampling behavior and console coverage on PS5's gfx1013 remain unverified. - [sceNpEntitlementAccessGetEntitlementKey](../../core/libs/prx/libSceNpEntitlementAccess/Export.cpp) - the three-argument signature, 16-byte key and errors `0x817D0002`/`0x817D0007` follow [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/main/src/libs/libNet.cpp); PPSA01521 passes a service label, entitlement label and output pointer. Only the no-key path is implemented: local entitlement labels carry no keys, and the output is untouched on failure. Console error precedence and key provisioning are unverified - [sceNpEntitlementAccess{Request,Poll}{Unified,Service}EntitlementInfoList](../../core/libs/prx/libSceNpEntitlementAccess/Export.cpp) - signatures unknown; no reference implements them, so they are declared without parameters. Like the consume requests in the same library, they report the signed-out NP state with `SCE_NP_ERROR_SIGNED_OUT` (`0x80550006`) and write nothing. PPSA15657 calls the unified request during start-up and continues past the error. Whether the console fails at the request or at the poll when signed out is unverified -- [sceRudpGetStatus](../../core/libs/prx/libSceRudp/Export.cpp) - PPSA01521 passes an output pointer and byte count `0xF8`; the status layout and size rules are unknown, so the buffer is zeroed like `sceRudpGetContextStatus` (state 0 = idle, no connections) and the size is not validated. It returns `RUDP_ERROR_NOT_INITIALIZED` before initialization and 0 otherwise, as `sceRudpGetContextStatus` does. [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/rudp/rudp.cpp) declares it without arguments and returns success without writing status +- [sceRudpGetStatus](../../core/libs/prx/libSceRudp/Export.cpp) - PPSA01521 passes an output pointer and byte count `0xF8`. Measured on a PS5 (firmware 13.60): the status is 0xF8 bytes; before `sceRudpInit` every call returns `RUDP_ERROR_NOT_INITIALIZED`, and after it a null pointer or a byte count of 0 or above 0xF8 returns `RUDP_ERROR_INVALID_ARGUMENT`. The layout matches `SceRudpStatus` in [Vita3K](https://github.com/AndnixSH/Vita3K-PS-Vita-OS-Fork/blob/master/vita3k/modules/SceRudp/SceLibRudp.cpp) with `currentContexts` at 0x64 and `allocs`, `frees`, `memCurrent` and `memPeak` at 0xC8 to 0xD4. Only `currentContexts` is reported: the traffic counters stay 0 because nothing is sent, and the allocator counters stay 0 because the library does not allocate from the title's pool (the console reports 6 allocations and 0x1260 bytes right after `sceRudpInit`, and 4 more allocations and 0x3F0 bytes per context). The console also writes `currentContexts` and the allocator counters when the byte count is smaller than their offset; only the requested bytes are written here. - [sceHttpSetCookieEnabled](../../core/libs/prx/libSceHttp/Export.cpp) - the two-argument signature and 0/1 enable values follow [VitaSDK](https://github.com/vitasdk/vita-headers/blob/master/include/psp2/net/http.h) and [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.cpp); PPSA01521 passes 0 to disable cookies. Disabling succeeds because the offline HTTP backend has no cookie engine; enabling throws. Invalid enable values use the same error as `sceHttpSetInflateGZIPEnabled`. Like the existing offline HTTP setters, the handle is not validated; requests still fail with the network error. Console behavior is unverified - [sceHmd2Initialize](../../core/libs/prx/libSceHmd2/Export.cpp) (libSceHmd2) - the error the console returns when the PS VR2 runtime is unavailable is unknown; it returns libSceHmd's `UNSUPPORTED_FEATURE` (0x81110016). PPSA01317 passes a pointer to 16 zeroed bytes and only tests the result for non-zero From 86db61ca30e0507c85c755342428a215f171eece Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Fri, 9 Oct 2026 20:49:16 +0900 Subject: [PATCH 277/460] feat(libc): implement rand_r Share rand's generator step with rand_r, which advances the caller's state instead of the global one, as FreeBSD's rand and rand_r share do_rand. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libc/src/Math.cpp | 20 ++++++++++++++++---- core/libs/tests/GuestMath.cpp | 15 +++++++++++++++ 2 files changed, 31 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libc/src/Math.cpp b/core/libs/prx/libc/src/Math.cpp index 8beda4557..992f358e3 100644 --- a/core/libs/prx/libc/src/Math.cpp +++ b/core/libs/prx/libc/src/Math.cpp @@ -127,15 +127,27 @@ int APS5_VABI __signbitf_nid_postfix(float x) { return std::signbit(x) ? 1 : 0; static std::mutex g_randLock; static std::uint32_t g_randState = 1; -int APS5_VABI rand_nid_postfix() { - std::lock_guard lock(g_randLock); - const std::int64_t x = static_cast(g_randState % 0x7ffffffeu) + 1; +static int NextRand_nid_no_patch(std::uint32_t& state) { + const std::int64_t x = static_cast(state % 0x7ffffffeu) + 1; std::int64_t next = 16807 * (x % 127773) - 2836 * (x / 127773); if (next < 0) next += 0x7fffffff; - g_randState = static_cast(next - 1); + state = static_cast(next - 1); return static_cast(next - 1); } +int APS5_VABI rand_nid_postfix() { + std::lock_guard lock(g_randLock); + return NextRand_nid_no_patch(g_randState); +} + +int APS5_VABI rand_r_nid_postfix(unsigned int* state) { + if (state == nullptr) throw std::invalid_argument("rand_r: null state"); + std::uint32_t value = *state; + const int result = NextRand_nid_no_patch(value); + *state = value; + return result; +} + void APS5_VABI srand_nid_postfix(unsigned int seed) { std::lock_guard lock(g_randLock); g_randState = seed; diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index b0a96e78b..ae2333740 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -35,6 +35,9 @@ float APS5_VABI atan2f_nid_postfix(float, float); float APS5_VABI hypotf_nid_postfix(float, float); double APS5_VABI hypot_nid_postfix(double, double); float APS5_VABI tanf_nid_postfix(float); +int APS5_VABI rand_nid_postfix(); +void APS5_VABI srand_nid_postfix(unsigned int); +int APS5_VABI rand_r_nid_postfix(unsigned int*); float APS5_VABI log10f_nid_postfix(float); float APS5_VABI logbf_nid_postfix(float); double APS5_VABI exp2_nid_postfix(double); @@ -234,6 +237,18 @@ int main() { Require(std::abs(acosf_nid_postfix(0.5f) - 1.0471976f) < 0.000001f); Require(std::abs(atan2f_nid_postfix(1.f, -1.f) - 2.3561945f) < 0.000001f); Require(tanf_nid_postfix(0.f) == 0.f); + unsigned int randomState = 1; + Require(rand_r_nid_postfix(&randomState) == 33613 && randomState == 33613u); + randomState = 0; + Require(rand_r_nid_postfix(&randomState) == 16806 && randomState == 16806u); + for (const unsigned int seed : {0u, 1u, 12345u, 0x7ffffffeu, 0xffffffffu}) { + srand_nid_postfix(seed); + randomState = seed; + for (int step = 0; step < 8; ++step) Require(rand_r_nid_postfix(&randomState) == rand_nid_postfix()); + } + bool threw = false; + try { rand_r_nid_postfix(nullptr); } catch (...) { threw = true; } + Require(threw); Require(hypot_nid_postfix(3.0, 4.0) == 5.0 && hypot_nid_postfix(-3.0, -4.0) == 5.0 && hypotf_nid_postfix(3.f, -4.f) == 5.f); Require(std::abs(hypot_nid_postfix(1e308, 1e308) / 1.4142135623730951e308 - 1.0) < 1e-15); Require(std::abs(hypotf_nid_postfix(2e38f, 2e38f) / 2.8284271e38f - 1.f) < 1e-6f); From 0f480c6acca733aab7f6679dd42d9601e542c734 Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Fri, 9 Oct 2026 19:57:29 +0900 Subject: [PATCH 278/460] feat(libc): implement strsep Split at the first delimiter like FreeBSD's strsep: empty fields are returned, and the cursor becomes null after the last token. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libc/src/StringExtensions.cpp | 13 +++++++++++++ core/libs/tests/GuestStrings.cpp | 16 ++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/core/libs/prx/libc/src/StringExtensions.cpp b/core/libs/prx/libc/src/StringExtensions.cpp index 155ec1443..dc9882dd5 100644 --- a/core/libs/prx/libc/src/StringExtensions.cpp +++ b/core/libs/prx/libc/src/StringExtensions.cpp @@ -47,6 +47,19 @@ std::size_t APS5_VABI strnlen_s_nid_postfix(const char* text, std::size_t limit) return text ? strnlen_nid_postfix(text, limit) : 0; } +char* APS5_VABI strsep_nid_postfix(char** text, const char* delimiters) { + char* token = *text; + if (token == nullptr) return nullptr; + char* end = token + std::strcspn(token, delimiters); + if (*end == '\0') { + *text = nullptr; + } else { + *end = '\0'; + *text = end + 1; + } + return token; +} + char* APS5_VABI strncat_nid_postfix(char* destination, const char* source, std::size_t limit) { return std::strncat(destination, source, limit); } diff --git a/core/libs/tests/GuestStrings.cpp b/core/libs/tests/GuestStrings.cpp index c7b048a8a..b20fd6319 100644 --- a/core/libs/tests/GuestStrings.cpp +++ b/core/libs/tests/GuestStrings.cpp @@ -19,6 +19,7 @@ std::size_t APS5_VABI strlcat_nid_postfix(char*, const char*, std::size_t); char* APS5_VABI stpcpy_nid_postfix(char*, const char*); char* APS5_VABI strtok_r_nid_postfix(char*, const char*, char**); char* APS5_VABI strtok_nid_postfix(char*, const char*); +char* APS5_VABI strsep_nid_postfix(char**, const char*); char* APS5_VABI strcasestr_nid_postfix(const char*, const char*); int APS5_VABI strcpy_s_nid_postfix(char*, std::size_t, const char*); int APS5_VABI strcat_s_nid_postfix(char*, std::size_t, const char*); @@ -219,6 +220,21 @@ int main() { Require(strtok_nid_postfix(nullptr, ",") == nullptr); Require(strtok_nid_postfix(nullptr, ",") == nullptr); Require(std::strcmp(std::strtok(nullptr, ":"), "next") == 0); + char fields[] = ",a,,b;c"; + char* cursor = fields; + Require(strsep_nid_postfix(&cursor, ",;") == fields && fields[0] == '\0' && cursor == fields + 1); + Require(std::strcmp(strsep_nid_postfix(&cursor, ",;"), "a") == 0); + Require(std::strcmp(strsep_nid_postfix(&cursor, ",;"), "") == 0); + Require(std::strcmp(strsep_nid_postfix(&cursor, ",;"), "b") == 0); + char* last = cursor; + Require(strsep_nid_postfix(&cursor, ",;") == last && std::strcmp(last, "c") == 0 && cursor == nullptr); + Require(strsep_nid_postfix(&cursor, ",;") == nullptr && cursor == nullptr); + char whole[] = "a,b"; + cursor = whole; + Require(strsep_nid_postfix(&cursor, "") == whole && std::strcmp(whole, "a,b") == 0 && cursor == nullptr); + char empty[] = ""; + cursor = empty; + Require(strsep_nid_postfix(&cursor, ",") == empty && cursor == nullptr); const char text[] = "aABAbC"; Require(strcasestr_nid_postfix(text, "ababc") == text + 1); Require(strcasestr_nid_postfix(text, "") == text); From e7605c8fd6639bb1eb8f042b6bec894e010e040b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 02:54:24 +0200 Subject: [PATCH 279/460] feat(libc): implement rewind Co-Authored-By: Claude --- core/libs/prx/libc/src/FileAndHeap.cpp | 6 +++++ core/libs/tests/GuestStandardStreams.cpp | 32 ++++++++++++++++++++++++ 2 files changed, 38 insertions(+) diff --git a/core/libs/prx/libc/src/FileAndHeap.cpp b/core/libs/prx/libc/src/FileAndHeap.cpp index f295fdb7f..311c4973c 100644 --- a/core/libs/prx/libc/src/FileAndHeap.cpp +++ b/core/libs/prx/libc/src/FileAndHeap.cpp @@ -246,6 +246,12 @@ int APS5_VABI fsetpos_nid_postfix(FileStream* stream, const std::int64_t* positi return fseeko_nid_postfix(stream, *position, SEEK_SET); } +void APS5_VABI rewind_nid_postfix(FileStream* stream) { + const int saved = errno; + if (fseeko_nid_postfix(stream, 0, SEEK_SET) == 0) errno = saved; + stream->ClearError(); +} + int APS5_VABI fputs_nid_postfix(const char* str, FileStream* stream) { if (!str) throw std::runtime_error("fputs: null string"); const int result = std::fputs(str, GetNativeStream(stream)); diff --git a/core/libs/tests/GuestStandardStreams.cpp b/core/libs/tests/GuestStandardStreams.cpp index 66fda23d6..d30be63a6 100644 --- a/core/libs/tests/GuestStandardStreams.cpp +++ b/core/libs/tests/GuestStandardStreams.cpp @@ -41,8 +41,10 @@ int APS5_VABI __swbuf_nid_postfix(int, FileStream*); int APS5_VABI ungetc_nid_postfix(int, FileStream*); char* APS5_VABI fgets_nid_postfix(char*, int, FileStream*); int APS5_VABI feof_nid_postfix(FileStream*); +int APS5_VABI ferror_nid_postfix(FileStream*); int APS5_VABI fileno_nid_postfix(FileStream*); void APS5_VABI clearerr_nid_postfix(FileStream*); +void APS5_VABI rewind_nid_postfix(FileStream*); int APS5_VABI setvbuf_nid_postfix(FileStream*, char*, int, std::size_t); void APS5_VABI setbuf_nid_postfix(FileStream*, char*); FileStream* APS5_VABI fdopen_nid_postfix(int, const char*); @@ -93,6 +95,35 @@ static bool CheckFileBytes(const std::string& filename, const std::string& expec return false; } +static void CheckRewind() { + FileStream stream(std::tmpfile()); + Require(fwrite_nid_postfix("ab", 1, 2, &stream) == 2); + *__error_nid_postfix() = 7; + rewind_nid_postfix(&stream); + Require(*__error_nid_postfix() == 7 && ftello_nid_postfix(&stream) == 0); + Require(fgetc_nid_postfix(&stream) == 'a' && ungetc_nid_postfix('z', &stream) == 'z'); + rewind_nid_postfix(&stream); + Require(fgetc_nid_postfix(&stream) == 'a' && fgetc_nid_postfix(&stream) == 'b' && fgetc_nid_postfix(&stream) == EOF); + Require(feof_nid_postfix(&stream) != 0); + stream.SetEncodingError(); + Require(ferror_nid_postfix(&stream) != 0); + rewind_nid_postfix(&stream); + Require(feof_nid_postfix(&stream) == 0 && ferror_nid_postfix(&stream) == 0 && fgetc_nid_postfix(&stream) == 'a'); + stream.Close(); +#ifndef _WIN32 + int pipeEnds[2]; + Require(::pipe(pipeEnds) == 0); + Require(::write(pipeEnds[1], "p", 1) == 1); + ::close(pipeEnds[1]); + FileStream pipe(::fdopen(pipeEnds[0], "r")); + Require(fgetc_nid_postfix(&pipe) == 'p' && fgetc_nid_postfix(&pipe) == EOF && feof_nid_postfix(&pipe) != 0); + *__error_nid_postfix() = 0; + rewind_nid_postfix(&pipe); + Require(*__error_nid_postfix() == 29 && feof_nid_postfix(&pipe) == 0 && ferror_nid_postfix(&pipe) == 0); + pipe.Close(); +#endif +} + static bool CheckBinaryModes() { const auto directory = "anyps5-byte-stream-" + std::to_string(std::chrono::steady_clock::now().time_since_epoch().count()); Require(std::filesystem::create_directory(directory)); @@ -333,5 +364,6 @@ int main() { Require(lockedWrite && fseeko_nid_postfix(&locked, 0, SEEK_SET) == 0); Require(fgetc_nid_postfix(&locked) == 'A' && fgetc_nid_postfix(&locked) == 'B'); locked.Close(); + CheckRewind(); return CheckBinaryModes() ? 0 : 1; } From 5543c73b370ed6045c38eff8eb34ba99f85e1efd Mon Sep 17 00:00:00 2001 From: Dean Galvin Date: Sun, 11 Oct 2026 13:56:11 +1300 Subject: [PATCH 280/460] fix(libkernel): reserve the AMM virtual address ranges at the top of the guest arena The AMM ranges were reserved address-less on first use, so they took the lowest free address of the arena, directly after the title's last mapping, where the title's next hinted maps go. They are now reserved at a fixed place, the last 64 GiB of the guest arena, which maps without MAP_FIXED and the hints derived from them do not reach; a reservation that cannot be made there throws. Fixes #3146 --- core/libs/prx/libkernel/Apr/src/Apr.cpp | 9 +++++-- core/libs/tests/GuestAmpr.cpp | 34 +++++++++++++++++++++++++ docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 42 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libkernel/Apr/src/Apr.cpp b/core/libs/prx/libkernel/Apr/src/Apr.cpp index 1417f6fd4..279debe50 100644 --- a/core/libs/prx/libkernel/Apr/src/Apr.cpp +++ b/core/libs/prx/libkernel/Apr/src/Apr.cpp @@ -245,6 +245,7 @@ TCommand _read(const Apr::CommandBufferObject& buffer, std::uint32_t cursor) { constexpr std::uint64_t AmmRangeBytes = 32ull << 30; constexpr int GuestMapFixed = 0x10; +constexpr int GuestMapNoOverwrite = 0x80; struct AmmPage { std::uint64_t physical; @@ -270,8 +271,12 @@ AmmState& _amm() { std::uintptr_t _ammBase(AmmState& state) { if (state.base == 0) { - void* address = nullptr; - if (DoReserveVirtual(&address, 2 * AmmRangeBytes, 0, 0x200000) != 0) throw std::runtime_error("AMM: cannot reserve the virtual address range"); + std::uintptr_t arenaBase = 0; + std::size_t arenaBytes = 0; + GuestArena::GuestArenaRange_nid_postfix(&arenaBase, &arenaBytes); + if (arenaBase == 0 || arenaBytes < 2 * AmmRangeBytes) throw std::runtime_error("AMM: the guest address space arena cannot hold the virtual address range"); + void* address = reinterpret_cast(arenaBase + arenaBytes - 2 * AmmRangeBytes); + if (DoReserveVirtual(&address, 2 * AmmRangeBytes, GuestMapFixed | GuestMapNoOverwrite, 0) != 0) throw std::runtime_error("AMM: cannot reserve the virtual address range"); state.base = reinterpret_cast(address); } return state.base; diff --git a/core/libs/tests/GuestAmpr.cpp b/core/libs/tests/GuestAmpr.cpp index 7f45f4ff3..23bdcee6b 100644 --- a/core/libs/tests/GuestAmpr.cpp +++ b/core/libs/tests/GuestAmpr.cpp @@ -1,5 +1,6 @@ #include "SceTypes.hpp" #include "prx/libc/include/general/VabiMacros.hpp" +#include "prx/libc/include/GuestArena.hpp" #include "prx/libkernel/Apr/include/AprCommandBuffer.hpp" #include #include @@ -93,6 +94,9 @@ std::uint64_t APS5_VABI sceAmprMeasureCommandSizeMapBegin(std::uint64_t, std::ui std::uint64_t APS5_VABI sceAmprMeasureCommandSizeMapDirectBegin(std::uint64_t, std::uint64_t, std::uint64_t, std::uint32_t, std::uint32_t); std::uint64_t APS5_VABI sceAmprMeasureCommandSizeMapEnd(); int APS5_VABI sceKernelQueryMemoryProtection(void*, void**, void**, int*); +int APS5_VABI sceKernelReserveVirtualRange(void**, std::size_t, int, std::size_t); +int APS5_VABI sceKernelMapFlexibleMemory(void**, std::size_t, int, int); +int APS5_VABI sceKernelMunmap(void*, std::size_t); int APS5_VABI sceAmprAmmCommandBufferMapAsPrt(Apr::CommandBufferObject*, std::uint64_t, std::uint64_t); int APS5_VABI sceAmprAmmCommandBufferAllocatePaForPrt(Apr::CommandBufferObject*, std::uint64_t, std::uint64_t, std::int32_t, std::int32_t); int APS5_VABI sceAmprAmmCommandBufferRemapIntoPrt(Apr::CommandBufferObject*, std::uint64_t, std::uint64_t, std::uint64_t, std::int32_t, std::uint32_t); @@ -637,6 +641,35 @@ volatile std::uint64_t& At(std::uint64_t address) { return *reinterpret_cast(address); } +void TestAmmRangesAboveHintedMaps() { + constexpr std::uint64_t step = 0x400000; + void* span = nullptr; + Require(sceKernelReserveVirtualRange(&span, 4 * step, 0, 0) == 0); + Require(sceKernelMunmap(span, 4 * step) == 0); + const auto heap = reinterpret_cast(span); + void* placed = span; + Require(sceKernelMapFlexibleMemory(&placed, page, 3, 0) == 0 && placed == span); + + std::uint64_t start = 0; + std::uint64_t end = 0; + std::uint64_t multimapStart = 0; + std::uint64_t multimapEnd = 0; + Require(sceAmprAmmGetVirtualAddressRanges(&start, &end, &multimapStart, &multimapEnd) == 0); + std::uintptr_t arenaBase = 0; + std::size_t arenaBytes = 0; + GuestArena::GuestArenaRange_nid_postfix(&arenaBase, &arenaBytes); + Require(multimapEnd == arenaBase + arenaBytes); + + for (std::uint64_t hint = heap + step; hint < heap + 4 * step; hint += step) { + void* mapped = reinterpret_cast(hint); + Require(sceKernelMapFlexibleMemory(&mapped, page, 3, 0) == 0); + Require(reinterpret_cast(mapped) == hint); + Require(hint + page <= start || hint >= multimapEnd); + Require(sceKernelMunmap(mapped, page) == 0); + } + Require(sceKernelMunmap(span, page) == 0); +} + void TestAmm() { std::uint64_t start = 0; std::uint64_t end = 0; @@ -989,6 +1022,7 @@ int main() { TestConstructed(); TestZeroFilledBuffer(); TestGatherScatter(); + TestAmmRangesAboveHintedMaps(); TestAmm(); TestAmmRemapAndProtect(); TestAmmPrt(); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 74a13e50a..fac18a731 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -94,7 +94,7 @@ Throughout the project, every function at every stage either **does exactly what - [AMPR command sizes](../../core/libs/prx/libSceAmpr/Export.cpp) (libSceAmpr) - commands use the emulator's own encoding with an 8-byte header, so `Nop(n)` takes 8 + 4n bytes where the console record is n dwords including its header (`NopWithData(n)`: (n + 1) dwords). The measure functions agree with the appended sizes, but a title that pads to an exact offset or the end of the buffer with `Nop(remaining / 4)` gets `SCE_AMPR_ERROR_BUFFER_FULL` or a shifted offset. The at-start flag of the `_04_00` writes is ignored; commands run in order, each after the previous one completes - [AMPR counter access](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) (libSceAmpr) - signatures and checks of `WaitOnCounter_04_00`, `WriteCounter_04_00`, `ConstructNop` and `ConstructMarker` taken from the [ampr_emu](https://github.com/drakmor/ampr_emu) reimplementation, which hands counter commands to the console and does not show their effect. The counters are taken as 32 bits each: the 8-byte access of counter n spans n (low) and n + 1 (high), the 2- and 1-byte accesses select a half or a byte of the counter by its offset, waits compare at the access width (signed and wrapped compares included), the AND mask applies to both sides and the atomic writes wrap within the access. The `ConstructNop` type is not recorded - [APR gather and scatter reads](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) (libSceAmpr) - signatures, argument checks and the read cursor taken from the [ampr_emu](https://github.com/drakmor/ampr_emu) reimplementation, not confirmed on a title: each read leaves the file, the next file offset and the next destination; a gather reads that file at a new offset into the next destination, a scatter continues in the file into a new destination, a gather-scatter takes both. Recording one needs a read file since the last `ResetGatherScatterState`; like ampr_emu, `sceAmprCommandBufferReset` keeps that state, and a gather or scatter with no read before it in the submitted buffer throws instead of returning `SCE_AMPR_ERROR_APR_INVALIDGATHERSCATTERSTATE` -- [AMM map, map direct and unmap](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) (libSceAmpr) - signatures and the protection mask (CPU, GPU, AMPR and ACP read and write) taken from the [ampr_emu](https://github.com/drakmor/ampr_emu) reimplementation, which hands the remaining checks and the mapping to the console kernel; the mapper model taken from [PS5PCEM](https://github.com/iStark/PS5PCEM): `sceAmprAmmGiveDirectMemory` usage 1 gives the allocated direct memory to the pool plain maps take pages from, usage 0 only allocates it. Not confirmed on a title. The AMM virtual address ranges are a 32 GiB range and a 32 GiB multimap range reserved in the guest arena on first use, not the console's; maps outside them throw. Addresses, sizes and direct offsets must be 16 KiB aligned. The memory type and the GPU mask are ignored, AMPR and ACP access maps as CPU access, unmapped pages go back to the pool and stay reserved. Submission runs the buffer before returning, the submit result is always 0 and a failing command throws; the AMM measures return errors sign-extended, as ampr_emu does. `sceAmprAmmGetVirtualAddressRanges` throws for a null output +- [AMM map, map direct and unmap](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) (libSceAmpr) - signatures and the protection mask (CPU, GPU, AMPR and ACP read and write) taken from the [ampr_emu](https://github.com/drakmor/ampr_emu) reimplementation, which hands the remaining checks and the mapping to the console kernel; the mapper model taken from [PS5PCEM](https://github.com/iStark/PS5PCEM): `sceAmprAmmGiveDirectMemory` usage 1 gives the allocated direct memory to the pool plain maps take pages from, usage 0 only allocates it. Not confirmed on a title. The AMM virtual address ranges are a 32 GiB range and a 32 GiB multimap range reserved on first use at the top of the guest arena, above the addresses that maps without `MAP_FIXED` are given lowest first and that titles derive their next hints from; the console's addresses are not known (ampr_emu takes them from `sceKernelMapperGetParam`), and a reservation that cannot be made there throws. Maps outside them throw. Addresses, sizes and direct offsets must be 16 KiB aligned. The memory type and the GPU mask are ignored, AMPR and ACP access maps as CPU access, unmapped pages go back to the pool and stay reserved. Submission runs the buffer before returning, the submit result is always 0 and a failing command throws; the AMM measures return errors sign-extended, as ampr_emu does. `sceAmprAmmGetVirtualAddressRanges` throws for a null output - [AMM remap, multimap and protection changes](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) (libSceAmpr) - signatures and the protection and mask checks taken from ampr_emu; the effects are inferred from the names, not confirmed on a title. A remap moves every page of a fully mapped range to the new address with the given protection and leaves the old range reserved; a multimap maps the same pages at a second address, anywhere in the AMM ranges, and a pool page goes back to the pool once its last mapping is unmapped; a multimap onto its own source range throws. A protection change sets the masked bits on every page of a fully mapped range; the memory type is recorded and ignored. A command on a range that is not fully mapped throws - [sceVideodec2GetAvcPictureInfo](../../core/libs/prx/libSceVideodec2/Export.cpp) (libSceVideodec2) - runs the `sceVideodec2GetPictureInfo` body (KytyPS5 binds both NIDs to one function; SharpProspero gives both the same `outputInfo, first, second` signature); the second (bottom field) picture info is never written, since only progressive H.264 output is modelled. prosper calls the AVC layout unestablished (no title observed calling it) - [AMM PRT ranges](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) (libSceAmpr) - signatures, checks and their order taken from ampr_emu, where `MapAsPrt` is a map with the PRT flag and protection 0 and `AllocatePaForPrt` a memory type and protection change with mask 1019, both checking the buffer memory after its size; the effects are inferred from the names, not confirmed on a title. Unbacked PRT pages are zero-filled pages with GPU read access, so CPU reads see zeros instead of faulting and GPU writes fault instead of being dropped. `AllocatePaForPrt` backs each unbacked page with a pool page and sets the type and protection of backed ones; `RemapIntoPrt` moves a fully mapped range into a PRT range, its opcode argument (1011 when 0, as in ampr_emu) is recorded and ignored; `UnmapToPrt` returns the pages to the pool and zero-fills them again. A plain map, unmap or remap over a PRT range ends it; `AllocatePaForPrt`, `RemapIntoPrt` or `UnmapToPrt` outside a PRT range throws From ef2d56eb9c22b39dca374704ceef8f0cf4a02494 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 02:58:30 +0200 Subject: [PATCH 281/460] feat(libc): implement ctime Co-Authored-By: Claude --- core/libs/prx/libc/src/Time.cpp | 9 +++++++++ core/libs/tests/GuestTimeConversion.cpp | 9 +++++++++ 2 files changed, 18 insertions(+) diff --git a/core/libs/prx/libc/src/Time.cpp b/core/libs/prx/libc/src/Time.cpp index fc6609b9a..c149f207d 100644 --- a/core/libs/prx/libc/src/Time.cpp +++ b/core/libs/prx/libc/src/Time.cpp @@ -1,5 +1,6 @@ #include #include +#include #include #include #include @@ -213,6 +214,14 @@ char* APS5_VABI asctime_nid_postfix(const GuestTm* timeptr) { return std::asctime(&host); } +char* APS5_VABI ctime_nid_postfix(const int64_t* timer) { + if (const GuestTm* local = localtime_nid_postfix(timer)) return asctime_nid_postfix(local); + static char unknown[26]; + std::memcpy(unknown, "??? ??? ?? ??:??:?? ????\n", sizeof(unknown)); + errno = EINVAL; + return unknown; +} + size_t APS5_VABI strftime_nid_postfix(char* str, size_t count, const char* format, const GuestTm* timeptr) { return libc_strftime_nid_postfix(str, count, format, timeptr); } diff --git a/core/libs/tests/GuestTimeConversion.cpp b/core/libs/tests/GuestTimeConversion.cpp index 4f83dbd38..3745c359d 100644 --- a/core/libs/tests/GuestTimeConversion.cpp +++ b/core/libs/tests/GuestTimeConversion.cpp @@ -21,6 +21,8 @@ GuestTm* APS5_VABI localtime_s_nid_postfix(const std::int64_t*, GuestTm*); std::int64_t APS5_VABI mktime_nid_postfix(GuestTm*); std::size_t APS5_VABI strftime_nid_postfix(char*, std::size_t, const char*, const GuestTm*); void APS5_VABI tzset_nid_postfix(void); +char* APS5_VABI ctime_nid_postfix(const std::int64_t*); +int* APS5_VABI __error_nid_postfix(); } using Converter = GuestTm* (APS5_VABI *)(const std::int64_t*); @@ -128,6 +130,13 @@ int main() { Require(roundTrip.tm_wday == 4 && roundTrip.tm_yday == 0, "mktime did not normalize the calendar fields"); Require(roundTrip.tm_gmtoff == 7200 && roundTrip.tm_zone != nullptr, "mktime did not set the zone fields"); + Require(std::strcmp(ctime_nid_postfix(&epoch), "Thu Jan 1 02:00:00 1970\n") == 0, "ctime did not format local time"); + const std::int64_t leap = 951782400 + 13 * 3600 + 4 * 60 + 5; + Require(std::strcmp(ctime_nid_postfix(&leap), "Tue Feb 29 15:04:05 2000\n") == 0, "ctime formatted the wrong date"); + *__error_nid_postfix() = 0; + Require(std::strcmp(ctime_nid_postfix(&underflow), "??? ??? ?? ??:??:?? ????\n") == 0 && *__error_nid_postfix() == 22, + "ctime of an unrepresentable time did not report it"); + GuestTm utc{}; Require(gmtime_s_nid_postfix(&epoch, &utc) == &utc, "UTC epoch conversion failed"); char formatted[64]{}; From a6520d0c8b9139300c30de4b4d2fcab6e554bd41 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sat, 10 Oct 2026 16:24:41 -0500 Subject: [PATCH 282/460] feat(libkernel): implement sceKernelGetAppInfo sceKernelGetAppInfo fills the app id and the title id of the calling process: app id 1, the running title's id in libSceSystemService, and the titleId of /app0/sce_sys/param.json at offset 0x10, up to 9 characters and a terminator. Only those fields are written, since callers declare the structure with different sizes. Another pid returns ESRCH and a null info EFAULT. AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libkernel/CMakeLists.txt | 10 +++++ .../libs/prx/libkernel/System/src/Process.cpp | 26 +++++++++++ core/libs/tests/GuestAppInfo.cpp | 44 +++++++++++++++++++ 3 files changed, 80 insertions(+) create mode 100644 core/libs/tests/GuestAppInfo.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index d4eaa4550..0cfc3d427 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -263,6 +263,16 @@ if(BUILD_TESTING) set_tests_properties(guest_kernel_is_stack PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_app_info_tests ${LIBS_INCLUDE_DIR}/tests/GuestAppInfo.cpp) + target_include_directories(guest_app_info_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_app_info_tests PRIVATE libkernel libc) + configure_windows_unwind(guest_app_info_tests) + add_test(NAME guest_app_info COMMAND guest_app_info_tests) + set_tests_properties(guest_app_info PROPERTIES TIMEOUT 20) + if(WIN32) + set_tests_properties(guest_app_info PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_thread_self_tests ${LIBS_INCLUDE_DIR}/tests/GuestThreadSelf.cpp) target_include_directories(guest_thread_self_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_thread_self_tests PRIVATE libkernel libc) diff --git a/core/libs/prx/libkernel/System/src/Process.cpp b/core/libs/prx/libkernel/System/src/Process.cpp index 3cc2d722b..1ee262b02 100644 --- a/core/libs/prx/libkernel/System/src/Process.cpp +++ b/core/libs/prx/libkernel/System/src/Process.cpp @@ -14,7 +14,9 @@ #include "prx/libc/include/ApplicationHeap.hpp" #include "prx/libc/include/Shutdown.hpp" #include "prx/libkernel/DirectMemory/DirectMemory.hpp" +#include "prx/libkernel/KernelErrors.hpp" #include "prx/libkernel/Pthread/include/Pthread.hpp" +#include #include #include #include @@ -23,7 +25,9 @@ #include #include #include +#include #include +#include #include #include #include @@ -197,6 +201,28 @@ int APS5_VABI getpid_nid_postfix(void) { return static_cast(pid); } +int APS5_VABI sceKernelGetAppInfo(int pid, void* info) { + if (info == nullptr) return SCE_KERNEL_ERROR_EFAULT; + if (pid != getpid_nid_postfix()) return SCE_KERNEL_ERROR_ESRCH; + constexpr std::size_t TitleIdOffset = 0x10; + constexpr std::size_t TitleIdBytes = 10; + char titleId[TitleIdBytes] = {}; + std::ifstream param(ResolvePath_nid_no_patch("/app0/sce_sys/param.json"), std::ios::binary); + if (param) { + const std::string text((std::istreambuf_iterator(param)), std::istreambuf_iterator()); + const auto key = text.find("\"titleId\""); + const auto colon = key == std::string::npos ? std::string::npos : text.find(':', key); + const auto open = colon == std::string::npos ? std::string::npos : text.find('"', colon); + const auto close = open == std::string::npos ? std::string::npos : text.find('"', open + 1); + if (close != std::string::npos) text.copy(titleId, std::min(close - open - 1, TitleIdBytes - 1), open + 1); + } + auto* bytes = static_cast(info); + const std::uint32_t appId = 1; + std::memcpy(bytes, &appId, sizeof(appId)); + std::memcpy(bytes + TitleIdOffset, titleId, TitleIdBytes); + return 0; +} + int APS5_VABI getuid_nid_postfix(void) { return 0; } diff --git a/core/libs/tests/GuestAppInfo.cpp b/core/libs/tests/GuestAppInfo.cpp new file mode 100644 index 000000000..cb01a1ba3 --- /dev/null +++ b/core/libs/tests/GuestAppInfo.cpp @@ -0,0 +1,44 @@ +#include "SceTypes.hpp" +#include +#include +#include +#include +#include +#include + +extern "C" { +int APS5_VABI sceKernelGetAppInfo(int pid, void* info); +int APS5_VABI getpid_nid_postfix(void); +} + +static constexpr int SCE_KERNEL_ERROR_ESRCH = static_cast(0x80020003); +static constexpr int SCE_KERNEL_ERROR_EFAULT = static_cast(0x8002000e); + +static void Require(bool value) { if (!value) std::abort(); } + +struct AppInfo { + std::uint32_t appId; + std::uint64_t unknown1; + char titleId[14]; + char unknown2[0x3c]; +}; + +int main() { + const auto previous = std::filesystem::current_path(); + const auto root = std::filesystem::temp_directory_path() / ("guest_app_info_" + std::to_string(getpid_nid_postfix())); + std::filesystem::remove_all(root); + std::filesystem::create_directories(root / "app0" / "sce_sys"); + std::ofstream(root / "app0" / "sce_sys" / "param.json") << "{\n \"contentId\": \"IV0000-PPSA97358_00-XPSEMU0000000000\",\n \"titleId\": \"PPSA97358\"\n}\n"; + std::filesystem::current_path(root); + AppInfo info; + std::memset(&info, 0x5a, sizeof(info)); + const int self = getpid_nid_postfix(); + Require(sceKernelGetAppInfo(self, &info) == 0); + Require(info.appId == 1); + Require(std::strcmp(info.titleId, "PPSA97358") == 0); + Require(info.unknown2[0] == 0x5a); + Require(sceKernelGetAppInfo(self + 1, &info) == SCE_KERNEL_ERROR_ESRCH); + Require(sceKernelGetAppInfo(self, nullptr) == SCE_KERNEL_ERROR_EFAULT); + std::filesystem::current_path(previous); + std::filesystem::remove_all(root); +} From 3f32c12fd858a0ed80cd00519793a4647d7d75f2 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sat, 10 Oct 2026 18:12:17 -0500 Subject: [PATCH 283/460] fix(libkernel): read sceKernelGetAppInfo's title id through AppMetadata The title id now comes from GetAppTitleId, so param.json has one parser and its validation (a missing file or titleId throws). Another pid returns EPERM and a null info returns 0 without writing, as shadPS4 does. app_id stays 1, the running title's id in libSceSystemService, which is inferred, not measured. guest_app_info runs the call without param.json in a child process and checks that the field between app_id and the title id is left alone. AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- .../libs/prx/libkernel/System/src/Process.cpp | 23 ++++------ core/libs/tests/GuestAppInfo.cpp | 42 +++++++++++++++---- 2 files changed, 40 insertions(+), 25 deletions(-) diff --git a/core/libs/prx/libkernel/System/src/Process.cpp b/core/libs/prx/libkernel/System/src/Process.cpp index 1ee262b02..b90c2b628 100644 --- a/core/libs/prx/libkernel/System/src/Process.cpp +++ b/core/libs/prx/libkernel/System/src/Process.cpp @@ -13,10 +13,10 @@ #include "prx/libc/include/General.hpp" #include "prx/libc/include/ApplicationHeap.hpp" #include "prx/libc/include/Shutdown.hpp" +#include "prx/libkernel/AppMetadata/include/AppMetadata.hpp" #include "prx/libkernel/DirectMemory/DirectMemory.hpp" #include "prx/libkernel/KernelErrors.hpp" #include "prx/libkernel/Pthread/include/Pthread.hpp" -#include #include #include #include @@ -25,9 +25,7 @@ #include #include #include -#include #include -#include #include #include #include @@ -202,24 +200,17 @@ int APS5_VABI getpid_nid_postfix(void) { } int APS5_VABI sceKernelGetAppInfo(int pid, void* info) { - if (info == nullptr) return SCE_KERNEL_ERROR_EFAULT; - if (pid != getpid_nid_postfix()) return SCE_KERNEL_ERROR_ESRCH; + if (pid != getpid_nid_postfix()) return SCE_KERNEL_ERROR_EPERM; + if (info == nullptr) return 0; constexpr std::size_t TitleIdOffset = 0x10; constexpr std::size_t TitleIdBytes = 10; - char titleId[TitleIdBytes] = {}; - std::ifstream param(ResolvePath_nid_no_patch("/app0/sce_sys/param.json"), std::ios::binary); - if (param) { - const std::string text((std::istreambuf_iterator(param)), std::istreambuf_iterator()); - const auto key = text.find("\"titleId\""); - const auto colon = key == std::string::npos ? std::string::npos : text.find(':', key); - const auto open = colon == std::string::npos ? std::string::npos : text.find('"', colon); - const auto close = open == std::string::npos ? std::string::npos : text.find('"', open + 1); - if (close != std::string::npos) text.copy(titleId, std::min(close - open - 1, TitleIdBytes - 1), open + 1); - } + const auto titleId = GetAppTitleId_nid_postfix(); + char field[TitleIdBytes] = {}; + std::strncpy(field, titleId.value, sizeof(field) - 1); auto* bytes = static_cast(info); const std::uint32_t appId = 1; std::memcpy(bytes, &appId, sizeof(appId)); - std::memcpy(bytes + TitleIdOffset, titleId, TitleIdBytes); + std::memcpy(bytes + TitleIdOffset, field, TitleIdBytes); return 0; } diff --git a/core/libs/tests/GuestAppInfo.cpp b/core/libs/tests/GuestAppInfo.cpp index cb01a1ba3..e39662613 100644 --- a/core/libs/tests/GuestAppInfo.cpp +++ b/core/libs/tests/GuestAppInfo.cpp @@ -5,14 +5,14 @@ #include #include #include +#include extern "C" { int APS5_VABI sceKernelGetAppInfo(int pid, void* info); int APS5_VABI getpid_nid_postfix(void); } -static constexpr int SCE_KERNEL_ERROR_ESRCH = static_cast(0x80020003); -static constexpr int SCE_KERNEL_ERROR_EFAULT = static_cast(0x8002000e); +static constexpr int SCE_KERNEL_ERROR_EPERM = static_cast(0x80020001); static void Require(bool value) { if (!value) std::abort(); } @@ -23,22 +23,46 @@ struct AppInfo { char unknown2[0x3c]; }; -int main() { - const auto previous = std::filesystem::current_path(); - const auto root = std::filesystem::temp_directory_path() / ("guest_app_info_" + std::to_string(getpid_nid_postfix())); +static std::filesystem::path EnterTemporaryRoot(std::string_view name) { + const auto root = std::filesystem::temp_directory_path() / (std::string(name) + std::to_string(getpid_nid_postfix())); std::filesystem::remove_all(root); - std::filesystem::create_directories(root / "app0" / "sce_sys"); - std::ofstream(root / "app0" / "sce_sys" / "param.json") << "{\n \"contentId\": \"IV0000-PPSA97358_00-XPSEMU0000000000\",\n \"titleId\": \"PPSA97358\"\n}\n"; + std::filesystem::create_directories(root); std::filesystem::current_path(root); + return root; +} + +int main(int argc, char** argv) { + if (argc == 3 && std::string_view(argv[1]) == "--without-param-json") { + std::filesystem::current_path(argv[2]); + AppInfo info{}; + sceKernelGetAppInfo(getpid_nid_postfix(), &info); + return 0; + } + const auto previous = std::filesystem::current_path(); + const auto root = EnterTemporaryRoot("guest_app_info_"); + std::filesystem::create_directories(root / "app0" / "sce_sys"); + std::ofstream(root / "app0" / "sce_sys" / "param.json") << "{\n \"contentId\": \"IV0000-PPSA97358_00-XPSEMU0000000000\",\n \"localizedParameters\": {\"defaultLanguage\": \"en-US\", \"en-US\": {\"titleName\": \"XPSemu\"}},\n \"titleId\": \"PPSA97358\"\n}\n"; AppInfo info; std::memset(&info, 0x5a, sizeof(info)); const int self = getpid_nid_postfix(); Require(sceKernelGetAppInfo(self, &info) == 0); Require(info.appId == 1); Require(std::strcmp(info.titleId, "PPSA97358") == 0); + Require(info.unknown1 == 0x5a5a5a5a5a5a5a5aull); Require(info.unknown2[0] == 0x5a); - Require(sceKernelGetAppInfo(self + 1, &info) == SCE_KERNEL_ERROR_ESRCH); - Require(sceKernelGetAppInfo(self, nullptr) == SCE_KERNEL_ERROR_EFAULT); + Require(sceKernelGetAppInfo(self + 1, &info) == SCE_KERNEL_ERROR_EPERM); + Require(sceKernelGetAppInfo(self, nullptr) == 0); std::filesystem::current_path(previous); std::filesystem::remove_all(root); + const auto empty = std::filesystem::temp_directory_path() / ("guest_app_info_empty_" + std::to_string(self)); + std::filesystem::remove_all(empty); + std::filesystem::create_directories(empty); + const std::string arguments = "\"" + std::string(argv[0]) + "\" --without-param-json \"" + empty.string() + "\""; +#ifdef _WIN32 + const std::string command = "\"" + arguments + "\""; +#else + const std::string command = arguments; +#endif + Require(std::system(command.c_str()) != 0); + std::filesystem::remove_all(empty); } From d8af71c0cad1745d35ae6bb1b25065ed026c7065 Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Sat, 10 Oct 2026 17:04:12 -0400 Subject: [PATCH 284/460] feat(libkernel): implement pthread_kill --- core/libs/prx/libkernel/System/src/Signals.cpp | 10 ++++++++++ core/libs/tests/GuestSignals.cpp | 13 +++++++++++++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 24 insertions(+) diff --git a/core/libs/prx/libkernel/System/src/Signals.cpp b/core/libs/prx/libkernel/System/src/Signals.cpp index 8650e8f7f..35bfc4e4b 100644 --- a/core/libs/prx/libkernel/System/src/Signals.cpp +++ b/core/libs/prx/libkernel/System/src/Signals.cpp @@ -1,3 +1,4 @@ +#include "SceTypes.hpp" #include "prx/libc/include/General.hpp" #include "prx/libc/include/general/VabiMacros.hpp" #include @@ -11,10 +12,12 @@ #include #include #include +#include #include extern "C" int* APS5_VABI __error_nid_postfix(); extern "C" int APS5_VABI getpid_nid_postfix(void); +extern "C" Pthread APS5_VABI scePthreadSelf(void); struct GuestSignalSet { std::uint32_t bits[4]; @@ -199,6 +202,13 @@ int APS5_VABI kill_nid_postfix(int pid, int guest) { if (pid != self && pid != 0 && pid != -self) { *__error_nid_postfix() = 3; return -1; } return guest == 0 ? 0 : raise_nid_postfix(guest); } +int APS5_VABI pthread_kill_nid_postfix(Pthread thread, int guest) { + if (guest < 0 || guest > MaxSignal) return 22; + if (thread == nullptr) return 3; + if (guest == 0) return 0; + if (thread != scePthreadSelf()) NotImplemented_nid_no_patch(("pthread_kill signal " + std::to_string(guest) + " to another thread").c_str()); + return raise_nid_postfix(guest) == 0 ? 0 : *__error_nid_postfix(); +} int APS5_VABI sigaltstack_nid_postfix(const GuestStack* stack, GuestStack* previous) { GuestStack replacement = alternateStack; if (stack != nullptr) { diff --git a/core/libs/tests/GuestSignals.cpp b/core/libs/tests/GuestSignals.cpp index a1453fc98..187e6ae36 100644 --- a/core/libs/tests/GuestSignals.cpp +++ b/core/libs/tests/GuestSignals.cpp @@ -10,6 +10,8 @@ using Handler = void (APS5_VABI *)(int); extern "C" { Handler APS5_VABI signal_nid_postfix(int, Handler); int APS5_VABI raise_nid_postfix(int); +int APS5_VABI pthread_kill_nid_postfix(void*, int); +void* APS5_VABI scePthreadSelf(); int APS5_VABI sigaction_nid_postfix(int, const void*, void*); int APS5_VABI sigprocmask_nid_postfix(int, const void*, void*); int APS5_VABI pthread_sigmask_nid_postfix(int, const void*, void*); @@ -49,6 +51,17 @@ int main() { Require(_is_signal_return_nid_postfix(0) == 0); received = 0; Require(raise_nid_postfix(15) == 0 && received == 15); + received = 0; + Require(pthread_kill_nid_postfix(scePthreadSelf(), 15) == 0 && received == 15); + received = 0; + Require(pthread_kill_nid_postfix(scePthreadSelf(), 0) == 0 && received == 0); + Require(pthread_kill_nid_postfix(scePthreadSelf(), 200) == 22); + Require(pthread_kill_nid_postfix(scePthreadSelf(), -1) == 22); + Require(pthread_kill_nid_postfix(nullptr, 15) == 3); + bool threw = false; + try { pthread_kill_nid_postfix(reinterpret_cast(std::uintptr_t{0x10}), 15); } + catch (const std::runtime_error&) { threw = true; } + Require(threw && received == 0); Require(signal_nid_postfix(15, ignore) != invalid); received = 0; Require(raise_nid_postfix(15) == 0 && received == 0); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 74a13e50a..ccd7c89fa 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -512,6 +512,7 @@ Throughout the project, every function at every stage either **does exactly what - [sigaction](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) records the handler, flags and mask of every signal from 1 to 128 and reports them back, following FreeBSD's `sigaction` (`EINVAL` for other signals, and for a handler on `SIGKILL` or `SIGSTOP`); `signal` records its handler with `SA_RESTART`, as FreeBSD's libc does. Only handlers without `SA_SIGINFO` on the signals `signal` already delivers (`SIGINT`, `SIGILL`, `SIGABRT`, `SIGFPE`, `SIGSEGV`, `SIGTERM`) run, the same way `signal` handlers do; their mask and flags are not applied. `SA_SIGINFO` handlers and other signals (`SIGBUS`, `SIGHUP`, ...) are recorded but never called, since that needs FreeBSD's `siginfo_t` and `ucontext_t`; `raise` of an `SA_SIGINFO` handler on a signal `signal` delivers throws instead of terminating, and `raise` of the other signals still returns `EINVAL` - [kill](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) follows FreeBSD's `kern_kill` and `killpg1` for the only guest process: `EINVAL` for signals outside 0 to 128, signal 0 only checks that the target exists, and its own pid, 0 and minus its own pid (the guest is taken to lead its own process group) signal it through `raise`, so they share `raise`'s limits above. Any other pid, and -1, which skips the caller, report `ESRCH`, so `EPERM` is never reported - [sceKernelFcntl](../../core/libs/prx/libkernel/File/src/Open.cpp) (libkernel) handles socket descriptors through the POSIX `fcntl` export (`F_GETFL`, `F_SETFL` with `O_NONBLOCK`) and returns its guest errno as `SCE_KERNEL_ERROR_*`, the form the other `sceKernel` file exports use; the command semantics follow FreeBSD's [fcntl](https://github.com/freebsd/freebsd-src/blob/stable/11/sys/kern/kern_descrip.c), and a closed descriptor returns `SCE_KERNEL_ERROR_EBADF`. File descriptors throw: their open flags are not tracked, and `F_DUPFD`, `F_GETFD` and `F_SETFD` are unsupported on all descriptors. The return values and error codes are unverified on a console +- [pthread_kill](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) returns the error number like FreeBSD's `pthread_kill`: `EINVAL` for signals outside 0 to 128 and `ESRCH` for a null thread. Signal 0 reports any other thread handle as existing, without checking that the thread is still alive. A signal to the calling thread goes through `raise`, so it shares `raise`'s limits; a signal to another thread throws, since guest signals are not delivered to a chosen thread - [AJM](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp), [APR](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) and [NGS2](../../core/libs/prx/libSceNgs2.native/src/System.cpp) state is allocated with `new` and never freed on purpose: title threads still call into these libraries while `std::exit` runs static destructors, so ordinary statics would be destroyed under them. Keep them leaked; `guest_ajm_teardown` covers the AJM case on Unix. - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields From 0e88f3014de0045f472e2ca73650063cae95bc9d Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Sat, 10 Oct 2026 19:21:40 -0400 Subject: [PATCH 285/460] fix(libkernel): name pthread_kill's errors and test the signal range edge --- core/libs/prx/libkernel/System/src/Signals.cpp | 5 +++-- core/libs/tests/GuestSignals.cpp | 2 ++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libkernel/System/src/Signals.cpp b/core/libs/prx/libkernel/System/src/Signals.cpp index 35bfc4e4b..0ce268890 100644 --- a/core/libs/prx/libkernel/System/src/Signals.cpp +++ b/core/libs/prx/libkernel/System/src/Signals.cpp @@ -1,5 +1,6 @@ #include "SceTypes.hpp" #include "prx/libc/include/General.hpp" +#include "prx/libkernel/Pthread/Posix/Common.hpp" #include "prx/libc/include/general/VabiMacros.hpp" #include #include @@ -203,8 +204,8 @@ int APS5_VABI kill_nid_postfix(int pid, int guest) { return guest == 0 ? 0 : raise_nid_postfix(guest); } int APS5_VABI pthread_kill_nid_postfix(Pthread thread, int guest) { - if (guest < 0 || guest > MaxSignal) return 22; - if (thread == nullptr) return 3; + if (guest < 0 || guest > MaxSignal) return PosixThread::GUEST_EINVAL; + if (thread == nullptr) return PosixThread::GUEST_ESRCH; if (guest == 0) return 0; if (thread != scePthreadSelf()) NotImplemented_nid_no_patch(("pthread_kill signal " + std::to_string(guest) + " to another thread").c_str()); return raise_nid_postfix(guest) == 0 ? 0 : *__error_nid_postfix(); diff --git a/core/libs/tests/GuestSignals.cpp b/core/libs/tests/GuestSignals.cpp index 187e6ae36..0f048e414 100644 --- a/core/libs/tests/GuestSignals.cpp +++ b/core/libs/tests/GuestSignals.cpp @@ -58,6 +58,8 @@ int main() { Require(pthread_kill_nid_postfix(scePthreadSelf(), 200) == 22); Require(pthread_kill_nid_postfix(scePthreadSelf(), -1) == 22); Require(pthread_kill_nid_postfix(nullptr, 15) == 3); + Require(pthread_kill_nid_postfix(nullptr, 128) == 3); + Require(pthread_kill_nid_postfix(nullptr, 129) == 22); bool threw = false; try { pthread_kill_nid_postfix(reinterpret_cast(std::uintptr_t{0x10}), 15); } catch (const std::runtime_error&) { threw = true; } From 39748573b09986daf0e0791dd6caa9e284624286 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sat, 10 Oct 2026 18:26:52 -0500 Subject: [PATCH 286/460] docs(technical-debt): record what sceKernelGetAppInfo infers AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- docs/dev/TechnicalDebt.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 74a13e50a..e9735e514 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -47,6 +47,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceVideoOutGetOutputStatusWithVrr](../../core/libs/prx/libSceVideoOut/Unimplemented.cpp) (libSceVideoOut, NID `gWT7X8H0bYs`) - the name hashes to the imported NID. PPSA26344 version 02.002.000 passes a VideoOut handle and an output pointer at three call sites. It compares a 32-bit value at byte 4 with 2, tests bit 0x10 at byte 16, and, when set, compares low 16-bit values at bytes 48, 50 and 52 with 48, 60 and 120. The full output layout, size, field meanings and console errors are unknown. Every call throws without reading or writing caller memory. - [getuid, geteuid, getgid, getegid and issetugid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) report the root identity (0) and a process that never changed credentials (`issetugid` 0), as a jailbroken process has; the host user and group are not exposed to the guest. The values a title sees on a console are unverified +- [sceKernelGetAppInfo](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - only `app_id` and the title ID (offset 0x10, up to 9 characters plus a terminator) are written; callers declare the structure with different sizes, so the rest is left alone. `app_id` is 1, the running title's id in libSceSystemService, which is a guess, not something measured. The error codes follow shadPS4: `EPERM` for another process, and 0 without writing anything for a null pointer. - [sceFontSetScriptLanguage, sceFontGetScriptLanguage, sceFontSetTypographicDesign, sceFontGetTypographicDesign](../../core/libs/prx/libSceFont/src/Style.cpp) (libSceFont) - signatures follow [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.cs), not confirmed for the PS5. The handle keeps the language set for each script and the feature set for each typographic design and returns them, with 0 for one never set; the console default and its valid ranges are unknown, so any value is accepted. Neither setting changes shaping or rendering yet - [Packed pixel ancillary word](../../core/shader/recompiler/Optimization/src/ConstantFolder.cpp) (shader recompiler) - the sample index at bits 8-11 and the render target index at bits 16-28 follow #295, and their equality with the SampleId and Layer builtins is inferred, not measured. A value passed through a select, move or phi is rebuilt from those two fields with bits 0-7, 12-15 and 29-31 zero. That is exact only because every read of the word must be an extract inside one modelled field; any other read throws. The other bits are unmodelled and their RDNA 2 layout is unknown: the RDNA 3.5 ISA is reported to give prim type at bits 0-1, VRS rates at bits 2-5 and eye id at bit 7, which was not checked against RDNA 2 - [sceAudioPropagationSourceSetAudioPath](../../core/libs/prx/libSceAudioPropagation/Export.cpp) (libSceAudioPropagation, NID `tKSmk2JsMAA`) - `(uint64_t path, const void* data, float gain)` follows [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/5dfdb4b0b166e83dc9ae275001afb695f12c7174/src/SharpProspero/Interop/Audio/AudioPropagation.cs) and SAROS's call, which passes a handle from `sceAudioPropagationSourceGetAudioPath`, path data and gain 1.0. Path updates are unsupported and every call throws without reading or writing caller memory. The no-propagation model yields no path handles; path allocation, ownership, rendering and console error codes remain unknown. From bbe26912dc5c3c5b36dfa65609c8a5a8f55823f9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 03:13:39 +0200 Subject: [PATCH 287/460] fix(agc): view 4_4_4_4 textures as B4G4R4A4, which every Vulkan device can sample Co-Authored-By: Claude --- .../libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp | 6 +++--- core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp | 2 +- docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp index 6926b8df7..fdee455cf 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp @@ -67,7 +67,7 @@ constexpr FormatEntry kFormatLookup[] = { {132, VK_FORMAT_E5B9G9R9_UFLOAT_PACK32, 4, false}, {133, VK_FORMAT_R5G6B5_UNORM_PACK16, 2, false}, {134, VK_FORMAT_A1R5G5B5_UNORM_PACK16, 2, false}, - {136, VK_FORMAT_R4G4B4A4_UNORM_PACK16, 2, false}, + {136, VK_FORMAT_B4G4R4A4_UNORM_PACK16, 2, false}, {169, VK_FORMAT_BC1_RGBA_UNORM_BLOCK, 8, true}, {170, VK_FORMAT_BC1_RGBA_SRGB_BLOCK, 8, true}, {171, VK_FORMAT_BC2_UNORM_BLOCK, 16, true}, @@ -159,11 +159,11 @@ VkComponentSwizzle TextureComponentChannel(std::uint32_t guestFormat, VkComponen const auto index = static_cast(component - VK_COMPONENT_SWIZZLE_R); constexpr std::array bgrb{VK_COMPONENT_SWIZZLE_B, VK_COMPONENT_SWIZZLE_G, VK_COMPONENT_SWIZZLE_R, VK_COMPONENT_SWIZZLE_B}; constexpr std::array bgra{VK_COMPONENT_SWIZZLE_B, VK_COMPONENT_SWIZZLE_G, VK_COMPONENT_SWIZZLE_R, VK_COMPONENT_SWIZZLE_A}; - constexpr std::array abgr{VK_COMPONENT_SWIZZLE_A, VK_COMPONENT_SWIZZLE_B, VK_COMPONENT_SWIZZLE_G, VK_COMPONENT_SWIZZLE_R}; + constexpr std::array argb{VK_COMPONENT_SWIZZLE_A, VK_COMPONENT_SWIZZLE_R, VK_COMPONENT_SWIZZLE_G, VK_COMPONENT_SWIZZLE_B}; switch (guestFormat) { case 133: return bgrb[index]; case 134: return bgra[index]; - case 136: return abgr[index]; + case 136: return argb[index]; default: return component; } } diff --git a/core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp b/core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp index 00a5f561a..26ad1c130 100644 --- a/core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/TextureFormat.cpp @@ -62,7 +62,7 @@ int channelLowBit(VkFormat format, VkComponentSwizzle channel) { switch (format) { case VK_FORMAT_R5G6B5_UNORM_PACK16: return std::array{11, 5, 0, -1}[index]; case VK_FORMAT_A1R5G5B5_UNORM_PACK16: return std::array{10, 5, 0, 15}[index]; - case VK_FORMAT_R4G4B4A4_UNORM_PACK16: return std::array{12, 8, 4, 0}[index]; + case VK_FORMAT_B4G4R4A4_UNORM_PACK16: return std::array{4, 8, 12, 0}[index]; default: throw std::runtime_error("no channel layout for packed format " + std::to_string(format)); } } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 74a13e50a..237d82bed 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -465,7 +465,7 @@ Throughout the project, every function at every stage either **does exactly what - [Negative color-texture comparison wrap indices](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (libSceAgcDriver) use explicit `OpSRem` normalization to implement modulo; the negative-offset point-wrap regression fails with direct `OpSMod` and passes with normalization on Mesa 22.3.6 llvmpipe (LLVM 15.0.6), but negative-index wrap has not been measured on PS5 hardware. - [libScePlayerInvitationDialog](../../core/libs/prx/libScePlayerInvitationDialog/libScePlayerInvitationDialog.cpp) simulates dialog completion without displaying UI or sending invitations, so `scePlayerInvitationDialogGetResult` reports `USER_CANCELED`; its parameter ABI remains unverified. The 40-byte result layout (error code, result, 32 reserved bytes) is taken from [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Dialog/PlayerInvitationDialog.cs) and the error codes from the other common dialogs, not confirmed on a title. - [Converted unorm images](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (10_11_11_UNORM, read and written through an R32_UINT view): samples, gathers, LOD queries, 16-bit data and a DST_SEL of the missing W channel throw. -- [Packed 16-bit textures](../../core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp) (libSceAgcDriver) - `5_6_5`, `1_5_5_5` and `4_4_4_4` are viewed as R5G6B5, A1R5G5B5 and R4G4B4A4 with a swizzle that reads X from the low bits, and a DST_SEL of the missing W of `5_6_5` reads X. Measured on RDNA2 (gfx1035), not on the console: through `image_load` and `image_sample_lz`, X, Y, Z (and W of `1_5_5_5` and `4_4_4_4`) come from the low, middle and high fields, and W of `5_6_5` returns X (the low 5 bits), not 1. +- [Packed 16-bit textures](../../core/libs/prx/libSceAgcDriver/Graphics/src/TextureFormat.cpp) (libSceAgcDriver) - `5_6_5`, `1_5_5_5` and `4_4_4_4` are viewed as R5G6B5, A1R5G5B5 and B4G4R4A4 (the 4-bit format every Vulkan device can sample; R4G4B4A4 is optional and lavapipe lacks it) with a swizzle that reads X from the low bits, and a DST_SEL of the missing W of `5_6_5` reads X. Measured on RDNA2 (gfx1035), not on the console: through `image_load` and `image_sample_lz`, X, Y, Z (and W of `1_5_5_5` and `4_4_4_4`) come from the low, middle and high fields, and W of `5_6_5` returns X (the low 5 bits), not 1. - [Converted unorm image loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) divide by 2^n - 1 with the buffer format path's `OpFDiv`, which Vulkan allows to be up to 2.5 ULP off, so some drivers are not bit-exact. - [64-bit LDS atomics](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (`ds_*_u64`, `ds_*_b64`, `ds_*_f64`) read and write the two dwords under a workgroup lock stored after the guest LDS, as LDS is a 32-bit array without 64-bit atomics. On RDNA2 (gfx1036) the LDS serialises a 32-bit atomic on a slot's high dword and a 64-bit atomic on that slot: 128 lanes of `ds_add_u32` and 128 lanes of `ds_add_u64` on one slot sum exactly. Only that pair was measured. Every 32-bit LDS atomic of a program that contains a 64-bit LDS atomic therefore takes the same lock, including a 32-bit `flat_atomic_*` whose address lands in the LDS aperture (a 64-bit one takes it already); the flat forms were not measured against a 64-bit atomic either (the other 32-bit LDS atomics, `ds_min/max`, `ds_cmpst`, the float forms, `ds_inc/dec`, `ds_mskor` and `ds_wrxchg`, are extrapolated from `ds_add_u32` and were not measured against a 64-bit atomic); a program without a 64-bit LDS atomic keeps its native 32-bit atomics. `ds_condxchg32_rtn_b64` is two `ds_mskor` halves, each under the lock but not atomic as a pair, so a 64-bit atomic between the two halves is not excluded. Sub-dword writes (`ds_write_b8`, `ds_write_b16` and their `_d16_hi` forms) and plain `ds_read_*`, `ds_write_*`, `ds_append_u32` and `ds_consume_u32` are not taken under the lock, so a plain access racing a 64-bit atomic on the same dwords can see or overwrite one half of it; whether the hardware orders them was not measured. The lock serialises the workgroup's LDS atomics of such a program; the throughput cost was not measured. `VK_KHR_workgroup_memory_explicit_layout` with `shaderSharedInt64Atomics` would make both widths native and is not used. A 64-bit atomic at an address that is 4-byte but not 8-byte aligned faults the wave on gfx1036, as on gfx1035. - [LDS float atomics](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (`ds_min/max/cmpst_f32`, `ds_min/max/cmpst_f64` and their `_rtn` forms) follow the hardware measured on RDNA2 (gfx1036) over 14 x 14 operand sets (zeros, ones, infinities, denormals, quiet and signalling NaNs of both signs and with payloads) in IEEE 0 and 1 with every value 0-3 of the type's denormal field: both operands are flushed to a signed zero when bit 0 of the field is clear (0 and 2; 1 and 3 keep them; the f32 field for the f32 forms, the f16/f64 field for the f64 forms; a shader without a known float mode flushes f32 and keeps f64), then a signalling-NaN data operand is stored quieted, else a signalling-NaN old value is stored quieted, else a quiet-NaN old value is replaced by the data, else a quiet-NaN data operand loses, else the ordered compare with -0 below +0 decides, and the flushed value is what is written, also when the old value wins. `cmpst` compares the flushed bits (both zeros equal, NaN never equal) and leaves memory unchanged when they differ. The returned value is always the old value as it was in memory. `IEEE_MODE` has no effect. The same bit rule serves the buffer float atomics (`AtomicFloatBits`). Not measured: a denormal or signalling-NaN `data1` of `cmpst` (stored as given), and `ds_add_f32`, which keeps the host's `OpFAdd`. From 7755e1d05c416385a8f79cfdb5676e0f9ab2ff9c Mon Sep 17 00:00:00 2001 From: Chad Date: Sat, 10 Oct 2026 20:14:23 -0500 Subject: [PATCH 288/460] fix(relinker): leave needed libraries nothing imports from out of the start-up load list Co-Authored-By: Claude Sonnet 5.5 --- .../src/windows/WindowsEntryStubBuilder.cpp | 19 ++++--------------- .../tests/test_windows_import_modules.py | 2 +- 2 files changed, 5 insertions(+), 16 deletions(-) diff --git a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp index 74dbb30ee..de1e413cc 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp @@ -301,6 +301,7 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con requireNonzero(directoryError, 0xc000000du); for (std::size_t index = 0; index < libraries.size(); ++index) { + if (index >= guestModules.size() && !imported[index]) continue; if (absolutePath && index >= guestModules.size()) { code.Rip({0x48, 0x8d, 0x0d}, libraryPaths[index]); } else { @@ -335,12 +336,6 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con code.Emit({0x48, 0x85, 0xc0}); const auto loadSucceeded = code.Branch({0x0f, 0x85}); captureLastError(); - std::optional unusedMissing; - if (index >= guestModules.size() && !imported[index]) { - code.Rip({0x8b, 0x05}, lastError); - code.Emit({0x83, 0xf8, 0x7e}); - unusedMissing = code.Branch({0x0f, 0x84}); - } writeString(loadFailed, true); writeString(failedModule, true); writeString(resolvedPaths[index], true); @@ -358,13 +353,6 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con if (dependencyDiagnostics) { writeString(loaded); } - if (unusedMissing) { - const auto next = code.Branch({0xe9}); - code.PatchBranch(*unusedMissing, code.GetRva()); - if (!dependencyDiagnostics) writeString(resolvedPaths[index], true); - writeString(skippedModule, true); - code.PatchBranch(next, code.GetRva()); - } } std::vector unresolvedBranches; @@ -480,9 +468,10 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con for (const auto branch : unresolvedBranches) code.PatchBranch(branch, code.GetRva()); writeString(searched, true); - for (const auto resolvedPath : resolvedPaths) { + for (std::size_t index = 0; index < resolvedPaths.size(); ++index) { + if (index >= guestModules.size() && !imported[index]) continue; writeString(indent, true); - writeString(resolvedPath, true); + writeString(resolvedPaths[index], true); writeString(newline, true); } writeLastError(); diff --git a/core/relinker/relinker/tests/test_windows_import_modules.py b/core/relinker/relinker/tests/test_windows_import_modules.py index cd96d1276..0eeaabd36 100644 --- a/core/relinker/relinker/tests/test_windows_import_modules.py +++ b/core/relinker/relinker/tests/test_windows_import_modules.py @@ -342,7 +342,7 @@ def main(): assert result.returncode == 0, (symbol, module_name, result.stdout, result.stderr) if os.name == 'nt': run = subprocess.run([str(output)], capture_output=True, text=True, timeout=30) - assert run.returncode == 11 and 'unused.prx -> not found; nothing is imported from it' in run.stderr, (symbol, module_name, run.returncode, run.stdout, run.stderr) + assert run.returncode == 11 and 'unused.prx' not in run.stderr, (symbol, module_name, run.returncode, run.stdout, run.stderr) print('Windows module-scoped import tests passed') From 60f12c888d85e6bac7ec3cf6d09a3e4eb1977cae Mon Sep 17 00:00:00 2001 From: Dean Galvin Date: Sun, 11 Oct 2026 14:15:27 +1300 Subject: [PATCH 289/460] fix(agc): compare host imports when device writes reach the write watch The import probe only checked whether a GPU read of an imported range reports its pages as written. lavapipe runs command buffers on CPU threads, so its writes into an import go through the page tables and are collected as CPU stores. In agc_unit_shadow the whole publish's copy into the import is then collected as a CPU store over unit 3, and the next partial publish drops the slab's newer bytes as stale: "the partial publish did not copy exactly the partly covered unit". The probe now also fills the scratch range on the GPU and collects again. When the read or the fill reports pages as written, host and dma-buf imports are compared. lavapipe reports 0 of 4 pages after the read and 4 of 4 after the fill; NVIDIA reports 0 after the fill. The probe change is the first commit of #1046. --- .../Graphics/include/GuestBufferMemory.hpp | 1 + .../Graphics/src/GuestBufferMemory.cpp | 46 +++++++++++-------- .../prx/libSceAgcDriver/tests/Recorder.cpp | 2 +- 3 files changed, 30 insertions(+), 19 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp index f180e1e9b..b260da581 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp @@ -44,6 +44,7 @@ struct ImportProbe { std::uint32_t writtenAtImport = 0; std::uint32_t writtenAfterSubmit = 0; std::uint32_t writtenByCpu = 0; + std::uint32_t writtenAfterFill = 0; }; ImportProbe ProbeImportWriteProtection(const Context& context); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp index 1f6091d71..9ffe299e6 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp @@ -499,8 +499,8 @@ void decideImportWatch(const Context& context, HostImports& state) { state.unwatchImports = true; std::fprintf(stderr, "[write-watch] host imports resolve write protection: unknown (probe failed at %s, %d); imported ranges are compared\n", probe.failure, static_cast(probe.result)); } else { - state.unwatchImports = probe.writtenAfterSubmit != 0; - std::fprintf(stderr, "[write-watch] host imports resolve write protection: %s (%u of %u scratch pages written after a GPU read, %u after the import); imported ranges %s\n", state.unwatchImports ? "yes" : "no", probe.writtenAfterSubmit, probe.pages, probe.writtenAtImport, state.unwatchImports ? "are compared" : "stay watched"); + state.unwatchImports = probe.writtenAfterSubmit != 0 || probe.writtenAfterFill != 0; + std::fprintf(stderr, "[write-watch] host imports resolve write protection: %s (%u of %u scratch pages written after a GPU read, %u after a GPU write, %u after the import); imported ranges %s\n", state.unwatchImports ? "yes" : "no", probe.writtenAfterSubmit, probe.pages, probe.writtenAfterFill, probe.writtenAtImport, state.unwatchImports ? "are compared" : "stay watched"); } state.unwatchDmaBufImports = state.unwatchImports; if (!context.dmaBufImport) return; @@ -510,8 +510,8 @@ void decideImportWatch(const Context& context, HostImports& state) { std::fprintf(stderr, "[write-watch] dma-buf imports keep write protection: unknown (probe failed at %s, %d); imported ranges are compared\n", dmaBuf.failure, static_cast(dmaBuf.result)); return; } - state.unwatchDmaBufImports = dmaBuf.writtenAfterSubmit != 0 || dmaBuf.writtenByCpu == 0; - std::fprintf(stderr, "[write-watch] dma-buf imports keep write protection: %s (%u of %u scratch pages written after a GPU read, %u after the import, %u seen after a CPU store); imported ranges %s\n", state.unwatchDmaBufImports ? "no" : "yes", dmaBuf.writtenAfterSubmit, dmaBuf.pages, dmaBuf.writtenAtImport, dmaBuf.writtenByCpu, state.unwatchDmaBufImports ? "are compared" : "stay watched"); + state.unwatchDmaBufImports = dmaBuf.writtenAfterSubmit != 0 || dmaBuf.writtenAfterFill != 0 || dmaBuf.writtenByCpu == 0; + std::fprintf(stderr, "[write-watch] dma-buf imports keep write protection: %s (%u of %u scratch pages written after a GPU read, %u after a GPU write, %u after the import, %u seen after a CPU store); imported ranges %s\n", state.unwatchDmaBufImports ? "no" : "yes", dmaBuf.writtenAfterSubmit, dmaBuf.pages, dmaBuf.writtenAfterFill, dmaBuf.writtenAtImport, dmaBuf.writtenByCpu, state.unwatchDmaBufImports ? "are compared" : "stay watched"); #endif } @@ -1510,7 +1510,7 @@ void runImportProbe(const Context& context, std::uint64_t base, std::uint64_t by HostImport import{base, bytes, VK_NULL_HANDLE, VK_NULL_HANDLE, 0}; VkBuffer destination = VK_NULL_HANDLE; VkDeviceMemory destinationMemory = VK_NULL_HANDLE; - VkCommandBuffer commands = VK_NULL_HANDLE; + VkCommandBuffer commands[2]{}; VkFence fence = VK_NULL_HANDLE; bool submitted = false; const auto collect = [&](std::uint32_t& pages) { @@ -1540,24 +1540,34 @@ void runImportProbe(const Context& context, std::uint64_t base, std::uint64_t by VkCommandBufferAllocateInfo allocate{VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO}; allocate.commandPool = context.pool; allocate.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY; - allocate.commandBufferCount = 1; - if ((probe.result = context.Function("vkAllocateCommandBuffers")(context.device, &allocate, &commands)) != VK_SUCCESS) return "vkAllocateCommandBuffers"; + allocate.commandBufferCount = 2; + if ((probe.result = context.Function("vkAllocateCommandBuffers")(context.device, &allocate, commands)) != VK_SUCCESS) return "vkAllocateCommandBuffers"; VkCommandBufferBeginInfo begin{VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO}; begin.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT; - if ((probe.result = context.Function("vkBeginCommandBuffer")(commands, &begin)) != VK_SUCCESS) return "vkBeginCommandBuffer"; + if ((probe.result = context.Function("vkBeginCommandBuffer")(commands[0], &begin)) != VK_SUCCESS) return "vkBeginCommandBuffer"; const VkBufferCopy region{0, 0, bytes}; - context.Function("vkCmdCopyBuffer")(commands, import.buffer, destination, 1, ®ion); - if ((probe.result = context.Function("vkEndCommandBuffer")(commands)) != VK_SUCCESS) return "vkEndCommandBuffer"; + context.Function("vkCmdCopyBuffer")(commands[0], import.buffer, destination, 1, ®ion); + if ((probe.result = context.Function("vkEndCommandBuffer")(commands[0])) != VK_SUCCESS) return "vkEndCommandBuffer"; + if ((probe.result = context.Function("vkBeginCommandBuffer")(commands[1], &begin)) != VK_SUCCESS) return "vkBeginCommandBuffer"; + context.Function("vkCmdFillBuffer")(commands[1], import.buffer, 0, bytes, 0x02020202u); + if ((probe.result = context.Function("vkEndCommandBuffer")(commands[1])) != VK_SUCCESS) return "vkEndCommandBuffer"; const VkFenceCreateInfo fenceInfo{VK_STRUCTURE_TYPE_FENCE_CREATE_INFO}; if ((probe.result = context.Function("vkCreateFence")(context.device, &fenceInfo, nullptr, &fence)) != VK_SUCCESS) return "vkCreateFence"; - VkSubmitInfo submit{VK_STRUCTURE_TYPE_SUBMIT_INFO}; - submit.commandBufferCount = 1; - submit.pCommandBuffers = &commands; - if ((probe.result = context.Function("vkQueueSubmit")(context.queue, 1, &submit, fence)) != VK_SUCCESS) return "vkQueueSubmit"; - submitted = true; - if ((probe.result = context.Function("vkWaitForFences")(context.device, 1, &fence, VK_TRUE, 10'000'000'000ull)) != VK_SUCCESS) return "vkWaitForFences"; - submitted = false; + const auto execute = [&](VkCommandBuffer buffer) -> const char* { + if ((probe.result = context.Function("vkResetFences")(context.device, 1, &fence)) != VK_SUCCESS) return "vkResetFences"; + VkSubmitInfo submit{VK_STRUCTURE_TYPE_SUBMIT_INFO}; + submit.commandBufferCount = 1; + submit.pCommandBuffers = &buffer; + if ((probe.result = context.Function("vkQueueSubmit")(context.queue, 1, &submit, fence)) != VK_SUCCESS) return "vkQueueSubmit"; + submitted = true; + if ((probe.result = context.Function("vkWaitForFences")(context.device, 1, &fence, VK_TRUE, 10'000'000'000ull)) != VK_SUCCESS) return "vkWaitForFences"; + submitted = false; + return nullptr; + }; + if (const char* step = execute(commands[0])) return step; if (!collect(probe.writtenAfterSubmit)) return "the collect after the submission"; + if (const char* step = execute(commands[1])) return step; + if (!collect(probe.writtenAfterFill)) return "the collect after the fill"; *reinterpret_cast(base) = 2; if (!collect(probe.writtenByCpu)) return "the collect after a CPU store"; return nullptr; @@ -1565,7 +1575,7 @@ void runImportProbe(const Context& context, std::uint64_t base, std::uint64_t by probe.failure = run(); if (submitted) context.Function("vkQueueWaitIdle")(context.queue); if (fence != VK_NULL_HANDLE) context.Function("vkDestroyFence")(context.device, fence, nullptr); - if (commands != VK_NULL_HANDLE) context.Function("vkFreeCommandBuffers")(context.device, context.pool, 1, &commands); + if (commands[0] != VK_NULL_HANDLE) context.Function("vkFreeCommandBuffers")(context.device, context.pool, 2, commands); if (destination != VK_NULL_HANDLE) context.Function("vkDestroyBuffer")(context.device, destination, nullptr); if (destinationMemory != VK_NULL_HANDLE) context.Function("vkFreeMemory")(context.device, destinationMemory, nullptr); if (import.buffer != VK_NULL_HANDLE) destroyImport(context, import); diff --git a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp index cc3c4bff4..adb35a6d7 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp @@ -2294,7 +2294,7 @@ void importWatchTests(const Device& device) { } const auto probe = ProbeImportWriteProtection(context); Require(probe.failure == nullptr, std::string("(u) the import probe failed at ") + (probe.failure != nullptr ? probe.failure : "") + " (" + std::to_string(static_cast(probe.result)) + ")"); - std::cout << "import probe: " << probe.writtenAfterSubmit << " of " << probe.pages << " scratch pages written after a GPU read, " << probe.writtenAtImport << " after the import\n"; + std::cout << "import probe: " << probe.writtenAfterSubmit << " of " << probe.pages << " scratch pages written after a GPU read, " << probe.writtenAfterFill << " after a GPU write, " << probe.writtenAtImport << " after the import\n"; Require(probe.writtenByCpu != 0, "(u) a CPU store after the import probe's GPU read was not collected"); if (context.dmaBufImport) { const auto dmaBuf = ProbeDmaBufImportWriteProtection(context); From d78af2c7105287f97d499215386e96630997188a Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 23:19:30 +0000 Subject: [PATCH 290/460] fix(libkernel): route vector I/O on guest sockets --- core/libs/prx/libkernel/File/src/Stdio.cpp | 14 ++++++ .../Socket/include/SocketRuntime.hpp | 6 +++ core/libs/prx/libkernel/Socket/src/Udp.cpp | 47 +++++++++++++++++++ core/libs/tests/GuestSocketReadWrite.cpp | 29 ++++++++++++ core/libs/tests/GuestUdp.cpp | 13 +++++ docs/dev/TechnicalDebt.md | 2 +- 6 files changed, 110 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index daf19052f..61f56c803 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -27,6 +27,7 @@ struct KernelIovec { void* base; std::size_t length; }; +static_assert(sizeof(KernelIovec) == sizeof(GuestSockets::Iovec)); static constexpr int KERNEL_IOV_MAX = 1024; @@ -809,6 +810,11 @@ static std::int64_t TransferIovecs(int d, const KernelIovec* iov, int iovcnt, co if (offset != nullptr && total > static_cast(std::numeric_limits::max() - *offset)) return SceErrorFromErrno(GUEST_EINVAL); std::deque destinations; if (!write && !OpenIovecs(iov, iovcnt, destinations)) return SceErrorFromErrno(GUEST_EFAULT); + if (offset == nullptr && d >= GuestSockets::FirstDescriptor) { + const auto result = write ? GuestSockets::Writev(d, reinterpret_cast(iov), iovcnt) + : GuestSockets::Readv(d, reinterpret_cast(iov), iovcnt); + return result < 0 ? SceErrorFromErrno(*__error_nid_postfix()) : result; + } if (!write && File::IsRandomDevice(d)) return ReadRandomIovecs(d, iov, iovcnt); if (total == 0) { char none = 0; @@ -870,6 +876,10 @@ int64_t APS5_VABI sceKernelReadv(int d, const KernelIovec* iov, int iovcnt) { if (const int error = CheckIovecs(iov, iovcnt)) return error; std::deque destinations; if (!OpenIovecs(iov, iovcnt, destinations)) return SceErrorFromErrno(GUEST_EFAULT); + if (d >= GuestSockets::FirstDescriptor) { + const auto result = GuestSockets::Readv(d, reinterpret_cast(iov), iovcnt); + return result < 0 ? SceErrorFromErrno(*__error_nid_postfix()) : result; + } if (File::IsRandomDevice(d)) return ReadRandomIovecs(d, iov, iovcnt); const auto result = static_cast(::readv(d, NativeIovecs(iov), iovcnt)); return result < 0 ? SceErrorFromErrno(errno) : result; @@ -877,6 +887,10 @@ int64_t APS5_VABI sceKernelReadv(int d, const KernelIovec* iov, int iovcnt) { int64_t APS5_VABI sceKernelWritev(int d, const KernelIovec* iov, int iovcnt) { if (const int error = CheckIovecs(iov, iovcnt)) return error; + if (d >= GuestSockets::FirstDescriptor) { + const auto result = GuestSockets::Writev(d, reinterpret_cast(iov), iovcnt); + return result < 0 ? SceErrorFromErrno(*__error_nid_postfix()) : result; + } const auto result = static_cast(::writev(d, NativeIovecs(iov), iovcnt)); return result < 0 ? SceErrorFromErrno(errno) : result; } diff --git a/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp b/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp index 0703293dc..41bf8caf3 100644 --- a/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp +++ b/core/libs/prx/libkernel/Socket/include/SocketRuntime.hpp @@ -4,6 +4,10 @@ #include namespace GuestSockets { constexpr int FirstDescriptor = 0x10000000; +struct Iovec { + void* base; + std::size_t length; +}; int Duplicate(int descriptor); int DuplicateTo(int descriptor, int target); int Close(int descriptor); @@ -19,6 +23,8 @@ struct Interest { std::uintptr_t CurrentWaker(); void Wake(std::uintptr_t waker); bool WaitAny(const std::vector& interests, std::uint64_t deadlineNanos); +std::int64_t Readv(int descriptor, const Iovec* iov, int iovcnt); +std::int64_t Writev(int descriptor, const Iovec* iov, int iovcnt); } extern "C" bool GuestSocketIsOpen_nid_no_patch(int descriptor); diff --git a/core/libs/prx/libkernel/Socket/src/Udp.cpp b/core/libs/prx/libkernel/Socket/src/Udp.cpp index 9ff5d7b33..2512c433d 100644 --- a/core/libs/prx/libkernel/Socket/src/Udp.cpp +++ b/core/libs/prx/libkernel/Socket/src/Udp.cpp @@ -915,3 +915,50 @@ std::int64_t GuestSockets::Read(int descriptor, void* buffer, std::size_t length std::int64_t GuestSockets::Write(int descriptor, const void* buffer, std::size_t length) { return send_nid_postfix(descriptor, buffer, length, 0); } + +std::int64_t GuestSockets::Readv(int descriptor, const Iovec* iov, int iovcnt) { + std::size_t total = 0; + for (int i = 0; i < iovcnt; ++i) { + if (iov[i].base == nullptr && iov[i].length != 0) return Fail(14); + if (iov[i].length > static_cast(INT_MAX) - total) return Fail(22); + total += iov[i].length; + } + if (total == 0) return IsOpen(descriptor) ? 0 : Fail(9); + std::vector buffer; + try { + buffer.resize(total); + } catch (const std::bad_alloc&) { + return Fail(12); + } + const auto result = Read(descriptor, buffer.data(), total); + if (result < 0) return result; + std::size_t copied = 0; + for (int i = 0; i < iovcnt && copied < static_cast(result); ++i) { + const auto amount = std::min(iov[i].length, static_cast(result) - copied); + if (amount != 0) std::memcpy(iov[i].base, buffer.data() + copied, amount); + copied += amount; + } + return result; +} + +std::int64_t GuestSockets::Writev(int descriptor, const Iovec* iov, int iovcnt) { + std::size_t total = 0; + for (int i = 0; i < iovcnt; ++i) { + if (iov[i].base == nullptr && iov[i].length != 0) return Fail(14); + if (iov[i].length > static_cast(INT_MAX) - total) return Fail(22); + total += iov[i].length; + } + if (total == 0) return Write(descriptor, nullptr, 0); + std::vector buffer; + try { + buffer.resize(total); + } catch (const std::bad_alloc&) { + return Fail(12); + } + std::size_t copied = 0; + for (int i = 0; i < iovcnt; ++i) { + if (iov[i].length != 0) std::memcpy(buffer.data() + copied, iov[i].base, iov[i].length); + copied += iov[i].length; + } + return Write(descriptor, buffer.data(), total); +} diff --git a/core/libs/tests/GuestSocketReadWrite.cpp b/core/libs/tests/GuestSocketReadWrite.cpp index ad6eae0f8..3d0707482 100644 --- a/core/libs/tests/GuestSocketReadWrite.cpp +++ b/core/libs/tests/GuestSocketReadWrite.cpp @@ -15,6 +15,11 @@ struct PollDescriptor { short revents; }; +struct Iovec { + void* base; + std::size_t length; +}; + extern "C" { int APS5_VABI socket_nid_postfix(int, int, int); int APS5_VABI bind_nid_postfix(int, const void*, std::uint32_t); @@ -31,6 +36,8 @@ std::int64_t APS5_VABI _read_nid_postfix(int, void*, std::size_t); std::int64_t APS5_VABI _write_nid_postfix(int, const void*, std::size_t); std::int64_t APS5_VABI sceKernelRead(int, void*, std::size_t); std::int64_t APS5_VABI sceKernelWrite(int, const void*, std::size_t); +std::int64_t APS5_VABI sceKernelReadv(int, const Iovec*, int); +std::int64_t APS5_VABI sceKernelWritev(int, const Iovec*, int); int* APS5_VABI __error_nid_postfix(); } @@ -86,6 +93,28 @@ int main() { Require(Readable(server) && sceKernelRead(server, received, sizeof(received)) == sizeof(kernel)); Require(std::strcmp(received, kernel) == 0); + const char vectorFirst[] = "vector "; + const char vectorSecond[] = "write"; + Iovec vectorWrite[] = {{const_cast(vectorFirst), sizeof(vectorFirst) - 1}, + {const_cast(vectorSecond), sizeof(vectorSecond)}}; + Require(sceKernelWritev(client, vectorWrite, 2) == sizeof(vectorFirst) + sizeof(vectorSecond) - 1); + char vectorReadFirst[sizeof(vectorFirst) - 1]{}; + char vectorReadSecond[sizeof(vectorSecond)]{}; + Iovec vectorRead[] = {{vectorReadFirst, sizeof(vectorReadFirst)}, {vectorReadSecond, sizeof(vectorReadSecond)}}; + Require(Readable(server) && sceKernelReadv(server, vectorRead, 2) == sizeof(vectorFirst) + sizeof(vectorSecond) - 1); + Require(std::memcmp(vectorReadFirst, "vector ", sizeof(vectorReadFirst)) == 0 && std::strcmp(vectorReadSecond, "write") == 0); + + const char sceVectorFirst[] = "sceKernel"; + const char sceVectorSecond[] = "Writev"; + Iovec sceVectorWrite[] = {{const_cast(sceVectorFirst), sizeof(sceVectorFirst) - 1}, + {const_cast(sceVectorSecond), sizeof(sceVectorSecond)}}; + Require(sceKernelWritev(client, sceVectorWrite, 2) == sizeof(sceVectorFirst) + sizeof(sceVectorSecond) - 1); + char sceVectorReadFirst[sizeof(sceVectorFirst) - 1]{}; + char sceVectorReadSecond[sizeof(sceVectorSecond)]{}; + Iovec sceVectorRead[] = {{sceVectorReadFirst, sizeof(sceVectorReadFirst)}, {sceVectorReadSecond, sizeof(sceVectorReadSecond)}}; + Require(Readable(server) && sceKernelReadv(server, sceVectorRead, 2) == sizeof(sceVectorFirst) + sizeof(sceVectorSecond) - 1); + Require(std::memcmp(sceVectorReadFirst, "sceKernel", sizeof(sceVectorReadFirst)) == 0 && std::strcmp(sceVectorReadSecond, "Writev") == 0); + Require(fcntl_nid_postfix(server, 4, 4) == 0); *__error_nid_postfix() = 0; Require(read_nid_postfix(server, received, sizeof(received)) == -1 && *__error_nid_postfix() == 35); diff --git a/core/libs/tests/GuestUdp.cpp b/core/libs/tests/GuestUdp.cpp index ac9b76ea1..7628ae588 100644 --- a/core/libs/tests/GuestUdp.cpp +++ b/core/libs/tests/GuestUdp.cpp @@ -7,6 +7,7 @@ #include extern "C" { int APS5_VABI socket_nid_postfix(int, int, int); +int APS5_VABI connect_nid_postfix(int, const void*, std::uint32_t); int APS5_VABI fcntl_nid_postfix(int, int, ...); int APS5_VABI setsockopt_nid_postfix(int, int, int, const void*, std::uint32_t); int APS5_VABI getsockopt_nid_postfix(int, int, int, void*, std::uint32_t*); @@ -36,6 +37,8 @@ struct Msghdr { extern "C" { std::int64_t APS5_VABI sendmsg_nid_postfix(int, const Msghdr*, int); std::int64_t APS5_VABI recvmsg_nid_postfix(int, Msghdr*, int); +std::int64_t APS5_VABI sceKernelReadv(int, const Iovec*, int); +std::int64_t APS5_VABI sceKernelWritev(int, const Iovec*, int); } static void Require(bool value) { if (!value) std::abort(); } int main() { @@ -47,6 +50,16 @@ int main() { std::uint32_t size = destination.size(); Require(getsockname_nid_postfix(receiver, destination.data(), &size) == 0 && size == 16); Require(destination[2] || destination[3]); + Require(connect_nid_postfix(sender, destination.data(), destination.size()) == 0); + char vectorHead[] = "atomic "; + char vectorTail[] = "datagram"; + Iovec vectorOutput[]{{vectorHead, sizeof(vectorHead) - 1}, {vectorTail, sizeof(vectorTail)}}; + Require(sceKernelWritev(sender, vectorOutput, 2) == sizeof(vectorHead) + sizeof(vectorTail) - 1); + char vectorFirst[sizeof(vectorHead) - 1]{}; + char vectorSecond[sizeof(vectorTail)]{}; + Iovec vectorInput[]{{vectorFirst, sizeof(vectorFirst)}, {vectorSecond, sizeof(vectorSecond)}}; + Require(sceKernelReadv(receiver, vectorInput, 2) == sizeof(vectorHead) + sizeof(vectorTail) - 1); + Require(std::strcmp(vectorFirst, "atomic ") == 0 && std::strcmp(vectorSecond, "datagram") == 0); int enabled = 1; Require(setsockopt_nid_postfix(sender, 0xffff, 0x20, &enabled, sizeof(enabled)) == 0); int option = 0; diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 74a13e50a..688ff214d 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -483,7 +483,7 @@ Throughout the project, every function at every stage either **does exactly what - [collate::_Getcat](../../core/libs/prx/libc/src/LocaleSupport.cpp) (libc) builds the facet for the C locale only and throws for any other locale name. The console's `libc.prx` stores pointers to its collation state tables at offsets 0x10 and 0x18 of the facet; ours leaves them null, as only the facet's own functions read them. `do_transform` throws for input with embedded null characters, which the console drops depending on the output buffer size, and for results of 4095 characters or more, whose string storage uses the guest's over-aligned large allocation. - [The scanf family](../../core/libs/prx/libc/src/Formatting.cpp) (libc) hands `%i` to glibc on Linux, which since 2.38 reads the C23 `0b`/`0B` binary prefix when the library is built as GNU C++: `sscanf("0b101", "%i", &value)` stores 5 where C17 stores 0 and stops at the `b`. The strtol family stops at the prefix as C17 does - [socketpair](../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) - the pair is the host's `AF_UNIX` pair on Linux and, on Windows, a loopback TCP connection for `SOCK_STREAM` or two connected UDP sockets for `SOCK_DGRAM`; `SOCK_SEQPACKET` throws on Windows. A datagram sent to a full receiver gets `ENOBUFS` on the console, waits or gets `EAGAIN` on Linux and is dropped on Windows, and on Windows a datagram `recv` after the peer closed reports `ECONNRESET` instead of the console's `ENOTCONN`, and a stream `send` after the peer closed reports `ECONNABORTED` instead of the console's `EPIPE`. `read` and `write` on a pair end go to the host like on every guest socket. `getpeername` on a pair end succeeds after the peer closed, where the console returns `ENOTCONN`, and `connect`, `bind` and `sendto` with an address on a pair end report an address-family error instead of the console's `EISCONN`. On Windows `poll` on a stream end reports `POLLHUP` without `POLLIN` after the peer's `shutdown(SHUT_WR)` or `close`, where the console reports `POLLIN`, with `POLLHUP` only after `close` -- [read and write](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) on a guest socket are `recv` and `send` without flags, and `sceKernelRead` and `sceKernelWrite` return the kernel error for them; `EPIPE` raises the host's `SIGPIPE` on Linux, which the guest can neither ignore (`signal` maps only six signals) nor suppress (`SO_NOSIGPIPE` is not an option here), and raises nothing on Windows; a `write` longer than `INT_MAX` bytes is `EMSGSIZE` where the console sends it; `readv`, `writev`, `pread`, `pwrite`, their `sceKernel` forms and AIO on a guest socket still go to the host, which has no such descriptor, where the console reads and writes through `readv` and `writev` and returns `ESPIPE` for the positioned forms +- [read, write and vector I/O](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) on a guest socket use `recv` and `send` without flags; `sceKernelReadv` and `sceKernelWritev` gather or scatter through one socket operation, preserving stream short reads and datagram boundaries. The positioned `pread`, `pwrite`, `preadv` and `pwritev` forms still go to the host, which has no such descriptor; the console returns `ESPIPE` for these calls. `EPIPE` from a write raises the host's `SIGPIPE` on Linux, which the guest can neither ignore (`signal` maps only six signals) nor suppress (`SO_NOSIGPIPE` is not an option here), and raises nothing on Windows; a `write` longer than `INT_MAX` bytes is `EMSGSIZE` where the console sends it. AIO on a guest socket still goes to the host, which has no such descriptor - [poll](../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) waits only on guest sockets, through `WSAPoll` on Windows and `poll` on Linux; the `pollfd` layout and flag values are FreeBSD's. A descriptor below the socket range (files, pipes, stdio) throws, as do event bits other than `POLLIN`, `POLLPRI`, `POLLOUT`, `POLLRDNORM`, `POLLRDBAND` and `POLLWRBAND` (the `POLLERR`, `POLLHUP` and `POLLNVAL` bits are ignored in `events`), `POLLPRI` on Windows, which `WSAPoll` rejects, and an infinite timeout with no socket to wait on. A closed socket descriptor reports `POLLNVAL`; the console's limit on the descriptor count is unknown and not checked - [msync](../../core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp) (libkernel) follows FreeBSD 11's `kern_msync` and `vm_map_sync`, not measured on the console: the address is rounded down to the page, a range that wraps is `EINVAL`, as is `MS_ASYNC` with `MS_INVALIDATE`, and a range whose start or any page is not mapped (as `sceKernelVirtualQuery` reports it, reservations included) is `ENOMEM`; a page unmapped from inside a larger mapping still counts as mapped there, since `sceKernelVirtualQuery` keeps reporting the whole mapping. The guest has no file mappings, so a successful call writes nothing back. `MS_INVALIDATE` on a mapped range throws: FreeBSD returns `EBUSY` when the range holds `mlock`ed pages, and locked pages are not tracked - [Thread exit destructors](../../core/libs/prx/libSceLibcInternal/Export.cpp) (libSceLibcInternal) run when a libkernel thread finishes; on the main or a host thread only `_sceLibcInternalThreadDtors` runs them, as `exit` does, else they are dropped From eb9efe109e93bffbbf5779bad14313131a48a1b4 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sun, 11 Oct 2026 01:17:37 +0000 Subject: [PATCH 291/460] fix(libkernel): preserve vector socket error codes --- core/libs/prx/libkernel/File/src/Stdio.cpp | 6 +++--- core/libs/tests/GuestSocketReadWrite.cpp | 2 ++ 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index 61f56c803..1dda34d95 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -813,7 +813,7 @@ static std::int64_t TransferIovecs(int d, const KernelIovec* iov, int iovcnt, co if (offset == nullptr && d >= GuestSockets::FirstDescriptor) { const auto result = write ? GuestSockets::Writev(d, reinterpret_cast(iov), iovcnt) : GuestSockets::Readv(d, reinterpret_cast(iov), iovcnt); - return result < 0 ? SceErrorFromErrno(*__error_nid_postfix()) : result; + return result < 0 ? SceKernelError(*__error_nid_postfix()) : result; } if (!write && File::IsRandomDevice(d)) return ReadRandomIovecs(d, iov, iovcnt); if (total == 0) { @@ -878,7 +878,7 @@ int64_t APS5_VABI sceKernelReadv(int d, const KernelIovec* iov, int iovcnt) { if (!OpenIovecs(iov, iovcnt, destinations)) return SceErrorFromErrno(GUEST_EFAULT); if (d >= GuestSockets::FirstDescriptor) { const auto result = GuestSockets::Readv(d, reinterpret_cast(iov), iovcnt); - return result < 0 ? SceErrorFromErrno(*__error_nid_postfix()) : result; + return result < 0 ? SceKernelError(*__error_nid_postfix()) : result; } if (File::IsRandomDevice(d)) return ReadRandomIovecs(d, iov, iovcnt); const auto result = static_cast(::readv(d, NativeIovecs(iov), iovcnt)); @@ -889,7 +889,7 @@ int64_t APS5_VABI sceKernelWritev(int d, const KernelIovec* iov, int iovcnt) { if (const int error = CheckIovecs(iov, iovcnt)) return error; if (d >= GuestSockets::FirstDescriptor) { const auto result = GuestSockets::Writev(d, reinterpret_cast(iov), iovcnt); - return result < 0 ? SceErrorFromErrno(*__error_nid_postfix()) : result; + return result < 0 ? SceKernelError(*__error_nid_postfix()) : result; } const auto result = static_cast(::writev(d, NativeIovecs(iov), iovcnt)); return result < 0 ? SceErrorFromErrno(errno) : result; diff --git a/core/libs/tests/GuestSocketReadWrite.cpp b/core/libs/tests/GuestSocketReadWrite.cpp index 3d0707482..34952e5dd 100644 --- a/core/libs/tests/GuestSocketReadWrite.cpp +++ b/core/libs/tests/GuestSocketReadWrite.cpp @@ -119,6 +119,8 @@ int main() { *__error_nid_postfix() = 0; Require(read_nid_postfix(server, received, sizeof(received)) == -1 && *__error_nid_postfix() == 35); Require(sceKernelRead(server, received, sizeof(received)) == KernelAgain); + Iovec emptyRead[] = {{received, sizeof(received)}}; + Require(sceKernelReadv(server, emptyRead, 1) == KernelAgain); Require(read_nid_postfix(server, received, 0) == 0 && sceKernelRead(server, received, 0) == 0); Require(close_nid_postfix(client) == 0); From 4fd6192d5c7f376b7350c3e4605dad49b365d0df Mon Sep 17 00:00:00 2001 From: Scramblrrrr Date: Sat, 10 Oct 2026 19:03:32 -0300 Subject: [PATCH 292/460] fix(agc): enforce retained-byte budget before buffer-only work --- .../Execution/src/VulkanDevice.cpp | 2 ++ .../prx/libSceAgcDriver/Graphics/src/Draw.cpp | 2 ++ .../tests/execution/RecordedKeptBytes.cpp | 20 +++++++++++++++++++ 3 files changed, 24 insertions(+) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index 77913ce42..91fcc5c40 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -3596,6 +3596,7 @@ void VulkanDevice::recordDispatch(RecordedDispatch& record) { } VulkanDevice::IndirectOutcome VulkanDevice::dispatch(const ShaderRecompiler::RecompileResult& shader, std::uint32_t x, std::uint32_t y, std::uint32_t z, std::uint64_t arguments, std::span snapshots, std::uint64_t programAddress, std::shared_ptr prepared, std::shared_ptr* recipeOut) { + state->recorder->BoundKeptBytes(); ShaderRecompiler::RuntimeAbi::RequireVersion(shader.runtimeAbiVersion); PerformanceTimer timing("Vulkan.Dispatch"); if (recipeOut != nullptr) *recipeOut = nullptr; @@ -3890,6 +3891,7 @@ VulkanDevice::IndirectOutcome VulkanDevice::dispatch(const ShaderRecompiler::Rec } RecipeOutcome VulkanDevice::DispatchRecipe(const ShaderRecompiler::RecompileResult& shader, std::uint32_t x, std::uint32_t y, std::uint32_t z, std::uint64_t arguments, std::uint64_t programAddress, const std::shared_ptr& hit, IndirectOutcome& outcome, const std::shared_ptr& verify, bool refreshByWords) { + state->recorder->BoundKeptBytes(); ShaderRecompiler::RuntimeAbi::RequireVersion(shader.runtimeAbiVersion); PerformanceTimer timing("Vulkan.DispatchRecipe"); outcome = {0, 0}; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp index a36593d10..1824f74ec 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp @@ -1807,6 +1807,7 @@ std::optional KnownValidationFailure(const Context& context, std::s } void Draw(const Context& context, const State& state, const Pm4::DrawParameters& draw, std::span shaders, std::span snapshots, std::shared_ptr* recipeOut) { + if (context.recorder != nullptr) context.recorder->BoundKeptBytes(); PerformanceTimer timing("Graphics.Draw"); // APS5_PROFILE_DRAW prints the time of each phase of the draw (microseconds) and the [draws] totals. static const bool profile = std::getenv("APS5_PROFILE_DRAW") != nullptr; @@ -2352,6 +2353,7 @@ void Draw(const Context& context, const State& state, const Pm4::DrawParameters& } DrawRecipeOutcome DrawWithRecipe(const Context& context, const State& state, const Pm4::DrawParameters& draw, std::span shaders, std::span snapshots, const DrawRecipe& recipe) { + if (context.recorder != nullptr) context.recorder->BoundKeptBytes(); static_cast(snapshots); PerformanceTimer timing("Graphics.DrawWithRecipe"); static const bool profile = std::getenv("APS5_PROFILE_DRAW") != nullptr; diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp index 11b98c82d..a41fe4456 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/RecordedKeptBytes.cpp @@ -24,6 +24,8 @@ #include #include #include +#include +#include #include #include #include @@ -156,6 +158,7 @@ void Dispatch(AgcDriver::VulkanDevice& device, std::uint8_t* target) { }; request.useCache = false; const auto result = ShaderRecompiler::Recompile(request); + std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex()); device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); } @@ -236,6 +239,22 @@ void CopiedDispatch(AgcDriver::VulkanDevice& device) { device.WaitIdle(); } +void BufferOnlyDispatchBounds(AgcDriver::VulkanDevice& device) { + GuestBuffer first(false); + GuestBuffer second(false); + device.WaitIdle(); + Dispatch(device, first.Data()); + auto& recorder = ActiveRecorder(); + { + std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex()); + recorder.Keep(std::make_shared(0), Recorder::KeptBytesBudget); + } + const auto submitted = recorder.Submissions(); + Dispatch(device, second.Data()); + Require(recorder.Submissions() > submitted, "a buffer-only dispatch did not submit the preceding batch at its kept-byte budget"); + device.WaitIdle(); +} + void ResidentTarget(AgcDriver::VulkanDevice& device, GuestBuffer& pixels) { GuestBuffer vertices(false); FillVertices(vertices); @@ -284,6 +303,7 @@ int main() { const auto device = OpenVulkanTestDevice(); if (!device) return VulkanTestSkipped; CopiedDispatch(*device); + BufferOnlyDispatchBounds(*device); GuestBuffer pixels(true, TargetLayout.Bytes()); ResidentTarget(*device, pixels); CopiedDraw(*device, pixels); From ab0babdc9bb83061cffb2f1bd49b89eaf4066f0e Mon Sep 17 00:00:00 2001 From: fma-rc <188743521+fma-rc@users.noreply.github.com> Date: Sun, 11 Oct 2026 03:31:37 +0200 Subject: [PATCH 293/460] fix(libs): bind the system libraries eagerly on Linux A title calls our exports on stacks it sized for the console. With lazy binding the first call through a PLT slot runs the dynamic linker's symbol lookup on that stack: on the 2 KiB fiber context Astro's Playroom uses it wrote below the context into the neighbouring heap chunks. --- core/libs/CMakeLists.txt | 6 ++++++ core/libs/tests/Fiber.cpp | 25 +++++++++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/core/libs/CMakeLists.txt b/core/libs/CMakeLists.txt index cb240d7dd..6abb0fdd0 100644 --- a/core/libs/CMakeLists.txt +++ b/core/libs/CMakeLists.txt @@ -53,6 +53,9 @@ set(LIBS_INCLUDE_DIR ${CMAKE_CURRENT_SOURCE_DIR}) add_subdirectory(${LIBC_DIR} ${CMAKE_CURRENT_BINARY_DIR}/prx/libc) target_include_directories(libc PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}) configure_windows_unwind(libc) +if(CMAKE_SYSTEM_NAME STREQUAL "Linux") + target_link_options(libc PRIVATE LINKER:-z,now) +endif() set(UNPATCHED_DIR ${CMAKE_CURRENT_BINARY_DIR}/libs/unpatched) set(PATCHED_DIR ${CMAKE_CURRENT_BINARY_DIR}/libs) @@ -102,6 +105,9 @@ foreach(LIB_DIR ${LIBS_DIRS}) else() target_link_libraries(${LIB_NAME} PRIVATE libkernel libc) endif() + if(CMAKE_SYSTEM_NAME STREQUAL "Linux") + target_link_options(${LIB_NAME} PRIVATE LINKER:-z,now) + endif() if(WIN32 AND LIB_NAME STREQUAL "libSceNet") target_link_libraries(${LIB_NAME} PRIVATE ws2_32) target_compile_definitions(${LIB_NAME} PRIVATE _WIN32_WINNT=0x0602) diff --git a/core/libs/tests/Fiber.cpp b/core/libs/tests/Fiber.cpp index 707a59a80..da0721409 100644 --- a/core/libs/tests/Fiber.cpp +++ b/core/libs/tests/Fiber.cpp @@ -81,6 +81,30 @@ void APS5_VABI CheckedEntry(std::uint64_t, std::uint64_t) { for (;;) sceFiberReturnToThread(0, &received); } +constexpr std::size_t SmallContextSize = 2048; +constexpr unsigned char SmallGuardFill = 0xa5; +alignas(16) std::array g_smallArea; +FiberStorage g_smallStorage; + +void APS5_VABI SmallEntry(std::uint64_t, std::uint64_t) { + std::uint64_t received = 0; + for (;;) sceFiberReturnToThread(0, &received); +} + +void RunSmallContextFirst() { + auto* small = reinterpret_cast(&g_smallStorage); + auto* context = g_smallArea.data() + g_smallArea.size() - SmallContextSize; + g_smallArea.fill(SmallGuardFill); + Require(sceFiberReturnToThread(0, nullptr) == FiberErrorPermission, "return to thread outside a fiber"); + Require(_sceFiberInitializeImpl_nid_postfix(small, "small", SmallEntry, 0, context, SmallContextSize, nullptr, 0) == 0, "initialize small"); + Require(sceFiberRun_nid_postfix(small, 0, nullptr) == 0, "first fiber run of the process on a 2 KiB context"); + Require(sceFiberRun_nid_postfix(small, 0, nullptr) == 0, "resume on a 2 KiB context"); + Require(sceFiberFinalize(small) == 0, "finalize small"); + for (auto* byte = g_smallArea.data(); byte != context; ++byte) { + Require(*byte == SmallGuardFill, "nothing is written below a 2 KiB context"); + } +} + [[gnu::noinline]] std::int32_t RunFirst(std::uint64_t* returned) { g_threadFramePointer = reinterpret_cast(__builtin_frame_address(0)); const std::int32_t result = sceFiberRun_nid_postfix(g_first, 100, returned); @@ -91,6 +115,7 @@ void APS5_VABI CheckedEntry(std::uint64_t, std::uint64_t) { } int main() { + RunSmallContextFirst(); FiberObject* self = nullptr; Require(sceFiberGetSelf(&self) == FiberErrorPermission, "no fiber on the thread"); Require(_sceFiberInitializeImpl_nid_postfix(g_first, "first", FirstEntry, 11, g_firstContext.data(), g_firstContext.size(), nullptr, 0) == 0, "initialize first"); From aa013f59ac8b8a600635aebbcfde8c97512c2ae8 Mon Sep 17 00:00:00 2001 From: raghav314 Date: Sun, 11 Oct 2026 06:52:21 +0530 Subject: [PATCH 294/460] fix(libc): register guest heap blocks without GPU access AnyPS5's libc heap registered every block with GPU access, so every address-based build imported or mirrored each malloc block the title had made. The console's libc.prx maps its heap with protection 3 (CPU read and write, no GPU bits), so the GPU never sees it there either. Heap blocks are now registered without GPU access; reallocating keeps the block's access. The AvPlayer fallback for a title that passes no texture allocator stands in for that allocator, which hands out memory the GPU reads, so it allocates through a new GuestHeapAlignForGpu. Co-Authored-By: Claude Opus 5.5 --- .../libSceAgcDriver/tests/BdaResources.cpp | 29 ++++++++++++++++++- .../libs/prx/libSceAgcDriver/tests/Submit.cpp | 4 +-- core/libs/prx/libSceAvPlayer/src/Player.cpp | 6 +++- core/libs/prx/libc/include/GuestHeap.hpp | 1 + core/libs/prx/libc/src/GuestHeap.cpp | 17 +++++++---- 5 files changed, 47 insertions(+), 10 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp index fa8240bd6..3174ec625 100644 --- a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp @@ -741,11 +741,38 @@ void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { tail.Upload(true); reject([&] { tail.Descriptor(address + 58, 4, adjustment); }, "exceeds its GPU owner"); } + { + void* plain = GuestHeap::GuestHeapAllocate_nid_postfix(64); + void* gpu = GuestHeap::GuestHeapAlignForGpu_nid_postfix(16, 64); + const auto lookup = [](const void* pointer) { + GuestAllocations::Mutation mutation; + return mutation.Find(pointer); + }; + Require(!lookup(plain).gpu && lookup(gpu).gpu, "a guest heap block's GPU access does not follow how it was allocated"); + gpu = GuestHeap::GuestHeapReallocate_nid_postfix(gpu, 128); + plain = GuestHeap::GuestHeapRealign_nid_postfix(plain, 128, 64); + Require(!lookup(plain).gpu && lookup(gpu).gpu, "a reallocated guest heap block changed its GPU access"); + { + GuestBufferMemory leased(context); + leased.AcquireRegistered(); + leased.Upload(true); + const auto ranges = leased.AddressRanges(); + const auto listed = [&](const void* pointer) { + const auto address = reinterpret_cast(pointer); + return std::any_of(ranges.begin(), ranges.end(), [&](const auto& range) { return range.begin <= address && address < range.end; }); + }; + Require(!listed(plain), "a guest heap block without GPU access is in an address-based build"); + Require(listed(gpu), "a guest heap block with GPU access is missing from an address-based build"); + leased.WriteBack(); + } + GuestHeap::GuestHeapFree_nid_postfix(plain); + GuestHeap::GuestHeapFree_nid_postfix(gpu); + } { // The cached address space: a second build in an unchanged registry takes the first one's // space, and a guest free of a range pinned only by the cache goes through the pin waiter's // drop (no GPU work to wait for) and empties the registry of it. - void* block = GuestHeap::GuestHeapAllocate_nid_postfix(64); + void* block = GuestHeap::GuestHeapAlignForGpu_nid_postfix(16, 64); const auto blockAddress = reinterpret_cast(block); std::memset(block, 0x5a, 64); const auto registered = [&] { diff --git a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp index cd6f89778..1bbce7d61 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp @@ -450,8 +450,8 @@ std::array memoryCopy(const void* destination, const void* sou void testCopyIntoHostMemory() { constexpr std::size_t words = 32768; constexpr auto bytes = static_cast(words * 4); - auto* produced = static_cast(GuestHeap::GuestHeapAlign_nid_postfix(65536, bytes)); - auto* source = static_cast(GuestHeap::GuestHeapAlign_nid_postfix(65536, bytes)); + auto* produced = static_cast(GuestHeap::GuestHeapAlignForGpu_nid_postfix(65536, bytes)); + auto* source = static_cast(GuestHeap::GuestHeapAlignForGpu_nid_postfix(65536, bytes)); std::vector destination(words); for (const bool label : {false, true}) { for (std::size_t i = 0; i < words; ++i) { diff --git a/core/libs/prx/libSceAvPlayer/src/Player.cpp b/core/libs/prx/libSceAvPlayer/src/Player.cpp index bdf69bfc3..ab1aebb40 100644 --- a/core/libs/prx/libSceAvPlayer/src/Player.cpp +++ b/core/libs/prx/libSceAvPlayer/src/Player.cpp @@ -29,13 +29,17 @@ void APS5_VABI GuestDeallocate_nid_no_patch(void*, void* memory) { GuestHeap::GuestHeapFree_nid_postfix(memory); } +void* APS5_VABI GuestAllocateTexture_nid_no_patch(void*, std::uint32_t alignment, std::uint32_t size) { + return GuestHeap::GuestHeapAlignForGpu_nid_postfix(std::max(alignment, 16), size); +} + AvPlayerMemAllocator WithGuestHeapFallback(AvPlayerMemAllocator memory) { if (!memory.allocate || !memory.deallocate) { memory.allocate = GuestAllocate_nid_no_patch; memory.deallocate = GuestDeallocate_nid_no_patch; } if (!memory.allocate_texture || !memory.deallocate_texture) { - memory.allocate_texture = GuestAllocate_nid_no_patch; + memory.allocate_texture = GuestAllocateTexture_nid_no_patch; memory.deallocate_texture = GuestDeallocate_nid_no_patch; } return memory; diff --git a/core/libs/prx/libc/include/GuestHeap.hpp b/core/libs/prx/libc/include/GuestHeap.hpp index 65f9c0d42..f87205fee 100644 --- a/core/libs/prx/libc/include/GuestHeap.hpp +++ b/core/libs/prx/libc/include/GuestHeap.hpp @@ -12,6 +12,7 @@ void GuestHeapFree_nid_postfix(void* pointer); void* GuestHeapReallocate_nid_postfix(void* pointer, std::size_t bytes); void* GuestHeapAlign_nid_postfix(std::size_t alignment, std::size_t bytes); void* GuestHeapRealign_nid_postfix(void* pointer, std::size_t bytes, std::size_t alignment); +void* GuestHeapAlignForGpu_nid_postfix(std::size_t alignment, std::size_t bytes); } diff --git a/core/libs/prx/libc/src/GuestHeap.cpp b/core/libs/prx/libc/src/GuestHeap.cpp index 24442855c..378830dcb 100644 --- a/core/libs/prx/libc/src/GuestHeap.cpp +++ b/core/libs/prx/libc/src/GuestHeap.cpp @@ -193,7 +193,7 @@ void rawFree(void* raw, std::size_t blockBytes) { else std::free(raw); } -void* allocate(GuestAllocations::Mutation& mutation, std::size_t alignment, std::size_t bytes) { +void* allocate(GuestAllocations::Mutation& mutation, std::size_t alignment, std::size_t bytes, bool gpu) { if (alignment == 0 || (alignment & (alignment - 1)) != 0) throw std::invalid_argument("invalid guest heap alignment"); alignment = std::max(alignment, MinimumAlignment); const std::size_t padding = alignment > RawAlignment ? alignment : 0; @@ -205,7 +205,7 @@ void* allocate(GuestAllocations::Mutation& mutation, std::size_t alignment, std: reinterpret_cast(pointer)[-2] = raw; reinterpret_cast(pointer)[-1] = blockBytes; try { - mutation.Add(pointer, bytes, true, true, true); + mutation.Add(pointer, bytes, true, true, gpu); } catch (...) { rawFree(raw, blockBytes); throw; @@ -222,7 +222,7 @@ void free(GuestAllocations::Mutation& mutation, void* pointer) { void* GuestHeapAllocate_nid_postfix(std::size_t bytes) { GuestAllocations::Mutation mutation; - return allocate(mutation, MinimumAlignment, bytes); + return allocate(mutation, MinimumAlignment, bytes, false); } void GuestHeapFree_nid_postfix(void* pointer) { @@ -242,7 +242,7 @@ void* GuestHeapReallocate_nid_postfix(void* pointer, std::size_t bytes) { free(mutation, pointer); return nullptr; } - void* result = allocate(mutation, MinimumAlignment, bytes); + void* result = allocate(mutation, MinimumAlignment, bytes, range.gpu); std::memcpy(result, pointer, std::min(bytes, range.bytes)); free(mutation, pointer); return result; @@ -257,7 +257,7 @@ void* GuestHeapRealign_nid_postfix(void* pointer, std::size_t bytes, std::size_t free(mutation, pointer); return nullptr; } - void* result = allocate(mutation, alignment, bytes); + void* result = allocate(mutation, alignment, bytes, range.gpu); std::memcpy(result, pointer, std::min(bytes, range.bytes)); free(mutation, pointer); return result; @@ -265,7 +265,12 @@ void* GuestHeapRealign_nid_postfix(void* pointer, std::size_t bytes, std::size_t void* GuestHeapAlign_nid_postfix(std::size_t alignment, std::size_t bytes) { GuestAllocations::Mutation mutation; - return allocate(mutation, alignment, bytes); + return allocate(mutation, alignment, bytes, false); +} + +void* GuestHeapAlignForGpu_nid_postfix(std::size_t alignment, std::size_t bytes) { + GuestAllocations::Mutation mutation; + return allocate(mutation, alignment, bytes, true); } } From 967de4039395f3c26840128799e6f9188176c07a Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Sat, 10 Oct 2026 19:33:29 -0400 Subject: [PATCH 295/460] feat(libkernel_sys): export fork and implement setsid and umask --- core/libs/prx/libkernel_sys/Export.cpp | 17 +++++++++++++++++ core/libs/tests/GuestKernelSys.cpp | 6 ++++++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 24 insertions(+) diff --git a/core/libs/prx/libkernel_sys/Export.cpp b/core/libs/prx/libkernel_sys/Export.cpp index 923ebaa27..82a7cee18 100644 --- a/core/libs/prx/libkernel_sys/Export.cpp +++ b/core/libs/prx/libkernel_sys/Export.cpp @@ -1,3 +1,4 @@ +#include #include #include #include @@ -13,10 +14,13 @@ extern "C" int* APS5_VABI __error_nid_postfix(); namespace { constexpr int GuestEnoent = 2; +constexpr int GuestEperm = 1; constexpr int GuestEio = 5; constexpr int GuestEfault = 14; constexpr int GuestEinval = 22; +std::atomic creationMask{022}; + std::int64_t Fail(int error) { *__error_nid_postfix() = error; return -1; @@ -30,6 +34,19 @@ int GuestError(const std::error_code& error) { extern "C" { +int APS5_VABI fork_nid_postfix(void) { + NotImplemented_nid_no_patch("fork: child processes"); + return -1; +} + +int APS5_VABI setsid_nid_postfix(void) { + return static_cast(Fail(GuestEperm)); +} + +std::uint16_t APS5_VABI umask_nid_postfix(std::uint16_t mask) { + return creationMask.exchange(static_cast(mask & 0777)); +} + std::int64_t APS5_VABI readlink_nid_postfix(const char* path, char* buffer, std::size_t size) { if (path == nullptr) return Fail(GuestEfault); if (*path == '\0') return Fail(GuestEnoent); diff --git a/core/libs/tests/GuestKernelSys.cpp b/core/libs/tests/GuestKernelSys.cpp index 6b523f471..5e340f500 100644 --- a/core/libs/tests/GuestKernelSys.cpp +++ b/core/libs/tests/GuestKernelSys.cpp @@ -11,6 +11,8 @@ extern "C" { std::int64_t APS5_VABI readlink_nid_postfix(const char*, char*, std::size_t); +int APS5_VABI setsid_nid_postfix(void); +std::uint16_t APS5_VABI umask_nid_postfix(std::uint16_t); int* APS5_VABI __error_nid_postfix(); } @@ -26,6 +28,10 @@ int Error() { } int main() { + Require(setsid_nid_postfix() == -1 && Error() == 1); + Require(umask_nid_postfix(0) == 022); + Require(umask_nid_postfix(07777) == 0); + Require(umask_nid_postfix(022) == 0777); const auto root = std::filesystem::path("anyps5-kernel-sys-test-" + std::to_string(std::chrono::steady_clock::now().time_since_epoch().count())); Require(std::filesystem::create_directory(root)); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 237d82bed..d021ab4f7 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -358,6 +358,7 @@ Throughout the project, every function at every stage either **does exactly what - [Packed horizontal gathers](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_gather4h_pck`, MIMG opcode 98): the four window texels are read raw and concatenated into a little-endian bitstream, with DMASK bit i writing stream word i to the next destination register; measured on RDNA2 (gfx1031) at every edge: the anchor is floor(x-0.5) with elements at -1, 0, +1 and +2 clamped horizontally into the width, the row is floor(y), negative rows clamp to row 0, rows at or past the height return zeros, sampler addressing modes do not affect the window, and formats wider than 32 bits per texel write zeros in every register. The emitter fetches each window texel at LOD 0 and packs its bits like `image_load_pck`, so the sampler's filter, addressing and LOD state are not used; formats whose bits are not recoverable from the view and D16 data throw. The anchor and row multiply the normalized coordinate by the image size without contraction. The stream rule was measured for R8_UNORM, RG8_UNORM and RGBA8; only RGBA8 has an execution test. `image_gather8h_pck` (opcode 99) never returns texel data on this hardware - across dmasks 0x1 to 0xFF including the upper nibble, RGBA8, RGBA16 and RGBA32 formats, 1D, 2D and 3D descriptors and normalized and unnormalized sampling it only zero-fills popcount(dmask) registers (sentinel-prefilled destinations) - so it decodes but throws at translation; its eight-texel semantics remain unmeasured. - [waitpid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) returns -1 with `ECHILD` (`EINVAL` for unknown options): nothing here creates a child process; child processes would need a real `waitpid`. - [execvp](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) throws: replacing the process image is not supported. PPSA85300 imports it and fails to load without it; its `fork` and `vfork` fail with `ENOSYS` ([libc_missing.c](https://github.com/ZiZc3/VLC/blob/f3253020a64368153233ede29183014c0b6c8979/ps5/compat/libc_missing.c#L290-L292)), so the child process that would call it is never created. +- [fork](../../core/libs/prx/libkernel_sys/Export.cpp) (libkernel_sys) throws: creating a child process is not supported. PPSA19111 imports it and fails to load without it; it calls `fork` and then `setsid` from one routine that does not run during start-up. [setsid](../../core/libs/prx/libkernel_sys/Export.cpp) fails with `EPERM`, as FreeBSD's does for a process group leader, since the guest is taken to lead its own process group (see `kill`). [umask](../../core/libs/prx/libkernel_sys/Export.cpp) keeps the guest's mask (initially 022) and returns the previous one, but files are still created with the host process's mask - [MIMG BY2/BY4 loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_load[_mip]_by2/by4`, opcodes 0x42, 0x43, 0x4a and 0x4b) were measured on RDNA2 (gfx1035) with the [hardware oracle](HW_ORACLE.md) for 2D images with identity swizzle. They read N texels (BY2: 2, BY4: 4) from `x & ~(N - 1)` on row y, texel by texel and component by component within each texel, with each texel converted like `image_load`. Every register is zero when the mip level is past the last level, `x + N - 1` is outside the level's width (unaligned x, unsigned), or y is outside its height. Data only comes back when the N texels fit in one dword: R8 and R16 (DMASK 0x3) or RG8 (0xf) for BY2, and R8 (0xf) for BY4. With 32-bit texels (R32, RG32, RG16, RGBA8) the hardware returns zeros. Other formats, masks, dimensions, swizzles, D16/A16 and indirect descriptors throw. The stores (0x52, 0x53, 0x5a and 0x5b) decode but throw at translation. Measured on hardware, they write from the unaligned x up to the end of its dword, which is not implemented. - [2D stores to 1D storage images](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp): an `image_store` with DIM 2D through a 1D T# writes texel x and ignores y, as measured on RDNA2 (gfx1035) for y from 0 to 4 and 0xffffffff, single- and four-component formats and a 1x1 8_8_8_8_UNORM T#: the T# type decides the addressing, DIM only how many address VGPRs are read. Storage loads, atomics and `image_store_mip` through such a pair still throw - [MIMG PCK2/PCK4 loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_load[_mip]_pck2/pck4`, opcodes 0x70, 0x71, 0x73 and 0x74) were measured on RDNA2 (gfx1035) with the [hardware oracle](HW_ORACLE.md) for 2D images with identity swizzle. With data mask 0x1, they write one dword: the raw bits of the N texels (PCK2: 2, PCK4: 4) from `x & ~(N - 1)` on row y, concatenated (element k at bit `k * texel bits`) and zero-extended. The dword is zero when the mip level is past the last level, `x + N - 1` is outside the level's width (unaligned x, unsigned), or y is outside its height. Data only comes back when N texels fit in 32 bits: R8, R16 or RG8 for PCK2, R8 for PCK4. Wider texels (R32, RG16, RGBA8) and masks other than 0x1 write zeros on hardware. Those masks throw at decode, and formats outside UNORM/UINT/SINT R8, R16 and RG8 throw. #1314's earlier measurement saw only zeros because it used masks 0x3 and 0xf. The stores (0x76, 0x77, 0x79 and 0x7a) decode but throw at translation. They are not measured, and opcode 0x76 on 32_32_32_32_FLOAT hung the GPU (#1314). From 9c38ea16cf313dbff3e55e2aba1690867f456ad0 Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Sat, 10 Oct 2026 20:31:26 -0400 Subject: [PATCH 296/460] docs: say where XashPS5 calls fork --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index d021ab4f7..572036bf9 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -358,7 +358,7 @@ Throughout the project, every function at every stage either **does exactly what - [Packed horizontal gathers](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_gather4h_pck`, MIMG opcode 98): the four window texels are read raw and concatenated into a little-endian bitstream, with DMASK bit i writing stream word i to the next destination register; measured on RDNA2 (gfx1031) at every edge: the anchor is floor(x-0.5) with elements at -1, 0, +1 and +2 clamped horizontally into the width, the row is floor(y), negative rows clamp to row 0, rows at or past the height return zeros, sampler addressing modes do not affect the window, and formats wider than 32 bits per texel write zeros in every register. The emitter fetches each window texel at LOD 0 and packs its bits like `image_load_pck`, so the sampler's filter, addressing and LOD state are not used; formats whose bits are not recoverable from the view and D16 data throw. The anchor and row multiply the normalized coordinate by the image size without contraction. The stream rule was measured for R8_UNORM, RG8_UNORM and RGBA8; only RGBA8 has an execution test. `image_gather8h_pck` (opcode 99) never returns texel data on this hardware - across dmasks 0x1 to 0xFF including the upper nibble, RGBA8, RGBA16 and RGBA32 formats, 1D, 2D and 3D descriptors and normalized and unnormalized sampling it only zero-fills popcount(dmask) registers (sentinel-prefilled destinations) - so it decodes but throws at translation; its eight-texel semantics remain unmeasured. - [waitpid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) returns -1 with `ECHILD` (`EINVAL` for unknown options): nothing here creates a child process; child processes would need a real `waitpid`. - [execvp](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) throws: replacing the process image is not supported. PPSA85300 imports it and fails to load without it; its `fork` and `vfork` fail with `ENOSYS` ([libc_missing.c](https://github.com/ZiZc3/VLC/blob/f3253020a64368153233ede29183014c0b6c8979/ps5/compat/libc_missing.c#L290-L292)), so the child process that would call it is never created. -- [fork](../../core/libs/prx/libkernel_sys/Export.cpp) (libkernel_sys) throws: creating a child process is not supported. PPSA19111 imports it and fails to load without it; it calls `fork` and then `setsid` from one routine that does not run during start-up. [setsid](../../core/libs/prx/libkernel_sys/Export.cpp) fails with `EPERM`, as FreeBSD's does for a process group leader, since the guest is taken to lead its own process group (see `kill`). [umask](../../core/libs/prx/libkernel_sys/Export.cpp) keeps the guest's mask (initially 022) and returns the previous one, but files are still created with the host process's mask +- [fork](../../core/libs/prx/libkernel_sys/Export.cpp) (libkernel_sys) throws: creating a child process is not supported. PPSA19111 imports it and fails to load without it. Its engine calls `fork` in `Posix_Daemonize`, only with `-daemonize`, and in `Platform_ShellExecute` before `execvp("xdg-open")`, which the main menu's Previews button reaches, so pressing Previews ends in this throw. [setsid](../../core/libs/prx/libkernel_sys/Export.cpp) fails with `EPERM`, as FreeBSD's does for a process group leader, since the guest is taken to lead its own process group (see `kill`). [umask](../../core/libs/prx/libkernel_sys/Export.cpp) keeps the guest's mask (initially 022) and returns the previous one, but files are still created with the host process's mask - [MIMG BY2/BY4 loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_load[_mip]_by2/by4`, opcodes 0x42, 0x43, 0x4a and 0x4b) were measured on RDNA2 (gfx1035) with the [hardware oracle](HW_ORACLE.md) for 2D images with identity swizzle. They read N texels (BY2: 2, BY4: 4) from `x & ~(N - 1)` on row y, texel by texel and component by component within each texel, with each texel converted like `image_load`. Every register is zero when the mip level is past the last level, `x + N - 1` is outside the level's width (unaligned x, unsigned), or y is outside its height. Data only comes back when the N texels fit in one dword: R8 and R16 (DMASK 0x3) or RG8 (0xf) for BY2, and R8 (0xf) for BY4. With 32-bit texels (R32, RG32, RG16, RGBA8) the hardware returns zeros. Other formats, masks, dimensions, swizzles, D16/A16 and indirect descriptors throw. The stores (0x52, 0x53, 0x5a and 0x5b) decode but throw at translation. Measured on hardware, they write from the unaligned x up to the end of its dword, which is not implemented. - [2D stores to 1D storage images](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp): an `image_store` with DIM 2D through a 1D T# writes texel x and ignores y, as measured on RDNA2 (gfx1035) for y from 0 to 4 and 0xffffffff, single- and four-component formats and a 1x1 8_8_8_8_UNORM T#: the T# type decides the addressing, DIM only how many address VGPRs are read. Storage loads, atomics and `image_store_mip` through such a pair still throw - [MIMG PCK2/PCK4 loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_load[_mip]_pck2/pck4`, opcodes 0x70, 0x71, 0x73 and 0x74) were measured on RDNA2 (gfx1035) with the [hardware oracle](HW_ORACLE.md) for 2D images with identity swizzle. With data mask 0x1, they write one dword: the raw bits of the N texels (PCK2: 2, PCK4: 4) from `x & ~(N - 1)` on row y, concatenated (element k at bit `k * texel bits`) and zero-extended. The dword is zero when the mip level is past the last level, `x + N - 1` is outside the level's width (unaligned x, unsigned), or y is outside its height. Data only comes back when N texels fit in 32 bits: R8, R16 or RG8 for PCK2, R8 for PCK4. Wider texels (R32, RG16, RGBA8) and masks other than 0x1 write zeros on hardware. Those masks throw at decode, and formats outside UNORM/UINT/SINT R8, R16 and RG8 throw. #1314's earlier measurement saw only zeros because it used masks 0x3 and 0xf. The stores (0x76, 0x77, 0x79 and 0x7a) decode but throw at translation. They are not measured, and opcode 0x76 on 32_32_32_32_FLOAT hung the GPU (#1314). From d4b98087059d3a5bd41878075328cd5cdf954e6a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 03:36:40 +0200 Subject: [PATCH 297/460] fix(libkernel_sys): keep umask bits up to 07777 like FreeBSD Co-Authored-By: Claude --- core/libs/prx/libkernel_sys/Export.cpp | 2 +- core/libs/tests/GuestKernelSys.cpp | 2 +- docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libkernel_sys/Export.cpp b/core/libs/prx/libkernel_sys/Export.cpp index 82a7cee18..a37212fa9 100644 --- a/core/libs/prx/libkernel_sys/Export.cpp +++ b/core/libs/prx/libkernel_sys/Export.cpp @@ -44,7 +44,7 @@ int APS5_VABI setsid_nid_postfix(void) { } std::uint16_t APS5_VABI umask_nid_postfix(std::uint16_t mask) { - return creationMask.exchange(static_cast(mask & 0777)); + return creationMask.exchange(static_cast(mask & 07777)); } std::int64_t APS5_VABI readlink_nid_postfix(const char* path, char* buffer, std::size_t size) { diff --git a/core/libs/tests/GuestKernelSys.cpp b/core/libs/tests/GuestKernelSys.cpp index 5e340f500..5069aac4e 100644 --- a/core/libs/tests/GuestKernelSys.cpp +++ b/core/libs/tests/GuestKernelSys.cpp @@ -31,7 +31,7 @@ int main() { Require(setsid_nid_postfix() == -1 && Error() == 1); Require(umask_nid_postfix(0) == 022); Require(umask_nid_postfix(07777) == 0); - Require(umask_nid_postfix(022) == 0777); + Require(umask_nid_postfix(022) == 07777); const auto root = std::filesystem::path("anyps5-kernel-sys-test-" + std::to_string(std::chrono::steady_clock::now().time_since_epoch().count())); Require(std::filesystem::create_directory(root)); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 572036bf9..bde6ad1ad 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -358,7 +358,7 @@ Throughout the project, every function at every stage either **does exactly what - [Packed horizontal gathers](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_gather4h_pck`, MIMG opcode 98): the four window texels are read raw and concatenated into a little-endian bitstream, with DMASK bit i writing stream word i to the next destination register; measured on RDNA2 (gfx1031) at every edge: the anchor is floor(x-0.5) with elements at -1, 0, +1 and +2 clamped horizontally into the width, the row is floor(y), negative rows clamp to row 0, rows at or past the height return zeros, sampler addressing modes do not affect the window, and formats wider than 32 bits per texel write zeros in every register. The emitter fetches each window texel at LOD 0 and packs its bits like `image_load_pck`, so the sampler's filter, addressing and LOD state are not used; formats whose bits are not recoverable from the view and D16 data throw. The anchor and row multiply the normalized coordinate by the image size without contraction. The stream rule was measured for R8_UNORM, RG8_UNORM and RGBA8; only RGBA8 has an execution test. `image_gather8h_pck` (opcode 99) never returns texel data on this hardware - across dmasks 0x1 to 0xFF including the upper nibble, RGBA8, RGBA16 and RGBA32 formats, 1D, 2D and 3D descriptors and normalized and unnormalized sampling it only zero-fills popcount(dmask) registers (sentinel-prefilled destinations) - so it decodes but throws at translation; its eight-texel semantics remain unmeasured. - [waitpid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) returns -1 with `ECHILD` (`EINVAL` for unknown options): nothing here creates a child process; child processes would need a real `waitpid`. - [execvp](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) throws: replacing the process image is not supported. PPSA85300 imports it and fails to load without it; its `fork` and `vfork` fail with `ENOSYS` ([libc_missing.c](https://github.com/ZiZc3/VLC/blob/f3253020a64368153233ede29183014c0b6c8979/ps5/compat/libc_missing.c#L290-L292)), so the child process that would call it is never created. -- [fork](../../core/libs/prx/libkernel_sys/Export.cpp) (libkernel_sys) throws: creating a child process is not supported. PPSA19111 imports it and fails to load without it. Its engine calls `fork` in `Posix_Daemonize`, only with `-daemonize`, and in `Platform_ShellExecute` before `execvp("xdg-open")`, which the main menu's Previews button reaches, so pressing Previews ends in this throw. [setsid](../../core/libs/prx/libkernel_sys/Export.cpp) fails with `EPERM`, as FreeBSD's does for a process group leader, since the guest is taken to lead its own process group (see `kill`). [umask](../../core/libs/prx/libkernel_sys/Export.cpp) keeps the guest's mask (initially 022) and returns the previous one, but files are still created with the host process's mask +- [fork](../../core/libs/prx/libkernel_sys/Export.cpp) (libkernel_sys) throws: creating a child process is not supported. PPSA19111 imports it and fails to load without it. Its engine calls `fork` in `Posix_Daemonize`, only with `-daemonize`, and in `Platform_ShellExecute` before `execvp("xdg-open")`, which the main menu's Previews button reaches, so pressing Previews ends in this throw. [setsid](../../core/libs/prx/libkernel_sys/Export.cpp) fails with `EPERM`, as FreeBSD's does for a process group leader, since the guest is taken to lead its own process group (see `kill`). [umask](../../core/libs/prx/libkernel_sys/Export.cpp) keeps the guest's mask (initially 022, masked to 07777) and returns the previous one, but files are still created with the host process's mask - [MIMG BY2/BY4 loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_load[_mip]_by2/by4`, opcodes 0x42, 0x43, 0x4a and 0x4b) were measured on RDNA2 (gfx1035) with the [hardware oracle](HW_ORACLE.md) for 2D images with identity swizzle. They read N texels (BY2: 2, BY4: 4) from `x & ~(N - 1)` on row y, texel by texel and component by component within each texel, with each texel converted like `image_load`. Every register is zero when the mip level is past the last level, `x + N - 1` is outside the level's width (unaligned x, unsigned), or y is outside its height. Data only comes back when the N texels fit in one dword: R8 and R16 (DMASK 0x3) or RG8 (0xf) for BY2, and R8 (0xf) for BY4. With 32-bit texels (R32, RG32, RG16, RGBA8) the hardware returns zeros. Other formats, masks, dimensions, swizzles, D16/A16 and indirect descriptors throw. The stores (0x52, 0x53, 0x5a and 0x5b) decode but throw at translation. Measured on hardware, they write from the unaligned x up to the end of its dword, which is not implemented. - [2D stores to 1D storage images](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp): an `image_store` with DIM 2D through a 1D T# writes texel x and ignores y, as measured on RDNA2 (gfx1035) for y from 0 to 4 and 0xffffffff, single- and four-component formats and a 1x1 8_8_8_8_UNORM T#: the T# type decides the addressing, DIM only how many address VGPRs are read. Storage loads, atomics and `image_store_mip` through such a pair still throw - [MIMG PCK2/PCK4 loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_load[_mip]_pck2/pck4`, opcodes 0x70, 0x71, 0x73 and 0x74) were measured on RDNA2 (gfx1035) with the [hardware oracle](HW_ORACLE.md) for 2D images with identity swizzle. With data mask 0x1, they write one dword: the raw bits of the N texels (PCK2: 2, PCK4: 4) from `x & ~(N - 1)` on row y, concatenated (element k at bit `k * texel bits`) and zero-extended. The dword is zero when the mip level is past the last level, `x + N - 1` is outside the level's width (unaligned x, unsigned), or y is outside its height. Data only comes back when N texels fit in 32 bits: R8, R16 or RG8 for PCK2, R8 for PCK4. Wider texels (R32, RG16, RGBA8) and masks other than 0x1 write zeros on hardware. Those masks throw at decode, and formats outside UNORM/UINT/SINT R8, R16 and RG8 throw. #1314's earlier measurement saw only zeros because it used masks 0x3 and 0xf. The stores (0x76, 0x77, 0x79 and 0x7a) decode but throw at translation. They are not measured, and opcode 0x76 on 32_32_32_32_FLOAT hung the GPU (#1314). From 9fe5d90cbedc3fc971277c4bd2efc377da13fce6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 03:47:10 +0200 Subject: [PATCH 298/460] test(agc): split agc_driver_flat_apertures into six parts Co-Authored-By: Claude --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 11 ++++++++++- .../tests/execution/FlatApertures.cpp | 17 +++++++++++++++-- 2 files changed, 25 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 866503983..6d03e7feb 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -955,7 +955,7 @@ if(BUILD_TESTING) set_tests_properties(agc_shader_bindless agc_shader_frontend_pair agc_shader_frontend_pair_off PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() foreach(agcTest IN ITEMS agc_shared_write_tests agc_driver_tests agc_shader_memory_tests agc_structurizer_tests agc_spirv_module_tests agc_dpp_write_tests agc_masked_select_tests agc_constant_bit_test_tests agc_scalar_select_mask_tests agc_vop1_control_decode_tests agc_validate_program_tests agc_pass_hazards_tests agc_shader_disk_cache_tests agc_shader_device_profile_tests agc_shader_control_flow_tests agc_driver_flip_tests agc_driver_pm4_tests agc_driver_graphics_tests agc_driver_bda_device_tests agc_driver_mesh_tests agc_driver_cmask_fast_clear_tests agc_formatted_buffer_once_tests ${agcExecutionTests}) - if(agcTest STREQUAL "agc_driver_constant_store_tests") + if(agcTest STREQUAL "agc_driver_constant_store_tests" OR agcTest STREQUAL "agc_driver_flat_apertures_tests") continue() endif() string(REGEX REPLACE "_tests$" "" agcTestName "${agcTest}") @@ -974,6 +974,15 @@ if(BUILD_TESTING) endif() set_tests_properties(agc_driver_provoking_vertex PROPERTIES TIMEOUT 30) set_tests_properties(agc_driver_flat_store PROPERTIES RUN_SERIAL TRUE) + set_target_properties(agc_driver_flat_apertures_tests PROPERTIES EXCLUDE_FROM_ALL OFF) + foreach(flatAperturesPart RANGE 5) + add_test(NAME agc_driver_flat_apertures_${flatAperturesPart} COMMAND agc_driver_flat_apertures_tests --part ${flatAperturesPart}/6) + set_tests_properties(agc_driver_flat_apertures_${flatAperturesPart} PROPERTIES ENVIRONMENT "ANYPS5_NO_SHADER_CACHE=1" SKIP_RETURN_CODE 77 WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}") + if(WIN32) + set_tests_properties(agc_driver_flat_apertures_${flatAperturesPart} PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() + endforeach() set_tests_properties(agc_driver_storage_cache_budget PROPERTIES TIMEOUT 30) add_test(NAME agc_driver_sampled_cache_budget_memory COMMAND agc_driver_sampled_cache_budget_tests memory) set_tests_properties(agc_driver_sampled_cache_budget_memory PROPERTIES ENVIRONMENT "ANYPS5_NO_SHADER_CACHE=1" SKIP_RETURN_CODE 77) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp index 810f6a16d..2fb3c5932 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp @@ -21,6 +21,7 @@ #include #include #include +#include #include #include @@ -478,14 +479,26 @@ std::vector Rows() { } -int main() { +int main(int argc, char** argv) { try { + std::size_t part = 0, parts = 1; + if (argc == 3 && std::string_view(argv[1]) == "--part") { + const std::string_view value(argv[2]); + const auto slash = value.find('/'); + part = std::stoul(std::string(value.substr(0, slash))); + parts = slash == std::string_view::npos ? 0 : std::stoul(std::string(value.substr(slash + 1))); + } else if (argc != 1) { + parts = 0; + } + if (parts == 0 || part >= parts) throw std::invalid_argument("usage: [--part K/N] with K < N"); const auto device = OpenVulkanTestDevice(); if (!device) return VulkanTestSkipped; const bool wave64 = device->Target().subgroupSize >= 32u; GuestBlock guest; std::size_t checked = 0; - for (const Row& row : Rows()) { + const auto rows = Rows(); + for (std::size_t index = part; index < rows.size(); index += parts) { + const Row& row = rows[index]; if (row.waveSize == 64u && !wave64) continue; Run(*device, row, guest); ++checked; From 15ae919325f2fe4571c48b09795058d69b44d513 Mon Sep 17 00:00:00 2001 From: Gustavo Ariel Gamarra Rojas <111788638+GustavoGamarra95@users.noreply.github.com> Date: Sat, 10 Oct 2026 21:22:22 -0300 Subject: [PATCH 299/460] perf(libkernel): look up the mapping with PROCMAP_QUERY in sceKernelGetModuleInfoForUnwind On Linux every call opened /proc/self/maps and scanned it line by line with sscanf. Unity titles call it on each stack walk, and with about 900 mappings God of War Sons of Sparta (PPSA28997) spent most of its main thread there. The covering mapping is now asked of the kernel with PROCMAP_QUERY (Linux 6.11), on one descriptor opened once, as GuestMemory does; older kernels and APS5_NO_PROCMAP_QUERY keep the line scan. --- .../include/specifics/linux/HostMapping.hpp | 106 ++++++++++++++++++ core/libs/prx/libkernel/Module/src/Module.cpp | 42 +++---- 2 files changed, 119 insertions(+), 29 deletions(-) create mode 100644 core/libs/prx/libc/include/specifics/linux/HostMapping.hpp diff --git a/core/libs/prx/libc/include/specifics/linux/HostMapping.hpp b/core/libs/prx/libc/include/specifics/linux/HostMapping.hpp new file mode 100644 index 000000000..aeecd02b3 --- /dev/null +++ b/core/libs/prx/libc/include/specifics/linux/HostMapping.hpp @@ -0,0 +1,106 @@ +#ifndef PRX_LIBC_INCLUDE_SPECIFICS_LINUX_HOSTMAPPING_HPP +#define PRX_LIBC_INCLUDE_SPECIFICS_LINUX_HOSTMAPPING_HPP + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#if __has_include() +#include +#endif +#if !defined(PROCMAP_QUERY) +struct procmap_query { + std::uint64_t size; + std::uint64_t query_flags; + std::uint64_t query_addr; + std::uint64_t vma_start; + std::uint64_t vma_end; + std::uint64_t vma_flags; + std::uint64_t vma_page_size; + std::uint64_t vma_offset; + std::uint64_t inode; + std::uint32_t dev_major; + std::uint32_t dev_minor; + std::uint32_t vma_name_size; + std::uint32_t build_id_size; + std::uint64_t vma_name_addr; + std::uint64_t build_id_addr; +}; +static_assert(sizeof(procmap_query) == 104); +enum : std::uint64_t { + PROCMAP_QUERY_COVERING_OR_NEXT_VMA = 0x10, +}; +#define PROCMAP_QUERY _IOWR('f', 17, struct procmap_query) +#endif + +namespace HostMapping { + +struct Mapping { + std::uint64_t start; + std::uint64_t end; + std::string path; +}; + +inline int QueryFd() { + static const int fd = [] { + if (std::getenv("APS5_NO_PROCMAP_QUERY") != nullptr) return -1; + const int opened = open("/proc/self/maps", O_RDONLY | O_CLOEXEC); + if (opened < 0) return -1; + procmap_query probe{}; + probe.size = sizeof(probe); + probe.query_flags = PROCMAP_QUERY_COVERING_OR_NEXT_VMA; + if (ioctl(opened, PROCMAP_QUERY, &probe) == 0 || errno == ENOENT) return opened; + close(opened); + return -1; + }(); + return fd; +} + +inline std::optional ScanMaps(std::uint64_t address, const char* caller) { + std::ifstream maps("/proc/self/maps"); + if (!maps) throw std::runtime_error(std::string(caller) + ": failed to open /proc/self/maps"); + std::string line; + while (std::getline(maps, line)) { + std::uint64_t start = 0; + std::uint64_t end = 0; + char perms[8] = {}; + std::uint64_t offset = 0; + unsigned int devMajor = 0; + unsigned int devMinor = 0; + std::uint64_t inode = 0; + char path[4096] = {}; + const int parsed = std::sscanf(line.c_str(), "%llx-%llx %7s %llx %x:%x %llu %4095s", + (unsigned long long*)&start, (unsigned long long*)&end, perms, + (unsigned long long*)&offset, &devMajor, &devMinor, (unsigned long long*)&inode, path); + if (parsed < 7 || address < start || address >= end) continue; + return Mapping{start, end, parsed >= 8 ? path : ""}; + } + return std::nullopt; +} + +inline std::optional Find(std::uint64_t address, const char* caller) { + const int fd = QueryFd(); + if (fd < 0) return ScanMaps(address, caller); + char name[4096] = {}; + procmap_query query{}; + query.size = sizeof(query); + query.query_addr = address; + query.vma_name_size = sizeof(name); + query.vma_name_addr = reinterpret_cast(name); + if (ioctl(fd, PROCMAP_QUERY, &query) != 0) { + if (errno == ENOENT) return std::nullopt; + throw std::runtime_error(std::string(caller) + ": PROCMAP_QUERY failed"); + } + return Mapping{query.vma_start, query.vma_end, query.vma_name_size != 0 ? name : ""}; +} + +} + +#endif diff --git a/core/libs/prx/libkernel/Module/src/Module.cpp b/core/libs/prx/libkernel/Module/src/Module.cpp index 76dd6d63e..9a46bd0db 100644 --- a/core/libs/prx/libkernel/Module/src/Module.cpp +++ b/core/libs/prx/libkernel/Module/src/Module.cpp @@ -17,8 +17,8 @@ #include #include #else -#include #include +#include "prx/libc/include/specifics/linux/HostMapping.hpp" #endif #ifdef _WIN32 @@ -163,34 +163,18 @@ int APS5_VABI sceKernelGetModuleInfoForUnwind(uint64_t addr, int flags, ModuleIn info->seg0_size = textSize; return 0; #else - std::ifstream maps("/proc/self/maps"); - if (!maps) throw std::runtime_error("sceKernelGetModuleInfoForUnwind: failed to open /proc/self/maps"); - std::string line; - while (std::getline(maps, line)) { - std::uint64_t start = 0; - std::uint64_t end = 0; - char perms[8] = {}; - std::uint64_t offset = 0; - unsigned int devMajor = 0; - unsigned int devMinor = 0; - std::uint64_t inode = 0; - char path[4096] = {}; - int parsed = std::sscanf(line.c_str(), "%llx-%llx %7s %llx %x:%x %llu %4095s", - (unsigned long long*)&start, (unsigned long long*)&end, perms, - (unsigned long long*)&offset, &devMajor, &devMinor, (unsigned long long*)&inode, path); - if (parsed < 7 || addr < start || addr >= end) continue; - info->st_size = sizeof(ModuleInfoForUnwind); - std::strncpy(info->name, parsed >= 8 ? path : "", sizeof(info->name) - 1); - info->name[sizeof(info->name) - 1] = '\0'; - info->eh_frame_hdr_addr = 0; - info->eh_frame_addr = 0; - info->eh_frame_size = 0; - info->seg0_addr = start; - info->seg0_size = end - start; - if (IsRelinkedImage(addr)) FillGuestUnwindInfo(addr, info); - return 0; - } - return SCE_KERNEL_ERROR_ESRCH; + const auto mapping = HostMapping::Find(addr, "sceKernelGetModuleInfoForUnwind"); + if (!mapping) return SCE_KERNEL_ERROR_ESRCH; + info->st_size = sizeof(ModuleInfoForUnwind); + std::strncpy(info->name, mapping->path.c_str(), sizeof(info->name) - 1); + info->name[sizeof(info->name) - 1] = '\0'; + info->eh_frame_hdr_addr = 0; + info->eh_frame_addr = 0; + info->eh_frame_size = 0; + info->seg0_addr = mapping->start; + info->seg0_size = mapping->end - mapping->start; + if (IsRelinkedImage(addr)) FillGuestUnwindInfo(addr, info); + return 0; #endif } From d152ff833c796d9dd742d2c084f7b8dc3054837d Mon Sep 17 00:00:00 2001 From: Gustavo Ariel Gamarra Rojas <111788638+GustavoGamarra95@users.noreply.github.com> Date: Sat, 10 Oct 2026 22:07:32 -0300 Subject: [PATCH 300/460] feat(shader): turn the MRT0 alpha into a sample mask for alpha-to-coverage A pixel shader whose stage info asks for alpha-to-coverage declares SampleMask, starts with every sample covered and, at its MRT0 export, clears the mask when the alpha is below one half or NaN, running with late fragment tests. The shader keeps running, as a zero coverage mask does on the GPU. The flag is part of the static shader identity and of the request serialization, now version 16. --- .../libSceAgcDriver/tests/ShaderMemory.cpp | 16 +++++++++----- core/shader/recompiler/CacheKey.hpp | 1 + .../ControlFlow/src/RequestSerializer.cpp | 6 +++-- .../Optimization/ShaderStageInputInfo.hpp | 1 + core/shader/recompiler/Recompiler.hpp | 1 + .../src/SpirvMemory/SpirvEmitterHelpers.cpp | 4 ++++ .../src/SpirvMemory/SpirvModuleSetup.cpp | 2 +- .../SpirvBackend/src/SpirvModuleEmitter.cpp | 22 +++++++++++++++++++ .../src/ShaderInputInfoBuilder.cpp | 1 + 9 files changed, 46 insertions(+), 8 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp b/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp index c1c8977ac..7ddeec523 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp @@ -1295,7 +1295,7 @@ void verifyPixelRequestSerialization() { value.posZ, value.posW, value.frontFace, value.ancillary, value.sampleShading, value.noPerspective, value.linearCentroid, value.pixelKillEnable, value.depthExportEnable, value.sampleMaskExportEnable, value.earlyZ, value.executeOnNoop, value.conservativeZExport, value.orderedPixelShader, - value.targetOutputMode, value.targetExportMapping); + value.targetOutputMode, value.targetExportMapping, value.alphaToCoverage); }; const std::array code{0xbf810000u}; const std::array userData{0x12345678u, 0xabcdef01u, 0x87654321u}; @@ -1338,6 +1338,7 @@ void verifyPixelRequestSerialization() { .targetOutputMode = {1u, 2u, 3u, 4u, 5u, 6u, 7u, 8u} }; for (std::uint32_t i = 0; i < pixel.interpolatorSettings.size(); ++i) pixel.interpolatorSettings[i] = 0x10101010u + i; + pixel.alphaToCoverage = true; const std::array, 3> mappings{{ {0x00u, 0xe4u, 0xc6u, 0x1bu, 0xffu, 0x80u, 0x55u, 0xaau}, {0xe4u, 0xe4u, 0xe4u, 0xe4u, 0xe4u, 0xe4u, 0xe4u, 0xe4u}, @@ -1362,6 +1363,10 @@ void verifyPixelRequestSerialization() { RecompileCacheKey::Build(changed, replayKey); require(key == replayKey && RecompileCacheKey::ContextHash(request) == RecompileCacheKey::ContextHash(changed), "runtime pixel export mapping changed the static shader identity"); } + auto withoutCoverage = request; + withoutCoverage.context.pixel->alphaToCoverage = false; + RecompileCacheKey::Build(withoutCoverage, replayKey); + require(key != replayKey && RecompileCacheKey::ContextHash(request) != RecompileCacheKey::ContextHash(withoutCoverage), "alpha-to-coverage did not change the static shader identity"); } request.context.pixel.reset(); request.shader.stage = ShaderStage::Vertex; @@ -1373,12 +1378,12 @@ void verifyPixelRequestSerialization() { minimal.context.waveSize = 64; minimal.context.pixel = ShaderPixelStageInfo{}; const auto encoded = serializer.Serialize(minimal); - require(requestPrefix(encoded, 8u) == "NVNQQQ8AAAA=", "new requests did not use serialization version 15"); + require(requestPrefix(encoded, 8u) == "NVNQQRAAAAA=", "new requests did not use serialization version 16"); constexpr std::size_t mappingOffset = 8u + 37u + 18u + 163u; - for (std::size_t bytes = 0; bytes < 17u; ++bytes) { - expectFailure([&] { static_cast(serializer.Deserialize(requestPrefix(encoded, mappingOffset + bytes))); }, "truncated data", "a truncated pixel mapping, packing or dual-source flag was accepted"); + for (std::size_t bytes = 0; bytes < 18u; ++bytes) { + expectFailure([&] { static_cast(serializer.Deserialize(requestPrefix(encoded, mappingOffset + bytes))); }, "truncated data", "a truncated pixel mapping, packing, dual-source or alpha-to-coverage flag was accepted"); } - for (const auto unsupported : {"NVNQQQAAAAA=", "NVNQQRAAAAA="}) { + for (const auto unsupported : {"NVNQQQAAAAA=", "NVNQQREAAAA="}) { expectFailure([&] { static_cast(serializer.Deserialize(unsupported)); }, "serialization version", "an unsupported request version was accepted"); } } @@ -1467,6 +1472,7 @@ void verifyLegacyPixelRequests() { require(pixel.targetExportMapping == std::array{}, "legacy pixel mapping no longer defaults to zero"); require(pixel.targetExportPacking == std::array{}, "a legacy request packed a color export"); require(!pixel.dualSourceBlend, "a legacy request enabled dual-source blending"); + require(!pixel.alphaToCoverage, "a legacy request enabled alpha-to-coverage"); require(pixel.inputAddr == 2u && pixel.hasPerspectiveCenterVgpr && pixel.targetOutputMode[0] == 9u, "legacy pixel layout was misread"); require(pixel.conservativeZExport == (version >= 7u ? ConservativeZExport::GreaterThanZ : ConservativeZExport::AnyZ), "legacy conservative Z layout was misread"); require(!pixel.orderedPixelShader, "a legacy request became a primitive-ordered pixel shader"); diff --git a/core/shader/recompiler/CacheKey.hpp b/core/shader/recompiler/CacheKey.hpp index 4207ad6ab..ddffedd3c 100644 --- a/core/shader/recompiler/CacheKey.hpp +++ b/core/shader/recompiler/CacheKey.hpp @@ -176,6 +176,7 @@ private: append(key, value.executeOnNoop); append(key, value.conservativeZExport); append(key, value.orderedPixelShader); + append(key, value.alphaToCoverage); append(key, value.targetOutputMode); } diff --git a/core/shader/recompiler/ControlFlow/src/RequestSerializer.cpp b/core/shader/recompiler/ControlFlow/src/RequestSerializer.cpp index e5b8a2f7c..60ad4c460 100644 --- a/core/shader/recompiler/ControlFlow/src/RequestSerializer.cpp +++ b/core/shader/recompiler/ControlFlow/src/RequestSerializer.cpp @@ -324,6 +324,7 @@ void writePixelInfo(Writer& writer, const ShaderPixelStageInfo& info) { writer.WriteU8(static_cast(value)); } writer.WriteBool(info.dualSourceBlend); + writer.WriteBool(info.alphaToCoverage); } ShaderPixelStageInfo readPixelInfo(Reader& reader, std::uint32_t version) { @@ -371,6 +372,7 @@ ShaderPixelStageInfo readPixelInfo(Reader& reader, std::uint32_t version) { } } if (version >= 15u) info.dualSourceBlend = reader.ReadBool(); + if (version >= 16u) info.alphaToCoverage = reader.ReadBool(); if (version < 5u) { const auto input = [](PixelInput value, bool present) { return present ? PixelInputBit(value) : 0u; }; info.inputAddr = input(PixelInput::PerspectiveSample, info.hasPerspectiveCenterVgpr && inputAddrOrCenterVgpr == 2u) | input(PixelInput::PerspectiveCenter, info.hasPerspectiveCenterVgpr) | @@ -732,7 +734,7 @@ std::string RequestSerializer::Serialize(const RecompileRequest& request) const std::string buffer; Writer writer(buffer); writer.WriteU32(0x41505335u); - writer.WriteU32(15u); + writer.WriteU32(16u); writeShaderBinary(writer, request.shader); writeGuestContext(writer, request.context); writeSpirvTarget(writer, request.target); @@ -759,7 +761,7 @@ DeserializedRequest RequestSerializer::Deserialize(std::string_view text) const Reader reader(decoded); if (reader.ReadU32() != 0x41505335u) throw std::runtime_error("invalid recompile request signature"); const auto version = reader.ReadU32(); - if (version < 1u || version > 15u) throw std::runtime_error("unsupported recompile request serialization version"); + if (version < 1u || version > 16u) throw std::runtime_error("unsupported recompile request serialization version"); DeserializedRequest result{}; result.request.shader = readShaderBinary(reader, result.shaderCode, result.shaderHeader); result.request.context = readGuestContext(reader, result, version); diff --git a/core/shader/recompiler/Optimization/include/Optimization/ShaderStageInputInfo.hpp b/core/shader/recompiler/Optimization/include/Optimization/ShaderStageInputInfo.hpp index 8f72d9821..45f4984f5 100644 --- a/core/shader/recompiler/Optimization/include/Optimization/ShaderStageInputInfo.hpp +++ b/core/shader/recompiler/Optimization/include/Optimization/ShaderStageInputInfo.hpp @@ -173,6 +173,7 @@ struct ShaderPixelInputInfo { bool psExecuteOnNoop = false; ConservativeZExport psConservativeZExport = ConservativeZExport::AnyZ; bool psOrderedPixelShader = false; + bool psAlphaToCoverage = false; ShaderStageRuntime stage; bool HasPositionInput() const { diff --git a/core/shader/recompiler/Recompiler.hpp b/core/shader/recompiler/Recompiler.hpp index 5688af8e8..951101c82 100644 --- a/core/shader/recompiler/Recompiler.hpp +++ b/core/shader/recompiler/Recompiler.hpp @@ -140,6 +140,7 @@ struct ShaderPixelStageInfo { std::array targetExportMapping; std::array targetExportPacking; bool dualSourceBlend; + bool alphaToCoverage; }; struct ShaderVertexBufferResource { diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp index 79c762646..daf4a4110 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp @@ -355,6 +355,10 @@ void DefineOutputs(SpirvEmitterState& state) { } } } + if (state.program.Resources().stage == IrShaderStage::Pixel && state.inputInfo.pixel != nullptr && state.inputInfo.pixel->psAlphaToCoverage) { + if (state.inputInfo.pixel->psSampleMaskExportEnable) throw std::runtime_error("alpha-to-coverage with a sample-mask export is unsupported"); + BuiltIn(state.sampleMaskVariable, SampleMaskArrayType(state), "gl_SampleMask", spv::BuiltInSampleMask); + } } void DefineDescriptors(SpirvEmitterState& state) { diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvModuleSetup.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvModuleSetup.cpp index b8d68e795..26e0c761a 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvModuleSetup.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvModuleSetup.cpp @@ -176,7 +176,7 @@ void DefineModule(SpirvEmitterState& state) { state.module.AddExecutionMode(state.mainFunc, spv::ExecutionModeDepthGreater); } } - if (pixel.psEarlyZ && !pixel.psPixelKillEnable && !pixel.psDepthExportEnable && !pixel.psSampleMaskExportEnable) { + if (pixel.psEarlyZ && !pixel.psPixelKillEnable && !pixel.psDepthExportEnable && !pixel.psSampleMaskExportEnable && !pixel.psAlphaToCoverage) { state.module.AddExecutionMode(state.mainFunc, spv::ExecutionModeEarlyFragmentTests); } if (pixel.psOrderedPixelShader) { diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp index 07f3ca948..45145cfc6 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp @@ -272,6 +272,26 @@ std::uint32_t SelectExportPacking(SpirvEmitterState& state, std::uint32_t target return result; } +bool AlphaToCoverage(const SpirvEmitterState& state) { + return state.program.Resources().stage == IrShaderStage::Pixel && state.inputInfo.pixel != nullptr && state.inputInfo.pixel->psAlphaToCoverage; +} + +void StoreCoverage(SpirvEmitterState& state, std::uint32_t mask) { + const auto pointer = state.module.AllocateId(); + state.module.AddFunction(spv::OpAccessChain, TypePointer(state, spv::StorageClassOutput, TypeI32(state)), pointer, state.sampleMaskVariable, ConstantU32(state, 0u)); + state.module.AddFunction(spv::OpStore, pointer, mask); +} + +void StoreAlphaCoverage(SpirvEmitterState& state, std::uint32_t color) { + const auto alpha = state.module.AllocateId(); + state.module.AddFunction(spv::OpCompositeExtract, TypeF32(state), alpha, color, 3u); + const auto below = state.module.AllocateId(); + state.module.AddFunction(spv::OpFUnordLessThan, TypeBool(state), below, alpha, ConstantF32(state, 0x3f000000u)); + const auto mask = state.module.AllocateId(); + state.module.AddFunction(spv::OpSelect, TypeI32(state), mask, below, ConstantI32(state, 0), ConstantI32(state, -1)); + StoreCoverage(state, mask); +} + std::uint32_t ExportRawComponent(SpirvValueEmitContext& ctx, std::uint32_t vector, std::uint32_t component) { const auto value = ctx.state.module.AllocateId(); ctx.state.module.AddFunction(spv::OpCompositeExtract, TypeU32(ctx.state), value, vector, component); @@ -983,6 +1003,7 @@ void EmitProgram(SpirvEmitterState& state) { if (state.pixelValidMaskVariable != 0u) { state.module.AddFunction(spv::OpStore, state.pixelValidMaskVariable, ConstantU32(state, 1u)); } + if (AlphaToCoverage(state)) StoreCoverage(state, ConstantI32(state, -1)); if (OrderedPixelShader(state)) { state.module.AddFunction(spv::OpBeginInvocationInterlockEXT); } @@ -1235,6 +1256,7 @@ void EmitSetAttribute(SpirvValueEmitContext& ctx, const IrValue& inst) { const auto mapped = state.module.AllocateId(); state.module.AddFunction(spv::OpCompositeConstruct, vectorType, mapped, components[0], components[1], components[2], components[3]); value = mapped; + if (exp.index == 0u && !uintOutput && AlphaToCoverage(state)) StoreAlphaCoverage(state, value); const auto mode = MrtOutputMode(state, exp); if (!uintOutput && mode != 5u && mode != 6u) value = SelectExportPacking(state, exp.index, value); } diff --git a/core/shader/recompiler/Translation/src/ShaderInputInfoBuilder.cpp b/core/shader/recompiler/Translation/src/ShaderInputInfoBuilder.cpp index c14e244be..18d89e989 100644 --- a/core/shader/recompiler/Translation/src/ShaderInputInfoBuilder.cpp +++ b/core/shader/recompiler/Translation/src/ShaderInputInfoBuilder.cpp @@ -141,6 +141,7 @@ ShaderStageInputInfo BuildShaderStageInputInfo(ShaderStageKind stage, const Gues pixelStorage.psExecuteOnNoop = pixel.executeOnNoop; pixelStorage.psConservativeZExport = pixel.conservativeZExport; pixelStorage.psOrderedPixelShader = pixel.orderedPixelShader; + pixelStorage.psAlphaToCoverage = pixel.alphaToCoverage; ShaderStageInputInfo result; result.pixel = &pixelStorage; return result; From ad4add2b1a85549a257eead82412b684bfecdf82 Mon Sep 17 00:00:00 2001 From: Gustavo Ariel Gamarra Rojas <111788638+GustavoGamarra95@users.noreply.github.com> Date: Sat, 10 Oct 2026 21:22:42 -0300 Subject: [PATCH 301/460] perf(libSceSysmodule): look up the mapping with PROCMAP_QUERY in sceSysmoduleGetModuleInfoForUnwind It carried its own copy of the /proc/self/maps line scan and is called on the same Unity stack walks as sceKernelGetModuleInfoForUnwind. It now uses the shared HostMapping lookup. --- core/libs/prx/libSceSysmodule/Export.cpp | 53 ++++++------------------ 1 file changed, 13 insertions(+), 40 deletions(-) diff --git a/core/libs/prx/libSceSysmodule/Export.cpp b/core/libs/prx/libSceSysmodule/Export.cpp index 83f56fb54..61b73e6af 100644 --- a/core/libs/prx/libSceSysmodule/Export.cpp +++ b/core/libs/prx/libSceSysmodule/Export.cpp @@ -9,7 +9,7 @@ #include #include #else -#include +#include "prx/libc/include/specifics/linux/HostMapping.hpp" #endif #include "SceTypes.hpp" @@ -47,46 +47,19 @@ bool fillModuleInfoForUnwind(std::uint64_t addr, ModuleInfoForUnwind* info) { info->name[sizeof(info->name) - 1] = '\0'; return true; #else - std::ifstream maps("/proc/self/maps"); - if (!maps) { - throw std::runtime_error("sceSysmoduleGetModuleInfoForUnwind: failed to open /proc/self/maps"); + const auto mapping = HostMapping::Find(addr, "sceSysmoduleGetModuleInfoForUnwind"); + if (!mapping) { + return false; } - std::string line; - while (std::getline(maps, line)) { - std::uint64_t start = 0; - std::uint64_t end = 0; - char perms[8] = {}; - std::uint64_t offset = 0; - unsigned int devMajor = 0; - unsigned int devMinor = 0; - std::uint64_t inode = 0; - char path[4096] = {}; - int parsed = std::sscanf( - line.c_str(), - "%llx-%llx %7s %llx %x:%x %llu %4095s", - (unsigned long long*)&start, - (unsigned long long*)&end, - perms, - (unsigned long long*)&offset, - &devMajor, - &devMinor, - (unsigned long long*)&inode, - path - ); - if (parsed < 7 || addr < start || addr >= end) { - continue; - } - info->st_size = sizeof(ModuleInfoForUnwind); - std::strncpy(info->name, parsed >= 8 ? path : "", sizeof(info->name) - 1); - info->name[sizeof(info->name) - 1] = '\0'; - info->eh_frame_hdr_addr = 0; - info->eh_frame_addr = 0; - info->eh_frame_size = 0; - info->seg0_addr = start; - info->seg0_size = end - start; - return true; - } - return false; + info->st_size = sizeof(ModuleInfoForUnwind); + std::strncpy(info->name, mapping->path.c_str(), sizeof(info->name) - 1); + info->name[sizeof(info->name) - 1] = '\0'; + info->eh_frame_hdr_addr = 0; + info->eh_frame_addr = 0; + info->eh_frame_size = 0; + info->seg0_addr = mapping->start; + info->seg0_size = mapping->end - mapping->start; + return true; #endif } From 7d92bf70f9487adbe7bc1e7eeae664ad18a830aa Mon Sep 17 00:00:00 2001 From: Gustavo Ariel Gamarra Rojas <111788638+GustavoGamarra95@users.noreply.github.com> Date: Sat, 10 Oct 2026 22:07:40 -0300 Subject: [PATCH 302/460] feat(agc): decode undithered alpha-to-coverage DB_ALPHA_TO_MASK 0xaa01, the enable bit with the four offsets at 2 and no OFFSET_ROUND, reaches the pixel stage, which keeps a pixel when its MRT0 alpha is at least one half. Dithered or rounded settings, and an MRT0 that is missing, swizzled, packed or exported without a floating-point alpha, still throw. God of War Sons of Sparta (PPSA28997) stopped on it a few minutes into play. --- .../Execution/src/Driver/Draw/DrawDecode.cpp | 1 + .../Execution/src/Driver/Draw/DrawLookup.cpp | 1 + .../src/Driver/Draw/DrawRegisterKey.cpp | 2 +- .../Graphics/include/State.hpp | 1 + .../libSceAgcDriver/Graphics/src/State.cpp | 15 ++++++++++-- .../prx/libSceAgcDriver/tests/Graphics.cpp | 23 ++++++++++++++++++- docs/dev/TechnicalDebt.md | 1 + 7 files changed, 40 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp index f4444a808..b784526d6 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp @@ -114,6 +114,7 @@ std::shared_ptr Driver::decodeDraw(const QueueState& queue, const Su product->pixel = Graphics::DecodePixelStageInfo(queue.context, Graphics::ExportMappings(product->state), Graphics::PixelProgramSkipped(queue)); product->pixel.targetExportPacking = Graphics::ExportPackings(product->state); product->pixel.dualSourceBlend = product->state.dualSourceBlend; + product->pixel.alphaToCoverage = product->state.alphaToCoverage; return product; } diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawLookup.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawLookup.cpp index 090b9b10d..14090d411 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawLookup.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawLookup.cpp @@ -68,6 +68,7 @@ void Driver::lookupDraw(const Submission& submission, const std::shared_ptr(value)); mix(pixel.dualSourceBlend); + mix(pixel.alphaToCoverage); std::lock_guard cacheLock(drawCacheMutex); ++drawEntryCounters.lookups; const auto found = drawCache.find(drawKey); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawRegisterKey.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawRegisterKey.cpp index 1759f417e..3e1fd6a77 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawRegisterKey.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawRegisterKey.cpp @@ -105,7 +105,7 @@ bool Driver::sameDecode(const DrawDecode& a, const DrawDecode& b) { const auto& y = b.programs[i]; if (x.binary.stage != y.binary.stage || x.binary.codeAddress != y.binary.codeAddress || x.userDataBase != y.userDataBase || x.firstUserSgpr != y.firstUserSgpr || x.userData != y.userData || x.snapshot != y.snapshot || x.codeOffset != y.codeOffset) return false; } - if (p.targetExportPacking != q.targetExportPacking || p.dualSourceBlend != q.dualSourceBlend) return false; + if (p.targetExportPacking != q.targetExportPacking || p.dualSourceBlend != q.dualSourceBlend || p.alphaToCoverage != q.alphaToCoverage) return false; return true; } diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp index 05cb27663..df0a89c64 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp @@ -100,6 +100,7 @@ struct State { VkViewport viewport; bool negativeOneToOne; bool depthClamp = false; + bool alphaToCoverage = false; VkConservativeRasterizationModeEXT conservativeRasterization = VK_CONSERVATIVE_RASTERIZATION_MODE_DISABLED_EXT; VkRect2D scissor; VkCullModeFlags cullMode; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp index f29320922..f5818214d 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp @@ -104,6 +104,7 @@ constexpr std::uint32_t DepthControlMask = ~0x007007f0u; constexpr std::uint32_t ShaderControlMask = ~(0x0000f870u | 0x00020600u | 0x00010000u); constexpr std::uint32_t PixelStageRunsMask = 0x00020747u; constexpr std::uint32_t AlphaToCoverageMask = ~0x0001ff00u; +constexpr std::uint32_t AlphaToCoverageUndithered = 0x0000aa01u; constexpr std::uint32_t ScanModeMask = ~0x22u; constexpr std::uint32_t ScanControlMask = ~0x76023fffu; constexpr std::uint32_t ScreenOffsetMask = ~0x01ff01ffu; @@ -597,7 +598,8 @@ State DecodeState(const QueueState& queue) { Require((depthControl & 0xc0000000u) == 0, "depth-conditional color writes are unsupported"); } zero(cx, 0x203, shaderControlMask(read(cx, 0x1c4)), "depth export, shader coverage or ordered fragment execution"); - zero(cx, 0x2dc, AlphaToCoverageMask, "alpha-to-coverage"); + result.alphaToCoverage = read(cx, 0x2dc) == AlphaToCoverageUndithered; + if (!result.alphaToCoverage) zero(cx, 0x2dc, AlphaToCoverageMask, "dithered or rounded alpha-to-coverage"); zero(cx, 0x2f8, ~0u, "multisampling or coverage conversion"); zero(cx, 0x292, ScanModeMask, "scan conversion mode"); zero(cx, 0x293, ScanControlMask, "sample iteration, primitive discard or out-of-order rasterization"); @@ -770,6 +772,13 @@ State DecodeState(const QueueState& queue) { Require(((exportFormat >> 4u) & 0xfu) == (exportFormat & 0xfu), "dual-source blending needs the MRT1 export in the MRT0 export format"); Require(((shaderMask >> 4u) & 0xfu) == (shaderMask & 0xfu), "dual-source blending needs the MRT1 export with the MRT0 shader mask"); } + if (result.alphaToCoverage) { + const auto mrt0 = std::find_if(result.colors.begin(), result.colors.end(), [](const ColorTarget& color) { return color.exportIndex == 0; }); + Require(mrt0 != result.colors.end(), "alpha-to-coverage without a written MRT0 color target is unsupported"); + const auto format = exportFormat & 0xfu; + Require(format == 4u || format == 5u || format == 6u || format == 9u, "alpha-to-coverage needs an MRT0 export format with a floating-point alpha"); + Require(mrt0->componentMapping == 0xe4u && mrt0->packing == ShaderRecompiler::ColorExportPacking::None, "alpha-to-coverage into a component-swapped or packed MRT0 target is unsupported"); + } APS5_LOG_OUT_DEBUG("DecodeState done colorTarget=%u render=%ux%u topology=%u", result.hasColorTarget ? 1u : 0u, result.renderExtent.width, result.renderExtent.height, static_cast(result.topology)); return result; } @@ -983,7 +992,9 @@ std::string DrawRejection(const QueueState& queue, bool indexed) { std::uint32_t zFormat = 0; static_cast(value(cx, 0x1c4, zFormat)); if (auto reason = nonzero(cx, 0x203, shaderControlMask(zFormat), "depth export, shader coverage or ordered fragment execution"); !reason.empty()) return reason; - if (auto reason = nonzero(cx, 0x2dc, AlphaToCoverageMask, "alpha-to-coverage"); !reason.empty()) return reason; + if (value(cx, 0x2dc, word) && word != AlphaToCoverageUndithered) { + if (auto reason = nonzero(cx, 0x2dc, AlphaToCoverageMask, "dithered or rounded alpha-to-coverage"); !reason.empty()) return reason; + } if (auto reason = nonzero(cx, 0x2f8, ~0u, "multisampling or coverage conversion"); !reason.empty()) return reason; if (auto reason = nonzero(cx, 0x292, ScanModeMask, "scan conversion mode"); !reason.empty()) return reason; if (auto reason = nonzero(cx, 0x293, ScanControlMask, "sample iteration, primitive discard or out-of-order rasterization"); !reason.empty()) return reason; diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index cf7a795c0..52aff8942 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -1516,6 +1516,26 @@ void DepthClipTests() { } } +void AlphaToCoverageTests() { + auto queue = makeState(); + queue.context[0x1b3] = 2; + queue.context[0x1b4] = 2; + Require(!AgcDriver::Graphics::DecodeState(queue).alphaToCoverage, "alpha-to-coverage decoded while disabled"); + queue.context[0x2dc] = 0xaa01; + Require(AgcDriver::Graphics::DecodeState(queue).alphaToCoverage, "undithered alpha-to-coverage was not decoded"); + Require(AgcDriver::Graphics::DrawRejection(queue, false).empty(), "the precheck rejected undithered alpha-to-coverage"); + for (const auto control : {0x8701u, 0x1aa01u, 0xaa03u}) { + queue.context[0x2dc] = control; + expectFailure([&] { AgcDriver::Graphics::DecodeState(queue); }, "alpha-to-coverage"); + Require(AgcDriver::Graphics::DrawRejection(queue, false).find("alpha-to-coverage") != std::string::npos, "the precheck accepted unsupported alpha-to-coverage"); + } + queue.context[0x2dc] = 0xaa01; + const auto exportFormat = queue.context.at(0x1c5); + queue.context[0x1c5] = (exportFormat & ~0xfu) | 1u; + expectFailure([&] { AgcDriver::Graphics::DecodeState(queue); }, "floating-point alpha"); + queue.context[0x1c5] = exportFormat; +} + void InitialContextTests() { const auto configured = makeState(); AgcDriver::QueueState queue; @@ -1533,7 +1553,7 @@ void InitialContextTests() { } queue.context[0x206] = 0x43f; queue.context[0x2dc] |= 1; - expectFailure([&] { AgcDriver::Graphics::DecodeState(queue); }, "alpha-to-coverage"); + Require(AgcDriver::Graphics::DecodeState(queue).alphaToCoverage, "undithered alpha-to-coverage was not decoded"); queue.context.erase(0x2dc); expectFailure([&] { AgcDriver::Graphics::DecodeState(queue); }, "missing register"); queue.ClearContext(); @@ -4041,6 +4061,7 @@ int main() { ComputeScratchTests(); shaderUserDataTailPaddingTests(); InitialContextTests(); + AlphaToCoverageTests(); pushConstantTests(); resourceTests(); descriptorSnapshotTests(); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 237d82bed..df0e2020f 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -117,6 +117,7 @@ Throughout the project, every function at every stage either **does exactly what - [libSceVdecsw](../../core/libs/prx/libSceVdecsw/Export.cpp) (libSceVdecsw) - structure layouts follow shadPS4's `vdecsw.h` and were checked against PPSA05684's movie decoder (`movie-decoder-vdecsw.cpp`), which asserts `pictureCount == 1` at offset 0xb of `OutputInfo` and reads `InputResult` offset 0x10 as the number of frames an access unit produced. The compute, decoder memory and decode input structures and the first 0x28 bytes of the 0x50-byte decoder config are those of libSceVideodec2, and decoding goes through it. The 0x78-byte AVC picture info is filled with Videodec2's leading fields and the cropping fields only. OUTPUT_PENDING (0x81510115), INPUT_QUEUE_EMPTY (0x81510116) and DECODE_PENDING (0x81510117) follow from the codes each call's assert accepts. At most four decoded pictures are queued and the last one is held until `sceVdecswFinalizeDecodeSequence`; the console's queue depth is unknown - [sceGameLiveStreamingGetCurrentStatus2](../../core/libs/prx/libSceGameLiveStreaming/Export.cpp) (libSceGameLiveStreaming) - the status layout (user id, on-air flag, spectator, text and command message counts, broadcast resolution, 48 reserved bytes) is taken from shadPS4's `OrbisGameLiveStreamingStatus2`; the call reports that no stream is on air, with an invalid user id - [Depth clear draws](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (AGC driver) - a draw with `DB_RENDER_CONTROL.DEPTH_CLEAR_ENABLE` is taken to write `DB_DEPTH_CLEAR` to every pixel it covers whatever the depth test, depth bounds and depth bias say: it is drawn with an always-passing depth write and the viewport depth range pinned to the clear value. The draw is rasterized as submitted, so its coverage is the draw's own. Whether the DB still applies the depth test or HiZ to such a draw is not measured +- [Alpha-to-coverage](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (AGC driver) - only `DB_ALPHA_TO_MASK = 0xaa01` is accepted: the enable bit with the four quad offsets at 2 and no `OFFSET_ROUND`, the undithered setting [radeonsi](https://gitlab.freedesktop.org/mesa/mesa/-/blob/bffd91947efc1932da99aca69fc540d6d4201b0c/src/gallium/drivers/radeonsi/si_state.c#L371-L381) programs. Draws are single-sampled, so the pixel shader [writes](../../core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp) `SampleMask` 0 when its MRT0 alpha is below one half or NaN and all ones otherwise, and runs with late fragment tests. That rule was measured on a Radeon 660M (gfx1035, RADV with `RADV_DEBUG=noatocdithering`, which programs the same register) for 0, -0, 0.25, the two floats below one half, one half, the two above, 0.75, 1, 2, -1, both infinities and both NaNs, and is pinned in [AlphaToCoverage.cpp](../../core/libs/prx/libSceAgcDriver/tests/execution/AlphaToCoverage.cpp); it is not measured on gfx1013. Vulkan's `alphaToCoverageEnable` is not used: the NVIDIA driver 615.78.08 dithers single-sample coverage (an alpha of 0.25 writes half the pixels of an RTX 3050 Laptop target). Other offsets, `OFFSET_ROUND`, a sample-mask export, and an MRT0 that is missing, swizzled, packed or exported without a floating-point alpha throw - [GPU flips](../../core/libs/prx/libSceVideoOut/src/VideoOutDriver.cpp) (libSceVideoOut) - a command-buffer flip of a buffer index with no registered buffer when the GPU reaches it presents black and completes like any flip (flip event, count, `flipPendingNum`). The console's behaviour is unknown; PPSA05684 requires that it completes: its renderer submits a flip of buffer 0 while another thread waits for `sceVideoOutIsFlipPending` to reach 0 before registering the buffers, and `flipPendingNum` counts that flip from its submission as in shadPS4 - [sceAgcDriverGetEqContextId](../../core/libs/prx/libSceAgcDriver/Eq/src/Query.cpp) (libSceAgcDriver) - returns `ident`: an end-of-pipe interrupt is delivered only to the registration whose id equals the raising queue, so `ident` is that context's id (titles register 0 or 1 for graphics and `0x20 + 8 * pipe + queue` for compute). The console's value is not known; KytyPS5 returns the event's `data`, which it fills with the interrupt's context id, whereas `data` here is the EV_CLEAR trigger count. Other event filters throw - [sceAgcSetSubmitMode](../../core/libs/prx/libSceAgc/Misc/src/Suspend.cpp) (libSceAgc) - mode values unknown; only 0 is accepted From 869d73e4d82f4711cf973dbc4e3fc0d1bbc5dedd Mon Sep 17 00:00:00 2001 From: Gustavo Ariel Gamarra Rojas <111788638+GustavoGamarra95@users.noreply.github.com> Date: Sat, 10 Oct 2026 21:23:19 -0300 Subject: [PATCH 303/460] test(libSceSysmodule): compare sceSysmoduleGetModuleInfoForUnwind with /proc/self/maps Covers code, stack and an anonymous page, and an unmapped address. On Linux the test runs again with APS5_NO_PROCMAP_QUERY so the line scan stays covered. --- core/libs/prx/libSceSysmodule/CMakeLists.txt | 4 ++ core/libs/tests/GuestSysmodule.cpp | 67 ++++++++++++++++++++ 2 files changed, 71 insertions(+) diff --git a/core/libs/prx/libSceSysmodule/CMakeLists.txt b/core/libs/prx/libSceSysmodule/CMakeLists.txt index b9111e032..4d5f4a73e 100644 --- a/core/libs/prx/libSceSysmodule/CMakeLists.txt +++ b/core/libs/prx/libSceSysmodule/CMakeLists.txt @@ -7,6 +7,10 @@ if(BUILD_TESTING) target_include_directories(guest_sysmodule_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_sysmodule_tests PRIVATE libSceSysmodule) add_test(NAME guest_sysmodule COMMAND guest_sysmodule_tests) + if(UNIX AND NOT APPLE) + add_test(NAME guest_sysmodule_maps_scan COMMAND guest_sysmodule_tests) + set_tests_properties(guest_sysmodule_maps_scan PROPERTIES ENVIRONMENT "APS5_NO_PROCMAP_QUERY=1") + endif() if(WIN32) set_tests_properties(guest_sysmodule PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") diff --git a/core/libs/tests/GuestSysmodule.cpp b/core/libs/tests/GuestSysmodule.cpp index f3ffdecb0..f4f999aa3 100644 --- a/core/libs/tests/GuestSysmodule.cpp +++ b/core/libs/tests/GuestSysmodule.cpp @@ -1,6 +1,15 @@ +#include "SceTypes.hpp" #include "prx/libc/include/general/VabiMacros.hpp" #include #include +#ifdef __linux__ +#include +#include +#include +#include +#include +#include +#endif extern "C" { int APS5_VABI sceSysmoduleIsLoaded(std::uint16_t id); @@ -8,6 +17,7 @@ int APS5_VABI sceSysmoduleLoadModule(std::uint16_t id); int APS5_VABI sceSysmoduleUnloadModule(std::uint16_t id); int APS5_VABI sceSysmoduleLoadModuleInternal(std::uint32_t id); int APS5_VABI sceSysmoduleUnloadModuleInternal(std::uint32_t id); +int APS5_VABI sceSysmoduleGetModuleInfoForUnwind(std::uint64_t addr, int flags, ModuleInfoForUnwind* info); } namespace { @@ -18,6 +28,60 @@ constexpr std::uint16_t kFiberModuleId = 0x0006; constexpr std::uint16_t kUltModuleId = 0x0007; constexpr int kModuleUnloaded = static_cast(0x805A1001); +#ifdef __linux__ +struct Mapping { + std::uint64_t start = 0; + std::uint64_t end = 0; + std::string path; +}; + +Mapping MapsEntry(std::uint64_t address) { + std::ifstream maps("/proc/self/maps"); + std::string line; + while (std::getline(maps, line)) { + unsigned long long start = 0; + unsigned long long end = 0; + char path[4096] = {}; + const int parsed = std::sscanf(line.c_str(), "%llx-%llx %*s %*x %*x:%*x %*u %4095s", &start, &end, path); + if (parsed >= 2 && address >= start && address < end) return Mapping{start, end, parsed == 3 ? path : ""}; + } + std::abort(); +} + +void RequireUnwindMatchesMaps(const void* pointer) { + const auto address = reinterpret_cast(pointer); + const auto expected = MapsEntry(address); + ModuleInfoForUnwind info{}; + Require(sceSysmoduleGetModuleInfoForUnwind(address, 0, &info) == 0); + Require(info.st_size == sizeof(ModuleInfoForUnwind)); + Require(info.seg0_addr == expected.start && info.seg0_size == expected.end - expected.start); + Require(std::string(info.name) == expected.path.substr(0, sizeof(info.name) - 1)); + Require(info.eh_frame_hdr_addr == 0 && info.eh_frame_addr == 0 && info.eh_frame_size == 0); +} + +void CheckUnwindInfo() { + RequireUnwindMatchesMaps(reinterpret_cast(&RequireUnwindMatchesMaps)); + RequireUnwindMatchesMaps(reinterpret_cast(&sceSysmoduleGetModuleInfoForUnwind)); + int local = 0; + RequireUnwindMatchesMaps(&local); + const auto page = static_cast(sysconf(_SC_PAGESIZE)); + void* anonymous = mmap(nullptr, page, PROT_READ, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + Require(anonymous != MAP_FAILED); + RequireUnwindMatchesMaps(anonymous); + Require(munmap(anonymous, page) == 0); + void* gap = mmap(nullptr, page, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + Require(gap != MAP_FAILED && munmap(gap, page) == 0); + ModuleInfoForUnwind info{}; + bool threw = false; + try { + sceSysmoduleGetModuleInfoForUnwind(reinterpret_cast(gap), 0, &info); + } catch (const std::runtime_error&) { + threw = true; + } + Require(threw); +} +#endif + } int main() { @@ -37,4 +101,7 @@ int main() { Require(sceSysmoduleIsLoaded(kUltModuleId) == 0); Require(sceSysmoduleUnloadModule(kUltModuleId) == 0); Require(sceSysmoduleIsLoaded(kUltModuleId) == kModuleUnloaded); +#ifdef __linux__ + CheckUnwindInfo(); +#endif } From 50e84c7cf77fd1badbcb336462e18493b40bb2ca Mon Sep 17 00:00:00 2001 From: Gustavo Ariel Gamarra Rojas <111788638+GustavoGamarra95@users.noreply.github.com> Date: Sat, 10 Oct 2026 22:07:40 -0300 Subject: [PATCH 304/460] test(agc): pin alpha-to-coverage coverage measured on RDNA2 The expected rows were measured with Vulkan alpha-to-coverage on a Radeon 660M (gfx1035) under RADV_DEBUG=noatocdithering, which programs DB_ALPHA_TO_MASK 0xaa01: an alpha of at least one half, 2 and +inf keep the pixel; below one half, -0, negatives, -inf and NaN drop it. --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 2 + .../tests/execution/AlphaToCoverage.cpp | 327 ++++++++++++++++++ 2 files changed, 329 insertions(+) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/AlphaToCoverage.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 866503983..4d7fb8317 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -905,6 +905,8 @@ foreach(agcExecutionTestSource IN LISTS agcExecutionTestSources) list(APPEND agcExecutionTests ${agcExecutionTest}) endforeach() +target_include_directories(agc_driver_alpha_to_coverage_tests PRIVATE $) + target_include_directories(agc_driver_dual_source_blend_tests PRIVATE $) target_include_directories(agc_driver_packed_unorm_color_target_tests PRIVATE $) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/AlphaToCoverage.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/AlphaToCoverage.cpp new file mode 100644 index 000000000..5fa517874 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/AlphaToCoverage.cpp @@ -0,0 +1,327 @@ +#include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp" +#include "prx/libSceAgcDriver/Graphics/include/ColorTargetLayout.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "prx/libSceAgcDriver/Graphics/include/ShaderInputState.hpp" +#include "prx/libSceAgcDriver/Graphics/include/State.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Texture.hpp" +#include "prx/libc/include/GuestAllocations.hpp" +#include "prx/libc/include/GuestArena.hpp" +#include "prx/libc/include/GuestWriteWatch.hpp" +#include "Recompiler.hpp" +#include "Optimization/ResourceProgram.hpp" +#include "VulkanTestDevice.hpp" +#ifdef _WIN32 +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include +#else +#include +#endif +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::ColorTargetLayout; +using AgcDriver::Graphics::ColorTileMode; +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; +using Color = std::array; + +constexpr std::uint32_t Width = 256; +constexpr std::uint32_t Height = 128; +constexpr std::uint32_t ScissorWidth = 128; +constexpr std::size_t BlockBytes = 65536; +constexpr std::uint32_t Float16x4Info = 0x0730u; +constexpr std::uint32_t TiledAttrib3 = 0x4dc6c000u; +constexpr std::uint32_t Float32Export = 9; +constexpr std::uint32_t Float16Export = 4; + +alignas(256) constexpr std::array VertexCode{ + 0xe0382000, 0x80000005, 0xbf8c3f70, 0xf80008cf, 0x03020100, 0xbf810000, +}; + +constexpr std::array, 3> Triangle{{ + {-1.0f, -1.0f, 0.5f, 1.0f}, {3.0f, -1.0f, 0.5f, 1.0f}, {-1.0f, 3.0f, 0.5f, 1.0f} +}}; + +constexpr std::array Destinations{{ + {0.5f, 0.25f, 0.75f, 0.5f}, {1.0f, 0.0f, 0.125f, 0.25f}, {0.0625f, 0.875f, 0.5f, 1.0f}, {0.25f, 0.5f, 0.375f, 0.75f} +}}; + +alignas(256) std::array, 8> Programs{}; +std::size_t programCount = 0; + +std::uint16_t Half(float value) { + Require(value == 0.0f || (value >= 0.0625f && value <= 1.0f), "the test's half values are normal and at most 1"); + if (value == 0.0f) return 0; + const auto bits = std::bit_cast(value); + Require((bits & 0x1fffu) == 0, "the test's half values are exact"); + return static_cast(((((bits >> 23u) & 0xffu) - 112u) << 10u) | ((bits >> 13u) & 0x3ffu)); +} + +float FromHalf(std::uint16_t half) { + const auto exponent = (half >> 10u) & 0x1fu; + const auto mantissa = half & 0x3ffu; + Require((half & 0x8000u) == 0 && exponent != 31u, "a blended texel is negative, infinite or NaN"); + return exponent == 0 ? std::ldexp(static_cast(mantissa), -24) : std::ldexp(1.0f + static_cast(mantissa) / 1024.0f, static_cast(exponent) - 15); +} + +std::span Program(const Color& color, bool compressed) { + Require(programCount < Programs.size(), "too many test programs"); + auto& code = Programs[programCount++]; + std::size_t at = 0; + const auto move = [&](std::uint32_t vgpr, std::uint32_t value) { + code[at++] = 0x7e0002ffu | (vgpr << 17u); + code[at++] = value; + }; + if (compressed) { + move(4, Half(color[0]) | (Half(color[1]) << 16u)); + move(5, Half(color[2]) | (Half(color[3]) << 16u)); + code[at++] = 0xf8001c0fu; + code[at++] = 0x00000504u; + } else { + for (std::uint32_t channel = 0; channel < 4; ++channel) move(4 + channel, std::bit_cast(color[channel])); + code[at++] = 0xf800180fu; + code[at++] = 0x07060504u; + } + code[at++] = 0xbf810000u; + return std::span(code.data(), at); +} + +struct Block { + explicit Block(std::size_t bytes) : bytes(bytes), watched(AgcDriver::GuestMemory::WriteWatched()) { +#ifdef _WIN32 + if (watched) { + data = static_cast(GuestArena::GuestArenaAllocate_nid_postfix(bytes, BlockBytes)); + GuestArena::GuestArenaCommit_nid_postfix(data, bytes, PAGE_READWRITE, bytes); + } else { + data = static_cast(VirtualAlloc(nullptr, bytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); + } +#else + if (watched) { + void* raw = mmap(nullptr, bytes + BlockBytes, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + Require(raw != MAP_FAILED, "cannot map the color target"); + const auto begin = reinterpret_cast(raw); + const auto aligned = (begin + BlockBytes - 1) & ~(static_cast(BlockBytes) - 1); + if (aligned != begin) munmap(raw, aligned - begin); + if (aligned + bytes != begin + bytes + BlockBytes) munmap(reinterpret_cast(aligned + bytes), begin + BlockBytes - aligned); + data = reinterpret_cast(aligned); + GuestWriteWatch::GuestWriteWatchRegister_nid_postfix(data, bytes); + } else { + data = static_cast(std::aligned_alloc(BlockBytes, bytes)); + } +#endif + Require(data != nullptr, "cannot allocate the color target"); + Require(!watched || AgcDriver::GuestMemory::Watched(Address(), bytes), "the color target is not write-watched"); + GuestAllocations::Mutation mutation; + mutation.Add(data, bytes, true, true, true); + } + ~Block() { + AgcDriver::Graphics::StorageTexture::FlushPending(Address(), bytes, nullptr, "test release"); + { + GuestAllocations::Mutation mutation; + mutation.Remove(data); + } +#ifdef _WIN32 + if (watched) { + GuestArena::GuestArenaReset_nid_postfix(data, bytes); + GuestArena::GuestArenaRelease_nid_postfix(data, bytes); + } else { + VirtualFree(data, 0, MEM_RELEASE); + } +#else + if (watched) { + munmap(data, bytes); + GuestWriteWatch::GuestWriteWatchUnregister_nid_postfix(data, bytes); + } else { + std::free(data); + } +#endif + } + Block(const Block&) = delete; + Block& operator=(const Block&) = delete; + std::uint64_t Address() const { return reinterpret_cast(data); } + std::size_t bytes; + bool watched; + std::uint8_t* data = nullptr; +}; + +AgcDriver::Graphics::ColorTarget DecodeTarget(const Block& block) { + AgcDriver::Registers cx; + cx[0x318] = static_cast(block.Address() >> 8u); + cx[0x31b] = 0; + cx[0x31c] = Float16x4Info; + cx[0x31d] = 0; + cx[0x3b0] = ((Width - 1u) << 14u) | (Height - 1u); + cx[0x3b8] = TiledAttrib3; + cx[0x390] = static_cast(block.Address() >> 40u); + const auto color = AgcDriver::Graphics::DecodeColorBuffer(cx, 0); + Require(color.format == VK_FORMAT_R16G16B16A16_SFLOAT && color.elementBytes == 8 && color.bytes == block.bytes, "the 16_16_16_16 float target did not decode as a 256x128 RGBA16F surface"); + return color; +} + +void Draw(AgcDriver::VulkanDevice& device, const AgcDriver::Graphics::ColorTarget& color, bool alphaToCoverage, std::uint32_t exportFormat, std::span pixelCode) { + VkPipelineColorBlendAttachmentState blend{}; + blend.colorWriteMask = 0xf; + const auto target = device.Target(); + constexpr std::uint32_t waveSize = 64; + std::vector vertexUserData(4, 0u); + const auto address = reinterpret_cast(Triangle.data()); + const std::array vertexBuffer{static_cast(address), static_cast((address >> 32u) & 0xffffu) | (16u << 16u), static_cast(Triangle.size()), 0x01016facu}; + std::copy(vertexBuffer.begin(), vertexBuffer.end(), vertexUserData.begin()); + const std::array vertexMemory{{{reinterpret_cast(VertexCode.data()), std::as_bytes(std::span(VertexCode))}}}; + ShaderRecompiler::RecompileRequest vertex{ + {ShaderStage::Vertex, reinterpret_cast(VertexCode.data()), VertexCode, 0, {}}, + {waveSize, 0, vertexUserData, std::nullopt, std::nullopt, ShaderRecompiler::ShaderVertexStageInfo{}, vertexMemory}, + target, + {0, 0, 0, 64} + }; + vertex.useCache = false; + const auto vertexResult = ShaderRecompiler::Recompile(vertex); + const auto vertexPush = static_cast(vertexResult.pushConstants.size()); + + AgcDriver::Graphics::State state{}; + state.stages = {AgcDriver::Graphics::ShaderPath::Vertex, 0u, waveSize, waveSize, std::nullopt, std::nullopt}; + state.colors = {color}; + state.color = color; + state.hasColorTarget = true; + state.renderExtent = {Width, Height}; + state.topology = VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST; + state.viewport = {0, static_cast(Height), static_cast(Width), -static_cast(Height), 0, 1}; + state.negativeOneToOne = false; + state.scissor = {{0, 0}, {ScissorWidth, Height}}; + state.cullMode = VK_CULL_MODE_NONE; + state.frontFace = VK_FRONT_FACE_COUNTER_CLOCKWISE; + state.blends = {blend}; + state.blend = blend; + state.blendConstants = {}; + state.alphaToCoverage = alphaToCoverage; + + const auto codeAddress = reinterpret_cast(pixelCode.data()); + const AgcDriver::Registers context{{0x1b6, 0x8000}, {0x1b3, 0}, {0x1b4, 0}, {0x203, 0}, {0x1c5, exportFormat}}; + const std::array pixelMemory{{{codeAddress, std::as_bytes(pixelCode)}}}; + auto prepared = AgcDriver::Graphics::DecodePixelStageInfo(context, {}); + prepared.alphaToCoverage = alphaToCoverage; + ShaderRecompiler::RecompileRequest fragment{ + {ShaderStage::Fragment, codeAddress, pixelCode, 0, {}}, + {waveSize, 0, {}, std::nullopt, prepared, std::nullopt, pixelMemory}, + target, + PixelPushLayout(vertexPush, target) + }; + const auto handle = ShaderRecompiler::PrepareShader(fragment); + AgcDriver::DriverDetail::ShaderSnapshot snapshot{codeAddress, 0, 1, {pixelCode.begin(), pixelCode.end()}, {}}; + snapshot.prepared->entries.push_back({0, handle}); + fragment.shader.code = snapshot.code; + fragment.context.pixel = AgcDriver::Graphics::DecodePixelStageInfo(context, AgcDriver::Graphics::ExportMappings(state)); + fragment.context.pixel->alphaToCoverage = state.alphaToCoverage; + const ShaderRecompiler::SrtRuntime runtime{}; + const auto materialize = [&](const ShaderRecompiler::RecompileRequest& request) { + const auto invocation = AgcDriver::DriverDetail::InvocationFor(snapshot, 0, request); + return invocation.Materialize(*invocation.Capture(runtime)); + }; + const auto pixelResult = materialize(fragment); + Require(pixelResult->variantId == ShaderRecompiler::GetPreparedArtifact(*handle).variantId, "the draw did not use the registered pixel shader"); + const std::array shaders{{ + {ShaderStage::Vertex, &vertexResult, 0}, + {ShaderStage::Fragment, pixelResult.get(), PixelPushOffset(vertexPush, target)} + }}; + const AgcDriver::Pm4::DrawParameters draw{0, static_cast(Triangle.size()), 0, 1, 0, false}; + device.Draw(state, draw, shaders); +} + +std::vector ReadBack(AgcDriver::VulkanDevice& device, const Block& block) { + AgcDriver::Graphics::StorageTexture::FlushPending(block.Address(), block.bytes, nullptr, "test read-back"); + device.WaitIdle(); + std::vector stored(block.bytes / 2u); + AgcDriver::GuestMemory::Read(block.Address(), std::as_writable_bytes(std::span(stored)), 1); + return stored; +} + +std::vector Fill(Block& block, const ColorTargetLayout& layout) { + std::vector halves(block.bytes / 2u); + for (std::uint32_t y = 0; y < Height; ++y) { + for (std::uint32_t x = 0; x < Width; ++x) { + const auto& color = Destinations[(x + y) % Destinations.size()]; + const auto index = layout.Offset(x, y) / 2u; + for (std::uint32_t channel = 0; channel < 4; ++channel) halves[index + channel] = Half(color[channel]); + } + } + std::memcpy(block.data, halves.data(), block.bytes); + return halves; +} + +void Check(AgcDriver::VulkanDevice& device, const Block& block, const ColorTargetLayout& layout, const std::vector& before, const std::function& expectedColor, const std::string& what) { + const auto stored = ReadBack(device, block); + for (std::uint32_t y = 0; y < Height; ++y) { + for (std::uint32_t x = 0; x < Width; ++x) { + const auto index = layout.Offset(x, y) / 2u; + const auto expected = expectedColor(Destinations[(x + y) % Destinations.size()]); + for (std::uint32_t channel = 0; channel < 3; ++channel) { + const bool inside = x < ScissorWidth; + const bool match = inside ? std::fabs(FromHalf(stored[index + channel]) - expected[channel]) <= 1.0f / 2048.0f : stored[index + channel] == before[index + channel]; + if (match) continue; + char detail[160]; + std::snprintf(detail, sizeof(detail), ": channel %u of pixel (%u, %u) %s the scissor holds half 0x%04x, expected %.6g", channel, x, y, inside ? "inside" : "outside", stored[index + channel], inside ? expected[channel] : FromHalf(before[index + channel])); + throw std::runtime_error(what + detail); + } + } + } +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + const ColorTargetLayout layout(Width, Height, ColorTileMode::RenderTarget, 8); + Block block(layout.Bytes()); + struct Case { + std::uint32_t alpha; + bool alphaToCoverage; + bool written; + }; + constexpr std::array cases{{ + {0x00000000u, true, false}, {0x80000000u, true, false}, {0x3e800000u, true, false}, {0x3efffffeu, true, false}, + {0x3effffffu, true, false}, {0x3f000000u, true, true}, {0x3f000001u, true, true}, {0x3f000002u, true, true}, + {0x3f400000u, true, true}, {0x3f800000u, true, true}, {0x40000000u, true, true}, {0xbf800000u, true, false}, + {0x7f800000u, true, true}, {0xff800000u, true, false}, {0x7fc00000u, true, false}, {0xffc00000u, true, false}, + {0x3e800000u, false, true}, + }}; + for (const auto& test : cases) { + const Color source{0.25f, 0.5f, 0.125f, std::bit_cast(test.alpha)}; + const auto before = Fill(block, layout); + Draw(*device, DecodeTarget(block), test.alphaToCoverage, Float32Export, Program(source, false)); + char name[96]; + std::snprintf(name, sizeof(name), "alpha 0x%08x with alpha-to-coverage %s", test.alpha, test.alphaToCoverage ? "on" : "off"); + Check(*device, block, layout, before, [&](const Color& destination) { return test.written ? source : destination; }, name); + programCount = 0; + } + for (const Case test : {Case{0x3f400000u, true, true}, Case{0x3e800000u, true, false}}) { + const Color source{0.25f, 0.5f, 0.125f, std::bit_cast(test.alpha)}; + const auto before = Fill(block, layout); + Draw(*device, DecodeTarget(block), true, Float16Export, Program(source, true)); + Check(*device, block, layout, before, [&](const Color& destination) { return test.written ? source : destination; }, test.written ? "alpha 0.75 with FP16_ABGR exports" : "alpha 0.25 with FP16_ABGR exports"); + programCount = 0; + } + std::puts("alpha-to-coverage tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} From eb2c43a2ebbc237d15e8785dda877a6026cc6c4e Mon Sep 17 00:00:00 2001 From: fma-rc <188743521+fma-rc@users.noreply.github.com> Date: Sun, 11 Oct 2026 03:54:50 +0200 Subject: [PATCH 305/460] fix(shader): sample a 1D T# through a 2D implicit-LOD image_sample A plain image_sample that says DIM 2D on a 1D T# takes its texel and its LOD from the first coordinate alone, measured on RDNA2 (gfx1036) with the hardware oracle. Astro's Playroom draws with such a pair. --- .../ImagePlaneAddressOnLineImplicitLod.cpp | 196 ++++++++++++++++++ .../Optimization/src/ResourceTracker.cpp | 2 +- docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 198 insertions(+), 2 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/ImagePlaneAddressOnLineImplicitLod.cpp diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/ImagePlaneAddressOnLineImplicitLod.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/ImagePlaneAddressOnLineImplicitLod.cpp new file mode 100644 index 000000000..cc39b8471 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/ImagePlaneAddressOnLineImplicitLod.cpp @@ -0,0 +1,196 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "prx/libSceAgcDriver/Graphics/include/TextureTiling.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Width = 64; +constexpr std::uint32_t Height = 16; +constexpr std::uint32_t Size = 16; +constexpr std::uint32_t Mips = 5; +constexpr std::uint32_t Format8888UNorm = 56; +constexpr std::uint32_t Type1D = 8; +constexpr float SecondCoordinateScale = 3.0f; +alignas(256) std::array Pixels{}; +alignas(256) std::array Line{}; + +alignas(256) constexpr std::array VertexCode{ + 0xe0382000, 0x80000005, 0xbf8c3f70, 0xf80008cf, 0x03020100, 0xbf810000, +}; + +alignas(256) constexpr std::array PixelCode{ + 0x10040010, 0x06060300, 0x10060611, 0xf0800f08, 0x00610402, 0xbf8c3f70, 0xf800180f, 0x07060504, 0xbf810000, +}; + +alignas(256) constexpr std::array BiasPixelCode{ + 0x10040010, 0x06060300, 0x10060611, 0xf0940f08, 0x00610401, 0xbf8c3f70, 0xf800180f, 0x07060504, 0xbf810000, +}; + +constexpr std::array, 3> Triangle{{{-1.0f, -1.0f, 0.5f, 1.0f}, {3.0f, -1.0f, 0.5f, 1.0f}, {-1.0f, 3.0f, 0.5f, 1.0f}}}; + +std::array BufferDescriptor(const void* data, std::uint32_t stride, std::uint32_t count) { + const auto address = reinterpret_cast(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu) | (stride << 16u), count, 0x01016facu}; +} + +std::array LineDescriptor() { + const auto address = static_cast(reinterpret_cast(Line.data())); + return { + static_cast(address >> 8u), + static_cast((address >> 40u) & 0xffu) | (Format8888UNorm << 20u) | (((Size - 1u) & 3u) << 30u), + (Size - 1u) >> 2u, + 0xfacu | ((Mips - 1u) << 16u) | (Type1D << 28u), + 0u, + (Mips - 1u) << 4u, + 0u, + 0u, + }; +} + +std::array SamplerDescriptor() { + return {0u, 0xfffu << 12u, 1u << 26u, 0u}; +} + +std::uint8_t LevelMark(std::uint32_t level) { + return static_cast(10u + 40u * level); +} + +void FillTexture() { + const auto geometry = AgcDriver::Graphics::DescribeSurface(AgcDriver::Graphics::DecodeTextureResource(LineDescriptor())); + Require(geometry.guestBytes <= Line.size(), "implicit LOD through a 1D texture: the texture does not fit the texel storage"); + Require(geometry.mips.size() == Mips, "implicit LOD through a 1D texture: the texture has " + std::to_string(geometry.mips.size()) + " mips"); + std::fill(Line.begin(), Line.end(), 0xeeu); + for (std::uint32_t level = 0; level < Mips; ++level) { + const auto& mip = geometry.mips[level]; + for (std::uint32_t x = 0; x < (Size >> level); ++x) { + auto* texel = &Line[geometry.GuestLayerOffset(0) + mip.tiledOffset + x * 4u]; + texel[0] = LevelMark(level); + texel[1] = static_cast(x * 8u); + texel[2] = 99u; + texel[3] = 255u; + } + } +} + +void Draw(AgcDriver::VulkanDevice& device, std::span pixelCode, float firstCoordinateScale) { + const auto& target = device.Target(); + Pixels.fill(std::byte{0}); + + std::vector vertexUserData(4, 0u); + const auto vertexBuffer = BufferDescriptor(Triangle.data(), 16u, static_cast(Triangle.size())); + std::copy(vertexBuffer.begin(), vertexBuffer.end(), vertexUserData.begin()); + const std::array vertexMemory{{{reinterpret_cast(VertexCode.data()), std::as_bytes(std::span(VertexCode))}}}; + ShaderRecompiler::RecompileRequest vertex{ + {ShaderStage::Vertex, reinterpret_cast(VertexCode.data()), VertexCode, 0, {}}, + {64, 0, vertexUserData, std::nullopt, std::nullopt, ShaderRecompiler::ShaderVertexStageInfo{}, vertexMemory}, + target, + {0, 0, 0, 64} + }; + vertex.useCache = false; + const auto vertexResult = ShaderRecompiler::Recompile(vertex); + const auto vertexPush = static_cast(vertexResult.pushConstants.size()); + + ShaderRecompiler::ShaderPixelStageInfo pixel{}; + pixel.wave32 = false; + pixel.inputAddr = ShaderRecompiler::PixelInputBit(ShaderRecompiler::PixelInput::PositionX) | ShaderRecompiler::PixelInputBit(ShaderRecompiler::PixelInput::PositionY); + pixel.posX = true; + pixel.posY = true; + pixel.targetOutputMode[0] = 9; + pixel.targetExportMapping.fill(0xe4u); + std::vector pixelUserData(18, 0u); + const auto texture = LineDescriptor(); + const auto sampler = SamplerDescriptor(); + std::copy(texture.begin(), texture.end(), pixelUserData.begin() + 4); + std::copy(sampler.begin(), sampler.end(), pixelUserData.begin() + 12); + pixelUserData[16] = std::bit_cast(firstCoordinateScale); + pixelUserData[17] = std::bit_cast(SecondCoordinateScale); + const std::array pixelMemory{{{reinterpret_cast(pixelCode.data()), std::as_bytes(pixelCode)}}}; + ShaderRecompiler::RecompileRequest fragment{ + {ShaderStage::Fragment, reinterpret_cast(pixelCode.data()), pixelCode, 0, {}}, + {64, 0, pixelUserData, std::nullopt, pixel, std::nullopt, pixelMemory}, + target, + PixelPushLayout(vertexPush, target) + }; + fragment.useCache = false; + const auto pixelResult = ShaderRecompiler::Recompile(fragment); + const std::array shaders{{ + {ShaderStage::Vertex, &vertexResult, 0}, + {ShaderStage::Fragment, &pixelResult, PixelPushOffset(vertexPush, target)} + }}; + + AgcDriver::Graphics::State state{}; + state.stages = {AgcDriver::Graphics::ShaderPath::Vertex, 0u, 64u, 64u, std::nullopt, std::nullopt}; + state.color = {reinterpret_cast(Pixels.data()), {Width, Height}, VK_FORMAT_R8G8B8A8_UNORM, Pixels.size(), 0xe4u}; + state.colors = {state.color}; + state.hasColorTarget = true; + state.renderExtent = {Width, Height}; + state.topology = VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST; + state.viewport = {0, static_cast(Height), static_cast(Width), -static_cast(Height), 0, 1}; + state.negativeOneToOne = false; + state.scissor = {{0, 0}, {Width, Height}}; + state.cullMode = VK_CULL_MODE_NONE; + state.frontFace = VK_FRONT_FACE_COUNTER_CLOCKWISE; + state.blend.colorWriteMask = 15; + state.blends = {state.blend}; + state.blendConstants = {}; + const AgcDriver::Pm4::DrawParameters draw{0, static_cast(Triangle.size()), 0, 1, 0, false}; + device.Draw(state, draw, shaders); + device.WaitIdle(); +} + +void Check(std::uint32_t level) { + const auto what = "image_sample 2d on a 1D texture with the first coordinate advancing " + std::to_string(1u << level) + " texels per pixel"; + for (std::uint32_t y = 0; y < Height; ++y) { + for (std::uint32_t x = 0; x < Width; ++x) { + const std::array expected{LevelMark(level), (x % (Size >> level)) * 8u, 99u, 255u}; + for (std::uint32_t component = 0; component < 4u; ++component) { + const auto value = std::to_integer(Pixels[(y * Width + x) * 4u + component]); + Require(value == expected[component], what + ": pixel " + std::to_string(x) + "," + std::to_string(y) + " component " + std::to_string(component) + " is " + std::to_string(value) + ", expected " + std::to_string(expected[component])); + } + } + } +} + +void RequireRefused(AgcDriver::VulkanDevice& device) { + std::string refusal; + try { + Draw(device, BiasPixelCode, 1.0f / static_cast(Size)); + } catch (const std::exception& error) { + refusal = error.what(); + } + Require(refusal.find("incompatible with the static runtime image interface") != std::string::npos, "image_sample_b 2d on a 1D texture was not refused: " + refusal); +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + FillTexture(); + for (std::uint32_t level = 0; level < 3u; ++level) { + Draw(*device, PixelCode, static_cast(1u << level) / static_cast(Size)); + Check(level); + } + RequireRefused(*device); + std::puts("image plane address on line implicit LOD tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/Optimization/src/ResourceTracker.cpp b/core/shader/recompiler/Optimization/src/ResourceTracker.cpp index 20be15285..16807e7d1 100644 --- a/core/shader/recompiler/Optimization/src/ResourceTracker.cpp +++ b/core/shader/recompiler/Optimization/src/ResourceTracker.cpp @@ -1108,7 +1108,7 @@ private: const bool flatVolumeSample = op == IrOpcode::ImageSampleRaw && memory.imageSampleFlags == RdnaImageSampleFlagLevelZero; image.flatVolumeCompatible = image.flatVolumeCompatible && (flatVolumeLoad || flatVolumeSample) && !memory.imagePacked && memory.imageByElements == 0u; const bool flatLineLoad = op == IrOpcode::ImageRead && !memory.imageHasMip && memory.imageSampleFlags == 0u; - const bool flatLineSample = op == IrOpcode::ImageSampleRaw && memory.imageSampleFlags == RdnaImageSampleFlagLevelZero; + const bool flatLineSample = op == IrOpcode::ImageSampleRaw && (memory.imageSampleFlags == RdnaImageSampleFlagLevelZero || memory.imageSampleFlags == 0u); image.flatLineCompatible = image.flatLineCompatible && (flatLineLoad || flatLineSample) && !memory.imagePacked && memory.imageByElements == 0u; if ((memory.imageSampleFlags & RdnaImageSampleFlagCompare) != 0u) { constexpr auto unsupported = RdnaImageSampleFlagLod | RdnaImageSampleFlagDerivative; diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 237d82bed..44b985f26 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -381,7 +381,7 @@ Throughout the project, every function at every stage either **does exactly what - A value read through a null or unmapped pointer still throws at record time if a [branch condition, uniform fill or table base](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp) needs it, or if the pointer comes from user data and is not null, even if the console skips the read. - A [V# or flat SRT word](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtEvaluator.cpp) read through a null pointer taken from user data is zero and counts as a [poisoned SRT read](../../core/shader/recompiler/Recompiler.cpp), so a dispatch that reads it is not cached. The console faults if the program executes the read; PPSA28416 branches around it with `s_cbranch_execz`. The fault is not modeled. - A [V#, T# or S#](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp) loaded through a null or unmapped pointer read from memory is zero when the work is recorded, and the capture throws if no flat SRT slot records that read. -- [2D accesses through a 1D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) read the texel the 1D addressing selects and ignore the second coordinate, as simonbbby measured on RDNA2 (gfx1036) in #2237 for `image_sample_lz` and mip-less `image_load` with point filtering and wrap, clamp and border addressing; the PS5's gfx1013 is not measured. Other 2D accesses through a 1D T# (storage, depth, packed, BY loads, mips, derivatives, comparison) still throw. +- [2D accesses through a 1D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) read the texel the 1D addressing selects and ignore the second coordinate, as simonbbby measured on RDNA2 (gfx1036) in #2237 for `image_sample_lz` and mip-less `image_load` with point filtering and wrap, clamp and border addressing; the PS5's gfx1013 is not measured. A plain `image_sample` (implicit LOD) through such a pair takes its LOD from the derivatives of the first coordinate alone, measured on RDNA2 (gfx1036) with the [hardware oracle](HW_ORACLE.md) on the quads of a compute kernel (wave32 and wave64, IEEE off, DX10 clamp on, denormals preserved, round to nearest even): on a 16-texel, 5-mip 32_UINT T# with point filters it picks the same mip and texel as DIM 1D for first-coordinate steps of 0 to 8 texels per lane in either direction, whatever the second coordinate does (steps up to 16 per lane), while the same rows on a 16x16 2D T# move the mip with the second coordinate. Other 2D accesses through a 1D T# (storage, depth, packed, BY loads, mips, explicit LOD, bias, offsets, derivatives, comparison) still throw. - [Emulated comparison of a 2D-array access through a 2D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) samples the 2D image and ignores the slice, as the native modes of a 2D-array access through a 2D T# already do. This follows the T# type deciding the addressing; it is not measured on hardware for comparison samples - [Scalar aperture base sources](../../core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp) (core/shader/recompiler) - `src_shared_base`, `src_shared_limit`, `src_private_base` and `src_private_limit` (scalar source codes 235 to 238) are not modelled: decoding throws `unsupported scalar source operand code` with the code and the program counter, so a shader that reads one does not compile. Operand 235 (`src_shared_base`) was reported on [#1182](https://github.com/boykopovar/AnyPS5/pull/1182) from PPSA26344; the flat aperture routing does not read the base, so a shader that builds its own flat addresses from it stops at decode. - [Flat aperture bases](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (core/shader/recompiler) - the layout follows the gfx9+ one in the AMD KFD driver's `kfd_flat_memory.c` (`MAKE_LDS_APP_BASE_V9`, `MAKE_LDS_APP_LIMIT`): each aperture is a 4 GB window whose base is the high dword, so a flat address is routed by its whole high dword, as measured on RDNA2 (gfx1035: `src_shared_base` 0x00010000_00000000, `src_shared_limit` 0x00010000_ffffffff). The PS5's base values, high dwords 0x80000000 (shared, `SharedApertureHigh`) and 0x70000000 (private, `PrivateApertureHigh`), are inferred from GTA V's traversal addresses, not measured on a console or read from SH_MEM_BASES From 1ce9790481171e0e7f9f5597f068f8876b23acd5 Mon Sep 17 00:00:00 2001 From: krishnaraya Date: Fri, 9 Oct 2026 13:31:08 -0400 Subject: [PATCH 306/460] fix(agc): pick float or integer sampler border colours from the paired images Co-Authored-By: Claude Opus 5.5 (1M context) --- .../Graphics/include/GuestSamplerResource.hpp | 2 +- .../Graphics/include/Sampler.hpp | 2 +- .../Graphics/include/ShaderResources.hpp | 1 + .../Graphics/src/GuestSamplerResource.cpp | 10 +- .../libSceAgcDriver/Graphics/src/Sampler.cpp | 6 +- .../Graphics/src/ShaderResources.cpp | 12 +- .../tests/GuestSamplerResource.cpp | 13 +- .../libSceAgcDriver/tests/ShaderDiskCache.cpp | 3 +- .../execution/ImageSampleBorderColor.cpp | 114 ++++++++++++++++++ .../src/DescriptorBindingBuilder.cpp | 13 +- core/shader/recompiler/Recompiler.hpp | 1 + core/shader/recompiler/ShaderDiskCache.cpp | 2 + core/shader/recompiler/ShaderDiskCache.hpp | 2 +- 13 files changed, 157 insertions(+), 24 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/ImageSampleBorderColor.cpp diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestSamplerResource.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestSamplerResource.hpp index 74e86e866..55442662b 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestSamplerResource.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestSamplerResource.hpp @@ -33,7 +33,7 @@ struct GuestSamplerResource { bool nonSeamlessCube = false; }; -GuestSamplerResource DecodeSamplerResource(std::span words, bool unnormalizedProven = false, bool forceDegammaPaired = false); +GuestSamplerResource DecodeSamplerResource(std::span words, bool unnormalizedProven = false, bool forceDegammaPaired = false, bool integerBorder = false); std::optional> SingleLevelSamplerWords(std::span words, bool singleLevelImage, bool mipmappedImage); } diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/Sampler.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/Sampler.hpp index a1f894545..76804ccfc 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/Sampler.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/Sampler.hpp @@ -47,7 +47,7 @@ public: explicit SamplerCache(std::size_t capacity = 1024); SamplerCache(const SamplerCache&) = delete; SamplerCache& operator=(const SamplerCache&) = delete; - std::shared_ptr Get(const Context& context, std::span words, bool compareEnable, bool unnormalizedProven = false, bool forceDegammaPaired = false); + std::shared_ptr Get(const Context& context, std::span words, bool compareEnable, bool unnormalizedProven = false, bool forceDegammaPaired = false, bool integerBorder = false); // APS5_PROFILE_DRAW counters: lookups served by an existing sampler, and samplers created. std::uint64_t Hits() const { return hits; } std::uint64_t Misses() const { return misses; } diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp index ee1a0a5c5..4cd044a8d 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp @@ -444,6 +444,7 @@ private: std::array words{}; bool compareEnable = false; bool unnormalized = false; + bool integerBorder = false; bool singleLevelImage = false; bool mipmappedImage = false; }; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestSamplerResource.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestSamplerResource.cpp index 86713d404..87a8c37b6 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestSamplerResource.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestSamplerResource.cpp @@ -54,7 +54,7 @@ float toSignedLodBias(std::uint32_t raw) { } -GuestSamplerResource DecodeSamplerResource(std::span words, bool unnormalizedProven, bool forceDegammaPaired) { +GuestSamplerResource DecodeSamplerResource(std::span words, bool unnormalizedProven, bool forceDegammaPaired, bool integerBorder) { Require(words.size() == 4, "guest sampler descriptor must contain 4 dwords"); const auto clampX = (words[0] >> 0u) & 0x7u; @@ -135,12 +135,12 @@ GuestSamplerResource DecodeSamplerResource(std::span words, VkBorderColor border; switch (borderColorType) { - case 0: border = VK_BORDER_COLOR_INT_TRANSPARENT_BLACK; break; - case 1: border = VK_BORDER_COLOR_INT_OPAQUE_BLACK; break; - case 2: border = VK_BORDER_COLOR_INT_OPAQUE_WHITE; break; + case 0: border = integerBorder ? VK_BORDER_COLOR_INT_TRANSPARENT_BLACK : VK_BORDER_COLOR_FLOAT_TRANSPARENT_BLACK; break; + case 1: border = integerBorder ? VK_BORDER_COLOR_INT_OPAQUE_BLACK : VK_BORDER_COLOR_FLOAT_OPAQUE_BLACK; break; + case 2: border = integerBorder ? VK_BORDER_COLOR_INT_OPAQUE_WHITE : VK_BORDER_COLOR_FLOAT_OPAQUE_WHITE; break; default: if (readsBorderColor(clampX) || readsBorderColor(clampY) || readsBorderColor(clampZ)) throw std::runtime_error("AGC graphics: guest sampler descriptor uses a border color table which is not implemented"); - border = VK_BORDER_COLOR_INT_TRANSPARENT_BLACK; + border = integerBorder ? VK_BORDER_COLOR_INT_TRANSPARENT_BLACK : VK_BORDER_COLOR_FLOAT_TRANSPARENT_BLACK; break; } diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Sampler.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Sampler.cpp index 1e8dcd657..3bf2f9556 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Sampler.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Sampler.cpp @@ -67,9 +67,9 @@ namespace AgcDriver::Graphics { SamplerCache::SamplerCache(std::size_t capacity) : capacity(std::max(capacity, 1)) {} - std::shared_ptr SamplerCache::Get(const Context& context, std::span words, bool compareEnable, bool unnormalizedProven, bool forceDegammaPaired) { + std::shared_ptr SamplerCache::Get(const Context& context, std::span words, bool compareEnable, bool unnormalizedProven, bool forceDegammaPaired, bool integerBorder) { Require(words.size() == 4, "guest sampler descriptor must contain 4 dwords"); - const std::array key{words[0], words[1], words[2], words[3], (compareEnable ? 1u : 0u) | (unnormalizedProven ? 2u : 0u)}; + const std::array key{words[0], words[1], words[2], words[3], (compareEnable ? 1u : 0u) | (unnormalizedProven ? 2u : 0u) | (integerBorder ? 4u : 0u)}; std::lock_guard lock(mutex); ++clock; if (const auto found = entries.find(key); found != entries.end()) { @@ -79,7 +79,7 @@ namespace AgcDriver::Graphics { return found->second.sampler; } ++misses; - auto resource = DecodeSamplerResource(words, unnormalizedProven, forceDegammaPaired); + auto resource = DecodeSamplerResource(words, unnormalizedProven, forceDegammaPaired, integerBorder); resource.compareEnable = compareEnable; auto sampler = std::make_shared(context, resource); // The cap keeps live samplers well below the device's limit (NVIDIA: ~4000); a set in flight diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index 9ac9ff9bd..21258a371 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -1643,6 +1643,7 @@ void VisitContentKey(const CompiledShader& shader, bool dataWords, bool movableB packBits(binding.imageDepthCompare); packBits(binding.imageAtomic); packBits(binding.samplerUnnormalized); + packBits(binding.samplerIntegerBorder); packBits(binding.imageUnnormalized); // Read-only elements are bound without a write set: an object built for one written set // must not serve a build with another (the variant implies it, this makes it explicit). @@ -2908,14 +2909,15 @@ void ShaderResources::addImageBinding(const ShaderRecompiler::DescriptorBinding& const auto words = std::span(binding.guestDescriptor).subspan(static_cast(element) * elementWords, elementWords); const bool compareEnable = binding.samplerDepthCompare.at(element); const bool unnormalized = element < binding.samplerUnnormalized.size() && binding.samplerUnnormalized[element]; + const bool integerBorder = element < binding.samplerIntegerBorder.size() && binding.samplerIntegerBorder[element]; if (context.samplerCache != nullptr && !SamplerCacheDisabled()) { - samplers.push_back(context.samplerCache->Get(context, words, compareEnable, unnormalized, true)); + samplers.push_back(context.samplerCache->Get(context, words, compareEnable, unnormalized, true, integerBorder)); } else { - auto resource = DecodeSamplerResource(words, unnormalized, true); + auto resource = DecodeSamplerResource(words, unnormalized, true, integerBorder); resource.compareEnable = compareEnable; samplers.push_back(std::make_shared(context, resource)); } - samplerSources.push_back({{words[0], words[1], words[2], words[3]}, compareEnable, unnormalized}); + samplerSources.push_back({{words[0], words[1], words[2], words[3]}, compareEnable, unnormalized, integerBorder}); item.imageAllocations.push_back(samplers.size() - 1); } Require(samplers.size() <= context.limits.maxDescriptorSetSamplers, "pipeline sampler descriptors exceed device limits"); @@ -3073,9 +3075,9 @@ void ShaderResources::applyAnisoOverride() { if (!variant.has_value()) continue; const auto& words = *variant; if (context.samplerCache != nullptr && !SamplerCacheDisabled()) { - samplers[index] = context.samplerCache->Get(context, words, source.compareEnable, source.unnormalized); + samplers[index] = context.samplerCache->Get(context, words, source.compareEnable, source.unnormalized, false, source.integerBorder); } else { - auto resource = DecodeSamplerResource(words, source.unnormalized); + auto resource = DecodeSamplerResource(words, source.unnormalized, false, source.integerBorder); resource.compareEnable = source.compareEnable; samplers[index] = std::make_shared(context, resource); } diff --git a/core/libs/prx/libSceAgcDriver/tests/GuestSamplerResource.cpp b/core/libs/prx/libSceAgcDriver/tests/GuestSamplerResource.cpp index b266d39a4..ecf28b1e9 100644 --- a/core/libs/prx/libSceAgcDriver/tests/GuestSamplerResource.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/GuestSamplerResource.cpp @@ -288,7 +288,7 @@ void RunGuestSamplerResourceTests() { Require(result.addressModeU == VK_SAMPLER_ADDRESS_MODE_CLAMP_TO_EDGE, "clamp mode 2 must decode to clamp-to-edge"); Require(!result.anisotropyEnable && result.maxAnisotropy == 1.0f, "non-anisotropic filter must leave anisotropy disabled"); Require(nearlyEqual(result.maxLod, 12.0f), "max LOD decoded incorrectly"); - Require(result.borderColor == VK_BORDER_COLOR_INT_TRANSPARENT_BLACK, "border color type 0 must decode to transparent black"); + Require(result.borderColor == VK_BORDER_COLOR_FLOAT_TRANSPARENT_BLACK, "border color type 0 must decode to transparent black"); Fields nearest = base; nearest.xyMagFilter = 0; @@ -360,7 +360,7 @@ void RunGuestSamplerResourceTests() { requireUnnormalized(unnormalized, "captured unnormalized S#"); Require(unnormalized.magFilter == VK_FILTER_LINEAR && unnormalized.minFilter == VK_FILTER_LINEAR, "captured unnormalized S# filters decoded incorrectly"); Require(unnormalized.addressModeU == VK_SAMPLER_ADDRESS_MODE_CLAMP_TO_EDGE && unnormalized.addressModeV == VK_SAMPLER_ADDRESS_MODE_CLAMP_TO_EDGE, "captured unnormalized S# clamp modes decoded incorrectly"); - Require(unnormalized.borderColor == VK_BORDER_COLOR_INT_TRANSPARENT_BLACK, "captured unnormalized S# border color decoded incorrectly"); + Require(unnormalized.borderColor == VK_BORDER_COLOR_FLOAT_TRANSPARENT_BLACK, "captured unnormalized S# border color decoded incorrectly"); Require(!DecodeSamplerResource(pack(base)).unnormalizedCoordinates, "a normalized S# decoded to unnormalized coordinates"); Fields unnormalizedBase = base; @@ -371,7 +371,7 @@ void RunGuestSamplerResourceTests() { const auto point = DecodeSamplerResource(pack(unnormalizedPoint), true); requireUnnormalized(point, "unnormalized point S#"); Require(point.magFilter == VK_FILTER_NEAREST && point.minFilter == VK_FILTER_NEAREST, "unnormalized point S# filters decoded incorrectly"); - const std::array borders{VK_BORDER_COLOR_INT_TRANSPARENT_BLACK, VK_BORDER_COLOR_INT_OPAQUE_BLACK, VK_BORDER_COLOR_INT_OPAQUE_WHITE}; + const std::array borders{VK_BORDER_COLOR_FLOAT_TRANSPARENT_BLACK, VK_BORDER_COLOR_FLOAT_OPAQUE_BLACK, VK_BORDER_COLOR_FLOAT_OPAQUE_WHITE}; for (std::uint32_t type = 0; type < borders.size(); ++type) { Fields border = unnormalizedBase; border.clampX = 6; @@ -504,10 +504,13 @@ void RunGuestSamplerResourceTests() { Fields opaqueBlack = base; opaqueBlack.borderColorType = 1; - Require(DecodeSamplerResource(pack(opaqueBlack)).borderColor == VK_BORDER_COLOR_INT_OPAQUE_BLACK, "border color type 1 must decode to opaque black"); + Require(DecodeSamplerResource(pack(opaqueBlack)).borderColor == VK_BORDER_COLOR_FLOAT_OPAQUE_BLACK, "border color type 1 must decode to opaque black"); + Require(DecodeSamplerResource(pack(opaqueBlack), false, false, true).borderColor == VK_BORDER_COLOR_INT_OPAQUE_BLACK, "border color type 1 of a sampler for integer images must decode to integer opaque black"); Fields opaqueWhite = base; opaqueWhite.borderColorType = 2; - Require(DecodeSamplerResource(pack(opaqueWhite)).borderColor == VK_BORDER_COLOR_INT_OPAQUE_WHITE, "border color type 2 must decode to opaque white"); + Require(DecodeSamplerResource(pack(opaqueWhite)).borderColor == VK_BORDER_COLOR_FLOAT_OPAQUE_WHITE, "border color type 2 must decode to opaque white"); + Require(DecodeSamplerResource(pack(opaqueWhite), false, false, true).borderColor == VK_BORDER_COLOR_INT_OPAQUE_WHITE, "border color type 2 of a sampler for integer images must decode to integer opaque white"); + Require(DecodeSamplerResource(pack(base), false, false, true).borderColor == VK_BORDER_COLOR_INT_TRANSPARENT_BLACK, "border color type 0 of a sampler for integer images must decode to integer transparent black"); Fields positiveBias = base; positiveBias.lodBiasRaw = 256; diff --git a/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp b/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp index 0aeb9425e..119291ba5 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp @@ -41,7 +41,7 @@ void setEnvironment(const char* name, const std::string& value) { } bool sameBinding(const DescriptorBinding& left, const DescriptorBinding& right) { - return left.kind == right.kind && left.role == right.role && left.descriptorSet == right.descriptorSet && left.binding == right.binding && left.count == right.count && left.guestDescriptor == right.guestDescriptor && left.readOnly == right.readOnly && left.imageShape == right.imageShape && left.samplerDepthCompare == right.samplerDepthCompare && left.imageWritten == right.imageWritten && left.imageDepthCompare == right.imageDepthCompare && left.imageAtomic == right.imageAtomic && left.bufferAtomic == right.bufferAtomic && left.bufferWritten == right.bufferWritten && left.bufferRead == right.bufferRead && left.samplerUnnormalized == right.samplerUnnormalized && left.imageUnnormalized == right.imageUnnormalized && left.imageSamplers == right.imageSamplers; + return left.kind == right.kind && left.role == right.role && left.descriptorSet == right.descriptorSet && left.binding == right.binding && left.count == right.count && left.guestDescriptor == right.guestDescriptor && left.readOnly == right.readOnly && left.imageShape == right.imageShape && left.samplerDepthCompare == right.samplerDepthCompare && left.imageWritten == right.imageWritten && left.imageDepthCompare == right.imageDepthCompare && left.imageAtomic == right.imageAtomic && left.bufferAtomic == right.bufferAtomic && left.bufferWritten == right.bufferWritten && left.bufferRead == right.bufferRead && left.samplerUnnormalized == right.samplerUnnormalized && left.samplerIntegerBorder == right.samplerIntegerBorder && left.imageUnnormalized == right.imageUnnormalized && left.imageSamplers == right.imageSamplers; } bool sameBindings(const std::vector& left, const std::vector& right) { @@ -119,6 +119,7 @@ DescriptorBinding sampleBinding(std::uint32_t seed) { binding.bufferAtomic = {true}; binding.bufferWritten = {false, false, true, true, false}; binding.samplerUnnormalized = {false, true, true}; + binding.samplerIntegerBorder = {true, false, seed % 2 == 1}; binding.imageUnnormalized = {true, false, seed % 2 == 0}; binding.imageSamplers = {0x5u, 0u, 0x80000000u}; binding.bufferRead = {true, false, false, true, true, false}; diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/ImageSampleBorderColor.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/ImageSampleBorderColor.cpp new file mode 100644 index 000000000..b4a5db512 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/ImageSampleBorderColor.cpp @@ -0,0 +1,114 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Texture.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 32; +constexpr std::uint32_t Width = 4; +constexpr std::uint32_t Type2D = 9; +constexpr std::uint32_t Unorm8_8_8_8 = 56; +constexpr std::uint32_t Uint8_8_8_8 = 60; +constexpr std::uint32_t SwizzleXYZW = 0xfacu; +constexpr std::uint32_t One = 0x3f800000u; +constexpr std::uint32_t ClampBorder = 0x1b6u; +constexpr std::uint32_t AnisoOverride = (1u << 29u) | (2u << 22u) | (2u << 20u); + +alignas(256) std::array Output{}; +alignas(256) std::array Texels{}; + +alignas(256) constexpr std::array SampleOutside{ + 0x34060084u, 0x7e2802f4u, 0x7e2a02f4u, 0xf09c0f08u, 0x00610a14u, 0xbf8c3f70u, 0xe0781000u, 0x80000a03u, + 0xbf810000u, +}; + +std::string Hex(std::uint32_t value) { + char text[16]; + std::snprintf(text, sizeof(text), "0x%08x", value); + return text; +} + +std::array TextureDescriptor(std::uint32_t format) { + const auto address = static_cast(reinterpret_cast(Texels.data())); + return { + static_cast(address >> 8u), + static_cast((address >> 40u) & 0xffu) | (format << 20u) | (((Width - 1u) & 3u) << 30u), + (Width - 1u) >> 2u, + SwizzleXYZW | (Type2D << 28u), + 0u, + 0u, + 0u, + 0u, + }; +} + +void Run(AgcDriver::VulkanDevice& device, std::uint32_t format, std::uint32_t borderType, std::uint32_t filters) { + Output.fill(0xdeadbeefu); + Texels.fill(0u); + std::vector userData(24, 0u); + const auto outputAddress = reinterpret_cast(Output.data()); + const std::array output{static_cast(outputAddress), static_cast((outputAddress >> 32u) & 0xffffu), static_cast(Output.size() * 4u), 0x31016facu}; + const auto texture = TextureDescriptor(format); + const std::array sampler{ClampBorder, 0x00fff000u, filters, borderType << 30u}; + std::copy(output.begin(), output.end(), userData.begin()); + std::copy(texture.begin(), texture.end(), userData.begin() + 4); + std::copy(sampler.begin(), sampler.end(), userData.begin() + 12); + const std::span code(SampleOutside); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {true, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); +} + +void Check(AgcDriver::VulkanDevice& device, std::uint32_t format, std::uint32_t borderType, const std::array& expected, const std::string& what, std::uint32_t filters = 0u) { + Run(device, format, borderType, filters); + for (std::uint32_t thread = 0; thread < Threads; ++thread) { + for (std::uint32_t component = 0; component < 4u; ++component) { + const auto actual = Output[thread * 4u + component]; + Require(actual == expected[component], what + ": thread " + std::to_string(thread) + " component " + std::to_string(component) + " is " + Hex(actual) + ", expected " + Hex(expected[component])); + } + } +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + Check(*device, Unorm8_8_8_8, 0u, {0u, 0u, 0u, 0u}, "8_8_8_8_UNORM transparent black border"); + Check(*device, Unorm8_8_8_8, 1u, {0u, 0u, 0u, One}, "8_8_8_8_UNORM opaque black border"); + Check(*device, Unorm8_8_8_8, 2u, {One, One, One, One}, "8_8_8_8_UNORM opaque white border"); + Check(*device, Uint8_8_8_8, 0u, {0u, 0u, 0u, 0u}, "8_8_8_8_UINT transparent black border"); + Check(*device, Uint8_8_8_8, 1u, {0u, 0u, 0u, 1u}, "8_8_8_8_UINT opaque black border"); + Check(*device, Uint8_8_8_8, 2u, {1u, 1u, 1u, 1u}, "8_8_8_8_UINT opaque white border"); + Check(*device, Unorm8_8_8_8, 2u, {One, One, One, One}, "8_8_8_8_UNORM opaque white border under ANISO_OVERRIDE", AnisoOverride); + Check(*device, Uint8_8_8_8, 2u, {1u, 1u, 1u, 1u}, "8_8_8_8_UINT opaque white border under ANISO_OVERRIDE", AnisoOverride); + std::puts("image sample border color tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp b/core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp index 7455a1384..4dfca8bb8 100644 --- a/core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp +++ b/core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp @@ -298,13 +298,20 @@ DescriptorBindingPlan DescriptorBindingBuilder::Prepare(const IrBindingLayout& l for (std::uint32_t component = 0; component < 4u; ++component) plan.specialization.push_back({PipelineSpecialization::ExportBase + target * 4u + component, (exportMappings[target] >> (component * 2u)) & 3u}); } std::vector samplerModes(info.samplers.size(), 0u); + std::vector samplerIntegerModes(info.samplers.size(), 0u); + std::vector samplerFloatModes(info.samplers.size(), 0u); const auto samplerMode = [](const ImageResource& image) { return !image.constantSwizzle && (image.numericClass == IrTextureNumericClass::Sint || image.conversionFormat != IrBufferFormat::Invalid || image.depthBits) ? 2u : 1u; }; + const auto pairSampler = [&](std::uint32_t sampler, const ImageResource& image) { + const auto mode = samplerMode(image); + samplerModes.at(sampler) |= mode; + (image.numericClass == IrTextureNumericClass::Uint || image.numericClass == IrTextureNumericClass::Sint ? samplerIntegerModes : samplerFloatModes).at(sampler) |= mode; + }; for (const auto& pair : info.sampledPairs) { const auto& image = info.images.at(pair.image); - if (image.indirectRoot == ImageResource::NoIndirectImage) samplerModes.at(pair.sampler) |= samplerMode(info.runtimeImageModes.at(pair.image).at(imageModes[pair.image])); + if (image.indirectRoot == ImageResource::NoIndirectImage) pairSampler(pair.sampler, info.runtimeImageModes.at(pair.image).at(imageModes[pair.image])); else { const auto& root = info.images.at(image.indirectRoot); - for (const auto slot : root.indirectResources) samplerModes.at(pair.sampler) |= samplerMode(info.runtimeImageModes.at(slot).at(imageModes[slot])); + for (const auto slot : root.indirectResources) pairSampler(pair.sampler, info.runtimeImageModes.at(slot).at(imageModes[slot])); } } for (std::uint32_t index = 0; index < info.images.size(); ++index) { @@ -411,6 +418,8 @@ DescriptorBindingPlan DescriptorBindingBuilder::Prepare(const IrBindingLayout& l for (const auto& pair : info.sampledPairs) if (pair.sampler == resource) compare |= info.images.at(pair.image).depthCompare && compareStates.at(pair.image) == 0u; physical.samplerDepthCompare.push_back(compare); physical.samplerUnnormalized.push_back(unnormalized.samplers.at(resource)); + const auto mode = 1u << (originals[element] & 1u); + physical.samplerIntegerBorder.push_back((samplerIntegerModes.at(resource) & mode) != 0u && (samplerFloatModes.at(resource) & mode) == 0u); if ((originals[element] & 1u) != 0u) entry.samplerFilterElements.push_back(element); } } else if (physical.role == DescriptorRole::ShaderData && layout.UsesPushData()) { diff --git a/core/shader/recompiler/Recompiler.hpp b/core/shader/recompiler/Recompiler.hpp index 5688af8e8..d77c76088 100644 --- a/core/shader/recompiler/Recompiler.hpp +++ b/core/shader/recompiler/Recompiler.hpp @@ -364,6 +364,7 @@ struct DescriptorBinding { // (a producer that does not fill it) must be treated as written. std::vector bufferWritten; std::vector samplerUnnormalized; + std::vector samplerIntegerBorder; std::vector imageUnnormalized; std::vector imageSamplers; std::vector bufferRead; diff --git a/core/shader/recompiler/ShaderDiskCache.cpp b/core/shader/recompiler/ShaderDiskCache.cpp index fe8b08896..6225925ac 100644 --- a/core/shader/recompiler/ShaderDiskCache.cpp +++ b/core/shader/recompiler/ShaderDiskCache.cpp @@ -256,6 +256,7 @@ void encodeBinding(Writer& writer, const DescriptorBinding& binding) { writer.Flags(binding.bufferWritten); writer.Flags(binding.samplerUnnormalized); writer.Flags(binding.imageUnnormalized); + writer.Flags(binding.samplerIntegerBorder); writer.Values(std::span(binding.imageSamplers)); writer.Flags(binding.bufferRead); } @@ -280,6 +281,7 @@ void decodeBinding(Reader& reader, DescriptorBinding& binding) { reader.Flags(binding.bufferWritten); reader.Flags(binding.samplerUnnormalized); reader.Flags(binding.imageUnnormalized); + reader.Flags(binding.samplerIntegerBorder); reader.Values(binding.imageSamplers); reader.Flags(binding.bufferRead); } diff --git a/core/shader/recompiler/ShaderDiskCache.hpp b/core/shader/recompiler/ShaderDiskCache.hpp index 6111e47c1..08b1d6c8d 100644 --- a/core/shader/recompiler/ShaderDiskCache.hpp +++ b/core/shader/recompiler/ShaderDiskCache.hpp @@ -12,7 +12,7 @@ namespace ShaderRecompiler::ShaderDiskCache { -inline constexpr std::uint32_t FormatVersion = 24; +inline constexpr std::uint32_t FormatVersion = 25; enum class LoadStatus { Loaded, From 8132ac7a218cc924a963250b6558dc0682aadd29 Mon Sep 17 00:00:00 2001 From: fma-rc <188743521+fma-rc@users.noreply.github.com> Date: Sun, 11 Oct 2026 03:57:24 +0200 Subject: [PATCH 307/460] docs(debt): state the measured steps and the non-pixel LOD of a 2D implicit-LOD sample on a 1D T# --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 44b985f26..b1285744c 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -381,7 +381,7 @@ Throughout the project, every function at every stage either **does exactly what - A value read through a null or unmapped pointer still throws at record time if a [branch condition, uniform fill or table base](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp) needs it, or if the pointer comes from user data and is not null, even if the console skips the read. - A [V# or flat SRT word](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtEvaluator.cpp) read through a null pointer taken from user data is zero and counts as a [poisoned SRT read](../../core/shader/recompiler/Recompiler.cpp), so a dispatch that reads it is not cached. The console faults if the program executes the read; PPSA28416 branches around it with `s_cbranch_execz`. The fault is not modeled. - A [V#, T# or S#](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtDescriptorEvaluation.cpp) loaded through a null or unmapped pointer read from memory is zero when the work is recorded, and the capture throws if no flat SRT slot records that read. -- [2D accesses through a 1D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) read the texel the 1D addressing selects and ignore the second coordinate, as simonbbby measured on RDNA2 (gfx1036) in #2237 for `image_sample_lz` and mip-less `image_load` with point filtering and wrap, clamp and border addressing; the PS5's gfx1013 is not measured. A plain `image_sample` (implicit LOD) through such a pair takes its LOD from the derivatives of the first coordinate alone, measured on RDNA2 (gfx1036) with the [hardware oracle](HW_ORACLE.md) on the quads of a compute kernel (wave32 and wave64, IEEE off, DX10 clamp on, denormals preserved, round to nearest even): on a 16-texel, 5-mip 32_UINT T# with point filters it picks the same mip and texel as DIM 1D for first-coordinate steps of 0 to 8 texels per lane in either direction, whatever the second coordinate does (steps up to 16 per lane), while the same rows on a 16x16 2D T# move the mip with the second coordinate. Other 2D accesses through a 1D T# (storage, depth, packed, BY loads, mips, explicit LOD, bias, offsets, derivatives, comparison) still throw. +- [2D accesses through a 1D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) read the texel the 1D addressing selects and ignore the second coordinate, as simonbbby measured on RDNA2 (gfx1036) in #2237 for `image_sample_lz` and mip-less `image_load` with point filtering and wrap, clamp and border addressing; the PS5's gfx1013 is not measured. A plain `image_sample` (implicit LOD) through such a pair takes its LOD from the derivatives of the first coordinate alone, measured on RDNA2 (gfx1036) with the [hardware oracle](HW_ORACLE.md) on the quads of a compute kernel (wave32 and wave64, IEEE off, DX10 clamp on, denormals preserved, round to nearest even): on a 16-texel, 5-mip 32_UINT T# with point filters it picks the same mip and texel as DIM 1D for first-coordinate steps of 0, 1, 2, 4 and 8 texels per lane along x or along y, whatever the second coordinate does (normalised steps up to 16 per lane), while the same rows on a 16x16 2D T# move the mip with the second coordinate. Outside the pixel stage a plain `image_sample` is emitted with LOD 0 for every image type, so this pair reads mip 0 there, where the kernel measurement shows the quad's derivatives being used. Other 2D accesses through a 1D T# (storage, depth, packed, BY loads, mips, explicit LOD, bias, offsets, derivatives, comparison) still throw. - [Emulated comparison of a 2D-array access through a 2D T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) samples the 2D image and ignores the slice, as the native modes of a 2D-array access through a 2D T# already do. This follows the T# type deciding the addressing; it is not measured on hardware for comparison samples - [Scalar aperture base sources](../../core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp) (core/shader/recompiler) - `src_shared_base`, `src_shared_limit`, `src_private_base` and `src_private_limit` (scalar source codes 235 to 238) are not modelled: decoding throws `unsupported scalar source operand code` with the code and the program counter, so a shader that reads one does not compile. Operand 235 (`src_shared_base`) was reported on [#1182](https://github.com/boykopovar/AnyPS5/pull/1182) from PPSA26344; the flat aperture routing does not read the base, so a shader that builds its own flat addresses from it stops at decode. - [Flat aperture bases](../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp) (core/shader/recompiler) - the layout follows the gfx9+ one in the AMD KFD driver's `kfd_flat_memory.c` (`MAKE_LDS_APP_BASE_V9`, `MAKE_LDS_APP_LIMIT`): each aperture is a 4 GB window whose base is the high dword, so a flat address is routed by its whole high dword, as measured on RDNA2 (gfx1035: `src_shared_base` 0x00010000_00000000, `src_shared_limit` 0x00010000_ffffffff). The PS5's base values, high dwords 0x80000000 (shared, `SharedApertureHigh`) and 0x70000000 (private, `PrivateApertureHigh`), are inferred from GTA V's traversal addresses, not measured on a console or read from SH_MEM_BASES From 3cd36b0135fff6312563be26d15b44a230eb944b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 03:57:15 +0200 Subject: [PATCH 308/460] feat(libc): implement acosl and frexpl Co-Authored-By: Claude --- core/libs/prx/libc/src/Math.cpp | 6 ++++++ core/libs/tests/GuestMath.cpp | 35 +++++++++++++++++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 42 insertions(+) diff --git a/core/libs/prx/libc/src/Math.cpp b/core/libs/prx/libc/src/Math.cpp index 8beda4557..614e7e1a5 100644 --- a/core/libs/prx/libc/src/Math.cpp +++ b/core/libs/prx/libc/src/Math.cpp @@ -46,6 +46,12 @@ int APS5_VABI __isinff_nid_postfix(float x) { return std::isinf(x) ? 1 : 0; } double APS5_VABI cbrt_nid_postfix(double x) { return std::cbrt(x); } double APS5_VABI asin_nid_postfix(double x) { return std::asin(x); } double APS5_VABI acos_nid_postfix(double x) { return std::acos(x); } +long double APS5_VABI acosl_nid_postfix(long double x) { return std::acos(x); } + +long double APS5_VABI frexpl_nid_postfix(long double x, int* exponent) { + if (!std::isfinite(x)) return x; + return std::frexp(x, exponent); +} double APS5_VABI exp_nid_postfix(double x) { return std::exp(x); } double APS5_VABI atan_nid_postfix(double x) { return std::atan(x); } double APS5_VABI tan_nid_postfix(double x) { return std::tan(x); } diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index b0a96e78b..5bc32fad4 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -23,6 +23,8 @@ std::uint64_t APS5_VABI _Stoul_nid_postfix(const char*, char**, int); std::int64_t APS5_VABI atol_nid_postfix(const char*); long long APS5_VABI atoll_nid_postfix(const char*); int* APS5_VABI __error_nid_postfix(); +long double APS5_VABI acosl_nid_postfix(long double); +long double APS5_VABI frexpl_nid_postfix(long double, int*); struct LibcFloatConstant { std::uint32_t bits[4]; }; extern LibcFloatConstant _FInf_nid_postfix; extern LibcFloatConstant _FNan_nid_postfix; @@ -59,6 +61,38 @@ std::lldiv_t APS5_VABI ldiv_nid_postfix(std::int64_t, std::int64_t); } static void Require(bool value) { if (!value) std::abort(); } +static bool ExtendedBits(long double value, std::uint64_t mantissa, std::uint16_t signExponent) { + unsigned char bytes[sizeof(long double)]{}; + std::memcpy(bytes, &value, sizeof(value)); + std::uint64_t actualMantissa = 0; + std::uint16_t actualSignExponent = 0; + std::memcpy(&actualMantissa, bytes, 8); + std::memcpy(&actualSignExponent, bytes + 8, 2); + return actualMantissa == mantissa && actualSignExponent == signExponent; +} + +static void CheckLongDouble() { + Require(ExtendedBits(acosl_nid_postfix(-1.0L), 0xc90fdaa22168c235ull, 0x4000)); + Require(ExtendedBits(acosl_nid_postfix(0.0L), 0xc90fdaa22168c235ull, 0x3fff)); + Require(ExtendedBits(acosl_nid_postfix(1.0L), 0, 0)); + Require(std::isnan(acosl_nid_postfix(1.5L)) && std::isnan(acosl_nid_postfix(-2.0L))); + + int exponent = 77; + Require(frexpl_nid_postfix(8.0L, &exponent) == 0.5L && exponent == 4); + Require(frexpl_nid_postfix(-3.0L, &exponent) == -0.75L && exponent == 2); + Require(frexpl_nid_postfix(0.375L, &exponent) == 0.75L && exponent == -1); + long double smallest = 0; + const std::uint64_t one = 1; + std::memcpy(&smallest, &one, sizeof(one)); + Require(ExtendedBits(frexpl_nid_postfix(smallest, &exponent), 0x8000000000000000ull, 0x3ffe) && exponent == -16444); + exponent = 77; + Require(ExtendedBits(frexpl_nid_postfix(-0.0L, &exponent), 0, 0x8000) && exponent == 0); + exponent = 77; + Require(frexpl_nid_postfix(INFINITY, &exponent) == INFINITY && exponent == 77); + Require(frexpl_nid_postfix(-INFINITY, &exponent) == -INFINITY && exponent == 77); + Require(std::isnan(frexpl_nid_postfix(NAN, &exponent)) && exponent == 77); +} + static void CheckIntegerConversions() { for (const long long numerator : {4294967301LL, -4294967301LL}) { for (const long long denominator : {3LL, -3LL}) { @@ -255,6 +289,7 @@ int main() { Require(std::isnan(logbf_nid_postfix(std::numeric_limits::quiet_NaN()))); Require(exp2_nid_postfix(-3.) == 0.125); Require(ldexp_nid_postfix(0.75, 4) == 12.); + CheckLongDouble(); Require(scalbn_nid_postfix(0.75, -2) == 0.1875); Require(scalbnf_nid_postfix(0.75f, 4) == 12.f); Require(nextafterf_nid_postfix(1.f, 2.f) == 0x1.000002p0f && nextafterf_nid_postfix(1.f, 0.f) == 0x1.fffffep-1f); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 237d82bed..ea6808edc 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -166,6 +166,7 @@ Throughout the project, every function at every stage either **does exactly what - [_sceKernelRtldThreadAtexitIncrement, _sceKernelRtldThreadAtexitDecrement](../../core/libs/prx/libkernel/System/src/Rtld.cpp) (libkernel) - the argument is the `dso` address a title's `libc.prx` passes to `__cxa_thread_atexit` (PPSA14632), which fails when Increment returns nonzero. As in the [fpPS4](https://github.com/red-prig/fpPS4) reimplementation, Increment takes a reference on the loaded image containing the address and Decrement releases one; the variants without the leading underscore are assumed to match - [sceImeGetPanelPositionAndForm](../../core/libs/prx/libSceIme/Export.cpp) (libSceIme) - the output is taken to be the `PositionAndForm` that `sceImeDialogGetPanelPositionAndForm` fills; it is never written, since no panel can be open. That a closed panel is reported before a null output is assumed by analogy with `sceImeSetCaret`, not confirmed on a PS5 title - [_sceLibcInternalForceTlsDestructor](../../core/libs/prx/libSceLibcInternal/Export.cpp) (libSceLibcInternal) - behaviour follows `sceLibcForceTlsDestructor(int)` in a title's `libc.prx` (disassembly): the calling thread's destructors whose dso belongs to the module run newest first, only if the dso's module is loaded and the dso points to itself, in up to 4 passes. The return type is unknown (that function returns nothing), 0 is returned. The module of each dso is looked up when the destructors are forced, not when they are registered as `libc.prx` does, and the lookup is Linux only, so on Windows the call throws when the thread has destructors +- [acosl](../../core/libs/prx/libc/src/Math.cpp) (libc) - computed by the host's math library, like the other libc math functions here, not by the console's (FreeBSD msun ld80 code in Sony's `libc.prx`): results can differ from the console's in the last bit, and on Linux glibc sets `errno` to `EDOM` for `acosl` outside [-1, 1] where msun only raises the floating-point exception. `long double` is the x87 80-bit format, as Sony's `frexpl` uses it (`fldt 0x8(%rsp)`, result in `st(0)`) - [std_execute_once](../../core/libs/prx/libc/Export.cpp) (libc) - assumed to be the plain-named twin of `std::_Execute_once` (`_ZSt13_Execute_onceRSt9once_flagPFiPvS1_PS1_ES1_`, the callee of Dinkumware `call_once`): it delegates there and gets the [thread.once.callonce] semantics already verified for that symbol (one invocation per flag, concurrent callers block until it completes, and a failing or throwing invocation leaves the flag unready so the next caller runs it). The name equivalence is not confirmed on a title, and the callback convention (called with (null, arg, null), non-zero return meaning success) is the twin's, unverified for this symbol - [Dinkumware mutexes and conditions](../../core/libs/prx/libc/src/Threads.cpp) (libc) use pointer-to-handle arguments, matching the console aerolib signatures reported in [#3079](https://github.com/boykopovar/AnyPS5/pull/3079). The title's `_Mtx_lock` maps `0` to `_Thrd_success` (0), `0x8002000b` to `_Thrd_busy` (3), and other failures to `_Thrd_error` (4); its internal mutex init recognizes recursive flag `0x100`, while every other type uses default attributes. The observed relock failure for a plain mutex suggests those defaults are error-checking, but that has not been confirmed on hardware. The title's `_Mtx_unlock` and `_Cnd_wait` return success regardless of their internal result. The host implementation treats invalid ownership preconditions as no-ops, and throws for invalid handles, host wait errors and destroying a locked mutex as debug guards; those cases are not confirmed against the console. - [tcgetattr, tcsetattr](../../core/libs/prx/libkernel/File/src/Terminal.cpp) (libkernel) - every file and pipe descriptor is taken as not a terminal (`ENOTTY`), and a socket gets `EADDRNOTAVAIL` for `AF_INET` and `EOPNOTSUPP` otherwise, as FreeBSD's `in_control` and `pru_control_notsupp` answer a termios ioctl; what descriptors 0 to 2 of a title are on the console, and whether any of them is a terminal there, is unknown From 05b3f6e40d555aa310d4ead8b50e66920405c51b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 03:59:03 +0200 Subject: [PATCH 309/460] fix(agc): update the DescriptorBinding size assert for samplerIntegerBorder Co-Authored-By: Claude --- core/shader/recompiler/ShaderDiskCache.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/shader/recompiler/ShaderDiskCache.cpp b/core/shader/recompiler/ShaderDiskCache.cpp index 6225925ac..7100a5806 100644 --- a/core/shader/recompiler/ShaderDiskCache.cpp +++ b/core/shader/recompiler/ShaderDiskCache.cpp @@ -67,7 +67,7 @@ namespace ShaderRecompiler::ShaderDiskCache { static_assert(sizeof(CompiledShaderArtifact) == 192, "CompiledShaderArtifact changed: update the artifact encoder"); static_assert(sizeof(ShaderInvocation) == 112, "ShaderInvocation changed: update the invocation encoder"); static_assert(sizeof(RecompileResult) == 312, "RecompileResult changed: update EncodeResult and DecodeResult"); -static_assert(sizeof(DescriptorBinding) == 488, "DescriptorBinding changed: update the binding encoder"); +static_assert(sizeof(DescriptorBinding) == 528, "DescriptorBinding changed: update the binding encoder"); static_assert(sizeof(VertexAttribute) == 32, "VertexAttribute changed: update the attribute encoder"); static_assert(sizeof(VertexInput) == 16, "VertexInput changed: update the vertex input encoder"); static_assert(sizeof(FragmentParameter) == 12, "FragmentParameter changed: update the parameter encoder"); From ba0b042925959eda6acfdcb085a7c53d49ddee72 Mon Sep 17 00:00:00 2001 From: XairRrR Date: Sun, 11 Oct 2026 02:27:09 +0200 Subject: [PATCH 310/460] feat(agc-driver): blend into COLOR_8_8 SWAP_ALT targets with a separate alpha equation An 8_8 target with SWAP_ALT keeps the x export in its first channel and the alpha export in its second, and the color block blends that second channel with the alpha equation. Blending into such a target threw. With one equation for color and alpha the draw blends as before. With a separate alpha equation the pipeline draws twice: the first pass writes the first channel with the color equation, the second writes the second channel with the alpha factors and operation moved into its color equation. The split is refused, by throwing, for draws that write depth or stencil, use per-face depth bias, or whose pixel shader writes memory, since those would repeat; factors of the destination alpha throw since the target keeps it in its second channel. Pipeline libraries are not used for split pipelines. --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 2 + .../Graphics/include/Pipeline.hpp | 3 + .../Graphics/include/State.hpp | 1 + .../prx/libSceAgcDriver/Graphics/src/Draw.cpp | 8 + .../libSceAgcDriver/Graphics/src/Pipeline.cpp | 64 +++- .../libSceAgcDriver/Graphics/src/State.cpp | 14 +- .../prx/libSceAgcDriver/tests/Graphics.cpp | 7 +- .../tests/execution/SwapAltBlend.cpp | 300 ++++++++++++++++++ 8 files changed, 394 insertions(+), 5 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/SwapAltBlend.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 6d03e7feb..5afafbe20 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -907,6 +907,8 @@ endforeach() target_include_directories(agc_driver_dual_source_blend_tests PRIVATE $) +target_include_directories(agc_driver_swap_alt_blend_tests PRIVATE $) + target_include_directories(agc_driver_packed_unorm_color_target_tests PRIVATE $) foreach(stage IN ITEMS vert frag) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp index d053fd79f..c4fa93353 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp @@ -54,6 +54,8 @@ public: bool SplitsFaces() const { return backFaces != VK_NULL_HANDLE; } bool Optimized() const { return optimized != nullptr && optimized->handle.load() != VK_NULL_HANDLE; } void ContinueBackFaces(VkCommandBuffer commands, const State& state) const; + bool SplitsSecondChannel() const { return secondChannel != VK_NULL_HANDLE; } + void ContinueSecondChannel(VkCommandBuffer commands) const; void PushConstants(VkCommandBuffer commands, VkShaderStageFlags stages, std::span bytes) const; // Forgets the Vulkan objects without destroying them: for entries of a device that is already gone. void Abandon() noexcept; @@ -73,6 +75,7 @@ private: VkRenderPass renderPass = VK_NULL_HANDLE; VkPipeline pipeline = VK_NULL_HANDLE; VkPipeline backFaces = VK_NULL_HANDLE; + VkPipeline secondChannel = VK_NULL_HANDLE; std::size_t attachments = 0; std::size_t colorAttachments = 0; bool depthBounds = false; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp index 05cb27663..fe63d81cf 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp @@ -179,6 +179,7 @@ inline constexpr std::array DrawKeyRegisters{{ // GE_PRIM_TYPE, GE_MULTI_PRIM_IB_RESET_EN, the geometry subgroup sizes. {RegisterBank::UserConfig, 0x242, 1}, {RegisterBank::UserConfig, 0x24b, 1}, {RegisterBank::UserConfig, 0x25b, 1}, }}; +bool SplitsAlternateAlpha(const State& state); // Whether DrawKeyRegisters holds the read. bool DrawKeyCovers(RegisterRead read); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp index a36593d10..849e0a59c 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp @@ -1736,6 +1736,10 @@ void recordDraw(const Context& context, const State& state, const Pm4::DrawParam record.pipeline->ContinueBackFaces(commands, state); recordDrawCommands(context, commands, state, draw, inputs, record.indirect, argumentBuffer, argumentOffset); } + if (record.pipeline->SplitsSecondChannel()) { + record.pipeline->ContinueSecondChannel(commands); + recordDrawCommands(context, commands, state, draw, inputs, record.indirect, argumentBuffer, argumentOffset); + } if (meshArguments != nullptr) recorder->Keep(meshArguments); if (args != nullptr) CountIndirectDraw(record.indirect->path, record.indirect->readMs, rewritten); auto checkRecords = indirectRecordCheck(record.indirect); @@ -2208,6 +2212,10 @@ void Draw(const Context& context, const State& state, const Pm4::DrawParameters& pipeline->ContinueBackFaces(commands, state); recordDrawCommands(context, commands, state, draw, inputs, args != nullptr ? &indirect : nullptr, argumentBuffer, argumentOffset); } + if (pipeline->SplitsSecondChannel()) { + pipeline->ContinueSecondChannel(commands); + recordDrawCommands(context, commands, state, draw, inputs, args != nullptr ? &indirect : nullptr, argumentBuffer, argumentOffset); + } if (meshArguments != nullptr && recorded) recorder->Keep(meshArguments); if (args != nullptr) CountIndirectDraw(indirect.path, indirect.readMs, rewritten); auto checkRecords = indirectRecordCheck(args != nullptr ? &indirect : nullptr); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp index ae8f7b15c..d900e5e89 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp @@ -128,9 +128,35 @@ void appendKey(std::vector& key, const TValue& value) { key.insert(key.end(), bytes.begin(), bytes.end()); } +bool fragmentWritesMemory(std::span shaders) { + const auto any = [](const std::vector& written, std::uint32_t count) { return written.size() < count || std::ranges::any_of(written, [](bool element) { return element; }); }; + for (const auto& shader : shaders) { + if (shader.stage != ShaderRecompiler::ShaderStage::Fragment || shader.program == nullptr) continue; + for (const auto& binding : shader.program->bindings) { + switch (binding.role) { + case ShaderRecompiler::DescriptorRole::GuestBuffers: + if (binding.kind != ShaderRecompiler::DescriptorKind::UniformBuffer && binding.kind != ShaderRecompiler::DescriptorKind::UniformTexelBuffer && any(binding.bufferWritten, binding.count)) return true; + break; + case ShaderRecompiler::DescriptorRole::GuestImages: + if (binding.kind == ShaderRecompiler::DescriptorKind::StorageImage && any(binding.imageWritten, binding.count)) return true; + break; + case ShaderRecompiler::DescriptorRole::Gds: + case ShaderRecompiler::DescriptorRole::BdaPagetable: + return true; + case ShaderRecompiler::DescriptorRole::GuestSamplers: + case ShaderRecompiler::DescriptorRole::FaultBuffer: + case ShaderRecompiler::DescriptorRole::FlattenedSrt: + case ShaderRecompiler::DescriptorRole::ShaderData: + break; + } + } + } + return false; +} + bool libraryKeys(PipelineLibraryKeys& keys, const Context& context, const State& state, const VertexInputLayout& input, const ShaderResources& resources, std::span shaders) { using Stage = ShaderRecompiler::ShaderStage; - if (!context.graphicsPipelineLibrary || context.pipelineExecutableInfo || state.depthBiasPerFace || state.rectList) return false; + if (!context.graphicsPipelineLibrary || context.pipelineExecutableInfo || state.depthBiasPerFace || state.rectList || SplitsAlternateAlpha(state)) return false; keys = {}; for (const auto& shader : shaders) { if (shader.stage == Stage::Geometry || shader.program->PipelineVariantId() == 0) return false; @@ -347,8 +373,28 @@ Pipeline::Pipeline(const Context& context, const State& state, const VertexInput depthStencil.front = state.stencilFront; depthStencil.back = state.stencilBack; VkPipelineColorBlendStateCreateInfo blend{VK_STRUCTURE_TYPE_PIPELINE_COLOR_BLEND_STATE_CREATE_INFO}; - blend.attachmentCount = static_cast(state.blends.size()); - blend.pAttachments = state.blends.empty() ? nullptr : state.blends.data(); + const bool alternateAlpha = SplitsAlternateAlpha(state); + std::vector firstChannels(state.blends.begin(), state.blends.end()); + std::vector secondChannels(state.blends.begin(), state.blends.end()); + if (alternateAlpha) { + const auto keepsStencil = [](const VkStencilOpState& face) { return face.writeMask == 0 || (face.failOp == VK_STENCIL_OP_KEEP && face.passOp == VK_STENCIL_OP_KEEP && face.depthFailOp == VK_STENCIL_OP_KEEP); }; + Require(!(state.depth && (state.depthWrite || (state.stencilTest && (!keepsStencil(state.stencilFront) || !keepsStencil(state.stencilBack))))) && !state.depthBiasPerFace, "blending into an 8_8 color target with the alternate component order and a separate alpha blend from a draw that writes depth or stencil is not implemented"); + Require(!fragmentWritesMemory(shaders), "blending into an 8_8 color target with the alternate component order and a separate alpha blend from a pixel shader that writes memory is not implemented"); + for (std::size_t index = 0; index < secondChannels.size(); ++index) { + const bool split = index < state.colors.size() && state.colors[index].componentMapping == 0xecu && state.blends[index].blendEnable; + if (!split) { + secondChannels[index].colorWriteMask = 0; + continue; + } + firstChannels[index].colorWriteMask &= ~VK_COLOR_COMPONENT_G_BIT; + secondChannels[index].colorWriteMask &= VK_COLOR_COMPONENT_G_BIT; + secondChannels[index].srcColorBlendFactor = state.blends[index].srcAlphaBlendFactor; + secondChannels[index].dstColorBlendFactor = state.blends[index].dstAlphaBlendFactor; + secondChannels[index].colorBlendOp = state.blends[index].alphaBlendOp; + } + } + blend.attachmentCount = static_cast(firstChannels.size()); + blend.pAttachments = firstChannels.empty() ? nullptr : firstChannels.data(); std::copy(state.blendConstants.begin(), state.blendConstants.end(), blend.blendConstants); VkGraphicsPipelineCreateInfo pipelineInfo{VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO}; pipelineInfo.flags = context.pipelineExecutableInfo ? VK_PIPELINE_CREATE_CAPTURE_STATISTICS_BIT_KHR : 0; @@ -373,6 +419,10 @@ Pipeline::Pipeline(const Context& context, const State& state, const VertexInput timing.Mark("modules_and_state"); if (libraries) pipeline = LinkPipelineFromLibraries(context, pipelineInfo, rendering, layoutInfo, keys, &optimized); else Check(context.Function("vkCreateGraphicsPipelines")(context.device, context.pipelineCache, 1, &pipelineInfo, nullptr, &pipeline), "vkCreateGraphicsPipelines"); + if (alternateAlpha) { + blend.pAttachments = secondChannels.data(); + Check(context.Function("vkCreateGraphicsPipelines")(context.device, context.pipelineCache, 1, &pipelineInfo, nullptr, &secondChannel), "vkCreateGraphicsPipelines"); + } if (state.depthBiasPerFace) { raster.cullMode = VK_CULL_MODE_FRONT_BIT; Check(context.Function("vkCreateGraphicsPipelines")(context.device, context.pipelineCache, 1, &pipelineInfo, nullptr, &backFaces), "vkCreateGraphicsPipelines"); @@ -397,6 +447,7 @@ void Pipeline::release() noexcept { optimized.reset(); if (pipeline) context.Function("vkDestroyPipeline")(context.device, pipeline, nullptr); if (backFaces) context.Function("vkDestroyPipeline")(context.device, backFaces, nullptr); + if (secondChannel) context.Function("vkDestroyPipeline")(context.device, secondChannel, nullptr); if (renderPass) context.Function("vkDestroyRenderPass")(context.device, renderPass, nullptr); if (layout) context.Function("vkDestroyPipelineLayout")(context.device, layout, nullptr); for (auto module : _modules) { @@ -404,6 +455,7 @@ void Pipeline::release() noexcept { } pipeline = VK_NULL_HANDLE; backFaces = VK_NULL_HANDLE; + secondChannel = VK_NULL_HANDLE; renderPass = VK_NULL_HANDLE; layout = VK_NULL_HANDLE; _modules.clear(); @@ -416,6 +468,7 @@ void Pipeline::Abandon() noexcept { framebuffers.clear(); pipeline = VK_NULL_HANDLE; backFaces = VK_NULL_HANDLE; + secondChannel = VK_NULL_HANDLE; renderPass = VK_NULL_HANDLE; layout = VK_NULL_HANDLE; _modules.clear(); @@ -540,6 +593,11 @@ void Pipeline::setLibraryState(VkCommandBuffer commands, const State& state) con stencil(VK_STENCIL_FACE_BACK_BIT, depth ? state.stencilBack : VkStencilOpState{}); } +void Pipeline::ContinueSecondChannel(VkCommandBuffer commands) const { + Require(secondChannel != VK_NULL_HANDLE, "the pipeline blends every channel in one pass"); + context.Resolved(&DeviceFunctions::cmdBindPipeline, "vkCmdBindPipeline")(commands, VK_PIPELINE_BIND_POINT_GRAPHICS, secondChannel); +} + void Pipeline::ContinueBackFaces(VkCommandBuffer commands, const State& state) const { Require(backFaces != VK_NULL_HANDLE, "the pipeline draws both faces in one pass"); context.Resolved(&DeviceFunctions::cmdBindPipeline, "vkCmdBindPipeline")(commands, VK_PIPELINE_BIND_POINT_GRAPHICS, backFaces); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp index f29320922..b29634aad 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp @@ -727,7 +727,6 @@ State DecodeState(const QueueState& queue) { state.blendEnable = (blend >> 30u) & 1u; if (state.blendEnable && (mapping == 0x1bu || mapping == 0x93u)) throw std::runtime_error("AGC graphics: blending into a color target with a reversed component order is not implemented"); Require(!state.blendEnable || !color.uintExport, "blending into an unsigned integer target is unsupported"); - if (state.blendEnable && mapping == 0xecu) throw std::runtime_error("AGC graphics: blending into an 8_8 color target with the alternate component order is not implemented"); if (state.blendEnable) { Require((read(cx, 0x31c + slot * 0xfu) & 0x10000u) == 0, "blend bypass conflicts with enabled blending"); state.srcColorBlendFactor = blendFactor(blend & 0x1fu); @@ -749,6 +748,10 @@ State DecodeState(const QueueState& queue) { state.dstAlphaBlendFactor = alphaFactorWithOpaqueDestination(state.dstAlphaBlendFactor); } for (std::uint32_t i = 0; i < 4; ++i) result.blendConstants[i] = readFloat(cx, 0x105 + i); + if (mapping == 0xecu) { + const auto destinationAlpha = [](VkBlendFactor factor) { return factor == VK_BLEND_FACTOR_DST_ALPHA || factor == VK_BLEND_FACTOR_ONE_MINUS_DST_ALPHA || factor == VK_BLEND_FACTOR_SRC_ALPHA_SATURATE; }; + if (destinationAlpha(state.srcColorBlendFactor) || destinationAlpha(state.dstColorBlendFactor) || destinationAlpha(state.srcAlphaBlendFactor) || destinationAlpha(state.dstAlphaBlendFactor)) throw std::runtime_error("AGC graphics: blending into an 8_8 color target with the alternate component order with a factor of the destination alpha, which the target keeps in its second channel, is not implemented"); + } } if (color.packing != ShaderRecompiler::ColorExportPacking::None) { const auto slotExport = (exportFormat >> (4u * color.exportIndex)) & 0xfu; @@ -1036,6 +1039,15 @@ std::vector*& RegisterReadLog() { return log; } +bool SplitsAlternateAlpha(const State& state) { + for (std::size_t index = 0; index < state.colors.size() && index < state.blends.size(); ++index) { + const auto& blend = state.blends[index]; + if (state.colors[index].componentMapping != 0xecu || !blend.blendEnable) continue; + if (blend.srcColorBlendFactor != blend.srcAlphaBlendFactor || blend.dstColorBlendFactor != blend.dstAlphaBlendFactor || blend.colorBlendOp != blend.alphaBlendOp) return true; + } + return false; +} + bool DrawKeyCovers(RegisterRead read) { // One bit per register offset of each bank, set from the table once. static const auto covered = [] { diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index cf7a795c0..9b99b9bc0 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -767,7 +767,12 @@ void ReversedComponentOrderTests() { const auto state = AgcDriver::Graphics::DecodeState(queue); Require(state.colors.size() == 1 && state.colors[0].format == VK_FORMAT_R8G8_UNORM && state.colors[0].componentMapping == 0xecu, "an 8_8 target with SWAP_ALT did not store the x and w exports in its two channels"); queue.context[0x1e0] = 0x40010001u; - expectFailure([&] { AgcDriver::Graphics::DecodeState(queue); }, "alternate component order"); + for (const auto offset : {0x105u, 0x106u, 0x107u, 0x108u}) queue.context[offset] = 0; + const auto blended = AgcDriver::Graphics::DecodeState(queue); + Require(blended.blends.size() == 1 && blended.blends[0].blendEnable, "blending into an 8_8 SWAP_ALT target with one equation for color and alpha was rejected"); + Require(!AgcDriver::Graphics::SplitsAlternateAlpha(blended), "one blend equation for color and alpha split an 8_8 SWAP_ALT draw"); + queue.context[0x1e0] = 0x60000001u; + Require(AgcDriver::Graphics::SplitsAlternateAlpha(AgcDriver::Graphics::DecodeState(queue)), "a separate alpha blend into an 8_8 SWAP_ALT target did not split its second channel"); } for (const auto& [swap, mapping] : {std::pair{2u, 0x1bu}, std::pair{3u, 0x93u}}) { auto queue = makeState(); diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/SwapAltBlend.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/SwapAltBlend.cpp new file mode 100644 index 000000000..432c19bb8 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/SwapAltBlend.cpp @@ -0,0 +1,300 @@ +#include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp" +#include "prx/libSceAgcDriver/Graphics/include/ColorTargetLayout.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "prx/libSceAgcDriver/Graphics/include/ShaderInputState.hpp" +#include "prx/libSceAgcDriver/Graphics/include/State.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Texture.hpp" +#include "prx/libc/include/GuestAllocations.hpp" +#include "prx/libc/include/GuestArena.hpp" +#include "prx/libc/include/GuestWriteWatch.hpp" +#include "Recompiler.hpp" +#include "Optimization/ResourceProgram.hpp" +#include "VulkanTestDevice.hpp" +#ifdef _WIN32 +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include +#else +#include +#endif +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::ColorTargetLayout; +using AgcDriver::Graphics::ColorTileMode; +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; +using Color = std::array; + +constexpr std::uint32_t Width = 256; +constexpr std::uint32_t Height = 128; +constexpr std::uint32_t ScissorWidth = 128; +constexpr std::size_t BlockBytes = 65536; +constexpr std::uint32_t SwapAlt8x8Info = (3u << 2u) | (1u << 11u) | 0x8000u; +constexpr std::uint32_t TiledAttrib3 = 0x4dc6c000u; +constexpr std::uint32_t Float32Export = 9; + +alignas(256) constexpr std::array VertexCode{ + 0xe0382000, 0x80000005, 0xbf8c3f70, 0xf80008cf, 0x03020100, 0xbf810000, +}; + +constexpr std::array, 3> Triangle{{ + {-1.0f, -1.0f, 0.5f, 1.0f}, {3.0f, -1.0f, 0.5f, 1.0f}, {-1.0f, 3.0f, 0.5f, 1.0f} +}}; + +constexpr std::array, 4> Destinations{{ + {64, 128}, {255, 0}, {32, 224}, {128, 64} +}}; + +alignas(256) std::array, 8> Programs{}; +std::size_t programCount = 0; + +std::span Program(const Color& exported) { + Require(programCount < Programs.size(), "too many test programs"); + auto& code = Programs[programCount++]; + std::size_t at = 0; + for (std::uint32_t channel = 0; channel < 4; ++channel) { + code[at++] = 0x7e0002ffu | ((4u + channel) << 17u); + code[at++] = std::bit_cast(exported[channel]); + } + code[at++] = 0xf800180fu; + code[at++] = 0x07060504u; + code[at++] = 0xbf810000u; + return std::span(code.data(), at); +} + +struct Block { + explicit Block(std::size_t bytes) : bytes(bytes), watched(AgcDriver::GuestMemory::WriteWatched()) { +#ifdef _WIN32 + if (watched) { + data = static_cast(GuestArena::GuestArenaAllocate_nid_postfix(bytes, BlockBytes)); + GuestArena::GuestArenaCommit_nid_postfix(data, bytes, PAGE_READWRITE, bytes); + } else { + data = static_cast(VirtualAlloc(nullptr, bytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); + } +#else + if (watched) { + void* raw = mmap(nullptr, bytes + BlockBytes, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + Require(raw != MAP_FAILED, "cannot map the color target"); + const auto begin = reinterpret_cast(raw); + const auto aligned = (begin + BlockBytes - 1) & ~(static_cast(BlockBytes) - 1); + if (aligned != begin) munmap(raw, aligned - begin); + if (aligned + bytes != begin + bytes + BlockBytes) munmap(reinterpret_cast(aligned + bytes), begin + BlockBytes - aligned); + data = reinterpret_cast(aligned); + GuestWriteWatch::GuestWriteWatchRegister_nid_postfix(data, bytes); + } else { + data = static_cast(std::aligned_alloc(BlockBytes, bytes)); + } +#endif + Require(data != nullptr, "cannot allocate the color target"); + Require(!watched || AgcDriver::GuestMemory::Watched(Address(), bytes), "the color target is not write-watched"); + GuestAllocations::Mutation mutation; + mutation.Add(data, bytes, true, true, true); + } + ~Block() { + AgcDriver::Graphics::StorageTexture::FlushPending(Address(), bytes, nullptr, "test release"); + { + GuestAllocations::Mutation mutation; + mutation.Remove(data); + } +#ifdef _WIN32 + if (watched) { + GuestArena::GuestArenaReset_nid_postfix(data, bytes); + GuestArena::GuestArenaRelease_nid_postfix(data, bytes); + } else { + VirtualFree(data, 0, MEM_RELEASE); + } +#else + if (watched) { + munmap(data, bytes); + GuestWriteWatch::GuestWriteWatchUnregister_nid_postfix(data, bytes); + } else { + std::free(data); + } +#endif + } + Block(const Block&) = delete; + Block& operator=(const Block&) = delete; + std::uint64_t Address() const { return reinterpret_cast(data); } + std::size_t bytes; + bool watched; + std::uint8_t* data = nullptr; +}; + +AgcDriver::Graphics::ColorTarget DecodeTarget(const Block& block) { + AgcDriver::Registers cx; + cx[0x318] = static_cast(block.Address() >> 8u); + cx[0x31b] = 0; + cx[0x31c] = SwapAlt8x8Info; + cx[0x31d] = 0; + cx[0x3b0] = ((Width - 1u) << 14u) | (Height - 1u); + cx[0x3b8] = TiledAttrib3; + cx[0x390] = static_cast(block.Address() >> 40u); + const auto color = AgcDriver::Graphics::DecodeColorBuffer(cx, 0); + Require(color.format == VK_FORMAT_R8G8_UNORM && color.componentMapping == 0xecu && color.elementBytes == 2 && color.bytes == block.bytes, "the 8_8 SWAP_ALT target did not decode as a 256x128 R8G8 surface holding the x and w exports"); + return color; +} + +VkPipelineColorBlendAttachmentState Blend(VkBlendFactor source, VkBlendFactor destination, VkBlendOp operation, VkBlendFactor sourceAlpha, VkBlendFactor destinationAlpha, VkBlendOp alphaOperation) { + VkPipelineColorBlendAttachmentState blend{}; + blend.blendEnable = VK_TRUE; + blend.srcColorBlendFactor = source; + blend.dstColorBlendFactor = destination; + blend.colorBlendOp = operation; + blend.srcAlphaBlendFactor = sourceAlpha; + blend.dstAlphaBlendFactor = destinationAlpha; + blend.alphaBlendOp = alphaOperation; + blend.colorWriteMask = 0xf; + return blend; +} + +void Draw(AgcDriver::VulkanDevice& device, const AgcDriver::Graphics::ColorTarget& color, const VkPipelineColorBlendAttachmentState& blend, std::span pixelCode) { + const auto target = device.Target(); + constexpr std::uint32_t waveSize = 64; + std::vector vertexUserData(4, 0u); + const auto address = reinterpret_cast(Triangle.data()); + const std::array vertexBuffer{static_cast(address), static_cast((address >> 32u) & 0xffffu) | (16u << 16u), static_cast(Triangle.size()), 0x01016facu}; + std::copy(vertexBuffer.begin(), vertexBuffer.end(), vertexUserData.begin()); + const std::array vertexMemory{{{reinterpret_cast(VertexCode.data()), std::as_bytes(std::span(VertexCode))}}}; + ShaderRecompiler::RecompileRequest vertex{ + {ShaderStage::Vertex, reinterpret_cast(VertexCode.data()), VertexCode, 0, {}}, + {waveSize, 0, vertexUserData, std::nullopt, std::nullopt, ShaderRecompiler::ShaderVertexStageInfo{}, vertexMemory}, + target, + {0, 0, 0, 64} + }; + vertex.useCache = false; + const auto vertexResult = ShaderRecompiler::Recompile(vertex); + const auto vertexPush = static_cast(vertexResult.pushConstants.size()); + + AgcDriver::Graphics::State state{}; + state.stages = {AgcDriver::Graphics::ShaderPath::Vertex, 0u, waveSize, waveSize, std::nullopt, std::nullopt}; + state.colors = {color}; + state.color = color; + state.hasColorTarget = true; + state.renderExtent = {Width, Height}; + state.topology = VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST; + state.viewport = {0, static_cast(Height), static_cast(Width), -static_cast(Height), 0, 1}; + state.negativeOneToOne = false; + state.scissor = {{0, 0}, {ScissorWidth, Height}}; + state.cullMode = VK_CULL_MODE_NONE; + state.frontFace = VK_FRONT_FACE_COUNTER_CLOCKWISE; + state.blends = {blend}; + state.blend = blend; + state.blendConstants = {}; + + const auto codeAddress = reinterpret_cast(pixelCode.data()); + const AgcDriver::Registers context{{0x1b6, 0x8000}, {0x1b3, 0}, {0x1b4, 0}, {0x203, 0}, {0x1c5, Float32Export}}; + const std::array pixelMemory{{{codeAddress, std::as_bytes(pixelCode)}}}; + ShaderRecompiler::RecompileRequest fragment{ + {ShaderStage::Fragment, codeAddress, pixelCode, 0, {}}, + {waveSize, 0, {}, std::nullopt, AgcDriver::Graphics::DecodePixelStageInfo(context, AgcDriver::Graphics::ExportMappings(state)), std::nullopt, pixelMemory}, + target, + PixelPushLayout(vertexPush, target) + }; + fragment.useCache = false; + const auto pixelResult = ShaderRecompiler::Recompile(fragment); + const std::array shaders{{ + {ShaderStage::Vertex, &vertexResult, 0}, + {ShaderStage::Fragment, &pixelResult, PixelPushOffset(vertexPush, target)} + }}; + const AgcDriver::Pm4::DrawParameters draw{0, static_cast(Triangle.size()), 0, 1, 0, false}; + device.Draw(state, draw, shaders); +} + +std::vector Fill(Block& block, const ColorTargetLayout& layout) { + std::vector bytes(block.bytes); + for (std::uint32_t y = 0; y < Height; ++y) { + for (std::uint32_t x = 0; x < Width; ++x) { + const auto& color = Destinations[(x + y) % Destinations.size()]; + const auto index = layout.Offset(x, y); + bytes[index] = color[0]; + bytes[index + 1] = color[1]; + } + } + std::memcpy(block.data, bytes.data(), block.bytes); + return bytes; +} + +void Check(AgcDriver::VulkanDevice& device, const Block& block, const ColorTargetLayout& layout, const std::vector& before, const std::function(float first, float second)>& blended, const std::string& what) { + AgcDriver::Graphics::StorageTexture::FlushPending(block.Address(), block.bytes, nullptr, "test read-back"); + device.WaitIdle(); + std::vector stored(block.bytes); + AgcDriver::GuestMemory::Read(block.Address(), std::as_writable_bytes(std::span(stored)), 1); + for (std::uint32_t y = 0; y < Height; ++y) { + for (std::uint32_t x = 0; x < Width; ++x) { + const auto index = layout.Offset(x, y); + const auto& destination = Destinations[(x + y) % Destinations.size()]; + const auto expected = blended(static_cast(destination[0]) / 255.0f, static_cast(destination[1]) / 255.0f); + for (std::uint32_t channel = 0; channel < 2; ++channel) { + const bool inside = x < ScissorWidth; + const auto want = std::clamp(expected[channel], 0.0f, 1.0f) * 255.0f; + const bool match = inside ? std::fabs(static_cast(stored[index + channel]) - want) <= 1.0f : stored[index + channel] == before[index + channel]; + if (match) continue; + char detail[160]; + std::snprintf(detail, sizeof(detail), ": channel %u of pixel (%u, %u) %s the scissor holds %u, expected %.2f", channel, x, y, inside ? "inside" : "outside", stored[index + channel], inside ? want : static_cast(before[index + channel])); + throw std::runtime_error(what + detail); + } + } + } +} + +struct BlendCase { + const char* name; + VkPipelineColorBlendAttachmentState blend; + bool split; + std::function(float first, float second)> blended; +}; + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + const ColorTargetLayout layout(Width, Height, ColorTileMode::RenderTarget, 2); + Block block(layout.Bytes()); + const Color exported{0.25f, 0.75f, 0.125f, 0.5f}; + const float x = exported[0]; + const float w = exported[3]; + const std::array cases{{ + {"SRC_ALPHA + ONE_MINUS_SRC_ALPHA, alpha ONE + ZERO", Blend(VK_BLEND_FACTOR_SRC_ALPHA, VK_BLEND_FACTOR_ONE_MINUS_SRC_ALPHA, VK_BLEND_OP_ADD, VK_BLEND_FACTOR_ONE, VK_BLEND_FACTOR_ZERO, VK_BLEND_OP_ADD), true, + [&](float r, float) { return std::array{x * w + r * (1.0f - w), w}; }}, + {"ONE + ONE, alpha ZERO + ONE_MINUS_SRC_ALPHA", Blend(VK_BLEND_FACTOR_ONE, VK_BLEND_FACTOR_ONE, VK_BLEND_OP_ADD, VK_BLEND_FACTOR_ZERO, VK_BLEND_FACTOR_ONE_MINUS_SRC_ALPHA, VK_BLEND_OP_ADD), true, + [&](float r, float g) { return std::array{x + r, g * (1.0f - w)}; }}, + {"ONE + ZERO, alpha ONE reverse-subtract ONE", Blend(VK_BLEND_FACTOR_ONE, VK_BLEND_FACTOR_ZERO, VK_BLEND_OP_ADD, VK_BLEND_FACTOR_ONE, VK_BLEND_FACTOR_ONE, VK_BLEND_OP_REVERSE_SUBTRACT), true, + [&](float, float g) { return std::array{x, g - w}; }}, + {"ONE + ONE_MINUS_SRC_ALPHA for color and alpha", Blend(VK_BLEND_FACTOR_ONE, VK_BLEND_FACTOR_ONE_MINUS_SRC_ALPHA, VK_BLEND_OP_ADD, VK_BLEND_FACTOR_ONE, VK_BLEND_FACTOR_ONE_MINUS_SRC_ALPHA, VK_BLEND_OP_ADD), false, + [&](float r, float g) { return std::array{x + r * (1.0f - w), w + g * (1.0f - w)}; }}, + }}; + for (const auto& test : cases) { + const auto before = Fill(block, layout); + const auto color = DecodeTarget(block); + AgcDriver::Graphics::State probe{}; + probe.colors = {color}; + probe.blends = {test.blend}; + Require(AgcDriver::Graphics::SplitsAlternateAlpha(probe) == test.split, (std::string(test.name) + ": the split decision is wrong").c_str()); + Draw(*device, color, test.blend, Program(exported)); + Check(*device, block, layout, before, test.blended, test.name); + } + std::puts("8_8 SWAP_ALT blend tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} From 250b617f14562e96337b3398bed0e8e2091c1c63 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 04:00:52 +0200 Subject: [PATCH 311/460] docs: record the two-pass blend into 8_8 SWAP_ALT targets --- docs/dev/ARCHITECTURE.md | 2 +- docs/dev/TechnicalDebt.md | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/dev/ARCHITECTURE.md b/docs/dev/ARCHITECTURE.md index 5f2ffb9cf..32e6742d8 100644 --- a/docs/dev/ARCHITECTURE.md +++ b/docs/dev/ARCHITECTURE.md @@ -84,7 +84,7 @@ flowchart LR - The shader registry maps a code address to every header registered for that code, in registration order. Static, graphics and rectangle ABI resolution pick the header at the bound address (or one equal to it, `SameHeader`); draws and dispatches pick the program by the stage's binary type (`RegisteredProgram`), and a front entry matching both type 2 and type 4 throws. Within a type the last registration wins; re-registering an identical earlier header makes its snapshot current again without preparing it again, and changed code drops the older headers. - The snapshot is published as pending. `SourceHandleFor`, `InvocationFor` and `ShaderPreparationTransaction::Edit` / `Read` wait for the pending job before reading `PreparedShaders`, and rethrow its failure. `APS5_SYNC_SHADER_PREPARE=1` prepares on the registering thread instead. - A draw or dispatch whose state matches no artifact prepared at registration (a title that builds its static ABI state itself, without `sceAgcDriverResolve*Abi`) gets one prepared at use by `SourceHandleFor` or `InvocationFor`, logged once per artifact and kept in the snapshot for the next use. -- A draw's `Graphics::Pipeline` comes from `CachedPipeline`. When the device has `VK_EXT_graphics_pipeline_library` with fast linking, `VK_EXT_extended_dynamic_state` and `VK_KHR_dynamic_rendering`, draws render with `vkCmdBeginRenderingKHR` instead of render passes and the pipeline is linked from four libraries (vertex input, pre-rasterization shaders, fragment shader, fragment output) kept per device in [`PipelineLibrary.cpp`](../../core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp). A library built for one pipeline is reused by the next: another blend, vertex layout or target format builds only that part (the shader libraries name no attachment format), and cull mode, front face, depth and stencil state are dynamic, so changing them only links. The fast link is what the draw waits for; a background thread then links the same libraries with link-time optimization and the pipeline binds that one from the next draw on. Push constants then use a 256-byte block with a 128-byte slot per side: the pre-rasterization stages stack from byte 0, the pixel shader always starts at byte 128 (`StagePushOffset`), so its library does not depend on how much push data its vertex shader uses. Pipelines with per-face depth bias, rect lists, geometry shaders or a stage without a variant id are created whole, still with dynamic rendering; devices without the extensions (or with `APS5_NO_GPL=1`) keep render passes and whole pipelines. +- A draw's `Graphics::Pipeline` comes from `CachedPipeline`. When the device has `VK_EXT_graphics_pipeline_library` with fast linking, `VK_EXT_extended_dynamic_state` and `VK_KHR_dynamic_rendering`, draws render with `vkCmdBeginRenderingKHR` instead of render passes and the pipeline is linked from four libraries (vertex input, pre-rasterization shaders, fragment shader, fragment output) kept per device in [`PipelineLibrary.cpp`](../../core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp). A library built for one pipeline is reused by the next: another blend, vertex layout or target format builds only that part (the shader libraries name no attachment format), and cull mode, front face, depth and stencil state are dynamic, so changing them only links. The fast link is what the draw waits for; a background thread then links the same libraries with link-time optimization and the pipeline binds that one from the next draw on. Push constants then use a 256-byte block with a 128-byte slot per side: the pre-rasterization stages stack from byte 0, the pixel shader always starts at byte 128 (`StagePushOffset`), so its library does not depend on how much push data its vertex shader uses. Pipelines with per-face depth bias, a separate alpha blend into an 8_8 target with the alternate component order (both draw twice), rect lists, geometry shaders or a stage without a variant id are created whole, still with dynamic rendering; devices without the extensions (or with `APS5_NO_GPL=1`) keep render passes and whole pipelines. ### Shader MODE register access diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 237d82bed..637376a05 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -315,6 +315,7 @@ Throughout the project, every function at every stage either **does exactly what - [Windows file identity](../../core/libs/prx/libkernel/File/src/NativeStat.cpp) (libkernel) - `stat` and `fstat` report the volume serial number as `st_dev` and the 64-bit file index folded into the guest's 32-bit `st_ino` (never 0), so two different files can in principle share an `st_ino`; the fold is not taken from a reference. `st_nlink` is clamped to 65535, the width of the guest field. The call keeps the `_wstat64` values when `GetFileInformationByHandle` fails - On Linux, [direct memory](../../core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp) lives in one sparse memfd at file offset = physical address, so the [GPU imports](../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp) can share one udmabuf buffer object per 64 MiB of it. Released direct memory is not punched out of the file, because a live import of its chunk pins the pages and the GPU would keep reading the old ones: its pages stay committed (the process keeps the high-water mark of the direct memory it used) and are zeroed when the range is allocated again. - [Per-face depth bias](../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver): Vulkan has one depth bias per draw, so a draw that rasterizes both faces with different front and back bias (PA_SU_SC_MODE_CNTL, PA_SU_POLY_OFFSET_FRONT/BACK_*) is recorded twice, front faces with the back faces culled and then back faces with the front faces culled. The vertex stages run twice, so a vertex shader that writes memory writes twice, and all front faces land before all back faces instead of in primitive order, which differs only for blending or stencil updates that depend on the order. +- [Separate alpha blend into 8_8 SWAP_ALT targets](../../core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp) (libSceAgcDriver): the second channel of a two-channel target with `COMP_SWAP` 1 holds the alpha export, but Vulkan blends it with the color equation of the R8G8 attachment, so a draw whose alpha equation differs from its color equation is recorded twice, the first channel with the color equation and the second with the alpha factors and operation moved into the color equation. The vertex stages run twice, so a vertex shader that writes memory writes twice, and occlusion counts include both passes. Draws that write depth or stencil, pixel shaders that write memory and factors of the destination alpha throw. That the hardware blends the second channel with the alpha equation is taken from Mesa's `si_translate_colorswap` and radeonsi's use of it for luminance-alpha formats, not measured on hardware. - [Compute LDS beyond the device's shared-memory limit](../../core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp) (libSceAgcDriver) lives in a coherent storage buffer with one slice of `LDS bytes / 4 + 1` dwords per workgroup (the last dword is the atomic lock), shared by all such dispatches and grown in 1 MiB steps up to `maxStorageBufferRange`; a dispatch whose slices do not fit throws. It is slower than shared memory and its cost was not measured. An indirect dispatch of such a program always reads its group counts on the CPU. Its results are pinned against the in-limit semantics in `agc_driver_lds_beyond_device_limit`, not measured on hardware. - [Graphics pipeline libraries](../../core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp) (libSceAgcDriver): a pipeline is fast-linked at the draw and relinked with `VK_PIPELINE_CREATE_LINK_TIME_OPTIMIZATION_BIT_EXT` on a background thread, and binds the optimized one from the next draw on. Draws recorded before that use the fast link, whose vertex stage keeps the outputs the fragment stage does not read: on RADV (Radeon 660M, Mesa 26.2.4) 346 of 483 Astro's Playroom vertex shaders paired with a fragment stage that reads none of them used more VGPRs (median 8 more, up to 2.5 times), and on an RTX 4090 13 of 15 PPSA36082 vertex shaders up to 68 instead of 38 (#2242). The optimized link compiles to the same code as a whole pipeline - [sceAvPlayerGetVideoData](../../core/libs/prx/libSceAvPlayer/src/Source.cpp) (libSceAvPlayer) in sync mode `SCE_AVPLAYER_AV_SYNC_MODE_NONE` skips queued frames that the audio clock has already passed while the title pulls audio, so a title that requests video slower than the film's frame rate keeps the picture with the sound. Whether the console hands out every frame in this mode is unknown. PPSA12544 requests 7-19 frames per second for a 30 fps cinematic From 1671179c4fb52ec132bd57b372858f1c3badfb30 Mon Sep 17 00:00:00 2001 From: David Date: Sun, 11 Oct 2026 01:20:55 +0200 Subject: [PATCH 312/460] feat(libc): implement random, srandom, initstate and setstate Port FreeBSD 11 lib/libc/stdlib/random.c (freebsd-src 4ab2e064): the additive feedback generator with the default 128-byte TYPE_3 table, good_rand seeding, state sizes rounded down to 8, 32, 64, 128 or 256 bytes, setstate returning the previous state, and initstate returning null below 8 bytes without setting errno. Calls are serialised by one lock; a null state throws. The import audit of the homebrew catalog on main 262c0316 lists them as absent libSceLibcInternal imports: - random: PPSA74126 Stremio Plus, PPSA85300 VLC, PPSA99169 RetroArch, PPSA99177 PS5VR, PPSA99420 PS5MC, PPSA99505 Jelly5, PPSA99515 Emby5, PPSA99810 PSBox Cloud Gaming - srandom: PPSA85300, PPSA99169, PPSA99420, PPSA99810 - initstate, setstate: PPSA99177, PPSA99420, PPSA99505, PPSA99515 Co-Authored-By: Claude Fable 5.1 --- core/libs/prx/libc/CMakeLists.txt | 10 ++ core/libs/prx/libc/src/Random.cpp | 146 +++++++++++++++++++++++++ core/libs/tests/GuestRandomState.cpp | 154 +++++++++++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 311 insertions(+) create mode 100644 core/libs/prx/libc/src/Random.cpp create mode 100644 core/libs/tests/GuestRandomState.cpp diff --git a/core/libs/prx/libc/CMakeLists.txt b/core/libs/prx/libc/CMakeLists.txt index 9538fce00..53df25bad 100644 --- a/core/libs/prx/libc/CMakeLists.txt +++ b/core/libs/prx/libc/CMakeLists.txt @@ -12,6 +12,7 @@ target_sources(libc PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src/RuneLocale.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/StringExtensions.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/Math.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/src/Random.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/Time.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/FileAndHeap.cpp ${CMAKE_CURRENT_SOURCE_DIR}/src/StdioExtras.cpp @@ -148,6 +149,15 @@ if(BUILD_TESTING) set_tests_properties(guest_math PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_random_state_tests ${LIBS_INCLUDE_DIR}/tests/GuestRandomState.cpp) + target_include_directories(guest_random_state_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_random_state_tests PRIVATE libc) + configure_windows_unwind(guest_random_state_tests) + add_test(NAME guest_random_state COMMAND guest_random_state_tests) + if(WIN32) + set_tests_properties(guest_random_state PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_execute_once_tests ${LIBS_INCLUDE_DIR}/tests/GuestExecuteOnce.cpp) target_include_directories(guest_execute_once_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_execute_once_tests PRIVATE libc) diff --git a/core/libs/prx/libc/src/Random.cpp b/core/libs/prx/libc/src/Random.cpp new file mode 100644 index 000000000..c5128a6ea --- /dev/null +++ b/core/libs/prx/libc/src/Random.cpp @@ -0,0 +1,146 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include + +namespace { + +constexpr int MaxTypes = 5; +constexpr int LinearCongruentialType = 0; +constexpr int LinearCongruentialShuffles = 50; +constexpr std::array MinimumStateBytes {8, 32, 64, 128, 256}; +constexpr std::array Degrees {0, 7, 15, 31, 63}; +constexpr std::array Separations {0, 3, 1, 3, 1}; + +std::uint32_t g_defaultTable[] { + 3, + 0x2cf41758, 0x27bb3711, 0x4916d4d1, 0x7b02f59f, 0x9b8e28eb, 0xc0e80269, + 0x696f5c16, 0x878f1ff5, 0x52d9c07f, 0x916a06cd, 0xb50b3a20, 0x2776970a, + 0xee4eb2a6, 0xe94640ec, 0xb1d65612, 0x9d1ed968, 0x1043f6b7, 0xa3432a76, + 0x17eacbb9, 0x3c09e2eb, 0x04f8c2b3, 0x708a1f57, 0xee341814, 0x95d0e4d2, + 0xb06f216c, 0x8bd2e72e, 0x8f7c38d7, 0xcfc6a8fc, 0x02a59495, 0xa20d2a69, + 0xe29d12d1 +}; + +struct AdditiveFeedbackState { + char* table; + int type; + int degree; + int separation; + int front; + int rear; +}; + +std::mutex g_randomLock; +AdditiveFeedbackState g_random {reinterpret_cast(g_defaultTable), 3, 31, 3, 3, 0}; + +std::uint32_t LoadWord(const char* table, int index) { + std::uint32_t value; + std::memcpy(&value, table + sizeof(value) * static_cast(index), sizeof(value)); + return value; +} + +void StoreWord(char* table, int index, std::uint32_t value) { + std::memcpy(table + sizeof(value) * static_cast(index), &value, sizeof(value)); +} + +std::uint32_t GoodRand(std::uint32_t context) { + std::int32_t x = static_cast(context % 0x7ffffffeu) + 1; + const std::int32_t high = x / 127773; + const std::int32_t low = x % 127773; + x = 16807 * low - 2836 * high; + if (x < 0) x += 0x7fffffff; + return static_cast(x - 1); +} + +std::uint32_t NextLocked() { + auto& state = g_random; + if (state.type == LinearCongruentialType) { + const std::uint32_t value = GoodRand(LoadWord(state.table, 1)); + StoreWord(state.table, 1, value); + return value; + } + const std::uint32_t sum = LoadWord(state.table, 1 + state.front) + LoadWord(state.table, 1 + state.rear); + StoreWord(state.table, 1 + state.front, sum); + if (++state.front >= state.degree) { + state.front = 0; + ++state.rear; + } else if (++state.rear >= state.degree) { + state.rear = 0; + } + return sum >> 1; +} + +void SeedLocked(std::uint32_t seed) { + auto& state = g_random; + StoreWord(state.table, 1, seed); + int rounds = LinearCongruentialShuffles; + if (state.type != LinearCongruentialType) { + for (int i = 1; i < state.degree; ++i) StoreWord(state.table, 1 + i, GoodRand(LoadWord(state.table, i))); + state.front = state.separation; + state.rear = 0; + rounds = 10 * state.degree; + } + for (int i = 0; i < rounds; ++i) NextLocked(); +} + +void SaveTypeWordLocked() { + const auto& state = g_random; + const int word = state.type == LinearCongruentialType ? state.type : MaxTypes * state.rear + state.type; + StoreWord(state.table, 0, static_cast(word)); +} + +} + +extern "C" { + +std::int64_t APS5_VABI random_nid_postfix() { + std::lock_guard lock(g_randomLock); + return static_cast(NextLocked()); +} + +void APS5_VABI srandom_nid_postfix(std::uint64_t seed) { + std::lock_guard lock(g_randomLock); + SeedLocked(static_cast(seed)); +} + +char* APS5_VABI initstate_nid_postfix(std::uint64_t seed, char* state, std::int64_t size) { + if (size < MinimumStateBytes[0]) return nullptr; + if (state == nullptr) throw std::invalid_argument("initstate: null state"); + std::lock_guard lock(g_randomLock); + char* previous = g_random.table; + SaveTypeWordLocked(); + int type = MaxTypes - 1; + while (size < MinimumStateBytes[static_cast(type)]) --type; + g_random = {state, type, Degrees[static_cast(type)], Separations[static_cast(type)], 0, 0}; + SeedLocked(static_cast(seed)); + SaveTypeWordLocked(); + return previous; +} + +char* APS5_VABI setstate_nid_postfix(char* state) { + if (state == nullptr) throw std::invalid_argument("setstate: null state"); + std::lock_guard lock(g_randomLock); + const std::uint32_t word = LoadWord(state, 0); + const int type = static_cast(word % MaxTypes); + const std::uint32_t rear = word / MaxTypes; + const int degree = Degrees[static_cast(type)]; + if (type != LinearCongruentialType && rear >= static_cast(degree)) return nullptr; + char* previous = g_random.table; + SaveTypeWordLocked(); + const int separation = Separations[static_cast(type)]; + g_random.table = state; + g_random.type = type; + g_random.degree = degree; + g_random.separation = separation; + if (type != LinearCongruentialType) { + g_random.rear = static_cast(rear); + g_random.front = (static_cast(rear) + separation) % degree; + } + return previous; +} + +} diff --git a/core/libs/tests/GuestRandomState.cpp b/core/libs/tests/GuestRandomState.cpp new file mode 100644 index 000000000..8e79c298f --- /dev/null +++ b/core/libs/tests/GuestRandomState.cpp @@ -0,0 +1,154 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include + +extern "C" { +std::int64_t APS5_VABI random_nid_postfix(); +void APS5_VABI srandom_nid_postfix(std::uint64_t); +char* APS5_VABI initstate_nid_postfix(std::uint64_t, char*, std::int64_t); +char* APS5_VABI setstate_nid_postfix(char*); +int* APS5_VABI __error_nid_postfix(); +} + +static int failures = 0; + +static void Require(bool condition, const char* message) { + if (!condition) { + std::fprintf(stderr, "%s\n", message); + ++failures; + } +} + +using Sequence = std::array; + +static Sequence Draw() { + Sequence values{}; + for (auto& value : values) value = random_nid_postfix(); + return values; +} + +static std::uint32_t Word(const char* table, std::size_t index) { + std::uint32_t value; + std::memcpy(&value, table + sizeof(value) * index, sizeof(value)); + return value; +} + +int main() { + constexpr Sequence seedOne {1408992891, 1638182910, 83848093, 145962118, 627851256}; + constexpr Sequence seed12345 {1052984060, 1639322404, 1971525204, 2097956667, 2012147915}; + constexpr std::array seed42ByType {{ + {2060028338, 1164936638, 481681973, 1763071474, 962919018}, + {660273387, 1356916325, 287587791, 1132382070, 1497594867}, + {1015209508, 12952757, 1418769128, 2005808989, 1212998434}, + {745791173, 418148062, 949332769, 1480536337, 177711995}, + {996638292, 1209745284, 1148231710, 400392963, 1149215164}, + }}; + constexpr std::array seed7Type1 {1271472800, 27686551, 1252069488, 1609001632, 28873941, 508017694}; + + Require(Draw() == seedOne, "unseeded random must match srandom(1)"); + srandom_nid_postfix(1); + Require(Draw() == seedOne, "srandom(1) sequence"); + srandom_nid_postfix(0x100000001ull); + Require(Draw() == seedOne, "srandom must use the low 32 bits of the seed"); + srandom_nid_postfix(12345); + Require(Draw() == seed12345, "srandom(12345) sequence"); + + struct SizeCase { + std::int64_t size; + int type; + std::size_t usedBytes; + }; + constexpr std::array sizes {{ + {8, 0, 8}, {31, 0, 8}, {32, 1, 32}, {63, 1, 32}, {64, 2, 64}, + {127, 2, 64}, {128, 3, 128}, {255, 3, 128}, {256, 4, 256}, {1000, 4, 256}, + }}; + std::vector> buffers; + char* defaultTable = nullptr; + char* active = nullptr; + for (const auto& sizeCase : sizes) { + auto& buffer = buffers.emplace_back(static_cast(sizeCase.size) + 4, static_cast(0xa5)); + char* previous = initstate_nid_postfix(42, buffer.data(), sizeCase.size); + if (defaultTable == nullptr) defaultTable = previous; + else Require(previous == active, "initstate must return the previous state"); + Require(Word(buffer.data(), 0) == static_cast(sizeCase.type), "initstate type word"); + Require(Draw() == seed42ByType[static_cast(sizeCase.type)], "initstate(42) sequence for its size"); + Require(std::all_of(buffer.begin() + static_cast(sizeCase.usedBytes), buffer.end(), + [](char value) { return value == static_cast(0xa5); }), "initstate must use only the rounded state size"); + active = buffer.data(); + } + Require(defaultTable != nullptr && Word(defaultTable, 0) == 5 * 5 + 3, "default table must hold its rear position"); + Require(setstate_nid_postfix(defaultTable) == active, "setstate must return the previous state"); + Require(random_nid_postfix() == 1533042628 && random_nid_postfix() == 1225903521, + "setstate on the default table must continue the srandom(12345) sequence"); + + alignas(4) std::array small{}; + alignas(4) std::array large{}; + initstate_nid_postfix(7, small.data(), static_cast(small.size())); + for (std::size_t i = 0; i < 3; ++i) Require(random_nid_postfix() == seed7Type1[i], "initstate(7) on 32 bytes"); + initstate_nid_postfix(7, large.data(), static_cast(large.size())); + Require(setstate_nid_postfix(small.data()) == large.data(), "setstate must return the 64-byte state"); + Require(setstate_nid_postfix(large.data()) == small.data(), "setstate must return the 32-byte state"); + Require(Word(small.data(), 0) == 5 * 3 + 1, "setstate must save the rear position of the old state"); + Require(setstate_nid_postfix(small.data()) == large.data(), "setstate back to the 32-byte state"); + for (std::size_t i = 3; i < seed7Type1.size(); ++i) Require(random_nid_postfix() == seed7Type1[i], "setstate must resume the sequence"); + + initstate_nid_postfix(7, small.data(), static_cast(small.size())); + random_nid_postfix(); + alignas(4) std::array garbled{}; + const std::uint32_t garbledWord = 5 * 7 + 1; + std::memcpy(garbled.data(), &garbledWord, sizeof(garbledWord)); + Require(setstate_nid_postfix(garbled.data()) == nullptr, "setstate must reject a rear position past the degree"); + Require(random_nid_postfix() == seed7Type1[1], "a rejected setstate must keep the current state"); + + *__error_nid_postfix() = 1234; + std::array tiny{}; + tiny.fill(static_cast(0x5a)); + for (const std::int64_t size : {std::int64_t{7}, std::int64_t{0}, std::int64_t{-1}}) + Require(initstate_nid_postfix(1, tiny.data(), size) == nullptr, "initstate below 8 bytes must return null"); + Require(initstate_nid_postfix(1, nullptr, 4) == nullptr, "initstate below 8 bytes must return null before reading the state"); + Require(*__error_nid_postfix() == 1234, "initstate below 8 bytes must leave errno unchanged"); + Require(std::all_of(tiny.begin(), tiny.end(), [](char value) { return value == static_cast(0x5a); }), + "initstate below 8 bytes must not write the state"); + Require(random_nid_postfix() == seed7Type1[2], "a rejected initstate must keep the current state"); + + bool threw = false; + try { + initstate_nid_postfix(1, nullptr, 128); + } catch (const std::invalid_argument&) { + threw = true; + } + Require(threw, "initstate with a null state must throw"); + threw = false; + try { + setstate_nid_postfix(nullptr); + } catch (const std::invalid_argument&) { + threw = true; + } + Require(threw, "setstate with a null state must throw"); + + constexpr std::size_t threadCount = 4; + constexpr std::size_t callsPerThread = 2000; + srandom_nid_postfix(99); + std::vector serial(threadCount * callsPerThread); + for (auto& value : serial) value = random_nid_postfix(); + srandom_nid_postfix(99); + std::vector concurrent(threadCount * callsPerThread); + std::vector threads; + for (std::size_t t = 0; t < threadCount; ++t) + threads.emplace_back([&concurrent, t] { + for (std::size_t i = 0; i < callsPerThread; ++i) concurrent[t * callsPerThread + i] = random_nid_postfix(); + }); + for (auto& thread : threads) thread.join(); + std::sort(serial.begin(), serial.end()); + std::sort(concurrent.begin(), concurrent.end()); + Require(serial == concurrent, "concurrent calls must draw the serial sequence"); + + return failures == 0 ? 0 : 1; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index ea6808edc..91f4161eb 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -517,3 +517,4 @@ Throughout the project, every function at every stage either **does exactly what - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert +- [random, srandom, initstate, setstate](../../core/libs/prx/libc/src/Random.cpp) (libc) follow FreeBSD 11 [random.c](https://github.com/freebsd/freebsd-src/blob/4ab2e064d7950be84256d671a7ae93f87cc6aa36/lib/libc/stdlib/random.c): the additive feedback generator on a default 128-byte `TYPE_3` table, `good_rand` seeding, state sizes rounded down to 8, 32, 64, 128 or 256 bytes, and `setstate` returning the previous state or null for a rear position past the degree. `initstate` below 8 bytes returns null without setting errno, as that file and its manual do (glibc sets `EINVAL`). FreeBSD does not lock the generator; here every call is serialised by one lock, so concurrent callers draw the serial sequence between them. A null state throws, where FreeBSD faults. That the console's `libSceLibcInternal` uses this FreeBSD code and not another generator is unverified From b1267b33886668510f33a707f6ce62402b268eef Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 04:07:37 +0200 Subject: [PATCH 313/460] docs(technical-debt): list the random family under unknown function info Co-Authored-By: Claude --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 91f4161eb..ed3724fc9 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -169,6 +169,7 @@ Throughout the project, every function at every stage either **does exactly what - [acosl](../../core/libs/prx/libc/src/Math.cpp) (libc) - computed by the host's math library, like the other libc math functions here, not by the console's (FreeBSD msun ld80 code in Sony's `libc.prx`): results can differ from the console's in the last bit, and on Linux glibc sets `errno` to `EDOM` for `acosl` outside [-1, 1] where msun only raises the floating-point exception. `long double` is the x87 80-bit format, as Sony's `frexpl` uses it (`fldt 0x8(%rsp)`, result in `st(0)`) - [std_execute_once](../../core/libs/prx/libc/Export.cpp) (libc) - assumed to be the plain-named twin of `std::_Execute_once` (`_ZSt13_Execute_onceRSt9once_flagPFiPvS1_PS1_ES1_`, the callee of Dinkumware `call_once`): it delegates there and gets the [thread.once.callonce] semantics already verified for that symbol (one invocation per flag, concurrent callers block until it completes, and a failing or throwing invocation leaves the flag unready so the next caller runs it). The name equivalence is not confirmed on a title, and the callback convention (called with (null, arg, null), non-zero return meaning success) is the twin's, unverified for this symbol - [Dinkumware mutexes and conditions](../../core/libs/prx/libc/src/Threads.cpp) (libc) use pointer-to-handle arguments, matching the console aerolib signatures reported in [#3079](https://github.com/boykopovar/AnyPS5/pull/3079). The title's `_Mtx_lock` maps `0` to `_Thrd_success` (0), `0x8002000b` to `_Thrd_busy` (3), and other failures to `_Thrd_error` (4); its internal mutex init recognizes recursive flag `0x100`, while every other type uses default attributes. The observed relock failure for a plain mutex suggests those defaults are error-checking, but that has not been confirmed on hardware. The title's `_Mtx_unlock` and `_Cnd_wait` return success regardless of their internal result. The host implementation treats invalid ownership preconditions as no-ops, and throws for invalid handles, host wait errors and destroying a locked mutex as debug guards; those cases are not confirmed against the console. +- [random, srandom, initstate, setstate](../../core/libs/prx/libc/src/Random.cpp) (libc) follow FreeBSD 11 [random.c](https://github.com/freebsd/freebsd-src/blob/4ab2e064d7950be84256d671a7ae93f87cc6aa36/lib/libc/stdlib/random.c): the additive feedback generator on a default 128-byte `TYPE_3` table, `good_rand` seeding, state sizes rounded down to 8, 32, 64, 128 or 256 bytes, and `setstate` returning the previous state or null for a rear position past the degree. `initstate` below 8 bytes returns null without setting errno, as that file and its manual do (glibc sets `EINVAL`). FreeBSD does not lock the generator; here every call is serialised by one lock, so concurrent callers draw the serial sequence between them. A null state throws, where FreeBSD faults. That the console's `libSceLibcInternal` uses this FreeBSD code and not another generator is unverified - [tcgetattr, tcsetattr](../../core/libs/prx/libkernel/File/src/Terminal.cpp) (libkernel) - every file and pipe descriptor is taken as not a terminal (`ENOTTY`), and a socket gets `EADDRNOTAVAIL` for `AF_INET` and `EOPNOTSUPP` otherwise, as FreeBSD's `in_control` and `pru_control_notsupp` answer a termios ioctl; what descriptors 0 to 2 of a title are on the console, and whether any of them is a terminal there, is unknown - [__progname](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - unknown data export - [environ](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - data export pointing to a fixed empty environment so titles that import it load; it is not the environment libc's `getenv` reads, so it lists none of those variables and `setenv`/`putenv` changes do not appear in it @@ -517,4 +518,3 @@ Throughout the project, every function at every stage either **does exactly what - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert -- [random, srandom, initstate, setstate](../../core/libs/prx/libc/src/Random.cpp) (libc) follow FreeBSD 11 [random.c](https://github.com/freebsd/freebsd-src/blob/4ab2e064d7950be84256d671a7ae93f87cc6aa36/lib/libc/stdlib/random.c): the additive feedback generator on a default 128-byte `TYPE_3` table, `good_rand` seeding, state sizes rounded down to 8, 32, 64, 128 or 256 bytes, and `setstate` returning the previous state or null for a rear position past the degree. `initstate` below 8 bytes returns null without setting errno, as that file and its manual do (glibc sets `EINVAL`). FreeBSD does not lock the generator; here every call is serialised by one lock, so concurrent callers draw the serial sequence between them. A null state throws, where FreeBSD faults. That the console's `libSceLibcInternal` uses this FreeBSD code and not another generator is unverified From 70a6dde689f85c78ade359e36e1254af44b304f2 Mon Sep 17 00:00:00 2001 From: raghav314 Date: Sun, 11 Oct 2026 07:40:41 +0530 Subject: [PATCH 314/460] perf(agc): continue render passes across draws with only a fault buffer A recorded draw counted as reading and writing any guest memory when its resources needed completion work, which a fault buffer alone (guarded descriptor slots, rect lists) also needs. Such a draw reaches no guest memory through it: the buffer is the draw's own, written only on a fault. Its pass hazards now come from its known ranges and images like any other draw; address-based draws, draws through the BDA table and draws with copied writes still count as touching any guest memory. Co-Authored-By: Claude Opus 5.5 --- .../prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp | 1 + core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp | 3 ++- core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp | 2 ++ 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp index ee1a0a5c5..28370a397 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp @@ -149,6 +149,7 @@ public: // Whether a written buffer was copied (its results reach guest memory by the CPU write-back). bool HasCopiedWrites() const { return guestMemory.HasCopiedWrites(); } bool HoldsLease() const { return guestMemory.HoldsLease(); } + bool UsesAddressTable() const { return usesBda; } std::size_t CopiedBytes() const { return guestMemory.CopiedBytes(); } bool WritesOverlap(std::uint64_t address, std::size_t bytes) const { return guestMemory.WritesOverlap(address, bytes); } // Whether a region the recorded work reads in place through a host import overlaps the range. diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp index a36593d10..344261741 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp @@ -1668,7 +1668,8 @@ void recordDraw(const Context& context, const State& state, const Pm4::DrawParam for (const auto& target : record.targets) { if (target != nullptr) passAttachments.emplace_back(target->Image(), true); } - const PassAccess passAccess{passReads, resources.GpuWrites(), passImages, passAttachments, addressBased || resources.NeedsCompletion(), addressBased || resources.NeedsCompletion()}; + const bool unknownMemory = addressBased || resources.UsesAddressTable() || resources.HasCopiedWrites(); + const PassAccess passAccess{passReads, resources.GpuWrites(), passImages, passAttachments, unknownMemory, unknownMemory}; const bool fenced = state.depth.has_value() || !record.proxies.empty(); const auto start = recorder->StartDrawPass(passKey, capture || readsTarget || gpuIndirect || meshIndirect, fenced, passAccess); const bool continued = start.continued; diff --git a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp index fa8240bd6..ce9d84870 100644 --- a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp @@ -667,6 +667,7 @@ void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { const std::array unusedSnapshots{{{0, source}}}; ShaderResources resources(disabled, stages, ColorTarget{}, 0, 0, 0, unusedSnapshots); Require(ShaderResources::NeverReusable(stages) && !resources.Reusable(), "a fault-buffer build is reusable"); + Require(resources.NeedsCompletion() && !resources.UsesAddressTable(), "a fault-buffer build reaches guest memory through the BDA table"); const auto fault = access.bytes(access.descriptor(5).buffer); for (const auto byte : fault) Require(byte == std::byte{}, "rect-list fault buffer was not initialized"); const ShaderRecompiler::BdaAbi::Fault report{ShaderRecompiler::BdaAbi::FaultState::Ready, ShaderRecompiler::BdaAbi::FaultReason::InvalidRectangle, 0, 0, 0, 0, 0}; @@ -681,6 +682,7 @@ void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { { ShaderResources resources(context, compiled, snapshots); Require(ShaderResources::NeverReusable(std::span(&compiled, 1)) && !resources.Reusable(), "an address-based build is reusable"); + Require(resources.UsesAddressTable(), "an address-based build does not use the BDA table"); ShaderRecompiler::RecompileResult plain; plain.bindings = {binding(Role::GuestBuffers, 6)}; const CompiledShader plainStage{ShaderRecompiler::ShaderStage::Compute, &plain, 0}; From 859f2f3b92a2d761951864237b6b59bd27a9c349 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 04:03:40 +0200 Subject: [PATCH 315/460] feat(libc): implement malloc_usable_size Co-Authored-By: Claude --- .../libs/prx/libc/include/ApplicationHeap.hpp | 1 + core/libs/prx/libc/include/GuestHeap.hpp | 1 + core/libs/prx/libc/src/ApplicationHeap.cpp | 18 ++++++++++++++ core/libs/prx/libc/src/FileAndHeap.cpp | 4 ++++ core/libs/prx/libc/src/GuestHeap.cpp | 6 +++++ core/libs/tests/ApplicationHeapStats.cpp | 24 ++++++++++++++++++- docs/dev/TechnicalDebt.md | 1 + 7 files changed, 54 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libc/include/ApplicationHeap.hpp b/core/libs/prx/libc/include/ApplicationHeap.hpp index 22212a31d..ea9e7ad41 100644 --- a/core/libs/prx/libc/include/ApplicationHeap.hpp +++ b/core/libs/prx/libc/include/ApplicationHeap.hpp @@ -16,6 +16,7 @@ void* ApplicationHeapRealign_nid_no_patch(void* pointer, std::size_t bytes, std: void* ApplicationHeapCalloc_nid_no_patch(std::size_t count, std::size_t bytes); int ApplicationHeapPosixAlign_nid_no_patch(void** pointer, std::size_t alignment, std::size_t bytes); int ApplicationHeapStatsFast_nid_no_patch(void* stats); +std::size_t ApplicationHeapUsableSize_nid_no_patch(void* pointer); } diff --git a/core/libs/prx/libc/include/GuestHeap.hpp b/core/libs/prx/libc/include/GuestHeap.hpp index 65f9c0d42..0c7b87a87 100644 --- a/core/libs/prx/libc/include/GuestHeap.hpp +++ b/core/libs/prx/libc/include/GuestHeap.hpp @@ -7,6 +7,7 @@ namespace GuestHeap { extern "C" { +std::size_t GuestHeapUsableSize_nid_postfix(const void* pointer); void* GuestHeapAllocate_nid_postfix(std::size_t bytes); void GuestHeapFree_nid_postfix(void* pointer); void* GuestHeapReallocate_nid_postfix(void* pointer, std::size_t bytes); diff --git a/core/libs/prx/libc/src/ApplicationHeap.cpp b/core/libs/prx/libc/src/ApplicationHeap.cpp index 3affde65e..f05e94132 100644 --- a/core/libs/prx/libc/src/ApplicationHeap.cpp +++ b/core/libs/prx/libc/src/ApplicationHeap.cpp @@ -23,6 +23,7 @@ using Realign = void* (APS5_VABI *)(void*, std::size_t, std::size_t); using PosixAlign = int (APS5_VABI *)(void**, std::size_t, std::size_t); using Initialize = void (APS5_VABI *)(); using Stats = int (APS5_VABI *)(void*); +using UsableSize = std::size_t (APS5_VABI *)(void*); std::mutex heapMutex; std::array heapApi{}; @@ -249,3 +250,20 @@ int ApplicationHeapStatsFast_nid_no_patch(void* stats) { CallbackScope scope; return statsFast(stats); } + +std::size_t ApplicationHeapUsableSize_nid_no_patch(void* pointer) { + UsableSize usableSize = nullptr; + { + std::lock_guard lock(heapMutex); + if (heapFailure) std::rethrow_exception(heapFailure); + if (heapFinalized) throw std::runtime_error("application heap: allocator has been finalized"); + if (heapApi[0] == nullptr) throw std::runtime_error("application heap: allocator API is not registered"); + if (heapApi != defaultApi()) { + if (heapApi[9] == nullptr) throw std::runtime_error("malloc_usable_size: implemented only for an application allocator that provides it"); + std::memcpy(&usableSize, &heapApi[9], sizeof(usableSize)); + } + } + if (usableSize == nullptr) return GuestHeap::GuestHeapUsableSize_nid_postfix(pointer); + CallbackScope scope; + return usableSize(pointer); +} diff --git a/core/libs/prx/libc/src/FileAndHeap.cpp b/core/libs/prx/libc/src/FileAndHeap.cpp index 311c4973c..20527e876 100644 --- a/core/libs/prx/libc/src/FileAndHeap.cpp +++ b/core/libs/prx/libc/src/FileAndHeap.cpp @@ -268,6 +268,10 @@ int APS5_VABI malloc_stats_fast_nid_postfix(void* stats) { return ApplicationHeapStatsFast_nid_no_patch(stats); } +size_t APS5_VABI malloc_usable_size_nid_postfix(void* pointer) { + return ApplicationHeapUsableSize_nid_no_patch(pointer); +} + void* APS5_VABI malloc_nid_postfix(size_t size) { return ApplicationHeapAllocate_nid_no_patch(size); } diff --git a/core/libs/prx/libc/src/GuestHeap.cpp b/core/libs/prx/libc/src/GuestHeap.cpp index 24442855c..e651faa3a 100644 --- a/core/libs/prx/libc/src/GuestHeap.cpp +++ b/core/libs/prx/libc/src/GuestHeap.cpp @@ -263,6 +263,12 @@ void* GuestHeapRealign_nid_postfix(void* pointer, std::size_t bytes, std::size_t return result; } +std::size_t GuestHeapUsableSize_nid_postfix(const void* pointer) { + if (pointer == nullptr) return 0; + GuestAllocations::Mutation mutation; + return mutation.Find(pointer).bytes; +} + void* GuestHeapAlign_nid_postfix(std::size_t alignment, std::size_t bytes) { GuestAllocations::Mutation mutation; return allocate(mutation, alignment, bytes); diff --git a/core/libs/tests/ApplicationHeapStats.cpp b/core/libs/tests/ApplicationHeapStats.cpp index 42d97d01d..d2694001d 100644 --- a/core/libs/tests/ApplicationHeapStats.cpp +++ b/core/libs/tests/ApplicationHeapStats.cpp @@ -8,6 +8,7 @@ extern "C" { int APS5_VABI malloc_stats_fast_nid_postfix(void*); +std::size_t APS5_VABI malloc_usable_size_nid_postfix(void*); } namespace { @@ -46,6 +47,15 @@ int APS5_VABI statsFast(void* stats) { return 0x2a; } +unsigned usableCalls = 0; +void* lastUsable = nullptr; + +std::size_t APS5_VABI usableSize(void* pointer) { + ++usableCalls; + lastUsable = pointer; + return 0x55; +} + template void write(std::array& data, std::size_t offset, TValue value) { require(offset <= data.size() && sizeof(value) <= data.size() - offset); @@ -75,19 +85,29 @@ int main(int argc, char** argv) { write(replacement, 0x48, &realign); write(replacement, 0x50, &posixAlign); write(replacement, 0x60, &statsFast); + write(replacement, 0x68, &usableSize); if (argc > 1 && std::strcmp(argv[1], "default") == 0) { std::memset(replacement.data() + 0x20, 0, 10 * sizeof(void*)); ApplicationHeapInitialize_nid_no_patch(process.data()); require(ApplicationHeapAllocate_nid_no_patch(16) != nullptr); reject([&] { malloc_stats_fast_nid_postfix(stats.data()); }); + auto* small = ApplicationHeapAllocate_nid_no_patch(16); + auto* large = ApplicationHeapAllocate_nid_no_patch(100000); + require(malloc_usable_size_nid_postfix(small) == 16 && malloc_usable_size_nid_postfix(large) == 100000); + require(malloc_usable_size_nid_postfix(nullptr) == 0 && usableCalls == 0); + ApplicationHeapFree_nid_no_patch(small); + ApplicationHeapFree_nid_no_patch(large); return 0; } if (argc > 1 && std::strcmp(argv[1], "missing") == 0) { write(replacement, 0x60, static_cast(nullptr)); + write(replacement, 0x68, static_cast(nullptr)); ApplicationHeapInitialize_nid_no_patch(process.data()); require(ApplicationHeapAllocate_nid_no_patch(16) == storage.data()); reject([&] { malloc_stats_fast_nid_postfix(stats.data()); }); - require(statsCalls == 0); + reject([&] { malloc_usable_size_nid_postfix(storage.data()); }); + reject([&] { malloc_usable_size_nid_postfix(nullptr); }); + require(statsCalls == 0 && usableCalls == 0); return 0; } ApplicationHeapInitialize_nid_no_patch(process.data()); @@ -99,4 +119,6 @@ int main(int argc, char** argv) { require(statsCalls == 3); require(malloc_stats_fast_nid_postfix(stats.data()) == 0x2a && statsCalls == 4); require(ApplicationHeapAllocate_nid_no_patch(16) == storage.data()); + require(malloc_usable_size_nid_postfix(storage.data()) == 0x55 && usableCalls == 1 && lastUsable == storage.data()); + require(malloc_usable_size_nid_postfix(nullptr) == 0x55 && usableCalls == 2 && lastUsable == nullptr); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index ed3724fc9..3d6495951 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -162,6 +162,7 @@ Throughout the project, every function at every stage either **does exactly what - [Guest arena](../../core/libs/prx/libc/src/GuestArena.cpp) (libc) - the application map area is assumed to end at 0xFC_0000_0000, as `SCE_KERNEL_APP_MAP_AREA_END_ADDR` does on the PS4. On Windows, host allocations made in the arena before libc loads stay, and a fixed guest mapping over one throws - [sceLibcInternalBacktraceForGame](../../core/libs/prx/libc/src/HeapDiagnostics.cpp) (libSceLibcInternal, implemented in libc) - unknown signature - [sceLibcInternalHeapErrorReportForGame](../../core/libs/prx/libc/src/HeapDiagnostics.cpp) (libSceLibcInternal, implemented in libc) - unknown signature +- [malloc_usable_size](../../core/libs/prx/libc/src/ApplicationHeap.cpp) (libc) - with the default allocator it returns the size the block was requested with (the range registered for it), not the rounded-up chunk size Sony's allocator reports; both satisfy "at least the requested size", and bytes past the request would not be write-watched. With an application allocator it calls entry 9 of the replacement table (`malloc_usable_size` in KytyPS5's `MallocReplace` order) and throws when the table leaves it null, like `malloc_stats_fast` - [_sceLibcInternalThreadAtexit, _sceLibcInternalThreadDtors](../../core/libs/prx/libSceLibcInternal/Export.cpp) (libSceLibcInternal) - FreeBSD `__cxa_thread_atexit` semantics assumed; a null or non-image destructor and an allocation failure throw - [_sceKernelRtldThreadAtexitIncrement, _sceKernelRtldThreadAtexitDecrement](../../core/libs/prx/libkernel/System/src/Rtld.cpp) (libkernel) - the argument is the `dso` address a title's `libc.prx` passes to `__cxa_thread_atexit` (PPSA14632), which fails when Increment returns nonzero. As in the [fpPS4](https://github.com/red-prig/fpPS4) reimplementation, Increment takes a reference on the loaded image containing the address and Decrement releases one; the variants without the leading underscore are assumed to match - [sceImeGetPanelPositionAndForm](../../core/libs/prx/libSceIme/Export.cpp) (libSceIme) - the output is taken to be the `PositionAndForm` that `sceImeDialogGetPanelPositionAndForm` fills; it is never written, since no panel can be open. That a closed panel is reported before a null output is assumed by analogy with `sceImeSetCaret`, not confirmed on a PS5 title From 660abec2d3fb9bca7bb60b32feefea491ef452ac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 04:44:39 +0200 Subject: [PATCH 316/460] feat(libc): implement sqrt and sqrtf Co-Authored-By: Claude --- core/libs/prx/libc/src/Math.cpp | 2 ++ core/libs/tests/GuestMath.cpp | 14 ++++++++++++++ 2 files changed, 16 insertions(+) diff --git a/core/libs/prx/libc/src/Math.cpp b/core/libs/prx/libc/src/Math.cpp index 614e7e1a5..0f9236c92 100644 --- a/core/libs/prx/libc/src/Math.cpp +++ b/core/libs/prx/libc/src/Math.cpp @@ -46,6 +46,8 @@ int APS5_VABI __isinff_nid_postfix(float x) { return std::isinf(x) ? 1 : 0; } double APS5_VABI cbrt_nid_postfix(double x) { return std::cbrt(x); } double APS5_VABI asin_nid_postfix(double x) { return std::asin(x); } double APS5_VABI acos_nid_postfix(double x) { return std::acos(x); } +double APS5_VABI sqrt_nid_postfix(double x) { return std::sqrt(x); } +float APS5_VABI sqrtf_nid_postfix(float x) { return std::sqrt(x); } long double APS5_VABI acosl_nid_postfix(long double x) { return std::acos(x); } long double APS5_VABI frexpl_nid_postfix(long double x, int* exponent) { diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index 5bc32fad4..ec4d5e4a7 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -23,6 +23,8 @@ std::uint64_t APS5_VABI _Stoul_nid_postfix(const char*, char**, int); std::int64_t APS5_VABI atol_nid_postfix(const char*); long long APS5_VABI atoll_nid_postfix(const char*); int* APS5_VABI __error_nid_postfix(); +double APS5_VABI sqrt_nid_postfix(double); +float APS5_VABI sqrtf_nid_postfix(float); long double APS5_VABI acosl_nid_postfix(long double); long double APS5_VABI frexpl_nid_postfix(long double, int*); struct LibcFloatConstant { std::uint32_t bits[4]; }; @@ -61,6 +63,17 @@ std::lldiv_t APS5_VABI ldiv_nid_postfix(std::int64_t, std::int64_t); } static void Require(bool value) { if (!value) std::abort(); } +static void CheckSquareRoot() { + Require(sqrt_nid_postfix(4.) == 2. && sqrtf_nid_postfix(6.25f) == 2.5f); + Require(std::bit_cast(sqrt_nid_postfix(2.)) == 0x3ff6a09e667f3bcdull); + Require(std::bit_cast(sqrtf_nid_postfix(2.f)) == 0x3fb504f3u); + Require(std::bit_cast(sqrt_nid_postfix(0x1p-1074)) == 0x1e60000000000000ull); + Require(std::bit_cast(sqrt_nid_postfix(-0.)) == 0x8000000000000000ull); + Require(std::bit_cast(sqrtf_nid_postfix(-0.f)) == 0x80000000u); + Require(sqrt_nid_postfix(INFINITY) == INFINITY && sqrtf_nid_postfix(INFINITY) == INFINITY); + Require(std::isnan(sqrt_nid_postfix(-1.)) && std::isnan(sqrtf_nid_postfix(-1.f))); +} + static bool ExtendedBits(long double value, std::uint64_t mantissa, std::uint16_t signExponent) { unsigned char bytes[sizeof(long double)]{}; std::memcpy(bytes, &value, sizeof(value)); @@ -289,6 +302,7 @@ int main() { Require(std::isnan(logbf_nid_postfix(std::numeric_limits::quiet_NaN()))); Require(exp2_nid_postfix(-3.) == 0.125); Require(ldexp_nid_postfix(0.75, 4) == 12.); + CheckSquareRoot(); CheckLongDouble(); Require(scalbn_nid_postfix(0.75, -2) == 0.1875); Require(scalbnf_nid_postfix(0.75f, 4) == 12.f); From d024709cffdd70bf40be3bdf9ebf6476f295ebfb Mon Sep 17 00:00:00 2001 From: Joey Fisher Date: Wed, 7 Oct 2026 11:28:52 -0600 Subject: [PATCH 317/460] perf(agc): sample comparison-read R32F/R16 surfaces from a GPU depth copy of their storage image --- .../Graphics/include/Texture.hpp | 11 + .../Graphics/src/ShaderResources.cpp | 38 +++- .../libSceAgcDriver/Graphics/src/Texture.cpp | 147 ++++++++++++++ .../tests/execution/DepthCompareCopy.cpp | 188 ++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 5 files changed, 381 insertions(+), 4 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/DepthCompareCopy.cpp diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp index e357ff77c..8d739e591 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp @@ -61,6 +61,12 @@ public: // CanCopyFrom says whether the two descriptors address the same surface compatibly. Texture(const Context& context, const std::shared_ptr& source, const GuestTextureResource& descriptor, VkComponentMapping components); Texture(const Context& context, VkImage depthImage, VkFormat depthFormat, VkImageAspectFlags aspect, VkComponentMapping components); + struct DepthCopyTag {}; + Texture(const Context& context, const std::shared_ptr& source, const GuestTextureResource& descriptor, VkComponentMapping components, DepthCopyTag); + static VkFormat DepthCopyFormat(VkFormat colorFormat); + static bool CanDepthCopyFrom(const StorageTexture& source, const GuestTextureResource& descriptor); + bool IsDepthCopy() const { return depthCopy; } + bool RefreshDepthCopy(); static bool CanCopyFrom(const StorageTexture& source, const GuestTextureResource& descriptor); ~Texture(); Texture(const Texture&) = delete; @@ -108,6 +114,11 @@ private: std::shared_ptr source; std::shared_ptr storageSource; std::unique_ptr upload; + bool depthCopy = false; + bool copiedOnce = false; + std::uint64_t copiedVersion = 0; + VkFormat depthFormat = VK_FORMAT_UNDEFINED; + std::uint32_t copyWidth = 0, copyHeight = 0, copyLevels = 0, copyLayers = 0, copyElementBytes = 0; }; VkFormat AttachmentProxyFormat(const Context& context, VkFormat format); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index 9ac9ff9bd..18c5a6f11 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -410,6 +410,19 @@ bool ClearedViewEnabled() { return !disabled; } +bool DepthCopyEnabled() { + static const bool disabled = std::getenv("APS5_NO_DEPTH_COPY") != nullptr; + return !disabled; +} + +std::uint64_t DepthCopyMinBytes() { + static const std::uint64_t bytes = [] { + const char* text = std::getenv("APS5_DEPTH_COPY_MIN_KIB"); + return (text != nullptr ? std::strtoull(text, nullptr, 10) : 1024ull) << 10u; + }(); + return bytes; +} + // Surfaces whose storage image could not be made (no writable committed pages, say): remembered // so the sampled lookup does not throw and fall back on every use. A failure is keyed by the surface // (address and size: the heap reuses addresses) and holds only while the guest mappings are what @@ -537,8 +550,10 @@ std::shared_ptr cachedTexture(const Context& context, std::span= DepthCopyMinBytes() && Texture::DepthCopyFormat(ResolveTextureFormat(resource.format)) != VK_FORMAT_UNDEFINED && resource.dimension != TextureDimension::k3D; auto source = StorageTexture::FindPending(address, guestBytes); - if (source != nullptr && (depthCompare || !Texture::CanCopyFrom(*source, resource) || MetadataMoved(*source, resource))) source.reset(); + if (source != nullptr && (depthCompare ? !depthCopy || !Texture::CanDepthCopyFrom(*source, resource) : !Texture::CanCopyFrom(*source, resource))) source.reset(); + if (source != nullptr && MetadataMoved(*source, resource)) source.reset(); std::optional keys; bool clearThroughKeys = false; if (source != nullptr && resource.dccAddress != 0 && IsDccClear(source->FilledKeys())) { @@ -562,12 +577,13 @@ std::shared_ptr cachedTexture(const Context& context, std::span cachedTexture(const Context& context, std::span(context, source, resource, components, Texture::DepthCopyTag{}); + texture->RefreshDepthCopy(); + return texture; + } if (source != nullptr) return std::make_shared(context, source, resource, components); std::vector snapshot(bytes); ReadTextureSurface(resource, *keys, snapshot); @@ -618,6 +639,7 @@ std::shared_ptr cachedTexture(const Context& context, std::spansource) && (source != nullptr || *keys == DccKeys::Uncompressed) && StorageImageServesKeys(*it->source, resource.dccAddress)) { if (!GuestMemory::UnchangedSince(address, bytes, it->source->Generation())) it->source->Refresh(); + if (it->texture->IsDepthCopy()) it->texture->RefreshDepthCopy(); it->keys = *keys; touchTexture(cache, it); logLookup({it->texture.get(), resource, guestBytes, *keys, 0, it->source.get()}); @@ -650,7 +672,12 @@ std::shared_ptr cachedTexture(const Context& context, std::spanVersion(); - entry.texture = std::make_shared(context, source, resource, components); + if (depthCompare) { + entry.texture = std::make_shared(context, source, resource, components, Texture::DepthCopyTag{}); + entry.texture->RefreshDepthCopy(); + } else { + entry.texture = std::make_shared(context, source, resource, components); + } entry.accounted = source->Cached() ? 0u : heldBytes(*source); counters.fromStorage.fetch_add(1, std::memory_order_relaxed); } else { @@ -2313,6 +2340,9 @@ bool ShaderResources::Revalidate(std::span shaders, ProofR } } pendingSerialSeen = EpochRevalidate() && StorageTexture::PendingSerial() == serialBefore ? serialBefore : 0; + for (const auto& texture : textures) { + if (texture != nullptr && texture->IsDepthCopy()) texture->RefreshDepthCopy(); + } return finish(fast, true); } diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp index 2e4b692a9..14ea2ab6b 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp @@ -457,6 +457,153 @@ Texture::Texture(const Context& context, TextureDetiler& detiler, const GuestTex } } +VkFormat Texture::DepthCopyFormat(VkFormat colorFormat) { + switch (colorFormat) { + case VK_FORMAT_R32_SFLOAT: return VK_FORMAT_D32_SFLOAT; + case VK_FORMAT_R16_UNORM: return VK_FORMAT_D16_UNORM; + default: return VK_FORMAT_UNDEFINED; + } +} + +bool Texture::CanDepthCopyFrom(const StorageTexture& source, const GuestTextureResource& descriptor) { + if (!CanCopyFrom(source, descriptor) || descriptor.dimension == TextureDimension::k3D) return false; + if (DepthCopyFormat(ResolveTextureFormat(descriptor.format)) == VK_FORMAT_UNDEFINED) return false; + if (ResolveTextureFormat(source.Descriptor().format) != ResolveTextureFormat(descriptor.format)) return false; + const auto geometry = DescribeSurface(descriptor); + return geometry.imageDepth == 1 && source.ImageDepth() == 1 && source.ImageLayers() >= geometry.imageLayers && source.Descriptor().mipCount >= descriptor.mipCount; +} + +Texture::Texture(const Context& context, const std::shared_ptr& source, const GuestTextureResource& descriptor, VkComponentMapping components, DepthCopyTag) : context(context), storageSource(source), depthCopy(true) { + try { + Require(source != nullptr && CanDepthCopyFrom(*source, descriptor), "storage image cannot serve a depth copy of the sampled texture"); + const auto colorFormat = ResolveTextureFormat(descriptor.format); + depthFormat = DepthCopyFormat(colorFormat); + const auto geometry = DescribeSurface(descriptor); + copyWidth = descriptor.width; + copyHeight = descriptor.height; + copyLevels = descriptor.mipCount; + copyLayers = geometry.imageLayers; + copyElementBytes = BytesPerElement(descriptor.format); + APS5_LOG_OUT("Texture address=0x%llx %ux%u mips=%u layers=%u depth copy of storage image (vk=%d -> %d)", static_cast(descriptor.baseAddress), descriptor.width, descriptor.height, descriptor.mipCount, copyLayers, static_cast(colorFormat), static_cast(depthFormat)); + + VkImageCreateInfo imageInfo{VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO}; + imageInfo.flags = descriptor.dimension == TextureDimension::kCube ? VK_IMAGE_CREATE_CUBE_COMPATIBLE_BIT : 0u; + imageInfo.imageType = ImageTypeFor(descriptor.dimension); + imageInfo.format = depthFormat; + imageInfo.extent = {descriptor.width, descriptor.height, 1u}; + imageInfo.mipLevels = copyLevels; + imageInfo.arrayLayers = copyLayers; + imageInfo.samples = VK_SAMPLE_COUNT_1_BIT; + imageInfo.tiling = VK_IMAGE_TILING_OPTIMAL; + imageInfo.usage = VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_SAMPLED_BIT; + imageInfo.sharingMode = VK_SHARING_MODE_EXCLUSIVE; + imageInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED; + Check(context.Function("vkCreateImage")(context.device, &imageInfo, nullptr, &image), "vkCreateImage depth copy"); + owned = std::make_shared(context, image, VK_NULL_HANDLE); + VkMemoryRequirements requirements{}; + context.Function("vkGetImageMemoryRequirements")(context.device, image, &requirements); + VkMemoryAllocateInfo allocation{VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO}; + allocation.allocationSize = requirements.size; + allocationBytes = requirements.size; + allocation.memoryTypeIndex = context.MemoryType(requirements.memoryTypeBits, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT); + Check(context.Function("vkAllocateMemory")(context.device, &allocation, nullptr, &owned->memory), "vkAllocateMemory depth copy"); + Check(context.Function("vkBindImageMemory")(context.device, image, owned->memory, 0), "vkBindImageMemory depth copy"); + + const auto viewLevelCount = std::min(descriptor.lastLevel, descriptor.mipCount - 1u) - descriptor.baseLevel + 1u; + const auto viewLayerCount = geometry.imageLayers - descriptor.baseArray; + if (descriptor.dimension == TextureDimension::kCube) { + Require(viewLayerCount % 6u == 0, "guest cube texture view does not contain a multiple of 6 array slices"); + } + VkImageViewCreateInfo viewInfo{VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO}; + viewInfo.image = image; + viewInfo.viewType = ViewTypeFor(descriptor.dimension, viewLayerCount); + viewInfo.format = depthFormat; + viewInfo.components = VkComponentMapping{}; + viewInfo.subresourceRange = {VK_IMAGE_ASPECT_DEPTH_BIT, descriptor.baseLevel, viewLevelCount, descriptor.baseArray, viewLayerCount}; + VkImageViewMinLodCreateInfoEXT minLod{VK_STRUCTURE_TYPE_IMAGE_VIEW_MIN_LOD_CREATE_INFO_EXT}; + ChainMinLod(context, descriptor, viewInfo, minLod); + Check(context.Function("vkCreateImageView")(context.device, &viewInfo, nullptr, &view), "vkCreateImageView depth copy"); + createFirstLayerView(descriptor, viewInfo); + static_cast(components); + } catch (...) { + release(); + throw; + } +} + +bool Texture::RefreshDepthCopy() { + if (!depthCopy || storageSource == nullptr) return false; + if (copiedOnce && storageSource->Version() == copiedVersion) return false; + std::uint64_t linearBytes = 0; + for (std::uint32_t level = 0; level < copyLevels; ++level) linearBytes += static_cast(std::max(copyWidth >> level, 1u)) * std::max(copyHeight >> level, 1u) * copyElementBytes * copyLayers; + auto linear = std::make_shared(context, static_cast(linearBytes), VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT); + std::vector fromSource, toDepth; + std::uint64_t offset = 0; + for (std::uint32_t level = 0; level < copyLevels; ++level) { + const VkExtent3D extent{std::max(copyWidth >> level, 1u), std::max(copyHeight >> level, 1u), 1u}; + VkBufferImageCopy region{}; + region.bufferOffset = offset; + region.bufferRowLength = 0; + region.bufferImageHeight = 0; + region.imageSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, level, 0, copyLayers}; + region.imageOffset = {0, 0, 0}; + region.imageExtent = extent; + fromSource.push_back(region); + region.imageSubresource.aspectMask = VK_IMAGE_ASPECT_DEPTH_BIT; + toDepth.push_back(region); + offset += static_cast(extent.width) * extent.height * copyElementBytes * copyLayers; + } + auto* recorder = Recorder::Active(); + std::unique_ptr batch; + VkCommandBuffer commands = VK_NULL_HANDLE; + if (recorder != nullptr) { + commands = recorder->Commands(); + recorder->Keep(linear); + recorder->Keep(owned); + recorder->Keep(storageSource); + } else { + batch = std::make_unique(context); + commands = batch->Handle(); + } + const auto barrier = context.Function("vkCmdPipelineBarrier"); + const VkMemoryBarrier produced{VK_STRUCTURE_TYPE_MEMORY_BARRIER, nullptr, VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_TRANSFER_WRITE_BIT | VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT, VK_ACCESS_TRANSFER_READ_BIT | VK_ACCESS_TRANSFER_WRITE_BIT}; + VkImageMemoryBarrier toTransferDst{VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER}; + toTransferDst.srcAccessMask = VK_ACCESS_SHADER_READ_BIT; + toTransferDst.dstAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT; + toTransferDst.oldLayout = copiedOnce ? VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL : VK_IMAGE_LAYOUT_UNDEFINED; + toTransferDst.newLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL; + toTransferDst.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED; + toTransferDst.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED; + toTransferDst.image = image; + toTransferDst.subresourceRange = {VK_IMAGE_ASPECT_DEPTH_BIT, 0, copyLevels, 0, copyLayers}; + barrier(commands, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, 1, &produced, 0, nullptr, 1, &toTransferDst); + context.Function("vkCmdCopyImageToBuffer")(commands, storageSource->Image(), VK_IMAGE_LAYOUT_GENERAL, linear->Handle(), static_cast(fromSource.size()), fromSource.data()); + VkBufferMemoryBarrier linearReady{VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER}; + linearReady.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT; + linearReady.dstAccessMask = VK_ACCESS_TRANSFER_READ_BIT; + linearReady.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED; + linearReady.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED; + linearReady.buffer = linear->Handle(); + linearReady.offset = 0; + linearReady.size = VK_WHOLE_SIZE; + barrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, 0, nullptr, 1, &linearReady, 0, nullptr); + context.Function("vkCmdCopyBufferToImage")(commands, linear->Handle(), image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, static_cast(toDepth.size()), toDepth.data()); + VkImageMemoryBarrier toShaderRead{VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER}; + toShaderRead.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT; + toShaderRead.dstAccessMask = VK_ACCESS_SHADER_READ_BIT; + toShaderRead.oldLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL; + toShaderRead.newLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL; + toShaderRead.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED; + toShaderRead.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED; + toShaderRead.image = image; + toShaderRead.subresourceRange = toTransferDst.subresourceRange; + barrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, 0, 0, nullptr, 0, nullptr, 1, &toShaderRead); + if (batch) batch->SubmitAndWait(); + copiedOnce = true; + copiedVersion = storageSource->Version(); + return true; +} + bool Texture::CanCopyFrom(const StorageTexture& source, const GuestTextureResource& descriptor) { const auto& from = source.Descriptor(); if (IsBlockCompressed(descriptor.format) || IsBlockCompressed(from.format)) return false; diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/DepthCompareCopy.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/DepthCompareCopy.cpp new file mode 100644 index 000000000..f7bf5a3d9 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/DepthCompareCopy.cpp @@ -0,0 +1,188 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Texture.hpp" +#include "prx/libSceAgcDriver/Graphics/include/TextureTiling.hpp" +#include "prx/libc/include/GuestAllocations.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#ifdef _WIN32 +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include +#endif +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 32; +constexpr std::uint32_t Side = 512; +constexpr std::uint32_t Format32Float = 22; +constexpr std::uint32_t Type2D = 9; +constexpr std::uint32_t LessEqual = 3; +constexpr std::uint32_t ClampEdge = 2; +constexpr std::size_t BlockBytes = 2u << 20u; + +alignas(256) constexpr std::array StoreHalfCode{ + 0x7e280d00, 0x102828ff, 0x3d000000, 0x7e3c0300, 0x7e3e0280, 0xf0201108, 0x0001141e, 0xbf810000, 0xbf810000, +}; + +alignas(256) constexpr std::array StoreQuarterCode{ + 0x7e280d00, 0x102828ff, 0x3c800000, 0x7e3c0300, 0x7e3e0280, 0xf0201108, 0x0001141e, 0xbf810000, 0xbf810000, +}; + +alignas(256) constexpr std::array SampleCode{ + 0x1614008c, 0xe03c1000, 0x8000010a, 0xbf8c3f70, 0xf0bc0108, 0x00820401, 0xbf8c3f70, + 0x34160082, 0xe0701000, 0x8001040b, 0xbf810000, 0xbf810000, 0xbf810000, +}; + +alignas(256) std::array Input{}; +alignas(256) std::array Output{}; + +class GuestBlock { +public: + GuestBlock() { +#ifdef _WIN32 + block = static_cast(VirtualAlloc(nullptr, BlockBytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); +#else + block = static_cast(std::aligned_alloc(BlockBytes, BlockBytes)); +#endif + Require(block != nullptr, "depth compare copy: cannot allocate the guest block"); + std::memset(block, 0, BlockBytes); + GuestAllocations::Mutation().Add(block, BlockBytes, true, true, true); + } + + ~GuestBlock() { + GuestAllocations::Mutation().Remove(block); +#ifdef _WIN32 + VirtualFree(block, 0, MEM_RELEASE); +#else + std::free(block); +#endif + } + + GuestBlock(const GuestBlock&) = delete; + GuestBlock& operator=(const GuestBlock&) = delete; + + std::uint8_t* Data() { return block; } + +private: + std::uint8_t* block = nullptr; +}; + +std::array BufferDescriptor(const void* data, std::uint32_t bytes) { + const auto address = reinterpret_cast(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; +} + +std::array TextureDescriptor(const void* data) { + const auto address = static_cast(reinterpret_cast(data)); + return { + static_cast(address >> 8u), + static_cast((address >> 40u) & 0xffu) | (Format32Float << 20u) | (((Side - 1u) & 3u) << 30u), + ((Side - 1u) >> 2u) | ((Side - 1u) << 14u), + 0xfacu | (Type2D << 28u), + 0u, 0u, 0u, 0u, + }; +} + +std::array SamplerDescriptor() { + return {ClampEdge | (ClampEdge << 3u) | (ClampEdge << 6u) | (LessEqual << 12u), 0u, 0u, 0u}; +} + +ShaderRecompiler::RecompileResult Compile(AgcDriver::VulkanDevice& device, std::span code, std::span userData) { + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {false, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + return ShaderRecompiler::Recompile(request); +} + +void Store(AgcDriver::VulkanDevice& device, std::uint8_t* texels, std::span code) { + std::vector userData(16, 0u); + const auto texture = TextureDescriptor(texels); + std::copy(texture.begin(), texture.end(), userData.begin() + 4); + const auto result = Compile(device, code, userData); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); +} + +void Sample(AgcDriver::VulkanDevice& device, std::uint8_t* texels, float scale, const std::string& what) { + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + const float value = static_cast(tid) * scale; + Input[tid * 3u + 0u] = (tid & 1u) != 0u ? value - scale / 4.0f : value + scale / 4.0f; + Input[tid * 3u + 1u] = (static_cast(tid) + 0.5f) / static_cast(Side); + Input[tid * 3u + 2u] = 0.5f / static_cast(Side); + } + Output.fill(-1.0f); + std::vector userData(24, 0u); + const auto input = BufferDescriptor(Input.data(), static_cast(sizeof(Input))); + const auto output = BufferDescriptor(Output.data(), static_cast(sizeof(Output))); + const auto texture = TextureDescriptor(texels); + const auto sampler = SamplerDescriptor(); + std::copy(input.begin(), input.end(), userData.begin()); + std::copy(output.begin(), output.end(), userData.begin() + 4); + std::copy(texture.begin(), texture.end(), userData.begin() + 8); + std::copy(sampler.begin(), sampler.end(), userData.begin() + 16); + const auto result = Compile(device, SampleCode, userData); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(SampleCode.data())); + device.WaitIdle(); + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + const float expected = (tid & 1u) != 0u ? 1.0f : 0.0f; + Require(Output[tid] == expected, what + ": thread " + std::to_string(tid) + " compared to " + std::to_string(Output[tid]) + ", expected " + std::to_string(expected)); + } +} + +float Texel(const std::uint8_t* texels, std::uint32_t x) { + float value = 0.0f; + std::memcpy(&value, texels + static_cast(x) * sizeof(float), sizeof(float)); + return value; +} + +void RequireGuestTexels(const std::uint8_t* texels, float scale, const std::string& what) { + for (std::uint32_t x = 0; x < Threads; ++x) { + const float expected = static_cast(x) * scale; + Require(Texel(texels, x) == expected, what + ": texel " + std::to_string(x) + " is " + std::to_string(Texel(texels, x)) + ", expected " + std::to_string(expected)); + } +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + GuestBlock block; + auto* texels = block.Data(); + const auto surface = AgcDriver::Graphics::DescribeSurface(AgcDriver::Graphics::DecodeTextureResource(TextureDescriptor(texels))); + Require(surface.guestBytes >= (1u << 20u) && surface.guestBytes <= BlockBytes, "depth compare copy: the surface does not fit the guest block"); + Store(*device, texels, StoreHalfCode); + Sample(*device, texels, 1.0f / 32.0f, "comparison sample of pending results"); + Store(*device, texels, StoreQuarterCode); + Sample(*device, texels, 1.0f / 64.0f, "comparison sample after a second store"); + AgcDriver::Graphics::StorageTexture::FlushPending(reinterpret_cast(texels), surface.guestBytes, nullptr, "test"); + device->WaitIdle(); + RequireGuestTexels(texels, 1.0f / 64.0f, "flushed results"); + std::puts("depth compare copy tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 3d6495951..8c628a6dd 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -519,3 +519,4 @@ Throughout the project, every function at every stage either **does exactly what - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert +- [Comparison-sampled R32 float and R16 unorm textures](../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp) (libSceAgcDriver) whose surface a storage image holds are sampled from a D32_SFLOAT or D16_UNORM image that the GPU refills from the storage image whenever its version moves, because Vulkan compares only depth formats. Only surfaces of at least 1 MiB (`APS5_DEPTH_COPY_MIN_KIB`) that are not 3D take the copy; smaller ones stay CPU snapshots compared against guest memory. With copies of small surfaces, one title hit an intermittent boot-time crash that was not explained. `APS5_NO_DEPTH_COPY=1` turns the copies off. From b38418350502eb3c3b16e86012ed939c0ab0c654 Mon Sep 17 00:00:00 2001 From: Joey Fisher Date: Sat, 10 Oct 2026 20:15:34 -0600 Subject: [PATCH 318/460] test(agc): pin the depth copy path, its refresh on reuse, R16 and several mips and layers; drop APS5_NO_DEPTH_COPY --- .../Graphics/include/Texture.hpp | 5 + .../Graphics/src/ShaderResources.cpp | 7 +- .../libSceAgcDriver/Graphics/src/Texture.cpp | 7 + .../tests/execution/DepthCompareCopy.cpp | 209 ++++++++++++++---- docs/dev/TechnicalDebt.md | 2 +- 5 files changed, 182 insertions(+), 48 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp index 8d739e591..6bc130edb 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/Texture.hpp @@ -67,6 +67,11 @@ public: static bool CanDepthCopyFrom(const StorageTexture& source, const GuestTextureResource& descriptor); bool IsDepthCopy() const { return depthCopy; } bool RefreshDepthCopy(); + struct DepthCopyCounts { + std::uint64_t textures; + std::uint64_t refreshes; + }; + static DepthCopyCounts TakeDepthCopyCounts(); static bool CanCopyFrom(const StorageTexture& source, const GuestTextureResource& descriptor); ~Texture(); Texture(const Texture&) = delete; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index 18c5a6f11..ac89460b7 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -410,11 +410,6 @@ bool ClearedViewEnabled() { return !disabled; } -bool DepthCopyEnabled() { - static const bool disabled = std::getenv("APS5_NO_DEPTH_COPY") != nullptr; - return !disabled; -} - std::uint64_t DepthCopyMinBytes() { static const std::uint64_t bytes = [] { const char* text = std::getenv("APS5_DEPTH_COPY_MIN_KIB"); @@ -550,7 +545,7 @@ std::shared_ptr cachedTexture(const Context& context, std::span= DepthCopyMinBytes() && Texture::DepthCopyFormat(ResolveTextureFormat(resource.format)) != VK_FORMAT_UNDEFINED && resource.dimension != TextureDimension::k3D; + const bool depthCopy = depthCompare && guestBytes >= DepthCopyMinBytes() && Texture::DepthCopyFormat(ResolveTextureFormat(resource.format)) != VK_FORMAT_UNDEFINED && resource.dimension != TextureDimension::k3D; auto source = StorageTexture::FindPending(address, guestBytes); if (source != nullptr && (depthCompare ? !depthCopy || !Texture::CanDepthCopyFrom(*source, resource) : !Texture::CanCopyFrom(*source, resource))) source.reset(); if (source != nullptr && MetadataMoved(*source, resource)) source.reset(); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp index 14ea2ab6b..bd296af03 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp @@ -87,6 +87,7 @@ std::atomic pretestSkipped{0}, unregisteredDropped{0}, keyFlipKep // Images stored by a FlushPending after a hook skip of theirs (AccessKeptByCpu), of which by the // hook for the read site of the last skip; images evictStale dropped (the [hooksync] and [storage] lines). std::atomic flushedAfterSkip{0}, flushedAfterSkipSameSite{0}, staleEvicted{0}; +std::atomic depthCopyTextures{0}, depthCopyRefreshes{0}; std::atomic refreshesProved{0}; struct StorageTraffic { @@ -525,6 +526,7 @@ Texture::Texture(const Context& context, const std::shared_ptr& Check(context.Function("vkCreateImageView")(context.device, &viewInfo, nullptr, &view), "vkCreateImageView depth copy"); createFirstLayerView(descriptor, viewInfo); static_cast(components); + depthCopyTextures.fetch_add(1, std::memory_order_relaxed); } catch (...) { release(); throw; @@ -601,9 +603,14 @@ bool Texture::RefreshDepthCopy() { if (batch) batch->SubmitAndWait(); copiedOnce = true; copiedVersion = storageSource->Version(); + depthCopyRefreshes.fetch_add(1, std::memory_order_relaxed); return true; } +Texture::DepthCopyCounts Texture::TakeDepthCopyCounts() { + return {depthCopyTextures.exchange(0, std::memory_order_relaxed), depthCopyRefreshes.exchange(0, std::memory_order_relaxed)}; +} + bool Texture::CanCopyFrom(const StorageTexture& source, const GuestTextureResource& descriptor) { const auto& from = source.Descriptor(); if (IsBlockCompressed(descriptor.format) || IsBlockCompressed(from.format)) return false; diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/DepthCompareCopy.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/DepthCompareCopy.cpp index f7bf5a3d9..1db56eaa8 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/DepthCompareCopy.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/DepthCompareCopy.cpp @@ -1,7 +1,10 @@ +#include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" #include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" #include "prx/libSceAgcDriver/Graphics/include/Texture.hpp" #include "prx/libSceAgcDriver/Graphics/include/TextureTiling.hpp" #include "prx/libc/include/GuestAllocations.hpp" +#include "prx/libc/include/GuestArena.hpp" +#include "prx/libc/include/GuestWriteWatch.hpp" #include "Recompiler.hpp" #include "VulkanTestDevice.hpp" #ifdef _WIN32 @@ -9,6 +12,8 @@ #define NOMINMAX #endif #include +#else +#include #endif #include #include @@ -27,13 +32,23 @@ using AgcDriver::Graphics::Require; using ShaderRecompiler::ShaderStage; constexpr std::uint32_t Threads = 32; -constexpr std::uint32_t Side = 512; +constexpr std::uint32_t Format16Unorm = 7; constexpr std::uint32_t Format32Float = 22; constexpr std::uint32_t Type2D = 9; +constexpr std::uint32_t Type2DArray = 13; constexpr std::uint32_t LessEqual = 3; constexpr std::uint32_t ClampEdge = 2; constexpr std::size_t BlockBytes = 2u << 20u; +struct Shape { + std::uint32_t format; + std::uint32_t width; + std::uint32_t height; + std::uint32_t layers; + std::uint32_t mips; + bool array; +}; + alignas(256) constexpr std::array StoreHalfCode{ 0x7e280d00, 0x102828ff, 0x3d000000, 0x7e3c0300, 0x7e3e0280, 0xf0201108, 0x0001141e, 0xbf810000, 0xbf810000, }; @@ -42,33 +57,56 @@ alignas(256) constexpr std::array StoreQuarterCode{ 0x7e280d00, 0x102828ff, 0x3c800000, 0x7e3c0300, 0x7e3e0280, 0xf0201108, 0x0001141e, 0xbf810000, 0xbf810000, }; +alignas(256) constexpr std::array StoreLayerMipCode{ + 0x7e280d00, 0x1028280e, 0x7e3c0300, 0x7e3e0280, 0x7e40020c, 0x7e42020d, 0xf0241128, 0x0001141e, 0xbf810000, 0xbf810000, +}; + alignas(256) constexpr std::array SampleCode{ 0x1614008c, 0xe03c1000, 0x8000010a, 0xbf8c3f70, 0xf0bc0108, 0x00820401, 0xbf8c3f70, 0x34160082, 0xe0701000, 0x8001040b, 0xbf810000, 0xbf810000, 0xbf810000, }; -alignas(256) std::array Input{}; -alignas(256) std::array Output{}; +alignas(256) constexpr std::array SampleLayerMipCode{ + 0x1614008c, 0xe03c1000, 0x8000010a, 0x7e080214, 0x7e0a0215, 0xbf8c3f70, 0xf0b00128, 0x00820601, 0xbf8c3f70, + 0x34160082, 0xe0701000, 0x8001060b, 0xbf810000, 0xbf810000, 0xbf810000, +}; + +std::span Input; +std::span Output; class GuestBlock { public: GuestBlock() { #ifdef _WIN32 - block = static_cast(VirtualAlloc(nullptr, BlockBytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); + block = static_cast(GuestArena::GuestArenaAllocate_nid_postfix(BlockBytes, 65536)); + if (block != nullptr) GuestArena::GuestArenaCommit_nid_postfix(block, BlockBytes, PAGE_READWRITE, BlockBytes); #else - block = static_cast(std::aligned_alloc(BlockBytes, BlockBytes)); + constexpr std::uintptr_t alignment = 65536; + void* mapped = mmap(nullptr, BlockBytes + alignment, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (mapped != MAP_FAILED) { + const auto begin = reinterpret_cast(mapped); + const auto aligned = (begin + alignment - 1) & ~(alignment - 1); + if (aligned != begin) munmap(mapped, aligned - begin); + if (aligned + BlockBytes != begin + BlockBytes + alignment) munmap(reinterpret_cast(aligned + BlockBytes), begin + alignment - aligned); + block = reinterpret_cast(aligned); + GuestWriteWatch::GuestWriteWatchRegister_nid_postfix(block, BlockBytes); + } #endif Require(block != nullptr, "depth compare copy: cannot allocate the guest block"); + Require(AgcDriver::GuestMemory::Watched(reinterpret_cast(block), BlockBytes), "depth compare copy: the guest block is not write-watched"); std::memset(block, 0, BlockBytes); GuestAllocations::Mutation().Add(block, BlockBytes, true, true, true); + AgcDriver::GuestMemory::CollectWritesUncached(reinterpret_cast(block), BlockBytes); } ~GuestBlock() { GuestAllocations::Mutation().Remove(block); #ifdef _WIN32 - VirtualFree(block, 0, MEM_RELEASE); + GuestArena::GuestArenaReset_nid_postfix(block, BlockBytes); + GuestArena::GuestArenaRelease_nid_postfix(block, BlockBytes); #else - std::free(block); + munmap(block, BlockBytes); + GuestWriteWatch::GuestWriteWatchUnregister_nid_postfix(block, BlockBytes); #endif } @@ -86,19 +124,27 @@ std::array BufferDescriptor(const void* data, std::uint32_t by return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; } -std::array TextureDescriptor(const void* data) { +std::array TextureDescriptor(const void* data, const Shape& shape) { const auto address = static_cast(reinterpret_cast(data)); return { static_cast(address >> 8u), - static_cast((address >> 40u) & 0xffu) | (Format32Float << 20u) | (((Side - 1u) & 3u) << 30u), - ((Side - 1u) >> 2u) | ((Side - 1u) << 14u), - 0xfacu | (Type2D << 28u), - 0u, 0u, 0u, 0u, + static_cast((address >> 40u) & 0xffu) | (shape.format << 20u) | (((shape.width - 1u) & 3u) << 30u), + ((shape.width - 1u) >> 2u) | ((shape.height - 1u) << 14u), + 0xfacu | ((shape.mips - 1u) << 16u) | ((shape.array ? Type2DArray : Type2D) << 28u), + shape.array ? shape.layers - 1u : 0u, + (shape.mips - 1u) << 4u, + 0u, 0u, }; } -std::array SamplerDescriptor() { - return {ClampEdge | (ClampEdge << 3u) | (ClampEdge << 6u) | (LessEqual << 12u), 0u, 0u, 0u}; +std::array SamplerDescriptor(bool mips) { + return {ClampEdge | (ClampEdge << 3u) | (ClampEdge << 6u) | (LessEqual << 12u), mips ? 0xf00u << 12u : 0u, mips ? 1u << 26u : 0u, 0u}; +} + +std::uint32_t Bits(float value) { + std::uint32_t bits = 0; + std::memcpy(&bits, &value, sizeof(bits)); + return bits; } ShaderRecompiler::RecompileResult Compile(AgcDriver::VulkanDevice& device, std::span code, std::span userData) { @@ -114,34 +160,45 @@ ShaderRecompiler::RecompileResult Compile(AgcDriver::VulkanDevice& device, std:: return ShaderRecompiler::Recompile(request); } -void Store(AgcDriver::VulkanDevice& device, std::uint8_t* texels, std::span code) { +void Store(AgcDriver::VulkanDevice& device, std::uint8_t* texels, const Shape& shape, std::span code, std::uint32_t layer = 0, std::uint32_t mip = 0, float scale = 0.0f) { std::vector userData(16, 0u); - const auto texture = TextureDescriptor(texels); + const auto texture = TextureDescriptor(texels, shape); std::copy(texture.begin(), texture.end(), userData.begin() + 4); + userData[12] = layer; + userData[13] = mip; + userData[14] = Bits(scale); const auto result = Compile(device, code, userData); device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); device.WaitIdle(); } -void Sample(AgcDriver::VulkanDevice& device, std::uint8_t* texels, float scale, const std::string& what) { - for (std::uint32_t tid = 0; tid < Threads; ++tid) { - const float value = static_cast(tid) * scale; - Input[tid * 3u + 0u] = (tid & 1u) != 0u ? value - scale / 4.0f : value + scale / 4.0f; - Input[tid * 3u + 1u] = (static_cast(tid) + 0.5f) / static_cast(Side); - Input[tid * 3u + 2u] = 0.5f / static_cast(Side); - } - Output.fill(-1.0f); +ShaderRecompiler::RecompileResult SampleProgram(AgcDriver::VulkanDevice& device, std::uint8_t* texels, const Shape& shape, std::uint32_t layer = 0, std::uint32_t mip = 0) { std::vector userData(24, 0u); - const auto input = BufferDescriptor(Input.data(), static_cast(sizeof(Input))); - const auto output = BufferDescriptor(Output.data(), static_cast(sizeof(Output))); - const auto texture = TextureDescriptor(texels); - const auto sampler = SamplerDescriptor(); + const auto input = BufferDescriptor(Input.data(), static_cast(Input.size_bytes())); + const auto output = BufferDescriptor(Output.data(), static_cast(Output.size_bytes())); + const auto texture = TextureDescriptor(texels, shape); + const auto sampler = SamplerDescriptor(shape.mips > 1u); std::copy(input.begin(), input.end(), userData.begin()); std::copy(output.begin(), output.end(), userData.begin() + 4); std::copy(texture.begin(), texture.end(), userData.begin() + 8); std::copy(sampler.begin(), sampler.end(), userData.begin() + 16); - const auto result = Compile(device, SampleCode, userData); - device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(SampleCode.data())); + userData[20] = Bits(static_cast(layer)); + userData[21] = Bits(static_cast(mip)); + return Compile(device, shape.array ? std::span(SampleLayerMipCode) : std::span(SampleCode), userData); +} + +void Sample(AgcDriver::VulkanDevice& device, const ShaderRecompiler::RecompileResult& program, const Shape& shape, float scale, const std::string& what, std::uint32_t layer = 0, std::uint32_t mip = 0) { + const auto width = std::max(shape.width >> mip, 1u); + const auto height = std::max(shape.height >> mip, 1u); + for (std::uint32_t tid = 0; tid < Threads; ++tid) { + const float value = static_cast(tid) * scale; + Input[tid * 3u + 0u] = (tid & 1u) != 0u ? value - scale / 4.0f : value + scale / 4.0f; + Input[tid * 3u + 1u] = (static_cast(tid) + 0.5f) / static_cast(width); + Input[tid * 3u + 2u] = 0.5f / static_cast(height); + } + std::fill(Output.begin(), Output.end(), -1.0f); + const auto* code = shape.array ? SampleLayerMipCode.data() : SampleCode.data(); + device.Dispatch(program, 1, 1, 1, {}, reinterpret_cast(code)); device.WaitIdle(); for (std::uint32_t tid = 0; tid < Threads; ++tid) { const float expected = (tid & 1u) != 0u ? 1.0f : 0.0f; @@ -149,6 +206,33 @@ void Sample(AgcDriver::VulkanDevice& device, std::uint8_t* texels, float scale, } } +std::uint64_t SurfaceBytes(std::uint8_t* texels, const Shape& shape) { + const auto surface = AgcDriver::Graphics::DescribeSurface(AgcDriver::Graphics::DecodeTextureResource(TextureDescriptor(texels, shape))); + Require(surface.guestBytes >= (1u << 20u) && surface.guestBytes <= BlockBytes, "depth compare copy: the surface does not fit the guest block or is below the depth copy floor"); + return surface.guestBytes; +} + +std::uint64_t PendingVersion(std::uint8_t* texels, std::uint64_t bytes, const std::string& what) { + const auto source = AgcDriver::Graphics::StorageTexture::FindPending(reinterpret_cast(texels), bytes); + Require(source != nullptr, what + ": no storage image holds the surface's results"); + return source->Version(); +} + +void Settle(AgcDriver::VulkanDevice& device, std::uint8_t* texels, std::uint64_t bytes) { + AgcDriver::Graphics::StorageTexture::FlushPending(reinterpret_cast(texels), bytes, nullptr, "test"); + device.WaitIdle(); + AgcDriver::GuestMemory::CollectWritesUncached(reinterpret_cast(texels), bytes); +} + +void RequireCounts(std::uint64_t textures, std::uint64_t refreshes, const std::string& what) { + const auto counts = AgcDriver::Graphics::Texture::TakeDepthCopyCounts(); + Require(counts.textures == textures && counts.refreshes == refreshes, what + ": " + std::to_string(counts.textures) + " depth copy textures and " + std::to_string(counts.refreshes) + " refreshes, expected " + std::to_string(textures) + " and " + std::to_string(refreshes)); +} + +void RequireNewerVersion(std::uint64_t before, std::uint64_t after, const std::string& what) { + Require(after > before, what + ": the storage image version stayed at " + std::to_string(before)); +} + float Texel(const std::uint8_t* texels, std::uint32_t x) { float value = 0.0f; std::memcpy(&value, texels + static_cast(x) * sizeof(float), sizeof(float)); @@ -162,23 +246,66 @@ void RequireGuestTexels(const std::uint8_t* texels, float scale, const std::stri } } +void SingleLevelCase(AgcDriver::VulkanDevice& device, const Shape& shape, const std::string& name) { + GuestBlock block; + auto* texels = block.Data(); + const auto bytes = SurfaceBytes(texels, shape); + Store(device, texels, shape, StoreQuarterCode); + Settle(device, texels, bytes); + AgcDriver::Graphics::Texture::TakeDepthCopyCounts(); + Store(device, texels, shape, StoreHalfCode); + const auto program = SampleProgram(device, texels, shape); + Sample(device, program, shape, 1.0f / 32.0f, name + ": comparison sample of pending results"); + RequireCounts(1, 1, name + ": first comparison sample"); + const auto before = PendingVersion(texels, bytes, name); + Store(device, texels, shape, StoreQuarterCode); + RequireNewerVersion(before, PendingVersion(texels, bytes, name), name + ": second store"); + Sample(device, program, shape, 1.0f / 64.0f, name + ": comparison sample of the same descriptor set after a second store"); + RequireCounts(0, 1, name + ": comparison sample after a second store"); + AgcDriver::Graphics::StorageTexture::FlushPending(reinterpret_cast(texels), bytes, nullptr, "test"); + device.WaitIdle(); + if (shape.format == Format32Float) RequireGuestTexels(texels, 1.0f / 64.0f, name + ": flushed results"); +} + +void LayersAndMipsCase(AgcDriver::VulkanDevice& device) { + const Shape shape{Format32Float, 256, 256, 4, 3, true}; + const std::string name = "R32F 256x256, 4 layers, 3 mips"; + GuestBlock block; + auto* texels = block.Data(); + const auto bytes = SurfaceBytes(texels, shape); + Store(device, texels, shape, StoreLayerMipCode, 0, 0, 1.0f / 512.0f); + Settle(device, texels, bytes); + AgcDriver::Graphics::Texture::TakeDepthCopyCounts(); + Store(device, texels, shape, StoreLayerMipCode, 0, 0, 1.0f / 32.0f); + Store(device, texels, shape, StoreLayerMipCode, 2, 1, 1.0f / 64.0f); + Store(device, texels, shape, StoreLayerMipCode, 3, 2, 1.0f / 128.0f); + const auto first = SampleProgram(device, texels, shape, 0, 0); + const auto second = SampleProgram(device, texels, shape, 2, 1); + const auto third = SampleProgram(device, texels, shape, 3, 2); + Sample(device, first, shape, 1.0f / 32.0f, name + ": layer 0 mip 0", 0, 0); + Sample(device, second, shape, 1.0f / 64.0f, name + ": layer 2 mip 1", 2, 1); + Sample(device, third, shape, 1.0f / 128.0f, name + ": layer 3 mip 2", 3, 2); + RequireCounts(1, 1, name + ": first comparison samples"); + const auto before = PendingVersion(texels, bytes, name); + Store(device, texels, shape, StoreLayerMipCode, 2, 1, 1.0f / 256.0f); + RequireNewerVersion(before, PendingVersion(texels, bytes, name), name + ": second store"); + Sample(device, second, shape, 1.0f / 256.0f, name + ": layer 2 mip 1 after a second store", 2, 1); + Sample(device, first, shape, 1.0f / 32.0f, name + ": layer 0 mip 0 after a second store", 0, 0); + RequireCounts(0, 1, name + ": comparison samples after a second store"); +} + } int main() { try { const auto device = OpenVulkanTestDevice(); if (!device) return VulkanTestSkipped; - GuestBlock block; - auto* texels = block.Data(); - const auto surface = AgcDriver::Graphics::DescribeSurface(AgcDriver::Graphics::DecodeTextureResource(TextureDescriptor(texels))); - Require(surface.guestBytes >= (1u << 20u) && surface.guestBytes <= BlockBytes, "depth compare copy: the surface does not fit the guest block"); - Store(*device, texels, StoreHalfCode); - Sample(*device, texels, 1.0f / 32.0f, "comparison sample of pending results"); - Store(*device, texels, StoreQuarterCode); - Sample(*device, texels, 1.0f / 64.0f, "comparison sample after a second store"); - AgcDriver::Graphics::StorageTexture::FlushPending(reinterpret_cast(texels), surface.guestBytes, nullptr, "test"); - device->WaitIdle(); - RequireGuestTexels(texels, 1.0f / 64.0f, "flushed results"); + GuestBlock buffers; + Input = std::span(reinterpret_cast(buffers.Data()), Threads * 3u); + Output = std::span(reinterpret_cast(buffers.Data() + 4096u), Threads); + SingleLevelCase(*device, {Format32Float, 512, 512, 1, 1, false}, "R32F 512x512"); + SingleLevelCase(*device, {Format16Unorm, 1024, 512, 1, 1, false}, "R16 unorm 1024x512"); + LayersAndMipsCase(*device); std::puts("depth compare copy tests passed"); return 0; } catch (const std::exception& error) { diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 8c628a6dd..1f9a7a38e 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -519,4 +519,4 @@ Throughout the project, every function at every stage either **does exactly what - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert -- [Comparison-sampled R32 float and R16 unorm textures](../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp) (libSceAgcDriver) whose surface a storage image holds are sampled from a D32_SFLOAT or D16_UNORM image that the GPU refills from the storage image whenever its version moves, because Vulkan compares only depth formats. Only surfaces of at least 1 MiB (`APS5_DEPTH_COPY_MIN_KIB`) that are not 3D take the copy; smaller ones stay CPU snapshots compared against guest memory. With copies of small surfaces, one title hit an intermittent boot-time crash that was not explained. `APS5_NO_DEPTH_COPY=1` turns the copies off. +- [Comparison-sampled R32 float and R16 unorm textures](../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp) (libSceAgcDriver) whose surface a storage image holds are sampled from a D32_SFLOAT or D16_UNORM image that the GPU refills from the storage image whenever its version moves, because Vulkan compares only depth formats. Only surfaces of at least 1 MiB (`APS5_DEPTH_COPY_MIN_KIB`) that are not 3D take the copy; smaller ones stay CPU snapshots compared against guest memory. With copies of small surfaces, one title hit an intermittent boot-time crash that was not explained. From bf779b2d717bdb6b16916b302843471b73f2529e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 04:52:43 +0200 Subject: [PATCH 319/460] docs(technical-debt): move the depth copy line off the end of the file --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 1f9a7a38e..8c55773ae 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -516,7 +516,7 @@ Throughout the project, every function at every stage either **does exactly what - [kill](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) follows FreeBSD's `kern_kill` and `killpg1` for the only guest process: `EINVAL` for signals outside 0 to 128, signal 0 only checks that the target exists, and its own pid, 0 and minus its own pid (the guest is taken to lead its own process group) signal it through `raise`, so they share `raise`'s limits above. Any other pid, and -1, which skips the caller, report `ESRCH`, so `EPERM` is never reported - [sceKernelFcntl](../../core/libs/prx/libkernel/File/src/Open.cpp) (libkernel) handles socket descriptors through the POSIX `fcntl` export (`F_GETFL`, `F_SETFL` with `O_NONBLOCK`) and returns its guest errno as `SCE_KERNEL_ERROR_*`, the form the other `sceKernel` file exports use; the command semantics follow FreeBSD's [fcntl](https://github.com/freebsd/freebsd-src/blob/stable/11/sys/kern/kern_descrip.c), and a closed descriptor returns `SCE_KERNEL_ERROR_EBADF`. File descriptors throw: their open flags are not tracked, and `F_DUPFD`, `F_GETFD` and `F_SETFD` are unsupported on all descriptors. The return values and error codes are unverified on a console - [AJM](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp), [APR](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) and [NGS2](../../core/libs/prx/libSceNgs2.native/src/System.cpp) state is allocated with `new` and never freed on purpose: title threads still call into these libraries while `std::exit` runs static destructors, so ordinary statics would be destroyed under them. Keep them leaked; `guest_ajm_teardown` covers the AJM case on Unix. +- [Comparison-sampled R32 float and R16 unorm textures](../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp) (libSceAgcDriver) whose surface a storage image holds are sampled from a D32_SFLOAT or D16_UNORM image that the GPU refills from the storage image whenever its version moves, because Vulkan compares only depth formats. Only surfaces of at least 1 MiB (`APS5_DEPTH_COPY_MIN_KIB`) that are not 3D take the copy; smaller ones stay CPU snapshots compared against guest memory. With copies of small surfaces, one title hit an intermittent boot-time crash that was not explained. - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert -- [Comparison-sampled R32 float and R16 unorm textures](../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp) (libSceAgcDriver) whose surface a storage image holds are sampled from a D32_SFLOAT or D16_UNORM image that the GPU refills from the storage image whenever its version moves, because Vulkan compares only depth formats. Only surfaces of at least 1 MiB (`APS5_DEPTH_COPY_MIN_KIB`) that are not 3D take the copy; smaller ones stay CPU snapshots compared against guest memory. With copies of small surfaces, one title hit an intermittent boot-time crash that was not explained. From 86d66243233ae72afaa73d2658cd01cbdd3bff56 Mon Sep 17 00:00:00 2001 From: Chad Date: Sat, 10 Oct 2026 14:56:19 -0500 Subject: [PATCH 320/460] fix(agc): copy a sparse descriptor heap only up to its last backed byte Co-authored-by: Scramblrrrr --- .../Graphics/include/GuestBufferMemory.hpp | 4 +- .../Graphics/src/GuestBufferMemory.cpp | 34 ++++++++-- .../libSceAgcDriver/tests/BdaResources.cpp | 66 +++++++++++++++++++ 3 files changed, 98 insertions(+), 6 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp index f180e1e9b..f6e5f3e5c 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp @@ -247,7 +247,7 @@ public: bool HoldsLease() const { return !lease.empty() || space != nullptr; } std::size_t CopiedBytes() const { std::size_t bytes = 0; - for (const auto& region : regions) if (region.buffer != nullptr) bytes += static_cast(region.end - region.begin); + for (const auto& region : regions) if (region.buffer != nullptr) bytes += static_cast((region.bufferEnd != 0 ? region.bufferEnd : region.end) - region.begin); return bytes; } // Every uploaded region as [begin, end) when all of them are served by host imports, in place or @@ -281,6 +281,8 @@ private: // memory; the rest reads as zeros and is never stored. bool sparse = false; std::vector> backed {}; + std::uint64_t lastView = 0; + std::uint64_t bufferEnd = 0; // Set when the region is served by an image mirror: nothing is copied; writable mirrors are // written back by comparing with the mirror's shadow. Kept alive here for recorded work. std::shared_ptr mirror {}; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp index 1f6091d71..818b39d65 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp @@ -2053,6 +2053,7 @@ void GuestBufferMemory::addDescriptorRegion(std::uint64_t address, std::size_t b Region region{begin, end, true, {}, nullptr}; region.atomic = atomic; region.swept = swept; + region.lastView = begin; auto committed = GuestMemory::DescribeCommitted(begin, static_cast(end - begin)); if (!committed.whole) { // A GPU heap bound whole while the guest commits its pages on demand, or a descriptor left @@ -2376,6 +2377,7 @@ void GuestBufferMemory::UploadPrepare(bool addressable) { previous.hostBacked = true; } mergeBacked(previous, region); + previous.lastView = std::max({previous.lastView, region.lastView, region.begin}); if (region.end > previous.end) previous.direct = nullptr; previous.atomic = previous.atomic || region.atomic; previous.swept = previous.swept || region.swept; @@ -2702,8 +2704,17 @@ void GuestBufferMemory::copyRegion(Region& region, bool addressable) { } } const auto allocateStart = std::chrono::steady_clock::now(); + if (region.sparse && region.buffer == nullptr && (region.hostBacked || region.writable)) { + const auto backedEnd = region.backed.empty() ? region.begin : region.backed.back().second; + const auto viewEnd = std::max(region.lastView, region.begin) + 4u; + constexpr std::uint64_t granule = 256; + const auto extent = std::max(backedEnd, viewEnd) - region.begin; + const auto end = region.begin + std::min((extent + granule - 1) / granule * granule, bytes); + region.bufferEnd = end < region.end ? end : 0; + } + const auto allocated = region.bufferEnd != 0 ? region.bufferEnd - region.begin : bytes; // A buffer made by UploadPrepare for a GPU copy that fell back here (its import went) is kept. - if (region.buffer == nullptr) region.buffer = std::make_shared(context, static_cast(bytes), usage); + if (region.buffer == nullptr) region.buffer = std::make_shared(context, static_cast(allocated), usage); const auto readStart = std::chrono::steady_clock::now(); if (profile) allocateUs.fetch_add(microsecondsSince(allocateStart), std::memory_order_relaxed); // Named for the [hooksync] attribution: the reads below go through the flush hook. @@ -2717,11 +2728,18 @@ void GuestBufferMemory::copyRegion(Region& region, bool addressable) { if (written) region.uploaded.assign(region.buffer->Bytes().begin(), region.buffer->Bytes().end()); } else { // `uploaded` holds the backed pieces back to back. + const auto target = region.buffer->Bytes().first(static_cast(allocated)); + const bool zeroGaps = region.bufferEnd != 0; + std::size_t cursor = 0; for (const auto& [first, last] : region.backed) { - const auto piece = region.buffer->Bytes().subspan(static_cast(first - region.begin), static_cast(last - first)); + const auto at = static_cast(first - region.begin); + if (zeroGaps && at > cursor) std::memset(target.data() + cursor, 0, at - cursor); + const auto piece = target.subspan(at, static_cast(last - first)); GuestMemory::Read(first, piece); if (written) region.uploaded.insert(region.uploaded.end(), piece.begin(), piece.end()); + cursor = at + piece.size(); } + if (zeroGaps && cursor < target.size()) std::memset(target.data() + cursor, 0, target.size() - cursor); } } else std::memcpy(region.buffer->Bytes().data(), region.snapshot.data(), region.snapshot.size()); @@ -2911,8 +2929,13 @@ VkDescriptorBufferInfo GuestBufferMemory::Descriptor(std::uint64_t address, std: Require(context.limits.minStorageBufferOffsetAlignment != 0, "no storage buffer offset alignment"); Require(base % 4 == 0, "a guest buffer view in a GPU owner that does not start at a DWORD boundary is not implemented"); adjustment = static_cast(offset % std::max(context.limits.minStorageBufferOffsetAlignment, 4)); - const auto range = ViewBytes(bytes, adjustment); - const auto end = region.direct != nullptr ? region.direct->base + region.direct->bytes : region.end; + auto range = ViewBytes(bytes, adjustment); + const bool cut = region.direct == nullptr && region.mirror == nullptr && region.bufferEnd != 0; + const auto end = region.direct != nullptr ? region.direct->base + region.direct->bytes : cut ? region.bufferEnd : region.end; + if (cut) { + Require(address - adjustment < end, "guest buffer view starts past its copy"); + range = std::min(range, end - (address - adjustment)); + } if (address - adjustment + range > end) Require(false, describe("rounded up to its view range exceeds")); Require(range <= context.limits.maxStorageBufferRange, "guest buffer view exceeds descriptor range limit"); const auto handle = region.direct != nullptr ? region.direct->buffer : region.mirror != nullptr ? region.mirror->buffer->Handle() : region.buffer->Handle(); @@ -2928,7 +2951,8 @@ ShaderRecompiler::BdaAbi::Range GuestBufferMemory::addressRange(const Region& re const auto address = region.direct != nullptr ? region.direct->address + (region.begin - region.direct->base) : region.mirror != nullptr ? region.mirror->buffer->DeviceAddress() + (region.begin - region.mirror->base) : region.buffer->DeviceAddress(); Require(region.end - region.begin <= std::numeric_limits::max() - address, "GPU address range overflow"); const auto permissions = ShaderRecompiler::BdaAbi::Read | (region.direct != nullptr && region.writable ? ShaderRecompiler::BdaAbi::Write : 0u); - return {region.begin, region.end, address, permissions, 0}; + const bool cut = region.direct == nullptr && region.mirror == nullptr && region.bufferEnd != 0; + return {region.begin, cut ? region.bufferEnd : region.end, address, permissions, 0}; } std::vector GuestBufferMemory::AddressRanges() const { diff --git a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp index fa8240bd6..5baca0ff9 100644 --- a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp @@ -618,6 +618,71 @@ void stridedOverhangTests(const Context& context, const BdaTestAccess& access) { ::operator delete(block, std::align_val_t{allocation}); } +std::byte* reserveSparse(std::size_t reserved, std::size_t committed) { +#ifdef _WIN32 + auto* block = static_cast(VirtualAlloc(nullptr, reserved, MEM_RESERVE, PAGE_NOACCESS)); + Require(block != nullptr && VirtualAlloc(block, committed, MEM_COMMIT, PAGE_READWRITE) != nullptr, "cannot reserve the sparse heap"); +#else + void* mapped = mmap(nullptr, reserved, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + Require(mapped != MAP_FAILED && mprotect(mapped, committed, PROT_READ | PROT_WRITE) == 0, "cannot reserve the sparse heap"); + auto* block = static_cast(mapped); +#endif + return block; +} + +void releaseSparse(std::byte* block, std::size_t reserved) { +#ifdef _WIN32 + static_cast(reserved); + VirtualFree(block, 0, MEM_RELEASE); +#else + munmap(block, reserved); +#endif +} + +void sparseHeapTests(const Context& context, const BdaTestAccess& access) { + constexpr std::size_t reserved = 64u << 20u; + constexpr std::size_t committed = 65536; + auto* block = reserveSparse(reserved, committed); + std::memset(block, 0x5a, committed); + const auto base = reinterpret_cast(block); + access.limitMemory(4u << 20u); + { + GuestBufferMemory memory(context); + memory.AddWritable(base, reserved); + memory.Upload(false); + std::uint32_t adjustment = 0; + const auto view = memory.Descriptor(base, reserved, adjustment); + Require(adjustment == 0 && view.range == committed, "a sparse heap view is not cut at its last committed byte"); + Require(memory.CopiedBytes() == committed, "a cut sparse heap is charged at its reserved size"); + const auto bytes = access.bytes(view.buffer); + Require(bytes.size() < reserved && bytes[view.offset + committed - 1] == std::byte{0x5a}, "a sparse heap was not copied up to its last committed byte"); + bytes[view.offset + 8] = std::byte{0x77}; + memory.WriteBack(); + Require(block[8] == std::byte{0x77}, "a store to a cut sparse heap was not written back"); + } + { + constexpr std::size_t late = 1u << 20u; + GuestBufferMemory memory(context); + memory.AddWritable(base, reserved); + memory.AddReadable(base + late, 16); + memory.Upload(true); + std::uint32_t adjustment = 0; + const auto view = memory.Descriptor(base + late, 16, adjustment); + Require(view.range == 16 + adjustment, "a view in the uncommitted part of a sparse heap is not bound"); + const auto bytes = access.bytes(view.buffer); + const auto whole = memory.Descriptor(base, reserved, adjustment); + Require(whole.buffer == view.buffer && bytes.size() >= whole.offset + late + 16, "a view in the uncommitted part of a sparse heap has another owner"); + Require(bytes[whole.offset + committed - 1] == std::byte{0x5a} && std::all_of(bytes.begin() + static_cast(whole.offset + committed), bytes.begin() + static_cast(whole.offset + late + 16), [](std::byte byte) { return byte == std::byte{}; }), "the uncommitted part of a cut sparse heap does not read as zeros"); + Require(whole.range == late + 256, "a sparse heap copy does not end past its last view's start"); + Require(memory.CopiedBytes() == late + 256, "a cut sparse heap charge does not include its last view"); + const auto ranges = memory.AddressRanges(); + Require(ranges.size() == 1 && ranges[0].begin == base && ranges[0].end == base + late + 256, "the BDA range of a cut sparse heap is not clamped to its copy"); + memory.WriteBack(); + } + access.limitMemory(std::nullopt); + releaseSparse(block, reserved); +} + void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { alignas(64) std::array guest{}; guest[0] = 123; @@ -781,6 +846,7 @@ void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { importedFreshTests(context); gpuMappingTests(context); stridedOverhangTests(context, access); + sparseHeapTests(context, access); Require(AddressCopyOverflow({{0x1000, 0x3000, 0x2000, "uncommitted pages"}}, 0x2000).empty(), "copies within the limit were refused"); const auto copies = AddressCopyOverflow({{0x1000, 0x2000, 0x1000, "not mirrored"}, {0x10000, 0x30000, 0x18000, "uncommitted pages"}}, 0x2000); Require(!copies.empty() && copies.find("0x10000+0x20000 (0.1 MiB committed, uncommitted pages)") < copies.find("0x1000+0x1000"), "the copy limit does not name the largest copy first"); From 8d96b44917ef0593eec4ab459e87ce675a09c3a9 Mon Sep 17 00:00:00 2001 From: Cesar Rodriguez <11452991+Tamalero@users.noreply.github.com> Date: Sat, 10 Oct 2026 20:05:51 -0700 Subject: [PATCH 321/460] fix(libSceAgc): accept a null base in sceAgcDcbSetBaseIndirectArgs The builder threw on a null indirect base, while the PM4 executor accepts SET_BASE with address 0 (it only requires the alignment and address bits) and an indirect draw that later reads the base is checked there. Titles clear the base with 0, as with sceAgcDcbSetIndexBuffer, which already accepts 0 to unbind. A non-null base is still checked for 8-byte alignment. Co-Authored-By: Claude Opus 5.5 --- .../libs/prx/libSceAgc/DcbDraw/src/Instancing.cpp | 2 +- core/libs/prx/libSceAgc/tests/Commands.cpp | 15 +++++++++++++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgc/DcbDraw/src/Instancing.cpp b/core/libs/prx/libSceAgc/DcbDraw/src/Instancing.cpp index 35a5fcba8..21ec62d11 100644 --- a/core/libs/prx/libSceAgc/DcbDraw/src/Instancing.cpp +++ b/core/libs/prx/libSceAgc/DcbDraw/src/Instancing.cpp @@ -20,7 +20,7 @@ uint32_t* APS5_VABI sceAgcDcbSetBaseIndirectArgs(CommandBuffer* buf, uint32_t sh // SET_BASE with base index 1; header bit 1 selects the dispatch (compute) base over the draw base. Agc::Command::CheckBits(shader_type, 1, __func__); const auto address = reinterpret_cast(indirect_base_addr); - Agc::Command::CheckAddress(address, 8, __func__); + if (address != 0) Agc::Command::CheckAddress(address, 8, __func__); Agc::Command::CheckBits(address, 0xffffffffffffull, __func__); auto* packet = Agc::Command::Allocate(buf, 4, __func__); packet[0] = Agc::Command::Header(0x11u, 4, shader_type != 0 ? 2u : 0u); diff --git a/core/libs/prx/libSceAgc/tests/Commands.cpp b/core/libs/prx/libSceAgc/tests/Commands.cpp index 1f02cc134..c2ea940d6 100644 --- a/core/libs/prx/libSceAgc/tests/Commands.cpp +++ b/core/libs/prx/libSceAgc/tests/Commands.cpp @@ -34,6 +34,7 @@ extern "C" std::uint32_t* APS5_VABI sceAgcAcbPushMarker(CommandBuffer* buf, cons extern "C" std::uint32_t* APS5_VABI sceAgcAcbPopMarker(CommandBuffer* buf); extern "C" std::uint32_t* APS5_VABI sceAgcAcbSetMarker(CommandBuffer* buf, const char* str, std::uint32_t color); extern "C" std::uint32_t* APS5_VABI sceAgcDcbSetIndexBuffer(CommandBuffer* buf, std::uint64_t indexAddress); +extern "C" std::uint32_t* APS5_VABI sceAgcDcbSetBaseIndirectArgs(CommandBuffer* buf, std::uint32_t shaderType, const volatile void* indirectBaseAddr); extern "C" std::uint32_t* APS5_VABI sceAgcSetNop(CommandBuffer* buf, std::uint32_t sizeDw); namespace { @@ -221,6 +222,19 @@ void testIndexBuffer() { throw std::runtime_error("misaligned index buffer was accepted"); } +void testIndirectBase() { + Storage storage; + alignas(8) std::uint32_t arguments[4]{}; + const auto address = reinterpret_cast(arguments); + const auto* bound = sceAgcDcbSetBaseIndirectArgs(&storage.buffer, 0, arguments); + check(bound[1] == 1 && bound[2] == static_cast(address) && bound[3] == static_cast(address >> 32u), "indirect base address mismatch"); + const auto* cleared = sceAgcDcbSetBaseIndirectArgs(&storage.buffer, 1, nullptr); + check(cleared == bound + 4 && (cleared[0] & 2u) != 0 && cleared[1] == 1 && cleared[2] == 0 && cleared[3] == 0, "indirect base was not cleared"); + const auto before = storage.words; + expectFailure([&] { sceAgcDcbSetBaseIndirectArgs(&storage.buffer, 0, reinterpret_cast(address + 4)); }); + check(storage.words == before, "misaligned indirect base modified packet memory"); +} + void testContextState() { const std::array, 4> sizes{{{5}, {5, 8, 9, 3, 2}, {3, 5, 8, 9, 2}, {5, 8, 9, 3, 2, 5}}}; const std::array, 4> reservations{{{5}, {22, 3, 2}, {3, 22, 2}, {22, 3, 2, 5}}}; @@ -425,6 +439,7 @@ int main(int argc, char** argv) { testIndexedIndirectDraws(); testMarkers(); testIndexBuffer(); + testIndirectBase(); testContextState(); testFlip(); testRegisters(); From 13c03910a48caef62359c6c4fc67e85358409078 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Thu, 8 Oct 2026 21:14:28 -0500 Subject: [PATCH 322/460] feat(agc): run RADV's dispatches with a partial last thread group RADV sets PARTIAL_TG_EN in the dispatch initiator for an unaligned direct dispatch, with each axis's last-group size in the upper half of COMPUTE_NUM_THREAD_X/Y/Z, and TUNNEL_ENABLE on every GFX10 dispatch. The validator rejected PARTIAL_TG_EN, and DecodeComputeStageInfo read the whole register, so a partial size would have widened the workgroup. A direct dispatch with PARTIAL_TG_EN now runs (groups - 1) * full + last threads on each axis through the dispatch-size limit USE_THREAD_DIMENSIONS already uses, or plain groups when every last group is complete; a zero or oversized last group throws. The workgroup size is NUM_THREAD_FULL. Co-Authored-By: Claude Opus 5.5 --- .../libSceAgcDriver/Execution/include/Pm4.hpp | 1 + .../Execution/src/Driver/Dispatch/Dispatch.cpp | 4 ++++ .../prx/libSceAgcDriver/Execution/src/Pm4.cpp | 16 +++++++++++++++- .../Graphics/src/ShaderInputState.cpp | 6 +++--- core/libs/prx/libSceAgcDriver/tests/Graphics.cpp | 3 +++ core/libs/prx/libSceAgcDriver/tests/Pm4.cpp | 8 ++++++++ 6 files changed, 34 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/Pm4.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/Pm4.hpp index e1a24ec60..08493a9a7 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/Pm4.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/Pm4.hpp @@ -146,6 +146,7 @@ std::optional DecodeMemoryCopy(std::span packet // recorded GPU work that writes them), and both steps in one for the packet. std::uint64_t DispatchArgumentAddress(std::span packet, const QueueState& queue); std::array ReadDispatchArguments(std::uint64_t arguments, std::uint32_t initiator); +std::array PartialGroupThreads(const std::array& groups, const std::array& numThreads); std::array ResolveDispatch(std::span packet, const QueueState& queue); DrawParameters ResolveDraw(std::span packet, const QueueState& queue); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp index dcfcc848f..7b113d655 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp @@ -85,6 +85,10 @@ void Driver::dispatch(QueueState& queue, std::span packet, if (threads[axis] % compute.numThreads[axis] != 0) compute.partialThreads = threads; } } + if (indirectArguments == 0 && (packet[4] & 0x2u) != 0) { + const auto threads = Pm4::PartialGroupThreads({packet[1], packet[2], packet[3]}, {readRegister(queue.shader, 0x207), readRegister(queue.shader, 0x208), readRegister(queue.shader, 0x209)}); + if (threads != std::array{}) compute.partialThreads = threads; + } ShaderRecompiler::RecompileRequest request{ {ShaderRecompiler::ShaderStage::Compute, address, std::span(snapshot.code).subspan(codeOffset), snapshot.headerAddress, snapshot.header}, {(packet[4] & 0x8000u) != 0 ? 32u : 64u, 0, userData, compute, std::nullopt, std::nullopt, memory, RegisteredFloatMode(snapshot)}, diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp index 7520ab41b..9ac51ef5a 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp @@ -323,7 +323,7 @@ void Validate(std::span packet, std::uint32_t queue) { require((packet[8] & 3u) == 0 && packet[8] >= (opcode == 0x2c ? 16u : 20u), "invalid indirect draw stride"); require((packet[9] & ~0x20u) == (opcode == 0x2c ? 2u : 0u), "unsupported indirect draw initiator"); break; - case 0x15: size(5); if ((packet[4] & ~0xa024u) != 0x41u) throw std::runtime_error("dispatch modifiers 0x" + ToHex(packet[4]) + " are not implemented"); break; + case 0x15: size(5); if ((packet[4] & ~0xa026u) != 0x41u) throw std::runtime_error("dispatch modifiers 0x" + ToHex(packet[4]) + " are not implemented"); break; case 0x16: require(packet.size() == 3 || packet.size() == 4, "invalid indirect dispatch size"); require((packet.back() & ~0xa024u) == 0x41u, "indirect dispatch modifiers are not implemented"); @@ -636,6 +636,20 @@ std::uint64_t DispatchArgumentAddress(std::span packet, con return queue.dispatchIndirectBase + packet[1]; } +std::array PartialGroupThreads(const std::array& groups, const std::array& numThreads) { + std::array threads{}; + bool partial = false; + for (std::size_t axis = 0; axis < 3; ++axis) { + const auto full = numThreads[axis] & 0xffffu; + const auto last = numThreads[axis] >> 16u; + require(last != 0 && last <= full, "COMPUTE_NUM_THREAD partial group size is zero or exceeds the full group"); + if (groups[axis] == 0) return {}; + threads[axis] = (groups[axis] - 1) * full + last; + partial |= last != full; + } + return partial ? threads : std::array{}; +} + std::array ReadDispatchArguments(std::uint64_t arguments, std::uint32_t initiator) { std::array result{0xc0031500u, 0, 0, 0, initiator}; // Named for the [hooksync] attribution (the read goes through the flush hook). diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderInputState.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderInputState.cpp index 1939acc1e..fa18f41d1 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderInputState.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderInputState.cpp @@ -64,9 +64,9 @@ template void _readHeaderArray(std::span header, s } ShaderRecompiler::ShaderComputeStageInfo DecodeComputeStageInfo(const Registers& shader, std::span header) { - const auto numThreadX = read(shader, computeNumThreadX, RegisterBank::Shader); - const auto numThreadY = read(shader, computeNumThreadY, RegisterBank::Shader); - const auto numThreadZ = read(shader, computeNumThreadZ, RegisterBank::Shader); + const auto numThreadX = read(shader, computeNumThreadX, RegisterBank::Shader) & 0xffffu; + const auto numThreadY = read(shader, computeNumThreadY, RegisterBank::Shader) & 0xffffu; + const auto numThreadZ = read(shader, computeNumThreadZ, RegisterBank::Shader) & 0xffffu; if (numThreadX == 0 || numThreadY == 0 || numThreadZ == 0) { throw std::runtime_error("AGC graphics: COMPUTE_NUM_THREAD_X/Y/Z must be nonzero"); } diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index cf7a795c0..dac6062e3 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -564,6 +564,9 @@ void ComputeScratchTests() { expectFailure([&] { static_cast(AgcDriver::Graphics::DecodeComputeStageInfo(shader, header)); }, "zero scratch size"); shader[0x213] = 0u; Require(AgcDriver::Graphics::DecodeComputeStageInfo(shader, {}).scratchDwords == 0u, "a dispatch without SCRATCH_EN got scratch"); + shader[0x207] = 0x00100040u; + Require(AgcDriver::Graphics::DecodeComputeStageInfo(shader, {}).numThreads[0] == 64u, "the partial group size leaked into the group size"); + shader[0x207] = 64u; ShaderRecompiler::RecompileRequest request{}; const std::array code{0xbf810000u}; request.shader = {ShaderRecompiler::ShaderStage::Compute, 0x30000u, code, 0, {}}; diff --git a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp index e6ef42a26..12844b6ab 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp @@ -182,6 +182,14 @@ void testContextAndBases() { expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x15, {1, 1, 1, 0x4041}), 0); }, "dispatch modifiers"); AgcDriver::Pm4::Validate(makePacket(0x15, {0x280, 1, 1, 0x2045}, 2), 0); AgcDriver::Pm4::Validate(makePacket(0x16, {0, 0x2045}, 2), 0); + AgcDriver::Pm4::Validate(makePacket(0x15, {0xc0, 1, 1, 0x2047}, 2), 0); + expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x16, {0, 0x2047}, 2), 0); }, "indirect dispatch modifiers"); + using Threads = std::array; + check(AgcDriver::Pm4::PartialGroupThreads({0xc0, 1, 1}, {0x00400040, 0x00010001, 0x00010001}) == Threads{}, "complete last groups reported partial threads"); + check(AgcDriver::Pm4::PartialGroupThreads({3, 2, 1}, {0x00100040, 0x00080008, 0x00010001}) == Threads{144, 16, 1}, "partial last group threads lost"); + check(AgcDriver::Pm4::PartialGroupThreads({0, 1, 1}, {0x00100040, 0x00010001, 0x00010001}) == Threads{}, "an empty dispatch reported partial threads"); + expectFailure([] { static_cast(AgcDriver::Pm4::PartialGroupThreads({1, 1, 1}, {0x00000040, 0x00010001, 0x00010001})); }, "partial group size"); + expectFailure([] { static_cast(AgcDriver::Pm4::PartialGroupThreads({1, 1, 1}, {0x00410040, 0x00010001, 0x00010001})); }, "partial group size"); expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x15, {1, 1, 1, 0x41}, 4), 0); }, "header flags"); expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x15, {1, 1, 1, 0x49}), 0); }, "dispatch modifiers"); execute(state, makePacket(0x13, {32})); From 2dad44f8dfa7abe40560952c2b6abd5643971525 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Thu, 8 Oct 2026 22:26:12 -0500 Subject: [PATCH 323/460] feat(agc): load context registers from memory with COPY_DATA Where the GPU has no LOAD_CONTEXT_REG_INDEX, RADV loads an image's clear values into DB_STENCIL_CLEAR/DB_DEPTH_CLEAR and CB_COLORn_CLEAR_WORD0/1 with COPY_DATA from memory to a register (DST_SEL 0), one or two dwords. The validator rejected every register destination. A COPY_DATA from memory to one or two context registers is now accepted. It is not a memory store, so the packet drains like other COPY_DATA the driver cannot record on the GPU, and executing it reads the source then and writes the registers. Register, GDS, immediate and clock sources, and destinations outside the context registers, are still rejected. Co-Authored-By: Claude Opus 5.5 --- .../prx/libSceAgcDriver/Execution/src/Pm4.cpp | 27 +++++++++++++++++-- core/libs/prx/libSceAgcDriver/tests/Pm4.cpp | 10 +++++++ 2 files changed, 35 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp index 9ac51ef5a..ed7240676 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp @@ -135,6 +135,15 @@ void copyMemory(std::uint64_t source, std::uint64_t destination, std::size_t byt constexpr std::uint32_t CopyDataGpuClockSource = 18; constexpr std::uint32_t CopyDataCachePolicy = (3u << 13u) | (3u << 25u); +constexpr std::uint32_t ContextRegisterBase = 0xa000; + +bool copyDataToRegister(std::span packet) { + return ((packet[1] >> 8u) & 0xfu) == 0; +} + +std::uint32_t copyDataCount(std::span packet) { + return (packet[1] & 0x10000u) != 0 ? 2u : 1u; +} std::uint64_t gpuClockCount() { return static_cast(std::chrono::duration_cast(std::chrono::steady_clock::now().time_since_epoch()).count() / 10); @@ -453,7 +462,13 @@ void Validate(std::span packet, std::uint32_t queue) { require((packet[1] & ~(0x40110f0fu | CopyDataCachePolicy)) == 0, "COPY_DATA engine or reserved fields are not implemented"); const auto source = ((packet[1] & 0xfu) << 1u) | ((packet[1] >> 30u) & 1u); const auto destination = ((packet[1] >> 8u) & 0xfu) << 1u; - require(destination == 2 || destination == 4, "COPY_DATA register or GDS destination is not implemented"); + if (copyDataToRegister(packet)) { + require(source == 2 || source == 4, "COPY_DATA to a register from a register, GDS, immediate or clock source is not implemented"); + require((packet[2] & 3u) == 0, "misaligned COPY_DATA source"); + require(packet[5] == 0 && packet[4] >= ContextRegisterBase && packet[4] - ContextRegisterBase <= 0x400u - copyDataCount(packet), "COPY_DATA to a register outside the context registers is not implemented"); + break; + } + require(destination == 2 || destination == 4, "COPY_DATA GDS destination is not implemented"); require(source == 2 || source == 4 || source == 5 || source == 10 || source == 11 || source == CopyDataGpuClockSource, "COPY_DATA register, GDS or reference-clock source is not implemented"); require(source < 10 || source == CopyDataGpuClockSource || ((packet[1] & 0x10000u) == 0 && packet[3] == 0), "64-bit immediate COPY_DATA is not implemented"); break; @@ -579,7 +594,7 @@ std::optional ResolveStore(std::span packet, co const auto fits = [limit](std::uint64_t destination, std::size_t bytes) { return bytes <= limit && destination % 4 == 0 && bytes % 4 == 0; }; switch ((packet[0] >> 8u) & 0xffu) { case 0x40: { - if (packet.size() < 6) return std::nullopt; + if (packet.size() < 6 || copyDataToRegister(packet)) return std::nullopt; const auto source = ((packet[1] & 0xfu) << 1u) | ((packet[1] >> 30u) & 1u); const std::size_t bytes = (packet[1] & 0x10000u) != 0 ? 8 : 4; const auto destination = address(packet[4], packet[5]); @@ -905,6 +920,14 @@ void Execute(std::span packet, QueueState& queue) { return; } case 0x40: { + if (copyDataToRegister(packet)) { + std::array values{}; + const auto count = copyDataCount(packet); + const GuestMemory::ReadSiteScope site(GuestMemory::ReadSite::Registers); + GuestMemory::Read(address(packet[2], packet[3]), std::as_writable_bytes(std::span(values).first(count)), 4); + for (std::uint32_t i = 0; i < count; ++i) writeRegister(queue, 0x69, packet[4] - ContextRegisterBase + i, values[i]); + return; + } const auto destination = address(packet[4], packet[5]); const auto data = copyDataSource(packet, (packet[1] & 0x10000u) != 0 ? 8 : 4); GuestMemory::CheckRange(reinterpret_cast(destination), data.size(), 1, true); diff --git a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp index 12844b6ab..6693461bc 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp @@ -359,6 +359,16 @@ void testCopies() { check(destination[0] == 11 && destination[1] == 12 && destination[2] == 0, "64-bit COPY_DATA failed"); execute(state, makePacket(0x40, {0x105, 0x12345678, 0, low(destination.data()), high(destination.data())})); check(destination[0] == 0x12345678, "immediate COPY_DATA failed"); + alignas(8) std::array clearValues{0x55, 0x3f800000}; + execute(state, makePacket(0x40, {0x10001, low(clearValues.data()), high(clearValues.data()), 0xa00a, 0})); + check(state.context.at(0x0a) == 0x55 && state.context.at(0x0b) == 0x3f800000, "COPY_DATA to two context registers failed"); + execute(state, makePacket(0x40, {0x1, low(&clearValues[1]), high(&clearValues[1]), 0xa323, 0})); + check(state.context.at(0x323) == 0x3f800000 && !state.context.contains(0x324), "COPY_DATA to one context register failed"); + check(!AgcDriver::Pm4::ResolveStore(makePacket(0x40, {0x1, low(clearValues.data()), high(clearValues.data()), 0xa00a, 0}), state, 64).has_value(), "COPY_DATA to a register resolved as a memory store"); + expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x40, {0x5, 7, 0, 0xa00a, 0}), 0); }, "to a register from a register, GDS, immediate"); + expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x40, {0x1, low(clearValues.data()), high(clearValues.data()), 0x2e40, 0}), 0); }, "outside the context registers"); + expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x40, {0x10001, low(clearValues.data()), high(clearValues.data()), 0xa3ff, 0}), 0); }, "outside the context registers"); + expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x40, {0x1, low(clearValues.data()) + 2, high(clearValues.data()), 0xa00a, 0}), 0); }, "misaligned COPY_DATA source"); alignas(8) std::array clock{}; const auto clockCopy = [&](std::uint64_t* target) { return makePacket(0x40, {0x06016209, 0, 0, low(target), high(target)}); }; execute(state, clockCopy(&clock[0])); From 4390d4b7a545f36e6ffc18dbc1f648127ebb7d71 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Thu, 8 Oct 2026 22:29:20 -0500 Subject: [PATCH 324/460] feat(agc): accept COPY_DATA to memory through DST_SEL 5 RADV writes every COPY_DATA memory destination with DST_SEL 5 (COPY_DATA_DST_MEM in Mesa's sid.h), the current form of the deprecated GRBM-synchronized DST_SEL 1 that was already accepted. vkQuake's timestamp writes use it. It is now validated, resolved and executed as memory like 1 and 2; GDS and performance-counter destinations are still rejected. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp | 2 +- core/libs/prx/libSceAgcDriver/tests/Pm4.cpp | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp index ed7240676..e8f42a21d 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp @@ -468,7 +468,7 @@ void Validate(std::span packet, std::uint32_t queue) { require(packet[5] == 0 && packet[4] >= ContextRegisterBase && packet[4] - ContextRegisterBase <= 0x400u - copyDataCount(packet), "COPY_DATA to a register outside the context registers is not implemented"); break; } - require(destination == 2 || destination == 4, "COPY_DATA GDS destination is not implemented"); + require(destination == 2 || destination == 4 || destination == 10, "COPY_DATA GDS or performance-counter destination is not implemented"); require(source == 2 || source == 4 || source == 5 || source == 10 || source == 11 || source == CopyDataGpuClockSource, "COPY_DATA register, GDS or reference-clock source is not implemented"); require(source < 10 || source == CopyDataGpuClockSource || ((packet[1] & 0x10000u) == 0 && packet[3] == 0), "64-bit immediate COPY_DATA is not implemented"); break; diff --git a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp index 6693461bc..119f20056 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp @@ -359,6 +359,12 @@ void testCopies() { check(destination[0] == 11 && destination[1] == 12 && destination[2] == 0, "64-bit COPY_DATA failed"); execute(state, makePacket(0x40, {0x105, 0x12345678, 0, low(destination.data()), high(destination.data())})); check(destination[0] == 0x12345678, "immediate COPY_DATA failed"); + execute(state, makePacket(0x40, {0x505, 0x7654321, 0, low(destination.data()), high(destination.data())})); + check(destination[0] == 0x7654321, "immediate COPY_DATA to DST_SEL 5 memory failed"); + alignas(8) std::array stamp{}; + execute(state, makePacket(0x40, {0x110509, 0, 0, low(stamp.data()), high(stamp.data())})); + check(stamp[0] != 0, "GPU timestamp COPY_DATA to DST_SEL 5 memory failed"); + expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x40, {0x305, 1, 0, 0, 0}), 0); }, "GDS or performance-counter destination"); alignas(8) std::array clearValues{0x55, 0x3f800000}; execute(state, makePacket(0x40, {0x10001, low(clearValues.data()), high(clearValues.data()), 0xa00a, 0})); check(state.context.at(0x0a) == 0x55 && state.context.at(0x0b) == 0x3f800000, "COPY_DATA to two context registers failed"); From 01909c914e1aec1ebc749d2ab611c36cddda3b1e Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Thu, 8 Oct 2026 22:44:37 -0500 Subject: [PATCH 325/460] feat(agc): accept dispatches that launch waves in order RADV clears ORDER_MODE in the dispatch initiator for a dispatch that needs its waves launched in order, and the validator required it set. In-order launch is the stricter guarantee and the driver does not reorder waves either way, so a direct or indirect dispatch now needs only COMPUTE_SHADER_EN; vkQuake's map load issues one. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp | 4 ++-- core/libs/prx/libSceAgcDriver/tests/Pm4.cpp | 3 +++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp index e8f42a21d..42830764c 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp @@ -332,10 +332,10 @@ void Validate(std::span packet, std::uint32_t queue) { require((packet[8] & 3u) == 0 && packet[8] >= (opcode == 0x2c ? 16u : 20u), "invalid indirect draw stride"); require((packet[9] & ~0x20u) == (opcode == 0x2c ? 2u : 0u), "unsupported indirect draw initiator"); break; - case 0x15: size(5); if ((packet[4] & ~0xa026u) != 0x41u) throw std::runtime_error("dispatch modifiers 0x" + ToHex(packet[4]) + " are not implemented"); break; + case 0x15: size(5); if ((packet[4] & ~0xa066u) != 0x1u) throw std::runtime_error("dispatch modifiers 0x" + ToHex(packet[4]) + " are not implemented"); break; case 0x16: require(packet.size() == 3 || packet.size() == 4, "invalid indirect dispatch size"); - require((packet.back() & ~0xa024u) == 0x41u, "indirect dispatch modifiers are not implemented"); + require((packet.back() & ~0xa064u) == 0x1u, "indirect dispatch modifiers are not implemented"); break; case 0x22: size(5); diff --git a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp index 119f20056..3c5bbb9b0 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp @@ -183,6 +183,9 @@ void testContextAndBases() { AgcDriver::Pm4::Validate(makePacket(0x15, {0x280, 1, 1, 0x2045}, 2), 0); AgcDriver::Pm4::Validate(makePacket(0x16, {0, 0x2045}, 2), 0); AgcDriver::Pm4::Validate(makePacket(0x15, {0xc0, 1, 1, 0x2047}, 2), 0); + AgcDriver::Pm4::Validate(makePacket(0x15, {0xc0, 1, 1, 0x2007}, 2), 0); + AgcDriver::Pm4::Validate(makePacket(0x16, {0, 0x2005}, 2), 0); + expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x15, {1, 1, 1, 0x40}), 0); }, "dispatch modifiers"); expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x16, {0, 0x2047}, 2), 0); }, "indirect dispatch modifiers"); using Threads = std::array; check(AgcDriver::Pm4::PartialGroupThreads({0xc0, 1, 1}, {0x00400040, 0x00010001, 0x00010001}) == Threads{}, "complete last groups reported partial threads"); From 40f243facc808cf93f025da0deaa416d120f936d Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Fri, 9 Oct 2026 00:45:26 -0500 Subject: [PATCH 326/460] feat(agc): read indirect draw register location 0 as no register RADV emits 0 as the start-instance location of DRAW_INDIRECT and DRAW_INDEX_INDIRECT when the vertex shader does not read the base instance, and as the draw-index location of the multi forms when draw IDs are off. The validator accepted only 0x280 as "no register". Location 0 is now accepted and decoded as 0x280; vkQuake's draws use it. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp | 9 +++++---- core/libs/prx/libSceAgcDriver/tests/Pm4.cpp | 6 ++++++ 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp index 42830764c..49898a481 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp @@ -72,7 +72,7 @@ bool IndirectDrawsDisabled() { // nowhere) or a user-data register of the vertex-side stage banks (0x8c.. for the vertex / geometry // programs, 0x10c.. for the local / hull programs). bool drawLocation(std::uint32_t value) { - return value == 0x280u || value - 0x8cu < 32u || value - 0x10cu < 32u; + return value == 0 || value == 0x280u || value - 0x8cu < 32u || value - 0x10cu < 32u; } Registers& registersFor(QueueState& queue, std::uint32_t opcode) { @@ -702,9 +702,10 @@ DrawParameters resolveIndirectDraw(std::span packet, const indirect.count = multi ? packet[5] : 1; indirect.countIndirect = multi && (packet[4] & (1u << 30u)) != 0; indirect.countAddress = multi ? address(packet[6], packet[7]) : 0; - indirect.baseVertexLocation = packet[2]; - indirect.startInstanceLocation = packet[3]; - indirect.drawIndexLocation = multi ? packet[4] & 0xffffu : 0x280u; + const auto location = [](std::uint32_t value) { return value == 0 ? 0x280u : value; }; + indirect.baseVertexLocation = location(packet[2]); + indirect.startInstanceLocation = location(packet[3]); + indirect.drawIndexLocation = multi ? location(packet[4] & 0xffffu) : 0x280u; indirect.drawIndexEnabled = multi && (packet[4] >> 31u) != 0; indirect.indxOffset = 0; require(indirect.count <= (std::numeric_limits::max() - indirect.arguments - indirect.recordBytes) / indirect.stride, "indirect draw record range overflow"); diff --git a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp index 3c5bbb9b0..a877a3958 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp @@ -304,6 +304,12 @@ void testIndirectDraw() { check(draw.indirect->baseVertexLocation == 0x280 && draw.indirect->startInstanceLocation == 0x8e && draw.indirect->drawIndexLocation == 0x280 && !draw.indirect->drawIndexEnabled && draw.indirect->indxOffset == 5 && draw.firstVertex == 5 && !draw.indexed && draw.indexCount == 0 && draw.instanceCount == 0, "indirect draw locations mismatch"); check(draw.indirect->RangeBytes() == 16 && draw.indirect->VertexDwordOffset() == 8 && draw.indirect->InstanceDwordOffset() == 12, "indirect draw record geometry mismatch"); check(AgcDriver::Pm4::ResolveDraw(makePacket(0x24, {0x60, 0x280, 0x8e, 2}), state).indirect->arguments == 0x105d87fca0ull, "indirect draw offset not applied"); + AgcDriver::Pm4::Validate(makePacket(0x24, {0, 0x8d, 0, 2}), 0); + const auto unused = AgcDriver::Pm4::ResolveDraw(makePacket(0x24, {0, 0x8d, 0, 2}), state); + check(unused.indirect->baseVertexLocation == 0x8d && unused.indirect->startInstanceLocation == 0x280, "an indirect draw location 0 did not mean no register"); + const auto unusedMulti = AgcDriver::Pm4::ResolveDraw(makePacket(0x2c, {0x20, 0x8c, 0, 0, 3, 0, 0, 32, 0x22}), state); + check(unusedMulti.indirect->startInstanceLocation == 0x280 && unusedMulti.indirect->drawIndexLocation == 0x280, "a multi indirect draw location 0 did not mean no register"); + expectFailure([] { AgcDriver::Pm4::Validate(makePacket(0x24, {0, 0x8d, 0x200, 2}), 0); }, "invalid indirect draw register location"); const auto multi = AgcDriver::Pm4::ResolveDraw(makePacket(0x2c, {0x20, 0x8c, 0x280, 0x8d | (1u << 31u), 3, 0, 0, 32, 0x22}), state); check(multi.indirect->count == 3 && multi.indirect->stride == 32 && multi.indirect->drawIndexEnabled && multi.indirect->drawIndexLocation == 0x8d && !multi.indirect->countIndirect && multi.flags == 0x20 && multi.indirect->RangeBytes() == 80, "indirect multi-draw mismatch"); expectFailure([&] { AgcDriver::Pm4::ResolveDraw(makePacket(0x25, {0, 0x8c, 0x280, 0}), state); }, "index base"); From 44abcc43b19e5af9c12317107a35c01e33541638 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Fri, 9 Oct 2026 00:49:28 -0500 Subject: [PATCH 327/460] feat(agc): accept 4-byte-aligned indirect argument bases RADV sets the draw and dispatch indirect base with SET_BASE to the argument buffer address plus the command's offset, which Vulkan requires only to be a multiple of 4; vkQuake's base ends in 0x14. The validator required 8-byte alignment. The records are read as 32-bit words, so a 4-byte-aligned base is now accepted. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp | 2 +- core/libs/prx/libSceAgcDriver/tests/Pm4.cpp | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp index 49898a481..ba8f5bb2f 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp @@ -276,7 +276,7 @@ void Validate(std::span packet, std::uint32_t queue) { switch (opcode) { case 0x11: size(4); - require(packet[1] == 1 && (packet[2] & 7u) == 0 && packet[3] <= 0xffffu, "unsupported indirect base index, alignment or address bits"); + require(packet[1] == 1 && (packet[2] & 3u) == 0 && packet[3] <= 0xffffu, "unsupported indirect base index, alignment or address bits"); if ((header & 2u) == 0) graphics(); break; case 0x12: graphics(); size(2); require((packet[1] & ~0xfu) == 0, "unsupported CLEAR_STATE payload bits"); break; diff --git a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp index a877a3958..36ca54041 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp @@ -305,6 +305,8 @@ void testIndirectDraw() { check(draw.indirect->RangeBytes() == 16 && draw.indirect->VertexDwordOffset() == 8 && draw.indirect->InstanceDwordOffset() == 12, "indirect draw record geometry mismatch"); check(AgcDriver::Pm4::ResolveDraw(makePacket(0x24, {0x60, 0x280, 0x8e, 2}), state).indirect->arguments == 0x105d87fca0ull, "indirect draw offset not applied"); AgcDriver::Pm4::Validate(makePacket(0x24, {0, 0x8d, 0, 2}), 0); + AgcDriver::Pm4::Validate(makePacket(0x11, {1, 0x10400014, 0x40}), 0); + expectFailure([] { AgcDriver::Pm4::Validate(makePacket(0x11, {1, 0x10400012, 0x40}), 0); }, "alignment"); const auto unused = AgcDriver::Pm4::ResolveDraw(makePacket(0x24, {0, 0x8d, 0, 2}), state); check(unused.indirect->baseVertexLocation == 0x8d && unused.indirect->startInstanceLocation == 0x280, "an indirect draw location 0 did not mean no register"); const auto unusedMulti = AgcDriver::Pm4::ResolveDraw(makePacket(0x2c, {0x20, 0x8c, 0, 0, 3, 0, 0, 32, 0x22}), state); From fdf683ac1b3af82938e4e3856cf361f52da9d943 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sat, 10 Oct 2026 14:40:25 -0500 Subject: [PATCH 328/460] fix(agc): run a partial last thread group through the shader and validate every axis The buffer fill and copy fast paths returned before the partial last group was applied, so a PARTIAL_TG_EN dispatch of the fill kernel filled whole groups, and a zero last group did not throw. The partial sizes are now decoded before the fast paths, which are skipped when the last group is partial; the dispatch then runs the shader with the partial thread counts. PartialGroupThreads validates all three axes before an empty dispatch returns, and throws when (groups - 1) * full + last does not fit in 32 bits instead of wrapping. Validate rejects PARTIAL_TG_EN combined with USE_THREAD_DIMENSIONS (initiator 0x63). agc_driver_pattern_buffer_fill now runs the fill kernel with a 16-of-64 last group through the driver and checks that 80 dwords are written. AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- .../src/Driver/Dispatch/Dispatch.cpp | 18 ++++++++------- .../prx/libSceAgcDriver/Execution/src/Pm4.cpp | 14 +++++++---- core/libs/prx/libSceAgcDriver/tests/Pm4.cpp | 3 +++ .../tests/execution/PatternBufferFill.cpp | 23 ++++++++++++++++++- 4 files changed, 45 insertions(+), 13 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp index 7b113d655..8a467113c 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp @@ -65,16 +65,21 @@ void Driver::dispatch(QueueState& queue, std::span packet, packet = resolved; indirectArguments = 0; } + std::array partialGroups{}; + if (indirectArguments == 0 && (packet[4] & 0x2u) != 0) { + partialGroups = Pm4::PartialGroupThreads({packet[1], packet[2], packet[3]}, {readRegister(queue.shader, 0x207), readRegister(queue.shader, 0x208), readRegister(queue.shader, 0x209)}); + } + const bool partialLastGroup = partialGroups != std::array{}; if (resolving) { - if (matchesFillKernel(std::span(snapshot.code).subspan(codeOffset), userData, compute)) return; - if (indirectArguments == 0 && matchesCopyKernel(std::span(snapshot.code).subspan(codeOffset), userData, compute)) return; + if (!partialLastGroup && matchesFillKernel(std::span(snapshot.code).subspan(codeOffset), userData, compute)) return; + if (!partialLastGroup && indirectArguments == 0 && matchesCopyKernel(std::span(snapshot.code).subspan(codeOffset), userData, compute)) return; } else { if (const auto htile = Graphics::HtileDepthClearAddress(std::span(snapshot.code).subspan(codeOffset), userData, compute.numThreads); htile != 0) Graphics::NoteHtileDepthClear(htile); - if (fillBuffer(queue, submission.queue, packet, std::span(snapshot.code).subspan(codeOffset), userData, compute, localDevice)) { + if (!partialLastGroup && fillBuffer(queue, submission.queue, packet, std::span(snapshot.code).subspan(codeOffset), userData, compute, localDevice)) { pendingDispatchPhases().outcome = DispatchOutcome::FillHle; return; } - if (indirectArguments == 0 && copyBuffer(queue, submission.queue, packet, std::span(snapshot.code).subspan(codeOffset), userData, compute, localDevice, address)) { + if (!partialLastGroup && indirectArguments == 0 && copyBuffer(queue, submission.queue, packet, std::span(snapshot.code).subspan(codeOffset), userData, compute, localDevice, address)) { pendingDispatchPhases().outcome = DispatchOutcome::CopyHle; return; } @@ -85,10 +90,7 @@ void Driver::dispatch(QueueState& queue, std::span packet, if (threads[axis] % compute.numThreads[axis] != 0) compute.partialThreads = threads; } } - if (indirectArguments == 0 && (packet[4] & 0x2u) != 0) { - const auto threads = Pm4::PartialGroupThreads({packet[1], packet[2], packet[3]}, {readRegister(queue.shader, 0x207), readRegister(queue.shader, 0x208), readRegister(queue.shader, 0x209)}); - if (threads != std::array{}) compute.partialThreads = threads; - } + if (partialLastGroup) compute.partialThreads = partialGroups; ShaderRecompiler::RecompileRequest request{ {ShaderRecompiler::ShaderStage::Compute, address, std::span(snapshot.code).subspan(codeOffset), snapshot.headerAddress, snapshot.header}, {(packet[4] & 0x8000u) != 0 ? 32u : 64u, 0, userData, compute, std::nullopt, std::nullopt, memory, RegisteredFloatMode(snapshot)}, diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp index ba8f5bb2f..e76a0572d 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp @@ -332,7 +332,7 @@ void Validate(std::span packet, std::uint32_t queue) { require((packet[8] & 3u) == 0 && packet[8] >= (opcode == 0x2c ? 16u : 20u), "invalid indirect draw stride"); require((packet[9] & ~0x20u) == (opcode == 0x2c ? 2u : 0u), "unsupported indirect draw initiator"); break; - case 0x15: size(5); if ((packet[4] & ~0xa066u) != 0x1u) throw std::runtime_error("dispatch modifiers 0x" + ToHex(packet[4]) + " are not implemented"); break; + case 0x15: size(5); if ((packet[4] & ~0xa066u) != 0x1u || (packet[4] & 0x22u) == 0x22u) throw std::runtime_error("dispatch modifiers 0x" + ToHex(packet[4]) + " are not implemented"); break; case 0x16: require(packet.size() == 3 || packet.size() == 4, "invalid indirect dispatch size"); require((packet.back() & ~0xa064u) == 0x1u, "indirect dispatch modifiers are not implemented"); @@ -654,15 +654,21 @@ std::uint64_t DispatchArgumentAddress(std::span packet, con std::array PartialGroupThreads(const std::array& groups, const std::array& numThreads) { std::array threads{}; bool partial = false; + bool empty = false; for (std::size_t axis = 0; axis < 3; ++axis) { const auto full = numThreads[axis] & 0xffffu; const auto last = numThreads[axis] >> 16u; require(last != 0 && last <= full, "COMPUTE_NUM_THREAD partial group size is zero or exceeds the full group"); - if (groups[axis] == 0) return {}; - threads[axis] = (groups[axis] - 1) * full + last; partial |= last != full; + if (groups[axis] == 0) { + empty = true; + continue; + } + const auto total = static_cast(groups[axis] - 1) * full + last; + require(total <= std::numeric_limits::max(), "partial thread group dispatch size exceeds 32 bits"); + threads[axis] = static_cast(total); } - return partial ? threads : std::array{}; + return partial && !empty ? threads : std::array{}; } std::array ReadDispatchArguments(std::uint64_t arguments, std::uint32_t initiator) { diff --git a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp index 36ca54041..106d50ba2 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp @@ -193,6 +193,9 @@ void testContextAndBases() { check(AgcDriver::Pm4::PartialGroupThreads({0, 1, 1}, {0x00100040, 0x00010001, 0x00010001}) == Threads{}, "an empty dispatch reported partial threads"); expectFailure([] { static_cast(AgcDriver::Pm4::PartialGroupThreads({1, 1, 1}, {0x00000040, 0x00010001, 0x00010001})); }, "partial group size"); expectFailure([] { static_cast(AgcDriver::Pm4::PartialGroupThreads({1, 1, 1}, {0x00410040, 0x00010001, 0x00010001})); }, "partial group size"); + expectFailure([] { static_cast(AgcDriver::Pm4::PartialGroupThreads({1, 0, 1}, {0x00100040, 0x00010001, 0x00000001})); }, "partial group size"); + expectFailure([] { static_cast(AgcDriver::Pm4::PartialGroupThreads({0x10002, 1, 1}, {0x0001ffff, 0x00010001, 0x00010001})); }, "exceeds 32 bits"); + expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x15, {1, 1, 1, 0x2063}, 2), 0); }, "dispatch modifiers"); expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x15, {1, 1, 1, 0x41}, 4), 0); }, "header flags"); expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x15, {1, 1, 1, 0x49}), 0); }, "dispatch modifiers"); execute(state, makePacket(0x13, {32})); diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/PatternBufferFill.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/PatternBufferFill.cpp index 9e0872423..f56fbe9c0 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/PatternBufferFill.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/PatternBufferFill.cpp @@ -167,6 +167,24 @@ void ExecuteTests() { } } +void PartialGroupTests() { + for (const auto wave : {32u, 64u}) { + Output.fill(0xcafebabeu); + Pattern[0] = 0x5a5a5a5au; + Control = {256, 0, 0, 0}; + auto words = Commands(UserData(Output.data(), 256), 128, wave, false, false); + words[2] = 0x00100040u; + words[3] = 0x00010001u; + words[4] = 0x00010001u; + words.back() |= 0x2u; + Submit(words); + for (std::size_t i = 0; i < Output.size(); ++i) { + const auto expected = i < 80 ? 0x5a5a5a5au : 0xcafebabeu; + Require(Output[i] == expected, "partial last thread group fill mismatch at " + std::to_string(i) + " wave " + std::to_string(wave)); + } + } +} + void Benchmark() { constexpr std::size_t bytes = 64u << 20u; std::vector storage(bytes / 4 + 4); @@ -196,7 +214,10 @@ int main(int argc, char** argv) { if (!device) return VulkanTestSkipped; } if (argc == 2 && std::string_view(argv[1]) == "--benchmark") Benchmark(); - else ExecuteTests(); + else { + ExecuteTests(); + PartialGroupTests(); + } AgcDriverShutdown_nid_postfix(); std::puts("pattern buffer fill tests passed"); return 0; From a585ec1d62579c34a79f5bce4d95ccab1c77ca45 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 05:05:58 +0200 Subject: [PATCH 329/460] test(agc): resolve the register COPY_DATA from an aligned address 0xa00a is not 4-byte aligned, so ResolveStore rejected it before the register guard. --- core/libs/prx/libSceAgcDriver/tests/Pm4.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp index 106d50ba2..d2cfaa5a0 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Pm4.cpp @@ -384,7 +384,7 @@ void testCopies() { check(state.context.at(0x0a) == 0x55 && state.context.at(0x0b) == 0x3f800000, "COPY_DATA to two context registers failed"); execute(state, makePacket(0x40, {0x1, low(&clearValues[1]), high(&clearValues[1]), 0xa323, 0})); check(state.context.at(0x323) == 0x3f800000 && !state.context.contains(0x324), "COPY_DATA to one context register failed"); - check(!AgcDriver::Pm4::ResolveStore(makePacket(0x40, {0x1, low(clearValues.data()), high(clearValues.data()), 0xa00a, 0}), state, 64).has_value(), "COPY_DATA to a register resolved as a memory store"); + check(!AgcDriver::Pm4::ResolveStore(makePacket(0x40, {0x1, low(clearValues.data()), high(clearValues.data()), 0xa00c, 0}), state, 64).has_value(), "COPY_DATA to a register resolved as a memory store"); expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x40, {0x5, 7, 0, 0xa00a, 0}), 0); }, "to a register from a register, GDS, immediate"); expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x40, {0x1, low(clearValues.data()), high(clearValues.data()), 0x2e40, 0}), 0); }, "outside the context registers"); expectFailure([&] { AgcDriver::Pm4::Validate(makePacket(0x40, {0x10001, low(clearValues.data()), high(clearValues.data()), 0xa3ff, 0}), 0); }, "outside the context registers"); From 24c4dbae1e71851219f286b1aa407caa2f3d525a Mon Sep 17 00:00:00 2001 From: ThatsLeo Date: Sat, 10 Oct 2026 19:23:42 +0200 Subject: [PATCH 330/460] fix(shader): keep f64 denormals where the device can preserve them Shaders with f64 instructions now declare DenormPreserve for 64-bit floats when the device reports shaderDenormPreserveFloat64. Without it Mesa ANV flushed the subnormal result of v_fma_f64 to zero. (cherry picked from commit 7df6c709de994cea0019707d45b6e7b96698fad1) --- .../prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp | 4 ++++ core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp | 1 + core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp | 9 ++++++--- .../libSceAgcDriver/tests/execution/F64FmaSubnormals.cpp | 4 ++++ core/shader/recompiler/CacheKey.hpp | 1 + .../recompiler/ControlFlow/src/RequestSerializer.cpp | 6 ++++-- core/shader/recompiler/Recompiler.cpp | 1 + core/shader/recompiler/Recompiler.hpp | 1 + .../SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp | 1 + .../include/SpirvBackend/SpirvEmitterState.hpp | 1 + core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp | 1 + .../SpirvBackend/src/SpirvMemory/SpirvModuleSetup.cpp | 4 ++++ docs/dev/TechnicalDebt.md | 2 +- 13 files changed, 30 insertions(+), 6 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index 77913ce42..f67c2f87f 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -219,6 +219,7 @@ struct VulkanDevice::State { bool sampleRateShading = false; bool shaderClock = false; bool narrowSubgroupClock = false; + bool float64DenormPreserve = false; // VK_EXT_descriptor_indexing with non-uniform image array indexing (bindless image tables in // graphics stages, and compute workgroups wider than a wave). bool descriptorIndexing = false; @@ -1078,6 +1079,8 @@ VulkanDevice::VulkanDevice(const PresentationWindow* window) : state(std::make_u enabled.shaderInt64 = VK_TRUE; enabled.shaderFloat64 = available.shaderFloat64 && floatControls.shaderSignedZeroInfNanPreserveFloat64; if (enabled.shaderFloat64) state->capabilities.push_back(spv::CapabilityFloat64); + state->float64DenormPreserve = enabled.shaderFloat64 && floatControls.shaderDenormPreserveFloat64 == VK_TRUE; + if (state->float64DenormPreserve && std::find(state->capabilities.begin(), state->capabilities.end(), static_cast(spv::CapabilityDenormPreserve)) == state->capabilities.end()) state->capabilities.push_back(spv::CapabilityDenormPreserve); enabled.vertexPipelineStoresAndAtomics = VK_TRUE; enabled.fragmentStoresAndAtomics = VK_TRUE; enabled.tessellationShader = available.tessellationShader; @@ -2646,6 +2649,7 @@ ShaderRecompiler::SpirvTarget VulkanDevice::buildTarget() const { target.fragmentShaderBarycentricEnabled = state->fragmentShaderBarycentric; target.nonConstantImageOffsets = state->maintenance8; target.narrowSubgroupClock = state->narrowSubgroupClock; + target.float64DenormPreserve = state->float64DenormPreserve; target.fixedPushSlots = state->graphicsPipelineLibrary; target.srgbDecodeFormats = state->srgbDecodeFormats; if (state->meshShader) { diff --git a/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp b/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp index 0aeb9425e..7b31dafe1 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp @@ -397,6 +397,7 @@ void verifyKeySensitivity() { changes("non-constant texel offsets", [](SampleRequest& sample) { sample.request.target.nonConstantImageOffsets = true; }); changes("the sRGB formats decoded in the shader", [](SampleRequest& sample) { sample.request.target.srgbDecodeFormats = 2u; }); changes("the narrow subgroup clock", [](SampleRequest& sample) { sample.request.target.narrowSubgroupClock = true; }); + changes("f64 denormal preservation", [](SampleRequest& sample) { sample.request.target.float64DenormPreserve = true; }); changes("the workgroup size limit", [](SampleRequest& sample) { sample.request.target.maxWorkgroupSize[2] = 128; }); changes("the invocation limit", [](SampleRequest& sample) { sample.request.target.maxWorkgroupInvocations = 512; }); changes("the shared memory limit", [](SampleRequest& sample) { sample.request.target.maxWorkgroupSharedMemoryBytes = 32768; }); diff --git a/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp b/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp index c1c8977ac..29c4aeae7 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp @@ -1373,12 +1373,12 @@ void verifyPixelRequestSerialization() { minimal.context.waveSize = 64; minimal.context.pixel = ShaderPixelStageInfo{}; const auto encoded = serializer.Serialize(minimal); - require(requestPrefix(encoded, 8u) == "NVNQQQ8AAAA=", "new requests did not use serialization version 15"); + require(requestPrefix(encoded, 8u) == "NVNQQRAAAAA=", "new requests did not use serialization version 16"); constexpr std::size_t mappingOffset = 8u + 37u + 18u + 163u; for (std::size_t bytes = 0; bytes < 17u; ++bytes) { expectFailure([&] { static_cast(serializer.Deserialize(requestPrefix(encoded, mappingOffset + bytes))); }, "truncated data", "a truncated pixel mapping, packing or dual-source flag was accepted"); } - for (const auto unsupported : {"NVNQQQAAAAA=", "NVNQQRAAAAA="}) { + for (const auto unsupported : {"NVNQQQAAAAA=", "NVNQQREAAAA="}) { expectFailure([&] { static_cast(serializer.Deserialize(unsupported)); }, "serialization version", "an unsupported request version was accepted"); } } @@ -1472,7 +1472,7 @@ void verifyLegacyPixelRequests() { require(!pixel.orderedPixelShader, "a legacy request became a primitive-ordered pixel shader"); require(request.shader.stage == ShaderStage::Fragment && request.shader.code.size() == 1u && request.shader.code[0] == 0xbf810000u && !request.context.vertex.has_value() && request.context.memory.empty(), "legacy guest context was misaligned"); require(request.target.vulkanVersion == 0x00401000u && request.target.spirvVersion == 0x00010300u && request.target.subgroupSize == 64u && request.layout.firstBinding == 11u && request.layout.pushConstantSizeBytes == 128u, "legacy target or binding layout was misaligned"); - require(request.useCache == (version == 1u) && request.target.nonConstantImageOffsets == (version >= 6u) && request.target.srgbDecodeFormats == 0u && !request.target.narrowSubgroupClock, "legacy request trailer was misread"); + require(request.useCache == (version == 1u) && request.target.nonConstantImageOffsets == (version >= 6u) && request.target.srgbDecodeFormats == 0u && !request.target.narrowSubgroupClock && !request.target.float64DenormPreserve, "legacy request trailer was misread"); const auto upgraded = serializer.Deserialize(serializer.Serialize(request)); require(upgraded.request.context.pixel->targetExportMapping == pixel.targetExportMapping && RecompileCacheKey::ContextHash(upgraded.request) == RecompileCacheKey::ContextHash(request), "upgrading a legacy capture changed its pixel mapping"); } @@ -2706,6 +2706,9 @@ int main(int argc, char** argv) { auto narrowClock = uncached; narrowClock.target.narrowSubgroupClock = true; require(RequestSerializer{}.Deserialize(RequestSerializer{}.Serialize(narrowClock)).request.target.narrowSubgroupClock, "the narrow subgroup clock was lost in serialization"); + auto float64Denormals = uncached; + float64Denormals.target.float64DenormPreserve = true; + require(RequestSerializer{}.Deserialize(RequestSerializer{}.Serialize(float64Denormals)).request.target.float64DenormPreserve, "f64 denormal preservation was lost in serialization"); auto changedLayout = request; changedLayout.layout.pushConstantSizeBytes = 64; require(!Recompile(changedLayout).cacheHit, "binding layout change reused an incompatible variant"); diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/F64FmaSubnormals.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/F64FmaSubnormals.cpp index b77fe2b37..d2f91d35c 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/F64FmaSubnormals.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/F64FmaSubnormals.cpp @@ -144,6 +144,10 @@ int main() { try { const auto device = OpenVulkanTestDevice(); if (!device) return VulkanTestSkipped; + if (!device->Target().float64DenormPreserve) { + std::puts("skipped, the device has no shaderDenormPreserveFloat64"); + return VulkanTestSkipped; + } Run(*device, Code, Rows); Check("v_fma_f64", Rows); Run(*device, FmasCode, FmasRows); diff --git a/core/shader/recompiler/CacheKey.hpp b/core/shader/recompiler/CacheKey.hpp index 4207ad6ab..d1cc7b042 100644 --- a/core/shader/recompiler/CacheKey.hpp +++ b/core/shader/recompiler/CacheKey.hpp @@ -238,6 +238,7 @@ private: append(key, value.nonConstantImageOffsets); append(key, value.srgbDecodeFormats); append(key, value.narrowSubgroupClock); + append(key, value.float64DenormPreserve); } }; diff --git a/core/shader/recompiler/ControlFlow/src/RequestSerializer.cpp b/core/shader/recompiler/ControlFlow/src/RequestSerializer.cpp index e5b8a2f7c..6049290be 100644 --- a/core/shader/recompiler/ControlFlow/src/RequestSerializer.cpp +++ b/core/shader/recompiler/ControlFlow/src/RequestSerializer.cpp @@ -732,7 +732,7 @@ std::string RequestSerializer::Serialize(const RecompileRequest& request) const std::string buffer; Writer writer(buffer); writer.WriteU32(0x41505335u); - writer.WriteU32(15u); + writer.WriteU32(16u); writeShaderBinary(writer, request.shader); writeGuestContext(writer, request.context); writeSpirvTarget(writer, request.target); @@ -751,6 +751,7 @@ std::string RequestSerializer::Serialize(const RecompileRequest& request) const writer.WriteU32(request.target.srgbDecodeFormats); if (request.context.compute.has_value()) writer.WriteU32(request.context.compute->scratchDwords); writer.WriteBool(request.target.narrowSubgroupClock); + writer.WriteBool(request.target.float64DenormPreserve); return base64Encode(buffer); } @@ -759,7 +760,7 @@ DeserializedRequest RequestSerializer::Deserialize(std::string_view text) const Reader reader(decoded); if (reader.ReadU32() != 0x41505335u) throw std::runtime_error("invalid recompile request signature"); const auto version = reader.ReadU32(); - if (version < 1u || version > 15u) throw std::runtime_error("unsupported recompile request serialization version"); + if (version < 1u || version > 16u) throw std::runtime_error("unsupported recompile request serialization version"); DeserializedRequest result{}; result.request.shader = readShaderBinary(reader, result.shaderCode, result.shaderHeader); result.request.context = readGuestContext(reader, result, version); @@ -779,6 +780,7 @@ DeserializedRequest RequestSerializer::Deserialize(std::string_view text) const if (version >= 9u) result.request.target.srgbDecodeFormats = reader.ReadU32(); if (version >= 10u && result.request.context.compute.has_value()) result.request.context.compute->scratchDwords = reader.ReadU32(); if (version >= 12u) result.request.target.narrowSubgroupClock = reader.ReadBool(); + if (version >= 16u) result.request.target.float64DenormPreserve = reader.ReadBool(); return result; } diff --git a/core/shader/recompiler/Recompiler.cpp b/core/shader/recompiler/Recompiler.cpp index bbaece613..492b6bff1 100644 --- a/core/shader/recompiler/Recompiler.cpp +++ b/core/shader/recompiler/Recompiler.cpp @@ -393,6 +393,7 @@ CompiledVariant compileVariant(const RecompileRequest& request, IrProgram progra targetOptions.supportedExtensions = request.target.supportedExtensions; targetOptions.nonConstantImageOffsets = request.target.nonConstantImageOffsets; targetOptions.narrowSubgroupClock = request.target.narrowSubgroupClock; + targetOptions.float64DenormPreserve = request.target.float64DenormPreserve; constexpr SpirvEmitter spirvEmitter; CompiledShaderArtifact result; diff --git a/core/shader/recompiler/Recompiler.hpp b/core/shader/recompiler/Recompiler.hpp index 5688af8e8..ab00913d3 100644 --- a/core/shader/recompiler/Recompiler.hpp +++ b/core/shader/recompiler/Recompiler.hpp @@ -226,6 +226,7 @@ struct SpirvTarget { std::uint32_t srgbDecodeFormats = 0; bool narrowSubgroupClock = false; bool fixedPushSlots = false; + bool float64DenormPreserve = false; }; struct BindingLayout { diff --git a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp index d221ffe2e..0d18fb6e5 100644 --- a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp +++ b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitter.hpp @@ -20,6 +20,7 @@ struct SpirvTargetOptions { std::span supportedExtensions; bool nonConstantImageOffsets = false; bool narrowSubgroupClock = false; + bool float64DenormPreserve = false; }; class SpirvEmitter { diff --git a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp index a89724a0a..d6dde888f 100644 --- a/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp +++ b/core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp @@ -89,6 +89,7 @@ struct SpirvEmitterState { std::uint32_t hostSubgroupSize = 0; bool splitSubgroup = false; bool narrowSubgroupClock = false; + bool float64DenormPreserve = false; std::unordered_set sharedLaneValues; std::uint32_t laneHalf = 0; // The target's SPIR-V version and what the device accepts, for capabilities an emitter adds diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp index b663800a2..8a0951153 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp @@ -249,6 +249,7 @@ std::vector SpirvEmitter::Emit(const IrProgram& program, const Sh state.supportedExtensions = target.supportedExtensions; state.nonConstantImageOffsets = target.nonConstantImageOffsets; state.narrowSubgroupClock = target.narrowSubgroupClock; + state.float64DenormPreserve = target.float64DenormPreserve; state.hostSubgroupSize = target.subgroupSize; state.splitSubgroup = program.WaveSize() == 32u && target.subgroupSize > 32u; if (state.splitSubgroup && (state.requirements.subgroupBallot || state.requirements.subgroupShuffle)) state.requirements.subgroupLocalInvocationId = true; diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvModuleSetup.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvModuleSetup.cpp index b8d68e795..9c3faaa61 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvModuleSetup.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvModuleSetup.cpp @@ -147,6 +147,10 @@ void DefineModule(SpirvEmitterState& state) { } state.module.EmitCapability(spv::CapabilityFloat64); state.module.AddExecutionMode(state.mainFunc, spv::ExecutionModeSignedZeroInfNanPreserve, 64u); + if (state.float64DenormPreserve) { + state.module.EmitCapability(spv::CapabilityDenormPreserve); + state.module.AddExecutionMode(state.mainFunc, spv::ExecutionModeDenormPreserve, 64u); + } } if (const auto* workgroup = ShaderWorkgroupInput(state)) { const std::uint32_t derivativeDefault = state.requirements.computeDerivatives ? 2u : 1u; diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 3d6495951..d2793eb4c 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -398,7 +398,7 @@ Throughout the project, every function at every stage either **does exactly what - [Shader recompilation](../../core/shader/recompiler/Recompiler.cpp) currently occurs right before it was transferred to Vulkan with caching, but should be moved to the [relinker](../../core/relinker/main.cpp) stage. For this purpose, [shader/recompiler](../../core/shader/recompiler) was written completely independently from [libs/prx](../../core/libs/prx). - [v_interp_p1ll_f16, v_interp_p1lv_f16 and v_interp_p2_f16](../../core/shader/recompiler/Translation/src/AttributeInstructions.cpp) read the 16-bit halves of each vertex's parameter through `VK_KHR_fragment_shader_barycentric` and throw without it, as fixed-function interpolation cannot split a 32-bit attribute. P10 and P20 are the vertex differences rounded to f16 to nearest even, which is not confirmed on hardware. v_interp_p1ll_f16 and v_interp_p1lv_f16 round the exact `P10 * I + P0` once to f32 and v_interp_p2_f16 rounds the exact `P20 * J + S2` once to f16, as measured on RDNA2 (#697); the exact sums are computed in f64, so these instructions need the Float64 capability. They follow the shader's float mode, measured on RDNA2 (gfx1035) in all eight combinations of `IEEE_MODE` and kept/flushed f32 and f16 denormals over 768 special and 1,024 random rows. With IEEE=1 a NaN operand is returned quieted; with IEEE=0 it is returned unquieted, except that v_interp_p2_f16's f16 result is always quiet. Flushed f32 denormals flush I/J, v_interp_p2_f16's f32 S2 and the f32 result; flushed f16 denormals flush P0, P10/P20, v_interp_p1lv_f16's f16 S2 and the f16 result. Denormal modes 1 and 2 (flush only inputs or only outputs) and rounding other than to nearest even are not measured and throw. An unknown mode follows IEEE=0 with denormals kept. Lavapipe has no barycentrics, so the draw test is skipped in CI; `agc_driver_interpolation_modes` checks the arithmetic in every mode against the hardware rows on any device with Float64. - Without `VK_KHR_fragment_shader_barycentric`, a pixel shader whose I/J VGPRs [only feed `v_interp_p1_f32`/`v_interp_p2_f32` of their own pair](../../core/shader/recompiler/Optimization/src/HostInterpolationChecker.cpp) reads the attribute Vulkan interpolated. Any other read of them or of a raw vertex parameter takes the weights and the three vertex values from a [generated geometry stage](../../core/shader/recompiler/SpirvBackend/src/BarycentricGeometryShader.cpp), which needs `geometryShader` and a vertex shader before it: tessellation, mesh and rect-list pipelines, primitives other than triangles, more than 32 fragment input locations and interfaces beyond the device's geometry limits throw. -- [v_fma_f64 and v_div_fmas_f64](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterFloat64.cpp) with a subnormal operand or result round once, as IEEE 754 fusedMultiplyAdd does with f64 denormals kept, measured on RDNA2 (gfx1035) at IEEE=0 and IEEE=1, `v_div_fmas_f64` with VCC clear and set. The f16/f64 denormal field of `FLOAT_MODE` (bits 7:6) is not read: when it flushes, the hardware flushes the operands and the result of `v_fma_f64` (tininess before rounding) and only the result of `v_div_fmas_f64`, while these keep them. +- [v_fma_f64 and v_div_fmas_f64](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterFloat64.cpp) with a subnormal operand or result round once, as IEEE 754 fusedMultiplyAdd does with f64 denormals kept, measured on RDNA2 (gfx1035) at IEEE=0 and IEEE=1, `v_div_fmas_f64` with VCC clear and set. The f16/f64 denormal field of `FLOAT_MODE` (bits 7:6) is not read: when it flushes, the hardware flushes the operands and the result of `v_fma_f64` (tininess before rounding) and only the result of `v_div_fmas_f64`, while these keep them. Shaders with f64 instructions declare `DenormPreserve` for 64-bit floats only where the device reports `shaderDenormPreserveFloat64`; elsewhere (lavapipe) the driver may flush f64 subnormals, and `agc_driver_f64_fma_subnormals` is skipped. - [SINT storage images](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) of 8- and 16-bit components saturate each stored 32-bit value to the component range, as adriadam10 measured for image stores on RDNA2 (gfx1035) in #944 (448 rows match); the PS5's gfx1013 is not measured. - [`StoreTexel`](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) writes channel 1 of a two-channel format from the data slot whose DST_SEL is Y, for SINT and UINT alike. With DST_SEL X,Y,Z,W, gfx1035 writes it from the W slot, as adriadam10 measured in #944. - [COND_EXEC](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Packets/PacketExecution.cpp) reads its condition after the earlier packets of its queue complete. Without PFP_SYNC_ME the console's graphics PFP reads it ahead of the ME, so a title that races the read sees a later value here. From ebaa687d36d361b6ff720fe2b7483555372a32b0 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sun, 11 Oct 2026 01:32:50 +0000 Subject: [PATCH 331/460] fix(shader): respect float control independence --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 5 ++ .../Execution/src/VulkanDevice.cpp | 3 +- .../tests/execution/F64FmaSubnormals.cpp | 68 ++++++++++++++++++- docs/dev/TechnicalDebt.md | 2 +- 4 files changed, 75 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 6d03e7feb..561f9647a 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -967,6 +967,11 @@ if(BUILD_TESTING) "PATH=path_list_prepend:$") endif() endforeach() + add_test(NAME agc_driver_f64_denorm_preserve_spirv COMMAND agc_driver_f64_fma_subnormals_tests --verify-spv-denorm-mode) + set_tests_properties(agc_driver_f64_denorm_preserve_spirv PROPERTIES ENVIRONMENT "ANYPS5_NO_SHADER_CACHE=1" WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}") + if(WIN32) + set_tests_properties(agc_driver_f64_denorm_preserve_spirv PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") + endif() add_test(NAME agc_structurizer_intersection_off COMMAND agc_structurizer_tests) set_tests_properties(agc_structurizer_intersection_off PROPERTIES ENVIRONMENT "ANYPS5_NO_SHADER_CACHE=1;APS5_NO_PERF_DOM_INTERSECTION=1" WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}") if(WIN32) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index f67c2f87f..47050e545 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -1079,7 +1079,8 @@ VulkanDevice::VulkanDevice(const PresentationWindow* window) : state(std::make_u enabled.shaderInt64 = VK_TRUE; enabled.shaderFloat64 = available.shaderFloat64 && floatControls.shaderSignedZeroInfNanPreserveFloat64; if (enabled.shaderFloat64) state->capabilities.push_back(spv::CapabilityFloat64); - state->float64DenormPreserve = enabled.shaderFloat64 && floatControls.shaderDenormPreserveFloat64 == VK_TRUE; + state->float64DenormPreserve = enabled.shaderFloat64 && floatControls.shaderDenormPreserveFloat64 == VK_TRUE && + floatControls.denormBehaviorIndependence == VK_SHADER_FLOAT_CONTROLS_INDEPENDENCE_ALL; if (state->float64DenormPreserve && std::find(state->capabilities.begin(), state->capabilities.end(), static_cast(spv::CapabilityDenormPreserve)) == state->capabilities.end()) state->capabilities.push_back(spv::CapabilityDenormPreserve); enabled.vertexPipelineStoresAndAtomics = VK_TRUE; enabled.fragmentStoresAndAtomics = VK_TRUE; diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/F64FmaSubnormals.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/F64FmaSubnormals.cpp index d2f91d35c..aaccc5329 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/F64FmaSubnormals.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/F64FmaSubnormals.cpp @@ -2,12 +2,15 @@ #include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" #include "Recompiler.hpp" #include "VulkanTestDevice.hpp" +#include #include #include #include #include #include +#include #include +#include #include namespace { @@ -94,6 +97,64 @@ std::array BufferDescriptor(const void* data, std::uint32_t by return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; } +std::vector Recompile(bool float64DenormPreserve) { + constexpr std::array capabilities{ + spv::CapabilityShader, spv::CapabilityGroupNonUniform, spv::CapabilityGroupNonUniformBallot, + spv::CapabilityInt64, spv::CapabilityFloat64, spv::CapabilityPhysicalStorageBufferAddresses, + spv::CapabilityStorageBuffer8BitAccess, spv::CapabilityDenormPreserve, spv::CapabilitySignedZeroInfNanPreserve, + }; + constexpr std::array extensions{ + "SPV_KHR_physical_storage_buffer", "SPV_KHR_8bit_storage", "SPV_KHR_float_controls", + "SPV_KHR_storage_buffer_storage_class", + }; + std::array input{}; + std::array output{}; + std::array userData{}; + const auto inputDescriptor = BufferDescriptor(input.data(), static_cast(input.size() * sizeof(input[0]))); + const auto outputDescriptor = BufferDescriptor(output.data(), static_cast(output.size() * sizeof(output[0]))); + std::copy(inputDescriptor.begin(), inputDescriptor.end(), userData.begin()); + std::copy(outputDescriptor.begin(), outputDescriptor.end(), userData.begin() + 4); + const std::span code(Code); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + ShaderRecompiler::RecompileRequest request{}; + request.shader = {ShaderStage::Compute, 0x20000u, code, 0, {}}; + request.context.waveSize = 32; + request.context.userDataBaseRegister = 0; + request.context.userData = userData; + request.context.compute = ShaderRecompiler::ShaderComputeStageInfo{{Threads, 1, 1}, 0u, {false, false, false}, false, 1}; + request.context.memory = memory; + request.target.vulkanVersion = 0x00403000u; + request.target.spirvVersion = 0x00010600u; + request.target.subgroupSize = 32; + request.target.bdaAbiVersion = 1; + request.target.supportedCapabilities = capabilities; + request.target.supportedExtensions = extensions; + request.target.maxWorkgroupSize = {1024, 1024, 64}; + request.target.maxWorkgroupInvocations = 1024; + request.target.maxWorkgroupSharedMemoryBytes = 49152; + request.target.float64DenormPreserve = float64DenormPreserve; + request.layout = {0, 0, 0, 128}; + request.useCache = false; + return ShaderRecompiler::Recompile(request).spirv; +} + +bool HasDenormPreserve64(std::span words) { + for (std::size_t cursor = 5; cursor < words.size();) { + const auto count = words[cursor] >> 16u; + Require(count != 0 && cursor + count <= words.size(), "malformed SPIR-V while scanning denorm modes"); + if ((words[cursor] & 0xffffu) == spv::OpExecutionMode && count == 4 && words[cursor + 2] == spv::ExecutionModeDenormPreserve && words[cursor + 3] == 64u) return true; + cursor += count; + } + return false; +} + +void VerifySpirvDenormMode() { + const auto enabled = Recompile(true); + const auto disabled = Recompile(false); + Require(HasDenormPreserve64(enabled), "float64DenormPreserve=true did not emit DenormPreserve 64"); + Require(!HasDenormPreserve64(disabled), "float64DenormPreserve=false emitted DenormPreserve 64"); +} + std::string Hex(std::uint64_t value) { char text[24]; std::snprintf(text, sizeof(text), "0x%016llx", static_cast(value)); @@ -140,8 +201,13 @@ void Check(const char* name, const std::uint64_t (&rows)[Count][4]) { } -int main() { +int main(int argc, char** argv) { try { + if (argc == 2 && std::string_view(argv[1]) == "--verify-spv-denorm-mode") { + VerifySpirvDenormMode(); + std::puts("f64 DenormPreserve execution mode follows the target capability"); + return 0; + } const auto device = OpenVulkanTestDevice(); if (!device) return VulkanTestSkipped; if (!device->Target().float64DenormPreserve) { diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index d2793eb4c..7b3182c5b 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -398,7 +398,7 @@ Throughout the project, every function at every stage either **does exactly what - [Shader recompilation](../../core/shader/recompiler/Recompiler.cpp) currently occurs right before it was transferred to Vulkan with caching, but should be moved to the [relinker](../../core/relinker/main.cpp) stage. For this purpose, [shader/recompiler](../../core/shader/recompiler) was written completely independently from [libs/prx](../../core/libs/prx). - [v_interp_p1ll_f16, v_interp_p1lv_f16 and v_interp_p2_f16](../../core/shader/recompiler/Translation/src/AttributeInstructions.cpp) read the 16-bit halves of each vertex's parameter through `VK_KHR_fragment_shader_barycentric` and throw without it, as fixed-function interpolation cannot split a 32-bit attribute. P10 and P20 are the vertex differences rounded to f16 to nearest even, which is not confirmed on hardware. v_interp_p1ll_f16 and v_interp_p1lv_f16 round the exact `P10 * I + P0` once to f32 and v_interp_p2_f16 rounds the exact `P20 * J + S2` once to f16, as measured on RDNA2 (#697); the exact sums are computed in f64, so these instructions need the Float64 capability. They follow the shader's float mode, measured on RDNA2 (gfx1035) in all eight combinations of `IEEE_MODE` and kept/flushed f32 and f16 denormals over 768 special and 1,024 random rows. With IEEE=1 a NaN operand is returned quieted; with IEEE=0 it is returned unquieted, except that v_interp_p2_f16's f16 result is always quiet. Flushed f32 denormals flush I/J, v_interp_p2_f16's f32 S2 and the f32 result; flushed f16 denormals flush P0, P10/P20, v_interp_p1lv_f16's f16 S2 and the f16 result. Denormal modes 1 and 2 (flush only inputs or only outputs) and rounding other than to nearest even are not measured and throw. An unknown mode follows IEEE=0 with denormals kept. Lavapipe has no barycentrics, so the draw test is skipped in CI; `agc_driver_interpolation_modes` checks the arithmetic in every mode against the hardware rows on any device with Float64. - Without `VK_KHR_fragment_shader_barycentric`, a pixel shader whose I/J VGPRs [only feed `v_interp_p1_f32`/`v_interp_p2_f32` of their own pair](../../core/shader/recompiler/Optimization/src/HostInterpolationChecker.cpp) reads the attribute Vulkan interpolated. Any other read of them or of a raw vertex parameter takes the weights and the three vertex values from a [generated geometry stage](../../core/shader/recompiler/SpirvBackend/src/BarycentricGeometryShader.cpp), which needs `geometryShader` and a vertex shader before it: tessellation, mesh and rect-list pipelines, primitives other than triangles, more than 32 fragment input locations and interfaces beyond the device's geometry limits throw. -- [v_fma_f64 and v_div_fmas_f64](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterFloat64.cpp) with a subnormal operand or result round once, as IEEE 754 fusedMultiplyAdd does with f64 denormals kept, measured on RDNA2 (gfx1035) at IEEE=0 and IEEE=1, `v_div_fmas_f64` with VCC clear and set. The f16/f64 denormal field of `FLOAT_MODE` (bits 7:6) is not read: when it flushes, the hardware flushes the operands and the result of `v_fma_f64` (tininess before rounding) and only the result of `v_div_fmas_f64`, while these keep them. Shaders with f64 instructions declare `DenormPreserve` for 64-bit floats only where the device reports `shaderDenormPreserveFloat64`; elsewhere (lavapipe) the driver may flush f64 subnormals, and `agc_driver_f64_fma_subnormals` is skipped. +- [v_fma_f64 and v_div_fmas_f64](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterFloat64.cpp) with a subnormal operand or result round once, as IEEE 754 fusedMultiplyAdd does with f64 denormals kept, measured on RDNA2 (gfx1035) at IEEE=0 and IEEE=1, `v_div_fmas_f64` with VCC clear and set. The f16/f64 denormal field of `FLOAT_MODE` (bits 7:6) is not read: when it flushes, the hardware flushes the operands and the result of `v_fma_f64` (tininess before rounding) and only the result of `v_div_fmas_f64`, while these keep them. Shaders with f64 instructions declare `DenormPreserve` for 64-bit floats only when `shaderDenormPreserveFloat64` is true and Vulkan reports `denormBehaviorIndependence` as `ALL`; other independence modes are not handled by the current backend because they require coupled execution modes. See [VkShaderFloatControlsIndependence](https://docs.vulkan.org/refpages/latest/refpages/source/VkShaderFloatControlsIndependence.html). Elsewhere (including lavapipe), the driver may flush f64 subnormals, and `agc_driver_f64_fma_subnormals` is skipped. - [SINT storage images](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) of 8- and 16-bit components saturate each stored 32-bit value to the component range, as adriadam10 measured for image stores on RDNA2 (gfx1035) in #944 (448 rows match); the PS5's gfx1013 is not measured. - [`StoreTexel`](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) writes channel 1 of a two-channel format from the data slot whose DST_SEL is Y, for SINT and UINT alike. With DST_SEL X,Y,Z,W, gfx1035 writes it from the W slot, as adriadam10 measured in #944. - [COND_EXEC](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Packets/PacketExecution.cpp) reads its condition after the earlier packets of its queue complete. Without PFP_SYNC_ME the console's graphics PFP reads it ahead of the ME, so a title that races the read sees a later value here. From 5e00b269ca2b42493cd05581391dc580a468be63 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sun, 11 Oct 2026 03:09:34 +0000 Subject: [PATCH 332/460] test(shader): run f64 denorm cases without preserve mode --- .../tests/execution/F64FmaSubnormals.cpp | 29 +++++++++++++++---- 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/F64FmaSubnormals.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/F64FmaSubnormals.cpp index aaccc5329..cb9823b3a 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/F64FmaSubnormals.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/F64FmaSubnormals.cpp @@ -192,11 +192,15 @@ void Run(AgcDriver::VulkanDevice& device, const std::array } template -void Check(const char* name, const std::uint64_t (&rows)[Count][4]) { +bool Check(const char* name, const std::uint64_t (&rows)[Count][4]) { for (std::uint32_t tid = 0; tid < Count; ++tid) { const std::uint64_t actual = Output[tid * Results] | (static_cast(Output[tid * Results + 1]) << 32u); - Require(actual == rows[tid][3], std::string(name) + ": lane " + std::to_string(tid) + " (" + Hex(rows[tid][0]) + ", " + Hex(rows[tid][1]) + ", " + Hex(rows[tid][2]) + ") is " + Hex(actual) + ", expected " + Hex(rows[tid][3])); + if (actual != rows[tid][3]) { + std::cerr << name << ": lane " << tid << " (" << Hex(rows[tid][0]) << ", " << Hex(rows[tid][1]) << ", " << Hex(rows[tid][2]) << ") is " << Hex(actual) << ", expected " << Hex(rows[tid][3]) << '\n'; + return false; + } } + return true; } } @@ -210,14 +214,27 @@ int main(int argc, char** argv) { } const auto device = OpenVulkanTestDevice(); if (!device) return VulkanTestSkipped; - if (!device->Target().float64DenormPreserve) { - std::puts("skipped, the device has no shaderDenormPreserveFloat64"); + if (!TargetHasCapability(device->Target(), spv::CapabilityFloat64)) { + std::puts("skipped, the device has no shaderFloat64"); return VulkanTestSkipped; } + const bool canPreserveDenormals = device->Target().float64DenormPreserve; Run(*device, Code, Rows); - Check("v_fma_f64", Rows); + if (!Check("v_fma_f64", Rows)) { + if (!canPreserveDenormals) { + std::puts("skipped, the device flushes f64 subnormals without DenormPreserve support"); + return VulkanTestSkipped; + } + return 1; + } Run(*device, FmasCode, FmasRows); - Check("v_div_fmas_f64 with VCC set", FmasRows); + if (!Check("v_div_fmas_f64 with VCC set", FmasRows)) { + if (!canPreserveDenormals) { + std::puts("skipped, the device flushes f64 subnormals without DenormPreserve support"); + return VulkanTestSkipped; + } + return 1; + } std::puts("v_fma_f64 subnormal tests passed"); return 0; } catch (const std::exception& error) { From ef727569cc100dab1021f67e996c0829cae77da6 Mon Sep 17 00:00:00 2001 From: Get-fix <100056769+Get-fix@users.noreply.github.com> Date: Sat, 10 Oct 2026 20:46:33 +0300 Subject: [PATCH 333/460] fix(relinker): read SCE dynamic tables from PT_SCE_DYNLIBDATA DT_SCE_STRTAB, DT_SCE_SYMTAB, DT_SCE_RELA and DT_SCE_JMPREL are offsets into the PT_SCE_DYNLIBDATA segment, as the code analysis and the guest image reader already read them, but the pipeline read them as file offsets. That only works when the segment starts at file offset 0, as in the test fixtures. A PS4 executable (CUSA00900) keeps the segment at 0x53e9b40, so the relinker read its relocations from the code and failed with "Dynamic string offset is outside DT_STRSZ". Without the segment the values are still read as file offsets. (cherry picked from commit 5dbf89a57e096f2891ae786f44c3f57935af2285) --- .../relinker/src/pipeline/RelinkerPipeline.cpp | 14 ++++++++++++-- .../relinker/tests/test_string_table_bounds.py | 18 +++++++++++++----- docs/dev/TechnicalDebt.md | 1 + 3 files changed, 26 insertions(+), 7 deletions(-) diff --git a/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp b/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp index d631eea28..c862ee74c 100644 --- a/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp +++ b/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp @@ -5,6 +5,7 @@ #include #include #include +#include #include namespace Relinker { @@ -59,6 +60,11 @@ RelinkResult RelinkerPipeline::Relink(const std::vector& sourceElf } } + FileByteOffset sceTableBase = 0; + for (const auto& ph : programHeaders) + if (ph.Type == PT_SCE_DYNLIBDATA) + sceTableBase = ph.Offset; + std::vector dynTags; bool hasDynamicSegment = false; @@ -99,8 +105,12 @@ RelinkResult RelinkerPipeline::Relink(const std::vector& sourceElf }; auto readAsOffset = [&](const std::int64_t osTag, const std::int64_t sysvTag, const char* name) -> FileByteOffset { - if (requireExactlyOneOf(osTag, sysvTag, name)) - return getTagValue(osTag); + if (requireExactlyOneOf(osTag, sysvTag, name)) { + const std::uint64_t value = getTagValue(osTag); + if (value > std::numeric_limits::max() - sceTableBase) + throw RelinkerException(std::string(name) + " is outside the file", value); + return sceTableBase + value; + } return _elfReader->TranslateVirtualAddress(getTagValue(sysvTag)); }; diff --git a/core/relinker/relinker/tests/test_string_table_bounds.py b/core/relinker/relinker/tests/test_string_table_bounds.py index 10669542c..f130b3d7f 100644 --- a/core/relinker/relinker/tests/test_string_table_bounds.py +++ b/core/relinker/relinker/tests/test_string_table_bounds.py @@ -23,20 +23,22 @@ DT_OS_RELASZ = 0x61000031 DT_OS_RELAENT = 0x61000033 -def fixture(os_tags=False, str_size=16, needed_offset=1, symbol_offset=8, table_bytes=None, outside_bytes=b""): +def fixture(os_tags=False, str_size=16, needed_offset=1, symbol_offset=8, table_bytes=None, outside_bytes=b"", + block_offset=0, strtab_value=None): image = bytearray(0x1000) + table_base = block_offset if os_tags else 0 image[:16] = b"\x7fELF\x02\x01\x01" + bytes(9) struct.pack_into("` when one does not fit 15 bytes), not the console's. A Linux point-to-point address without a peer loses `IFF_POINTOPOINT`, and an IPv4 broadcast address the host omits is computed from the netmask. On Windows the flags are only `IFF_UP`, `IFF_RUNNING`, `IFF_LOOPBACK`, `IFF_BROADCAST` and `IFF_MULTICAST`, derived from the adapter: PPP and tunnel adapters get neither `IFF_BROADCAST` nor `IFF_POINTOPOINT` because their peer address is not reported, and an address whose prefix length the host reports as unspecified gets a host mask. - [gethostname](../../core/libs/prx/libc/src/Hostname.cpp) (libc) reports the fixed guest name `AnyPS5` without querying the host machine name. Buffer copying, null queries and `ENAMETOOLONG` follow [FreeBSD gethostname](https://github.com/freebsd/freebsd-src/blob/main/lib/libc/gen/gethostname.c); the console's configurable hostname and its integration with `sysctl` and name resolution are unverified and not implemented. From 16adcd4871b8779bfa695e1fa54cfe122faf4786 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sun, 11 Oct 2026 03:30:09 +0000 Subject: [PATCH 334/460] fix(relinker): validate SCE dynamic data segment --- .../relinker/src/pipeline/RelinkerPipeline.cpp | 14 ++++++++++---- .../relinker/tests/test_string_table_bounds.py | 18 +++++++++++++----- docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 24 insertions(+), 10 deletions(-) diff --git a/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp b/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp index c862ee74c..483eaed3b 100644 --- a/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp +++ b/core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp @@ -61,9 +61,15 @@ RelinkResult RelinkerPipeline::Relink(const std::vector& sourceElf } FileByteOffset sceTableBase = 0; - for (const auto& ph : programHeaders) - if (ph.Type == PT_SCE_DYNLIBDATA) - sceTableBase = ph.Offset; + bool hasSceDynlibData = false; + for (const auto& ph : programHeaders) { + if (ph.Type != PT_SCE_DYNLIBDATA) + continue; + if (hasSceDynlibData) + throw RelinkerException("Multiple PT_SCE_DYNLIBDATA segments", ph.Offset); + hasSceDynlibData = true; + sceTableBase = ph.Offset; + } std::vector dynTags; bool hasDynamicSegment = false; @@ -108,7 +114,7 @@ RelinkResult RelinkerPipeline::Relink(const std::vector& sourceElf if (requireExactlyOneOf(osTag, sysvTag, name)) { const std::uint64_t value = getTagValue(osTag); if (value > std::numeric_limits::max() - sceTableBase) - throw RelinkerException(std::string(name) + " is outside the file", value); + throw RelinkerException(std::string("Adding PT_SCE_DYNLIBDATA base to ") + name + " overflows", value); return sceTableBase + value; } return _elfReader->TranslateVirtualAddress(getTagValue(sysvTag)); diff --git a/core/relinker/relinker/tests/test_string_table_bounds.py b/core/relinker/relinker/tests/test_string_table_bounds.py index f130b3d7f..c5f66e25c 100644 --- a/core/relinker/relinker/tests/test_string_table_bounds.py +++ b/core/relinker/relinker/tests/test_string_table_bounds.py @@ -24,12 +24,12 @@ DT_OS_RELAENT = 0x61000033 def fixture(os_tags=False, str_size=16, needed_offset=1, symbol_offset=8, table_bytes=None, outside_bytes=b"", - block_offset=0, strtab_value=None): + block_offset=0, strtab_value=None, duplicate_sce_dynlibdata=False): image = bytearray(0x1000) table_base = block_offset if os_tags else 0 image[:16] = b"\x7fELF\x02\x01\x01" + bytes(9) struct.pack_into("` when one does not fit 15 bytes), not the console's. A Linux point-to-point address without a peer loses `IFF_POINTOPOINT`, and an IPv4 broadcast address the host omits is computed from the netmask. On Windows the flags are only `IFF_UP`, `IFF_RUNNING`, `IFF_LOOPBACK`, `IFF_BROADCAST` and `IFF_MULTICAST`, derived from the adapter: PPP and tunnel adapters get neither `IFF_BROADCAST` nor `IFF_POINTOPOINT` because their peer address is not reported, and an address whose prefix length the host reports as unspecified gets a host mask. - [gethostname](../../core/libs/prx/libc/src/Hostname.cpp) (libc) reports the fixed guest name `AnyPS5` without querying the host machine name. Buffer copying, null queries and `ENAMETOOLONG` follow [FreeBSD gethostname](https://github.com/freebsd/freebsd-src/blob/main/lib/libc/gen/gethostname.c); the console's configurable hostname and its integration with `sysctl` and name resolution are unverified and not implemented. From 7f9ca519b420c3b92bf120738f611392cee73c9b Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Sun, 11 Oct 2026 12:40:04 +0900 Subject: [PATCH 335/460] fix(tools): accept NIDs starting with a dash in nid_names --nid NIDs use the base64 alphabet with '-', so some start with a dash, such as -nvxBWa0iDs (gethostname). argparse read such a value after --nid as an option and nid_names.py stopped with "unrecognized arguments". Every value after --nid up to the next option of the tool is now passed as --nid=, and --nid collects them with append. Co-Authored-By: Claude Opus 5.5 --- tools/nid_names.py | 20 ++++++++++++++++++-- tools/tests/test_nid_names.py | 18 ++++++++++++++++++ 2 files changed, 36 insertions(+), 2 deletions(-) diff --git a/tools/nid_names.py b/tools/nid_names.py index e4c0c98f0..72a60fe97 100644 --- a/tools/nid_names.py +++ b/tools/nid_names.py @@ -85,16 +85,32 @@ def collect_real_names(): return names +def nid_arguments(arguments): + options = {"--db", "--json", "--nid", "-h", "--help"} + normalized = [] + collecting = False + for argument in arguments: + if argument == "--nid": + collecting = True + continue + if collecting and argument not in options and not argument.startswith("--db="): + normalized.append("--nid=" + argument) + continue + collecting = False + normalized.append(argument) + return normalized + + def main(): parser = argparse.ArgumentParser( description="Suggest real names for Unknown NID stubs") parser.add_argument("--db", default=os.environ.get("ANYPS5_NID_DB"), help="NID database CSV (downloaded to a cache dir by default)") - parser.add_argument("--nid", nargs="*", default=[], + parser.add_argument("--nid", action="append", default=[], help="resolve only these NIDs instead of scanning the tree") parser.add_argument("--json", action="store_true", help="emit machine-readable JSON") - args = parser.parse_args() + args = parser.parse_args(nid_arguments(sys.argv[1:])) db_path = Path(args.db) if args.db else DEFAULT_CACHE db = load_db(db_path) diff --git a/tools/tests/test_nid_names.py b/tools/tests/test_nid_names.py index ca268c741..5f50053e0 100644 --- a/tools/tests/test_nid_names.py +++ b/tools/tests/test_nid_names.py @@ -1,3 +1,4 @@ +import json import tempfile import sys import unittest @@ -67,6 +68,23 @@ class DatabaseCacheTests(unittest.TestCase): self.assertEqual(nid_names.load_db(cache), {"nid1": "sceExisting"}) download.assert_not_called() +class NidArgumentTests(unittest.TestCase): + def test_nids_starting_with_a_dash_are_accepted(self): + self.assertEqual( + nid_names.nid_arguments(["--nid", "-nvxBWa0iDs", "ZjtRqSMJwdw", "--json"]), + ["--nid=-nvxBWa0iDs", "--nid=ZjtRqSMJwdw", "--json"]) + + def test_main_resolves_every_nid_given_after_nid(self): + database = {"-nvxBWa0iDs": "gethostname", "ZjtRqSMJwdw": "sinh"} + arguments = ["nid_names.py", "--json", "--nid", "-nvxBWa0iDs", "ZjtRqSMJwdw"] + with (patch.object(sys, "argv", arguments), + patch.object(nid_names, "load_db", return_value=database), + patch.object(nid_names, "collect_real_names", return_value=set()), + patch("builtins.print") as printed): + self.assertEqual(nid_names.main(), 0) + rows = json.loads(printed.call_args.args[0])["rows"] + self.assertEqual({row["nid"]: row["suggestion"] for row in rows}, database) + if __name__ == "__main__": unittest.main() From 247e3f5eb2e38745c27d00f21891fe8dc11938c8 Mon Sep 17 00:00:00 2001 From: Nahuel Genchi <65604505+NahuelGenchi@users.noreply.github.com> Date: Fri, 9 Oct 2026 19:46:22 -0300 Subject: [PATCH 336/460] perf(agc): take a 2D storage image's results for a one-slice array on the device A 2D surface and a one-slice 2D array over the same memory are separate storage images, since the storage key holds the dimension. pendingAlias only took a pending image of the same dimension, so a refresh of one view wrote the other view's results back and uploaded them again. Sonic Origins Plus (PPSA05324) switches between the two views of the same surfaces about 50 times a frame in its opening animation. pendingAlias now counts a 2D surface and a one-slice 2D array as the same dimension. The rest of the shape test (size, array range, mips, tiling, element size, image layers and depth, tracked units) and the conditions of the device copy (uncompressed keys, a host import, the surface's write stamps) are unchanged. agc_driver_storage_one_slice_alias stores through the 2D view twice and loads through the array view after each store. It checks the texels read through the array, that the 2D image is still pending after the array image's refresh (where the imported block is write-watched), and the guest memory after the final flush. Without the change the pending check fails on lavapipe. Co-Authored-By: Claude Opus 5.5 --- .../libSceAgcDriver/Graphics/src/Texture.cpp | 4 +- .../tests/execution/StorageOneSliceAlias.cpp | 207 ++++++++++++++++++ 2 files changed, 210 insertions(+), 1 deletion(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/StorageOneSliceAlias.cpp diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp index 2e4b692a9..43e3eb315 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp @@ -3628,7 +3628,9 @@ std::shared_ptr StorageTexture::pendingAlias() const { const auto& theirs = candidate->descriptor; // The same texels under another format of the same size: the device copy moves them bit // for bit, as the guest bytes would. - const bool sameShape = mine.width == theirs.width && mine.height == theirs.height && mine.depthOrLastArray == theirs.depthOrLastArray && mine.baseArray == theirs.baseArray && mine.mipCount == theirs.mipCount && mine.tileMode == theirs.tileMode && mine.dimension == theirs.dimension && BytesPerElement(mine.format) == BytesPerElement(theirs.format) && geometry.imageLayers == candidate->geometry.imageLayers && geometry.imageDepth == candidate->geometry.imageDepth && trackedLayers == candidate->trackedLayers; + const auto oneSlice = [](const GuestTextureResource& resource) { return resource.dimension == TextureDimension::k2D || (resource.dimension == TextureDimension::k2DArray && resource.depthOrLastArray == 0); }; + const bool sameDimension = mine.dimension == theirs.dimension || (oneSlice(mine) && oneSlice(theirs)); + const bool sameShape = mine.width == theirs.width && mine.height == theirs.height && mine.depthOrLastArray == theirs.depthOrLastArray && mine.baseArray == theirs.baseArray && mine.mipCount == theirs.mipCount && mine.tileMode == theirs.tileMode && sameDimension && BytesPerElement(mine.format) == BytesPerElement(theirs.format) && geometry.imageLayers == candidate->geometry.imageLayers && geometry.imageDepth == candidate->geometry.imageDepth && trackedLayers == candidate->trackedLayers; if (sameShape && candidate->Cached() && candidate->uploadedKeys == DccKeys::Uncompressed) return candidate; } return nullptr; diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/StorageOneSliceAlias.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/StorageOneSliceAlias.cpp new file mode 100644 index 000000000..ccfa31bf3 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/StorageOneSliceAlias.cpp @@ -0,0 +1,207 @@ +#include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Texture.hpp" +#include "prx/libc/include/GuestAllocations.hpp" +#include "prx/libc/include/GuestArena.hpp" +#include "prx/libc/include/GuestWriteWatch.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#ifdef _WIN32 +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include +#else +#include +#endif +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using AgcDriver::Graphics::StorageTexture; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 32; +constexpr std::uint32_t Side = 128; +constexpr std::uint32_t Format32UInt = 20; +constexpr std::uint32_t TileR64KBX = 0x1b; +constexpr std::uint32_t Type2D = 9; +constexpr std::uint32_t Type2DArray = 13; +constexpr std::size_t SurfaceBytes = 65536; +constexpr std::size_t BlockBytes = 65536; +constexpr std::uint32_t Initial = 0x5a5a5a5au; + +alignas(256) constexpr std::array StoreCode{ + 0x7e020280, 0x7e04020c, 0xf0201108, 0x00010200, 0xbf810000, +}; + +alignas(256) constexpr std::array LoadArrayCode{ + 0x7e020280, 0x7e040280, 0xf0001128, 0x00010300, 0x34080082, 0xbf8c3f70, 0xe0701000, 0x80000304, 0xbf810000, +}; + +alignas(256) std::array Output{}; + +std::uint64_t AddressOf(const void* data) { + return static_cast(reinterpret_cast(data)); +} + +std::string Hex(std::uint32_t value) { + char text[16]; + std::snprintf(text, sizeof(text), "0x%08x", value); + return text; +} + +class GuestBlock { +public: + explicit GuestBlock(bool watched) { +#ifdef _WIN32 + block = static_cast(GuestArena::GuestArenaAllocate_nid_postfix(BlockBytes, 65536)); + if (block != nullptr) GuestArena::GuestArenaCommit_nid_postfix(block, BlockBytes, PAGE_READWRITE, BlockBytes); + static_cast(watched); +#else + constexpr std::uintptr_t alignment = 65536; + void* mapped = mmap(nullptr, BlockBytes + alignment, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (mapped != MAP_FAILED) { + const auto begin = reinterpret_cast(mapped); + const auto aligned = (begin + alignment - 1) & ~(alignment - 1); + if (aligned != begin) munmap(mapped, aligned - begin); + if (aligned + BlockBytes != begin + BlockBytes + alignment) munmap(reinterpret_cast(aligned + BlockBytes), begin + alignment - aligned); + block = reinterpret_cast(aligned); + if (watched) GuestWriteWatch::GuestWriteWatchRegister_nid_postfix(block, BlockBytes); + } +#endif + Require(block != nullptr, "one-slice alias: cannot allocate the guest block"); + Require(!watched || AgcDriver::GuestMemory::Watched(AddressOf(block), BlockBytes), "one-slice alias: the guest block is not write-watched"); + GuestAllocations::Mutation().Add(block, BlockBytes, true, true); + } + + ~GuestBlock() { + GuestAllocations::Mutation().Remove(block); +#ifdef _WIN32 + GuestArena::GuestArenaReset_nid_postfix(block, BlockBytes); + GuestArena::GuestArenaRelease_nid_postfix(block, BlockBytes); +#else + munmap(block, BlockBytes); + GuestWriteWatch::GuestWriteWatchUnregister_nid_postfix(block, BlockBytes); +#endif + } + + GuestBlock(const GuestBlock&) = delete; + GuestBlock& operator=(const GuestBlock&) = delete; + + std::uint8_t* Data() { return block; } + +private: + std::uint8_t* block = nullptr; +}; + +std::array TextureDescriptor(const std::uint8_t* texels, std::uint32_t type) { + const auto address = AddressOf(texels); + return { + static_cast(address >> 8u), + static_cast((address >> 40u) & 0xffu) | (Format32UInt << 20u) | (((Side - 1u) & 3u) << 30u), + ((Side - 1u) >> 2u) | ((Side - 1u) << 14u), + 0xfacu | (TileR64KBX << 20u) | (type << 28u), + 0u, + 0u, + 0u, + 0u, + }; +} + +std::array BufferDescriptor(const void* data, std::uint32_t bytes) { + const auto address = AddressOf(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; +} + +void Dispatch(AgcDriver::VulkanDevice& device, std::span code, const std::vector& userData) { + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {false, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); +} + +void Store(AgcDriver::VulkanDevice& device, const std::uint8_t* texels, std::uint32_t value) { + std::vector userData(16, 0u); + const auto texture = TextureDescriptor(texels, Type2D); + std::copy(texture.begin(), texture.end(), userData.begin() + 4); + userData[12] = value; + Dispatch(device, StoreCode, userData); +} + +void LoadArray(AgcDriver::VulkanDevice& device, const std::uint8_t* texels) { + Output.fill(0xdeadbeefu); + std::vector userData(16, 0u); + const auto buffer = BufferDescriptor(Output.data(), static_cast(Output.size() * 4u)); + const auto texture = TextureDescriptor(texels, Type2DArray); + std::copy(buffer.begin(), buffer.end(), userData.begin()); + std::copy(texture.begin(), texture.end(), userData.begin() + 4); + Dispatch(device, LoadArrayCode, userData); +} + +void RequireOutput(std::uint32_t expected, const std::string& what) { + for (std::uint32_t x = 0; x < Threads; ++x) { + Require(Output[x] == expected, what + ": texel (" + std::to_string(x) + ", 0) reads " + Hex(Output[x]) + ", expected " + Hex(expected)); + } +} + +void Run(AgcDriver::VulkanDevice& device, std::uint8_t* texels, bool watched) { + const auto surface = AddressOf(texels); + std::fill_n(reinterpret_cast(texels), SurfaceBytes / 4u, Initial); + AgcDriver::GuestMemory::CollectWrites(surface, SurfaceBytes); + AgcDriver::GuestMemory::BumpCollectEpoch(); + std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex()); + Store(device, texels, 0x11111111u); + Require(StorageTexture::FindPending(surface, SurfaceBytes) != nullptr, "the 2D store left no pending image"); + LoadArray(device, texels); + RequireOutput(0x11111111u, "the first load through the one-slice array"); + Store(device, texels, 0x22222222u); + const auto written = StorageTexture::FindPending(surface, SurfaceBytes); + Require(written != nullptr && written->Descriptor().dimension == AgcDriver::Graphics::TextureDimension::k2D, "the second 2D store left no pending 2D image"); + LoadArray(device, texels); + RequireOutput(0x22222222u, "the second load through the one-slice array"); + if (watched && AgcDriver::GuestMemory::Watched(surface, SurfaceBytes)) { + Require(StorageTexture::FindPending(surface, SurfaceBytes) == written, "the array image's refresh wrote the 2D image's results back instead of taking them on the device"); + } else { + std::puts("the imported block is compared, not watched: taking the results on the device is not checked"); + } + StorageTexture::FlushPending(surface, SurfaceBytes, nullptr, "test"); + device.WaitIdle(); + Require(std::count(reinterpret_cast(texels), reinterpret_cast(texels) + SurfaceBytes / 4u, 0x22222222u) == static_cast(Threads), "guest memory does not hold the 32 texels of the second store after the flush"); + AgcDriver::Graphics::ClearCachedTextures(device.Device()); +} + +} + +int main() { + try { + const bool watched = AgcDriver::GuestMemory::WriteWatched(); + GuestBlock block(watched); + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + if (!watched) std::puts("guest memory has no write watch: only the texels read through the array are checked"); + Run(*device, block.Data(), watched); + std::puts("storage one-slice alias tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} From aa91d9144c4ce1e8b73086468d3c3c75f90546da Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 05:41:07 +0200 Subject: [PATCH 337/460] test(agc): pass the GPU flag to GuestAllocations Add in the one-slice alias test --- .../libSceAgcDriver/tests/execution/StorageOneSliceAlias.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/StorageOneSliceAlias.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/StorageOneSliceAlias.cpp index ccfa31bf3..78437ac99 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/StorageOneSliceAlias.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/StorageOneSliceAlias.cpp @@ -81,7 +81,7 @@ public: #endif Require(block != nullptr, "one-slice alias: cannot allocate the guest block"); Require(!watched || AgcDriver::GuestMemory::Watched(AddressOf(block), BlockBytes), "one-slice alias: the guest block is not write-watched"); - GuestAllocations::Mutation().Add(block, BlockBytes, true, true); + GuestAllocations::Mutation().Add(block, BlockBytes, true, true, true); } ~GuestBlock() { From 1f94d93d06e1160b3ed775d2a374f9b6e545620d Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Fri, 9 Oct 2026 19:37:42 +0900 Subject: [PATCH 338/460] feat(libc): implement log1p, log1pf and expm1 Export them forwarding to the C++ standard library, like log and exp. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libc/src/Math.cpp | 3 +++ core/libs/tests/GuestMath.cpp | 15 +++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/core/libs/prx/libc/src/Math.cpp b/core/libs/prx/libc/src/Math.cpp index 0f9236c92..171e96c99 100644 --- a/core/libs/prx/libc/src/Math.cpp +++ b/core/libs/prx/libc/src/Math.cpp @@ -59,6 +59,9 @@ double APS5_VABI atan_nid_postfix(double x) { return std::atan(x); } double APS5_VABI tan_nid_postfix(double x) { return std::tan(x); } double APS5_VABI log2_nid_postfix(double x) { return std::log2(x); } double APS5_VABI log_nid_postfix(double x) { return std::log(x); } +double APS5_VABI log1p_nid_postfix(double x) { return std::log1p(x); } +float APS5_VABI log1pf_nid_postfix(float x) { return std::log1p(x); } +double APS5_VABI expm1_nid_postfix(double x) { return std::expm1(x); } float APS5_VABI sinf_nid_postfix(float x) { return std::sin(x); } float APS5_VABI cosf_nid_postfix(float x) { return std::cos(x); } diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index ec4d5e4a7..8ae31c158 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -40,6 +40,9 @@ float APS5_VABI hypotf_nid_postfix(float, float); double APS5_VABI hypot_nid_postfix(double, double); float APS5_VABI tanf_nid_postfix(float); float APS5_VABI log10f_nid_postfix(float); +double APS5_VABI log1p_nid_postfix(double); +float APS5_VABI log1pf_nid_postfix(float); +double APS5_VABI expm1_nid_postfix(double); float APS5_VABI logbf_nid_postfix(float); double APS5_VABI exp2_nid_postfix(double); double APS5_VABI ldexp_nid_postfix(double, int); @@ -295,6 +298,18 @@ int main() { Require(std::isinf(result) && result > 0.f && std::fetestexcept(FE_INVALID) != 0); } Require(log10f_nid_postfix(100.f) == 2.f); + Require(log1p_nid_postfix(0.) == 0. && std::signbit(log1p_nid_postfix(-0.))); + Require(log1p_nid_postfix(-1.) == -std::numeric_limits::infinity() && std::isnan(log1p_nid_postfix(-2.))); + Require(log1p_nid_postfix(std::numeric_limits::infinity()) == std::numeric_limits::infinity()); + Require(log1p_nid_postfix(1e-300) == 1e-300 && log1pf_nid_postfix(1e-30f) == 1e-30f); + Require(std::abs(log1p_nid_postfix(1.) - 0.6931471805599453) < 1e-16); + Require(std::abs(log1p_nid_postfix(1e-10) - 9.9999999995e-11) < 1e-25); + Require(std::abs(log1pf_nid_postfix(1.f) - 0.6931472f) < 1e-7f && std::isnan(log1pf_nid_postfix(-2.f))); + Require(expm1_nid_postfix(0.) == 0. && std::signbit(expm1_nid_postfix(-0.))); + Require(expm1_nid_postfix(1e-300) == 1e-300 && expm1_nid_postfix(-1000.) == -1.); + Require(expm1_nid_postfix(-std::numeric_limits::infinity()) == -1.); + Require(std::abs(expm1_nid_postfix(1.) - 1.718281828459045) < 1e-15); + Require(std::abs(expm1_nid_postfix(1e-10) - 1.00000000005e-10) < 1e-25); Require(logbf_nid_postfix(8.f) == 3.f && logbf_nid_postfix(-0.75f) == -1.f); Require(logbf_nid_postfix(std::numeric_limits::denorm_min()) == -149.f); Require(logbf_nid_postfix(0.f) == -std::numeric_limits::infinity()); From 2c012b4fe59a4aded989e7af4062cb3531ddcf5a Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 17:50:31 +0000 Subject: [PATCH 339/460] test(net): skip unavailable IPv6 loopback checks (cherry picked from commit 9f2a0062d580b399424cb417e7f2a17cd093785b) --- core/libs/prx/libSceNet/CMakeLists.txt | 1 + core/libs/tests/GuestSceNet.cpp | 49 ++++++++++++++++++++++++-- 2 files changed, 48 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceNet/CMakeLists.txt b/core/libs/prx/libSceNet/CMakeLists.txt index 84e0b910f..aa2b08ccd 100644 --- a/core/libs/prx/libSceNet/CMakeLists.txt +++ b/core/libs/prx/libSceNet/CMakeLists.txt @@ -10,6 +10,7 @@ if(BUILD_TESTING) add_test(NAME guest_sce_net COMMAND guest_sce_net_tests) set_tests_properties(guest_sce_net PROPERTIES TIMEOUT 20 SKIP_RETURN_CODE 77) if(WIN32) + target_link_libraries(guest_sce_net_tests PRIVATE ws2_32) set_tests_properties(guest_sce_net PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() diff --git a/core/libs/tests/GuestSceNet.cpp b/core/libs/tests/GuestSceNet.cpp index cce92d275..b1f0fe4fb 100644 --- a/core/libs/tests/GuestSceNet.cpp +++ b/core/libs/tests/GuestSceNet.cpp @@ -1,3 +1,12 @@ +#ifdef _WIN32 +#define WIN32_LEAN_AND_MEAN +#include +#else +#include +#include +#include +#include +#endif #include "prx/libc/include/general/VabiMacros.hpp" #include "SceTypes.hpp" #include @@ -213,6 +222,35 @@ static void CheckAddressText(int family, const char* text) { static bool ipv6Unavailable = false; +static bool HostIpv6LoopbackUnavailable() { + const auto hostSocket = ::socket(AF_INET6, SOCK_DGRAM, IPPROTO_UDP); +#ifdef _WIN32 + if (hostSocket == INVALID_SOCKET) { + const int error = WSAGetLastError(); + return error == WSAEAFNOSUPPORT || error == WSAEPROTONOSUPPORT; + } +#else + if (hostSocket < 0) return errno == EAFNOSUPPORT || errno == EPROTONOSUPPORT; +#endif + + sockaddr_in6 loopback{}; + loopback.sin6_family = AF_INET6; + loopback.sin6_port = htons(9); + loopback.sin6_addr = in6addr_loopback; + const char payload = 0; + const auto sent = ::sendto(hostSocket, &payload, sizeof(payload), 0, + reinterpret_cast(&loopback), sizeof(loopback)); +#ifdef _WIN32 + const int error = sent < 0 ? WSAGetLastError() : 0; + closesocket(hostSocket); + return sent < 0 && (error == WSAEADDRNOTAVAIL || error == WSAENETUNREACH || error == WSAEHOSTUNREACH); +#else + const int error = sent < 0 ? errno : 0; + close(hostSocket); + return sent < 0 && (error == EADDRNOTAVAIL || error == ENETUNREACH || error == EHOSTUNREACH); +#endif +} + static void CheckUnspecifiedIpv6() { std::array unspecified{}; Require(sceNetInetPton(28, "::", unspecified.data()) == 1); @@ -239,7 +277,14 @@ static void CheckUnspecifiedIpv6() { Require(sceNetInetPton(28, "::1", address.data() + 8) == 1); const char payload[] = "IPv6 wildcard receive"; - Require(sceNetSendto(sender, payload, sizeof(payload), 0, address.data(), address.size()) == sizeof(payload)); + const auto sent = sceNetSendto(sender, payload, sizeof(payload), 0, address.data(), address.size()); + if (sent < 0 && *sceNetErrnoLoc() == 51 && HostIpv6LoopbackUnavailable()) { + Require(sceNetSocketClose(sender) == 0); + Require(sceNetSocketClose(receiver) == 0); + ipv6Unavailable = true; + return; + } + Require(sent == sizeof(payload)); char received[sizeof(payload)]{}; std::array peer{}; size = peer.size(); @@ -430,7 +475,7 @@ int main() { Require(sceNetCtlGetState(&state) == 0); Require(state == 0 || state == 3); if (ipv6Unavailable) { - std::puts("skipped the IPv6 socket checks, the host has no IPv6"); + std::puts("skipped the IPv6 socket checks, the host has no usable IPv6 loopback"); return 77; } } From 3d79d426630b8cc5933402b8a51d4121d0aac210 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 20:14:24 +0000 Subject: [PATCH 340/460] test(net): include Windows IPv6 declarations (cherry picked from commit 811d8c6b155fbfdae5e5277f90b435e68d42ffdc) --- core/libs/tests/GuestSceNet.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/core/libs/tests/GuestSceNet.cpp b/core/libs/tests/GuestSceNet.cpp index b1f0fe4fb..57d88cfc7 100644 --- a/core/libs/tests/GuestSceNet.cpp +++ b/core/libs/tests/GuestSceNet.cpp @@ -1,6 +1,7 @@ #ifdef _WIN32 #define WIN32_LEAN_AND_MEAN #include +#include #else #include #include From eecba6888dcf370f11d3e934ee1efa06dfe0b22b Mon Sep 17 00:00:00 2001 From: fma-rc <188743521+fma-rc@users.noreply.github.com> Date: Sun, 11 Oct 2026 05:48:07 +0200 Subject: [PATCH 341/460] feat(font): order writing line runs by embedding level and report their spacing --- core/libs/prx/libSceFont/src/WritingLine.cpp | 304 ++++++++++++++++--- core/libs/tests/GuestFontWritingLine.cpp | 165 +++++++--- docs/dev/TechnicalDebt.md | 6 +- 3 files changed, 381 insertions(+), 94 deletions(-) diff --git a/core/libs/prx/libSceFont/src/WritingLine.cpp b/core/libs/prx/libSceFont/src/WritingLine.cpp index 216a03deb..81c06114e 100644 --- a/core/libs/prx/libSceFont/src/WritingLine.cpp +++ b/core/libs/prx/libSceFont/src/WritingLine.cpp @@ -1,30 +1,60 @@ -#include +#include #include #include +#include #include #include #include #include -#include #include "prx/libSceFont/include/FontTypes.hpp" namespace { constexpr std::int32_t WRITING_FORM_HORIZONTAL = 0x10; +constexpr std::int32_t WRITING_FORM_HORIZONTAL_RTL = 0x11; constexpr std::int32_t WRITING_FORM_HORIZONTAL_LTR = 0x12; constexpr std::uint16_t CREATE_WRITING_LINE_DETAIL_ID = 0x0FD5; +constexpr std::uint64_t WRITING_ATTRIBUTE_ID_MASK = 0xFF; +constexpr std::uint64_t WRITING_ATTRIBUTE_ID = 0xFA; +constexpr std::uint64_t WRITING_ATTRIBUTE_SPACE_BEFORE = std::uint64_t{1} << 30; +constexpr std::uint64_t WRITING_ATTRIBUTE_SPACE_AFTER = std::uint64_t{1} << 31; +constexpr std::size_t WRITING_LEVEL_COUNT = 0x7F; struct CreateWritingLineDetail { std::uint16_t detailId; }; -struct WritingLine { - std::vector steps; - std::size_t cursor = 0; +struct WritingOrder { + void* orderer = nullptr; + WritingOrder* prev = nullptr; + WritingOrder* next = nullptr; + std::uint64_t attribute = 0; FontWritingMetrics metrics{}; }; +struct WritingLevel { + WritingOrder* first = nullptr; + WritingOrder* last = nullptr; +}; + +struct WritingLine { + std::int32_t form = 0; + FontWritingMetrics renderMetrics{}; + FontWritingLineStep step{}; + float advanceX = 0.0f; + float advanceY = 0.0f; + float orderingSpace = 0.0f; + float steppedSpace = 0.0f; + std::size_t lastLevel = 0; + WritingOrder* last = nullptr; + WritingOrder* head = nullptr; + WritingOrder* tail = nullptr; + WritingOrder* cursor = nullptr; + std::array levels{}; + std::deque orders; +}; + std::mutex linesMutex; std::unordered_set lines; @@ -39,6 +69,147 @@ WritingLine* GetLine(const char* function, void* writingLine) { return line; } +bool IsRightToLeft(const WritingLine& line) { + return line.form == WRITING_FORM_HORIZONTAL_RTL; +} + +bool SpaceAfter(const WritingOrder& order) { + return (order.attribute & WRITING_ATTRIBUTE_SPACE_AFTER) != 0; +} + +bool SpaceBefore(const WritingOrder& order) { + return (order.attribute & WRITING_ATTRIBUTE_SPACE_BEFORE) != 0; +} + +bool IsSpaced(const WritingOrder& before, const WritingOrder& after) { + return SpaceAfter(before) || SpaceBefore(after); +} + +float HeadSpace(const WritingLine& line, const WritingOrder& order) { + return IsRightToLeft(line) ? order.metrics.advanceX - order.metrics.Extent.left : order.metrics.Extent.left; +} + +float TailSpace(const WritingLine& line, const WritingOrder& order) { + return IsRightToLeft(line) ? order.metrics.Extent.right : order.metrics.advanceX - order.metrics.Extent.right; +} + +void ResetOrders(WritingLine& line) { + line.orders.clear(); + line.levels.fill({}); + line.advanceX = 0.0f; + line.advanceY = 0.0f; + line.orderingSpace = 0.0f; + line.last = nullptr; + line.head = nullptr; + line.tail = nullptr; + line.cursor = nullptr; +} + +void InsertAfter(WritingOrder* anchor, WritingOrder* order) { + order->prev = anchor; + order->next = anchor->next; + if (anchor->next) anchor->next->prev = order; + anchor->next = order; +} + +void InsertBefore(WritingOrder* anchor, WritingOrder* order) { + order->next = anchor; + order->prev = anchor->prev; + if (anchor->prev) anchor->prev->next = order; + anchor->prev = order; +} + +void Link(WritingLine& line, WritingOrder* order, std::size_t level) { + WritingLevel& slot = line.levels[level]; + if (!line.last) { + slot = {order, order}; + return; + } + if (level >= line.lastLevel) { + if (line.lastLevel % 2 == 0) { + InsertAfter(line.last, order); + } else { + InsertBefore(line.last, order); + } + if (!slot.last || order->prev == slot.last) slot.last = order; + if (!slot.first || order->next == slot.first) slot.first = order; + return; + } + WritingOrder* first = line.levels[line.lastLevel].first; + WritingOrder* last = line.levels[line.lastLevel].last; + line.levels[line.lastLevel] = {}; + for (std::size_t closing = line.lastLevel - 1; closing > level; --closing) { + WritingLevel& closed = line.levels[closing]; + if (closing % 2 == 0) { + if (closed.first) first = closed.first; + } else { + if (closed.last) last = closed.last; + } + closed = {}; + } + if (level % 2 == 0) { + if (!slot.first) slot.first = first; + InsertAfter(last, order); + slot.last = order; + } else { + if (!slot.last) slot.last = last; + InsertBefore(first, order); + slot.first = order; + } +} + +float OrderingSpaceDelta(const WritingLine& line, const WritingOrder& order) { + const WritingOrder* before = order.prev; + const WritingOrder* after = order.next; + const bool split = before && after && IsSpaced(*before, *after); + const float sign = split ? -1.0f : 1.0f; + float delta = 0.0f; + if (before) { + const bool spaced = IsSpaced(*before, order); + if (spaced) delta = HeadSpace(line, order); + if (spaced != split) delta += sign * TailSpace(line, *before); + } + if (after) { + const bool spaced = IsSpaced(order, *after); + if (spaced) delta += TailSpace(line, order); + if (spaced != split) delta += sign * HeadSpace(line, *after); + } + return delta; +} + +void ExtendRenderMetrics(WritingLine& line) { + const FontWritingMetrics& run = line.step.Metrics; + const float x = line.step.x + line.step.Adjusting.x; + const float y = line.step.y + line.step.Adjusting.y; + line.step.x += run.advanceX; + line.step.y += run.advanceY; + FontWritingMetrics& total = line.renderMetrics; + total.advanceX = x + run.advanceX; + total.advanceY = y + run.advanceY; + if (!(run.Extent.left == run.Extent.right)) { + const float left = x + run.Extent.left; + const float right = x + run.Extent.right; + if (total.Extent.left == 0.0f && total.Extent.right == 0.0f) { + total.Extent.left = left; + total.Extent.right = right; + } else { + if (total.Extent.left > left) total.Extent.left = left; + if (right > total.Extent.right) total.Extent.right = right; + } + } + if (!(run.Extent.top == run.Extent.bottom)) { + const float top = y + run.Extent.top; + const float bottom = y + run.Extent.bottom; + if (total.Extent.top == 0.0f && total.Extent.bottom == 0.0f) { + total.Extent.top = top; + total.Extent.bottom = bottom; + } else { + if (total.Extent.top > top) total.Extent.top = top; + if (bottom > total.Extent.bottom) total.Extent.bottom = bottom; + } + } +} + } #pragma GCC visibility push(default) @@ -48,11 +219,12 @@ extern "C" { int APS5_VABI sceFontCreateWritingLine(const FontMemory* memory, std::int32_t writingForm, const void* writingLineDetail, void** pWritingLine) { (void)memory; if (!pWritingLine) Unsupported(__func__, "null writing line pointer (error code not verified)"); - if (writingForm != WRITING_FORM_HORIZONTAL && writingForm != WRITING_FORM_HORIZONTAL_LTR) + if (writingForm != WRITING_FORM_HORIZONTAL && writingForm != WRITING_FORM_HORIZONTAL_RTL && writingForm != WRITING_FORM_HORIZONTAL_LTR) Unsupported(__func__, "writing form " + std::to_string(writingForm) + " is not modelled"); if (writingLineDetail && static_cast(writingLineDetail)->detailId != CREATE_WRITING_LINE_DETAIL_ID) Unsupported(__func__, "unknown writing line detail id"); auto* line = new WritingLine{}; + line->form = writingForm; { std::lock_guard lock(linesMutex); lines.insert(line); @@ -74,63 +246,107 @@ int APS5_VABI sceFontDestroyWritingLine(void** pWritingLine) { } int APS5_VABI sceFontWritingLineClear(void* writingLine) { + if (!writingLine) return SCE_FONT_ERROR_INVALID_PARAMETER; auto* line = GetLine(__func__, writingLine); - line->steps.clear(); - line->cursor = 0; - line->metrics = {}; + ResetOrders(*line); return SCE_FONT_OK; } int APS5_VABI sceFontWritingLineWritesOrder(void* writingLine, std::uint64_t writingAttribute, const FontWritingMetrics* writingMetrics, void* writingOrderer) { + if (!writingLine || !writingOrderer) return SCE_FONT_ERROR_INVALID_PARAMETER; + if ((writingAttribute & WRITING_ATTRIBUTE_ID_MASK) != WRITING_ATTRIBUTE_ID) return SCE_FONT_ERROR_INVALID_PARAMETER; + const std::size_t level = (writingAttribute >> 8) & 0xFF; + if (level >= WRITING_LEVEL_COUNT) return SCE_FONT_ERROR_INVALID_PARAMETER; auto* line = GetLine(__func__, writingLine); - if (writingAttribute != 0) Unsupported(__func__, "writing attribute " + std::to_string(writingAttribute) + " is not modelled"); - if (!writingMetrics) Unsupported(__func__, "null writing metrics are not modelled"); - const FontWritingMetrics& order = *writingMetrics; - const float pen = line->metrics.advanceX; - FontWritingLineStep step{}; - step.x = pen; - step.advanceX = order.advanceX; - step.advanceY = order.advanceY; - step.writingOrderer = writingOrderer; - step.Metrics = order; - const FontWritingExtent extent{order.Extent.top, order.Extent.bottom, pen + order.Extent.left, pen + order.Extent.right}; - if (line->steps.empty()) { - line->metrics.Extent = extent; - } else { - line->metrics.Extent.top = std::min(line->metrics.Extent.top, extent.top); - line->metrics.Extent.bottom = std::max(line->metrics.Extent.bottom, extent.bottom); - line->metrics.Extent.left = std::min(line->metrics.Extent.left, extent.left); - line->metrics.Extent.right = std::max(line->metrics.Extent.right, extent.right); - } - line->metrics.advanceX += order.advanceX; - line->metrics.advanceY += order.advanceY; - line->steps.push_back(step); - line->cursor = 0; + if (line->cursor) ResetOrders(*line); + WritingOrder* order = &line->orders.emplace_back(); + order->orderer = writingOrderer; + order->attribute = writingAttribute; + if (writingMetrics) order->metrics = *writingMetrics; + Link(*line, order, level); + if (!order->prev) line->head = order; + if (!order->next) line->tail = order; + line->orderingSpace += OrderingSpaceDelta(*line, *order); + line->advanceX += order->metrics.advanceX; + line->advanceY += order->metrics.advanceY; + line->last = order; + line->lastLevel = level; return SCE_FONT_OK; } const FontWritingLineStep* APS5_VABI sceFontWritingLineRefersRenderStep(void* writingLine) { if (!writingLine) return nullptr; auto* line = GetLine(__func__, writingLine); - if (line->cursor >= line->steps.size()) return nullptr; - return &line->steps[line->cursor++]; + if (line->cursor) { + ExtendRenderMetrics(*line); + } else { + line->steppedSpace = 0.0f; + line->step = {}; + line->renderMetrics = {}; + } + const bool rightToLeft = IsRightToLeft(*line); + WritingOrder* next = nullptr; + if (rightToLeft) { + next = line->cursor ? line->cursor->prev : line->tail; + } else { + next = line->cursor ? line->cursor->next : line->head; + } + if (!next) { + line->cursor = nullptr; + line->step.Metrics = {}; + return nullptr; + } + if (rightToLeft) { + const WritingOrder* stepped = next->next; + if (stepped && IsSpaced(*next, *stepped)) + line->steppedSpace += stepped->metrics.advanceX - stepped->metrics.Extent.left + next->metrics.Extent.right; + } else { + const WritingOrder* stepped = next->prev; + if (stepped && IsSpaced(*stepped, *next)) + line->steppedSpace += stepped->metrics.advanceX - stepped->metrics.Extent.right + next->metrics.Extent.left; + } + line->cursor = next; + line->step.Metrics = next->metrics; + line->step.writingOrderer = next->orderer; + line->step.Adjusting = {}; + line->step.spacingProgress = 0.0f; + if (!(line->steppedSpace == 0.0f) && !(line->orderingSpace == 0.0f)) line->step.spacingProgress = line->steppedSpace / line->orderingSpace; + return &line->step; } int APS5_VABI sceFontWritingLineGetRenderMetrics(void* writingLine, FontWritingMetrics* writingMetrics) { + if (!writingLine || !writingMetrics) { + if (writingMetrics) *writingMetrics = {}; + return SCE_FONT_ERROR_INVALID_PARAMETER; + } auto* line = GetLine(__func__, writingLine); - if (!writingMetrics) Unsupported(__func__, "null metrics pointer (error code not verified)"); - if (line->steps.empty()) Unsupported(__func__, "metrics of an empty line are not modelled"); - *writingMetrics = line->metrics; + *writingMetrics = line->renderMetrics; return SCE_FONT_OK; } -int APS5_VABI sceFontWritingLineGetOrderingSpace(void* writingLine, float* headSpace, float* inlineSpace, float* tailSpace, float* advanceSpace) { - GetLine(__func__, writingLine); - if (!headSpace || !inlineSpace || !tailSpace || !advanceSpace) Unsupported(__func__, "null space pointer (error code not verified)"); - *headSpace = 0.0f; - *inlineSpace = 0.0f; - *tailSpace = 0.0f; - *advanceSpace = 0.0f; +int APS5_VABI sceFontWritingLineGetOrderingSpace(void* writingLine, float* advanceSpace, float* headSpace, float* inlineSpace, float* tailSpace) { + if (!writingLine) { + if (advanceSpace) *advanceSpace = 0.0f; + if (headSpace) *headSpace = 0.0f; + if (inlineSpace) *inlineSpace = 0.0f; + if (tailSpace) *tailSpace = 0.0f; + return SCE_FONT_ERROR_INVALID_PARAMETER; + } + auto* line = GetLine(__func__, writingLine); + if (headSpace && !tailSpace) Unsupported(__func__, "a head space pointer without a tail space pointer is not modelled"); + if (advanceSpace) *advanceSpace = line->advanceX; + if (headSpace) { + const WritingOrder* leading = IsRightToLeft(*line) ? line->tail : line->head; + const WritingOrder* trailing = IsRightToLeft(*line) ? line->head : line->tail; + if (IsRightToLeft(*line)) { + *headSpace = leading ? leading->metrics.Extent.right : 0.0f; + *tailSpace = trailing ? trailing->metrics.advanceX - trailing->metrics.Extent.left : 0.0f; + } else { + *headSpace = leading ? leading->metrics.Extent.left : 0.0f; + *tailSpace = trailing ? trailing->metrics.advanceX - trailing->metrics.Extent.right : 0.0f; + } + } + if (inlineSpace) *inlineSpace = line->orderingSpace; return SCE_FONT_OK; } diff --git a/core/libs/tests/GuestFontWritingLine.cpp b/core/libs/tests/GuestFontWritingLine.cpp index 7d5e40f08..07542e3a7 100644 --- a/core/libs/tests/GuestFontWritingLine.cpp +++ b/core/libs/tests/GuestFontWritingLine.cpp @@ -38,23 +38,30 @@ struct WritingLineDetail { void* reservedPointers[3]; }; +constexpr std::uint64_t SPACE_BEFORE = std::uint64_t{1} << 30; +constexpr std::uint64_t SPACE_AFTER = std::uint64_t{1} << 31; + +static std::uint64_t Attribute(std::uint64_t level, std::uint64_t flags = 0) { + return 0xFA | (level << 8) | flags; +} + static bool Same(const FontWritingMetrics& a, const FontWritingMetrics& b) { return a.advanceX == b.advanceX && a.advanceY == b.advanceY && a.Extent.top == b.Extent.top && a.Extent.bottom == b.Extent.bottom && a.Extent.left == b.Extent.left && a.Extent.right == b.Extent.right; } -static bool NoSpace(void* line) { - float head = 1.0f; - float inner = 1.0f; - float tail = 1.0f; - float advance = 1.0f; - return sceFontWritingLineGetOrderingSpace(line, &head, &inner, &tail, &advance) == SCE_FONT_OK && head == 0.0f && inner == 0.0f && - tail == 0.0f && advance == 0.0f; +static bool SpaceIs(void* line, float advance, float head, float inner, float tail) { + float gotAdvance = -1.0f; + float gotHead = -1.0f; + float gotInner = -1.0f; + float gotTail = -1.0f; + return sceFontWritingLineGetOrderingSpace(line, &gotAdvance, &gotHead, &gotInner, &gotTail) == SCE_FONT_OK && gotAdvance == advance && + gotHead == head && gotInner == inner && gotTail == tail; } -static bool StepIs(const FontWritingLineStep* step, float x, const FontWritingMetrics& run, void* orderer) { - return step && step->x == x && step->y == 0.0f && step->advanceX == run.advanceX && step->advanceY == run.advanceY && - step->spacingProgress == 0.0f && step->writingOrderer == orderer && step->Adjusting.x == 0.0f && +static bool StepIs(const FontWritingLineStep* step, float x, float progress, const FontWritingMetrics& run, void* orderer) { + return step && step->x == x && step->y == 0.0f && step->advanceX == 0.0f && step->advanceY == 0.0f && + step->spacingProgress == progress && step->writingOrderer == orderer && step->Adjusting.x == 0.0f && step->Adjusting.y == 0.0f && Same(step->Metrics, run); } @@ -62,7 +69,7 @@ int main() { FontMemory memory{}; void* line = nullptr; Require(Throws([&] { sceFontCreateWritingLine(&memory, 0x10, nullptr, nullptr); })); - Require(Throws([&] { sceFontCreateWritingLine(&memory, 0x11, nullptr, &line); }) && line == nullptr); + Require(Throws([&] { sceFontCreateWritingLine(&memory, 0x13, nullptr, &line); }) && line == nullptr); Require(Throws([&] { sceFontCreateWritingLine(&memory, 0, nullptr, &line); }) && line == nullptr); WritingLineDetail detail{0x0FD4, {}, {}}; Require(Throws([&] { sceFontCreateWritingLine(&memory, 0x10, &detail, &line); }) && line == nullptr); @@ -70,58 +77,121 @@ int main() { Require(sceFontCreateWritingLine(&memory, 0x10, &detail, &line) == SCE_FONT_OK && line != nullptr); void* ltr = nullptr; Require(sceFontCreateWritingLine(&memory, 0x12, nullptr, <r) == SCE_FONT_OK && ltr != nullptr && ltr != line); + void* rtl = nullptr; + Require(sceFontCreateWritingLine(&memory, 0x11, nullptr, &rtl) == SCE_FONT_OK && rtl != nullptr); - FontWritingMetrics metrics{}; - Require(NoSpace(line)); + FontWritingMetrics metrics{1.0f, 1.0f, {1.0f, 1.0f, 1.0f, 1.0f}}; + Require(SpaceIs(line, 0.0f, 0.0f, 0.0f, 0.0f)); Require(sceFontWritingLineRefersRenderStep(line) == nullptr); - Require(Throws([&] { sceFontWritingLineGetRenderMetrics(line, &metrics); })); + Require(sceFontWritingLineGetRenderMetrics(line, &metrics) == SCE_FONT_OK && Same(metrics, {})); - int ordererA = 0; - int ordererB = 0; + int a = 0; + int b = 0; + int c = 0; + int d = 0; const FontWritingMetrics runA{10.0f, 0.0f, {-8.0f, 2.0f, 1.0f, 9.0f}}; const FontWritingMetrics runB{6.0f, 0.0f, {-12.0f, 3.0f, -1.0f, 5.0f}}; - Require(sceFontWritingLineWritesOrder(line, 0, &runA, &ordererA) == SCE_FONT_OK); - Require(sceFontWritingLineGetRenderMetrics(line, &metrics) == SCE_FONT_OK && Same(metrics, runA)); - Require(sceFontWritingLineWritesOrder(line, 0, &runB, &ordererB) == SCE_FONT_OK); - Require(sceFontWritingLineGetRenderMetrics(line, &metrics) == SCE_FONT_OK); - Require(Same(metrics, {16.0f, 0.0f, {-12.0f, 3.0f, 1.0f, 15.0f}})); - Require(NoSpace(line)); - float space = 0.0f; - Require(Throws([&] { sceFontWritingLineGetOrderingSpace(line, nullptr, &space, &space, &space); })); - Require(Throws([&] { sceFontWritingLineGetOrderingSpace(line, &space, &space, &space, nullptr); })); + const FontWritingMetrics runC{8.0f, 0.0f, {-4.0f, 1.0f, 2.0f, 5.0f}}; + const FontWritingMetrics runD{4.0f, 0.0f, {-4.0f, 1.0f, 0.0f, 20.0f}}; - Require(StepIs(sceFontWritingLineRefersRenderStep(line), 0.0f, runA, &ordererA)); - Require(StepIs(sceFontWritingLineRefersRenderStep(line), 10.0f, runB, &ordererB)); - Require(sceFontWritingLineRefersRenderStep(line) == nullptr); + Require(sceFontWritingLineWritesOrder(nullptr, Attribute(0), &runA, &a) == SCE_FONT_ERROR_INVALID_PARAMETER); + Require(sceFontWritingLineWritesOrder(line, 0, &runA, &a) == SCE_FONT_ERROR_INVALID_PARAMETER); + Require(sceFontWritingLineWritesOrder(line, 0x1FB, &runA, &a) == SCE_FONT_ERROR_INVALID_PARAMETER); + Require(sceFontWritingLineWritesOrder(line, Attribute(0), &runA, nullptr) == SCE_FONT_ERROR_INVALID_PARAMETER); + Require(sceFontWritingLineWritesOrder(line, Attribute(0x7F), &runA, &a) == SCE_FONT_ERROR_INVALID_PARAMETER); Require(sceFontWritingLineRefersRenderStep(line) == nullptr); - const FontWritingMetrics runC{4.0f, 0.0f, {-4.0f, 1.0f, 0.0f, 20.0f}}; - Require(sceFontWritingLineWritesOrder(line, 0, &runC, nullptr) == SCE_FONT_OK); - Require(sceFontWritingLineGetRenderMetrics(line, &metrics) == SCE_FONT_OK); - Require(Same(metrics, {20.0f, 0.0f, {-12.0f, 3.0f, 1.0f, 36.0f}})); - Require(StepIs(sceFontWritingLineRefersRenderStep(line), 0.0f, runA, &ordererA)); - Require(StepIs(sceFontWritingLineRefersRenderStep(line), 10.0f, runB, &ordererB)); - Require(StepIs(sceFontWritingLineRefersRenderStep(line), 16.0f, runC, nullptr)); + Require(sceFontWritingLineWritesOrder(line, Attribute(0), &runA, &a) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(line, Attribute(0), &runB, &b) == SCE_FONT_OK); + Require(SpaceIs(line, 16.0f, 1.0f, 0.0f, 1.0f)); + Require(sceFontWritingLineGetRenderMetrics(line, &metrics) == SCE_FONT_OK && Same(metrics, {})); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 0.0f, 0.0f, runA, &a)); + Require(sceFontWritingLineGetRenderMetrics(line, &metrics) == SCE_FONT_OK && Same(metrics, {})); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 10.0f, 0.0f, runB, &b)); + Require(sceFontWritingLineGetRenderMetrics(line, &metrics) == SCE_FONT_OK && Same(metrics, {10.0f, 0.0f, {-8.0f, 2.0f, 1.0f, 9.0f}})); Require(sceFontWritingLineRefersRenderStep(line) == nullptr); + Require(sceFontWritingLineGetRenderMetrics(line, &metrics) == SCE_FONT_OK && Same(metrics, {16.0f, 0.0f, {-12.0f, 3.0f, 1.0f, 15.0f}})); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 0.0f, 0.0f, runA, &a)); + Require(sceFontWritingLineGetRenderMetrics(line, &metrics) == SCE_FONT_OK && Same(metrics, {})); - Require(Throws([&] { sceFontWritingLineWritesOrder(line, 1, &runA, nullptr); })); - Require(Throws([&] { sceFontWritingLineWritesOrder(line, 0, nullptr, nullptr); })); - Require(Throws([&] { sceFontWritingLineGetRenderMetrics(line, nullptr); })); - Require(sceFontWritingLineGetRenderMetrics(line, &metrics) == SCE_FONT_OK && metrics.advanceX == 20.0f); + Require(sceFontWritingLineWritesOrder(line, Attribute(0x7E), nullptr, &d) == SCE_FONT_OK); + Require(SpaceIs(line, 0.0f, 0.0f, 0.0f, 0.0f)); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 0.0f, 0.0f, {}, &d)); + Require(sceFontWritingLineRefersRenderStep(line) == nullptr); Require(sceFontWritingLineClear(line) == SCE_FONT_OK); Require(sceFontWritingLineRefersRenderStep(line) == nullptr); - Require(Throws([&] { sceFontWritingLineGetRenderMetrics(line, &metrics); })); - Require(sceFontWritingLineWritesOrder(line, 0, &runB, &ordererB) == SCE_FONT_OK); - Require(sceFontWritingLineGetRenderMetrics(line, &metrics) == SCE_FONT_OK && Same(metrics, runB)); - Require(StepIs(sceFontWritingLineRefersRenderStep(line), 0.0f, runB, &ordererB)); + Require(sceFontWritingLineWritesOrder(line, Attribute(0), &runA, &a) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(line, Attribute(1), &runB, &b) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(line, Attribute(1), &runC, &c) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(line, Attribute(0), &runD, &d) == SCE_FONT_OK); + Require(SpaceIs(line, 28.0f, 1.0f, 0.0f, -16.0f)); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 0.0f, 0.0f, runA, &a)); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 10.0f, 0.0f, runC, &c)); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 18.0f, 0.0f, runB, &b)); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 24.0f, 0.0f, runD, &d)); + Require(sceFontWritingLineRefersRenderStep(line) == nullptr); - Require(sceFontWritingLineWritesOrder(ltr, 0, &runC, nullptr) == SCE_FONT_OK); - Require(sceFontWritingLineGetRenderMetrics(ltr, &metrics) == SCE_FONT_OK && Same(metrics, runC)); + Require(sceFontWritingLineClear(line) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(line, Attribute(0), &runA, &a) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(line, Attribute(1), &runB, &b) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(line, Attribute(2), &runC, &c) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(line, Attribute(2), &runD, &d) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(line, Attribute(1), &runA, &b) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(line, Attribute(0), &runB, &a) == SCE_FONT_OK); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 0.0f, 0.0f, runA, &a)); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 10.0f, 0.0f, runA, &b)); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 20.0f, 0.0f, runC, &c)); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 28.0f, 0.0f, runD, &d)); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 32.0f, 0.0f, runB, &b)); + Require(StepIs(sceFontWritingLineRefersRenderStep(line), 38.0f, 0.0f, runB, &a)); + Require(sceFontWritingLineRefersRenderStep(line) == nullptr); + + Require(sceFontWritingLineWritesOrder(ltr, Attribute(0, SPACE_AFTER), &runA, &a) == SCE_FONT_OK); + Require(SpaceIs(ltr, 10.0f, 1.0f, 0.0f, 1.0f)); + Require(sceFontWritingLineWritesOrder(ltr, Attribute(0), &runC, &c) == SCE_FONT_OK); + Require(SpaceIs(ltr, 18.0f, 1.0f, 3.0f, 3.0f)); + Require(sceFontWritingLineWritesOrder(ltr, Attribute(0, SPACE_BEFORE), &runB, &b) == SCE_FONT_OK); + Require(SpaceIs(ltr, 24.0f, 1.0f, 5.0f, 1.0f)); + Require(StepIs(sceFontWritingLineRefersRenderStep(ltr), 0.0f, 0.0f, runA, &a)); + Require(StepIs(sceFontWritingLineRefersRenderStep(ltr), 10.0f, 3.0f / 5.0f, runC, &c)); + Require(StepIs(sceFontWritingLineRefersRenderStep(ltr), 18.0f, 1.0f, runB, &b)); + Require(sceFontWritingLineRefersRenderStep(ltr) == nullptr); + + Require(sceFontWritingLineClear(ltr) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(ltr, Attribute(0, SPACE_AFTER), &runA, &a) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(ltr, Attribute(1), &runC, &c) == SCE_FONT_OK); + Require(SpaceIs(ltr, 18.0f, 1.0f, 3.0f, 3.0f)); + Require(sceFontWritingLineWritesOrder(ltr, Attribute(1), &runB, &b) == SCE_FONT_OK); + Require(SpaceIs(ltr, 24.0f, 1.0f, 0.0f, 3.0f)); + Require(sceFontWritingLineWritesOrder(ltr, Attribute(1, SPACE_AFTER), &runD, &d) == SCE_FONT_OK); + Require(SpaceIs(ltr, 28.0f, 1.0f, -16.0f, 3.0f)); + Require(StepIs(sceFontWritingLineRefersRenderStep(ltr), 0.0f, 0.0f, runA, &a)); + Require(StepIs(sceFontWritingLineRefersRenderStep(ltr), 10.0f, 1.0f / -16.0f, runD, &d)); + Require(StepIs(sceFontWritingLineRefersRenderStep(ltr), 14.0f, 1.0f, runB, &b)); + Require(StepIs(sceFontWritingLineRefersRenderStep(ltr), 20.0f, 1.0f, runC, &c)); + Require(sceFontWritingLineRefersRenderStep(ltr) == nullptr); + + Require(sceFontWritingLineWritesOrder(rtl, Attribute(0, SPACE_AFTER), &runA, &a) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(rtl, Attribute(0), &runC, &c) == SCE_FONT_OK); + Require(SpaceIs(rtl, 18.0f, 5.0f, 15.0f, 9.0f)); + Require(StepIs(sceFontWritingLineRefersRenderStep(rtl), 0.0f, 0.0f, runC, &c)); + Require(StepIs(sceFontWritingLineRefersRenderStep(rtl), 8.0f, 1.0f, runA, &a)); + Require(sceFontWritingLineRefersRenderStep(rtl) == nullptr); + Require(sceFontWritingLineGetRenderMetrics(rtl, &metrics) == SCE_FONT_OK && Same(metrics, {18.0f, 0.0f, {-8.0f, 2.0f, 2.0f, 17.0f}})); + + float space = 7.0f; + Require(sceFontWritingLineGetOrderingSpace(rtl, nullptr, nullptr, &space, nullptr) == SCE_FONT_OK && space == 15.0f); + Require(Throws([&] { sceFontWritingLineGetOrderingSpace(rtl, &space, &space, &space, nullptr); })); + space = 7.0f; + Require(sceFontWritingLineGetOrderingSpace(nullptr, &space, nullptr, nullptr, nullptr) == SCE_FONT_ERROR_INVALID_PARAMETER && space == 0.0f); + metrics = runA; + Require(sceFontWritingLineGetRenderMetrics(nullptr, &metrics) == SCE_FONT_ERROR_INVALID_PARAMETER && Same(metrics, {})); + Require(sceFontWritingLineGetRenderMetrics(rtl, nullptr) == SCE_FONT_ERROR_INVALID_PARAMETER); + Require(sceFontWritingLineClear(nullptr) == SCE_FONT_ERROR_INVALID_PARAMETER); int notALine = 0; Require(sceFontWritingLineRefersRenderStep(nullptr) == nullptr); - Require(Throws([&] { sceFontWritingLineClear(nullptr); })); Require(Throws([&] { sceFontWritingLineClear(¬ALine); })); Require(Throws([&] { sceFontWritingLineRefersRenderStep(¬ALine); })); Require(Throws([&] { sceFontWritingLineGetOrderingSpace(¬ALine, &space, &space, &space, &space); })); @@ -133,7 +203,8 @@ int main() { Require(sceFontDestroyWritingLine(&line) == SCE_FONT_OK && line == nullptr); Require(sceFontDestroyWritingLine(nullptr) == SCE_FONT_OK); Require(Throws([&] { sceFontWritingLineClear(destroyed); })); - Require(Throws([&] { sceFontWritingLineWritesOrder(destroyed, 0, &runA, nullptr); })); + Require(Throws([&] { sceFontWritingLineWritesOrder(destroyed, Attribute(0), &runA, &a); })); Require(sceFontDestroyWritingLine(<r) == SCE_FONT_OK && ltr == nullptr); + Require(sceFontDestroyWritingLine(&rtl) == SCE_FONT_OK && rtl == nullptr); std::puts("font writing line: ok"); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 3d6495951..b599c50aa 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -105,8 +105,8 @@ Throughout the project, every function at every stage either **does exactly what - [sceFontDettachDeviceCacheBuffer](../../core/libs/prx/libSceFont/src/Library.cpp) (libSceFont) - signature taken from the IDA type database of [Orbital](https://github.com/AlexAltea/orbital/blob/master/tools/ida/db_types.json), not confirmed on a title. It detaches the cache buffer attached by `sceFontAttachDeviceCacheBuffer`, returns the buffer and its size, and reports `SCE_FONT_ERROR_NOT_ATTACHED_CACHE_BUFFER` when no cache buffer is attached - [sceFontGetFontGlyphsCount, sceFontGetCharGlyphCode](../../core/libs/prx/libSceFont/src/Open.cpp) (libSceFont) - signatures taken from the IDA type database of [Orbital](https://github.com/AlexAltea/orbital/blob/master/tools/ida/db_types.json) and matching the [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.Engine.cs) bindings, not confirmed on a title. The count is the face's glyph count and the glyph code its glyph index, as the library's driver reports them; a code with no glyph returns `NO_SUPPORT_GLYPH` and code 0 `NO_SUPPORT_CODE`, as `sceFontRenderCharGlyphImage` does. The argument checks and the cleared output follow `sceFontGetPixelResolution` - [sceFontGetFontResolution](../../core/libs/prx/libSceFont/src/Open.cpp) (libSceFont) - signature taken from the IDA type database of [Orbital](https://github.com/AlexAltea/orbital/blob/master/tools/ida/db_types.json), not confirmed on a title; the outputs are inferred from the names. The resolution is the face's units per EM and the scale pixel the driver's face scale (units per EM / 64 for FreeType), the pixel scale a font has when opened. Either output may be null, as in `sceFontGetResolutionDpi` -- [sceFontCreateWritingLine, sceFontWritingLineWritesOrder](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - signatures and the `SceFontWritingLineStep`/`SceFontCreateWritingLineDetail` layouts (detail id `0x0FD5`) taken from [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFontStructs.cs), not confirmed on a title; the meaning of the 64-bit writing attribute is unknown -- [sceFontWritingLineGetOrderingSpace](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - signature taken from the IDA type database of [Orbital](https://github.com/AlexAltea/orbital/blob/master/tools/ida/db_types.json) and the [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.Engine.cs) bindings, not confirmed on a title. The modelled line applies no spacing, so the head, inline, tail and advance spaces are 0; what spacing the console applies is unknown. A null output throws +- [sceFontCreateWritingLine, sceFontWritingLineWritesOrder](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - signatures and the `SceFontWritingLineStep`/`SceFontCreateWritingLineDetail` layouts (detail id `0x0FD5`) taken from [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFontStructs.cs), not confirmed on a title. The 64-bit writing attribute is read as the PS4 11.00 `libSceFont.sprx` reads it: low byte `0xFA`, embedding level in bits 8-15 (0 to 126), bit 30 and bit 31 marking space before and after the run; the PS5 library was not measured, and the names of the two bits are ours +- [sceFontWritingLineGetOrderingSpace](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - the four outputs are written in the order the PS4 11.00 `libSceFont.sprx` writes them (line advance, leading space, space between spaced runs, trailing space); the parameter names in the [Orbital](https://github.com/AlexAltea/orbital/blob/master/tools/ida/db_types.json) IDA type database and the [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.Engine.cs) bindings order them differently, and the PS5 library was not measured. The PS4 library writes the trailing space through its pointer without a null check whenever the leading-space pointer is set; that call throws here - [sceSaveDataTransferringMountPs4, sceSaveDataDirNameSearchPs4](../../core/libs/prx/libSceSaveData.native/Export.cpp) (libSceSaveData.native) - arguments assumed to be those of `sceSaveDataTransferringMount` and `sceSaveDataDirNameSearch`: KytyPS5 binds both NIDs to the PS5 functions, and [prosper](https://github.com/mattias800/prosper) captured the same result layout on a title for the search. There is no PS4 save area, so the mount returns `NOT_FOUND` and the search reports zero hits - [sceCoredumpWriteUserString](../../core/libs/prx/libkernel/System/src/Coredump.cpp) and [sceSaveDataConvert, sceSaveDataGetConvertProgress](../../core/libs/prx/libSceSaveData.native/Export.cpp) - imported by PPSA16064, not called during its start-up and added without an implementation; unknown signatures - [sceSaveDataCancel](../../core/libs/prx/libSceSaveData.native/Export.cpp) (libSceSaveData.native) - imported by PPSA23126, which fails to load without it; added without an implementation because its signature is unknown. Every save data call here finishes before it returns, so nothing is ever in flight to cancel, but what the console returns then is unknown @@ -492,7 +492,7 @@ Throughout the project, every function at every stage either **does exactly what - [Thread exit destructors](../../core/libs/prx/libSceLibcInternal/Export.cpp) (libSceLibcInternal) run when a libkernel thread finishes; on the main or a host thread only `_sceLibcInternalThreadDtors` runs them, as `exit` does, else they are dropped - [sceFontOpenFontSet](../../core/libs/prx/libSceFont/src/SystemFont.cpp) (libSceFont) opens only the primary file of a system font set. Characters the console takes from the set's other files (Hangul, Thai and Arabic in Japanese and Chinese sets, Arabic in European sets) are reported as unsupported glyphs. The console file names are those of the PS4 firmware; the names of the PS5-only JG2 sets (`0x1A......`) are unknown, so they use the SSTJpPro files. When a console file is missing, an openly licensed substitute (Noto Sans, Noto Sans Mono, Noto Sans Thai, Noto Sans CJK) is loaded instead. It is not the system font: its metrics differ, so text width, line height and wrapping differ from the console. System font faces are loaded by FreeType in host memory, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. - [sceFontGetCharGlyphMetrics, sceFontGetRenderCharGlyphMetrics](../../core/libs/prx/libSceFont/src/Layout.cpp) (libSceFont) report the vertical metrics the way the [FreeType driver](../../core/libs/prx/libSceFontFt/src/Driver.cpp) reports them for a rendered glyph, not confirmed on a title: FreeType's vertical advance, and for a face without vertical metrics a bearing of half the horizontal advance left of the horizontal bearing and of half the advance left over by the glyph height. -- [Font writing lines](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - `sceFontCreateWritingLine`, `sceFontWritingLineWritesOrder`, `sceFontWritingLineRefersRenderStep`, `sceFontWritingLineGetRenderMetrics` and `sceFontWritingLineClear` follow [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/main/src/libs/libFont.cpp): runs are placed left to right, each at the line's advance so far, with no spacing or adjustment. Only horizontal and left-to-right lines, writing attribute 0 and non-null run metrics are modelled; other values, the metrics of an empty line and unknown or null handles throw, since no reference gives their result or error code. `sceFontWritingLineGetOrderingSpace` still throws (KytyPS5's values are placeholders). Lines are allocated on the host, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. +- [Font writing lines](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - `sceFontWritingLineWritesOrder`, `sceFontWritingLineRefersRenderStep`, `sceFontWritingLineGetRenderMetrics`, `sceFontWritingLineGetOrderingSpace` and `sceFontWritingLineClear` follow the PS4 11.00 `libSceFont.sprx` (firmware module, read from its disassembly and compared call for call on random sequences): runs are linked in visual order by embedding level, the space at each spaced join is summed for the line, steps walk the visual order from the left (form `0x12`) or from the right (form `0x11`) and the render metrics grow as steps are consumed. The PS5 library was not measured. Not from that module: form `0x10` is accepted and treated as left to right and a detail with id `0x0FD5` is accepted (the PS4 library rejects both); an unknown writing form, another detail id, a null line pointer on creation and unknown handles throw instead of returning the PS4 error codes; `sceFontDestroyWritingLine` was not compared. Lines are allocated on the host, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. - [s_swappc_b64](../../core/shader/recompiler/Translation/src/ControlFlowInstructions.cpp) writes the link pair with the same model as `s_getpc_b64`: the guest return address `pc + 4` as a byte offset over `GetShaderBase`. Any other use of the link pair (arithmetic, compare, hash, store, or a second `s_swappc_b64` sharing it) throws; constant-offset folding over the link is not modeled. The special encoding `s_swappc_b64` with SDST 125 still decodes as `s_setpc_b64`, which matches the hardware: measured on RDNA2 (gfx1036) in wave32 and wave64, it branches to the source pair and leaves a marker SGPR pair untouched, and with an SGPR destination the link is `PC + 4`. - The [s_swappc_b64 fetch-shader call at vertex/local/tessellation/mesh entry](../../core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp) assumes the called fetch-shader blob ends in `s_setpc_b64` to the link pair (the GNM fetch-shader shape shadPS4's `EmitFetch` copies); the blob is not part of the shader program, so its return closes the pairing by assumption and any in-program reader of the link pair throws. Attribute fetches are inlined from the request's parsed fetch-shader plan through the same `GetAttribute` path as the embedded fetch analyzer, honoring the buffer resource `dst_sel`; an attribute writing more than four registers, or an entry call without a parsed plan, throws. The vertex/instance offset accumulator inside the fetch blob is not replayed for external blobs. - The scalar aperture sources [src_shared_base, src_shared_limit, src_private_base, src_private_limit](../../core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp) (core/shader/recompiler) decode to their own operand kinds (scalar source codes 235 to 238, named as in LLVM's `SIRegisterInfo.td`), but their value is not modelled: reading one refuses with the source it refused named, because the wave's shared and private aperture bases and limits are unknown. Both the scalar-code read and the raw-operand read refuse; a shader that only carries the value into an address still prepares, one that reads it does not From 25afd95d3598f3aa030d4efe88bb09fcb9ab5455 Mon Sep 17 00:00:00 2001 From: fma-rc <188743521+fma-rc@users.noreply.github.com> Date: Sun, 11 Oct 2026 05:55:25 +0200 Subject: [PATCH 342/460] fix(font): keep the step fields the library leaves alone and test multi-level closing --- core/libs/prx/libSceFont/src/WritingLine.cpp | 5 ++- core/libs/tests/GuestFontWritingLine.cpp | 38 ++++++++++++++++++++ docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 43 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceFont/src/WritingLine.cpp b/core/libs/prx/libSceFont/src/WritingLine.cpp index 81c06114e..5a91c6b27 100644 --- a/core/libs/prx/libSceFont/src/WritingLine.cpp +++ b/core/libs/prx/libSceFont/src/WritingLine.cpp @@ -281,7 +281,10 @@ const FontWritingLineStep* APS5_VABI sceFontWritingLineRefersRenderStep(void* wr ExtendRenderMetrics(*line); } else { line->steppedSpace = 0.0f; - line->step = {}; + line->step.x = 0.0f; + line->step.y = 0.0f; + line->step.spacingProgress = 0.0f; + line->step.Metrics = {}; line->renderMetrics = {}; } const bool rightToLeft = IsRightToLeft(*line); diff --git a/core/libs/tests/GuestFontWritingLine.cpp b/core/libs/tests/GuestFontWritingLine.cpp index 07542e3a7..4697f74ab 100644 --- a/core/libs/tests/GuestFontWritingLine.cpp +++ b/core/libs/tests/GuestFontWritingLine.cpp @@ -65,6 +65,20 @@ static bool StepIs(const FontWritingLineStep* step, float x, float progress, con step->Adjusting.y == 0.0f && Same(step->Metrics, run); } +static bool OrderIs(void* line, const std::uint64_t* levels, const std::size_t* order, std::size_t count) { + static int orderers[16]; + const FontWritingMetrics run{1.0f, 0.0f, {-1.0f, 1.0f, 0.0f, 1.0f}}; + if (sceFontWritingLineClear(line) != SCE_FONT_OK) return false; + for (std::size_t i = 0; i < count; ++i) { + if (sceFontWritingLineWritesOrder(line, Attribute(levels[i]), &run, &orderers[i]) != SCE_FONT_OK) return false; + } + for (std::size_t i = 0; i < count; ++i) { + const FontWritingLineStep* step = sceFontWritingLineRefersRenderStep(line); + if (!step || step->writingOrderer != &orderers[order[i]]) return false; + } + return sceFontWritingLineRefersRenderStep(line) == nullptr; +} + int main() { FontMemory memory{}; void* line = nullptr; @@ -180,6 +194,30 @@ int main() { Require(sceFontWritingLineRefersRenderStep(rtl) == nullptr); Require(sceFontWritingLineGetRenderMetrics(rtl, &metrics) == SCE_FONT_OK && Same(metrics, {18.0f, 0.0f, {-8.0f, 2.0f, 2.0f, 17.0f}})); + const std::uint64_t nested[] = {0, 1, 2, 3, 1, 1, 0}; + const std::size_t nestedOrder[] = {0, 5, 4, 2, 3, 1, 6}; + Require(OrderIs(ltr, nested, nestedOrder, 7)); + const std::uint64_t repeated[] = {0, 1, 1, 2, 2, 0, 1, 2, 2, 0}; + const std::size_t repeatedOrder[] = {0, 3, 4, 2, 1, 5, 7, 8, 6, 9}; + Require(OrderIs(ltr, repeated, repeatedOrder, 10)); + const std::uint64_t skipped[] = {0, 2, 3, 3, 2, 1, 0}; + const std::size_t skippedOrder[] = {0, 5, 1, 3, 2, 4, 6}; + Require(OrderIs(ltr, skipped, skippedOrder, 7)); + + const FontWritingMetrics runE{10.0f, 3.0f, {-8.0f, 2.0f, 1.0f, 9.0f}}; + const FontWritingMetrics runF{6.0f, -1.0f, {-12.0f, 3.0f, -1.0f, 5.0f}}; + Require(sceFontWritingLineClear(ltr) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(ltr, Attribute(0), &runE, &a) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(ltr, Attribute(0), &runF, &b) == SCE_FONT_OK); + auto* adjusted = const_cast(sceFontWritingLineRefersRenderStep(ltr)); + Require(adjusted && adjusted->x == 0.0f && adjusted->y == 0.0f && adjusted->writingOrderer == &a); + adjusted->Adjusting = {2.0f, 5.0f}; + const FontWritingLineStep* second = sceFontWritingLineRefersRenderStep(ltr); + Require(second && second->x == 10.0f && second->y == 3.0f && second->Adjusting.x == 0.0f && second->Adjusting.y == 0.0f); + Require(sceFontWritingLineGetRenderMetrics(ltr, &metrics) == SCE_FONT_OK && Same(metrics, {12.0f, 8.0f, {-3.0f, 7.0f, 3.0f, 11.0f}})); + Require(sceFontWritingLineRefersRenderStep(ltr) == nullptr); + Require(sceFontWritingLineGetRenderMetrics(ltr, &metrics) == SCE_FONT_OK && Same(metrics, {16.0f, 2.0f, {-9.0f, 7.0f, 3.0f, 15.0f}})); + float space = 7.0f; Require(sceFontWritingLineGetOrderingSpace(rtl, nullptr, nullptr, &space, nullptr) == SCE_FONT_OK && space == 15.0f); Require(Throws([&] { sceFontWritingLineGetOrderingSpace(rtl, &space, &space, &space, nullptr); })); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index b599c50aa..ad20b74a8 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -492,7 +492,7 @@ Throughout the project, every function at every stage either **does exactly what - [Thread exit destructors](../../core/libs/prx/libSceLibcInternal/Export.cpp) (libSceLibcInternal) run when a libkernel thread finishes; on the main or a host thread only `_sceLibcInternalThreadDtors` runs them, as `exit` does, else they are dropped - [sceFontOpenFontSet](../../core/libs/prx/libSceFont/src/SystemFont.cpp) (libSceFont) opens only the primary file of a system font set. Characters the console takes from the set's other files (Hangul, Thai and Arabic in Japanese and Chinese sets, Arabic in European sets) are reported as unsupported glyphs. The console file names are those of the PS4 firmware; the names of the PS5-only JG2 sets (`0x1A......`) are unknown, so they use the SSTJpPro files. When a console file is missing, an openly licensed substitute (Noto Sans, Noto Sans Mono, Noto Sans Thai, Noto Sans CJK) is loaded instead. It is not the system font: its metrics differ, so text width, line height and wrapping differ from the console. System font faces are loaded by FreeType in host memory, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. - [sceFontGetCharGlyphMetrics, sceFontGetRenderCharGlyphMetrics](../../core/libs/prx/libSceFont/src/Layout.cpp) (libSceFont) report the vertical metrics the way the [FreeType driver](../../core/libs/prx/libSceFontFt/src/Driver.cpp) reports them for a rendered glyph, not confirmed on a title: FreeType's vertical advance, and for a face without vertical metrics a bearing of half the horizontal advance left of the horizontal bearing and of half the advance left over by the glyph height. -- [Font writing lines](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - `sceFontWritingLineWritesOrder`, `sceFontWritingLineRefersRenderStep`, `sceFontWritingLineGetRenderMetrics`, `sceFontWritingLineGetOrderingSpace` and `sceFontWritingLineClear` follow the PS4 11.00 `libSceFont.sprx` (firmware module, read from its disassembly and compared call for call on random sequences): runs are linked in visual order by embedding level, the space at each spaced join is summed for the line, steps walk the visual order from the left (form `0x12`) or from the right (form `0x11`) and the render metrics grow as steps are consumed. The PS5 library was not measured. Not from that module: form `0x10` is accepted and treated as left to right and a detail with id `0x0FD5` is accepted (the PS4 library rejects both); an unknown writing form, another detail id, a null line pointer on creation and unknown handles throw instead of returning the PS4 error codes; `sceFontDestroyWritingLine` was not compared. Lines are allocated on the host, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. +- [Font writing lines](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - `sceFontWritingLineWritesOrder`, `sceFontWritingLineRefersRenderStep`, `sceFontWritingLineGetRenderMetrics`, `sceFontWritingLineGetOrderingSpace` and `sceFontWritingLineClear` follow the PS4 11.00 `libSceFont.sprx` (firmware module, read from its disassembly and compared call for call on random sequences): runs are linked in visual order by embedding level, the space at each spaced join is summed for the line, steps walk the visual order from the left (form `0x12`) or from the right (form `0x11`) and the render metrics grow as steps are consumed. The PS5 library was not measured. Not from that module: form `0x10` is accepted and treated as left to right and a detail with id `0x0FD5` is accepted (the PS4 library rejects both); the memory argument of `sceFontCreateWritingLine` and the rest of the detail are not read (the PS4 library validates its library object there); an unknown writing form, another detail id, a null line pointer on creation and unknown handles throw instead of returning the PS4 error codes; `sceFontDestroyWritingLine` was not compared. Lines are allocated on the host, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. - [s_swappc_b64](../../core/shader/recompiler/Translation/src/ControlFlowInstructions.cpp) writes the link pair with the same model as `s_getpc_b64`: the guest return address `pc + 4` as a byte offset over `GetShaderBase`. Any other use of the link pair (arithmetic, compare, hash, store, or a second `s_swappc_b64` sharing it) throws; constant-offset folding over the link is not modeled. The special encoding `s_swappc_b64` with SDST 125 still decodes as `s_setpc_b64`, which matches the hardware: measured on RDNA2 (gfx1036) in wave32 and wave64, it branches to the source pair and leaves a marker SGPR pair untouched, and with an SGPR destination the link is `PC + 4`. - The [s_swappc_b64 fetch-shader call at vertex/local/tessellation/mesh entry](../../core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp) assumes the called fetch-shader blob ends in `s_setpc_b64` to the link pair (the GNM fetch-shader shape shadPS4's `EmitFetch` copies); the blob is not part of the shader program, so its return closes the pairing by assumption and any in-program reader of the link pair throws. Attribute fetches are inlined from the request's parsed fetch-shader plan through the same `GetAttribute` path as the embedded fetch analyzer, honoring the buffer resource `dst_sel`; an attribute writing more than four registers, or an entry call without a parsed plan, throws. The vertex/instance offset accumulator inside the fetch blob is not replayed for external blobs. - The scalar aperture sources [src_shared_base, src_shared_limit, src_private_base, src_private_limit](../../core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp) (core/shader/recompiler) decode to their own operand kinds (scalar source codes 235 to 238, named as in LLVM's `SIRegisterInfo.td`), but their value is not modelled: reading one refuses with the source it refused named, because the wave's shared and private aperture bases and limits are unknown. Both the scalar-code read and the raw-operand read refuse; a shader that only carries the value into an address still prepares, one that reads it does not From 3dbcc5f1c71c78771a77bfb17cd83afda40f5f23 Mon Sep 17 00:00:00 2001 From: GalaxyGamingFR <153150716+GalaxyGamingFR@users.noreply.github.com> Date: Sat, 10 Oct 2026 21:21:07 -0700 Subject: [PATCH 343/460] fix(tools): keep apostrophes in comments from opening character literals Co-Authored-By: Claude Fable 5.1 --- tools/check_conventions.py | 3 ++- tools/tests/test_check_conventions_comments.py | 10 ++++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/tools/check_conventions.py b/tools/check_conventions.py index eb53a431d..8c3c27239 100644 --- a/tools/check_conventions.py +++ b/tools/check_conventions.py @@ -30,6 +30,7 @@ AGENT_FILES = {"AGENTS.md", "CLAUDE.md", "GEMINI.md", ".cursorrules", "copilot-i CONVENTIONAL = re.compile(r"^(feat|fix|docs|style|refactor|perf|test|build|ci|chore|revert)(\([^)]+\))?!?: \S") STRINGS = re.compile(r'"(?:\\.|[^"\\])*"|\'(?:\\.|[^\'\\])*\'') +TOKENS = re.compile(r'"(?:\\.|[^"\\\n])*"|\'(?:\\.|[^\'\\\n])*\'|//[^\n]*|/\*.*?\*/', re.S) ALLOWED_COMMENT = re.compile(r"^\s*(#\s*endif\b|}\s*;?\s*//\s*(end\s+)?namespace\b)") EXPORT = re.compile(r"\bAPS5_VABI\s+(\w+)\s*\(") STUB_BODY = re.compile(r"^(?:\(void\)\s*\w+\s*;|static_cast\(\s*\w+\s*\)\s*;)*" @@ -216,7 +217,7 @@ def allowed_notes(path): def function_body(lines, start): text = "\n".join(lines[start:start + 400]) - text = re.sub(r"//[^\r\n]*|/\*.*?\*/", "", STRINGS.sub('""', text), flags=re.S) + text = TOKENS.sub(lambda match: "" if match.group(0).startswith("/") else '""', text) brace, semicolon = text.find("{"), text.find(";") if brace < 0 or 0 <= semicolon < brace: return None diff --git a/tools/tests/test_check_conventions_comments.py b/tools/tests/test_check_conventions_comments.py index 8f577a370..c4af4a8a0 100644 --- a/tools/tests/test_check_conventions_comments.py +++ b/tools/tests/test_check_conventions_comments.py @@ -37,6 +37,16 @@ class FunctionBodyTests(unittest.TestCase): source = 'int APS5_VABI sceExample() {\nuse("// }", "/* { */");\nreturn 0;\n}' self.assertEqual(function_body(source.splitlines(), 0), 'use("","");return0;') + def test_apostrophes_in_comments_do_not_open_character_literals(self): + source = "int APS5_VABI sceExample() {\n// don't fail here\nvalidate();\n// it's fine\nreturn 0;\n}" + body = function_body(source.splitlines(), 0) + self.assertEqual(body, "validate();return0;") + self.assertIsNone(STUB_BODY.fullmatch(body)) + source = "int APS5_VABI sceExample() {\n/* don't */\nreturn 0;\n}\nstatic const char Separator = '/';" + self.assertEqual(function_body(source.splitlines(), 0), "return0;") + source = "int APS5_VABI sceExample() {\nuse(\"http://x\"); // isn't a stub\nreturn 0;\n}" + self.assertEqual(function_body(source.splitlines(), 0), 'use("");return0;') + def test_declaration_has_no_body(self): source = "int APS5_VABI sceExample();\nint APS5_VABI sceOther() { return 0; }" self.assertIsNone(function_body(source.splitlines(), 0)) From d8172fb7ddb43d9715c73e064264ca3d3c3574f2 Mon Sep 17 00:00:00 2001 From: Abdulrahman Elkordy <196347165+Silverash-i8@users.noreply.github.com> Date: Sun, 11 Oct 2026 04:01:16 +0300 Subject: [PATCH 344/460] test(agc-driver): give the shared-code header draw a [0, 1] depth range SharedCodeHeaders draws on the initial context, whose PA_CL_CLIP_CNTL selects the [-1, 1] clip space with viewport Z scale 1 and offset 0, so the draw asks for a viewport depth range of [-1, 1]. Devices without VK_EXT_depth_range_unrestricted, such as Intel GPUs on Windows, reject that and the test fails before it reaches registration. Set DX_CLIP_SPACE_DEF as the other draw tests do. (cherry picked from commit 3c3bcf8f80cceaba4aa1f8742c32251fcc143e6b) --- .../prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp index 6513746f8..93dffc420 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp @@ -388,7 +388,7 @@ void SharedCodeHeaders() { std::vector commands; for (const auto reg : vertex.registers) commands.insert(commands.end(), {0xc0017600u, reg.offset, reg.value}); for (const auto reg : vertex.context) commands.insert(commands.end(), {0xc0016900u, reg.offset, reg.value}); - commands.insert(commands.end(), {0xc0017600u, 0x8, 0, 0xc0016900u, 0xd, 0x00100010u, 0xc0016900u, 0x1c3, 4, 0xc0017900u, 0x242, 4, 0xc0012d00u, 3, 2}); + commands.insert(commands.end(), {0xc0017600u, 0x8, 0, 0xc0016900u, 0xd, 0x00100010u, 0xc0016900u, 0x204, 0x80000u, 0xc0016900u, 0x1c3, 4, 0xc0017900u, 0x242, 4, 0xc0012d00u, 3, 2}); Packet packet{commands.data(), static_cast(commands.size()), 0, {}}; for (unsigned iteration = 0; iteration < 2; ++iteration) { Require(sceAgcDriverSubmitDcb(&packet) == 0, "shared-code draw submission failed"); From dd9c78aa1bf91095971e758a3d44196529194acf Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Sun, 11 Oct 2026 13:24:35 +0900 Subject: [PATCH 345/460] fix(tools): stop collecting NIDs at abbreviated options and --nid= Resolve each argument after --nid against the parser's option strings the way argparse does, so --nid=B, abbreviations such as --j and --d=, and the -- marker end the NID list instead of being taken as NIDs. Co-Authored-By: Claude Opus 5.5 --- tools/nid_names.py | 30 ++++++++++++++++++++++++------ tools/tests/test_nid_names.py | 20 ++++++++++++++++++-- 2 files changed, 42 insertions(+), 8 deletions(-) diff --git a/tools/nid_names.py b/tools/nid_names.py index 72a60fe97..35ee96582 100644 --- a/tools/nid_names.py +++ b/tools/nid_names.py @@ -85,15 +85,32 @@ def collect_real_names(): return names -def nid_arguments(arguments): - options = {"--db", "--json", "--nid", "-h", "--help"} +def option_name(argument, options): + if not argument.startswith("-") or argument == "-": + return None + name = argument.split("=", 1)[0] + if name in options: + return name + if not name.startswith("--"): + return None + matches = [option for option in options if option.startswith(name)] + if len(matches) == 1: + return matches[0] + return name if matches else None + + +def nid_arguments(arguments, options): normalized = [] collecting = False - for argument in arguments: - if argument == "--nid": + for index, argument in enumerate(arguments): + if argument == "--": + normalized.extend(arguments[index:]) + break + name = option_name(argument, options) + if name == "--nid" and "=" not in argument: collecting = True continue - if collecting and argument not in options and not argument.startswith("--db="): + if collecting and name is None: normalized.append("--nid=" + argument) continue collecting = False @@ -110,7 +127,8 @@ def main(): help="resolve only these NIDs instead of scanning the tree") parser.add_argument("--json", action="store_true", help="emit machine-readable JSON") - args = parser.parse_args(nid_arguments(sys.argv[1:])) + options = [option for action in parser._actions for option in action.option_strings] + args = parser.parse_args(nid_arguments(sys.argv[1:], options)) db_path = Path(args.db) if args.db else DEFAULT_CACHE db = load_db(db_path) diff --git a/tools/tests/test_nid_names.py b/tools/tests/test_nid_names.py index 5f50053e0..1c7453e93 100644 --- a/tools/tests/test_nid_names.py +++ b/tools/tests/test_nid_names.py @@ -69,10 +69,26 @@ class DatabaseCacheTests(unittest.TestCase): download.assert_not_called() class NidArgumentTests(unittest.TestCase): + options = ["-h", "--help", "--db", "--nid", "--json"] + + def normalize(self, arguments): + return nid_names.nid_arguments(arguments, self.options) + def test_nids_starting_with_a_dash_are_accepted(self): self.assertEqual( - nid_names.nid_arguments(["--nid", "-nvxBWa0iDs", "ZjtRqSMJwdw", "--json"]), - ["--nid=-nvxBWa0iDs", "--nid=ZjtRqSMJwdw", "--json"]) + self.normalize(["--nid", "-nvxBWa0iDs", "ZjtRqSMJwdw", "--ll-AgiK2M", "--json"]), + ["--nid=-nvxBWa0iDs", "--nid=ZjtRqSMJwdw", "--nid=--ll-AgiK2M", "--json"]) + + def test_nid_with_value_after_a_bare_nid_is_kept(self): + self.assertEqual(self.normalize(["--nid", "A", "--nid=B"]), ["--nid=A", "--nid=B"]) + + def test_abbreviated_options_end_the_nids(self): + self.assertEqual(self.normalize(["--nid", "A", "--j"]), ["--nid=A", "--j"]) + self.assertEqual(self.normalize(["--nid", "A", "--d=x.csv"]), ["--nid=A", "--d=x.csv"]) + self.assertEqual(self.normalize(["--ni", "-nvxBWa0iDs"]), ["--nid=-nvxBWa0iDs"]) + + def test_end_of_options_marker_is_kept(self): + self.assertEqual(self.normalize(["--nid", "A", "--", "B"]), ["--nid=A", "--", "B"]) def test_main_resolves_every_nid_given_after_nid(self): database = {"-nvxBWa0iDs": "gethostname", "ZjtRqSMJwdw": "sinh"} From 4d82201f72621ef2519f4acad027496cb2ba2232 Mon Sep 17 00:00:00 2001 From: Abdulrahman Elkordy <196347165+Silverash-i8@users.noreply.github.com> Date: Sun, 11 Oct 2026 04:44:29 +0000 Subject: [PATCH 346/460] test(agc): skip fp16 overflow without shaderFloat64 The fp16 overflow shader translates to 64-bit float instructions, so a device without shaderFloat64 cannot run this test. Skip before recompiling on unsupported devices. Carried from issue #3230, commit 922d4015e05c34f616912e1913854007a5d689bf; original author preserved. --- .../libs/prx/libSceAgcDriver/tests/execution/Fp16Overflow.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/Fp16Overflow.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/Fp16Overflow.cpp index 880b0b65b..f83aa30c3 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/Fp16Overflow.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/Fp16Overflow.cpp @@ -226,6 +226,10 @@ int main() { try { const auto device = OpenVulkanTestDevice(); if (!device) return VulkanTestSkipped; + if (!TargetHasCapability(device->Target(), spv::CapabilityFloat64)) { + std::puts("skipped, the device has no shaderFloat64"); + return VulkanTestSkipped; + } Run(*device, std::nullopt); Check(ExpectedInfinity, "no float mode"); Run(*device, ShaderRecompiler::ShaderFloatMode{0xc0u, true, false, false}); From 4bc36b19b8f8d584c603ba5d1ac7a04a3432cf45 Mon Sep 17 00:00:00 2001 From: ArjenS-LA Date: Sun, 11 Oct 2026 04:48:55 +0000 Subject: [PATCH 347/460] fix(relinker): read GNU hash symbols during code analysis When the dynamic symbol count is absent from the SCE or SysV hash metadata, derive it from the GNU hash buckets and chain. Keep all table reads within mapped file-backed ranges. Carried from issue #3224, commit 1a10126089882a2306318f18d19d05b04a764d6c; original author preserved. --- core/relinker/CMakeLists.txt | 2 + .../src/analysis/CodeInstructionCollector.cpp | 33 ++++- .../relinker/tests/test_gnu_hash_analysis.py | 126 ++++++++++++++++++ 3 files changed, 160 insertions(+), 1 deletion(-) create mode 100644 core/relinker/relinker/tests/test_gnu_hash_analysis.py diff --git a/core/relinker/CMakeLists.txt b/core/relinker/CMakeLists.txt index ada19f309..1ebdec341 100644 --- a/core/relinker/CMakeLists.txt +++ b/core/relinker/CMakeLists.txt @@ -332,6 +332,8 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_entry_point_bounds.py $) add_test(NAME init_array_roots COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_init_array_roots.py $) + add_test(NAME gnu_hash_analysis COMMAND ${Python3_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_gnu_hash_analysis.py $) add_test(NAME to_rosetta COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_to_rosetta.py $) add_test(NAME syscall_number COMMAND ${Python3_EXECUTABLE} diff --git a/core/relinker/relinker/src/analysis/CodeInstructionCollector.cpp b/core/relinker/relinker/src/analysis/CodeInstructionCollector.cpp index 540ef359a..3d78157a2 100644 --- a/core/relinker/relinker/src/analysis/CodeInstructionCollector.cpp +++ b/core/relinker/relinker/src/analysis/CodeInstructionCollector.cpp @@ -65,7 +65,7 @@ std::set CodeInstructionCollector::Collect(const std::ve const auto tag = Io::ReadU64(bytes, offset); if (tag == 0) { terminated = true; break; } const auto value = Io::ReadU64(bytes, offset + 8); - if (tag == 1 || (tag >= 0x61000000 && tag != 0x6100003f && (tag < 0x61000027 || tag > 0x6100003b))) continue; + if (tag == 1 || (tag >= 0x61000000 && tag != 0x6ffffef5 && tag != 0x6100003f && (tag < 0x61000027 || tag > 0x6100003b))) continue; if (!tags.emplace(tag, value).second) throw Domain::RelinkerException("Code analysis: duplicate dynamic tag", offset); } if (!terminated) throw Domain::RelinkerException("Code analysis: unterminated dynamic segment"); @@ -122,6 +122,37 @@ std::set CodeInstructionCollector::Collect(const std::ve if (*symbolBytes % 24 != 0) throw Domain::RelinkerException("Code analysis: invalid symbol table size"); symbolCount = *symbolBytes / 24; } else if (tags.contains(4)) symbolCount = Io::ReadU32(bytes, fileOffset(tags.at(4), 8) + 4); + else if (tags.contains(0x6ffffef5)) { + const auto address = tags.at(0x6ffffef5); + const auto header = fileOffset(address, 16); + const auto bucketCount = Io::ReadU32(bytes, header); + const auto firstSymbol = Io::ReadU32(bytes, header + 4); + const auto bloomCount = Io::ReadU32(bytes, header + 8); + if (bucketCount == 0 || bloomCount == 0 || firstSymbol == 0) + throw Domain::RelinkerException("Code analysis: invalid GNU hash header", address); + const std::uint64_t bucketOffset = 16 + static_cast(bloomCount) * 8; + const auto chainOffset = bucketOffset + static_cast(bucketCount) * 4; + const auto start = fileOffset(address, chainOffset); + std::uint64_t lastBucket = 0; + for (std::uint64_t index = 0; index < bucketCount; ++index) { + const auto bucket = Io::ReadU32(bytes, start + bucketOffset + index * 4); + if (bucket != 0 && bucket < firstSymbol) + throw Domain::RelinkerException("Code analysis: invalid GNU hash bucket", address); + lastBucket = std::max(lastBucket, static_cast(bucket)); + } + symbolCount = firstSymbol; + if (lastBucket != 0) { + symbolCount = lastBucket; + for (;;) { + const auto offset = chainOffset + (symbolCount - firstSymbol) * 4; + if (offset > std::numeric_limits::max() - address) + throw Domain::RelinkerException("Code analysis: invalid GNU hash chain", address); + const auto chain = Io::ReadU32(bytes, fileOffset(address + offset, 4)); + ++symbolCount; + if (chain & 1) break; + } + } + } std::uint64_t symbols = 0; if (symbolCount != 0) { if (value(11, 0x6100003b) != 24) throw Domain::RelinkerException("Code analysis: invalid symbol entry size"); diff --git a/core/relinker/relinker/tests/test_gnu_hash_analysis.py b/core/relinker/relinker/tests/test_gnu_hash_analysis.py new file mode 100644 index 000000000..01354a669 --- /dev/null +++ b/core/relinker/relinker/tests/test_gnu_hash_analysis.py @@ -0,0 +1,126 @@ +from pathlib import Path +import struct +import subprocess +import sys +import tempfile + +from test_jmprel_table_bounds import fixture, DT_OS_SYMTABSZ, DT_STRTAB, DT_STRSZ + + +DT_GNU_HASH = 0x6FFFFEF5 +HASH = 0x800 +SYMBOLS = 0x620 +STRINGS = 0x900 + + +def name_hash(name): + value = 5381 + for byte in name.encode(): + value = (value * 33 + byte) & 0xFFFFFFFF + return value + + +def image(names=("alpha", "bravo", "charlie"), bucket_count=3): + elf = fixture(os_tags=False) + struct.pack_into("> 5) % 64)) + if chains: + chains[-1] |= 1 + struct.pack_into(" Date: Sun, 11 Oct 2026 04:53:40 +0000 Subject: [PATCH 348/460] test(libkernel): report native filesystem errors --- core/libs/tests/GuestFilesystem.cpp | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index d84df7903..5a36e6c9c 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -2,6 +2,7 @@ #include "prx/libc/include/general/VabiMacros.hpp" #include #include +#include #include #include #include @@ -52,7 +53,19 @@ int APS5_VABI sceKernelDebugOutText(int, const char*); } static void Check(bool value, int line) { if (!value) { - std::fprintf(stderr, "Filesystem check failed at line %d (guest errno %d)\n", line, *__error_nid_postfix()); +#ifdef _WIN32 + const auto nativeError = ::GetLastError(); +#else + const auto nativeError = errno; +#endif + const auto guestError = *__error_nid_postfix(); +#ifdef _WIN32 + std::fprintf(stderr, "Filesystem check failed at line %d (guest errno %d, Win32 error %lu)\n", + line, guestError, static_cast(nativeError)); +#else + std::fprintf(stderr, "Filesystem check failed at line %d (guest errno %d, host errno %d)\n", + line, guestError, nativeError); +#endif std::abort(); } } From c2a48416b1c103d4c129baaa2659cf071e38d9ed Mon Sep 17 00:00:00 2001 From: AlxcBTW <187763517+AlxcBTW@users.noreply.github.com> Date: Sat, 10 Oct 2026 18:04:48 +0200 Subject: [PATCH 349/460] fix(shader): follow exec phis and long kill chains when proving masked lanes unread Taxi Chaos 2 (PPSA30279) pixel shader 0x18c07bf000 is refused when it is registered, and the title stops: packed pixel ancillary input has an unsupported live use (used by SelectU32 ? ?) The shader reads the ancillary VGPR (v2) once, with v_bfe_u32 v3, v2, 16, 11, which #1904 lowers to Layer. The next write to v2, v_rcp_f32 v2, s33, runs before any kill under the initial exec and becomes SelectU32(IEqual32(HelperInvocation, 0), rcp, ancillary). The raw word therefore survives only in helper lanes. The shader has no s_wqm, no DPP, readlane or swizzle, no stores, and only *_lz samples and gathers. Every helper-lane value that comes from v2 is dropped by a later exec select, a compare that is ANDed with exec before its ballot, or an export whose exec excludes helper lanes. MaskedSelectEliminator exists to remove selects like this one, but it could not prove the case. The kill paths save and restore exec, so the later execs are phis at block merges, and the AND chains the kills build go deeper than its eight-level limit. When it checks whether a later exec implies the write's exec, it now: - accepts a phi when the phi and each incoming value lie outside loops and every incoming value implies that exec; - pairs low and high exec words merged by phis in one block, predecessor by predecessor; - memoises results for each select and follows up to 64 levels. A cycle counts as not implied. Once the select is gone, nothing reads the ancillary word, so no bit of it is produced. A select whose false lanes can still be observed is kept and the shader is still refused. That covers a data test that leaves the word in non-helper lanes that later export, an export under the full wave, and an unmasked ballot. Related: #2548 Co-Authored-By: Claude Opus 5.5 --- .../libSceAgcDriver/tests/MaskedSelect.cpp | 110 +++++++++++++++ core/libs/tests/PixelAncillary.cpp | 89 ++++++++++++ .../src/MaskedSelectEliminator.cpp | 127 ++++++++++++++---- 3 files changed, 297 insertions(+), 29 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp b/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp index 119f87925..ea19428f1 100644 --- a/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp @@ -275,6 +275,116 @@ int main() { return count == 0u && !gone; }); + const auto mergedExec = [](bool wideBranch, bool loopMerge) { + Builder b; + auto& wide = b.mask(48u); + auto& exec = b.logicalAnd(wide, b.mask(32u)); + auto& old = b.lane(); + auto& written = b.select(exec, b.add(old, 1u), old); + IrBlock& head = *b.block; + IrBlock& body = b.newBlock(); + IrBlock& merge = b.newBlock(); + head.AddBranch(&body); + head.AddBranch(&merge); + body.AddBranch(&merge); + b.block = &body; + auto& killed = b.logicalAnd(wideBranch ? wide : exec, b.mask(16u)); + auto& phi = b.program.CreateValue(IrOpcode::Phi, IrType::Bool); + merge.AppendInstruction(&phi); + phi.AddPhiOperand(&head, &exec); + phi.AddPhiOperand(&body, &killed); + if (loopMerge) { + IrBlock& exit = b.newBlock(); + merge.AddBranch(&merge); + merge.AddBranch(&exit); + phi.AddPhiOperand(&merge, &phi); + b.block = &exit; + } else { + b.block = &merge; + } + b.keep(b.select(phi, b.add(written, 2u), old)); + return std::pair{b.eliminate(), removed(written)}; + }; + + passed &= run("an exec merged by a phi whose every incoming exec implies the write's exec drops the select", [&] { + const auto [count, gone] = mergedExec(false, false); + return count == 1u && gone; + }); + + passed &= run("an exec merged by a phi with a wider incoming exec keeps the select", [&] { + const auto [count, gone] = mergedExec(true, false); + return count == 0u && !gone; + }); + + passed &= run("an exec merged by a phi inside a loop keeps the select", [&] { + const auto [count, gone] = mergedExec(false, true); + return count == 0u && !gone; + }); + + const auto mergedWords = [](bool wideBranch) { + Builder b; + auto& wide = b.mask(48u); + auto& exec = b.logicalAnd(wide, b.mask(40u)); + auto& old = b.lane(); + auto& written = b.select(exec, b.add(old, 1u), old); + auto& saved = b.emit(IrOpcode::Ballot, IrType::U32x4, {&exec}); + auto& savedLow = b.emit(IrOpcode::CompositeExtractU32x4, IrType::U32, {&saved, &b.constant(0u)}); + auto& savedHigh = b.emit(IrOpcode::CompositeExtractU32x4, IrType::U32, {&saved, &b.constant(1u)}); + IrBlock& head = *b.block; + IrBlock& body = b.newBlock(); + IrBlock& merge = b.newBlock(); + head.AddBranch(&body); + head.AddBranch(&merge); + body.AddBranch(&merge); + b.block = &body; + auto& survived = b.emit(IrOpcode::Ballot, IrType::U32x4, {&b.logicalAnd(wideBranch ? wide : exec, b.mask(24u))}); + auto& survivedLow = b.emit(IrOpcode::CompositeExtractU32x4, IrType::U32, {&survived, &b.constant(0u)}); + auto& survivedHigh = b.emit(IrOpcode::CompositeExtractU32x4, IrType::U32, {&survived, &b.constant(1u)}); + auto& low = b.program.CreateValue(IrOpcode::Phi, IrType::U32); + auto& high = b.program.CreateValue(IrOpcode::Phi, IrType::U32); + merge.AppendInstruction(&low); + merge.AppendInstruction(&high); + low.AddPhiOperand(&head, &savedLow); + low.AddPhiOperand(&body, &survivedLow); + high.AddPhiOperand(&head, &savedHigh); + high.AddPhiOperand(&body, &survivedHigh); + b.block = &merge; + b.keep(b.select(b.threadBit(low, high), b.add(written, 2u), old)); + return std::pair{b.eliminate(), removed(written)}; + }; + + passed &= run("an exec rebuilt from mask words merged by phis implies the write's exec", [&] { + const auto [count, gone] = mergedWords(false); + return count == 1u && gone; + }); + + passed &= run("an exec rebuilt from merged mask words with a wider incoming ballot keeps the select", [&] { + const auto [count, gone] = mergedWords(true); + return count == 0u && !gone; + }); + + const auto chained = [](bool wideRoot) { + Builder b; + auto& wide = b.mask(48u); + auto& exec = b.logicalAnd(wide, b.mask(32u)); + auto& old = b.lane(); + auto& written = b.select(exec, b.add(old, 1u), old); + IrValue* later = wideRoot ? &wide : &exec; + for (std::uint32_t kill = 0; kill < 24u; ++kill) later = &b.logicalAnd(*later, b.mask(31u - kill)); + b.keep(b.select(*later, b.add(written, 2u), old)); + return std::pair{b.eliminate(), removed(written)}; + }; + + passed &= run("an exec narrowed by a long chain of kills still implies the write's exec", [&] { + const auto [count, gone] = chained(false); + return count == 1u && gone; + }); + + passed &= run("a long chain of kills rooted at a wider exec keeps the select", [&] { + const auto [count, gone] = chained(true); + return count == 0u && !gone; + }); + passed &= run("a write in a loop must keep its select", [] { Builder b; auto& exec = b.mask(16u); diff --git a/core/libs/tests/PixelAncillary.cpp b/core/libs/tests/PixelAncillary.cpp index b7be6fc81..2100713ec 100644 --- a/core/libs/tests/PixelAncillary.cpp +++ b/core/libs/tests/PixelAncillary.cpp @@ -1,6 +1,7 @@ #include "IntermediateRepresentation/IrBuilder.hpp" #include "Optimization/ConstantFolder.hpp" #include "Optimization/DeadCodeEliminator.hpp" +#include "Optimization/MaskedSelectEliminator.hpp" #include "Optimization/ShaderInfoCollector.hpp" #include #include @@ -253,6 +254,90 @@ static void MergedRefused(IrOpcode opcode, std::uint32_t offset, std::uint32_t c } Require(false); } +enum class HelperRead { Export, FullWaveExport, DataTestExport, Ballot }; + +struct HelperProgram { + IrValue* written = nullptr; + IrValue* layer = nullptr; + IrValue* exported = nullptr; +}; + +static HelperProgram HelperOnly(IrProgram& program, HelperRead read) { + auto& entry = Begin(program); + auto& body = program.CreateBlock(); + auto& merge = program.CreateBlock(); + program.BlockOrder().push_back(&body); + program.BlockOrder().push_back(&merge); + entry.AddBranch(&body); + entry.AddBranch(&merge); + body.AddBranch(&merge); + IrBuilder builder(program); + builder.SetInsertionPoint(entry); + IrValue& helper = builder.Emit(IrOpcode::GetBuiltin, IrType::U32, {&builder.Constant(static_cast(StageInputKind::HelperInvocation)), &builder.Constant(0u)}); + IrValue& live = builder.IEqual(helper, builder.Constant(0u)); + IrValue& ancillary = AncillaryBuiltin(builder); + IrValue& layer = builder.Emit(IrOpcode::BitFieldUExtract, IrType::U32, {&ancillary, &builder.Constant(16u), &builder.Constant(11u)}); + IrValue& target = builder.Select(live, layer, builder.Constant(0u)); + IrValue& tested = builder.LogicalAnd(live, FrontFacingBuiltin(builder)); + IrValue& written = builder.Select(read == HelperRead::DataTestExport ? tested : live, builder.Constant(SelectOther), ancillary); + builder.SetInsertionPoint(body); + IrValue& squared = builder.Emit(IrOpcode::FPMul32, IrType::F32, {&builder.BitCastF32(written), &builder.BitCastF32(written)}); + IrValue& killed = builder.Select(tested, builder.BitCastU32(squared), written); + IrValue& exec = program.CreateValue(IrOpcode::Phi, IrType::Bool); + IrValue& value = program.CreateValue(IrOpcode::Phi, IrType::U32); + merge.AppendInstruction(&exec); + merge.AppendInstruction(&value); + exec.AddPhiOperand(&entry, &live); + exec.AddPhiOperand(&body, &tested); + value.AddPhiOperand(&entry, &written); + value.AddPhiOperand(&body, &killed); + builder.SetInsertionPoint(merge); + IrValue& data = builder.Emit(IrOpcode::CompositeConstructU32x4, IrType::U32x4, {&value, &target, &builder.Constant(0u), &builder.Constant(0u)}); + program.Metadata().exportInfo.push_back(ExportInfo {.kind = ExportTargetKind::Mrt, .index = 0u, .en = 0xFu, .done = true, .vm = true}); + IrValue& guard = read == HelperRead::FullWaveExport ? builder.ConstantBool(true) : exec; + static_cast(builder.Emit(IrOpcode::SetAttribute, IrType::Void, {&data, &guard}, ExportFlags {0u, 0u})); + if (read == HelperRead::Ballot) { + IrValue& ballot = builder.Emit(IrOpcode::Ballot, IrType::U32x4, {&builder.INotEqual(value, builder.Constant(0u))}); + static_cast(builder.Emit(IrOpcode::ReferenceU32, IrType::Void, {&builder.CompositeExtract(ballot, 0u)})); + } + static_cast(builder.Emit(IrOpcode::Return, IrType::Void, {})); + return HelperProgram {&written, &layer, &value}; +} +static void LowerMasked(IrProgram& program) { + ConstantFolder().Fold(program); + DeadCodeEliminator().RemoveIdentities(program); + DeadCodeEliminator().Eliminate(program); + static_cast(MaskedSelectEliminator().Eliminate(program)); + DeadCodeEliminator().Eliminate(program); + const ShaderPixelInputInfo pixel {}; + ShaderInfoCollector().Collect(program, ShaderStageInputInfo {nullptr, &pixel, nullptr}); +} +static void HelperLanesOnly() { + IrProgram program; + const HelperProgram built = HelperOnly(program, HelperRead::Export); + LowerMasked(program); + for (const IrBlock* block : program.BlockOrder()) { + for (const IrValue* inst : block->Instructions()) { + Require(inst->Opcode() != IrOpcode::GetBuiltin || static_cast(inst->Argument(0)->Resolve()->ImmediateU32()) != StageInputKind::PackedAncillary); + } + } + Require(built.written->Parent() == nullptr); + const IrValue* field = built.layer->Argument(0)->Resolve(); + Require(field->Opcode() == IrOpcode::GetBuiltin && static_cast(field->Argument(0)->Resolve()->ImmediateU32()) == StageInputKind::Layer); + const IrValue* entryValue = built.exported->Argument(0)->Resolve(); + Require(entryValue->HasImmediate() && entryValue->ImmediateU32() == SelectOther); +} +static void HelperLanesRefused(HelperRead read) { + IrProgram program; + static_cast(HelperOnly(program, read)); + try { + LowerMasked(program); + } catch (const std::runtime_error& error) { + Require(std::string(error.what()).find("unsupported live use") != std::string::npos); + return; + } + Require(false); +} int main() { Extract(IrOpcode::BitFieldUExtract, 8u, 4u, StageInputKind::SampleId, 0u); Extract(IrOpcode::BitFieldUExtract, 9u, 2u, StageInputKind::SampleId, 1u); @@ -292,4 +377,8 @@ int main() { MergedRefused(IrOpcode::BitwiseOr32, 1u, 0u, false); MergedRefused(IrOpcode::BitFieldUExtract, 12u, 4u, false); MergedRefused(IrOpcode::BitFieldUExtract, 0u, 2u, true); + HelperLanesOnly(); + HelperLanesRefused(HelperRead::FullWaveExport); + HelperLanesRefused(HelperRead::DataTestExport); + HelperLanesRefused(HelperRead::Ballot); } diff --git a/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp b/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp index 74f4cda6f..1966437f1 100644 --- a/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp +++ b/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp @@ -266,36 +266,105 @@ bool alwaysTrue(const IrValue* value, std::uint32_t waveSize, std::uint32_t dept return bit && isImmediate(bit->low, 0xffffffffu) && isImmediate(bit->high, 0xffffffffu); } -bool implies(const IrValue* condition, const IrValue* mask, std::uint32_t waveSize, std::uint32_t depth = 0); +struct ImplicationKey { + const IrValue* first = nullptr; + const IrValue* second = nullptr; -bool wordsImply(const IrValue* low, const IrValue* high, const IrValue* mask, std::uint32_t waveSize, std::uint32_t depth) { - if (depth > DepthLimit) return false; - if (isImmediate(low, 0u) && (waveSize == 32u || isImmediate(high, 0u))) return true; - if (low->Opcode() == IrOpcode::CompositeExtractU32x4 && isImmediate(low->Argument(1)->Resolve(), 0u)) { - const IrValue* ballot = low->Argument(0)->Resolve(); - const bool sameBallot = waveSize == 32u || (high->Opcode() == IrOpcode::CompositeExtractU32x4 && high->Argument(0)->Resolve() == ballot && isImmediate(high->Argument(1)->Resolve(), 1u)); - if (ballot->Opcode() == IrOpcode::Ballot && sameBallot && implies(ballot->Argument(0), mask, waveSize, depth + 1u)) return true; + bool operator==(const ImplicationKey&) const = default; +}; + +struct ImplicationKeyHash { + std::size_t operator()(const ImplicationKey& key) const { + return std::hash{}(key.first) * 31u ^ std::hash{}(key.second); } - if (low->Opcode() != IrOpcode::BitwiseAnd32) return false; - if (waveSize == 32u) return wordsImply(low->Argument(0)->Resolve(), high, mask, waveSize, depth + 1u) || wordsImply(low->Argument(1)->Resolve(), high, mask, waveSize, depth + 1u); - if (high->Opcode() != IrOpcode::BitwiseAnd32) return false; - for (std::size_t lowIndex = 0; lowIndex < 2u; ++lowIndex) { - for (std::size_t highIndex = 0; highIndex < 2u; ++highIndex) { - if (wordsImply(low->Argument(lowIndex)->Resolve(), high->Argument(highIndex)->Resolve(), mask, waveSize, depth + 1u)) return true; +}; + +class Implications { +public: + Implications(const std::unordered_set& once, const IrValue* mask, std::uint32_t waveSize) : once(once), mask(mask), waveSize(waveSize), always(alwaysTrue(mask, waveSize)) {} + + bool Implies(const IrValue* condition, std::uint32_t depth = 0) { + condition = condition->Resolve(); + if (condition == mask || always) return true; + if (condition->HasImmediate()) return condition->Type() == IrType::Bool && !condition->ImmediateBool(); + const ImplicationKey key{condition, nullptr}; + if (const auto known = results.find(key); known != results.end()) return known->second; + if (depth > ImplicationDepthLimit) return false; + results.emplace(key, false); + bool result = false; + if (condition->Opcode() == IrOpcode::LogicalAnd) { + result = Implies(condition->Argument(0), depth + 1u) || Implies(condition->Argument(1), depth + 1u); + } else if (condition->IsPhi()) { + result = mergesOutsideLoops(*condition); + for (std::size_t index = 0; result && index < condition->ArgumentCount(); ++index) { + result = Implies(condition->Argument(index), depth + 1u); + } + } else if (const auto bit = threadBit(condition, waveSize)) { + result = wordsImply(bit->low, bit->high, depth + 1u); } + results[key] = result; + return result; } - return false; -} -bool implies(const IrValue* condition, const IrValue* mask, std::uint32_t waveSize, std::uint32_t depth) { - condition = condition->Resolve(); - if (condition == mask || alwaysTrue(mask, waveSize)) return true; - if (condition->HasImmediate()) return condition->Type() == IrType::Bool && !condition->ImmediateBool(); - if (depth > DepthLimit) return false; - if (condition->Opcode() == IrOpcode::LogicalAnd) return implies(condition->Argument(0), mask, waveSize, depth + 1u) || implies(condition->Argument(1), mask, waveSize, depth + 1u); - const auto bit = threadBit(condition, waveSize); - return bit && wordsImply(bit->low, bit->high, mask, waveSize, depth + 1u); -} +private: + static constexpr std::uint32_t ImplicationDepthLimit = 64; + + bool mergesOutsideLoops(const IrValue& phi) const { + if (phi.Parent() == nullptr || !once.contains(phi.Parent()) || phi.ArgumentCount() != phi.PhiBlockCount()) return false; + for (std::size_t index = 0; index < phi.ArgumentCount(); ++index) { + const IrValue* incoming = phi.Argument(index)->Resolve(); + if (!incoming->HasImmediate() && (incoming->Parent() == nullptr || !once.contains(incoming->Parent()))) return false; + } + return true; + } + + static const IrValue* incomingFrom(const IrValue& phi, const IrBlock* predecessor) { + for (std::size_t index = 0; index < phi.PhiBlockCount(); ++index) { + if (phi.PhiBlock(index) == predecessor) return phi.Argument(index)->Resolve(); + } + return nullptr; + } + + bool wordsImply(const IrValue* low, const IrValue* high, std::uint32_t depth) { + if (isImmediate(low, 0u) && (waveSize == 32u || isImmediate(high, 0u))) return true; + const ImplicationKey key{low, high}; + if (const auto known = results.find(key); known != results.end()) return known->second; + if (depth > ImplicationDepthLimit) return false; + results.emplace(key, false); + bool result = false; + if (low->Opcode() == IrOpcode::CompositeExtractU32x4 && isImmediate(low->Argument(1)->Resolve(), 0u)) { + const IrValue* ballot = low->Argument(0)->Resolve(); + const bool sameBallot = waveSize == 32u || (high->Opcode() == IrOpcode::CompositeExtractU32x4 && high->Argument(0)->Resolve() == ballot && isImmediate(high->Argument(1)->Resolve(), 1u)); + result = ballot->Opcode() == IrOpcode::Ballot && sameBallot && Implies(ballot->Argument(0), depth + 1u); + } + if (!result && low->Opcode() == IrOpcode::BitwiseAnd32) { + if (waveSize == 32u) { + result = wordsImply(low->Argument(0)->Resolve(), high, depth + 1u) || wordsImply(low->Argument(1)->Resolve(), high, depth + 1u); + } else if (high->Opcode() == IrOpcode::BitwiseAnd32) { + for (std::size_t lowIndex = 0; !result && lowIndex < 2u; ++lowIndex) { + for (std::size_t highIndex = 0; !result && highIndex < 2u; ++highIndex) { + result = wordsImply(low->Argument(lowIndex)->Resolve(), high->Argument(highIndex)->Resolve(), depth + 1u); + } + } + } + } + if (!result && low->IsPhi() && mergesOutsideLoops(*low) && (waveSize == 32u || (high->IsPhi() && high->Parent() == low->Parent() && mergesOutsideLoops(*high)))) { + result = true; + for (std::size_t index = 0; result && index < low->PhiBlockCount(); ++index) { + const IrValue* highIncoming = waveSize == 32u ? high : incomingFrom(*high, low->PhiBlock(index)); + result = highIncoming != nullptr && wordsImply(low->Argument(index)->Resolve(), highIncoming, depth + 1u); + } + } + results[key] = result; + return result; + } + + const std::unordered_set& once; + const IrValue* mask; + std::uint32_t waveSize; + bool always; + std::unordered_map results; +}; std::unordered_set blocksOutsideLoops(const IrProgram& program) { const auto& order = program.BlockOrder(); @@ -338,7 +407,7 @@ std::unordered_set blocksOutsideLoops(const IrProgram& program) } bool unobservedWhereMasked(const IrProgram& program, const std::unordered_set& once, const IrValue& select, const IrValue* mask) { - const auto waveSize = program.WaveSize(); + Implications implications(once, mask, program.WaveSize()); std::vector pending{&select}; std::unordered_set visited{&select}; while (!pending.empty()) { @@ -347,9 +416,9 @@ bool unobservedWhereMasked(const IrProgram& program, const std::unordered_setOperandUses()) { const IrValue* user = use.user; if (user->Parent() == nullptr || !once.contains(user->Parent())) return false; - if (isSelect(user->Opcode()) && use.operand == 1u && implies(user->Argument(0), mask, waveSize)) continue; - if (user->Opcode() == IrOpcode::LogicalAnd && use.operand < 2u && implies(user->Argument(1u - use.operand), mask, waveSize)) continue; - if (readsOnlyWhereActive(user->Opcode()) && use.operand + 1u < user->ArgumentCount() && implies(user->Argument(user->ArgumentCount() - 1u), mask, waveSize)) continue; + if (isSelect(user->Opcode()) && use.operand == 1u && implications.Implies(user->Argument(0))) continue; + if (user->Opcode() == IrOpcode::LogicalAnd && use.operand < 2u && implications.Implies(user->Argument(1u - use.operand))) continue; + if (readsOnlyWhereActive(user->Opcode()) && use.operand + 1u < user->ArgumentCount() && implications.Implies(user->Argument(user->ArgumentCount() - 1u))) continue; if (!user->IsPhi() && !isLaneLocal(user->Opcode()) && !isExplicitLodSample(program, *user)) return false; if (visited.insert(user).second) { if (visited.size() > VisitLimit) return false; From cd4eaacf7c7300fee08a30c4c1188a86177e85db Mon Sep 17 00:00:00 2001 From: AlxcBTW <187763517+AlxcBTW@users.noreply.github.com> Date: Sat, 10 Oct 2026 19:43:36 +0200 Subject: [PATCH 350/460] perf(shader): scale the masked-lane walk limit with the program LEGO Horizon Adventures (PPSA14632) still refuses a pixel shader with packed pixel ancillary input has an unsupported live use (used by SelectU32 ? ?) although it has the form the previous commit handles: the only whole-word reader is SelectU32(IEqual32(HelperInvocation, 0), BitFieldUExtract(x, 0, 11), ancillary), the exec words are Ballot(not helper) & Ballot(true), and there are no image ops, stores or loop phis. About 10,250 IR values derive from the word, so the walk from that select stops at its fixed limit of 4096 visited values and the select is kept. With the limit raised to 8192 the shader prepares (reported on Linux with an RX 9070 XT). A walk may now take max(4096, 2 * instructions) steps. A step is a value visited for the first time or an implication evaluated for the first time. The walk only visits values derived from the select, at most one per instruction, so a walk is only cut short when the implications it evaluates cost as much as the program itself. On the 198 Taxi Chaos 2 requests no walk needs more than 0.79 * instructions steps. One walk stays linear in the program, O(instructions + operands): each value is expanded once, and with the per-walk memo each implication is evaluated once. The walk is also cheaper, so that the larger limit costs nothing: - visited values are marked in a table indexed by value id instead of a hash set, and the table, the work list and the memo are reused across walks; - blocks outside loops are looked up in a table indexed by block id instead of a hash set. This lookup ran once per use and was most of the pass's time. Time in MaskedSelectEliminator per prepare (median of 3, the pass runs twice per prepare), 54 Taxi Chaos 2 requests: shader before 40a8a5a7 this commit main 5edae353 0x1ac0413000 689 ms 285 ms 401 ms 0x19002bd200 102 ms 34 ms 100 ms 0x18c07bf000 191 ms 52 ms (refused) all 54 1294 ms 484 ms 919 ms 0x1ac0413000 and its three copies in the NVIDIA dump now remove 659 selects instead of 625: their walks no longer stop at 4096. No other request changes what it removes. 197 of the 198 requests prepare before and after; the other one fails the same way in both (v_readlane_b32 of lane 63 on a 32-lane host subgroup). Related: #2548 Co-Authored-By: Claude Opus 5.5 --- .../libSceAgcDriver/tests/MaskedSelect.cpp | 23 ++++ core/libs/tests/PixelAncillary.cpp | 16 ++- .../src/MaskedSelectEliminator.cpp | 127 +++++++++++++----- 3 files changed, 126 insertions(+), 40 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp b/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp index ea19428f1..839a465a6 100644 --- a/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp @@ -162,6 +162,29 @@ int main() { return b.eliminate() == 2u && removed(first) && removed(second) && !removed(sink); }); + passed &= run("a write read through thousands of lane-local values still loses its select", [] { + Builder b; + auto& exec = b.mask(16u); + auto& old = b.lane(); + auto& written = b.select(exec, b.add(old, 1u), old); + IrValue* value = &written; + for (std::uint32_t step = 0; step < 6000u; ++step) value = &b.add(*value, step); + b.keep(b.select(exec, *value, old)); + return b.eliminate() == 1u && removed(written); + }); + + passed &= run("thousands of lane-local values ending in a wider read keep the select", [] { + Builder b; + auto& wide = b.mask(32u); + auto& exec = b.logicalAnd(wide, b.mask(16u)); + auto& old = b.lane(); + auto& written = b.select(exec, b.add(old, 1u), old); + IrValue* value = &written; + for (std::uint32_t step = 0; step < 6000u; ++step) value = &b.add(*value, step); + b.keep(b.select(wide, *value, old)); + return b.eliminate() == 0u && !removed(written); + }); + passed &= run("cross-lane reads must keep the select", [] { bool ok = true; for (const IrOpcode opcode : {IrOpcode::ReadFirstLane, IrOpcode::DppMoveU32, IrOpcode::BpermuteU32, IrOpcode::Permlane16U32, IrOpcode::SwizzleU32}) { diff --git a/core/libs/tests/PixelAncillary.cpp b/core/libs/tests/PixelAncillary.cpp index 2100713ec..9832c0ebe 100644 --- a/core/libs/tests/PixelAncillary.cpp +++ b/core/libs/tests/PixelAncillary.cpp @@ -262,7 +262,7 @@ struct HelperProgram { IrValue* exported = nullptr; }; -static HelperProgram HelperOnly(IrProgram& program, HelperRead read) { +static HelperProgram HelperOnly(IrProgram& program, HelperRead read, std::uint32_t chain = 0u) { auto& entry = Begin(program); auto& body = program.CreateBlock(); auto& merge = program.CreateBlock(); @@ -292,7 +292,9 @@ static HelperProgram HelperOnly(IrProgram& program, HelperRead read) { value.AddPhiOperand(&entry, &written); value.AddPhiOperand(&body, &killed); builder.SetInsertionPoint(merge); - IrValue& data = builder.Emit(IrOpcode::CompositeConstructU32x4, IrType::U32x4, {&value, &target, &builder.Constant(0u), &builder.Constant(0u)}); + IrValue* shaded = &value; + for (std::uint32_t step = 0; step < chain; ++step) shaded = &builder.IAdd(*shaded, helper); + IrValue& data = builder.Emit(IrOpcode::CompositeConstructU32x4, IrType::U32x4, {shaded, &target, &builder.Constant(0u), &builder.Constant(0u)}); program.Metadata().exportInfo.push_back(ExportInfo {.kind = ExportTargetKind::Mrt, .index = 0u, .en = 0xFu, .done = true, .vm = true}); IrValue& guard = read == HelperRead::FullWaveExport ? builder.ConstantBool(true) : exec; static_cast(builder.Emit(IrOpcode::SetAttribute, IrType::Void, {&data, &guard}, ExportFlags {0u, 0u})); @@ -312,9 +314,9 @@ static void LowerMasked(IrProgram& program) { const ShaderPixelInputInfo pixel {}; ShaderInfoCollector().Collect(program, ShaderStageInputInfo {nullptr, &pixel, nullptr}); } -static void HelperLanesOnly() { +static void HelperLanesOnly(std::uint32_t chain = 0u) { IrProgram program; - const HelperProgram built = HelperOnly(program, HelperRead::Export); + const HelperProgram built = HelperOnly(program, HelperRead::Export, chain); LowerMasked(program); for (const IrBlock* block : program.BlockOrder()) { for (const IrValue* inst : block->Instructions()) { @@ -327,9 +329,9 @@ static void HelperLanesOnly() { const IrValue* entryValue = built.exported->Argument(0)->Resolve(); Require(entryValue->HasImmediate() && entryValue->ImmediateU32() == SelectOther); } -static void HelperLanesRefused(HelperRead read) { +static void HelperLanesRefused(HelperRead read, std::uint32_t chain = 0u) { IrProgram program; - static_cast(HelperOnly(program, read)); + static_cast(HelperOnly(program, read, chain)); try { LowerMasked(program); } catch (const std::runtime_error& error) { @@ -378,7 +380,9 @@ int main() { MergedRefused(IrOpcode::BitFieldUExtract, 12u, 4u, false); MergedRefused(IrOpcode::BitFieldUExtract, 0u, 2u, true); HelperLanesOnly(); + HelperLanesOnly(6000u); HelperLanesRefused(HelperRead::FullWaveExport); HelperLanesRefused(HelperRead::DataTestExport); HelperLanesRefused(HelperRead::Ballot); + HelperLanesRefused(HelperRead::FullWaveExport, 6000u); } diff --git a/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp b/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp index 1966437f1..e9d218b80 100644 --- a/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp +++ b/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp @@ -1,6 +1,7 @@ #include "Optimization/MaskedSelectEliminator.hpp" #include "Optimization/DeadCodeEliminator.hpp" #include "RdnaDecoder/RdnaInstruction.hpp" +#include #include #include #include @@ -10,7 +11,8 @@ namespace ShaderRecompiler { namespace { -constexpr std::size_t VisitLimit = 4096; +constexpr std::size_t MinimumWalkLimit = 4096; +constexpr std::size_t WalkLimitPerInstruction = 2; constexpr std::uint32_t DepthLimit = 8; bool isSelect(IrOpcode opcode) { @@ -266,6 +268,12 @@ bool alwaysTrue(const IrValue* value, std::uint32_t waveSize, std::uint32_t dept return bit && isImmediate(bit->low, 0xffffffffu) && isImmediate(bit->high, 0xffffffffu); } +enum class Region : std::uint8_t { Unreached, Once, Loop }; + +Region regionOf(const std::vector& regions, const IrBlock* block) { + return block != nullptr && block->Id() < regions.size() ? regions[block->Id()] : Region::Unreached; +} + struct ImplicationKey { const IrValue* first = nullptr; const IrValue* second = nullptr; @@ -281,7 +289,13 @@ struct ImplicationKeyHash { class Implications { public: - Implications(const std::unordered_set& once, const IrValue* mask, std::uint32_t waveSize) : once(once), mask(mask), waveSize(waveSize), always(alwaysTrue(mask, waveSize)) {} + Implications(const std::vector& regions, std::uint32_t waveSize, std::size_t& work) : regions(regions), waveSize(waveSize), work(work) {} + + void Reset(const IrValue* writeMask) { + mask = writeMask; + always = alwaysTrue(writeMask, waveSize); + results.clear(); + } bool Implies(const IrValue* condition, std::uint32_t depth = 0) { condition = condition->Resolve(); @@ -289,7 +303,8 @@ public: if (condition->HasImmediate()) return condition->Type() == IrType::Bool && !condition->ImmediateBool(); const ImplicationKey key{condition, nullptr}; if (const auto known = results.find(key); known != results.end()) return known->second; - if (depth > ImplicationDepthLimit) return false; + if (depth > ImplicationDepthLimit || work == 0) return false; + --work; results.emplace(key, false); bool result = false; if (condition->Opcode() == IrOpcode::LogicalAnd) { @@ -310,10 +325,10 @@ private: static constexpr std::uint32_t ImplicationDepthLimit = 64; bool mergesOutsideLoops(const IrValue& phi) const { - if (phi.Parent() == nullptr || !once.contains(phi.Parent()) || phi.ArgumentCount() != phi.PhiBlockCount()) return false; + if (regionOf(regions, phi.Parent()) != Region::Once || phi.ArgumentCount() != phi.PhiBlockCount()) return false; for (std::size_t index = 0; index < phi.ArgumentCount(); ++index) { const IrValue* incoming = phi.Argument(index)->Resolve(); - if (!incoming->HasImmediate() && (incoming->Parent() == nullptr || !once.contains(incoming->Parent()))) return false; + if (!incoming->HasImmediate() && regionOf(regions, incoming->Parent()) != Region::Once) return false; } return true; } @@ -329,7 +344,8 @@ private: if (isImmediate(low, 0u) && (waveSize == 32u || isImmediate(high, 0u))) return true; const ImplicationKey key{low, high}; if (const auto known = results.find(key); known != results.end()) return known->second; - if (depth > ImplicationDepthLimit) return false; + if (depth > ImplicationDepthLimit || work == 0) return false; + --work; results.emplace(key, false); bool result = false; if (low->Opcode() == IrOpcode::CompositeExtractU32x4 && isImmediate(low->Argument(1)->Resolve(), 0u)) { @@ -359,14 +375,15 @@ private: return result; } - const std::unordered_set& once; - const IrValue* mask; + const std::vector& regions; std::uint32_t waveSize; - bool always; + std::size_t& work; + const IrValue* mask = nullptr; + bool always = false; std::unordered_map results; }; -std::unordered_set blocksOutsideLoops(const IrProgram& program) { +std::vector blockRegions(const IrProgram& program) { const auto& order = program.BlockOrder(); if (order.empty()) return {}; std::unordered_map state; @@ -399,46 +416,88 @@ std::unordered_set blocksOutsideLoops(const IrProgram& program) looping.insert(body.begin(), body.end()); } } - std::unordered_set result; + std::vector result; for (const auto& [block, visited] : state) { - if (visited == 2 && !looping.contains(block)) result.insert(block); + if (visited != 2) continue; + if (block->Id() >= result.size()) result.resize(block->Id() + 1u, Region::Unreached); + result[block->Id()] = looping.contains(block) ? Region::Loop : Region::Once; } return result; } -bool unobservedWhereMasked(const IrProgram& program, const std::unordered_set& once, const IrValue& select, const IrValue* mask) { - Implications implications(once, mask, program.WaveSize()); - std::vector pending{&select}; - std::unordered_set visited{&select}; - while (!pending.empty()) { - const IrValue* value = pending.back(); - pending.pop_back(); - for (const IrUse& use : value->OperandUses()) { - const IrValue* user = use.user; - if (user->Parent() == nullptr || !once.contains(user->Parent())) return false; - if (isSelect(user->Opcode()) && use.operand == 1u && implications.Implies(user->Argument(0))) continue; - if (user->Opcode() == IrOpcode::LogicalAnd && use.operand < 2u && implications.Implies(user->Argument(1u - use.operand))) continue; - if (readsOnlyWhereActive(user->Opcode()) && use.operand + 1u < user->ArgumentCount() && implications.Implies(user->Argument(user->ArgumentCount() - 1u))) continue; - if (!user->IsPhi() && !isLaneLocal(user->Opcode()) && !isExplicitLodSample(program, *user)) return false; - if (visited.insert(user).second) { - if (visited.size() > VisitLimit) return false; - pending.push_back(user); +std::size_t walkLimit(const IrProgram& program) { + std::size_t instructions = 0; + for (const IrBlock* block : program.BlockOrder()) instructions += block->Instructions().size(); + return std::max(MinimumWalkLimit, WalkLimitPerInstruction * instructions); +} + +class MaskedLaneWalk { +public: + MaskedLaneWalk(const IrProgram& program, const std::vector& regions) : program(program), regions(regions), limit(walkLimit(program)), implications(regions, program.WaveSize(), work) {} + + bool UnobservedWhereMasked(const IrValue& select, const IrValue* mask) { + work = limit; + implications.Reset(mask); + startVisits(); + pending.clear(); + pending.push_back(&select); + static_cast(firstVisit(select)); + while (!pending.empty()) { + const IrValue* value = pending.back(); + pending.pop_back(); + for (const IrUse& use : value->OperandUses()) { + const IrValue* user = use.user; + if (regionOf(regions, user->Parent()) != Region::Once) return false; + if (isSelect(user->Opcode()) && use.operand == 1u && implications.Implies(user->Argument(0))) continue; + if (user->Opcode() == IrOpcode::LogicalAnd && use.operand < 2u && implications.Implies(user->Argument(1u - use.operand))) continue; + if (readsOnlyWhereActive(user->Opcode()) && use.operand + 1u < user->ArgumentCount() && implications.Implies(user->Argument(user->ArgumentCount() - 1u))) continue; + if (!user->IsPhi() && !isLaneLocal(user->Opcode()) && !isExplicitLodSample(program, *user)) return false; + if (firstVisit(*user)) { + if (work == 0) return false; + --work; + pending.push_back(user); + } } } + return true; } - return true; -} + +private: + void startVisits() { + if (++epoch != 0u) return; + std::fill(visits.begin(), visits.end(), 0u); + epoch = 1u; + } + + bool firstVisit(const IrValue& value) { + const std::uint32_t id = value.Id(); + if (id >= visits.size()) visits.resize(id + 1u, 0u); + if (visits[id] == epoch) return false; + visits[id] = epoch; + return true; + } + + const IrProgram& program; + const std::vector& regions; + std::size_t limit; + std::size_t work = 0; + Implications implications; + std::vector pending; + std::vector visits; + std::uint32_t epoch = 0; +}; } MaskedSelectEliminationStats MaskedSelectEliminator::Eliminate(IrProgram& program) const { MaskedSelectEliminationStats stats; - const auto once = blocksOutsideLoops(program); + const auto regions = blockRegions(program); + MaskedLaneWalk walk(program, regions); bool changed = true; while (changed) { changed = false; for (auto blockIt = program.BlockOrder().rbegin(); blockIt != program.BlockOrder().rend(); ++blockIt) { - const bool single = once.contains(*blockIt); + const bool single = regionOf(regions, *blockIt) == Region::Once; auto& instructions = (*blockIt)->Instructions(); auto it = instructions.end(); while (it != instructions.begin()) { @@ -446,7 +505,7 @@ MaskedSelectEliminationStats MaskedSelectEliminator::Eliminate(IrProgram& progra IrValue* inst = *it; if (!isSelect(inst->Opcode()) || inst->ArgumentCount() != 3u || !inst->HasUses()) continue; const IrValue* mask = inst->Argument(0)->Resolve(); - if (!alwaysTrue(mask, program.WaveSize()) && (!single || !unobservedWhereMasked(program, once, *inst, mask))) continue; + if (!alwaysTrue(mask, program.WaveSize()) && (!single || !walk.UnobservedWhereMasked(*inst, mask))) continue; inst->ReplaceAllUsesWith(inst->Argument(1)); inst->Invalidate(); inst->SetParent(nullptr); From 98f99d38836c8626db22dfe50f6ed4115dc3d03d Mon Sep 17 00:00:00 2001 From: AlxcBTW <187763517+AlxcBTW@users.noreply.github.com> Date: Sat, 10 Oct 2026 20:06:44 +0200 Subject: [PATCH 351/460] fix(shader): prove masked lanes unread across loops Kena: Bridge of Spirits (PPSA01802) refuses ten pixel shaders with packed pixel ancillary input has an unsupported live use (used by SelectU32 ? ?) at any visit limit. All ten have 30 to 33 phis on loop back edges (reported on Linux with an RX 9070 XT). Two rules stopped the proof at a loop, and the first one fires before the second is ever asked: - The walk from the select gave up at the first use inside a loop. A value derived from the select reaches a loop phi as soon as its register is written inside the loop, because the masked write becomes Select(loopExec, new, phi). A loop that leaves exec alone needs nothing more than this. - An exec that a loop narrows and carries around its back edge is a phi inside the loop, and such phis counted as not implying the write's exec. The walk now follows uses into loops. A phi inside a loop implies the write's exec when all of its incoming values do, where the phi itself counts as implying it while its back-edge values are checked. This is sound because each rule of the proof takes a lane bit of one value from a value computed earlier: a phi's value on each iteration is the incoming value computed before the edge was taken, a set lane of an And is set in both operands, a set lane of a ballot is a lane whose predicate held. If any value in the proof had a lane set where the write's exec is clear, the earliest such value would take that lane from an earlier value of the proof that is set there too, which contradicts it being the earliest. Two conditions keep this true: - The write's exec must be computed once, outside every loop. Otherwise a phi can carry a ballot of it from an earlier iteration, which is not the value the select used. Masks computed inside a loop keep the old rules. - A result may only be kept if the assumptions it used held. When a value that was assumed turns out not to be implied, every result recorded since it was assumed is dropped. Results that say "not implied" are kept; they only make the proof refuse more. A lane that is turned back on inside the loop makes the back-edge exec wider than the write's exec, so the phi is not implied and a read under it keeps the select. A lane that is turned back on but never reads the derived value does not. New tests cover a loop that carries the write with exec saved, narrowed and restored (read inside and after the loop), a loop that leaves exec alone, and the helper-only ancillary select feeding such loops. The refusing tests each fail when the matching check is removed: a revived lane that reads the write inside the loop (ignoring back-edge values), a result proven under an assumption that later fails (keeping it), and a write mask computed inside an earlier loop (allowing such masks). Two existing tests expected a loop to keep the select although every read is under the write's own exec; they now expect it to go, next to new counterparts with a wider exec. On the Taxi Chaos 2 requests, compute shaders with a loop inside a branch now lose the branch's masked writes: 0x194000b400 removes 207 selects instead of 36, 0x19002bd200 111 instead of 28, and seven more gain 5 to 43. In 0x16400cf700 the 26 new removals are writes under the branch condition tid < count; the loop's exec starts as a ballot of that condition, only loses lanes at the latch, and every store and atomic inside the loop runs under it. No proof on these requests had to drop a result. Walks now go on into loops, so the pass takes longer on loop shaders, but it stays below main. Time in MaskedSelectEliminator per prepare (median of 3, the pass runs twice per prepare), 54 Taxi Chaos 2 requests, one run: shader previous commit this commit main 5edae353 0x1ac0413000 334 ms 349 ms 424 ms 0x19002bd200 38 ms 103 ms 112 ms 0x19c016aa00 9 ms 74 ms 16 ms 0x194000b400 13 ms 31 ms 32 ms all 54 547 ms 805 ms 977 ms 0x19c016aa00 removes nothing more; its walks only end later inside its loops. The 144 requests of the NVIDIA dump go from 1283 to 1589 ms and gain removals in the copies of the same shaders. 197 of the 198 requests prepare, as before. Related: #2548 Co-Authored-By: Claude Opus 5.5 --- .../libSceAgcDriver/tests/MaskedSelect.cpp | 161 +++++++++++++++++- core/libs/tests/PixelAncillary.cpp | 91 ++++++++++ .../src/MaskedSelectEliminator.cpp | 73 ++++++-- 3 files changed, 304 insertions(+), 21 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp b/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp index 839a465a6..56dc21e87 100644 --- a/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp @@ -77,6 +77,22 @@ struct Builder { return threadBit(low, high); } + std::pair ballotWords(IrValue& predicate) { + auto& ballot = emit(IrOpcode::Ballot, IrType::U32x4, {&predicate}); + return {&emit(IrOpcode::CompositeExtractU32x4, IrType::U32, {&ballot, &constant(0u)}), &emit(IrOpcode::CompositeExtractU32x4, IrType::U32, {&ballot, &constant(1u)})}; + } + + IrValue& wordAnd(IrValue& left, IrValue& right) { + return emit(IrOpcode::BitwiseAnd32, IrType::U32, {&left, &right}); + } + + IrValue& phi(IrType type, IrBlock& predecessor, IrValue& incoming) { + auto& created = program.CreateValue(IrOpcode::Phi, type); + block->AppendInstruction(&created); + created.AddPhiOperand(&predecessor, &incoming); + return created; + } + void keep(IrValue& value) { (void)emit(IrOpcode::ReferenceU32, IrType::Void, {&value}); } @@ -339,8 +355,13 @@ int main() { return count == 0u && !gone; }); - passed &= run("an exec merged by a phi inside a loop keeps the select", [&] { + passed &= run("an exec carried around a loop whose every incoming exec implies the write's exec drops the select", [&] { const auto [count, gone] = mergedExec(false, true); + return count == 1u && gone; + }); + + passed &= run("an exec carried around a loop with a wider incoming exec keeps the select", [&] { + const auto [count, gone] = mergedExec(true, true); return count == 0u && !gone; }); @@ -425,9 +446,10 @@ int main() { return b.eliminate() == 0u && !removed(written); }); - passed &= run("a write read inside a later loop must keep its select", [] { + const auto laterLoop = [](bool wideRead) { Builder b; - auto& exec = b.mask(16u); + auto& wide = b.mask(48u); + auto& exec = b.logicalAnd(wide, b.mask(16u)); auto& old = b.lane(); auto& written = b.select(exec, b.add(old, 1u), old); IrBlock& head = *b.block; @@ -435,7 +457,138 @@ int main() { head.AddBranch(&loop); loop.AddBranch(&loop); b.block = &loop; - b.keep(b.select(exec, b.add(written, 2u), old)); + b.keep(b.select(wideRead ? wide : exec, b.add(written, 2u), old)); + return std::pair{b.eliminate(), removed(written)}; + }; + + passed &= run("a write read inside a later loop under its own exec loses its select", [&] { + const auto [count, gone] = laterLoop(false); + return count == 1u && gone; + }); + + passed &= run("a write read inside a later loop under a wider exec keeps its select", [&] { + const auto [count, gone] = laterLoop(true); + return count == 0u && !gone; + }); + + enum class Latch { Narrowed, Revived }; + const auto carried = [](Latch latch, bool readInside) { + Builder b; + auto& wide = b.mask(48u); + auto& exec = b.logicalAnd(wide, b.mask(40u)); + auto& old = b.lane(); + auto& written = b.select(exec, b.add(old, 1u), old); + const auto [savedLow, savedHigh] = b.ballotWords(exec); + IrBlock& entry = *b.block; + IrBlock& header = b.newBlock(); + IrBlock& exit = b.newBlock(); + entry.AddBranch(&header); + header.AddBranch(&header); + header.AddBranch(&exit); + b.block = &header; + auto& low = b.phi(IrType::U32, entry, *savedLow); + auto& high = b.phi(IrType::U32, entry, *savedHigh); + auto& value = b.phi(IrType::U32, entry, written); + auto& loopExec = b.threadBit(low, high); + auto& next = b.select(loopExec, b.add(value, 2u), value); + if (readInside) b.keep(b.select(loopExec, b.add(value, 3u), old)); + if (latch == Latch::Narrowed) { + const auto [stayLow, stayHigh] = b.ballotWords(b.mask(24u)); + low.AddPhiOperand(&header, &b.wordAnd(low, *stayLow)); + high.AddPhiOperand(&header, &b.wordAnd(high, *stayHigh)); + } else { + const auto [revivedLow, revivedHigh] = b.ballotWords(wide); + low.AddPhiOperand(&header, revivedLow); + high.AddPhiOperand(&header, revivedHigh); + } + value.AddPhiOperand(&header, &next); + b.block = &exit; + b.keep(b.select(b.threadBit(*savedLow, *savedHigh), b.add(value, 4u), old)); + return std::pair{b.eliminate(), removed(written)}; + }; + + passed &= run("a write carried through a loop whose exec is saved, narrowed and restored loses its select", [&] { + const auto [count, gone] = carried(Latch::Narrowed, false); + return count == 1u && gone; + }); + + passed &= run("a write read inside a loop under the loop's narrowed exec loses its select", [&] { + const auto [count, gone] = carried(Latch::Narrowed, true); + return count == 1u && gone; + }); + + passed &= run("a lane that becomes active again inside a loop and reads the write keeps the select", [&] { + const auto [count, gone] = carried(Latch::Revived, true); + return count == 0u && !gone; + }); + + passed &= run("a lane that becomes active again inside a loop but never reads the write lets the select go", [&] { + const auto [count, gone] = carried(Latch::Revived, false); + return count == 1u && gone; + }); + + passed &= run("an exec proven only by assuming a loop phi is forgotten once that phi turns out wider", [] { + Builder b; + auto& wide = b.mask(48u); + auto& exec = b.logicalAnd(wide, b.mask(40u)); + auto& old = b.lane(); + auto& written = b.select(exec, b.add(old, 1u), old); + const auto [savedLow, savedHigh] = b.ballotWords(exec); + IrBlock& entry = *b.block; + IrBlock& header = b.newBlock(); + IrBlock& kill = b.newBlock(); + IrBlock& revive = b.newBlock(); + IrBlock& latch = b.newBlock(); + IrBlock& exit = b.newBlock(); + entry.AddBranch(&header); + header.AddBranch(&kill); + header.AddBranch(&revive); + kill.AddBranch(&latch); + revive.AddBranch(&latch); + latch.AddBranch(&header); + latch.AddBranch(&exit); + b.block = &header; + auto& low = b.phi(IrType::U32, entry, *savedLow); + auto& high = b.phi(IrType::U32, entry, *savedHigh); + (void)b.select(b.threadBit(low, high), written, old); + b.block = &kill; + const auto [stayLow, stayHigh] = b.ballotWords(b.mask(24u)); + auto& killedLow = b.wordAnd(low, *stayLow); + auto& killedHigh = b.wordAnd(high, *stayHigh); + b.keep(b.select(b.threadBit(killedLow, killedHigh), written, old)); + b.block = &revive; + const auto [revivedLow, revivedHigh] = b.ballotWords(wide); + b.block = &latch; + auto& mergedLow = b.phi(IrType::U32, kill, killedLow); + auto& mergedHigh = b.phi(IrType::U32, kill, killedHigh); + mergedLow.AddPhiOperand(&revive, revivedLow); + mergedHigh.AddPhiOperand(&revive, revivedHigh); + low.AddPhiOperand(&latch, &mergedLow); + high.AddPhiOperand(&latch, &mergedHigh); + return b.eliminate() == 0u && !removed(written); + }); + + passed &= run("a write mask computed inside an earlier loop does not let that loop's phis stand for it", [] { + Builder b; + auto& old = b.lane(); + IrBlock& entry = *b.block; + IrBlock& header = b.newBlock(); + IrBlock& exit = b.newBlock(); + entry.AddBranch(&header); + header.AddBranch(&header); + header.AddBranch(&exit); + b.block = &header; + auto& low = b.phi(IrType::U32, entry, b.constant(0u)); + auto& high = b.phi(IrType::U32, entry, b.constant(0u)); + auto& bound = b.phi(IrType::U32, entry, b.constant(64u)); + auto& exec = b.emit(IrOpcode::ULessThan32, IrType::Bool, {&b.lane(), &bound}); + const auto [stayLow, stayHigh] = b.ballotWords(exec); + low.AddPhiOperand(&header, stayLow); + high.AddPhiOperand(&header, stayHigh); + bound.AddPhiOperand(&header, &b.emit(IrOpcode::ISub32, IrType::U32, {&bound, &b.constant(8u)})); + b.block = &exit; + auto& written = b.select(exec, b.add(old, 1u), old); + b.keep(b.select(b.threadBit(low, high), b.add(written, 2u), old)); return b.eliminate() == 0u && !removed(written); }); diff --git a/core/libs/tests/PixelAncillary.cpp b/core/libs/tests/PixelAncillary.cpp index 9832c0ebe..15740c514 100644 --- a/core/libs/tests/PixelAncillary.cpp +++ b/core/libs/tests/PixelAncillary.cpp @@ -340,6 +340,93 @@ static void HelperLanesRefused(HelperRead read, std::uint32_t chain = 0u) { } Require(false); } +enum class LoopRead { ExportAfter, ExportInside, UniformLoop, RevivedExportInside }; + +static IrValue& LaneBit(IrBuilder& builder, IrValue& low, IrValue& high) { + IrValue& lane = builder.Emit(IrOpcode::LaneId, IrType::U32, {}); + IrValue& word = builder.Select(builder.Emit(IrOpcode::ULessThan32, IrType::Bool, {&lane, &builder.Constant(32u)}), low, high); + IrValue& shifted = builder.Emit(IrOpcode::ShiftRightLogical32, IrType::U32, {&word, &builder.BitwiseAnd(lane, builder.Constant(31u))}); + return builder.INotEqual(builder.BitwiseAnd(shifted, builder.Constant(1u)), builder.Constant(0u)); +} +static HelperProgram HelperLoop(IrProgram& program, LoopRead read) { + auto& entry = Begin(program); + auto& header = program.CreateBlock(); + auto& exit = program.CreateBlock(); + program.BlockOrder().push_back(&header); + program.BlockOrder().push_back(&exit); + entry.AddBranch(&header); + header.AddBranch(&header); + header.AddBranch(&exit); + IrBuilder builder(program); + builder.SetInsertionPoint(entry); + IrValue& helper = builder.Emit(IrOpcode::GetBuiltin, IrType::U32, {&builder.Constant(static_cast(StageInputKind::HelperInvocation)), &builder.Constant(0u)}); + IrValue& live = builder.IEqual(helper, builder.Constant(0u)); + IrValue& ancillary = AncillaryBuiltin(builder); + IrValue& layer = builder.Emit(IrOpcode::BitFieldUExtract, IrType::U32, {&ancillary, &builder.Constant(16u), &builder.Constant(11u)}); + IrValue& target = builder.Select(live, layer, builder.Constant(0u)); + IrValue& written = builder.Select(live, builder.Constant(SelectOther), ancillary); + IrValue& frontFacing = FrontFacingBuiltin(builder); + IrValue& saved = builder.Emit(IrOpcode::Ballot, IrType::U32x4, {&live}); + IrValue& savedLow = builder.CompositeExtract(saved, 0u); + IrValue& savedHigh = builder.CompositeExtract(saved, 1u); + IrValue& low = program.CreateValue(IrOpcode::Phi, IrType::U32); + IrValue& high = program.CreateValue(IrOpcode::Phi, IrType::U32); + IrValue& value = program.CreateValue(IrOpcode::Phi, IrType::U32); + header.AppendInstruction(&low); + header.AppendInstruction(&high); + header.AppendInstruction(&value); + low.AddPhiOperand(&entry, &savedLow); + high.AddPhiOperand(&entry, &savedHigh); + value.AddPhiOperand(&entry, &written); + builder.SetInsertionPoint(header); + IrValue& loopExec = read == LoopRead::UniformLoop ? live : LaneBit(builder, low, high); + IrValue& squared = builder.Emit(IrOpcode::FPMul32, IrType::F32, {&builder.BitCastF32(value), &builder.BitCastF32(value)}); + IrValue& next = builder.Select(loopExec, builder.BitCastU32(squared), value); + IrValue& stay = builder.Emit(IrOpcode::Ballot, IrType::U32x4, {&frontFacing}); + IrValue& stayLow = builder.CompositeExtract(stay, 0u); + IrValue& stayHigh = builder.CompositeExtract(stay, 1u); + const bool revived = read == LoopRead::RevivedExportInside; + low.AddPhiOperand(&header, revived ? &stayLow : &builder.BitwiseAnd(low, stayLow)); + high.AddPhiOperand(&header, revived ? &stayHigh : &builder.BitwiseAnd(high, stayHigh)); + value.AddPhiOperand(&header, &next); + program.Metadata().exportInfo.push_back(ExportInfo {.kind = ExportTargetKind::Mrt, .index = 0u, .en = 0xFu, .done = true, .vm = true}); + const auto exportValue = [&](IrValue& guard) { + IrValue& data = builder.Emit(IrOpcode::CompositeConstructU32x4, IrType::U32x4, {&value, &target, &builder.Constant(0u), &builder.Constant(0u)}); + static_cast(builder.Emit(IrOpcode::SetAttribute, IrType::Void, {&data, &guard}, ExportFlags {0u, 0u})); + }; + if (read == LoopRead::ExportInside || revived) exportValue(loopExec); + builder.SetInsertionPoint(exit); + if (read == LoopRead::ExportAfter) exportValue(LaneBit(builder, savedLow, savedHigh)); + if (read == LoopRead::UniformLoop) exportValue(live); + static_cast(builder.Emit(IrOpcode::Return, IrType::Void, {})); + return HelperProgram {&written, &layer, &value}; +} +static void HelperLoopOnly(LoopRead read) { + IrProgram program; + const HelperProgram built = HelperLoop(program, read); + LowerMasked(program); + for (const IrBlock* block : program.BlockOrder()) { + for (const IrValue* inst : block->Instructions()) { + Require(inst->Opcode() != IrOpcode::GetBuiltin || static_cast(inst->Argument(0)->Resolve()->ImmediateU32()) != StageInputKind::PackedAncillary); + } + } + Require(built.written->Parent() == nullptr); + const IrValue* field = built.layer->Argument(0)->Resolve(); + Require(field->Opcode() == IrOpcode::GetBuiltin && static_cast(field->Argument(0)->Resolve()->ImmediateU32()) == StageInputKind::Layer); + const IrValue* entryValue = built.exported->Argument(0)->Resolve(); + Require(entryValue->HasImmediate() && entryValue->ImmediateU32() == SelectOther); +} +static void HelperLoopRefused(LoopRead read) { + IrProgram program; + static_cast(HelperLoop(program, read)); + try { + LowerMasked(program); + } catch (const std::runtime_error& error) { + Require(std::string(error.what()).find("unsupported live use") != std::string::npos); + return; + } + Require(false); +} int main() { Extract(IrOpcode::BitFieldUExtract, 8u, 4u, StageInputKind::SampleId, 0u); Extract(IrOpcode::BitFieldUExtract, 9u, 2u, StageInputKind::SampleId, 1u); @@ -385,4 +472,8 @@ int main() { HelperLanesRefused(HelperRead::DataTestExport); HelperLanesRefused(HelperRead::Ballot); HelperLanesRefused(HelperRead::FullWaveExport, 6000u); + HelperLoopOnly(LoopRead::ExportAfter); + HelperLoopOnly(LoopRead::ExportInside); + HelperLoopOnly(LoopRead::UniformLoop); + HelperLoopRefused(LoopRead::RevivedExportInside); } diff --git a/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp b/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp index e9d218b80..be902084d 100644 --- a/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp +++ b/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp @@ -274,6 +274,11 @@ Region regionOf(const std::vector& regions, const IrBlock* block) { return block != nullptr && block->Id() < regions.size() ? regions[block->Id()] : Region::Unreached; } +bool followable(const std::vector& regions, const IrBlock* block, bool acrossLoops) { + const Region region = regionOf(regions, block); + return region == Region::Once || (acrossLoops && region == Region::Loop); +} + struct ImplicationKey { const IrValue* first = nullptr; const IrValue* second = nullptr; @@ -291,10 +296,13 @@ class Implications { public: Implications(const std::vector& regions, std::uint32_t waveSize, std::size_t& work) : regions(regions), waveSize(waveSize), work(work) {} - void Reset(const IrValue* writeMask) { + void Reset(const IrValue* writeMask, bool acrossLoops) { mask = writeMask; + loops = acrossLoops; always = alwaysTrue(writeMask, waveSize); results.clear(); + proven.clear(); + assumptionReads = 0; } bool Implies(const IrValue* condition, std::uint32_t depth = 0) { @@ -302,33 +310,61 @@ public: if (condition == mask || always) return true; if (condition->HasImmediate()) return condition->Type() == IrType::Bool && !condition->ImmediateBool(); const ImplicationKey key{condition, nullptr}; - if (const auto known = results.find(key); known != results.end()) return known->second; + if (const auto known = lookup(key)) return *known; if (depth > ImplicationDepthLimit || work == 0) return false; --work; - results.emplace(key, false); + const Attempt attempt = assume(key); bool result = false; if (condition->Opcode() == IrOpcode::LogicalAnd) { result = Implies(condition->Argument(0), depth + 1u) || Implies(condition->Argument(1), depth + 1u); } else if (condition->IsPhi()) { - result = mergesOutsideLoops(*condition); + result = merges(*condition); for (std::size_t index = 0; result && index < condition->ArgumentCount(); ++index) { result = Implies(condition->Argument(index), depth + 1u); } } else if (const auto bit = threadBit(condition, waveSize)) { result = wordsImply(bit->low, bit->high, depth + 1u); } - results[key] = result; - return result; + return settle(key, attempt, result); } private: static constexpr std::uint32_t ImplicationDepthLimit = 64; - bool mergesOutsideLoops(const IrValue& phi) const { - if (regionOf(regions, phi.Parent()) != Region::Once || phi.ArgumentCount() != phi.PhiBlockCount()) return false; + enum class Proof : std::uint8_t { Assumed, Implied, NotImplied }; + + struct Attempt { + std::size_t proven = 0; + std::size_t assumptionReads = 0; + }; + + std::optional lookup(const ImplicationKey& key) { + const auto known = results.find(key); + if (known == results.end()) return std::nullopt; + if (known->second == Proof::Assumed) ++assumptionReads; + return known->second != Proof::NotImplied; + } + + Attempt assume(const ImplicationKey& key) { + results.emplace(key, Proof::Assumed); + return Attempt{proven.size(), assumptionReads}; + } + + bool settle(const ImplicationKey& key, const Attempt& attempt, bool implied) { + if (!implied && assumptionReads != attempt.assumptionReads) { + for (std::size_t index = attempt.proven; index < proven.size(); ++index) results.erase(proven[index]); + proven.resize(attempt.proven); + } + results[key] = implied ? Proof::Implied : Proof::NotImplied; + if (implied) proven.push_back(key); + return implied; + } + + bool merges(const IrValue& phi) const { + if (!followable(regions, phi.Parent(), loops) || phi.ArgumentCount() != phi.PhiBlockCount()) return false; for (std::size_t index = 0; index < phi.ArgumentCount(); ++index) { const IrValue* incoming = phi.Argument(index)->Resolve(); - if (!incoming->HasImmediate() && regionOf(regions, incoming->Parent()) != Region::Once) return false; + if (!incoming->HasImmediate() && !followable(regions, incoming->Parent(), loops)) return false; } return true; } @@ -343,10 +379,10 @@ private: bool wordsImply(const IrValue* low, const IrValue* high, std::uint32_t depth) { if (isImmediate(low, 0u) && (waveSize == 32u || isImmediate(high, 0u))) return true; const ImplicationKey key{low, high}; - if (const auto known = results.find(key); known != results.end()) return known->second; + if (const auto known = lookup(key)) return *known; if (depth > ImplicationDepthLimit || work == 0) return false; --work; - results.emplace(key, false); + const Attempt attempt = assume(key); bool result = false; if (low->Opcode() == IrOpcode::CompositeExtractU32x4 && isImmediate(low->Argument(1)->Resolve(), 0u)) { const IrValue* ballot = low->Argument(0)->Resolve(); @@ -364,23 +400,25 @@ private: } } } - if (!result && low->IsPhi() && mergesOutsideLoops(*low) && (waveSize == 32u || (high->IsPhi() && high->Parent() == low->Parent() && mergesOutsideLoops(*high)))) { + if (!result && low->IsPhi() && merges(*low) && (waveSize == 32u || (high->IsPhi() && high->Parent() == low->Parent() && merges(*high)))) { result = true; for (std::size_t index = 0; result && index < low->PhiBlockCount(); ++index) { const IrValue* highIncoming = waveSize == 32u ? high : incomingFrom(*high, low->PhiBlock(index)); result = highIncoming != nullptr && wordsImply(low->Argument(index)->Resolve(), highIncoming, depth + 1u); } } - results[key] = result; - return result; + return settle(key, attempt, result); } const std::vector& regions; std::uint32_t waveSize; std::size_t& work; const IrValue* mask = nullptr; + bool loops = false; bool always = false; - std::unordered_map results; + std::unordered_map results; + std::vector proven; + std::size_t assumptionReads = 0; }; std::vector blockRegions(const IrProgram& program) { @@ -436,8 +474,9 @@ public: MaskedLaneWalk(const IrProgram& program, const std::vector& regions) : program(program), regions(regions), limit(walkLimit(program)), implications(regions, program.WaveSize(), work) {} bool UnobservedWhereMasked(const IrValue& select, const IrValue* mask) { + const bool acrossLoops = mask->HasImmediate() || regionOf(regions, mask->Parent()) == Region::Once; work = limit; - implications.Reset(mask); + implications.Reset(mask, acrossLoops); startVisits(); pending.clear(); pending.push_back(&select); @@ -447,7 +486,7 @@ public: pending.pop_back(); for (const IrUse& use : value->OperandUses()) { const IrValue* user = use.user; - if (regionOf(regions, user->Parent()) != Region::Once) return false; + if (!followable(regions, user->Parent(), acrossLoops)) return false; if (isSelect(user->Opcode()) && use.operand == 1u && implications.Implies(user->Argument(0))) continue; if (user->Opcode() == IrOpcode::LogicalAnd && use.operand < 2u && implications.Implies(user->Argument(1u - use.operand))) continue; if (readsOnlyWhereActive(user->Opcode()) && use.operand + 1u < user->ArgumentCount() && implications.Implies(user->Argument(user->ArgumentCount() - 1u))) continue; From 83f8c193c980de696f2b593ee7802be75e190003 Mon Sep 17 00:00:00 2001 From: xXPBSXx <20614131+xXPBSXx@users.noreply.github.com> Date: Sat, 10 Oct 2026 22:26:51 +0200 Subject: [PATCH 352/460] fix(shader): treat an image read as a use that only reads where its exec is active An ImageRead carries a trailing exec argument like the buffer loads, but it was in neither the read-only-where-active list nor the lane-local list, so a masked select feeding its address was kept. Co-Authored-By: Claude Sonnet 5.5 --- core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp | 11 +++++++++-- .../Optimization/src/MaskedSelectEliminator.cpp | 1 + 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp b/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp index 56dc21e87..ecf3a581e 100644 --- a/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp @@ -636,6 +636,13 @@ int main() { case IrOpcode::LoadBufferU32: b.keep(b.select(exec, b.emit(opcode, IrType::U32, {&b.emit(IrOpcode::GetBufferResource, IrType::BufferResource, {}), &written, &b.constant(0u), &b.constant(0u), &active}), old)); break; + case IrOpcode::ImageRead: { + auto& address = b.emit(IrOpcode::MakeImageAddress, IrType::ImageAddress, {&written, &old}); + auto& image = b.emit(IrOpcode::GetImageResource, IrType::ImageResource, {}); + auto& read = b.emit(opcode, IrType::U32x4, {&image, &address, &active}); + b.keep(b.select(exec, b.emit(IrOpcode::CompositeExtractU32x4, IrType::U32, {&read, &b.constant(0u)}), old)); + break; + } default: b.emit(opcode, IrType::Void, {&b.emit(IrOpcode::CompositeConstructU32x4, IrType::U32x4, {&written, &old, &old, &old}), &active}); break; @@ -645,11 +652,11 @@ int main() { }; passed &= run("a store, load or export whose exec implies the write's exec drops the select", [&] { - return guarded(IrOpcode::WriteSharedU32, true, false) && guarded(IrOpcode::LoadBufferU32, true, false) && guarded(IrOpcode::SetAttribute, true, false); + return guarded(IrOpcode::WriteSharedU32, true, false) && guarded(IrOpcode::LoadBufferU32, true, false) && guarded(IrOpcode::ImageRead, true, false) && guarded(IrOpcode::SetAttribute, true, false); }); passed &= run("a store, load or export under a wider exec keeps the select", [&] { - return !guarded(IrOpcode::WriteSharedU32, false, false) && !guarded(IrOpcode::LoadBufferU32, false, false) && !guarded(IrOpcode::SetAttribute, false, false); + return !guarded(IrOpcode::WriteSharedU32, false, false) && !guarded(IrOpcode::LoadBufferU32, false, false) && !guarded(IrOpcode::ImageRead, false, false) && !guarded(IrOpcode::SetAttribute, false, false); }); passed &= run("a value used as a store's exec keeps the select", [&] { diff --git a/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp b/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp index be902084d..30331f633 100644 --- a/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp +++ b/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp @@ -204,6 +204,7 @@ bool readsOnlyWhereActive(IrOpcode opcode) { case IrOpcode::WriteSharedU32x3: case IrOpcode::WriteSharedU32x4: case IrOpcode::SetAttribute: + case IrOpcode::ImageRead: return true; default: return false; From fda4aa90c9f81fa3b5160150b3bee926b6923955 Mon Sep 17 00:00:00 2001 From: AlxcBTW <187763517+AlxcBTW@users.noreply.github.com> Date: Sat, 10 Oct 2026 22:53:22 +0200 Subject: [PATCH 353/460] fix(shader): let image writes, explicit-LOD gathers and size queries pass the masked-lane walk Following the ImageRead case, every image operation was checked against the two lists the masked-lane walk uses: accesses guarded by a trailing exec operand, and lane-local values. - ImageWrite carries its exec last like StoreBuffer, and EmitWriteOp stores only inside an if on it (bindless tables and FMASK refuse the store). It joins the exec-guarded list. - A gather with an LOD of its own (gather4_l, gather4_lz) or outside a pixel shader reads only its lane's address, like the explicit-LOD samples the walk already follows. A pixel gather without one still keeps the select, since on the GPU it takes its LOD from the quad. - ImageQueryDimensions returns the size of its lane's level of the image and is lane-local. Left out on purpose: - ImageQueryLod and implicit-LOD samples read their quad. - ImageBvhIntersectRay has a trailing exec operand, but its emitter ignores it and records faults in every lane. - Image, buffer, shared and address atomics are guarded by their exec in the emitter too, but the test from 60c65521 pins them as keeping the select, and no refused shader has needed them so far. Co-Authored-By: Claude Opus 5.5 --- .../libSceAgcDriver/tests/MaskedSelect.cpp | 34 +++++++++++++++++-- .../src/MaskedSelectEliminator.cpp | 4 ++- 2 files changed, 35 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp b/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp index ecf3a581e..17c06cd64 100644 --- a/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/MaskedSelect.cpp @@ -592,7 +592,7 @@ int main() { return b.eliminate() == 0u && !removed(written); }); - const auto sample = [](IrShaderStage stage, std::uint32_t sampleFlags) { + const auto sample = [](IrShaderStage stage, std::uint32_t sampleFlags, IrOpcode opcode = IrOpcode::ImageSampleRaw) { Builder b; b.program.Resources().stage = stage; MemoryInfo memory; @@ -608,7 +608,7 @@ int main() { MemoryFlags flags{0u, 0u}; std::uint64_t bits = 0; std::memcpy(&bits, &flags, sizeof(flags)); - auto& sampled = b.emit(IrOpcode::ImageSampleRaw, IrType::U32x4, {&image, &sampler, &address}, bits); + auto& sampled = b.emit(opcode, IrType::U32x4, {&image, &sampler, &address}, bits); b.keep(b.select(exec, b.emit(IrOpcode::CompositeExtractU32x4, IrType::U32, {&sampled, &b.constant(0u)}), old)); b.eliminate(); return removed(written); @@ -622,6 +622,26 @@ int main() { return sample(IrShaderStage::Pixel, RdnaImageSampleFlagLevelZero) && sample(IrShaderStage::Mesh, 0u); }); + passed &= run("a pixel gather without an LOD of its own takes it from the quad and keeps the select", [&] { + return !sample(IrShaderStage::Pixel, 0u, IrOpcode::ImageGatherRaw) && !sample(IrShaderStage::Pixel, RdnaImageSampleFlagBias, IrOpcode::ImageGatherRaw); + }); + + passed &= run("a gather with an explicit LOD reads only its lane and drops the select", [&] { + return sample(IrShaderStage::Pixel, RdnaImageSampleFlagLevelZero, IrOpcode::ImageGatherRaw) && sample(IrShaderStage::Pixel, RdnaImageSampleFlagLod, IrOpcode::ImageGatherRaw) && + sample(IrShaderStage::Mesh, 0u, IrOpcode::ImageGatherRaw); + }); + + passed &= run("an image size query reads only its lane's level and drops the select", [] { + Builder b; + auto& exec = b.mask(16u); + auto& old = b.lane(); + auto& written = b.select(exec, b.add(old, 1u), old); + auto& address = b.emit(IrOpcode::MakeImageAddress, IrType::ImageAddress, {&written}); + auto& size = b.emit(IrOpcode::ImageQueryDimensions, IrType::U32x4, {&b.emit(IrOpcode::GetImageResource, IrType::ImageResource, {}), &address}); + b.keep(b.select(exec, b.emit(IrOpcode::CompositeExtractU32x4, IrType::U32, {&size, &b.constant(0u)}), old)); + return b.eliminate() == 1u && removed(written); + }); + const auto guarded = [](IrOpcode opcode, bool sameExec, bool asActive) { Builder b; auto& wide = b.mask(48u); @@ -643,6 +663,12 @@ int main() { b.keep(b.select(exec, b.emit(IrOpcode::CompositeExtractU32x4, IrType::U32, {&read, &b.constant(0u)}), old)); break; } + case IrOpcode::ImageWrite: { + auto& address = b.emit(IrOpcode::MakeImageAddress, IrType::ImageAddress, {&old, &old}); + auto& texel = b.emit(IrOpcode::CompositeConstructU32x4, IrType::U32x4, {&written, &old, &old, &old}); + b.emit(opcode, IrType::Void, {&b.emit(IrOpcode::GetImageResource, IrType::ImageResource, {}), &address, &texel, &active}); + break; + } default: b.emit(opcode, IrType::Void, {&b.emit(IrOpcode::CompositeConstructU32x4, IrType::U32x4, {&written, &old, &old, &old}), &active}); break; @@ -659,6 +685,10 @@ int main() { return !guarded(IrOpcode::WriteSharedU32, false, false) && !guarded(IrOpcode::LoadBufferU32, false, false) && !guarded(IrOpcode::ImageRead, false, false) && !guarded(IrOpcode::SetAttribute, false, false); }); + passed &= run("an image write under the write's exec drops the select, under a wider exec or as its exec keeps it", [&] { + return guarded(IrOpcode::ImageWrite, true, false) && !guarded(IrOpcode::ImageWrite, false, false) && !guarded(IrOpcode::ImageWrite, true, true); + }); + passed &= run("a value used as a store's exec keeps the select", [&] { return !guarded(IrOpcode::WriteSharedU32, true, true); }); diff --git a/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp b/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp index 30331f633..e500ead2a 100644 --- a/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp +++ b/core/shader/recompiler/Optimization/src/MaskedSelectEliminator.cpp @@ -171,6 +171,7 @@ bool isLaneLocal(IrOpcode opcode) { case IrOpcode::FPFrexpMant64: case IrOpcode::FPFrexpExp64: case IrOpcode::MakeImageAddress: + case IrOpcode::ImageQueryDimensions: return true; default: return false; @@ -205,6 +206,7 @@ bool readsOnlyWhereActive(IrOpcode opcode) { case IrOpcode::WriteSharedU32x4: case IrOpcode::SetAttribute: case IrOpcode::ImageRead: + case IrOpcode::ImageWrite: return true; default: return false; @@ -212,7 +214,7 @@ bool readsOnlyWhereActive(IrOpcode opcode) { } bool isExplicitLodSample(const IrProgram& program, const IrValue& value) { - if (value.Opcode() != IrOpcode::ImageSampleRaw) return false; + if (value.Opcode() != IrOpcode::ImageSampleRaw && value.Opcode() != IrOpcode::ImageGatherRaw) return false; if (program.Resources().stage != IrShaderStage::Pixel) return true; const auto index = value.Flags().index; if (index >= program.Resources().memoryInfo.size()) return false; From 007cc62555beff1067f0c2d1ae1bbb69ff36789b Mon Sep 17 00:00:00 2001 From: Fishybeing <328519762+Fishybeing@users.noreply.github.com> Date: Sun, 11 Oct 2026 00:23:45 +0200 Subject: [PATCH 354/460] fix(libc): follow FreeBSD's memset_s errors FreeBSD's memset_s returns EINVAL for every constraint violation and writes nothing when smax or n exceeds RSIZE_MAX. The guest returned ERANGE when n exceeded smax and wrote up to min(n, smax) bytes even when smax or n exceeded RSIZE_MAX. (cherry picked from commit bf08837cf929e40f1b646df82e140a33076d5587) --- core/libs/prx/libc/src/Strings.cpp | 6 +++--- core/libs/tests/GuestStrings.cpp | 8 +++++++- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libc/src/Strings.cpp b/core/libs/prx/libc/src/Strings.cpp index 4399cbe08..d301d270e 100644 --- a/core/libs/prx/libc/src/Strings.cpp +++ b/core/libs/prx/libc/src/Strings.cpp @@ -323,12 +323,12 @@ int APS5_VABI memmove_s_nid_postfix(void* dest, size_t destsz, const void* src, int APS5_VABI memset_s_nid_postfix(void* dest, size_t destsz, int value, size_t count) { constexpr int GuestEinval = 22; - constexpr int GuestErange = 34; - if (!dest) return GuestEinval; + constexpr size_t RsizeMax = SIZE_MAX >> 1; + if (!dest || destsz > RsizeMax || count > RsizeMax) return GuestEinval; const auto length = count > destsz ? destsz : count; auto* bytes = static_cast(dest); for (size_t index = 0; index < length; ++index) bytes[index] = static_cast(value); - return count > destsz ? GuestErange : 0; + return count > destsz ? GuestEinval : 0; } char* APS5_VABI strnstr_nid_postfix(const char* haystack, const char* needle, size_t length) { diff --git a/core/libs/tests/GuestStrings.cpp b/core/libs/tests/GuestStrings.cpp index c7b048a8a..3cefefffd 100644 --- a/core/libs/tests/GuestStrings.cpp +++ b/core/libs/tests/GuestStrings.cpp @@ -82,7 +82,13 @@ static void CheckBoundsCheckedFunctions() { Require(memcpy_s_nid_postfix(bytes, 2, source, 4) == 34 && bytes[0] == 0 && bytes[1] == 0 && bytes[2] == 3); char overlap[6] = "abcde"; Require(memmove_s_nid_postfix(overlap + 1, 5, overlap, 3) == 0 && std::strcmp(overlap, "aabce") == 0); - Require(memset_s_nid_postfix(bytes, sizeof(bytes), 9, 8) == 34 && bytes[3] == 9); + Require(memset_s_nid_postfix(bytes, sizeof(bytes), 9, 8) == 22 && bytes[0] == 9 && bytes[3] == 9); + char sentinel[4] = {1, 2, 3, 4}; + Require(memset_s_nid_postfix(sentinel, sizeof(sentinel), 7, 2) == 0 && sentinel[1] == 7 && sentinel[2] == 3); + Require(memset_s_nid_postfix(sentinel, (SIZE_MAX >> 1) + 1, 0, 1) == 22 && sentinel[0] == 7); + Require(memset_s_nid_postfix(sentinel, sizeof(sentinel), 0, (SIZE_MAX >> 1) + 1) == 22 && sentinel[0] == 7 && sentinel[3] == 4); + Require(memset_s_nid_postfix(sentinel, SIZE_MAX >> 1, 5, 1) == 0 && sentinel[0] == 5 && sentinel[1] == 7); + Require(memset_s_nid_postfix(nullptr, sizeof(sentinel), 0, 1) == 22); Require(strnstr_nid_postfix("haystack", "st", 4) == nullptr); const char haystack[] = "haystack"; Require(strnstr_nid_postfix(haystack, "st", 6) == haystack + 3); From 865e530505ea0fe0984bba8de95c5dcf62dfdf16 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Thu, 8 Oct 2026 18:39:35 +0000 Subject: [PATCH 355/460] feat(libc): implement dirname --- core/libs/prx/libc/src/StringExtensions.cpp | 24 +++++++++++++++++++++ core/libs/tests/GuestStrings.cpp | 14 ++++++++++++ 2 files changed, 38 insertions(+) diff --git a/core/libs/prx/libc/src/StringExtensions.cpp b/core/libs/prx/libc/src/StringExtensions.cpp index 155ec1443..178bd0218 100644 --- a/core/libs/prx/libc/src/StringExtensions.cpp +++ b/core/libs/prx/libc/src/StringExtensions.cpp @@ -37,6 +37,30 @@ char* APS5_VABI __inet_ntoa_nid_postfix(unsigned int address) { return buffer; } +char* APS5_VABI dirname_nid_postfix(char* path) { + thread_local char dot[] = "."; + thread_local char slash[] = "/"; + if (path == nullptr || *path == '\0') { + dot[0] = '.'; + return dot; + } + + auto length = std::strlen(path); + while (length > 0 && path[length - 1] == '/') --length; + if (length == 0) return slash; + path[length] = '\0'; + + auto separator = length; + while (separator > 0 && path[separator - 1] != '/') --separator; + if (separator == 0) { + dot[0] = '.'; + return dot; + } + while (separator > 1 && path[separator - 1] == '/') --separator; + path[separator] = '\0'; + return path; +} + std::size_t APS5_VABI strnlen_nid_postfix(const char* text, std::size_t limit) { std::size_t length = 0; while (length < limit && text[length] != '\0') ++length; diff --git a/core/libs/tests/GuestStrings.cpp b/core/libs/tests/GuestStrings.cpp index c7b048a8a..31194f5e4 100644 --- a/core/libs/tests/GuestStrings.cpp +++ b/core/libs/tests/GuestStrings.cpp @@ -9,6 +9,7 @@ extern "C" { char* APS5_VABI basename_nid_postfix(const char*); char* APS5_VABI __inet_ntoa_nid_postfix(unsigned int); +char* APS5_VABI dirname_nid_postfix(char*); int* APS5_VABI __error_nid_postfix(); std::size_t APS5_VABI strnlen_nid_postfix(const char*, std::size_t); std::size_t APS5_VABI strnlen_s_nid_postfix(const char*, std::size_t); @@ -114,6 +115,19 @@ static void CheckSscanfS() { int main() { CheckBoundsCheckedFunctions(); CheckSscanfS(); + Require(std::strcmp(dirname_nid_postfix(nullptr), ".") == 0); + char emptyPath[] = ""; + Require(std::strcmp(dirname_nid_postfix(emptyPath), ".") == 0); + char leaf[] = "file"; + Require(std::strcmp(dirname_nid_postfix(leaf), ".") == 0); + char nested[] = "one/two///"; + Require(std::strcmp(dirname_nid_postfix(nested), "one") == 0 && std::strcmp(nested, "one") == 0); + char absolute[] = "/one/two"; + Require(std::strcmp(dirname_nid_postfix(absolute), "/one") == 0); + char rootChild[] = "/file"; + Require(std::strcmp(dirname_nid_postfix(rootChild), "/") == 0); + char onlySlashes[] = "////"; + Require(std::strcmp(dirname_nid_postfix(onlySlashes), "/") == 0); Require(std::strcmp(basename_nid_postfix(nullptr), ".") == 0); Require(std::strcmp(basename_nid_postfix(""), ".") == 0); Require(std::strcmp(basename_nid_postfix("////"), "/") == 0); From 7d7b405b3ca810fc97dfb91b78de2b8751ede925 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sun, 11 Oct 2026 04:41:32 +0000 Subject: [PATCH 356/460] test(libc): cover dirname separator and boundary cases --- core/libs/tests/GuestStrings.cpp | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/core/libs/tests/GuestStrings.cpp b/core/libs/tests/GuestStrings.cpp index 31194f5e4..dd6fa6054 100644 --- a/core/libs/tests/GuestStrings.cpp +++ b/core/libs/tests/GuestStrings.cpp @@ -122,6 +122,14 @@ int main() { Require(std::strcmp(dirname_nid_postfix(leaf), ".") == 0); char nested[] = "one/two///"; Require(std::strcmp(dirname_nid_postfix(nested), "one") == 0 && std::strcmp(nested, "one") == 0); + char repeatedSeparators[] = "one//two"; + Require(std::strcmp(dirname_nid_postfix(repeatedSeparators), "one") == 0); + char absoluteRepeatedSeparators[] = "/one//two"; + Require(std::strcmp(dirname_nid_postfix(absoluteRepeatedSeparators), "/one") == 0); + char rootPath[] = "/"; + Require(std::strcmp(dirname_nid_postfix(rootPath), "/") == 0); + char trailingSlash[] = "foo/"; + Require(std::strcmp(dirname_nid_postfix(trailingSlash), ".") == 0); char absolute[] = "/one/two"; Require(std::strcmp(dirname_nid_postfix(absolute), "/one") == 0); char rootChild[] = "/file"; From 6c87593a16164ee1961e1161cc5289aa43b9d53a Mon Sep 17 00:00:00 2001 From: Hussein Kadhim Date: Sat, 10 Oct 2026 14:14:47 +0300 Subject: [PATCH 357/460] fix(libc): return LONG_MIN from lround, lroundf and llround out of range FreeBSD amd64 converts the rounded value with a plain cast, so NaN, infinities and values outside the 64-bit range give 0x8000000000000000 with FE_INVALID raised. The Windows host llround returns 0 for them. (cherry picked from commit 0ee2741534bdfcc9d818b1d93f2eb793a68f111e) --- core/libs/prx/libc/src/Math.cpp | 19 ++++++++++++++++--- core/libs/tests/GuestMath.cpp | 16 ++++++++++++++++ 2 files changed, 32 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libc/src/Math.cpp b/core/libs/prx/libc/src/Math.cpp index 0f9236c92..ba1271b24 100644 --- a/core/libs/prx/libc/src/Math.cpp +++ b/core/libs/prx/libc/src/Math.cpp @@ -1,5 +1,6 @@ #include #include +#include #include #include #include @@ -8,6 +9,18 @@ #include "prx/libc/include/General.hpp" +namespace { + +template +std::int64_t RoundToInt64(TFloat value) { + const std::int64_t native = std::llround(value); + const TFloat rounded = std::round(value); + const bool inRange = rounded >= static_cast(INT64_MIN) && rounded < static_cast(INT64_MAX); + return inRange ? native : INT64_MIN; +} + +} + extern "C" { std::lldiv_t APS5_VABI lldiv_nid_postfix(long long numerator, long long denominator) { @@ -34,9 +47,9 @@ float APS5_VABI scalbnf_nid_postfix(float x, int exponent) { return std::scalbn( double APS5_VABI frexp_nid_postfix(double x, int* exponent) { return std::frexp(x, exponent); } float APS5_VABI frexpf_nid_postfix(float x, int* exponent) { return std::frexp(x, exponent); } // Guest long is 64-bit, including on Windows where native long is 32-bit. -std::int64_t APS5_VABI lround_nid_postfix(double x) { return std::llround(x); } -std::int64_t APS5_VABI lroundf_nid_postfix(float x) { return std::llround(x); } -std::int64_t APS5_VABI llround_nid_postfix(double x) { return std::llround(x); } +std::int64_t APS5_VABI lround_nid_postfix(double x) { return RoundToInt64(x); } +std::int64_t APS5_VABI lroundf_nid_postfix(float x) { return RoundToInt64(x); } +std::int64_t APS5_VABI llround_nid_postfix(double x) { return RoundToInt64(x); } std::int64_t APS5_VABI llroundf_nid_postfix(float x) { return std::llround(x); } int APS5_VABI __isfinitef_nid_postfix(float x) { return std::isfinite(x) ? 1 : 0; } int APS5_VABI __isnormal_nid_postfix(double x) { return std::isnormal(x) ? 1 : 0; } diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index ec4d5e4a7..9e1e56a1f 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -324,6 +324,22 @@ int main() { Require(llroundf_nid_postfix(2.5f) == 3 && llroundf_nid_postfix(-2.5f) == -3 && llroundf_nid_postfix(-0.4f) == 0); Require(llroundf_nid_postfix(8589934592.f) == INT64_C(8589934592) && llroundf_nid_postfix(-0x1p62f) == -(INT64_C(1) << 62)); Require(llroundf_nid_postfix(0.49999997f) == 0 && llroundf_nid_postfix(16777215.f) == 16777215); + Require(lround_nid_postfix(0.49999999999999994) == 0 && lround_nid_postfix(-0.49999999999999994) == 0); + Require(lround_nid_postfix(-9223372036854775808.0) == INT64_MIN); + Require(lround_nid_postfix(9223372036854774784.0) == INT64_C(9223372036854774784)); + Require(lroundf_nid_postfix(9223371487098961920.f) == INT64_C(9223371487098961920)); + std::feclearexcept(FE_ALL_EXCEPT); + Require(lround_nid_postfix(4503599627370497.0) == INT64_C(4503599627370497) && std::fetestexcept(FE_INVALID) == 0); + const double outOfRange[] = {9223372036854775808.0, -9223374235878031360.0, 1e30, -1e30, std::numeric_limits::infinity(), -std::numeric_limits::infinity(), std::numeric_limits::quiet_NaN()}; + for (const double value : outOfRange) { + std::feclearexcept(FE_ALL_EXCEPT); + Require(lround_nid_postfix(value) == INT64_MIN && std::fetestexcept(FE_INVALID) != 0); + std::feclearexcept(FE_ALL_EXCEPT); + Require(llround_nid_postfix(value) == INT64_MIN && std::fetestexcept(FE_INVALID) != 0); + std::feclearexcept(FE_ALL_EXCEPT); + Require(lroundf_nid_postfix(static_cast(value)) == INT64_MIN && std::fetestexcept(FE_INVALID) != 0); + } + std::feclearexcept(FE_ALL_EXCEPT); const auto infinity = std::numeric_limits::infinity(); const auto nan = std::numeric_limits::quiet_NaN(); Require(__isinff_nid_postfix(infinity) == 1 && __isinff_nid_postfix(-infinity) == 1); From 03dde075c9a819fce88e01c4d6de569f8088e9e6 Mon Sep 17 00:00:00 2001 From: Hussein Kadhim Date: Sat, 10 Oct 2026 14:40:59 +0300 Subject: [PATCH 358/460] fix(libc): raise FE_INVALID explicitly and leave errno untouched in lround family (cherry picked from commit a56de63b8ae0e4bef888ec24ed7f737ab9cab9f7) --- core/libs/prx/libc/src/Math.cpp | 8 ++++-- core/libs/tests/GuestMath.cpp | 49 ++++++++++++++++++++++++--------- 2 files changed, 41 insertions(+), 16 deletions(-) diff --git a/core/libs/prx/libc/src/Math.cpp b/core/libs/prx/libc/src/Math.cpp index ba1271b24..807d6116b 100644 --- a/core/libs/prx/libc/src/Math.cpp +++ b/core/libs/prx/libc/src/Math.cpp @@ -13,10 +13,12 @@ namespace { template std::int64_t RoundToInt64(TFloat value) { - const std::int64_t native = std::llround(value); const TFloat rounded = std::round(value); - const bool inRange = rounded >= static_cast(INT64_MIN) && rounded < static_cast(INT64_MAX); - return inRange ? native : INT64_MIN; + if (!(rounded >= static_cast(INT64_MIN) && rounded < static_cast(INT64_MAX))) { + std::feraiseexcept(FE_INVALID); + return INT64_MIN; + } + return static_cast(rounded); } } diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index 9e1e56a1f..69fa3c362 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -1,5 +1,6 @@ #include "prx/libc/include/general/VabiMacros.hpp" #include +#include #include #include #include @@ -260,6 +261,26 @@ static void CheckFloatClassification() { Require(_FDtest_nid_postfix(reinterpret_cast(&_FNan_nid_postfix)) == 2); } +static void CheckDoubleRounding(double value, std::int64_t expected, bool invalid) { + for (const bool useLlround : {false, true}) { + std::feclearexcept(FE_ALL_EXCEPT); + errno = 12345; + const std::int64_t result = useLlround ? llround_nid_postfix(value) : lround_nid_postfix(value); + Require(result == expected); + Require((std::fetestexcept(FE_INVALID) != 0) == invalid); + Require(errno == 12345); + } +} + +static void CheckFloatRounding(float value, std::int64_t expected, bool invalid) { + std::feclearexcept(FE_ALL_EXCEPT); + errno = 12345; + const std::int64_t result = lroundf_nid_postfix(value); + Require(result == expected); + Require((std::fetestexcept(FE_INVALID) != 0) == invalid); + Require(errno == 12345); +} + int main() { CheckFloatClassification(); CheckIntegerConversions(); @@ -325,20 +346,22 @@ int main() { Require(llroundf_nid_postfix(8589934592.f) == INT64_C(8589934592) && llroundf_nid_postfix(-0x1p62f) == -(INT64_C(1) << 62)); Require(llroundf_nid_postfix(0.49999997f) == 0 && llroundf_nid_postfix(16777215.f) == 16777215); Require(lround_nid_postfix(0.49999999999999994) == 0 && lround_nid_postfix(-0.49999999999999994) == 0); - Require(lround_nid_postfix(-9223372036854775808.0) == INT64_MIN); - Require(lround_nid_postfix(9223372036854774784.0) == INT64_C(9223372036854774784)); - Require(lroundf_nid_postfix(9223371487098961920.f) == INT64_C(9223371487098961920)); - std::feclearexcept(FE_ALL_EXCEPT); - Require(lround_nid_postfix(4503599627370497.0) == INT64_C(4503599627370497) && std::fetestexcept(FE_INVALID) == 0); - const double outOfRange[] = {9223372036854775808.0, -9223374235878031360.0, 1e30, -1e30, std::numeric_limits::infinity(), -std::numeric_limits::infinity(), std::numeric_limits::quiet_NaN()}; - for (const double value : outOfRange) { - std::feclearexcept(FE_ALL_EXCEPT); - Require(lround_nid_postfix(value) == INT64_MIN && std::fetestexcept(FE_INVALID) != 0); - std::feclearexcept(FE_ALL_EXCEPT); - Require(llround_nid_postfix(value) == INT64_MIN && std::fetestexcept(FE_INVALID) != 0); - std::feclearexcept(FE_ALL_EXCEPT); - Require(lroundf_nid_postfix(static_cast(value)) == INT64_MIN && std::fetestexcept(FE_INVALID) != 0); + constexpr double kTwo63 = 9223372036854775808.0; + constexpr double kTwo63Below = 9223372036854774784.0; + constexpr float kTwo63f = 9223372036854775808.f; + constexpr float kTwo63fBelow = 9223371487098961920.f; + CheckDoubleRounding(-kTwo63, INT64_MIN, false); + CheckDoubleRounding(kTwo63Below, INT64_C(9223372036854774784), false); + CheckDoubleRounding(-kTwo63Below, -INT64_C(9223372036854774784), false); + CheckDoubleRounding(4503599627370497.0, INT64_C(4503599627370497), false); + CheckFloatRounding(-kTwo63f, INT64_MIN, false); + CheckFloatRounding(kTwo63fBelow, INT64_C(9223371487098961920), false); + CheckFloatRounding(-kTwo63fBelow, -INT64_C(9223371487098961920), false); + for (const double value : {kTwo63, -9223374235878031360.0, 1e30, -1e30, std::numeric_limits::infinity(), -std::numeric_limits::infinity(), std::numeric_limits::quiet_NaN()}) { + CheckDoubleRounding(value, INT64_MIN, true); + CheckFloatRounding(static_cast(value), INT64_MIN, true); } + CheckFloatRounding(kTwo63f, INT64_MIN, true); std::feclearexcept(FE_ALL_EXCEPT); const auto infinity = std::numeric_limits::infinity(); const auto nan = std::numeric_limits::quiet_NaN(); From fcc3145abf5581badc4379f512687d9f7febe234 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 21:32:08 +0000 Subject: [PATCH 359/460] fix(libc): apply range handling to llroundf The original fix predates llroundf, which was added in #2933. Use the same guest range and exception behavior for the fourth wrapper and include it in the edge-case checks. --- core/libs/prx/libc/src/Math.cpp | 2 +- core/libs/tests/GuestMath.cpp | 14 ++++++++------ 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/core/libs/prx/libc/src/Math.cpp b/core/libs/prx/libc/src/Math.cpp index 807d6116b..7d5fc2161 100644 --- a/core/libs/prx/libc/src/Math.cpp +++ b/core/libs/prx/libc/src/Math.cpp @@ -52,7 +52,7 @@ float APS5_VABI frexpf_nid_postfix(float x, int* exponent) { return std::frexp(x std::int64_t APS5_VABI lround_nid_postfix(double x) { return RoundToInt64(x); } std::int64_t APS5_VABI lroundf_nid_postfix(float x) { return RoundToInt64(x); } std::int64_t APS5_VABI llround_nid_postfix(double x) { return RoundToInt64(x); } -std::int64_t APS5_VABI llroundf_nid_postfix(float x) { return std::llround(x); } +std::int64_t APS5_VABI llroundf_nid_postfix(float x) { return RoundToInt64(x); } int APS5_VABI __isfinitef_nid_postfix(float x) { return std::isfinite(x) ? 1 : 0; } int APS5_VABI __isnormal_nid_postfix(double x) { return std::isnormal(x) ? 1 : 0; } int APS5_VABI __isnormalf_nid_postfix(float x) { return std::isnormal(x) ? 1 : 0; } diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index 69fa3c362..ec8080d0a 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -273,12 +273,14 @@ static void CheckDoubleRounding(double value, std::int64_t expected, bool invali } static void CheckFloatRounding(float value, std::int64_t expected, bool invalid) { - std::feclearexcept(FE_ALL_EXCEPT); - errno = 12345; - const std::int64_t result = lroundf_nid_postfix(value); - Require(result == expected); - Require((std::fetestexcept(FE_INVALID) != 0) == invalid); - Require(errno == 12345); + for (const bool useLlround : {false, true}) { + std::feclearexcept(FE_ALL_EXCEPT); + errno = 12345; + const std::int64_t result = useLlround ? llroundf_nid_postfix(value) : lroundf_nid_postfix(value); + Require(result == expected); + Require((std::fetestexcept(FE_INVALID) != 0) == invalid); + Require(errno == 12345); + } } int main() { From bde216731e4efd6c49021c0f22add8679b4e4182 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sun, 11 Oct 2026 05:12:21 +0000 Subject: [PATCH 360/460] fix(libc): preserve dirname input paths --- core/libs/prx/libc/src/StringExtensions.cpp | 38 +++++++++++++-------- core/libs/tests/GuestStrings.cpp | 11 +++--- 2 files changed, 30 insertions(+), 19 deletions(-) diff --git a/core/libs/prx/libc/src/StringExtensions.cpp b/core/libs/prx/libc/src/StringExtensions.cpp index 178bd0218..0e26972f8 100644 --- a/core/libs/prx/libc/src/StringExtensions.cpp +++ b/core/libs/prx/libc/src/StringExtensions.cpp @@ -37,28 +37,38 @@ char* APS5_VABI __inet_ntoa_nid_postfix(unsigned int address) { return buffer; } -char* APS5_VABI dirname_nid_postfix(char* path) { - thread_local char dot[] = "."; - thread_local char slash[] = "/"; +char* APS5_VABI dirname_nid_postfix(const char* path) { + thread_local char buffer[1024]; if (path == nullptr || *path == '\0') { - dot[0] = '.'; - return dot; + buffer[0] = '.'; + buffer[1] = '\0'; + return buffer; } auto length = std::strlen(path); - while (length > 0 && path[length - 1] == '/') --length; - if (length == 0) return slash; - path[length] = '\0'; + if (length >= sizeof(buffer)) { + errno = 63; + return nullptr; + } + std::memcpy(buffer, path, length + 1); + while (length > 0 && buffer[length - 1] == '/') --length; + if (length == 0) { + buffer[0] = '/'; + buffer[1] = '\0'; + return buffer; + } + buffer[length] = '\0'; auto separator = length; - while (separator > 0 && path[separator - 1] != '/') --separator; + while (separator > 0 && buffer[separator - 1] != '/') --separator; if (separator == 0) { - dot[0] = '.'; - return dot; + buffer[0] = '.'; + buffer[1] = '\0'; + return buffer; } - while (separator > 1 && path[separator - 1] == '/') --separator; - path[separator] = '\0'; - return path; + while (separator > 1 && buffer[separator - 1] == '/') --separator; + buffer[separator] = '\0'; + return buffer; } std::size_t APS5_VABI strnlen_nid_postfix(const char* text, std::size_t limit) { diff --git a/core/libs/tests/GuestStrings.cpp b/core/libs/tests/GuestStrings.cpp index dd6fa6054..9222cae2d 100644 --- a/core/libs/tests/GuestStrings.cpp +++ b/core/libs/tests/GuestStrings.cpp @@ -9,7 +9,7 @@ extern "C" { char* APS5_VABI basename_nid_postfix(const char*); char* APS5_VABI __inet_ntoa_nid_postfix(unsigned int); -char* APS5_VABI dirname_nid_postfix(char*); +char* APS5_VABI dirname_nid_postfix(const char*); int* APS5_VABI __error_nid_postfix(); std::size_t APS5_VABI strnlen_nid_postfix(const char*, std::size_t); std::size_t APS5_VABI strnlen_s_nid_postfix(const char*, std::size_t); @@ -118,10 +118,9 @@ int main() { Require(std::strcmp(dirname_nid_postfix(nullptr), ".") == 0); char emptyPath[] = ""; Require(std::strcmp(dirname_nid_postfix(emptyPath), ".") == 0); - char leaf[] = "file"; - Require(std::strcmp(dirname_nid_postfix(leaf), ".") == 0); - char nested[] = "one/two///"; - Require(std::strcmp(dirname_nid_postfix(nested), "one") == 0 && std::strcmp(nested, "one") == 0); + Require(std::strcmp(dirname_nid_postfix("file"), ".") == 0); + const char nested[] = "one/two///"; + Require(std::strcmp(dirname_nid_postfix(nested), "one") == 0 && std::strcmp(nested, "one/two///") == 0); char repeatedSeparators[] = "one//two"; Require(std::strcmp(dirname_nid_postfix(repeatedSeparators), "one") == 0); char absoluteRepeatedSeparators[] = "/one//two"; @@ -136,6 +135,8 @@ int main() { Require(std::strcmp(dirname_nid_postfix(rootChild), "/") == 0); char onlySlashes[] = "////"; Require(std::strcmp(dirname_nid_postfix(onlySlashes), "/") == 0); + const std::string longDirnamePath(1024, 'x'); + Require(dirname_nid_postfix(longDirnamePath.c_str()) == nullptr && *__error_nid_postfix() == 63); Require(std::strcmp(basename_nid_postfix(nullptr), ".") == 0); Require(std::strcmp(basename_nid_postfix(""), ".") == 0); Require(std::strcmp(basename_nid_postfix("////"), "/") == 0); From d041c183df18cf45cf94f753c8f5e510a02de9ee Mon Sep 17 00:00:00 2001 From: krishnaraya Date: Sun, 11 Oct 2026 01:21:40 -0400 Subject: [PATCH 361/460] test(agc): skip flat aperture 64-bit atomic rows without shaderBufferInt64Atomics Co-Authored-By: Claude Opus 5.5 (1M context) --- .../libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp | 3 +++ 1 file changed, 3 insertions(+) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp index 2fb3c5932..c404a90f9 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/FlatApertures.cpp @@ -494,16 +494,19 @@ int main(int argc, char** argv) { const auto device = OpenVulkanTestDevice(); if (!device) return VulkanTestSkipped; const bool wave64 = device->Target().subgroupSize >= 32u; + const bool int64Atomics = TargetHasCapability(device->Target(), spv::CapabilityInt64Atomics); GuestBlock guest; std::size_t checked = 0; const auto rows = Rows(); for (std::size_t index = part; index < rows.size(); index += parts) { const Row& row = rows[index]; if (row.waveSize == 64u && !wave64) continue; + if (row.atomic && row.width == 64u && !int64Atomics) continue; Run(*device, row, guest); ++checked; } if (!wave64) std::printf("skipped the wave64 rows, subgroup size %u cannot hold a wave64 in two lanes\n", device->Target().subgroupSize); + if (!int64Atomics) std::puts("skipped the 64-bit atomic rows, the device has no shaderBufferInt64Atomics"); std::printf("flat aperture rows checked: %zu\n", checked); std::puts("flat aperture tests passed"); return 0; From 68900523c416676dd0959c3799cea1e5f8139662 Mon Sep 17 00:00:00 2001 From: tototomate123 Date: Thu, 8 Oct 2026 16:47:53 +0200 Subject: [PATCH 362/460] fix(apr): report guest errors and clear failed file resolutions --- core/libs/prx/libkernel/Apr/src/Apr.cpp | 13 +++++++++++-- core/libs/tests/GuestAmpr.cpp | 21 +++++++++++++++++++++ 2 files changed, 32 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libkernel/Apr/src/Apr.cpp b/core/libs/prx/libkernel/Apr/src/Apr.cpp index 1417f6fd4..82dafbcb5 100644 --- a/core/libs/prx/libkernel/Apr/src/Apr.cpp +++ b/core/libs/prx/libkernel/Apr/src/Apr.cpp @@ -30,6 +30,8 @@ #include #endif +extern "C" int* APS5_VABI __error_nid_postfix(); + namespace { constexpr int GUEST_ENOENT = 2; @@ -99,7 +101,7 @@ bool _fileSize(const std::filesystem::path& path, std::uint64_t& bytes) { } int _fail(int guestErrno) { - errno = guestErrno; + *__error_nid_postfix() = guestErrno; return -1; } @@ -131,6 +133,7 @@ constexpr int SCE_KERNEL_ERROR_ENOENT = static_cast(0x80020002); int _resolveForEach(const char* prefix, const char** paths, uint32_t count, uint32_t* ids, uint64_t* sizes, int* results) { if (!paths || !ids) return _fail(GUEST_EINVAL); + std::uint32_t resolvedCount = 0; for (uint32_t index = 0; index < count; ++index) { const std::string path = prefix ? (paths[index] ? std::string(prefix) + paths[index] : std::string()) : (paths[index] ? std::string(paths[index]) : std::string()); const bool resolved = !path.empty() && _resolve(path.c_str(), &ids[index], sizes ? &sizes[index] : nullptr); @@ -138,9 +141,10 @@ int _resolveForEach(const char* prefix, const char** paths, uint32_t count, uint ids[index] = INVALID_FILE_ID; if (sizes) sizes[index] = 0; } + if (resolved) ++resolvedCount; if (results) results[index] = resolved ? 0 : SCE_KERNEL_ERROR_ENOENT; } - return 0; + return static_cast(resolvedCount); } void _readFile(const Apr::ReadFileCommand& command) { @@ -657,6 +661,7 @@ int APS5_VABI sceKernelAprResolveFilepathsToIds(const char** paths, uint32_t cou if (!paths || !ids) return _fail(GUEST_EINVAL); for (uint32_t index = 0; index < count; ++index) { if (!_resolve(paths[index], &ids[index], nullptr)) { + ids[index] = INVALID_FILE_ID; if (error_index) *error_index = index; return _fail(GUEST_ENOENT); } @@ -668,6 +673,8 @@ int APS5_VABI sceKernelAprResolveFilepathsToIdsAndFileSizes(const char** paths, if (!paths || !ids || !sizes) return _fail(GUEST_EINVAL); for (uint32_t index = 0; index < count; ++index) { if (!_resolve(paths[index], &ids[index], &sizes[index])) { + ids[index] = INVALID_FILE_ID; + if (sizes) sizes[index] = 0; if (error_index) *error_index = index; return _fail(GUEST_ENOENT); } @@ -680,6 +687,8 @@ int APS5_VABI sceKernelAprResolveFilepathsWithPrefixToIdsAndFileSizes(const char for (uint32_t index = 0; index < count; ++index) { const std::string path = paths[index] ? std::string(prefix) + paths[index] : std::string(); if (path.empty() || !_resolve(path.c_str(), &ids[index], sizes ? &sizes[index] : nullptr)) { + ids[index] = INVALID_FILE_ID; + if (sizes) sizes[index] = 0; if (error_index) *error_index = index; return _fail(GUEST_ENOENT); } diff --git a/core/libs/tests/GuestAmpr.cpp b/core/libs/tests/GuestAmpr.cpp index 7f45f4ff3..7accd9917 100644 --- a/core/libs/tests/GuestAmpr.cpp +++ b/core/libs/tests/GuestAmpr.cpp @@ -59,6 +59,9 @@ std::uint64_t APS5_VABI sceAmprMeasureCommandSizeWriteCounter_04_00(std::uint8_t int APS5_VABI sceAmprCommandBufferConstructNop(Apr::CommandBufferObject*, std::int16_t, const void*, std::uint32_t, const std::uint32_t*); int APS5_VABI sceAmprCommandBufferConstructMarker(Apr::CommandBufferObject*, std::uint32_t, const char*, const std::uint32_t*); int APS5_VABI sceKernelAprResolveFilepathsToIds(const char**, std::uint32_t, std::uint32_t*, std::uint32_t*); +int APS5_VABI sceKernelAprResolveFilepathsToIdsAndFileSizes(const char**, std::uint32_t, std::uint32_t*, std::uint64_t*, std::uint32_t*); +int APS5_VABI sceKernelAprResolveFilepathsToIdsAndFileSizesForEach(const char**, std::uint32_t, std::uint32_t*, std::uint64_t*, int*); +int* APS5_VABI __error_nid_postfix(); int APS5_VABI sceAmprAprCommandBufferReadFile(Apr::CommandBufferObject*, std::uint64_t*, std::uint64_t*, std::uint32_t, void*, std::uint64_t, std::uint64_t); int APS5_VABI sceAmprAprCommandBufferReadFileGather(Apr::CommandBufferObject*, std::uint64_t*, std::uint64_t*, std::uint64_t, std::uint64_t); int APS5_VABI sceAmprAprCommandBufferReadFileScatter(Apr::CommandBufferObject*, std::uint64_t*, std::uint64_t*, void*, std::uint64_t); @@ -552,6 +555,23 @@ void TestZeroFilledBuffer() { std::remove(path); } +void TestResolveFailures() { + const char* paths[]{"ampr_resolve_existing.bin", "ampr_resolve_missing.bin"}; + { std::ofstream file(paths[0], std::ios::binary); file << "data"; } + std::uint32_t ids[]{123, 123}; + std::uint64_t sizes[]{123, 123}; + std::uint32_t failed = 99; + *__error_nid_postfix() = 0; + Require(sceKernelAprResolveFilepathsToIdsAndFileSizes(paths, 2, ids, sizes, &failed) == -1); + Require(*__error_nid_postfix() == 2 && failed == 1 && ids[1] == 0xffffffffu && sizes[1] == 0); + Require(sizes[0] == 4); + int results[]{123, 123}; + Require(sceKernelAprResolveFilepathsToIdsAndFileSizesForEach(paths, 2, ids, sizes, results) == 1); + Require(results[0] == 0 && results[1] == static_cast(0x80020002)); + Require(ids[1] == 0xffffffffu && sizes[0] == 4 && sizes[1] == 0); + std::remove(paths[0]); +} + void TestGatherScatter() { const char* path = "ampr_gather_scatter.bin"; { @@ -988,6 +1008,7 @@ int main() { TestClearBuffer(); TestConstructed(); TestZeroFilledBuffer(); + TestResolveFailures(); TestGatherScatter(); TestAmm(); TestAmmRemapAndProtect(); From f514df98de86cc21abaf56c0ab97cd2f7c0f4a63 Mon Sep 17 00:00:00 2001 From: tototomate123 Date: Thu, 8 Oct 2026 16:47:53 +0200 Subject: [PATCH 363/460] fix(apr): match the guest command buffer object layout --- core/libs/prx/libSceAmpr/Export.cpp | 2 +- .../prx/libkernel/Apr/include/AprCommandBuffer.hpp | 13 ++++++++----- core/libs/prx/libkernel/Apr/src/Apr.cpp | 2 +- core/libs/tests/GuestAmpr.cpp | 2 +- 4 files changed, 11 insertions(+), 8 deletions(-) diff --git a/core/libs/prx/libSceAmpr/Export.cpp b/core/libs/prx/libSceAmpr/Export.cpp index 22af4667a..d169d2498 100644 --- a/core/libs/prx/libSceAmpr/Export.cpp +++ b/core/libs/prx/libSceAmpr/Export.cpp @@ -398,7 +398,7 @@ int APS5_VABI sceAmprCommandBufferConstructNop(Apr::CommandBufferObject* buffer, } int APS5_VABI sceAmprCommandBufferConstructor(Apr::CommandBufferObject* buffer) { - *buffer = {nullptr, 0, 0, 0, Apr::BufferType::Generic}; + *buffer = {Apr::BufferType::Generic, 0, 0, 0, 0, nullptr}; return 0; } diff --git a/core/libs/prx/libkernel/Apr/include/AprCommandBuffer.hpp b/core/libs/prx/libkernel/Apr/include/AprCommandBuffer.hpp index ebd001627..40cdef33b 100644 --- a/core/libs/prx/libkernel/Apr/include/AprCommandBuffer.hpp +++ b/core/libs/prx/libkernel/Apr/include/AprCommandBuffer.hpp @@ -5,7 +5,6 @@ #include // Shared between libSceAmpr (which records commands) and libkernel (which executes them). -// The guest treats both the command buffer object and its memory as opaque, so the encoding is ours. namespace Apr { enum class BufferType : std::uint16_t { @@ -14,15 +13,19 @@ enum class BufferType : std::uint16_t { }; struct CommandBufferObject { - std::uint8_t* base; - std::uint32_t size; - std::uint32_t offset; - std::uint32_t numCommands; BufferType type; std::uint16_t recording; + std::uint32_t offset; + std::uint32_t numCommands; + std::uint32_t size; + std::uint8_t* base; }; static_assert(sizeof(CommandBufferObject) == 0x18, "guest reserves 0x18 bytes for sce::Ampr::CommandBuffer"); +static_assert(offsetof(CommandBufferObject, offset) == 0x04); +static_assert(offsetof(CommandBufferObject, size) == 0x0c); +static_assert(offsetof(CommandBufferObject, base) == 0x10); + inline constexpr std::uint16_t ScatterGatherValid = 1; inline constexpr std::uint16_t MapActive = 2; diff --git a/core/libs/prx/libkernel/Apr/src/Apr.cpp b/core/libs/prx/libkernel/Apr/src/Apr.cpp index 82dafbcb5..58d604628 100644 --- a/core/libs/prx/libkernel/Apr/src/Apr.cpp +++ b/core/libs/prx/libkernel/Apr/src/Apr.cpp @@ -645,7 +645,7 @@ void AmmVirtualAddressRanges_nid_no_patch(std::uint64_t* start, std::uint64_t* e } std::uint32_t AmmSubmit_nid_no_patch(void* base, std::uint32_t bytes) { - _execute(Apr::CommandBufferObject{static_cast(base), bytes, bytes, 0, Apr::BufferType::Generic, 0}); + _execute(Apr::CommandBufferObject{Apr::BufferType::Generic, 0, bytes, 0, bytes, static_cast(base)}); auto& last = _amm().lastSubmit; std::uint32_t id = last.fetch_add(1) + 1; while (id == 0) id = last.fetch_add(1) + 1; diff --git a/core/libs/tests/GuestAmpr.cpp b/core/libs/tests/GuestAmpr.cpp index 7accd9917..0b8f2ec47 100644 --- a/core/libs/tests/GuestAmpr.cpp +++ b/core/libs/tests/GuestAmpr.cpp @@ -980,7 +980,7 @@ void TestAmmPrt() { Require(sceAmprAmmCommandBufferRemapIntoPrt(&unbound, prt, source, page, cpuReadWrite, 0) == permissionDenied); Require(sceAmprAmmCommandBufferUnmapToPrt(&unbound, prt, page) == permissionDenied); std::array loose{}; - Apr::CommandBufferObject sized{nullptr, static_cast(loose.size()), 0, 0, Apr::BufferType::Generic, 0}; + Apr::CommandBufferObject sized{Apr::BufferType::Generic, 0, 0, 0, static_cast(loose.size()), nullptr}; Require(sceAmprAmmCommandBufferMapAsPrt(&sized, prt, page) == permissionDenied); Require(sceAmprAmmMeasureAmmCommandSizeMapAsPrt(prt, 0) == ammRejected); Require(sceAmprAmmMeasureAmmCommandSizeAllocatePaForPrt(prt, page, 0, 0x08) == ammRejected); From 65c8ef656f99f98b619a0fde316c37495bfdf104 Mon Sep 17 00:00:00 2001 From: tototomate123 Date: Thu, 8 Oct 2026 16:47:54 +0200 Subject: [PATCH 364/460] fix(apr): compact gather records to avoid command buffer exhaustion --- core/libs/prx/libSceAmpr/Export.cpp | 8 ++++++-- .../libkernel/Apr/include/AprCommandBuffer.hpp | 12 ++++++++++++ core/libs/prx/libkernel/Apr/src/Apr.cpp | 9 +++++++++ core/libs/tests/GuestAmpr.cpp | 18 +++++++++++++++--- docs/dev/TechnicalDebt.md | 1 + 5 files changed, 43 insertions(+), 5 deletions(-) diff --git a/core/libs/prx/libSceAmpr/Export.cpp b/core/libs/prx/libSceAmpr/Export.cpp index d169d2498..33d351884 100644 --- a/core/libs/prx/libSceAmpr/Export.cpp +++ b/core/libs/prx/libSceAmpr/Export.cpp @@ -337,7 +337,11 @@ int APS5_VABI sceAmprAprCommandBufferReadFileGather(Apr::CommandBufferObject* bu (void)mapState; (void)scatterGatherState; if (!ValidReadLength(size) || !ValidReadOffset(offset)) return SCE_KERNEL_ERROR_EINVAL; - return AppendRead(buffer, Apr::Opcode::ReadFileGather, 0, nullptr, size, offset); + if (!buffer || (buffer->recording & Apr::ScatterGatherValid) == 0u) return SCE_KERNEL_ERROR_EINVAL; + const Apr::CompactReadFileGatherCommand command{ + static_cast(Apr::Opcode::CompactReadFileGather) | static_cast(offset >> 32u) << 8u, + static_cast(size - 1u), static_cast(offset)}; + return Append(buffer, &command, sizeof(command)); } int APS5_VABI sceAmprAprCommandBufferReadFileGatherScatter(Apr::CommandBufferObject* buffer, std::uint64_t* mapState, std::uint64_t* scatterGatherState, void* destination, std::uint64_t size, std::uint64_t offset) { @@ -561,7 +565,7 @@ uint32_t APS5_VABI sceAmprMeasureCommandSizeReadFile(void) { std::uint64_t APS5_VABI sceAmprMeasureCommandSizeReadFileGather(std::uint64_t size, std::uint64_t offset) { if (!ValidReadLength(size) || !ValidReadOffset(offset)) return MeasureInvalid; - return sizeof(Apr::ReadFileCommand); + return sizeof(Apr::CompactReadFileGatherCommand); } std::uint64_t APS5_VABI sceAmprMeasureCommandSizeReadFileGatherScatter(void* destination, std::uint64_t size, std::uint64_t offset) { diff --git a/core/libs/prx/libkernel/Apr/include/AprCommandBuffer.hpp b/core/libs/prx/libkernel/Apr/include/AprCommandBuffer.hpp index 40cdef33b..7b12ec2ea 100644 --- a/core/libs/prx/libkernel/Apr/include/AprCommandBuffer.hpp +++ b/core/libs/prx/libkernel/Apr/include/AprCommandBuffer.hpp @@ -5,6 +5,7 @@ #include // Shared between libSceAmpr (which records commands) and libkernel (which executes them). +// The object layout is guest ABI; command records use the shared host encoding. namespace Apr { enum class BufferType : std::uint16_t { @@ -22,6 +23,9 @@ struct CommandBufferObject { }; static_assert(sizeof(CommandBufferObject) == 0x18, "guest reserves 0x18 bytes for sce::Ampr::CommandBuffer"); +static_assert(offsetof(CommandBufferObject, type) == 0x00); +static_assert(offsetof(CommandBufferObject, recording) == 0x02); +static_assert(offsetof(CommandBufferObject, numCommands) == 0x08); static_assert(offsetof(CommandBufferObject, offset) == 0x04); static_assert(offsetof(CommandBufferObject, size) == 0x0c); static_assert(offsetof(CommandBufferObject, base) == 0x10); @@ -59,6 +63,7 @@ enum class Opcode : std::uint32_t { AmmAllocatePaForPrt = 26, AmmRemapIntoPrt = 27, AmmUnmapToPrt = 28, + CompactReadFileGather = 29, }; struct CommandHeader { @@ -75,6 +80,13 @@ struct ReadFileCommand { std::uint64_t offset; }; +struct CompactReadFileGatherCommand { + std::uint32_t opcodeAndOffsetHigh; + std::uint32_t sizeMinusOne; + std::uint32_t offsetLow; +}; +static_assert(sizeof(CompactReadFileGatherCommand) == 12); + struct ResetGatherScatterStateCommand { CommandHeader header; }; diff --git a/core/libs/prx/libkernel/Apr/src/Apr.cpp b/core/libs/prx/libkernel/Apr/src/Apr.cpp index 58d604628..326a98fb9 100644 --- a/core/libs/prx/libkernel/Apr/src/Apr.cpp +++ b/core/libs/prx/libkernel/Apr/src/Apr.cpp @@ -525,6 +525,15 @@ void _execute(const Apr::CommandBufferObject& buffer) { if (cursor + sizeof(Apr::CommandHeader) > buffer.offset) throw std::runtime_error("APR: truncated command buffer"); Apr::CommandHeader header; std::memcpy(&header, buffer.base + cursor, sizeof(header)); + if ((static_cast(header.opcode) & 0xffu) == static_cast(Apr::Opcode::CompactReadFileGather)) { + if (sizeof(Apr::CompactReadFileGatherCommand) > buffer.offset - cursor) throw std::runtime_error("APR: truncated compact gather command"); + const auto command = _read(buffer, cursor); + const auto offset = static_cast(command.opcodeAndOffsetHigh >> 8u) << 32u | command.offsetLow; + if (offset >= 0x10000000000ull) throw std::runtime_error("APR: malformed compact gather offset"); + _readResolved(Apr::Opcode::ReadFileGather, Apr::ReadFileCommand{{}, 0, 0, 0, static_cast(command.sizeMinusOne) + 1u, offset}, read); + cursor += sizeof(command); + continue; + } if (header.bytes < sizeof(header) || cursor + header.bytes > buffer.offset) throw std::runtime_error("APR: malformed command"); switch (header.opcode) { case Apr::Opcode::Nop: diff --git a/core/libs/tests/GuestAmpr.cpp b/core/libs/tests/GuestAmpr.cpp index 0b8f2ec47..4f040b920 100644 --- a/core/libs/tests/GuestAmpr.cpp +++ b/core/libs/tests/GuestAmpr.cpp @@ -164,7 +164,10 @@ void RequireAppended(const Recorder& recorder, std::uint32_t offset, std::uint32 Require(recorder.Commands() == commands + 1); Apr::CommandHeader header; std::memcpy(&header, recorder.memory.data() + offset, sizeof(header)); - Require(header.opcode == opcode && header.bytes == measured); + if (opcode == Apr::Opcode::CompactReadFileGather) { + Require((static_cast(header.opcode) & 0xffu) == static_cast(opcode)); + Require(measured == sizeof(Apr::CompactReadFileGatherCommand)); + } else Require(header.opcode == opcode && header.bytes == measured); } void RequireRecorded(const Recorder& recorder, std::uint32_t offset, std::uint32_t commands, Apr::Opcode opcode, std::uint64_t measured, const std::string& text) { @@ -582,6 +585,15 @@ void TestGatherScatter() { std::uint32_t failed = 0; Require(sceKernelAprResolveFilepathsToIds(&path, 1, &fileId, &failed) == 0); + Recorder compact(64); + std::array compactData{}; + Require(sceAmprAprCommandBufferReadFile(&compact.buffer, &compact.gatherState, &compact.scatterState, fileId, compactData.data(), 8, 100) == 0); + Require(sceAmprAprCommandBufferReadFileGather(&compact.buffer, &compact.gatherState, &compact.scatterState, 8, 108) == 0); + Require(sceAmprAprCommandBufferReadFileGather(&compact.buffer, &compact.gatherState, &compact.scatterState, 8, 116) == 0); + Require(compact.Offset() == 64); + Require(sceKernelAprSubmitCommandBuffer(&compact.buffer, 0) == 0); + Require(MatchesFile(compactData.data(), 100, compactData.size())); + Recorder recorder; auto* buffer = &recorder.buffer; auto* map = &recorder.gatherState; @@ -597,7 +609,7 @@ void TestGatherScatter() { auto offset = recorder.Offset(); auto commands = recorder.Commands(); Require(sceAmprAprCommandBufferReadFileGather(buffer, map, state, 8, 500) == 0); - RequireAppended(recorder, offset, commands, Apr::Opcode::ReadFileGather, sceAmprMeasureCommandSizeReadFileGather(8, 500)); + RequireAppended(recorder, offset, commands, Apr::Opcode::CompactReadFileGather, sceAmprMeasureCommandSizeReadFileGather(8, 500)); offset = recorder.Offset(); commands = recorder.Commands(); Require(sceAmprAprCommandBufferReadFileScatter(buffer, map, state, second.data(), 8) == 0); @@ -637,7 +649,7 @@ void TestGatherScatter() { Require(sceAmprAprCommandBufferReadFile(nullptr, map, state, fileId, first.data(), 4, 0) == invalidArgument); Require(empty.Offset() == 0 && empty.Commands() == 0); Require(sceAmprMeasureCommandSizeReadFileGather(0, 0) == rejected); - Require(sceAmprMeasureCommandSizeReadFileGather(0x100000000ull, 0x10000000000ull - 1u) == sizeof(Apr::ReadFileCommand)); + Require(sceAmprMeasureCommandSizeReadFileGather(0x100000000ull, 0x10000000000ull - 1u) == sizeof(Apr::CompactReadFileGatherCommand)); Require(sceAmprMeasureCommandSizeReadFileGather(4, 0x10000000000ull) == rejected); Require(sceAmprMeasureCommandSizeReadFileScatter(first.data(), 0x100000001ull) == rejected); Require(sceAmprMeasureCommandSizeReadFileScatter(high, 4) == rejected); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index bf76b1588..86b1c3956 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -92,6 +92,7 @@ Throughout the project, every function at every stage either **does exactly what - [s_sethalt](../../core/shader/recompiler/Translation/src/ScalarInstructions.cpp) (core/shader/recompiler) - with SIMM16[0] clear it is a no-op: the RDNA 2 ISA guide describes that value as resuming a halted wave, and a wave that executes it is not halted; measured on a gfx1035 (RDNA 2) wave: `s_sethalt 0` followed by a VALU op completes and gives the normal result. Only SIMM16[0] is read; with it set the instruction throws - [AMPR WriteAddressFromCounterPairOnCompletion](../../core/libs/prx/libSceAmpr/Export.cpp) (libSceAmpr) - the pair of an even counter n is assumed to give counter n in the low 32 bits and n + 1 in the high 32 bits. Wait compare values, wait flush and counter index ranges follow the [ampr_emu](https://github.com/drakmor/ampr_emu) reimplementation; the flush is accepted and has no effect, as commands run in order - [AMPR command sizes](../../core/libs/prx/libSceAmpr/Export.cpp) (libSceAmpr) - commands use the emulator's own encoding with an 8-byte header, so `Nop(n)` takes 8 + 4n bytes where the console record is n dwords including its header (`NopWithData(n)`: (n + 1) dwords). The measure functions agree with the appended sizes, but a title that pads to an exact offset or the end of the buffer with `Nop(remaining / 4)` gets `SCE_AMPR_ERROR_BUFFER_FULL` or a shifted offset. The at-start flag of the `_04_00` writes is ignored; commands run in order, each after the previous one completes +- [AMPR object layout and compact gathers](../../core/libs/prx/libkernel/Apr/include/AprCommandBuffer.hpp) - the 24-byte object and offsets (type 0, cursor 4, count 8, capacity 12, pointer 16) follow [ampr_emu assertions](https://github.com/drakmor/ampr_emu/blob/cfa85df379f6eeeb165d7badf9b648e266fe77b7/include/ampr_emu_command_buffer_types.h#L138-L143) and its [C declaration](https://github.com/drakmor/ampr_emu/blob/cfa85df379f6eeeb165d7badf9b648e266fe77b7/include/ampr.h#L41-L47). AnyPS5 represents the 32-bit type word as low 16-bit type and high 16-bit recording flags. Command encoding is AnyPS5 internal: opcode 29 and three 32-bit words encode the high offset byte, length minus one and low offset word. The 12-byte size reduces HFW's 1 MiB buffer pressure; it is not a claim about console opcode encoding. Existing 40-bit offsets and lengths 1 through 2^32 are retained; console behavior outside those limits is unverified. - [AMPR counter access](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) (libSceAmpr) - signatures and checks of `WaitOnCounter_04_00`, `WriteCounter_04_00`, `ConstructNop` and `ConstructMarker` taken from the [ampr_emu](https://github.com/drakmor/ampr_emu) reimplementation, which hands counter commands to the console and does not show their effect. The counters are taken as 32 bits each: the 8-byte access of counter n spans n (low) and n + 1 (high), the 2- and 1-byte accesses select a half or a byte of the counter by its offset, waits compare at the access width (signed and wrapped compares included), the AND mask applies to both sides and the atomic writes wrap within the access. The `ConstructNop` type is not recorded - [APR gather and scatter reads](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) (libSceAmpr) - signatures, argument checks and the read cursor taken from the [ampr_emu](https://github.com/drakmor/ampr_emu) reimplementation, not confirmed on a title: each read leaves the file, the next file offset and the next destination; a gather reads that file at a new offset into the next destination, a scatter continues in the file into a new destination, a gather-scatter takes both. Recording one needs a read file since the last `ResetGatherScatterState`; like ampr_emu, `sceAmprCommandBufferReset` keeps that state, and a gather or scatter with no read before it in the submitted buffer throws instead of returning `SCE_AMPR_ERROR_APR_INVALIDGATHERSCATTERSTATE` - [AMM map, map direct and unmap](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) (libSceAmpr) - signatures and the protection mask (CPU, GPU, AMPR and ACP read and write) taken from the [ampr_emu](https://github.com/drakmor/ampr_emu) reimplementation, which hands the remaining checks and the mapping to the console kernel; the mapper model taken from [PS5PCEM](https://github.com/iStark/PS5PCEM): `sceAmprAmmGiveDirectMemory` usage 1 gives the allocated direct memory to the pool plain maps take pages from, usage 0 only allocates it. Not confirmed on a title. The AMM virtual address ranges are a 32 GiB range and a 32 GiB multimap range reserved in the guest arena on first use, not the console's; maps outside them throw. Addresses, sizes and direct offsets must be 16 KiB aligned. The memory type and the GPU mask are ignored, AMPR and ACP access maps as CPU access, unmapped pages go back to the pool and stay reserved. Submission runs the buffer before returning, the submit result is always 0 and a failing command throws; the AMM measures return errors sign-extended, as ampr_emu does. `sceAmprAmmGetVirtualAddressRanges` throws for a null output From b299350a7a008cc3ce2a39c8c6140390faa1ab3e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 07:27:53 +0200 Subject: [PATCH 365/460] docs(debt): record the unverified ForEach resolve count --- docs/dev/TechnicalDebt.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 86b1c3956..586cab99d 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -93,6 +93,7 @@ Throughout the project, every function at every stage either **does exactly what - [AMPR WriteAddressFromCounterPairOnCompletion](../../core/libs/prx/libSceAmpr/Export.cpp) (libSceAmpr) - the pair of an even counter n is assumed to give counter n in the low 32 bits and n + 1 in the high 32 bits. Wait compare values, wait flush and counter index ranges follow the [ampr_emu](https://github.com/drakmor/ampr_emu) reimplementation; the flush is accepted and has no effect, as commands run in order - [AMPR command sizes](../../core/libs/prx/libSceAmpr/Export.cpp) (libSceAmpr) - commands use the emulator's own encoding with an 8-byte header, so `Nop(n)` takes 8 + 4n bytes where the console record is n dwords including its header (`NopWithData(n)`: (n + 1) dwords). The measure functions agree with the appended sizes, but a title that pads to an exact offset or the end of the buffer with `Nop(remaining / 4)` gets `SCE_AMPR_ERROR_BUFFER_FULL` or a shifted offset. The at-start flag of the `_04_00` writes is ignored; commands run in order, each after the previous one completes - [AMPR object layout and compact gathers](../../core/libs/prx/libkernel/Apr/include/AprCommandBuffer.hpp) - the 24-byte object and offsets (type 0, cursor 4, count 8, capacity 12, pointer 16) follow [ampr_emu assertions](https://github.com/drakmor/ampr_emu/blob/cfa85df379f6eeeb165d7badf9b648e266fe77b7/include/ampr_emu_command_buffer_types.h#L138-L143) and its [C declaration](https://github.com/drakmor/ampr_emu/blob/cfa85df379f6eeeb165d7badf9b648e266fe77b7/include/ampr.h#L41-L47). AnyPS5 represents the 32-bit type word as low 16-bit type and high 16-bit recording flags. Command encoding is AnyPS5 internal: opcode 29 and three 32-bit words encode the high offset byte, length minus one and low offset word. The 12-byte size reduces HFW's 1 MiB buffer pressure; it is not a claim about console opcode encoding. Existing 40-bit offsets and lengths 1 through 2^32 are retained; console behavior outside those limits is unverified. +- [APR resolve ForEach result](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) (libkernel) - `sceKernelAprResolveFilepathsToIdsAndFileSizesForEach` returns the number of paths that resolved, with the per-path status in `results` and the failed ids/sizes set to the invalid id and 0. The [ampr_emu](https://github.com/drakmor/ampr_emu) bridge passes the count of its resolver through and does not show what is counted; counting successes is not confirmed on a title. The guest errno of the failing non-`ForEach` resolves is set through `__error_nid_postfix` - [AMPR counter access](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) (libSceAmpr) - signatures and checks of `WaitOnCounter_04_00`, `WriteCounter_04_00`, `ConstructNop` and `ConstructMarker` taken from the [ampr_emu](https://github.com/drakmor/ampr_emu) reimplementation, which hands counter commands to the console and does not show their effect. The counters are taken as 32 bits each: the 8-byte access of counter n spans n (low) and n + 1 (high), the 2- and 1-byte accesses select a half or a byte of the counter by its offset, waits compare at the access width (signed and wrapped compares included), the AND mask applies to both sides and the atomic writes wrap within the access. The `ConstructNop` type is not recorded - [APR gather and scatter reads](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) (libSceAmpr) - signatures, argument checks and the read cursor taken from the [ampr_emu](https://github.com/drakmor/ampr_emu) reimplementation, not confirmed on a title: each read leaves the file, the next file offset and the next destination; a gather reads that file at a new offset into the next destination, a scatter continues in the file into a new destination, a gather-scatter takes both. Recording one needs a read file since the last `ResetGatherScatterState`; like ampr_emu, `sceAmprCommandBufferReset` keeps that state, and a gather or scatter with no read before it in the submitted buffer throws instead of returning `SCE_AMPR_ERROR_APR_INVALIDGATHERSCATTERSTATE` - [AMM map, map direct and unmap](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) (libSceAmpr) - signatures and the protection mask (CPU, GPU, AMPR and ACP read and write) taken from the [ampr_emu](https://github.com/drakmor/ampr_emu) reimplementation, which hands the remaining checks and the mapping to the console kernel; the mapper model taken from [PS5PCEM](https://github.com/iStark/PS5PCEM): `sceAmprAmmGiveDirectMemory` usage 1 gives the allocated direct memory to the pool plain maps take pages from, usage 0 only allocates it. Not confirmed on a title. The AMM virtual address ranges are a 32 GiB range and a 32 GiB multimap range reserved in the guest arena on first use, not the console's; maps outside them throw. Addresses, sizes and direct offsets must be 16 KiB aligned. The memory type and the GPU mask are ignored, AMPR and ACP access maps as CPU access, unmapped pages go back to the pool and stay reserved. Submission runs the buffer before returning, the submit result is always 0 and a failing command throws; the AMM measures return errors sign-extended, as ampr_emu does. `sceAmprAmmGetVirtualAddressRanges` throws for a null output From 63d29116b478102d78256cfc9d9b0cdfe94ad1df Mon Sep 17 00:00:00 2001 From: tototomate123 Date: Thu, 8 Oct 2026 16:47:53 +0200 Subject: [PATCH 366/460] fix(http): retain request status callbacks with the guest ABI --- core/libs/prx/libSceHttp/Export.cpp | 19 ++++++++++++++++--- core/libs/tests/GuestHttp.cpp | 12 ++++++++++++ docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 29 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceHttp/Export.cpp b/core/libs/prx/libSceHttp/Export.cpp index c1ab8550f..3b9216532 100644 --- a/core/libs/prx/libSceHttp/Export.cpp +++ b/core/libs/prx/libSceHttp/Export.cpp @@ -6,6 +6,8 @@ #include #include #include +#include +#include #include #include #include @@ -13,6 +15,13 @@ // No network is emulated: contexts, templates and requests can be created, but any request // that would touch the network fails with the library's network error. static std::atomic g_nextHandle{1}; +using HttpRequestStatusCallback = void (APS5_VABI *)(int, int, void*); +struct RequestStatusRegistration { + HttpRequestStatusCallback callback; + void* argument; +}; +static std::mutex g_statusMutex; +static std::map g_statusCallbacks; extern "C" { @@ -82,7 +91,8 @@ int APS5_VABI sceHttpDeleteConnection(int conn_id) { } int APS5_VABI sceHttpDeleteRequest(int req_id) { - (void)req_id; + std::lock_guard lock(g_statusMutex); + g_statusCallbacks.erase(req_id); return 0; } @@ -287,8 +297,11 @@ int APS5_VABI sceHttpSetInflateGZIPEnabled(int id, int enable) { return 0; } -int APS5_VABI sceHttpSetRequestStatusCallback(void) { - NotImplemented_nid_no_patch(__func__); +int APS5_VABI sceHttpSetRequestStatusCallback(int id, HttpRequestStatusCallback callback, void* argument) { + if (id <= 0 || id >= g_nextHandle.load(std::memory_order_relaxed)) return static_cast(0x80431100u); + std::lock_guard lock(g_statusMutex); + if (callback) g_statusCallbacks[id] = {callback, argument}; + else g_statusCallbacks.erase(id); return 0; } diff --git a/core/libs/tests/GuestHttp.cpp b/core/libs/tests/GuestHttp.cpp index 1a5dcc2d3..9d18283f0 100644 --- a/core/libs/tests/GuestHttp.cpp +++ b/core/libs/tests/GuestHttp.cpp @@ -9,6 +9,8 @@ extern "C" { int APS5_VABI sceHttpUriParse(SceHttpUriElement*, const char*, void*, std::size_t*, std::size_t); int APS5_VABI sceHttpUriMerge(char*, const char*, const char*, std::size_t*, std::size_t, std::uint32_t); int APS5_VABI sceHttpSetInflateGZIPEnabled(int, int); +int APS5_VABI sceHttpSetRequestStatusCallback(int, void (APS5_VABI *)(int, int, void*), void*); +int APS5_VABI sceHttpSendRequest(int, const void*, std::size_t); int APS5_VABI sceHttpUriBuild(char*, std::size_t*, std::size_t, const SceHttpUriElement*, std::uint32_t); int APS5_VABI sceHttpUriEscape(char*, std::size_t*, std::size_t, const char*); int APS5_VABI sceHttpUriUnescape(char*, std::size_t*, std::size_t, const char*); @@ -38,8 +40,18 @@ static void Require(bool value) { if (!value) std::abort(); } static bool Equal(const char* left, const char* right) { return std::strcmp(left, right) == 0; } static int AuthInfo(int, int, const char*, char*, char*, int, std::uint8_t**, std::uint64_t*, int*, void*) { std::abort(); } +static void APS5_VABI RequestStatusCallback(int, int, void* argument) { + ++*static_cast(argument); +} int main() { + int callbackCount = 0; + const auto request = sceHttpCreateRequest2(1, "GET", "/", 0); + Require(sceHttpSetRequestStatusCallback(request, RequestStatusCallback, &callbackCount) == 0); + Require(sceHttpSendRequest(request, nullptr, 0) == static_cast(0x80431063u)); + Require(callbackCount == 0); + Require(sceHttpSetRequestStatusCallback(request, nullptr, nullptr) == 0); + Require(sceHttpSetRequestStatusCallback(-1, RequestStatusCallback, &callbackCount) == static_cast(0x80431100u)); constexpr int outOfMemory = static_cast(0x80431022); constexpr int invalidValue = static_cast(0x804311FE); constexpr int invalidUrl = static_cast(0x80433060); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index bf76b1588..9636f6eb5 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -243,10 +243,10 @@ Throughout the project, every function at every stage either **does exactly what - [sceHttpUriSweepPath](../../core/libs/prx/libSceHttp/src/Uri.cpp) (libSceHttp) - behaviour follows the [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.cpp) reimplementation of the PS4 library and its tests (including the PS4 SDK example), not confirmed on a PS5 title or hardware. It is not RFC 3986 5.2.4: a path that doesn't start with `/` is copied unchanged and a trailing `.` or `..` without a slash is kept. Unlike shadPS4, nothing at or past `srcSize - 1` is read, and `srcSize == 1` writes only the terminator - [sceHttpSetResponseHeaderMaxSize, sceHttpRedirectCacheFlush, sceHttpsUnloadCert](../../core/libs/prx/libSceHttp/Export.cpp) (libSceHttp) - signatures follow [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.cpp) (PS4); no PS5 reference has them - [sceNpSetTimeout](../../core/libs/prx/libSceNpManager/Export.cpp) (libSceNpManager) - the signature follows [orbital](https://github.com/AlexAltea/orbital/blob/b321fcdbd700cc02da1cb1ad25412781d5ec73c6/tools/ida/db_types.json), [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/0fe263a4760dfbfa973366890061749b4af0de97/src/core/libraries/np/np_manager.cpp) and [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/5dfdb4b0b166e83dc9ae275001afb695f12c7174/src/SharpProspero/Interop/Np/NpAccount.cs); the rejected values (all five 0, negative retries, a resolve timeout under 1 s or another timeout under 10 s, 0 meaning the default) are shadPS4's, not checked on a console. An id that was never created or was deleted returns `REQUEST_NOT_FOUND`, as in shadPS4; there is no network, so the timeouts are not stored +- [sceHttpSetRequestStatusCallback](../../core/libs/prx/libSceHttp/Export.cpp) - the three-argument ABI follows [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.cpp). Registrations and user data are retained and removed on request deletion; the offline backend never dispatches callbacks. Invalid ids return `0x80431100`, an uncited assumption, not a verified console error. Handle hierarchy, inheritance and deleted-handle validation remain incomplete; callback statuses and dispatch timing are unknown. - [sceHttpSetRedirectCallback, sceHttpSetCookieRecvCallback, sceHttpsSetSslVersion, sceHttpsGetSslError](../../core/libs/prx/libSceHttp/Export.cpp) (libSceHttp) - signatures follow [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.cpp) (PS4), which only logs and returns 0; a null `errNum` or `detail` in `sceHttpsGetSslError` is assumed to return `SCE_HTTP_ERROR_INVALID_VALUE` like `sceHttpGetLastErrno` - [sceHttpSetAuthInfoCallback](../../core/libs/prx/libSceHttp/Export.cpp) (libSceHttp) - signature and callback type follow [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.h) (PS4); no PS5 reference has them. The `id` is not checked, as in the other libSceHttp setters - [libSceHttp2 cookie boxes](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) - `sceHttp2CreateCookieBox` is taken to take the library context id first, as in the [prosper](https://github.com/mattias800/prosper) reimplementation; further arguments are unknown and ignored. `sceHttp2SetRequestNoContentLength` is assumed to take only the request id. Cookies are never stored, so `sceHttp2CookieFlush` has nothing to discard -- [sceHttpSetRequestStatusCallback](../../core/libs/prx/libSceHttp/Export.cpp) (libSceHttp) - no reference implements it (shadPS4 only logs and returns 0); when the callback runs and what status it gets is unknown - [sceNpSessionSignalingGetMemoryInfo](../../core/libs/prx/libSceNpSessionSignaling/Export.cpp) (libSceNpSessionSignaling) - unknown signature - [sceNpSessionSignalingGetConnectionStatistics](../../core/libs/prx/libSceNpSessionSignaling/Export.cpp) (libSceNpSessionSignaling) - unknown signature - [sceUserServiceGetAgeLevel](../../core/libs/prx/libSceUserService/Export.cpp) (libSceUserService) - the meaning of the value is unknown; 0 is reported, as for the unset game presets. PPSA12544 (Unity) only stores it for its user profile From e2a7aac32d070c3ba2425fc921c5a4b84ee3661a Mon Sep 17 00:00:00 2001 From: fma-rc <188743521+fma-rc@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:50:03 +0200 Subject: [PATCH 367/460] feat(libSceJobManager): implement CPU job sequences --- core/libs/prx/libSceJobManager/CMakeLists.txt | 18 + core/libs/prx/libSceJobManager/Export.cpp | 578 ++++++++++++++++++ core/libs/tests/GuestJobManager.cpp | 384 ++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 981 insertions(+) create mode 100644 core/libs/prx/libSceJobManager/CMakeLists.txt create mode 100644 core/libs/prx/libSceJobManager/Export.cpp create mode 100644 core/libs/tests/GuestJobManager.cpp diff --git a/core/libs/prx/libSceJobManager/CMakeLists.txt b/core/libs/prx/libSceJobManager/CMakeLists.txt new file mode 100644 index 000000000..74bfa4368 --- /dev/null +++ b/core/libs/prx/libSceJobManager/CMakeLists.txt @@ -0,0 +1,18 @@ +add_library(libSceJobManager SHARED EXCLUDE_FROM_ALL Export.cpp) +target_include_directories(libSceJobManager PRIVATE ${LIBS_INCLUDE_DIR}) +configure_windows_unwind(libSceJobManager) + +if(BUILD_TESTING) + add_test_executable(guest_job_manager_tests ${LIBS_INCLUDE_DIR}/tests/GuestJobManager.cpp) + target_include_directories(guest_job_manager_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_job_manager_tests PRIVATE libSceJobManager libc) + configure_windows_unwind(guest_job_manager_tests) + foreach(testCase IN ITEMS memory validation dispatch ordering limits priority shutdown) + add_test(NAME guest_job_manager_${testCase} COMMAND guest_job_manager_tests ${testCase}) + set_tests_properties(guest_job_manager_${testCase} PROPERTIES TIMEOUT 20) + if(WIN32) + set_tests_properties(guest_job_manager_${testCase} PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() + endforeach() +endif() diff --git a/core/libs/prx/libSceJobManager/Export.cpp b/core/libs/prx/libSceJobManager/Export.cpp new file mode 100644 index 000000000..838c1e6d3 --- /dev/null +++ b/core/libs/prx/libSceJobManager/Export.cpp @@ -0,0 +1,578 @@ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "SceTypes.hpp" +#include "prx/libc/include/General.hpp" + +extern "C" { +int APS5_VABI scePthreadAttrInit(PthreadAttr* attr); +int APS5_VABI scePthreadAttrDestroy(PthreadAttr* attr); +int APS5_VABI scePthreadAttrSetinheritsched(PthreadAttr* attr, int inheritSched); +int APS5_VABI scePthreadAttrSetschedparam(PthreadAttr* attr, const KernelSchedParam* param); +int APS5_VABI scePthreadAttrSetstacksize(PthreadAttr* attr, std::size_t stackSize); +int APS5_VABI scePthreadAttrSetaffinity(PthreadAttr* attr, KernelCpumask mask); +int APS5_VABI scePthreadCreate(Pthread* thread, const PthreadAttr* attr, PthreadEntry entry, void* arg, const char* name); +int APS5_VABI scePthreadJoin(Pthread thread, void** result); +} + +namespace { + +constexpr int ManagerInvalidArgument = static_cast(0x81581000u); +constexpr int ManagerNullMemory = static_cast(0x81581001u); +constexpr int ManagerMemoryTooSmall = static_cast(0x81581002u); +constexpr int ManagerNotInitialized = static_cast(0x81581010u); +constexpr int ManagerAlreadyInitialized = static_cast(0x81581020u); +constexpr int ManagerMisalignedMemory = static_cast(0x81581030u); +constexpr int SequenceOutOfRange = static_cast(0x81580000u); +constexpr int SequenceNullArgument = static_cast(0x81580001u); +constexpr int SequenceNoNodes = static_cast(0x81580002u); +constexpr int SequenceLimitExceeded = static_cast(0x81580003u); +constexpr int SequenceNoneFree = static_cast(0x81580006u); +constexpr int SequenceInvalid = static_cast(0x81580007u); + +constexpr std::uint32_t MaxWorkers = 13; +constexpr std::uint32_t MaxSequences = 32; +constexpr std::uint32_t MaxInvocations = 0x7fff; +constexpr std::uint32_t MaxDispatches = 0x3ff; +constexpr std::uint32_t MaxSyncPoints = 0xff; +constexpr std::uint32_t MaxWaits = 0x3f; +constexpr std::uint64_t StateOffset = 0x10; +constexpr std::uint64_t StateEnd = 0x2d8; +constexpr std::uint64_t ThreadRecordBytes = 0x48; +constexpr std::uint64_t SequenceBytes = 0x240; +constexpr std::uint64_t NodeBytes = 72; + +struct MemorySizeQueryParams { + std::uint32_t version; + std::uint32_t workers; + std::uint32_t sequences; + std::uint32_t invocations; + std::uint32_t dispatches; + std::uint32_t syncPoints; + std::uint32_t waits; + std::uint32_t computeSequences; + std::uint64_t computeQueueMask; +}; + +struct ConfigParams { + std::uint32_t version; + std::uint32_t threadFlags; + std::uint32_t stackSize; + std::uint32_t priority; + const char* threadName; + std::uint32_t spinMicroseconds; + std::uint32_t computeAffinity; + std::uint32_t computePriority; + std::uint32_t reserved; +}; + +constexpr ConfigParams DefaultConfig{0, 0x00021fff, 0x00010000, 0x2bc, nullptr, 0, 0x1fff, 0x100, 0}; + +struct BlockHeader { + std::uint16_t magic; + std::uint16_t stateOffset; + std::uint32_t version; + std::uint32_t size; + std::uint32_t reserved; +}; + +using JobEntry = void (APS5_VABI*)(void* argument, std::uint32_t index); + +std::uint64_t AlignUp(std::uint64_t value, std::uint64_t alignment) { + return (value + alignment - 1) & ~(alignment - 1); +} + +struct Manager; +class Sequence; + +enum class ItemType { Dispatch, SyncPoint, Wait }; + +struct Item { + ItemType type; + Sequence* sequence; + JobEntry entry; + void* argument; + std::uint32_t count; + std::uint32_t handedOut; + std::uint32_t completed; + bool* passed; +}; + +struct Counters { + std::uint32_t invocations = 0; + std::uint32_t dispatches = 0; + std::uint32_t syncPoints = 0; + std::uint32_t waits = 0; +}; + +class SequenceInterface { +public: + APS5_VABI virtual int Dispatch(JobEntry entry, void* argument, std::uint32_t count, const char* name) = 0; + APS5_VABI virtual int SyncPoint() = 0; + APS5_VABI virtual int Wait() = 0; + APS5_VABI virtual ~SequenceInterface() = default; + APS5_VABI virtual int DispatchCompute(const void* shader, void* userData, std::int32_t x, std::int32_t y, std::int32_t z, const char* name) = 0; + APS5_VABI virtual int DispatchComputeIndirect(const void* shader, void* userData, const void* indirectArguments, const char* name) = 0; + static void operator delete(void*) { throw std::runtime_error("libSceJobManager: a sequence was deleted by its caller"); } +}; + +class SequenceFactoryInterface { +public: + APS5_VABI virtual int Create(SequenceInterface** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char* name) = 0; + APS5_VABI virtual int Destroy(SequenceInterface* sequence) = 0; + APS5_VABI virtual ~SequenceFactoryInterface() = default; + APS5_VABI virtual int CreateCompute(SequenceInterface** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char* name) = 0; + APS5_VABI virtual int DestroyCompute(SequenceInterface* sequence) = 0; + static void operator delete(void*) { throw std::runtime_error("libSceJobManager: a sequence factory was deleted by its caller"); } +}; + +class Sequence final : public SequenceInterface { +public: + APS5_VABI int Dispatch(JobEntry entry, void* argument, std::uint32_t count, const char* name) override; + APS5_VABI int SyncPoint() override; + APS5_VABI int Wait() override; + APS5_VABI int DispatchCompute(const void*, void*, std::int32_t, std::int32_t, std::int32_t, const char*) override { + NotImplemented_nid_no_patch("sce::Job compute dispatch"); + return 0; + } + APS5_VABI int DispatchComputeIndirect(const void*, void*, const void*, const char*) override { + NotImplemented_nid_no_patch("sce::Job indirect compute dispatch"); + return 0; + } + + bool Open(Manager* owner, const Counters& limit, std::uint32_t runQueue); + void Close(); + void Advance(); + void DispatchCompleted(std::uint32_t invocations); + + Manager* manager = nullptr; + bool open = false; + +private: + int Latch(int code); + + Counters limits; + Counters submitted; + Counters completed; + std::uint32_t priority = 0; + int error = 0; + std::uint32_t outstanding = 0; + std::size_t cursor = 0; + std::deque items; + std::mutex waiters; + std::condition_variable passed; +}; + +class Factory final : public SequenceFactoryInterface { +public: + APS5_VABI int Create(SequenceInterface** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char* name) override; + APS5_VABI int Destroy(SequenceInterface* sequence) override; + APS5_VABI int CreateCompute(SequenceInterface** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char* name) override; + APS5_VABI int DestroyCompute(SequenceInterface* sequence) override; + + Manager* manager = nullptr; + std::uint32_t priority = 0; +}; + +struct Manager { + std::mutex lock; + std::condition_variable work; + std::condition_variable nodeFreed; + bool running = false; + std::uint32_t blockedWaiters = 0; + std::uint32_t freeNodes = 0; + std::uint32_t freeSequences = 0; + std::uint32_t persistentThreads = 0; + std::array, 2> runQueues; + std::vector sequences; + std::array factories; + std::array unifiedFactories; + std::vector workers; +}; + +Manager*& Implementation(void* self) { + return *static_cast(self); +} + +bool TakeNode(Manager& manager) { + if (manager.freeNodes == 0) return false; + --manager.freeNodes; + return true; +} + +void ReturnNodes(Manager& manager, std::uint32_t count) { + manager.freeNodes += count; + manager.nodeFreed.notify_all(); +} + +int Sequence::Latch(int code) { + if (error == 0) error = code; + return code; +} + +bool Sequence::Open(Manager* owner, const Counters& limit, std::uint32_t runQueue) { + manager = owner; + limits = limit; + submitted = {}; + completed = {}; + priority = runQueue; + error = 0; + outstanding = 0; + cursor = 0; + items.clear(); + if (!TakeNode(*owner)) return false; + open = true; + return true; +} + +void Sequence::Close() { + if (!open) return; + ReturnNodes(*manager, static_cast(items.size()) + 1); + items.clear(); + cursor = 0; + outstanding = 0; + open = false; +} + +void Sequence::Advance() { + while (cursor < items.size()) { + Item& item = items[cursor]; + if (item.type == ItemType::Dispatch) { + ++outstanding; + ++cursor; + manager->runQueues[priority].push_back(&item); + manager->work.notify_all(); + continue; + } + if (outstanding != 0) return; + if (item.type == ItemType::SyncPoint) { + ++completed.syncPoints; + ++cursor; + continue; + } + bool* const waiter = item.passed; + const auto retired = static_cast(cursor) + 1; + items.erase(items.begin(), items.begin() + static_cast(retired)); + cursor = 0; + if (error == 0) { + submitted.invocations -= completed.invocations; + submitted.dispatches -= completed.dispatches; + submitted.syncPoints -= completed.syncPoints; + --submitted.waits; + completed = {}; + } + ReturnNodes(*manager, retired); + *waiter = true; + passed.notify_all(); + } +} + +void Sequence::DispatchCompleted(std::uint32_t invocations) { + completed.invocations += invocations; + ++completed.dispatches; + --outstanding; + if (outstanding == 0) Advance(); +} + +int Sequence::Dispatch(JobEntry entry, void* argument, std::uint32_t count, const char*) { + if (entry == nullptr) return SequenceNullArgument; + if (count == 0 || count > MaxInvocations) return SequenceOutOfRange; + const std::lock_guard guard(manager->lock); + if (error != 0) return SequenceInvalid; + const Counters before = submitted; + submitted.invocations += count; + ++submitted.dispatches; + if (before.invocations + count > limits.invocations || before.dispatches >= limits.dispatches) return Latch(SequenceLimitExceeded); + if (!TakeNode(*manager)) return Latch(SequenceNoNodes); + items.push_back({ItemType::Dispatch, this, entry, argument, count, 0, 0, nullptr}); + Advance(); + return 0; +} + +int Sequence::SyncPoint() { + const std::lock_guard guard(manager->lock); + if (error != 0) return SequenceInvalid; + const std::uint32_t before = submitted.syncPoints++; + if (before >= limits.syncPoints) return Latch(SequenceLimitExceeded); + if (!TakeNode(*manager)) return Latch(SequenceNoNodes); + items.push_back({ItemType::SyncPoint, this, nullptr, nullptr, 0, 0, 0, nullptr}); + Advance(); + return 0; +} + +int Sequence::Wait() { + const std::lock_guard serial(waiters); + std::unique_lock guard(manager->lock); + int result = 0; + if (error != 0) { + result = SequenceInvalid; + } else if (submitted.waits++ >= limits.waits) { + result = Latch(SequenceLimitExceeded); + } + if (manager->freeNodes == 0 && result == 0) result = Latch(SequenceNoNodes); + ++manager->blockedWaiters; + manager->nodeFreed.wait(guard, [this] { return manager->freeNodes != 0; }); + --manager->freeNodes; + bool done = false; + items.push_back({ItemType::Wait, this, nullptr, nullptr, 0, 0, 0, &done}); + Advance(); + passed.wait(guard, [&done] { return done; }); + --manager->blockedWaiters; + return result; +} + +bool ValidLimits(std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits) { + if (invocations > MaxInvocations || dispatches > MaxDispatches || syncPoints > MaxSyncPoints || waits > MaxWaits) return false; + return invocations != 0 || dispatches != 0 || syncPoints != 0 || waits != 0; +} + +int Factory::Create(SequenceInterface** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char*) { + if (out == nullptr) return SequenceNullArgument; + if (!ValidLimits(invocations, dispatches, syncPoints, waits)) return SequenceOutOfRange; + const std::lock_guard guard(manager->lock); + if (manager->freeSequences == 0) return SequenceNoneFree; + const int index = std::countr_zero(manager->freeSequences); + manager->freeSequences &= ~(std::uint32_t{1} << index); + Sequence& sequence = manager->sequences[static_cast(index)]; + if (!sequence.Open(manager, {invocations, dispatches, syncPoints, waits}, priority)) return SequenceNoNodes; + *out = &sequence; + return 0; +} + +int Factory::Destroy(SequenceInterface* sequence) { + Sequence* found = nullptr; + std::uint32_t index = 0; + { + const std::lock_guard guard(manager->lock); + for (; index < manager->sequences.size(); ++index) { + if (&manager->sequences[index] == sequence) break; + } + if (index == manager->sequences.size() || (manager->freeSequences & (std::uint32_t{1} << index)) != 0) return SequenceInvalid; + found = &manager->sequences[index]; + } + static_cast(found->Wait()); + const std::lock_guard guard(manager->lock); + found->Close(); + manager->freeSequences |= std::uint32_t{1} << index; + return 0; +} + +int Factory::CreateCompute(SequenceInterface** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char*) { + if (out == nullptr) return SequenceNullArgument; + if (!ValidLimits(invocations, dispatches, syncPoints, waits)) return SequenceOutOfRange; + return SequenceNoneFree; +} + +int Factory::DestroyCompute(SequenceInterface*) { + return SequenceInvalid; +} + +bool RunOne(Manager& manager, std::unique_lock& guard) { + Item* item = nullptr; + for (const std::size_t queue : {std::size_t{1}, std::size_t{0}}) { + auto& pending = manager.runQueues[queue]; + if (pending.empty()) continue; + item = pending.front(); + if (item->handedOut + 1 == item->count) pending.pop_front(); + break; + } + if (item == nullptr) return false; + const std::uint32_t index = item->handedOut++; + const JobEntry entry = item->entry; + void* const argument = item->argument; + guard.unlock(); + entry(argument, index); + guard.lock(); + if (++item->completed == item->count) item->sequence->DispatchCompleted(item->count); + return true; +} + +void* APS5_VABI WorkerEntry(void* argument) { + Manager& manager = *static_cast(argument); + std::unique_lock guard(manager.lock); + while (manager.running) { + if (RunOne(manager, guard)) continue; + manager.work.wait(guard, [&manager] { return !manager.running || !manager.runQueues[0].empty() || !manager.runQueues[1].empty(); }); + } + return nullptr; +} + +std::uint32_t NodeCount(std::uint64_t memory, std::uint64_t size, std::uint32_t workers, std::uint32_t sequences, bool* fits) { + const std::uint64_t end = memory + size; + std::uint64_t address = AlignUp(memory + StateEnd, 8) + workers * ThreadRecordBytes; + for (const std::uint64_t bytes : {std::uint64_t{0x100}, std::uint64_t{0x80}, std::uint64_t{0x80}, std::uint64_t{0x80}}) address = AlignUp(address, 64) + bytes; + address = AlignUp(AlignUp(address, 64) + sequences * SequenceBytes, 64); + *fits = address <= end; + return *fits ? static_cast(((end - address) / NodeBytes) & 0xffff) : 0; +} + +int ValidateWorkers(const ConfigParams& config, std::uint32_t workers) { + const std::uint32_t mode = config.threadFlags & 0x30000; + if (config.reserved != 0 || config.priority < 0x100 || config.priority > 0x2ff || config.stackSize < 0x4000) return ManagerInvalidArgument; + if ((config.threadFlags & 0xfffce000) != 0 || static_cast(std::popcount(config.threadFlags & 0x1fff)) < workers) return ManagerInvalidArgument; + if ((mode != 0x10000 && mode != 0x20000) || config.spinMicroseconds > 0x32) return ManagerInvalidArgument; + return 0; +} + +void StartWorkers(Manager& manager, const ConfigParams& config, std::uint32_t workers) { + std::uint32_t remaining = config.threadFlags & 0x1fff; + for (std::uint32_t index = 0; index < workers; ++index) { + KernelCpumask affinity = config.threadFlags & 0x1fff; + if ((config.threadFlags & 0x30000) == 0x10000) { + affinity = remaining & (~remaining + 1); + remaining &= remaining - 1; + } + const std::string name = config.threadName != nullptr ? std::string(config.threadName).substr(0, 31) : "SceJobManagerWorker" + std::to_string(index); + const KernelSchedParam schedule{static_cast(config.priority)}; + PthreadAttr attributes = nullptr; + if (scePthreadAttrInit(&attributes) != 0) throw std::runtime_error("libSceJobManager: scePthreadAttrInit failed"); + scePthreadAttrSetinheritsched(&attributes, 0); + scePthreadAttrSetschedparam(&attributes, &schedule); + scePthreadAttrSetstacksize(&attributes, config.stackSize); + scePthreadAttrSetaffinity(&attributes, affinity); + Pthread thread = nullptr; + const int created = scePthreadCreate(&thread, &attributes, WorkerEntry, &manager, name.c_str()); + scePthreadAttrDestroy(&attributes); + if (created != 0) throw std::runtime_error("libSceJobManager: scePthreadCreate failed for a worker"); + manager.workers.push_back(thread); + } +} + +int Initialize(void* self, void* memory, std::uint64_t size, std::uint32_t workers, std::uint32_t sequences, std::uint32_t computeSequences, std::uint64_t computeQueueMask, const ConfigParams* config, bool unified) { + if (Implementation(self) != nullptr) return ManagerAlreadyInitialized; + if (memory == nullptr) return ManagerNullMemory; + const auto base = reinterpret_cast(memory); + if ((base & 0xff) != 0) return ManagerMisalignedMemory; + if (size == 0 || workers == 0 || workers > MaxWorkers) return ManagerInvalidArgument; + if (unified) { + if ((sequences == 0 && computeSequences == 0) || sequences + computeSequences > MaxSequences || (computeQueueMask >> 56) != 0 || static_cast(std::popcount(computeQueueMask)) < computeSequences) return ManagerInvalidArgument; + } else if (sequences == 0 || sequences > MaxSequences) { + return ManagerInvalidArgument; + } + if (size < StateOffset) return ManagerMemoryTooSmall; + if (size < StateEnd) throw std::runtime_error("sce::Job::JobManager::initialize: a memory block smaller than 0x2d8 bytes is not supported"); + std::memset(static_cast(memory) + StateOffset, 0, StateEnd - StateOffset); + const BlockHeader header{0x4d4a, static_cast(StateOffset), 0x05000001, static_cast(size), 0}; + std::memcpy(memory, &header, sizeof(header)); + if (config != nullptr && config->version > 1) return ManagerInvalidArgument; + if (computeSequences != 0) NotImplemented_nid_no_patch("sce::Job::JobManager::initialize with compute sequences"); + const ConfigParams& settings = config != nullptr ? *config : DefaultConfig; + bool fits = false; + const std::uint32_t nodes = NodeCount(base, size, workers, sequences, &fits); + if (!fits || nodes == 0) return ManagerMemoryTooSmall; + if (const int invalid = ValidateWorkers(settings, workers); invalid != 0) return invalid; + auto* manager = new Manager; + manager->running = true; + manager->freeNodes = nodes - 1; + manager->freeSequences = sequences == MaxSequences ? ~std::uint32_t{0} : (std::uint32_t{1} << sequences) - 1; + manager->sequences = std::vector(sequences); + for (std::uint32_t priority = 0; priority < 2; ++priority) { + manager->factories[priority].manager = manager; + manager->factories[priority].priority = priority; + manager->unifiedFactories[priority].manager = manager; + manager->unifiedFactories[priority].priority = priority; + } + Implementation(self) = manager; + StartWorkers(*manager, settings, workers); + return 0; +} + +SequenceFactoryInterface* FactoryFor(void* self, std::uint32_t priority, bool unified) { + Manager* manager = Implementation(self); + if (manager == nullptr) return nullptr; + if (priority > 1) throw std::runtime_error("sce::Job::JobManager: sequence factory priority " + std::to_string(priority) + " does not exist"); + return unified ? &manager->unifiedFactories[priority] : &manager->factories[priority]; +} + +} + +extern "C" { + +void APS5_VABI _ZN3sce3Job10JobManagerC1Ev(void* self) { + Implementation(self) = nullptr; +} + +void APS5_VABI _ZN3sce3Job10JobManagerC2Ev(void* self) { + Implementation(self) = nullptr; +} + +void APS5_VABI _ZN3sce3Job10JobManagerD1Ev(void*) { +} + +void APS5_VABI _ZN3sce3Job10JobManagerD2Ev(void*) { +} + +std::uint64_t APS5_VABI _ZN3sce3Job10JobManager27calculateRequiredMemorySizeEPKNS1_21MemorySizeQueryParamsE(const MemorySizeQueryParams* params) { + if (params == nullptr || params->version > 1 || params->workers == 0 || params->workers > MaxWorkers || params->sequences > MaxSequences) return 0; + const bool unified = params->version == 1; + const std::uint32_t computeSequences = unified ? params->computeSequences : 0; + if (unified) { + if (computeSequences > MaxSequences || params->sequences + computeSequences == 0 || params->sequences + computeSequences > MaxSequences) return 0; + if ((params->computeQueueMask >> 56) != 0 || static_cast(std::popcount(params->computeQueueMask)) < computeSequences) return 0; + } else if (params->sequences == 0) { + return 0; + } + const std::uint64_t total = params->sequences + computeSequences; + if (params->invocations > total * MaxInvocations || params->dispatches > total * MaxDispatches || params->syncPoints > total * MaxSyncPoints || params->waits > total * MaxWaits) return 0; + if (computeSequences != 0) NotImplemented_nid_no_patch("sce::Job::JobManager::calculateRequiredMemorySize with compute sequences"); + std::uint64_t bytes = (NodeBytes * params->workers + 0x317) & 0xfc0; + bytes = (bytes + 0x13f) & ~std::uint64_t{0x3f}; + bytes = (bytes + 0x0bf) & ~std::uint64_t{0x3f}; + bytes = (bytes + 0x13f) & ~std::uint64_t{0x3f}; + bytes += params->sequences * SequenceBytes; + const std::uint32_t nodes = params->dispatches + params->syncPoints + params->waits + static_cast(total) + 1; + return AlignUp(bytes, 64) + std::uint64_t{nodes} * NodeBytes + 0x800; +} + +int APS5_VABI _ZN3sce3Job10JobManager10initializeEPvmjjjmPKNS1_12ConfigParamsE(void* self, void* memory, std::uint64_t memorySize, std::uint32_t workers, std::uint32_t sequences, std::uint32_t computeSequences, std::uint64_t computeQueueMask, const ConfigParams* config) { + return Initialize(self, memory, memorySize, workers, sequences, computeSequences, computeQueueMask, config, true); +} + +int APS5_VABI _ZN3sce3Job10JobManager10initializeEPvmjjPKNS1_12ConfigParamsE(void* self, void* memory, std::uint64_t memorySize, std::uint32_t workers, std::uint32_t sequences, const ConfigParams* config) { + return Initialize(self, memory, memorySize, workers, sequences, 0, 0, config, false); +} + +int APS5_VABI _ZN3sce3Job10JobManager8shutdownEv(void* self) { + Manager* manager = Implementation(self); + if (manager == nullptr) return ManagerNotInitialized; + { + const std::lock_guard guard(manager->lock); + if (manager->blockedWaiters != 0) throw std::runtime_error("sce::Job::JobManager::shutdown: a thread is still blocked in a sequence wait"); + manager->running = false; + manager->work.notify_all(); + } + for (const Pthread worker : manager->workers) { + if (scePthreadJoin(worker, nullptr) != 0) throw std::runtime_error("sce::Job::JobManager::shutdown: scePthreadJoin failed"); + } + Implementation(self) = nullptr; + delete manager; + return 0; +} + +SequenceFactoryInterface* APS5_VABI _ZN3sce3Job10JobManager27getSequenceFactoryInterfaceENS1_8PriorityE(void* self, std::uint32_t priority) { + return FactoryFor(self, priority, false); +} + +SequenceFactoryInterface* APS5_VABI _ZN3sce3Job10JobManager34getUnifiedSequenceFactoryInterfaceENS1_8PriorityE(void* self, std::uint32_t priority) { + return FactoryFor(self, priority, true); +} + +int APS5_VABI _ZN3sce3Job10JobManager24setPersistentThreadCountEj(void* self, std::uint32_t count) { + Manager* manager = Implementation(self); + if (manager == nullptr) return ManagerNotInitialized; + if (count > manager->workers.size()) return ManagerInvalidArgument; + const std::lock_guard guard(manager->lock); + manager->persistentThreads = count; + return 0; +} + +} diff --git a/core/libs/tests/GuestJobManager.cpp b/core/libs/tests/GuestJobManager.cpp new file mode 100644 index 000000000..c95519c39 --- /dev/null +++ b/core/libs/tests/GuestJobManager.cpp @@ -0,0 +1,384 @@ +#include "SceTypes.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct MemorySizeQueryParams { + std::uint32_t version; + std::uint32_t workers; + std::uint32_t sequences; + std::uint32_t invocations; + std::uint32_t dispatches; + std::uint32_t syncPoints; + std::uint32_t waits; + std::uint32_t computeSequences; + std::uint64_t computeQueueMask; +}; + +struct ConfigParams { + std::uint32_t version; + std::uint32_t threadFlags; + std::uint32_t stackSize; + std::uint32_t priority; + const char* threadName; + std::uint32_t spinMicroseconds; + std::uint32_t computeAffinity; + std::uint32_t computePriority; + std::uint32_t reserved; +}; + +using JobEntry = void (APS5_VABI*)(void*, std::uint32_t); + +struct Sequence { + APS5_VABI virtual int Dispatch(JobEntry, void*, std::uint32_t, const char*) = 0; + APS5_VABI virtual int SyncPoint() = 0; + APS5_VABI virtual int Wait() = 0; + APS5_VABI virtual ~Sequence() = default; +}; + +struct SequenceFactory { + APS5_VABI virtual int Create(Sequence**, std::uint32_t, std::uint32_t, std::uint32_t, std::uint32_t, const char*) = 0; + APS5_VABI virtual int Destroy(Sequence*) = 0; + APS5_VABI virtual ~SequenceFactory() = default; + APS5_VABI virtual int CreateCompute(Sequence**, std::uint32_t, std::uint32_t, std::uint32_t, std::uint32_t, const char*) = 0; + APS5_VABI virtual int DestroyCompute(Sequence*) = 0; +}; + +extern "C" { +void APS5_VABI _ZN3sce3Job10JobManagerC1Ev(void*); +void APS5_VABI _ZN3sce3Job10JobManagerD1Ev(void*); +std::uint64_t APS5_VABI _ZN3sce3Job10JobManager27calculateRequiredMemorySizeEPKNS1_21MemorySizeQueryParamsE(const MemorySizeQueryParams*); +int APS5_VABI _ZN3sce3Job10JobManager10initializeEPvmjjjmPKNS1_12ConfigParamsE(void*, void*, std::uint64_t, std::uint32_t, std::uint32_t, std::uint32_t, std::uint64_t, const ConfigParams*); +int APS5_VABI _ZN3sce3Job10JobManager10initializeEPvmjjPKNS1_12ConfigParamsE(void*, void*, std::uint64_t, std::uint32_t, std::uint32_t, const ConfigParams*); +int APS5_VABI _ZN3sce3Job10JobManager8shutdownEv(void*); +SequenceFactory* APS5_VABI _ZN3sce3Job10JobManager27getSequenceFactoryInterfaceENS1_8PriorityE(void*, std::uint32_t); +SequenceFactory* APS5_VABI _ZN3sce3Job10JobManager34getUnifiedSequenceFactoryInterfaceENS1_8PriorityE(void*, std::uint32_t); +int APS5_VABI _ZN3sce3Job10JobManager24setPersistentThreadCountEj(void*, std::uint32_t); +} + +#define Construct _ZN3sce3Job10JobManagerC1Ev +#define Destruct _ZN3sce3Job10JobManagerD1Ev +#define RequiredSize _ZN3sce3Job10JobManager27calculateRequiredMemorySizeEPKNS1_21MemorySizeQueryParamsE +#define InitializeUnified _ZN3sce3Job10JobManager10initializeEPvmjjjmPKNS1_12ConfigParamsE +#define Initialize _ZN3sce3Job10JobManager10initializeEPvmjjPKNS1_12ConfigParamsE +#define Shutdown _ZN3sce3Job10JobManager8shutdownEv +#define Factories _ZN3sce3Job10JobManager27getSequenceFactoryInterfaceENS1_8PriorityE +#define UnifiedFactories _ZN3sce3Job10JobManager34getUnifiedSequenceFactoryInterfaceENS1_8PriorityE +#define SetPersistentThreadCount _ZN3sce3Job10JobManager24setPersistentThreadCountEj + +static constexpr int InvalidArgument = static_cast(0x81581000u); +static constexpr int NullMemory = static_cast(0x81581001u); +static constexpr int MemoryTooSmall = static_cast(0x81581002u); +static constexpr int NotInitialized = static_cast(0x81581010u); +static constexpr int AlreadyInitialized = static_cast(0x81581020u); +static constexpr int MisalignedMemory = static_cast(0x81581030u); +static constexpr int OutOfRange = static_cast(0x81580000u); +static constexpr int NullArgument = static_cast(0x81580001u); +static constexpr int NoNodes = static_cast(0x81580002u); +static constexpr int LimitExceeded = static_cast(0x81580003u); +static constexpr int NoneFree = static_cast(0x81580006u); +static constexpr int Invalid = static_cast(0x81580007u); + +static void Check(bool value, int line) { + if (!value) { + std::fprintf(stderr, "job manager check failed at line %d\n", line); + std::abort(); + } +} + +#define Require(value) Check((value), __LINE__) + +struct Block { + explicit Block(std::uint64_t size) : bytes(size), memory(std::aligned_alloc(256, (size + 255) & ~std::uint64_t{255})) {} + ~Block() { std::free(memory); } + std::uint64_t bytes; + void* memory; +}; + +struct ManagerObject { + ManagerObject() { Construct(storage); } + ~ManagerObject() { Destruct(storage); } + alignas(64) unsigned char storage[0x100]{}; +}; + +static std::uint64_t SizeFor(std::uint32_t workers, std::uint32_t sequences, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits) { + const MemorySizeQueryParams query{0, workers, sequences, 0, dispatches, syncPoints, waits, 0, 0}; + return RequiredSize(&query); +} + +static void MemoryCase() { + const MemorySizeQueryParams query{0, 4, 2, 100, 8, 4, 2, 0, 0}; + Require(RequiredSize(&query) == 0x17c8); + const MemorySizeQueryParams unified{1, 4, 2, 100, 8, 4, 2, 0, 0}; + Require(RequiredSize(&unified) == 0x17c8); + const MemorySizeQueryParams single{0, 1, 1, 0, 0, 0, 0, 0, 0}; + Require(RequiredSize(&single) == 0x1090); + Require(RequiredSize(nullptr) == 0); + for (const MemorySizeQueryParams& bad : { + MemorySizeQueryParams{2, 4, 2, 0, 0, 0, 0, 0, 0}, MemorySizeQueryParams{0, 0, 2, 0, 0, 0, 0, 0, 0}, + MemorySizeQueryParams{0, 14, 2, 0, 0, 0, 0, 0, 0}, MemorySizeQueryParams{0, 4, 0, 0, 0, 0, 0, 0, 0}, + MemorySizeQueryParams{0, 4, 33, 0, 0, 0, 0, 0, 0}, MemorySizeQueryParams{0, 4, 2, 2 * 0x7fff + 1, 0, 0, 0, 0, 0}, + MemorySizeQueryParams{0, 4, 2, 0, 2 * 0x3ff + 1, 0, 0, 0, 0}, MemorySizeQueryParams{0, 4, 2, 0, 0, 2 * 0xff + 1, 0, 0, 0}, + MemorySizeQueryParams{0, 4, 2, 0, 0, 0, 2 * 0x3f + 1, 0, 0}, MemorySizeQueryParams{1, 4, 0, 0, 0, 0, 0, 0, 0}, + MemorySizeQueryParams{1, 4, 2, 0, 0, 0, 0, 0, std::uint64_t{1} << 56}}) { + Require(RequiredSize(&bad) == 0); + } +} + +static void APS5_VABI Nothing(void*, std::uint32_t) { +} + +static void ValidationCase() { + Block block(SizeFor(2, 2, 16, 4, 4)); + ManagerObject manager; + Require(Shutdown(manager.storage) == NotInitialized); + Require(SetPersistentThreadCount(manager.storage, 0) == NotInitialized); + Require(Factories(manager.storage, 0) == nullptr && UnifiedFactories(manager.storage, 1) == nullptr); + Require(Initialize(manager.storage, nullptr, block.bytes, 2, 2, nullptr) == NullMemory); + Require(Initialize(manager.storage, static_cast(block.memory) + 16, block.bytes - 16, 2, 2, nullptr) == MisalignedMemory); + Require(Initialize(manager.storage, block.memory, 0, 2, 2, nullptr) == InvalidArgument); + Require(Initialize(manager.storage, block.memory, block.bytes, 0, 2, nullptr) == InvalidArgument); + Require(Initialize(manager.storage, block.memory, block.bytes, 14, 2, nullptr) == InvalidArgument); + Require(Initialize(manager.storage, block.memory, block.bytes, 2, 0, nullptr) == InvalidArgument); + Require(Initialize(manager.storage, block.memory, block.bytes, 2, 33, nullptr) == InvalidArgument); + Require(InitializeUnified(manager.storage, block.memory, block.bytes, 2, 0, 0, 0, nullptr) == InvalidArgument); + Require(InitializeUnified(manager.storage, block.memory, block.bytes, 2, 2, 0, std::uint64_t{1} << 56, nullptr) == InvalidArgument); + Require(Initialize(manager.storage, block.memory, 8, 2, 2, nullptr) == MemoryTooSmall); + Require(Initialize(manager.storage, block.memory, 0x400, 2, 2, nullptr) == MemoryTooSmall); + ConfigParams config{0, 0x00021fff, 0x10000, 0x2bc, nullptr, 0, 0x1fff, 0x100, 0}; + for (int field = 0; field < 7; ++field) { + ConfigParams bad = config; + if (field == 0) bad.version = 2; + if (field == 1) bad.threadFlags = 0x00001fff; + if (field == 2) bad.threadFlags = 0x00020001; + if (field == 3) bad.stackSize = 0x3fff; + if (field == 4) bad.priority = 0x300; + if (field == 5) bad.spinMicroseconds = 0x33; + if (field == 6) bad.reserved = 1; + Require(Initialize(manager.storage, block.memory, block.bytes, 2, 2, &bad) == InvalidArgument); + Require(Factories(manager.storage, 0) == nullptr); + } + config.threadFlags = 0x00010003; + config.threadName = "JobManagerTestWorker"; + Require(Initialize(manager.storage, block.memory, block.bytes, 2, 2, &config) == 0); + std::uint16_t magic = 0; + std::memcpy(&magic, block.memory, sizeof(magic)); + Require(magic == 0x4d4a); + Require(Initialize(manager.storage, block.memory, block.bytes, 2, 2, nullptr) == AlreadyInitialized); + Require(SetPersistentThreadCount(manager.storage, 3) == InvalidArgument); + Require(SetPersistentThreadCount(manager.storage, 2) == 0); + SequenceFactory* factory = Factories(manager.storage, 0); + Require(factory != nullptr && Factories(manager.storage, 1) != nullptr && factory != Factories(manager.storage, 1)); + Require(UnifiedFactories(manager.storage, 0) != nullptr); + Sequence* sequence = nullptr; + Require(factory->Create(nullptr, 1, 1, 1, 1, nullptr) == NullArgument); + Require(factory->Create(&sequence, 0, 0, 0, 0, nullptr) == OutOfRange); + Require(factory->Create(&sequence, 0x8000, 1, 1, 1, nullptr) == OutOfRange); + Require(factory->Create(&sequence, 1, 0x400, 1, 1, nullptr) == OutOfRange); + Require(factory->Create(&sequence, 1, 1, 0x100, 1, nullptr) == OutOfRange); + Require(factory->Create(&sequence, 1, 1, 1, 0x40, nullptr) == OutOfRange); + Require(factory->CreateCompute(&sequence, 1, 1, 1, 1, nullptr) == NoneFree && sequence == nullptr); + Sequence* first = nullptr; + Sequence* second = nullptr; + Require(factory->Create(&first, 8, 8, 2, 2, "first") == 0 && first != nullptr); + Require(factory->Create(&second, 8, 8, 2, 2, "second") == 0 && second != nullptr && second != first); + Require(factory->Create(&sequence, 8, 8, 2, 2, "third") == NoneFree); + Require(factory->DestroyCompute(first) == Invalid); + Require(factory->Destroy(reinterpret_cast(block.memory)) == Invalid); + Require(factory->Destroy(first) == 0 && factory->Destroy(first) == Invalid); + Require(factory->Create(&sequence, 8, 8, 2, 2, "again") == 0 && sequence == first); + Require(sequence->Dispatch(nullptr, nullptr, 1, nullptr) == NullArgument); + Require(sequence->Dispatch(Nothing, nullptr, 0, nullptr) == OutOfRange); + Require(sequence->Dispatch(Nothing, nullptr, 0x8000, nullptr) == OutOfRange); + Require(factory->Destroy(sequence) == 0 && factory->Destroy(second) == 0); + Require(Shutdown(manager.storage) == 0); +} + +struct Tally { + std::atomic calls{0}; + std::vector> seen; + explicit Tally(std::size_t count) : seen(count) {} +}; + +static void APS5_VABI CountJob(void* argument, std::uint32_t index) { + auto& tally = *static_cast(argument); + ++tally.seen[index]; + ++tally.calls; +} + +static void DispatchCase() { + Block block(SizeFor(4, 1, 8, 2, 2)); + ManagerObject manager; + Require(Initialize(manager.storage, block.memory, block.bytes, 4, 1, nullptr) == 0); + SequenceFactory* factory = Factories(manager.storage, 0); + Sequence* sequence = nullptr; + Require(factory->Create(&sequence, 0x7fff, 8, 2, 2, "dispatch") == 0); + Tally tally(1000); + Tally single(1); + Require(sequence->Dispatch(CountJob, &tally, 1000, "many") == 0); + Require(sequence->Dispatch(CountJob, &single, 1, "one") == 0); + Require(sequence->Wait() == 0); + Require(tally.calls == 1000 && single.calls == 1); + for (const auto& count : tally.seen) Require(count == 1); + Require(sequence->Wait() == 0); + Require(factory->Destroy(sequence) == 0); + Require(Shutdown(manager.storage) == 0); +} + +struct Stage { + std::atomic* finished; + std::uint32_t expected; + std::atomic* early; +}; + +static void APS5_VABI StageJob(void* argument, std::uint32_t) { + auto& stage = *static_cast(argument); + if (stage.finished->load() < stage.expected) ++*stage.early; + std::this_thread::sleep_for(std::chrono::milliseconds(2)); + ++*stage.finished; +} + +static void OrderingCase() { + Block block(SizeFor(4, 1, 8, 4, 2)); + ManagerObject manager; + Require(Initialize(manager.storage, block.memory, block.bytes, 4, 1, nullptr) == 0); + SequenceFactory* factory = Factories(manager.storage, 1); + Sequence* sequence = nullptr; + Require(factory->Create(&sequence, 256, 8, 4, 2, "ordering") == 0); + std::atomic finished{0}; + std::atomic early{0}; + Stage first{&finished, 0, &early}; + Stage second{&finished, 16, &early}; + Stage third{&finished, 32, &early}; + Require(sequence->Dispatch(StageJob, &first, 8, "a") == 0); + Require(sequence->Dispatch(StageJob, &first, 8, "b") == 0); + Require(sequence->SyncPoint() == 0); + Require(sequence->Dispatch(StageJob, &second, 16, "c") == 0); + Require(sequence->SyncPoint() == 0); + Require(sequence->SyncPoint() == 0); + Require(sequence->Dispatch(StageJob, &third, 4, "d") == 0); + Require(sequence->Wait() == 0); + Require(finished == 36 && early == 0); + Require(factory->Destroy(sequence) == 0); + Require(Shutdown(manager.storage) == 0); +} + +static void LimitsCase() { + Block block(SizeFor(2, 2, 2, 1, 1)); + ManagerObject manager; + Require(Initialize(manager.storage, block.memory, block.bytes, 2, 2, nullptr) == 0); + SequenceFactory* factory = Factories(manager.storage, 0); + Sequence* sequence = nullptr; + Require(factory->Create(&sequence, 4, 2, 1, 1, "limits") == 0); + for (int round = 0; round < 3; ++round) { + Require(sequence->Dispatch(Nothing, nullptr, 2, nullptr) == 0); + Require(sequence->Dispatch(Nothing, nullptr, 2, nullptr) == 0); + Require(sequence->SyncPoint() == 0); + Require(sequence->Wait() == 0); + } + Require(sequence->Dispatch(Nothing, nullptr, 2, nullptr) == 0); + Require(sequence->Dispatch(Nothing, nullptr, 3, nullptr) == LimitExceeded); + Require(sequence->Dispatch(Nothing, nullptr, 1, nullptr) == Invalid); + Require(sequence->SyncPoint() == Invalid); + Require(sequence->Wait() == Invalid); + Require(factory->Destroy(sequence) == 0); + Require(factory->Create(&sequence, 4, 2, 1, 1, "dispatches") == 0); + Require(sequence->Dispatch(Nothing, nullptr, 1, nullptr) == 0 && sequence->Dispatch(Nothing, nullptr, 1, nullptr) == 0); + Require(sequence->Dispatch(Nothing, nullptr, 1, nullptr) == LimitExceeded); + Require(factory->Destroy(sequence) == 0); + Require(factory->Create(&sequence, 4, 2, 1, 1, "sync points") == 0); + Require(sequence->SyncPoint() == 0 && sequence->SyncPoint() == LimitExceeded); + Require(factory->Destroy(sequence) == 0); + Require(factory->Create(&sequence, 4, 2, 1, 1, "waits") == 0); + Require(sequence->Wait() == 0 && sequence->Wait() == 0); + Require(factory->Destroy(sequence) == 0); + Require(factory->Create(&sequence, 0x7fff, 0x3ff, 0xff, 1, "nodes") == 0); + int result = 0; + int accepted = 0; + while (accepted < 0x3ff && (result = sequence->Dispatch(Nothing, nullptr, 1, nullptr)) == 0) ++accepted; + Require(result == NoNodes && accepted > 0); + Require(sequence->Dispatch(Nothing, nullptr, 1, nullptr) == Invalid); + Require(Shutdown(manager.storage) == 0); +} + +struct Order { + std::atomic release{false}; + std::atomic started{false}; + std::atomic next{0}; + std::uint32_t low = 0; + std::uint32_t high = 0; +}; + +static void APS5_VABI GateJob(void* argument, std::uint32_t) { + auto& order = *static_cast(argument); + order.started = true; + while (!order.release) std::this_thread::yield(); +} + +static void APS5_VABI LowJob(void* argument, std::uint32_t) { + auto& order = *static_cast(argument); + order.low = ++order.next; +} + +static void APS5_VABI HighJob(void* argument, std::uint32_t) { + auto& order = *static_cast(argument); + order.high = ++order.next; +} + +static void PriorityCase() { + Block block(SizeFor(1, 3, 8, 2, 4)); + ManagerObject manager; + Require(Initialize(manager.storage, block.memory, block.bytes, 1, 3, nullptr) == 0); + Sequence* gate = nullptr; + Sequence* low = nullptr; + Sequence* high = nullptr; + Require(Factories(manager.storage, 0)->Create(&gate, 4, 4, 1, 1, "gate") == 0); + Require(Factories(manager.storage, 0)->Create(&low, 4, 4, 1, 1, "low") == 0); + Require(Factories(manager.storage, 1)->Create(&high, 4, 4, 1, 1, "high") == 0); + Order order; + Require(gate->Dispatch(GateJob, &order, 1, nullptr) == 0); + while (!order.started) std::this_thread::yield(); + Require(low->Dispatch(LowJob, &order, 1, nullptr) == 0); + Require(high->Dispatch(HighJob, &order, 1, nullptr) == 0); + order.release = true; + Require(gate->Wait() == 0 && low->Wait() == 0 && high->Wait() == 0); + Require(order.high == 1 && order.low == 2); + Require(Factories(manager.storage, 0)->Destroy(gate) == 0 && Factories(manager.storage, 0)->Destroy(low) == 0); + Require(Factories(manager.storage, 1)->Destroy(high) == 0); + Require(Shutdown(manager.storage) == 0); +} + +static void ShutdownCase() { + Block block(SizeFor(3, 2, 8, 2, 2)); + ManagerObject manager; + for (int round = 0; round < 3; ++round) { + Require(Initialize(manager.storage, block.memory, block.bytes, 3, 2, nullptr) == 0); + SequenceFactory* factory = UnifiedFactories(manager.storage, 0); + Sequence* sequence = nullptr; + Tally tally(64); + Require(factory->Create(&sequence, 64, 8, 2, 2, "shutdown") == 0); + Require(sequence->Dispatch(CountJob, &tally, 64, nullptr) == 0 && sequence->Wait() == 0 && tally.calls == 64); + Require(Shutdown(manager.storage) == 0); + Require(Shutdown(manager.storage) == NotInitialized); + Require(Factories(manager.storage, 0) == nullptr); + } +} + +int main(int argc, char** argv) { + const std::string name = argc > 1 ? argv[1] : ""; + if (name == "memory") MemoryCase(); + else if (name == "validation") ValidationCase(); + else if (name == "dispatch") DispatchCase(); + else if (name == "ordering") OrderingCase(); + else if (name == "limits") LimitsCase(); + else if (name == "priority") PriorityCase(); + else if (name == "shutdown") ShutdownCase(); + else return 2; + return 0; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index ba7e7fd0c..73a84aba9 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -521,3 +521,4 @@ Throughout the project, every function at every stage either **does exactly what - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert +- [sce::Job::JobManager](../../core/libs/prx/libSceJobManager/Export.cpp) (libSceJobManager) is written from the disassembly of the copies PPSA01474 and PPSA36082 ship in `sce_module`, and `guest_job_manager_tests` passes against both of those modules as well: the memory size formula, the error codes, sync point ordering, the per-sequence limits and the priority order are the original's. Only CPU sequences exist: `initialize` and `calculateRequiredMemorySize` with compute sequences, and both compute dispatches, are not implemented, and `createCompute` and `destroyCompute` return the errors the original returns when no compute sequence was configured. The manager's state lives on the host heap, with only the block header written to the title's memory; the original keeps everything in that block, and a block smaller than 0x2d8 bytes throws here. The worker sleep policy, the spin count and the persistent thread count are validated and stored but change nothing, a factory priority above 1 throws, and `shutdown` throws while a thread is blocked in a sequence wait. The order of the sequence and factory virtual functions is read from the modules' vtables only, since no title that calls them has been run From 8897c5ceacdb40fe016e97af90bf27a2fd10b0d8 Mon Sep 17 00:00:00 2001 From: fma-rc <188743521+fma-rc@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:56:27 +0200 Subject: [PATCH 368/460] fix(libSceJobManager): count a wait before serialising it and close under the lock --- core/libs/prx/libSceJobManager/CMakeLists.txt | 2 +- core/libs/prx/libSceJobManager/Export.cpp | 96 +++++++++++-------- core/libs/tests/GuestJobManager.cpp | 56 ++++++++++- docs/dev/TechnicalDebt.md | 2 +- 4 files changed, 111 insertions(+), 45 deletions(-) diff --git a/core/libs/prx/libSceJobManager/CMakeLists.txt b/core/libs/prx/libSceJobManager/CMakeLists.txt index 74bfa4368..c029f72ce 100644 --- a/core/libs/prx/libSceJobManager/CMakeLists.txt +++ b/core/libs/prx/libSceJobManager/CMakeLists.txt @@ -7,7 +7,7 @@ if(BUILD_TESTING) target_include_directories(guest_job_manager_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_job_manager_tests PRIVATE libSceJobManager libc) configure_windows_unwind(guest_job_manager_tests) - foreach(testCase IN ITEMS memory validation dispatch ordering limits priority shutdown) + foreach(testCase IN ITEMS memory validation dispatch ordering nested limits priority shutdown) add_test(NAME guest_job_manager_${testCase} COMMAND guest_job_manager_tests ${testCase}) set_tests_properties(guest_job_manager_${testCase} PROPERTIES TIMEOUT 20) if(WIN32) diff --git a/core/libs/prx/libSceJobManager/Export.cpp b/core/libs/prx/libSceJobManager/Export.cpp index 838c1e6d3..94d53929a 100644 --- a/core/libs/prx/libSceJobManager/Export.cpp +++ b/core/libs/prx/libSceJobManager/Export.cpp @@ -114,28 +114,28 @@ struct Counters { std::uint32_t waits = 0; }; -class SequenceInterface { +class ISequence { public: APS5_VABI virtual int Dispatch(JobEntry entry, void* argument, std::uint32_t count, const char* name) = 0; APS5_VABI virtual int SyncPoint() = 0; APS5_VABI virtual int Wait() = 0; - APS5_VABI virtual ~SequenceInterface() = default; + APS5_VABI virtual void CompleteDestructor() {} + APS5_VABI virtual void DeletingDestructor() { throw std::runtime_error("libSceJobManager: a sequence was deleted by its caller"); } APS5_VABI virtual int DispatchCompute(const void* shader, void* userData, std::int32_t x, std::int32_t y, std::int32_t z, const char* name) = 0; APS5_VABI virtual int DispatchComputeIndirect(const void* shader, void* userData, const void* indirectArguments, const char* name) = 0; - static void operator delete(void*) { throw std::runtime_error("libSceJobManager: a sequence was deleted by its caller"); } }; -class SequenceFactoryInterface { +class ISequenceFactory { public: - APS5_VABI virtual int Create(SequenceInterface** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char* name) = 0; - APS5_VABI virtual int Destroy(SequenceInterface* sequence) = 0; - APS5_VABI virtual ~SequenceFactoryInterface() = default; - APS5_VABI virtual int CreateCompute(SequenceInterface** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char* name) = 0; - APS5_VABI virtual int DestroyCompute(SequenceInterface* sequence) = 0; - static void operator delete(void*) { throw std::runtime_error("libSceJobManager: a sequence factory was deleted by its caller"); } + APS5_VABI virtual int Create(ISequence** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char* name) = 0; + APS5_VABI virtual int Destroy(ISequence* sequence) = 0; + APS5_VABI virtual void CompleteDestructor() {} + APS5_VABI virtual void DeletingDestructor() { throw std::runtime_error("libSceJobManager: a sequence factory was deleted by its caller"); } + APS5_VABI virtual int CreateCompute(ISequence** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char* name) = 0; + APS5_VABI virtual int DestroyCompute(ISequence* sequence) = 0; }; -class Sequence final : public SequenceInterface { +class Sequence final : public ISequence { public: APS5_VABI int Dispatch(JobEntry entry, void* argument, std::uint32_t count, const char* name) override; APS5_VABI int SyncPoint() override; @@ -150,7 +150,7 @@ public: } bool Open(Manager* owner, const Counters& limit, std::uint32_t runQueue); - void Close(); + int WaitAndClose(std::uint32_t index); void Advance(); void DispatchCompleted(std::uint32_t invocations); @@ -159,6 +159,8 @@ public: private: int Latch(int code); + void RequireOpen() const; + int WaitLocked(std::unique_lock& guard); Counters limits; Counters submitted; @@ -172,12 +174,12 @@ private: std::condition_variable passed; }; -class Factory final : public SequenceFactoryInterface { +class Factory final : public ISequenceFactory { public: - APS5_VABI int Create(SequenceInterface** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char* name) override; - APS5_VABI int Destroy(SequenceInterface* sequence) override; - APS5_VABI int CreateCompute(SequenceInterface** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char* name) override; - APS5_VABI int DestroyCompute(SequenceInterface* sequence) override; + APS5_VABI int Create(ISequence** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char* name) override; + APS5_VABI int Destroy(ISequence* sequence) override; + APS5_VABI int CreateCompute(ISequence** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char* name) override; + APS5_VABI int DestroyCompute(ISequence* sequence) override; Manager* manager = nullptr; std::uint32_t priority = 0; @@ -234,13 +236,8 @@ bool Sequence::Open(Manager* owner, const Counters& limit, std::uint32_t runQueu return true; } -void Sequence::Close() { - if (!open) return; - ReturnNodes(*manager, static_cast(items.size()) + 1); - items.clear(); - cursor = 0; - outstanding = 0; - open = false; +void Sequence::RequireOpen() const { + if (!open) throw std::runtime_error("libSceJobManager: a destroyed sequence was used"); } void Sequence::Advance() { @@ -287,6 +284,7 @@ int Sequence::Dispatch(JobEntry entry, void* argument, std::uint32_t count, cons if (entry == nullptr) return SequenceNullArgument; if (count == 0 || count > MaxInvocations) return SequenceOutOfRange; const std::lock_guard guard(manager->lock); + RequireOpen(); if (error != 0) return SequenceInvalid; const Counters before = submitted; submitted.invocations += count; @@ -300,6 +298,7 @@ int Sequence::Dispatch(JobEntry entry, void* argument, std::uint32_t count, cons int Sequence::SyncPoint() { const std::lock_guard guard(manager->lock); + RequireOpen(); if (error != 0) return SequenceInvalid; const std::uint32_t before = submitted.syncPoints++; if (before >= limits.syncPoints) return Latch(SequenceLimitExceeded); @@ -310,8 +309,24 @@ int Sequence::SyncPoint() { } int Sequence::Wait() { - const std::lock_guard serial(waiters); std::unique_lock guard(manager->lock); + return WaitLocked(guard); +} + +int Sequence::WaitAndClose(std::uint32_t index) { + std::unique_lock guard(manager->lock); + const int result = WaitLocked(guard); + ReturnNodes(*manager, static_cast(items.size()) + 1); + items.clear(); + cursor = 0; + outstanding = 0; + open = false; + manager->freeSequences |= std::uint32_t{1} << index; + return result; +} + +int Sequence::WaitLocked(std::unique_lock& guard) { + RequireOpen(); int result = 0; if (error != 0) { result = SequenceInvalid; @@ -322,6 +337,9 @@ int Sequence::Wait() { ++manager->blockedWaiters; manager->nodeFreed.wait(guard, [this] { return manager->freeNodes != 0; }); --manager->freeNodes; + guard.unlock(); + const std::lock_guard serial(waiters); + guard.lock(); bool done = false; items.push_back({ItemType::Wait, this, nullptr, nullptr, 0, 0, 0, &done}); Advance(); @@ -335,7 +353,7 @@ bool ValidLimits(std::uint32_t invocations, std::uint32_t dispatches, std::uint3 return invocations != 0 || dispatches != 0 || syncPoints != 0 || waits != 0; } -int Factory::Create(SequenceInterface** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char*) { +int Factory::Create(ISequence** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char*) { if (out == nullptr) return SequenceNullArgument; if (!ValidLimits(invocations, dispatches, syncPoints, waits)) return SequenceOutOfRange; const std::lock_guard guard(manager->lock); @@ -348,7 +366,7 @@ int Factory::Create(SequenceInterface** out, std::uint32_t invocations, std::uin return 0; } -int Factory::Destroy(SequenceInterface* sequence) { +int Factory::Destroy(ISequence* sequence) { Sequence* found = nullptr; std::uint32_t index = 0; { @@ -359,20 +377,17 @@ int Factory::Destroy(SequenceInterface* sequence) { if (index == manager->sequences.size() || (manager->freeSequences & (std::uint32_t{1} << index)) != 0) return SequenceInvalid; found = &manager->sequences[index]; } - static_cast(found->Wait()); - const std::lock_guard guard(manager->lock); - found->Close(); - manager->freeSequences |= std::uint32_t{1} << index; + static_cast(found->WaitAndClose(index)); return 0; } -int Factory::CreateCompute(SequenceInterface** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char*) { +int Factory::CreateCompute(ISequence** out, std::uint32_t invocations, std::uint32_t dispatches, std::uint32_t syncPoints, std::uint32_t waits, const char*) { if (out == nullptr) return SequenceNullArgument; if (!ValidLimits(invocations, dispatches, syncPoints, waits)) return SequenceOutOfRange; return SequenceNoneFree; } -int Factory::DestroyCompute(SequenceInterface*) { +int Factory::DestroyCompute(ISequence*) { return SequenceInvalid; } @@ -435,10 +450,10 @@ void StartWorkers(Manager& manager, const ConfigParams& config, std::uint32_t wo const KernelSchedParam schedule{static_cast(config.priority)}; PthreadAttr attributes = nullptr; if (scePthreadAttrInit(&attributes) != 0) throw std::runtime_error("libSceJobManager: scePthreadAttrInit failed"); - scePthreadAttrSetinheritsched(&attributes, 0); - scePthreadAttrSetschedparam(&attributes, &schedule); - scePthreadAttrSetstacksize(&attributes, config.stackSize); - scePthreadAttrSetaffinity(&attributes, affinity); + if (scePthreadAttrSetinheritsched(&attributes, 0) != 0 || scePthreadAttrSetschedparam(&attributes, &schedule) != 0 || + scePthreadAttrSetstacksize(&attributes, config.stackSize) != 0 || scePthreadAttrSetaffinity(&attributes, affinity) != 0) { + throw std::runtime_error("libSceJobManager: a worker thread attribute was rejected"); + } Pthread thread = nullptr; const int created = scePthreadCreate(&thread, &attributes, WorkerEntry, &manager, name.c_str()); scePthreadAttrDestroy(&attributes); @@ -486,9 +501,10 @@ int Initialize(void* self, void* memory, std::uint64_t size, std::uint32_t worke return 0; } -SequenceFactoryInterface* FactoryFor(void* self, std::uint32_t priority, bool unified) { +ISequenceFactory* FactoryFor(void* self, std::uint32_t priority, bool unified) { Manager* manager = Implementation(self); if (manager == nullptr) return nullptr; + priority &= 0xffff; if (priority > 1) throw std::runtime_error("sce::Job::JobManager: sequence factory priority " + std::to_string(priority) + " does not exist"); return unified ? &manager->unifiedFactories[priority] : &manager->factories[priority]; } @@ -558,11 +574,11 @@ int APS5_VABI _ZN3sce3Job10JobManager8shutdownEv(void* self) { return 0; } -SequenceFactoryInterface* APS5_VABI _ZN3sce3Job10JobManager27getSequenceFactoryInterfaceENS1_8PriorityE(void* self, std::uint32_t priority) { +ISequenceFactory* APS5_VABI _ZN3sce3Job10JobManager27getSequenceFactoryInterfaceENS1_8PriorityE(void* self, std::uint32_t priority) { return FactoryFor(self, priority, false); } -SequenceFactoryInterface* APS5_VABI _ZN3sce3Job10JobManager34getUnifiedSequenceFactoryInterfaceENS1_8PriorityE(void* self, std::uint32_t priority) { +ISequenceFactory* APS5_VABI _ZN3sce3Job10JobManager34getUnifiedSequenceFactoryInterfaceENS1_8PriorityE(void* self, std::uint32_t priority) { return FactoryFor(self, priority, true); } diff --git a/core/libs/tests/GuestJobManager.cpp b/core/libs/tests/GuestJobManager.cpp index c95519c39..0dca62187 100644 --- a/core/libs/tests/GuestJobManager.cpp +++ b/core/libs/tests/GuestJobManager.cpp @@ -39,13 +39,15 @@ struct Sequence { APS5_VABI virtual int Dispatch(JobEntry, void*, std::uint32_t, const char*) = 0; APS5_VABI virtual int SyncPoint() = 0; APS5_VABI virtual int Wait() = 0; - APS5_VABI virtual ~Sequence() = default; + APS5_VABI virtual void CompleteDestructor() = 0; + APS5_VABI virtual void DeletingDestructor() = 0; }; struct SequenceFactory { APS5_VABI virtual int Create(Sequence**, std::uint32_t, std::uint32_t, std::uint32_t, std::uint32_t, const char*) = 0; APS5_VABI virtual int Destroy(Sequence*) = 0; - APS5_VABI virtual ~SequenceFactory() = default; + APS5_VABI virtual void CompleteDestructor() = 0; + APS5_VABI virtual void DeletingDestructor() = 0; APS5_VABI virtual int CreateCompute(Sequence**, std::uint32_t, std::uint32_t, std::uint32_t, std::uint32_t, const char*) = 0; APS5_VABI virtual int DestroyCompute(Sequence*) = 0; }; @@ -269,6 +271,53 @@ static void OrderingCase() { Require(Shutdown(manager.storage) == 0); } +struct Nested { + Sequence* sequence; + std::atomic barrierAppended{false}; + std::atomic followerAppended{false}; + std::atomic leaderFinished{false}; + std::atomic followerResult{-1}; + std::atomic followerSawLeaderFinished{-1}; +}; + +static void APS5_VABI FollowerJob(void* argument, std::uint32_t) { + auto& nested = *static_cast(argument); + nested.followerSawLeaderFinished = nested.leaderFinished ? 1 : 0; +} + +static void APS5_VABI LeaderJob(void* argument, std::uint32_t) { + auto& nested = *static_cast(argument); + while (!nested.barrierAppended) std::this_thread::yield(); + nested.followerResult = nested.sequence->Dispatch(FollowerJob, &nested, 1, "follower"); + nested.followerAppended = true; + std::this_thread::sleep_for(std::chrono::milliseconds(20)); + nested.leaderFinished = true; +} + +static void NestedCase() { + Block block(SizeFor(2, 1, 4, 2, 2)); + ManagerObject manager; + Require(Initialize(manager.storage, block.memory, block.bytes, 2, 1, nullptr) == 0); + SequenceFactory* factory = Factories(manager.storage, 0); + Nested nested{}; + Require(factory->Create(&nested.sequence, 16, 4, 2, 2, "nested") == 0); + Require(nested.sequence->Dispatch(LeaderJob, &nested, 1, "leader") == 0); + Require(nested.sequence->SyncPoint() == 0); + nested.barrierAppended = true; + while (!nested.followerAppended) std::this_thread::yield(); + Require(nested.sequence->Wait() == 0); + Require(nested.followerResult == 0 && nested.followerSawLeaderFinished == 1); + nested.sequence->CompleteDestructor(); + Require(nested.sequence->Wait() == 0); + Require(factory->Destroy(nested.sequence) == 0); + Require(factory->Create(&nested.sequence, 1, 1, 0, 0, "no waits") == 0); + Require(nested.sequence->Wait() == LimitExceeded); + Require(nested.sequence->Dispatch(Nothing, nullptr, 1, nullptr) == Invalid); + Require(factory->Destroy(nested.sequence) == 0); + Require(Factories(manager.storage, 0x10000) == factory); + Require(Shutdown(manager.storage) == 0); +} + static void LimitsCase() { Block block(SizeFor(2, 2, 2, 1, 1)); ManagerObject manager; @@ -302,7 +351,7 @@ static void LimitsCase() { int result = 0; int accepted = 0; while (accepted < 0x3ff && (result = sequence->Dispatch(Nothing, nullptr, 1, nullptr)) == 0) ++accepted; - Require(result == NoNodes && accepted > 0); + Require(result == NoNodes && accepted == 33); Require(sequence->Dispatch(Nothing, nullptr, 1, nullptr) == Invalid); Require(Shutdown(manager.storage) == 0); } @@ -377,6 +426,7 @@ int main(int argc, char** argv) { else if (name == "dispatch") DispatchCase(); else if (name == "ordering") OrderingCase(); else if (name == "limits") LimitsCase(); + else if (name == "nested") NestedCase(); else if (name == "priority") PriorityCase(); else if (name == "shutdown") ShutdownCase(); else return 2; diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 73a84aba9..a24521fb7 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -521,4 +521,4 @@ Throughout the project, every function at every stage either **does exactly what - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert -- [sce::Job::JobManager](../../core/libs/prx/libSceJobManager/Export.cpp) (libSceJobManager) is written from the disassembly of the copies PPSA01474 and PPSA36082 ship in `sce_module`, and `guest_job_manager_tests` passes against both of those modules as well: the memory size formula, the error codes, sync point ordering, the per-sequence limits and the priority order are the original's. Only CPU sequences exist: `initialize` and `calculateRequiredMemorySize` with compute sequences, and both compute dispatches, are not implemented, and `createCompute` and `destroyCompute` return the errors the original returns when no compute sequence was configured. The manager's state lives on the host heap, with only the block header written to the title's memory; the original keeps everything in that block, and a block smaller than 0x2d8 bytes throws here. The worker sleep policy, the spin count and the persistent thread count are validated and stored but change nothing, a factory priority above 1 throws, and `shutdown` throws while a thread is blocked in a sequence wait. The order of the sequence and factory virtual functions is read from the modules' vtables only, since no title that calls them has been run +- [sce::Job::JobManager](../../core/libs/prx/libSceJobManager/Export.cpp) (libSceJobManager) is written from the disassembly of the copies PPSA01474 and PPSA36082 ship in `sce_module`, and `guest_job_manager_tests` passes against both of those modules as well: the memory size formula, the error codes, sync point ordering, the per-sequence limits and the priority order are the original's. Only CPU sequences exist: `initialize` and `calculateRequiredMemorySize` with compute sequences, and both compute dispatches, are not implemented, and `createCompute` and `destroyCompute` return the errors the original returns when no compute sequence was configured. The manager's state lives on the host heap, with only the block header written to the title's memory; the original keeps everything in that block, and a block smaller than 0x2d8 bytes throws here. The worker sleep policy, the spin count and the persistent thread count are validated and stored but change nothing, a factory priority above 1 throws, `shutdown` throws while a thread is blocked in a sequence wait, and using a sequence after its `destroy` throws. `initialize` returns 0 where the original returns the result of its last import (`BG26hBGiNlw`, not resolved to a name), the `sceRazorCpuJobManager*` profiler calls are not made, and a worker thread that cannot be created or configured throws where the original does not check. The order of the sequence and factory virtual functions is read from the modules' vtables only, since no title that calls them has been run From 44745482237b844c78e3cc87353dce4669920f29 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 07:40:46 +0200 Subject: [PATCH 369/460] docs(libSceJobManager): move the TechnicalDebt line off the end of the section --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index a24521fb7..b8dfb8b38 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -428,6 +428,7 @@ Throughout the project, every function at every stage either **does exactly what - [Dual-source blending](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) takes the MRT1 export as the second source of the MRT0 blend, as RADV and radeonsi program it; what the CB does with other export or target layouts is unknown, so they throw. - `--to-intel` does not lower RDPRU/MCOMMIT; the [matcher](../../core/relinker/codegen/src/x86/Amd64OnlyInstructionMatcher.cpp) fails the relink instead. - [Ordered not-equal compares](../../core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp) (shader recompiler) - `FPOrdNotEqual32` is emitted as `OpFUnordNotEqual` with both operands checked for NaN on their bits instead of `OpFOrdNotEqual`; the two agree for every input. The AMD Windows driver 32.0.13058.8001 (Radeon 860M, RDNA 3.5) returns 1 for a NaN operand when any float form of the compare feeds a ballot on a 32-wide subgroup (#825) +- [sce::Job::JobManager](../../core/libs/prx/libSceJobManager/Export.cpp) (libSceJobManager) is written from the disassembly of the copies PPSA01474 and PPSA36082 ship in `sce_module`, and `guest_job_manager_tests` passes against both of those modules as well: the memory size formula, the error codes, sync point ordering, the per-sequence limits and the priority order are the original's. Only CPU sequences exist: `initialize` and `calculateRequiredMemorySize` with compute sequences, and both compute dispatches, are not implemented, and `createCompute` and `destroyCompute` return the errors the original returns when no compute sequence was configured. The manager's state lives on the host heap, with only the block header written to the title's memory; the original keeps everything in that block, and a block smaller than 0x2d8 bytes throws here. The worker sleep policy, the spin count and the persistent thread count are validated and stored but change nothing, a factory priority above 1 throws, `shutdown` throws while a thread is blocked in a sequence wait, and using a sequence after its `destroy` throws. `initialize` returns 0 where the original returns the result of its last import (`BG26hBGiNlw`, not resolved to a name), the `sceRazorCpuJobManager*` profiler calls are not made, and a worker thread that cannot be created or configured throws where the original does not check. The order of the sequence and factory virtual functions is read from the modules' vtables only, since no title that calls them has been run - The length-changing path of the [instruction rewriter](../../core/relinker/codegen/src/x86/X64InstructionRewriter.cpp) is not used by the [converter](../../core/relinker/codegen/src/Amd64OnlyConverter.cpp): it does not adjust data-to-code references (relocations, FDEs, jump tables) or segment sizes, so every substitution keeps the instruction length. - [DB_SHADER_CONTROL.CONSERVATIVE_Z_EXPORT](../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderInputState.cpp) (libSceAgcDriver) becomes `DepthLess`/`DepthGreater`. For a pixel shader that breaks the promise, the host driver decides whether its interpolated Z is depth-tested before shading; what the console does then was not measured. - [DB_SHADER_CONTROL.PRIMITIVE_ORDERED_PIXEL_SHADER](../../core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp) (shader recompiler) - the host pixel interlock spans the whole shader, not only up to `s_sendmsg` ORDERED_PS_DONE. @@ -521,4 +522,3 @@ Throughout the project, every function at every stage either **does exactly what - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert -- [sce::Job::JobManager](../../core/libs/prx/libSceJobManager/Export.cpp) (libSceJobManager) is written from the disassembly of the copies PPSA01474 and PPSA36082 ship in `sce_module`, and `guest_job_manager_tests` passes against both of those modules as well: the memory size formula, the error codes, sync point ordering, the per-sequence limits and the priority order are the original's. Only CPU sequences exist: `initialize` and `calculateRequiredMemorySize` with compute sequences, and both compute dispatches, are not implemented, and `createCompute` and `destroyCompute` return the errors the original returns when no compute sequence was configured. The manager's state lives on the host heap, with only the block header written to the title's memory; the original keeps everything in that block, and a block smaller than 0x2d8 bytes throws here. The worker sleep policy, the spin count and the persistent thread count are validated and stored but change nothing, a factory priority above 1 throws, `shutdown` throws while a thread is blocked in a sequence wait, and using a sequence after its `destroy` throws. `initialize` returns 0 where the original returns the result of its last import (`BG26hBGiNlw`, not resolved to a name), the `sceRazorCpuJobManager*` profiler calls are not made, and a worker thread that cannot be created or configured throws where the original does not check. The order of the sequence and factory virtual functions is read from the modules' vtables only, since no title that calls them has been run From 6aa9128eb39de344c4d95dc825af0630f2375f38 Mon Sep 17 00:00:00 2001 From: Leonardo Date: Sun, 11 Oct 2026 04:59:47 +0200 Subject: [PATCH 370/460] feat(libSceAppContent): implement sceAppContentGetAddcontDownloadProgress --- core/libs/prx/libSceAppContent/Export.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/core/libs/prx/libSceAppContent/Export.cpp b/core/libs/prx/libSceAppContent/Export.cpp index 695437ff9..ed74e8f72 100644 --- a/core/libs/prx/libSceAppContent/Export.cpp +++ b/core/libs/prx/libSceAppContent/Export.cpp @@ -167,4 +167,9 @@ int APS5_VABI sceAppContentAddcontEnqueueDownload(void) { NotImplemented_nid_no_patch(__func__); return 0; } + +int APS5_VABI sceAppContentGetAddcontDownloadProgress(void) { + NotImplemented_nid_no_patch(__func__); + return 0; +} } From 228c8a0d6e083253b6a8d4d4d082b0d5e8b21dec Mon Sep 17 00:00:00 2001 From: Leonardo <69599495+leorossi2005@users.noreply.github.com> Date: Sun, 11 Oct 2026 07:43:20 +0200 Subject: [PATCH 371/460] docs: document sceAppContentGetAddcontDownloadProgress in technical debt --- docs/dev/TechnicalDebt.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index b2c670076..5193136fb 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -308,6 +308,7 @@ Throughout the project, every function at every stage either **does exactly what - [sched_get_priority_max, sched_get_priority_min](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - `SCHED_OTHER` (2) reports the same 256 to 767 range as `SCHED_FIFO` and `SCHED_RR`; the console's answer is unverified. FreeBSD accepts `SCHED_OTHER` with its own range, and shadPS4 rejects it with `EINVAL`. `pthread_setschedparam` ignores the policy, so a priority taken from this range applies - [gethostbyname](../../core/libs/prx/libScePosixForWebKit/Resolver.cpp) (libScePosixForWebKit) - numeric IPv4 forms, trailing whitespace and 64-bit accumulation follow FreeBSD 11.4 [inet_aton](https://github.com/freebsd/freebsd-src/blob/releng/11.4/lib/libc/inet/inet_addr.c); clearing the resolver error on numeric success follows [fakeaddr](https://github.com/freebsd/freebsd-src/blob/releng/11.4/lib/libc/net/gethostnamadr.c). Exact PS5 behavior is unverified; hostname lookup still uses the host resolver +- [sceAppContentGetAddcontDownloadProgress](../../core/libs/prx/libSceAppContent/Export.cpp) (libSceAppContent) - imported by PPSA08755, which fails to load without it; added without an implementation because its signature is unknown ### Functional From 41b2841b0a74c5bd0f871953b3a3880fd0ed2820 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 07:48:17 +0200 Subject: [PATCH 372/460] docs: format sceAppContentGetAddcontDownloadProgress technical debt line Co-Authored-By: Claude --- docs/dev/TechnicalDebt.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 5193136fb..bdd5201ea 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -308,7 +308,8 @@ Throughout the project, every function at every stage either **does exactly what - [sched_get_priority_max, sched_get_priority_min](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - `SCHED_OTHER` (2) reports the same 256 to 767 range as `SCHED_FIFO` and `SCHED_RR`; the console's answer is unverified. FreeBSD accepts `SCHED_OTHER` with its own range, and shadPS4 rejects it with `EINVAL`. `pthread_setschedparam` ignores the policy, so a priority taken from this range applies - [gethostbyname](../../core/libs/prx/libScePosixForWebKit/Resolver.cpp) (libScePosixForWebKit) - numeric IPv4 forms, trailing whitespace and 64-bit accumulation follow FreeBSD 11.4 [inet_aton](https://github.com/freebsd/freebsd-src/blob/releng/11.4/lib/libc/inet/inet_addr.c); clearing the resolver error on numeric success follows [fakeaddr](https://github.com/freebsd/freebsd-src/blob/releng/11.4/lib/libc/net/gethostnamadr.c). Exact PS5 behavior is unverified; hostname lookup still uses the host resolver -- [sceAppContentGetAddcontDownloadProgress](../../core/libs/prx/libSceAppContent/Export.cpp) (libSceAppContent) - imported by PPSA08755, which fails to load without it; added without an implementation because its signature is unknown + +- [sceAppContentGetAddcontDownloadProgress](../../core/libs/prx/libSceAppContent/Export.cpp) (libSceAppContent) - imported by PPSA08755, which fails to load without it; added without an implementation because its behavior is unverified. [SharpProspero](https://github.com/SvenGDK/SharpProspero) lists `(uint32_t serviceLabel, entitlementLabel, SceAppContentAddcontDownloadProgress*)`, but no PS5 reference gives the progress layout or the error codes ### Functional From 056534a969e102a47b88dc493e5a41e91a046bf5 Mon Sep 17 00:00:00 2001 From: MrMha <114929652+MrMha@users.noreply.github.com> Date: Sun, 11 Oct 2026 09:54:17 +0400 Subject: [PATCH 373/460] test(agc): finish persistent label waits with real producers Apply the three-case test correction offered by Scramblrrrr in 36b26122a5458028f76fe7c7710333fcfca80f6d on issue #2832. Preserve all current-main test additions outside those three functions. Credit the supplied implementation to its contributor. Independent Windows Vulkan results and the timeout-return negative control are reported in issue #2832. Local validation: exact three-function source match, preservation of unrelated Submit.cpp content, conventions and whitespace. Native Windows/GPU execution was not performed here. Shutdown cancellation and persistent-wait policy remain unresolved. AI-assisted: yes. Co-authored-by: Scramblrrrr <70121301+Scramblrrrr@users.noreply.github.com> --- .../libs/prx/libSceAgcDriver/tests/Submit.cpp | 30 +++++++++++++------ 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp index 4137e06b3..0dccf2821 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp @@ -279,7 +279,7 @@ void testEndOfPipeLabelsWithoutWork() { } void testLabelStoredSinceSubmission() { - alignas(64) static volatile std::uint32_t gate = 0, label = 0, done = 0, late = 0; + alignas(64) static volatile std::uint32_t gate = 0, label = 0, done = 0, late = 0, waiting = 0; submit(0x20, commands(waitEqual(&gate, 1), waitEqual(&label, 1), writeData(&done, 1))); submit(0, commands(writeData(&label, 1))); waitFor(&label, 1, "producer label never landed"); @@ -287,13 +287,17 @@ void testLabelStoredSinceSubmission() { gate = 1; check(waitFor(&done, 1, "consumer never passed its waits") < std::chrono::milliseconds(500), "a label stored after the wait's submission did not satisfy it"); AgcDriverWaitIdle_nid_postfix(); - submit(0x20, commands(waitEqual(&label, 1), writeData(&late, 1))); - check(waitFor(&late, 1, "consumer never passed its wait") >= std::chrono::milliseconds(900), "a label stored before the wait's submission satisfied it"); + submit(0x20, commands(writeData(&waiting, 1), waitEqual(&label, 1), writeData(&late, 1))); + waitFor(&waiting, 1, "consumer never reached its label wait"); + std::this_thread::sleep_for(std::chrono::milliseconds(1200)); + check(late == 0, "a label stored before the wait's submission satisfied it or the wait expired"); + submit(0, commands(writeData(&label, 1))); + waitFor(&late, 1, "consumer did not resume after a new producer label"); AgcDriverWaitIdle_nid_postfix(); } void testWideLabelStoredSinceSubmission() { - alignas(64) static volatile std::uint32_t gate = 0, done = 0, late = 0; + alignas(64) static volatile std::uint32_t gate = 0, done = 0, late = 0, waiting = 0; alignas(64) static volatile std::uint32_t label[2] = {0, 0x5eed}; submit(0x20, commands(waitEqual(&gate, 1), waitEqual64(label, 1, 0xffffffffu), writeData(&done, 1))); submit(0, commands(writeData(label, 1))); @@ -303,17 +307,21 @@ void testWideLabelStoredSinceSubmission() { check(waitFor(&done, 1, "consumer never passed its waits") < std::chrono::milliseconds(500), "a 32-bit label stored after a low-dword 64-bit wait's submission did not satisfy it"); AgcDriverWaitIdle_nid_postfix(); gate = 0; - submit(0x20, commands(waitEqual(&gate, 1), waitEqual64(label, 1, ~0ull), writeData(&late, 1))); + submit(0x20, commands(waitEqual(&gate, 1), writeData(&waiting, 1), waitEqual64(label, 1, ~0ull), writeData(&late, 1))); submit(0, commands(writeData(label, 1))); waitFor(label, 1, "producer label never landed"); label[0] = 0; gate = 1; - check(waitFor(&late, 1, "consumer never passed its wait") >= std::chrono::milliseconds(900), "a 32-bit store satisfied a 64-bit wait whose high dword never matched"); + waitFor(&waiting, 1, "consumer never reached its wide label wait"); + std::this_thread::sleep_for(std::chrono::milliseconds(1200)); + check(late == 0, "a mismatched high dword satisfied the 64-bit wait or the wait expired"); + submit(0, commands(writeData(label + 1, 0), writeData(label, 1))); + waitFor(&late, 1, "consumer did not resume after a matching 64-bit label"); AgcDriverWaitIdle_nid_postfix(); } void testLabelHeldAtSubmission() { - alignas(64) static volatile std::uint32_t gate = 0, label = 1, done = 0, reset = 0; + alignas(64) static volatile std::uint32_t gate = 0, label = 1, done = 0, reset = 0, waiting = 0; submit(0x20, commands(waitEqual(&gate, 1), waitEqual(&label, 1), writeData(&done, 1))); label = 0; gate = 1; @@ -321,9 +329,13 @@ void testLabelHeldAtSubmission() { AgcDriverWaitIdle_nid_postfix(); gate = 0; label = 1; - submit(0x20, commands(waitEqual(&gate, 1), writeData(&label, 0), waitEqual(&label, 1), writeData(&reset, 1))); + submit(0x20, commands(waitEqual(&gate, 1), writeData(&label, 0), writeData(&waiting, 1), waitEqual(&label, 1), writeData(&reset, 1))); gate = 1; - check(waitFor(&reset, 1, "consumer never passed its wait") >= std::chrono::milliseconds(900), "a label its own queue stored first counted as held at the submission"); + waitFor(&waiting, 1, "consumer never reached its reset label wait"); + std::this_thread::sleep_for(std::chrono::milliseconds(1200)); + check(reset == 0, "a label reset by its own queue satisfied the wait or the wait expired"); + submit(0x28, commands(writeData(&label, 1))); + waitFor(&reset, 1, "consumer did not resume after another queue restored the label"); AgcDriverWaitIdle_nid_postfix(); } From a48b39e9339ba9855d6726f2109d66bab973d35b Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Thu, 8 Oct 2026 15:27:34 -0500 Subject: [PATCH 374/460] feat(agc): run unregistered NGG vertex programs that need a workgroup on the mesh path Co-Authored-By: Claude Opus 5.5 --- .../include/Driver/Shaders/ShaderRegistry.hpp | 2 ++ .../Execution/src/Driver/Draw/DrawDecode.cpp | 11 ++++++++++- .../src/Driver/Shaders/ShaderRegistry.cpp | 16 ++++++++++++++++ .../libSceAgcDriver/Graphics/include/State.hpp | 4 ++-- .../prx/libSceAgcDriver/Graphics/src/State.cpp | 18 ++++++++++-------- 5 files changed, 40 insertions(+), 11 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp index 297686f5e..ffec875b5 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/Driver/Shaders/ShaderRegistry.hpp @@ -81,6 +81,7 @@ struct ShaderSnapshot { std::vector header; std::shared_ptr prepared = std::make_shared(); std::shared_ptr registeredState; + std::shared_ptr> workgroupUse = std::make_shared>(0u); }; @@ -88,6 +89,7 @@ std::shared_ptr ReadRawShader(std::uint64_t address); std::shared_ptr ProgramSnapshot(const ShaderRegistry& shaders, std::uint64_t address, std::initializer_list types); void NoteRawGraphicsDraw(); bool RawGraphicsDrawn(); +bool ProgramUsesWorkgroup(const ShaderSnapshot& snapshot, std::size_t codeOffset); std::shared_ptr PrepareShaderWithDiagnostics(const ShaderRecompiler::RecompileRequest& request, ShaderRecompiler::ShaderPreparationContext* preparation = nullptr); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp index be0f1794d..1a8180fd9 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/DrawDecode.cpp @@ -109,7 +109,16 @@ void DecodeGraphicsPrograms(DrawDecode& decoded, const QueueState& queue, const std::shared_ptr Driver::decodeDraw(const QueueState& queue, const Submission& submission) { auto product = std::make_shared(); - product->state = Graphics::DecodeState(queue); + bool vertexWorkgroup = false; + const auto stages = queue.context.find(0x2d5); + const auto low = queue.shader.find(0xc8); + const auto high = queue.shader.find(0xc9); + if (stages != queue.context.end() && (stages->second & 0x02002024u) == 0x2000u && low != queue.shader.end() && high != queue.shader.end()) { + const auto address = (static_cast(low->second) << 8u) | (static_cast(high->second & 0xffu) << 40u); + const auto front = ProgramSnapshot(*submission.shaders, address, {2}); + vertexWorkgroup = front->header.empty() && ProgramUsesWorkgroup(*front, static_cast((address - front->codeAddress) / sizeof(std::uint32_t))); + } + product->state = Graphics::DecodeState(queue, vertexWorkgroup); DecodeGraphicsPrograms(*product, queue, *submission.shaders, false, true); product->pixel = Graphics::DecodePixelStageInfo(queue.context, Graphics::ExportMappings(product->state), Graphics::PixelProgramSkipped(queue)); product->pixel.targetExportPacking = Graphics::ExportPackings(product->state); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp index 75fb76947..2d564856c 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -131,6 +132,21 @@ void AwaitRegisteredPreparation(PreparedShaders& prepared, std::unique_lockload(std::memory_order_acquire); + if (cached != Unknown) return cached == Used; + } + const auto decoded = ShaderRecompiler::RdnaInstructionDecoder{}.Decode(std::span(snapshot.code).subspan(codeOffset)); + const bool used = std::any_of(decoded.instructions.begin(), decoded.instructions.end(), [](const auto& instruction) { + return instruction.family == ShaderRecompiler::RdnaInstructionFamily::DS || instruction.op == ShaderRecompiler::RdnaOpcode::SBarrier; + }); + if (codeOffset == 0) snapshot.workgroupUse->store(used ? Used : Unused, std::memory_order_release); + return used; +} + + struct ShaderPreparationTransaction::State { struct Change { std::shared_ptr destination; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp index 05cb27663..b3a5e2970 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp @@ -109,8 +109,8 @@ struct State { std::array blendConstants; }; -ShaderStages DecodeShaderStages(const QueueState& queue); -State DecodeState(const QueueState& queue); +ShaderStages DecodeShaderStages(const QueueState& queue, bool vertexWorkgroup = false); +State DecodeState(const QueueState& queue, bool vertexWorkgroup = false); std::array ExportMappings(const State& state); std::array ExportPackings(const State& state); ColorTarget DecodeColorBuffer(const Registers& context, std::uint32_t slot); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp index f29320922..6098e17bd 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp @@ -477,7 +477,7 @@ void intersect(VkRect2D& result, const Registers& registers, std::uint32_t offse } -ShaderStages DecodeShaderStages(const QueueState& queue) { +ShaderStages DecodeShaderStages(const QueueState& queue, bool vertexWorkgroup) { const auto value = read(queue.context, 0x2d5); const auto validate = [&](bool condition, const char* reason) { if (condition) return; @@ -489,7 +489,8 @@ ShaderStages DecodeShaderStages(const QueueState& queue) { validate((value & 3u) != 3u && ((value >> 3u) & 3u) != 3u && ((value >> 6u) & 3u) != 3u, "reserved LS_EN, ES_EN or VS_EN encoding"); const auto primitive = read(queue.userConfig, 0x242, RegisterBank::UserConfig); const bool tessellation = primitive == 9; - const bool geometry = (value & 0x20u) != 0; + validate(!vertexWorkgroup || ((value & 0x02002020u) == 0x2000u && !tessellation), "a vertex program that needs a workgroup requires PRIMGEN_EN without GS_EN, PRIMGEN_PASSTHRU_EN or tessellation"); + const bool geometry = (value & 0x20u) != 0 || vertexWorkgroup; validate(tessellation == ((value & 4u) != 0), "Patch topology and HS_EN disagree"); validate(!tessellation || !geometry, "combined tessellation and geometry is unsupported by the reference path"); const auto path = tessellation ? ShaderPath::Tessellation : geometry ? ShaderPath::Geometry : ShaderPath::Vertex; @@ -513,15 +514,16 @@ ShaderStages DecodeShaderStages(const QueueState& queue) { const auto vertices = (group >> 9u) & 0x1ffu; const auto primitives = group & 0x1ffu; const auto maxVertices = read(queue.context, 0x1ff); - const auto verticesPerPrimitive = read(queue.context, 0x2ce); + const auto verticesPerPrimitive = vertexWorkgroup ? 3u : read(queue.context, 0x2ce); validate((primitive == 1 || primitive == 2 || primitive == 4 || primitive == 5 || primitive == 6) && read(queue.context, 0x29b) == 2 && verticesPerPrimitive >= 3, "unsupported geometry input or output assembly"); const auto inputSize = primitive == 1 ? 1u : primitive == 2 ? 2u : 3u; - validate(vertices >= inputSize && maxVertices != 0 && maxVertices <= 256 && verticesPerPrimitive <= 256, "invalid geometry subgroup output"); + validate(vertices >= inputSize && maxVertices != 0 && (!vertexWorkgroup || maxVertices >= inputSize) && maxVertices <= 256 && verticesPerPrimitive <= 256, "invalid geometry subgroup output"); const auto inputStep = primitive == 5 || primitive == 6 ? 1u : inputSize; - const auto groupPrimitives = std::min({primitives, (vertices - inputSize) / inputStep + 1u, maxVertices / verticesPerPrimitive}); + const auto outputLimit = vertexWorkgroup ? (maxVertices - inputSize) / inputStep + 1u : maxVertices / verticesPerPrimitive; + const auto groupPrimitives = std::min({primitives, (vertices - inputSize) / inputStep + 1u, outputLimit}); validate(groupPrimitives != 0, "geometry subgroup contains no primitives"); const auto resources = read(queue.shader, 0x8b, RegisterBank::Shader); - validate(((read(queue.shader, 0x8a, RegisterBank::Shader) >> 29u) & 3u) == 3 && ((resources >> 16u) & 3u) == 3, "unsupported geometry VGPR allocation"); + validate((vertexWorkgroup || ((read(queue.shader, 0x8a, RegisterBank::Shader) >> 29u) & 3u) == 3) && ((resources >> 16u) & 3u) == 3, "unsupported geometry VGPR allocation"); const auto esgsItemSize = read(queue.context, 0x2ab); validate(esgsItemSize != 0 && esgsItemSize * vertices <= 0xffffu, "invalid VGT_ESGS_RING_ITEMSIZE"); const auto threads = std::max({(groupPrimitives - 1u) * inputStep + inputSize, primitives, maxVertices, primitives * (verticesPerPrimitive - 2u)}); @@ -539,12 +541,12 @@ std::string DepthMaintenanceRejection(const QueueState& queue) { return zeroMessage(0x000, control->second, "DB_RENDER_CONTROL depth copy, resummarize or decompress"); } -State DecodeState(const QueueState& queue) { +State DecodeState(const QueueState& queue, bool vertexWorkgroup) { if (auto reason = DepthMaintenanceRejection(queue); !reason.empty()) throw std::runtime_error(reason); const auto& cx = queue.context; State result{}; result.conservativeRasterization = decodeConservativeRasterization(queue); - result.stages = DecodeShaderStages(queue); + result.stages = DecodeShaderStages(queue, vertexWorkgroup); const auto primitive = read(queue.userConfig, 0x242, RegisterBank::UserConfig); APS5_LOG_OUT_DEBUG("DecodeState primitive=%u path=%u vertexWave=%u", primitive, static_cast(result.stages.path), result.stages.vertexWaveSize); switch (primitive) { From 0a1b31f5c4e53015abd56696e4e01b4e0ea37d49 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Thu, 8 Oct 2026 15:31:07 -0500 Subject: [PATCH 375/460] test(agc): cover NGG vertex programs routed to the mesh path Co-Authored-By: Claude Opus 5.5 --- .../prx/libSceAgcDriver/tests/Graphics.cpp | 31 +++++++++++++++++++ .../libs/prx/libSceAgcDriver/tests/Submit.cpp | 16 ++++++++++ 2 files changed, 47 insertions(+) diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index 7deea9485..516c3b165 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -468,6 +468,36 @@ void ShaderStageTests() { expectFailure([&] { AgcDriver::Graphics::DecodeState(queue); }, "missing register"); } +void NggVertexWorkgroupTests() { + auto queue = makeState(); + queue.context[0x2d5] = 0x12010; + queue.userConfig[0x242] = 4; + queue.userConfig[0x25b] = 0x10080; + queue.context[0x1ff] = 0x80; + queue.context[0x2ce] = 0; + queue.context[0x29b] = 2; + queue.context[0x2ab] = 1; + queue.shader[0x8a] = 0x222c0009; + queue.shader[0x8b] = 0x43001e; + queue.context[0x1b3] = 2; + queue.context[0x1b4] = 2; + Require(AgcDriver::Graphics::DecodeState(queue).stages.path == AgcDriver::Graphics::ShaderPath::Vertex, "an NGG vertex program without a workgroup left the vertex path"); + auto state = AgcDriver::Graphics::DecodeState(queue, true); + Require(state.stages.path == AgcDriver::Graphics::ShaderPath::Geometry && state.stages.mesh, "an NGG vertex program that needs a workgroup did not take the mesh path"); + auto mesh = *state.stages.mesh; + Require(mesh.inputPrimitive == 4 && mesh.primitivesPerGroup == 42 && mesh.verticesPerGroup == 126 && mesh.maxVertices == 128 && mesh.maxPrimitives == 128 && mesh.threadsPerGroup == 128 && mesh.ldsSizeDwords == 1024 && mesh.esgsItemSize == 1, "the NGG vertex subgroup of a triangle list changed"); + queue.userConfig[0x242] = 6; + mesh = *AgcDriver::Graphics::DecodeState(queue, true).stages.mesh; + Require(mesh.primitivesPerGroup == 126 && mesh.verticesPerGroup == 128 && mesh.maxPrimitives == 128, "the NGG vertex subgroup of a triangle strip is not bounded by its vertex budget"); + queue.context[0x1ff] = 2; + expectFailure([&] { AgcDriver::Graphics::DecodeState(queue, true); }, "invalid geometry subgroup output"); + queue.context[0x1ff] = 0x80; + for (const auto stages : {0x02012010u, 0x00012030u, 0x00010010u}) { + queue.context[0x2d5] = stages; + expectFailure([&] { AgcDriver::Graphics::DecodeState(queue, true); }, "requires PRIMGEN_EN without GS_EN, PRIMGEN_PASSTHRU_EN or tessellation"); + } +} + constexpr std::array IdentityExports{0xe4u, 0xe4u, 0xe4u, 0xe4u, 0xe4u, 0xe4u, 0xe4u, 0xe4u}; std::vector pixelBuiltinsRead(std::uint32_t ena, std::uint32_t addr, std::uint32_t source) { @@ -4043,6 +4073,7 @@ int main() { uint16ExportTests(); uint8x4TargetTests(); ShaderStageTests(); + NggVertexWorkgroupTests(); TuningFieldTests(); PixelInputLayoutTests(); ColorPipeBankXorTests(); diff --git a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp index 4137e06b3..e6a9d0384 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Submit.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Submit.cpp @@ -602,6 +602,21 @@ void testProgramSnapshots() { check(AgcDriver::DriverDetail::DecodeProgramsCurrent(decode, shaders), "a decode with the rewritten program was not current"); } +void testProgramWorkgroupUse() { + const auto snapshot = [](std::vector code) { + AgcDriver::DriverDetail::ShaderSnapshot result{0x1000, 0, 2, std::move(code), {}}; + return result; + }; + const auto plain = snapshot({0xbf800000, 0xbf810000}); + check(!AgcDriver::DriverDetail::ProgramUsesWorkgroup(plain, 0) && !AgcDriver::DriverDetail::ProgramUsesWorkgroup(plain, 0), "a program without LDS or barriers needs a workgroup"); + const auto barrier = snapshot({0xbf8a0000, 0xbf810000}); + check(AgcDriver::DriverDetail::ProgramUsesWorkgroup(barrier, 0), "an s_barrier did not need a workgroup"); + const auto lds = snapshot({0xd8340000, 0x00000100, 0xbf810000}); + check(AgcDriver::DriverDetail::ProgramUsesWorkgroup(lds, 0) && AgcDriver::DriverDetail::ProgramUsesWorkgroup(lds, 0), "an LDS write did not need a workgroup"); + const auto entry = snapshot({0xbf8a0000, 0xbf810000, 0xbf800000, 0xbf810000}); + check(!AgcDriver::DriverDetail::ProgramUsesWorkgroup(entry, 2) && AgcDriver::DriverDetail::ProgramUsesWorkgroup(entry, 0), "the workgroup check did not start at the program's entry"); +} + void testWorkerFailure() { std::array words{0xc0031500, 1, 1, 1, 0x41}; Packet packet{words.data(), static_cast(words.size()), 0, {}}; @@ -694,6 +709,7 @@ int main() { check(!unterminated.empty() && bounded->code.size() == 64, "raw compute crossed inaccessible memory or missed its last instruction"); #endif testProgramSnapshots(); + testProgramWorkgroupUse(); testEvents(); testValidation(); testClearState(); From 0e3449129e1047d64c47a8b0858f23becda9c1e3 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Thu, 8 Oct 2026 22:32:58 -0500 Subject: [PATCH 376/460] fix(agc): accept any ES VGPR count for NGG vertex programs on the mesh path vkQuake's NGG culling vertex shaders load only the vertex ID (SPI_SHADER_PGM_RSRC2_GS ES_VGPR_COMP_CNT 0), and DecodeShaderStages still required 3 for a program routed to the mesh path. The merged ES inputs keep their positions whatever the count, and the mesh entry loads all of them, so the count is now checked only for real geometry programs, as the GS VGPR count already was. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp | 2 +- core/libs/prx/libSceAgcDriver/tests/Graphics.cpp | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp index 6098e17bd..2785bf7bf 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp @@ -523,7 +523,7 @@ ShaderStages DecodeShaderStages(const QueueState& queue, bool vertexWorkgroup) { const auto groupPrimitives = std::min({primitives, (vertices - inputSize) / inputStep + 1u, outputLimit}); validate(groupPrimitives != 0, "geometry subgroup contains no primitives"); const auto resources = read(queue.shader, 0x8b, RegisterBank::Shader); - validate((vertexWorkgroup || ((read(queue.shader, 0x8a, RegisterBank::Shader) >> 29u) & 3u) == 3) && ((resources >> 16u) & 3u) == 3, "unsupported geometry VGPR allocation"); + validate(vertexWorkgroup || (((read(queue.shader, 0x8a, RegisterBank::Shader) >> 29u) & 3u) == 3 && ((resources >> 16u) & 3u) == 3), "unsupported geometry VGPR allocation"); const auto esgsItemSize = read(queue.context, 0x2ab); validate(esgsItemSize != 0 && esgsItemSize * vertices <= 0xffffu, "invalid VGT_ESGS_RING_ITEMSIZE"); const auto threads = std::max({(groupPrimitives - 1u) * inputStep + inputSize, primitives, maxVertices, primitives * (verticesPerPrimitive - 2u)}); diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index 516c3b165..59ecfc305 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -489,6 +489,11 @@ void NggVertexWorkgroupTests() { queue.userConfig[0x242] = 6; mesh = *AgcDriver::Graphics::DecodeState(queue, true).stages.mesh; Require(mesh.primitivesPerGroup == 126 && mesh.verticesPerGroup == 128 && mesh.maxPrimitives == 128, "the NGG vertex subgroup of a triangle strip is not bounded by its vertex budget"); + queue.shader[0x8a] = 0x222c0005; + queue.shader[0x8b] = 0x400030; + Require(AgcDriver::Graphics::DecodeState(queue, true).stages.mesh.has_value(), "an NGG vertex program that loads only its vertex ID was rejected"); + queue.shader[0x8a] = 0x222c0009; + queue.shader[0x8b] = 0x43001e; queue.context[0x1ff] = 2; expectFailure([&] { AgcDriver::Graphics::DecodeState(queue, true); }, "invalid geometry subgroup output"); queue.context[0x1ff] = 0x80; From fdc0b5268be96fbe279907bf0f1c533eef30edb4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 07:56:42 +0200 Subject: [PATCH 377/460] docs(technical-debt): record the NGG vertex mesh routing assumptions --- docs/dev/TechnicalDebt.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 8dae145c5..eb9495555 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -126,6 +126,7 @@ Throughout the project, every function at every stage either **does exactly what - [SET_PREDICATION](../../core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp) (AGC driver) - boolean predication follows the AMD PM4 encoding [Mesa](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/common/sid.h) uses: operation 3 reads 64 bits and 4 reads 32 bits, and bit 8 runs predicated packets when the value is non-zero (clear: when it is zero); the bit 12 hint is ignored. The address must have its low 4 bits clear, as `sceAgcDcbSetPredication` writes it. Query predication (Z-pass, primitive count), a predicated COND_EXEC, predicated command buffer chains and flips in predicated command buffers throw - [sceAgcGetGsOversubscription](../../core/libs/prx/libSceAgc/Shader/src/GsOversubscription.cpp) (libSceAgc) - signature and occupancy formula (base and expanded vertex/export capacities, budget shift, `GE_PC_ALLOC` / `SPI_SHADER_PGM_RSRC4_GS` encoding) follow KytyPS5 only, not confirmed on hardware or a title; a negative or NaN factor that gives a negative target, a missing GS context register or a zero `GE_MAX_OUTPUT_PER_SUBGROUP` throws - [Vertex entry merged wave info](../../core/shader/recompiler/Translation/src/InstructionTranslator.cpp) (recompiler) - `s3` of every vertex-path shader is set to `waveSize | waveSize << 8` (vertex and primitive count of a full wave, wave index 0). Layout taken from the prologue Mesa's ACO emits for NGG vertex shaders (`s_bfe_u64 exec` from `s3`), not measured on hardware. The driver gives every graphics program user data from `s8`, so the condition does not tell NGG from legacy vertex shaders: they just never read `s3`. A draw whose last wave is partial still reports a full wave. +- [NGG vertex programs on the mesh path](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (AGC driver) - an unregistered vertex program with `PRIMGEN_EN` that uses LDS or `s_barrier` runs on the geometry (mesh) path. Each input primitive is assumed to yield at most one output primitive, the subgroup is bounded by `GE_MAX_OUTPUT_PER_SUBGROUP` as a vertex budget, and the GS/ES VGPR count checks are skipped, because the merged inputs keep their positions. Taken from the code Mesa's ACO emits for NGG culling and from PPSA59322 and PPSA99010 draws, not measured on hardware. Registered programs keep the vertex path - [s_barrier in vertex-path programs](../../core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp) (recompiler) - emits no instruction when the program has no LDS or GDS access and no memory write or atomic, so no wave of the subgroup can observe another; the `GS_ALLOC_REQ` such a barrier orders on NGG passthrough programs is already dropped on the vertex path. The Vertex execution model has no workgroup barrier. Programs with LDS, GDS or memory writes throw. Not measured on hardware - [Strided buffer overhang](../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp) (libSceAgcDriver) - a strided buffer descriptor that ends less than one stride past the end of the readable registered allocation holding its base is bound only up to that allocation's end, even when another readable allocation starts right there. This assumes the bytes past the end are the rounded-up `num_records` of the last record (RADV rounds it up), which no fetch inside the buffer reads; a guest that really keeps a vertex split across two adjacent allocations would read zeros for the cut tail. Taken from the RetroArch Dolphin core case in #2912, not measured on hardware. - [Zero-stride vertex fetch range](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - for a stride-0 V# with OOB_SELECT 2 the whole fetch reads memory unless NUM_RECORDS is 0; with OOB_SELECT 3 it reads zero when NUM_RECORDS is 0 and throws when the fetch only partly fits. Rules taken from the GFX10 OOB_SELECT table in Mesa [ac_descriptors.c](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/common/ac_descriptors.c), not measured on hardware. Whether OOB_SELECT 3 checks each dword of a multi-dword fetch separately is unknown. OOB_SELECT 0 and 1 keep the structured record checks. From 8ad1bcb58988e53eb1558c18def4351d215ba296 Mon Sep 17 00:00:00 2001 From: SynnZer0 <243459655+SynnZer0@users.noreply.github.com> Date: Sun, 11 Oct 2026 02:02:38 -0400 Subject: [PATCH 378/460] fix(libkernel): ignore the input st_size of sceKernelGetModuleInfoFromAddr The bundled libc.prx of PPSA21703 calls sceKernelGetModuleInfoFromAddr with a 0x1A8-byte stack buffer and never sets st_size, so the field holds whatever was on the stack (983040 in one run) and the call threw. The console cannot require the field on input. The full size is still written back. The flags check is unchanged. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libkernel/Module/src/ModuleInfo.cpp | 2 -- core/libs/tests/GuestModuleInfo.cpp | 5 +++-- docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/core/libs/prx/libkernel/Module/src/ModuleInfo.cpp b/core/libs/prx/libkernel/Module/src/ModuleInfo.cpp index 2a8d8a5ac..66dc83653 100644 --- a/core/libs/prx/libkernel/Module/src/ModuleInfo.cpp +++ b/core/libs/prx/libkernel/Module/src/ModuleInfo.cpp @@ -276,8 +276,6 @@ extern "C" { int APS5_VABI sceKernelGetModuleInfoFromAddr(std::uint64_t address, int flags, ModuleInfoEx* info) { if (!info) return SCE_KERNEL_ERROR_EFAULT; if (flags != 2) throw std::invalid_argument("sceKernelGetModuleInfoFromAddr: unsupported flags " + std::to_string(flags)); - if (info->st_size != sizeof(ModuleInfoEx)) - throw std::invalid_argument("sceKernelGetModuleInfoFromAddr: unsupported st_size " + std::to_string(info->st_size)); ModuleInfoEx result{}; result.st_size = sizeof(ModuleInfoEx); #ifdef _WIN32 diff --git a/core/libs/tests/GuestModuleInfo.cpp b/core/libs/tests/GuestModuleInfo.cpp index 9cfc631e0..b63e20a04 100644 --- a/core/libs/tests/GuestModuleInfo.cpp +++ b/core/libs/tests/GuestModuleInfo.cpp @@ -111,7 +111,8 @@ int main(int argc, char** argv) { ModuleInfoEx invalid{}; invalid.st_size = sizeof(ModuleInfoEx); Require(ThrowsInvalidArgument([&] { sceKernelGetModuleInfoFromAddr(reinterpret_cast(add), 1, &invalid); })); - invalid.st_size = sizeof(ModuleInfoEx) - 8; - Require(ThrowsInvalidArgument([&] { sceKernelGetModuleInfoFromAddr(reinterpret_cast(add), 2, &invalid); })); + ModuleInfoEx unset{}; + unset.st_size = 0xf0000; + Require(sceKernelGetModuleInfoFromAddr(reinterpret_cast(add), 2, &unset) == 0 && unset.st_size == sizeof(ModuleInfoEx)); Require(dlclose_nid_postfix(module) == 0); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 8dae145c5..2d9576016 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -145,7 +145,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceAgcDcbQueueEndOfShaderActionGetSize](../../core/libs/prx/libSceAgc/DcbState/src/ContextState.cpp) (libSceAgc) - the builder is inline in the SDK and not exported; the size is assumed to be that of the RELEASE_MEM packet `sceAgcCbReleaseMem` writes (32 bytes), the same as `sceAgcAcbQueueEndOfShaderActionGetSize`, as in [OpenAGC](https://github.com/OpenAGC/OpenAGC/blob/main/src/game_compat.c) and [REmu](https://github.com/xnetcat/REmu/blob/main/crates/remu-runtime/src/agc_dcb.rs) - [sceKernelInternalMemoryGetModuleSegmentInfo](../../core/libs/prx/libkernel/Module/src/Module.cpp) (libkernel) - unknown signature - [sceKernelGetModuleInfoForUnwind](../../core/libs/prx/libkernel/Module/src/Module.cpp) (libkernel) - on Linux the tables come from `sceKernelGetModuleInfoFromAddr`, whose `eh_frame_size` includes the zero terminator, as measured on firmware 13.60 (reported in [#2419](https://github.com/boykopovar/AnyPS5/issues/2419), not reproduced here); on Windows it is assumed to stop before the terminator. LLVM libunwind's `CFI_Parser::findFDE` stops at the terminator or the length, whichever comes first, so it finds the same FDEs either way. Only the main executable and `.guest.prx` images report tables; other host images report none, as on Windows -- [sceKernelGetModuleInfoFromAddr](../../core/libs/prx/libkernel/Module/src/ModuleInfo.cpp) (libkernel) - the 0x1A8-byte info layout follows the [shadPS4](https://github.com/shadps4-emu/shadPS4) reimplementation; a title's `libc.prx` reads only `id` (PPSA14632). The meaning of `flags` is unknown; only 2 is accepted, as is only the full `st_size`. The id is the `sceKernelLoadStartModule` handle of an image opened through the loader and otherwise a stable id from the same range. The name is the host file name without `.guest.prx`. An image with more than 4 load segments (the converted executable has 7) reports its first 4. `tls_offset` is 0 and `ref_count` is 1, as the host loader exposes neither. On Windows the module is the PE image holding the address: a relinked image reports its `.elf` sections as segments and the eh_frame_hdr recorded in `.ehmeta`, any other image its first 4 sections with memory access and the `.ehfram` section without an eh_frame_hdr; the TLS fields and `init_proc_addr`/`fini_proc_addr` are 0 +- [sceKernelGetModuleInfoFromAddr](../../core/libs/prx/libkernel/Module/src/ModuleInfo.cpp) (libkernel) - the 0x1A8-byte info layout follows the [shadPS4](https://github.com/shadps4-emu/shadPS4) reimplementation; a title's `libc.prx` reads only `id` (PPSA14632). The meaning of `flags` is unknown; only 2 is accepted. The input `st_size` is ignored, because PPSA21703's `libc.prx` passes an uninitialized 0x1A8-byte buffer; the full size is written back. The id is the `sceKernelLoadStartModule` handle of an image opened through the loader and otherwise a stable id from the same range. The name is the host file name without `.guest.prx`. An image with more than 4 load segments (the converted executable has 7) reports its first 4. `tls_offset` is 0 and `ref_count` is 1, as the host loader exposes neither. On Windows the module is the PE image holding the address: a relinked image reports its `.elf` sections as segments and the eh_frame_hdr recorded in `.ehmeta`, any other image its first 4 sections with memory access and the `.ehfram` section without an eh_frame_hdr; the TLS fields and `init_proc_addr`/`fini_proc_addr` are 0 - [sceKernelGetModuleList](../../core/libs/prx/libkernel/Module/src/ModuleInfo.cpp) (libkernel) - lists the main program and every loaded `.prx` image in load order; host runtime libraries are left out. The handles are the ids `sceKernelGetModuleInfoFromAddr` reports, not the console's. The `(int32_t* handles, size_t count, size_t* actual)` signature and returning `ENOMEM` without writing `actual` after filling the first `count` handles when more images are loaded follow [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/0fe263a4760dfbfa973366890061749b4af0de97/src/core/libraries/kernel/process.cpp#L244-L264); whether the console writes any handles in that case is unknown - [sceKernelGetModuleInfo](../../core/libs/prx/libkernel/Module/src/ModuleInfo.cpp) (libkernel) - the 0x160-byte layout (`st_size`, `name[256]`, 4 segments, `segment_count`, `fingerprint[20]`) follows [fpPS4](https://github.com/red-prig/fpPS4/blob/04cefd43e6fddd1ab033e7980cd356d14c964905/sys/sys_types.pas#L319-L337) and [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/0fe263a4760dfbfa973366890061749b4af0de97/src/core/module.h#L15-L31). Only that `st_size` is accepted, as in shadPS4; other sizes return `EINVAL`, a null pointer `EFAULT` and a handle `sceKernelGetModuleList` does not list `ESRCH`. Name and segments are the ones `sceKernelGetModuleInfoFromAddr` reports, on Windows from the same PE sections, so `prot` uses the read/write/execute bits 1/2/4, while fpPS4 and shadPS4 store the ELF `p_flags`. The fingerprint is left zero, as the host image does not carry it - [sceKernelReleaseFlexibleMemory](../../core/libs/prx/libkernel/DirectMemory/Export.cpp) (libkernel) - unmaps the range like sceKernelMunmap, as fpPS4 and prosper do; whether the console keeps the virtual range reserved after releasing its pages is unverified From b6cc4fe7f9e3521dc4653fbc6f2a83ec4454d886 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 20:08:38 -0400 Subject: [PATCH 379/460] feat(videoout): implement sceVideoOutDeleteOutputModeEvent Unsubscribe an event queue from the output mode events that sceVideoOutAddOutputModeEvent registers, the same way the flip and vblank events are removed. Deleting a registration that does not exist succeeds, and a null queue or an invalid handle throws. Co-Authored-By: Gemini 3.8 Flash Co-Authored-By: Claude Sonnet 5.5 --- core/libs/prx/libSceVideoOut/include/Event.hpp | 1 + core/libs/prx/libSceVideoOut/src/Event.cpp | 6 ++++++ core/libs/prx/libSceVideoOut/tests/Flip.cpp | 8 ++++++++ 3 files changed, 15 insertions(+) diff --git a/core/libs/prx/libSceVideoOut/include/Event.hpp b/core/libs/prx/libSceVideoOut/include/Event.hpp index af8d10be4..46dade9d3 100644 --- a/core/libs/prx/libSceVideoOut/include/Event.hpp +++ b/core/libs/prx/libSceVideoOut/include/Event.hpp @@ -15,6 +15,7 @@ int APS5_VABI sceVideoOutAddVblankEvent(KernelEqueue eq, int handle, void* udata int APS5_VABI sceVideoOutAddVrrActiveStatusEvent(KernelEqueue eq, int handle, void* udata); int APS5_VABI sceVideoOutDeleteFlipEvent(KernelEqueue eq, int handle); +int APS5_VABI sceVideoOutDeleteOutputModeEvent(KernelEqueue eq, int handle); int APS5_VABI sceVideoOutDeletePreVblankStartEvent(KernelEqueue eq, int handle); int APS5_VABI sceVideoOutDeleteVblankEvent(KernelEqueue eq, int handle); diff --git a/core/libs/prx/libSceVideoOut/src/Event.cpp b/core/libs/prx/libSceVideoOut/src/Event.cpp index adef7235c..a4853cd0a 100644 --- a/core/libs/prx/libSceVideoOut/src/Event.cpp +++ b/core/libs/prx/libSceVideoOut/src/Event.cpp @@ -144,6 +144,12 @@ int APS5_VABI sceVideoOutDeleteFlipEvent(KernelEqueue eq, int handle) try { LibcAwaitExit_nid_postfix(); } +int APS5_VABI sceVideoOutDeleteOutputModeEvent(KernelEqueue eq, int handle) try { + return deleteVideoOutEvent(handle, eq, VIDEO_OUT_EVENT_SET_MODE); +} catch (const ProcessShutdown&) { + LibcAwaitExit_nid_postfix(); +} + int APS5_VABI sceVideoOutDeleteVblankEvent(KernelEqueue eq, int handle) try { return deleteVideoOutEvent(handle, eq, VIDEO_OUT_EVENT_VBLANK); } catch (const ProcessShutdown&) { diff --git a/core/libs/prx/libSceVideoOut/tests/Flip.cpp b/core/libs/prx/libSceVideoOut/tests/Flip.cpp index c883e240c..607870626 100644 --- a/core/libs/prx/libSceVideoOut/tests/Flip.cpp +++ b/core/libs/prx/libSceVideoOut/tests/Flip.cpp @@ -307,6 +307,14 @@ void testControls() { check(owner->GetTriggeredEvents(&event, 1) == 1, "initial output mode event missing"); int64_t mode = 0; check(sceVideoOutGetEventData(&event, &mode) == 0 && mode == VIDEO_OUT_OUTPUT_MODE_DEFAULT && sceVideoOutGetEventCount(&event) == 1, "initial output mode event encoding is wrong"); + check(sceVideoOutDeleteOutputModeEvent(queue, handle) == 0, "output mode unsubscription failed"); + { + std::lock_guard lock(cfg->mutex); + check(cfg->outputModeEvents.empty(), "output mode unsubscription did not clear registration"); + } + check(sceVideoOutDeleteOutputModeEvent(queue, handle) == 0, "duplicate output mode unsubscription failed"); + expectFailure([&] { sceVideoOutDeleteOutputModeEvent(0, handle); }); + expectFailure([&] { sceVideoOutDeleteOutputModeEvent(queue, -1); }); check(sceVideoOutAddVblankEvent(queue, handle, nullptr) == 0 && sceVideoOutAddVblankEvent(queue, handle, &settings) == 0, "vblank subscription failed"); { std::lock_guard lock(cfg->mutex); From 54d4024c2b97637482904ebce11bfdc0f12302b7 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 19:33:26 -0400 Subject: [PATCH 380/460] fix(shader): keep f32 subnormals in v_ldexp_f32 when denormals are kept With the f32 denormal field of FLOAT_MODE set to keep, v_ldexp_f32 flushed subnormal inputs and results to zero. Keep subnormal inputs and round a result that lands in the subnormal range in the f32 round mode of FLOAT_MODE. An overflow follows the round mode as well, with or without denormals kept: the largest finite value instead of infinity when the mode rounds that sign toward zero. This matches RDNA2 (gfx1036) on 2163 rows in each round mode. Denormal modes 1 and 2 now throw. Co-Authored-By: Claude Sonnet 5.5 --- .../tests/execution/LdexpF32Denormal.cpp | 163 ++++++++++++++++++ .../Translation/src/FloatInstructions.cpp | 47 ++++- docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 209 insertions(+), 3 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/LdexpF32Denormal.cpp diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/LdexpF32Denormal.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/LdexpF32Denormal.cpp new file mode 100644 index 000000000..fabe12346 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/LdexpF32Denormal.cpp @@ -0,0 +1,163 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::uint32_t Threads = 64; +constexpr std::uint32_t Inputs = 4; +constexpr std::uint32_t Results = 16; +alignas(256) std::array Input{}; +alignas(256) std::array Output{}; + +alignas(256) constexpr std::array Code{ + 0x34020084u, 0x34060086u, 0xe0381000u, 0x80000401u, 0xbf8c3f70u, 0xd762000au, 0x02020b04u, 0xe0701000u, + 0x80010a03u, 0xbf810000u, +}; + +struct LdexpVector { + std::uint32_t value; + std::uint32_t exponent; + std::uint32_t kept; + std::uint32_t keptIeee; + std::uint32_t flushed; + std::uint32_t keptTowardPositive; + std::uint32_t keptTowardNegative; + std::uint32_t keptTowardZero; + std::uint32_t flushedTowardPositive; + std::uint32_t flushedTowardNegative; + std::uint32_t flushedTowardZero; +}; + +constexpr std::array Vectors{{ + {0x00000001u, 0x00000003u, 0x00000008u, 0x00000008u, 0x00000000u, 0x00000008u, 0x00000008u, 0x00000008u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x00000001u, 0x00000016u, 0x00400000u, 0x00400000u, 0x00000000u, 0x00400000u, 0x00400000u, 0x00400000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x007fffffu, 0xffffffe9u, 0x00000001u, 0x00000001u, 0x00000000u, 0x00000001u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x007fffffu, 0xfffffffdu, 0x00100000u, 0x00100000u, 0x00000000u, 0x00100000u, 0x000fffffu, 0x000fffffu, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x00000001u, 0x00000019u, 0x01800000u, 0x01800000u, 0x00000000u, 0x01800000u, 0x01800000u, 0x01800000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x00000001u, 0x0000007du, 0x33800000u, 0x33800000u, 0x00000000u, 0x33800000u, 0x33800000u, 0x33800000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x00000001u, 0x000003e8u, 0x7f800000u, 0x7f800000u, 0x00000000u, 0x7f800000u, 0x7f7fffffu, 0x7f7fffffu, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x80000001u, 0x000003e8u, 0xff800000u, 0xff800000u, 0x80000000u, 0xff7fffffu, 0xff800000u, 0xff7fffffu, 0x80000000u, 0x80000000u, 0x80000000u}, + {0x80000001u, 0x00000003u, 0x80000008u, 0x80000008u, 0x80000000u, 0x80000008u, 0x80000008u, 0x80000008u, 0x80000000u, 0x80000000u, 0x80000000u}, + {0x3f800000u, 0xffffff6cu, 0x00000002u, 0x00000002u, 0x00000000u, 0x00000002u, 0x00000002u, 0x00000002u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x3f000000u, 0xffffff6cu, 0x00000001u, 0x00000001u, 0x00000000u, 0x00000001u, 0x00000001u, 0x00000001u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x3f000000u, 0xffffff82u, 0x00400000u, 0x00400000u, 0x00000000u, 0x00400000u, 0x00400000u, 0x00400000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0xbf000000u, 0xffffff6cu, 0x80000001u, 0x80000001u, 0x80000000u, 0x80000001u, 0x80000001u, 0x80000001u, 0x80000000u, 0x80000000u, 0x80000000u}, + {0x00800000u, 0xffffffe8u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000001u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x3eaaaaabu, 0xffffff83u, 0x00555556u, 0x00555556u, 0x00000000u, 0x00555556u, 0x00555555u, 0x00555555u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x80800000u, 0xffffffe8u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000001u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u}, + {0x3f7fffffu, 0xffffff82u, 0x00800000u, 0x00800000u, 0x00000000u, 0x00800000u, 0x007fffffu, 0x007fffffu, 0x00000000u, 0x00000000u, 0x00000000u}, + {0xbfc00000u, 0xffffff6au, 0x80000001u, 0x80000001u, 0x80000000u, 0x80000000u, 0x80000001u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u}, + {0x3f7fffffu, 0xffffff6cu, 0x00000002u, 0x00000002u, 0x00000000u, 0x00000002u, 0x00000001u, 0x00000001u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x3fc00000u, 0xffffff6cu, 0x00000003u, 0x00000003u, 0x00000000u, 0x00000003u, 0x00000003u, 0x00000003u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x00000000u, 0x00000003u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x80000000u, 0x00000003u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u}, + {0x7f800000u, 0x00000003u, 0x7f800000u, 0x7f800000u, 0x7f800000u, 0x7f800000u, 0x7f800000u, 0x7f800000u, 0x7f800000u, 0x7f800000u, 0x7f800000u}, + {0xff800000u, 0xfffffffdu, 0xff800000u, 0xff800000u, 0xff800000u, 0xff800000u, 0xff800000u, 0xff800000u, 0xff800000u, 0xff800000u, 0xff800000u}, + {0x7fc00000u, 0x00000003u, 0x7fc00000u, 0x7fc00000u, 0x7fc00000u, 0x7fc00000u, 0x7fc00000u, 0x7fc00000u, 0x7fc00000u, 0x7fc00000u, 0x7fc00000u}, + {0x7fa00000u, 0x00000003u, 0x7fa00000u, 0x7fe00000u, 0x7fa00000u, 0x7fa00000u, 0x7fa00000u, 0x7fa00000u, 0x7fa00000u, 0x7fa00000u, 0x7fa00000u}, + {0xffbfffffu, 0xfffffffdu, 0xffbfffffu, 0xffffffffu, 0xffbfffffu, 0xffbfffffu, 0xffbfffffu, 0xffbfffffu, 0xffbfffffu, 0xffbfffffu, 0xffbfffffu}, + {0x7f800001u, 0x00000019u, 0x7f800001u, 0x7fc00001u, 0x7f800001u, 0x7f800001u, 0x7f800001u, 0x7f800001u, 0x7f800001u, 0x7f800001u, 0x7f800001u}, + {0x3f800000u, 0x000003e8u, 0x7f800000u, 0x7f800000u, 0x7f800000u, 0x7f800000u, 0x7f7fffffu, 0x7f7fffffu, 0x7f800000u, 0x7f7fffffu, 0x7f7fffffu}, + {0x3f800000u, 0xfffffc17u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000001u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0x3f800000u, 0x00000081u, 0x7f800000u, 0x7f800000u, 0x7f800000u, 0x7f800000u, 0x7f7fffffu, 0x7f7fffffu, 0x7f800000u, 0x7f7fffffu, 0x7f7fffffu}, + {0xbf800000u, 0x00000081u, 0xff800000u, 0xff800000u, 0xff800000u, 0xff7fffffu, 0xff800000u, 0xff7fffffu, 0xff7fffffu, 0xff800000u, 0xff7fffffu}, + {0x40200000u, 0xffffff6au, 0x00000001u, 0x00000001u, 0x00000000u, 0x00000002u, 0x00000001u, 0x00000001u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0xc0e80000u, 0xffffff6cu, 0x8000000eu, 0x8000000eu, 0x80000000u, 0x8000000eu, 0x8000000fu, 0x8000000eu, 0x80000000u, 0x80000000u, 0x80000000u}, + {0x3f800000u, 0xfffe7960u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000001u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u, 0x00000000u}, + {0xbf800000u, 0xfffe7960u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000001u, 0x80000000u, 0x80000000u, 0x80000000u, 0x80000000u}, +}}; + +std::array BufferDescriptor(const void* data, std::uint32_t bytes) { + const auto address = reinterpret_cast(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu), bytes, 0x31016facu}; +} + +std::string Hex(std::uint32_t value) { + char text[16]; + std::snprintf(text, sizeof(text), "0x%08x", value); + return text; +} + +void Run(AgcDriver::VulkanDevice& device, const std::optional& floatMode) { + Input.fill(0u); + for (std::uint32_t lane = 0; lane < Vectors.size(); ++lane) { + Input[lane * Inputs] = Vectors[lane].value; + Input[lane * Inputs + 1] = Vectors[lane].exponent; + } + Output.fill(0xdeadbeefu); + std::vector userData(8, 0u); + const auto input = BufferDescriptor(Input.data(), static_cast(Input.size() * 4u)); + const auto output = BufferDescriptor(Output.data(), static_cast(Output.size() * 4u)); + std::copy(input.begin(), input.end(), userData.begin()); + std::copy(output.begin(), output.end(), userData.begin() + 4); + const std::span code(Code); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0, {false, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + request.context.floatMode = floatMode; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); +} + +void Check(std::uint32_t LdexpVector::*expected, const char* mode) { + for (std::uint32_t lane = 0; lane < Vectors.size(); ++lane) { + const auto& vector = Vectors[lane]; + const auto actual = Output[lane * Results]; + Require(actual == vector.*expected, std::string("ldexp f32 denormals: ") + mode + " v_ldexp_f32 (" + Hex(vector.value) + ", " + Hex(vector.exponent) + ") is " + Hex(actual) + ", expected " + Hex(vector.*expected)); + } +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + Run(*device, ShaderRecompiler::ShaderFloatMode{0xf0u, true, false, false}); + Check(&LdexpVector::kept, "denormals kept, IEEE=0"); + Run(*device, ShaderRecompiler::ShaderFloatMode{0xf0u, true, true, false}); + Check(&LdexpVector::keptIeee, "denormals kept, IEEE=1"); + Run(*device, ShaderRecompiler::ShaderFloatMode{0xf1u, true, false, false}); + Check(&LdexpVector::keptTowardPositive, "denormals kept, round toward +inf"); + Run(*device, ShaderRecompiler::ShaderFloatMode{0xf2u, true, false, false}); + Check(&LdexpVector::keptTowardNegative, "denormals kept, round toward -inf"); + Run(*device, ShaderRecompiler::ShaderFloatMode{0xf3u, true, false, false}); + Check(&LdexpVector::keptTowardZero, "denormals kept, round toward zero"); + Run(*device, ShaderRecompiler::ShaderFloatMode{0xc0u, true, false, false}); + Check(&LdexpVector::flushed, "denormals flushed"); + Run(*device, ShaderRecompiler::ShaderFloatMode{0xc1u, true, false, false}); + Check(&LdexpVector::flushedTowardPositive, "denormals flushed, round toward +inf"); + Run(*device, ShaderRecompiler::ShaderFloatMode{0xc2u, true, false, false}); + Check(&LdexpVector::flushedTowardNegative, "denormals flushed, round toward -inf"); + Run(*device, ShaderRecompiler::ShaderFloatMode{0xc3u, true, false, false}); + Check(&LdexpVector::flushedTowardZero, "denormals flushed, round toward zero"); + Run(*device, std::nullopt); + Check(&LdexpVector::flushed, "no float mode"); + std::puts("ldexp f32 denormal tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/Translation/src/FloatInstructions.cpp b/core/shader/recompiler/Translation/src/FloatInstructions.cpp index b74c1be2d..3262eadda 100644 --- a/core/shader/recompiler/Translation/src/FloatInstructions.cpp +++ b/core/shader/recompiler/Translation/src/FloatInstructions.cpp @@ -698,10 +698,53 @@ bool TranslationContext::vLdexpF32(const RdnaInstruction& inst) { const IrU32 sign(ir.BitwiseAnd(bits.Value(), ir.Constant(0x80000000u))); const IrU32 exponent(ir.Emit(IrOpcode::BitFieldUExtract, IrType::U32, {&bits.Value(), &ir.Constant(23u), &ir.Constant(8u)})); const IrU32 clamped(ir.Emit(IrOpcode::SMax32, IrType::U32, {&ir.Emit(IrOpcode::SMin32, IrType::U32, {&offset.Value(), &ir.Constant(512u)}), &ir.Constant(static_cast(-512))})); + const IrU32 overflow(ir.BitwiseOr(sign.Value(), ir.Constant(0x7f800000u))); + const std::uint32_t denorm32 = floatMode.has_value() ? (floatMode->floatMode >> 4u) & 3u : 0u; + const std::uint32_t roundMode = floatMode.has_value() ? floatMode->floatMode & 3u : 0u; + IrValue* positive = roundMode == 1u || roundMode == 2u ? &ir.IEqual(sign.Value(), ir.Constant(0u)) : nullptr; + IrValue* overflowBits = &overflow.Value(); + if (roundMode == 1u) overflowBits = &ir.Select(*positive, ir.Constant(0x7f800000u), ir.Constant(0xff7fffffu)); + else if (roundMode == 2u) overflowBits = &ir.Select(*positive, ir.Constant(0x7f7fffffu), ir.Constant(0xff800000u)); + else if (roundMode == 3u) overflowBits = &ir.BitwiseOr(sign.Value(), ir.Constant(0x7f7fffffu)); + if (denorm32 == 1u || denorm32 == 2u) { + throw std::runtime_error("f32 denormal mode " + std::to_string(denorm32) + " at pc " + std::to_string(inst.programCounter) + " is not implemented"); + } + if (denorm32 == 3u) { + const IrU32 mantissa(ir.BitwiseAnd(bits.Value(), ir.Constant(0x007fffffu))); + const IrU1 isSubnormal(ir.IEqual(exponent.Value(), ir.Constant(0u))); + const IrU32 sig(ir.Select(isSubnormal.Value(), mantissa.Value(), ir.BitwiseOr(mantissa.Value(), ir.Constant(0x00800000u)))); + const IrU32 msb(ir.Emit(IrOpcode::FindUMsb32, IrType::U32, {&sig.Value()})); + const IrU32 normShift(ir.ISub(ir.Constant(23u), msb.Value())); + const IrU32 sigNorm(ir.ShiftLeftLogical(sig.Value(), normShift.Value())); + const IrU32 effExp(ir.Select(isSubnormal.Value(), ir.Constant(1u), exponent.Value())); + const IrU32 expNorm(ir.ISub(ir.IAdd(effExp.Value(), clamped.Value()), normShift.Value())); + const IrU32 normResult(ir.BitwiseOr(sign.Value(), ir.BitwiseOr(ir.ShiftLeftLogical(expNorm.Value(), ir.Constant(23u)), ir.BitwiseAnd(sigNorm.Value(), ir.Constant(0x007fffffu))))); + const IrU32 underShiftRaw(ir.ISub(ir.Constant(1u), expNorm.Value())); + const IrU32 underShift(ir.Emit(IrOpcode::SMin32, IrType::U32, {&ir.Emit(IrOpcode::SMax32, IrType::U32, {&underShiftRaw.Value(), &ir.Constant(1u)}), &ir.Constant(25u)})); + const IrU32 unrounded(ir.ShiftRightLogical(sigNorm.Value(), underShift.Value())); + const IrU32 half(ir.ShiftLeftLogical(ir.Constant(1u), ir.ISub(underShift.Value(), ir.Constant(1u)))); + const IrU32 remMask(ir.BitwiseOr(ir.ISub(half.Value(), ir.Constant(1u)), half.Value())); + const IrU32 rem(ir.BitwiseAnd(sigNorm.Value(), remMask.Value())); + const IrU1 remGreater(ir.UGreaterThan(rem.Value(), half.Value())); + const IrU1 remEqual(ir.IEqual(rem.Value(), half.Value())); + const IrU1 lsbOdd(ir.INotEqual(ir.BitwiseAnd(unrounded.Value(), ir.Constant(1u)), ir.Constant(0u))); + IrValue* roundUp = &ir.LogicalOr(remGreater.Value(), ir.LogicalAnd(remEqual.Value(), lsbOdd.Value())); + if (roundMode == 1u) roundUp = &ir.LogicalAnd(ir.INotEqual(rem.Value(), ir.Constant(0u)), *positive); + else if (roundMode == 2u) roundUp = &ir.LogicalAnd(ir.INotEqual(rem.Value(), ir.Constant(0u)), ir.LogicalNot(*positive)); + const IrU32 magnitude(roundMode == 3u ? unrounded : IrU32(ir.Select(*roundUp, ir.IAdd(unrounded.Value(), ir.Constant(1u)), unrounded.Value()))); + const IrU32 subResult(ir.BitwiseOr(sign.Value(), magnitude.Value())); + IrU32 result(ir.Select(ir.Emit(IrOpcode::SGreaterThanEqual32, IrType::U1, {&expNorm.Value(), &ir.Constant(255u)}), *overflowBits, normResult.Value())); + result = IrU32(ir.Select(ir.Emit(IrOpcode::SLessThanEqual32, IrType::U1, {&expNorm.Value(), &ir.Constant(0u)}), subResult.Value(), result.Value())); + const IrU1 zero(ir.IEqual(ir.BitwiseAnd(bits.Value(), ir.Constant(0x7fffffffu)), ir.Constant(0u))); + result = IrU32(ir.Select(zero.Value(), bits.Value(), result.Value())); + const IrU1 nan(ir.UGreaterThan(ir.BitwiseAnd(bits.Value(), ir.Constant(0x7fffffffu)), ir.Constant(0x7f800000u))); + result = IrU32(ir.Select(ir.IEqual(exponent.Value(), ir.Constant(0xffu)), ir.Select(nan.Value(), quietNan32(bits).Value(), bits.Value()), result.Value())); + writeOperand(inst.destination, &ir.BitCastF32(result.Value())); + return true; + } const IrU32 scaled(ir.IAdd(exponent.Value(), clamped.Value())); const IrU32 normal(ir.BitwiseOr(ir.BitwiseAnd(bits.Value(), ir.Constant(0x807fffffu)), ir.ShiftLeftLogical(scaled.Value(), ir.Constant(23u)))); - const IrU32 overflow(ir.BitwiseOr(sign.Value(), ir.Constant(0x7f800000u))); - IrU32 result(ir.Select(ir.Emit(IrOpcode::SGreaterThanEqual32, IrType::U1, {&scaled.Value(), &ir.Constant(255u)}), overflow.Value(), normal.Value())); + IrU32 result(ir.Select(ir.Emit(IrOpcode::SGreaterThanEqual32, IrType::U1, {&scaled.Value(), &ir.Constant(255u)}), *overflowBits, normal.Value())); result = IrU32(ir.Select(ir.Emit(IrOpcode::SLessThanEqual32, IrType::U1, {&scaled.Value(), &ir.Constant(0u)}), sign.Value(), result.Value())); const IrU1 nan(ir.UGreaterThan(ir.BitwiseAnd(bits.Value(), ir.Constant(0x7fffffffu)), ir.Constant(0x7f800000u))); result = IrU32(ir.Select(ir.IEqual(exponent.Value(), ir.Constant(0xffu)), ir.Select(nan.Value(), quietNan32(bits).Value(), bits.Value()), result.Value())); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 06bdafa65..55c93d7f9 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -419,7 +419,7 @@ Throughout the project, every function at every stage either **does exactly what - [Signaling NaN inputs](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) of the f32 and f16 unary instructions (`v_sqrt/rcp/rsq/log/exp/sin/cos/fract/floor/ceil/trunc/rndne/rcp_iflag/frexp_mant`), `v_ldexp_f32/f16`, `v_frexp_mant_f16`, `v_div_fixup_f32/f16` and `v_mullit_f32` are returned quieted only with IEEE=1 and unchanged with IEEE=0, measured on RDNA2 (gfx1035) in both modes, and so are those of the f64 forms (`v_sqrt/rcp/rsq/fract/floor/ceil/trunc/rndne/frexp_mant/ldexp/div_fixup_f64`) and `v_cvt_f64_f32`, measured on RDNA2 (gfx1036) in both modes; shaders without a known float mode follow IEEE=0. `v_pack_b32_f16` quiets each signaling NaN half by the same rule, measured on RDNA2 (gfx1036) in both modes. A signaling NaN result of `v_cubesc/cubetc/cubema_f32` is quieted by the same rule and keeps the sign the instruction gives it, measured on RDNA2 (gfx1036) in both modes. `v_add/sub/subrev/mul/fma/fmac/fmamk/fmaak/mul_legacy_f32` and `v_fma_mix_f32` with f32 sources choose their NaN result on the bits, measured over 1000 operand triples in both modes: the first NaN source in operand order after its modifiers (the subtrahend negated, as the hardware adds its negation), quieted only with IEEE=1, or the default NaN 0xffc00000 for an invalid operation; in the fma forms an invalid product (inf * 0) gives the default NaN even when src2 is a NaN, and in the legacy forms a zero factor makes the product 0 even when the other factor is a NaN (a NaN src2 still wins). `v_mad_legacy_f32` and `v_mac_legacy_f32` follow the same rule with their separately rounded product, as measured on gfx1010 (#1281, #1290): a NaN source quieted only with IEEE=1, inf - inf giving 0xffc00000, the zero rule hiding a NaN factor and a NaN src2 winning. The f16 and f64 arithmetic, `v_fma_mix_f32` with f16 sources and the separately rounded mad/mac forms (not measurable on RDNA2) follow the same rule on hardware but their NaN bits are left to the driver. `v_cvt_f16_f32`, `v_cvt_f32_f64`, `v_dot2_f32_f16`, `v_fma_mixlo_f16` and `v_fma_mixhi_f16` always quiet and `v_div_scale_f32` never does: the f32 signaling NaN 0x7f812345 as src0 or src2 gives 0x7e09 from mixlo and mixhi at IEEE=0 and IEEE=1 on RDNA2 (gfx1036), where `v_fma_mix_f32` gives 0x7f812345 at IEEE=0 and 0x7fc12345 at IEEE=1. The payload of an f16 NaN source of mixlo and mixhi is left to the driver's f16 to f32 conversion, which NVIDIA replaces. - [Scalar f16 add, sub, subrev and mul denormals](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) follow the `FLOAT_MODE` f16 denormal field at nearest-even: bit 0 preserves inputs and bit 1 preserves the result; otherwise a denormal is flushed to its signed zero. Measured on RDNA2 (gfx1036) with IEEE=0/1 and FP16_OVFL=0/1, over 32 operand pairs in all four denormal modes, including signed zeros, denormals, minimum normals, infinities and NaNs. Multiply detects tininess after rounding to 11 significant bits with an unbounded exponent, before gradual half packing: `0x3bff * 0x0400` flushes, while `0x3c01 * 0x03ff` rounds to `0x0400` and is kept. Directed rounding and ternary arithmetic keep their previous denormal lowering. Shaders without a known float mode keep the previous lowering. Scalar `v_sub_f16`/`v_subrev_f16` share `float16Binary` with add/mul and were measured on gfx1036 over the same operand set as add (1156 non-NaN-input rows per mode); NaN result bits still depend on the driver as recorded above. - [Packed f16 NaN results](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) (`v_pk_add/mul/fma/min/max_f16`) follow the hardware measured on RDNA2 (gfx1035) at IEEE=0 and IEEE=1 with f16 round-to-nearest-even and f16 denormals kept: with IEEE=1 a signaling NaN input is quieted and min/max return the other operand only for a quiet NaN; with IEEE=0 NaN inputs pass through unquieted and min/max ignore signaling. In `v_pk_fma_f16` an invalid product (inf * 0) gives the default NaN even when src2 is a NaN. A shader without a known float mode is translated as IEEE=0, the mode titles use; a mode with another f16 rounding or that flushes f16 denormals throws. -- [f32 denormals](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follow the shader's `FLOAT_MODE` f32 denormal field (bits 5:4) when it flushes them (0, and shaders without a known float mode, as in the 0xc0 most titles use), measured on RDNA2 (gfx1035) over 1444 rows per instruction at IEEE=0 and IEEE=1. `v_add/sub/subrev/mul/mul_legacy/mullit/fma/fmac/fmamk/fmaak/min/max/min3/max3/med3/fract/floor/ceil/frexp_mant/sin/cos_f32` flush their f32 inputs and their result to a zero of the same sign (`v_mullit_f32` measured on RDNA2 (gfx1036) over 6907 rows per mode at IEEE=0 and IEEE=1, so a positive denormal src2 counts as zero); the f32 compares, `v_cvt_i32/u32/rpi_i32/flr_i32_f32`, `v_frexp_exp_i32_f32`, `v_cvt_f64_f32`, `v_cvt_f16_f32`, `v_cvt_pkrtz_f16_f32`, `v_cvt_pknorm_i16/u16_f32` and `v_cvt_pk_u8_f32` flush their f32 inputs. `v_div_scale_f32` flushes all three inputs to signed zeros, measured on RDNA2 (gfx1036) over 2744 operand triples per mode at IEEE=0 and IEEE=1 with f32 denormals flushed and kept; its scale-down behaviour and host denormal preservation remain incomplete. `v_div_fixup_f32` flushes its src1 and src2 and keeps a denormal src0 and result, measured on RDNA2 (gfx1036) over 6859 rows per mode at IEEE=0 and IEEE=1. Tininess is detected after rounding, measured on RDNA2 (gfx1036) for `v_mul_f32` and `v_mullit_f32` at IEEE=0 and IEEE=1: a product is flushed when its value rounded to 24 bits with an unbounded exponent is below 2^-126, so `0x3f7ffffe * 0x00800001` (2^-126 * (1 - 2^-46)) gives `0x00800000` and `0x3f7fffff * 0x00800000` (2^-126 * (1 - 2^-24)) gives zero. The check rounds in the f32 round mode of `FLOAT_MODE` (bits 1:0), measured in all four modes, while the products themselves are still rounded to nearest even. This applies to `v_mul_f32`, `v_mul_legacy_f32`, `v_mullit_f32`, and to `v_fma_f32`, `v_fmac_f32`, `v_fmamk_f32`, `v_fmaak_f32` and `v_fma_mix_f32` when the addend is zero; with a nonzero addend the fused result is kept, so an exact result just below 2^-126 that rounds to it is not flushed. In the three directed modes the check is computed exactly from the significands. To nearest even (0, the mode of shaders without a known float mode) one factor is multiplied by 2^64, which is exact below 2^64, and the product with the other factor is compared with 2^-62. That multiply is a plain `OpFMul` like the product, and no f32 rounding mode is requested in the SPIR-V, so the check relies on the driver rounding f32 to nearest even by default, as the products do, and a product that rounds up to 2^-126 is the emitted product. `v_fma_mix_f32` follows this rule with mixed f16/f32 sources too, measured on RDNA2 (gfx1036) with f16 in either multiplicand slot, low and high half selection, and an f32 or f16 zero addend: f32 `0x00801002` times f16 `0x3bff` gives `0x00800000` to nearest at IEEE=0 and IEEE=1, while the neighbours `0x00801001` and `0x00801003` give zero and `0x00800001`. The tiny-product boundary was also measured for both signs in all four f32 round modes at IEEE=0; f16 inputs retain their previous conversion and denormal handling. `v_fma_mixlo_f16` and `v_fma_mixhi_f16` flush their f32 sources too and keep a product below 2^-126, measured on RDNA2 (gfx1036). `v_mad_legacy_f32` and `v_mac_legacy_f32` flush their inputs, their product and their result in every mode, see its entry; their product is still checked for tininess on its exact value, so one below 2^-126 gives zero even when it rounds to 2^-126, as the console's separately rounded encoding is not measured at this boundary. Denormal modes 1 and 2 throw for the other instructions. Not covered, and not measurable on RDNA2 (no such opcodes) or not measured: `v_mac_f32`, `v_mad_f32`, `v_madak_f32` and `v_madmk_f32` (rounded separately), `v_dot2_f32_f16` and `v_div_fmas_f32` (on hardware `v_div_fmas_f32` flushes only its result), f32 interpolation and `ds_add_f32` keep the previous lowering; `ds_min/max/cmpst_f32` flush both operands by the field, see the LDS float atomics entry. With f32 denormals kept (3) the results depend on the driver's default denormal behaviour, as no `DenormPreserve` execution mode is requested, and `v_ldexp_f32` flushes in every mode. +- [f32 denormals](../../core/shader/recompiler/Translation/src/TranslationContext/OperandAccess.cpp) follow the shader's `FLOAT_MODE` f32 denormal field (bits 5:4) when it flushes them (0, and shaders without a known float mode, as in the 0xc0 most titles use), measured on RDNA2 (gfx1035) over 1444 rows per instruction at IEEE=0 and IEEE=1. `v_add/sub/subrev/mul/mul_legacy/mullit/fma/fmac/fmamk/fmaak/min/max/min3/max3/med3/fract/floor/ceil/frexp_mant/sin/cos_f32` flush their f32 inputs and their result to a zero of the same sign (`v_mullit_f32` measured on RDNA2 (gfx1036) over 6907 rows per mode at IEEE=0 and IEEE=1, so a positive denormal src2 counts as zero); the f32 compares, `v_cvt_i32/u32/rpi_i32/flr_i32_f32`, `v_frexp_exp_i32_f32`, `v_cvt_f64_f32`, `v_cvt_f16_f32`, `v_cvt_pkrtz_f16_f32`, `v_cvt_pknorm_i16/u16_f32` and `v_cvt_pk_u8_f32` flush their f32 inputs. `v_div_scale_f32` flushes all three inputs to signed zeros, measured on RDNA2 (gfx1036) over 2744 operand triples per mode at IEEE=0 and IEEE=1 with f32 denormals flushed and kept; its scale-down behaviour and host denormal preservation remain incomplete. `v_div_fixup_f32` flushes its src1 and src2 and keeps a denormal src0 and result, measured on RDNA2 (gfx1036) over 6859 rows per mode at IEEE=0 and IEEE=1. Tininess is detected after rounding, measured on RDNA2 (gfx1036) for `v_mul_f32` and `v_mullit_f32` at IEEE=0 and IEEE=1: a product is flushed when its value rounded to 24 bits with an unbounded exponent is below 2^-126, so `0x3f7ffffe * 0x00800001` (2^-126 * (1 - 2^-46)) gives `0x00800000` and `0x3f7fffff * 0x00800000` (2^-126 * (1 - 2^-24)) gives zero. The check rounds in the f32 round mode of `FLOAT_MODE` (bits 1:0), measured in all four modes, while the products themselves are still rounded to nearest even. This applies to `v_mul_f32`, `v_mul_legacy_f32`, `v_mullit_f32`, and to `v_fma_f32`, `v_fmac_f32`, `v_fmamk_f32`, `v_fmaak_f32` and `v_fma_mix_f32` when the addend is zero; with a nonzero addend the fused result is kept, so an exact result just below 2^-126 that rounds to it is not flushed. In the three directed modes the check is computed exactly from the significands. To nearest even (0, the mode of shaders without a known float mode) one factor is multiplied by 2^64, which is exact below 2^64, and the product with the other factor is compared with 2^-62. That multiply is a plain `OpFMul` like the product, and no f32 rounding mode is requested in the SPIR-V, so the check relies on the driver rounding f32 to nearest even by default, as the products do, and a product that rounds up to 2^-126 is the emitted product. `v_fma_mix_f32` follows this rule with mixed f16/f32 sources too, measured on RDNA2 (gfx1036) with f16 in either multiplicand slot, low and high half selection, and an f32 or f16 zero addend: f32 `0x00801002` times f16 `0x3bff` gives `0x00800000` to nearest at IEEE=0 and IEEE=1, while the neighbours `0x00801001` and `0x00801003` give zero and `0x00800001`. The tiny-product boundary was also measured for both signs in all four f32 round modes at IEEE=0; f16 inputs retain their previous conversion and denormal handling. `v_fma_mixlo_f16` and `v_fma_mixhi_f16` flush their f32 sources too and keep a product below 2^-126, measured on RDNA2 (gfx1036). `v_mad_legacy_f32` and `v_mac_legacy_f32` flush their inputs, their product and their result in every mode, see its entry; their product is still checked for tininess on its exact value, so one below 2^-126 gives zero even when it rounds to 2^-126, as the console's separately rounded encoding is not measured at this boundary. Denormal modes 1 and 2 throw for the other instructions. Not covered, and not measurable on RDNA2 (no such opcodes) or not measured: `v_mac_f32`, `v_mad_f32`, `v_madak_f32` and `v_madmk_f32` (rounded separately), `v_dot2_f32_f16` and `v_div_fmas_f32` (on hardware `v_div_fmas_f32` flushes only its result), f32 interpolation and `ds_add_f32` keep the previous lowering; `ds_min/max/cmpst_f32` flush both operands by the field, see the LDS float atomics entry. With f32 denormals kept (3) the results depend on the driver's default denormal behaviour, as no `DenormPreserve` execution mode is requested; `v_ldexp_f32` is the exception: it preserves subnormal inputs and rounds a subnormal result in the f32 round mode of `FLOAT_MODE`, measured on RDNA2 (gfx1036) over 2163 rows in each of the four round modes at IEEE=0 and IEEE=1. An overflow rounds like any other result (infinity, or the largest finite value when the mode rounds that sign toward zero). With f32 denormals flushed it flushes a subnormal input and result, measured over the same rows in each round mode at IEEE=0 only. - The executable file that [relinker](../../core/relinker/elfpatcher/src/windows/WindowsPeWriter.cpp) generates opens the console when launched, which is inconvenient for playability. - [Overlay output bus](../../core/libs/prx/libSceVideoOut/src/VideoOutDriver.cpp) (libSceVideoOut) - flips on a port opened with `SCE_VIDEO_OUT_BUS_TYPE_OVERLAY` complete and signal their events like main-bus flips, but their buffers are not shown: the overlay is not composited over the main output. Presenting them as whole frames made the window alternate between the two buses - [PA_CL_CLIP_CNTL](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) - `DX_LINEAR_ATTR_CLIP_ENA` (bit 24) clear is drawn like bit 24 set, with Vulkan's clipping of `NoPerspective` outputs; what a clear bit changes on the console is undocumented. From 3b21873b355ac0225275b8a8b48f94c57d189140 Mon Sep 17 00:00:00 2001 From: MrMha <114929652+MrMha@users.noreply.github.com> Date: Sun, 11 Oct 2026 10:38:27 +0400 Subject: [PATCH 381/460] test(agc): cover shutdown of unsatisfied memory waits Extend the independently Windows-tested WAIT_REG_MEM candidate 056534a9 with a focused shutdown regression. Keep labels unsatisfied on graphics and two compute queues beyond the configured diagnostic, then require prompt shutdown without executing the commands or pending submissions behind those waits. The production cancellation path already calls CheckFailure/checkStopping and catches ProcessShutdown in the worker; no runtime behavior is changed. This test does not explicitly open a Vulkan device. Validation: Linux/GCC 13.3, C++20, -O2 -Wall -Wextra -Werror object compilation against the production headers; conventions and whitespace pass. CMake configuration, runtime linking and execution, and native Windows execution were not performed in this environment. Runtime verification remains pending. Depends on the persistent-wait candidate for #2832/#2985; maintainer policy approval is pending. AI-assisted: yes. --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 17 ++++ .../libSceAgcDriver/tests/WaitShutdown.cpp | 94 +++++++++++++++++++ 2 files changed, 111 insertions(+) create mode 100644 core/libs/prx/libSceAgcDriver/tests/WaitShutdown.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index b1ec6ae52..d65026bbc 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -301,6 +301,23 @@ target_link_libraries(agc_driver_tests PRIVATE set_target_properties(agc_driver_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_driver_tests) +add_test_executable(agc_driver_wait_shutdown_tests EXCLUDE_FROM_ALL tests/WaitShutdown.cpp) +target_include_directories(agc_driver_wait_shutdown_tests PRIVATE ${LIBS_INCLUDE_DIR}) +target_link_libraries(agc_driver_wait_shutdown_tests PRIVATE libSceAgcDriver libkernel libc) +set_target_properties(agc_driver_wait_shutdown_tests PROPERTIES CXX_EXTENSIONS OFF) +configure_windows_unwind(agc_driver_wait_shutdown_tests) +if(BUILD_TESTING) + set_target_properties(agc_driver_wait_shutdown_tests PROPERTIES EXCLUDE_FROM_ALL OFF) + add_test(NAME agc_driver_wait_shutdown COMMAND agc_driver_wait_shutdown_tests) + set_tests_properties(agc_driver_wait_shutdown PROPERTIES + ENVIRONMENT "ANYPS5_NO_SHADER_CACHE=1;APS5_GPU_WAIT_TIMEOUT_MS=20" + TIMEOUT 10 WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}") + if(WIN32) + set_tests_properties(agc_driver_wait_shutdown PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() +endif() + add_test_executable(agc_shader_memory_tests EXCLUDE_FROM_ALL tests/ShaderMemory.cpp Execution/src/ShaderMemory.cpp diff --git a/core/libs/prx/libSceAgcDriver/tests/WaitShutdown.cpp b/core/libs/prx/libSceAgcDriver/tests/WaitShutdown.cpp new file mode 100644 index 000000000..5bfd23f7f --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/WaitShutdown.cpp @@ -0,0 +1,94 @@ +#include "prx/libSceAgcDriver/Submit/include/Acb.hpp" +#include "prx/libSceAgcDriver/Submit/include/Dcb.hpp" +#include "prx/libc/include/Shutdown.hpp" +#include +#include +#include +#include +#include +#include +#include + +namespace { + +struct alignas(64) LabelState { + volatile std::uint32_t label = 0; + volatile std::uint32_t entered = 0; + volatile std::uint32_t after = 0; + volatile std::uint32_t queued = 0; +}; + +void Require(bool condition, const char* message) { + if (!condition) throw std::runtime_error(message); +} + +std::array WriteData(volatile std::uint32_t* destination) { + const auto address = static_cast(reinterpret_cast(destination)); + return {0xc0033700, 0x00100200, static_cast(address), static_cast(address >> 32u), 1}; +} + +std::vector WaitingCommands(LabelState& state) { + const auto address = static_cast(reinterpret_cast(&state.label)); + const auto entered = WriteData(&state.entered); + const std::array wait{0xc0053c00, 0x13, static_cast(address), static_cast(address >> 32u), 1, 0xffffffffu, 0x19}; + const auto after = WriteData(&state.after); + std::vector words(entered.begin(), entered.end()); + words.insert(words.end(), wait.begin(), wait.end()); + words.insert(words.end(), after.begin(), after.end()); + return words; +} + +void Submit(std::uint32_t queue, const std::vector& words) { + Packet packet{const_cast(words.data()), static_cast(words.size()), 0, {}}; + const auto result = queue == 0 ? sceAgcDriverSubmitDcb(&packet) : sceAgcDriverSubmitAcb(queue, &packet); + Require(result == 0, "shutdown wait submission failed"); +} + +void CheckShutdownCancellation() { + static std::array states{}; + constexpr std::array queues{0, 0x20, 0x28}; + for (std::size_t index = 0; index < queues.size(); ++index) { + Submit(queues[index], WaitingCommands(states[index])); + const auto queued = WriteData(&states[index].queued); + Submit(queues[index], {queued.begin(), queued.end()}); + } + const auto deadline = std::chrono::steady_clock::now() + std::chrono::seconds(5); + for (const auto& state : states) { + while (state.entered != 1) { + Require(std::chrono::steady_clock::now() < deadline, "queue did not reach its unsatisfied wait"); + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + } + } + std::this_thread::sleep_for(std::chrono::milliseconds(100)); + for (const auto& state : states) { + Require(state.label == 0, "unsatisfied label was unexpectedly signalled"); + Require(state.after == 0 && state.queued == 0, "unsatisfied wait completed before shutdown"); + } + const auto started = std::chrono::steady_clock::now(); + LibcRunShutdown_nid_postfix(); + Require(std::chrono::steady_clock::now() - started < std::chrono::seconds(2), "shutdown did not promptly cancel waiting queues"); + for (const auto& state : states) { + Require(state.label == 0, "shutdown signalled an unsatisfied label"); + Require(state.after == 0, "shutdown executed commands after an unsatisfied wait"); + Require(state.queued == 0, "shutdown executed a pending submission behind an unsatisfied wait"); + } + LibcRunShutdown_nid_postfix(); +} + +} + +int main() { + try { + CheckShutdownCancellation(); + std::puts("unsatisfied label waits cancelled cleanly at shutdown"); + return 0; + } catch (const std::exception& error) { + std::fprintf(stderr, "%s\n", error.what()); + try { + LibcRunShutdown_nid_postfix(); + } catch (const std::exception& shutdown) { + std::fprintf(stderr, "shutdown: %s\n", shutdown.what()); + } + return 1; + } +} From ffeb26365474436638f56e2f3351560c686e7595 Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Sun, 11 Oct 2026 15:54:23 +0900 Subject: [PATCH 382/460] feat(libc): implement fabs, trunc and fma Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libc/src/Math.cpp | 3 +++ core/libs/tests/GuestMath.cpp | 20 ++++++++++++++++++++ 2 files changed, 23 insertions(+) diff --git a/core/libs/prx/libc/src/Math.cpp b/core/libs/prx/libc/src/Math.cpp index aa26f615b..dc46f7c1a 100644 --- a/core/libs/prx/libc/src/Math.cpp +++ b/core/libs/prx/libc/src/Math.cpp @@ -48,6 +48,9 @@ double APS5_VABI asin_nid_postfix(double x) { return std::asin(x); } double APS5_VABI acos_nid_postfix(double x) { return std::acos(x); } double APS5_VABI sqrt_nid_postfix(double x) { return std::sqrt(x); } float APS5_VABI sqrtf_nid_postfix(float x) { return std::sqrt(x); } +double APS5_VABI fabs_nid_postfix(double x) { return std::fabs(x); } +double APS5_VABI trunc_nid_postfix(double x) { return std::trunc(x); } +double APS5_VABI fma_nid_postfix(double x, double y, double z) { return std::fma(x, y, z); } long double APS5_VABI acosl_nid_postfix(long double x) { return std::acos(x); } long double APS5_VABI frexpl_nid_postfix(long double x, int* exponent) { diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index 4c4967c46..b02fcbea1 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -25,6 +25,9 @@ long long APS5_VABI atoll_nid_postfix(const char*); int* APS5_VABI __error_nid_postfix(); double APS5_VABI sqrt_nid_postfix(double); float APS5_VABI sqrtf_nid_postfix(float); +double APS5_VABI fabs_nid_postfix(double); +double APS5_VABI trunc_nid_postfix(double); +double APS5_VABI fma_nid_postfix(double, double, double); long double APS5_VABI acosl_nid_postfix(long double); long double APS5_VABI frexpl_nid_postfix(long double, int*); struct LibcFloatConstant { std::uint32_t bits[4]; }; @@ -85,6 +88,22 @@ static void CheckSquareRoot() { Require(std::isnan(sqrt_nid_postfix(-1.)) && std::isnan(sqrtf_nid_postfix(-1.f))); } +static void CheckExactOperations() { + const double nan = std::numeric_limits::quiet_NaN(); + Require(fabs_nid_postfix(-3.5) == 3.5 && fabs_nid_postfix(-INFINITY) == INFINITY); + Require(std::bit_cast(fabs_nid_postfix(-0.)) == 0); + Require(std::isnan(fabs_nid_postfix(-nan)) && !std::signbit(fabs_nid_postfix(-nan))); + Require(trunc_nid_postfix(2.7) == 2. && trunc_nid_postfix(-2.7) == -2.); + Require(std::bit_cast(trunc_nid_postfix(-0.5)) == 0x8000000000000000ull); + Require(trunc_nid_postfix(4503599627370497.) == 4503599627370497. && trunc_nid_postfix(-1e300) == -1e300); + Require(trunc_nid_postfix(-INFINITY) == -INFINITY && std::isnan(trunc_nid_postfix(nan))); + Require(fma_nid_postfix(2., 3., 4.) == 10.); + volatile double tenth = 0.1; + Require(fma_nid_postfix(tenth, 10., -1.) == 0x1p-54); + Require(fma_nid_postfix(0x1p1023, 2., -0x1p1023) == 0x1p1023); + Require(std::isnan(fma_nid_postfix(INFINITY, 0., 1.))); +} + static bool ExtendedBits(long double value, std::uint64_t mantissa, std::uint16_t signExponent) { unsigned char bytes[sizeof(long double)]{}; std::memcpy(bytes, &value, sizeof(value)); @@ -350,6 +369,7 @@ int main() { Require(exp2_nid_postfix(-3.) == 0.125); Require(ldexp_nid_postfix(0.75, 4) == 12.); CheckSquareRoot(); + CheckExactOperations(); CheckLongDouble(); Require(scalbn_nid_postfix(0.75, -2) == 0.1875); Require(scalbnf_nid_postfix(0.75f, 4) == 12.f); From 024cb1fedaac34a8d9ee86f4cef78f75895b7fdf Mon Sep 17 00:00:00 2001 From: raghav314 Date: Sat, 10 Oct 2026 11:13:08 +0530 Subject: [PATCH 383/460] perf(agc): cache resources whose only completion work is the fault check A draw or dispatch whose shaders have guarded descriptor slots (or a rect list) but no BDA table gets a fault buffer, and with it NeedsCompletion(), which kept its resources out of the resource cache, and NeverReusable skipped the cache lookup for it. In PPSA36082 nearly every draw in the hub is such a build, so the cache served none of them. Such an object has no table to revalidate, no stores through BDA and no copied writes, so it is now reusable. Each use of a reused object checks the fault record when its batch completes (CheckFaults) instead of running the once-only write-back; draw recipes over such a template register the same check, and dispatch recipes still skip templates with completion work. NeverReusable now refuses what noteReusable refuses: a BDA table, or a fault buffer with APS5_LOOP_GUARD set (its fault resets the buffer). Co-Authored-By: Claude Opus 5.5 --- .../Execution/src/VulkanDevice.cpp | 7 +++++-- .../Graphics/include/ShaderResources.hpp | 2 ++ .../prx/libSceAgcDriver/Graphics/src/Draw.cpp | 6 ++++-- .../Graphics/src/ShaderResources.cpp | 20 ++++++++++++++++--- .../libSceAgcDriver/tests/BdaResources.cpp | 16 ++++++++++++++- 5 files changed, 43 insertions(+), 8 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index 5f2e43115..660622656 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -3585,7 +3585,10 @@ void VulkanDevice::recordDispatch(RecordedDispatch& record) { recordStep(PhaseRecordMarks); // Only copied written buffers (and BDA fault checks) need work once the GPU is done; without them // the batch can signal its labels from the GPU. - if (resources.NeedsCompletion()) { + if (resources.NeedsCompletion() && resources.Reusable()) { + auto kept = record.resources; + recorder.OnComplete([kept] { kept->CheckFaults(); }); + } else if (resources.NeedsCompletion()) { // Listed for DispatchIndirect until the write-back ran (a non-reusable object, so once). auto writers = state->copiedWriters; auto kept = record.resources; @@ -3873,7 +3876,7 @@ VulkanDevice::IndirectOutcome VulkanDevice::dispatch(const ShaderRecompiler::Rec // object the resource cache serves under this content key (reusable: no lease, no copied // writes, every direct region import- or mirror-served), so a hit's proof is the template's // Revalidate and nothing needs completion work. - if (recipeOut != nullptr && DispatchRecipes() && cacheable && resources->Reusable() && objects != nullptr && !contentKey.empty()) { + if (recipeOut != nullptr && DispatchRecipes() && cacheable && resources->Reusable() && !resources->NeedsCompletion() && objects != nullptr && !contentKey.empty()) { auto recipe = std::make_shared(); recipe->device = state->device; recipe->templateRef = resources; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp index 28370a397..737327b9b 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp @@ -188,6 +188,8 @@ public: bool Reusable() const { return reusable; } static bool NeverReusable(std::span shaders); const std::vector>& SampledTextures() const { return textures; } + bool ChecksFaults() const { return bda != nullptr; } + void CheckFaults() const; // `shaders` are the stages the object was built from, in build order (a recorded draw's vertex // and fragment stages, or one compute stage): their bindings are walked like the build did. // How a Revalidate proved (or refused) the object, for the [recipe] line: the proof path taken diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp index 344261741..77d53b380 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp @@ -1498,7 +1498,8 @@ void keepRecordedDraw(Recorder& recorder, const std::shared_ptr }); writers->push_back(resources); } else if (completion) { - recorder.OnComplete([resources] { resources->WriteBackBuffers(); }); + if (resources->Reusable()) recorder.OnComplete([resources] { resources->CheckFaults(); }); + else recorder.OnComplete([resources] { resources->WriteBackBuffers(); }); } for (const auto& resident : residents) { if (resident != nullptr) resident->MarkDirty(); @@ -2421,7 +2422,7 @@ DrawRecipeOutcome DrawWithRecipe(const Context& context, const State& state, con SharedResourceCache().Touch(recipe.key); countCache(&DrawProfile::cacheHits); timer.phase(PhaseLookup); - outcome.completion = false; + outcome.completion = resources->ChecksFaults(); outcome.recorded = true; // The pipeline is the recipe's while the pipeline store still holds it (its device's teardown // or its eviction bound drops it: a miss). The framebuffer is the recipe's while the pipeline's @@ -2435,6 +2436,7 @@ DrawRecipeOutcome DrawWithRecipe(const Context& context, const State& state, con const auto recordStart = profile ? std::chrono::steady_clock::now() : std::chrono::steady_clock::time_point{}; RecordedDraw record; record.recorder = recorder; + record.completion = outcome.completion; record.resources = std::move(resources); record.pipeline = std::move(pipeline); record.framebuffer = std::move(framebuffer); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index 9ac9ff9bd..0fbb3ef2a 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -1563,12 +1563,21 @@ std::atomic resourceCacheTouches{0}; } +namespace { + +bool LoopGuardEnabled() { + static const bool enabled = std::getenv("APS5_LOOP_GUARD") != nullptr; + return enabled; +} + +} + void ShaderResources::noteReusable() { // Taken whether or not the object turns out reusable: the lookups' records are this build's. captureValidation(); reusable = false; directRegions.clear(); - if (NeedsCompletion() || HoldsLease()) return; + if (guestMemory.HasCopiedWrites() || HoldsLease() || usesBda || (bda != nullptr && LoopGuardEnabled())) return; if (TemplateDataRefresh() && std::any_of(allocations.begin(), allocations.end(), [](const Allocation& allocation) { return allocation.buffer != nullptr && !allocation.guest && allocation.size > MaxRefreshBytes; })) return; const auto regions = guestMemory.DirectRegions(); if (!regions.has_value()) return; @@ -1586,8 +1595,9 @@ void ShaderResources::noteReusable() { } bool ShaderResources::NeverReusable(std::span shaders) { - return std::any_of(shaders.begin(), shaders.end(), [](const CompiledShader& shader) { - return shader.program != nullptr && std::any_of(shader.program->bindings.begin(), shader.program->bindings.end(), [](const ShaderRecompiler::DescriptorBinding& binding) { return binding.role == ShaderRecompiler::DescriptorRole::BdaPagetable || binding.role == ShaderRecompiler::DescriptorRole::FaultBuffer; }); + const bool loopGuard = LoopGuardEnabled(); + return std::any_of(shaders.begin(), shaders.end(), [loopGuard](const CompiledShader& shader) { + return shader.program != nullptr && std::any_of(shader.program->bindings.begin(), shader.program->bindings.end(), [loopGuard](const ShaderRecompiler::DescriptorBinding& binding) { return binding.role == ShaderRecompiler::DescriptorRole::BdaPagetable || (loopGuard && binding.role == ShaderRecompiler::DescriptorRole::FaultBuffer); }); }); } @@ -3446,6 +3456,10 @@ void ShaderResources::MarkGpuWrites(Recorder& recorder) { AgcDriver::ProfilePrint_nid_no_patch("[buffers] descriptor elements bound: %llu total, %llu written, %llu read-only; %llu pending-write notes skipped\n", static_cast(elements), static_cast(elements - readOnly), static_cast(readOnly), static_cast(counters.notesSkipped.load())); } +void ShaderResources::CheckFaults() const { + if (bda) bda->CheckFault(); +} + void ShaderResources::WriteBackBuffers() { if (bda) bda->CheckFault(); if (SkipWriteBack()) return; diff --git a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp index 932eb95ac..43db11be7 100644 --- a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp @@ -697,7 +697,8 @@ void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { const std::array stages{{{ShaderRecompiler::ShaderStage::TessellationControl, &control, 0}}}; const std::array unusedSnapshots{{{0, source}}}; ShaderResources resources(disabled, stages, ColorTarget{}, 0, 0, 0, unusedSnapshots); - Require(ShaderResources::NeverReusable(stages) && !resources.Reusable(), "a fault-buffer build is reusable"); + Require(!ShaderResources::NeverReusable(stages), "a fault-buffer build is refused the resource cache"); + Require(!resources.Reusable(), "a fault-buffer build with copied writes is reusable"); Require(resources.NeedsCompletion() && !resources.UsesAddressTable(), "a fault-buffer build reaches guest memory through the BDA table"); const auto fault = access.bytes(access.descriptor(5).buffer); for (const auto byte : fault) Require(byte == std::byte{}, "rect-list fault buffer was not initialized"); @@ -710,6 +711,19 @@ void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { resources.WriteBack(); Require(guest[1] == changed, "rect-list fault-only path lost guest buffer writes"); } + { + ShaderRecompiler::RecompileResult control; + control.bdaAbiVersion = ShaderRecompiler::BdaAbi::Version; + control.bindings = {binding(Role::FaultBuffer, 5)}; + const std::array stages{{{ShaderRecompiler::ShaderStage::TessellationControl, &control, 0}}}; + ShaderResources resources(disabled, stages, ColorTarget{}, 0, 0, 0, {}); + Require(!ShaderResources::NeverReusable(stages) && resources.Reusable() && resources.ChecksFaults(), "a fault-only build is not reusable"); + resources.CheckFaults(); + const auto fault = access.bytes(access.descriptor(5).buffer); + const ShaderRecompiler::BdaAbi::Fault report{ShaderRecompiler::BdaAbi::FaultState::Ready, ShaderRecompiler::BdaAbi::FaultReason::InvalidRectangle, 0, 0, 0, 0, 0}; + std::memcpy(fault.data(), &report, sizeof(report)); + reject([&] { resources.CheckFaults(); }, "rect-list requires"); + } { ShaderResources resources(context, compiled, snapshots); Require(ShaderResources::NeverReusable(std::span(&compiled, 1)) && !resources.Reusable(), "an address-based build is reusable"); From e864e687b389d3e98b910af7f76b378f2739c881 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 23:21:41 +0000 Subject: [PATCH 384/460] fix(libkernel): saturate oversized select timeouts --- .../Socket/include/SocketSelectDeadline.hpp | 18 ++++++++++++++++++ core/libs/prx/libkernel/Socket/src/Socket.cpp | 6 ++++-- core/libs/tests/GuestSceNet.cpp | 8 ++++++++ 3 files changed, 30 insertions(+), 2 deletions(-) create mode 100644 core/libs/prx/libkernel/Socket/include/SocketSelectDeadline.hpp diff --git a/core/libs/prx/libkernel/Socket/include/SocketSelectDeadline.hpp b/core/libs/prx/libkernel/Socket/include/SocketSelectDeadline.hpp new file mode 100644 index 000000000..c643de494 --- /dev/null +++ b/core/libs/prx/libkernel/Socket/include/SocketSelectDeadline.hpp @@ -0,0 +1,18 @@ +#pragma once +#include +#include + +namespace KernelSocketSelect { + +inline std::chrono::steady_clock::time_point DeadlineAfter(std::chrono::steady_clock::time_point start, + std::int64_t seconds, std::int64_t microseconds) { + using Clock = std::chrono::steady_clock; + constexpr std::int64_t MicrosecondsPerSecond = 1000000; + const auto remaining = std::chrono::duration_cast(Clock::time_point::max() - start).count(); + if (seconds > remaining / MicrosecondsPerSecond) return Clock::time_point::max(); + const auto requested = seconds * MicrosecondsPerSecond; + if (microseconds > remaining - requested) return Clock::time_point::max(); + return start + std::chrono::duration_cast(std::chrono::microseconds(requested + microseconds)); +} + +} diff --git a/core/libs/prx/libkernel/Socket/src/Socket.cpp b/core/libs/prx/libkernel/Socket/src/Socket.cpp index d2f176af1..95da4bf78 100644 --- a/core/libs/prx/libkernel/Socket/src/Socket.cpp +++ b/core/libs/prx/libkernel/Socket/src/Socket.cpp @@ -3,6 +3,7 @@ #include "SceTypes.hpp" #include "prx/libc/include/General.hpp" #include "prx/libkernel/Socket/include/SocketPoll.hpp" +#include "prx/libkernel/Socket/include/SocketSelectDeadline.hpp" #include #include @@ -67,12 +68,13 @@ int APS5_VABI select_nid_postfix(int nfds, void* readfds, void* writefds, void* } const auto start = std::chrono::steady_clock::now(); const auto deadline = limit == nullptr ? std::chrono::steady_clock::time_point::max() - : start + std::chrono::seconds(limit->seconds) + std::chrono::microseconds(limit->microseconds); + : KernelSocketSelect::DeadlineAfter(start, limit->seconds, limit->microseconds); for (;;) { int waitMilliseconds = 100; if (limit != nullptr) { const auto remaining = std::chrono::duration_cast(deadline - std::chrono::steady_clock::now()).count(); - waitMilliseconds = static_cast(std::clamp((remaining + 999) / 1000, 0, 100)); + const auto roundedMilliseconds = remaining > 0 ? remaining / 1000 + (remaining % 1000 != 0) : 0; + waitMilliseconds = static_cast(std::clamp(roundedMilliseconds, 0, 100)); } if (!entries.empty()) { const auto poller = g_socketPoller.load(); diff --git a/core/libs/tests/GuestSceNet.cpp b/core/libs/tests/GuestSceNet.cpp index 57d88cfc7..c4b8c8004 100644 --- a/core/libs/tests/GuestSceNet.cpp +++ b/core/libs/tests/GuestSceNet.cpp @@ -9,6 +9,7 @@ #include #endif #include "prx/libc/include/general/VabiMacros.hpp" +#include "prx/libkernel/Socket/include/SocketSelectDeadline.hpp" #include "SceTypes.hpp" #include #include @@ -298,6 +299,13 @@ static void CheckUnspecifiedIpv6() { } int main() { + const auto epoch = std::chrono::steady_clock::time_point{}; + Require(KernelSocketSelect::DeadlineAfter(epoch, 2, 250000) == epoch + std::chrono::milliseconds(2250)); + Require(KernelSocketSelect::DeadlineAfter(epoch, INT64_MAX, 0) == std::chrono::steady_clock::time_point::max()); + Require(KernelSocketSelect::DeadlineAfter(epoch, 10000000000LL, 0) == std::chrono::steady_clock::time_point::max()); + const auto almostMax = std::chrono::steady_clock::time_point::max() - std::chrono::microseconds(1); + Require(KernelSocketSelect::DeadlineAfter(almostMax, 0, 1) == std::chrono::steady_clock::time_point::max()); + Require(KernelSocketSelect::DeadlineAfter(almostMax, 0, 2) == std::chrono::steady_clock::time_point::max()); for (int i = 0; i < 16; ++i) { Require(in6addr_any_nid_postfix[i] == 0); Require(in6addr_loopback_nid_postfix[i] == (i == 15 ? 1 : 0)); From c6c9e51439b70ca4d838da703abe65f726bf0bb4 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sun, 11 Oct 2026 03:35:41 +0000 Subject: [PATCH 385/460] docs(libkernel): document oversized select timeout --- docs/dev/TechnicalDebt.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 96afffd2e..0d9df26ab 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -526,6 +526,7 @@ Throughout the project, every function at every stage either **does exactly what - [kill](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) follows FreeBSD's `kern_kill` and `killpg1` for the only guest process: `EINVAL` for signals outside 0 to 128, signal 0 only checks that the target exists, and its own pid, 0 and minus its own pid (the guest is taken to lead its own process group) signal it through `raise`, so they share `raise`'s limits above. Any other pid, and -1, which skips the caller, report `ESRCH`, so `EPERM` is never reported - [sceKernelFcntl](../../core/libs/prx/libkernel/File/src/Open.cpp) (libkernel) handles socket descriptors through the POSIX `fcntl` export (`F_GETFL`, `F_SETFL` with `O_NONBLOCK`) and returns its guest errno as `SCE_KERNEL_ERROR_*`, the form the other `sceKernel` file exports use; the command semantics follow FreeBSD's [fcntl](https://github.com/freebsd/freebsd-src/blob/stable/11/sys/kern/kern_descrip.c), and a closed descriptor returns `SCE_KERNEL_ERROR_EBADF`. File descriptors throw: their open flags are not tracked, and `F_DUPFD`, `F_GETFD` and `F_SETFD` are unsupported on all descriptors. The return values and error codes are unverified on a console - [pthread_kill](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) returns the error number like FreeBSD's `pthread_kill`: `EINVAL` for signals outside 0 to 128 and `ESRCH` for a null thread. Signal 0 reports any other thread handle as existing, without checking that the thread is still alive. A signal to the calling thread goes through `raise`, so it shares `raise`'s limits; a signal to another thread throws, since guest signals are not delivered to a chosen thread +- [select](../../core/libs/prx/libkernel/Socket/src/Socket.cpp) (libkernel) validates `tv_usec` but saturates a valid timeout beyond the remaining `steady_clock` range at `time_point::max()`, so the wait can continue effectively indefinitely. The console's behavior for a valid timeout outside the host clock's range is unknown. - [AJM](../../core/libs/prx/libSceAjm.native/src/Ajm.cpp), [APR](../../core/libs/prx/libkernel/Apr/src/Apr.cpp) and [NGS2](../../core/libs/prx/libSceNgs2.native/src/System.cpp) state is allocated with `new` and never freed on purpose: title threads still call into these libraries while `std::exit` runs static destructors, so ordinary statics would be destroyed under them. Keep them leaked; `guest_ajm_teardown` covers the AJM case on Unix. - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields From e720d3cd6132a50d7da0f18c9d8dfe9981644835 Mon Sep 17 00:00:00 2001 From: DariusHQ Date: Sat, 10 Oct 2026 14:30:11 +0300 Subject: [PATCH 386/460] fix(relinker): skip zero-return guest libc exports for internal libc imports Since #1454, a libSceLibcInternal import that no native library exports on Windows falls back to the title's own libc.prx. The loader shim that ps5-native-app-boilerplate builds into homebrew titles exports 1874 functions at three addresses whose code is `xor eax, eax; ret` or `pxor xmm0, xmm0; xor eax, eax; ret`. Such an import then binds to a function that returns 0, and the title starts instead of stopping with `unresolved ELF import`. The Windows guest module writer records the exported functions whose code is one of these two sequences, and the entry stub leaves the guest libc.prx out of the search for those names. A guest export with code of its own still provides a missing libSceLibcInternal function, and native libraries keep their priority. --- .../domain/include/domain/GuestRuntime.hpp | 2 ++ .../src/windows/WindowsEntryStubBuilder.cpp | 2 +- .../src/windows/WindowsGuestModuleWriter.cpp | 13 ++++++++++ .../tests/test_windows_import_modules.py | 25 +++++++++++-------- 4 files changed, 31 insertions(+), 11 deletions(-) diff --git a/core/relinker/domain/include/domain/GuestRuntime.hpp b/core/relinker/domain/include/domain/GuestRuntime.hpp index 6ce00f5a1..45f720ae8 100644 --- a/core/relinker/domain/include/domain/GuestRuntime.hpp +++ b/core/relinker/domain/include/domain/GuestRuntime.hpp @@ -2,6 +2,7 @@ #define DOMAIN_GUESTRUNTIME_HPP #include +#include #include #include @@ -25,6 +26,7 @@ struct GuestRuntime { bool UsePlatformTlsResolver = true; std::vector Names; std::vector UnmatchedImportModules; + std::set ZeroReturnExports; }; } diff --git a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp index 6406e8e15..a07231184 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp @@ -149,7 +149,7 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con if (import.Library == "libSceLibcInternal.prx") { for (std::size_t index = 0; index < guestModules.size(); ++index) { const auto& names = guestModules[index].Names; - if (std::find(names.begin(), names.end(), "libc.prx") != names.end()) + if (std::find(names.begin(), names.end(), "libc.prx") != names.end() && !guestModules[index].ZeroReturnExports.contains(import.Name)) order.push_back(CheckedRva(index)); } } diff --git a/core/relinker/elfpatcher/src/windows/WindowsGuestModuleWriter.cpp b/core/relinker/elfpatcher/src/windows/WindowsGuestModuleWriter.cpp index d3a913de2..107c446c6 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsGuestModuleWriter.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsGuestModuleWriter.cpp @@ -107,6 +107,18 @@ std::vector GuestModuleWriter::WriteWindows(const Relinker::GuestI sections.push_back({".ehmeta", nextRva, SectionRead | 0x40u, std::move(metadata)}); nextRva = AlignRva(nextRva + sections.back().Data.size()); } + const auto returnsZero = [&](const std::uint64_t address) { + for (const auto& header : guest.Headers) { + if (header.Type != 1 || (header.Flags & 1) == 0 || address < header.MappedAddress || address - header.MappedAddress >= header.FileSize) continue; + const auto start = guest.Bytes.begin() + static_cast(header.Offset + (address - header.MappedAddress)); + const auto available = header.FileSize - (address - header.MappedAddress); + const auto matches = [&](const std::vector& code) { + return code.size() <= available && std::equal(code.begin(), code.end(), start); + }; + return matches({0x31, 0xc0, 0xc3}) || matches({0x66, 0x0f, 0xef, 0xc0, 0x31, 0xc0, 0xc3}); + } + return false; + }; std::map exports; PeSection tlsExports{".tlsrefs", nextRva, SectionRead | 0x40u, {}}; for (const auto& symbol : guest.Symbols) { @@ -121,6 +133,7 @@ std::vector GuestModuleWriter::WriteWindows(const Relinker::GuestI relocations.push_back(rva); } else rva = image.GetRva(symbol.Value, std::max(symbol.Size, 1)); if (!exports.emplace(symbol.Name, rva).second) throw Domain::RelinkerException("Duplicate guest export: " + symbol.Name); + if ((symbol.Info & 15) == 2 && returnsZero(symbol.Value)) runtime.ZeroReturnExports.insert(symbol.Name); } if (!tlsExports.Data.empty()) { nextRva = AlignRva(nextRva + tlsExports.Data.size()); diff --git a/core/relinker/relinker/tests/test_windows_import_modules.py b/core/relinker/relinker/tests/test_windows_import_modules.py index 0eeaabd36..c3542f4b1 100644 --- a/core/relinker/relinker/tests/test_windows_import_modules.py +++ b/core/relinker/relinker/tests/test_windows_import_modules.py @@ -17,10 +17,10 @@ def dynamic(image, offset, phoff, tags): struct.pack_into(' Date: Sat, 10 Oct 2026 12:55:09 +0200 Subject: [PATCH 387/460] fix(content): export the content calls PPSA06465 imports Co-Authored-By: Claude Sonnet 5.5 --- core/libs/prx/libSceContentDelete/Export.cpp | 5 +++ .../prx/libSceContentExport/CMakeLists.txt | 10 +++++ core/libs/prx/libSceContentExport/Export.cpp | 15 +++++++ core/libs/prx/libSceContentSearch/Export.cpp | 20 +++++++++ core/libs/tests/GuestContentUnimplemented.cpp | 45 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 6 files changed, 96 insertions(+) create mode 100644 core/libs/tests/GuestContentUnimplemented.cpp diff --git a/core/libs/prx/libSceContentDelete/Export.cpp b/core/libs/prx/libSceContentDelete/Export.cpp index 16dd322fb..4c80330e3 100644 --- a/core/libs/prx/libSceContentDelete/Export.cpp +++ b/core/libs/prx/libSceContentDelete/Export.cpp @@ -24,6 +24,11 @@ int APS5_VABI sceContentDeleteByPath(void) { return 0; } +int APS5_VABI sceContentDeleteById(void) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + int APS5_VABI sceContentDeleteTerminate(void) { bool expected = true; if (!g_initialized.compare_exchange_strong(expected, false)) throw std::logic_error(std::string(__func__) + ": not initialized"); diff --git a/core/libs/prx/libSceContentExport/CMakeLists.txt b/core/libs/prx/libSceContentExport/CMakeLists.txt index 7c401ea5f..7efbba37e 100644 --- a/core/libs/prx/libSceContentExport/CMakeLists.txt +++ b/core/libs/prx/libSceContentExport/CMakeLists.txt @@ -12,4 +12,14 @@ if(BUILD_TESTING) set_tests_properties(guest_content_export PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + + add_test_executable(guest_content_unimplemented_tests ${LIBS_INCLUDE_DIR}/tests/GuestContentUnimplemented.cpp) + target_include_directories(guest_content_unimplemented_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_content_unimplemented_tests PRIVATE libSceContentExport libSceContentSearch libSceContentDelete libc) + configure_windows_unwind(guest_content_unimplemented_tests) + add_test(NAME guest_content_unimplemented COMMAND guest_content_unimplemented_tests) + if(WIN32) + set_tests_properties(guest_content_unimplemented PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() endif() diff --git a/core/libs/prx/libSceContentExport/Export.cpp b/core/libs/prx/libSceContentExport/Export.cpp index 1a7fde306..a626f6a42 100644 --- a/core/libs/prx/libSceContentExport/Export.cpp +++ b/core/libs/prx/libSceContentExport/Export.cpp @@ -46,6 +46,21 @@ int APS5_VABI sceContentExportStart(void) { return 0; } +int APS5_VABI sceContentExportGetProgress(void) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceContentExportCancel(void) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceContentExportFromDataWithThumbnail(void) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + int APS5_VABI sceContentExportTerm(void) { bool expected = true; if (!g_initialized.compare_exchange_strong(expected, false)) throw std::logic_error(std::string(__func__) + ": not initialized"); diff --git a/core/libs/prx/libSceContentSearch/Export.cpp b/core/libs/prx/libSceContentSearch/Export.cpp index cfb70402a..7a40aabad 100644 --- a/core/libs/prx/libSceContentSearch/Export.cpp +++ b/core/libs/prx/libSceContentSearch/Export.cpp @@ -44,6 +44,26 @@ int APS5_VABI sceContentSearchSearchContent(void) { return 0; } +int APS5_VABI sceContentSearchGetContentLastUpdateId(void) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceContentSearchOpenMetadataByContentId(void) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceContentSearchGetTotalContentSize(void) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + +int APS5_VABI sceContentSearchGetNumOfContent(void) { + NotImplemented_nid_no_patch(__func__); + return 0; +} + int APS5_VABI sceContentSearchTerm(void) { bool expected = true; if (!g_initialized.compare_exchange_strong(expected, false)) throw std::logic_error(std::string(__func__) + ": not initialized"); diff --git a/core/libs/tests/GuestContentUnimplemented.cpp b/core/libs/tests/GuestContentUnimplemented.cpp new file mode 100644 index 000000000..00a291f52 --- /dev/null +++ b/core/libs/tests/GuestContentUnimplemented.cpp @@ -0,0 +1,45 @@ +#include "prx/libc/include/general/VabiMacros.hpp" + +#include +#include +#include + +extern "C" { +int APS5_VABI sceContentDeleteById(); +int APS5_VABI sceContentExportGetProgress(); +int APS5_VABI sceContentExportCancel(); +int APS5_VABI sceContentExportFromDataWithThumbnail(); +int APS5_VABI sceContentSearchGetContentLastUpdateId(); +int APS5_VABI sceContentSearchOpenMetadataByContentId(); +int APS5_VABI sceContentSearchGetTotalContentSize(); +int APS5_VABI sceContentSearchGetNumOfContent(); +} + +namespace { + +bool ReportsNotImplemented(const char* name, int (APS5_VABI *function)()) { + try { + function(); + } catch (const std::runtime_error& error) { + if (std::string(error.what()) == std::string(name) + " not implemented") return true; + std::fprintf(stderr, "%s threw: %s\n", name, error.what()); + return false; + } + std::fprintf(stderr, "%s returned instead of reporting that it is not implemented\n", name); + return false; +} + +} + +int main() { + bool correct = true; + correct &= ReportsNotImplemented("sceContentDeleteById", sceContentDeleteById); + correct &= ReportsNotImplemented("sceContentExportGetProgress", sceContentExportGetProgress); + correct &= ReportsNotImplemented("sceContentExportCancel", sceContentExportCancel); + correct &= ReportsNotImplemented("sceContentExportFromDataWithThumbnail", sceContentExportFromDataWithThumbnail); + correct &= ReportsNotImplemented("sceContentSearchGetContentLastUpdateId", sceContentSearchGetContentLastUpdateId); + correct &= ReportsNotImplemented("sceContentSearchOpenMetadataByContentId", sceContentSearchOpenMetadataByContentId); + correct &= ReportsNotImplemented("sceContentSearchGetTotalContentSize", sceContentSearchGetTotalContentSize); + correct &= ReportsNotImplemented("sceContentSearchGetNumOfContent", sceContentSearchGetNumOfContent); + return correct ? 0 : 1; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 727f26c3c..b00ae11ac 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -116,6 +116,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceSaveDataCancel](../../core/libs/prx/libSceSaveData.native/Export.cpp) (libSceSaveData.native) - imported by PPSA23126, which fails to load without it; added without an implementation because its signature is unknown. Every save data call here finishes before it returns, so nothing is ever in flight to cancel, but what the console returns then is unknown - [sceAudioOutExClose, sceAudioOutExConfigureOutput, sceAudioOutExGetMonitorInfo, sceAudioOutExOpen](../../core/libs/prx/libSceAudioOut/src/AudioOut.cpp) (libSceAudioOut) - imported by PPSA99505, which fails to load without them; added without an implementation and declared without parameters. PPSA99505 calls them only for its HDMI bitstream output, an option that is off by default, to read the formats the TV or receiver lists, switch the HDMI output mode and open a port for IEC 61937 bursts ([Jelly5](https://github.com/02dnot/Jelly5/blob/552b866d89b2a58b27f7f7b0484dc743ce235526/app/src/jelly5_bitstream.cpp#L110-L179)); [EVO-PLAYER-PS5](https://github.com/sainsaji/EVO-PLAYER-PS5/blob/da0c0451333bab82e79fffd2de4718e917f9dcb0/projects/evoplayer/core/src/audio/evo_pt.cpp) also calls `sceAudioOutExOpen`, `sceAudioOutExConfigureOutput` and `sceAudioOutExClose` for HDMI bitstream output. The two declare the arguments of `sceAudioOutExConfigureOutput` differently and neither is confirmed on a console. Audio reaches the host device through SDL as PCM, so there is no encoded HDMI output to switch to - [sceOpusCeltDecCreateEx, sceOpusCeltDecDecode, sceOpusCeltDecDestroy, sceOpusCeltDecGetSize, sceOpusCeltDecInitialize, sceOpusCeltDecTerminate](../../core/libs/prx/libSceOpusCeltDec/Export.cpp) (libSceOpusCeltDec) - imported by PPSA99001, which fails to load without the module; added without an implementation. Their signatures match libSceOpusDec's in [ProsperoRadio](https://github.com/blackbearreloaded/ProsperoRadio/blob/2c4029c3a8f43e9e1dfef98374d94d99b14d745d/src/opus_decoder.cpp). PPSA99001 opens this decoder only as a fallback when libSceOpusDec reports a retryable error for a CELT packet, which [libSceOpusDec](../../core/libs/prx/libSceOpusDec/Export.cpp) here never returns, so the fallback is not reached +- [sceContentDeleteById](../../core/libs/prx/libSceContentDelete/Export.cpp) (libSceContentDelete), [sceContentExportGetProgress, sceContentExportCancel, sceContentExportFromDataWithThumbnail](../../core/libs/prx/libSceContentExport/Export.cpp) (libSceContentExport) and [sceContentSearchGetContentLastUpdateId, sceContentSearchOpenMetadataByContentId, sceContentSearchGetTotalContentSize, sceContentSearchGetNumOfContent](../../core/libs/prx/libSceContentSearch/Export.cpp) (libSceContentSearch) - imported by PPSA06465, which fails to load without them; added without an implementation because their signatures are unknown - [sce::Json::Parser::parse](../../core/libs/prx/libSceJson2/Export.cpp) (libSceJson2) - a number that overflows a double (e.g. `1e309`) throws: whether the console rejects the document or stores an infinite real is unknown - [sceUserServiceGetNpAccountId](../../core/libs/prx/libSceUserService/Export.cpp) (libSceUserService) - the account id for a user with no linked PSN account is assumed to be 0 with success; libSceNpManager's `sceNpGetAccountIdA` answers the same question with `SCE_NP_ERROR_SIGNED_OUT` - [sceUserServiceGetUserColor](../../core/libs/prx/libSceUserService/Export.cpp) (libSceUserService) - the default profile colour is assumed to be blue (0) From c0348a02bfd14627dc5b57dacb717a431a7e8be4 Mon Sep 17 00:00:00 2001 From: raghav314 Date: Sun, 11 Oct 2026 05:07:36 +0530 Subject: [PATCH 388/460] fix(agc): note BDA-written pages at full address width and read back only the claimed ones A shader store through the BDA table records its 4 KiB page in the fault buffer, and the driver marks those pages written after the use. The page number was truncated to 32 bits, so a store above 16 TiB marked a page at another address. And the driver scanned all 4096 slots after every use, even when nothing was stored. Slots now hold a 128 KiB chunk (its number plus one, 32 bits for any address below 512 TiB) and a mask of its 4 KiB pages. A shader that claims a slot appends it to a list, with a count. The driver reads the count, decodes only the claimed slots, marks each run of pages once and clears only those slots. A writable BDA range past 512 TiB is refused when the table is built. Co-Authored-By: Claude Opus 5.5 --- .../Graphics/src/BdaResources.cpp | 20 +++--- .../libSceAgcDriver/tests/BdaExecution.cpp | 69 ++++++++++++++++--- .../tests/execution/FlatStore.cpp | 68 ++++++++++++++++-- core/shader/recompiler/BdaAbi.hpp | 40 ++++++++++- .../SpirvBackend/src/SpirvBdaLookup.cpp | 44 +++++++++--- 5 files changed, 206 insertions(+), 35 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/BdaResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/BdaResources.cpp index 605809a50..0b94d5e0c 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/BdaResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/BdaResources.cpp @@ -124,7 +124,7 @@ BdaResources::BdaResources(const Context& context) { static_assert(std::endian::native == std::endian::little); Require(ShaderRecompiler::BdaAbi::FaultBufferBytes <= context.limits.maxStorageBufferRange, "BDA fault buffer exceeds storage buffer range limit"); fault = std::make_unique(context, ShaderRecompiler::BdaAbi::FaultBufferBytes, VK_BUFFER_USAGE_STORAGE_BUFFER_BIT); - std::memset(fault->Bytes().data(), 0, fault->Bytes().size()); + std::memset(fault->Bytes().data(), 0, ShaderRecompiler::BdaAbi::WrittenListWord * sizeof(std::uint32_t)); } BdaResources::BdaResources(const Context& context, const GuestBufferMemory& memory) : BdaResources(context) { @@ -192,6 +192,7 @@ BdaResources::BdaResources(const Context& context, const GuestBufferMemory& memo ++it; } } + for (const auto& range : ranges) Require((range.permissions & ShaderRecompiler::BdaAbi::Write) == 0 || range.end <= ShaderRecompiler::BdaAbi::WrittenAddressLimit, "a writable BDA range ends past the addresses the written-page record can hold"); table = std::make_shared(context, tableBytes, VK_BUFFER_USAGE_STORAGE_BUFFER_BIT); const ShaderRecompiler::BdaAbi::Header header{ShaderRecompiler::BdaAbi::Version, static_cast(ranges.size()), sizeof(ShaderRecompiler::BdaAbi::Range), 0}; std::memcpy(table->Bytes().data(), &header, sizeof(header)); @@ -276,15 +277,16 @@ namespace AgcDriver::Graphics { void BdaResources::markWrittenPages() const { namespace Abi = ShaderRecompiler::BdaAbi; auto* words = reinterpret_cast(fault->Bytes().data()); - Require(words[Abi::WrittenOverflowWord] == 0, "more than " + std::to_string(Abi::WrittenPageSlots) + " pages stored to through the BDA table in one use are not implemented"); - bool any = false; - for (std::uint32_t slot = 0; slot < Abi::WrittenPageSlots; ++slot) { - const auto page = words[Abi::WrittenSlotsWord + slot]; - if (page == 0) continue; - GuestMemory::MarkWritten(static_cast(page - 1u) << Abi::WrittenPageShift, std::size_t{1} << Abi::WrittenPageShift); - any = true; + Require(words[Abi::WrittenOverflowWord] == 0, "more than " + std::to_string(Abi::WrittenPageSlots) + " chunks of " + std::to_string(Abi::WrittenChunkPages) + " pages stored to through the BDA table in one use are not implemented"); + const auto count = words[Abi::WrittenCountWord]; + if (count == 0) return; + Require(Abi::ForEachWrittenRange(words, [](std::uint64_t address, std::size_t bytes) { GuestMemory::MarkWritten(address, bytes); }), "invalid BDA written-page record"); + for (std::uint32_t claim = 0; claim < count; ++claim) { + const auto slot = words[Abi::WrittenListWord + claim]; + words[Abi::WrittenSlotsWord + 2 * slot] = 0; + words[Abi::WrittenSlotsWord + 2 * slot + 1] = 0; } - if (any) std::memset(words + Abi::WrittenSlotsWord, 0, Abi::WrittenPageSlots * sizeof(std::uint32_t)); + words[Abi::WrittenCountWord] = 0; } } diff --git a/core/libs/prx/libSceAgcDriver/tests/BdaExecution.cpp b/core/libs/prx/libSceAgcDriver/tests/BdaExecution.cpp index 7c4c2adc2..bf7674cc4 100644 --- a/core/libs/prx/libSceAgcDriver/tests/BdaExecution.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/BdaExecution.cpp @@ -90,7 +90,46 @@ private: } +static void RunWrittenRangeDecodeTests() { + namespace Abi = ShaderRecompiler::BdaAbi; + std::vector words(Abi::FaultBufferBytes / sizeof(std::uint32_t)); + std::vector> marked; + const auto decode = [&] { + marked.clear(); + return Abi::ForEachWrittenRange(words.data(), [&](std::uint64_t address, std::size_t bytes) { marked.emplace_back(address, bytes); }); + }; + const auto claim = [&](std::uint32_t index, std::uint32_t slot, std::uint64_t address, std::uint32_t mask) { + words[Abi::WrittenListWord + index] = slot; + words[Abi::WrittenSlotsWord + 2 * slot] = Abi::WrittenChunkKey(address); + words[Abi::WrittenSlotsWord + 2 * slot + 1] = mask; + words[Abi::WrittenCountWord] = std::max(words[Abi::WrittenCountWord], index + 1); + }; + constexpr std::uint64_t page = std::uint64_t{1} << Abi::WrittenPageShift; + Require(decode() && marked.empty(), "an empty written-page record marked pages"); + constexpr std::uint64_t top = 0x7ffffffe0000ULL; + claim(0, 5, top, 0b1011u); + Require(decode() && marked == std::vector>{{top, 2 * page}, {top + 3 * page, page}}, "written pages near the top of the address space were not decoded as written"); + std::fill(words.begin(), words.end(), 0u); + constexpr std::uint64_t high = 0x200000001000ULL; + claim(0, Abi::WrittenPageSlots - 1, high, 1u << ((high >> Abi::WrittenPageShift) & (Abi::WrittenChunkPages - 1u))); + Require(decode() && marked == std::vector>{{high, page}}, "a written page above 16 TiB was decoded at another address"); + std::fill(words.begin(), words.end(), 0u); + constexpr std::uint64_t chunk = 0x3e0000000ULL; + claim(0, 7, chunk, 0xffffffffu); + claim(1, 8, chunk + (std::uint64_t{1} << Abi::WrittenChunkShift), 1u); + claim(2, 9, chunk - (std::uint64_t{1} << Abi::WrittenChunkShift), 0x80000000u); + Require(decode() && marked == std::vector>{{chunk, 32 * page}, {chunk + 32 * page, page}, {chunk - page, page}}, "whole, first-page and last-page chunks were not decoded exactly"); + words[Abi::WrittenListWord + 1] = Abi::WrittenPageSlots; + Require(!decode(), "a claim outside the slots was accepted"); + std::fill(words.begin(), words.end(), 0u); + words[Abi::WrittenCountWord] = 1; + Require(!decode(), "a claim of an empty slot was accepted"); + words[Abi::WrittenCountWord] = Abi::WrittenPageSlots + 1; + Require(!decode(), "more claims than slots were accepted"); +} + static void RunBdaDwordWriteTests(const Context& context) { + RunWrittenRangeDecodeTests(); namespace Abi = ShaderRecompiler::BdaAbi; constexpr std::uint64_t guest = 0x7fff12340f80ULL; constexpr std::size_t firstBytes = 160; @@ -114,20 +153,23 @@ static void RunBdaDwordWriteTests(const Context& context) { } return nullptr; }; - const auto noted = [&](std::uint64_t address) { - const auto page = static_cast(address >> Abi::WrittenPageShift) + 1u; - std::vector slots(Abi::WrittenPageSlots); - std::memcpy(slots.data(), fault.Bytes().data() + Abi::WrittenSlotsWord * sizeof(std::uint32_t), slots.size() * sizeof(std::uint32_t)); - return std::find(slots.begin(), slots.end(), page) != slots.end(); + const auto notedBytes = [&](std::uint64_t first, std::uint64_t last) { + std::uint64_t covered = 0, total = 0; + const auto* faultWords = reinterpret_cast(fault.Bytes().data()); + Require(Abi::ForEachWrittenRange(faultWords, [&](std::uint64_t begin, std::size_t bytes) { + total += bytes; + for (const auto address : {first, last}) covered += address >= begin && address < begin + bytes ? 1u : 0u; + }), "a BDA dword write left an invalid written-page record"); + return std::pair{covered, total}; }; - const auto write = [&](std::uint64_t address, std::uint32_t dwords, std::uint32_t storedBytes, Abi::FaultReason reason) { + const auto write = [&](std::uint64_t address, std::uint32_t dwords, std::uint32_t storedBytes, Abi::FaultReason reason, std::uint32_t groups = 1) { const Abi::Header header{Abi::Version, static_cast(ranges.size()), sizeof(Abi::Range), 0}; std::memcpy(table.Bytes().data(), &header, sizeof(header)); std::memcpy(table.Bytes().data() + sizeof(header), ranges.data(), sizeof(ranges)); std::memset(fault.Bytes().data(), 0, fault.Bytes().size()); for (auto* buffer : {&first, &second, &readOnly}) std::memset(buffer->Bytes().data(), 0xa5, buffer->Bytes().size()); Pipeline pipeline(context, MakeBdaDwordWriteTestShader(address, dwords, values.data()), {&table, &fault, &output}); - pipeline.Run(1); + pipeline.Run(groups); Abi::Fault report{}; std::memcpy(&report, fault.Bytes().data(), sizeof(report)); for (std::uint32_t byte = 0; byte < dwords * 4u; ++byte) { @@ -142,7 +184,17 @@ static void RunBdaDwordWriteTests(const Context& context) { } else { Require(report.state == Abi::FaultState::Ready && report.reason == reason && report.instruction == 0x1234, "a BDA dword write did not publish its fault"); } - if (storedBytes != 0) Require(noted(address) && noted(address + storedBytes - 1u), "a BDA dword write did not note its written pages"); + const auto [covered, total] = notedBytes(address, address + (storedBytes != 0 ? storedBytes - 1u : 0u)); + if (storedBytes == 0) { + Require(total == 0, "a BDA dword write that stored nothing noted pages"); + } else { + const auto pages = ((address + storedBytes - 1u) >> Abi::WrittenPageShift) - (address >> Abi::WrittenPageShift) + 1u; + Require(covered == 2 && total == pages << Abi::WrittenPageShift, "a BDA dword write did not note exactly its written pages"); + const auto chunks = ((address + storedBytes - 1u) >> Abi::WrittenChunkShift) - (address >> Abi::WrittenChunkShift) + 1u; + std::uint32_t claims = 0; + std::memcpy(&claims, fault.Bytes().data() + Abi::WrittenCountWord * sizeof(std::uint32_t), sizeof(claims)); + Require(claims == chunks, "a BDA dword write claimed a written-page slot more than once per chunk"); + } }; write(guest + 16, 4, 16, static_cast(0)); write(guest + 20, 4, 16, static_cast(0)); @@ -154,6 +206,7 @@ static void RunBdaDwordWriteTests(const Context& context) { write(guest + 41, 4, 16, static_cast(0)); write(guest + firstBytes - 8, 4, 16, static_cast(0)); write(guest + firstBytes + 12, 4, 4, Abi::FaultReason::Permission); + write(guest + 0x7c, 2, 8, static_cast(0), 64); } void RunBdaExecutionTests(const Context& context) { diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/FlatStore.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/FlatStore.cpp index 259929ce0..77db354f6 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/FlatStore.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/FlatStore.cpp @@ -1,6 +1,8 @@ +#include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" #include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" #include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" #include "prx/libc/include/GuestAllocations.hpp" +#include "prx/libc/include/GuestWriteWatch.hpp" #include "Recompiler.hpp" #include "VulkanTestDevice.hpp" #ifdef _WIN32 @@ -8,6 +10,8 @@ #define NOMINMAX #endif #include +#else +#include #endif #include #include @@ -27,6 +31,9 @@ constexpr std::uint32_t MaskedThreads = 16; constexpr std::size_t BlockBytes = 65536; constexpr std::uint32_t CheckedBytes = 0x2000; constexpr std::uint8_t Fill = 0xcd; +#ifdef __linux__ +constexpr std::array GuestWindowAddresses{0x80000000000, 0x200000000000}; +#endif alignas(256) constexpr std::array FlatStoreCode{ 0x34020082, 0x34040084, 0x340c0081, 0x340e0083, 0x4a060200, 0x7e080201, 0x4a1806ff, 0x00000204, @@ -39,28 +46,54 @@ alignas(256) constexpr std::array FlatStoreCode{ class GuestBlock { public: - explicit GuestBlock(bool writable) { + explicit GuestBlock(bool writable, bool watched = false, std::uintptr_t at = 0) : bytes(watched ? 2 * BlockBytes : BlockBytes), watched(watched) { #ifdef _WIN32 - block = static_cast(VirtualAlloc(nullptr, BlockBytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); + Require(!watched, "flat store: watched guest blocks are mapped only on Linux"); + block = static_cast(VirtualAlloc(nullptr, bytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); #else - block = static_cast(std::aligned_alloc(BlockBytes, BlockBytes)); + if (watched) { +#ifdef __linux__ + void* raw = mmap(reinterpret_cast(at), bytes, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED_NOREPLACE, -1, 0); + Require(raw == reinterpret_cast(at), "flat store: cannot map the watched guest block at its address"); + block = static_cast(raw); + GuestWriteWatch::GuestWriteWatchRegister_nid_postfix(block, bytes); +#else + Require(false, "flat store: watched guest blocks are mapped only on Linux"); +#endif + } else { + block = static_cast(std::aligned_alloc(BlockBytes, bytes)); + } #endif Require(block != nullptr, "flat store: cannot allocate the guest block"); + Require(!watched || AgcDriver::GuestMemory::Watched(Address(), bytes), "flat store: the guest block is not write-watched"); Clear(); - GuestAllocations::Mutation().Add(block, BlockBytes, true, writable, true); + GuestAllocations::Mutation().Add(block, bytes, true, writable, true); } ~GuestBlock() { GuestAllocations::Mutation().Remove(block); +#ifdef _WIN32 + VirtualFree(block, 0, MEM_RELEASE); +#else + if (watched) { + GuestWriteWatch::GuestWriteWatchUnregister_nid_postfix(block, bytes); + munmap(block, bytes); + } else { + std::free(block); + } +#endif } GuestBlock(const GuestBlock&) = delete; GuestBlock& operator=(const GuestBlock&) = delete; - void Clear() { std::memset(block, Fill, BlockBytes); } + void Clear() { std::memset(block, Fill, bytes); } const std::uint8_t* Data() const { return block; } + std::uint64_t Address() const { return reinterpret_cast(block); } private: + std::size_t bytes; + bool watched; std::uint8_t* block = nullptr; }; @@ -116,6 +149,23 @@ void RunStores(AgcDriver::VulkanDevice& device, GuestBlock& guest, std::uint32_t } } +#ifdef __linux__ +void RunStampedBlocks(AgcDriver::VulkanDevice& device, GuestBlock& guest) { + namespace GuestMemory = AgcDriver::GuestMemory; + const auto base = guest.Address(); + Dispatch(device, 32, guest.Data()); + for (int use = 0; use < 2; ++use) { + guest.Clear(); + const auto collected = GuestMemory::CollectWrites(base, 2 * BlockBytes); + Require(collected != 0, "flat store: the watched guest blocks are not collected"); + Require(GuestMemory::UnchangedSince(base, 2 * BlockBytes, collected), "flat store: the guest blocks changed before the store"); + Dispatch(device, 32, guest.Data()); + Require(!GuestMemory::UnchangedSince(base, BlockBytes, collected), "flat store: the stored block is not stamped as written"); + Require(GuestMemory::UnchangedSince(base + BlockBytes, BlockBytes, collected), "flat store: a block no store reached is stamped as written"); + } +} +#endif + void RunReadOnly(AgcDriver::VulkanDevice& device, const GuestBlock& guest) { Dispatch(device, 32, guest.Data()); for (std::uint32_t offset = 0; offset < CheckedBytes; ++offset) { @@ -134,6 +184,14 @@ int main() { RunStores(*device, writable, 32); RunStores(*device, writable, 64); RunReadOnly(*device, readOnly); +#ifdef __linux__ + if (AgcDriver::GuestMemory::WriteWatched()) { + for (const auto at : GuestWindowAddresses) { + GuestBlock watched(true, true, at); + RunStampedBlocks(*device, watched); + } + } +#endif std::puts("flat store tests passed"); return 0; } catch (const std::exception& error) { diff --git a/core/shader/recompiler/BdaAbi.hpp b/core/shader/recompiler/BdaAbi.hpp index aa31e43e9..f964d7fe9 100644 --- a/core/shader/recompiler/BdaAbi.hpp +++ b/core/shader/recompiler/BdaAbi.hpp @@ -1,6 +1,7 @@ #ifndef CORE_SHADER_RECOMPILER_BDAABI_HPP #define CORE_SHADER_RECOMPILER_BDAABI_HPP +#include #include #include #include @@ -46,12 +47,45 @@ static_assert(std::is_standard_layout_v && std::is_trivially_copyable_v(address >> WrittenChunkShift) + 1u; +} + +template +bool ForEachWrittenRange(const std::uint32_t* words, TMark&& mark) { + const auto count = words[WrittenCountWord]; + if (count > WrittenPageSlots) return false; + for (std::uint32_t claim = 0; claim < count; ++claim) { + const auto slot = words[WrittenListWord + claim]; + if (slot >= WrittenPageSlots || words[WrittenSlotsWord + 2 * slot] == 0) return false; + } + for (std::uint32_t claim = 0; claim < count; ++claim) { + const auto slot = words[WrittenListWord + claim]; + const auto key = words[WrittenSlotsWord + 2 * slot]; + auto mask = words[WrittenSlotsWord + 2 * slot + 1]; + const auto base = static_cast(key - 1u) << WrittenChunkShift; + while (mask != 0) { + const auto first = static_cast(std::countr_zero(mask)); + const auto pages = static_cast(std::countr_one(mask >> first)); + mark(base + (static_cast(first) << WrittenPageShift), static_cast(pages) << WrittenPageShift); + mask = pages + first >= 32u ? 0u : mask & ~((1u << (first + pages)) - 1u); + } + } + return true; +} } diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvBdaLookup.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvBdaLookup.cpp index 8cc0fa6d2..f4290dcf5 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvBdaLookup.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvBdaLookup.cpp @@ -132,10 +132,17 @@ static std::uint32_t DefineBdaNoteWrite(SpirvEmitterState& state) { state.module.AddFunction(spv::OpVariable, TypePointer(state, spv::StorageClassFunction, boolean), done, spv::StorageClassFunction); const auto counter = state.module.AllocateId(); state.module.AddFunction(spv::OpVariable, TypePointer(state, spv::StorageClassFunction, u32), counter, spv::StorageClassFunction); + const auto held = state.module.AllocateId(); + state.module.AddFunction(spv::OpVariable, TypePointer(state, spv::StorageClassFunction, u32), held, spv::StorageClassFunction); state.module.AddFunction(spv::OpStore, done, ConstantBool(state, false)); state.module.AddFunction(spv::OpStore, counter, constant(0u)); - const auto page = binary(spv::OpIAdd, u32, Unary(state, spv::OpUConvert, u32, binary(spv::OpShiftRightLogical, TypeScalarU64(state), address, BdaConstant(state, BdaAbi::WrittenPageShift))), constant(1)); - const auto hash = binary(spv::OpShiftRightLogical, u32, binary(spv::OpIMul, u32, page, constant(0x9e3779b1u)), constant(32u - static_cast(std::countr_zero(BdaAbi::WrittenPageSlots)))); + state.module.AddFunction(spv::OpStore, held, constant(0u)); + const auto u64 = TypeScalarU64(state); + const auto key = binary(spv::OpIAdd, u32, Unary(state, spv::OpUConvert, u32, binary(spv::OpShiftRightLogical, u64, address, BdaConstant(state, BdaAbi::WrittenChunkShift))), constant(1)); + const auto page = binary(spv::OpBitwiseAnd, u32, Unary(state, spv::OpUConvert, u32, binary(spv::OpShiftRightLogical, u64, address, BdaConstant(state, BdaAbi::WrittenPageShift))), constant(BdaAbi::WrittenChunkPages - 1u)); + const auto bit = binary(spv::OpShiftLeftLogical, u32, constant(1u), page); + const auto hash = binary(spv::OpShiftRightLogical, u32, binary(spv::OpIMul, u32, key, constant(0x9e3779b1u)), constant(32u - static_cast(std::countr_zero(BdaAbi::WrittenPageSlots)))); + const auto keyWordOf = [&](std::uint32_t slot) { return binary(spv::OpIAdd, u32, constant(BdaAbi::WrittenSlotsWord), binary(spv::OpShiftLeftLogical, u32, slot, constant(1u))); }; const auto scope = constant(spv::ScopeDevice); const auto relaxed = constant(spv::MemorySemanticsMaskNone); const auto header = state.module.AllocateId(); @@ -152,16 +159,23 @@ static std::uint32_t DefineBdaNoteWrite(SpirvEmitterState& state) { state.module.AddFunction(spv::OpLoopMerge, merge, continuation, spv::LoopControlMaskNone); state.module.AddFunction(spv::OpBranchConditional, searching, body, merge); EmitLabel(state, body); - const auto slot = binary(spv::OpIAdd, u32, constant(BdaAbi::WrittenSlotsWord), binary(spv::OpBitwiseAnd, u32, binary(spv::OpIAdd, u32, hash, probe), constant(BdaAbi::WrittenPageSlots - 1u))); + const auto slot = binary(spv::OpBitwiseAnd, u32, binary(spv::OpIAdd, u32, hash, probe), constant(BdaAbi::WrittenPageSlots - 1u)); + const auto keyWord = keyWordOf(slot); + state.module.AddFunction(spv::OpStore, held, slot); const auto current = state.module.AllocateId(); - state.module.AddFunction(spv::OpAtomicLoad, u32, current, BdaWord(state, state.faultBufferVariable, slot), scope, relaxed); - const auto held = binary(spv::OpIEqual, boolean, current, page); - state.module.AddFunction(spv::OpStore, done, held); - EmitIfCondition(state, Unary(state, spv::OpLogicalNot, boolean, held), [&] { + state.module.AddFunction(spv::OpAtomicLoad, u32, current, BdaWord(state, state.faultBufferVariable, keyWord), scope, relaxed); + const auto same = binary(spv::OpIEqual, boolean, current, key); + state.module.AddFunction(spv::OpStore, done, same); + EmitIfCondition(state, Unary(state, spv::OpLogicalNot, boolean, same), [&] { const auto previous = state.module.AllocateId(); - state.module.AddFunction(spv::OpAtomicCompareExchange, u32, previous, BdaWord(state, state.faultBufferVariable, slot), scope, relaxed, relaxed, page, constant(0u)); - const auto taken = binary(spv::OpLogicalOr, boolean, binary(spv::OpIEqual, boolean, previous, constant(0u)), binary(spv::OpIEqual, boolean, previous, page)); - state.module.AddFunction(spv::OpStore, done, taken); + state.module.AddFunction(spv::OpAtomicCompareExchange, u32, previous, BdaWord(state, state.faultBufferVariable, keyWord), scope, relaxed, relaxed, key, constant(0u)); + const auto claimed = binary(spv::OpIEqual, boolean, previous, constant(0u)); + state.module.AddFunction(spv::OpStore, done, binary(spv::OpLogicalOr, boolean, claimed, binary(spv::OpIEqual, boolean, previous, key))); + EmitIfCondition(state, claimed, [&] { + const auto index = state.module.AllocateId(); + state.module.AddFunction(spv::OpAtomicIAdd, u32, index, BdaWord(state, state.faultBufferVariable, constant(BdaAbi::WrittenCountWord)), scope, relaxed, constant(1u)); + state.module.AddFunction(spv::OpAtomicStore, BdaWord(state, state.faultBufferVariable, binary(spv::OpIAdd, u32, constant(BdaAbi::WrittenListWord), index)), scope, relaxed, slot); + }); }); state.module.AddFunction(spv::OpBranch, continuation); EmitLabel(state, continuation); @@ -170,6 +184,16 @@ static std::uint32_t DefineBdaNoteWrite(SpirvEmitterState& state) { EmitLabel(state, merge); const auto noted = state.module.AllocateId(); state.module.AddFunction(spv::OpLoad, boolean, noted, done); + EmitIfCondition(state, noted, [&] { + const auto heldSlot = state.module.AllocateId(); + state.module.AddFunction(spv::OpLoad, u32, heldSlot, held); + const auto maskWord = binary(spv::OpIAdd, u32, keyWordOf(heldSlot), constant(1u)); + const auto mask = state.module.AllocateId(); + state.module.AddFunction(spv::OpAtomicLoad, u32, mask, BdaWord(state, state.faultBufferVariable, maskWord), scope, relaxed); + EmitIfCondition(state, binary(spv::OpIEqual, boolean, binary(spv::OpBitwiseAnd, u32, mask, bit), constant(0u)), [&] { + state.module.AddFunction(spv::OpAtomicOr, u32, state.module.AllocateId(), BdaWord(state, state.faultBufferVariable, maskWord), scope, relaxed, bit); + }); + }); EmitIfCondition(state, Unary(state, spv::OpLogicalNot, boolean, noted), [&] { state.module.AddFunction(spv::OpAtomicStore, BdaWord(state, state.faultBufferVariable, constant(BdaAbi::WrittenOverflowWord)), scope, relaxed, constant(1u)); }); From ef3a3818246018fdeda99209557029f9a5e4ff60 Mon Sep 17 00:00:00 2001 From: raghav314 Date: Sun, 11 Oct 2026 12:54:02 +0530 Subject: [PATCH 389/460] test(agc): flat store stamped blocks skip when host imports are compared, not watched Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceAgcDriver/tests/execution/FlatStore.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/FlatStore.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/FlatStore.cpp index 77db354f6..f249083ba 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/FlatStore.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/FlatStore.cpp @@ -154,6 +154,10 @@ void RunStampedBlocks(AgcDriver::VulkanDevice& device, GuestBlock& guest) { namespace GuestMemory = AgcDriver::GuestMemory; const auto base = guest.Address(); Dispatch(device, 32, guest.Data()); + if (!GuestMemory::Watched(base, 2 * BlockBytes)) { + std::printf("host imports are compared, not watched: stamped blocks at 0x%llx not tested\n", static_cast(base)); + return; + } for (int use = 0; use < 2; ++use) { guest.Clear(); const auto collected = GuestMemory::CollectWrites(base, 2 * BlockBytes); From aa653ec50e6b516fef73d428e3c358e59d634b66 Mon Sep 17 00:00:00 2001 From: Abdulrahman Elkordy <196347165+Silverash-i8@users.noreply.github.com> Date: Sun, 11 Oct 2026 10:24:24 +0300 Subject: [PATCH 390/460] fix(agc): import a span's ranges one at a time when Windows refuses its alias #3158 imports a run of address-adjacent registered ranges as one span. On Windows a shared mapping in the span needs a read-write alias, and MapAlias refuses one unless the range is a single contiguous run of views of one section. Ranges that are adjacent in guest address can be separate views, so the refusal escaped from importAllocation and agc_driver_dcc_keys_first_write failed on main with "read-write alias of shared guest memory ...: the range is not one contiguous run of views of a section". A span whose alias is refused is now remembered, and its ranges are imported one at a time as before spans; a single range that is refused still throws. Claude-Session: https://claude.ai/code/session_012SLQbhAiexxW6TtjYToW8a --- .../libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp | 10 +++++++++- docs/dev/TechnicalDebt.md | 2 +- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp index a9460626e..bdce3bb0f 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp @@ -145,6 +145,7 @@ struct HostImports { std::map imports; std::uint64_t liveBytes = 0; std::set failed; + std::set unaliasedSpans; // Registry generation the imports were last reconciled with. std::uint64_t refreshedGeneration = 0; // Bumped whenever an import is dropped, so HostImport pointers taken under the lock earlier can be @@ -546,6 +547,7 @@ ImportRun importRun(const GuestAllocations::Lease& lease, std::uint64_t address) while (first > 0 && continuesRun(*lease[first - 1], *lease[first])) --first; auto last = index; while (last + 1 < lease.size() && continuesRun(*lease[last], *lease[last + 1])) ++last; + if (first != last && Imports().unaliasedSpans.contains(lease[first]->address)) return {lease[index]->address, lease[index]->bytes, {lease[index]}}; ImportRun run{lease[first]->address, lease[last]->address + lease[last]->bytes - lease[first]->address, {}}; run.ranges.assign(lease.begin() + static_cast(first), lease.begin() + static_cast(last) + 1); return run; @@ -643,7 +645,13 @@ const HostImport* importAllocation(const Context& context, HostImports& state, s std::snprintf(text, sizeof(text), "AGC graphics: host import of 0x%llx+0x%llx: memory at 0x%llx (type 0x%lx, protection 0x%lx) is neither read-write, read-only nor a shared mapping", static_cast(base), static_cast(bytes), reinterpret_cast(refused.BaseAddress), refused.Type, refused.Protect); throw std::runtime_error(text); } - entry.alias = GuestArena::GuestArenaMapAlias_nid_postfix(static_cast(base), static_cast(bytes)); + try { + entry.alias = GuestArena::GuestArenaMapAlias_nid_postfix(static_cast(base), static_cast(bytes)); + } catch (const std::runtime_error&) { + if (entry.ranges.size() == 1) throw; + state.unaliasedSpans.insert(base); + return importAllocation(context, state, address, lease); + } } #else if (!GuestMemory::Accessible(reinterpret_cast(base), static_cast(bytes), true)) { diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index f47fe5a3a..d41241ba3 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -449,7 +449,7 @@ Throughout the project, every function at every stage either **does exactly what - [Descriptor pool chain](../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp) (libSceAgcDriver) - each pool holds two storage buffer descriptors more than a set can take, as a workaround: RADV (Mesa 26.0 and later) lets a `FREE_DESCRIPTOR_SET` pool place a set up to 32 bytes past the pool's buffer ([`util_vma_heap_init`](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/vulkan/radv_descriptor_pool.c) is given `bo_size + RADV_POOL_HEAP_OFFSET` as a size), and with a page-multiple pool size those bytes are an unmapped page - [CMASK fast clears](../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver) - each fast-cleared CMASK tile (code 0) is stored as `CB_COLOR_CLEAR_WORD` into its 8x8 texels at the first draw into the target or at its fast-clear eliminate or DCC decompress pass, not as the color block writes the tile, and the CMASK is then stored as expanded (0xF): a texture or copy read of the target after that draw and before the title's eliminate sees the clear value in tiles no draw wrote, where the console keeps their previous bytes. Tile addresses follow the pipe-aligned SW_64KB_Z_X CMASK equation for 16 pipes; other CMASK swizzle modes, codes other than 0 and 0xF, a cleared tile of a DCC target and a fast clear of a mipmapped, 3D or array color target throw. When the clear runs as a compute pass (a resident target whose CMASK is host-imported), an invalid code throws at the next fast-clear materialization, not at the one that read it - [HTILE depth fast clears](../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver) - HTILE is not modeled. Only RADV's `clear_htile_mask` compute kernel (17 GFX10 words, 64x1x1 groups) is recognized, and only when it leaves ZMASK at 0 (`user[4]` and `user[5]` low nibbles both 0); the dispatch still runs, and the host depth image is cleared to `DB_DEPTH_CLEAR` the next time the surface with that `DB_HTILE_DATA_BASE` address is bound. The address match is exact, so a clear of a different HTILE range does not hit. HTILE initialization, the stencil-only update, and fast clears made by the SDK's own paths, CP DMA fills or other kernels are not covered and leave the depth image as it was. The kernel words are the ones the PR author saw RADV submit; they are not compared with Mesa's source. -- [Host imports](../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp) (libSceAgcDriver) on Linux skip ranges that are not mapped read-write, which are copied instead. The NVIDIA driver (595.91.07) refuses `VK_EXT_external_memory_host` imports of read-only or executable mappings (and shared mappings of regular files) with `VK_ERROR_OUT_OF_DEVICE_MEMORY`, after which every `vkAllocateMemory` on the device fails the same way. Read-only ranges are never imported, even on drivers that accept them. Shared mappings of regular files are not checked for. +- [Host imports](../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp) (libSceAgcDriver) on Linux skip ranges that are not mapped read-write, which are copied instead. The NVIDIA driver (595.91.07) refuses `VK_EXT_external_memory_host` imports of read-only or executable mappings (and shared mappings of regular files) with `VK_ERROR_OUT_OF_DEVICE_MEMORY`, after which every `vkAllocateMemory` on the device fails the same way. Read-only ranges are never imported, even on drivers that accept them. Shared mappings of regular files are not checked for. On Windows, a run of adjacent registered ranges whose shared views are not one contiguous run of views of a section gets no read-write alias, so its ranges are imported one at a time, as before runs were imported whole, and a buffer that crosses two of them is copied. - [Cube texture descriptors](../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestTextureResource.cpp) with equal `BASE_ARRAY` and `DEPTH` expose six faces starting at `BASE_ARRAY`. Measured with `image_load`, `image_sample` and `image_get_resinfo` on RX 6600 (gfx1032), using 64x64 R32_UINT faces and base slices 0, 5 and 6. Other ranges containing fewer than six faces or incomplete cube arrays still throw: hardware load bounds and filtered-sample addressing differ. Out-of-range cube-array sampling remains unimplemented. Samplers with `DISABLE_CUBE_WRAP` are created with `VK_SAMPLER_CREATE_NON_SEAMLESS_CUBE_MAP_BIT_EXT`, the flag RADV maps to that bit (Mesa `src/amd/vulkan/radv_sampler.c`, `src/amd/common/ac_descriptors.c`); without `VK_EXT_non_seamless_cube_map` a cube texture sampled through such a sampler throws. PS5 equivalence has not been measured on console hardware. - [DCC display buffers](../../core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp) are presented only when all keys of the console's DCC extent agree on uncompressed or a fast-clear code, padding keys included; other keys, and register-clear keys over a pending image, throw. - [Storage image write-back](../../core/libs/prx/libSceAgcDriver/Graphics/src/Texture.cpp) (libSceAgcDriver) stores the GPU results over a CPU-stamped range whose guest bytes equal the copy taken when the image's layers got their generation (images up to 1 MiB whose guest bytes were settled at that point, with no unit shadow, pending image results or recorded GPU store over them; other images keep the previous rule: the CPU's bytes win). A CPU store that rewrites the baseline value after the GPU changed it is not told apart from no store, and the GPU result wins there. The 1 MiB limit is a choice, not a console value. From 19dc22b010c5ae977f3bd7007c3a83a4c23e52d8 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 16:36:41 -0400 Subject: [PATCH 391/460] feat(libSceSysmodule): implement sceSysmoduleGetModuleHandleInternal Return a host module handle for a loaded sysmodule id so callers can sceKernelDlsym after sceSysmoduleLoadModule. Unknown or null inputs throw like neighbouring internal load paths; unloaded ids return 0x805A1001. Unblocks Prospero Win past NID D8cuU4d72xM at load. Co-Authored-By: Composer Co-authored-by: Cursor --- core/libs/prx/libSceSysmodule/CMakeLists.txt | 7 +++- core/libs/prx/libSceSysmodule/Export.cpp | 44 ++++++++++++++++++++ core/libs/tests/GuestSysmodule.cpp | 36 +++++++++++++++- 3 files changed, 83 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceSysmodule/CMakeLists.txt b/core/libs/prx/libSceSysmodule/CMakeLists.txt index 4d5f4a73e..3704a2419 100644 --- a/core/libs/prx/libSceSysmodule/CMakeLists.txt +++ b/core/libs/prx/libSceSysmodule/CMakeLists.txt @@ -5,11 +5,14 @@ configure_windows_unwind(libSceSysmodule) if(BUILD_TESTING) add_test_executable(guest_sysmodule_tests ${LIBS_INCLUDE_DIR}/tests/GuestSysmodule.cpp) target_include_directories(guest_sysmodule_tests PRIVATE ${LIBS_INCLUDE_DIR}) - target_link_libraries(guest_sysmodule_tests PRIVATE libSceSysmodule) + target_link_libraries(guest_sysmodule_tests PRIVATE libSceSysmodule libkernel) + add_dependencies(guest_sysmodule_tests libSceFiber) add_test(NAME guest_sysmodule COMMAND guest_sysmodule_tests) + set_tests_properties(guest_sysmodule PROPERTIES WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/core/libs) if(UNIX AND NOT APPLE) add_test(NAME guest_sysmodule_maps_scan COMMAND guest_sysmodule_tests) - set_tests_properties(guest_sysmodule_maps_scan PROPERTIES ENVIRONMENT "APS5_NO_PROCMAP_QUERY=1") + set_tests_properties(guest_sysmodule_maps_scan PROPERTIES ENVIRONMENT "APS5_NO_PROCMAP_QUERY=1" + WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/core/libs) endif() if(WIN32) set_tests_properties(guest_sysmodule PROPERTIES ENVIRONMENT_MODIFICATION diff --git a/core/libs/prx/libSceSysmodule/Export.cpp b/core/libs/prx/libSceSysmodule/Export.cpp index 61b73e6af..9e20b417b 100644 --- a/core/libs/prx/libSceSysmodule/Export.cpp +++ b/core/libs/prx/libSceSysmodule/Export.cpp @@ -16,9 +16,12 @@ #include "ModuleTable.hpp" #include "prx/libc/include/General.hpp" +extern "C" void* APS5_VABI dlopen_nid_postfix(const char* path, int flags); + namespace { constexpr int SCE_SYSMODULE_ERROR_UNLOADED = static_cast(0x805A1001); +constexpr int kRtldNow = 2; const char* findModuleName(const std::uint32_t id) { const auto it = kModuleTable.find(id); @@ -27,6 +30,16 @@ const char* findModuleName(const std::uint32_t id) { std::mutex gMutex; std::unordered_map gLoadCount; +std::unordered_map gHandles; + +void* openHostModule(const char* name) { + const std::string relative = std::string("libs/") + name + ".prx"; + if (void* module = dlopen_nid_postfix(relative.c_str(), kRtldNow)) { + return module; + } + const std::string bare = std::string(name) + ".prx"; + return dlopen_nid_postfix(bare.c_str(), kRtldNow); +} bool fillModuleInfoForUnwind(std::uint64_t addr, ModuleInfoForUnwind* info) { #ifdef _WIN32 @@ -75,6 +88,37 @@ int APS5_VABI sceSysmoduleGetModuleInfoForUnwind(std::uint64_t addr, int flags, return 0; } +int APS5_VABI sceSysmoduleGetModuleHandleInternal(std::uint32_t id, std::int32_t* handle) { + if ((id & 0x7fffffffu) == 0) { + throw std::runtime_error("sceSysmoduleGetModuleHandleInternal: invalid id 0"); + } + if (!handle) { + throw std::runtime_error("sceSysmoduleGetModuleHandleInternal: null handle"); + } + const char* name = findModuleName(id); + if (!name) { + throw std::runtime_error(std::string("sceSysmoduleGetModuleHandleInternal: unknown id ") + std::to_string(id)); + } + std::lock_guard lock(gMutex); + const auto loaded = gLoadCount.find(id); + if (loaded == gLoadCount.end() || loaded->second < 1) { + return SCE_SYSMODULE_ERROR_UNLOADED; + } + const auto existing = gHandles.find(id); + if (existing != gHandles.end()) { + *handle = existing->second; + return 0; + } + void* module = openHostModule(name); + if (!module) { + throw std::runtime_error(std::string("sceSysmoduleGetModuleHandleInternal: failed to open ") + name); + } + const auto value = static_cast(reinterpret_cast(module)); + gHandles.emplace(id, value); + *handle = value; + return 0; +} + int APS5_VABI sceSysmoduleIsLoaded(std::uint16_t id) { if (id == 0) { throw std::runtime_error("sceSysmoduleIsLoaded: invalid id 0"); diff --git a/core/libs/tests/GuestSysmodule.cpp b/core/libs/tests/GuestSysmodule.cpp index f4f999aa3..5929ed7c0 100644 --- a/core/libs/tests/GuestSysmodule.cpp +++ b/core/libs/tests/GuestSysmodule.cpp @@ -2,11 +2,11 @@ #include "prx/libc/include/general/VabiMacros.hpp" #include #include +#include +#include #ifdef __linux__ #include #include -#include -#include #include #include #endif @@ -18,12 +18,24 @@ int APS5_VABI sceSysmoduleUnloadModule(std::uint16_t id); int APS5_VABI sceSysmoduleLoadModuleInternal(std::uint32_t id); int APS5_VABI sceSysmoduleUnloadModuleInternal(std::uint32_t id); int APS5_VABI sceSysmoduleGetModuleInfoForUnwind(std::uint64_t addr, int flags, ModuleInfoForUnwind* info); +int APS5_VABI sceSysmoduleGetModuleHandleInternal(std::uint32_t id, std::int32_t* handle); +int APS5_VABI sceKernelDlsym(std::int32_t handle, const char* symbol, void** addr); } namespace { void Require(bool value) { if (!value) std::abort(); } +template +bool ThrowsRuntimeError(TFunction function) { + try { + function(); + } catch (const std::runtime_error&) { + return true; + } + return false; +} + constexpr std::uint16_t kFiberModuleId = 0x0006; constexpr std::uint16_t kUltModuleId = 0x0007; constexpr int kModuleUnloaded = static_cast(0x805A1001); @@ -104,4 +116,24 @@ int main() { #ifdef __linux__ CheckUnwindInfo(); #endif + + std::int32_t handle = -1; + Require(sceSysmoduleGetModuleHandleInternal(kFiberModuleId, &handle) == kModuleUnloaded); + Require(handle == -1); + Require(ThrowsRuntimeError([] { sceSysmoduleGetModuleHandleInternal(0, nullptr); })); + Require(ThrowsRuntimeError([&] { sceSysmoduleGetModuleHandleInternal(0, &handle); })); + Require(ThrowsRuntimeError([] { sceSysmoduleGetModuleHandleInternal(kFiberModuleId, nullptr); })); + Require(ThrowsRuntimeError([&] { sceSysmoduleGetModuleHandleInternal(0x00ffffffu, &handle); })); + + Require(sceSysmoduleLoadModule(kFiberModuleId) == 0); + Require(sceSysmoduleGetModuleHandleInternal(kFiberModuleId, &handle) == 0); + Require(handle >= 0); + std::int32_t again = -1; + Require(sceSysmoduleGetModuleHandleInternal(kFiberModuleId, &again) == 0); + Require(again == handle); + void* symbol = nullptr; + Require(sceKernelDlsym(handle, "sceFiberOptParamInitialize", &symbol) == 0); + Require(symbol != nullptr); + Require(sceSysmoduleUnloadModule(kFiberModuleId) == 0); + Require(sceSysmoduleGetModuleHandleInternal(kFiberModuleId, &handle) == kModuleUnloaded); } From 3bfb0445e58245ae169058138f0b80e7ad8cb267 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 11 Oct 2026 01:54:00 +0000 Subject: [PATCH 392/460] fix(libSceSysmodule): link and run guest_sysmodule_tests like the other guest tests Link libc, configure Windows unwinding for the test so its runtime_error checks are caught, and run it from libSceFiber's output directory so openHostModule finds the unpatched libSceFiber.prx next to the modules the test already loads. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01K9MzdVqDf6wHUKzNymQSze --- core/libs/prx/libSceSysmodule/CMakeLists.txt | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/core/libs/prx/libSceSysmodule/CMakeLists.txt b/core/libs/prx/libSceSysmodule/CMakeLists.txt index 3704a2419..2cfcfe11e 100644 --- a/core/libs/prx/libSceSysmodule/CMakeLists.txt +++ b/core/libs/prx/libSceSysmodule/CMakeLists.txt @@ -5,14 +5,15 @@ configure_windows_unwind(libSceSysmodule) if(BUILD_TESTING) add_test_executable(guest_sysmodule_tests ${LIBS_INCLUDE_DIR}/tests/GuestSysmodule.cpp) target_include_directories(guest_sysmodule_tests PRIVATE ${LIBS_INCLUDE_DIR}) - target_link_libraries(guest_sysmodule_tests PRIVATE libSceSysmodule libkernel) + target_link_libraries(guest_sysmodule_tests PRIVATE libSceSysmodule libkernel libc) + configure_windows_unwind(guest_sysmodule_tests) add_dependencies(guest_sysmodule_tests libSceFiber) add_test(NAME guest_sysmodule COMMAND guest_sysmodule_tests) - set_tests_properties(guest_sysmodule PROPERTIES WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/core/libs) + set_tests_properties(guest_sysmodule PROPERTIES WORKING_DIRECTORY $) if(UNIX AND NOT APPLE) add_test(NAME guest_sysmodule_maps_scan COMMAND guest_sysmodule_tests) set_tests_properties(guest_sysmodule_maps_scan PROPERTIES ENVIRONMENT "APS5_NO_PROCMAP_QUERY=1" - WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/core/libs) + WORKING_DIRECTORY $) endif() if(WIN32) set_tests_properties(guest_sysmodule PROPERTIES ENVIRONMENT_MODIFICATION From 67bccf6dd4dfb8fa81af0ba9412c968f7117416f Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sat, 10 Oct 2026 14:19:15 -0500 Subject: [PATCH 393/460] fix(libkernel): change the protection of guest thread stack pages On Windows a guest thread runs on its host thread's stack, which is not a registered guest allocation, so mprotect on any of its pages failed with EINVAL. A guest that guards the bottom of its own stack (Dolphin's JIT read-protects 64 KiB there to catch stack overflows) got an error where the PS5 changes the protection. A range that lies inside a live guest thread stack, with no GPU access requested, now takes the native protection directly. Ranges that cross the stack's bounds are still rejected. AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libkernel/CMakeLists.txt | 10 ++++ .../libkernel/DirectMemory/DirectMemory.cpp | 8 +++ core/libs/tests/GuestKernelStackProtect.cpp | 56 +++++++++++++++++++ 3 files changed, 74 insertions(+) create mode 100644 core/libs/tests/GuestKernelStackProtect.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index 0cfc3d427..1f9977767 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -273,6 +273,16 @@ if(BUILD_TESTING) set_tests_properties(guest_app_info PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_kernel_stack_protect_tests ${LIBS_INCLUDE_DIR}/tests/GuestKernelStackProtect.cpp) + target_include_directories(guest_kernel_stack_protect_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_kernel_stack_protect_tests PRIVATE libkernel libc) + configure_windows_unwind(guest_kernel_stack_protect_tests) + add_test(NAME guest_kernel_stack_protect COMMAND guest_kernel_stack_protect_tests) + set_tests_properties(guest_kernel_stack_protect PROPERTIES TIMEOUT 20) + if(WIN32) + set_tests_properties(guest_kernel_stack_protect PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_thread_self_tests ${LIBS_INCLUDE_DIR}/tests/GuestThreadSelf.cpp) target_include_directories(guest_thread_self_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_thread_self_tests PRIVATE libkernel libc) diff --git a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp index 8fd7a4170..eaf62b4ff 100644 --- a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp +++ b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp @@ -1,4 +1,5 @@ #include "prx/libkernel/DirectMemory/DirectMemory.hpp" +#include "prx/libkernel/Pthread/include/Pthread.hpp" #include "prx/libc/include/GuestAllocations.hpp" #include "prx/libc/include/GuestArena.hpp" #include "prx/libc/include/GuestWriteWatch.hpp" @@ -858,6 +859,13 @@ int DoMprotect(const void* addr, size_t len, int prot) { const auto bytes = static_cast(end - first); const auto* pointer = reinterpret_cast(first); const auto nativeProtection = LinuxProtFromSce(prot); + std::uintptr_t stackStart = 0; + std::uintptr_t stackEnd = 0; + if ((prot & GuestProtGpuReadWrite) == 0 && GuestThreadStack(first, &stackStart, &stackEnd) && first >= stackStart && end <= stackEnd) { + if (mprotect(const_cast(pointer), bytes, nativeProtection) != 0) throw std::system_error(errno, std::generic_category(), "mprotect failed"); + RecordProtection(pointer, bytes, prot); + return 0; + } GuestAllocations::Mutation mutation; #ifdef _WIN32 MEMORY_BASIC_INFORMATION memory{}; diff --git a/core/libs/tests/GuestKernelStackProtect.cpp b/core/libs/tests/GuestKernelStackProtect.cpp new file mode 100644 index 000000000..baae22806 --- /dev/null +++ b/core/libs/tests/GuestKernelStackProtect.cpp @@ -0,0 +1,56 @@ +#include "SceTypes.hpp" +#include +#include +#include +#ifdef _WIN32 +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include +#endif + +extern "C" { +int APS5_VABI scePthreadCreate(Pthread* thread, const PthreadAttr* attr, PthreadEntry entry, void* arg, const char* name); +int APS5_VABI scePthreadJoin(Pthread thread, void** retval); +int APS5_VABI sceKernelIsStack(void* addr, void** start, void** end); +int APS5_VABI mprotect_nid_postfix(void* address, std::size_t length, int protection) noexcept; +} + +static constexpr int SCE_OK = 0; +static constexpr std::uintptr_t Page = 0x4000; +static constexpr std::size_t GuardBytes = 0x10000; + +static void Require(bool value) { if (!value) std::abort(); } + +static void* APS5_VABI Worker(void* arg) { + auto& passed = *static_cast*>(arg); + int local = 0; + void* start = nullptr; + void* end = nullptr; + Require(sceKernelIsStack(&local, &start, &end) == SCE_OK); + const auto low = (reinterpret_cast(start) + Page - 1) & ~(Page - 1); + Require(low + GuardBytes + Page < reinterpret_cast(&local)); + auto* guard = reinterpret_cast(low); + Require(mprotect_nid_postfix(const_cast(guard), GuardBytes, 0) == 0); +#ifdef _WIN32 + MEMORY_BASIC_INFORMATION memory{}; + Require(VirtualQuery(const_cast(guard), &memory, sizeof(memory)) == sizeof(memory)); + Require(memory.Protect == PAGE_NOACCESS && memory.RegionSize >= GuardBytes); +#endif + Require(mprotect_nid_postfix(const_cast(guard), GuardBytes, 3) == 0); + guard[0] = 1; + guard[GuardBytes - 1] = 2; + Require(guard[0] == 1 && guard[GuardBytes - 1] == 2); + const auto outside = reinterpret_cast(start) & ~(Page - 1); + if (outside >= Page) Require(mprotect_nid_postfix(reinterpret_cast(outside - Page), 2 * Page, 3) == -1); + passed.store(true); + return nullptr; +} + +int main() { + std::atomic passed{false}; + Pthread thread = nullptr; + Require(scePthreadCreate(&thread, nullptr, Worker, &passed, nullptr) == SCE_OK); + Require(scePthreadJoin(thread, nullptr) == SCE_OK); + Require(passed.load()); +} From 24ae0a98d0227e55f89ce97bbf48f2ea8b18f709 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sat, 10 Oct 2026 18:57:46 -0500 Subject: [PATCH 394/460] fix(libkernel): check the requested stack range and restore guard pages at thread exit A stack base need not be page aligned (it is not on Linux), so rounding the range down first could leave the stack and miss the stack path. The range the guest asked for is now checked against the guest stack, and the rounded pages are protected, as FreeBSD rounds the address down. A thread that exits with part of its stack protected left those pages to the next thread that reused the stack. Protected stack pages are recorded, and a thread's stack is made readable and writable again when it is unregistered at exit. AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- .../libkernel/DirectMemory/DirectMemory.cpp | 3 +- .../prx/libkernel/Pthread/include/Pthread.hpp | 1 + .../libs/prx/libkernel/Pthread/src/Thread.cpp | 32 +++++++++++++++++++ core/libs/tests/GuestKernelStackProtect.cpp | 32 +++++++++++++++++++ 4 files changed, 67 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp index eaf62b4ff..657d0da9f 100644 --- a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp +++ b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp @@ -861,8 +861,9 @@ int DoMprotect(const void* addr, size_t len, int prot) { const auto nativeProtection = LinuxProtFromSce(prot); std::uintptr_t stackStart = 0; std::uintptr_t stackEnd = 0; - if ((prot & GuestProtGpuReadWrite) == 0 && GuestThreadStack(first, &stackStart, &stackEnd) && first >= stackStart && end <= stackEnd) { + if ((prot & GuestProtGpuReadWrite) == 0 && GuestThreadStack(address, &stackStart, &stackEnd) && address >= stackStart && address + len <= stackEnd) { if (mprotect(const_cast(pointer), bytes, nativeProtection) != 0) throw std::system_error(errno, std::generic_category(), "mprotect failed"); + NoteGuestStackProtection(first, bytes, (prot & 3) == 3); RecordProtection(pointer, bytes, prot); return 0; } diff --git a/core/libs/prx/libkernel/Pthread/include/Pthread.hpp b/core/libs/prx/libkernel/Pthread/include/Pthread.hpp index 9e6b73c73..8ea240202 100644 --- a/core/libs/prx/libkernel/Pthread/include/Pthread.hpp +++ b/core/libs/prx/libkernel/Pthread/include/Pthread.hpp @@ -143,6 +143,7 @@ struct PthreadPrivate { }; bool GuestThreadStack(std::uintptr_t address, std::uintptr_t* start, std::uintptr_t* end); +void NoteGuestStackProtection(std::uintptr_t start, std::size_t bytes, bool readWrite); int GuestThreadCpuClockId(const PthreadPrivate* thread); bool GuestThreadCpuNanos(int cpuClockThread, std::uint64_t* nanos); diff --git a/core/libs/prx/libkernel/Pthread/src/Thread.cpp b/core/libs/prx/libkernel/Pthread/src/Thread.cpp index 60f902c82..4805a2f9b 100644 --- a/core/libs/prx/libkernel/Pthread/src/Thread.cpp +++ b/core/libs/prx/libkernel/Pthread/src/Thread.cpp @@ -23,6 +23,7 @@ #include #include #include +#include #include #endif @@ -72,6 +73,7 @@ static GuestThreadState& CurrentGuestState() { } static std::mutex stackLock; static std::map> liveStacks; +static std::map protectedStackPages; static bool HostStackLimits(std::uintptr_t* low, std::uintptr_t* high) { #ifdef _WIN32 @@ -122,8 +124,38 @@ static void RegisterStack(PthreadPrivate* self) { liveStacks[self] = {low, high}; } +static void RestoreStackPages(std::uintptr_t start, std::size_t bytes) { +#ifdef _WIN32 + DWORD previous = 0; + if (!VirtualProtect(reinterpret_cast(start), bytes, PAGE_READWRITE, &previous)) + throw std::runtime_error("Cannot restore a guest thread stack's protection"); +#else + if (::mprotect(reinterpret_cast(start), bytes, PROT_READ | PROT_WRITE) != 0) + throw std::system_error(errno, std::generic_category(), "Cannot restore a guest thread stack's protection"); +#endif +} + +void NoteGuestStackProtection(std::uintptr_t start, std::size_t bytes, bool readWrite) { + std::lock_guard lock(stackLock); + auto it = protectedStackPages.upper_bound(start); + if (it != protectedStackPages.begin() && std::prev(it)->first + std::prev(it)->second > start) --it; + while (it != protectedStackPages.end() && it->first < start + bytes) it = protectedStackPages.erase(it); + if (!readWrite) protectedStackPages[start] = bytes; +} + static void UnregisterStack(PthreadPrivate* self) { std::lock_guard lock(stackLock); + if (const auto found = liveStacks.find(self); found != liveStacks.end()) { + const auto [low, high] = found->second; + for (auto it = protectedStackPages.begin(); it != protectedStackPages.end();) { + if (it->first < high && it->first + it->second > low) { + RestoreStackPages(it->first, it->second); + it = protectedStackPages.erase(it); + } else { + ++it; + } + } + } liveStacks.erase(self); } diff --git a/core/libs/tests/GuestKernelStackProtect.cpp b/core/libs/tests/GuestKernelStackProtect.cpp index baae22806..6aad702b5 100644 --- a/core/libs/tests/GuestKernelStackProtect.cpp +++ b/core/libs/tests/GuestKernelStackProtect.cpp @@ -41,16 +41,48 @@ static void* APS5_VABI Worker(void* arg) { guard[0] = 1; guard[GuardBytes - 1] = 2; Require(guard[0] == 1 && guard[GuardBytes - 1] == 2); + Require(mprotect_nid_postfix(start, GuardBytes, 0) == 0); + Require(mprotect_nid_postfix(start, GuardBytes, 3) == 0); + static_cast(start)[0] = 3; const auto outside = reinterpret_cast(start) & ~(Page - 1); if (outside >= Page) Require(mprotect_nid_postfix(reinterpret_cast(outside - Page), 2 * Page, 3) == -1); passed.store(true); return nullptr; } +static std::atomic leftGuard{0}; + +static void* APS5_VABI LeaveGuard(void*) { + int local = 0; + void* start = nullptr; + void* end = nullptr; + Require(sceKernelIsStack(&local, &start, &end) == SCE_OK); + const auto low = (reinterpret_cast(start) + Page - 1) & ~(Page - 1); + Require(mprotect_nid_postfix(reinterpret_cast(low), GuardBytes, 0) == 0); + leftGuard.store(low); + return nullptr; +} + +static void* APS5_VABI WriteLeftGuard(void*) { + int local = 0; + void* start = nullptr; + void* end = nullptr; + Require(sceKernelIsStack(&local, &start, &end) == SCE_OK); + const auto guard = leftGuard.load(); + if (guard >= reinterpret_cast(start) && guard + GuardBytes <= reinterpret_cast(&local)) { + reinterpret_cast(guard)[0] = 4; + } + return nullptr; +} + int main() { std::atomic passed{false}; Pthread thread = nullptr; Require(scePthreadCreate(&thread, nullptr, Worker, &passed, nullptr) == SCE_OK); Require(scePthreadJoin(thread, nullptr) == SCE_OK); Require(passed.load()); + Require(scePthreadCreate(&thread, nullptr, LeaveGuard, nullptr, nullptr) == SCE_OK); + Require(scePthreadJoin(thread, nullptr) == SCE_OK); + Require(scePthreadCreate(&thread, nullptr, WriteLeftGuard, nullptr, nullptr) == SCE_OK); + Require(scePthreadJoin(thread, nullptr) == SCE_OK); } From f38408ef5c1d483cf86f0219da0a80b3bbb67e67 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sun, 11 Oct 2026 02:19:56 -0500 Subject: [PATCH 395/460] fix(libkernel): keep stack protection inside the stack and never throw while restoring it Co-Authored-By: Claude Opus 5.5 --- .../libkernel/DirectMemory/DirectMemory.cpp | 12 ++++-- .../libs/prx/libkernel/Pthread/src/Thread.cpp | 10 ++--- core/libs/tests/GuestKernelStackProtect.cpp | 40 +++++++++++++++++++ 3 files changed, 54 insertions(+), 8 deletions(-) diff --git a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp index 657d0da9f..1fae6ec14 100644 --- a/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp +++ b/core/libs/prx/libkernel/DirectMemory/DirectMemory.cpp @@ -862,9 +862,15 @@ int DoMprotect(const void* addr, size_t len, int prot) { std::uintptr_t stackStart = 0; std::uintptr_t stackEnd = 0; if ((prot & GuestProtGpuReadWrite) == 0 && GuestThreadStack(address, &stackStart, &stackEnd) && address >= stackStart && address + len <= stackEnd) { - if (mprotect(const_cast(pointer), bytes, nativeProtection) != 0) throw std::system_error(errno, std::generic_category(), "mprotect failed"); - NoteGuestStackProtection(first, bytes, (prot & 3) == 3); - RecordProtection(pointer, bytes, prot); + constexpr std::uintptr_t hostPageMask = 0xfff; + const auto inside = std::max(first, (stackStart + hostPageMask) & ~hostPageMask); + const auto insideEnd = std::min(end, stackEnd & ~hostPageMask); + if (inside >= insideEnd) return 0; + auto* insidePointer = reinterpret_cast(inside); + const auto insideBytes = static_cast(insideEnd - inside); + if (mprotect(insidePointer, insideBytes, nativeProtection) != 0) throw std::system_error(errno, std::generic_category(), "mprotect failed"); + NoteGuestStackProtection(inside, insideBytes, (prot & 3) == 3); + RecordProtection(insidePointer, insideBytes, prot); return 0; } GuestAllocations::Mutation mutation; diff --git a/core/libs/prx/libkernel/Pthread/src/Thread.cpp b/core/libs/prx/libkernel/Pthread/src/Thread.cpp index 4805a2f9b..696d5fcb2 100644 --- a/core/libs/prx/libkernel/Pthread/src/Thread.cpp +++ b/core/libs/prx/libkernel/Pthread/src/Thread.cpp @@ -6,6 +6,7 @@ #include "prx/libkernel/KernelErrors.hpp" #include "prx/libc/include/CpuTopology.hpp" #include +#include #include #include #include @@ -124,15 +125,14 @@ static void RegisterStack(PthreadPrivate* self) { liveStacks[self] = {low, high}; } -static void RestoreStackPages(std::uintptr_t start, std::size_t bytes) { +static void RestoreStackPages(std::uintptr_t start, std::size_t bytes) noexcept { #ifdef _WIN32 DWORD previous = 0; - if (!VirtualProtect(reinterpret_cast(start), bytes, PAGE_READWRITE, &previous)) - throw std::runtime_error("Cannot restore a guest thread stack's protection"); + const bool restored = VirtualProtect(reinterpret_cast(start), bytes, PAGE_READWRITE, &previous) != 0; #else - if (::mprotect(reinterpret_cast(start), bytes, PROT_READ | PROT_WRITE) != 0) - throw std::system_error(errno, std::generic_category(), "Cannot restore a guest thread stack's protection"); + const bool restored = ::mprotect(reinterpret_cast(start), bytes, PROT_READ | PROT_WRITE) == 0; #endif + if (!restored) std::fprintf(stderr, "[thread] cannot restore the protection of guest stack pages 0x%llx+0x%zx at thread exit\n", static_cast(start), bytes); } void NoteGuestStackProtection(std::uintptr_t start, std::size_t bytes, bool readWrite) { diff --git a/core/libs/tests/GuestKernelStackProtect.cpp b/core/libs/tests/GuestKernelStackProtect.cpp index 6aad702b5..861844474 100644 --- a/core/libs/tests/GuestKernelStackProtect.cpp +++ b/core/libs/tests/GuestKernelStackProtect.cpp @@ -7,6 +7,8 @@ #define NOMINMAX #endif #include +#else +#include #endif extern "C" { @@ -22,6 +24,36 @@ static constexpr std::size_t GuardBytes = 0x10000; static void Require(bool value) { if (!value) std::abort(); } +struct PageState { + bool mapped; + bool writable; + std::uint64_t raw; +}; + +static PageState QueryPage(std::uintptr_t address) { +#ifdef _WIN32 + MEMORY_BASIC_INFORMATION memory{}; + Require(VirtualQuery(reinterpret_cast(address), &memory, sizeof(memory)) == sizeof(memory)); + const bool mapped = memory.State == MEM_COMMIT; + return {mapped, mapped && (memory.Protect & (PAGE_READWRITE | PAGE_EXECUTE_READWRITE)) != 0, (static_cast(memory.State) << 32) | memory.Protect}; +#else + std::FILE* maps = std::fopen("/proc/self/maps", "r"); + Require(maps != nullptr); + PageState state{false, false, 0}; + char line[512]; + while (std::fgets(line, sizeof(line), maps) != nullptr) { + unsigned long long low = 0; + unsigned long long high = 0; + char permissions[5] = {}; + if (std::sscanf(line, "%llx-%llx %4s", &low, &high, permissions) != 3 || address < low || address >= high) continue; + state = {true, permissions[0] == 'r' && permissions[1] == 'w', 1ull | (permissions[0] == 'r' ? 2ull : 0) | (permissions[1] == 'w' ? 4ull : 0) | (permissions[2] == 'x' ? 8ull : 0)}; + break; + } + std::fclose(maps); + return state; +#endif +} + static void* APS5_VABI Worker(void* arg) { auto& passed = *static_cast*>(arg); int local = 0; @@ -41,8 +73,12 @@ static void* APS5_VABI Worker(void* arg) { guard[0] = 1; guard[GuardBytes - 1] = 2; Require(guard[0] == 1 && guard[GuardBytes - 1] == 2); + const auto below = reinterpret_cast(start) - 1; + const auto belowBefore = QueryPage(below).raw; Require(mprotect_nid_postfix(start, GuardBytes, 0) == 0); + Require(QueryPage(below).raw == belowBefore); Require(mprotect_nid_postfix(start, GuardBytes, 3) == 0); + Require(QueryPage(below).raw == belowBefore); static_cast(start)[0] = 3; const auto outside = reinterpret_cast(start) & ~(Page - 1); if (outside >= Page) Require(mprotect_nid_postfix(reinterpret_cast(outside - Page), 2 * Page, 3) == -1); @@ -83,6 +119,10 @@ int main() { Require(passed.load()); Require(scePthreadCreate(&thread, nullptr, LeaveGuard, nullptr, nullptr) == SCE_OK); Require(scePthreadJoin(thread, nullptr) == SCE_OK); + for (std::uintptr_t offset = 0; offset < GuardBytes; offset += Page) { + const auto left = QueryPage(leftGuard.load() + offset); + Require(!left.mapped || left.writable); + } Require(scePthreadCreate(&thread, nullptr, WriteLeftGuard, nullptr, nullptr) == SCE_OK); Require(scePthreadJoin(thread, nullptr) == SCE_OK); } From f1af6683a11483a9156ce81a3427c88ede74adb7 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sun, 11 Oct 2026 02:13:30 -0400 Subject: [PATCH 396/460] fix(libSceSysmodule): open host modules only through libs/.prx Drop the retry on .prx after libs/.prx fails. The test now runs from its own directory holding libs/libSceFiber.prx, copied from the libSceFiber output, so the single path resolves in the test too. Co-Authored-By: Claude Sonnet 5.5 --- core/libs/prx/libSceSysmodule/CMakeLists.txt | 14 +++++++++++--- core/libs/prx/libSceSysmodule/Export.cpp | 6 +----- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/core/libs/prx/libSceSysmodule/CMakeLists.txt b/core/libs/prx/libSceSysmodule/CMakeLists.txt index 2cfcfe11e..5f204ccf4 100644 --- a/core/libs/prx/libSceSysmodule/CMakeLists.txt +++ b/core/libs/prx/libSceSysmodule/CMakeLists.txt @@ -7,13 +7,21 @@ if(BUILD_TESTING) target_include_directories(guest_sysmodule_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_sysmodule_tests PRIVATE libSceSysmodule libkernel libc) configure_windows_unwind(guest_sysmodule_tests) - add_dependencies(guest_sysmodule_tests libSceFiber) + set(guestSysmoduleRun ${CMAKE_CURRENT_BINARY_DIR}/guest_sysmodule) + file(MAKE_DIRECTORY ${guestSysmoduleRun}/libs) + add_custom_target(guest_sysmodule_runtime + COMMAND "${CMAKE_COMMAND}" -E copy_if_different + "$" "${guestSysmoduleRun}/libs/libSceFiber.prx" + VERBATIM + ) + add_dependencies(guest_sysmodule_runtime libSceFiber) + add_dependencies(guest_sysmodule_tests guest_sysmodule_runtime) add_test(NAME guest_sysmodule COMMAND guest_sysmodule_tests) - set_tests_properties(guest_sysmodule PROPERTIES WORKING_DIRECTORY $) + set_tests_properties(guest_sysmodule PROPERTIES WORKING_DIRECTORY ${guestSysmoduleRun}) if(UNIX AND NOT APPLE) add_test(NAME guest_sysmodule_maps_scan COMMAND guest_sysmodule_tests) set_tests_properties(guest_sysmodule_maps_scan PROPERTIES ENVIRONMENT "APS5_NO_PROCMAP_QUERY=1" - WORKING_DIRECTORY $) + WORKING_DIRECTORY ${guestSysmoduleRun}) endif() if(WIN32) set_tests_properties(guest_sysmodule PROPERTIES ENVIRONMENT_MODIFICATION diff --git a/core/libs/prx/libSceSysmodule/Export.cpp b/core/libs/prx/libSceSysmodule/Export.cpp index 9e20b417b..0dd8ea293 100644 --- a/core/libs/prx/libSceSysmodule/Export.cpp +++ b/core/libs/prx/libSceSysmodule/Export.cpp @@ -34,11 +34,7 @@ std::unordered_map gHandles; void* openHostModule(const char* name) { const std::string relative = std::string("libs/") + name + ".prx"; - if (void* module = dlopen_nid_postfix(relative.c_str(), kRtldNow)) { - return module; - } - const std::string bare = std::string(name) + ".prx"; - return dlopen_nid_postfix(bare.c_str(), kRtldNow); + return dlopen_nid_postfix(relative.c_str(), kRtldNow); } bool fillModuleInfoForUnwind(std::uint64_t addr, ModuleInfoForUnwind* info) { From 26bd36df8412aac47204812184681a9ab7915b71 Mon Sep 17 00:00:00 2001 From: raghav314 Date: Sun, 11 Oct 2026 11:42:51 +0530 Subject: [PATCH 397/460] fix(ajm): decode a single input buffer in place instead of copying its stated size A job's inputs were joined into one vector before decoding, copying every input buffer at its stated size. PPSA36082's FMOD submits Opus run jobs whose one input buffer states 0xFFFFFE74 bytes (a remaining size that went below zero) ahead of a single packet, so each such job copied about 4 GiB and sceAjmBatchStart took about a second; the mixer missed its deadline and the title pushed silence. A job with one input buffer now decodes from that buffer in place, reading only the packets it decodes; jobs with several input buffers are joined as before. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceAjm.native/src/Ajm.cpp | 37 +++++++++++---- core/libs/tests/GuestAjm.cpp | 54 ++++++++++++++++++++++ 2 files changed, 81 insertions(+), 10 deletions(-) diff --git a/core/libs/prx/libSceAjm.native/src/Ajm.cpp b/core/libs/prx/libSceAjm.native/src/Ajm.cpp index 6f207a53e..878733d18 100644 --- a/core/libs/prx/libSceAjm.native/src/Ajm.cpp +++ b/core/libs/prx/libSceAjm.native/src/Ajm.cpp @@ -20,6 +20,7 @@ extern "C" { #include #include #include +#include // The batch buffer and its job records are private to this library: the guest only reserves the // memory and reads the "used bytes" field of AjmBatchInfo. Sideband results written back to guest @@ -422,14 +423,30 @@ bool consumedFirstRun(Instance& instance) { return true; } -std::vector JoinInputs(const JobHeader& job, const AjmBuffer* inputs) { - std::vector input; - for (std::uint32_t index = 0; index < job.inputCount; ++index) { - const auto* data = static_cast(inputs[index].ptr); - input.insert(input.end(), data, data + inputs[index].size); +class JobInput { +public: + JobInput(const JobHeader& job, const AjmBuffer* inputs) { + if (job.inputCount == 1) { + bytes = {static_cast(inputs[0].ptr), static_cast(inputs[0].size)}; + return; + } + for (std::uint32_t index = 0; index < job.inputCount; ++index) { + const auto* data = static_cast(inputs[index].ptr); + joined.insert(joined.end(), data, data + inputs[index].size); + } + bytes = joined; } - return input; -} + JobInput(const JobInput&) = delete; + JobInput& operator=(const JobInput&) = delete; + std::size_t size() const { return bytes.size(); } + bool empty() const { return bytes.empty(); } + const std::uint8_t* data() const { return bytes.data(); } + std::uint8_t operator[](std::size_t index) const { return bytes[index]; } + +private: + std::vector joined; + std::span bytes; +}; struct PcmOutputs { const AjmBuffer* buffers; @@ -606,7 +623,7 @@ void WriteRunSideband(const JobHeader& job, const Instance& instance, std::int32 } void RunAt9(Instance& instance, const JobHeader& job, const AjmBuffer* inputs, const AjmBuffer* outputs) { - const auto input = JoinInputs(job, inputs); + const JobInput input(job, inputs); PcmOutputs pcmOutputs(outputs, job.outputCount); const auto channels = static_cast(instance.info.channels); @@ -845,7 +862,7 @@ void OpenMp3(Instance& instance) { } void RunMp3(Instance& instance, const JobHeader& job, const AjmBuffer* inputs, const AjmBuffer* outputs) { - const auto input = JoinInputs(job, inputs); + const JobInput input(job, inputs); PcmOutputs pcmOutputs(outputs, job.outputCount); const auto encoding = PcmEncoding(instance); if (encoding > 2) throw std::runtime_error("AJM MP3: PCM encoding " + std::to_string(encoding) + " is not implemented"); @@ -955,7 +972,7 @@ std::size_t OpusPacketSamples(const std::uint8_t* packet, std::size_t size) { } void RunOpus(Instance& instance, const JobHeader& job, const AjmBuffer* inputs, const AjmBuffer* outputs) { - const auto input = JoinInputs(job, inputs); + const JobInput input(job, inputs); PcmOutputs pcmOutputs(outputs, job.outputCount); const auto encoding = PcmEncoding(instance); if (encoding > 2) throw std::runtime_error("AJM Opus: PCM encoding " + std::to_string(encoding) + " is not implemented"); diff --git a/core/libs/tests/GuestAjm.cpp b/core/libs/tests/GuestAjm.cpp index f11cb7c10..193e1f445 100644 --- a/core/libs/tests/GuestAjm.cpp +++ b/core/libs/tests/GuestAjm.cpp @@ -8,6 +8,13 @@ #include #include +#ifdef _WIN32 +#include +#else +#include +#include +#endif + extern "C" { int APS5_VABI sceAjmInitialize(std::int64_t, std::uint32_t*); int APS5_VABI sceAjmFinalize(std::uint32_t); @@ -164,6 +171,52 @@ void TestOpus(std::uint32_t context) { Require(sceAjmInstanceDestroy(context, instance) == 0); } +void TestOpusInputBeyondStatedData(std::uint32_t context) { +#ifdef _WIN32 + SYSTEM_INFO system{}; + GetSystemInfo(&system); + const std::size_t pageSize = system.dwPageSize; + auto* pages = static_cast(VirtualAlloc(nullptr, 2 * pageSize, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); + Require(pages != nullptr); + DWORD oldProtection = 0; + Require(VirtualProtect(pages + pageSize, pageSize, PAGE_NOACCESS, &oldProtection) != 0); +#else + const long hostPageSize = sysconf(_SC_PAGESIZE); + Require(hostPageSize > 0); + const std::size_t pageSize = static_cast(hostPageSize); + auto* pages = static_cast(mmap(nullptr, 2 * pageSize, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0)); + Require(pages != MAP_FAILED); + Require(mprotect(pages + pageSize, pageSize, PROT_NONE) == 0); +#endif + const std::size_t packetBytes = 2 + (OPUS_STEREO[0] | (std::size_t{OPUS_STEREO[1]} << 8u)); + auto* packet = pages + pageSize - packetBytes; + std::memcpy(packet, OPUS_STEREO, packetBytes); + + std::uint32_t instance = 0; + Require(sceAjmInstanceCreate(context, 24, 0, &instance) == 0); + const std::uint32_t parameters[3] = {2, 48000, 0}; + std::vector batch(4096); + AjmBatchInfo info{}; + std::int64_t initResult[2] = {-1, -1}; + Require(sceAjmBatchInitialize(batch.data(), batch.size(), &info) == 0); + Require(sceAjmBatchJobInitialize(&info, instance, parameters, sizeof(parameters), initResult) == 0); + std::uint32_t id = 0; + AjmBatchError error{}; + Require(sceAjmBatchStart(context, &info, 0, &error, &id) == 0 && sceAjmBatchWait(context, id, 0, &error) == 0); + Require(initResult[0] == 0); + + std::vector pcm(960 * 2); + DecodeSideband sideband{}; + RunDecode(context, instance, packet, 0xFFFFFE74u, pcm.data(), pcm.size() * sizeof(std::int16_t), sideband); + Require(sideband.result == 0 && static_cast(sideband.inputConsumed) == packetBytes && sideband.outputWritten == 960 * 2 * 2); + Require(sceAjmInstanceDestroy(context, instance) == 0); +#ifdef _WIN32 + Require(VirtualFree(pages, 0, MEM_RELEASE) != 0); +#else + Require(munmap(pages, 2 * pageSize) == 0); +#endif +} + bool ParsesTo(const std::uint8_t* stream, std::uint32_t streamSize, std::uint64_t frameSize, std::uint32_t channels, std::uint32_t samples, std::uint32_t bitrate, std::uint32_t sampleRate) { AjmDecMp3ParseFrame frame; std::memset(&frame, 0xFF, sizeof(frame)); @@ -1103,6 +1156,7 @@ int main() { TestMp3ParseOfl(); TestMp3(context); TestOpus(context); + TestOpusInputBeyondStatedData(context); TestDecodeSingle(context); TestDecodeSplit(context); TestGaplessDecode(context); From 29c0814c90980e0c168e810543862168be38d221 Mon Sep 17 00:00:00 2001 From: raghav314 Date: Sun, 11 Oct 2026 13:00:49 +0530 Subject: [PATCH 398/460] fix(ajm): an Opus run stops before the next packet once the output has no room for its frame Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceAjm.native/src/Ajm.cpp | 3 ++- core/libs/tests/GuestAjm.cpp | 25 +++++++++++++++++----- 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/core/libs/prx/libSceAjm.native/src/Ajm.cpp b/core/libs/prx/libSceAjm.native/src/Ajm.cpp index 878733d18..4d7cd26fa 100644 --- a/core/libs/prx/libSceAjm.native/src/Ajm.cpp +++ b/core/libs/prx/libSceAjm.native/src/Ajm.cpp @@ -690,7 +690,7 @@ void RunAt9(Instance& instance, const JobHeader& job, const AjmBuffer* inputs, c std::fprintf(stderr, "[ajm] instance %u run flags 0x%llx: %zu input bytes in %u buffers, %zu output bytes in %u buffers, sideband %llu bytes; superframe %d (%d frames), %d channels, %d samples/frame, %d Hz, config %02x %02x %02x %02x -> result 0x%x, decoder status 0x%x, %u frames, consumed %zu, produced %zu, total samples %llu, gapless total %u skip %u skipped %u, input", job.instance, static_cast(job.flags), input.size(), job.inputCount, pcmOutputs.capacity, job.outputCount, static_cast(job.sidebandSize), instance.info.superframeSize, instance.info.framesInSuperframe, instance.info.channels, instance.info.frameSamples, instance.info.samplingRate, instance.info.configData[0], instance.info.configData[1], instance.info.configData[2], instance.info.configData[3], static_cast(result), static_cast(decodeStatus), frames, consumed, pcmOutputs.produced, static_cast(instance.totalDecodedSamples), instance.gapless.totalSamples, instance.gapless.skipSamples, instance.gapless.skippedSamples); - for (std::size_t i = 0; i < input.size() && i < 16; ++i) std::fprintf(stderr, " %02x", input[i]); + for (std::size_t i = 0; i < consumed && i < 16; ++i) std::fprintf(stderr, " %02x", input[i]); std::fprintf(stderr, "\n"); } WriteRunSideband(job, instance, result, consumed, pcmOutputs.produced, frames, CurrentFormat(instance)); @@ -1000,6 +1000,7 @@ void RunOpus(Instance& instance, const JobHeader& job, const AjmBuffer* inputs, instance.opusPending.erase(instance.opusPending.begin(), instance.opusPending.begin() + static_cast(bytes)); } while (consumed < input.size() && instance.opusPending.empty()) { + if (frames != 0 && pcmOutputs.Room() < frameBytes) break; if (input.size() - consumed < 2) { result |= AJM_RESULT_PARTIAL_INPUT; break; diff --git a/core/libs/tests/GuestAjm.cpp b/core/libs/tests/GuestAjm.cpp index 193e1f445..c35ca5e78 100644 --- a/core/libs/tests/GuestAjm.cpp +++ b/core/libs/tests/GuestAjm.cpp @@ -171,7 +171,7 @@ void TestOpus(std::uint32_t context) { Require(sceAjmInstanceDestroy(context, instance) == 0); } -void TestOpusInputBeyondStatedData(std::uint32_t context) { +void TestOpusInputBeyondStatedData(std::uint32_t context, bool multipleFrames) { #ifdef _WIN32 SYSTEM_INFO system{}; GetSystemInfo(&system); @@ -206,9 +206,23 @@ void TestOpusInputBeyondStatedData(std::uint32_t context) { Require(initResult[0] == 0); std::vector pcm(960 * 2); - DecodeSideband sideband{}; - RunDecode(context, instance, packet, 0xFFFFFE74u, pcm.data(), pcm.size() * sizeof(std::int16_t), sideband); - Require(sideband.result == 0 && static_cast(sideband.inputConsumed) == packetBytes && sideband.outputWritten == 960 * 2 * 2); + if (multipleFrames) { + constexpr std::uint64_t runMultipleFrames = 1ull << 12; + constexpr std::uint64_t sidebandStream = 1ull << 47; + struct { + DecodeSideband stream; + std::uint32_t frames; + std::uint32_t reserved; + } decoded{{-1, -1, 0, 0, 0}, 0, 0}; + Require(sceAjmBatchInitialize(batch.data(), batch.size(), &info) == 0); + Require(sceAjmBatchJobRun(&info, instance, sidebandStream | runMultipleFrames, packet, 0xFFFFFE74u, pcm.data(), pcm.size() * sizeof(std::int16_t), &decoded, sizeof(decoded)) == 0); + Require(sceAjmBatchStart(context, &info, 0, &error, &id) == 0 && sceAjmBatchWait(context, id, 0, &error) == 0); + Require(decoded.stream.result == 0 && static_cast(decoded.stream.inputConsumed) == packetBytes && decoded.stream.outputWritten == 960 * 2 * 2 && decoded.frames == 1); + } else { + DecodeSideband sideband{}; + RunDecode(context, instance, packet, 0xFFFFFE74u, pcm.data(), pcm.size() * sizeof(std::int16_t), sideband); + Require(sideband.result == 0 && static_cast(sideband.inputConsumed) == packetBytes && sideband.outputWritten == 960 * 2 * 2); + } Require(sceAjmInstanceDestroy(context, instance) == 0); #ifdef _WIN32 Require(VirtualFree(pages, 0, MEM_RELEASE) != 0); @@ -1156,7 +1170,8 @@ int main() { TestMp3ParseOfl(); TestMp3(context); TestOpus(context); - TestOpusInputBeyondStatedData(context); + TestOpusInputBeyondStatedData(context, false); + TestOpusInputBeyondStatedData(context, true); TestDecodeSingle(context); TestDecodeSplit(context); TestGaplessDecode(context); From 01ed49c98e8c4a6db368d2eff18e99fcf7897b2c Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Sat, 10 Oct 2026 19:38:51 -0400 Subject: [PATCH 399/460] feat(libkernel): implement creat --- core/libs/prx/libkernel/File/src/Stdio.cpp | 4 ++++ core/libs/tests/GuestFilesystem.cpp | 10 ++++++++++ 2 files changed, 14 insertions(+) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index ecddd0935..d5ca88470 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -545,6 +545,10 @@ int APS5_VABI open_nid_postfix(const char* path, int flags, int mode) { return PosixResult(sceKernelOpen(path, flags, static_cast(mode))); } +int APS5_VABI creat_nid_postfix(const char* path, int mode) { + return open_nid_postfix(path, SCE_KERNEL_O_WRONLY | SCE_KERNEL_O_CREAT | SCE_KERNEL_O_TRUNC, mode); +} + int APS5_VABI _open_nid_postfix(const char* path, int flags, ...) { int mode = 0; if (flags & SCE_KERNEL_O_CREAT) { diff --git a/core/libs/tests/GuestFilesystem.cpp b/core/libs/tests/GuestFilesystem.cpp index cb9c3cbdc..1a053eb18 100644 --- a/core/libs/tests/GuestFilesystem.cpp +++ b/core/libs/tests/GuestFilesystem.cpp @@ -33,6 +33,7 @@ int APS5_VABI sceKernelFtruncate(int, long long); int APS5_VABI sceKernelTruncate_nid_postfix(const char*, long long); int APS5_VABI sceKernelUtimes_nid_postfix(const char*, const void*); int APS5_VABI open_nid_postfix(const char*, int, int); +int APS5_VABI creat_nid_postfix(const char*, int); int APS5_VABI _open_nid_postfix(const char*, int, ...); int APS5_VABI close_nid_postfix(int); int APS5_VABI flock_nid_postfix(int, int); @@ -113,6 +114,15 @@ int main() { Require(read_nid_postfix(descriptors[0], received, sizeof(received)) == sizeof(received)); Require(std::memcmp(received, payload, sizeof(payload)) == 0); Require(read_nid_postfix(descriptors[0], received, sizeof(received)) == 0); + { + std::filesystem::remove("creat_probe.bin"); + int created = creat_nid_postfix("creat_probe.bin", 0644); + Require(created >= 0 && write_nid_postfix(created, "abc", 3) == 3); + Require(close_nid_postfix(created) == 0 && std::filesystem::file_size("creat_probe.bin") == 3); + created = creat_nid_postfix("creat_probe.bin", 0644); + Require(created >= 0 && close_nid_postfix(created) == 0 && std::filesystem::file_size("creat_probe.bin") == 0); + Require(std::filesystem::remove("creat_probe.bin")); + } Require(close_nid_postfix(descriptors[0]) == 0); const auto root = std::filesystem::path("anyps5-filesystem-test-" + std::to_string(std::chrono::steady_clock::now().time_since_epoch().count())); From bb4b938e34b704aec4030945caa91bad73de1997 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sat, 10 Oct 2026 21:31:24 -0500 Subject: [PATCH 400/460] feat(libkernel): implement cpuset_setaffinity for the calling thread or process Co-Authored-By: Claude Opus 5.5 --- .../prx/libkernel/Pthread/Posix/Thread.cpp | 29 ++++++++++ .../prx/libkernel/Pthread/include/Pthread.hpp | 1 + .../libs/prx/libkernel/Pthread/src/Thread.cpp | 8 +++ core/libs/tests/GuestAvailableCpumask.cpp | 56 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 5 files changed, 95 insertions(+) diff --git a/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp b/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp index 78af2d4fe..73111721c 100644 --- a/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp +++ b/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp @@ -32,6 +32,9 @@ static constexpr int GUEST_SCHED_FIFO = 1; extern "C" { int* APS5_VABI __error_nid_postfix(); int APS5_VABI scePthreadGetaffinity(Pthread thread, KernelCpumask* mask); +int APS5_VABI scePthreadSetaffinity(Pthread thread, KernelCpumask mask); +KernelCpumask APS5_VABI sceKernelGetAvailableCpumask(void); +int APS5_VABI getpid_nid_postfix(void); int APS5_VABI cpuset_getaffinity_nid_postfix(int level, int which, std::int64_t id, std::size_t size, void* mask) { constexpr int LevelWhich = 3, WhichThread = 1; @@ -53,6 +56,32 @@ int APS5_VABI cpuset_getaffinity_nid_postfix(int level, int which, std::int64_t return 0; } +int APS5_VABI cpuset_setaffinity_nid_postfix(int level, int which, std::int64_t id, std::size_t size, const void* mask) { + constexpr int LevelWhich = 3, WhichThread = 1, WhichProcess = 2; + constexpr int GuestEdeadlk = 11, GuestEfault = 14, GuestEinval = 22, GuestErange = 34; + constexpr std::size_t MaximumSize = 256 / 8; + const auto fail = [](int error) { + *__error_nid_postfix() = error; + return -1; + }; + if (size < sizeof(KernelCpumask) || size > MaximumSize) return fail(GuestErange); + if (!mask) return fail(GuestEfault); + const auto* bytes = static_cast(mask); + for (std::size_t index = sizeof(KernelCpumask); index < size; ++index) { + if (bytes[index] != 0) return fail(GuestEinval); + } + const bool thread = level == LevelWhich && which == WhichThread && id == -1; + const bool process = level == LevelWhich && which == WhichProcess && (id == -1 || id == getpid_nid_postfix()); + if (!thread && !process) NotImplemented_nid_no_patch("cpuset_setaffinity other than the calling thread or process"); + KernelCpumask requested = 0; + std::memcpy(&requested, bytes, sizeof(requested)); + const KernelCpumask affinity = requested & sceKernelGetAvailableCpumask(); + if (affinity == 0) return fail(GuestEdeadlk); + if (thread) scePthreadSetaffinity(scePthreadSelf(), affinity); + else SetLiveGuestThreadsAffinity(affinity); + return 0; +} + int APS5_VABI pthread_create_nid_postfix(Pthread* thread, const PthreadAttr* attr, PthreadEntry entry, void* arg) { return PosixThread::ToErrno(scePthreadCreate(thread, attr, entry, arg, nullptr)); } diff --git a/core/libs/prx/libkernel/Pthread/include/Pthread.hpp b/core/libs/prx/libkernel/Pthread/include/Pthread.hpp index 9e6b73c73..6b43d19e7 100644 --- a/core/libs/prx/libkernel/Pthread/include/Pthread.hpp +++ b/core/libs/prx/libkernel/Pthread/include/Pthread.hpp @@ -145,5 +145,6 @@ struct PthreadPrivate { bool GuestThreadStack(std::uintptr_t address, std::uintptr_t* start, std::uintptr_t* end); int GuestThreadCpuClockId(const PthreadPrivate* thread); bool GuestThreadCpuNanos(int cpuClockThread, std::uint64_t* nanos); +void SetLiveGuestThreadsAffinity(KernelCpumask mask); #endif diff --git a/core/libs/prx/libkernel/Pthread/src/Thread.cpp b/core/libs/prx/libkernel/Pthread/src/Thread.cpp index 60f902c82..c4add9bd7 100644 --- a/core/libs/prx/libkernel/Pthread/src/Thread.cpp +++ b/core/libs/prx/libkernel/Pthread/src/Thread.cpp @@ -191,6 +191,14 @@ int GuestThreadCpuClockId(const PthreadPrivate* thread) { return static_cast(CPU_CLOCK_BIT | static_cast(thread->cpuClockThread)); } +void SetLiveGuestThreadsAffinity(KernelCpumask mask) { + auto& registry = CpuClocks(); + std::lock_guard lock(registry.lock); + for (const auto& [id, thread] : registry.threads) { + if (!thread->_finished.load(std::memory_order_acquire)) thread->affinity.store(mask, std::memory_order_relaxed); + } +} + bool GuestThreadCpuNanos(int cpuClockThread, std::uint64_t* nanos) { auto& registry = CpuClocks(); std::lock_guard lock(registry.lock); diff --git a/core/libs/tests/GuestAvailableCpumask.cpp b/core/libs/tests/GuestAvailableCpumask.cpp index 171d43459..3e5d6b343 100644 --- a/core/libs/tests/GuestAvailableCpumask.cpp +++ b/core/libs/tests/GuestAvailableCpumask.cpp @@ -8,12 +8,15 @@ extern "C" { KernelCpumask APS5_VABI sceKernelGetAvailableCpumask(void); int APS5_VABI scePthreadCreate(Pthread* thread, const PthreadAttr* attr, PthreadEntry entry, void* arg, const char* name); int APS5_VABI scePthreadJoin(Pthread thread, void** retval); +Pthread APS5_VABI scePthreadSelf(); int APS5_VABI scePthreadGetaffinity(Pthread thread, KernelCpumask* mask); int APS5_VABI scePthreadSetaffinity(Pthread thread, KernelCpumask mask); int APS5_VABI scePthreadAttrInit(PthreadAttr* attr); int APS5_VABI scePthreadAttrDestroy(PthreadAttr* attr); int APS5_VABI scePthreadAttrGetaffinity(const PthreadAttr* attr, KernelCpumask* mask); int APS5_VABI cpuset_getaffinity_nid_postfix(int level, int which, std::int64_t id, std::size_t size, void* mask); +int APS5_VABI cpuset_setaffinity_nid_postfix(int level, int which, std::int64_t id, std::size_t size, const void* mask); +int APS5_VABI getpid_nid_postfix(void); int* APS5_VABI __error_nid_postfix(); } @@ -41,6 +44,34 @@ static void* APS5_VABI ReportCpuset(void* arg) { return nullptr; } +struct SetCpusetResult { + KernelCpumask cpu; + int results[8]; + KernelCpumask affinity[3]; +}; + +static void* APS5_VABI SetCpuset(void* arg) { + auto* result = static_cast(arg); + const KernelCpumask cpu = result->cpu; + std::uint64_t set[4] = {cpu | (1ull << 40), 0, 0, 0}; + result->results[0] = cpuset_setaffinity_nid_postfix(3, 1, -1, 8, set); + scePthreadGetaffinity(scePthreadSelf(), &result->affinity[0]); + set[1] = 1; + result->results[1] = cpuset_setaffinity_nid_postfix(3, 1, -1, 16, set) == -1 && *__error_nid_postfix() == 22; + set[1] = 0; + result->results[2] = cpuset_setaffinity_nid_postfix(3, 1, -1, sizeof(set), set); + result->results[3] = cpuset_setaffinity_nid_postfix(3, 1, -1, 4, set) == -1 && *__error_nid_postfix() == 34; + result->results[4] = cpuset_setaffinity_nid_postfix(3, 1, -1, 33, set) == -1 && *__error_nid_postfix() == 34; + result->results[5] = cpuset_setaffinity_nid_postfix(3, 1, -1, 8, nullptr) == -1 && *__error_nid_postfix() == 14; + const std::uint64_t unavailable[1] = {1ull << 40}; + result->results[6] = cpuset_setaffinity_nid_postfix(3, 1, -1, 8, unavailable) == -1 && *__error_nid_postfix() == 11; + scePthreadGetaffinity(scePthreadSelf(), &result->affinity[1]); + const std::uint64_t process[1] = {cpu}; + result->results[7] = cpuset_setaffinity_nid_postfix(3, 2, getpid_nid_postfix(), 8, process); + scePthreadGetaffinity(scePthreadSelf(), &result->affinity[2]); + return nullptr; +} + static void* APS5_VABI ReportMask(void* arg) { *static_cast(arg) = sceKernelGetAvailableCpumask(); return nullptr; @@ -86,5 +117,30 @@ int main() { Require(cpuset[0] == 0 && cpuset[1] == available && cpuset[2] == ~0ull); Require(cpuset[3] == 0 && cpuset[4] == available); Require(cpuset[5] == 1 && cpuset[6] == 1 && cpuset[7] == 1); + + KernelCpumask first = available & (~available + 1); + KernelCpumask last = first; + for (KernelCpumask cpu = first; cpu != 0; cpu <<= 1) if ((available & cpu) != 0) last = cpu; + std::promise hold; + auto held = hold.get_future(); + Pthread bystander = nullptr; + Require(scePthreadCreate(&bystander, &attr, Worker, &held, nullptr) == SCE_OK); + SetCpusetResult set{last, {}, {}}; + Require(scePthreadCreate(&thread, &attr, SetCpuset, &set, nullptr) == SCE_OK); + Require(scePthreadJoin(thread, nullptr) == SCE_OK); + Require(set.results[0] == 0 && set.affinity[0] == last); + Require(set.results[1] == 1 && set.results[2] == 0); + Require(set.results[3] == 1 && set.results[4] == 1 && set.results[5] == 1); + Require(set.results[6] == 1 && set.affinity[1] == last); + Require(set.results[7] == 0 && set.affinity[2] == last); + KernelCpumask bystanderAffinity = 0; + Require(scePthreadGetaffinity(bystander, &bystanderAffinity) == SCE_OK); + Require(bystanderAffinity == last); + const std::uint64_t everything[1] = {available}; + Require(cpuset_setaffinity_nid_postfix(3, 2, -1, 8, everything) == 0); + Require(scePthreadGetaffinity(bystander, &bystanderAffinity) == SCE_OK); + Require(bystanderAffinity == available); + hold.set_value(); + Require(scePthreadJoin(bystander, nullptr) == SCE_OK); Require(scePthreadAttrDestroy(&attr) == SCE_OK); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 03f1c4859..8b4faa104 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -363,6 +363,7 @@ Throughout the project, every function at every stage either **does exactly what - [sigaltstack](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) records each thread's alternate signal stack and reports it back, following FreeBSD's `kern_sigaltstack` (`EINVAL` for flags other than `SS_DISABLE`, `ENOMEM` below `MINSIGSTKSZ` 2048), but handlers never run on it, so `SS_ONSTACK` and `EPERM` are never reported - [libSceUlt queues](../../core/libs/prx/libSceUlt/Export.cpp) use preallocated host storage instead of the supplied guest work area; its existing size formula and the 512-byte object layouts remain unverified. Waiting-queue pool thread limits and waiter priority ordering are not implemented. Finalization wakes blocked queue callers with `ULT_ERROR_STATE`. - [Thread cancellation](../../core/libs/prx/libkernel/Pthread/src/Thread.cpp) (libkernel) follows FreeBSD libthr (states and types 0/1 and 0/2, `PTHREAD_CANCELED` 1, the `_pthread_cleanup_info` node layout), not measured on the console. A request is acted on only in `scePthreadTestcancel`, the condition variable, `sem` and `scePthreadSem` waits, `scePthreadJoin` and the sleeps, and their POSIX names; file and socket calls are not cancellation points. A sleep is not interrupted: the thread exits when it ends. The asynchronous type acts like the deferred one, except at the two set calls. +- [cpuset_setaffinity](../../core/libs/prx/libkernel/Pthread/Posix/Thread.cpp) (libkernel) - only the calling thread or the calling process is handled. For the process, the mask is stored on every guest thread that is alive at that moment; FreeBSD gives the whole process one cpuset that later threads join, but here a thread created afterwards still starts with its attribute's mask. Bits outside `sceKernelGetAvailableCpumask` are dropped, and a mask with none of those bits fails with `EDEADLK`, as FreeBSD does when the new set has no CPU in common with the parent set - [dup and dup2](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) throw for the combinations they do not model: `dup2` between a guest socket and a host descriptor in either direction, and on Windows duplicating a directory descriptor. Guest sockets share one socket object between the duplicates and the descriptor numbers come from the guest socket counter, not the lowest free number as in FreeBSD `kern_dup`. The errno values for descriptor exhaustion are not verified against the console. - [Cancellation coverage](ARCHITECTURE.md#thread-cancellation) checks deferred request handling and recovery, not full asynchronous support or console ABI compatibility. Cancellation during file/socket I/O, sleep interruption, cancellation versus exit/detach, condition signal conservation under cancellation, and safe rejection of non-null invalid or reclaimed thread handles remain unverified. No console title or homebrew establishes the cleanup NIDs or buffer layout. Accepted asynchronous mode currently has no diagnostic for its deferred behavior. - [MIMG `tfe` and `lwe`](../../core/shader/recompiler/RdnaDecoder/src/RdnaImageOpDecoder.cpp) never write the status VGPR after the data. On hardware it is written only when a texel fetch hits an unmapped page of a partially resident texture, or, for `lwe` on sample/gather, when the LOD is below the T# `MIN_LOD_WARN`; texture memory is always resident here, the LOD warning (texture-streaming feedback, like the T# `MIN_LOD_WARN` itself) is never reported, and `lwe` on `image_get_lod` throws. From bb9bd2183c125b12e749738682de7805f1c103a2 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 23:21:38 +0000 Subject: [PATCH 401/460] fix(libkernel): return ESPIPE for positioned socket I/O --- core/libs/prx/libkernel/File/src/Stdio.cpp | 25 ++++++++++++++++++++ core/libs/tests/GuestSocketReadWrite.cpp | 27 ++++++++++++++++++++++ docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 53 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libkernel/File/src/Stdio.cpp b/core/libs/prx/libkernel/File/src/Stdio.cpp index ecddd0935..ffa1d9a4b 100644 --- a/core/libs/prx/libkernel/File/src/Stdio.cpp +++ b/core/libs/prx/libkernel/File/src/Stdio.cpp @@ -266,11 +266,25 @@ static constexpr int GUEST_EBADF = 9; static constexpr int GUEST_EFAULT = 14; static constexpr int GUEST_EEXIST = 17; static constexpr int GUEST_EINVAL = 22; +static constexpr int GUEST_ESPIPE = 29; static constexpr int GUEST_ENAMETOOLONG = 63; static constexpr int GUEST_ENOTDIR = 20; static constexpr int GUEST_EWOULDBLOCK = 35; static constexpr int GUEST_ENOTEMPTY = 66; +static int SceErrorFromErrno(int error); + +static std::int64_t PositionedGuestSocketError(int descriptor) { + if (descriptor < GuestSockets::FirstDescriptor) return 0; + return SceErrorFromErrno(GuestSockets::IsOpen(descriptor) ? GUEST_ESPIPE : GUEST_EBADF); +} + +static bool FailPosixPositionedGuestSocket(int descriptor) { + if (descriptor < GuestSockets::FirstDescriptor) return false; + errno = GuestSockets::IsOpen(descriptor) ? GUEST_ESPIPE : GUEST_EBADF; + return true; +} + static int SceErrorFromErrno(int error) { return static_cast(0x80020000u | static_cast(error > 0 && error <= 34 ? error : error == GUEST_ENOTEMPTY ? error : GUEST_EIO)); } @@ -570,6 +584,7 @@ int64_t APS5_VABI pread_nid_postfix(int d, void* buf, size_t nbytes, int64_t off if (offset < 0) { APS5_INVALID_ARG_EX; } + if (FailPosixPositionedGuestSocket(d)) return -1; const GuestArena::HostWrite destination(buf, nbytes); if (!destination.Open()) errno = EFAULT; else if (File::ReadRandomDevice(d, buf, nbytes)) return static_cast(nbytes); @@ -587,6 +602,7 @@ int64_t APS5_VABI pwrite_nid_disambig1_nid_postfix(int d, const void* buf, size_ if (offset < 0) { APS5_INVALID_ARG_EX; } + if (FailPosixPositionedGuestSocket(d)) return -1; auto n = NativePwrite(d, buf, nbytes, offset); if (n < 0) { throw std::runtime_error(std::string(__func__) + ": pwrite failed, fd=" + std::to_string(d) + ", errno=" + std::to_string(errno)); @@ -791,6 +807,7 @@ static bool OpenIovecs(const KernelIovec* iov, int iovcnt, std::deque(nbytes); @@ -802,6 +819,7 @@ int64_t APS5_VABI sceKernelPread(int d, void* buf, size_t nbytes, int64_t offset int64_t APS5_VABI sceKernelPwrite(int d, const void* buf, size_t nbytes, int64_t offset) { if (buf == nullptr && nbytes != 0) return SceErrorFromErrno(GUEST_EFAULT); if (offset < 0) return SceErrorFromErrno(GUEST_EINVAL); + if (const auto error = PositionedGuestSocketError(d); error != 0) return error; char empty = 0; const auto result = NativePwrite(d, nbytes == 0 && buf == nullptr ? &empty : buf, nbytes, offset); return result < 0 ? SceErrorFromErrno(errno) : result; @@ -817,6 +835,9 @@ static std::int64_t TransferIovecs(int d, const KernelIovec* iov, int iovcnt, co total += iov[i].length; } if (offset != nullptr && total > static_cast(std::numeric_limits::max() - *offset)) return SceErrorFromErrno(GUEST_EINVAL); + if (offset != nullptr) { + if (const auto error = PositionedGuestSocketError(d); error != 0) return error; + } std::deque destinations; if (!write && !OpenIovecs(iov, iovcnt, destinations)) return SceErrorFromErrno(GUEST_EFAULT); if (offset == nullptr && d >= GuestSockets::FirstDescriptor) { @@ -867,6 +888,7 @@ int64_t APS5_VABI sceKernelPwritev(int d, const KernelIovec* iov, int iovcnt, in int64_t APS5_VABI sceKernelPread(int d, void* buf, size_t nbytes, int64_t offset) { if (buf == nullptr && nbytes != 0) return SceErrorFromErrno(GUEST_EFAULT); if (offset < 0) return SceErrorFromErrno(GUEST_EINVAL); + if (const auto error = PositionedGuestSocketError(d); error != 0) return error; const GuestArena::HostWrite destination(buf, nbytes); if (!destination.Open()) return SceErrorFromErrno(GUEST_EFAULT); if (File::ReadRandomDevice(d, buf, nbytes)) return static_cast(nbytes); @@ -877,6 +899,7 @@ int64_t APS5_VABI sceKernelPread(int d, void* buf, size_t nbytes, int64_t offset int64_t APS5_VABI sceKernelPwrite(int d, const void* buf, size_t nbytes, int64_t offset) { if (buf == nullptr && nbytes != 0) return SceErrorFromErrno(GUEST_EFAULT); if (offset < 0) return SceErrorFromErrno(GUEST_EINVAL); + if (const auto error = PositionedGuestSocketError(d); error != 0) return error; const auto result = NativePwrite(d, buf, nbytes, offset); return result < 0 ? SceErrorFromErrno(errno) : result; } @@ -907,6 +930,7 @@ int64_t APS5_VABI sceKernelWritev(int d, const KernelIovec* iov, int iovcnt) { int64_t APS5_VABI sceKernelPreadv(int d, const KernelIovec* iov, int iovcnt, int64_t offset) { if (const int error = CheckIovecs(iov, iovcnt)) return error; if (offset < 0) return SceErrorFromErrno(GUEST_EINVAL); + if (const auto error = PositionedGuestSocketError(d); error != 0) return error; std::deque destinations; if (!OpenIovecs(iov, iovcnt, destinations)) return SceErrorFromErrno(GUEST_EFAULT); if (File::IsRandomDevice(d)) return ReadRandomIovecs(d, iov, iovcnt); @@ -917,6 +941,7 @@ int64_t APS5_VABI sceKernelPreadv(int d, const KernelIovec* iov, int iovcnt, int int64_t APS5_VABI sceKernelPwritev(int d, const KernelIovec* iov, int iovcnt, int64_t offset) { if (const int error = CheckIovecs(iov, iovcnt)) return error; if (offset < 0) return SceErrorFromErrno(GUEST_EINVAL); + if (const auto error = PositionedGuestSocketError(d); error != 0) return error; const auto result = static_cast(::pwritev(d, NativeIovecs(iov), iovcnt, static_cast(offset))); return result < 0 ? SceErrorFromErrno(errno) : result; } diff --git a/core/libs/tests/GuestSocketReadWrite.cpp b/core/libs/tests/GuestSocketReadWrite.cpp index 34952e5dd..b89370b8c 100644 --- a/core/libs/tests/GuestSocketReadWrite.cpp +++ b/core/libs/tests/GuestSocketReadWrite.cpp @@ -38,6 +38,14 @@ std::int64_t APS5_VABI sceKernelRead(int, void*, std::size_t); std::int64_t APS5_VABI sceKernelWrite(int, const void*, std::size_t); std::int64_t APS5_VABI sceKernelReadv(int, const Iovec*, int); std::int64_t APS5_VABI sceKernelWritev(int, const Iovec*, int); +std::int64_t APS5_VABI pread_nid_postfix(int, void*, std::size_t, std::int64_t); +std::int64_t APS5_VABI pwrite_nid_disambig1_nid_postfix(int, const void*, std::size_t, std::int64_t); +std::int64_t APS5_VABI preadv_nid_postfix(int, const Iovec*, int, std::int64_t); +std::int64_t APS5_VABI pwritev_nid_postfix(int, const Iovec*, int, std::int64_t); +std::int64_t APS5_VABI sceKernelPread(int, void*, std::size_t, std::int64_t); +std::int64_t APS5_VABI sceKernelPwrite(int, const void*, std::size_t, std::int64_t); +std::int64_t APS5_VABI sceKernelPreadv(int, const Iovec*, int, std::int64_t); +std::int64_t APS5_VABI sceKernelPwritev(int, const Iovec*, int, std::int64_t); int* APS5_VABI __error_nid_postfix(); } @@ -53,6 +61,7 @@ constexpr short In = 0x1; constexpr short Hup = 0x10; constexpr int KernelAgain = static_cast(0x80020023u); constexpr int KernelBadDescriptor = static_cast(0x80020009u); +constexpr int KernelIllegalSeek = static_cast(0x8002001du); static bool Readable(int descriptor, short accepted = In) { PollDescriptor waited{descriptor, In, 0}; @@ -115,6 +124,18 @@ int main() { Require(Readable(server) && sceKernelReadv(server, sceVectorRead, 2) == sizeof(sceVectorFirst) + sizeof(sceVectorSecond) - 1); Require(std::memcmp(sceVectorReadFirst, "sceKernel", sizeof(sceVectorReadFirst)) == 0 && std::strcmp(sceVectorReadSecond, "Writev") == 0); + const char positionedWrite[] = "positioned"; + Iovec positionedIov[] = {{const_cast(positionedWrite), sizeof(positionedWrite)}}; + *__error_nid_postfix() = 0; + Require(pread_nid_postfix(server, received, 1, 0) == -1 && *__error_nid_postfix() == 29); + Require(pwrite_nid_disambig1_nid_postfix(client, positionedWrite, 1, 0) == -1 && *__error_nid_postfix() == 29); + Require(preadv_nid_postfix(server, positionedIov, 1, 0) == -1 && *__error_nid_postfix() == 29); + Require(pwritev_nid_postfix(client, positionedIov, 1, 0) == -1 && *__error_nid_postfix() == 29); + Require(sceKernelPread(server, received, 1, 0) == KernelIllegalSeek); + Require(sceKernelPwrite(client, positionedWrite, 1, 0) == KernelIllegalSeek); + Require(sceKernelPreadv(server, positionedIov, 1, 0) == KernelIllegalSeek); + Require(sceKernelPwritev(client, positionedIov, 1, 0) == KernelIllegalSeek); + Require(fcntl_nid_postfix(server, 4, 4) == 0); *__error_nid_postfix() = 0; Require(read_nid_postfix(server, received, sizeof(received)) == -1 && *__error_nid_postfix() == 35); @@ -140,6 +161,12 @@ int main() { Require(_read_nid_postfix(server, received, sizeof(received)) == -1 && *__error_nid_postfix() == 9); Require(sceKernelRead(server, received, sizeof(received)) == KernelBadDescriptor); Require(sceKernelWrite(server, request, sizeof(request)) == KernelBadDescriptor); + Require(sceKernelPread(server, received, 1, 0) == KernelBadDescriptor); + Require(sceKernelPwrite(server, request, 1, 0) == KernelBadDescriptor); + Require(pread_nid_postfix(server, received, 1, 0) == -1 && *__error_nid_postfix() == 9); + Require(pwrite_nid_disambig1_nid_postfix(server, request, 1, 0) == -1 && *__error_nid_postfix() == 9); + Require(preadv_nid_postfix(server, positionedIov, 1, 0) == -1 && *__error_nid_postfix() == 9); + Require(pwritev_nid_postfix(server, positionedIov, 1, 0) == -1 && *__error_nid_postfix() == 9); Require(close_nid_postfix(listener) == 0); return 0; } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 572482f0d..451b5c606 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -505,7 +505,7 @@ Throughout the project, every function at every stage either **does exactly what - [collate::_Getcat](../../core/libs/prx/libc/src/LocaleSupport.cpp) (libc) builds the facet for the C locale only and throws for any other locale name. The console's `libc.prx` stores pointers to its collation state tables at offsets 0x10 and 0x18 of the facet; ours leaves them null, as only the facet's own functions read them. `do_transform` throws for input with embedded null characters, which the console drops depending on the output buffer size, and for results of 4095 characters or more, whose string storage uses the guest's over-aligned large allocation. - [The scanf family](../../core/libs/prx/libc/src/Formatting.cpp) (libc) hands `%i` to glibc on Linux, which since 2.38 reads the C23 `0b`/`0B` binary prefix when the library is built as GNU C++: `sscanf("0b101", "%i", &value)` stores 5 where C17 stores 0 and stops at the `b`. The strtol family stops at the prefix as C17 does - [socketpair](../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) - the pair is the host's `AF_UNIX` pair on Linux and, on Windows, a loopback TCP connection for `SOCK_STREAM` or two connected UDP sockets for `SOCK_DGRAM`; `SOCK_SEQPACKET` throws on Windows. A datagram sent to a full receiver gets `ENOBUFS` on the console, waits or gets `EAGAIN` on Linux and is dropped on Windows, and on Windows a datagram `recv` after the peer closed reports `ECONNRESET` instead of the console's `ENOTCONN`, and a stream `send` after the peer closed reports `ECONNABORTED` instead of the console's `EPIPE`. `read` and `write` on a pair end go to the host like on every guest socket. `getpeername` on a pair end succeeds after the peer closed, where the console returns `ENOTCONN`, and `connect`, `bind` and `sendto` with an address on a pair end report an address-family error instead of the console's `EISCONN`. On Windows `poll` on a stream end reports `POLLHUP` without `POLLIN` after the peer's `shutdown(SHUT_WR)` or `close`, where the console reports `POLLIN`, with `POLLHUP` only after `close` -- [read, write and vector I/O](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) on a guest socket use `recv` and `send` without flags; `sceKernelReadv` and `sceKernelWritev` gather or scatter through one socket operation, preserving stream short reads and datagram boundaries. The positioned `pread`, `pwrite`, `preadv` and `pwritev` forms still go to the host, which has no such descriptor; the console returns `ESPIPE` for these calls. `EPIPE` from a write raises the host's `SIGPIPE` on Linux, which the guest can neither ignore (`signal` maps only six signals) nor suppress (`SO_NOSIGPIPE` is not an option here), and raises nothing on Windows; a `write` longer than `INT_MAX` bytes is `EMSGSIZE` where the console sends it. AIO on a guest socket still goes to the host, which has no such descriptor +- [read, write and vector I/O](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) on a guest socket use `recv` and `send` without flags; `sceKernelReadv` and `sceKernelWritev` gather or scatter through one socket operation, preserving stream short reads and datagram boundaries. The positioned `pread`, `pwrite`, `preadv` and `pwritev` forms, and their `sceKernel` forms, return `ESPIPE` for an open guest socket and `EBADF` for a closed one. `EPIPE` from a write raises the host's `SIGPIPE` on Linux, which the guest can neither ignore (`signal` maps only six signals) nor suppress (`SO_NOSIGPIPE` is not an option here), and raises nothing on Windows; a `write` longer than `INT_MAX` bytes is `EMSGSIZE` where the console sends it. AIO on a guest socket still goes to the host, which has no such descriptor - [poll](../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) waits only on guest sockets, through `WSAPoll` on Windows and `poll` on Linux; the `pollfd` layout and flag values are FreeBSD's. A descriptor below the socket range (files, pipes, stdio) throws, as do event bits other than `POLLIN`, `POLLPRI`, `POLLOUT`, `POLLRDNORM`, `POLLRDBAND` and `POLLWRBAND` (the `POLLERR`, `POLLHUP` and `POLLNVAL` bits are ignored in `events`), `POLLPRI` on Windows, which `WSAPoll` rejects, and an infinite timeout with no socket to wait on. A closed socket descriptor reports `POLLNVAL`; the console's limit on the descriptor count is unknown and not checked - [msync](../../core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp) (libkernel) follows FreeBSD 11's `kern_msync` and `vm_map_sync`, not measured on the console: the address is rounded down to the page, a range that wraps is `EINVAL`, as is `MS_ASYNC` with `MS_INVALIDATE`, and a range whose start or any page is not mapped (as `sceKernelVirtualQuery` reports it, reservations included) is `ENOMEM`; a page unmapped from inside a larger mapping still counts as mapped there, since `sceKernelVirtualQuery` keeps reporting the whole mapping. The guest has no file mappings, so a successful call writes nothing back. `MS_INVALIDATE` on a mapped range throws: FreeBSD returns `EBUSY` when the range holds `mlock`ed pages, and locked pages are not tracked - [Thread exit destructors](../../core/libs/prx/libSceLibcInternal/Export.cpp) (libSceLibcInternal) run when a libkernel thread finishes; on the main or a host thread only `_sceLibcInternalThreadDtors` runs them, as `exit` does, else they are dropped From 4b3f02ab13b7ea089ce247927a25d2a929245dd0 Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sun, 11 Oct 2026 03:22:09 +0000 Subject: [PATCH 402/460] docs(libkernel): clarify guest socket descriptor range --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 451b5c606..8211cd700 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -505,7 +505,7 @@ Throughout the project, every function at every stage either **does exactly what - [collate::_Getcat](../../core/libs/prx/libc/src/LocaleSupport.cpp) (libc) builds the facet for the C locale only and throws for any other locale name. The console's `libc.prx` stores pointers to its collation state tables at offsets 0x10 and 0x18 of the facet; ours leaves them null, as only the facet's own functions read them. `do_transform` throws for input with embedded null characters, which the console drops depending on the output buffer size, and for results of 4095 characters or more, whose string storage uses the guest's over-aligned large allocation. - [The scanf family](../../core/libs/prx/libc/src/Formatting.cpp) (libc) hands `%i` to glibc on Linux, which since 2.38 reads the C23 `0b`/`0B` binary prefix when the library is built as GNU C++: `sscanf("0b101", "%i", &value)` stores 5 where C17 stores 0 and stops at the `b`. The strtol family stops at the prefix as C17 does - [socketpair](../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) - the pair is the host's `AF_UNIX` pair on Linux and, on Windows, a loopback TCP connection for `SOCK_STREAM` or two connected UDP sockets for `SOCK_DGRAM`; `SOCK_SEQPACKET` throws on Windows. A datagram sent to a full receiver gets `ENOBUFS` on the console, waits or gets `EAGAIN` on Linux and is dropped on Windows, and on Windows a datagram `recv` after the peer closed reports `ECONNRESET` instead of the console's `ENOTCONN`, and a stream `send` after the peer closed reports `ECONNABORTED` instead of the console's `EPIPE`. `read` and `write` on a pair end go to the host like on every guest socket. `getpeername` on a pair end succeeds after the peer closed, where the console returns `ENOTCONN`, and `connect`, `bind` and `sendto` with an address on a pair end report an address-family error instead of the console's `EISCONN`. On Windows `poll` on a stream end reports `POLLHUP` without `POLLIN` after the peer's `shutdown(SHUT_WR)` or `close`, where the console reports `POLLIN`, with `POLLHUP` only after `close` -- [read, write and vector I/O](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) on a guest socket use `recv` and `send` without flags; `sceKernelReadv` and `sceKernelWritev` gather or scatter through one socket operation, preserving stream short reads and datagram boundaries. The positioned `pread`, `pwrite`, `preadv` and `pwritev` forms, and their `sceKernel` forms, return `ESPIPE` for an open guest socket and `EBADF` for a closed one. `EPIPE` from a write raises the host's `SIGPIPE` on Linux, which the guest can neither ignore (`signal` maps only six signals) nor suppress (`SO_NOSIGPIPE` is not an option here), and raises nothing on Windows; a `write` longer than `INT_MAX` bytes is `EMSGSIZE` where the console sends it. AIO on a guest socket still goes to the host, which has no such descriptor +- [read, write and vector I/O](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) on a guest socket use `recv` and `send` without flags; `sceKernelReadv` and `sceKernelWritev` gather or scatter through one socket operation, preserving stream short reads and datagram boundaries. The positioned `pread`, `pwrite`, `preadv` and `pwritev` forms, and their `sceKernel` forms, return `ESPIPE` for an open guest socket and `EBADF` for a closed one. Guest socket descriptors start at `0x10000000` and are allocated upward, so the descriptor range selects the guest socket table or the host descriptor table. `EPIPE` from a write raises the host's `SIGPIPE` on Linux, which the guest can neither ignore (`signal` maps only six signals) nor suppress (`SO_NOSIGPIPE` is not an option here), and raises nothing on Windows; a `write` longer than `INT_MAX` bytes is `EMSGSIZE` where the console sends it. AIO on a guest socket still goes to the host, which has no such descriptor - [poll](../../core/libs/prx/libkernel/Socket/src/Udp.cpp) (libkernel) waits only on guest sockets, through `WSAPoll` on Windows and `poll` on Linux; the `pollfd` layout and flag values are FreeBSD's. A descriptor below the socket range (files, pipes, stdio) throws, as do event bits other than `POLLIN`, `POLLPRI`, `POLLOUT`, `POLLRDNORM`, `POLLRDBAND` and `POLLWRBAND` (the `POLLERR`, `POLLHUP` and `POLLNVAL` bits are ignored in `events`), `POLLPRI` on Windows, which `WSAPoll` rejects, and an infinite timeout with no socket to wait on. A closed socket descriptor reports `POLLNVAL`; the console's limit on the descriptor count is unknown and not checked - [msync](../../core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp) (libkernel) follows FreeBSD 11's `kern_msync` and `vm_map_sync`, not measured on the console: the address is rounded down to the page, a range that wraps is `EINVAL`, as is `MS_ASYNC` with `MS_INVALIDATE`, and a range whose start or any page is not mapped (as `sceKernelVirtualQuery` reports it, reservations included) is `ENOMEM`; a page unmapped from inside a larger mapping still counts as mapped there, since `sceKernelVirtualQuery` keeps reporting the whole mapping. The guest has no file mappings, so a successful call writes nothing back. `MS_INVALIDATE` on a mapped range throws: FreeBSD returns `EBUSY` when the range holds `mlock`ed pages, and locked pages are not tracked - [Thread exit destructors](../../core/libs/prx/libSceLibcInternal/Export.cpp) (libSceLibcInternal) run when a libkernel thread finishes; on the main or a host thread only `_sceLibcInternalThreadDtors` runs them, as `exit` does, else they are dropped From 520b9facb6d1bdab3c7eb3f64dbc6cc8bc982d63 Mon Sep 17 00:00:00 2001 From: Get-fix <100056769+Get-fix@users.noreply.github.com> Date: Sat, 10 Oct 2026 19:26:37 +0300 Subject: [PATCH 403/460] fix(libc): export daemon VLC (PPSA85300) imports daemon (Wn6I3wVATUE) and fails to load because no library exports it. Forking into the background is not supported, so it throws if called. VLC 3.0 calls daemon only when its daemon option is set, and the title starts libVLC with --ignore-config and without --daemon. --- core/libs/prx/libc/CMakeLists.txt | 9 +++++++++ core/libs/prx/libc/src/Process.cpp | 7 +++++++ core/libs/tests/GuestDaemon.cpp | 19 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 36 insertions(+) create mode 100644 core/libs/tests/GuestDaemon.cpp diff --git a/core/libs/prx/libc/CMakeLists.txt b/core/libs/prx/libc/CMakeLists.txt index 63e873da7..b1914d250 100644 --- a/core/libs/prx/libc/CMakeLists.txt +++ b/core/libs/prx/libc/CMakeLists.txt @@ -263,6 +263,15 @@ if(BUILD_TESTING) set_tests_properties(guest_memrchr PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_daemon_tests ${LIBS_INCLUDE_DIR}/tests/GuestDaemon.cpp) + target_include_directories(guest_daemon_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_daemon_tests PRIVATE libc) + configure_windows_unwind(guest_daemon_tests) + add_test(NAME guest_daemon COMMAND guest_daemon_tests) + if(WIN32) + set_tests_properties(guest_daemon PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_lifecycle_tests ${LIBS_INCLUDE_DIR}/tests/GuestLifecycle.cpp) target_include_directories(guest_lifecycle_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_lifecycle_tests PRIVATE libc) diff --git a/core/libs/prx/libc/src/Process.cpp b/core/libs/prx/libc/src/Process.cpp index ce307b378..0e2341b02 100644 --- a/core/libs/prx/libc/src/Process.cpp +++ b/core/libs/prx/libc/src/Process.cpp @@ -196,4 +196,11 @@ int APS5_VABI at_quick_exit_nid_postfix(GuestExitCallback func) { _Exit_nid_postfix(status); } +int APS5_VABI daemon_nid_postfix(int noChdir, int noClose) { + (void)noChdir; + (void)noClose; + NotImplemented_nid_no_patch("daemon: forking into the background"); + return -1; +} + } diff --git a/core/libs/tests/GuestDaemon.cpp b/core/libs/tests/GuestDaemon.cpp new file mode 100644 index 000000000..0e87384eb --- /dev/null +++ b/core/libs/tests/GuestDaemon.cpp @@ -0,0 +1,19 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#include +#include +extern "C" int APS5_VABI daemon_nid_postfix(int, int); +static void Require(bool value) { if (!value) std::abort(); } +static bool ReportsNotImplemented(int noChdir, int noClose) { + try { + daemon_nid_postfix(noChdir, noClose); + } catch (const std::runtime_error& error) { + return std::string(error.what()) == "daemon: forking into the background not implemented"; + } + return false; +} +int main() { + Require(ReportsNotImplemented(1, 0)); + Require(ReportsNotImplemented(0, 0)); + Require(ReportsNotImplemented(1, 1)); +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 572482f0d..e52830975 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -531,6 +531,7 @@ Throughout the project, every function at every stage either **does exactly what - [s_call_b64](../../core/shader/recompiler/Translation/src/ScalarInstructions.cpp) saves `PC + 4` over `GetShaderBase` and branches to `PC + 4 + signext(SIMM16) * 4`, following [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) sections 3.6.3, 12.2 and 12.3. Forward callees after `s_endpgm` are decoded. Measured on RDNA2 (gfx1036) in wave32 and wave64 with SIMM16 = 3: the link pair receives `PC + 4`, the branch goes to `PC + 4 + signext(SIMM16) * 4`, and `s_setpc_b64` of the link returns to the instruction after the call. Execution on the PS5 (gfx1013) and title compatibility for these bounded scalar calls remain unverified; `s_rfe_b64` still throws in translation. - [sceVideodec2CreateDecoder](../../core/libs/prx/libSceVideodec2/Export.cpp) (libSceVideodec2) decodes HEVC (codec type 974921) and VP9 (2382845) through FFmpeg, besides H.264. The tags come from SharpProspero's `Videodec2CodecType` and EVO-PLAYER-PS5's `sce_videodec2.h`, which records them from hardware runs. HEVC input is converted from length-prefixed NAL units to Annex B as for H.264, and VP9 frames are passed through. Output is the same 8-bit NV12 for every codec; a decoder for HEVC Main10 or VP9 Profile 2 is created, but a frame deeper than 8 bits throws, since the console's 10-bit output layout is unknown, and `QueryDecoderMemoryInfo` reports the 8-bit frame size for it. `sceVideodec2GetPictureInfo` and `sceVideodec2GetAvcPictureInfo` throw for HEVC and VP9 pictures, which use other layouts (`SceVideodec2HevcPictureInfo`, `SceVideodec2Vp9PictureInfo`), and `sceVideodec2GetHevcPictureInfo` and `sceVideodec2GetVp9PictureInfo` are not exported - [syslog](../../core/libs/prx/libc/src/Syslog.cpp) (libc) writes each message to `stderr` as `[syslog:] `, with a newline added when missing, instead of sending it to the console's system log. Priority and facility handling follow FreeBSD's `vsyslog`: `%m` expands to the guest errno's message, unknown bits are reported (as priority 0x23, since FreeBSD passes `LOG_ERR|LOG_CONS|LOG_PERROR|LOG_PID` as the priority) and masked, a missing facility becomes `LOG_USER`, and errno is preserved. `openlog`, `setlogmask` and `closelog` are not implemented, so the identity, options and mask they set are not applied +- [daemon](../../core/libs/prx/libc/src/Process.cpp) (libc) throws: forking into the background is not supported. PPSA85300 imports it and fails to load without it. VLC 3.0 calls it only when its `daemon` option is set ([libvlc.c](https://github.com/videolan/vlc/blob/6de05adcbaf2e8b85fe86aad4169393098628119/src/libvlc.c#L190-L197)), and the title starts libVLC with `--ignore-config` and without `--daemon` ([player.cc](https://github.com/ZiZc3/VLC/blob/f3253020a64368153233ede29183014c0b6c8979/src/player.cc#L960-L981)) - [sigaction](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) records the handler, flags and mask of every signal from 1 to 128 and reports them back, following FreeBSD's `sigaction` (`EINVAL` for other signals, and for a handler on `SIGKILL` or `SIGSTOP`); `signal` records its handler with `SA_RESTART`, as FreeBSD's libc does. Only handlers without `SA_SIGINFO` on the signals `signal` already delivers (`SIGINT`, `SIGILL`, `SIGABRT`, `SIGFPE`, `SIGSEGV`, `SIGTERM`) run, the same way `signal` handlers do; their mask and flags are not applied. `SA_SIGINFO` handlers and other signals (`SIGBUS`, `SIGHUP`, ...) are recorded but never called, since that needs FreeBSD's `siginfo_t` and `ucontext_t`; `raise` of an `SA_SIGINFO` handler on a signal `signal` delivers throws instead of terminating, and `raise` of the other signals still returns `EINVAL` - [kill](../../core/libs/prx/libkernel/System/src/Signals.cpp) (libkernel) follows FreeBSD's `kern_kill` and `killpg1` for the only guest process: `EINVAL` for signals outside 0 to 128, signal 0 only checks that the target exists, and its own pid, 0 and minus its own pid (the guest is taken to lead its own process group) signal it through `raise`, so they share `raise`'s limits above. Any other pid, and -1, which skips the caller, report `ESRCH`, so `EPERM` is never reported - [sceKernelFcntl](../../core/libs/prx/libkernel/File/src/Open.cpp) (libkernel) handles socket descriptors through the POSIX `fcntl` export (`F_GETFL`, `F_SETFL` with `O_NONBLOCK`) and returns its guest errno as `SCE_KERNEL_ERROR_*`, the form the other `sceKernel` file exports use; the command semantics follow FreeBSD's [fcntl](https://github.com/freebsd/freebsd-src/blob/stable/11/sys/kern/kern_descrip.c), and a closed descriptor returns `SCE_KERNEL_ERROR_EBADF`. File descriptors throw: their open flags are not tracked, and `F_DUPFD`, `F_GETFD` and `F_SETFD` are unsupported on all descriptors. The return values and error codes are unverified on a console From d102e186c97b517d6a846bbd53c140b9d3c3d3bf Mon Sep 17 00:00:00 2001 From: Badr <168379979+Jnbla6@users.noreply.github.com> Date: Sun, 11 Oct 2026 09:41:09 +0300 Subject: [PATCH 404/460] fix(relinker): print the Windows stub's ' -> FAILED' only with dependency diagnostics --- .../elfpatcher/src/windows/WindowsEntryStubBuilder.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp index 6406e8e15..b158d671e 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsEntryStubBuilder.cpp @@ -391,7 +391,9 @@ WindowsEntryStub WindowsEntryStubBuilder::Build(const std::uint32_t dataRva, con code.Emit({0x48, 0x85, 0xc0}); const auto loadSucceeded = code.Branch({0x0f, 0x85}); captureLastError(); - writeString(loadFailed, true); + if (dependencyDiagnostics) { + writeString(loadFailed, true); + } writeString(failedModule, true); writeString(resolvedPaths[index], true); writeString(newline, true); From 73e0890c20935e453ff59da1a51aa627dc0c3da0 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Fri, 9 Oct 2026 07:45:47 -0500 Subject: [PATCH 405/460] fix(shader): read GLC/DLC BDA dwords with device-scope atomic loads Coherent (GLC/DLC) global loads were emitted as Volatile loads through physical pointers. Volatile does not make another workgroup's stores visible within the same dispatch, so a workgroup spinning on a flag written by another one (decoupled look-back scans, as in RADV's acceleration-structure build) never saw it on NVIDIA. Aligned coherent dword reads now use OpAtomicLoad at device scope; the byte path keeps the old reads for unaligned addresses. AI-assisted (Claude Code). Co-Authored-By: Claude Opus 5.5 --- .../tests/execution/CrossWorkgroupFlag.cpp | 96 +++++++++++++++++++ .../SpirvBackend/src/SpirvBdaRead.cpp | 33 ++++++- 2 files changed, 127 insertions(+), 2 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/CrossWorkgroupFlag.cpp diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/CrossWorkgroupFlag.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/CrossWorkgroupFlag.cpp new file mode 100644 index 000000000..10da1c562 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/CrossWorkgroupFlag.cpp @@ -0,0 +1,96 @@ +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "prx/libc/include/GuestAllocations.hpp" +#include "Recompiler.hpp" +#include "VulkanTestDevice.hpp" +#ifdef _WIN32 +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include +#endif +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +constexpr std::size_t BlockBytes = 65536; + +alignas(256) constexpr std::array CrossWorkgroupFlagCode{ + 0xbe840380, 0x7e000280, 0xbf068002, 0xbf840004, 0x7e020281, 0xdc718000, 0x00000100, 0xbf810000, + 0xdc319000, 0x01000000, 0xbf8c3f70, 0x7d840280, 0x80048104, 0xbf0aff04, 0x000f4240, 0xbf840005, + 0xbf87fff7, 0x7e020282, 0xdc718004, 0x00000100, 0xbf810000, 0x7e020283, 0xdc718004, 0x00000100, + 0xbf810000, +}; + +class GuestBlock { +public: + GuestBlock() { +#ifdef _WIN32 + block = static_cast(VirtualAlloc(nullptr, BlockBytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); +#else + block = static_cast(std::aligned_alloc(BlockBytes, BlockBytes)); +#endif + Require(block != nullptr, "cross-workgroup flag: cannot allocate the guest block"); + GuestAllocations::Mutation().Add(block, BlockBytes, true, true); + } + ~GuestBlock() { GuestAllocations::Mutation().Remove(block); } + GuestBlock(const GuestBlock&) = delete; + GuestBlock& operator=(const GuestBlock&) = delete; + std::uint8_t* Data() { return block; } + +private: + std::uint8_t* block = nullptr; +}; + +std::uint32_t Run(AgcDriver::VulkanDevice& device, GuestBlock& guest, std::uint32_t waveSize, std::uint32_t preset) { + std::memset(guest.Data(), 0, BlockBytes); + std::memcpy(guest.Data(), &preset, sizeof(preset)); + const auto address = static_cast(reinterpret_cast(guest.Data())); + const std::vector userData{static_cast(address), static_cast(address >> 32u)}; + const std::span code(CrossWorkgroupFlagCode); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{64, 1, 1}, 0, {true, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {waveSize, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 2, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); + std::uint32_t words[2]{}; + std::memcpy(words, guest.Data(), sizeof(words)); + std::printf("wave%u: flag %u, result %u\n", waveSize, words[0], words[1]); + return words[1]; +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + GuestBlock guest; + Require(Run(*device, guest, 64, 1) == 2u && Run(*device, guest, 32, 1) == 2u, "a workgroup did not see a flag the CPU stored before the dispatch"); + const auto wave64 = Run(*device, guest, 64, 0); + const auto wave32 = Run(*device, guest, 32, 0); + Require(wave64 == 2u && wave32 == 2u, "a workgroup did not see another workgroup's flag store (2 = seen, 3 = gave up)"); + std::puts("cross-workgroup flag tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvBdaRead.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvBdaRead.cpp index 371d7fc06..f07e159ee 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvBdaRead.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvBdaRead.cpp @@ -173,7 +173,8 @@ std::uint32_t DefineBdaSpanReadFunction(SpirvEmitterState& state, bool coherent) const auto pointer = state.module.AllocateId(); state.module.AddFunction(spv::OpConvertUToPtr, TypePhysicalU32Pointer(state), pointer, Binary(state, spv::OpIAdd, u64, physical, delta)); const auto value = state.module.AllocateId(); - state.module.AddFunction(spv::OpLoad, u32, value, pointer, BdaAccessMask(coherent), 4u); + if (coherent) state.module.AddFunction(spv::OpAtomicLoad, u32, value, pointer, ConstantU32(state, spv::ScopeDevice), ConstantU32(state, 0u)); + else state.module.AddFunction(spv::OpLoad, u32, value, pointer, BdaAccessMask(coherent), 4u); return value; }); } @@ -234,7 +235,35 @@ void DefineBdaDwordReadFunctions(SpirvEmitterState& state) { EmitLabel(state, state.module.AllocateId()); state.bdaStopsInvocations = stop; EmitBdaOverflowCheck(state, address, 4u, instruction); - const auto value = EmitBdaBytes(state, address, 4u, instruction, BdaAccessMask(coherent)); + std::uint32_t value = 0; + if (coherent && state.bdaProbeFunction != 0) { + const auto u64t = TypeScalarU64(state); + const auto physical = state.module.AllocateId(); + state.module.AddFunction(spv::OpFunctionCall, u64t, physical, state.bdaProbeFunction, address, ConstantU32(state, 4u), instruction); + const auto usable = Binary(state, spv::OpLogicalAnd, TypeBool(state), Binary(state, spv::OpINotEqual, TypeBool(state), physical, BdaConstant(state, 0u)), Binary(state, spv::OpIEqual, TypeBool(state), Binary(state, spv::OpBitwiseAnd, u64t, physical, BdaConstant(state, 3u)), BdaConstant(state, 0u))); + const auto atomicLabel = state.module.AllocateId(); + const auto byteLabel = state.module.AllocateId(); + const auto byteExit = state.module.AllocateId(); + const auto merge = state.module.AllocateId(); + state.module.AddFunction(spv::OpSelectionMerge, merge, spv::SelectionControlMaskNone); + state.module.AddFunction(spv::OpBranchConditional, usable, atomicLabel, byteLabel); + EmitLabel(state, atomicLabel); + const auto pointer = state.module.AllocateId(); + state.module.AddFunction(spv::OpConvertUToPtr, TypePhysicalU32Pointer(state), pointer, physical); + const auto atomicValue = state.module.AllocateId(); + state.module.AddFunction(spv::OpAtomicLoad, u32, atomicValue, pointer, ConstantU32(state, spv::ScopeDevice), ConstantU32(state, 0u)); + state.module.AddFunction(spv::OpBranch, merge); + EmitLabel(state, byteLabel); + const auto byteValue = EmitBdaBytes(state, address, 4u, instruction, BdaAccessMask(coherent)); + state.module.AddFunction(spv::OpBranch, byteExit); + EmitLabel(state, byteExit); + state.module.AddFunction(spv::OpBranch, merge); + EmitLabel(state, merge); + value = state.module.AllocateId(); + state.module.AddFunction(spv::OpPhi, u32, value, atomicValue, atomicLabel, byteValue, byteExit); + } else { + value = EmitBdaBytes(state, address, 4u, instruction, BdaAccessMask(coherent)); + } const auto result = state.module.AllocateId(); state.module.AddFunction(spv::OpCompositeConstruct, pair, result, value, ConstantU32(state, 0u)); state.module.AddFunction(spv::OpReturnValue, result); From 62d6cbe42dc7a4e1735b6a82e721aa242f94f2aa Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Sun, 11 Oct 2026 16:41:25 +0900 Subject: [PATCH 406/460] test(relinker): check that a Windows load failure starts with its own message Co-Authored-By: Claude Opus 5.5 --- core/relinker/relinker/tests/test_windows_import_modules.py | 1 + 1 file changed, 1 insertion(+) diff --git a/core/relinker/relinker/tests/test_windows_import_modules.py b/core/relinker/relinker/tests/test_windows_import_modules.py index 0eeaabd36..c582d5d4b 100644 --- a/core/relinker/relinker/tests/test_windows_import_modules.py +++ b/core/relinker/relinker/tests/test_windows_import_modules.py @@ -329,6 +329,7 @@ def main(): if os.name == 'nt': run = subprocess.run([str(output)], capture_output=True, text=True, timeout=30) assert run.returncode != 0 and 'Failed to load module:' in run.stderr and 'missing.prx' in run.stderr, (run.returncode, run.stderr) + assert run.stderr.startswith('Failed to load module:'), run.stderr for index, (symbol, module_name) in enumerate([('shared#A#B', None), ('shared', None), ('shared#A#B', 'libSceJson')]): case = work / ('missing-unused-dependency-' + str(index)) From 5680c323815d511088e42e5a2aa95dade77a93b8 Mon Sep 17 00:00:00 2001 From: fma-rc <188743521+fma-rc@users.noreply.github.com> Date: Sun, 11 Oct 2026 09:41:52 +0200 Subject: [PATCH 407/460] fix(font): throw where the writing line error codes come from the PS4 module only and accept attribute 0 --- core/libs/prx/libSceFont/src/WritingLine.cpp | 23 ++++++-------------- core/libs/tests/GuestFontWritingLine.cpp | 22 +++++++++---------- docs/dev/TechnicalDebt.md | 6 ++--- 3 files changed, 20 insertions(+), 31 deletions(-) diff --git a/core/libs/prx/libSceFont/src/WritingLine.cpp b/core/libs/prx/libSceFont/src/WritingLine.cpp index 5a91c6b27..35e014c3c 100644 --- a/core/libs/prx/libSceFont/src/WritingLine.cpp +++ b/core/libs/prx/libSceFont/src/WritingLine.cpp @@ -246,18 +246,19 @@ int APS5_VABI sceFontDestroyWritingLine(void** pWritingLine) { } int APS5_VABI sceFontWritingLineClear(void* writingLine) { - if (!writingLine) return SCE_FONT_ERROR_INVALID_PARAMETER; auto* line = GetLine(__func__, writingLine); ResetOrders(*line); return SCE_FONT_OK; } int APS5_VABI sceFontWritingLineWritesOrder(void* writingLine, std::uint64_t writingAttribute, const FontWritingMetrics* writingMetrics, void* writingOrderer) { - if (!writingLine || !writingOrderer) return SCE_FONT_ERROR_INVALID_PARAMETER; - if ((writingAttribute & WRITING_ATTRIBUTE_ID_MASK) != WRITING_ATTRIBUTE_ID) return SCE_FONT_ERROR_INVALID_PARAMETER; - const std::size_t level = (writingAttribute >> 8) & 0xFF; - if (level >= WRITING_LEVEL_COUNT) return SCE_FONT_ERROR_INVALID_PARAMETER; auto* line = GetLine(__func__, writingLine); + if (writingAttribute == 0) writingAttribute = WRITING_ATTRIBUTE_ID; + if ((writingAttribute & WRITING_ATTRIBUTE_ID_MASK) != WRITING_ATTRIBUTE_ID) + Unsupported(__func__, "writing attribute " + std::to_string(writingAttribute) + " is not modelled"); + const std::size_t level = (writingAttribute >> 8) & 0xFF; + if (level >= WRITING_LEVEL_COUNT) Unsupported(__func__, "embedding level " + std::to_string(level) + " (error code not verified)"); + if (!writingOrderer) Unsupported(__func__, "null writing orderer (error code not verified)"); if (line->cursor) ResetOrders(*line); WritingOrder* order = &line->orders.emplace_back(); order->orderer = writingOrderer; @@ -318,23 +319,13 @@ const FontWritingLineStep* APS5_VABI sceFontWritingLineRefersRenderStep(void* wr } int APS5_VABI sceFontWritingLineGetRenderMetrics(void* writingLine, FontWritingMetrics* writingMetrics) { - if (!writingLine || !writingMetrics) { - if (writingMetrics) *writingMetrics = {}; - return SCE_FONT_ERROR_INVALID_PARAMETER; - } auto* line = GetLine(__func__, writingLine); + if (!writingMetrics) Unsupported(__func__, "null metrics pointer (error code not verified)"); *writingMetrics = line->renderMetrics; return SCE_FONT_OK; } int APS5_VABI sceFontWritingLineGetOrderingSpace(void* writingLine, float* advanceSpace, float* headSpace, float* inlineSpace, float* tailSpace) { - if (!writingLine) { - if (advanceSpace) *advanceSpace = 0.0f; - if (headSpace) *headSpace = 0.0f; - if (inlineSpace) *inlineSpace = 0.0f; - if (tailSpace) *tailSpace = 0.0f; - return SCE_FONT_ERROR_INVALID_PARAMETER; - } auto* line = GetLine(__func__, writingLine); if (headSpace && !tailSpace) Unsupported(__func__, "a head space pointer without a tail space pointer is not modelled"); if (advanceSpace) *advanceSpace = line->advanceX; diff --git a/core/libs/tests/GuestFontWritingLine.cpp b/core/libs/tests/GuestFontWritingLine.cpp index 4697f74ab..7b40b889c 100644 --- a/core/libs/tests/GuestFontWritingLine.cpp +++ b/core/libs/tests/GuestFontWritingLine.cpp @@ -108,14 +108,14 @@ int main() { const FontWritingMetrics runC{8.0f, 0.0f, {-4.0f, 1.0f, 2.0f, 5.0f}}; const FontWritingMetrics runD{4.0f, 0.0f, {-4.0f, 1.0f, 0.0f, 20.0f}}; - Require(sceFontWritingLineWritesOrder(nullptr, Attribute(0), &runA, &a) == SCE_FONT_ERROR_INVALID_PARAMETER); - Require(sceFontWritingLineWritesOrder(line, 0, &runA, &a) == SCE_FONT_ERROR_INVALID_PARAMETER); - Require(sceFontWritingLineWritesOrder(line, 0x1FB, &runA, &a) == SCE_FONT_ERROR_INVALID_PARAMETER); - Require(sceFontWritingLineWritesOrder(line, Attribute(0), &runA, nullptr) == SCE_FONT_ERROR_INVALID_PARAMETER); - Require(sceFontWritingLineWritesOrder(line, Attribute(0x7F), &runA, &a) == SCE_FONT_ERROR_INVALID_PARAMETER); + Require(Throws([&] { sceFontWritingLineWritesOrder(nullptr, Attribute(0), &runA, &a); })); + Require(Throws([&] { sceFontWritingLineWritesOrder(line, 1, &runA, &a); })); + Require(Throws([&] { sceFontWritingLineWritesOrder(line, 0x1FB, &runA, &a); })); + Require(Throws([&] { sceFontWritingLineWritesOrder(line, Attribute(0), &runA, nullptr); })); + Require(Throws([&] { sceFontWritingLineWritesOrder(line, Attribute(0x7F), &runA, &a); })); Require(sceFontWritingLineRefersRenderStep(line) == nullptr); - Require(sceFontWritingLineWritesOrder(line, Attribute(0), &runA, &a) == SCE_FONT_OK); + Require(sceFontWritingLineWritesOrder(line, 0, &runA, &a) == SCE_FONT_OK); Require(sceFontWritingLineWritesOrder(line, Attribute(0), &runB, &b) == SCE_FONT_OK); Require(SpaceIs(line, 16.0f, 1.0f, 0.0f, 1.0f)); Require(sceFontWritingLineGetRenderMetrics(line, &metrics) == SCE_FONT_OK && Same(metrics, {})); @@ -221,12 +221,10 @@ int main() { float space = 7.0f; Require(sceFontWritingLineGetOrderingSpace(rtl, nullptr, nullptr, &space, nullptr) == SCE_FONT_OK && space == 15.0f); Require(Throws([&] { sceFontWritingLineGetOrderingSpace(rtl, &space, &space, &space, nullptr); })); - space = 7.0f; - Require(sceFontWritingLineGetOrderingSpace(nullptr, &space, nullptr, nullptr, nullptr) == SCE_FONT_ERROR_INVALID_PARAMETER && space == 0.0f); - metrics = runA; - Require(sceFontWritingLineGetRenderMetrics(nullptr, &metrics) == SCE_FONT_ERROR_INVALID_PARAMETER && Same(metrics, {})); - Require(sceFontWritingLineGetRenderMetrics(rtl, nullptr) == SCE_FONT_ERROR_INVALID_PARAMETER); - Require(sceFontWritingLineClear(nullptr) == SCE_FONT_ERROR_INVALID_PARAMETER); + Require(Throws([&] { sceFontWritingLineGetOrderingSpace(nullptr, &space, nullptr, nullptr, nullptr); })); + Require(Throws([&] { sceFontWritingLineGetRenderMetrics(nullptr, &metrics); })); + Require(Throws([&] { sceFontWritingLineGetRenderMetrics(rtl, nullptr); })); + Require(Throws([&] { sceFontWritingLineClear(nullptr); })); int notALine = 0; Require(sceFontWritingLineRefersRenderStep(nullptr) == nullptr); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index ad20b74a8..6bf58ab25 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -105,8 +105,8 @@ Throughout the project, every function at every stage either **does exactly what - [sceFontDettachDeviceCacheBuffer](../../core/libs/prx/libSceFont/src/Library.cpp) (libSceFont) - signature taken from the IDA type database of [Orbital](https://github.com/AlexAltea/orbital/blob/master/tools/ida/db_types.json), not confirmed on a title. It detaches the cache buffer attached by `sceFontAttachDeviceCacheBuffer`, returns the buffer and its size, and reports `SCE_FONT_ERROR_NOT_ATTACHED_CACHE_BUFFER` when no cache buffer is attached - [sceFontGetFontGlyphsCount, sceFontGetCharGlyphCode](../../core/libs/prx/libSceFont/src/Open.cpp) (libSceFont) - signatures taken from the IDA type database of [Orbital](https://github.com/AlexAltea/orbital/blob/master/tools/ida/db_types.json) and matching the [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.Engine.cs) bindings, not confirmed on a title. The count is the face's glyph count and the glyph code its glyph index, as the library's driver reports them; a code with no glyph returns `NO_SUPPORT_GLYPH` and code 0 `NO_SUPPORT_CODE`, as `sceFontRenderCharGlyphImage` does. The argument checks and the cleared output follow `sceFontGetPixelResolution` - [sceFontGetFontResolution](../../core/libs/prx/libSceFont/src/Open.cpp) (libSceFont) - signature taken from the IDA type database of [Orbital](https://github.com/AlexAltea/orbital/blob/master/tools/ida/db_types.json), not confirmed on a title; the outputs are inferred from the names. The resolution is the face's units per EM and the scale pixel the driver's face scale (units per EM / 64 for FreeType), the pixel scale a font has when opened. Either output may be null, as in `sceFontGetResolutionDpi` -- [sceFontCreateWritingLine, sceFontWritingLineWritesOrder](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - signatures and the `SceFontWritingLineStep`/`SceFontCreateWritingLineDetail` layouts (detail id `0x0FD5`) taken from [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFontStructs.cs), not confirmed on a title. The 64-bit writing attribute is read as the PS4 11.00 `libSceFont.sprx` reads it: low byte `0xFA`, embedding level in bits 8-15 (0 to 126), bit 30 and bit 31 marking space before and after the run; the PS5 library was not measured, and the names of the two bits are ours -- [sceFontWritingLineGetOrderingSpace](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - the four outputs are written in the order the PS4 11.00 `libSceFont.sprx` writes them (line advance, leading space, space between spaced runs, trailing space); the parameter names in the [Orbital](https://github.com/AlexAltea/orbital/blob/master/tools/ida/db_types.json) IDA type database and the [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.Engine.cs) bindings order them differently, and the PS5 library was not measured. The PS4 library writes the trailing space through its pointer without a null check whenever the leading-space pointer is set; that call throws here +- [sceFontCreateWritingLine, sceFontWritingLineWritesOrder](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - signatures and the `SceFontWritingLineStep`/`SceFontCreateWritingLineDetail` layouts (detail id `0x0FD5`) taken from [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFontStructs.cs), not confirmed on a title. The 64-bit writing attribute is read as the PS4 11.00 `libSceFont.sprx` reads it: low byte `0xFA`, embedding level in bits 8-15 (0 to 126), bit 30 and bit 31 marking space before and after the run; the PS5 library was not measured, and the names of the two bits are ours. PPSA01325 builds its attribute the same way (`0xFA | level << 8`, bits 30 and 31) and passes a null metrics pointer when its run has no metrics object. Attribute 0 is accepted and treated as `0xFA` with level 0 and no spacing bits, as before this reading; the PS4 library rejects it and what the PS5 library does with it is unknown. Any other attribute, a level above 126 and a null orderer throw: the PS4 library returns `0x80460002` for them, not confirmed for the PS5 +- [sceFontWritingLineGetOrderingSpace](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - the four outputs are written in the order the PS4 11.00 `libSceFont.sprx` writes them (line advance, leading space, space between spaced runs, trailing space); the parameter names in the [Orbital](https://github.com/AlexAltea/orbital/blob/master/tools/ida/db_types.json) IDA type database and the [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.Engine.cs) bindings order them differently (head, inline, tail, advance), and the PS5 library was not measured. PPSA01325's only call uses them in the PS4 order: it computes first - second - fourth - 0.25 * third and compares that, divided by a scale, with its width limit. The PS4 library writes the trailing space through its pointer without a null check whenever the leading-space pointer is set; that call throws here - [sceSaveDataTransferringMountPs4, sceSaveDataDirNameSearchPs4](../../core/libs/prx/libSceSaveData.native/Export.cpp) (libSceSaveData.native) - arguments assumed to be those of `sceSaveDataTransferringMount` and `sceSaveDataDirNameSearch`: KytyPS5 binds both NIDs to the PS5 functions, and [prosper](https://github.com/mattias800/prosper) captured the same result layout on a title for the search. There is no PS4 save area, so the mount returns `NOT_FOUND` and the search reports zero hits - [sceCoredumpWriteUserString](../../core/libs/prx/libkernel/System/src/Coredump.cpp) and [sceSaveDataConvert, sceSaveDataGetConvertProgress](../../core/libs/prx/libSceSaveData.native/Export.cpp) - imported by PPSA16064, not called during its start-up and added without an implementation; unknown signatures - [sceSaveDataCancel](../../core/libs/prx/libSceSaveData.native/Export.cpp) (libSceSaveData.native) - imported by PPSA23126, which fails to load without it; added without an implementation because its signature is unknown. Every save data call here finishes before it returns, so nothing is ever in flight to cancel, but what the console returns then is unknown @@ -492,7 +492,7 @@ Throughout the project, every function at every stage either **does exactly what - [Thread exit destructors](../../core/libs/prx/libSceLibcInternal/Export.cpp) (libSceLibcInternal) run when a libkernel thread finishes; on the main or a host thread only `_sceLibcInternalThreadDtors` runs them, as `exit` does, else they are dropped - [sceFontOpenFontSet](../../core/libs/prx/libSceFont/src/SystemFont.cpp) (libSceFont) opens only the primary file of a system font set. Characters the console takes from the set's other files (Hangul, Thai and Arabic in Japanese and Chinese sets, Arabic in European sets) are reported as unsupported glyphs. The console file names are those of the PS4 firmware; the names of the PS5-only JG2 sets (`0x1A......`) are unknown, so they use the SSTJpPro files. When a console file is missing, an openly licensed substitute (Noto Sans, Noto Sans Mono, Noto Sans Thai, Noto Sans CJK) is loaded instead. It is not the system font: its metrics differ, so text width, line height and wrapping differ from the console. System font faces are loaded by FreeType in host memory, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. - [sceFontGetCharGlyphMetrics, sceFontGetRenderCharGlyphMetrics](../../core/libs/prx/libSceFont/src/Layout.cpp) (libSceFont) report the vertical metrics the way the [FreeType driver](../../core/libs/prx/libSceFontFt/src/Driver.cpp) reports them for a rendered glyph, not confirmed on a title: FreeType's vertical advance, and for a face without vertical metrics a bearing of half the horizontal advance left of the horizontal bearing and of half the advance left over by the glyph height. -- [Font writing lines](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - `sceFontWritingLineWritesOrder`, `sceFontWritingLineRefersRenderStep`, `sceFontWritingLineGetRenderMetrics`, `sceFontWritingLineGetOrderingSpace` and `sceFontWritingLineClear` follow the PS4 11.00 `libSceFont.sprx` (firmware module, read from its disassembly and compared call for call on random sequences): runs are linked in visual order by embedding level, the space at each spaced join is summed for the line, steps walk the visual order from the left (form `0x12`) or from the right (form `0x11`) and the render metrics grow as steps are consumed. The PS5 library was not measured. Not from that module: form `0x10` is accepted and treated as left to right and a detail with id `0x0FD5` is accepted (the PS4 library rejects both); the memory argument of `sceFontCreateWritingLine` and the rest of the detail are not read (the PS4 library validates its library object there); an unknown writing form, another detail id, a null line pointer on creation and unknown handles throw instead of returning the PS4 error codes; `sceFontDestroyWritingLine` was not compared. Lines are allocated on the host, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. +- [Font writing lines](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - `sceFontWritingLineWritesOrder`, `sceFontWritingLineRefersRenderStep`, `sceFontWritingLineGetRenderMetrics`, `sceFontWritingLineGetOrderingSpace` and `sceFontWritingLineClear` follow the PS4 11.00 `libSceFont.sprx` (firmware module, read from its disassembly and compared call for call on random sequences): runs are linked in visual order by embedding level, the space at each spaced join is summed for the line, steps walk the visual order from the left (form `0x12`) or from the right (form `0x11`) and the render metrics grow as steps are consumed. The PS5 library was not measured. Not from that module: form `0x10` is accepted and treated as left to right and a detail with id `0x0FD5` is accepted (the PS4 library rejects both); the memory argument of `sceFontCreateWritingLine` and the rest of the detail are not read (the PS4 library validates its library object there); an unknown writing form, another detail id, a null line pointer on creation, unknown and null handles (except in `sceFontWritingLineRefersRenderStep`, which returns null for a null line) and a null output of `sceFontWritingLineGetRenderMetrics` throw instead of returning the PS4 error codes, which are not confirmed for the PS5; `sceFontDestroyWritingLine` was not compared. Lines are allocated on the host, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. - [s_swappc_b64](../../core/shader/recompiler/Translation/src/ControlFlowInstructions.cpp) writes the link pair with the same model as `s_getpc_b64`: the guest return address `pc + 4` as a byte offset over `GetShaderBase`. Any other use of the link pair (arithmetic, compare, hash, store, or a second `s_swappc_b64` sharing it) throws; constant-offset folding over the link is not modeled. The special encoding `s_swappc_b64` with SDST 125 still decodes as `s_setpc_b64`, which matches the hardware: measured on RDNA2 (gfx1036) in wave32 and wave64, it branches to the source pair and leaves a marker SGPR pair untouched, and with an SGPR destination the link is `PC + 4`. - The [s_swappc_b64 fetch-shader call at vertex/local/tessellation/mesh entry](../../core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp) assumes the called fetch-shader blob ends in `s_setpc_b64` to the link pair (the GNM fetch-shader shape shadPS4's `EmitFetch` copies); the blob is not part of the shader program, so its return closes the pairing by assumption and any in-program reader of the link pair throws. Attribute fetches are inlined from the request's parsed fetch-shader plan through the same `GetAttribute` path as the embedded fetch analyzer, honoring the buffer resource `dst_sel`; an attribute writing more than four registers, or an entry call without a parsed plan, throws. The vertex/instance offset accumulator inside the fetch blob is not replayed for external blobs. - The scalar aperture sources [src_shared_base, src_shared_limit, src_private_base, src_private_limit](../../core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp) (core/shader/recompiler) decode to their own operand kinds (scalar source codes 235 to 238, named as in LLVM's `SIRegisterInfo.td`), but their value is not modelled: reading one refuses with the source it refused named, because the wave's shared and private aperture bases and limits are unknown. Both the scalar-code read and the raw-operand read refuse; a shader that only carries the value into an address still prepares, one that reads it does not From 5890160278ccceb703f23726a1aee46093023578 Mon Sep 17 00:00:00 2001 From: pushkarreddyy <212133856+pushkarreddyy@users.noreply.github.com> Date: Sat, 10 Oct 2026 23:37:02 +0530 Subject: [PATCH 408/460] feat(libkernel): implement sceKernelGetSystemSwVersion (cherry picked from commit 7143d0b46e389254b0b3bd52812a196ace0c7a6b) --- .../System/src/SystemConfiguration.cpp | 23 +++++++++++++ core/libs/tests/GuestSystemConfiguration.cpp | 34 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 58 insertions(+) diff --git a/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp b/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp index 3b245aa97..242f25749 100644 --- a/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp +++ b/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp @@ -15,6 +15,13 @@ extern "C" int* APS5_VABI __error_nid_postfix(); +struct SceKernelSystemSwVersion { + std::uint32_t size; + char versionString[0x1C]; + std::uint32_t version; + std::uint32_t unk_24; +}; + namespace { int CopyOut(const void* value, std::size_t size, void* oldValue, std::size_t* oldLength) { @@ -106,4 +113,20 @@ int APS5_VABI sysctlbyname_nid_postfix(const char* name, void* oldValue, std::si static constexpr int mib[] = {6, 3}; return sysctl_nid_postfix(mib, 2, oldValue, oldLength, newValue, newLength); } + +int APS5_VABI sceKernelGetSystemSwVersion_nid_postfix(SceKernelSystemSwVersion* version) { + if (version == nullptr) { + *__error_nid_postfix() = 14; + return -1; + } + if (version->size != sizeof(SceKernelSystemSwVersion)) { + *__error_nid_postfix() = 22; + return -1; + } + std::memset(version->versionString, 0, sizeof(version->versionString)); + std::memcpy(version->versionString, "01.000.000", sizeof("01.000.000")); + version->version = 0x01000000; + version->unk_24 = 0; + return 0; +} } diff --git a/core/libs/tests/GuestSystemConfiguration.cpp b/core/libs/tests/GuestSystemConfiguration.cpp index aed9e28e3..a7bca8628 100644 --- a/core/libs/tests/GuestSystemConfiguration.cpp +++ b/core/libs/tests/GuestSystemConfiguration.cpp @@ -2,13 +2,21 @@ #include #include #include +#include #include +struct SceKernelSystemSwVersion { + std::uint32_t size; + char versionString[0x1C]; + std::uint32_t version; + std::uint32_t unk_24; +}; extern "C" { std::int64_t APS5_VABI sysconf_nid_postfix(int); int APS5_VABI getpagesize_nid_postfix(); int* APS5_VABI __error_nid_postfix(); int APS5_VABI sysctl_nid_postfix(const int*, std::uint32_t, void*, std::size_t*, const void*, std::size_t); int APS5_VABI sysctlbyname_nid_postfix(const char*, void*, std::size_t*, const void*, std::size_t); +int APS5_VABI sceKernelGetSystemSwVersion_nid_postfix(SceKernelSystemSwVersion*); extern char** environ_nid_postfix; } static void Require(bool value) { if (!value) std::abort(); } @@ -52,8 +60,34 @@ static void CheckProcessorCountSysctl() { } Require(threw); } +static void CheckSystemSwVersion() { + *__error_nid_postfix() = 13; + Require(sceKernelGetSystemSwVersion_nid_postfix(nullptr) == -1 && *__error_nid_postfix() == 14); + SceKernelSystemSwVersion version{}; + version.size = 0; + *__error_nid_postfix() = 13; + Require(sceKernelGetSystemSwVersion_nid_postfix(&version) == -1 && *__error_nid_postfix() == 22); + version.size = sizeof(SceKernelSystemSwVersion) - 1; + *__error_nid_postfix() = 13; + Require(sceKernelGetSystemSwVersion_nid_postfix(&version) == -1 && *__error_nid_postfix() == 22); + version.size = sizeof(SceKernelSystemSwVersion) + 1; + *__error_nid_postfix() = 13; + Require(sceKernelGetSystemSwVersion_nid_postfix(&version) == -1 && *__error_nid_postfix() == 22); + std::memset(&version, 0xaa, sizeof(version)); + version.size = sizeof(SceKernelSystemSwVersion); + *__error_nid_postfix() = 13; + Require(sceKernelGetSystemSwVersion_nid_postfix(&version) == 0 && *__error_nid_postfix() == 13); + Require(version.size == sizeof(SceKernelSystemSwVersion)); + Require(std::strcmp(version.versionString, "01.000.000") == 0); + for (std::size_t i = sizeof("01.000.000"); i < sizeof(version.versionString); ++i) { + Require(version.versionString[i] == 0); + } + Require(version.version == 0x01000000); + Require(version.unk_24 == 0); +} int main() { CheckProcessorCountSysctl(); + CheckSystemSwVersion(); *__error_nid_postfix() = 13; Require(sysconf_nid_postfix(47) == 0x4000); Require(getpagesize_nid_postfix() == sysconf_nid_postfix(47)); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 572482f0d..8e481b1bc 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -47,6 +47,7 @@ Throughout the project, every function at every stage either **does exactly what ### Unknown function info - [sceNpGetSandboxInfo](../../core/libs/prx/libSceNpManager/Export.cpp) (libSceNpManager, NID `ZVZ++68nNI8`) - the name hashes to the imported NID (found with nidhunt, confirmed with `tools/nid_names.py`). STAR WARS: Galactic Racer (PPSA28416) imports it; its OnlineSubsystemPS5 calls it with one pointer to a 0x50-byte stack buffer and reads the buffer as a string (discussions/3086). The returned string, the exact parameter struct and console errors are unknown, so every call throws without reading or writing caller memory. A working implementation belongs with that title's bring-up. +- [sceKernelGetSystemSwVersion](../../core/libs/prx/libkernel/System/src/SystemConfiguration.cpp) (libkernel) reports baseline system software version `0x01000000` ("01.000.000") for ProsperoStore (PPSA99000); console firmware version reporting and hardware behavior remain unverified - [sceVideoOutGetOutputStatusWithVrr](../../core/libs/prx/libSceVideoOut/Unimplemented.cpp) (libSceVideoOut, NID `gWT7X8H0bYs`) - the name hashes to the imported NID. PPSA26344 version 02.002.000 passes a VideoOut handle and an output pointer at three call sites. It compares a 32-bit value at byte 4 with 2, tests bit 0x10 at byte 16, and, when set, compares low 16-bit values at bytes 48, 50 and 52 with 48, 60 and 120. The full output layout, size, field meanings and console errors are unknown. Every call throws without reading or writing caller memory. - [getuid, geteuid, getgid, getegid and issetugid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) report the root identity (0) and a process that never changed credentials (`issetugid` 0), as a jailbroken process has; the host user and group are not exposed to the guest. The values a title sees on a console are unverified - [sceKernelGetAppInfo](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - only `app_id` and the title ID (offset 0x10, up to 9 characters plus a terminator) are written; callers declare the structure with different sizes, so the rest is left alone. `app_id` is 1, the running title's id in libSceSystemService, which is a guess, not something measured. The error codes follow shadPS4: `EPERM` for another process, and 0 without writing anything for a null pointer. From cbacc46edec9818c0d457142f43b46025986b90e Mon Sep 17 00:00:00 2001 From: pushkarreddyy <212133856+pushkarreddyy@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:26:06 +0530 Subject: [PATCH 409/460] fix(libkernel): return SCE_KERNEL_ERROR codes in sceKernelGetSystemSwVersion --- .../prx/libkernel/System/src/SystemConfiguration.cpp | 7 +++---- core/libs/tests/GuestSystemConfiguration.cpp | 9 +++++---- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp b/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp index 242f25749..1d169efe0 100644 --- a/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp +++ b/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp @@ -1,5 +1,6 @@ #include "prx/libc/include/general/VabiMacros.hpp" #include "prx/libkernel/DirectMemory/DirectMemory.hpp" +#include "prx/libkernel/KernelErrors.hpp" #include #include #include @@ -116,12 +117,10 @@ int APS5_VABI sysctlbyname_nid_postfix(const char* name, void* oldValue, std::si int APS5_VABI sceKernelGetSystemSwVersion_nid_postfix(SceKernelSystemSwVersion* version) { if (version == nullptr) { - *__error_nid_postfix() = 14; - return -1; + return SCE_KERNEL_ERROR_EFAULT; } if (version->size != sizeof(SceKernelSystemSwVersion)) { - *__error_nid_postfix() = 22; - return -1; + return SCE_KERNEL_ERROR_EINVAL; } std::memset(version->versionString, 0, sizeof(version->versionString)); std::memcpy(version->versionString, "01.000.000", sizeof("01.000.000")); diff --git a/core/libs/tests/GuestSystemConfiguration.cpp b/core/libs/tests/GuestSystemConfiguration.cpp index a7bca8628..0343c69f0 100644 --- a/core/libs/tests/GuestSystemConfiguration.cpp +++ b/core/libs/tests/GuestSystemConfiguration.cpp @@ -1,4 +1,5 @@ #include "prx/libc/include/general/VabiMacros.hpp" +#include "prx/libkernel/KernelErrors.hpp" #include #include #include @@ -62,17 +63,17 @@ static void CheckProcessorCountSysctl() { } static void CheckSystemSwVersion() { *__error_nid_postfix() = 13; - Require(sceKernelGetSystemSwVersion_nid_postfix(nullptr) == -1 && *__error_nid_postfix() == 14); + Require(sceKernelGetSystemSwVersion_nid_postfix(nullptr) == SCE_KERNEL_ERROR_EFAULT && *__error_nid_postfix() == 13); SceKernelSystemSwVersion version{}; version.size = 0; *__error_nid_postfix() = 13; - Require(sceKernelGetSystemSwVersion_nid_postfix(&version) == -1 && *__error_nid_postfix() == 22); + Require(sceKernelGetSystemSwVersion_nid_postfix(&version) == SCE_KERNEL_ERROR_EINVAL && *__error_nid_postfix() == 13); version.size = sizeof(SceKernelSystemSwVersion) - 1; *__error_nid_postfix() = 13; - Require(sceKernelGetSystemSwVersion_nid_postfix(&version) == -1 && *__error_nid_postfix() == 22); + Require(sceKernelGetSystemSwVersion_nid_postfix(&version) == SCE_KERNEL_ERROR_EINVAL && *__error_nid_postfix() == 13); version.size = sizeof(SceKernelSystemSwVersion) + 1; *__error_nid_postfix() = 13; - Require(sceKernelGetSystemSwVersion_nid_postfix(&version) == -1 && *__error_nid_postfix() == 22); + Require(sceKernelGetSystemSwVersion_nid_postfix(&version) == SCE_KERNEL_ERROR_EINVAL && *__error_nid_postfix() == 13); std::memset(&version, 0xaa, sizeof(version)); version.size = sizeof(SceKernelSystemSwVersion); *__error_nid_postfix() = 13; From e0e0b66cbbb37bc5b08c4608525f943cb7535cac Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sat, 10 Oct 2026 23:01:30 +0000 Subject: [PATCH 410/460] docs(libkernel): clarify system version ABI assumption --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 8e481b1bc..22a3b9dac 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -47,7 +47,7 @@ Throughout the project, every function at every stage either **does exactly what ### Unknown function info - [sceNpGetSandboxInfo](../../core/libs/prx/libSceNpManager/Export.cpp) (libSceNpManager, NID `ZVZ++68nNI8`) - the name hashes to the imported NID (found with nidhunt, confirmed with `tools/nid_names.py`). STAR WARS: Galactic Racer (PPSA28416) imports it; its OnlineSubsystemPS5 calls it with one pointer to a 0x50-byte stack buffer and reads the buffer as a string (discussions/3086). The returned string, the exact parameter struct and console errors are unknown, so every call throws without reading or writing caller memory. A working implementation belongs with that title's bring-up. -- [sceKernelGetSystemSwVersion](../../core/libs/prx/libkernel/System/src/SystemConfiguration.cpp) (libkernel) reports baseline system software version `0x01000000` ("01.000.000") for ProsperoStore (PPSA99000); console firmware version reporting and hardware behavior remain unverified +- [sceKernelGetSystemSwVersion](../../core/libs/prx/libkernel/System/src/SystemConfiguration.cpp) (libkernel) reports baseline system software version `0x01000000` ("01.000.000") for ProsperoStore (PPSA99000). The 0x28-byte layout (`size` at 0, `versionString` at 4, `version` at 0x20 and `unk_24` at 0x24) is an unverified assumption: [#2906](https://github.com/boykopovar/AnyPS5/issues/2906) proposes it but records no source for the field offsets. No title caller disassembly or console measurement is available. The guest test uses the same declaration as the implementation and therefore does not independently verify the ABI. Console firmware version reporting and hardware behavior remain unverified - [sceVideoOutGetOutputStatusWithVrr](../../core/libs/prx/libSceVideoOut/Unimplemented.cpp) (libSceVideoOut, NID `gWT7X8H0bYs`) - the name hashes to the imported NID. PPSA26344 version 02.002.000 passes a VideoOut handle and an output pointer at three call sites. It compares a 32-bit value at byte 4 with 2, tests bit 0x10 at byte 16, and, when set, compares low 16-bit values at bytes 48, 50 and 52 with 48, 60 and 120. The full output layout, size, field meanings and console errors are unknown. Every call throws without reading or writing caller memory. - [getuid, geteuid, getgid, getegid and issetugid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) report the root identity (0) and a process that never changed credentials (`issetugid` 0), as a jailbroken process has; the host user and group are not exposed to the guest. The values a title sees on a console are unverified - [sceKernelGetAppInfo](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - only `app_id` and the title ID (offset 0x10, up to 9 characters plus a terminator) are written; callers declare the structure with different sizes, so the rest is left alone. `app_id` is 1, the running title's id in libSceSystemService, which is a guess, not something measured. The error codes follow shadPS4: `EPERM` for another process, and 0 without writing anything for a null pointer. From d8ba169df81041283e14d09fa46dd882ff7a0abf Mon Sep 17 00:00:00 2001 From: MarcZX100 Date: Sun, 11 Oct 2026 01:16:06 +0000 Subject: [PATCH 411/460] fix(libkernel): correct system version ABI layout --- .../libs/prx/libkernel/System/src/SystemConfiguration.cpp | 4 +--- core/libs/tests/GuestSystemConfiguration.cpp | 8 +++++--- docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp b/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp index 1d169efe0..98a4a2111 100644 --- a/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp +++ b/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp @@ -17,10 +17,9 @@ extern "C" int* APS5_VABI __error_nid_postfix(); struct SceKernelSystemSwVersion { - std::uint32_t size; + std::uint64_t size; char versionString[0x1C]; std::uint32_t version; - std::uint32_t unk_24; }; namespace { @@ -125,7 +124,6 @@ int APS5_VABI sceKernelGetSystemSwVersion_nid_postfix(SceKernelSystemSwVersion* std::memset(version->versionString, 0, sizeof(version->versionString)); std::memcpy(version->versionString, "01.000.000", sizeof("01.000.000")); version->version = 0x01000000; - version->unk_24 = 0; return 0; } } diff --git a/core/libs/tests/GuestSystemConfiguration.cpp b/core/libs/tests/GuestSystemConfiguration.cpp index 0343c69f0..ea0fe752a 100644 --- a/core/libs/tests/GuestSystemConfiguration.cpp +++ b/core/libs/tests/GuestSystemConfiguration.cpp @@ -6,11 +6,14 @@ #include #include struct SceKernelSystemSwVersion { - std::uint32_t size; + std::uint64_t size; char versionString[0x1C]; std::uint32_t version; - std::uint32_t unk_24; }; +static_assert(offsetof(SceKernelSystemSwVersion, size) == 0x00); +static_assert(offsetof(SceKernelSystemSwVersion, versionString) == 0x08); +static_assert(offsetof(SceKernelSystemSwVersion, version) == 0x24); +static_assert(sizeof(SceKernelSystemSwVersion) == 0x28); extern "C" { std::int64_t APS5_VABI sysconf_nid_postfix(int); int APS5_VABI getpagesize_nid_postfix(); @@ -84,7 +87,6 @@ static void CheckSystemSwVersion() { Require(version.versionString[i] == 0); } Require(version.version == 0x01000000); - Require(version.unk_24 == 0); } int main() { CheckProcessorCountSysctl(); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 22a3b9dac..c0df68321 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -47,7 +47,7 @@ Throughout the project, every function at every stage either **does exactly what ### Unknown function info - [sceNpGetSandboxInfo](../../core/libs/prx/libSceNpManager/Export.cpp) (libSceNpManager, NID `ZVZ++68nNI8`) - the name hashes to the imported NID (found with nidhunt, confirmed with `tools/nid_names.py`). STAR WARS: Galactic Racer (PPSA28416) imports it; its OnlineSubsystemPS5 calls it with one pointer to a 0x50-byte stack buffer and reads the buffer as a string (discussions/3086). The returned string, the exact parameter struct and console errors are unknown, so every call throws without reading or writing caller memory. A working implementation belongs with that title's bring-up. -- [sceKernelGetSystemSwVersion](../../core/libs/prx/libkernel/System/src/SystemConfiguration.cpp) (libkernel) reports baseline system software version `0x01000000` ("01.000.000") for ProsperoStore (PPSA99000). The 0x28-byte layout (`size` at 0, `versionString` at 4, `version` at 0x20 and `unk_24` at 0x24) is an unverified assumption: [#2906](https://github.com/boykopovar/AnyPS5/issues/2906) proposes it but records no source for the field offsets. No title caller disassembly or console measurement is available. The guest test uses the same declaration as the implementation and therefore does not independently verify the ABI. Console firmware version reporting and hardware behavior remain unverified +- [sceKernelGetSystemSwVersion](../../core/libs/prx/libkernel/System/src/SystemConfiguration.cpp) (libkernel) reports baseline system software version `0x01000000` ("01.000.000") for ProsperoStore (PPSA99000). The 0x28-byte layout (`uint64_t size` at 0, `versionString` at 8 and `version` at 0x24) matches [ProsperoStore's caller](https://github.com/blackbearreloaded/ProsperoStore/blob/20584011/src/diag/trace.cpp#L20-L26), [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/811bfcd7d1/src/core/libraries/kernel/kernel.h#L45-L49), [fpPS4](https://github.com/red-prig/fpPS4/blob/04cefd43e6/kernel/ps4_libkernel.pas#L502-L506) and [OpenOrbis](https://github.com/OpenOrbis/OpenOrbis-PS4-Toolchain/blob/0a1aaf9dd4/include/orbis/_types/kernel.h#L256-L260). These sources support the structure layout, but not the reported version value: the console firmware version remains unverified. - [sceVideoOutGetOutputStatusWithVrr](../../core/libs/prx/libSceVideoOut/Unimplemented.cpp) (libSceVideoOut, NID `gWT7X8H0bYs`) - the name hashes to the imported NID. PPSA26344 version 02.002.000 passes a VideoOut handle and an output pointer at three call sites. It compares a 32-bit value at byte 4 with 2, tests bit 0x10 at byte 16, and, when set, compares low 16-bit values at bytes 48, 50 and 52 with 48, 60 and 120. The full output layout, size, field meanings and console errors are unknown. Every call throws without reading or writing caller memory. - [getuid, geteuid, getgid, getegid and issetugid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) report the root identity (0) and a process that never changed credentials (`issetugid` 0), as a jailbroken process has; the host user and group are not exposed to the guest. The values a title sees on a console are unverified - [sceKernelGetAppInfo](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - only `app_id` and the title ID (offset 0x10, up to 9 characters plus a terminator) are written; callers declare the structure with different sizes, so the rest is left alone. `app_id` is 1, the running title's id in libSceSystemService, which is a guess, not something measured. The error codes follow shadPS4: `EPERM` for another process, and 0 without writing anything for a null pointer. From 1583126a3566b2ed58166d509b812312832a5ad8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 05:42:41 +0200 Subject: [PATCH 412/460] docs(libkernel): record unverified sceKernelGetSystemSwVersion argument checks --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index c0df68321..c84cdf70c 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -47,7 +47,7 @@ Throughout the project, every function at every stage either **does exactly what ### Unknown function info - [sceNpGetSandboxInfo](../../core/libs/prx/libSceNpManager/Export.cpp) (libSceNpManager, NID `ZVZ++68nNI8`) - the name hashes to the imported NID (found with nidhunt, confirmed with `tools/nid_names.py`). STAR WARS: Galactic Racer (PPSA28416) imports it; its OnlineSubsystemPS5 calls it with one pointer to a 0x50-byte stack buffer and reads the buffer as a string (discussions/3086). The returned string, the exact parameter struct and console errors are unknown, so every call throws without reading or writing caller memory. A working implementation belongs with that title's bring-up. -- [sceKernelGetSystemSwVersion](../../core/libs/prx/libkernel/System/src/SystemConfiguration.cpp) (libkernel) reports baseline system software version `0x01000000` ("01.000.000") for ProsperoStore (PPSA99000). The 0x28-byte layout (`uint64_t size` at 0, `versionString` at 8 and `version` at 0x24) matches [ProsperoStore's caller](https://github.com/blackbearreloaded/ProsperoStore/blob/20584011/src/diag/trace.cpp#L20-L26), [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/811bfcd7d1/src/core/libraries/kernel/kernel.h#L45-L49), [fpPS4](https://github.com/red-prig/fpPS4/blob/04cefd43e6/kernel/ps4_libkernel.pas#L502-L506) and [OpenOrbis](https://github.com/OpenOrbis/OpenOrbis-PS4-Toolchain/blob/0a1aaf9dd4/include/orbis/_types/kernel.h#L256-L260). These sources support the structure layout, but not the reported version value: the console firmware version remains unverified. +- [sceKernelGetSystemSwVersion](../../core/libs/prx/libkernel/System/src/SystemConfiguration.cpp) (libkernel) reports baseline system software version `0x01000000` ("01.000.000") for ProsperoStore (PPSA99000). The 0x28-byte layout (`uint64_t size` at 0, `versionString` at 8 and `version` at 0x24) matches [ProsperoStore's caller](https://github.com/blackbearreloaded/ProsperoStore/blob/20584011/src/diag/trace.cpp#L20-L26), [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/811bfcd7d1/src/core/libraries/kernel/kernel.h#L45-L49), [fpPS4](https://github.com/red-prig/fpPS4/blob/04cefd43e6/kernel/ps4_libkernel.pas#L502-L506) and [OpenOrbis](https://github.com/OpenOrbis/OpenOrbis-PS4-Toolchain/blob/0a1aaf9dd4/include/orbis/_types/kernel.h#L256-L260). These sources support the structure layout, but not the reported version value: the console firmware version remains unverified. The checks are unverified too: a null pointer returns `SCE_KERNEL_ERROR_EFAULT` and a `size` other than 0x28 returns `SCE_KERNEL_ERROR_EINVAL`, while shadPS4 returns success for null and never reads `size`. - [sceVideoOutGetOutputStatusWithVrr](../../core/libs/prx/libSceVideoOut/Unimplemented.cpp) (libSceVideoOut, NID `gWT7X8H0bYs`) - the name hashes to the imported NID. PPSA26344 version 02.002.000 passes a VideoOut handle and an output pointer at three call sites. It compares a 32-bit value at byte 4 with 2, tests bit 0x10 at byte 16, and, when set, compares low 16-bit values at bytes 48, 50 and 52 with 48, 60 and 120. The full output layout, size, field meanings and console errors are unknown. Every call throws without reading or writing caller memory. - [getuid, geteuid, getgid, getegid and issetugid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) report the root identity (0) and a process that never changed credentials (`issetugid` 0), as a jailbroken process has; the host user and group are not exposed to the guest. The values a title sees on a console are unverified - [sceKernelGetAppInfo](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - only `app_id` and the title ID (offset 0x10, up to 9 characters plus a terminator) are written; callers declare the structure with different sizes, so the rest is left alone. `app_id` is 1, the running title's id in libSceSystemService, which is a guess, not something measured. The error codes follow shadPS4: `EPERM` for another process, and 0 without writing anything for a null pointer. From 082177e1b74ea168c84a1d0afc36c074c6f294d1 Mon Sep 17 00:00:00 2001 From: Cesar Rodriguez <11452991+Tamalero@users.noreply.github.com> Date: Sat, 10 Oct 2026 20:05:42 -0700 Subject: [PATCH 413/460] fix(libScePlayGo): accept chunk ids when the dump has no chunk definitions PS5 dumps carry no playgo-chunkdefs.xml (only sce_sys/playgo-scenario.json, which lists no chunks), so the valid set fell back to chunk 0 and every query for another chunk returned SCE_PLAYGO_ERROR_BAD_CHUNK_ID. A title that asks for all of its chunks in one scePlayGoGetLocus call then never sees them as installed and waits forever. Without definitions the chunk table is unknown and a dump is fully installed, so any chunk id is accepted and reported as local. With playgo-chunkdefs.xml present, validation is unchanged. scePlayGoGetChunkId still reports the known set. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libScePlayGo/Export.cpp | 30 ++++++++++++++++++--------- core/libs/tests/GuestPlayGo.cpp | 9 ++++++-- 2 files changed, 27 insertions(+), 12 deletions(-) diff --git a/core/libs/prx/libScePlayGo/Export.cpp b/core/libs/prx/libScePlayGo/Export.cpp index 58d081c3d..f95f2ad6c 100644 --- a/core/libs/prx/libScePlayGo/Export.cpp +++ b/core/libs/prx/libScePlayGo/Export.cpp @@ -27,11 +27,18 @@ static constexpr int SCE_PLAYGO_ERROR_BAD_LOCUS = static_cast(0x80B20010); // The package's chunk table is not part of the dump, so the chunk set comes from the title's // playgo-chunkdefs.xml: every listed chunk plus chunks 0 through the default chunk. Games probe // chunk IDs and size arrays from the result, so unknown IDs must be rejected. -static const std::set& ValidChunks() { - static const std::set chunks = [] { - std::set result{0}; +struct ChunkTable { + std::set ids{0}; + bool defined = false; +}; + +static const ChunkTable& Chunks() { + static const ChunkTable chunks = [] { + ChunkTable table; std::ifstream file(ResolvePath_nid_no_patch("/app0/playgo-chunkdefs.xml")); - if (!file) return result; + if (!file) return table; + table.defined = true; + auto& result = table.ids; const std::string text((std::istreambuf_iterator(file)), std::istreambuf_iterator()); static const std::regex chunk(R"re(& ValidChunks() { const auto last = std::stoul(match[1].str()); for (unsigned long id = 0; id <= last && id <= 0xFFFF; ++id) result.insert(static_cast(id)); } - return result; + return table; }(); return chunks; } +static bool IsKnownChunk(uint16_t id) { + const auto& chunks = Chunks(); + return !chunks.defined || chunks.ids.contains(id); +} + static int ValidateChunks(const uint16_t* chunk_ids, uint32_t number_of_entries) { if (!chunk_ids) return SCE_PLAYGO_ERROR_BAD_POINTER; if (number_of_entries == 0) return SCE_PLAYGO_ERROR_BAD_SIZE; - const auto& valid = ValidChunks(); for (uint32_t index = 0; index < number_of_entries; ++index) - if (!valid.contains(chunk_ids[index])) return SCE_PLAYGO_ERROR_BAD_CHUNK_ID; + if (!IsKnownChunk(chunk_ids[index])) return SCE_PLAYGO_ERROR_BAD_CHUNK_ID; return 0; } @@ -66,7 +77,7 @@ int APS5_VABI scePlayGoClose(int handle) { int APS5_VABI scePlayGoGetChunkId(int handle, uint16_t* out_chunk_id_list, uint32_t number_of_entries, uint32_t* out_entries) { if (handle != PLAYGO_HANDLE) return SCE_PLAYGO_ERROR_BAD_HANDLE; if (!out_entries) return SCE_PLAYGO_ERROR_BAD_POINTER; - const auto& valid = ValidChunks(); + const auto& valid = Chunks().ids; if (!out_chunk_id_list) { *out_entries = static_cast(valid.size()); return 0; @@ -183,9 +194,8 @@ int APS5_VABI scePlayGoSetToDoList(int handle, const PlayGoToDo* todo_list, uint if (handle != PLAYGO_HANDLE) return SCE_PLAYGO_ERROR_BAD_HANDLE; if (!todo_list) return SCE_PLAYGO_ERROR_BAD_POINTER; if (number_of_entries == 0) return SCE_PLAYGO_ERROR_BAD_SIZE; - const auto& valid = ValidChunks(); for (uint32_t index = 0; index < number_of_entries; ++index) { - if (!valid.contains(todo_list[index].chunk_id)) return SCE_PLAYGO_ERROR_BAD_CHUNK_ID; + if (!IsKnownChunk(todo_list[index].chunk_id)) return SCE_PLAYGO_ERROR_BAD_CHUNK_ID; const int8_t locus = todo_list[index].locus; if (locus != PLAYGO_LOCUS_NOT_DOWNLOADED && locus != PLAYGO_LOCUS_LOCAL_SLOW && locus != PLAYGO_LOCUS_LOCAL_FAST) return SCE_PLAYGO_ERROR_BAD_LOCUS; } diff --git a/core/libs/tests/GuestPlayGo.cpp b/core/libs/tests/GuestPlayGo.cpp index 84ee10f08..ba63ad1a8 100644 --- a/core/libs/tests/GuestPlayGo.cpp +++ b/core/libs/tests/GuestPlayGo.cpp @@ -11,6 +11,7 @@ int APS5_VABI scePlayGoGetLanguageMask(int, std::uint64_t*); int APS5_VABI scePlayGoSetToDoList(int, const PlayGoToDo*, std::uint32_t); int APS5_VABI scePlayGoGetToDoList(int, PlayGoToDo*, std::uint32_t, std::uint32_t*); int APS5_VABI scePlayGoGetLocus(int, const std::uint16_t*, std::uint32_t, std::int8_t*); +int APS5_VABI scePlayGoGetChunkId(int, std::uint16_t*, std::uint32_t, std::uint32_t*); } static void Require(bool value) { if (!value) std::abort(); } @@ -19,7 +20,6 @@ int main() { constexpr int badHandle = static_cast(0x80B20009); constexpr int badPointer = static_cast(0x80B2000A); constexpr int badSize = static_cast(0x80B2000B); - constexpr int badChunkId = static_cast(0x80B2000C); constexpr int badLocus = static_cast(0x80B20010); PlayGoInitParams init{}; @@ -42,7 +42,7 @@ int main() { todo[1].locus = 1; Require(scePlayGoSetToDoList(handle, todo, 2) == badLocus); todo[1] = {0xFFFF, 3, 0}; - Require(scePlayGoSetToDoList(handle, todo, 2) == badChunkId); + Require(scePlayGoSetToDoList(handle, todo, 2) == 0); std::uint32_t entries = 1; Require(scePlayGoGetToDoList(handle, todo, 2, &entries) == 0 && entries == 0); @@ -50,6 +50,11 @@ int main() { std::int8_t locus = 0; Require(scePlayGoGetLocus(handle, &chunk, 0, &locus) == badSize); Require(scePlayGoGetLocus(handle, &chunk, 1, &locus) == 0 && locus == 3); + const std::uint16_t undeclared[2] = {1, 10}; + std::int8_t loci[2] = {0, 0}; + Require(scePlayGoGetLocus(handle, undeclared, 2, loci) == 0 && loci[0] == 3 && loci[1] == 3); + std::uint32_t chunkCount = 0; + Require(scePlayGoGetChunkId(handle, nullptr, 0, &chunkCount) == 0 && chunkCount == 1); Require(scePlayGoClose(handle + 1) == badHandle); Require(scePlayGoClose(handle) == 0); } From db3c429e427c2d046889d7bb1d033df10e6f5720 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 07:45:06 +0200 Subject: [PATCH 414/460] docs(playgo): update chunk-set comment and record unchecked chunk ids in TechnicalDebt Co-Authored-By: Claude --- core/libs/prx/libScePlayGo/Export.cpp | 4 ++-- docs/dev/TechnicalDebt.md | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libScePlayGo/Export.cpp b/core/libs/prx/libScePlayGo/Export.cpp index f95f2ad6c..dfddd8c42 100644 --- a/core/libs/prx/libScePlayGo/Export.cpp +++ b/core/libs/prx/libScePlayGo/Export.cpp @@ -25,8 +25,8 @@ static constexpr int SCE_PLAYGO_ERROR_BAD_SIZE = static_cast(0x80B2000B); static constexpr int SCE_PLAYGO_ERROR_BAD_LOCUS = static_cast(0x80B20010); // The package's chunk table is not part of the dump, so the chunk set comes from the title's -// playgo-chunkdefs.xml: every listed chunk plus chunks 0 through the default chunk. Games probe -// chunk IDs and size arrays from the result, so unknown IDs must be rejected. +// playgo-chunkdefs.xml: every listed chunk plus chunks 0 through the default chunk. When that file +// exists, IDs outside the set are rejected; without it the table is unknown and every ID is accepted. struct ChunkTable { std::set ids{0}; bool defined = false; diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 8211cd700..63c0ce5a8 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -50,6 +50,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceVideoOutGetOutputStatusWithVrr](../../core/libs/prx/libSceVideoOut/Unimplemented.cpp) (libSceVideoOut, NID `gWT7X8H0bYs`) - the name hashes to the imported NID. PPSA26344 version 02.002.000 passes a VideoOut handle and an output pointer at three call sites. It compares a 32-bit value at byte 4 with 2, tests bit 0x10 at byte 16, and, when set, compares low 16-bit values at bytes 48, 50 and 52 with 48, 60 and 120. The full output layout, size, field meanings and console errors are unknown. Every call throws without reading or writing caller memory. - [getuid, geteuid, getgid, getegid and issetugid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) report the root identity (0) and a process that never changed credentials (`issetugid` 0), as a jailbroken process has; the host user and group are not exposed to the guest. The values a title sees on a console are unverified - [sceKernelGetAppInfo](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - only `app_id` and the title ID (offset 0x10, up to 9 characters plus a terminator) are written; callers declare the structure with different sizes, so the rest is left alone. `app_id` is 1, the running title's id in libSceSystemService, which is a guess, not something measured. The error codes follow shadPS4: `EPERM` for another process, and 0 without writing anything for a null pointer. +- [libScePlayGo](../../core/libs/prx/libScePlayGo/Export.cpp) - without `playgo-chunkdefs.xml` every chunk id is reported as installed (`LOCAL_FAST`) while `scePlayGoGetChunkId` still lists only chunk 0; the console's answer for an id outside the package's chunk table is not checked. - [sceFontSetScriptLanguage, sceFontGetScriptLanguage, sceFontSetTypographicDesign, sceFontGetTypographicDesign](../../core/libs/prx/libSceFont/src/Style.cpp) (libSceFont) - signatures follow [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.cs), not confirmed for the PS5. The handle keeps the language set for each script and the feature set for each typographic design and returns them, with 0 for one never set; the console default and its valid ranges are unknown, so any value is accepted. Neither setting changes shaping or rendering yet - [Packed pixel ancillary word](../../core/shader/recompiler/Optimization/src/ConstantFolder.cpp) (shader recompiler) - the sample index at bits 8-11 and the render target index at bits 16-28 follow #295, and their equality with the SampleId and Layer builtins is inferred, not measured. A value passed through a select, move or phi is rebuilt from those two fields with bits 0-7, 12-15 and 29-31 zero. That is exact only because every read of the word must be an extract inside one modelled field; any other read throws. The other bits are unmodelled and their RDNA 2 layout is unknown: the RDNA 3.5 ISA is reported to give prim type at bits 0-1, VRS rates at bits 2-5 and eye id at bit 7, which was not checked against RDNA 2 - [sceAudioPropagationSourceSetAudioPath](../../core/libs/prx/libSceAudioPropagation/Export.cpp) (libSceAudioPropagation, NID `tKSmk2JsMAA`) - `(uint64_t path, const void* data, float gain)` follows [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/5dfdb4b0b166e83dc9ae275001afb695f12c7174/src/SharpProspero/Interop/Audio/AudioPropagation.cs) and SAROS's call, which passes a handle from `sceAudioPropagationSourceGetAudioPath`, path data and gain 1.0. Path updates are unsupported and every call throws without reading or writing caller memory. The no-propagation model yields no path handles; path allocation, ownership, rendering and console error codes remain unknown. From 271e31a5a3d5bc62fd10a814f6a19368d8695c86 Mon Sep 17 00:00:00 2001 From: Blazing Cook <> Date: Sun, 11 Oct 2026 00:28:24 -0500 Subject: [PATCH 415/460] fix(conventions): restrict silent stub exemptions to their section --- tools/check_conventions.py | 18 ++++- .../tests/test_check_conventions_comments.py | 79 +++++++++++++++++++ 2 files changed, 96 insertions(+), 1 deletion(-) diff --git a/tools/check_conventions.py b/tools/check_conventions.py index 8c3c27239..2f073419e 100644 --- a/tools/check_conventions.py +++ b/tools/check_conventions.py @@ -83,7 +83,7 @@ class Check: return git("show", f"{self.head}:{path}") def code(self): - debt = [(text, links("docs/dev/TechnicalDebt.md", text)) for text in self.show("docs/dev/TechnicalDebt.md").splitlines()] if "docs/dev/TechnicalDebt.md" in self.tree else [] + debt = silent_stub_debt(self.show("docs/dev/TechnicalDebt.md")) if "docs/dev/TechnicalDebt.md" in self.tree else [] exports = {} for path, lines in self.files(lambda p: p.suffix in CPP): core = path.startswith("core/") and "tests" not in PurePosixPath(path).parts @@ -193,6 +193,22 @@ def links(path, text): return result +def silent_stub_debt(text): + debt = [] + section_level = 0 + for line in text.splitlines(): + heading = re.match(r"^(#{1,6})\s+(.+?)\s*$", line) + if heading: + level = len(heading[1]) + if heading[2] == "Silent stubs": + section_level = level + elif level <= section_level: + section_level = 0 + if section_level: + debt.append((line, links("docs/dev/TechnicalDebt.md", line))) + return debt + + def listed(debt, name, path): own = PurePosixPath(path).parts[:4] for text, targets in debt: diff --git a/tools/tests/test_check_conventions_comments.py b/tools/tests/test_check_conventions_comments.py index c4af4a8a0..3bc70b7bd 100644 --- a/tools/tests/test_check_conventions_comments.py +++ b/tools/tests/test_check_conventions_comments.py @@ -92,5 +92,84 @@ class SilentStubTests(unittest.TestCase): self.assertNotIn("[comment]", result.stdout) +class SilentStubSectionTests(unittest.TestCase): + path = "core/libs/prx/libSceExample/Export.cpp" + entry = f"- [Existing limitation](../../{path}) - console layout is unknown.\n" + + def check_stub(self, debt_text): + checker = Path(__file__).resolve().parents[1] / "check_conventions.py" + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + + def git(*args): + return subprocess.run( + ["git", "-c", f"safe.directory={root}", "-c", "user.name=Conventions Test", + "-c", "user.email=test@example.com", "-c", "commit.gpgsign=false", *args], + cwd=root, capture_output=True, text=True, check=True, + ) + + git("init", "-q") + debt = root / "docs/dev/TechnicalDebt.md" + debt.parent.mkdir(parents=True) + debt.write_text("# Project technical debt\n", encoding="utf-8") + for library in ("libSceExample", "libSceOther"): + reference = root / "core/libs/prx" / library / "Reference.hpp" + reference.parent.mkdir(parents=True) + reference.write_text("", encoding="utf-8") + git("add", ".") + git("commit", "-qm", "docs: record technical debt") + source = root / self.path + source.write_text("int APS5_VABI sceExample() { return 0; }\n", encoding="utf-8") + debt.write_text(debt_text, encoding="utf-8") + git("add", ".") + git("commit", "-qm", "feat: add example export") + env = {key: value for key, value in os.environ.items() + if key not in ("GITHUB_EVENT_PATH", "GITHUB_STEP_SUMMARY", "GITHUB_ACTIONS")} + env.update(GIT_CONFIG_COUNT="1", GIT_CONFIG_KEY_0="safe.directory", GIT_CONFIG_VALUE_0=str(root)) + return subprocess.run( + [sys.executable, str(checker), "--base", "HEAD~1"], + cwd=root, capture_output=True, text=True, env=env, timeout=10, + ) + + def require_rejected(self, debt): + result = self.check_stub(debt) + self.assertEqual(result.returncode, 1, result.stdout + result.stderr) + self.assertIn("[silent-stub]", result.stdout) + self.assertNotIn("[doc-link]", result.stdout) + + def require_accepted(self, debt): + result = self.check_stub(debt) + self.assertEqual(result.returncode, 0, result.stdout + result.stderr) + self.assertIn("All conventions checks passed", result.stdout) + + def test_debt_outside_silent_stubs_does_not_exempt_export(self): + for section in ("Build", "Unknown function info", "Functional"): + with self.subTest(section=section): + self.require_rejected(f"# Project technical debt\n\n### {section}\n\n{self.entry}") + + def test_entries_after_silent_stubs_do_not_exempt_export(self): + for heading in ("# Other debt", "## Other debt", "### Functional"): + with self.subTest(heading=heading): + self.require_rejected(f"# Project technical debt\n\n### Silent stubs\n\n{heading}\n\n{self.entry}") + + def test_file_entry_in_silent_stubs_exempts_export(self): + self.require_accepted(f"# Project technical debt\n\n### Silent stubs\n\n{self.entry}") + + def test_named_entry_in_same_library_exempts_export(self): + entry = "- sceExample: [reference](../../core/libs/prx/libSceExample/Reference.hpp)\n" + self.require_accepted(f"# Project technical debt\n\n### Silent stubs\n\n{entry}") + + def test_named_entry_in_another_library_does_not_exempt_export(self): + entry = "- sceExample: [reference](../../core/libs/prx/libSceOther/Reference.hpp)\n" + self.require_rejected(f"# Project technical debt\n\n### Silent stubs\n\n{entry}") + + def test_different_function_in_same_library_does_not_exempt_export(self): + entry = "- sceExampleOther: [reference](../../core/libs/prx/libSceExample/Reference.hpp)\n" + self.require_rejected(f"# Project technical debt\n\n### Silent stubs\n\n{entry}") + + def test_subsection_in_silent_stubs_keeps_exemption(self): + self.require_accepted(f"# Project technical debt\n\n### Silent stubs\n\n#### Dialogs\n\n{self.entry}") + + if __name__ == "__main__": unittest.main() From eb33094e20fc0e041c87f84d5f40439f9e4d5a99 Mon Sep 17 00:00:00 2001 From: Scott Schuster Date: Sun, 11 Oct 2026 02:46:00 -0500 Subject: [PATCH 416/460] test(shader): register the cross-workgroup test block as GPU-accessible Co-Authored-By: Claude Opus 5.5 --- .../prx/libSceAgcDriver/tests/execution/CrossWorkgroupFlag.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/CrossWorkgroupFlag.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/CrossWorkgroupFlag.cpp index 10da1c562..1238181a8 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/CrossWorkgroupFlag.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/CrossWorkgroupFlag.cpp @@ -41,7 +41,7 @@ public: block = static_cast(std::aligned_alloc(BlockBytes, BlockBytes)); #endif Require(block != nullptr, "cross-workgroup flag: cannot allocate the guest block"); - GuestAllocations::Mutation().Add(block, BlockBytes, true, true); + GuestAllocations::Mutation().Add(block, BlockBytes, true, true, true); } ~GuestBlock() { GuestAllocations::Mutation().Remove(block); } GuestBlock(const GuestBlock&) = delete; From 94d6a82d83b340b39e5b43cd756134ec1bc50189 Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Sun, 11 Oct 2026 16:46:16 +0900 Subject: [PATCH 417/460] docs(debt): list sceAudioInInit with the silent stubs Co-Authored-By: Claude Opus 5.5 --- docs/dev/TechnicalDebt.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 572482f0d..b6cfa5e91 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -43,6 +43,7 @@ Throughout the project, every function at every stage either **does exactly what - [ulobjmgr](../../core/libs/prx/ulobjmgr/Export.cpp) registers no object: `_sceUlobjmgrRegisterObject` always hands out id 0 and `_sceUlobjmgrUnregisterObject` releases nothing, as shadPS4 does - [libSceHttp](../../core/libs/prx/libSceHttp/Export.cpp) - no request reaches the network, so `sceHttpSetResponseHeaderMaxSize` has no response header to limit, `sceHttpSetRecvBlockSize` (`(int id, u32 blockSize)` as in [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/network/http.h)) no response body to read in blocks and `sceHttpRedirectCacheFlush` no redirect to forget; `sceHttpsUnloadCert` returns success like `sceHttpsLoadCert`, which keeps no certificate; `sceHttpSetRedirectCallback` and `sceHttpSetCookieRecvCallback` accept callbacks that are never called, `sceHttpsSetSslVersion` has no TLS connection to configure and `sceHttpsGetSslError` reports error 0 with detail 0; `sceHttpSetAuthInfoCallback` accepts a callback that is never called, as no server asks for authentication - [sceFontGlyphDefineAttribute](../../core/libs/prx/libSceFont/src/Render.cpp) (libSceFont) checks the glyph and ignores the attribute and its value, whose meaning is unknown. PPSA01325 sets attribute 0x11 to 0 on a glyph from `sceFontGenerateCharGlyph` that it deletes unused, and draws the character with `sceFontRenderCharGlyphImageHorizontal` +- [sceAudioInInit](../../core/libs/prx/libSceAudioIn/Export.cpp) (libSceAudioIn) - returns 0 without library-wide setup, as `sceAudioOutInit` does; each `sceAudioInOpen` opens its own SDL capture device ### Unknown function info From 26cad4e8c826f17be921ad0ae1708cf0d658a100 Mon Sep 17 00:00:00 2001 From: overlordxrz-source Date: Sun, 11 Oct 2026 02:38:19 +0000 Subject: [PATCH 418/460] feat(libkernel): implement sceKernelGetProsperoSystemSwVersion AI-assisted: yes --- core/libs/prx/libkernel/CMakeLists.txt | 10 ++++ .../libkernel/System/src/SoftwareVersion.cpp | 36 +++++++++++++ core/libs/tests/GuestSoftwareVersion.cpp | 50 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 97 insertions(+) create mode 100644 core/libs/prx/libkernel/System/src/SoftwareVersion.cpp create mode 100644 core/libs/tests/GuestSoftwareVersion.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index 0cfc3d427..deaa24180 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -53,6 +53,7 @@ add_library(libkernel SHARED EXCLUDE_FROM_ALL ${CMAKE_CURRENT_SOURCE_DIR}/System/src/Process.cpp ${CMAKE_CURRENT_SOURCE_DIR}/System/src/Sandbox.cpp ${CMAKE_CURRENT_SOURCE_DIR}/System/src/SystemConfiguration.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/System/src/SoftwareVersion.cpp ${CMAKE_CURRENT_SOURCE_DIR}/System/src/Signals.cpp ${CMAKE_CURRENT_SOURCE_DIR}/System/src/SignalSets.cpp ${CMAKE_CURRENT_SOURCE_DIR}/System/src/Notification.cpp @@ -677,6 +678,15 @@ if(BUILD_TESTING) set_tests_properties(guest_system_configuration PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_software_version_tests ${LIBS_INCLUDE_DIR}/tests/GuestSoftwareVersion.cpp) + target_include_directories(guest_software_version_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_software_version_tests PRIVATE libkernel libc) + configure_windows_unwind(guest_software_version_tests) + add_test(NAME guest_software_version COMMAND guest_software_version_tests) + if(WIN32) + set_tests_properties(guest_software_version PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_system_command_tests ${LIBS_INCLUDE_DIR}/tests/GuestSystemCommand.cpp) target_include_directories(guest_system_command_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_system_command_tests PRIVATE libkernel libc) diff --git a/core/libs/prx/libkernel/System/src/SoftwareVersion.cpp b/core/libs/prx/libkernel/System/src/SoftwareVersion.cpp new file mode 100644 index 000000000..0987eb6fa --- /dev/null +++ b/core/libs/prx/libkernel/System/src/SoftwareVersion.cpp @@ -0,0 +1,36 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include "prx/libkernel/KernelErrors.hpp" +#include +#include +#include + +struct SceKernelProsperoSystemSwVersion { + std::uint64_t size; + char versionString[0x1C]; + std::uint32_t version; +}; + +static_assert(offsetof(SceKernelProsperoSystemSwVersion, size) == 0x00); +static_assert(offsetof(SceKernelProsperoSystemSwVersion, versionString) == 0x08); +static_assert(offsetof(SceKernelProsperoSystemSwVersion, version) == 0x24); +static_assert(sizeof(SceKernelProsperoSystemSwVersion) == 0x28); + +namespace { + +constexpr std::uint32_t SystemSoftwareVersion = 0x01000000; +constexpr char SystemSoftwareVersionString[] = "01.000.000"; + +} + +extern "C" { + +int APS5_VABI sceKernelGetProsperoSystemSwVersion_nid_postfix(SceKernelProsperoSystemSwVersion* version) { + if (version == nullptr) return SCE_KERNEL_ERROR_EFAULT; + if (version->size != sizeof(SceKernelProsperoSystemSwVersion)) return SCE_KERNEL_ERROR_EINVAL; + std::memset(version->versionString, 0, sizeof(version->versionString)); + std::memcpy(version->versionString, SystemSoftwareVersionString, sizeof(SystemSoftwareVersionString)); + version->version = SystemSoftwareVersion; + return 0; +} + +} diff --git a/core/libs/tests/GuestSoftwareVersion.cpp b/core/libs/tests/GuestSoftwareVersion.cpp new file mode 100644 index 000000000..16a704685 --- /dev/null +++ b/core/libs/tests/GuestSoftwareVersion.cpp @@ -0,0 +1,50 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include "prx/libkernel/KernelErrors.hpp" +#include +#include +#include +#include + +struct SceKernelProsperoSystemSwVersion { + std::uint64_t size; + char versionString[0x1C]; + std::uint32_t version; +}; + +static_assert(offsetof(SceKernelProsperoSystemSwVersion, versionString) == 0x08); +static_assert(offsetof(SceKernelProsperoSystemSwVersion, version) == 0x24); +static_assert(sizeof(SceKernelProsperoSystemSwVersion) == 0x28); + +extern "C" { +int* APS5_VABI __error_nid_postfix(); +int APS5_VABI sceKernelGetProsperoSystemSwVersion_nid_postfix(SceKernelProsperoSystemSwVersion*); +} + +static void Require(bool value) { if (!value) std::abort(); } + +static void CheckProsperoSystemSwVersion() { + *__error_nid_postfix() = 13; + Require(sceKernelGetProsperoSystemSwVersion_nid_postfix(nullptr) == SCE_KERNEL_ERROR_EFAULT); + const std::uint64_t sizes[] = {0, 0x18, 0x27, 0x30}; + for (const std::uint64_t size : sizes) { + SceKernelProsperoSystemSwVersion version{}; + std::memset(&version, 0xaa, sizeof(version)); + version.size = size; + Require(sceKernelGetProsperoSystemSwVersion_nid_postfix(&version) == SCE_KERNEL_ERROR_EINVAL); + Require(version.size == size && version.version == 0xaaaaaaaa && static_cast(version.versionString[0]) == 0xaa); + } + SceKernelProsperoSystemSwVersion version{}; + std::memset(&version, 0xaa, sizeof(version)); + version.size = sizeof(version); + Require(sceKernelGetProsperoSystemSwVersion_nid_postfix(&version) == 0); + Require(version.size == sizeof(version)); + Require(std::strcmp(version.versionString, "01.000.000") == 0); + for (std::size_t i = sizeof("01.000.000"); i < sizeof(version.versionString); ++i) Require(version.versionString[i] == 0); + Require(version.version == 0x01000000); + Require(*__error_nid_postfix() == 13); +} + +int main() { + CheckProsperoSystemSwVersion(); + return 0; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 8211cd700..aaa484ecd 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -48,6 +48,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceNpGetSandboxInfo](../../core/libs/prx/libSceNpManager/Export.cpp) (libSceNpManager, NID `ZVZ++68nNI8`) - the name hashes to the imported NID (found with nidhunt, confirmed with `tools/nid_names.py`). STAR WARS: Galactic Racer (PPSA28416) imports it; its OnlineSubsystemPS5 calls it with one pointer to a 0x50-byte stack buffer and reads the buffer as a string (discussions/3086). The returned string, the exact parameter struct and console errors are unknown, so every call throws without reading or writing caller memory. A working implementation belongs with that title's bring-up. - [sceVideoOutGetOutputStatusWithVrr](../../core/libs/prx/libSceVideoOut/Unimplemented.cpp) (libSceVideoOut, NID `gWT7X8H0bYs`) - the name hashes to the imported NID. PPSA26344 version 02.002.000 passes a VideoOut handle and an output pointer at three call sites. It compares a 32-bit value at byte 4 with 2, tests bit 0x10 at byte 16, and, when set, compares low 16-bit values at bytes 48, 50 and 52 with 48, 60 and 120. The full output layout, size, field meanings and console errors are unknown. Every call throws without reading or writing caller memory. +- [sceKernelGetProsperoSystemSwVersion](../../core/libs/prx/libkernel/System/src/SoftwareVersion.cpp) (libkernel) reports system software version `0x01000000` ("01.000.000"); the console's firmware version is not modeled. The 0x28-byte block (a 64-bit size at 0, the version string at 8 and the packed version at 0x24) follows [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Kernel/KernelSystem.cs), which Prospero Multi Tools (PPSA99110) and Prospero Explorer (PPSA99109) are built with and which sets the size to 40, and the same fields in [etaHEN](https://github.com/etaHEN/etaHEN/blob/main/Source%20Code/daemon/include/ipc.hpp) and [shsrv](https://github.com/ps5-payload-dev/shsrv/blob/master/sh.c). A size other than 0x28 returns `SCE_KERNEL_ERROR_EINVAL` without writing; whether the console checks the size is unverified, since etaHEN and shsrv declare a 0x30-byte block and do not set it - [getuid, geteuid, getgid, getegid and issetugid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) report the root identity (0) and a process that never changed credentials (`issetugid` 0), as a jailbroken process has; the host user and group are not exposed to the guest. The values a title sees on a console are unverified - [sceKernelGetAppInfo](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - only `app_id` and the title ID (offset 0x10, up to 9 characters plus a terminator) are written; callers declare the structure with different sizes, so the rest is left alone. `app_id` is 1, the running title's id in libSceSystemService, which is a guess, not something measured. The error codes follow shadPS4: `EPERM` for another process, and 0 without writing anything for a null pointer. - [sceFontSetScriptLanguage, sceFontGetScriptLanguage, sceFontSetTypographicDesign, sceFontGetTypographicDesign](../../core/libs/prx/libSceFont/src/Style.cpp) (libSceFont) - signatures follow [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.cs), not confirmed for the PS5. The handle keeps the language set for each script and the feature set for each typographic design and returns them, with 0 for one never set; the console default and its valid ranges are unknown, so any value is accepted. Neither setting changes shaping or rendering yet From aeee59614e80219b6ced09e224fddcbbe361e354 Mon Sep 17 00:00:00 2001 From: overlordxrz-source Date: Sun, 11 Oct 2026 02:38:32 +0000 Subject: [PATCH 419/460] feat(libkernel): implement sceKernelGetAllowedSdkVersionOnSystem AI-assisted: yes --- .../prx/libkernel/System/src/SoftwareVersion.cpp | 7 +++++++ core/libs/tests/GuestSoftwareVersion.cpp | 14 ++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 22 insertions(+) diff --git a/core/libs/prx/libkernel/System/src/SoftwareVersion.cpp b/core/libs/prx/libkernel/System/src/SoftwareVersion.cpp index 0987eb6fa..6f834ce2f 100644 --- a/core/libs/prx/libkernel/System/src/SoftwareVersion.cpp +++ b/core/libs/prx/libkernel/System/src/SoftwareVersion.cpp @@ -19,6 +19,7 @@ namespace { constexpr std::uint32_t SystemSoftwareVersion = 0x01000000; constexpr char SystemSoftwareVersionString[] = "01.000.000"; +constexpr std::uint32_t SdkPatchMask = 0xfff; } @@ -33,4 +34,10 @@ int APS5_VABI sceKernelGetProsperoSystemSwVersion_nid_postfix(SceKernelProsperoS return 0; } +int APS5_VABI sceKernelGetAllowedSdkVersionOnSystem_nid_postfix(std::uint32_t* version) { + if (version == nullptr) return SCE_KERNEL_ERROR_EINVAL; + *version = SystemSoftwareVersion | SdkPatchMask; + return 0; +} + } diff --git a/core/libs/tests/GuestSoftwareVersion.cpp b/core/libs/tests/GuestSoftwareVersion.cpp index 16a704685..d572e1722 100644 --- a/core/libs/tests/GuestSoftwareVersion.cpp +++ b/core/libs/tests/GuestSoftwareVersion.cpp @@ -18,6 +18,7 @@ static_assert(sizeof(SceKernelProsperoSystemSwVersion) == 0x28); extern "C" { int* APS5_VABI __error_nid_postfix(); int APS5_VABI sceKernelGetProsperoSystemSwVersion_nid_postfix(SceKernelProsperoSystemSwVersion*); +int APS5_VABI sceKernelGetAllowedSdkVersionOnSystem_nid_postfix(std::uint32_t*); } static void Require(bool value) { if (!value) std::abort(); } @@ -44,7 +45,20 @@ static void CheckProsperoSystemSwVersion() { Require(*__error_nid_postfix() == 13); } +static void CheckAllowedSdkVersionOnSystem() { + *__error_nid_postfix() = 13; + Require(sceKernelGetAllowedSdkVersionOnSystem_nid_postfix(nullptr) == SCE_KERNEL_ERROR_EINVAL); + std::uint32_t allowed = 0xaaaaaaaa; + Require(sceKernelGetAllowedSdkVersionOnSystem_nid_postfix(&allowed) == 0); + SceKernelProsperoSystemSwVersion system{}; + system.size = sizeof(system); + Require(sceKernelGetProsperoSystemSwVersion_nid_postfix(&system) == 0); + Require(allowed == (system.version | 0xfff) && allowed >> 16 == system.version >> 16); + Require(*__error_nid_postfix() == 13); +} + int main() { CheckProsperoSystemSwVersion(); + CheckAllowedSdkVersionOnSystem(); return 0; } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index aaa484ecd..b9113cce3 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -49,6 +49,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceNpGetSandboxInfo](../../core/libs/prx/libSceNpManager/Export.cpp) (libSceNpManager, NID `ZVZ++68nNI8`) - the name hashes to the imported NID (found with nidhunt, confirmed with `tools/nid_names.py`). STAR WARS: Galactic Racer (PPSA28416) imports it; its OnlineSubsystemPS5 calls it with one pointer to a 0x50-byte stack buffer and reads the buffer as a string (discussions/3086). The returned string, the exact parameter struct and console errors are unknown, so every call throws without reading or writing caller memory. A working implementation belongs with that title's bring-up. - [sceVideoOutGetOutputStatusWithVrr](../../core/libs/prx/libSceVideoOut/Unimplemented.cpp) (libSceVideoOut, NID `gWT7X8H0bYs`) - the name hashes to the imported NID. PPSA26344 version 02.002.000 passes a VideoOut handle and an output pointer at three call sites. It compares a 32-bit value at byte 4 with 2, tests bit 0x10 at byte 16, and, when set, compares low 16-bit values at bytes 48, 50 and 52 with 48, 60 and 120. The full output layout, size, field meanings and console errors are unknown. Every call throws without reading or writing caller memory. - [sceKernelGetProsperoSystemSwVersion](../../core/libs/prx/libkernel/System/src/SoftwareVersion.cpp) (libkernel) reports system software version `0x01000000` ("01.000.000"); the console's firmware version is not modeled. The 0x28-byte block (a 64-bit size at 0, the version string at 8 and the packed version at 0x24) follows [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Kernel/KernelSystem.cs), which Prospero Multi Tools (PPSA99110) and Prospero Explorer (PPSA99109) are built with and which sets the size to 40, and the same fields in [etaHEN](https://github.com/etaHEN/etaHEN/blob/main/Source%20Code/daemon/include/ipc.hpp) and [shsrv](https://github.com/ps5-payload-dev/shsrv/blob/master/sh.c). A size other than 0x28 returns `SCE_KERNEL_ERROR_EINVAL` without writing; whether the console checks the size is unverified, since etaHEN and shsrv declare a 0x30-byte block and do not set it +- [sceKernelGetAllowedSdkVersionOnSystem](../../core/libs/prx/libkernel/System/src/SoftwareVersion.cpp) (libkernel) reports the system software version with the 12 patch bits set, `0x01000fff`, as [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/98fd0689/src/core/libraries/kernel/kernel.cpp) does on the PS4 and as [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Kernel/KernelSystem.cs) describes it, which Prospero Explorer (PPSA99109) reads it through; a null pointer returns `SCE_KERNEL_ERROR_EINVAL`. The console's value and its errors are unverified - [getuid, geteuid, getgid, getegid and issetugid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) report the root identity (0) and a process that never changed credentials (`issetugid` 0), as a jailbroken process has; the host user and group are not exposed to the guest. The values a title sees on a console are unverified - [sceKernelGetAppInfo](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - only `app_id` and the title ID (offset 0x10, up to 9 characters plus a terminator) are written; callers declare the structure with different sizes, so the rest is left alone. `app_id` is 1, the running title's id in libSceSystemService, which is a guess, not something measured. The error codes follow shadPS4: `EPERM` for another process, and 0 without writing anything for a null pointer. - [sceFontSetScriptLanguage, sceFontGetScriptLanguage, sceFontSetTypographicDesign, sceFontGetTypographicDesign](../../core/libs/prx/libSceFont/src/Style.cpp) (libSceFont) - signatures follow [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.cs), not confirmed for the PS5. The handle keeps the language set for each script and the feature set for each typographic design and returns them, with 0 for one never set; the console default and its valid ranges are unknown, so any value is accepted. Neither setting changes shaping or rendering yet From aaf5754d2798de99c398185e2aead247172d3b15 Mon Sep 17 00:00:00 2001 From: overlordxrz-source Date: Sun, 11 Oct 2026 03:33:20 +0000 Subject: [PATCH 420/460] feat(libSceSystemService): export sceSystemServiceLaunchApp AI-assisted: yes --- core/libs/prx/libSceSystemService/Export.cpp | 8 ++++++++ core/libs/tests/GuestSystemService.cpp | 18 ++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 27 insertions(+) diff --git a/core/libs/prx/libSceSystemService/Export.cpp b/core/libs/prx/libSceSystemService/Export.cpp index a5519684e..da7831be4 100644 --- a/core/libs/prx/libSceSystemService/Export.cpp +++ b/core/libs/prx/libSceSystemService/Export.cpp @@ -163,6 +163,14 @@ int APS5_VABI sceSystemServiceLaunchWebBrowser(const char* uri, void* param) { return SCE_KERNEL_ERROR_EOPNOTSUPP; } +int APS5_VABI sceSystemServiceLaunchApp(const char* titleId, const char* const* arguments, const void* param) { + (void)titleId; + (void)arguments; + (void)param; + NotImplemented_nid_no_patch("sceSystemServiceLaunchApp: launching another application"); + return 0; +} + int APS5_VABI sceSystemServiceDisableMusicPlayer(void) { return SYSTEM_SERVICE_OK; } diff --git a/core/libs/tests/GuestSystemService.cpp b/core/libs/tests/GuestSystemService.cpp index 84d4454e9..13b062c85 100644 --- a/core/libs/tests/GuestSystemService.cpp +++ b/core/libs/tests/GuestSystemService.cpp @@ -7,6 +7,7 @@ extern "C" int APS5_VABI sceSystemServiceGetHdrToneMapLuminance(SystemServiceHdrToneMapLuminance* luminance); extern "C" int APS5_VABI sceSystemServiceParamGetString(int paramId, char* buf, std::size_t bufSize); +extern "C" int APS5_VABI sceSystemServiceLaunchApp(const char* titleId, const char* const* arguments, const void* param); namespace { @@ -23,6 +24,15 @@ bool ParamGetStringThrows(int paramId, char* buf, std::size_t bufSize) { return false; } +bool LaunchAppThrows(const char* titleId, const char* const* arguments, const void* param) { + try { + static_cast(sceSystemServiceLaunchApp(titleId, arguments, param)); + } catch (const std::runtime_error&) { + return true; + } + return false; +} + } extern "C" int APS5_VABI sceSystemServicePowerTick(void); @@ -54,6 +64,14 @@ int main() { for (unsigned char byte : browserParam) Require(byte == 0x5a); Require(sceSystemServiceLaunchWebBrowser("", nullptr) == browserUnavailable); Require(sceSystemServiceLaunchWebBrowser(nullptr, nullptr) == browserUnavailable); + Require(LaunchAppThrows("PPSA99109", nullptr, nullptr)); + Require(LaunchAppThrows(nullptr, nullptr, nullptr)); + const char* launchArguments[] = {"--backup", nullptr}; + unsigned char launchParam[0x20]; + std::memset(launchParam, 0x5a, sizeof(launchParam)); + Require(LaunchAppThrows("PPSA99110", launchArguments, launchParam)); + for (unsigned char byte : launchParam) Require(byte == 0x5a); + Require(std::strcmp(launchArguments[0], "--backup") == 0 && launchArguments[1] == nullptr); Require(sceSystemServiceGetHdrToneMapLuminance(nullptr) == SYSTEM_SERVICE_ERROR_PARAMETER); SystemServiceHdrToneMapLuminance luminance{-1.0f, -1.0f, -1.0f}; Require(sceSystemServiceGetHdrToneMapLuminance(&luminance) == SYSTEM_SERVICE_OK); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index b9113cce3..4278ba365 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -50,6 +50,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceVideoOutGetOutputStatusWithVrr](../../core/libs/prx/libSceVideoOut/Unimplemented.cpp) (libSceVideoOut, NID `gWT7X8H0bYs`) - the name hashes to the imported NID. PPSA26344 version 02.002.000 passes a VideoOut handle and an output pointer at three call sites. It compares a 32-bit value at byte 4 with 2, tests bit 0x10 at byte 16, and, when set, compares low 16-bit values at bytes 48, 50 and 52 with 48, 60 and 120. The full output layout, size, field meanings and console errors are unknown. Every call throws without reading or writing caller memory. - [sceKernelGetProsperoSystemSwVersion](../../core/libs/prx/libkernel/System/src/SoftwareVersion.cpp) (libkernel) reports system software version `0x01000000` ("01.000.000"); the console's firmware version is not modeled. The 0x28-byte block (a 64-bit size at 0, the version string at 8 and the packed version at 0x24) follows [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Kernel/KernelSystem.cs), which Prospero Multi Tools (PPSA99110) and Prospero Explorer (PPSA99109) are built with and which sets the size to 40, and the same fields in [etaHEN](https://github.com/etaHEN/etaHEN/blob/main/Source%20Code/daemon/include/ipc.hpp) and [shsrv](https://github.com/ps5-payload-dev/shsrv/blob/master/sh.c). A size other than 0x28 returns `SCE_KERNEL_ERROR_EINVAL` without writing; whether the console checks the size is unverified, since etaHEN and shsrv declare a 0x30-byte block and do not set it - [sceKernelGetAllowedSdkVersionOnSystem](../../core/libs/prx/libkernel/System/src/SoftwareVersion.cpp) (libkernel) reports the system software version with the 12 patch bits set, `0x01000fff`, as [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/98fd0689/src/core/libraries/kernel/kernel.cpp) does on the PS4 and as [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Kernel/KernelSystem.cs) describes it, which Prospero Explorer (PPSA99109) reads it through; a null pointer returns `SCE_KERNEL_ERROR_EINVAL`. The console's value and its errors are unverified +- [sceSystemServiceLaunchApp](../../core/libs/prx/libSceSystemService/Export.cpp) (libSceSystemService) - AnyPS5 runs one title and cannot start another, so every call throws without reading its arguments. The (title ID, null-terminated argv, launch parameters) signature follows [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/5dfdb4b0b166e83dc9ae275001afb695f12c7174/src/SharpProspero/Interop/SystemService/SystemService.cs#L141-L148), whose `AppLauncher` Prospero Explorer (PPSA99109) and Prospero Multi Tools (PPSA99110) import it through and which passes a 0x20-byte launch context or null; the console's argument checks and errors are unknown - [getuid, geteuid, getgid, getegid and issetugid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) report the root identity (0) and a process that never changed credentials (`issetugid` 0), as a jailbroken process has; the host user and group are not exposed to the guest. The values a title sees on a console are unverified - [sceKernelGetAppInfo](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - only `app_id` and the title ID (offset 0x10, up to 9 characters plus a terminator) are written; callers declare the structure with different sizes, so the rest is left alone. `app_id` is 1, the running title's id in libSceSystemService, which is a guess, not something measured. The error codes follow shadPS4: `EPERM` for another process, and 0 without writing anything for a null pointer. - [sceFontSetScriptLanguage, sceFontGetScriptLanguage, sceFontSetTypographicDesign, sceFontGetTypographicDesign](../../core/libs/prx/libSceFont/src/Style.cpp) (libSceFont) - signatures follow [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Font/SceFont.cs), not confirmed for the PS5. The handle keeps the language set for each script and the feature set for each typographic design and returns them, with 0 for one never set; the console default and its valid ranges are unknown, so any value is accepted. Neither setting changes shaping or rendering yet From 3bb39589f3d190d9e3d70a9d413ebfc1d3a25bb3 Mon Sep 17 00:00:00 2001 From: fma-rc <188743521+fma-rc@users.noreply.github.com> Date: Sun, 11 Oct 2026 00:10:55 +0200 Subject: [PATCH 421/460] test(agc): fix two stale expectations in the full recorder run --- core/libs/prx/libSceAgcDriver/tests/Recorder.cpp | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp index adb35a6d7..21528bb0d 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp @@ -2483,6 +2483,11 @@ void staleGenerationTests(const Device& device, Recorder& recorder) { recorder.Submit(); device.WaitQueue(); recorder.Sync(); + Require(StorageTexture::FindPending(address, surfaceBytes) == image, "(s) the image is not pending after the refresh"); + Require(StorageTexture::FlushPending(address, surfaceBytes, nullptr, "stale generation test"), "(s) the pending image was not flushed"); + recorder.Submit(); + device.WaitQueue(); + recorder.Sync(); const auto pixels = readback.Bytes(); constexpr std::array expected{255, 0, 0, 255}; for (std::size_t i = 0; i < pixels.size(); ++i) Require(std::to_integer(pixels[i]) == expected[i % 4], "(s) results pending when their memory left the watch were dropped"); @@ -3230,9 +3235,9 @@ void importWindowTests(const Device& device, Recorder& recorder) { RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_TRANSFER_WRITE_BIT | VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT, VK_ACCESS_TRANSFER_WRITE_BIT); context.Function("vkCmdClearColorImage")(commands, image->Image(), VK_IMAGE_LAYOUT_GENERAL, &red, 1, &range); RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_TRANSFER_READ_BIT | VK_ACCESS_TRANSFER_WRITE_BIT); - image->MarkDirty(); const auto cached = CollectWritesUncached(address, surfaceBytes); Require(cached != 0 && UnchangedSince(address, surfaceBytes, cached), "(w) the cache's generation is not current before the import"); + image->MarkDirty(); const auto* import = HostImportFor(base, address, bytes); if (import == nullptr) { std::cout << "host import of the import window block refused: the import window not tested\n"; From 433302be58224f82e070f4818222b8f1e16e9cce Mon Sep 17 00:00:00 2001 From: fma-rc <188743521+fma-rc@users.noreply.github.com> Date: Sun, 11 Oct 2026 09:47:24 +0200 Subject: [PATCH 422/460] test(agc): check the recorder's (s) and (w) generations around the storage image's own import --- core/libs/prx/libSceAgcDriver/tests/Recorder.cpp | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp index 21528bb0d..2968dfe4a 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp @@ -2454,11 +2454,12 @@ void staleGenerationTests(const Device& device, Recorder& recorder) { resource.dstSelW = 7; resource.dccAddress = address + surfaceBytes; { + Require(AgcDriver::GuestMemory::Watched(address, bytes), "(s) the block is not watched before the storage image takes its generation"); auto image = std::make_shared(context, detiler, resource, 0); - Require(AgcDriver::GuestMemory::Watched(address, bytes), "(s) the storage image imported its memory before a generation was taken"); const VkImageSubresourceRange range{VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1}; const auto commands = recorder.Commands(); recorder.Keep(image); + Require(image->Generation() != 0, "(s) the storage image took no generation while its memory was watched"); const VkClearColorValue red{{1.0f, 0.0f, 0.0f, 1.0f}}; RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_TRANSFER_WRITE_BIT | VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT, VK_ACCESS_TRANSFER_WRITE_BIT); context.Function("vkCmdClearColorImage")(commands, image->Image(), VK_IMAGE_LAYOUT_GENERAL, &red, 1, &range); @@ -3226,8 +3227,10 @@ void importWindowTests(const Device& device, Recorder& recorder) { resource.dstSelW = 7; resource.dccAddress = address + surfaceBytes; { + Require(!HostImportCovers(base, address, bytes), "(w) the block is imported before the cache takes its generation"); + const auto cached = CollectWritesUncached(address, surfaceBytes); + Require(cached != 0 && UnchangedSince(address, surfaceBytes, cached), "(w) the cache's generation is not current before the import"); auto image = std::make_shared(context, detiler, resource, 0); - Require(!HostImportCovers(base, address, bytes), "(w) the storage image imported its memory before a generation was taken"); const VkImageSubresourceRange range{VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1}; const auto commands = recorder.Commands(); recorder.Keep(image); @@ -3235,8 +3238,6 @@ void importWindowTests(const Device& device, Recorder& recorder) { RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_TRANSFER_WRITE_BIT | VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT, VK_ACCESS_TRANSFER_WRITE_BIT); context.Function("vkCmdClearColorImage")(commands, image->Image(), VK_IMAGE_LAYOUT_GENERAL, &red, 1, &range); RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_TRANSFER_READ_BIT | VK_ACCESS_TRANSFER_WRITE_BIT); - const auto cached = CollectWritesUncached(address, surfaceBytes); - Require(cached != 0 && UnchangedSince(address, surfaceBytes, cached), "(w) the cache's generation is not current before the import"); image->MarkDirty(); const auto* import = HostImportFor(base, address, bytes); if (import == nullptr) { @@ -3248,6 +3249,7 @@ void importWindowTests(const Device& device, Recorder& recorder) { CollectWritesUncached(address, surfaceBytes); if (importWrites) { Require(!UnchangedSince(address, surfaceBytes, cached), "(w) a cache over the pages the import reported sees no change"); + Require(image->Generation() != 0 && !UnchangedSince(address, surfaceBytes, image->Generation()), "(w) the storage image took its generation after the import window"); std::array changed{}; const std::array generations{cached, cached, cached, cached}; Require(ChangedBlocks(address, surfaceBytes, generations, changed) && std::all_of(changed.begin(), changed.end(), [](std::uint8_t value) { return value == BlockMaybeWritten; }), "(w) the pages the import reported read as written by the CPU"); From 7e5a34c58c9871d1bd931730989500b5ef849763 Mon Sep 17 00:00:00 2001 From: hugodearaujo Date: Sat, 10 Oct 2026 22:38:09 -0300 Subject: [PATCH 423/460] fix(net): return the console's errors from the memory pool stats calls Measured on a PS5 (firmware 13.60): - sceSslGetMemoryPoolStats returns 0x8095F007 for a null output, before the id check, and 0x8095F006 for an unknown or terminated context, zeroing the 32-byte output. - sceHttp2GetMemoryPoolStats returns 0x817B1225 for a null output, before the id check, and 0x817B1100 for an unknown or terminated context, zeroing the 32-byte output. - sceNetGetMemoryPoolStats checks the pool before the output, so an unknown pool with a null output returns EBADF. Co-Authored-By: Claude Opus 5.5 --- core/libs/CMakeLists.txt | 10 +++ core/libs/prx/libSceHttp2/Export.cpp | 10 ++- core/libs/prx/libSceNet/Export.cpp | 6 +- core/libs/prx/libSceSsl/Export.cpp | 10 ++- core/libs/tests/GuestMemoryPoolStats.cpp | 90 ++++++++++++++++++++++++ docs/dev/TechnicalDebt.md | 2 +- 6 files changed, 120 insertions(+), 8 deletions(-) create mode 100644 core/libs/tests/GuestMemoryPoolStats.cpp diff --git a/core/libs/CMakeLists.txt b/core/libs/CMakeLists.txt index 9c5f89017..427e70bd2 100644 --- a/core/libs/CMakeLists.txt +++ b/core/libs/CMakeLists.txt @@ -189,6 +189,16 @@ if(BUILD_TESTING) "PATH=path_list_prepend:$") endif() + add_test_executable(guest_memory_pool_stats_tests tests/GuestMemoryPoolStats.cpp) + target_include_directories(guest_memory_pool_stats_tests PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}) + target_link_libraries(guest_memory_pool_stats_tests PRIVATE libSceSsl libSceHttp2 libSceNet libkernel libc) + configure_windows_unwind(guest_memory_pool_stats_tests) + add_test(NAME guest_memory_pool_stats COMMAND guest_memory_pool_stats_tests) + if(WIN32) + set_tests_properties(guest_memory_pool_stats PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() + add_test_executable(guest_system_gesture_tests tests/GuestSystemGesture.cpp) target_include_directories(guest_system_gesture_tests PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}) target_link_libraries(guest_system_gesture_tests PRIVATE libSceSystemGesture libc) diff --git a/core/libs/prx/libSceHttp2/Export.cpp b/core/libs/prx/libSceHttp2/Export.cpp index 215d674e5..d7874e9c9 100644 --- a/core/libs/prx/libSceHttp2/Export.cpp +++ b/core/libs/prx/libSceHttp2/Export.cpp @@ -1,5 +1,6 @@ #include #include +#include #include "SceTypes.hpp" #include "prx/libc/include/General.hpp" #include "prx/libkernel/Equeue/Equeue.hpp" @@ -13,6 +14,8 @@ // No network is emulated: contexts, templates and requests can be created, but any request // that would touch the network fails with the library's network error. static constexpr int ERROR_NETWORK = static_cast(0x80436063); +static constexpr int ERROR_INVALID_ID = static_cast(0x817B1100); +static constexpr int ERROR_NULL_POINTER = static_cast(0x817B1225); static std::atomic g_nextHandle{1}; static std::mutex g_poolsMutex; static std::map g_pools; @@ -256,10 +259,13 @@ int APS5_VABI sceHttp2AbortRequest(int req_id) { } int APS5_VABI sceHttp2GetMemoryPoolStats(int lib_http2_ctx_id, Http2MemoryPoolStats* stats) { - if (stats == nullptr) APS5_INVALID_ARG_EX; + if (stats == nullptr) return ERROR_NULL_POINTER; std::lock_guard lock(g_poolsMutex); const auto pool = g_pools.find(lib_http2_ctx_id); - if (pool == g_pools.end()) throw std::invalid_argument("sceHttp2GetMemoryPoolStats: unknown context"); + if (pool == g_pools.end()) { + std::memset(stats, 0, sizeof(*stats)); + return ERROR_INVALID_ID; + } *stats = {pool->second, 0, 0, 0}; return 0; } diff --git a/core/libs/prx/libSceNet/Export.cpp b/core/libs/prx/libSceNet/Export.cpp index 2bd2c3eb7..85ba8b7f3 100644 --- a/core/libs/prx/libSceNet/Export.cpp +++ b/core/libs/prx/libSceNet/Export.cpp @@ -912,14 +912,14 @@ int APS5_VABI sceNetEpollCreate(const char* name, int flags) { } int APS5_VABI sceNetGetMemoryPoolStats(int memid, NetMemoryPoolStats* stats) { - if (stats == nullptr) { - return fail(NET_EINVAL); - } std::lock_guard lk(g_mutex); const auto pool = g_pools.find(memid); if (pool == g_pools.end()) { return fail(NET_EBADF); } + if (stats == nullptr) { + return fail(NET_EINVAL); + } *stats = {pool->second, 0, 0}; return 0; } diff --git a/core/libs/prx/libSceSsl/Export.cpp b/core/libs/prx/libSceSsl/Export.cpp index a1623ff2a..e8fd49499 100644 --- a/core/libs/prx/libSceSsl/Export.cpp +++ b/core/libs/prx/libSceSsl/Export.cpp @@ -1,5 +1,6 @@ #include #include +#include #include "SceTypes.hpp" #include "prx/libc/include/General.hpp" #include @@ -19,6 +20,8 @@ namespace { constexpr int ERROR_NOT_FOUND = static_cast(0x8095F004); constexpr int ERROR_INVALID_ARG = static_cast(0x8095177A); +constexpr int ERROR_INVALID_ID = static_cast(0x8095F006); +constexpr int ERROR_INVALID_VALUE = static_cast(0x8095F007); struct SslData { char* ptr; @@ -115,10 +118,13 @@ int APS5_VABI sceSslUnloadCert(int sslCtxId) { } int APS5_VABI sceSslGetMemoryPoolStats(int ssl_ctx_id, SslMemoryPoolStats* stats) { - if (stats == nullptr) APS5_INVALID_ARG_EX; + if (stats == nullptr) return ERROR_INVALID_VALUE; std::lock_guard lock(g_poolsMutex); const auto pool = g_pools.find(ssl_ctx_id); - if (pool == g_pools.end()) throw std::invalid_argument("sceSslGetMemoryPoolStats: unknown context"); + if (pool == g_pools.end()) { + std::memset(stats, 0, sizeof(*stats)); + return ERROR_INVALID_ID; + } *stats = {static_cast(pool->second), 0, 0, 0}; return 0; } diff --git a/core/libs/tests/GuestMemoryPoolStats.cpp b/core/libs/tests/GuestMemoryPoolStats.cpp new file mode 100644 index 000000000..7aec461e1 --- /dev/null +++ b/core/libs/tests/GuestMemoryPoolStats.cpp @@ -0,0 +1,90 @@ +#include "prx/libc/include/general/VabiMacros.hpp" + +#include +#include +#include +#include +#include + +struct PoolStats { + std::size_t pool_size; + std::size_t max_inuse_size; + std::size_t current_inuse_size; + std::int32_t reserved; +}; + +struct NetPoolStats { + std::size_t pool_size; + std::size_t max_inuse_size; + std::size_t current_inuse_size; +}; + +extern "C" { +int APS5_VABI sceSslInit_nid_postfix(std::size_t); +int APS5_VABI sceSslTerm_nid_postfix(int); +int APS5_VABI sceSslGetMemoryPoolStats(int, void*); +int APS5_VABI sceHttp2Init(int, int, std::size_t, int); +int APS5_VABI sceHttp2Term(int); +int APS5_VABI sceHttp2GetMemoryPoolStats(int, void*); +int APS5_VABI sceNetPoolCreate(const char*, int, int); +int APS5_VABI sceNetPoolDestroy(int); +int APS5_VABI sceNetGetMemoryPoolStats(int, void*); +} + +static void Require(bool value) { if (!value) std::abort(); } + +using StatsCall = int(APS5_VABI*)(int, void*); + +static void CheckRejected(StatsCall call, int id, int expected) { + std::array buffer; + buffer.fill(0x5a); + Require(call(id, buffer.data()) == expected); + for (std::size_t i = 0; i < sizeof(PoolStats); ++i) Require(buffer[i] == 0); + for (std::size_t i = sizeof(PoolStats); i < buffer.size(); ++i) Require(buffer[i] == 0x5a); +} + +int main() { + constexpr int sslInvalidId = static_cast(0x8095F006); + constexpr int sslInvalidValue = static_cast(0x8095F007); + constexpr int http2InvalidId = static_cast(0x817B1100); + constexpr int http2NullPointer = static_cast(0x817B1225); + constexpr int netBadDescriptor = static_cast(0x80410109); + constexpr int netInvalid = static_cast(0x80410116); + static_assert(sizeof(PoolStats) == 0x20 && sizeof(NetPoolStats) == 0x18); + + const int ssl = sceSslInit_nid_postfix(0x40000); + Require(ssl > 0); + PoolStats stats{}; + Require(sceSslGetMemoryPoolStats(ssl, &stats) == 0 && stats.pool_size == 0x40000); + Require(sceSslGetMemoryPoolStats(ssl, nullptr) == sslInvalidValue); + Require(sceSslGetMemoryPoolStats(12345, nullptr) == sslInvalidValue); + CheckRejected(sceSslGetMemoryPoolStats, 12345, sslInvalidId); + CheckRejected(sceSslGetMemoryPoolStats, -1, sslInvalidId); + + const int http2 = sceHttp2Init(1, ssl, 0x40000, 1); + Require(http2 > 0); + Require(sceHttp2GetMemoryPoolStats(http2, &stats) == 0 && stats.pool_size == 0x40000); + Require(sceHttp2GetMemoryPoolStats(http2, nullptr) == http2NullPointer); + Require(sceHttp2GetMemoryPoolStats(12345, nullptr) == http2NullPointer); + CheckRejected(sceHttp2GetMemoryPoolStats, 12345, http2InvalidId); + CheckRejected(sceHttp2GetMemoryPoolStats, -1, http2InvalidId); + Require(sceHttp2Term(http2) == 0); + CheckRejected(sceHttp2GetMemoryPoolStats, http2, http2InvalidId); + + Require(sceSslTerm_nid_postfix(ssl) == 0); + CheckRejected(sceSslGetMemoryPoolStats, ssl, sslInvalidId); + + const int pool = sceNetPoolCreate("stats", 0x20000, 0); + Require(pool > 0); + NetPoolStats netStats{}; + Require(sceNetGetMemoryPoolStats(pool, &netStats) == 0 && netStats.pool_size == 0x20000); + Require(sceNetGetMemoryPoolStats(pool, nullptr) == netInvalid); + Require(sceNetGetMemoryPoolStats(12345, nullptr) == netBadDescriptor); + std::array untouched; + untouched.fill(0x5a); + auto buffer = untouched; + Require(sceNetGetMemoryPoolStats(12345, buffer.data()) == netBadDescriptor && buffer == untouched); + Require(sceNetPoolDestroy(pool) == 0); + Require(sceNetGetMemoryPoolStats(pool, nullptr) == netBadDescriptor); + return 0; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 090027395..ba5d8c340 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -212,7 +212,7 @@ Throughout the project, every function at every stage either **does exactly what - [scePadSetAngularVelocityBiasCorrectionState](../../core/libs/prx/libScePad/Export.cpp) (libScePad) - inferred signature `(int handle, bool enabled)` follows `scePadSetTiltCorrectionState` and `scePadSetAngularVelocityDeadbandState`; an invalid handle returns `PAD_ERROR_INVALID_HANDLE`, disabling is a no-op, and enabling throws because bias correction is not modelled - [sceKernelGetOperationMode](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - the signature `(int* mode, int* submode)` comes from PPSA12544, which logs both values after the call; the values are unknown and 0 is reported for both, as in the [prosper](https://github.com/mattias800/prosper/pull/4153) reimplementation. Null outputs throw - Font, Http2, Net, Ssl, SystemService and libkernel exports imported by PPSA12544 and added without an implementation have unknown signatures -- [sceHttp2GetMemoryPoolStats, sceSslGetMemoryPoolStats, sceNetGetMemoryPoolStats](../../core/libs/prx/libSceNet/Export.cpp) (libSceHttp2, libSceSsl, libSceNet) - signatures `(int id, stats*)` and the statistics layouts (pool size, peak and current use, and for Http2 and Ssl a 32-bit reserved field) follow the PS4 SDK's `SceHttp2MemoryPoolStats`, `SceSslMemoryPoolStats` and `SceNetMemoryPoolStats`, not confirmed for the PS5. The pool size is the one given at init or pool creation; nothing is allocated from these pools, so the peak and current use are 0. Net returns `EBADF` for an unknown pool and `EINVAL` for a null output, as its other pool calls do; for Http2 and Ssl the error codes are unknown, so both cases throw +- [sceHttp2GetMemoryPoolStats, sceSslGetMemoryPoolStats, sceNetGetMemoryPoolStats](../../core/libs/prx/libSceNet/Export.cpp) (libSceHttp2, libSceSsl, libSceNet) - the signatures `(int id, stats*)`, the layouts (pool size, peak and current use, and for Http2 and Ssl a 32-bit reserved field) and the error codes were measured on a PS5 (firmware 13.60). The pool size is the one given at init or pool creation; nothing is allocated from these pools, so the peak and current use are 0, where the console already reports use right after initialization (0x15C0 bytes for Ssl, 0x1640 for Http2 and 4 for a Net pool) - [sceSslGetCaList, sceSslFreeCaList, sceSslUnloadCert](../../core/libs/prx/libSceSsl/Export.cpp) (libSceSsl) - signatures and the 16-byte `SceSslCaList` layout (an array of certificate handles and a 32-bit count) follow [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/5dfdb4b0b166e83dc9ae275001afb695f12c7174/src/SharpProspero/Interop/Net/Http.cs) and shadPS4's `OrbisSslCaList`, not confirmed for the PS5. On the console the list holds the certificate authorities the SSL service has; [shadPS4](https://github.com/shadps4-emu/shadPS4/pull/4104) returns a placeholder certificate from `sceSslGetCaCerts` because Rise of the Tomb Raider fails its initialization checks without one. No root certificate bundle is shipped and no loaded certificate is kept, so here a context holds none: `sceSslGetCaList` clears the list and returns `NOT_FOUND`, as `sceSslGetCaCerts` does, `sceSslFreeCaList` accepts only that cleared list, and `sceSslUnloadCert` (imported by PPSA15508) has nothing to drop and is assumed to return 0. A null list returns `0x8095177A`, which `sceSslGetCaCerts` and [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/59183dbfa81ed7efdff6069177fed50de0552725/src/libs/network.cpp) return for a null argument (shadPS4, PS4, returns `0x8095F007`). An unknown context throws (KytyPS5 returns `SSL_ERROR_INVALID_ID`, `0x8095F006`, from `sceSslGetCaCerts`), and so does a list `sceSslGetCaList` did not return - [sceHttp2SetResolveRetry](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) - signature `(int id, int32_t retry)` from libSceHttp's `sceHttpSetResolveRetry`; the value is accepted and ignored like the other Http2 settings, since no request reaches the network - [sceHttp2WebSocketCreateRequest, sceHttp2WebSocketCloseAsync, sceHttp2WebSocketSendTextMessageAsync, sceHttp2WebSocketSendDataMessageAsync](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) - imported by PPSA16064 and not called during its start-up; unknown signatures From 04835bdcd1bb8bfabb7223b7fda7c0b14f56e5aa Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Sat, 10 Oct 2026 21:45:28 -0400 Subject: [PATCH 424/460] feat(libkernel): support kern.rng_pseudo in sysctlbyname --- .../System/src/SystemConfiguration.cpp | 12 +++++++++++ core/libs/tests/GuestSystemConfiguration.cpp | 20 +++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 33 insertions(+) diff --git a/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp b/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp index 98a4a2111..abbd8d0ea 100644 --- a/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp +++ b/core/libs/prx/libkernel/System/src/SystemConfiguration.cpp @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #ifdef _WIN32 @@ -109,6 +110,17 @@ int APS5_VABI sysctlbyname_nid_postfix(const char* name, void* oldValue, std::si *__error_nid_postfix() = 14; return -1; } + if (std::strcmp(name, "kern.rng_pseudo") == 0) { + if (newValue != nullptr) { + *__error_nid_postfix() = 1; + return -1; + } + if (oldValue == nullptr || oldLength == nullptr) throw std::runtime_error(std::string(__func__) + ": kern.rng_pseudo without an output buffer"); + static std::random_device device; + auto* bytes = static_cast(oldValue); + for (std::size_t i = 0; i < *oldLength; ++i) bytes[i] = static_cast(device()); + return 0; + } if (std::strcmp(name, "hw.ncpu") != 0) throw std::runtime_error(std::string(__func__) + ": unsupported name " + name); static constexpr int mib[] = {6, 3}; return sysctl_nid_postfix(mib, 2, oldValue, oldLength, newValue, newLength); diff --git a/core/libs/tests/GuestSystemConfiguration.cpp b/core/libs/tests/GuestSystemConfiguration.cpp index ea0fe752a..ffba40d37 100644 --- a/core/libs/tests/GuestSystemConfiguration.cpp +++ b/core/libs/tests/GuestSystemConfiguration.cpp @@ -88,9 +88,29 @@ static void CheckSystemSwVersion() { } Require(version.version == 0x01000000); } +static void CheckPseudoRandomSysctl() { + *__error_nid_postfix() = 13; + unsigned char first[64] = {}; + unsigned char second[64] = {}; + std::size_t length = sizeof(first); + Require(sysctlbyname_nid_postfix("kern.rng_pseudo", first, &length, nullptr, 0) == 0 && length == sizeof(first)); + length = sizeof(second); + Require(sysctlbyname_nid_postfix("kern.rng_pseudo", second, &length, nullptr, 0) == 0 && length == sizeof(second)); + Require(std::memcmp(first, second, sizeof(first)) != 0); + Require(*__error_nid_postfix() == 13); + Require(sysctlbyname_nid_postfix("kern.rng_pseudo", first, &length, first, sizeof(first)) == -1 && *__error_nid_postfix() == 1); + bool threw = false; + try { + sysctlbyname_nid_postfix("kern.rng_pseudo", nullptr, &length, nullptr, 0); + } catch (const std::runtime_error&) { + threw = true; + } + Require(threw); +} int main() { CheckProcessorCountSysctl(); CheckSystemSwVersion(); + CheckPseudoRandomSysctl(); *__error_nid_postfix() = 13; Require(sysconf_nid_postfix(47) == 0x4000); Require(getpagesize_nid_postfix() == sysconf_nid_postfix(47)); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index c7a461544..c31f8c92f 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -48,6 +48,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceNpGetSandboxInfo](../../core/libs/prx/libSceNpManager/Export.cpp) (libSceNpManager, NID `ZVZ++68nNI8`) - the name hashes to the imported NID (found with nidhunt, confirmed with `tools/nid_names.py`). STAR WARS: Galactic Racer (PPSA28416) imports it; its OnlineSubsystemPS5 calls it with one pointer to a 0x50-byte stack buffer and reads the buffer as a string (discussions/3086). The returned string, the exact parameter struct and console errors are unknown, so every call throws without reading or writing caller memory. A working implementation belongs with that title's bring-up. - [sceKernelGetSystemSwVersion](../../core/libs/prx/libkernel/System/src/SystemConfiguration.cpp) (libkernel) reports baseline system software version `0x01000000` ("01.000.000") for ProsperoStore (PPSA99000). The 0x28-byte layout (`uint64_t size` at 0, `versionString` at 8 and `version` at 0x24) matches [ProsperoStore's caller](https://github.com/blackbearreloaded/ProsperoStore/blob/20584011/src/diag/trace.cpp#L20-L26), [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/811bfcd7d1/src/core/libraries/kernel/kernel.h#L45-L49), [fpPS4](https://github.com/red-prig/fpPS4/blob/04cefd43e6/kernel/ps4_libkernel.pas#L502-L506) and [OpenOrbis](https://github.com/OpenOrbis/OpenOrbis-PS4-Toolchain/blob/0a1aaf9dd4/include/orbis/_types/kernel.h#L256-L260). These sources support the structure layout, but not the reported version value: the console firmware version remains unverified. The checks are unverified too: a null pointer returns `SCE_KERNEL_ERROR_EFAULT` and a `size` other than 0x28 returns `SCE_KERNEL_ERROR_EINVAL`, while shadPS4 returns success for null and never reads `size`. +- [kern.rng_pseudo](../../core/libs/prx/libkernel/System/src/SystemConfiguration.cpp) (libkernel) - `sysctlbyname` fills the whole output buffer with bytes from the host's `std::random_device` and leaves the length unchanged; a write returns -1 with `EPERM` and a call without an output buffer throws. The only caller seen is mbedTLS's `mbedtls_hardware_poll` in the catalog's ProsperoLight boilerplate, which asks for 64-byte blocks and fails if fewer come back. The console's generator, its per-call maximum and its answer to a size query are unknown - [sceVideoOutGetOutputStatusWithVrr](../../core/libs/prx/libSceVideoOut/Unimplemented.cpp) (libSceVideoOut, NID `gWT7X8H0bYs`) - the name hashes to the imported NID. PPSA26344 version 02.002.000 passes a VideoOut handle and an output pointer at three call sites. It compares a 32-bit value at byte 4 with 2, tests bit 0x10 at byte 16, and, when set, compares low 16-bit values at bytes 48, 50 and 52 with 48, 60 and 120. The full output layout, size, field meanings and console errors are unknown. Every call throws without reading or writing caller memory. - [getuid, geteuid, getgid, getegid and issetugid](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) report the root identity (0) and a process that never changed credentials (`issetugid` 0), as a jailbroken process has; the host user and group are not exposed to the guest. The values a title sees on a console are unverified - [sceKernelGetAppInfo](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - only `app_id` and the title ID (offset 0x10, up to 9 characters plus a terminator) are written; callers declare the structure with different sizes, so the rest is left alone. `app_id` is 1, the running title's id in libSceSystemService, which is a guess, not something measured. The error codes follow shadPS4: `EPERM` for another process, and 0 without writing anything for a null pointer. From b88ad020906fbcc5af261178b0b370594ee0e109 Mon Sep 17 00:00:00 2001 From: Chad Date: Sat, 10 Oct 2026 15:52:30 -0500 Subject: [PATCH 425/460] perf(agc): mirror a sparse descriptor heap instead of copying it per dispatch --- .../Graphics/include/GuestBufferMemory.hpp | 6 + .../Graphics/src/GuestBufferMemory.cpp | 310 ++++++++++++++++-- .../libSceAgcDriver/tests/BdaResources.cpp | 123 +++++++ 3 files changed, 410 insertions(+), 29 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp index df3640c0d..73e9077f6 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp @@ -157,6 +157,7 @@ struct MirrorStats { std::uint64_t heapRefills = 0; std::uint64_t sweeps = 0; std::uint64_t heapChecks = 0; + std::uint64_t sparseBinds = 0; }; MirrorStats MirrorCounters(); void ClearImageMirrors(VkDevice device); @@ -282,6 +283,7 @@ private: // Set when the region covers uncommitted pages: only the `backed` parts (possibly none) are guest // memory; the rest reads as zeros and is never stored. bool sparse = false; + bool sparseHeap = false; std::vector> backed {}; std::uint64_t lastView = 0; std::uint64_t bufferEnd = 0; @@ -336,11 +338,15 @@ private: void addCopiedRange(const CopiedRange& range); // The BDA table entry of an uploaded region. static ShaderRecompiler::BdaAbi::Range addressRange(const Region& region); + static std::uint64_t servedEnd(const Region& region); void validate(std::uint64_t address, std::size_t bytes) const; // The region of a descriptor-bound range (AddWritable/AddReadable), committed pages only. A // range inside a base region of the space adds nothing: the region serves it, as today's merge // of the two did. void addDescriptorRegion(std::uint64_t address, std::size_t bytes, bool atomic, bool swept); + static bool servesView(const Region& region, std::uint64_t view); + static bool sparseHeapCandidate(const Region& region); + bool bindSparseMirror(Region& region, const GuestAllocations::Lease& lease); // Gives a region a buffer of its own with its bytes (guest memory for host-backed and writable // ranges, plus the write-back's reference copy for a range a descriptor writes; else its snapshot). void copyRegion(Region& region, bool addressable); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp index 3c58b415b..0e8359f04 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp @@ -20,6 +20,7 @@ #include #include #endif +#include #include #include #include @@ -57,6 +58,9 @@ struct ImageMirror { std::uint64_t bytes = 0; bool writable = false; bool heap = false; + bool sparse = false; + std::uint64_t reserved = 0; + std::vector> committed; std::vector generations; std::weak_ptr range; std::shared_ptr buffer; @@ -98,6 +102,7 @@ struct GuestBufferMemory::AddressSpace { std::vector writableMirrors; std::vector heapMirrors; std::vector heapRuns; + std::vector sparseMirrors; std::unique_ptr[]> verified; std::vector copied; // The BDA table entries of `base`, in its order. @@ -788,6 +793,7 @@ struct ImageMirrors { std::uint64_t heapImported = 0; std::uint64_t heapImportedBytes = 0; std::uint64_t subranges = 0; + std::uint64_t sparseBinds = 0; std::uint64_t rebuilds = 0; std::uint64_t refreshes = 0; std::uint64_t blocksCompared = 0; @@ -883,12 +889,22 @@ struct RefreshBlock { std::uint64_t address; std::size_t length; std::uint64_t generation = 0; + std::uint8_t fill = 0; }; +constexpr std::uint8_t SparseCopy = 1; +constexpr std::uint8_t SparseZero = 2; + // Compares a block's guest bytes with the mirror's shadow and copies a changed block into the device bool compareBlock(const RefreshBlock& block) { const auto offset = static_cast(block.address - block.mirror->base); const auto* guest = reinterpret_cast(block.address); + if (block.fill != 0) { + auto* target = block.mirror->buffer->Bytes().data() + offset; + if (block.fill == SparseZero) std::memset(target, 0, block.length); + else std::memcpy(target, guest, block.length); + return true; + } if (block.mirror->heap) { std::memcpy(block.mirror->buffer->Bytes().data() + offset, guest, block.length); block.mirror->generations[static_cast(block.address / 65536 - block.mirror->base / 65536)] = block.generation; @@ -997,10 +1013,7 @@ private: // completes first, so its results reach the guest and the shadow before the compare and no batch // writes a block being replaced; GPU reads of the mirror by earlier recorded work (an address-based // dispatch whose lease is released at completion) see the newer bytes, as they would on hardware. -void prepareRange(std::uint64_t address, std::uint64_t bytes) { - // The compare reads the pages directly; a mirror is only reused while its Range exists, but a - // lease of a build in flight can keep a replaced Range alive past a protection change. - Require(GuestMemory::Accessible(reinterpret_cast(address), static_cast(bytes)), "image mirror range became inaccessible"); +void syncRange(std::uint64_t address, std::uint64_t bytes) { // Named for the [hooksync] attribution: the flush goes through the hook, which waits when // recorded work notes a write inside the range (a descriptor written inside a writable exe // range; the lease itself maps the range read-only). The access is the whole range. @@ -1015,6 +1028,13 @@ void prepareRange(std::uint64_t address, std::uint64_t bytes) { ++state.refreshes; } +void prepareRange(std::uint64_t address, std::uint64_t bytes) { + // The compare reads the pages directly; a mirror is only reused while its Range exists, but a + // lease of a build in flight can keep a replaced Range alive past a protection change. + Require(GuestMemory::Accessible(reinterpret_cast(address), static_cast(bytes)), "image mirror range became inaccessible"); + syncRange(address, bytes); +} + bool prepareRefresh(ImageMirror& mirror, std::uint64_t address, std::uint64_t bytes) { if ((!mirror.writable && !mirror.heap) || bytes == 0) return false; prepareRange(address, bytes); @@ -1085,6 +1105,105 @@ void refreshHeapMirrors(std::vector& mirrors, std::vector>; + +std::uint64_t sparseExtent(std::uint64_t base, std::uint64_t reserved, const Pieces& pieces) { + constexpr std::uint64_t granule = std::uint64_t{1} << 20u; + const auto used = pieces.empty() ? 0 : pieces.back().second - base; + const auto wanted = used + used / 8; + return std::min(reserved, std::max(granule, (wanted + granule - 1) / granule * granule)); +} + +std::uint64_t collectPieces(const Pieces& pieces) { + std::uint64_t generation = 0; + for (const auto& [first, last] : pieces) { + generation = GuestMemory::CollectWrites(first, static_cast(last - first)); + if (generation == 0) return 0; + } + return generation != 0 ? generation : GuestMemory::TrackerGeneration(); +} + +bool covers(const Pieces& pieces, std::uint64_t address) { + return std::any_of(pieces.begin(), pieces.end(), [&](const auto& piece) { return piece.first <= address && address < piece.second; }); +} + +void forgetCommitChanges(ImageMirror& mirror, const Pieces& pieces) { + if (pieces == mirror.committed) return; + std::vector edges; + for (const auto* list : std::array{&pieces, &mirror.committed}) { + for (const auto& [first, last] : *list) { + edges.push_back(first); + edges.push_back(last); + } + } + std::sort(edges.begin(), edges.end()); + edges.erase(std::unique(edges.begin(), edges.end()), edges.end()); + constexpr std::uint64_t block = 65536; + const auto end = mirror.base + mirror.bytes; + for (std::size_t at = 0; at + 1 < edges.size(); ++at) { + const auto from = std::max(edges[at], mirror.base); + const auto to = std::min(edges[at + 1], end); + if (from >= to || covers(pieces, from) == covers(mirror.committed, from)) continue; + for (auto index = from / block - mirror.base / block; index <= (to - 1) / block - mirror.base / block; ++index) mirror.generations[static_cast(index)] = 0; + } +} + +void appendSparseBlocks(ImageMirror& mirror, Pieces pieces, std::uint64_t generation, std::vector& blocks) { + constexpr std::uint64_t block = 65536; + thread_local std::vector* changedSlot = nullptr; + auto& changed = ShaderRecompiler::ThreadOwned(changedSlot); + forgetCommitChanges(mirror, pieces); + ++Mirrors().heapChecks; + changed.assign(mirror.generations.size(), 0); + GuestMemory::ChangedBlocks(mirror.base, static_cast(mirror.bytes), mirror.generations, changed); + const auto aligned = mirror.base / block * block; + const auto end = mirror.base + mirror.bytes; + bool refilled = false; + std::size_t piece = 0; + for (std::size_t at = 0; at < changed.size(); ++at) { + if (changed[at] == 0) continue; + refilled = true; + mirror.generations[at] = generation; + const auto from = std::max(mirror.base, aligned + at * block); + const auto to = std::min(end, aligned + (at + 1) * block); + auto cursor = from; + while (piece < pieces.size() && pieces[piece].second <= from) ++piece; + for (auto next = piece; next < pieces.size() && pieces[next].first < to; ++next) { + const auto first = std::max(pieces[next].first, from); + const auto last = std::min(pieces[next].second, to); + if (cursor < first) blocks.push_back({&mirror, cursor, static_cast(first - cursor), 0, SparseZero}); + blocks.push_back({&mirror, first, static_cast(last - first), 0, SparseCopy}); + cursor = last; + } + if (cursor < to) blocks.push_back({&mirror, cursor, static_cast(to - cursor), 0, SparseZero}); + } + if (refilled) ++Mirrors().heapRefills; + mirror.committed = std::move(pieces); +} + +Pieces clipPieces(const Pieces& pieces, std::uint64_t begin, std::uint64_t end) { + Pieces result; + for (const auto& [first, last] : pieces) { + if (std::max(first, begin) < std::min(last, end)) result.emplace_back(std::max(first, begin), std::min(last, end)); + } + return result; +} + +bool refreshSparseMirror(ImageMirror& mirror, std::vector& blocks) { + auto pieces = GuestMemory::DescribeCommitted(mirror.base, static_cast(mirror.reserved)).ranges; + if (!pieces.empty() && pieces.back().second > mirror.base + mirror.bytes) return false; + syncRange(mirror.base, mirror.bytes); + const auto generation = collectPieces(pieces); + Require(generation != 0, "a sparse heap mirror's range is no longer write-watched"); + appendSparseBlocks(mirror, std::move(pieces), generation, blocks); + return true; +} + void planSpaceMirrors(GuestBufferMemory::AddressSpace& space) { for (std::size_t index = 0; index < space.base.size(); ++index) { const auto& region = space.base[index]; @@ -1095,6 +1214,10 @@ void planSpaceMirrors(GuestBufferMemory::AddressSpace& space) { continue; } auto* mirror = region.mirror.get(); + if (mirror->sparse) { + space.sparseMirrors.push_back(mirror); + continue; + } if (space.heapRuns.empty() || mirror->base != space.heapMirrors.back()->base + space.heapMirrors.back()->bytes) space.heapRuns.push_back({mirror->base, 0, space.heapMirrors.size(), 0}); space.heapMirrors.push_back(mirror); auto& run = space.heapRuns.back(); @@ -1136,36 +1259,53 @@ std::shared_ptr acquireMirror(const Context& context, const std::sh } if (state.failed.contains(range->address)) return nullptr; const auto found = state.entries.find(range->address); - if (found != state.entries.end() && sameRange(found->second->range, range) && found->second->bytes == range->bytes && found->second->writable == range->writable && found->second->heap == heap) mirror = found->second; + if (found != state.entries.end() && sameRange(found->second->range, range) && found->second->reserved == range->bytes && found->second->writable == range->writable && found->second->heap == heap) mirror = found->second; } if (mirror != nullptr) { - if (!heap && prepareRefresh(*mirror, range->address, range->bytes)) appendBlocks(blocks, *mirror, range->address, range->bytes); - return mirror; + if (!mirror->sparse) { + if (!heap && prepareRefresh(*mirror, range->address, range->bytes)) appendBlocks(blocks, *mirror, range->address, range->bytes); + return mirror; + } + if (refreshSparseMirror(*mirror, blocks)) return mirror; } - if (!GuestMemory::Accessible(reinterpret_cast(range->address), range->bytes, range->writable && !heap)) return nullptr; + Pieces pieces; + if (!GuestMemory::Accessible(reinterpret_cast(range->address), range->bytes, range->writable && !heap)) { + if (!heap || !sparseMirrorsEnabled()) return nullptr; + pieces = GuestMemory::DescribeCommitted(range->address, range->bytes).ranges; + if (pieces.empty()) return nullptr; + } + const bool sparse = !pieces.empty(); mirror = std::make_shared(); mirror->base = range->address; - mirror->bytes = range->bytes; + mirror->bytes = sparse ? sparseExtent(range->address, range->bytes, pieces) : range->bytes; + mirror->reserved = range->bytes; + mirror->sparse = sparse; mirror->writable = range->writable; mirror->heap = heap; mirror->range = range; if (heap) { - prepareRefresh(*mirror, range->address, range->bytes); - const auto generation = GuestMemory::CollectWrites(range->address, range->bytes); + std::uint64_t generation = 0; + if (sparse) { + syncRange(mirror->base, mirror->bytes); + if (GuestMemory::Watched(mirror->base, static_cast(mirror->bytes))) generation = collectPieces(pieces); + } else { + prepareRefresh(*mirror, range->address, range->bytes); + generation = GuestMemory::CollectWrites(range->address, range->bytes); + } constexpr std::uint64_t block = 65536; - mirror->generations.assign(static_cast(((range->address + range->bytes + block - 1) / block) - range->address / block), generation); + mirror->generations.assign(static_cast(((mirror->base + mirror->bytes + block - 1) / block) - mirror->base / block), generation); if (generation == 0) { std::lock_guard lock(state.mutex); ++state.heapUnwatched; return nullptr; } - if (heapMirrorsBeside(*range) + range->bytes > heapMirrorBudget()) { + if (heapMirrorsBeside(*range) + mirror->bytes > heapMirrorBudget()) { sweepMirrors(); - if (const auto held = heapMirrorsBeside(*range); held + range->bytes > heapMirrorBudget()) throw std::runtime_error(heapMirrorError(*range, held, "past the budget")); + if (const auto held = heapMirrorsBeside(*range); held + mirror->bytes > heapMirrorBudget()) throw std::runtime_error(heapMirrorError(*range, held, "past the budget")); } } try { - mirror->buffer = std::make_shared(context, range->bytes, VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT); + mirror->buffer = std::make_shared(context, static_cast(mirror->bytes), VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT); } catch (const std::runtime_error& error) { if (heap) throw std::runtime_error(heapMirrorError(*range, heapMirrorsBeside(*range), error.what())); std::fprintf(stderr, "[gpu] image mirror of 0x%llx+0x%llx failed: %s; falling back to copies\n", static_cast(range->address), static_cast(range->bytes), error.what()); @@ -1177,7 +1317,17 @@ std::shared_ptr acquireMirror(const Context& context, const std::sh // named for the [hooksync] attribution like the refresh's flush. { const GuestMemory::ReadSiteScope site(GuestMemory::ReadSite::MirrorRefresh); - GuestMemory::Read(range->address, mirror->buffer->Bytes()); + const auto target = mirror->buffer->Bytes().first(static_cast(mirror->bytes)); + if (!sparse) GuestMemory::Read(range->address, target); + std::size_t cursor = 0; + for (const auto& [first, last] : pieces) { + const auto at = static_cast(first - mirror->base); + std::memset(target.data() + cursor, 0, at - cursor); + GuestMemory::Read(first, target.subspan(at, static_cast(last - first))); + cursor = static_cast(last - mirror->base); + } + if (sparse) std::memset(target.data() + cursor, 0, target.size() - cursor); + mirror->committed = std::move(pieces); } if (range->writable && !heap) mirror->shadow.assign(mirror->buffer->Bytes().begin(), mirror->buffer->Bytes().end()); std::lock_guard lock(state.mutex); @@ -1268,7 +1418,7 @@ void reportMirrors() { } heapBytes = state.heapBytes; } - AgcDriver::ProfilePrint_nid_no_patch("[buffers] image mirrors: %zu ranges (%.1f MiB, %zu writable, %zu heap %.1f MiB), %llu address-based builds served, %llu descriptor sub-ranges bound, %llu rebuilds, %llu refreshes: %llu blocks compared, %llu copied, %llu refresh syncs; heap refills %llu, unwatched heaps %llu, %llu heap mirrors released for imports (%.1f MiB)\n", count, bytes / 1048576.0, writable, heaps, heapBytes / 1048576.0, static_cast(state.builds), static_cast(state.subranges), static_cast(state.rebuilds), static_cast(state.refreshes), static_cast(state.blocksCompared), static_cast(state.blocksCopied), static_cast(state.syncs), static_cast(state.heapRefills), static_cast(state.heapUnwatched), static_cast(state.heapImported), state.heapImportedBytes / 1048576.0); + AgcDriver::ProfilePrint_nid_no_patch("[buffers] image mirrors: %zu ranges (%.1f MiB, %zu writable, %zu heap %.1f MiB), %llu address-based builds served, %llu descriptor sub-ranges bound, %llu sparse heaps bound, %llu rebuilds, %llu refreshes: %llu blocks compared, %llu copied, %llu refresh syncs; heap refills %llu, unwatched heaps %llu, %llu heap mirrors released for imports (%.1f MiB)\n", count, bytes / 1048576.0, writable, heaps, heapBytes / 1048576.0, static_cast(state.builds), static_cast(state.subranges), static_cast(state.sparseBinds), static_cast(state.rebuilds), static_cast(state.refreshes), static_cast(state.blocksCompared), static_cast(state.blocksCopied), static_cast(state.syncs), static_cast(state.heapRefills), static_cast(state.heapUnwatched), static_cast(state.heapImported), state.heapImportedBytes / 1048576.0); } // Deferred lease release. The lease an address-based build takes (AcquireRegistered) is dropped by its @@ -1524,7 +1674,7 @@ std::string AddressCopyOverflow(std::vector copies, std::uint64_t l MirrorStats MirrorCounters() { auto& state = Mirrors(); std::lock_guard lock(state.mutex); - MirrorStats stats{0, state.heapBytes, state.rebuilds, state.blocksCopied, state.heapRefills, state.sweeps, state.heapChecks}; + MirrorStats stats{0, state.heapBytes, state.rebuilds, state.blocksCopied, state.heapRefills, state.sweeps, state.heapChecks, state.sparseBinds}; for (const auto& [base, mirror] : state.entries) stats.heapMirrors += mirror->heap ? 1 : 0; return stats; } @@ -1877,6 +2027,15 @@ void GuestBufferMemory::AcquireRegistered() { else if (current->importsEpoch != state.epoch) spaces.rebuiltEpoch.fetch_add(1, std::memory_order_relaxed); else hit = true; } + if (hit) { + for (auto* mirror : current->sparseMirrors) { + if (refreshSparseMirror(*mirror, blocks)) continue; + hit = false; + blocks.clear(); + spaces.current.compare_exchange_strong(current, std::shared_ptr{}); + break; + } + } if (hit) { spaces.hits.fetch_add(1, std::memory_order_relaxed); space = std::move(current); @@ -1942,7 +2101,12 @@ void GuestBufferMemory::AcquireRegistered() { lap(timing.mirrorsUs, at); if (region.mirror != nullptr) { mirrored = true; - if (region.mirror->heap) heaps.push_back(region.mirror.get()); + if (region.mirror->sparse) { + region.sparse = true; + region.backed = region.mirror->committed; + } else if (region.mirror->heap) { + heaps.push_back(region.mirror.get()); + } regions.push_back(std::move(region)); continue; } @@ -2110,8 +2274,13 @@ void GuestBufferMemory::addDescriptorRegion(std::uint64_t address, std::size_t b validate(address, bytes); const auto begin = address & ~std::uint64_t{3}; const auto end = begin == address ? address + bytes : (address + bytes + 3) & ~std::uint64_t{3}; - switch (baseOverlap(begin, end, nullptr)) { + const Region* inside = nullptr; + switch (baseOverlap(begin, end, &inside)) { case BaseOverlap::Inside: + if (!servesView(*inside, begin)) { + dissolveSpace(false); + break; + } return; case BaseOverlap::Partial: Spaces().dissolvedOverlap.fetch_add(1, std::memory_order_relaxed); @@ -2130,12 +2299,54 @@ void GuestBufferMemory::addDescriptorRegion(std::uint64_t address, std::size_t b if (!committed.whole) { // A GPU heap bound whole while the guest commits its pages on demand, or a descriptor left region.sparse = true; + region.sparseHeap = true; region.backed = std::move(committed.ranges); } regions.push_back(std::move(region)); regionsSorted = false; } +bool GuestBufferMemory::servesView(const Region& region, std::uint64_t view) { + return region.mirror == nullptr || view + 4 <= region.mirror->base + region.mirror->bytes; +} + +bool GuestBufferMemory::sparseHeapCandidate(const Region& region) { + return region.sparseHeap && region.sparse && region.mirror == nullptr && region.direct == nullptr && (region.hostBacked || region.writable) && mirrorsEnabled() && sparseMirrorsEnabled(); +} + +bool GuestBufferMemory::bindSparseMirror(Region& region, const GuestAllocations::Lease& lease) { + const auto backedEnd = region.backed.empty() ? region.begin : region.backed.back().second; + const auto neededEnd = std::min(region.end, std::max(backedEnd, std::max(region.lastView, region.begin) + 4u)); + const auto* range = containingRange(lease, region.begin, neededEnd); + if (range == nullptr || !range->releasable) return false; + std::vector blocks; + std::shared_ptr mirror; + try { + mirror = acquireMirror(context, leasedRangeOwner(lease, range->address), blocks, true); + } catch (const std::runtime_error& error) { + static std::atomic reported{0}; + if (reported.fetch_add(1, std::memory_order_relaxed) < 4) std::fprintf(stderr, "[gpu] %s; copying the sparse heap instead\n", error.what()); + return false; + } + if (mirror != nullptr && mirror->heap && !mirror->sparse) { + std::vector whole{mirror.get()}; + refreshHeapMirrors(whole, blocks); + } + compareBlocks(blocks); + if (mirror == nullptr || !mirror->heap || region.begin < mirror->base || (region.begin - mirror->base) % context.limits.minStorageBufferOffsetAlignment != 0) return false; + region.mirror = std::move(mirror); + if (!servesView(region, std::max(region.lastView, region.begin))) { + region.mirror = nullptr; + return false; + } + if (region.mirror->sparse) region.backed = clipPieces(region.mirror->committed, region.begin, region.end); + else region.backed = clipPieces({{region.mirror->base, region.mirror->base + region.mirror->bytes}}, region.begin, region.end); + region.snapshot.clear(); + std::lock_guard lock(Mirrors().mutex); + ++Mirrors().sparseBinds; + return true; +} + void GuestBufferMemory::AddWritable(std::uint64_t address, std::size_t bytes, bool atomic, bool swept) { addDescriptorRegion(address, bytes, atomic, swept); writes.emplace_back(address, address + bytes); @@ -2444,7 +2655,8 @@ void GuestBufferMemory::UploadPrepare(bool addressable) { // crossing the mirror's end cannot be served by it: this build copies the range. if (previous.mirror == nullptr) std::swap(previous, region); Require(region.mirror == nullptr, "image mirrors overlap"); - if (region.begin >= previous.begin && region.end <= previous.end) continue; + if (region.begin >= previous.begin && region.end <= previous.end && servesView(previous, std::max(region.lastView, region.begin))) continue; + if (previous.mirror->sparse) previous.backed = clipPieces(previous.mirror->committed, previous.begin, previous.end); previous.mirror = nullptr; previous.hostBacked = true; } @@ -2453,6 +2665,7 @@ void GuestBufferMemory::UploadPrepare(bool addressable) { if (region.end > previous.end) previous.direct = nullptr; previous.atomic = previous.atomic || region.atomic; previous.swept = previous.swept || region.swept; + previous.sparseHeap = previous.sparseHeap || region.sparseHeap; // A range starting before the mirror (only possible after the swap) keeps its prefix; the // earlier merged region ends at or before it, so the merged list stays sorted. previous.begin = std::min(previous.begin, region.begin); @@ -2506,6 +2719,11 @@ void GuestBufferMemory::UploadPrepare(bool addressable) { const bool stale = importsStale(context, state); for (auto& region : regions) { if (region.mirror != nullptr) continue; + if (sparseHeapCandidate(region)) { + region.direct = nullptr; + region.pending = true; + continue; + } // An import found when the lease was acquired is reused while none was dropped since. const HostImport* entry = region.direct != nullptr && state.epoch == importsEpoch ? region.direct : nullptr; region.direct = nullptr; @@ -2562,7 +2780,7 @@ void GuestBufferMemory::UploadPrepare(bool addressable) { // GuestMemory::Read stores pending GPU results first (the flush hook locks for itself). for (auto& region : regions) { if (region.pending || region.mirror != nullptr || region.direct != nullptr) continue; - if (region.writable && WritesOverlap(region.begin, static_cast(region.end - region.begin))) { + if (sparseHeapCandidate(region) || (region.writable && WritesOverlap(region.begin, static_cast(region.end - region.begin)))) { region.pending = true; continue; } @@ -2640,6 +2858,12 @@ void GuestBufferMemory::UploadFinish(bool addressable) { if (profile) refreshUs += microsecondsSince(refreshStart); continue; } + if (sparseHeapCandidate(region)) { + const auto refreshStart = profile ? std::chrono::steady_clock::now() : std::chrono::steady_clock::time_point{}; + const bool bound = bindSparseMirror(region, importRanges()); + if (profile) refreshUs += microsecondsSince(refreshStart); + if (bound) continue; + } bool copyOnGpu = false; if (context.hostImportAlignment != 0) { auto& state = Imports(); @@ -3002,8 +3226,8 @@ VkDescriptorBufferInfo GuestBufferMemory::Descriptor(std::uint64_t address, std: Require(base % 4 == 0, "a guest buffer view in a GPU owner that does not start at a DWORD boundary is not implemented"); adjustment = static_cast(offset % std::max(context.limits.minStorageBufferOffsetAlignment, 4)); auto range = ViewBytes(bytes, adjustment); - const bool cut = region.direct == nullptr && region.mirror == nullptr && region.bufferEnd != 0; - const auto end = region.direct != nullptr ? region.direct->base + region.direct->bytes : cut ? region.bufferEnd : region.end; + const auto end = region.direct != nullptr ? region.direct->base + region.direct->bytes : servedEnd(region); + const bool cut = region.direct == nullptr && end < region.end; if (cut) { Require(address - adjustment < end, "guest buffer view starts past its copy"); range = std::min(range, end - (address - adjustment)); @@ -3023,8 +3247,12 @@ ShaderRecompiler::BdaAbi::Range GuestBufferMemory::addressRange(const Region& re const auto address = region.direct != nullptr ? region.direct->address + (region.begin - region.direct->base) : region.mirror != nullptr ? region.mirror->buffer->DeviceAddress() + (region.begin - region.mirror->base) : region.buffer->DeviceAddress(); Require(region.end - region.begin <= std::numeric_limits::max() - address, "GPU address range overflow"); const auto permissions = ShaderRecompiler::BdaAbi::Read | (region.direct != nullptr && region.writable ? ShaderRecompiler::BdaAbi::Write : 0u); - const bool cut = region.direct == nullptr && region.mirror == nullptr && region.bufferEnd != 0; - return {region.begin, cut ? region.bufferEnd : region.end, address, permissions, 0}; + return {region.begin, region.direct != nullptr ? region.end : servedEnd(region), address, permissions, 0}; +} + +std::uint64_t GuestBufferMemory::servedEnd(const Region& region) { + if (region.mirror != nullptr) return std::min(region.end, region.mirror->base + region.mirror->bytes); + return region.bufferEnd != 0 ? region.bufferEnd : region.end; } std::vector GuestBufferMemory::AddressRanges() const { @@ -3151,6 +3379,25 @@ void storeChanged(std::uint64_t address, std::span current, std if (cursor < address + current.size()) unchanged(cursor, address + current.size()); } +void storeSparse(ImageMirror& mirror, std::uint64_t address, std::span current, std::span original) { + constexpr std::uint64_t block = 65536; + const auto end = address + current.size(); + const auto forget = [&](std::uint64_t from, std::uint64_t to) { + const auto at = static_cast(from - address); + if (from >= to || std::memcmp(current.data() + at, original.data() + at, static_cast(to - from)) == 0) return; + for (auto index = from / block - mirror.base / block; index <= (to - 1) / block - mirror.base / block; ++index) mirror.generations[static_cast(index)] = 0; + }; + auto cursor = address; + for (const auto& [first, last] : GuestMemory::DescribeCommitted(address, current.size(), true).ranges) { + forget(cursor, first); + const auto at = static_cast(first - address); + const auto length = static_cast(last - first); + GuestMemory::WriteChanged(first, current.subspan(at, length), original.subspan(at, length)); + cursor = last; + } + forget(cursor, end); +} + std::vector> mergeWrites(std::vector> writes) { std::sort(writes.begin(), writes.end()); std::vector> merged; @@ -3170,7 +3417,9 @@ void GuestBufferMemory::takeHeapReferences() { if (found == nullptr || found->mirror == nullptr || !found->mirror->heap || !found->mirror->writable) continue; Require(end <= found->end, "written range exceeds its heap mirror"); const auto& mirror = *found->mirror; - const auto bytes = mirror.buffer->Bytes().subspan(static_cast(begin - mirror.base), static_cast(end - begin)); + const auto served = std::min(end, servedEnd(*found)); + if (begin >= served) continue; + const auto bytes = mirror.buffer->Bytes().subspan(static_cast(begin - mirror.base), static_cast(served - begin)); heapReferences.emplace_back(begin, std::vector(bytes.begin(), bytes.end())); } } @@ -3198,12 +3447,15 @@ void GuestBufferMemory::WriteBack() { // refresh does not mistake the GPU's results for a guest change. auto& mirror = *region.mirror; const auto offset = static_cast(begin - mirror.base); - const auto length = static_cast(end - begin); + const auto served = std::min(end, servedEnd(region)); + if (mirror.heap && begin >= served) continue; + const auto length = static_cast((mirror.heap ? served : end) - begin); const auto current = mirror.buffer->Bytes().subspan(offset, length); if (mirror.heap) { const auto reference = std::find_if(heapReferences.begin(), heapReferences.end(), [&](const auto& entry) { return entry.first == begin; }); Require(reference != heapReferences.end() && reference->second.size() == length, "heap mirror write-back has no reference"); - storeChanged(begin, current, reference->second); + if (mirror.sparse) storeSparse(mirror, begin, current, reference->second); + else storeChanged(begin, current, reference->second); continue; } GuestMemory::WriteChanged(begin, current, std::span(mirror.shadow).subspan(offset, length)); diff --git a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp index 22ab44c39..2613ff889 100644 --- a/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/BdaResources.cpp @@ -714,6 +714,128 @@ void sparseHeapTests(const Context& context, const BdaTestAccess& access) { releaseSparse(block, reserved); } +void sparseHeapMirrorTests(const Context& context, const BdaTestAccess& access) { +#ifdef _WIN32 + if (!GuestArena::GuestArenaAvailable_nid_postfix() || !GuestArena::GuestArenaWriteWatched_nid_postfix()) { + std::cout << "guest arena unavailable or not write-watched: sparse heap mirrors not tested\n"; + return; + } + constexpr std::size_t reserved = 64u << 20u; + constexpr std::size_t committed = 2 * 65536; + constexpr std::size_t extent = 1u << 20u; + constexpr std::size_t late = 512u << 10u; + constexpr std::size_t grown = 2u << 20u; + constexpr std::size_t past = 3584u << 10u; + void* block = GuestArena::GuestArenaAllocate_nid_postfix(reserved, 65536); + auto* guest = static_cast(block); + GuestArena::GuestArenaCommit_nid_postfix(block, committed, PAGE_READWRITE, 65536); + std::memset(block, 0x5a, committed); + const auto base = reinterpret_cast(block); + const auto registry = [&](bool add) { + auto* mutation = GuestAllocations::GuestAllocationsBegin_nid_postfix(); + if (add) GuestAllocations::GuestAllocationsAdd_nid_postfix(mutation, block, reserved, true, true, true); + else GuestAllocations::GuestAllocationsRemove_nid_postfix(mutation, block); + GuestAllocations::GuestAllocationsEnd_nid_postfix(mutation); + }; + const auto bind = [&](bool written, const std::function, const VkDescriptorBufferInfo&)>& gpu) { + GuestBufferMemory memory(context); + if (written) memory.AddWritable(base, reserved); + else memory.AddReadable(base, reserved); + memory.Upload(false); + std::uint32_t adjustment = 0; + const auto view = memory.Descriptor(base, reserved, adjustment); + Require(adjustment == 0, "a sparse heap view at its base is adjusted"); + gpu(access.bytes(view.buffer), view); + memory.WriteBack(); + return view; + }; + const auto address = [&] { + GuestBufferMemory memory(context); + memory.AcquireRegistered(); + memory.Upload(true); + const auto ranges = memory.AddressRanges(); + const auto found = std::find_if(ranges.begin(), ranges.end(), [&](const auto& range) { return range.begin == base; }); + const auto result = found != ranges.end() ? std::pair{found->end, found->deviceAddress} : std::pair{0, 0}; + memory.WriteBack(); + return result; + }; + const auto sweep = [&] { + GuestBufferMemory memory(context); + memory.AcquireRegistered(); + memory.Upload(true); + memory.WriteBack(); + }; + sweep(); + registry(true); + const auto before = MirrorCounters(); + const auto first = bind(true, [&](std::span bytes, const VkDescriptorBufferInfo& view) { + Require(view.range == extent && bytes[view.offset + committed - 1] == std::byte{0x5a} && bytes[view.offset + committed] == std::byte{}, "a sparse heap is not bound to a mirror of its committed extent"); + bytes[view.offset + 8] = std::byte{0x77}; + guest[24] = 0x55; + }); + const auto made = MirrorCounters(); + Require(made.heapMirrors == before.heapMirrors + 1 && made.heapBytes == before.heapBytes + extent && made.rebuilds == before.rebuilds + 1 && made.sparseBinds == before.sparseBinds + 1, "a sparse heap was not mirrored once"); + Require(guest[8] == 0x77 && guest[24] == 0x55, "a sparse heap mirror's write-back lost the GPU's store or rolled back the CPU's"); + const auto second = bind(false, [&](std::span bytes, const VkDescriptorBufferInfo& view) { + Require(bytes[view.offset + 8] == std::byte{0x77} && bytes[view.offset + 24] == std::byte{0x55}, "the sparse heap mirror missed the stores"); + }); + const auto again = MirrorCounters(); + Require(second.buffer == first.buffer && again.rebuilds == made.rebuilds && again.blocksCopied == made.blocksCopied + 1, "a sparse heap bound again was read again beyond its written block"); + bind(false, [](std::span, const VkDescriptorBufferInfo&) {}); + Require(MirrorCounters().blocksCopied == again.blocksCopied && MirrorCounters().rebuilds == again.rebuilds, "an unchanged sparse heap was read again"); + GuestArena::GuestArenaCommit_nid_postfix(guest + late, 65536, PAGE_READWRITE, 65536); + guest[late + 4] = 0x66; + bind(false, [&](std::span bytes, const VkDescriptorBufferInfo& view) { + Require(view.buffer == first.buffer && bytes[view.offset + late + 4] == std::byte{0x66} && bytes[view.offset + late + 65536] == std::byte{}, "a page committed after the mirror was made was not read into it"); + }); + const auto [end, device] = address(); + Require(end == base + extent && device != 0 && access.addressBytes(device)[late + 4] == std::byte{0x66} && MirrorCounters().rebuilds == again.rebuilds, "an address-based build does not map the sparse heap mirror up to its extent"); + GuestArena::GuestArenaCommit_nid_postfix(guest + grown, 65536, PAGE_READWRITE, 65536); + guest[grown + 2] = 0x44; + const auto [grownEnd, grownDevice] = address(); + Require(grownEnd == base + 3 * extent && grownDevice != device && access.addressBytes(grownDevice)[grown + 2] == std::byte{0x44} && access.addressBytes(grownDevice)[8] == std::byte{0x77}, "a sparse heap committed past its mirror was not mirrored again"); + { + GuestBufferMemory memory(context); + memory.AddReadable(base, reserved); + memory.AddReadable(base + past, 16); + memory.Upload(false); + std::uint32_t adjustment = 0; + const auto view = memory.Descriptor(base + past, 16, adjustment); + const auto whole = memory.Descriptor(base, reserved, adjustment); + Require(view.buffer == whole.buffer && access.bytes(whole.buffer)[whole.offset + grown + 2] == std::byte{0x44}, "a view past the sparse heap mirror is not copied with the heap"); + memory.WriteBack(); + } + registry(false); + sweep(); + Require(MirrorCounters().heapMirrors == before.heapMirrors && MirrorCounters().heapBytes == before.heapBytes, "the sparse heap mirror outlived its range"); + { + constexpr std::size_t whole = 1u << 20u; + GuestArena::GuestArenaReset_nid_postfix(block, reserved); + GuestArena::GuestArenaCommit_nid_postfix(guest, whole, PAGE_READWRITE, 65536); + std::memset(guest, 0x31, whole); + auto* mutation = GuestAllocations::GuestAllocationsBegin_nid_postfix(); + GuestAllocations::GuestAllocationsAdd_nid_postfix(mutation, block, whole, true, true, true); + GuestAllocations::GuestAllocationsEnd_nid_postfix(mutation); + const auto start = MirrorCounters(); + const auto view = bind(true, [&](std::span bytes, const VkDescriptorBufferInfo& info) { + Require(info.range == whole, "a heap bound whole over a smaller registered range is not clamped to that range"); + Require(bytes[info.offset + whole - 1] == std::byte{0x31}, "a heap bound whole over a smaller registered range misses its last committed byte"); + bytes[info.offset + 16] = std::byte{0x12}; + }); + Require(view.range == whole && MirrorCounters().sparseBinds == start.sparseBinds + 1 && guest[16] == 0x12, "a heap bound whole over a smaller registered range was copied"); + mutation = GuestAllocations::GuestAllocationsBegin_nid_postfix(); + GuestAllocations::GuestAllocationsRemove_nid_postfix(mutation, block); + GuestAllocations::GuestAllocationsEnd_nid_postfix(mutation); + sweep(); + } + GuestArena::GuestArenaReset_nid_postfix(block, reserved); + GuestArena::GuestArenaRelease_nid_postfix(block, reserved); +#else + static_cast(context); + static_cast(access); +#endif +} + void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { alignas(64) std::array guest{}; guest[0] = 123; @@ -922,6 +1044,7 @@ void RunBdaResourceTests(const Context& context, const BdaTestAccess& access) { gpuMappingTests(context); stridedOverhangTests(context, access); sparseHeapTests(context, access); + sparseHeapMirrorTests(context, access); Require(AddressCopyOverflow({{0x1000, 0x3000, 0x2000, "uncommitted pages"}}, 0x2000).empty(), "copies within the limit were refused"); const auto copies = AddressCopyOverflow({{0x1000, 0x2000, 0x1000, "not mirrored"}, {0x10000, 0x30000, 0x18000, "uncommitted pages"}}, 0x2000); Require(!copies.empty() && copies.find("0x10000+0x20000 (0.1 MiB committed, uncommitted pages)") < copies.find("0x1000+0x1000"), "the copy limit does not name the largest copy first"); From 38012ec35c4790f1e0df777c17d7fe75cc28ae70 Mon Sep 17 00:00:00 2001 From: Chad Date: Sat, 10 Oct 2026 19:05:04 -0500 Subject: [PATCH 426/460] perf(agc): drop a comment the conventions check refuses --- .../libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp | 2 -- 1 file changed, 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp index 0e8359f04..251a48902 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp @@ -1029,8 +1029,6 @@ void syncRange(std::uint64_t address, std::uint64_t bytes) { } void prepareRange(std::uint64_t address, std::uint64_t bytes) { - // The compare reads the pages directly; a mirror is only reused while its Range exists, but a - // lease of a build in flight can keep a replaced Range alive past a protection change. Require(GuestMemory::Accessible(reinterpret_cast(address), static_cast(bytes)), "image mirror range became inaccessible"); syncRange(address, bytes); } From f65e63fafa1844c5c6937c9c90e285058eed20f4 Mon Sep 17 00:00:00 2001 From: David Date: Sun, 11 Oct 2026 09:50:41 +0200 Subject: [PATCH 427/460] test(libSceJobManager): allocate aligned test blocks without std::aligned_alloc MinGW-w64's UCRT has no aligned_alloc, so the Windows build of guest_job_manager_tests fails since #3170. Use the aligned operator new and delete instead, which are standard C++17 on every host. Co-Authored-By: Claude Fable 5.1 --- core/libs/tests/GuestJobManager.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/core/libs/tests/GuestJobManager.cpp b/core/libs/tests/GuestJobManager.cpp index 0dca62187..e82703be1 100644 --- a/core/libs/tests/GuestJobManager.cpp +++ b/core/libs/tests/GuestJobManager.cpp @@ -4,6 +4,7 @@ #include #include #include +#include #include #include #include @@ -97,8 +98,8 @@ static void Check(bool value, int line) { #define Require(value) Check((value), __LINE__) struct Block { - explicit Block(std::uint64_t size) : bytes(size), memory(std::aligned_alloc(256, (size + 255) & ~std::uint64_t{255})) {} - ~Block() { std::free(memory); } + explicit Block(std::uint64_t size) : bytes(size), memory(::operator new((size + 255) & ~std::uint64_t{255}, std::align_val_t{256})) {} + ~Block() { ::operator delete(memory, std::align_val_t{256}); } std::uint64_t bytes; void* memory; }; From 6d9321fc57ab1d4698d390acebcdc675a7607d0b Mon Sep 17 00:00:00 2001 From: krishnaraya Date: Sun, 11 Oct 2026 01:52:30 -0400 Subject: [PATCH 428/460] feat(shader): link compute shaders' user-data callees at dispatch Co-Authored-By: Claude Opus 5.5 (1M context) --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 5 + .../src/Driver/Shaders/ShaderRegistry.cpp | 68 +++++- .../tests/execution/ComputeUserDataCalls.cpp | 135 +++++++++++ core/libs/tests/ScalarCall.cpp | 86 +++++++ core/shader/recompiler/CMakeLists.txt | 3 + .../include/ControlFlow/GraphBuilder.hpp | 10 + .../ControlFlow/src/GraphBuilder.cpp | 221 ++++++++++++++++-- docs/dev/TechnicalDebt.md | 5 +- 8 files changed, 513 insertions(+), 20 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/ComputeUserDataCalls.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 849984a3a..273bb2a0a 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -1076,6 +1076,11 @@ if(BUILD_TESTING) set_tests_properties(${preparedTest} PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() endforeach() + add_test(NAME agc_driver_compute_user_data_calls_registered COMMAND agc_driver_compute_user_data_calls_tests --registered) + set_tests_properties(agc_driver_compute_user_data_calls_registered PROPERTIES ENVIRONMENT "ANYPS5_NO_SHADER_CACHE=1" SKIP_RETURN_CODE 77 WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}") + if(WIN32) + set_tests_properties(agc_driver_compute_user_data_calls_registered PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") + endif() set_tests_properties(agc_driver_prepared_shaders agc_driver_prepared_shaders_indirect agc_driver_prepared_shaders_uncached agc_driver_prepared_shaders_debug agc_driver_prepared_shaders_debug_failure diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp index 2d564856c..c8e298cab 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp @@ -357,24 +357,82 @@ ShaderRecompiler::RectListShaders DrawRectangle(const ShaderSnapshot& front, con return PreparedRectangle(front, vertexId, fragmentId); } +namespace { + +std::string hexText(std::uint64_t value) { + char text[24]; + std::snprintf(text, sizeof(text), "0x%llx", static_cast(value)); + return text; +} + +std::vector ReadUserDataCallee(std::uint64_t address, std::uint32_t linkRegister) { + constexpr std::uint64_t page = 4096u; + constexpr std::size_t limitWords = 16384u; + if (address % 4u != 0u) throw std::runtime_error("its callee address " + hexText(address) + " is not dword aligned"); + std::vector words; + while (words.size() < limitWords) { + const auto next = address + words.size() * 4u; + const auto bytes = static_cast(page - next % page); + if (!GuestMemory::Accessible(reinterpret_cast(static_cast(next)), bytes)) throw std::runtime_error("its callee at " + hexText(address) + " is not readable at " + hexText(next)); + const auto old = words.size(); + words.resize(old + bytes / 4u); + GuestMemory::Read(next, std::as_writable_bytes(std::span(words).subspan(old))); + if (const auto length = ShaderRecompiler::UserDataCalleeLength(words, linkRegister)) { + words.resize(*length); + return words; + } + } + throw std::runtime_error("its callee at " + hexText(address) + " does not return within " + std::to_string(limitWords * 4u) + " bytes"); +} + +// A compute shader that calls a user-data pointer gets one copy of each callee after its code, read at +// the address the call's user data holds now; the prepared key follows the linked code. +std::vector LinkUserDataCallees(const ShaderRecompiler::RecompileRequest& request) { + if (request.shader.stage != ShaderRecompiler::ShaderStage::Compute) return {}; + const auto code = request.shader.code; + if (std::none_of(code.begin(), code.end(), [](std::uint32_t word) { return (word & 0xff80ff00u) == 0xbe802100u; })) return {}; + const auto program = ShaderRecompiler::RdnaInstructionDecoder{}.Decode(code); + const ShaderRecompiler::SwappcInfo swappc{false, request.context.userDataBaseRegister, static_cast(request.context.userData.size())}; + const auto calls = ShaderRecompiler::FindUserDataCalls(program, swappc); + if (calls.empty()) return {}; + std::vector> callees; + for (const auto& call : calls) { + const auto slot = call.userDataRegister - swappc.userDataBaseRegister; + const auto address = static_cast(request.context.userData[slot]) | static_cast(request.context.userData[slot + 1u]) << 32u; + try { + callees.push_back(ReadUserDataCallee(address, call.linkRegister)); + } catch (const std::exception& error) { + throw std::runtime_error("AGC driver: compute shader " + hexText(request.shader.codeAddress) + " calls user data s[" + std::to_string(call.userDataRegister) + ":" + + std::to_string(call.userDataRegister + 1u) + "] at program counter " + hexText(call.programCounter) + ", but " + error.what()); + } + } + return ShaderRecompiler::LinkUserDataCalls(program, calls, callees); +} + +} + ShaderRecompiler::PreparedShaderInvocation InvocationFor(const ShaderSnapshot& snapshot, std::size_t codeOffset, const ShaderRecompiler::RecompileRequest& request) { require(codeOffset < snapshot.code.size(), "prepared shader code offset is outside the snapshot"); const auto code = std::span(snapshot.code).subspan(codeOffset); require(request.shader.code.data() == code.data() && request.shader.code.size() == code.size(), "prepared invocation does not refer to registered code"); - auto invocationRequest = request; + const auto linked = LinkUserDataCallees(request); + auto source = request; + if (!linked.empty()) source.shader.code = linked; + auto invocationRequest = source; struct PreparedKeyStorage {}; auto& key = HostThreadLocal, PreparedKeyStorage>(); - ShaderRecompiler::BuildPreparedShaderKey(request, key); + ShaderRecompiler::BuildPreparedShaderKey(source, key); std::unique_lock lock(snapshot.prepared->mutex); AwaitRegisteredPreparation(*snapshot.prepared, lock); for (const auto& entry : snapshot.prepared->entries) { if (entry.codeOffset != codeOffset) continue; invocationRequest.shader.code = ShaderRecompiler::GetPreparedCode(*entry.handle); + if (!linked.empty() && !std::ranges::equal(invocationRequest.shader.code, linked)) continue; if (auto invocation = ShaderRecompiler::PreparedShaderInvocation::TryCreate(invocationRequest, entry.handle, key)) return std::move(*invocation); } - ReportPreparedAtUse(snapshot, request); - auto handle = PrepareShaderWithDiagnostics(request); - invocationRequest = request; + ReportPreparedAtUse(snapshot, source); + auto handle = PrepareShaderWithDiagnostics(source); + invocationRequest = source; invocationRequest.shader.code = ShaderRecompiler::GetPreparedCode(*handle); auto invocation = ShaderRecompiler::PreparedShaderInvocation::TryCreate(invocationRequest, handle, key); if (!invocation.has_value()) throw std::runtime_error("AGC driver: an artifact prepared at use does not match its invocation"); diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/ComputeUserDataCalls.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/ComputeUserDataCalls.cpp new file mode 100644 index 000000000..f69cfaf20 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/ComputeUserDataCalls.cpp @@ -0,0 +1,135 @@ +#include "prx/libSceAgcDriver/Execution/include/Driver.hpp" +#include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "SceShaders.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using AgcDriver::Graphics::Require; + +constexpr std::uint32_t Threads = 64; +constexpr std::uint32_t Untouched = 0xcafebabeu; + +alignas(256) constexpr std::array Caller{ + 0x7e020280u, 0xbe940383u, 0xbe8e0300u, 0xbe8f0301u, 0xbe8e210eu, 0xbe8e0302u, 0xbe8f0303u, 0xbe8e210eu, + 0x80948114u, 0xbf078014u, 0xbf85fff7u, 0x34040082u, 0xe0701000u, 0x80010102u, 0xbf810000u, 0xbf9f0000u, + 0x30306c73u, 0x00000040u, 0x00000008u}; +alignas(256) constexpr std::array AddFive{0x4a020285u, 0xbe80200eu}; +alignas(256) constexpr std::array AddSixtyFour{0x4a0202c0u, 0xbe80200eu}; +alignas(256) constexpr std::array AddOne{0x4a020281u, 0xbe80200eu}; +alignas(256) std::array Output{}; + +std::array UserData(const void* first, const void* second) { + const auto low = reinterpret_cast(first); + const auto high = reinterpret_cast(second); + const auto output = reinterpret_cast(Output.data()); + return {static_cast(low), static_cast(low >> 32u), static_cast(high), static_cast(high >> 32u), + static_cast(output), static_cast((output >> 32u) & 0xffffu), static_cast(sizeof(Output)), 0x31016facu}; +} + +std::vector Commands(const std::array& data, bool wave32) { + std::vector words; + const auto registers = [&](std::uint32_t first, std::span values) { + words.push_back(0xc0007600u | (static_cast(values.size()) << 16u)); + words.push_back(first); + words.insert(words.end(), values.begin(), values.end()); + }; + const std::array shape{Threads, 1, 1}; + registers(0x207, shape); + const auto address = reinterpret_cast(Caller.data()); + const std::array program{static_cast(address >> 8u), static_cast(address >> 40u)}; + registers(0x20c, program); + const std::array resource{static_cast(data.size()) << 1u}; + registers(0x213, resource); + registers(0x240, data); + words.insert(words.end(), {0xc0031500u, 1, 1, 1, 0x41u | (wave32 ? 0x8000u : 0u)}); + return words; +} + +void Dispatch(const void* first, const void* second, bool wave32) { + Output.fill(Untouched); + const auto words = Commands(UserData(first, second), wave32); + Packet packet{const_cast(words.data()), static_cast(words.size()), 0, {}}; + AgcDriver::Submit(&packet, 0); + AgcDriverWaitIdle_nid_postfix(); + AgcDriver::GuestMemory::FlushGpuWrites(reinterpret_cast(Output.data()), sizeof(Output)); +} + +void Check(const void* first, const void* second, bool wave32, std::uint32_t expected, const std::string& what) { + Dispatch(first, second, wave32); + for (std::uint32_t lane = 0; lane < Threads; ++lane) { + Require(Output[lane] == expected, what + (wave32 ? " (wave32)" : " (wave64)") + ": lane " + std::to_string(lane) + " is " + std::to_string(Output[lane]) + ", expected " + std::to_string(expected)); + } +} + +void Register(bool wave32) { + struct Header { + Shader shader{}; + std::array registers{}; + ShaderSpecialRegs specials{}; + }; + alignas(8) static Header header{}; + const auto address = reinterpret_cast(Caller.data()); + header.shader.file_header = 0x34333231u; + header.shader.version = 0x18; + header.shader.header_size = sizeof(Header); + header.shader.shader_size = sizeof(Caller); + header.shader.code = Caller.data(); + header.shader.sh_registers = header.registers.data(); + header.shader.num_sh_registers = header.registers.size(); + header.shader.specials = &header.specials; + header.specials.dispatch_modifier = wave32 ? 0x8000u : 0u; + header.registers = {{{0x20c, static_cast(address >> 8u)}, {0x20d, static_cast(address >> 40u)}, {0x207, Threads}, {0x208, 1}, {0x209, 1}, {0x212, 0}, {0x213, 8u << 1u}}}; + AgcDriverRegisterShader_nid_postfix(&header.shader); +} + +template +void ExpectFailure(TAction action, const char* expected) { + try { + action(); + } catch (const std::exception& error) { + Require(std::string(error.what()).find(expected) != std::string::npos, std::string("unexpected failure: ") + error.what()); + return; + } + throw std::runtime_error(std::string("expected a failure containing ") + expected); +} + +} + +int main(int argc, char** argv) { + try { + Require(argc == 1 || (argc == 2 && std::string(argv[1]) == "--registered"), "usage: [--registered]"); + const bool registered = argc == 2; + { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + } + if (registered) Register(true); + for (const bool wave32 : registered ? std::vector{true} : std::vector{false, true}) { + Check(AddFive.data(), AddSixtyFour.data(), wave32, 3u * (5u + 64u), "two user-data callees called from a loop"); + Check(AddOne.data(), AddSixtyFour.data(), wave32, 3u * (1u + 64u), "the first callee replaced"); + Check(AddSixtyFour.data(), AddSixtyFour.data(), wave32, 3u * (64u + 64u), "one callee behind both pointers"); + } + const auto* unmapped = reinterpret_cast(std::uintptr_t{0x1000u}); + ExpectFailure([&] { Dispatch(AddFive.data(), unmapped, registered); }, "is not readable"); + ExpectFailure([] { AgcDriverShutdown_nid_postfix(); }, "is not readable"); + std::puts("compute user-data call tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + try { + AgcDriverShutdown_nid_postfix(); + } catch (const std::exception&) { + } + return 1; + } +} diff --git a/core/libs/tests/ScalarCall.cpp b/core/libs/tests/ScalarCall.cpp index 3058f6577..e797b7aeb 100644 --- a/core/libs/tests/ScalarCall.cpp +++ b/core/libs/tests/ScalarCall.cpp @@ -6,10 +6,13 @@ #include "ControlFlow/GraphBuilder.hpp" #include #include +#include #include +#include #include #include #include +#include using namespace ShaderRecompiler; @@ -96,6 +99,88 @@ static void CheckCallAfterEnd() { static_cast(InstructionTranslator{}.Translate(decoded, cfg, options)); } +static std::vector UserDataCaller() { + return {0x7e020280u, 0xbe940383u, 0xbe8e0300u, 0xbe8f0301u, 0xbe8e210eu, 0xbe8e0302u, 0xbe8f0303u, 0xbe8e210eu, + 0x80948114u, 0xbf078014u, 0xbf85fff7u, 0xbf810000u, 0x30306c73u}; +} + +static std::vector UserDataCalls(const std::vector& code, std::uint32_t userDataCount = 8u) { + const SwappcInfo swappc{false, 0u, userDataCount}; + return FindUserDataCalls(RdnaInstructionDecoder{}.Decode(code), swappc); +} + +static void CheckUserDataCalls() { + const SwappcInfo swappc{false, 0u, 8u}; + const auto caller = UserDataCaller(); + const auto program = RdnaInstructionDecoder{}.Decode(caller); + const auto calls = FindUserDataCalls(program, swappc); + Require(calls.size() == 2u); + Require(calls[0].programCounter == 0x10u && calls[0].linkRegister == 14u && calls[0].userDataRegister == 0u); + Require(calls[1].programCounter == 0x1cu && calls[1].linkRegister == 14u && calls[1].userDataRegister == 2u); + ExpectThrow("not statically resolvable", [&] { static_cast(GraphBuilder{}.Build(program, &swappc)); }); + + auto overwritten = caller; + overwritten[1] = 0xbe800380u; + const auto second = UserDataCalls(overwritten); + Require(second.size() == 1u && second[0].programCounter == 0x1cu); + auto entered = caller; + entered[10] = 0xbf85fff8u; + const auto afterLabel = UserDataCalls(entered); + Require(afterLabel.size() == 1u && afterLabel[0].programCounter == 0x1cu); + auto looped = caller; + looped[8] = 0xbe820380u; + const auto beforeWrite = UserDataCalls(looped); + Require(beforeWrite.size() == 1u && beforeWrite[0].programCounter == 0x10u); + Require(UserDataCalls({0xbe842104u, 0xbf810000u}, 6u).empty()); + const auto direct = UserDataCalls({0xbe8e2104u, 0xbf810000u}, 6u); + Require(direct.size() == 1u && direct[0].linkRegister == 14u && direct[0].userDataRegister == 4u); + + const std::vector addFive{0x4a020285u, 0xbe80200eu}; + const std::vector addSixtyFour{0x4a0202c0u, 0xbe80200eu}; + Require(UserDataCalleeLength(addFive, 14u) == 2u); + Require(!UserDataCalleeLength(std::vector{0x4a020285u}, 14u).has_value()); + ExpectThrow("reads its program counter", [] { static_cast(UserDataCalleeLength(std::vector{0xbe901f00u, 0xbe80200eu}, 14u)); }); + ExpectThrow("makes a call", [] { static_cast(UserDataCalleeLength(std::vector{0xbe92210eu, 0xbe80200eu}, 14u)); }); + ExpectThrow("ends the program", [] { static_cast(UserDataCalleeLength(std::vector{0xbf810000u}, 14u)); }); + ExpectThrow("jumps through another register pair", [] { static_cast(UserDataCalleeLength(std::vector{0xbe802010u}, 14u)); }); + ExpectThrow("before code it branches to", [] { static_cast(UserDataCalleeLength(std::vector{0xbf840001u, 0xbe80200eu, 0x4a020285u, 0xbe80200eu}, 14u)); }); + + const std::vector> callees{addFive, addSixtyFour}; + const auto linked = LinkUserDataCalls(program, calls, callees); + Require(linked.size() == 17u && linked[4] == 0xbb0e0007u && linked[7] == 0xbb0e0006u && linked[16] == 0xbf9f0000u); + Require(std::equal(addFive.begin(), addFive.end(), linked.begin() + 12) && std::equal(addSixtyFour.begin(), addSixtyFour.end(), linked.begin() + 14)); + auto readsPc = caller; + readsPc[1] = 0xbe941f00u; + ExpectThrow("cannot have user-data callees linked", [&] { + const auto readsPcProgram = RdnaInstructionDecoder{}.Decode(readsPc); + static_cast(LinkUserDataCalls(readsPcProgram, FindUserDataCalls(readsPcProgram, swappc), callees)); + }); + + const auto linkedProgram = RdnaInstructionDecoder{}.Decode(linked); + const auto cfg = GraphBuilder{}.Build(linkedProgram, &swappc); + const auto block = [&](std::uint32_t programCounter) { return cfg.FindBlockByProgramCounter(programCounter).id; }; + Require(cfg.FindBlockByProgramCounter(0x08u).terminator.trueBlock == block(0x30u)); + Require(cfg.FindBlockByProgramCounter(0x30u).terminator.trueBlock == block(0x14u)); + Require(cfg.FindBlockByProgramCounter(0x14u).terminator.trueBlock == block(0x38u)); + Require(cfg.FindBlockByProgramCounter(0x38u).terminator.trueBlock == block(0x20u)); + ShaderComputeInputInfo computeInfo{}; + TranslateOptions options{}; + options.stage = ShaderStageKind::Compute; + options.inputInfo.compute = &computeInfo; + options.userDataCount = 8u; + static_cast(InstructionTranslator{}.Translate(linkedProgram, cfg, options)); + + auto readsLink = linked; + readsLink[8] = 0xbe94030eu; + ExpectThrow("escapes the constant-offset call/return model", [&] { static_cast(GraphBuilder{}.Build(RdnaInstructionDecoder{}.Decode(readsLink), &swappc)); }); + auto writesInside = linked; + writesInside[12] = 0xbe8e0300u; + ExpectThrow("escapes the constant-offset call/return model", [&] { static_cast(GraphBuilder{}.Build(RdnaInstructionDecoder{}.Decode(writesInside), &swappc)); }); + auto sameCopy = linked; + sameCopy[7] = 0xbb0e0004u; + ExpectThrow("is shared with another call", [&] { static_cast(GraphBuilder{}.Build(RdnaInstructionDecoder{}.Decode(sameCopy), &swappc)); }); +} + static void CheckRejections() { ExpectThrow("invalid instruction boundary", [] { Build({Call(0u, 8u, 16u), Sopp(4u, 1u)}); @@ -194,6 +279,7 @@ int main(int argc, char** argv) { CheckLink(); CheckCallAfterEnd(); CheckRejections(); + CheckUserDataCalls(); CheckGuards(""); } else { Require(argc == 2); diff --git a/core/shader/recompiler/CMakeLists.txt b/core/shader/recompiler/CMakeLists.txt index 7e9584de0..727e2b4d3 100644 --- a/core/shader/recompiler/CMakeLists.txt +++ b/core/shader/recompiler/CMakeLists.txt @@ -240,6 +240,9 @@ function(add_shader_recompiler target runtimeTarget) SrgbUnormFormat ThreadOwned UnresolvableSwappcTarget + FindUserDataCalls + UserDataCalleeLength + LinkUserDataCalls WriteFileAtomically ) set(shaderCacheManifest) diff --git a/core/shader/recompiler/ControlFlow/include/ControlFlow/GraphBuilder.hpp b/core/shader/recompiler/ControlFlow/include/ControlFlow/GraphBuilder.hpp index 1b450325e..ada6f1084 100644 --- a/core/shader/recompiler/ControlFlow/include/ControlFlow/GraphBuilder.hpp +++ b/core/shader/recompiler/ControlFlow/include/ControlFlow/GraphBuilder.hpp @@ -5,6 +5,7 @@ #include "RdnaDecoder/RdnaProgram.hpp" #include #include +#include #include namespace ShaderRecompiler { @@ -25,7 +26,16 @@ struct SwappcCall { std::uint32_t returnTargetProgramCounter = 0; }; +struct UserDataCall { + std::uint32_t programCounter = 0; + std::uint32_t linkRegister = 0; + std::uint32_t userDataRegister = 0; +}; + [[nodiscard]] std::optional UnresolvableSwappcTarget(const RdnaProgram& program, const SwappcInfo* swappc); +[[nodiscard]] std::vector FindUserDataCalls(const RdnaProgram& program, const SwappcInfo& swappc); +[[nodiscard]] std::optional UserDataCalleeLength(std::span callee, std::uint32_t linkRegister); +[[nodiscard]] std::vector LinkUserDataCalls(const RdnaProgram& program, std::span calls, std::span> callees); class GraphBuilder { public: diff --git a/core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp b/core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp index 0128c6474..3b623565e 100644 --- a/core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp +++ b/core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp @@ -1,5 +1,6 @@ #include "ControlFlow/GraphBuilder.hpp" #include "ControlFlow/ControlFlowHelpers.hpp" +#include "RdnaDecoder/RdnaInstructionDecoder.hpp" #include #include #include @@ -370,7 +371,7 @@ bool pairOverlapsRegister(const RdnaOperand& operand, std::uint32_t linkRegister return slot != NoScalarRegister && slot <= linkRegister + 1u && linkRegister < slot + count; } -bool touchesLinkRegister(const RdnaInstruction& instruction, std::uint32_t linkRegister) { +bool readsLinkRegister(const RdnaInstruction& instruction, std::uint32_t linkRegister) { if (instruction.op == RdnaOpcode::SMovrelsB32 || instruction.op == RdnaOpcode::SMovrelsB64) return true; std::uint32_t source0Count = 1u, source1Count = 1u, source2Count = 1u; switch (instruction.family) { @@ -397,10 +398,36 @@ bool touchesLinkRegister(const RdnaInstruction& instruction, std::uint32_t linkR break; default: break; } + return pairOverlapsRegister(instruction.source0, linkRegister, source0Count) || pairOverlapsRegister(instruction.source1, linkRegister, source1Count) || + pairOverlapsRegister(instruction.source2, linkRegister, source2Count) || pairOverlapsRegister(instruction.source3, linkRegister); +} + +bool touchesLinkRegister(const RdnaInstruction& instruction, std::uint32_t linkRegister) { return writesScalar(instruction, linkRegister) || writesScalar(instruction, linkRegister + 1u) || pairOverlapsRegister(instruction.destination, linkRegister) || pairOverlapsRegister(instruction.destination2, linkRegister) || - pairOverlapsRegister(instruction.source0, linkRegister, source0Count) || pairOverlapsRegister(instruction.source1, linkRegister, source1Count) || - pairOverlapsRegister(instruction.source2, linkRegister, source2Count) || pairOverlapsRegister(instruction.source3, linkRegister); + readsLinkRegister(instruction, linkRegister); +} + +bool isScalarLoad(RdnaOpcode op) { + switch (op) { + case RdnaOpcode::SLoadDword: + case RdnaOpcode::SLoadDwordx2: + case RdnaOpcode::SLoadDwordx4: + case RdnaOpcode::SLoadDwordx8: + case RdnaOpcode::SLoadDwordx16: + case RdnaOpcode::SBufferLoadDword: + case RdnaOpcode::SBufferLoadDwordx2: + case RdnaOpcode::SBufferLoadDwordx4: + case RdnaOpcode::SBufferLoadDwordx8: + case RdnaOpcode::SBufferLoadDwordx16: return true; + default: return false; + } +} + +bool onlyWritesLinkRegister(const RdnaInstruction& instruction, std::uint32_t linkRegister) { + const bool overwrites = instruction.family == RdnaInstructionFamily::SOP2 || instruction.op == RdnaOpcode::SMovB32 || instruction.op == RdnaOpcode::SMovB64 || + (instruction.family == RdnaInstructionFamily::SMEM && isScalarLoad(instruction.op)); + return overwrites && !readsLinkRegister(instruction, linkRegister); } bool isFetchCallPrefixOpcode(const RdnaInstruction& instruction) { @@ -462,12 +489,6 @@ std::vector analyzeSwappcCalls(const RdnaProgram& program, const Swa SwappcCall call; call.callIndex = index; call.linkRegister = linkRegister; - for (const auto& other : calls) { - if (call.linkRegister <= other.linkRegister + 1u && other.linkRegister <= call.linkRegister + 1u) { - throw std::invalid_argument("unclosable scalar call/return pairing at program counter " + toHexString(instruction.programCounter) + - ": link register s[" + std::to_string(call.linkRegister) + "] is shared with another call"); - } - } std::uint32_t target = instruction.branchTarget; const bool staticTarget = instruction.op == RdnaOpcode::SCallB64 || resolveSetpcTarget(program, index, target); call.targetIndex = staticTarget ? instructionIndexOfProgramCounter(program, target) : InvalidControlFlowId; @@ -481,13 +502,63 @@ std::vector analyzeSwappcCalls(const RdnaProgram& program, const Swa continue; } if (!staticTarget || call.targetIndex == InvalidControlFlowId) { + if (std::ranges::any_of(calls, [&](const SwappcCall& other) { return other.linkRegister == call.linkRegister; })) { + throw std::invalid_argument("unclosable scalar call/return pairing at program counter " + toHexString(instruction.programCounter) + + ": link register s[" + std::to_string(call.linkRegister) + "] is shared with another call"); + } throw std::invalid_argument("computed/data-dependent s_swappc_b64 call target at program counter " + toHexString(instruction.programCounter) + " is not statically resolvable"); } call.targetProgramCounter = target; calls.push_back(call); } + const auto sharedLinkError = [&](const SwappcCall& call) { + return std::invalid_argument("unclosable scalar call/return pairing at program counter " + toHexString(program.instructions[call.callIndex].programCounter) + + ": link register s[" + std::to_string(call.linkRegister) + "] is shared with another call"); + }; + const auto escapeError = [&](const SwappcCall& call, const RdnaInstruction& instruction) { + return std::invalid_argument("scalar call link register s[" + std::to_string(call.linkRegister) + "] written at program counter " + + toHexString(program.instructions[call.callIndex].programCounter) + " escapes the constant-offset call/return model (used at program counter " + + toHexString(instruction.programCounter) + ")"); + }; + const auto sharesLink = [&](const SwappcCall& call) { + return std::ranges::count_if(calls, [&](const SwappcCall& other) { return other.linkRegister == call.linkRegister; }) > 1; + }; + for (std::size_t first = 0; first < calls.size(); ++first) { + for (std::size_t second = first + 1u; second < calls.size(); ++second) { + if (calls[first].linkRegister == calls[second].linkRegister && (calls[first].fetch || calls[second].fetch)) throw sharedLinkError(calls[second]); + } + } + for (auto& call : calls) { + if (sharesLink(call)) { + call.returnIndex = InvalidControlFlowId; + for (auto index = call.targetIndex; index < program.instructions.size(); ++index) { + const auto& instruction = program.instructions[index]; + if (instruction.op == RdnaOpcode::SSetpcB64 && isScalar(instruction.source0, call.linkRegister)) { + call.returnIndex = index; + break; + } + } + if (call.returnIndex == InvalidControlFlowId) { + throw std::invalid_argument("unclosable scalar call/return pairing at program counter " + toHexString(program.instructions[call.callIndex].programCounter) + + ": no paired s_setpc_b64 return reads link register s[" + std::to_string(call.linkRegister) + "]"); + } + call.returnTargetProgramCounter = instructionEndProgramCounter(program.instructions[call.callIndex]); + continue; + } + std::uint32_t regionEnd = InvalidControlFlowId; + if (!call.fetch) { + std::uint32_t returns = 0; + for (std::uint32_t index = 0; index < program.instructions.size(); ++index) { + const auto& instruction = program.instructions[index]; + if (index != call.callIndex && instruction.op == RdnaOpcode::SSetpcB64 && isScalar(instruction.source0, call.linkRegister)) { + ++returns; + regionEnd = index; + } + } + if (returns != 1u || regionEnd < call.targetIndex) regionEnd = InvalidControlFlowId; + } std::uint32_t returnIndex = InvalidControlFlowId; for (std::uint32_t index = 0; index < program.instructions.size(); ++index) { if (index == call.callIndex) { @@ -502,10 +573,9 @@ std::vector analyzeSwappcCalls(const RdnaProgram& program, const Swa returnIndex = index; continue; } - if (touchesLinkRegister(instruction, call.linkRegister)) { - throw std::invalid_argument("scalar call link register s[" + std::to_string(call.linkRegister) + "] written at program counter " + - toHexString(program.instructions[call.callIndex].programCounter) + " escapes the constant-offset call/return model (used at program counter " + - toHexString(instruction.programCounter) + ")"); + const bool outsideRegion = regionEnd != InvalidControlFlowId && (index < call.targetIndex || index > regionEnd); + if (touchesLinkRegister(instruction, call.linkRegister) && !(outsideRegion && onlyWritesLinkRegister(instruction, call.linkRegister))) { + throw escapeError(call, instruction); } } if (call.fetch) { @@ -527,6 +597,26 @@ std::vector analyzeSwappcCalls(const RdnaProgram& program, const Swa call.returnTargetProgramCounter = instructionEndProgramCounter(program.instructions[call.callIndex]); } + for (const auto& call : calls) { + if (!sharesLink(call)) continue; + for (const auto& other : calls) { + if (&other == &call || other.linkRegister != call.linkRegister) continue; + const bool callInside = other.callIndex >= call.targetIndex && other.callIndex <= call.returnIndex; + const bool regionsMeet = other.targetIndex <= call.returnIndex && call.targetIndex <= other.returnIndex; + if (callInside || regionsMeet) throw sharedLinkError(other); + } + for (std::uint32_t index = 0; index < program.instructions.size(); ++index) { + if (std::ranges::any_of(calls, [&](const SwappcCall& other) { return other.callIndex == index && other.linkRegister == call.linkRegister; })) continue; + const auto& instruction = program.instructions[index]; + const auto region = std::ranges::find_if(calls, [&](const SwappcCall& other) { return other.linkRegister == call.linkRegister && index >= other.targetIndex && index <= other.returnIndex; }); + if (region != calls.end()) { + if (index != region->returnIndex && touchesLinkRegister(instruction, call.linkRegister)) throw escapeError(*region, instruction); + } else if (touchesLinkRegister(instruction, call.linkRegister) && !onlyWritesLinkRegister(instruction, call.linkRegister)) { + throw escapeError(call, instruction); + } + } + } + for (std::size_t first = 0; first < calls.size(); ++first) { if (!calls[first].fetch && calls[first].callIndex >= calls[first].targetIndex && calls[first].callIndex <= calls[first].returnIndex) { throw std::invalid_argument("recursive scalar call at program counter " + toHexString(program.instructions[calls[first].callIndex].programCounter) + @@ -868,6 +958,111 @@ std::optional UnresolvableSwappcTarget(const RdnaProgram& program return std::nullopt; } +std::vector FindUserDataCalls(const RdnaProgram& program, const SwappcInfo& swappc) { + std::set labels; + for (const auto& instruction : program.instructions) { + if (IsDirectBranchOpcode(instruction.op) || instruction.op == RdnaOpcode::SCallB64) labels.insert(instruction.branchTarget); + } + const auto userData = [&](std::uint32_t reg) { return reg >= swappc.userDataBaseRegister && reg - swappc.userDataBaseRegister < swappc.userDataCount; }; + const auto unchangedUntil = [&](std::uint32_t reg, std::uint32_t reader) { + for (std::uint32_t index = 0; index < program.instructions.size(); ++index) { + if (!writesScalar(program.instructions[index], reg)) continue; + if (index <= reader) return false; + for (std::uint32_t later = index; later < program.instructions.size(); ++later) { + const auto& instruction = program.instructions[later]; + if ((IsDirectBranchOpcode(instruction.op) || instruction.op == RdnaOpcode::SCallB64) && instruction.branchTarget <= program.instructions[reader].programCounter) return false; + } + } + return true; + }; + std::vector calls; + for (std::uint32_t index = 0; index < program.instructions.size(); ++index) { + const auto& instruction = program.instructions[index]; + if (instruction.op != RdnaOpcode::SSwappcB64) continue; + std::uint32_t target = 0; + if (resolveSetpcTarget(program, index, target)) continue; + const auto link = scalarIndex(instruction.destination); + const auto source = scalarIndex(instruction.source0); + if (link == NoScalarRegister || source == NoScalarRegister || source % 2u != 0u) continue; + std::uint32_t pair = NoScalarRegister; + if (source != link && userData(source) && userData(source + 1u) && unchangedUntil(source, index) && unchangedUntil(source + 1u, index)) { + pair = source; + } else { + std::uint32_t low = 0, high = 0; + if (!findLastWriter(program, index, {source}, low) || !findLastWriter(program, index, {source + 1u}, high)) continue; + const auto first = std::min(low, high); + const bool sameBlock = std::none_of(labels.begin(), labels.end(), [&](std::uint32_t label) { + return label > program.instructions[first].programCounter && label <= instruction.programCounter; + }); + const auto& lowWriter = program.instructions[low]; + const auto& highWriter = program.instructions[high]; + std::uint32_t from = NoScalarRegister; + if (low == high && lowWriter.op == RdnaOpcode::SMovB64 && isScalar(lowWriter.destination, source)) { + from = scalarIndex(lowWriter.source0); + } else if (lowWriter.op == RdnaOpcode::SMovB32 && highWriter.op == RdnaOpcode::SMovB32 && isScalar(lowWriter.destination, source) && isScalar(highWriter.destination, source + 1u)) { + from = scalarIndex(lowWriter.source0); + if (from != NoScalarRegister && !isScalar(highWriter.source0, from + 1u)) from = NoScalarRegister; + } + if (!sameBlock || from == NoScalarRegister || from % 2u != 0u || !userData(from) || !userData(from + 1u) || !unchangedUntil(from, low) || !unchangedUntil(from + 1u, high)) continue; + pair = from; + } + calls.push_back({instruction.programCounter, link, pair}); + } + return calls; +} + +std::optional UserDataCalleeLength(std::span callee, std::uint32_t linkRegister) { + std::uint32_t furthest = 0; + for (std::uint32_t word = 0; word < callee.size();) { + const auto programCounter = word * 4u; + RdnaInstruction instruction; + try { + instruction = DecodeRdnaInstruction(programCounter, callee, word); + } catch (const std::out_of_range&) { + if (callee.size() - word < 16u) return std::nullopt; + throw; + } + word += instruction.wordCount; + const auto at = " at callee offset " + toHexString(programCounter); + if (instruction.op == RdnaOpcode::SGetpcB64) throw std::invalid_argument("user-data callee reads its program counter" + at + ", which a linked copy does not keep"); + if (instruction.op == RdnaOpcode::SSwappcB64 || instruction.op == RdnaOpcode::SCallB64) throw std::invalid_argument("user-data callee makes a call" + at); + if (instruction.op == RdnaOpcode::SEndpgm || instruction.op == RdnaOpcode::SCodeEnd) throw std::invalid_argument("user-data callee ends the program" + at + " before it returns"); + if (IsDirectBranchOpcode(instruction.op)) { + if (instruction.branchTarget >= 0x80000000u) throw std::invalid_argument("user-data callee branches before its start" + at); + furthest = std::max(furthest, instruction.branchTarget / 4u); + } + if (instruction.op != RdnaOpcode::SSetpcB64) continue; + if (!isScalar(instruction.source0, linkRegister)) throw std::invalid_argument("user-data callee jumps through another register pair than its link s[" + std::to_string(linkRegister) + "]" + at); + if (furthest >= word) throw std::invalid_argument("user-data callee returns" + at + " before code it branches to"); + return word; + } + return std::nullopt; +} + +std::vector LinkUserDataCalls(const RdnaProgram& program, std::span calls, std::span> callees) { + if (calls.size() != callees.size()) throw std::logic_error("user-data calls and callees differ in number"); + for (const auto& instruction : program.instructions) { + if (instruction.op == RdnaOpcode::SGetpcB64) { + throw std::invalid_argument("a program that reads its program counter (s_getpc_b64 at program counter " + toHexString(instruction.programCounter) + ") cannot have user-data callees linked after it"); + } + } + const auto end = instructionEndProgramCounter(program.instructions.back()) / 4u; + std::vector linked(program.code.begin(), program.code.begin() + end); + for (std::size_t call = 0; call < calls.size(); ++call) { + const auto index = instructionIndexOfProgramCounter(program, calls[call].programCounter); + if (index == InvalidControlFlowId || program.instructions[index].op != RdnaOpcode::SSwappcB64 || program.instructions[index].wordCount != 1u) { + throw std::logic_error("user-data call does not name an s_swappc_b64"); + } + const auto word = calls[call].programCounter / 4u; + const auto offset = static_cast(linked.size()) - static_cast(word) - 1; + if (offset > 0x7fff) throw std::invalid_argument("user-data callee of the call at program counter " + toHexString(calls[call].programCounter) + " lands out of s_call_b64 range"); + linked[word] = 0xbb000000u | (calls[call].linkRegister << 16u) | static_cast(offset); + linked.insert(linked.end(), callees[call].begin(), callees[call].end()); + } + linked.push_back(0xbf9f0000u); + return linked; +} + ControlFlowGraph GraphBuilder::Build(const RdnaProgram& program, const SwappcInfo* swappc) const { const std::vector calls = analyzeSwappcCalls(program, swappc); ControlFlowGraph graph; diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index a735e619b..3b840c910 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -521,7 +521,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceFontOpenFontSet](../../core/libs/prx/libSceFont/src/SystemFont.cpp) (libSceFont) opens only the primary file of a system font set. Characters the console takes from the set's other files (Hangul, Thai and Arabic in Japanese and Chinese sets, Arabic in European sets) are reported as unsupported glyphs. The console file names are those of the PS4 firmware; the names of the PS5-only JG2 sets (`0x1A......`) are unknown, so they use the SSTJpPro files. When a console file is missing, an openly licensed substitute (Noto Sans, Noto Sans Mono, Noto Sans Thai, Noto Sans CJK) is loaded instead. It is not the system font: its metrics differ, so text width, line height and wrapping differ from the console. System font faces are loaded by FreeType in host memory, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. - [sceFontGetCharGlyphMetrics, sceFontGetRenderCharGlyphMetrics](../../core/libs/prx/libSceFont/src/Layout.cpp) (libSceFont) report the vertical metrics the way the [FreeType driver](../../core/libs/prx/libSceFontFt/src/Driver.cpp) reports them for a rendered glyph, not confirmed on a title: FreeType's vertical advance, and for a face without vertical metrics a bearing of half the horizontal advance left of the horizontal bearing and of half the advance left over by the glyph height. - [Font writing lines](../../core/libs/prx/libSceFont/src/WritingLine.cpp) (libSceFont) - `sceFontWritingLineWritesOrder`, `sceFontWritingLineRefersRenderStep`, `sceFontWritingLineGetRenderMetrics`, `sceFontWritingLineGetOrderingSpace` and `sceFontWritingLineClear` follow the PS4 11.00 `libSceFont.sprx` (firmware module, read from its disassembly and compared call for call on random sequences): runs are linked in visual order by embedding level, the space at each spaced join is summed for the line, steps walk the visual order from the left (form `0x12`) or from the right (form `0x11`) and the render metrics grow as steps are consumed. The PS5 library was not measured. Not from that module: form `0x10` is accepted and treated as left to right and a detail with id `0x0FD5` is accepted (the PS4 library rejects both); the memory argument of `sceFontCreateWritingLine` and the rest of the detail are not read (the PS4 library validates its library object there); an unknown writing form, another detail id, a null line pointer on creation, unknown and null handles (except in `sceFontWritingLineRefersRenderStep`, which returns null for a null line) and a null output of `sceFontWritingLineGetRenderMetrics` throw instead of returning the PS4 error codes, which are not confirmed for the PS5; `sceFontDestroyWritingLine` was not compared. Lines are allocated on the host, not in the font memory the title passes to the library: FreeType needs about 1 MiB per CJK face, which the title's font memory is not sized for. -- [s_swappc_b64](../../core/shader/recompiler/Translation/src/ControlFlowInstructions.cpp) writes the link pair with the same model as `s_getpc_b64`: the guest return address `pc + 4` as a byte offset over `GetShaderBase`. Any other use of the link pair (arithmetic, compare, hash, store, or a second `s_swappc_b64` sharing it) throws; constant-offset folding over the link is not modeled. The special encoding `s_swappc_b64` with SDST 125 still decodes as `s_setpc_b64`, which matches the hardware: measured on RDNA2 (gfx1036) in wave32 and wave64, it branches to the source pair and leaves a marker SGPR pair untouched, and with an SGPR destination the link is `PC + 4`. +- [s_swappc_b64](../../core/shader/recompiler/Translation/src/ControlFlowInstructions.cpp) writes the link pair with the same model as `s_getpc_b64`: the guest return address `pc + 4` as a byte offset over `GetShaderBase`. Any other use of the link pair (arithmetic, compare, hash, store) throws, apart from the overwrites and shared links that [static scalar calls](../../core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp) allow; constant-offset folding over the link is not modeled. The special encoding `s_swappc_b64` with SDST 125 still decodes as `s_setpc_b64`, which matches the hardware: measured on RDNA2 (gfx1036) in wave32 and wave64, it branches to the source pair and leaves a marker SGPR pair untouched, and with an SGPR destination the link is `PC + 4`. - The [s_swappc_b64 fetch-shader call at vertex/local/tessellation/mesh entry](../../core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp) assumes the called fetch-shader blob ends in `s_setpc_b64` to the link pair (the GNM fetch-shader shape shadPS4's `EmitFetch` copies); the blob is not part of the shader program, so its return closes the pairing by assumption and any in-program reader of the link pair throws. Attribute fetches are inlined from the request's parsed fetch-shader plan through the same `GetAttribute` path as the embedded fetch analyzer, honoring the buffer resource `dst_sel`; an attribute writing more than four registers, or an entry call without a parsed plan, throws. The vertex/instance offset accumulator inside the fetch blob is not replayed for external blobs. - The scalar aperture sources [src_shared_base, src_shared_limit, src_private_base, src_private_limit](../../core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp) (core/shader/recompiler) decode to their own operand kinds (scalar source codes 235 to 238, named as in LLVM's `SIRegisterInfo.td`), but their value is not modelled: reading one refuses with the source it refused named, because the wave's shared and private aperture bases and limits are unknown. Both the scalar-code read and the raw-operand read refuse; a shader that only carries the value into an address still prepares, one that reads it does not - [VertexBufferReadSize](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) and the [vertex staging copy](../../core/libs/prx/libSceAgcDriver/Graphics/src/Draw.cpp) (libSceAgcDriver) - a vertex fetch whose index is at or past the descriptor's `num_records` no longer throws for `OOB_SELECT` 0 and 1. The copy still spans the whole fetch, but the bytes from `num_records * stride` onward read as zeros, so the out-of-range vertex is `(0, 0, 0, 0)`. Measured with `buffer_load_format_xyzw` (`idxen`) through the [hardware oracle](HW_ORACLE.md) on RDNA2 gfx1036 for PPSA01325's word 3 `0x0004dfac` (`OOB_SELECT` 0) in #2241: indices 3, 4 and 5 over 3 records each return `00000000 00000000 00000000 00000000`, W included, unlike Vulkan `robustBufferAccess`, which returns `(0, 0, 0, 1)`. The zeros come from the record check, not from memory: stored values placed past the records are ignored, and a `num_records` 6 control returns them. `RESOURCE_LEVEL` 1, `OOB_SELECT` 1 and FORMAT 74 `32_32_32_FLOAT` give the same zeros, in wave32 and wave64; gfx1013 (the console) is not measured. `OOB_SELECT` 2 returns the stored values past the records and `OOB_SELECT` 3 compares `num_records` against the byte offset, both measured on the same setup in #2241; the two keep throwing past `num_records` here, since each needs its own rule. Zero-stride descriptors (`stride` 0) keep their separate byte-count check and are not covered here. Each out-of-range fetch is copied alone and outside the draw-snapshot cache, which is neither read nor written for it: a reused snapshot would carry another draw's bytes into the zero region, and a stored zero-padded copy would hand zeros to a later in-range draw at the same address. The valid prefix is still flushed for GPU writes before it is read. @@ -535,7 +535,8 @@ Throughout the project, every function at every stage either **does exactly what - [localtime, mktime](../../core/libs/prx/libc/src/Time.cpp) (libc) on Windows convert through the CRT's `localtime_s` and `mktime`, which accept about 1970 to January 3001 UTC and apply the CRT's daylight-saving rule; outside that range `localtime` returns null and `mktime` returns -1, where the console converts. `gmtime` converts every time whose year fits an `int` on both hosts. - [LINE_STIPPLE_TEX](../../core/shader/recompiler/Translation/src/InstructionTranslator.cpp) (recompiler) reads as 0: line stipple is not emulated. RADV enables the input only as the required pixel input of a shader that reads no interpolants, because it initializes the fewest VGPRs ([Mesa](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/vulkan/radv_shader.c) `radv_compute_spi_ps_input`), and never reads it - [Draw snapshot buffer descriptors](../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp) rewrite each affected array from element zero, preserving unchanged descriptors. MoltenVK 1.4.2 writes auxiliary buffer lengths at index `i` instead of `start + i` for partial array updates ([MVKDescriptorSet.mm, BufferAuxSize](https://github.com/KhronosGroup/MoltenVK/blob/v1.4.2/MoltenVK/MoltenVK/GPUObjects/MVKDescriptorSet.mm#L1051)). Per-element writes therefore let one buffer length replace another even when Vulkan buffer handles and ranges are correct. -- [Static scalar calls](../../core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp) (`s_call_b64` and `s_swappc_b64`) require one call per ordinary even-aligned SGPR link pair (s0:s1 through s104:s105) and one later `s_setpc_b64` return reading that exact pair. Regions must nest or be disjoint; recursion, shared links, escaping link values (including overlapping tuples and resource descriptors), invalid targets, interleaved regions, cross-region branches or fall-through, relative SGPR reads and dynamic `s_setpc_b64` in a program with calls throw. The analysis checks all decoded instructions, including unreachable code. Arbitrary indirect calls and general call frames are not implemented. +- [Static scalar calls](../../core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp) (`s_call_b64` and `s_swappc_b64`) require an ordinary even-aligned SGPR link pair (s0:s1 through s104:s105) and a later `s_setpc_b64` return reading that exact pair. A pair serves one call, or several calls whose regions are disjoint, each closed by the first return reading the pair after its target. Outside every region of its calls the pair may only be overwritten whole, by `s_mov_b32`, `s_mov_b64`, an SOP2 result or a scalar load, as the caller of a user-data callee does before each call. Regions must nest or be disjoint; recursion, other shared links, escaping link values (including overlapping tuples and resource descriptors), invalid targets, interleaved regions, cross-region branches or fall-through, relative SGPR reads and dynamic `s_setpc_b64` in a program with calls throw. The analysis checks all decoded instructions, including unreachable code. Arbitrary indirect calls and general call frames are not implemented. +- [User-data callees](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp) (libSceAgcDriver) - a compute shader's `s_swappc_b64` whose source pair is a user-data pair, or a copy of one made with `s_mov_b32` or `s_mov_b64` in the same block, calls the code at the address that user data holds. At dispatch the callee is read from guest memory up to its `s_setpc_b64` on the link pair, one copy of it is appended per call after the caller's last instruction, and the call is rewritten to `s_call_b64` to that copy. The words after the caller's last instruction are dropped; in the shaders seen they are metadata, and a caller that reads its program counter, which could address them, throws. A copy runs at another program counter than the callee, so a callee that reads its program counter, makes a call, ends the program, jumps through another pair, returns before code it branches to, spans more than 64 KiB or is not readable throws. Callees are read again whenever a dispatch misses the dispatch cache; a dispatch that hits it reuses the linked program, so a title that rewrote a callee in place at the same address would keep running the old code. - [DB_RENDER_CONTROL resummarize draws](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) - HTILE is not modeled, so a RESUMMARIZE_ENABLE draw without depth or stencil tests and without color writes changes no modeled state and draws nothing; any other resummarize, copy or decompress draw throws - [s_call_b64](../../core/shader/recompiler/Translation/src/ScalarInstructions.cpp) saves `PC + 4` over `GetShaderBase` and branches to `PC + 4 + signext(SIMM16) * 4`, following [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) sections 3.6.3, 12.2 and 12.3. Forward callees after `s_endpgm` are decoded. Measured on RDNA2 (gfx1036) in wave32 and wave64 with SIMM16 = 3: the link pair receives `PC + 4`, the branch goes to `PC + 4 + signext(SIMM16) * 4`, and `s_setpc_b64` of the link returns to the instruction after the call. Execution on the PS5 (gfx1013) and title compatibility for these bounded scalar calls remain unverified; `s_rfe_b64` still throws in translation. - [sceVideodec2CreateDecoder](../../core/libs/prx/libSceVideodec2/Export.cpp) (libSceVideodec2) decodes HEVC (codec type 974921) and VP9 (2382845) through FFmpeg, besides H.264. The tags come from SharpProspero's `Videodec2CodecType` and EVO-PLAYER-PS5's `sce_videodec2.h`, which records them from hardware runs. HEVC input is converted from length-prefixed NAL units to Annex B as for H.264, and VP9 frames are passed through. Output is the same 8-bit NV12 for every codec; a decoder for HEVC Main10 or VP9 Profile 2 is created, but a frame deeper than 8 bits throws, since the console's 10-bit output layout is unknown, and `QueryDecoderMemoryInfo` reports the 8-bit frame size for it. `sceVideodec2GetPictureInfo` and `sceVideodec2GetAvcPictureInfo` throw for HEVC and VP9 pictures, which use other layouts (`SceVideodec2HevcPictureInfo`, `SceVideodec2Vp9PictureInfo`), and `sceVideodec2GetHevcPictureInfo` and `sceVideodec2GetVp9PictureInfo` are not exported From 8c619f16daca3d02cd5f05a2cfb8bdeea44f4da9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 08:08:21 +0200 Subject: [PATCH 429/460] docs(architecture): user-data callees; drop code comment Co-Authored-By: Claude --- .../Execution/src/Driver/Shaders/ShaderRegistry.cpp | 2 -- docs/dev/ARCHITECTURE.md | 1 + 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp index c8e298cab..98bea57e5 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp @@ -385,8 +385,6 @@ std::vector ReadUserDataCallee(std::uint64_t address, std::uint32 throw std::runtime_error("its callee at " + hexText(address) + " does not return within " + std::to_string(limitWords * 4u) + " bytes"); } -// A compute shader that calls a user-data pointer gets one copy of each callee after its code, read at -// the address the call's user data holds now; the prepared key follows the linked code. std::vector LinkUserDataCallees(const ShaderRecompiler::RecompileRequest& request) { if (request.shader.stage != ShaderRecompiler::ShaderStage::Compute) return {}; const auto code = request.shader.code; diff --git a/docs/dev/ARCHITECTURE.md b/docs/dev/ARCHITECTURE.md index 32e6742d8..37557e4a0 100644 --- a/docs/dev/ARCHITECTURE.md +++ b/docs/dev/ARCHITECTURE.md @@ -84,6 +84,7 @@ flowchart LR - The shader registry maps a code address to every header registered for that code, in registration order. Static, graphics and rectangle ABI resolution pick the header at the bound address (or one equal to it, `SameHeader`); draws and dispatches pick the program by the stage's binary type (`RegisteredProgram`), and a front entry matching both type 2 and type 4 throws. Within a type the last registration wins; re-registering an identical earlier header makes its snapshot current again without preparing it again, and changed code drops the older headers. - The snapshot is published as pending. `SourceHandleFor`, `InvocationFor` and `ShaderPreparationTransaction::Edit` / `Read` wait for the pending job before reading `PreparedShaders`, and rethrow its failure. `APS5_SYNC_SHADER_PREPARE=1` prepares on the registering thread instead. - A draw or dispatch whose state matches no artifact prepared at registration (a title that builds its static ABI state itself, without `sceAgcDriverResolve*Abi`) gets one prepared at use by `SourceHandleFor` or `InvocationFor`, logged once per artifact and kept in the snapshot for the next use. +- A compute shader whose `s_swappc_b64` calls a user-data pointer cannot be linked at registration, because the callee's address is only known at dispatch. `InvocationFor` in [`ShaderRegistry.cpp`](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp) reads each callee from guest memory, appends one copy per call after the caller (`LinkUserDataCalls`) and prepares that linked code, so a different callee gets its own prepared entry. - A draw's `Graphics::Pipeline` comes from `CachedPipeline`. When the device has `VK_EXT_graphics_pipeline_library` with fast linking, `VK_EXT_extended_dynamic_state` and `VK_KHR_dynamic_rendering`, draws render with `vkCmdBeginRenderingKHR` instead of render passes and the pipeline is linked from four libraries (vertex input, pre-rasterization shaders, fragment shader, fragment output) kept per device in [`PipelineLibrary.cpp`](../../core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp). A library built for one pipeline is reused by the next: another blend, vertex layout or target format builds only that part (the shader libraries name no attachment format), and cull mode, front face, depth and stencil state are dynamic, so changing them only links. The fast link is what the draw waits for; a background thread then links the same libraries with link-time optimization and the pipeline binds that one from the next draw on. Push constants then use a 256-byte block with a 128-byte slot per side: the pre-rasterization stages stack from byte 0, the pixel shader always starts at byte 128 (`StagePushOffset`), so its library does not depend on how much push data its vertex shader uses. Pipelines with per-face depth bias, a separate alpha blend into an 8_8 target with the alternate component order (both draw twice), rect lists, geometry shaders or a stage without a variant id are created whole, still with dynamic rendering; devices without the extensions (or with `APS5_NO_GPL=1`) keep render passes and whole pipelines. ### Shader MODE register access From fb2b7b8dce1e6a90303becb1323f0d23a8fa6c38 Mon Sep 17 00:00:00 2001 From: pushkarreddyy <212133856+pushkarreddyy@users.noreply.github.com> Date: Sun, 11 Oct 2026 12:52:49 +0530 Subject: [PATCH 430/460] feat(libSceHttp2): export sceHttp2WebSocketSetPingInterval Export sceHttp2WebSocketSetPingInterval (YfaXPnc8PJE) in libSceHttp2. Resolves unresolved ELF import in Palworld (PPSA23126, Issue #2648) bundled libdiscord_partner_sdk.prx. --- core/libs/prx/libSceHttp2/Export.cpp | 6 ++++++ core/libs/tests/GuestHttp2.cpp | 2 ++ docs/dev/TechnicalDebt.md | 2 +- 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/core/libs/prx/libSceHttp2/Export.cpp b/core/libs/prx/libSceHttp2/Export.cpp index d7874e9c9..2e96dab09 100644 --- a/core/libs/prx/libSceHttp2/Export.cpp +++ b/core/libs/prx/libSceHttp2/Export.cpp @@ -318,4 +318,10 @@ int APS5_VABI sceHttp2WebSocketSendDataMessageAsync() { return 0; } +int APS5_VABI sceHttp2WebSocketSetPingInterval(int req_id, uint32_t interval_ms) { + (void)req_id; + (void)interval_ms; + return 0; +} + } diff --git a/core/libs/tests/GuestHttp2.cpp b/core/libs/tests/GuestHttp2.cpp index dcc8347c3..81694da8c 100644 --- a/core/libs/tests/GuestHttp2.cpp +++ b/core/libs/tests/GuestHttp2.cpp @@ -35,6 +35,7 @@ struct Http2MemoryPoolStats { extern "C" { int APS5_VABI sceHttp2GetMemoryPoolStats(int, Http2MemoryPoolStats*); int APS5_VABI sceHttp2SetResolveRetry(int, std::int32_t); +int APS5_VABI sceHttp2WebSocketSetPingInterval(int, std::uint32_t); } static void Require(bool value) { if (!value) std::abort(); } @@ -67,6 +68,7 @@ int main() { Require(request > 0); Require(sceHttp2SetCookieBox(request, other) == 0); Require(sceHttp2SetRequestNoContentLength(request) == 0); + Require(sceHttp2WebSocketSetPingInterval(request, 30000) == 0); Require(sceHttp2CookieFlush(context) == 0); Require(sceHttp2SendRequest(request, nullptr, 0) == static_cast(0x80436063)); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 3b76446d5..aa95ddc7a 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -36,7 +36,7 @@ Throughout the project, every function at every stage either **does exactly what - [libSceAudio3d](../../core/libs/prx/libSceAudio3d/Export.cpp) - the port produces no sound: objects can be reserved and `sceAudio3dObjectSetAttributes` accepts their attributes, but the library has no bed exports or mixing to give them audio, the attributes have no effect and `sceAudio3dPortPush` only paces the queue at `granularity` samples per frame at 48 kHz. `sceAudio3dPortSetAttribute` accepts the late reverb level and the downmix spread attributes and ignores them. `sceAudio3dObjectSetAttributes` validates the port, the reserved object and the attribute array (an object id that was never reserved returns the invalid-object error, while shadPS4 returns success for it and the error only for a released object; neither is measured on a console), treating the reset-state attribute as valid with a null value and requiring a value for the others; the attribute identifiers and the value of each are not interpreted. `sceAudio3dPortFlush` drops the frames queued by `sceAudio3dPortAdvance` instead of mixing and submitting them to an output, as there is no output - [libSceTextToSpeech2](../../core/libs/prx/libSceTextToSpeech2/Export.cpp) has no speech engine: `sceTextToSpeech2Initialize` fails and the other exports throw. PPSA01317 skips its narration set-up when initialization fails - [libSceAvPlayer](../../core/libs/prx/libSceAvPlayer/Export.cpp): `sceAvPlayerSetLogCallback` accepts a callback that is never called, as the player produces no log messages, and `sceAvPlayerSetAvailableBandwidth` has no effect, as it governs HLS sources, which `sceAvPlayerAddSource` does not implement. -- [libSceHttp2](../../core/libs/prx/libSceHttp2/Export.cpp) - no request reaches the network, so `sceHttp2SetMinSslVersion` has no TLS connection to restrict and returns success for any `id`, like `sceHttp2SslEnableOption` (prosper returns the invalid-id error for an unknown one) +- [libSceHttp2](../../core/libs/prx/libSceHttp2/Export.cpp) - no request reaches the network, so `sceHttp2SetMinSslVersion` has no TLS connection to restrict and returns success for any `id`, like `sceHttp2SslEnableOption` (prosper returns the invalid-id error for an unknown one); `sceHttp2WebSocketSetPingInterval` records no ping interval and returns success without network activity, unblocking Palworld (PPSA23126) - [libSceNpTrophy2](../../core/libs/prx/libSceNpTrophy2/src/GameInfo.cpp): `sceNpTrophy2RegisterUnlockCallback` accepts a callback that is never called, as no trophy is ever unlocked, and `sceNpTrophy2UnregisterUnlockCallback` only returns success. - [sceAvPlayerSetTrickSpeed](../../core/libs/prx/libSceAvPlayer/src/Source.cpp) (libSceAvPlayer) with a negative speed runs the clock backwards but delivers no frames; when a forward speed is set again, playback resumes from the rewound time. - [sceRudpActivate](../../core/libs/prx/libSceRudp/Export.cpp) (libSceRudp) - there is no reliable-UDP stack on the host to activate, so it returns success without doing anything, as [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/rudp/rudp.cpp) does for its own `sceRudpActivate`. PPSA01325 calls it with no arguments during start-up; what the console activates, and whether it reports an error before `sceRudpInit`, is unverified From cfb372a035ba30b841fdda43ed444cc13319888d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 10:06:21 +0200 Subject: [PATCH 431/460] fix(libSceHttp2): make sceHttp2WebSocketSetPingInterval throw, document under unknown function info Co-Authored-By: Claude --- core/libs/prx/libSceHttp2/Export.cpp | 5 ++--- core/libs/tests/GuestHttp2.cpp | 4 ++-- docs/dev/TechnicalDebt.md | 3 ++- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/core/libs/prx/libSceHttp2/Export.cpp b/core/libs/prx/libSceHttp2/Export.cpp index 2e96dab09..e4869b148 100644 --- a/core/libs/prx/libSceHttp2/Export.cpp +++ b/core/libs/prx/libSceHttp2/Export.cpp @@ -318,9 +318,8 @@ int APS5_VABI sceHttp2WebSocketSendDataMessageAsync() { return 0; } -int APS5_VABI sceHttp2WebSocketSetPingInterval(int req_id, uint32_t interval_ms) { - (void)req_id; - (void)interval_ms; +int APS5_VABI sceHttp2WebSocketSetPingInterval() { + NotImplemented_nid_no_patch(__func__); return 0; } diff --git a/core/libs/tests/GuestHttp2.cpp b/core/libs/tests/GuestHttp2.cpp index 81694da8c..4bfb83c4f 100644 --- a/core/libs/tests/GuestHttp2.cpp +++ b/core/libs/tests/GuestHttp2.cpp @@ -35,7 +35,7 @@ struct Http2MemoryPoolStats { extern "C" { int APS5_VABI sceHttp2GetMemoryPoolStats(int, Http2MemoryPoolStats*); int APS5_VABI sceHttp2SetResolveRetry(int, std::int32_t); -int APS5_VABI sceHttp2WebSocketSetPingInterval(int, std::uint32_t); +int APS5_VABI sceHttp2WebSocketSetPingInterval(); } static void Require(bool value) { if (!value) std::abort(); } @@ -68,7 +68,7 @@ int main() { Require(request > 0); Require(sceHttp2SetCookieBox(request, other) == 0); Require(sceHttp2SetRequestNoContentLength(request) == 0); - Require(sceHttp2WebSocketSetPingInterval(request, 30000) == 0); + Require(Throws([] { sceHttp2WebSocketSetPingInterval(); })); Require(sceHttp2CookieFlush(context) == 0); Require(sceHttp2SendRequest(request, nullptr, 0) == static_cast(0x80436063)); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index aa95ddc7a..ed6370ad9 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -36,7 +36,7 @@ Throughout the project, every function at every stage either **does exactly what - [libSceAudio3d](../../core/libs/prx/libSceAudio3d/Export.cpp) - the port produces no sound: objects can be reserved and `sceAudio3dObjectSetAttributes` accepts their attributes, but the library has no bed exports or mixing to give them audio, the attributes have no effect and `sceAudio3dPortPush` only paces the queue at `granularity` samples per frame at 48 kHz. `sceAudio3dPortSetAttribute` accepts the late reverb level and the downmix spread attributes and ignores them. `sceAudio3dObjectSetAttributes` validates the port, the reserved object and the attribute array (an object id that was never reserved returns the invalid-object error, while shadPS4 returns success for it and the error only for a released object; neither is measured on a console), treating the reset-state attribute as valid with a null value and requiring a value for the others; the attribute identifiers and the value of each are not interpreted. `sceAudio3dPortFlush` drops the frames queued by `sceAudio3dPortAdvance` instead of mixing and submitting them to an output, as there is no output - [libSceTextToSpeech2](../../core/libs/prx/libSceTextToSpeech2/Export.cpp) has no speech engine: `sceTextToSpeech2Initialize` fails and the other exports throw. PPSA01317 skips its narration set-up when initialization fails - [libSceAvPlayer](../../core/libs/prx/libSceAvPlayer/Export.cpp): `sceAvPlayerSetLogCallback` accepts a callback that is never called, as the player produces no log messages, and `sceAvPlayerSetAvailableBandwidth` has no effect, as it governs HLS sources, which `sceAvPlayerAddSource` does not implement. -- [libSceHttp2](../../core/libs/prx/libSceHttp2/Export.cpp) - no request reaches the network, so `sceHttp2SetMinSslVersion` has no TLS connection to restrict and returns success for any `id`, like `sceHttp2SslEnableOption` (prosper returns the invalid-id error for an unknown one); `sceHttp2WebSocketSetPingInterval` records no ping interval and returns success without network activity, unblocking Palworld (PPSA23126) +- [libSceHttp2](../../core/libs/prx/libSceHttp2/Export.cpp) - no request reaches the network, so `sceHttp2SetMinSslVersion` has no TLS connection to restrict and returns success for any `id`, like `sceHttp2SslEnableOption` (prosper returns the invalid-id error for an unknown one) - [libSceNpTrophy2](../../core/libs/prx/libSceNpTrophy2/src/GameInfo.cpp): `sceNpTrophy2RegisterUnlockCallback` accepts a callback that is never called, as no trophy is ever unlocked, and `sceNpTrophy2UnregisterUnlockCallback` only returns success. - [sceAvPlayerSetTrickSpeed](../../core/libs/prx/libSceAvPlayer/src/Source.cpp) (libSceAvPlayer) with a negative speed runs the clock backwards but delivers no frames; when a forward speed is set again, playback resumes from the rewound time. - [sceRudpActivate](../../core/libs/prx/libSceRudp/Export.cpp) (libSceRudp) - there is no reliable-UDP stack on the host to activate, so it returns success without doing anything, as [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/main/src/core/libraries/rudp/rudp.cpp) does for its own `sceRudpActivate`. PPSA01325 calls it with no arguments during start-up; what the console activates, and whether it reports an error before `sceRudpInit`, is unverified @@ -51,6 +51,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceKernelGetSystemSwVersion](../../core/libs/prx/libkernel/System/src/SystemConfiguration.cpp) (libkernel) reports baseline system software version `0x01000000` ("01.000.000") for ProsperoStore (PPSA99000). The 0x28-byte layout (`uint64_t size` at 0, `versionString` at 8 and `version` at 0x24) matches [ProsperoStore's caller](https://github.com/blackbearreloaded/ProsperoStore/blob/20584011/src/diag/trace.cpp#L20-L26), [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/811bfcd7d1/src/core/libraries/kernel/kernel.h#L45-L49), [fpPS4](https://github.com/red-prig/fpPS4/blob/04cefd43e6/kernel/ps4_libkernel.pas#L502-L506) and [OpenOrbis](https://github.com/OpenOrbis/OpenOrbis-PS4-Toolchain/blob/0a1aaf9dd4/include/orbis/_types/kernel.h#L256-L260). These sources support the structure layout, but not the reported version value: the console firmware version remains unverified. The checks are unverified too: a null pointer returns `SCE_KERNEL_ERROR_EFAULT` and a `size` other than 0x28 returns `SCE_KERNEL_ERROR_EINVAL`, while shadPS4 returns success for null and never reads `size`. - [kern.rng_pseudo](../../core/libs/prx/libkernel/System/src/SystemConfiguration.cpp) (libkernel) - `sysctlbyname` fills the whole output buffer with bytes from the host's `std::random_device` and leaves the length unchanged; a write returns -1 with `EPERM` and a call without an output buffer throws. The only caller seen is mbedTLS's `mbedtls_hardware_poll` in the catalog's ProsperoLight boilerplate, which asks for 64-byte blocks and fails if fewer come back. The console's generator, its per-call maximum and its answer to a size query are unknown - [sceVideoOutGetOutputStatusWithVrr](../../core/libs/prx/libSceVideoOut/Unimplemented.cpp) (libSceVideoOut, NID `gWT7X8H0bYs`) - the name hashes to the imported NID. PPSA26344 version 02.002.000 passes a VideoOut handle and an output pointer at three call sites. It compares a 32-bit value at byte 4 with 2, tests bit 0x10 at byte 16, and, when set, compares low 16-bit values at bytes 48, 50 and 52 with 48, 60 and 120. The full output layout, size, field meanings and console errors are unknown. Every call throws without reading or writing caller memory. +- [sceHttp2WebSocketSetPingInterval](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2, NID `YfaXPnc8PJE`) - imported by `libdiscord_partner_sdk.prx` of PPSA23126 (issue 2648). The parameters, the unit of the interval and the console errors are unknown. Every call throws. - [sceKernelGetProsperoSystemSwVersion](../../core/libs/prx/libkernel/System/src/SoftwareVersion.cpp) (libkernel) reports system software version `0x01000000` ("01.000.000"); the console's firmware version is not modeled. The 0x28-byte block (a 64-bit size at 0, the version string at 8 and the packed version at 0x24) follows [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Kernel/KernelSystem.cs), which Prospero Multi Tools (PPSA99110) and Prospero Explorer (PPSA99109) are built with and which sets the size to 40, and the same fields in [etaHEN](https://github.com/etaHEN/etaHEN/blob/main/Source%20Code/daemon/include/ipc.hpp) and [shsrv](https://github.com/ps5-payload-dev/shsrv/blob/master/sh.c). A size other than 0x28 returns `SCE_KERNEL_ERROR_EINVAL` without writing; whether the console checks the size is unverified, since etaHEN and shsrv declare a 0x30-byte block and do not set it - [sceKernelGetAllowedSdkVersionOnSystem](../../core/libs/prx/libkernel/System/src/SoftwareVersion.cpp) (libkernel) reports the system software version with the 12 patch bits set, `0x01000fff`, as [shadPS4](https://github.com/shadps4-emu/shadPS4/blob/98fd0689/src/core/libraries/kernel/kernel.cpp) does on the PS4 and as [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/main/src/SharpProspero/Interop/Kernel/KernelSystem.cs) describes it, which Prospero Explorer (PPSA99109) reads it through; a null pointer returns `SCE_KERNEL_ERROR_EINVAL`. The console's value and its errors are unverified - [sceSystemServiceLaunchApp](../../core/libs/prx/libSceSystemService/Export.cpp) (libSceSystemService) - AnyPS5 runs one title and cannot start another, so every call throws without reading its arguments. The (title ID, null-terminated argv, launch parameters) signature follows [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/5dfdb4b0b166e83dc9ae275001afb695f12c7174/src/SharpProspero/Interop/SystemService/SystemService.cs#L141-L148), whose `AppLauncher` Prospero Explorer (PPSA99109) and Prospero Multi Tools (PPSA99110) import it through and which passes a 0x20-byte launch context or null; the console's argument checks and errors are unknown From d0d127b17d37b5b783382308234ec57594efd08d Mon Sep 17 00:00:00 2001 From: Celegans12 <53881228+Celegans12@users.noreply.github.com> Date: Thu, 8 Oct 2026 20:36:04 +0200 Subject: [PATCH 432/460] feat(agc): accept texel offsets with unnormalized sampler coordinates Explicit-LOD offset samples through FORCE_UNNORMALIZED samplers now add sign-extended texel offsets to f32 coordinates before clamping, using OpFAdd with NoContraction and a sampler specialization selector. Without native nonconstant-offset support, retain only runtime image modes valid for unnormalized sampling and reject normalized samplers. Preserve constant-swizzle handling, indirect-image rejection and main's flat-volume and flat-line compatibility rules when selecting those modes. Serialize image compatibility and folding flags and the sampler folding flag in cache format 34, with roundtrip, preparation and execution coverage. Retain main's request format 15 and runtime ABI 11. --- .../libSceAgcDriver/tests/ShaderDiskCache.cpp | 5 +- .../libSceAgcDriver/tests/ShaderMemory.cpp | 89 ++++++++++++++++++- .../execution/ImageSampleUnnormalized.cpp | 68 +++++++++++--- .../IrMetadata/Resources.hpp | 6 ++ .../Optimization/ResourceMaterializer.hpp | 1 + .../src/DescriptorBindingBuilder.cpp | 28 +++--- .../Optimization/src/ResourceMaterializer.cpp | 23 ++++- .../recompiler/PipelineSpecialization.hpp | 1 + core/shader/recompiler/ShaderDiskCache.cpp | 8 +- core/shader/recompiler/ShaderDiskCache.hpp | 2 +- .../SpirvBackend/src/SpirvImageEmitter.cpp | 74 +++++++++++---- docs/dev/TechnicalDebt.md | 3 +- 12 files changed, 253 insertions(+), 55 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp b/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp index 7d803f39f..fc10c39cb 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp @@ -201,8 +201,11 @@ CompiledVariant sampleVariant() { image.r128 = true; image.fmaskCompatible = false; image.depthBitsCompatible = false; + image.constantSwizzle = true; + image.constantSwizzleCompatible = false; image.flatVolumeCompatible = false; image.flatLineCompatible = false; + image.foldTexelOffsetsOnly = true; image.byElements = 4; image.byComponents = 1; image.indirectRoot = 0; @@ -211,7 +214,7 @@ CompiledVariant sampleVariant() { image.indirectResources = {1, 2, 3}; info.info.images = {image}; ResourceMaterializer::PrepareImageModes(info.info); - info.info.samplers = {{7, 0x10, 3, true, false, SamplerUseExplicitLod | SamplerUseGather}}; + info.info.samplers = {{7, 0x10, 3, true, false, SamplerUseExplicitLod | SamplerUseGather, true}}; info.info.sampledPairs = {{0, 0, 0x10}}; StageInput input{}; input.kind = StageInputKind::GlobalInvocationId; diff --git a/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp b/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp index 0e43ded53..a933ca489 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp @@ -1862,6 +1862,7 @@ void verifyUnnormalizedSamplers() { constexpr std::uint32_t Format8888UNorm = 56; constexpr std::uint32_t Format11_11_10UInt = 34; constexpr std::uint32_t Format32Float = 22; + constexpr std::uint32_t Type1D = 8; constexpr std::uint32_t Type2D = 9; constexpr std::uint32_t Type3D = 10; constexpr std::uint32_t TypeCube = 11; @@ -1887,7 +1888,7 @@ void verifyUnnormalizedSamplers() { }; const std::array capabilities{1u, static_cast(spv::CapabilityImageGatherExtended), static_cast(spv::CapabilityMinLod)}; std::uint64_t nextAddress = 0x40000u; - const auto recompile = [&](const std::vector& code, const std::array& data, std::uint64_t address = 0u) { + const auto recompile = [&](const std::vector& code, const std::array& data, std::uint64_t address = 0u, bool offsets = false) { RecompileRequest request{}; request.shader = {ShaderStage::Compute, address != 0u ? address : (nextAddress += 0x1000u), code, 0, {}}; request.context.waveSize = 32; @@ -1899,6 +1900,7 @@ void verifyUnnormalizedSamplers() { request.target.subgroupSize = 32; request.target.supportedCapabilities = capabilities; request.target.fragmentShaderBarycentricEnabled = false; + request.target.nonConstantImageOffsets = offsets; request.layout.pushConstantSizeBytes = 128; AgcDriver::ShaderMemory memory({}); static_cast(memory.Capture(request)); @@ -1947,7 +1949,6 @@ void verifyUnnormalizedSamplers() { const auto reject = [&](std::uint32_t mimg, const std::array& data, const char* reason, const char* what) { expectFailure([&] { static_cast(recompile(program(mimg), data)); }, reason, what); }; - reject(0xf0c00f08u, userData(unnormalized), "unnormalized guest sampler is used with a texel offset, which is not implemented", "unnormalized samplers: image_sample_o was accepted"); reject(0xf0bc0f08u, imageData(unnormalized, Type2D, Format32Float, 0u), "unnormalized guest sampler is used with depth comparison, which is not implemented", "unnormalized samplers: image_sample_c_lz was accepted"); reject(0xf11c0108u, userData(unnormalized), "unnormalized guest sampler is used by a gather, which is not implemented", "unnormalized samplers: image_gather4_lz was accepted"); reject(0xf1800308u, userData(unnormalized), "unnormalized guest sampler is used by image_get_lod, which is not implemented", "unnormalized samplers: image_get_lod was accepted"); @@ -1957,12 +1958,75 @@ void verifyUnnormalizedSamplers() { reject(0xf09c0f10u, imageData(unnormalized, Type3D, Format8888UNorm, 3u), "unnormalized guest sampler samples a 1D-array, 2D-array, 3D, cube or multisampled image", "unnormalized samplers: a 3D T# was accepted"); reject(0xf09c0f28u, imageData(unnormalized, Type2DArray, Format8888UNorm, 5u), "unnormalized guest sampler samples a 1D-array, 2D-array, 3D, cube or multisampled image", "unnormalized samplers: a 2D-array sample was accepted"); reject(0xf09c0f08u, imageData(unnormalized, Type2D, Format11_11_10UInt, 0u), "unnormalized guest sampler samples an image that needs a format conversion or packed access", "unnormalized samplers: a T# with a format conversion was accepted"); + reject(0xf15c0108u, userData(unnormalized), "unnormalized guest sampler is used by a gather, which is not implemented", "unnormalized samplers: image_gather4_lz_o was accepted"); + reject(0xf0fc0f08u, imageData(unnormalized, Type2D, Format32Float, 0u), "unnormalized guest sampler is used with depth comparison, which is not implemented", "unnormalized samplers: image_sample_c_lz_o was accepted"); + reject(0xf0dc0f28u, imageData(unnormalized, Type2DArray, Format8888UNorm, 5u), "unnormalized guest sampler samples a 1D-array, 2D-array, 3D, cube or multisampled image", "unnormalized samplers: a 2D-array image_sample_lz_o was accepted"); for (const std::uint32_t mimg : {0xf0c00f08u, 0xf11c0108u, 0xf09c0f18u}) { static_cast(recompile(program(mimg), imageData(normalized, mimg == 0xf09c0f18u ? TypeCube : Type2D, Format8888UNorm, mimg == 0xf09c0f18u ? 5u : 0u))); } const std::vector latchSample{0xbe900380u, 0x7e020280u, 0x7e040280u, 0x7e060280u, 0x7e080280u, 0x7e0a0280u, 0x7e0c0280u, 0xf09c0f28u, 0x00400801u, 0x80108110u, 0xbf0a8410u, 0xbf85fffbu, 0xe0700000u, 0x80030800u, 0xbf810000u}; static_cast(recompile(latchSample, imageData(normalized, Type2DArray, Format8888UNorm, 5u))); + const auto offsetProgram = [](std::uint32_t mimg, std::uint32_t offsetSource, std::uint32_t literal) { + std::vector code{0x7e020200u | offsetSource}; + if (offsetSource == 0xffu) code.push_back(literal); + code.insert(code.end(), {0x7e040280u, 0x7e060280u, 0x7e080280u, 0x7e0a0280u, 0x7e0c0280u, mimg, 0x00400801u, 0xe0700000u, 0x80030800u, 0xbf810000u}); + return code; + }; + const auto foldedSamples = [](const std::vector& words) { + std::vector exact; + std::vector sums; + std::vector coordinates; + for (std::size_t cursor = 5; cursor < words.size();) { + const auto count = words[cursor] >> 16u; + require(count != 0 && count <= words.size() - cursor, "unnormalized texel offsets: truncated SPIR-V instruction"); + const auto op = words[cursor] & 0xffffu; + if (op == spv::OpDecorate && count == 3u && words[cursor + 2] == spv::DecorationNoContraction) exact.push_back(words[cursor + 1]); + if (op == spv::OpFAdd) sums.push_back(words[cursor + 2]); + if (op == spv::OpImageSampleExplicitLod) coordinates.push_back(words[cursor + 4]); + cursor += count; + } + return !coordinates.empty() && std::ranges::all_of(coordinates, [&](std::uint32_t id) { return std::ranges::count(sums, id) != 0 && std::ranges::count(exact, id) != 0; }); + }; + const auto sampleMasks = [](const std::vector& words) { + std::vector masks; + for (std::size_t cursor = 5; cursor < words.size();) { + const auto count = words[cursor] >> 16u; + require(count != 0 && count <= words.size() - cursor, "unnormalized texel offsets: truncated SPIR-V instruction"); + if ((words[cursor] & 0xffffu) == spv::OpImageSampleExplicitLod && count > 5u) masks.push_back(words[cursor + 5]); + cursor += count; + } + return masks; + }; + for (const std::uint32_t source : {0xffu, 0x100u}) { + const auto oneD = recompile(offsetProgram(0xf0dc0f00u, source, 0x3du), imageData(unnormalized, Type1D, Format8888UNorm, 0u)); + require(proven(oneD, true), "unnormalized texel offsets: a 1D image_sample_lz_o through an unnormalized S# was not flagged"); + auditSpirv(oneD.spirv.Words()); + require(foldedSamples(oneD.spirv.Words()), "unnormalized texel offsets: a 1D offset was not added to the coordinate"); + } + const auto constantLz = offsetProgram(0xf0dc0f08u, 0xffu, 0x3du | (2u << 8u)); + const auto foldedLz = recompile(constantLz, userData(unnormalized), 0x48000u); + require(proven(foldedLz, true), "unnormalized texel offsets: image_sample_lz_o through an unnormalized S# was not flagged"); + auditSpirv(foldedLz.spirv.Words()); + require(foldedSamples(foldedLz.spirv.Words()), "unnormalized texel offsets: a constant offset was not added to the coordinates"); + const auto operandLz = recompile(constantLz, userData(normalized), 0x48000u); + require(proven(operandLz, false) && operandLz.variantId == foldedLz.variantId && operandLz.PipelineVariantId() != foldedLz.PipelineVariantId(), "unnormalized texel offsets: a normalized S# did not get its own specialization of the shared artifact"); + require(sampleMasks(operandLz.spirv.Words()) == std::vector{spv::ImageOperandsLodMask | spv::ImageOperandsConstOffsetMask} && !foldedSamples(operandLz.spirv.Words()), "unnormalized texel offsets: a normalized image_sample_lz_o lost its ConstOffset operand"); + require(recompile(constantLz, userData(unnormalized), 0x48000u).PipelineVariantId() == foldedLz.PipelineVariantId(), "unnormalized texel offsets: the unnormalized S# did not reuse its specialization"); + for (const std::uint32_t mimg : {0xf0dc0f08u, 0xf0d00f08u, 0xf0c00f08u, 0xf0d40f08u, 0xf0c40f08u}) { + const auto computed = offsetProgram(mimg, 0x100u, 0u); + for (const bool offsets : {false, true}) { + const auto result = recompile(computed, userData(unnormalized), 0u, offsets); + require(proven(result, true), "unnormalized texel offsets: a compute image_sample_lz_o, _l_o, _o, _b_o or _cl_o with a computed offset was not flagged"); + auditSpirv(result.spirv.Words()); + require(foldedSamples(result.spirv.Words()), "unnormalized texel offsets: a computed offset was not added to the coordinates"); + } + expectFailure([&] { static_cast(recompile(computed, userData(normalized))); }, "normalized guest sampler is used with a texel offset that is not a constant", "unnormalized texel offsets: a normalized computed offset was accepted without maintenance8"); + expectFailure([&] { static_cast(recompile(computed, imageData(unnormalized, Type2D, Format11_11_10UInt, 0u))); }, "takes only images an unnormalized S# can sample", "unnormalized texel offsets: a computed offset sampled a converted image without maintenance8"); + const auto masks = sampleMasks(recompile(computed, userData(normalized), 0u, true).spirv.Words()); + require(!masks.empty() && std::ranges::all_of(masks, [](std::uint32_t mask) { return (mask & spv::ImageOperandsOffsetMask) != 0u; }), "unnormalized texel offsets: a normalized computed offset lost its Offset operand"); + } + const std::array pixelCode{0xf0800f08u, 0x00400801u, 0xf800180fu, 0x0b0a0908u, 0xbf810000u}; const auto pixelData = userData(unnormalized); RecompileRequest pixel{}; @@ -2013,9 +2077,30 @@ void verifyUnusedUnnormalizedSampler() { auto compared = info; compared.samplers[0].depthCompare = true; expectFailure([&] { static_cast(populate(compared)); }, "unnormalized guest sampler is used with depth comparison, which is not implemented", "unnormalized samplers: a depth-compare S# without live uses was accepted"); + auto offset = info; + offset.samplers[0].uses = SamplerUseExplicitLod | SamplerUseOffset; + const auto folded = populate(offset); + require(folded[0].imageUnnormalized == std::vector{true} && folded[1].samplerUnnormalized == std::vector{true}, "unnormalized samplers: an offset use was not flagged"); + auto gathered = offset; + gathered.samplers[0].uses |= SamplerUseGather; + expectFailure([&] { static_cast(populate(gathered)); }, "unnormalized guest sampler is used by a gather, which is not implemented", "unnormalized samplers: an offset gather was accepted"); + auto foldOnly = offset; + foldOnly.samplers[0].foldTexelOffsetsOnly = true; + static_cast(populate(foldOnly)); + auto foldVolume = foldOnly; + foldVolume.images[0].dimension = RdnaImageDimension::Dim3D; + foldVolume.images[0].foldTexelOffsetsOnly = true; + snapshot.images[0].dwords[3] = 0xa0000041u; + const auto foldedConstant = populate(foldVolume); + require(foldedConstant.size() == 1u && foldedConstant[0].samplerUnnormalized == std::vector{true}, "unnormalized texel offsets: a 3D view whose channels select constants was bound or refused"); + snapshot.images[0].dwords[3] = 0xa0000facu; + expectFailure([&] { static_cast(populate(foldVolume)); }, "takes only images an unnormalized S# can sample", "unnormalized texel offsets: an ordinary 3D view matched a constant-only mode"); + snapshot.images[0].dwords[3] = 0x90000facu; snapshot.samplers[0].dwords[0] = 0x00000092u; const auto normalized = populate(info); require(normalized[0].imageUnnormalized == std::vector{false} && normalized[1].samplerUnnormalized == std::vector{false}, "unnormalized samplers: a normalized S# was flagged"); + static_cast(populate(offset)); + expectFailure([&] { static_cast(populate(foldOnly)); }, "normalized guest sampler is used with a texel offset that is not a constant", "unnormalized samplers: a normalized S# was accepted for an offset that only folds"); } void verifyWaveUniformValues() { diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/ImageSampleUnnormalized.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/ImageSampleUnnormalized.cpp index 4d1274912..7422e5e65 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/ImageSampleUnnormalized.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/ImageSampleUnnormalized.cpp @@ -47,7 +47,27 @@ alignas(256) constexpr std::array Code{ 0x80001001, 0xe0701034, 0x80001101, 0xe0701038, 0x80001201, 0xe070103c, 0x80001301, 0xbf810000, }; -constexpr std::array Instructions{"image_sample_lz", "image_sample_l 2.7", "image_sample"}; +alignas(256) constexpr std::array OffsetCode{ + 0x34020086, 0xe0301008, 0x80000201, 0xe0301000, 0x80000301, 0xe0301004, 0x80000401, 0xbf8c3f70, + 0x7e0a02ff, 0x402ccccd, 0x7e2802ff, 0x0000023d, 0xf0dc0f08, 0x00610802, 0xf0d00f08, 0x00610c02, + 0xf0dc0f0a, 0x00611014, 0x00000403, 0xbf8c3f70, 0xe0701010, 0x80000801, 0xe0701014, 0x80000901, + 0xe0701018, 0x80000a01, 0xe070101c, 0x80000b01, 0xe0701020, 0x80000c01, 0xe0701024, 0x80000d01, + 0xe0701028, 0x80000e01, 0xe070102c, 0x80000f01, 0xe0701030, 0x80001001, 0xe0701034, 0x80001101, + 0xe0701038, 0x80001201, 0xe070103c, 0x80001301, 0xbf810000, +}; + +enum class Offset { None, Lane, Constant }; +constexpr std::int32_t ConstantOffsetU = -3; +constexpr std::int32_t ConstantOffsetV = 2; + +struct Program { + std::span code; + std::array instructions; + std::array offsets; +}; + +const Program Plain{Code, {"image_sample_lz", "image_sample_l 2.7", "image_sample"}, {Offset::None, Offset::None, Offset::None}}; +const Program Offsets{OffsetCode, {"image_sample_lz_o", "image_sample_l_o 2.7", "image_sample_lz_o (-3, 2) NSA"}, {Offset::Lane, Offset::Lane, Offset::Constant}}; alignas(256) constexpr std::array ConstantCode{ 0x34020086, 0xe0301000, 0x80000201, 0xe0301004, 0x80000301, 0xbf8c3f70, 0xf09c0608, 0x00610802, @@ -75,6 +95,8 @@ constexpr std::array Samplers{{ struct Coordinate { float u; float v; + std::int32_t du = 0; + std::int32_t dv = 0; }; std::vector Coordinates() { @@ -98,6 +120,17 @@ std::vector Coordinates() { return result; } +std::vector OffsetCoordinates() { + constexpr std::array us{-32, -5, -1, 0, 1, 3, 31}; + constexpr std::array vs{-2, -1, 0, 2, -32, 31}; + auto result = Coordinates(); + for (std::size_t index = 0; index < result.size(); ++index) { + result[index].du = us[index % us.size()]; + result[index].dv = vs[index % vs.size()]; + } + return result; +} + std::array TexelOf(std::int32_t x, std::int32_t y) { return {240.0 * ((x + y) & 1), 16.0 * x + 16.0, 32.0 * y + 32.0, 240.0}; } @@ -147,7 +180,7 @@ std::array TextureDescriptor(const void* data, std::uint32_t l using Samples = std::vector>; -Samples Run(AgcDriver::VulkanDevice& device, const std::array& texture, const std::array& sampler, const std::vector& coordinates, std::span code = Code) { +Samples Run(AgcDriver::VulkanDevice& device, const Program& program, const std::array& texture, const std::array& sampler, const std::vector& coordinates) { Samples samples; for (std::size_t first = 0; first < coordinates.size(); first += Threads) { Buffer.fill(0xdeadbeefu); @@ -155,12 +188,14 @@ Samples Run(AgcDriver::VulkanDevice& device, const std::array& const auto coordinate = first + lane < coordinates.size() ? coordinates[first + lane] : Coordinate{0.5f, 0.5f}; Buffer[lane * Words] = std::bit_cast(coordinate.u); Buffer[lane * Words + 1u] = std::bit_cast(coordinate.v); + Buffer[lane * Words + 2u] = 0xffffc0c0u | (static_cast(coordinate.du) & 0x3fu) | ((static_cast(coordinate.dv) & 0x3fu) << 8u); } std::vector userData(16, 0u); const auto buffer = BufferDescriptor(Buffer.data(), static_cast(Buffer.size() * 4u)); std::copy(buffer.begin(), buffer.end(), userData.begin()); std::copy(texture.begin(), texture.end(), userData.begin() + 4); std::copy(sampler.begin(), sampler.end(), userData.begin() + 12); + const auto code = program.code; const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {false, false, false}, false, 1}; ShaderRecompiler::RecompileRequest request{ @@ -203,15 +238,19 @@ std::array Reference(const SamplerCase& sampler, const Coordinate& co return result; } -void Check(const SamplerCase& sampler, const char* image, const std::vector& coordinates, const Samples& samples) { +void Check(const Program& program, const SamplerCase& sampler, const char* image, const std::vector& coordinates, const Samples& samples) { Require(samples.size() == coordinates.size(), "image sample unnormalized: a dispatch lost samples"); const double tolerance = sampler.linear ? 0.5 : 0.25; for (std::size_t index = 0; index < coordinates.size(); ++index) { - const auto expected = Reference(sampler, coordinates[index]); - for (std::uint32_t instruction = 0; instruction < Instructions.size(); ++instruction) { + const auto& coordinate = coordinates[index]; + for (std::uint32_t instruction = 0; instruction < program.instructions.size(); ++instruction) { + const auto offset = program.offsets[instruction]; + const std::int32_t du = offset == Offset::Lane ? coordinate.du : offset == Offset::Constant ? ConstantOffsetU : 0; + const std::int32_t dv = offset == Offset::Lane ? coordinate.dv : offset == Offset::Constant ? ConstantOffsetV : 0; + const auto expected = Reference(sampler, {coordinate.u + static_cast(du), coordinate.v + static_cast(dv)}); for (std::uint32_t component = 0; component < 4u; ++component) { const double value = static_cast(std::bit_cast(samples[index][instruction * 4u + component])) * 255.0; - Require(std::fabs(value - expected[component]) <= tolerance, std::string(Instructions[instruction]) + ", " + sampler.name + ", " + image + ": (" + std::to_string(coordinates[index].u) + ", " + std::to_string(coordinates[index].v) + ") component " + std::to_string(component) + " is " + std::to_string(value) + ", expected " + std::to_string(expected[component])); + Require(std::fabs(value - expected[component]) <= tolerance, std::string(program.instructions[instruction]) + ", " + sampler.name + ", " + image + ": (" + std::to_string(coordinate.u) + ", " + std::to_string(coordinate.v) + ") offset (" + std::to_string(du) + ", " + std::to_string(dv) + ") component " + std::to_string(component) + " is " + std::to_string(value) + ", expected " + std::to_string(expected[component])); } } } @@ -220,7 +259,7 @@ void Check(const SamplerCase& sampler, const char* image, const std::vector& texture, const std::array& sampler, std::string_view reason, const char* what) { const std::vector one{{0.5f, 0.5f}}; try { - static_cast(Run(device, texture, sampler, one)); + static_cast(Run(device, Plain, texture, sampler, one)); } catch (const std::exception& error) { Require(std::string_view(error.what()).find(reason) != std::string_view::npos, std::string(what) + ": unexpected error: " + error.what()); return; @@ -239,12 +278,14 @@ int main() { const auto coordinates = Coordinates(); const auto single = TextureDescriptor(SingleLevel.data(), 0u, 0u); const auto multi = TextureDescriptor(MultiLevel.data(), 0u, StorageLevels - 1u); + const auto offsetCoordinates = OffsetCoordinates(); for (const auto& sampler : Samplers) { - const auto singleSamples = Run(*device, single, sampler.words, coordinates); - Check(sampler, "1-level image", coordinates, singleSamples); - const auto multiSamples = Run(*device, multi, sampler.words, coordinates); - Check(sampler, "first level of a 4-level image", coordinates, multiSamples); + const auto singleSamples = Run(*device, Plain, single, sampler.words, coordinates); + Check(Plain, sampler, "1-level image", coordinates, singleSamples); + const auto multiSamples = Run(*device, Plain, multi, sampler.words, coordinates); + Check(Plain, sampler, "first level of a 4-level image", coordinates, multiSamples); Require(multiSamples == singleSamples, std::string(sampler.name) + ": the first level of a 4-level image does not sample like a 1-level image"); + Check(Offsets, sampler, "1-level image", offsetCoordinates, Run(*device, Offsets, single, sampler.words, offsetCoordinates)); } ExpectFailure(*device, TextureDescriptor(MultiLevel.data(), 2u, 2u), Samplers[0].words, "single-level", "a 3-level view"); ExpectFailure(*device, single, {0x00008092u, 0x00fff000u, 0x05100000u, 0u}, "different minification", "unequal minification and magnification filters"); @@ -256,7 +297,7 @@ int main() { return descriptor; }; for (const auto& sampler : Samplers) { - for (const auto& sample : Run(*device, constantView(Type3D, Format8888UNorm, OneZeroOneZero), sampler.words, coordinates)) { + for (const auto& sample : Run(*device, Plain, constantView(Type3D, Format8888UNorm, OneZeroOneZero), sampler.words, coordinates)) { for (std::uint32_t index = 0; index < Results; ++index) { const std::uint32_t expected = index % 2u == 0u ? 0x3f800000u : 0u; Require(sample[index] == expected, std::string(sampler.name) + ": a 3D view whose channels select (1, 0, 1, 0) returned " + Hex(sample[index]) + " for result " + std::to_string(index)); @@ -275,10 +316,11 @@ int main() { {"a 1D view selecting (0, 0, 0, 0)", constantView(Type1D, Format8888UNorm, 0u), 0u}, }}; const std::array constantInstructions{"image_sample_lz dmask 0x6", "image_gather4_lz dmask 0x2", "image_gather4_lz dmask 0x4"}; + const Program constantProgram{ConstantCode, constantInstructions, Plain.offsets}; for (const auto& constant : constants) { auto normalized = Samplers[0].words; normalized[0] &= ~0x8000u; - for (const auto& sample : Run(*device, constant.texture, normalized, coordinates, ConstantCode)) { + for (const auto& sample : Run(*device, constantProgram, constant.texture, normalized, coordinates)) { const std::array expected{0u, constant.one, 0u, 0u, 0u, 0u, constant.one, constant.one, constant.one, constant.one}; for (std::uint32_t index = 0; index < expected.size(); ++index) { const auto instruction = constantInstructions[index < 2u ? 0u : index < 6u ? 1u : 2u]; diff --git a/core/shader/recompiler/IntermediateRepresentation/include/IntermediateRepresentation/IrMetadata/Resources.hpp b/core/shader/recompiler/IntermediateRepresentation/include/IntermediateRepresentation/IrMetadata/Resources.hpp index 882457f94..27099a8ca 100644 --- a/core/shader/recompiler/IntermediateRepresentation/include/IntermediateRepresentation/IrMetadata/Resources.hpp +++ b/core/shader/recompiler/IntermediateRepresentation/include/IntermediateRepresentation/IrMetadata/Resources.hpp @@ -87,6 +87,7 @@ struct ImageResource { bool constantSwizzleCompatible = true; bool flatVolumeCompatible = true; bool flatLineCompatible = true; + bool foldTexelOffsetsOnly = false; std::uint32_t byElements = 0; std::uint32_t byComponents = 0; IrBufferFormat packedFormat = IrBufferFormat::Invalid; @@ -123,10 +124,15 @@ struct SamplerResource { bool forcePointFiltering = false; bool depthCompare = false; std::uint8_t uses = 0; + bool foldTexelOffsetsOnly = false; bool operator==(const SamplerResource& other) const = default; }; +inline bool FoldsTexelOffsets(const SamplerResource& sampler) { + return (sampler.uses & SamplerUseOffset) != 0u && (sampler.uses & ~static_cast(SamplerUseExplicitLod | SamplerUseOffset)) == 0u && !sampler.depthCompare; +} + struct SampledResourcePair { std::uint32_t image = 0; std::uint32_t sampler = 0; diff --git a/core/shader/recompiler/Optimization/include/Optimization/ResourceMaterializer.hpp b/core/shader/recompiler/Optimization/include/Optimization/ResourceMaterializer.hpp index 5bd14939e..2dbf39823 100644 --- a/core/shader/recompiler/Optimization/include/Optimization/ResourceMaterializer.hpp +++ b/core/shader/recompiler/Optimization/include/Optimization/ResourceMaterializer.hpp @@ -19,6 +19,7 @@ public: static std::vector RuntimeImageModes(const ImageResource& image); static std::uint32_t RuntimeImageMode(const ImageResource& image, const DescriptorValue& descriptor, std::span modes); static void PrepareImageModes(ShaderInfo& info); + static const char* UnnormalizedSampleMismatch(const ImageResource& image); [[nodiscard]] IrResourcePlan ExtractPlan(const IrProgram& program) const; void Materialize(const IrResourcePlan& program, const SrtRuntime& runtime, ResourceSnapshot& snapshot) const; static std::uint64_t SpecializationNanoseconds(); diff --git a/core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp b/core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp index 4dfca8bb8..b2c07bab2 100644 --- a/core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp +++ b/core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp @@ -150,7 +150,6 @@ const char* UnnormalizedUseReason(std::uint32_t uses) { switch (uses & (~uses + 1u)) { case SamplerUseImplicitLod: return "is used by an implicit-LOD sample"; case SamplerUseGradient: return "is used by a sample with derivatives"; - case SamplerUseOffset: return "is used with a texel offset"; case SamplerUseCompare: return "is used with depth comparison"; case SamplerUseGather: return "is used by a gather"; case SamplerUseQueryLod: return "is used by image_get_lod"; @@ -167,11 +166,14 @@ UnnormalizedProof ProveUnnormalized(const ShaderInfo& info, const ResourceSnapsh UnnormalizedProof proof{std::vector(info.samplers.size()), std::vector(info.images.size())}; for (std::uint32_t r = 0; r < info.samplers.size(); r++) { if (snapshot.samplers.at(r).dwordCount != 4u) fail("sampler descriptor must contain four dwords"); + const auto& sampler = info.samplers[r]; if ((snapshot.samplers[r].dwords[0] & ForceUnnormalizedBit) == 0u) { + if (sampler.foldTexelOffsetsOnly) { + fail("DescriptorBindingBuilder::Populate normalized guest sampler is used with a texel offset that is not a constant, which image sampling takes only with VK_KHR_maintenance8 and shaderImageGatherExtended"); + } continue; } - const auto& sampler = info.samplers[r]; - const std::uint32_t unsupported = sampler.uses & ~static_cast(SamplerUseExplicitLod); + const std::uint32_t unsupported = sampler.uses & ~static_cast(SamplerUseExplicitLod | SamplerUseOffset); if (unsupported != 0u) { failUnnormalized(UnnormalizedUseReason(unsupported)); } @@ -184,21 +186,10 @@ UnnormalizedProof ProveUnnormalized(const ShaderInfo& info, const ResourceSnapsh } const auto& base = info.images.at(pair.image); const auto& image = info.runtimeImageModes.at(pair.image).at(ResourceMaterializer::RuntimeImageMode(base, snapshot.images.at(pair.image), info.runtimeImageModes.at(pair.image))); - if (image.indirectRoot != ImageResource::NoIndirectImage) { - failUnnormalized("samples an image selected at run time"); - } - if (image.constantSwizzle) { - continue; - } - if ((image.dimension != RdnaImageDimension::Dim1D && image.dimension != RdnaImageDimension::Dim2D) || image.cube) { - failUnnormalized("samples a 1D-array, 2D-array, 3D, cube or multisampled image"); - } - if (image.depthCompare) { - failUnnormalized("is used with depth comparison"); - } - if (image.conversionFormat != IrBufferFormat::Invalid || image.packed) { - failUnnormalized("samples an image that needs a format conversion or packed access"); + if (const auto* reason = ResourceMaterializer::UnnormalizedSampleMismatch(image)) { + failUnnormalized(reason); } + if (image.constantSwizzle) continue; proof.images[pair.image] = true; } proof.samplers[r] = true; @@ -253,6 +244,9 @@ DescriptorBindingPlan DescriptorBindingBuilder::Prepare(const IrBindingLayout& l if (info.images.size() > ShaderInfo::MaxImages) fail("runtime image count exceeds the static capacity"); DescriptorBindingPlan plan; const auto unnormalized = ProveUnnormalized(info, snapshot); + for (std::uint32_t index = 0; index < info.samplers.size(); ++index) { + if (FoldsTexelOffsets(info.samplers[index])) plan.specialization.push_back({PipelineSpecialization::FoldOffsetBase + index, unnormalized.samplers[index] ? 1u : 0u}); + } std::vector compareStates(info.images.size()); for (std::uint32_t index = 0; index < info.images.size(); ++index) { if (!info.images[index].depthCompare) continue; diff --git a/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp b/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp index 070aaef87..6b9f1ec19 100644 --- a/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp +++ b/core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp @@ -746,9 +746,21 @@ std::vector ResourceMaterializer::RuntimeImageModes(const ImageRe } } if (modes.empty()) throw std::runtime_error("image instruction has no supported runtime modes"); + if (image.foldTexelOffsetsOnly && std::ranges::any_of(modes, [](const ImageResource& mode) { return UnnormalizedSampleMismatch(mode) == nullptr; })) { + std::erase_if(modes, [](const ImageResource& mode) { return UnnormalizedSampleMismatch(mode) != nullptr; }); + } return modes; } +const char* ResourceMaterializer::UnnormalizedSampleMismatch(const ImageResource& image) { + if (image.indirectRoot != ImageResource::NoIndirectImage) return "samples an image selected at run time"; + if (image.constantSwizzle) return nullptr; + if ((image.dimension != RdnaImageDimension::Dim1D && image.dimension != RdnaImageDimension::Dim2D) || image.cube) return "samples a 1D-array, 2D-array, 3D, cube or multisampled image"; + if (image.depthCompare) return "is used with depth comparison"; + if (image.conversionFormat != IrBufferFormat::Invalid || image.packed) return "samples an image that needs a format conversion or packed access"; + return nullptr; +} + std::uint32_t ResourceMaterializer::RuntimeImageMode(const ImageResource& image, const DescriptorValue& descriptor, std::span modes) { if (descriptor.dwordCount != 8u) throw std::runtime_error("runtime image descriptor must contain eight dwords"); if (modes.empty()) throw std::runtime_error("prepared runtime image modes are missing"); @@ -776,6 +788,7 @@ std::uint32_t ResourceMaterializer::RuntimeImageMode(const ImageResource& image, } for (std::uint32_t index = 0u; index < modes.size(); ++index) { const auto& mode = modes[index]; + if (mode.constantSwizzle) continue; if (((mode.emulatedCompare & EmulatedCompare::Enabled) != 0u) != emulated) continue; if (decoded.fmask) { if (mode.packedFormat == IrBufferFormat::Fmask8_S2_F1) return index; @@ -784,6 +797,7 @@ std::uint32_t ResourceMaterializer::RuntimeImageMode(const ImageResource& image, if (mode.numericClass == decoded.numericClass && mode.dimension == decoded.dimension && mode.conversionFormat == decoded.conversionFormat && mode.packedFormat == decoded.packedFormat && mode.cube == decoded.cube && mode.depthBits == decoded.depthBits && mode.depthUnorm16 == decoded.depthUnorm16 && mode.srgbDecode == decoded.srgbDecode) return index; } if (image.dimension == RdnaImageDimension::Dim1D && decoded.dimension != RdnaImageDimension::Dim1D) throw std::runtime_error("image address has too few coordinate components"); + if (image.foldTexelOffsetsOnly) throw std::runtime_error("image sample with a texel offset that is not a constant takes only images an unnormalized S# can sample without VK_KHR_maintenance8 and shaderImageGatherExtended"); const auto describe = [](IrTextureNumericClass numeric, RdnaImageDimension dimension, IrBufferFormat conversion, IrBufferFormat packed, bool cube, bool depthBits, bool depthUnorm16, bool srgb) { return "{class " + std::to_string(static_cast(numeric)) + ", dim " + std::to_string(static_cast(dimension)) + ", conversion " + std::to_string(static_cast(conversion)) + ", packed " + std::to_string(static_cast(packed)) + ", cube " + std::to_string(cube) + ", depthBits " + std::to_string(depthBits) + ", unorm16 " + std::to_string(depthUnorm16) + ", srgb " + std::to_string(srgb) + "}"; }; @@ -807,12 +821,17 @@ void ResourceMaterializer::ApplyStaticInterface(IrProgram& program, bool nativeS for (const auto* inst : block->Instructions()) { if (inst->Opcode() != IrOpcode::ImageSampleRaw) continue; const auto& memory = resources.memoryInfo.at(inst->Flags().index); - if ((memory.imageSampleFlags & (RdnaImageSampleFlagCompare | RdnaImageSampleFlagOffset)) != (RdnaImageSampleFlagCompare | RdnaImageSampleFlagOffset)) continue; + if ((memory.imageSampleFlags & RdnaImageSampleFlagOffset) == 0u) continue; const auto* address = inst->Argument(2)->Resolve(); const auto component = GetRdnaImageAddressComponentLayout(memory.imageSampleFlags, 0u); const auto argument = component.bitOffset / 32u; if (component.bitWidth == 32u && argument < address->ArgumentCount() && address->Argument(argument)->Resolve()->HasImmediate()) continue; - images.at(memory.resource).emulatedCompare |= EmulatedCompare::NativeOffsetUnsupported; + if ((memory.imageSampleFlags & RdnaImageSampleFlagCompare) != 0u) { + images.at(memory.resource).emulatedCompare |= EmulatedCompare::NativeOffsetUnsupported; + } else { + images.at(memory.resource).foldTexelOffsetsOnly = true; + resources.info.samplers.at(memory.sampler).foldTexelOffsetsOnly = true; + } } } } diff --git a/core/shader/recompiler/PipelineSpecialization.hpp b/core/shader/recompiler/PipelineSpecialization.hpp index f6ae06676..878b0a74d 100644 --- a/core/shader/recompiler/PipelineSpecialization.hpp +++ b/core/shader/recompiler/PipelineSpecialization.hpp @@ -32,6 +32,7 @@ inline constexpr std::uint32_t DescriptorIndexBase = 16384u; inline constexpr std::uint32_t DescriptorIndexStride = 128u; inline constexpr std::uint32_t ImageModeBase = 32768u; inline constexpr std::uint32_t ImageModeStride = 256u; +inline constexpr std::uint32_t FoldOffsetBase = 57344u; inline constexpr std::uint32_t DescriptorIndex(std::uint32_t binding, std::uint32_t element) { return DescriptorIndexBase + binding * DescriptorIndexStride + element; diff --git a/core/shader/recompiler/ShaderDiskCache.cpp b/core/shader/recompiler/ShaderDiskCache.cpp index 7100a5806..a9bc8fb13 100644 --- a/core/shader/recompiler/ShaderDiskCache.cpp +++ b/core/shader/recompiler/ShaderDiskCache.cpp @@ -488,6 +488,7 @@ void encodeInfo(Writer& writer, const CompiledShaderInfo& compiled) { out.Value(image.constantSwizzleCompatible); out.Value(image.flatVolumeCompatible); out.Value(image.flatLineCompatible); + out.Value(image.foldTexelOffsetsOnly); out.Value(image.byElements); out.Value(image.byComponents); out.Value(image.packedFormat); @@ -505,6 +506,7 @@ void encodeInfo(Writer& writer, const CompiledShaderInfo& compiled) { out.Value(sampler.forcePointFiltering); out.Value(sampler.depthCompare); out.Value(sampler.uses); + out.Value(sampler.foldTexelOffsetsOnly); }); writer.List(info.sampledPairs, [](Writer& out, const SampledResourcePair& pair) { out.Value(pair.image); @@ -564,7 +566,7 @@ void decodeInfo(Reader& reader, CompiledShaderInfo& compiled) { in.Value(buffer.scalar); in.Value(buffer.typedAlignment); }); - reader.List(info.images, 74, [](Reader& in, ImageResource& image) { + reader.List(info.images, 75, [](Reader& in, ImageResource& image) { in.Value(image.source); in.Value(image.firstUsePc); in.Value(image.resourceClass); @@ -593,6 +595,7 @@ void decodeInfo(Reader& reader, CompiledShaderInfo& compiled) { in.Value(image.constantSwizzleCompatible); in.Value(image.flatVolumeCompatible); in.Value(image.flatLineCompatible); + in.Value(image.foldTexelOffsetsOnly); in.Value(image.byElements); in.Value(image.byComponents); in.Value(image.packedFormat); @@ -605,13 +608,14 @@ void decodeInfo(Reader& reader, CompiledShaderInfo& compiled) { bool preparedImageModes = false; reader.Value(preparedImageModes); if (preparedImageModes) ResourceMaterializer::PrepareImageModes(info); - reader.List(info.samplers, 15, [](Reader& in, SamplerResource& sampler) { + reader.List(info.samplers, 16, [](Reader& in, SamplerResource& sampler) { in.Value(sampler.source); in.Value(sampler.firstUsePc); in.Value(sampler.copyOf); in.Value(sampler.forcePointFiltering); in.Value(sampler.depthCompare); in.Value(sampler.uses); + in.Value(sampler.foldTexelOffsetsOnly); }); reader.List(info.sampledPairs, 12, [](Reader& in, SampledResourcePair& pair) { in.Value(pair.image); diff --git a/core/shader/recompiler/ShaderDiskCache.hpp b/core/shader/recompiler/ShaderDiskCache.hpp index 08b1d6c8d..df13ae723 100644 --- a/core/shader/recompiler/ShaderDiskCache.hpp +++ b/core/shader/recompiler/ShaderDiskCache.hpp @@ -12,7 +12,7 @@ namespace ShaderRecompiler::ShaderDiskCache { -inline constexpr std::uint32_t FormatVersion = 25; +inline constexpr std::uint32_t FormatVersion = 26; enum class LoadStatus { Loaded, diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp index 3d65d9ea1..a2c4943a8 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp @@ -470,6 +470,15 @@ std::uint32_t PackedOffset(SpirvValueEmitContext& ctx, const ImageEmitAccess& ac return result; } +std::uint32_t FoldedOffsetCoord(SpirvValueEmitContext& ctx, const ImageEmitAccess& access, const SampleSetup& setup) { + auto& state = ctx.state; + const auto components = setup.dimensionInfo.spatialComponents; + const auto type = components == 1u ? TypeF32(state) : TypeF32Vector(state, components); + const auto sum = Binary(state, spv::OpFAdd, type, setup.coord, Unary(state, spv::OpConvertSToF, type, PackedOffset(ctx, access, setup.layout))); + state.module.AddAnnotation(spv::OpDecorate, sum, spv::DecorationNoContraction); + return sum; +} + std::uint32_t HorizontalOffsets(SpirvValueEmitContext& ctx, const ImageEmitAccess& access) { auto& state = ctx.state; const auto components = RdnaImageDimensionInfoFor(access.image.dimension).spatialComponents; @@ -1424,15 +1433,20 @@ void EmitSampleOp(SpirvValueEmitContext& ctx, const ImageEmitAccess& access, con const auto* value = access.address.Argument(argument)->Resolve(); return value->HasImmediate() ? value : nullptr; }; - if (setup.layout.offset != NoImageComponent && constantOffset() == nullptr) { - const bool gatherExtended = std::find(state.supportedCapabilities.begin(), state.supportedCapabilities.end(), static_cast(spv::CapabilityImageGatherExtended)) != state.supportedCapabilities.end(); - if (!state.nonConstantImageOffsets || !gatherExtended) { - ctx.Fail(access.inst, "has a texel offset that is not a constant, which image sampling takes only with VK_KHR_maintenance8 and shaderImageGatherExtended"); - } + const bool hasOffset = setup.layout.offset != NoImageComponent; + const bool foldOffset = hasOffset && FoldsTexelOffsets(state.program.Info().samplers.at(mem.sampler)) && ResourceMaterializer::UnnormalizedSampleMismatch(image) == nullptr; + const auto foldedMask = operandMask; + const auto foldedOperands = operands; + const bool gatherExtended = std::find(state.supportedCapabilities.begin(), state.supportedCapabilities.end(), static_cast(spv::CapabilityImageGatherExtended)) != state.supportedCapabilities.end(); + const bool offsetOperand = hasOffset && (constantOffset() != nullptr || (state.nonConstantImageOffsets && gatherExtended)); + if (hasOffset && !offsetOperand && !foldOffset) { + ctx.Fail(access.inst, "has a texel offset that is not a constant, which image sampling takes only with VK_KHR_maintenance8 and shaderImageGatherExtended"); + } + if (offsetOperand && constantOffset() == nullptr) { state.module.EmitCapability(spv::CapabilityImageGatherExtended); operandMask |= spv::ImageOperandsOffsetMask; operands.insert(operands.end() - ((operandMask & spv::ImageOperandsMinLodMask) != 0u ? 1 : 0), PackedOffset(ctx, access, setup.layout)); - } else if (setup.layout.offset != NoImageComponent) { + } else if (offsetOperand) { const auto bits = constantOffset()->ImmediateU32(); std::array values{}; for (std::uint32_t index = 0; index < setup.dimensionInfo.spatialComponents; index++) { @@ -1447,17 +1461,45 @@ void EmitSampleOp(SpirvValueEmitContext& ctx, const ImageEmitAccess& access, con operands.insert(operands.end() - ((operandMask & spv::ImageOperandsMinLodMask) != 0u ? 1 : 0), offset); } - const auto sampled = MakeSampledImage(state, mem.resource, mem.sampler, access.slot); - const auto sample = state.module.AllocateId(); - std::vector words = {opcode, resultType, sample, sampled, setup.coord}; - if (setup.dref) { - words.push_back(drefValue); + const auto emitSample = [&](std::uint32_t coord, std::uint32_t mask, const std::vector& arguments) { + const auto sampled = MakeSampledImage(state, mem.resource, mem.sampler, access.slot); + const auto sample = state.module.AllocateId(); + std::vector words = {opcode, resultType, sample, sampled, coord}; + if (setup.dref) { + words.push_back(drefValue); + } + if (mask != 0u) { + words.push_back(mask); + words.insert(words.end(), arguments.begin(), arguments.end()); + } + state.module.AddFunction(words); + return sample; + }; + std::uint32_t sample = 0; + if (!foldOffset) { + sample = emitSample(setup.coord, operandMask, operands); + } else if (!offsetOperand) { + sample = emitSample(FoldedOffsetCoord(ctx, access, setup), foldedMask, foldedOperands); + } else { + const auto selector = state.module.SpecializationConstant(TypeU32(state), PipelineSpecialization::FoldOffsetBase + mem.sampler, 0u); + const auto folded = Binary(state, spv::OpINotEqual, TypeBool(state), selector, ConstantU32(state, 0u)); + const auto foldLabel = state.module.AllocateId(); + const auto operandLabel = state.module.AllocateId(); + const auto merge = state.module.AllocateId(); + state.module.AddFunction(spv::OpSelectionMerge, merge, spv::SelectionControlMaskNone); + state.module.AddFunction(spv::OpBranchConditional, folded, foldLabel, operandLabel); + EmitLabel(state, foldLabel); + const auto foldSample = emitSample(FoldedOffsetCoord(ctx, access, setup), foldedMask, foldedOperands); + const auto foldEnd = state.currentLabel; + state.module.AddFunction(spv::OpBranch, merge); + EmitLabel(state, operandLabel); + const auto operandSample = emitSample(setup.coord, operandMask, operands); + const auto operandEnd = state.currentLabel; + state.module.AddFunction(spv::OpBranch, merge); + EmitLabel(state, merge); + sample = state.module.AllocateId(); + state.module.AddFunction(spv::OpPhi, resultType, sample, foldSample, foldEnd, operandSample, operandEnd); } - if (operandMask != 0u) { - words.push_back(operandMask); - words.insert(words.end(), operands.begin(), operands.end()); - } - state.module.AddFunction(words); const auto result = setup.dref ? sample : UnpackImageTexel(ctx, access, sample); ctx.Define(access.inst, TableResult(ctx, access, ResultVector(ctx, access, result, setup.numericClass, setup.dref, false))); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 542edd980..6a44b99e8 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -389,8 +389,9 @@ Throughout the project, every function at every stage either **does exactly what - [MIMG PCK2/PCK4 loads](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) (`image_load[_mip]_pck2/pck4`, opcodes 0x70, 0x71, 0x73 and 0x74) were measured on RDNA2 (gfx1035) with the [hardware oracle](HW_ORACLE.md) for 2D images with identity swizzle. With data mask 0x1, they write one dword: the raw bits of the N texels (PCK2: 2, PCK4: 4) from `x & ~(N - 1)` on row y, concatenated (element k at bit `k * texel bits`) and zero-extended. The dword is zero when the mip level is past the last level, `x + N - 1` is outside the level's width (unaligned x, unsigned), or y is outside its height. Data only comes back when N texels fit in 32 bits: R8, R16 or RG8 for PCK2, R8 for PCK4. Wider texels (R32, RG16, RGBA8) and masks other than 0x1 write zeros on hardware. Those masks throw at decode, and formats outside UNORM/UINT/SINT R8, R16 and RG8 throw. #1314's earlier measurement saw only zeros because it used masks 0x3 and 0xf. The stores (0x76, 0x77, 0x79 and 0x7a) decode but throw at translation. They are not measured, and opcode 0x76 on 32_32_32_32_FLOAT hung the GPU (#1314). - [3D image accesses bound to a 2D or 2D-array T#](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) follow CouchRiv's [hardware oracle](HW_ORACLE.md) measurement on RDNA2 (gfx1036, #2023) and adriadam10's (gfx1035, #2140), not the PS5's gfx1013. On a 2D T#, `image_sample_lz` ignores r and reads the (u,v) texel, `image_load` and `image_load_mip` read the (x,y) texel when z is 0 and return zero otherwise, and `image_get_resinfo` returns (width, height, 1, levels). On a 2D-array T#, these accesses behave like dim 2D-array: r or z is the layer, and `image_get_resinfo` returns the layer count as its third component. Every other access of such an image, including samples with an LOD, bias, derivatives, offset or comparison, gathers, `image_get_lod` and stores, keeps throwing because it is not measured. - [Unnormalized S# coordinates](../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestSamplerResource.cpp) (libSceAgcDriver) - `FORCE_UNNORMALIZED` uses a Vulkan `unnormalizedCoordinates` sampler, so the S# LOD range, LOD bias and mip filter are not applied, which is exact only on a one-level view. -- [Unnormalized S#s](../../core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp) are accepted only when direct and used only by explicit-LOD `image_sample` without offset, comparison, derivatives or `_a`, on a single-level, single-layer 1D or 2D view of mip 0, even when the image is also sampled through a normalized S#. +- [Unnormalized S#s](../../core/shader/recompiler/Optimization/src/DescriptorBindingBuilder.cpp) are accepted only when direct and used only by explicit-LOD `image_sample` without comparison, derivatives or `_a`, on a single-level, single-layer 1D or 2D view of mip 0, even when the image is also sampled through a normalized S#. - An [unnormalized S#](../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestSamplerResource.cpp) (libSceAgcDriver) throws with unequal minification and magnification filters, anisotropy, `TRUNC_COORD`, `MC_COORD_TRUNC` or X/Y clamping other than last-texel or border. +- [Unnormalized texel offsets](../../core/shader/recompiler/SpirvBackend/src/SpirvImageEmitter.cpp) are added to the f32 coordinates: exact on RDNA2 (gfx1035), which applies the offset before the clamp ([measured by adriadam10](https://github.com/boykopovar/AnyPS5/pull/822#issuecomment-6033192725)). - [Image atomics](../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) on 32_SINT and 32_FLOAT surfaces act on the raw dword through an R32_UINT storage view; their float results are assumed to match those measured on 32_UINT surfaces. Other formats throw. - [`s_memtime`](../../core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp) on RADV with a GFX10.3 to GFX11.7 GPU (RDNA 2, RDNA 3, RDNA 3.5) reads the Vulkan device clock instead of the subgroup clock, which [ACO](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/compiler/instruction_selection/aco_select_nir_intrinsics.cpp) reads from the 20-bit `SHADER_CYCLES` register there and which wraps every 2^20 cycles. The device clock is a monotonic 64-bit counter, but it counts the 100 MHz reference clock (the same one as `s_memrealtime`), not shader cycles, so `s_memtime` deltas are about 20 times smaller than on hardware. The families are matched by the name RADV reports ([`NarrowSubgroupClock`](../../core/libs/prx/libSceAgcDriver/Execution/src/SubgroupClock.cpp)); a family added to Mesa later reads the subgroup clock until it is listed. AMD's proprietary Windows driver also exposes a 20-bit subgroup clock on device 0x1114 (Radeon 860M, measured in #1615), device 0x744c (Radeon RX 7900 XT, measured in #1632), and device 0x747e (Radeon RX 7800 XT, measured in #1632); other AMD proprietary devices remain unverified. - [`s_memtime` and `s_memrealtime`](../../core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp) on llvmpipe read only the low 32 bits of the host's nanosecond clock at either scope: Mesa's [`emit_clock`](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/gallium/auxiliary/gallivm/lp_bld_nir_soa.c) builds the high dword with a left shift by 32, so it is always 0 and both counters wrap every 2^32 ns (about 4.3 s). The [shader clock test](../../core/libs/prx/libSceAgcDriver/tests/execution/ShaderClock.cpp) compares them modulo 2^32 there From 1c6eac425d5e60c8480890092d0c10d6d79db936 Mon Sep 17 00:00:00 2001 From: Merijn Date: Sun, 11 Oct 2026 10:11:31 +0200 Subject: [PATCH 433/460] feat(relinker): add --module-dir for modules loaded at run time Relinks the ELF modules of extra directories under the module parent directory. Modules the executable or another module needs start with the executable; the others are initialized when the title loads them with sceKernelLoadStartModule, with its start arguments (Windows: __aps5_guest_initialize export; Linux: left out of DT_NEEDED). Squashed onto main by the reviewer (history had a merge commit). --- core/libs/prx/libkernel/Module/ModuleArgs.hpp | 16 +++ .../libkernel/Module/src/DynamicLoader.cpp | 25 ++++ core/libs/prx/libkernel/Module/src/Module.cpp | 5 +- core/relinker/CMakeLists.txt | 2 + core/relinker/cli/include/Cli.hpp | 2 + core/relinker/cli/src/CliArgs.cpp | 9 +- .../domain/include/domain/GuestRuntime.hpp | 1 + .../src/windows/WindowsGuestModuleWriter.cpp | 9 ++ .../src/windows/WindowsGuestStartup.cpp | 2 + core/relinker/main.cpp | 2 +- .../include/relinker/guest/GuestImage.hpp | 4 +- .../relinker/src/guest/GuestModuleBuilder.cpp | 25 +++- .../relinker/tests/test_guest_module_dir.py | 108 ++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + docs/user/USAGE.md | 1 + 15 files changed, 203 insertions(+), 9 deletions(-) create mode 100644 core/libs/prx/libkernel/Module/ModuleArgs.hpp create mode 100644 core/relinker/relinker/tests/test_guest_module_dir.py diff --git a/core/libs/prx/libkernel/Module/ModuleArgs.hpp b/core/libs/prx/libkernel/Module/ModuleArgs.hpp new file mode 100644 index 000000000..67e847114 --- /dev/null +++ b/core/libs/prx/libkernel/Module/ModuleArgs.hpp @@ -0,0 +1,16 @@ +#ifndef CORE_LIBS_PRX_LIBKERNEL_MODULE_MODULEARGS_HPP +#define CORE_LIBS_PRX_LIBKERNEL_MODULE_MODULEARGS_HPP + +#include + +struct PendingModuleArgs { + std::size_t args = 0; + const void* argp = nullptr; +}; + +extern "C" { +const void* __aps5_get_pending_module_args_nid_no_patch(); +void __aps5_set_module_init_result_nid_no_patch(int result); +} + +#endif diff --git a/core/libs/prx/libkernel/Module/src/DynamicLoader.cpp b/core/libs/prx/libkernel/Module/src/DynamicLoader.cpp index ffe065082..0a20bc08d 100644 --- a/core/libs/prx/libkernel/Module/src/DynamicLoader.cpp +++ b/core/libs/prx/libkernel/Module/src/DynamicLoader.cpp @@ -1,5 +1,6 @@ #include "prx/libc/include/General.hpp" #include "prx/libc/include/GuestAllocations.hpp" +#include "prx/libkernel/Module/ModuleArgs.hpp" #include #include #include @@ -8,6 +9,7 @@ #include #include #include +#include #include #ifdef _WIN32 #ifndef NOMINMAX @@ -104,6 +106,28 @@ char* APS5_VABI dlerror_nid_postfix() { pendingError = false; return loaderError.data(); } +#ifdef _WIN32 +static void InitializeDeferredModule(HMODULE native) { + using Entry = int (APS5_VABI *)(std::size_t, const void*, void*); + using Initializer = void (APS5_VABI *)(int, char**, char**); + static std::mutex initializedLock; + static std::set initialized; + const auto* table = reinterpret_cast(GetProcAddress(native, "__aps5_guest_initialize")); + if (!table) return; + { + std::lock_guard lock(initializedLock); + if (!initialized.insert(native).second) return; + } + auto* base = reinterpret_cast(native); + const auto* pending = static_cast(__aps5_get_pending_module_args_nid_no_patch()); + if (table[0] != 0) __aps5_set_module_init_result_nid_no_patch(reinterpret_cast(base + table[0])(pending->args, pending->argp, nullptr)); + for (std::uint32_t index = 0; index < table[1]; ++index) { + const auto initializer = *reinterpret_cast(base + table[2 + index]); + if (initializer) initializer(0, nullptr, nullptr); + } +} +#endif + static std::filesystem::path RelinkedModulePath(const std::filesystem::path& path) { auto relinked = path; relinked += ".guest.prx"; @@ -126,6 +150,7 @@ void* APS5_VABI dlopen_nid_postfix(const char* path, int flags) { if (!*path) { Error("dlopen: empty module path"); return nullptr; } const auto resolved = RelinkedModulePath(ResolvePath_nid_no_patch(path)); module->native = LoadLibraryExW(resolved.c_str(), nullptr, LOAD_WITH_ALTERED_SEARCH_PATH); + if (module->native) InitializeDeferredModule(static_cast(module->native)); } if (!module->native) { char message[128]; diff --git a/core/libs/prx/libkernel/Module/src/Module.cpp b/core/libs/prx/libkernel/Module/src/Module.cpp index 9a46bd0db..08a153b1e 100644 --- a/core/libs/prx/libkernel/Module/src/Module.cpp +++ b/core/libs/prx/libkernel/Module/src/Module.cpp @@ -9,6 +9,7 @@ #include "prx/libkernel/DirectMemory/DirectMemory.hpp" #include "prx/libkernel/KernelErrors.hpp" #include "prx/libkernel/Module/EhFrame.hpp" +#include "prx/libkernel/Module/ModuleArgs.hpp" #ifdef _WIN32 #include #include @@ -180,10 +181,6 @@ int APS5_VABI sceKernelGetModuleInfoForUnwind(uint64_t addr, int flags, ModuleIn namespace { -struct PendingModuleArgs { - std::size_t args = 0; - const void* argp = nullptr; -}; thread_local PendingModuleArgs pendingModuleArgs; thread_local int pendingModuleInitResult = 0; diff --git a/core/relinker/CMakeLists.txt b/core/relinker/CMakeLists.txt index 6e0692d73..04d90be58 100644 --- a/core/relinker/CMakeLists.txt +++ b/core/relinker/CMakeLists.txt @@ -282,6 +282,8 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_guest_tls.py $) add_test(NAME guest_needed_modules COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_guest_needed_modules.py $) + add_test(NAME guest_module_dir COMMAND ${Python3_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_guest_module_dir.py $) add_test(NAME guest_module_directories COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_guest_module_directories.py $) add_test(NAME sce_module_path COMMAND ${Python3_EXECUTABLE} diff --git a/core/relinker/cli/include/Cli.hpp b/core/relinker/cli/include/Cli.hpp index 38776589d..6f7985c1c 100644 --- a/core/relinker/cli/include/Cli.hpp +++ b/core/relinker/cli/include/Cli.hpp @@ -3,6 +3,7 @@ #include #include +#include #include namespace Cli { @@ -26,6 +27,7 @@ struct Args { std::string sceModulePath; std::string runPath = "$ORIGIN/libs"; std::set excludedSceModules; + std::vector moduleDirectories; }; Args ParseArgs(int argc, char* argv[]); diff --git a/core/relinker/cli/src/CliArgs.cpp b/core/relinker/cli/src/CliArgs.cpp index f2c6a95a6..0528e8419 100644 --- a/core/relinker/cli/src/CliArgs.cpp +++ b/core/relinker/cli/src/CliArgs.cpp @@ -8,7 +8,7 @@ namespace Cli { const char* Usage() { - return "Usage: relinker [--help] [--windows | --macos] [--windows-diagnostics] [--windows-gui] [--skip-syscall-check] [--skip-sce-module | --sce-module-path ] [--exclude-sce-module ]... [--to-intel] [--to-rosetta] [unused-filter=0|1|2] [--registry] [--rpath ] [--lazy-binding] [--autorun] \n" + return "Usage: relinker [--help] [--windows | --macos] [--windows-diagnostics] [--windows-gui] [--skip-syscall-check] [--skip-sce-module | --sce-module-path ] [--exclude-sce-module ]... [--module-dir ]... [--to-intel] [--to-rosetta] [unused-filter=0|1|2] [--registry] [--rpath ] [--lazy-binding] [--autorun] \n" "Example: relinker input.elf output.elf"; } @@ -34,6 +34,10 @@ Args ParseArgs(int argc, char* argv[]) { if (i + 1 >= argc) throw std::runtime_error("--exclude-sce-module requires a file name"); args.excludedSceModules.insert(argv[++i]); + } else if (arg == "--module-dir") { + if (i + 1 >= argc) + throw std::runtime_error("--module-dir requires a directory"); + args.moduleDirectories.push_back(argv[++i]); } else if (arg == "--to-intel") { args.toIntel = true; } else if (arg == "--to-rosetta") { @@ -79,6 +83,9 @@ Args ParseArgs(int argc, char* argv[]) { if (args.skipSceModule && !args.excludedSceModules.empty()) throw std::runtime_error("--exclude-sce-module conflicts with --skip-sce-module"); + if (args.skipSceModule && !args.moduleDirectories.empty()) + throw std::runtime_error("--module-dir conflicts with --skip-sce-module"); + if (args.toWindows && args.toMacos) throw std::runtime_error("--windows conflicts with --macos"); if (args.toRosetta && args.toWindows) diff --git a/core/relinker/domain/include/domain/GuestRuntime.hpp b/core/relinker/domain/include/domain/GuestRuntime.hpp index 45f720ae8..b7e81a307 100644 --- a/core/relinker/domain/include/domain/GuestRuntime.hpp +++ b/core/relinker/domain/include/domain/GuestRuntime.hpp @@ -27,6 +27,7 @@ struct GuestRuntime { std::vector Names; std::vector UnmatchedImportModules; std::set ZeroReturnExports; + bool DeferInitialization = false; }; } diff --git a/core/relinker/elfpatcher/src/windows/WindowsGuestModuleWriter.cpp b/core/relinker/elfpatcher/src/windows/WindowsGuestModuleWriter.cpp index 107c446c6..f7ca50fa2 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsGuestModuleWriter.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsGuestModuleWriter.cpp @@ -139,6 +139,15 @@ std::vector GuestModuleWriter::WriteWindows(const Relinker::GuestI nextRva = AlignRva(nextRva + tlsExports.Data.size()); sections.push_back(std::move(tlsExports)); } + if (runtime.DeferInitialization) { + PeSection lifecycle{".ginit", nextRva, SectionRead | 0x40u, {}}; + Io::AppendU32(lifecycle.Data, guest.Init == 0 ? 0 : image.GetRva(guest.Init)); + Io::AppendU32(lifecycle.Data, CheckedRva(guest.InitArray.size())); + for (const auto slot : guest.InitArray) Io::AppendU32(lifecycle.Data, image.GetRva(slot, 8)); + if (!exports.emplace(Relinker::GuestInitializeExport, nextRva).second) throw Domain::RelinkerException("Duplicate guest export: " + std::string(Relinker::GuestInitializeExport)); + nextRva = AlignRva(nextRva + lifecycle.Data.size()); + sections.push_back(std::move(lifecycle)); + } if (exports.size() > 65535) throw Domain::RelinkerException("Too many guest PE exports"); PeSection exportSection{".edata", nextRva, SectionRead | 0x40u, std::vector(40)}; auto& data = exportSection.Data; diff --git a/core/relinker/elfpatcher/src/windows/WindowsGuestStartup.cpp b/core/relinker/elfpatcher/src/windows/WindowsGuestStartup.cpp index 0d6068306..8d241798f 100644 --- a/core/relinker/elfpatcher/src/windows/WindowsGuestStartup.cpp +++ b/core/relinker/elfpatcher/src/windows/WindowsGuestStartup.cpp @@ -29,6 +29,7 @@ void WindowsGuestStartup::callLifecycle(WindowsStubEmitter& code, const std::uin void WindowsGuestStartup::Initialize(WindowsStubEmitter& code, const std::vector& modules, const std::uint32_t handles) const { for (std::size_t index = 0; index < modules.size(); ++index) { + if (modules[index].DeferInitialization) continue; const auto handle = CheckedRva(handles + index * 8); callLifecycle(code, handle, modules[index].InitRva); for (const auto slot : modules[index].InitArrayRvas) callLifecycle(code, handle, slot, true); @@ -43,6 +44,7 @@ void WindowsGuestStartup::Finalize(WindowsStubEmitter& code, const std::vector 0; --index) { const auto handle = CheckedRva(handles + (index - 1) * 8); const auto& module = modules[index - 1]; + if (module.DeferInitialization) continue; for (auto slot = module.FiniArrayRvas.rbegin(); slot != module.FiniArrayRvas.rend(); ++slot) callLifecycle(code, handle, *slot, true); callLifecycle(code, handle, module.FiniRva); } diff --git a/core/relinker/main.cpp b/core/relinker/main.cpp index 922290659..c43d97743 100644 --- a/core/relinker/main.cpp +++ b/core/relinker/main.cpp @@ -108,7 +108,7 @@ int Run(const int argc, char* argv[]) { std::vector guestArtifacts; if (!args.skipSceModule) { - guestArtifacts = Relinker::GuestModuleBuilder().Build(Io::NativePath(args.inputPath), Io::NativePath(absPath), result.DynamicSection, args.toWindows, args.toMacos, args.toIntel, args.toRosetta, *syscallScanner, args.lazyBinding, args.runPath, args.excludedSceModules, Io::NativePath(args.sceModulePath)); + guestArtifacts = Relinker::GuestModuleBuilder().Build(Io::NativePath(args.inputPath), Io::NativePath(absPath), result.DynamicSection, args.toWindows, args.toMacos, args.toIntel, args.toRosetta, *syscallScanner, args.lazyBinding, args.runPath, args.excludedSceModules, Io::NativePath(args.sceModulePath), args.moduleDirectories); } if (args.writeRegistry) { diff --git a/core/relinker/relinker/include/relinker/guest/GuestImage.hpp b/core/relinker/relinker/include/relinker/guest/GuestImage.hpp index 8ba1296d6..c3d6692fa 100644 --- a/core/relinker/relinker/include/relinker/guest/GuestImage.hpp +++ b/core/relinker/relinker/include/relinker/guest/GuestImage.hpp @@ -7,6 +7,7 @@ #include #include #include +#include namespace Relinker { @@ -14,6 +15,7 @@ inline constexpr char GuestSymbolSuffix[] = "#guest"; inline constexpr char GuestModuleSuffix[] = ".guest.prx"; inline constexpr char GuestModulePattern[] = "sce_module/*.prx"; inline constexpr std::uint16_t AbsoluteSection = 0xfff1; +inline constexpr char GuestInitializeExport[] = "__aps5_guest_initialize"; struct GuestSymbol { std::string Name; @@ -58,7 +60,7 @@ struct GuestArtifact { class GuestModuleBuilder { public: - std::vector Build(const std::filesystem::path& inputPath, const std::filesystem::path& outputPath, Domain::SysVDynamicSection& dynamic, bool windows, bool macos, bool toIntel, bool toRosetta, ISyscallScanner& syscallScanner, bool lazyBinding, const std::string& runPath, const std::set& excludedModules, const std::filesystem::path& sceModulePath) const; + std::vector Build(const std::filesystem::path& inputPath, const std::filesystem::path& outputPath, Domain::SysVDynamicSection& dynamic, bool windows, bool macos, bool toIntel, bool toRosetta, ISyscallScanner& syscallScanner, bool lazyBinding, const std::string& runPath, const std::set& excludedModules, const std::filesystem::path& sceModulePath, const std::vector& moduleDirectories = {}) const; }; } diff --git a/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp b/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp index 0fc02b314..238fdf832 100644 --- a/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp +++ b/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp @@ -50,7 +50,7 @@ std::string ModuleStem(std::string name, const bool windows) { } -std::vector GuestModuleBuilder::Build(const std::filesystem::path& inputPath, const std::filesystem::path& outputPath, Domain::SysVDynamicSection& dynamic, const bool windows, const bool macos, const bool toIntel, const bool toRosetta, ISyscallScanner& syscallScanner, const bool lazyBinding, const std::string& runPath, const std::set& excludedModules, const std::filesystem::path& sceModulePath) const { +std::vector GuestModuleBuilder::Build(const std::filesystem::path& inputPath, const std::filesystem::path& outputPath, Domain::SysVDynamicSection& dynamic, const bool windows, const bool macos, const bool toIntel, const bool toRosetta, ISyscallScanner& syscallScanner, const bool lazyBinding, const std::string& runPath, const std::set& excludedModules, const std::filesystem::path& sceModulePath, const std::vector& moduleDirectories) const { const auto root = std::filesystem::absolute(sceModulePath).lexically_normal(); if (!std::filesystem::exists(root)) throw Domain::RelinkerException("Guest module parent directory does not exist: " + Io::Utf8Path(root)); if (!std::filesystem::is_directory(root)) throw Domain::RelinkerException("Guest module parent path is not a directory: " + Io::Utf8Path(root)); @@ -66,6 +66,17 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path std::vector directories; if (hasSingular || hasPlural) directories.push_back(hasSingular ? singular : plural); if (hasPrx) directories.push_back(prx); + std::vector runtimeDirectories; + for (const auto& name : moduleDirectories) { + const auto relative = Io::NativePath(name); + if (relative.empty() || relative.is_absolute() || relative.lexically_normal().generic_string().starts_with("..")) + throw Domain::RelinkerException("--module-dir must be a directory inside the guest module parent directory: " + name); + const auto directory = (root / relative).lexically_normal(); + if (!std::filesystem::is_directory(directory)) throw Domain::RelinkerException("Guest module path is not a directory: " + Io::Utf8Path(directory)); + if (std::find(directories.begin(), directories.end(), directory) != directories.end()) throw Domain::RelinkerException("Duplicate guest module directory: " + Io::Utf8Path(directory)); + directories.push_back(directory); + runtimeDirectories.push_back(directory); + } std::vector paths; std::set unmatchedExclusions = excludedModules; const auto isElf = [](const std::filesystem::path& path) { @@ -263,6 +274,15 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path if (providers.empty()) systemImports[index].insert(symbolIndex); } } + std::vector required(images.size()); + for (const auto& name : neededNames) { + const auto found = findGuest(name); + if (found != guestNames.end()) required[found->second] = true; + } + for (const auto& needed : dependencies) for (const auto dependency : needed) required[dependency] = true; + const auto deferred = [&](const std::size_t index) { + return !required[index] && std::find(runtimeDirectories.begin(), runtimeDirectories.end(), images[index].SourcePath.parent_path().lexically_normal()) != runtimeDirectories.end(); + }; if (!windows) { for (auto& image : images) { for (std::size_t index = 1; index < image.Symbols.size(); ++index) { @@ -299,7 +319,7 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path Io::AppendU64(dynamic.DynamicSegmentData, dynamic.DynStrData.size()); Io::AppendString(dynamic.DynStrData, name); }; - for (const auto index : order) if (!windows) addNeeded("$ORIGIN/app0/" + images[index].SourcePath.parent_path().lexically_relative(root).generic_string() + "/" + images[index].OutputName); + for (const auto index : order) if (!windows && !deferred(index)) addNeeded("$ORIGIN/app0/" + images[index].SourcePath.parent_path().lexically_relative(root).generic_string() + "/" + images[index].OutputName); for (const auto& name : hostLibraries) addNeeded(name); if (!windows && runPath != "$ORIGIN" && !runPath.starts_with("$ORIGIN/") && !std::filesystem::path(runPath).is_absolute()) throw Domain::RelinkerException("Guest Linux run path must be absolute or begin with $ORIGIN"); const auto outputDirectory = std::filesystem::absolute(outputPath).parent_path().lexically_normal(); @@ -316,6 +336,7 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path runtime.UsePlatformTlsResolver = image.UsePlatformTlsResolver; runtime.Path = relativeDirectory + "/" + image.OutputName; runtime.Names = {Io::Utf8Path(image.SourcePath.filename()), image.Soname}; + runtime.DeferInitialization = deferred(index); std::vector output; if (windows) output = Elfpatcher::GuestModuleWriter().WriteWindows(image, runtime); else { diff --git a/core/relinker/relinker/tests/test_guest_module_dir.py b/core/relinker/relinker/tests/test_guest_module_dir.py new file mode 100644 index 000000000..97f3c852c --- /dev/null +++ b/core/relinker/relinker/tests/test_guest_module_dir.py @@ -0,0 +1,108 @@ +from pathlib import Path +import os +import struct +import subprocess +import sys +import tempfile + +from test_guest_intel_trampolines import main_fixture, pe_sections +from test_guest_module_directories import module_with_symbol, needed_libraries +from test_guest_needed_modules import executable_with_needed + + +def pe_export_names(data): + header, = struct.unpack_from("` | Set the system library search path. Quote `$ORIGIN` to prevent shell expansion, for example `--rpath '$ORIGIN/libs'` in Bash or PowerShell. Linux guest modules require an absolute path or a path beginning with `$ORIGIN`. Windows requires a nonempty ASCII path and supports `$ORIGIN` as the executable directory. | | `--autorun` | Run the output after conversion, print its exit code, and wait for Enter. Adds executable permissions for Linux output. Requires the target OS and prepared runtime layout. | | `--sce-module-path ` | Parent of the module directories, not the module directory itself. Default: input ELF parent; relative input paths resolve from the process working directory. Explicit absolute path: used directly. Explicit relative path (including `.` and `..`): relative to the process working directory. Output path has no effect. Requires one nonempty value; repeats, nonexistent or nondirectory paths, access failures, and combination with `--skip-sce-module` are errors. | +| `--module-dir ` | Relink the ELF modules of a directory under the module parent directory, besides `sce_module`/`sce_modules`/`prx`; outputs keep the relative path. A module the executable or another module needs is initialized at startup, any other when the title loads it with `sceKernelLoadStartModule`, with its start arguments (for example Unity plugins in `Media/Plugins`). Repeat for multiple directories. Conflicts with `--skip-sce-module`. | | `--skip-sce-module` | Deprecated. Disable bundled module processing and module directory checks. Conflicts with explicit `--sce-module-path`. | | `--exclude-sce-module ` | Deprecated. Exclude a bundled module by exact filename, not path. Repeat for multiple files; a missing filename is an error. Conflicts with `--skip-sce-module`. | | `--skip-syscall-check` | Deprecated. Disable syscall scanning in the executable and bundled modules. | From 00fba4472c6dda2bc599891a1f33910118d5a8d5 Mon Sep 17 00:00:00 2001 From: xXPBSXx <20614131+xXPBSXx@users.noreply.github.com> Date: Sun, 11 Oct 2026 01:02:08 +0200 Subject: [PATCH 434/460] fix(libSceSharePlay): export sceSharePlayGetCurrentConnectionInfoA Co-Authored-By: Claude Sonnet 5.5 --- core/libs/prx/libSceSharePlay/Export.cpp | 5 +++++ core/libs/tests/GuestSharePlay.cpp | 4 ++++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 10 insertions(+) diff --git a/core/libs/prx/libSceSharePlay/Export.cpp b/core/libs/prx/libSceSharePlay/Export.cpp index 7db889918..c2af9d13c 100644 --- a/core/libs/prx/libSceSharePlay/Export.cpp +++ b/core/libs/prx/libSceSharePlay/Export.cpp @@ -15,4 +15,9 @@ int APS5_VABI sceSharePlayTerminate(void) { return 0; } +int APS5_VABI sceSharePlayGetCurrentConnectionInfoA(void* info) { + if (info == nullptr) APS5_INVALID_ARG_EX; + return 0; +} + } diff --git a/core/libs/tests/GuestSharePlay.cpp b/core/libs/tests/GuestSharePlay.cpp index 5acffe557..cdc064419 100644 --- a/core/libs/tests/GuestSharePlay.cpp +++ b/core/libs/tests/GuestSharePlay.cpp @@ -6,6 +6,7 @@ extern "C" { int APS5_VABI sceSharePlayInitialize(void*, std::size_t); int APS5_VABI sceSharePlayTerminate(void); +int APS5_VABI sceSharePlayGetCurrentConnectionInfoA(void*); } static void Require(bool value) { if (!value) std::abort(); } @@ -28,4 +29,7 @@ int main() { Require(sceSharePlayTerminate() == 0); Require(Throws([&] { sceSharePlayInitialize(heap, 0); })); Require(Throws([&] { sceSharePlayInitialize(nullptr, 0); })); + unsigned char info[0x100] = {}; + Require(sceSharePlayGetCurrentConnectionInfoA(info) == 0); + Require(Throws([&] { sceSharePlayGetCurrentConnectionInfoA(nullptr); })); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 542edd980..aafe0531c 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -552,3 +552,4 @@ Throughout the project, every function at every stage either **does exactly what - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert +- [sceSharePlayGetCurrentConnectionInfoA](../../core/libs/prx/libSceSharePlay/Export.cpp) (libSceSharePlay) - succeeds without writing the connection info, because the library keeps no Share Play session. A null pointer throws; the real error code and the layout of the info are unknown. From 0d2eeb87ebe0aa8e8444496b694e2a91b5498d82 Mon Sep 17 00:00:00 2001 From: Abdulrahman Elkordy <196347165+Silverash-i8@users.noreply.github.com> Date: Sun, 11 Oct 2026 04:10:22 +0300 Subject: [PATCH 435/460] fix(libSceSharePlay): fill the connection info with no Share Play session sceSharePlayGetCurrentConnectionInfoA now writes a 72-byte SceSharePlayConnectionInfoA that reports no session: status DORMANT, mode 0, empty online IDs, account IDs 0 and both user IDs invalid (-1). A null pointer returns SCE_SHARE_PLAY_ERROR_INVALID_ARGS (0x810E0001) instead of throwing. KytyPS5 and fpPS4 agree on the layout and the error code; KytyPS5 sets the user IDs to -1, fpPS4 leaves them 0. Claude-Session: https://claude.ai/code/session_012SLQbhAiexxW6TtjYToW8a --- core/libs/SceTypes.hpp | 12 ++++++++++++ core/libs/prx/libSceSharePlay/Export.cpp | 10 ++++++++-- core/libs/tests/GuestSharePlay.cpp | 14 ++++++++++---- docs/dev/TechnicalDebt.md | 2 +- 4 files changed, 31 insertions(+), 7 deletions(-) diff --git a/core/libs/SceTypes.hpp b/core/libs/SceTypes.hpp index 99211616a..0ad23c379 100644 --- a/core/libs/SceTypes.hpp +++ b/core/libs/SceTypes.hpp @@ -1278,6 +1278,18 @@ struct NpTitleId { char data[13]; char pad[3]; }; struct NpTitleSecret { std::uint8_t data[128]; }; struct NpContentRestriction { std::uint8_t opaque[128]; }; struct NpOnlineId { char data[17]; char pad[3]; }; + +struct SharePlayConnectionInfoA { + std::int32_t status; + std::int32_t mode; + NpOnlineId hostOnlineId; + NpOnlineId visitorOnlineId; + std::uint64_t hostAccountId; + std::uint64_t visitorAccountId; + std::int32_t hostUserId; + std::int32_t visitorUserId; +}; +static_assert(sizeof(SharePlayConnectionInfoA) == 72 && offsetof(SharePlayConnectionInfoA, hostAccountId) == 48 && offsetof(SharePlayConnectionInfoA, hostUserId) == 64); struct NpId { NpOnlineId online_id; std::uint8_t opaque[4]; }; struct NpCreateAsyncRequestParameter { std::uint8_t opaque[64]; }; struct NpCheckPremiumParameter { std::uint8_t opaque[64]; }; diff --git a/core/libs/prx/libSceSharePlay/Export.cpp b/core/libs/prx/libSceSharePlay/Export.cpp index c2af9d13c..6b0387b94 100644 --- a/core/libs/prx/libSceSharePlay/Export.cpp +++ b/core/libs/prx/libSceSharePlay/Export.cpp @@ -3,6 +3,9 @@ #include "SceTypes.hpp" #include "prx/libc/include/General.hpp" +static constexpr int SCE_SHARE_PLAY_ERROR_INVALID_ARGS = static_cast(0x810E0001); +static constexpr std::int32_t SCE_USER_SERVICE_USER_ID_INVALID = -1; + extern "C" { int APS5_VABI sceSharePlayInitialize(void* heap, size_t heapSize) { @@ -15,8 +18,11 @@ int APS5_VABI sceSharePlayTerminate(void) { return 0; } -int APS5_VABI sceSharePlayGetCurrentConnectionInfoA(void* info) { - if (info == nullptr) APS5_INVALID_ARG_EX; +int APS5_VABI sceSharePlayGetCurrentConnectionInfoA(SharePlayConnectionInfoA* info) { + if (info == nullptr) return SCE_SHARE_PLAY_ERROR_INVALID_ARGS; + *info = {}; + info->hostUserId = SCE_USER_SERVICE_USER_ID_INVALID; + info->visitorUserId = SCE_USER_SERVICE_USER_ID_INVALID; return 0; } diff --git a/core/libs/tests/GuestSharePlay.cpp b/core/libs/tests/GuestSharePlay.cpp index cdc064419..ba62ad7da 100644 --- a/core/libs/tests/GuestSharePlay.cpp +++ b/core/libs/tests/GuestSharePlay.cpp @@ -1,12 +1,14 @@ +#include "SceTypes.hpp" #include "prx/libc/include/general/VabiMacros.hpp" #include #include +#include #include extern "C" { int APS5_VABI sceSharePlayInitialize(void*, std::size_t); int APS5_VABI sceSharePlayTerminate(void); -int APS5_VABI sceSharePlayGetCurrentConnectionInfoA(void*); +int APS5_VABI sceSharePlayGetCurrentConnectionInfoA(SharePlayConnectionInfoA*); } static void Require(bool value) { if (!value) std::abort(); } @@ -29,7 +31,11 @@ int main() { Require(sceSharePlayTerminate() == 0); Require(Throws([&] { sceSharePlayInitialize(heap, 0); })); Require(Throws([&] { sceSharePlayInitialize(nullptr, 0); })); - unsigned char info[0x100] = {}; - Require(sceSharePlayGetCurrentConnectionInfoA(info) == 0); - Require(Throws([&] { sceSharePlayGetCurrentConnectionInfoA(nullptr); })); + SharePlayConnectionInfoA info; + std::memset(&info, 0x5a, sizeof(info)); + Require(sceSharePlayGetCurrentConnectionInfoA(&info) == 0); + Require(info.status == 0 && info.mode == 0 && info.hostAccountId == 0 && info.visitorAccountId == 0); + Require(info.hostOnlineId.data[0] == 0 && info.visitorOnlineId.data[0] == 0 && info.hostOnlineId.pad[2] == 0); + Require(info.hostUserId == -1 && info.visitorUserId == -1); + Require(sceSharePlayGetCurrentConnectionInfoA(nullptr) == static_cast(0x810E0001)); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index aafe0531c..3dfb8c913 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -552,4 +552,4 @@ Throughout the project, every function at every stage either **does exactly what - [Null vertex buffer V#](../../core/libs/prx/libSceAgcDriver/Graphics/include/VertexInput.hpp) (libSceAgcDriver) - a V# whose four dwords are all zero is bound to a zeroed vertex buffer of the attribute's size with stride 0, so every component of its attribute reads 0. This follows the V# fields in the [AMD RDNA2 ISA](https://gpuopen.com/download/rdna2-shader-instruction-set-architecture.pdf) (`NUM_RECORDS` 0, `DST_SEL` 0 selects 0), not measured on hardware. - [_fstatfs](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) fills the FreeBSD 11 `struct statfs` of the PS5 payload SDK with the host volume that holds the file: block size, transfer size, total, free and available blocks, maximum name length and the read-only, `nosuid` and `noexec` flags (on Windows only read-only, and `f_files` and `f_ffree` are 0, as NTFS has no fixed number of files). The filesystem type number and name, mount paths, owner, filesystem id and I/O counters are left 0 or empty, since they would describe a host mount, not a console one. Sockets and pipes return `EINVAL`, as FreeBSD's `getvnode` does; a Windows descriptor that is neither a file nor a pipe throws. Stremio Plus (PPSA74126) reads the free space before a download and only logs the other fields - [_WStoul](../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../core/libs/prx/libc/src/RuneLocale.cpp) and [swscanf_s](../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert -- [sceSharePlayGetCurrentConnectionInfoA](../../core/libs/prx/libSceSharePlay/Export.cpp) (libSceSharePlay) - succeeds without writing the connection info, because the library keeps no Share Play session. A null pointer throws; the real error code and the layout of the info are unknown. +- [sceSharePlayGetCurrentConnectionInfoA](../../core/libs/prx/libSceSharePlay/Export.cpp) (libSceSharePlay) - reports no Share Play session, as the library keeps none: status 0 (DORMANT), mode 0, empty online IDs, account IDs 0 and both user IDs -1 (`SCE_USER_SERVICE_USER_ID_INVALID`). A null pointer returns `SCE_SHARE_PLAY_ERROR_INVALID_ARGS` (0x810E0001). The 72-byte `SceSharePlayConnectionInfoA` layout and the error code follow [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/main/src/libs/libNet.cpp) and [fpPS4](https://github.com/red-prig/fpPS4/blob/trunk/src/ps4_libsceshareplay.pas), which agree on both; KytyPS5 sets the user IDs to -1 and fpPS4 leaves them 0, and which one the console returns is not verified. KytyPS5 also returns `SCE_SHARE_PLAY_ERROR_NOT_INITIALIZED` before `sceSharePlayInitialize`; this library keeps no initialization state and does not. From f3216874c586e455aa5017df0b36e938f8660d13 Mon Sep 17 00:00:00 2001 From: GalaxyGamingFR <153150716+GalaxyGamingFR@users.noreply.github.com> Date: Sat, 10 Oct 2026 21:21:04 -0700 Subject: [PATCH 436/460] fix(progress): ignore braces in literals and comments when scanning bodies Co-Authored-By: Claude Fable 5.1 --- docs/dev/PROGRESS.md | 2 +- tools/progress.py | 12 +++++++++++- tools/tests/test_progress.py | 17 +++++++++++++++++ 3 files changed, 29 insertions(+), 2 deletions(-) diff --git a/docs/dev/PROGRESS.md b/docs/dev/PROGRESS.md index 03517e712..9daf9fc04 100644 --- a/docs/dev/PROGRESS.md +++ b/docs/dev/PROGRESS.md @@ -4,7 +4,7 @@ ## Library counts -The scanner reads `APS5_VABI` function definitions under `core/libs/prx//`, excluding `tests` subdirectories and their descendants. Test callbacks do not contribute to library counts. A definition is pending when its body calls `NotImplemented_nid_no_patch`, directly or through a recognized local stub wrapper. Other definitions are counted as implemented. These counts describe declared functions, not every firmware export, verified ABI compatibility or playable games. Silent behavior and unverified assumptions remain in [Technical debt](TechnicalDebt.md). +The scanner reads `APS5_VABI` function definitions under `core/libs/prx//`, excluding `tests` subdirectories and their descendants. Test callbacks do not contribute to library counts. A definition is pending when its body calls `NotImplemented_nid_no_patch`, directly or through a recognized local stub wrapper. Other definitions are counted as implemented. Braces inside string literals, character literals and comments do not delimit bodies. These counts describe declared functions, not every firmware export, verified ABI compatibility or playable games. Silent behavior and unverified assumptions remain in [Technical debt](TechnicalDebt.md). A library with no scanned definitions can include a sibling library's source builder. The scanner recognizes literal `include(${CMAKE_CURRENT_SOURCE_DIR}/..//.cmake)` references, including quoted paths. It does not evaluate arbitrary CMake expressions. It also recognizes a top-level `.cpp` that includes another library's source with `#include "prx//.cpp"`. diff --git a/tools/progress.py b/tools/progress.py index 1b32f8db1..33c27cf60 100644 --- a/tools/progress.py +++ b/tools/progress.py @@ -16,6 +16,8 @@ STUB = "NotImplemented_nid_no_patch" CMAKE_SHARED = re.compile(r'^\s*include\(\s*"?\$\{CMAKE_CURRENT_SOURCE_DIR\}/\.\./([\w.-]+)/[\w.-]+\.cmake"?\s*\)', re.M) SOURCE_SHARED = re.compile(r'^\s*#\s*include\s*"prx/([\w.-]+)/[\w./-]+\.cpp"', re.M) STUB_WRAPPER = re.compile(r"\bstatic\s+(?:\[\[noreturn\]\]\s+)?[\w:<>,\s*&]+?\s+(\w+)\s*\([^;{]*\)\s*\{") +LITERAL = re.compile(r'"(?:\\.|[^"\\\n])*"|\'(?:\\.|[^\'\\\n])*\'') +COMMENT = re.compile(r"//[^\n]*|/\*.*?\*/", re.S) FLAT_SEGMENTS = ("GLOBAL_", "SCRATCH_") OPCODE_SENTINELS = {"Invalid", "Count", "Unknown", "Unsupported"} OPCODE_ALIASES = { @@ -62,6 +64,14 @@ PANEL_WIDTH, GAP, MAP_HEIGHT, HEADER = 495, 10, 280, 30 DONE_COLOR, TODO_COLOR, BORDER, TEXT = "#2ea043", "#6e7681", "#0d1117", "#ffffff" +def blank(match): + return re.sub(r"[^\n]", " ", match.group(0)) + + +def code(text): + return COMMENT.sub(blank, LITERAL.sub(blank, text)) + + def body_end(text, start): depth = 0 for i in range(start, len(text)): @@ -87,7 +97,7 @@ def scan_library(path): for source in path.rglob("*.cpp"): if "tests" in source.relative_to(path).parts: continue - text = source.read_text(errors="ignore") + text = code(source.read_text(errors="ignore")) calls = stub_calls(text) for match in DEFINITION.finditer(text): name = match.group(1) diff --git a/tools/tests/test_progress.py b/tools/tests/test_progress.py index dfff23f69..d927b88f4 100644 --- a/tools/tests/test_progress.py +++ b/tools/tests/test_progress.py @@ -20,6 +20,23 @@ class ProgressTests(unittest.TestCase): self.assertEqual(group["done_names"], []) self.assertEqual(group["todo_names"], ["sceExampleSupportFormat"]) + def test_braces_in_literals_and_comments_do_not_delimit_bodies(self): + with tempfile.TemporaryDirectory() as directory: + library = Path(directory) / "libSceExample" + library.mkdir() + (library / "Export.cpp").write_text( + "static int Helper() { const char* brace = \"}\"; NotImplemented_nid_no_patch(__func__); return 0; }\n" + "int APS5_VABI sceExampleOpen(char* out) { *out = '{'; return 0; }\n" + "int APS5_VABI sceExampleClose(char* out) { *out = '}'; NotImplemented_nid_no_patch(__func__); return 0; }\n" + "int APS5_VABI sceExampleQuote(std::string& out) { out += \"{\\\"\"; return 0; }\n" + "int APS5_VABI sceExampleNoted() { // }\n NotImplemented_nid_no_patch(__func__); return 0; }\n" + "int APS5_VABI sceExampleBlock() { /* { */ return 0; }\n" + "int APS5_VABI sceExampleWrapped() { return Helper(); }\n" + "int APS5_VABI sceExampleReady() { return 0; }\n") + group = progress.scan_library(library) + self.assertEqual(group["done_names"], ["sceExampleBlock", "sceExampleOpen", "sceExampleQuote", "sceExampleReady"]) + self.assertEqual(group["todo_names"], ["sceExampleClose", "sceExampleNoted", "sceExampleWrapped"]) + def test_test_sources_do_not_change_library_progress(self): with tempfile.TemporaryDirectory() as directory: root = Path(directory) / "tests" / "prx" From ef94a36b90f602c20a54a133bd4a33ed6f98c516 Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Sun, 11 Oct 2026 17:17:40 +0900 Subject: [PATCH 437/460] fix(progress): blank literals and comments in one pass An apostrophe in a block comment paired with a later character literal on the same line when literals were blanked before comments, exposing a brace and ending the body early. Co-Authored-By: Claude Opus 5.5 --- tools/progress.py | 5 ++--- tools/tests/test_progress.py | 12 ++++++++++++ 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/tools/progress.py b/tools/progress.py index 33c27cf60..c052dab1b 100644 --- a/tools/progress.py +++ b/tools/progress.py @@ -16,8 +16,7 @@ STUB = "NotImplemented_nid_no_patch" CMAKE_SHARED = re.compile(r'^\s*include\(\s*"?\$\{CMAKE_CURRENT_SOURCE_DIR\}/\.\./([\w.-]+)/[\w.-]+\.cmake"?\s*\)', re.M) SOURCE_SHARED = re.compile(r'^\s*#\s*include\s*"prx/([\w.-]+)/[\w./-]+\.cpp"', re.M) STUB_WRAPPER = re.compile(r"\bstatic\s+(?:\[\[noreturn\]\]\s+)?[\w:<>,\s*&]+?\s+(\w+)\s*\([^;{]*\)\s*\{") -LITERAL = re.compile(r'"(?:\\.|[^"\\\n])*"|\'(?:\\.|[^\'\\\n])*\'') -COMMENT = re.compile(r"//[^\n]*|/\*.*?\*/", re.S) +NON_CODE = re.compile(r'"(?:\\.|[^"\\\n])*"|\'(?:\\.|[^\'\\\n])*\'|//[^\n]*|/\*.*?\*/', re.S) FLAT_SEGMENTS = ("GLOBAL_", "SCRATCH_") OPCODE_SENTINELS = {"Invalid", "Count", "Unknown", "Unsupported"} OPCODE_ALIASES = { @@ -69,7 +68,7 @@ def blank(match): def code(text): - return COMMENT.sub(blank, LITERAL.sub(blank, text)) + return NON_CODE.sub(blank, text) def body_end(text, start): diff --git a/tools/tests/test_progress.py b/tools/tests/test_progress.py index d927b88f4..15f88dd2c 100644 --- a/tools/tests/test_progress.py +++ b/tools/tests/test_progress.py @@ -37,6 +37,18 @@ class ProgressTests(unittest.TestCase): self.assertEqual(group["done_names"], ["sceExampleBlock", "sceExampleOpen", "sceExampleQuote", "sceExampleReady"]) self.assertEqual(group["todo_names"], ["sceExampleClose", "sceExampleNoted", "sceExampleWrapped"]) + def test_apostrophe_in_a_comment_does_not_pair_with_a_later_literal(self): + with tempfile.TemporaryDirectory() as directory: + library = Path(directory) / "libSceExample" + library.mkdir() + (library / "Export.cpp").write_text( + "int APS5_VABI sceExamplePending() {\n /* it's */ log('}');\n" + " NotImplemented_nid_no_patch(__func__);\n return 0;\n}\n" + "int APS5_VABI sceExampleReady() { return 0; }\n") + group = progress.scan_library(library) + self.assertEqual(group["done_names"], ["sceExampleReady"]) + self.assertEqual(group["todo_names"], ["sceExamplePending"]) + def test_test_sources_do_not_change_library_progress(self): with tempfile.TemporaryDirectory() as directory: root = Path(directory) / "tests" / "prx" From 96ad950914702b384614dce6b21a63428c93cb7a Mon Sep 17 00:00:00 2001 From: "husari.orozco" Date: Tue, 6 Oct 2026 11:13:31 -0600 Subject: [PATCH 438/460] build: add ANYPS5_USE_SYSTEM_DEPS option Resolve SDL2, freetype, Vulkan-Headers, SPIRV-Headers, glslang and SPIRV-Tools from the system instead of the bundled submodules when the option is ON. It is OFF by default, so the bundled build is unchanged. Every package is required: a missing one, or Vulkan-Headers older than 1.4.310 (the agc driver uses VK_KHR_maintenance8), fails the configure with an error and nothing falls back to the submodule. The system packages are exposed under the same target names the bundled path uses, so the rest of the CMake is untouched. SDL2 is linked from the system as a shared library because the prx libraries are shared objects and distribution static SDL2 is usually not PIC. ffmpeg-core, libjpeg-turbo, stb, VulkanMemoryAllocator and LibAtrac9 stay bundled. --- CMakeLists.txt | 26 ++++--- cmake/SystemDeps.cmake | 101 ++++++++++++++++++++++++++ core/shader/recompiler/CMakeLists.txt | 30 ++++---- docs/dev/BUILD.md | 3 + 4 files changed, 136 insertions(+), 24 deletions(-) create mode 100644 cmake/SystemDeps.cmake diff --git a/CMakeLists.txt b/CMakeLists.txt index ff5546acc..d7757486b 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -25,7 +25,9 @@ endif() option(ANYPS5_ENABLE_SPIRV_TOOLS "Enable SPIRV-Tools validation and optimization" OFF) option(APS5_ENABLE_TIMING_LOG "Write frame timings asynchronously to frame-timing.log" OFF) +option(ANYPS5_USE_SYSTEM_DEPS "Resolve SDL2, freetype, Vulkan-Headers, SPIRV-Headers, glslang and SPIRV-Tools from the system instead of the bundled submodules" OFF) add_compile_definitions(APS5_ENABLE_TIMING_LOG=$) +include(cmake/SystemDeps.cmake) set(CMAKE_CXX_STANDARD 20) set(CMAKE_CXX_STANDARD_REQUIRED ON) @@ -82,10 +84,12 @@ set(SDL_HAPTIC OFF CACHE BOOL "" FORCE) set(SDL_HIDAPI ON CACHE BOOL "" FORCE) set(SDL_SENSOR OFF CACHE BOOL "" FORCE) set(SDL_STATIC_PIC ON CACHE BOOL "" FORCE) -set(ANYPS5_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS}") -set(CMAKE_EXE_LINKER_FLAGS "") -add_subdirectory(3rdparty/SDL2) -set(CMAKE_EXE_LINKER_FLAGS "${ANYPS5_EXE_LINKER_FLAGS}") +if(NOT ANYPS5_USE_SYSTEM_DEPS) + set(ANYPS5_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS}") + set(CMAKE_EXE_LINKER_FLAGS "") + add_subdirectory(3rdparty/SDL2) + set(CMAKE_EXE_LINKER_FLAGS "${ANYPS5_EXE_LINKER_FLAGS}") +endif() set(ARCHITECTURE x86_64) add_subdirectory(3rdparty/ffmpeg-core) if(MINGW) @@ -100,12 +104,14 @@ set(FT_DISABLE_PNG ON CACHE BOOL "" FORCE) set(FT_DISABLE_HARFBUZZ ON CACHE BOOL "" FORCE) set(FT_DISABLE_BROTLI ON CACHE BOOL "" FORCE) set(SKIP_INSTALL_ALL ON CACHE BOOL "" FORCE) -add_subdirectory(3rdparty/freetype EXCLUDE_FROM_ALL) -set_target_properties(freetype PROPERTIES POSITION_INDEPENDENT_CODE ON) -if(WIN32) - get_target_property(freetypeSources freetype SOURCES) - list(FILTER freetypeSources EXCLUDE REGEX "ftver\\.rc$") - set_target_properties(freetype PROPERTIES SOURCES "${freetypeSources}") +if(NOT ANYPS5_USE_SYSTEM_DEPS) + add_subdirectory(3rdparty/freetype EXCLUDE_FROM_ALL) + set_target_properties(freetype PROPERTIES POSITION_INDEPENDENT_CODE ON) + if(WIN32) + get_target_property(freetypeSources freetype SOURCES) + list(FILTER freetypeSources EXCLUDE REGEX "ftver\\.rc$") + set_target_properties(freetype PROPERTIES SOURCES "${freetypeSources}") + endif() endif() add_subdirectory(core/shader/recompiler) diff --git a/cmake/SystemDeps.cmake b/cmake/SystemDeps.cmake new file mode 100644 index 000000000..7b5923ace --- /dev/null +++ b/cmake/SystemDeps.cmake @@ -0,0 +1,101 @@ +# Resolves the third-party libraries from the system instead of the bundled +# submodules when ANYPS5_USE_SYSTEM_DEPS is ON. Every package is required: a +# missing one fails the configure and nothing falls back to the submodule. +# +# The system packages are exposed under the same target names the bundled +# submodules use (SDL2-static, freetype, glslang, SPIRV, glslang-standalone, +# SPIRV-Tools-opt, SPIRV-Tools-static) so the rest of the build does not need +# to know which path was taken. + +if(NOT ANYPS5_USE_SYSTEM_DEPS) + return() +endif() + +find_package(SDL2 CONFIG REQUIRED) +# The prx libraries are shared objects, so they need a position-independent +# SDL2. Distribution static builds are usually not PIC, so link the shared one. +if(NOT TARGET SDL2::SDL2) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: SDL2 was found but does not provide SDL2::SDL2") +endif() +add_library(SDL2-static INTERFACE) +target_link_libraries(SDL2-static INTERFACE SDL2::SDL2) + +find_package(Freetype REQUIRED) +if(NOT TARGET Freetype::Freetype) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: freetype was found but does not provide Freetype::Freetype") +endif() +add_library(freetype INTERFACE) +target_link_libraries(freetype INTERFACE Freetype::Freetype) + +find_package(VulkanHeaders CONFIG REQUIRED) +if(NOT TARGET Vulkan::Headers) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: Vulkan-Headers was found but does not provide Vulkan::Headers") +endif() +get_target_property(ANYPS5_VULKAN_INCLUDE_DIRS Vulkan::Headers INTERFACE_INCLUDE_DIRECTORIES) +include(CheckCXXSourceCompiles) +set(CMAKE_REQUIRED_INCLUDES ${ANYPS5_VULKAN_INCLUDE_DIRS}) +check_cxx_source_compiles("#include \nint main() { VkPhysicalDeviceMaintenance8FeaturesKHR features{}; (void)features; return 0; }" ANYPS5_VULKAN_HAS_MAINTENANCE8) +unset(CMAKE_REQUIRED_INCLUDES) +if(NOT ANYPS5_VULKAN_HAS_MAINTENANCE8) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: the system Vulkan-Headers do not provide VkPhysicalDeviceMaintenance8FeaturesKHR (VK_KHR_maintenance8), which the agc driver uses; install Vulkan-Headers 1.4.310 or newer") +endif() + +find_package(SPIRV-Headers CONFIG REQUIRED) +if(NOT TARGET SPIRV-Headers::SPIRV-Headers) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: SPIRV-Headers was found but does not provide SPIRV-Headers::SPIRV-Headers") +endif() + +find_package(glslang CONFIG REQUIRED) +if(NOT TARGET glslang::glslang OR NOT TARGET glslang::SPIRV) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: glslang was found but does not provide glslang::glslang and glslang::SPIRV") +endif() +add_library(glslang INTERFACE) +target_link_libraries(glslang INTERFACE glslang::glslang) +add_library(SPIRV INTERFACE) +target_link_libraries(SPIRV INTERFACE glslang::SPIRV) + +# The bundled glslang has both and under one root; the +# system install keeps under /include and the rest, +# including , under /include/glslang. +find_path(ANYPS5_GLSLANG_PUBLIC_INCLUDE_DIR NAMES glslang/Public/ShaderLang.h) +if(NOT ANYPS5_GLSLANG_PUBLIC_INCLUDE_DIR) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: glslang/Public/ShaderLang.h was not found; the glslang headers are incomplete") +endif() +set(ANYPS5_GLSLANG_SPIRV_INCLUDE_DIR "${ANYPS5_GLSLANG_PUBLIC_INCLUDE_DIR}/glslang") +if(NOT EXISTS "${ANYPS5_GLSLANG_SPIRV_INCLUDE_DIR}/SPIRV/GlslangToSpv.h") + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: ${ANYPS5_GLSLANG_SPIRV_INCLUDE_DIR}/SPIRV/GlslangToSpv.h was not found") +endif() + +find_program(ANYPS5_GLSLANG_STANDALONE_EXECUTABLE NAMES glslang glslangValidator) +if(NOT ANYPS5_GLSLANG_STANDALONE_EXECUTABLE) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: the glslang compiler (glslang or glslangValidator) was not found; install the glslang tools package") +endif() +add_executable(glslang-standalone IMPORTED GLOBAL) +set_target_properties(glslang-standalone PROPERTIES IMPORTED_LOCATION "${ANYPS5_GLSLANG_STANDALONE_EXECUTABLE}") + +if(ANYPS5_ENABLE_SPIRV_TOOLS) + find_package(SPIRV-Tools-opt CONFIG REQUIRED) + if(NOT TARGET SPIRV-Tools-opt OR NOT TARGET SPIRV-Tools-static) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: SPIRV-Tools was found but does not provide SPIRV-Tools-opt and SPIRV-Tools-static") + endif() +endif() + +set(_anyps5_system_include_targets + SDL2::SDL2 + Freetype::Freetype + Vulkan::Headers + SPIRV-Headers::SPIRV-Headers + glslang::glslang + glslang::SPIRV +) +set(_anyps5_system_includes "") +foreach(_target IN LISTS _anyps5_system_include_targets) + if(TARGET ${_target}) + get_target_property(_target_includes ${_target} INTERFACE_INCLUDE_DIRECTORIES) + if(_target_includes) + list(APPEND _anyps5_system_includes ${_target_includes}) + endif() + endif() +endforeach() +list(APPEND _anyps5_system_includes ${ANYPS5_GLSLANG_SPIRV_INCLUDE_DIR} ${ANYPS5_GLSLANG_PUBLIC_INCLUDE_DIR}) +include_directories(SYSTEM ${_anyps5_system_includes}) diff --git a/core/shader/recompiler/CMakeLists.txt b/core/shader/recompiler/CMakeLists.txt index 727e2b4d3..d25cf641a 100644 --- a/core/shader/recompiler/CMakeLists.txt +++ b/core/shader/recompiler/CMakeLists.txt @@ -186,24 +186,26 @@ function(add_shader_recompiler target runtimeTarget) list(REMOVE_ITEM shaderCacheGuardSources ${shaderCacheRecompilerSources}) find_package(Git QUIET) - set(shaderCacheSubmodules "${PROJECT_SOURCE_DIR}/3rdparty/SPIRV-Headers" "${PROJECT_SOURCE_DIR}/3rdparty/glslang") - if(ANYPS5_ENABLE_SPIRV_TOOLS) - list(APPEND shaderCacheSubmodules "${PROJECT_SOURCE_DIR}/3rdparty/SPIRV-Tools") - endif() set(shaderCacheSubmoduleStamps) - foreach(submodule IN LISTS shaderCacheSubmodules) - list(APPEND shaderCacheSubmoduleStamps "${submodule}/CMakeLists.txt") - if(NOT GIT_FOUND) - continue() + if(NOT ANYPS5_USE_SYSTEM_DEPS) + set(shaderCacheSubmodules "${PROJECT_SOURCE_DIR}/3rdparty/SPIRV-Headers" "${PROJECT_SOURCE_DIR}/3rdparty/glslang") + if(ANYPS5_ENABLE_SPIRV_TOOLS) + list(APPEND shaderCacheSubmodules "${PROJECT_SOURCE_DIR}/3rdparty/SPIRV-Tools") endif() - foreach(ref IN ITEMS HEAD refs/heads) - execute_process(COMMAND "${GIT_EXECUTABLE}" -C "${submodule}" rev-parse --path-format=absolute --git-path ${ref} - OUTPUT_VARIABLE stamp OUTPUT_STRIP_TRAILING_WHITESPACE RESULT_VARIABLE status ERROR_QUIET) - if(status EQUAL 0 AND EXISTS "${stamp}" AND NOT IS_DIRECTORY "${stamp}") - list(APPEND shaderCacheSubmoduleStamps "${stamp}") + foreach(submodule IN LISTS shaderCacheSubmodules) + list(APPEND shaderCacheSubmoduleStamps "${submodule}/CMakeLists.txt") + if(NOT GIT_FOUND) + continue() endif() + foreach(ref IN ITEMS HEAD refs/heads) + execute_process(COMMAND "${GIT_EXECUTABLE}" -C "${submodule}" rev-parse --path-format=absolute --git-path ${ref} + OUTPUT_VARIABLE stamp OUTPUT_STRIP_TRAILING_WHITESPACE RESULT_VARIABLE status ERROR_QUIET) + if(status EQUAL 0 AND EXISTS "${stamp}" AND NOT IS_DIRECTORY "${stamp}") + list(APPEND shaderCacheSubmoduleStamps "${stamp}") + endif() + endforeach() endforeach() - endforeach() + endif() get_target_property(shaderCacheIncludeDirectories ${target} INCLUDE_DIRECTORIES) string(TOUPPER "${CMAKE_BUILD_TYPE}" shaderCacheBuildType) diff --git a/docs/dev/BUILD.md b/docs/dev/BUILD.md index 38264ca7c..1c30aa947 100644 --- a/docs/dev/BUILD.md +++ b/docs/dev/BUILD.md @@ -57,6 +57,7 @@ Project switches accept `ON` or `OFF`: | `-DBUILD_TESTING=ON` | `OFF` | Build and register tests. | | `-DANYPS5_RELINKER_ONLY=ON` | `OFF` | Build only the relinker and its tests, without third-party dependencies. | | `-DANYPS5_ENABLE_SPIRV_TOOLS=ON` | `OFF` | Enable SPIR-V validation and optimization. | +| `-DANYPS5_USE_SYSTEM_DEPS=ON` | `OFF` | Resolve SDL2, freetype, Vulkan-Headers, SPIRV-Headers, glslang and SPIRV-Tools from the system instead of the submodules. | | `-DAPS5_ENABLE_TIMING_LOG=ON` | `OFF` | Compile frame timing logging. | | `-DAPS5_AGC_CREATE_LOG=OFF` | `ON` | Disable successful `sceAgcCreateShader` logging. | | `-DAGC_BUILD_VISUAL_TEST=ON` | `OFF` | Build the standalone AGC SPIR-V visual test. | @@ -74,6 +75,8 @@ Build configuration parameters: SDL and FreeType settings forced by the root `CMakeLists.txt` cannot be overridden with `-D`. +With `ANYPS5_USE_SYSTEM_DEPS=ON` those libraries come from the system (on Debian and Ubuntu: `libsdl2-dev`, `libfreetype-dev`, `libvulkan-dev`, `spirv-headers`, `glslang-dev`, `glslang-tools` and `spirv-tools`). A missing package, or Vulkan-Headers older than 1.4.310 (the agc driver uses `VK_KHR_maintenance8`), fails the configure with an error and never falls back to the submodule. The prx libraries are shared objects, so SDL2 is linked from the system as a shared library; the default build keeps its own static SDL2. `ffmpeg-core`, `libjpeg-turbo`, `stb`, `VulkanMemoryAllocator` and `LibAtrac9` always stay bundled and still need `git submodule update --init`. + ## Write watch tests The macOS write watch can be built and tested separately, without submodules or the prx libraries: From 6ad5c61f1a32ab0438348c57e166794417c59985 Mon Sep 17 00:00:00 2001 From: "husari.orozco" Date: Wed, 7 Oct 2026 09:09:40 -0600 Subject: [PATCH 439/460] build: route system dependency include paths and pin the shader cache to package versions --- cmake/SystemDeps.cmake | 63 ++++++++++---------- core/libs/prx/libSceAgc/CMakeLists.txt | 2 +- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 42 ++++++------- core/libs/prx/libSceAudioIn/CMakeLists.txt | 2 +- core/libs/prx/libSceAudioOut/CMakeLists.txt | 2 +- core/libs/prx/libSceKeyboard/CMakeLists.txt | 2 +- core/libs/prx/libSceMouse/CMakeLists.txt | 2 +- core/libs/prx/libScePad/CMakeLists.txt | 2 +- core/libs/prx/libSceVideoOut/CMakeLists.txt | 6 +- core/shader/recompiler/CMakeLists.txt | 11 ++-- 10 files changed, 67 insertions(+), 67 deletions(-) diff --git a/cmake/SystemDeps.cmake b/cmake/SystemDeps.cmake index 7b5923ace..e08c7b504 100644 --- a/cmake/SystemDeps.cmake +++ b/cmake/SystemDeps.cmake @@ -1,24 +1,24 @@ -# Resolves the third-party libraries from the system instead of the bundled -# submodules when ANYPS5_USE_SYSTEM_DEPS is ON. Every package is required: a -# missing one fails the configure and nothing falls back to the submodule. -# -# The system packages are exposed under the same target names the bundled -# submodules use (SDL2-static, freetype, glslang, SPIRV, glslang-standalone, -# SPIRV-Tools-opt, SPIRV-Tools-static) so the rest of the build does not need -# to know which path was taken. +set(ANYPS5_VULKAN_HEADERS_INCLUDE_DIR "${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include") +set(ANYPS5_SPIRV_HEADERS_INCLUDE_DIR "${CMAKE_SOURCE_DIR}/3rdparty/SPIRV-Headers/include") +set(ANYPS5_SDL2_INCLUDE_DIR "${CMAKE_SOURCE_DIR}/3rdparty/SDL2/include") +set(ANYPS5_GLSLANG_INCLUDE_DIRS "${CMAKE_SOURCE_DIR}/3rdparty/glslang") +set(ANYPS5_SPIRV_HEADERS_SOURCE_DIR "${CMAKE_SOURCE_DIR}/3rdparty/SPIRV-Headers") +set(ANYPS5_GLSLANG_SOURCE_DIR "${CMAKE_SOURCE_DIR}/3rdparty/glslang") if(NOT ANYPS5_USE_SYSTEM_DEPS) return() endif() find_package(SDL2 CONFIG REQUIRED) -# The prx libraries are shared objects, so they need a position-independent -# SDL2. Distribution static builds are usually not PIC, so link the shared one. if(NOT TARGET SDL2::SDL2) message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: SDL2 was found but does not provide SDL2::SDL2") endif() add_library(SDL2-static INTERFACE) target_link_libraries(SDL2-static INTERFACE SDL2::SDL2) +get_target_property(ANYPS5_SDL2_INCLUDE_DIR SDL2::SDL2 INTERFACE_INCLUDE_DIRECTORIES) +if(NOT ANYPS5_SDL2_INCLUDE_DIR) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: SDL2::SDL2 exposes no INTERFACE_INCLUDE_DIRECTORIES") +endif() find_package(Freetype REQUIRED) if(NOT TARGET Freetype::Freetype) @@ -31,9 +31,12 @@ find_package(VulkanHeaders CONFIG REQUIRED) if(NOT TARGET Vulkan::Headers) message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: Vulkan-Headers was found but does not provide Vulkan::Headers") endif() -get_target_property(ANYPS5_VULKAN_INCLUDE_DIRS Vulkan::Headers INTERFACE_INCLUDE_DIRECTORIES) +get_target_property(ANYPS5_VULKAN_HEADERS_INCLUDE_DIR Vulkan::Headers INTERFACE_INCLUDE_DIRECTORIES) +if(NOT ANYPS5_VULKAN_HEADERS_INCLUDE_DIR) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: Vulkan::Headers exposes no INTERFACE_INCLUDE_DIRECTORIES") +endif() include(CheckCXXSourceCompiles) -set(CMAKE_REQUIRED_INCLUDES ${ANYPS5_VULKAN_INCLUDE_DIRS}) +set(CMAKE_REQUIRED_INCLUDES ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) check_cxx_source_compiles("#include \nint main() { VkPhysicalDeviceMaintenance8FeaturesKHR features{}; (void)features; return 0; }" ANYPS5_VULKAN_HAS_MAINTENANCE8) unset(CMAKE_REQUIRED_INCLUDES) if(NOT ANYPS5_VULKAN_HAS_MAINTENANCE8) @@ -44,6 +47,10 @@ find_package(SPIRV-Headers CONFIG REQUIRED) if(NOT TARGET SPIRV-Headers::SPIRV-Headers) message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: SPIRV-Headers was found but does not provide SPIRV-Headers::SPIRV-Headers") endif() +get_target_property(ANYPS5_SPIRV_HEADERS_INCLUDE_DIR SPIRV-Headers::SPIRV-Headers INTERFACE_INCLUDE_DIRECTORIES) +if(NOT ANYPS5_SPIRV_HEADERS_INCLUDE_DIR) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: SPIRV-Headers::SPIRV-Headers exposes no INTERFACE_INCLUDE_DIRECTORIES") +endif() find_package(glslang CONFIG REQUIRED) if(NOT TARGET glslang::glslang OR NOT TARGET glslang::SPIRV) @@ -53,10 +60,6 @@ add_library(glslang INTERFACE) target_link_libraries(glslang INTERFACE glslang::glslang) add_library(SPIRV INTERFACE) target_link_libraries(SPIRV INTERFACE glslang::SPIRV) - -# The bundled glslang has both and under one root; the -# system install keeps under /include and the rest, -# including , under /include/glslang. find_path(ANYPS5_GLSLANG_PUBLIC_INCLUDE_DIR NAMES glslang/Public/ShaderLang.h) if(NOT ANYPS5_GLSLANG_PUBLIC_INCLUDE_DIR) message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: glslang/Public/ShaderLang.h was not found; the glslang headers are incomplete") @@ -65,6 +68,7 @@ set(ANYPS5_GLSLANG_SPIRV_INCLUDE_DIR "${ANYPS5_GLSLANG_PUBLIC_INCLUDE_DIR}/glsla if(NOT EXISTS "${ANYPS5_GLSLANG_SPIRV_INCLUDE_DIR}/SPIRV/GlslangToSpv.h") message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: ${ANYPS5_GLSLANG_SPIRV_INCLUDE_DIR}/SPIRV/GlslangToSpv.h was not found") endif() +set(ANYPS5_GLSLANG_INCLUDE_DIRS "${ANYPS5_GLSLANG_PUBLIC_INCLUDE_DIR};${ANYPS5_GLSLANG_SPIRV_INCLUDE_DIR}") find_program(ANYPS5_GLSLANG_STANDALONE_EXECUTABLE NAMES glslang glslangValidator) if(NOT ANYPS5_GLSLANG_STANDALONE_EXECUTABLE) @@ -80,22 +84,15 @@ if(ANYPS5_ENABLE_SPIRV_TOOLS) endif() endif() -set(_anyps5_system_include_targets - SDL2::SDL2 - Freetype::Freetype - Vulkan::Headers - SPIRV-Headers::SPIRV-Headers - glslang::glslang - glslang::SPIRV -) -set(_anyps5_system_includes "") -foreach(_target IN LISTS _anyps5_system_include_targets) - if(TARGET ${_target}) - get_target_property(_target_includes ${_target} INTERFACE_INCLUDE_DIRECTORIES) - if(_target_includes) - list(APPEND _anyps5_system_includes ${_target_includes}) - endif() +set(ANYPS5_SYSTEM_DEPS_STAMP "") +list(APPEND ANYPS5_SYSTEM_DEPS_PACKAGES VulkanHeaders SPIRV-Headers glslang) +if(ANYPS5_ENABLE_SPIRV_TOOLS) + list(APPEND ANYPS5_SYSTEM_DEPS_PACKAGES SPIRV-Tools-opt) +endif() +foreach(_package IN LISTS ANYPS5_SYSTEM_DEPS_PACKAGES) + set(_version "unknown") + if(DEFINED ${_package}_VERSION) + set(_version "${${_package}_VERSION}") endif() + string(APPEND ANYPS5_SYSTEM_DEPS_STAMP " ${_package}=${_version}") endforeach() -list(APPEND _anyps5_system_includes ${ANYPS5_GLSLANG_SPIRV_INCLUDE_DIR} ${ANYPS5_GLSLANG_PUBLIC_INCLUDE_DIR}) -include_directories(SYSTEM ${_anyps5_system_includes}) diff --git a/core/libs/prx/libSceAgc/CMakeLists.txt b/core/libs/prx/libSceAgc/CMakeLists.txt index b6f691fb4..2441a415c 100644 --- a/core/libs/prx/libSceAgc/CMakeLists.txt +++ b/core/libs/prx/libSceAgc/CMakeLists.txt @@ -209,7 +209,7 @@ if(BUILD_TESTING) endif() add_test_executable(agc_shader_fusion_tests EXCLUDE_FROM_ALL tests/ShaderFusion.cpp) -target_include_directories(agc_shader_fusion_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include ${CMAKE_SOURCE_DIR}/3rdparty/SPIRV-Headers/include) +target_include_directories(agc_shader_fusion_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR} ${ANYPS5_SPIRV_HEADERS_INCLUDE_DIR}) target_link_libraries(agc_shader_fusion_tests PRIVATE libSceAgc libSceAgcDriver libc) set_target_properties(agc_shader_fusion_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_shader_fusion_tests) diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 273bb2a0a..aeff4141d 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -224,8 +224,8 @@ target_include_directories(libSceAgcDriver PRIVATE ${LIBS_INCLUDE_DIR} ${agcTextureDetileGeneratedRoot} ${CMAKE_SOURCE_DIR}/core/shader/recompiler/RdnaDecoder/include - ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include - ${CMAKE_SOURCE_DIR}/3rdparty/SPIRV-Headers/include + ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR} + ${ANYPS5_SPIRV_HEADERS_INCLUDE_DIR} ) file(GLOB_RECURSE agcShaderCacheGuardSources CONFIGURE_DEPENDS RELATIVE "${CMAKE_CURRENT_SOURCE_DIR}" @@ -288,8 +288,8 @@ add_test_executable(agc_driver_tests EXCLUDE_FROM_ALL target_include_directories(agc_driver_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler - ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include - ${CMAKE_SOURCE_DIR}/3rdparty/SPIRV-Headers/include + ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR} + ${ANYPS5_SPIRV_HEADERS_INCLUDE_DIR} ) target_link_libraries(agc_driver_tests PRIVATE @@ -376,7 +376,7 @@ add_test_executable(agc_subgroup_clock_tests EXCLUDE_FROM_ALL tests/SubgroupClock.cpp Execution/src/SubgroupClock.cpp ) -target_include_directories(agc_subgroup_clock_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include) +target_include_directories(agc_subgroup_clock_tests PRIVATE ${LIBS_INCLUDE_DIR} ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) set_target_properties(agc_subgroup_clock_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_subgroup_clock_tests) if(BUILD_TESTING) @@ -515,7 +515,7 @@ set_target_properties(agc_pass_hazards_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_pass_hazards_tests) add_test_executable(agc_dcc_key_sharing_tests EXCLUDE_FROM_ALL tests/DccKeySharing.cpp) -target_include_directories(agc_dcc_key_sharing_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include) +target_include_directories(agc_dcc_key_sharing_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) target_link_libraries(agc_dcc_key_sharing_tests PRIVATE libc) set_target_properties(agc_dcc_key_sharing_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_dcc_key_sharing_tests) @@ -529,7 +529,7 @@ if(BUILD_TESTING) endif() add_test_executable(agc_buffer_pool_tests EXCLUDE_FROM_ALL tests/BufferPool.cpp) -target_include_directories(agc_buffer_pool_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include) +target_include_directories(agc_buffer_pool_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) target_link_libraries(agc_buffer_pool_tests PRIVATE libSceAgcDriver libc) set_target_properties(agc_buffer_pool_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_buffer_pool_tests) @@ -543,7 +543,7 @@ if(BUILD_TESTING) endif() add_test_executable(agc_display_buffer_dcc_tests EXCLUDE_FROM_ALL tests/DisplayBufferDcc.cpp) -target_include_directories(agc_display_buffer_dcc_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include) +target_include_directories(agc_display_buffer_dcc_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) target_link_libraries(agc_display_buffer_dcc_tests PRIVATE libSceAgcDriver libc) set_target_properties(agc_display_buffer_dcc_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_display_buffer_dcc_tests) @@ -573,7 +573,7 @@ set_target_properties(agc_shader_disk_cache_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_shader_disk_cache_tests) add_test_executable(agc_shader_device_profile_tests EXCLUDE_FROM_ALL tests/ShaderDeviceProfile.cpp Execution/src/ShaderDeviceProfile.cpp) -target_include_directories(agc_shader_device_profile_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include) +target_include_directories(agc_shader_device_profile_tests PRIVATE ${LIBS_INCLUDE_DIR} ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) target_link_libraries(agc_shader_device_profile_tests PRIVATE shader_recompiler_agc_driver libc) set_target_properties(agc_shader_device_profile_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_shader_device_profile_tests) @@ -701,7 +701,7 @@ target_include_directories(agc_driver_recorder_tests PRIVATE ${LIBS_INCLUDE_DIR} ${agcTestShaderGeneratedRoot} ${CMAKE_SOURCE_DIR}/core/shader/recompiler - ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include + ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR} ) target_link_libraries(agc_driver_recorder_tests PRIVATE @@ -754,7 +754,7 @@ if(BUILD_TESTING) endif() add_test_executable(agc_recorder_kept_bytes_tests EXCLUDE_FROM_ALL tests/RecorderKeptBytes.cpp) -target_include_directories(agc_recorder_kept_bytes_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include) +target_include_directories(agc_recorder_kept_bytes_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) target_link_libraries(agc_recorder_kept_bytes_tests PRIVATE libSceAgcDriver libc) set_target_properties(agc_recorder_kept_bytes_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_recorder_kept_bytes_tests) @@ -840,8 +840,8 @@ target_include_directories(agc_driver_graphics_tests PRIVATE ${CMAKE_SOURCE_DIR}/core/shader/recompiler/IntermediateRepresentation/include ${CMAKE_SOURCE_DIR}/core/shader/recompiler/ControlFlow/include ${CMAKE_SOURCE_DIR}/core/shader/recompiler/RdnaDecoder/include - ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include - ${CMAKE_SOURCE_DIR}/3rdparty/SPIRV-Headers/include + ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR} + ${ANYPS5_SPIRV_HEADERS_INCLUDE_DIR} ) target_link_libraries(agc_driver_graphics_tests PRIVATE @@ -866,7 +866,7 @@ add_test_executable(agc_driver_bda_device_tests EXCLUDE_FROM_ALL target_include_directories(agc_driver_bda_device_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler - ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include + ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR} ) target_link_libraries(agc_driver_bda_device_tests PRIVATE SDL2-static libc shader_recompiler_agc_driver) set_target_properties(agc_driver_bda_device_tests PROPERTIES CXX_EXTENSIONS OFF) @@ -879,8 +879,8 @@ add_test_executable(agc_driver_mesh_tests EXCLUDE_FROM_ALL target_include_directories(agc_driver_mesh_tests PRIVATE ${LIBS_INCLUDE_DIR} ${agcTextureDetileGeneratedRoot} - ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include - ${CMAKE_SOURCE_DIR}/3rdparty/SPIRV-Headers/include + ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR} + ${ANYPS5_SPIRV_HEADERS_INCLUDE_DIR} ) target_link_libraries(agc_driver_mesh_tests PRIVATE shader_recompiler_agc_driver SDL2-static libc libkernel ${CMAKE_DL_LIBS}) set_target_properties(agc_driver_mesh_tests PROPERTIES CXX_EXTENSIONS OFF) @@ -896,8 +896,8 @@ foreach(agcExecutionTestSource IN LISTS agcExecutionTestSources) target_include_directories(${agcExecutionTest} PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler - ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include - ${CMAKE_SOURCE_DIR}/3rdparty/SPIRV-Headers/include + ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR} + ${ANYPS5_SPIRV_HEADERS_INCLUDE_DIR} ) target_link_libraries(${agcExecutionTest} PRIVATE libSceAgcDriver SDL2-static libc) set_target_properties(${agcExecutionTest} PROPERTIES CXX_EXTENSIONS OFF) @@ -1127,8 +1127,8 @@ if(AGC_BUILD_VISUAL_TEST) ${CMAKE_SOURCE_DIR}/core/shader/recompiler/IntermediateRepresentation/include ${CMAKE_SOURCE_DIR}/core/shader/recompiler/ControlFlow/include ${CMAKE_SOURCE_DIR}/core/shader/recompiler/RdnaDecoder/include - ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include - ${CMAKE_SOURCE_DIR}/3rdparty/SPIRV-Headers/include + ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR} + ${ANYPS5_SPIRV_HEADERS_INCLUDE_DIR} ) target_link_libraries(agc_driver_visual_test PRIVATE @@ -1190,7 +1190,7 @@ configure_windows_unwind(agc_shader_replay) if(BUILD_TESTING) add_test_executable(texture_format_lookup_tests ${LIBS_INCLUDE_DIR}/tests/TextureFormatLookup.cpp) - target_include_directories(texture_format_lookup_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include) + target_include_directories(texture_format_lookup_tests PRIVATE ${LIBS_INCLUDE_DIR} ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) target_link_libraries(texture_format_lookup_tests PRIVATE libSceAgcDriver libc) configure_windows_unwind(texture_format_lookup_tests) add_test(NAME texture_format_lookup COMMAND texture_format_lookup_tests) diff --git a/core/libs/prx/libSceAudioIn/CMakeLists.txt b/core/libs/prx/libSceAudioIn/CMakeLists.txt index 405b20d4a..f25187b9c 100644 --- a/core/libs/prx/libSceAudioIn/CMakeLists.txt +++ b/core/libs/prx/libSceAudioIn/CMakeLists.txt @@ -1,5 +1,5 @@ add_library(libSceAudioIn SHARED EXCLUDE_FROM_ALL Export.cpp) -target_include_directories(libSceAudioIn PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/3rdparty/SDL2/include) +target_include_directories(libSceAudioIn PRIVATE ${LIBS_INCLUDE_DIR} ${ANYPS5_SDL2_INCLUDE_DIR}) target_link_libraries(libSceAudioIn PRIVATE SDL2-static) target_link_options(libSceAudioIn PRIVATE LINKER:--exclude-libs,ALL) configure_windows_unwind(libSceAudioIn) diff --git a/core/libs/prx/libSceAudioOut/CMakeLists.txt b/core/libs/prx/libSceAudioOut/CMakeLists.txt index dfd3f5ea7..968a17129 100644 --- a/core/libs/prx/libSceAudioOut/CMakeLists.txt +++ b/core/libs/prx/libSceAudioOut/CMakeLists.txt @@ -11,7 +11,7 @@ add_library(libSceAudioOut SHARED EXCLUDE_FROM_ALL target_include_directories(libSceAudioOut PRIVATE ${LIBS_INCLUDE_DIR} - ${CMAKE_SOURCE_DIR}/3rdparty/SDL2/include + ${ANYPS5_SDL2_INCLUDE_DIR} ) target_link_libraries(libSceAudioOut PRIVATE libc libkernel SDL2-static) configure_windows_unwind(libSceAudioOut) diff --git a/core/libs/prx/libSceKeyboard/CMakeLists.txt b/core/libs/prx/libSceKeyboard/CMakeLists.txt index 0e3fe276f..6f23ed265 100644 --- a/core/libs/prx/libSceKeyboard/CMakeLists.txt +++ b/core/libs/prx/libSceKeyboard/CMakeLists.txt @@ -11,7 +11,7 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/tests/Keyboard.cpp ${CMAKE_CURRENT_SOURCE_DIR}/../libSceVideoOut/src/KeyboardInput.cpp ) - target_include_directories(keyboard_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/3rdparty/SDL2/include) + target_include_directories(keyboard_tests PRIVATE ${LIBS_INCLUDE_DIR} ${ANYPS5_SDL2_INCLUDE_DIR}) target_link_libraries(keyboard_tests PRIVATE libSceKeyboard libkernel libc) configure_windows_unwind(keyboard_tests) add_test(NAME keyboard_input COMMAND keyboard_tests) diff --git a/core/libs/prx/libSceMouse/CMakeLists.txt b/core/libs/prx/libSceMouse/CMakeLists.txt index a8284f73a..d98fdf1a5 100644 --- a/core/libs/prx/libSceMouse/CMakeLists.txt +++ b/core/libs/prx/libSceMouse/CMakeLists.txt @@ -11,7 +11,7 @@ if(BUILD_TESTING) ${CMAKE_CURRENT_SOURCE_DIR}/tests/Mouse.cpp ${CMAKE_CURRENT_SOURCE_DIR}/../libSceVideoOut/src/MouseInput.cpp ) - target_include_directories(mouse_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/3rdparty/SDL2/include) + target_include_directories(mouse_tests PRIVATE ${LIBS_INCLUDE_DIR} ${ANYPS5_SDL2_INCLUDE_DIR}) target_link_libraries(mouse_tests PRIVATE libSceMouse libkernel libc) configure_windows_unwind(mouse_tests) add_test(NAME mouse_input COMMAND mouse_tests) diff --git a/core/libs/prx/libScePad/CMakeLists.txt b/core/libs/prx/libScePad/CMakeLists.txt index a4ed0f81f..16919e233 100644 --- a/core/libs/prx/libScePad/CMakeLists.txt +++ b/core/libs/prx/libScePad/CMakeLists.txt @@ -8,7 +8,7 @@ configure_windows_unwind(libScePad) if(BUILD_TESTING) add_test_executable(guest_pad_tests ${LIBS_INCLUDE_DIR}/tests/GuestPad.cpp) - target_include_directories(guest_pad_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/3rdparty/SDL2/include) + target_include_directories(guest_pad_tests PRIVATE ${LIBS_INCLUDE_DIR} ${ANYPS5_SDL2_INCLUDE_DIR}) target_link_libraries(guest_pad_tests PRIVATE libScePad libc) configure_windows_unwind(guest_pad_tests) add_test(NAME guest_pad COMMAND guest_pad_tests) diff --git a/core/libs/prx/libSceVideoOut/CMakeLists.txt b/core/libs/prx/libSceVideoOut/CMakeLists.txt index cf2807919..e8f4694ff 100644 --- a/core/libs/prx/libSceVideoOut/CMakeLists.txt +++ b/core/libs/prx/libSceVideoOut/CMakeLists.txt @@ -13,8 +13,8 @@ add_library(libSceVideoOut SHARED EXCLUDE_FROM_ALL target_include_directories(libSceVideoOut PRIVATE ${LIBS_INCLUDE_DIR} - ${CMAKE_SOURCE_DIR}/3rdparty/SDL2/include - ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include + ${ANYPS5_SDL2_INCLUDE_DIR} + ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR} ) target_link_libraries(libSceVideoOut PRIVATE libSceAgcDriver libScePad libSceMouse libSceKeyboard libc libkernel decoder_png SDL2-static) if(WIN32) @@ -27,7 +27,7 @@ foreach(runtimeLinkFlag IN ITEMS -static-libgcc -static-libstdc++ -static) string(REPLACE "${runtimeLinkFlag}" "" CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS}") endforeach() add_test_executable(video_out_flip_tests EXCLUDE_FROM_ALL tests/Flip.cpp) -target_include_directories(video_out_flip_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/3rdparty/SDL2/include ${CMAKE_SOURCE_DIR}/core/shader/recompiler ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include) +target_include_directories(video_out_flip_tests PRIVATE ${LIBS_INCLUDE_DIR} ${ANYPS5_SDL2_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) target_link_libraries(video_out_flip_tests PRIVATE libSceVideoOut libSceAgcDriver libkernel libc) set_target_properties(video_out_flip_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(video_out_flip_tests) diff --git a/core/shader/recompiler/CMakeLists.txt b/core/shader/recompiler/CMakeLists.txt index d25cf641a..14af1548c 100644 --- a/core/shader/recompiler/CMakeLists.txt +++ b/core/shader/recompiler/CMakeLists.txt @@ -5,7 +5,7 @@ function(add_shader_recompiler target runtimeTarget) endif() if(ANYPS5_ENABLE_SPIRV_TOOLS AND NOT TARGET SPIRV-Tools-opt) - set(SPIRV-Headers_SOURCE_DIR "${PROJECT_SOURCE_DIR}/3rdparty/SPIRV-Headers") + set(SPIRV-Headers_SOURCE_DIR "${ANYPS5_SPIRV_HEADERS_SOURCE_DIR}") set(SPIRV_SKIP_EXECUTABLES ON) set(SPIRV_SKIP_TESTS ON) set(SPIRV_TOOLS_BUILD_STATIC ON) @@ -156,8 +156,8 @@ function(add_shader_recompiler target runtimeTarget) ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/Translation/include ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/Optimization/include ${CMAKE_CURRENT_FUNCTION_LIST_DIR}/SpirvBackend/include - ${PROJECT_SOURCE_DIR}/3rdparty/SPIRV-Headers/include - ${PROJECT_SOURCE_DIR}/3rdparty/glslang + ${ANYPS5_SPIRV_HEADERS_INCLUDE_DIR} + ${ANYPS5_GLSLANG_INCLUDE_DIRS} ) target_compile_features(${target} PUBLIC cxx_std_20) @@ -188,7 +188,7 @@ function(add_shader_recompiler target runtimeTarget) find_package(Git QUIET) set(shaderCacheSubmoduleStamps) if(NOT ANYPS5_USE_SYSTEM_DEPS) - set(shaderCacheSubmodules "${PROJECT_SOURCE_DIR}/3rdparty/SPIRV-Headers" "${PROJECT_SOURCE_DIR}/3rdparty/glslang") + set(shaderCacheSubmodules "${ANYPS5_SPIRV_HEADERS_SOURCE_DIR}" "${ANYPS5_GLSLANG_SOURCE_DIR}") if(ANYPS5_ENABLE_SPIRV_TOOLS) list(APPEND shaderCacheSubmodules "${PROJECT_SOURCE_DIR}/3rdparty/SPIRV-Tools") endif() @@ -210,6 +210,9 @@ function(add_shader_recompiler target runtimeTarget) get_target_property(shaderCacheIncludeDirectories ${target} INCLUDE_DIRECTORIES) string(TOUPPER "${CMAKE_BUILD_TYPE}" shaderCacheBuildType) set(shaderCacheExtra "${CMAKE_CXX_COMPILER_ID} ${CMAKE_CXX_COMPILER_VERSION} ${CMAKE_BUILD_TYPE} spirv-tools=${ANYPS5_ENABLE_SPIRV_TOOLS} ${CMAKE_SYSTEM_PROCESSOR} flags=${CMAKE_CXX_FLAGS} ${CMAKE_CXX_FLAGS_${shaderCacheBuildType}}") + if(ANYPS5_USE_SYSTEM_DEPS) + string(APPEND shaderCacheExtra " system-deps=${ANYPS5_SYSTEM_DEPS_STAMP}") + endif() set(shaderCacheReviewedNames BuildBarycentricGeometryShader BuildPreparedShaderKey From ce6edc5a0022c2664f52996bf9f6475b0de691f2 Mon Sep 17 00:00:00 2001 From: "husari.orozco" Date: Wed, 7 Oct 2026 16:35:06 -0600 Subject: [PATCH 440/460] build: reject an SDL2 without HIDAPI and fall back to shared SPIRV-Tools A system SDL2 that does not define SDL_JOYSTICK_HIDAPI (sdl2-compat over SDL3 ships no such macro, and neither does a build with the driver disabled) fails pad_sdl and makes the disk audio driver open the port with a different channel count, so ANYPS5_USE_SYSTEM_DEPS=ON now rejects it with a named error, like the VK_KHR_maintenance8 check. Ubuntu's real SDL2 defines the macro through its SDL_config.h indirection and is unaffected. SPIRV-Tools only exports SPIRV-Tools-static when built with SPIRV_TOOLS_BUILD_STATIC=ON; distributions that ship it shared expose SPIRV-Tools instead, so a shared target is aliased to the static name. --- cmake/SystemDeps.cmake | 26 +++++++++++++++++++++++--- docs/dev/BUILD.md | 2 +- 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/cmake/SystemDeps.cmake b/cmake/SystemDeps.cmake index e08c7b504..c679aaab3 100644 --- a/cmake/SystemDeps.cmake +++ b/cmake/SystemDeps.cmake @@ -9,6 +9,8 @@ if(NOT ANYPS5_USE_SYSTEM_DEPS) return() endif() +include(CheckCXXSourceCompiles) + find_package(SDL2 CONFIG REQUIRED) if(NOT TARGET SDL2::SDL2) message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: SDL2 was found but does not provide SDL2::SDL2") @@ -19,6 +21,17 @@ get_target_property(ANYPS5_SDL2_INCLUDE_DIR SDL2::SDL2 INTERFACE_INCLUDE_DIRECTO if(NOT ANYPS5_SDL2_INCLUDE_DIR) message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: SDL2::SDL2 exposes no INTERFACE_INCLUDE_DIRECTORIES") endif() +set(ANYPS5_SDL2_PROBE_INCLUDES ${ANYPS5_SDL2_INCLUDE_DIR}) +foreach(_dir IN LISTS ANYPS5_SDL2_INCLUDE_DIR) + list(APPEND ANYPS5_SDL2_PROBE_INCLUDES "${_dir}/SDL2") +endforeach() +list(REMOVE_DUPLICATES ANYPS5_SDL2_PROBE_INCLUDES) +set(CMAKE_REQUIRED_INCLUDES ${ANYPS5_SDL2_PROBE_INCLUDES}) +check_cxx_source_compiles("#include \"SDL_config.h\"\n#ifndef SDL_JOYSTICK_HIDAPI\n#error SDL_JOYSTICK_HIDAPI is not defined\n#endif\nint main() { return 0; }" ANYPS5_SDL2_HAS_HIDAPI) +unset(CMAKE_REQUIRED_INCLUDES) +if(NOT ANYPS5_SDL2_HAS_HIDAPI) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: the system SDL2 does not define SDL_JOYSTICK_HIDAPI, so a DualSense gets no light bar, touchpad or trigger effects and the disk audio driver does not preserve the requested format; sdl2-compat (SDL3 underneath) and SDL2 builds with HIDAPI disabled both fail the pad and audio tests; install a real SDL2 2.x built with HIDAPI") +endif() find_package(Freetype REQUIRED) if(NOT TARGET Freetype::Freetype) @@ -35,7 +48,6 @@ get_target_property(ANYPS5_VULKAN_HEADERS_INCLUDE_DIR Vulkan::Headers INTERFACE_ if(NOT ANYPS5_VULKAN_HEADERS_INCLUDE_DIR) message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: Vulkan::Headers exposes no INTERFACE_INCLUDE_DIRECTORIES") endif() -include(CheckCXXSourceCompiles) set(CMAKE_REQUIRED_INCLUDES ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) check_cxx_source_compiles("#include \nint main() { VkPhysicalDeviceMaintenance8FeaturesKHR features{}; (void)features; return 0; }" ANYPS5_VULKAN_HAS_MAINTENANCE8) unset(CMAKE_REQUIRED_INCLUDES) @@ -79,8 +91,16 @@ set_target_properties(glslang-standalone PROPERTIES IMPORTED_LOCATION "${ANYPS5_ if(ANYPS5_ENABLE_SPIRV_TOOLS) find_package(SPIRV-Tools-opt CONFIG REQUIRED) - if(NOT TARGET SPIRV-Tools-opt OR NOT TARGET SPIRV-Tools-static) - message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: SPIRV-Tools was found but does not provide SPIRV-Tools-opt and SPIRV-Tools-static") + if(NOT TARGET SPIRV-Tools-opt) + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: SPIRV-Tools was found but does not provide SPIRV-Tools-opt") + endif() + if(NOT TARGET SPIRV-Tools-static) + if(TARGET SPIRV-Tools) + add_library(SPIRV-Tools-static INTERFACE) + target_link_libraries(SPIRV-Tools-static INTERFACE SPIRV-Tools) + else() + message(FATAL_ERROR "ANYPS5_USE_SYSTEM_DEPS: SPIRV-Tools was found but provides neither SPIRV-Tools-static nor SPIRV-Tools") + endif() endif() endif() diff --git a/docs/dev/BUILD.md b/docs/dev/BUILD.md index 1c30aa947..50e7023fe 100644 --- a/docs/dev/BUILD.md +++ b/docs/dev/BUILD.md @@ -75,7 +75,7 @@ Build configuration parameters: SDL and FreeType settings forced by the root `CMakeLists.txt` cannot be overridden with `-D`. -With `ANYPS5_USE_SYSTEM_DEPS=ON` those libraries come from the system (on Debian and Ubuntu: `libsdl2-dev`, `libfreetype-dev`, `libvulkan-dev`, `spirv-headers`, `glslang-dev`, `glslang-tools` and `spirv-tools`). A missing package, or Vulkan-Headers older than 1.4.310 (the agc driver uses `VK_KHR_maintenance8`), fails the configure with an error and never falls back to the submodule. The prx libraries are shared objects, so SDL2 is linked from the system as a shared library; the default build keeps its own static SDL2. `ffmpeg-core`, `libjpeg-turbo`, `stb`, `VulkanMemoryAllocator` and `LibAtrac9` always stay bundled and still need `git submodule update --init`. +With `ANYPS5_USE_SYSTEM_DEPS=ON` those libraries come from the system (on Debian and Ubuntu: `libsdl2-dev`, `libfreetype-dev`, `libvulkan-dev`, `spirv-headers`, `glslang-dev`, `glslang-tools` and `spirv-tools`). A missing package, Vulkan-Headers older than 1.4.310 (the agc driver uses `VK_KHR_maintenance8`), or an SDL2 without the HIDAPI joystick driver (sdl2-compat over SDL3, or a build with HIDAPI disabled, which breaks the pad and disk-audio tests), fails the configure with an error and never falls back to the submodule. The forced SDL and FreeType settings from the root `CMakeLists.txt` do not apply with the flag ON: the system SDL2 comes with whatever its distribution enabled. The prx libraries are shared objects, so SDL2 is linked from the system as a shared library; the default build keeps its own static SDL2. `ffmpeg-core`, `libjpeg-turbo`, `stb`, `VulkanMemoryAllocator` and `LibAtrac9` always stay bundled and still need `git submodule update --init`. ## Write watch tests From 636bfc7298f81388e5cce82e69011c86207371fe Mon Sep 17 00:00:00 2001 From: "husari.orozco" Date: Thu, 8 Oct 2026 09:45:35 -0600 Subject: [PATCH 441/460] docs(contributing): document the ANYPS5_USE_SYSTEM_DEPS exception --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8fb682627..c85c888c6 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -11,7 +11,7 @@ - Don't add replacements for modules that titles ship themselves in `sce_module/`, `sce_modules/` or `prx/`: engine or middleware modules (Cohtml, FMOD, GOG Galaxy) and SDK libraries that only wrap other system libraries (NpCppWebApi over NpWebApi2). The relinker converts and loads the title's own module, and a host library with the same name is left out of `DT_NEEDED`. Only system libraries, which reach the kernel or the hardware, are reimplemented in [core/libs/prx](core/libs/prx). - Avoid non-standard extensions (`__attribute__`, etc.) where standard C++ is enough. Helper symbols that must not become NIDs use the `_nid_no_patch` or `_nid_no_patch_cut` suffix. - The relinker uses only the C++20 standard library. -- Third-party code is added as a submodule under `3rdparty/` and built from source, not found on the system. +- Third-party code is added as a submodule under `3rdparty/` and built from source, not found on the system. With `ANYPS5_USE_SYSTEM_DEPS=ON` the packages packaged by the distributions (SDL2, freetype, Vulkan-Headers, SPIRV-Headers, glslang and SPIRV-Tools) come from the system instead; that resolution lives in [cmake/SystemDeps.cmake](cmake/SystemDeps.cmake) and the default build never reads the system. - Don't add tests that only check that a symbol is exported: a missing export already fails at startup. - Each test finishes in 30 s on the CI runners, Vulkan tests included (Linux runs them on lavapipe), counting the first run with no shader cache. ctest runs in parallel there, so a slow test competes for the cores and times out on some runs and not others. Split or shrink a test that gets close instead of raising its timeout. CI reports every test over 30 s as a warning on the pull request. From abcfa7c23d2160eef38f6ab4cc5654588d8f3a30 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 10:18:29 +0200 Subject: [PATCH 442/460] build: route the Vulkan-Headers include of tests added since the rebase through ANYPS5_VULKAN_HEADERS_INCLUDE_DIR Co-Authored-By: Claude --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index aeff4141d..8357c6c20 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -364,7 +364,7 @@ set_target_properties(agc_structurizer_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_structurizer_tests) add_test_executable(agc_fragment_barycentric_tests EXCLUDE_FROM_ALL tests/FragmentBarycentric.cpp) -target_include_directories(agc_fragment_barycentric_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include) +target_include_directories(agc_fragment_barycentric_tests PRIVATE ${LIBS_INCLUDE_DIR} ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) set_target_properties(agc_fragment_barycentric_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_fragment_barycentric_tests) if(BUILD_TESTING) @@ -509,7 +509,7 @@ if(BUILD_TESTING) endif() add_test_executable(agc_pass_hazards_tests EXCLUDE_FROM_ALL tests/PassHazards.cpp Graphics/src/PassHazards.cpp) -target_include_directories(agc_pass_hazards_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include) +target_include_directories(agc_pass_hazards_tests PRIVATE ${LIBS_INCLUDE_DIR} ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) target_link_libraries(agc_pass_hazards_tests PRIVATE libc) set_target_properties(agc_pass_hazards_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_pass_hazards_tests) @@ -676,7 +676,7 @@ target_include_directories(agc_driver_cmask_fast_clear_tests PRIVATE ${LIBS_INCLUDE_DIR} ${agcTestShaderGeneratedRoot} ${CMAKE_SOURCE_DIR}/core/shader/recompiler - ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include + ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR} ) target_link_libraries(agc_driver_cmask_fast_clear_tests PRIVATE @@ -1199,7 +1199,7 @@ if(BUILD_TESTING) "PATH=path_list_prepend:$") endif() add_test_executable(linear_display_buffer_tests ${LIBS_INCLUDE_DIR}/tests/LinearDisplayBuffer.cpp) - target_include_directories(linear_display_buffer_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include) + target_include_directories(linear_display_buffer_tests PRIVATE ${LIBS_INCLUDE_DIR} ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) target_link_libraries(linear_display_buffer_tests PRIVATE libSceAgcDriver libc) configure_windows_unwind(linear_display_buffer_tests) add_test(NAME linear_display_buffer COMMAND linear_display_buffer_tests) From c133e23ddeccf9eb0bfd5775da5c45e69f25270d Mon Sep 17 00:00:00 2001 From: Michon <231849@buas.nl> Date: Sat, 10 Oct 2026 10:09:51 +0200 Subject: [PATCH 443/460] fix(agc): depth fast clears through the HTILE metadata Rebased onto main as one commit (the six commits of #374). A uniform fill over a depth surface's whole HTILE (DMA_DATA, the fill kernels or a CPU store) clears the depth and stencil aspects its ZMask and SMem fields say, to DB_DEPTH_CLEAR / DB_STENCIL_CLEAR, when the surface is next bound or sampled. Main's RADV clear_htile_mask recognition (#2229) now goes through the same pending clear: an HTILE clear marks the matching surfaces' depth aspect, and one for a surface not created yet is applied when it is first bound. Main's HTILE address decode and this one are merged into one (TILE_SURFACE_ENABLE, a nonzero DB_HTILE_DATA_BASE and a depth format). Co-Authored-By: Claude Opus 5.5 (1M context) --- .../libSceAgcDriver/Execution/include/Pm4.hpp | 2 + .../src/Driver/Memory/BufferFill.cpp | 2 + .../src/Driver/Packets/PacketMemory.cpp | 4 + .../prx/libSceAgcDriver/Execution/src/Pm4.cpp | 8 + .../Graphics/include/DepthSurface.hpp | 3 + .../Graphics/include/State.hpp | 1 + .../Graphics/src/DepthSurface.cpp | 60 +++++- .../libSceAgcDriver/Graphics/src/State.cpp | 5 +- .../prx/libSceAgcDriver/tests/Graphics.cpp | 29 ++- .../tests/execution/DepthFastClear.cpp | 49 +++++ .../tests/execution/DepthFastClearHarness.hpp | 146 ++++++++++++++ .../tests/execution/DepthFastClearSubmit.cpp | 181 ++++++++++++++++++ docs/dev/TechnicalDebt.md | 4 + 13 files changed, 481 insertions(+), 13 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClear.cpp create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearHarness.hpp create mode 100644 core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearSubmit.cpp diff --git a/core/libs/prx/libSceAgcDriver/Execution/include/Pm4.hpp b/core/libs/prx/libSceAgcDriver/Execution/include/Pm4.hpp index 08493a9a7..ce6f1638e 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/include/Pm4.hpp +++ b/core/libs/prx/libSceAgcDriver/Execution/include/Pm4.hpp @@ -140,6 +140,8 @@ struct MemoryCopy { std::size_t bytes; }; std::optional DecodeMemoryCopy(std::span packet); +std::uint32_t DmaSource(std::span packet); +std::uint32_t DmaDestination(std::span packet); // A DISPATCH_INDIRECT's arguments: the guest address of its three group-count dwords (no memory // access, so the GPU can read them in place: VulkanDevice::DispatchIndirect), the DISPATCH_DIRECT // packet made by reading them there (through the checked guest memory path, which waits for diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/BufferFill.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/BufferFill.cpp index d0b2593e3..5e7f9a53c 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/BufferFill.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/BufferFill.cpp @@ -5,6 +5,7 @@ #include "prx/libSceAgcDriver/Execution/include/Driver/Memory/DwordPatternFill.hpp" #include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" #include +#include "prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp" #include #include #include @@ -264,6 +265,7 @@ bool Driver::fillBuffer(QueueState& queue, std::uint32_t queueId, std::span(block).first(std::min(chunk, bytes - done)), 16); phase(FillCpu); } + if (pattern[0] == pattern[1] && pattern[1] == pattern[2] && pattern[2] == pattern[3]) Graphics::NoteDepthMetadataFill(base, bytes, pattern[0]); if (profile) { const auto now = std::chrono::steady_clock::now(); const auto cover = static_cast(coverage.cover); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Packets/PacketMemory.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Packets/PacketMemory.cpp index 1e381a778..c7b1198a1 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Packets/PacketMemory.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Driver/Packets/PacketMemory.cpp @@ -5,6 +5,7 @@ #include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" #include "prx/libSceAgcDriver/Execution/include/Pm4.hpp" #include "prx/libSceAgcDriver/Eq/include/Event.hpp" +#include "prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp" #include "prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp" #include "prx/libSceAgcDriver/Graphics/include/Recorder.hpp" #include @@ -58,6 +59,9 @@ bool Driver::preparePacketMemory(const Submission& submission, QueueState& queue } } } + if (opcode == 0x50 && packet.size() >= 7) { + if (Pm4::DmaSource(packet) == 2 && Pm4::DmaDestination(packet) != 1) Graphics::NoteDepthMetadataFill(packet[4] | (static_cast(packet[5]) << 32u), packet[6] & 0x3ffffffu, packet[2]); + } if (!drainAll && !endOfPipeInterrupt && (opcode == 0x49 || opcode == 0x37)) { if (const auto label = Pm4::DecodeLabelWrite(packet)) { const auto bytes = label->Bytes(); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp index e76a0572d..2b8fbede8 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/Pm4.cpp @@ -621,6 +621,14 @@ std::optional ResolveStore(std::span packet, co } } +std::uint32_t DmaSource(std::span packet) { + return dmaSource(packet); +} + +std::uint32_t DmaDestination(std::span packet) { + return dmaDestination(packet); +} + std::optional DecodeMemoryCopy(std::span packet) { if (packet.size() != 7 || ((packet[0] >> 8u) & 0xffu) != 0x50) return std::nullopt; if (!memorySelector(dmaSource(packet)) || !memorySelector(dmaDestination(packet))) return std::nullopt; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp index e2ab1bf30..aa7515e90 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp @@ -19,6 +19,9 @@ bool DepthSurfaceAt(std::uint64_t address); std::uint64_t HtileDepthClearAddress(std::span code, std::span userData, const std::array& numThreads); void NoteHtileDepthClear(std::uint64_t htileAddress); bool DepthSurfaceHolds(const Context& context, const GuestTextureResource& resource, const Texture* texture); +void NoteDepthMetadataFill(std::uint64_t address, std::size_t bytes, std::uint32_t pattern); +VkImageAspectFlags HtileFillClears(std::uint32_t pattern, bool stencilInHtile); +bool HtileFillCovers(std::uint64_t htile, VkExtent2D extent, std::uint64_t address, std::size_t bytes); std::shared_ptr DepthSurfaceTexture(const Context& context, std::span words, const GuestTextureResource& resource, VkComponentMapping components); } diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp index 3d004e9c8..6fc853dcd 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp @@ -67,6 +67,7 @@ struct DepthTarget { float clearDepth; std::uint8_t clearStencil; std::uint64_t htileAddress = 0; + bool htileStencil = false; }; struct State { diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp index c25bbea6e..d8d6929a3 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp @@ -78,22 +78,27 @@ public: } } - void ClearDepth(float value) { + ~DepthSurface() { release(); } + DepthSurface(const DepthSurface&) = delete; + DepthSurface& operator=(const DepthSurface&) = delete; + + void ApplyFastClear() { + const auto aspects = pendingClear & (VK_IMAGE_ASPECT_DEPTH_BIT | (target.stencilAddress != 0 ? VK_IMAGE_ASPECT_STENCIL_BIT : 0u)); + pendingClear = 0; + if (aspects == 0) return; auto* recorder = Recorder::Active(); std::unique_ptr batch; if (recorder == nullptr) batch = std::make_unique(context); const auto commands = recorder != nullptr ? recorder->Commands() : batch->Handle(); - RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT | VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT | VK_ACCESS_SHADER_READ_BIT, VK_ACCESS_TRANSFER_WRITE_BIT); - const VkClearDepthStencilValue clear{value, 0}; - const VkImageSubresourceRange range{VK_IMAGE_ASPECT_DEPTH_BIT, 0, 1, 0, 1}; + const VkImageSubresourceRange range{aspects, 0, 1, 0, 1}; + constexpr VkAccessFlags access = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_TRANSFER_READ_BIT | VK_ACCESS_TRANSFER_WRITE_BIT | VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT | VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT; + RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, access, VK_ACCESS_TRANSFER_WRITE_BIT); + const VkClearDepthStencilValue clear{clearDepth, clearStencil}; context.Function("vkCmdClearDepthStencilImage")(commands, image, VK_IMAGE_LAYOUT_GENERAL, &clear, 1, &range); - RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT | VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT | VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT | VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT | VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT | VK_ACCESS_SHADER_READ_BIT); + RecordMemoryBarrier(context, commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, VK_ACCESS_TRANSFER_WRITE_BIT, access); if (batch) batch->SubmitAndWait(); else Recorder::CountBarriers(Recorder::CommandClass::Draw, 2); } - ~DepthSurface() { release(); } - DepthSurface(const DepthSurface&) = delete; - DepthSurface& operator=(const DepthSurface&) = delete; std::shared_ptr Sampled(std::span words, const GuestTextureResource& resource, VkComponentMapping components) { std::array key{}; @@ -124,6 +129,9 @@ public: return std::any_of(textures.begin(), textures.end(), [&](const auto& entry) { return entry.second.get() == texture; }); } + VkImageAspectFlags pendingClear = 0; + float clearDepth = 0.0f; + std::uint8_t clearStencil = 0; const Context context; const DepthTarget target; VkImage image = VK_NULL_HANDLE; @@ -178,7 +186,10 @@ VkImageView DepthSurfaceView(const Context& context, const DepthTarget& target) const bool cleared = target.htileAddress != 0 && clearedHtiles().erase(target.htileAddress) != 0; for (const auto& surface : surfaces()) { if (surface->context.device != context.device || !sameSurface(surface->target, target)) continue; - if (cleared) surface->ClearDepth(target.clearDepth); + surface->clearDepth = target.clearDepth; + surface->clearStencil = target.clearStencil; + if (cleared) surface->pendingClear |= VK_IMAGE_ASPECT_DEPTH_BIT; + surface->ApplyFastClear(); return surface->view; } surfaces().push_back(std::make_unique(context, target)); @@ -195,7 +206,13 @@ std::uint64_t HtileDepthClearAddress(std::span code, std::s void NoteHtileDepthClear(std::uint64_t htileAddress) { std::lock_guard lock(surfacesMutex()); - clearedHtiles().insert(htileAddress); + bool existing = false; + for (const auto& surface : surfaces()) { + if (surface->target.htileAddress != htileAddress) continue; + surface->pendingClear |= VK_IMAGE_ASPECT_DEPTH_BIT; + existing = true; + } + if (!existing) clearedHtiles().insert(htileAddress); } void ClearDepthSurfaces(VkDevice device) { @@ -220,6 +237,7 @@ std::shared_ptr DepthSurfaceTexture(const Context& context, std::spanApplyFastClear(); return (*found)->Sampled(words, resource, components); } @@ -234,6 +252,28 @@ bool DepthSurfaceHolds(const Context& context, const GuestTextureResource& resou return found != list.rend() && (*found)->Holds(texture); } +VkImageAspectFlags HtileFillClears(std::uint32_t pattern, bool stencilInHtile) { + VkImageAspectFlags cleared = (pattern & 0xfu) == 0 ? VK_IMAGE_ASPECT_DEPTH_BIT : 0u; + if (stencilInHtile && ((pattern >> 8u) & 0x3u) == 0) cleared |= VK_IMAGE_ASPECT_STENCIL_BIT; + return cleared; +} + +bool HtileFillCovers(std::uint64_t htile, VkExtent2D extent, std::uint64_t address, std::size_t bytes) { + if (htile == 0 || htile < address || htile >= address + bytes) return false; + const auto tiles = static_cast((extent.width + 7u) / 8u) * ((extent.height + 7u) / 8u); + return address + bytes - htile >= tiles * 4u; +} + +void NoteDepthMetadataFill(std::uint64_t address, std::size_t bytes, std::uint32_t pattern) { + std::lock_guard lock(surfacesMutex()); + for (const auto& surface : surfaces()) { + const auto& target = surface->target; + if (!HtileFillCovers(target.htileAddress, target.extent, address, bytes)) continue; + const VkImageAspectFlags written = VK_IMAGE_ASPECT_DEPTH_BIT | (target.htileStencil ? VK_IMAGE_ASPECT_STENCIL_BIT : 0u); + surface->pendingClear = (surface->pendingClear & ~written) | HtileFillClears(pattern, target.htileStencil); + } +} + bool DepthSurfaceAt(std::uint64_t address) { std::lock_guard lock(surfacesMutex()); return std::any_of(surfaces().begin(), surfaces().end(), [&](const auto& surface) { return surface->target.address == address || surface->target.stencilAddress == address; }); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp index 2a728c9a1..2210c33d0 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp @@ -227,8 +227,11 @@ void decodeDepth(const Registers& cx, std::uint32_t depthControl, State& result) } depth.format = zFormat == 1 ? (stencil ? VK_FORMAT_D16_UNORM_S8_UINT : VK_FORMAT_D16_UNORM) : (stencil ? VK_FORMAT_D32_SFLOAT_S8_UINT : VK_FORMAT_D32_SFLOAT); depth.clearDepth = readFloat(cx, 0x00b); - if (zFormat != 0 && (read(cx, 0x010) & 0x20000000u) != 0 && find(cx, 0x005) != cx.end()) depth.htileAddress = base(0x005, 0x01e); depth.clearStencil = static_cast(read(cx, 0x00a) & 0xffu); + if (const auto htile = find(cx, 0x005); zFormat != 0 && htile != cx.end() && htile->second != 0 && (read(cx, 0x010) & (1u << 29u)) != 0) { + depth.htileAddress = base(0x005, 0x01e); + depth.htileStencil = stencil && (read(cx, 0x011) & (1u << 29u)) == 0; + } result.depth = depth; result.depthTest = (depthControl & 2u) != 0; result.depthWrite = result.depthTest && (depthControl & 4u) != 0 && !depthReadOnly; diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index 92f660ee5..ce902f814 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -7,6 +7,7 @@ #include "prx/libSceAgcDriver/Graphics/include/VertexInput.hpp" #include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" #include "prx/libSceAgcDriver/Graphics/include/BufferPool.hpp" +#include "prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp" #include "prx/libSceAgcDriver/Graphics/include/ShaderInputState.hpp" #include "prx/libSceAgcDriver/Graphics/include/DccMetadata.hpp" #include "prx/libSceAgcDriver/Graphics/include/ParallelCompare.hpp" @@ -883,6 +884,8 @@ void DepthStencilTests() { queue.context[0x011] = 0x20000181; for (const auto offset : {0x012u, 0x014u}) queue.context[offset] = 0x100; for (const auto offset : {0x013u, 0x015u}) queue.context[offset] = 0x200; + queue.context[0x005] = 0x300; + queue.context[0x01e] = 0x1; queue.context[0x10b] = 0x00050050; queue.context[0x10c] = 0x01ffff01; queue.context[0x10d] = 0x01000001; @@ -896,10 +899,32 @@ void DepthStencilTests() { queue.context[0x1b4] = 2; const auto rejection = AgcDriver::Graphics::DrawRejection(queue, false); Require(rejection.empty(), "precheck rejected a stencil draw with a surface: " + rejection); - Require(state.depth && state.depth->address == 0x10000 && state.depth->stencilAddress == 0x20000 && state.depth->format == VK_FORMAT_D32_SFLOAT_S8_UINT && state.depth->clearStencil == 7, "depth surface decode changed"); + Require(state.depth && state.depth->address == 0x10000 && state.depth->stencilAddress == 0x20000 && state.depth->format == VK_FORMAT_D32_SFLOAT_S8_UINT && state.depth->clearStencil == 7 && state.depth->htileAddress == 0x10000030000ull && !state.depth->htileStencil, "depth surface decode changed"); + { + queue.context[0x011] &= ~(1u << 29u); + const auto stencilTiled = AgcDriver::Graphics::DecodeState(queue); + Require(stencilTiled.depth && stencilTiled.depth->htileStencil, "HTILE holds the stencil state without TILE_STENCIL_DISABLE"); + queue.context[0x011] |= 1u << 29u; + queue.context[0x010] &= ~(1u << 29u); + const auto untiled = AgcDriver::Graphics::DecodeState(queue); + Require(untiled.depth && untiled.depth->htileAddress == 0, "HTILE must be ignored without TILE_SURFACE_ENABLE"); + queue.context[0x010] |= 1u << 29u; + using AgcDriver::Graphics::HtileFillClears; + using AgcDriver::Graphics::HtileFillCovers; + constexpr VkImageAspectFlags both = VK_IMAGE_ASPECT_DEPTH_BIT | VK_IMAGE_ASPECT_STENCIL_BIT; + Require(HtileFillClears(0x0003fff0u, false) == VK_IMAGE_ASPECT_DEPTH_BIT, "a depth-only HTILE fill with ZMask 0 clears the depth"); + Require(HtileFillClears(0x000000f0u, true) == both, "a depth and stencil fast clear (ZMask 0, SMem 0) clears both aspects"); + Require(HtileFillClears(0x000003f0u, true) == VK_IMAGE_ASPECT_DEPTH_BIT, "SMem 3 leaves the stencil uncleared"); + Require(HtileFillClears(0xfffff0ffu, true) == VK_IMAGE_ASPECT_STENCIL_BIT, "ZMask 0xf with SMem 0 clears only the stencil"); + Require(HtileFillClears(0xffffffffu, true) == 0 && HtileFillClears(0xffffffffu, false) == 0, "an expanded fill clears nothing"); + const VkExtent2D extent{16, 8}; + Require(HtileFillCovers(0x1000, extent, 0x1000, 8) && HtileFillCovers(0x1000, extent, 0xff0, 0x20), "a fill over every HTILE word covers the surface"); + Require(!HtileFillCovers(0x1000, extent, 0x1000, 4) && !HtileFillCovers(0x1000, extent, 0x1004, 8) && !HtileFillCovers(0, extent, 0, 64), "a partial fill, one starting past the HTILE base, or no HTILE covers nothing"); + } Require(state.renderExtent.width == 4 && state.renderExtent.height == 2, "render extent ignores the depth surface"); Require(!state.depthTest && !state.depthWrite && state.stencilTest, "depth/stencil enables changed"); - Require(state.depth->htileAddress == 0, "an HTILE surface without DB_HTILE_DATA_BASE decoded an HTILE address"); + queue.context.erase(0x005); + Require(AgcDriver::Graphics::DecodeState(queue).depth->htileAddress == 0, "an HTILE surface without DB_HTILE_DATA_BASE decoded an HTILE address"); queue.context[0x005] = 0x300; queue.context[0x01e] = 0x01; log.clear(); diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClear.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClear.cpp new file mode 100644 index 000000000..159eb8223 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClear.cpp @@ -0,0 +1,49 @@ +#include "prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp" +#include "DepthFastClearHarness.hpp" +#include "VulkanTestDevice.hpp" +#include +#include +#include + +namespace { + +using DepthFastClearHarness::Compile; +using DepthFastClearHarness::Covered; +using DepthFastClearHarness::Draw; +using DepthFastClearHarness::Height; +using DepthFastClearHarness::HtileBytes; +using DepthFastClearHarness::Require; +using DepthFastClearHarness::Surface; +using DepthFastClearHarness::Width; + +alignas(4096) std::array Depth{}; +alignas(256) std::array Htile{}; + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + Htile.fill(0xffffffffu); + const auto shaders = Compile(*device); + const Surface surface{reinterpret_cast(Depth.data()), 0, reinterpret_cast(Htile.data()), false, VK_FORMAT_D32_SFLOAT}; + const auto htile = surface.htile; + Require(Draw(*device, shaders, surface, {.depthWrite = true}) == Covered, "a new depth surface was not cleared to DB_DEPTH_CLEAR (1.0)"); + Require(Draw(*device, shaders, surface, {}) == 0u, "a draw at depth 0.5 passed LESS against the 0.5 written before it"); + AgcDriver::Graphics::NoteDepthMetadataFill(htile, HtileBytes - 4u, 0u); + Require(Draw(*device, shaders, surface, {}) == 0u, "a fill shorter than the HTILE cleared the depth"); + AgcDriver::Graphics::NoteDepthMetadataFill(htile, HtileBytes, 0xffffffffu); + Require(Draw(*device, shaders, surface, {}) == 0u, "an expanded HTILE fill (ZMask 0xf) cleared the depth"); + AgcDriver::Graphics::NoteDepthMetadataFill(htile, HtileBytes, 0u); + Require(Draw(*device, shaders, surface, {.depthWrite = true}) == Covered, "a fill of ZMask 0 over the whole HTILE did not clear the depth to DB_DEPTH_CLEAR (1.0)"); + Require(Draw(*device, shaders, surface, {}) == 0u, "the depth written after a fast clear was lost"); + AgcDriver::Graphics::NoteDepthMetadataFill(htile, HtileBytes, 0u); + Require(Draw(*device, shaders, surface, {}) == Covered, "a second whole-HTILE fill of ZMask 0 did not clear the depth again"); + std::puts("depth fast clear tests passed"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearHarness.hpp b/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearHarness.hpp new file mode 100644 index 000000000..e8c94851b --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearHarness.hpp @@ -0,0 +1,146 @@ +#ifndef CORE_LIBS_PRX_LIBSCEAGCDRIVER_TESTS_EXECUTION_DEPTHFASTCLEARHARNESS_HPP +#define CORE_LIBS_PRX_LIBSCEAGCDRIVER_TESTS_EXECUTION_DEPTHFASTCLEARHARNESS_HPP + +#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" +#include "Recompiler.hpp" +#include +#include +#include +#include +#include +#include +#include + +namespace DepthFastClearHarness { + +using AgcDriver::Graphics::Require; +using ShaderRecompiler::ShaderStage; + +inline constexpr std::uint32_t Width = 64; +inline constexpr std::uint32_t Height = 32; +inline constexpr std::uint32_t Covered = Width * Height; +inline constexpr std::size_t HtileBytes = ((Width + 7u) / 8u) * ((Height + 7u) / 8u) * 4u; + +alignas(256) inline std::array Pixels{}; + +alignas(256) inline constexpr std::array VertexCode{ + 0xe0382000, 0x80000005, 0xbf8c3f70, 0xf80008cf, 0x03020100, 0xbf810000, +}; + +alignas(256) inline constexpr std::array PixelCode{ + 0x7e0002f2, 0xf800180f, 0x00000000, 0xbf810000, +}; + +struct Surface { + std::uint64_t depth; + std::uint64_t stencil; + std::uint64_t htile; + bool htileStencil; + VkFormat format; +}; + +struct DrawOptions { + bool depthTest = true; + bool depthWrite = false; + VkCompareOp depthCompare = VK_COMPARE_OP_LESS; + bool stencilTest = false; + VkStencilOpState stencil{}; +}; + +inline std::array BufferDescriptor(const void* data, std::uint32_t stride, std::uint32_t count) { + const auto address = reinterpret_cast(data); + return {static_cast(address), static_cast((address >> 32u) & 0xffffu) | (stride << 16u), count, 0x01016facu}; +} + +inline const std::vector>& Triangles() { + static const std::vector> vertices{ + {-1.0f, -1.0f, 0.5f, 1.0f}, {3.0f, -1.0f, 0.5f, 1.0f}, {-1.0f, 3.0f, 0.5f, 1.0f}, + }; + return vertices; +} + +struct Shaders { + ShaderRecompiler::RecompileResult vertex; + ShaderRecompiler::RecompileResult pixel; + std::uint32_t vertexPush = 0; +}; + +inline Shaders Compile(AgcDriver::VulkanDevice& device) { + const auto target = device.Target(); + Shaders shaders; + std::vector vertexUserData(4, 0u); + const auto vertexBuffer = BufferDescriptor(Triangles().data(), 16u, static_cast(Triangles().size())); + std::copy(vertexBuffer.begin(), vertexBuffer.end(), vertexUserData.begin()); + const std::array vertexMemory{{{reinterpret_cast(VertexCode.data()), std::as_bytes(std::span(VertexCode))}}}; + ShaderRecompiler::RecompileRequest vertex{ + {ShaderStage::Vertex, reinterpret_cast(VertexCode.data()), VertexCode, 0, {}}, + {32, 0, vertexUserData, std::nullopt, std::nullopt, ShaderRecompiler::ShaderVertexStageInfo{}, vertexMemory}, + target, + {0, 0, 0, 64} + }; + vertex.useCache = false; + shaders.vertex = ShaderRecompiler::Recompile(vertex); + shaders.vertexPush = static_cast(shaders.vertex.pushConstants.size()); + + ShaderRecompiler::ShaderPixelStageInfo pixel{}; + pixel.wave32 = true; + pixel.inputAddr = ShaderRecompiler::PixelInputBit(ShaderRecompiler::PixelInput::PositionX) | ShaderRecompiler::PixelInputBit(ShaderRecompiler::PixelInput::PositionY); + pixel.posX = true; + pixel.posY = true; + pixel.targetOutputMode[0] = 9; + pixel.targetExportMapping.fill(0xe4u); + std::vector pixelUserData(4, 0u); + const std::array pixelMemory{{{reinterpret_cast(PixelCode.data()), std::as_bytes(std::span(PixelCode))}}}; + ShaderRecompiler::RecompileRequest fragment{ + {ShaderStage::Fragment, reinterpret_cast(PixelCode.data()), PixelCode, 0, {}}, + {32, 0, pixelUserData, std::nullopt, pixel, std::nullopt, pixelMemory}, + target, + {0, 0, shaders.vertexPush, 128 - shaders.vertexPush} + }; + fragment.useCache = false; + shaders.pixel = ShaderRecompiler::Recompile(fragment); + return shaders; +} + +inline std::uint32_t Draw(AgcDriver::VulkanDevice& device, const Shaders& compiled, const Surface& surface, const DrawOptions& options) { + Pixels.fill(std::byte{0}); + const std::array shaders{{ + {ShaderStage::Vertex, &compiled.vertex, 0}, + {ShaderStage::Fragment, &compiled.pixel, compiled.vertexPush} + }}; + AgcDriver::Graphics::State state{}; + state.stages = {AgcDriver::Graphics::ShaderPath::Vertex, 0u, 32u, 32u, std::nullopt, std::nullopt}; + state.color = {reinterpret_cast(Pixels.data()), {Width, Height}, VK_FORMAT_R8G8B8A8_UNORM, Pixels.size(), 0xe4u}; + state.colors = {state.color}; + state.hasColorTarget = true; + state.depth = AgcDriver::Graphics::DepthTarget{surface.depth, surface.stencil, {Width, Height}, surface.format, 1.0f, 0, surface.htile, surface.htileStencil}; + state.depthTest = options.depthTest; + state.depthWrite = options.depthWrite; + state.depthCompare = options.depthCompare; + state.stencilTest = options.stencilTest; + state.stencilFront = options.stencil; + state.stencilBack = options.stencil; + state.renderExtent = {Width, Height}; + state.topology = VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST; + state.viewport = {0, static_cast(Height), static_cast(Width), -static_cast(Height), 0, 1}; + state.negativeOneToOne = false; + state.scissor = {{0, 0}, {Width, Height}}; + state.cullMode = VK_CULL_MODE_NONE; + state.frontFace = VK_FRONT_FACE_COUNTER_CLOCKWISE; + state.blend.colorWriteMask = 15; + state.blends = {state.blend}; + state.blendConstants = {}; + const AgcDriver::Pm4::DrawParameters draw{0, static_cast(Triangles().size()), 0, 1, 0, false}; + device.Draw(state, draw, shaders); + device.WaitIdle(); + std::uint32_t covered = 0; + for (std::size_t pixel = 0; pixel < Pixels.size() / 4u; ++pixel) { + if (std::to_integer(Pixels[pixel * 4u]) == 255u) ++covered; + } + return covered; +} + +} + +#endif diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearSubmit.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearSubmit.cpp new file mode 100644 index 000000000..a15f2d1e9 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearSubmit.cpp @@ -0,0 +1,181 @@ +#include "prx/libSceAgcDriver/Execution/include/Driver.hpp" +#include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp" +#include "prx/libSceAgcDriver/Submit/include/Dcb.hpp" +#include "prx/libc/include/GuestAllocations.hpp" +#include "prx/libc/include/Shutdown.hpp" +#include "DepthFastClearHarness.hpp" +#include "VulkanTestDevice.hpp" +#ifdef _WIN32 +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include +#endif +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using DepthFastClearHarness::Compile; +using DepthFastClearHarness::Covered; +using DepthFastClearHarness::Draw; +using DepthFastClearHarness::HtileBytes; +using DepthFastClearHarness::Require; +using DepthFastClearHarness::Shaders; +using DepthFastClearHarness::Surface; + +constexpr auto HtileFillBytes = static_cast(HtileBytes); + +class GuestBlock { +public: + GuestBlock() { + constexpr std::size_t bytes = 65536; +#ifdef _WIN32 + block = static_cast(VirtualAlloc(nullptr, bytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE)); +#else + block = static_cast(std::aligned_alloc(bytes, bytes)); +#endif + Require(block != nullptr, "depth fast clear submit: cannot allocate a guest block"); + std::memset(block, 0, bytes); + GuestAllocations::Mutation().Add(block, bytes, true, true); + } + + GuestBlock(const GuestBlock&) = delete; + GuestBlock& operator=(const GuestBlock&) = delete; + + std::uint8_t* Data() { return block; } + + std::uint64_t Address() const { return static_cast(reinterpret_cast(block)); } + +private: + std::uint8_t* block = nullptr; +}; + +GuestBlock& NewBlock() { + static auto* blocks = new std::deque(); + return blocks->emplace_back(); +} + +std::uint32_t Low(std::uint64_t value) { return static_cast(value); } + +std::uint32_t High(std::uint64_t value) { return static_cast(value >> 32u); } + +std::vector Pm4Packet(std::uint32_t opcode, std::initializer_list body) { + std::vector words{0xc0000000u | (static_cast(body.size() - 1) << 16u) | (opcode << 8u)}; + words.insert(words.end(), body); + return words; +} + +void Submit(std::initializer_list> packets) { + std::vector words; + for (const auto& packet : packets) words.insert(words.end(), packet.begin(), packet.end()); + Packet dcb{words.data(), static_cast(words.size()), 0, {}}; + Require(sceAgcDriverSubmitDcb(&dcb) == 0, "depth fast clear submit: DCB submission failed"); + AgcDriverWaitIdle_nid_postfix(); +} + +void DmaFill(const GuestBlock& htile, std::uint32_t bytes, std::uint32_t pattern) { + Submit({Pm4Packet(0x50, {0x40000000u, pattern, 0, Low(htile.Address()), High(htile.Address()), bytes})}); +} + +constexpr std::array FillKernel{0xd7460004u, 0x04010c08u, 0x7e000204u, 0x7e020205u, 0x7e040206u, 0x7e060207u, 0xe01c2000u, 0x80000004u, 0xbf810000u}; + +void DispatchFill(const GuestBlock& code, const GuestBlock& htile, std::uint32_t records) { + const auto kernel = code.Address(); + const auto base = htile.Address(); + Submit({ + Pm4Packet(0x76, {0x20c, Low(kernel >> 8u)}), + Pm4Packet(0x76, {0x20d, High(kernel >> 8u) & 0xffu}), + Pm4Packet(0x76, {0x207, 64}), + Pm4Packet(0x76, {0x208, 1}), + Pm4Packet(0x76, {0x209, 1}), + Pm4Packet(0x76, {0x213, 8u << 1u}), + Pm4Packet(0x76, {0x240, Low(base)}), + Pm4Packet(0x76, {0x241, (High(base) & 0xffffu) | (16u << 16u)}), + Pm4Packet(0x76, {0x242, records}), + Pm4Packet(0x76, {0x243, 0x4bfacu}), + Pm4Packet(0x76, {0x244, 0}), + Pm4Packet(0x76, {0x245, 0}), + Pm4Packet(0x76, {0x246, 0}), + Pm4Packet(0x76, {0x247, 0}), + Pm4Packet(0x15, {1, 1, 1, 0x41}), + }); +} + +bool StoresFills(AgcDriver::VulkanDevice& device, const GuestBlock& probe) { + std::lock_guard lock(AgcDriver::GuestMemory::GpuMutex()); + std::array pattern{}; + const bool stored = device.FillBuffer(probe.Address(), 4096, pattern); + device.WaitIdle(); + return stored; +} + +void DmaFillClearsDepth(AgcDriver::VulkanDevice& device, const Shaders& shaders) { + const auto& htile = NewBlock(); + const Surface surface{NewBlock().Address(), 0, htile.Address(), false, VK_FORMAT_D32_SFLOAT}; + Require(Draw(device, shaders, surface, {.depthWrite = true}) == Covered, "a new depth surface was not cleared to DB_DEPTH_CLEAR (1.0)"); + Require(Draw(device, shaders, surface, {}) == 0u, "a draw at depth 0.5 passed LESS against the 0.5 written before it"); + DmaFill(htile, HtileFillBytes - 4u, 0u); + Require(Draw(device, shaders, surface, {}) == 0u, "a DMA_DATA fill shorter than the HTILE cleared the depth"); + DmaFill(htile, HtileFillBytes, 0xffffffffu); + Require(Draw(device, shaders, surface, {}) == 0u, "a DMA_DATA fill expanded to ZMask 0xf cleared the depth"); + DmaFill(htile, HtileFillBytes, 0u); + Require(Draw(device, shaders, surface, {.depthWrite = true}) == Covered, "a DMA_DATA fill of 0 over the HTILE did not clear the depth to DB_DEPTH_CLEAR (1.0)"); + Require(Draw(device, shaders, surface, {}) == 0u, "the depth written after the DMA_DATA clear was lost"); +} + +void DispatchFillClearsDepth(AgcDriver::VulkanDevice& device, const Shaders& shaders) { + const auto& htile = NewBlock(); + auto& code = NewBlock(); + std::memcpy(code.Data(), FillKernel.data(), sizeof(FillKernel)); + const Surface surface{NewBlock().Address(), 0, htile.Address(), false, VK_FORMAT_D32_SFLOAT}; + Require(Draw(device, shaders, surface, {.depthWrite = true}) == Covered, "a new depth surface was not cleared to DB_DEPTH_CLEAR (1.0)"); + Require(Draw(device, shaders, surface, {}) == 0u, "a draw at depth 0.5 passed LESS against the 0.5 written before it"); + DispatchFill(code, htile, HtileFillBytes / 16u); + Require(Draw(device, shaders, surface, {.depthWrite = true}) == Covered, "a compute fill of 0 over the HTILE did not clear the depth to DB_DEPTH_CLEAR (1.0)"); + Require(Draw(device, shaders, surface, {}) == 0u, "the depth written after the compute fill clear was lost"); +} + +void DepthOnlyFillKeepsStencil(AgcDriver::VulkanDevice& device, const Shaders& shaders) { + const auto& htile = NewBlock(); + const Surface surface{NewBlock().Address(), NewBlock().Address(), htile.Address(), true, VK_FORMAT_D32_SFLOAT_S8_UINT}; + const VkStencilOpState writeFive{VK_STENCIL_OP_KEEP, VK_STENCIL_OP_REPLACE, VK_STENCIL_OP_KEEP, VK_COMPARE_OP_ALWAYS, 0xffu, 0xffu, 5u}; + const VkStencilOpState equalFive{VK_STENCIL_OP_KEEP, VK_STENCIL_OP_KEEP, VK_STENCIL_OP_KEEP, VK_COMPARE_OP_EQUAL, 0xffu, 0u, 5u}; + const VkStencilOpState equalZero{VK_STENCIL_OP_KEEP, VK_STENCIL_OP_KEEP, VK_STENCIL_OP_KEEP, VK_COMPARE_OP_EQUAL, 0xffu, 0u, 0u}; + Require(Draw(device, shaders, surface, {.depthWrite = true, .stencilTest = true, .stencil = writeFive}) == Covered, "the stencil write over a new depth-stencil surface did not cover the target"); + DmaFill(htile, HtileFillBytes, 0x3f0u); + Require(Draw(device, shaders, surface, {.depthTest = false, .stencilTest = true, .stencil = equalFive}) == Covered, "a depth-only fill (SMem 3) cleared the stencil"); + Require(Draw(device, shaders, surface, {}) == Covered, "a depth-only fill did not clear the depth to DB_DEPTH_CLEAR (1.0)"); + DmaFill(htile, HtileFillBytes, 0xf0u); + Require(Draw(device, shaders, surface, {.depthTest = false, .stencilTest = true, .stencil = equalZero}) == Covered, "a depth and stencil fast clear (SMem 0) did not clear the stencil to 0"); +} + +} + +int main() { + try { + const auto device = OpenVulkanTestDevice(); + if (!device) return VulkanTestSkipped; + const auto shaders = Compile(*device); + DmaFillClearsDepth(*device, shaders); + DepthOnlyFillKeepsStencil(*device, shaders); + const bool computeRan = StoresFills(*device, NewBlock()); + if (computeRan) DispatchFillClearsDepth(*device, shaders); + else std::printf("compute fill not run: the device does not store fills into imported guest memory\n"); + LibcRunShutdown_nid_postfix(); + std::puts(computeRan ? "depth fast clear submit tests passed" : "depth fast clear submit tests passed without the compute fill"); + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + try { LibcRunShutdown_nid_postfix(); } catch (...) {} + return 1; + } +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 542edd980..69a5eb358 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -314,6 +314,10 @@ Throughout the project, every function at every stage either **does exactly what - [sceAgcDriverRegisterMultipleResources](../../core/libs/prx/libSceAgcDriver/Resource/src/Registration.cpp) uses the existing unavailable-registration state and leaves caller memory untouched. Its four arguments (uint32 handle output, uint32 owner, descriptor pointer, uint32 count) follow the original PPSA34502 / 01.010.000 codzm call. The caller builds 40-byte descriptors and explicitly accepts 0x8A6C9018 with an invalid output handle. Enabled registration and console argument-validation precedence are unverified. - [Shader registration](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp) (libSceAgcDriver) - a shader header is accepted at any address: PPSA26344 passes headers 4 and 1 bytes past an 8-byte boundary; whether the console refuses an alignment is unknown - [Shader header copies](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp) (libSceAgcDriver) - a header bound for a draw or dispatch at another address than the registered one is accepted when its fixed fields match the registered header except `user_data`; the registered header's metadata is used. PPSA21567 binds such a copy of a vertex shader header with `user_data` set to null. Whether the console reads the copy's `user_data` is unknown +- [HtileFillCovers](../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver) - a fill is taken to cover a depth surface's whole HTILE when it spans `ceil(width / 8) * ceil(height / 8) * 4` bytes from DB_HTILE_DATA_BASE. The console's gfx10 HTILE is pipe-aligned and larger (addrlib), so a fill of only its first part would clear every tile; not measured. +- [Depth fast clear through HTILE](../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver) - a surface cleared by an HTILE fill is cleared to DB_DEPTH_CLEAR, also when it is next sampled. For a depth-only HTILE the clear word also carries a value (radv writes `minz = maxz = round(depth * 0x3fff)`, `HTILE_Z_CLEAR_REG` in Mesa's `src/amd/common/ac_descriptors.h`, and allows TC-compatible fast clears only to 0.0 or 1.0); which of the two the texture unit returns for a cleared tile was not measured. +- [Depth-only and stencil HTILE fill](../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver) - in a whole-HTILE fill, ZMask (bits 0-3) 0 clears the depth and SMem (bits 8-9) 0 clears the stencil, while SMem 3 leaves the stencil alone (`HtileFillClears`); other SMem values are taken to leave it alone too. The bit meanings follow the gfx10 HTILE layout as the code reads it, not a measurement; `agc_driver_depth_fast_clear_submit` pins them for the paths the Driver takes. +- [HTILE address of a depth array slice](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) - `decodeDepth` adds the `DB_DEPTH_VIEW` slice offset to the Z and stencil addresses but not to `htileAddress`, so every slice of a layered depth target shares slice 0's HTILE address and a fill of slice 0's HTILE marks all slices cleared. Astro's Playroom fills the HTILE of each of its three shadow cascades 0x40000 apart (0x24000 bytes of tiles each) back to back before drawing, so the result is still right there; a title that clears one slice between draws to another would lose depth. The per-slice HTILE stride is not derived yet. - [sceNgs2VoiceControl sampler filter](../../core/libs/prx/libSceNgs2.native/src/Voice.cpp) (libSceNgs2.native) - param `0x1000000a`, its layout and meaning follow KytyPS5 only: location 1 type 1 is an RBJ low-pass biquad at the system sample rate after resampling, scaled by `level`, a set `channel_mask` bit bypasses that channel, type 0 turns the filter off. Other locations and types throw. The voice keeps playing a filter tail after its last block until the history is zero, with outputs below 1e-20 flushed to zero (a KytyPS5 choice); the console's tail length is unknown - [sceNgs2PanInit / sceNgs2PanGetVolumeMatrix](../../core/libs/prx/libSceNgs2.native/src/Pan.cpp) (libSceNgs2.native) - follow the PS4 firmware 11.00 `libSceNgs2.sprx`, read and checked against its output for 2, 4, 5 and 7 speakers into 1, 2, 6 and 8 channels; not yet confirmed on the PS5 library. Speakers are taken in the order L R, L R Ls Rs, L R C Ls Rs or L R C Ls Rs Lb Rb and kept in a fixed circular order, not sorted; without angles the defaults are 90/270, 40/110/250/320, 0/40/110/250/320 and 0/40/110/150/210/250/320 degrees. As in that library, defaults converted to radians keep the wrap-around entry in degrees, so angles past the last speaker pan almost fully to it. Error codes are not known for the PS5, so invalid arguments throw, including a speaker angle equal to the unit angle, on which that library loops forever. Angles so large that adding or subtracting a turn leaves them unchanged throw for the same reason - [sceRtcGetDosTime, sceRtcSetDosTime](../../core/libs/prx/libSceRtc/Export.cpp) (libSceRtc) - bit layout from shadPS4 (PS4, the standard FAT date/time). A year outside 1980-2107 makes `GetDosTime` return `INVALID_YEAR` after writing 0 for an earlier one and `0xff9fbf7d` (2107-12-31 23:59:58, the largest encodable time) for a later one, and `SetDosTime` decodes the year as the unsigned 7-bit FAT field (1980-2107) without validating the fields, as the disassembly-derived [fpPS4](https://github.com/red-prig/fpPS4) reimplementation of the PS4 library does; not confirmed on a PS5 title, and shadPS4 instead masks out-of-range years into the packed value and sign-extends the 16-bit date From 30a0545b21510d949c4925d4894e4f5074b463ca Mon Sep 17 00:00:00 2001 From: Michon <231849@buas.nl> Date: Sat, 10 Oct 2026 10:10:14 +0200 Subject: [PATCH 444/460] fix(agc): a depth surface's clears record on its own device `Recorder::Active()` is process-wide. With two devices (agc_driver_depth_fast_clear_submit draws on its own device; its compute-fill DCB makes the driver create a second one, whose recorder becomes the active one) a surface's fast clear was recorded into the other device's command buffer, unordered with the draw: the compute-fill case failed about once in 60-90 runs. A surface now records into the active recorder only when that recorder is its own device's, and submits on its own device otherwise. Co-Authored-By: Claude Opus 5.5 (1M context) --- .../prx/libSceAgcDriver/Graphics/include/Recorder.hpp | 1 + .../prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp | 9 +++++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp index 1f042f2ca..7a863048d 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp @@ -383,6 +383,7 @@ public: // The recorder of the current device, for code that only has guest addresses (the flush hook) // and for helpers that must not recycle resources the recorded work still uses. static Recorder* Active(); + VkDevice Device() const { return context.device; } // Whether the calling thread runs inside a batch's completion action (finish()): a store made // there must not have a publish recorded for it (UnitShadow), see SyncThrough. static bool InCompletion(); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp index d8d6929a3..9fefda326 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp @@ -17,6 +17,11 @@ namespace AgcDriver::Graphics { namespace { +Recorder* RecorderFor(const Context& context) { + auto* recorder = Recorder::Active(); + return recorder != nullptr && recorder->Device() == context.device ? recorder : nullptr; +} + class DepthSurface { public: DepthSurface(const Context& context, const DepthTarget& target) : context(context), target(target) { @@ -53,7 +58,7 @@ public: viewInfo.format = target.format; viewInfo.subresourceRange = {aspects, 0, 1, 0, 1}; Check(context.Function("vkCreateImageView")(context.device, &viewInfo, nullptr, &view), "vkCreateImageView depth"); - auto* recorder = Recorder::Active(); + auto* recorder = RecorderFor(context); std::unique_ptr batch; if (recorder == nullptr) batch = std::make_unique(context); const auto commands = recorder != nullptr ? recorder->Commands() : batch->Handle(); @@ -86,7 +91,7 @@ public: const auto aspects = pendingClear & (VK_IMAGE_ASPECT_DEPTH_BIT | (target.stencilAddress != 0 ? VK_IMAGE_ASPECT_STENCIL_BIT : 0u)); pendingClear = 0; if (aspects == 0) return; - auto* recorder = Recorder::Active(); + auto* recorder = RecorderFor(context); std::unique_ptr batch; if (recorder == nullptr) batch = std::make_unique(context); const auto commands = recorder != nullptr ? recorder->Commands() : batch->Handle(); From ce65ab251cd3f149f1535f7c0ba1b6952c8f85c8 Mon Sep 17 00:00:00 2001 From: Michon <231849@buas.nl> Date: Sat, 10 Oct 2026 11:51:39 +0200 Subject: [PATCH 445/460] test(agc): the fast clear tests on main's Mutation::Add gpu flag; one DepthSurface include Co-Authored-By: Claude Opus 5.5 (1M context) --- core/libs/prx/libSceAgcDriver/tests/Graphics.cpp | 1 - .../libSceAgcDriver/tests/execution/DepthFastClearSubmit.cpp | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index ce902f814..834a29628 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -7,7 +7,6 @@ #include "prx/libSceAgcDriver/Graphics/include/VertexInput.hpp" #include "prx/libSceAgcDriver/Graphics/include/Draw.hpp" #include "prx/libSceAgcDriver/Graphics/include/BufferPool.hpp" -#include "prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp" #include "prx/libSceAgcDriver/Graphics/include/ShaderInputState.hpp" #include "prx/libSceAgcDriver/Graphics/include/DccMetadata.hpp" #include "prx/libSceAgcDriver/Graphics/include/ParallelCompare.hpp" diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearSubmit.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearSubmit.cpp index a15f2d1e9..674317831 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearSubmit.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearSubmit.cpp @@ -45,7 +45,7 @@ public: #endif Require(block != nullptr, "depth fast clear submit: cannot allocate a guest block"); std::memset(block, 0, bytes); - GuestAllocations::Mutation().Add(block, bytes, true, true); + GuestAllocations::Mutation().Add(block, bytes, true, true, true); } GuestBlock(const GuestBlock&) = delete; From 56ea14f44c4a600681598bffca80f28a40f6828a Mon Sep 17 00:00:00 2001 From: Michon <231849@buas.nl> Date: Sun, 11 Oct 2026 09:49:53 +0200 Subject: [PATCH 446/460] fix(agc): an HTILE fill covers a depth surface only over its whole pipe-aligned metadata A fill was taken to cover the surface when it spanned ceil(w/8)*ceil(h/8)*4 bytes, a linear HTILE layout. In the pipe-aligned SW_64KB_Z_X layout a surface's HTILE words lie across a 32 KiB metadata block per 1024x512 pixels (AMD PAL gfx10 addrlib: a 16x16 surface has a valid word at offset 260), so a fill of the linear size cleared tiles it never wrote. HtileMetadataBytes gives that extent; the fast clear tests fill it (the compute fixture now dispatches one group per KiB with TGID_X). Reported with a PAL reproducer by @Amoenus. Co-Authored-By: Amoenus <5391396+Amoenus@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 (1M context) --- .../libSceAgcDriver/Graphics/include/DepthSurface.hpp | 1 + .../prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp | 8 ++++++-- core/libs/prx/libSceAgcDriver/tests/Graphics.cpp | 8 ++++++-- .../tests/execution/DepthFastClearHarness.hpp | 2 +- .../tests/execution/DepthFastClearSubmit.cpp | 10 +++++----- docs/dev/TechnicalDebt.md | 2 +- 6 files changed, 20 insertions(+), 11 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp index aa7515e90..f5b6cad4a 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp @@ -21,6 +21,7 @@ void NoteHtileDepthClear(std::uint64_t htileAddress); bool DepthSurfaceHolds(const Context& context, const GuestTextureResource& resource, const Texture* texture); void NoteDepthMetadataFill(std::uint64_t address, std::size_t bytes, std::uint32_t pattern); VkImageAspectFlags HtileFillClears(std::uint32_t pattern, bool stencilInHtile); +std::uint64_t HtileMetadataBytes(VkExtent2D extent); bool HtileFillCovers(std::uint64_t htile, VkExtent2D extent, std::uint64_t address, std::size_t bytes); std::shared_ptr DepthSurfaceTexture(const Context& context, std::span words, const GuestTextureResource& resource, VkComponentMapping components); diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp index 9fefda326..b29e8d1f7 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp @@ -263,10 +263,14 @@ VkImageAspectFlags HtileFillClears(std::uint32_t pattern, bool stencilInHtile) { return cleared; } +std::uint64_t HtileMetadataBytes(VkExtent2D extent) { + const auto blocks = static_cast((extent.width + 1023u) / 1024u) * ((extent.height + 511u) / 512u); + return blocks * 32768u; +} + bool HtileFillCovers(std::uint64_t htile, VkExtent2D extent, std::uint64_t address, std::size_t bytes) { if (htile == 0 || htile < address || htile >= address + bytes) return false; - const auto tiles = static_cast((extent.width + 7u) / 8u) * ((extent.height + 7u) / 8u); - return address + bytes - htile >= tiles * 4u; + return address + bytes - htile >= HtileMetadataBytes(extent); } void NoteDepthMetadataFill(std::uint64_t address, std::size_t bytes, std::uint32_t pattern) { diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index 834a29628..3c5a1c1c2 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -917,8 +917,12 @@ void DepthStencilTests() { Require(HtileFillClears(0xfffff0ffu, true) == VK_IMAGE_ASPECT_STENCIL_BIT, "ZMask 0xf with SMem 0 clears only the stencil"); Require(HtileFillClears(0xffffffffu, true) == 0 && HtileFillClears(0xffffffffu, false) == 0, "an expanded fill clears nothing"); const VkExtent2D extent{16, 8}; - Require(HtileFillCovers(0x1000, extent, 0x1000, 8) && HtileFillCovers(0x1000, extent, 0xff0, 0x20), "a fill over every HTILE word covers the surface"); - Require(!HtileFillCovers(0x1000, extent, 0x1000, 4) && !HtileFillCovers(0x1000, extent, 0x1004, 8) && !HtileFillCovers(0, extent, 0, 64), "a partial fill, one starting past the HTILE base, or no HTILE covers nothing"); + Require(HtileFillCovers(0x10000, extent, 0x10000, 0x8000) && HtileFillCovers(0x10000, extent, 0xfff0, 0x8010), "a fill over the whole 32 KiB metadata block covers the surface"); + Require(!HtileFillCovers(0x10000, extent, 0x10000, 0x7ffc) && !HtileFillCovers(0x10000, extent, 0x10004, 0x8000) && !HtileFillCovers(0, extent, 0, 0x10000), "a partial fill, one starting past the HTILE base, or no HTILE covers nothing"); + // Pipe-aligned SW_64KB_Z_X places a 16x16 surface's HTILE words across its metadata block (a valid word at offset + // 260, AMD PAL), so a fill of the words a linear layout would hold covers nothing. + Require(!HtileFillCovers(0x10000, {16, 16}, 0x10000, 16) && !HtileFillCovers(0x10000, {16, 16}, 0x10000, 264), "a fill of the linear HTILE size of a 16x16 surface misses its pipe-aligned words"); + Require(AgcDriver::Graphics::HtileMetadataBytes({1024, 512}) == 0x8000u && AgcDriver::Graphics::HtileMetadataBytes({1025, 512}) == 0x10000u && AgcDriver::Graphics::HtileMetadataBytes({3840, 2160}) == 20u * 0x8000u, "the HTILE metadata is 32 KiB per 1024x512 block"); } Require(state.renderExtent.width == 4 && state.renderExtent.height == 2, "render extent ignores the depth surface"); Require(!state.depthTest && !state.depthWrite && state.stencilTest, "depth/stencil enables changed"); diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearHarness.hpp b/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearHarness.hpp index e8c94851b..18d3083b4 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearHarness.hpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearHarness.hpp @@ -20,7 +20,7 @@ using ShaderRecompiler::ShaderStage; inline constexpr std::uint32_t Width = 64; inline constexpr std::uint32_t Height = 32; inline constexpr std::uint32_t Covered = Width * Height; -inline constexpr std::size_t HtileBytes = ((Width + 7u) / 8u) * ((Height + 7u) / 8u) * 4u; +inline constexpr std::size_t HtileBytes = ((Width + 1023u) / 1024u) * ((Height + 511u) / 512u) * 32768u; alignas(256) inline std::array Pixels{}; diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearSubmit.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearSubmit.cpp index 674317831..6f43c7492 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearSubmit.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/DepthFastClearSubmit.cpp @@ -88,7 +88,7 @@ void DmaFill(const GuestBlock& htile, std::uint32_t bytes, std::uint32_t pattern constexpr std::array FillKernel{0xd7460004u, 0x04010c08u, 0x7e000204u, 0x7e020205u, 0x7e040206u, 0x7e060207u, 0xe01c2000u, 0x80000004u, 0xbf810000u}; -void DispatchFill(const GuestBlock& code, const GuestBlock& htile, std::uint32_t records) { +void DispatchFill(const GuestBlock& code, const GuestBlock& htile, std::uint32_t bytes) { const auto kernel = code.Address(); const auto base = htile.Address(); Submit({ @@ -97,16 +97,16 @@ void DispatchFill(const GuestBlock& code, const GuestBlock& htile, std::uint32_t Pm4Packet(0x76, {0x207, 64}), Pm4Packet(0x76, {0x208, 1}), Pm4Packet(0x76, {0x209, 1}), - Pm4Packet(0x76, {0x213, 8u << 1u}), + Pm4Packet(0x76, {0x213, (8u << 1u) | 0x80u}), Pm4Packet(0x76, {0x240, Low(base)}), Pm4Packet(0x76, {0x241, (High(base) & 0xffffu) | (16u << 16u)}), - Pm4Packet(0x76, {0x242, records}), + Pm4Packet(0x76, {0x242, bytes / 16u}), Pm4Packet(0x76, {0x243, 0x4bfacu}), Pm4Packet(0x76, {0x244, 0}), Pm4Packet(0x76, {0x245, 0}), Pm4Packet(0x76, {0x246, 0}), Pm4Packet(0x76, {0x247, 0}), - Pm4Packet(0x15, {1, 1, 1, 0x41}), + Pm4Packet(0x15, {bytes / 1024u, 1, 1, 0x41}), }); } @@ -139,7 +139,7 @@ void DispatchFillClearsDepth(AgcDriver::VulkanDevice& device, const Shaders& sha const Surface surface{NewBlock().Address(), 0, htile.Address(), false, VK_FORMAT_D32_SFLOAT}; Require(Draw(device, shaders, surface, {.depthWrite = true}) == Covered, "a new depth surface was not cleared to DB_DEPTH_CLEAR (1.0)"); Require(Draw(device, shaders, surface, {}) == 0u, "a draw at depth 0.5 passed LESS against the 0.5 written before it"); - DispatchFill(code, htile, HtileFillBytes / 16u); + DispatchFill(code, htile, HtileFillBytes); Require(Draw(device, shaders, surface, {.depthWrite = true}) == Covered, "a compute fill of 0 over the HTILE did not clear the depth to DB_DEPTH_CLEAR (1.0)"); Require(Draw(device, shaders, surface, {}) == 0u, "the depth written after the compute fill clear was lost"); } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 69a5eb358..2d64af84c 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -314,7 +314,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceAgcDriverRegisterMultipleResources](../../core/libs/prx/libSceAgcDriver/Resource/src/Registration.cpp) uses the existing unavailable-registration state and leaves caller memory untouched. Its four arguments (uint32 handle output, uint32 owner, descriptor pointer, uint32 count) follow the original PPSA34502 / 01.010.000 codzm call. The caller builds 40-byte descriptors and explicitly accepts 0x8A6C9018 with an invalid output handle. Enabled registration and console argument-validation precedence are unverified. - [Shader registration](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp) (libSceAgcDriver) - a shader header is accepted at any address: PPSA26344 passes headers 4 and 1 bytes past an 8-byte boundary; whether the console refuses an alignment is unknown - [Shader header copies](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp) (libSceAgcDriver) - a header bound for a draw or dispatch at another address than the registered one is accepted when its fixed fields match the registered header except `user_data`; the registered header's metadata is used. PPSA21567 binds such a copy of a vertex shader header with `user_data` set to null. Whether the console reads the copy's `user_data` is unknown -- [HtileFillCovers](../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver) - a fill is taken to cover a depth surface's whole HTILE when it spans `ceil(width / 8) * ceil(height / 8) * 4` bytes from DB_HTILE_DATA_BASE. The console's gfx10 HTILE is pipe-aligned and larger (addrlib), so a fill of only its first part would clear every tile; not measured. +- [HtileFillCovers](../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver) - a fill is taken to cover a depth surface's whole HTILE when it spans 32 KiB per 1024x512 pixel block from DB_HTILE_DATA_BASE, the pipe-aligned SW_64KB_Z_X metadata extent AMD PAL's gfx10 addrlib gives for GB_ADDR_CONFIG 0x00100044 (1-8 samples, D16/D32); other metadata layouts are not decoded. A partial fill is ignored (the surface keeps its contents); not measured on the console. - [Depth fast clear through HTILE](../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver) - a surface cleared by an HTILE fill is cleared to DB_DEPTH_CLEAR, also when it is next sampled. For a depth-only HTILE the clear word also carries a value (radv writes `minz = maxz = round(depth * 0x3fff)`, `HTILE_Z_CLEAR_REG` in Mesa's `src/amd/common/ac_descriptors.h`, and allows TC-compatible fast clears only to 0.0 or 1.0); which of the two the texture unit returns for a cleared tile was not measured. - [Depth-only and stencil HTILE fill](../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver) - in a whole-HTILE fill, ZMask (bits 0-3) 0 clears the depth and SMem (bits 8-9) 0 clears the stencil, while SMem 3 leaves the stencil alone (`HtileFillClears`); other SMem values are taken to leave it alone too. The bit meanings follow the gfx10 HTILE layout as the code reads it, not a measurement; `agc_driver_depth_fast_clear_submit` pins them for the paths the Driver takes. - [HTILE address of a depth array slice](../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (libSceAgcDriver) - `decodeDepth` adds the `DB_DEPTH_VIEW` slice offset to the Z and stencil addresses but not to `htileAddress`, so every slice of a layered depth target shares slice 0's HTILE address and a fill of slice 0's HTILE marks all slices cleared. Astro's Playroom fills the HTILE of each of its three shadow cascades 0x40000 apart (0x24000 bytes of tiles each) back to back before drawing, so the result is still right there; a title that clears one slice between draws to another would lose depth. The per-slice HTILE stride is not derived yet. From 2806eb57a2c6a7d869c1a0f7a6c73995b021a923 Mon Sep 17 00:00:00 2001 From: overlordxrz-source Date: Sun, 11 Oct 2026 03:05:00 -0400 Subject: [PATCH 447/460] feat(libSceLibcInternal): implement getprogname --- .../prx/libSceLibcInternal/CMakeLists.txt | 10 ++++++ core/libs/prx/libSceLibcInternal/Export.cpp | 5 +++ core/libs/tests/GuestProgramName.cpp | 32 +++++++++++++++++++ 3 files changed, 47 insertions(+) create mode 100644 core/libs/tests/GuestProgramName.cpp diff --git a/core/libs/prx/libSceLibcInternal/CMakeLists.txt b/core/libs/prx/libSceLibcInternal/CMakeLists.txt index da7eb464b..a0aae315f 100644 --- a/core/libs/prx/libSceLibcInternal/CMakeLists.txt +++ b/core/libs/prx/libSceLibcInternal/CMakeLists.txt @@ -19,6 +19,16 @@ if(BUILD_TESTING) set_tests_properties(guest_utime PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_program_name_tests ${LIBS_INCLUDE_DIR}/tests/GuestProgramName.cpp) + target_include_directories(guest_program_name_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_program_name_tests PRIVATE libSceLibcInternal libkernel libc) + configure_windows_unwind(guest_program_name_tests) + add_test(NAME guest_program_name COMMAND guest_program_name_tests) + set_tests_properties(guest_program_name PROPERTIES TIMEOUT 20) + if(WIN32) + set_tests_properties(guest_program_name PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_thread_atexit_tests ${LIBS_INCLUDE_DIR}/tests/GuestThreadAtexit.cpp) target_include_directories(guest_thread_atexit_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_thread_atexit_tests PRIVATE libSceLibcInternal libkernel libc) diff --git a/core/libs/prx/libSceLibcInternal/Export.cpp b/core/libs/prx/libSceLibcInternal/Export.cpp index 89afdbc73..567f6271c 100644 --- a/core/libs/prx/libSceLibcInternal/Export.cpp +++ b/core/libs/prx/libSceLibcInternal/Export.cpp @@ -15,6 +15,7 @@ extern "C" void APS5_VABI sceKernelSetThreadDtors(thread_dtors_func_t dtors); extern "C" int APS5_VABI sceKernelGetModuleInfoFromAddr(std::uint64_t address, int flags, ModuleInfoEx* info); +extern "C" const char* __progname_nid_postfix; namespace { @@ -104,6 +105,10 @@ extern "C" { int Need_sceLibcInternal_nid_postfix = 1; +const char* APS5_VABI getprogname_nid_postfix() { + return __progname_nid_postfix; +} + void APS5_VABI __cxa_finalize_nid_postfix(void* dsoHandle) { CxaFinalize_nid_no_patch(dsoHandle); } diff --git a/core/libs/tests/GuestProgramName.cpp b/core/libs/tests/GuestProgramName.cpp new file mode 100644 index 000000000..48be372cc --- /dev/null +++ b/core/libs/tests/GuestProgramName.cpp @@ -0,0 +1,32 @@ +#include "SceTypes.hpp" +#include +#include + +extern "C" { +const char* APS5_VABI getprogname_nid_postfix(); +extern const char* __progname_nid_postfix; +int* APS5_VABI __error_nid_postfix(); +} + +static void Require(bool value) { if (!value) std::abort(); } + +int main() { + const char* const original = __progname_nid_postfix; + *__error_nid_postfix() = 45; + Require(getprogname_nid_postfix() == original); + Require(std::strcmp(getprogname_nid_postfix(), "eboot.bin") == 0); + Require(*__error_nid_postfix() == 45); + + char renamed[] = "quake2"; + __progname_nid_postfix = renamed; + Require(getprogname_nid_postfix() == renamed); + renamed[0] = 'Q'; + Require(std::strcmp(getprogname_nid_postfix(), "Quake2") == 0); + + __progname_nid_postfix = ""; + Require(getprogname_nid_postfix()[0] == '\0'); + + __progname_nid_postfix = original; + Require(getprogname_nid_postfix() == original); + Require(*__error_nid_postfix() == 45); +} From 45eb4923357b85e7ce9547568d689b8abbe0f072 Mon Sep 17 00:00:00 2001 From: Ramzi Date: Fri, 9 Oct 2026 01:05:15 +0100 Subject: [PATCH 448/460] feat(libkernel): implement pthread_getaffinity_np Report the thread's affinity mask as a 32-byte FreeBSD cpuset_t, with FreeBSD 11's errors: ERANGE for any other set size, ESRCH for a null thread and EFAULT for a null set. EVO-PLAYER (PPSA99039 v0.11.0) imports it (Jb2uGFMr688) and calls it once with a 32-byte set on pthread_self(). --- .../prx/libkernel/Pthread/Posix/Thread.cpp | 15 ++++++++++ core/libs/tests/GuestAvailableCpumask.cpp | 28 +++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp b/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp index 73111721c..13a72d1b8 100644 --- a/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp +++ b/core/libs/prx/libkernel/Pthread/Posix/Thread.cpp @@ -9,6 +9,7 @@ #include "Common.hpp" #include "prx/libkernel/KernelErrors.hpp" #include +#include #include extern "C" { @@ -28,6 +29,8 @@ int APS5_VABI scePthreadGetprio(Pthread thread, int* prio); } static constexpr int GUEST_SCHED_FIFO = 1; +static constexpr std::size_t GUEST_CPUSET_BYTES = 32; +static constexpr int GUEST_ERANGE = 34; extern "C" { int* APS5_VABI __error_nid_postfix(); @@ -116,6 +119,18 @@ int APS5_VABI pthread_getschedparam_nid_postfix(Pthread thread, int* policy, Ker return PosixThread::ToErrno(scePthreadGetprio(thread, ¶m->sched_priority)); } +int APS5_VABI pthread_getaffinity_np_nid_postfix(Pthread thread, std::size_t cpusetSize, void* cpuset) { + if (!thread) return PosixThread::GUEST_EINVAL; + if (cpusetSize != GUEST_CPUSET_BYTES) return GUEST_ERANGE; + if (!cpuset) return PosixThread::GUEST_EFAULT; + KernelCpumask mask = 0; + const int result = scePthreadGetaffinity(thread, &mask); + if (result != 0) return PosixThread::ToErrno(result); + std::memset(cpuset, 0, GUEST_CPUSET_BYTES); + std::memcpy(cpuset, &mask, sizeof(mask)); + return 0; +} + int APS5_VABI pthread_join_nid_postfix(Pthread thread, void** value) { return PosixThread::ToErrno(scePthreadJoin(thread, value)); } diff --git a/core/libs/tests/GuestAvailableCpumask.cpp b/core/libs/tests/GuestAvailableCpumask.cpp index 3e5d6b343..4a1076604 100644 --- a/core/libs/tests/GuestAvailableCpumask.cpp +++ b/core/libs/tests/GuestAvailableCpumask.cpp @@ -2,6 +2,7 @@ #include #include #include +#include #include extern "C" { @@ -18,6 +19,8 @@ int APS5_VABI cpuset_getaffinity_nid_postfix(int level, int which, std::int64_t int APS5_VABI cpuset_setaffinity_nid_postfix(int level, int which, std::int64_t id, std::size_t size, const void* mask); int APS5_VABI getpid_nid_postfix(void); int* APS5_VABI __error_nid_postfix(); +Pthread APS5_VABI pthread_self_nid_postfix(void); +int APS5_VABI pthread_getaffinity_np_nid_postfix(Pthread thread, std::size_t cpusetSize, void* cpuset); } static constexpr int SCE_OK = 0; @@ -25,6 +28,22 @@ static constexpr int PS5_LOGICAL_CPUS = 16; static void Require(bool value) { if (!value) std::abort(); } +static constexpr std::size_t GUEST_CPUSET_BYTES = 32; +static constexpr int GUEST_EINVAL = 22; +static constexpr int GUEST_EFAULT = 14; +static constexpr int GUEST_ERANGE = 34; + +static KernelCpumask ReadCpuset(Pthread thread) { + std::uint8_t cpuset[GUEST_CPUSET_BYTES + 8]; + std::memset(cpuset, 0xAB, sizeof(cpuset)); + Require(pthread_getaffinity_np_nid_postfix(thread, GUEST_CPUSET_BYTES, cpuset) == 0); + for (std::size_t i = sizeof(KernelCpumask); i < GUEST_CPUSET_BYTES; ++i) Require(cpuset[i] == 0); + for (std::size_t i = GUEST_CPUSET_BYTES; i < sizeof(cpuset); ++i) Require(cpuset[i] == 0xAB); + KernelCpumask mask = 0; + std::memcpy(&mask, cpuset, sizeof(mask)); + return mask; +} + static void* APS5_VABI Worker(void* arg) { static_cast*>(arg)->get(); return nullptr; @@ -102,6 +121,7 @@ int main() { Require(scePthreadSetaffinity(thread, cpu) == SCE_OK); Require(scePthreadGetaffinity(thread, &threadAffinity) == SCE_OK); Require(threadAffinity == cpu); + Require(ReadCpuset(thread) == cpu); } release.set_value(); Require(scePthreadJoin(thread, nullptr) == SCE_OK); @@ -143,4 +163,12 @@ int main() { hold.set_value(); Require(scePthreadJoin(bystander, nullptr) == SCE_OK); Require(scePthreadAttrDestroy(&attr) == SCE_OK); + + Require(ReadCpuset(pthread_self_nid_postfix()) == available); + std::uint8_t rawSet[GUEST_CPUSET_BYTES]{}; + for (const std::size_t size : {std::size_t{0}, std::size_t{8}, std::size_t{31}, std::size_t{33}, std::size_t{64}}) + Require(pthread_getaffinity_np_nid_postfix(pthread_self_nid_postfix(), size, rawSet) == GUEST_ERANGE); + Require(pthread_getaffinity_np_nid_postfix(nullptr, GUEST_CPUSET_BYTES, rawSet) == GUEST_EINVAL); + Require(pthread_getaffinity_np_nid_postfix(nullptr, 0, rawSet) == GUEST_EINVAL); + Require(pthread_getaffinity_np_nid_postfix(pthread_self_nid_postfix(), GUEST_CPUSET_BYTES, nullptr) == GUEST_EFAULT); } From 63b2bcb3548c9af1d23a0ef50ad751165e3ccfe3 Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sat, 10 Oct 2026 16:38:04 -0400 Subject: [PATCH 449/460] fix(shader): flush v_ldexp_f16 denormals per the f16 denormal mode v_ldexp_f16 ignored the FLOAT_MODE denorm16 flush bits on main. Flush a denormal half input when bit 0 is clear. When bit 1 is clear, flush the result when its exact value is below the minimum normal half, decided on the f32 product before the half conversion rounds it: 0x07ff scaled by 2^-1 is zero on RDNA2, not the minimum normal. NaN handling and destination high-half preservation stay the same. Co-Authored-By: Claude Opus 5.5 Co-authored-by: Cursor --- .../tests/execution/Float16Rounding.cpp | 73 +++++++++++++++++++ .../Translation/src/FloatInstructions.cpp | 18 ++++- docs/dev/TechnicalDebt.md | 1 + 3 files changed, 88 insertions(+), 4 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/execution/Float16Rounding.cpp b/core/libs/prx/libSceAgcDriver/tests/execution/Float16Rounding.cpp index 60d3ce0bf..838b17713 100644 --- a/core/libs/prx/libSceAgcDriver/tests/execution/Float16Rounding.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/execution/Float16Rounding.cpp @@ -725,6 +725,75 @@ void Check(std::uint32_t first, std::uint32_t count, std::span vec } } +alignas(256) constexpr std::array LdexpDenormalCode{ + 0x34020082u, 0x34060084u, 0xe0302000u, 0x80000401u, 0xe0302004u, 0x80000501u, 0x7e1402ffu, 0xabcdabcdu, + 0xbf8c3f70u, 0x76140b04u, 0xe0702000u, 0x80010a03u, 0xbf810000u, +}; + +struct LdexpDenormalVector { + std::uint32_t half; + std::uint32_t exponent; + std::array expected; +}; + +constexpr LdexpDenormalVector LdexpDenormalVectors[] = { + {0x00000401u, 0xffffffffu, {0xabcd0000u, 0xabcd0000u, 0xabcd0200u, 0xabcd0200u}}, + {0x000003ffu, 0x00000000u, {0xabcd0000u, 0xabcd0000u, 0xabcd0000u, 0xabcd03ffu}}, + {0x000003ffu, 0x00000001u, {0xabcd0000u, 0xabcd07feu, 0xabcd0000u, 0xabcd07feu}}, + {0x00000400u, 0xffffffffu, {0xabcd0000u, 0xabcd0000u, 0xabcd0200u, 0xabcd0200u}}, + {0x00008400u, 0xffffffffu, {0xabcd8000u, 0xabcd8000u, 0xabcd8200u, 0xabcd8200u}}, + {0x000007ffu, 0xffffffffu, {0xabcd0000u, 0xabcd0000u, 0xabcd0400u, 0xabcd0400u}}, + {0x000087ffu, 0xffffffffu, {0xabcd8000u, 0xabcd8000u, 0xabcd8400u, 0xabcd8400u}}, + {0x00000001u, 0x00000000u, {0xabcd0000u, 0xabcd0000u, 0xabcd0000u, 0xabcd0001u}}, + {0x00000001u, 0x00000001u, {0xabcd0000u, 0xabcd0000u, 0xabcd0000u, 0xabcd0002u}}, + {0x00008001u, 0x00000000u, {0xabcd8000u, 0xabcd8000u, 0xabcd8000u, 0xabcd8001u}}, + {0x00000200u, 0x00000000u, {0xabcd0000u, 0xabcd0000u, 0xabcd0000u, 0xabcd0200u}}, + {0x00000200u, 0x00000001u, {0xabcd0000u, 0xabcd0400u, 0xabcd0000u, 0xabcd0400u}}, + {0x00003c01u, 0x00000000u, {0xabcd3c01u, 0xabcd3c01u, 0xabcd3c01u, 0xabcd3c01u}}, + {0x00003c01u, 0xffffffffu, {0xabcd3801u, 0xabcd3801u, 0xabcd3801u, 0xabcd3801u}}, + {0x00007c00u, 0x00000001u, {0xabcd7c00u, 0xabcd7c00u, 0xabcd7c00u, 0xabcd7c00u}}, + {0x00007e55u, 0x00000000u, {0xabcd7e55u, 0xabcd7e55u, 0xabcd7e55u, 0xabcd7e55u}}, + {0x00000000u, 0x00000040u, {0xabcd0000u, 0xabcd0000u, 0xabcd0000u, 0xabcd0000u}}, + {0x00008000u, 0xffffffffu, {0xabcd8000u, 0xabcd8000u, 0xabcd8000u, 0xabcd8000u}}, +}; + +void RunLdexpDenormal(AgcDriver::VulkanDevice& device, std::uint32_t denorm16) { + Input.fill(0u); + for (std::uint32_t lane = 0; lane < std::size(LdexpDenormalVectors); ++lane) { + Input[lane * Inputs] = LdexpDenormalVectors[lane].half; + Input[lane * Inputs + 1] = LdexpDenormalVectors[lane].exponent; + } + Output.fill(0xdeadbeefu); + std::vector userData(8, 0u); + const auto input = BufferDescriptor(Input.data(), static_cast(Input.size())); + const auto output = BufferDescriptor(Output.data(), static_cast(Output.size())); + std::copy(input.begin(), input.end(), userData.begin()); + std::copy(output.begin(), output.end(), userData.begin() + 4); + const std::span code(LdexpDenormalCode); + const std::array memory{{{reinterpret_cast(code.data()), std::as_bytes(code)}}}; + const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0, {false, false, false}, false, 1}; + ShaderRecompiler::RecompileRequest request{ + {ShaderStage::Compute, reinterpret_cast(code.data()), code, 0, {}}, + {32, 0, userData, compute, std::nullopt, std::nullopt, memory}, + device.Target(), + {0, 0, 0, 128} + }; + request.context.floatMode = ShaderRecompiler::ShaderFloatMode{denorm16 << 6u, true, true, false}; + request.useCache = false; + const auto result = ShaderRecompiler::Recompile(request); + device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast(code.data())); + device.WaitIdle(); +} + +void CheckLdexpDenormal(std::uint32_t denorm16) { + for (std::uint32_t lane = 0; lane < std::size(LdexpDenormalVectors); ++lane) { + const auto& vector = LdexpDenormalVectors[lane]; + const auto expected = vector.expected[denorm16]; + const auto actual = Output[lane * Results]; + Require(Matches(actual, expected), std::string("f16 ldexp denormal: denorm16 ") + std::to_string(denorm16) + " vector " + std::to_string(lane) + " (" + Hex(vector.half) + ", " + Hex(vector.exponent) + ") is " + Hex(actual) + ", expected " + Hex(expected)); + } +} + } int main() { @@ -753,6 +822,10 @@ int main() { } } } + for (std::uint32_t denorm16 = 0u; denorm16 < 4u; ++denorm16) { + RunLdexpDenormal(*device, denorm16); + CheckLdexpDenormal(denorm16); + } std::puts("f16 rounding tests passed"); return 0; } catch (const std::exception& error) { diff --git a/core/shader/recompiler/Translation/src/FloatInstructions.cpp b/core/shader/recompiler/Translation/src/FloatInstructions.cpp index 3262eadda..5bad7a11e 100644 --- a/core/shader/recompiler/Translation/src/FloatInstructions.cpp +++ b/core/shader/recompiler/Translation/src/FloatInstructions.cpp @@ -324,7 +324,12 @@ IrU32 TranslationContext::normF32(IrU32 bits, bool signedValue) { } bool TranslationContext::vLdexpF16(const RdnaInstruction& inst) { - const IrU32 bits = readF16Bits(sourceAt(inst, 0u)); + const IrU32 source = readF16Bits(sourceAt(inst, 0u)); + IrU32 bits = source; + if (floatMode.has_value() && (floatMode->floatMode & 0x40u) == 0u) { + const IrU1 tiny(ir.ULessThan(ir.BitwiseAnd(bits.Value(), ir.Constant(0x7fffu)), ir.Constant(0x0400u))); + bits = IrU32(ir.Select(tiny.Value(), ir.BitwiseAnd(bits.Value(), ir.Constant(0x8000u)), bits.Value())); + } const IrF16 half(ir.Emit(IrOpcode::BitCastF16U16, IrType::F16, {&ir.Emit(IrOpcode::ConvertU16U32, IrType::U16, {&bits.Value()})})); const IrF32 value(ir.Emit(IrOpcode::ConvertF32F16, IrType::F32, {&half.Value()})); const RdnaOperand& exponentSource = sourceAt(inst, 1u); @@ -332,9 +337,14 @@ bool TranslationContext::vLdexpF16(const RdnaInstruction& inst) { const IrU32 exponent(ir.Emit(IrOpcode::BitFieldSExtract, IrType::U32, {&exponentBits.Value(), &ir.Constant(0u), &ir.Constant(16u)})); const IrU32 clamped(ir.Emit(IrOpcode::SMax32, IrType::U32, {&ir.Emit(IrOpcode::SMin32, IrType::U32, {&exponent.Value(), &ir.Constant(64u)}), &ir.Constant(static_cast(-64))})); IrValue& power = ir.BitCastF32(ir.ShiftLeftLogical(ir.IAdd(clamped.Value(), ir.Constant(127u)), ir.Constant(23u))); - const IrU32 result = convertF32ToF16Bits(IrF32(ir.Emit(IrOpcode::FPMul32, IrType::F32, {&value.Value(), &power})), {&value.Value()}); - const IrU1 nan(ir.UGreaterThan(ir.BitwiseAnd(bits.Value(), ir.Constant(0x7fffu)), ir.Constant(0x7c00u))); - write16Bits(inst.destination, clampF16Bits(inst.destination, IrU32(ir.Select(nan.Value(), quietNan16(bits).Value(), result.Value())))); + const IrF32 product(ir.Emit(IrOpcode::FPMul32, IrType::F32, {&value.Value(), &power})); + IrU32 result = convertF32ToF16Bits(product, {&value.Value()}); + if (floatMode.has_value() && (floatMode->floatMode & 0x80u) == 0u) { + const IrU1 tiny(ir.ULessThan(ir.BitwiseAnd(ir.BitCastU32(product.Value()), ir.Constant(0x7fffffffu)), ir.Constant(0x38800000u))); + result = IrU32(ir.Select(tiny.Value(), ir.BitwiseAnd(bits.Value(), ir.Constant(0x8000u)), result.Value())); + } + const IrU1 nan(ir.UGreaterThan(ir.BitwiseAnd(source.Value(), ir.Constant(0x7fffu)), ir.Constant(0x7c00u))); + write16Bits(inst.destination, clampF16Bits(inst.destination, IrU32(ir.Select(nan.Value(), quietNan16(source).Value(), result.Value())))); return true; } diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 879ccbff4..76d775e30 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -434,6 +434,7 @@ Throughout the project, every function at every stage either **does exactly what - A [label or store](../../core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp) (`RELEASE_MEM`, `WRITE_DATA`, `DMA_DATA`) into guest memory the device has no host import of waits for all recorded work and is then written by the CPU, so the queue worker stalls once per such label. The console's CP stores it in stream order without that stall. - [SET_*_REG_INDIRECT](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp) (libSceAgcDriver) reads its register list when the command buffer is submitted, with the packets. The console's graphics PFP reads it when it reaches the packet, so a list written after the submission (by the CPU, or by GPU work ahead of the packet behind PFP_SYNC_ME) is read here before that write. - The [shader float mode](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp) (`FLOAT_MODE`, `DX10_CLAMP`, `IEEE_MODE` and `FP16_OVFL` of `COMPUTE_PGM_RSRC1` and `SPI_SHADER_PGM_RSRC1_PS/GS/HS`, bit positions from Mesa's `gfx10.json` and `gfx103.json`) is decoded into `GuestContext::floatMode` and is part of the shader cache keys and the serialized request (#981). `IEEE_MODE` is read by `v_min_f32`, `v_max_f32`, `v_min3_f32`, `v_max3_f32` and `v_med3_f32`, their f16 forms, `v_min_f64` and `v_max_f64`, `IEEE_MODE` and the `FLOAT_MODE` output denormal bits by the output modifier, `DX10_CLAMP` by the clamp modifier, `FP16_OVFL` by every instruction with an f16 result (see its entry), and the whole mode by the 16-bit interpolation instructions (see their entries). With IEEE=1 a signaling NaN operand returns that NaN quieted (the first operand checked first, min3/max3 as two nested pairs, med3 as min3 when any operand is NaN), measured on RDNA2 (gfx1035) over every combination of ±0, ±1, ±inf, denormals, quiet and signaling NaNs with f32 denormals kept and flushed. IEEE=0 and unknown modes keep the previous f32 lowering, which matches IEEE=0. The f16 forms choose their NaN results on the bits: a NaN operand returns the other operand unchanged (src0 when both are NaN), and with IEEE=1 a signaling NaN operand returns that NaN quieted, in the same order as f32. `v_min_f64` and `v_max_f64` quiet a signaling NaN operand the same way with IEEE=1; with IEEE=0 they return src0 unchanged when src1 is NaN, otherwise src1 when src0 is NaN. Both were measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1 with f32 denormals kept and flushed and f16/f64 denormals kept; RDNA2 has no f64 min3, max3 or med3. Unknown modes follow IEEE=0. `v_min_f16` and `v_max_f16` flush each denormal input to its signed zero when bit 0 of the f16/f64 denormal field is clear (0 and 2); fields 1 and 3 preserve inputs, even when output flushing is enabled. Measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1, including signed zeros, NaNs and neg/abs modifiers. Unknown modes keep f16 denormals. With f16/f64 denormals flushed the hardware also flushes denormal inputs of f64 min/max, which those lowerings still do not. NaN quieting in f16 arithmetic (the unary instructions and the f32 arithmetic follow it, see below) and `FLOAT_MODE` (f32 denormals flushed before the min/max compare in Astro's Playroom's 0xc0) still follow a single mode, as the entries below record. The mode comes from the registered shader header's `sh_registers`, the same source at registration and at draw/dispatch so the prepared artifact keys match. A title that rewrites RSRC1 at run time, or a raw compute program with no header, compiles with that mode or none. +- [v_ldexp_f16](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) follows the `FLOAT_MODE` f16 denormal field: bit 0 preserves the half input and bit 1 the half result; otherwise a denormal is flushed to its signed zero before the scale, and a result whose exact (unrounded) value is below the minimum normal half is flushed to its signed zero, so `0x07ff` scaled by 2^-1 gives zero rather than rounding up to `0x0400`. Measured on RDNA2 (gfx1036) at IEEE=1 with denorm16 0/1/2/3 over the `v_ldexp_f16_controls` operand grid (328 rows per mode): denorm16 0/1/2 fixed 82 mismatched rows against the previous keep-both lowering, with no regressions. NaN and infinity handling and destination high-half preservation are unchanged. Directed `round16` applies only with f16 denormals preserved (see the directed f16 rounding entry); shaders without a known float mode keep the previous lowering. - A [WAIT_REG_MEM](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/WaitMemory.cpp) (libSceAgcDriver) that a compute queue passes on another queue's recorded label store, before the GPU has executed it, is landed before the queue's next dispatch, draw, COND_EXEC or predicated packet reads guest memory: the worker polls until memory satisfies the wait or no unfinished batch writes the label. The graphics queue (queue 0) is exempt, as landing its waits costs PPSA01341 about a quarter of its frame rate; a title that fills memory a graphics command buffer reads after it submits the buffer behind such a wait reads it early there. `APS5_LAND_WAITS_ALL_QUEUES=1` lands the graphics queue's waits too. - The [dispatches that follow a compute queue's wait](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/DispatchAhead.cpp) (libSceAgcDriver), up to the next wait that memory does not satisfy, a packet that writes guest memory other than a label, or a conditional packet, are resolved (memory captured, shader recompiled) when the wait passes, as the CP reads them right after it; the graphics queue is exempt (`APS5_GROUP_CAPTURE_ALL_QUEUES=1` includes it). The device still reads guest memory when the dispatch is issued; when that memory no longer matches the capture, the dispatch is resolved again from current memory. - [v_mullit_f32](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) follows the hardware measured with the MODE register at IEEE=1 and denormals kept (a signaling NaN src0 is quieted only with IEEE=1, as measured at IEEE=0 too; a positive denormal src2 counts as positive); with f32 denormals flushed it follows the f32 denormals entry below, and denormal modes 1 and 2 throw. From b87dd8d845c4173741d8569e01eb25e1f49cd227 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 11 Oct 2026 01:54:16 +0000 Subject: [PATCH 450/460] docs(shader): note the per-bit f16 denormal field against the f32 mode 1 and 2 refusals Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01K9MzdVqDf6wHUKzNymQSze --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 76d775e30..1ae162b5b 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -434,7 +434,7 @@ Throughout the project, every function at every stage either **does exactly what - A [label or store](../../core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp) (`RELEASE_MEM`, `WRITE_DATA`, `DMA_DATA`) into guest memory the device has no host import of waits for all recorded work and is then written by the CPU, so the queue worker stalls once per such label. The console's CP stores it in stream order without that stall. - [SET_*_REG_INDIRECT](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp) (libSceAgcDriver) reads its register list when the command buffer is submitted, with the packets. The console's graphics PFP reads it when it reaches the packet, so a list written after the submission (by the CPU, or by GPU work ahead of the packet behind PFP_SYNC_ME) is read here before that write. - The [shader float mode](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp) (`FLOAT_MODE`, `DX10_CLAMP`, `IEEE_MODE` and `FP16_OVFL` of `COMPUTE_PGM_RSRC1` and `SPI_SHADER_PGM_RSRC1_PS/GS/HS`, bit positions from Mesa's `gfx10.json` and `gfx103.json`) is decoded into `GuestContext::floatMode` and is part of the shader cache keys and the serialized request (#981). `IEEE_MODE` is read by `v_min_f32`, `v_max_f32`, `v_min3_f32`, `v_max3_f32` and `v_med3_f32`, their f16 forms, `v_min_f64` and `v_max_f64`, `IEEE_MODE` and the `FLOAT_MODE` output denormal bits by the output modifier, `DX10_CLAMP` by the clamp modifier, `FP16_OVFL` by every instruction with an f16 result (see its entry), and the whole mode by the 16-bit interpolation instructions (see their entries). With IEEE=1 a signaling NaN operand returns that NaN quieted (the first operand checked first, min3/max3 as two nested pairs, med3 as min3 when any operand is NaN), measured on RDNA2 (gfx1035) over every combination of ±0, ±1, ±inf, denormals, quiet and signaling NaNs with f32 denormals kept and flushed. IEEE=0 and unknown modes keep the previous f32 lowering, which matches IEEE=0. The f16 forms choose their NaN results on the bits: a NaN operand returns the other operand unchanged (src0 when both are NaN), and with IEEE=1 a signaling NaN operand returns that NaN quieted, in the same order as f32. `v_min_f64` and `v_max_f64` quiet a signaling NaN operand the same way with IEEE=1; with IEEE=0 they return src0 unchanged when src1 is NaN, otherwise src1 when src0 is NaN. Both were measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1 with f32 denormals kept and flushed and f16/f64 denormals kept; RDNA2 has no f64 min3, max3 or med3. Unknown modes follow IEEE=0. `v_min_f16` and `v_max_f16` flush each denormal input to its signed zero when bit 0 of the f16/f64 denormal field is clear (0 and 2); fields 1 and 3 preserve inputs, even when output flushing is enabled. Measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1, including signed zeros, NaNs and neg/abs modifiers. Unknown modes keep f16 denormals. With f16/f64 denormals flushed the hardware also flushes denormal inputs of f64 min/max, which those lowerings still do not. NaN quieting in f16 arithmetic (the unary instructions and the f32 arithmetic follow it, see below) and `FLOAT_MODE` (f32 denormals flushed before the min/max compare in Astro's Playroom's 0xc0) still follow a single mode, as the entries below record. The mode comes from the registered shader header's `sh_registers`, the same source at registration and at draw/dispatch so the prepared artifact keys match. A title that rewrites RSRC1 at run time, or a raw compute program with no header, compiles with that mode or none. -- [v_ldexp_f16](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) follows the `FLOAT_MODE` f16 denormal field: bit 0 preserves the half input and bit 1 the half result; otherwise a denormal is flushed to its signed zero before the scale, and a result whose exact (unrounded) value is below the minimum normal half is flushed to its signed zero, so `0x07ff` scaled by 2^-1 gives zero rather than rounding up to `0x0400`. Measured on RDNA2 (gfx1036) at IEEE=1 with denorm16 0/1/2/3 over the `v_ldexp_f16_controls` operand grid (328 rows per mode): denorm16 0/1/2 fixed 82 mismatched rows against the previous keep-both lowering, with no regressions. NaN and infinity handling and destination high-half preservation are unchanged. Directed `round16` applies only with f16 denormals preserved (see the directed f16 rounding entry); shaders without a known float mode keep the previous lowering. +- [v_ldexp_f16](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) follows the `FLOAT_MODE` f16 denormal field: bit 0 preserves the half input and bit 1 the half result; otherwise a denormal is flushed to its signed zero before the scale, and a result whose exact (unrounded) value is below the minimum normal half is flushed to its signed zero, so `0x07ff` scaled by 2^-1 gives zero rather than rounding up to `0x0400`. Measured on RDNA2 (gfx1036) at IEEE=1 with denorm16 0/1/2/3 over the `v_ldexp_f16_controls` operand grid (328 rows per mode): denorm16 0/1/2 fixed 82 mismatched rows against the previous keep-both lowering, with no regressions. NaN and infinity handling and destination high-half preservation are unchanged. Directed `round16` applies only with f16 denormals preserved (see the directed f16 rounding entry); shaders without a known float mode keep the previous lowering. The f16 field is read per bit here (bit 0 inputs, bit 1 results), while f32 denormal modes 1 and 2 still throw (see the f32 denormals entry); if the f32 field has the same per-bit meaning, those refusals block legal modes, which is not yet measured on RDNA2 (gfx1036). - A [WAIT_REG_MEM](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/WaitMemory.cpp) (libSceAgcDriver) that a compute queue passes on another queue's recorded label store, before the GPU has executed it, is landed before the queue's next dispatch, draw, COND_EXEC or predicated packet reads guest memory: the worker polls until memory satisfies the wait or no unfinished batch writes the label. The graphics queue (queue 0) is exempt, as landing its waits costs PPSA01341 about a quarter of its frame rate; a title that fills memory a graphics command buffer reads after it submits the buffer behind such a wait reads it early there. `APS5_LAND_WAITS_ALL_QUEUES=1` lands the graphics queue's waits too. - The [dispatches that follow a compute queue's wait](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/DispatchAhead.cpp) (libSceAgcDriver), up to the next wait that memory does not satisfy, a packet that writes guest memory other than a label, or a conditional packet, are resolved (memory captured, shader recompiled) when the wait passes, as the CP reads them right after it; the graphics queue is exempt (`APS5_GROUP_CAPTURE_ALL_QUEUES=1` includes it). The device still reads guest memory when the dispatch is issued; when that memory no longer matches the capture, the dispatch is resolved again from current memory. - [v_mullit_f32](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) follows the hardware measured with the MODE register at IEEE=1 and denormals kept (a signaling NaN src0 is quieted only with IEEE=1, as measured at IEEE=0 too; a positive denormal src2 counts as positive); with f32 denormals flushed it follows the f32 denormals entry below, and denormal modes 1 and 2 throw. From 327bf794a21348b5b6b0db76333843e6cb6b972f Mon Sep 17 00:00:00 2001 From: Alex <270190108+CouchRiv@users.noreply.github.com> Date: Sun, 11 Oct 2026 03:25:24 -0400 Subject: [PATCH 451/460] docs(shader): flush v_ldexp_f16 results before rounding in the debt entry The entry said the result is flushed before or after the scale. The lowering flushes the exact scaled value below the minimum normal before rounding. Measured on RDNA2 (gfx1036) with round to nearest over every half magnitude 0x0000..0x0fff of both signs, scaled by 2^-1 and 2^0, IEEE 0 and 1, denorm16 0..3: no mismatch. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/dev/TechnicalDebt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index 1ae162b5b..dfaaf220b 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -434,7 +434,7 @@ Throughout the project, every function at every stage either **does exactly what - A [label or store](../../core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp) (`RELEASE_MEM`, `WRITE_DATA`, `DMA_DATA`) into guest memory the device has no host import of waits for all recorded work and is then written by the CPU, so the queue worker stalls once per such label. The console's CP stores it in stream order without that stall. - [SET_*_REG_INDIRECT](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp) (libSceAgcDriver) reads its register list when the command buffer is submitted, with the packets. The console's graphics PFP reads it when it reaches the packet, so a list written after the submission (by the CPU, or by GPU work ahead of the packet behind PFP_SYNC_ME) is read here before that write. - The [shader float mode](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp) (`FLOAT_MODE`, `DX10_CLAMP`, `IEEE_MODE` and `FP16_OVFL` of `COMPUTE_PGM_RSRC1` and `SPI_SHADER_PGM_RSRC1_PS/GS/HS`, bit positions from Mesa's `gfx10.json` and `gfx103.json`) is decoded into `GuestContext::floatMode` and is part of the shader cache keys and the serialized request (#981). `IEEE_MODE` is read by `v_min_f32`, `v_max_f32`, `v_min3_f32`, `v_max3_f32` and `v_med3_f32`, their f16 forms, `v_min_f64` and `v_max_f64`, `IEEE_MODE` and the `FLOAT_MODE` output denormal bits by the output modifier, `DX10_CLAMP` by the clamp modifier, `FP16_OVFL` by every instruction with an f16 result (see its entry), and the whole mode by the 16-bit interpolation instructions (see their entries). With IEEE=1 a signaling NaN operand returns that NaN quieted (the first operand checked first, min3/max3 as two nested pairs, med3 as min3 when any operand is NaN), measured on RDNA2 (gfx1035) over every combination of ±0, ±1, ±inf, denormals, quiet and signaling NaNs with f32 denormals kept and flushed. IEEE=0 and unknown modes keep the previous f32 lowering, which matches IEEE=0. The f16 forms choose their NaN results on the bits: a NaN operand returns the other operand unchanged (src0 when both are NaN), and with IEEE=1 a signaling NaN operand returns that NaN quieted, in the same order as f32. `v_min_f64` and `v_max_f64` quiet a signaling NaN operand the same way with IEEE=1; with IEEE=0 they return src0 unchanged when src1 is NaN, otherwise src1 when src0 is NaN. Both were measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1 with f32 denormals kept and flushed and f16/f64 denormals kept; RDNA2 has no f64 min3, max3 or med3. Unknown modes follow IEEE=0. `v_min_f16` and `v_max_f16` flush each denormal input to its signed zero when bit 0 of the f16/f64 denormal field is clear (0 and 2); fields 1 and 3 preserve inputs, even when output flushing is enabled. Measured on RDNA2 (gfx1036) at IEEE=0 and IEEE=1, including signed zeros, NaNs and neg/abs modifiers. Unknown modes keep f16 denormals. With f16/f64 denormals flushed the hardware also flushes denormal inputs of f64 min/max, which those lowerings still do not. NaN quieting in f16 arithmetic (the unary instructions and the f32 arithmetic follow it, see below) and `FLOAT_MODE` (f32 denormals flushed before the min/max compare in Astro's Playroom's 0xc0) still follow a single mode, as the entries below record. The mode comes from the registered shader header's `sh_registers`, the same source at registration and at draw/dispatch so the prepared artifact keys match. A title that rewrites RSRC1 at run time, or a raw compute program with no header, compiles with that mode or none. -- [v_ldexp_f16](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) follows the `FLOAT_MODE` f16 denormal field: bit 0 preserves the half input and bit 1 the half result; otherwise a denormal is flushed to its signed zero before the scale, and a result whose exact (unrounded) value is below the minimum normal half is flushed to its signed zero, so `0x07ff` scaled by 2^-1 gives zero rather than rounding up to `0x0400`. Measured on RDNA2 (gfx1036) at IEEE=1 with denorm16 0/1/2/3 over the `v_ldexp_f16_controls` operand grid (328 rows per mode): denorm16 0/1/2 fixed 82 mismatched rows against the previous keep-both lowering, with no regressions. NaN and infinity handling and destination high-half preservation are unchanged. Directed `round16` applies only with f16 denormals preserved (see the directed f16 rounding entry); shaders without a known float mode keep the previous lowering. The f16 field is read per bit here (bit 0 inputs, bit 1 results), while f32 denormal modes 1 and 2 still throw (see the f32 denormals entry); if the f32 field has the same per-bit meaning, those refusals block legal modes, which is not yet measured on RDNA2 (gfx1036). +- [v_ldexp_f16](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) follows the `FLOAT_MODE` f16 denormal field: bit 0 preserves the half input and bit 1 the half result. Otherwise a denormal input is flushed to its signed zero, and a result whose exact scaled value is below the minimum normal (2^-14) is flushed to its signed zero before rounding, so `v_ldexp_f16 0x07ff, -1` gives +0 instead of 0x0400. Measured on RDNA2 (gfx1036) with round to nearest, IEEE 0 and 1 and denorm16 0..3, over every half magnitude 0x0000..0x0fff of both signs scaled by 2^-1 and 2^0 (16384 rows per mode): no mismatch. NaN and infinity handling and destination high-half preservation are unchanged. Shaders without a known float mode keep the previous lowering. - A [WAIT_REG_MEM](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Synchronization/WaitMemory.cpp) (libSceAgcDriver) that a compute queue passes on another queue's recorded label store, before the GPU has executed it, is landed before the queue's next dispatch, draw, COND_EXEC or predicated packet reads guest memory: the worker polls until memory satisfies the wait or no unfinished batch writes the label. The graphics queue (queue 0) is exempt, as landing its waits costs PPSA01341 about a quarter of its frame rate; a title that fills memory a graphics command buffer reads after it submits the buffer behind such a wait reads it early there. `APS5_LAND_WAITS_ALL_QUEUES=1` lands the graphics queue's waits too. - The [dispatches that follow a compute queue's wait](../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/DispatchAhead.cpp) (libSceAgcDriver), up to the next wait that memory does not satisfy, a packet that writes guest memory other than a label, or a conditional packet, are resolved (memory captured, shader recompiled) when the wait passes, as the CP reads them right after it; the graphics queue is exempt (`APS5_GROUP_CAPTURE_ALL_QUEUES=1` includes it). The device still reads guest memory when the dispatch is issued; when that memory no longer matches the capture, the dispatch is resolved again from current memory. - [v_mullit_f32](../../core/shader/recompiler/Translation/src/FloatInstructions.cpp) follows the hardware measured with the MODE register at IEEE=1 and denormals kept (a signaling NaN src0 is quieted only with IEEE=1, as measured at IEEE=0 too; a positive denormal src2 counts as positive); with f32 denormals flushed it follows the f32 denormals entry below, and denormal modes 1 and 2 throw. From 17cc2ce5b48b66292e1d4f65956bf3b3c97365b8 Mon Sep 17 00:00:00 2001 From: TibixDev Date: Wed, 7 Oct 2026 09:49:41 +0300 Subject: [PATCH 452/460] perf(agc): write draw descriptors from retained CPU metadata Replace per-draw Vulkan descriptor copies with a single write from retained buffer and image metadata. Keep per-draw guest snapshots, sets and resource ownership unchanged. Baseline RX 6600 profiling attributed 28% of graphics-worker cycles to descriptor copies. Validation: mixed descriptor and snapshot regression passes on RADV RX 6600 and llvmpipe with Vulkan validation; graphics and guest-memory tests pass. Written with AI assistance. --- .../Graphics/include/ShaderResources.hpp | 4 +- .../Graphics/src/ShaderResources.cpp | 62 ++++------- .../prx/libSceAgcDriver/tests/Recorder.cpp | 100 ++++++++++++++++++ 3 files changed, 122 insertions(+), 44 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp index 49ae6fbf0..43cc45c10 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp @@ -426,7 +426,9 @@ private: // The cache pool the set was allocated from, freed back to it on release. VkDescriptorPool cachePool = VK_NULL_HANDLE; std::vector allocations; - std::vector allocationDescriptors; + std::vector descriptorBuffers; + std::vector descriptorImages; + std::vector descriptorWrites; // Guest buffer elements bound read-only: each use of this object skips that many pending-write // notes (counted in MarkGpuWrites for the [buffers] line). std::size_t readOnlyBuffers = 0; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index 18a32727c..c3d670a7a 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -1477,12 +1477,11 @@ void ShaderResources::buildComplete() { bufferCount += binding.allocations.size(); if (binding.layout.descriptorType != VK_DESCRIPTOR_TYPE_STORAGE_BUFFER) imageCount += binding.layout.descriptorCount; } - allocationDescriptors.resize(allocations.size()); - std::vector buffers; - std::vector images; + auto& buffers = descriptorBuffers; + auto& images = descriptorImages; buffers.reserve(bufferCount); images.reserve(imageCount); - std::vector writes; + auto& writes = descriptorWrites; writes.reserve(bindings.size()); for (const auto& binding : bindings) { VkWriteDescriptorSet write{VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET}; @@ -1494,8 +1493,7 @@ void ShaderResources::buildComplete() { case VK_DESCRIPTOR_TYPE_STORAGE_BUFFER: write.pBufferInfo = buffers.data() + buffers.size(); for (const auto index : binding.allocations) { - allocationDescriptors[index] = descriptor(allocations[index]); - buffers.push_back(allocationDescriptors[index]); + buffers.push_back(descriptor(allocations[index])); } break; case VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE: @@ -3386,47 +3384,25 @@ std::shared_ptr ShaderResources::PrepareDrawBindi result->cache = context.descriptorCache; result->allocation = result->cache->Allocate(_layout, sizes); Require(result->allocation.set != VK_NULL_HANDLE, "draw snapshot descriptor allocation failed"); - std::vector copies; - for (const auto& binding : bindings) { - VkCopyDescriptorSet copy{VK_STRUCTURE_TYPE_COPY_DESCRIPTOR_SET}; - copy.srcSet = _set; - copy.srcBinding = binding.layout.binding; - copy.dstSet = result->allocation.set; - copy.dstBinding = binding.layout.binding; - copy.descriptorCount = binding.layout.descriptorCount; - copies.push_back(copy); - } - const auto update = context.Resolved(&DeviceFunctions::updateDescriptorSets, "vkUpdateDescriptorSets"); - update(context.device, 0, nullptr, static_cast(copies.size()), copies.data()); + auto buffers = descriptorBuffers; + auto writes = descriptorWrites; std::vector snapshotIndices(allocations.size(), selected.size()); for (std::size_t index = 0; index < selected.size(); ++index) snapshotIndices[selected[index]] = index; - std::size_t infoCount = 0; - for (const auto& binding : bindings) infoCount += binding.allocations.size(); - std::vector infos; - infos.reserve(infoCount); - std::vector writes; - for (const auto& binding : bindings) { - if (std::none_of(binding.allocations.begin(), binding.allocations.end(), - [&](std::size_t index) { return snapshotIndices[index] != selected.size(); })) continue; - const auto first = infos.size(); - for (const auto index : binding.allocations) { - const auto snapshotIndex = snapshotIndices[index]; - if (snapshotIndex == selected.size()) infos.push_back(allocationDescriptors[index]); - else { - const auto& snapshot = result->snapshots[snapshotIndex]; - infos.push_back({snapshot.buffer->Handle(), 0, snapshot.buffer->Bytes().size()}); - } - } - VkWriteDescriptorSet write{VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET}; + for (std::size_t index = 0; index < bindings.size(); ++index) { + const auto& binding = bindings[index]; + auto& write = writes[index]; write.dstSet = result->allocation.set; - write.dstBinding = binding.layout.binding; - write.dstArrayElement = 0; - write.descriptorCount = static_cast(binding.allocations.size()); - write.descriptorType = binding.layout.descriptorType; - write.pBufferInfo = infos.data() + first; - writes.push_back(write); + if (binding.allocations.empty()) continue; + const auto offset = static_cast(write.pBufferInfo - descriptorBuffers.data()); + write.pBufferInfo = buffers.data() + offset; + for (std::size_t element = 0; element < binding.allocations.size(); ++element) { + const auto snapshotIndex = snapshotIndices[binding.allocations[element]]; + if (snapshotIndex == selected.size()) continue; + const auto& snapshot = result->snapshots[snapshotIndex]; + buffers[offset + element] = {snapshot.buffer->Handle(), 0, snapshot.buffer->Bytes().size()}; + } } - update(context.device, static_cast(writes.size()), writes.data(), 0, nullptr); + context.Resolved(&DeviceFunctions::updateDescriptorSets, "vkUpdateDescriptorSets")(context.device, static_cast(writes.size()), writes.data(), 0, nullptr); recorder.Keep(result); for (const auto& snapshot : result->snapshots) recorder.KeepBytes(snapshot.buffer.get(), snapshot.buffer->Bytes().size()); return result; diff --git a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp index 2968dfe4a..9b894726c 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Recorder.cpp @@ -1465,6 +1465,99 @@ void misalignedSnapshotTests(const Device& device, Recorder& recorder) { recorder.Activate(); } +class DescriptorUpdates { +public: + struct Contents { + std::map, VkDescriptorBufferInfo> buffers; + std::map, VkDescriptorImageInfo> images; + }; + explicit DescriptorUpdates(Context& context) : context(context), previous(context.functions), + update(context.Resolved(&DeviceFunctions::updateDescriptorSets, "vkUpdateDescriptorSets")) { + Require(active == nullptr, "nested descriptor update capture"); + if (previous != nullptr) functions = *previous; + functions.updateDescriptorSets = captureUpdate; + context.functions = &functions; + active = this; + } + ~DescriptorUpdates() { context.functions = previous; active = nullptr; } + std::vector calls; + std::uint32_t copies = 0; + +private: + static VKAPI_ATTR void VKAPI_CALL captureUpdate(VkDevice device, std::uint32_t count, const VkWriteDescriptorSet* writes, std::uint32_t copyCount, const VkCopyDescriptorSet* copies) { + Contents contents; + for (std::uint32_t i = 0; i < count; ++i) { + const auto& write = writes[i]; + for (std::uint32_t element = 0; element < write.descriptorCount; ++element) { + const auto key = std::pair{write.dstBinding, write.dstArrayElement + element}; + if (write.descriptorType == VK_DESCRIPTOR_TYPE_STORAGE_BUFFER) contents.buffers[key] = write.pBufferInfo[element]; + else contents.images[key] = write.pImageInfo[element]; + } + } + active->calls.push_back(std::move(contents)); + active->copies += copyCount; + active->update(device, count, writes, copyCount, copies); + } + Context& context; + const DeviceFunctions* previous; + DeviceFunctions functions{}; + PFN_vkUpdateDescriptorSets update; + static inline DescriptorUpdates* active = nullptr; +}; + +void drawDescriptorWriteTests(const Device& device, Recorder& recorder) { + auto context = device.GetContext(); + DescriptorUpdates updates(context); + { + DescriptorCache cache(context); + context.descriptorCache = &cache; + Recorder snapshots(context); + snapshots.Activate(); + ShaderRecompiler::RecompileResult program; + ShaderRecompiler::DescriptorBinding data{}; + data.kind = ShaderRecompiler::DescriptorKind::StorageBuffer; + data.role = ShaderRecompiler::DescriptorRole::FlattenedSrt; + data.binding = 2; + data.count = 1; + data.guestDescriptor = {1, 2, 3, 4}; + program.bindings.push_back(data); + data.binding = 7; + program.bindings.push_back(data); + ShaderRecompiler::DescriptorBinding sampler{}; + sampler.kind = ShaderRecompiler::DescriptorKind::Sampler; + sampler.role = ShaderRecompiler::DescriptorRole::GuestSamplers; + sampler.binding = static_cast(ShaderRecompiler::RuntimeAbi::Binding::Samplers); + sampler.count = 2; + sampler.guestDescriptor.resize(8); + sampler.samplerDepthCompare = {false, false}; + program.bindings.push_back(sampler); + const CompiledShader shader{ShaderRecompiler::ShaderStage::Compute, &program, 0}; + ShaderResources resources(context, shader); + Require(updates.calls.size() == 1, "building resources did not write one descriptor set"); + const auto original = updates.calls.front(); + Require(original.buffers.size() == 2 && original.images.size() == 2, "descriptor test lacks mixed bindings and an image array"); + for (std::size_t allocation = 0; allocation < 2; ++allocation) { + const ShaderResources::MovedBuffer moved{allocation, 0, 16, {5, 6, 7, 8}}; + const auto draw = resources.PrepareDrawBindings(snapshots, std::span(&moved, 1)); + Require(draw != nullptr && draw->snapshots.size() == 1, "moved shader data did not create a draw snapshot"); + Require(updates.calls.size() == allocation + 2 && updates.copies == 0, "draw descriptors copied GPU memory or needed multiple updates"); + const auto& actual = updates.calls.back(); + Require(actual.buffers.size() == original.buffers.size() && actual.images.size() == original.images.size(), "snapshot update omitted unchanged descriptors"); + for (const auto& [key, before] : original.buffers) { + const auto after = actual.buffers.at(key); + const auto expected = key.first == (allocation == 0 ? 2u : 7u) ? draw->snapshots.front().buffer->Handle() : before.buffer; + Require(after.buffer == expected && after.offset == before.offset && after.range == before.range, "snapshot update replaced the wrong buffer or changed its range"); + } + for (const auto& [key, before] : original.images) { + const auto after = actual.images.at(key); + Require(after.sampler == before.sampler && after.imageView == before.imageView && after.imageLayout == before.imageLayout, "snapshot update changed an image descriptor"); + } + } + snapshots.Sync(); + } + recorder.Activate(); +} + void drawSnapshotReuseTests(const Device& device, Recorder& recorder) { const auto& context = device.GetContext(); constexpr std::size_t bytes = 65536; @@ -4843,6 +4936,12 @@ int main(int argc, char** argv) { std::cout << "CMASK copied clear, write-back and failure tests passed\n"; return 0; } + if (argc == 2 && std::string_view(argv[1]) == "--draw-descriptors-only") { + drawDescriptorWriteTests(device, recorder); + std::cout << "Draw descriptor writes and snapshot overrides passed\n"; + return 0; + } + readTrackingTests(device, recorder); writeSettledTests(device, recorder); completionCountTests(device, recorder); @@ -4863,6 +4962,7 @@ int main(int argc, char** argv) { misalignedSnapshotTests(device, recorder); misalignedRegionTests(device, recorder); drawSnapshotReuseTests(device, recorder); + drawDescriptorWriteTests(device, recorder); drawSnapshotEvictionTests(device); drawInputReuseTests(device, recorder); drawInputInPlaceTests(device, recorder); From 1dbf97622636cc8a38e855da4b5fc84fb277fd05 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 10:27:55 +0200 Subject: [PATCH 453/460] test(agc): snapshot descriptor sets are written whole, without copies Update descriptorSnapshotTests for the single-update path of this PR. --- core/libs/prx/libSceAgcDriver/tests/Graphics.cpp | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp index 3c5a1c1c2..9cd184dfd 100644 --- a/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/Graphics.cpp @@ -2106,6 +2106,7 @@ void descriptorSnapshotTests() { const auto state = AgcDriver::Graphics::DecodeState(makeState()); ShaderResources resources(context, vertex, fragment, state.color, 0, 0); const auto original = findWrite(0).buffers; + const auto originalThird = findWrite(4).buffers; std::vector moved; for (std::size_t index = 0; index < 2; ++index) { if ((selected & (1u << index)) == 0) continue; @@ -2115,9 +2116,11 @@ void descriptorSnapshotTests() { mock.writes.clear(); const auto snapshots = resources.PrepareDrawBindings(recorder, moved); Require(snapshots != nullptr && snapshots->snapshots.size() == moved.size(), "draw snapshots are missing"); - Require(mock.copies.size() == 2 && mock.copies[0].descriptorCount == 2 && mock.copies[1].descriptorCount == 1, "snapshot descriptor set did not copy every original binding"); - Require(mock.writes.size() == 1, "snapshot updates must rewrite one complete affected array"); + Require(mock.copies.empty(), "snapshot descriptor set must be written without descriptor copies"); + Require(mock.writes.size() == 2, "snapshot update must write every binding once"); const auto& write = mock.writes.front(); + const auto& untouched = mock.writes.back(); + Require(untouched.binding == 4 && untouched.count == 1 && untouched.buffers[0].buffer == originalThird[0].buffer && untouched.buffers[0].offset == originalThird[0].offset && untouched.buffers[0].range == originalThird[0].range, "snapshot update changed a binding without snapshots"); Require(write.binding == 0 && write.arrayElement == 0 && write.count == 2, "snapshot update did not start from array element zero"); std::size_t snapshot = 0; for (std::size_t index = 0; index < 2; ++index) { From 0cca339e4b92de7df5f4c29f8efa6f6dc365fbdd Mon Sep 17 00:00:00 2001 From: Hannah Lyons <7751607+babyhuey@users.noreply.github.com> Date: Sun, 11 Oct 2026 03:37:52 -0400 Subject: [PATCH 454/460] feat(libkernel): implement getppid --- core/libs/prx/libkernel/CMakeLists.txt | 9 +++++++ .../libs/prx/libkernel/System/src/Process.cpp | 27 +++++++++++++++++++ core/libs/tests/GuestParentProcess.cpp | 25 +++++++++++++++++ 3 files changed, 61 insertions(+) create mode 100644 core/libs/tests/GuestParentProcess.cpp diff --git a/core/libs/prx/libkernel/CMakeLists.txt b/core/libs/prx/libkernel/CMakeLists.txt index e94d2cf78..858d258b2 100644 --- a/core/libs/prx/libkernel/CMakeLists.txt +++ b/core/libs/prx/libkernel/CMakeLists.txt @@ -485,6 +485,15 @@ if(BUILD_TESTING) set_tests_properties(guest_signals PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() + add_test_executable(guest_parent_process_tests ${LIBS_INCLUDE_DIR}/tests/GuestParentProcess.cpp) + target_include_directories(guest_parent_process_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_parent_process_tests PRIVATE libkernel libc) + configure_windows_unwind(guest_parent_process_tests) + add_test(NAME guest_parent_process COMMAND guest_parent_process_tests) + if(WIN32) + set_tests_properties(guest_parent_process PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() add_test_executable(guest_kill_tests ${LIBS_INCLUDE_DIR}/tests/GuestKill.cpp) target_include_directories(guest_kill_tests PRIVATE ${LIBS_INCLUDE_DIR}) target_link_libraries(guest_kill_tests PRIVATE libkernel libc) diff --git a/core/libs/prx/libkernel/System/src/Process.cpp b/core/libs/prx/libkernel/System/src/Process.cpp index b90c2b628..45528cd7a 100644 --- a/core/libs/prx/libkernel/System/src/Process.cpp +++ b/core/libs/prx/libkernel/System/src/Process.cpp @@ -4,6 +4,7 @@ #endif #include #include +#include #else #include #include @@ -199,6 +200,32 @@ int APS5_VABI getpid_nid_postfix(void) { return static_cast(pid); } +int APS5_VABI getppid_nid_postfix(void) { +#ifdef _WIN32 + const DWORD self = GetCurrentProcessId(); + const HANDLE snapshot = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0); + if (snapshot == INVALID_HANDLE_VALUE) + throw std::system_error(static_cast(GetLastError()), std::system_category(), "getppid: CreateToolhelp32Snapshot"); + PROCESSENTRY32W entry{}; + entry.dwSize = sizeof(entry); + bool found = false; + DWORD pid = 0; + for (BOOL more = Process32FirstW(snapshot, &entry); more && !found; more = Process32NextW(snapshot, &entry)) { + if (entry.th32ProcessID == self) { + pid = entry.th32ParentProcessID; + found = true; + } + } + CloseHandle(snapshot); + if (!found) throw std::runtime_error("getppid: the process is missing from the process snapshot"); +#else + const auto pid = ::getppid(); +#endif + if (static_cast(pid) > static_cast(std::numeric_limits::max())) + throw std::runtime_error("Parent process identifier is outside the guest range"); + return static_cast(pid); +} + int APS5_VABI sceKernelGetAppInfo(int pid, void* info) { if (pid != getpid_nid_postfix()) return SCE_KERNEL_ERROR_EPERM; if (info == nullptr) return 0; diff --git a/core/libs/tests/GuestParentProcess.cpp b/core/libs/tests/GuestParentProcess.cpp new file mode 100644 index 000000000..c705785c2 --- /dev/null +++ b/core/libs/tests/GuestParentProcess.cpp @@ -0,0 +1,25 @@ +#include "prx/libc/include/general/VabiMacros.hpp" +#include +#ifdef _WIN32 +#include +#else +#include +#endif +extern "C" { +int APS5_VABI getpid_nid_postfix(void); +int APS5_VABI getppid_nid_postfix(void); +} +static void Require(bool value) { if (!value) std::abort(); } +int main() { + const int parent = getppid_nid_postfix(); + Require(parent > 0 && parent != getpid_nid_postfix()); + Require(getppid_nid_postfix() == parent); +#ifdef _WIN32 + const HANDLE process = OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, FALSE, static_cast(parent)); + Require(process != nullptr); + CloseHandle(process); +#else + Require(parent == static_cast(::getppid())); +#endif + return 0; +} From 97ee575d6c6a598e72f14b1ff3642abdf751a4ae Mon Sep 17 00:00:00 2001 From: Achraf Reyani Date: Sun, 11 Oct 2026 17:32:47 +0900 Subject: [PATCH 455/460] fix(shader): emit unordered f32 relations as negated ordered ones NVIDIA's Vulkan compiler (driver 616.92) aborts while building a pipeline whose OpFUnordLessThanEqual or OpFUnordGreaterThanEqual compares against +infinity. The negated inverse ordered relation gives the same result for every input, NaN included. Co-Authored-By: Claude Opus 5.5 --- .../SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp b/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp index 06aa6f6ab..a64e65e8a 100644 --- a/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp +++ b/core/shader/recompiler/SpirvBackend/src/SpirvAlu/SpirvAluEmitterMath.cpp @@ -1114,7 +1114,7 @@ std::uint32_t EmitFPOrdLessThan32(SpirvEmitterState& state, std::uint32_t arg0, } std::uint32_t EmitFPUnordLessThan32(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1) { - return EmitNative(state, arg0, arg1); + return EmitNative(state, EmitNative(state, arg0, arg1)); } std::uint32_t EmitFPOrdGreaterThan32(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1) { @@ -1122,7 +1122,7 @@ std::uint32_t EmitFPOrdGreaterThan32(SpirvEmitterState& state, std::uint32_t arg } std::uint32_t EmitFPUnordGreaterThan32(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1) { - return EmitNative(state, arg0, arg1); + return EmitNative(state, EmitNative(state, arg0, arg1)); } std::uint32_t EmitFPOrdLessThanEqual32(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1) { @@ -1130,7 +1130,7 @@ std::uint32_t EmitFPOrdLessThanEqual32(SpirvEmitterState& state, std::uint32_t a } std::uint32_t EmitFPUnordLessThanEqual32(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1) { - return EmitNative(state, arg0, arg1); + return EmitNative(state, EmitNative(state, arg0, arg1)); } std::uint32_t EmitFPOrdGreaterThanEqual32(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1) { @@ -1138,7 +1138,7 @@ std::uint32_t EmitFPOrdGreaterThanEqual32(SpirvEmitterState& state, std::uint32_ } std::uint32_t EmitFPUnordGreaterThanEqual32(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1) { - return EmitNative(state, arg0, arg1); + return EmitNative(state, EmitNative(state, arg0, arg1)); } std::uint32_t EmitFPAdd32(SpirvEmitterState& state, std::uint32_t arg0, std::uint32_t arg1) { From 72166215452da9bd89619f3dc8f2df04ddd332fb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Arvind=20Fr=C3=B8iland?= Date: Fri, 9 Oct 2026 14:05:52 +0900 Subject: [PATCH 456/460] fix(agc): typed image heaps on devices without nullDescriptor The typed descriptor heaps write a null descriptor for a null guest image, and the device therefore required VK_EXT_robustness2's nullDescriptor. MoltenVK has the extension but not the feature, so on macOS no Vulkan device opened and every GPU test was skipped. The feature is now enabled when the device has it. Without it, the device creates a placeholder image per typed image heap: 1x1, cleared to zero, of the heap's dimension, sample count, numeric class and access, and a null guest image binds the placeholder of its heap. A heap whose placeholder the device cannot create throws when a null image reaches it. TechnicalDebt.md lists where a placeholder differs from a null descriptor. agc_placeholder_images checks every image resource the recompiler maps to a heap against that heap's placeholder, so the two cannot drift apart, and agc_shader_device_profile now expects a device without nullDescriptor to be accepted. Co-Authored-By: Claude Opus 5.5 Co-Authored-By: Claude Sonnet 5.5 --- core/libs/prx/libSceAgcDriver/CMakeLists.txt | 11 +- .../Execution/src/ShaderDeviceProfile.cpp | 1 - .../Execution/src/VulkanDevice.cpp | 33 ++++-- .../Graphics/include/Context.hpp | 2 + .../Graphics/include/PlaceholderImages.hpp | 75 ++++++++++++ .../Graphics/src/PlaceholderImages.cpp | 111 ++++++++++++++++++ .../Graphics/src/ShaderResources.cpp | 12 +- .../tests/PlaceholderImages.cpp | 98 ++++++++++++++++ .../tests/ShaderDeviceProfile.cpp | 8 +- docs/dev/TechnicalDebt.md | 1 + 10 files changed, 334 insertions(+), 18 deletions(-) create mode 100644 core/libs/prx/libSceAgcDriver/Graphics/include/PlaceholderImages.hpp create mode 100644 core/libs/prx/libSceAgcDriver/Graphics/src/PlaceholderImages.cpp create mode 100644 core/libs/prx/libSceAgcDriver/tests/PlaceholderImages.cpp diff --git a/core/libs/prx/libSceAgcDriver/CMakeLists.txt b/core/libs/prx/libSceAgcDriver/CMakeLists.txt index 8357c6c20..7232e14a3 100644 --- a/core/libs/prx/libSceAgcDriver/CMakeLists.txt +++ b/core/libs/prx/libSceAgcDriver/CMakeLists.txt @@ -179,6 +179,7 @@ set(agcDriverSources ${CMAKE_CURRENT_SOURCE_DIR}/Graphics/src/PipelineLibrary.cpp ${CMAKE_CURRENT_SOURCE_DIR}/Graphics/src/PipelineCache.cpp ${CMAKE_CURRENT_SOURCE_DIR}/Graphics/src/Resources.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/Graphics/src/PlaceholderImages.cpp ${CMAKE_CURRENT_SOURCE_DIR}/Graphics/src/Recorder.cpp ${CMAKE_CURRENT_SOURCE_DIR}/Graphics/src/ParallelCompare.cpp ${CMAKE_CURRENT_SOURCE_DIR}/Graphics/src/ShaderInputState.cpp @@ -572,6 +573,12 @@ target_link_libraries(agc_shader_disk_cache_tests PRIVATE shader_recompiler_agc_ set_target_properties(agc_shader_disk_cache_tests PROPERTIES CXX_EXTENSIONS OFF) configure_windows_unwind(agc_shader_disk_cache_tests) +add_test_executable(agc_placeholder_images_tests EXCLUDE_FROM_ALL tests/PlaceholderImages.cpp) +target_include_directories(agc_placeholder_images_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/3rdparty/Vulkan-Headers/include) +target_link_libraries(agc_placeholder_images_tests PRIVATE shader_recompiler_agc_driver libc) +set_target_properties(agc_placeholder_images_tests PROPERTIES CXX_EXTENSIONS OFF) +configure_windows_unwind(agc_placeholder_images_tests) + add_test_executable(agc_shader_device_profile_tests EXCLUDE_FROM_ALL tests/ShaderDeviceProfile.cpp Execution/src/ShaderDeviceProfile.cpp) target_include_directories(agc_shader_device_profile_tests PRIVATE ${LIBS_INCLUDE_DIR} ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR}) target_link_libraries(agc_shader_device_profile_tests PRIVATE shader_recompiler_agc_driver libc) @@ -804,6 +811,7 @@ add_test_executable(agc_driver_graphics_tests EXCLUDE_FROM_ALL Graphics/src/ColorTargetLayout.cpp Graphics/src/ColorTargetTransfer.cpp Graphics/src/Resources.cpp + Graphics/src/PlaceholderImages.cpp Graphics/src/ShaderResources.cpp Graphics/src/ShaderAddressResources.cpp Graphics/src/GuestBufferMemory.cpp @@ -860,6 +868,7 @@ add_test_executable(agc_driver_bda_device_tests EXCLUDE_FROM_ALL tests/BdaContracts.cpp Execution/src/BdaFeatures.cpp Graphics/src/Resources.cpp + Graphics/src/PlaceholderImages.cpp Graphics/src/BufferAddress.cpp ) @@ -960,7 +969,7 @@ if(BUILD_TESTING) if(WIN32) set_tests_properties(agc_shader_bindless agc_shader_frontend_pair agc_shader_frontend_pair_off PROPERTIES ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$") endif() - foreach(agcTest IN ITEMS agc_shared_write_tests agc_driver_tests agc_shader_memory_tests agc_structurizer_tests agc_spirv_module_tests agc_dpp_write_tests agc_masked_select_tests agc_constant_bit_test_tests agc_scalar_select_mask_tests agc_vop1_control_decode_tests agc_validate_program_tests agc_pass_hazards_tests agc_shader_disk_cache_tests agc_shader_device_profile_tests agc_shader_control_flow_tests agc_driver_flip_tests agc_driver_pm4_tests agc_driver_graphics_tests agc_driver_bda_device_tests agc_driver_mesh_tests agc_driver_cmask_fast_clear_tests agc_formatted_buffer_once_tests ${agcExecutionTests}) + foreach(agcTest IN ITEMS agc_shared_write_tests agc_driver_tests agc_shader_memory_tests agc_structurizer_tests agc_spirv_module_tests agc_dpp_write_tests agc_masked_select_tests agc_constant_bit_test_tests agc_scalar_select_mask_tests agc_vop1_control_decode_tests agc_validate_program_tests agc_pass_hazards_tests agc_shader_disk_cache_tests agc_shader_device_profile_tests agc_placeholder_images_tests agc_shader_control_flow_tests agc_driver_flip_tests agc_driver_pm4_tests agc_driver_graphics_tests agc_driver_bda_device_tests agc_driver_mesh_tests agc_driver_cmask_fast_clear_tests agc_formatted_buffer_once_tests ${agcExecutionTests}) if(agcTest STREQUAL "agc_driver_constant_store_tests" OR agcTest STREQUAL "agc_driver_flat_apertures_tests") continue() endif() diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/ShaderDeviceProfile.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/ShaderDeviceProfile.cpp index 5eec16c82..3d3794279 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/ShaderDeviceProfile.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/ShaderDeviceProfile.cpp @@ -34,7 +34,6 @@ ShaderDeviceProfile::ShaderDeviceProfile(const ShaderRecompiler::SpirvTarget& ta Require(target.bdaAbiVersion == ShaderRecompiler::BdaAbi::Version, "shader device profile has an incompatible BDA ABI"); Require(bda != nullptr && bda->bufferDeviceAddress == VK_TRUE && core.shaderInt64 == VK_TRUE, "shader runtime requires enabled bufferDeviceAddress and shaderInt64"); Require(bytes != nullptr && bytes->storageBuffer8BitAccess == VK_TRUE, "shader runtime requires enabled storageBuffer8BitAccess"); - Require(nullDescriptors, "shader runtime requires enabled nullDescriptor"); Require(core.vertexPipelineStoresAndAtomics == VK_TRUE && core.fragmentStoresAndAtomics == VK_TRUE, "shader runtime requires enabled graphics stores and atomics"); Require(limits.maxPushConstantsSize >= ShaderRecompiler::RuntimeAbi::PushConstantDwords * sizeof(std::uint32_t) && limits.maxBoundDescriptorSets > ShaderRecompiler::RuntimeAbi::DescriptorSet, "shader runtime ABI exceeds device limits"); Require(limits.maxStorageBufferRange >= sizeof(ShaderRecompiler::RuntimeAbi::ShaderData), "shader runtime ShaderData exceeds device limits"); diff --git a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp index b6ed20d7d..246837582 100644 --- a/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp +++ b/core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp @@ -11,6 +11,7 @@ #include "prx/libSceAgcDriver/Execution/include/ShaderDeviceProfile.hpp" #include "prx/libSceAgcDriver/Execution/include/PresentationScaler.hpp" #include "prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp" +#include "prx/libSceAgcDriver/Graphics/include/PlaceholderImages.hpp" #include "prx/libSceAgcDriver/Graphics/include/DccMetadata.hpp" #include "prx/libSceAgcDriver/Execution/include/DisplayFormat.hpp" #include "prx/libSceAgcDriver/Graphics/include/TextureDetiler.hpp" @@ -283,6 +284,7 @@ struct VulkanDevice::State { std::unique_ptr colorTransfer; std::shared_ptr bufferPool; std::unique_ptr emptyBuffer; + std::unique_ptr placeholderImages; std::unique_ptr textureCache; std::unique_ptr pipelineCache; std::unique_ptr descriptorCache; @@ -616,6 +618,7 @@ struct VulkanDevice::State { if (workgroupMemoryLayout != VK_NULL_HANDLE) reinterpret_cast(deviceProc(device, "vkDestroyDescriptorSetLayout"))(device, workgroupMemoryLayout, nullptr); descriptorCache.reset(); emptyBuffer.reset(); + placeholderImages.reset(); samplerCache.reset(); textureCache.reset(); detiler.reset(); @@ -1287,17 +1290,19 @@ VulkanDevice::VulkanDevice(const PresentationWindow* window) : state(std::make_u } bdaFeatures.pNext = &byteFeatures; VkPhysicalDeviceRobustness2FeaturesEXT robustness2{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_ROBUSTNESS_2_FEATURES_EXT}; - Graphics::Require(hasExtension(VK_EXT_ROBUSTNESS_2_EXTENSION_NAME), "shader runtime requires VK_EXT_robustness2"); - VkPhysicalDeviceFeatures2 queried{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2, &robustness2}; - state->InstanceFunction("vkGetPhysicalDeviceFeatures2")(selected, &queried); - Graphics::Require(robustness2.nullDescriptor == VK_TRUE, "shader runtime requires nullDescriptor"); - robustness2 = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_ROBUSTNESS_2_FEATURES_EXT}; - robustness2.nullDescriptor = VK_TRUE; - robustness2.pNext = bdaFeatures.pNext; - bdaFeatures.pNext = &robustness2; - deviceExtensions.push_back(VK_EXT_ROBUSTNESS_2_EXTENSION_NAME); - deviceInfo.enabledExtensionCount = static_cast(deviceExtensions.size()); - deviceInfo.ppEnabledExtensionNames = deviceExtensions.data(); + if (hasExtension(VK_EXT_ROBUSTNESS_2_EXTENSION_NAME)) { + VkPhysicalDeviceFeatures2 queried{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2, &robustness2}; + state->InstanceFunction("vkGetPhysicalDeviceFeatures2")(selected, &queried); + } + if (robustness2.nullDescriptor == VK_TRUE) { + robustness2 = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_ROBUSTNESS_2_FEATURES_EXT}; + robustness2.nullDescriptor = VK_TRUE; + robustness2.pNext = bdaFeatures.pNext; + bdaFeatures.pNext = &robustness2; + deviceExtensions.push_back(VK_EXT_ROBUSTNESS_2_EXTENSION_NAME); + deviceInfo.enabledExtensionCount = static_cast(deviceExtensions.size()); + deviceInfo.ppEnabledExtensionNames = deviceExtensions.data(); + } deviceInfo.pNext = &bdaFeatures; auto shaderProfile = std::make_unique(buildTarget(), deviceInfo, state->properties.limits); check(state->InstanceFunction("vkCreateDevice")(selected, &deviceInfo, nullptr, &state->device), "vkCreateDevice"); @@ -1323,6 +1328,11 @@ VulkanDevice::VulkanDevice(const PresentationWindow* window) : state(std::make_u state->descriptorCache = std::make_unique(graphicsContext()); state->samplerCache = std::make_unique(); state->CreateWorkgroupMemoryLayout(); + if (!state->shaderProfile->NullDescriptors()) { + std::unique_lock gpuLock(GuestMemory::GpuMutex(), std::defer_lock); + if (!GuestMemory::GpuMutex().HeldByThisThread()) gpuLock.lock(); + state->placeholderImages = std::make_unique(graphicsContext()); + } state->recorder = std::make_unique(graphicsContext(), state->timelineSemaphores); state->recorder->Activate(); state->context = buildContext(); @@ -2758,6 +2768,7 @@ Graphics::Context VulkanDevice::buildContext() const { context.sampleRateShading = state->sampleRateShading; context.nullDescriptors = state->shaderProfile != nullptr && state->shaderProfile->NullDescriptors(); context.bufferInt64Atomics = state->bufferInt64Atomics; + context.placeholderImages = state->placeholderImages.get(); context.primitiveListRestart = state->primitiveListRestart; context.imageViewMinLod = state->imageViewMinLod; context.pipelineExecutableInfo = state->pipelineExecutableInfo; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/Context.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/Context.hpp index 83a15159e..4b8cc7e4c 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/include/Context.hpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/Context.hpp @@ -24,6 +24,7 @@ class Recorder; class DescriptorCache; class SamplerCache; class ShaderResources; +class PlaceholderImages; inline void Require(bool condition, const std::string& reason) { if (!condition) throw std::runtime_error("AGC graphics: " + reason); @@ -159,6 +160,7 @@ struct Context { bool sampleRateShading = false; bool nullDescriptors = false; bool bufferInt64Atomics = false; + const PlaceholderImages* placeholderImages = nullptr; bool primitiveListRestart = false; bool imageViewMinLod = false; bool pipelineExecutableInfo = false; diff --git a/core/libs/prx/libSceAgcDriver/Graphics/include/PlaceholderImages.hpp b/core/libs/prx/libSceAgcDriver/Graphics/include/PlaceholderImages.hpp new file mode 100644 index 000000000..7e9a4dc18 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/Graphics/include/PlaceholderImages.hpp @@ -0,0 +1,75 @@ +#ifndef CORE_LIBS_PRX_LIBSCEAGCDRIVER_GRAPHICS_INCLUDE_PLACEHOLDERIMAGES_HPP +#define CORE_LIBS_PRX_LIBSCEAGCDRIVER_GRAPHICS_INCLUDE_PLACEHOLDERIMAGES_HPP + +#include "prx/libSceAgcDriver/Graphics/include/Context.hpp" +#include "RuntimeAbi.hpp" +#include +#include + +namespace AgcDriver::Graphics { + +inline constexpr std::uint32_t PlaceholderDimensionsPerClass = (ShaderRecompiler::RuntimeAbi::FirstComparisonImageBinding - ShaderRecompiler::RuntimeAbi::FirstImageBinding) / 3u; +static_assert(ShaderRecompiler::RuntimeAbi::FirstStorageImageBinding == ShaderRecompiler::RuntimeAbi::FirstComparisonImageBinding + PlaceholderDimensionsPerClass); +static_assert(ShaderRecompiler::RuntimeAbi::ImageBindingCount == 8u * PlaceholderDimensionsPerClass && PlaceholderDimensionsPerClass == 7u); + +struct PlaceholderImageShape { + VkFormat format; + VkFormatFeatureFlags features; + VkImageUsageFlags usage; + VkImageAspectFlags aspect; + VkImageType type; + VkImageViewType viewType; + VkSampleCountFlagBits samples; +}; + +constexpr PlaceholderImageShape PlaceholderImageShapeOf(std::uint32_t heap) { + constexpr std::array formats{VK_FORMAT_R8G8B8A8_UNORM, VK_FORMAT_R32_UINT, VK_FORMAT_R32_SINT, VK_FORMAT_D16_UNORM, + VK_FORMAT_R8G8B8A8_UNORM, VK_FORMAT_R32_UINT, VK_FORMAT_R32_UINT, VK_FORMAT_R64_UINT}; + constexpr std::array types{VK_IMAGE_TYPE_1D, VK_IMAGE_TYPE_1D, VK_IMAGE_TYPE_2D, VK_IMAGE_TYPE_2D, + VK_IMAGE_TYPE_2D, VK_IMAGE_TYPE_2D, VK_IMAGE_TYPE_3D}; + constexpr std::array views{VK_IMAGE_VIEW_TYPE_1D, VK_IMAGE_VIEW_TYPE_1D_ARRAY, VK_IMAGE_VIEW_TYPE_2D, + VK_IMAGE_VIEW_TYPE_2D_ARRAY, VK_IMAGE_VIEW_TYPE_2D, VK_IMAGE_VIEW_TYPE_2D_ARRAY, + VK_IMAGE_VIEW_TYPE_3D}; + constexpr std::array samples{VK_SAMPLE_COUNT_1_BIT, VK_SAMPLE_COUNT_1_BIT, VK_SAMPLE_COUNT_1_BIT, + VK_SAMPLE_COUNT_1_BIT, VK_SAMPLE_COUNT_4_BIT, VK_SAMPLE_COUNT_4_BIT, + VK_SAMPLE_COUNT_1_BIT}; + const auto index = heap - ShaderRecompiler::RuntimeAbi::FirstImageBinding; + const auto kind = index / PlaceholderDimensionsPerClass; + const auto format = formats[kind]; + const auto dimension = index % PlaceholderDimensionsPerClass; + const bool storage = heap >= ShaderRecompiler::RuntimeAbi::FirstStorageImageBinding; + const bool atomic = kind >= 6u; + const VkFormatFeatureFlags access = atomic ? VK_FORMAT_FEATURE_STORAGE_IMAGE_BIT | VK_FORMAT_FEATURE_STORAGE_IMAGE_ATOMIC_BIT + : storage ? VK_FORMAT_FEATURE_STORAGE_IMAGE_BIT + : VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT; + return {format, + static_cast(access | VK_FORMAT_FEATURE_TRANSFER_DST_BIT), + static_cast((storage ? VK_IMAGE_USAGE_STORAGE_BIT : VK_IMAGE_USAGE_SAMPLED_BIT) | VK_IMAGE_USAGE_TRANSFER_DST_BIT), + static_cast(format == VK_FORMAT_D16_UNORM ? VK_IMAGE_ASPECT_DEPTH_BIT : VK_IMAGE_ASPECT_COLOR_BIT), + types[dimension], + views[dimension], + samples[dimension]}; +} + +class PlaceholderImages { +public: + explicit PlaceholderImages(const Context& context); + ~PlaceholderImages(); + PlaceholderImages(const PlaceholderImages&) = delete; + PlaceholderImages& operator=(const PlaceholderImages&) = delete; + VkImageView View(std::uint32_t heap) const; + +private: + struct Image { + VkImage image = VK_NULL_HANDLE; + VkDeviceMemory memory = VK_NULL_HANDLE; + VkImageView view = VK_NULL_HANDLE; + }; + void release() noexcept; + Context context; + std::array images{}; +}; + +} + +#endif diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/PlaceholderImages.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/PlaceholderImages.cpp new file mode 100644 index 000000000..cc352d0d9 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/PlaceholderImages.cpp @@ -0,0 +1,111 @@ +#include "prx/libSceAgcDriver/Graphics/include/PlaceholderImages.hpp" +#include "prx/libSceAgcDriver/Graphics/include/Resources.hpp" +#include +#include + +namespace AgcDriver::Graphics { +namespace { + +bool Supported(const Context& context, const PlaceholderImageShape& shape) { + VkFormatProperties properties{}; + context.formatProperties(context.physical, shape.format, &properties); + if ((properties.optimalTilingFeatures & shape.features) != shape.features) return false; + VkImageFormatProperties limits{}; + if (context.imageFormatProperties(context.physical, shape.format, shape.type, VK_IMAGE_TILING_OPTIMAL, shape.usage, 0, &limits) != VK_SUCCESS) return false; + return (limits.sampleCounts & shape.samples) != 0; +} + +} + +PlaceholderImages::PlaceholderImages(const Context& context) : context(context) { + try { + std::vector toClear; + for (std::uint32_t index = 0; index < images.size(); ++index) { + const auto shape = PlaceholderImageShapeOf(ShaderRecompiler::RuntimeAbi::FirstImageBinding + index); + if (!Supported(context, shape)) continue; + auto& image = images[index]; + VkImageCreateInfo info{VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO}; + info.imageType = shape.type; + info.format = shape.format; + info.extent = {1, 1, 1}; + info.mipLevels = 1; + info.arrayLayers = 1; + info.samples = shape.samples; + info.tiling = VK_IMAGE_TILING_OPTIMAL; + info.usage = shape.usage; + info.sharingMode = VK_SHARING_MODE_EXCLUSIVE; + info.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED; + Check(context.Function("vkCreateImage")(context.device, &info, nullptr, &image.image), "vkCreateImage placeholder"); + VkMemoryRequirements requirements{}; + context.Function("vkGetImageMemoryRequirements")(context.device, image.image, &requirements); + VkMemoryAllocateInfo allocation{VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO}; + allocation.allocationSize = requirements.size; + allocation.memoryTypeIndex = context.MemoryType(requirements.memoryTypeBits, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT); + Check(context.Function("vkAllocateMemory")(context.device, &allocation, nullptr, &image.memory), "vkAllocateMemory placeholder"); + Check(context.Function("vkBindImageMemory")(context.device, image.image, image.memory, 0), "vkBindImageMemory placeholder"); + VkImageViewCreateInfo viewInfo{VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO}; + viewInfo.image = image.image; + viewInfo.viewType = shape.viewType; + viewInfo.format = shape.format; + viewInfo.subresourceRange = {shape.aspect, 0, 1, 0, 1}; + Check(context.Function("vkCreateImageView")(context.device, &viewInfo, nullptr, &image.view), "vkCreateImageView placeholder"); + VkImageMemoryBarrier barrier{VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER}; + barrier.dstAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT; + barrier.oldLayout = VK_IMAGE_LAYOUT_UNDEFINED; + barrier.newLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL; + barrier.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED; + barrier.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED; + barrier.image = image.image; + barrier.subresourceRange = viewInfo.subresourceRange; + toClear.push_back(barrier); + } + if (toClear.empty()) return; + CommandBatch batch(context); + const auto commands = batch.Handle(); + const auto pipelineBarrier = context.Function("vkCmdPipelineBarrier"); + pipelineBarrier(commands, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, 0, nullptr, 0, nullptr, static_cast(toClear.size()), toClear.data()); + std::vector toGeneral; + for (const auto& barrier : toClear) { + if (barrier.subresourceRange.aspectMask == VK_IMAGE_ASPECT_DEPTH_BIT) { + const VkClearDepthStencilValue zero{0.0f, 0}; + context.Function("vkCmdClearDepthStencilImage")(commands, barrier.image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, &zero, 1, &barrier.subresourceRange); + } else { + const VkClearColorValue zero{}; + context.Function("vkCmdClearColorImage")(commands, barrier.image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, &zero, 1, &barrier.subresourceRange); + } + auto general = barrier; + general.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT; + general.dstAccessMask = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_SHADER_WRITE_BIT; + general.oldLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL; + general.newLayout = VK_IMAGE_LAYOUT_GENERAL; + toGeneral.push_back(general); + } + pipelineBarrier(commands, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_ALL_COMMANDS_BIT, 0, 0, nullptr, 0, nullptr, static_cast(toGeneral.size()), toGeneral.data()); + batch.SubmitAndWait(); + } catch (...) { + release(); + throw; + } +} + +PlaceholderImages::~PlaceholderImages() { + release(); +} + +void PlaceholderImages::release() noexcept { + for (auto& image : images) { + if (image.view) context.Function("vkDestroyImageView")(context.device, image.view, nullptr); + if (image.image) context.Function("vkDestroyImage")(context.device, image.image, nullptr); + if (image.memory) context.Function("vkFreeMemory")(context.device, image.memory, nullptr); + image = {}; + } +} + +VkImageView PlaceholderImages::View(std::uint32_t heap) const { + const auto index = heap - ShaderRecompiler::RuntimeAbi::FirstImageBinding; + Require(heap >= ShaderRecompiler::RuntimeAbi::FirstImageBinding && index < images.size() && images[index].view != VK_NULL_HANDLE, + "the device has no placeholder image for typed image heap " + std::to_string(heap)); + return images[index].view; +} + +} diff --git a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp index 18a32727c..222b73753 100644 --- a/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp +++ b/core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp @@ -3,6 +3,7 @@ #include "prx/libSceAgcDriver/Execution/include/CaptureTrace.hpp" #include "ThreadOwned.hpp" #include "prx/libSceAgcDriver/Graphics/include/DepthSurface.hpp" +#include "prx/libSceAgcDriver/Graphics/include/PlaceholderImages.hpp" #include "prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp" #include "prx/libSceAgcDriver/Graphics/include/Recorder.hpp" #include @@ -1484,6 +1485,11 @@ void ShaderResources::buildComplete() { images.reserve(imageCount); std::vector writes; writes.reserve(bindings.size()); + const auto absentImage = [this](std::uint32_t binding) { + constexpr auto bindingCount = static_cast(ShaderRecompiler::RuntimeAbi::Binding::Count); + const auto view = context.nullDescriptors ? VK_NULL_HANDLE : context.placeholderImages->View(binding % bindingCount); + return VkDescriptorImageInfo{VK_NULL_HANDLE, view, VK_IMAGE_LAYOUT_GENERAL}; + }; for (const auto& binding : bindings) { VkWriteDescriptorSet write{VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET}; write.dstSet = _set; @@ -1501,7 +1507,7 @@ void ShaderResources::buildComplete() { case VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE: write.pImageInfo = images.data() + images.size(); for (const auto index : binding.imageAllocations) { - if (index == std::numeric_limits::max()) images.push_back({VK_NULL_HANDLE, VK_NULL_HANDLE, VK_IMAGE_LAYOUT_GENERAL}); + if (index == std::numeric_limits::max()) images.push_back(absentImage(binding.layout.binding)); else images.push_back({VK_NULL_HANDLE, textureFirstLayer[index] ? textures[index]->FirstLayerView() : textures[index]->View(), textures[index]->Layout()}); } Require(binding.imageAllocations.size() == binding.layout.descriptorCount, "descriptor allocations disagree with compact binding"); @@ -1509,7 +1515,7 @@ void ShaderResources::buildComplete() { case VK_DESCRIPTOR_TYPE_STORAGE_IMAGE: write.pImageInfo = images.data() + images.size(); for (const auto index : binding.imageAllocations) { - if (index == std::numeric_limits::max()) images.push_back({VK_NULL_HANDLE, VK_NULL_HANDLE, VK_IMAGE_LAYOUT_GENERAL}); + if (index == std::numeric_limits::max()) images.push_back(absentImage(binding.layout.binding)); else images.push_back({VK_NULL_HANDLE, storageAtomic64[index] ? storageTextures[index]->Atomic64View(storageMips[index], storageFirstLayer[index]) : storageAtomic[index] ? storageTextures[index]->AtomicView(storageMips[index], storageFirstLayer[index]) : storageTextures[index]->StorageView(storageMips[index], storageFirstLayer[index]), VK_IMAGE_LAYOUT_GENERAL}); } Require(binding.imageAllocations.size() == binding.layout.descriptorCount, "descriptor allocations disagree with compact binding"); @@ -2906,7 +2912,7 @@ void ShaderResources::addImageBinding(const ShaderRecompiler::DescriptorBinding& const bool sampledHeap = localBinding >= ShaderRecompiler::RuntimeAbi::FirstImageBinding && localBinding < ShaderRecompiler::RuntimeAbi::FirstStorageImageBinding; const bool storageHeap = localBinding >= ShaderRecompiler::RuntimeAbi::FirstStorageImageBinding && localBinding < static_cast(ShaderRecompiler::RuntimeAbi::Binding::Samplers); Require((binding.kind == ShaderRecompiler::DescriptorKind::SampledImage && sampledHeap) || (binding.kind == ShaderRecompiler::DescriptorKind::StorageImage && storageHeap) || (binding.kind == ShaderRecompiler::DescriptorKind::Sampler && heap == ShaderRecompiler::RuntimeAbi::Binding::Samplers), "resource class disagrees with typed descriptor heap"); - Require(binding.kind == ShaderRecompiler::DescriptorKind::Sampler || context.nullDescriptors, "typed image heaps require enabled nullDescriptor"); + Require(binding.kind == ShaderRecompiler::DescriptorKind::Sampler || context.nullDescriptors || context.placeholderImages != nullptr, "typed image heaps require nullDescriptor or placeholder images"); if (binding.kind == ShaderRecompiler::DescriptorKind::StorageImage) { // Storage images are guest textures the shader writes (looked up in stage B). Require(binding.role == ShaderRecompiler::DescriptorRole::GuestImages, "storage image binding has a non-image role"); diff --git a/core/libs/prx/libSceAgcDriver/tests/PlaceholderImages.cpp b/core/libs/prx/libSceAgcDriver/tests/PlaceholderImages.cpp new file mode 100644 index 000000000..e00241e37 --- /dev/null +++ b/core/libs/prx/libSceAgcDriver/tests/PlaceholderImages.cpp @@ -0,0 +1,98 @@ +#include "prx/libSceAgcDriver/Graphics/include/PlaceholderImages.hpp" +#include "IntermediateRepresentation/IrProgram.hpp" +#include +#include +#include +#include +#include +#include +#include + +using namespace ShaderRecompiler; + +namespace { + +void Require(bool condition, const std::string& message) { + if (!condition) throw std::runtime_error(message); +} + +VkImageViewType ViewType(RdnaImageDimension dimension) { + switch (dimension) { + case RdnaImageDimension::Dim1D: return VK_IMAGE_VIEW_TYPE_1D; + case RdnaImageDimension::Dim1DArray: return VK_IMAGE_VIEW_TYPE_1D_ARRAY; + case RdnaImageDimension::Dim2D: + case RdnaImageDimension::Dim2DMsaa: return VK_IMAGE_VIEW_TYPE_2D; + case RdnaImageDimension::Dim2DArray: + case RdnaImageDimension::Dim2DMsaaArray: return VK_IMAGE_VIEW_TYPE_2D_ARRAY; + case RdnaImageDimension::Dim3D: return VK_IMAGE_VIEW_TYPE_3D; + default: throw std::runtime_error("unknown image dimension"); + } +} + +VkFormat Format(const ImageResource& image) { + if (image.depthCompare) return VK_FORMAT_D16_UNORM; + if (image.atomic64) return VK_FORMAT_R64_UINT; + switch (image.numericClass) { + case IrTextureNumericClass::Float: return VK_FORMAT_R8G8B8A8_UNORM; + case IrTextureNumericClass::Uint: return VK_FORMAT_R32_UINT; + case IrTextureNumericClass::Sint: return VK_FORMAT_R32_SINT; + default: throw std::runtime_error("unknown numeric class"); + } +} + +void CheckEveryHeap() { + constexpr std::array dimensions{RdnaImageDimension::Dim1D, RdnaImageDimension::Dim1DArray, RdnaImageDimension::Dim2D, RdnaImageDimension::Dim2DArray, + RdnaImageDimension::Dim2DMsaa, RdnaImageDimension::Dim2DMsaaArray, RdnaImageDimension::Dim3D}; + std::set heaps; + for (const auto resourceClass : {ImageResourceClass::Sampled, ImageResourceClass::Storage}) { + for (const auto numericClass : {IrTextureNumericClass::Float, IrTextureNumericClass::Uint, IrTextureNumericClass::Sint}) { + for (const bool depthCompare : {false, true}) { + for (const bool atomic : {false, true}) { + for (const bool atomic64 : {false, true}) { + if ((depthCompare && resourceClass == ImageResourceClass::Storage) || (atomic64 && !atomic)) continue; + for (const auto dimension : dimensions) { + ImageResource image; + image.resourceClass = resourceClass; + image.numericClass = numericClass; + image.dimension = dimension; + image.depthCompare = depthCompare; + image.atomic = atomic; + image.atomic64 = atomic64; + std::uint32_t heap = 0; + try { + heap = static_cast(DescriptorBindingForImage(image)); + } catch (const std::exception&) { + continue; + } + const auto shape = AgcDriver::Graphics::PlaceholderImageShapeOf(heap); + const auto where = " for heap " + std::to_string(heap); + const bool storage = resourceClass == ImageResourceClass::Storage; + const bool multisampled = dimension == RdnaImageDimension::Dim2DMsaa || dimension == RdnaImageDimension::Dim2DMsaaArray; + Require(((shape.usage & VK_IMAGE_USAGE_STORAGE_BIT) != 0) == storage && ((shape.usage & VK_IMAGE_USAGE_SAMPLED_BIT) != 0) == !storage, "placeholder usage disagrees with the resource class" + where); + Require((shape.aspect == VK_IMAGE_ASPECT_DEPTH_BIT) == depthCompare, "placeholder aspect disagrees with depth comparison" + where); + Require(shape.format == Format(image), "placeholder format disagrees with the numeric class" + where); + Require(shape.viewType == ViewType(dimension), "placeholder view type disagrees with the dimension" + where); + Require((shape.samples != VK_SAMPLE_COUNT_1_BIT) == multisampled, "placeholder sample count disagrees with multisampling" + where); + Require(((shape.features & VK_FORMAT_FEATURE_STORAGE_IMAGE_ATOMIC_BIT) != 0) == atomic, "placeholder format features disagree with atomics" + where); + heaps.insert(heap); + } + } + } + } + } + } + Require(heaps.size() == RuntimeAbi::ImageBindingCount, "not every typed image heap is reachable from an image resource: " + std::to_string(heaps.size())); +} + +} + +int main() { + try { + CheckEveryHeap(); + std::cout << "placeholder image tests passed\n"; + return 0; + } catch (const std::exception& error) { + std::cerr << error.what() << '\n'; + return 1; + } +} diff --git a/core/libs/prx/libSceAgcDriver/tests/ShaderDeviceProfile.cpp b/core/libs/prx/libSceAgcDriver/tests/ShaderDeviceProfile.cpp index 11e9c1dd1..c5f9c8d5d 100644 --- a/core/libs/prx/libSceAgcDriver/tests/ShaderDeviceProfile.cpp +++ b/core/libs/prx/libSceAgcDriver/tests/ShaderDeviceProfile.cpp @@ -82,8 +82,12 @@ void CheckProfile() { invalid([](auto& value) { value.device.pNext = nullptr; }, "bufferDeviceAddress"); invalid([](auto& value) { value.core.shaderInt64 = VK_FALSE; }, "shaderInt64"); invalid([](auto& value) { value.bytes.storageBuffer8BitAccess = VK_FALSE; }, "storageBuffer8BitAccess"); - invalid([](auto& value) { value.robustness.nullDescriptor = VK_FALSE; }, "nullDescriptor"); - invalid([](auto& value) { value.indexing.pNext = nullptr; }, "nullDescriptor"); + ProfileInput withoutNull; + withoutNull.robustness.nullDescriptor = VK_FALSE; + Require(!AgcDriver::ShaderDeviceProfile(withoutNull.Target(), withoutNull.device, withoutNull.limits).NullDescriptors(), "a device without nullDescriptor was profiled with it"); + ProfileInput withoutRobustness; + withoutRobustness.indexing.pNext = nullptr; + Require(!AgcDriver::ShaderDeviceProfile(withoutRobustness.Target(), withoutRobustness.device, withoutRobustness.limits).NullDescriptors(), "a device without VK_EXT_robustness2 was profiled with nullDescriptor"); invalid([](auto& value) { value.core.fragmentStoresAndAtomics = VK_FALSE; }, "graphics stores and atomics"); invalid([](auto& value) { value.limits.maxPushConstantsSize = 127u; }, "exceeds device limits"); invalid([](auto& value) { value.limits.maxBoundDescriptorSets = 0u; }, "exceeds device limits"); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index e6360c572..62f01fa77 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -348,6 +348,7 @@ Throughout the project, every function at every stage either **does exactly what - [Graphics pipeline libraries](../../core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp) (libSceAgcDriver): a pipeline is fast-linked at the draw and relinked with `VK_PIPELINE_CREATE_LINK_TIME_OPTIMIZATION_BIT_EXT` on a background thread, and binds the optimized one from the next draw on. Draws recorded before that use the fast link, whose vertex stage keeps the outputs the fragment stage does not read: on RADV (Radeon 660M, Mesa 26.2.4) 346 of 483 Astro's Playroom vertex shaders paired with a fragment stage that reads none of them used more VGPRs (median 8 more, up to 2.5 times), and on an RTX 4090 13 of 15 PPSA36082 vertex shaders up to 68 instead of 38 (#2242). The optimized link compiles to the same code as a whole pipeline - [sceAvPlayerGetVideoData](../../core/libs/prx/libSceAvPlayer/src/Source.cpp) (libSceAvPlayer) in sync mode `SCE_AVPLAYER_AV_SYNC_MODE_NONE` skips queued frames that the audio clock has already passed while the title pulls audio, so a title that requests video slower than the film's frame rate keeps the picture with the sound. Whether the console hands out every frame in this mode is unknown. PPSA12544 requests 7-19 frames per second for a 30 fps cinematic - [`APS5_THREAD_PRIORITY=rt`](../../core/libs/prx/libSceAgcDriver/Execution/src/ThreadPriority.cpp) (libSceAgcDriver) lowers the process's hard `RLIMIT_RTTIME` when it is above `APS5_THREAD_RTTIME_US` or rtkit's `RTTimeUSecMax`, because [rtkit](https://github.com/heftig/rtkit/blob/c295fa849f52b487be6433e69e08b46251950399/rtkit-daemon.c#L592-L637) (`verify_process_rttime`) reads the hard limit from `/proc//limits` and refuses real-time scheduling when it is higher. An unprivileged process cannot raise the hard limit again, so it stays lowered for the process and its threads even when rtkit then refuses the request (burst limit, user or start-time checks) and the thread throws. rtkit's `MaxRealtimePriority` and `RTTimeUSecMax` are read first, so an unreachable rtkit or a priority above its maximum leaves the limit untouched. The lowered limit only matters for threads that later run under a real-time policy +- On a device without VK_EXT_robustness2's `nullDescriptor`, such as MoltenVK, a null image descriptor in a typed heap is a [placeholder image](../../core/libs/prx/libSceAgcDriver/Graphics/src/PlaceholderImages.cpp) of that heap's type, 1×1 and cleared to zero, which every null descriptor of the heap shares. Reads give zeros, as `nullDescriptor` allows, but stores and atomics to it are kept, so a later read of a null storage image can return them, and a depth comparison compares against depth 0. A heap whose placeholder the device cannot create, for example 1D or 3D depth, 64-bit atomics without 64-bit image atomics, or multisampled storage without `shaderStorageImageMultisample`, throws when a null descriptor reaches it. - [libSceHttp2 asynchronous requests](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) never reach the network: `sceHttp2SendRequestAsync` accepts the request, queues a completion whose result is the library's network error and triggers the user event named by the kqueue option (equeue at offset 0, event id at 8, user data at 16, layout inferred from PPSA17221); `sceHttp2WaitAsync` returns that completion. The completion's `event_type` value is unknown and left 0. PPSA17221 completes its request object only after `sceHttp2SendRequestAsync` returns, and its handler for a synchronous failure calls `shared_from_this` before that, so a send must not fail synchronously. A kqueue option or user event that is missing, an extra option, and a wait with no completed operation throw - [system](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) runs no command. It follows FreeBSD 11's `lib/libc/stdlib/system.c` for a sandbox without `/bin/sh`: a null command returns 1 unconditionally, as that version does, and any command returns the wait status of a child that could not exec the shell and called `_exit(127)` (`0x7f00`). Whether the console's application sandbox has no shell, and whether `vfork` succeeds there (FreeBSD returns -1 when it fails), is unverified - [sceKernelWriteThrottlingStatus](../../core/libs/prx/libkernel/File/src/Stdio.cpp) (libkernel) reports no write throttling: the host SSD has no write budget, so the first qword of the 32-byte status is set to `UINT32_MAX` and the rest is zeroed. PPSA17221 reads only that qword: it multiplies it by 64 KiB and compares the product with byte thresholds up to 0x4AF00000 to pick one of five tiers. When the call fails the title assumes 0x4B000001 bytes, which selects the same top tier as `UINT32_MAX`. What the value measures on hardware (a remaining budget or a rate) and the meaning of the other three qwords are unknown From 144b1ea8f83b29729956162a7a541d324dbebf17 Mon Sep 17 00:00:00 2001 From: exefer Date: Sun, 11 Oct 2026 10:29:50 +0200 Subject: [PATCH 457/460] fix(relinker): bind Linux guest imports by declared module identity --- .../relinker/src/guest/GuestModuleBuilder.cpp | 22 +++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp b/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp index 238fdf832..fed193013 100644 --- a/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp +++ b/core/relinker/relinker/src/guest/GuestModuleBuilder.cpp @@ -237,7 +237,19 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path if (end == dynamic.DynStrData.end()) throw Domain::RelinkerException("Unterminated executable symbol name"); const std::string name(start, end); if (declaresHost(executableModules[offset / 24])) continue; - rejectSharedImport(name.substr(0, name.find('#')), Io::Utf8Path(inputPath)); + const std::string base = name.substr(0, name.find('#')); + const auto shared = sharedExports.find(base); + std::size_t needed = 0; + if (shared != sharedExports.end()) { + for (const auto provider : shared->second) { + const auto& file = images[provider].SourcePath.filename().string(); + const auto& soname = images[provider].Soname; + if (std::find(neededNames.begin(), neededNames.end(), file) == neededNames.end() && + (soname.empty() || std::find(neededNames.begin(), neededNames.end(), soname) == neededNames.end())) continue; + if (++needed > 1) break; + } + } + if (needed != 1) rejectSharedImport(base, Io::Utf8Path(inputPath)); if (!windows && exports.contains(name)) rename(dynamic.DynSymData, dynamic.DynStrData, offset / 24, name); } std::vector> dependencies(images.size()); @@ -255,13 +267,15 @@ std::vector GuestModuleBuilder::Build(const std::filesystem::path for (std::size_t symbolIndex = 0; symbolIndex < images[index].Symbols.size(); ++symbolIndex) { const auto& symbol = images[index].Symbols[symbolIndex]; if (symbol.Section != 0 || symbol.Name.empty() || declaresHost(symbol.Library)) continue; - rejectSharedImport(symbol.Name, Io::Utf8Path(images[index].SourcePath)); + const auto declared = symbol.Library.empty() ? guestNames.end() : findGuest(symbol.Library); + const auto shared = sharedExports.find(symbol.Name); + if (shared == sharedExports.end() || declared == guestNames.end() || !shared->second.contains(declared->second)) + rejectSharedImport(symbol.Name, Io::Utf8Path(images[index].SourcePath)); const auto found = exports.find(symbol.Name); std::vector providers; if (found != exports.end()) { for (const auto provider : found->second) { - const auto declared = findGuest(symbol.Library); - if (!windows || symbol.Library.empty() || (declared != guestNames.end() && declared->second == provider)) providers.push_back(provider); + if (symbol.Library.empty() || (declared != guestNames.end() && declared->second == provider)) providers.push_back(provider); } } if (providers.size() > 1) throw Domain::RelinkerException("Ambiguous guest import after stripping #: " + symbol.Name); From 364c8936da0fe0ce884ef3396de280968b890fe1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Adri=C3=A0=20Franch?= Date: Sun, 11 Oct 2026 10:36:00 +0200 Subject: [PATCH 458/460] test(relinker): pin executable import binding to the needed provider --- .../relinker/tests/test_guest_symbol_names.py | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/core/relinker/relinker/tests/test_guest_symbol_names.py b/core/relinker/relinker/tests/test_guest_symbol_names.py index e37ca64ca..e9853120e 100644 --- a/core/relinker/relinker/tests/test_guest_symbol_names.py +++ b/core/relinker/relinker/tests/test_guest_symbol_names.py @@ -165,6 +165,19 @@ def main(): imports = imported_symbols(declared.read_bytes()) assert imports == {"AAAAAAAAAAA#guest", "EEEEEEEEEEE"}, imports + twins = {"libc.prx": guest_module([("AAAAAAAAAAA", True)]), "other.prx": guest_module([("AAAAAAAAAAA", True)])} + bound = relink(relinker, work / "needed-one", importing_executable(["AAAAAAAAAAA#B#B"], ["libc"]), twins) + imports = imported_symbols(bound.read_bytes()) + assert imports == {"AAAAAAAAAAA#guest"}, imports + + (work / "needed-both" / "sce_module").mkdir(parents=True) + (work / "needed-both" / "input.elf").write_bytes(importing_executable(["AAAAAAAAAAA#B#B"], ["libc", "other"])) + for name, image in twins.items(): + (work / "needed-both" / "sce_module" / name).write_bytes(image) + result = subprocess.run([str(relinker), str(work / "needed-both" / "input.elf"), str(work / "needed-both" / "output.elf")], + capture_output=True, text=True, timeout=30) + assert result.returncode != 0 and "Ambiguous guest import AAAAAAAAAAA" in result.stderr, (result.stdout, result.stderr) + guests = relink(relinker, work / "guest-declared", importing_executable([]), { "b.prx": provider(7), "c.prx": consumer("libSceLibcInternal.prx", module_name="libSceLibcInternal"), From c5bcc085ba2c1e8750fb719efae78481dd018183 Mon Sep 17 00:00:00 2001 From: SynnZer0 <243459655+SynnZer0@users.noreply.github.com> Date: Sun, 11 Oct 2026 02:01:11 -0400 Subject: [PATCH 459/460] feat(libSceNpWebApi2): implement sceNpWebApi2GetMemoryPoolStats PPSA21703's Unity PSN plugin imports sceNpWebApi2GetMemoryPoolStats (Xweb+naPZ8Y), so the title did not start. sceNpWebApi2Initialize now records the pool size of each context and sceNpWebApi2Terminate forgets it. The statistics report that size with no peak or current use, as sceHttp2GetMemoryPoolStats does, since nothing is allocated from the pool. A null output or an unknown context throws. The signature and layout follow the Http2 statistics and are recorded in TechnicalDebt. The library gets its own CMakeLists.txt, equivalent to the generic rule, to register its test. Co-Authored-By: Claude Opus 5.5 --- core/libs/prx/libSceNpWebApi2/CMakeLists.txt | 16 +++++++ core/libs/prx/libSceNpWebApi2/Export.cpp | 31 ++++++++++++-- core/libs/tests/GuestNpWebApi2.cpp | 45 ++++++++++++++++++++ docs/dev/TechnicalDebt.md | 1 + 4 files changed, 90 insertions(+), 3 deletions(-) create mode 100644 core/libs/prx/libSceNpWebApi2/CMakeLists.txt create mode 100644 core/libs/tests/GuestNpWebApi2.cpp diff --git a/core/libs/prx/libSceNpWebApi2/CMakeLists.txt b/core/libs/prx/libSceNpWebApi2/CMakeLists.txt new file mode 100644 index 000000000..1fbafd283 --- /dev/null +++ b/core/libs/prx/libSceNpWebApi2/CMakeLists.txt @@ -0,0 +1,16 @@ +add_library(libSceNpWebApi2 SHARED EXCLUDE_FROM_ALL Export.cpp) +target_include_directories(libSceNpWebApi2 PRIVATE ${LIBS_INCLUDE_DIR}) +target_link_libraries(libSceNpWebApi2 PRIVATE libc libkernel) +configure_windows_unwind(libSceNpWebApi2) + +if(BUILD_TESTING) + add_test_executable(guest_np_web_api2_tests ${LIBS_INCLUDE_DIR}/tests/GuestNpWebApi2.cpp) + target_include_directories(guest_np_web_api2_tests PRIVATE ${LIBS_INCLUDE_DIR}) + target_link_libraries(guest_np_web_api2_tests PRIVATE libSceNpWebApi2 libkernel libc) + configure_windows_unwind(guest_np_web_api2_tests) + add_test(NAME guest_np_web_api2 COMMAND guest_np_web_api2_tests) + if(WIN32) + set_tests_properties(guest_np_web_api2 PROPERTIES ENVIRONMENT_MODIFICATION + "PATH=path_list_prepend:$") + endif() +endif() diff --git a/core/libs/prx/libSceNpWebApi2/Export.cpp b/core/libs/prx/libSceNpWebApi2/Export.cpp index 2c3e02d70..1f12e8b1e 100644 --- a/core/libs/prx/libSceNpWebApi2/Export.cpp +++ b/core/libs/prx/libSceNpWebApi2/Export.cpp @@ -3,12 +3,25 @@ #include "SceTypes.hpp" #include "prx/libc/include/General.hpp" #include +#include +#include +#include // The PSN web API has no service behind it: handles can be created, but requests fail. static constexpr int SCE_NP_WEBAPI2_ERROR_INVALID_ARGUMENT = static_cast(0x80553402); static constexpr int SCE_NP_WEBAPI2_ERROR_UNAVAILABLE = static_cast(0x80553406); static std::atomic g_nextHandle{1}; +struct NpWebApi2MemoryPoolStats { + size_t pool_size; + size_t max_inuse_size; + size_t current_inuse_size; + int32_t reserved; +}; + +static std::mutex g_poolsMutex; +static std::unordered_map g_pools; + extern "C" { int APS5_VABI sceNpWebApi2AbortRequest(int64_t request_id) { @@ -70,8 +83,19 @@ int APS5_VABI sceNpWebApi2GetHttpResponseHeaderValueLength(int64_t request_id, c int APS5_VABI sceNpWebApi2Initialize(int lib_http_ctx_id, size_t pool_size) { (void)lib_http_ctx_id; - (void)pool_size; - return g_nextHandle.fetch_add(1, std::memory_order_relaxed); + const int lib_ctx_id = g_nextHandle.fetch_add(1, std::memory_order_relaxed); + std::lock_guard lock(g_poolsMutex); + g_pools[lib_ctx_id] = pool_size; + return lib_ctx_id; +} + +int APS5_VABI sceNpWebApi2GetMemoryPoolStats(int lib_ctx_id, NpWebApi2MemoryPoolStats* stats) { + if (stats == nullptr) APS5_INVALID_ARG_EX; + std::lock_guard lock(g_poolsMutex); + const auto pool = g_pools.find(lib_ctx_id); + if (pool == g_pools.end()) throw std::invalid_argument("sceNpWebApi2GetMemoryPoolStats: unknown context"); + *stats = {pool->second, 0, 0, 0}; + return 0; } int APS5_VABI sceNpWebApi2PushEventCreateFilter(int lib_ctx_id, int handle_id, const char* np_service_name, uint32_t np_service_label, const void* filter_param, size_t filter_param_num) { @@ -125,7 +149,8 @@ int APS5_VABI sceNpWebApi2SendRequest(int64_t request_id, const void* data, size } int APS5_VABI sceNpWebApi2Terminate(int lib_ctx_id) { - (void)lib_ctx_id; + std::lock_guard lock(g_poolsMutex); + g_pools.erase(lib_ctx_id); return 0; } diff --git a/core/libs/tests/GuestNpWebApi2.cpp b/core/libs/tests/GuestNpWebApi2.cpp new file mode 100644 index 000000000..ea167e654 --- /dev/null +++ b/core/libs/tests/GuestNpWebApi2.cpp @@ -0,0 +1,45 @@ +#include "SceTypes.hpp" +#include +#include +#include +#include + +struct NpWebApi2MemoryPoolStats { + std::size_t pool_size; + std::size_t max_inuse_size; + std::size_t current_inuse_size; + std::int32_t reserved; +}; + +extern "C" { +int APS5_VABI sceNpWebApi2Initialize(int, std::size_t); +int APS5_VABI sceNpWebApi2Terminate(int); +int APS5_VABI sceNpWebApi2GetMemoryPoolStats(int, NpWebApi2MemoryPoolStats*); +} + +static void Require(bool value) { if (!value) std::abort(); } + +template +static bool Throws(TCall call) { + try { + call(); + } catch (const std::exception&) { + return true; + } + return false; +} + +int main() { + const int context = sceNpWebApi2Initialize(1, 0x20000); + const int other = sceNpWebApi2Initialize(1, 0x8000); + Require(context > 0 && other > 0 && context != other); + NpWebApi2MemoryPoolStats stats{1, 1, 1, 1}; + Require(sceNpWebApi2GetMemoryPoolStats(context, &stats) == 0); + Require(stats.pool_size == 0x20000 && stats.max_inuse_size == 0 && stats.current_inuse_size == 0 && stats.reserved == 0); + Require(sceNpWebApi2GetMemoryPoolStats(other, &stats) == 0 && stats.pool_size == 0x8000); + Require(Throws([&] { sceNpWebApi2GetMemoryPoolStats(context, nullptr); })); + Require(sceNpWebApi2Terminate(context) == 0); + Require(Throws([&] { sceNpWebApi2GetMemoryPoolStats(context, &stats); })); + Require(sceNpWebApi2Terminate(other) == 0); + return 0; +} diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index e6360c572..e69c8144b 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -222,6 +222,7 @@ Throughout the project, every function at every stage either **does exactly what - [sceKernelGetOperationMode](../../core/libs/prx/libkernel/System/src/Process.cpp) (libkernel) - the signature `(int* mode, int* submode)` comes from PPSA12544, which logs both values after the call; the values are unknown and 0 is reported for both, as in the [prosper](https://github.com/mattias800/prosper/pull/4153) reimplementation. Null outputs throw - Font, Http2, Net, Ssl, SystemService and libkernel exports imported by PPSA12544 and added without an implementation have unknown signatures - [sceHttp2GetMemoryPoolStats, sceSslGetMemoryPoolStats, sceNetGetMemoryPoolStats](../../core/libs/prx/libSceNet/Export.cpp) (libSceHttp2, libSceSsl, libSceNet) - the signatures `(int id, stats*)`, the layouts (pool size, peak and current use, and for Http2 and Ssl a 32-bit reserved field) and the error codes were measured on a PS5 (firmware 13.60). The pool size is the one given at init or pool creation; nothing is allocated from these pools, so the peak and current use are 0, where the console already reports use right after initialization (0x15C0 bytes for Ssl, 0x1640 for Http2 and 4 for a Net pool) +- [sceNpWebApi2GetMemoryPoolStats](../../core/libs/prx/libSceNpWebApi2/Export.cpp) (libSceNpWebApi2) - imported by PPSA21703's Unity PSN plugin. The signature `(int libCtxId, stats*)` and the layout of the Http2 statistics above (measured on a PS5 for Http2, Ssl and Net) are assumed for this library, not measured. The pool size is the one given to `sceNpWebApi2Initialize`, and the peak and current use are 0. The error codes are unknown, so a null output or an unknown context throws - [sceSslGetCaList, sceSslFreeCaList, sceSslUnloadCert](../../core/libs/prx/libSceSsl/Export.cpp) (libSceSsl) - signatures and the 16-byte `SceSslCaList` layout (an array of certificate handles and a 32-bit count) follow [SharpProspero](https://github.com/SvenGDK/SharpProspero/blob/5dfdb4b0b166e83dc9ae275001afb695f12c7174/src/SharpProspero/Interop/Net/Http.cs) and shadPS4's `OrbisSslCaList`, not confirmed for the PS5. On the console the list holds the certificate authorities the SSL service has; [shadPS4](https://github.com/shadps4-emu/shadPS4/pull/4104) returns a placeholder certificate from `sceSslGetCaCerts` because Rise of the Tomb Raider fails its initialization checks without one. No root certificate bundle is shipped and no loaded certificate is kept, so here a context holds none: `sceSslGetCaList` clears the list and returns `NOT_FOUND`, as `sceSslGetCaCerts` does, `sceSslFreeCaList` accepts only that cleared list, and `sceSslUnloadCert` (imported by PPSA15508) has nothing to drop and is assumed to return 0. A null list returns `0x8095177A`, which `sceSslGetCaCerts` and [KytyPS5](https://github.com/KytyPS5/KytyPS5/blob/59183dbfa81ed7efdff6069177fed50de0552725/src/libs/network.cpp) return for a null argument (shadPS4, PS4, returns `0x8095F007`). An unknown context throws (KytyPS5 returns `SSL_ERROR_INVALID_ID`, `0x8095F006`, from `sceSslGetCaCerts`), and so does a list `sceSslGetCaList` did not return - [sceHttp2SetResolveRetry](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) - signature `(int id, int32_t retry)` from libSceHttp's `sceHttpSetResolveRetry`; the value is accepted and ignored like the other Http2 settings, since no request reaches the network - [sceHttp2WebSocketCreateRequest, sceHttp2WebSocketCloseAsync, sceHttp2WebSocketSendTextMessageAsync, sceHttp2WebSocketSendDataMessageAsync](../../core/libs/prx/libSceHttp2/Export.cpp) (libSceHttp2) - imported by PPSA16064 and not called during its start-up; unknown signatures From 31906bf9b0018354976be45d18c06df43707b8c2 Mon Sep 17 00:00:00 2001 From: pushkarreddyy <212133856+pushkarreddyy@users.noreply.github.com> Date: Sun, 11 Oct 2026 12:36:33 +0530 Subject: [PATCH 460/460] feat(libc): implement nan and nanf Add standard quiet NaN generators nanf (DZU+K1wozGI) and nan (zck+6bVj5pA) to libc. Resolves startup failure in VLC for PS5 (PPSA85300) where execution halts at nanf on Linux once strsep is available. --- core/libs/prx/libc/src/Math.cpp | 9 +++++++++ core/libs/tests/GuestMath.cpp | 10 ++++++++++ docs/dev/TechnicalDebt.md | 1 + 3 files changed, 20 insertions(+) diff --git a/core/libs/prx/libc/src/Math.cpp b/core/libs/prx/libc/src/Math.cpp index 8bf4d3d6e..259c13735 100644 --- a/core/libs/prx/libc/src/Math.cpp +++ b/core/libs/prx/libc/src/Math.cpp @@ -1,4 +1,5 @@ #include +#include #include #include #include @@ -125,6 +126,14 @@ int APS5_VABI __isfinite_nid_postfix(double x) { return std::isfinite(x) ? 1 : 0 int APS5_VABI __isnan_nid_postfix(double x) { return std::isnan(x) ? 1 : 0; } int APS5_VABI __isinf_nid_postfix(double x) { return std::isinf(x) ? 1 : 0; } int APS5_VABI __signbit_nid_postfix(double x) { return std::signbit(x) ? 1 : 0; } +float APS5_VABI nanf_nid_postfix(const char* tagp) { + (void)tagp; + return std::bit_cast(0x7fc00000u); +} +double APS5_VABI nan_nid_postfix(const char* tagp) { + (void)tagp; + return std::bit_cast(0x7ff8000000000000ull); +} double APS5_VABI modf_nid_postfix(double x, double* integral) { return std::modf(x, integral); } float APS5_VABI modff_nid_postfix(float x, float* integral) { return std::modf(x, integral); } diff --git a/core/libs/tests/GuestMath.cpp b/core/libs/tests/GuestMath.cpp index 7f668ef75..84ea0db56 100644 --- a/core/libs/tests/GuestMath.cpp +++ b/core/libs/tests/GuestMath.cpp @@ -78,6 +78,8 @@ int APS5_VABI __isinff_nid_postfix(float); int APS5_VABI __isinf_nid_postfix(double); std::lldiv_t APS5_VABI lldiv_nid_postfix(long long, long long); std::lldiv_t APS5_VABI ldiv_nid_postfix(std::int64_t, std::int64_t); +float APS5_VABI nanf_nid_postfix(const char*); +double APS5_VABI nan_nid_postfix(const char*); } static void Require(bool value) { if (!value) std::abort(); } @@ -357,6 +359,14 @@ int main() { Require(fmodf_nid_postfix(-5.5f, 2.f) == -1.5f); Require(std::signbit(fmodf_nid_postfix(-4.f, 2.f))); Require(std::isnan(fmodf_nid_postfix(1.f, 0.f))); + Require(std::bit_cast(nanf_nid_postfix(nullptr)) == 0x7fc00000u); + Require(std::bit_cast(nanf_nid_postfix("")) == 0x7fc00000u); + Require(std::bit_cast(nanf_nid_postfix("123")) == 0x7fc00000u); + Require(!std::signbit(nanf_nid_postfix(""))); + Require(std::bit_cast(nan_nid_postfix(nullptr)) == 0x7ff8000000000000ull); + Require(std::bit_cast(nan_nid_postfix("")) == 0x7ff8000000000000ull); + Require(std::bit_cast(nan_nid_postfix("123")) == 0x7ff8000000000000ull); + Require(!std::signbit(nan_nid_postfix(""))); Require(std::abs(asinf_nid_postfix(0.5f) - 0.5235988f) < 0.000001f); Require(std::abs(acosf_nid_postfix(0.5f) - 1.0471976f) < 0.000001f); Require(std::abs(atan2f_nid_postfix(1.f, -1.f) - 2.3561945f) < 0.000001f); diff --git a/docs/dev/TechnicalDebt.md b/docs/dev/TechnicalDebt.md index e6360c572..fabec5f7d 100644 --- a/docs/dev/TechnicalDebt.md +++ b/docs/dev/TechnicalDebt.md @@ -354,6 +354,7 @@ Throughout the project, every function at every stage either **does exactly what - [/dev/random and /dev/urandom](../../core/libs/prx/libkernel/File/src/RandomDevice.cpp) (libkernel) opened for reading return a descriptor backed by the host null device whose reads are filled from `std::random_device`; both paths behave the same, never block, and positioned reads ignore the offset (FreeBSD `random(4)`). Only the exact paths are recognised, a `dup`ed descriptor reads as an empty file, `fstat`, `lseek`, `mmap` and `select` see the null device, and opening them for writing throws - [Wide-character conversions](../../core/libs/prx/libc/src/Multibyte.cpp) (libc) - The wide `wcsto*` functions parse the ASCII prefix of the guest string, as the C locale accepts only ASCII white space and digits. [`wcsrtombs`](../../core/libs/prx/libc/src/Multibyte.cpp) uses the guest's single-byte C locale through `wcrtomb`; other encodings and shift states are not supported - [realpath](../../core/libs/prx/libc/src/RealPath.cpp) (libc) follows FreeBSD 11's [`realpath.c`](https://github.com/freebsd/freebsd-src/blob/stable/11/lib/libc/stdlib/realpath.c) except for symlinks. The console's guest filesystem has none, so a host symlink under the guest root is followed for existence and type but not expanded, and `..` drops the previous component lexically, as `Resolve` does for every guest path. Through a host link `linkdir -> sub/deep`, `linkdir/../x` gives `x` beside the link, not beside its target. +- [nan and nanf](../../core/libs/prx/libc/src/Math.cpp) (libc) return fixed quiet NaN values (`0x7fc00000` and `0x7ff8000000000000`), ignoring the tag argument as observed in the console's `libc.prx` (both exports load a global constant via `mov rax,[rip+..]; vmovsd xmm0,[rax]; ret`); VLC (PPSA85300) stops on nanf at start-up after strsep. - [SPIRV-Tools optimization](../../core/shader/recompiler/SpirvBackend/src/SpirvOptimizer.cpp) skips CCP and Simplification for modules with storage-only 8- or 16-bit types because the pinned folder produces illegal narrow constants. This also disables unrelated constant folding; the restricted full pipeline mirrors the pinned performance recipe. - [SRT signed comparison](../../core/shader/recompiler/Optimization/src/SrtWalker/SrtEvaluator.cpp) evaluates `SLessThan32` as signed 32-bit less-than, matching the existing SPIR-V backend and [SPIR-V `OpSLessThan`](https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html#OpSLessThan). Signed boundary descriptor expressions are tested on the host; native RDNA2/PS5 execution and title behavior remain unverified. - On macOS the [write watch](../../core/libs/prx/libc/src/GuestWriteWatch.cpp) write-protects watched pages and records the first write to each from a SIGBUS/SIGSEGV handler, because macOS has no counterpart to `userfaultfd` write-protect or `MEM_WRITE_WATCH`. The protection is real, so: