shifathosn
/
AnyPS5
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into fix/libscenet-epoll-out

This commit is contained in:
Yunus Shaikh 2026-10-12 11:56:48 -07:00
parent b19a74f5bf a60f52e318
commit 78d594d9f6
218 changed files with 5562 additions and 1037 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/pull_request_template.md vendored
View File

@ -8,7 +8,7 @@
- [ ] Based on current `main`; no other open PR implements the same functions
- [ ] One topic per PR; follow-ups go in a new PR
- [ ] No comments in code except [technical debt](https://github.com/boykopovar/AnyPS5/blob/main/docs/dev/CONVENTIONS.md)
- [ ] Unimplemented paths throw (`NotImplemented_nid_no_patch`); silent stubs are listed in [TechnicalDebt](https://github.com/boykopovar/AnyPS5/blob/main/docs/dev/TechnicalDebt.md#silent-stubs)
- [ ] Unimplemented paths throw (`NotImplemented_nid_no_patch`); silent stubs are listed under Silent stubs in their library's [technical debt](https://github.com/boykopovar/AnyPS5/tree/main/docs/dev/debt) file
- [ ] No notes, investigation `.md` files or images added to the repository (attach them to this PR)
- [ ] New third-party code is a submodule built from source
- [ ] Generic behaviour, not specific to one title

1
.github/workflows/labels.yml vendored
View File

@ -35,3 +35,4 @@ jobs:
label prx c5def5 'Other PRX libraries'
label build bfd4f2 'CMake, submodules, 3rdparty'
label docs 0075ca 'Documentation'
label stale ededed 'Conflicting with main and no activity for 7 days'

28
.github/workflows/stale-conflicts.yml vendored Normal file
View File

@ -0,0 +1,28 @@
name: Stale conflicts
on:
schedule:
- cron: '30 3 * * *'
workflow_dispatch:
permissions: {}
jobs:
warn:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v11
with:
only-pr-labels: needs rebase
days-before-issue-stale: -1
days-before-issue-close: -1
days-before-pr-stale: 7
days-before-pr-close: -1
stale-pr-label: stale
stale-pr-message: 'This pull request has conflicted with `main` and had no activity for 7 days. Rebase it on current `main` so it can be reviewed and merged. It will not be closed automatically.'
remove-pr-stale-when-updated: true
exempt-draft-pr: true
operations-per-run: 200

5
CMakeLists.txt
View File

@ -82,6 +82,8 @@ if(BUILD_TESTING)
${CMAKE_CURRENT_SOURCE_DIR}/tools/tests/test_check_skips.py)
add_test(NAME stubgen COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/tools/tests/test_stubgen.py)
add_test(NAME debt_split COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/tools/tests/test_debt_split.py)
add_test(NAME hardware_oracle COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/tools/hw-oracle/test_hw_oracle.py)
add_test(NAME pr_overlap_exports COMMAND ${Python3_EXECUTABLE}
@ -92,6 +94,9 @@ if(BUILD_TESTING)
${CMAKE_CURRENT_SOURCE_DIR}/tools/tests/test_nid_names.py)
add_test(NAME release_package COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/tools/tests/test_package_release.py)
add_test(NAME pad_input COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/tools/tests/test_pad_input.py)
set_tests_properties(pad_input PROPERTIES TIMEOUT 20)
endif()
endif()

7
CONTRIBUTING.md
View File

@ -5,7 +5,7 @@
- Follow the [coding conventions](docs/dev/CONVENTIONS.md): naming, no comments except [technical debt](docs/dev/TechnicalDebt.md), Conventional Commits.
- Every function either does exactly what it is supposed to or throws. Unimplemented exports call `NotImplemented_nid_no_patch(__func__)` (see [libSceAudioIn](core/libs/prx/libSceAudioIn/Export.cpp)).
- Pull requests that add or change shader instruction semantics say where they come from: measured on hardware (which GPU and what was checked, e.g. with the [hardware oracle](docs/dev/HW_ORACLE.md)) or the exact source (ISA section, LLVM, ACO or Mesa file). Reviewers check the semantics on hardware. Cases the source doesn't settle throw, and behaviour not verified on hardware is recorded in [technical debt](docs/dev/TechnicalDebt.md).
- A silent stub is allowed only when it unblocks a title and only affects the UI; add it to [silent stubs](docs/dev/TechnicalDebt.md#silent-stubs).
- A silent stub is allowed only when it unblocks a title and only affects the UI; add it under Silent stubs in its library's [technical debt](docs/dev/debt/) file.
- Missing imports fail at startup with a clear error; don't replace them with fallbacks that keep running.
- Implement the general behaviour of a function, not what one title happens to need.
- Don't add replacements for modules that titles ship themselves in `sce_module/`, `sce_modules/` or `prx/`: engine or middleware modules (Cohtml, FMOD, GOG Galaxy) and SDK libraries that only wrap other system libraries (NpCppWebApi over NpWebApi2). The relinker converts and loads the title's own module, and a host library with the same name is left out of `DT_NEEDED`. Only system libraries, which reach the kernel or the hardware, are reimplemented in [core/libs/prx](core/libs/prx).
@ -39,7 +39,7 @@ gh workflow run build.yml --repo boykopovar/AnyPS5 --ref main --field pr=1234
This manual run also builds and tests the relinker on Linux, Windows and macOS, regardless of the author's previous contributions or the changed files. The relinker matrix also runs automatically when a pull request targeting `main` is opened, updated or reopened, if its author already has a commit in `main` and it changes a file the relinker-only build uses; full builds never run automatically for pull requests. Every job that runs automatically for pull requests runs only when the pull request changes its files: a `paths` filter on the `pull_request` trigger, or `needs: eligibility` with an `if` on the files output it sets in `build.yml`. The Conventions check fails a workflow change that adds a job without one. All build jobs build the selected pull request merged into the current `main` (GitHub's merge commit), so a pull request that no longer merges cleanly must be rebased first. New commits require another manual run for full builds.
The Conventions check runs automatically for every contributor, including first-time contributors. It uses the base branch checker and reads pull request Git objects without executing pull request code. The Conventions check runs on every pull request and fails when a rule on this page is broken. It accepts code comments only when the pull request also changes [TechnicalDebt](docs/dev/TechnicalDebt.md), and only UTF-8 text files. Run it locally before pushing:
The Conventions check runs automatically for every contributor, including first-time contributors. It uses the base branch checker and reads pull request Git objects without executing pull request code. The Conventions check runs on every pull request and fails when a rule on this page is broken. It accepts code comments only when the pull request also changes [technical debt](docs/dev/TechnicalDebt.md), the index or a file in `docs/dev/debt/`, and only UTF-8 text files. Run it locally before pushing:
```
python3 tools/check_conventions.py --base origin/main
@ -53,6 +53,7 @@ The [pull request template](.github/pull_request_template.md) is the checklist f
- Open pull requests per author are limited by how many of theirs are merged into `main`: 1 with none merged, 20 with at least 1, no limit from 20. A pull request opened or reopened over the limit is closed automatically; reopen it once another one is merged or closed.
- Before starting, check that no open pull request already implements the same functions.
- Keep the branch up to date with `main` and resolve conflicts yourself; rebasing and force-pushing is fine.
- A pull request that conflicts with `main` and has no activity for 7 days is labeled `stale` with a reminder. It is not closed.
- If a pull request needs another one first, say so on the checklist line (`Depends on: #N`; `none` otherwise).
- If a pull request fixes a stop or a wrong result seen on a title, name it on the checklist line with the title ID and its [Game bring-up](https://github.com/boykopovar/AnyPS5/discussions/categories/game-bring-up) discussion (`Unblocks: PPSA01325 #3232`; `none` otherwise). Reviewers review these first.
- Fixes a title needs go to `main` as pull requests. A Game bring-up discussion lists the open pull requests and local changes the title still needs; update the list when one is merged, closed or replaced.
@ -63,4 +64,4 @@ The [pull request template](.github/pull_request_template.md) is the checklist f
## Documentation
Text in the repository is dry, strict and concise. Record known gaps, unknown NIDs and signatures in [technical debt](docs/dev/TechnicalDebt.md) instead of new files.
Text in the repository is dry, strict and concise. Record known gaps, unknown NIDs and signatures in [technical debt](docs/dev/TechnicalDebt.md) instead of new files. An entry goes in `docs/dev/debt/<name>.md`, named after the first `core/` path it links (`<name>` for `core/libs/prx/<name>/...`, otherwise for `core/<name>/...`), under the same `###` sections as the index; entries that link neither stay in the index. A pull request whose `TechnicalDebt.md` change conflicts after the split rebases and runs `python3 tools/debt_split.py resolve` during the stopped rebase, which moves its added, edited and removed entries to their files; `python3 tools/debt_split.py split` moves entries left in the index.

27
core/libs/prx/libSceAgcDriver/CMakeLists.txt
View File

@ -633,6 +633,19 @@ target_link_libraries(agc_formatted_buffer_once_tests PRIVATE shader_recompiler_
set_target_properties(agc_formatted_buffer_once_tests PROPERTIES CXX_EXTENSIONS OFF)
configure_windows_unwind(agc_formatted_buffer_once_tests)
add_test_executable(agc_rect_list_position_invariant_tests EXCLUDE_FROM_ALL tests/RectListPositionInvariant.cpp)
target_link_libraries(agc_rect_list_position_invariant_tests PRIVATE shader_recompiler_agc_driver libc)
set_target_properties(agc_rect_list_position_invariant_tests PROPERTIES CXX_EXTENSIONS OFF)
configure_windows_unwind(agc_rect_list_position_invariant_tests)
if(BUILD_TESTING)
set_target_properties(agc_rect_list_position_invariant_tests PROPERTIES EXCLUDE_FROM_ALL OFF)
add_test(NAME agc_rect_list_position_invariant COMMAND agc_rect_list_position_invariant_tests)
if(WIN32)
set_tests_properties(agc_rect_list_position_invariant PROPERTIES ENVIRONMENT_MODIFICATION
"PATH=path_list_prepend:$<TARGET_FILE_DIR:libc>")
endif()
endif()
add_test_executable(agc_shader_disk_cache_tests EXCLUDE_FROM_ALL tests/ShaderDiskCache.cpp)
target_include_directories(agc_shader_disk_cache_tests PRIVATE ${LIBS_INCLUDE_DIR})
target_link_libraries(agc_shader_disk_cache_tests PRIVATE shader_recompiler_agc_driver libc)
@ -858,6 +871,20 @@ if(BUILD_TESTING)
endif()
endif()
add_test_executable(agc_host_import_retire_tests EXCLUDE_FROM_ALL tests/HostImportRetireHold.cpp)
target_include_directories(agc_host_import_retire_tests PRIVATE ${LIBS_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/core/shader/recompiler ${ANYPS5_VULKAN_HEADERS_INCLUDE_DIR})
target_link_libraries(agc_host_import_retire_tests PRIVATE libSceAgcDriver libc)
set_target_properties(agc_host_import_retire_tests PROPERTIES CXX_EXTENSIONS OFF)
configure_windows_unwind(agc_host_import_retire_tests)
if(BUILD_TESTING)
set_target_properties(agc_host_import_retire_tests PROPERTIES EXCLUDE_FROM_ALL OFF)
add_test(NAME agc_host_import_retire COMMAND agc_host_import_retire_tests)
if(WIN32)
set_tests_properties(agc_host_import_retire PROPERTIES ENVIRONMENT_MODIFICATION
"PATH=path_list_prepend:$<TARGET_FILE_DIR:libc>")
endif()
endif()
add_test_executable(agc_driver_pm4_tests EXCLUDE_FROM_ALL
tests/Pm4.cpp
)

1
core/libs/prx/libSceAgcDriver/Execution/include/GuestMemory.hpp
View File

@ -150,6 +150,7 @@ GpuMutexType& GpuMutex();
// work the hold deferred to run without it (the recorder's release of the objects completed batches
// kept). One hook, set once; it must not take GpuMutex itself.
void SetGpuUnlockHook(void (*hook)());
std::uint64_t GpuHoldSerial();
// Debug aid (APS5_ASSERT_GPU_LOCK=1): aborts with a [lock] line naming `where` when the calling
// thread does not hold GpuMutex. Recording, submits, command batches and the detiler's pools must
// only be reached under it (a resource build's stage A runs without it, see ShaderResources).

3
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/BufferFill.cpp
View File

@ -49,8 +49,9 @@ std::optional<DwordPatternFill> MatchDwordPatternFill(std::span<const std::uint3
const bool swizzled = ((userData[1] >> 31u) & 1u) != 0;
const bool addTid = ((userData[3] >> 23u) & 1u) != 0;
const auto type = userData[3] >> 30u;
const auto format = (userData[3] >> 12u) & 0x7fu;
const auto period = userData[9];
if (type != 0 || stride != 4 || swizzled || addTid || (period != 1u && period != 2u && period != 4u) || (packet[4] & 0x20u) != 0 || packet[2] != 1 || packet[3] != 1) return std::nullopt;
if (type != 0 || stride != 4 || swizzled || addTid || format < 20u || format > 22u || (period != 1u && period != 2u && period != 4u) || (packet[4] & 0x20u) != 0 || packet[2] != 1 || packet[3] != 1) return std::nullopt;
const auto dwords = std::min<std::uint64_t>({static_cast<std::uint64_t>(packet[1]) * 64u, userData[8], userData[2]});
const auto base = userData[0] | (static_cast<std::uint64_t>(userData[1] & 0xffffu) << 32u);
if (dwords == 0 || dwords % 4u != 0 || base % 16u != 0) return std::nullopt;

8
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
View File

@ -1597,11 +1597,13 @@ const void* ThreadToken() {
thread_local char token;
return &token;
}
std::atomic<std::uint64_t> gpuHoldSerial{0};
}
void GpuMutexType::acquired() {
owner.store(ThreadToken(), std::memory_order_relaxed);
++depth;
if (++depth == 1) gpuHoldSerial.fetch_add(1, std::memory_order_relaxed);
}
bool GpuMutexType::try_lock() {
@ -1632,6 +1634,10 @@ void SetGpuUnlockHook(void (*hook)()) {
gpuUnlockHook.store(hook, std::memory_order_release);
}
std::uint64_t GpuHoldSerial() {
return gpuHoldSerial.load(std::memory_order_relaxed);
}
void GpuMutexType::unlock() {
const bool outermost = --depth == 0;
if (outermost) {

1
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
View File

@ -2965,6 +2965,7 @@ ShaderRecompiler::SpirvTarget VulkanDevice::buildTarget() const {
target.fixedPushSlots = state->graphicsPipelineLibrary;
target.registrationModules = state->graphicsPipelineLibrary;
target.srgbDecodeFormats = state->srgbDecodeFormats;
target.vertexSubgroups = (state->subgroup.supportedStages & VK_SHADER_STAGE_VERTEX_BIT) != 0;
if (state->meshShader) {
const auto& mesh = state->meshLimits;
target.mesh = ShaderRecompiler::MeshTargetLimits{{mesh.maxMeshWorkGroupSize[0], mesh.maxMeshWorkGroupSize[1], mesh.maxMeshWorkGroupSize[2]}, mesh.maxMeshWorkGroupInvocations, std::min(mesh.maxMeshSharedMemorySize, mesh.maxMeshPayloadAndSharedMemorySize), mesh.maxMeshOutputVertices, mesh.maxMeshOutputPrimitives, mesh.maxMeshOutputComponents, std::min(mesh.maxMeshOutputMemorySize, mesh.maxMeshPayloadAndOutputMemorySize), mesh.meshOutputPerVertexGranularity, mesh.meshOutputPerPrimitiveGranularity};

4
core/libs/prx/libSceAgcDriver/Graphics/include/Recorder.hpp
View File

@ -86,6 +86,7 @@ public:
// Whether recorded work still has completion actions (write-backs the CPU must see) to run.
bool HasCompletions() const;
void Keep(std::shared_ptr<void> object, std::size_t bytes = 0);
void KeepThroughHold(std::shared_ptr<void> object);
void KeepBytes(const void* owner, std::size_t bytes);
std::size_t OpenKeptBytes() const { return open != nullptr ? open->keptBytes : 0; }
void FlushDeferredReleases();
@ -715,6 +716,7 @@ private:
void recordBarrier(VkCommandBuffer commands, VkPipelineStageFlags sourceStage, VkPipelineStageFlags destinationStage, VkAccessFlags sourceAccess, VkAccessFlags destinationAccess) const;
// Opens a batch when none is open, without closing a store run (Commands() does both).
void ensureOpen();
void dropStaleHoldKept();
// Records the run's pending store (vkCmdUpdateBuffer), if any.
void flushPendingStore();
// Records the pending store and the run's trailing barrier; the next store starts a new run.
@ -782,6 +784,8 @@ private:
// table is empty (most of the time between label groups).
std::atomic<std::size_t> recordedLabels{0};
std::unique_ptr<Batch> open;
std::vector<std::shared_ptr<void>> holdKept;
std::uint64_t holdKeptSerial = 0;
std::deque<std::unique_ptr<Batch>> inFlight;
// Batches popped from inFlight whose completions have not run yet: their writes stay in the
// snapshot (a rebuild from inside a completion must not drop them) until finish returns.

37
core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
View File

@ -187,14 +187,17 @@ void destroyImport(const Context& context, const HostImport& entry) {
#endif
}
using ImportNode = std::map<std::uint64_t, HostImport>::node_type;
// Frees a dropped import's Vulkan objects when it is released. Never copied: a copy would destroy the
// same handles twice (and a temporary would destroy them at once).
struct RetiredImport {
RetiredImport(VkDevice device, PFN_vkDestroyBuffer destroyBuffer, PFN_vkFreeMemory freeMemory, const HostImport& entry)
: device(device), destroyBuffer(destroyBuffer), freeMemory(freeMemory), entry(entry) {}
RetiredImport(VkDevice device, PFN_vkDestroyBuffer destroyBuffer, PFN_vkFreeMemory freeMemory, ImportNode node)
: device(device), destroyBuffer(destroyBuffer), freeMemory(freeMemory), node(std::move(node)) {}
RetiredImport(const RetiredImport&) = delete;
RetiredImport& operator=(const RetiredImport&) = delete;
~RetiredImport() {
const auto& entry = node.mapped();
destroyBuffer(device, entry.buffer, nullptr);
freeMemory(device, entry.memory, nullptr);
#ifdef _WIN32
@ -204,25 +207,24 @@ struct RetiredImport {
VkDevice device;
PFN_vkDestroyBuffer destroyBuffer;
PFN_vkFreeMemory freeMemory;
HostImport entry;
ImportNode node;
};
// Drops an import. Recorded batches (dispatches, recorded draws, GPU label writes) may still read it,
// so the objects live until the batch open now completed; with no batches in flight all GPU work that
// used it was synchronous and it is destroyed at once.
const GuestAllocations::Range* containingRange(const GuestAllocations::Lease& lease, std::uint64_t begin, std::uint64_t end);
bool runCovers(const GuestAllocations::Lease& lease, std::uint64_t begin, std::uint64_t end);
void retireImport(VkDevice device, HostImports& state, std::map<std::uint64_t, HostImport>::iterator it, const GuestAllocations::Lease& lease) {
void retireImport(VkDevice device, HostImports& state, std::map<std::uint64_t, HostImport>::iterator it, const GuestAllocations::Lease& lease, bool hold = true) {
// Results shadowed for the import reach its (old) buffer first where the memory is still a
// readable registered range (the import retires because its registration vanished or changed
// size); the holder below outlives the batch that copies them.
RetireShadow(device, it->second, [&lease](std::uint64_t begin, std::uint64_t end) { return runCovers(lease, begin, end); });
auto holder = std::make_shared<RetiredImport>(state.device, state.destroyBuffer, state.freeMemory, it->second);
if (auto* recorder = Recorder::Active(); recorder != nullptr && !recorder->Idle()) recorder->Keep(std::move(holder));
++state.epoch;
state.liveBytes -= it->second.bytes;
state.imports.erase(it);
auto holder = std::make_shared<RetiredImport>(state.device, state.destroyBuffer, state.freeMemory, state.imports.extract(it));
auto* recorder = Recorder::Active();
if (recorder == nullptr) return;
if (hold) recorder->KeepThroughHold(std::move(holder));
else if (!recorder->Idle()) recorder->Keep(std::move(holder));
}
bool releaseParkedImports(HostImports& state, std::uint64_t begin, std::uint64_t end) {
@ -232,7 +234,7 @@ bool releaseParkedImports(HostImports& state, std::uint64_t begin, std::uint64_t
for (auto device = state.parked.begin(); device != state.parked.end();) {
auto& list = device->second;
for (auto it = list.begin(); it != list.end();) {
if ((*it)->entry.base < end && begin < (*it)->entry.base + (*it)->entry.bytes) {
if ((*it)->node.mapped().base < end && begin < (*it)->node.mapped().base + (*it)->node.mapped().bytes) {
released.push_back(std::move(*it));
it = list.erase(it);
} else {
@ -795,16 +797,16 @@ const GuestAllocations::Range* containingRange(const GuestAllocations::Lease& le
// guest addresses. Walks the imports only when the registry changed since the last walk.
void refreshImports(const Context& context, HostImports& state, const GuestAllocations::Lease& lease) {
if (state.device != context.device) {
for (const auto& [address, entry] : state.imports) {
while (!state.imports.empty()) {
auto node = state.imports.extract(state.imports.begin());
if (state.device != VK_NULL_HANDLE && state.destroyBuffer != nullptr && state.freeMemory != nullptr) {
state.parked[state.device].push_back(std::make_shared<RetiredImport>(state.device, state.destroyBuffer, state.freeMemory, entry));
state.parked[state.device].push_back(std::make_shared<RetiredImport>(state.device, state.destroyBuffer, state.freeMemory, std::move(node)));
continue;
}
#ifdef _WIN32
GuestArena::GuestArenaUnmapAlias_nid_postfix(entry.alias);
GuestArena::GuestArenaUnmapAlias_nid_postfix(node.mapped().alias);
#endif
}
state.imports.clear();
state.liveBytes = 0;
state.failed.clear();
state.device = context.device;
@ -1576,7 +1578,7 @@ bool WaitForLeasesAndImports(std::uintptr_t address, std::size_t bytes) noexcept
continue;
}
const auto next = std::next(it);
retireImport(imports.device, imports, it, lease);
retireImport(imports.device, imports, it, lease, false);
it = next;
retiredImport = true;
}
@ -1614,6 +1616,7 @@ bool WaitForLeasesAndImports(std::uintptr_t address, std::size_t bytes) noexcept
// yet to record (a test's, or a build whose worker gave up the lock); give it time.
std::this_thread::yield();
}
if (recorder != nullptr) recorder->FlushDeferredReleases();
const auto end = static_cast<std::uint64_t>(address) + bytes;
auto& imports = Imports();
@ -1645,7 +1648,7 @@ bool WaitForLeasesAndImports(std::uintptr_t address, std::size_t bytes) noexcept
continue;
}
const auto next = std::next(it);
retireImport(imports.device, imports, it, lease);
retireImport(imports.device, imports, it, lease, false);
it = next;
retiredImport = true;
}

20
core/libs/prx/libSceAgcDriver/Graphics/src/Recorder.cpp
View File

@ -1117,6 +1117,7 @@ Recorder::~Recorder() {
} catch (const std::exception& error) {
std::fprintf(stderr, "[gpu] recorder teardown: %s\n", error.what());
}
holdKept.clear();
if (activeRecorder == this) {
activeRecorder = nullptr;
pendingWrites.store(nullptr, std::memory_order_release);
@ -1516,6 +1517,8 @@ void Recorder::ensureOpen() {
}
batch->queue = GuestMemory::GpuLockThreadTag();
if (countingSamples && !segmentMode()) beginSamples(*batch);
dropStaleHoldKept();
batch->kept.insert(batch->kept.end(), holdKept.begin(), holdKept.end());
open = std::move(batch);
// The whole-batch range: its start stamp waits for the previous batches like any
// bottom-of-pipe stamp, so it marks when this batch's execution began.
@ -2452,6 +2455,20 @@ void Recorder::Keep(std::shared_ptr<void> object, std::size_t bytes) {
open->keptBytes += bytes;
}
void Recorder::KeepThroughHold(std::shared_ptr<void> object) {
GuestMemory::AssertGpuLockHeld("Recorder::KeepThroughHold");
if (const auto hold = GuestMemory::GpuHoldSerial(); hold != holdKeptSerial) {
holdKept.clear();
holdKeptSerial = hold;
}
Keep(object);
holdKept.push_back(std::move(object));
}
void Recorder::dropStaleHoldKept() {
if (!holdKept.empty() && holdKeptSerial != GuestMemory::GpuHoldSerial() && GuestMemory::GpuMutex().HeldByThisThread()) holdKept.clear();
}
void Recorder::KeepBytes(const void* owner, std::size_t bytes) {
if (owner == nullptr || bytes == 0) return;
ensureOpen();
@ -3116,6 +3133,7 @@ void Recorder::Submit() {
// count behind, and the callers that act on it would otherwise take the mutex for nothing at
// every packet or submission end until a real batch is next submitted.
if (activeRecorder == this) workSinceSubmit.store(0, std::memory_order_relaxed);
dropStaleHoldKept();
if (open == nullptr) return;
GuestMemory::AssertGpuLockHeld("Recorder::Submit");
if (!open->keyStores.empty()) recordKeyStores(false);
@ -3270,6 +3288,7 @@ void Recorder::Sync() {
}
void Recorder::FlushDeferredReleases() {
holdKept.clear();
ReleaseDeferredKeeps();
}
@ -3532,6 +3551,7 @@ bool Recorder::syncThroughUnlocked(std::uint64_t address, std::uint64_t end, int
bool Recorder::Reap() {
PerformanceTimer frameTiming("Recorder.Reap");
dropStaleHoldKept();
const bool profile = DrawProfiled();
const auto start = profile ? std::chrono::steady_clock::now() : std::chrono::steady_clock::time_point{};
std::uint64_t retired = 0;

5
core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp
View File

@ -797,6 +797,11 @@ State DecodeState(const QueueState& queue, bool vertexWorkgroup) {
if (slotExport == 7 && ((read(cx, 0x31c + slot * 0xfu) >> 8u) & 7u) != 4u) throw std::runtime_error("AGC graphics: color export format 7 (UINT16_ABGR) into a target that is not unsigned integer is unsupported");
color.exportIndex = index;
color.uintExport = slotExport == 7;
if (color.samples > 1 && result.samples == 1 && result.colors.empty() && !result.depth && !result.alphaToCoverage && (read(cx, 0x292) & 1u) == 0) {
result.samples = color.samples;
result.customSampleLocations = true;
result.sampleLocations.fill(0);
}
Require(color.samples == result.samples, "a color target's sample count differs from the rasterizer's");
APS5_LOG_OUT_DEBUG("Color %u address=0x%llx extent=%ux%u bytes=%llu VkFormat=%u", slot, static_cast<unsigned long long>(color.address), color.extent.width, color.extent.height, static_cast<unsigned long long>(color.bytes), static_cast<unsigned>(color.format));
if (result.colors.empty()) {

9
core/libs/prx/libSceAgcDriver/tests/BdaExecution.cpp
View File

@ -9,6 +9,7 @@
#include <limits>
#include <set>
#include <string>
#include <utility>
#include <vector>
namespace {
@ -345,15 +346,16 @@ void RunBdaStoreExecutionTests(const Context& context) {
constexpr std::uint32_t writable = Abi::Read | Abi::Write;
const Abi::Header header{Abi::Version, 2, sizeof(Abi::Range), 0};
std::array<Abi::Range, 2> ranges{};
bool runtime = false;
const auto run = [&](const char* name, std::uint64_t address, std::uint32_t bits, bool coherent, const std::array<std::uint8_t, 5>& expected, std::initializer_list<std::uint64_t> writtenAddresses, Abi::FaultReason reason = {}, std::uint64_t faultAddress = 0, std::uint32_t faultBytes = 0) {
const std::string label = std::string("BDA store ") + name + (coherent ? " coherent: " : ": ");
const std::string label = std::string(runtime ? "BDA runtime write " : "BDA store ") + name + (coherent ? " coherent: " : ": ");
std::memset(first.Bytes().data(), 0xcd, first.Bytes().size());
std::memset(second.Bytes().data(), 0xcd, second.Bytes().size());
std::memcpy(table.Bytes().data(), &header, sizeof(header));
std::memcpy(table.Bytes().data() + sizeof(header), ranges.data(), sizeof(ranges));
std::memset(fault.Bytes().data(), 0, fault.Bytes().size());
std::memset(output.Bytes().data(), 0, output.Bytes().size());
Pipeline pipeline(context, MakeBdaStoreTestShader(address, 0xa1b2c3d4u, bits, coherent), {&table, &fault, &output});
Pipeline pipeline(context, runtime ? MakeBdaWriteTestShader(address, 0xa1b2c3d4u, bits, coherent) : MakeBdaStoreTestShader(address, 0xa1b2c3d4u, bits, coherent), {&table, &fault, &output});
pipeline.Run(1);
std::array<std::uint8_t, 5> actual{};
std::memcpy(actual.data(), first.Bytes().data(), 2);
@ -387,7 +389,8 @@ void RunBdaStoreExecutionTests(const Context& context) {
std::memcpy(&completed, output.Bytes().data(), sizeof(completed));
Require(completed == 1, label + "store did not finish its per-byte accesses");
};
for (const bool coherent : {false, true}) {
for (const auto& [coherent, writes] : std::initializer_list<std::pair<bool, bool>>{{false, false}, {false, true}, {true, false}, {true, true}}) {
runtime = writes;
ranges = {{{guest, guest + 2, first.DeviceAddress(), writable, 0}, {guest + 2, guest + 5, second.DeviceAddress(), writable, 0}}};
run("byte", guest + 1, 8, coherent, {0xcd, 0xd4, 0xcd, 0xcd, 0xcd}, {guest + 1});
run("split short", guest + 1, 16, coherent, {0xcd, 0xd4, 0xc3, 0xcd, 0xcd}, {guest + 1, guest + 2});

12
core/libs/prx/libSceAgcDriver/tests/BdaShader.cpp
View File

@ -121,3 +121,15 @@ std::vector<std::uint32_t> MakeBdaStoreTestShader(std::uint64_t address, std::ui
#endif
return shader;
}
std::vector<std::uint32_t> MakeBdaWriteTestShader(std::uint64_t address, std::uint32_t value, std::uint32_t bits, bool coherent) {
using namespace ShaderRecompiler;
auto shader = MakeShader(address, coherent, true, [&](SpirvValueEmitContext& ctx, const IrValue& instruction, std::uint32_t base) {
EmitBdaWrite(ctx, instruction, base, ConstantU32(ctx.state, value), bits);
return std::vector<std::uint32_t>{ConstantU32(ctx.state, 1u)};
}, 0u, true);
#if ANYPS5_ENABLE_SPIRV_TOOLS
shader = ValidateAndOptimizeSpirv(shader, VK_API_VERSION_1_1, shader[1], false, false);
#endif
return shader;
}

1
core/libs/prx/libSceAgcDriver/tests/BdaShader.hpp
View File

@ -10,6 +10,7 @@ std::vector<std::uint32_t> MakeBdaDwordReadTestShader(std::uint64_t address, std
std::vector<std::uint32_t> MakeBdaSpanReadTestShader(std::uint64_t address, std::uint32_t offset, std::uint32_t extracted, bool coherent, bool stops);
std::vector<std::uint32_t> MakeBdaDwordWriteTestShader(std::uint64_t address, std::uint32_t dwords, const std::uint32_t* values);
std::vector<std::uint32_t> MakeBdaStoreTestShader(std::uint64_t address, std::uint32_t value, std::uint32_t bits, bool coherent);
std::vector<std::uint32_t> MakeBdaWriteTestShader(std::uint64_t address, std::uint32_t value, std::uint32_t bits, bool coherent);
void RunBdaStoreExecutionTests(const AgcDriver::Graphics::Context& context);
void RunBdaExecutionTests(const AgcDriver::Graphics::Context& context);
void RunBdaContractTests();

23
core/libs/prx/libSceAgcDriver/tests/ControlFlow.cpp
View File

@ -1119,6 +1119,29 @@ early_exit:
{0x7e020280u, 0xbf06c002u, 0xbf850003u, 0x4a020081u, 0xbf068003u, 0xbf850006u, 0x7d880088u, 0xbf870001u, 0x4a020282u, 0xe0700000u,
0x80000100u, 0xbf810000u, 0xbefe0480u, 0xbf810000u},
Split::None},
{"early exit arm beside a parent arm that reaches the same merge", R"(
v_mov_b32 v1, 0
s_cmp_eq_u32 s2, 64
s_cbranch_scc1 other
v_add_nc_u32 v1, 1, v0
s_cmp_eq_u32 s3, 0
s_cbranch_scc1 early_exit
join:
v_cmp_gt_u32 vcc, 8, v0
s_cbranch_vccnz store
v_add_nc_u32 v1, 2, v1
store:
buffer_store_dword v1, off, s[0:3], 0
s_endpgm
other:
v_add_nc_u32 v1, 3, v0
s_branch join
early_exit:
s_mov_b64 exec, 0
s_endpgm)",
{0x7e020280u, 0xbf06c002u, 0xbf850009u, 0x4a020081u, 0xbf068003u, 0xbf850008u, 0x7d880088u, 0xbf870001u, 0x4a020282u, 0xe0700000u,
0x80000100u, 0xbf810000u, 0x4a020083u, 0xbf82fff8u, 0xbefe0480u, 0xbf810000u},
Split::None},
{"return block entered from the return block after it", R"(
v_mov_b32 v1, 0
s_cmp_eq_u32 s2, 64

10
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
View File

@ -1892,6 +1892,16 @@ void multisampleTests() {
queue.context[0x302] = 0x44cc;
expectFailure([&] { DecodeState(queue); }, "nonstandard sample locations");
queue = multisampled();
queue.context[0x2f8] = 0;
queue.context[0x292] = 2;
queue.context[0x31d] = 0x12000;
state = DecodeState(queue);
Require(state.samples == 4 && state.colors.size() == 1 && state.colors[0].samples == 4 && state.customSampleLocations, "single-sample scan conversion into a 4x target did not rasterize the target's samples");
const auto centered = AgcDriver::Graphics::SampleLocations(state);
for (std::uint32_t sample = 0; sample < state.samples; ++sample) Require(centered[sample].x == 0.5f && centered[sample].y == 0.5f, "single-sample scan conversion did not place every sample at the pixel center");
queue.context[0x292] = 3;
expectFailure([&] { DecodeState(queue); }, "MSAA_ENABLE on a single-sample target");
queue = multisampled();
queue.context[0x2f8] = 0x00300003;
queue.context[0x31d] = 0x1b000;
for (std::uint32_t pixel = 0; pixel < 4; ++pixel) {

349
core/libs/prx/libSceAgcDriver/tests/HostImportRetireHold.cpp Normal file
View File

@ -0,0 +1,349 @@
#include "prx/libSceAgcDriver/Graphics/include/GuestBufferMemory.hpp"
#include "prx/libSceAgcDriver/Graphics/include/Recorder.hpp"
#include "prx/libSceAgcDriver/Execution/include/GuestMemory.hpp"
#include "prx/libc/include/GuestAllocations.hpp"
#ifdef _WIN32
#include <windows.h>
#endif
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <exception>
#include <map>
#include <memory>
#include <mutex>
#include <set>
#include <stdexcept>
#include <string>
#include <string_view>
#include <vector>
using namespace AgcDriver::Graphics;
namespace {
int failures = 0;
void Expect(bool condition, const std::string& what) {
if (condition) return;
std::fprintf(stderr, "FAIL: %s\n", what.c_str());
++failures;
}
constexpr std::size_t Alignment = 65536;
constexpr std::size_t RangeBytes = std::size_t{1} << 20u;
struct MockDevice {
std::uintptr_t next = 0x1000;
std::map<VkFence, bool> signaled;
std::vector<VkFence> submitted;
std::set<VkBuffer> buffers;
std::map<VkBuffer, VkDeviceSize> sizes;
std::set<VkDeviceMemory> memories;
std::uint64_t nextAddress = 0x500000000ull;
};
MockDevice mock;
bool Live(VkBuffer buffer) {
return mock.buffers.contains(buffer);
}
VKAPI_ATTR VkResult VKAPI_CALL mockAllocateCommandBuffers(VkDevice, const VkCommandBufferAllocateInfo*, VkCommandBuffer* commands) {
*commands = reinterpret_cast<VkCommandBuffer>(mock.next++);
return VK_SUCCESS;
}
VKAPI_ATTR void VKAPI_CALL mockFreeCommandBuffers(VkDevice, VkCommandPool, std::uint32_t, const VkCommandBuffer*) {}
VKAPI_ATTR VkResult VKAPI_CALL mockCreateFence(VkDevice, const VkFenceCreateInfo*, const VkAllocationCallbacks*, VkFence* fence) {
*fence = reinterpret_cast<VkFence>(mock.next++);
mock.signaled[*fence] = false;
return VK_SUCCESS;
}
VKAPI_ATTR void VKAPI_CALL mockDestroyFence(VkDevice, VkFence fence, const VkAllocationCallbacks*) {
mock.signaled.erase(fence);
}
VKAPI_ATTR VkResult VKAPI_CALL mockGetFenceStatus(VkDevice, VkFence fence) {
return mock.signaled.at(fence) ? VK_SUCCESS : VK_NOT_READY;
}
VKAPI_ATTR VkResult VKAPI_CALL mockWaitForFences(VkDevice, std::uint32_t count, const VkFence* fences, VkBool32, std::uint64_t) {
for (std::uint32_t i = 0; i < count; ++i) mock.signaled.at(fences[i]) = true;
return VK_SUCCESS;
}
VKAPI_ATTR VkResult VKAPI_CALL mockResetFences(VkDevice, std::uint32_t count, const VkFence* fences) {
for (std::uint32_t i = 0; i < count; ++i) mock.signaled.at(fences[i]) = false;
return VK_SUCCESS;
}
VKAPI_ATTR VkResult VKAPI_CALL mockBeginCommandBuffer(VkCommandBuffer, const VkCommandBufferBeginInfo*) {
return VK_SUCCESS;
}
VKAPI_ATTR VkResult VKAPI_CALL mockEndCommandBuffer(VkCommandBuffer) {
return VK_SUCCESS;
}
VKAPI_ATTR VkResult VKAPI_CALL mockQueueSubmit(VkQueue, std::uint32_t, const VkSubmitInfo*, VkFence fence) {
mock.submitted.push_back(fence);
return VK_SUCCESS;
}
VKAPI_ATTR void VKAPI_CALL mockCmdPipelineBarrier(VkCommandBuffer, VkPipelineStageFlags, VkPipelineStageFlags, VkDependencyFlags, std::uint32_t, const VkMemoryBarrier*, std::uint32_t, const VkBufferMemoryBarrier*, std::uint32_t, const VkImageMemoryBarrier*) {}
VKAPI_ATTR void VKAPI_CALL mockCmdUpdateBuffer(VkCommandBuffer, VkBuffer, VkDeviceSize, VkDeviceSize, const void*) {}
VKAPI_ATTR void VKAPI_CALL mockCmdBeginQuery(VkCommandBuffer, VkQueryPool, std::uint32_t, VkQueryControlFlags) {}
VKAPI_ATTR void VKAPI_CALL mockCmdEndQuery(VkCommandBuffer, VkQueryPool, std::uint32_t) {}
VKAPI_ATTR void VKAPI_CALL mockCmdResetQueryPool(VkCommandBuffer, VkQueryPool, std::uint32_t, std::uint32_t) {}
VKAPI_ATTR void VKAPI_CALL mockCmdCopyQueryPoolResults(VkCommandBuffer, VkQueryPool, std::uint32_t, std::uint32_t, VkBuffer, VkDeviceSize, VkDeviceSize, VkQueryResultFlags) {}
VKAPI_ATTR void VKAPI_CALL mockCmdBindPipeline(VkCommandBuffer, VkPipelineBindPoint, VkPipeline) {}
VKAPI_ATTR void VKAPI_CALL mockCmdPushConstants(VkCommandBuffer, VkPipelineLayout, VkShaderStageFlags, std::uint32_t, std::uint32_t, const void*) {}
VKAPI_ATTR void VKAPI_CALL mockCmdDispatch(VkCommandBuffer, std::uint32_t, std::uint32_t, std::uint32_t) {}
VKAPI_ATTR VkResult VKAPI_CALL mockGetMemoryHostPointerProperties(VkDevice, VkExternalMemoryHandleTypeFlagBits, const void*, VkMemoryHostPointerPropertiesEXT* properties) {
properties->memoryTypeBits = 1;
return VK_SUCCESS;
}
VKAPI_ATTR VkResult VKAPI_CALL mockCreateBuffer(VkDevice, const VkBufferCreateInfo* info, const VkAllocationCallbacks*, VkBuffer* buffer) {
*buffer = reinterpret_cast<VkBuffer>(mock.next++);
mock.buffers.insert(*buffer);
mock.sizes[*buffer] = info->size;
return VK_SUCCESS;
}
VKAPI_ATTR void VKAPI_CALL mockDestroyBuffer(VkDevice, VkBuffer buffer, const VkAllocationCallbacks*) {
mock.buffers.erase(buffer);
}
VKAPI_ATTR void VKAPI_CALL mockGetBufferMemoryRequirements(VkDevice, VkBuffer buffer, VkMemoryRequirements* requirements) {
requirements->size = mock.sizes.at(buffer);
requirements->alignment = Alignment;
requirements->memoryTypeBits = 1;
}
VKAPI_ATTR VkResult VKAPI_CALL mockAllocateMemory(VkDevice, const VkMemoryAllocateInfo*, const VkAllocationCallbacks*, VkDeviceMemory* memory) {
*memory = reinterpret_cast<VkDeviceMemory>(mock.next++);
mock.memories.insert(*memory);
return VK_SUCCESS;
}
VKAPI_ATTR void VKAPI_CALL mockFreeMemory(VkDevice, VkDeviceMemory memory, const VkAllocationCallbacks*) {
mock.memories.erase(memory);
}
VKAPI_ATTR VkResult VKAPI_CALL mockBindBufferMemory(VkDevice, VkBuffer, VkDeviceMemory, VkDeviceSize) {
return VK_SUCCESS;
}
VKAPI_ATTR VkDeviceAddress VKAPI_CALL mockGetBufferDeviceAddress(VkDevice, const VkBufferDeviceAddressInfo* info) {
const auto address = mock.nextAddress;
mock.nextAddress += (mock.sizes.at(info->buffer) + Alignment - 1) / Alignment * Alignment;
return address;
}
PFN_vkVoidFunction VKAPI_CALL mockProc(VkDevice, const char* name) {
static const std::map<std::string_view, PFN_vkVoidFunction> table{
{"vkAllocateCommandBuffers", reinterpret_cast<PFN_vkVoidFunction>(mockAllocateCommandBuffers)},
{"vkFreeCommandBuffers", reinterpret_cast<PFN_vkVoidFunction>(mockFreeCommandBuffers)},
{"vkCreateFence", reinterpret_cast<PFN_vkVoidFunction>(mockCreateFence)},
{"vkDestroyFence", reinterpret_cast<PFN_vkVoidFunction>(mockDestroyFence)},
{"vkGetFenceStatus", reinterpret_cast<PFN_vkVoidFunction>(mockGetFenceStatus)},
{"vkWaitForFences", reinterpret_cast<PFN_vkVoidFunction>(mockWaitForFences)},
{"vkResetFences", reinterpret_cast<PFN_vkVoidFunction>(mockResetFences)},
{"vkBeginCommandBuffer", reinterpret_cast<PFN_vkVoidFunction>(mockBeginCommandBuffer)},
{"vkEndCommandBuffer", reinterpret_cast<PFN_vkVoidFunction>(mockEndCommandBuffer)},
{"vkQueueSubmit", reinterpret_cast<PFN_vkVoidFunction>(mockQueueSubmit)},
{"vkCmdPipelineBarrier", reinterpret_cast<PFN_vkVoidFunction>(mockCmdPipelineBarrier)},
{"vkCmdUpdateBuffer", reinterpret_cast<PFN_vkVoidFunction>(mockCmdUpdateBuffer)},
{"vkCmdBeginQuery", reinterpret_cast<PFN_vkVoidFunction>(mockCmdBeginQuery)},
{"vkCmdEndQuery", reinterpret_cast<PFN_vkVoidFunction>(mockCmdEndQuery)},
{"vkCmdResetQueryPool", reinterpret_cast<PFN_vkVoidFunction>(mockCmdResetQueryPool)},
{"vkCmdCopyQueryPoolResults", reinterpret_cast<PFN_vkVoidFunction>(mockCmdCopyQueryPoolResults)},
{"vkCmdBindPipeline", reinterpret_cast<PFN_vkVoidFunction>(mockCmdBindPipeline)},
{"vkCmdPushConstants", reinterpret_cast<PFN_vkVoidFunction>(mockCmdPushConstants)},
{"vkCmdDispatch", reinterpret_cast<PFN_vkVoidFunction>(mockCmdDispatch)},
{"vkGetMemoryHostPointerPropertiesEXT", reinterpret_cast<PFN_vkVoidFunction>(mockGetMemoryHostPointerProperties)},
{"vkCreateBuffer", reinterpret_cast<PFN_vkVoidFunction>(mockCreateBuffer)},
{"vkDestroyBuffer", reinterpret_cast<PFN_vkVoidFunction>(mockDestroyBuffer)},
{"vkGetBufferMemoryRequirements", reinterpret_cast<PFN_vkVoidFunction>(mockGetBufferMemoryRequirements)},
{"vkAllocateMemory", reinterpret_cast<PFN_vkVoidFunction>(mockAllocateMemory)},
{"vkFreeMemory", reinterpret_cast<PFN_vkVoidFunction>(mockFreeMemory)},
{"vkBindBufferMemory", reinterpret_cast<PFN_vkVoidFunction>(mockBindBufferMemory)},
{"vkGetBufferDeviceAddressKHR", reinterpret_cast<PFN_vkVoidFunction>(mockGetBufferDeviceAddress)},
};
const auto it = table.find(name);
return it == table.end() ? nullptr : it->second;
}
Context mockContext() {
static const auto device = reinterpret_cast<VkDevice>(mock.next++);
static const auto queue = reinterpret_cast<VkQueue>(mock.next++);
Context context{};
context.device = device;
context.queue = queue;
context.deviceProc = mockProc;
context.hostImportAlignment = Alignment;
return context;
}
struct Heap {
std::byte* base = nullptr;
bool first = false;
bool second = false;
Heap() {
#ifdef _WIN32
base = static_cast<std::byte*>(VirtualAlloc(nullptr, 2 * RangeBytes, MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE));
#else
base = static_cast<std::byte*>(std::aligned_alloc(Alignment, 2 * RangeBytes));
#endif
if (base == nullptr) throw std::runtime_error("cannot allocate the test heap");
GuestAllocations::Mutation mutation;
mutation.Add(base, RangeBytes, true, true, true);
first = true;
}
void Grow() {
GuestAllocations::Mutation mutation;
mutation.Add(base + RangeBytes, RangeBytes, true, true, true);
second = true;
}
std::uint64_t Address(std::size_t offset = 0) const {
return reinterpret_cast<std::uint64_t>(base) + offset;
}
~Heap() {
try {
GuestAllocations::Mutation mutation;
if (second) mutation.Remove(base + RangeBytes);
if (first) mutation.Remove(base);
} catch (...) {
}
#ifdef _WIN32
VirtualFree(base, 0, MEM_RELEASE);
#else
std::free(base);
#endif
}
};
struct ActiveRecorder {
std::unique_ptr<Recorder> recorder;
ActiveRecorder() {
std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex());
recorder = std::make_unique<Recorder>(mockContext());
recorder->Activate();
}
~ActiveRecorder() {
std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex());
recorder.reset();
}
};
void IdleRetireOutlivesTheHold() {
Heap heap;
ActiveRecorder active;
auto& recorder = *active.recorder;
const auto context = mockContext();
VkBuffer retired = VK_NULL_HANDLE;
{
std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex());
const auto* taken = HostImportFor(context, heap.Address(), 4096);
Expect(taken != nullptr, "the first range was not imported");
if (taken == nullptr) return;
retired = taken->buffer;
Expect(recorder.Idle(), "the recorder is not idle before the drop");
heap.Grow();
const auto* grown = HostImportFor(context, heap.Address(RangeBytes), 4096);
Expect(grown != nullptr && grown->buffer != retired && grown->base == heap.Address() && grown->bytes == 2 * RangeBytes, "the grown run was not imported as one import replacing the first");
Expect(Live(retired), "an import dropped with the recorder idle was destroyed at once, while the hold that took it can still record with it");
if (!Live(retired)) return;
Expect(taken->buffer == retired && taken->base == heap.Address(), "the dropped import's entry changed under the hold that took it");
recorder.Commands();
recorder.Submit();
recorder.Sync();
Expect(Live(retired), "an import dropped in this hold was destroyed by a Sync inside the same hold");
recorder.Commands();
recorder.Submit();
Expect(Live(retired), "an import dropped in this hold was destroyed while a batch the hold recorded afterwards is in flight");
}
{
std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex());
Expect(Live(retired), "an import dropped in a hold was destroyed at the hold's unlock while a batch of that hold is in flight");
}
{
std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex());
recorder.Sync();
}
Expect(!Live(retired), "the dropped import outlived every batch of its hold");
}
void BusyRetireOutlivesALaterBatch() {
Heap heap;
ActiveRecorder active;
auto& recorder = *active.recorder;
const auto context = mockContext();
VkBuffer retired = VK_NULL_HANDLE;
{
std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex());
recorder.Keep(std::make_shared<int>(0));
const auto* taken = HostImportFor(context, heap.Address(), 4096);
Expect(taken != nullptr, "the first range was not imported");
if (taken == nullptr) return;
retired = taken->buffer;
heap.Grow();
Expect(HostImportFor(context, heap.Address(RangeBytes), 4096) != nullptr, "the grown run was not imported");
mock.submitted.clear();
recorder.Submit();
recorder.Commands();
recorder.Submit();
Expect(mock.submitted.size() == 2, "the hold did not submit the keeping batch and the later one apart");
if (mock.submitted.size() != 2) return;
mock.signaled.at(mock.submitted[0]) = true;
recorder.Reap();
Expect(Live(retired), "the dropped import was destroyed when the batch open at the drop completed, while a later batch of the hold that may use it is in flight");
}
{
std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex());
Expect(Live(retired), "the dropped import was destroyed at the hold's unlock, while a later batch of the hold that may use it is in flight");
mock.signaled.at(mock.submitted[1]) = true;
recorder.Reap();
}
{
std::lock_guard gpu(AgcDriver::GuestMemory::GpuMutex());
recorder.Commands();
recorder.Sync();
}
Expect(!Live(retired), "the dropped import outlived every batch of its hold");
}
}
int main() {
#ifdef _WIN32
_putenv_s("APS5_RELEASE_ON_UNLOCK", "1");
#else
setenv("APS5_RELEASE_ON_UNLOCK", "1", 1);
setenv("APS5_WRITE_WATCH_IMPORTS", "unwatch", 1);
#endif
try {
IdleRetireOutlivesTheHold();
BusyRetireOutlivesALaterBatch();
} catch (const std::exception& error) {
std::fprintf(stderr, "%s\n", error.what());
return 1;
}
if (failures != 0) {
std::fprintf(stderr, "%d host import retire checks failed\n", failures);
return 1;
}
std::printf("host import retire tests passed\n");
return 0;
}

66
core/libs/prx/libSceAgcDriver/tests/RectListPositionInvariant.cpp Normal file
View File

@ -0,0 +1,66 @@
#include "Recompiler.hpp"
#include <algorithm>
#include <array>
#include <cstdint>
#include <cstdio>
#include <exception>
#include <span>
#include <spirv/unified1/spirv.hpp>
#include <stdexcept>
#include <string>
#include <utility>
#include <vector>
namespace {
using namespace ShaderRecompiler;
void Check(bool condition, const std::string& message) {
if (!condition) throw std::runtime_error(message);
}
bool PositionIsInvariant(std::span<const std::uint32_t> words) {
Check(words.size() >= 5 && words[0] == spv::MagicNumber, "rect-list invariant: missing SPIR-V module");
std::vector<std::pair<std::uint32_t, std::uint32_t>> positionMembers;
std::vector<std::pair<std::uint32_t, std::uint32_t>> invariantMembers;
for (std::size_t offset = 5; offset < words.size();) {
const auto count = words[offset] >> spv::WordCountShift;
Check(count != 0 && count <= words.size() - offset, "rect-list invariant: malformed SPIR-V instruction");
const auto opcode = words[offset] & spv::OpCodeMask;
if (opcode == spv::OpMemberDecorate && count == 5 && words[offset + 3] == spv::DecorationBuiltIn && words[offset + 4] == spv::BuiltInPosition) {
positionMembers.emplace_back(words[offset + 1], words[offset + 2]);
}
if (opcode == spv::OpMemberDecorate && count == 4 && words[offset + 3] == spv::DecorationInvariant) {
invariantMembers.emplace_back(words[offset + 1], words[offset + 2]);
}
offset += count;
}
Check(!positionMembers.empty(), "rect-list invariant: shader has no gl_Position");
for (const auto& member : positionMembers) {
if (std::find(invariantMembers.begin(), invariantMembers.end(), member) == invariantMembers.end()) return false;
}
return true;
}
}
int main() {
try {
static constexpr std::array<std::uint32_t, 2> capabilities{spv::CapabilityShader, spv::CapabilityTessellation};
SpirvTarget target{};
target.vulkanVersion = 0x00401000u;
target.spirvVersion = 0x00010400u;
target.supportedCapabilities = capabilities;
target.tessellation = TessellationTargetLimits{32, 128, 128, 120, 4096, 128, 128};
RecompileResult vertex;
RecompileResult fragment;
const auto shaders = BuildRectListShaders(vertex, fragment, target);
Check(PositionIsInvariant(shaders.control.spirv.Words()), "rect-list control gl_Position is not Invariant");
Check(PositionIsInvariant(shaders.evaluation.spirv.Words()), "rect-list evaluation gl_Position is not Invariant");
std::printf("rect-list position invariant tests passed\n");
return 0;
} catch (const std::exception& error) {
std::fprintf(stderr, "%s\n", error.what());
return 1;
}
}

93
core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
View File

@ -1464,6 +1464,94 @@ void verifyNarrowVertexWave() {
require(GetPreparedArtifact(*PrepareShader(vertex(wave32, 32u, 32u, 8u))).spirv == GetPreparedArtifact(*PrepareShader(vertex(wave32, 32u, 64u, 8u))).spirv, "narrow vertex wave: s3 of a wave32 vertex program depended on the subgroup width");
}
void verifySingleLaneVertexWave() {
using namespace ShaderRecompiler;
static constexpr std::array<std::uint32_t, 5> capabilities{spv::CapabilityGroupNonUniform, spv::CapabilityGroupNonUniformBallot, spv::CapabilityGroupNonUniformShuffle, spv::CapabilityGroupNonUniformArithmetic, spv::CapabilityClipDistance};
const auto vertex = [](std::span<const std::uint32_t> code, std::uint32_t subgroupSize, bool vertexSubgroups) {
RecompileRequest request{};
request.shader = {ShaderStage::Vertex, 0x58000u, code, 0, {}};
request.context.waveSize = 64;
request.context.userDataBaseRegister = 8;
request.context.vertex = ShaderVertexStageInfo{};
request.target.vulkanVersion = 0x00401000u;
request.target.spirvVersion = 0x00010300u;
request.target.subgroupSize = subgroupSize;
request.target.supportedCapabilities = capabilities;
request.target.vertexSubgroups = vertexSubgroups;
request.layout.pushConstantSizeBytes = 128;
request.useCache = false;
return request;
};
const auto compiled = [&vertex](std::span<const std::uint32_t> code, std::uint32_t subgroupSize, bool vertexSubgroups) {
return Recompile(vertex(code, subgroupSize, vertexSubgroups)).spirv;
};
const auto usesSubgroups = [](const std::vector<std::uint32_t>& spirv) {
for (std::size_t cursor = 5; cursor < spirv.size() && (spirv[cursor] >> 16u) != 0u; cursor += spirv[cursor] >> 16u) {
const auto op = spirv[cursor] & 0xffffu;
if (op == spv::OpCapability && spirv[cursor + 1u] >= spv::CapabilityGroupNonUniform && spirv[cursor + 1u] <= spv::CapabilityGroupNonUniformQuad) return true;
if (op >= spv::OpGroupNonUniformElect && op <= spv::OpGroupNonUniformQuadSwap) return true;
if (op == spv::OpDecorate && spirv[cursor + 2u] == spv::DecorationBuiltIn && (spirv[cursor + 3u] == spv::BuiltInSubgroupLocalInvocationId || spirv[cursor + 3u] == spv::BuiltInSubgroupSize)) return true;
}
return false;
};
const auto hasConstant = [](const std::vector<std::uint32_t>& spirv, std::uint32_t value) {
for (std::size_t cursor = 5; cursor < spirv.size() && (spirv[cursor] >> 16u) != 0u; cursor += spirv[cursor] >> 16u) {
if ((spirv[cursor] & 0xffffu) == spv::OpConstant && (spirv[cursor] >> 16u) == 4u && spirv[cursor + 3u] == value) return true;
}
return false;
};
static constexpr std::array<std::uint32_t, 4> waveInfo{0x7e020203u, 0xf80008cfu, 0x01010101u, 0xbf810000u};
static constexpr std::array<std::uint32_t, 12> laneOps{
0xd7650015u, 0x000100c1u, 0xd7660015u, 0x00022ac1u, 0x7e0c0500u, 0xd7600007u, 0x00010100u, 0x4a022a06u,
0x4a020207u, 0xf80008cfu, 0x01010101u, 0xbf810000u,
};
static constexpr std::array<std::uint32_t, 11> waterfall{0xbe80047eu, 0xbe821400u, 0xd7600003u, 0x00000500u, 0xbe801c02u, 0xbf138000u, 0xbf85fffau, 0x7e020203u, 0xf80008cfu, 0x01010101u, 0xbf810000u};
static constexpr std::array<std::uint32_t, 6> lane31{0xd7600006u, 0x00013f00u, 0x7e020206u, 0xf80008cfu, 0x01010101u, 0xbf810000u};
const auto single = compiled(waveInfo, 32u, false);
const auto wide = compiled(waveInfo, 64u, true);
require(hasConstant(single, 0x0101u) && !hasConstant(single, 0x4040u) && !hasConstant(single, 0x2020u), "single-lane vertex wave: s3 of a vertex program without vertex subgroups was not one vertex and one primitive");
require(hasConstant(wide, 0x4040u) && !hasConstant(wide, 0x0101u), "single-lane vertex wave: s3 of a vertex program with vertex subgroups on 64 lanes changed");
require(single == compiled(waveInfo, 4u, false), "single-lane vertex wave: the program without vertex subgroups depended on the device subgroup size");
for (const auto code : {std::span<const std::uint32_t>(laneOps), std::span<const std::uint32_t>(waterfall)}) {
const auto withSubgroups = compiled(code, 32u, true);
const auto withoutSubgroups = compiled(code, 32u, false);
require(usesSubgroups(withSubgroups), "single-lane vertex wave: the program did not use subgroup operations with vertex subgroups");
require(!withoutSubgroups.empty() && !usesSubgroups(withoutSubgroups), "single-lane vertex wave: a vertex program without vertex subgroups declared a subgroup capability, built-in or instruction");
}
expectFailure([&] { static_cast<void>(compiled(lane31, 32u, false)); }, "v_readlane_b32 of lane 31 is outside the 1-lane host subgroup", "single-lane vertex wave: a vertex program without vertex subgroups read lane 31");
require(!compiled(lane31, 32u, true).empty(), "single-lane vertex wave: lane 31 of a 32-lane vertex subgroup did not build");
ShaderPixelStageInfo info{};
info.inputAddr = PixelInputBit(PixelInput::PerspectiveCenter);
info.hasPerspectiveCenterVgpr = true;
info.targetOutputMode[0] = 9u;
info.targetExportMapping.fill(0xe4u);
static constexpr std::array<std::uint32_t, 6> pixelLane31{0xd7600006u, 0x00013f00u, 0x7e020206u, 0xf800180fu, 0x01010101u, 0xbf810000u};
RecompileRequest pixel{};
pixel.shader = {ShaderStage::Fragment, 0x30000u, pixelLane31, 0, {}};
pixel.context.waveSize = 64;
pixel.context.pixel = info;
pixel.target.vulkanVersion = 0x00401000u;
pixel.target.spirvVersion = 0x00010300u;
pixel.target.subgroupSize = 32u;
pixel.target.vertexSubgroups = false;
pixel.layout.pushConstantSizeBytes = 128;
pixel.useCache = false;
require(usesSubgroups(Recompile(pixel).spirv), "single-lane vertex wave: a pixel program stopped using subgroups when only the vertex stage lacks them");
auto request = vertex(waveInfo, 32u, false);
const auto replay = RequestSerializer{}.Deserialize(RequestSerializer{}.Serialize(request));
require(!replay.request.target.vertexSubgroups && replay.request.target.subgroupSize == 32u, "single-lane vertex wave: missing vertex subgroups were lost in serialization");
require(Recompile(replay.request).spirv == single, "single-lane vertex wave: the replayed request built another program");
std::vector<std::uint64_t> withoutKey;
std::vector<std::uint64_t> withKey;
RecompileCacheKey::Build(request, withoutKey);
request.target.vertexSubgroups = true;
RecompileCacheKey::Build(request, withKey);
require(withoutKey != withKey, "single-lane vertex wave: vertex subgroup support did not change the cache key");
}
void verifyMeshConfiguration() {
using namespace ShaderRecompiler;
ShaderMeshInputInfo list;
@ -1637,12 +1725,12 @@ void verifyPixelRequestSerialization() {
minimal.context.waveSize = 64;
minimal.context.pixel = ShaderPixelStageInfo{};
const auto encoded = serializer.Serialize(minimal);
require(requestPrefix(encoded, 8u) == "NVNQQREAAAA=", "new requests did not use serialization version 17");
require(requestPrefix(encoded, 8u) == "NVNQQRIAAAA=", "new requests did not use serialization version 18");
constexpr std::size_t mappingOffset = 8u + 37u + 18u + 163u;
for (std::size_t bytes = 0; bytes < 19u; ++bytes) {
expectFailure([&] { static_cast<void>(serializer.Deserialize(requestPrefix(encoded, mappingOffset + bytes))); }, "truncated data", "a truncated pixel mapping, packing, dual-source, alpha-to-coverage or stencil reference export flag was accepted");
}
for (const auto unsupported : {"NVNQQQAAAAA=", "NVNQQRIAAAA="}) {
for (const auto unsupported : {"NVNQQQAAAAA=", "NVNQQRMAAAA="}) {
expectFailure([&] { static_cast<void>(serializer.Deserialize(unsupported)); }, "serialization version", "an unsupported request version was accepted");
}
}
@ -3107,6 +3195,7 @@ int main(int argc, char** argv) {
verifyLanesOutsideHostSubgroup();
verifyHalfWaveReduction();
verifyNarrowVertexWave();
verifySingleLaneVertexWave();
verifyMeshConfiguration();
verifyPixelInputs();
verifyPixelRequestSerialization();

7
core/libs/prx/libSceAgcDriver/tests/Structurizer.cpp
View File

@ -328,6 +328,13 @@ int main() {
requireExactPostDominators(earlyReturn, "an early return inside a selection that joins its parent's merge");
requireStructuredBranches(earlyReturn, "an early return inside a selection that joins its parent's merge");
requireDominatedMerges(earlyReturn, "an early return inside a selection that joins its parent's merge");
const std::vector<std::vector<std::uint32_t>> elseReturnSuccessors{{1, 2}, {4}, {3, 4}, {}, {5, 6}, {7}, {7}, {}};
auto elseReturn = makeGraph(elseReturnSuccessors);
Structurizer{}.Structurize(elseReturn);
requireExactPostDominators(elseReturn, "an early return in one arm of a selection whose other arm reaches the same merge");
requireStructuredBranches(elseReturn, "an early return in one arm of a selection whose other arm reaches the same merge");
requireDominatedMerges(elseReturn, "an early return in one arm of a selection whose other arm reaches the same merge");
requireOriginalBranches(elseReturn, elseReturnSuccessors, "an early return in one arm of a selection whose other arm reaches the same merge");
std::vector<std::vector<std::uint32_t>> diamonds;
for (std::uint32_t diamond = 0; diamond < 50u; ++diamond) {
const auto head = diamond * 3u;

6
core/libs/prx/libSceAgcDriver/tests/execution/DwordPatternFill.cpp
View File

@ -55,6 +55,12 @@ void MatchTests() {
Require(!MatchDwordPatternFill(packet, Code, UserData(Output.data(), 1024, 250, 4), compute), "a count of whole dwords not filling 16-byte units matched");
Require(!MatchDwordPatternFill(packet, Code, UserData(Output.data() + 1, 1024, 256, 4), compute), "an unaligned destination matched");
Require(!MatchDwordPatternFill(packet, Code, UserData(Output.data(), 1024, 256, 4, 0x00814004u), compute), "an add-tid descriptor matched");
for (const std::uint32_t format : {0x00015004u, 0x00016004u}) {
Require(MatchDwordPatternFill(packet, Code, UserData(Output.data(), 1024, 256, 4, format), compute).has_value(), "a 32-bit output format was refused");
}
for (const std::uint32_t format : {0x00005004u, 0x0000b004u, 0x0000d004u, 0x0001b004u, 0x00027facu, 0x0004b004u}) {
Require(!MatchDwordPatternFill(packet, Code, UserData(Output.data(), 1024, 256, 4, format), compute), "an output format whose elements are not one 32-bit value matched");
}
auto extra = data;
extra.push_back(0);
Require(!MatchDwordPatternFill(packet, Code, extra, compute), "a different user SGPR count matched (the group id moves)");

4
core/libs/prx/libSceAgcDriver/tests/execution/F32Dot2Rounding.cpp
View File

@ -231,6 +231,10 @@ int main() {
try {
const auto device = OpenVulkanTestDevice();
if (!device) return VulkanTestSkipped;
if (!TargetHasCapability(device->Target(), spv::CapabilityFloat64)) {
std::puts("skipped, v_dot2_f32_f16 takes an f64 branch and the device has no shaderFloat64");
return VulkanTestSkipped;
}
const auto cases = Cases();
for (std::size_t index = 0; index < cases.size(); ++index) {
Input[index * 4u] = cases[index].a;

211
core/libs/prx/libSceAgcDriver/tests/execution/PackedIntegerNegate.cpp Normal file
View File

@ -0,0 +1,211 @@
#include "prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp"
#include "prx/libSceAgcDriver/Graphics/include/Draw.hpp"
#include "Recompiler.hpp"
#include "VulkanTestDevice.hpp"
#include <algorithm>
#include <array>
#include <cstdint>
#include <cstdio>
#include <iostream>
#include <string>
#include <vector>
namespace {
using AgcDriver::Graphics::Require;
using ShaderRecompiler::ShaderStage;
constexpr std::uint32_t Threads = 32;
constexpr std::uint32_t Inputs = 4;
constexpr std::uint32_t Results = 16;
alignas(256) std::array<std::uint32_t, Threads * Inputs> Input{};
alignas(256) std::array<std::uint32_t, Threads * Results> Output{};
alignas(256) constexpr std::array<std::uint32_t, 47> Code{
0x34020084, 0x34060086, 0xe0381000, 0x80000401, 0xbf8c3f70, 0xcc07410a, 0x300208ff, 0x00007fff,
0xcc0a410b, 0x58020b04, 0xcc03420c, 0x38020b04, 0xcc08430d, 0x78020b04, 0xcc0c420e, 0x58020b04,
0xcc0d410f, 0x38020b04, 0xcc014210, 0x38020b04, 0xcc004511, 0x9c1a0b04, 0xcc094612, 0x7c1a0b04,
0xcc044213, 0x58020b04, 0xcc064314, 0x58020b04, 0xcc054215, 0x78020b04, 0xcc024a16, 0x30020b04,
0xcc0b5317, 0x68020b04, 0xcc02c218, 0x58020b04, 0xcc074919, 0x30020906, 0xe0781000, 0x80010a03,
0xe0781010, 0x80010e03, 0xe0781020, 0x80011203, 0xe0781030, 0x80011603, 0xbf810000,
};
constexpr std::uint32_t Rows[32][4] = {
{0x1234021bu, 0xffff7fffu, 0x8001edccu, 0x00000000u},
{0xc000021bu, 0x0002021bu, 0x000fedccu, 0x00000000u},
{0xedcc0001u, 0x00ff0218u, 0x0002fffeu, 0x00000000u},
{0x000f00ffu, 0x000f0218u, 0x80100218u, 0x00000000u},
{0x000f8000u, 0x000f021bu, 0x00ff021bu, 0x00000000u},
{0x7fff0218u, 0x4000000fu, 0x80018000u, 0x00000000u},
{0xffff4000u, 0x00ff8000u, 0x00021234u, 0x00000000u},
{0x7ffeffffu, 0x021bffffu, 0x8010c000u, 0x00000000u},
{0x80107fffu, 0x80010010u, 0xedcc8001u, 0x00000000u},
{0xc0000000u, 0x80008000u, 0x0218021bu, 0x00000000u},
{0x00020001u, 0x00027fffu, 0x40000218u, 0x00000000u},
{0x00ffc000u, 0xffff0002u, 0xfffefffeu, 0x00000000u},
{0x00100000u, 0x00ff000fu, 0xfffe4000u, 0x00000000u},
{0x021bfffeu, 0x00ff8001u, 0x00fffffeu, 0x00000000u},
{0xedccedccu, 0x021b021bu, 0x7ffe8001u, 0x00000000u},
{0x00007ffeu, 0x12340218u, 0xfffe0001u, 0x00000000u},
{0xc0000218u, 0x0010ffffu, 0xffff8001u, 0x00000000u},
{0x400000ffu, 0x7ffec000u, 0x80008000u, 0x00000000u},
{0x000f4000u, 0xedcc0001u, 0x80008000u, 0x00000000u},
{0x7fff0218u, 0x8010c000u, 0x80108010u, 0x00000000u},
{0x80008000u, 0x80000218u, 0x00007ffeu, 0x00000000u},
{0x021bffffu, 0x000fc000u, 0x7fffedccu, 0x00000000u},
{0x02180002u, 0x000f8000u, 0x0001edccu, 0x00000000u},
{0x021b4000u, 0x021b0001u, 0x02188001u, 0x00000000u},
{0x7fff8000u, 0x40008010u, 0xffff1234u, 0x00000000u},
{0x80100000u, 0x00010001u, 0x80000218u, 0x00000000u},
{0x4000c000u, 0xffff021bu, 0x40008010u, 0x00000000u},
{0x7fff4000u, 0x7ffe0002u, 0x000f00ffu, 0x00000000u},
{0x000fedccu, 0x8000fffeu, 0x021b4000u, 0x00000000u},
{0xedcc0001u, 0x8000021bu, 0xedcc8000u, 0x00000000u},
{0x02180000u, 0x00ffedccu, 0x12344000u, 0x00000000u},
{0xc000ffffu, 0x80010218u, 0x7fffc000u, 0x00000000u},
};
enum class Kind { MaxI, MinI, MaxU, MinU, AddU, AddI, SubU, SubI, MulLo, MadI, MadU, Lshl, Ashr, Lshr };
constexpr std::int32_t Literal = -1;
struct Case {
const char* name;
Kind kind;
std::array<std::int32_t, 3> sources;
std::array<bool, 3> opSel;
std::array<bool, 3> opSelHi;
std::array<bool, 3> neg;
std::array<bool, 3> negHi;
bool clamp;
};
constexpr std::uint32_t LiteralValue = 0x00007fffu;
constexpr std::array<Case, Results> Cases{{
{"v_pk_max_i16 v10, lit 0x7fff, v4 op_sel_hi:[0,1] neg_lo:[1,0] neg_hi:[1,0]", Kind::MaxI, {Literal, 0, 0}, {false, false, false}, {false, true, true}, {true, false, false}, {true, false, false}, false},
{"v_pk_add_u16 v11, v4, v5 neg_lo:[0,1] neg_hi:[1,0]", Kind::AddU, {0, 1, 0}, {}, {true, true, true}, {false, true, false}, {true, false, false}, false},
{"v_pk_sub_i16 v12, v4, v5 neg_lo:[1,0] neg_hi:[0,1]", Kind::SubI, {0, 1, 0}, {}, {true, true, true}, {true, false, false}, {false, true, false}, false},
{"v_pk_min_i16 v13, v4, v5 neg_lo:[1,1] neg_hi:[1,1]", Kind::MinI, {0, 1, 0}, {}, {true, true, true}, {true, true, false}, {true, true, false}, false},
{"v_pk_max_u16 v14, v4, v5 neg_lo:[0,1] neg_hi:[0,1]", Kind::MaxU, {0, 1, 0}, {}, {true, true, true}, {false, true, false}, {false, true, false}, false},
{"v_pk_min_u16 v15, v4, v5 neg_lo:[1,0] neg_hi:[1,0]", Kind::MinU, {0, 1, 0}, {}, {true, true, true}, {true, false, false}, {true, false, false}, false},
{"v_pk_mul_lo_u16 v16, v4, v5 neg_lo:[1,0] neg_hi:[0,1]", Kind::MulLo, {0, 1, 0}, {}, {true, true, true}, {true, false, false}, {false, true, false}, false},
{"v_pk_mad_i16 v17, v4, v5, v6 neg_lo:[0,0,1] neg_hi:[1,0,1]", Kind::MadI, {0, 1, 2}, {}, {true, true, true}, {false, false, true}, {true, false, true}, false},
{"v_pk_mad_u16 v18, v4, v5, v6 neg_lo:[1,1,0] neg_hi:[0,1,1]", Kind::MadU, {0, 1, 2}, {}, {true, true, true}, {true, true, false}, {false, true, true}, false},
{"v_pk_lshlrev_b16 v19, v4, v5 neg_lo:[0,1] neg_hi:[0,1]", Kind::Lshl, {0, 1, 0}, {}, {true, true, true}, {false, true, false}, {false, true, false}, false},
{"v_pk_ashrrev_i16 v20, v4, v5 neg_lo:[0,1] neg_hi:[1,1]", Kind::Ashr, {0, 1, 0}, {}, {true, true, true}, {false, true, false}, {true, true, false}, false},
{"v_pk_lshrrev_b16 v21, v4, v5 neg_lo:[1,1] neg_hi:[0,1]", Kind::Lshr, {0, 1, 0}, {}, {true, true, true}, {true, true, false}, {false, true, false}, false},
{"v_pk_add_i16 v22, v4, v5 op_sel:[1,0] op_sel_hi:[0,1] neg_lo:[1,0] neg_hi:[0,1]", Kind::AddI, {0, 1, 0}, {true, false, false}, {false, true, true}, {true, false, false}, {false, true, false}, false},
{"v_pk_sub_u16 v23, v4, v5 op_sel:[0,1] op_sel_hi:[1,0] neg_lo:[1,1] neg_hi:[1,1]", Kind::SubU, {0, 1, 0}, {false, true, false}, {true, false, true}, {true, true, false}, {true, true, false}, false},
{"v_pk_add_i16 v24, v4, v5 neg_lo:[0,1] neg_hi:[0,1] clamp", Kind::AddI, {0, 1, 0}, {}, {true, true, true}, {false, true, false}, {false, true, false}, true},
{"v_pk_max_i16 v25, v6, v4 op_sel:[1,0] op_sel_hi:[0,1] neg_lo:[1,0] neg_hi:[1,0]", Kind::MaxI, {2, 0, 0}, {true, false, false}, {false, true, true}, {true, false, false}, {true, false, false}, false},
}};
std::int32_t Signed(std::uint32_t half) {
return static_cast<std::int16_t>(static_cast<std::uint16_t>(half));
}
std::uint32_t Source(const Case& c, const std::uint32_t* in, std::size_t i, bool highLane) {
const std::uint32_t word = c.sources[i] == Literal ? LiteralValue : in[c.sources[i]];
const bool high = highLane ? c.opSelHi[i] : c.opSel[i];
const bool negate = highLane ? c.negHi[i] : c.neg[i];
return ((high ? word >> 16u : word) & 0xffffu) ^ (negate ? 0x8000u : 0u);
}
std::uint32_t Lane(const Case& c, const std::uint32_t* in, bool highLane) {
const std::uint32_t a = Source(c, in, 0, highLane);
const std::uint32_t b = Source(c, in, 1, highLane);
const std::uint32_t s = Source(c, in, 2, highLane);
std::int64_t value = 0;
switch (c.kind) {
case Kind::MaxI: value = std::max(Signed(a), Signed(b)); break;
case Kind::MinI: value = std::min(Signed(a), Signed(b)); break;
case Kind::MaxU: value = std::max(a, b); break;
case Kind::MinU: value = std::min(a, b); break;
case Kind::AddU: value = a + b; break;
case Kind::AddI: value = static_cast<std::int64_t>(Signed(a)) + Signed(b); break;
case Kind::SubU: value = static_cast<std::int64_t>(a) - b; break;
case Kind::SubI: value = static_cast<std::int64_t>(Signed(a)) - Signed(b); break;
case Kind::MulLo: value = static_cast<std::int64_t>(a) * b; break;
case Kind::MadI: value = static_cast<std::int64_t>(Signed(a)) * Signed(b) + Signed(s); break;
case Kind::MadU: value = static_cast<std::int64_t>(a) * b + s; break;
case Kind::Lshl: value = static_cast<std::int64_t>(b) << (a & 15u); break;
case Kind::Ashr: value = Signed(b) >> (a & 15u); break;
case Kind::Lshr: value = b >> (a & 15u); break;
}
if (c.clamp) value = std::clamp<std::int64_t>(value, -32768, 32767);
return static_cast<std::uint32_t>(value) & 0xffffu;
}
std::uint32_t Reference(const Case& c, const std::uint32_t* in) {
return Lane(c, in, false) | (Lane(c, in, true) << 16u);
}
std::array<std::uint32_t, 4> BufferDescriptor(const void* data, std::uint32_t bytes) {
const auto address = reinterpret_cast<std::uintptr_t>(data);
return {static_cast<std::uint32_t>(address), static_cast<std::uint32_t>((address >> 32u) & 0xffffu), bytes, 0x31016facu};
}
std::string Hex(std::uint32_t value) {
char text[16];
std::snprintf(text, sizeof(text), "0x%08x", value);
return text;
}
void Run(AgcDriver::VulkanDevice& device) {
for (std::uint32_t tid = 0; tid < Threads; ++tid) std::copy(std::begin(Rows[tid]), std::end(Rows[tid]), &Input[tid * Inputs]);
Output.fill(0xdeadbeefu);
std::vector<std::uint32_t> userData(8, 0u);
const auto input = BufferDescriptor(Input.data(), static_cast<std::uint32_t>(Input.size() * 4u));
const auto output = BufferDescriptor(Output.data(), static_cast<std::uint32_t>(Output.size() * 4u));
std::copy(input.begin(), input.end(), userData.begin());
std::copy(output.begin(), output.end(), userData.begin() + 4);
const std::span<const std::uint32_t> code(Code);
const std::array<ShaderRecompiler::MemoryRegion, 1> memory{{{reinterpret_cast<std::uintptr_t>(code.data()), std::as_bytes(code)}}};
const ShaderRecompiler::ShaderComputeStageInfo compute{{Threads, 1, 1}, 0u, {false, false, false}, false, 1};
ShaderRecompiler::RecompileRequest request{
{ShaderStage::Compute, reinterpret_cast<std::uintptr_t>(code.data()), code, 0, {}},
{32, 0, userData, compute, std::nullopt, std::nullopt, memory},
device.Target(),
{0, 0, 0, 128}
};
request.useCache = false;
const auto result = ShaderRecompiler::Recompile(request);
device.Dispatch(result, 1, 1, 1, {}, reinterpret_cast<std::uintptr_t>(code.data()));
device.WaitIdle();
}
void Check() {
std::uint32_t failures = 0;
std::string report;
for (std::uint32_t tid = 0; tid < Threads; ++tid) {
const std::uint32_t* in = &Input[tid * Inputs];
const std::uint32_t* out = &Output[tid * Results];
const auto clampLane = [&](std::uint32_t x) { return static_cast<std::uint32_t>(std::max<std::int32_t>(-1, Signed(x))) & 0xffffu; };
const std::uint32_t tsr = clampLane(in[0] & 0xffffu) | (clampLane(in[0] >> 16u) << 16u);
Require(Reference(Cases[0], in) == tsr, "packed integer negate: reference disagrees with max(-1, x)");
for (std::uint32_t i = 0; i < Results; ++i) {
const std::uint32_t expected = Reference(Cases[i], in);
if (out[i] != expected && failures++ < 24u) {
report += std::string("packed integer negate: lane ") + std::to_string(tid) + " " + Cases[i].name + " is " + Hex(out[i]) + ", expected " + Hex(expected) + "\n";
}
}
}
Require(failures == 0u, report + std::to_string(failures) + " mismatches");
}
}
int main() {
try {
const auto device = OpenVulkanTestDevice();
if (!device) return VulkanTestSkipped;
Run(*device);
Check();
std::puts("packed integer negate tests passed");
return 0;
} catch (const std::exception& error) {
std::cerr << error.what() << '\n';
return 1;
}
}

71
core/libs/prx/libSceNet/Export.cpp
View File

@ -45,6 +45,7 @@ constexpr int NET_EACCES = 13;
constexpr int NET_EFAULT = 14;
constexpr int NET_EINVAL = 22;
constexpr int NET_ENOSPC = 28;
constexpr int NET_EDOM = 33;
constexpr int NET_EAGAIN = 35;
constexpr int NET_EINPROGRESS = 36;
constexpr int NET_EALREADY = 37;
@ -198,6 +199,10 @@ bool guest_to_native_address(const void* address, std::uint32_t length,
bool native_to_guest_address(const sockaddr_storage& native, void* address, std::uint32_t* length) {
if (!address || !length) return false;
if (native.ss_family == AF_UNSPEC) {
*length = 0;
return true;
}
std::uint8_t bytes[28]{};
if (native.ss_family == AF_INET) {
const auto& v4 = reinterpret_cast<const sockaddr_in&>(native);
@ -321,6 +326,45 @@ int fail(int err) {
return NET_ERROR_BASE | err;
}
int set_native_timeout(NativeSocket native, int optname, std::int64_t microseconds) {
const int native_option = optname == NET_SO_RCVTIMEO ? SO_RCVTIMEO : SO_SNDTIMEO;
#ifdef _WIN32
const std::int64_t milliseconds = (microseconds + 999) / 1000;
const DWORD timeout_ms = milliseconds > static_cast<std::int64_t>(MAXDWORD) ? MAXDWORD : static_cast<DWORD>(milliseconds);
return ::setsockopt(native, SOL_SOCKET, native_option, reinterpret_cast<const char*>(&timeout_ms), sizeof(timeout_ms));
#else
timeval timeout{};
timeout.tv_sec = static_cast<time_t>(microseconds / 1000000);
timeout.tv_usec = static_cast<suseconds_t>(microseconds % 1000000);
return ::setsockopt(native, SOL_SOCKET, native_option, &timeout, sizeof(timeout));
#endif
}
int set_kernel_timeout(int descriptor, int optname, const void* value, std::uint32_t length) {
KernelTimeval timeout{};
if (value == nullptr && length != 0) return NET_EFAULT;
if (length < sizeof(timeout)) return NET_EINVAL;
std::memcpy(&timeout, value, sizeof(timeout));
if (timeout.tv_sec < 0 || timeout.tv_usec < 0 || timeout.tv_usec >= 1000000) return NET_EDOM;
const std::int64_t microseconds = std::min<std::int64_t>(timeout.tv_sec, INT32_MAX) * 1000000 + timeout.tv_usec;
Sock socket;
{
std::lock_guard lk(g_mutex);
const auto it = g_socks.find(descriptor);
if (it == g_socks.end()) return NET_EBADF;
socket = it->second;
}
if (set_native_timeout(socket.native->value, optname, microseconds) != 0) return native_error();
std::lock_guard lk(g_mutex);
const auto it = g_socks.find(descriptor);
if (it != g_socks.end()) {
const int stored = static_cast<int>(std::min<std::int64_t>(microseconds, INT_MAX));
if (optname == NET_SO_RCVTIMEO) it->second.rcv_timeout_us = stored;
else it->second.snd_timeout_us = stored;
}
return 0;
}
void set_resolver_error(int rid, int error) {
std::lock_guard lk(g_mutex);
const auto resolver = g_resolvers.find(rid);
@ -483,6 +527,11 @@ bool SetSocketOptionForKernel(int descriptor, int level, int option, const void*
std::lock_guard lk(g_mutex);
if (g_socks.count(descriptor) == 0) return false;
}
if (level == NET_SOL_SOCKET && (option == NET_SO_RCVTIMEO || option == NET_SO_SNDTIMEO)) {
*error = set_kernel_timeout(descriptor, option, value, length);
if (*error != 0) set_errno(*error);
return true;
}
*error = sceNetSetsockopt(descriptor, level, option, value, length) == 0 ? 0 : *errno_slot();
return true;
}
@ -529,6 +578,13 @@ int APS5_VABI sceNetSocket(const char* name, int family, int type, int protocol)
if (!initialize_sockets()) return fail(5);
const NativeSocket native = ::socket(family == NET_AF_INET ? AF_INET : AF_INET6, type, protocol);
if (native == INVALID_NATIVE_SOCKET) return fail(native_error());
const int ipv6_only = 1;
if (family == NET_AF_INET6 &&
::setsockopt(native, IPPROTO_IPV6, IPV6_V6ONLY, reinterpret_cast<const char*>(&ipv6_only), sizeof(ipv6_only)) != 0) {
const int error = native_error();
close_socket(native);
return fail(error);
}
std::shared_ptr<NativeSocketHandle> handle;
try {
handle = std::make_shared<NativeSocketHandle>(native);
@ -871,20 +927,7 @@ int APS5_VABI sceNetSetsockopt(int s, int level, int optname, const void* optval
#endif
} else if (optname == NET_SO_RCVTIMEO || optname == NET_SO_SNDTIMEO) {
if (value < 0) return fail(NET_EINVAL);
#ifdef _WIN32
const DWORD timeout_ms = value == 0 ? 0 : static_cast<DWORD>((value + 999) / 1000);
const int native_option = optname == NET_SO_RCVTIMEO ? SO_RCVTIMEO : SO_SNDTIMEO;
if (::setsockopt(socket.native->value, SOL_SOCKET, native_option,
reinterpret_cast<const char*>(&timeout_ms), sizeof(timeout_ms)) != 0)
return fail(native_error());
#else
timeval timeout{};
timeout.tv_sec = value / 1000000;
timeout.tv_usec = value % 1000000;
const int native_option = optname == NET_SO_RCVTIMEO ? SO_RCVTIMEO : SO_SNDTIMEO;
if (::setsockopt(socket.native->value, SOL_SOCKET, native_option, &timeout, sizeof(timeout)) != 0)
return fail(native_error());
#endif
if (set_native_timeout(socket.native->value, optname, value) != 0) return fail(native_error());
} else if (optname == NET_SO_REUSEADDR || optname == NET_SO_BROADCAST || optname == NET_SO_SNDBUF || optname == NET_SO_RCVBUF) {
const int native_option = optname == NET_SO_REUSEADDR ? SO_REUSEADDR : optname == NET_SO_BROADCAST ? SO_BROADCAST : optname == NET_SO_SNDBUF ? SO_SNDBUF : SO_RCVBUF;
if (::setsockopt(socket.native->value, SOL_SOCKET, native_option, reinterpret_cast<const char*>(&value), sizeof(value)) != 0)

3
core/libs/prx/libSceSaveData.native/Export.cpp
View File

@ -62,6 +62,9 @@ static bool dir_name_match(const char* str, const char* pattern) {
str++;
pattern++;
}
while (*pattern == '%') {
pattern++;
}
return *str == '\0' && *pattern == '\0';
}

29
core/libs/prx/libSceVideodec2/Export.cpp
View File

@ -147,6 +147,7 @@ struct Decoder {
std::mutex lock;
std::map<std::uint64_t, std::shared_ptr<Decoder>> decoders;
std::map<const void*, std::uint64_t> pictureOwners;
std::uint64_t nextDecoder = 1;
std::uint64_t nextQueue = 1;
@ -266,7 +267,7 @@ bool WriteNv12(Decoder& decoder, const AVFrame& frame, std::uint8_t* target, std
return sws_scale(decoder.scaler, frame.data, frame.linesize, 0, frame.height, planes, strides) == frame.height;
}
void EmitDecoded(Decoder& decoder, FrameBuffer* frame, OutputInfo* output) {
void EmitDecoded(std::uint64_t handle, Decoder& decoder, FrameBuffer* frame, OutputInfo* output) {
output->isValid = false;
output->isErrorFrame = false;
output->pictureCount = 0;
@ -302,6 +303,8 @@ void EmitDecoded(Decoder& decoder, FrameBuffer* frame, OutputInfo* output) {
output->framePitchInBytes = pitch;
}
decoder.pictures[frame->frameBuffer] = timestamps;
std::lock_guard guard(lock);
if (decoders.contains(handle)) pictureOwners[frame->frameBuffer] = handle;
}
}
@ -358,6 +361,7 @@ int APS5_VABI sceVideodec2CreateDecoder_nid_postfix(const DecoderConfigInfo* con
int APS5_VABI sceVideodec2DeleteDecoder_nid_postfix(std::uint64_t handle) {
std::lock_guard guard(lock);
if (decoders.erase(handle) == 0) throw std::runtime_error("Videodec2: invalid decoder handle");
std::erase_if(pictureOwners, [handle](const auto& owner) { return owner.second == handle; });
return 0;
}
@ -383,7 +387,7 @@ int APS5_VABI sceVideodec2Decode_nid_postfix(std::uint64_t handle, const InputDa
packet->pts = decoder.nextKey;
decoder.inputs.emplace(decoder.nextKey++, picture);
Send(decoder, packet);
EmitDecoded(decoder, frame, output);
EmitDecoded(handle, decoder, frame, output);
return 0;
}
@ -397,7 +401,7 @@ int APS5_VABI sceVideodec2Flush_nid_postfix(std::uint64_t handle, FrameBuffer* f
Send(decoder, nullptr);
decoder.flushing = true;
}
EmitDecoded(decoder, frame, output);
EmitDecoded(handle, decoder, frame, output);
return 0;
}
@ -407,6 +411,10 @@ int APS5_VABI sceVideodec2Reset_nid_postfix(std::uint64_t handle) {
auto& decoder = *found;
std::lock_guard guard(decoder.mutex);
decoder.pictures.clear();
{
std::lock_guard ownersGuard(lock);
std::erase_if(pictureOwners, [handle](const auto& owner) { return owner.second == handle; });
}
avcodec_flush_buffers(decoder.context);
decoder.ready.clear();
decoder.inputs.clear();
@ -417,15 +425,18 @@ int APS5_VABI sceVideodec2Reset_nid_postfix(std::uint64_t handle) {
int APS5_VABI sceVideodec2GetPictureInfo_nid_postfix(const OutputInfo* output, AvcPictureInfo* first, void* second) {
(void)second;
if (!output || !first) throw std::runtime_error("Videodec2: invalid argument pointer");
std::vector<std::shared_ptr<Decoder>> all;
{
const auto owner = [output]() -> std::uint64_t {
std::lock_guard guard(lock);
for (const auto& [handle, decoder] : decoders) all.push_back(decoder);
}
for (const auto& decoder : all) {
const auto found = pictureOwners.find(output->frameBuffer);
return found != pictureOwners.end() ? found->second : 0;
};
for (auto handle = owner(); handle != 0; handle = owner()) {
const auto decoder = FindDecoder(handle);
if (!decoder) continue;
std::lock_guard guard(decoder->mutex);
if (owner() != handle) continue;
const auto found = decoder->pictures.find(output->frameBuffer);
if (found == decoder->pictures.end()) continue;
if (found == decoder->pictures.end()) throw std::runtime_error("Videodec2: owned picture is missing");
if (decoder->codec != CodecAvc) throw std::runtime_error("Videodec2: picture info is only implemented for H.264");
const auto& picture = found->second;
first->isValid = true;

25
core/libs/prx/libc/CMakeLists.txt
View File

@ -17,6 +17,7 @@ target_sources(libc PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}/src/HostEntropy.cpp
${CMAKE_CURRENT_SOURCE_DIR}/src/Time.cpp
${CMAKE_CURRENT_SOURCE_DIR}/src/FileAndHeap.cpp
${CMAKE_CURRENT_SOURCE_DIR}/src/FileLock.cpp
${CMAKE_CURRENT_SOURCE_DIR}/src/StdioExtras.cpp
${CMAKE_CURRENT_SOURCE_DIR}/src/StandardStreams.cpp
${CMAKE_CURRENT_SOURCE_DIR}/src/GuestAllocations.cpp
@ -334,6 +335,16 @@ if(BUILD_TESTING)
set_tests_properties(guest_interfaces PROPERTIES ENVIRONMENT_MODIFICATION
"PATH=path_list_prepend:$<TARGET_FILE_DIR:libc>")
endif()
add_test_executable(guest_qsort_tests ${LIBS_INCLUDE_DIR}/tests/GuestQsort.cpp)
target_include_directories(guest_qsort_tests PRIVATE ${LIBS_INCLUDE_DIR})
target_link_libraries(guest_qsort_tests PRIVATE libc)
configure_windows_unwind(guest_qsort_tests)
add_test(NAME guest_qsort COMMAND guest_qsort_tests)
set_tests_properties(guest_qsort PROPERTIES TIMEOUT 20)
if(WIN32)
set_tests_properties(guest_qsort PROPERTIES ENVIRONMENT_MODIFICATION
"PATH=path_list_prepend:$<TARGET_FILE_DIR:libc>")
endif()
add_test_executable(guest_atoi_tests ${LIBS_INCLUDE_DIR}/tests/GuestAtoi.cpp)
target_include_directories(guest_atoi_tests PRIVATE ${LIBS_INCLUDE_DIR})
target_link_libraries(guest_atoi_tests PRIVATE libc)
@ -417,6 +428,11 @@ if(BUILD_TESTING)
set_tests_properties(guest_new_handler PROPERTIES ENVIRONMENT_MODIFICATION
"PATH=path_list_prepend:$<TARGET_FILE_DIR:libc>")
endif()
add_test(NAME guest_new_handler_default_heap COMMAND guest_new_handler_tests default)
if(WIN32)
set_tests_properties(guest_new_handler_default_heap PROPERTIES ENVIRONMENT_MODIFICATION
"PATH=path_list_prepend:$<TARGET_FILE_DIR:libc>")
endif()
add_test_executable(guest_search_tests ${LIBS_INCLUDE_DIR}/tests/GuestSearch.cpp)
target_include_directories(guest_search_tests PRIVATE ${LIBS_INCLUDE_DIR})
target_link_libraries(guest_search_tests PRIVATE libc)
@ -462,6 +478,15 @@ if(BUILD_TESTING)
set_tests_properties(guest_wcsrtombs_s PROPERTIES ENVIRONMENT_MODIFICATION
"PATH=path_list_prepend:$<TARGET_FILE_DIR:libc>")
endif()
add_test_executable(guest_overflow_error_tests ${LIBS_INCLUDE_DIR}/tests/GuestOverflowError.cpp)
target_include_directories(guest_overflow_error_tests PRIVATE ${LIBS_INCLUDE_DIR})
target_link_libraries(guest_overflow_error_tests PRIVATE libc)
configure_windows_unwind(guest_overflow_error_tests)
add_test(NAME guest_overflow_error COMMAND guest_overflow_error_tests)
if(WIN32)
set_tests_properties(guest_overflow_error PROPERTIES ENVIRONMENT_MODIFICATION
"PATH=path_list_prepend:$<TARGET_FILE_DIR:libc>")
endif()
if(UNIX)
add_test_executable(exception_runtime_tests ${LIBS_INCLUDE_DIR}/tests/ExceptionRuntime.cpp)
target_include_directories(exception_runtime_tests PRIVATE ${LIBS_INCLUDE_DIR})

12
core/libs/prx/libc/include/FileLock.hpp Normal file
View File

@ -0,0 +1,12 @@
#ifndef CORE_LIBS_PRX_LIBC_INCLUDE_FILELOCK_HPP
#define CORE_LIBS_PRX_LIBC_INCLUDE_FILELOCK_HPP
#ifdef _WIN32
extern "C" {
int GuestFileLock_nid_no_patch(int fd, int operation);
void GuestFileLockShare_nid_no_patch(int from, int to);
void GuestFileLockForget_nid_no_patch(int fd);
}
#endif
#endif

5
core/libs/prx/libc/include/FileStream.hpp
View File

@ -12,6 +12,7 @@
#include <type_traits>
#ifdef _WIN32
#include <io.h>
#include "prx/libc/include/FileLock.hpp"
#endif
struct GuestFilePrefix {
@ -76,6 +77,7 @@ public:
#ifdef _WIN32
std::wstring wideMode;
for (const char character : std::string_view(mode)) wideMode.push_back(static_cast<unsigned char>(character));
GuestFileLockForget_nid_no_patch(::_fileno(previous));
_handle = ::_wfreopen(filename.c_str(), wideMode.c_str(), previous);
#else
_handle = std::freopen(filename.c_str(), mode, previous);
@ -124,6 +126,9 @@ public:
int Close() {
GetHandle();
#ifdef _WIN32
GuestFileLockForget_nid_no_patch(::_fileno(_handle));
#endif
const int result = std::fclose(std::exchange(_handle, nullptr));
_guest.flags = 0;
_guest.descriptor = -1;

22
core/libs/prx/libc/include/SonyCrt.hpp
View File

@ -181,8 +181,7 @@ inline int SonyFltrounds(void) {
}
}
inline unsigned __int128 SonyUdivti3(unsigned __int128 dividend, unsigned __int128 divisor) {
if (divisor == 0) throw std::runtime_error("__udivti3: integer divide by zero");
inline unsigned __int128 SonyUdivmod128(unsigned __int128 dividend, unsigned __int128 divisor, unsigned __int128* remainderOut) {
unsigned __int128 quotient = 0;
unsigned __int128 remainder = 0;
for (int i = 127; i >= 0; --i) {
@ -192,9 +191,28 @@ inline unsigned __int128 SonyUdivti3(unsigned __int128 dividend, unsigned __int1
quotient |= (unsigned __int128)1 << i;
}
}
*remainderOut = remainder;
return quotient;
}
inline unsigned __int128 SonyUdivti3(unsigned __int128 dividend, unsigned __int128 divisor) {
if (divisor == 0) throw std::runtime_error("__udivti3: integer divide by zero");
unsigned __int128 remainder = 0;
return SonyUdivmod128(dividend, divisor, &remainder);
}
inline unsigned __int128 SonyUmodti3(unsigned __int128 dividend, unsigned __int128 divisor) {
if (divisor == 0) {
char message[96];
std::snprintf(message, sizeof(message), "__umodti3: integer divide by zero (0x%016llx%016llx %% 0)",
(unsigned long long)(dividend >> 64), (unsigned long long)dividend);
throw std::runtime_error(message);
}
unsigned __int128 remainder = 0;
SonyUdivmod128(dividend, divisor, &remainder);
return remainder;
}
inline __int128 SonyDivti3(__int128 dividend, __int128 divisor) {
if (divisor == 0) throw std::runtime_error("__divti3: integer divide by zero");
const bool negative = (dividend < 0) != (divisor < 0);

8
core/libs/prx/libc/include/WindowsFormatting.hpp
View File

@ -3,6 +3,7 @@
#include "SceTypes.hpp"
#include <array>
#include <cerrno>
#include <cstdint>
#include <cstdio>
#include <climits>
@ -13,6 +14,8 @@
#include <vector>
#include <type_traits>
extern "C" int APS5_VABI strerror_r_nid_postfix(int error, char* buffer, std::size_t length);
namespace LibcDetail {
enum class ArgumentClass : unsigned char { Integer, Double, LongDouble };
@ -240,6 +243,7 @@ inline std::vector<ArgumentClass> NumberedArgumentClasses(const char* format) {
inline int FormatWindows(char* buffer, size_t size, const char* format, const void* source, std::string* complete = nullptr) {
if (!format || !source) throw std::invalid_argument("Null formatting argument");
const int savedError = errno;
const char* const formatStart = format;
FormatArguments args(source);
args.Number(NumberedArgumentClasses(format));
@ -360,6 +364,10 @@ inline int FormatWindows(char* buffer, size_t size, const char* format, const vo
} else if (conversion == 'p' && length.empty()) {
const std::string text = FormatPointer(spec, args.Next<void*>());
output.Append(text.data(), text.size());
} else if (conversion == 'm') {
char message[128];
strerror_r_nid_postfix(savedError, message, sizeof(message));
output.Value(spec + 's', static_cast<const char*>(message));
} else if (conversion == 'n' && integerLength && spec == "%") {
void* pointer = args.Next<void*>();
if (!pointer) throw std::invalid_argument("Null format count pointer");

136
core/libs/prx/libc/src/FileAndHeap.cpp
View File

@ -1,3 +1,4 @@
#include <algorithm>
#include <cstddef>
#include <cstdint>
#include <cstdio>
@ -26,10 +27,15 @@
#include "prx/libc/include/FilesystemError.hpp"
#include "prx/libc/include/GuestArena.hpp"
#ifdef __linux__
#include <stdio_ext.h>
#endif
struct GuestFileMode {
std::string native;
bool writes = false;
bool exclusive = false;
bool append = false;
};
static std::optional<GuestFileMode> ParseFileMode(const char* mode) {
@ -42,7 +48,7 @@ static std::optional<GuestFileMode> ParseFileMode(const char* mode) {
if (*flag == 'x') exclusive = true;
}
if (exclusive && access == 'r' && !update) return std::nullopt;
GuestFileMode result{std::string(1, access), access != 'r' || update, exclusive};
GuestFileMode result{std::string(1, access), access != 'r' || update, exclusive, access == 'a'};
if (update) result.native += '+';
#ifdef _WIN32
result.native += 'b';
@ -81,6 +87,100 @@ static std::FILE* OpenNative(const std::filesystem::path& path, const std::strin
#endif
}
using GuestCompare = int (APS5_VABI *)(const void*, const void*);
static void SwapElements(unsigned char* left, unsigned char* right, size_t size) {
for (size_t index = 0; index < size; ++index) std::swap(left[index], right[index]);
}
static void SwapRanges(unsigned char* left, unsigned char* right, size_t bytes) {
if (bytes > 0) SwapElements(left, right, bytes);
}
static unsigned char* MedianOfThree(unsigned char* a, unsigned char* b, unsigned char* c, GuestCompare compare) {
return compare(a, b) < 0
? (compare(b, c) < 0 ? b : (compare(a, c) < 0 ? c : a))
: (compare(b, c) > 0 ? b : (compare(a, c) < 0 ? a : c));
}
static void InsertionSort(unsigned char* base, size_t count, size_t size, GuestCompare compare) {
for (unsigned char* next = base + size; next < base + count * size; next += size)
for (unsigned char* element = next; element > base && compare(element - size, element) > 0; element -= size)
SwapElements(element, element - size, size);
}
static void SortElements(unsigned char* base, size_t count, size_t size, GuestCompare compare) {
for (;;) {
if (count < 7) {
InsertionSort(base, count, size, compare);
return;
}
unsigned char* middle = base + (count / 2) * size;
if (count > 7) {
unsigned char* low = base;
unsigned char* high = base + (count - 1) * size;
if (count > 40) {
const size_t step = (count / 8) * size;
low = MedianOfThree(low, low + step, low + 2 * step, compare);
middle = MedianOfThree(middle - step, middle, middle + step, compare);
high = MedianOfThree(high - 2 * step, high - step, high, compare);
}
middle = MedianOfThree(low, middle, high, compare);
}
SwapElements(base, middle, size);
unsigned char* equalLow = base + size;
unsigned char* scanLow = equalLow;
unsigned char* scanHigh = base + (count - 1) * size;
unsigned char* equalHigh = scanHigh;
bool swapped = false;
for (;;) {
int result = 0;
while (scanLow <= scanHigh && (result = compare(scanLow, base)) <= 0) {
if (result == 0) {
swapped = true;
SwapElements(equalLow, scanLow, size);
equalLow += size;
}
scanLow += size;
}
while (scanLow <= scanHigh && (result = compare(scanHigh, base)) >= 0) {
if (result == 0) {
swapped = true;
SwapElements(scanHigh, equalHigh, size);
equalHigh -= size;
}
scanHigh -= size;
}
if (scanLow > scanHigh) break;
SwapElements(scanLow, scanHigh, size);
swapped = true;
scanLow += size;
scanHigh -= size;
}
if (!swapped) {
InsertionSort(base, count, size, compare);
return;
}
unsigned char* const end = base + count * size;
size_t moved = std::min<size_t>(equalLow - base, scanLow - equalLow);
SwapRanges(base, scanLow - moved, moved);
moved = std::min<size_t>(equalHigh - scanHigh, end - equalHigh - size);
SwapRanges(scanLow, end - moved, moved);
const size_t lowBytes = scanLow - equalLow;
const size_t highBytes = equalHigh - scanHigh;
if (lowBytes <= highBytes) {
if (lowBytes > size) SortElements(base, lowBytes / size, size, compare);
if (highBytes <= size) return;
base = end - highBytes;
count = highBytes / size;
} else {
if (highBytes > size) SortElements(end - highBytes, highBytes / size, size, compare);
if (lowBytes <= size) return;
count = lowBytes / size;
}
}
}
extern "C" {
[[noreturn]] void APS5_VABI _ZSt11_Xbad_allocv_nid_postfix();
@ -123,6 +223,7 @@ FileStream* APS5_VABI freopen_nid_postfix(const char* filename, const char* mode
}
}
if (stream->Reopen(path, parsed->native.c_str())) {
if (parsed->append) static_cast<void>(std::fseek(stream->GetHandle(), 0, SEEK_END));
if (!path.empty() && parsed->writes) RecordWrittenPath_nid_no_patch(path);
return stream;
}
@ -168,6 +269,7 @@ FileStream* APS5_VABI fopen_nid_postfix(const char* filename, const char* mode)
throw std::runtime_error(std::string(__func__) + ": " + FOPEN_MSG_OPEN_FAILED + ": \"" + abs_path + "\": " + reason);
}
// APS5_LOG_OUT("success: \"%s\"", abs_path.c_str());
if (parsed->append) static_cast<void>(std::fseek(handle.get(), 0, SEEK_END));
if (parsed->writes) RecordWrittenPath_nid_no_patch(fpath);
auto stream = std::make_unique<FileStream>(handle.get(), true);
handle.release();
@ -228,7 +330,6 @@ size_t APS5_VABI fread_nid_postfix(void* buffer, size_t size, size_t count, File
if (!destination.Open()) throw std::runtime_error("fread: buffer is not writable");
const auto result = std::fread(buffer, size, count, handle);
stream->SyncStatus();
if (std::ferror(handle)) throw std::runtime_error("fread: read failed");
return result;
}
@ -238,7 +339,6 @@ size_t APS5_VABI fwrite_nid_postfix(const void* buffer, size_t size, size_t coun
if (!buffer) throw std::runtime_error("fwrite: null buffer");
const auto result = std::fwrite(buffer, size, count, handle);
stream->SyncStatus();
if (result != count || std::ferror(handle)) throw std::runtime_error("fwrite: write failed");
return result;
}
@ -295,12 +395,16 @@ void APS5_VABI rewind_nid_postfix(FileStream* stream) {
int APS5_VABI fputs_nid_postfix(const char* str, FileStream* stream) {
if (!str) throw std::runtime_error("fputs: null string");
const int result = std::fputs(str, GetNativeStream(stream));
if (result == EOF) throw std::runtime_error("fputs: write failed");
stream->SyncStatus();
return result;
}
int APS5_VABI fflush_nid_postfix(FileStream* stream) {
if (std::fflush(stream ? GetNativeStream(stream) : nullptr) != 0) throw std::runtime_error("fflush: flush failed");
std::FILE* native = stream ? GetNativeStream(stream) : nullptr;
#ifdef __linux__
if (native && !__fwriting(native)) return 0;
#endif
if (std::fflush(native) != 0) throw std::runtime_error("fflush: flush failed");
return 0;
}
@ -388,27 +492,7 @@ void APS5_VABI qsort_nid_postfix(void* base, size_t count, size_t size, int (APS
if (count > std::numeric_limits<size_t>::max() / size) throw std::overflow_error("qsort: array size overflow");
if (count == 1) return;
auto* bytes = static_cast<unsigned char*>(base);
const auto swapElements = [bytes, size](size_t left, size_t right) {
auto* leftElement = bytes + left * size;
auto* rightElement = bytes + right * size;
for (size_t index = 0; index < size; ++index) std::swap(leftElement[index], rightElement[index]);
};
const auto siftDown = [bytes, size, compare, &swapElements](size_t root, size_t heapSize) {
while (root < heapSize / 2) {
size_t child = root * 2 + 1;
if (child + 1 < heapSize && compare(bytes + child * size, bytes + (child + 1) * size) < 0) ++child;
if (compare(bytes + root * size, bytes + child * size) >= 0) return;
swapElements(root, child);
root = child;
}
};
for (size_t parent = count / 2; parent != 0; --parent) siftDown(parent - 1, count);
for (size_t heapSize = count; heapSize > 1;) {
swapElements(0, --heapSize);
siftDown(0, heapSize);
}
SortElements(static_cast<unsigned char*>(base), count, size, compare);
}
}

83
core/libs/prx/libc/src/FileLock.cpp Normal file
View File

@ -0,0 +1,83 @@
#include "prx/libc/include/FileLock.hpp"
#ifdef _WIN32
#include <cerrno>
#include <io.h>
#include <map>
#include <memory>
#include <mutex>
#include <windows.h>
namespace {
constexpr int LockShared = 1;
constexpr int LockExclusive = 2;
constexpr int LockNonblocking = 4;
constexpr int LockUnlock = 8;
struct LockState {
std::mutex mutex;
int mode = 0;
};
std::mutex g_mutex;
std::map<int, std::shared_ptr<LockState>> g_locks;
std::shared_ptr<LockState> Materialize(int fd) {
const auto found = g_locks.find(fd);
if (found != g_locks.end()) return found->second;
return g_locks.emplace(fd, std::make_shared<LockState>()).first->second;
}
}
extern "C" {
int GuestFileLock_nid_no_patch(int fd, int operation) {
const int mode = operation & (LockShared | LockExclusive | LockUnlock);
if (mode != LockShared && mode != LockExclusive && mode != LockUnlock) {
errno = EINVAL;
::SetLastError(ERROR_INVALID_PARAMETER);
return -1;
}
const auto handle = reinterpret_cast<HANDLE>(::_get_osfhandle(fd));
if (handle == INVALID_HANDLE_VALUE) {
errno = EBADF;
::SetLastError(ERROR_INVALID_HANDLE);
return -1;
}
const auto state = [&] {
std::lock_guard lock(g_mutex);
return Materialize(fd);
}();
std::lock_guard lock(state->mutex);
if (state->mode == mode) return 0;
if (state->mode != 0) {
OVERLAPPED overlapped{};
if (!::UnlockFileEx(handle, 0, MAXDWORD, MAXDWORD, &overlapped)) return -1;
state->mode = 0;
}
if (mode == LockUnlock) return 0;
DWORD flags = mode == LockExclusive ? LOCKFILE_EXCLUSIVE_LOCK : 0;
if (operation & LockNonblocking) flags |= LOCKFILE_FAIL_IMMEDIATELY;
OVERLAPPED overlapped{};
if (!::LockFileEx(handle, flags, 0, MAXDWORD, MAXDWORD, &overlapped)) return -1;
state->mode = mode;
return 0;
}
void GuestFileLockShare_nid_no_patch(int from, int to) {
std::lock_guard lock(g_mutex);
auto state = Materialize(from);
g_locks[to] = std::move(state);
}
void GuestFileLockForget_nid_no_patch(int fd) {
std::lock_guard lock(g_mutex);
g_locks.erase(fd);
}
}
#endif

43
core/libs/prx/libc/src/Formatting.cpp
View File

@ -21,6 +21,13 @@
#include "prx/libc/include/WindowsScanning.hpp"
#endif
#if defined(__GLIBC__)
extern "C" {
int __isoc99_vsscanf(const char*, const char*, std::va_list) noexcept;
int __isoc99_vfscanf(std::FILE*, const char*, std::va_list);
}
#endif
namespace {
#ifndef _WIN32
@ -30,7 +37,7 @@ bool HasGuestWideFormat(const char* format) {
++format;
if (*format == '%') { ++format; continue; }
while (*format && std::strchr("-+ #0'0123456789.*$", *format)) ++format;
if (*format == 'S' || *format == 'C' || (*format == 'l' && (format[1] == 's' || format[1] == 'c'))) return true;
if (*format == 'S' || *format == 'C' || *format == 'm' || (*format == 'l' && (format[1] == 's' || format[1] == 'c'))) return true;
if (*format) ++format;
}
return false;
@ -64,6 +71,32 @@ std::string HostLength(const std::string& length, char conversion) {
return length;
}
int HostVsscanf(const char* input, const char* format, std::va_list args) {
#if defined(__GLIBC__)
return __isoc99_vsscanf(input, format, args);
#else
return std::vsscanf(input, format, args);
#endif
}
int HostSscanf(const char* input, const char* format, ...) {
std::va_list args;
va_start(args, format);
const int result = HostVsscanf(input, format, args);
va_end(args);
return result;
}
#ifndef _WIN32
int HostVfscanf(std::FILE* stream, const char* format, std::va_list args) {
#if defined(__GLIBC__)
return __isoc99_vfscanf(stream, format, args);
#else
return std::vfscanf(stream, format, args);
#endif
}
#endif
template <typename NextPointer, typename NextCapacity>
int ScanGuest(const char* buffer, const char* format, bool secure, NextPointer nextPointer, NextCapacity nextCapacity) {
if (buffer == nullptr || format == nullptr) return EOF;
@ -147,7 +180,7 @@ int ScanGuest(const char* buffer, const char* format, bool secure, NextPointer n
directive += specifier;
directive += "%n";
int consumed = -1;
const int matched = suppress ? std::sscanf(input, directive.c_str(), &consumed) : std::sscanf(input, directive.c_str(), target, &consumed);
const int matched = suppress ? HostSscanf(input, directive.c_str(), &consumed) : HostSscanf(input, directive.c_str(), target, &consumed);
if (consumed < 0 || (!suppress && matched != 1)) return finish(!converted && *input == '\0' ? EOF : assigned);
if (capped && conversion != 'c') {
const char next = input[consumed];
@ -257,7 +290,7 @@ int APS5_VABI fscanf_nid_postfix(FileStream* stream, const char* format, ...) {
RejectGuestWideScan(format);
std::va_list args;
va_start(args, format);
const int result = std::vfscanf(native, format, args);
const int result = HostVfscanf(native, format, args);
va_end(args);
#endif
stream->SyncStatus();
@ -341,7 +374,7 @@ int APS5_VABI vsscanf_nid_postfix(const char* input, const char* format, VaList*
RejectGuestWideScan(format);
std::va_list copy;
va_copy(copy, *reinterpret_cast<std::va_list*>(args));
const int result = std::vsscanf(input, format, copy);
const int result = HostVsscanf(input, format, copy);
va_end(copy);
return result;
#endif
@ -360,7 +393,7 @@ int APS5_VABI sscanf_nid_postfix(const char* input, const char* format, ...) {
RejectGuestWideScan(format);
std::va_list args;
va_start(args, format);
const int result = std::vsscanf(input, format, args);
const int result = HostVsscanf(input, format, args);
va_end(args);
return result;
}

24
core/libs/prx/libc/src/FormattingWide.cpp
View File

@ -91,22 +91,38 @@ void AppendPadded(std::u16string& out, const std::u16string& text, bool left, in
std::u16string FormatWide(const char16_t* format, VaList* source, bool secure = false) {
if (format == nullptr || source == nullptr) throw std::invalid_argument("Null formatting argument");
LibcDetail::FormatArguments args(source);
const char16_t* const start = format;
std::string ascii;
for (const char16_t* unit = format; *unit != 0; ++unit) ascii.push_back(*unit < 0x80 ? static_cast<char>(*unit) : '?');
args.Number(LibcDetail::NumberedArgumentClasses(ascii.c_str()));
const auto position = [&] {
const char* cursor = ascii.c_str() + (format - start);
const std::size_t found = LibcDetail::ArgumentPosition(cursor);
format = start + (cursor - ascii.c_str());
return found;
};
const auto star = [&] {
const std::size_t found = position();
return found ? args.At<int>(found) : args.Next<int>();
};
std::u16string out;
while (*format) {
if (*format != u'%') { out.push_back(*format++); continue; }
++format;
if (*format == u'%') { out.push_back(*format++); continue; }
if (const std::size_t found = position()) args.Seek(found);
std::string spec = "%";
bool left = false;
int width = 0;
int precision = -1;
while (In(*format, "-+ #0")) {
while (In(*format, "-+ #0'")) {
if (*format == u'-') left = true;
spec += static_cast<char>(*format++);
if (*format != u'\'') spec += static_cast<char>(*format);
++format;
}
if (*format == u'*') {
++format;
width = args.Next<int>();
width = star();
if (width < 0) { left = true; spec += '-'; width = -width; }
spec += std::to_string(width);
} else {
@ -117,7 +133,7 @@ std::u16string FormatWide(const char16_t* format, VaList* source, bool secure =
precision = 0;
if (*format == u'*') {
++format;
precision = args.Next<int>();
precision = star();
if (precision >= 0) spec += "." + std::to_string(precision);
} else {
spec += '.';

4
core/libs/prx/libc/src/RuntimeSupport.cpp
View File

@ -188,4 +188,8 @@ unsigned __int128 APS5_VABI __udivti3_nid_postfix(unsigned __int128 dividend, un
return SonyUdivti3(dividend, divisor);
}
unsigned __int128 APS5_VABI __umodti3_nid_postfix(unsigned __int128 dividend, unsigned __int128 divisor) {
return SonyUmodti3(dividend, divisor);
}
}

4
core/libs/prx/libc/src/StdioExtras.cpp
View File

@ -5,6 +5,7 @@
#include <cstdlib>
#include <cstring>
#include <new>
#include <stdexcept>
#include "prx/libc/include/FileStream.hpp"
#include "prx/libc/include/ApplicationHeap.hpp"
@ -29,6 +30,9 @@ void* Allocate(std::size_t size) {
} catch (const std::bad_alloc&) {
handler = g_newHandler.load();
if (handler == nullptr) throw;
} catch (const std::length_error&) {
handler = g_newHandler.load();
if (handler == nullptr) throw std::bad_alloc();
}
handler();
}

6
core/libs/prx/libc/src/Strings.cpp
View File

@ -311,7 +311,9 @@ int APS5_VABI strcat_s_nid_postfix(char* dest, size_t destsz, const char* src) {
int APS5_VABI memcpy_s_nid_postfix(void* dest, size_t destsz, const void* src, size_t count) {
constexpr int GuestEinval = 22;
constexpr int GuestErange = 34;
constexpr size_t RsizeMax = SIZE_MAX >> 1;
if (!dest) return GuestEinval;
if (destsz > RsizeMax) return GuestErange;
if (!src || count > destsz) {
std::memset(dest, 0, destsz);
return src ? GuestErange : GuestEinval;
@ -330,7 +332,9 @@ int APS5_VABI memcpy_s_nid_postfix(void* dest, size_t destsz, const void* src, s
int APS5_VABI memmove_s_nid_postfix(void* dest, size_t destsz, const void* src, size_t count) {
constexpr int GuestEinval = 22;
constexpr int GuestErange = 34;
constexpr size_t RsizeMax = SIZE_MAX >> 1;
if (!dest) return GuestEinval;
if (destsz > RsizeMax) return GuestErange;
if (!src || count > destsz) {
std::memset(dest, 0, destsz);
return src ? GuestErange : GuestEinval;
@ -504,7 +508,7 @@ size_t APS5_VABI wcsxfrm_nid_postfix(char16_t* destination, const char16_t* sour
}
size_t APS5_VABI strxfrm_nid_postfix(char* destination, const char* source, size_t count) {
return std::strxfrm(destination, source, count);
return strlcpy_nid_postfix(destination, source, count);
}
char* APS5_VABI strtok_s_nid_postfix(char* str, const char* delim, char** context) {

10
core/libs/prx/libc/src/Syslog.cpp
View File

@ -12,6 +12,7 @@
extern "C" {
int* APS5_VABI __error_nid_postfix();
int APS5_VABI strerror_r_nid_postfix(int error, char* buffer, std::size_t length);
int APS5_VABI vsnprintf_nid_postfix(char* buffer, std::size_t size, const char* format, VaList* args);
}
namespace {
@ -59,15 +60,10 @@ std::string Format(const char* format, VaList* args) {
LibcDetail::FormatWindows(nullptr, 0, format, args, &buffer);
return buffer;
#else
std::va_list copy;
va_copy(copy, *reinterpret_cast<std::va_list*>(args));
const int size = std::vsnprintf(nullptr, 0, format, copy);
va_end(copy);
const int size = vsnprintf_nid_postfix(nullptr, 0, format, args);
if (size <= 0) return {};
std::string buffer(static_cast<std::size_t>(size) + 1, '\0');
va_copy(copy, *reinterpret_cast<std::va_list*>(args));
std::vsnprintf(buffer.data(), buffer.size(), format, copy);
va_end(copy);
vsnprintf_nid_postfix(buffer.data(), buffer.size(), format, args);
buffer.resize(static_cast<std::size_t>(size));
return buffer;
#endif

4
core/libs/prx/libc/src/exception/StandardExceptions.cpp
View File

@ -291,6 +291,10 @@ void APS5_VABI _ZNSt11range_errorD1Ev_nid_postfix(LibcException::ExceptionObject
void APS5_VABI _ZNSt11range_errorD2Ev_nid_postfix(LibcException::ExceptionObject* self) { LibcException::DestroyMessage(self); }
void APS5_VABI _ZNSt11range_errorD0Ev_nid_postfix(LibcException::ExceptionObject* self) { LibcException::DeleteMessage(self); }
LibcException::TypeRecord _ZTISt14overflow_error_nid_postfix {LibcException::SingleTypeVtable + 2, "St14overflow_error", &_ZTISt13runtime_error_nid_postfix};
LibcException::ExceptionVtable _ZTVSt14overflow_error_nid_postfix {0, &_ZTISt14overflow_error_nid_postfix, LibcException::GuestDestroyMessage_nid_no_patch, LibcException::GuestDeleteMessage_nid_no_patch, LibcException::GuestMessageWhat_nid_no_patch};
void APS5_VABI _ZNSt14overflow_errorD1Ev_nid_postfix(LibcException::ExceptionObject* self) { LibcException::DestroyMessage(self); }
LibcException::TypeRecord _ZTINSt8ios_base7failureE_nid_postfix {LibcException::SingleTypeVtable + 2, "NSt8ios_base7failureE", &_ZTISt12system_error_nid_postfix};
LibcException::ExceptionVtable _ZTVNSt8ios_base7failureE_nid_postfix {0, &_ZTINSt8ios_base7failureE_nid_postfix, LibcException::GuestDestroyMessage_nid_no_patch, LibcException::GuestDeleteMessage_nid_no_patch, LibcException::GuestMessageWhat_nid_no_patch};
void APS5_VABI _ZNSt8ios_base7failureD1Ev_nid_postfix(LibcException::ExceptionObject* self) { LibcException::DestroyMessage(self); }

71
core/libs/prx/libc/src/exception/Unwind.cpp
View File

@ -90,6 +90,10 @@ struct Frame {
std::intptr_t dataAlign {};
unsigned returnRegister {};
bool signal {};
#ifdef _WIN32
bool foreign {};
_Unwind_Context caller {};
#endif
};
bool DecodeRange(Frame& frame, const Lookup& query, const Byte*& p, const char*& augmentation, Byte& lsdaEncoding) {
@ -192,6 +196,42 @@ const FrameIndex& IndexFrames(const ImageKey& key, const Byte* begin, const Byte
std::sort(index.begin(), index.end());
return index;
}
const IMAGE_NT_HEADERS64* ImageHeaders(Word pc, const Byte*& base) {
void* image = nullptr;
if (!RtlPcToFileHeader(reinterpret_cast<void*>(pc), &image) || !image) return nullptr;
base = static_cast<const Byte*>(image);
const auto* dos = reinterpret_cast<const IMAGE_DOS_HEADER*>(base);
if (dos->e_magic != IMAGE_DOS_SIGNATURE) return nullptr;
const auto* nt = reinterpret_cast<const IMAGE_NT_HEADERS64*>(base + dos->e_lfanew);
return nt->Signature == IMAGE_NT_SIGNATURE ? nt : nullptr;
}
bool VirtualUnwind(const _Unwind_Context& context, _Unwind_Context& caller) {
const Word pc = context.registers[16];
const Byte* base = nullptr;
const auto* nt = ImageHeaders(pc, base);
if (!nt) return false;
const auto* sections = IMAGE_FIRST_SECTION(nt);
for (unsigned i = 0; i < nt->FileHeader.NumberOfSections; ++i)
if (std::memcmp(sections[i].Name, ".ehmeta", 8) == 0 || std::memcmp(sections[i].Name, ".ehfram", 8) == 0) return false;
DWORD64 imageBase = 0;
auto* function = RtlLookupFunctionEntry(pc, &imageBase, nullptr);
if (!function) return false;
CONTEXT state {};
DWORD64* const registers[17] {&state.Rax, &state.Rdx, &state.Rcx, &state.Rbx, &state.Rsi, &state.Rdi, &state.Rbp, &state.Rsp,
&state.R8, &state.R9, &state.R10, &state.R11, &state.R12, &state.R13, &state.R14, &state.R15, &state.Rip};
for (unsigned i = 0; i < 17; ++i) *registers[i] = context.registers[i];
for (unsigned i = 0; i < 16; ++i) std::memcpy(&state.FltSave.XmmRegisters[i], context.vectorRegisters[i], 16);
void* handlerData = nullptr;
DWORD64 establisher = 0;
if (RtlVirtualUnwind(UNW_FLAG_EHANDLER | UNW_FLAG_UHANDLER, imageBase, pc, function, &state, &handlerData, &establisher, nullptr)) return false;
caller = context;
for (unsigned i = 0; i < 17; ++i) caller.registers[i] = *registers[i];
for (unsigned i = 0; i < 16; ++i) std::memcpy(caller.vectorRegisters[i], &state.FltSave.XmmRegisters[i], 16);
caller.signalFrame = false;
return true;
}
#endif
bool DecodeFrame(_Unwind_Context& context, Frame& frame) {
@ -224,13 +264,9 @@ bool DecodeFrame(_Unwind_Context& context, Frame& frame) {
#endif
return DecodeCandidate(context, frame, query);
#else
void* image = nullptr;
if (!RtlPcToFileHeader(reinterpret_cast<void*>(query.pc), &image) || !image) return false;
const auto* base = static_cast<const Byte*>(image);
const auto* dos = reinterpret_cast<const IMAGE_DOS_HEADER*>(base);
if (dos->e_magic != IMAGE_DOS_SIGNATURE) return false;
const auto* nt = reinterpret_cast<const IMAGE_NT_HEADERS64*>(base + dos->e_lfanew);
if (nt->Signature != IMAGE_NT_SIGNATURE) return false;
const Byte* base = nullptr;
const auto* nt = ImageHeaders(query.pc, base);
if (!nt) return false;
const auto* sections = IMAGE_FIRST_SECTION(nt);
for (unsigned i = 0; i < nt->FileHeader.NumberOfSections; ++i) {
const auto& section = sections[i];
@ -385,7 +421,18 @@ bool Expression(const Byte* p, const _Unwind_Context& context, Word cfa, Word& r
}
bool GetRules(_Unwind_Context& context, Frame& frame, Rules& rules) {
if (!DecodeFrame(context, frame)) return false;
if (!DecodeFrame(context, frame)) {
#ifdef _WIN32
frame = {};
if (!VirtualUnwind(context, frame.caller)) return false;
frame.foreign = true;
context.cfa = frame.caller.registers[7];
context.region = context.lsda = context.personality = context.textBase = context.dataBase = 0;
return true;
#else
return false;
#endif
}
Rules initial;
if (!Instructions(frame.cieBegin, frame.cieEnd, frame, ~Word(0), initial, {})) return false;
#ifdef _WIN32
@ -406,6 +453,14 @@ bool GetRules(_Unwind_Context& context, Frame& frame, Rules& rules) {
bool Step(_Unwind_Context& context) {
Frame frame; Rules rules;
if (!GetRules(context, frame, rules)) return false;
#ifdef _WIN32
if (frame.foreign) {
const auto& next = frame.caller;
if (!next.registers[16] || (next.registers[7] == context.registers[7] && next.registers[16] == context.registers[16])) return false;
context = next;
return true;
}
#endif
_Unwind_Context next = context;
for (unsigned i = 0; i < 17; ++i) {
const auto& rule = rules.registers[i];

27
core/libs/prx/libkernel/CMakeLists.txt
View File

@ -178,6 +178,24 @@ if(BUILD_TESTING)
if(WIN32)
set_tests_properties(guest_dynamic_loader PROPERTIES ENVIRONMENT_MODIFICATION
"PATH=path_list_prepend:$<TARGET_FILE_DIR:libkernel>")
enable_language(ASM)
add_library(guest_deferred_module_fixture SHARED
${LIBS_INCLUDE_DIR}/tests/GuestDeferredModuleFixture.cpp
${LIBS_INCLUDE_DIR}/tests/GuestDeferredModuleFixture.S)
target_include_directories(guest_deferred_module_fixture PRIVATE ${LIBS_INCLUDE_DIR})
target_link_options(guest_deferred_module_fixture PRIVATE -Wl,--image-base,0x38000000 -Wl,--disable-dynamicbase)
configure_windows_unwind(guest_deferred_module_fixture)
set_target_properties(guest_deferred_module_fixture PROPERTIES PREFIX "" SUFFIX ".guest.prx")
add_test_executable(guest_deferred_module_tests ${LIBS_INCLUDE_DIR}/tests/GuestDeferredModule.cpp)
target_include_directories(guest_deferred_module_tests PRIVATE ${LIBS_INCLUDE_DIR})
target_link_libraries(guest_deferred_module_tests PRIVATE libkernel libc)
configure_windows_unwind(guest_deferred_module_tests)
add_dependencies(guest_deferred_module_tests guest_deferred_module_fixture)
foreach(mode IN ITEMS reload pinned parallel reentrant)
add_test(NAME guest_deferred_module_${mode} COMMAND guest_deferred_module_tests $<TARGET_FILE:guest_deferred_module_fixture> ${mode})
set_tests_properties(guest_deferred_module_${mode} PROPERTIES TIMEOUT 20 ENVIRONMENT_MODIFICATION
"PATH=path_list_prepend:$<TARGET_FILE_DIR:libkernel>")
endforeach()
endif()
add_test_executable(guest_module_info_tests ${LIBS_INCLUDE_DIR}/tests/GuestModuleInfo.cpp)
target_include_directories(guest_module_info_tests PRIVATE ${LIBS_INCLUDE_DIR})
@ -1119,6 +1137,15 @@ if(BUILD_TESTING)
if(WIN32)
set_tests_properties(guest_kernel_fcntl PROPERTIES ENVIRONMENT_MODIFICATION
"PATH=path_list_prepend:$<TARGET_FILE_DIR:libkernel>")
add_test_executable(guest_file_lock_tests ${LIBS_INCLUDE_DIR}/tests/GuestFileLock.cpp)
target_include_directories(guest_file_lock_tests PRIVATE ${LIBS_INCLUDE_DIR})
target_link_libraries(guest_file_lock_tests PRIVATE libkernel libc)
configure_windows_unwind(guest_file_lock_tests)
foreach(fileLockCase IN ITEMS duplicate lifetime conversion threads streams process)
add_test(NAME guest_file_lock_${fileLockCase} COMMAND guest_file_lock_tests ${fileLockCase})
set_tests_properties(guest_file_lock_${fileLockCase} PROPERTIES TIMEOUT 20 ENVIRONMENT_MODIFICATION
"PATH=path_list_prepend:$<TARGET_FILE_DIR:libkernel>")
endforeach()
endif()
add_test_executable(guest_sandbox_random_word_tests ${LIBS_INCLUDE_DIR}/tests/GuestSandboxRandomWord.cpp)
target_include_directories(guest_sandbox_random_word_tests PRIVATE ${LIBS_INCLUDE_DIR})

4
core/libs/prx/libkernel/File/include/File.hpp
View File

@ -25,4 +25,8 @@ std::int64_t NativePositioned_nid_no_patch(int descriptor, void* buf, std::size_
bool DescriptorIsOpen_nid_no_patch(int descriptor);
namespace File {
int HostStandardError();
}
#endif

7
core/libs/prx/libkernel/File/src/DescriptorControl.cpp
View File

@ -3,6 +3,7 @@
#include "prx/libkernel/File/include/File.hpp"
#include "prx/libkernel/File/include/RandomDevice.hpp"
#include "prx/libc/include/GuestArena.hpp"
#include "prx/libc/include/FileLock.hpp"
#include "prx/libc/include/general/VabiMacros.hpp"
#include <cerrno>
@ -494,12 +495,18 @@ int Ioctl(int fd, std::uint64_t request, void* argument) {
void RememberDescriptor(int fd, int openFlags) {
std::lock_guard lock(g_mutex);
g_entries[fd] = Entry{(openFlags & OpenCloseOnExec) != 0, std::make_shared<int>(openFlags & OpenStatusFlags)};
#ifdef _WIN32
GuestFileLockForget_nid_no_patch(fd);
#endif
}
void ShareDescriptor(int from, int to) {
std::lock_guard lock(g_mutex);
auto status = Materialize(from).status;
g_entries[to] = Entry{false, std::move(status)};
#ifdef _WIN32
GuestFileLockShare_nid_no_patch(from, to);
#endif
}
void ForgetDescriptor(int fd) {

63
core/libs/prx/libkernel/File/src/FileLock.cpp
View File

@ -1,73 +1,16 @@
#include "prx/libkernel/File/include/FileLock.hpp"
#include "prx/libc/include/FileLock.hpp"
#ifdef _WIN32
#include <cerrno>
#include <io.h>
#include <map>
#include <mutex>
#include <windows.h>
namespace {
constexpr int LockShared = 1;
constexpr int LockExclusive = 2;
constexpr int LockNonblocking = 4;
constexpr int LockUnlock = 8;
std::mutex g_mutex;
std::map<int, int> g_locks;
int HeldLock(int fd) {
std::lock_guard lock(g_mutex);
const auto found = g_locks.find(fd);
return found == g_locks.end() ? 0 : found->second;
}
void SetHeldLock(int fd, int mode) {
std::lock_guard lock(g_mutex);
if (mode == 0) {
g_locks.erase(fd);
} else {
g_locks[fd] = mode;
}
}
}
namespace File {
int Flock(int fd, int operation) {
const int mode = operation & (LockShared | LockExclusive | LockUnlock);
if (mode != LockShared && mode != LockExclusive && mode != LockUnlock) {
errno = EINVAL;
::SetLastError(ERROR_INVALID_PARAMETER);
return -1;
}
const auto handle = reinterpret_cast<HANDLE>(::_get_osfhandle(fd));
if (handle == INVALID_HANDLE_VALUE) {
errno = EBADF;
::SetLastError(ERROR_INVALID_HANDLE);
return -1;
}
const int held = HeldLock(fd);
if (held == mode) return 0;
if (held != 0) {
OVERLAPPED overlapped{};
if (!::UnlockFileEx(handle, 0, MAXDWORD, MAXDWORD, &overlapped)) return -1;
SetHeldLock(fd, 0);
}
if (mode == LockUnlock) return 0;
DWORD flags = mode == LockExclusive ? LOCKFILE_EXCLUSIVE_LOCK : 0;
if (operation & LockNonblocking) flags |= LOCKFILE_FAIL_IMMEDIATELY;
OVERLAPPED overlapped{};
if (!::LockFileEx(handle, flags, 0, MAXDWORD, MAXDWORD, &overlapped)) return -1;
SetHeldLock(fd, mode);
return 0;
return GuestFileLock_nid_no_patch(fd, operation);
}
void ForgetFileLock(int fd) {
SetHeldLock(fd, 0);
GuestFileLockForget_nid_no_patch(fd);
}
}

8
core/libs/prx/libkernel/File/src/Open.cpp
View File

@ -174,12 +174,14 @@ static int OpenNative(const std::filesystem::path& native, int flags, std::uint1
}
int fd = NativeOpen(native, MapFlags(flags), mode);
#ifdef _WIN32
if (fd < 0 && errno != ENOENT) {
if (fd < 0 && (errno != ENOENT || !native.has_filename())) {
auto directory = native;
while (!directory.has_filename() && directory.has_relative_path()) directory = directory.parent_path();
std::error_code error;
if (std::filesystem::is_directory(native, error)) {
if (std::filesystem::is_directory(directory, error)) {
if ((flags & (SCE_KERNEL_O_CREAT | SCE_KERNEL_O_EXCL)) == (SCE_KERNEL_O_CREAT | SCE_KERNEL_O_EXCL)) errno = EEXIST;
else if ((flags & SCE_KERNEL_O_ACCMODE) != SCE_KERNEL_O_RDONLY || (flags & (SCE_KERNEL_O_CREAT | SCE_KERNEL_O_TRUNC))) errno = EISDIR;
else fd = File::OpenDirectoryDescriptor(native);
else fd = File::OpenDirectoryDescriptor(directory);
}
}
#endif

33
core/libs/prx/libkernel/File/src/Stdio.cpp
View File

@ -18,6 +18,7 @@
#include "prx/libkernel/KernelErrors.hpp"
#include "prx/libkernel/Socket/include/SocketRuntime.hpp"
#include "prx/libkernel/Socket/include/SocketPoll.hpp"
#include <atomic>
#include <cerrno>
#include <cstring>
#include <cstdarg>
@ -513,6 +514,36 @@ int APS5_VABI _close_nid_postfix(int descriptor) {
return close_nid_postfix(descriptor);
}
}
namespace {
std::atomic<int> g_hostStandardError{-1};
void PreserveHostStandardError() {
if (g_hostStandardError.load() >= 0) return;
#ifdef _WIN32
const int saved = WithoutParameterHandler([] { return ::_dup(2); });
#else
const int saved = ::dup(2);
#endif
if (saved < 0) throw std::runtime_error("dup2: cannot keep the host standard error");
int expected = -1;
if (!g_hostStandardError.compare_exchange_strong(expected, saved)) {
#ifdef _WIN32
_close(saved);
#else
::close(saved);
#endif
}
}
}
int File::HostStandardError() {
return g_hostStandardError.load();
}
extern "C" {
int APS5_VABI dup_nid_postfix(int d) {
if (d >= GuestSockets::FirstDescriptor) return GuestSockets::Duplicate(d);
if (d < 0) return PosixFailure(GUEST_EBADF);
@ -546,6 +577,7 @@ int APS5_VABI dup2_nid_postfix(int from, int to) {
if (WithoutParameterHandler([from] { return ::_get_osfhandle(from) == -1 ? -1 : 0; }) != 0) return PosixFailure(GUEST_EBADF);
if (from == to) return to;
RejectDirectoryDuplicate(from, __func__);
if (to == 2) PreserveHostStandardError();
if (WithoutParameterHandler([from, to] { return ::_dup2(from, to); }) != 0) return PosixFailure(SceErrorFromErrno(errno) & 0xffff);
File::ForgetDirectoryDescriptor(to);
File::ForgetFileLock(to);
@ -558,6 +590,7 @@ int APS5_VABI dup2_nid_postfix(int from, int to) {
if (::fcntl(from, F_GETFL) < 0) return PosixFailure(GUEST_EBADF);
return to;
}
if (to == 2) PreserveHostStandardError();
const int result = ::dup2(from, to);
if (result < 0) return PosixFailure(SceErrorFromErrno(errno) & 0xffff);
File::ForgetRandomDevice(to);

58
core/libs/prx/libkernel/Module/src/DynamicLoader.cpp
View File

@ -3,13 +3,16 @@
#include "prx/libkernel/Module/ModuleArgs.hpp"
#include <nid/NidCompute.hpp>
#include <array>
#include <condition_variable>
#include <filesystem>
#include <cstdio>
#include <map>
#include <memory>
#include <limits>
#include <mutex>
#include <string>
#include <set>
#include <stdexcept>
#include <system_error>
#include <vector>
#ifdef _WIN32
#ifndef NOMINMAX
@ -110,21 +113,52 @@ char* APS5_VABI dlerror_nid_postfix() {
static void InitializeDeferredModule(HMODULE native) {
using Entry = int (APS5_VABI *)(std::size_t, const void*, void*);
using Initializer = void (APS5_VABI *)(int, char**, char**);
constexpr auto initializing = std::numeric_limits<std::uint32_t>::max();
static std::mutex initializedLock;
static std::set<HMODULE> initialized;
const auto* table = reinterpret_cast<const std::uint32_t*>(GetProcAddress(native, "__aps5_guest_initialize"));
static std::condition_variable initializedChanged;
auto* table = reinterpret_cast<std::uint32_t*>(GetProcAddress(native, "__aps5_guest_initialize"));
if (!table) return;
{
std::lock_guard lock(initializedLock);
if (!initialized.insert(native).second) return;
const auto owner = GetCurrentThreadId();
std::unique_lock lock(initializedLock);
while (table[1] == initializing) {
if (table[0] == 0) throw std::runtime_error("dlopen: deferred module initialization failed");
if (table[0] == owner) return;
initializedChanged.wait(lock);
}
auto* base = reinterpret_cast<std::uint8_t*>(native);
const auto* pending = static_cast<const PendingModuleArgs*>(__aps5_get_pending_module_args_nid_no_patch());
if (table[0] != 0) __aps5_set_module_init_result_nid_no_patch(reinterpret_cast<Entry>(base + table[0])(pending->args, pending->argp, nullptr));
for (std::uint32_t index = 0; index < table[1]; ++index) {
const auto initializer = *reinterpret_cast<Initializer*>(base + table[2 + index]);
if (initializer) initializer(0, nullptr, nullptr);
const auto entry = table[0];
const auto count = table[1];
if (entry == 0 && count == 0) return;
DWORD protection = 0;
if (!VirtualProtect(table, 2 * sizeof(*table), PAGE_READWRITE, &protection))
throw std::system_error(static_cast<int>(GetLastError()), std::system_category(), "open deferred module initialization metadata");
table[0] = owner;
table[1] = initializing;
lock.unlock();
const auto finish = [&](bool succeeded) {
lock.lock();
table[0] = 0;
table[1] = succeeded ? 0 : initializing;
DWORD previous = 0;
const bool restored = VirtualProtect(table, 2 * sizeof(*table), protection, &previous);
const auto error = restored ? ERROR_SUCCESS : GetLastError();
if (!restored) table[1] = initializing;
lock.unlock();
initializedChanged.notify_all();
if (!restored) throw std::system_error(static_cast<int>(error), std::system_category(), "restore deferred module initialization metadata");
};
try {
auto* base = reinterpret_cast<std::uint8_t*>(native);
const auto* pending = static_cast<const PendingModuleArgs*>(__aps5_get_pending_module_args_nid_no_patch());
if (entry != 0) __aps5_set_module_init_result_nid_no_patch(reinterpret_cast<Entry>(base + entry)(pending->args, pending->argp, nullptr));
for (std::uint32_t index = 0; index < count; ++index) {
const auto initializer = *reinterpret_cast<Initializer*>(base + table[2 + index]);
if (initializer) initializer(0, nullptr, nullptr);
}
} catch (...) {
finish(false);
throw;
}
finish(true);
}
#endif

77
core/libs/prx/libkernel/Module/src/ModuleInfo.cpp
View File

@ -205,6 +205,12 @@ std::uint64_t GuestSegmentBytes() {
dl_iterate_phdr(AddGuestSegmentBytes, &bytes);
return bytes;
}
bool SystemImage(const GuestImage& image) {
const auto* name = static_cast<const link_map*>(image.native)->l_name;
const std::string_view path = name ? name : "";
return !path.empty() && !path.ends_with(GuestModuleSuffix);
}
#else
bool SectionNamed(const IMAGE_SECTION_HEADER& section, const char* name) {
return std::strncmp(reinterpret_cast<const char*>(section.Name), name, IMAGE_SIZEOF_SHORT_NAME) == 0;
@ -341,8 +347,46 @@ std::uint64_t GuestSegmentBytes() {
}
return bytes;
}
bool SystemImage(const GuestImage& image) {
if (image.address == reinterpret_cast<std::uintptr_t>(GetModuleHandleW(nullptr))) return false;
constexpr std::wstring_view suffix = L".guest.prx";
const auto path = ModulePath(reinterpret_cast<HMODULE>(image.address));
return path.size() < suffix.size() || CompareStringOrdinal(path.data() + path.size() - suffix.size(), static_cast<int>(suffix.size()), suffix.data(), static_cast<int>(suffix.size()), TRUE) != CSTR_EQUAL;
}
#endif
int FillModuleInfo(KernelModule handle, ModuleInfo* info, bool titleOnly) {
if (!info) return SCE_KERNEL_ERROR_EFAULT;
if (info->st_size != sizeof(ModuleInfo)) return SCE_KERNEL_ERROR_EINVAL;
for (const auto& image : GuestImages()) {
if (ModuleIdForImage_nid_no_patch(image.native) != handle) continue;
if (titleOnly && SystemImage(image)) return SCE_KERNEL_ERROR_EPERM;
ModuleInfo result{};
result.st_size = sizeof(ModuleInfo);
#ifdef _WIN32
if (!Fill(image.address, result)) return SCE_KERNEL_ERROR_ESRCH;
#else
ImageSearch<ModuleInfo> search{image.address, &result, false};
dl_iterate_phdr(FindImage<ModuleInfo>, &search);
if (!search.found) return SCE_KERNEL_ERROR_ESRCH;
#endif
*info = result;
return 0;
}
return SCE_KERNEL_ERROR_ESRCH;
}
int FillModuleList(KernelModule* handles, std::size_t count, std::size_t* actual, bool titleOnly) {
if (!handles || !actual) return SCE_KERNEL_ERROR_EFAULT;
auto images = GuestImages();
if (titleOnly) std::erase_if(images, SystemImage);
for (std::size_t i = 0; i < images.size() && i < count; ++i) handles[i] = ModuleIdForImage_nid_no_patch(images[i].native);
if (images.size() > count) return SCE_KERNEL_ERROR_ENOMEM;
*actual = images.size();
return 0;
}
}
extern "C" {
@ -369,32 +413,19 @@ int APS5_VABI sceKernelGetModuleInfoFromAddr(std::uint64_t address, int flags, M
}
int APS5_VABI sceKernelGetModuleList(KernelModule* handles, std::size_t count, std::size_t* actual) {
if (!handles || !actual) return SCE_KERNEL_ERROR_EFAULT;
const auto images = GuestImages();
for (std::size_t i = 0; i < images.size() && i < count; ++i) handles[i] = ModuleIdForImage_nid_no_patch(images[i].native);
if (images.size() > count) return SCE_KERNEL_ERROR_ENOMEM;
*actual = images.size();
return 0;
return FillModuleList(handles, count, actual, false);
}
int APS5_VABI sceKernelGetModuleList2(KernelModule* handles, std::size_t count, std::size_t* actual) {
return FillModuleList(handles, count, actual, true);
}
int APS5_VABI sceKernelGetModuleInfo(KernelModule handle, ModuleInfo* info) {
if (!info) return SCE_KERNEL_ERROR_EFAULT;
if (info->st_size != sizeof(ModuleInfo)) return SCE_KERNEL_ERROR_EINVAL;
for (const auto& image : GuestImages()) {
if (ModuleIdForImage_nid_no_patch(image.native) != handle) continue;
ModuleInfo result{};
result.st_size = sizeof(ModuleInfo);
#ifdef _WIN32
if (!Fill(image.address, result)) return SCE_KERNEL_ERROR_ESRCH;
#else
ImageSearch<ModuleInfo> search{image.address, &result, false};
dl_iterate_phdr(FindImage<ModuleInfo>, &search);
if (!search.found) return SCE_KERNEL_ERROR_ESRCH;
#endif
*info = result;
return 0;
}
return SCE_KERNEL_ERROR_ESRCH;
return FillModuleInfo(handle, info, false);
}
int APS5_VABI sceKernelGetModuleInfo2(KernelModule handle, ModuleInfo* info) {
return FillModuleInfo(handle, info, true);
}
std::uint64_t GuestImageSegmentBytes_nid_no_patch() {

9
core/libs/prx/libkernel/Socket/src/Udp.cpp
View File

@ -536,6 +536,9 @@ int APS5_VABI socket_nid_postfix(int family, int type, int protocol) {
const auto native = ::socket(family == 2 ? AF_INET : AF_INET6, nativeType, protocol);
if (native == Invalid) return Fail(NativeError());
Socket guard(native, family, plainType);
const int ipv6Only = 1;
if (family == 28 && ::setsockopt(native, IPPROTO_IPV6, IPV6_V6ONLY, reinterpret_cast<const char*>(&ipv6Only), sizeof(ipv6Only)))
return Fail(NativeError());
try {
auto socket = std::make_shared<Socket>(native, family, plainType);
guard.value = Invalid;
@ -811,10 +814,10 @@ std::int64_t APS5_VABI recvmsg_nid_postfix(int descriptor, GuestMsghdr* message,
const auto socket = Lookup(descriptor);
if (!socket) return -1;
if ((flags & ~2) != 0) return Fail(45);
const bool datagram = socket->type == 2;
const bool records = socket->type == 2 || socket->type == 5;
std::vector<char> buffer;
try {
buffer.resize(static_cast<std::size_t>(total) + (datagram ? 1u : 0u));
buffer.resize(static_cast<std::size_t>(total) + (records ? 1u : 0u));
} catch (const std::bad_alloc&) {
return Fail(55);
}
@ -824,7 +827,7 @@ std::int64_t APS5_VABI recvmsg_nid_postfix(int descriptor, GuestMsghdr* message,
flags & 2 ? MSG_PEEK : 0, reinterpret_cast<sockaddr*>(&native), &size);
if (received < 0) {
#ifdef _WIN32
if (!datagram || WSAGetLastError() != WSAEMSGSIZE) return Fail(NativeError());
if (!records || WSAGetLastError() != WSAEMSGSIZE) return Fail(NativeError());
received = static_cast<std::int64_t>(buffer.size());
#else
return Fail(NativeError());

12
core/libs/prx/libkernel/System/src/Notification.cpp
View File

@ -1,4 +1,6 @@
#include "prx/libc/include/general/VabiMacros.hpp"
#include "prx/libkernel/File/include/File.hpp"
#include <string>
#include <array>
#include <cerrno>
#include <cstddef>
@ -8,6 +10,7 @@
#include <mutex>
#ifdef _WIN32
#include <windows.h>
#include <io.h>
#else
#include <unistd.h>
#endif
@ -17,6 +20,15 @@ extern "C" int APS5_VABI sceKernelDebugOutText(int channel, const char* text) {
if (!text) return static_cast<int>(0x8002000eu);
static std::mutex outputMutex;
const std::lock_guard lock(outputMutex);
if (const int host = File::HostStandardError(); host >= 0) {
const auto line = "[debug:" + std::to_string(channel) + "] " + text;
#ifdef _WIN32
const auto written = _write(host, line.data(), static_cast<unsigned>(line.size()));
#else
const auto written = ::write(host, line.data(), line.size());
#endif
return written == static_cast<decltype(written)>(line.size()) ? 0 : static_cast<int>(0x80020005u);
}
if (std::fprintf(stderr, "[debug:%d] %s", channel, text) < 0 || std::fflush(stderr) != 0)
return static_cast<int>(0x80020005u);
return 0;

8
core/libs/prx/libkernel/System/src/Process.cpp
View File

@ -338,6 +338,14 @@ int APS5_VABI sceKernelGetCurrentCpu(void) {
return GuestCpuFromHost(index, scePthreadSelf()->affinity.load(std::memory_order_relaxed));
}
int APS5_VABI sceKernelGetMainSocId(void) {
return 0;
}
int APS5_VABI sceKernelGetCpumode(void) {
return 0;
}
std::uint64_t APS5_VABI sceKernelGetGPI(void) {
return gpoBits.load(std::memory_order_relaxed);
}

4
core/libs/prx/libkernel/Time/Time.cpp
View File

@ -614,6 +614,10 @@ uint64_t APS5_VABI sceKernelGetTscFrequency(void) {
return frequency;
}
std::int64_t APS5_VABI sceKernelGetCpuFrequency(void) {
return static_cast<std::int64_t>(sceKernelGetTscFrequency());
}
unsigned int APS5_VABI sceKernelSleep(unsigned int seconds) {
CancellableSleep(static_cast<std::uint64_t>(seconds) * 1000000000ULL);
return 0;

18
core/libs/tests/FileStreams.cpp
View File

@ -21,6 +21,8 @@ int APS5_VABI fputs_nid_postfix(const char* str, FileStream* stream);
int APS5_VABI fflush_nid_postfix(FileStream* stream);
int APS5_VABI remove_nid_postfix(const char* path);
int APS5_VABI rename_nid_postfix(const char* from, const char* to);
int APS5_VABI ferror_nid_postfix(FileStream* stream);
void APS5_VABI clearerr_nid_postfix(FileStream* stream);
}
static void Require(bool condition) {
@ -114,5 +116,21 @@ int main(int argc, char** argv) {
std::array<char, 5> appended{};
Require(stream != nullptr && fread_nid_postfix(appended.data(), 1, 4, stream) == 4 && std::strcmp(appended.data(), "abcd") == 0);
Require(fclose_nid_postfix(stream) == 0 && remove_nid_postfix(heldName.c_str()) == 0);
constexpr char marker[] = "marker";
stream = fopen_nid_postfix(argv[1], "wb");
Require(stream != nullptr);
std::array<char, sizeof(marker)> markerBuffer{};
Require(fread_nid_postfix(markerBuffer.data(), 1, markerBuffer.size(), stream) == 0);
Require(ferror_nid_postfix(stream) != 0);
Require(fclose_nid_postfix(stream) == 0);
stream = fopen_nid_postfix(argv[1], "rb");
Require(stream != nullptr);
Require(fwrite_nid_postfix(marker, 1, sizeof(marker), stream) == 0);
Require(ferror_nid_postfix(stream) != 0);
clearerr_nid_postfix(stream);
Require(fputs_nid_postfix(marker, stream) == EOF);
Require(ferror_nid_postfix(stream) != 0);
Require(fclose_nid_postfix(stream) == 0);
Require(std::remove(argv[1]) == 0);
std::cout << "PASS: stream objects, file operations, EOF and error handling\n";
}

25
core/libs/tests/Formatting.cpp
View File

@ -18,9 +18,11 @@ int APS5_VABI sscanf_nid_postfix(const char*, const char*, ...);
int APS5_VABI vsnprintf_nid_postfix(char*, size_t, const char*, VaList*);
int APS5_VABI vsnprintf_s_nid_postfix(char*, size_t, const char*, VaList*);
int APS5_VABI vsscanf_s_nid_postfix(const char*, const char*, VaList*);
int APS5_VABI vsscanf_nid_postfix(const char*, const char*, VaList*);
int APS5_VABI vprintf_nid_postfix(const char*, VaList*);
int APS5_VABI asprintf_nid_postfix(char**, const char*, ...);
void APS5_VABI free_nid_postfix(void*);
int* APS5_VABI __error_nid_postfix();
}
static void Require(bool condition) {
@ -49,6 +51,16 @@ static int APS5_VABI FormatListS(char* buffer, size_t size, const char* format,
return result;
}
static int APS5_VABI ScanList(const char* buffer, const char* format, ...) {
__builtin_sysv_va_list args;
__builtin_sysv_va_start(args, format);
VaList list;
std::memcpy(&list, args, sizeof(list));
const int result = vsscanf_nid_postfix(buffer, format, &list);
__builtin_sysv_va_end(args);
return result;
}
static int APS5_VABI ScanListS(const char* buffer, const char* format, ...) {
__builtin_sysv_va_list args;
__builtin_sysv_va_start(args, format);
@ -199,6 +211,13 @@ __attribute__((noinline)) static void APS5_VABI RunChecks() {
&first, &second, &third, &fourth, &fifth, &sixth, word, &real) == 8);
Require(first == 1 && second == 2 && third == 3 && fourth == 4 && fifth == 5 && sixth == 6);
Require(std::strcmp(word, "seven") == 0 && real == 8.5);
int binary = -1, binaryEnd = -1;
char binaryRest[8] = {};
Require(sscanf_nid_postfix("0b101", "%i%n%7s", &binary, &binaryEnd, binaryRest) == 2);
Require(binary == 0 && binaryEnd == 1 && std::strcmp(binaryRest, "b101") == 0);
Require(ScanList(" -0B11", "%i", &binary) == 1 && binary == 0);
Require(ScanListS("0b1", "%i", &binary) == 1 && binary == 0);
Require(sscanf_nid_postfix("0x1f", "%i", &binary) == 1 && binary == 31);
#ifdef _WIN32
int scanned = -1;
int position = -1;
@ -235,6 +254,12 @@ __attribute__((noinline)) static void APS5_VABI RunChecks() {
Require(count == 4 && smallCount == 6 && std::strcmp(buffer, "ab") == 0);
Require(snprintf_nid_postfix(buffer, sizeof(buffer), "a%cb", 0) == 3);
Require(buffer[0] == 'a' && buffer[1] == 0 && buffer[2] == 'b' && buffer[3] == 0);
*__error_nid_postfix() = 35;
Require(snprintf_nid_postfix(buffer, sizeof(buffer), "[%m][%.8m][%-10.5m][%d]", 7) == 59);
Require(std::strcmp(buffer, "[Resource temporarily unavailable][Resource][Resou ][7]") == 0);
Require(*__error_nid_postfix() == 35);
*__error_nid_postfix() = 63;
Require(FormatList(buffer, sizeof(buffer), "%s: %m", "open") == 24 && std::strcmp(buffer, "open: File name too long") == 0);
Require(printf_nid_postfix("printf: %d %.1f\n", 7, 2.5) == 14);
Require(libc_printf_nid_postfix("libc_printf: %s\n", "OK") == 16);
Require(libc_printf_nid_postfix("%d %d %d %d %d %d %d\n", 1, 22, 333, 4444, 55555, 666666, 7777777) == 35);

24
core/libs/tests/GuestCrt.cpp
View File

@ -32,6 +32,7 @@ char* APS5_VABI strtok_s_nid_postfix(char* str, const char* delim, char** contex
int APS5_VABI __fpclassifyd_nid_postfix(double value);
unsigned __int128 APS5_VABI __udivti3_nid_postfix(unsigned __int128 dividend, unsigned __int128 divisor);
__int128 APS5_VABI __divti3_nid_postfix(__int128 dividend, __int128 divisor);
unsigned __int128 APS5_VABI __umodti3_nid_postfix(unsigned __int128 dividend, unsigned __int128 divisor);
SonyLconv* APS5_VABI localeconv_nid_postfix(void);
int APS5_VABI sceAmprFpclassifyd(double value);
unsigned int APS5_VABI _ZSt14_Random_devicev_nid_postfix();
@ -109,6 +110,29 @@ int main() {
}
Require(trapped);
const unsigned __int128 unsignedMaximum = ~(unsigned __int128)0;
const unsigned __int128 largeDivisor = ((unsigned __int128)1 << 64) + 3;
Require(__umodti3_nid_postfix(100, 3) == 1);
Require(__umodti3_nid_postfix(0, 5) == 0);
Require(__umodti3_nid_postfix(3, 100) == 3);
Require(__umodti3_nid_postfix(unsignedMaximum, 1) == 0);
Require(__umodti3_nid_postfix(unsignedMaximum, unsignedMaximum) == 0);
Require(__umodti3_nid_postfix(largeDivisor, largeDivisor) == 0);
Require(__umodti3_nid_postfix(unsignedMaximum, 10) == 5);
Require(__umodti3_nid_postfix((unsigned __int128)1 << 100, 3) == 1);
Require(__umodti3_nid_postfix(unsignedMaximum, (unsigned __int128)1 << 64) == 0xFFFFFFFFFFFFFFFFull);
Require(__umodti3_nid_postfix(unsignedMaximum, largeDivisor) == 8);
Require(__umodti3_nid_postfix((unsigned __int128)1 << 127, largeDivisor) == 0x8000000000000006ull);
Require(__umodti3_nid_postfix(unsignedMaximum - 1, unsignedMaximum) == unsignedMaximum - 1);
Require(__umodti3_nid_postfix(unsignedMaximum, (unsigned __int128)1 << 127) == ((unsigned __int128)1 << 127) - 1);
trapped = false;
try {
__umodti3_nid_postfix(((unsigned __int128)0x1234 << 64) | 0x5678, 0);
} catch (const std::runtime_error& error) {
trapped = std::strcmp(error.what(), "__umodti3: integer divide by zero (0x00000000000012340000000000005678 % 0)") == 0;
}
Require(trapped);
void* previous = sceAmprSetConstraintHandlerS(reinterpret_cast<void*>(&CountingHandler));
Require(previous != nullptr);
Require(previous != reinterpret_cast<void*>(&CountingHandler));

6
core/libs/tests/GuestCurrentCpu.cpp
View File

@ -6,6 +6,10 @@
extern "C" {
int APS5_VABI sceKernelGetCurrentCpu(void);
int APS5_VABI sceKernelGetMainSocId(void);
int APS5_VABI sceKernelGetCpumode(void);
std::int64_t APS5_VABI sceKernelGetCpuFrequency(void);
std::uint64_t APS5_VABI sceKernelGetTscFrequency(void);
KernelCpumask APS5_VABI sceKernelGetAvailableCpumask(void);
Pthread APS5_VABI scePthreadSelf();
int APS5_VABI scePthreadCreate(Pthread* thread, const PthreadAttr* attr, PthreadEntry entry, void* arg, const char* name);
@ -38,6 +42,8 @@ static void* APS5_VABI ReportWhilePinned(void* arg) {
int main() {
const KernelCpumask available = sceKernelGetAvailableCpumask();
Require(sceKernelGetMainSocId() == 0 && sceKernelGetCpumode() == 0);
Require(sceKernelGetCpuFrequency() > 0 && static_cast<std::uint64_t>(sceKernelGetCpuFrequency()) == sceKernelGetTscFrequency());
const std::array<KernelCpumask, 8> affinities{available, 0x1FFB, 0x3, 0x3F, 0x1000, 0x1, 0xFFFF, 0x10000};
for (const KernelCpumask affinity : affinities) {

142
core/libs/tests/GuestDeferredModule.cpp Normal file
View File

@ -0,0 +1,142 @@
#define WIN32_LEAN_AND_MEAN
#include "GuestDeferredModule.hpp"
#include <chrono>
#include <cstdio>
#include <cstdlib>
#include <filesystem>
#include <future>
extern "C" {
void* APS5_VABI dlopen_nid_postfix(const char*, int);
void* APS5_VABI dlsym_nid_postfix(void*, const char*);
int APS5_VABI dlclose_nid_postfix(void*);
char* APS5_VABI dlerror_nid_postfix();
int APS5_VABI sceKernelLoadStartModule(const char*, std::size_t, const void*, unsigned, const void*, int*);
int APS5_VABI sceKernelStopUnloadModule(int, std::size_t, const void*, unsigned, const void*, int*);
}
static void Require(bool value, const char* what) {
if (!value) {
std::fprintf(stderr, "%s\n", what);
std::exit(1);
}
}
static void* Handle(int id) { return reinterpret_cast<void*>(static_cast<std::uintptr_t>(id)); }
static int Open(const std::string& path, const GuestDeferredModuleArgs* args = nullptr, bool initialized = true) {
int result = -1;
const int id = sceKernelLoadStartModule(path.c_str(), args ? sizeof(*args) : 0, args, 0, nullptr, &result);
Require(id >= 0, "load deferred module");
Require(result == (initialized ? GuestDeferredModuleResult : 0), "module start result");
return id;
}
static void Close(int id) { Require(sceKernelStopUnloadModule(id, 0, nullptr, 0, nullptr, nullptr) == 0, "close module"); }
static void Counts(int id, int expectedStarts, int expectedConstructors) {
using Count = int (APS5_VABI*)();
const auto starts = reinterpret_cast<Count>(dlsym_nid_postfix(Handle(id), "DeferredStarts_nid_no_patch"));
const auto constructors = reinterpret_cast<Count>(dlsym_nid_postfix(Handle(id), "DeferredConstructors_nid_no_patch"));
Require(starts && constructors, "find fixture counters");
if (starts() != expectedStarts || constructors() != expectedConstructors) {
std::fprintf(stderr, "starts=%d constructors=%d, expected=%d/%d\n", starts(), constructors(), expectedStarts, expectedConstructors);
std::exit(1);
}
}
static HMODULE Native(int id) {
const auto symbol = dlsym_nid_postfix(Handle(id), "DeferredStarts_nid_no_patch");
HMODULE native = nullptr;
Require(GetModuleHandleExW(GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS | GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNT,
reinterpret_cast<LPCWSTR>(symbol), &native), "native image handle");
return native;
}
static void MetadataProtected(int id) {
const auto table = GetProcAddress(Native(id), "__aps5_guest_initialize");
MEMORY_BASIC_INFORMATION information{};
Require(table && VirtualQuery(reinterpret_cast<const void*>(table), &information, sizeof(information)), "query init metadata");
Require(information.Protect == PAGE_READONLY, "init metadata protection restored");
}
static void APS5_VABI Reenter(const char* path) {
void* nested = dlopen_nid_postfix(path, 2);
Require(nested != nullptr, "recursive module load");
Require(dlclose_nid_postfix(nested) == 0, "recursive module close");
}
int main(int argc, char** argv) {
Require(argc == 3, "fixture and mode required");
const auto file = std::filesystem::absolute(argv[1]);
const auto path = file.string();
const std::string mode = argv[2];
if (mode == "reload") {
HMODULE firstBase = nullptr;
for (unsigned cycle = 0; cycle < 3; ++cycle) {
const int first = Open(path);
Counts(first, 1, 1);
const auto native = Native(first);
if (cycle == 0) firstBase = native;
Require(native == firstBase, "reload reuses the fixture base");
MetadataProtected(first);
const int second = Open(path, nullptr, false);
Counts(second, 1, 1);
Close(first);
Require(GetModuleHandleW(file.c_str()) != nullptr, "second reference retains the module");
Counts(second, 1, 1);
Close(second);
Require(GetModuleHandleW(file.c_str()) == nullptr, "last close unloads the module");
}
} else if (mode == "pinned") {
HMODULE pin = LoadLibraryW(file.c_str());
Require(pin != nullptr, "native pin");
const int first = Open(path);
Counts(first, 1, 1);
Close(first);
const int second = Open(path, nullptr, false);
Counts(second, 1, 1);
MetadataProtected(second);
Close(second);
Require(FreeLibrary(pin) && GetModuleHandleW(file.c_str()) == nullptr, "release native pin");
const int reloaded = Open(path);
Counts(reloaded, 1, 1);
Close(reloaded);
} else if (mode == "parallel") {
GuestDeferredModuleArgs args;
args.Mode = 1;
args.Entered = CreateEventW(nullptr, TRUE, FALSE, nullptr);
args.Release = CreateEventW(nullptr, TRUE, FALSE, nullptr);
Require(args.Entered && args.Release, "create initializer gates");
auto first = std::async(std::launch::async, [&] { return Open(path, &args); });
Require(WaitForSingleObject(args.Entered, 3000) == WAIT_OBJECT_0, "first initializer entered");
std::promise<void> attempting;
auto started = attempting.get_future();
auto second = std::async(std::launch::async, [&] {
attempting.set_value();
return Open(path, nullptr, false);
});
started.wait();
const bool returnedEarly = second.wait_for(std::chrono::milliseconds(200)) == std::future_status::ready;
Require(SetEvent(args.Release), "release initializer");
const int one = first.get();
const int two = second.get();
Require(!returnedEarly, "concurrent load waits for initialization");
Counts(one, 1, 1);
Counts(two, 1, 1);
MetadataProtected(one);
Close(one);
Close(two);
CloseHandle(args.Entered);
CloseHandle(args.Release);
} else if (mode == "reentrant") {
GuestDeferredModuleArgs args;
args.Mode = 2;
args.Reenter = Reenter;
args.Path = path.c_str();
const int id = Open(path, &args);
Counts(id, 1, 1);
MetadataProtected(id);
Close(id);
} else Require(false, "unknown fixture mode");
}

18
core/libs/tests/GuestDeferredModule.hpp Normal file
View File

@ -0,0 +1,18 @@
#ifndef CORE_LIBS_TESTS_GUESTDEFERREDMODULE_HPP
#define CORE_LIBS_TESTS_GUESTDEFERREDMODULE_HPP
#include "prx/libc/include/general/VabiMacros.hpp"
#include <windows.h>
#include <cstdint>
struct GuestDeferredModuleArgs {
unsigned Mode = 0;
HANDLE Entered = nullptr;
HANDLE Release = nullptr;
void (APS5_VABI* Reenter)(const char*) = nullptr;
const char* Path = nullptr;
};
inline constexpr int GuestDeferredModuleResult = 0x5a17;
#endif

7
core/libs/tests/GuestDeferredModuleFixture.S Normal file
View File

@ -0,0 +1,7 @@
.section .ginit,"dr"
.p2align 3
.globl __aps5_guest_initialize
__aps5_guest_initialize:
.rva DeferredStart_nid_no_patch
.long 1
.rva DeferredInitializerSlot_nid_no_patch

32
core/libs/tests/GuestDeferredModuleFixture.cpp Normal file
View File

@ -0,0 +1,32 @@
#include "GuestDeferredModule.hpp"
#include <atomic>
#include <cstddef>
#include <cstdlib>
static std::atomic<int> starts{0};
static std::atomic<int> constructors{0};
extern "C" int APS5_VABI DeferredStart_nid_no_patch(std::size_t bytes, const void* argument, void*) {
++starts;
if (bytes != 0) {
if (bytes != sizeof(GuestDeferredModuleArgs) || !argument) std::abort();
const auto& args = *static_cast<const GuestDeferredModuleArgs*>(argument);
if (args.Mode == 1) {
if (!SetEvent(args.Entered) || WaitForSingleObject(args.Release, 10000) != WAIT_OBJECT_0)
std::abort();
} else if (args.Mode == 2) args.Reenter(args.Path);
}
return GuestDeferredModuleResult;
}
extern "C" void APS5_VABI DeferredConstructor_nid_no_patch(int, char**, char**) {
++constructors;
}
using Initializer = void (APS5_VABI*)(int, char**, char**);
extern "C" {
Initializer DeferredInitializerSlot_nid_no_patch = DeferredConstructor_nid_no_patch;
}
extern "C" int APS5_VABI DeferredStarts_nid_no_patch() { return starts.load(); }
extern "C" int APS5_VABI DeferredConstructors_nid_no_patch() { return constructors.load(); }

306
core/libs/tests/GuestFileLock.cpp Normal file
View File

@ -0,0 +1,306 @@
#include "prx/libc/include/general/VabiMacros.hpp"
#include <barrier>
#include <chrono>
#include <cstdio>
#include <cstdlib>
#include <filesystem>
#include <future>
#include <set>
#include <string>
#include <thread>
#include <windows.h>
class FileStream;
extern "C" {
int APS5_VABI open_nid_postfix(const char*, int, int);
int APS5_VABI close_nid_postfix(int);
int APS5_VABI sceKernelClose(int);
int APS5_VABI dup_nid_postfix(int);
int APS5_VABI dup2_nid_postfix(int, int);
int APS5_VABI fcntl_nid_postfix(int, int, ...);
int APS5_VABI flock_nid_postfix(int, int);
int* APS5_VABI __error_nid_postfix();
FileStream* APS5_VABI fopen_nid_postfix(const char*, const char*);
FileStream* APS5_VABI fdopen_nid_postfix(int, const char*);
FileStream* APS5_VABI freopen_nid_postfix(const char*, const char*, FileStream*);
int APS5_VABI fclose_nid_postfix(FileStream*);
int APS5_VABI fileno_nid_postfix(FileStream*);
}
static void Require(bool condition) {
if (!condition) std::abort();
}
static int Duplicate(int source, int kind) {
if (kind == 0) return dup_nid_postfix(source);
if (kind == 1) return dup2_nid_postfix(source, 20);
return fcntl_nid_postfix(source, 0, 20);
}
static bool Acquires(int descriptor, int mode) {
const int result = flock_nid_postfix(descriptor, mode | 4);
if (result == 0) {
Require(flock_nid_postfix(descriptor, 8) == 0);
return true;
}
Require(result == -1 && *__error_nid_postfix() == 35);
return false;
}
struct Fixture {
static inline unsigned serial = 0;
std::string path = (std::filesystem::temp_directory_path() / ("anyps5-flock-" + std::to_string(::GetCurrentProcessId()) + "-" + std::to_string(serial++) + ".bin")).string();
std::set<int> descriptors;
int source = Open(true);
int other = Open(false);
int Open(bool create) {
const int descriptor = open_nid_postfix(path.c_str(), create ? 0x602 : 2, 0644);
Require(descriptor >= 0);
descriptors.insert(descriptor);
return descriptor;
}
int Copy(int kind, int from = -1) {
const int descriptor = Duplicate(from < 0 ? source : from, kind);
Require(descriptor >= 0);
descriptors.insert(descriptor);
return descriptor;
}
void Close(int descriptor, bool kernel = false) {
Require((kernel ? sceKernelClose(descriptor) : close_nid_postfix(descriptor)) == 0);
descriptors.erase(descriptor);
}
~Fixture() {
for (const int descriptor : descriptors) Require(close_nid_postfix(descriptor) == 0);
Require(std::remove(path.c_str()) == 0);
}
};
static void Duplicates() {
for (int kind = 0; kind != 3; ++kind) {
for (bool before : {false, true}) {
for (int mode : {1, 2}) {
Fixture fixture;
int copy = before ? fixture.Copy(kind) : -1;
Require(flock_nid_postfix(fixture.source, mode | 4) == 0);
if (!before) copy = fixture.Copy(kind);
Require(flock_nid_postfix(copy, mode | 4) == 0);
Require(flock_nid_postfix(fixture.source, mode | 4) == 0);
Require(!Acquires(fixture.other, 2));
Require(flock_nid_postfix(copy, 8) == 0);
Require(Acquires(fixture.other, 2));
Require(flock_nid_postfix(fixture.source, 8) == 0);
}
}
}
}
static void Lifetime() {
for (int kind = 0; kind != 3; ++kind) {
for (bool kernel : {false, true}) {
Fixture fixture;
const int copy = fixture.Copy(kind);
Require(flock_nid_postfix(fixture.source, 2 | 4) == 0);
const int chain = fixture.Copy(0, copy);
fixture.Close(fixture.source, kernel);
fixture.Close(copy, !kernel);
Require(!Acquires(fixture.other, 2));
Require(flock_nid_postfix(chain, 8) == 0);
Require(Acquires(fixture.other, 2));
Require(flock_nid_postfix(chain, 2 | 4) == 0);
fixture.Close(chain, kernel);
Require(Acquires(fixture.other, 2));
}
}
{
Fixture fixture;
const int copy = fixture.Copy(0);
Require(flock_nid_postfix(fixture.source, 2 | 4) == 0);
Require(dup2_nid_postfix(copy, fixture.source) == fixture.source);
Require(dup2_nid_postfix(fixture.source, fixture.source) == fixture.source);
Require(!Acquires(fixture.other, 2));
Require(flock_nid_postfix(fixture.source, 8) == 0);
Require(Acquires(fixture.other, 2));
}
{
Fixture first;
Fixture second;
const int copy = second.Copy(0);
Require(flock_nid_postfix(second.source, 2 | 4) == 0);
Require(dup2_nid_postfix(first.source, second.source) == second.source);
Require(!Acquires(second.other, 2));
Require(flock_nid_postfix(copy, 8) == 0);
Require(Acquires(second.other, 2));
Require(flock_nid_postfix(second.source, 2 | 4) == 0);
Require(!Acquires(first.other, 2));
Require(flock_nid_postfix(first.source, 8) == 0);
Require(Acquires(first.other, 2));
}
{
Fixture fixture;
const int previous = fixture.source;
Require(flock_nid_postfix(previous, 2 | 4) == 0);
fixture.Close(previous);
const int recycled = fixture.Open(false);
Require(recycled == previous && Acquires(recycled, 2));
Require(Acquires(fixture.other, 2));
}
}
static void Conversions() {
for (int kind = 0; kind != 3; ++kind) {
Fixture fixture;
const int copy = fixture.Copy(kind);
for (int repeat = 0; repeat != 20; ++repeat) {
Require(flock_nid_postfix(fixture.source, 2 | 4) == 0);
Require(flock_nid_postfix(copy, 1 | 4) == 0);
Require(Acquires(fixture.other, 1) && !Acquires(fixture.other, 2));
Require(flock_nid_postfix(fixture.source, 2 | 4) == 0);
Require(!Acquires(fixture.other, 1));
Require(flock_nid_postfix(copy, 8) == 0 && Acquires(fixture.other, 2));
}
Require(flock_nid_postfix(fixture.source, 1 | 4) == 0);
Require(flock_nid_postfix(fixture.other, 1 | 4) == 0);
Require(flock_nid_postfix(copy, 2 | 4) == -1 && *__error_nid_postfix() == 35);
Require(flock_nid_postfix(fixture.other, 8) == 0);
Require(Acquires(fixture.other, 2));
}
}
static void Threads() {
for (int kind = 0; kind != 3; ++kind) {
Fixture fixture;
Require(flock_nid_postfix(fixture.source, 2 | 4) == 0);
const int copy = fixture.Copy(kind);
auto repeated = std::async(std::launch::async, [&] { return flock_nid_postfix(copy, 2); });
Require(repeated.wait_for(std::chrono::seconds(2)) == std::future_status::ready);
Require(repeated.get() == 0 && flock_nid_postfix(copy, 8) == 0);
}
for (bool duplicate : {false, true}) {
Fixture fixture;
const int copy = duplicate ? fixture.Copy(0) : fixture.source;
std::barrier gate(3);
int results[2] = {-2, -2};
std::thread first([&] { gate.arrive_and_wait(); results[0] = flock_nid_postfix(fixture.source, 1 | 4); });
std::thread second([&] { gate.arrive_and_wait(); results[1] = flock_nid_postfix(copy, 1 | 4); });
gate.arrive_and_wait();
first.join();
second.join();
Require(results[0] == 0 && results[1] == 0);
Require(flock_nid_postfix(copy, 8) == 0 && Acquires(fixture.other, 2));
}
Fixture first;
Fixture unrelated;
Require(flock_nid_postfix(first.source, 2 | 4) == 0);
std::promise<void> entered;
auto entry = entered.get_future();
auto waiting = std::async(std::launch::async, [&] {
entered.set_value();
return flock_nid_postfix(first.other, 2);
});
entry.wait();
Require(waiting.wait_for(std::chrono::milliseconds(200)) == std::future_status::timeout);
Require(Acquires(unrelated.source, 2));
Require(flock_nid_postfix(first.source, 8) == 0);
Require(waiting.wait_for(std::chrono::seconds(2)) == std::future_status::ready && waiting.get() == 0);
Require(flock_nid_postfix(first.other, 8) == 0);
}
static void Streams() {
Fixture first;
Fixture second;
auto* stream = fopen_nid_postfix(first.path.c_str(), "r+b");
Require(stream != nullptr);
const int descriptor = fileno_nid_postfix(stream);
Require(flock_nid_postfix(descriptor, 2 | 4) == 0);
Require(fclose_nid_postfix(stream) == 0 && Acquires(first.other, 2));
for (const auto* fixture : {&second, &first}) {
stream = fopen_nid_postfix(fixture->path.c_str(), "r+b");
Require(stream != nullptr && fileno_nid_postfix(stream) == descriptor);
Require(flock_nid_postfix(descriptor, 2 | 4) == 0 && !Acquires(fixture->other, 2));
Require(fclose_nid_postfix(stream) == 0);
}
const int recycled = first.Open(false);
Require(recycled == descriptor && flock_nid_postfix(recycled, 2 | 4) == 0);
Require(!Acquires(first.other, 2));
const int copy = first.Copy(0, recycled);
stream = fdopen_nid_postfix(recycled, "r+");
Require(stream != nullptr && !Acquires(first.other, 2));
Require(fclose_nid_postfix(stream) == 0);
first.descriptors.erase(recycled);
Require(!Acquires(first.other, 2));
Require(flock_nid_postfix(copy, 8) == 0 && Acquires(first.other, 2));
stream = fopen_nid_postfix(first.path.c_str(), "r+b");
Require(stream != nullptr);
const int reopened = fileno_nid_postfix(stream);
Require(flock_nid_postfix(reopened, 2 | 4) == 0);
const int oldCopy = first.Copy(0, reopened);
Require(freopen_nid_postfix(second.path.c_str(), "r+b", stream) == stream);
Require(!Acquires(first.other, 2));
Require(flock_nid_postfix(oldCopy, 8) == 0 && Acquires(first.other, 2));
Require(flock_nid_postfix(fileno_nid_postfix(stream), 2 | 4) == 0 && !Acquires(second.other, 2));
const auto missing = first.path + ".missing";
Require(freopen_nid_postfix(missing.c_str(), "r+b", stream) == nullptr);
Require(Acquires(second.other, 2));
stream = fopen_nid_postfix(second.path.c_str(), "r+b");
Require(stream != nullptr);
Require(flock_nid_postfix(fileno_nid_postfix(stream), 2 | 4) == 0 && !Acquires(second.other, 2));
Require(fclose_nid_postfix(stream) == 0);
}
static int ChildTry(const std::string& path, int mode) {
wchar_t executable[32768];
Require(::GetModuleFileNameW(nullptr, executable, 32768) != 0);
std::wstring command = L"\"" + std::wstring(executable) + L"\" child \"" + std::filesystem::path(path).wstring() + L"\" " + std::to_wstring(mode | 4);
STARTUPINFOW startup{};
startup.cb = sizeof(startup);
PROCESS_INFORMATION process{};
Require(::CreateProcessW(nullptr, command.data(), nullptr, nullptr, FALSE, CREATE_NO_WINDOW, nullptr, nullptr, &startup, &process) != 0);
::CloseHandle(process.hThread);
if (::WaitForSingleObject(process.hProcess, 3000) != WAIT_OBJECT_0) {
::TerminateProcess(process.hProcess, 99);
std::abort();
}
DWORD result = 99;
Require(::GetExitCodeProcess(process.hProcess, &result) != 0);
::CloseHandle(process.hProcess);
return static_cast<int>(result);
}
static void Processes() {
for (int kind = 0; kind != 3; ++kind) {
for (int mode : {1, 2}) {
Fixture fixture;
Require(flock_nid_postfix(fixture.source, mode | 4) == 0);
const int copy = fixture.Copy(kind);
Require(ChildTry(fixture.path, 2) == 35);
Require(ChildTry(fixture.path, 1) == (mode == 1 ? 0 : 35));
fixture.Close(fixture.source);
Require(ChildTry(fixture.path, 2) == 35);
Require(flock_nid_postfix(copy, 8) == 0);
Require(ChildTry(fixture.path, 2) == 0);
}
}
}
int main(int argc, char** argv) {
if (argc == 4 && std::string(argv[1]) == "child") {
const int descriptor = open_nid_postfix(argv[2], 2, 0);
Require(descriptor >= 0);
const int result = flock_nid_postfix(descriptor, std::atoi(argv[3]));
const int error = result == 0 ? 0 : *__error_nid_postfix();
if (result == 0) Require(flock_nid_postfix(descriptor, 8) == 0);
Require(close_nid_postfix(descriptor) == 0);
return error;
}
Require(argc == 2);
const std::string mode = argv[1];
if (mode == "duplicate") Duplicates();
else if (mode == "lifetime") Lifetime();
else if (mode == "conversion") Conversions();
else if (mode == "threads") Threads();
else if (mode == "streams") Streams();
else if (mode == "process") Processes();
else std::abort();
}

36
core/libs/tests/GuestFilesystem.cpp
View File

@ -12,6 +12,8 @@
#include <cstdio>
#include <cstring>
#include <stdexcept>
#include <string>
#include <vector>
#ifdef _WIN32
#include <io.h>
#include <windows.h>
@ -48,6 +50,7 @@ int APS5_VABI rmdir_nid_postfix(const char*);
int APS5_VABI mkdir_nid_postfix(const char*, unsigned short);
int APS5_VABI sceKernelOpen(const char*, int, unsigned short);
int APS5_VABI sceKernelClose(int);
int APS5_VABI sceKernelGetdents(int, char*, int);
int APS5_VABI sceKernelStat(const char*, FileStat*);
int APS5_VABI sceKernelCheckReachability(const char*);
int APS5_VABI sceKernelFstat(int, FileStat*);
@ -64,6 +67,7 @@ FileStream* APS5_VABI fopen_nid_postfix(const char*, const char*);
FileStream* APS5_VABI freopen_nid_postfix(const char*, const char*, FileStream*);
int* APS5_VABI __error_nid_postfix();
int APS5_VABI pipe_nid_postfix(int*);
int APS5_VABI dup2_nid_postfix(int, int);
std::int64_t APS5_VABI read_nid_postfix(int, void*, std::size_t);
std::int64_t APS5_VABI write_nid_postfix(int, const void*, std::size_t);
int APS5_VABI sceKernelDebugOutText(int, const char*);
@ -223,6 +227,29 @@ int main() {
Require(std::filesystem::remove("creat_probe.bin"));
}
Require(close_nid_postfix(descriptors[0]) == 0);
auto* hostCapture = std::tmpfile();
Require(hostCapture != nullptr && pipe_nid_postfix(descriptors) == 0);
#ifdef _WIN32
const int hostError = ::_dup(2);
Require(hostError >= 0 && ::_dup2(::_fileno(hostCapture), 2) == 0);
#else
const int hostError = ::dup(2);
Require(hostError >= 0 && ::dup2(::fileno(hostCapture), 2) == 2);
#endif
Require(dup2_nid_postfix(descriptors[1], 2) == 2);
Require(sceKernelDebugOutText(1, "kept\n") == 0);
Require(write_nid_postfix(2, "P", 1) == 1);
#ifdef _WIN32
Require(::_dup2(hostError, 2) == 0 && ::_close(hostError) == 0);
#else
Require(::dup2(hostError, 2) == 2 && ::close(hostError) == 0);
#endif
Require(close_nid_postfix(descriptors[1]) == 0);
Require(read_nid_postfix(descriptors[0], received, sizeof(received)) == 1 && received[0] == 'P');
Require(close_nid_postfix(descriptors[0]) == 0);
std::rewind(hostCapture);
Require(std::fread(debugText, 1, sizeof(debugText), hostCapture) == 15 && std::memcmp(debugText, "[debug:1] kept\n", 15) == 0);
Require(std::fclose(hostCapture) == 0);
const auto root = std::filesystem::path("anyps5-filesystem-test-" +
std::to_string(std::chrono::steady_clock::now().time_since_epoch().count()));
Require(mkdir_nid_postfix(root.string().c_str(), 0700) == 0);
@ -578,6 +605,15 @@ int main() {
Require(directoryDescriptor >= 0 && close_nid_postfix(directoryDescriptor) == 0);
const int flaggedDirectory = sceKernelOpen(rootName.c_str(), 0x20000, 0);
Require(flaggedDirectory >= 0 && sceKernelClose(flaggedDirectory) == 0);
std::vector<char> entries(4096);
for (const auto& directoryName : {std::string("/"), std::string("."), rootName + "/", rootName + "/.."}) {
const int listed = sceKernelOpen(directoryName.c_str(), 0x20000, 0);
Require(listed >= 0);
int read = sceKernelGetdents(listed, entries.data(), static_cast<int>(entries.size()));
Require(read > 0);
while (read > 0) read = sceKernelGetdents(listed, entries.data(), static_cast<int>(entries.size()));
Require(read == 0 && sceKernelClose(listed) == 0);
}
Require(open_nid_postfix(presentName.c_str(), 0x20000, 0) == -1 && *__error_nid_postfix() == 20);
Require(sceKernelOpen(presentName.c_str(), 0x20000, 0) == static_cast<int>(0x80020014u));
Require(sceKernelOpen(presentName.c_str(), 0x20402, 0) == static_cast<int>(0x80020014u));

32
core/libs/tests/GuestModuleInfo.cpp
View File

@ -16,6 +16,8 @@ int APS5_VABI dlclose_nid_postfix(void*);
int APS5_VABI sceKernelGetModuleInfoFromAddr(std::uint64_t, int, ModuleInfoEx*);
int APS5_VABI sceKernelGetModuleList(KernelModule*, std::size_t, std::size_t*);
int APS5_VABI sceKernelGetModuleInfo(KernelModule, ModuleInfo*);
int APS5_VABI sceKernelGetModuleList2(KernelModule*, std::size_t, std::size_t*);
int APS5_VABI sceKernelGetModuleInfo2(KernelModule, ModuleInfo*);
}
static void Require(bool value) { if (!value) std::abort(); }
template<typename TFunction>
@ -105,6 +107,36 @@ int main(int argc, char** argv) {
unknown.st_size = sizeof(ModuleInfo);
Require(sceKernelGetModuleInfo(info.id, &unknown) == SCE_KERNEL_ERROR_ESRCH);
Require(sceKernelGetModuleInfo(-1, &unknown) == SCE_KERNEL_ERROR_ESRCH && unknown.name[0] == '\0');
std::vector<KernelModule> titleHandles(handles.size() + 1, -1);
std::size_t titleCount = 0;
Require(sceKernelGetModuleList2(titleHandles.data(), titleHandles.size(), &titleCount) == 0);
Require(titleCount >= 1 && titleCount < count && titleHandles[titleCount] == -1);
titleHandles.resize(titleCount);
Require(titleHandles.front() == self.id);
for (const auto handle : titleHandles) {
Require(std::count(handles.begin(), handles.end(), handle) == 1);
ModuleInfo title{}, listed{};
title.st_size = listed.st_size = sizeof(ModuleInfo);
Require(sceKernelGetModuleInfo2(handle, &title) == 0 && sceKernelGetModuleInfo(handle, &listed) == 0);
Require(std::string(title.name) == listed.name && title.segment_count == listed.segment_count);
}
std::size_t titleUntouched = 99;
Require(sceKernelGetModuleList2(titleHandles.data(), titleCount - 1, &titleUntouched) == SCE_KERNEL_ERROR_ENOMEM && titleUntouched == 99);
Require(sceKernelGetModuleList2(nullptr, 0, &titleUntouched) == SCE_KERNEL_ERROR_EFAULT);
Require(sceKernelGetModuleList2(titleHandles.data(), titleHandles.size(), nullptr) == SCE_KERNEL_ERROR_EFAULT);
for (const auto handle : handles) {
if (std::count(titleHandles.begin(), titleHandles.end(), handle) != 0) continue;
ModuleInfo system{};
system.st_size = sizeof(ModuleInfo);
system.name[0] = 'x';
Require(sceKernelGetModuleInfo2(handle, &system) == SCE_KERNEL_ERROR_EPERM && system.name[0] == 'x');
Require(sceKernelGetModuleInfo2(handle, nullptr) == SCE_KERNEL_ERROR_EFAULT);
system.st_size = 0x158;
Require(sceKernelGetModuleInfo2(handle, &system) == SCE_KERNEL_ERROR_EINVAL);
}
ModuleInfo absent{};
absent.st_size = sizeof(ModuleInfo);
Require(sceKernelGetModuleInfo2(-1, &absent) == SCE_KERNEL_ERROR_ESRCH && absent.name[0] == 0);
int local = 0;
Query(&local, SCE_KERNEL_ERROR_ESRCH);
Require(sceKernelGetModuleInfoFromAddr(reinterpret_cast<std::uintptr_t>(add), 2, nullptr) == SCE_KERNEL_ERROR_EFAULT);

36
core/libs/tests/GuestNewHandler.cpp
View File

@ -4,7 +4,10 @@
#include <cstddef>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <limits>
#include <new>
#include <stdexcept>
extern "C" {
using Handler = void (APS5_VABI*)();
@ -14,6 +17,7 @@ void* APS5_VABI _Znwm_nid_postfix(std::size_t);
void* APS5_VABI _Znam_nid_postfix(std::size_t);
void* APS5_VABI _ZnwmRKSt9nothrow_t_nid_postfix(std::size_t, const void*) noexcept;
void* APS5_VABI _ZnamRKSt9nothrow_t_nid_postfix(std::size_t, const void*) noexcept;
void APS5_VABI _ZdlPv_nid_postfix(void*);
}
namespace {
@ -85,7 +89,37 @@ bool Throws(TAction action) {
}
int main() {
int RunDefaultHeap() {
std::array<void*, 10> none{};
ApplicationHeapRegister_nid_no_patch(none.data());
constexpr std::size_t huge = std::numeric_limits<std::size_t>::max();
REQUIRE(Throws([] { _Znwm_nid_postfix(huge); }));
REQUIRE(Throws([] { _Znam_nid_postfix(huge); }));
REQUIRE(_ZnwmRKSt9nothrow_t_nid_postfix(huge, nullptr) == nullptr);
REQUIRE(_ZnamRKSt9nothrow_t_nid_postfix(huge, nullptr) == nullptr);
handlerCalls = 0;
REQUIRE(_ZSt15set_new_handlerPFvvE_nid_postfix(&uninstallingHandler) == nullptr);
REQUIRE(Throws([] { _Znwm_nid_postfix(huge); }));
REQUIRE(handlerCalls == 3 && _ZSt15get_new_handlerv_nid_postfix() == nullptr);
handlerCalls = 0;
REQUIRE(_ZSt15set_new_handlerPFvvE_nid_postfix(&throwingHandler) == nullptr);
REQUIRE(_ZnwmRKSt9nothrow_t_nid_postfix(huge, nullptr) == nullptr && handlerCalls == 1);
REQUIRE(_ZSt15set_new_handlerPFvvE_nid_postfix(nullptr) == &throwingHandler);
void* small = _Znwm_nid_postfix(24);
REQUIRE(small != nullptr);
std::memset(small, 0x5a, 24);
_ZdlPv_nid_postfix(small);
std::puts("Guest new handler default heap checks passed");
return 0;
}
int main(int argc, char** argv) {
if (argc > 1 && std::strcmp(argv[1], "default") == 0) return RunDefaultHeap();
std::array<void*, 10> api{};
api[0] = reinterpret_cast<void*>(&allocate);
api[1] = reinterpret_cast<void*>(&release);

38
core/libs/tests/GuestOverflowError.cpp Normal file
View File

@ -0,0 +1,38 @@
#include "prx/libc/include/general/VabiMacros.hpp"
#include <cstddef>
#include <cstdlib>
#include <cstring>
struct TypeRecord { const void* const* vtable; const char* name; const TypeRecord* base; };
struct Object { const void* vtable; const char* message; };
struct Vtable { std::ptrdiff_t offset; const TypeRecord* type; const void* destroy; const void* deleteObject; const void* what; };
extern "C" {
extern const TypeRecord _ZTISt13runtime_error_nid_postfix;
extern const TypeRecord _ZTISt14overflow_error_nid_postfix;
extern const Vtable _ZTVSt13runtime_error_nid_postfix;
extern const Vtable _ZTVSt14overflow_error_nid_postfix;
void APS5_VABI _ZNSt13runtime_errorC1EPKc_nid_postfix(Object*, const char*);
void APS5_VABI _ZNSt14overflow_errorD1Ev_nid_postfix(Object*);
}
static void Require(bool value) { if (!value) std::abort(); }
int main() {
const auto& type = _ZTISt14overflow_error_nid_postfix;
const auto& runtimeType = _ZTISt13runtime_error_nid_postfix;
Require(std::strcmp(type.name, "St14overflow_error") == 0);
Require(type.base == &runtimeType && type.vtable == runtimeType.vtable);
const auto& table = _ZTVSt14overflow_error_nid_postfix;
const auto& runtimeTable = _ZTVSt13runtime_error_nid_postfix;
Require(table.offset == 0 && table.type == &type);
Require(table.destroy == runtimeTable.destroy && table.deleteObject == runtimeTable.deleteObject && table.what == runtimeTable.what);
Object object{};
_ZNSt13runtime_errorC1EPKc_nid_postfix(&object, "value too large");
Require(object.message != nullptr && std::strcmp(object.message, "value too large") == 0);
_ZNSt14overflow_errorD1Ev_nid_postfix(&object);
Require(object.message == nullptr);
_ZNSt14overflow_errorD1Ev_nid_postfix(&object);
Require(object.message == nullptr);
return 0;
}

84
core/libs/tests/GuestQsort.cpp Normal file
View File

@ -0,0 +1,84 @@
#include "prx/libc/include/general/VabiMacros.hpp"
#include <cstddef>
#include <cstdio>
#include <cstring>
#include <vector>
extern "C" {
void APS5_VABI qsort_nid_postfix(void*, std::size_t, std::size_t, int (APS5_VABI*)(const void*, const void*));
}
namespace {
struct Element {
int key;
int tag;
};
int APS5_VABI CompareKeys(const void* left, const void* right) {
const int first = static_cast<const Element*>(left)->key;
const int second = static_cast<const Element*>(right)->key;
return (first > second) - (first < second);
}
int APS5_VABI CompareFirstByte(const void* left, const void* right) {
const int first = *static_cast<const unsigned char*>(left);
const int second = *static_cast<const unsigned char*>(right);
return (first > second) - (first < second);
}
bool ExpectOrder(const char* name, const std::vector<int>& keys, const std::vector<int>& tags) {
std::vector<Element> elements;
for (std::size_t index = 0; index < keys.size(); ++index) elements.push_back({keys[index], static_cast<int>(index)});
qsort_nid_postfix(elements.data(), elements.size(), sizeof(Element), CompareKeys);
for (std::size_t index = 0; index < tags.size(); ++index) {
if (elements[index].tag != tags[index]) {
std::fprintf(stderr, "%s: element %zu has tag %d, expected %d\n", name, index, elements[index].tag, tags[index]);
return false;
}
}
return true;
}
bool ExpectSorted(std::size_t size, std::size_t count) {
std::vector<unsigned char> bytes(size * count);
unsigned int state = static_cast<unsigned int>(size * 31 + count);
for (auto& byte : bytes) {
state = state * 1103515245u + 12345u;
byte = static_cast<unsigned char>(state >> 16);
}
std::vector<unsigned char> counts(256);
for (std::size_t index = 0; index < count; ++index) ++counts[bytes[index * size]];
qsort_nid_postfix(bytes.data(), count, size, CompareFirstByte);
for (std::size_t index = 0; index < count; ++index) {
if (index > 0 && bytes[(index - 1) * size] > bytes[index * size]) {
std::fprintf(stderr, "qsort of %zu %zu-byte elements is not sorted at %zu\n", count, size, index);
return false;
}
--counts[bytes[index * size]];
}
for (const unsigned char remaining : counts) {
if (remaining != 0) {
std::fprintf(stderr, "qsort of %zu %zu-byte elements lost or duplicated elements\n", count, size);
return false;
}
}
return true;
}
}
int main() {
bool correct = true;
correct &= ExpectOrder("insertion sort", {1, 0, 1, 0}, {1, 3, 0, 2});
correct &= ExpectOrder("median of three", {2, 1, 2, 0, 1, 2, 0, 1, 0, 2, 1, 0}, {11, 3, 6, 8, 10, 1, 7, 4, 9, 5, 0, 2});
std::vector<int> keys;
for (int index = 0; index < 48; ++index) keys.push_back((index * 7 + 3) % 5);
correct &= ExpectOrder("pseudomedian of nine", keys,
{21, 11, 36, 6, 1, 31, 16, 46, 41, 26, 29, 4, 24, 44, 9, 39, 34, 14, 19, 12, 2, 7, 17, 22,
27, 32, 37, 42, 47, 5, 40, 30, 15, 35, 25, 20, 0, 45, 10, 28, 33, 13, 38, 8, 3, 23, 43, 18});
for (const std::size_t size : {1, 3, 8, 24})
for (const std::size_t count : {0, 1, 2, 6, 7, 8, 40, 41, 1000})
correct &= ExpectSorted(size, count);
return correct ? 0 : 1;
}

26
core/libs/tests/GuestSaveDataSearch.cpp
View File

@ -81,6 +81,31 @@ static void SortByKey() {
Require(sceSaveDataDirNameSearch(&cond, &result) == SaveDataErrorParameter, "an unknown sort order must be rejected");
}
static std::vector<std::string> SearchPattern(const char* text) {
SceSaveDataDirName pattern{};
std::snprintf(pattern.data, sizeof(pattern.data), "%s", text);
SaveDataDirNameSearchCond cond{};
cond.user_id = 1;
cond.dir_name = &pattern;
std::array<SceSaveDataDirName, 4> names{};
SaveDataDirNameSearchResult result{};
result.dir_names = names.data();
result.dir_names_num = static_cast<std::uint32_t>(names.size());
Require(sceSaveDataDirNameSearch(&cond, &result) == 0, "pattern search failed");
std::vector<std::string> found;
for (std::uint32_t i = 0; i < result.set_num; ++i) found.emplace_back(names[i].data);
return found;
}
static void TrailingWildcards() {
for (const char* name : std::array{"TRAIL", "TRAIL1"}) std::filesystem::create_directories(std::filesystem::path("_sd") / name);
using Names = std::vector<std::string>;
Require(SearchPattern("TRAIL%") == Names{"TRAIL", "TRAIL1"}, "a trailing % must also match no characters");
Require(SearchPattern("TRAIL%%") == Names{"TRAIL", "TRAIL1"}, "trailing %% must also match no characters");
Require(SearchPattern("TRAI_%") == Names{"TRAIL", "TRAIL1"}, "_ then % must match one character and then none");
Require(SearchPattern("TRAIL_%") == Names{"TRAIL1"}, "_ must still match exactly one character");
}
static void SearchAndMount() {
const std::array names{std::string(31, 'A'), std::string(31, 'B')};
for (const auto& name : names) std::filesystem::create_directories(std::filesystem::path("_sd") / name);
@ -139,6 +164,7 @@ int main() {
try {
SearchAndMount();
SortByKey();
TrailingWildcards();
} catch (const std::exception& error) {
std::fprintf(stderr, "%s\n", error.what());
status = 1;

129
core/libs/tests/GuestSceNet.cpp
View File

@ -39,6 +39,7 @@ std::int64_t APS5_VABI sceNetRecvfrom(int, void*, std::size_t, int, void*, std::
int APS5_VABI sceNetSocketClose(int);
int APS5_VABI sceNetShutdown(int, int);
int APS5_VABI sceNetSetsockopt(int, int, int, const void*, std::uint32_t);
int APS5_VABI sceNetGetsockopt(int, int, int, void*, std::uint32_t*);
int* APS5_VABI sceNetErrnoLoc(void);
int APS5_VABI sceNetEpollCreate(const char*, int);
int APS5_VABI sceNetEpollControl(int, int, int, const NetEpollEvent*);
@ -100,6 +101,63 @@ static bool Failed(std::int64_t result, int error) {
return result == static_cast<int>(0x80410100u | static_cast<unsigned>(error)) && *sceNetErrnoLoc() == error;
}
static void CheckStreamSender(int family) {
const auto addressSize = family == 2 ? 16u : 28u;
const int listener = sceNetSocket(nullptr, family, 1, 6);
Require(listener >= 0);
std::array<std::uint8_t, 28> destination{};
destination[0] = static_cast<std::uint8_t>(addressSize);
destination[1] = static_cast<std::uint8_t>(family);
if (family == 2) { destination[4] = 127; destination[7] = 1; }
else destination[23] = 1;
Require(sceNetBind_nid_postfix(listener, destination.data(), addressSize) == 0 && sceNetListen(listener, 1) == 0);
std::uint32_t size = destination.size();
Require(sceNetGetsockname(listener, destination.data(), &size) == 0 && size == addressSize);
const int client = sceNetSocket(nullptr, family, 1, 6);
Require(client >= 0 && sceNetConnect(client, destination.data(), addressSize) == 0);
const int accepted = sceNetAccept(listener, nullptr, nullptr);
Require(accepted >= 0);
const char payload[] = "stream";
for (const auto endpoints : {std::array{accepted, client}, std::array{client, accepted}}) {
const auto receiver = endpoints[0], sender = endpoints[1];
for (const std::uint32_t capacity : {0u, 1u, 16u, 28u, UINT32_MAX}) {
Require(sceNetSend(sender, payload, sizeof(payload), 0) == sizeof(payload));
std::array<std::uint8_t, 28> address;
address.fill(0xa5);
char received[sizeof(payload)]{};
size = capacity;
*sceNetErrnoLoc() = 123;
Require(sceNetRecvfrom(receiver, received, 0, 2, address.data(), &size) == 0);
Require(size == 0 && received[0] == 0 && *sceNetErrnoLoc() == 123);
Require(std::all_of(address.begin(), address.end(), [](auto byte) { return byte == 0xa5; }));
for (const int flags : {2, 0}) {
size = capacity;
*sceNetErrnoLoc() = 123;
Require(sceNetRecvfrom(receiver, received, sizeof(received), flags, address.data(), &size) == sizeof(payload));
Require(size == 0 && std::memcmp(received, payload, sizeof(payload)) == 0 && *sceNetErrnoLoc() == 123);
Require(std::all_of(address.begin(), address.end(), [](auto byte) { return byte == 0xa5; }));
}
}
Require(sceNetSend(sender, payload, sizeof(payload), 0) == sizeof(payload));
std::array<std::uint8_t, 28> address;
address.fill(0xa5);
char received[sizeof(payload)]{};
Require(Failed(sceNetRecvfrom(receiver, received, sizeof(received), 0, address.data(), nullptr), 22));
size = 123;
Require(sceNetRecvfrom(receiver, received, sizeof(received), 0, nullptr, &size) == sizeof(payload));
Require(size == 123 && std::memcmp(received, payload, sizeof(payload)) == 0);
Require(sceNetShutdown(sender, 1) == 0);
for (const int flags : {2, 0}) {
size = address.size();
*sceNetErrnoLoc() = 123;
Require(sceNetRecvfrom(receiver, received, sizeof(received), flags, address.data(), &size) == 0);
Require(size == 0 && *sceNetErrnoLoc() == 123);
Require(std::all_of(address.begin(), address.end(), [](auto byte) { return byte == 0xa5; }));
}
}
for (const int descriptor : {client, accepted, listener}) Require(sceNetSocketClose(descriptor) == 0);
}
static void CheckPoolStats() {
const int pool = sceNetPoolCreate("stats", 0x4000, 0);
Require(pool > 0);
@ -256,6 +314,49 @@ static void CheckMessages(int receiver, int sender, const std::array<std::uint8_
Require(threw);
}
static void RequireTimeout(int socket, int option, int expected) {
int microseconds = -1;
std::uint32_t size = sizeof(microseconds);
Require(sceNetGetsockopt(socket, 0xffff, option, &microseconds, &size) == 0 && size == sizeof(microseconds));
if (microseconds != expected) {
std::fprintf(stderr, "setsockopt(0x%x) with a struct timeval on a libSceNet socket set %d microseconds, expected %d\n",
option, microseconds, expected);
std::abort();
}
}
static void CheckKernelTimeouts() {
const int descriptor = sceNetSocket("kernel-timeouts", 2, 2, 17);
Require(descriptor >= 0);
const KernelTimeval receive{1, 250000};
Require(setsockopt_nid_postfix(descriptor, 0xffff, 0x1006, &receive, sizeof(receive)) == 0);
RequireTimeout(descriptor, 0x1006, 1250000);
const KernelTimeval send{0, 500000};
Require(setsockopt_nid_postfix(descriptor, 0xffff, 0x1005, &send, sizeof(send)) == 0);
RequireTimeout(descriptor, 0x1005, 500000);
const KernelTimeval invalid{0, 1000000};
Require(setsockopt_nid_postfix(descriptor, 0xffff, 0x1006, &invalid, sizeof(invalid)) == -1 && *__error_nid_postfix() == 33);
const KernelTimeval negative{-1, 0};
Require(setsockopt_nid_postfix(descriptor, 0xffff, 0x1005, &negative, sizeof(negative)) == -1 && *__error_nid_postfix() == 33);
const int microseconds = 1000;
Require(setsockopt_nid_postfix(descriptor, 0xffff, 0x1006, &microseconds, sizeof(microseconds)) == -1 && *__error_nid_postfix() == 22);
RequireTimeout(descriptor, 0x1006, 1250000);
RequireTimeout(descriptor, 0x1005, 500000);
Require(sceNetSetsockopt(descriptor, 0xffff, 0x1006, &microseconds, sizeof(microseconds)) == 0);
RequireTimeout(descriptor, 0x1006, microseconds);
#ifndef _WIN32
std::array<std::uint8_t, 16> loopback{16, 2, 0, 0, 127, 0, 0, 1};
Require(sceNetBind_nid_postfix(descriptor, loopback.data(), loopback.size()) == 0);
const KernelTimeval brief{0, 50000};
Require(setsockopt_nid_postfix(descriptor, 0xffff, 0x1006, &brief, sizeof(brief)) == 0);
char buffer[8]{};
const auto start = std::chrono::steady_clock::now();
Require(Failed(sceNetRecv(descriptor, buffer, sizeof(buffer), 0), 35));
Require(std::chrono::steady_clock::now() - start >= std::chrono::milliseconds(40));
#endif
Require(sceNetSocketClose(descriptor) == 0);
}
static void CheckAddressText(int family, const char* text) {
std::array<std::uint8_t, 16> address{};
Require(sceNetInetPton(family, text, address.data()) == 1);
@ -370,6 +471,25 @@ static void CheckUnspecifiedIpv6() {
Require(sceNetSocketClose(receiver) == 0);
}
static void CheckIpv6SocketsAreIpv6Only() {
const int inet = sceNetSocket("ipv4-any", 2, 2, 17);
Require(inet >= 0);
std::array<std::uint8_t, 16> any{16, 2};
Require(sceNetBind_nid_postfix(inet, any.data(), any.size()) == 0);
std::uint32_t size = any.size();
Require(sceNetGetsockname(inet, any.data(), &size) == 0 && size == any.size());
const int inet6 = sceNetSocket("ipv6-any-same-port", 28, 2, 17);
Require(inet6 >= 0);
std::array<std::uint8_t, 28> any6{28, 28, any[2], any[3]};
if (sceNetBind_nid_postfix(inet6, any6.data(), any6.size()) != 0) {
std::fprintf(stderr, "sceNetBind of [::]:%d failed with errno %d while an IPv4 socket holds 0.0.0.0:%d\n",
(any[2] << 8) | any[3], *sceNetErrnoLoc(), (any[2] << 8) | any[3]);
std::abort();
}
Require(sceNetSocketClose(inet6) == 0);
Require(sceNetSocketClose(inet) == 0);
}
int main() {
const auto epoch = std::chrono::steady_clock::time_point{};
Require(KernelSocketSelect::DeadlineAfter(epoch, 2, 250000) == epoch + std::chrono::milliseconds(2250));
@ -392,6 +512,7 @@ int main() {
CheckAddressText(28, "1234:5678:9abc:def0:1234:5678:9abc:def0");
CheckLinkLocalAllNodesIpv6();
CheckUnspecifiedIpv6();
if (!ipv6Unavailable) CheckIpv6SocketsAreIpv6Only();
const int closed = sceNetSocket("closed-by-kernel", 2, 2, 17);
Require(closed >= 0);
@ -401,6 +522,7 @@ int main() {
Require(sceNetSetsockopt(closed, 0xffff, 0x4, &enabled, sizeof(enabled)) == 0);
Require(close_nid_postfix(closed) == 0);
Require(Failed(sceNetSocketClose(closed), 9));
CheckKernelTimeouts();
const int listener = sceNetSocket(nullptr, 2, 1, 6);
Require(listener >= 0);
@ -545,7 +667,7 @@ int main() {
int resolver_error = -1;
Require(sceNetResolverGetError(resolver, &resolver_error) == 0 && resolver_error == 0);
std::array<std::uint8_t, 4> ipv4{};
Require(sceNetResolverStartNtoa(resolver, "guest-sce-net.invalid", ipv4.data(), 5000000, 1, 0) ==
Require(sceNetResolverStartNtoa(resolver, "guest-sce-net.invalid.", ipv4.data(), 5000000, 1, 0) ==
static_cast<int>(0x804101E1));
Require(sceNetResolverGetError(resolver, &resolver_error) == 0 &&
resolver_error == static_cast<int>(0x804101E1));
@ -578,7 +700,7 @@ int main() {
static_cast<int>(0x80410116) && *sceNetErrnoLoc() == 22 && records == resolved);
Require(sceNetResolverStartNtoaMultipleRecordsEx(resolver, "localhost", nullptr, 5000000, 1, 0) ==
static_cast<int>(0x80410116) && *sceNetErrnoLoc() == 22);
Require(sceNetResolverStartNtoaMultipleRecordsEx(resolver, "guest-sce-net.invalid", records.data(), 5000000, 1, 0) ==
Require(sceNetResolverStartNtoaMultipleRecordsEx(resolver, "guest-sce-net.invalid.", records.data(), 5000000, 1, 0) ==
static_cast<int>(0x804101E1) && records == resolved);
Require(sceNetResolverGetError(resolver, nullptr) == static_cast<int>(0x80410116) && *sceNetErrnoLoc() == 22);
Require(sceNetResolverDestroy(resolver) == 0);
@ -594,6 +716,9 @@ int main() {
Require(sceNetInetNtop(28, ipv6.data(), text, sizeof(text)) == text);
Require(std::strcmp(text, "::1") == 0);
CheckStreamSender(2);
if (!ipv6Unavailable) CheckStreamSender(28);
int state = -1;
Require(sceNetCtlGetState(&state) == 0);
Require(state == 0 || state == 3);

33
core/libs/tests/GuestSocketPair.cpp
View File

@ -1,6 +1,7 @@
#include "prx/libc/include/general/VabiMacros.hpp"
#include <array>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <cstring>
@ -10,11 +11,27 @@ struct PollDescriptor {
short revents;
};
struct Iovec {
void* base;
std::uint64_t length;
};
struct Msghdr {
void* name;
std::uint32_t nameLength;
Iovec* iov;
int iovLength;
void* control;
std::uint32_t controlLength;
int flags;
};
extern "C" {
int APS5_VABI socketpair_nid_postfix(int, int, int, int*);
std::int64_t APS5_VABI send_nid_postfix(int, const void*, std::uint64_t, int);
std::int64_t APS5_VABI recv_nid_postfix(int, void*, std::uint64_t, int);
std::int64_t APS5_VABI recvfrom_nid_postfix(int, void*, std::uint64_t, int, void*, std::uint32_t*);
std::int64_t APS5_VABI recvmsg_nid_postfix(int, Msghdr*, int);
int APS5_VABI getsockname_nid_postfix(int, void*, std::uint32_t*);
int APS5_VABI getpeername_nid_postfix(int, void*, std::uint32_t*);
int APS5_VABI shutdown_nid_postfix(int, int);
@ -134,6 +151,22 @@ int main() {
Require(send_nid_postfix(pair[0], "bb", 2, 0) == 2);
Require(recv_nid_postfix(pair[1], datagram, sizeof(datagram), 0) == 1);
Require(recv_nid_postfix(pair[1], datagram, sizeof(datagram), 0) == 2);
Require(send_nid_postfix(pair[0], "record", 6, 0) == 6);
Require(send_nid_postfix(pair[0], "next", 4, 0) == 4);
char part[2]{};
Iovec piece{part, sizeof(part)};
Msghdr message{nullptr, 0, &piece, 1, nullptr, 0, -1};
const auto truncated = recvmsg_nid_postfix(pair[1], &message, 0);
if (truncated != 2 || message.flags != 0x10) {
std::fprintf(stderr, "recvmsg of a 6-byte SOCK_SEQPACKET record into 2 bytes returned %lld with msg_flags 0x%x, expected 2 with MSG_TRUNC (0x10)\n",
static_cast<long long>(truncated), static_cast<unsigned>(message.flags));
std::abort();
}
Require(std::memcmp(part, "re", 2) == 0);
char whole[8]{};
piece = {whole, sizeof(whole)};
message.flags = -1;
Require(recvmsg_nid_postfix(pair[1], &message, 0) == 4 && message.flags == 0 && std::memcmp(whole, "next", 4) == 0);
Require(close_nid_postfix(pair[0]) == 0 && close_nid_postfix(pair[1]) == 0);
#endif

38
core/libs/tests/GuestStandardStreams.cpp
View File

@ -53,6 +53,7 @@ int APS5_VABI setvbuf_nid_postfix(FileStream*, char*, int, std::size_t);
void APS5_VABI setbuf_nid_postfix(FileStream*, char*);
FileStream* APS5_VABI fdopen_nid_postfix(int, const char*);
int APS5_VABI fclose_nid_postfix(FileStream*);
int APS5_VABI fflush_nid_postfix(FileStream*);
int APS5_VABI _Getmbcurmax_nid_postfix();
int APS5_VABI ___mb_cur_max_nid_postfix();
void APS5_VABI flockfile_nid_postfix(FileStream*);
@ -280,6 +281,28 @@ static void CheckRewind() {
#endif
}
static void CheckFlushWhileReading() {
const auto filename = "anyps5-flush-read-" + std::to_string(std::chrono::steady_clock::now().time_since_epoch().count());
{ std::ofstream seed(filename, std::ios::binary); seed << "abcdef\n"; Require(seed.good()); }
for (const char* mode : {"r", "r+"}) {
FileStream* stream = fopen_nid_postfix(filename.c_str(), mode);
Require(stream != nullptr);
Require(fgetc_nid_postfix(stream) == 'a' && ungetc_nid_postfix('Z', stream) == 'Z');
Require(fflush_nid_postfix(stream) == 0);
char rest[16]{};
const auto count = fread_nid_postfix(rest, 1, sizeof(rest) - 1, stream);
if (count != 7 || std::strcmp(rest, "Zbcdef\n") != 0) {
std::string shown;
for (std::size_t index = 0; index < count; ++index) shown += rest[index] == '\n' ? std::string("\\n") : std::string(1, rest[index]);
std::fprintf(stderr, "fflush of a \"%s\" stream that is reading changed the data read next: \"%s\" (%zu bytes), expected \"Zbcdef\\n\" (7 bytes)\n",
mode, shown.c_str(), count);
std::abort();
}
Require(fclose_nid_postfix(stream) == 0);
}
std::filesystem::remove(filename);
}
static bool CheckBinaryModes() {
const auto directory = "anyps5-byte-stream-" + std::to_string(std::chrono::steady_clock::now().time_since_epoch().count());
Require(std::filesystem::create_directory(directory));
@ -295,6 +318,12 @@ static bool CheckBinaryModes() {
FileStream* stream = reopen ? freopen_nid_postfix(filename.c_str(), mode, &redirected) : fopen_nid_postfix(filename.c_str(), mode);
Require(stream != nullptr);
const bool update = std::strchr(mode, '+') != nullptr;
if (*mode == 'a') {
const bool atEnd = ftello_nid_postfix(stream) == static_cast<std::int64_t>(original.size()) &&
(!update || fgetc_nid_postfix(stream) == EOF);
if (!atEnd) std::fprintf(stderr, "%s %s: the append stream does not start at the end of the file\n", reopen ? "freopen" : "fopen", mode);
correct &= atEnd;
}
if (*mode == 'r' || (*mode == 'a' && update)) {
Require(fseeko_nid_postfix(stream, 0, SEEK_SET) == 0);
char bytes[16]{};
@ -365,6 +394,7 @@ static void CheckDescriptorModes(std::FILE* original) {
int main() {
Require(_Getmbcurmax_nid_postfix() == 1 && _Getmbcurmax_nid_postfix() == ___mb_cur_max_nid_postfix());
CheckFlushWhileReading();
Require(fdopen_nid_postfix(-1, "rb") == nullptr && *__error_nid_postfix() == 9);
Require(fdopen_nid_postfix(0, nullptr) == nullptr && *__error_nid_postfix() == 22);
Require(fdopen_nid_postfix(0, "invalid") == nullptr && *__error_nid_postfix() == 22);
@ -475,6 +505,14 @@ int main() {
Require(feof_nid_postfix(&scanned) && (scanned.GuestState().flags & 0x20));
scanned.Close();
FileStream scanBinary(std::tmpfile());
Require(std::fputs("0b101", scanBinary.GetHandle()) >= 0);
std::rewind(scanBinary.GetHandle());
int binaryValue = -1;
Require(fscanf_nid_postfix(&scanBinary, "%i", &binaryValue) == 1 && binaryValue == 0);
Require(fgetc_nid_postfix(&scanBinary) == 'b');
scanBinary.Close();
#ifdef _WIN32
FileStream scanSuppressed(std::tmpfile());
Require(std::fputs("12", scanSuppressed.GetHandle()) >= 0);

50
core/libs/tests/GuestStrings.cpp
View File

@ -31,6 +31,7 @@ int APS5_VABI memcpy_s_nid_postfix(void*, std::size_t, const void*, std::size_t)
int APS5_VABI memmove_s_nid_postfix(void*, std::size_t, const void*, std::size_t);
int APS5_VABI memset_s_nid_postfix(void*, std::size_t, int, std::size_t);
char* APS5_VABI strnstr_nid_postfix(const char*, const char*, std::size_t);
std::size_t APS5_VABI strxfrm_nid_postfix(char*, const char*, std::size_t);
int APS5_VABI snprintf_s_nid_postfix(char*, std::size_t, const char*, ...);
int APS5_VABI sscanf_s_nid_postfix(const char*, const char*, ...);
int APS5_VABI __inet_aton_nid_postfix(const char*, void*);
@ -71,6 +72,40 @@ static bool CheckMemcpyOverlap() {
return correct;
}
static void CheckSecureCopySizeLimits() {
using Copy = int (APS5_VABI*)(void*, std::size_t, const void*, std::size_t);
constexpr std::size_t RsizeMax = SIZE_MAX >> 1;
const std::array<unsigned char, 8> original = {1, 2, 3, 4, 5, 6, 7, 8};
const unsigned char source[] = {9, 10, 11, 12};
for (Copy copy : {memcpy_s_nid_postfix, memmove_s_nid_postfix}) {
for (const auto size : {RsizeMax + 1, SIZE_MAX}) {
for (const auto count : {std::size_t{1}, std::size_t{0}, RsizeMax + 1, SIZE_MAX}) {
for (const void* input : {static_cast<const void*>(source), static_cast<const void*>(nullptr)}) {
auto bytes = original;
Require(copy(bytes.data() + 2, size, input, count) != 0);
Require(bytes == original);
}
}
}
for (const auto count : {RsizeMax + 1, SIZE_MAX}) {
auto bytes = original;
Require(copy(bytes.data() + 2, 4, source, count) != 0);
const std::array<unsigned char, 8> cleared = {1, 2, 0, 0, 0, 0, 7, 8};
Require(bytes == cleared);
}
auto bytes = original;
Require(copy(bytes.data() + 2, RsizeMax, source, 1) == 0);
const std::array<unsigned char, 8> copied = {1, 2, 9, 4, 5, 6, 7, 8};
Require(bytes == copied);
Require(copy(bytes.data() + 2, 4, source, 0) == 0 && bytes == copied);
Require(copy(nullptr, SIZE_MAX, source, 1) == 22);
bytes = original;
Require(copy(bytes.data() + 2, 4, nullptr, 1) == 22);
const std::array<unsigned char, 8> cleared = {1, 2, 0, 0, 0, 0, 7, 8};
Require(bytes == cleared);
}
}
static void CheckBoundsCheckedFunctions() {
char small[4] = "zz";
Require(strcpy_s_nid_postfix(small, sizeof(small), "abc") == 0 && std::strcmp(small, "abc") == 0);
@ -249,6 +284,20 @@ int main() {
Require(stpcpy_nid_postfix(end, "") == end && chained[5] == 'z');
Require(strncat_nid_postfix(buffer, "xyz", 2) == buffer);
Require(std::strcmp(buffer, "xy") == 0);
char transformed[8] = "zzzzzzz";
Require(strxfrm_nid_postfix(transformed, "abc", sizeof(transformed)) == 3);
Require(std::strcmp(transformed, "abc") == 0 && transformed[4] == 'z');
std::memset(transformed, 'z', sizeof(transformed));
Require(strxfrm_nid_postfix(transformed, "abcdefgh", 4) == 8);
Require(std::strcmp(transformed, "abc") == 0 && transformed[4] == 'z');
std::memset(transformed, 'z', sizeof(transformed));
Require(strxfrm_nid_postfix(transformed, "abcdefg", sizeof(transformed)) == 7 && std::strcmp(transformed, "abcdefg") == 0);
std::memset(transformed, 'z', sizeof(transformed));
Require(strxfrm_nid_postfix(transformed, "abcdefgh", sizeof(transformed)) == 8 && std::strcmp(transformed, "abcdefg") == 0);
std::memset(transformed, 'z', sizeof(transformed));
Require(strxfrm_nid_postfix(transformed, "", sizeof(transformed)) == 0 && transformed[0] == '\0' && transformed[1] == 'z');
Require(strxfrm_nid_postfix(transformed, "abc", 1) == 3 && transformed[0] == '\0' && transformed[1] == 'z');
Require(strxfrm_nid_postfix(nullptr, "abc", 0) == 3);
Require(strpbrk_nid_postfix(buffer, "ay") == buffer + 1);
Require(strpbrk_nid_postfix(buffer, "") == nullptr);
Require(strcspn_nid_postfix(buffer, "y") == 1);
@ -329,5 +378,6 @@ int main() {
Require(strcasestr_nid_postfix("", "a") == nullptr);
const char highBytes[] = {static_cast<char>(0xff), 'A', 0};
Require(strcasestr_nid_postfix(highBytes, "a") == highBytes + 1);
CheckSecureCopySizeLimits();
return CheckMemcpyOverlap() ? 0 : 1;
}

4
core/libs/tests/GuestSyslog.cpp
View File

@ -59,6 +59,7 @@ int main() {
std::thread([] { Require(setlogmask_nid_postfix(0xff) == 0x55); }).join();
Require(setlogmask_nid_postfix(0) == 0xff);
syslog_nid_postfix(7, "mask shared across threads");
syslog_nid_postfix(7, "wide %ls %d %S", u"guest", 5, u"text");
std::fflush(stderr);
#ifdef _WIN32
Require(::_dup2(saved, ::_fileno(stderr)) == 0 && ::_close(saved) == 0);
@ -83,7 +84,8 @@ int main() {
"[syslog:18] level 2\n"
"[syslog:20] level 4\n"
"[syslog:22] level 6\n"
"[syslog:15] mask shared across threads\n";
"[syslog:15] mask shared across threads\n"
"[syslog:15] wide guest 5 text\n";
Require(length == std::strlen(expected) && std::strcmp(text, expected) == 0);
Require(std::fclose(captured) == 0);
}

29
core/libs/tests/GuestTcp.cpp
View File

@ -2,6 +2,7 @@
#include <algorithm>
#include <array>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <cstring>
@ -48,8 +49,36 @@ static void StreamReceiveReportsNoSender() {
for (const int descriptor : {client, accepted, listener}) Require(close_nid_postfix(descriptor) == 0);
}
static void ListenersShareAPortAcrossFamilies() {
const int inet = socket_nid_postfix(2, 1, 0);
Require(inet >= 0);
std::array<std::uint8_t, 16> any{16, 2};
Require(bind_nid_postfix(inet, any.data(), any.size()) == 0 && listen_nid_postfix(inet, 4) == 0);
std::uint32_t size = any.size();
Require(getsockname_nid_postfix(inet, any.data(), &size) == 0 && size == any.size());
const int inet6 = socket_nid_postfix(28, 1, 0);
if (inet6 < 0) {
Require(*__error_nid_postfix() == 47);
Require(close_nid_postfix(inet) == 0);
return;
}
std::array<std::uint8_t, 28> any6{28, 28, any[2], any[3]};
if (bind_nid_postfix(inet6, any6.data(), any6.size()) != 0) {
if (*__error_nid_postfix() != 49) {
std::fprintf(stderr, "bind of [::]:%d failed with errno %d while an IPv4 listener holds 0.0.0.0:%d\n",
(any[2] << 8) | any[3], *__error_nid_postfix(), (any[2] << 8) | any[3]);
std::abort();
}
} else {
Require(listen_nid_postfix(inet6, 4) == 0);
}
Require(close_nid_postfix(inet6) == 0);
Require(close_nid_postfix(inet) == 0);
}
int main() {
StreamReceiveReportsNoSender();
ListenersShareAPortAcrossFamilies();
const int listener = socket_nid_postfix(2, 1, 6);
Require(listener >= 0);
std::array<std::uint8_t, 16> address{16, 2, 0, 0, 127, 0, 0, 1};

169
core/libs/tests/GuestVideodec2.cpp
View File

@ -1,9 +1,11 @@
#include <algorithm>
#include <array>
#include <atomic>
#include <cstdint>
#include <cstdio>
#include <stdexcept>
#include <string>
#include <thread>
#include <vector>
#include "prx/libc/include/General.hpp"
#include "H264Fixture.hpp"
@ -444,6 +446,171 @@ void testDecode(bool lengthPrefixed) {
checkThrows([&] { sceVideodec2DeleteDecoder_nid_postfix(handle); });
}
std::uint64_t createDecoder(std::uint32_t codec, std::uint32_t profile) {
const DecoderConfigInfo config{sizeof(DecoderConfigInfo), 1, codec, profile, 0, static_cast<std::int32_t>(Width), static_cast<std::int32_t>(Height), 4, 1};
DecoderMemoryInfo memory{sizeof(DecoderMemoryInfo)};
check(sceVideodec2QueryDecoderMemoryInfo_nid_postfix(&config, &memory) == 0, "memory query failed");
std::uint64_t handle = 0;
check(sceVideodec2CreateDecoder_nid_postfix(&config, &memory, &handle) == 0, "decoder creation failed");
return handle;
}
AvcPictureInfo query(std::vector<std::uint8_t>& buffer) {
OutputInfo output{sizeof(OutputInfo)};
output.frameBuffer = buffer.data();
AvcPictureInfo info{sizeof(AvcPictureInfo)};
check(sceVideodec2GetAvcPictureInfo_nid_postfix(&output, &info, nullptr) == 0, "picture info query failed");
return info;
}
bool owned(std::vector<std::uint8_t>& buffer, std::uint64_t pts) {
const auto info = query(buffer);
return info.isValid && info.ptsData == pts;
}
std::uint64_t publishAvc(std::uint64_t handle, std::uint64_t base, std::vector<std::uint8_t>& buffer) {
const auto units = accessUnits(false);
std::size_t pictures = 0;
const auto take = [&](const OutputInfo& output) {
if (!output.isValid) return false;
check(owned(buffer, base + DisplayOrderUnits[pictures]), "picture info does not follow the latest publication");
++pictures;
return true;
};
for (std::size_t unit = 0; unit < units.size(); ++unit) {
const InputData input{sizeof(InputData), units[unit].data(), units[unit].size(), base + unit, unit, base + unit};
FrameBuffer frame{sizeof(FrameBuffer), buffer.data(), buffer.size(), false};
OutputInfo output{sizeof(OutputInfo)};
check(sceVideodec2Decode_nid_postfix(handle, &input, &frame, &output) == 0, "decode failed");
take(output);
}
for (;;) {
FrameBuffer frame{sizeof(FrameBuffer), buffer.data(), buffer.size(), false};
OutputInfo output{sizeof(OutputInfo)};
check(sceVideodec2Flush_nid_postfix(handle, &frame, &output) == 0, "flush failed");
if (!take(output)) break;
}
check(pictures == PictureHashes.size(), "missing pictures after flush");
return base + DisplayOrderUnits[pictures - 1];
}
void publishClip(std::uint64_t handle, const std::vector<std::vector<std::uint8_t>>& units, std::vector<std::uint8_t>& buffer) {
for (std::size_t unit = 0; unit < units.size(); ++unit) {
const InputData input{sizeof(InputData), units[unit].data(), units[unit].size(), unit, unit, 0};
FrameBuffer frame{sizeof(FrameBuffer), buffer.data(), buffer.size(), false};
OutputInfo output{sizeof(OutputInfo)};
check(sceVideodec2Decode_nid_postfix(handle, &input, &frame, &output) == 0, "decode failed");
}
for (;;) {
FrameBuffer frame{sizeof(FrameBuffer), buffer.data(), buffer.size(), false};
OutputInfo output{sizeof(OutputInfo)};
check(sceVideodec2Flush_nid_postfix(handle, &frame, &output) == 0, "flush failed");
if (!output.isValid) break;
}
}
std::uint64_t frameBufferBytes() {
const DecoderConfigInfo config{sizeof(DecoderConfigInfo), 1, 1, 100, 0, static_cast<std::int32_t>(Width), static_cast<std::int32_t>(Height), 4, 1};
DecoderMemoryInfo memory{sizeof(DecoderMemoryInfo)};
check(sceVideodec2QueryDecoderMemoryInfo_nid_postfix(&config, &memory) == 0, "memory query failed");
return memory.maxFrameBufferSize;
}
void testPictureOwnership() {
std::vector<std::uint8_t> shared(frameBufferBytes()), other(frameBufferBytes());
{
const auto older = createDecoder(1, 100);
const auto newer = createDecoder(1, 100);
publishAvc(older, 1000, shared);
const auto latest = publishAvc(newer, 5000, shared);
check(owned(shared, latest), "an older decoder answered for a buffer a newer decoder wrote last");
check(sceVideodec2Reset_nid_postfix(newer) == 0, "reset failed");
check(!query(shared).isValid, "a stale picture survived the reset of the decoder that last wrote the buffer");
check(sceVideodec2DeleteDecoder_nid_postfix(older) == 0 && sceVideodec2DeleteDecoder_nid_postfix(newer) == 0, "delete failed");
}
{
const auto older = createDecoder(1, 100);
const auto newer = createDecoder(1, 100);
publishAvc(older, 1000, shared);
publishAvc(newer, 5000, shared);
check(sceVideodec2DeleteDecoder_nid_postfix(newer) == 0, "delete failed");
check(!query(shared).isValid, "a stale picture survived the deletion of the decoder that last wrote the buffer");
check(sceVideodec2DeleteDecoder_nid_postfix(older) == 0, "delete failed");
}
{
const auto older = createDecoder(1, 100);
const auto newer = createDecoder(1, 100);
publishAvc(older, 1000, shared);
const auto latest = publishAvc(newer, 5000, shared);
check(sceVideodec2Reset_nid_postfix(older) == 0, "reset failed");
check(owned(shared, latest), "resetting a decoder that no longer owns the buffer dropped its picture");
check(sceVideodec2DeleteDecoder_nid_postfix(older) == 0, "delete failed");
check(owned(shared, latest), "deleting a decoder that no longer owns the buffer dropped its picture");
check(sceVideodec2DeleteDecoder_nid_postfix(newer) == 0, "delete failed");
check(!query(shared).isValid, "a deleted decoder still owns its picture");
}
{
const auto older = createDecoder(1, 100);
const auto newer = createDecoder(1, 100);
publishAvc(newer, 5000, shared);
const auto latest = publishAvc(older, 1000, shared);
const auto third = createDecoder(1, 100);
const auto separate = publishAvc(third, 9000, other);
check(owned(shared, latest) && owned(other, separate), "distinct buffers are not independent");
check(sceVideodec2Reset_nid_postfix(older) == 0, "reset failed");
check(!query(shared).isValid && owned(other, separate), "resetting one buffer's owner changed another buffer");
check(sceVideodec2DeleteDecoder_nid_postfix(older) == 0 && sceVideodec2DeleteDecoder_nid_postfix(newer) == 0 && sceVideodec2DeleteDecoder_nid_postfix(third) == 0, "delete failed");
}
for (const auto codec : {CodecHevc, CodecVp9}) {
const auto units = codec == CodecHevc ? hevcUnits(HevcStream.data(), HevcStream.size()) : ivfFrames();
const auto avc = createDecoder(1, 100);
const auto foreign = createDecoder(codec, codec == CodecHevc ? 1 : 0);
publishAvc(avc, 1000, shared);
publishClip(foreign, units, shared);
checkThrows([&] { query(shared); });
check(sceVideodec2DeleteDecoder_nid_postfix(avc) == 0, "delete failed");
const auto newer = createDecoder(1, 100);
const auto latest = publishAvc(newer, 5000, shared);
check(owned(shared, latest), "a decoder of another codec answered for an AVC picture");
check(sceVideodec2DeleteDecoder_nid_postfix(foreign) == 0 && sceVideodec2DeleteDecoder_nid_postfix(newer) == 0, "delete failed");
}
}
void testConcurrentOwnership() {
std::vector<std::uint8_t> shared(frameBufferBytes());
const auto keeper = createDecoder(1, 100);
const auto kept = publishAvc(keeper, 1000, shared);
std::atomic<bool> done = false;
std::string failure;
std::thread reader([&] {
try {
while (!done.load()) {
const auto info = query(shared);
check(!info.isValid || info.ptsData == kept || (info.ptsData >= 5000 && info.ptsData < 5000 + PictureHashes.size()), "a query returned metadata no decoder published");
}
} catch (const std::exception& error) {
failure = error.what();
}
});
try {
for (int round = 0; round < 8; ++round) {
const auto writer = createDecoder(1, 100);
publishAvc(writer, 5000, shared);
if (round % 2 == 0) check(sceVideodec2Reset_nid_postfix(writer) == 0, "reset failed");
check(sceVideodec2DeleteDecoder_nid_postfix(writer) == 0, "delete failed");
check(!query(shared).isValid, "a removed writer left its picture behind");
}
} catch (...) {
done = true;
reader.join();
throw;
}
done = true;
reader.join();
check(failure.empty(), failure);
check(sceVideodec2DeleteDecoder_nid_postfix(keeper) == 0, "delete failed");
}
}
int main() {
@ -475,6 +642,8 @@ int main() {
testClip(CodecHevc, 1, hevcLengthPrefixed, HevcHashes);
testClip(CodecVp9, 0, ivfFrames(), Vp9Hashes);
testTenBit();
testPictureOwnership();
testConcurrentOwnership();
std::puts("Videodec2 tests passed");
return 0;
} catch (const std::exception& error) {

24
core/libs/tests/GuestWideFormatting.cpp
View File

@ -89,6 +89,20 @@ static void CheckBounded() {
Require(Format(buffer, 16, u"%s", nullNarrow) == 6 && buffer == std::u16string(u"(null)"), "vswprintf null %s argument");
}
static void CheckPositional() {
char16_t buffer[32];
Require(swprintf_nid_postfix(buffer, 32, u"%2$d-%1$d", 1, 2) == 3 && buffer == std::u16string(u"2-1"),
"swprintf positional integers");
Require(swprintf_nid_postfix(buffer, 32, u"%2$ls %1$ls", u"world", u"hello") == 11 && buffer == std::u16string(u"hello world"),
"swprintf positional wide strings");
Require(swprintf_nid_postfix(buffer, 32, u"[%1$*2$d|%3$.*2$f]", 7, 2, 2.5) == 9 && buffer == std::u16string(u"[ 7|2.50]"),
"swprintf positional width and precision");
Require(swprintf_nid_postfix(buffer, 32, u"%1$Lg %2$d %d", 0.5L, 3, 9) == 7 && buffer == std::u16string(u"0.5 3 9"),
"swprintf positional long double and the next argument");
Require(snwprintf_s_nid_postfix(buffer, 32, u"%2$s%1$lc", 0x20ac, "\xc3\xa9") == 2 && buffer == std::u16string(u"\u00e9\u20ac"),
"snwprintf_s positional arguments");
}
static void CheckPointer() {
char16_t buffer[64];
const auto address = [](std::uintptr_t value) { return reinterpret_cast<void*>(value); };
@ -121,6 +135,14 @@ static void CheckCount() {
Require(Format(buffer, 16, u"a%n", nullCount) < 0, "vswprintf null %n argument");
}
static void CheckGroupingFlag() {
char16_t buffer[48];
Require(swprintf_nid_postfix(buffer, 48, u"%'d|%'-6u|%'.2f|%'+05d|%'g", 1234567, 42u, 1234.5, 12, 1e6) == 34 &&
buffer == std::u16string(u"1234567|42 |1234.50|+0012|1e+06"), "swprintf ' flag");
Require(snwprintf_s_nid_postfix(buffer, 48, u"%'lld %ls", 9876543210ll, u"x") == 12 && buffer == std::u16string(u"9876543210 x"),
"snwprintf_s ' flag");
}
static void CheckSwprintf() {
char16_t buffer[8];
for (auto& unit : buffer) unit = u'x';
@ -147,6 +169,7 @@ static void CheckSwprintf() {
int main() {
CheckBounded();
CheckPositional();
CheckPointer();
CheckCount();
CheckSwprintf();
@ -161,4 +184,5 @@ int main() {
Check(u"%.2s", "\xf0\x9f\x98\x80x", u"\U0001f600");
Check(u"%.1s", "\xf0\x9f\x98\x80x", u"");
Check(u"%.3s", "\xf0\x9f\x98\x80x", u"\U0001f600x");
CheckGroupingFlag();
}

23
core/libs/tests/WindowsExceptions.cpp
View File

@ -248,6 +248,28 @@ static void testUnwindThroughGuestFrames() {
}
}
static int onceDestroyed;
struct OnceGuard { ~OnceGuard() { ++onceDestroyed; } };
static void ThrowThroughOnce(int depth) {
OnceGuard guard;
if (depth) { ThrowThroughOnce(depth - 1); return; }
std::once_flag flag;
std::call_once(flag, [] { throw std::runtime_error("through runtime frames"); });
}
static void testUnwindThroughRuntimeModules() {
for (int depth : {0, 1, 7, 64}) {
onceDestroyed = 0;
try {
ThrowThroughOnce(depth);
throw std::logic_error("call_once did not propagate");
} catch (const std::runtime_error& error) {
if (std::strcmp(error.what(), "through runtime frames")) throw;
}
if (onceDestroyed != depth + 1) throw std::runtime_error("unwinding through runtime DLL frames skipped a destructor");
}
}
int main(int argc, char** argv) {
if (argc > 1) {
if (std::strcmp(argv[1], "throw-uncaught") == 0) return RunThrowChild();
@ -260,6 +282,7 @@ int main(int argc, char** argv) {
testMadeExceptionPointer();
testUncaughtExceptions();
testUnwindThroughGuestFrames();
testUnwindThroughRuntimeModules();
try {
Rethrow();
return 1;

20
core/relinker/CMakeLists.txt
View File

@ -181,6 +181,19 @@ if(BUILD_TESTING)
add_test(NAME call_site_index COMMAND call_site_index_tests)
set_tests_properties(call_site_index PROPERTIES TIMEOUT 20)
add_test_executable(sdk_mman_syscall_tests
${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/SdkMmanSyscallTests.cpp
${CMAKE_CURRENT_SOURCE_DIR}/relinker/src/analysis/SyscallScanner.cpp
${CMAKE_CURRENT_SOURCE_DIR}/codegen/src/InstructionScanner.cpp
${CMAKE_CURRENT_SOURCE_DIR}/codegen/src/x86/X64InstructionDecoder.cpp
${CMAKE_CURRENT_SOURCE_DIR}/codegen/src/x86/DecodedInstruction.cpp)
target_include_directories(sdk_mman_syscall_tests PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}/domain/include
${CMAKE_CURRENT_SOURCE_DIR}/relinker/include
${CMAKE_CURRENT_SOURCE_DIR}/codegen/include)
add_test(NAME sdk_mman_syscall COMMAND sdk_mman_syscall_tests)
set_tests_properties(sdk_mman_syscall PROPERTIES TIMEOUT 20)
add_test_executable(autorun_exit_tests
${CMAKE_CURRENT_SOURCE_DIR}/cli/tests/AutorunExitTests.cpp
${CMAKE_CURRENT_SOURCE_DIR}/cli/src/Autorun.cpp
@ -450,6 +463,9 @@ if(BUILD_TESTING)
${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_guest_absolute_symbols.py $<TARGET_FILE:relinker>)
add_test(NAME guest_module_identity COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_guest_module_identity.py $<TARGET_FILE:relinker>)
add_test(NAME guest_export_lookup COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_guest_export_lookup.py $<TARGET_FILE:relinker>)
set_tests_properties(guest_export_lookup PROPERTIES TIMEOUT 30)
add_test(NAME guest_needed_nested_modules COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_guest_needed_nested_modules.py $<TARGET_FILE:relinker>)
add_test(NAME windows_import_modules COMMAND ${Python3_EXECUTABLE}
@ -462,12 +478,16 @@ if(BUILD_TESTING)
${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_input_preservation.py $<TARGET_FILE:relinker>)
add_test(NAME input_file COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_input_file.py $<TARGET_FILE:relinker>)
add_test(NAME null_symbol_relocations COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_null_symbol_relocations.py $<TARGET_FILE:relinker>)
add_test(NAME input_nonascii COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_input_nonascii.py $<TARGET_FILE:relinker>)
add_test(NAME string_table_bounds COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_string_table_bounds.py $<TARGET_FILE:relinker>)
add_test(NAME sce_code_extent COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_sce_code_extent.py $<TARGET_FILE:relinker>)
add_test(NAME sce_relro COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_sce_relro.py $<TARGET_FILE:relinker>)
add_test(NAME call_registry COMMAND ${Python3_EXECUTABLE}
${CMAKE_CURRENT_SOURCE_DIR}/relinker/tests/test_call_registry.py $<TARGET_FILE:relinker>)
add_test(NAME jmprel_table_bounds COMMAND ${Python3_EXECUTABLE}

2
core/relinker/codegen/include/codegen/IAmd64OnlyConverter.hpp
View File

@ -14,7 +14,7 @@ public:
[[nodiscard]] virtual ConvertResult Convert(
std::vector<std::uint8_t> fileBytes,
const std::vector<Domain::ProgramHeader>& codeSegments
const std::vector<Domain::ProgramHeader>& segments
) const = 0;
};

60
core/relinker/codegen/src/Amd64OnlyConverter.cpp
View File

@ -7,6 +7,7 @@
#include <codegen/x86/IAmd64OnlyInstructionMatcher.hpp>
#include <codegen/x86/StubBodyBuilder.hpp>
#include <algorithm>
#include <cstring>
#include <limits>
#include <memory>
#include <optional>
@ -29,6 +30,9 @@ auto _atFileOffset(const Domain::FileByteOffset base, const TOperation& operatio
}
constexpr std::int64_t kShortJumpSize = 2;
constexpr std::uint32_t kLoadSegment = 1;
constexpr std::uint32_t kExecutableSegment = 1;
constexpr std::uint8_t kLea = 0x8D;
constexpr std::uint64_t kIslandSearchBytes = 0x100;
struct Placement {
@ -209,7 +213,7 @@ public:
[[nodiscard]] ConvertResult Convert(
std::vector<std::uint8_t> fileBytes,
const std::vector<Domain::ProgramHeader>& codeSegments
const std::vector<Domain::ProgramHeader>& segments
) const override;
private:
@ -225,30 +229,59 @@ private:
void _convertSegment(
std::vector<std::uint8_t>& fileBytes,
const Domain::ProgramHeader& ph,
const std::vector<Domain::ProgramHeader>& segments,
ConvertResult& result
) const;
[[nodiscard]] static std::set<std::uint64_t> _collectBranchTargets(
const std::vector<std::uint8_t>& seg,
const std::vector<InstructionMatch>& matches,
const Domain::ProgramHeader& ph
const Domain::ProgramHeader& ph,
const std::vector<std::uint8_t>& fileBytes,
const std::vector<Domain::ProgramHeader>& segments
);
};
std::set<std::uint64_t> Amd64OnlyConverter::_collectBranchTargets(
const std::vector<std::uint8_t>& seg,
const std::vector<InstructionMatch>& matches,
const Domain::ProgramHeader& ph
const Domain::ProgramHeader& ph,
const std::vector<std::uint8_t>& fileBytes,
const std::vector<Domain::ProgramHeader>& segments
) {
const auto startsInstruction = [&](const std::uint64_t address) {
const auto offset = address - ph.MappedAddress;
const auto match = std::lower_bound(matches.begin(), matches.end(), offset, [](const InstructionMatch& item, const std::uint64_t value) { return item.Offset < value; });
return match != matches.end() && match->Offset == offset;
};
const X64InstructionDecoder decoder;
std::set<std::uint64_t> targets;
for (const auto& match : matches) {
const auto info = decoder.DecodeInstruction(seg.data() + match.Offset, match.Length);
if (!info.HasBranchTarget || info.HasRipRelativeDisp)
continue;
const auto target = static_cast<std::int64_t>(ph.MappedAddress + match.Offset + info.Length) + info.BranchDisp;
if (target >= 0)
targets.insert(static_cast<std::uint64_t>(target));
const auto next = static_cast<std::int64_t>(ph.MappedAddress + match.Offset + info.Length);
if (info.HasBranchTarget && !info.HasRipRelativeDisp) {
const auto target = next + info.BranchDisp;
if (target >= 0)
targets.insert(static_cast<std::uint64_t>(target));
} else if (info.HasRipRelativeDisp && info.Opcode == kLea) {
std::int32_t displacement;
std::memcpy(&displacement, seg.data() + match.Offset + info.RipRelativeDispOffset, sizeof(displacement));
const auto table = static_cast<std::uint64_t>(next + displacement);
for (const auto& data : segments) {
if (data.Type != kLoadSegment || table < data.MappedAddress || table - data.MappedAddress >= data.FileSize)
continue;
if (data.Offset > fileBytes.size() || data.FileSize > fileBytes.size() - data.Offset)
throw CodegenException("Segment exceeds the file", data.Offset);
for (auto entry = table; entry - data.MappedAddress + sizeof(std::int32_t) <= data.FileSize; entry += sizeof(std::int32_t)) {
std::int32_t relative;
std::memcpy(&relative, fileBytes.data() + data.Offset + (entry - data.MappedAddress), sizeof(relative));
const auto target = table + static_cast<std::uint64_t>(static_cast<std::int64_t>(relative));
if (!startsInstruction(target))
break;
targets.insert(target);
}
}
}
}
return targets;
}
@ -256,6 +289,7 @@ std::set<std::uint64_t> Amd64OnlyConverter::_collectBranchTargets(
void Amd64OnlyConverter::_convertSegment(
std::vector<std::uint8_t>& fileBytes,
const Domain::ProgramHeader& ph,
const std::vector<Domain::ProgramHeader>& segments,
ConvertResult& result
) const {
const auto segOffset = static_cast<std::size_t>(ph.Offset);
@ -283,7 +317,7 @@ void Amd64OnlyConverter::_convertSegment(
const bool needsBranchTargets = std::any_of(pending.begin(), pending.end(), [](const Pending& item) {
return item.Substitution.Lowering == Amd64OnlyLowering::Trampoline;
});
const auto branchTargets = needsBranchTargets ? _collectBranchTargets(seg, matches, ph) : std::set<std::uint64_t>{};
const auto branchTargets = needsBranchTargets ? _collectBranchTargets(seg, matches, ph, fileBytes, segments) : std::set<std::uint64_t>{};
std::set<std::size_t> pendingIndices;
for (const auto& item : pending)
@ -395,11 +429,13 @@ void Amd64OnlyConverter::_convertSegment(
ConvertResult Amd64OnlyConverter::Convert(
std::vector<std::uint8_t> fileBytes,
const std::vector<Domain::ProgramHeader>& codeSegments
const std::vector<Domain::ProgramHeader>& segments
) const {
ConvertResult result{};
for (const auto& ph : codeSegments)
_convertSegment(fileBytes, ph, result);
for (const auto& ph : segments) {
if (ph.Type == kLoadSegment && (ph.Flags & kExecutableSegment) != 0)
_convertSegment(fileBytes, ph, segments, result);
}
result.Bytes = std::move(fileBytes);
return result;
}

38
core/relinker/codegen/tests/Amd64OnlyConverterTests.cpp
View File

@ -886,6 +886,43 @@ void converterWideStoreWindows() {
requireFailure([&] { (void)convert(withStore(far, tail)); }, "A 256-bit store whose movable neighbours are all beyond a short jump was kept");
}
void converterJumpTableTargets() {
const auto converter = Codegen::MakeAmd64OnlyConverter(Codegen::Amd64OnlyTarget::Rosetta);
const Bytes leaTable = {0x48, 0x8D, 0x0D, 0xF9, 0x1F, 0x00, 0x00};
const Bytes dispatch = {0x48, 0x63, 0x04, 0x81, 0x48, 0x01, 0xC8, 0xFF, 0xE0, 0x48, 0x83, 0xC1, 0x01, 0xC5, 0xFC, 0x11, 0x07, 0x48, 0x83, 0xC2, 0x01, 0xC3};
const Bytes store = {0xC5, 0xFC, 0x11, 0x07};
const Domain::ProgramHeader code{1, 5, 0x200, 0x1000, 0, 29, 29, 16};
const Domain::ProgramHeader data{1, 4, 0x300, 0x3000, 0, 0x40, 0x40, 16};
const auto table = [](const std::uint64_t base, const std::vector<std::uint64_t>& targets) {
Bytes bytes;
for (const auto target : targets) {
const auto relative = static_cast<std::uint32_t>(target - base);
for (std::size_t byte = 0; byte < 4; ++byte)
bytes.push_back(static_cast<std::uint8_t>(relative >> (byte * 8)));
}
return bytes;
};
const auto convert = [&](const Bytes& first, const std::vector<std::pair<std::size_t, Bytes>>& placed, const std::vector<Domain::ProgramHeader>& headers) {
Bytes file(0x400, 0xCC);
std::copy(first.begin(), first.end(), file.begin() + 0x200);
std::copy(dispatch.begin(), dispatch.end(), file.begin() + 0x207);
for (const auto& [offset, bytes] : placed)
std::copy(bytes.begin(), bytes.end(), file.begin() + static_cast<std::ptrdiff_t>(offset));
return converter->Convert(file, headers);
};
const auto placedAt = [](const Codegen::ConvertResult& result, const Domain::FileByteOffset offset) {
return result.Trampolines.size() == 1 && result.Trampolines[0].Offset == offset && result.Trampolines[0].Length == 8;
};
require(placedAt(convert(leaTable, {{0x300, table(0x3000, {0x1010, 0x1018})}, {0x330, store}}, {code, data}), 0x210), "A 256-bit store moved the instruction after it although a jump table enters that instruction");
require(placedAt(convert(leaTable, {{0x300, table(0x3000, {0x1010, 0x1019, 0x1018})}}, {code, data}), 0x214), "A jump table was read past an entry that does not start an instruction");
require(placedAt(convert({0x48, 0x8B, 0x0D, 0xF9, 0x1F, 0x00, 0x00}, {{0x300, table(0x3000, {0x1010, 0x1018})}}, {code, data}), 0x214), "A jump table was read at an address loaded rather than formed by LEA");
require(placedAt(convert({0x48, 0x8D, 0x0D, 0xF7, 0x1F, 0x00, 0x00}, {{0x2FE, table(0x2FFE, {0x1018})}}, {code, data}), 0x214), "A jump table was read at an address outside every segment");
require(placedAt(convert(leaTable, {{0x300, table(0x3000, {0x1010})}, {0x380, table(0x3000, {0x1018})}}, {code, data, {0x61000000, 4, 0x380, 0x3000, 0, 0x10, 0x10, 16}}), 0x214), "A jump table was read from a segment that is not loaded");
require(placedAt(convert(leaTable, {{0x300, table(0x3000, {0x1010, 0x1018})}, {0x3A0, store}}, {code, data, {2, 5, 0x3A0, 0x2000, 0, 4, 4, 16}}), 0x210), "A segment that is not loaded was converted");
requireFailure([&] { (void)convert(leaTable, {{0x300, table(0x3000, {0x1010, 0x1018})}}, {code, {1, 4, 0x300, 0x3000, 0, 0x1000, 0x1000, 16}}); }, "A jump table was read from a segment that exceeds the file");
}
void converterRosetta() {
const auto converter = Codegen::MakeAmd64OnlyConverter(Codegen::Amd64OnlyTarget::Rosetta);
Bytes file(0x300, 0xCC);
@ -1637,6 +1674,7 @@ int main() {
converterRosetta();
wideStoreSplit();
converterWideStoreWindows();
converterJumpTableTargets();
converterReciprocal();
converterStrayRex();
rewriterStrayRex();

1
core/relinker/elfpatcher/include/elfpatcher/general/ElfConstants.hpp
View File

@ -141,6 +141,7 @@ inline constexpr std::uint32_t R_X86_64_REX_GOTPCRELX = 42;
inline constexpr std::uint8_t STB_GLOBAL = 1;
inline constexpr std::uint8_t STB_WEAK = 2;
inline constexpr std::uint8_t STT_NOTYPE = 0;
inline constexpr std::uint8_t STT_FUNC = 2;
inline constexpr std::uint8_t STV_DEFAULT = 0;
inline constexpr std::uint16_t SHN_UNDEF = 0;

7
core/relinker/elfpatcher/src/windows/WindowsRelocationBuilder.cpp
View File

@ -1,4 +1,5 @@
#include <elfpatcher/windows/WindowsRelocationBuilder.hpp>
#include <elfpatcher/general/ElfConstants.hpp>
#include <io/BufferUtils.hpp>
#include <map>
@ -38,11 +39,15 @@ PeRelocations WindowsRelocationBuilder::Apply(WindowsLoadImage& image, const Dom
if (Io::ReadU16(dynamicSection.DynSymData, symbolOffset + 6) != 0)
throw Domain::RelinkerException("Defined symbols require explicit relocation support", target);
const auto name = ReadString(dynamicSection.DynStrData, Io::ReadU32(dynamicSection.DynSymData, symbolOffset));
if (name.empty())
if (name.empty() && dynamicSection.DynSymData[symbolOffset + 4] >> 4 != STB_WEAK)
throw Domain::RelinkerException("Empty import symbol name", target);
if (type != 1 && addend != 0)
throw Domain::RelinkerException("GLOB_DAT/JUMP_SLOT relocation has a nonzero addend", target);
image.RequireWritable(target, 8);
if (name.empty()) {
image.WritePointer(target, addend);
continue;
}
image.WritePointer(target, 0);
result.Imports.push_back({name, rva, addend});
if (const auto module = dynamicSection.ImportModules.find(target); module != dynamicSection.ImportModules.end())

4
core/relinker/main.cpp
View File

@ -66,8 +66,8 @@ int Run(const int argc, char* argv[]) {
std::vector<Codegen::TrampolineSite> trampolines;
if (args.toIntel) {
const auto codeSegments = Relinker::ElfReader(sourceBytes).ReadCodeSegments();
auto converted = Codegen::MakeAmd64OnlyConverter(args.toRosetta ? Codegen::Amd64OnlyTarget::Rosetta : Codegen::Amd64OnlyTarget::Intel)->Convert(std::move(sourceBytes), codeSegments);
const auto segments = Relinker::ElfReader(sourceBytes).ReadProgramHeaders();
auto converted = Codegen::MakeAmd64OnlyConverter(args.toRosetta ? Codegen::Amd64OnlyTarget::Rosetta : Codegen::Amd64OnlyTarget::Intel)->Convert(std::move(sourceBytes), segments);
sourceBytes = std::move(converted.Bytes);
trampolines = std::move(converted.Trampolines);
std::map<std::string, std::size_t> stubsByName;

1
core/relinker/relinker/include/relinker/domain/IElfReader.hpp
View File

@ -18,7 +18,6 @@ public:
[[nodiscard]] virtual FileByteOffset TranslateVirtualAddress(VirtualAddress address) const = 0;
[[nodiscard]] virtual std::vector<std::uint8_t> ReadSection(const SectionHeader& header) const = 0;
[[nodiscard]] virtual std::vector<std::uint8_t> ReadSegment(const ProgramHeader& header) const = 0;
[[nodiscard]] virtual std::vector<ProgramHeader> ReadCodeSegments() const = 0;
[[nodiscard]] virtual std::uint64_t GetFileSize() const = 0;
[[nodiscard]] virtual const std::vector<std::uint8_t>& GetRawBytes() const = 0;
};

1
core/relinker/relinker/include/relinker/parsing/ElfReader.hpp
View File

@ -17,7 +17,6 @@ public:
FileByteOffset TranslateVirtualAddress(VirtualAddress address) const override;
std::vector<std::uint8_t> ReadSection(const SectionHeader& header) const override;
std::vector<std::uint8_t> ReadSegment(const ProgramHeader& header) const override;
std::vector<ProgramHeader> ReadCodeSegments() const override;
std::uint64_t GetFileSize() const override;
const std::vector<std::uint8_t>& GetRawBytes() const override;

42
core/relinker/relinker/src/analysis/SyscallScanner.cpp
View File

@ -50,6 +50,45 @@ enum class ForbiddenInstruction {
Return,
};
static constexpr std::array<std::uint8_t, 62> SDK_MPROTECT_INLINE{
0x55, 0x48, 0x89, 0xE5, 0x53, 0x50, 0x89, 0xD1, 0x49, 0x89, 0xF0, 0x48, 0x89, 0xFE,
0xF6, 0xC1, 0x04, 0x75, 0x31, 0x48, 0x63, 0xD1, 0xB8, 0x4A, 0x00, 0x00, 0x00,
0x48, 0x89, 0xF7, 0x4C, 0x89, 0xC6, 0x45, 0x31, 0xD2, 0x45, 0x31, 0xC0, 0x45, 0x31, 0xC9,
0x0F, 0x05, 0x0F, 0x93, 0xC1, 0x48, 0x89, 0xC3, 0x48, 0xF7, 0xDB, 0x84, 0xC9,
0x48, 0x0F, 0x45, 0xD8, 0x48, 0x85, 0xDB,
};
static constexpr std::array<std::uint8_t, 37> SDK_MMAP_INLINE{
0x49, 0x63, 0xD6, 0x4C, 0x63, 0xD1, 0x4D, 0x63, 0xC0, 0xB8, 0xDD, 0x01, 0x00, 0x00,
0x48, 0x89, 0xDE, 0x0F, 0x05, 0x48, 0x89, 0xC1, 0x0F, 0x93, 0xC2, 0x48, 0xF7, 0xD8,
0x84, 0xD2, 0x48, 0x0F, 0x45, 0xC1, 0x48, 0x85, 0xC0,
};
static constexpr std::array<std::uint8_t, 43> SDK_MMAP_EXEC_INLINE{
0x44, 0x89, 0xF0, 0x83, 0xE0, 0xFB, 0x48, 0x63, 0xD0, 0x4C, 0x63, 0xD1, 0x4D, 0x63, 0xC0,
0xB8, 0xDD, 0x01, 0x00, 0x00, 0x48, 0x89, 0xDE, 0x0F, 0x05, 0x0F, 0x93, 0xC1,
0x49, 0x89, 0xC7, 0x49, 0xF7, 0xDF, 0x84, 0xC9, 0x4C, 0x0F, 0x45, 0xF8, 0x4D, 0x85, 0xFF,
};
template<std::size_t TSize>
static bool MatchesSdkInlineStub(const std::vector<std::uint8_t>& code, std::size_t syscallOffset, std::size_t limit,
const std::array<std::uint8_t, TSize>& pattern, std::size_t beforeSyscall,
const std::vector<Codegen::InstructionMatch>& matches, FileByteOffset codeOffset) {
if (syscallOffset < beforeSyscall || syscallOffset > limit) return false;
const auto start = syscallOffset - beforeSyscall;
if (pattern.size() > limit - start || !std::equal(pattern.begin(), pattern.end(), code.begin() + static_cast<std::ptrdiff_t>(start))) return false;
const auto instruction = std::lower_bound(matches.begin(), matches.end(), codeOffset + start,
[](const Codegen::InstructionMatch& match, FileByteOffset offset) { return match.Offset < offset; });
return instruction != matches.end() && instruction->Offset == codeOffset + start;
}
static bool IsSdkMmanSyscallStub(const std::vector<std::uint8_t>& code, std::size_t syscallOffset, std::size_t limit,
const std::vector<Codegen::InstructionMatch>& matches, FileByteOffset codeOffset) {
return MatchesSdkInlineStub(code, syscallOffset, limit, SDK_MPROTECT_INLINE, 42, matches, codeOffset)
|| MatchesSdkInlineStub(code, syscallOffset, limit, SDK_MMAP_INLINE, 17, matches, codeOffset)
|| MatchesSdkInlineStub(code, syscallOffset, limit, SDK_MMAP_EXEC_INLINE, 23, matches, codeOffset);
}
static ForbiddenInstruction ClassifyForbiddenInstruction(
const std::vector<std::uint8_t>& codeSection,
const std::size_t opcodeOffset,
@ -158,7 +197,8 @@ std::vector<std::size_t> SyscallScanner::ScanCodeSectionForSyscalls(
if (kind == ForbiddenInstruction::None)
continue;
if (_trapSdkStubs && kind == ForbiddenInstruction::Invocation && IsSdkSyscallStub(codeSection, opcode)) {
if (_trapSdkStubs && kind == ForbiddenInstruction::Invocation &&
(IsSdkSyscallStub(codeSection, opcode) || IsSdkMmanSyscallStub(codeSection, opcode, limit, matches, codeSectionOffset))) {
trapped.push_back(opcode);
std::cout << "Trapping SDK syscall instruction at code offset 0x" << std::hex << match.Offset << std::dec << '\n';
continue;

38
core/relinker/relinker/src/guest/GuestModuleBuilder.cpp
View File

@ -16,6 +16,7 @@
#include <sstream>
#include <string>
#include <string_view>
#include <unordered_map>
#include <utility>
#include <vector>
@ -23,6 +24,11 @@ namespace Relinker {
namespace {
struct GuestExport {
std::size_t Provider;
std::uint8_t Type;
};
std::vector<std::string> ReadNeededNames(const Domain::SysVDynamicSection& dynamic) {
if (dynamic.DynamicSegmentData.size() % 16 != 0) throw Domain::RelinkerException("Invalid executable dependency table");
std::vector<std::string> names;
@ -136,7 +142,7 @@ std::vector<GuestArtifact> GuestModuleBuilder::Build(const std::filesystem::path
if (std::filesystem::equivalent(path, outputPath))
throw Domain::RelinkerException("Executable output would overwrite an input module: " + Io::Utf8Path(outputPath));
}
std::map<std::string, std::vector<std::size_t>> exports;
std::unordered_map<std::string, std::vector<GuestExport>> exports;
std::map<std::string, std::set<std::size_t>> sharedExports;
std::set<std::string> outputNames;
for (const auto& path : paths) {
@ -153,10 +159,11 @@ std::vector<GuestArtifact> GuestModuleBuilder::Build(const std::filesystem::path
for (const auto& symbol : image.Symbols) {
if (symbol.Section == 0 || symbol.Section == AbsoluteSection || (symbol.Info >> 4) == 0 || symbol.Visibility == 1 || symbol.Visibility == 2) continue;
auto& providers = exports[symbol.Name];
const bool repeated = std::find(providers.begin(), providers.end(), images.size()) != providers.end();
if (!windows && repeated) throw Domain::RelinkerException("Duplicate guest export after stripping #: " + symbol.Name + " in " + Io::Utf8Path(path) + " and " + Io::Utf8Path(images[providers.front()].SourcePath));
if (!repeated) providers.push_back(images.size());
if (!windows && providers.size() > 1) sharedExports[symbol.Name].insert(providers.begin(), providers.end());
const bool repeated = !providers.empty() && providers.back().Provider == images.size();
if (!windows && repeated) throw Domain::RelinkerException("Duplicate guest export after stripping #: " + symbol.Name + " in " + Io::Utf8Path(path) + " and " + Io::Utf8Path(images[providers.front().Provider].SourcePath));
if (!repeated) providers.push_back({images.size(), static_cast<std::uint8_t>(symbol.Info & 15)});
if (!windows && providers.size() > 1)
for (const auto& provider : providers) sharedExports[symbol.Name].insert(provider.Provider);
}
images.push_back(std::move(image));
}
@ -264,20 +271,19 @@ std::vector<GuestArtifact> GuestModuleBuilder::Build(const std::filesystem::path
if (shared == sharedExports.end() || declared == guestNames.end() || !shared->second.contains(declared->second))
rejectSharedImport(symbol.Name, Io::Utf8Path(images[index].SourcePath));
const auto found = exports.find(symbol.Name);
std::vector<std::size_t> providers;
const GuestExport* provider = nullptr;
if (found != exports.end()) {
for (const auto provider : found->second) {
if (symbol.Library.empty() || (declared != guestNames.end() && declared->second == provider)) providers.push_back(provider);
for (const auto& candidate : found->second) {
if (!symbol.Library.empty() && (declared == guestNames.end() || declared->second != candidate.Provider)) continue;
if (provider != nullptr) throw Domain::RelinkerException("Ambiguous guest import after stripping #: " + symbol.Name);
provider = &candidate;
}
}
if (providers.size() > 1) throw Domain::RelinkerException("Ambiguous guest import after stripping #: " + symbol.Name);
if (!providers.empty()) {
const auto& provider = images[providers.front()];
const auto exported = std::find_if(provider.Symbols.begin(), provider.Symbols.end(), [&](const auto& candidate) { return candidate.Section != 0 && candidate.Section != AbsoluteSection && candidate.Name == symbol.Name && (candidate.Info >> 4) != 0 && candidate.Visibility != 1 && candidate.Visibility != 2; });
if (exported == provider.Symbols.end() || ((symbol.Info & 15) != 0 && (symbol.Info & 15) != (exported->Info & 15))) throw Domain::RelinkerException("Guest import/export type mismatch: " + symbol.Name);
if (providers.front() != index) dependencies[index].insert(providers.front());
if (provider != nullptr) {
if ((symbol.Info & 15) != 0 && (symbol.Info & 15) != provider->Type) throw Domain::RelinkerException("Guest import/export type mismatch: " + symbol.Name);
if (provider->Provider != index) dependencies[index].insert(provider->Provider);
} else if (windows && (symbol.Info & 15) == 6) throw Domain::RelinkerException("Windows guest TLS import requires a guest TLS export: " + symbol.Name);
if (providers.empty()) systemImports[index].insert(symbolIndex);
if (provider == nullptr) systemImports[index].insert(symbolIndex);
}
}
std::vector<bool> required(images.size());
@ -342,7 +348,7 @@ std::vector<GuestArtifact> GuestModuleBuilder::Build(const std::filesystem::path
std::vector<Domain::ProgramHeader> codeHeaders;
for (const auto& header : image.Headers) if (header.Type == 1 && (header.Flags & 1) != 0) codeHeaders.push_back(header);
if (toIntel) {
auto converted = Codegen::MakeAmd64OnlyConverter(toRosetta ? Codegen::Amd64OnlyTarget::Rosetta : Codegen::Amd64OnlyTarget::Intel)->Convert(std::move(image.Bytes), codeHeaders);
auto converted = Codegen::MakeAmd64OnlyConverter(toRosetta ? Codegen::Amd64OnlyTarget::Rosetta : Codegen::Amd64OnlyTarget::Intel)->Convert(std::move(image.Bytes), image.Headers);
image.Trampolines = std::move(converted.Trampolines);
image.Bytes = std::move(converted.Bytes);
}

11
core/relinker/relinker/src/output/SysVDynamicSectionBuilder.cpp
View File

@ -82,6 +82,11 @@ SysVDynamicSection SysVDynamicSectionBuilder::BuildDynamicSection(
return value.substr(0, hashPos);
};
auto symbolInfo = [](const NidReference& ref) -> std::uint8_t {
if (ref.Nid.empty()) return static_cast<std::uint8_t>((STB_WEAK << 4) | STT_NOTYPE);
return static_cast<std::uint8_t>((STB_GLOBAL << 4) | STT_FUNC);
};
static constexpr std::size_t kRelaEntSize = 24;
std::vector<const NidReference*> pltSlots(originalJmprelCount, nullptr);
@ -132,8 +137,7 @@ SysVDynamicSection SysVDynamicSectionBuilder::BuildDynamicSection(
const NidReference& ref = *slot;
const std::uint32_t nameOff = _appendStr(result.DynStrData, stripHashSuffix(ref.Nid));
const auto info = static_cast<std::uint8_t>((STB_GLOBAL << 4) | STT_FUNC);
_appendElfSym(result.DynSymData, nameOff, info, STV_DEFAULT, 0, 0, 0);
_appendElfSym(result.DynSymData, nameOff, symbolInfo(ref), STV_DEFAULT, 0, 0, 0);
const std::uint64_t relaInfo = (static_cast<std::uint64_t>(symIdx) << 32) | R_X86_64_JUMP_SLOT;
_appendRela(result.RelaPltData, ref.RelocationAddress, relaInfo, ref.Addend);
@ -143,8 +147,7 @@ SysVDynamicSection SysVDynamicSectionBuilder::BuildDynamicSection(
for (const NidReference* slotPtr : nonPltRefs) {
const NidReference& ref = *slotPtr;
const std::uint32_t nameOff = _appendStr(result.DynStrData, stripHashSuffix(ref.Nid));
const auto info = static_cast<std::uint8_t>((STB_GLOBAL << 4) | STT_FUNC);
_appendElfSym(result.DynSymData, nameOff, info, STV_DEFAULT, 0, 0, 0);
_appendElfSym(result.DynSymData, nameOff, symbolInfo(ref), STV_DEFAULT, 0, 0, 0);
const std::uint64_t relaInfo = (static_cast<std::uint64_t>(symIdx) << 32) | ref.RelocationTypeValue;
_appendRela(result.RelaData, ref.RelocationAddress, relaInfo, ref.Addend);

29
core/relinker/relinker/src/parsing/ElfReader.cpp
View File

@ -130,6 +130,23 @@ std::vector<ProgramHeader> ElfReader::ReadProgramHeaders() const {
offset += header.ProgramHeaderEntrySize;
}
for (auto& relro : headers) {
if (relro.Type != PT_OS_RELRO || relro.MemorySize == 0) continue;
bool covered = false;
for (const auto& load : headers) {
if (load.Type != PT_LOAD || load.MemorySize == 0) continue;
const bool overlaps = relro.MappedAddress < load.MappedAddress ? load.MappedAddress - relro.MappedAddress < relro.MemorySize : relro.MappedAddress - load.MappedAddress < load.MemorySize;
if (!overlaps) continue;
if (relro.MappedAddress < load.MappedAddress || relro.MemorySize > load.MemorySize - (relro.MappedAddress - load.MappedAddress))
throw RelinkerException("PT_SCE_RELRO partially overlaps a PT_LOAD segment", relro.MappedAddress);
covered = true;
}
if (!covered) {
relro.Type = PT_LOAD;
relro.Flags = PF_R | PF_W;
}
}
return headers;
}
@ -263,18 +280,6 @@ std::vector<std::uint8_t> ElfReader::ReadSegment(const ProgramHeader& header) co
_fileBuffer.begin() + header.Offset + header.FileSize);
}
std::vector<ProgramHeader> ElfReader::ReadCodeSegments() const {
static constexpr std::uint32_t kPtLoad = 1;
static constexpr std::uint32_t kPfX = 0x1;
std::vector<ProgramHeader> result;
for (const auto& ph : ReadProgramHeaders()) {
if (ph.Type == kPtLoad && (ph.Flags & kPfX)) {
result.push_back(ph);
}
}
return result;
}
std::uint64_t ElfReader::GetFileSize() const {
return _fileBuffer.size();
}

5
core/relinker/relinker/src/pipeline/RelinkerPipeline.cpp
View File

@ -258,6 +258,10 @@ RelinkResult RelinkerPipeline::Relink(const std::vector<std::uint8_t>& sourceElf
std::memcpy(&nameOff, raw.data() + symOff, 4);
const auto name = readCStr(nameOff);
if (name.empty() && (raw.size() - symOff < symEntSize
|| std::any_of(raw.begin() + static_cast<std::ptrdiff_t>(symOff), raw.begin() + static_cast<std::ptrdiff_t>(symOff + symEntSize),
[](const std::uint8_t byte) { return byte != 0; })))
throw RelinkerException("Unnamed import symbol is not a null symbol entry", symOff);
nidRefs.push_back({name, Domain::ImportModule(name, importModules, neededLibraries), relType, pos, rOffset, rAddend});
}
};
@ -426,6 +430,7 @@ RelinkResult RelinkerPipeline::Relink(const std::vector<std::uint8_t>& sourceElf
std::vector<CallRegistryEntry> entries;
entries.reserve(nidRefs.size());
for (const auto& ref : nidRefs) {
if (ref.Nid.empty()) continue;
CallRegistryEntry entry;
entry.Nid = ref.Nid;
entry.Library = ref.Library;

87
core/relinker/relinker/tests/SdkMmanSyscallTests.cpp Normal file
View File

@ -0,0 +1,87 @@
#include <relinker/analysis/SyscallScanner.hpp>
#include <cstdint>
#include <iostream>
#include <stdexcept>
#include <string>
#include <vector>
namespace {
using Bytes = std::vector<std::uint8_t>;
struct Fixture { Bytes code; std::size_t syscall; };
const std::vector<Fixture> Fixtures = {
{{0x55, 0x48, 0x89, 0xE5, 0x53, 0x50, 0x89, 0xD1, 0x49, 0x89, 0xF0, 0x48, 0x89, 0xFE,
0xF6, 0xC1, 0x04, 0x75, 0x31, 0x48, 0x63, 0xD1, 0xB8, 0x4A, 0x00, 0x00, 0x00,
0x48, 0x89, 0xF7, 0x4C, 0x89, 0xC6, 0x45, 0x31, 0xD2, 0x45, 0x31, 0xC0, 0x45, 0x31, 0xC9,
0x0F, 0x05, 0x0F, 0x93, 0xC1, 0x48, 0x89, 0xC3, 0x48, 0xF7, 0xDB, 0x84, 0xC9,
0x48, 0x0F, 0x45, 0xD8, 0x48, 0x85, 0xDB}, 42},
{{0x49, 0x63, 0xD6, 0x4C, 0x63, 0xD1, 0x4D, 0x63, 0xC0, 0xB8, 0xDD, 0x01, 0x00, 0x00,
0x48, 0x89, 0xDE, 0x0F, 0x05, 0x48, 0x89, 0xC1, 0x0F, 0x93, 0xC2, 0x48, 0xF7, 0xD8,
0x84, 0xD2, 0x48, 0x0F, 0x45, 0xC1, 0x48, 0x85, 0xC0}, 17},
{{0x44, 0x89, 0xF0, 0x83, 0xE0, 0xFB, 0x48, 0x63, 0xD0, 0x4C, 0x63, 0xD1, 0x4D, 0x63, 0xC0,
0xB8, 0xDD, 0x01, 0x00, 0x00, 0x48, 0x89, 0xDE, 0x0F, 0x05, 0x0F, 0x93, 0xC1,
0x49, 0x89, 0xC7, 0x49, 0xF7, 0xDF, 0x84, 0xC9, 0x4C, 0x0F, 0x45, 0xF8, 0x4D, 0x85, 0xFF}, 23},
};
void Require(bool value, const std::string& message) {
if (!value) throw std::runtime_error(message);
}
bool Rejected(const Bytes& code, bool trap, std::size_t size) {
try {
Relinker::MakeSyscallScanner(trap)->ScanCodeSectionForSyscalls(code, 0, size);
return false;
} catch (const std::runtime_error&) {
return true;
}
}
void CheckFixtures() {
std::size_t rejectedMutations = 0;
for (const auto& fixture : Fixtures) {
Require(Rejected(fixture.code, false, fixture.code.size()), "default mode accepted an SDK inline syscall");
for (const auto padding : {std::size_t{0}, std::size_t{1}, std::size_t{7}, std::size_t{64}}) {
Bytes code(padding, 0x90);
code.insert(code.end(), fixture.code.begin(), fixture.code.end());
const auto trapped = Relinker::MakeSyscallScanner(true)->ScanCodeSectionForSyscalls(code, 0x123400, code.size());
Require(trapped == std::vector<std::size_t>{padding + fixture.syscall}, "SDK inline syscall returned the wrong patch offset");
}
for (std::size_t length = fixture.syscall + 2; length < fixture.code.size(); ++length)
Require(Rejected(fixture.code, true, length), "SDK matching read its suffix beyond the code section");
for (std::size_t removed = 1; removed <= fixture.syscall; ++removed) {
const Bytes fragment(fixture.code.begin() + static_cast<std::ptrdiff_t>(removed), fixture.code.end());
if (Rejected(fragment, false, fragment.size()))
Require(Rejected(fragment, true, fragment.size()), "SDK matching accepted a truncated prefix");
}
for (std::size_t offset = 0; offset < fixture.code.size(); ++offset) {
if (offset == fixture.syscall || offset == fixture.syscall + 1) continue;
for (unsigned bit = 0; bit < 8; ++bit) {
auto code = fixture.code;
code[offset] ^= static_cast<std::uint8_t>(1u << bit);
if (!Rejected(code, false, code.size())) continue;
Require(Rejected(code, true, code.size()), "SDK matching accepted a changed instruction sequence");
++rejectedMutations;
}
}
}
Require(rejectedMutations > 1000, "too few near-match rejection cases ran");
Bytes embedded = {0x48, 0xB8};
embedded.insert(embedded.end(), Fixtures.front().code.begin(), Fixtures.front().code.end());
Require(Rejected(embedded, false, embedded.size()), "embedded fixture lost its syscall instruction");
Require(Rejected(embedded, true, embedded.size()), "SDK matching accepted a prefix that starts inside an immediate");
for (const auto& code : std::vector<Bytes>{{0xB8, 0xDD, 0x01, 0x00, 0x00, 0x0F, 0x05, 0xC3}, {0xCD, 0x80}, {0x0F, 0x34}, {0x0F, 0x07}})
Require(Rejected(code, true, code.size()), "SDK trap mode accepted an unrelated privileged instruction");
std::cout << "SDK inline syscall checks passed, " << rejectedMutations << " near-match rejections\n";
}
}
int main() {
try {
CheckFixtures();
} catch (const std::exception& error) {
std::cerr << error.what() << '\n';
return 1;
}
}

128
core/relinker/relinker/tests/test_guest_export_lookup.py Normal file
View File

@ -0,0 +1,128 @@
from pathlib import Path
import os
import struct
import subprocess
import sys
import tempfile
from test_guest_intel_trampolines import PLAIN_SITE, guest_fixture, main_fixture
from test_guest_module_directories import needed_libraries
from test_guest_symbol_names import imported_symbols, module_symbols
from test_windows_import_modules import executable
def guest_symbols(symbols, dependency=None):
image = guest_fixture(PLAIN_SITE)
image[0x400:0x406] = b'\xb8\x2a\x00\x00\x00\xc3'
strings = bytearray(b'\0')
table = bytearray(24)
for name, info, visibility, section in symbols:
offset = len(strings)
strings.extend(name.encode() + b'\0')
table.extend(struct.pack('<IBBHQQ', offset, info, visibility, section,
0x1000 if section else 0, 6 if section else 0))
dependency_offset = len(strings)
if dependency:
strings.extend(dependency.encode() + b'\0')
string_offset = 0x800
symbol_offset = (string_offset + len(strings) + 7) & ~7
hash_offset = symbol_offset + len(table)
end = hash_offset + 8 + 4 * (1 + len(table) // 24)
image.extend(bytes(max(0, end - len(image))))
image[string_offset:string_offset + len(strings)] = strings
image[symbol_offset:symbol_offset + len(table)] = table
image[hash_offset:end] = bytes(end - hash_offset)
struct.pack_into('<II', image, hash_offset, 1, len(table) // 24)
address = lambda offset: offset + 0x1a00
tags = [(5, address(string_offset)), (10, len(strings)),
(6, address(symbol_offset)), (11, 24), (4, address(hash_offset))]
if dependency:
tags.append((1, dependency_offset))
tags.append((0, 0))
image[0x600:0x800] = bytes(0x200)
for index, tag in enumerate(tags):
struct.pack_into('<qQ', image, 0x600 + index * 16, *tag)
struct.pack_into('<QQ', image, 120 + 32, end - 0x600, end - 0x600)
struct.pack_into('<QQ', image, 176 + 32, len(tags) * 16, len(tags) * 16)
return image
def large_modules(count):
names = [f'entry{index:06d}' for index in range(count)]
return {
'provider.prx': guest_symbols((name, 0x12, 0, 1) for name in names),
'consumer.prx': guest_symbols(((name, 0x12, 0, 0) for name in reversed(names)),
'provider.prx'),
}
def convert(relinker, case, windows, modules, source=None):
directory = case / 'sce_module'
directory.mkdir(parents=True)
for name, image in modules.items():
(directory / name).write_bytes(image)
input_path = case / 'input.elf'
input_path.write_bytes(main_fixture() if source is None else source)
output = case / ('output.exe' if windows else 'output.elf')
result = subprocess.run([str(relinker), *(['--windows'] if windows else []),
str(input_path), str(output)], capture_output=True,
text=True, timeout=30)
return result, output
def main():
relinker = Path(sys.argv[1]).resolve()
cases = 0
with tempfile.TemporaryDirectory(prefix='anyps5-export-lookup-') as directory:
work = Path(directory)
for windows in (False, True):
modules = large_modules(4096)
result, output = convert(relinker, work / f'large-{windows}', windows, modules,
executable('provider.prx', symbol='entry004095'))
assert result.returncode == 0, (result.stdout, result.stderr)
provider = output.parent / 'app0/sce_module/provider.prx.guest.prx'
consumer = output.parent / 'app0/sce_module/consumer.prx.guest.prx'
assert provider.is_file() and consumer.is_file()
if not windows:
names = {f'entry{index:06d}#guest' for index in range(4096)}
assert set(module_symbols(provider.read_bytes())) == names
assert set(module_symbols(consumer.read_bytes())) == names
assert needed_libraries(consumer.read_bytes()) == ['$ORIGIN/provider.prx.guest.prx']
assert imported_symbols(output.read_bytes()) == {'entry004095#guest'}
elif os.name == 'nt':
run = subprocess.run([str(output)], capture_output=True, text=True, timeout=30)
assert run.returncode == 42, (run.returncode, run.stdout, run.stderr)
cases += 1
for label, exports, imports, error in (
('object', [('entry', 0x11, 0, 1)], [('entry', 0x11, 0, 0)], None),
('unspecified', [('entry', 0x12, 0, 1)], [('entry', 0x10, 0, 0)], None),
('mismatch', [('entry', 0x11, 0, 1)], [('entry', 0x12, 0, 0)],
'Guest import/export type mismatch: entry'),
('filtered-first', [('entry', 0x12, 2, 1), ('entry', 0x11, 0, 1)],
[('entry', 0x11, 0, 0)], None),
('filtered-mismatch', [('entry', 0x12, 2, 1), ('entry', 0x11, 0, 1)],
[('entry', 0x12, 0, 0)], 'Guest import/export type mismatch: entry'),
('internal-first', [('entry', 0x12, 1, 1), ('entry', 0x11, 0, 1)],
[('entry', 0x11, 0, 0)], None),
('local-first', [('entry', 0x02, 0, 1), ('entry', 0x11, 0, 1)],
[('entry', 0x11, 0, 0)], None),
('duplicate-first', [('entry#A#A', 0x12, 0, 1), ('entry#B#A', 0x12, 0, 1)],
[('entry', 0x12, 0, 0)], None if windows else 'Duplicate or empty export after stripping #: entry'),
('duplicate-mismatch', [('entry#A#A', 0x12, 0, 1), ('entry#B#A', 0x12, 0, 1)],
[('entry', 0x11, 0, 0)], 'Guest import/export type mismatch: entry' if windows else 'Duplicate or empty export after stripping #: entry'),
('weak', [('entry', 0x22, 0, 1)], [('entry', 0x12, 0, 0)], None),
('protected', [('entry', 0x12, 3, 1)], [('entry', 0x12, 0, 0)], None)):
result, _ = convert(relinker, work / f'{label}-{windows}', windows, {
'provider.prx': guest_symbols(exports),
'consumer.prx': guest_symbols(imports, 'provider.prx'),
})
if error:
assert result.returncode != 0 and error in result.stderr, (result.stdout, result.stderr)
else:
assert result.returncode == 0, (result.stdout, result.stderr)
cases += 1
print(f'{cases} guest export lookup cases passed')
if __name__ == '__main__':
main()

60
core/relinker/relinker/tests/test_guest_syscall_trap.py
View File

@ -4,38 +4,60 @@ import subprocess
import sys
import tempfile
from test_guest_intel_trampolines import elf_bytes_at, elf_loads, guest_fixture, main_fixture
from test_guest_intel_trampolines import elf_bytes_at, elf_loads, guest_fixture, main_fixture, pe_bytes_at, pe_sections
GENERIC_STUB = bytes.fromhex(
"48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 "
"4c 8b 4c 24 08 0f 05 c3")
NUMBERED_STUB = bytes.fromhex("48 c7 c0 09 00 00 00 49 89 ca 0f 05 c3")
MPROTECT_INLINE = bytes.fromhex(
"55 48 89 e5 53 50 89 d1 49 89 f0 48 89 fe f6 c1 04 75 31 48 63 d1 "
"b8 4a 00 00 00 48 89 f7 4c 89 c6 45 31 d2 45 31 c0 45 31 c9 "
"0f 05 0f 93 c1 48 89 c3 48 f7 db 84 c9 48 0f 45 d8 48 85 db")
MMAP_INLINE = bytes.fromhex(
"49 63 d6 4c 63 d1 4d 63 c0 b8 dd 01 00 00 48 89 de "
"0f 05 48 89 c1 0f 93 c2 48 f7 d8 84 d2 48 0f 45 c1 48 85 c0")
MMAP_EXEC_INLINE = bytes.fromhex(
"44 89 f0 83 e0 fb 48 63 d0 4c 63 d1 4d 63 c0 b8 dd 01 00 00 48 89 de "
"0f 05 0f 93 c1 49 89 c7 49 f7 df 84 c9 4c 0f 45 f8 4d 85 ff")
def main():
relinker = Path(sys.argv[1]).resolve()
with tempfile.TemporaryDirectory(prefix="anyps5-guest-syscall-trap-") as directory:
work = Path(directory)
modules = work / "sce_module"
modules.mkdir()
module = bytearray(guest_fixture(b"\x90" * 6))
module[0x400:0x400 + len(GENERIC_STUB)] = GENERIC_STUB
module[0x440:0x440 + len(NUMBERED_STUB)] = NUMBERED_STUB
(modules / "sample.prx").write_bytes(module)
source = work / "input.elf"
source.write_bytes(main_fixture())
output = work / "output.elf"
result = subprocess.run(
[str(relinker), "--trap-sdk-syscall-stubs", str(source), str(output)],
capture_output=True, text=True, timeout=30)
assert result.returncode == 0, (result.stdout, result.stderr)
for windows in (False, True):
case = work / ("windows" if windows else "linux")
modules = case / "sce_module"
modules.mkdir(parents=True)
module = bytearray(guest_fixture(b"\x90" * 6))
for offset, stub in ((0x400, GENERIC_STUB), (0x440, NUMBERED_STUB),
(0x450, MPROTECT_INLINE), (0x490, MMAP_INLINE), (0x4b8, MMAP_EXEC_INLINE)):
module[offset:offset + len(stub)] = stub
(modules / "sample.prx").write_bytes(module)
source = case / "input.elf"
source.write_bytes(main_fixture())
output = case / ("output.exe" if windows else "output.elf")
options = ["--windows"] if windows else []
result = subprocess.run(
[str(relinker), *options, "--trap-sdk-syscall-stubs", str(source), str(output)],
capture_output=True, text=True, timeout=30)
assert result.returncode == 0, (windows, result.stdout, result.stderr)
converted = output.parent / "app0" / "sce_module" / "sample.prx.guest.prx"
image = converted.read_bytes()
loads = elf_loads(image)
assert elf_bytes_at(image, loads, 0x1017, 3) == b"\x0f\x0b\xc3"
assert elf_bytes_at(image, loads, 0x104a, 3) == b"\x0f\x0b\xc3"
converted = output.parent / "app0" / "sce_module" / "sample.prx.guest.prx"
image = converted.read_bytes()
if windows:
sections = pe_sections(image)
base = next(section[1] for section in sections if section[0] == b".elf0")
converted_code = pe_bytes_at(image, sections, base, 0x100)
else:
converted_code = elf_bytes_at(image, elf_loads(image), 0x1000, 0x100)
expected = bytearray(module[0x400:0x500])
for offset in (0x17, 0x4a, 0x7a, 0xa1, 0xcf):
assert expected[offset:offset + 2] == b"\x0f\x05"
expected[offset:offset + 2] = b"\x0f\x0b"
assert converted_code == expected, (windows, converted_code.hex(), expected.hex())
if __name__ == "__main__":

87
core/relinker/relinker/tests/test_null_symbol_relocations.py Normal file
View File

@ -0,0 +1,87 @@
import json
import os
from pathlib import Path
import struct
import subprocess
import sys
import tempfile
from test_dynamic_segment import fixture
from test_linux_dynamic_strings import dynamic_strings, file_offset
from test_linux_load_alignment import loads
GLOB_DAT_SLOT = 0x308
ABSOLUTE_SLOT = 0x310
ADDEND = 0x1234
CHECK_SLOTS = bytes.fromhex(
"488b0501010000" "4885c0" "751a"
"488b0dfd000000" "4881f934120000" "750a"
"b82a000000c3" "cccccccc" "b807000000c3")
TARGETS = (("linux", [], b"\x7fELF"), ("windows", ["--windows"], b"MZ"), ("macos", ["--macos"], b"\xcf\xfa\xed\xfe"))
def image_with(symbol=b""):
image = fixture()
relocations = [(GLOB_DAT_SLOT, (1 << 32) | 6, 0), (ABSOLUTE_SLOT, (1 << 32) | 1, ADDEND)]
struct.pack_into("<Q", image, 0x458, len(relocations) * 24)
for index, relocation in enumerate(relocations):
struct.pack_into("<QQq", image, 0x700 + index * 24, *relocation)
image[0x200:0x200 + len(CHECK_SLOTS)] = CHECK_SLOTS
struct.pack_into("<QQ", image, GLOB_DAT_SLOT, 0x5A5A5A5A5A5A5A5A, 0x5A5A5A5A5A5A5A5A)
image[0x638:0x638 + len(symbol)] = symbol
struct.pack_into("<qQ", image, 0x470, 0x6100003f, 48)
struct.pack_into("<QQ", image, 152, 0x90, 0x90)
return image
def linux_imports(data):
tags, strings = dynamic_strings(data)
segments = loads(data)
symbols = file_offset(data, segments, tags[6])
relocations = file_offset(data, segments, tags[7])
imports = []
for offset in range(relocations, relocations + tags[8], 24):
target, info, addend = struct.unpack_from("<QQq", data, offset)
if info & 0xffffffff == 8:
continue
name, bind_type, _, section, value, size = struct.unpack_from("<IBBHQQ", data, symbols + (info >> 32) * 24)
imports.append((target, info & 0xffffffff, addend, strings[name:strings.index(b"\0", name)], bind_type, section, value, size))
return sorted(imports)
def main():
relinker = Path(sys.argv[1]).resolve()
with tempfile.TemporaryDirectory(prefix="anyps5-null-symbol-") as directory:
work = Path(directory)
source = work / "null-entry.elf"
source.write_bytes(image_with())
for target, options, magic in TARGETS:
for level in (0, 1):
output = work / f"null-entry-{target}-{level}.out"
result = subprocess.run([str(relinker), "--skip-sce-module", "--registry", f"unused-filter={level}",
*options, str(source), str(output)], capture_output=True, text=True, timeout=20)
assert result.returncode == 0, (target, level, result.stdout, result.stderr)
data = output.read_bytes()
assert data.startswith(magic), target
registry = json.loads(output.with_suffix(".registry.json").read_text(encoding="utf-8"))
assert registry == [], (target, level, registry)
if target == "linux":
assert linux_imports(data) == [(GLOB_DAT_SLOT, 6, 0, b"", 0x20, 0, 0, 0),
(ABSOLUTE_SLOT, 1, ADDEND, b"", 0x20, 0, 0, 0)], level
if target == "windows" and os.name == "nt":
run = subprocess.run([str(output)], capture_output=True, timeout=20)
assert run.returncode == 42, (level, run.returncode, run.stderr)
source = work / "named-unnamed.elf"
source.write_bytes(image_with(struct.pack("<IBBHQQ", 0, 0x12, 0, 0, 0, 0)))
for target, options, _ in TARGETS:
output = work / f"named-unnamed-{target}.out"
result = subprocess.run([str(relinker), "--skip-sce-module", *options, str(source), str(output)],
capture_output=True, text=True, timeout=20)
assert result.returncode == 2 and "Unnamed import symbol is not a null symbol entry" in result.stderr, (target, result.stderr)
assert not output.exists()
print("Null symbol relocation tests passed")
if __name__ == "__main__":
main()

74
core/relinker/relinker/tests/test_sce_relro.py Normal file
View File

@ -0,0 +1,74 @@
import os
from pathlib import Path
import struct
import subprocess
import sys
import tempfile
from test_linux_dynamic_strings import dynamic_strings, file_offset
from test_linux_load_alignment import loads
PT_LOAD = 1
PT_DYNAMIC = 2
PT_SCE_RELRO = 0x61000010
PT_SCE_VERSION = 0x6FFFFF01
def fixture(extra):
image = bytearray(0x3000)
image[:16] = b"\x7fELF\x02\x01\x01" + bytes(9)
headers = [(PT_LOAD, 5, 0, 0, 0x1000, 0x1000, 0x1000), *extra]
tags = [(5, 0x600), (10, 1), (6, 0x620), (11, 24), (7, 0x700), (8, 24), (9, 24), (0, 0)]
headers.append((PT_DYNAMIC, 6, 0x400, 0x400, len(tags) * 16, len(tags) * 16, 8))
headers += [(PT_SCE_VERSION, 0, 0, 0, 0, 0, 1)] * (7 - len(headers))
struct.pack_into("<HHIQQQIHHHHHH", image, 16, 3, 62, 1, 0x200, 64, 0, 0, 64, 56, len(headers), 64, 0, 0)
for index, (kind, flags, offset, address, file_size, memory_size, align) in enumerate(headers):
struct.pack_into("<IIQQQQQQ", image, 64 + index * 56, kind, flags, offset, address, address, file_size, memory_size, align)
image[0x200:0x206] = b"\xff\x25\xfa\x0d\x00\x00"
image[0x210:0x216] = b"\xb8\x2a\x00\x00\x00\xc3"
for index, tag in enumerate(tags):
struct.pack_into("<qQ", image, 0x400 + index * 16, *tag)
struct.pack_into("<QQq", image, 0x700, 0x1000, 8, 0x210)
return image
def relink(relinker, work, name, image, options):
source = work / f"{name}.elf"
output = work / f"{name}.out"
source.write_bytes(image)
result = subprocess.run([str(relinker), "--skip-sce-module", *options, str(source), str(output)],
capture_output=True, text=True, timeout=20)
return result, output
def main():
relinker = Path(sys.argv[1]).resolve()
standalone = [(PT_SCE_RELRO, 4, 0x1000, 0x1000, 0x1000, 0x1000, 0x1000)]
covered = [(PT_LOAD, 6, 0x1000, 0x1000, 0x1000, 0x1000, 0x1000), (PT_SCE_RELRO, 4, 0x1000, 0x1000, 0x1000, 0x1000, 0x1000)]
with tempfile.TemporaryDirectory(prefix="anyps5-sce-relro-") as directory:
work = Path(directory)
for name, extra in (("standalone", standalone), ("covered", covered)):
for target, options in (("linux", []), ("windows", ["--windows"]), ("macos", ["--macos"])):
result, output = relink(relinker, work, f"{name}-{target}", fixture(extra), options)
assert result.returncode == 0, (name, target, result.stdout, result.stderr)
if target == "linux":
data = output.read_bytes()
relro = [load for load in loads(data) if load[3] <= 0x1000 < load[3] + load[6]]
assert len(relro) == 1 and relro[0][1] == 6 and relro[0][3] == 0x1000 and relro[0][5:7] == (0x1000, 0x1000), (name, relro)
tags, _ = dynamic_strings(data)
start = file_offset(data, loads(data), tags[7])
assert [struct.unpack_from("<QQq", data, offset) for offset in range(start, start + tags[8], 24)] == [(0x1000, 8, 0x210)], name
if target == "windows" and os.name == "nt":
run = subprocess.run([str(output)], capture_output=True, timeout=20)
assert run.returncode == 42, (name, run.returncode, run.stderr)
for name, load in (("tail", (PT_LOAD, 6, 0x1000, 0x1000, 0x800, 0x800, 0x1000)),
("head", (PT_LOAD, 6, 0x1800, 0x1800, 0x800, 0x800, 0x1000))):
for options in ([], ["--windows"]):
result, output = relink(relinker, work, f"partial-{name}", fixture([load, standalone[0]]), options)
assert result.returncode == 2 and "PT_SCE_RELRO partially overlaps a PT_LOAD segment" in result.stderr, (name, result.stderr)
assert not output.exists()
print("PT_SCE_RELRO tests passed")
if __name__ == "__main__":
main()

1
core/shader/recompiler/CacheKey.hpp
View File

@ -243,6 +243,7 @@ private:
append(key, value.float64DenormPreserve);
append(key, value.registrationModules);
append(key, value.registrationWordLimit);
append(key, value.vertexSubgroups);
}
};

Some files were not shown because too many files have changed in this diff Show More