shifathosn
/
AnyPS5
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #2917 from XairRrR/feat/agc-prepare-unpredicted-variants 

feat(agc-driver): prepare shader variants registration did not predict
This commit is contained in:
Adrià 2026-10-11 01:31:32 +02:00 committed by GitHub
parent ad3369236d 1232fd6cdf
commit 2cae6cc3cd
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 60 additions and 44 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

52
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp
View File

@ -264,20 +264,24 @@ void PublishRegisteredShader(std::shared_ptr<ShaderRegistry>& registry, const st
namespace {
bool PreparedAtUse(const ShaderSnapshot& snapshot, const ShaderRecompiler::RecompileRequest& request) {
void ReportPreparedAtUse(const ShaderSnapshot& snapshot, const ShaderRecompiler::RecompileRequest& request) {
if (snapshot.codeAddress == NullPixelProgramAddress() && request.shader.stage == ShaderRecompiler::ShaderStage::Fragment && request.context.waveSize == 32u) {
APS5_LOG_ERR("The null pixel program has no wave%u artifact for this draw; preparing it at draw", request.context.waveSize);
return true;
return;
}
if (!snapshot.header.empty()) {
if (!snapshot.prepared->deferred) APS5_LOG_ERR("Shader 0x%llx stage %u has no artifact prepared at registration for the state of this draw or dispatch; preparing it at use", static_cast<unsigned long long>(request.shader.codeAddress), static_cast<std::uint32_t>(request.shader.stage));
return;
}
if (!snapshot.header.empty()) return snapshot.prepared->deferred;
if (snapshot.type != 0 || request.shader.stage != ShaderRecompiler::ShaderStage::Compute) throw std::runtime_error("AGC driver: unregistered program is not a compute shader");
APS5_LOG_ERR("Compute shader 0x%llx was not registered; preparing its artifact at dispatch", static_cast<unsigned long long>(snapshot.codeAddress));
return true;
}
}
std::shared_ptr<const ShaderRecompiler::SourceHandle> SourceHandleFor(const ShaderSnapshot& snapshot, std::size_t codeOffset, const ShaderRecompiler::RecompileRequest& request) {
require(codeOffset < snapshot.code.size(), "prepared shader code offset is outside the snapshot");
require(std::ranges::equal(request.shader.code, std::span(snapshot.code).subspan(codeOffset)), "prepared shader request does not refer to registered code");
struct PreparedKeyStorage {};
auto& key = HostThreadLocal<std::vector<std::uint64_t>, PreparedKeyStorage>();
ShaderRecompiler::BuildPreparedShaderKey(request, key);
@ -286,18 +290,10 @@ std::shared_ptr<const ShaderRecompiler::SourceHandle> SourceHandleFor(const Shad
for (const auto& entry : snapshot.prepared->entries) {
if (entry.codeOffset == codeOffset && ShaderRecompiler::MatchesPreparedShader(request, *entry.handle, key)) return entry.handle;
}
if (PreparedAtUse(snapshot, request)) {
auto handle = PrepareShaderWithDiagnostics(request);
snapshot.prepared->entries.push_back({codeOffset, handle});
return handle;
}
std::string layouts;
for (const auto& entry : snapshot.prepared->entries) {
if (entry.codeOffset != codeOffset || entry.handle == nullptr || entry.handle->artifact == nullptr) continue;
const auto& layout = entry.handle->artifact->layout;
layouts += " [" + std::to_string(layout.pushConstantOffsetBytes) + "," + std::to_string(layout.pushConstantSizeBytes) + "]";
}
throw std::runtime_error("AGC driver: prepared shader artifact is missing for the requested static ABI: address=" + std::to_string(request.shader.codeAddress) + " stage=" + std::to_string(static_cast<std::uint32_t>(request.shader.stage)) + " wave=" + std::to_string(request.context.waveSize) + " pushOffset=" + std::to_string(request.layout.pushConstantOffsetBytes) + " pushCapacity=" + std::to_string(request.layout.pushConstantSizeBytes) + " preparedLayouts=" + layouts);
ReportPreparedAtUse(snapshot, request);
auto handle = PrepareShaderWithDiagnostics(request);
snapshot.prepared->entries.push_back({codeOffset, handle});
return handle;
}
ShaderRecompiler::RectListShaders PreparedRectangle(const ShaderSnapshot& snapshot, std::uint64_t vertexId, std::uint64_t fragmentId) {
@ -337,22 +333,14 @@ ShaderRecompiler::PreparedShaderInvocation InvocationFor(const ShaderSnapshot& s
invocationRequest.shader.code = ShaderRecompiler::GetPreparedCode(*entry.handle);
if (auto invocation = ShaderRecompiler::PreparedShaderInvocation::TryCreate(invocationRequest, entry.handle, key)) return std::move(*invocation);
}
if (PreparedAtUse(snapshot, request)) {
auto handle = PrepareShaderWithDiagnostics(request);
invocationRequest = request;
invocationRequest.shader.code = ShaderRecompiler::GetPreparedCode(*handle);
auto invocation = ShaderRecompiler::PreparedShaderInvocation::TryCreate(invocationRequest, handle, key);
if (!invocation.has_value()) throw std::runtime_error("AGC driver: raw compute artifact does not match its invocation");
snapshot.prepared->entries.push_back({codeOffset, std::move(handle)});
return std::move(*invocation);
}
std::string layouts;
for (const auto& entry : snapshot.prepared->entries) {
if (entry.codeOffset != codeOffset || entry.handle == nullptr || entry.handle->artifact == nullptr) continue;
const auto& layout = entry.handle->artifact->layout;
layouts += " [" + std::to_string(layout.pushConstantOffsetBytes) + "," + std::to_string(layout.pushConstantSizeBytes) + "]";
}
throw std::runtime_error("AGC driver: prepared shader artifact is missing for the requested static ABI: address=" + std::to_string(request.shader.codeAddress) + " stage=" + std::to_string(static_cast<std::uint32_t>(request.shader.stage)) + " wave=" + std::to_string(request.context.waveSize) + " pushOffset=" + std::to_string(request.layout.pushConstantOffsetBytes) + " pushCapacity=" + std::to_string(request.layout.pushConstantSizeBytes) + " preparedLayouts=" + layouts);
ReportPreparedAtUse(snapshot, request);
auto handle = PrepareShaderWithDiagnostics(request);
invocationRequest = request;
invocationRequest.shader.code = ShaderRecompiler::GetPreparedCode(*handle);
auto invocation = ShaderRecompiler::PreparedShaderInvocation::TryCreate(invocationRequest, handle, key);
if (!invocation.has_value()) throw std::runtime_error("AGC driver: an artifact prepared at use does not match its invocation");
snapshot.prepared->entries.push_back({codeOffset, std::move(handle)});
return std::move(*invocation);
}
std::optional<ShaderRecompiler::ShaderFloatMode> RegisteredFloatMode(const ShaderSnapshot& snapshot) {
if (snapshot.registeredState == nullptr) return std::nullopt;

8
core/libs/prx/libSceAgcDriver/tests/execution/PreparedGraphics.cpp
View File

@ -231,13 +231,17 @@ void Check(AgcDriver::VulkanDevice& device, AgcDriver::Graphics::ShaderPath path
request.layout.pushConstantSizeBytes -= 4;
static_cast<void>(InvocationFor(*program.snapshot, program.codeOffset, request));
request.layout.pushConstantOffsetBytes = 2;
Reject([&] { static_cast<void>(InvocationFor(*program.snapshot, program.codeOffset, request)); }, "artifact is missing");
Reject([&] { static_cast<void>(InvocationFor(*program.snapshot, program.codeOffset, request)); }, "not dword-aligned");
request.layout = {0, 0, 0, mesh ? MeshDrawPushOffsetBytes : 128u};
if (pixel) request.context.pixel->interpolatorSettings[0] ^= 1u;
else if (tessellation) ++request.graphics->tessellation->outputControlPoints;
else if (mesh) ++request.graphics->mesh->maxVertices;
else ++request.context.userDataBaseRegister;
Reject([&] { static_cast<void>(InvocationFor(*program.snapshot, program.codeOffset, request)); }, "artifact is missing");
const auto predicted = program.snapshot->prepared->entries.size();
static_cast<void>(InvocationFor(*program.snapshot, program.codeOffset, request));
Require(program.snapshot->prepared->entries.size() == predicted + 1, "a draw state that registration did not predict was not prepared");
static_cast<void>(InvocationFor(*program.snapshot, program.codeOffset, request));
Require(program.snapshot->prepared->entries.size() == predicted + 1, "an artifact prepared at a draw was not reused");
}
ShaderRegistry invalidRegistry;
DrawDecode invalid{};

43
core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp
View File

@ -20,6 +20,8 @@
#include <vector>
#include <future>
#include <barrier>
#include <atomic>
#include <chrono>
#include <cstdlib>
#include <cstdio>
#include <fstream>
@ -93,12 +95,18 @@ void Run(AgcDriver::VulkanDevice& device) {
const ShaderRecompiler::ShaderComputeStageInfo compute{{1, 1, 1}, 0, {false, false, false}, false, 1, {}};
ShaderRecompiler::RecompileRequest request{{ShaderRecompiler::ShaderStage::Compute, reinterpret_cast<std::uintptr_t>(code.data()), code, 0, {}}, {32, 0, users, compute, {}, {}, {}}, device.ComputeTarget(32), {0, 0, 0, 128}};
RunUnregistered(device, request);
{
AgcDriver::DriverDetail::ShaderSnapshot unpredicted{request.shader.codeAddress, 0, 0, {code.begin(), code.end()}, {}};
unpredicted.header.resize(sizeof(Shader));
const auto prepared = AgcDriver::DriverDetail::SourceHandleFor(unpredicted, 0, request);
Require(unpredicted.prepared->entries.size() == 1 && unpredicted.prepared->entries.front().handle == prepared, "a registered shader with no matching artifact was not prepared");
Require(AgcDriver::DriverDetail::SourceHandleFor(unpredicted, 0, request) == prepared && unpredicted.prepared->entries.size() == 1, "an artifact prepared at a dispatch was not reused");
}
AgcDriver::DriverDetail::ShaderSnapshot snapshot{request.shader.codeAddress, 0, 0, {code.begin(), code.end()}, {}};
snapshot.header.resize(sizeof(Shader));
ExpectFailure([&] { static_cast<void>(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request)); }, "artifact is missing");
const auto handle = ShaderRecompiler::PrepareShader(request);
snapshot.prepared->entries.push_back({0, handle});
ExpectFailure([&] { static_cast<void>(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 1, request)); }, "artifact is missing");
ExpectFailure([&] { static_cast<void>(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 1, request)); }, "outside the snapshot");
const auto& artifact = ShaderRecompiler::GetPreparedArtifact(*handle);
for (std::uint32_t value = 0; value < 8; ++value) {
users.fill(value);
@ -119,7 +127,10 @@ void Run(AgcDriver::VulkanDevice& device) {
ExpectFailure([&] { static_cast<void>(AgcDriver::DriverDetail::InvocationFor(snapshot, 0, request)); }, "does not refer to registered code");
ExpectFailure([&] { static_cast<void>(AgcDriver::DriverDetail::InvocationFor(snapshot, 1, registeredRequest)); }, "outside the snapshot");
registeredRequest.context.compute->numThreads[0] = 2;
ExpectFailure([&] { static_cast<void>(AgcDriver::DriverDetail::InvocationFor(snapshot, 0, registeredRequest)); }, "artifact is missing");
static_cast<void>(AgcDriver::DriverDetail::InvocationFor(snapshot, 0, registeredRequest));
Require(snapshot.prepared->entries.size() == 2, "a dispatch state that registration did not predict was not prepared");
static_cast<void>(AgcDriver::DriverDetail::InvocationFor(snapshot, 0, registeredRequest));
Require(snapshot.prepared->entries.size() == 2, "an artifact prepared at a dispatch was not reused");
ShaderRecompiler::SrtRuntime preparedRuntime{};
preparedRuntime.userData = users;
const auto preparedCapture = invocation.Capture(preparedRuntime);
@ -145,12 +156,13 @@ void Run(AgcDriver::VulkanDevice& device) {
replacement->Dispatch(*replacementResult, 1, 1, 1);
replacement->WaitIdle();
replacementRequest.target.nonConstantImageOffsets = !replacementRequest.target.nonConstantImageOffsets;
ExpectFailure([&] { static_cast<void>(AgcDriver::DriverDetail::InvocationFor(snapshot, 0, replacementRequest)); }, "artifact is missing");
static_cast<void>(AgcDriver::DriverDetail::InvocationFor(snapshot, 0, replacementRequest));
Require(snapshot.prepared->entries.size() == 3, "a target that registration did not prepare for was not prepared");
ShaderRecompiler::SrtRuntime runtime{};
runtime.userData = users;
const auto capture = ShaderRecompiler::CaptureResources(request, runtime, *handle);
request.context.compute->numThreads[0] = 2;
ExpectFailure([&] { static_cast<void>(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request)); }, "artifact is missing");
Require(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request) == snapshot.prepared->entries[1].handle, "a dispatch did not find the artifact prepared for its state");
ExpectFailure([&] { static_cast<void>(ShaderRecompiler::CaptureResources(request, runtime, *handle)); }, "does not match the static ABI");
ExpectFailure([&] { static_cast<void>(ShaderRecompiler::MaterializeShader(request, *capture, *handle)); }, "does not match the static ABI");
const auto otherHandle = ShaderRecompiler::PrepareShader(request);
@ -161,10 +173,10 @@ void Run(AgcDriver::VulkanDevice& device) {
request.layout.pushConstantSizeBytes = 124;
Require(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request) == handle, "compatible push constant capacity discarded the prepared artifact");
request.layout.pushConstantSizeBytes = 126;
ExpectFailure([&] { static_cast<void>(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request)); }, "artifact is missing");
ExpectFailure([&] { static_cast<void>(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request)); }, "not dword-aligned");
request.layout.pushConstantSizeBytes = 128;
code[0] = 0xffffffffu;
ExpectFailure([&] { static_cast<void>(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request)); }, "artifact is missing");
ExpectFailure([&] { static_cast<void>(AgcDriver::DriverDetail::SourceHandleFor(snapshot, 0, request)); }, "does not refer to registered code");
}
void FailureCapture(AgcDriver::VulkanDevice& device) {
@ -496,9 +508,20 @@ void Registration(bool indirect) {
commands.insert(commands.end(), {0xc0033700u, 0x00100200u, static_cast<std::uint32_t>(destinationAddress), static_cast<std::uint32_t>(destinationAddress >> 32u), 7});
packet = Packet{commands.data(), static_cast<std::uint32_t>(commands.size()), 0, {}};
sceAgcDriverSubmitAcb(0x20, &packet);
ExpectFailure([] { AgcDriverWaitIdle_nid_postfix(); }, "artifact is missing");
ExpectFailure([] { AgcDriverWaitIdle_nid_postfix(); }, "artifact is missing");
ExpectFailure([] { AgcDriverShutdown_nid_postfix(); }, "artifact is missing");
AgcDriverWaitIdle_nid_postfix();
const auto deadline = std::chrono::steady_clock::now() + std::chrono::seconds(10);
while (std::atomic_ref<std::uint32_t>(destination).load() != 7 && std::chrono::steady_clock::now() < deadline) std::this_thread::yield();
Require(destination == 7, "a dispatch whose state registration did not predict did not run");
destination = 0;
alignas(256) static const std::array<std::uint32_t, 1> invalid{0xffffffffu};
const auto invalidAddress = reinterpret_cast<std::uintptr_t>(invalid.data());
commands[2] = static_cast<std::uint32_t>(invalidAddress >> 8u);
commands[5] = static_cast<std::uint32_t>(invalidAddress >> 40u);
packet = Packet{commands.data(), static_cast<std::uint32_t>(commands.size()), 0, {}};
sceAgcDriverSubmitAcb(0x20, &packet);
ExpectFailure([] { AgcDriverWaitIdle_nid_postfix(); }, "");
ExpectFailure([] { AgcDriverWaitIdle_nid_postfix(); }, "");
ExpectFailure([] { AgcDriverShutdown_nid_postfix(); }, "");
Require(destination == 0, "failed dispatch executed a subsequent memory write");
}

1
docs/dev/ARCHITECTURE.md
View File

@ -83,6 +83,7 @@ flowchart LR
- `Driver::RegisterShader` splits shader preparation in two. `PlanRegistered` runs on the registering guest thread: it validates the registers and entry point, decodes the code and builds the recompile request, so invalid registrations still fail there. The RDNA to IR translation and resource plan (`PrepareShader`) run on a background pool of `hardware_concurrency() / 2` detached threads (at least 2, 64 MiB stacks).
- The shader registry maps a code address to every header registered for that code, in registration order. Static, graphics and rectangle ABI resolution pick the header at the bound address (or one equal to it, `SameHeader`); draws and dispatches pick the program by the stage's binary type (`RegisteredProgram`), and a front entry matching both type 2 and type 4 throws. Within a type the last registration wins; re-registering an identical earlier header makes its snapshot current again without preparing it again, and changed code drops the older headers.
- The snapshot is published as pending. `SourceHandleFor`, `InvocationFor` and `ShaderPreparationTransaction::Edit` / `Read` wait for the pending job before reading `PreparedShaders`, and rethrow its failure. `APS5_SYNC_SHADER_PREPARE=1` prepares on the registering thread instead.
- A draw or dispatch whose state matches no artifact prepared at registration (a title that builds its static ABI state itself, without `sceAgcDriverResolve*Abi`) gets one prepared at use by `SourceHandleFor` or `InvocationFor`, logged once per artifact and kept in the snapshot for the next use.
- A draw's `Graphics::Pipeline` comes from `CachedPipeline`. When the device has `VK_EXT_graphics_pipeline_library` with fast linking, `VK_EXT_extended_dynamic_state` and `VK_KHR_dynamic_rendering`, draws render with `vkCmdBeginRenderingKHR` instead of render passes and the pipeline is linked from four libraries (vertex input, pre-rasterization shaders, fragment shader, fragment output) kept per device in [`PipelineLibrary.cpp`](../../core/libs/prx/libSceAgcDriver/Graphics/src/PipelineLibrary.cpp). A library built for one pipeline is reused by the next: another blend, vertex layout or target format builds only that part (the shader libraries name no attachment format), and cull mode, front face, depth and stencil state are dynamic, so changing them only links. The fast link is what the draw waits for; a background thread then links the same libraries with link-time optimization and the pipeline binds that one from the next draw on. Push constants then use a 256-byte block with a 128-byte slot per side: the pre-rasterization stages stack from byte 0, the pixel shader always starts at byte 128 (`StagePushOffset`), so its library does not depend on how much push data its vertex shader uses. Pipelines with per-face depth bias, rect lists, geometry shaders or a stage without a variant id are created whole, still with dynamic rendering; devices without the extensions (or with `APS5_NO_GPL=1`) keep render passes and whole pipelines.
### Shader MODE register access