[HLE/Cpu] Remove host lock contention from POSIX memory queries, thread resumes and timed waits (#1031)
* [HLE] Answer POSIX host memory queries without the mapping lock Every HLE import that reads stack arguments queries the guest stack's protection first. On POSIX the query took the same lock Protect holds across its mprotect call, and GPU write tracking protects pages constantly, so each query waited tens of microseconds; Silent Hill: The Short Message spent up to a second of every five seconds in queries. Regions are now also published as an immutable snapshot (replaced under the lock when a region is added or removed), and a query of untracked memory or of a region without per-page overrides is answered from it without the lock. Regions with overrides keep the locked path, which now finds where a protection run ends through an ordered set of overridden pages instead of visiting every override (write tracking leaves tens of thousands), and the next region by binary search. Query time drops from 200-1000 ms to 5-50 ms per 5 s. This is the POSIX counterpart of replacing page-by-page VirtualQuery scans on Windows. (cherry picked from commit b6ffe31f28fa93842726b71315c7731f97b5fda8) * [Cpu] Sleep through nanosleep, resumed on EINTR, in the POSIX Sleep stub The POSIX stand-in for kernel32!Sleep (which the native sceKernelUsleep intrinsic calls with milliseconds) converted to microseconds and called usleep. macOS rejects usleep(>= 1 s) with EINVAL and returns at once, so every guest sleep of a second or more came back immediately; and a signal (the runtime's thread activations, for one) ended any sleep early through an error the stub never looked at. The stub now builds a timespec and calls nanosleep, and when nanosleep returns EINTR it resumes with the remaining time. A test exercises the stub with the Win64 argument register for both a 1.2 s and a 20 ms sleep. (cherry picked from commit 43b770f86b5e3aa2332121f1bc2ddf2acc9d2968) * [HLE] Query POSIX view regions from a lock-free snapshot Every HLE import first queries the guest stack's protection, and the view-region query took the same gate the GPU write tracker holds across its mprotect calls, so each import waited on page-protection traffic from other threads (the main guest thread spent ~2% of its time in that wait at Silent Hill: The Short Message's menu). The region list is now also published as an array snapshot whenever it changes, and queries read the snapshot without the gate. Per-page protections already changed in place and were read racily; nothing changes there. (cherry picked from commit d7e72cb4974388883696516420b2e9d462778715) * [Cpu] Skip the scheduler gate when a host thread re-registers the same guest context scePthreadSelf on a guest thread the scheduler does not own registers the thread's context on every call, and that took the scheduler-wide gate each time - tens of thousands of times a second on Silent Hill: The Short Message's main thread, contending with the dispatcher and every blocked-thread wake. Each host thread remembers the handle and context it registered last; a repeat registration only refreshes the external-handle hint and returns. New or changed registrations still take the gate. (cherry picked from commit d1991b2b75261cd1f1ccb4e005b10805f0da24c1) * [Cpu] Keep each thread's guest continuation stub instead of mapping one per resume Every resume of a blocked guest thread allocated a 512-byte executable stub page and a host-RSP slot, and released both when the thread blocked again: an mmap/munmap pair plus two acquisitions of the host-memory gate per resume, which the GPU write tracker holds across its own protection changes. Contended on that gate, resumes were 62% of all lock waiting in Silent Hill: The Short Message's menu. Each host thread now keeps a stub page and slot per nesting depth for its lifetime; a resume reopens the page for writing with mprotect directly (the page is emulator-private, so it bypasses the tracked host-memory layer), re-emits the stub and seals it again. Lock waiting across threads halves (13.6 s to 6.5 s per 30 s). (cherry picked from commit 2c2293abdd5d19dd1be3d6941dcedf9b482c6420) * [Kernel] Round host condvar waits up to whole milliseconds Monitor.Wait truncates its timeout to milliseconds, so a timed wait with less than 1 ms left became a zero timeout and spun until the deadline. UE's game thread waits for the render fence in 1 ms slices, which kept a host core busy for the whole wait and contended the condvar lock with the signaling threads. Timed waits may wake late, so round up instead. SH menu 4.29 -> 4.64 fps together with the readback retry change. (cherry picked from commit 66eaa0095fe594ff8e285cbf31c32bd84274ac85) * [Tests] Run the POSIX Sleep stub test only in x64 processes The stub is x86-64 machine code; a native arm64 test host cannot map or run it, so the macOS arm64 CI runner failed to allocate the stub page.
This commit is contained in:
kuba
GitHub
parent
d809f84491
commit
4e6d1e0d9a
|
|
@ -3978,6 +3978,11 @@ public sealed unsafe partial class DirectExecutionBackend : INativeCpuBackend, I
|
|||
|
||||
public bool SupportsGuestContextTransfer => true;
|
||||
|
||||
// The handle and context this host thread registered last; a repeat registration (every
|
||||
// scePthreadSelf on an external guest thread, tens of thousands a second) then skips the
|
||||
// scheduler gate.
|
||||
[ThreadStatic] private static ulong _registeredExternalHandle;
|
||||
[ThreadStatic] private static CpuContext? _registeredExternalContext;
|
||||
public bool TryGetGuestThreadStackBounds(
|
||||
ulong threadHandle,
|
||||
out ulong stackBase,
|
||||
|
|
@ -4012,6 +4017,12 @@ public sealed unsafe partial class DirectExecutionBackend : INativeCpuBackend, I
|
|||
return;
|
||||
}
|
||||
|
||||
if (_registeredExternalHandle == threadHandle && ReferenceEquals(_registeredExternalContext, context))
|
||||
{
|
||||
Volatile.Write(ref _currentExternalGuestThreadHandle, threadHandle);
|
||||
return;
|
||||
}
|
||||
|
||||
lock (_guestThreadGate)
|
||||
{
|
||||
_currentExternalGuestThreadHandle = threadHandle;
|
||||
|
|
@ -4025,15 +4036,19 @@ public sealed unsafe partial class DirectExecutionBackend : INativeCpuBackend, I
|
|||
{
|
||||
existing.Context = context;
|
||||
Volatile.Write(ref existing.HostThreadId, hostThreadId);
|
||||
return;
|
||||
}
|
||||
else
|
||||
{
|
||||
_externalGuestThreads[threadHandle] = new ExternalGuestThreadState
|
||||
{
|
||||
Context = context,
|
||||
Name = $"External-{threadHandle:X}",
|
||||
HostThreadId = hostThreadId,
|
||||
};
|
||||
}
|
||||
|
||||
_externalGuestThreads[threadHandle] = new ExternalGuestThreadState
|
||||
{
|
||||
Context = context,
|
||||
Name = $"External-{threadHandle:X}",
|
||||
HostThreadId = hostThreadId,
|
||||
};
|
||||
_registeredExternalHandle = threadHandle;
|
||||
_registeredExternalContext = context;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -6382,19 +6397,15 @@ public sealed unsafe partial class DirectExecutionBackend : INativeCpuBackend, I
|
|||
return GuestNativeCallExitReason.Exception;
|
||||
}
|
||||
const uint stubSize = 512u;
|
||||
void* ptr = VirtualAlloc(null, stubSize, 12288u, 4u);
|
||||
if (ptr == null)
|
||||
// The stub page and host-RSP slot are kept per host thread and nesting depth: allocating
|
||||
// and releasing them on every resume cost an mmap/munmap pair and two acquisitions of the
|
||||
// host-memory gate, which the GPU write tracker holds across its own protection changes.
|
||||
var depth = _continuationDepth;
|
||||
if (!TryAcquireContinuationStub(depth, stubSize, out var ptr, out var hostRspStorage, out reason))
|
||||
{
|
||||
reason = "failed to allocate executable memory for guest thread stub";
|
||||
return GuestNativeCallExitReason.Exception;
|
||||
}
|
||||
void* hostRspStorage = NativeMemory.AllocZeroed((nuint)HostStackStateBytes);
|
||||
if (hostRspStorage == null)
|
||||
{
|
||||
VirtualFree(ptr, 0u, 32768u);
|
||||
reason = "failed to allocate writable host-RSP storage for guest continuation stub";
|
||||
return GuestNativeCallExitReason.Exception;
|
||||
}
|
||||
_continuationDepth = depth + 1;
|
||||
var previousActiveBackend = _activeExecutionBackend;
|
||||
var previousActiveContext = _activeCpuContext;
|
||||
var previousSentinel = _activeEntryReturnSentinelRip;
|
||||
|
|
@ -6468,8 +6479,7 @@ public sealed unsafe partial class DirectExecutionBackend : INativeCpuBackend, I
|
|||
reason = $"failed to patch guest continuation return slot at 0x{returnSlotAddress:X16}";
|
||||
return GuestNativeCallExitReason.Exception;
|
||||
}
|
||||
uint oldProtect = default(uint);
|
||||
if (!VirtualProtect(ptr, stubSize, 32u, &oldProtect))
|
||||
if (!ProtectContinuationStub(ptr, stubSize, executable: true))
|
||||
{
|
||||
reason = "failed to seal guest continuation stub execute-read";
|
||||
return GuestNativeCallExitReason.Exception;
|
||||
|
|
@ -6529,11 +6539,85 @@ public sealed unsafe partial class DirectExecutionBackend : INativeCpuBackend, I
|
|||
previousForcedExit,
|
||||
previousYieldRequested,
|
||||
previousYieldReason);
|
||||
NativeMemory.Free(hostRspStorage);
|
||||
VirtualFree(ptr, 0u, 32768u);
|
||||
_continuationDepth = depth;
|
||||
}
|
||||
}
|
||||
|
||||
[ThreadStatic] private static nint[]? _continuationStubPages;
|
||||
[ThreadStatic] private static nint[]? _continuationHostRspSlots;
|
||||
[ThreadStatic] private static int _continuationDepth;
|
||||
|
||||
// Returns this thread's stub page for the given nesting depth, writable and with a zeroed
|
||||
// host-RSP slot; the first use at a depth allocates them, and they live as long as the thread.
|
||||
private unsafe bool TryAcquireContinuationStub(int depth, uint stubSize, out void* stub, out void* hostRspStorage, out string? reason)
|
||||
{
|
||||
reason = null;
|
||||
stub = null;
|
||||
hostRspStorage = null;
|
||||
var pages = _continuationStubPages ??= new nint[4];
|
||||
var slots = _continuationHostRspSlots ??= new nint[4];
|
||||
if (depth >= pages.Length)
|
||||
{
|
||||
Array.Resize(ref pages, depth + 2);
|
||||
Array.Resize(ref slots, depth + 2);
|
||||
_continuationStubPages = pages;
|
||||
_continuationHostRspSlots = slots;
|
||||
}
|
||||
|
||||
if (pages[depth] == 0)
|
||||
{
|
||||
var page = VirtualAlloc(null, stubSize, 12288u, 4u);
|
||||
if (page == null)
|
||||
{
|
||||
reason = "failed to allocate executable memory for guest thread stub";
|
||||
return false;
|
||||
}
|
||||
|
||||
var slot = NativeMemory.AllocZeroed((nuint)HostStackStateBytes);
|
||||
if (slot == null)
|
||||
{
|
||||
VirtualFree(page, 0u, 32768u);
|
||||
reason = "failed to allocate writable host-RSP storage for guest continuation stub";
|
||||
return false;
|
||||
}
|
||||
|
||||
pages[depth] = (nint)page;
|
||||
slots[depth] = (nint)slot;
|
||||
}
|
||||
else
|
||||
{
|
||||
if (!ProtectContinuationStub((void*)pages[depth], stubSize, executable: false))
|
||||
{
|
||||
reason = "failed to reopen guest continuation stub for writing";
|
||||
return false;
|
||||
}
|
||||
|
||||
NativeMemory.Clear((void*)slots[depth], (nuint)HostStackStateBytes);
|
||||
}
|
||||
|
||||
stub = (void*)pages[depth];
|
||||
hostRspStorage = (void*)slots[depth];
|
||||
return true;
|
||||
}
|
||||
|
||||
// The stub page is emulator-private, so on POSIX its protection changes bypass the tracked
|
||||
// host-memory layer and its gate.
|
||||
private static unsafe bool ProtectContinuationStub(void* stub, uint stubSize, bool executable)
|
||||
{
|
||||
if (OperatingSystem.IsWindows())
|
||||
{
|
||||
uint oldProtect = 0;
|
||||
return VirtualProtect(stub, stubSize, executable ? 32u : 4u, &oldProtect);
|
||||
}
|
||||
|
||||
var pageSize = (nint)Environment.SystemPageSize;
|
||||
var page = (nint)stub & ~(pageSize - 1);
|
||||
return PosixMprotect(page, (nuint)pageSize, executable ? 1 | 4 : 1 | 2) == 0;
|
||||
}
|
||||
|
||||
[DllImport("libc", EntryPoint = "mprotect")]
|
||||
private static extern int PosixMprotect(nint address, nuint length, int protection);
|
||||
|
||||
// The continuation trampoline is rebuilt on every blocked-thread resume.
|
||||
// Keep its tiny writer on the stack: capturing local emit functions create a
|
||||
// managed display-class allocation on this extremely hot path.
|
||||
|
|
|
|||
|
|
@ -322,20 +322,43 @@ internal static unsafe class PosixHostStubs
|
|||
|
||||
private static nint EmitSleep(byte* page, ref int offset)
|
||||
{
|
||||
// void Sleep(DWORD milliseconds in ecx) -> usleep(microseconds in edi).
|
||||
var usleep = ResolveLibcExport("usleep");
|
||||
// void Sleep(DWORD milliseconds in ecx) -> nanosleep(&req, &rem), resumed on EINTR. usleep
|
||||
// would be shorter, but macOS rejects usleep(>= 1 s) with EINVAL, and a signal (the
|
||||
// runtime's thread activations, for one) cuts either short: a sleep that returned at once
|
||||
// on an ignored error turned UE's idle HTTP thread into a hot spin.
|
||||
var nanosleep = ResolveLibcExport("nanosleep");
|
||||
var errorLocation = ResolveLibcExport(OperatingSystem.IsMacOS() ? "__error" : "__errno_location");
|
||||
var start = (nint)(page + offset);
|
||||
Emit(page, ref offset, 0x56); // push rsi
|
||||
Emit(page, ref offset, 0x57); // push rdi
|
||||
Emit(page, ref offset, 0x48, 0x83, 0xEC, 0x08); // sub rsp, 8
|
||||
Emit(page, ref offset, 0x89, 0xCF); // mov edi, ecx
|
||||
Emit(page, ref offset, 0x81, 0xFF, 0xFF, 0x0F, 0x00, 0x00); // cmp edi, 0xFFF
|
||||
Emit(page, ref offset, 0x76, 0x05); // jbe +5
|
||||
Emit(page, ref offset, 0xBF, 0xFF, 0x0F, 0x00, 0x00); // mov edi, 0xFFF (cap at ~4s)
|
||||
Emit(page, ref offset, 0x69, 0xFF, 0xE8, 0x03, 0x00, 0x00); // imul edi, edi, 1000
|
||||
EmitMovRaxImm64(page, ref offset, usleep); // mov rax, imm64
|
||||
Emit(page, ref offset, 0x53); // push rbx
|
||||
Emit(page, ref offset, 0x48, 0x83, 0xEC, 0x20); // sub rsp, 0x20: req at [rsp], rem at [rsp+16]
|
||||
Emit(page, ref offset, 0x89, 0xC8); // mov eax, ecx
|
||||
Emit(page, ref offset, 0x31, 0xD2); // xor edx, edx
|
||||
Emit(page, ref offset, 0xB9, 0xE8, 0x03, 0x00, 0x00); // mov ecx, 1000
|
||||
Emit(page, ref offset, 0xF7, 0xF1); // div ecx: eax = seconds, edx = milliseconds
|
||||
Emit(page, ref offset, 0x48, 0x89, 0x04, 0x24); // mov [rsp], rax (tv_sec)
|
||||
Emit(page, ref offset, 0x69, 0xD2, 0x40, 0x42, 0x0F, 0x00); // imul edx, edx, 1000000
|
||||
Emit(page, ref offset, 0x48, 0x89, 0x54, 0x24, 0x08); // mov [rsp+8], rdx (tv_nsec)
|
||||
// loop:
|
||||
Emit(page, ref offset, 0x48, 0x89, 0xE7); // mov rdi, rsp
|
||||
Emit(page, ref offset, 0x48, 0x8D, 0x74, 0x24, 0x10); // lea rsi, [rsp+16]
|
||||
EmitMovRaxImm64(page, ref offset, nanosleep); // mov rax, imm64
|
||||
Emit(page, ref offset, 0xFF, 0xD0); // call rax
|
||||
Emit(page, ref offset, 0x48, 0x83, 0xC4, 0x08); // add rsp, 8
|
||||
Emit(page, ref offset, 0x85, 0xC0); // test eax, eax
|
||||
Emit(page, ref offset, 0x74, 0x26); // jz done
|
||||
EmitMovRaxImm64(page, ref offset, errorLocation); // mov rax, imm64
|
||||
Emit(page, ref offset, 0xFF, 0xD0); // call rax
|
||||
Emit(page, ref offset, 0x83, 0x38, 0x04); // cmp dword [rax], EINTR
|
||||
Emit(page, ref offset, 0x75, 0x15); // jne done
|
||||
Emit(page, ref offset, 0x48, 0x8B, 0x44, 0x24, 0x10); // mov rax, [rsp+16]
|
||||
Emit(page, ref offset, 0x48, 0x89, 0x04, 0x24); // mov [rsp], rax
|
||||
Emit(page, ref offset, 0x48, 0x8B, 0x44, 0x24, 0x18); // mov rax, [rsp+24]
|
||||
Emit(page, ref offset, 0x48, 0x89, 0x44, 0x24, 0x08); // mov [rsp+8], rax
|
||||
Emit(page, ref offset, 0xEB, 0xC2); // jmp loop
|
||||
// done:
|
||||
Emit(page, ref offset, 0x48, 0x83, 0xC4, 0x20); // add rsp, 0x20
|
||||
Emit(page, ref offset, 0x5B); // pop rbx
|
||||
Emit(page, ref offset, 0x5F); // pop rdi
|
||||
Emit(page, ref offset, 0x5E); // pop rsi
|
||||
Emit(page, ref offset, 0xC3); // ret
|
||||
|
|
|
|||
|
|
@ -462,20 +462,43 @@ internal static unsafe class PosixHostStubs
|
|||
|
||||
private static nint EmitSleep(byte* page, ref int offset)
|
||||
{
|
||||
// void Sleep(DWORD milliseconds in ecx) -> usleep(microseconds in edi).
|
||||
var usleep = ResolveLibcExport("usleep");
|
||||
// void Sleep(DWORD milliseconds in ecx) -> nanosleep(&req, &rem), resumed on EINTR. usleep
|
||||
// would be shorter, but macOS rejects usleep(>= 1 s) with EINVAL, and a signal (the
|
||||
// runtime's thread activations, for one) cuts either short: a sleep that returned at once
|
||||
// on an ignored error turned UE's idle HTTP thread into a hot spin.
|
||||
var nanosleep = ResolveLibcExport("nanosleep");
|
||||
var errorLocation = ResolveLibcExport(OperatingSystem.IsMacOS() ? "__error" : "__errno_location");
|
||||
var start = (nint)(page + offset);
|
||||
Emit(page, ref offset, 0x56); // push rsi
|
||||
Emit(page, ref offset, 0x57); // push rdi
|
||||
Emit(page, ref offset, 0x48, 0x83, 0xEC, 0x08); // sub rsp, 8
|
||||
Emit(page, ref offset, 0x89, 0xCF); // mov edi, ecx
|
||||
Emit(page, ref offset, 0x81, 0xFF, 0xFF, 0x0F, 0x00, 0x00); // cmp edi, 0xFFF
|
||||
Emit(page, ref offset, 0x76, 0x05); // jbe +5
|
||||
Emit(page, ref offset, 0xBF, 0xFF, 0x0F, 0x00, 0x00); // mov edi, 0xFFF (cap at ~4s)
|
||||
Emit(page, ref offset, 0x69, 0xFF, 0xE8, 0x03, 0x00, 0x00); // imul edi, edi, 1000
|
||||
EmitMovRaxImm64(page, ref offset, usleep); // mov rax, imm64
|
||||
Emit(page, ref offset, 0x53); // push rbx
|
||||
Emit(page, ref offset, 0x48, 0x83, 0xEC, 0x20); // sub rsp, 0x20: req at [rsp], rem at [rsp+16]
|
||||
Emit(page, ref offset, 0x89, 0xC8); // mov eax, ecx
|
||||
Emit(page, ref offset, 0x31, 0xD2); // xor edx, edx
|
||||
Emit(page, ref offset, 0xB9, 0xE8, 0x03, 0x00, 0x00); // mov ecx, 1000
|
||||
Emit(page, ref offset, 0xF7, 0xF1); // div ecx: eax = seconds, edx = milliseconds
|
||||
Emit(page, ref offset, 0x48, 0x89, 0x04, 0x24); // mov [rsp], rax (tv_sec)
|
||||
Emit(page, ref offset, 0x69, 0xD2, 0x40, 0x42, 0x0F, 0x00); // imul edx, edx, 1000000
|
||||
Emit(page, ref offset, 0x48, 0x89, 0x54, 0x24, 0x08); // mov [rsp+8], rdx (tv_nsec)
|
||||
// loop:
|
||||
Emit(page, ref offset, 0x48, 0x89, 0xE7); // mov rdi, rsp
|
||||
Emit(page, ref offset, 0x48, 0x8D, 0x74, 0x24, 0x10); // lea rsi, [rsp+16]
|
||||
EmitMovRaxImm64(page, ref offset, nanosleep); // mov rax, imm64
|
||||
Emit(page, ref offset, 0xFF, 0xD0); // call rax
|
||||
Emit(page, ref offset, 0x48, 0x83, 0xC4, 0x08); // add rsp, 8
|
||||
Emit(page, ref offset, 0x85, 0xC0); // test eax, eax
|
||||
Emit(page, ref offset, 0x74, 0x26); // jz done
|
||||
EmitMovRaxImm64(page, ref offset, errorLocation); // mov rax, imm64
|
||||
Emit(page, ref offset, 0xFF, 0xD0); // call rax
|
||||
Emit(page, ref offset, 0x83, 0x38, 0x04); // cmp dword [rax], EINTR
|
||||
Emit(page, ref offset, 0x75, 0x15); // jne done
|
||||
Emit(page, ref offset, 0x48, 0x8B, 0x44, 0x24, 0x10); // mov rax, [rsp+16]
|
||||
Emit(page, ref offset, 0x48, 0x89, 0x04, 0x24); // mov [rsp], rax
|
||||
Emit(page, ref offset, 0x48, 0x8B, 0x44, 0x24, 0x18); // mov rax, [rsp+24]
|
||||
Emit(page, ref offset, 0x48, 0x89, 0x44, 0x24, 0x08); // mov [rsp+8], rax
|
||||
Emit(page, ref offset, 0xEB, 0xC2); // jmp loop
|
||||
// done:
|
||||
Emit(page, ref offset, 0x48, 0x83, 0xC4, 0x20); // add rsp, 0x20
|
||||
Emit(page, ref offset, 0x5B); // pop rbx
|
||||
Emit(page, ref offset, 0x5F); // pop rdi
|
||||
Emit(page, ref offset, 0x5E); // pop rsi
|
||||
Emit(page, ref offset, 0xC3); // ret
|
||||
|
|
|
|||
|
|
@ -1,6 +1,8 @@
|
|||
// Copyright (C) 2026 SharpEmu Emulator Project
|
||||
// SPDX-License-Identifier: GPL-2.0-or-later
|
||||
|
||||
using System.Threading;
|
||||
|
||||
namespace SharpEmu.HLE.Host.Posix;
|
||||
|
||||
// View mappings bypass the anonymous-allocation tables. Keep their query state together.
|
||||
|
|
@ -11,6 +13,12 @@ internal static class PosixViewRegions
|
|||
uint[]? PageProtections, ulong PageBase);
|
||||
private static readonly List<Region> Regions = new();
|
||||
|
||||
// The regions in address order, replaced whenever the list changes. Queries read it without
|
||||
// Gate: every HLE import queries the guest stack's protection first, and Gate is held across
|
||||
// mprotect by the GPU write tracker, so a locked query made each import wait on page traffic.
|
||||
// Page protections change in place and are read racily, as a concurrent change always was.
|
||||
private static Region[] _snapshot = [];
|
||||
|
||||
internal static void Replace(ulong address, ulong size, uint state, uint protection)
|
||||
{
|
||||
HostMemory.OnMappingChanged();
|
||||
|
|
@ -46,6 +54,8 @@ internal static class PosixViewRegions
|
|||
else
|
||||
Regions.Add(region);
|
||||
}
|
||||
|
||||
Volatile.Write(ref _snapshot, [.. Regions]);
|
||||
}
|
||||
|
||||
// Mapping creates the storage. Fault-time protection updates must not allocate.
|
||||
|
|
@ -66,9 +76,8 @@ internal static class PosixViewRegions
|
|||
|
||||
internal static bool TryQuery(ulong address, out HostMemory.BasicInfo info)
|
||||
{
|
||||
lock (Gate)
|
||||
{
|
||||
foreach (var region in Regions)
|
||||
foreach (var region in Volatile.Read(ref _snapshot))
|
||||
{
|
||||
if (region.Start > address) break;
|
||||
if (address >= region.End) continue;
|
||||
|
|
|
|||
|
|
@ -149,6 +149,14 @@ public static unsafe class HostMemory
|
|||
private static readonly object Gate = new();
|
||||
private static readonly SortedList<ulong, Region> Regions = new();
|
||||
|
||||
// The regions in address order, replaced under Gate whenever a region is added or removed.
|
||||
// Queries read it without the lock: Protect holds Gate across its mprotect call, and GPU
|
||||
// write tracking protects pages constantly, so a locked query (one per HLE call reading
|
||||
// stack arguments) waited tens of microseconds behind those syscalls.
|
||||
private static Region[] RegionSnapshot = [];
|
||||
|
||||
private static void PublishRegionsLocked() => Volatile.Write(ref RegionSnapshot, [.. Regions.Values]);
|
||||
|
||||
private sealed class Region
|
||||
{
|
||||
public ulong Base;
|
||||
|
|
@ -156,6 +164,10 @@ public static unsafe class HostMemory
|
|||
public uint DefaultProtect;
|
||||
public Dictionary<ulong, uint>? PageProtects;
|
||||
|
||||
// The overridden pages in address order, so a query finds where a protection run ends
|
||||
// without visiting every override: write tracking leaves tens of thousands of them.
|
||||
public SortedSet<ulong>? OverridePages;
|
||||
|
||||
public ulong End => Base + Size;
|
||||
|
||||
public uint ProtectAt(ulong pageAddress)
|
||||
|
|
@ -269,6 +281,7 @@ public static unsafe class HostMemory
|
|||
Size = alignedSize,
|
||||
DefaultProtect = protect
|
||||
};
|
||||
PublishRegionsLocked();
|
||||
|
||||
return (void*)result;
|
||||
}
|
||||
|
|
@ -288,6 +301,7 @@ public static unsafe class HostMemory
|
|||
|
||||
HostMemory.OnMappingChanged();
|
||||
Regions.Remove((ulong)address);
|
||||
PublishRegionsLocked();
|
||||
return munmap((nint)address, (nuint)region.Size) == 0;
|
||||
}
|
||||
}
|
||||
|
|
@ -325,6 +339,10 @@ public static unsafe class HostMemory
|
|||
{
|
||||
info = default;
|
||||
var pageAddress = AlignDown((ulong)address, PageSize);
|
||||
if (TryQueryUnlocked(pageAddress, ref info))
|
||||
{
|
||||
return (nuint)sizeof(BasicInfo);
|
||||
}
|
||||
|
||||
lock (Gate)
|
||||
{
|
||||
|
|
@ -340,30 +358,37 @@ public static unsafe class HostMemory
|
|||
var pageProtects = region.PageProtects;
|
||||
uint protect;
|
||||
ulong runEnd;
|
||||
if (pageProtects is null || pageProtects.Count == 0)
|
||||
if (pageProtects is null || pageProtects.Count == 0 || region.OverridePages is not { } overridePages)
|
||||
{
|
||||
protect = region.DefaultProtect;
|
||||
runEnd = region.End;
|
||||
}
|
||||
else if (pageProtects.TryGetValue(pageAddress, out protect))
|
||||
{
|
||||
// Overrides run while the next page is overridden with the same protection.
|
||||
runEnd = pageAddress + PageSize;
|
||||
while (runEnd < region.End &&
|
||||
pageProtects.TryGetValue(runEnd, out var nextProtect) &&
|
||||
nextProtect == protect)
|
||||
foreach (var overrideAddress in runEnd < region.End ? overridePages.GetViewBetween(runEnd, region.End - 1) : [])
|
||||
{
|
||||
if (overrideAddress != runEnd || pageProtects[overrideAddress] != protect)
|
||||
{
|
||||
break;
|
||||
}
|
||||
|
||||
runEnd += PageSize;
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
// The default protection runs to the next override.
|
||||
protect = region.DefaultProtect;
|
||||
runEnd = region.End;
|
||||
foreach (var overrideAddress in pageProtects.Keys)
|
||||
if (pageAddress + PageSize < region.End)
|
||||
{
|
||||
if (overrideAddress > pageAddress && overrideAddress < runEnd)
|
||||
// The first element only: a view's Count walks the whole view.
|
||||
foreach (var overrideAddress in overridePages.GetViewBetween(pageAddress + PageSize, region.End - 1))
|
||||
{
|
||||
runEnd = overrideAddress;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -382,12 +407,18 @@ public static unsafe class HostMemory
|
|||
// reported as a free block reaching to the next tracked region
|
||||
// so scanning callers keep advancing.
|
||||
var nextBase = ulong.MaxValue;
|
||||
foreach (var regionBase in Regions.Keys)
|
||||
var keys = Regions.Keys;
|
||||
for (int low = 0, high = keys.Count - 1; low <= high;)
|
||||
{
|
||||
if (regionBase > pageAddress)
|
||||
var middle = low + ((high - low) >> 1);
|
||||
if (keys[middle] > pageAddress)
|
||||
{
|
||||
nextBase = regionBase;
|
||||
break;
|
||||
nextBase = keys[middle];
|
||||
high = middle - 1;
|
||||
}
|
||||
else
|
||||
{
|
||||
low = middle + 1;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -416,6 +447,56 @@ public static unsafe class HostMemory
|
|||
return false;
|
||||
}
|
||||
|
||||
// Answers without Gate for untracked memory and for regions without per-page overrides;
|
||||
// a region with overrides needs their dictionary, which only Gate protects. A concurrent
|
||||
// Protect may land just before or after either way, as with the locked query.
|
||||
private static bool TryQueryUnlocked(ulong pageAddress, ref BasicInfo info)
|
||||
{
|
||||
var regions = Volatile.Read(ref RegionSnapshot);
|
||||
Region? candidate = null;
|
||||
var nextBase = ulong.MaxValue;
|
||||
for (int low = 0, high = regions.Length - 1; low <= high;)
|
||||
{
|
||||
var middle = low + ((high - low) >> 1);
|
||||
if (regions[middle].Base <= pageAddress)
|
||||
{
|
||||
candidate = regions[middle];
|
||||
low = middle + 1;
|
||||
}
|
||||
else
|
||||
{
|
||||
nextBase = regions[middle].Base;
|
||||
high = middle - 1;
|
||||
}
|
||||
}
|
||||
|
||||
if (candidate is not null && pageAddress < candidate.End)
|
||||
{
|
||||
if (Volatile.Read(ref candidate.PageProtects) is { Count: > 0 })
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
info.BaseAddress = pageAddress;
|
||||
info.AllocationBase = candidate.Base;
|
||||
info.AllocationProtect = candidate.DefaultProtect;
|
||||
info.RegionSize = candidate.End - pageAddress;
|
||||
info.State = MEM_COMMIT;
|
||||
info.Protect = candidate.DefaultProtect;
|
||||
info.Type = MEM_PRIVATE;
|
||||
return true;
|
||||
}
|
||||
|
||||
info.BaseAddress = pageAddress;
|
||||
info.AllocationBase = 0;
|
||||
info.AllocationProtect = PAGE_NOACCESS;
|
||||
info.RegionSize = (nextBase == ulong.MaxValue ? pageAddress + PageSize : nextBase) - pageAddress;
|
||||
info.State = MEM_FREE_STATE;
|
||||
info.Protect = PAGE_NOACCESS;
|
||||
info.Type = 0;
|
||||
return true;
|
||||
}
|
||||
|
||||
private static bool TryFindRegionLocked(ulong address, out Region region)
|
||||
{
|
||||
region = null!;
|
||||
|
|
@ -454,20 +535,26 @@ public static unsafe class HostMemory
|
|||
{
|
||||
region.DefaultProtect = protect;
|
||||
region.PageProtects = null;
|
||||
region.OverridePages = null;
|
||||
return;
|
||||
}
|
||||
|
||||
region.PageProtects ??= new Dictionary<ulong, uint>();
|
||||
region.OverridePages ??= new SortedSet<ulong>();
|
||||
var end = start + size;
|
||||
for (var pageAddress = start; pageAddress < end; pageAddress += PageSize)
|
||||
{
|
||||
if (protect == region.DefaultProtect)
|
||||
{
|
||||
region.PageProtects.Remove(pageAddress);
|
||||
if (region.PageProtects.Remove(pageAddress))
|
||||
{
|
||||
region.OverridePages.Remove(pageAddress);
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
region.PageProtects[pageAddress] = protect;
|
||||
region.OverridePages.Add(pageAddress);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1988,7 +1988,7 @@ public static class KernelPthreadCompatExports
|
|||
break;
|
||||
}
|
||||
|
||||
var waitDuration = TimeSpan.FromMilliseconds(10);
|
||||
var waitMilliseconds = 10;
|
||||
if (timed)
|
||||
{
|
||||
var remaining = GetRemainingTimeout(deadline);
|
||||
|
|
@ -1998,13 +1998,14 @@ public static class KernelPthreadCompatExports
|
|||
break;
|
||||
}
|
||||
|
||||
if (remaining < waitDuration)
|
||||
{
|
||||
waitDuration = remaining;
|
||||
}
|
||||
// Monitor.Wait truncates to whole milliseconds, so a sub-millisecond remainder
|
||||
// became a zero timeout and the wait spun until the deadline. UE's game thread
|
||||
// waits for the render fence in 1 ms slices, which kept a core busy. A timed wait
|
||||
// may wake late, so round up instead.
|
||||
waitMilliseconds = (int)Math.Min(waitMilliseconds, Math.Ceiling(remaining.TotalMilliseconds));
|
||||
}
|
||||
|
||||
_ = Monitor.Wait(state.SyncRoot, waitDuration);
|
||||
_ = Monitor.Wait(state.SyncRoot, waitMilliseconds);
|
||||
if (waiter.CompletionState == 0 &&
|
||||
timed &&
|
||||
GetRemainingTimeout(deadline) <= TimeSpan.Zero)
|
||||
|
|
|
|||
|
|
@ -144,7 +144,8 @@ public static class KernelSyncOnAddressCompatExports
|
|||
{
|
||||
if (Volatile.Read(ref _wakeRequested) == 0)
|
||||
{
|
||||
Monitor.Wait(_hostGate, waitSlice);
|
||||
// Round up so a sub-millisecond remainder does not spin.
|
||||
Monitor.Wait(_hostGate, (int)Math.Ceiling(waitSlice.TotalMilliseconds));
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,27 @@
|
|||
// Copyright (C) 2026 SharpEmu Emulator Project
|
||||
// SPDX-License-Identifier: GPL-2.0-or-later
|
||||
|
||||
using System.Diagnostics;
|
||||
using SharpEmu.Core.Cpu.Native;
|
||||
using Xunit;
|
||||
|
||||
namespace SharpEmu.Libs.Tests.Cpu;
|
||||
|
||||
// The POSIX stand-in for kernel32!Sleep takes milliseconds in ecx. It must sleep for a second or
|
||||
// more as well: macOS usleep rejects such lengths with EINVAL, which made the stub return at once.
|
||||
public sealed unsafe class PosixSleepStubTests
|
||||
{
|
||||
[NativeX64Theory]
|
||||
[InlineData(1200u, 1.0, 4.0)]
|
||||
[InlineData(20u, 0.015, 1.0)]
|
||||
public void SleepStubSleepsForTheRequestedMilliseconds(uint milliseconds, double minSeconds, double maxSeconds)
|
||||
{
|
||||
if (OperatingSystem.IsWindows()) return;
|
||||
// SysV passes the fourth integer argument in rcx, where the stub reads its milliseconds.
|
||||
var sleep = (delegate* unmanaged<nuint, nuint, nuint, uint, void>)PosixHostStubs.SleepStubAddress;
|
||||
var started = Stopwatch.GetTimestamp();
|
||||
sleep(0, 0, 0, milliseconds);
|
||||
var elapsed = Stopwatch.GetElapsedTime(started).TotalSeconds;
|
||||
Assert.InRange(elapsed, minSeconds, maxSeconds);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
// Copyright (C) 2026 SharpEmu Emulator Project
|
||||
// SPDX-License-Identifier: GPL-2.0-or-later
|
||||
|
||||
using SharpEmu.HLE;
|
||||
using Xunit;
|
||||
|
||||
namespace SharpEmu.Libs.Tests.Memory;
|
||||
|
||||
// A POSIX query reports the run of pages sharing the queried page's protection, ending at the
|
||||
// next protection change, as Win32 VirtualQuery does.
|
||||
public sealed unsafe class HostMemoryQueryRunTests
|
||||
{
|
||||
[Fact]
|
||||
public void QueryRunsEndAtTheNextProtectionChange()
|
||||
{
|
||||
if (OperatingSystem.IsWindows()) return;
|
||||
var page = (ulong)Environment.SystemPageSize;
|
||||
var memory = (byte*)HostMemory.Alloc(null, (nuint)(page * 16), HostMemory.MEM_RESERVE | HostMemory.MEM_COMMIT, HostMemory.PAGE_READWRITE);
|
||||
Assert.True(memory != null);
|
||||
var start = (ulong)memory;
|
||||
try
|
||||
{
|
||||
Assert.True(HostMemory.Protect((void*)(start + 3 * page), (nuint)(3 * page), HostMemory.PAGE_READONLY, out _));
|
||||
Assert.True(HostMemory.Protect((void*)(start + 9 * page), (nuint)page, HostMemory.PAGE_NOACCESS, out _));
|
||||
Assert.True(HostMemory.Protect((void*)(start + 15 * page), (nuint)page, HostMemory.PAGE_READONLY, out _));
|
||||
|
||||
AssertRun(start, 0, 3, HostMemory.PAGE_READWRITE);
|
||||
AssertRun(start, 3, 3, HostMemory.PAGE_READONLY);
|
||||
AssertRun(start, 4, 2, HostMemory.PAGE_READONLY);
|
||||
AssertRun(start, 6, 3, HostMemory.PAGE_READWRITE);
|
||||
AssertRun(start, 9, 1, HostMemory.PAGE_NOACCESS);
|
||||
AssertRun(start, 10, 5, HostMemory.PAGE_READWRITE);
|
||||
// A run at the last page of the mapping.
|
||||
AssertRun(start, 15, 1, HostMemory.PAGE_READONLY);
|
||||
|
||||
// Restoring the default protection removes the overrides from the run.
|
||||
Assert.True(HostMemory.Protect((void*)(start + 3 * page), (nuint)(7 * page), HostMemory.PAGE_READWRITE, out _));
|
||||
AssertRun(start, 0, 15, HostMemory.PAGE_READWRITE);
|
||||
}
|
||||
finally
|
||||
{
|
||||
Assert.True(HostMemory.Free(memory, 0, HostMemory.MEM_RELEASE));
|
||||
}
|
||||
|
||||
void AssertRun(ulong baseAddress, ulong firstPage, ulong pages, uint protect)
|
||||
{
|
||||
Assert.NotEqual(0u, (uint)HostMemory.Query((void*)(baseAddress + firstPage * page + 8), out var info));
|
||||
Assert.Equal(baseAddress + firstPage * page, info.BaseAddress);
|
||||
Assert.Equal(pages * page, info.RegionSize);
|
||||
Assert.Equal(protect, info.Protect);
|
||||
}
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue