Merge pull request #4958 from milescoviello/carry/flock-advisory-windows-4957

fix(libc): make flock advisory on Windows so other descriptors can read and write

core/libs/prx/libc/src/FileLock.cpp

@@ -15,6 +15,8 @@ constexpr int LockShared = 1;
 constexpr int LockExclusive = 2;
 constexpr int LockNonblocking = 4;
 constexpr int LockUnlock = 8;
+constexpr DWORD LockByteLow = 0xFFFFFFFE;
+constexpr DWORD LockByteHigh = 0x7FFFFFFF;
 
 struct LockState {
     std::mutex mutex;
@@ -55,14 +57,18 @@ int GuestFileLock_nid_no_patch(int fd, int operation) {
     if (state->mode == mode) return 0;
     if (state->mode != 0) {
         OVERLAPPED overlapped{};
-        if (!::UnlockFileEx(handle, 0, MAXDWORD, MAXDWORD, &overlapped)) return -1;
+        overlapped.Offset = LockByteLow;
+        overlapped.OffsetHigh = LockByteHigh;
+        if (!::UnlockFileEx(handle, 0, 1, 0, &overlapped)) return -1;
         state->mode = 0;
     }
     if (mode == LockUnlock) return 0;
     DWORD flags = mode == LockExclusive ? LOCKFILE_EXCLUSIVE_LOCK : 0;
     if (operation & LockNonblocking) flags |= LOCKFILE_FAIL_IMMEDIATELY;
     OVERLAPPED overlapped{};
-    if (!::LockFileEx(handle, flags, 0, MAXDWORD, MAXDWORD, &overlapped)) return -1;
+    overlapped.Offset = LockByteLow;
+    overlapped.OffsetHigh = LockByteHigh;
+    if (!::LockFileEx(handle, flags, 0, 1, 0, &overlapped)) return -1;
     state->mode = mode;
     return 0;
 }

core/libs/tests/GuestFileLock.cpp

@@ -20,6 +20,9 @@ int APS5_VABI dup_nid_postfix(int);
 int APS5_VABI dup2_nid_postfix(int, int);
 int APS5_VABI fcntl_nid_postfix(int, int, ...);
 int APS5_VABI flock_nid_postfix(int, int);
+std::int64_t APS5_VABI sceKernelWrite(int, const void*, std::size_t);
+std::int64_t APS5_VABI sceKernelPread(int, void*, std::size_t, std::int64_t);
+std::int64_t APS5_VABI sceKernelPwrite(int, const void*, std::size_t, std::int64_t);
 int* APS5_VABI __error_nid_postfix();
 FileStream* APS5_VABI fopen_nid_postfix(const char*, const char*);
 FileStream* APS5_VABI fdopen_nid_postfix(int, const char*);
@@ -165,6 +168,12 @@ static void Conversions() {
         Require(flock_nid_postfix(copy, 2 | 4) == -1 && *__error_nid_postfix() == 35);
         Require(flock_nid_postfix(fixture.other, 8) == 0);
         Require(Acquires(fixture.other, 2));
+        Require(flock_nid_postfix(fixture.source, 2 | 4) == 0);
+        char byte = 'x';
+        Require(sceKernelPwrite(fixture.other, &byte, 1, 0) == 1 && sceKernelWrite(fixture.other, "yz", 2) == 2);
+        Require(sceKernelPread(fixture.other, &byte, 1, 1) == 1 && byte == 'z');
+        Require(flock_nid_postfix(fixture.other, 1 | 4) == -1 && *__error_nid_postfix() == 35);
+        Require(flock_nid_postfix(fixture.source, 8) == 0);
     }
 }
 

docs/dev/debt/libc.md

@@ -63,7 +63,7 @@
 - [daemon](../../../core/libs/prx/libc/src/Process.cpp) (libc) throws: forking into the background is not supported. PPSA85300 imports it and fails to load without it. VLC 3.0 calls it only when its `daemon` option is set ([libvlc.c](https://github.com/videolan/vlc/blob/6de05adcbaf2e8b85fe86aad4169393098628119/src/libvlc.c#L190-L197)), and the title starts libVLC with `--ignore-config` and without `--daemon` ([player.cc](https://github.com/ZiZc3/VLC/blob/f3253020a64368153233ede29183014c0b6c8979/src/player.cc#L960-L981))
 - [Call tracing](../../../core/libs/prx/libc/src/specifics/windows/CallTrace.cpp) (`APS5_TRACE_CALLS`) exists on Windows only and records the entry of a call, not its arguments or its result. Nothing passes imports through `TraceImport_nid_no_patch` yet, so no title is traced. One process-wide ring keeps the last 65536 calls, so a thread that calls often pushes the others out. Calls are also counted per traced target since start-up; an import bound to a target already traced under another name is counted under the first name. The thunk saves the integer argument registers and `xmm0` to `xmm7`; arguments in wider vector registers pass only as long as the recording code stays free of AVX instructions, which is not tested
 - [_WStoul](../../../core/libs/prx/libc/src/Strings.cpp), [_Iswctype](../../../core/libs/prx/libc/src/RuneLocale.cpp), [swscanf_s and swscanf](../../../core/libs/prx/libc/src/ScanningWide.cpp) (libc) - `_WStoul` is `wcstoul` and throws for a base other than 0 or 2 to 36. `_Iswctype` takes the class numbers of Dinkumware's [xwcc.h](https://github.com/superna9999/ti-cgt6x/blob/cgt6x_7_3_4/include/xwcc.h) (1 `alnum` to 11 `xdigit`, 12 `blank`) and throws for any other value, including the FreeBSD masks `wctype` returns; the numbering is not confirmed on the console's `libc.prx`. `swscanf_s` follows C11 7.29.2.2 and K.3.9.1.5, but reads the size argument as `unsigned int` like `sscanf_s` where K.3.9.1.5 specifies `rsize_t`; the console's width is unverified. A runtime-constraint violation (null buffer, format or receiving pointer) only returns `EOF`, and a too small array gets a null first element as in `sscanf_s`. A literal mismatch is a matching failure and a field that is only a prefix, such as `0x` or `1e`, does not convert, as C11 requires; FreeBSD's `vfwscanf` reports an input failure for the first and converts the second. Integers out of range are truncated from `strtoll`/`strtoull`. It throws for `%p`, `%S`, `%C`, unknown length modifiers, scanset ranges, `%n` with suppression or width, and narrow `%s`, `%c` or `%[` input that the C-locale `wcrtomb` cannot convert. `swscanf` runs the same scanner without the size arguments of `%s`, `%c` and `%[`, as FreeBSD's [swscanf.c](https://github.com/freebsd/freebsd-src/blob/d106002967eff712894ab8b5337bbbff106ce5e0/lib/libc/stdio/swscanf.c) passes its arguments to `vswscanf`, so everything above except the sizes applies to it; it also keeps the null buffer, format and receiving pointer checks returning `EOF`, where C leaves those undefined. Neither variant is confirmed on a PS5 title
-- [flock](../../../core/libs/prx/libc/src/FileLock.cpp) (libkernel) on Windows shares one lock state between duplicated descriptors, following FreeBSD 11.4's [flock(2)](https://github.com/freebsd/freebsd-src/blob/releng/11.4/lib/libc/sys/flock.2). Independent opens have separate states; closing a descriptor or reopening a stream drops only its reference. Lock transitions on one open file description are serialized. The host's [LockFileEx](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex) locks also restrict ordinary reads and writes through other handles, unlike FreeBSD's advisory locks, and do not restrict mapped views. Console behavior, signal interruption and concurrent closure of a descriptor while its lock call is pending are unverified.
+- [flock](../../../core/libs/prx/libc/src/FileLock.cpp) (libkernel) on Windows shares one lock state between duplicated descriptors, following FreeBSD 11.4's [flock(2)](https://github.com/freebsd/freebsd-src/blob/releng/11.4/lib/libc/sys/flock.2). Independent opens have separate states; closing a descriptor or reopening a stream drops only its reference. Lock transitions on one open file description are serialized. The lock is a host [LockFileEx](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex) lock on one byte at offset 2^63 - 2, which no file reaches, so it excludes other `flock` calls and leaves reads, writes and mapped views alone, like FreeBSD's advisory locks. Console behavior, signal interruption and concurrent closure of a descriptor while its lock call is pending are unverified.
 - [std::_Throw_Cpp_error](../../../core/libs/prx/libc/src/exception/StandardExceptions.cpp) (libSceAmpr) follows Dinkumware's `thread0.cpp` as shipped with [Visual C++ 2013](https://github.com/ganboing/crt.vc12/blob/f1d47126bf13d1c9dccaa6c53b72217fd0474968/src/thread0.cpp) and in [microsoft/STL](https://github.com/microsoft/STL/blob/da52dc0fcadbd83690cfd1ed32408d2847439493/stl/src/thread0.cpp), without console verification. `_Cpp_error` codes 0 to 6 throw a guest `std::system_error` whose `error_code` holds the FreeBSD errno of the `errc` value (EBUSY 16, EINVAL 22, ESRCH 3, ENOMEM 12, EPERM 1, EDEADLK 11, EAGAIN 35) and the generic category, and whose `what()` is the table message, `": "` and the generic category's message for that errno, selected by the first matching errno entry in VC12's [`syserror.cpp`](https://github.com/ganboing/crt.vc12/blob/f1d47126bf13d1c9dccaa6c53b72217fd0474968/src/syserror.cpp); another code throws `std::out_of_range` on the host instead of reading past the table. The guest `system_error` is taken to be [VC12's](https://github.com/ganboing/crt.vc12/blob/f1d47126bf13d1c9dccaa6c53b72217fd0474968/src/system_error): the 16-byte `runtime_error` followed by `error_code {int, const error_category*}`, 32 bytes, so its copy constructor and assignment copy the code too. The generic category is a static libc object with VC12's `error_category` virtual order (destructors, `name`, `message`, `default_error_condition`, the two `equivalent`), not the console's `_Error_objects<int>::_Generic_object`, and the new `generic_category()` export returns that object; a title's inline category object in a bundled libc can still have a different identity. Its `message` and direct message export construct the existing [GuestLocale::String](../../../core/libs/prx/libc/include/GuestLocale.hpp) model: a reserved eight-byte word, 16-byte inline buffer, length and capacity, with guest `operator new` storage above 15 bytes. This 40-byte string layout and virtual return ABI have not been verified against a console library. The deleting destructor still throws. `_Syserror_map` covers the 72 distinct Dinkumware `errc` values present in FreeBSD 11.4 [errno.h](https://github.com/freebsd/freebsd-src/blob/releng/11.4/sys/sys/errno.h), returning null for other values; category messages use `unknown error` for those. First-match aliases are `EAGAIN`/`EWOULDBLOCK` -> `operation would block` and `ENOTSUP`/`EOPNOTSUPP` -> `not supported`. STREAMS-only errno assignments absent from that header (`ENODATA`, `ENOSR`, `ENOSTR`, `ETIME`) are not inferred
 - [Call tracing](../../../core/libs/prx/libc/src/specifics/windows/CallTrace.cpp) (`--trace-imports`, `APS5_TRACE_CALLS`) exists on Windows only and records the entry of a call, not its arguments or its result. One process-wide ring keeps the last 65536 calls, so a thread that calls often pushes the others out. Calls are also counted per traced target since start-up; an import bound to a target already traced under another name is counted under the first name. The thunk saves the integer argument registers and `xmm0` to `xmm7`; arguments in wider vector registers pass only as long as the recording code stays free of AVX instructions, which is not tested
 - [_Dtest](../../../core/libs/prx/libc/src/Math.cpp) (libc) follows Dinkumware double classification matching _FDtest: finite values return -1, subnormals -2, zeros 0, infinities 1, and NaNs 2.