fix(libc): size collate<char>::do_transform strings as the console does

collate<char>::do_transform gave the result std::string a capacity of
exactly its length and threw for results of 4095 characters or more.
The console's libc.prx grows the string from its inline capacity of 15
to length | 15, so a 26-character result has capacity 31 and a 32-byte
allocation. Once capacity + 1 reaches 0x1000 bytes it takes the
over-aligned large allocation that the guest's inlined deallocation
expects: operator new(bytes + 0x27), the data aligned to 32 bytes and
the raw pointer stored just before it. This is the scheme of
Dinkumware's basic_string (_Newsize | _ALLOC_MASK) and of its
_Allocate in <xmemory0> (_BIG_ALLOCATION_THRESHOLD 4096,
_BIG_ALLOCATION_ALIGNMENT 32, _NON_USER_SIZE sizeof(void*) + 31),
including its bad_alloc when bytes + 0x27 would wrap.

do_transform now uses that capacity and allocation; the throw for long
results is gone. The debt entry in docs/dev/debt/libc.md now records
the capacity and large storage as unverified on hardware. guest_collate
checks the capacity and allocation size of a 26-character result, the
last small result (4079 characters, a 4080-byte block) and the first
large one (4080 characters, capacity 4095, a 4096 + 0x27-byte block
with the data aligned to 32 and the raw pointer before it). With main's
LocaleSupport.cpp it aborts at the allocation-size check of the
26-character result (lastSize 27 instead of 32).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

core/libs/prx/libc/src/LocaleSupport.cpp

@@ -8,6 +8,7 @@
 #include <ios>
 #include <locale>
 #include <mutex>
+#include <new>
 #include <atomic>
 #include <cstring>
 #include <vector>
@@ -81,12 +82,24 @@ GuestLocale::Facet* g_classicFacets[1]{};
 GuestLocale::Implementation g_classicLocale{{&g_localeVtable, 1, 0}, g_classicFacets, 1, 0, false, "C"};
 
 constexpr std::size_t GuestStringInlineCapacity = 15;
+constexpr std::size_t GuestStringCapacityMask = 15;
 constexpr std::size_t GuestStringLargeAllocation = 0x1000;
+constexpr std::size_t GuestLargeAllocationPadding = 0x27;
+constexpr std::uintptr_t GuestLargeAllocationAlignment = 0x20;
 
 void ValidateCharacterRange(const char* first, const char* last, const char* function) {
     if ((first == nullptr) != (last == nullptr) || last < first) throw std::invalid_argument(std::string(function) + ": invalid character range");
 }
 
+void* AllocateStringStorage(std::size_t bytes) {
+    if (bytes < GuestStringLargeAllocation) return _Znwm_nid_postfix(bytes);
+    if (bytes > std::numeric_limits<std::size_t>::max() - GuestLargeAllocationPadding) throw std::bad_alloc();
+    auto* raw = static_cast<unsigned char*>(_Znwm_nid_postfix(bytes + GuestLargeAllocationPadding));
+    const auto aligned = (reinterpret_cast<std::uintptr_t>(raw) + GuestLargeAllocationPadding) & ~(GuestLargeAllocationAlignment - 1);
+    reinterpret_cast<void**>(aligned)[-1] = raw;
+    return reinterpret_cast<void*>(aligned);
+}
+
 void APS5_VABI DestroyCollate(GuestLocale::Facet*) {}
 
 void APS5_VABI DeleteCollate(GuestLocale::Facet* self) {
@@ -121,12 +134,11 @@ GuestLocale::String* APS5_VABI CollateTransform(GuestLocale::String* result, con
     ValidateCharacterRange(first, last, "collate::do_transform");
     const auto length = static_cast<std::size_t>(last - first);
     if (length != 0 && std::memchr(first, 0, length) != nullptr) throw std::runtime_error("collate::do_transform: embedded null characters are not supported");
-    if (length + 1 >= GuestStringLargeAllocation) throw std::runtime_error("collate::do_transform: results of 4095 characters or more are not supported");
     char* data = result->buffer;
     std::size_t capacity = GuestStringInlineCapacity;
     if (length > GuestStringInlineCapacity) {
-        capacity = length;
-        data = static_cast<char*>(_Znwm_nid_postfix(capacity + 1));
+        capacity = length | GuestStringCapacityMask;
+        data = static_cast<char*>(AllocateStringStorage(capacity + 1));
     }
     if (length != 0) std::memcpy(data, first, length);
     data[length] = 0;

core/libs/tests/GuestCollate.cpp

@@ -127,21 +127,28 @@ int main() {
 
     const std::string longText = "collation of a longer text";
     require(Vtable(facet).transform(&result, collate, longText.data(), longText.data() + longText.size()) == &result);
-    require(allocations == 2 && lastSize == longText.size() + 1 && result.pointer == lastAllocation);
-    require(result.size == longText.size() && result.capacity == longText.size() && std::string(result.pointer) == longText);
+    require(allocations == 2 && lastSize == 32 && result.pointer == lastAllocation);
+    require(result.size == longText.size() && result.capacity == 31 && std::string(result.pointer) == longText);
     std::free(result.pointer);
 
-    const std::string largest(4094, 'z');
-    require(Vtable(facet).transform(&result, collate, largest.data(), largest.data() + largest.size()) == &result);
-    require(allocations == 3 && lastSize == 4095 && result.capacity == 4094 && std::string(result.pointer) == largest);
+    const std::string largestSmall(4079, 'z');
+    require(Vtable(facet).transform(&result, collate, largestSmall.data(), largestSmall.data() + largestSmall.size()) == &result);
+    require(allocations == 3 && lastSize == 4080 && result.pointer == lastAllocation && result.capacity == 4079 && std::string(result.pointer) == largestSmall);
     std::free(result.pointer);
 
-    const std::string tooLong(4095, 'z');
-    reject([&] { Vtable(facet).transform(&result, collate, tooLong.data(), tooLong.data() + tooLong.size()); });
+    const std::string smallestLarge(4080, 'y');
+    require(Vtable(facet).transform(&result, collate, smallestLarge.data(), smallestLarge.data() + smallestLarge.size()) == &result);
+    const auto raw = reinterpret_cast<std::uintptr_t>(lastAllocation);
+    const auto aligned = reinterpret_cast<std::uintptr_t>(result.pointer);
+    require(allocations == 4 && lastSize == 4096 + 0x27);
+    require(aligned % 0x20 == 0 && aligned >= raw + 8 && aligned <= raw + 0x27 && reinterpret_cast<void**>(result.pointer)[-1] == lastAllocation);
+    require(result.size == 4080 && result.capacity == 4095 && std::string(result.pointer) == smallestLarge);
+    std::free(lastAllocation);
+
     const std::string embedded("a\0b", 3);
     reject([&] { Vtable(facet).transform(&result, collate, embedded.data(), embedded.data() + embedded.size()); });
     reject([&] { Vtable(facet).transform(nullptr, collate, text, text + 1); });
-    require(allocations == 3);
+    require(allocations == 4);
 
     Vtable(facet).facet.retain(facet);
     Vtable(facet).facet.retain(facet);

docs/dev/debt/libc.md

@@ -46,7 +46,7 @@
 - [gethostname](../../../core/libs/prx/libc/src/Hostname.cpp) (libc) reports the fixed guest name `AnyPS5` without querying the host machine name. Buffer copying, null queries and `ENAMETOOLONG` follow [FreeBSD gethostname](https://github.com/freebsd/freebsd-src/blob/main/lib/libc/gen/gethostname.c); the console's configurable hostname and its integration with `sysctl` and name resolution are unverified and not implemented.
 - [wcsrtombs_s](../../../core/libs/prx/libc/src/Multibyte.cpp) (libc) follows the console's `libc.prx`, but a runtime-constraint violation only returns its error: the default handler there writes the message and a newline to `stderr` before returning, which is not done here. Conversion uses the C locale of `wcrtomb` (`MB_CUR_MAX` 1), so the path where a multibyte character does not fit the remaining space is not reached
 - [The narrow printf family](../../../core/libs/prx/libc/src/Formatting.cpp) (libc) converts 16-bit guest wide arguments to UTF-8 on both platforms, independently of the host locale. It does not use the guest's single-byte `wcrtomb`; the console's treatment of invalid UTF-16 and locale-specific encoding remains unverified. Linux retains host formatting for formats without wide arguments or `%m`, so a narrow `%p` there still follows the host (glibc prints a null pointer as `(nil)`, where FreeBSD prints `0x0`); argument positions above 99 (FreeBSD's `NL_ARGMAX`) throw when a format requires the guest formatter
-- [collate<char>::_Getcat](../../../core/libs/prx/libc/src/LocaleSupport.cpp) (libc) builds the facet for the C locale only and throws for any other locale name. The console's `libc.prx` stores pointers to its collation state tables at offsets 0x10 and 0x18 of the facet; ours leaves them null, as only the facet's own functions read them. `do_transform` throws for input with embedded null characters, which the console drops depending on the output buffer size, and for results of 4095 characters or more, whose string storage uses the guest's over-aligned large allocation.
+- [collate<char>::_Getcat](../../../core/libs/prx/libc/src/LocaleSupport.cpp) (libc) builds the facet for the C locale only and throws for any other locale name. The console's `libc.prx` stores pointers to its collation state tables at offsets 0x10 and 0x18 of the facet; ours leaves them null, as only the facet's own functions read them. `do_transform` throws for input with embedded null characters, which the console drops depending on the output buffer size. Its result capacity (`length | 15`) and the over-aligned storage of results of 4080 characters or more follow the console's `libc.prx` code and are not checked on hardware.
 - [The scanf family](../../../core/libs/prx/libc/src/Formatting.cpp) (libc), the `_s` forms included, on Linux throws for the 16-bit wide conversions (`%ls`, `%lc`, `%l[`, `%S`, `%C`): glibc would store a 32-bit `wchar_t` per character into the guest's 16-bit buffer. Windows hands `%ls`, `%lc` and `%l[` to the host, whose `wchar_t` is 16 bits
 - [The scanf family](../../../core/libs/prx/libc/include/WindowsScanning.hpp) (libc) on Windows returns 0, as C17 and FreeBSD do, when input fails after only suppressed conversions have completed (`sscanf("12", "%*d%d", &value)`). Linux hands the format to glibc, which returns `EOF` there. On Windows the host stops a `%x`, `%X`, `%i` or `%p` after the `x` of a `0x` that no hex digit follows (or that the width cuts off), and a float conversion (`%e`, `%f`, `%g`, `%a`) after the `e` of an exponent with no digits (`1e`), where C17 and FreeBSD stop before the `x` or the `e`
 - [Guest path resolution](../../../core/libs/prx/libc/src/General.cpp) falls back to ASCII case-insensitive component lookup, following [shadPS4's host filesystem](https://github.com/shadps4-emu/shadPS4/blob/945dbc3cc3eee80ac3e053b438502ed936fa6bb2/src/core/file_sys/backends/host_fs.cpp). PPSA03747 requests `Skyrim.INI`, `Data/Video` and lowercase archive/plugin names that differ from the dump's filenames. Exact host names take precedence; multiple fallback matches throw instead of selecting an arbitrary file. Existing parent names are resolved when creating a new file, and guest mount aliases use the same ASCII comparison. Results are not cached, so renames and newly created files remain visible. Unicode case folding and PS5 behaviour for case-colliding host entries are unverified; host roots are used as supplied. Explicit native Windows drive and UNC paths bypass guest parent-component validation.