feat(ghidra): decode Swift String literals in function dossiers (#2049)
* feat(ghidra): decode Swift String literals in function dossiers Swift passes a String as two words. Small literals keep their UTF-8 bytes inside those words, and immortal large literals keep a count plus a pointer to their bytes biased by 32. Ghidra creates no data reference for either, so analyze_function reported no strings for Swift code. On AArch64 programs that contain Swift metadata, the Ghidra bridge now runs constant propagation over the function and decodes register pairs where Swift consumes a String: consecutive argument registers at calls and tail calls, x0 and x1 at returns, and both registers of one STP. Large literals must resolve to terminated bytes inside one initialized block that is not executable. A pair whose registers both come from conditional selects is skipped, because Ghidra forks each select without correlating the condition and could otherwise pair words from different cases. Decoded values are a derivation from the ABI layout rather than observed references, so they appear in a new optional swift_string_literals field with their source instruction, registers and raw words, and in a matching inspect_analysis_view facet. referenced_strings is unchanged. Refs #2030 * fix(ghidra): bound Swift literal reads and compare decoded strings Large immortal literals are measured against the remaining heap before allocation, and a failed fit stays visible as a limitation. The canonical empty string is kept. Small and large literals now require matching address nullability, and function comparison treats decoded literals as string evidence without equating a missing record to an empty one. * fix(ghidra): budget Swift literal decoding and tighten use sites Large literals now draw on one heap budget per dossier, measured once before decoding, instead of a separate check per literal. A literal that does not fit is listed in a new swift_string_literal_omissions field with its address, byte length and reason, so consumers read omissions from structured data rather than from limitation text. strings_names comparison checks referenced strings and names first and reports a change there even when Swift literal coverage differs; it returns unknown only when those facts match and literal coverage is absent on one side or partial. A computed jump counts as a tail call only when it has no resolved target inside the function, so jump tables no longer qualify, and a return requires a RETURN p-code operation, so traps no longer qualify. Decoded literals keep control characters, which Swift strings may contain and JSON escapes. The Swift verifier builds the literal fixture for arm64 on any Mac host and now also covers the empty string and a literal containing a tab. * fix(ghidra): charge escaped JSON and follow copied selects The Swift literal budget now charges the worst case per UTF-8 byte: UTF-16 text, six escaped JSON characters held in Gson's growing buffer and in the returned String, and the UTF-8 bytes written to the socket. Literals made of control characters can no longer exceed the reserved heap. A jump counts as a tail call only when every resolved target lies outside the function. The conditional select check follows register copies back to their definition, so selects copied into argument registers are recognized like selects written there directly. strings_names reports a change when both dossiers decoded different literals at the same use site, even when either side has omissions; it returns unknown only when no such use site disagrees. * fix(ghidra): bound literal records and trace selects across branches Every Swift literal record now pays a fixed cost against the dossier budget in addition to its text, so many empty or small literals are bounded as well. The first record that does not fit is kept as an omission and decoding stops there, which keeps the omission record itself bounded; omissions now describe small or large literals. The conditional select check searches every predecessor path for the reaching definitions, following register copies, so selects made before a branch into the use block are recognized. A dossier records Swift literals and their omissions together. Coverage reads omissions first, and strings_names compares every use site that neither side omitted: a decoded literal is known where it was recorded, and an absence is known only in a dossier whose decoding examined every use site. * fix(ghidra): stop every flow at the literal budget and route omissions Ending one flow from the evaluator leaves other queued flows running, so the budget now records a shared stop that every flow checks before it examines another instruction. A record that exceeds the budget is routed by that capacity result rather than by whether text was decoded, so a small literal over budget becomes an omission instead of a literal. When coverage is partial, the per site comparison keeps each literal's representation as well as its value, matching the full strings_names projection. * fix(ghidra): keep unresolved Swift tail-call sites unknown --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
This commit is contained in:
Omisakin Joshua
GitHub
parent
2f3dee3fcf
commit
8823c55b0c
|
|
@ -13,6 +13,8 @@ import java.net.UnixDomainSocketAddress;
|
|||
import java.nio.channels.Channels;
|
||||
import java.nio.channels.ServerSocketChannel;
|
||||
import java.nio.channels.SocketChannel;
|
||||
import java.nio.ByteBuffer;
|
||||
import java.nio.charset.CharacterCodingException;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.AtomicMoveNotSupportedException;
|
||||
import java.nio.file.Files;
|
||||
|
|
@ -22,7 +24,9 @@ import java.nio.file.StandardCopyOption;
|
|||
import java.nio.file.StandardOpenOption;
|
||||
import java.nio.file.attribute.PosixFilePermissions;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.ArrayDeque;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Arrays;
|
||||
import java.util.Comparator;
|
||||
import java.util.HashSet;
|
||||
import java.util.IdentityHashMap;
|
||||
|
|
@ -82,6 +86,7 @@ import ghidra.program.model.address.SegmentedAddress;
|
|||
import ghidra.program.model.reloc.Relocation;
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import ghidra.program.model.symbol.Namespace;
|
||||
import ghidra.program.model.symbol.FlowType;
|
||||
import ghidra.program.model.symbol.RefType;
|
||||
import ghidra.program.model.symbol.Reference;
|
||||
import ghidra.program.model.symbol.ReferenceIterator;
|
||||
|
|
@ -100,6 +105,9 @@ import ghidra.program.model.pcode.PcodeBlock;
|
|||
import ghidra.program.model.pcode.PcodeBlockBasic;
|
||||
import ghidra.program.model.pcode.Varnode;
|
||||
import ghidra.program.model.pcode.VarnodeAST;
|
||||
import ghidra.program.util.ContextEvaluatorAdapter;
|
||||
import ghidra.program.util.SymbolicPropogator;
|
||||
import ghidra.program.util.VarnodeContext;
|
||||
|
||||
public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private static final int MAX_HIGH_PCODE_OPS = 3000;
|
||||
|
|
@ -1319,6 +1327,13 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
result.add("outgoing_references", outgoing);
|
||||
result.add("referenced_strings", referencedStrings);
|
||||
result.add("referenced_names", referencedNames);
|
||||
boolean swift = swiftProgram();
|
||||
boolean aarch64 = currentProgram.getLanguageID().getIdAsString().startsWith("AARCH64:");
|
||||
SwiftStringLiteralDecode swiftLiterals = swift && aarch64 ? swiftStringLiterals(function) : null;
|
||||
if (swiftLiterals != null) {
|
||||
result.add("swift_string_literals", swiftLiterals.literals());
|
||||
result.add("swift_string_literal_omissions", swiftLiterals.omissions());
|
||||
}
|
||||
result.add("basic_blocks", blocks);
|
||||
result.add(
|
||||
"native_api",
|
||||
|
|
@ -1338,6 +1353,20 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
limitations.add(
|
||||
"Synthetic Ghidra entry-point references without actionable memory sources are omitted."
|
||||
);
|
||||
if (swift && aarch64) {
|
||||
limitations.add(
|
||||
"Swift String literals are decoded from constant register pairs at calls, returns and paired stores; literals stored one word at a time, chosen by conditional selects, passed through unresolved jumps, or built at runtime are not reported."
|
||||
);
|
||||
if (swiftLiterals.omissions().size() > 0) {
|
||||
limitations.add(
|
||||
"Swift String literal records listed in swift_string_literal_omissions could not be retained or decoded within this dossier's remaining heap budget; later symbolic flows remain unexamined."
|
||||
);
|
||||
}
|
||||
} else if (swift) {
|
||||
limitations.add(
|
||||
"Swift String literal decoding is implemented for AArch64 register pairs only; no Swift literals were decoded for this architecture."
|
||||
);
|
||||
}
|
||||
List<String> boundaryLimitations = noReturnCallLimitations(scan.instructions);
|
||||
for (String limitation : boundaryLimitations) {
|
||||
limitations.add(limitation);
|
||||
|
|
@ -3391,6 +3420,349 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
return result;
|
||||
}
|
||||
|
||||
private boolean swiftProgram() {
|
||||
for (MemoryBlock block : currentProgram.getMemory().getBlocks()) {
|
||||
String name = block.getName();
|
||||
if (name.startsWith("__swift5_") || name.startsWith("swift5_")) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Swift passes a String as two words: small literals carry their bytes in the words and
|
||||
* immortal large literals carry their count and a biased pointer to the bytes. Ghidra keeps
|
||||
* neither as a data reference, so decode constant pairs where Swift consumes them: argument
|
||||
* and return registers at calls, tail calls and returns, and both registers of one STP.
|
||||
* The canonical empty string is a small literal with a zero count.
|
||||
*/
|
||||
private SwiftStringLiteralDecode swiftStringLiterals(Function function) throws Exception {
|
||||
Register[] registers = new Register[31];
|
||||
for (int index = 0; index < registers.length; index++) {
|
||||
registers[index] = currentProgram.getRegister("x" + index);
|
||||
}
|
||||
TreeMap<String, JsonObject> observed = new TreeMap<>();
|
||||
TreeMap<String, JsonObject> omitted = new TreeMap<>();
|
||||
SwiftLiteralBudget budget = new SwiftLiteralBudget();
|
||||
Map<String, Boolean> selectedValues = new HashMap<>();
|
||||
SymbolicPropogator propagator = new SymbolicPropogator(currentProgram, false);
|
||||
propagator.flowConstants(function.getEntryPoint(), function.getBody(), new ContextEvaluatorAdapter() {
|
||||
@Override
|
||||
public boolean evaluateContextBefore(VarnodeContext context, Instruction instruction) {
|
||||
// Returning true ends only the current flow, so every queued flow checks the shared stop.
|
||||
if (budget.stopped()) return true;
|
||||
for (Register[] pair : swiftStringPairs(function, instruction, registers)) {
|
||||
Long word = registerConstant(context, pair[0]);
|
||||
Long object = registerConstant(context, pair[1]);
|
||||
if (word == null || object == null) continue;
|
||||
SwiftLiteralLayout layout = swiftLiteralLayout(word, object);
|
||||
if (layout == null) continue;
|
||||
// A selected word can combine with a path-dependent ordinary move. Cache the
|
||||
// provenance question per site/register, charging retained cache state to the
|
||||
// same dossier budget before allocating it.
|
||||
boolean selected = false;
|
||||
boolean provenanceOverCapacity = false;
|
||||
for (Register register : pair) {
|
||||
String provenanceKey = canonicalAddress(instruction.getAddress()) + "\u0000" +
|
||||
register.getBaseRegister().getName();
|
||||
Boolean value = selectedValues.get(provenanceKey);
|
||||
if (value == null) {
|
||||
if (!budget.takeState()) {
|
||||
provenanceOverCapacity = true;
|
||||
break;
|
||||
}
|
||||
value = selectedValue(function, instruction, register);
|
||||
selectedValues.put(provenanceKey, value);
|
||||
}
|
||||
if (value) {
|
||||
selected = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (selected) continue;
|
||||
// Forks can reach one site with different constants, so the words are part of the key.
|
||||
String key = canonicalAddress(instruction.getAddress()) + "\u0000" + pair[0].getName() +
|
||||
"\u0000" + pair[1].getName() + "\u0000" + Long.toUnsignedString(word, 16) +
|
||||
"\u0000" + Long.toUnsignedString(object, 16);
|
||||
if (observed.containsKey(key) || omitted.containsKey(key)) continue;
|
||||
// Every record draws on the budget; the first one that does not fit is kept as an
|
||||
// omission and decoding stops, so later sites are never examined.
|
||||
String value = layout.smallValue();
|
||||
boolean overCapacity = provenanceOverCapacity || !budget.take(layout.byteLength());
|
||||
if (!overCapacity && value == null) {
|
||||
try {
|
||||
value = swiftLargeLiteralText(layout);
|
||||
} catch (OutOfMemoryError exhausted) {
|
||||
overCapacity = true;
|
||||
}
|
||||
}
|
||||
if (value == null && !overCapacity) continue;
|
||||
JsonObject item = overCapacity
|
||||
? swiftStringOmission(layout, word, object)
|
||||
: swiftStringItem(value, layout, word, object);
|
||||
item.addProperty("source_address", canonicalAddress(instruction.getAddress()));
|
||||
JsonArray pairRegisters = new JsonArray();
|
||||
pairRegisters.add(pair[0].getName());
|
||||
pairRegisters.add(pair[1].getName());
|
||||
item.add("registers", pairRegisters);
|
||||
(overCapacity ? omitted : observed).put(key, item);
|
||||
if (overCapacity) {
|
||||
budget.stop();
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}, false, monitor);
|
||||
JsonArray literals = new JsonArray();
|
||||
observed.values().forEach(literals::add);
|
||||
JsonArray omissions = new JsonArray();
|
||||
omitted.values().forEach(omissions::add);
|
||||
return new SwiftStringLiteralDecode(literals, omissions);
|
||||
}
|
||||
|
||||
private static List<Register[]> swiftStringPairs(
|
||||
Function function,
|
||||
Instruction instruction,
|
||||
Register[] registers
|
||||
) {
|
||||
List<Register[]> pairs = new ArrayList<>();
|
||||
if (instruction.getMnemonicString().equals("stp")) {
|
||||
Register first = instruction.getRegister(0);
|
||||
Register second = instruction.getRegister(1);
|
||||
if (first != null && second != null && first.getBitLength() == 64 && second.getBitLength() == 64) {
|
||||
pairs.add(new Register[] {first, second});
|
||||
}
|
||||
return pairs;
|
||||
}
|
||||
FlowType flow = instruction.getFlowType();
|
||||
boolean returns = false;
|
||||
for (PcodeOp operation : instruction.getPcode()) {
|
||||
if (operation.getOpcode() == PcodeOp.RETURN) returns = true;
|
||||
}
|
||||
// A jump is a proven tail call only when it has resolved targets and all leave the
|
||||
// body. A targetless computed jump can still dispatch inside the function.
|
||||
Address[] targets = instruction.getFlows();
|
||||
boolean tailCall = flow.isJump() && !flow.isConditional() && targets.length > 0 &&
|
||||
Arrays.stream(targets).allMatch(target -> !function.getBody().contains(target));
|
||||
if (!flow.isCall() && !returns && !tailCall) return pairs;
|
||||
// A String occupies two consecutive registers among x0-x7, or x0 and x1 when returned.
|
||||
int lastFirstRegister = returns ? 0 : 6;
|
||||
for (int index = 0; index <= lastFirstRegister; index++) {
|
||||
pairs.add(new Register[] {registers[index], registers[index + 1]});
|
||||
}
|
||||
return pairs;
|
||||
}
|
||||
|
||||
private static Long registerConstant(VarnodeContext context, Register register) {
|
||||
Varnode value = context.getRegisterVarnodeValue(register);
|
||||
return value != null && context.isConstant(value) ? value.getOffset() : null;
|
||||
}
|
||||
|
||||
// Search every predecessor path for the reaching definitions, following register copies,
|
||||
// since a select copied into an argument register or defined before a branch forks just as
|
||||
// independently as one written there directly.
|
||||
private boolean selectedValue(Function function, Instruction use, Register register) {
|
||||
ArrayDeque<Instruction> instructions = new ArrayDeque<>();
|
||||
ArrayDeque<Register> targets = new ArrayDeque<>();
|
||||
Set<String> visited = new HashSet<>();
|
||||
for (Instruction predecessor : predecessors(function, use)) {
|
||||
instructions.push(predecessor);
|
||||
targets.push(register.getBaseRegister());
|
||||
}
|
||||
while (!instructions.isEmpty()) {
|
||||
Instruction instruction = instructions.pop();
|
||||
Register target = targets.pop();
|
||||
if (!visited.add(canonicalAddress(instruction.getAddress()) + "\u0000" + target.getName())) continue;
|
||||
boolean defines = false;
|
||||
for (Object result : instruction.getResultObjects()) {
|
||||
if (result instanceof Register written && written.getBaseRegister().equals(target)) defines = true;
|
||||
}
|
||||
Register next = target;
|
||||
if (defines) {
|
||||
PcodeOp[] operations = instruction.getPcode();
|
||||
for (PcodeOp operation : operations) {
|
||||
if (operation.getOpcode() == PcodeOp.CBRANCH && operation.getInput(0).isConstant()) return true;
|
||||
}
|
||||
Register source = copiedRegister(operations);
|
||||
if (source == null) continue;
|
||||
next = source.getBaseRegister();
|
||||
}
|
||||
for (Instruction predecessor : predecessors(function, instruction)) {
|
||||
instructions.push(predecessor);
|
||||
targets.push(next);
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private List<Instruction> predecessors(Function function, Instruction instruction) {
|
||||
List<Instruction> result = new ArrayList<>();
|
||||
Address from = instruction.getFallFrom();
|
||||
if (from != null) {
|
||||
Instruction previous = currentProgram.getListing().getInstructionAt(from);
|
||||
if (previous != null) result.add(previous);
|
||||
}
|
||||
for (Reference reference : currentProgram.getReferenceManager().getReferencesTo(instruction.getAddress())) {
|
||||
if (!reference.getReferenceType().isFlow() || reference.getReferenceType().isCall()) continue;
|
||||
if (!function.getBody().contains(reference.getFromAddress())) continue;
|
||||
Instruction source = currentProgram.getListing().getInstructionAt(reference.getFromAddress());
|
||||
if (source != null) result.add(source);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private Register copiedRegister(PcodeOp[] operations) {
|
||||
if (operations.length != 1) return null;
|
||||
int opcode = operations[0].getOpcode();
|
||||
Varnode input = operations[0].getInput(0);
|
||||
if ((opcode != PcodeOp.COPY && opcode != PcodeOp.INT_ZEXT) || input == null || !input.isRegister()) return null;
|
||||
return currentProgram.getRegister(input);
|
||||
}
|
||||
|
||||
/** Validate the ABI layout of a pair; a small literal is decoded here, a large one only located. */
|
||||
private SwiftLiteralLayout swiftLiteralLayout(long word, long object) {
|
||||
int discriminator = (int) (object >>> 60);
|
||||
if (discriminator == 0xE || discriminator == 0xA) {
|
||||
int count = (int) ((object >>> 56) & 0xF);
|
||||
byte[] bytes = new byte[15];
|
||||
for (int index = 0; index < 8; index++) bytes[index] = (byte) (word >>> (8 * index));
|
||||
for (int index = 0; index < 7; index++) bytes[8 + index] = (byte) (object >>> (8 * index));
|
||||
for (int index = count; index < bytes.length; index++) {
|
||||
if (bytes[index] != 0) return null;
|
||||
}
|
||||
// Count zero is the canonical empty string (object 0xE000000000000000, word 0).
|
||||
String value = swiftStringText(bytes, count, discriminator == 0xE);
|
||||
return value == null ? null : new SwiftLiteralLayout("small", null, count, true, value);
|
||||
}
|
||||
int flags = (int) (word >>> 60);
|
||||
long count = word & 0xFFFFFFFFFFFFL;
|
||||
// Immortal literals: ASCII and NFC or neither, tail allocated, with a pointer biased by 32.
|
||||
if ((object >>> 56) != 0x80 || (flags != 0xD && flags != 0x1) || ((word >>> 48) & 0xFFF) != 0) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
Address address = currentProgram.getAddressFactory().getDefaultAddressSpace()
|
||||
.getAddress((object & 0x00FFFFFFFFFFFFFFL) + 0x20);
|
||||
MemoryBlock block = currentProgram.getMemory().getBlock(address);
|
||||
// The bytes and their terminator must lie in one initialized, non-executable block,
|
||||
// and Java arrays bound one read.
|
||||
if (block == null || !block.isInitialized() || block.isExecute() ||
|
||||
count >= block.getEnd().subtract(address) + 1 || count >= Integer.MAX_VALUE) {
|
||||
return null;
|
||||
}
|
||||
return new SwiftLiteralLayout("large", address, count, flags == 0xD, null);
|
||||
} catch (Exception unaddressable) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private String swiftLargeLiteralText(SwiftLiteralLayout layout) {
|
||||
int count = (int) layout.byteLength();
|
||||
byte[] bytes = new byte[count + 1];
|
||||
try {
|
||||
if (currentProgram.getMemory().getBytes(layout.address(), bytes) != bytes.length) return null;
|
||||
} catch (Exception unreadable) {
|
||||
return null;
|
||||
}
|
||||
return bytes[count] == 0 ? swiftStringText(bytes, count, layout.ascii()) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Every literal record of one dossier draws on half of the heap that was free when decoding
|
||||
* began. Text is charged at the worst case per UTF-8 byte: two bytes of UTF-16 text, then up
|
||||
* to six escaped JSON characters ("\\u0000") held as UTF-16 in Gson's growing writer buffer
|
||||
* (up to twice its length), again in the returned String, and once more as UTF-8 on the
|
||||
* socket. Each record also pays a fixed cost for its map key, JSON members, addresses, words
|
||||
* and their serialized form, so many empty or small literals are bounded too. Retained
|
||||
* provenance-cache entries draw on that same fixed-state allowance.
|
||||
*/
|
||||
private static final class SwiftLiteralBudget {
|
||||
private static final long BYTES_PER_LITERAL_BYTE = 2 + 6 * 2 * 2 + 6 * 2 + 6;
|
||||
private static final long BYTES_PER_RECORD = 4096;
|
||||
private long remaining;
|
||||
private boolean stopped;
|
||||
|
||||
SwiftLiteralBudget() {
|
||||
Runtime runtime = Runtime.getRuntime();
|
||||
remaining = Math.max(0, (runtime.maxMemory() - runtime.totalMemory() + runtime.freeMemory()) / 2);
|
||||
}
|
||||
|
||||
boolean takeState() {
|
||||
if (remaining < BYTES_PER_RECORD) return false;
|
||||
remaining -= BYTES_PER_RECORD;
|
||||
return true;
|
||||
}
|
||||
|
||||
boolean take(long bytes) {
|
||||
if (remaining < BYTES_PER_RECORD ||
|
||||
bytes > (remaining - BYTES_PER_RECORD) / BYTES_PER_LITERAL_BYTE) {
|
||||
return false;
|
||||
}
|
||||
remaining -= BYTES_PER_RECORD + bytes * BYTES_PER_LITERAL_BYTE;
|
||||
return true;
|
||||
}
|
||||
|
||||
void stop() {
|
||||
stopped = true;
|
||||
}
|
||||
|
||||
boolean stopped() {
|
||||
return stopped;
|
||||
}
|
||||
}
|
||||
|
||||
// Swift String values may contain any scalar, including controls; JSON escapes them.
|
||||
private static String swiftStringText(byte[] bytes, int length, boolean ascii) {
|
||||
if (length < 0 || length > bytes.length) return null;
|
||||
for (int index = 0; index < length; index++) {
|
||||
if (ascii && bytes[index] < 0) return null;
|
||||
}
|
||||
try {
|
||||
return StandardCharsets.UTF_8.newDecoder().decode(ByteBuffer.wrap(bytes, 0, length)).toString();
|
||||
} catch (CharacterCodingException invalid) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private static JsonObject swiftStringOmission(SwiftLiteralLayout layout, long word, long object) {
|
||||
JsonObject item = new JsonObject();
|
||||
item.addProperty("representation", layout.representation());
|
||||
if (layout.address() == null) {
|
||||
item.add("address", JsonNull.INSTANCE);
|
||||
} else {
|
||||
item.addProperty("address", canonicalAddress(layout.address()));
|
||||
}
|
||||
item.addProperty("byte_length", layout.byteLength());
|
||||
item.addProperty("reason", "capacity");
|
||||
item.add("words", swiftStringWords(word, object));
|
||||
return item;
|
||||
}
|
||||
|
||||
private static JsonObject swiftStringItem(
|
||||
String value,
|
||||
SwiftLiteralLayout layout,
|
||||
long word,
|
||||
long object
|
||||
) {
|
||||
JsonObject item = new JsonObject();
|
||||
item.addProperty("value", value);
|
||||
item.addProperty("representation", layout.representation());
|
||||
if (layout.address() == null) {
|
||||
item.add("address", JsonNull.INSTANCE);
|
||||
} else {
|
||||
item.addProperty("address", canonicalAddress(layout.address()));
|
||||
}
|
||||
item.add("words", swiftStringWords(word, object));
|
||||
return item;
|
||||
}
|
||||
|
||||
private static JsonArray swiftStringWords(long word, long object) {
|
||||
JsonArray words = new JsonArray();
|
||||
words.add("0x" + Long.toUnsignedString(word, 16));
|
||||
words.add("0x" + Long.toUnsignedString(object, 16));
|
||||
return words;
|
||||
}
|
||||
|
||||
private static String exclusiveAddress(Address inclusive) {
|
||||
Address next = inclusive.next();
|
||||
if (next != null && next.getAddressSpace().equals(inclusive.getAddressSpace())) {
|
||||
|
|
@ -3930,6 +4302,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
private record AddressNameRequest(String inputAddress, Address address, String name) {}
|
||||
private record FunctionEntry(Function function, InventoryItem item) {}
|
||||
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations) {}
|
||||
private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {}
|
||||
private record SwiftLiteralLayout(
|
||||
String representation,
|
||||
Address address,
|
||||
long byteLength,
|
||||
boolean ascii,
|
||||
String smallValue
|
||||
) {}
|
||||
private record InstructionScan(List<Instruction> instructions, boolean truncated) {}
|
||||
private record SessionDescriptor(
|
||||
String transport,
|
||||
|
|
|
|||
|
|
@ -45,8 +45,13 @@ const transport = new StdioClientTransport({
|
|||
const source = fileURLToPath(
|
||||
new URL("../tests/conformance/swift/type-inventory.swift", import.meta.url),
|
||||
);
|
||||
const literalSource = fileURLToPath(
|
||||
new URL("../tests/conformance/swift/string-literals.swift", import.meta.url),
|
||||
);
|
||||
const binary = join(root, "swift-inventory");
|
||||
const literalBinary = join(root, "swift-literals");
|
||||
const reports = [];
|
||||
let literalReport;
|
||||
try {
|
||||
await execute("swiftc", ["-O", "-g", source, "-o", binary], {
|
||||
timeout: 60_000,
|
||||
|
|
@ -54,6 +59,19 @@ try {
|
|||
await mkdir(join(root, "copied"));
|
||||
const copied = join(root, "copied", "swift-inventory");
|
||||
await copyFile(binary, copied);
|
||||
// Decoding covers AArch64 register pairs, so build that slice on any Mac host.
|
||||
await execute(
|
||||
"swiftc",
|
||||
[
|
||||
"-O",
|
||||
"-target",
|
||||
"arm64-apple-macosx13.0",
|
||||
literalSource,
|
||||
"-o",
|
||||
literalBinary,
|
||||
],
|
||||
{ timeout: 60_000 },
|
||||
);
|
||||
await client.connect(transport);
|
||||
for (const [variant, path] of [
|
||||
["debug", binary],
|
||||
|
|
@ -132,6 +150,7 @@ try {
|
|||
await call("close_binary", {});
|
||||
}
|
||||
}
|
||||
literalReport = await verifyStringLiterals();
|
||||
} finally {
|
||||
await client.close();
|
||||
await transport.close();
|
||||
|
|
@ -143,6 +162,7 @@ console.log(
|
|||
ok: true,
|
||||
provider: "ghidra",
|
||||
fixtures: reports,
|
||||
swift_string_literals: literalReport,
|
||||
verifier_run: await completeVerifierRun(run),
|
||||
},
|
||||
null,
|
||||
|
|
@ -159,3 +179,51 @@ async function call(name, arguments_) {
|
|||
toolContract(name).outputSchema.parse(response.structuredContent);
|
||||
return requireMcpOperationResult(response, name);
|
||||
}
|
||||
|
||||
// Literal text lives only in register constants, so it must come from the decoded pairs.
|
||||
async function verifyStringLiterals() {
|
||||
await call("open_binary", { path: literalBinary, provider_id: "ghidra" });
|
||||
try {
|
||||
const procedures = await call("list_procedures", {});
|
||||
const expected = {
|
||||
smallLiteral: [["SleepDisabled", "small"]],
|
||||
largeLiteral: [["The power setting could not be changed.", "large"]],
|
||||
unicodeLiterals: [
|
||||
["café", "small"],
|
||||
["Résumé of the naïve décor, written twice", "large"],
|
||||
],
|
||||
emptyAndControlLiterals: [
|
||||
["", "small"],
|
||||
["tab\there", "small"],
|
||||
],
|
||||
returnedLiteral: [["Holding awake", "small"]],
|
||||
};
|
||||
const decoded = {};
|
||||
for (const [name, literals] of Object.entries(expected)) {
|
||||
const procedure = procedures.find(({ value }) => value.includes(name));
|
||||
assert(procedure, `${name} is missing from the procedure inventory`);
|
||||
const dossier = await call("analyze_function", {
|
||||
procedure: procedure.address,
|
||||
});
|
||||
const observed = dossier.swift_string_literals;
|
||||
assert(Array.isArray(observed), `${name} recorded no Swift literals`);
|
||||
assert.deepEqual(dossier.swift_string_literal_omissions, []);
|
||||
assert.deepEqual(
|
||||
observed.map(({ value, representation }) => [value, representation]),
|
||||
literals,
|
||||
);
|
||||
for (const literal of observed) {
|
||||
assert.deepEqual(literal.registers, ["x0", "x1"]);
|
||||
assert.equal(
|
||||
literal.address === null,
|
||||
literal.representation === "small",
|
||||
`${literal.value} storage does not match its representation`,
|
||||
);
|
||||
}
|
||||
decoded[name] = observed.length;
|
||||
}
|
||||
return decoded;
|
||||
} finally {
|
||||
await call("close_binary", {});
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -169,7 +169,8 @@ it("pages every retained dossier collection and the native API record", () => {
|
|||
| "comments"
|
||||
| "unresolved_calls"
|
||||
| "referenced_strings"
|
||||
| "referenced_names",
|
||||
| "referenced_names"
|
||||
| "swift_string_literals",
|
||||
result: unknown = original,
|
||||
) => {
|
||||
const view = projectAnalysisView(nativeParent(result), {
|
||||
|
|
@ -195,6 +196,23 @@ it("pages every retained dossier collection and the native API record", () => {
|
|||
}),
|
||||
).toMatchObject({ item: [{ text: "entry" }] });
|
||||
expect(page("referenced_names")).toMatchObject({ coverage: { total: 0 } });
|
||||
const literal = {
|
||||
source_address: "0x401004",
|
||||
value: "SleepDisabled",
|
||||
representation: "small",
|
||||
address: null,
|
||||
registers: ["x0", "x1"],
|
||||
words: ["0x7369447065656c53", "0xed000064656c6261"],
|
||||
};
|
||||
expect(
|
||||
page("swift_string_literals", {
|
||||
...original,
|
||||
swift_string_literals: [literal],
|
||||
}),
|
||||
).toMatchObject({ item: [literal], coverage: { total: 1 } });
|
||||
expect(page("swift_string_literals")).toMatchObject({
|
||||
item: { available: false, reason: "not recorded" },
|
||||
});
|
||||
const { unresolved_calls: _omitted, ...withoutUnresolved } = original;
|
||||
expect(
|
||||
page("unresolved_calls", {
|
||||
|
|
|
|||
|
|
@ -160,6 +160,8 @@ const nativeViewSchema = z.strictObject({
|
|||
"unresolved_calls",
|
||||
"referenced_strings",
|
||||
"referenced_names",
|
||||
"swift_string_literals",
|
||||
"swift_string_literal_omissions",
|
||||
"native_api",
|
||||
"value_flow_summary",
|
||||
"value_flow_operations",
|
||||
|
|
|
|||
|
|
@ -79,6 +79,8 @@ const nativeRows = (
|
|||
unresolved_calls: dossier.unresolved_calls,
|
||||
referenced_strings: dossier.referenced_strings,
|
||||
referenced_names: dossier.referenced_names,
|
||||
swift_string_literals: dossier.swift_string_literals,
|
||||
swift_string_literal_omissions: dossier.swift_string_literal_omissions,
|
||||
value_flow_operations: flow?.operations,
|
||||
value_flow_def_use: flow?.def_use,
|
||||
value_flow_effects: flow?.effects,
|
||||
|
|
@ -163,7 +165,8 @@ export const projectNativeFunctionView = (
|
|||
singleton({
|
||||
available: false,
|
||||
// Only value-flow facets inherit the value-flow failure reason; an
|
||||
// older dossier simply has no unresolved_calls record.
|
||||
// older dossier simply has no unresolved_calls record, and a target
|
||||
// whose provider does not decode Swift literals has no such record.
|
||||
reason:
|
||||
view.facet.startsWith("value_flow_") && flow?.available === false
|
||||
? flow.reason
|
||||
|
|
|
|||
|
|
@ -716,6 +716,236 @@ describe("function comparison identity and malformed CFG", () => {
|
|||
});
|
||||
});
|
||||
|
||||
const withLiterals = (base: "0x1000" | "0x2000", values: readonly string[]) =>
|
||||
functionDossierSchema.parse({
|
||||
...dossier("return 0;", base),
|
||||
swift_string_literals: values.map((value) => ({
|
||||
source_address: base,
|
||||
value,
|
||||
representation: "small" as const,
|
||||
address: null,
|
||||
registers: ["x0", "x1"] as const,
|
||||
words: ["0x0", "0xe"] as const,
|
||||
})),
|
||||
});
|
||||
const stringsDimension = (
|
||||
leftDigit: string,
|
||||
left: ReturnType<typeof withLiterals> | ReturnType<typeof dossier>,
|
||||
rightDigit: string,
|
||||
right: ReturnType<typeof withLiterals> | ReturnType<typeof dossier>,
|
||||
) =>
|
||||
compareFunctions(
|
||||
observe(leftDigit, left),
|
||||
observe(rightDigit, right),
|
||||
).dimensions.find(({ dimension }) => dimension === "strings_names");
|
||||
|
||||
describe("function comparison Swift string literals", () => {
|
||||
it("treats differing decoded literals as a strings and names change", () => {
|
||||
expect(
|
||||
stringsDimension(
|
||||
"1",
|
||||
withLiterals("0x1000", ["SleepDisabled"]),
|
||||
"2",
|
||||
withLiterals("0x2000", ["Holding awake"]),
|
||||
),
|
||||
).toMatchObject({ status: "changed" });
|
||||
});
|
||||
|
||||
it("matches the same decoded literal after the function is relocated", () => {
|
||||
expect(
|
||||
stringsDimension(
|
||||
"3",
|
||||
withLiterals("0x1000", ["SleepDisabled"]),
|
||||
"4",
|
||||
withLiterals("0x2000", ["SleepDisabled"]),
|
||||
),
|
||||
).toMatchObject({ status: "unchanged" });
|
||||
});
|
||||
|
||||
it("keeps strings and names unknown when only one dossier recorded Swift literals", () => {
|
||||
expect(
|
||||
stringsDimension(
|
||||
"5",
|
||||
withLiterals("0x1000", []),
|
||||
"6",
|
||||
dossier("return 0;", "0x2000"),
|
||||
),
|
||||
).toMatchObject({
|
||||
status: "unknown",
|
||||
limitations: [
|
||||
"Swift String literal coverage differs or is partial; referenced strings and names match, and complete coverage does not establish a differing literal.",
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it("reports a referenced name change even when Swift literal coverage differs", () => {
|
||||
const named = functionDossierSchema.parse({
|
||||
...dossier("return 0;", "0x2000"),
|
||||
referenced_names: [
|
||||
{ address: "0x3000", value: "helper", source_address: "0x2000" },
|
||||
],
|
||||
});
|
||||
expect(
|
||||
stringsDimension("7", withLiterals("0x1000", []), "8", named),
|
||||
).toMatchObject({ status: "changed" });
|
||||
});
|
||||
});
|
||||
|
||||
describe("function comparison partial Swift literal coverage", () => {
|
||||
it("does not equate a partial literal record with a complete one", () => {
|
||||
const partial = functionDossierSchema.parse({
|
||||
...withLiterals("0x2000", ["SleepDisabled"]),
|
||||
swift_string_literal_omissions: [
|
||||
{
|
||||
source_address: "0x2000",
|
||||
representation: "large",
|
||||
address: "0x3000",
|
||||
byte_length: 64,
|
||||
reason: "capacity",
|
||||
registers: ["x0", "x1"],
|
||||
words: ["0xd000000000000040", "0x8000000000002fe0"],
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(
|
||||
stringsDimension(
|
||||
"9",
|
||||
withLiterals("0x1000", ["SleepDisabled"]),
|
||||
"a",
|
||||
partial,
|
||||
),
|
||||
).toMatchObject({ status: "unknown" });
|
||||
});
|
||||
|
||||
it("reports a literal that a complete dossier shows absent at a site the partial one decoded", () => {
|
||||
const partial = functionDossierSchema.parse({
|
||||
...withLiterals("0x2000", ["SleepDisabled"]),
|
||||
swift_string_literal_omissions: [
|
||||
{
|
||||
source_address: "0x2004",
|
||||
representation: "small",
|
||||
address: null,
|
||||
byte_length: 4,
|
||||
reason: "capacity",
|
||||
registers: ["x0", "x1"],
|
||||
words: ["0x732f4220", "0xe400000000000000"],
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(
|
||||
stringsDimension("d", withLiterals("0x1000", []), "e", partial),
|
||||
).toMatchObject({ status: "changed" });
|
||||
});
|
||||
|
||||
it("does not interpret an unexamined path as a missing literal at a retained site", () => {
|
||||
const partial = functionDossierSchema.parse({
|
||||
...withLiterals("0x2000", ["A"]),
|
||||
swift_string_literal_omissions: [
|
||||
{
|
||||
source_address: "0x2004",
|
||||
representation: "small",
|
||||
address: null,
|
||||
byte_length: 0,
|
||||
reason: "capacity",
|
||||
registers: ["x0", "x1"],
|
||||
words: ["0x0", "0xe000000000000000"],
|
||||
},
|
||||
],
|
||||
});
|
||||
const complete = withLiterals("0x1000", ["A", "B"]);
|
||||
expect(stringsDimension("1", complete, "2", partial)).toMatchObject({
|
||||
status: "unknown",
|
||||
});
|
||||
expect(stringsDimension("3", partial, "4", complete)).toMatchObject({
|
||||
status: "unknown",
|
||||
});
|
||||
expect(
|
||||
stringsDimension("5", withLiterals("0x1000", []), "6", partial),
|
||||
).toMatchObject({ status: "changed" });
|
||||
});
|
||||
|
||||
it("treats an omission record without decoded literals as partial coverage", () => {
|
||||
const omissionOnly = functionDossierSchema.parse({
|
||||
...dossier("return 0;", "0x2000"),
|
||||
swift_string_literal_omissions: [
|
||||
{
|
||||
source_address: "0x2004",
|
||||
representation: "small",
|
||||
address: null,
|
||||
byte_length: 4,
|
||||
reason: "capacity",
|
||||
registers: ["x0", "x1"],
|
||||
words: ["0x732f4220", "0xe400000000000000"],
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(
|
||||
stringsDimension("f", dossier("return 0;", "0x1000"), "0", omissionOnly),
|
||||
).toMatchObject({ status: "unknown" });
|
||||
});
|
||||
|
||||
it("keeps different retained representations unknown when both dossiers are partial", () => {
|
||||
const omission = {
|
||||
source_address: "0x1004",
|
||||
representation: "small",
|
||||
address: null,
|
||||
byte_length: 4,
|
||||
reason: "capacity",
|
||||
registers: ["x0", "x1"],
|
||||
words: ["0x732f4220", "0xe400000000000000"],
|
||||
} as const;
|
||||
const small = functionDossierSchema.parse({
|
||||
...withLiterals("0x1000", ["SleepDisabled"]),
|
||||
swift_string_literal_omissions: [omission],
|
||||
});
|
||||
const large = functionDossierSchema.parse({
|
||||
...dossier("return 0;", "0x2000"),
|
||||
swift_string_literals: [
|
||||
{
|
||||
source_address: "0x2000",
|
||||
value: "SleepDisabled",
|
||||
representation: "large",
|
||||
address: "0x3000",
|
||||
registers: ["x0", "x1"],
|
||||
words: ["0xd00000000000000d", "0x8000000000002fe0"],
|
||||
},
|
||||
],
|
||||
swift_string_literal_omissions: [
|
||||
{ ...omission, source_address: "0x2004" },
|
||||
],
|
||||
});
|
||||
expect(stringsDimension("1", small, "2", large)).toMatchObject({
|
||||
status: "unknown",
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps different retained literals unknown when both dossiers are partial", () => {
|
||||
const partial = (base: "0x1000" | "0x2000", value: string) =>
|
||||
functionDossierSchema.parse({
|
||||
...withLiterals(base, [value]),
|
||||
swift_string_literal_omissions: [
|
||||
{
|
||||
source_address: base === "0x1000" ? "0x1004" : "0x2004",
|
||||
representation: "large",
|
||||
address: "0x3000",
|
||||
byte_length: 64,
|
||||
reason: "capacity",
|
||||
registers: ["x0", "x1"],
|
||||
words: ["0xd000000000000040", "0x8000000000002fe0"],
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(
|
||||
stringsDimension(
|
||||
"b",
|
||||
partial("0x1000", "SleepDisabled"),
|
||||
"c",
|
||||
partial("0x2000", "Holding awake"),
|
||||
),
|
||||
).toMatchObject({ status: "unknown" });
|
||||
});
|
||||
});
|
||||
|
||||
const expectInlineComparisonEvidence = (
|
||||
result: ReturnType<typeof compareFunctions>,
|
||||
): void => {
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ import {
|
|||
referenceProjection,
|
||||
sorted,
|
||||
stringAndNameProjection,
|
||||
swiftLiteralSitesConflict,
|
||||
} from "./functionComparisonNormalization.js";
|
||||
import type {
|
||||
DimensionName,
|
||||
|
|
@ -139,8 +140,12 @@ const compareStringsAndNames = (
|
|||
right: FunctionSnapshot,
|
||||
context: ComparisonContext,
|
||||
): FunctionDimension => {
|
||||
const leftValues = stringAndNameProjection(left);
|
||||
const rightValues = stringAndNameProjection(right);
|
||||
const complete =
|
||||
swiftLiteralCoverage(left) === swiftLiteralCoverage(right) &&
|
||||
swiftLiteralCoverage(left) !== "partial";
|
||||
// Facts both dossiers record decide a change before Swift literal coverage can.
|
||||
const leftValues = stringAndNameProjection(left, complete);
|
||||
const rightValues = stringAndNameProjection(right, complete);
|
||||
if (leftValues === null || rightValues === null)
|
||||
return unresolvedDimension({
|
||||
dimension: "strings_names",
|
||||
|
|
@ -151,9 +156,40 @@ const compareStringsAndNames = (
|
|||
"Reference source locations could not be normalized to function offsets.",
|
||||
],
|
||||
});
|
||||
return compareValues("strings_names", leftValues, rightValues, context);
|
||||
const compared = compareValues(
|
||||
"strings_names",
|
||||
leftValues,
|
||||
rightValues,
|
||||
context,
|
||||
);
|
||||
if (complete || compared.status === "changed") return compared;
|
||||
// A retained literal differs only when the opposite dossier establishes its absence.
|
||||
if (swiftLiteralSitesConflict(left, right)) {
|
||||
const leftAll = stringAndNameProjection(left);
|
||||
const rightAll = stringAndNameProjection(right);
|
||||
if (leftAll !== null && rightAll !== null)
|
||||
return compareValues("strings_names", leftAll, rightAll, context);
|
||||
}
|
||||
return unresolvedDimension({
|
||||
dimension: "strings_names",
|
||||
links: context.links,
|
||||
leftCount: null,
|
||||
rightCount: null,
|
||||
limitations: [
|
||||
"Swift String literal coverage differs or is partial; referenced strings and names match, and complete coverage does not establish a differing literal.",
|
||||
],
|
||||
});
|
||||
};
|
||||
|
||||
const swiftLiteralCoverage = (
|
||||
snapshot: FunctionSnapshot,
|
||||
): "absent" | "complete" | "partial" =>
|
||||
(snapshot.dossier.swift_string_literal_omissions ?? []).length > 0
|
||||
? "partial"
|
||||
: snapshot.dossier.swift_string_literals === undefined
|
||||
? "absent"
|
||||
: "complete";
|
||||
|
||||
const compareReferences = (
|
||||
left: FunctionSnapshot,
|
||||
right: FunctionSnapshot,
|
||||
|
|
|
|||
|
|
@ -142,6 +142,7 @@ export const commentProjection = (
|
|||
|
||||
export const stringAndNameProjection = (
|
||||
snapshot: FunctionSnapshot,
|
||||
includeSwiftLiterals = true,
|
||||
): readonly unknown[] | null => {
|
||||
const values = [
|
||||
...snapshot.dossier.referenced_strings.map(
|
||||
|
|
@ -164,12 +165,90 @@ export const stringAndNameProjection = (
|
|||
value,
|
||||
}),
|
||||
),
|
||||
...(includeSwiftLiterals
|
||||
? (snapshot.dossier.swift_string_literals ?? [])
|
||||
: []
|
||||
).map(
|
||||
({
|
||||
source_address: sourceAddress,
|
||||
value,
|
||||
representation,
|
||||
registers,
|
||||
}) => ({
|
||||
kind: "swift_string",
|
||||
source_offset: relativeAddress(
|
||||
sourceAddress,
|
||||
snapshot.dossier.procedure.address,
|
||||
),
|
||||
value,
|
||||
representation,
|
||||
registers,
|
||||
}),
|
||||
),
|
||||
];
|
||||
return values.some(({ source_offset: offset }) => offset === null)
|
||||
? null
|
||||
: sorted(values);
|
||||
};
|
||||
|
||||
/**
|
||||
* A retained literal proves presence, not that every path through its site was examined.
|
||||
* A differing retained fact proves a change only when the opposite dossier is complete
|
||||
* and therefore establishes its absence.
|
||||
*/
|
||||
export const swiftLiteralSitesConflict = (
|
||||
left: FunctionSnapshot,
|
||||
right: FunctionSnapshot,
|
||||
): boolean => {
|
||||
const leftSites = swiftLiteralSites(left);
|
||||
const rightSites = swiftLiteralSites(right);
|
||||
for (const site of new Set([
|
||||
...leftSites.values.keys(),
|
||||
...rightSites.values.keys(),
|
||||
])) {
|
||||
const leftValues =
|
||||
leftSites.values.get(site) ?? (leftSites.complete ? [] : undefined);
|
||||
const rightValues =
|
||||
rightSites.values.get(site) ?? (rightSites.complete ? [] : undefined);
|
||||
if (leftValues === undefined || rightValues === undefined) continue;
|
||||
const leftFacts = new Set(leftValues);
|
||||
const rightFacts = new Set(rightValues);
|
||||
if (
|
||||
(rightSites.complete &&
|
||||
leftValues.some((value) => !rightFacts.has(value))) ||
|
||||
(leftSites.complete && rightValues.some((value) => !leftFacts.has(value)))
|
||||
)
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
const swiftLiteralSites = (snapshot: FunctionSnapshot) => {
|
||||
const base = snapshot.dossier.procedure.address;
|
||||
const omissions = snapshot.dossier.swift_string_literal_omissions ?? [];
|
||||
const site = (sourceAddress: string, registers: readonly string[]) => {
|
||||
const offset = relativeAddress(sourceAddress, base);
|
||||
return offset === null ? null : canonicalJson([offset, registers]);
|
||||
};
|
||||
// Keep the same literal facts the full strings_names projection compares.
|
||||
const values = new Map<string, string[]>();
|
||||
for (const literal of snapshot.dossier.swift_string_literals ?? []) {
|
||||
const key = site(literal.source_address, literal.registers);
|
||||
if (key === null) continue;
|
||||
values.set(
|
||||
key,
|
||||
[
|
||||
...(values.get(key) ?? []),
|
||||
canonicalJson([literal.value, literal.representation]),
|
||||
].sort(compareUnicodeCodePoints),
|
||||
);
|
||||
}
|
||||
const complete =
|
||||
snapshot.dossier.swift_string_literals !== undefined &&
|
||||
omissions.length === 0;
|
||||
return { values, complete };
|
||||
};
|
||||
|
||||
export const sorted = (values: readonly unknown[]): readonly unknown[] =>
|
||||
values
|
||||
.map((value) => ({
|
||||
|
|
|
|||
|
|
@ -290,6 +290,70 @@ export const unresolvedCallSchema = z.strictObject({
|
|||
reason: z.string(),
|
||||
});
|
||||
|
||||
const swiftStringLiteralFields = {
|
||||
source_address: z
|
||||
.string()
|
||||
.describe(
|
||||
"Instruction that consumes both words: a call, tail call, return, or paired store.",
|
||||
),
|
||||
value: z
|
||||
.string()
|
||||
.describe("Decoded UTF-8 text, including the canonical empty string."),
|
||||
registers: z.tuple([z.string(), z.string()]),
|
||||
words: z
|
||||
.tuple([z.string(), z.string()])
|
||||
.describe("Observed constant register values, in register order."),
|
||||
};
|
||||
|
||||
/** A Swift String literal decoded from the two words of its ABI value, not a data reference. */
|
||||
export const swiftStringLiteralSchema = z.discriminatedUnion("representation", [
|
||||
z.strictObject({
|
||||
...swiftStringLiteralFields,
|
||||
representation: z
|
||||
.literal("small")
|
||||
.describe("The UTF-8 bytes are inside the two words."),
|
||||
address: z.null().describe("A small literal has no storage of its own."),
|
||||
}),
|
||||
z.strictObject({
|
||||
...swiftStringLiteralFields,
|
||||
representation: z
|
||||
.literal("large")
|
||||
.describe(
|
||||
"An immortal literal whose words hold its count and a pointer to its bytes.",
|
||||
),
|
||||
address: z.string().describe("Address of a large literal's bytes."),
|
||||
}),
|
||||
]);
|
||||
|
||||
const swiftStringLiteralOmissionFields = {
|
||||
source_address: swiftStringLiteralFields.source_address,
|
||||
byte_length: z.number().int().min(0).max(Number.MAX_SAFE_INTEGER),
|
||||
reason: z
|
||||
.literal("capacity")
|
||||
.describe(
|
||||
"The record did not fit the provider's memory budget for this dossier; decoding stopped here, so later use sites were not examined.",
|
||||
),
|
||||
registers: swiftStringLiteralFields.registers,
|
||||
words: swiftStringLiteralFields.words,
|
||||
};
|
||||
|
||||
/** A Swift literal whose layout matched but whose record did not fit; decoding stopped there. */
|
||||
export const swiftStringLiteralOmissionSchema = z.discriminatedUnion(
|
||||
"representation",
|
||||
[
|
||||
z.strictObject({
|
||||
...swiftStringLiteralOmissionFields,
|
||||
representation: z.literal("small"),
|
||||
address: z.null(),
|
||||
}),
|
||||
z.strictObject({
|
||||
...swiftStringLiteralOmissionFields,
|
||||
representation: z.literal("large"),
|
||||
address: z.string().describe("Address of the literal's bytes."),
|
||||
}),
|
||||
],
|
||||
);
|
||||
|
||||
/** Observed endpoints with complete or partial reference classification evidence. */
|
||||
export const referenceEdgeSchema = z
|
||||
.object({
|
||||
|
|
@ -341,6 +405,18 @@ export const functionDossierSchema = z
|
|||
})
|
||||
.strict(),
|
||||
),
|
||||
swift_string_literals: z
|
||||
.array(swiftStringLiteralSchema)
|
||||
.exactOptional()
|
||||
.describe(
|
||||
"Swift String literals decoded from constant register values. Absent when the provider does not decode them for this target.",
|
||||
),
|
||||
swift_string_literal_omissions: z
|
||||
.array(swiftStringLiteralOmissionSchema)
|
||||
.exactOptional()
|
||||
.describe(
|
||||
"The literal at which decoding stopped for capacity. Present exactly when swift_string_literals is present; empty when decoding examined every use site.",
|
||||
),
|
||||
basic_blocks: z.array(
|
||||
z
|
||||
.object({
|
||||
|
|
|
|||
|
|
@ -195,6 +195,51 @@ describe("Ghidra function-analysis result values", () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe("Ghidra Swift String literals", () => {
|
||||
it("keeps a small Swift literal without storage apart from a large literal's bytes", () => {
|
||||
const dossier = ghidraFunctionDossier();
|
||||
if (
|
||||
typeof dossier !== "object" ||
|
||||
dossier === null ||
|
||||
Array.isArray(dossier)
|
||||
)
|
||||
throw new TypeError("Ghidra dossier fixture is invalid");
|
||||
const literals = [
|
||||
{
|
||||
source_address: "0x401004",
|
||||
value: "The user name ",
|
||||
representation: "small",
|
||||
address: null,
|
||||
registers: ["x0", "x1"],
|
||||
words: ["0x7265737520656854", "0xee0020656d616e20"],
|
||||
},
|
||||
{
|
||||
source_address: "0x401006",
|
||||
value: "",
|
||||
representation: "small",
|
||||
address: null,
|
||||
registers: ["x0", "x1"],
|
||||
words: ["0x0", "0xe000000000000000"],
|
||||
},
|
||||
{
|
||||
source_address: "0x401008",
|
||||
value: " cannot be written into a sudoers rule safely.",
|
||||
representation: "large",
|
||||
address: "0x402270",
|
||||
registers: ["x0", "x1"],
|
||||
words: ["0xd00000000000002e", "0x8000000000402250"],
|
||||
},
|
||||
];
|
||||
expect(
|
||||
parseFunctionValueFixture("analyze_function", {
|
||||
...dossier,
|
||||
swift_string_literals: literals,
|
||||
swift_string_literal_omissions: [],
|
||||
}),
|
||||
).toMatchObject({ ok: true, value: { swift_string_literals: literals } });
|
||||
});
|
||||
});
|
||||
|
||||
describe("Ghidra compact reference provenance", () => {
|
||||
it("distinguishes incoming references, examined emptiness and unavailable coverage at data addresses", () => {
|
||||
const instruction = {
|
||||
|
|
@ -423,6 +468,34 @@ describe("Ghidra function-analysis malformed results", () => {
|
|||
);
|
||||
});
|
||||
|
||||
const malformedSwiftStringLiterals = (
|
||||
dossier: Record<string, JsonValue>,
|
||||
): Array<[string, GhidraFunctionOperation, JsonValue]> =>
|
||||
(
|
||||
[
|
||||
["non-canonical Swift literal address", "large", "402240"],
|
||||
["small Swift literal with storage", "small", "0x402270"],
|
||||
["large Swift literal without storage", "large", null],
|
||||
] as const
|
||||
).map(([name, representation, address]) => [
|
||||
name,
|
||||
"analyze_function",
|
||||
{
|
||||
...dossier,
|
||||
swift_string_literal_omissions: [],
|
||||
swift_string_literals: [
|
||||
{
|
||||
source_address: "0x401004",
|
||||
value: "The power setting could not be changed.",
|
||||
representation,
|
||||
address,
|
||||
registers: ["x0", "x1"],
|
||||
words: ["0xd000000000000027", "0x8000000000402220"],
|
||||
},
|
||||
],
|
||||
},
|
||||
]);
|
||||
|
||||
const malformedOutputs = (): Array<
|
||||
[string, GhidraFunctionOperation, JsonValue]
|
||||
> => {
|
||||
|
|
@ -437,6 +510,11 @@ const malformedOutputs = (): Array<
|
|||
throw new TypeError("Ghidra native API fixture is invalid");
|
||||
return [
|
||||
["non-canonical address", "procedure_callers", ["401000"]],
|
||||
[
|
||||
"Swift literals without their omission record",
|
||||
"analyze_function",
|
||||
{ ...dossier, swift_string_literals: [] },
|
||||
],
|
||||
[
|
||||
"missing classification",
|
||||
"procedure_info",
|
||||
|
|
@ -467,6 +545,26 @@ const malformedOutputs = (): Array<
|
|||
Object.entries(dossier).filter(([key]) => key !== "native_api"),
|
||||
),
|
||||
],
|
||||
...malformedSwiftStringLiterals(dossier),
|
||||
[
|
||||
"non-canonical Swift literal omission address",
|
||||
"analyze_function",
|
||||
{
|
||||
...dossier,
|
||||
swift_string_literals: [],
|
||||
swift_string_literal_omissions: [
|
||||
{
|
||||
source_address: "0x401004",
|
||||
representation: "large",
|
||||
address: "402240",
|
||||
byte_length: 39,
|
||||
reason: "capacity",
|
||||
registers: ["x0", "x1"],
|
||||
words: ["0xd000000000000027", "0x8000000000402220"],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
[
|
||||
"non-canonical native API target",
|
||||
"analyze_function",
|
||||
|
|
|
|||
|
|
@ -293,6 +293,15 @@ const ghidraFunctionDossier = functionDossierSchema
|
|||
})
|
||||
.strict()
|
||||
.superRefine((value, context) => {
|
||||
if (
|
||||
(value.swift_string_literals === undefined) !==
|
||||
(value.swift_string_literal_omissions === undefined)
|
||||
)
|
||||
context.addIssue({
|
||||
code: "custom",
|
||||
message:
|
||||
"Ghidra dossier must record Swift literals and their omissions together",
|
||||
});
|
||||
const addresses = [
|
||||
value.procedure.address,
|
||||
...value.comments.map(({ address }) => address),
|
||||
|
|
@ -308,6 +317,14 @@ const ghidraFunctionDossier = functionDossierSchema
|
|||
address,
|
||||
source_address,
|
||||
]),
|
||||
...(value.swift_string_literals ?? []).flatMap(
|
||||
({ address, source_address }) =>
|
||||
address === null ? [source_address] : [address, source_address],
|
||||
),
|
||||
...(value.swift_string_literal_omissions ?? []).flatMap(
|
||||
({ address, source_address }) =>
|
||||
address === null ? [source_address] : [address, source_address],
|
||||
),
|
||||
...value.basic_blocks.flatMap(({ start, end, successors }) => [
|
||||
start,
|
||||
end,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,23 @@
|
|||
@inline(never)
|
||||
func consume(_ value: String) { print(value) }
|
||||
@inline(never)
|
||||
func smallLiteral() { consume("SleepDisabled") }
|
||||
@inline(never)
|
||||
func largeLiteral() { consume("The power setting could not be changed.") }
|
||||
@inline(never)
|
||||
func unicodeLiterals() {
|
||||
consume("café")
|
||||
consume("Résumé of the naïve décor, written twice")
|
||||
}
|
||||
@inline(never)
|
||||
func emptyAndControlLiterals() {
|
||||
consume("")
|
||||
consume("tab\there")
|
||||
}
|
||||
@inline(never)
|
||||
func returnedLiteral() -> String { "Holding awake" }
|
||||
smallLiteral()
|
||||
largeLiteral()
|
||||
unicodeLiterals()
|
||||
emptyAndControlLiterals()
|
||||
print(returnedLiteral(), CommandLine.arguments.count)
|
||||
Loading…
Reference in New Issue