test(ghidra): align alias restoration regressions with upstream
Reuse the equivalent shared verifier restoration merged independently in main. Retain distinct uncertain-write, rejected-restoration, incorrect-readback and combined-failure regressions without duplicating the producing implementation. Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
This commit is contained in:
commit
a4d293b61b
|
|
@ -2,7 +2,7 @@
|
|||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "41"
|
||||
version: "42"
|
||||
---
|
||||
|
||||
# REA
|
||||
|
|
@ -104,6 +104,10 @@ explicit path or endpoint and do not need it.
|
|||
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
|
||||
optional metadata recovery and its supported host/layout boundary.
|
||||
- User-owned browser page already open: `list_browser_targets`.
|
||||
- Browser action the agent must perform (click, type, navigate):
|
||||
`capture_browser_scenario` with declared actions. See
|
||||
[runtime observation](references/runtime-observation.md) for connect/launch
|
||||
and the focused follow-up browser tools.
|
||||
- Fiddler Everywhere traffic or a native `.saz` capture: read the
|
||||
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
|
||||
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
|
||||
|
|
@ -323,7 +327,8 @@ as complete while required questions remain open.
|
|||
[references/javascript-applications.md](references/javascript-applications.md)
|
||||
- Android APK declarations, classes, methods, and static references:
|
||||
[references/android-applications.md](references/android-applications.md)
|
||||
- Passive browser/Electron observation and static/runtime reconciliation:
|
||||
- Browser/Electron observation (passive and scenario) and static/runtime
|
||||
reconciliation:
|
||||
[references/runtime-observation.md](references/runtime-observation.md)
|
||||
- Evidence paging, comparisons, residual unknowns, and verification:
|
||||
[references/evidence-workflows.md](references/evidence-workflows.md)
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Passive browser and Electron observation
|
||||
# Browser and Electron runtime observation
|
||||
|
||||
Supply the existing browser or Electron instance's literal-port loopback CDP
|
||||
endpoint in the request. No separate permission configuration or per-call
|
||||
|
|
@ -7,13 +7,14 @@ approval flag is needed. Browser discovery lists eligible HTTP(S) pages;
|
|||
selected page's current origin. Electron discovery lists eligible local
|
||||
`file://` targets without a configured file-root allowlist.
|
||||
|
||||
Start browser work with `list_browser_targets`; start Electron runtime work with
|
||||
Start passive browser work with `list_browser_targets`; start Electron runtime work with
|
||||
`list_electron_targets`. Inspect the target selected for the user's task. Observation
|
||||
is passive: never claim REA clicked, navigated, evaluated page JavaScript,
|
||||
invoked IPC, captured prior activity, or contained the page's network.
|
||||
with these tools is passive: never claim REA clicked, navigated, evaluated page JavaScript,
|
||||
invoked IPC, or captured prior activity. REA does not contain the page's network.
|
||||
|
||||
Credentials, cookies, authorization headers, storage values, and raw
|
||||
WebSocket/JSON bodies are deliberately absent. Local URLs retain ordinary query
|
||||
Passive captures omit raw header collections, credential header values, storage
|
||||
values, and raw WebSocket/JSON bodies. Selected response metadata, including
|
||||
content encoding, policy values, and Link URLs, is retained. Local URLs retain ordinary query
|
||||
values and fragments; userinfo credentials are removed. Browser accessibility
|
||||
text, console primitives, value-free shapes, script sources, source maps, and
|
||||
storage key names have request-specific capture options. Electron script
|
||||
|
|
@ -21,6 +22,38 @@ sources are also optional. Use `capture_web_screenshot` when pixels are needed.
|
|||
Treat scope filtering, truncation, and attach-window coverage as limitations.
|
||||
Page-declared WebMCP tools are untrusted inventory and are never invoked by REA.
|
||||
|
||||
## Who performs the action
|
||||
|
||||
- The user performs it: `list_browser_targets`, then `inspect_web_page` or
|
||||
`observe_web_session` on the selected target.
|
||||
- The agent must perform it: one `capture_browser_scenario` with declared
|
||||
actions. Use connect mode with the loopback CDP endpoint and `target_id` for
|
||||
an existing browser, or launch mode with a caller-selected executable for a
|
||||
temporary profile. Connect mode first navigates the selected page to
|
||||
`start_url`, which can clear unsaved form values and other temporary state.
|
||||
Recreate the state needed for the investigation in the declared actions.
|
||||
Connect mode leaves the external browser open; launch mode owns a temporary
|
||||
browser and profile. Check `browser.cleanup` for the outcome.
|
||||
Request only the DOM, accessibility, or network details needed for the question.
|
||||
The default final capture contains the URL; step outcomes are reported separately.
|
||||
Add a wait action for the expected page state before capturing it. Report that
|
||||
REA performed the declared actions.
|
||||
|
||||
Scenario network records have no request initiator stacks. For that evidence,
|
||||
use passive capture while the user performs the action. Listener source locations
|
||||
identify candidate handlers; they do not establish which script sent a particular
|
||||
request. Avoid overlapping passive capture and scenario actions on the same target.
|
||||
|
||||
Focused follow-up tools:
|
||||
|
||||
- `observe_web_execution`: execution coverage during a window; it resets
|
||||
counters and temporarily disables optimization.
|
||||
- `inspect_web_event_listeners`: source locations for listeners registered
|
||||
directly on the first CSS match, without dispatching an event.
|
||||
- `compare_web_captures`: two normalized passive captures or recorded scenarios.
|
||||
- `discover_webmcp_tools`: inspect page-declared WebMCP inventory; never invoke
|
||||
the discovered tools.
|
||||
|
||||
Use `reconcile_javascript_runtime` only after static application Evidence and
|
||||
passive browser/Electron Evidence exist. Prefer captured-source digest identity;
|
||||
report path/digest disagreement as a mismatch and preserve ambiguity. Explicit
|
||||
|
|
|
|||
|
|
@ -634,6 +634,7 @@ jobs:
|
|||
tests/boundary/cli/nativeSignatureCli.test.ts
|
||||
tests/boundary/mcp/nativeSignatureTargetBindingMcp.test.ts
|
||||
src/process/capture/FilesystemSnapshot.test.ts
|
||||
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
|
||||
tests/boundary/process/darwinProcessRunTokenReader.test.ts
|
||||
tests/boundary/process/directAnalysisSignals.test.ts
|
||||
tests/boundary/process/processCaptureLifecycle.test.ts
|
||||
|
|
@ -714,6 +715,8 @@ jobs:
|
|||
- name: Run curated Windows capability tests
|
||||
run: >-
|
||||
npx vitest run --coverage.enabled=false
|
||||
src/process/capture/FilesystemSnapshot.test.ts
|
||||
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
|
||||
tests/boundary/filesystem/binaryTarget.test.ts
|
||||
tests/boundary/filesystem/safeOutputTree.test.ts
|
||||
tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts
|
||||
|
|
|
|||
|
|
@ -5,9 +5,11 @@ import java.io.BufferedReader;
|
|||
import java.io.BufferedWriter;
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.io.IOException;
|
||||
import java.io.InputStream;
|
||||
import java.math.BigInteger;
|
||||
import java.net.InetAddress;
|
||||
import java.net.InetSocketAddress;
|
||||
import java.net.SocketTimeoutException;
|
||||
import java.net.StandardProtocolFamily;
|
||||
import java.net.UnixDomainSocketAddress;
|
||||
import java.nio.channels.Channels;
|
||||
|
|
@ -298,7 +300,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
Files.move(pendingPath, endpointPath);
|
||||
}
|
||||
}
|
||||
acceptClient(server, descriptor);
|
||||
acceptAuthenticatedLoopbackClient(server, descriptor);
|
||||
}
|
||||
finally {
|
||||
Files.deleteIfExists(pendingPath);
|
||||
|
|
@ -310,6 +312,50 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
return System.getProperty("os.name", "").startsWith("Windows");
|
||||
}
|
||||
|
||||
private static String readLoopbackHandshake(
|
||||
SocketChannel client, int maxBytes, long timeoutMillis) throws IOException {
|
||||
long deadline = System.nanoTime() + timeoutMillis * 1_000_000L;
|
||||
InputStream input = client.socket().getInputStream();
|
||||
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
|
||||
while (true) {
|
||||
long remaining = deadline - System.nanoTime();
|
||||
if (remaining <= 0) throw new SocketTimeoutException("Bridge authentication timed out");
|
||||
client.socket().setSoTimeout((int) Math.max(1,
|
||||
Math.min(Integer.MAX_VALUE, (remaining + 999_999L) / 1_000_000L)));
|
||||
int value = input.read();
|
||||
if (value == -1) return null;
|
||||
if (value == '\n') return bytes.toString(StandardCharsets.UTF_8);
|
||||
if (bytes.size() == maxBytes) throw new IOException("Bridge authentication frame is too large");
|
||||
bytes.write(value);
|
||||
}
|
||||
}
|
||||
|
||||
private void acceptAuthenticatedLoopbackClient(
|
||||
ServerSocketChannel server, SessionDescriptor descriptor) throws Exception {
|
||||
while (true) {
|
||||
try (SocketChannel client = server.accept()) {
|
||||
Request first;
|
||||
try {
|
||||
String line = readLoopbackHandshake(client, 64 * 1024, 1_000);
|
||||
if (line == null) continue;
|
||||
first = parseRequest(line, descriptor.token);
|
||||
}
|
||||
catch (IOException | RuntimeException rejected) {
|
||||
// A rejected unauthenticated peer cannot consume the bridge session.
|
||||
continue;
|
||||
}
|
||||
client.socket().setSoTimeout(0);
|
||||
try (BufferedReader reader = new BufferedReader(
|
||||
Channels.newReader(client, StandardCharsets.UTF_8));
|
||||
BufferedWriter writer = new BufferedWriter(
|
||||
Channels.newWriter(client, StandardCharsets.UTF_8))) {
|
||||
serveClient(descriptor, reader, writer, first);
|
||||
}
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void acceptClient(
|
||||
ServerSocketChannel server,
|
||||
SessionDescriptor descriptor) throws Exception {
|
||||
|
|
@ -320,26 +366,28 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
BufferedWriter writer = new BufferedWriter(
|
||||
Channels.newWriter(client, StandardCharsets.UTF_8)
|
||||
)) {
|
||||
serveClient(descriptor, reader, writer);
|
||||
serveClient(descriptor, reader, writer, null);
|
||||
}
|
||||
}
|
||||
|
||||
private void serveClient(
|
||||
SessionDescriptor descriptor,
|
||||
BufferedReader reader,
|
||||
BufferedWriter writer) throws Exception {
|
||||
BufferedWriter writer,
|
||||
Request first) throws Exception {
|
||||
while (true) {
|
||||
String line = reader.readLine();
|
||||
if (line == null) {
|
||||
return;
|
||||
}
|
||||
Request request;
|
||||
try {
|
||||
request = parseRequest(line, descriptor.token);
|
||||
}
|
||||
catch (RuntimeException exception) {
|
||||
writeFailure(writer, 1, "invalid_request", "Bridge request is invalid");
|
||||
return;
|
||||
Request request = first;
|
||||
first = null;
|
||||
if (request == null) {
|
||||
String line = reader.readLine();
|
||||
if (line == null) return;
|
||||
try {
|
||||
request = parseRequest(line, descriptor.token);
|
||||
}
|
||||
catch (RuntimeException exception) {
|
||||
writeFailure(writer, 1, "invalid_request", "Bridge request is invalid");
|
||||
return;
|
||||
}
|
||||
}
|
||||
try {
|
||||
if (request.method.equals("shutdown")) {
|
||||
|
|
@ -887,6 +935,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
JsonObject result = functionObservation(
|
||||
value == null ? JsonNull.INSTANCE : GSON.toJsonTree(value),
|
||||
noReturnCallLimitations(
|
||||
function,
|
||||
currentProgram.getListing().getInstructions(function.getBody(), true)
|
||||
)
|
||||
);
|
||||
|
|
@ -978,7 +1027,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
|
||||
return functionObservation(
|
||||
GSON.toJsonTree(renderAssembly(scan.instructions)),
|
||||
noReturnCallLimitations(scan.instructions)
|
||||
noReturnCallLimitations(function, scan.instructions)
|
||||
);
|
||||
}
|
||||
|
||||
|
|
@ -997,7 +1046,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
limitations.add(
|
||||
"The fast path does not invoke the decompiler or scan whole-program names and strings."
|
||||
);
|
||||
List<String> boundaryLimitations = noReturnCallLimitations(scan.instructions);
|
||||
List<String> boundaryLimitations = noReturnCallLimitations(function, scan.instructions);
|
||||
for (String limitation : boundaryLimitations) limitations.add(limitation);
|
||||
JsonObject result = new JsonObject();
|
||||
result.add("procedure", procedureIdentity(function));
|
||||
|
|
@ -1020,7 +1069,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
result.add(canonicalAddress(item.getEntryPoint()));
|
||||
}
|
||||
if (callers) {
|
||||
JsonObject observation = functionObservation(result, noReturnCallLimitations(
|
||||
JsonObject observation = functionObservation(result, noReturnCallLimitations(function,
|
||||
currentProgram.getListing().getInstructions(function.getBody(), true)
|
||||
));
|
||||
observation.add("relationship_coverage", callRelationshipCoverage(null,
|
||||
|
|
@ -1051,6 +1100,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
return functionObservation(
|
||||
result,
|
||||
noReturnCallLimitations(
|
||||
function,
|
||||
currentProgram.getListing().getInstructions(function.getBody(), true)
|
||||
)
|
||||
);
|
||||
|
|
@ -1571,7 +1621,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
);
|
||||
if (swift && aarch64) {
|
||||
limitations.add(
|
||||
"Swift String literals are decoded from constant register pairs at calls, returns and paired stores; literals stored one word at a time, chosen by conditional selects, passed through unresolved jumps, or built at runtime are not reported."
|
||||
"Swift String literals are decoded from constant register pairs at calls, returns and paired stores. Literals stored one word at a time, chosen by conditional selects, or built at runtime are not reported; candidate pairs at unresolved jumps are retained as unknown omissions."
|
||||
);
|
||||
if (swiftLiterals.omissions().size() > 0) {
|
||||
limitations.add(
|
||||
|
|
@ -3380,13 +3430,23 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
return omitted[0];
|
||||
}
|
||||
|
||||
private List<String> noReturnCallLimitations(Iterable<Instruction> instructions)
|
||||
private List<String> noReturnCallLimitations(
|
||||
Function function, Iterable<Instruction> instructions)
|
||||
throws Exception {
|
||||
List<String> limitations = new ArrayList<>();
|
||||
Instruction beforePrevious = null;
|
||||
Instruction previous = null;
|
||||
for (Instruction instruction : instructions) {
|
||||
monitor.checkCancelled();
|
||||
String limitation = terminalCallLimitation(instruction);
|
||||
if (limitation != null) limitations.add(limitation);
|
||||
if (beforePrevious != null && previous != null) {
|
||||
String seh = borlandSehHandlerLimitation(
|
||||
function, beforePrevious, previous, instruction);
|
||||
if (seh != null) limitations.add(seh);
|
||||
}
|
||||
beforePrevious = previous;
|
||||
previous = instruction;
|
||||
}
|
||||
return limitations;
|
||||
}
|
||||
|
|
@ -3397,6 +3457,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
throws Exception {
|
||||
List<String> limitations = new ArrayList<>();
|
||||
CallBoundaryReferences calls = new CallBoundaryReferences(returnedCallees, includeUnresolvedSites);
|
||||
Instruction beforePrevious = null;
|
||||
Instruction previous = null;
|
||||
if (observedReferences == null) visitReferences(function, "outgoing", calls::observe);
|
||||
else for (Reference reference : observedReferences) {
|
||||
monitor.checkCancelled();
|
||||
|
|
@ -3406,6 +3468,13 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
monitor.checkCancelled();
|
||||
String limitation = terminalCallLimitation(instruction);
|
||||
if (limitation != null) limitations.add(limitation);
|
||||
if (beforePrevious != null && previous != null) {
|
||||
String seh = borlandSehHandlerLimitation(
|
||||
function, beforePrevious, previous, instruction);
|
||||
if (seh != null) limitations.add(seh);
|
||||
}
|
||||
beforePrevious = previous;
|
||||
previous = instruction;
|
||||
if (!instruction.getFlowType().isCall()) continue;
|
||||
boolean destination = false;
|
||||
for (Reference reference : instruction.getReferencesFrom()) {
|
||||
|
|
@ -3423,6 +3492,61 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls());
|
||||
}
|
||||
|
||||
private String borlandSehHandlerLimitation(
|
||||
Function function, Instruction handlerPush, Instruction framePush,
|
||||
Instruction frameLink) {
|
||||
if (!immediatelyPrecedes(handlerPush, framePush) ||
|
||||
!immediatelyPrecedes(framePush, frameLink) ||
|
||||
!handlerPush.getMnemonicString().equalsIgnoreCase("PUSH") ||
|
||||
handlerPush.getNumOperands() != 1 ||
|
||||
!framePush.getMnemonicString().equalsIgnoreCase("PUSH") ||
|
||||
framePush.getNumOperands() != 1 ||
|
||||
!frameLink.getMnemonicString().equalsIgnoreCase("MOV") ||
|
||||
frameLink.getNumOperands() != 2 ||
|
||||
!sameFsMemoryOperand(framePush, frameLink) ||
|
||||
!hasRegisterOperand(frameLink, 1, "ESP"))
|
||||
return null;
|
||||
Object[] handlerOperand = handlerPush.getOpObjects(0);
|
||||
if (handlerOperand.length != 1 || !(handlerOperand[0] instanceof Scalar scalar) ||
|
||||
scalar.bitLength() != 32)
|
||||
return null;
|
||||
Address handler = currentProgram.getAddressFactory().getDefaultAddressSpace()
|
||||
.getAddress(scalar.getUnsignedValue());
|
||||
if (function.getBody().contains(handler)) return null;
|
||||
return "Ghidra Listing shows a Borland/Delphi-style SEH frame registration at " +
|
||||
canonicalAddress(frameLink.getAddress()) + " (push " + canonicalAddress(handler) +
|
||||
" followed by fs-segment stack chaining); handler " + canonicalAddress(handler) +
|
||||
" lies outside this procedure's body, so callees, strings, and cross-references may omit the handler region.";
|
||||
}
|
||||
|
||||
private static boolean immediatelyPrecedes(Instruction first, Instruction second) {
|
||||
Address fallThrough = first.getFallThrough();
|
||||
return fallThrough != null && fallThrough.equals(second.getAddress());
|
||||
}
|
||||
|
||||
private static boolean sameFsMemoryOperand(Instruction first, Instruction second) {
|
||||
String firstOperand = normalizedFsMemoryOperand(first);
|
||||
String secondOperand = normalizedFsMemoryOperand(second);
|
||||
return firstOperand != null && firstOperand.equalsIgnoreCase(secondOperand);
|
||||
}
|
||||
|
||||
private static String normalizedFsMemoryOperand(Instruction instruction) {
|
||||
String representation = instruction.getDefaultOperandRepresentation(0);
|
||||
representation = representation.replaceAll("(?i)\\b(?:byte|word|dword|qword)\\s+ptr\\s+", "")
|
||||
.replaceAll("\\s+", "");
|
||||
return representation.regionMatches(true, 0, "fs:", 0, 3) &&
|
||||
representation.indexOf('[') >= 0 ? representation : null;
|
||||
}
|
||||
|
||||
private static boolean hasRegisterOperand(
|
||||
Instruction instruction, int operand, String name) {
|
||||
for (Object object : instruction.getOpObjects(operand)) {
|
||||
if (object instanceof Register register && register.getName().equalsIgnoreCase(name))
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private final class CallBoundaryReferences {
|
||||
private final Set<Function> returnedCallees;
|
||||
private final TreeMap<Address, String> unresolved;
|
||||
|
|
@ -3827,13 +3951,20 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
* The canonical empty string is a small literal with a zero count.
|
||||
*/
|
||||
private SwiftStringLiteralDecode swiftStringLiterals(Function function) throws Exception {
|
||||
return swiftStringLiterals(function, new SwiftLiteralBudget());
|
||||
}
|
||||
|
||||
// An explicit budget makes capacity omissions deterministic in producer tests.
|
||||
private SwiftStringLiteralDecode swiftStringLiterals(
|
||||
Function function,
|
||||
SwiftLiteralBudget budget
|
||||
) throws Exception {
|
||||
Register[] registers = new Register[31];
|
||||
for (int index = 0; index < registers.length; index++) {
|
||||
registers[index] = currentProgram.getRegister("x" + index);
|
||||
}
|
||||
TreeMap<String, JsonObject> observed = new TreeMap<>();
|
||||
TreeMap<String, JsonObject> omitted = new TreeMap<>();
|
||||
SwiftLiteralBudget budget = new SwiftLiteralBudget();
|
||||
Map<String, Boolean> selectedValues = new HashMap<>();
|
||||
SymbolicPropogator propagator = new SymbolicPropogator(currentProgram, false);
|
||||
propagator.flowConstants(function.getEntryPoint(), function.getBody(), new ContextEvaluatorAdapter() {
|
||||
|
|
@ -3841,6 +3972,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
public boolean evaluateContextBefore(VarnodeContext context, Instruction instruction) {
|
||||
// Returning true ends only the current flow, so every queued flow checks the shared stop.
|
||||
if (budget.stopped()) return true;
|
||||
FlowType flow = instruction.getFlowType();
|
||||
Address[] flowTargets = instruction.getFlows();
|
||||
boolean unresolvedJump = flow.isJump() && !flow.isConditional() && flowTargets.length == 0;
|
||||
for (Register[] pair : swiftStringPairs(function, instruction, registers)) {
|
||||
Long word = registerConstant(context, pair[0]);
|
||||
Long object = registerConstant(context, pair[1]);
|
||||
|
|
@ -3890,12 +4024,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
JsonObject item = overCapacity
|
||||
? swiftStringOmission(layout, word, object)
|
||||
: swiftStringItem(value, layout, word, object);
|
||||
if (unresolvedJump && !overCapacity)
|
||||
item = swiftStringOmission(layout, word, object, "unresolved_jump");
|
||||
item.addProperty("source_address", canonicalAddress(instruction.getAddress()));
|
||||
JsonArray pairRegisters = new JsonArray();
|
||||
pairRegisters.add(pair[0].getName());
|
||||
pairRegisters.add(pair[1].getName());
|
||||
item.add("registers", pairRegisters);
|
||||
(overCapacity ? omitted : observed).put(key, item);
|
||||
(overCapacity || unresolvedJump ? omitted : observed).put(key, item);
|
||||
if (overCapacity) {
|
||||
budget.stop();
|
||||
return true;
|
||||
|
|
@ -3931,11 +4067,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
if (operation.getOpcode() == PcodeOp.RETURN) returns = true;
|
||||
}
|
||||
// A jump is a proven tail call only when it has resolved targets and all leave the
|
||||
// body. A targetless computed jump can still dispatch inside the function.
|
||||
// body. For unresolved targetless jumps, candidate literals remain unknown omissions.
|
||||
Address[] targets = instruction.getFlows();
|
||||
boolean tailCall = flow.isJump() && !flow.isConditional() && targets.length > 0 &&
|
||||
Arrays.stream(targets).allMatch(target -> !function.getBody().contains(target));
|
||||
if (!flow.isCall() && !returns && !tailCall) return pairs;
|
||||
boolean unresolvedJump = flow.isJump() && !flow.isConditional() && targets.length == 0;
|
||||
if (!flow.isCall() && !returns && !tailCall && !unresolvedJump) return pairs;
|
||||
// A String occupies two consecutive registers among x0-x7, or x0 and x1 when returned.
|
||||
int lastFirstRegister = returns ? 0 : 6;
|
||||
for (int index = 0; index <= lastFirstRegister; index++) {
|
||||
|
|
@ -4078,6 +4215,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
remaining = Math.max(0, (runtime.maxMemory() - runtime.totalMemory() + runtime.freeMemory()) / 2);
|
||||
}
|
||||
|
||||
SwiftLiteralBudget(long remaining) {
|
||||
if (remaining < 0) throw new IllegalArgumentException("Swift literal budget must be nonnegative");
|
||||
this.remaining = remaining;
|
||||
}
|
||||
|
||||
boolean takeState() {
|
||||
if (remaining < BYTES_PER_RECORD) return false;
|
||||
remaining -= BYTES_PER_RECORD;
|
||||
|
|
@ -4116,6 +4258,15 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
}
|
||||
|
||||
private static JsonObject swiftStringOmission(SwiftLiteralLayout layout, long word, long object) {
|
||||
return swiftStringOmission(layout, word, object, "capacity");
|
||||
}
|
||||
|
||||
private static JsonObject swiftStringOmission(
|
||||
SwiftLiteralLayout layout,
|
||||
long word,
|
||||
long object,
|
||||
String reason
|
||||
) {
|
||||
JsonObject item = new JsonObject();
|
||||
item.addProperty("representation", layout.representation());
|
||||
if (layout.address() == null) {
|
||||
|
|
@ -4124,7 +4275,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
|||
item.addProperty("address", canonicalAddress(layout.address()));
|
||||
}
|
||||
item.addProperty("byte_length", layout.byteLength());
|
||||
item.addProperty("reason", "capacity");
|
||||
item.addProperty("reason", reason);
|
||||
item.add("words", swiftStringWords(word, object));
|
||||
return item;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -736,7 +736,12 @@ def _analyze_function(document, params):
|
|||
def _dispatch(method, params):
|
||||
"""Dispatch only the closed operation set implemented by REA's public tools."""
|
||||
if method == "health":
|
||||
return {"name": "REA Hopper bridge", "version": "1.0.0", "run_id": REA_RUN_ID}
|
||||
return {
|
||||
"name": "REA Hopper bridge",
|
||||
"version": "1.0.0",
|
||||
"run_id": REA_RUN_ID,
|
||||
"process_id": os.getpid(),
|
||||
}
|
||||
if method in ("shutdown", "shutdown_document"):
|
||||
document = _session_document()
|
||||
if document is None:
|
||||
|
|
|
|||
|
|
@ -41,6 +41,87 @@ class BoundedReader:
|
|||
return self.handle.read(count)
|
||||
|
||||
|
||||
SPECIAL_SCHEMES = {"ftp", "file", "http", "https", "ws", "wss"}
|
||||
SCHEME_PREFIX = r"^[\x00-\x20]*([a-z][a-z0-9+.-]*):"
|
||||
NETWORK_PREFIX = r"^[\x00-\x20]*[/\\]{2}"
|
||||
URL_IGNORABLE_CONTROLS = "\t\n\r"
|
||||
URL_CONTROL_TRANSLATION = {ord(char): None for char in URL_IGNORABLE_CONTROLS}
|
||||
|
||||
|
||||
def authority_start(text, base_scheme=None):
|
||||
"""Return where an authority begins and whether it is special, or None.
|
||||
|
||||
Follows WHATWG URL parsing as src/domain/recordedTransportUrl.ts does:
|
||||
special schemes (and scheme-relative references) skip every leading "/"
|
||||
or "\\" before the authority, so `https://\\user:pass@host` still has
|
||||
userinfo. File URLs carry no userinfo and keep their "//" boundary.
|
||||
"""
|
||||
scheme = re.match(SCHEME_PREFIX, text, flags=re.I)
|
||||
name = scheme.group(1).lower() if scheme is not None else None
|
||||
if name is not None and name in SPECIAL_SCHEMES and name != "file":
|
||||
# A same-scheme Location without two slash-equivalent separators is
|
||||
# relative to the recorded request, so its @ belongs to the path.
|
||||
following = text[scheme.end():scheme.end() + 2]
|
||||
if name == base_scheme and (len(following) < 2 or any(char not in "/\\" for char in following)):
|
||||
return None
|
||||
start = scheme.end()
|
||||
else:
|
||||
if scheme is not None:
|
||||
if text[scheme.end():scheme.end() + 2] != "//":
|
||||
return None
|
||||
return scheme.end() + 2, name in SPECIAL_SCHEMES
|
||||
# A non-special base treats backslashes as authority data and only
|
||||
# recognizes two forward slashes as a network reference.
|
||||
if base_scheme is not None and base_scheme not in SPECIAL_SCHEMES:
|
||||
prefix = re.match(r"^[\x00-\x20]*//", text)
|
||||
return None if prefix is None else (prefix.end(), False)
|
||||
prefix = re.match(NETWORK_PREFIX, text)
|
||||
if prefix is None:
|
||||
return None
|
||||
start = prefix.end()
|
||||
if base_scheme == "file":
|
||||
return start, True
|
||||
while start < len(text) and text[start] in "/\\":
|
||||
start += 1
|
||||
return start, True
|
||||
|
||||
|
||||
def without_userinfo(value, base_scheme=None):
|
||||
"""Remove an authority's userinfo using the WHATWG authority boundary.
|
||||
|
||||
Special schemes and scheme-relative references also end the authority
|
||||
at a backslash, so later path text containing "@" is not userinfo.
|
||||
"""
|
||||
binary = isinstance(value, bytes)
|
||||
text = value.decode("latin-1") if binary else value
|
||||
# WHATWG removes these controls before interpreting scheme/separators.
|
||||
# Keep that parse view separate from the exact source spelling and bytes.
|
||||
parsed_text = text.translate(URL_CONTROL_TRANSLATION)
|
||||
located = authority_start(parsed_text, base_scheme)
|
||||
if located is None:
|
||||
return value
|
||||
start, special = located
|
||||
terminators = "/?#\\" if special else "/?#"
|
||||
end = start
|
||||
while end < len(parsed_text) and parsed_text[end] not in terminators:
|
||||
end += 1
|
||||
at = parsed_text.rfind("@", start, end)
|
||||
if at < start:
|
||||
return value
|
||||
parsed_offset = 0
|
||||
raw_start = 0
|
||||
for raw_offset, char in enumerate(text):
|
||||
if char in URL_IGNORABLE_CONTROLS:
|
||||
continue
|
||||
if parsed_offset == start:
|
||||
raw_start = raw_offset
|
||||
if parsed_offset == at:
|
||||
safe = text[:raw_start] + text[raw_offset + 1 :]
|
||||
return safe.encode("latin-1") if binary else safe
|
||||
parsed_offset += 1
|
||||
raise CaptureFailure("decoder", "Normalized URL authority could not be bound to its source.")
|
||||
|
||||
|
||||
def project_record(state, secrets):
|
||||
binaries, numbers, redactions = [], [], []
|
||||
byte_secrets = []
|
||||
|
|
@ -58,16 +139,23 @@ def project_record(state, secrets):
|
|||
return None
|
||||
return text
|
||||
|
||||
def transport_value(value, path, transport_url):
|
||||
def transport_value(value, path, transport_url, base_scheme):
|
||||
if not (transport_url or re.fullmatch(r"/(?:backup/)*request/(?:path|authority)", path)):
|
||||
return value, False
|
||||
binary = isinstance(value, bytes)
|
||||
pattern = rb"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@" if binary else r"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@"
|
||||
replacement = rb"\1" if binary else r"\1"
|
||||
separator = b"@" if binary else "@"
|
||||
safe_url = re.sub(pattern, replacement, value, flags=re.I)
|
||||
if path.endswith("/authority") and separator in safe_url:
|
||||
safe_url = safe_url.rsplit(separator, 1)[1]
|
||||
if transport_url:
|
||||
safe_url = without_userinfo(value, base_scheme)
|
||||
elif path.endswith("/authority"):
|
||||
safe_url = value.rsplit(separator, 1)[-1]
|
||||
else:
|
||||
# Native request.path is origin-form unless it explicitly carries
|
||||
# a scheme (absolute-form). Leading separators are path evidence,
|
||||
# not a header URL reference's network authority.
|
||||
text = value.decode("latin-1") if binary else value
|
||||
safe_url = without_userinfo(value) if re.match(
|
||||
SCHEME_PREFIX, text.translate(URL_CONTROL_TRANSLATION), flags=re.I
|
||||
) else value
|
||||
changed = safe_url != value
|
||||
if changed:
|
||||
redactions.append({"pointer": path, "reason": "transport-credential"})
|
||||
|
|
@ -80,7 +168,7 @@ def project_record(state, secrets):
|
|||
del numbers[before[1]:]
|
||||
del redactions[before[2]:]
|
||||
|
||||
def visit(value, path, depth=0, credential=False, transport_url=False):
|
||||
def visit(value, path, depth=0, credential=False, transport_url=False, base_scheme=None):
|
||||
if depth > 64:
|
||||
raise CaptureFailure("input-limit", "Native capture exceeds the 64-level evidence nesting budget.", path)
|
||||
if credential:
|
||||
|
|
@ -90,7 +178,7 @@ def project_record(state, secrets):
|
|||
binaries.append({"pointer": path, "representation": "producer-bytes", "state": "redacted", "content_base64": None, "bytes": None, "sha256": None})
|
||||
return None
|
||||
if isinstance(value, bytes):
|
||||
safe_bytes, credential_url = transport_value(value, path, transport_url)
|
||||
safe_bytes, credential_url = transport_value(value, path, transport_url, base_scheme)
|
||||
content_base64 = base64.b64encode(value).decode("ascii")
|
||||
digest = hashlib.sha256(value).hexdigest()
|
||||
hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest or secret in str(len(value)) for secret in secrets)
|
||||
|
|
@ -107,7 +195,7 @@ def project_record(state, secrets):
|
|||
return None
|
||||
return safe_text
|
||||
if isinstance(value, str):
|
||||
safe_text, _ = transport_value(value, path, transport_url)
|
||||
safe_text, _ = transport_value(value, path, transport_url, base_scheme)
|
||||
return redact(safe_text, path, value)
|
||||
if isinstance(value, bool) or value is None:
|
||||
return value
|
||||
|
|
@ -125,10 +213,16 @@ def project_record(state, secrets):
|
|||
for index, field in enumerate(value):
|
||||
if not isinstance(field, (list, tuple)) or len(field) != 2 or not isinstance(field[0], bytes):
|
||||
raise CaptureFailure("format", "Native header/trailer is not an ordered byte pair.", pointer(path, index))
|
||||
result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"})])
|
||||
result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"}, base_scheme if field[0].lower() == b"location" else None)])
|
||||
return result
|
||||
return [visit(item, pointer(path, index), depth + 1) for index, item in enumerate(value)]
|
||||
return [visit(item, pointer(path, index), depth + 1, base_scheme=base_scheme) for index, item in enumerate(value)]
|
||||
if isinstance(value, dict):
|
||||
if path == "" or re.fullmatch(r"/backup(?:/backup)*", path):
|
||||
request = value.get("request")
|
||||
scheme = request.get("scheme") if isinstance(request, dict) else None
|
||||
if isinstance(scheme, bytes):
|
||||
scheme = scheme.decode("latin-1")
|
||||
base_scheme = scheme.lower() if isinstance(scheme, str) and re.fullmatch(r"[a-z][a-z0-9+.-]*", scheme, flags=re.I) else None
|
||||
result = {}
|
||||
for key, item in value.items():
|
||||
if not isinstance(key, str):
|
||||
|
|
@ -139,7 +233,7 @@ def project_record(state, secrets):
|
|||
validate_omitted(item, pointer(path, key), depth + 1)
|
||||
redactions.append({"pointer": path, "reason": "explicit-sensitive-value", "scope": "property-name"})
|
||||
continue
|
||||
result[key] = visit(item, pointer(path, key), depth + 1)
|
||||
result[key] = visit(item, pointer(path, key), depth + 1, base_scheme=base_scheme)
|
||||
return result
|
||||
raise CaptureFailure("decoder", "Native parser returned an unexpected value type: " + type(value).__name__ + ".", path)
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,174 @@
|
|||
"""Source-owned tests for the mitmproxy adapter's record projection."""
|
||||
|
||||
import importlib.util
|
||||
import os
|
||||
import sys
|
||||
import types
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
|
||||
def load_capture():
|
||||
"""Load the adapter with fresh mitmproxy stand-ins scoped to its import."""
|
||||
fake_mitmproxy = types.ModuleType("mitmproxy")
|
||||
fake_mitmproxy.ctx = types.SimpleNamespace()
|
||||
fake_mitmproxy.version = types.SimpleNamespace(VERSION="12.2.3")
|
||||
fake_io = types.ModuleType("mitmproxy.io")
|
||||
fake_io.tnetstring = types.ModuleType("mitmproxy.io.tnetstring")
|
||||
stand_ins = {
|
||||
"mitmproxy": fake_mitmproxy,
|
||||
"mitmproxy.io": fake_io,
|
||||
"mitmproxy.io.tnetstring": fake_io.tnetstring,
|
||||
}
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"rea_mitmproxy_capture", os.path.join(os.path.dirname(__file__), "capture.py")
|
||||
)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
with mock.patch.dict(sys.modules, stand_ins):
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
capture = load_capture()
|
||||
|
||||
|
||||
def location(value, scheme=None):
|
||||
state = {"response": {"headers": [[b"Location", value]]}}
|
||||
if scheme is not None:
|
||||
state["request"] = {"scheme": scheme}
|
||||
record = capture.project_record(state, [])
|
||||
sidecar = next(
|
||||
field for field in record["binary_fields"] if field["pointer"] == "/response/headers/0/1"
|
||||
)
|
||||
return record["reported"]["response"]["headers"][0][1], sidecar, record["redactions"]
|
||||
|
||||
|
||||
class TransportUserinfoTests(unittest.TestCase):
|
||||
def test_ignorable_controls_cannot_hide_authority_credentials(self):
|
||||
for control in [b"\t", b"\n", b"\r"]:
|
||||
for value in [b"https://" + control + b"/user:pass@host/path", b"ht" + control + b"tps://user:" + control + b"pass@host/path"]:
|
||||
with self.subTest(value=value):
|
||||
text, sidecar, redactions = location(value, b"https")
|
||||
self.assertNotIn("user", text)
|
||||
self.assertNotIn("pass", text)
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(len(redactions), 1)
|
||||
|
||||
def test_mixed_network_separators_remove_credentials(self):
|
||||
for prefix in [b"\\\\", b"/\\", b"\\/"]:
|
||||
with self.subTest(prefix=prefix):
|
||||
text, sidecar, redactions = location(prefix + b"user:pass@host/path", b"https")
|
||||
self.assertEqual(text, prefix.decode("ascii") + "host/path")
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(len(redactions), 1)
|
||||
|
||||
def test_non_special_base_keeps_backslashes_inside_network_authorities(self):
|
||||
text, sidecar, redactions = location(b"//user\\name:pass@host/x", b"foo")
|
||||
self.assertEqual(text, "//host/x")
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(len(redactions), 1)
|
||||
text, sidecar, redactions = location(b"/\\invoice@customer", b"foo")
|
||||
self.assertEqual(text, "/\\invoice@customer")
|
||||
self.assertEqual(sidecar["state"], "retained")
|
||||
self.assertEqual(redactions, [])
|
||||
|
||||
def test_native_request_paths_preserve_origin_form_and_redact_absolute_form(self):
|
||||
for value in [b"/\\invoice@customer", b"//invoice@customer", b"\\\\invoice@customer"]:
|
||||
with self.subTest(value=value):
|
||||
record = capture.project_record({"request": {"path": value}}, [])
|
||||
self.assertEqual(record["reported"]["request"]["path"], value.decode("ascii"))
|
||||
self.assertEqual(record["binary_fields"][0]["state"], "retained")
|
||||
self.assertEqual(record["redactions"], [])
|
||||
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [])
|
||||
self.assertEqual(record["reported"]["request"]["path"], "https://host/x")
|
||||
self.assertEqual(record["binary_fields"][0]["state"], "redacted")
|
||||
|
||||
def test_same_scheme_locations_preserve_relative_path_text_and_bytes(self):
|
||||
for value in [b"https:invoice@customer", b"https:/invoice@customer", b"https:\\invoice@customer"]:
|
||||
with self.subTest(value=value):
|
||||
text, sidecar, redactions = location(value, b"https")
|
||||
self.assertEqual(text, value.decode("ascii"))
|
||||
self.assertEqual(sidecar["state"], "retained")
|
||||
self.assertEqual(redactions, [])
|
||||
|
||||
def test_location_authorities_still_remove_credentials_with_a_base_scheme(self):
|
||||
for value, scheme, expected in [
|
||||
(b"https://user:pass@host/path", b"https", "https://host/path"),
|
||||
(b"https:\\\\user:pass@host/path", b"https", "https:\\\\host/path"),
|
||||
(b"https:user:pass@host/path", b"http", "https:host/path"),
|
||||
]:
|
||||
with self.subTest(value=value, scheme=scheme):
|
||||
text, sidecar, redactions = location(value, scheme)
|
||||
self.assertEqual(text, expected)
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(len(redactions), 1)
|
||||
|
||||
def test_backup_location_uses_its_own_request_scheme(self):
|
||||
record = capture.project_record({
|
||||
"request": {"scheme": b"http"},
|
||||
"backup": {
|
||||
"request": {"scheme": b"https"},
|
||||
"response": {"headers": [[b"Location", b"https:invoice@customer"]]},
|
||||
},
|
||||
}, [])
|
||||
self.assertEqual(record["reported"]["backup"]["response"]["headers"][0][1], "https:invoice@customer")
|
||||
self.assertEqual(record["redactions"], [])
|
||||
|
||||
def test_backslash_ends_a_special_scheme_authority(self):
|
||||
text, sidecar, redactions = location(b"https://example.test\\part@tail")
|
||||
self.assertEqual(text, "https://example.test\\part@tail")
|
||||
self.assertEqual(sidecar["state"], "retained")
|
||||
self.assertIsNotNone(sidecar["content_base64"])
|
||||
self.assertEqual(redactions, [])
|
||||
|
||||
def test_userinfo_before_a_backslash_is_still_removed(self):
|
||||
text, sidecar, redactions = location(b"https://user:pw@example.test\\path")
|
||||
self.assertEqual(text, "https://example.test\\path")
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(
|
||||
redactions,
|
||||
[{"pointer": "/response/headers/0/1", "reason": "transport-credential"}],
|
||||
)
|
||||
|
||||
def test_ordinary_userinfo_is_removed(self):
|
||||
text, _sidecar, _redactions = location(b"https://user:pw@example.test/a@b")
|
||||
self.assertEqual(text, "https://example.test/a@b")
|
||||
|
||||
def test_empty_userinfo_marker_is_removed(self):
|
||||
self.assertEqual(
|
||||
capture.without_userinfo("https://@host"), "https://host"
|
||||
)
|
||||
|
||||
def test_special_schemes_skip_leading_slashes_before_userinfo(self):
|
||||
for value, expected in [
|
||||
(b"https://\\user:pass@host/path", b"https://\\host/path"),
|
||||
(b"https:\\\\user:pass@host/path", b"https:\\\\host/path"),
|
||||
(b"https:user:pass@host", b"https:host"),
|
||||
(b"//\\user:pass@host", b"//\\host"),
|
||||
]:
|
||||
with self.subTest(value=value):
|
||||
self.assertEqual(capture.without_userinfo(value), expected)
|
||||
text, sidecar, redactions = location(b"https://\\user:pass@host/path")
|
||||
self.assertEqual(text, "https://\\host/path")
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(len(redactions), 1)
|
||||
|
||||
def test_file_base_keeps_paths_after_the_fixed_network_boundary(self):
|
||||
for value in [b"///u@h", b"////u@h"]:
|
||||
with self.subTest(value=value):
|
||||
text, sidecar, redactions = location(value, b"file")
|
||||
self.assertEqual(text, value.decode("ascii"))
|
||||
self.assertEqual(sidecar["state"], "retained")
|
||||
self.assertIsNotNone(sidecar["content_base64"])
|
||||
self.assertEqual(redactions, [])
|
||||
|
||||
def test_file_urls_keep_path_text(self):
|
||||
self.assertEqual(capture.without_userinfo("file:///a@b"), "file:///a@b")
|
||||
|
||||
def test_non_special_schemes_keep_backslashes_in_the_authority(self):
|
||||
self.assertEqual(capture.without_userinfo("foo://a\\b@host/x"), "foo://host/x")
|
||||
self.assertEqual(capture.without_userinfo(b"//a\\b@host"), b"//a\\b@host")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -73,32 +73,32 @@ existing analysis families directly, for example
|
|||
|
||||
## Operations
|
||||
|
||||
| Tool | Kind | What it does |
|
||||
| ----------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `inspect_adb_client` | read | Resolved binary, version, install path; never contacts the server |
|
||||
| `list_adb_devices` | read | Serial, state, transport, model metadata, inferred emulator/physical kind with basis |
|
||||
| `inspect_adb_device` | read | Fixed `getprop` whitelist: build, SDK, patch, fingerprint, product, ABIs, debuggable, `ro.kernel.qemu` |
|
||||
| `inspect_adb_display` | read | `wm size` and `wm density` as declared |
|
||||
| `list_adb_features` | read | `pm list features` (hardware/software features, hex GL versions) |
|
||||
| `list_adb_services` | read | `service list` binder services and interfaces |
|
||||
| `list_adb_processes` | read | `ps -A`: user, pid, ppid, rss, state, name |
|
||||
| `inspect_adb_window` | read | `dumpsys window` focus declarations (what is on screen) |
|
||||
| `read_adb_setting` | read | One `settings get` value; device-side null is reported as such |
|
||||
| `read_adb_logcat` | read | Bounded `logcat -d -t` dump for one buffer, optional pid filter |
|
||||
| `list_adb_packages` | read | `pm list packages -f` with scope; base APK device paths |
|
||||
| `resolve_adb_packages` | read | Case-insensitive package-name substring search with exact-match flag |
|
||||
| `inspect_adb_package` | read | `dumpsys package` projection: versions, install times, installer, flags, permission count |
|
||||
| `list_adb_directory` | read | NUL-framed immediate children, including hidden entries: kinds, permissions, owner, group, sizes, symlink targets |
|
||||
| `pull_adb_package` | read | Complete split APK set from `pm path`, each file digested |
|
||||
| `pull_adb_file` | read | One absolute device path pulled and digested |
|
||||
| `capture_adb_screen` | read | One `screencap -p` frame (exec-out, no device-side file) with dimensions |
|
||||
| `collect_adb_bugreport` | read | Device-generated bugreport zip, digested |
|
||||
| `push_adb_file` | **mutates device** | One local file to a device path (e.g. pushing tools into `/data/local/tmp`); refuses existing targets without explicit overwrite; verifies via the device's own sha256sum |
|
||||
| `install_adb_package` | **mutates device** | Local APK via `adb install [-r]`; preserves the device's Success/Failure verdict |
|
||||
| `uninstall_adb_package` | **mutates device** | Removes a package and its data for user 0 |
|
||||
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -n`) |
|
||||
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
|
||||
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
|
||||
| Tool | Kind | What it does |
|
||||
| ----------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `inspect_adb_client` | read | Resolved binary, version, install path; never contacts the server |
|
||||
| `list_adb_devices` | read | Serial, state, transport, model metadata, inferred emulator/physical kind with basis |
|
||||
| `inspect_adb_device` | read | Fixed `getprop` whitelist: build, SDK, patch, fingerprint, product, ABIs, debuggable, `ro.kernel.qemu` |
|
||||
| `inspect_adb_display` | read | `wm size` and `wm density` as declared |
|
||||
| `list_adb_features` | read | `pm list features` (hardware/software features, hex GL versions) |
|
||||
| `list_adb_services` | read | `service list` binder services and interfaces |
|
||||
| `list_adb_processes` | read | `ps -A`: user, pid, ppid, rss, state, name |
|
||||
| `inspect_adb_window` | read | `dumpsys window` focus declarations (what is on screen) |
|
||||
| `read_adb_setting` | read | One `settings get` value; device-side null is reported as such |
|
||||
| `read_adb_logcat` | read | Bounded `logcat -d -t` dump for one buffer, optional pid filter |
|
||||
| `list_adb_packages` | read | `pm list packages -f` with scope; base APK device paths |
|
||||
| `resolve_adb_packages` | read | Case-insensitive package-name substring search with exact-match flag |
|
||||
| `inspect_adb_package` | read | `dumpsys package` projection: versions, install times, installer, flags, permission count |
|
||||
| `list_adb_directory` | read | NUL-framed immediate children, including hidden entries: kinds, permissions, owner, group, sizes, symlink targets |
|
||||
| `pull_adb_package` | read | Complete split APK set from `pm path`, each file digested |
|
||||
| `pull_adb_file` | read | One absolute device path pulled and digested |
|
||||
| `capture_adb_screen` | read | One `screencap -p` frame (exec-out, no device-side file) with dimensions |
|
||||
| `collect_adb_bugreport` | read | Device-generated bugreport zip, digested |
|
||||
| `push_adb_file` | **mutates device** | One local file to a device path (e.g. pushing tools into `/data/local/tmp`); checks for existing targets and refuses them without explicit overwrite (the check is not atomic); compares digests when device `sha256sum` is available |
|
||||
| `install_adb_package` | **mutates device** | Local APK via `adb install [-r]`; preserves the device's Success/Failure verdict |
|
||||
| `uninstall_adb_package` | **mutates device** | Removes a package and its data for user 0 |
|
||||
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -n`) |
|
||||
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
|
||||
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
|
||||
|
||||
## Interpret the result
|
||||
|
||||
|
|
@ -118,6 +118,16 @@ existing analysis families directly, for example
|
|||
while POSIX hosts retain valid POSIX filenames such as names with spaces or
|
||||
colons. A package pull checks reported names against the destination
|
||||
filesystem before pulling, and refuses names that would share a local path.
|
||||
- **Package path refusals are diagnosable.** Refusals from `pull_adb_package`
|
||||
return `unreadable_output` with `retryable: false`, keep the `pm path` output
|
||||
in `details.captured_output`, and name the refusal in `details.reason`.
|
||||
`unsafe_device_path` lists refused entries in
|
||||
`details.diagnostics.rejected_paths` as `{device_path, reason}`;
|
||||
`duplicate_local_destination` reports the shared `file_name` and both
|
||||
`device_paths`; `local_destination_unavailable` reports the `device_path`,
|
||||
`file_name`, and filesystem `error_code`. Diagnostics retain raw device
|
||||
strings, while the caller-facing message escapes invisible and bidirectional
|
||||
characters so they cannot hide text or reorder the path.
|
||||
- **Splits are part of the answer.** `pm path` enumerates the base APK and
|
||||
every installed split; pulls cover the whole set. System packages whose
|
||||
APKs keep other names report the `unknown` role rather than guessing.
|
||||
|
|
@ -127,8 +137,10 @@ existing analysis families directly, for example
|
|||
partial coverage instead of dropping rows.
|
||||
- **Mutations are explicit.** `push_adb_file`, `install_adb_package`, and
|
||||
`uninstall_adb_package` declare `mutatesTarget` and never run implicitly:
|
||||
pushes refuse existing device files unless `overwrite` is set and verify
|
||||
the transfer with the device's own `sha256sum`; installs preserve the
|
||||
pushes check for existing device files before transfer and refuse them
|
||||
unless `overwrite` is set. The check is not atomic, so a different process
|
||||
can create the path before `adb push` writes it. When available, the device's
|
||||
`sha256sum` verifies the transfer; installs preserve the
|
||||
device's `Failure [...]` reason verbatim.
|
||||
- **Producer identity stays attached.** Every result embeds the producing
|
||||
adb client (`binary_path`, `path_source`, `version`, `installed_path`).
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ Inspect native SAZ or HAR 1.2 offline with REA, or connect Everywhere's separate
|
|||
- `inspect_web_page` / `rea inspect-web-page` captures DOM structure, accessibility nodes, scripts, resources, safe response/DOM metadata, attach-window network and console metadata, WebSocket frame sizes, workers, quota, and optionally selected storage key names, redacted storage-content fingerprints, or script sources.
|
||||
- `analyze_web_bundle` / `rea analyze-web-bundle` parses selected script artifacts without execution and derives chunk edges, route and endpoint candidates, vendor fingerprints, static WebMCP declarations, and optionally source-map/original-source evidence.
|
||||
- `observe_web_session` / `rea observe-web-session` arms an observation window of the requested duration for an external user action and records ordered reload, SPA navigation, redirect, failure, lifecycle, and target-termination metadata.
|
||||
- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. REA never exposes `WebMCP.invokeTool`; malformed protocol payloads fail explicitly instead of returning incomplete tool metadata.
|
||||
- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. Declared `readOnly`, `untrustedContent`, `consequential`, and `autosubmit` annotation hints are reported per registration: `true`/`false` are declared values, `null` means the declaration did not provide the hint in that capture, and a missing `consequential` key means the hint was not recorded, either in a historical capture or because the observed annotation was invalid. Consult discovery completeness and exclusions to distinguish these cases. Malformed annotation values retain the registration with incomplete coverage and an `invalid_protocol_value` exclusion. Other malformed protocol payloads can fail discovery explicitly. REA never exposes `WebMCP.invokeTool`.
|
||||
- `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence.
|
||||
- `capture_web_screenshot` / `rea capture-web-screenshot` returns a content-addressed visible-viewport PNG.
|
||||
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
|
||||
|
|
@ -190,6 +190,36 @@ rea discover-webmcp-tools http://127.0.0.1:9222 TARGET_ID \
|
|||
--json
|
||||
```
|
||||
|
||||
### WebMCP discovery prerequisites
|
||||
|
||||
`discover_webmcp_tools` reports only what the attached browser's experimental
|
||||
`WebMCP` CDP domain exposes. When a browser rejects the unknown domain, the
|
||||
result reports `status: "unavailable"`. Whether a page can register tools at
|
||||
all depends on several conditions that are not interchangeable:
|
||||
|
||||
- A site's own origin-trial token can enable the API for that origin
|
||||
**provided the trial is still active, the browser version and platform
|
||||
support it, and user settings permit it** — token presence alone is not a
|
||||
guarantee. The trial is announced for Chrome 149 and later.
|
||||
- The `WebMCPTesting` feature — `--enable-features=WebMCPTesting` at launch, or
|
||||
`chrome://flags/#enable-webmcp-testing` in a normal profile — enables the
|
||||
API page-side for that browser instance, independent of per-site tokens.
|
||||
In one pilot run, a site whose registration code was present but whose
|
||||
served token was scoped for third-party use registered 0 tools under a
|
||||
default launch and 4 under the flag; that the token did not enroll the
|
||||
first-party document is consistent with this but is an inference, not a
|
||||
measured Chrome behavior.
|
||||
|
||||
In Chrome 154 specifically (one tested version), `WebMCP.enable` answered
|
||||
under a default launch — the debug domain was not flag-gated in that run.
|
||||
Other versions may differ, so an unattributed zero-tool result should be
|
||||
read as registration state only after domain availability is confirmed.
|
||||
|
||||
Discovery is attach-only: the command needs a reachable loopback CDP endpoint
|
||||
and a target ID from `list_browser_targets`, and it never navigates, reloads,
|
||||
or evaluates the page. Observe the page in the state you want to measure;
|
||||
registered tools reflect the document as loaded.
|
||||
|
||||
Accessibility names/descriptions, console primitive text, JSON body shapes, WebSocket shapes, script content, and storage key names are selected independently:
|
||||
|
||||
```bash
|
||||
|
|
|
|||
|
|
@ -15,6 +15,12 @@ there is no separate origin allowlist or request-interception layer. Environment
|
|||
settings (including service workers blocked by default), empty storage, and a
|
||||
final sanitized URL capture are supplied by default. Set
|
||||
`environment.service_workers` to `allow` when the application requires them.
|
||||
For attached browsers, `block` rejects captures when the selected page or its
|
||||
frames are controlled by a service worker or already have a registration,
|
||||
including after navigation. Existing caller registrations are never removed.
|
||||
Navigation can run destination code before a newly discovered worker is rejected;
|
||||
that rejection does not roll back browser effects. REA removes its owned
|
||||
future-document scripts, restores live frame methods, then disconnects.
|
||||
Add secret declarations, storage seeds, and additional artifact or event
|
||||
capture only when needed. Action, secret, and storage counts are not capped.
|
||||
Duration, action, and navigation
|
||||
|
|
@ -249,8 +255,12 @@ Each observed request object receives a capture-scoped `transaction_id`.
|
|||
Requests sharing a URL remain distinct. Redirects link to the preceding
|
||||
transaction only when it was observed; a null predecessor does not establish
|
||||
that no earlier hop existed. Request events require both transaction fields;
|
||||
an absent ID is malformed capture data, while a null redirect predecessor is
|
||||
an explicit unknown.
|
||||
an absent ID is malformed capture data. `redirected_from_transaction_state`
|
||||
records `root` when Playwright observed no predecessor and `unknown` when a
|
||||
predecessor exists but was outside the capture. A linked predecessor is carried
|
||||
by `redirected_from_transaction_id` with no state field. Older captures without
|
||||
the state field retain the conservative meaning that a null predecessor is
|
||||
unknown.
|
||||
|
||||
`scenario.network_content` records the resolved content selection. Comparing
|
||||
captures with different selections reports network evidence as unknown with a
|
||||
|
|
@ -284,8 +294,10 @@ browser cleanup, with a provider-owned five-second timeout per read and
|
|||
cancellation support. Failed or unexposed reads retain per-content states and make selected
|
||||
event coverage incomplete. Body events can arrive during later steps or final
|
||||
drain; event bounds describe receipt during the step, while the source event
|
||||
reference establishes the transaction phase. Comparison attributes content to
|
||||
its source event's step and excludes the sequence reference from normalization.
|
||||
reference establishes the transaction phase. Comparison attributes content, like
|
||||
every other event of its transaction, to the step that sent the request (the
|
||||
receipt step when that request was not observed) and excludes the sequence
|
||||
reference from normalization.
|
||||
Playwright scenario capture does not supply initiator stacks; passive CDP
|
||||
observation retains its own initiator evidence.
|
||||
|
||||
|
|
@ -300,7 +312,22 @@ an unchanged claim; observed differences may still prove a changed result.
|
|||
The comparison records its full normalization commitment and SHA-256 digest.
|
||||
Built-in rules exclude step elapsed time and event sequence/index fields,
|
||||
compare screenshots by content digest, and compare DOM/accessibility captures
|
||||
by normalized text. Optional caller rules are bounded exact-literal
|
||||
by normalized text. Network events are grouped by transaction without their
|
||||
capture-local transaction IDs, attributed to the step that sent the request,
|
||||
and compared as an unordered collection after caller rules apply, because
|
||||
concurrent responses arrive in any order and can land in a later step; a
|
||||
redirect retains the complete predecessor ancestry, including each hop's own
|
||||
request, response outcome, and selected network content. Each transaction's
|
||||
capture-local IDs, sequence/index fields, and content source-sequence anchor
|
||||
are removed. Caller rules normalize retained strings before hashing. The
|
||||
`preserve_redirect_transaction_ancestry` built-in rule fingerprints each hop
|
||||
as canonical SHA-256 of `{ events, redirected_from }`: `events` is the sorted
|
||||
list of its normalized event SHA-256 digests, and `redirected_from` is its
|
||||
predecessor fingerprint or null for a root. Selected transaction fingerprints
|
||||
are sorted for the step's network comparison. Temporary numeric graph indices
|
||||
are lookup references only and do not enter the final digest. Missing or cyclic
|
||||
predecessor evidence yields unknown comparability instead of an unchanged claim.
|
||||
Other events keep receipt order. Optional caller rules are bounded exact-literal
|
||||
replacements over named artifact kinds. Rules and artifact-kind lists are
|
||||
sorted canonically before application, making the same policy reproducible
|
||||
regardless of input order. Rules operate only on already-redacted durable
|
||||
|
|
@ -318,3 +345,7 @@ selection. Captures with different selections report event evidence as unknown
|
|||
with a recovery instruction, even when no selected family emitted an event.
|
||||
Captures written before this field existed omit it, so their event evidence is
|
||||
also unknown.
|
||||
|
||||
Network normalization also commits the removal of capture-local transaction IDs,
|
||||
attribution to the requesting step, and unordered transaction comparison. These
|
||||
built-in rule identities participate in the normalization digest.
|
||||
|
|
|
|||
|
|
@ -263,7 +263,12 @@ tree matches that commit remains unknown. It honors the source tree's `.gitignor
|
|||
and applies default exclusions for common generated and dependency paths such
|
||||
as `node_modules/`, `dist/`, and `*.log`. Pattern-based exclusions record the
|
||||
matched pattern and whether it came from project, default, caller, or sensitive
|
||||
path policy. Set `REA_REFERENCE_SECRET_PATTERNS_JSON` to a JSON array of ignore
|
||||
path policy. Links into excluded paths retain their internal target metadata;
|
||||
the exclusion explains why the target has no inventory entry, and the inventory
|
||||
remains partial. Exclusions record `entry_kind` when the importer observed it;
|
||||
older saved exclusions without that field have unknown kind. An excluded
|
||||
ancestor proves an internal descendant only when its kind is `directory`.
|
||||
Set `REA_REFERENCE_SECRET_PATTERNS_JSON` to a JSON array of ignore
|
||||
patterns when you want to mark selected paths as sensitive; those patterns are
|
||||
reported separately as configured-secret exclusions.
|
||||
|
||||
|
|
|
|||
|
|
@ -129,6 +129,12 @@ and Claude Code reports that autocompact is thrashing. To avoid this:
|
|||
The sizes above come from REA's own `tools/list`. A live Claude Code session
|
||||
against DeepSeek's API is unverified.
|
||||
|
||||
For a client whose stdio reader rejects the complete catalog frame, add
|
||||
`"REA_MCP_OUTPUT_SCHEMA_PROFILE": "off"` to REA's `env` object and restart the
|
||||
client. This omits only advertised output schemas; REA still validates complete
|
||||
structured results against its canonical contracts. See [MCP output schema
|
||||
profiles](mcp-contracts.md#output-schemas).
|
||||
|
||||
## Kimi Code
|
||||
|
||||
Setup registers `rea` in `~/.kimi-code/mcp.json`, or
|
||||
|
|
@ -294,7 +300,11 @@ the first native server into an empty table, it copies the fallback's
|
|||
`mcpServers` entries into the native table so they keep loading, and leaves
|
||||
the fallback file untouched. Existing REA environment overrides are preserved
|
||||
when refreshing its command. A malformed or unreadable fallback file fails setup instead of
|
||||
being silently suppressed; repair or remove it and rerun. ZCode disables a
|
||||
being silently suppressed; repair or remove it and rerun. Uninstall removes an
|
||||
REA-owned `rea` entry from both the native table and the fallback, backing up
|
||||
each changed file, so an emptied native table cannot load a leftover fallback
|
||||
registration; other fallback servers, settings and a foreign `rea` entry are
|
||||
kept. ZCode disables a
|
||||
server with `enable: false` on its entry; doctor reports such a registration
|
||||
as stale, and rerunning setup removes the flag to repair it. Native configuration
|
||||
uses strict JSON; setup refuses comments, trailing commas and a BOM rather than
|
||||
|
|
|
|||
|
|
@ -103,6 +103,23 @@ build's source revision, so that field remains unknown.
|
|||
engine finished its configured extraction without those reported conditions;
|
||||
default skip rules and format coverage still apply. It does not establish
|
||||
complete firmware reconstruction.
|
||||
- `extraction_gaps` makes those conditions recoverable: a row identifies an
|
||||
unextracted region or a failed task/stage. A chunk retains its identity and
|
||||
byte range, plus its original-file range when it belongs to the selected input.
|
||||
A multi-file stage retains its reported ID and handler with unknown byte ranges;
|
||||
a standalone task failure keeps unavailable stage identity null. Rows retain exact dependency names from
|
||||
the producer's missing-dependency reports; an accepted report with no names
|
||||
remains a failure with unknown dependencies; no executable identity or specific installation step is inferred. Each row's `continuation_id`
|
||||
references one instruction in `extraction_continuations`, so many regions
|
||||
share recovery prose without losing any region identity. Instructions use
|
||||
already configured capabilities where one exists. Every extraction rerun
|
||||
requires a fresh absent `output_directory`, preserving the prior evidence.
|
||||
Nested unknown chunks use `inspect_firmware_regions` on the actual published
|
||||
child rather than repeating the same extraction with unchanged handlers.
|
||||
Inspection requires separately configured Binwalk (`REA_BINWALK_COMMAND`);
|
||||
without it, the inspection step is unavailable and the region remains unexamined. A listed gap does not
|
||||
mean the target is corrupt or that a retained region is empty; REA installs
|
||||
nothing.
|
||||
- The destination must be absent, absolute, with an existing parent. Publication
|
||||
uses exclusive, symlink-resistant writes and durable digest readback. Existing
|
||||
destinations are never overwritten. This is not an atomic directory rename:
|
||||
|
|
@ -167,6 +184,12 @@ records gzip offset/length and extracted file hashes. Real verification checks
|
|||
CLI/MCP, full-file and selected-range extraction, unknown chunks, depth limits,
|
||||
content parity and unchanged source bytes. No binary fixture is committed.
|
||||
|
||||
The optional missing-extractor lane asserts dependency names and the owning
|
||||
handler in `extraction_gaps`. Those assertions must be run against the pinned
|
||||
Unblob release to establish real-provider coverage; synthetic producer fixtures
|
||||
alone do not verify Unblob's report nesting or stage identities. The gap ownership
|
||||
claims remain unverified against real Unblob until that lane runs.
|
||||
|
||||
Additional lanes are explicit, so the base lane stays small:
|
||||
|
||||
```sh
|
||||
|
|
|
|||
|
|
@ -52,13 +52,15 @@ section names or the record being inside the initial 4 KiB header probe.
|
|||
Program and section declarations must agree; duplicate, conflicting, truncated
|
||||
or out-of-file records are rejected. Stripped section tables and section-only
|
||||
records are supported. Extended table numbering is explicitly unsupported.
|
||||
For carved ELFs whose section table extends beyond the captured file, REA
|
||||
examines every complete captured section header and records the unavailable
|
||||
headers as a limitation. A valid program-header ABI record can still establish
|
||||
the supported declaration. Available section declarations must agree with it;
|
||||
their duplicate, conflicting or unreadable records remain errors. A missing
|
||||
program-header record does not make ABI declarations in uncaptured sections
|
||||
absent or supply a default ABI.
|
||||
GNU `SHT_GNU_ATTRIBUTES` sections are refused by numeric section type until REA
|
||||
parses and reconciles their declarations; section names and payload sizes do not
|
||||
control detection or allocation. For carved ELFs whose section table extends
|
||||
beyond the captured file, REA examines every complete captured section header
|
||||
and records the unavailable headers as a limitation. The provider refuses an
|
||||
incomplete ABI declaration inspection even when a program-header ABI record is
|
||||
valid, because an unavailable section header could carry an unsupported or
|
||||
conflicting declaration. A missing program-header record does not make ABI
|
||||
declarations in uncaptured sections absent or supply a default ABI.
|
||||
The field layout follows
|
||||
[`Elf_MIPS_ABIFlags_v0`](https://github.com/bminor/glibc/blob/master/elf/elf.h).
|
||||
|
||||
|
|
|
|||
|
|
@ -154,12 +154,15 @@ An inferred no-return call can terminate that body before subsequent bytes.
|
|||
Pseudocode, assembly, procedure info, instruction windows, callee and caller
|
||||
reads, and dossiers report observed terminal call sites in Evidence limitations,
|
||||
including resolved callee names, addresses, and FunctionManager flags when
|
||||
available. Instruction windows
|
||||
and dossiers also retain these warnings in their result limitations. An
|
||||
unresolved callee or unestablished terminal-flow basis remains explicit. These
|
||||
observations qualify the analysis; they do not prove that a no-return flag is
|
||||
incorrect or recover excluded code. Analysis profiles version this observation
|
||||
contract so older snapshots cannot replay unqualified results.
|
||||
available. On x86-32, the same views also report the exact Borland/Delphi-style
|
||||
FS exception-registration pattern when its handler address falls outside the
|
||||
function body. This is a static registration observation; it does not establish
|
||||
handler execution or change the body. Instruction windows and dossiers also
|
||||
retain these warnings in their result limitations. An unresolved callee or
|
||||
unestablished terminal-flow basis remains explicit. These observations qualify
|
||||
the analysis; they do not prove that a no-return flag is incorrect or recover
|
||||
excluded code. Analysis profiles version this observation contract so older
|
||||
snapshots cannot replay unqualified results.
|
||||
|
||||
On non-Windows hosts, the ephemeral import repair clears no-return flags on the
|
||||
known returning external ABI names `malloc`, `__tls_get_addr`, `memcpy`, `memset`,
|
||||
|
|
@ -189,7 +192,7 @@ host/provider acceptance run with only the host compiler.
|
|||
|
||||
`npm run verify:ghidra:windows` checks source-owned native x86-64 PE applications;
|
||||
`-- --x86` selects x86 and `-- --dll` selects DLLs, and the flags can be combined.
|
||||
These lanes check all 25 read-only operations, target/snapshot/import SHA-256 linkage,
|
||||
These lanes check all 26 read-only operations, target/snapshot/import SHA-256 linkage,
|
||||
and project, endpoint, process, and runtime cleanup. The independent native
|
||||
lane checks DACLs, handle admission, cancellation, and Job Object lifecycle.
|
||||
`npm run verify:ghidra:windows:package` packs and installs REA into an isolated
|
||||
|
|
|
|||
|
|
@ -58,6 +58,12 @@ original stable file snapshot, including pointer-backed strings. It does not
|
|||
scan arbitrary byte strings as compiler metadata or interpret a fat Mach-O
|
||||
container as one selected architecture.
|
||||
|
||||
For ELF images without a named build-info section, discovery checks each
|
||||
nonempty writable, non-executable load segment in program-header order. This
|
||||
includes sectionless PIE images with metadata in a later data segment. Named
|
||||
sections retain priority, and alignment, file mapping and record validation
|
||||
remain required.
|
||||
|
||||
Historical moduleless Go binaries can store their empty module-string header in
|
||||
declared zero-initialized virtual memory rather than file-backed bytes. The reader
|
||||
recognizes this only when the complete optional module header lies in that memory
|
||||
|
|
@ -78,9 +84,13 @@ host permission denial, changed artifacts and cancellation retain distinct
|
|||
errors. The selected file is never modified; this inspection creates no
|
||||
temporary files, starts no subprocesses and accesses no network endpoint.
|
||||
|
||||
Resource guards bound the selected file to 256 MiB, aggregate embedded strings
|
||||
Resource guards bound the selected file and cumulative header searches to 256 MiB
|
||||
each, aggregate embedded strings
|
||||
to 1 MiB, structural table decoding to 16 MiB and the complete normalized metadata
|
||||
report to 8 MiB. Oversized results fail without partial success. MCP also applies
|
||||
report to 8 MiB. Identical search spans with the same virtual alignment are
|
||||
searched once; partially overlapping spans consume the cumulative search budget
|
||||
and fail with a CPU resource constraint when it is exhausted. Oversized results
|
||||
fail without partial success. MCP also applies
|
||||
its [response transport budget](mcp-contracts.md), which counts both text and
|
||||
structured content; a report within the metadata limit can still require export
|
||||
from retained Evidence. These limits are not operating-system memory or CPU
|
||||
|
|
|
|||
|
|
@ -6,7 +6,9 @@ Choose one lifecycle mode in an upstream MCP JSON registration. Set `REA_IDA_MCP
|
|||
|
||||
## Supported upstream versions
|
||||
|
||||
Each mode accepts one upstream tool profile, checked against `tools/list` before any analysis runs. Upstream's `main` branch has since moved to 2.0.0, which renamed the GUI tools and dropped `idb_close`, so it serves neither profile. Installing from that branch fails at startup with the missing tool names in the error diagnostics.
|
||||
Each mode accepts one upstream tool profile, checked against `tools/list` before any analysis runs. The attached profile requires the legacy GUI tool names; upstream 2.0.0 renamed those tools, and REA has not established an equivalent attached-mode lifecycle and ownership contract for that catalog. It therefore fails closed with the missing tool names in the error diagnostics.
|
||||
|
||||
The headless profile is defined by the database-supervisor API, not by an upstream version string. The upstream source snapshot [`c133c385`](https://github.com/mrexodia/ida-pro-mcp/tree/c133c3853faa111a9b00ee615c013b720d0c4acd) includes `idb_close` (introduced by [upstream PR #495](https://github.com/mrexodia/ida-pro-mcp/pull/495)) alongside `idb_open` and `idb_list`. Earlier builds may report the same `2.0.0` version while lacking that tool, so REA checks the live catalog rather than treating a version number or branch name as a compatibility guarantee.
|
||||
|
||||
| Mode | Upstream profile | Known working |
|
||||
| -------- | ------------------- | ------------------------------------------------------ |
|
||||
|
|
@ -137,11 +139,11 @@ For guided registration, set `REA_IDA_MCP_CONFIG` before `rea setup --client <cl
|
|||
|
||||
All profiles support procedure/string inventory and literal or regex search, procedure resolution, pseudocode, assembly, raw function instructions, resolved callees, incoming address references, and a function dossier. Inventories and modern pseudocode/assembly follow upstream pagination. Upstream limits on modern xrefs and callees are retained as explicit truncation limitations. In the modern headless profile, targets that are not proven x86-only use processor-neutral `callgraph` edges instead of the x86 instruction checks in upstream `callees` (mrexodia/ida-pro-mcp#549). REA requests depth zero: the pinned upstream emits the root's outgoing edges without scanning child functions, then REA batches function-name lookup for those addresses. The graph covers recognized function destinations only; imports and other non-function targets remain unknown. A server without `callgraph` returns an empty relationship list with an explicit coverage limitation. Graph and path workflows retain those limitations alongside observed edges. A failed or inconsistent graph reply remains an error. A transport-level truncated preview is rejected rather than reported as complete evidence; the error preserves upstream retrieval guidance.
|
||||
|
||||
The modern headless profile also supports `resolve_containing_procedure` and `list_documents`. An address resolves through upstream `lookup_funcs` to the canonical entry of the function that contains it; the body stays unavailable like other IDA identities. When no function contains it, REA reads the database-scoped minimal `survey_binary` and reports `not_in_procedure` for an address inside one of its segments (ends are exclusive) and `outside_segments` otherwise. A missing, truncated or inconsistent survey fails the read instead of guessing a reason. `list_documents` returns the single admitted program path, so `inspect_address_context` and `trace_feature` need no explicit document. `trace_feature` marks a trace truncated, with a residual unknown for that address, when upstream caps the references it returned. The legacy and native profiles do not declare these operations.
|
||||
All profiles expose `list_documents` as the single target admitted for the session. Attached mode verifies its digest against IDA metadata before returning it; headless mode binds the digest-checked private input. `inspect_address_context`, `trace_feature`, and `binary_overview` can therefore resolve that identity without upstream document-listing tools. Segment enumeration is optional: when the selected profile cannot provide it, `binary_overview` still returns document and procedure/string counts and omits `segments` and `segment_count`; omission means unobserved, not empty. The modern headless profile also supports `resolve_containing_procedure`. An address resolves through upstream `lookup_funcs` to the canonical entry of the function that contains it; the body stays unavailable like other IDA identities. When no function contains it, REA reads the database-scoped minimal `survey_binary` and reports `not_in_procedure` for an address inside one of its segments (ends are exclusive) and `outside_segments` otherwise. A missing, truncated or inconsistent survey fails the read instead of guessing a reason. `trace_feature` marks a trace truncated, with a residual unknown for that address, when upstream caps the references it returned. Attached profiles do not declare address containment resolution.
|
||||
|
||||
The legacy profile additionally supports direct callers. Its producer may report call-site or interior addresses; REA resolves the reported function symbol to its canonical entry. The modern and native profiles do not prove direct callers from its code/data xref labels, so that capability is unavailable and the dossier marks it unknown.
|
||||
|
||||
Complete body ranges, typed reference edges, CFG, referenced data, source locals, comment kinds, and database revision are unavailable. Empty unsupported dossier facets mean **unknown**, not observed absence. `binary_overview`, whole-symbol inventory, mutation, debugging, arbitrary Python forwarding, and requested GUI navigation are outside this adapter's coverage. Check session availability before composing a workflow that needs those facets.
|
||||
Complete body ranges, typed reference edges, CFG, referenced data, source locals, comment kinds, and database revision are unavailable. Empty unsupported dossier facets mean **unknown**, not observed absence. Whole-symbol inventory, mutation, debugging, arbitrary Python forwarding, and requested GUI navigation are outside this adapter's coverage. Check session availability before composing a workflow that needs those facets.
|
||||
|
||||
`provider.version` identifies the REA adapter; the profile explicitly records `version_scope: "rea-ida-adapter"`. The MCP handshake's `serverInfo` is separate and does not establish IDA or Python distribution versions. Raw observations preserve producer metadata and output. Attached hashes identify the recorded original input; headless identity uses the digest-verified private copy and observed worker input path. Neither proves source-code recovery or an unmodified analysis database.
|
||||
|
||||
|
|
|
|||
|
|
@ -64,11 +64,21 @@ preserves a verified `.rea.backup.<sha256>.recovery` copy of the current bytes.
|
|||
Both snapshots remain available; a failed rollback reports the current recovery
|
||||
copy's path and the restore failure.
|
||||
Setup also retains valid environment overrides in REA's existing server entry,
|
||||
such as `REA_MCP_INPUT_SCHEMA_PROFILE` and `REA_MCP_MAX_RESPONSE_BYTES`.
|
||||
such as `REA_MCP_INPUT_SCHEMA_PROFILE`, `REA_MCP_OUTPUT_SCHEMA_PROFILE`, and
|
||||
`REA_MCP_MAX_RESPONSE_BYTES`.
|
||||
Detected provider settings replace only matching environment keys.
|
||||
Setup and doctor report a malformed REA environment map or invalid runtime
|
||||
setting before setup writes files. Repair the named setting in the client
|
||||
configuration; setup does not silently discard the rest of the map.
|
||||
This includes malformed loader/exclusion JSON arrays and a Ghidra compiler
|
||||
specification without its required language identifier. Valid overrides keep
|
||||
their original strings, including empty Hopper arguments. DeepSeek Harness
|
||||
`!!js` environment expressions remain unevaluated and preserved; registration
|
||||
alignment does not verify the runtime values those expressions will produce.
|
||||
For layered OpenCode files, each file's values are checked before merging, while
|
||||
setting dependencies are checked against the effective merged environment.
|
||||
Setup preserves each layer's overrides instead of copying inherited settings
|
||||
into the file it updates.
|
||||
|
||||
After setup, REA reports which agents, analysis tools, and workflow files passed
|
||||
its final checks. Restart any agent named in the completion message, then begin
|
||||
|
|
@ -129,6 +139,150 @@ Setup pins package-runner MCP registrations to the exact installed REA version,
|
|||
installs the matching skill and on-demand references in the same plan, and adds
|
||||
`startup_timeout_sec = 30` for Codex and Grok Build.
|
||||
|
||||
## Limit REA to one project or agent profile
|
||||
|
||||
Setup registers REA for the whole client. For Claude Code it writes
|
||||
`mcpServers.rea` in `~/.claude.json` and installs the bundled skill under
|
||||
`~/.claude/skills`, so every project that Claude Code opens can call REA. To
|
||||
offer REA's tools only where they're wanted, use one of the routes below.
|
||||
|
||||
Both routes control which sessions discover REA's MCP server and skill. They
|
||||
leave shell commands, filesystem access and network access available. An agent
|
||||
under the same OS account can run `rea` from `PATH` or `npx` and read other
|
||||
profiles and REA output. For clean-room separation, use an implementation
|
||||
environment with filesystem and network restrictions that prevent reading or
|
||||
fetching REA, the target and analysis output, including through shared
|
||||
installations and package registries. A separate account or container must
|
||||
enforce those restrictions.
|
||||
|
||||
For Claude Code, check existing default-profile state before choosing either
|
||||
route. Setup is additive: a previous user-level REA registration and personal
|
||||
skill remain active in other projects. Back up the default configuration and
|
||||
REA skill first, then, with `CLAUDE_CONFIG_DIR` unset, remove the REA user
|
||||
registration with `claude mcp remove --scope user rea` and remove the known
|
||||
REA-installed `~/.claude/skills/reverse-engineer-anything` directory. Preserve
|
||||
unrelated registrations and skills. `rea uninstall` applies to other clients
|
||||
as well, so it is broader than this Claude-only migration.
|
||||
|
||||
**A separate Claude Code profile.** Claude Code keeps settings, skills and session
|
||||
history under `CLAUDE_CONFIG_DIR`. Setup follows the same
|
||||
variable: it writes the registration to `$CLAUDE_CONFIG_DIR/.claude.json` and
|
||||
the skill to `$CLAUDE_CONFIG_DIR/skills`.
|
||||
|
||||
```bash
|
||||
CLAUDE_CONFIG_DIR=~/.claude-rea npx rea-agents setup --client claude_code
|
||||
CLAUDE_CONFIG_DIR=~/.claude-rea claude
|
||||
```
|
||||
|
||||
A new profile may need authentication. Console sign-ins without an API key are
|
||||
stored outside `CLAUDE_CONFIG_DIR`; see
|
||||
[Claude Code authentication](https://code.claude.com/docs/en/authentication#log-in-with-multiple-accounts).
|
||||
Sessions started without the variable use the default profile's registration,
|
||||
skill and history.
|
||||
|
||||
Inline assignments apply only to that command. Keep the same prefix for every
|
||||
profile check, setup rerun, update, or removal; bare commands use the current
|
||||
environment and can otherwise inspect or recreate default-profile state:
|
||||
|
||||
```bash
|
||||
CLAUDE_CONFIG_DIR=~/.claude-rea claude mcp list
|
||||
CLAUDE_CONFIG_DIR=~/.claude-rea npx rea-agents doctor --client claude_code --skill --json
|
||||
CLAUDE_CONFIG_DIR=~/.claude-rea npx -y rea-agents@latest setup --client claude_code --dry-run
|
||||
```
|
||||
|
||||
After reviewing an update plan, rerun that same prefixed setup command without
|
||||
`--dry-run` and approve its exact changes. Profile removal likewise needs the
|
||||
prefix on `claude mcp remove --scope user rea` and any `rea uninstall` command;
|
||||
keep the broader client scope of uninstall in mind. Restart the selected
|
||||
profile after changing its pinned registration.
|
||||
|
||||
`claude mcp remove` removes only the registration. To remove REA from this
|
||||
profile without running the broader uninstall flow, first back up the profile's
|
||||
REA-managed skill directory, then remove
|
||||
`$CLAUDE_CONFIG_DIR/skills/reverse-engineer-anything` as well. Keep other skills
|
||||
in the profile intact.
|
||||
|
||||
The commands above use POSIX shell assignment syntax. In native Windows
|
||||
PowerShell, select the profile for the whole setup/launch/check session and
|
||||
restore the previous selection afterward:
|
||||
|
||||
```powershell
|
||||
$previousClaudeConfigDir = $env:CLAUDE_CONFIG_DIR
|
||||
try {
|
||||
$env:CLAUDE_CONFIG_DIR = Join-Path $HOME ".claude-rea"
|
||||
npx rea-agents setup --client claude_code
|
||||
claude mcp list
|
||||
npx rea-agents doctor --client claude_code --skill --json
|
||||
npx -y rea-agents@latest setup --client claude_code --dry-run
|
||||
# After reviewing the plan, repeat setup without --dry-run and approve it.
|
||||
claude
|
||||
# Profile removal, when intended, also belongs inside this selected scope:
|
||||
# claude mcp remove --scope user rea
|
||||
} finally {
|
||||
if ($null -eq $previousClaudeConfigDir) {
|
||||
Remove-Item Env:CLAUDE_CONFIG_DIR -ErrorAction SilentlyContinue
|
||||
} else {
|
||||
$env:CLAUDE_CONFIG_DIR = $previousClaudeConfigDir
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Use PowerShell for these Windows examples rather than pasting POSIX assignments
|
||||
into cmd.exe. Keep profile updates, readiness checks and removal inside the
|
||||
same selected environment; the `finally` block restores the previous profile
|
||||
or unsets the variable when it was absent.
|
||||
|
||||
**One project.** Install without configuring Claude Code: use the curl
|
||||
installer's `--no-setup`, or run setup only for other clients. Then register
|
||||
REA from the project directory at Claude Code's local scope.
|
||||
|
||||
This route uses the default Claude profile. Keep `CLAUDE_CONFIG_DIR` unset for
|
||||
the entire local lifecycle: registration, backup, listing, version replacement
|
||||
and removal. In POSIX shells run `unset CLAUDE_CONFIG_DIR`; in PowerShell run
|
||||
`Remove-Item Env:CLAUDE_CONFIG_DIR -ErrorAction SilentlyContinue` before these
|
||||
commands. Record an existing selection first if you intend to restore it after
|
||||
finishing. The configuration path below is the default profile's
|
||||
`~/.claude.json`; a different active profile uses its own config root instead.
|
||||
|
||||
Register with the variable unset:
|
||||
|
||||
<!-- x-release-please-start-version -->
|
||||
|
||||
```bash
|
||||
claude mcp add --scope local rea -- npx -y rea-agents@6.4.0 mcp
|
||||
```
|
||||
|
||||
On native Windows, use the same project directory and wrap the package-runner
|
||||
shim so the stdio client can launch it:
|
||||
|
||||
```powershell
|
||||
claude mcp add --scope local rea -- cmd.exe /d /c npx -y rea-agents@6.4.0 mcp
|
||||
```
|
||||
|
||||
<!-- x-release-please-end -->
|
||||
|
||||
Local scope stores the entry under that directory's path in `~/.claude.json`,
|
||||
so other projects don't load it. This route doesn't install the bundled skill.
|
||||
Avoid `--scope project`, which writes `.mcp.json` into the repository where it
|
||||
can be committed or copied. `rea doctor` and `rea uninstall` read only the
|
||||
user-level entry, so they don't report or remove a local-scope registration;
|
||||
remove it with `claude mcp remove --scope local rea` from the same directory.
|
||||
`rea update` doesn't refresh it either, so it stays on the version it names.
|
||||
To move it to a newer release, first back up the project entry in
|
||||
`~/.claude.json` and record its environment and other metadata. In particular,
|
||||
preserve caller-selected overrides such as `REA_MCP_INPUT_SCHEMA_PROFILE` and
|
||||
`REA_MCP_MAX_RESPONSE_BYTES`. Run `claude mcp remove --scope local rea` from the
|
||||
same directory, then recreate it with the platform-appropriate command above
|
||||
and the new version. Reapply every recorded environment value with Claude
|
||||
Code's `mcp add --env NAME=value` options and restore any other retained entry
|
||||
metadata before restarting the project session. Removal and re-addition do not
|
||||
preserve these values automatically. Running setup for Claude Code instead
|
||||
would add a user-level entry for every project.
|
||||
|
||||
Check the separate profile with the prefixed `claude mcp list` above and compare
|
||||
with the default profile with `CLAUDE_CONFIG_DIR` unset. For local scope, run
|
||||
`claude mcp list` from the selected project and again from another project.
|
||||
|
||||
## Check readiness for your task
|
||||
|
||||
Run `rea doctor` when you need diagnosis. It reads host prerequisites and agent
|
||||
|
|
@ -136,6 +290,15 @@ configuration without changing them. Without options, it audits every detected
|
|||
registration, the installed skill and optional analysis engines; its overall
|
||||
`healthy` value can be false while your chosen workflow works.
|
||||
|
||||
Doctor's `providerInspections` are host-level engine and registration checks,
|
||||
not a readiness row for every advertised MCP adapter surface. `tools/list`
|
||||
remains the complete stable catalog. Call `binary_session` with `{}` before
|
||||
choosing an operation; it works before a target is opened, and
|
||||
`result.tool_availability` reports per-operation availability, reasons and
|
||||
client requirements for the current session state. It does not guarantee that
|
||||
a particular caller-supplied input will be valid. See
|
||||
[MCP discovery](mcp-contracts.md#identity-and-discovery).
|
||||
|
||||
Select the part you want to check:
|
||||
|
||||
| Task | Readiness check |
|
||||
|
|
@ -335,6 +498,20 @@ document for the same target. REA does not currently identify, focus, or reuse
|
|||
that existing GUI document, and the supported launcher exposes no attach or
|
||||
reuse action for it.
|
||||
|
||||
REA currently declines saved Hopper database (`.hop`) targets on macOS because
|
||||
the native `open database` launch route does not run REA's bridge script. To
|
||||
analyze that program with REA, select its original executable as the target and
|
||||
open it through Hopper; this starts the supported bridge route. This does not
|
||||
recover database-only annotations or analysis changes. Saved-database analysis
|
||||
on Linux continues to use its supported demo-launch route.
|
||||
|
||||
Hopper lease directories must be readable and owned by the current user.
|
||||
REA validates and secures an opened directory before reserving a lease. If
|
||||
directory access fails, the error identifies the path and recovery steps;
|
||||
REA leaves its permissions unchanged. Check ownership and parent-directory
|
||||
access, then restore owner-only read, write, and search permissions (`0700`)
|
||||
on a directory you own before retrying.
|
||||
|
||||
On supported Linux distributions, approved setup verifies Hopper's official
|
||||
`.deb`, `.rpm`, or Arch package before invoking the native package manager.
|
||||
REA runs the supported demo build on a private Xvfb display and selects Hopper's
|
||||
|
|
@ -347,6 +524,13 @@ host mount and the rest of `/tmp` remain unchanged; this fallback never invokes
|
|||
`sudo`. `rea doctor --provider hopper --json` reports the selected
|
||||
strategy and both host and effective mount facts.
|
||||
|
||||
When the package-manager step fails, setup reports the failing command and its
|
||||
captured output, so a package conflict is distinguishable from an authorization
|
||||
failure. On CachyOS, an unversioned compatibility package such as
|
||||
`zlib-ng-compat` may conflict with Hopper's versioned `zlib` dependency. Resolve
|
||||
the package conflict yourself or install Hopper's official package manually,
|
||||
then rerun `rea setup`.
|
||||
|
||||
The supported Linux demo forwards additional launches into its existing
|
||||
application, even across private displays. REA therefore reserves one Linux
|
||||
Hopper application per user across cooperating REA processes. A competing
|
||||
|
|
@ -371,8 +555,9 @@ sessions before removing that recovery directory and retrying.
|
|||
On macOS, REA uses
|
||||
`/Applications/Hopper Disassembler.app/Contents/MacOS/hopper` by default.
|
||||
On Linux, it prefers executable `/opt/hopper/bin/Hopper`, then executable
|
||||
`~/.local/share/rea/hopper/bin/Hopper`; if neither exists, the former remains
|
||||
the diagnostic fallback. `HOPPER_LAUNCHER_PATH` overrides those choices:
|
||||
`~/.local/share/rea/hopper/bin/Hopper`, followed by `/opt/hopper-v4/bin/hopper`
|
||||
and `/usr/bin/hopper`; if none exists, the former remains the diagnostic
|
||||
fallback. `HOPPER_LAUNCHER_PATH` overrides those choices:
|
||||
|
||||
```bash
|
||||
export HOPPER_LAUNCHER_PATH=/absolute/path/to/Hopper
|
||||
|
|
@ -512,7 +697,7 @@ reported as incompatible instead of being admitted from its directory name, and
|
|||
a header that cannot be read or recognized is reported as unknown. REA does not
|
||||
build or install native tools, or change Gatekeeper quarantine settings.
|
||||
|
||||
The adapter exposes 25 read-only operations: thirteen inventory/name/search
|
||||
The adapter exposes 26 read-only operations: fourteen inventory/name/search
|
||||
operations and twelve function-analysis operations. These cover metadata,
|
||||
decompilation, assembly, resolved calls, typed references, xrefs, function
|
||||
dossiers, instructions, recovered data types, measured load mappings, loaded
|
||||
|
|
@ -560,6 +745,14 @@ installation remains unavailable on Windows. Setup never installs Ghidra,
|
|||
Java, or Python. It preserves valid detected Ghidra/JDK settings in agent
|
||||
registrations. See [Windows Ghidra P0](windows-ghidra-p0.md) for provider diagnostics.
|
||||
|
||||
When `GHIDRA_INSTALL_DIR` is unset, doctor and the Ghidra provider also search
|
||||
`PATH` for `ghidra` or `ghidraRun`, resolve the selected launcher through its
|
||||
real path, and accept it only when that location belongs to a Ghidra tree with
|
||||
the application metadata and headless launcher. An explicitly supplied
|
||||
`GHIDRA_INSTALL_DIR` always takes precedence, including when it is invalid;
|
||||
REA reports that error instead of silently selecting another installation.
|
||||
This discovery is read-only and does not execute the Ghidra launcher.
|
||||
|
||||
Doctor validates the platform, architecture, Ghidra 12.1.x application version,
|
||||
`support/analyzeHeadless` or `support/analyzeHeadless.bat`, the installation's
|
||||
Java major range, 64-bit JDK bitness, and the presence of `javac`/`javac.exe`.
|
||||
|
|
@ -800,7 +993,7 @@ On a controlled Windows x64 runner, use
|
|||
`npm run verify:ghidra:windows`. The verifier generates a deterministic native
|
||||
PE fixture from source bytes and requires the Windows native authority before
|
||||
opening the provider. The controls are implemented on main; see the
|
||||
[release boundary](#released-package-and-main). The lane verifies all 25 admitted
|
||||
[release boundary](#released-package-and-main). The lane verifies all 26 admitted
|
||||
read-only operations, digest identity, and runtime cleanup. The separate
|
||||
`verify:ghidra:windows:package` lane checks the packed CLI/MCP boundary with an
|
||||
ordinary user. It requires the built Windows native bundle and an existing
|
||||
|
|
|
|||
|
|
@ -282,9 +282,14 @@ suffix. Utility calls such as `useUtils().invoice.list.fetch()`, `useQueries`,
|
|||
and links whose callee was renamed by a minifier are not recognized.
|
||||
|
||||
Each recovered bundle module retains the exact factory-source digest. A complete
|
||||
bounded AST also receives a `babel-ast-v1` structural fingerprint that ignores
|
||||
bounded AST also receives a `babel-ast-v2` structural fingerprint that ignores
|
||||
ordinary identifier names while retaining syntax, literals, operators, object
|
||||
keys, and member properties. If fingerprint construction reaches its structural
|
||||
keys, and member properties. Computed member identifiers retain their access
|
||||
syntax without treating a local variable's spelling as a fixed property; template
|
||||
literal contents remain part of the fingerprint. Version comparison keeps the
|
||||
algorithm in its matching key, so retained `babel-ast-v1` fingerprints cannot match
|
||||
these new fingerprints. Re-analyze both snapshots to compare their new structure.
|
||||
If fingerprint construction reaches its structural
|
||||
bound, the fingerprint is unavailable and its status is `truncated`; a bounded
|
||||
prefix is never presented as a complete stable fingerprint.
|
||||
|
||||
|
|
@ -619,3 +624,11 @@ Unresolved HTML references retain their declaration, source range, and resolutio
|
|||
reason in the renderer observations.
|
||||
HTML script source ranges follow the HTML parser across LF, CRLF, and bare CR
|
||||
line endings, preserving UTF-16 columns.
|
||||
Inline JavaScript in local HTML, meaning classic or module `<script>` bodies
|
||||
without a `src` attribute, is not analyzed. Each HTML file that has one gets an
|
||||
unknown `parse-html-inline-scripts` scope with the exact body count and the
|
||||
script type and source range of the first 100 bodies; beyond that, coverage is
|
||||
truncated under the `html_inline_script_locations_per_file` limit, and the
|
||||
scope's evidence counts the omitted locations. The graph names the
|
||||
file and up to 10 starting lines in its limitations, and coverage is partial. Import maps, speculation rules, data blocks, and blank
|
||||
bodies are not JavaScript and are not reported.
|
||||
|
|
|
|||
|
|
@ -193,7 +193,8 @@ after target content was read.
|
|||
## Retained application Evidence inputs
|
||||
|
||||
`analyze_javascript_application`, `inspect_binary_layout`, `inspect_artifact`,
|
||||
`inspect_wasm_artifact`, and `inspect_macho` accept `detail`:
|
||||
`inspect_wasm_artifact`, `inspect_macho`, and `decode_interface_builder`
|
||||
accept `detail`:
|
||||
`complete` (default) returns the complete analysis Evidence; `summary` records
|
||||
that complete Evidence in the session and returns the derived
|
||||
`inspect_analysis_view` summary Evidence instead. Its
|
||||
|
|
@ -276,6 +277,17 @@ symbols (Electron Framework: 2.5 MB of imports and exports, 3.3 KB as a
|
|||
summary), so select `detail: "summary"` on the first call. Delivery detail
|
||||
never reaches the native tools or changes the source Evidence ID.
|
||||
|
||||
Completed `decode_interface_builder` Evidence supports a summary and
|
||||
`view: {kind: "interface-builder", collection: "nodes", offset: 0, limit: 64}`.
|
||||
Collections are `documents`, `nodes`, `edges`, and `coverage`; for nodes and
|
||||
edges an optional `document` keeps rows whose evidence names that exact decoded
|
||||
`relative_path`, and an undecoded path is refused. The summary keeps every
|
||||
decoded document, the provider, node-kind, edge-relation and coverage-status
|
||||
counts, and whether the decoded graph is incomplete. Each page repeats
|
||||
`source_truncated`. A complete app graph often exceeds the default MCP response
|
||||
budget (XLD.app: 6.7 MB), so select `detail: "summary"` on the first call.
|
||||
Delivery detail never reaches the decoder or changes the source Evidence ID.
|
||||
|
||||
`trace_application_feature`, `trace_javascript_semantics`,
|
||||
`compare_application_versions`, `compare_source_to_bundle`, and
|
||||
`compare_javascript_export_shapes` accept complete inline application Evidence
|
||||
|
|
@ -671,7 +683,13 @@ advertised input schema within a 13,312-byte budget, measured to sit below
|
|||
Moonshot's observed cap (community measurements: a 13.7 KB schema is accepted
|
||||
while a ~15 KB one is rejected). The setting is read at startup; restart the
|
||||
server to apply changes, as with `REA_MCP_MAX_RESPONSE_BYTES`. The default
|
||||
value `full` keeps the complete advertisement unchanged.
|
||||
value `full` keeps the complete advertisement unchanged. Compact presentation
|
||||
also reduces schemas whose expanded JSON tree exceeds ten levels, the limit
|
||||
reported by strict-depth MCP clients in [#2813](https://github.com/morluto/rea/issues/2813).
|
||||
WorkBuddy/CodeBuddy gateways have also rejected full REA catalogs with an
|
||||
opaque `11133` error while accepting the compact profile
|
||||
([#2999](https://github.com/morluto/rea/issues/2999)); compatibility depends on
|
||||
the selected gateway and model.
|
||||
|
||||
The compact profile renders the same tool set, output schemas, annotations,
|
||||
and canonical server-side validation. It changes only the advertised input
|
||||
|
|
@ -696,6 +714,9 @@ JSON Schema:
|
|||
canonical schema still rejects malformed calls server-side, so callers
|
||||
following the reduced advertisement receive the canonical typed error
|
||||
instead of silent acceptance.
|
||||
- Schemas deeper than ten nodes use the same reduced projection, with
|
||||
root-property names, types, guidance, and required fields retained. The
|
||||
server continues to enforce the full canonical input contract.
|
||||
|
||||
Boundary tests pin every advertised compact schema to the budget, keep them
|
||||
valid JSON Schema that still accepts every canonical example. Client verification
|
||||
|
|
@ -711,6 +732,30 @@ identity. Explicit schema IDs and relative external reference bases are
|
|||
preserved. This keeps every tool and validation rule in discovery and does not
|
||||
change the SDK's catalog invalidation or availability checks.
|
||||
|
||||
Some stdio hosts reject a complete `tools/list` frame before they load any
|
||||
tools. The opt-in `REA_MCP_OUTPUT_SCHEMA_PROFILE=off` setting omits `outputSchema`
|
||||
from the advertised catalog for hosts with a small frame limit (the measured
|
||||
pi-web-ui failure and catalog breakdown are in [#2512](https://github.com/morluto/rea/issues/2512)).
|
||||
The default `full` profile is unchanged. With `off`, REA validates successful
|
||||
structured results against the same named canonical tool contract before
|
||||
delivery; it still returns complete `structuredContent` and text results.
|
||||
Only client-side schema discovery and validation are omitted, so hosts using
|
||||
this profile should rely on REA's validation and retain their usual response
|
||||
size limits.
|
||||
|
||||
### Function comparison limitations
|
||||
|
||||
`compare_functions` compares the `pseudocode` and `assembly` dimensions as
|
||||
exact provider text. A `changed` status establishes a textual difference, not a
|
||||
behavior change. Addresses and generated names are not normalized, so relocated
|
||||
labels can change these dimensions while `cfg` remains unchanged. Both text
|
||||
dimensions carry this limitation inline. Numeric constants remain part of the
|
||||
exact comparison. Reconstruction verification and static/runtime correlation
|
||||
retain limitations from the selected dimension; an overall function claim
|
||||
includes all dimension limitations. Equal CFGs establish only equal observed
|
||||
block sizes and successor relationships. Text from different provider
|
||||
identities remains unknown.
|
||||
|
||||
### Comparison input shapes
|
||||
|
||||
`compare_web_captures` accepts exactly one of two input shapes:
|
||||
|
|
|
|||
|
|
@ -227,6 +227,10 @@ nodes and edges. Node payloads are bounded to 8 MiB and serialized results to
|
|||
32 MiB. Pagination returns edges whose source is on the node page and preserves
|
||||
stable IDs for endpoints outside that page.
|
||||
|
||||
Within one value trace, each exact procedure selector is attempted at most once,
|
||||
including failed analyses. Failures remain unknown observations and do not
|
||||
establish canonical function identity. A new trace can retry those selectors.
|
||||
|
||||
The raw Java `is_dead` flag can remain set after Ghidra decodes a live block;
|
||||
`block_membership` reports actual syntax-tree membership separately.
|
||||
|
||||
|
|
@ -319,7 +323,16 @@ complete observation.
|
|||
digest are preserved; key interpretation follows the pinned plist decoder.
|
||||
- `decode_interface_builder` / `rea decode-interface-builder <app>` reads
|
||||
compiled storyboard and nib resources into objects, outlets, actions,
|
||||
connections and controller/class names. `verify:interface-builder` compiles
|
||||
connections and controller/class names. A full app graph is often several
|
||||
megabytes (XLD.app: 6.7 MB, about 2M tokens, which also exceeds the default
|
||||
MCP response budget), so MCP callers should pass `detail: "summary"`. The
|
||||
summary keeps every decoded document and node/edge/coverage counts and
|
||||
retains the complete Evidence; page rows with `inspect_analysis_view`
|
||||
`view: {kind: "interface-builder", collection: "nodes", document:
|
||||
"Contents/Resources/Base.lproj/MainMenu.nib", offset: 0, limit: 64}`.
|
||||
`document` selects rows whose evidence names that exact
|
||||
`documents[].relative_path` (nodes and edges only); `source_truncated` keeps
|
||||
graph incompleteness visible on every page. `verify:interface-builder` compiles
|
||||
a source-owned AppKit XIB with `ibtool`. Storyboards require an installed
|
||||
iOS platform; unsupported archive forms remain explicit.
|
||||
XML plist archives accept UTF-8 and BOM-marked UTF-16 in either byte order.
|
||||
|
|
|
|||
|
|
@ -380,6 +380,9 @@ Every capture requires `truncation_details`, with separate accounting for:
|
|||
When a directory exceeds that capacity, the retained subset follows filesystem
|
||||
enumeration order and is then sorted; it is explicitly incomplete and cannot
|
||||
establish that an omitted path is absent.
|
||||
An unreadable directory is recorded in `enumeration_omissions` with its
|
||||
root-aliased path and operating-system error code. Entries observed before
|
||||
that directory remain available, while descendants beneath it stay unknown.
|
||||
Each retained regular file without a digest has an aliased path, size,
|
||||
remaining budget and reason: `file_bytes_budget`,
|
||||
`file_changed_or_short_read`, or `file_unavailable`. Its `system_code` is
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ behavior, and complete CI evidence before merging.
|
|||
| `npm run test:boundary` | Boundary, CLI boundary, MCP boundary, process boundary, and process-global projects |
|
||||
| `npm run test:mcp` | MCP boundary project |
|
||||
| `npm run test:acceptance` | Complete compiled CLI and MCP acceptance workflows |
|
||||
| `npm run test:python` | LLDB producer retention and launch contracts with a fake LLDB API; no native debugger required |
|
||||
| `npm run test:python` | LLDB and mitmproxy producer contracts plus analysis-efficiency checks; no real debugger or proxy required |
|
||||
| `npm run test:watch` | Dirty source tests in watch mode, without build |
|
||||
| `npm run test:watch:all` | Changed tests from every project; builds at startup, so rebuild after production edits before relying on compiled tests |
|
||||
| `npm run check:changed` | Cached typecheck/lint and branch-related source feedback |
|
||||
|
|
@ -449,8 +449,9 @@ adds in-place reparse, breakaway, environment, and token observations.
|
|||
The separate `scripts/verify-windows-native-inspection.mjs` lane uses the
|
||||
`.cache/windows-native-inspection-fixture` artifact built on Linux with
|
||||
`node scripts/build-windows-native.mjs --inspection-fixture`. Its controlled
|
||||
first deletion failure checks retained NTFS ownership, later-query recovery,
|
||||
successful-admission caching, and environment-teardown cleanup. It neither
|
||||
first deletion failure checks retained NTFS ownership across a same-volume
|
||||
rename, later-query handle-based recovery, successful-admission caching, and
|
||||
environment-teardown cleanup. It neither
|
||||
establishes the cause of the intermittent #1701 host failure nor replaces
|
||||
verification with the shipped production artifact.
|
||||
`npm run verify:ghidra:windows:package` additionally packs and installs REA into
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ additional permission flag or degraded mode is required.
|
|||
- a 64-bit full JDK inside that installation's declared Java range (JDK 21 or newer, with no maximum, for current 12.1 releases);
|
||||
- an explicit native, non-managed x86 or x86-64 PE application/DLL, or a
|
||||
[raw NMOS 6502 image](raw-binary.md#nmos-6502) with a caller-supplied entry; and
|
||||
- the 25 read-only Ghidra inventory, memory, and function-analysis operations.
|
||||
- the 26 read-only Ghidra inventory, memory, and function-analysis operations.
|
||||
|
||||
Loaded memory reads and file offsets preserve Ghidra's observed source mappings.
|
||||
PE load-image inspection returns an explicit unsupported attestation status.
|
||||
|
|
@ -78,6 +78,19 @@ OS failures. It checks the Ghidra release, headless launcher, full JDK, x64
|
|||
host, and native controls. A missing or mismatched addon requires the matching
|
||||
Windows x64 package; changing MCP registration cannot supply that artifact.
|
||||
A filesystem or access failure identifies the failed native constraint.
|
||||
An incompatible host uses `unsupported_host`. Missing, mismatched or unloadable
|
||||
native artifacts use `runtime_missing` and doctor classification
|
||||
`missing_dependency`. A failed native invocation or invalid inspection uses
|
||||
`runtime_unavailable` and doctor classification `runtime_failure`; it does not
|
||||
mean Windows x64 itself is unsupported. Provider diagnostics retain the failure
|
||||
stage under `windows_native_failure`, together with the corresponding remediation.
|
||||
Self-test failures preserve the native operation and OS error text. Check the
|
||||
process's writable local NTFS temporary directory and the reported filesystem,
|
||||
DACL, Job Object or cleanup constraint, then rerun doctor after resolving it.
|
||||
Ghidra or Java reinstallation cannot resolve a native self-test failure. The
|
||||
provider remains unavailable until all native controls are proven; a later query
|
||||
may retry a failed native invocation, while artifact or malformed-inspection
|
||||
failures remain stable for the process lifetime.
|
||||
Other doctor checks, including unconfigured agent integrations, can remain
|
||||
unready while Ghidra itself is available.
|
||||
|
||||
|
|
|
|||
|
|
@ -16,6 +16,7 @@ prerelease_identifier="(${numeric_identifier}|[0-9A-Za-z-]*[A-Za-z-][0-9A-Za-z-]
|
|||
build_identifier='[0-9A-Za-z-]+'
|
||||
version_pattern="${numeric_identifier}\\.${numeric_identifier}\\.${numeric_identifier}(-${prerelease_identifier}(\\.${prerelease_identifier})*)?(\\+${build_identifier}(\\.${build_identifier})*)?"
|
||||
version="${REA_VERSION:-}"
|
||||
version_requested=false
|
||||
dry_run=false
|
||||
start_setup=true
|
||||
verbose=false
|
||||
|
|
@ -81,6 +82,7 @@ main() {
|
|||
--version)
|
||||
(($# >= 2)) || fail "--version requires a value."
|
||||
version="$2"
|
||||
version_requested=true
|
||||
shift 2
|
||||
;;
|
||||
--dry-run)
|
||||
|
|
@ -121,7 +123,7 @@ main() {
|
|||
fail_node_version "$node_version"
|
||||
fi
|
||||
|
||||
if [ -n "$version" ]; then
|
||||
if [ "$version_requested" = true ] || [ -n "$version" ]; then
|
||||
matches "$version" "^${version_pattern}$" || fail "version must be an exact semantic version."
|
||||
else
|
||||
release_json="$(curl -fsSL --proto '=https' --tlsv1.2 --retry 3 -H 'Accept: application/vnd.github+json' "https://api.github.com/repos/$REPOSITORY/releases/latest")" || fail "the latest REA release could not be resolved. Check network access or pass --version VERSION, then retry."
|
||||
|
|
@ -164,6 +166,7 @@ main() {
|
|||
npm "${install_args[@]}" || fail "npm could not install REA. Check registry access and npm permissions, then retry."
|
||||
[ -x "$install_bin" ] || fail "npm completed without installing the rea command. Check the npm global bin directory and PATH, then retry."
|
||||
installed_version="$("$install_bin" --version 2>/dev/null | tr -d '[:space:]')" || fail "the installed REA version could not be read. Reinstall the requested version, then retry."
|
||||
[ -n "$installed_version" ] || fail "the installed REA version could not be read. Reinstall the requested version, then retry."
|
||||
[ "$installed_version" = "$version" ] || fail "installed version $installed_version does not match $version. Reinstall the requested version, then retry."
|
||||
|
||||
printf 'REA %s is installed.\n' "$version"
|
||||
|
|
|
|||
|
|
@ -9,8 +9,8 @@
|
|||
"src/application/RealHopperAssertions.ts",
|
||||
"scripts/verify/fridaBindingsCompatibility.ts",
|
||||
"scripts/verify/termux/browser.mjs",
|
||||
"scripts/verify-real-cutter.mjs",
|
||||
"scripts/verify-real-ghidra-noreturn.mjs",
|
||||
"scripts/run-exact-tests.mjs",
|
||||
"tests/processOwnershipRunner.ts"
|
||||
],
|
||||
"project": ["src/**/*.ts", "tests/**/*.ts", "scripts/**/*.mjs"],
|
||||
|
|
|
|||
|
|
@ -214,18 +214,32 @@ std::unique_ptr<Process> spawnProcess(const std::wstring& command, const std::ws
|
|||
}
|
||||
|
||||
static napi_value readOutput(napi_env env, Handle& read, bool& ended) {
|
||||
std::array<unsigned char, 65536> bytes;
|
||||
DWORD available = 0, count = 0;
|
||||
constexpr size_t kMaximumReadPerPoll = 1024 * 1024;
|
||||
std::array<unsigned char, 65536> chunk;
|
||||
std::vector<unsigned char> bytes;
|
||||
if (!read.valid()) ended = true;
|
||||
else if (!PeekNamedPipe(read.get(), nullptr, 0, nullptr, &available, nullptr)) {
|
||||
require(GetLastError() == ERROR_BROKEN_PIPE, "Observe owned process output failed");
|
||||
read.reset(); ended = true;
|
||||
} else if (available > 0) {
|
||||
require(ReadFile(read.get(), bytes.data(), std::min<DWORD>(available, bytes.size()), &count, nullptr),
|
||||
"Read owned process output failed");
|
||||
else {
|
||||
while (bytes.size() < kMaximumReadPerPoll) {
|
||||
DWORD available = 0;
|
||||
if (!PeekNamedPipe(read.get(), nullptr, 0, nullptr, &available, nullptr)) {
|
||||
require(GetLastError() == ERROR_BROKEN_PIPE,
|
||||
"Observe owned process output failed");
|
||||
read.reset();
|
||||
ended = true;
|
||||
break;
|
||||
}
|
||||
if (available == 0) break;
|
||||
const auto requested = static_cast<DWORD>(std::min<size_t>(
|
||||
{available, chunk.size(), kMaximumReadPerPoll - bytes.size()}));
|
||||
DWORD count = 0;
|
||||
require(ReadFile(read.get(), chunk.data(), requested, &count, nullptr) &&
|
||||
count > 0,
|
||||
"Read owned process output failed");
|
||||
bytes.insert(bytes.end(), chunk.begin(), chunk.begin() + count);
|
||||
}
|
||||
}
|
||||
napi_value result;
|
||||
check(napi_create_buffer_copy(env, count, bytes.data(), nullptr, &result));
|
||||
check(napi_create_buffer_copy(env, bytes.size(), bytes.data(), nullptr, &result));
|
||||
return result;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -138,7 +138,7 @@
|
|||
"test": "npm run test:with-build",
|
||||
"test:with-build": "npm run test:prepare && vitest run",
|
||||
"test:run": "vitest run",
|
||||
"test:python": "python3 -m unittest bridge/native/test_rea_lldb_tracer.py -v && python3 -m unittest discover -s experiments/analysis-efficiency -p 'test_*.py' -v",
|
||||
"test:python": "python3 -m unittest bridge/native/test_rea_lldb_tracer.py bridge/mitmproxy/test_capture.py -v && python3 -m unittest discover -s experiments/analysis-efficiency -p 'test_*.py' -v",
|
||||
"test:local": "node scripts/run-development-tests.mjs local",
|
||||
"test:fast": "node scripts/ci/test-group.mjs source",
|
||||
"test:built": "node scripts/ci/test-group.mjs built",
|
||||
|
|
|
|||
|
|
@ -202,6 +202,9 @@ if (tier === "reused") {
|
|||
if (GUARDED_WORKFLOWS.has(file)) selected.code = true;
|
||||
if (file !== ".github/workflows/ci.yml") continue;
|
||||
}
|
||||
// The Notes example is also the source-owned agent evaluation fixture.
|
||||
if (file.startsWith("website/public/examples/notes-electron/"))
|
||||
selected.code = true;
|
||||
if (owners.length === 1 && owners[0] === "website") continue;
|
||||
if (/^src\/(contracts|domain)\//.test(file)) selected.docs = true;
|
||||
if (file === "package.json") {
|
||||
|
|
|
|||
|
|
@ -79,6 +79,7 @@ export const SCOPES = {
|
|||
"tests/boundary/cli/nativeSignatureCli.test.ts",
|
||||
"tests/boundary/mcp/nativeSignatureTargetBindingMcp.test.ts",
|
||||
"src/process/capture/FilesystemSnapshot.test.ts",
|
||||
"tests/process-global/filesystemSnapshotFallbackCancellation.test.ts",
|
||||
"tests/boundary/process/darwinProcessRunTokenReader.test.ts",
|
||||
"tests/boundary/process/directAnalysisSignals.test.ts",
|
||||
"tests/boundary/process/processCaptureLifecycle.test.ts",
|
||||
|
|
@ -89,6 +90,9 @@ export const SCOPES = {
|
|||
package: [
|
||||
"src/windows/**",
|
||||
"native/windows/**",
|
||||
"src/process/capture/FilesystemSnapshot.ts",
|
||||
"src/process/capture/FilesystemSnapshot.test.ts",
|
||||
"tests/process-global/filesystemSnapshotFallbackCancellation.test.ts",
|
||||
"src/inspector/**",
|
||||
"scripts/verify/inspector/**",
|
||||
"scripts/verify-package.mjs",
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { dirname, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { writeWindowsGhidraByteSearchFixture } from "./lib/ghidra-windows-byte-search.mjs";
|
||||
|
||||
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
||||
// Source-owned static fixtures: no compiler, target loading or execution.
|
||||
|
|
@ -61,7 +62,7 @@ for (const architecture of ["x86_64", "x86"]) {
|
|||
}
|
||||
|
||||
bytes.write(".text", sectionTable, "ascii");
|
||||
bytes.writeUInt32LE(dll ? 0x180 : 0x50, sectionTable + 8);
|
||||
bytes.writeUInt32LE(0x200, sectionTable + 8);
|
||||
bytes.writeUInt32LE(0x1000, sectionTable + 12);
|
||||
bytes.writeUInt32LE(0x200, sectionTable + 16);
|
||||
bytes.writeUInt32LE(0x200, sectionTable + 20);
|
||||
|
|
@ -136,6 +137,8 @@ for (const architecture of ["x86_64", "x86"]) {
|
|||
bytes.write("fixture_return42\0", 0x340, "binary");
|
||||
}
|
||||
|
||||
writeWindowsGhidraByteSearchFixture(bytes);
|
||||
|
||||
await mkdir(dirname(output), { recursive: true });
|
||||
await writeFile(output, bytes);
|
||||
if (!x86 && !dll) process.stdout.write(`${output}\n`);
|
||||
|
|
|
|||
|
|
@ -3,6 +3,8 @@ export interface ManagedPeFixtureMethod {
|
|||
readonly flags?: number;
|
||||
readonly implFlags?: number;
|
||||
readonly name: string;
|
||||
/** An explicit metadata RVA; zero represents a method without an IL body. */
|
||||
readonly rva?: number;
|
||||
}
|
||||
|
||||
export interface ManagedPeFixtureOptions {
|
||||
|
|
@ -19,6 +21,7 @@ export interface ManagedPeFixtureOptions {
|
|||
readonly pinvoke?: {
|
||||
readonly importName?: string;
|
||||
readonly mappingFlags?: number;
|
||||
readonly memberRow?: number;
|
||||
readonly moduleName?: string;
|
||||
};
|
||||
readonly readyToRun?: boolean;
|
||||
|
|
|
|||
|
|
@ -280,7 +280,7 @@ const buildManagedMetadata = (options, methods) => {
|
|||
methodDefRow({
|
||||
name: stringsForMethods[index],
|
||||
signature: methodSignature,
|
||||
rva: method.rva ?? 0x2800 + index * 0x20,
|
||||
rva: method.rva,
|
||||
flags: method.flags ?? 0x0016,
|
||||
implFlags: method.implFlags ?? 0,
|
||||
}),
|
||||
|
|
@ -325,13 +325,64 @@ const buildManagedMetadata = (options, methods) => {
|
|||
return metadata;
|
||||
};
|
||||
|
||||
const align = (value, boundary) => Math.ceil(value / boundary) * boundary;
|
||||
|
||||
const managedFixtureLayout = (options, methods, metadataLength) => {
|
||||
const resourceLength =
|
||||
4 + (options.resourceData ?? Buffer.from("resource-data")).length;
|
||||
const metadataOffset = 0x0300;
|
||||
const resourceOffset = Math.max(
|
||||
0x0800,
|
||||
align(metadataOffset + metadataLength, 0x200),
|
||||
);
|
||||
const readyToRunOffset = Math.max(
|
||||
0x0900,
|
||||
align(resourceOffset + resourceLength, 0x200),
|
||||
);
|
||||
const methodOffsets = [];
|
||||
let nextMethodOffset = Math.max(
|
||||
0x0a00,
|
||||
align(readyToRunOffset + (options.readyToRun === true ? 4 : 0), 0x200),
|
||||
);
|
||||
for (const method of methods) {
|
||||
// RVA zero denotes a method without a mapped IL body, such as P/Invoke.
|
||||
if (method.rva === 0) {
|
||||
methodOffsets.push(null);
|
||||
continue;
|
||||
}
|
||||
const offset =
|
||||
method.rva === undefined
|
||||
? nextMethodOffset
|
||||
: 0x0200 + (method.rva - 0x2000);
|
||||
methodOffsets.push(offset);
|
||||
nextMethodOffset =
|
||||
Math.max(offset, nextMethodOffset) +
|
||||
align(Math.max(0x20, (method.body ?? retOnlyIlBody).length), 0x20);
|
||||
}
|
||||
return { metadataOffset, resourceOffset, readyToRunOffset, methodOffsets };
|
||||
};
|
||||
|
||||
const buildManagedImage = (options, methods, metadata) => {
|
||||
const resourceData = options.resourceData ?? Buffer.from("resource-data");
|
||||
const resourceDirectory = Buffer.concat([
|
||||
u32(resourceData.length),
|
||||
resourceData,
|
||||
]);
|
||||
const image = Buffer.alloc(0x1000);
|
||||
const { metadataOffset, resourceOffset, readyToRunOffset, methodOffsets } =
|
||||
managedFixtureLayout(options, methods, metadata.length);
|
||||
const contentEnd = Math.max(
|
||||
metadataOffset + metadata.length,
|
||||
resourceOffset + resourceDirectory.length,
|
||||
readyToRunOffset + (options.readyToRun === true ? 4 : 0),
|
||||
...methodOffsets.flatMap((offset, index) =>
|
||||
offset === null
|
||||
? []
|
||||
: [offset + (methods[index].body ?? retOnlyIlBody).length],
|
||||
),
|
||||
0x1000,
|
||||
);
|
||||
const imageSize = align(contentEnd, 0x200);
|
||||
const image = Buffer.alloc(imageSize);
|
||||
image.write("MZ", 0, "ascii");
|
||||
image.writeUInt32LE(0x80, 0x3c);
|
||||
image.writeUInt32LE(0x0000_4550, 0x80);
|
||||
|
|
@ -351,16 +402,16 @@ const buildManagedImage = (options, methods, metadata) => {
|
|||
else image.writeUInt32LE(0x0040_0000, optional + 28);
|
||||
image.writeUInt32LE(0x1000, optional + 32);
|
||||
image.writeUInt32LE(0x200, optional + 36);
|
||||
image.writeUInt32LE(0x3000, optional + 56);
|
||||
image.writeUInt32LE(align(0x2000 + imageSize - 0x200, 0x1000), optional + 56);
|
||||
image.writeUInt32LE(0x200, optional + 60);
|
||||
image.writeUInt32LE(16, optional + directoryOffset - 4);
|
||||
image.writeUInt32LE(0x2000, optional + directoryOffset + 14 * 8);
|
||||
image.writeUInt32LE(72, optional + directoryOffset + 14 * 8 + 4);
|
||||
const section = optional + optionalSize;
|
||||
image.write(".text\0\0\0", section, "ascii");
|
||||
image.writeUInt32LE(0x1000, section + 8);
|
||||
image.writeUInt32LE(imageSize - 0x200, section + 8);
|
||||
image.writeUInt32LE(0x2000, section + 12);
|
||||
image.writeUInt32LE(0x0e00, section + 16);
|
||||
image.writeUInt32LE(imageSize - 0x200, section + 16);
|
||||
image.writeUInt32LE(0x0200, section + 20);
|
||||
image.writeUInt32LE(0x6000_0020, section + 36);
|
||||
const cli = 0x0200;
|
||||
|
|
@ -371,27 +422,41 @@ const buildManagedImage = (options, methods, metadata) => {
|
|||
image.writeUInt32LE(metadata.length, cli + 12);
|
||||
image.writeUInt32LE(options.cliFlags ?? 1, cli + 16);
|
||||
image.writeUInt32LE(0x0600_0001, cli + 20);
|
||||
image.writeUInt32LE(0x2600, cli + 24);
|
||||
image.writeUInt32LE(0x2000 + (resourceOffset - 0x0200), cli + 24);
|
||||
image.writeUInt32LE(resourceDirectory.length, cli + 28);
|
||||
if (options.readyToRun === true) {
|
||||
image.writeUInt32LE(0x2700, cli + 64);
|
||||
image.writeUInt32LE(0x2000 + (readyToRunOffset - 0x0200), cli + 64);
|
||||
image.writeUInt32LE(4, cli + 68);
|
||||
image.write("RTR\0", 0x0900, "ascii");
|
||||
image.write("RTR\0", readyToRunOffset, "ascii");
|
||||
}
|
||||
methods.forEach((method, index) => {
|
||||
const offset = 0x0a00 + index * 0x20;
|
||||
const offset = methodOffsets[index];
|
||||
if (offset === null) return;
|
||||
(method.body ?? retOnlyIlBody).copy(image, offset);
|
||||
});
|
||||
metadata.copy(image, 0x0300);
|
||||
resourceDirectory.copy(image, 0x0800);
|
||||
metadata.copy(image, metadataOffset);
|
||||
resourceDirectory.copy(image, resourceOffset);
|
||||
return image;
|
||||
};
|
||||
|
||||
/** Build a source-owned PE/CLI fixture without committing compiled binaries. */
|
||||
export const buildManagedPeFixture = (options = {}) => {
|
||||
const methods = managedFixtureMethods(options);
|
||||
const metadata = buildManagedMetadata(options, methods);
|
||||
return buildManagedImage(options, methods, metadata);
|
||||
const provisionalMetadata = buildManagedMetadata(
|
||||
options,
|
||||
methods.map((method) => ({ ...method, rva: method.rva ?? 0x2800 })),
|
||||
);
|
||||
const layout = managedFixtureLayout(
|
||||
options,
|
||||
methods,
|
||||
provisionalMetadata.length,
|
||||
);
|
||||
const methodsWithRvas = methods.map((method, index) => ({
|
||||
...method,
|
||||
rva: method.rva ?? 0x2000 + (layout.methodOffsets[index] - 0x0200),
|
||||
}));
|
||||
const metadata = buildManagedMetadata(options, methodsWithRvas);
|
||||
return buildManagedImage(options, methodsWithRvas, metadata);
|
||||
};
|
||||
export const buildNativePeFixture = () => {
|
||||
const image = buildManagedPeFixture();
|
||||
|
|
|
|||
|
|
@ -54,10 +54,12 @@ export async function verifyBrowserCaptureMetadataBudget(endpoint) {
|
|||
if (address === null || typeof address === "string")
|
||||
throw new Error("Browser metadata fixture did not bind a TCP port");
|
||||
fixtureOrigin = `http://127.0.0.1:${String(address.port)}`;
|
||||
// Each independent capture opens a new Debugger session. Keep each script
|
||||
// reachable so V8 cannot collect it before the later sessions attach.
|
||||
const annotations = Array.from(
|
||||
{ length: SCRIPT_COUNT },
|
||||
(_, index) =>
|
||||
`<script>globalThis["${marker}${index}"]=true;\n//# sourceMappingURL=${fixtureOrigin}/maps/${String(index)}-${"x".repeat(ANNOTATION_BYTES)}.map\n</script>`,
|
||||
`<script>globalThis["${marker}${index}"]=()=>true;\n//# sourceMappingURL=${fixtureOrigin}/maps/${String(index)}-${"x".repeat(ANNOTATION_BYTES)}.map\n</script>`,
|
||||
).join("");
|
||||
page = `<!doctype html><meta charset="utf-8">${annotations}<script>fetch("/capture-metadata-ready")</script>`;
|
||||
const fixtureUrl = `${fixtureOrigin}/capture-metadata-budget`;
|
||||
|
|
|
|||
|
|
@ -0,0 +1,260 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { once } from "node:events";
|
||||
import { createServer, request } from "node:http";
|
||||
import { connect } from "node:net";
|
||||
|
||||
import { createBrowserScenarioProvider } from "../../dist/composition/browserScenario.js";
|
||||
import { browserScenarioSchema } from "../../dist/domain/browserScenario.js";
|
||||
import { PlaywrightScenarioServiceWorkers } from "../../dist/browser/PlaywrightScenarioServiceWorkers.js";
|
||||
|
||||
const beforeDeadline = async (promise, label, timeoutMs = 2_000) => {
|
||||
let timer;
|
||||
try {
|
||||
return await Promise.race([
|
||||
promise,
|
||||
new Promise((_, reject) => {
|
||||
timer = setTimeout(
|
||||
() => reject(new Error(`${label} did not settle`)),
|
||||
timeoutMs,
|
||||
);
|
||||
}),
|
||||
]);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
};
|
||||
|
||||
// Real loopback proxy: Playwright's first transport is forwarded to Chrome,
|
||||
// then only the policy owner's discovery/handshake is deliberately withheld.
|
||||
const startProxy = async (endpoint, mode, ownerOnly = false) => {
|
||||
const upstream = new URL(endpoint);
|
||||
const sockets = new Set();
|
||||
let versions = 0;
|
||||
let upgrades = 0;
|
||||
let enabled = true;
|
||||
let entered;
|
||||
const stalled = new Promise((resolve) => {
|
||||
entered = resolve;
|
||||
});
|
||||
let closed;
|
||||
let stalledClose;
|
||||
const stall = (socket) => {
|
||||
socket.resume();
|
||||
socket.once("end", () => socket.end());
|
||||
stalledClose = new Promise((resolve) => {
|
||||
closed = resolve;
|
||||
});
|
||||
socket.once("close", () => closed());
|
||||
entered();
|
||||
};
|
||||
const server = createServer((incoming, response) => {
|
||||
const versionRequest = incoming.url.replace(/\/$/, "") === "/json/version";
|
||||
if (versionRequest) versions += 1;
|
||||
if (
|
||||
enabled &&
|
||||
mode === "http-discovery" &&
|
||||
versions === 2 &&
|
||||
versionRequest
|
||||
) {
|
||||
stall(incoming.socket);
|
||||
return;
|
||||
}
|
||||
const outgoing = request(new URL(incoming.url, upstream), (received) => {
|
||||
const chunks = [];
|
||||
received.on("data", (chunk) => chunks.push(chunk));
|
||||
received.on("end", () => {
|
||||
const content = Buffer.concat(chunks).toString("utf8");
|
||||
response.writeHead(received.statusCode, {
|
||||
"content-type": "application/json",
|
||||
});
|
||||
response.end(
|
||||
content.replaceAll(
|
||||
`127.0.0.1:${upstream.port}`,
|
||||
`127.0.0.1:${server.address().port}`,
|
||||
),
|
||||
);
|
||||
});
|
||||
});
|
||||
outgoing.on("error", () => response.destroy());
|
||||
response.on("close", () => outgoing.destroy());
|
||||
outgoing.end();
|
||||
});
|
||||
server.on("connection", (socket) => {
|
||||
sockets.add(socket);
|
||||
socket.on("close", () => sockets.delete(socket));
|
||||
});
|
||||
server.on("upgrade", (incoming, socket, head) => {
|
||||
upgrades += 1;
|
||||
if (
|
||||
enabled &&
|
||||
mode !== "http-discovery" &&
|
||||
upgrades === (ownerOnly ? 1 : 2)
|
||||
) {
|
||||
stall(socket);
|
||||
return;
|
||||
}
|
||||
const outgoing = connect(Number(upstream.port), "127.0.0.1", () => {
|
||||
const headers = incoming.rawHeaders.flatMap((value, index, all) =>
|
||||
index % 2 === 0 ? [`${value}: ${all[index + 1]}`] : [],
|
||||
);
|
||||
outgoing.write(
|
||||
`${incoming.method} ${incoming.url} HTTP/${incoming.httpVersion}\r\n${headers.join("\r\n")}\r\n\r\n`,
|
||||
);
|
||||
if (head.length > 0) outgoing.write(head);
|
||||
outgoing.pipe(socket);
|
||||
socket.pipe(outgoing);
|
||||
});
|
||||
outgoing.on("error", () => socket.destroy());
|
||||
socket.on("error", () => outgoing.destroy());
|
||||
socket.on("close", () => outgoing.destroy());
|
||||
});
|
||||
server.listen(0, "127.0.0.1");
|
||||
await once(server, "listening");
|
||||
const origin = `http://127.0.0.1:${server.address().port}`;
|
||||
let proxyEndpoint = origin;
|
||||
if (mode === "browser-websocket") {
|
||||
const version = await (await fetch(`${endpoint}/json/version`)).json();
|
||||
proxyEndpoint = version.webSocketDebuggerUrl.replace(
|
||||
`127.0.0.1:${upstream.port}`,
|
||||
`127.0.0.1:${server.address().port}`,
|
||||
);
|
||||
}
|
||||
return {
|
||||
endpoint: proxyEndpoint,
|
||||
stalled,
|
||||
waitClosed: () => stalledClose,
|
||||
resume: () => {
|
||||
enabled = false;
|
||||
},
|
||||
close: async () => {
|
||||
const done = once(server, "close");
|
||||
server.close();
|
||||
for (const socket of sockets) socket.destroy();
|
||||
await done;
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
export const verifyScenarioPolicySetup = async (
|
||||
endpoint,
|
||||
targetId,
|
||||
origin,
|
||||
page,
|
||||
) => {
|
||||
const result = {};
|
||||
for (const mode of ["http-websocket", "http-discovery", "deadline"]) {
|
||||
const proxy = await startProxy(endpoint, mode);
|
||||
const provider = createBrowserScenarioProvider(process.env);
|
||||
const scenario = browserScenarioSchema.parse({
|
||||
browser: {
|
||||
mode: "connect",
|
||||
cdp_endpoint: proxy.endpoint,
|
||||
target_id: targetId,
|
||||
},
|
||||
start_url: { url: origin },
|
||||
environment: { service_workers: "block" },
|
||||
actions: [
|
||||
{ step_id: "settle", action: "wait_for_timeout", duration_ms: 1 },
|
||||
],
|
||||
capture: { events: [], after_each_step: ["dom"], at_end: ["dom"] },
|
||||
});
|
||||
const cancellation = new AbortController();
|
||||
let failure;
|
||||
try {
|
||||
const pending = provider.captureScenario(scenario, {
|
||||
signal: cancellation.signal,
|
||||
});
|
||||
await beforeDeadline(proxy.stalled, `${mode} setup`);
|
||||
if (mode !== "deadline") cancellation.abort();
|
||||
const cancelled = await beforeDeadline(
|
||||
pending,
|
||||
`${mode} termination`,
|
||||
mode === "deadline" ? 12_000 : 2_000,
|
||||
);
|
||||
assert.equal(cancelled.ok, false);
|
||||
assert.equal(cancelled.error.cleanupIncomplete, false);
|
||||
if (mode === "deadline") assert.equal(cancelled.error.reason, "timeout");
|
||||
await beforeDeadline(proxy.waitClosed(), `${mode} socket release`);
|
||||
assert.equal(
|
||||
await page.evaluate(() =>
|
||||
navigator.serviceWorker.register.toString().includes("[native code]"),
|
||||
),
|
||||
true,
|
||||
);
|
||||
proxy.resume();
|
||||
const next = await beforeDeadline(
|
||||
provider.captureScenario(scenario),
|
||||
`${mode} target reacquisition`,
|
||||
);
|
||||
if (!next.ok) throw next.error;
|
||||
assert.equal(next.value.browser.cleanup, "disconnected-external");
|
||||
result[mode] = {
|
||||
...(mode === "deadline" ? { timed_out: true } : { cancelled: true }),
|
||||
socket_closed: true,
|
||||
target_reused: true,
|
||||
};
|
||||
} catch (cause) {
|
||||
failure = cause;
|
||||
throw cause;
|
||||
} finally {
|
||||
cancellation.abort();
|
||||
try {
|
||||
await proxy.close();
|
||||
} finally {
|
||||
try {
|
||||
await provider.close();
|
||||
} catch (cleanupFailure) {
|
||||
if (failure !== undefined)
|
||||
throw new AggregateError([failure, cleanupFailure]);
|
||||
throw cleanupFailure;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
const proxy = await startProxy(endpoint, "browser-websocket", true);
|
||||
const policy = new PlaywrightScenarioServiceWorkers(
|
||||
page,
|
||||
proxy.endpoint,
|
||||
targetId,
|
||||
);
|
||||
try {
|
||||
const rejected = assert.rejects(
|
||||
policy.initialize(),
|
||||
(error) => error._tag === "AnalysisCancelledError",
|
||||
);
|
||||
await beforeDeadline(proxy.stalled, "browser websocket approval handshake");
|
||||
await beforeDeadline(policy.close(), "browser websocket cleanup");
|
||||
await rejected;
|
||||
await beforeDeadline(
|
||||
proxy.waitClosed(),
|
||||
"browser websocket socket release",
|
||||
);
|
||||
proxy.resume();
|
||||
const next = new PlaywrightScenarioServiceWorkers(
|
||||
page,
|
||||
proxy.endpoint,
|
||||
targetId,
|
||||
);
|
||||
try {
|
||||
await beforeDeadline(
|
||||
next.initialize(),
|
||||
"browser websocket reacquisition",
|
||||
);
|
||||
} finally {
|
||||
await next.close();
|
||||
}
|
||||
result["browser-websocket"] = {
|
||||
cancelled: true,
|
||||
socket_closed: true,
|
||||
target_reused: true,
|
||||
direct_owner: true,
|
||||
};
|
||||
} finally {
|
||||
try {
|
||||
await policy.close();
|
||||
} finally {
|
||||
await proxy.close();
|
||||
}
|
||||
}
|
||||
return result;
|
||||
};
|
||||
|
|
@ -0,0 +1,605 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { once } from "node:events";
|
||||
import { createServer } from "node:http";
|
||||
import { setTimeout as delay } from "node:timers/promises";
|
||||
|
||||
import { chromium } from "playwright-core";
|
||||
|
||||
import { PlaywrightScenarioSession } from "../../dist/browser/PlaywrightScenarioSession.js";
|
||||
import { PlaywrightScenarioServiceWorkers } from "../../dist/browser/PlaywrightScenarioServiceWorkers.js";
|
||||
import { createBrowserScenarioProvider } from "../../dist/composition/browserScenario.js";
|
||||
import { browserScenarioSchema } from "../../dist/domain/browserScenario.js";
|
||||
import { runScenarioCli } from "./browser-scenario-verifier.mjs";
|
||||
import { verifyScenarioPolicySetup } from "./browser-scenario-service-worker-setup-e2e.mjs";
|
||||
|
||||
const startSite = async () => {
|
||||
let otherOrigin;
|
||||
const server = createServer((request, response) => {
|
||||
const path = new URL(request.url, "http://localhost").pathname;
|
||||
if (path === "/sw.js") {
|
||||
response.setHeader("content-type", "application/javascript");
|
||||
response.end(`self.addEventListener("install", () => self.skipWaiting());
|
||||
self.addEventListener("activate", event => event.waitUntil(clients.claim()));
|
||||
self.addEventListener("fetch", event => {
|
||||
if (new URL(event.request.url).pathname === "/probe")
|
||||
event.respondWith(new Response("worker"));
|
||||
});`);
|
||||
} else if (path === "/probe") {
|
||||
response.end("network");
|
||||
} else if (path === "/failed-navigation") {
|
||||
response.destroy();
|
||||
} else if (path === "/stalled-image.png") {
|
||||
response.writeHead(200, { "content-type": "image/png" });
|
||||
response.flushHeaders();
|
||||
} else if (path === "/pending-image" || path === "/pending-child") {
|
||||
response.setHeader("content-type", "text/html");
|
||||
response.end(
|
||||
'<!doctype html><img id="pending-image" src="/stalled-image.png">',
|
||||
);
|
||||
} else if (path === "/pending-frame") {
|
||||
response.setHeader("content-type", "text/html");
|
||||
response.end(
|
||||
'<!doctype html><iframe id="pending-frame" src="/pending-child"></iframe>',
|
||||
);
|
||||
} else {
|
||||
response.setHeader("content-type", "text/html");
|
||||
response.end(`<!doctype html><p id="state">ready</p>
|
||||
<a id="navigate" href="${otherOrigin}/">Navigate</a>
|
||||
${path === "/frame" ? `<iframe src="${otherOrigin}/"></iframe>` : ""}
|
||||
${
|
||||
path === "/register"
|
||||
? `<script>
|
||||
navigator.serviceWorker.register('/sw.js').then(
|
||||
() => document.querySelector('#state').textContent = 'registered',
|
||||
() => {
|
||||
document.querySelector('#state').textContent = 'blocked';
|
||||
document.querySelector('#state').dataset.blocked = 'true';
|
||||
});
|
||||
</script>`
|
||||
: ""
|
||||
}`);
|
||||
}
|
||||
});
|
||||
server.listen(0, "127.0.0.1");
|
||||
await once(server, "listening");
|
||||
return {
|
||||
origin: `http://127.0.0.1:${server.address().port}`,
|
||||
setOtherOrigin: (origin) => {
|
||||
otherOrigin = origin;
|
||||
},
|
||||
close: async () => {
|
||||
const closed = once(server, "close");
|
||||
server.close();
|
||||
server.closeAllConnections();
|
||||
await closed;
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
/** Real attached-page policy checks, with caller-owned workers on a second origin. */
|
||||
export async function verifyScenarioServiceWorkers(endpoint, targetId) {
|
||||
const first = await startSite();
|
||||
const second = await startSite();
|
||||
first.setOtherOrigin(second.origin);
|
||||
second.setOtherOrigin(first.origin);
|
||||
const observer = await chromium.connectOverCDP(endpoint);
|
||||
let provider = createBrowserScenarioProvider(process.env);
|
||||
let session;
|
||||
let failure;
|
||||
try {
|
||||
let page;
|
||||
for (const context of observer.contexts()) {
|
||||
for (const candidate of context.pages()) {
|
||||
const cdp = await context.newCDPSession(candidate);
|
||||
const { targetInfo } = await cdp.send("Target.getTargetInfo");
|
||||
await cdp.detach();
|
||||
if (targetInfo.targetId === targetId) page = candidate;
|
||||
}
|
||||
}
|
||||
assert.ok(page, "External fixture page disappeared");
|
||||
const scenarioFor = (url, actions = [], policy = "block") =>
|
||||
browserScenarioSchema.parse({
|
||||
browser: {
|
||||
mode: "connect",
|
||||
cdp_endpoint: endpoint,
|
||||
target_id: targetId,
|
||||
},
|
||||
start_url: { url },
|
||||
environment: { service_workers: policy },
|
||||
actions:
|
||||
actions.length === 0
|
||||
? [
|
||||
{
|
||||
step_id: "settle",
|
||||
action: "wait_for_timeout",
|
||||
duration_ms: 1,
|
||||
},
|
||||
]
|
||||
: actions,
|
||||
capture: { events: [], after_each_step: ["dom"], at_end: ["dom"] },
|
||||
});
|
||||
const assertRestored = async () => {
|
||||
assert.equal(page.isClosed(), false);
|
||||
assert.equal(
|
||||
await page.evaluate(() =>
|
||||
ServiceWorkerContainer.prototype.register
|
||||
.toString()
|
||||
.includes("[native code]"),
|
||||
),
|
||||
true,
|
||||
);
|
||||
};
|
||||
const assertRejected = async (scenario) => {
|
||||
const result = await provider.captureScenario(scenario);
|
||||
assert.equal(
|
||||
result.ok,
|
||||
false,
|
||||
"Worker-influenced capture unexpectedly succeeded",
|
||||
);
|
||||
assert.equal(result.error.reason, "target_not_allowed");
|
||||
assert.equal(
|
||||
result.error.partialObservation.capture.browser.cleanup,
|
||||
"disconnected-external",
|
||||
);
|
||||
await assertRestored();
|
||||
};
|
||||
|
||||
// A fresh attached origin blocks registration, then restores both the
|
||||
// current document and future documents when its transport disconnects.
|
||||
const clean = scenarioFor(`${first.origin}/register`, [
|
||||
{
|
||||
step_id: "registration-rejected",
|
||||
action: "wait_for",
|
||||
locator: { kind: "css", selector: "#state[data-blocked='true']" },
|
||||
state: "visible",
|
||||
},
|
||||
]);
|
||||
const cleanCapture = (await runScenarioCli(clean)).normalized_result;
|
||||
assert.ok(cleanCapture.steps.every(({ status }) => status === "completed"));
|
||||
assert.equal(cleanCapture.browser.cleanup, "disconnected-external");
|
||||
await assertRestored();
|
||||
await page.reload();
|
||||
await page.getByText("registered", { exact: true }).waitFor();
|
||||
await page.evaluate(async () => {
|
||||
for (const registration of await navigator.serviceWorker.getRegistrations())
|
||||
await registration.unregister();
|
||||
});
|
||||
await page.goto("about:blank");
|
||||
|
||||
// The caller installs a real worker at a different origin, then moves the
|
||||
// selected target away before REA attaches. Its registration must survive.
|
||||
await page.goto(second.origin);
|
||||
await page.evaluate(async () => {
|
||||
await navigator.serviceWorker.register("/sw.js");
|
||||
await navigator.serviceWorker.ready;
|
||||
});
|
||||
await page.reload();
|
||||
assert.equal(
|
||||
await page.evaluate(async () => (await fetch("/probe")).text()),
|
||||
"worker",
|
||||
);
|
||||
await page.goto("about:blank");
|
||||
await assertRejected(scenarioFor(second.origin));
|
||||
await page.goto("about:blank");
|
||||
await assertRejected(
|
||||
scenarioFor(first.origin, [
|
||||
{
|
||||
step_id: "cross-origin",
|
||||
action: "goto",
|
||||
destination: { url: second.origin },
|
||||
wait_until: "load",
|
||||
},
|
||||
]),
|
||||
);
|
||||
await page.goto("about:blank");
|
||||
await assertRejected(
|
||||
scenarioFor(first.origin, [
|
||||
{
|
||||
step_id: "link-navigation",
|
||||
action: "click",
|
||||
locator: { kind: "css", selector: "#navigate" },
|
||||
},
|
||||
{ step_id: "loaded", action: "wait_for_timeout", duration_ms: 100 },
|
||||
]),
|
||||
);
|
||||
await page.goto("about:blank");
|
||||
await assertRejected(scenarioFor(`${first.origin}/frame`));
|
||||
|
||||
const allowed = await provider.captureScenario(
|
||||
scenarioFor(second.origin, [], "allow"),
|
||||
);
|
||||
if (!allowed.ok) throw allowed.error;
|
||||
assert.ok(
|
||||
allowed.value.steps.every(({ status }) => status === "completed"),
|
||||
);
|
||||
assert.equal(
|
||||
await page.evaluate(async () => (await fetch("/probe")).text()),
|
||||
"worker",
|
||||
);
|
||||
assert.equal(
|
||||
await page.evaluate(
|
||||
async () => (await navigator.serviceWorker.getRegistrations()).length,
|
||||
),
|
||||
1,
|
||||
);
|
||||
|
||||
// Restoration also runs after navigation failure and active cancellation.
|
||||
await page.goto("about:blank");
|
||||
const failed = await provider.captureScenario(
|
||||
scenarioFor(`${first.origin}/failed-navigation`),
|
||||
);
|
||||
assert.equal(failed.ok, false);
|
||||
// Chromium completes its error-document transition after goto rejects.
|
||||
await page.waitForSelector("#main-frame-error", { state: "attached" });
|
||||
// A context replacement can defer restoration. Retained cleanup must
|
||||
// restore the replacement document before releasing the external page.
|
||||
await provider.close();
|
||||
provider = createBrowserScenarioProvider(process.env);
|
||||
await page.goto(first.origin);
|
||||
await assertRestored();
|
||||
const cancellation = new AbortController();
|
||||
const pending = provider.captureScenario(
|
||||
scenarioFor(`${first.origin}/register`, [
|
||||
{
|
||||
step_id: "wait",
|
||||
action: "wait_for",
|
||||
locator: { kind: "css", selector: "#absent" },
|
||||
state: "visible",
|
||||
timeout_ms: 30_000,
|
||||
},
|
||||
]),
|
||||
{ signal: cancellation.signal },
|
||||
);
|
||||
try {
|
||||
const deadline = Date.now() + 10_000;
|
||||
while ((await page.locator("#state").textContent()) !== "blocked") {
|
||||
assert.ok(
|
||||
Date.now() < deadline,
|
||||
"Scenario never applied registration policy",
|
||||
);
|
||||
await delay(25);
|
||||
}
|
||||
} finally {
|
||||
cancellation.abort();
|
||||
}
|
||||
const cancelled = await pending;
|
||||
assert.ok(
|
||||
!cancelled.ok ||
|
||||
cancelled.value.steps.some(({ status }) => status === "cancelled"),
|
||||
);
|
||||
await assertRestored();
|
||||
|
||||
// Caller changes made after installation belong to the caller, too.
|
||||
session = await PlaywrightScenarioSession.open(
|
||||
scenarioFor(first.origin),
|
||||
process.env,
|
||||
{},
|
||||
);
|
||||
await page.evaluate(() => {
|
||||
ServiceWorkerContainer.prototype.register =
|
||||
function callerReplacement() {};
|
||||
});
|
||||
await session.close();
|
||||
session = undefined;
|
||||
assert.equal(
|
||||
await page.evaluate(() => ServiceWorkerContainer.prototype.register.name),
|
||||
"callerReplacement",
|
||||
);
|
||||
await page.reload();
|
||||
await assertRestored();
|
||||
await page.evaluate(() => {
|
||||
delete ServiceWorkerContainer.prototype.register;
|
||||
});
|
||||
const policyBrowser = await chromium.connectOverCDP(endpoint);
|
||||
try {
|
||||
let policyPage;
|
||||
for (const context of policyBrowser.contexts()) {
|
||||
for (const candidate of context.pages()) {
|
||||
const cdp = await context.newCDPSession(candidate);
|
||||
const { targetInfo } = await cdp.send("Target.getTargetInfo");
|
||||
await cdp.detach();
|
||||
if (targetInfo.targetId === targetId) policyPage = candidate;
|
||||
}
|
||||
}
|
||||
assert.ok(policyPage);
|
||||
const policy = new PlaywrightScenarioServiceWorkers(
|
||||
policyPage,
|
||||
endpoint,
|
||||
targetId,
|
||||
);
|
||||
await policy.initialize();
|
||||
await policy.close();
|
||||
} finally {
|
||||
await policyBrowser.close();
|
||||
}
|
||||
assert.equal(
|
||||
await page.evaluate(() =>
|
||||
Object.hasOwn(ServiceWorkerContainer.prototype, "register"),
|
||||
),
|
||||
false,
|
||||
);
|
||||
await page.reload();
|
||||
await assertRestored();
|
||||
const policyCleanup = await verifyPolicyCleanup({
|
||||
page,
|
||||
endpoint,
|
||||
targetId,
|
||||
provider,
|
||||
scenarioFor,
|
||||
origin: first.origin,
|
||||
});
|
||||
return {
|
||||
mocked: false,
|
||||
cli: true,
|
||||
production_provider: true,
|
||||
initial_navigation: true,
|
||||
action_navigation: true,
|
||||
cross_origin_frame: true,
|
||||
allow_preserves_existing_worker: true,
|
||||
registration_blocked: true,
|
||||
live_document_and_reload_restored: true,
|
||||
failure_and_cancellation_cleanup: true,
|
||||
caller_replacement_preserved: true,
|
||||
caller_original_absence_preserved: true,
|
||||
policyCleanup,
|
||||
external_target_preserved: true,
|
||||
};
|
||||
} catch (cause) {
|
||||
failure = cause;
|
||||
throw cause;
|
||||
} finally {
|
||||
await session?.close();
|
||||
try {
|
||||
await provider.close();
|
||||
} catch (cleanupFailure) {
|
||||
if (failure !== undefined)
|
||||
throw new AggregateError([failure, cleanupFailure]);
|
||||
throw cleanupFailure;
|
||||
} finally {
|
||||
await observer.close();
|
||||
await first.close();
|
||||
await second.close();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const verifyPolicyCleanup = async ({
|
||||
page,
|
||||
endpoint,
|
||||
targetId,
|
||||
provider,
|
||||
scenarioFor,
|
||||
origin,
|
||||
}) => {
|
||||
const nativeRegister = async () =>
|
||||
page.evaluate(() =>
|
||||
navigator.serviceWorker.register.toString().includes("[native code]"),
|
||||
);
|
||||
for (const path of ["/pending-image", "/pending-frame"]) {
|
||||
const cancellation = new AbortController();
|
||||
const pending = provider.captureScenario(scenarioFor(`${origin}${path}`), {
|
||||
signal: cancellation.signal,
|
||||
});
|
||||
try {
|
||||
await page
|
||||
.locator(
|
||||
path === "/pending-image" ? "#pending-image" : "#pending-frame",
|
||||
)
|
||||
.waitFor({ state: "attached" });
|
||||
if (path === "/pending-frame") {
|
||||
const child = page
|
||||
.frames()
|
||||
.find((frame) => frame.parentFrame() !== null);
|
||||
assert.ok(child);
|
||||
await child.waitForURL(`${origin}/pending-child`, {
|
||||
waitUntil: "domcontentloaded",
|
||||
});
|
||||
}
|
||||
} finally {
|
||||
cancellation.abort();
|
||||
}
|
||||
const result = await pending;
|
||||
assert.equal(result.ok, false);
|
||||
assert.equal(
|
||||
result.error.partialObservation.capture.browser.cleanup,
|
||||
"disconnected-external",
|
||||
);
|
||||
assert.equal(await nativeRegister(), true);
|
||||
const next = await provider.captureScenario(scenarioFor(origin));
|
||||
if (!next.ok) throw next.error;
|
||||
assert.equal(next.value.browser.cleanup, "disconnected-external");
|
||||
}
|
||||
|
||||
await page.addInitScript("globalThis.reaCallerInitScript = 'preserved'");
|
||||
const policy = new PlaywrightScenarioServiceWorkers(page, endpoint, targetId);
|
||||
await policy.initialize();
|
||||
const exposure = await page.evaluate(async () => {
|
||||
const register = navigator.serviceWorker.register;
|
||||
const originals = Object.getOwnPropertySymbols(register).map(
|
||||
(symbol) => register[symbol],
|
||||
);
|
||||
const restore = Object.getOwnPropertyNames(globalThis).find((name) =>
|
||||
name.startsWith("rea-restore-"),
|
||||
);
|
||||
const denied = globalThis[restore]("not-the-session-secret");
|
||||
let blocked = false;
|
||||
try {
|
||||
await navigator.serviceWorker.register("/sw.js");
|
||||
} catch {
|
||||
blocked = true;
|
||||
}
|
||||
return {
|
||||
exposesOriginal: originals.some(
|
||||
(value) => typeof value?.value === "function",
|
||||
),
|
||||
denied,
|
||||
blocked,
|
||||
registrations: (await navigator.serviceWorker.getRegistrations()).length,
|
||||
};
|
||||
});
|
||||
assert.deepEqual(exposure, {
|
||||
exposesOriginal: false,
|
||||
denied: false,
|
||||
blocked: true,
|
||||
registrations: 0,
|
||||
});
|
||||
|
||||
// Crossing the renderer boundary must not layer a wrapper over an earlier
|
||||
// wrapper in a surviving realm, or leave an OOPIF unprotected.
|
||||
const crossSite = `${origin.replace("127.0.0.1", "localhost")}/register`;
|
||||
await page.evaluate((url) => {
|
||||
const child = document.createElement("iframe");
|
||||
child.id = "cross-site";
|
||||
child.src = url;
|
||||
document.body.append(child);
|
||||
}, crossSite);
|
||||
const child = page.frames().find((frame) => frame.parentFrame() !== null);
|
||||
assert.ok(child);
|
||||
await child.waitForURL(crossSite, { waitUntil: "load" });
|
||||
assert.equal(await child.locator("#state").textContent(), "blocked");
|
||||
await page.locator("#cross-site").evaluate((frame, url) => {
|
||||
frame.src = url;
|
||||
}, `${origin}/register`);
|
||||
await child.waitForURL(`${origin}/register`, { waitUntil: "load" });
|
||||
assert.equal(await child.locator("#state").textContent(), "blocked");
|
||||
|
||||
// Gate one real frame evaluation to force a new main document between live
|
||||
// frame restorations. Browser replies and navigation remain real.
|
||||
const originalEvaluate = child.evaluate.bind(child);
|
||||
let release;
|
||||
const gate = new Promise((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
let entered;
|
||||
const started = new Promise((resolve) => {
|
||||
entered = resolve;
|
||||
});
|
||||
child.evaluate = async (...args) => {
|
||||
entered();
|
||||
await gate;
|
||||
return originalEvaluate(...args);
|
||||
};
|
||||
const closing = policy.close();
|
||||
void closing.catch(() => undefined);
|
||||
try {
|
||||
await started;
|
||||
await page.goto(origin);
|
||||
} finally {
|
||||
release();
|
||||
child.evaluate = originalEvaluate;
|
||||
}
|
||||
await closing;
|
||||
assert.equal(await nativeRegister(), true);
|
||||
await page.reload();
|
||||
assert.equal(await nativeRegister(), true);
|
||||
assert.equal(
|
||||
await page.evaluate(() => globalThis.reaCallerInitScript),
|
||||
"preserved",
|
||||
);
|
||||
assert.equal(
|
||||
await page.evaluate(() =>
|
||||
Object.getOwnPropertyNames(globalThis).some((name) =>
|
||||
name.startsWith("rea-restore-"),
|
||||
),
|
||||
),
|
||||
false,
|
||||
);
|
||||
await verifyRejectedPolicyInstallation(page.context(), endpoint, origin);
|
||||
const setup = await verifyScenarioPolicySetup(
|
||||
endpoint,
|
||||
targetId,
|
||||
origin,
|
||||
page,
|
||||
);
|
||||
return {
|
||||
stalled_image_cancelled: true,
|
||||
stalled_frame_cancelled: true,
|
||||
target_reused_after_cancel: true,
|
||||
private_original_descriptor: true,
|
||||
wrong_restore_token_rejected: true,
|
||||
renderer_transition: true,
|
||||
navigation_during_restoration: true,
|
||||
caller_init_script_preserved: true,
|
||||
failed_main_installation_rejected: true,
|
||||
failed_accessor_installation_rejected: true,
|
||||
failed_iframe_installation_rejected: true,
|
||||
replaced_iframe_policy_rejected: true,
|
||||
setup,
|
||||
};
|
||||
};
|
||||
|
||||
const verifyRejectedPolicyInstallation = async (context, endpoint, origin) => {
|
||||
for (const realm of ["main", "accessor", "existing-iframe", "new-iframe"]) {
|
||||
const page = await context.newPage();
|
||||
let policy;
|
||||
try {
|
||||
await page.goto(origin);
|
||||
const cdp = await context.newCDPSession(page);
|
||||
const { targetInfo } = await cdp.send("Target.getTargetInfo");
|
||||
await cdp.detach();
|
||||
policy = new PlaywrightScenarioServiceWorkers(
|
||||
page,
|
||||
endpoint,
|
||||
targetInfo.targetId,
|
||||
);
|
||||
if (realm === "new-iframe") await policy.initialize();
|
||||
let frame = page.mainFrame();
|
||||
if (realm === "existing-iframe" || realm === "new-iframe") {
|
||||
const crossSite = origin.replace("127.0.0.1", "localhost");
|
||||
await page.evaluate((url) => {
|
||||
const child = document.createElement("iframe");
|
||||
child.src = url;
|
||||
document.body.append(child);
|
||||
}, crossSite);
|
||||
frame = page
|
||||
.frames()
|
||||
.find((candidate) => candidate.parentFrame() !== null);
|
||||
assert.ok(frame);
|
||||
await frame.waitForURL(crossSite, { waitUntil: "load" });
|
||||
}
|
||||
await frame.evaluate((kind) => {
|
||||
const prototype = ServiceWorkerContainer.prototype;
|
||||
const descriptor = Object.getOwnPropertyDescriptor(
|
||||
prototype,
|
||||
"register",
|
||||
);
|
||||
if (kind === "accessor") {
|
||||
const original = descriptor.value;
|
||||
Object.defineProperty(prototype, "register", {
|
||||
configurable: true,
|
||||
enumerable: descriptor.enumerable,
|
||||
get: () => original,
|
||||
});
|
||||
return;
|
||||
}
|
||||
Object.defineProperty(prototype, "register", {
|
||||
...descriptor,
|
||||
configurable: false,
|
||||
...(kind === "new-iframe"
|
||||
? { value: function callerReplacement() {} }
|
||||
: {}),
|
||||
});
|
||||
}, realm);
|
||||
await assert.rejects(
|
||||
realm === "new-iframe" ? policy.assertBlocked() : policy.initialize(),
|
||||
(error) => error.reason === "target_not_allowed",
|
||||
);
|
||||
await policy.close();
|
||||
assert.equal(
|
||||
await frame.evaluate((replaced) => {
|
||||
const register = navigator.serviceWorker.register;
|
||||
return replaced
|
||||
? register.name === "callerReplacement"
|
||||
: register.toString().includes("[native code]");
|
||||
}, realm === "new-iframe"),
|
||||
true,
|
||||
);
|
||||
} finally {
|
||||
try {
|
||||
await policy?.close();
|
||||
} finally {
|
||||
await page.close();
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
|
@ -1,8 +1,36 @@
|
|||
import { execFile } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
import { readdir, stat } from "node:fs/promises";
|
||||
import { readFile, readdir, stat } from "node:fs/promises";
|
||||
import { basename } from "node:path";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
// CPython's interface option ends interpreter parsing; following values are
|
||||
// script arguments. -W/-X consume either an attached value or the next operand.
|
||||
// https://docs.python.org/3.12/using/cmdline.html
|
||||
const pythonScriptOperand = (arguments_) => {
|
||||
for (let index = 1; index < arguments_.length; index++) {
|
||||
const option = arguments_[index];
|
||||
if (option === "--") return arguments_[index + 1];
|
||||
if (option === "-" || option === "") return undefined;
|
||||
if (!option.startsWith("-")) return option;
|
||||
if (option === "--check-hash-based-pycs") {
|
||||
index++;
|
||||
continue;
|
||||
}
|
||||
if (/^--(?:help(?:-env|-xoptions|-all)?|version)$/u.test(option))
|
||||
return undefined;
|
||||
if (option.startsWith("--"))
|
||||
throw new Error(`Unknown Python helper invocation option: ${option}`);
|
||||
for (let character = 1; character < option.length; character++) {
|
||||
const flag = option[character];
|
||||
if ("cmh?V".includes(flag)) return undefined;
|
||||
if (flag === "W" || flag === "X") {
|
||||
if (character === option.length - 1) index++;
|
||||
break;
|
||||
}
|
||||
if (!"bBdEiIOPqRsSuvx".includes(flag))
|
||||
throw new Error(`Unknown Python helper invocation option: ${option}`);
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
|
||||
/**
|
||||
* Observe leftover demo processes on a dedicated clean Linux runner, including
|
||||
|
|
@ -18,9 +46,6 @@ export const assertCleanLinuxHopperRuntime = async (
|
|||
],
|
||||
procRoot = "/proc",
|
||||
) => {
|
||||
const { stdout } = await execFileAsync("ps", ["-ax", "-o", "command="]);
|
||||
if (/\/opt\/hopper\/bin\/Hopper|\bXvfb\b|hopper-demo-x11\.py/u.test(stdout))
|
||||
throw new Error("Clean Linux runner retained a Hopper demo process");
|
||||
const launchers = [];
|
||||
for (const path of launcherPaths) {
|
||||
try {
|
||||
|
|
@ -29,20 +54,55 @@ export const assertCleanLinuxHopperRuntime = async (
|
|||
if (error?.code !== "ENOENT") throw error;
|
||||
}
|
||||
}
|
||||
if (launchers.length === 0) return;
|
||||
// /proc exposes the executed file even when argv[0] names a removed symlink.
|
||||
// Compare file identity, not a display path or a process name.
|
||||
let inspectionError;
|
||||
for (const entry of await readdir(procRoot, { withFileTypes: true })) {
|
||||
if (!entry.isDirectory() || !/^\d+$/u.test(entry.name)) continue;
|
||||
let executable;
|
||||
let retainedRuntime;
|
||||
try {
|
||||
executable = await stat(`${procRoot}/${entry.name}/exe`, {
|
||||
bigint: true,
|
||||
});
|
||||
// procfs preserves argument boundaries. A caller merely mentioning a
|
||||
// runtime in its script text or other arguments is not that runtime.
|
||||
const commandLine = await readFile(
|
||||
`${procRoot}/${entry.name}/cmdline`,
|
||||
"utf8",
|
||||
);
|
||||
const arguments_ = commandLine.split("\0");
|
||||
// The kernel command field identifies renamed runtime processes such
|
||||
// as Xvfb. Keep it separate from arguments so ordinary script text does
|
||||
// not count as a retained runtime.
|
||||
const kernelCommand = (
|
||||
await readFile(`${procRoot}/${entry.name}/comm`, "utf8")
|
||||
).replace(/\n$/u, "");
|
||||
if (
|
||||
commandLine.length === 0 &&
|
||||
/^python3(?:\.\d+)?$/u.test(kernelCommand)
|
||||
)
|
||||
throw new Error(
|
||||
"Cannot identify a command-less Python process on the clean Linux runner",
|
||||
);
|
||||
retainedRuntime =
|
||||
kernelCommand === "Hopper" ||
|
||||
kernelCommand === "Xvfb" ||
|
||||
arguments_[0] === "/opt/hopper/bin/Hopper" ||
|
||||
basename(arguments_[0] ?? "") === "Xvfb" ||
|
||||
(/^python3(?:\.\d+)?$/u.test(basename(arguments_[0] ?? "")) &&
|
||||
arguments_.some(
|
||||
(value) => basename(value) === "hopper-demo-x11.py",
|
||||
) &&
|
||||
basename(pythonScriptOperand(arguments_) ?? "") ===
|
||||
"hopper-demo-x11.py");
|
||||
if (!retainedRuntime && launchers.length > 0) {
|
||||
const executable = await stat(`${procRoot}/${entry.name}/exe`, {
|
||||
bigint: true,
|
||||
});
|
||||
retainedRuntime = launchers.some(
|
||||
({ dev, ino }) => dev === executable.dev && ino === executable.ino,
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
if (error?.code === "ENOENT" || error?.code === "ESRCH") continue;
|
||||
// Other users' executables may be unreadable. The setup lane launches
|
||||
// Other users' process facts may be unreadable. The setup lane launches
|
||||
// providers as the caller; unreadable caller-owned processes fail closed.
|
||||
if (error?.code === "EACCES") {
|
||||
const owner = await stat(`${procRoot}/${entry.name}`).catch((cause) => {
|
||||
|
|
@ -56,11 +116,7 @@ export const assertCleanLinuxHopperRuntime = async (
|
|||
inspectionError ??= error;
|
||||
continue;
|
||||
}
|
||||
if (
|
||||
launchers.some(
|
||||
({ dev, ino }) => dev === executable.dev && ino === executable.ino,
|
||||
)
|
||||
)
|
||||
if (retainedRuntime)
|
||||
throw new Error("Clean Linux runner retained a Hopper demo process");
|
||||
}
|
||||
if (inspectionError !== undefined) throw inspectionError;
|
||||
|
|
|
|||
|
|
@ -0,0 +1,149 @@
|
|||
import assert from "node:assert/strict";
|
||||
|
||||
export const WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE = Object.freeze({
|
||||
pattern: "a0 ?? c0",
|
||||
mask: "f0 ff f0",
|
||||
startFileOffset: 0x380,
|
||||
endFileOffset: 0x392,
|
||||
markerOffsets: Object.freeze([0x380, 0x390]),
|
||||
});
|
||||
|
||||
/** Put two masked, wildcarded signatures in loaded, source-owned PE bytes. */
|
||||
export function writeWindowsGhidraByteSearchFixture(bytes) {
|
||||
bytes.set([0xa1, 0xb2, 0xc3], 0x380);
|
||||
bytes.set([0xa4, 0xb5, 0xc6], 0x390);
|
||||
}
|
||||
|
||||
/** Verify real Ghidra search pages against the immutable source fixture. */
|
||||
export async function verifyWindowsGhidraByteSearch({
|
||||
invoke,
|
||||
sourceBytes,
|
||||
architecture,
|
||||
sourceOwnedFixture = true,
|
||||
fallbackAddress,
|
||||
fallbackFileOffset,
|
||||
runCliPage,
|
||||
}) {
|
||||
const imageBase = architecture === "x86" ? 0x400000n : 0x140000000n;
|
||||
const query = sourceOwnedFixture
|
||||
? {
|
||||
pattern: WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE.pattern,
|
||||
mask: WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE.mask,
|
||||
start: addressAtFileOffset(
|
||||
imageBase,
|
||||
WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE.startFileOffset,
|
||||
),
|
||||
end: addressAtFileOffset(
|
||||
imageBase,
|
||||
WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE.endFileOffset,
|
||||
),
|
||||
limit: 1,
|
||||
}
|
||||
: {
|
||||
pattern: sourceBytes
|
||||
.subarray(fallbackFileOffset, fallbackFileOffset + 1)
|
||||
.toString("hex"),
|
||||
start: fallbackAddress,
|
||||
end: fallbackAddress,
|
||||
limit: 1,
|
||||
};
|
||||
|
||||
if (!sourceOwnedFixture) {
|
||||
const result = await invoke("search_bytes", query);
|
||||
assert.equal(result.complete, true);
|
||||
assert.equal(result.next_cursor, null);
|
||||
assert.deepEqual(
|
||||
result.matches.map(({ address, file_offset }) => ({
|
||||
address,
|
||||
file_offset,
|
||||
})),
|
||||
[{ address: fallbackAddress, file_offset: fallbackFileOffset }],
|
||||
);
|
||||
assert.ok(result.searched.bytes > 0);
|
||||
return { bounded_source_match: true };
|
||||
}
|
||||
|
||||
const { startFileOffset, endFileOffset } = WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE;
|
||||
const expected = [];
|
||||
for (
|
||||
let fileOffset = startFileOffset;
|
||||
fileOffset + 2 <= endFileOffset;
|
||||
fileOffset += 1
|
||||
) {
|
||||
const first = sourceBytes[fileOffset];
|
||||
const last = sourceBytes[fileOffset + 2];
|
||||
if (first === undefined || last === undefined) break;
|
||||
if ((first & 0xf0) === 0xa0 && (last & 0xf0) === 0xc0)
|
||||
expected.push({
|
||||
address: addressAtFileOffset(imageBase, fileOffset),
|
||||
file_offset: fileOffset,
|
||||
});
|
||||
}
|
||||
assert.deepEqual(
|
||||
expected.map(({ file_offset }) => file_offset),
|
||||
[...WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE.markerOffsets],
|
||||
"The source-owned byte oracle must retain both distinct signatures",
|
||||
);
|
||||
|
||||
const first = await invoke("search_bytes", query);
|
||||
assert.deepEqual(
|
||||
first.matches.map(({ address, file_offset }) => ({ address, file_offset })),
|
||||
[expected[0]],
|
||||
);
|
||||
assert.equal(first.complete, false);
|
||||
assert.equal(typeof first.next_cursor, "string");
|
||||
assert.ok(first.searched.bytes > 0);
|
||||
if (runCliPage !== undefined) {
|
||||
const cli = await runCliPage(query);
|
||||
assert.deepEqual(cli.matches, first.matches);
|
||||
assert.deepEqual(cli.searched, first.searched);
|
||||
assert.equal(cli.complete, first.complete);
|
||||
}
|
||||
|
||||
const second = await invoke("search_bytes", {
|
||||
...query,
|
||||
cursor: first.next_cursor,
|
||||
});
|
||||
assert.deepEqual(
|
||||
second.matches.map(({ address, file_offset }) => ({
|
||||
address,
|
||||
file_offset,
|
||||
})),
|
||||
[expected[1]],
|
||||
);
|
||||
assert.equal(second.complete, true);
|
||||
assert.equal(second.next_cursor, null);
|
||||
assert.ok(second.searched.bytes > 0);
|
||||
|
||||
const covered = new Set();
|
||||
for (const page of [first, second]) {
|
||||
assert.equal(
|
||||
page.searched.bytes,
|
||||
page.searched.blocks.reduce((sum, block) => sum + block.bytes, 0),
|
||||
);
|
||||
for (const block of page.searched.blocks) {
|
||||
const start = BigInt(block.start);
|
||||
const end = BigInt(block.end);
|
||||
assert.ok(start >= BigInt(query.start) && end <= BigInt(query.end));
|
||||
for (let offset = start; offset <= end; offset += 1n)
|
||||
covered.add(offset.toString());
|
||||
}
|
||||
}
|
||||
assert.equal(
|
||||
covered.size,
|
||||
endFileOffset - startFileOffset + 1,
|
||||
"Paged byte-search coverage must account for the complete bounded source range",
|
||||
);
|
||||
return {
|
||||
bounded: true,
|
||||
wildcardAndMask: true,
|
||||
matches: expected,
|
||||
continuation: true,
|
||||
examinedBytes: covered.size,
|
||||
cliMcpParity: runCliPage !== undefined,
|
||||
};
|
||||
}
|
||||
|
||||
function addressAtFileOffset(imageBase, fileOffset) {
|
||||
return `0x${(imageBase + 0x1000n + BigInt(fileOffset - 0x200)).toString(16)}`;
|
||||
}
|
||||
|
|
@ -62,11 +62,14 @@ export async function verifyGhidraAddressBoundaries(client, entry, cli) {
|
|||
await query("read_bytes", { address: padded, length: 1 }),
|
||||
await query("read_bytes", { address: entry, length: 1 }),
|
||||
);
|
||||
const aliasKeys = [entry, padded];
|
||||
const originalName = await query("address_name", { address: entry });
|
||||
assert.equal(typeof originalName, "string", "Entry has no restorable name");
|
||||
const failures = [];
|
||||
let restorationFailed = false;
|
||||
assert.equal(
|
||||
typeof originalName,
|
||||
"string",
|
||||
"The function entry needs an observed name that can be restored",
|
||||
);
|
||||
const aliasKeys = [entry, padded];
|
||||
let primaryFailure;
|
||||
try {
|
||||
assert.deepEqual(
|
||||
await query("set_addresses_names", {
|
||||
|
|
@ -78,10 +81,10 @@ export async function verifyGhidraAddressBoundaries(client, entry, cli) {
|
|||
Object.fromEntries(aliasKeys.map((address) => [address, true])),
|
||||
"Equivalent address spellings with one requested name share one edit",
|
||||
);
|
||||
assert.equal(
|
||||
await query("address_name", { address: entry }),
|
||||
"rea_same_address_alias",
|
||||
);
|
||||
const nameBeforeAliasedBatch = await query("address_name", {
|
||||
address: entry,
|
||||
});
|
||||
assert.equal(nameBeforeAliasedBatch, "rea_same_address_alias");
|
||||
assert.deepEqual(
|
||||
await query("set_addresses_names", {
|
||||
names: {
|
||||
|
|
@ -94,36 +97,33 @@ export async function verifyGhidraAddressBoundaries(client, entry, cli) {
|
|||
);
|
||||
assert.equal(
|
||||
await query("address_name", { address: entry }),
|
||||
"rea_same_address_alias",
|
||||
nameBeforeAliasedBatch,
|
||||
"A rejected alias collision must leave the preexisting target name unchanged",
|
||||
);
|
||||
} catch (cause) {
|
||||
failures.push(cause);
|
||||
primaryFailure = cause;
|
||||
throw cause;
|
||||
} finally {
|
||||
// Restore even when a failed reply leaves the write outcome uncertain.
|
||||
// The public API restores the observed name, not private symbol provenance.
|
||||
try {
|
||||
assert.equal(
|
||||
await query("set_address_name", { address: entry, name: originalName }),
|
||||
true,
|
||||
"Temporary address alias restoration was rejected",
|
||||
"The temporary alias edit must be restored",
|
||||
);
|
||||
assert.equal(
|
||||
await query("address_name", { address: entry }),
|
||||
originalName,
|
||||
"Temporary address alias restoration did not restore the observed name",
|
||||
"The original function-entry name must survive restoration",
|
||||
);
|
||||
} catch (cause) {
|
||||
restorationFailed = true;
|
||||
failures.push(cause);
|
||||
} catch (cleanupCause) {
|
||||
if (primaryFailure !== undefined)
|
||||
throw new AggregateError(
|
||||
[primaryFailure, cleanupCause],
|
||||
"Ghidra address-boundary verification and alias restoration failed",
|
||||
);
|
||||
throw cleanupCause;
|
||||
}
|
||||
}
|
||||
if (restorationFailed)
|
||||
throw new AggregateError(
|
||||
failures,
|
||||
"Ghidra address boundary verification failed to restore its temporary name",
|
||||
);
|
||||
if (failures.length > 0) throw failures[0];
|
||||
for (const [command, flags, operation] of [
|
||||
["read-bytes", ["--length", "1"], "read_bytes"],
|
||||
[
|
||||
|
|
|
|||
|
|
@ -1,3 +1,7 @@
|
|||
import type { z } from "zod";
|
||||
|
||||
import type { analysisStringSchema } from "../../src/domain/hopperValues.js";
|
||||
|
||||
type VerifierCall = (
|
||||
name: string,
|
||||
arguments_?: Record<string, unknown>,
|
||||
|
|
@ -44,6 +48,3 @@ export function verifyHopperStringObjects(
|
|||
}[];
|
||||
readonly displayEscapesExcludedFromMatching: true;
|
||||
}>;
|
||||
import type { z } from "zod";
|
||||
|
||||
import type { analysisStringSchema } from "../../src/domain/hopperValues.js";
|
||||
|
|
|
|||
|
|
@ -68,7 +68,11 @@ try {
|
|||
}
|
||||
if (!process.exitCode && !shutdownSignal) {
|
||||
const tested = await run(process.execPath, [
|
||||
resolve("node_modules/vitest/vitest.mjs"),
|
||||
resolve(
|
||||
request.paths.length > 0
|
||||
? "scripts/run-exact-tests.mjs"
|
||||
: "node_modules/vitest/vitest.mjs",
|
||||
),
|
||||
...plan.vitestArguments,
|
||||
]);
|
||||
process.exitCode = tested.code;
|
||||
|
|
|
|||
|
|
@ -0,0 +1,134 @@
|
|||
import { lstat, realpath, stat } from "node:fs/promises";
|
||||
import { parse, relative, resolve, sep } from "node:path";
|
||||
|
||||
import { createVitest, parseCLI } from "vitest/node";
|
||||
|
||||
// CLI file filters are substring matches. Select discovered specifications by
|
||||
// identity instead, retaining each project's include/exclude and runner settings.
|
||||
const { filter, options } = parseCLI(["vitest", ...process.argv.slice(2)]);
|
||||
const vitest = await createVitest({ ...options, watch: false });
|
||||
const signalExitCodes = { SIGINT: 130, SIGTERM: 143 };
|
||||
let shutdownSignal;
|
||||
let closePromise;
|
||||
const close = () => (closePromise ??= vitest.close());
|
||||
for (const signal of Object.keys(signalExitCodes))
|
||||
process.on(signal, () => {
|
||||
if (shutdownSignal !== undefined) return;
|
||||
shutdownSignal = signal;
|
||||
void close().catch((cause) => {
|
||||
console.error(cause instanceof Error ? cause.message : String(cause));
|
||||
process.exitCode = 1;
|
||||
});
|
||||
});
|
||||
try {
|
||||
if (shutdownSignal === undefined) await vitest.standalone();
|
||||
if (shutdownSignal === undefined) {
|
||||
const discovered = new Map();
|
||||
for (const specification of await vitest.globTestSpecifications()) {
|
||||
const path = resolve(specification.moduleId);
|
||||
const specifications = discovered.get(path) ?? [];
|
||||
specifications.push(specification);
|
||||
discovered.set(path, specifications);
|
||||
}
|
||||
const selected = new Set();
|
||||
const missing = [];
|
||||
let discoveredByIdentity;
|
||||
for (const path of filter) {
|
||||
const absolute = resolve(path);
|
||||
let matches = discovered.get(absolute) ?? [];
|
||||
if (matches.length === 0) {
|
||||
try {
|
||||
await lstat(absolute);
|
||||
} catch (cause) {
|
||||
if (cause?.code === "ENOENT" || cause?.code === "ENOTDIR") {
|
||||
missing.push(path);
|
||||
continue;
|
||||
}
|
||||
throw cause;
|
||||
}
|
||||
const [canonical, selectedIdentity] = await Promise.all([
|
||||
realpath(absolute),
|
||||
stat(absolute, { bigint: true }),
|
||||
]);
|
||||
if (discoveredByIdentity === undefined) {
|
||||
discoveredByIdentity = new Map();
|
||||
for (const [candidate, specifications] of discovered) {
|
||||
try {
|
||||
const entry = await stat(candidate, { bigint: true });
|
||||
const identity = `${entry.dev}:${entry.ino}`;
|
||||
const entries = discoveredByIdentity.get(identity) ?? [];
|
||||
entries.push([candidate, specifications]);
|
||||
discoveredByIdentity.set(identity, entries);
|
||||
} catch (cause) {
|
||||
if (cause?.code === "ENOENT" || cause?.code === "ENOTDIR")
|
||||
continue;
|
||||
throw cause;
|
||||
}
|
||||
}
|
||||
}
|
||||
const identity = `${selectedIdentity.dev}:${selectedIdentity.ino}`;
|
||||
for (const [candidate, specifications] of discoveredByIdentity.get(
|
||||
identity,
|
||||
) ?? []) {
|
||||
try {
|
||||
await lstat(candidate);
|
||||
} catch (cause) {
|
||||
if (cause?.code === "ENOENT" || cause?.code === "ENOTDIR") continue;
|
||||
throw cause;
|
||||
}
|
||||
if (
|
||||
!(await hasCaseMismatchedSymlinkComponent(absolute, candidate)) &&
|
||||
(await realpath(candidate)) === canonical
|
||||
)
|
||||
matches = [...matches, ...specifications];
|
||||
}
|
||||
}
|
||||
if (matches.length === 0) missing.push(path);
|
||||
for (const specification of matches) selected.add(specification);
|
||||
}
|
||||
if (missing.length > 0 || selected.size === 0)
|
||||
throw new Error(
|
||||
`Selected test files were not discovered: ${missing.join(", ")}`,
|
||||
);
|
||||
if (shutdownSignal === undefined)
|
||||
await vitest.runTestSpecifications([...selected]);
|
||||
}
|
||||
} finally {
|
||||
await close();
|
||||
}
|
||||
if (shutdownSignal !== undefined)
|
||||
process.exitCode = signalExitCodes[shutdownSignal];
|
||||
|
||||
async function hasCaseMismatchedSymlinkComponent(selected, candidate) {
|
||||
const selectedAbsolute = resolve(selected);
|
||||
const candidateAbsolute = resolve(candidate);
|
||||
const root = parse(selectedAbsolute).root;
|
||||
if (root !== parse(candidateAbsolute).root) return true;
|
||||
const selectedParts = relative(root, selectedAbsolute).split(sep);
|
||||
const candidateParts = relative(root, candidateAbsolute).split(sep);
|
||||
let selectedPath = root;
|
||||
let candidatePath = root;
|
||||
for (let index = 0; index < selectedParts.length; index += 1) {
|
||||
const selectedPart = selectedParts[index];
|
||||
const candidatePart = candidateParts[index];
|
||||
if (selectedPart === undefined || candidatePart === undefined) return true;
|
||||
selectedPath = selectedPath
|
||||
? resolve(selectedPath, selectedPart)
|
||||
: selectedPart;
|
||||
candidatePath = candidatePath
|
||||
? resolve(candidatePath, candidatePart)
|
||||
: candidatePart;
|
||||
if (
|
||||
selectedPart !== candidatePart &&
|
||||
selectedPart.toLowerCase() === candidatePart.toLowerCase()
|
||||
) {
|
||||
const [selectedEntry, candidateEntry] = await Promise.all([
|
||||
lstat(selectedPath),
|
||||
lstat(candidatePath),
|
||||
]);
|
||||
if (selectedEntry.isSymbolicLink() || candidateEntry.isSymbolicLink())
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
|
@ -155,6 +155,18 @@ try {
|
|||
requiredAnswerTermGroups: [["profileapi"], ["profile:read"]],
|
||||
prompt: `請靜態分析已交付的 Electron 程式 ${targets.javascript},說明 renderer 如何透過 preload 讀取 profile,以及 main handler 的關係。不要執行目標或安裝分析引擎;引用完整的產出 Evidence,保留 runtime 未知事項。`,
|
||||
},
|
||||
{
|
||||
id: "notes-export",
|
||||
expectedFirstTool: "analyze_javascript_application",
|
||||
requiresEvidence: true,
|
||||
requiredAnswerTermGroups: [],
|
||||
requiredToolSubsequence: [
|
||||
"analyze_javascript_application",
|
||||
"trace_application_feature",
|
||||
"trace_javascript_semantics",
|
||||
],
|
||||
prompt: `Use REA on the shipped Electron application at ${targets.notesElectron} to investigate its notes export API, preload exposure and main handler. Do not read target files directly or launch Electron. Use the returned application Evidence to trace the exact notes:export IPC channel in both directions, then query semantic relations for the property invoke with forward influence, excluding ambiguous dynamic edges and without an expected sink. Report the selected static facts, source locations and coverage limits using the closed claim contract.`,
|
||||
},
|
||||
{
|
||||
id: "javascript-module-view",
|
||||
expectedFirstTool: "analyze_javascript_application",
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ import assert from "node:assert/strict";
|
|||
import { isUtf8 } from "node:buffer";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { lstat, mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { isAbsolute, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
|
|
@ -24,6 +24,8 @@ if (process.env.REA_VERIFY_GO_COMMAND !== undefined && !isAbsolute(compiler))
|
|||
let compilerVersion = null;
|
||||
const run = createVerifierRun();
|
||||
const root = await PrivateRuntimeRoot.create({ prefix: "rea-go-verifier-" });
|
||||
// Keep Go's directory enumeration below the private root's Windows ownership lease.
|
||||
const sourceRoot = join(root.path, "source");
|
||||
const entrypoint = fileURLToPath(new URL("./rea.mjs", import.meta.url));
|
||||
const environment = Object.fromEntries(
|
||||
Object.entries(process.env).filter(([, value]) => typeof value === "string"),
|
||||
|
|
@ -115,14 +117,54 @@ try {
|
|||
],
|
||||
{ GOOS: os, GOARCH: arch },
|
||||
);
|
||||
await verify(path, target, oracle, null, "stripped");
|
||||
const strippedInfo = await verify(path, target, oracle, null, "stripped");
|
||||
if (os === "linux" && arch === "amd64") {
|
||||
const sectionless = join(root.path, "linux-amd64-sectionless");
|
||||
const bytes = Buffer.from(await readFile(path));
|
||||
bytes.fill(0, 40, 48); // ELF64 e_shoff.
|
||||
bytes.fill(0, 58, 64); // e_shentsize, e_shnum and e_shstrndx.
|
||||
await writeFile(sectionless, bytes);
|
||||
await verify(sectionless, target, oracle, null, "sectionless");
|
||||
const sectionlessInfo = await verify(
|
||||
sectionless,
|
||||
target,
|
||||
oracle,
|
||||
null,
|
||||
"sectionless",
|
||||
path,
|
||||
);
|
||||
assert.deepEqual(sectionlessInfo, strippedInfo);
|
||||
|
||||
const pie = join(root.path, "linux-amd64-pie");
|
||||
await go(
|
||||
[
|
||||
"build",
|
||||
"-trimpath",
|
||||
"-buildvcs=false",
|
||||
"-ldflags=-s -w",
|
||||
"-buildmode=pie",
|
||||
"-o",
|
||||
pie,
|
||||
".",
|
||||
],
|
||||
{ GOOS: os, GOARCH: arch },
|
||||
);
|
||||
const originalInfo = await verify(pie, target, oracle, null, "pie");
|
||||
const pieBytes = Buffer.from(await readFile(pie));
|
||||
pieBytes.fill(0, 40, 48);
|
||||
pieBytes.fill(0, 58, 64);
|
||||
const pieSectionless = `${pie}-sectionless`;
|
||||
await writeFile(pieSectionless, pieBytes);
|
||||
// Keep the producer image as the independent oracle. Removing section
|
||||
// headers must preserve every metadata field and original source span.
|
||||
const retainedInfo = await verify(
|
||||
pieSectionless,
|
||||
target,
|
||||
oracle,
|
||||
null,
|
||||
"sectionless-pie",
|
||||
pie,
|
||||
);
|
||||
assert.deepEqual(retainedInfo, originalInfo);
|
||||
}
|
||||
}
|
||||
for (const target of targets.filter(([os]) => os === "darwin")) {
|
||||
|
|
@ -143,7 +185,7 @@ try {
|
|||
const expected = JSON.parse(
|
||||
(
|
||||
await execute(oracle, ["--build-byte-fixture", compiler, path], {
|
||||
cwd: root.path,
|
||||
cwd: sourceRoot,
|
||||
env: { ...goEnvironment, GOOS: "linux", GOARCH: "amd64" },
|
||||
timeout: 120_000,
|
||||
maxBuffer: 1024 * 1024,
|
||||
|
|
@ -169,7 +211,7 @@ try {
|
|||
}
|
||||
const verifier = await completeVerifierRun(run);
|
||||
try {
|
||||
await assert.rejects(readFile(join(root.path, "go.mod")), { code: "ENOENT" });
|
||||
await assert.rejects(lstat(root.path), { code: "ENOENT" });
|
||||
if (verifier.process_lineage.status === "verified")
|
||||
assert.deepEqual(verifier.process_lineage.descendants, []);
|
||||
} catch (cause) {
|
||||
|
|
@ -214,7 +256,7 @@ console.log(
|
|||
|
||||
async function go(args, overrides = {}) {
|
||||
return execute(compiler, args, {
|
||||
cwd: root.path,
|
||||
cwd: sourceRoot,
|
||||
env: { ...goEnvironment, ...overrides },
|
||||
timeout: 120_000,
|
||||
maxBuffer: 1024 * 1024,
|
||||
|
|
@ -222,29 +264,30 @@ async function go(args, overrides = {}) {
|
|||
}
|
||||
|
||||
async function prepareFixture() {
|
||||
await mkdir(sourceRoot);
|
||||
await mkdir(join(root.path, "tmp"));
|
||||
await mkdir(join(root.path, "telemetry"));
|
||||
await writeFile(join(root.path, "telemetry", "mode"), "off\n");
|
||||
await mkdir(join(root.path, "dependency"));
|
||||
await mkdir(join(root.path, "oracle"));
|
||||
await mkdir(join(sourceRoot, "dependency"));
|
||||
await mkdir(join(sourceRoot, "oracle"));
|
||||
await writeFile(
|
||||
join(root.path, "go.mod"),
|
||||
join(sourceRoot, "go.mod"),
|
||||
"module example.test/rea-go-fixture\n\ngo 1.20\n\nrequire example.test/dependency v1.2.3\nreplace example.test/dependency => ./dependency\n",
|
||||
);
|
||||
await writeFile(
|
||||
join(root.path, "dependency", "go.mod"),
|
||||
join(sourceRoot, "dependency", "go.mod"),
|
||||
"module example.test/dependency\n\ngo 1.20\n",
|
||||
);
|
||||
await writeFile(
|
||||
join(root.path, "dependency", "dependency.go"),
|
||||
join(sourceRoot, "dependency", "dependency.go"),
|
||||
'package dependency\nfunc Value() string { return "REA fixture" }\n',
|
||||
);
|
||||
await writeFile(
|
||||
join(root.path, "main.go"),
|
||||
join(sourceRoot, "main.go"),
|
||||
'package main\nimport ("fmt"; "example.test/dependency")\nfunc main() { fmt.Println(dependency.Value()) }\n',
|
||||
);
|
||||
await writeFile(
|
||||
join(root.path, "oracle", "main.go"),
|
||||
join(sourceRoot, "oracle", "main.go"),
|
||||
String.raw`package main
|
||||
|
||||
import (
|
||||
|
|
@ -332,13 +375,14 @@ async function verify(
|
|||
oracle,
|
||||
byteExpected = null,
|
||||
fileKind = "stripped",
|
||||
oraclePath = path,
|
||||
) {
|
||||
const bytes = await readFile(path);
|
||||
const expected =
|
||||
byteExpected ??
|
||||
JSON.parse(
|
||||
(
|
||||
await execute(oracle, [path], {
|
||||
await execute(oracle, [oraclePath], {
|
||||
timeout: 10_000,
|
||||
maxBuffer: 1024 * 1024,
|
||||
})
|
||||
|
|
@ -408,6 +452,7 @@ async function verify(
|
|||
? {}
|
||||
: { go_version_bytes_base64: expected.GoVersionBytesBase64 }),
|
||||
});
|
||||
return reports[0].build_info;
|
||||
}
|
||||
|
||||
function verifyByteStrings(info, expected) {
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
import { execFile } from "node:child_process";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { resolve } from "node:path";
|
||||
import { readdir, readFile } from "node:fs/promises";
|
||||
import { join, resolve } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import { inspectModuleBoundaries } from "./lib/module-boundaries.mjs";
|
||||
|
|
@ -11,24 +11,44 @@ const selected = process.argv.slice(2);
|
|||
for (const file of selected)
|
||||
if (!file.endsWith(".ts"))
|
||||
throw new TypeError(`Expected an exact TypeScript source file: ${file}`);
|
||||
const files =
|
||||
selected.length > 0
|
||||
? selected
|
||||
: (
|
||||
await execute(
|
||||
"git",
|
||||
[
|
||||
"ls-files",
|
||||
"-z",
|
||||
"--cached",
|
||||
"--others",
|
||||
"--exclude-standard",
|
||||
"--",
|
||||
"src",
|
||||
],
|
||||
{ cwd: root, maxBuffer: 8 * 1024 * 1024 },
|
||||
)
|
||||
).stdout.split("\0");
|
||||
|
||||
const enumerateSourceFiles = async () => {
|
||||
if (selected.length > 0) return selected;
|
||||
let isGitRoot = false;
|
||||
try {
|
||||
const { stdout } = await execute("git", ["rev-parse", "--show-cdup"], {
|
||||
cwd: root,
|
||||
maxBuffer: 8 * 1024 * 1024,
|
||||
});
|
||||
isGitRoot = stdout.trim() === "";
|
||||
} catch {
|
||||
// Archives and source snapshots may not be in a Git worktree.
|
||||
}
|
||||
if (isGitRoot) {
|
||||
try {
|
||||
const { stdout } = await execute(
|
||||
"git",
|
||||
[
|
||||
"ls-files",
|
||||
"-z",
|
||||
"--cached",
|
||||
"--others",
|
||||
"--exclude-standard",
|
||||
"--",
|
||||
"src",
|
||||
],
|
||||
{ cwd: root, maxBuffer: 8 * 1024 * 1024 },
|
||||
);
|
||||
return stdout.split("\0");
|
||||
} catch {
|
||||
// Fall back to disk when Git's index is unavailable or unreadable.
|
||||
}
|
||||
}
|
||||
const entries = await readdir(resolve(root, "src"), { recursive: true });
|
||||
return entries.map((entry) => join("src", entry));
|
||||
};
|
||||
|
||||
const files = await enumerateSourceFiles();
|
||||
|
||||
let checked = 0;
|
||||
for (const file of new Set(files.filter((path) => path.endsWith(".ts")))) {
|
||||
|
|
@ -36,7 +56,11 @@ for (const file of new Set(files.filter((path) => path.endsWith(".ts")))) {
|
|||
try {
|
||||
source = await readFile(resolve(root, file), "utf8");
|
||||
} catch (cause) {
|
||||
if (selected.length === 0 && cause.code === "ENOENT") continue;
|
||||
if (
|
||||
selected.length === 0 &&
|
||||
(cause.code === "ENOENT" || cause.code === "EISDIR")
|
||||
)
|
||||
continue;
|
||||
throw cause;
|
||||
}
|
||||
checked += 1;
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import assert from "node:assert/strict";
|
|||
import { createHash } from "node:crypto";
|
||||
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { isAbsolute, join, relative, sep } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import Ajv2020 from "ajv/dist/2020.js";
|
||||
import {
|
||||
|
|
@ -42,6 +42,21 @@ const skillDirectory = join(
|
|||
"skills",
|
||||
"reverse-engineer-anything",
|
||||
);
|
||||
const isManagedSkillTarget = (target) => {
|
||||
const value = relative(skillDirectory, target);
|
||||
return (
|
||||
value === "" ||
|
||||
(value !== ".." && !value.startsWith(`..${sep}`) && !isAbsolute(value))
|
||||
);
|
||||
};
|
||||
assert.equal(
|
||||
isManagedSkillTarget(`${skillDirectory}-sibling/AGENTS.md`),
|
||||
false,
|
||||
);
|
||||
assert.equal(
|
||||
isManagedSkillTarget(join(skillDirectory, "..", "outside", "AGENTS.md")),
|
||||
false,
|
||||
);
|
||||
await Promise.all([mkdir(account), mkdir(profile), mkdir(workspace)]);
|
||||
await exec("git", ["init", "--quiet", workspace], { timeout: 30_000 });
|
||||
const environment = {
|
||||
|
|
@ -264,8 +279,7 @@ try {
|
|||
assert(parsed.plannedActions.some((a) => a.target === skillDirectory));
|
||||
assert(
|
||||
parsed.plannedActions.every(
|
||||
(a) =>
|
||||
a.target === configPath || a.target.startsWith(skillDirectory),
|
||||
(a) => a.target === configPath || isManagedSkillTarget(a.target),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -31,7 +31,14 @@ export async function verifyPackageCapabilitiesAndSearch({ cli, environment }) {
|
|||
const searchResult = json(
|
||||
await run(
|
||||
cli,
|
||||
["search", process.execPath, "fixture", "--json"],
|
||||
[
|
||||
"search",
|
||||
process.execPath,
|
||||
"fixture",
|
||||
"--provider",
|
||||
"hopper",
|
||||
"--json",
|
||||
],
|
||||
environment,
|
||||
),
|
||||
);
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ export async function verifyPackagePlatform({
|
|||
if (platform === "linux") {
|
||||
const unsupportedExecution = await runWithStatus(
|
||||
cli,
|
||||
["analyze", process.execPath, "--json"],
|
||||
["analyze", process.execPath, "--provider", "hopper", "--json"],
|
||||
environment,
|
||||
);
|
||||
assertLinuxPackageProviderFailure(unsupportedExecution);
|
||||
|
|
@ -18,7 +18,7 @@ export async function verifyPackagePlatform({
|
|||
const overview = json(
|
||||
await runCommand(
|
||||
cli,
|
||||
["analyze", process.execPath, "--json"],
|
||||
["analyze", process.execPath, "--provider", "hopper", "--json"],
|
||||
environment,
|
||||
),
|
||||
);
|
||||
|
|
@ -29,7 +29,7 @@ export async function verifyPackagePlatform({
|
|||
const inspected = json(
|
||||
await runCommand(
|
||||
cli,
|
||||
["inspect", process.execPath, "--json"],
|
||||
["inspect", process.execPath, "--provider", "hopper", "--json"],
|
||||
environment,
|
||||
),
|
||||
);
|
||||
|
|
@ -38,7 +38,14 @@ export async function verifyPackagePlatform({
|
|||
const functionResult = json(
|
||||
await runCommand(
|
||||
cli,
|
||||
["function", process.execPath, "0x1000", "--json"],
|
||||
[
|
||||
"function",
|
||||
process.execPath,
|
||||
"0x1000",
|
||||
"--provider",
|
||||
"hopper",
|
||||
"--json",
|
||||
],
|
||||
environment,
|
||||
),
|
||||
);
|
||||
|
|
@ -53,7 +60,7 @@ export async function verifyPackagePlatform({
|
|||
const xrefs = json(
|
||||
await runCommand(
|
||||
cli,
|
||||
["xrefs", process.execPath, "0x1000", "--json"],
|
||||
["xrefs", process.execPath, "0x1000", "--provider", "hopper", "--json"],
|
||||
environment,
|
||||
),
|
||||
);
|
||||
|
|
@ -65,7 +72,14 @@ export async function verifyPackagePlatform({
|
|||
const trace = json(
|
||||
await runCommand(
|
||||
cli,
|
||||
["trace", process.execPath, "fixture", "--json"],
|
||||
[
|
||||
"trace",
|
||||
process.execPath,
|
||||
"fixture",
|
||||
"--provider",
|
||||
"hopper",
|
||||
"--json",
|
||||
],
|
||||
environment,
|
||||
),
|
||||
);
|
||||
|
|
|
|||
|
|
@ -0,0 +1,6 @@
|
|||
/** Validate a live registration and its byte-identical pre-setup backup. */
|
||||
export function assertClientConfig(
|
||||
cli: string,
|
||||
configPath: string,
|
||||
original: Uint8Array,
|
||||
): Promise<void>;
|
||||
|
|
@ -414,7 +414,8 @@ const verifySetupApply = async ({
|
|||
};
|
||||
};
|
||||
|
||||
const assertClientConfig = async (cli, configPath) => {
|
||||
/** Verify the live registration and exact pre-setup backup bytes. */
|
||||
export const assertClientConfig = async (cli, configPath, original) => {
|
||||
const config = json(await readFile(configPath, "utf8"));
|
||||
if (
|
||||
config.existing !== true ||
|
||||
|
|
@ -422,9 +423,7 @@ const assertClientConfig = async (cli, configPath) => {
|
|||
JSON.stringify(config.mcpServers?.rea?.args) !== JSON.stringify(["mcp"])
|
||||
)
|
||||
throw new Error("packaged client readback failed");
|
||||
if (
|
||||
!(await readFile(`${configPath}.rea.backup`, "utf8")).includes("existing")
|
||||
)
|
||||
if (!(await readFile(`${configPath}.rea.backup`)).equals(original))
|
||||
throw new Error("packaged client backup failed");
|
||||
};
|
||||
|
||||
|
|
@ -467,9 +466,7 @@ const assertCodexSymlink = async ({
|
|||
!codex.includes(`command = "${cli}"`) ||
|
||||
!codex.includes("startup_timeout_sec = 30") ||
|
||||
!codex.startsWith(CODEX_ORIGINAL) ||
|
||||
(await readFile(`${codexConfig}.rea.backup`, "utf8")) !== CODEX_ORIGINAL ||
|
||||
(await readFile(`${cursorConfig}.rea.backup`, "utf8")) !==
|
||||
'{"existing":true}\n'
|
||||
(await readFile(`${codexConfig}.rea.backup`, "utf8")) !== CODEX_ORIGINAL
|
||||
)
|
||||
throw new Error("packaged setup did not preserve config symlinks");
|
||||
};
|
||||
|
|
@ -534,9 +531,11 @@ const verifyConfigReadback = async ({
|
|||
siblingSkillPath,
|
||||
opencodeConfig,
|
||||
opencodeOriginal,
|
||||
claudeOriginal,
|
||||
cursorOriginal,
|
||||
}) => {
|
||||
await assertClientConfig(cli, claudeConfig);
|
||||
await assertClientConfig(cli, cursorConfig);
|
||||
await assertClientConfig(cli, claudeConfig, claudeOriginal);
|
||||
await assertClientConfig(cli, cursorConfig, cursorOriginal);
|
||||
await assertCodexSymlink({
|
||||
cli,
|
||||
codexConfig,
|
||||
|
|
@ -729,6 +728,9 @@ export async function verifyPackageSetup({
|
|||
opencodeConfig,
|
||||
skillPath,
|
||||
];
|
||||
const [claudeOriginal, cursorOriginal] = supportedSetupHost
|
||||
? await Promise.all([readFile(claudeConfig), readFile(cursorConfig)])
|
||||
: [];
|
||||
await verifyInteractiveSetup({
|
||||
command: packageRunnerCli,
|
||||
environment,
|
||||
|
|
@ -772,6 +774,8 @@ export async function verifyPackageSetup({
|
|||
siblingSkillPath,
|
||||
opencodeConfig,
|
||||
opencodeOriginal: OPENCODE_ORIGINAL,
|
||||
claudeOriginal,
|
||||
cursorOriginal,
|
||||
});
|
||||
await verifySetupFailureRecovery({
|
||||
cli,
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ import {
|
|||
requireMcpOperationResult,
|
||||
} from "./lib/mcp-verifier-results.mjs";
|
||||
import { verifyGhidraCompactReferences } from "./lib/real-ghidra-compact-references.mjs";
|
||||
import { verifyWindowsGhidraByteSearch } from "./lib/ghidra-windows-byte-search.mjs";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import {
|
||||
|
|
@ -358,11 +359,12 @@ try {
|
|||
const mapping = await call("address_to_file_offset", {
|
||||
address: procedure.address,
|
||||
});
|
||||
const sourceBytes = await readFile(target);
|
||||
assert.equal(memory.complete, true);
|
||||
assert.equal(memory.returned_bytes, 16);
|
||||
assert.equal(
|
||||
memory.bytes_hex,
|
||||
(await readFile(target))
|
||||
sourceBytes
|
||||
.subarray(mapping.file_offset, mapping.file_offset + 16)
|
||||
.toString("hex"),
|
||||
);
|
||||
|
|
@ -374,6 +376,35 @@ try {
|
|||
await call("resolve_containing_procedure", { address: procedure.address });
|
||||
await call("search_procedures", { pattern: procedure.value });
|
||||
await call("search_strings", { pattern: "rea" });
|
||||
report.byteSearch = await verifyWindowsGhidraByteSearch({
|
||||
invoke: call,
|
||||
sourceBytes,
|
||||
architecture: parsedTarget.value.architecture,
|
||||
sourceOwnedFixture: targetArgument === undefined,
|
||||
fallbackAddress: procedure.address,
|
||||
fallbackFileOffset: mapping.file_offset,
|
||||
runCliPage: async (query) => {
|
||||
const cliArguments = [entry, "search-bytes", target, query.pattern];
|
||||
for (const [option, value] of [
|
||||
["--mask", query.mask],
|
||||
["--start", query.start],
|
||||
["--end", query.end],
|
||||
["--limit", String(query.limit)],
|
||||
]) {
|
||||
if (value !== undefined) cliArguments.push(option, value);
|
||||
}
|
||||
cliArguments.push("--provider", "ghidra", "--json");
|
||||
const result = await exec(process.execPath, cliArguments, {
|
||||
env: environment,
|
||||
cwd: callerDirectory,
|
||||
timeout: 360_000,
|
||||
maxBuffer: 8 * 1024 * 1024,
|
||||
});
|
||||
const evidence = JSON.parse(result.stdout);
|
||||
assert.ok(evidence.error === undefined, "CLI byte search failed");
|
||||
return evidence.normalized_result;
|
||||
},
|
||||
});
|
||||
for (const name of [
|
||||
"procedure_assembly",
|
||||
"procedure_callees",
|
||||
|
|
@ -429,9 +460,24 @@ try {
|
|||
name !== "shutdown" &&
|
||||
!GHIDRA_MUTATING_OPERATIONS.has(name),
|
||||
);
|
||||
assert.ok(
|
||||
expected.every((name) => report.mcpOperations.includes(name)),
|
||||
"Packaged verification omitted an admitted operation.",
|
||||
const expectedOperations = new Set([
|
||||
...expected,
|
||||
"open_binary",
|
||||
"binary_overview",
|
||||
"binary_session",
|
||||
"close_binary",
|
||||
]);
|
||||
const observedOperations = new Set(report.mcpOperations);
|
||||
const missingOperations = expected.filter(
|
||||
(name) => !observedOperations.has(name),
|
||||
);
|
||||
const unexpectedOperations = [...observedOperations].filter(
|
||||
(name) => !expectedOperations.has(name),
|
||||
);
|
||||
assert.deepEqual(
|
||||
{ missing: missingOperations, unexpected: unexpectedOperations },
|
||||
{ missing: [], unexpected: [] },
|
||||
"Packaged verification operation coverage drifted.",
|
||||
);
|
||||
const session = await call("binary_session", {});
|
||||
assert.equal(
|
||||
|
|
|
|||
|
|
@ -50,6 +50,7 @@ import { verifyBrowserCaptureMetadataBudget } from "./lib/browser-capture-metada
|
|||
import { artifactCliEvidence, artifactMcpResult } from "./lib/artifact-e2e.mjs";
|
||||
import { verifyScenarioEnvironment } from "./lib/browser-scenario-environment-e2e.mjs";
|
||||
import { verifyScenarioStorage } from "./lib/browser-scenario-storage-e2e.mjs";
|
||||
import { verifyScenarioServiceWorkers } from "./lib/browser-scenario-service-workers-e2e.mjs";
|
||||
|
||||
const SCENARIO_SECRET_VALUE = "rea-browser-verifier-secret";
|
||||
const SCENARIO_URL_SECRET_VALUE = "rea-browser-url-verifier-secret";
|
||||
|
|
@ -299,6 +300,11 @@ try {
|
|||
endpoint,
|
||||
targetId: target,
|
||||
});
|
||||
process.stderr.write("Browser verifier: scenario service workers\n");
|
||||
const scenarioServiceWorkers = await verifyScenarioServiceWorkers(
|
||||
endpoint,
|
||||
target,
|
||||
);
|
||||
const launchedScenario = await createBrowserScenarioProvider(
|
||||
process.env,
|
||||
).captureScenario(
|
||||
|
|
@ -400,6 +406,7 @@ try {
|
|||
scenarioFailure,
|
||||
scenarioEnvironment,
|
||||
scenarioStorage,
|
||||
scenarioServiceWorkers,
|
||||
verified: true,
|
||||
};
|
||||
} finally {
|
||||
|
|
|
|||
|
|
@ -1,7 +1,14 @@
|
|||
#!/usr/bin/env node
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { copyFile, mkdir, mkdtemp, rm } from "node:fs/promises";
|
||||
import {
|
||||
copyFile,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
readFile,
|
||||
readdir,
|
||||
rm,
|
||||
} from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
|
@ -9,6 +16,11 @@ import { promisify } from "node:util";
|
|||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { inspectGhidraInstallation } from "../dist/ghidra/GhidraInstallation.js";
|
||||
import { cleanupOwnedProcessGroup } from "../dist/process/ProcessOwnership.js";
|
||||
import {
|
||||
ProviderProcessSupervisor,
|
||||
spawnOwnedProviderProcess,
|
||||
} from "../dist/process/ProviderProcess.js";
|
||||
import { toolContract } from "../dist/contracts/toolContracts.js";
|
||||
import { requireMcpOperationResult } from "./lib/mcp-verifier-results.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
|
|
@ -48,10 +60,21 @@ const source = fileURLToPath(
|
|||
const literalSource = fileURLToPath(
|
||||
new URL("../tests/conformance/swift/string-literals.swift", import.meta.url),
|
||||
);
|
||||
const unresolvedJumpSource = fileURLToPath(
|
||||
new URL(
|
||||
"../tests/conformance/swift/unresolved-string-jump.s",
|
||||
import.meta.url,
|
||||
),
|
||||
);
|
||||
const binary = join(root, "swift-inventory");
|
||||
const literalBinary = join(root, "swift-literals");
|
||||
const literalAssembly = fileURLToPath(
|
||||
new URL("../tests/conformance/swift/decoder-safeguards.S", import.meta.url),
|
||||
);
|
||||
const literalObject = join(root, "decoder-safeguards.o");
|
||||
const reports = [];
|
||||
let literalReport;
|
||||
let decoderCleaned = true;
|
||||
try {
|
||||
await execute("swiftc", ["-O", "-g", source, "-o", binary], {
|
||||
timeout: 60_000,
|
||||
|
|
@ -59,6 +82,19 @@ try {
|
|||
await mkdir(join(root, "copied"));
|
||||
const copied = join(root, "copied", "swift-inventory");
|
||||
await copyFile(binary, copied);
|
||||
await execute(
|
||||
"xcrun",
|
||||
[
|
||||
"clang",
|
||||
"-target",
|
||||
"arm64-apple-macosx13.0",
|
||||
"-c",
|
||||
literalAssembly,
|
||||
"-o",
|
||||
literalObject,
|
||||
],
|
||||
{ timeout: 60_000 },
|
||||
);
|
||||
// Decoding covers AArch64 register pairs, so build that slice on any Mac host.
|
||||
await execute(
|
||||
"swiftc",
|
||||
|
|
@ -67,11 +103,14 @@ try {
|
|||
"-target",
|
||||
"arm64-apple-macosx13.0",
|
||||
literalSource,
|
||||
literalObject,
|
||||
unresolvedJumpSource,
|
||||
"-o",
|
||||
literalBinary,
|
||||
],
|
||||
{ timeout: 60_000 },
|
||||
);
|
||||
await verifyDecoderSafeguards();
|
||||
await client.connect(transport);
|
||||
for (const [variant, path] of [
|
||||
["debug", binary],
|
||||
|
|
@ -154,7 +193,7 @@ try {
|
|||
} finally {
|
||||
await client.close();
|
||||
await transport.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
if (decoderCleaned) await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
console.log(
|
||||
JSON.stringify(
|
||||
|
|
@ -222,8 +261,144 @@ async function verifyStringLiterals() {
|
|||
}
|
||||
decoded[name] = observed.length;
|
||||
}
|
||||
const unresolvedJump = procedures.find(({ value }) =>
|
||||
value.includes("rea_unresolved_string_jump"),
|
||||
);
|
||||
assert(
|
||||
unresolvedJump,
|
||||
"unresolved jump fixture is missing from the procedure inventory",
|
||||
);
|
||||
const unresolvedDossier = await call("analyze_function", {
|
||||
procedure: unresolvedJump.address,
|
||||
});
|
||||
assert.deepEqual(unresolvedDossier.swift_string_literals, []);
|
||||
assert.deepEqual(
|
||||
unresolvedDossier.swift_string_literal_omissions.map(
|
||||
({ reason }) => reason,
|
||||
),
|
||||
["unresolved_jump"],
|
||||
);
|
||||
decoded.unresolvedStringJump =
|
||||
unresolvedDossier.swift_string_literal_omissions.length;
|
||||
return decoded;
|
||||
} finally {
|
||||
await call("close_binary", {});
|
||||
}
|
||||
}
|
||||
|
||||
// Exercise the Java producer inside an ephemeral Ghidra database. The explicit
|
||||
// budget seam makes exhaustion deterministic without exposing a runtime option.
|
||||
async function verifyDecoderSafeguards() {
|
||||
const scripts = join(root, "decoder-scripts");
|
||||
await mkdir(scripts);
|
||||
const bridgeSources = fileURLToPath(
|
||||
new URL("../bridge/ghidra/", import.meta.url),
|
||||
);
|
||||
for (const name of await readdir(bridgeSources)) {
|
||||
if (name.endsWith(".java"))
|
||||
await copyFile(join(bridgeSources, name), join(scripts, name));
|
||||
}
|
||||
await copyFile(
|
||||
fileURLToPath(
|
||||
new URL(
|
||||
"../tests/conformance/swift/ReaSwiftDecoderRegressions.java",
|
||||
import.meta.url,
|
||||
),
|
||||
),
|
||||
join(scripts, "ReaSwiftDecoderRegressions.java"),
|
||||
);
|
||||
const report = join(root, "decoder-safeguards.txt");
|
||||
await runDecoderProbe(scripts, report);
|
||||
assert.equal(
|
||||
await readFile(report, "utf8"),
|
||||
"REA_SWIFT_DECODER_SAFEGUARDS_OK\n",
|
||||
);
|
||||
}
|
||||
|
||||
async function runDecoderProbe(scripts, report) {
|
||||
const controller = new AbortController();
|
||||
let interrupt;
|
||||
const interrupted = new Promise((resolve) => {
|
||||
interrupt = () => {
|
||||
controller.abort(new Error("Swift decoder probe interrupted"));
|
||||
resolve(false);
|
||||
};
|
||||
});
|
||||
process.on("SIGINT", interrupt);
|
||||
process.on("SIGTERM", interrupt);
|
||||
let supervisor;
|
||||
let launch;
|
||||
try {
|
||||
launch = await spawnOwnedProviderProcess({
|
||||
command: installation.analyzeHeadlessPath,
|
||||
arguments: [
|
||||
root,
|
||||
"decoder-regressions",
|
||||
"-import",
|
||||
literalBinary,
|
||||
"-scriptPath",
|
||||
scripts,
|
||||
"-postScript",
|
||||
"ReaSwiftDecoderRegressions.java",
|
||||
report,
|
||||
"-deleteProject",
|
||||
"-max-cpu",
|
||||
"2",
|
||||
],
|
||||
runId: run.run_id,
|
||||
expectedCommand: null,
|
||||
env: { JAVA_HOME: installation.javaHome },
|
||||
signal: controller.signal,
|
||||
});
|
||||
decoderCleaned = false;
|
||||
supervisor = new ProviderProcessSupervisor(
|
||||
{
|
||||
...launch,
|
||||
ownsProcessLifetime: true,
|
||||
cleanup: () => cleanupOwnedProcessGroup(launch.ownership),
|
||||
},
|
||||
{ maxDiagnosticBytes: 8 * 1024 * 1024 },
|
||||
);
|
||||
assert.ok(
|
||||
await Promise.race([supervisor.waitForExit(120_000), interrupted]),
|
||||
controller.signal.aborted
|
||||
? "Swift decoder probe interrupted"
|
||||
: "Swift decoder probe timed out",
|
||||
);
|
||||
assert.ok(
|
||||
await supervisor.waitForOutputClose(1_000),
|
||||
"Swift decoder probe output did not close",
|
||||
);
|
||||
const snapshot = supervisor.snapshot();
|
||||
assert.equal(
|
||||
snapshot.exitCode,
|
||||
0,
|
||||
`${snapshot.stdout.text}\n${snapshot.stderr.text}`,
|
||||
);
|
||||
assert.ok(
|
||||
!snapshot.diagnosticTruncated,
|
||||
"Swift decoder probe exceeded its diagnostic capacity",
|
||||
);
|
||||
} finally {
|
||||
try {
|
||||
if (supervisor !== undefined) {
|
||||
const stopped = await supervisor.stop();
|
||||
decoderCleaned = stopped.status === "verified-cleanup";
|
||||
assert.ok(
|
||||
decoderCleaned,
|
||||
`Owned Ghidra cleanup incomplete; retained ${root}: ${JSON.stringify(stopped)}`,
|
||||
);
|
||||
} else if (launch !== undefined) {
|
||||
const stopped = await cleanupOwnedProcessGroup(launch.ownership);
|
||||
decoderCleaned = stopped.cleaned;
|
||||
assert.ok(
|
||||
decoderCleaned,
|
||||
`Owned Ghidra cleanup incomplete; retained ${root}: ${JSON.stringify(stopped)}`,
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
process.off("SIGINT", interrupt);
|
||||
process.off("SIGTERM", interrupt);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
|
|||
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
|
||||
import { verifyGhidraCompactReferences } from "./lib/real-ghidra-compact-references.mjs";
|
||||
import { verifyGhidraFileOffsets } from "./lib/real-ghidra-file-offsets.mjs";
|
||||
import { verifyWindowsGhidraByteSearch } from "./lib/ghidra-windows-byte-search.mjs";
|
||||
|
||||
const verifierRun = createVerifierRun();
|
||||
const SESSION_CAPABILITIES = GHIDRA_SESSION_CAPABILITIES.filter(
|
||||
|
|
@ -215,6 +216,11 @@ try {
|
|||
});
|
||||
await inventory("search_procedures", { pattern: procedure.value });
|
||||
await inventory("search_strings", { pattern: "rea" });
|
||||
const byteSearch = await verifyWindowsGhidraByteSearch({
|
||||
invoke: inventory,
|
||||
sourceBytes,
|
||||
architecture,
|
||||
});
|
||||
|
||||
for (const operation of [
|
||||
"procedure_assembly",
|
||||
|
|
@ -242,12 +248,18 @@ try {
|
|||
...GHIDRA_INVENTORY_OPERATIONS,
|
||||
...GHIDRA_FUNCTION_OPERATIONS,
|
||||
].filter((name) => !GHIDRA_MUTATING_OPERATIONS.has(name));
|
||||
const expectedOperations = new Set(expected);
|
||||
const missingOperations = expected.filter((name) => !observed.has(name));
|
||||
const unexpectedOperations = [...observed].filter(
|
||||
(name) => !expectedOperations.has(name),
|
||||
);
|
||||
if (
|
||||
observed.size !== expected.length ||
|
||||
expected.some((operation) => !observed.has(operation))
|
||||
missingOperations.length > 0 ||
|
||||
unexpectedOperations.length > 0 ||
|
||||
observed.size !== expectedOperations.size
|
||||
)
|
||||
throw new Error(
|
||||
`Windows Ghidra operation proof was incomplete: ${JSON.stringify([...observed])}`,
|
||||
`Windows Ghidra operation proof was incomplete: ${JSON.stringify({ missing: missingOperations, unexpected: unexpectedOperations })}`,
|
||||
);
|
||||
|
||||
// The reference controls occupy the EXE fixture's additional function.
|
||||
|
|
@ -271,6 +283,7 @@ try {
|
|||
},
|
||||
transport: "authenticated-loopback-tcp",
|
||||
operations: [...observed].sort((left, right) => left.localeCompare(right)),
|
||||
byte_search: byteSearch,
|
||||
cleanup: "complete",
|
||||
compact_references: compactReferences,
|
||||
native_controls: {
|
||||
|
|
|
|||
|
|
@ -1,11 +1,12 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import {
|
||||
access,
|
||||
copyFile,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
readFile,
|
||||
readdir,
|
||||
rmdir,
|
||||
rm,
|
||||
stat,
|
||||
symlink,
|
||||
|
|
@ -54,7 +55,7 @@ try {
|
|||
join(workspace, "dist", "windows", "WindowsNativeLoader.js"),
|
||||
).href;
|
||||
const source = `import assert from 'node:assert/strict';
|
||||
import {existsSync,mkdirSync,readdirSync,renameSync,statSync} from 'node:fs';
|
||||
import {existsSync,mkdirSync,readFileSync,readdirSync,renameSync,statSync,writeFileSync} from 'node:fs';
|
||||
import {join} from 'node:path';
|
||||
import {systemWindowsNativeAuthority} from ${JSON.stringify(loader)};
|
||||
const temp=process.env.TEMP;
|
||||
|
|
@ -64,46 +65,61 @@ assert.equal(first.retryable,true);
|
|||
assert.match(first.reason,/Delete owned object by handle failed.*Win32 5/s);
|
||||
const [name,...extra]=readdirSync(temp);assert.ok(name);assert.equal(extra.length,0);
|
||||
const path=join(temp,name);const identity=statSync(path,{bigint:true});
|
||||
// Ownership follows the retained NTFS handle even when the directory moves.
|
||||
// A replacement at its old pathname must remain caller-owned.
|
||||
const moved=path+'-renamed';renameSync(path,moved);
|
||||
assert.equal(statSync(moved,{bigint:true}).ino,identity.ino);
|
||||
// Delete sharing permits rename; ownership follows the retained NTFS handle.
|
||||
writeFileSync(join(path,'owned-child.txt'),'owned child');
|
||||
const renamed=path+'-renamed';renameSync(path,renamed);
|
||||
const moved=statSync(renamed,{bigint:true});
|
||||
assert.equal(moved.dev,identity.dev);assert.equal(moved.ino,identity.ino);
|
||||
mkdirSync(path);const replacement=statSync(path,{bigint:true});
|
||||
assert.notEqual(replacement.ino,identity.ino);
|
||||
const sentinel=join(path,'unrelated.txt');writeFileSync(sentinel,'unrelated replacement');
|
||||
if(process.argv[2]!=='teardown'){
|
||||
const second=systemWindowsNativeAuthority();assert.equal(second.available,true);
|
||||
assert.equal(existsSync(moved),false);
|
||||
assert.equal(existsSync(renamed),false);
|
||||
assert.deepEqual(readdirSync(temp),[name]);
|
||||
assert.equal(readFileSync(sentinel,'utf8'),'unrelated replacement');
|
||||
assert.equal(statSync(path,{bigint:true}).ino,replacement.ino);
|
||||
assert.equal(systemWindowsNativeAuthority(),second);
|
||||
}
|
||||
console.log(JSON.stringify({mode:process.argv[2],firstFailed:true,retainedOwner:true,inode:identity.ino.toString(),replacementName:name,replacementInode:replacement.ino.toString()}));
|
||||
console.log(JSON.stringify({mode:process.argv[2],firstFailed:true,retainedOwner:true,name,renamed,device:identity.dev.toString(),inode:identity.ino.toString(),replacementInode:replacement.ino.toString()}));
|
||||
`;
|
||||
const fixture = join(workspace, "probe.mjs");
|
||||
await writeFile(fixture, source);
|
||||
for (const mode of ["retry", "teardown"]) {
|
||||
const temp = join(workspace, mode);
|
||||
await mkdir(temp);
|
||||
const observed = JSON.parse(
|
||||
execFileSync(process.execPath, [fixture, mode], {
|
||||
encoding: "utf8",
|
||||
timeout: 20_000,
|
||||
windowsHide: true,
|
||||
env: { ...process.env, TEMP: temp, TMP: temp },
|
||||
}),
|
||||
);
|
||||
assert.equal(observed.retainedOwner, true);
|
||||
assert.deepEqual(await readdir(temp), [observed.replacementName]);
|
||||
const replacement = join(temp, observed.replacementName);
|
||||
const child = spawnSync(process.execPath, [fixture, mode], {
|
||||
encoding: "utf8",
|
||||
timeout: 20_000,
|
||||
windowsHide: true,
|
||||
env: { ...process.env, TEMP: temp, TMP: temp },
|
||||
});
|
||||
if (child.error !== undefined) throw child.error;
|
||||
assert.equal(child.status, 0, child.stderr);
|
||||
assert.equal(
|
||||
(await stat(replacement, { bigint: true })).ino.toString(),
|
||||
observed.replacementInode,
|
||||
child.stderr,
|
||||
"",
|
||||
"Native environment teardown reported a cleanup failure",
|
||||
);
|
||||
const observed = JSON.parse(child.stdout);
|
||||
assert.equal(observed.mode, mode);
|
||||
assert.equal(observed.firstFailed, true);
|
||||
assert.equal(observed.retainedOwner, true);
|
||||
await assert.rejects(access(observed.renamed), { code: "ENOENT" });
|
||||
assert.deepEqual(await readdir(temp), [observed.name]);
|
||||
const replacement = join(temp, observed.name);
|
||||
const identity = await stat(replacement, { bigint: true });
|
||||
assert.equal(identity.dev.toString(), observed.device);
|
||||
assert.equal(identity.ino.toString(), observed.replacementInode);
|
||||
assert.notEqual(identity.ino.toString(), observed.inode);
|
||||
assert.deepEqual(await readdir(replacement), ["unrelated.txt"]);
|
||||
assert.equal(
|
||||
await readFile(join(replacement, "unrelated.txt"), "utf8"),
|
||||
"unrelated replacement",
|
||||
);
|
||||
await rmdir(replacement);
|
||||
assert.deepEqual(await readdir(temp), []);
|
||||
}
|
||||
process.stdout.write(
|
||||
`${JSON.stringify({ ok: true, controlledDispositionFailure: true, retainedOwner: true, renamedOwnerCleanup: true, replacementPreserved: true, retryRecovers: true, successfulAdmissionCached: true, environmentTeardownCleansProbe: true, issue1701HostCauseReproduced: false })}\n`,
|
||||
`${JSON.stringify({ ok: true, controlledDispositionFailure: true, retainedOwner: true, renamedOwnerCleansChildren: true, unrelatedReplacementPreserved: true, retryRecovers: true, successfulAdmissionCached: true, environmentTeardownCleansProbe: true, issue1701HostCauseReproduced: false })}\n`,
|
||||
);
|
||||
} finally {
|
||||
await rm(workspace, { recursive: true, force: true });
|
||||
|
|
|
|||
|
|
@ -0,0 +1,21 @@
|
|||
export interface OpenAiModelFixture {
|
||||
readonly baseUrl: string;
|
||||
readonly requests: readonly { readonly path: string }[];
|
||||
readonly probes: readonly {
|
||||
readonly path: string | undefined;
|
||||
readonly status: 404;
|
||||
}[];
|
||||
readonly failure: string | undefined;
|
||||
close(): Promise<void>;
|
||||
}
|
||||
|
||||
export function createOpenAiModelFixture(options: {
|
||||
readonly directory: string;
|
||||
readonly prefix: string;
|
||||
readonly onRequest: () =>
|
||||
| {
|
||||
readonly role: "assistant";
|
||||
readonly content: string;
|
||||
}
|
||||
| Promise<{ readonly role: "assistant"; readonly content: string }>;
|
||||
}): Promise<OpenAiModelFixture>;
|
||||
|
|
@ -15,7 +15,8 @@ export const createOpenAiModelFixture = async ({
|
|||
let failure;
|
||||
const server = createServer(async (request, response) => {
|
||||
try {
|
||||
if (request.method === "GET" && request.url?.endsWith("/models")) {
|
||||
const path = new URL(request.url ?? "/", "http://localhost").pathname;
|
||||
if (request.method === "GET" && path.endsWith("/models")) {
|
||||
response.setHeader("Content-Type", "application/json");
|
||||
response.end(
|
||||
JSON.stringify({
|
||||
|
|
@ -47,7 +48,7 @@ export const createOpenAiModelFixture = async ({
|
|||
join(directory, `${prefix}-incoming-${++incoming}.json`),
|
||||
JSON.stringify({ method: request.method, path: request.url, body }),
|
||||
);
|
||||
if (request.url !== "/v1/chat/completions") {
|
||||
if (path !== "/v1/chat/completions") {
|
||||
// Hermes probes custom endpoints for Ollama/llama.cpp metadata.
|
||||
// Unsupported HTTP endpoints return 404, as on an OpenAI-only server.
|
||||
probes.push({
|
||||
|
|
|
|||
|
|
@ -201,6 +201,16 @@ async function stageCli() {
|
|||
),
|
||||
);
|
||||
|
||||
assert.ok(
|
||||
files.extraction_gaps.some(
|
||||
(gap) =>
|
||||
gap.kind === "unknown-chunk-handler" &&
|
||||
gap.range !== null &&
|
||||
gap.range.offset === 0 &&
|
||||
gap.range.length === 64,
|
||||
),
|
||||
"The unknown chunk must carry its explicit recovery row",
|
||||
);
|
||||
return { regions, files };
|
||||
}
|
||||
|
||||
|
|
@ -247,6 +257,20 @@ async function stageMcp(cli) {
|
|||
);
|
||||
assert.equal(depthParsed.coverage, "partial");
|
||||
assert.ok(depthParsed.depth_limited_paths.length > 0);
|
||||
assert.ok(
|
||||
depthParsed.extraction_gaps.some(
|
||||
(gap) =>
|
||||
gap.kind === "depth-limited" &&
|
||||
gap.dependencies.length === 0 &&
|
||||
depthParsed.extraction_continuations.some(
|
||||
({ id, instruction }) =>
|
||||
id === gap.continuation_id &&
|
||||
instruction.includes("max_depth") &&
|
||||
instruction.includes("fresh output_directory"),
|
||||
),
|
||||
),
|
||||
"Depth-limited paths must carry the explicit depth continuation",
|
||||
);
|
||||
|
||||
console.log(
|
||||
"PASS real Binwalk/Unblob: CLI, MCP, independent offsets/digests, range extraction, unknown chunks and depth limits",
|
||||
|
|
@ -292,8 +316,17 @@ async function stageExt4() {
|
|||
);
|
||||
assert.equal(degradedParsed.coverage, "partial");
|
||||
assert.ok(JSON.stringify(degradedParsed.diagnostics).includes("debugfs"));
|
||||
assert.ok(
|
||||
degradedParsed.extraction_gaps.some(
|
||||
(gap) =>
|
||||
gap.kind === "missing-extractor-dependency" &&
|
||||
gap.dependencies.includes("debugfs") &&
|
||||
gap.handler === "extfs",
|
||||
),
|
||||
"The missing extractor must be named in its explicit recovery row",
|
||||
);
|
||||
console.log(
|
||||
"PASS real missing-extractor diagnostics retained despite Unblob exit 1",
|
||||
"PASS real missing-extractor diagnostics and recovery row retained despite Unblob exit 1",
|
||||
);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -192,6 +192,25 @@ export const parsePackageListOutput = (
|
|||
return { packages, unparsedLines };
|
||||
};
|
||||
|
||||
/** Quote raw device paths for prose without allowing bidi or invisible text. */
|
||||
export const displayDevicePath = (devicePath: string): string => {
|
||||
let excerpt = "";
|
||||
let count = 0;
|
||||
for (const character of devicePath) {
|
||||
if (count === 160) break;
|
||||
excerpt += character;
|
||||
count += 1;
|
||||
}
|
||||
const quoted = JSON.stringify(excerpt).replace(
|
||||
/[\p{C}\p{Z}\p{Default_Ignorable_Code_Point}\u2800]/gu,
|
||||
(character) =>
|
||||
character === " "
|
||||
? character
|
||||
: `\\u{${character.codePointAt(0)!.toString(16).toUpperCase()}}`,
|
||||
);
|
||||
return excerpt.length === devicePath.length ? quoted : `${quoted} (excerpt)`;
|
||||
};
|
||||
|
||||
/** One APK of an installed package exactly as `pm path` reports it. */
|
||||
export interface AdbPackagePathEntry {
|
||||
readonly device_path: string;
|
||||
|
|
|
|||
|
|
@ -240,11 +240,12 @@ const shellStub = (
|
|||
): ((root: string) => string) => {
|
||||
const cases = Object.entries(dispatch)
|
||||
.map(([key, output]) => {
|
||||
const [first, second] = key.split(":");
|
||||
const guard =
|
||||
second === undefined
|
||||
? `[ "$4" = "${first}" ]`
|
||||
: `[ "$4" = "${first}" ] && [ "$5" = "${second}" ]`;
|
||||
const [first, second, third] = key.split(":");
|
||||
const guard = [
|
||||
`[ "$4" = "${first}" ]`,
|
||||
...(second === undefined ? [] : [`[ "$5" = "${second}" ]`]),
|
||||
...(third === undefined ? [] : [`[ "$6" = "${third}" ]`]),
|
||||
].join(" && ");
|
||||
return ` if ${guard}; then printf '%b' '${output.replaceAll("\\", "\\\\").replaceAll("\0", "\\000").replace(/'/gu, "'\\''")}'; exit 0; fi`;
|
||||
})
|
||||
.join("\n");
|
||||
|
|
@ -379,6 +380,30 @@ exit 1`,
|
|||
expect(result.error.capturedOutput?.stdout).toContain(
|
||||
pmOutput.split("\n")[0],
|
||||
);
|
||||
const projected = projectAnalysisError(result.error);
|
||||
const duplicate = pmOutput.includes("\n");
|
||||
expect(projected).toMatchObject({
|
||||
code: "unreadable_output",
|
||||
message: expect.stringContaining("pm path for com.example"),
|
||||
remediation: { action: expect.stringContaining("details.diagnostics") },
|
||||
details: {
|
||||
reason: duplicate ? "duplicate_local_destination" : "unsafe_device_path",
|
||||
diagnostics: {
|
||||
package_name: "com.example",
|
||||
...(duplicate
|
||||
? { device_paths: ["/data/app/x/base.apk", "/data/app/y/base.apk"] }
|
||||
: {
|
||||
rejected_paths: [
|
||||
{
|
||||
device_path: "/data/app/x/",
|
||||
reason: "empty_file_name",
|
||||
},
|
||||
],
|
||||
}),
|
||||
},
|
||||
captured_output: { stdout: expect.stringContaining("package:") },
|
||||
},
|
||||
});
|
||||
await expect(readFile(join(root, "pull-ran"))).rejects.toThrow();
|
||||
await expect(lstat(output)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
});
|
||||
|
|
@ -410,7 +435,15 @@ it.each([
|
|||
if (aliases) {
|
||||
expect(result.ok).toBe(false);
|
||||
if (result.ok) return;
|
||||
expect(result.error.message).toContain("unique local destinations");
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||
message: expect.stringContaining("unique local destination"),
|
||||
details: {
|
||||
reason: "local_destination_unavailable",
|
||||
diagnostics: {
|
||||
device_path: `/data/app/y/${secondName}`,
|
||||
},
|
||||
},
|
||||
});
|
||||
await expect(readFile(pullLog)).rejects.toThrow();
|
||||
await expect(lstat(packageDirectory)).rejects.toThrow();
|
||||
return;
|
||||
|
|
@ -1015,6 +1048,60 @@ it("inspects the display and window focus", async () => {
|
|||
);
|
||||
});
|
||||
|
||||
it("reads Android 16 focus declarations from displays when windows omits them", async () => {
|
||||
const { binary } = await writeAdbStub(
|
||||
shellStub({
|
||||
"dumpsys:window:windows": "Window state has no focus fields\\n",
|
||||
"dumpsys:window:displays":
|
||||
"mCurrentFocus=Window{abc u0 com.example/.Main}\nmFocusedApp=ActivityRecord{def u0 com.example/.Main t1}\n",
|
||||
}),
|
||||
);
|
||||
const result = await evidenceResult(providerOver(binary), {
|
||||
operation: "inspect_adb_window",
|
||||
input: { serial: "emulator-5554" },
|
||||
});
|
||||
expect(result).toMatchObject({
|
||||
focused_window: "Window{abc u0 com.example/.Main}",
|
||||
focused_app: "ActivityRecord{def u0 com.example/.Main t1}",
|
||||
observed_lines: [
|
||||
"Window{abc u0 com.example/.Main}",
|
||||
"ActivityRecord{def u0 com.example/.Main t1}",
|
||||
],
|
||||
});
|
||||
const execution = await providerOver(binary).execute({
|
||||
operation: "inspect_adb_window",
|
||||
input: { serial: "emulator-5554" },
|
||||
});
|
||||
expect(execution.ok).toBe(true);
|
||||
if (!execution.ok) throw execution.error;
|
||||
expect(execution.value.limitations).toContainEqual(
|
||||
expect.stringContaining("dumpsys window displays"),
|
||||
);
|
||||
});
|
||||
|
||||
it("describes missing window focus as unknown after both sections omit declarations", async () => {
|
||||
const { binary } = await writeAdbStub(
|
||||
shellStub({
|
||||
"dumpsys:window:windows": "no focus\\n",
|
||||
"dumpsys:window:displays": "no focus\\n",
|
||||
}),
|
||||
);
|
||||
const result = await evidenceResult(providerOver(binary), {
|
||||
operation: "inspect_adb_window",
|
||||
input: { serial: "emulator-5554" },
|
||||
});
|
||||
expect(result).toMatchObject({ focused_window: null, focused_app: null });
|
||||
const execution = await providerOver(binary).execute({
|
||||
operation: "inspect_adb_window",
|
||||
input: { serial: "emulator-5554" },
|
||||
});
|
||||
expect(execution.ok).toBe(true);
|
||||
if (!execution.ok) throw execution.error;
|
||||
expect(execution.value.limitations).toContainEqual(
|
||||
expect.stringContaining("focus is unknown"),
|
||||
);
|
||||
});
|
||||
|
||||
it("reads one settings value and reports device-side nulls", async () => {
|
||||
const { binary } = await writeAdbStub(shellStub({ "settings:get": "1\n" }));
|
||||
const enabled = await evidenceResult(providerOver(binary), {
|
||||
|
|
|
|||
|
|
@ -56,6 +56,7 @@ import {
|
|||
type AdbClientIdentity,
|
||||
} from "./AdbConfiguration.js";
|
||||
import {
|
||||
displayDevicePath,
|
||||
parseAdbDevicesOutput,
|
||||
parseGetpropOutput,
|
||||
parsePackageListOutput,
|
||||
|
|
@ -417,8 +418,12 @@ const removeOwnedOutputDirectory = async (
|
|||
|
||||
const reservePackageDestinations = async (
|
||||
operation: string,
|
||||
packageName: string,
|
||||
packageDirectory: string,
|
||||
entries: readonly { readonly file_name: string }[],
|
||||
entries: readonly {
|
||||
readonly file_name: string;
|
||||
readonly device_path: string;
|
||||
}[],
|
||||
capturedOutput: {
|
||||
readonly stdout: string;
|
||||
readonly stderr: string;
|
||||
|
|
@ -427,6 +432,8 @@ const reservePackageDestinations = async (
|
|||
): Promise<AnalysisOutputError | undefined> => {
|
||||
const reservations: {
|
||||
readonly path: string;
|
||||
readonly file_name: string;
|
||||
readonly device_path: string;
|
||||
readonly identity: OutputDirectoryIdentity;
|
||||
}[] = [];
|
||||
for (const entry of entries) {
|
||||
|
|
@ -438,6 +445,8 @@ const reservePackageDestinations = async (
|
|||
throw new Error(`Could not reserve local filename ${entry.file_name}`);
|
||||
reservations.push({
|
||||
path,
|
||||
file_name: entry.file_name,
|
||||
device_path: entry.device_path,
|
||||
identity: {
|
||||
dev: created.dev,
|
||||
ino: created.ino,
|
||||
|
|
@ -445,10 +454,28 @@ const reservePackageDestinations = async (
|
|||
},
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
const errorCode =
|
||||
cause instanceof Error &&
|
||||
"code" in cause &&
|
||||
typeof cause.code === "string"
|
||||
? cause.code
|
||||
: null;
|
||||
return new AnalysisOutputError(
|
||||
operation,
|
||||
`Package Manager paths cannot be written to unique local destinations at ${JSON.stringify(entry.file_name)}; refusing before pulling any files`,
|
||||
{ cause, capturedOutput },
|
||||
"local_destination_unavailable",
|
||||
{
|
||||
cause,
|
||||
capturedOutput,
|
||||
userMessage: `pm path for ${packageName} reported ${displayDevicePath(entry.device_path)}, whose file name ${displayDevicePath(entry.file_name)} cannot be written to a unique local destination (${errorCode ?? "unknown error"}). No APK was pulled.`,
|
||||
remediationAction:
|
||||
"Review details.diagnostics and details.captured_output for the refused destination. Correct the output filesystem conflict before retrying; REA does not rename APKs.",
|
||||
diagnostics: {
|
||||
package_name: packageName,
|
||||
device_path: entry.device_path,
|
||||
file_name: entry.file_name,
|
||||
error_code: errorCode,
|
||||
},
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -467,10 +494,28 @@ const reservePackageDestinations = async (
|
|||
);
|
||||
await rmdir(reservation.path);
|
||||
} catch (cause: unknown) {
|
||||
const errorCode =
|
||||
cause instanceof Error &&
|
||||
"code" in cause &&
|
||||
typeof cause.code === "string"
|
||||
? cause.code
|
||||
: null;
|
||||
return new AnalysisOutputError(
|
||||
operation,
|
||||
`Could not release a local filename reservation before pulling: ${reservation.path}`,
|
||||
{ cause, capturedOutput },
|
||||
"local_destination_unavailable",
|
||||
{
|
||||
cause,
|
||||
capturedOutput,
|
||||
userMessage: `Could not release the local filename reservation for ${displayDevicePath(reservation.device_path)} (${errorCode ?? "unknown error"}); no APK was pulled.`,
|
||||
remediationAction:
|
||||
"Review details.diagnostics and details.captured_output for the refused destination. Correct the output filesystem conflict before retrying; REA does not rename APKs.",
|
||||
diagnostics: {
|
||||
package_name: packageName,
|
||||
device_path: reservation.device_path,
|
||||
file_name: reservation.file_name,
|
||||
error_code: errorCode,
|
||||
},
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -1095,14 +1140,27 @@ export class AdbProvider {
|
|||
);
|
||||
}
|
||||
case "inspect_adb_window": {
|
||||
const ran = await shell(["dumpsys", "window", "windows"], {
|
||||
let section = "windows";
|
||||
const ran = await shell(["dumpsys", "window", section], {
|
||||
timeoutMs: SHELL_TIMEOUT_MS,
|
||||
maxOutputBytes: DUMPSYS_WINDOW_MAX_OUTPUT_BYTES,
|
||||
});
|
||||
if (!ran.ok) return ran;
|
||||
const { focusedWindow, focusedApp } = parseWindowFocusOutput(
|
||||
let { focusedWindow, focusedApp } = parseWindowFocusOutput(
|
||||
ran.value.stdout,
|
||||
);
|
||||
if (focusedWindow === null && focusedApp === null) {
|
||||
section = "displays";
|
||||
const fallback = await shell(["dumpsys", "window", section], {
|
||||
timeoutMs: SHELL_TIMEOUT_MS,
|
||||
maxOutputBytes: DUMPSYS_WINDOW_MAX_OUTPUT_BYTES,
|
||||
});
|
||||
if (!fallback.ok) return fallback;
|
||||
({ focusedWindow, focusedApp } = parseWindowFocusOutput(
|
||||
fallback.value.stdout,
|
||||
));
|
||||
}
|
||||
const declared = focusedWindow !== null || focusedApp !== null;
|
||||
return ok(
|
||||
createAnalysisExecution(
|
||||
{
|
||||
|
|
@ -1119,6 +1177,9 @@ export class AdbProvider {
|
|||
{
|
||||
limitations: [
|
||||
"Focus values are the window manager's declarations at read time; no interaction occurred",
|
||||
declared
|
||||
? `Focus declarations were read from \`dumpsys window ${section}\``
|
||||
: "Neither `dumpsys window windows` nor `dumpsys window displays` declared mCurrentFocus or mFocusedApp; focus is unknown, not absent",
|
||||
],
|
||||
},
|
||||
),
|
||||
|
|
@ -1269,14 +1330,27 @@ export class AdbProvider {
|
|||
stderr: paths.value.stderr,
|
||||
truncated: false,
|
||||
};
|
||||
if (parsed.rejectedPaths.length > 0)
|
||||
if (parsed.rejectedPaths.length > 0) {
|
||||
const first = parsed.rejectedPaths[0]!;
|
||||
const otherCount = parsed.rejectedPaths.length - 1;
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
operation,
|
||||
`Package Manager reported paths that cannot be represented as host files: ${parsed.rejectedPaths.map((entry) => `${JSON.stringify(entry.device_path)} (${entry.reason})`).join(", ")}`,
|
||||
{ capturedOutput },
|
||||
),
|
||||
new AnalysisOutputError(operation, "unsafe_device_path", {
|
||||
capturedOutput,
|
||||
userMessage: `pm path for ${packageName} reported an APK path that cannot be written as a file on this host: ${displayDevicePath(first.device_path)} (${first.reason})${otherCount > 0 ? ` and ${String(otherCount)} other path(s)` : ""}. No APK was pulled.`,
|
||||
remediationAction:
|
||||
"Review details.diagnostics for the refused device paths and details.captured_output for the retained pm path output. REA does not rename device-reported APK paths; correct the device path policy before retrying.",
|
||||
diagnostics: {
|
||||
package_name: packageName,
|
||||
rejected_paths: parsed.rejectedPaths.map(
|
||||
({ device_path, reason }) => ({
|
||||
device_path,
|
||||
reason,
|
||||
}),
|
||||
),
|
||||
},
|
||||
}),
|
||||
);
|
||||
}
|
||||
if (parsed.paths.length === 0 && parsed.unparsedLines.length === 0)
|
||||
return err(
|
||||
new AnalysisUnsupportedTargetError(
|
||||
|
|
@ -1296,11 +1370,19 @@ export class AdbProvider {
|
|||
for (const entry of parsed.paths) {
|
||||
if (destinationNames.has(entry.file_name))
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
operation,
|
||||
`Package Manager reported multiple paths with the same local destination ${JSON.stringify(entry.file_name)}; refusing before pulling any files`,
|
||||
{ capturedOutput },
|
||||
),
|
||||
new AnalysisOutputError(operation, "duplicate_local_destination", {
|
||||
capturedOutput,
|
||||
userMessage: `pm path for ${packageName} reported multiple APK paths with the same local file name ${displayDevicePath(entry.file_name)}; no APK was pulled.`,
|
||||
remediationAction:
|
||||
"Review details.diagnostics and details.captured_output for the conflicting device paths. REA does not rename device-reported paths; correct Package Manager's path response before retrying.",
|
||||
diagnostics: {
|
||||
package_name: packageName,
|
||||
file_name: entry.file_name,
|
||||
device_paths: parsed.paths
|
||||
.filter((candidate) => candidate.file_name === entry.file_name)
|
||||
.map((candidate) => candidate.device_path),
|
||||
},
|
||||
}),
|
||||
);
|
||||
destinationNames.add(entry.file_name);
|
||||
}
|
||||
|
|
@ -1395,6 +1477,7 @@ export class AdbProvider {
|
|||
};
|
||||
const destinationError = await reservePackageDestinations(
|
||||
operation,
|
||||
packageName,
|
||||
packageDirectory,
|
||||
parsed.paths,
|
||||
capturedOutput,
|
||||
|
|
|
|||
|
|
@ -172,7 +172,7 @@ export const parseWmDensityOutput = (
|
|||
return { physical, override };
|
||||
};
|
||||
|
||||
/** Parse the focus declarations from `dumpsys window windows`. */
|
||||
/** Parse focus declarations from `dumpsys window windows` or `displays`. */
|
||||
export const parseWindowFocusOutput = (
|
||||
stdout: string,
|
||||
): {
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
import { constants } from "node:fs";
|
||||
import { access, realpath, stat } from "node:fs/promises";
|
||||
import { delimiter, isAbsolute, join, resolve } from "node:path";
|
||||
import { inspectWindowsNativeReadiness } from "../process/WindowsAuthority.js";
|
||||
import { systemWindowsNativeAuthority } from "../windows/WindowsNativeLoader.js";
|
||||
import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore.js";
|
||||
import type { AndroidOperation } from "../domain/android/androidAnalysis.js";
|
||||
|
|
@ -11,7 +12,15 @@ import {
|
|||
execFileOutput,
|
||||
execFileOutputFailure,
|
||||
} from "../process/ExecFileOutput.js";
|
||||
import { JADX_JAR_CONFIGURATION_REMEDIATION } from "./JadxRelease.js";
|
||||
import type { DoctorProviderCheck } from "../application/Doctor.js";
|
||||
import {
|
||||
JADX_JAR_CONFIGURATION_REMEDIATION,
|
||||
JADX_JAVA_PROBE_REMEDIATION,
|
||||
JADX_JAVA_RUNTIME_REMEDIATION,
|
||||
JADX_JVM_OPTIONS_REMEDIATION,
|
||||
JADX_UNSUPPORTED_HOST_REMEDIATION,
|
||||
JADX_WINDOWS_NATIVE_REMEDIATION,
|
||||
} from "./JadxRelease.js";
|
||||
|
||||
const JAVA_MODULE_LISTING_TIMEOUT_MS = 10_000;
|
||||
const JAVA_MODULE_LISTING_MAX_OUTPUT_BYTES = 1024 * 1024;
|
||||
|
|
@ -30,16 +39,110 @@ type ConfigurationFailureCode =
|
|||
| "not_configured"
|
||||
| "executable_missing"
|
||||
| "runtime_missing"
|
||||
| "runtime_unavailable"
|
||||
| "unsupported_host"
|
||||
| "unsupported_version"
|
||||
| "version_unresolved"
|
||||
| "open_options_invalid";
|
||||
|
||||
/**
|
||||
* Failure kinds tagged at the JADX configuration boundary. Consumers (the
|
||||
* doctor adapter) project the boundary's remediation and classification for
|
||||
* a kind; they must not reinterpret the phase/setting evidence to derive
|
||||
* them.
|
||||
*/
|
||||
export type JadxFailureKind =
|
||||
| "windows_native_artifact"
|
||||
| "windows_native_runtime"
|
||||
| "java_runtime"
|
||||
| "java_home"
|
||||
| "jvm_options"
|
||||
| "jar"
|
||||
| "java_probe"
|
||||
| "unsupported_host";
|
||||
|
||||
/** Runtime mirror of JadxFailureKind for validating the loose diagnostics tag. */
|
||||
export const JADX_FAILURE_KINDS: readonly JadxFailureKind[] = [
|
||||
"windows_native_artifact",
|
||||
"windows_native_runtime",
|
||||
"java_runtime",
|
||||
"java_home",
|
||||
"jvm_options",
|
||||
"jar",
|
||||
"java_probe",
|
||||
"unsupported_host",
|
||||
];
|
||||
|
||||
export interface JadxFailureOutcome {
|
||||
readonly remediation: string;
|
||||
readonly classification: DoctorProviderCheck["classification"];
|
||||
}
|
||||
|
||||
/**
|
||||
* Project the boundary-owned remediation and classification for a failure
|
||||
* kind. A null kind (missing or corrupt tag) keeps JAR guidance, matching
|
||||
* the previous fallthrough for unclassified failures.
|
||||
*/
|
||||
export const projectJadxFailureOutcome = (
|
||||
kind: JadxFailureKind | null,
|
||||
configured: boolean,
|
||||
): JadxFailureOutcome => {
|
||||
const drift = configured ? "config_drift" : "missing_analysis_engine";
|
||||
switch (kind) {
|
||||
case "windows_native_artifact":
|
||||
return {
|
||||
remediation: JADX_WINDOWS_NATIVE_REMEDIATION,
|
||||
classification: "missing_dependency",
|
||||
};
|
||||
case "windows_native_runtime":
|
||||
return {
|
||||
remediation: JADX_WINDOWS_NATIVE_REMEDIATION,
|
||||
classification: "runtime_failure",
|
||||
};
|
||||
case "java_runtime":
|
||||
return {
|
||||
remediation: JADX_JAVA_RUNTIME_REMEDIATION,
|
||||
classification: "missing_dependency",
|
||||
};
|
||||
case "java_home":
|
||||
return {
|
||||
remediation: JADX_JAVA_RUNTIME_REMEDIATION,
|
||||
classification: drift,
|
||||
};
|
||||
case "jvm_options":
|
||||
return {
|
||||
remediation: JADX_JVM_OPTIONS_REMEDIATION,
|
||||
classification: drift,
|
||||
};
|
||||
case "jar":
|
||||
return {
|
||||
remediation: JADX_JAR_CONFIGURATION_REMEDIATION,
|
||||
classification: drift,
|
||||
};
|
||||
case "java_probe":
|
||||
return {
|
||||
remediation: JADX_JAVA_PROBE_REMEDIATION,
|
||||
classification: drift,
|
||||
};
|
||||
case "unsupported_host":
|
||||
return {
|
||||
remediation: JADX_UNSUPPORTED_HOST_REMEDIATION,
|
||||
classification: "unsupported_host",
|
||||
};
|
||||
case null:
|
||||
return {
|
||||
remediation: JADX_JAR_CONFIGURATION_REMEDIATION,
|
||||
classification: drift,
|
||||
};
|
||||
}
|
||||
};
|
||||
|
||||
class JadxConfigurationFailure extends Error {
|
||||
constructor(
|
||||
readonly code: ConfigurationFailureCode,
|
||||
message: string,
|
||||
readonly diagnostics: Readonly<Record<string, JsonValue>>,
|
||||
readonly kind: JadxFailureKind,
|
||||
options?: ErrorOptions,
|
||||
) {
|
||||
super(message, options);
|
||||
|
|
@ -75,7 +178,7 @@ export const inspectJadxAvailability = async (
|
|||
status: "unavailable",
|
||||
code: cause.code,
|
||||
reason: cause.message,
|
||||
diagnostics: cause.diagnostics,
|
||||
diagnostics: { ...cause.diagnostics, failure_kind: cause.kind },
|
||||
};
|
||||
throw cause;
|
||||
}
|
||||
|
|
@ -191,6 +294,7 @@ const requireEngineArchive = async (
|
|||
archive_error: reason,
|
||||
maximum_metadata_read_bytes: JAR_METADATA_READ_MAX_BYTES,
|
||||
},
|
||||
"jar",
|
||||
cause,
|
||||
);
|
||||
} finally {
|
||||
|
|
@ -200,6 +304,7 @@ const requireEngineArchive = async (
|
|||
"unsupported_version",
|
||||
`REA_JADX_MCP_JAR ${jar} does not contain the required metadata bridge classes: ${[...missing].join(", ")}. ${JADX_JAR_CONFIGURATION_REMEDIATION}`,
|
||||
{ jar_path: jar, phase: "jar-inspection", missing_classes: [...missing] },
|
||||
"jar",
|
||||
);
|
||||
};
|
||||
|
||||
|
|
@ -207,25 +312,52 @@ const requireSupportedHost = (): void => {
|
|||
if (process.platform === "linux" || process.platform === "darwin") return;
|
||||
if (process.platform === "win32" && process.arch === "x64") {
|
||||
const loaded = systemWindowsNativeAuthority();
|
||||
if (loaded.available && loaded.authority.inspection.protocolStdin === true)
|
||||
return;
|
||||
if (loaded.available) {
|
||||
if (loaded.authority.inspection.protocolStdin === true) return;
|
||||
throw configurationFailure(
|
||||
"runtime_missing",
|
||||
"The bundled Windows native artifact lacks owned protocol stdin; use a matching current Windows x64 REA package.",
|
||||
{
|
||||
platform: process.platform,
|
||||
architecture: process.arch,
|
||||
phase: "process-ownership",
|
||||
owned_protocol_stdin: false,
|
||||
},
|
||||
"windows_native_artifact",
|
||||
);
|
||||
}
|
||||
const readiness = inspectWindowsNativeReadiness(process.platform, loaded);
|
||||
throw configurationFailure(
|
||||
"runtime_missing",
|
||||
loaded.available
|
||||
? "The bundled Windows native artifact lacks owned protocol stdin; use a matching current Windows x64 REA package."
|
||||
: loaded.reason,
|
||||
readiness.available ? "runtime_unavailable" : readiness.code,
|
||||
readiness.available
|
||||
? "Windows native readiness disagreed with the native loader result."
|
||||
: readiness.reason,
|
||||
{
|
||||
platform: process.platform,
|
||||
architecture: process.arch,
|
||||
phase: "process-ownership",
|
||||
owned_protocol_stdin: false,
|
||||
...(readiness.available
|
||||
? {}
|
||||
: {
|
||||
failure_stage: readiness.failureStage,
|
||||
remediation: readiness.remediation,
|
||||
}),
|
||||
},
|
||||
readiness.available
|
||||
? "windows_native_runtime"
|
||||
: readiness.code === "unsupported_host"
|
||||
? "unsupported_host"
|
||||
: readiness.code === "runtime_missing"
|
||||
? "windows_native_artifact"
|
||||
: "windows_native_runtime",
|
||||
);
|
||||
}
|
||||
throw configurationFailure(
|
||||
"unsupported_host",
|
||||
`Android JADX subprocess ownership is unsupported on ${process.platform}/${process.arch}; use Linux, macOS, or Windows x64 with its bundled native controls.`,
|
||||
{ platform: process.platform },
|
||||
"unsupported_host",
|
||||
);
|
||||
};
|
||||
|
||||
|
|
@ -238,6 +370,7 @@ const resolveJar = async (
|
|||
"not_configured",
|
||||
JADX_JAR_CONFIGURATION_REMEDIATION,
|
||||
{ jar_configured: false },
|
||||
"jar",
|
||||
);
|
||||
try {
|
||||
await access(jar, constants.R_OK);
|
||||
|
|
@ -246,6 +379,7 @@ const resolveJar = async (
|
|||
"executable_missing",
|
||||
`REA_JADX_MCP_JAR is not a regular file: ${jar}`,
|
||||
{ jar_path: jar },
|
||||
"jar",
|
||||
);
|
||||
return await realpath(jar);
|
||||
} catch (cause) {
|
||||
|
|
@ -254,6 +388,7 @@ const resolveJar = async (
|
|||
"executable_missing",
|
||||
`Cannot read REA_JADX_MCP_JAR ${jar}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
{ jar_path: jar },
|
||||
"jar",
|
||||
cause,
|
||||
);
|
||||
}
|
||||
|
|
@ -293,6 +428,7 @@ const resolveJava = async (
|
|||
"runtime_missing",
|
||||
`Cannot inspect selected Java executable ${candidate}: ${errorText(cause)}`,
|
||||
{ java_executable: candidate },
|
||||
"java_runtime",
|
||||
cause,
|
||||
);
|
||||
}
|
||||
|
|
@ -301,6 +437,7 @@ const resolveJava = async (
|
|||
"runtime_missing",
|
||||
"Cannot find java.exe on the selected Windows PATH. Select a full JDK with JAVA_HOME or PATH, then verify it with java --list-modules. REA does not install Java.",
|
||||
{ java_executable: executable },
|
||||
"java_runtime",
|
||||
);
|
||||
}
|
||||
if (isAbsolute(environment.JAVA_HOME))
|
||||
|
|
@ -309,6 +446,7 @@ const resolveJava = async (
|
|||
"open_options_invalid",
|
||||
"JAVA_HOME must select an existing JDK by absolute path; omit it to use java on PATH.",
|
||||
{ java_home: environment.JAVA_HOME },
|
||||
"java_home",
|
||||
);
|
||||
};
|
||||
|
||||
|
|
@ -324,6 +462,7 @@ const requireJavaHomeExecutable = async (
|
|||
"runtime_missing",
|
||||
`JAVA_HOME java is not a regular file: ${java}`,
|
||||
{ java_executable: java },
|
||||
"java_runtime",
|
||||
);
|
||||
} catch (cause) {
|
||||
if (cause instanceof JadxConfigurationFailure) throw cause;
|
||||
|
|
@ -331,6 +470,7 @@ const requireJavaHomeExecutable = async (
|
|||
"runtime_missing",
|
||||
`Cannot execute JAVA_HOME java ${java}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
{ java_executable: java },
|
||||
"java_runtime",
|
||||
cause,
|
||||
);
|
||||
}
|
||||
|
|
@ -351,6 +491,7 @@ const resolveJvmArguments = (
|
|||
"open_options_invalid",
|
||||
`${name} must be a positive safe integer; received ${value}.`,
|
||||
{ setting: name, value },
|
||||
"jvm_options",
|
||||
);
|
||||
if (
|
||||
name === "REA_JADX_ACTIVE_PROCESSOR_COUNT" &&
|
||||
|
|
@ -360,6 +501,7 @@ const resolveJvmArguments = (
|
|||
"open_options_invalid",
|
||||
`${name} exceeds the JVM's signed 32-bit processor-count range.`,
|
||||
{ setting: name, value },
|
||||
"jvm_options",
|
||||
);
|
||||
result.push(`${prefix}${value}${suffix}`);
|
||||
}
|
||||
|
|
@ -405,6 +547,7 @@ const inspectJavaRuntime = async (
|
|||
failure: "output_limit",
|
||||
output_limit_bytes: JAVA_MODULE_LISTING_MAX_OUTPUT_BYTES,
|
||||
},
|
||||
"java_probe",
|
||||
cause,
|
||||
);
|
||||
if (output.killed)
|
||||
|
|
@ -416,12 +559,14 @@ const inspectJavaRuntime = async (
|
|||
failure: "probe_terminated",
|
||||
deadline_ms: JAVA_MODULE_LISTING_TIMEOUT_MS,
|
||||
},
|
||||
"java_probe",
|
||||
cause,
|
||||
);
|
||||
throw configurationFailure(
|
||||
"runtime_missing",
|
||||
`Cannot inspect selected Java runtime ${java}: ${output.stderr.trim() || errorText(cause)}. Select a full JDK including jdk.compiler via JAVA_HOME or PATH; verify it with java --list-modules.`,
|
||||
diagnostics,
|
||||
"java_runtime",
|
||||
cause,
|
||||
);
|
||||
}
|
||||
|
|
@ -438,6 +583,7 @@ const inspectJavaRuntime = async (
|
|||
stderr,
|
||||
output_truncated: false,
|
||||
},
|
||||
"java_runtime",
|
||||
);
|
||||
const major = /^java\.base@(\d+)(?:[.+_-]|$)/mu.exec(stdout)?.[1];
|
||||
if (major === undefined)
|
||||
|
|
@ -453,6 +599,7 @@ const inspectJavaRuntime = async (
|
|||
stderr,
|
||||
output_truncated: false,
|
||||
},
|
||||
"java_probe",
|
||||
);
|
||||
if (Number(major) < 17)
|
||||
throw configurationFailure(
|
||||
|
|
@ -467,6 +614,7 @@ const inspectJavaRuntime = async (
|
|||
stderr,
|
||||
output_truncated: false,
|
||||
},
|
||||
"java_runtime",
|
||||
);
|
||||
};
|
||||
|
||||
|
|
@ -474,12 +622,14 @@ const configurationFailure = (
|
|||
code: ConfigurationFailureCode,
|
||||
message: string,
|
||||
diagnostics: Readonly<Record<string, JsonValue>>,
|
||||
kind: JadxFailureKind,
|
||||
cause?: unknown,
|
||||
): JadxConfigurationFailure =>
|
||||
new JadxConfigurationFailure(
|
||||
code,
|
||||
message,
|
||||
diagnostics,
|
||||
kind,
|
||||
cause === undefined ? undefined : { cause },
|
||||
);
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,155 @@
|
|||
import type {
|
||||
DoctorProviderCheck,
|
||||
DoctorProviderInspection,
|
||||
} from "../application/Doctor.js";
|
||||
import type { ProviderAvailability } from "../application/AnalysisProvider.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import {
|
||||
inspectJadxAvailability,
|
||||
JADX_FAILURE_KINDS,
|
||||
projectJadxFailureOutcome,
|
||||
type JadxFailureKind,
|
||||
} from "./JadxConfiguration.js";
|
||||
|
||||
/** Project the validated JADX configuration into the doctor contract. */
|
||||
export const inspectJadxProvider = async (
|
||||
environment: Readonly<Record<string, string | undefined>>,
|
||||
): Promise<DoctorProviderInspection> => {
|
||||
const configured = environment.REA_JADX_MCP_JAR !== undefined;
|
||||
const availability = await inspectJadxAvailability(environment);
|
||||
// diagnostics is a loosely-typed JsonValue map; guard the published path
|
||||
// even though the success path always sets jar_path today.
|
||||
const jarPath =
|
||||
availability.status === "available" &&
|
||||
typeof availability.diagnostics["jar_path"] === "string"
|
||||
? availability.diagnostics["jar_path"]
|
||||
: undefined;
|
||||
const available =
|
||||
availability.status === "available" && jarPath !== undefined;
|
||||
// Preserve the caller-selected JAR path (it may be a stable symlink) and
|
||||
// the validated JAVA_HOME selection; the resolved jarPath stays in
|
||||
// diagnostics as inspection evidence.
|
||||
const configuredJar = environment.REA_JADX_MCP_JAR;
|
||||
return {
|
||||
id: "jadx",
|
||||
configured,
|
||||
available,
|
||||
providerVersion: null,
|
||||
registrationEnvironment:
|
||||
jarPath === undefined || configuredJar === undefined
|
||||
? {}
|
||||
: {
|
||||
REA_JADX_MCP_JAR: configuredJar,
|
||||
...optionalEntry("JAVA_HOME", environment.JAVA_HOME),
|
||||
...optionalEntry(
|
||||
"REA_JADX_HEAP_MIB",
|
||||
environment.REA_JADX_HEAP_MIB,
|
||||
),
|
||||
...optionalEntry(
|
||||
"REA_JADX_ACTIVE_PROCESSOR_COUNT",
|
||||
environment.REA_JADX_ACTIVE_PROCESSOR_COUNT,
|
||||
),
|
||||
},
|
||||
checks: [configurationCheck(configured, availability, jarPath)],
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* Publish a caller-selected variable only when explicitly set; never persist
|
||||
* ambient inheritance. Values are validated before this point.
|
||||
*/
|
||||
const optionalEntry = (
|
||||
name: string,
|
||||
value: string | undefined,
|
||||
): Record<string, string> => (value === undefined ? {} : { [name]: value });
|
||||
|
||||
const configurationCheck = (
|
||||
configured: boolean,
|
||||
availability: ProviderAvailability,
|
||||
jarPath: string | undefined,
|
||||
): DoctorProviderCheck => {
|
||||
const code = availability.status === "unavailable" ? availability.code : null;
|
||||
if (availability.status === "available" && jarPath !== undefined) {
|
||||
return {
|
||||
name: "configuration",
|
||||
ok: true,
|
||||
code: null,
|
||||
detail: `JADX MCP bridge JAR validated at ${jarPath}; its path is recorded for agent registrations.`,
|
||||
remediation: null,
|
||||
classification: "config_drift",
|
||||
};
|
||||
}
|
||||
// The failure kind is tagged at the configuration boundary; project its
|
||||
// remediation and classification here instead of reinterpreting the
|
||||
// phase/setting evidence. Every failed check keeps a remediation.
|
||||
const kind =
|
||||
availability.status === "unavailable"
|
||||
? asFailureKind(availability.diagnostics["failure_kind"])
|
||||
: null;
|
||||
const outcome = projectJadxFailureOutcome(kind, configured);
|
||||
return {
|
||||
name: "configuration",
|
||||
ok: false,
|
||||
code,
|
||||
detail:
|
||||
availability.status === "available"
|
||||
? "Validated JADX configuration reported no JAR path."
|
||||
: availability.reason,
|
||||
remediation: outcome.remediation,
|
||||
classification: outcome.classification,
|
||||
...(kind === "java_probe" && availability.status === "unavailable"
|
||||
? { details: probeDetails(availability.diagnostics) }
|
||||
: kind !== null &&
|
||||
kind.startsWith("windows_native_") &&
|
||||
availability.status === "unavailable"
|
||||
? { details: windowsNativeFailureDetails(availability.diagnostics) }
|
||||
: {}),
|
||||
};
|
||||
};
|
||||
|
||||
const windowsNativeFailureDetails = (
|
||||
diagnostics: Readonly<Record<string, JsonValue>>,
|
||||
): Record<string, JsonValue> => {
|
||||
const details: Record<string, JsonValue> = {};
|
||||
for (const key of ["failure_stage", "remediation"] as const) {
|
||||
const value = diagnostics[key];
|
||||
if (value !== undefined) details[key] = value;
|
||||
}
|
||||
return details;
|
||||
};
|
||||
|
||||
/** Validate the loose diagnostics tag against the boundary's failure kinds. */
|
||||
const asFailureKind = (value: JsonValue | undefined): JadxFailureKind | null =>
|
||||
typeof value === "string" &&
|
||||
(JADX_FAILURE_KINDS as readonly string[]).includes(value)
|
||||
? (value as JadxFailureKind)
|
||||
: null;
|
||||
|
||||
/**
|
||||
* Project bounded Java probe metadata for the doctor report. The full
|
||||
* stdout/stderr can reach the 1 MiB probe budget, so only the bounded
|
||||
* outcome metadata is carried; the message already references the retained
|
||||
* output.
|
||||
*/
|
||||
const probeDetails = (
|
||||
diagnostics: Readonly<Record<string, JsonValue>>,
|
||||
): Record<string, JsonValue> => {
|
||||
const details: Record<string, JsonValue> = {};
|
||||
for (const key of [
|
||||
"java_executable",
|
||||
"failure",
|
||||
"output_truncated",
|
||||
"exit_code",
|
||||
"process_error_code",
|
||||
"signal",
|
||||
"killed",
|
||||
"output_limit_bytes",
|
||||
"deadline_ms",
|
||||
"jdk_compiler",
|
||||
"java_major",
|
||||
] as const) {
|
||||
const value = diagnostics[key];
|
||||
if (value !== undefined) details[key] = value;
|
||||
}
|
||||
return details;
|
||||
};
|
||||
|
|
@ -15,6 +15,26 @@ export const JADX_BRIDGE_IDENTITY = Object.freeze({
|
|||
/** Remediation when `REA_JADX_MCP_JAR` is missing or not an absolute path. */
|
||||
export const JADX_JAR_CONFIGURATION_REMEDIATION = `Set REA_JADX_MCP_JAR to the absolute path of a caller-supplied jadx-headless-mcp ${JADX_RELEASE.version} JAR. REA does not download or install it.`;
|
||||
|
||||
/** Remediation when the selected Java runtime cannot satisfy the JADX bridge. */
|
||||
export const JADX_JAVA_RUNTIME_REMEDIATION =
|
||||
"Select a full JDK including jdk.compiler via JAVA_HOME or PATH, then verify it with java --list-modules. REA does not install Java.";
|
||||
|
||||
/** Remediation when a JADX JVM resource override is malformed. */
|
||||
export const JADX_JVM_OPTIONS_REMEDIATION =
|
||||
"Set REA_JADX_HEAP_MIB to a positive integer (heap size in MiB) and REA_JADX_ACTIVE_PROCESSOR_COUNT to a positive integer within the JVM's signed 32-bit range.";
|
||||
|
||||
/** Remediation when the host cannot run the Android JADX subprocess. */
|
||||
export const JADX_UNSUPPORTED_HOST_REMEDIATION =
|
||||
"Use Linux, macOS, or Windows x64 with its bundled native controls for Android JADX analysis.";
|
||||
|
||||
/** Remediation when the Windows native backend cannot take process ownership. */
|
||||
export const JADX_WINDOWS_NATIVE_REMEDIATION =
|
||||
"Use a matching current Windows x64 REA package with its bundled native controls. The native failure above identifies the failed constraint; Java installation or registration changes cannot enable a missing native backend.";
|
||||
|
||||
/** Remediation when the Java module-listing probe cannot establish the runtime. */
|
||||
export const JADX_JAVA_PROBE_REMEDIATION =
|
||||
"Verify the selected Java launcher with java --list-modules: it must start promptly and emit a concise module listing including a parseable java.base version. REA does not install Java.";
|
||||
|
||||
/** Producing engine identity used by Evidence and the generated product catalog. */
|
||||
export const JADX_PROVIDER_IDENTITY = Object.freeze({
|
||||
id: "jadx",
|
||||
|
|
|
|||
|
|
@ -100,10 +100,12 @@ const decodeStub = (
|
|||
readonly manifestFile?: string;
|
||||
readonly stringsFile?: string;
|
||||
readonly stringsXml?: string;
|
||||
readonly extraValuesDirectories?: readonly string[];
|
||||
} = {},
|
||||
): ((root: string) => string) => {
|
||||
const withStrings = options.withStrings ?? true;
|
||||
const withGerman = options.withGerman ?? true;
|
||||
const extraValuesDirectories = options.extraValuesDirectories ?? [];
|
||||
const stringsXml = options.stringsXml ?? REAL_STRINGS;
|
||||
const ymlWrite =
|
||||
options.ymlFile === undefined
|
||||
|
|
@ -117,6 +119,12 @@ const decodeStub = (
|
|||
options.stringsFile === undefined
|
||||
? `printf '%s\\n' '${stringsXml.replace(/'/gu, "'\\''")}' > "$5/res/values/strings.xml"`
|
||||
: `cp '${options.stringsFile}' "$5/res/values/strings.xml"`;
|
||||
const qualifierStrings = extraValuesDirectories
|
||||
.map(
|
||||
(name) =>
|
||||
`printf '%s\\n' '<resources><string name="wrong">Wrong</string></resources>' > "$5/res/values-${name}/strings.xml"`,
|
||||
)
|
||||
.join("\\n ");
|
||||
return () => `if [ "$1" = "--version" ]; then ${VERSION_BLOCK}; exit 0; fi
|
||||
if [ "$1" = "d" ]; then
|
||||
${
|
||||
|
|
@ -124,9 +132,10 @@ if [ "$1" = "d" ]; then
|
|||
? `printf 'Exception in thread "main" brut.androlib.AndrolibException: Could not decode the APK\\n' >&2; exit 1;`
|
||||
: `${ymlWrite}
|
||||
${manifestWrite}
|
||||
mkdir -p "$5/res/values"${withGerman ? ' "$5/res/values-de"' : ""}
|
||||
mkdir -p "$5/res/values"${withGerman ? ' "$5/res/values-de"' : ""}${extraValuesDirectories.map((name) => ` "$5/res/values-${name}"`).join("")}
|
||||
${withStrings ? stringsWrite : ""}
|
||||
${withGerman ? `printf '%s\\n' '${DE_STRINGS.replace(/'/gu, "'\\''")}' > "$5/res/values-de/strings.xml"` : ""}
|
||||
${qualifierStrings}
|
||||
printf 'I: Using Apktool on stub\\n'
|
||||
exit 0`
|
||||
}
|
||||
|
|
@ -254,6 +263,30 @@ it("preserves empty resource strings and reports markup omissions as partial", a
|
|||
expect(result.coverage).toBe("partial");
|
||||
});
|
||||
|
||||
it("ignores Android resource qualifiers when discovering locales", async () => {
|
||||
const { command, root } = await writeApktoolStub(
|
||||
decodeStub({ extraValuesDirectories: ["car", "night", "v21"] }),
|
||||
);
|
||||
const apk = await apkFixture(root);
|
||||
const result = await evidenceResult(providerOver(command), {
|
||||
operation: "decode_android_resources",
|
||||
input: { path: apk.path, include_strings: true },
|
||||
});
|
||||
expect(result.locales).toEqual(["de"]);
|
||||
});
|
||||
|
||||
it.each(["car", "night", "v21"])(
|
||||
"rejects %s as a requested locale",
|
||||
(locale) => {
|
||||
expect(
|
||||
apktoolRequestSchema.safeParse({
|
||||
operation: "decode_android_resources",
|
||||
input: { path: "/tmp/probe.apk", locale },
|
||||
}).success,
|
||||
).toBe(false);
|
||||
},
|
||||
);
|
||||
|
||||
it("projects one locale's strings on request", async () => {
|
||||
const { command, root } = await writeApktoolStub(decodeStub());
|
||||
const apk = await apkFixture(root);
|
||||
|
|
|
|||
|
|
@ -25,7 +25,10 @@ import {
|
|||
createAnalysisExecution,
|
||||
type AnalysisExecution,
|
||||
} from "../application/AnalysisProvider.js";
|
||||
import type { ApktoolRequest } from "../domain/apktool/apktoolResourceAnalysis.js";
|
||||
import {
|
||||
androidLocaleQualifierSchema,
|
||||
type ApktoolRequest,
|
||||
} from "../domain/apktool/apktoolResourceAnalysis.js";
|
||||
import {
|
||||
AnalysisCapabilityUnavailableError,
|
||||
AnalysisCancelledError,
|
||||
|
|
@ -291,6 +294,7 @@ const countDecodedFiles = async (
|
|||
const match = /^values-(.+)$/u.exec(entry.name);
|
||||
if (
|
||||
match !== null &&
|
||||
androidLocaleQualifierSchema.safeParse(match[1]).success &&
|
||||
(await access(join(path, "strings.xml")).then(
|
||||
() => true,
|
||||
() => false,
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ import {
|
|||
} from "./AnalysisProvider.js";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisOutputError,
|
||||
AnalysisProtocolError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { err, ok } from "../domain/result.js";
|
||||
|
|
@ -17,6 +18,35 @@ import { err, ok } from "../domain/result.js";
|
|||
const provider = { id: "fixture", name: "Fixture", version: "1" };
|
||||
|
||||
describe("analysis context queries: navigation", () => {
|
||||
it.each([
|
||||
[null, "0x401000", "current_document"],
|
||||
["fixture", null, "current_address"],
|
||||
] as const)(
|
||||
"rejects malformed %s / %s provider selection before returning success",
|
||||
async (document, address, operation) => {
|
||||
const analysis: AnalysisOperationPort = {
|
||||
execute: async (name) =>
|
||||
ok(
|
||||
createAnalysisExecution(
|
||||
name === "current_document" ? document : address,
|
||||
provider,
|
||||
),
|
||||
),
|
||||
};
|
||||
const result = await getNavigationContext(analysis, {});
|
||||
expect(result).toEqual(
|
||||
err(
|
||||
new AnalysisOutputError(
|
||||
operation,
|
||||
operation === "current_document"
|
||||
? "expected a document identity string"
|
||||
: "expected an address string",
|
||||
),
|
||||
),
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it("aggregates volatile navigation and treats a missing procedure as null", async () => {
|
||||
const calls: {
|
||||
operation: string;
|
||||
|
|
@ -70,6 +100,23 @@ describe("analysis context queries: navigation", () => {
|
|||
});
|
||||
});
|
||||
|
||||
it("preserves empty string selections accepted by the output contract", async () => {
|
||||
const analysis: AnalysisOperationPort = {
|
||||
execute: async (operation) =>
|
||||
ok(
|
||||
createAnalysisExecution(
|
||||
operation === "resolve_containing_procedure"
|
||||
? { found: false, procedure: null, reason: "not_in_procedure" }
|
||||
: "",
|
||||
provider,
|
||||
),
|
||||
),
|
||||
};
|
||||
await expect(getNavigationContext(analysis, {})).resolves.toEqual(
|
||||
ok({ document: "", address: "", procedure: null }),
|
||||
);
|
||||
});
|
||||
|
||||
it("uses an explicit document without passing it to current_document and preserves procedure failures", async () => {
|
||||
const calls: string[] = [];
|
||||
const failure = new AnalysisProtocolError("provider response malformed");
|
||||
|
|
|
|||
|
|
@ -4,10 +4,11 @@ import type {
|
|||
AnalysisOperationPort,
|
||||
} from "./AnalysisProvider.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import { analysisBookmarkSchema } from "../domain/hopperValues.js";
|
||||
import { resolveAnalysisDocument } from "./AnalysisDocument.js";
|
||||
import { ok, type Result } from "../domain/result.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
|
||||
type Facet =
|
||||
| { readonly state: "available"; readonly value: JsonValue }
|
||||
|
|
@ -42,8 +43,14 @@ export const getNavigationContext = async (
|
|||
if (typeof document !== "string" && !document.ok) return document;
|
||||
const documentName =
|
||||
typeof document === "string" ? document : document.value.result;
|
||||
const documentArgs =
|
||||
typeof documentName === "string" ? parameters(documentName) : {};
|
||||
if (typeof documentName !== "string")
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
"current_document",
|
||||
"expected a document identity string",
|
||||
),
|
||||
);
|
||||
const documentArgs = parameters(documentName);
|
||||
const address = await analysis.execute(
|
||||
"current_address",
|
||||
documentArgs,
|
||||
|
|
@ -51,17 +58,12 @@ export const getNavigationContext = async (
|
|||
);
|
||||
if (!address.ok) return address;
|
||||
if (typeof address.value.result !== "string")
|
||||
return ok({
|
||||
document: documentName,
|
||||
address: address.value.result,
|
||||
procedure: null,
|
||||
});
|
||||
return err(
|
||||
new AnalysisOutputError("current_address", "expected an address string"),
|
||||
);
|
||||
const procedure = await analysis.execute(
|
||||
"resolve_containing_procedure",
|
||||
parameters(
|
||||
typeof documentName === "string" ? documentName : undefined,
|
||||
address.value.result,
|
||||
),
|
||||
parameters(documentName, address.value.result),
|
||||
executionOptions(signal),
|
||||
);
|
||||
if (!procedure.ok) {
|
||||
|
|
|
|||
|
|
@ -275,6 +275,12 @@ export type ProviderTargetSupport =
|
|||
readonly diagnostics: Readonly<Record<string, JsonValue>>;
|
||||
};
|
||||
|
||||
/** One owner-observed availability/capability snapshot for provider discovery. */
|
||||
export interface AnalysisProviderReadinessSnapshot {
|
||||
readonly availability: ProviderAvailability;
|
||||
readonly capabilities: readonly CapabilityDescriptor[];
|
||||
}
|
||||
|
||||
/** Deep provider candidate discoverable before one target-bound route is chosen. */
|
||||
export interface AnalysisProviderCandidate extends AnalysisProvider {
|
||||
resolveAnalysisProfile(
|
||||
|
|
@ -282,5 +288,6 @@ export interface AnalysisProviderCandidate extends AnalysisProvider {
|
|||
options?: AnalysisProfileResolutionOptions,
|
||||
): Promise<Result<AnalysisProfileResolution, AnalysisError>>;
|
||||
inspectAvailability(): ProviderAvailability;
|
||||
inspectReadinessSnapshot?(): AnalysisProviderReadinessSnapshot;
|
||||
inspectTargetSupport(target: BinaryTarget): ProviderTargetSupport;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,60 @@
|
|||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { resolveAppBundleExecutable } from "./AppBundleExecutable.js";
|
||||
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
|
||||
|
||||
describe("OpenStep Info.plist nesting during app target resolution", () => {
|
||||
it.each([127, 128, 3000])(
|
||||
"applies the shared bound to %s nested payload arrays",
|
||||
async (depth) => {
|
||||
const root = await createTestTempDirectory("rea-app-openstep-depth-");
|
||||
const app = join(root, "Example.app");
|
||||
const programs = join(app, "Contents", "MacOS");
|
||||
await mkdir(programs, { recursive: true });
|
||||
await writeFile(join(programs, "App"), "inert program-path fixture");
|
||||
await writeFile(
|
||||
join(app, "Contents", "Info.plist"),
|
||||
`{ CFBundleExecutable = App; payload = ${"(".repeat(depth)}x${")".repeat(depth)}; }`,
|
||||
);
|
||||
|
||||
const result = await resolveAppBundleExecutable(app);
|
||||
|
||||
if (depth === 127) {
|
||||
expect(result).toMatchObject({
|
||||
ok: true,
|
||||
value: { executable: join(programs, "App") },
|
||||
});
|
||||
} else {
|
||||
expect(result).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
path: join(app, "Contents", "Info.plist"),
|
||||
cause: expect.objectContaining({
|
||||
message: "Plist exceeds 128 nesting levels",
|
||||
}),
|
||||
},
|
||||
});
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it("resolves the first Info.plist root without decoding deeply nested trailing text", async () => {
|
||||
const root = await createTestTempDirectory("rea-app-openstep-trailing-");
|
||||
const app = join(root, "Example.app");
|
||||
const programs = join(app, "Contents", "MacOS");
|
||||
await mkdir(programs, { recursive: true });
|
||||
await writeFile(join(programs, "App"), "inert program-path fixture");
|
||||
await writeFile(
|
||||
join(app, "Contents", "Info.plist"),
|
||||
`{ CFBundleExecutable = App; }${"(".repeat(3000)}x${")".repeat(3000)}`,
|
||||
);
|
||||
|
||||
expect(await resolveAppBundleExecutable(app)).toMatchObject({
|
||||
ok: true,
|
||||
value: { executable: join(programs, "App") },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
@ -32,7 +32,11 @@ import {
|
|||
} from "../domain/rawBinary.js";
|
||||
import type { BinaryArchitecture } from "../domain/binaryTargetTypes.js";
|
||||
import { observeMzWindowsHeader } from "../domain/dosMz.js";
|
||||
import { hasBinaryPlistSignature } from "../domain/apple/appleArchiveFormat.js";
|
||||
import {
|
||||
createXmlPlistSignatureProbe,
|
||||
hasBinaryPlistSignature,
|
||||
MAX_XML_PLIST_PROBE_BYTES,
|
||||
} from "../domain/apple/appleArchiveFormat.js";
|
||||
import {
|
||||
hasZipSignature,
|
||||
isZipPackageFormat,
|
||||
|
|
@ -49,6 +53,8 @@ import {
|
|||
} from "../domain/binaryTarget.js";
|
||||
import type { PeCodeViewRecord } from "../domain/binaryTargetTypes.js";
|
||||
|
||||
const XML_PLIST_PROBE_CHUNK_BYTES = 64 * 1024;
|
||||
|
||||
/** Options for resolving a local binary target and its stable identity. */
|
||||
export interface BinaryTargetResolverOptions {
|
||||
readonly cwd?: string;
|
||||
|
|
@ -288,12 +294,12 @@ const detectArtifactFormat = async (
|
|||
> => {
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
const lower = path.toLowerCase();
|
||||
const magic = Buffer.alloc(8);
|
||||
const magic = Buffer.alloc(4096);
|
||||
const observed = await handle.read(magic, 0, magic.length, 0);
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
if (
|
||||
observed.bytesRead === 8 &&
|
||||
magic.equals(Buffer.from([0, 0x61, 0x73, 0x6d, 1, 0, 0, 0]))
|
||||
observed.bytesRead >= 8 &&
|
||||
magic.subarray(0, 8).equals(Buffer.from([0, 0x61, 0x73, 0x6d, 1, 0, 0, 0]))
|
||||
)
|
||||
throw new BinaryTargetError(
|
||||
path,
|
||||
|
|
@ -329,6 +335,37 @@ const detectArtifactFormat = async (
|
|||
return "dmg";
|
||||
}
|
||||
}
|
||||
// XML property lists can use archive-specific suffixes such as .archive.
|
||||
// The named-format check above keeps .js, .map, and .asar classifications.
|
||||
const xmlProbe = createXmlPlistSignatureProbe();
|
||||
let xmlState = xmlProbe.write(magic.subarray(0, observed.bytesRead));
|
||||
if (xmlState === "matched") return "plist";
|
||||
const chunk = Buffer.alloc(XML_PLIST_PROBE_CHUNK_BYTES);
|
||||
let bytesRead = observed.bytesRead;
|
||||
while (xmlState === "pending" && bytesRead < MAX_XML_PLIST_PROBE_BYTES) {
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
const read = await handle.read(
|
||||
chunk,
|
||||
0,
|
||||
Math.min(chunk.length, MAX_XML_PLIST_PROBE_BYTES - bytesRead),
|
||||
bytesRead,
|
||||
);
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
if (read.bytesRead === 0) break;
|
||||
bytesRead += read.bytesRead;
|
||||
xmlState = xmlProbe.write(chunk.subarray(0, read.bytesRead));
|
||||
if (xmlState === "matched") return "plist";
|
||||
}
|
||||
if (bytesRead === MAX_XML_PLIST_PROBE_BYTES) {
|
||||
const { size } = await handle.stat();
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
if (size > bytesRead)
|
||||
throw new BinaryTargetError(
|
||||
path,
|
||||
"XML plist prolog was not found within the 1 MiB format probe limit",
|
||||
{ constraint: "file_format_rejected" },
|
||||
);
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -11,7 +11,10 @@ import type {
|
|||
CaptureWebScreenshotInput,
|
||||
CompareWebScreenshotsInput,
|
||||
} from "../domain/webScreenshot.js";
|
||||
import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore.js";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisCapabilityUnavailableError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import type { ExecutionOptions } from "./AnalysisProvider.js";
|
||||
|
|
@ -27,6 +30,8 @@ export const listBrowserTargets = async (
|
|||
const ready = requireBrowser(browser, "list_browser_targets");
|
||||
if (!ready.ok) return ready;
|
||||
const result = await ready.value.listTargets(input, options);
|
||||
if (result.ok && options.signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError("list_browser_targets"));
|
||||
return result.ok
|
||||
? ok(
|
||||
createBrowserEvidence(
|
||||
|
|
@ -48,6 +53,8 @@ export const inspectWebPage = async (
|
|||
const ready = requireBrowser(browser, "inspect_web_page");
|
||||
if (!ready.ok) return ready;
|
||||
const result = await ready.value.inspectPage(input, options);
|
||||
if (result.ok && options.signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError("inspect_web_page"));
|
||||
return result.ok
|
||||
? ok(
|
||||
createBrowserEvidence(
|
||||
|
|
@ -69,6 +76,8 @@ export const analyzeWebBundle = async (
|
|||
const ready = requireBrowser(browser, "analyze_web_bundle");
|
||||
if (!ready.ok) return ready;
|
||||
const analyzed = await ready.value.analyzeBundle(input, options);
|
||||
if (analyzed.ok && options.signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError("analyze_web_bundle"));
|
||||
if (!analyzed.ok) return analyzed;
|
||||
return ok(
|
||||
createBrowserEvidence(
|
||||
|
|
@ -89,6 +98,8 @@ export const observeWebSession = async (
|
|||
const ready = requireBrowser(browser, "observe_web_session");
|
||||
if (!ready.ok) return ready;
|
||||
const observed = await ready.value.observeSession(input, options);
|
||||
if (observed.ok && options.signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError("observe_web_session"));
|
||||
return observed.ok
|
||||
? ok(
|
||||
createBrowserEvidence(
|
||||
|
|
@ -110,6 +121,8 @@ export const discoverWebMcpTools = async (
|
|||
const ready = requireBrowser(browser, "discover_webmcp_tools");
|
||||
if (!ready.ok) return ready;
|
||||
const discovered = await ready.value.discoverWebMcpTools(input, options);
|
||||
if (discovered.ok && options.signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError("discover_webmcp_tools"));
|
||||
return discovered.ok
|
||||
? ok(
|
||||
createBrowserEvidence(
|
||||
|
|
@ -151,6 +164,8 @@ export const captureWebScreenshot = async (
|
|||
const ready = requireBrowser(browser, "capture_web_screenshot");
|
||||
if (!ready.ok) return ready;
|
||||
const captured = await ready.value.captureScreenshot(input, options);
|
||||
if (captured.ok && options.signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError("capture_web_screenshot"));
|
||||
return captured.ok
|
||||
? ok(
|
||||
createBrowserEvidence(
|
||||
|
|
|
|||
|
|
@ -98,6 +98,21 @@ const entry = (
|
|||
};
|
||||
|
||||
describe("capability failure reasons", () => {
|
||||
it("keeps an unavailable Android runtime distinct from a missing provider", () => {
|
||||
const inventory = buildCapabilityInventory(status(), {
|
||||
...enabledPolicy,
|
||||
androidAnalysisAvailability: {
|
||||
status: "unavailable",
|
||||
code: "runtime_unavailable",
|
||||
reason: "The JADX native self-test failed.",
|
||||
diagnostics: {},
|
||||
},
|
||||
});
|
||||
expect(
|
||||
inventory.find(({ name }) => name === "inspect_android_package"),
|
||||
).toMatchObject({ available: false, reason: "provider_unavailable" });
|
||||
});
|
||||
|
||||
it("admits physical universal Mach-O targets for native Mach-O operations", () => {
|
||||
expect(
|
||||
entry(
|
||||
|
|
@ -501,7 +516,6 @@ it("reports host denial before target mismatch but keeps composed workflow prere
|
|||
describe("binary_overview capability requirements", () => {
|
||||
it("admits a provider with current_document and other mandatory capabilities", () => {
|
||||
const capabilities = [
|
||||
"list_segments",
|
||||
"current_document",
|
||||
"list_procedures",
|
||||
"list_strings",
|
||||
|
|
@ -520,7 +534,6 @@ describe("binary_overview capability requirements", () => {
|
|||
|
||||
it("admits a provider with list_documents and other mandatory capabilities", () => {
|
||||
const capabilities = [
|
||||
"list_segments",
|
||||
"list_documents",
|
||||
"list_procedures",
|
||||
"list_strings",
|
||||
|
|
@ -556,7 +569,7 @@ describe("binary_overview capability requirements", () => {
|
|||
});
|
||||
});
|
||||
|
||||
it("reports missing segment enumeration independently when segments are missing", () => {
|
||||
it("admits a provider without optional segment enumeration", () => {
|
||||
const capabilities = [
|
||||
"current_document",
|
||||
"list_procedures",
|
||||
|
|
@ -568,12 +581,31 @@ describe("binary_overview capability requirements", () => {
|
|||
status({ open: true, kind: "executable", capabilities }),
|
||||
),
|
||||
).toMatchObject({
|
||||
available: false,
|
||||
reason: "provider_missing",
|
||||
remediation: "Configure a provider for required operation list_segments.",
|
||||
available: true,
|
||||
reason: "available",
|
||||
remediation: null,
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps binary overview available when a declared segment facet is unavailable", () => {
|
||||
const capabilities = [
|
||||
{
|
||||
operation: "list_segments",
|
||||
available: false,
|
||||
reason: "This upstream profile does not expose segment enumeration.",
|
||||
},
|
||||
{ operation: "list_documents", available: true, reason: null },
|
||||
{ operation: "list_procedures", available: true, reason: null },
|
||||
{ operation: "list_strings", available: true, reason: null },
|
||||
];
|
||||
expect(
|
||||
entry(
|
||||
"binary_overview",
|
||||
status({ open: true, kind: "executable", capabilities }),
|
||||
),
|
||||
).toMatchObject({ available: true, reason: "available" });
|
||||
});
|
||||
|
||||
it("reports provider_unavailable when declared document selection capability is unavailable", () => {
|
||||
const capabilities = [
|
||||
{ operation: "list_segments", available: true, reason: null },
|
||||
|
|
|
|||
|
|
@ -115,7 +115,6 @@ const ENHANCED_REQUIREMENTS: Readonly<
|
|||
analyze_swift_types: ["list_procedures"],
|
||||
find_xrefs_to_name: ["list_names", "xrefs"],
|
||||
binary_overview: [
|
||||
"list_segments",
|
||||
["current_document", "list_documents"],
|
||||
"list_procedures",
|
||||
"list_strings",
|
||||
|
|
@ -462,7 +461,10 @@ const workflowAvailabilityFor = ({
|
|||
reason:
|
||||
policy.androidAnalysisAvailability?.code === "unsupported_host"
|
||||
? "unsupported_host"
|
||||
: "provider_missing",
|
||||
: policy.androidAnalysisAvailability?.code ===
|
||||
"runtime_unavailable"
|
||||
? "provider_unavailable"
|
||||
: "provider_missing",
|
||||
remediation:
|
||||
policy.androidAnalysisAvailability?.reason ??
|
||||
"Set REA_JADX_MCP_JAR to a caller-supplied jadx-headless-mcp 0.7.1 JAR and provide a full JDK on Linux, macOS, or Windows x64 with its matching bundled native controls.",
|
||||
|
|
|
|||
|
|
@ -167,7 +167,7 @@ export const parseZcodeFallbackConfiguration = (
|
|||
? {}
|
||||
: requireObject(document.mcpServers, "ZCode fallback mcpServers"),
|
||||
serversPath: ["mcpServers"],
|
||||
dialect: "json",
|
||||
dialect: "zcode_fallback",
|
||||
legacyServers: {},
|
||||
};
|
||||
};
|
||||
|
|
@ -183,6 +183,7 @@ export const parseClientConfiguration = (
|
|||
if (format === undefined || format === "unsupported")
|
||||
throw new TypeError("client does not have a supported MCP config format");
|
||||
// DeepSeek Harness stores a Cordis patch list, not a server table.
|
||||
if (format === "zcode_fallback") return parseZcodeFallbackConfiguration(text);
|
||||
if (format === "dsh") return parseDshCordisDocument(text);
|
||||
const document = parseDocument(text, format);
|
||||
const serversKey = clientConfigurationServersKey(format);
|
||||
|
|
|
|||
|
|
@ -11,6 +11,21 @@ import {
|
|||
import type { SetupClient } from "./SupportedClients.js";
|
||||
import { clientRegistrationEnvironment } from "./ClientRegistrationEnvironment.js";
|
||||
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
|
||||
import type { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
|
||||
/** A parsed layer retains its path so rereading the write target replaces only that layer. */
|
||||
export interface ClientConfigurationSource extends ClientConfigurationDocument {
|
||||
readonly sourcePath: string;
|
||||
}
|
||||
|
||||
const invalidEnvironment = (
|
||||
path: string,
|
||||
error: ConfigurationError,
|
||||
): ClientConfigurationFileError => ({
|
||||
kind: "malformed",
|
||||
path,
|
||||
detail: `Configuration ${path} has an invalid REA server environment: ${error.settings.map(({ setting, constraint }) => `${setting}: ${constraint}`).join("; ")}. Repair it before rerunning setup.`,
|
||||
});
|
||||
|
||||
/** A named source failure that setup can explain without losing the bad path. */
|
||||
export interface ClientConfigurationFileError {
|
||||
|
|
@ -74,9 +89,9 @@ export const readClientConfigurationFiles = async (
|
|||
client: SetupClient,
|
||||
resources?: ArtifactResourceScope,
|
||||
): Promise<
|
||||
Result<readonly ClientConfigurationDocument[], ClientConfigurationFileError>
|
||||
Result<readonly ClientConfigurationSource[], ClientConfigurationFileError>
|
||||
> => {
|
||||
const documents: ClientConfigurationDocument[] = [];
|
||||
const documents: ClientConfigurationSource[] = [];
|
||||
for (const path of client.configPaths ?? [client.configPath]) {
|
||||
const document = await readConfigurationFile(
|
||||
path,
|
||||
|
|
@ -85,16 +100,21 @@ export const readClientConfigurationFiles = async (
|
|||
);
|
||||
if (!document.ok) return document;
|
||||
if (document.value !== undefined) {
|
||||
const environment = clientRegistrationEnvironment(document.value);
|
||||
const environment = clientRegistrationEnvironment(
|
||||
document.value,
|
||||
{},
|
||||
"fragment",
|
||||
);
|
||||
if (!environment.ok)
|
||||
return err({
|
||||
kind: "malformed",
|
||||
path,
|
||||
detail: `Configuration ${path} has an invalid REA server environment: ${environment.error.settings.map(({ setting, constraint }) => `${setting}: ${constraint}`).join("; ")}. Repair it before rerunning setup.`,
|
||||
});
|
||||
documents.push(document.value);
|
||||
return err(invalidEnvironment(path, environment.error));
|
||||
documents.push({ ...document.value, sourcePath: path });
|
||||
}
|
||||
}
|
||||
const environment = clientRegistrationEnvironment(
|
||||
effectiveClientConfiguration(documents),
|
||||
);
|
||||
if (!environment.ok)
|
||||
return err(invalidEnvironment(client.configPath, environment.error));
|
||||
return ok(documents);
|
||||
};
|
||||
|
||||
|
|
@ -117,8 +137,24 @@ const isObject = (value: unknown): value is Record<string, unknown> =>
|
|||
|
||||
/** Project a client's effective user registration without modifying its source files. */
|
||||
export const effectiveClientConfiguration = (
|
||||
documents: readonly ClientConfigurationDocument[],
|
||||
sources: readonly ClientConfigurationSource[],
|
||||
target?: {
|
||||
readonly client: Pick<SetupClient, "configPath" | "configPaths">;
|
||||
readonly document: ClientConfigurationDocument | undefined;
|
||||
},
|
||||
): ClientConfigurationDocument | undefined => {
|
||||
const documents =
|
||||
target === undefined
|
||||
? sources
|
||||
: (target.client.configPaths ?? [target.client.configPath]).flatMap(
|
||||
(path) => {
|
||||
const document =
|
||||
path === target.client.configPath
|
||||
? target.document
|
||||
: sources.find(({ sourcePath }) => sourcePath === path);
|
||||
return document === undefined ? [] : [document];
|
||||
},
|
||||
);
|
||||
const last = documents.at(-1);
|
||||
if (last === undefined) return undefined;
|
||||
if (documents.length === 1) return last;
|
||||
|
|
|
|||
|
|
@ -1,6 +1,9 @@
|
|||
import { z } from "zod";
|
||||
|
||||
import { parseEnvironment } from "../config/environment.js";
|
||||
import {
|
||||
parseEnvironment,
|
||||
parseEnvironmentValues,
|
||||
} from "../config/environment.js";
|
||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import {
|
||||
|
|
@ -15,9 +18,10 @@ import { dshDocument } from "./DshCordisPatch.js";
|
|||
export const clientRegistrationEnvironment = (
|
||||
parsed: ClientConfigurationDocument | undefined,
|
||||
overrides: Readonly<Record<string, string>> = {},
|
||||
validation: "fragment" | "effective" = "effective",
|
||||
): Result<Readonly<Record<string, string>>, ConfigurationError> => {
|
||||
if (parsed?.dialect === "dsh")
|
||||
return dshDocument(parsed.document).environment(overrides);
|
||||
return dshDocument(parsed.document).environment(overrides, validation);
|
||||
const key =
|
||||
parsed?.dialect === "opencode" || parsed?.dialect === "opencode_v2"
|
||||
? "environment"
|
||||
|
|
@ -46,7 +50,10 @@ export const clientRegistrationEnvironment = (
|
|||
// Validate known settings with the runtime's parser, but retain the original
|
||||
// string overrides, including unknown client-specific keys and no defaults.
|
||||
const environment = { ...existing.data, ...overrides };
|
||||
const validated = parseEnvironment(environment);
|
||||
const validated =
|
||||
validation === "fragment"
|
||||
? parseEnvironmentValues(environment)
|
||||
: parseEnvironment(environment);
|
||||
if (!validated.ok) return validated;
|
||||
return ok(
|
||||
Object.fromEntries(
|
||||
|
|
|
|||
|
|
@ -76,18 +76,24 @@ const unwrapCmdCommand = (command: readonly string[]): readonly string[] => {
|
|||
return [];
|
||||
};
|
||||
|
||||
const packageRunnerReference = (
|
||||
/** Preserve the package argument from a supported package-runner command. */
|
||||
export const clientRegistrationPackageReference = (
|
||||
command: readonly string[],
|
||||
): string | undefined => {
|
||||
const arguments_ = unwrapCmdCommand(command);
|
||||
if (
|
||||
command.length === 4 &&
|
||||
command[0] === "npx" &&
|
||||
command[1] === "-y" &&
|
||||
command[3] === "mcp"
|
||||
arguments_.length === 4 &&
|
||||
arguments_[0] === "npx" &&
|
||||
arguments_[1] === "-y" &&
|
||||
arguments_[3] === "mcp"
|
||||
)
|
||||
return command[2];
|
||||
if (command.length === 3 && command[0] === "bunx" && command[2] === "mcp")
|
||||
return command[1];
|
||||
return arguments_[2];
|
||||
if (
|
||||
arguments_.length === 3 &&
|
||||
arguments_[0] === "bunx" &&
|
||||
arguments_[2] === "mcp"
|
||||
)
|
||||
return arguments_[1];
|
||||
return undefined;
|
||||
};
|
||||
|
||||
|
|
@ -129,7 +135,7 @@ export const isOwnedClientRegistrationCommand = (
|
|||
return true;
|
||||
}
|
||||
|
||||
const packageReference = packageRunnerReference(unwrapCmdCommand(command));
|
||||
const packageReference = clientRegistrationPackageReference(command);
|
||||
return (
|
||||
packageReference !== undefined && isOwnedPackageReference(packageReference)
|
||||
);
|
||||
|
|
|
|||
|
|
@ -13,10 +13,12 @@ import { access } from "node:fs/promises";
|
|||
import { resolve } from "node:path";
|
||||
|
||||
import { z } from "zod";
|
||||
import { valid } from "semver";
|
||||
|
||||
import { PRODUCT_IDENTITY } from "../identity.js";
|
||||
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
|
||||
import {
|
||||
clientRegistrationPackageReference,
|
||||
isOwnedClientRegistrationCommand,
|
||||
packageRunnerRegistrationCommand,
|
||||
} from "./ClientRegistrationIdentity.js";
|
||||
|
|
@ -33,6 +35,8 @@ import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.j
|
|||
interface ClientRegistrationStatusBase {
|
||||
readonly client: string;
|
||||
readonly config_path: string;
|
||||
/** Exact package reference requested by npx/bunx, when registration uses one. */
|
||||
readonly package_reference?: string;
|
||||
}
|
||||
|
||||
type RegistrationCommand = readonly [string, ...string[]];
|
||||
|
|
@ -201,12 +205,18 @@ export const readClientRegistrationInspections = async (
|
|||
registration.command,
|
||||
...registration.args,
|
||||
];
|
||||
const packageReference = clientRegistrationPackageReference(command);
|
||||
const activation = registrationActivation(registration, client, parsed);
|
||||
const packageVersion = exactRegisteredPackageVersion(packageReference);
|
||||
const packageVersionDrift =
|
||||
packageVersion !== undefined &&
|
||||
packageVersion !== PRODUCT_IDENTITY.packageVersion;
|
||||
record(
|
||||
configuredStatus(
|
||||
client,
|
||||
command,
|
||||
activation === "enabled" &&
|
||||
!packageVersionDrift &&
|
||||
registrationAligned(
|
||||
registration,
|
||||
client,
|
||||
|
|
@ -217,7 +227,11 @@ export const readClientRegistrationInspections = async (
|
|||
policyBlock === undefined
|
||||
? "aligned"
|
||||
: "stale",
|
||||
policyBlock,
|
||||
policyBlock ??
|
||||
(packageVersionDrift
|
||||
? packageVersionRemediation(packageReference ?? "", client)
|
||||
: undefined),
|
||||
packageReference,
|
||||
),
|
||||
activation,
|
||||
);
|
||||
|
|
@ -378,12 +392,16 @@ const configuredStatus = (
|
|||
command: RegistrationCommand,
|
||||
state: "aligned" | "stale",
|
||||
policyRemediation?: string,
|
||||
packageReference?: string,
|
||||
): ClientRegistrationStatus =>
|
||||
state === "aligned"
|
||||
? {
|
||||
client: client.name,
|
||||
config_path: client.configPath,
|
||||
command,
|
||||
...(packageReference === undefined
|
||||
? {}
|
||||
: { package_reference: packageReference }),
|
||||
state,
|
||||
remediation: null,
|
||||
}
|
||||
|
|
@ -391,10 +409,29 @@ const configuredStatus = (
|
|||
client: client.name,
|
||||
config_path: client.configPath,
|
||||
command,
|
||||
...(packageReference === undefined
|
||||
? {}
|
||||
: { package_reference: packageReference }),
|
||||
state,
|
||||
remediation: policyRemediation ?? remediation(client),
|
||||
};
|
||||
|
||||
const exactRegisteredPackageVersion = (
|
||||
packageReference: string | undefined,
|
||||
): string | undefined => {
|
||||
if (packageReference === undefined) return undefined;
|
||||
const prefix = `${PRODUCT_IDENTITY.packageName}@`;
|
||||
if (!packageReference.startsWith(prefix)) return undefined;
|
||||
const version = packageReference.slice(prefix.length);
|
||||
return valid(version) === version ? version : undefined;
|
||||
};
|
||||
|
||||
const packageVersionRemediation = (
|
||||
packageReference: string,
|
||||
client: SetupClient,
|
||||
): string =>
|
||||
`This registration requests ${packageReference}, while the current REA CLI package is ${PRODUCT_IDENTITY.packageVersion}. REA does not resolve package-runner requests during doctor; update the registration with rea setup --client ${client.name}, then restart the client.`;
|
||||
|
||||
const unavailableStatus = (
|
||||
client: SetupClient,
|
||||
state: "missing" | "invalid",
|
||||
|
|
|
|||
|
|
@ -42,6 +42,7 @@ import {
|
|||
workflowAnalysisProfile,
|
||||
} from "./InvestigationProviders.js";
|
||||
import {
|
||||
executeComposedWorkflow,
|
||||
createWorkflowEvidence,
|
||||
workflowSnapshotRecord,
|
||||
recordWorkflowUnknowns,
|
||||
|
|
@ -401,18 +402,17 @@ const executeAnalysisTool = async (input: {
|
|||
tool === "trace_native_ui_action"
|
||||
) {
|
||||
const { EnhancedTools } = await import("./EnhancedTools.js");
|
||||
const result = await new EnhancedTools(session).execute(
|
||||
tool,
|
||||
input.arguments,
|
||||
signal,
|
||||
const result = await executeComposedWorkflow(session, (observed) =>
|
||||
new EnhancedTools(observed).execute(tool, input.arguments, signal),
|
||||
);
|
||||
if (!result.ok) return { output: cliError(result.error) };
|
||||
const evidence = createWorkflowEvidence({
|
||||
target: input.openedTarget,
|
||||
operation: tool,
|
||||
parameters: input.arguments,
|
||||
result: result.value,
|
||||
result: result.value.result,
|
||||
upstreamProfile: evidenceProfile,
|
||||
upstreamLimitations: result.value.limitations,
|
||||
});
|
||||
return { output: evidence, evidence };
|
||||
}
|
||||
|
|
|
|||
|
|
@ -588,6 +588,9 @@ describe("doctor Linux and Hopper discovery", () => {
|
|||
result.checks.find(({ name }) => name === "hopper-version")?.remediation,
|
||||
).toContain("Unset or update HOPPER_LAUNCHER_PATH");
|
||||
});
|
||||
});
|
||||
|
||||
describe("doctor installed Hopper discovery", () => {
|
||||
it("detects a Homebrew cask installed outside /Applications", async () => {
|
||||
const path =
|
||||
"/opt/homebrew/Caskroom/hopper/Hopper Disassembler.app/Contents/MacOS/hopper";
|
||||
|
|
@ -612,7 +615,9 @@ describe("doctor Linux and Hopper discovery", () => {
|
|||
);
|
||||
expect(result.hopperPath).toBe(path);
|
||||
});
|
||||
});
|
||||
|
||||
describe("doctor target admission on Linux", () => {
|
||||
it("uses shared app target validation when a target is supplied", async () => {
|
||||
const result = await runDoctor(
|
||||
"/Applications/Notes.app",
|
||||
|
|
@ -652,6 +657,30 @@ describe("doctor Linux and Hopper discovery", () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe("doctor standard Linux Hopper paths", () => {
|
||||
it.each(["/opt/hopper-v4/bin/hopper", "/usr/bin/hopper"])(
|
||||
"detects Hopper at the standard path %s",
|
||||
async (path) => {
|
||||
const result = await runDoctor(
|
||||
undefined,
|
||||
host({
|
||||
platform: "linux",
|
||||
macosVersion: () => Promise.resolve(undefined),
|
||||
linuxDistribution: () =>
|
||||
Promise.resolve({
|
||||
id: "ubuntu",
|
||||
versionId: "24.04",
|
||||
packageFamily: "deb",
|
||||
hopperSetupSupported: true,
|
||||
}),
|
||||
executable: (candidate) => Promise.resolve(candidate === path),
|
||||
}),
|
||||
);
|
||||
expect(result.hopperPath).toBe(path);
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("doctor target admission diagnostics", () => {
|
||||
it("preserves route-specific target diagnostics from the host", async () => {
|
||||
const targetCheck = {
|
||||
|
|
|
|||
|
|
@ -48,6 +48,7 @@ export interface DoctorCheck {
|
|||
readonly classification:
|
||||
| "healthy"
|
||||
| "missing_dependency"
|
||||
| "runtime_failure"
|
||||
| "unsupported_host"
|
||||
| "unsupported_target"
|
||||
| "missing_analysis_engine"
|
||||
|
|
@ -65,6 +66,7 @@ export interface DoctorProviderCheck {
|
|||
readonly detail: string;
|
||||
readonly remediation: string | null;
|
||||
readonly classification: Exclude<DoctorCheck["classification"], "healthy">;
|
||||
readonly details?: Readonly<Record<string, JsonValue>>;
|
||||
}
|
||||
|
||||
/** Read-only provider inspection supplied by an outer composition adapter. */
|
||||
|
|
|
|||
|
|
@ -2,7 +2,11 @@ import {
|
|||
SUPPORTED_NODE_VERSION_PROSE,
|
||||
supportsNodeVersion,
|
||||
} from "../domain/runtimeVersion.js";
|
||||
import { linuxHopperLauncherPath, supportsReaHost } from "./LinuxHopper.js";
|
||||
import {
|
||||
linuxHopperLauncherCandidates,
|
||||
systemLinuxHopperLauncherCandidates,
|
||||
supportsReaHost,
|
||||
} from "./LinuxHopper.js";
|
||||
import type {
|
||||
DoctorCheck,
|
||||
DoctorHost,
|
||||
|
|
@ -11,7 +15,6 @@ import type {
|
|||
|
||||
const DEFAULT_HOPPER =
|
||||
"/Applications/Hopper Disassembler.app/Contents/MacOS/hopper";
|
||||
const SYSTEM_LINUX_HOPPER = "/opt/hopper/bin/Hopper";
|
||||
|
||||
/** Ordered environment diagnostics consumed by the Doctor report assembler. */
|
||||
export interface DoctorDiagnostics {
|
||||
|
|
@ -148,8 +151,9 @@ const findHopper = async (host: DoctorHost): Promise<string | undefined> => {
|
|||
host.configuredHopperPath === undefined
|
||||
? [
|
||||
DEFAULT_HOPPER,
|
||||
SYSTEM_LINUX_HOPPER,
|
||||
linuxHopperLauncherPath(host.homeDirectory),
|
||||
...(host.platform === "linux"
|
||||
? linuxHopperLauncherCandidates(host.homeDirectory)
|
||||
: [...systemLinuxHopperLauncherCandidates]),
|
||||
...(await host.manualHopperPaths()),
|
||||
]
|
||||
: [host.configuredHopperPath];
|
||||
|
|
@ -280,6 +284,7 @@ const providerDoctorChecks = (
|
|||
...(candidate.remediation === null
|
||||
? {}
|
||||
: { remediation: candidate.remediation }),
|
||||
...(candidate.details === undefined ? {} : { details: candidate.details }),
|
||||
}));
|
||||
|
||||
const unsupportedHopperRemediation = (configured: boolean): string =>
|
||||
|
|
|
|||
|
|
@ -1,2 +1,2 @@
|
|||
/** Deep-analysis providers whose readiness can be selected explicitly by doctor. */
|
||||
export const DOCTOR_PROVIDER_IDS = ["hopper", "ghidra", "ida"] as const;
|
||||
export const DOCTOR_PROVIDER_IDS = ["hopper", "ghidra", "ida", "jadx"] as const;
|
||||
|
|
|
|||
|
|
@ -1,5 +1,8 @@
|
|||
import { isMap, isSeq, type Node, type YAMLMap } from "yaml";
|
||||
import { parseEnvironment } from "../config/environment.js";
|
||||
import {
|
||||
parseEnvironment,
|
||||
parseEnvironmentValues,
|
||||
} from "../config/environment.js";
|
||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import type { ClientConfigurationDocument } from "./ClientConfigurationDocument.js";
|
||||
|
|
@ -181,6 +184,7 @@ export class DshCordisDocument {
|
|||
/** Validate literals without treating expression source as an environment value. */
|
||||
environment(
|
||||
overrides: Readonly<Record<string, string>> = {},
|
||||
validation: "fragment" | "effective" = "effective",
|
||||
): Result<Readonly<Record<string, string>>, ConfigurationError> {
|
||||
const config = this.#field("config");
|
||||
if (config instanceof CordisExpression)
|
||||
|
|
@ -215,8 +219,12 @@ export class DshCordisDocument {
|
|||
new ConfigurationError("Cordis server environment must be a mapping"),
|
||||
);
|
||||
const literals: Record<string, string> = {};
|
||||
const unresolvedSettings = new Set<string>();
|
||||
for (const [key, value] of Object.entries(raw ?? {})) {
|
||||
if (value instanceof CordisExpression) continue;
|
||||
if (value instanceof CordisExpression) {
|
||||
if (overrides[key] === undefined) unresolvedSettings.add(key);
|
||||
continue;
|
||||
}
|
||||
if (typeof value !== "string")
|
||||
return err(
|
||||
new ConfigurationError("Invalid Cordis server environment", {
|
||||
|
|
@ -232,7 +240,10 @@ export class DshCordisDocument {
|
|||
literals[key] = value;
|
||||
}
|
||||
const environment = { ...literals, ...overrides };
|
||||
const validated = parseEnvironment(environment);
|
||||
const validated =
|
||||
validation === "fragment"
|
||||
? parseEnvironmentValues(environment)
|
||||
: parseEnvironment(environment, unresolvedSettings);
|
||||
return validated.ok ? ok(environment) : validated;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -525,33 +525,47 @@ export class EnhancedTools {
|
|||
this.#call("list_strings", {}, signal),
|
||||
],
|
||||
);
|
||||
if (!segmentsResult.ok) return segmentsResult;
|
||||
if (
|
||||
!segmentsResult.ok &&
|
||||
!(segmentsResult.error instanceof AnalysisCapabilityUnavailableError)
|
||||
)
|
||||
return segmentsResult;
|
||||
if (!proceduresResult.ok) return proceduresResult;
|
||||
if (!stringsResult.ok) return stringsResult;
|
||||
|
||||
const segments = parseSegments(segmentsResult.value);
|
||||
if (!segments.ok) return segments;
|
||||
const segments = segmentsResult.ok
|
||||
? parseSegments(segmentsResult.value)
|
||||
: undefined;
|
||||
if (segments !== undefined && !segments.ok) return segments;
|
||||
const procedureCount = parseListCount(proceduresResult.value, "procedures");
|
||||
if (!procedureCount.ok) return procedureCount;
|
||||
const stringCount = parseListCount(stringsResult.value, "strings");
|
||||
if (!stringCount.ok) return stringCount;
|
||||
|
||||
const imageBase = parseSegmentImageBase(segments.value);
|
||||
const observedSegments = segments?.ok ? segments.value : undefined;
|
||||
const imageBase =
|
||||
observedSegments === undefined
|
||||
? undefined
|
||||
: parseSegmentImageBase(observedSegments);
|
||||
|
||||
return ok({
|
||||
document: document.value,
|
||||
...(imageBase === undefined ? {} : { image_base: imageBase }),
|
||||
segments: segments.value.map(
|
||||
({ name, start, end, image_base, address_space }) => ({
|
||||
name,
|
||||
start,
|
||||
end,
|
||||
length: addressDistance(start, end),
|
||||
...(image_base === undefined ? {} : { image_base }),
|
||||
...(address_space === undefined ? {} : { address_space }),
|
||||
}),
|
||||
),
|
||||
segment_count: segments.value.length,
|
||||
...(observedSegments === undefined
|
||||
? {}
|
||||
: {
|
||||
segments: observedSegments.map(
|
||||
({ name, start, end, image_base, address_space }) => ({
|
||||
name,
|
||||
start,
|
||||
end,
|
||||
length: addressDistance(start, end),
|
||||
...(image_base === undefined ? {} : { image_base }),
|
||||
...(address_space === undefined ? {} : { address_space }),
|
||||
}),
|
||||
),
|
||||
segment_count: observedSegments.length,
|
||||
}),
|
||||
procedure_count: procedureCount.value,
|
||||
string_count: stringCount.value,
|
||||
});
|
||||
|
|
|
|||
|
|
@ -2,19 +2,36 @@ import { createHash } from "node:crypto";
|
|||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
formatCommandForDisplay,
|
||||
installLinuxHopper,
|
||||
linuxHopperLauncherDigestSupported,
|
||||
linuxPackageManagerCommands,
|
||||
linuxSharedLibrariesAvailable,
|
||||
parseLinuxDistribution,
|
||||
supportsReaHost,
|
||||
LinuxHopperTemporaryCleanupOwner,
|
||||
type LinuxDistribution,
|
||||
type LinuxHopperDownload,
|
||||
type LinuxHopperInstallHost,
|
||||
type LinuxHopperLauncherStatus,
|
||||
type LinuxPackageFamily,
|
||||
type LinuxPackageInstallFailure,
|
||||
type LinuxPackageInstallOutcome,
|
||||
} from "./LinuxHopper.js";
|
||||
|
||||
describe("Linux package-manager command display", () => {
|
||||
it("preserves argument boundaries for paths with spaces and quotes", () => {
|
||||
expect(
|
||||
formatCommandForDisplay("pkexec", [
|
||||
"pacman",
|
||||
"-U",
|
||||
"/tmp/rea test/hopper.arch",
|
||||
"apostrophe'path",
|
||||
]),
|
||||
).toBe('pkexec pacman -U "/tmp/rea test/hopper.arch" "apostrophe\'path"');
|
||||
});
|
||||
});
|
||||
|
||||
class RecordingLinuxHost implements LinuxHopperInstallHost {
|
||||
distributionValue: LinuxDistribution | undefined = {
|
||||
id: "ubuntu",
|
||||
|
|
@ -25,6 +42,14 @@ class RecordingLinuxHost implements LinuxHopperInstallHost {
|
|||
archive = new Uint8Array([1, 2, 3]);
|
||||
archiveOk = true;
|
||||
installSucceeds = true;
|
||||
cleanupFails = false;
|
||||
packageFailure: LinuxPackageInstallFailure = {
|
||||
command: "pacman -U --noconfirm /tmp/rea test/hopper.arch",
|
||||
exitCode: 1,
|
||||
stdout: "",
|
||||
stderr: "error: failed to prepare transaction",
|
||||
outputTruncated: false,
|
||||
};
|
||||
launcherStatusValue: LinuxHopperLauncherStatus = "ready";
|
||||
downloads: string[] = [];
|
||||
installed: Array<{ family: LinuxPackageFamily; archive: string }> = [];
|
||||
|
|
@ -42,15 +67,20 @@ class RecordingLinuxHost implements LinuxHopperInstallHost {
|
|||
installPackage = (
|
||||
family: LinuxPackageFamily,
|
||||
archive: string,
|
||||
): Promise<boolean> => {
|
||||
): Promise<LinuxPackageInstallOutcome> => {
|
||||
this.installed.push({ family, archive });
|
||||
return Promise.resolve(this.installSucceeds);
|
||||
const outcome: LinuxPackageInstallOutcome = this.installSucceeds
|
||||
? { status: "installed" }
|
||||
: { status: "failed", failure: this.packageFailure };
|
||||
return Promise.resolve(outcome);
|
||||
};
|
||||
launcherStatus = (): Promise<LinuxHopperLauncherStatus> =>
|
||||
Promise.resolve(this.launcherStatusValue);
|
||||
cleanup = (path: string): Promise<void> => {
|
||||
this.cleaned.push(path);
|
||||
return Promise.resolve();
|
||||
return this.cleanupFails
|
||||
? Promise.reject(new Error("temporary path is busy"))
|
||||
: Promise.resolve();
|
||||
};
|
||||
}
|
||||
|
||||
|
|
@ -183,7 +213,9 @@ describe("Linux Hopper installation", () => {
|
|||
);
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("Linux Hopper package installation", () => {
|
||||
it.each(["deb", "rpm", "arch"] as const)(
|
||||
"downloads, verifies, installs, reads back, and cleans a %s package",
|
||||
async (family) => {
|
||||
|
|
@ -204,7 +236,9 @@ describe("Linux Hopper installation", () => {
|
|||
expect(host.cleaned).toEqual(["/tmp/rea test"]);
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("Linux Hopper installation failure classification", () => {
|
||||
it.each([
|
||||
["unsupported host", "unsupported_host"],
|
||||
["package request failure", "download"],
|
||||
|
|
@ -232,11 +266,65 @@ describe("Linux Hopper installation", () => {
|
|||
expect(await installFixture(host)).toEqual({
|
||||
status: "failed",
|
||||
reason,
|
||||
...(scenario === "package manager failure"
|
||||
? { packageFailure: host.packageFailure }
|
||||
: {}),
|
||||
});
|
||||
if (host.installed.length > 0)
|
||||
expect(host.cleaned).toEqual(["/tmp/rea test"]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("Linux Hopper cleanup and failure details", () => {
|
||||
it("preserves the failing package-manager command and its output", async () => {
|
||||
const host = new RecordingLinuxHost();
|
||||
host.installSucceeds = false;
|
||||
host.packageFailure = {
|
||||
command: "pkexec pacman -U --noconfirm /tmp/rea test/hopper.arch",
|
||||
exitCode: 1,
|
||||
stdout: "loading packages...",
|
||||
stderr:
|
||||
"\u001b[31merror: unresolvable package conflicts detected\u001b[0m",
|
||||
outputTruncated: false,
|
||||
};
|
||||
expect(await installFixture(host)).toEqual({
|
||||
status: "failed",
|
||||
reason: "authorization_or_package_manager",
|
||||
packageFailure: host.packageFailure,
|
||||
});
|
||||
expect(host.cleaned).toEqual(["/tmp/rea test"]);
|
||||
});
|
||||
|
||||
it("preserves a successful install result when temporary cleanup fails", async () => {
|
||||
const host = new RecordingLinuxHost();
|
||||
host.cleanupFails = true;
|
||||
expect(await installFixture(host)).toEqual({
|
||||
status: "installed",
|
||||
launcherPath: "/opt/hopper/bin/Hopper",
|
||||
cleanupFailure: {
|
||||
path: "/tmp/rea test",
|
||||
reason: "temporary path is busy",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("preserves an install failure when temporary cleanup also fails", async () => {
|
||||
const host = new RecordingLinuxHost();
|
||||
host.installSucceeds = false;
|
||||
host.cleanupFails = true;
|
||||
expect(await installFixture(host)).toMatchObject({
|
||||
status: "failed",
|
||||
reason: "authorization_or_package_manager",
|
||||
packageFailure: host.packageFailure,
|
||||
cleanupFailure: {
|
||||
path: "/tmp/rea test",
|
||||
reason: "temporary path is busy",
|
||||
},
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("Linux Hopper cancellation", () => {
|
||||
it("cancels before network or mutation", async () => {
|
||||
const host = new RecordingLinuxHost();
|
||||
const controller = new AbortController();
|
||||
|
|
@ -252,6 +340,27 @@ describe("Linux Hopper installation", () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe("Linux Hopper temporary cleanup ownership", () => {
|
||||
it("retains a failed cleanup for a later successful retry", async () => {
|
||||
const owner = new LinuxHopperTemporaryCleanupOwner();
|
||||
owner.retain({ path: "/tmp/rea-hopper", reason: "busy" });
|
||||
let attempts = 0;
|
||||
|
||||
expect(
|
||||
await owner.retry(async () => {
|
||||
attempts += 1;
|
||||
throw new Error("still busy");
|
||||
}),
|
||||
).toContain("/tmp/rea-hopper");
|
||||
expect(
|
||||
await owner.retry(async () => {
|
||||
attempts += 1;
|
||||
}),
|
||||
).toBeUndefined();
|
||||
expect(attempts).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
function releasesFor(
|
||||
bytes: Uint8Array,
|
||||
): Record<
|
||||
|
|
|
|||
|
|
@ -1,18 +1,18 @@
|
|||
import { createHash } from "node:crypto";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createReadStream } from "node:fs";
|
||||
import { access, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import { execFileBoundedOutput } from "../process/ExecFileBoundedOutput.js";
|
||||
import { execFileOutput } from "../process/ExecFileOutput.js";
|
||||
import {
|
||||
downloadHopperPackage,
|
||||
HOPPER_DOWNLOAD_BYTES,
|
||||
} from "./HopperDownload.js";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
/** Package-manager output retained per stream without stopping the command. */
|
||||
const PACKAGE_MANAGER_OUTPUT_BYTES = 64 * 1024;
|
||||
const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/";
|
||||
const SUPPORTED_HOPPER_SHA256 = new Set([
|
||||
"0294ced141cc373468ee22d8343e7dac41980cb05a937994ca81c9f09afe7ded",
|
||||
|
|
@ -67,7 +67,7 @@ export const linuxHopperInstallDisclosure = (
|
|||
family,
|
||||
`<verified-hopper.${family}>`,
|
||||
isRoot,
|
||||
).map(({ executable, args }) => `${executable} ${args.join(" ")}`),
|
||||
).map(({ executable, args }) => formatCommandForDisplay(executable, args)),
|
||||
};
|
||||
};
|
||||
|
||||
|
|
@ -111,6 +111,25 @@ export interface LinuxHopperDownload {
|
|||
readonly bytes: Uint8Array;
|
||||
}
|
||||
|
||||
/** One failed package-manager command, with the output that explains it. */
|
||||
export interface LinuxPackageInstallFailure {
|
||||
/** The executed executable and arguments, joined for display. */
|
||||
readonly command: string;
|
||||
readonly exitCode: string | number | null;
|
||||
readonly stdout: string;
|
||||
readonly stderr: string;
|
||||
/** Whether the command produced more output than was retained. */
|
||||
readonly outputTruncated: boolean;
|
||||
}
|
||||
|
||||
/** Whether the package-manager step installed the verified package. */
|
||||
export type LinuxPackageInstallOutcome =
|
||||
| { readonly status: "installed" }
|
||||
| {
|
||||
readonly status: "failed";
|
||||
readonly failure: LinuxPackageInstallFailure;
|
||||
};
|
||||
|
||||
/** External effects required to install the official Linux package. */
|
||||
export interface LinuxHopperInstallHost {
|
||||
distribution(): Promise<LinuxDistribution | undefined>;
|
||||
|
|
@ -120,15 +139,20 @@ export interface LinuxHopperInstallHost {
|
|||
): Promise<LinuxHopperDownload>;
|
||||
createTemporaryDirectory(): Promise<string>;
|
||||
writeArchive(path: string, bytes: Uint8Array): Promise<void>;
|
||||
installPackage(family: LinuxPackageFamily, archive: string): Promise<boolean>;
|
||||
installPackage(
|
||||
family: LinuxPackageFamily,
|
||||
archive: string,
|
||||
): Promise<LinuxPackageInstallOutcome>;
|
||||
launcherStatus(path: string): Promise<LinuxHopperLauncherStatus>;
|
||||
cleanup(path: string): Promise<void>;
|
||||
}
|
||||
|
||||
/** Structured outcome for every expected Linux installation failure. */
|
||||
export type LinuxHopperInstallResult =
|
||||
| { readonly status: "installed"; readonly launcherPath: string }
|
||||
| {
|
||||
| ({ readonly status: "installed"; readonly launcherPath: string } & {
|
||||
readonly cleanupFailure?: LinuxHopperTemporaryCleanupFailure;
|
||||
})
|
||||
| ({
|
||||
readonly status: "failed";
|
||||
readonly reason:
|
||||
| "unsupported_host"
|
||||
|
|
@ -140,7 +164,49 @@ export type LinuxHopperInstallResult =
|
|||
| "runtime_dependencies"
|
||||
| "unsupported_hopper_build"
|
||||
| "cancelled";
|
||||
};
|
||||
/**
|
||||
* The failing package-manager command and its output. Present only when
|
||||
* the package-manager step itself failed, so setup can report what the
|
||||
* package manager said instead of assuming an authorization problem.
|
||||
*/
|
||||
readonly packageFailure?: LinuxPackageInstallFailure;
|
||||
} & { readonly cleanupFailure?: LinuxHopperTemporaryCleanupFailure });
|
||||
|
||||
/** Failed removal of an owned temporary install directory. */
|
||||
export interface LinuxHopperTemporaryCleanupFailure {
|
||||
readonly path: string;
|
||||
readonly reason: string;
|
||||
}
|
||||
|
||||
/** Own failed temporary cleanup until a later retry succeeds. */
|
||||
export class LinuxHopperTemporaryCleanupOwner {
|
||||
private readonly pending = new Map<
|
||||
string,
|
||||
LinuxHopperTemporaryCleanupFailure
|
||||
>();
|
||||
|
||||
retain(failure: LinuxHopperTemporaryCleanupFailure): void {
|
||||
this.pending.set(failure.path, failure);
|
||||
}
|
||||
|
||||
async retry(
|
||||
cleanup: (path: string) => Promise<void>,
|
||||
): Promise<string | undefined> {
|
||||
const failures: string[] = [];
|
||||
for (const path of this.pending.keys()) {
|
||||
try {
|
||||
await cleanup(path);
|
||||
this.pending.delete(path);
|
||||
} catch (cause: unknown) {
|
||||
const reason = cause instanceof Error ? cause.message : String(cause);
|
||||
failures.push(
|
||||
`Linux Hopper temporary cleanup failed for ${path}: ${reason}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return failures.length === 0 ? undefined : failures.join("; ");
|
||||
}
|
||||
}
|
||||
|
||||
export type LinuxHopperLauncherStatus =
|
||||
| "ready"
|
||||
|
|
@ -215,33 +281,57 @@ export const installLinuxHopper = async (
|
|||
if (distribution?.hopperSetupSupported !== true)
|
||||
return { status: "failed", reason: "unsupported_host" };
|
||||
let temporary: string | undefined;
|
||||
let result: LinuxHopperInstallResult;
|
||||
try {
|
||||
const release = (options.releases ?? SUPPORTED_RELEASES)[
|
||||
distribution.packageFamily
|
||||
];
|
||||
const archiveDownload = await host.download(release.filename, options);
|
||||
if (!archiveDownload.ok) return { status: "failed", reason: "download" };
|
||||
if (!packageIntegrityMatches(archiveDownload.bytes, release))
|
||||
return { status: "failed", reason: "integrity" };
|
||||
temporary = await host.createTemporaryDirectory();
|
||||
const archive = join(temporary, `hopper.${distribution.packageFamily}`);
|
||||
await host.writeArchive(archive, archiveDownload.bytes);
|
||||
if (!(await host.installPackage(distribution.packageFamily, archive)))
|
||||
return { status: "failed", reason: "authorization_or_package_manager" };
|
||||
const launcherPath = "/opt/hopper/bin/Hopper";
|
||||
const launcherStatus = await host.launcherStatus(launcherPath);
|
||||
if (launcherStatus !== "ready")
|
||||
return {
|
||||
status: "failed",
|
||||
reason:
|
||||
launcherStatus === "missing" ? "launcher_missing" : launcherStatus,
|
||||
};
|
||||
return { status: "installed", launcherPath };
|
||||
result = await (async (): Promise<LinuxHopperInstallResult> => {
|
||||
const release = (options.releases ?? SUPPORTED_RELEASES)[
|
||||
distribution.packageFamily
|
||||
];
|
||||
const archiveDownload = await host.download(release.filename, options);
|
||||
if (!archiveDownload.ok) return { status: "failed", reason: "download" };
|
||||
if (!packageIntegrityMatches(archiveDownload.bytes, release))
|
||||
return { status: "failed", reason: "integrity" };
|
||||
temporary = await host.createTemporaryDirectory();
|
||||
const archive = join(temporary, `hopper.${distribution.packageFamily}`);
|
||||
await host.writeArchive(archive, archiveDownload.bytes);
|
||||
const install = await host.installPackage(
|
||||
distribution.packageFamily,
|
||||
archive,
|
||||
);
|
||||
if (install.status === "failed")
|
||||
return {
|
||||
status: "failed",
|
||||
reason: "authorization_or_package_manager",
|
||||
packageFailure: install.failure,
|
||||
};
|
||||
const launcherPath = "/opt/hopper/bin/Hopper";
|
||||
const launcherStatus = await host.launcherStatus(launcherPath);
|
||||
if (launcherStatus !== "ready")
|
||||
return {
|
||||
status: "failed",
|
||||
reason:
|
||||
launcherStatus === "missing" ? "launcher_missing" : launcherStatus,
|
||||
};
|
||||
return { status: "installed", launcherPath };
|
||||
})();
|
||||
} catch (cause: unknown) {
|
||||
if (isAbortCause(cause)) return { status: "failed", reason: "cancelled" };
|
||||
return { status: "failed", reason: "download" };
|
||||
} finally {
|
||||
if (temporary !== undefined) await host.cleanup(temporary);
|
||||
result = {
|
||||
status: "failed",
|
||||
reason: isAbortCause(cause) ? "cancelled" : "download",
|
||||
};
|
||||
}
|
||||
if (temporary === undefined) return result;
|
||||
try {
|
||||
await host.cleanup(temporary);
|
||||
return result;
|
||||
} catch (cause: unknown) {
|
||||
return {
|
||||
...result,
|
||||
cleanupFailure: {
|
||||
path: temporary,
|
||||
reason: cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
};
|
||||
}
|
||||
};
|
||||
|
||||
|
|
@ -249,6 +339,24 @@ export const installLinuxHopper = async (
|
|||
export const linuxHopperLauncherPath = (home: string): string =>
|
||||
join(home, ".local/share/rea/hopper/bin/Hopper");
|
||||
|
||||
/** REA's managed Linux install path, preferred when no explicit launcher is set. */
|
||||
export const REA_MANAGED_LINUX_HOPPER_LAUNCHER = "/opt/hopper/bin/Hopper";
|
||||
|
||||
/** Other system-wide Hopper paths used by package installations. */
|
||||
export const systemLinuxHopperLauncherCandidates: readonly string[] = [
|
||||
"/opt/hopper-v4/bin/hopper",
|
||||
"/usr/bin/hopper",
|
||||
];
|
||||
|
||||
/** Probe REA-managed and user-local installs before arbitrary system builds. */
|
||||
export const linuxHopperLauncherCandidates = (
|
||||
home: string,
|
||||
): readonly string[] => [
|
||||
REA_MANAGED_LINUX_HOPPER_LAUNCHER,
|
||||
linuxHopperLauncherPath(home),
|
||||
...systemLinuxHopperLauncherCandidates,
|
||||
];
|
||||
|
||||
/** Compose package-manager and linker checks from the selected caller environment. */
|
||||
export const systemLinuxHopperInstallHost = (
|
||||
environment: Readonly<NodeJS.ProcessEnv>,
|
||||
|
|
@ -304,24 +412,49 @@ const installSystemPackage = async (
|
|||
family: LinuxPackageFamily,
|
||||
archive: string,
|
||||
environment: Readonly<NodeJS.ProcessEnv>,
|
||||
): Promise<boolean> => {
|
||||
try {
|
||||
for (const command of linuxPackageManagerCommands(
|
||||
family,
|
||||
archive,
|
||||
process.getuid?.() === 0,
|
||||
))
|
||||
await execFileAsync(command.executable, command.args, {
|
||||
): Promise<LinuxPackageInstallOutcome> => {
|
||||
for (const command of linuxPackageManagerCommands(
|
||||
family,
|
||||
archive,
|
||||
process.getuid?.() === 0,
|
||||
)) {
|
||||
const output = await execFileBoundedOutput(
|
||||
command.executable,
|
||||
command.args,
|
||||
{
|
||||
env: environment,
|
||||
});
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
// Package-manager failure is reported by the false return.
|
||||
void cause;
|
||||
return false;
|
||||
limit: PACKAGE_MANAGER_OUTPUT_BYTES,
|
||||
},
|
||||
);
|
||||
if (output.exitCode !== 0) {
|
||||
return {
|
||||
status: "failed",
|
||||
failure: {
|
||||
command: formatCommandForDisplay(command.executable, command.args),
|
||||
exitCode: output.exitCode,
|
||||
// The streams stay separate; their original interleaved order
|
||||
// is not recoverable after capture.
|
||||
stdout: output.stdout,
|
||||
stderr: output.stderr,
|
||||
outputTruncated: output.outputTruncated,
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
return { status: "installed" };
|
||||
};
|
||||
|
||||
/** Preserve argument boundaries in a command shown to the user. */
|
||||
export const formatCommandForDisplay = (
|
||||
executable: string,
|
||||
args: readonly string[],
|
||||
): string =>
|
||||
[executable, ...args]
|
||||
.map((part) =>
|
||||
/^[A-Za-z0-9_./:@%+=,-]+$/u.test(part) ? part : JSON.stringify(part),
|
||||
)
|
||||
.join(" ");
|
||||
|
||||
/** Select the native package manager and authorization boundary without shell evaluation. */
|
||||
export const linuxPackageManagerCommands = (
|
||||
family: LinuxPackageFamily,
|
||||
|
|
|
|||
|
|
@ -10,6 +10,8 @@ import {
|
|||
type MipsElfRelocationSection,
|
||||
} from "./MipsElfSharedLoad.js";
|
||||
|
||||
const SHT_GNU_ATTRIBUTES = 0x6ffffff5;
|
||||
|
||||
/** Observed ABI and load declarations, retaining unavailable source coverage. */
|
||||
export interface MipsElfInspection {
|
||||
readonly abiFlags: MipsAbiFlags | null;
|
||||
|
|
@ -151,6 +153,10 @@ export const readMipsElfMetadata = async (
|
|||
checkCancelled();
|
||||
const position = i * table.entrySize;
|
||||
const type = u32(contents.value, position + table.typeOffset);
|
||||
if (table.optional && type === SHT_GNU_ATTRIBUTES)
|
||||
return err(
|
||||
"MIPS ELF declares GNU ABI attributes that REA does not inspect; refusing provider selection until the declared ABI is parsed and reconciled",
|
||||
);
|
||||
if (metadata.type === 3 && table.optional && (type === 4 || type === 9))
|
||||
relocationSections.push({
|
||||
type,
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.j
|
|||
import type { AnalysisCleanupObservation } from "../domain/analysisErrorBase.js";
|
||||
import { readReferenceSource } from "../reference/ReferenceSourceReader.js";
|
||||
import type {
|
||||
ReferenceSourceEntryKind,
|
||||
ReferenceSourceExclusionKind,
|
||||
ReferenceSourceRead,
|
||||
ReferenceSourceReaderError,
|
||||
} from "../reference/ReferenceSourceReaderTypes.js";
|
||||
|
|
@ -111,7 +111,8 @@ const deriveInventoryState = (
|
|||
entry.limitations.length > 0 ||
|
||||
(entry.kind === "file" && entry.content_state !== "hashed") ||
|
||||
(entry.kind === "directory" && entry.tree_state !== "enumerated") ||
|
||||
(entry.kind === "symlink" && entry.target_state !== "internal"),
|
||||
(entry.kind === "symlink" && entry.target_state !== "internal") ||
|
||||
entry.kind === "unknown",
|
||||
);
|
||||
if (partialEntry) return "partial";
|
||||
const partialRelationship = input.relationships.some(
|
||||
|
|
@ -142,22 +143,36 @@ const createShouldExclude =
|
|||
exclusions: HistoricalSourceGraphInput["exclusions"],
|
||||
secrets: PreparedReferenceSourceImport["secrets"],
|
||||
ignored: PreparedReferenceSourceImport["ignored"],
|
||||
): ((path: string, kind: ReferenceSourceEntryKind) => boolean) =>
|
||||
): ((path: string, kind: ReferenceSourceExclusionKind) => boolean) =>
|
||||
(path, kind) => {
|
||||
const patternPath = kind === "directory" ? `${path}/` : path;
|
||||
const secretMatch = secrets.test(patternPath);
|
||||
if (secretMatch.ignored) {
|
||||
// An entry of unknown kind may be a directory, so directory-only rules
|
||||
// must still exclude it; this fails closed for caller-marked secrets.
|
||||
const patternPaths =
|
||||
kind === "directory"
|
||||
? [`${path}/`]
|
||||
: kind === "unknown"
|
||||
? [path, `${path}/`]
|
||||
: [path];
|
||||
const firstMatch = (matcher: typeof secrets) => {
|
||||
const results = patternPaths.map((patternPath) =>
|
||||
matcher.test(patternPath),
|
||||
);
|
||||
return results.find(({ ignored }) => ignored) ?? results[0];
|
||||
};
|
||||
const secretMatch = firstMatch(secrets);
|
||||
if (secretMatch?.ignored === true) {
|
||||
if (!secretMatch.rule)
|
||||
throw new Error(`Ignored secret path has no matching rule: ${path}`);
|
||||
exclusions.push({
|
||||
path,
|
||||
entry_kind: kind,
|
||||
reason: "configured-secret",
|
||||
pattern: secretMatch.rule.pattern,
|
||||
});
|
||||
return true;
|
||||
}
|
||||
const match = ignored.test(patternPath);
|
||||
if (!match.ignored) return false;
|
||||
const match = firstMatch(ignored);
|
||||
if (match?.ignored !== true) return false;
|
||||
if (!match.rule)
|
||||
throw new Error(`Ignored path has no matching rule: ${path}`);
|
||||
const reason = match.rule.mark;
|
||||
|
|
@ -167,7 +182,12 @@ const createShouldExclude =
|
|||
reason !== "caller-excluded"
|
||||
)
|
||||
throw new Error(`Ignored path has unknown rule origin: ${path}`);
|
||||
exclusions.push({ path, reason, pattern: match.rule.pattern });
|
||||
exclusions.push({
|
||||
path,
|
||||
entry_kind: kind,
|
||||
reason,
|
||||
pattern: match.rule.pattern,
|
||||
});
|
||||
return true;
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -45,6 +45,8 @@ const entryRecoveryGuidance = (
|
|||
return "This directory could not be read. Check its permissions, then try again.";
|
||||
if (entry.kind === "symlink")
|
||||
return "This symbolic link could not be read safely. Check the link and its permissions, then try again.";
|
||||
if (entry.kind === "unknown")
|
||||
return "This entry's type could not be determined. Check the parent directory and try again.";
|
||||
return "This file could not be read. Check its permissions, then try again.";
|
||||
};
|
||||
|
||||
|
|
@ -73,6 +75,13 @@ const failedEntry = (
|
|||
classifications,
|
||||
limitations: [limitation],
|
||||
};
|
||||
if (entry.kind === "unknown")
|
||||
return {
|
||||
path: entry.path,
|
||||
kind: "unknown",
|
||||
classifications: ["unknown"],
|
||||
limitations: [limitation],
|
||||
};
|
||||
return {
|
||||
path: entry.path,
|
||||
kind: "file",
|
||||
|
|
|
|||
|
|
@ -162,14 +162,21 @@ const resolveSourceHead = async (
|
|||
// The library probes readFile() without a path to detect promise APIs.
|
||||
// Reject that invocation without attributing it to a source read.
|
||||
if (path === undefined)
|
||||
throw new TypeError("Git filesystem read requires a path");
|
||||
throw new TypeError("Git metadata path is required");
|
||||
try {
|
||||
const bytes = await readMetadataBytes(path);
|
||||
const encoding =
|
||||
typeof options === "string" ? options : options?.encoding;
|
||||
return encoding === undefined || encoding === null
|
||||
? bytes
|
||||
: bytes.toString(encoding);
|
||||
typeof options === "string"
|
||||
? options
|
||||
: typeof options === "object" &&
|
||||
options !== null &&
|
||||
"encoding" in options
|
||||
? options.encoding
|
||||
: undefined;
|
||||
if (encoding === undefined || encoding === null) return bytes;
|
||||
if (typeof encoding !== "string" || !Buffer.isEncoding(encoding))
|
||||
throw new TypeError("Unsupported Git metadata text encoding");
|
||||
return bytes.toString(encoding);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof ArtifactReaderFailure) {
|
||||
cleanupFailures.push(cause);
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ import { join, resolve } from "node:path";
|
|||
import { FakeSetupHost, options } from "./Setup.fixture.js";
|
||||
import { runSetup } from "./Setup.js";
|
||||
import { setupRegistrationCommand } from "./SetupHost.js";
|
||||
import { discoverSetupState } from "./SetupPlan.js";
|
||||
import type { MsixPackageView } from "./MsixPackageStorage.js";
|
||||
import type { SetupClient } from "./SupportedClients.js";
|
||||
|
||||
|
|
@ -191,6 +192,51 @@ describe("setup workflow", () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe("configured Linux Hopper selection", () => {
|
||||
it("keeps an explicitly configured launcher even when demo checks fail", async () => {
|
||||
const host = new FakeSetupHost("linux");
|
||||
host.distribution = {
|
||||
id: "ubuntu",
|
||||
versionId: "24.04",
|
||||
packageFamily: "deb",
|
||||
hopperSetupSupported: true,
|
||||
};
|
||||
host.hopper = "/opt/hopper/bin/Hopper";
|
||||
host.hopperLauncherConfigured = true;
|
||||
host.linuxDemoRuntimeMissing = true;
|
||||
const doctor = await host.doctor();
|
||||
const discovery = await discoverSetupState({
|
||||
host,
|
||||
providerEnvironment: { HOPPER_LAUNCHER_PATH: host.hopper },
|
||||
forceHopperInstall: false,
|
||||
proposeHopper: true,
|
||||
hopperLauncherConfigured: true,
|
||||
initialDoctor: doctor,
|
||||
});
|
||||
expect(discovery.installHopper).toBe(false);
|
||||
});
|
||||
|
||||
it("still offers repair for an unconfigured REA managed install", async () => {
|
||||
const host = new FakeSetupHost("linux");
|
||||
host.distribution = {
|
||||
id: "ubuntu",
|
||||
versionId: "24.04",
|
||||
packageFamily: "deb",
|
||||
hopperSetupSupported: true,
|
||||
};
|
||||
host.hopper = "/opt/hopper/bin/Hopper";
|
||||
host.unsupportedHopperVersion = true;
|
||||
const discovery = await discoverSetupState({
|
||||
host,
|
||||
providerEnvironment: { HOPPER_LAUNCHER_PATH: host.hopper },
|
||||
forceHopperInstall: false,
|
||||
proposeHopper: true,
|
||||
initialDoctor: await host.doctor(),
|
||||
});
|
||||
expect(discovery.installHopper).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("setup workflow action selection", () => {
|
||||
it("executes only actions selected by the interactive adapter", async () => {
|
||||
const host = new FakeSetupHost();
|
||||
|
|
|
|||
|
|
@ -52,6 +52,7 @@ export class FakeSetupHost implements SetupHost {
|
|||
scopedDoctorHealthy: boolean | undefined;
|
||||
linuxDemoRuntimeMissing = false;
|
||||
unsupportedHopperVersion = false;
|
||||
hopperLauncherConfigured = false;
|
||||
|
||||
constructor(platform: NodeJS.Platform = "darwin") {
|
||||
this.platform = platform;
|
||||
|
|
@ -64,6 +65,9 @@ export class FakeSetupHost implements SetupHost {
|
|||
Promise.resolve(this.distribution);
|
||||
initialSetupState = async (scope?: DoctorScope) => ({
|
||||
...(this.hopper === undefined ? {} : { hopperPath: this.hopper }),
|
||||
...(this.hopperLauncherConfigured
|
||||
? { hopperLauncherConfigured: true }
|
||||
: {}),
|
||||
providerEnvironment: {
|
||||
...(this.hopper === undefined
|
||||
? {}
|
||||
|
|
|
|||
|
|
@ -107,6 +107,7 @@ const runSetupWorkflow = async (
|
|||
host,
|
||||
providerEnvironment,
|
||||
forceHopperInstall: options.installHopper,
|
||||
hopperLauncherConfigured: initialState.hopperLauncherConfigured === true,
|
||||
proposeHopper:
|
||||
options.proposeHopper ?? (confirm !== undefined && !options.structured),
|
||||
initialDoctor: initialState.doctor,
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
import { readClientPolicyBlock } from "./GeminiClientSettings.js";
|
||||
import {
|
||||
readClientConfigurationFiles,
|
||||
effectiveClientConfiguration,
|
||||
readConfigurationText,
|
||||
type ClientConfigurationSource,
|
||||
} from "./ClientConfigurationFiles.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import {
|
||||
|
|
@ -18,9 +20,7 @@ import {
|
|||
type ClientConfigurationDocument,
|
||||
} from "./ClientConfigurationDocument.js";
|
||||
import { dshDocument, DshCordisDocument } from "./DshCordisPatch.js";
|
||||
import { constants as fsConstants } from "node:fs";
|
||||
import { createHash } from "node:crypto";
|
||||
import { copyFile, mkdir, rm, writeFile } from "node:fs/promises";
|
||||
import { mkdir, rm, writeFile } from "node:fs/promises";
|
||||
import { dirname } from "node:path";
|
||||
import writeFileAtomic from "write-file-atomic";
|
||||
import { clientRegistrationEnvironment } from "./ClientRegistrationEnvironment.js";
|
||||
|
|
@ -31,10 +31,13 @@ import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
|
|||
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
|
||||
import { packageReadTargetChanged } from "./MsixPackageStorage.js";
|
||||
import {
|
||||
readRegularFile,
|
||||
readRegularFileText,
|
||||
RegularFileCleanupFailure,
|
||||
} from "./RegularFileRead.js";
|
||||
import {
|
||||
preserveRecoveryBackup,
|
||||
selectCurrentRecoveryBackup,
|
||||
} from "./SetupRecoveryBackup.js";
|
||||
import type { ClientConfigurationFileError } from "./ClientConfigurationFiles.js";
|
||||
import type {
|
||||
ClientConfigurationInspection,
|
||||
|
|
@ -90,7 +93,9 @@ const configureClientDocument = async (
|
|||
throwCleanupFailure(files.error);
|
||||
return { status: "failed", reason: "readback" };
|
||||
}
|
||||
const existing = files.value.at(-1);
|
||||
const existing = files.value.find(
|
||||
({ sourcePath }) => sourcePath === client.configPath,
|
||||
);
|
||||
const editabilityError =
|
||||
existing?.dialect === "dsh"
|
||||
? dshDocument(existing.document).editabilityError
|
||||
|
|
@ -104,7 +109,8 @@ const configureClientDocument = async (
|
|||
client,
|
||||
environment,
|
||||
command,
|
||||
files.value.at(-1),
|
||||
existing,
|
||||
files.value,
|
||||
),
|
||||
);
|
||||
if (!policy.ok) {
|
||||
|
|
@ -137,7 +143,13 @@ const configureClientDocument = async (
|
|||
const policy = await readClientPolicyBlock(
|
||||
client,
|
||||
parsed,
|
||||
clientConfigurationDesired(client, environment, command, parsed),
|
||||
clientConfigurationDesired(
|
||||
client,
|
||||
environment,
|
||||
command,
|
||||
parsed,
|
||||
files.value,
|
||||
),
|
||||
);
|
||||
if (!policy.ok) {
|
||||
throwCleanupFailure(policy.error);
|
||||
|
|
@ -156,10 +168,16 @@ const configureClientDocument = async (
|
|||
throwCleanupFailure(fallback.error);
|
||||
return { status: "failed", reason: "readback" };
|
||||
}
|
||||
const desired = clientConfigurationDesired(client, environment, command, {
|
||||
...parsed,
|
||||
servers: { ...fallback.value, ...parsed.servers },
|
||||
});
|
||||
const desired = clientConfigurationDesired(
|
||||
client,
|
||||
environment,
|
||||
command,
|
||||
{
|
||||
...parsed,
|
||||
servers: { ...fallback.value, ...parsed.servers },
|
||||
},
|
||||
files.value,
|
||||
);
|
||||
if (registrationCurrent(parsed, desired)) return { status: "unchanged" };
|
||||
return persistClientConfiguration({
|
||||
client,
|
||||
|
|
@ -205,7 +223,10 @@ const persistClientConfiguration = async ({
|
|||
const selection = await selectCurrentRecoveryBackup(backupPath, original);
|
||||
if (!selection.ok) return { status: "failed", reason: "backup" };
|
||||
if (
|
||||
!(await preserveConfigBackup(transactionPath, selection.value.backupPath))
|
||||
!(await preserveRecoveryBackup(
|
||||
transactionPath,
|
||||
selection.value.backupPath,
|
||||
))
|
||||
)
|
||||
return { status: "failed", reason: "backup" };
|
||||
const verified = await selectCurrentRecoveryBackup(backupPath, original);
|
||||
|
|
@ -338,7 +359,13 @@ export const clientConfigurationAligned = async (
|
|||
const policy = await readClientPolicyBlock(
|
||||
client,
|
||||
parsed,
|
||||
clientConfigurationDesired(client, providerEnvironment, command, parsed),
|
||||
clientConfigurationDesired(
|
||||
client,
|
||||
providerEnvironment,
|
||||
command,
|
||||
parsed,
|
||||
files.value,
|
||||
),
|
||||
);
|
||||
if (!policy.ok) {
|
||||
throwCleanupFailure(policy.error);
|
||||
|
|
@ -353,6 +380,7 @@ export const clientConfigurationAligned = async (
|
|||
providerEnvironment,
|
||||
command,
|
||||
parsed,
|
||||
files.value,
|
||||
),
|
||||
)
|
||||
);
|
||||
|
|
@ -400,7 +428,9 @@ const inspectClientDocument = async (
|
|||
remediation: files.error.detail,
|
||||
};
|
||||
}
|
||||
const existing = files.value.at(-1);
|
||||
const existing = files.value.find(
|
||||
({ sourcePath }) => sourcePath === client.configPath,
|
||||
);
|
||||
const editabilityError =
|
||||
existing?.dialect === "dsh"
|
||||
? dshDocument(existing.document).editabilityError
|
||||
|
|
@ -413,7 +443,13 @@ const inspectClientDocument = async (
|
|||
const policy = await readClientPolicyBlock(
|
||||
client,
|
||||
undefined,
|
||||
clientConfigurationDesired(client, providerEnvironment, command, existing),
|
||||
clientConfigurationDesired(
|
||||
client,
|
||||
providerEnvironment,
|
||||
command,
|
||||
existing,
|
||||
files.value,
|
||||
),
|
||||
);
|
||||
if (!policy.ok) {
|
||||
throwCleanupFailure(policy.error);
|
||||
|
|
@ -456,7 +492,13 @@ const inspectClientDocument = async (
|
|||
const policy = await readClientPolicyBlock(
|
||||
client,
|
||||
parsed,
|
||||
clientConfigurationDesired(client, providerEnvironment, command, parsed),
|
||||
clientConfigurationDesired(
|
||||
client,
|
||||
providerEnvironment,
|
||||
command,
|
||||
parsed,
|
||||
files.value,
|
||||
),
|
||||
);
|
||||
if (!policy.ok) {
|
||||
throwCleanupFailure(policy.error);
|
||||
|
|
@ -472,6 +514,7 @@ const inspectClientDocument = async (
|
|||
providerEnvironment,
|
||||
command,
|
||||
parsed,
|
||||
files.value,
|
||||
);
|
||||
if (registrationCurrent(parsed, desired))
|
||||
return { status: "already_current" };
|
||||
|
|
@ -554,76 +597,6 @@ const readZcodeFallbackServers = async (
|
|||
}
|
||||
};
|
||||
|
||||
const preserveConfigBackup = async (
|
||||
source: string,
|
||||
destination: string,
|
||||
): Promise<boolean> => {
|
||||
try {
|
||||
await copyFile(source, destination, fsConstants.COPYFILE_EXCL);
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
return cause instanceof Error && "code" in cause && cause.code === "EEXIST";
|
||||
}
|
||||
};
|
||||
|
||||
interface RecoveryBackupSelection {
|
||||
readonly backupPath: string;
|
||||
readonly present: boolean;
|
||||
}
|
||||
|
||||
interface RecoveryBackupSelectionFailure {
|
||||
readonly kind: "unreadable" | "occupied";
|
||||
readonly path: string;
|
||||
readonly detail: string;
|
||||
}
|
||||
|
||||
const selectCurrentRecoveryBackup = async (
|
||||
firstBackupPath: string,
|
||||
original: string,
|
||||
): Promise<Result<RecoveryBackupSelection, RecoveryBackupSelectionFailure>> => {
|
||||
const expected = Buffer.from(original, "utf8");
|
||||
let firstBackup: Buffer | undefined;
|
||||
try {
|
||||
firstBackup = await readRegularFile(firstBackupPath, {
|
||||
symlinks: "reject",
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
||||
if (!isMissing(cause))
|
||||
return err({
|
||||
kind: "unreadable",
|
||||
path: firstBackupPath,
|
||||
detail: `The recovery backup could not be checked: ${errorDetail(cause)}`,
|
||||
});
|
||||
}
|
||||
if (firstBackup === undefined)
|
||||
return ok({ backupPath: firstBackupPath, present: false });
|
||||
if (firstBackup.equals(expected))
|
||||
return ok({ backupPath: firstBackupPath, present: true });
|
||||
|
||||
const digest = createHash("sha256").update(original, "utf8").digest("hex");
|
||||
const recoveryPath = `${firstBackupPath}.${digest}.recovery`;
|
||||
let recovery: Buffer | undefined;
|
||||
try {
|
||||
recovery = await readRegularFile(recoveryPath, { symlinks: "reject" });
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof RegularFileCleanupFailure) throw cause;
|
||||
if (!isMissing(cause))
|
||||
return err({
|
||||
kind: "unreadable",
|
||||
path: recoveryPath,
|
||||
detail: `The recovery backup could not be checked: ${errorDetail(cause)}`,
|
||||
});
|
||||
}
|
||||
if (recovery !== undefined && !recovery.equals(expected))
|
||||
return err({
|
||||
kind: "occupied",
|
||||
path: recoveryPath,
|
||||
detail: `The recovery path already contains different data: ${recoveryPath}`,
|
||||
});
|
||||
return ok({ backupPath: recoveryPath, present: recovery !== undefined });
|
||||
};
|
||||
|
||||
const restoreConfig = async (
|
||||
path: string,
|
||||
original: string | undefined,
|
||||
|
|
@ -678,8 +651,18 @@ const clientConfigurationDesired = (
|
|||
client: SetupClient,
|
||||
providerEnvironment: SetupProviderEnvironment,
|
||||
command: readonly string[],
|
||||
parsed?: ClientConfigurationDocument,
|
||||
parsed: ClientConfigurationDocument | undefined,
|
||||
sources: readonly ClientConfigurationSource[],
|
||||
) => {
|
||||
const effective = effectiveClientConfiguration(sources, {
|
||||
client,
|
||||
document: parsed,
|
||||
});
|
||||
const validated = clientRegistrationEnvironment(
|
||||
effective,
|
||||
providerEnvironment,
|
||||
);
|
||||
if (!validated.ok) throw validated.error;
|
||||
const format = parsed?.dialect ?? client.format ?? "json";
|
||||
const launcher =
|
||||
command.length === 0 ? [PRODUCT_IDENTITY.cliBinary, "mcp"] : command;
|
||||
|
|
@ -697,6 +680,7 @@ const clientConfigurationDesired = (
|
|||
const environment = clientRegistrationEnvironment(
|
||||
parsed,
|
||||
providerEnvironment,
|
||||
"fragment",
|
||||
);
|
||||
if (!environment.ok) throw environment.error;
|
||||
const registration = clientRegistrationEntry(
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@ import {
|
|||
import { runDoctor, systemDoctorHost, type DoctorHost } from "./Doctor.js";
|
||||
import {
|
||||
installLinuxHopper,
|
||||
LinuxHopperTemporaryCleanupOwner,
|
||||
readLinuxDistribution,
|
||||
systemLinuxHopperInstallHost,
|
||||
} from "./LinuxHopper.js";
|
||||
|
|
@ -142,6 +143,8 @@ export const systemSetupHost = (
|
|||
? systemMacHopperInstallHost(homeDirectory, environment)
|
||||
: undefined;
|
||||
const resources = new ArtifactResourceScope();
|
||||
const linuxHopperHost = systemLinuxHopperInstallHost(environment);
|
||||
const pendingLinuxHopperCleanup = new LinuxHopperTemporaryCleanupOwner();
|
||||
return {
|
||||
platform,
|
||||
homeDirectory,
|
||||
|
|
@ -151,7 +154,10 @@ export const systemSetupHost = (
|
|||
nodeVersion: process.versions.node,
|
||||
macosVersion: () => doctorHost.macosVersion(),
|
||||
linuxDistribution: readLinuxDistribution,
|
||||
close: () => closeSetupResources(resources, macHopperHost, doctorHost),
|
||||
close: () =>
|
||||
closeSetupResources(resources, macHopperHost, doctorHost, () =>
|
||||
pendingLinuxHopperCleanup.retry(linuxHopperHost.cleanup),
|
||||
),
|
||||
initialSetupState: async (
|
||||
scope?: DoctorScope,
|
||||
): Promise<SetupInitialState> => {
|
||||
|
|
@ -160,6 +166,11 @@ export const systemSetupHost = (
|
|||
...(diagnosis.hopperPath === undefined
|
||||
? {}
|
||||
: { hopperPath: diagnosis.hopperPath }),
|
||||
...(typeof environment.HOPPER_LAUNCHER_PATH === "string" &&
|
||||
environment.HOPPER_LAUNCHER_PATH.trim() !== "" &&
|
||||
diagnosis.hopperPath === environment.HOPPER_LAUNCHER_PATH
|
||||
? { hopperLauncherConfigured: true }
|
||||
: {}),
|
||||
providerEnvironment: {
|
||||
...providerRegistrationEnvironment(
|
||||
diagnosis.providerInspections ?? [],
|
||||
|
|
@ -173,12 +184,12 @@ export const systemSetupHost = (
|
|||
},
|
||||
installHopper: async (replaceExisting) => {
|
||||
if (platform === "linux") {
|
||||
const result = await installLinuxHopper(
|
||||
systemLinuxHopperInstallHost(environment),
|
||||
);
|
||||
const result = await installLinuxHopper(linuxHopperHost);
|
||||
if (result.cleanupFailure !== undefined)
|
||||
pendingLinuxHopperCleanup.retain(result.cleanupFailure);
|
||||
return result.status === "installed"
|
||||
? result
|
||||
: setupInstallFailure(result.reason);
|
||||
? { status: "installed", launcherPath: result.launcherPath }
|
||||
: setupInstallFailure(result.reason, result.packageFailure);
|
||||
}
|
||||
if (macHopperHost === undefined)
|
||||
return setupInstallFailure("unsupported_host");
|
||||
|
|
@ -264,12 +275,14 @@ const closeSetupResources = async (
|
|||
resources: ArtifactResourceScope,
|
||||
macHopperHost: ReturnType<typeof systemMacHopperInstallHost> | undefined,
|
||||
doctorHost: DoctorHost,
|
||||
retryLinuxHopperCleanup: () => Promise<string | undefined>,
|
||||
): Promise<string | undefined> => {
|
||||
const [resourceCleanup, hopperCleanup, doctorCleanup] =
|
||||
const [resourceCleanup, hopperCleanup, doctorCleanup, linuxCleanup] =
|
||||
await Promise.allSettled([
|
||||
resources.close(),
|
||||
Promise.resolve().then(() => macHopperHost?.close()),
|
||||
Promise.resolve().then(() => doctorHost.close?.()),
|
||||
retryLinuxHopperCleanup(),
|
||||
]);
|
||||
const failures: string[] = [];
|
||||
if (hopperCleanup.status === "rejected")
|
||||
|
|
@ -284,6 +297,11 @@ const closeSetupResources = async (
|
|||
);
|
||||
else if (doctorCleanup.value !== undefined)
|
||||
failures.push(doctorCleanup.value);
|
||||
if (linuxCleanup.status === "rejected")
|
||||
failures.push(
|
||||
`Linux Hopper cleanup retry failed: ${errorMessage(linuxCleanup.reason)}`,
|
||||
);
|
||||
else if (linuxCleanup.value !== undefined) failures.push(linuxCleanup.value);
|
||||
if (resourceCleanup.status === "rejected")
|
||||
failures.push(errorMessage(resourceCleanup.reason));
|
||||
return failures.length === 0 ? undefined : failures.join("; ");
|
||||
|
|
|
|||
|
|
@ -21,3 +21,51 @@ describe("macOS Hopper setup failure projection", () => {
|
|||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("Linux package-manager setup failure projection", () => {
|
||||
it("quotes the failing command and a sanitized bounded diagnostic", () => {
|
||||
const result = setupInstallFailure("authorization_or_package_manager", {
|
||||
command: 'pkexec pacman -U "/tmp/hopper package.pkg.tar.xz"',
|
||||
exitCode: 1,
|
||||
stdout: "",
|
||||
stderr: "\u001b[31merror:\u001b[0m package conflict\nrefused",
|
||||
outputTruncated: false,
|
||||
});
|
||||
expect(result).toMatchObject({
|
||||
status: "failed",
|
||||
code: "authorization_or_package_manager_failed",
|
||||
});
|
||||
if (result.status !== "failed") throw new Error("expected failure");
|
||||
expect(result.remediation).toContain(
|
||||
'pkexec pacman -U "/tmp/hopper package.pkg.tar.xz"; exit 1',
|
||||
);
|
||||
expect(result.remediation).toContain("error: package conflict refused");
|
||||
expect(result.remediation).not.toContain("\u001b");
|
||||
expect(result.remediation).not.toContain("\n");
|
||||
});
|
||||
|
||||
it("preserves the established remediation if no package output exists", () => {
|
||||
expect(setupInstallFailure("authorization_or_package_manager")).toEqual({
|
||||
status: "failed",
|
||||
code: "authorization_or_package_manager_failed",
|
||||
remediation:
|
||||
"Allow the system package-manager operation or install the planned Hopper package and Linux runtime dependencies, then rerun rea setup.",
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps truncated package output within its excerpt budget", () => {
|
||||
const result = setupInstallFailure("authorization_or_package_manager", {
|
||||
command: "pacman -U hopper.pkg.tar.xz",
|
||||
exitCode: 1,
|
||||
stdout: "",
|
||||
stderr: `${"warning ".repeat(100)}final package conflict`,
|
||||
outputTruncated: true,
|
||||
});
|
||||
if (result.status !== "failed") throw new Error("expected failure");
|
||||
const excerpt = result.remediation
|
||||
.split("its output: ")[1]
|
||||
?.split(" Resolve")[0];
|
||||
expect(excerpt).toContain("output truncated");
|
||||
expect(excerpt?.length).toBeLessThanOrEqual(400);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
import { HOPPER_PUBLISHER } from "../domain/hopperPublisher.js";
|
||||
import type { LinuxPackageInstallFailure } from "./LinuxHopper.js";
|
||||
|
||||
export type SetupFailureCode =
|
||||
| "unsupported_host"
|
||||
|
|
@ -45,9 +46,53 @@ export type HopperInstallFailureReason =
|
|||
| "publisher_verification_failed"
|
||||
| "copy";
|
||||
|
||||
const normalizePackageOutput = (output: string): string =>
|
||||
output
|
||||
.replace(/\u001b(?:\[[0-9;?]*[ -/]*[@-~]|\([0-9A-Za-z]|[@-Z\\^_])/gu, "")
|
||||
.replace(/[\u0000-\u0008\u000b\u000c\u000e-\u001f\u007f]/gu, "")
|
||||
.replace(/\s+/gu, " ")
|
||||
.trim();
|
||||
|
||||
const packageOutputExcerpt = (
|
||||
failure: LinuxPackageInstallFailure,
|
||||
limit = 400,
|
||||
): string => {
|
||||
const output = [failure.stderr, failure.stdout]
|
||||
.map(normalizePackageOutput)
|
||||
.filter((text) => text !== "")
|
||||
.join(" ");
|
||||
const suffix = failure.outputTruncated ? " (output truncated)" : "";
|
||||
const bodyLimit = limit - suffix.length;
|
||||
if (output.length <= bodyLimit) return `${output}${suffix}`;
|
||||
const half = Math.floor((bodyLimit - 3) / 2);
|
||||
const head = output
|
||||
.slice(0, half)
|
||||
.replace(/\s+\S*$/u, "")
|
||||
.trimEnd();
|
||||
const tail = output
|
||||
.slice(-half)
|
||||
.replace(/^\S*\s+/u, "")
|
||||
.trimStart();
|
||||
return `${head} … ${tail}${suffix}`;
|
||||
};
|
||||
|
||||
const packageManagerRemediation = (
|
||||
failure: LinuxPackageInstallFailure | undefined,
|
||||
): string => {
|
||||
if (failure === undefined)
|
||||
return "Allow the system package-manager operation or install the planned Hopper package and Linux runtime dependencies, then rerun rea setup.";
|
||||
const status =
|
||||
failure.exitCode === null ? "no exit status" : `exit ${failure.exitCode}`;
|
||||
const excerpt = packageOutputExcerpt(failure);
|
||||
const observed =
|
||||
excerpt === "" ? "no output was captured." : `its output: ${excerpt}`;
|
||||
return `The system package-manager command failed (${failure.command}; ${status}); ${observed} Resolve the reported problem, grant the authorization the command needs, or install the planned Hopper package and Linux runtime dependencies manually, then rerun rea setup.`;
|
||||
};
|
||||
|
||||
/** Map provider installer failures to stable setup codes and safe recovery. */
|
||||
export const setupInstallFailure = (
|
||||
reason: HopperInstallFailureReason,
|
||||
packageFailure?: LinuxPackageInstallFailure,
|
||||
): SetupHopperInstallResult => {
|
||||
switch (reason) {
|
||||
case "unsupported_host":
|
||||
|
|
@ -67,8 +112,7 @@ export const setupInstallFailure = (
|
|||
return {
|
||||
status: "failed",
|
||||
code: "authorization_or_package_manager_failed",
|
||||
remediation:
|
||||
"Allow the system package-manager operation or install the planned Hopper package and Linux runtime dependencies, then rerun rea setup.",
|
||||
remediation: packageManagerRemediation(packageFailure),
|
||||
};
|
||||
case "launcher_missing":
|
||||
return {
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue