diff --git a/.agents/skills/rea-ci-tests-audit/SKILL.md b/.agents/skills/rea-ci-tests-audit/SKILL.md new file mode 100644 index 000000000..4d7b44bf5 --- /dev/null +++ b/.agents/skills/rea-ci-tests-audit/SKILL.md @@ -0,0 +1,112 @@ +--- +name: rea-ci-tests-audit +description: Audit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. +--- + +# REA CI and tests audit + +Establish which checks still catch real regressions and which costs can be +removed without losing that protection. The outcome may be a verified fix, +an evidence-backed recommendation, or an explicit decision to stop. + +Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/testing.md), +and [contribution rules](../../../CONTRIBUTING.md). Use their current commands +and pinned toolchain, rather than commands remembered from an earlier audit. +Preserve the requested scope: investigation alone does not authorize edits; +this skill does not grant permission to publish issues, push, or merge. Existing +user authorization still applies. Keep audit notes and raw results outside the +tracked tree, normally in a task-specific `/tmp` directory. + +## Establish the executed baseline + +Fetch when the user asks for latest main. Pin the baseline SHA and inspected +workflow revision; use an isolated worktree when changing or comparing code. +Do not modify another task's checkout. Record dirty changes and the exact +command, toolchain, host, selected tests, instrumentation, and prerequisites. +An old run is evidence for its executed revision, even if main advances. + +Trace workflow triggers, changed-path selection, job dependencies, reusable +actions, package scripts, test discovery, artifact producers, and report gates +through to the commands that actually execute. Include scripts, generated data, +runtime registration, and subprocess boundaries missing from import graphs. +Check positive and negative selection paths: required files must be discovered +once per intended test configuration; deliberate platform/configuration repeats +are separate coverage. A missing explicit test or missing required report must +not turn green. Optional lanes may skip; preserve their stated prerequisites. + +Use existing timing and failure artifacts to select suspicious families. Run +relevant checks to establish a baseline; reserve a full local suite for broad +CI diagnosis or uncertainty that focused checks cannot resolve. Do not rebuild +or rerun an unchanged complete suite after every documentation edit. + +## Prove the failure or protection + +For each candidate, state the expected observable behavior and its owner, +then construct the smallest reproduction against production behavior. Follow +the producer and affected consumers before deciding where to fix it. + +- For drift, compare fixtures with current admission requirements, protocol + replies, named contracts, tool versions, and actual public entrypoints. A + plausible-looking mock can fail before the behavior its test claims to cover. + Search the whole affected fixture family, including other version strings. + Preserve intentionally malformed fixtures and test their intended rejection. +- For a suspicious green test, perturb the relevant behavior in an isolated + copy or inject a controlled failure. Verify that the existing assertion or + gate detects it. A test may merely repeat its mock, import abandoned code, + or hide a subprocess/checker failure. Restore mutations after the experiment. +- For shared setup or pruning, map distinct assertions to their retained + scenarios. Preserve malformed inputs, cancellation, cleanup failure, + permissions, capacity, identity, and meaningful concurrency interactions. + A faster run that skips a case or weakens its oracle is not an improvement. +- For provider coverage, separate fake-producer, boundary, and actual-engine + evidence. Inspect named skips and prerequisites; fixtures do not establish + real Ghidra, LLDB, browser, or other provider behavior. + +Classify findings as reproduced defects, supported costs, hypotheses, or +unavailable evidence. Record the baseline command, observed output/exit status, +expected result, affected scope, and competing explanation. A failure on main +may be pre-existing; that does not by itself make it irrelevant to a CI audit. +Attribute PR failures using the executed revision and a clean baseline, not +just their resemblance to a previous failure. Retry only to test a specific +transient explanation; retain the original failure and stop when repetition +adds no evidence. A passing retry does not establish a root cause or a fix. + +## Assess efficiency without losing the gate + +Read [measurement and CI checks](references/measurement-and-ci.md) when +investigating slowness, caching, runner waits, shared artifacts, or log noise. +Compare equivalent completed work before claiming a speedup. Distinguish +setup/build time, test execution, coverage instrumentation, dependency waits, +and runner scheduling; state which costs remain unmeasured. + +Prefer removing proven duplicate preparation, reusing immutable setup, and +reducing repeated subprocess work before shrinking correctness coverage. +Preserve independent sessions where lifecycle or concurrency is the oracle. +Keep live tests fresh; a cached previous pass cannot validate current host, +installed provider contents, permissions, or external state. + +## Fix, verify, and hand off within scope + +If implementation is authorized, make the smallest structural change at the +owning boundary and add a regression that fails on the baseline for the right +reason. Retest affected consumers and meaningful negative paths; verify cache +invalidation or discovery when those mechanisms change. Read the installed +Turbo documentation before changing its configuration or commands. + +Run the repository's applicable checks and inspect complete current-head CI +results before an authorized merge. Preserve required coverage and discovery +gates. Do not suppress a failure, add blanket retries, raise timeouts, or delete +distinct assertions solely to make CI green. Changes to these controls need +their own demonstrated reason and retained failure detection. + +Report each confirmed issue with exact revision, reproduction, consequence, +proposed correction, and validation limits. Report performance with workload +and measurements, not just a percentage. Use English issue/PR records when +requested; follow the user's assignment, commit identity, review, and merge +preferences without assuming them for other users. Open ready-for-review PRs +only when that lifecycle is authorized. Resolve necessary review findings and +check final main against the tested tree after merging. + +Stop when the authorized work is verified, remaining benefits are unproven, +or further changes would trade correctness for speculative speed. List genuine +uncertainties without turning them into additional optimization rounds. diff --git a/.agents/skills/rea-ci-tests-audit/agents/openai.yaml b/.agents/skills/rea-ci-tests-audit/agents/openai.yaml new file mode 100644 index 000000000..366dd1229 --- /dev/null +++ b/.agents/skills/rea-ci-tests-audit/agents/openai.yaml @@ -0,0 +1,4 @@ +interface: + display_name: "REA CI and tests audit" + short_description: "Audit CI/test correctness and measured performance" + default_prompt: "Use $rea-ci-tests-audit to investigate CI and test correctness, drift, and performance before proposing changes." diff --git a/.agents/skills/rea-ci-tests-audit/references/measurement-and-ci.md b/.agents/skills/rea-ci-tests-audit/references/measurement-and-ci.md new file mode 100644 index 000000000..e86ba9a4d --- /dev/null +++ b/.agents/skills/rea-ci-tests-audit/references/measurement-and-ci.md @@ -0,0 +1,65 @@ +# Measurement and CI checks + +## Equivalent completed work + +Record the source/workflow SHA, command, test identities and counts, skips, +assertions or output contract, exit status, coverage scope and thresholds, +toolchain/provider versions, host resources, and prepared artifacts. Inspect +both the child result and its report: a profiling wrapper may exit successfully +after the measured command failed. Focused tests can pass while global coverage +thresholds fail; describe both rather than calling the command green. + +Measure wall time, user/system CPU, and peak process-family memory for relevant +completed tasks. Parent-only RSS misses workers and their children. On Linux, +summed RSS can double-count shared pages; report RSS/PSS semantics, sampling +interval, missing samples, and observed cleanup limits. Use equivalent host +tools elsewhere. State unknown costs, including agent tokens; fewer log bytes +alone prove neither fewer tokens nor faster tests. + +Separate cold execution, warm build/file caches, and warm runtime caches. +Use repeated or alternating runs when variance affects the decision, with +enough observations to distinguish the proposed benefit from noise. Keep +workload, instrumentation, and host conditions comparable; record interference. +A run with fewer tests, early rejection, lost coverage, or a different runner +does not prove suite acceleration. CI-wide claims need end-to-end measurements, +not an extrapolation from one improved family. + +## Workflow, caches, and shared artifacts + +Trace each reused result from producer to consumers and identify its lifetime. +Check key/input coverage against the actual file inventory and result-affecting +configuration, environment, generated data, tool versions, and host state. +Exercise an affected input change to prove invalidation; exercise an irrelevant +change to support reuse where useful. Cache deterministic preparation and +static analysis only when their inputs fully describe the result. + +Verify downloaded artifacts match the expected revision, platform, runtime, +and schema. Check restore misses, failed/partial producers, stale reports, +removed output files, and consumers after producer failures. Shared writers +need output ownership or locks; tests need isolated state, not an unnecessary +global lock. Worker counts do not bound children spawned within each test. +Preserve intentionally concurrent scenarios while bounding unrelated batches. + +Check matrix expansion and all job pages before saying CI passed. Inspect +required gates after failed, cancelled, and skipped dependencies. `always()` +alone does not preserve failures; inspect the aggregation logic. Test/report +uploads should retain available diagnostics after failure, while absent or +malformed required results must fail closed. Do not demand an optional report +from a lane intentionally excluded by the selector. + +## Runner waits and console noise + +Use per-job and per-step timestamps where available. Workflow creation to job +start includes dependencies and other delays; it is not necessarily runner +queue time. Identify when the job became eligible before attributing waits to +runner capacity. If eligibility/queue timestamps are unavailable, report the +combined delay and unknown split. Compare equivalent lanes and runner types +across enough runs; a single busy runner is not a workflow regression. + +For console reductions, retain checker exit codes, actionable errors, useful +summary counts, and complete machine-readable artifacts. Verify authored code +remains in the actual checker inventory when excluding generated outputs. +Introduce an authored error as a control; it must still fail. An ignore pattern +can hide real source as well as noise. Show that cached reports are restored, +stale successful reports cannot mask current failures, and report parse errors +remain failures. Measure readability or byte reduction separately from runtime. diff --git a/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md b/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md index a4fc6d18f..3e185a8a2 100644 --- a/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md +++ b/.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md @@ -175,7 +175,9 @@ additional evidence only when the provider accepts and matches them. REA has a built-in read-only snapshot parser for x64 PE targets with .NET 8 RTR 9.1 metadata. An optional headless Ghidra adapter adds database annotations and broader metadata recovery on Linux; it requires an explicitly configured -NativeAOT JAR. These are separate routes. See the +NativeAOT JAR. These are separate routes. The built-in parser also runs on +Windows Ghidra P0; read frozen literals from its report, because +`search_strings` does not include them. See the [NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/ghidra-nativeaot.md) for setup, supported layouts, and host restrictions. Preserve binary identity and distinguish derived metadata from Ghidra database observations. Recovered @@ -247,3 +249,9 @@ resolution is explicit; missing references and size mismatch remain visible. Other payloads are opaque, and this operation does not decode images or simulate Windows resource loading. Consult the budgets and supported tree profile in [offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md). + +For exact core-module import/export differences, use `compare_wasm_interfaces` +with two complete inspection Evidence records (or current-session retained +references) and explicit scope. Preserve unknown types/duplicate names; static +interface equality is not runtime compatibility. CLI accepts complete inline +Evidence in `rea compare-wasm-interfaces input.json --json`. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 142e179bb..69c757b3b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -33,7 +33,7 @@ env: jobs: changes: name: Classify changes - runs-on: ubuntu-latest + runs-on: ubuntu-24.04 timeout-minutes: 5 permissions: contents: read @@ -400,6 +400,21 @@ jobs: name: windows-native-inspection-fixture path: .cache/windows-native-inspection-fixture/ if-no-files-found: error + # The process boundary observations are independent of the addon: they + # report the caller's and child's own token, environment and breakaway + # results from a separate executable. Cross-build it in the lane that + # already has MinGW so the Windows job can observe them too. + - name: Build Windows process boundary fixture + run: | + mkdir -p .cache/windows-process-boundary + x86_64-w64-mingw32-g++ -municode -std=c++17 -O2 -static -Wall -Wextra \ + -o .cache/windows-process-boundary/process-boundary.exe \ + tests/fixtures/windows/processBoundary.cc -ladvapi32 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 + with: + name: windows-process-boundary-fixture + path: .cache/windows-process-boundary/ + if-no-files-found: error - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: windows-native-controls @@ -596,6 +611,7 @@ jobs: tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts tests/boundary/filesystem/setupUnavailableProvider.test.ts tests/boundary/filesystem/windowsNpxRegistration.test.ts + tests/boundary/android/workspaceGuardianRetry.test.ts tests/acceptance/setup/homeDirectoryFallback.test.ts tests/boundary/filesystem/javascriptApplicationWorkflows.test.ts tests/boundary/filesystem/webScriptManifestPaths.test.ts @@ -622,7 +638,16 @@ jobs: src/application/Update.test.ts tests/process-global/updateMaintenance.test.ts tests/conformance/providers/ghidra/realGhidraWindowsWorkflow.test.ts - - run: npm run verify:windows-native + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 + with: + name: windows-process-boundary-fixture + path: .cache/windows-process-boundary/ + # Passing the fixture adds the breakaway, caller/child token, environment + # semantics and ancestor reparse observations, which the lane skips + # entirely when it is absent. + - run: >- + npm run verify:windows-native -- . + "${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe" - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 with: name: windows-native-inspection-fixture @@ -752,7 +777,8 @@ jobs: required: name: CI required - if: ${{ always() }} + # Report failed dependencies, but release superseded PR concurrency slots. + if: ${{ !cancelled() }} needs: [ changes, @@ -779,7 +805,7 @@ jobs: nativeaot, website, ] - runs-on: ubuntu-latest + runs-on: ubuntu-24.04 timeout-minutes: 5 steps: - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 diff --git a/.github/workflows/pr-title.yml b/.github/workflows/pr-title.yml index abe98bb48..bb2715b3e 100644 --- a/.github/workflows/pr-title.yml +++ b/.github/workflows/pr-title.yml @@ -14,7 +14,7 @@ concurrency: jobs: pr-title: name: Conventional PR title - runs-on: ubuntu-latest + runs-on: ubuntu-24.04 timeout-minutes: 2 steps: - name: Validate title for Release Please diff --git a/.github/workflows/real-evm-interface.yml b/.github/workflows/real-evm-interface.yml index 6e510c973..8588a269f 100644 --- a/.github/workflows/real-evm-interface.yml +++ b/.github/workflows/real-evm-interface.yml @@ -43,3 +43,5 @@ jobs: echo "REA_VERIFY_SOLC_MODULE=${evm_fixture_root}/node_modules/solc/index.js" >> "${GITHUB_ENV}" - name: Verify source-owned compiled bytecode through public CLI and MCP run: npm run verify:evm:interface + - name: Verify content-hash-free and CBOR-free owned compiler artifacts + run: npm run verify:solidity:metadata diff --git a/.github/workflows/real-ghidra-windows.yml b/.github/workflows/real-ghidra-windows.yml index db92cd0f8..ce1493206 100644 --- a/.github/workflows/real-ghidra-windows.yml +++ b/.github/workflows/real-ghidra-windows.yml @@ -36,6 +36,20 @@ jobs: name: real-ghidra-windows-native path: native/windows/build/ if-no-files-found: error + # The boundary observations need a separate Windows executable, which the + # native lane skips when it is absent. Cross-build it here, where MinGW is + # already installed, so the proof run observes them too. + - name: Build Windows process boundary fixture + run: | + mkdir -p .cache/windows-process-boundary + x86_64-w64-mingw32-g++ -municode -std=c++17 -O2 -static -Wall -Wextra \ + -o .cache/windows-process-boundary/process-boundary.exe \ + tests/fixtures/windows/processBoundary.cc -ladvapi32 + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 + with: + name: real-ghidra-windows-process-boundary + path: .cache/windows-process-boundary/ + if-no-files-found: error verify: name: Real Ghidra 12.1.4 Windows x64 PE proof @@ -55,7 +69,13 @@ jobs: name: real-ghidra-windows-native path: native/windows/build/ - run: npm run verify:windows-native:artifact - - run: npm run verify:windows-native + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 + with: + name: real-ghidra-windows-process-boundary + path: .cache/windows-process-boundary/ + - run: >- + npm run verify:windows-native -- . + "${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe" - name: Verify all admitted Windows Ghidra operations env: REA_ANALYSIS_PROVIDER: ghidra diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 50674afa7..96d8010ee 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -181,6 +181,11 @@ The separate GitHub Pages host uses only the manual ## Development feedback and PR verification +For routine merges, wait for the required checks to pass for the current PR +revision against the current `main`. Branch protection requires up-to-date +branches; administrators retain an override for deliberate maintainer recovery. +If `main` is broken, prioritize a focused repair before updating affected PRs. + Select checks for the behavior changed. For a source edit: ```bash @@ -202,6 +207,10 @@ CI diagnosis. Packaging/setup/distribution changes additionally require `npm run verify:package` and `npm pack --dry-run`; provider changes require the matching real-provider lane. State which host/provider workflows ran. +For CI/test correctness, drift, or performance investigations, use the +[`rea-ci-tests-audit` contributor skill](.agents/skills/rea-ci-tests-audit/SKILL.md). +It guides reproduction and comparable measurements before changing checks. + `check:pr` reuses build and static-check outputs while executing the test suite fresh, including the fast Python producer checks. `docs:check` also runs the onboarding example consistency suite for README-only changes. diff --git a/README.md b/README.md index c4a89bcee..53a86c5fc 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@
-**English** · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +**English** · [简体中文](docs/readme/README_zh.md) · [繁體中文](docs/readme/README_zh-TW.md) · [日本語](docs/readme/README_ja.md) · [한국어](docs/readme/README_ko.md) · [Türkçe](docs/readme/README_tr.md) · [Русский](docs/readme/README_ru.md) · [Tiếng Việt](docs/readme/README_vi.md) · [ไทย](docs/readme/README_th.md) · [Deutsch](docs/readme/README_de.md) · [Español](docs/readme/README_es.md) · [Français](docs/readme/README_fr.md) · [Українська](docs/readme/README_uk.md) · [Polski](docs/readme/README_pl.md) · [Português (Brasil)](docs/readme/README_pt-BR.md) · [العربية](docs/readme/README_ar.md) · [فارسی](docs/readme/README_fa.md) · [Bahasa Indonesia](docs/readme/README_id.md) · [Italiano](docs/readme/README_it.md) # REA: Reverse Engineer Anything @@ -322,7 +322,7 @@ Report vulnerabilities through [SECURITY.md](SECURITY.md). ## Star history -🎉 **170,000 GitHub stars — thank you!** +🎉 **180,000 GitHub stars — thank you!** Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes. diff --git a/bridge/android/ReaJadxBridge.java b/bridge/android/ReaJadxBridge.java index 620e2454d..e57b24943 100644 --- a/bridge/android/ReaJadxBridge.java +++ b/bridge/android/ReaJadxBridge.java @@ -3,8 +3,14 @@ import com.google.gson.*; import io.modelcontextprotocol.kotlin.sdk.server.*; import io.modelcontextprotocol.kotlin.sdk.types.*; import jadx.api.*; +import jadx.api.plugins.input.data.ICodeReader; +import jadx.api.plugins.input.data.IMethodRef; +import jadx.api.plugins.input.insns.InsnIndexType; +import jadx.api.plugins.input.insns.Opcode; +import jadx.core.dex.info.MethodInfo; import jadx.core.dex.nodes.*; import java.io.PrintStream; +import java.nio.charset.StandardCharsets; import java.util.*; import java.util.concurrent.CountDownLatch; import java.util.stream.Collectors; @@ -79,11 +85,17 @@ class ReaJadxBridge { ordered.sort(Comparator.comparing(MethodNode::getAlias)); List methods = ordered.stream().map(method -> { String name = method.getAlias(); + String shortId = method.getMethodInfo().getShortId(); + String rawName = method.getMethodInfo().getName(); + if (!shortId.startsWith(rawName + "(")) + throw new IllegalStateException("JADX method ID has no raw-name prefix: " + shortId); + String descriptor = shortId.substring(rawName.length()); String signature = name + "(" + method.getArgTypes().stream() .map(Object::toString).collect(Collectors.joining(", ")) + "): " + method.getReturnType().toString(); - return new MethodSnapshot(name, method.getMethodInfo().getShortId(), Map.of( + return new MethodSnapshot(name, shortId, Map.of( "name", name, "signature", signature, + "dex_descriptor", descriptor, "is_constructor", method.isConstructor(), "def_pos", 0)); }).toList(); List> fields = node.getFields().stream() @@ -140,6 +152,57 @@ class ReaJadxBridge { "smali", true, List.of("static_initializer_disassembly")); } + // The local transport admits 8 MiB frames. Reserve 1 KiB for the fixed + // JSON-RPC/CallToolResult envelope and generated numeric request id, and + // account for JSON text being escaped again inside TextContent. + private static final int MAX_XREF_TEXT_BYTES = 8 * 1024 * 1024 - 1024; + private static int escapedJsonBytes(Object value) { + return JSON.toJson(JSON.toJson(value)).getBytes(StandardCharsets.UTF_8).length; + } + private static final class InvokeBudget { + long remaining; + InvokeBudget(long remaining) { this.remaining = Math.max(0, remaining); } + boolean admit(Map site) { + // Check a conservative two-layer escape bound before serialization. + long bound = 256; + for (Object value : site.values()) + if (value instanceof String text) bound += 12L * text.length(); + if (bound > remaining) return false; + long bytes = escapedJsonBytes(site) - 2L + 1L; + if (bytes > remaining) return false; + remaining -= bytes; + return true; + } + } + private record DirectInvokes(List> sites, boolean exhaustive) {} + + /** Raw DEX invokes within one JADX using method; offsets are code units from its code start. */ + private static DirectInvokes directInvokes(JavaMethod source, MethodInfo target, InvokeBudget budget) { + MethodNode node = source.getMethodNode(); + ICodeReader code = node.getCodeReader(); + if (code == null) return null; + List> sites = new ArrayList<>(); + boolean[] exhaustive = {true}; + code.visitInstructions(insn -> { + insn.decode(); + if (insn.getIndexType() != InsnIndexType.METHOD_REF) return; + Opcode opcode = insn.getOpcode(); + if (!opcode.name().startsWith("INVOKE_")) return; + IMethodRef ref = insn.getIndexAsMethod(); + ref.load(); + if (!MethodInfo.fromRef(node.root(), ref).getRawFullId().equals(target.getRawFullId())) return; + Map site = Map.of( + "dex_entry", node.getInputFileName(), + "source_raw_method_id", node.getMethodInfo().getRawFullId(), + "dex_code_unit_offset", insn.getOffset(), + "dex_method_index", insn.getIndex(), + "opcode", opcode.name()); + if (budget.admit(site)) sites.add(site); + else exhaustive[0] = false; + }); + return new DirectInvokes(sites, exhaustive[0]); + } + private CallToolResult handle(String name, JsonObject args) { if (name.equals("rea_jvm_status")) return json(Map.of( "engine_reported_version", BuildInfo.VERSION, @@ -173,17 +236,52 @@ class ReaJadxBridge { } List uses; String target = string(args, "class_name"); + MethodSnapshot selectedMethod = null; + int overloadIndex = 0; if (name.equals("get_xrefs_to_class")) uses = cls.node().getUseIn(); else { List overloads = methods(cls, args); - if (overloads.size() != 1) throw new InputFailure("method_name", "Method references require a unique overload"); - target += "." + overloads.get(0).name(); - uses = method(cls, overloads.get(0)).getUseIn(); + if (overloads.size() != 1 && !args.containsKey("overload_index")) + throw new InputFailure("overload_index", "Select one of " + overloads.size() + " method overloads"); + overloadIndex = integer(args, "overload_index", 0); + if (overloadIndex >= overloads.size()) + throw new InputFailure("overload_index", "overload_index out of range: " + overloadIndex); + selectedMethod = overloads.get(overloadIndex); + target += "." + selectedMethod.name(); + uses = method(cls, selectedMethod).getUseIn(); } // REA consumes all references without materializing source line information. - List> items = uses.stream().map(node -> current.describeUsage(node, false)).toList(); - return json(Map.of("target", target, "total", uses.size(), "count", items.size(), - "resolve_line", false, "items", items)); + MethodInfo targetInfo = selectedMethod == null ? null : method(cls, selectedMethod).getMethodNode().getMethodInfo(); + List> items = uses.stream().map(node -> { + Map item = new LinkedHashMap<>(current.describeUsage(node, false)); + if (targetInfo != null && node instanceof JavaMethod) { + item.put("direct_invokes", List.of()); + item.put("direct_invokes_exhaustive", false); + } + return item; + }).toList(); + Map result = new LinkedHashMap<>(Map.of("target", target, + "total", uses.size(), "count", items.size(), "resolve_line", false, "items", items)); + if (selectedMethod != null) { + result.put("overload_index", overloadIndex); + result.put("reported_signature", selectedMethod.summary().get("signature")); + } + if (targetInfo != null) { + InvokeBudget budget = new InvokeBudget(MAX_XREF_TEXT_BYTES - escapedJsonBytes(result)); + for (int index = 0; index < uses.size(); index++) { + if (!(uses.get(index) instanceof JavaMethod source)) continue; + DirectInvokes invokes = directInvokes(source, targetInfo, budget); + Map item = items.get(index); + if (invokes == null) { + item.remove("direct_invokes"); + item.remove("direct_invokes_exhaustive"); + } else { + item.put("direct_invokes", invokes.sites()); + item.put("direct_invokes_exhaustive", invokes.exhaustive()); + } + } + } + return json(result); } public static void main(String[] args) throws Exception { @@ -217,7 +315,7 @@ class ReaJadxBridge { List fields = switch (name) { case "list_classes" -> List.of("offset", "limit"); case "get_method_body" -> List.of("class_name", "method_name", "overload_index"); - case "get_xrefs_to_method" -> List.of("class_name", "method_name"); + case "get_xrefs_to_method" -> List.of("class_name", "method_name", "overload_index"); default -> List.of("class_name"); }; Map properties = new LinkedHashMap<>(); @@ -226,7 +324,7 @@ class ReaJadxBridge { case "list_classes" -> "Indexed pages of all top-level class names, with offset and limit."; case "get_class_summary" -> "Parsed member metadata for one exact class name; no source generation."; case "get_method_body" -> "Source for an exact class and stable same-name overload index; static initializers use smali to preserve folded field initialization."; - default -> "Complete static references to an exact class or unique method; source lines are unresolved."; + default -> "Complete static using-node inventory for an exact class or selected method overload; one node may contain multiple invoke instructions, and source lines are unresolved."; }; Tool narrowed = new Tool(name, new ToolSchema(null, new JsonObject(properties), tool.getInputSchema().getRequired(), null), description, null, null, null, null, null, null); diff --git a/bridge/ghidra/ReaGhidraBridge.java b/bridge/ghidra/ReaGhidraBridge.java index c180c5866..84e809ccb 100644 --- a/bridge/ghidra/ReaGhidraBridge.java +++ b/bridge/ghidra/ReaGhidraBridge.java @@ -28,6 +28,7 @@ import java.util.ArrayDeque; import java.util.ArrayList; import java.util.Arrays; import java.util.Comparator; +import java.util.HashMap; import java.util.HashSet; import java.util.IdentityHashMap; import java.util.Iterator; @@ -56,6 +57,8 @@ import ghidra.app.decompiler.DecompiledFunction; import ghidra.app.decompiler.ClangCaseToken; import ghidra.app.decompiler.ClangNode; import ghidra.framework.Application; +import ghidra.app.util.bin.format.pdb.PdbParserConstants; +import ghidra.framework.options.Options; import ghidra.program.model.address.Address; import ghidra.program.model.address.AddressIterator; import ghidra.program.model.address.AddressSet; @@ -110,6 +113,8 @@ import ghidra.program.util.SymbolicPropogator; import ghidra.program.util.VarnodeContext; public final class ReaGhidraBridge extends HeadlessScript { + // At most 24 KiB after worst-case JSON escaping (six bytes per UTF-16 unit). + private static final int PDB_ATTRIBUTE_UTF16_BUDGET = 4096; private static final int MAX_HIGH_PCODE_OPS = 3000; private static final int MAX_HIGH_PCODE_INPUTS_PER_OP = 64; private static final int MAX_HIGH_PCODE_DEF_USE_EDGES = 12000; @@ -137,6 +142,7 @@ public final class ReaGhidraBridge extends HeadlessScript { "set_addresses_names", "inspect_native_load_image", "read_bytes", + "search_bytes", "address_to_file_offset", "ping", "shutdown", @@ -374,6 +380,7 @@ public final class ReaGhidraBridge extends HeadlessScript { return switch (request.method) { case "inspect_native_load_image" -> inspectNativeLoadImage(request.params); case "read_bytes" -> readMemoryBytes(request.params); + case "search_bytes" -> searchMemoryBytes(request.params, descriptor); case "address_to_file_offset" -> addressToFileOffset(request.params); case "address_name" -> addressName(request.params); case "list_documents" -> listDocuments(request.params); @@ -454,6 +461,7 @@ public final class ReaGhidraBridge extends HeadlessScript { .toArray(String[]::new) )); result.add("target", target); + result.add("pdb", pdbObservation()); JsonArray extensions = analysisExtensions.reports(); if (extensions.size() > 0) result.add("analysis_extensions", extensions); // Written by ReaGhidraApplySeeds before auto-analysis; REA checks it @@ -469,6 +477,41 @@ public final class ReaGhidraBridge extends HeadlessScript { return result; } + /** Read loader/analyzer-owned facts without searching for or applying companions. */ + private JsonObject pdbObservation() { + Options info = currentProgram.getOptions(ghidra.program.model.listing.Program.PROGRAM_INFO); + JsonObject pdb = new JsonObject(); + String[] names = {"file", "guid", "age", "signature", "version"}; + String[] attributes = { + PdbParserConstants.PDB_FILE, PdbParserConstants.PDB_GUID, + PdbParserConstants.PDB_AGE, PdbParserConstants.PDB_SIGNATURE, + PdbParserConstants.PDB_VERSION + }; + int remaining = PDB_ATTRIBUTE_UTF16_BUDGET; + JsonArray omitted = new JsonArray(); + for (int index = 0; index < names.length; index++) { + String value = info.getString(attributes[index], null); + if (value != null && value.length() > remaining) { + omitted.add(names[index]); + pdb.add(names[index], JsonNull.INSTANCE); + } else { + pdb.addProperty(names[index], value); + if (value != null) remaining -= value.length(); + } + } + if (omitted.size() > 0) { + JsonObject limit = new JsonObject(); + limit.addProperty("utf16_budget", PDB_ATTRIBUTE_UTF16_BUDGET); + limit.add("omitted_attributes", omitted); + pdb.add("resource_limit", limit); + } + if (info.contains(PdbParserConstants.PDB_LOADED)) + pdb.addProperty("loaded", info.getBoolean(PdbParserConstants.PDB_LOADED, false)); + else + pdb.add("loaded", JsonNull.INSTANCE); + return pdb; + } + private static String bytesHex(byte[] bytes) { return java.util.HexFormat.of().formatHex(bytes); } @@ -644,6 +687,160 @@ public final class ReaGhidraBridge extends HeadlessScript { return result; } + private JsonObject searchMemoryBytes(JsonObject params, SessionDescriptor descriptor) throws Exception { + // mask, start, end, and cursor are omitted when absent; JSON encoding drops those keys. + Set keys = params.keySet(); + if (!keys.containsAll(Set.of("document", "pattern", "limit")) || + !Set.of("document", "pattern", "mask", "start", "end", "limit", "cursor").containsAll(keys)) + throw new RequestFailure("invalid_request", "Bridge request fields are invalid"); + requireDocument(params); + // Public input validation owns the grammar; validate again at the socket boundary. + String pattern = requireString(params, "pattern").replaceAll("[\\s\\p{Z}\\uFEFF]", ""); + String mask = params.has("mask") ? requireString(params, "mask") : null; + if (mask != null) mask = mask.replaceAll("[\\s\\p{Z}\\uFEFF]", ""); + if (pattern.isEmpty() || pattern.length() % 2 != 0 || + (mask != null && mask.length() != pattern.length())) + throw new RequestFailure("invalid_request", "Mask length must equal a nonempty byte pattern length"); + byte[] bytes = new byte[pattern.length() / 2]; + byte[] masks = new byte[bytes.length]; + boolean constrained = false; + for (int index = 0; index < bytes.length; index++) { + monitor.checkCancelled(); + String pair = pattern.substring(index * 2, index * 2 + 2); + if ((!pair.equals("??") && !isHexadecimal(pair, 0, 2)) || + (mask != null && !isHexadecimal(mask, index * 2, index * 2 + 2))) + throw new RequestFailure("invalid_request", "Use hexadecimal byte pairs or ?? wildcards"); + int selected = mask == null ? 255 : Integer.parseInt(mask.substring(index * 2, index * 2 + 2), 16); + bytes[index] = (byte) (pair.equals("??") ? 0 : Integer.parseInt(pair, 16)); + masks[index] = (byte) (pair.equals("??") ? 0 : selected); + constrained |= masks[index] != 0; + } + if (!constrained) throw new RequestFailure("invalid_request", "Pattern must constrain at least one bit"); + int limit = requireBoundedInteger(params, "limit", 1, 1000); + Address start = params.has("start") ? requireAddress(params, "start") : null; + Address end = params.has("end") ? requireAddress(params, "end") : null; + if (start != null && end != null && + (!start.getAddressSpace().equals(end.getAddressSpace()) || start.compareTo(end) > 0)) + throw new RequestFailure("invalid_request", "Search bounds must be ordered in one address space"); + JsonArray identity = new JsonArray(); + for (String value : new String[] { descriptor.targetSha256, descriptor.profileDigest, + bytesHex(bytes), bytesHex(masks), start == null ? "" : canonicalAddress(start), + end == null ? "" : canonicalAddress(end), optionalString(params, "document") }) identity.add(value); + String binding = bytesHex(MessageDigest.getInstance("SHA-256").digest(identity.toString().getBytes(StandardCharsets.UTF_8))); + String cursor = params.has("cursor") ? requireString(params, "cursor") : null; + Address resume = null; + if (cursor != null) { + if (!cursor.startsWith(binding + ":")) + throw new RequestFailure("invalid_request", "Byte search cursor does not match the target and query"); + resume = parseReaAddress(cursor.substring(binding.length() + 1)); + MemoryBlock resumedBlock = currentProgram.getMemory().getBlock(resume); + Address bound = start == null ? end : start; + if (resumedBlock == null || !resumedBlock.isInitialized() || + (bound != null && !bound.getAddressSpace().equals(resume.getAddressSpace())) || + (start != null && resume.compareTo(start) < 0) || + (end != null && resume.compareTo(end) > 0)) + throw new RequestFailure("invalid_request", "Byte search cursor lies outside initialized query memory"); + } + List ordered = new ArrayList<>(List.of(currentProgram.getMemory().getBlocks())); + ordered.sort(Comparator.comparing(MemoryBlock::getStart)); + JsonArray matches = new JsonArray(); + JsonArray blocks = new JsonArray(); + JsonArray skipped = new JsonArray(); + JsonArray notLoaded = new JsonArray(); + long covered = 0; + String next = null; + for (MemoryBlock block : ordered) { + monitor.checkCancelled(); + Address low = block.getStart(); + Address high = block.getEnd(); + Address bound = start == null ? end : start; + if (bound != null && !bound.getAddressSpace().equals(low.getAddressSpace())) continue; + // Without bounds, search loaded memory only. Non-loaded file parts + // (such as .symtab) and overlays stay reachable through explicit bounds. + if (bound == null && (!block.isLoaded() || block.isOverlay())) { + notLoaded.add(byteSearchCoverage(block, low, high)); + continue; + } + if (start != null && low.compareTo(start) < 0) low = start; + if (end != null && high.compareTo(end) > 0) high = end; + if (low.compareTo(high) > 0) continue; + if (!block.isInitialized()) { + skipped.add(byteSearchCoverage(block, low, high)); + continue; + } + if (next != null) continue; + if (resume != null && low.compareTo(resume) < 0) low = resume; + if (low.compareTo(high) > 0 || !low.getAddressSpace().equals(high.getAddressSpace())) continue; + Address at = low; + Address examinedEnd = high; + while (at != null && at.compareTo(high) <= 0 && high.subtract(at) >= bytes.length - 1L) { + monitor.checkCancelled(); + // Ghidra's end address bounds candidate starts, not the final matched byte. + Address found = currentProgram.getMemory().findBytes(at, high.subtract(bytes.length - 1L), bytes, masks, true, monitor); + monitor.checkCancelled(); + if (found == null || high.subtract(found) < bytes.length - 1L) break; + if (matches.size() == limit) { + next = binding + ":" + canonicalAddress(found); + examinedEnd = found.addNoWrap(bytes.length - 1L); + break; + } + JsonObject match = new JsonObject(); + match.addProperty("address", canonicalAddress(found)); + match.addProperty("segment", block.getName()); + Long offset = targetFileOffset(block, found).offset(); + match.add("file_offset", offset == null ? JsonNull.INSTANCE : GSON.toJsonTree(offset)); + Function function = currentProgram.getFunctionManager().getFunctionContaining(found); + JsonObject procedure = new JsonObject(); + if (function != null) { + procedure.addProperty("address", canonicalAddress(function.getEntryPoint())); + procedure.addProperty("name", function.getName(true)); + } + match.add("procedure", function == null ? JsonNull.INSTANCE : procedure); + matches.add(match); + at = found.next(); + } + JsonObject coverage = byteSearchCoverage(block, low, examinedEnd); + covered = Math.addExact(covered, coverage.get("bytes").getAsLong()); + blocks.add(coverage); + } + JsonObject searched = new JsonObject(); + searched.add("blocks", blocks); + searched.addProperty("bytes", covered); + searched.add("skipped_uninitialized", skipped); + searched.add("skipped_not_loaded", notLoaded); + JsonObject result = new JsonObject(); + result.add("matches", matches); + result.add("searched", searched); + result.addProperty("complete", next == null); + result.add("next_cursor", next == null ? JsonNull.INSTANCE : GSON.toJsonTree(next)); + return result; + } + + private static JsonObject byteSearchCoverage(MemoryBlock block, Address low, Address high) { + JsonObject result = new JsonObject(); + result.addProperty("segment", block.getName()); + result.addProperty("start", canonicalAddress(low)); + result.addProperty("end", canonicalAddress(high)); + result.addProperty("bytes", high.subtract(low) + 1); + return result; + } + + private record TargetFileOffset(Long offset, int mappings) {} + + private TargetFileOffset targetFileOffset(MemoryBlock block, Address address) throws Exception { + Long offset = null; + int mappings = 0; + for (MemoryBlockSourceInfo source : block.getSourceInfos()) { + if (!source.contains(address) || source.getFileBytes().isEmpty() || + !isTargetSource(source.getFileBytes().get())) continue; + long candidate = source.getFileBytesOffset(address); + if (candidate < 0) continue; + offset = candidate; + mappings++; + } + return new TargetFileOffset(mappings == 1 && offset <= 9007199254740991L ? offset : null, mappings); + } + private JsonObject addressToFileOffset(JsonObject params) throws Exception { requireKeys(params, Set.of("document", "address")); requireDocument(params); @@ -651,18 +848,12 @@ public final class ReaGhidraBridge extends HeadlessScript { MemoryBlock block = currentProgram.getMemory().getBlock(address); if (block == null || !block.isInitialized()) throw new RequestFailure("invalid_request", "Address has no initialized file-backed memory: " + canonicalAddress(address)); - Long fileOffset = null; - int mappings = 0; - for (MemoryBlockSourceInfo source : block.getSourceInfos()) { - if (!source.contains(address) || source.getFileBytes().isEmpty() || - !isTargetSource(source.getFileBytes().get())) continue; - long offset = source.getFileBytesOffset(address); - if (offset < 0) continue; - fileOffset = offset; - mappings++; - } - if (mappings != 1) - throw new RequestFailure("invalid_request", "Address has " + mappings + " authoritative target-file mappings: " + canonicalAddress(address)); + TargetFileOffset mapping = targetFileOffset(block, address); + if (mapping.mappings() != 1) + throw new RequestFailure("invalid_request", "Address has " + mapping.mappings() + " authoritative target-file mappings: " + canonicalAddress(address)); + Long fileOffset = mapping.offset(); + if (fileOffset == null) + throw new RequestFailure("invalid_request", "Address lacks a safe file offset: " + canonicalAddress(address)); JsonObject result = new JsonObject(); result.addProperty("address", canonicalAddress(address)); result.addProperty("file_offset", fileOffset); diff --git a/bridge/pwntools/recorded_crash_impl.py b/bridge/pwntools/recorded_crash_impl.py index 208e59fe6..789c7ca94 100644 --- a/bridge/pwntools/recorded_crash_impl.py +++ b/bridge/pwntools/recorded_crash_impl.py @@ -11,6 +11,12 @@ import sys PROFILE = "pwntools@4.15.0;pyelftools@0.33;unicorn@2.1.2" +# Linux ET_CORE machine identifiers whose PRSTATUS layout is interpreted with +# unchanged pwntools 4.15.0 ctypes. Both share pr_cursig@12, pr_pid@32 and +# pr_reg@112; only the register window differs: amd64 is 27 x 8 bytes for a +# 336-byte descriptor, AArch64 is 34 x 8 bytes for a 392-byte descriptor. +CORE_MACHINES = (62, 183) + class CoreFailure(Exception): def __init__(self, reason, message): @@ -51,12 +57,27 @@ def inspect_core(path, cache): # Other declared OSABIs must never select Linux note ctypes. if content[7] not in (0, 3) or content[8] != 0: raise CoreFailure("unsupported-target", "Initial recorded-core profile requires System V or GNU/Linux OSABI with ABI version zero; selected OSABI=" + str(content[7]) + ", ABI version=" + str(content[8]) + ".") - if int.from_bytes(content[16:18], "little") != 4 or int.from_bytes(content[18:20], "little") != 62: - raise CoreFailure("unsupported-target", "Initial recorded-core profile requires x86-64 ET_CORE.") + machine = int.from_bytes(content[18:20], "little") + if int.from_bytes(content[16:18], "little") != 4 or machine not in CORE_MACHINES: + raise CoreFailure("unsupported-target", "Initial recorded-core profile requires ET_CORE for x86-64 or AArch64; selected ET_TYPE=" + str(int.from_bytes(content[16:18], "little")) + ", e_machine=" + str(machine) + ".") from elftools.common.exceptions import ELFError from elftools.common.utils import struct_parse from elftools.elf.elffile import ELFFile - from pwnlib.elf.datatypes import elf_prstatus_amd64, elf_siginfo_64, user_regs_struct_amd64 + from pwnlib.elf.datatypes import ( + elf_prstatus_aarch64, + elf_prstatus_amd64, + elf_siginfo_64, + user_regs_struct_aarch64, + user_regs_struct_amd64, + ) + if machine == 62: + profile_label = "x86-64" + architecture = "x86_64-little-endian" + prstatus_type, registers_type = elf_prstatus_amd64, user_regs_struct_amd64 + else: + profile_label = "AArch64" + architecture = "aarch64-little-endian" + prstatus_type, registers_type = elf_prstatus_aarch64, user_regs_struct_aarch64 try: image = ELFFile(io.BytesIO(content)) header = image.header @@ -107,17 +128,17 @@ def inspect_core(path, cache): if owner != b"CORE\0": continue if note.n_type == "NT_PRSTATUS": - if len(descriptor) < ctypes.sizeof(elf_prstatus_amd64): - raise CoreFailure("format", "Linux amd64 NT_PRSTATUS descriptor is truncated.") - status = elf_prstatus_amd64.from_buffer_copy(descriptor) + if len(descriptor) < ctypes.sizeof(prstatus_type): + raise CoreFailure("format", "Linux " + profile_label + " NT_PRSTATUS descriptor is truncated.") + status = prstatus_type.from_buffer_copy(descriptor) registers = [] - for name, kind in user_regs_struct_amd64._fields_: - relative = elf_prstatus_amd64.pr_reg.offset + getattr(user_regs_struct_amd64, name).offset + for name, kind in registers_type._fields_: + relative = prstatus_type.pr_reg.offset + getattr(registers_type, name).offset registers.append({"name": name, "value": hex(getattr(status.pr_reg, name)), "location": location(descriptor_start + relative, ctypes.sizeof(kind), end)}) threads.append({"note_index": note_index, "historical_pid": ctypes.c_int32(status.pr_pid).value, "recorded_current_signal": status.pr_cursig, "registers": registers}) elif note.n_type == "NT_SIGINFO": if len(descriptor) < ctypes.sizeof(elf_siginfo_64): - raise CoreFailure("format", "Linux amd64 NT_SIGINFO descriptor is truncated.") + raise CoreFailure("format", "Linux " + profile_label + " NT_SIGINFO descriptor is truncated.") signal = elf_siginfo_64.from_buffer_copy(descriptor) number = ctypes.c_int32(signal.si_signo).value code = ctypes.c_int32(signal.si_code).value @@ -125,7 +146,7 @@ def inspect_core(path, cache): signals.append({"note_index": note_index, "number": number, "code": code, "errno": ctypes.c_int32(signal.si_errno).value, "fault_address": hex(signal.sigfault_addr) if known else None, "fault_address_meaning": "recorded-sigsegv-address" if known else "unknown", "thread_association": "unknown"}) - return {"format": "elf-core", "architecture": "x86_64-little-endian", "target_execution": "not-performed", "live_process_identity": "unknown", + return {"format": "elf-core", "architecture": architecture, "target_execution": "not-performed", "live_process_identity": "unknown", "segments": segments, "notes": notes, "note_padding": padding, "threads": threads, "signals": signals, "note_interpretation_completeness": "unknown", "limitations": ["The initial Linux note profile accepts System V/unspecified or GNU/Linux OSABI with ABI version zero; an unspecified OSABI alone does not prove operating-system origin.", diff --git a/docs/.vitepress/config.ts b/docs/.vitepress/config.ts index 001c07fbe..5d7fcc2f3 100644 --- a/docs/.vitepress/config.ts +++ b/docs/.vitepress/config.ts @@ -6,6 +6,7 @@ export default defineConfig({ "Reverse engineer binaries, applications, and runtime behavior with one CLI and MCP server.", lang: "en-US", base: "/rea/", + srcExclude: ["readme/**"], themeConfig: { nav: [ { text: "Get started", link: "/installation" }, diff --git a/docs/android-analysis.md b/docs/android-analysis.md index d98d9b5da..50087f464 100644 --- a/docs/android-analysis.md +++ b/docs/android-analysis.md @@ -86,6 +86,7 @@ rea inspect-android-method /path/Example.apk example.MainActivity onCreate rea inspect-android-method /path/Example.apk example.MainActivity select --overload-index 1 rea trace-android-references /path/Example.apk example.MainActivity rea trace-android-references /path/Example.apk example.MainActivity --method-name onCreate +rea trace-android-references /path/Example.apk example.MainActivity --method-name select --overload-index 1 ``` Equivalent MCP methods are `inspect_android_package`, `search_android_classes`, @@ -106,9 +107,10 @@ Every request includes `path`. Class/method operations use `class_name`, ``` Use `inspect_android_class` to choose an overload. REA rejects an ambiguous -method request rather than silently selecting the first match. The engine's -method reference operation cannot select overloads, so REA reports that boundary -as unsupported when multiple same-name methods exist. +method request rather than silently selecting the first match. Method reference +queries accept the same `overload_index`; the bridge resolves the selected +method node and the result reports its index and displayed signature. These +indices are tied to the current artifact and engine, not stable across builds. The engine's smali fallback also joins same-name overloads; REA rejects that fallback for an overloaded method instead of attributing all bodies to one @@ -137,14 +139,28 @@ the same source byte limit and explicit truncation reporting. full reported display types, fields and inner class names. Synthetic members that code generation hides can appear in the inventory; it stays consistent after method decompilation within the retained session. - Exact DEX descriptors are unknown. Overload indices are tied to this artifact - and engine, not stable identities across different builds. + Exact DEX prototypes are reported from frozen JADX method metadata as + `dex_descriptor`, for example `(I)V`. Display names can be JADX aliases; + the descriptor alone is not a complete raw DEX method ID. Overload indices + are tied to this artifact and engine, not stable identities across builds. - Method inspection reports Java or smali, fallback markers and complete/partial returned text. Decompiled source is a derived representation. A complete text result does not establish complete semantics or a byte-identical reconstruction. - Reference results are incoming static relationships between provider nodes. - Source lines and DEX instruction offsets remain unknown. They are not observed - runtime calls. + `reference_unit=using_node` means `total` counts containing JADX nodes, not + individual invoke instructions; two calls from one method can appear as one + entry. `coverage=complete` covers this node inventory only. For a method + reference whose using node has raw code, `direct_invokes` lists matching DEX + invoke instructions with their DEX entry, raw source method ID, code-unit + offset, method index and opcode. An empty array means no direct invoke was + matched in that node; `null` means raw code was unavailable or the using node + was not a method. This list does not establish complete call-site coverage + outside JADX's using-node inventory or through reflection and dynamic loading. + A virtual or interface invoke also does not establish its runtime implementation. + Source lines and the containing node's own `dex_offset` remain unknown. + Method-reference results also carry the selected `dex_descriptor`; + class-reference results report `null`. + These relationships are not observed runtime calls. - Original requests, APK SHA-256, actual engine JAR SHA-256 and raw producer responses remain in Evidence. Source revision is populated only for bytes matching the audited release JAR. @@ -210,9 +226,11 @@ REA never writes to the original APK or launches target code. On Windows, stdin writes preserve stream backpressure and run outside the Node event loop. Cancellation terminates the retained job, including descendants, -before workspace cleanup. An abrupt CLI/server process termination closes the -job and stops Java, but cannot run JavaScript workspace cleanup; temporary -workspace files can remain after that forced exit. +before workspace cleanup. On Windows, a detached REA helper owns the private +workspace's native directory handle. An abrupt CLI/server owner termination +closes the Java job and the helper's input pipe; the helper then removes the +workspace by handle. The real Windows lane checks both effects. If the helper +also dies or the host loses power before it finishes, private files can remain. If Java cannot start the bridge, REA reports the selected executable and retained startup diagnostics. Select a full JDK through `JAVA_HOME` or `PATH`, and use @@ -275,12 +293,20 @@ The original APK and host Java configuration remain unchanged. The same lane runs on Windows x64 with the bundled native controls and a full JDK. It checks CLI/MCP results, real MCP cancellation and disconnect, and Java -exit after abrupt CLI owner termination. Windows forced termination is checked -separately from the POSIX CLI SIGTERM handler; it does not claim a structured -cancellation result or workspace removal after the owner is forcibly killed. +exit and workspace removal after abrupt CLI owner termination. Windows forced +termination is checked separately from the POSIX CLI SIGTERM handler. A +synthetic two-owner test confirms that one owner's exit does not remove the +other owner's workspace. The lane does not claim a structured cancellation +result or power-loss recovery. The download script verifies fixed SHA-256 values, reuses matching files and refuses to overwrite a different existing file. APK/JAR files live under ignored `_reference/` and are excluded from package contents. The lane exercises actual REA CLI and MCP results with the real engine, separately from synthetic protocol regressions. It needs no Gradle build, emulator, Android SDK or Ghidra. + +Direct invoke sites retain exact DEX code-unit offsets separately from each +containing node's unknown `dex_offset`. `direct_invokes_exhaustive: false` +means instruction rows were bounded by the local bridge frame budget; `null` +means the producer did not establish instruction coverage. Using-node totals +and coverage do not imply complete instruction evidence. diff --git a/docs/browser-observation.md b/docs/browser-observation.md index dca1f994b..72dde900b 100644 --- a/docs/browser-observation.md +++ b/docs/browser-observation.md @@ -107,6 +107,47 @@ first navigation. For a dedicated investigation profile, add then describe an unloaded page; REA continues to require an allowed, committed URL. See [the startup investigation](https://github.com/morluto/rea/issues/1384#issuecomment-6093781686). +### Browsers without HTTP discovery + +Some browsers expose a loopback browser WebSocket but no `/json/version` or +`/json/list`. Chrome 144 and later can enable remote debugging on a running +browser from `chrome://inspect/#remote-debugging`; this mode is reported to +serve only the browser socket. Since Chrome 136, `--remote-debugging-port` is +ignored for the default profile, so this is the remaining way to inspect a page +in an existing profile. + +For such a browser, pass the browser socket instead of the HTTP endpoint: + +```bash +rea list-browser-targets ws://127.0.0.1:9222/devtools/browser/ +``` + +Chrome writes the port and the `/devtools/browser/` path to the +`DevToolsActivePort` file in its user data directory. REA does not read that +file for you. + +- Only `list_browser_targets`, `inspect_web_page`, `analyze_web_bundle`, + `observe_web_session`, `discover_webmcp_tools`, and `capture_web_screenshot` + accept a browser socket. Website runtime attribution, Electron inspection, + and scenario attach mode still require HTTP discovery. +- Each call opens one WebSocket and uses it for discovery + (`Browser.getVersion`, `Target.getTargets`) and for the selected page. + Discovery must finish within the same 10-second deadline as HTTP discovery. + Chrome may ask you to approve each connection. +- The browser identifier controls access to the socket. Evidence records the + endpoint as `ws://127.0.0.1:9222/devtools/browser/[REDACTED]`. +- `browser.revision` is the socket's `Browser.getVersion` value, such as + `@e533e98…`. HTTP discovery reports `WebKit-Version`, such as + `537.36 (@e533e98…)`. +- `Target.getTargets` reports whether each target is attached. Chrome's + `/json/list` can omit it, so HTTP discovery may report `attached` as `null`. +- An existing profile can hold many unrelated pages. Target listing returns + each eligible page's URL and title, so pass `--allowed-origins` to list only + the origins you are investigating. + +A browser driven only through `--remote-debugging-pipe` or a vendor's private +automation channel has neither endpoint, and REA cannot attach to it. + ## Request boundary CDP connections enforce the shared 64 MiB limit per WebSocket message. A larger @@ -116,9 +157,13 @@ decoded strings or JSON objects. Discovery supplies a literal loopback CDP endpoint and returns all eligible page targets unless an exact HTTP(S) origin filter narrows the result. Follow-up -inspection requests select one target; their default scope is its current origin. `localhost`, private-LAN -addresses, HTTPS CDP endpoints, credentials, paths, queries, fragments, and -implicit ports are not valid CDP endpoints. Origin filters contain only scheme, +inspection requests select one target; their default scope is its current origin. +The endpoint is either `http://127.0.0.1:PORT` or a +[browser socket](#browsers-without-http-discovery), +`ws://127.0.0.1:PORT/devtools/browser/ID`; `[::1]` may replace `127.0.0.1`. +`localhost`, private-LAN addresses, HTTPS and `wss:` endpoints, credentials, +queries, fragments, implicit ports, and any other path are not valid CDP +endpoints. Origin filters contain only scheme, host, and port; paths, credentials, queries, fragments, and wildcards are not valid origins. diff --git a/docs/cli.md b/docs/cli.md index 6d81ec7c1..c43ec2181 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -214,8 +214,36 @@ Import an older source tree to compare with the current artifact: ```bash rea import-reference-source /absolute/path/to/source +rea import-reference-source /absolute/path/to/source --detail summary --outline-depth 2 --json ``` +The default (`--detail full`) returns the complete `HistoricalSourceGraph`. +Choose `--detail summary` for a `HistoricalSourceSummary` of that same committed +inventory: its `HistoricalSourceManifest` contains the full graph and root +digests, with provenance, VCS facts, languages, manifest paths, entry and +relationship counts, parse-failure counts, file/directory states, exclusions +and limitations. The summary is a presentation view, not a graph accepted by +source-to-bundle comparison; keep the full output for that workflow. + +`--outline-depth` requires `--detail summary` and accepts a nonnegative safe +integer. Depth 1 shows root children, depth 2 (the default) also shows their +children, and depth 0 omits all outline entries. Paths retain canonical Unicode +code-point order. `outline.omitted_entry_count` counts only inventoried entries +hidden by the chosen depth; it does not count unexamined children of excluded or +unreadable directories, or change `inventory_state`. Exclusions and source +limitations remain separate from this presentation omission. + +`file_sizes.known_bytes` is the exact sum of known inventoried file sizes as +decimal text, so large sums cannot lose integer precision. Unknown sizes count +in `file_sizes.unknown_file_count`; excluded or unexamined paths are not included +in that total. A zero known-byte sum therefore does not establish an empty source +tree. Full output retains individual file hashes, relationships and parse-failure +diagnostics. Summary mode performs the same import and cleanup, then projects +the committed result without rereading source files. It reduces the selected +output, not the amount of source examined. A shallow tree or many exclusions can +still produce a large summary; depth 0 removes the outline but preserves the +other facts. No fixed output-size or runtime improvement is promised. + The import records hashes and metadata for the supplied files, separately from observations of the current app. For a directory inside a Git checkout, it records the HEAD of the nearest enclosing repository; whether the working diff --git a/docs/client-configuration.md b/docs/client-configuration.md index e27866f7d..409bc7f19 100644 --- a/docs/client-configuration.md +++ b/docs/client-configuration.md @@ -33,6 +33,71 @@ schema constraints; REA still validates complete canonical inputs. Full-profile `gpt-5.4` metadata is also verified locally. Live model API acceptance and Windows remain unverified; see [native client verification](testing.md). +## Claude Desktop + +Setup registers `mcpServers.rea` in `claude_desktop_config.json`: under +`~/Library/Application Support/Claude` on macOS, under +`$XDG_CONFIG_HOME/Claude` (default `~/.config/Claude`) on Linux, and on Windows +in the file the installed app reads: + +- Without Claude Desktop's MSIX package storage, setup uses + `%APPDATA%\Claude\claude_desktop_config.json`. +- The MSIX build (package family `Claude_pzs8sxrjxfjjc`, installed under + `C:\Program Files\WindowsApps\Claude__x64__pzs8sxrjxfjjc`) + virtualizes AppData writes. Once it has saved its configuration, it reads + the package-private copy + `%LOCALAPPDATA%\Packages\Claude_pzs8sxrjxfjjc\LocalCache\Roaming\Claude\claude_desktop_config.json` + and ignores the `%APPDATA%` file. Setup, doctor and uninstall use that + private copy when it exists. Before the package has saved one, they use the + `%APPDATA%` file it still reads; with neither file, setup creates the + private copy. +- If `%LOCALAPPDATA%\Packages` holds a `Claude_` package from + another publisher, or cannot be read, REA cannot establish which file Claude + Desktop reads. Setup, doctor and uninstall report that instead of falling + back to `%APPDATA%`. Doctor then reports `claude_desktop` as invalid, and + uninstall stops before removing anything for any client. + +Quit Claude Desktop fully before approving setup; closing its window is not +enough. A running Claude Desktop can rewrite its configuration from memory and +drop the registration. This was observed with the Windows MSIX build. Start +Claude Desktop again after setup. If the package saves its private copy after +setup planned to write the `%APPDATA%` file, setup makes no change and asks you +to rerun it. + +A process that the MSIX app started, such as a terminal in Claude Desktop's +Code tab, sees the package's private files at their `%APPDATA%` paths even +though it reports no package identity. Setup compares one of those files at +both paths. When it finds that redirected view, the app is running, so setup +and uninstall refuse to change Claude Desktop's configuration; rerun them from +a terminal that Claude Desktop did not start. If the package has no private +file to compare, setup and uninstall cannot rule that out and also refuse; open +Claude Desktop once if it has never run, quit it fully, then rerun. Doctor +reads the same file from either kind of process. + +The same process also writes everything under `%APPDATA%` and +`%LOCALAPPDATA%` into the package's private storage, where other apps never +read it. From that redirected view, setup and uninstall refuse to change other +clients' files there (VS Code, Trae, Devin and Hermes) and skill directories +there. `npm install -g` run from such a process installs into the package's +`LocalCache\Roaming\npm`, which other processes cannot see at the `%APPDATA%` +path. Setup refuses to register a launcher stored in any MSIX package's +private storage. Install REA from an ordinary terminal and rerun setup there. + +Uninstall removes REA's entry from both the private copy and the `%APPDATA%` +file, including registrations that earlier releases wrote to `%APPDATA%`. + +REA recognizes only the MSIX package. A non-MSIX Claude Desktop installed +beside it, which would read the `%APPDATA%` file, is not detected. + +The package identity, both file locations, the redirected view, and these +setup and doctor results were observed on Windows 11 with the MSIX build +2.31226.1.0, using read-only `doctor` and `setup --dry-run` runs both inside +and outside the app. On that host, setup run from an ordinary terminal with +the app quit wrote the package-private copy; after the app restarted, the +entry was still present, doctor reported it aligned, and Claude Desktop's MCP +log showed REA starting and answering `tools/list`. Uninstall against the real +app is unverified. + ## Claude Code Claude Code normally defers MCP tool definitions through tool search. It @@ -64,6 +129,18 @@ and Claude Code reports that autocompact is thrashing. To avoid this: The sizes above come from REA's own `tools/list`. A live Claude Code session against DeepSeek's API is unverified. +## Kimi Code + +Setup registers `rea` in `~/.kimi-code/mcp.json`, or +`$KIMI_CODE_HOME/mcp.json` when set, while preserving unrelated settings and +creating a `.rea.backup` before updating an existing file. Kimi Code infers +stdio from the `command` entry, so setup does not add a `type` field. The +registration includes `startupTimeoutMs = 30000` and `toolTimeoutMs = 120000`. +Kimi Code also discovers personal skills in `~/.agents/skills` and project-local +`.agents/skills`, so no separate skill root is required; setup still installs +and verifies the matching skill files through the normal shared-skill flow. +Start a new Kimi Code session to pick up MCP and skill changes. + ## Pi For [Pi](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/mcp.md), diff --git a/docs/client-verification.md b/docs/client-verification.md index d9ece9aad..38d0bf399 100644 --- a/docs/client-verification.md +++ b/docs/client-verification.md @@ -458,7 +458,7 @@ artifact path. Its `expectedFirstTool` is `null`, and `targetClarificationPassed` grades that limited routing outcome independently of analysis prose heuristics. Analyzing a nearby example cannot pass this case. -The packaged desktop and parser-comparison scenarios now assess a closed set of +The packaged desktop, parser-comparison, and managed scenarios assess a closed set of known fixture claims. Each final answer must be a strict JSON manifest containing exactly the requested claim IDs, values, Evidence IDs, and Evidence authority and confidence. Golden values are not included in the prompt. The evaluator checks @@ -475,15 +475,24 @@ analyses and use the requested module and export selectors. Boundary tests packa the same source-owned desktop fixture and compare the same source-owned parser files through actual MCP tool results, then verify that fabricated answers fail. +The managed rubric checks the exact public top-level type set and the CLI entry +point's declaring type and method name. The exact type-set claim requires an +unfiltered member inventory: `selection` must be null and the Evidence parameters +must omit `type_names`. A selected inventory can still ground the entry-point +claim. Both inspections must describe the freshly built DLL's SHA-256. Entry-point +resolution additionally requires complete artifact and member metadata, matching +non-null module MVIDs, and a unique method at the header's metadata token. These are +static metadata claims, not runtime behavior; +compiler-generated member counts, token values, and MVIDs are not golden answers. + Per-scenario `factualCorrectness` is `passed`, `failed`, or `not_assessed` within the `configured_fixture_claims` scope. A pass establishes only the configured -claims, not unrestricted factual correctness. Native, managed, browser, +claims, not unrestricted factual correctness. Native, browser, navigation-context, and address-context scenarios have no factual rubric and -remain `not_assessed`; the managed workflow requires both artifact and member -inspection to answer its type and entry-point question. +remain `not_assessed`. The closed factual scenarios request complete producer Evidence on the initial -analysis so their source selectors can authenticate all configured facts. +inspection or analysis so their source selectors can authenticate all configured facts. Summary/page/item workflows are exercised separately by `javascript-module-view`; its heuristic gate does not establish full-producer factual correctness. diff --git a/docs/evm-bytecode.md b/docs/evm-bytecode.md index 972d8b92c..1e28542fb 100644 --- a/docs/evm-bytecode.md +++ b/docs/evm-bytecode.md @@ -39,6 +39,18 @@ license](https://github.com/morluto/rea/blob/main/third_party/evmole/README.md) - Upstream also returns terminal metadata during selector recovery. Its reported values remain in `raw_result`; integer precision and non-text-key coverage are unknown in the JS binding. Full bytecode retains the original encoded metadata. +- `bytecode.metadata` independently interprets a plausible terminal CBOR root + map from the selected bytes. Integers use decimal strings; byte strings use hex, + and scalar keys retain their types. The recorded offset/length addresses the + original bytecode, so summary delivery does not copy an opaque CBOR payload. + Nested containers have partial scalar coverage; unsupported or malformed + candidates report `not-decoded` with a diagnostic. Decoding is bounded to 64 + container levels and the terminal payload's encoded size. +- An unavailable trailer does not prove metadata absence or Solidity origin. + Compiler-version bytes/text are metadata declarations, not verified compiler, + source-file or deployed-contract identity. Historical omission parses as a null + unknown facet without changing the original Evidence record. Metadata/source + URLs are not fetched. - Selectors, argument strings and state mutability are **inferences**. Function names/signatures are not looked up. Optimized code may produce different mutability candidates; recovered types are not a verified source ABI. @@ -125,3 +137,15 @@ The real verification lane traces representative raw/hex CLI workflows and check that only declared Node/prlimit launchers and process ownership inspection run. Socket traces permit local stdio socket pairs and reject attempted network requests. These observations verify those fixtures; they are not a general network sandbox. + +For the independent metadata parser, `npm run verify:solidity:metadata` uses an +existing caller-selected solc 0.8.30 module: + +```sh +REA_VERIFY_SOLC_MODULE=/absolute/path/to/solc/index.js npm run verify:solidity:metadata +``` + +It compiles the owned fixture with unoptimized, viaIR, content-hash-free and +CBOR-free settings, verifies the observed trailer facts and reports local decode +timings. No contract is executed. This lane does not establish the complete +Linux EVM provider's CLI/MCP, resource-limit or cleanup behavior. diff --git a/docs/flutter-analysis.md b/docs/flutter-analysis.md index ebf0ccd84..df68ee9cc 100644 --- a/docs/flutter-analysis.md +++ b/docs/flutter-analysis.md @@ -60,6 +60,10 @@ server; older servers keep the complete request after schema discovery. [Dart 2.14 layout](https://github.com/dart-lang/sdk/blob/2.14.0/runtime/vm/snapshot.h). Kind numbers vary across Dart releases; other values are preserved with partial coverage, without guessing their interpretation. + Address resolution examines every overlapping `PT_LOAD` interpretation, + including partial overlaps. Equivalent mappings are coalesced; conflicting + mappings leave affected symbols unresolved and qualify coverage. Dynamic + hash, symbol and string tables use the same mapping checks. A payload without exported snapshot symbols is refused rather than guessed at. - **The snapshot hash** from the header at the start of each exported data @@ -133,8 +137,11 @@ server; older servers keep the complete request after schema discovery. `resource_constraint`. - **Engine notes.** Engine libraries are spooled into the owned root. ELF headers, note tables and note payloads share an 8 MiB metadata budget; - up to 64 distinct build IDs are retained. Exceeding either bound preserves - the candidates already examined and reports partial coverage. + up to 64 distinct build IDs and 1024 distinct nonempty note ranges are retained. + The range cap bounds bookkeeping and separate positional reads even for tiny + overlapping declarations. Exact duplicates share one read and empty ranges + consume no inventory slots. Exceeding a bound preserves the candidates examined + from retained ranges and reports partial coverage with `build_id: null`. - **Metadata and candidate capacity.** Both operations refuse duplicate relevant library paths. Inventory is limited to 32 ABIs and 1024 bytes per relevant library path, independently of compressed size. Identification @@ -174,3 +181,9 @@ engine build-ids, no labeled Dart version (as expected for a release engine), and the Android clang toolchain line; a plain aapt2-built APK reports not-detected with complete coverage. The suite uses self-contained synthetic ELF images and scanner payloads; it does not read private local files. + +The Gallery workflow and all 147 focused Flutter tests also pass on macOS Sonoma +14.8.9 x86-64 and Windows 11 x64 build 26300.9457 guests with Node 24.18.0, +including the conflicting-load and note-range-limit CLI/MCP regressions. Windows +verification runs under a non-administrator account. Verifiers use a shared +temporary parent so each CLI/MCP process can own and release its own snapshots. diff --git a/docs/ghidra-mips.md b/docs/ghidra-mips.md index d5b58f00b..f010b4f4c 100644 --- a/docs/ghidra-mips.md +++ b/docs/ghidra-mips.md @@ -22,6 +22,28 @@ lane. MIPS16 shared images have the separate bounded profile below; other shared or compressed images remain unsupported. Their family may be identified, but provider selection returns the specific unsupported constraint rather than guessing a compatible profile. +A second admission covers recordless legacy firmware images +([#1620](https://github.com/morluto/rea/issues/1620)): ELF32 `ET_EXEC` headers +declaring `EF_MIPS_ARCH_32` or `EF_MIPS_ARCH_32R2` with an unspecified ABI +field, `EF_MIPS_ABI2` clear, and an _inspected_ absence of any ABI flags +record (a target whose record was merely not observed stays refused). For +these images o32 is an analysis assumption derived from the ELF32/ET_EXEC +structure, recorded as `mips_abi_basis` in the profile and stated as a +limitation in Evidence, never a declared fact; floating-point ABI semantics +stay unverified. Ghidra's own header auto-selection picks a 64-bit language +for exactly these files, so the lane pins `-processor MIPS::32:default` +through a `provider-pinned-v1` profile selection and checks the +bridge-reported `language_id` after import. A caller-selected +`REA_GHIDRA_LANGUAGE_ID`/`REA_GHIDRA_COMPILER_SPEC_ID` still wins over the +pin; combining the pinned profile with an environment override is a typed +refusal. A header that declares an ABI field but carries no record is still +refused. Two further boundaries apply: an inspection that could not examine +every ABI declaration source (for example a carved ELF whose section table +extends past the captured bytes without a program-header record) does not +establish the recordless admission, and an `EF_MIPS_ARCH_32` header that +_declares_ an ABI is refused rather than relabeled, because the declared-ABI +lane is verified for `EF_MIPS_ARCH_32R2` headers only. + Explicit `EF_MIPS_FP64` and `EF_MIPS_NAN2008` declarations are refused. Their absence is insufficient: Clang can emit identical `e_flags` for FP32, FP64 and soft-float targets. Target resolution also reads the 24-byte ABI flags record @@ -40,12 +62,14 @@ absent or supply a default ABI. The field layout follows [`Elf_MIPS_ABIFlags_v0`](https://github.com/bminor/glibc/blob/master/elf/elf.h). -Provider admission requires record version 0, MIPS32r2, 32-bit GPRs, no CPR2, -no ISA extension, ASE or unknown general flags, and one of two floating-point -declarations: 32-bit CPR1 with a double-precision or FPXX ABI, or no CPR1 with -the soft-float ABI. Other combinations, including single-float, FP64 and -unspecified ABIs, are refused. Missing records remain unknown rather than -defaulting to a compatible ABI, and this lane refuses them. Unknown record +Provider admission on the declared-ABI lane above requires record version 0, +MIPS32r2, 32-bit GPRs, no CPR2, no ISA extension, ASE or unknown general +flags, and one of two floating-point declarations: 32-bit CPR1 with a +double-precision or FPXX ABI, or no CPR1 with the soft-float ABI. Other +combinations, including single-float, FP64 and unspecified ABIs, are refused. +Missing records remain unknown rather than defaulting to a compatible ABI, and +the declared-ABI lane refuses them; the recordless legacy admission above is +the only path that admits a recordless image. Unknown record values are retained for diagnosis but do not gain provider support. These checks bind the declared interpretation; the fixtures do not establish full floating-point behavioral conformance. diff --git a/docs/ghidra-nativeaot.md b/docs/ghidra-nativeaot.md index d9505a226..394d7fdf5 100644 --- a/docs/ghidra-nativeaot.md +++ b/docs/ghidra-nativeaot.md @@ -17,10 +17,14 @@ submodule and is a separate, more capable workflow. The parser currently handles x64 PE32+ and RTR 9.1, including bounded DEHYDRATED_DATA processing, MethodTable relationships, virtual slots, and frozen strings. Other RTR versions and PE architectures are reported as -unsupported or partial. The matching .NET 8.0.22 fixture is source-built and -the parser's Windows-host Ghidra workflow has not yet had a real Windows-host -run. This does not establish Windows Ghidra import or decompiler behavior. -Windows Ghidra P0 has no database mutation authority. A PE/CLI or ReadyToRun +unsupported or partial. The matching .NET 8.0.22 fixture is source-built. + +On a Windows host, Windows Ghidra P0 runs the parser without a JAR. Metadata, +MethodTable relationships, virtual slots and frozen strings come from the +parser report, and `analyze_function` decompiles slot targets. Tools that read +the Ghidra database do not see rehydrated bytes (see below), because only the +optional adapter writes them into the database and Windows Ghidra P0 has no +database mutation authority. A PE/CLI or ReadyToRun assembly should use `inspect_managed_artifact`; it is not NativeAOT. ## Build and configure @@ -57,7 +61,9 @@ returns `observations.metadata_recovery` inline from the captured source snapsho including the RTR header, MethodTable addresses, frozen strings, derived overlay identity, and examined/recovered counts. Its enclosing `unsupported` load-image status means independent PE load-image verification was not performed; it does -not mean the metadata parser failed or the executable failed to load. +not mean the metadata parser failed or the executable failed to load. Without +the optional adapter, each recovered type's `type` is `null`, because no Ghidra +DataType exists; select it by `address`. Pass a returned MethodTable address to `inspect_native_data_type`. A direct PE query also derives the snapshot metadata from the authenticated Ghidra session @@ -67,8 +73,12 @@ The database result remains `unavailable` when Ghidra has no DataType at that address; its `metadata_recovery.source` is `read-only-derived-overlay` and carries related type, interface, slot, and target observations. Generated address identities do not claim original managed names. The parser does not -merge frozen literals into Ghidra's database string inventory. Follow slot -targets with `analyze_function` and ordinary cross-references. +merge frozen literals into Ghidra's database string inventory: read them from +`metadata_recovery.frozen_strings`, not `search_strings`. Slot targets come +from the parser report; on Windows P0 without the adapter, a `read_bytes` of +a slot in the rehydrated region returns no bytes. The Linux adapter route +verifies a complete slot read and its target. Follow slot targets with `analyze_function` and ordinary +cross-references. The built-in CLI path uses a MethodTable address returned by the parser or another observation: @@ -129,3 +139,11 @@ compiler/linker symbols and raw directory bytes provide the oracle. Stripped fixtures exercise heuristic detection. Tests do not execute the resulting NativeAOT binaries. Fixture generation and adapter compilation are optional and never prerequisites of ordinary native analysis or setup. + +The real MCP lane runs only on a Linux x64 host with the adapter. The +Windows-host behavior above was checked manually over MCP on Windows 11 x64 +(non-elevated, Ghidra 12.1.4, JDK 21, no JAR) with the source-built win-x64 +symbols fixture. The format, RTR version and frozen strings matched the fixture +oracle; the derived MethodTable, its base type, its interface and a virtual +slot reaching the overridden method matched the linker map; and pseudocode was +readable. No CI lane repeats this check. diff --git a/docs/ghidra-provider.md b/docs/ghidra-provider.md index 5b83ee152..fd23dcd07 100644 --- a/docs/ghidra-provider.md +++ b/docs/ghidra-provider.md @@ -41,6 +41,26 @@ does not prove text is absent. Both list and search Evidence carry this limitation. When an address is known, use `read_bytes` and `xrefs` to inspect bytes and references. +## PE debug-symbol coverage + +The post-analysis handshake reports Ghidra's PDB Program Information attributes: +the referenced file, GUID, hexadecimal age, legacy signature, version, and +`PDB Loaded` state. Absent attributes remain null. PE Evidence carries a +limitation when a PDB reference is present but Ghidra reports it as not loaded, +or its load state is unknown. A bridge without this report leaves PDB coverage +unknown. A reported loaded state is a Ghidra observation, not an independent +PDB identity or complete symbol-coverage attestation, and retains an inline +coverage caveat. Handshake PDB strings share a 4096 UTF-16-unit serialization +budget (at most 24 KiB after JSON escaping); whole attributes that do not fit +are omitted with an explicit resource limitation, rather than truncated. + +Original names and types may be missing even when auto-analysis completes. +An empty name search does not establish that a function is absent; use +`list_procedures` and address-based analysis to inspect recovered functions. +REA imports a private executable copy and does not admit a caller-selected +PDB companion. This reporting does not search symbol servers, copy a sibling +PDB, or change analyzer settings. + ## Loaded addresses and file offsets `rea analyze --provider ghidra --json` and MCP `binary_overview` report @@ -178,3 +198,32 @@ It accepts the same architecture and DLL flags and verifies the selected PE role in the analysis profile. All lanes require matching native controls; package startup alone establishes only package compatibility. + +## Masked byte search + +`search_bytes` (`rea search-bytes PATH '48 8B ?? ?? 89' --provider ghidra`) +searches initialized analyzed memory, including loader fixups. Supply hexadecimal +byte pairs, with `??` for any byte. An optional equal-length hexadecimal `mask` +selects bits: `pattern: "a0"`, `mask: "f0"` matches bytes from `a0` through `af`. +Wildcards remain unconstrained even with an explicit mask. At least one bit +must be constrained. + +Optional `start` and `end` addresses are inclusive and select one address space. +A match must end at or before `end`, so `end` set inside a match excludes it. +Without bounds, loaded memory blocks are eligible, in Ghidra address order. +Blocks that are not loaded (file parts such as `.symtab`) and overlay blocks are +listed in `searched.skipped_not_loaded` and are not searched; pass `start` or +`end` in their address space to search them. +Overlapping matches are included; matches never cross block boundaries. +Each match reports its address, segment, unique source file offset (or null), +and containing procedure (or null). + +Pages default to 100 matches and accept 1 to 1000. Pass `next_cursor` as `cursor` +with the same target, profile, pattern, mask and bounds to continue. `complete` +means no further matches remain. `searched.blocks` and `searched.bytes` describe +the inclusive memory ranges examined for this page, including lookahead for the +next match. Coverage can overlap between pages; do not sum it as unique memory. +`searched.skipped_uninitialized` lists eligible uninitialized blocks on every +page. These blocks are never searched. Cancellation returns a typed error, +discards the active Ghidra database, and does not return a successful partial page. +Hopper byte search remains unavailable until its implementation is verified. diff --git a/docs/http3-capture-lab.md b/docs/http3-capture-lab.md new file mode 100644 index 000000000..b4a41ebb4 --- /dev/null +++ b/docs/http3-capture-lab.md @@ -0,0 +1,183 @@ +# Verify real HTTP/3 MITM captures + +This optional Linux laboratory checks native mitmproxy captures through REA's +public CLI and MCP. It changes no runtime contracts or adapters and establishes +only the specific producer, mode and clients exercised here. + +## Prepare a disposable guest + +The exercised image is [Omarchy 4.0.4, build 440](https://nightly.omarchy.org/stable/4.0.4/440/omarchy-4.0.4.qcow2), +SHA-256 `43412b896fe3d730e373a51a8acf99343d7c3e6dee78aa9f9bd7fca181127264`. +Download it without installing guest packages on the host, check its digest, +and use a **new qcow2 overlay** with that file as a read-only backing image. +The matching [boot artifacts](https://nightly.omarchy.org/stable/4.0.4/440/unpacked/) +include the kernel, initrd, cmdline and upstream vm-boot recipe. Use a fresh SSH +key, loopback forwarding and guest disk; do not reuse another investigation's +writable overlay or host trust configuration. Shut the guest down before +switching overlays. Preserve the image and boot artifact hashes with evidence. + +This sealed guest already supplies cURL 8.22.0 with OpenSSL 3.6.4, ngtcp2 1.25.0, +nghttp3 1.18.0 and nghttp2 1.70.0. **No guest rolling package upgrade is needed.** +Run `curl --version` and verify HTTP2 and HTTP3 features before starting. A stock +cURL from another distribution may omit HTTP/3. For a different guest, building +cURL requires a compatible TLS library, ngtcp2 and nghttp3; follow the +[official build instructions](https://curl.se/docs/http3.html), install to a guest +private prefix, and record exact source revisions and build inputs. That alternative +has not been exercised by this laboratory; it is not the pinned reproduction path. + +Use a fresh VM overlay over a sealed Linux image. Use user-mode networking with +SSH forwarding bound to host loopback; keep host directories unmounted. Install +and build dependencies inside the guest, then remove its default route before +generating traffic. Host trust, routes and installed packages stay unchanged. + +Record the image digest, REA source revision and these dependency versions: + +- REA's `.nvmrc`, `package.json#packageManager` and locked npm dependencies. +- Official Linux x86_64 mitmproxy **12.2.3** standalone archive. SHA-256: + `2e95286b618fa6fd33e5e62a78c2e5112571d85f42ec2bac29b97ee242bdb5c5`. +- A private Python virtual environment using the complete pinned graph in + `scripts/fixtures/http3-lab-requirements.txt`. Save `pip freeze` and build/download logs. +- OpenSSL and cURL advertising HTTP3 and HTTP2 in `curl --version`. + +The verified guest used Python 3.14.7, cURL 8.22.0, ngtcp2 1.25.0 and nghttp3 +1.18.0. Other dependency profiles need verification. Back up the guest's +`/etc/hosts` before adding `127.0.0.1 lab.test`. Reserve guest TCP/UDP loopback +ports 18443 and 19443. These changes belong only to the disposable guest. + +From the guest REA checkout, use its pinned Node/npm toolchain: + +```sh +npm ci +npm run build:cached +python3 -m venv /absolute/lab/venv +/absolute/lab/venv/bin/pip install --no-deps -r scripts/fixtures/http3-lab-requirements.txt +/absolute/lab/venv/bin/pip check +/absolute/lab/venv/bin/pip freeze > /absolute/lab/python-requirements.txt +``` + +Before removing the guest default route, download the official mitmproxy archive +and verify it, then extract to the guest's private tools directory: + +```sh +mkdir -p /absolute/lab/tools +curl --fail --location https://downloads.mitmproxy.org/12.2.3/mitmproxy-12.2.3-linux-x86_64.tar.gz \ + --output /absolute/lab/mitmproxy.tar.gz +printf '%s %s\n' 2e95286b618fa6fd33e5e62a78c2e5112571d85f42ec2bac29b97ee242bdb5c5 \ + /absolute/lab/mitmproxy.tar.gz | sha256sum --check --strict +tar -xzf /absolute/lab/mitmproxy.tar.gz -C /absolute/lab/tools +/absolute/lab/tools/mitmdump --version +``` + +Pin the REA revision with `git rev-parse HEAD` or archive SHA-256. Build using +the exact `.nvmrc` and npm version; do not copy a different checkout's `dist`. +For stronger offline dependency reproduction, preserve downloaded Python +archives, their digests and built wheels along with the image and toolchain. +The requirement versions pin the runtime graph, not every transient build tool; +source builds and alternate interpreters are not guaranteed byte-identical. + +## Run the narrow profile + +Select absolute executables and a new evidence directory. The producer refuses +to reuse a directory. No certificates are installed in a system trust store; +clients and proxy receive explicit paths to freshly generated test CAs. + +```sh +REA_MITMDUMP_COMMAND=/absolute/lab/tools/mitmdump \ +REA_HTTP3_PYTHON_COMMAND=/absolute/lab/venv/bin/python \ +npm run verify:web:http3 -- /absolute/evidence/run-1 +``` + +The producer sends three concurrent binary POSTs on one HTTP/3 connection to the +same URL through mitmproxy reverse mode to an aioquic HTTP/3 origin. The origin +waits for all complete requests, then dispatches responses in reverse order of +completion at the origin. QUIC stream send/receive orders need not coincide. A cURL HTTP/3-only +request checks a second client implementation. Mitmproxy's real save addon +writes `normal.mitm`; no native flows are synthesized. + +A second proxy run disables HTTP/3. cURL `--http3` falls back to HTTP/2, reaching +an HTTP/1.1 TLS test origin. `fallback.mitm` must retain those lower protocols, +so a successful HTTP request cannot masquerade as an HTTP/3 success. + +## Acceptance and evidence + +Each execution creates `verifications//` with its own diagnostics. +`verification-latest.json` belongs to the most recently registered attempt and +changes atomically from `running` to `passed` or `failed`. Registration and +the ownership check/replacement use the same SQLite transaction lock. An older +attempt finishing later keeps its own reports without replacing the marker. +The `.verification-lock.sqlite` pathname must remain in place while attempts +may run; SQLite releases the lock on connection close or process death. +Lock acquisition waits for the current owner without a timeout or lease +stealing. The verifier has per-command deadlines, but no overall run deadline; +a live stalled lock owner can therefore delay publication until it exits. +Only a successful attempt writes its own `verification.json`, after +all cleanup and input-integrity checks. `result.json` also records failures and +the original error separately from cleanup errors. Older successes remain in +identified history; a legacy root `verification.json` is archived before a rerun. +A killed verifier may leave `running`, which must never be interpreted as success. + +Capture inputs live in `captures/`. To inspect them again without new traffic: + +```sh +REA_MITMDUMP_COMMAND=/absolute/lab/tools/mitmdump \ +npm run verify:web:http3 -- /absolute/evidence/run-1 --inspect-existing +``` + +Acceptance for the current run: + +- Every positive native record reports HTTP/3, UDP, ALPN h3 and successful TLS + timestamps on both legs. Connection IDs, producer TLS labels, exact TLS setup + timestamps and connection errors match the UUID-matched producer observation + on each leg. Endpoint handshakes independently observe QUIC v1. +- Native flow UUIDs remain distinct. Three endpoint streams share a connection + on each leg. Origin dispatch reverses completed receipt. Client and capture + ordering is observed, while association is verified independent of their order. +- CLI and MCP agree. Request/response bytes, lengths, SHA-256, ordered duplicate + headers, method, scheme, authority and path match independent observations. + Readable header names and values, request URL fields and ALPN also match; + non-UTF-8 header bytes retain a null display value rather than replacement text. +- Actual MCP advertised schemas validate, Evidence preserves source digests and + provider provenance, and the complete returned Evidence matches the retained MCP session records. +- The fallback capture reports client HTTP/2 over TCP and origin HTTP/1.1, with + cURL independently reporting version 2. Input files remain unchanged. +- Every cleanup action is attempted on success and failure, and final lineage + contains no live descendants. Recorded producer groups are inspected separately + after token-verified cleanup, including after their launcher has exited. + +Retain captures, endpoint/producer JSONL, proxy diagnostics, cURL output, manifest, +CLI/MCP output, advertised schema and session bundle. Export public certificates +and dependency/source provenance separately. Do not commit evidence, private +keys, dependencies or VM disks. Shut down the guest after exporting evidence. + +The reverse proxy can preserve authority `lab.test:18443` while recording origin +port 19443. Mitmproxy's serialized flow/connection UUIDs are not stream IDs. +Endpoint stream IDs are scoped observations; this test invents no cross-leg +mapping. Mitmproxy 12.2.3's `QUICv1` label alone does not prove negotiation: the +fixture checks the negotiated `_quic._version` in pinned aioquic 1.2.0. + +Coverage is native capture, Linux reverse mode, QUIC v1 and these clients. +Empty/absent/incomplete/redacted body semantics, cancellations, resets, +certificate failures, malformed captures, HAR and qlog are separate increments. +Local/WireGuard, browsers, QUIC v2, migration, 0-RTT, mTLS, replay, sustained load +and loss tolerance remain unverified by this profile. Passing it does not +establish general HTTP/3 support in REA. + +Official references: [mitmproxy protocol support](https://github.com/mitmproxy/mitmproxy/blob/v12.2.3/docs/src/content/concepts/protocols.md), +[HTTP/3 implementation](https://github.com/mitmproxy/mitmproxy/blob/v12.2.3/mitmproxy/proxy/layers/http/_http3.py), +[serialized HTTP state](https://github.com/mitmproxy/mitmproxy/blob/v12.2.3/mitmproxy/http.py), +[aioquic configuration](https://github.com/aiortc/aioquic/blob/1.2.0/src/aioquic/quic/configuration.py). + +## Verifier regressions + +After building REA, run these inside the same isolated guest: + +```sh +node --test scripts/lib/http3-verifier-lifecycle.test.mjs scripts/lib/http3-capture-assertions.test.mjs +/absolute/lab/venv/bin/python scripts/fixtures/http3-lab.test.py +``` + +These cover different QUIC stream completion orders, stale success reports, +multiple failed closes, separately owned process groups and partially acquired +resources, plus producer TLS fact mismatches on both legs and altered readable +fields with unchanged binary evidence. They test verifier +semantics; they do not add HTTP/3 provider coverage. diff --git a/docs/ida-provider.md b/docs/ida-provider.md index fbf67c8a2..012f75bfb 100644 --- a/docs/ida-provider.md +++ b/docs/ida-provider.md @@ -4,6 +4,15 @@ REA adapts an existing IDA MCP server into its native analysis tools. Existing I Choose one lifecycle mode in an upstream MCP JSON registration. Set `REA_IDA_MCP_CONFIG` to that file, then select `ida` once through `REA_ANALYSIS_PROVIDER`, the CLI's `--provider ida`, or MCP's `open_binary.provider_id`. No provider startup occurs during discovery. +## Supported upstream versions + +Each mode accepts one upstream tool profile, checked against `tools/list` before any analysis runs. Upstream's `main` branch has since moved to 2.0.0, which renamed the GUI tools and dropped `idb_close`, so it serves neither profile. Installing from that branch fails at startup with the missing tool names in the error diagnostics. + +| Mode | Upstream profile | Known working | +| -------- | ------------------- | ------------------------------------------------------ | +| attached | legacy GUI tools | 1.4.0 | +| headless | database supervisor | builds serving `idb_open`, `idb_list`, and `idb_close` | + ## Analyze an already-open GUI target The attached profile supports upstream **1.4.0 legacy tools**, including `get_metadata`, `list_functions`, and `decompile_function`. Copy the upstream registration's `command`, `args`, and optional `env` into a JSON file. An existing `mcpServers` object with an `ida-pro-mcp` entry is also accepted. For example: diff --git a/docs/installation.md b/docs/installation.md index 9285883e3..a39471585 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -20,7 +20,9 @@ npx rea-agents setup Choose the agents to configure, review the proposed paths and changes, and approve the plan. npm's package-download approval and REA's setup approval are separate. Setup asks before changing agent configuration or installing Hopper. -Restart the agents named in the completion message. +Restart the agents named in the completion message. Quit Claude Desktop fully +before approving, because a running app can overwrite the registration; on +Windows, see [where Claude Desktop reads its configuration](client-configuration.md#claude-desktop). For the latest published release, use `npx rea-agents@latest setup`. [Package-runner and global-installation options](#package-runners-and-global-installation) @@ -277,6 +279,7 @@ hosted connector; see [its configuration](client-configuration.md#other-client-s | GitHub Copilot CLI | `copilot_cli` | | Command Code | `commandcode` | | Qwen Code | `qwen_code` | +| Kimi Code | `kimi` | | VS Code | `vscode` | | Grok Build | `grok_build` | | OMP | `omp` | @@ -288,8 +291,8 @@ hosted connector; see [its configuration](client-configuration.md#other-client-s | ZCode | `zcode` | | DeepSeek Harness | `dsh` | -`dsh` is main-only until a release after 6.4.0. The registry check above does -not cover it. +`dsh` and `kimi` require a release containing them. The registry check above does +not cover them. See [client configuration and troubleshooting](client-configuration.md) for configuration paths, profiles, trust policies, and schema-limit workarounds. @@ -308,9 +311,12 @@ Hopper is separate commercial software with its own license. Its free demo has vendor-defined limits, and a paid license is optional. REA reuses any detected installation and preserves Hopper during uninstall. -The supported native host baseline is macOS 12+, Ubuntu 24.04+, Fedora 41+, -Nobara 44+, 64-bit Arch Linux, CachyOS, or Omarchy. Ghidra and IDA have their own provider-specific -host requirements; Windows Ghidra uses the [experimental P0 boundary](windows-ghidra-p0.md). +The automated Hopper installer supports macOS 12+, Ubuntu 24.04+, Fedora 41+, +Nobara 44+, 64-bit Arch Linux, CachyOS, or Omarchy. REA itself runs on any +Linux distribution with a recognized apt, dnf, or pacman package family, where +an existing Hopper launcher is selected through `HOPPER_LAUNCHER_PATH`. Ghidra +and IDA have their own provider-specific host requirements; Windows Ghidra uses +the [experimental P0 boundary](windows-ghidra-p0.md). On macOS, approved setup downloads the official DMG, checks its published size and digest, validates the application bundle, and atomically installs it to diff --git a/docs/managed-code-analysis.md b/docs/managed-code-analysis.md index d60603ce3..1d1a579e6 100644 --- a/docs/managed-code-analysis.md +++ b/docs/managed-code-analysis.md @@ -233,8 +233,9 @@ these routes, their layout/host restrictions, derived-memory provenance, and verification coverage. REA's experimental Windows Ghidra boundary admits native x86 and x86-64 PE -applications and DLLs for static analysis. This does not establish Windows -NativeAOT metadata-recovery coverage. +applications and DLLs for static analysis. For x64 PE NativeAOT, the read-only +parser also runs there; the [NativeAOT guide](ghidra-nativeaot.md) lists what a +Windows host covers. For NativeAOT analysis, preserve the selected native artifact's path and digest and report recovered type data with its returned provenance. Linking those types diff --git a/docs/mcp-contracts.md b/docs/mcp-contracts.md index f3f7892f6..876a30f6d 100644 --- a/docs/mcp-contracts.md +++ b/docs/mcp-contracts.md @@ -21,6 +21,20 @@ directly to `compare_functions`, and `inspect_artifact` Evidence to `compare_artifacts`. Use `get_evidence_bundle` when the task needs broader retained session history or an explicit bundle for transfer. +Function dossiers can declare per-facet coverage in the optional +`normalized_result.facet_availability` map. Each supplied facet is either +`{ "available": true }` or `{ "available": false, "reason": "..." }`. +Supported keys are `pseudocode`, `assembly`, `comments`, `callers`, `callees`, +`references`, `strings_names`, and `cfg`. Availability describes whether that +facet was observed, not whether its collection contains any entries. + +`compare_functions` keeps a dimension `unknown` with null counts when either +side declares a required facet unavailable. An empty legacy facet without a +coverage declaration is also unknown. A producer that actually observed an +empty facet can declare it available so observed absence remains comparable. +Call comparison additionally retains the Evidence relationship coverage +checks. IDA declares the facets its selected adapter cannot provide; an empty +unsupported array does not prove observed absence. Evidence and bundle imports reject unsupported fields in their envelopes, identities, manifests and other fixed metadata before authentication. Imports leave retained session state unchanged when validation fails. Keep application @@ -51,7 +65,7 @@ unresolved relationship reads. Cancellation remains a tool error. REA prepares complete MCP results within the pinned stdio client's 10 MiB receive-buffer budget, including both text and structured representations and room for the JSON-RPC envelope. Tool errors carry only their text -representation, so only that text counts against the budget. Response budget settings are captured at startup; +representation and `isError` flag, so only those count against the budget. Response budget settings are captured at startup; restart or recreate the server to apply changes. If a result cannot fit, REA returns `resource_constraint` with `details.resource: "transport"` before constructing a document-sized string. Analysis Evidence remains complete in the current @@ -257,9 +271,11 @@ Evidence remain complete inline, and both input forms pass the same semantic, identity, authority, and provenance checks without running the producer again. References belong to the current connection's Evidence ledger. Opening another -target preserves retained records; `close_binary` clears them, even when no -binary is active. A fresh connection has its own ledger. A missing reference -reports its exact ID and `details.reason: "missing"`; the server cannot infer +target or restarting the current target preserves retained records, including +when provider cleanup fails; records remain available for export and retry. +`close_binary` clears them, even when no binary is active. A fresh connection +has its own ledger. A missing reference reports its exact ID and +`details.reason: "missing"`; the server cannot infer whether it was never recorded, cleared, or retained by another connection. Supply complete inline Evidence, repeat its producer, or import an exported Evidence bundle before referencing that imported record. Export a bundle before @@ -588,8 +604,12 @@ failures for such entries name this cause and the operation to rerun with Advertised input and output schemas contain no reachable recursive references. Schemas share repeated definitions through schema-local references while -preserving complete nested fields and validation rules. Input properties retain -their descriptions and literal examples. +preserving complete nested fields and validation rules. Input schemas whose +shared definitions would exhaust a client's declaration-rendering reference +budget (Pi 1.1.0 code-mode shares 32 reference expansions across an input and +renders every later reference as `unknown`) are re-projected without shared +definitions, so every advertised field renders with complete structure. +Input properties retain their descriptions and literal examples. When a host forwards input schemas to the model, their examples consume context. REA's advertised examples therefore never copy complete producer Evidence. @@ -636,7 +656,7 @@ JSON Schema: objects or arrays. - The few schemas whose validation structure alone exceeds the budget (`compare_web_captures`, `build_reconstruction_obligation_ledger`, - `evaluate_reconstruction_coverage`, `project_managed_application_graph`, + `evaluate_reconstruction_coverage`, `compare_application_versions`, `capture_browser_scenario`) are advertised in a reduced form: property names, reference-resolved types (including union alternatives), first-level guidance, and the root's required-field constraints, @@ -646,8 +666,7 @@ JSON Schema: instead of silent acceptance. Boundary tests pin every advertised compact schema to the budget, keep them -valid JSON Schema that still accepts every canonical example, and hold the -set of six reduced presentations as contracts evolve. Client verification +valid JSON Schema that still accepts every canonical example. Client verification also checks provider-facing sizes and canonical examples after conversion by the actual Kimi Code CLI; local captures do not establish Moonshot acceptance. diff --git a/docs/native-investigation.md b/docs/native-investigation.md index 5a2fcbd31..f963fd4d8 100644 --- a/docs/native-investigation.md +++ b/docs/native-investigation.md @@ -549,3 +549,17 @@ Value projection preserves serialized node kinds, IDs, references and pagination Exact and ambiguous value counts describe emitted root/object value instances; a root and an object may contain the same value. Archive digests identify the original bytes. + +## Locate a byte signature + +Use `search_bytes` after opening a target with Ghidra to locate a known constant +or instruction sequence. For example, `{"pattern":"48 8B ?? ?? 89","limit":100}` +returns ordered addresses with segments, source file offsets and containing +procedures. The CLI equivalent is +`rea search-bytes ./program '48 8B ?? ?? 89' --provider ghidra`. +Use `start`/`end` to constrain the inclusive address window, and continue with +`next_cursor` until `complete` is true. Inspect `searched` for page coverage and +skipped uninitialized blocks. A missing match does not establish absence from +uninitialized memory or across block boundaries. See +[Ghidra byte search](ghidra-provider.md#masked-byte-search) for mask and coverage +semantics. This query never executes or modifies target bytes. diff --git a/README_ar.md b/docs/readme/README_ar.md similarity index 77% rename from README_ar.md rename to docs/readme/README_ar.md index 1c0bc367e..8df2522bc 100644 --- a/README_ar.md +++ b/docs/readme/README_ar.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · **العربية** · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · **العربية** · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA: هندسة عكسية لأي شيء @@ -9,10 +9,10 @@ **وجدت ميزة تعجبك؟ افهم طريقة عملها حتى مستوى الشيفرة الثنائية.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA يشغّل جسر التحليل داخل Hopper لفحص ملف ثنائي للشيفرة الأصلية +REA يشغّل جسر التحليل داخل Hopper لفحص ملف ثنائي للشيفرة الأصلية

@@ -33,7 +33,7 @@ - Discord
+ Discord
انضم إلى مجتمع الهندسة العكسية
Discord · أسئلة وأجوبة · مشاركة النتائج @@ -67,7 +67,7 @@ npx rea-agents setup اختر الوكلاء، وراجع التغييرات المقترحة، ثم وافق عليها. يضيف الإعداد خادم MCP الخاص بـ REA وتعليمات سير العمل المطابقة، مع الاحتفاظ بنسخ احتياطية من الإعدادات الموجودة. أعد تشغيل وكيلك بعد ذلك. -يدعم الإعداد Claude Code وCodex وCursor وGemini CLI وGrok Build و[وكلاء آخرين](docs/installation.md#supported-agents). راجع [التثبيت والإعداد](docs/installation.md) لتهيئة موفّري التحليل وتسجيل MCP يدويًا. +يدعم الإعداد Claude Code وCodex وCursor وGemini CLI وGrok Build و[وكلاء آخرين](../installation.md#supported-agents). راجع [التثبيت والإعداد](../installation.md) لتهيئة موفّري التحليل وتسجيل MCP يدويًا. ### اطلب من وكيلك @@ -94,7 +94,7 @@ npm install --global rea-agents rea --help ``` -لتحليل الشيفرة الأصلية، هيّئ موفّرًا أولًا. راجع [دليل CLI وEvidence](docs/cli.md) لأوامر تحليل الشيفرة الأصلية، واختيار الموفّر، واللقطات، والاستخدام في السكربتات. +لتحليل الشيفرة الأصلية، هيّئ موفّرًا أولًا. راجع [دليل CLI وEvidence](../cli.md) لأوامر تحليل الشيفرة الأصلية، واختيار الموفّر، واللقطات، والاستخدام في السكربتات. ### تحديث REA @@ -120,9 +120,9 @@ npx rea-agents@latest setup يستدعي وكيلك REA عبر MCP لفحص الهدف وتتبع الشيفرة ذات الصلة. يعيد REA النتائج مع أدلتها. يستخدمها الوكيل لطرح أسئلة متابعة، أو شرح السلوك، أو كتابة تنفيذ واختباره. تستخدم أوامر CLI سير العمل نفسه. -![مسار الاستقصاء في REA: يسأل وكيلك عن هدف محلي، ويفحصه REA ويتتبعه بأدوات التحليل، ثم يستخدم الوكيل الشيفرة والمراجع والجوانب غير المحسومة التي أعادتها الأدوات لشرح السلوك وتنفيذه واختباره.](website/public/assets/figures/rea-investigation-flow.svg) +![مسار الاستقصاء في REA: يسأل وكيلك عن هدف محلي، ويفحصه REA ويتتبعه بأدوات التحليل، ثم يستخدم الوكيل الشيفرة والمراجع والجوانب غير المحسومة التي أعادتها الأدوات لشرح السلوك وتنفيذه واختباره.](../../website/public/assets/figures/rea-investigation-flow.svg) -[افتح الشكل بحجمه الكامل](website/public/assets/figures/rea-investigation-flow.svg). +[افتح الشكل بحجمه الكامل](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -130,37 +130,37 @@ npx rea-agents@latest setup يتطلب REA إصدار Node.js 22.x (>=22.19) أو 24.x (>=24.11) أو 26+، إضافة إلى npm. تعتمد الأدوات الإضافية والأنظمة المضيفة المدعومة على الهدف: -| الهدف | ما يعيده REA | المتطلبات والدليل | -| ------------------------------- | ---------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | -| الملفات الثنائية للشيفرة الأصلية | شيفرة شبه برمجية، وتعليمات تجميع، وسلاسل نصية، ورموز، واستدعاءات، ومراجع | Hopper أو Ghidra أو IDA؛ [تحليل الشيفرة الأصلية](https://rea.tools/ar/guides/native/) | -| بنية ELF دون تشغيل | الأقسام والمقاطع والرموز ومعلومات إعادة التموضع الأصلية، وآليات الحماية المحتملة المستدلّ عليها بالفحص الثابت | pwntools يوفّره المستدعي على Linux x64؛ [تشخيص الملفات الثنائية](docs/binary-diagnostics.md) | -| شيفرة EVM البايتية | محددات التوجيه، وإزاحات البايتات، والمعاملات المستنتجة، وقابلية تغيير الحالة | مدخل محلي يحمل بايتات خامًا أو تمثيلًا سداسيًا عشريًا على Linux x64؛ [دليل EVM دون تشغيل](docs/evm-bytecode.md) | -| أعطال Linux المسجّلة | سجلات note الخام، ومسجّلات المعالج/الإشارات لكل خيط مسجّل، وترشيحات اختيارية لخرائط الذاكرة | pwntools يوفّره المستدعي على Linux x64؛ GDB/pwndbg اختياريان؛ [الأعطال المسجّلة](docs/recorded-crashes.md) | -| JavaScript / Electron | الوحدات، وعمليات الاستيراد، وخرائط المصدر، والمسارات، وIPC، وعلاقات إضافات الشيفرة الأصلية | Node.js وnpm؛ [تحليل التطبيقات](https://rea.tools/ar/guides/javascript/) | -| المواقع | بنية الصفحة، والسكربتات، وملاحظات الشبكة، ولقطات الشاشة المطلوبة | متصفح من عائلة Chrome؛ [تحليل المتصفح](https://rea.tools/ar/guides/browser/) | -| تسجيلات الشبكة المحفوظة | الطلبات، والاستجابات، ومحتويات الحمولة المتاحة، ومواضعها في المصدر | HAR؛ وmitmdump على Linux لتسجيلات mitmproxy بصيغتها الأصلية؛ [دليل تسجيلات الشبكة](docs/web-network-captures.md) | -| التقاط الحزم | نقاط النهاية وDNS وTLS SNI وتوزيع البروتوكولات وطلبات HTTP | tshark وcapinfos يوفّرهما المُستدعي؛ [دليل pcap](docs/pcap-analysis.md) | -| تجميعات .NET | البيانات الوصفية، وتعليمات CIL، واعتماديات الشيفرة الأصلية المعلنة، ومقارنات البناء | فحص ثابت؛ [دليل الشيفرة المُدارة](docs/managed-code-analysis.md) | -| حزم Android APK | تصريحات ملف manifest، والأصناف، والدوال الناتجة عن فك الترجمة، والمراجع | JADX دون واجهة رسومية وJDK كامل على Linux/macOS/Windows x64؛ [دليل Android](docs/android-analysis.md) | -| أجهزة Android | الأجهزة والحزم والعمليات والسجلات والدامبات ولقطات الشاشة ونقل APK/الملفات | adb يوفّره المُستدعي (محاكي أو جهاز)؛ [دليل ADB](docs/adb-device-analysis.md) | -| مشاريع JEB | وحدات المشروع من المحرك وبصمات المصنّفات والكود الزائف للأنواع/الدوال | عميل JEB يبدأه المُستدعي ويقدّم MCP، ويُختار عبر REA_JEB_MCP_URL؛ [دليل JEB](docs/jeb-analysis.md) | -| موارد Android | البيان المفكوك الترميز وجداول السلاسل واللغات وحقائق الإصدار | مُشغّل Apktool يوفّره المُستدعي؛ [دليل الموارد](docs/apktool-resource-analysis.md) | -| تطبيقات Flutter | بصمة لقطة Dart وهشوش المكتبات لكل ABI وأصل بناء المحرك | لا شيء (تحليل APK ساكن)؛ [دليل Flutter](docs/flutter-analysis.md) | -| البرامج الثابتة | المناطق، ونتائج الاستخراج، وما يُمرّر إلى تحليل الشيفرة الأصلية | Binwalk / Unblob على Linux؛ [دليل البرامج الثابتة](docs/firmware-analysis.md) | -| الحزم المسجلة | حقول مختارة وقيم متكررة ومرشحات وقواعد فك الترميز | Linux: TShark 4.x + prlimit; [دليل التقاط الحزم](docs/packet-captures.md) | -| الإشارات المنطقية المسجلة | تعليقات مفكك الترميز ونطاقات العينات ومعلومات التشخيص | Linux: sigrok-cli 0.7.2 + prlimit; [دليل الإشارات](docs/signal-recordings.md) | -| الحزم والموارد | قوائم الملفات، والبصمات، وملفات plist، وبنية حزم Apple، والموارد المستخرجة | [دليل تحليل الملفات وJavaScript](docs/javascript-artifact-reconstruction.md)، [تطبيقات Apple](docs/apple-application-analysis.md) | -| سلوك العمليات | مخرجات الطرفية، والتفاعلات، وملاحظات الخروج ونظام الملفات، ومقارنات التشغيل | Linux/macOS مع PTY أصلي؛ [تسجيل سلوك العمليات](docs/process-capture.md) | +| الهدف | ما يعيده REA | المتطلبات والدليل | +| ------------------------------- | ---------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | +| الملفات الثنائية للشيفرة الأصلية | شيفرة شبه برمجية، وتعليمات تجميع، وسلاسل نصية، ورموز، واستدعاءات، ومراجع | Hopper أو Ghidra أو IDA؛ [تحليل الشيفرة الأصلية](https://rea.tools/ar/guides/native/) | +| بنية ELF دون تشغيل | الأقسام والمقاطع والرموز ومعلومات إعادة التموضع الأصلية، وآليات الحماية المحتملة المستدلّ عليها بالفحص الثابت | pwntools يوفّره المستدعي على Linux x64؛ [تشخيص الملفات الثنائية](../binary-diagnostics.md) | +| شيفرة EVM البايتية | محددات التوجيه، وإزاحات البايتات، والمعاملات المستنتجة، وقابلية تغيير الحالة | مدخل محلي يحمل بايتات خامًا أو تمثيلًا سداسيًا عشريًا على Linux x64؛ [دليل EVM دون تشغيل](../evm-bytecode.md) | +| أعطال Linux المسجّلة | سجلات note الخام، ومسجّلات المعالج/الإشارات لكل خيط مسجّل، وترشيحات اختيارية لخرائط الذاكرة | pwntools يوفّره المستدعي على Linux x64؛ GDB/pwndbg اختياريان؛ [الأعطال المسجّلة](../recorded-crashes.md) | +| JavaScript / Electron | الوحدات، وعمليات الاستيراد، وخرائط المصدر، والمسارات، وIPC، وعلاقات إضافات الشيفرة الأصلية | Node.js وnpm؛ [تحليل التطبيقات](https://rea.tools/ar/guides/javascript/) | +| المواقع | بنية الصفحة، والسكربتات، وملاحظات الشبكة، ولقطات الشاشة المطلوبة | متصفح من عائلة Chrome؛ [تحليل المتصفح](https://rea.tools/ar/guides/browser/) | +| تسجيلات الشبكة المحفوظة | الطلبات، والاستجابات، ومحتويات الحمولة المتاحة، ومواضعها في المصدر | HAR؛ وmitmdump على Linux لتسجيلات mitmproxy بصيغتها الأصلية؛ [دليل تسجيلات الشبكة](../web-network-captures.md) | +| التقاط الحزم | نقاط النهاية وDNS وTLS SNI وتوزيع البروتوكولات وطلبات HTTP | tshark وcapinfos يوفّرهما المُستدعي؛ [دليل pcap](../pcap-analysis.md) | +| تجميعات .NET | البيانات الوصفية، وتعليمات CIL، واعتماديات الشيفرة الأصلية المعلنة، ومقارنات البناء | فحص ثابت؛ [دليل الشيفرة المُدارة](../managed-code-analysis.md) | +| حزم Android APK | تصريحات ملف manifest، والأصناف، والدوال الناتجة عن فك الترجمة، والمراجع | JADX دون واجهة رسومية وJDK كامل على Linux/macOS/Windows x64؛ [دليل Android](../android-analysis.md) | +| أجهزة Android | الأجهزة والحزم والعمليات والسجلات والدامبات ولقطات الشاشة ونقل APK/الملفات | adb يوفّره المُستدعي (محاكي أو جهاز)؛ [دليل ADB](../adb-device-analysis.md) | +| مشاريع JEB | وحدات المشروع من المحرك وبصمات المصنّفات والكود الزائف للأنواع/الدوال | عميل JEB يبدأه المُستدعي ويقدّم MCP، ويُختار عبر REA_JEB_MCP_URL؛ [دليل JEB](../jeb-analysis.md) | +| موارد Android | البيان المفكوك الترميز وجداول السلاسل واللغات وحقائق الإصدار | مُشغّل Apktool يوفّره المُستدعي؛ [دليل الموارد](../apktool-resource-analysis.md) | +| تطبيقات Flutter | بصمة لقطة Dart وهشوش المكتبات لكل ABI وأصل بناء المحرك | لا شيء (تحليل APK ساكن)؛ [دليل Flutter](../flutter-analysis.md) | +| البرامج الثابتة | المناطق، ونتائج الاستخراج، وما يُمرّر إلى تحليل الشيفرة الأصلية | Binwalk / Unblob على Linux؛ [دليل البرامج الثابتة](../firmware-analysis.md) | +| الحزم المسجلة | حقول مختارة وقيم متكررة ومرشحات وقواعد فك الترميز | Linux: TShark 4.x + prlimit; [دليل التقاط الحزم](../packet-captures.md) | +| الإشارات المنطقية المسجلة | تعليقات مفكك الترميز ونطاقات العينات ومعلومات التشخيص | Linux: sigrok-cli 0.7.2 + prlimit; [دليل الإشارات](../signal-recordings.md) | +| الحزم والموارد | قوائم الملفات، والبصمات، وملفات plist، وبنية حزم Apple، والموارد المستخرجة | [دليل تحليل الملفات وJavaScript](../javascript-artifact-reconstruction.md)، [تطبيقات Apple](../apple-application-analysis.md) | +| سلوك العمليات | مخرجات الطرفية، والتفاعلات، وملاحظات الخروج ونظام الملفات، ومقارنات التشغيل | Linux/macOS مع PTY أصلي؛ [تسجيل سلوك العمليات](../process-capture.md) | يقرأ الفحص الثابت لـ JavaScript و.NET الملفات المقدّمة دون تشغيل التطبيق. يشغّل التسجيل أثناء التنفيذ الهدف المحدد أو يتفاعل معه بصلاحيات مستخدمك؛ ويشرح كل دليل للتسجيل أثناء التنفيذ آثاره. -تختلف صيغ ملفات الشيفرة الأصلية والأنظمة المضيفة المدعومة باختلاف الموفّر. راجع [إعداد Hopper وGhidra](docs/installation.md#hopper)، و[دليل IDA](docs/ida-provider.md)، و[دعم Ghidra التجريبي على Windows](docs/windows-ghidra-p0.md). يدعم Ghidra أيضًا [تحليل DOS ذي 16 بت](docs/ghidra-dos.md). لاختيار الموفّر، راجع [دليل CLI](docs/cli.md#choose-a-provider). تحقّق من [توفر الميزات في الإصدارات](docs/installation.md#released-package-and-main) للميزات المضافة بعد أحدث إصدار على npm. +تختلف صيغ ملفات الشيفرة الأصلية والأنظمة المضيفة المدعومة باختلاف الموفّر. راجع [إعداد Hopper وGhidra](../installation.md#hopper)، و[دليل IDA](../ida-provider.md)، و[دعم Ghidra التجريبي على Windows](../windows-ghidra-p0.md). يدعم Ghidra أيضًا [تحليل DOS ذي 16 بت](../ghidra-dos.md). لاختيار الموفّر، راجع [دليل CLI](../cli.md#choose-a-provider). تحقّق من [توفر الميزات في الإصدارات](../installation.md#released-package-and-main) للميزات المضافة بعد أحدث إصدار على npm. ## دراسات الحالة -[![رسوم توضيحية لأمثلة توزيع الصوت في DX-Ball وجسر الحافظة في Notion وحلقة الطلقات في TH04](docs/assets/rea-showcases.png)](https://rea.tools/ar/showcase/) +[![رسوم توضيحية لأمثلة توزيع الصوت في DX-Ball وجسر الحافظة في Notion وحلقة الطلقات في TH04](../assets/rea-showcases.png)](https://rea.tools/ar/showcase/) ### DX-Ball: إعادة بناء حساب توزيع الصوت بين القناتين @@ -189,28 +189,28 @@ npx rea-agents@latest setup
أي وكلاء يمكنهم استخدام REA؟ -أي وكيل يدعم خوادم MCP المحلية. يهيّئ الإعداد [الوكلاء المدعومين](docs/installation.md#supported-agents)؛ ويمكن للعملاء الآخرين استخدام [تسجيل MCP يدويًا](docs/installation.md#mcp-registry). +أي وكيل يدعم خوادم MCP المحلية. يهيّئ الإعداد [الوكلاء المدعومين](../installation.md#supported-agents)؛ ويمكن للعملاء الآخرين استخدام [تسجيل MCP يدويًا](../installation.md#mcp-registry).
هل أحتاج إلى Hopper أو Ghidra أو IDA؟ -يستخدم تحليل الشيفرة الأصلية المتعمق أحدها. يعمل الفحص الثابت لـ JavaScript و.NET دون محرك لتحليل الشيفرة الأصلية. يستطيع الإعداد تثبيت Hopper بعد الموافقة؛ أما Ghidra وIDA فيستخدمان تثبيتاتك الموجودة. راجع [إعداد الموفّرين](docs/installation.md#hopper). +يستخدم تحليل الشيفرة الأصلية المتعمق أحدها. يعمل الفحص الثابت لـ JavaScript و.NET دون محرك لتحليل الشيفرة الأصلية. يستطيع الإعداد تثبيت Hopper بعد الموافقة؛ أما Ghidra وIDA فيستخدمان تثبيتاتك الموجودة. راجع [إعداد الموفّرين](../installation.md#hopper).
هل أحتاج إلى تشغيل Hopper أولًا؟ -يشغّل REA برنامج Hopper عندما تحتاج العملية إليه. على macOS، قد تظهر عند التشغيل الأول نافذة تطلب اختيار وضع العرض التجريبي أو تفعيل الترخيص. راجع [تشغيل Hopper واستكشاف الأخطاء](docs/installation.md#launcher-paths-and-troubleshooting). +يشغّل REA برنامج Hopper عندما تحتاج العملية إليه. على macOS، قد تظهر عند التشغيل الأول نافذة تطلب اختيار وضع العرض التجريبي أو تفعيل الترخيص. راجع [تشغيل Hopper واستكشاف الأخطاء](../installation.md#launcher-paths-and-troubleshooting).
ماذا يفعل تثبيت المهارة من skills.sh؟ -توفّر المهارة تعليمات الاستقصاء لوكيلك. استخدم `npx rea-agents setup` لتسجيل خادم MCP الخاص بـ REA وتثبيت التعليمات المطابقة، ثم أعد تشغيل وكيلك. راجع [تثبيت المهارة وحدها](docs/installation.md#skill-only-installation). +توفّر المهارة تعليمات الاستقصاء لوكيلك. استخدم `npx rea-agents setup` لتسجيل خادم MCP الخاص بـ REA وتثبيت التعليمات المطابقة، ثم أعد تشغيل وكيلك. راجع [تثبيت المهارة وحدها](../installation.md#skill-only-installation).
@@ -253,14 +253,14 @@ npx rea-agents@latest setup ابدأ بـ [الأدلة العملية](https://rea.tools/ar/guides/) على الموقع. للخيارات الدقيقة والمتطلبات وعقود النتائج: -- [التثبيت والإعداد](docs/installation.md): تسجيل الوكلاء، وتهيئة الموفّرين، والتحديثات، وإلغاء التثبيت. -- [الجاهزية واستكشاف الأخطاء](docs/installation.md#check-readiness-for-your-task): تشخيص وكيل أو محرك تحليل محدد. -- [CLI وEvidence](docs/cli.md): الأوامر، واختيار الموفّر، واللقطات، والاستيراد/التصدير، وحالات الخروج. -- [عقود MCP](docs/mcp-contracts.md) و[تعليمات الوكلاء](docs/mcp-prompts.md): نتائج الأدوات، والجلسات، والاستقصاءات الموجّهة. -- [كتالوج الأدوات](docs/mcp-contracts.md#generated-catalog): قائمة الأدوات والموفّرين وأوامر CLI المولّدة أثناء البناء. -- [خطة العمل](docs/roadmap.md): الأعمال المخطط لها ومتابعة القدرات. +- [التثبيت والإعداد](../installation.md): تسجيل الوكلاء، وتهيئة الموفّرين، والتحديثات، وإلغاء التثبيت. +- [الجاهزية واستكشاف الأخطاء](../installation.md#check-readiness-for-your-task): تشخيص وكيل أو محرك تحليل محدد. +- [CLI وEvidence](../cli.md): الأوامر، واختيار الموفّر، واللقطات، والاستيراد/التصدير، وحالات الخروج. +- [عقود MCP](../mcp-contracts.md) و[تعليمات الوكلاء](../mcp-prompts.md): نتائج الأدوات، والجلسات، والاستقصاءات الموجّهة. +- [كتالوج الأدوات](../mcp-contracts.md#generated-catalog): قائمة الأدوات والموفّرين وأوامر CLI المولّدة أثناء البناء. +- [خطة العمل](../roadmap.md): الأعمال المخطط لها ومتابعة القدرات. -أبلغ عن الثغرات وفق [SECURITY.md](SECURITY.md). +أبلغ عن الثغرات وفق [SECURITY.md](../../SECURITY.md). ## سجل النجوم @@ -278,7 +278,7 @@ npx rea-agents@latest setup ## إخلاء المسؤولية -راجع [DISCLAIMER.md](DISCLAIMER.md) للحصول على إرشادات الاستخدام المسؤول. +راجع [DISCLAIMER.md](../../DISCLAIMER.md) للحصول على إرشادات الاستخدام المسؤول. REA مخصص للبحث والتعليم والتشغيل البيني وحفظ البرمجيات بشكل قانوني، ولتحليل البرمجيات التي تملكها أو لديك تفويض لفحصها. @@ -290,14 +290,14 @@ applicable licenses. نرحّب بمساعدتك في تطوير REA! [افتح issue](https://github.com/morluto/rea/issues) للإبلاغ عن خطأ أو اقتراح ميزة، أو [أرسل pull request](https://github.com/morluto/rea/pulls) لتحسين الشيفرة أو التوثيق. -راجع [CONTRIBUTING.md](CONTRIBUTING.md) لإعداد بيئة التطوير والفحوص، و[دليل الاختبار](docs/testing.md) لمسارات التحقق، و[خريطة البنية](docs/architecture.mermaid) لهيكل المشروع. +راجع [CONTRIBUTING.md](../../CONTRIBUTING.md) لإعداد بيئة التطوير والفحوص، و[دليل الاختبار](../testing.md) لمسارات التحقق، و[خريطة البنية](../architecture.mermaid) لهيكل المشروع. ## الترخيص -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_de.md b/docs/readme/README_de.md similarity index 76% rename from README_de.md rename to docs/readme/README_de.md index 2c6d20290..2dbb1d41a 100644 --- a/README_de.md +++ b/docs/readme/README_de.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · **Deutsch** · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · **Deutsch** · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA: Alles reverse-engineeren @@ -9,10 +9,10 @@ **Eine Funktion gefällt dir? Verstehe, wie sie funktioniert – bis hinunter auf die Binärebene.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA startet seine Analysebrücke in Hopper und untersucht eine native Binärdatei +REA startet seine Analysebrücke in Hopper und untersucht eine native Binärdatei

@@ -33,7 +33,7 @@ - Discord
+ Discord
Der Reverse-Engineering-Community beitreten
Discord · Fragen und Antworten · Ergebnisse teilen @@ -67,7 +67,7 @@ npx rea-agents setup Wähle deine Agenten aus, prüfe die geplanten Änderungen und bestätige sie. Die Einrichtung fügt den MCP-Server von REA und passende Workflow-Anweisungen hinzu und sichert vorhandene Konfigurationen. Starte deinen Agenten anschließend neu. -Die Einrichtung unterstützt Claude Code, Codex, Cursor, Gemini CLI, Grok Build und [weitere Agenten](docs/installation.md#supported-agents). Informationen zur Provider-Konfiguration und manuellen MCP-Registrierung findest du unter [Installation und Einrichtung](docs/installation.md). +Die Einrichtung unterstützt Claude Code, Codex, Cursor, Gemini CLI, Grok Build und [weitere Agenten](../installation.md#supported-agents). Informationen zur Provider-Konfiguration und manuellen MCP-Registrierung findest du unter [Installation und Einrichtung](../installation.md). ### Deinen Agenten fragen @@ -94,7 +94,7 @@ npm install --global rea-agents rea --help ``` -Konfiguriere für native Analysen zuerst einen Provider. Native Befehle, Provider-Auswahl, Snapshots und Skriptnutzung beschreibt der [CLI- und Evidence-Leitfaden](docs/cli.md). +Konfiguriere für native Analysen zuerst einen Provider. Native Befehle, Provider-Auswahl, Snapshots und Skriptnutzung beschreibt der [CLI- und Evidence-Leitfaden](../cli.md). ### REA aktualisieren @@ -120,9 +120,9 @@ Prüfe die Änderungen und starte deinen Agenten neu. Für einzelne CLI-Aufrufe Dein Agent ruft REA über MCP auf, um das Ziel zu untersuchen und relevanten Code zu verfolgen. REA liefert Ergebnisse mit ihren Belegen zurück. Der Agent nutzt sie für weitere Fragen, Erklärungen oder zum Schreiben und Testen einer Implementierung. CLI-Befehle verwenden dieselben Workflows. -![REA-Untersuchungsablauf: Dein Agent fragt nach einem lokalen Ziel, REA untersucht und verfolgt es mit Analysewerkzeugen, und der Agent nutzt den zurückgegebenen Code, Verweise und offene Fragen zum Erklären, Implementieren und Testen.](website/public/assets/figures/rea-investigation-flow.svg) +![REA-Untersuchungsablauf: Dein Agent fragt nach einem lokalen Ziel, REA untersucht und verfolgt es mit Analysewerkzeugen, und der Agent nutzt den zurückgegebenen Code, Verweise und offene Fragen zum Erklären, Implementieren und Testen.](../../website/public/assets/figures/rea-investigation-flow.svg) -[Abbildung in voller Größe öffnen](website/public/assets/figures/rea-investigation-flow.svg). +[Abbildung in voller Größe öffnen](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -130,37 +130,37 @@ Dein Agent ruft REA über MCP auf, um das Ziel zu untersuchen und relevanten Cod REA benötigt Node.js 22.x (>=22.19), 24.x (>=24.11) oder 26+ sowie npm. Zusätzliche Werkzeuge und unterstützte Hostsysteme hängen vom Ziel ab: -| Ziel | Was REA zurückgibt | Voraussetzungen und Anleitung | -| ----------------------------------- | --------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | -| Native Binärdateien | Pseudocode, Assemblercode, Zeichenketten, Symbole, Aufrufe und Verweise | Hopper, Ghidra oder IDA; [native Analyse](https://rea.tools/de/guides/native/) | -| Offline-ELF-Struktur | Sektionen, Segmente, ursprüngliche Symbole/Relokationen und mögliche Schutzmaßnahmen aus statischer Analyse | Vom Aufrufer bereitgestelltes pwntools unter Linux x64; [Binärdiagnostik](docs/binary-diagnostics.md) | -| EVM-Bytecode | Dispatch-Selektoren, Byte-Offsets, abgeleitete Argumente und Veränderbarkeit des Zustands | Lokale Eingabe mit Rohbytes oder Hexdarstellung unter Linux x64; [Offline-EVM-Leitfaden](docs/evm-bytecode.md) | -| Aufgezeichnete Linux-Abstürze | Unverarbeitete Note-Einträge, Register/Signale aller aufgezeichneten Threads und optionale Zuordnungskandidaten | Vom Aufrufer bereitgestelltes pwntools unter Linux x64; optional GDB/pwndbg; [aufgezeichnete Abstürze](docs/recorded-crashes.md) | -| JavaScript / Electron | Module, Imports, Source Maps, Routen, IPC und Beziehungen zu nativen Add-ons | Node.js und npm; [Anwendungsanalyse](https://rea.tools/de/guides/javascript/) | -| Websites | Seitenstruktur, Skripte, Netzwerkbeobachtungen und angeforderte Screenshots | Ein Browser aus der Chrome-Familie; [Browseranalyse](https://rea.tools/de/guides/browser/) | -| Gespeicherte Netzwerkaufzeichnungen | Anfragen, Antworten, zugängliche Nutzdaten und Quellpositionen | HAR; mitmdump unter Linux für Aufzeichnungen im nativen mitmproxy-Format; [Leitfaden zu Netzwerkaufzeichnungen](docs/web-network-captures.md) | -| Paketerfassungen | Endpunkte, DNS, TLS SNI, Protokollverteilung und HTTP-Anfragen | Vom Aufrufer bereitgestellte tshark- und capinfos-Binaries; [pcap-Leitfaden](docs/pcap-analysis.md) | -| .NET-Assemblies | Metadaten, CIL-Anweisungen, deklarierte native Abhängigkeiten und Build-Vergleiche | Statische Untersuchung; [Leitfaden für verwalteten Code](docs/managed-code-analysis.md) | -| Android-APKs | Manifest-Deklarationen, Klassen, dekompilierte Methoden und Verweise | JADX ohne grafische Oberfläche und ein vollständiges JDK unter Linux/macOS/Windows x64; [Android-Leitfaden](docs/android-analysis.md) | -| Android-Geräte | Geräte, Pakete, Prozesse, Logs, Dumps, Bildschirmaufnahmen sowie APK- und Dateiübertragungen | Vom Aufrufer bereitgestelltes adb (Emulator oder Gerät); [ADB-Leitfaden](docs/adb-device-analysis.md) | -| JEB-Projekte | Engine-gestützte Projekteinheiten, Artefakt-Prüfsummen und Pseudokode von Typen/Methoden | Ein vom Aufrufer gestarteter JEB-Client mit MCP, ausgewählt über REA_JEB_MCP_URL; [JEB-Leitfaden](docs/jeb-analysis.md) | -| Android-Ressourcen | Dekodiertes Manifest, String-Tabellen, Sprachen und Versionsfakten | Vom Aufrufer bereitgestellter Apktool-Starter; [Ressourcen-Leitfaden](docs/apktool-resource-analysis.md) | -| Flutter-Apps | Dart-Snapshot-Hash, Per-ABI-Bibliotheksprüfsummen und Engine-Build-Provenienz | Keine (statische APK-Analyse); [Flutter-Leitfaden](docs/flutter-analysis.md) | -| Firmware | Bereiche, Extraktionsergebnisse und Übergaben an die native Analyse | Binwalk / Unblob unter Linux; [Firmware-Leitfaden](docs/firmware-analysis.md) | -| Aufgezeichnete Pakete | Ausgewählte Felder, wiederholte Werte, Filter und Decode-as-Regeln | Linux: TShark 4.x + prlimit; [Mitschnitt-Anleitung](docs/packet-captures.md) | -| Aufgezeichnete Logiksignale | Decoder-Anmerkungen, Sample-Bereiche und Diagnosen | Linux: sigrok-cli 0.7.2 + prlimit; [Signalaufzeichnungen](docs/signal-recordings.md) | -| Pakete und Ressourcen | Dateiinventare, Hashwerte, plists, Aufbau von Apple-Bundles und extrahierte Ressourcen | [Artefakt- und JavaScript-Leitfaden](docs/javascript-artifact-reconstruction.md), [Apple-Anwendungen](docs/apple-application-analysis.md) | -| Prozessverhalten | Terminalausgabe, Interaktionen, Beobachtungen zu Prozessende und Dateisystem sowie Laufvergleiche | Linux/macOS mit nativer PTY; [Prozessaufzeichnung](docs/process-capture.md) | +| Ziel | Was REA zurückgibt | Voraussetzungen und Anleitung | +| ----------------------------------- | --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | +| Native Binärdateien | Pseudocode, Assemblercode, Zeichenketten, Symbole, Aufrufe und Verweise | Hopper, Ghidra oder IDA; [native Analyse](https://rea.tools/de/guides/native/) | +| Offline-ELF-Struktur | Sektionen, Segmente, ursprüngliche Symbole/Relokationen und mögliche Schutzmaßnahmen aus statischer Analyse | Vom Aufrufer bereitgestelltes pwntools unter Linux x64; [Binärdiagnostik](../binary-diagnostics.md) | +| EVM-Bytecode | Dispatch-Selektoren, Byte-Offsets, abgeleitete Argumente und Veränderbarkeit des Zustands | Lokale Eingabe mit Rohbytes oder Hexdarstellung unter Linux x64; [Offline-EVM-Leitfaden](../evm-bytecode.md) | +| Aufgezeichnete Linux-Abstürze | Unverarbeitete Note-Einträge, Register/Signale aller aufgezeichneten Threads und optionale Zuordnungskandidaten | Vom Aufrufer bereitgestelltes pwntools unter Linux x64; optional GDB/pwndbg; [aufgezeichnete Abstürze](../recorded-crashes.md) | +| JavaScript / Electron | Module, Imports, Source Maps, Routen, IPC und Beziehungen zu nativen Add-ons | Node.js und npm; [Anwendungsanalyse](https://rea.tools/de/guides/javascript/) | +| Websites | Seitenstruktur, Skripte, Netzwerkbeobachtungen und angeforderte Screenshots | Ein Browser aus der Chrome-Familie; [Browseranalyse](https://rea.tools/de/guides/browser/) | +| Gespeicherte Netzwerkaufzeichnungen | Anfragen, Antworten, zugängliche Nutzdaten und Quellpositionen | HAR; mitmdump unter Linux für Aufzeichnungen im nativen mitmproxy-Format; [Leitfaden zu Netzwerkaufzeichnungen](../web-network-captures.md) | +| Paketerfassungen | Endpunkte, DNS, TLS SNI, Protokollverteilung und HTTP-Anfragen | Vom Aufrufer bereitgestellte tshark- und capinfos-Binaries; [pcap-Leitfaden](../pcap-analysis.md) | +| .NET-Assemblies | Metadaten, CIL-Anweisungen, deklarierte native Abhängigkeiten und Build-Vergleiche | Statische Untersuchung; [Leitfaden für verwalteten Code](../managed-code-analysis.md) | +| Android-APKs | Manifest-Deklarationen, Klassen, dekompilierte Methoden und Verweise | JADX ohne grafische Oberfläche und ein vollständiges JDK unter Linux/macOS/Windows x64; [Android-Leitfaden](../android-analysis.md) | +| Android-Geräte | Geräte, Pakete, Prozesse, Logs, Dumps, Bildschirmaufnahmen sowie APK- und Dateiübertragungen | Vom Aufrufer bereitgestelltes adb (Emulator oder Gerät); [ADB-Leitfaden](../adb-device-analysis.md) | +| JEB-Projekte | Engine-gestützte Projekteinheiten, Artefakt-Prüfsummen und Pseudokode von Typen/Methoden | Ein vom Aufrufer gestarteter JEB-Client mit MCP, ausgewählt über REA_JEB_MCP_URL; [JEB-Leitfaden](../jeb-analysis.md) | +| Android-Ressourcen | Dekodiertes Manifest, String-Tabellen, Sprachen und Versionsfakten | Vom Aufrufer bereitgestellter Apktool-Starter; [Ressourcen-Leitfaden](../apktool-resource-analysis.md) | +| Flutter-Apps | Dart-Snapshot-Hash, Per-ABI-Bibliotheksprüfsummen und Engine-Build-Provenienz | Keine (statische APK-Analyse); [Flutter-Leitfaden](../flutter-analysis.md) | +| Firmware | Bereiche, Extraktionsergebnisse und Übergaben an die native Analyse | Binwalk / Unblob unter Linux; [Firmware-Leitfaden](../firmware-analysis.md) | +| Aufgezeichnete Pakete | Ausgewählte Felder, wiederholte Werte, Filter und Decode-as-Regeln | Linux: TShark 4.x + prlimit; [Mitschnitt-Anleitung](../packet-captures.md) | +| Aufgezeichnete Logiksignale | Decoder-Anmerkungen, Sample-Bereiche und Diagnosen | Linux: sigrok-cli 0.7.2 + prlimit; [Signalaufzeichnungen](../signal-recordings.md) | +| Pakete und Ressourcen | Dateiinventare, Hashwerte, plists, Aufbau von Apple-Bundles und extrahierte Ressourcen | [Artefakt- und JavaScript-Leitfaden](../javascript-artifact-reconstruction.md), [Apple-Anwendungen](../apple-application-analysis.md) | +| Prozessverhalten | Terminalausgabe, Interaktionen, Beobachtungen zu Prozessende und Dateisystem sowie Laufvergleiche | Linux/macOS mit nativer PTY; [Prozessaufzeichnung](../process-capture.md) | Statische JavaScript- und .NET-Untersuchungen lesen die bereitgestellten Dateien, ohne die Anwendung auszuführen. Laufzeitaufzeichnungen führen das gewählte Ziel mit deinen Benutzerrechten aus oder interagieren damit; die jeweiligen Anleitungen beschreiben die Auswirkungen. -Native Formate und unterstützte Hostsysteme unterscheiden sich je nach Provider. Siehe [Hopper- und Ghidra-Einrichtung](docs/installation.md#hopper), den [IDA-Leitfaden](docs/ida-provider.md) und die [experimentelle Ghidra-Unterstützung unter Windows](docs/windows-ghidra-p0.md). Ghidra unterstützt auch [16-Bit-DOS-Analyse](docs/ghidra-dos.md). Die Provider-Auswahl beschreibt der [CLI-Leitfaden](docs/cli.md#choose-a-provider). Prüfe die [Verfügbarkeit in Releases](docs/installation.md#released-package-and-main) für Funktionen, die seit der neuesten npm-Version hinzugekommen sind. +Native Formate und unterstützte Hostsysteme unterscheiden sich je nach Provider. Siehe [Hopper- und Ghidra-Einrichtung](../installation.md#hopper), den [IDA-Leitfaden](../ida-provider.md) und die [experimentelle Ghidra-Unterstützung unter Windows](../windows-ghidra-p0.md). Ghidra unterstützt auch [16-Bit-DOS-Analyse](../ghidra-dos.md). Die Provider-Auswahl beschreibt der [CLI-Leitfaden](../cli.md#choose-a-provider). Prüfe die [Verfügbarkeit in Releases](../installation.md#released-package-and-main) für Funktionen, die seit der neuesten npm-Version hinzugekommen sind. ## Fallstudien -[![Illustrationen der Fallstudien zu DX-Balls Klangbalance, Notions Zwischenablage-Brücke und TH04s Geschossring](docs/assets/rea-showcases.png)](https://rea.tools/de/showcase/) +[![Illustrationen der Fallstudien zu DX-Balls Klangbalance, Notions Zwischenablage-Brücke und TH04s Geschossring](../assets/rea-showcases.png)](https://rea.tools/de/showcase/) ### DX-Ball: Eine Stereo-Panning-Berechnung rekonstruieren @@ -187,28 +187,28 @@ Wenn du mit REA etwas Interessantes untersucht hast, würden wir es gern sehen.
Welche Agenten können REA verwenden? -Jeder Agent, der lokale MCP-Server unterstützt. Die Einrichtung konfiguriert die [unterstützten Agenten](docs/installation.md#supported-agents); andere Clients können die [manuelle MCP-Registrierung](docs/installation.md#mcp-registry) nutzen. +Jeder Agent, der lokale MCP-Server unterstützt. Die Einrichtung konfiguriert die [unterstützten Agenten](../installation.md#supported-agents); andere Clients können die [manuelle MCP-Registrierung](../installation.md#mcp-registry) nutzen.
Brauche ich Hopper, Ghidra oder IDA? -Für tiefgehende native Analysen wird eines dieser Programme verwendet. Statische JavaScript- und .NET-Untersuchungen funktionieren ohne native Analyse-Engine. Die Einrichtung kann Hopper nach Zustimmung installieren; Ghidra und IDA nutzen deine vorhandenen Installationen. Siehe [Provider-Einrichtung](docs/installation.md#hopper). +Für tiefgehende native Analysen wird eines dieser Programme verwendet. Statische JavaScript- und .NET-Untersuchungen funktionieren ohne native Analyse-Engine. Die Einrichtung kann Hopper nach Zustimmung installieren; Ghidra und IDA nutzen deine vorhandenen Installationen. Siehe [Provider-Einrichtung](../installation.md#hopper).
Muss ich Hopper vorher starten? -REA startet Hopper, wenn eine Operation es benötigt. Unter macOS kann beim ersten Start ein Dialog erscheinen, in dem du den Demomodus auswählst oder deine Lizenz aktivierst. Siehe [Hopper-Start und Fehlerbehebung](docs/installation.md#launcher-paths-and-troubleshooting). +REA startet Hopper, wenn eine Operation es benötigt. Unter macOS kann beim ersten Start ein Dialog erscheinen, in dem du den Demomodus auswählst oder deine Lizenz aktivierst. Siehe [Hopper-Start und Fehlerbehebung](../installation.md#launcher-paths-and-troubleshooting).
Was bewirkt die Installation des Skills über skills.sh? -Der Skill stellt Untersuchungsanweisungen für deinen Agenten bereit. Verwende `npx rea-agents setup`, um den MCP-Server von REA zu registrieren und passende Anweisungen zu installieren. Starte deinen Agenten anschließend neu. Siehe [Installation nur des Skills](docs/installation.md#skill-only-installation). +Der Skill stellt Untersuchungsanweisungen für deinen Agenten bereit. Verwende `npx rea-agents setup`, um den MCP-Server von REA zu registrieren und passende Anweisungen zu installieren. Starte deinen Agenten anschließend neu. Siehe [Installation nur des Skills](../installation.md#skill-only-installation).
@@ -251,14 +251,14 @@ Wenn du einen Agenten verwendest, schließe die [Aktualisierung der Einrichtung] Beginne mit den [praxisnahen Anleitungen](https://rea.tools/de/guides/) auf der Website. Genaue Optionen, Voraussetzungen und Ergebniskontrakte findest du hier: -- [Installation und Einrichtung](docs/installation.md): Agentenregistrierung, Provider-Konfiguration, Updates und Deinstallation. -- [Bereitschaft und Fehlerbehebung](docs/installation.md#check-readiness-for-your-task): Diagnose eines bestimmten Agenten oder einer Analyse-Engine. -- [CLI und Evidence](docs/cli.md): Befehle, Provider-Auswahl, Snapshots, Import/Export und Exit-Status. -- [MCP-Kontrakte](docs/mcp-contracts.md) und [Agenten-Prompts](docs/mcp-prompts.md): Werkzeugergebnisse, Sitzungen und geführte Untersuchungen. -- [Werkzeugkatalog](docs/mcp-contracts.md#generated-catalog): Beim Build erzeugtes Inventar von Werkzeugen, Providern und CLI-Befehlen. -- [Roadmap](docs/roadmap.md): Geplante Arbeiten und Fortschrittsübersichten zu Fähigkeiten. +- [Installation und Einrichtung](../installation.md): Agentenregistrierung, Provider-Konfiguration, Updates und Deinstallation. +- [Bereitschaft und Fehlerbehebung](../installation.md#check-readiness-for-your-task): Diagnose eines bestimmten Agenten oder einer Analyse-Engine. +- [CLI und Evidence](../cli.md): Befehle, Provider-Auswahl, Snapshots, Import/Export und Exit-Status. +- [MCP-Kontrakte](../mcp-contracts.md) und [Agenten-Prompts](../mcp-prompts.md): Werkzeugergebnisse, Sitzungen und geführte Untersuchungen. +- [Werkzeugkatalog](../mcp-contracts.md#generated-catalog): Beim Build erzeugtes Inventar von Werkzeugen, Providern und CLI-Befehlen. +- [Roadmap](../roadmap.md): Geplante Arbeiten und Fortschrittsübersichten zu Fähigkeiten. -Melde Sicherheitslücken gemäß [SECURITY.md](SECURITY.md). +Melde Sicherheitslücken gemäß [SECURITY.md](../../SECURITY.md). ## Sternverlauf @@ -276,7 +276,7 @@ Danke an alle, die REA nutzen, Fehler melden, Funktionswünsche äußern, Builds ## Haftungsausschluss -Hinweise zur verantwortungsvollen Nutzung findest du in [DISCLAIMER.md](DISCLAIMER.md). +Hinweise zur verantwortungsvollen Nutzung findest du in [DISCLAIMER.md](../../DISCLAIMER.md). REA dient der rechtmäßigen Forschung, Bildung, Interoperabilität, Softwareerhaltung und Analyse von Software, die dir gehört oder die du mit entsprechender Genehmigung untersuchen darfst. @@ -288,14 +288,14 @@ applicable licenses. Wir freuen uns über deine Hilfe bei REA! [Öffne ein Issue](https://github.com/morluto/rea/issues), um einen Fehler zu melden oder eine Funktion vorzuschlagen, oder [sende einen Pull Request](https://github.com/morluto/rea/pulls), um Code oder Dokumentation zu verbessern. -Die Entwicklungsumgebung und Prüfungen beschreibt [CONTRIBUTING.md](CONTRIBUTING.md), die Verifikationspfade der [Testleitfaden](docs/testing.md) und die Projektstruktur die [Architekturübersicht](docs/architecture.mermaid). +Die Entwicklungsumgebung und Prüfungen beschreibt [CONTRIBUTING.md](../../CONTRIBUTING.md), die Verifikationspfade der [Testleitfaden](../testing.md) und die Projektstruktur die [Architekturübersicht](../architecture.mermaid). ## Lizenz -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_es.md b/docs/readme/README_es.md similarity index 75% rename from README_es.md rename to docs/readme/README_es.md index f955b5891..d09be25e2 100644 --- a/README_es.md +++ b/docs/readme/README_es.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · **Español** · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · **Español** · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA: Ingeniería inversa de cualquier cosa @@ -9,10 +9,10 @@ **¿Ves una función que te gusta? Entiende cómo funciona, hasta el nivel binario.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA inicia su puente de análisis dentro de Hopper mientras inspecciona un binario nativo +REA inicia su puente de análisis dentro de Hopper mientras inspecciona un binario nativo

@@ -33,7 +33,7 @@ - Discord
+ Discord
Únete a la comunidad de ingeniería inversa
Discord · Preguntas y respuestas · Comparte tus resultados @@ -67,7 +67,7 @@ npx rea-agents setup Elige tus agentes, revisa los cambios propuestos y apruébalos. La configuración añade el servidor MCP de REA y las instrucciones de trabajo correspondientes, con copias de seguridad de la configuración existente. Reinicia tu agente al terminar. -La configuración admite Claude Code, Codex, Cursor, Gemini CLI, Grok Build y [otros agentes](docs/installation.md#supported-agents). Consulta [instalación y configuración](docs/installation.md) para configurar proveedores y registrar MCP manualmente. +La configuración admite Claude Code, Codex, Cursor, Gemini CLI, Grok Build y [otros agentes](../installation.md#supported-agents). Consulta [instalación y configuración](../installation.md) para configurar proveedores y registrar MCP manualmente. ### Pregunta a tu agente @@ -94,7 +94,7 @@ npm install --global rea-agents rea --help ``` -Para el análisis nativo, configura primero un proveedor. Consulta la [guía de CLI y Evidence](docs/cli.md) para los comandos nativos, la selección de proveedores, las instantáneas y el uso en scripts. +Para el análisis nativo, configura primero un proveedor. Consulta la [guía de CLI y Evidence](../cli.md) para los comandos nativos, la selección de proveedores, las instantáneas y el uso en scripts. ### Actualiza REA @@ -120,9 +120,9 @@ Revisa los cambios de configuración y reinicia tu agente. Para ejecutar comando Tu agente llama a REA mediante MCP para inspeccionar el objetivo y seguir el código relevante. REA devuelve los hallazgos junto con sus pruebas. El agente los utiliza para hacer preguntas de seguimiento, explicar el comportamiento o escribir y probar una implementación. Los comandos de CLI usan los mismos flujos de trabajo. -![Flujo de investigación de REA: tu agente pregunta sobre un objetivo local, REA lo inspecciona y rastrea con herramientas de análisis, y el agente utiliza el código, las referencias y los aspectos desconocidos devueltos para explicar, implementar y probar.](website/public/assets/figures/rea-investigation-flow.svg) +![Flujo de investigación de REA: tu agente pregunta sobre un objetivo local, REA lo inspecciona y rastrea con herramientas de análisis, y el agente utiliza el código, las referencias y los aspectos desconocidos devueltos para explicar, implementar y probar.](../../website/public/assets/figures/rea-investigation-flow.svg) -[Abre la figura a tamaño completo](website/public/assets/figures/rea-investigation-flow.svg). +[Abre la figura a tamaño completo](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -130,37 +130,37 @@ Tu agente llama a REA mediante MCP para inspeccionar el objetivo y seguir el có REA requiere Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, además de npm. Las herramientas adicionales y los sistemas anfitriones compatibles dependen del objetivo: -| Objetivo | Qué devuelve REA | Requisitos y guía | -| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ | -| Binarios nativos | Pseudocódigo, ensamblador, cadenas, símbolos, llamadas y referencias | Hopper, Ghidra o IDA; [análisis nativo](https://rea.tools/es/guides/native/) | -| Estructura ELF sin ejecutar | Secciones, segmentos, símbolos/reubicaciones originales y posibles medidas de mitigación detectadas mediante análisis estático | pwntools proporcionado por quien llama, en Linux x64; [diagnóstico de binarios](docs/binary-diagnostics.md) | -| Bytecode EVM | Selectores de despacho, desplazamientos en bytes, argumentos inferidos y mutabilidad | Entrada local con bytes sin procesar o representación hexadecimal, en Linux x64; [guía de EVM sin ejecutar](docs/evm-bytecode.md) | -| Fallos de Linux registrados | Registros note sin procesar, registros/señales de cada hilo registrado y posibles correspondencias de memoria opcionales | pwntools proporcionado por quien llama, en Linux x64; GDB/pwndbg opcionales; [fallos registrados](docs/recorded-crashes.md) | -| JavaScript / Electron | Módulos, importaciones, mapas de código fuente, rutas, IPC y relaciones con complementos nativos | Node.js y npm; [análisis de aplicaciones](https://rea.tools/es/guides/javascript/) | -| Sitios web | Estructura de la página, scripts, observaciones de red y capturas de pantalla solicitadas | Un navegador de la familia Chrome; [análisis del navegador](https://rea.tools/es/guides/browser/) | -| Capturas de red guardadas | Solicitudes, respuestas, contenido accesible de las cargas útiles y ubicaciones de origen | HAR; mitmdump en Linux para capturas en formato nativo de mitmproxy; [guía de capturas](docs/web-network-captures.md) | -| Capturas de paquetes | Endpoints, DNS, TLS SNI, distribución de protocolos y solicitudes HTTP | tshark y capinfos suministrados por el autor de la llamada; [guía pcap](docs/pcap-analysis.md) | -| Ensamblados .NET | Metadatos, instrucciones CIL, dependencias nativas declaradas y comparaciones de compilaciones | Inspección estática; [guía de código administrado](docs/managed-code-analysis.md) | -| APK de Android | Declaraciones del manifiesto, clases, métodos descompilados y referencias | JADX sin interfaz gráfica y un JDK completo en Linux/macOS/Windows x64; [guía de Android](docs/android-analysis.md) | -| Dispositivos Android | Dispositivos, paquetes, procesos, registros, volcados, capturas de pantalla y transferencias de APK/archivos | adb suministrado por el autor de la llamada (emulador o dispositivo); [guía de ADB](docs/adb-device-analysis.md) | -| Proyectos JEB | Unidades de proyecto del motor, resúmenes de artefactos y pseudocódigo de tipos/métodos | Un cliente JEB iniciado por el autor de la llamada que sirve MCP, seleccionado con REA_JEB_MCP_URL; [guía de JEB](docs/jeb-analysis.md) | -| Recursos de Android | Manifiesto decodificado, tablas de cadenas, idiomas y datos de versión | Lanzador de Apktool suministrado por el autor de la llamada; [guía de recursos](docs/apktool-resource-analysis.md) | -| Aplicaciones Flutter | Hash de snapshot de Dart, resúmenes de bibliotecas por ABI y procedencia de compilación del motor | Ninguna (análisis APK estático); [guía de Flutter](docs/flutter-analysis.md) | -| Firmware | Regiones, resultados de extracción y derivaciones al análisis nativo | Binwalk / Unblob en Linux; [guía de firmware](docs/firmware-analysis.md) | -| Paquetes grabados | Campos seleccionados, valores repetidos, filtros y reglas de decodificación | Linux: TShark 4.x + prlimit; [guía de capturas](docs/packet-captures.md) | -| Señales lógicas grabadas | Anotaciones del decodificador, rangos de muestras y diagnósticos | Linux: sigrok-cli 0.7.2 + prlimit; [guía de señales](docs/signal-recordings.md) | -| Paquetes y recursos | Inventarios de archivos, resúmenes criptográficos, plists, estructura de paquetes de Apple y recursos extraídos | [Guía de artefactos y JavaScript](docs/javascript-artifact-reconstruction.md), [aplicaciones de Apple](docs/apple-application-analysis.md) | -| Comportamiento de procesos | Salida de la terminal, interacciones, observaciones de salida y del sistema de archivos, y comparaciones entre ejecuciones | Linux/macOS con una PTY nativa; [captura de procesos](docs/process-capture.md) | +| Objetivo | Qué devuelve REA | Requisitos y guía | +| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- | +| Binarios nativos | Pseudocódigo, ensamblador, cadenas, símbolos, llamadas y referencias | Hopper, Ghidra o IDA; [análisis nativo](https://rea.tools/es/guides/native/) | +| Estructura ELF sin ejecutar | Secciones, segmentos, símbolos/reubicaciones originales y posibles medidas de mitigación detectadas mediante análisis estático | pwntools proporcionado por quien llama, en Linux x64; [diagnóstico de binarios](../binary-diagnostics.md) | +| Bytecode EVM | Selectores de despacho, desplazamientos en bytes, argumentos inferidos y mutabilidad | Entrada local con bytes sin procesar o representación hexadecimal, en Linux x64; [guía de EVM sin ejecutar](../evm-bytecode.md) | +| Fallos de Linux registrados | Registros note sin procesar, registros/señales de cada hilo registrado y posibles correspondencias de memoria opcionales | pwntools proporcionado por quien llama, en Linux x64; GDB/pwndbg opcionales; [fallos registrados](../recorded-crashes.md) | +| JavaScript / Electron | Módulos, importaciones, mapas de código fuente, rutas, IPC y relaciones con complementos nativos | Node.js y npm; [análisis de aplicaciones](https://rea.tools/es/guides/javascript/) | +| Sitios web | Estructura de la página, scripts, observaciones de red y capturas de pantalla solicitadas | Un navegador de la familia Chrome; [análisis del navegador](https://rea.tools/es/guides/browser/) | +| Capturas de red guardadas | Solicitudes, respuestas, contenido accesible de las cargas útiles y ubicaciones de origen | HAR; mitmdump en Linux para capturas en formato nativo de mitmproxy; [guía de capturas](../web-network-captures.md) | +| Capturas de paquetes | Endpoints, DNS, TLS SNI, distribución de protocolos y solicitudes HTTP | tshark y capinfos suministrados por el autor de la llamada; [guía pcap](../pcap-analysis.md) | +| Ensamblados .NET | Metadatos, instrucciones CIL, dependencias nativas declaradas y comparaciones de compilaciones | Inspección estática; [guía de código administrado](../managed-code-analysis.md) | +| APK de Android | Declaraciones del manifiesto, clases, métodos descompilados y referencias | JADX sin interfaz gráfica y un JDK completo en Linux/macOS/Windows x64; [guía de Android](../android-analysis.md) | +| Dispositivos Android | Dispositivos, paquetes, procesos, registros, volcados, capturas de pantalla y transferencias de APK/archivos | adb suministrado por el autor de la llamada (emulador o dispositivo); [guía de ADB](../adb-device-analysis.md) | +| Proyectos JEB | Unidades de proyecto del motor, resúmenes de artefactos y pseudocódigo de tipos/métodos | Un cliente JEB iniciado por el autor de la llamada que sirve MCP, seleccionado con REA_JEB_MCP_URL; [guía de JEB](../jeb-analysis.md) | +| Recursos de Android | Manifiesto decodificado, tablas de cadenas, idiomas y datos de versión | Lanzador de Apktool suministrado por el autor de la llamada; [guía de recursos](../apktool-resource-analysis.md) | +| Aplicaciones Flutter | Hash de snapshot de Dart, resúmenes de bibliotecas por ABI y procedencia de compilación del motor | Ninguna (análisis APK estático); [guía de Flutter](../flutter-analysis.md) | +| Firmware | Regiones, resultados de extracción y derivaciones al análisis nativo | Binwalk / Unblob en Linux; [guía de firmware](../firmware-analysis.md) | +| Paquetes grabados | Campos seleccionados, valores repetidos, filtros y reglas de decodificación | Linux: TShark 4.x + prlimit; [guía de capturas](../packet-captures.md) | +| Señales lógicas grabadas | Anotaciones del decodificador, rangos de muestras y diagnósticos | Linux: sigrok-cli 0.7.2 + prlimit; [guía de señales](../signal-recordings.md) | +| Paquetes y recursos | Inventarios de archivos, resúmenes criptográficos, plists, estructura de paquetes de Apple y recursos extraídos | [Guía de artefactos y JavaScript](../javascript-artifact-reconstruction.md), [aplicaciones de Apple](../apple-application-analysis.md) | +| Comportamiento de procesos | Salida de la terminal, interacciones, observaciones de salida y del sistema de archivos, y comparaciones entre ejecuciones | Linux/macOS con una PTY nativa; [captura de procesos](../process-capture.md) | La inspección estática de JavaScript y .NET lee los archivos suministrados sin ejecutar la aplicación. La captura en ejecución inicia el objetivo seleccionado o interactúa con él con los permisos de tu usuario; cada guía describe sus efectos. -Los formatos nativos y los sistemas anfitriones compatibles varían según el proveedor. Consulta [configuración de Hopper y Ghidra](docs/installation.md#hopper), la [guía de IDA](docs/ida-provider.md) y la [compatibilidad experimental de Ghidra en Windows](docs/windows-ghidra-p0.md). Ghidra también admite [análisis de DOS de 16 bits](docs/ghidra-dos.md). Para elegir un proveedor, consulta la [guía de CLI](docs/cli.md#choose-a-provider). Comprueba la [disponibilidad por versión](docs/installation.md#released-package-and-main) de las funciones añadidas después de la última versión publicada en npm. +Los formatos nativos y los sistemas anfitriones compatibles varían según el proveedor. Consulta [configuración de Hopper y Ghidra](../installation.md#hopper), la [guía de IDA](../ida-provider.md) y la [compatibilidad experimental de Ghidra en Windows](../windows-ghidra-p0.md). Ghidra también admite [análisis de DOS de 16 bits](../ghidra-dos.md). Para elegir un proveedor, consulta la [guía de CLI](../cli.md#choose-a-provider). Comprueba la [disponibilidad por versión](../installation.md#released-package-and-main) de las funciones añadidas después de la última versión publicada en npm. ## Casos prácticos -[![Ilustraciones de los casos de panoramización de sonido de DX-Ball, el puente del portapapeles de Notion y el anillo de proyectiles de TH04](docs/assets/rea-showcases.png)](https://rea.tools/es/showcase/) +[![Ilustraciones de los casos de panoramización de sonido de DX-Ball, el puente del portapapeles de Notion y el anillo de proyectiles de TH04](../assets/rea-showcases.png)](https://rea.tools/es/showcase/) ### DX-Ball: reconstruir un cálculo de panoramización de sonido @@ -187,28 +187,28 @@ Si has usado REA para investigar algo interesante, nos gustaría verlo. Comparte
¿Qué agentes pueden usar REA? -Cualquier agente que admita servidores MCP locales. La configuración prepara los [agentes compatibles](docs/installation.md#supported-agents); otros clientes pueden usar el [registro manual de MCP](docs/installation.md#mcp-registry). +Cualquier agente que admita servidores MCP locales. La configuración prepara los [agentes compatibles](../installation.md#supported-agents); otros clientes pueden usar el [registro manual de MCP](../installation.md#mcp-registry).
¿Necesito Hopper, Ghidra o IDA? -El análisis nativo en profundidad utiliza uno de ellos. La inspección estática de JavaScript y .NET funciona sin un motor de análisis nativo. La configuración puede instalar Hopper después de tu aprobación; Ghidra e IDA usan las instalaciones que ya tienes. Consulta [configuración de proveedores](docs/installation.md#hopper). +El análisis nativo en profundidad utiliza uno de ellos. La inspección estática de JavaScript y .NET funciona sin un motor de análisis nativo. La configuración puede instalar Hopper después de tu aprobación; Ghidra e IDA usan las instalaciones que ya tienes. Consulta [configuración de proveedores](../installation.md#hopper).
¿Tengo que iniciar Hopper primero? -REA inicia Hopper cuando una operación lo necesita. En macOS, puede aparecer un diálogo al iniciarlo por primera vez para elegir el modo de demostración o activar tu licencia. Consulta [inicio de Hopper y resolución de problemas](docs/installation.md#launcher-paths-and-troubleshooting). +REA inicia Hopper cuando una operación lo necesita. En macOS, puede aparecer un diálogo al iniciarlo por primera vez para elegir el modo de demostración o activar tu licencia. Consulta [inicio de Hopper y resolución de problemas](../installation.md#launcher-paths-and-troubleshooting).
¿Qué hace la instalación de la skill desde skills.sh? -La skill proporciona instrucciones de investigación a tu agente. Usa `npx rea-agents setup` para registrar el servidor MCP de REA e instalar las instrucciones correspondientes; después reinicia tu agente. Consulta [instalación solo de la skill](docs/installation.md#skill-only-installation). +La skill proporciona instrucciones de investigación a tu agente. Usa `npx rea-agents setup` para registrar el servidor MCP de REA e instalar las instrucciones correspondientes; después reinicia tu agente. Consulta [instalación solo de la skill](../installation.md#skill-only-installation).
@@ -251,14 +251,14 @@ Si usas un agente, completa la [actualización de la configuración](#actualiza- Empieza con las [guías prácticas](https://rea.tools/es/guides/) del sitio web. Para opciones exactas, requisitos y contratos de resultados: -- [Instalación y configuración](docs/installation.md): registro de agentes, configuración de proveedores, actualizaciones y desinstalación. -- [Preparación y resolución de problemas](docs/installation.md#check-readiness-for-your-task): diagnóstico de un agente o motor de análisis concreto. -- [CLI y Evidence](docs/cli.md): comandos, selección de proveedores, instantáneas, importación/exportación y estados de salida. -- [Contratos de MCP](docs/mcp-contracts.md) y [prompts para agentes](docs/mcp-prompts.md): resultados de herramientas, sesiones e investigaciones guiadas. -- [Catálogo de herramientas](docs/mcp-contracts.md#generated-catalog): inventario de herramientas, proveedores y comandos de CLI generado durante la compilación. -- [Hoja de ruta](docs/roadmap.md): trabajo previsto y seguimiento de capacidades. +- [Instalación y configuración](../installation.md): registro de agentes, configuración de proveedores, actualizaciones y desinstalación. +- [Preparación y resolución de problemas](../installation.md#check-readiness-for-your-task): diagnóstico de un agente o motor de análisis concreto. +- [CLI y Evidence](../cli.md): comandos, selección de proveedores, instantáneas, importación/exportación y estados de salida. +- [Contratos de MCP](../mcp-contracts.md) y [prompts para agentes](../mcp-prompts.md): resultados de herramientas, sesiones e investigaciones guiadas. +- [Catálogo de herramientas](../mcp-contracts.md#generated-catalog): inventario de herramientas, proveedores y comandos de CLI generado durante la compilación. +- [Hoja de ruta](../roadmap.md): trabajo previsto y seguimiento de capacidades. -Informa de vulnerabilidades siguiendo [SECURITY.md](SECURITY.md). +Informa de vulnerabilidades siguiendo [SECURITY.md](../../SECURITY.md). ## Historial de estrellas @@ -276,7 +276,7 @@ Gracias a quienes usan REA, informan de errores, solicitan funciones, prueban co ## Aviso legal -Consulta [DISCLAIMER.md](DISCLAIMER.md) para obtener orientación sobre el uso responsable. +Consulta [DISCLAIMER.md](../../DISCLAIMER.md) para obtener orientación sobre el uso responsable. REA está destinado a la investigación, la educación, la interoperabilidad, la preservación de software y el análisis de software propio o autorizado, dentro de la ley. @@ -288,14 +288,14 @@ applicable licenses. ¡Nos encantaría que ayudaras con REA! [Abre una issue](https://github.com/morluto/rea/issues) para comunicar un error o proponer una función, o [envía una pull request](https://github.com/morluto/rea/pulls) para mejorar el código o la documentación. -Consulta [CONTRIBUTING.md](CONTRIBUTING.md) para preparar el entorno de desarrollo y las comprobaciones, la [guía de pruebas](docs/testing.md) para los procesos de verificación y el [mapa de arquitectura](docs/architecture.mermaid) para la estructura del proyecto. +Consulta [CONTRIBUTING.md](../../CONTRIBUTING.md) para preparar el entorno de desarrollo y las comprobaciones, la [guía de pruebas](../testing.md) para los procesos de verificación y el [mapa de arquitectura](../architecture.mermaid) para la estructura del proyecto. ## Licencia -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_fa.md b/docs/readme/README_fa.md similarity index 74% rename from README_fa.md rename to docs/readme/README_fa.md index 54af87c57..b24e4380c 100644 --- a/README_fa.md +++ b/docs/readme/README_fa.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · **فارسی** · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · **فارسی** · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA: مهندسی معکوس هر چیزی @@ -9,27 +9,27 @@ **قابلیتی دیده‌اید که دوستش دارید؟ سازوکارش را تا سطح باینری درک کنید.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift **[وب‌سایت](https://rea.tools/fa/) · [راهنماها](https://rea.tools/fa/guides/) · [نمونه‌های عملی](https://rea.tools/fa/showcase/)** -[شروع سریع](#شروع-سریع) · [REA چگونه کار می‌کند؟](#rea-چگونه-کار-میکند) · [چه چیزهایی را میتوان تحلیل کرد؟](#چه-چیزهایی-را-میتوان-تحلیل-کرد) · [نمونه‌های عملی](#نمونههای-عملی) · [پرسش‌های متداول](#پرسشهای-متداول) · [مستندات](#مستندات) +[شروع سریع](#شروع-سریع) · [REA چگونه کار می‌کند؟](#rea-چگونه-کار-می‌کند) · [چه چیزهایی را میتوان تحلیل کرد؟](#چه-چیزهایی-را-می‌توان-تحلیل-کرد) · [نمونه‌های عملی](#نمونه‌های-عملی) · [پرسش‌های متداول](#پرسش‌های-متداول) · [مستندات](#مستندات) npx rea-agents setup
-اجرای پل تحلیل REA در Hopper هنگام بررسی یک فایل باینری بومی +اجرای پل تحلیل REA در Hopper هنگام بررسی یک فایل باینری بومی

-
Discord
به انجمن مهندسی معکوس بپیوندید
دیسکورد · پرسش و پاسخ · نمایش دستاوردها
+
Discord
به انجمن مهندسی معکوس بپیوندید
دیسکورد · پرسش و پاسخ · نمایش دستاوردها

@@ -59,7 +59,7 @@ npx rea-agents setup ایجنت‌های موردنظر را انتخاب کنید، تغییرات پیشنهادی را بررسی و تأیید کنید. فرایند راه‌اندازی، سرور MCP مربوط به REA و دستورالعمل‌های گردش‌کار متناظر را اضافه می‌کند و از پیکربندی موجود نسخهٔ پشتیبان می‌گیرد. پس از آن ایجنت خود را مجدداً راه‌اندازی کنید. -راه‌اندازی از Claude Code، Codex، Cursor، Gemini CLI، Grok Build و [ایجنت‌های دیگر](docs/installation.md#supported-agents) پشتیبانی می‌کند. برای پیکربندی ارائه‌دهنده و ثبت دستی MCP، [نصب و راه‌اندازی](docs/installation.md) را ببینید. +راه‌اندازی از Claude Code، Codex، Cursor، Gemini CLI، Grok Build و [ایجنت‌های دیگر](../installation.md#supported-agents) پشتیبانی می‌کند. برای پیکربندی ارائه‌دهنده و ثبت دستی MCP، [نصب و راه‌اندازی](../installation.md) را ببینید. ### از ایجنت خود بپرسید @@ -86,7 +86,7 @@ npm install --global rea-agents rea --help ``` -برای تحلیل بومی ابتدا یک ارائه‌دهنده پیکربندی کنید. برای دستورات بومی، انتخاب ارائه‌دهنده، snapshotها و اسکریپت‌نویسی، [راهنمای CLI و شواهد](docs/cli.md) را ببینید. +برای تحلیل بومی ابتدا یک ارائه‌دهنده پیکربندی کنید. برای دستورات بومی، انتخاب ارائه‌دهنده، snapshotها و اسکریپت‌نویسی، [راهنمای CLI و شواهد](../cli.md) را ببینید. ### به‌روزرسانی REA @@ -112,9 +112,9 @@ npx rea-agents@latest setup ایجنت شما از طریق MCP، REA را فراخوانی می‌کند تا هدف را بررسی و کد مرتبط را ردیابی کند. REA یافته‌ها را همراه با شواهد برمی‌گرداند. ایجنت از این نتایج برای طرح پرسش‌های تکمیلی، توضیح رفتار یا نوشتن و آزمایش یک پیاده‌سازی استفاده می‌کند. دستورات CLI نیز از همین گردش‌کارها بهره می‌برند. -![جریان بررسی REA: ایجنت دربارهٔ هدف محلی سؤال می‌پرسد، REA آن را با ابزارهای تحلیل بررسی و ردیابی می‌کند و ایجنت از کد، ارجاعات و ناشناخته‌های بازگردانده‌شده برای توضیح، پیاده‌سازی و آزمایش استفاده می‌کند.](website/public/assets/figures/rea-investigation-flow.svg) +![جریان بررسی REA: ایجنت دربارهٔ هدف محلی سؤال می‌پرسد، REA آن را با ابزارهای تحلیل بررسی و ردیابی می‌کند و ایجنت از کد، ارجاعات و ناشناخته‌های بازگردانده‌شده برای توضیح، پیاده‌سازی و آزمایش استفاده می‌کند.](../../website/public/assets/figures/rea-investigation-flow.svg) -[نمایش تصویر در اندازهٔ کامل](website/public/assets/figures/rea-investigation-flow.svg). +[نمایش تصویر در اندازهٔ کامل](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -122,37 +122,37 @@ npx rea-agents@latest setup REA به Node.js 22.x (>=22.19)، 24.x (>=24.11) یا 26+، به‌همراه npm نیاز دارد. ابزارهای اضافی و پشتیبانی سیستم میزبان به نوع هدف بستگی دارند: -| هدف | خروجی REA | پیش‌نیازها و راهنما | -| ---------------------- | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | -| باینری‌های بومی | شبه‌کد، اسمبلی، رشته‌ها، نمادها، فراخوانی‌ها و ارجاعات | Hopper، Ghidra یا IDA؛ [تحلیل بومی](https://rea.tools/fa/guides/native/) | -| ساختار ELF آفلاین | بخش‌ها، سگمنت‌ها، نمادها و relocationهای اصلی و گزینه‌های احتمالی کاهش آسیب‌پذیری ایستا | pwntools تأمین‌شده توسط فراخواننده روی Linux x64؛ [تشخیص باینری](docs/binary-diagnostics.md) | -| بایت‌کد EVM | selectorهای dispatch، offsetهای بایتی، آرگومان‌های استنباط‌شده و قابلیت تغییر وضعیت | دادهٔ خام/hex محلی روی Linux x64؛ [راهنمای EVM آفلاین](docs/evm-bytecode.md) | -| کرش‌های ثبت‌شدهٔ لینوکس | یادداشت‌های خام، رجیسترها/سیگنال‌های تمام threadهای ثبت‌شده و گزینه‌های احتمالی نگاشت | pwntools تأمین‌شده توسط فراخواننده روی Linux x64؛ GDB/pwndbg اختیاری؛ [کرش‌های ثبت‌شده](docs/recorded-crashes.md) | -| JavaScript / Electron | ماژول‌ها، importها، source mapها، مسیرها، IPC و ارتباط افزونه‌های بومی | Node.js و npm؛ [تحلیل برنامه](https://rea.tools/fa/guides/javascript/) | -| وب‌سایت‌ها | ساختار صفحه، اسکریپت‌ها، مشاهدات شبکه و اسکرین‌شات‌های درخواستی | مرورگر مبتنی بر Chrome؛ [تحلیل مرورگر](https://rea.tools/fa/guides/browser/) | -| داده‌های ضبط‌شدهٔ شبکه | درخواست‌ها، پاسخ‌ها، payloadهای آشکارشده و مکان آن‌ها در منبع | HAR؛ ابزار mitmdump در لینوکس برای ضبط‌های بومی mitmproxy؛ [راهنمای ضبط شبکه](docs/web-network-captures.md) | -| اسمبلی‌های .NET | فراداده، دستورالعمل‌های CIL، وابستگی‌های بومی اعلام‌شده و مقایسهٔ buildها | بررسی ایستا؛ [راهنمای کد مدیریت‌شده](docs/managed-code-analysis.md) | -| فایل‌های APK اندروید | تعاریف Manifest، کلاس‌ها، متدهای دیکامپایل‌شده و ارجاعات | JADX بدون رابط گرافیکی و JDK کامل در Linux/macOS/Windows x64؛ [راهنمای اندروید](docs/android-analysis.md) | -| دستگاه‌های اندروید | دستگاه‌ها، بسته‌ها، فرایندها، لاگ‌ها، دامپ‌ها، اسکرین‌شات‌ها و انتقال APK/فایل | adb تأمین‌شده توسط فراخوان‌کننده (شبیه‌ساز یا دستگاه)؛ [راهنمای ADB](docs/adb-device-analysis.md) | -| پروژه‌های JEB | واحدهای پروژه از موتور، هش‌های مصنوع و شبه‌کدِ نوع/متد | یک کلاینت JEB که توسط فراخوان‌کننده اجرا شده و MCP ارائه می‌دهد و با REA_JEB_MCP_URL انتخاب می‌شود؛ [راهنمای JEB](docs/jeb-analysis.md) | -| منابع اندروید | مانیفست رمزگشایی‌شده، جداول رشته‌ها، زبان‌ها و اطلاعات نسخه | راه‌انداز Apktool تأمین‌شده توسط فراخوان‌کننده؛ [راهنمای منابع](docs/apktool-resource-analysis.md) | -| برنامه‌های فلاتر | هش اسنپ‌شات دارت، هش کتابخانه‌ها به تفکیک ABI و اصالت بیلد موتور | هیچ (تحلیل ایستا APK)؛ [راهنمای فلاتر](docs/flutter-analysis.md) | -| Firmware | ناحیه‌ها، نتایج استخراج و تحویل به ابزارهای تحلیل بومی | Binwalk / Unblob در لینوکس؛ [راهنمای Firmware](docs/firmware-analysis.md) | -| بسته‌های ضبط‌شده | فیلدهای منتخب، مقادیر تکراری، فیلترها و قواعد رمزگشایی | Linux: TShark 4.x + prlimit; [راهنمای ضبط بسته](docs/packet-captures.md) | -| ضبط بسته‌ها | نقاط پایانی، DNS، TLS SNI، توزیع پروتکل و درخواست‌های HTTP | tshark و capinfos تأمین‌شده توسط فراخوان‌کننده؛ [راهنمای pcap](docs/pcap-analysis.md) | -| سیگنال‌های منطقی ضبط‌شده | توضیحات رمزگشا، بازه‌های نمونه و اطلاعات عیب‌یابی | Linux: sigrok-cli 0.7.2 + prlimit; [راهنمای سیگنال‌ها](docs/signal-recordings.md) | -| بسته‌ها و منابع | فهرست فایل‌ها، digestها، فایل‌های plist، ساختار bundle اپل و منابع استخراج‌شده | [راهنمای آرتیفکت و JavaScript](docs/javascript-artifact-reconstruction.md)، [برنامه‌های اپل](docs/apple-application-analysis.md) | -| رفتار فرایند | خروجی ترمینال، تعاملات، وضعیت خروج، مشاهدات سیستم فایل و مقایسهٔ اجراها | Linux/macOS با PTY بومی؛ [ضبط فرایند](docs/process-capture.md) | +| هدف | خروجی REA | پیش‌نیازها و راهنما | +| ---------------------- | ----------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| باینری‌های بومی | شبه‌کد، اسمبلی، رشته‌ها، نمادها، فراخوانی‌ها و ارجاعات | Hopper، Ghidra یا IDA؛ [تحلیل بومی](https://rea.tools/fa/guides/native/) | +| ساختار ELF آفلاین | بخش‌ها، سگمنت‌ها، نمادها و relocationهای اصلی و گزینه‌های احتمالی کاهش آسیب‌پذیری ایستا | pwntools تأمین‌شده توسط فراخواننده روی Linux x64؛ [تشخیص باینری](../binary-diagnostics.md) | +| بایت‌کد EVM | selectorهای dispatch، offsetهای بایتی، آرگومان‌های استنباط‌شده و قابلیت تغییر وضعیت | دادهٔ خام/hex محلی روی Linux x64؛ [راهنمای EVM آفلاین](../evm-bytecode.md) | +| کرش‌های ثبت‌شدهٔ لینوکس | یادداشت‌های خام، رجیسترها/سیگنال‌های تمام threadهای ثبت‌شده و گزینه‌های احتمالی نگاشت | pwntools تأمین‌شده توسط فراخواننده روی Linux x64؛ GDB/pwndbg اختیاری؛ [کرش‌های ثبت‌شده](../recorded-crashes.md) | +| JavaScript / Electron | ماژول‌ها، importها، source mapها، مسیرها، IPC و ارتباط افزونه‌های بومی | Node.js و npm؛ [تحلیل برنامه](https://rea.tools/fa/guides/javascript/) | +| وب‌سایت‌ها | ساختار صفحه، اسکریپت‌ها، مشاهدات شبکه و اسکرین‌شات‌های درخواستی | مرورگر مبتنی بر Chrome؛ [تحلیل مرورگر](https://rea.tools/fa/guides/browser/) | +| داده‌های ضبط‌شدهٔ شبکه | درخواست‌ها، پاسخ‌ها، payloadهای آشکارشده و مکان آن‌ها در منبع | HAR؛ ابزار mitmdump در لینوکس برای ضبط‌های بومی mitmproxy؛ [راهنمای ضبط شبکه](../web-network-captures.md) | +| اسمبلی‌های .NET | فراداده، دستورالعمل‌های CIL، وابستگی‌های بومی اعلام‌شده و مقایسهٔ buildها | بررسی ایستا؛ [راهنمای کد مدیریت‌شده](../managed-code-analysis.md) | +| فایل‌های APK اندروید | تعاریف Manifest، کلاس‌ها، متدهای دیکامپایل‌شده و ارجاعات | JADX بدون رابط گرافیکی و JDK کامل در Linux/macOS/Windows x64؛ [راهنمای اندروید](../android-analysis.md) | +| دستگاه‌های اندروید | دستگاه‌ها، بسته‌ها، فرایندها، لاگ‌ها، دامپ‌ها، اسکرین‌شات‌ها و انتقال APK/فایل | adb تأمین‌شده توسط فراخوان‌کننده (شبیه‌ساز یا دستگاه)؛ [راهنمای ADB](../adb-device-analysis.md) | +| پروژه‌های JEB | واحدهای پروژه از موتور، هش‌های مصنوع و شبه‌کدِ نوع/متد | یک کلاینت JEB که توسط فراخوان‌کننده اجرا شده و MCP ارائه می‌دهد و با REA_JEB_MCP_URL انتخاب می‌شود؛ [راهنمای JEB](../jeb-analysis.md) | +| منابع اندروید | مانیفست رمزگشایی‌شده، جداول رشته‌ها، زبان‌ها و اطلاعات نسخه | راه‌انداز Apktool تأمین‌شده توسط فراخوان‌کننده؛ [راهنمای منابع](../apktool-resource-analysis.md) | +| برنامه‌های فلاتر | هش اسنپ‌شات دارت، هش کتابخانه‌ها به تفکیک ABI و اصالت بیلد موتور | هیچ (تحلیل ایستا APK)؛ [راهنمای فلاتر](../flutter-analysis.md) | +| Firmware | ناحیه‌ها، نتایج استخراج و تحویل به ابزارهای تحلیل بومی | Binwalk / Unblob در لینوکس؛ [راهنمای Firmware](../firmware-analysis.md) | +| بسته‌های ضبط‌شده | فیلدهای منتخب، مقادیر تکراری، فیلترها و قواعد رمزگشایی | Linux: TShark 4.x + prlimit; [راهنمای ضبط بسته](../packet-captures.md) | +| ضبط بسته‌ها | نقاط پایانی، DNS، TLS SNI، توزیع پروتکل و درخواست‌های HTTP | tshark و capinfos تأمین‌شده توسط فراخوان‌کننده؛ [راهنمای pcap](../pcap-analysis.md) | +| سیگنال‌های منطقی ضبط‌شده | توضیحات رمزگشا، بازه‌های نمونه و اطلاعات عیب‌یابی | Linux: sigrok-cli 0.7.2 + prlimit; [راهنمای سیگنال‌ها](../signal-recordings.md) | +| بسته‌ها و منابع | فهرست فایل‌ها، digestها، فایل‌های plist، ساختار bundle اپل و منابع استخراج‌شده | [راهنمای آرتیفکت و JavaScript](../javascript-artifact-reconstruction.md)، [برنامه‌های اپل](../apple-application-analysis.md) | +| رفتار فرایند | خروجی ترمینال، تعاملات، وضعیت خروج، مشاهدات سیستم فایل و مقایسهٔ اجراها | Linux/macOS با PTY بومی؛ [ضبط فرایند](../process-capture.md) | بررسی ایستای JavaScript و .NET فایل‌های ارائه‌شده را بدون اجرای برنامه می‌خواند. ضبط زمان اجرا، هدف انتخاب‌شده را با مجوزهای کاربری شما اجرا می‌کند یا با آن تعامل دارد؛ آثار این عملیات در راهنمای مربوط به هر نوع ضبط توضیح داده شده‌اند. -فرمت‌های بومی و پشتیبانی سیستم میزبان بر اساس ارائه‌دهنده متفاوت‌اند. [راه‌اندازی Hopper و Ghidra](docs/installation.md#hopper)، [راهنمای IDA](docs/ida-provider.md) و [پشتیبانی آزمایشی Ghidra در ویندوز](docs/windows-ghidra-p0.md) را ببینید. Ghidra همچنین از [تحلیل DOS شانزده‌بیتی](docs/ghidra-dos.md) پشتیبانی می‌کند. برای انتخاب ارائه‌دهنده، به [راهنمای CLI](docs/cli.md#choose-a-provider) مراجعه کنید. برای قابلیت‌هایی که پس از آخرین انتشار npm اضافه شده‌اند، [وضعیت انتشار](docs/installation.md#released-package-and-main) را بررسی کنید. +فرمت‌های بومی و پشتیبانی سیستم میزبان بر اساس ارائه‌دهنده متفاوت‌اند. [راه‌اندازی Hopper و Ghidra](../installation.md#hopper)، [راهنمای IDA](../ida-provider.md) و [پشتیبانی آزمایشی Ghidra در ویندوز](../windows-ghidra-p0.md) را ببینید. Ghidra همچنین از [تحلیل DOS شانزده‌بیتی](../ghidra-dos.md) پشتیبانی می‌کند. برای انتخاب ارائه‌دهنده، به [راهنمای CLI](../cli.md#choose-a-provider) مراجعه کنید. برای قابلیت‌هایی که پس از آخرین انتشار npm اضافه شده‌اند، [وضعیت انتشار](../installation.md#released-package-and-main) را بررسی کنید. ## نمونه‌های عملی -[![تصویرهایی از نمونه‌های پنینگ صدای DX-Ball، پل کلیپ‌بورد Notion و حلقهٔ گلوله‌های TH04](docs/assets/rea-showcases.png)](https://rea.tools/fa/showcase/) +[![تصویرهایی از نمونه‌های پنینگ صدای DX-Ball، پل کلیپ‌بورد Notion و حلقهٔ گلوله‌های TH04](../assets/rea-showcases.png)](https://rea.tools/fa/showcase/) ### DX-Ball: بازسازی محاسبهٔ موقعیت استریوی صدا @@ -178,31 +178,31 @@ API کلیپ‌بورد در renderer را پیدا کنید، مسیر آن ر
کدام ایجنت‌ها می‌توانند از REA استفاده کنند؟ -هر ایجنتی که از سرورهای MCP محلی پشتیبانی کند. فرایند setup، [ایجنت‌های پشتیبانی‌شده](docs/installation.md#supported-agents) را پیکربندی می‌کند؛ سایر کلاینت‌ها می‌توانند از [ثبت دستی MCP](docs/installation.md#mcp-registry) استفاده کنند. +هر ایجنتی که از سرورهای MCP محلی پشتیبانی کند. فرایند setup، [ایجنت‌های پشتیبانی‌شده](../installation.md#supported-agents) را پیکربندی می‌کند؛ سایر کلاینت‌ها می‌توانند از [ثبت دستی MCP](../installation.md#mcp-registry) استفاده کنند.
آیا به Hopper، Ghidra یا IDA نیاز دارم؟ -تحلیل عمیق بومی به یکی از آن‌ها نیاز دارد. بررسی ایستای JavaScript و .NET بدون موتور تحلیل بومی کار می‌کند. setup می‌تواند با تأیید شما Hopper را نصب کند؛ Ghidra و IDA از نصب‌های موجود شما استفاده می‌کنند. [راه‌اندازی ارائه‌دهنده](docs/installation.md#hopper) را ببینید. +تحلیل عمیق بومی به یکی از آن‌ها نیاز دارد. بررسی ایستای JavaScript و .NET بدون موتور تحلیل بومی کار می‌کند. setup می‌تواند با تأیید شما Hopper را نصب کند؛ Ghidra و IDA از نصب‌های موجود شما استفاده می‌کنند. [راه‌اندازی ارائه‌دهنده](../installation.md#hopper) را ببینید.
آیا باید ابتدا Hopper را اجرا کنم؟ -REA زمانی که عملیاتی به Hopper نیاز داشته باشد آن را اجرا می‌کند. در macOS، ممکن است هنگام نخستین اجرا پنجره‌ای برای انتخاب حالت آزمایشی یا فعال‌سازی مجوز نمایش داده شود. [راه‌اندازی Hopper و رفع اشکال](docs/installation.md#launcher-paths-and-troubleshooting) را ببینید. +REA زمانی که عملیاتی به Hopper نیاز داشته باشد آن را اجرا می‌کند. در macOS، ممکن است هنگام نخستین اجرا پنجره‌ای برای انتخاب حالت آزمایشی یا فعال‌سازی مجوز نمایش داده شود. [راه‌اندازی Hopper و رفع اشکال](../installation.md#launcher-paths-and-troubleshooting) را ببینید.
نصب skill از skills.sh چه کاری انجام می‌دهد؟ -این skill دستورالعمل‌های بررسی را برای ایجنت فراهم می‌کند. برای ثبت سرور MCP مربوط به REA و نصب دستورالعمل‌های متناظر، `npx rea-agents setup` را اجرا کنید و سپس ایجنت را مجدداً راه‌اندازی کنید. [نصب مستقل skill](docs/installation.md#skill-only-installation) را ببینید. +این skill دستورالعمل‌های بررسی را برای ایجنت فراهم می‌کند. برای ثبت سرور MCP مربوط به REA و نصب دستورالعمل‌های متناظر، `npx rea-agents setup` را اجرا کنید و سپس ایجنت را مجدداً راه‌اندازی کنید. [نصب مستقل skill](../installation.md#skill-only-installation) را ببینید.
REA چه نوع کدی برمی‌گرداند؟ -تحلیل بومی، شبه‌کد و اسمبلی برمی‌گرداند. تحلیل JavaScript/Electron ماژول‌ها و روابط میان آن‌ها را بازیابی می‌کند. ایجنت شما از این یافته‌ها برای نوشتن و آزمایش یک پیاده‌سازی استفاده می‌کند؛ [نمونه‌های عملی](#نمونههای-عملی) مثال‌های گام‌به‌گام ارائه می‌دهند. +تحلیل بومی، شبه‌کد و اسمبلی برمی‌گرداند. تحلیل JavaScript/Electron ماژول‌ها و روابط میان آن‌ها را بازیابی می‌کند. ایجنت شما از این یافته‌ها برای نوشتن و آزمایش یک پیاده‌سازی استفاده می‌کند؛ [نمونه‌های عملی](#نمونه‌های-عملی) مثال‌های گام‌به‌گام ارائه می‌دهند.
@@ -228,7 +228,7 @@ rea update npx rea-agents@latest setup ``` -اگر از ایجنت استفاده می‌کنید، [به‌روزرسانی راه‌اندازی](#بهروزرسانی-rea) را تکمیل و ایجنت را مجدداً اجرا کنید. همان کار را دوباره امتحان کنید. اگر مشکل ادامه داشت، یک [issue ثبت کنید](https://github.com/morluto/rea/issues) و نسخهٔ REA، نوع هدف، مراحل بازتولید و خروجی خطا را ذکر کنید. +اگر از ایجنت استفاده می‌کنید، [به‌روزرسانی راه‌اندازی](#به‌روزرسانی-rea) را تکمیل و ایجنت را مجدداً اجرا کنید. همان کار را دوباره امتحان کنید. اگر مشکل ادامه داشت، یک [issue ثبت کنید](https://github.com/morluto/rea/issues) و نسخهٔ REA، نوع هدف، مراحل بازتولید و خروجی خطا را ذکر کنید. @@ -236,14 +236,14 @@ npx rea-agents@latest setup از [راهنماهای عملی وب‌سایت](https://rea.tools/fa/guides/) شروع کنید. برای گزینه‌های دقیق، پیش‌نیازها و قراردادهای خروجی: -- [نصب و راه‌اندازی](docs/installation.md): ثبت ایجنت، پیکربندی ارائه‌دهنده، به‌روزرسانی و حذف نصب. -- [آمادگی و رفع اشکال](docs/installation.md#check-readiness-for-your-task): عیب‌یابی یک ایجنت یا موتور تحلیل. -- [CLI و شواهد](docs/cli.md): دستورات، انتخاب ارائه‌دهنده، snapshotها، ورود/خروج داده و وضعیت‌های خروج. -- [قراردادهای MCP](docs/mcp-contracts.md) و [پرامپت‌های ایجنت](docs/mcp-prompts.md): نتایج ابزارها، نشست‌ها و بررسی‌های هدایت‌شده. -- [فهرست ابزارها](docs/mcp-contracts.md#generated-catalog): فهرست تولیدشده هنگام build از ابزارها، ارائه‌دهندگان و دستورات CLI. -- [نقشهٔ راه](docs/roadmap.md): برنامه‌های آینده و پیگیری قابلیت‌ها. +- [نصب و راه‌اندازی](../installation.md): ثبت ایجنت، پیکربندی ارائه‌دهنده، به‌روزرسانی و حذف نصب. +- [آمادگی و رفع اشکال](../installation.md#check-readiness-for-your-task): عیب‌یابی یک ایجنت یا موتور تحلیل. +- [CLI و شواهد](../cli.md): دستورات، انتخاب ارائه‌دهنده، snapshotها، ورود/خروج داده و وضعیت‌های خروج. +- [قراردادهای MCP](../mcp-contracts.md) و [پرامپت‌های ایجنت](../mcp-prompts.md): نتایج ابزارها، نشست‌ها و بررسی‌های هدایت‌شده. +- [فهرست ابزارها](../mcp-contracts.md#generated-catalog): فهرست تولیدشده هنگام build از ابزارها، ارائه‌دهندگان و دستورات CLI. +- [نقشهٔ راه](../roadmap.md): برنامه‌های آینده و پیگیری قابلیت‌ها. -آسیب‌پذیری‌ها را از طریق [SECURITY.md](SECURITY.md) گزارش کنید. +آسیب‌پذیری‌ها را از طریق [SECURITY.md](../../SECURITY.md) گزارش کنید. ## تاریخچهٔ ستاره‌ها @@ -255,7 +255,7 @@ npx rea-agents@latest setup ## سلب مسئولیت -برای راهنمای استفادهٔ مسئولانه، [DISCLAIMER.md](DISCLAIMER.md) را ببینید. +برای راهنمای استفادهٔ مسئولانه، [DISCLAIMER.md](../../DISCLAIMER.md) را ببینید. REA برای پژوهش، آموزش، تعامل‌پذیری، حفظ نرم‌افزار و تحلیل نرم‌افزارهایی است که مالک آن‌ها هستید یا اجازهٔ بررسی آن‌ها را دارید، در چارچوب قانون. @@ -267,14 +267,14 @@ applicable licenses. از کمک شما به REA استقبال می‌کنیم! برای گزارش خطا یا پیشنهاد قابلیت، یک [issue باز کنید](https://github.com/morluto/rea/issues) یا برای بهبود کد و مستندات [pull request بفرستید](https://github.com/morluto/rea/pulls). -برای راه‌اندازی محیط توسعه و بررسی‌ها، [CONTRIBUTING.md](CONTRIBUTING.md)، برای مراحل اعتبارسنجی [آزمایش‌ها](docs/testing.md) و برای ساختار پروژه [نقشهٔ معماری](docs/architecture.mermaid) را ببینید. +برای راه‌اندازی محیط توسعه و بررسی‌ها، [CONTRIBUTING.md](../../CONTRIBUTING.md)، برای مراحل اعتبارسنجی [آزمایش‌ها](../testing.md) و برای ساختار پروژه [نقشهٔ معماری](../architecture.mermaid) را ببینید. ## مجوز -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_fr.md b/docs/readme/README_fr.md similarity index 76% rename from README_fr.md rename to docs/readme/README_fr.md index 346210525..e6278ac93 100644 --- a/README_fr.md +++ b/docs/readme/README_fr.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · **Français** · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · **Français** · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA : rétro-ingénierie de tout @@ -9,10 +9,10 @@ **Repérez une fonctionnalité qui vous plaît. Comprenez son fonctionnement, jusqu'au niveau du binaire.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA lance son pont d'analyse dans Hopper pendant l'inspection d'un binaire natif +REA lance son pont d'analyse dans Hopper pendant l'inspection d'un binaire natif

@@ -33,7 +33,7 @@ - Discord
+ Discord
Rejoignez la communauté de rétro-ingénierie
Discord · Questions et réponses · Partage de projets @@ -67,7 +67,7 @@ npx rea-agents setup Choisissez vos agents, vérifiez les modifications proposées et approuvez-les. Setup ajoute le serveur MCP de REA et les instructions de workflow correspondantes, et sauvegarde la configuration existante. Redémarrez ensuite votre agent. -Setup prend en charge Claude Code, Codex, Cursor, Gemini CLI, Grok Build et [d'autres agents](docs/installation.md#supported-agents). Consultez [Installation et configuration](docs/installation.md) pour la configuration des fournisseurs et l'enregistrement MCP manuel. +Setup prend en charge Claude Code, Codex, Cursor, Gemini CLI, Grok Build et [d'autres agents](../installation.md#supported-agents). Consultez [Installation et configuration](../installation.md) pour la configuration des fournisseurs et l'enregistrement MCP manuel. ### Interroger votre agent @@ -95,7 +95,7 @@ npm install --global rea-agents rea --help ``` -Pour l'analyse native, configurez d'abord un fournisseur. Consultez le [guide CLI et Evidence](docs/cli.md) pour les commandes natives, le choix du fournisseur, les snapshots et l'utilisation dans des scripts. +Pour l'analyse native, configurez d'abord un fournisseur. Consultez le [guide CLI et Evidence](../cli.md) pour les commandes natives, le choix du fournisseur, les snapshots et l'utilisation dans des scripts. ### Mettre à jour REA @@ -123,9 +123,9 @@ Vérifiez les modifications proposées par Setup et redémarrez votre agent. Pou Votre agent appelle REA via MCP pour inspecter la cible et suivre le code pertinent. REA renvoie ses constatations avec les preuves correspondantes. L'agent s'en sert pour poser des questions complémentaires, expliquer le comportement, ou écrire et tester une implémentation. Les commandes CLI utilisent les mêmes workflows. -![Flux d'investigation de REA : votre agent pose une question sur une cible locale, REA l'inspecte et la trace avec des outils d'analyse, et l'agent utilise le code, les références et les inconnues renvoyés pour expliquer, implémenter et tester.](website/public/assets/figures/rea-investigation-flow.svg) +![Flux d'investigation de REA : votre agent pose une question sur une cible locale, REA l'inspecte et la trace avec des outils d'analyse, et l'agent utilise le code, les références et les inconnues renvoyés pour expliquer, implémenter et tester.](../../website/public/assets/figures/rea-investigation-flow.svg) -[Ouvrir la figure en taille réelle](website/public/assets/figures/rea-investigation-flow.svg). +[Ouvrir la figure en taille réelle](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -133,38 +133,38 @@ Votre agent appelle REA via MCP pour inspecter la cible et suivre le code pertin REA nécessite Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, ainsi que npm. Les outils supplémentaires et les hôtes pris en charge dépendent de la cible : -| Cible | Ce que REA renvoie | Prérequis et guide | -| ---------------------------- | ----------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -| Binaires natifs | Pseudo-code, assembleur, chaînes, symboles, appels et références | Hopper, Ghidra ou IDA ; [analyse native](https://rea.tools/fr/guides/native/) | -| Structure ELF hors ligne | Sections, segments, symboles et relocations d'origine, et protections statiques candidates | pwntools fourni par l'utilisateur sous Linux x64 ; [diagnostics binaires](docs/binary-diagnostics.md) | -| Artefact WASM | Validation, sections, formes WAT exactes d'import/export et WAT décodé | WABT 1.0.42 fourni par l'utilisateur ; `inspect_wasm_artifact` | -| Bytecode EVM | Sélecteurs de dispatch, offsets en octets, arguments inférés et mutabilité | Fichier local brut ou hexadécimal sous Linux x64 ; [guide EVM hors ligne](docs/evm-bytecode.md) | -| Crashs Linux enregistrés | Notes brutes, registres et signaux de chaque thread enregistré, et correspondances de mappings facultatives | pwntools fourni par l'utilisateur sous Linux x64 ; GDB/pwndbg en option ; [crashs enregistrés](docs/recorded-crashes.md) | -| JavaScript / Electron | Modules, imports, source maps, routes, IPC et relations avec les modules natifs | Node.js et npm ; [analyse d'applications](https://rea.tools/fr/guides/javascript/) | -| Sites web | Structure des pages, scripts, observations réseau et captures d'écran demandées | Un navigateur de la famille Chrome ; [analyse du navigateur](https://rea.tools/fr/guides/browser/) | -| Captures réseau enregistrées | Requêtes, réponses, payloads exposés et emplacements dans le code source | HAR ; mitmdump sous Linux pour les captures mitmproxy natives ; [guide des captures](docs/web-network-captures.md) | -| Captures de paquets | Points de terminaison, DNS, TLS SNI, distribution des protocoles et requêtes HTTP | tshark et capinfos fournis par l'appelant ; [guide pcap](docs/pcap-analysis.md) | -| Assemblys .NET | Métadonnées, instructions CIL, dépendances natives déclarées et comparaisons de builds | Inspection statique ; [guide du code managé](docs/managed-code-analysis.md) | -| APK Android | Déclarations du manifeste, classes, méthodes décompilées et références | JADX headless et un JDK complet sous Linux/macOS/Windows x64 ; [guide Android](docs/android-analysis.md) | -| Appareils Android | Appareils, paquets, processus, journaux, dumps, captures d'écran et transferts d'APK/fichiers | adb fourni par l'appelant (émulateur ou appareil) ; [guide ADB](docs/adb-device-analysis.md) | -| Projets JEB | Unités de projet fournies par le moteur, condensés d'artefacts et pseudo-code de types/méthodes | Un client JEB démarré par l'appelant servant MCP, sélectionné via REA_JEB_MCP_URL ; [guide JEB](docs/jeb-analysis.md) | -| Ressources Android | Manifeste décodé, tables de chaînes, langues et faits de version | Lanceur Apktool fourni par l'appelant ; [guide des ressources](docs/apktool-resource-analysis.md) | -| Applications Flutter | Empreinte de snapshot Dart, condensés de bibliothèques par ABI et provenance de build du moteur | Aucune (analyse APK statique) ; [guide Flutter](docs/flutter-analysis.md) | -| Firmware | Régions, résultats d'extraction et passage de relais vers l'analyse native | Binwalk / Unblob sous Linux ; [guide firmware](docs/firmware-analysis.md) | -| Paquets enregistrés | Champs sélectionnés, valeurs répétées, filtres et règles de décodage | Linux: TShark 4.x + prlimit; [guide des captures](docs/packet-captures.md) | -| Signaux logiques enregistrés | Annotations du décodeur, plages d’échantillons et diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [guide des signaux](docs/signal-recordings.md) | -| Paquets et ressources | Inventaires de fichiers, empreintes, plists, anatomie des bundles Apple et ressources extraites | [Guide artefacts et JavaScript](docs/javascript-artifact-reconstruction.md), [applications Apple](docs/apple-application-analysis.md) | -| Comportement des processus | Sortie du terminal, interactions, code de sortie, observations du système de fichiers et comparaisons | Linux/macOS avec un PTY natif ; [capture de processus](docs/process-capture.md) | +| Cible | Ce que REA renvoie | Prérequis et guide | +| ---------------------------- | ----------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| Binaires natifs | Pseudo-code, assembleur, chaînes, symboles, appels et références | Hopper, Ghidra ou IDA ; [analyse native](https://rea.tools/fr/guides/native/) | +| Structure ELF hors ligne | Sections, segments, symboles et relocations d'origine, et protections statiques candidates | pwntools fourni par l'utilisateur sous Linux x64 ; [diagnostics binaires](../binary-diagnostics.md) | +| Artefact WASM | Validation, sections, formes WAT exactes d'import/export et WAT décodé | WABT 1.0.42 fourni par l'utilisateur ; `inspect_wasm_artifact` | +| Bytecode EVM | Sélecteurs de dispatch, offsets en octets, arguments inférés et mutabilité | Fichier local brut ou hexadécimal sous Linux x64 ; [guide EVM hors ligne](../evm-bytecode.md) | +| Crashs Linux enregistrés | Notes brutes, registres et signaux de chaque thread enregistré, et correspondances de mappings facultatives | pwntools fourni par l'utilisateur sous Linux x64 ; GDB/pwndbg en option ; [crashs enregistrés](../recorded-crashes.md) | +| JavaScript / Electron | Modules, imports, source maps, routes, IPC et relations avec les modules natifs | Node.js et npm ; [analyse d'applications](https://rea.tools/fr/guides/javascript/) | +| Sites web | Structure des pages, scripts, observations réseau et captures d'écran demandées | Un navigateur de la famille Chrome ; [analyse du navigateur](https://rea.tools/fr/guides/browser/) | +| Captures réseau enregistrées | Requêtes, réponses, payloads exposés et emplacements dans le code source | HAR ; mitmdump sous Linux pour les captures mitmproxy natives ; [guide des captures](../web-network-captures.md) | +| Captures de paquets | Points de terminaison, DNS, TLS SNI, distribution des protocoles et requêtes HTTP | tshark et capinfos fournis par l'appelant ; [guide pcap](../pcap-analysis.md) | +| Assemblys .NET | Métadonnées, instructions CIL, dépendances natives déclarées et comparaisons de builds | Inspection statique ; [guide du code managé](../managed-code-analysis.md) | +| APK Android | Déclarations du manifeste, classes, méthodes décompilées et références | JADX headless et un JDK complet sous Linux/macOS/Windows x64 ; [guide Android](../android-analysis.md) | +| Appareils Android | Appareils, paquets, processus, journaux, dumps, captures d'écran et transferts d'APK/fichiers | adb fourni par l'appelant (émulateur ou appareil) ; [guide ADB](../adb-device-analysis.md) | +| Projets JEB | Unités de projet fournies par le moteur, condensés d'artefacts et pseudo-code de types/méthodes | Un client JEB démarré par l'appelant servant MCP, sélectionné via REA_JEB_MCP_URL ; [guide JEB](../jeb-analysis.md) | +| Ressources Android | Manifeste décodé, tables de chaînes, langues et faits de version | Lanceur Apktool fourni par l'appelant ; [guide des ressources](../apktool-resource-analysis.md) | +| Applications Flutter | Empreinte de snapshot Dart, condensés de bibliothèques par ABI et provenance de build du moteur | Aucune (analyse APK statique) ; [guide Flutter](../flutter-analysis.md) | +| Firmware | Régions, résultats d'extraction et passage de relais vers l'analyse native | Binwalk / Unblob sous Linux ; [guide firmware](../firmware-analysis.md) | +| Paquets enregistrés | Champs sélectionnés, valeurs répétées, filtres et règles de décodage | Linux: TShark 4.x + prlimit; [guide des captures](../packet-captures.md) | +| Signaux logiques enregistrés | Annotations du décodeur, plages d’échantillons et diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [guide des signaux](../signal-recordings.md) | +| Paquets et ressources | Inventaires de fichiers, empreintes, plists, anatomie des bundles Apple et ressources extraites | [Guide artefacts et JavaScript](../javascript-artifact-reconstruction.md), [applications Apple](../apple-application-analysis.md) | +| Comportement des processus | Sortie du terminal, interactions, code de sortie, observations du système de fichiers et comparaisons | Linux/macOS avec un PTY natif ; [capture de processus](../process-capture.md) | L'inspection statique JavaScript et .NET lit les fichiers fournis sans exécuter l'application. La capture à l'exécution lance la cible choisie ou interagit avec elle avec vos permissions utilisateur ; chaque guide d'exécution décrit ses effets. -Les formats natifs et les hôtes pris en charge varient selon le fournisseur. Consultez [la configuration de Hopper et Ghidra](docs/installation.md#hopper), le [guide IDA](docs/ida-provider.md) et [la prise en charge expérimentale de Ghidra sous Windows](docs/windows-ghidra-p0.md). Ghidra prend aussi en charge [l'analyse DOS 16 bits](docs/ghidra-dos.md). Pour les gros binaires, augmentez son délai de démarrage avec `REA_GHIDRA_STARTUP_TIMEOUT_MS`. Pour le choix du fournisseur, consultez le [guide CLI](docs/cli.md#choose-a-provider). Vérifiez la [disponibilité dans les versions publiées](docs/installation.md#released-package-and-main) pour les fonctionnalités ajoutées depuis la dernière version npm. +Les formats natifs et les hôtes pris en charge varient selon le fournisseur. Consultez [la configuration de Hopper et Ghidra](../installation.md#hopper), le [guide IDA](../ida-provider.md) et [la prise en charge expérimentale de Ghidra sous Windows](../windows-ghidra-p0.md). Ghidra prend aussi en charge [l'analyse DOS 16 bits](../ghidra-dos.md). Pour les gros binaires, augmentez son délai de démarrage avec `REA_GHIDRA_STARTUP_TIMEOUT_MS`. Pour le choix du fournisseur, consultez le [guide CLI](../cli.md#choose-a-provider). Vérifiez la [disponibilité dans les versions publiées](../installation.md#released-package-and-main) pour les fonctionnalités ajoutées depuis la dernière version npm. ## Études de cas -[![Illustrations des études sur la spatialisation sonore de DX-Ball, le pont du presse-papiers de Notion et le cercle de projectiles de TH04](docs/assets/rea-showcases.png)](https://rea.tools/fr/showcase/) +[![Illustrations des études sur la spatialisation sonore de DX-Ball, le pont du presse-papiers de Notion et le cercle de projectiles de TH04](../assets/rea-showcases.png)](https://rea.tools/fr/showcase/) ### DX-Ball : reconstruire un calcul de panoramique sonore @@ -191,28 +191,28 @@ Si vous avez utilisé REA sur un projet intéressant, nous serions ravis de le d
Quels agents peuvent utiliser REA ? -Tout agent compatible avec les serveurs MCP locaux. Setup configure les [agents pris en charge](docs/installation.md#supported-agents) ; les autres clients peuvent utiliser [l'enregistrement MCP manuel](docs/installation.md#mcp-registry). +Tout agent compatible avec les serveurs MCP locaux. Setup configure les [agents pris en charge](../installation.md#supported-agents) ; les autres clients peuvent utiliser [l'enregistrement MCP manuel](../installation.md#mcp-registry).
Ai-je besoin de Hopper, Ghidra ou IDA ? -L'analyse native approfondie utilise l'un d'eux. L'inspection statique JavaScript et .NET fonctionne sans moteur d'analyse native. Setup peut installer Hopper après votre accord ; Ghidra et IDA utilisent vos installations existantes. Consultez [la configuration des fournisseurs](docs/installation.md#hopper). +L'analyse native approfondie utilise l'un d'eux. L'inspection statique JavaScript et .NET fonctionne sans moteur d'analyse native. Setup peut installer Hopper après votre accord ; Ghidra et IDA utilisent vos installations existantes. Consultez [la configuration des fournisseurs](../installation.md#hopper).
Dois-je lancer Hopper au préalable ? -REA lance Hopper lorsqu'une opération en a besoin. Sous macOS, une boîte de dialogue de premier lancement peut vous demander de choisir le mode démo ou d'activer votre licence. Consultez [Démarrage et dépannage de Hopper](docs/installation.md#launcher-paths-and-troubleshooting). +REA lance Hopper lorsqu'une opération en a besoin. Sous macOS, une boîte de dialogue de premier lancement peut vous demander de choisir le mode démo ou d'activer votre licence. Consultez [Démarrage et dépannage de Hopper](../installation.md#launcher-paths-and-troubleshooting).
Que fait l'installation du skill depuis skills.sh ? -Le skill fournit des instructions d'investigation à votre agent. Utilisez `npx rea-agents setup` pour enregistrer le serveur MCP de REA et installer les instructions correspondantes, puis redémarrez votre agent. Consultez [l'installation du skill seul](docs/installation.md#skill-only-installation). +Le skill fournit des instructions d'investigation à votre agent. Utilisez `npx rea-agents setup` pour enregistrer le serveur MCP de REA et installer les instructions correspondantes, puis redémarrez votre agent. Consultez [l'installation du skill seul](../installation.md#skill-only-installation).
@@ -255,14 +255,14 @@ Si vous utilisez un agent, terminez l'[actualisation de la configuration](#mettr Commencez par les [guides pas à pas](https://rea.tools/fr/guides/) du site web. Pour les options exactes, les prérequis et les contrats de résultats : -- [Installation et configuration](docs/installation.md) : enregistrement des agents, configuration des fournisseurs, mises à jour et désinstallation. -- [Vérification et dépannage](docs/installation.md#check-readiness-for-your-task) : diagnostiquer un agent ou un moteur d'analyse donné. -- [CLI et Evidence](docs/cli.md) : commandes, choix du fournisseur, snapshots, import/export et codes de sortie. -- [Contrats MCP](docs/mcp-contracts.md) et [prompts pour agents](docs/mcp-prompts.md) : résultats des outils, sessions et investigations guidées. -- [Catalogue des outils](docs/mcp-contracts.md#generated-catalog) : inventaire des outils, fournisseurs et commandes CLI, généré lors du build. -- [Feuille de route](docs/roadmap.md) : travaux prévus et suivi des capacités. +- [Installation et configuration](../installation.md) : enregistrement des agents, configuration des fournisseurs, mises à jour et désinstallation. +- [Vérification et dépannage](../installation.md#check-readiness-for-your-task) : diagnostiquer un agent ou un moteur d'analyse donné. +- [CLI et Evidence](../cli.md) : commandes, choix du fournisseur, snapshots, import/export et codes de sortie. +- [Contrats MCP](../mcp-contracts.md) et [prompts pour agents](../mcp-prompts.md) : résultats des outils, sessions et investigations guidées. +- [Catalogue des outils](../mcp-contracts.md#generated-catalog) : inventaire des outils, fournisseurs et commandes CLI, généré lors du build. +- [Feuille de route](../roadmap.md) : travaux prévus et suivi des capacités. -Signalez les vulnérabilités via [SECURITY.md](SECURITY.md). +Signalez les vulnérabilités via [SECURITY.md](../../SECURITY.md). ## Historique des étoiles @@ -280,7 +280,7 @@ Merci à toutes les personnes qui utilisent REA, signalent des bugs, proposent d ## Avertissement -Consultez [DISCLAIMER.md](DISCLAIMER.md) pour les consignes d'utilisation responsable. +Consultez [DISCLAIMER.md](../../DISCLAIMER.md) pour les consignes d'utilisation responsable. REA est destiné à la recherche, à l’éducation, à l’interopérabilité, à la préservation logicielle et à l’analyse de logiciels que vous possédez ou êtes autorisé à examiner, dans le respect de la loi. @@ -292,14 +292,14 @@ applicable licenses. Votre aide sur REA est la bienvenue ! [Ouvrez une issue](https://github.com/morluto/rea/issues) pour signaler un bug ou proposer une fonctionnalité, ou [envoyez une pull request](https://github.com/morluto/rea/pulls) pour améliorer le code ou la documentation. -Consultez [CONTRIBUTING.md](CONTRIBUTING.md) pour l'environnement de développement et les vérifications, la page [tests](docs/testing.md) pour les procédures de vérification, et la [carte de l'architecture](docs/architecture.mermaid) pour la structure du projet. +Consultez [CONTRIBUTING.md](../../CONTRIBUTING.md) pour l'environnement de développement et les vérifications, la page [tests](../testing.md) pour les procédures de vérification, et la [carte de l'architecture](../architecture.mermaid) pour la structure du projet. ## Licence -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_id.md b/docs/readme/README_id.md similarity index 75% rename from README_id.md rename to docs/readme/README_id.md index 7a564c434..fba947fb2 100644 --- a/README_id.md +++ b/docs/readme/README_id.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · **Bahasa Indonesia** · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · **Bahasa Indonesia** · [Italiano](README_it.md) # REA: Rekayasa Balik Apa Pun @@ -9,10 +9,10 @@ **Melihat fitur yang Anda sukai. Pahami cara kerjanya hingga ke tingkat biner.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA menjalankan jembatan analisisnya di dalam Hopper saat memeriksa biner native +REA menjalankan jembatan analisisnya di dalam Hopper saat memeriksa biner native

@@ -33,7 +33,7 @@ - Discord
+ Discord
Bergabunglah dengan Komunitas Rekayasa Balik
Discord · Tanya Jawab · Berbagi Hasil @@ -67,7 +67,7 @@ npx rea-agents setup Pilih agen yang ingin digunakan, tinjau perubahan yang diusulkan, lalu setujui perubahan tersebut. Proses penyiapan menambahkan server MCP REA dan instruksi alur kerja yang sesuai, serta membuat cadangan konfigurasi yang sudah ada. Setelah itu, mulai ulang agen Anda. -Penyiapan mendukung Claude Code, Codex, Cursor, Gemini CLI, Grok Build, dan [agen lainnya](docs/installation.md#supported-agents). Lihat [instalasi dan penyiapan](docs/installation.md) untuk konfigurasi penyedia dan pendaftaran MCP secara manual. +Penyiapan mendukung Claude Code, Codex, Cursor, Gemini CLI, Grok Build, dan [agen lainnya](../installation.md#supported-agents). Lihat [instalasi dan penyiapan](../installation.md) untuk konfigurasi penyedia dan pendaftaran MCP secara manual. ### Minta agen Anda @@ -94,7 +94,7 @@ npm install --global rea-agents rea --help ``` -Untuk analisis native, konfigurasikan penyedia terlebih dahulu. Lihat [Panduan CLI dan Bukti](docs/cli.md) untuk perintah native, pemilihan penyedia, snapshot, dan pembuatan skrip. +Untuk analisis native, konfigurasikan penyedia terlebih dahulu. Lihat [Panduan CLI dan Bukti](../cli.md) untuk perintah native, pemilihan penyedia, snapshot, dan pembuatan skrip. ### Perbarui REA @@ -124,9 +124,9 @@ Agen Anda memanggil REA melalui MCP untuk memeriksa target dan menelusuri kode y Perintah CLI menggunakan alur kerja yang sama. -![Alur investigasi REA: agen Anda menanyakan sesuatu tentang target lokal, REA memeriksa dan menelusurinya menggunakan alat analisis, lalu agen menggunakan kode, referensi, dan hal-hal yang belum diketahui dari hasil tersebut untuk menjelaskan, mengimplementasikan, dan menguji.](website/public/assets/figures/rea-investigation-flow.svg) +![Alur investigasi REA: agen Anda menanyakan sesuatu tentang target lokal, REA memeriksa dan menelusurinya menggunakan alat analisis, lalu agen menggunakan kode, referensi, dan hal-hal yang belum diketahui dari hasil tersebut untuk menjelaskan, mengimplementasikan, dan menguji.](../../website/public/assets/figures/rea-investigation-flow.svg) -[Buka ilustrasi dalam ukuran penuh](website/public/assets/figures/rea-investigation-flow.svg). +[Buka ilustrasi dalam ukuran penuh](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -136,46 +136,46 @@ REA memerlukan Node.js 22.x (>=22.19), 24.x (>=24.11), atau 26+, beserta npm. Alat tambahan dan dukungan sistem host bergantung pada target yang dianalisis: -| Target | Hasil yang diberikan REA | Persyaratan dan panduan | -| -------------------------- | ----------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | -| Biner native | Pseudocode, assembly, string, simbol, pemanggilan, dan referensi | Hopper, Ghidra, atau IDA; [analisis native](https://rea.tools/id/guides/native/) | -| Tata letak ELF offline | Bagian, segmen, simbol/relokasi asli, dan kandidat mitigasi statis | pwntools yang disediakan pemanggil pada Linux x64; [diagnostik biner](docs/binary-diagnostics.md) | -| Artefak WASM | Validasi, bagian, bentuk WAT import/ekspor yang tepat, dan WAT yang didekode | WABT 1.0.42 yang disediakan pemanggil; `inspect_wasm_artifact` | -| Bytecode EVM | Selector dispatch, offset byte, argumen yang disimpulkan, dan sifat mutabilitas | Pembawa data raw/hex lokal; [panduan EVM offline](docs/evm-bytecode.md) | -| Crash Linux yang direkam | Catatan mentah, register/sinyal setiap thread yang direkam, dan kandidat pemetaan opsional | pwntools yang disediakan pemanggil; GDB/pwndbg opsional; [crash yang direkam](docs/recorded-crashes.md) | -| JavaScript / Electron | Modul, impor, source map, rute, IPC, dan hubungan add-on native | Node.js dan npm; [analisis aplikasi](https://rea.tools/id/guides/javascript/) | -| Situs web | Struktur halaman, skrip, pengamatan jaringan, dan tangkapan layar yang diminta | Peramban berbasis Chrome; [analisis peramban](https://rea.tools/id/guides/browser/) | -| Rekaman jaringan tersimpan | Permintaan, respons, payload yang terekspos, dan lokasi sumber | HAR; mitmdump di Linux untuk rekaman native mitmproxy; [panduan rekaman](docs/web-network-captures.md) | -| Capturan paket | Endpoint, DNS, TLS SNI, distribusi protokol, dan permintaan HTTP | tshark dan capinfos buatan pemanggil; [panduan pcap](docs/pcap-analysis.md) | -| Assembly .NET | Metadata, instruksi CIL, dependensi native yang dideklarasikan, dan perbandingan build | Tidak ada (hanya analisis statis); [panduan kode terkelola](docs/managed-code-analysis.md) | -| APK Android | Deklarasi manifest, kelas, metode hasil dekompilasi, dan referensi | JADX mode headless dan JDK lengkap di Linux/macOS/Windows x64; [panduan Android](docs/android-analysis.md) | -| Perangkat Android | Perangkat, paket, proses, log, dump, tangkapan layar, serta transfer APK/berkas | adb buatan pemanggil (emulator atau perangkat); [panduan ADB](docs/adb-device-analysis.md) | -| Proyek JEB | Unit proyek bertenaga mesin, hash artefak, dan pseudokode tipe/metode | Klien JEB yang dijalankan pemanggil dan menyajikan MCP, dipilih dengan REA_JEB_MCP_URL; [panduan JEB](docs/jeb-analysis.md) | -| Resource Android | Manifest terdekode, tabel string, lokal, dan fakta versi | Peluncur Apktool buatan pemanggil; [panduan resource](docs/apktool-resource-analysis.md) | -| Aplikasi Flutter | Hash snapshot Dart, hash pustaka per ABI, dan asal build engine | Tidak ada (parsing APK statis); [panduan Flutter](docs/flutter-analysis.md) | -| Firmware | Wilayah, hasil ekstraksi, dan pengalihan ke analisis native | Binwalk / Unblob di Linux; [panduan firmware](docs/firmware-analysis.md) | -| Paket rekaman | Kolom terpilih, nilai berulang, filter, dan aturan decode-as | Linux: TShark 4.x + prlimit; [panduan tangkapan](docs/packet-captures.md) | -| Sinyal logika terekam | Anotasi dekoder, rentang sampel, dan diagnostik | Linux: sigrok-cli 0.7.2 + prlimit; [panduan sinyal](docs/signal-recordings.md) | -| Paket dan sumber daya | Inventaris berkas, digest, plist, struktur paket Apple, dan sumber daya yang diekstrak | [Panduan artefak dan JavaScript](docs/javascript-artifact-reconstruction.md), [aplikasi Apple](docs/apple-application-analysis.md) | -| Perilaku proses | Keluaran terminal, interaksi, pengamatan proses keluar dan sistem berkas, serta perbandingan eksekusi | Linux/macOS dengan PTY native; [perekaman proses](docs/process-capture.md) | +| Target | Hasil yang diberikan REA | Persyaratan dan panduan | +| -------------------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | +| Biner native | Pseudocode, assembly, string, simbol, pemanggilan, dan referensi | Hopper, Ghidra, atau IDA; [analisis native](https://rea.tools/id/guides/native/) | +| Tata letak ELF offline | Bagian, segmen, simbol/relokasi asli, dan kandidat mitigasi statis | pwntools yang disediakan pemanggil pada Linux x64; [diagnostik biner](../binary-diagnostics.md) | +| Artefak WASM | Validasi, bagian, bentuk WAT import/ekspor yang tepat, dan WAT yang didekode | WABT 1.0.42 yang disediakan pemanggil; `inspect_wasm_artifact` | +| Bytecode EVM | Selector dispatch, offset byte, argumen yang disimpulkan, dan sifat mutabilitas | Pembawa data raw/hex lokal; [panduan EVM offline](../evm-bytecode.md) | +| Crash Linux yang direkam | Catatan mentah, register/sinyal setiap thread yang direkam, dan kandidat pemetaan opsional | pwntools yang disediakan pemanggil; GDB/pwndbg opsional; [crash yang direkam](../recorded-crashes.md) | +| JavaScript / Electron | Modul, impor, source map, rute, IPC, dan hubungan add-on native | Node.js dan npm; [analisis aplikasi](https://rea.tools/id/guides/javascript/) | +| Situs web | Struktur halaman, skrip, pengamatan jaringan, dan tangkapan layar yang diminta | Peramban berbasis Chrome; [analisis peramban](https://rea.tools/id/guides/browser/) | +| Rekaman jaringan tersimpan | Permintaan, respons, payload yang terekspos, dan lokasi sumber | HAR; mitmdump di Linux untuk rekaman native mitmproxy; [panduan rekaman](../web-network-captures.md) | +| Capturan paket | Endpoint, DNS, TLS SNI, distribusi protokol, dan permintaan HTTP | tshark dan capinfos buatan pemanggil; [panduan pcap](../pcap-analysis.md) | +| Assembly .NET | Metadata, instruksi CIL, dependensi native yang dideklarasikan, dan perbandingan build | Tidak ada (hanya analisis statis); [panduan kode terkelola](../managed-code-analysis.md) | +| APK Android | Deklarasi manifest, kelas, metode hasil dekompilasi, dan referensi | JADX mode headless dan JDK lengkap di Linux/macOS/Windows x64; [panduan Android](../android-analysis.md) | +| Perangkat Android | Perangkat, paket, proses, log, dump, tangkapan layar, serta transfer APK/berkas | adb buatan pemanggil (emulator atau perangkat); [panduan ADB](../adb-device-analysis.md) | +| Proyek JEB | Unit proyek bertenaga mesin, hash artefak, dan pseudokode tipe/metode | Klien JEB yang dijalankan pemanggil dan menyajikan MCP, dipilih dengan REA_JEB_MCP_URL; [panduan JEB](../jeb-analysis.md) | +| Resource Android | Manifest terdekode, tabel string, lokal, dan fakta versi | Peluncur Apktool buatan pemanggil; [panduan resource](../apktool-resource-analysis.md) | +| Aplikasi Flutter | Hash snapshot Dart, hash pustaka per ABI, dan asal build engine | Tidak ada (parsing APK statis); [panduan Flutter](../flutter-analysis.md) | +| Firmware | Wilayah, hasil ekstraksi, dan pengalihan ke analisis native | Binwalk / Unblob di Linux; [panduan firmware](../firmware-analysis.md) | +| Paket rekaman | Kolom terpilih, nilai berulang, filter, dan aturan decode-as | Linux: TShark 4.x + prlimit; [panduan tangkapan](../packet-captures.md) | +| Sinyal logika terekam | Anotasi dekoder, rentang sampel, dan diagnostik | Linux: sigrok-cli 0.7.2 + prlimit; [panduan sinyal](../signal-recordings.md) | +| Paket dan sumber daya | Inventaris berkas, digest, plist, struktur paket Apple, dan sumber daya yang diekstrak | [Panduan artefak dan JavaScript](../javascript-artifact-reconstruction.md), [aplikasi Apple](../apple-application-analysis.md) | +| Perilaku proses | Keluaran terminal, interaksi, pengamatan proses keluar dan sistem berkas, serta perbandingan eksekusi | Linux/macOS dengan PTY native; [perekaman proses](../process-capture.md) | Pemeriksaan JavaScript dan .NET statis membaca berkas yang disediakan tanpa menjalankan aplikasinya. Perekaman runtime menjalankan atau berinteraksi dengan target yang dipilih menggunakan izin pengguna Anda. Setiap panduan runtime menjelaskan dampaknya. -Format native dan dukungan sistem host berbeda-beda menurut penyedia. Lihat [penyiapan Hopper dan Ghidra](docs/installation.md#hopper), [panduan IDA](docs/ida-provider.md), dan [dukungan eksperimental Ghidra di Windows](docs/windows-ghidra-p0.md). +Format native dan dukungan sistem host berbeda-beda menurut penyedia. Lihat [penyiapan Hopper dan Ghidra](../installation.md#hopper), [panduan IDA](../ida-provider.md), dan [dukungan eksperimental Ghidra di Windows](../windows-ghidra-p0.md). -Ghidra juga mendukung [analisis DOS 16-bit](docs/ghidra-dos.md). +Ghidra juga mendukung [analisis DOS 16-bit](../ghidra-dos.md). Untuk biner berukuran besar, tingkatkan batas waktu startup menggunakan `REA_GHIDRA_STARTUP_TIMEOUT_MS`. -Untuk memilih penyedia, lihat [panduan CLI](docs/cli.md#choose-a-provider). +Untuk memilih penyedia, lihat [panduan CLI](../cli.md#choose-a-provider). -Periksa [ketersediaan rilis](docs/installation.md#released-package-and-main) untuk mengetahui fitur yang ditambahkan setelah rilis npm terbaru. +Periksa [ketersediaan rilis](../installation.md#released-package-and-main) untuk mengetahui fitur yang ditambahkan setelah rilis npm terbaru. ## Contoh Penggunaan -[![Ilustrasi contoh penggunaan DX-Ball sound-pan, Notion clipboard-bridge, dan TH04 bullet-ring](docs/assets/rea-showcases.png)](https://rea.tools/id/showcase/) +[![Ilustrasi contoh penggunaan DX-Ball sound-pan, Notion clipboard-bridge, dan TH04 bullet-ring](../assets/rea-showcases.png)](https://rea.tools/id/showcase/) ### DX-Ball: merekonstruksi perhitungan sound-pan @@ -202,7 +202,7 @@ Jika Anda telah menggunakan REA untuk menganalisis sesuatu yang menarik, kami in
Agen apa saja yang dapat menggunakan REA? -Agen apa pun yang mendukung server MCP lokal dapat menggunakan REA. Proses penyiapan mengonfigurasi [agen yang didukung](docs/installation.md#supported-agents). Klien lain dapat menggunakan [pendaftaran MCP manual](docs/installation.md#mcp-registry). +Agen apa pun yang mendukung server MCP lokal dapat menggunakan REA. Proses penyiapan mengonfigurasi [agen yang didukung](../installation.md#supported-agents). Klien lain dapat menggunakan [pendaftaran MCP manual](../installation.md#mcp-registry).
@@ -211,7 +211,7 @@ Agen apa pun yang mendukung server MCP lokal dapat menggunakan REA. Proses penyi Analisis native mendalam menggunakan salah satu dari ketiganya. Pemeriksaan JavaScript dan .NET statis dapat dilakukan tanpa mesin analisis native. -Proses penyiapan dapat memasang Hopper setelah memperoleh persetujuan. Ghidra dan IDA menggunakan instalasi yang sudah tersedia. Lihat [penyiapan penyedia](docs/installation.md#hopper). +Proses penyiapan dapat memasang Hopper setelah memperoleh persetujuan. Ghidra dan IDA menggunakan instalasi yang sudah tersedia. Lihat [penyiapan penyedia](../installation.md#hopper). @@ -220,7 +220,7 @@ Proses penyiapan dapat memasang Hopper setelah memperoleh persetujuan. Ghidra da REA akan menjalankan Hopper saat suatu operasi membutuhkannya. Di macOS, dialog saat pertama kali dijalankan mungkin meminta Anda memilih mode demo atau mengaktifkan lisensi. -Lihat [startup dan pemecahan masalah Hopper](docs/installation.md#launcher-paths-and-troubleshooting). +Lihat [startup dan pemecahan masalah Hopper](../installation.md#launcher-paths-and-troubleshooting). @@ -229,7 +229,7 @@ Lihat [startup dan pemecahan masalah Hopper](docs/installation.md#launcher-paths Skill tersebut menyediakan instruksi investigasi untuk agen Anda. Gunakan `npx rea-agents setup` untuk mendaftarkan server MCP REA dan memasang instruksi yang sesuai, lalu mulai ulang agen Anda. -Lihat [instalasi khusus skill](docs/installation.md#skill-only-installation). +Lihat [instalasi khusus skill](../installation.md#skill-only-installation). @@ -276,14 +276,14 @@ Mulailah dengan [panduan praktik di situs web](https://rea.tools/id/guides/). Untuk mengetahui opsi, prasyarat, dan kontrak hasil secara tepat: -- [Instalasi dan penyiapan](docs/installation.md): pendaftaran agen, konfigurasi penyedia, pembaruan, dan penghapusan instalasi. -- [Kesiapan dan pemecahan masalah](docs/installation.md#check-readiness-for-your-task): mendiagnosis satu agen atau mesin analisis. -- [CLI dan Bukti](docs/cli.md): perintah, pemilihan penyedia, snapshot, impor/ekspor, dan status keluar. -- [Kontrak MCP](docs/mcp-contracts.md) dan [prompt agen](docs/mcp-prompts.md): hasil alat, sesi, dan investigasi terpandu. -- [Katalog alat](docs/mcp-contracts.md#generated-catalog): inventaris alat, penyedia, dan perintah CLI yang dibuat secara otomatis. -- [Peta jalan](docs/roadmap.md): pekerjaan yang direncanakan dan pelacak kemampuan. +- [Instalasi dan penyiapan](../installation.md): pendaftaran agen, konfigurasi penyedia, pembaruan, dan penghapusan instalasi. +- [Kesiapan dan pemecahan masalah](../installation.md#check-readiness-for-your-task): mendiagnosis satu agen atau mesin analisis. +- [CLI dan Bukti](../cli.md): perintah, pemilihan penyedia, snapshot, impor/ekspor, dan status keluar. +- [Kontrak MCP](../mcp-contracts.md) dan [prompt agen](../mcp-prompts.md): hasil alat, sesi, dan investigasi terpandu. +- [Katalog alat](../mcp-contracts.md#generated-catalog): inventaris alat, penyedia, dan perintah CLI yang dibuat secara otomatis. +- [Peta jalan](../roadmap.md): pekerjaan yang direncanakan dan pelacak kemampuan. -Laporkan kerentanan melalui [SECURITY.md](SECURITY.md). +Laporkan kerentanan melalui [SECURITY.md](../../SECURITY.md). ## Riwayat Bintang @@ -301,7 +301,7 @@ Terima kasih kepada semua orang yang menggunakan REA, melaporkan bug, meminta fi ## Penafian -Lihat [DISCLAIMER.md](DISCLAIMER.md) untuk panduan penggunaan yang bertanggung jawab. +Lihat [DISCLAIMER.md](../../DISCLAIMER.md) untuk panduan penggunaan yang bertanggung jawab. REA ditujukan untuk penelitian, pendidikan, interoperabilitas, pelestarian perangkat lunak, dan analisis perangkat lunak yang Anda miliki atau diizinkan untuk Anda periksa, sesuai hukum. @@ -313,14 +313,14 @@ applicable licenses. Kami sangat menghargai bantuan Anda untuk REA! [Buat issue](https://github.com/morluto/rea/issues) untuk melaporkan bug atau menyarankan fitur, atau [kirim pull request](https://github.com/morluto/rea/pulls) untuk meningkatkan kode atau dokumentasi. -Lihat [CONTRIBUTING.md](CONTRIBUTING.md) untuk penyiapan pengembangan dan pemeriksaan, [pengujian](docs/testing.md) untuk tahapan verifikasi, serta [peta arsitektur](docs/architecture.mermaid) untuk struktur proyek. +Lihat [CONTRIBUTING.md](../../CONTRIBUTING.md) untuk penyiapan pengembangan dan pemeriksaan, [pengujian](../testing.md) untuk tahapan verifikasi, serta [peta arsitektur](../architecture.mermaid) untuk struktur proyek. ## Lisensi -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_it.md b/docs/readme/README_it.md similarity index 74% rename from README_it.md rename to docs/readme/README_it.md index 8b1c55afe..b7e7c6100 100644 --- a/README_it.md +++ b/docs/readme/README_it.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · **Italiano** +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · **Italiano** # REA: Reverse Engineer Anything @@ -9,10 +9,10 @@ **Vedi una funzionalità che ti piace. Capisci come funziona, fino al livello binario.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#cosa-puoi-analizzare) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA che avvia il suo bridge di analisi dentro Hopper mentre ispeziona un binario nativo +REA che avvia il suo bridge di analisi dentro Hopper mentre ispeziona un binario nativo

@@ -33,7 +33,7 @@ - Discord
+ Discord
Unisciti alla community di ingegneria inversa
Discord · Domande e risposte · Show and Tell @@ -67,7 +67,7 @@ npx rea-agents setup Scegli i tuoi agenti, rivedi le modifiche proposte e approvale. La configurazione aggiunge il server MCP di REA e le istruzioni di workflow corrispondenti, con backup della configurazione esistente. Riavvia quindi il tuo agente. -La configurazione supporta Claude Code, Codex, Cursor, Gemini CLI, Grok Build e [altri agenti](docs/installation.md#supported-agents). Vedi [installazione e configurazione](docs/installation.md) per la configurazione dei provider e la registrazione manuale MCP. +La configurazione supporta Claude Code, Codex, Cursor, Gemini CLI, Grok Build e [altri agenti](../installation.md#supported-agents). Vedi [installazione e configurazione](../installation.md) per la configurazione dei provider e la registrazione manuale MCP. ### Chiedi al tuo agente @@ -83,7 +83,7 @@ Sostituisci Note con la tua app di destinazione e la funzionalità che vuoi capi Ispeziona una directory di app JavaScript/Electron estratta o un ASAR: ```bash -npx -y rea-agents@latest analyze-javascript-application /percorso/assoluto/verso/app --json +npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json ``` Il risultato include moduli, import, confini Electron e le relative prove. Sostituisci il percorso con il tuo bersaglio, ad esempio `"D:/apps/example"` su Windows. @@ -95,7 +95,7 @@ npm install --global rea-agents rea --help ``` -Per l'analisi nativa, configura prima un provider. Vedi la [guida CLI ed Evidence](docs/cli.md) per i comandi nativi, la selezione del provider, gli snapshot e lo scripting. +Per l'analisi nativa, configura prima un provider. Vedi la [guida CLI ed Evidence](../cli.md) per i comandi nativi, la selezione del provider, gli snapshot e lo scripting. ### Aggiorna REA @@ -123,9 +123,9 @@ Rivedi le modifiche di configurazione e riavvia il tuo agente. Per comandi CLI s Il tuo agente chiama REA tramite MCP per ispezionare il bersaglio e tracciare il codice rilevante. REA restituisce i risultati con le relative prove. L'agente li usa per fare domande di approfondimento, spiegare il comportamento o scrivere e testare un'implementazione. Anche i comandi CLI usano gli stessi workflow. -![Flusso di indagine REA: il tuo agente pone domande su un bersaglio locale, REA lo ispeziona e ne traccia il codice usando strumenti di analisi, e l'agente usa il codice, i riferimenti e gli elementi sconosciuti restituiti per spiegare, implementare e testare.](website/public/assets/figures/rea-investigation-flow.svg) +![Flusso di indagine REA: il tuo agente pone domande su un bersaglio locale, REA lo ispeziona e ne traccia il codice usando strumenti di analisi, e l'agente usa il codice, i riferimenti e gli elementi sconosciuti restituiti per spiegare, implementare e testare.](../../website/public/assets/figures/rea-investigation-flow.svg) -[Apri la figura a dimensione intera](website/public/assets/figures/rea-investigation-flow.svg). +[Apri la figura a dimensione intera](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -133,38 +133,38 @@ Il tuo agente chiama REA tramite MCP per ispezionare il bersaglio e tracciare il REA richiede Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, oltre a npm. Strumenti aggiuntivi e supporto host dipendono dal bersaglio: -| Obiettivo | Cosa restituisce REA | Requisiti e guida | -| ---------------------------- | ------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | -| Binari nativi | Pseudocodice, assembly, stringhe, simboli, chiamate e riferimenti | Hopper, Ghidra o IDA; [analisi nativa](https://rea.tools/it/guides/native/) | -| Layout ELF offline | Sezioni, segmenti, simboli/relocazioni originali e candidati mitigazioni statiche | pwntools fornito dal chiamante su Linux x64; [diagnostica binaria](docs/binary-diagnostics.md) | -| Artefatto WASM | Convalida, sezioni, forme WAT di import/export esatte e WAT decodificato | WABT 1.0.42 fornito dal chiamante; `inspect_wasm_artifact` | -| Bytecode EVM | Selettori di dispatch, offset dei byte, argomenti inferiti e mutabilità | Carrier raw/hex locale; [guida EVM offline](docs/evm-bytecode.md) | -| Crash Linux registrati | Note grezze, registri/segnai di ogni thread registrato e candidati mapping opzionali | pwntools fornito dal chiamante; GDB/pwndbg opzionale; [crash registrati](docs/recorded-crashes.md) | -| JavaScript / Electron | Moduli, import, source map, route, IPC e relazioni con add-on nativi | Node.js e npm; [analisi delle applicazioni](https://rea.tools/it/guides/javascript/) | -| Siti web | Struttura della pagina, script, osservazioni di rete e screenshot richiesti | Un browser della famiglia Chrome; [analisi browser](https://rea.tools/it/guides/browser/) | -| Acquisizioni di rete salvate | Richieste, risposte, payload esposti e posizioni sorgente | HAR; mitmdump su Linux per acquisizioni mitmproxy native; [guida alle acquisizioni](docs/web-network-captures.md) | -| Captured di pacchetti | Endpoint, DNS, TLS SNI, distribuzione dei protocolli e richieste HTTP | tshark e capinfos forniti dal chiamante; [guida pcap](docs/pcap-analysis.md) | -| Assembly .NET | Metadati, istruzioni CIL, dipendenze native dichiarate e confronti di build | Nessuno (solo statico); [guida managed-code](docs/managed-code-analysis.md) | -| APK Android | Dichiarazioni manifest, classi, metodi decompilati e riferimenti | JADX headless e un JDK completo su Linux/macOS/Windows x64; [guida Android](docs/android-analysis.md) | -| Dispositivi Android | Dispositivi, pacchetti, processi, log, dump, screenshot e trasferimenti APK/file | adb fornito dal chiamante (emulatore o hardware); [guida ADB](docs/adb-device-analysis.md) | -| Progetti JEB | Unità di progetto dal motore, digest degli artifact e pseudocodice di tipi/metodi | Client JEB avviato dal chiamante via MCP, selezionato con REA_JEB_MCP_URL; [guida JEB](docs/jeb-analysis.md) | -| Risorse Android | Manifest decodificato, tabelle stringhe, lingue e dati di versione | Launcher Apktool fornito dal chiamante; [guida alle risorse](docs/apktool-resource-analysis.md) | -| App Flutter | Hash dello snapshot Dart, digest delle librerie per ABI e provenienza della build del motore | Nessuno (analisi APK statica); [guida Flutter](docs/flutter-analysis.md) | -| Firmware | Regioni, risultati di estrazione e passaggi all'analisi nativa | Binwalk / Unblob su Linux; [guida firmware](docs/firmware-analysis.md) | -| Pacchetti registrati | Campi selezionati, valori ripetuti, filtri e regole di decodifica | Linux: TShark 4.x + prlimit; [guida alle catture](docs/packet-captures.md) | -| Segnali logici registrati | Annotazioni del decoder, intervalli di campioni e diagnostica | Linux: sigrok-cli 0.7.2 + prlimit; [guida ai segnali](docs/signal-recordings.md) | -| Pacchetti e risorse | Inventari file, digest, plist, anatomia dei bundle Apple e risorse estratte | [Guida artifact e JavaScript](docs/javascript-artifact-reconstruction.md), [Applicazioni Apple](docs/apple-application-analysis.md) | -| Comportamento dei processi | Output del terminale, interazioni, osservazioni su uscita e filesystem ed confronti di esecuzione | Linux/macOS con un PTY nativo; [cattura dei processi](docs/process-capture.md) | +| Obiettivo | Cosa restituisce REA | Requisiti e guida | +| ---------------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | +| Binari nativi | Pseudocodice, assembly, stringhe, simboli, chiamate e riferimenti | Hopper, Ghidra o IDA; [analisi nativa](https://rea.tools/it/guides/native/) | +| Layout ELF offline | Sezioni, segmenti, simboli/relocazioni originali e candidati mitigazioni statiche | pwntools fornito dal chiamante su Linux x64; [diagnostica binaria](../binary-diagnostics.md) | +| Artefatto WASM | Convalida, sezioni, forme WAT di import/export esatte e WAT decodificato | WABT 1.0.42 fornito dal chiamante; `inspect_wasm_artifact` | +| Bytecode EVM | Selettori di dispatch, offset dei byte, argomenti inferiti e mutabilità | Carrier raw/hex locale; [guida EVM offline](../evm-bytecode.md) | +| Crash Linux registrati | Note grezze, registri/segnai di ogni thread registrato e candidati mapping opzionali | pwntools fornito dal chiamante; GDB/pwndbg opzionale; [crash registrati](../recorded-crashes.md) | +| JavaScript / Electron | Moduli, import, source map, route, IPC e relazioni con add-on nativi | Node.js e npm; [analisi delle applicazioni](https://rea.tools/it/guides/javascript/) | +| Siti web | Struttura della pagina, script, osservazioni di rete e screenshot richiesti | Un browser della famiglia Chrome; [analisi browser](https://rea.tools/it/guides/browser/) | +| Acquisizioni di rete salvate | Richieste, risposte, payload esposti e posizioni sorgente | HAR; mitmdump su Linux per acquisizioni mitmproxy native; [guida alle acquisizioni](../web-network-captures.md) | +| Captured di pacchetti | Endpoint, DNS, TLS SNI, distribuzione dei protocolli e richieste HTTP | tshark e capinfos forniti dal chiamante; [guida pcap](../pcap-analysis.md) | +| Assembly .NET | Metadati, istruzioni CIL, dipendenze native dichiarate e confronti di build | Nessuno (solo statico); [guida managed-code](../managed-code-analysis.md) | +| APK Android | Dichiarazioni manifest, classi, metodi decompilati e riferimenti | JADX headless e un JDK completo su Linux/macOS/Windows x64; [guida Android](../android-analysis.md) | +| Dispositivi Android | Dispositivi, pacchetti, processi, log, dump, screenshot e trasferimenti APK/file | adb fornito dal chiamante (emulatore o hardware); [guida ADB](../adb-device-analysis.md) | +| Progetti JEB | Unità di progetto dal motore, digest degli artifact e pseudocodice di tipi/metodi | Client JEB avviato dal chiamante via MCP, selezionato con REA_JEB_MCP_URL; [guida JEB](../jeb-analysis.md) | +| Risorse Android | Manifest decodificato, tabelle stringhe, lingue e dati di versione | Launcher Apktool fornito dal chiamante; [guida alle risorse](../apktool-resource-analysis.md) | +| App Flutter | Hash dello snapshot Dart, digest delle librerie per ABI e provenienza della build del motore | Nessuno (analisi APK statica); [guida Flutter](../flutter-analysis.md) | +| Firmware | Regioni, risultati di estrazione e passaggi all'analisi nativa | Binwalk / Unblob su Linux; [guida firmware](../firmware-analysis.md) | +| Pacchetti registrati | Campi selezionati, valori ripetuti, filtri e regole di decodifica | Linux: TShark 4.x + prlimit; [guida alle catture](../packet-captures.md) | +| Segnali logici registrati | Annotazioni del decoder, intervalli di campioni e diagnostica | Linux: sigrok-cli 0.7.2 + prlimit; [guida ai segnali](../signal-recordings.md) | +| Pacchetti e risorse | Inventari file, digest, plist, anatomia dei bundle Apple e risorse estratte | [Guida artifact e JavaScript](../javascript-artifact-reconstruction.md), [Applicazioni Apple](../apple-application-analysis.md) | +| Comportamento dei processi | Output del terminale, interazioni, osservazioni su uscita e filesystem ed confronti di esecuzione | Linux/macOS con un PTY nativo; [cattura dei processi](../process-capture.md) | L'ispezione statica di JavaScript e .NET legge i file forniti senza eseguire l'applicazione. La cattura a runtime esegue o interagisce con il bersaglio selezionato usando i tuoi permessi utente; ogni guida runtime descrive i suoi effetti. -I formati nativi e il supporto host variano in base al provider. Vedi la [configurazione di Hopper e Ghidra](docs/installation.md#hopper), la [guida IDA](docs/ida-provider.md) e il [supporto sperimentale Windows Ghidra](docs/windows-ghidra-p0.md). Ghidra supporta anche l'[analisi DOS a 16 bit](docs/ghidra-dos.md). Per binari di grandi dimensioni, aumenta il timeout di avvio con `REA_GHIDRA_STARTUP_TIMEOUT_MS`. Per la selezione del provider, vedi la [guida CLI](docs/cli.md#choose-a-provider). Controlla la [disponibilità delle release](docs/installation.md#released-package-and-main) per le funzionalità aggiunte dall'ultima release npm. +I formati nativi e il supporto host variano in base al provider. Vedi la [configurazione di Hopper e Ghidra](../installation.md#hopper), la [guida IDA](../ida-provider.md) e il [supporto sperimentale Windows Ghidra](../windows-ghidra-p0.md). Ghidra supporta anche l'[analisi DOS a 16 bit](../ghidra-dos.md). Per binari di grandi dimensioni, aumenta il timeout di avvio con `REA_GHIDRA_STARTUP_TIMEOUT_MS`. Per la selezione del provider, vedi la [guida CLI](../cli.md#choose-a-provider). Controlla la [disponibilità delle release](../installation.md#released-package-and-main) per le funzionalità aggiunte dall'ultima release npm. ## Showcase -[![Illustrazioni degli showcase del sound-pan di DX-Ball, del clipboard-bridge di Notion e dell'anello di proiettili di TH04](docs/assets/rea-showcases.png)](https://rea.tools/it/showcase/) +[![Illustrazioni degli showcase del sound-pan di DX-Ball, del clipboard-bridge di Notion e dell'anello di proiettili di TH04](../assets/rea-showcases.png)](https://rea.tools/it/showcase/) ### DX-Ball: ricostruisci un calcolo sound-pan @@ -191,28 +191,28 @@ Se hai usato REA su qualcosa di interessante, ci piacerebbe vederlo. Condividi i
Quali agenti possono usare REA? -Qualsiasi agente che supporti server MCP locali. La configurazione configura gli [agenti supportati](docs/installation.md#supported-agents); altri client possono usare la [registrazione MCP manuale](docs/installation.md#mcp-registry). +Qualsiasi agente che supporti server MCP locali. La configurazione configura gli [agenti supportati](../installation.md#supported-agents); altri client possono usare la [registrazione MCP manuale](../installation.md#mcp-registry).
Ho bisogno di Hopper, Ghidra o IDA? -L'analisi nativa profonda ne usa uno. L'ispezione statica di JavaScript e .NET funziona senza un motore di analisi nativo. La configurazione può installare Hopper previa approvazione; Ghidra e IDA usano le tue installazioni esistenti. Vedi [configurazione provider](docs/installation.md#hopper). +L'analisi nativa profonda ne usa uno. L'ispezione statica di JavaScript e .NET funziona senza un motore di analisi nativo. La configurazione può installare Hopper previa approvazione; Ghidra e IDA usano le tue installazioni esistenti. Vedi [configurazione provider](../installation.md#hopper).
Devo avviare Hopper per primo? -REA avvia Hopper quando un'operazione ne ha bisogno. Su macOS, una finestra di dialogo al primo avvio potrebbe chiederti di scegliere la modalità demo o di attivare la licenza. Vedi [avvio di Hopper e risoluzione dei problemi](docs/installation.md#launcher-paths-and-troubleshooting). +REA avvia Hopper quando un'operazione ne ha bisogno. Su macOS, una finestra di dialogo al primo avvio potrebbe chiederti di scegliere la modalità demo o di attivare la licenza. Vedi [avvio di Hopper e risoluzione dei problemi](../installation.md#launcher-paths-and-troubleshooting).
Cosa fa l'installazione dello skill da skills.sh? -Lo skill fornisce istruzioni di indagine per il tuo agente. Usa `npx rea-agents setup` per registrare il server MCP di REA e installare le istruzioni corrispondenti, quindi riavvia il tuo agente. Vedi [installazione solo skill](docs/installation.md#skill-only-installation). +Lo skill fornisce istruzioni di indagine per il tuo agente. Usa `npx rea-agents setup` per registrare il server MCP di REA e installare le istruzioni corrispondenti, quindi riavvia il tuo agente. Vedi [installazione solo skill](../installation.md#skill-only-installation).
@@ -255,14 +255,14 @@ Se stai usando un agente, completa l'[aggiornamento della configurazione](#aggio Inizia dalle [guide pratiche](https://rea.tools/it/guides/) del sito web. Per opzioni esatte, prerequisiti e contratti dei risultati: -- [Installazione e configurazione](docs/installation.md): registrazione agente, configurazione provider, aggiornamenti e disinstallazione. -- [Pronto all'uso e risoluzione dei problemi](docs/installation.md#check-readiness-for-your-task): diagnostica un agente o un motore di analisi. -- [CLI ed Evidence](docs/cli.md): comandi, selezione provider, snapshot, import/export e stati di uscita. -- [Contratti MCP](docs/mcp-contracts.md) e [prompt agente](docs/mcp-prompts.md): risultati degli strumenti, sessioni e indagini guidate. -- [Catalogo strumenti](docs/mcp-contracts.md#generated-catalog): inventario generato di strumenti, provider e comandi CLI. -- [Roadmap](docs/roadmap.md): lavoro pianificato e tracker delle capacità. +- [Installazione e configurazione](../installation.md): registrazione agente, configurazione provider, aggiornamenti e disinstallazione. +- [Pronto all'uso e risoluzione dei problemi](../installation.md#check-readiness-for-your-task): diagnostica un agente o un motore di analisi. +- [CLI ed Evidence](../cli.md): comandi, selezione provider, snapshot, import/export e stati di uscita. +- [Contratti MCP](../mcp-contracts.md) e [prompt agente](../mcp-prompts.md): risultati degli strumenti, sessioni e indagini guidate. +- [Catalogo strumenti](../mcp-contracts.md#generated-catalog): inventario generato di strumenti, provider e comandi CLI. +- [Roadmap](../roadmap.md): lavoro pianificato e tracker delle capacità. -Segnala vulnerabilità tramite [SECURITY.md](SECURITY.md). +Segnala vulnerabilità tramite [SECURITY.md](../../SECURITY.md). ## Cronologia delle stelle @@ -280,7 +280,7 @@ Grazie a tutti coloro che usano REA, segnalano bug, propongono funzionalità, te ## Disclaimer -Consulta [DISCLAIMER.md](DISCLAIMER.md) per le indicazioni sull'uso responsabile. +Consulta [DISCLAIMER.md](../../DISCLAIMER.md) per le indicazioni sull'uso responsabile. REA è destinato alla ricerca, alla formazione, all’interoperabilità, alla conservazione del software e all’analisi di software di tua proprietà o che sei autorizzato a esaminare, nel rispetto della legge. @@ -292,14 +292,14 @@ applicable licenses. Apprezzeremmo il tuo aiuto con REA! [Apri un issue](https://github.com/morluto/rea/issues) per segnalare un bug o suggerire una funzionalità, oppure [invia una pull request](https://github.com/morluto/rea/pulls) per migliorare il codice o la documentazione. -Vedi [CONTRIBUTING.md](CONTRIBUTING.md) per la configurazione di sviluppo e i controlli, [testing](docs/testing.md) per le lane di verifica e la [mappa dell'architettura](docs/architecture.mermaid) per la struttura del progetto. +Vedi [CONTRIBUTING.md](../../CONTRIBUTING.md) per la configurazione di sviluppo e i controlli, [testing](../testing.md) per le lane di verifica e la [mappa dell'architettura](../architecture.mermaid) per la struttura del progetto. ## License -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_ja.md b/docs/readme/README_ja.md similarity index 76% rename from README_ja.md rename to docs/readme/README_ja.md index 5fb42d096..c9d7981ba 100644 --- a/README_ja.md +++ b/docs/readme/README_ja.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · **日本語** · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · **日本語** · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA：あらゆるものをリバースエンジニアリング @@ -9,10 +9,10 @@ **気に入った機能を見つけたら、その仕組みをバイナリのレベルまで理解する。** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA が Hopper 内で分析ブリッジを起動し、ネイティブバイナリを調べる様子 +REA が Hopper 内で分析ブリッジを起動し、ネイティブバイナリを調べる様子

@@ -33,7 +33,7 @@ - Discord
+ Discord
リバースエンジニアリングのコミュニティに参加
Discord · 質問と回答 · 成果の共有 @@ -67,7 +67,7 @@ npx rea-agents setup 使うエージェントを選び、予定されている変更を確認して承認してください。セットアップは REA の MCP サーバーと対応するワークフローの手順を追加し、既存の設定をバックアップします。完了後にエージェントを再起動してください。 -セットアップは Claude Code、Codex、Cursor、Gemini CLI、Grok Build、および[その他のエージェント](docs/installation.md#supported-agents)に対応しています。プロバイダーの設定と MCP の手動登録については、[インストールとセットアップ](docs/installation.md)を参照してください。 +セットアップは Claude Code、Codex、Cursor、Gemini CLI、Grok Build、および[その他のエージェント](../installation.md#supported-agents)に対応しています。プロバイダーの設定と MCP の手動登録については、[インストールとセットアップ](../installation.md)を参照してください。 ### エージェントに依頼する @@ -94,7 +94,7 @@ npm install --global rea-agents rea --help ``` -ネイティブ分析には、先にプロバイダーの設定が必要です。ネイティブ分析のコマンド、プロバイダーの選択、スナップショット、スクリプトでの利用は、[CLI と Evidence のガイド](docs/cli.md)を参照してください。 +ネイティブ分析には、先にプロバイダーの設定が必要です。ネイティブ分析のコマンド、プロバイダーの選択、スナップショット、スクリプトでの利用は、[CLI と Evidence のガイド](../cli.md)を参照してください。 ### REA を更新する @@ -120,9 +120,9 @@ npx rea-agents@latest setup エージェントは MCP 経由で REA を呼び出し、対象を調べて関連するコードを追跡します。REA は調査結果と根拠を返します。エージェントはそれを使って追加の質問をしたり、動作を説明したり、実装を書いてテストしたりします。CLI コマンドも同じワークフローを使います。 -![REA の調査フロー：エージェントがローカルの対象について質問し、REA が分析ツールで調査・追跡します。エージェントは返されたコード、参照、未解明の点を使い、説明、実装、テストを行います。](website/public/assets/figures/rea-investigation-flow.svg) +![REA の調査フロー：エージェントがローカルの対象について質問し、REA が分析ツールで調査・追跡します。エージェントは返されたコード、参照、未解明の点を使い、説明、実装、テストを行います。](../../website/public/assets/figures/rea-investigation-flow.svg) -[原寸大の図を開く](website/public/assets/figures/rea-investigation-flow.svg)。 +[原寸大の図を開く](../../website/public/assets/figures/rea-investigation-flow.svg)。 @@ -130,37 +130,37 @@ npx rea-agents@latest setup REA には Node.js 22.x（>=22.19）、24.x（>=24.11）、または 26+ と npm が必要です。追加のツールや対応ホストは、対象によって異なります。 -| 対象 | REA が返すもの | 要件とガイド | -| -------------------------------- | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | -| ネイティブバイナリ | 疑似コード、アセンブリ、文字列、シンボル、呼び出し、参照 | Hopper、Ghidra、または IDA；[ネイティブ分析](https://rea.tools/ja/guides/native/) | -| オフライン ELF レイアウト | セクション、セグメント、元のシンボル／再配置情報、静的解析によるセキュリティ対策の候補 | Linux x64 で利用者が用意する pwntools；[バイナリ診断](docs/binary-diagnostics.md) | -| EVM バイトコード | ディスパッチセレクター、バイトオフセット、推定された引数と状態変更の可否 | Linux x64 上のローカルの生バイト／16 進数の入力ファイル；[オフライン EVM ガイド](docs/evm-bytecode.md) | -| 記録済みの Linux クラッシュ | 生の note レコード、記録された全スレッドのレジスター／シグナル、任意のマッピング候補 | Linux x64 で利用者が用意する pwntools；GDB/pwndbg は任意；[記録済みクラッシュ](docs/recorded-crashes.md) | -| JavaScript / Electron | モジュール、インポート、ソースマップ、ルート、IPC、ネイティブアドオンの関係 | Node.js と npm；[アプリケーション分析](https://rea.tools/ja/guides/javascript/) | -| ウェブサイト | ページ構造、スクリプト、ネットワークの観測結果、依頼されたスクリーンショット | Chrome 系ブラウザー；[ブラウザー分析](https://rea.tools/ja/guides/browser/) | -| 保存済みのネットワークキャプチャ | リクエスト、レスポンス、取得可能なペイロード、ソース内の位置 | HAR；mitmproxy ネイティブ形式のキャプチャには Linux 上の mitmdump；[キャプチャガイド](docs/web-network-captures.md) | -| パケットキャプチャ | エンドポイント、DNS、TLS SNI、プロトコル分布、HTTPリクエスト | 呼び出し側が用意した tshark と capinfos；[pcapガイド](docs/pcap-analysis.md) | -| .NET アセンブリ | メタデータ、CIL 命令、宣言されたネイティブ依存関係、ビルドの比較 | 静的な検査；[マネージドコードのガイド](docs/managed-code-analysis.md) | -| Android APK | マニフェストの宣言、クラス、逆コンパイルされたメソッド、参照 | Linux/macOS/Windows x64 上のヘッドレス JADX と完全な JDK；[Android ガイド](docs/android-analysis.md) | -| Android デバイス | デバイス、パッケージ、プロセス、ログ、ダンプ、スクリーンキャプチャ、APK/ファイル転送 | 呼び出し側が用意した adb（エミュレーターまたは実機）；[ADB ガイド](docs/adb-device-analysis.md) | -| JEBプロジェクト | エンジンによるプロジェクトユニット、アーティファクトのダイジェスト、型/メソッドの擬似コード | 呼び出し側が起動し MCP を提供する JEB クライアント（REA_JEB_MCP_URL で選択）；[JEB ガイド](docs/jeb-analysis.md) | -| Android リソース | デコードされたマニフェスト、文字列テーブル、ロケール、バージョン情報 | 呼び出し側が用意した Apktool ランチャー；[リソースガイド](docs/apktool-resource-analysis.md) | -| Flutter アプリ | Dart スナップショットハッシュ、ABI ごとのライブラリダイジェスト、エンジンビルドの来歴 | 不要（静的 APK 解析）；[Flutter ガイド](docs/flutter-analysis.md) | -| ファームウェア | 領域、抽出結果、ネイティブ分析への引き継ぎ | Linux 上の Binwalk / Unblob；[ファームウェアガイド](docs/firmware-analysis.md) | -| 記録済みパケット | 選択フィールド、繰り返し値、フィルター、デコード指定 | Linux: TShark 4.x + prlimit; [キャプチャガイド](docs/packet-captures.md) | -| 記録済みロジック信号 | デコーダ注釈、サンプル範囲、診断情報 | Linux: sigrok-cli 0.7.2 + prlimit; [信号記録ガイド](docs/signal-recordings.md) | -| パッケージとリソース | ファイル一覧、ダイジェスト、plist、Apple バンドルの構造、抽出されたリソース | [アーティファクトと JavaScript のガイド](docs/javascript-artifact-reconstruction.md)、[Apple アプリケーション](docs/apple-application-analysis.md) | -| プロセスの動作 | ターミナル出力、対話、終了とファイルシステムの観測結果、実行の比較 | ネイティブ PTY を備えた Linux/macOS；[プロセスキャプチャ](docs/process-capture.md) | +| 対象 | REA が返すもの | 要件とガイド | +| -------------------------------- | ------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | +| ネイティブバイナリ | 疑似コード、アセンブリ、文字列、シンボル、呼び出し、参照 | Hopper、Ghidra、または IDA；[ネイティブ分析](https://rea.tools/ja/guides/native/) | +| オフライン ELF レイアウト | セクション、セグメント、元のシンボル／再配置情報、静的解析によるセキュリティ対策の候補 | Linux x64 で利用者が用意する pwntools；[バイナリ診断](../binary-diagnostics.md) | +| EVM バイトコード | ディスパッチセレクター、バイトオフセット、推定された引数と状態変更の可否 | Linux x64 上のローカルの生バイト／16 進数の入力ファイル；[オフライン EVM ガイド](../evm-bytecode.md) | +| 記録済みの Linux クラッシュ | 生の note レコード、記録された全スレッドのレジスター／シグナル、任意のマッピング候補 | Linux x64 で利用者が用意する pwntools；GDB/pwndbg は任意；[記録済みクラッシュ](../recorded-crashes.md) | +| JavaScript / Electron | モジュール、インポート、ソースマップ、ルート、IPC、ネイティブアドオンの関係 | Node.js と npm；[アプリケーション分析](https://rea.tools/ja/guides/javascript/) | +| ウェブサイト | ページ構造、スクリプト、ネットワークの観測結果、依頼されたスクリーンショット | Chrome 系ブラウザー；[ブラウザー分析](https://rea.tools/ja/guides/browser/) | +| 保存済みのネットワークキャプチャ | リクエスト、レスポンス、取得可能なペイロード、ソース内の位置 | HAR；mitmproxy ネイティブ形式のキャプチャには Linux 上の mitmdump；[キャプチャガイド](../web-network-captures.md) | +| パケットキャプチャ | エンドポイント、DNS、TLS SNI、プロトコル分布、HTTPリクエスト | 呼び出し側が用意した tshark と capinfos；[pcapガイド](../pcap-analysis.md) | +| .NET アセンブリ | メタデータ、CIL 命令、宣言されたネイティブ依存関係、ビルドの比較 | 静的な検査；[マネージドコードのガイド](../managed-code-analysis.md) | +| Android APK | マニフェストの宣言、クラス、逆コンパイルされたメソッド、参照 | Linux/macOS/Windows x64 上のヘッドレス JADX と完全な JDK；[Android ガイド](../android-analysis.md) | +| Android デバイス | デバイス、パッケージ、プロセス、ログ、ダンプ、スクリーンキャプチャ、APK/ファイル転送 | 呼び出し側が用意した adb（エミュレーターまたは実機）；[ADB ガイド](../adb-device-analysis.md) | +| JEBプロジェクト | エンジンによるプロジェクトユニット、アーティファクトのダイジェスト、型/メソッドの擬似コード | 呼び出し側が起動し MCP を提供する JEB クライアント（REA_JEB_MCP_URL で選択）；[JEB ガイド](../jeb-analysis.md) | +| Android リソース | デコードされたマニフェスト、文字列テーブル、ロケール、バージョン情報 | 呼び出し側が用意した Apktool ランチャー；[リソースガイド](../apktool-resource-analysis.md) | +| Flutter アプリ | Dart スナップショットハッシュ、ABI ごとのライブラリダイジェスト、エンジンビルドの来歴 | 不要（静的 APK 解析）；[Flutter ガイド](../flutter-analysis.md) | +| ファームウェア | 領域、抽出結果、ネイティブ分析への引き継ぎ | Linux 上の Binwalk / Unblob；[ファームウェアガイド](../firmware-analysis.md) | +| 記録済みパケット | 選択フィールド、繰り返し値、フィルター、デコード指定 | Linux: TShark 4.x + prlimit; [キャプチャガイド](../packet-captures.md) | +| 記録済みロジック信号 | デコーダ注釈、サンプル範囲、診断情報 | Linux: sigrok-cli 0.7.2 + prlimit; [信号記録ガイド](../signal-recordings.md) | +| パッケージとリソース | ファイル一覧、ダイジェスト、plist、Apple バンドルの構造、抽出されたリソース | [アーティファクトと JavaScript のガイド](../javascript-artifact-reconstruction.md)、[Apple アプリケーション](../apple-application-analysis.md) | +| プロセスの動作 | ターミナル出力、対話、終了とファイルシステムの観測結果、実行の比較 | ネイティブ PTY を備えた Linux/macOS；[プロセスキャプチャ](../process-capture.md) | 静的な JavaScript と .NET の検査は、指定されたファイルを読み取り、アプリケーションを実行しません。実行時キャプチャは、あなたのユーザー権限で対象を実行したり、対象とやり取りしたりします。それぞれの実行時ガイドに、どのような操作が行われるかを記載しています。 -対応するネイティブ形式とホストは、プロバイダーによって異なります。[Hopper と Ghidra のセットアップ](docs/installation.md#hopper)、[IDA ガイド](docs/ida-provider.md)、[Windows 向け Ghidra の実験的対応](docs/windows-ghidra-p0.md)を参照してください。Ghidra は [16 ビット DOS 分析](docs/ghidra-dos.md)にも対応しています。プロバイダーの選択は [CLI ガイド](docs/cli.md#choose-a-provider)を参照してください。最新の npm リリース以降に追加された機能については、[リリースでの利用可否](docs/installation.md#released-package-and-main)を確認してください。 +対応するネイティブ形式とホストは、プロバイダーによって異なります。[Hopper と Ghidra のセットアップ](../installation.md#hopper)、[IDA ガイド](../ida-provider.md)、[Windows 向け Ghidra の実験的対応](../windows-ghidra-p0.md)を参照してください。Ghidra は [16 ビット DOS 分析](../ghidra-dos.md)にも対応しています。プロバイダーの選択は [CLI ガイド](../cli.md#choose-a-provider)を参照してください。最新の npm リリース以降に追加された機能については、[リリースでの利用可否](../installation.md#released-package-and-main)を確認してください。 ## 事例 -[![DX-Ball の音声パン、Notion のクリップボードブリッジ、TH04 の弾幕リングの事例を表すイラスト](docs/assets/rea-showcases.png)](https://rea.tools/ja/showcase/) +[![DX-Ball の音声パン、Notion のクリップボードブリッジ、TH04 の弾幕リングの事例を表すイラスト](../assets/rea-showcases.png)](https://rea.tools/ja/showcase/) ### DX-Ball：音声のパン計算を再構築する @@ -189,28 +189,28 @@ REA で興味深い対象を調べたら、ぜひ共有してください。[iss
どのエージェントで REA を使えますか？ -ローカル MCP サーバーに対応するエージェントなら使えます。セットアップは[対応エージェント](docs/installation.md#supported-agents)を設定します。その他のクライアントでは、[MCP の手動登録](docs/installation.md#mcp-registry)を利用できます。 +ローカル MCP サーバーに対応するエージェントなら使えます。セットアップは[対応エージェント](../installation.md#supported-agents)を設定します。その他のクライアントでは、[MCP の手動登録](../installation.md#mcp-registry)を利用できます。
Hopper、Ghidra、IDA は必要ですか？ -深いネイティブ分析には、そのいずれかを使います。静的な JavaScript と .NET の検査には、ネイティブ分析エンジンは必要ありません。セットアップは承認後に Hopper をインストールできます。Ghidra と IDA は既存のインストールを使います。[プロバイダーのセットアップ](docs/installation.md#hopper)を参照してください。 +深いネイティブ分析には、そのいずれかを使います。静的な JavaScript と .NET の検査には、ネイティブ分析エンジンは必要ありません。セットアップは承認後に Hopper をインストールできます。Ghidra と IDA は既存のインストールを使います。[プロバイダーのセットアップ](../installation.md#hopper)を参照してください。
先に Hopper を起動する必要はありますか？ -REA は必要な操作の際に Hopper を起動します。macOS では、初回起動時にデモモードを選ぶか、ライセンスを有効化するよう求められることがあります。[Hopper の起動とトラブルシューティング](docs/installation.md#launcher-paths-and-troubleshooting)を参照してください。 +REA は必要な操作の際に Hopper を起動します。macOS では、初回起動時にデモモードを選ぶか、ライセンスを有効化するよう求められることがあります。[Hopper の起動とトラブルシューティング](../installation.md#launcher-paths-and-troubleshooting)を参照してください。
skills.sh からスキルをインストールすると何ができますか？ -スキルはエージェントに調査の手順を提供します。`npx rea-agents setup` で REA の MCP サーバーを登録し、対応する手順をインストールしてから、エージェントを再起動してください。[スキルのみのインストール](docs/installation.md#skill-only-installation)を参照してください。 +スキルはエージェントに調査の手順を提供します。`npx rea-agents setup` で REA の MCP サーバーを登録し、対応する手順をインストールしてから、エージェントを再起動してください。[スキルのみのインストール](../installation.md#skill-only-installation)を参照してください。
@@ -253,14 +253,14 @@ npx rea-agents@latest setup まずはウェブサイトの[実践ガイド](https://rea.tools/ja/guides/)を参照してください。正確なオプション、前提条件、結果の契約については、以下を参照してください。 -- [インストールとセットアップ](docs/installation.md)：エージェントの登録、プロバイダーの設定、更新、アンインストール。 -- [準備状況の確認とトラブルシューティング](docs/installation.md#check-readiness-for-your-task)：特定のエージェントや分析エンジンを診断。 -- [CLI と Evidence](docs/cli.md)：コマンド、プロバイダーの選択、スナップショット、インポート／エクスポート、終了ステータス。 -- [MCP 契約](docs/mcp-contracts.md)と[エージェント向けプロンプト](docs/mcp-prompts.md)：ツールの結果、セッション、ガイド付き調査。 -- [ツールカタログ](docs/mcp-contracts.md#generated-catalog)：ビルド時に生成されるツール、プロバイダー、CLI コマンドの一覧。 -- [ロードマップ](docs/roadmap.md)：計画中の作業と機能の進捗。 +- [インストールとセットアップ](../installation.md)：エージェントの登録、プロバイダーの設定、更新、アンインストール。 +- [準備状況の確認とトラブルシューティング](../installation.md#check-readiness-for-your-task)：特定のエージェントや分析エンジンを診断。 +- [CLI と Evidence](../cli.md)：コマンド、プロバイダーの選択、スナップショット、インポート／エクスポート、終了ステータス。 +- [MCP 契約](../mcp-contracts.md)と[エージェント向けプロンプト](../mcp-prompts.md)：ツールの結果、セッション、ガイド付き調査。 +- [ツールカタログ](../mcp-contracts.md#generated-catalog)：ビルド時に生成されるツール、プロバイダー、CLI コマンドの一覧。 +- [ロードマップ](../roadmap.md)：計画中の作業と機能の進捗。 -脆弱性の報告は [SECURITY.md](SECURITY.md) に従ってください。 +脆弱性の報告は [SECURITY.md](../../SECURITY.md) に従ってください。 ## スター履歴 @@ -278,7 +278,7 @@ REA の利用、バグ報告、機能の要望、ビルドのテスト、修正 ## 免責事項 -責任ある利用については [DISCLAIMER.md](DISCLAIMER.md) を参照してください。 +責任ある利用については [DISCLAIMER.md](../../DISCLAIMER.md) を参照してください。 REA は、法令に従った研究、教育、相互運用性の確保、ソフトウェアの保存、および所有するソフトウェアや調査を許可されたソフトウェアの解析を目的としています。 @@ -290,14 +290,14 @@ applicable licenses. REA への貢献を歓迎します！[issue を作成](https://github.com/morluto/rea/issues)してバグや機能の提案を報告したり、[pull request を送信](https://github.com/morluto/rea/pulls)してコードやドキュメントを改善したりできます。 -開発環境とチェックは [CONTRIBUTING.md](CONTRIBUTING.md)、検証レーンは[テストガイド](docs/testing.md)、プロジェクト構造は[アーキテクチャ図](docs/architecture.mermaid)を参照してください。 +開発環境とチェックは [CONTRIBUTING.md](../../CONTRIBUTING.md)、検証レーンは[テストガイド](../testing.md)、プロジェクト構造は[アーキテクチャ図](../architecture.mermaid)を参照してください。 ## ライセンス -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_ko.md b/docs/readme/README_ko.md similarity index 74% rename from README_ko.md rename to docs/readme/README_ko.md index 5a9e7cd5d..26db0d824 100644 --- a/README_ko.md +++ b/docs/readme/README_ko.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · **한국어** · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · **한국어** · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA: 무엇이든 리버스 엔지니어링 @@ -9,10 +9,10 @@ **마음에 드는 기능을 발견했다면, 바이너리 수준까지 작동 원리를 이해하세요.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA가 Hopper 안에서 분석 브리지를 실행해 네이티브 바이너리를 검사하는 모습 +REA가 Hopper 안에서 분석 브리지를 실행해 네이티브 바이너리를 검사하는 모습

@@ -33,7 +33,7 @@ - Discord
+ Discord
리버스 엔지니어링 커뮤니티에 참여하세요
Discord · 질문과 답변 · 결과 공유 @@ -67,7 +67,7 @@ npx rea-agents setup 에이전트를 선택하고, 예정된 변경 사항을 검토한 뒤 승인하세요. 설정 과정은 기존 설정을 백업하고 REA의 MCP 서버와 버전에 맞는 워크플로 지침을 추가합니다. 완료 후 에이전트를 다시 시작하세요. -설정은 Claude Code, Codex, Cursor, Gemini CLI, Grok Build 및 [다른 에이전트](docs/installation.md#supported-agents)를 지원합니다. 제공자 설정과 수동 MCP 등록은 [설치 및 설정](docs/installation.md)을 참고하세요. +설정은 Claude Code, Codex, Cursor, Gemini CLI, Grok Build 및 [다른 에이전트](../installation.md#supported-agents)를 지원합니다. 제공자 설정과 수동 MCP 등록은 [설치 및 설정](../installation.md)을 참고하세요. ### 에이전트에게 요청 @@ -94,7 +94,7 @@ npm install --global rea-agents rea --help ``` -네이티브 분석에는 먼저 제공자를 설정해야 합니다. 네이티브 명령, 제공자 선택, 스냅샷, 스크립트 사용은 [CLI 및 Evidence 가이드](docs/cli.md)를 참고하세요. +네이티브 분석에는 먼저 제공자를 설정해야 합니다. 네이티브 명령, 제공자 선택, 스냅샷, 스크립트 사용은 [CLI 및 Evidence 가이드](../cli.md)를 참고하세요. ### REA 업데이트 @@ -120,9 +120,9 @@ npx rea-agents@latest setup 에이전트는 MCP를 통해 REA를 호출하여 대상을 검사하고 관련 코드를 추적합니다. REA는 조사 결과와 근거를 반환합니다. 에이전트는 이를 바탕으로 추가 질문을 하고, 동작을 설명하거나, 구현을 작성하고 테스트합니다. CLI 명령도 같은 워크플로를 사용합니다. -![REA 조사 흐름: 에이전트가 로컬 대상에 대해 질문하면 REA가 분석 도구로 대상을 검사하고 추적합니다. 에이전트는 반환된 코드, 참조, 확인되지 않은 사항을 이용해 설명하고 구현하고 테스트합니다.](website/public/assets/figures/rea-investigation-flow.svg) +![REA 조사 흐름: 에이전트가 로컬 대상에 대해 질문하면 REA가 분석 도구로 대상을 검사하고 추적합니다. 에이전트는 반환된 코드, 참조, 확인되지 않은 사항을 이용해 설명하고 구현하고 테스트합니다.](../../website/public/assets/figures/rea-investigation-flow.svg) -[원본 크기의 그림 열기](website/public/assets/figures/rea-investigation-flow.svg). +[원본 크기의 그림 열기](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -130,37 +130,37 @@ npx rea-agents@latest setup REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니다. 추가 도구와 호스트 지원은 대상에 따라 다릅니다. -| 대상 | REA가 반환하는 내용 | 요구 사항 및 가이드 | -| --------------------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -| 네이티브 바이너리 | 의사 코드, 어셈블리, 문자열, 심볼, 호출, 참조 | Hopper, Ghidra 또는 IDA; [네이티브 분석](https://rea.tools/ko/guides/native/) | -| 오프라인 ELF 레이아웃 | 섹션, 세그먼트, 원본 심볼/재배치 정보, 정적 분석으로 얻은 보안 완화 기법 후보 | Linux x64에서 호출자가 제공하는 pwntools; [바이너리 진단](docs/binary-diagnostics.md) | -| EVM 바이트코드 | 디스패치 선택자, 바이트 오프셋, 추론한 인자와 상태 변경 가능 여부 | Linux x64의 로컬 원시 바이트/16진수 입력 파일; [오프라인 EVM 가이드](docs/evm-bytecode.md) | -| 기록된 Linux 크래시 | 원시 note 레코드, 기록된 모든 스레드의 레지스터/시그널, 선택적 매핑 후보 | Linux x64에서 호출자가 제공하는 pwntools; GDB/pwndbg는 선택 사항; [기록된 크래시](docs/recorded-crashes.md) | -| JavaScript / Electron | 모듈, 임포트, 소스 맵, 라우트, IPC, 네이티브 애드온 관계 | Node.js와 npm; [애플리케이션 분석](https://rea.tools/ko/guides/javascript/) | -| 웹사이트 | 페이지 구조, 스크립트, 네트워크 관찰 결과, 요청한 스크린샷 | Chrome 계열 브라우저; [브라우저 분석](https://rea.tools/ko/guides/browser/) | -| 저장된 네트워크 캡처 | 요청, 응답, 접근 가능한 페이로드, 소스 위치 | HAR; mitmproxy 네이티브 캡처에는 Linux의 mitmdump; [캡처 가이드](docs/web-network-captures.md) | -| 패킷 캡처 | 엔드포인트, DNS, TLS SNI, 프로토콜 분포, HTTP 요청 | 호출자 제공 tshark 및 capinfos; [pcap 가이드](docs/pcap-analysis.md) | -| .NET 어셈블리 | 메타데이터, CIL 명령, 선언된 네이티브 의존성, 빌드 비교 | 정적 검사; [관리 코드 가이드](docs/managed-code-analysis.md) | -| Android APK | 매니페스트 선언, 클래스, 디컴파일된 메서드, 참조 | Linux/macOS/Windows x64의 헤드리스 JADX와 전체 JDK; [Android 가이드](docs/android-analysis.md) | -| Android 기기 | 기기, 패키지, 프로세스, 로그, 덤프, 화면 캡처, APK/파일 전송 | 호출자 제공 adb(에뮬레이터 또는 실제 기기); [ADB 가이드](docs/adb-device-analysis.md) | -| JEB 프로젝트 | 엔진 기반 프로젝트 유닛, 아티팩트 해시, 타입/메서드 의사코드 | 호출자가 시작하고 MCP를 제공하는 JEB 클라이언트(REA_JEB_MCP_URL로 선택); [JEB 가이드](docs/jeb-analysis.md) | -| Android 리소스 | 디코딩된 매니페스트, 문자열 테이블, 로캘, 버전 정보 | 호출자 제공 Apktool 런처; [리소스 가이드](docs/apktool-resource-analysis.md) | -| Flutter 앱 | Dart 스냅샷 해시, ABI별 라이브러리 해시, 엔진 빌드 출처 | 없음 (정적 APK 파싱); [Flutter 가이드](docs/flutter-analysis.md) | -| 펌웨어 | 영역, 추출 결과, 네이티브 분석으로 전달할 정보 | Linux의 Binwalk / Unblob; [펌웨어 가이드](docs/firmware-analysis.md) | -| 기록된 패킷 | 선택 필드, 반복 값, 필터 및 디코딩 규칙 | Linux: TShark 4.x + prlimit; [캡처 가이드](docs/packet-captures.md) | -| 기록된 논리 신호 | 디코더 주석, 샘플 범위, 진단 정보 | Linux: sigrok-cli 0.7.2 + prlimit; [신호 기록 가이드](docs/signal-recordings.md) | -| 패키지 및 리소스 | 파일 목록, 다이제스트, plist, Apple 번들 구조, 추출한 리소스 | [아티팩트 및 JavaScript 가이드](docs/javascript-artifact-reconstruction.md), [Apple 애플리케이션](docs/apple-application-analysis.md) | -| 프로세스 동작 | 터미널 출력, 상호작용, 종료 및 파일 시스템 관찰 결과, 실행 비교 | 네이티브 PTY를 지원하는 Linux/macOS; [프로세스 캡처](docs/process-capture.md) | +| 대상 | REA가 반환하는 내용 | 요구 사항 및 가이드 | +| --------------------- | ----------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| 네이티브 바이너리 | 의사 코드, 어셈블리, 문자열, 심볼, 호출, 참조 | Hopper, Ghidra 또는 IDA; [네이티브 분석](https://rea.tools/ko/guides/native/) | +| 오프라인 ELF 레이아웃 | 섹션, 세그먼트, 원본 심볼/재배치 정보, 정적 분석으로 얻은 보안 완화 기법 후보 | Linux x64에서 호출자가 제공하는 pwntools; [바이너리 진단](../binary-diagnostics.md) | +| EVM 바이트코드 | 디스패치 선택자, 바이트 오프셋, 추론한 인자와 상태 변경 가능 여부 | Linux x64의 로컬 원시 바이트/16진수 입력 파일; [오프라인 EVM 가이드](../evm-bytecode.md) | +| 기록된 Linux 크래시 | 원시 note 레코드, 기록된 모든 스레드의 레지스터/시그널, 선택적 매핑 후보 | Linux x64에서 호출자가 제공하는 pwntools; GDB/pwndbg는 선택 사항; [기록된 크래시](../recorded-crashes.md) | +| JavaScript / Electron | 모듈, 임포트, 소스 맵, 라우트, IPC, 네이티브 애드온 관계 | Node.js와 npm; [애플리케이션 분석](https://rea.tools/ko/guides/javascript/) | +| 웹사이트 | 페이지 구조, 스크립트, 네트워크 관찰 결과, 요청한 스크린샷 | Chrome 계열 브라우저; [브라우저 분석](https://rea.tools/ko/guides/browser/) | +| 저장된 네트워크 캡처 | 요청, 응답, 접근 가능한 페이로드, 소스 위치 | HAR; mitmproxy 네이티브 캡처에는 Linux의 mitmdump; [캡처 가이드](../web-network-captures.md) | +| 패킷 캡처 | 엔드포인트, DNS, TLS SNI, 프로토콜 분포, HTTP 요청 | 호출자 제공 tshark 및 capinfos; [pcap 가이드](../pcap-analysis.md) | +| .NET 어셈블리 | 메타데이터, CIL 명령, 선언된 네이티브 의존성, 빌드 비교 | 정적 검사; [관리 코드 가이드](../managed-code-analysis.md) | +| Android APK | 매니페스트 선언, 클래스, 디컴파일된 메서드, 참조 | Linux/macOS/Windows x64의 헤드리스 JADX와 전체 JDK; [Android 가이드](../android-analysis.md) | +| Android 기기 | 기기, 패키지, 프로세스, 로그, 덤프, 화면 캡처, APK/파일 전송 | 호출자 제공 adb(에뮬레이터 또는 실제 기기); [ADB 가이드](../adb-device-analysis.md) | +| JEB 프로젝트 | 엔진 기반 프로젝트 유닛, 아티팩트 해시, 타입/메서드 의사코드 | 호출자가 시작하고 MCP를 제공하는 JEB 클라이언트(REA_JEB_MCP_URL로 선택); [JEB 가이드](../jeb-analysis.md) | +| Android 리소스 | 디코딩된 매니페스트, 문자열 테이블, 로캘, 버전 정보 | 호출자 제공 Apktool 런처; [리소스 가이드](../apktool-resource-analysis.md) | +| Flutter 앱 | Dart 스냅샷 해시, ABI별 라이브러리 해시, 엔진 빌드 출처 | 없음 (정적 APK 파싱); [Flutter 가이드](../flutter-analysis.md) | +| 펌웨어 | 영역, 추출 결과, 네이티브 분석으로 전달할 정보 | Linux의 Binwalk / Unblob; [펌웨어 가이드](../firmware-analysis.md) | +| 기록된 패킷 | 선택 필드, 반복 값, 필터 및 디코딩 규칙 | Linux: TShark 4.x + prlimit; [캡처 가이드](../packet-captures.md) | +| 기록된 논리 신호 | 디코더 주석, 샘플 범위, 진단 정보 | Linux: sigrok-cli 0.7.2 + prlimit; [신호 기록 가이드](../signal-recordings.md) | +| 패키지 및 리소스 | 파일 목록, 다이제스트, plist, Apple 번들 구조, 추출한 리소스 | [아티팩트 및 JavaScript 가이드](../javascript-artifact-reconstruction.md), [Apple 애플리케이션](../apple-application-analysis.md) | +| 프로세스 동작 | 터미널 출력, 상호작용, 종료 및 파일 시스템 관찰 결과, 실행 비교 | 네이티브 PTY를 지원하는 Linux/macOS; [프로세스 캡처](../process-capture.md) | 정적 JavaScript 및 .NET 검사는 제공된 파일을 읽으며 애플리케이션을 실행하지 않습니다. 런타임 캡처는 사용자의 권한으로 선택한 대상을 실행하거나 대상과 상호작용합니다. 각 런타임 가이드에서 그 영향을 설명합니다. -지원하는 네이티브 형식과 호스트는 제공자마다 다릅니다. [Hopper 및 Ghidra 설정](docs/installation.md#hopper), [IDA 가이드](docs/ida-provider.md), [실험적 Windows Ghidra 지원](docs/windows-ghidra-p0.md)을 참고하세요. Ghidra는 [16비트 DOS 분석](docs/ghidra-dos.md)도 지원합니다. 제공자 선택은 [CLI 가이드](docs/cli.md#choose-a-provider)를 참고하세요. 최신 npm 릴리스 이후 추가된 기능은 [릴리스에서의 사용 가능 여부](docs/installation.md#released-package-and-main)를 확인하세요. +지원하는 네이티브 형식과 호스트는 제공자마다 다릅니다. [Hopper 및 Ghidra 설정](../installation.md#hopper), [IDA 가이드](../ida-provider.md), [실험적 Windows Ghidra 지원](../windows-ghidra-p0.md)을 참고하세요. Ghidra는 [16비트 DOS 분석](../ghidra-dos.md)도 지원합니다. 제공자 선택은 [CLI 가이드](../cli.md#choose-a-provider)를 참고하세요. 최신 npm 릴리스 이후 추가된 기능은 [릴리스에서의 사용 가능 여부](../installation.md#released-package-and-main)를 확인하세요. ## 사례 -[![DX-Ball 사운드 패닝, Notion 클립보드 브리지, TH04 탄환 고리 사례를 보여 주는 그림](docs/assets/rea-showcases.png)](https://rea.tools/ko/showcase/) +[![DX-Ball 사운드 패닝, Notion 클립보드 브리지, TH04 탄환 고리 사례를 보여 주는 그림](../assets/rea-showcases.png)](https://rea.tools/ko/showcase/) ### DX-Ball: 사운드 패닝 계산 재구현 @@ -189,28 +189,28 @@ REA로 흥미로운 대상을 분석했다면 알려 주세요. [issue](https://
어떤 에이전트에서 REA를 사용할 수 있나요? -로컬 MCP 서버를 지원하는 에이전트라면 사용할 수 있습니다. 설정 과정은 [지원하는 에이전트](docs/installation.md#supported-agents)를 구성합니다. 다른 클라이언트는 [수동 MCP 등록](docs/installation.md#mcp-registry)을 이용할 수 있습니다. +로컬 MCP 서버를 지원하는 에이전트라면 사용할 수 있습니다. 설정 과정은 [지원하는 에이전트](../installation.md#supported-agents)를 구성합니다. 다른 클라이언트는 [수동 MCP 등록](../installation.md#mcp-registry)을 이용할 수 있습니다.
Hopper, Ghidra 또는 IDA가 필요한가요? -심층 네이티브 분석에는 이 중 하나를 사용합니다. 정적 JavaScript 및 .NET 검사에는 네이티브 분석 엔진이 필요하지 않습니다. 설정 과정에서 승인 후 Hopper를 설치할 수 있으며, Ghidra와 IDA는 기존 설치를 사용합니다. [제공자 설정](docs/installation.md#hopper)을 참고하세요. +심층 네이티브 분석에는 이 중 하나를 사용합니다. 정적 JavaScript 및 .NET 검사에는 네이티브 분석 엔진이 필요하지 않습니다. 설정 과정에서 승인 후 Hopper를 설치할 수 있으며, Ghidra와 IDA는 기존 설치를 사용합니다. [제공자 설정](../installation.md#hopper)을 참고하세요.
Hopper를 먼저 실행해야 하나요? -REA는 작업에 필요할 때 Hopper를 실행합니다. macOS에서는 처음 실행할 때 데모 모드를 선택하거나 라이선스를 활성화하라는 대화 상자가 나타날 수 있습니다. [Hopper 실행 및 문제 해결](docs/installation.md#launcher-paths-and-troubleshooting)을 참고하세요. +REA는 작업에 필요할 때 Hopper를 실행합니다. macOS에서는 처음 실행할 때 데모 모드를 선택하거나 라이선스를 활성화하라는 대화 상자가 나타날 수 있습니다. [Hopper 실행 및 문제 해결](../installation.md#launcher-paths-and-troubleshooting)을 참고하세요.
skills.sh에서 설치하는 스킬은 어떤 역할을 하나요? -스킬은 에이전트에게 조사 지침을 제공합니다. `npx rea-agents setup`으로 REA의 MCP 서버를 등록하고 버전에 맞는 지침을 설치한 뒤 에이전트를 다시 시작하세요. [스킬만 설치하기](docs/installation.md#skill-only-installation)를 참고하세요. +스킬은 에이전트에게 조사 지침을 제공합니다. `npx rea-agents setup`으로 REA의 MCP 서버를 등록하고 버전에 맞는 지침을 설치한 뒤 에이전트를 다시 시작하세요. [스킬만 설치하기](../installation.md#skill-only-installation)를 참고하세요.
@@ -253,14 +253,14 @@ npx rea-agents@latest setup 웹사이트의 [실습 가이드](https://rea.tools/ko/guides/)부터 읽어 보세요. 정확한 옵션, 사전 요구 사항, 결과 계약은 다음 문서를 참고하세요. -- [설치 및 설정](docs/installation.md): 에이전트 등록, 제공자 설정, 업데이트, 제거. -- [준비 상태 확인 및 문제 해결](docs/installation.md#check-readiness-for-your-task): 특정 에이전트 또는 분석 엔진 진단. -- [CLI 및 Evidence](docs/cli.md): 명령, 제공자 선택, 스냅샷, 가져오기/내보내기, 종료 상태. -- [MCP 계약](docs/mcp-contracts.md) 및 [에이전트 프롬프트](docs/mcp-prompts.md): 도구 결과, 세션, 안내에 따른 조사. -- [도구 카탈로그](docs/mcp-contracts.md#generated-catalog): 빌드 시 생성되는 도구, 제공자, CLI 명령 목록. -- [로드맵](docs/roadmap.md): 계획된 작업과 기능 진행 상황. +- [설치 및 설정](../installation.md): 에이전트 등록, 제공자 설정, 업데이트, 제거. +- [준비 상태 확인 및 문제 해결](../installation.md#check-readiness-for-your-task): 특정 에이전트 또는 분석 엔진 진단. +- [CLI 및 Evidence](../cli.md): 명령, 제공자 선택, 스냅샷, 가져오기/내보내기, 종료 상태. +- [MCP 계약](../mcp-contracts.md) 및 [에이전트 프롬프트](../mcp-prompts.md): 도구 결과, 세션, 안내에 따른 조사. +- [도구 카탈로그](../mcp-contracts.md#generated-catalog): 빌드 시 생성되는 도구, 제공자, CLI 명령 목록. +- [로드맵](../roadmap.md): 계획된 작업과 기능 진행 상황. -취약점은 [SECURITY.md](SECURITY.md)에 따라 보고하세요. +취약점은 [SECURITY.md](../../SECURITY.md)에 따라 보고하세요. ## 스타 기록 @@ -278,7 +278,7 @@ REA를 사용하고, 버그를 보고하고, 기능을 요청하고, 빌드를 ## 면책 조항 -책임 있는 사용에 관한 안내는 [DISCLAIMER.md](DISCLAIMER.md)를 참고하세요. +책임 있는 사용에 관한 안내는 [DISCLAIMER.md](../../DISCLAIMER.md)를 참고하세요. REA는 법을 준수하는 연구, 교육, 상호운용성 확보, 소프트웨어 보존, 그리고 본인이 소유하거나 조사 권한을 가진 소프트웨어의 분석을 위한 도구입니다. @@ -290,14 +290,14 @@ applicable licenses. REA에 도움을 보태 주세요! [issue를 등록](https://github.com/morluto/rea/issues)해 버그나 기능을 제안하거나, [pull request를 보내](https://github.com/morluto/rea/pulls) 코드와 문서를 개선할 수 있습니다. -개발 환경과 검사 항목은 [CONTRIBUTING.md](CONTRIBUTING.md), 검증 절차는 [테스트 가이드](docs/testing.md), 프로젝트 구조는 [아키텍처 지도](docs/architecture.mermaid)를 참고하세요. +개발 환경과 검사 항목은 [CONTRIBUTING.md](../../CONTRIBUTING.md), 검증 절차는 [테스트 가이드](../testing.md), 프로젝트 구조는 [아키텍처 지도](../architecture.mermaid)를 참고하세요. ## 라이선스 -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_pl.md b/docs/readme/README_pl.md similarity index 76% rename from README_pl.md rename to docs/readme/README_pl.md index c54af50fe..48200087d 100644 --- a/README_pl.md +++ b/docs/readme/README_pl.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · **Polski** · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · **Polski** · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA: Reverse Engineer Anything @@ -9,10 +9,10 @@ **Widzisz funkcję, która Ci się podoba? Dowiedz się, jak działa, aż do poziomu kodu binarnego.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA uruchamia most analityczny w Hopperze podczas analizy natywnego pliku binarnego +REA uruchamia most analityczny w Hopperze podczas analizy natywnego pliku binarnego

@@ -33,7 +33,7 @@ - Discord
+ Discord
Dołącz do społeczności inżynierii wstecznej
Discord · Pytania i odpowiedzi · Pokaż swój projekt @@ -67,7 +67,7 @@ npx rea-agents setup Wybierz swoich agentów, sprawdź proponowane zmiany i zatwierdź je. Konfigurator dodaje serwer MCP REA i pasujące instrukcje pracy, tworząc kopie zapasowe istniejącej konfiguracji. Następnie uruchom agenta ponownie. -Konfigurator obsługuje Claude Code, Codex, Cursor, Gemini CLI, Grok Build i [innych agentów](docs/installation.md#supported-agents). Konfigurację dostawców analizy i ręczną rejestrację MCP opisuje przewodnik [instalacji i konfiguracji](docs/installation.md). +Konfigurator obsługuje Claude Code, Codex, Cursor, Gemini CLI, Grok Build i [innych agentów](../installation.md#supported-agents). Konfigurację dostawców analizy i ręczną rejestrację MCP opisuje przewodnik [instalacji i konfiguracji](../installation.md). ### Zapytaj agenta @@ -94,7 +94,7 @@ npm install --global rea-agents rea --help ``` -Przed analizą kodu natywnego skonfiguruj dostawcę analizy. Polecenia analizy natywnej, wybór dostawcy, migawki i użycie w skryptach opisuje [przewodnik CLI i Evidence](docs/cli.md). +Przed analizą kodu natywnego skonfiguruj dostawcę analizy. Polecenia analizy natywnej, wybór dostawcy, migawki i użycie w skryptach opisuje [przewodnik CLI i Evidence](../cli.md). ### Zaktualizuj REA @@ -120,9 +120,9 @@ Sprawdź zmiany konfiguracji i uruchom agenta ponownie. Do jednorazowych polece Twój agent wywołuje REA przez MCP, aby analizować cel i śledzić istotny kod. REA zwraca ustalenia wraz z dowodami. Agent wykorzystuje je do zadawania dalszych pytań, wyjaśniania zachowania lub pisania i testowania implementacji. Polecenia CLI korzystają z tych samych procedur. -![Przebieg badania z REA: agent pyta o lokalny cel, REA analizuje go i śledzi za pomocą narzędzi analitycznych, a agent korzysta ze zwróconego kodu, odwołań i niewiadomych, aby wyjaśnić działanie, stworzyć implementację i ją przetestować.](website/public/assets/figures/rea-investigation-flow.svg) +![Przebieg badania z REA: agent pyta o lokalny cel, REA analizuje go i śledzi za pomocą narzędzi analitycznych, a agent korzysta ze zwróconego kodu, odwołań i niewiadomych, aby wyjaśnić działanie, stworzyć implementację i ją przetestować.](../../website/public/assets/figures/rea-investigation-flow.svg) -[Otwórz ilustrację w pełnym rozmiarze](website/public/assets/figures/rea-investigation-flow.svg). +[Otwórz ilustrację w pełnym rozmiarze](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -130,37 +130,37 @@ Twój agent wywołuje REA przez MCP, aby analizować cel i śledzić istotny kod REA wymaga Node.js 22.x (>=22.19), 24.x (>=24.11) lub 26+ oraz npm. Dodatkowe narzędzia i obsługiwane platformy zależą od celu analizy: -| Cel | Co zwraca REA | Wymagania i przewodnik | -| --------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | -| Natywne pliki binarne | Pseudokod, kod asemblera, ciągi znaków, symbole, wywołania i odwołania | Hopper, Ghidra lub IDA; [analiza kodu natywnego](https://rea.tools/pl/guides/native/) | -| Układ pliku ELF w trybie offline | Sekcje, segmenty, oryginalne symbole/relokacje i potencjalne mechanizmy zabezpieczeń wskazane przez analizę statyczną | pwntools dostarczone przez użytkownika w systemie Linux x64; [diagnostyka plików binarnych](docs/binary-diagnostics.md) | -| Kod bajtowy EVM | Selektory wywołań, przesunięcia bajtowe, wywnioskowane argumenty i mutowalność | Lokalny plik z surowymi bajtami lub zapisem szesnastkowym w systemie Linux x64; [przewodnik EVM offline](docs/evm-bytecode.md) | -| Zarejestrowane awarie systemu Linux | Surowe rekordy note, rejestry/sygnały każdego zarejestrowanego wątku i opcjonalni kandydaci na mapowania | pwntools dostarczone przez użytkownika w systemie Linux x64; opcjonalnie GDB/pwndbg; [zarejestrowane awarie](docs/recorded-crashes.md) | -| JavaScript / Electron | Moduły, importy, mapy źródeł, trasy, IPC i powiązania z natywnymi dodatkami | Node.js i npm; [analiza aplikacji](https://rea.tools/pl/guides/javascript/) | -| Strony internetowe | Struktura strony, skrypty, obserwacje sieciowe i zamówione zrzuty ekranu | Przeglądarka z rodziny Chrome; [analiza w przeglądarce](https://rea.tools/pl/guides/browser/) | -| Zapisane przechwycenia ruchu sieciowego | Żądania, odpowiedzi, dostępne dane przesyłane i lokalizacje źródłowe | HAR; mitmdump w systemie Linux dla natywnych przechwyceń mitmproxy; [przewodnik przechwytywania](docs/web-network-captures.md) | -| Przechwytywanie pakietów | Punkty końcowe, DNS, TLS SNI, dystrybucja protokołów i żądania HTTP | Dostarczone przez wywołującego tshark i capinfos; [przewodnik pcap](docs/pcap-analysis.md) | -| Zestawy .NET | Metadane, instrukcje CIL, zadeklarowane zależności natywne i porównania kompilacji | Analiza statyczna; [przewodnik kodu zarządzanego](docs/managed-code-analysis.md) | -| Pakiety APK dla Androida | Deklaracje manifestu, klasy, zdekompilowane metody i odwołania | JADX bez interfejsu graficznego i pełny JDK w systemach Linux/macOS/Windows x64; [przewodnik Androida](docs/android-analysis.md) | -| Urządzenia z Androidem | Urządzenia, pakiety, procesy, logi, zrzuty, zrzuty ekranu oraz transfery APK/plików | adb dostarczony przez wywołującego (emulator lub urządzenie); [przewodnik ADB](docs/adb-device-analysis.md) | -| Projekty JEB | Jednostki projektu obsługiwane przez silnik, skróty artefaktów i pseudokod typów/metod | Uruchomiony przez wywołującego klient JEB udostępniający MCP, wybrany przez REA_JEB_MCP_URL; [przewodnik JEB](docs/jeb-analysis.md) | -| Zasoby Androida | Zdekodowany manifest, tablice ciągów, języki i informacje o wersji | Należący do wywołującego launcher Apktool; [przewodnik zasobów](docs/apktool-resource-analysis.md) | -| Aplikacje Flutter | Hash migawki Dart, sumy bibliotek per ABI i pochodzenie kompilacji silnika | Brak (statyczna analiza APK); [przewodnik Flutter](docs/flutter-analysis.md) | -| Oprogramowanie układowe | Regiony, wyniki wyodrębniania i przekazanie do analizy kodu natywnego | Binwalk / Unblob w systemie Linux; [przewodnik oprogramowania układowego](docs/firmware-analysis.md) | -| Zapisane pakiety | Wybrane pola, powtarzane wartości, filtry i reguły dekodowania | Linux: TShark 4.x + prlimit; [przewodnik po przechwyconych pakietach](docs/packet-captures.md) | -| Zapisane sygnały logiczne | Adnotacje dekodera, zakresy próbek i diagnostyka | Linux: sigrok-cli 0.7.2 + prlimit; [przewodnik po sygnałach](docs/signal-recordings.md) | -| Pakiety i zasoby | Spisy plików, skróty, pliki plist, struktura pakietów Apple i wyodrębnione zasoby | [Przewodnik artefaktów i JavaScript](docs/javascript-artifact-reconstruction.md), [aplikacje Apple](docs/apple-application-analysis.md) | -| Zachowanie procesów | Wyjście terminala, interakcje, obserwacje zakończenia i systemu plików oraz porównania uruchomień | Linux/macOS z natywnym PTY; [przechwytywanie procesów](docs/process-capture.md) | +| Cel | Co zwraca REA | Wymagania i przewodnik | +| --------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | +| Natywne pliki binarne | Pseudokod, kod asemblera, ciągi znaków, symbole, wywołania i odwołania | Hopper, Ghidra lub IDA; [analiza kodu natywnego](https://rea.tools/pl/guides/native/) | +| Układ pliku ELF w trybie offline | Sekcje, segmenty, oryginalne symbole/relokacje i potencjalne mechanizmy zabezpieczeń wskazane przez analizę statyczną | pwntools dostarczone przez użytkownika w systemie Linux x64; [diagnostyka plików binarnych](../binary-diagnostics.md) | +| Kod bajtowy EVM | Selektory wywołań, przesunięcia bajtowe, wywnioskowane argumenty i mutowalność | Lokalny plik z surowymi bajtami lub zapisem szesnastkowym w systemie Linux x64; [przewodnik EVM offline](../evm-bytecode.md) | +| Zarejestrowane awarie systemu Linux | Surowe rekordy note, rejestry/sygnały każdego zarejestrowanego wątku i opcjonalni kandydaci na mapowania | pwntools dostarczone przez użytkownika w systemie Linux x64; opcjonalnie GDB/pwndbg; [zarejestrowane awarie](../recorded-crashes.md) | +| JavaScript / Electron | Moduły, importy, mapy źródeł, trasy, IPC i powiązania z natywnymi dodatkami | Node.js i npm; [analiza aplikacji](https://rea.tools/pl/guides/javascript/) | +| Strony internetowe | Struktura strony, skrypty, obserwacje sieciowe i zamówione zrzuty ekranu | Przeglądarka z rodziny Chrome; [analiza w przeglądarce](https://rea.tools/pl/guides/browser/) | +| Zapisane przechwycenia ruchu sieciowego | Żądania, odpowiedzi, dostępne dane przesyłane i lokalizacje źródłowe | HAR; mitmdump w systemie Linux dla natywnych przechwyceń mitmproxy; [przewodnik przechwytywania](../web-network-captures.md) | +| Przechwytywanie pakietów | Punkty końcowe, DNS, TLS SNI, dystrybucja protokołów i żądania HTTP | Dostarczone przez wywołującego tshark i capinfos; [przewodnik pcap](../pcap-analysis.md) | +| Zestawy .NET | Metadane, instrukcje CIL, zadeklarowane zależności natywne i porównania kompilacji | Analiza statyczna; [przewodnik kodu zarządzanego](../managed-code-analysis.md) | +| Pakiety APK dla Androida | Deklaracje manifestu, klasy, zdekompilowane metody i odwołania | JADX bez interfejsu graficznego i pełny JDK w systemach Linux/macOS/Windows x64; [przewodnik Androida](../android-analysis.md) | +| Urządzenia z Androidem | Urządzenia, pakiety, procesy, logi, zrzuty, zrzuty ekranu oraz transfery APK/plików | adb dostarczony przez wywołującego (emulator lub urządzenie); [przewodnik ADB](../adb-device-analysis.md) | +| Projekty JEB | Jednostki projektu obsługiwane przez silnik, skróty artefaktów i pseudokod typów/metod | Uruchomiony przez wywołującego klient JEB udostępniający MCP, wybrany przez REA_JEB_MCP_URL; [przewodnik JEB](../jeb-analysis.md) | +| Zasoby Androida | Zdekodowany manifest, tablice ciągów, języki i informacje o wersji | Należący do wywołującego launcher Apktool; [przewodnik zasobów](../apktool-resource-analysis.md) | +| Aplikacje Flutter | Hash migawki Dart, sumy bibliotek per ABI i pochodzenie kompilacji silnika | Brak (statyczna analiza APK); [przewodnik Flutter](../flutter-analysis.md) | +| Oprogramowanie układowe | Regiony, wyniki wyodrębniania i przekazanie do analizy kodu natywnego | Binwalk / Unblob w systemie Linux; [przewodnik oprogramowania układowego](../firmware-analysis.md) | +| Zapisane pakiety | Wybrane pola, powtarzane wartości, filtry i reguły dekodowania | Linux: TShark 4.x + prlimit; [przewodnik po przechwyconych pakietach](../packet-captures.md) | +| Zapisane sygnały logiczne | Adnotacje dekodera, zakresy próbek i diagnostyka | Linux: sigrok-cli 0.7.2 + prlimit; [przewodnik po sygnałach](../signal-recordings.md) | +| Pakiety i zasoby | Spisy plików, skróty, pliki plist, struktura pakietów Apple i wyodrębnione zasoby | [Przewodnik artefaktów i JavaScript](../javascript-artifact-reconstruction.md), [aplikacje Apple](../apple-application-analysis.md) | +| Zachowanie procesów | Wyjście terminala, interakcje, obserwacje zakończenia i systemu plików oraz porównania uruchomień | Linux/macOS z natywnym PTY; [przechwytywanie procesów](../process-capture.md) | Statyczna analiza JavaScript i .NET odczytuje dostarczone pliki bez uruchamiania aplikacji. Przechwytywanie w czasie wykonywania uruchamia wybrany cel lub wchodzi z nim w interakcje z uprawnieniami Twojego użytkownika; skutki opisano w odpowiednich przewodnikach. -Obsługiwane formaty natywne i platformy zależą od dostawcy analizy. Zobacz [konfigurację Hoppera i Ghidry](docs/installation.md#hopper), [przewodnik IDA](docs/ida-provider.md) oraz [eksperymentalną obsługę Ghidry w systemie Windows](docs/windows-ghidra-p0.md). Ghidra obsługuje też [analizę 16-bitowego DOS-a](docs/ghidra-dos.md). Wybór dostawcy opisuje [przewodnik CLI](docs/cli.md#choose-a-provider). Sprawdź [dostępność w wydaniach](docs/installation.md#released-package-and-main) dla funkcji dodanych po najnowszym wydaniu npm. +Obsługiwane formaty natywne i platformy zależą od dostawcy analizy. Zobacz [konfigurację Hoppera i Ghidry](../installation.md#hopper), [przewodnik IDA](../ida-provider.md) oraz [eksperymentalną obsługę Ghidry w systemie Windows](../windows-ghidra-p0.md). Ghidra obsługuje też [analizę 16-bitowego DOS-a](../ghidra-dos.md). Wybór dostawcy opisuje [przewodnik CLI](../cli.md#choose-a-provider). Sprawdź [dostępność w wydaniach](../installation.md#released-package-and-main) dla funkcji dodanych po najnowszym wydaniu npm. ## Przykłady zastosowań -[![Ilustracje studiów przypadku panoramowania dźwięku w DX-Ball, mostka schowka w Notion i pierścienia pocisków w TH04](docs/assets/rea-showcases.png)](https://rea.tools/pl/showcase/) +[![Ilustracje studiów przypadku panoramowania dźwięku w DX-Ball, mostka schowka w Notion i pierścienia pocisków w TH04](../assets/rea-showcases.png)](https://rea.tools/pl/showcase/) ### DX-Ball: odtworzenie obliczania panoramy dźwięku @@ -187,28 +187,28 @@ Jeśli użyłeś REA do czegoś ciekawego, chętnie to zobaczymy. Opisz swój pr
Którzy agenci mogą korzystać z REA? -Każdy agent obsługujący lokalne serwery MCP. Konfigurator konfiguruje [obsługiwanych agentów](docs/installation.md#supported-agents); inne klienty mogą użyć [ręcznej rejestracji MCP](docs/installation.md#mcp-registry). +Każdy agent obsługujący lokalne serwery MCP. Konfigurator konfiguruje [obsługiwanych agentów](../installation.md#supported-agents); inne klienty mogą użyć [ręcznej rejestracji MCP](../installation.md#mcp-registry).
Czy potrzebuję Hoppera, Ghidry lub IDA? -Dogłębna analiza kodu natywnego używa jednego z nich. Statyczna analiza JavaScript i .NET działa bez silnika analizy natywnej. Konfigurator może zainstalować Hoppera po uzyskaniu zgody; Ghidra i IDA korzystają z istniejących instalacji. Zobacz [konfigurację dostawców analizy](docs/installation.md#hopper). +Dogłębna analiza kodu natywnego używa jednego z nich. Statyczna analiza JavaScript i .NET działa bez silnika analizy natywnej. Konfigurator może zainstalować Hoppera po uzyskaniu zgody; Ghidra i IDA korzystają z istniejących instalacji. Zobacz [konfigurację dostawców analizy](../installation.md#hopper).
Czy muszę najpierw uruchomić Hoppera? -REA uruchamia Hoppera, gdy wymaga tego operacja. W systemie macOS przy pierwszym uruchomieniu okno dialogowe może poprosić o wybór trybu demonstracyjnego lub aktywację licencji. Zobacz [uruchamianie Hoppera i rozwiązywanie problemów](docs/installation.md#launcher-paths-and-troubleshooting). +REA uruchamia Hoppera, gdy wymaga tego operacja. W systemie macOS przy pierwszym uruchomieniu okno dialogowe może poprosić o wybór trybu demonstracyjnego lub aktywację licencji. Zobacz [uruchamianie Hoppera i rozwiązywanie problemów](../installation.md#launcher-paths-and-troubleshooting).
Co daje zainstalowanie umiejętności z skills.sh? -Umiejętność dostarcza agentowi instrukcje prowadzenia analizy. Użyj `npx rea-agents setup`, aby zarejestrować serwer MCP REA i zainstalować pasujące instrukcje, a następnie uruchom agenta ponownie. Zobacz [instalację samej umiejętności](docs/installation.md#skill-only-installation). +Umiejętność dostarcza agentowi instrukcje prowadzenia analizy. Użyj `npx rea-agents setup`, aby zarejestrować serwer MCP REA i zainstalować pasujące instrukcje, a następnie uruchom agenta ponownie. Zobacz [instalację samej umiejętności](../installation.md#skill-only-installation).
@@ -251,14 +251,14 @@ Jeśli używasz agenta, wykonaj [odświeżenie konfiguracji](#zaktualizuj-rea) i Zacznij od [przewodników z przykładami](https://rea.tools/pl/guides/) na stronie internetowej. Dokładne opcje, wymagania wstępne i kontrakty wyników opisano tutaj: -- [Instalacja i konfiguracja](docs/installation.md): rejestracja agentów, konfiguracja dostawców analizy, aktualizacje i odinstalowanie. -- [Gotowość i rozwiązywanie problemów](docs/installation.md#check-readiness-for-your-task): diagnostyka jednego agenta lub silnika analizy. -- [CLI i Evidence](docs/cli.md): polecenia, wybór dostawcy, migawki, import/eksport i kody zakończenia. -- [Kontrakty MCP](docs/mcp-contracts.md) i [prompty dla agentów](docs/mcp-prompts.md): wyniki narzędzi, sesje i prowadzone analizy. -- [Katalog narzędzi](docs/mcp-contracts.md#generated-catalog): generowany podczas kompilacji wykaz narzędzi, dostawców i poleceń CLI. -- [Plan rozwoju](docs/roadmap.md): planowane prace i śledzenie funkcjonalności. +- [Instalacja i konfiguracja](../installation.md): rejestracja agentów, konfiguracja dostawców analizy, aktualizacje i odinstalowanie. +- [Gotowość i rozwiązywanie problemów](../installation.md#check-readiness-for-your-task): diagnostyka jednego agenta lub silnika analizy. +- [CLI i Evidence](../cli.md): polecenia, wybór dostawcy, migawki, import/eksport i kody zakończenia. +- [Kontrakty MCP](../mcp-contracts.md) i [prompty dla agentów](../mcp-prompts.md): wyniki narzędzi, sesje i prowadzone analizy. +- [Katalog narzędzi](../mcp-contracts.md#generated-catalog): generowany podczas kompilacji wykaz narzędzi, dostawców i poleceń CLI. +- [Plan rozwoju](../roadmap.md): planowane prace i śledzenie funkcjonalności. -Zgłaszaj podatności zgodnie z [SECURITY.md](SECURITY.md). +Zgłaszaj podatności zgodnie z [SECURITY.md](../../SECURITY.md). ## Historia gwiazdek @@ -276,7 +276,7 @@ Dziękujemy wszystkim, którzy używają REA, zgłaszają błędy, proponują no ## Zastrzeżenie -Wskazówki dotyczące odpowiedzialnego korzystania znajdziesz w [DISCLAIMER.md](DISCLAIMER.md). +Wskazówki dotyczące odpowiedzialnego korzystania znajdziesz w [DISCLAIMER.md](../../DISCLAIMER.md). REA służy zgodnym z prawem badaniom, edukacji, zapewnianiu interoperacyjności, zachowaniu oprogramowania oraz analizie oprogramowania, które posiadasz lub masz prawo badać. @@ -288,14 +288,14 @@ applicable licenses. Chętnie przyjmiemy Twoją pomoc w rozwoju REA! [Otwórz zgłoszenie](https://github.com/morluto/rea/issues), aby poinformować o błędzie lub zaproponować funkcję, albo [wyślij pull request](https://github.com/morluto/rea/pulls) z ulepszeniami kodu lub dokumentacji. -Zobacz [CONTRIBUTING.md](CONTRIBUTING.md), aby poznać konfigurację środowiska programistycznego i kontrole, [testowanie](docs/testing.md), aby poznać ścieżki weryfikacji, oraz [mapę architektury](docs/architecture.mermaid), aby poznać strukturę projektu. +Zobacz [CONTRIBUTING.md](../../CONTRIBUTING.md), aby poznać konfigurację środowiska programistycznego i kontrole, [testowanie](../testing.md), aby poznać ścieżki weryfikacji, oraz [mapę architektury](../architecture.mermaid), aby poznać strukturę projektu. ## Licencja -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_pt-BR.md b/docs/readme/README_pt-BR.md similarity index 75% rename from README_pt-BR.md rename to docs/readme/README_pt-BR.md index dd138f7ff..6c3f34e35 100644 --- a/README_pt-BR.md +++ b/docs/readme/README_pt-BR.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · **Português (Brasil)** · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · **Português (Brasil)** · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA: Faça engenharia reversa de qualquer coisa @@ -9,10 +9,10 @@ **Viu uma funcionalidade de que gostou? Entenda como ela funciona, até o nível binário.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA inicia sua ponte de análise dentro do Hopper enquanto inspeciona um binário nativo +REA inicia sua ponte de análise dentro do Hopper enquanto inspeciona um binário nativo

@@ -33,7 +33,7 @@ - Discord
+ Discord
Participe da comunidade de engenharia reversa
Discord · Perguntas e respostas · Compartilhe seus resultados @@ -67,7 +67,7 @@ npx rea-agents setup Escolha seus agentes, revise as alterações propostas e aprove-as. A configuração adiciona o servidor MCP do REA e as instruções de fluxo de trabalho correspondentes, com backups das configurações existentes. Reinicie seu agente depois. -A configuração oferece suporte ao Claude Code, Codex, Cursor, Gemini CLI, Grok Build e [outros agentes](docs/installation.md#supported-agents). Consulte [instalação e configuração](docs/installation.md) para configurar provedores e registrar o MCP manualmente. +A configuração oferece suporte ao Claude Code, Codex, Cursor, Gemini CLI, Grok Build e [outros agentes](../installation.md#supported-agents). Consulte [instalação e configuração](../installation.md) para configurar provedores e registrar o MCP manualmente. ### Peça ao seu agente @@ -94,7 +94,7 @@ npm install --global rea-agents rea --help ``` -Para análise nativa, configure primeiro um provedor. Consulte o [guia de CLI e Evidence](docs/cli.md) para comandos nativos, seleção de provedores, snapshots e uso em scripts. +Para análise nativa, configure primeiro um provedor. Consulte o [guia de CLI e Evidence](../cli.md) para comandos nativos, seleção de provedores, snapshots e uso em scripts. ### Atualize o REA @@ -120,9 +120,9 @@ Revise as alterações de configuração e reinicie seu agente. Para comandos av Seu agente chama o REA por MCP para inspecionar o alvo e rastrear o código relevante. O REA retorna os resultados com suas evidências. O agente usa esses resultados para fazer novas perguntas, explicar o comportamento ou escrever e testar uma implementação. Os comandos da CLI usam os mesmos fluxos de trabalho. -![Fluxo de investigação do REA: seu agente pergunta sobre um alvo local, o REA o inspeciona e rastreia com ferramentas de análise, e o agente usa o código, as referências e os pontos desconhecidos retornados para explicar, implementar e testar.](website/public/assets/figures/rea-investigation-flow.svg) +![Fluxo de investigação do REA: seu agente pergunta sobre um alvo local, o REA o inspeciona e rastreia com ferramentas de análise, e o agente usa o código, as referências e os pontos desconhecidos retornados para explicar, implementar e testar.](../../website/public/assets/figures/rea-investigation-flow.svg) -[Abra a figura em tamanho completo](website/public/assets/figures/rea-investigation-flow.svg). +[Abra a figura em tamanho completo](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -130,37 +130,37 @@ Seu agente chama o REA por MCP para inspecionar o alvo e rastrear o código rele O REA requer Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, além do npm. As ferramentas adicionais e os sistemas anfitriões compatíveis dependem do alvo: -| Alvo | O que o REA retorna | Requisitos e guia | -| --------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -| Binários nativos | Pseudocódigo, assembly, strings, símbolos, chamadas e referências | Hopper, Ghidra ou IDA; [análise nativa](https://rea.tools/pt-BR/guides/native/) | -| Estrutura ELF sem execução | Seções, segmentos, símbolos/relocações originais e possíveis mecanismos de mitigação identificados por análise estática | pwntools fornecido pelo chamador no Linux x64; [diagnóstico de binários](docs/binary-diagnostics.md) | -| Bytecode EVM | Seletores de despacho, offsets de bytes, argumentos inferidos e mutabilidade | Entrada local com bytes brutos ou representação hexadecimal no Linux x64; [guia de EVM sem execução](docs/evm-bytecode.md) | -| Falhas do Linux registradas | Registros note brutos, registradores/sinais de cada thread registrada e candidatos opcionais a mapeamentos | pwntools fornecido pelo chamador no Linux x64; GDB/pwndbg opcionais; [falhas registradas](docs/recorded-crashes.md) | -| JavaScript / Electron | Módulos, importações, source maps, rotas, IPC e relações com complementos nativos | Node.js e npm; [análise de aplicativos](https://rea.tools/pt-BR/guides/javascript/) | -| Sites | Estrutura da página, scripts, observações de rede e capturas de tela solicitadas | Um navegador da família Chrome; [análise do navegador](https://rea.tools/pt-BR/guides/browser/) | -| Capturas de rede salvas | Requisições, respostas, payloads acessíveis e localizações de origem | HAR; mitmdump no Linux para capturas no formato nativo do mitmproxy; [guia de capturas](docs/web-network-captures.md) | -| Capturas de pacotes | Endpoints, DNS, TLS SNI, distribuição de protocolos e requisições HTTP | tshark e capinfos fornecidos pelo chamador; [guia pcap](docs/pcap-analysis.md) | -| Assemblies .NET | Metadados, instruções CIL, dependências nativas declaradas e comparações de builds | Inspeção estática; [guia de código gerenciado](docs/managed-code-analysis.md) | -| APKs Android | Declarações do manifesto, classes, métodos descompilados e referências | JADX sem interface gráfica e um JDK completo no Linux/macOS/Windows x64; [guia de Android](docs/android-analysis.md) | -| Dispositivos Android | Dispositivos, pacotes, processos, logs, despejos, capturas de tela e transferências de APK/arquivos | adb fornecido pelo chamador (emulador ou aparelho); [guia de ADB](docs/adb-device-analysis.md) | -| Projetos JEB | Unidades de projeto do motor, hashes de artefatos e pseudocódigo de tipos/métodos | Um cliente JEB iniciado pelo chamador servindo MCP, selecionado com REA_JEB_MCP_URL; [guia de JEB](docs/jeb-analysis.md) | -| Recursos do Android | Manifesto decodificado, tabelas de strings, idiomas e fatos de versão | Lançador do Apktool fornecido pelo chamador; [guia de recursos](docs/apktool-resource-analysis.md) | -| Aplicativos Flutter | Hash de snapshot Dart, hashes de bibliotecas por ABI e procedência de build do motor | Nenhuma (análise estática de APK); [guia Flutter](docs/flutter-analysis.md) | -| Firmware | Regiões, resultados de extração e encaminhamentos à análise nativa | Binwalk / Unblob no Linux; [guia de firmware](docs/firmware-analysis.md) | -| Pacotes gravados | Campos selecionados, valores repetidos, filtros e regras de decodificação | Linux: TShark 4.x + prlimit; [guia de capturas](docs/packet-captures.md) | -| Sinais lógicos gravados | Anotações do decodificador, intervalos de amostras e diagnósticos | Linux: sigrok-cli 0.7.2 + prlimit; [guia de sinais](docs/signal-recordings.md) | -| Pacotes e recursos | Inventários de arquivos, digests, plists, estrutura de bundles Apple e recursos extraídos | [Guia de artefatos e JavaScript](docs/javascript-artifact-reconstruction.md), [aplicativos Apple](docs/apple-application-analysis.md) | -| Comportamento de processos | Saída do terminal, interações, observações de encerramento e do sistema de arquivos, e comparações entre execuções | Linux/macOS com PTY nativo; [captura de processos](docs/process-capture.md) | +| Alvo | O que o REA retorna | Requisitos e guia | +| --------------------------- | ----------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| Binários nativos | Pseudocódigo, assembly, strings, símbolos, chamadas e referências | Hopper, Ghidra ou IDA; [análise nativa](https://rea.tools/pt-BR/guides/native/) | +| Estrutura ELF sem execução | Seções, segmentos, símbolos/relocações originais e possíveis mecanismos de mitigação identificados por análise estática | pwntools fornecido pelo chamador no Linux x64; [diagnóstico de binários](../binary-diagnostics.md) | +| Bytecode EVM | Seletores de despacho, offsets de bytes, argumentos inferidos e mutabilidade | Entrada local com bytes brutos ou representação hexadecimal no Linux x64; [guia de EVM sem execução](../evm-bytecode.md) | +| Falhas do Linux registradas | Registros note brutos, registradores/sinais de cada thread registrada e candidatos opcionais a mapeamentos | pwntools fornecido pelo chamador no Linux x64; GDB/pwndbg opcionais; [falhas registradas](../recorded-crashes.md) | +| JavaScript / Electron | Módulos, importações, source maps, rotas, IPC e relações com complementos nativos | Node.js e npm; [análise de aplicativos](https://rea.tools/pt-BR/guides/javascript/) | +| Sites | Estrutura da página, scripts, observações de rede e capturas de tela solicitadas | Um navegador da família Chrome; [análise do navegador](https://rea.tools/pt-BR/guides/browser/) | +| Capturas de rede salvas | Requisições, respostas, payloads acessíveis e localizações de origem | HAR; mitmdump no Linux para capturas no formato nativo do mitmproxy; [guia de capturas](../web-network-captures.md) | +| Capturas de pacotes | Endpoints, DNS, TLS SNI, distribuição de protocolos e requisições HTTP | tshark e capinfos fornecidos pelo chamador; [guia pcap](../pcap-analysis.md) | +| Assemblies .NET | Metadados, instruções CIL, dependências nativas declaradas e comparações de builds | Inspeção estática; [guia de código gerenciado](../managed-code-analysis.md) | +| APKs Android | Declarações do manifesto, classes, métodos descompilados e referências | JADX sem interface gráfica e um JDK completo no Linux/macOS/Windows x64; [guia de Android](../android-analysis.md) | +| Dispositivos Android | Dispositivos, pacotes, processos, logs, despejos, capturas de tela e transferências de APK/arquivos | adb fornecido pelo chamador (emulador ou aparelho); [guia de ADB](../adb-device-analysis.md) | +| Projetos JEB | Unidades de projeto do motor, hashes de artefatos e pseudocódigo de tipos/métodos | Um cliente JEB iniciado pelo chamador servindo MCP, selecionado com REA_JEB_MCP_URL; [guia de JEB](../jeb-analysis.md) | +| Recursos do Android | Manifesto decodificado, tabelas de strings, idiomas e fatos de versão | Lançador do Apktool fornecido pelo chamador; [guia de recursos](../apktool-resource-analysis.md) | +| Aplicativos Flutter | Hash de snapshot Dart, hashes de bibliotecas por ABI e procedência de build do motor | Nenhuma (análise estática de APK); [guia Flutter](../flutter-analysis.md) | +| Firmware | Regiões, resultados de extração e encaminhamentos à análise nativa | Binwalk / Unblob no Linux; [guia de firmware](../firmware-analysis.md) | +| Pacotes gravados | Campos selecionados, valores repetidos, filtros e regras de decodificação | Linux: TShark 4.x + prlimit; [guia de capturas](../packet-captures.md) | +| Sinais lógicos gravados | Anotações do decodificador, intervalos de amostras e diagnósticos | Linux: sigrok-cli 0.7.2 + prlimit; [guia de sinais](../signal-recordings.md) | +| Pacotes e recursos | Inventários de arquivos, digests, plists, estrutura de bundles Apple e recursos extraídos | [Guia de artefatos e JavaScript](../javascript-artifact-reconstruction.md), [aplicativos Apple](../apple-application-analysis.md) | +| Comportamento de processos | Saída do terminal, interações, observações de encerramento e do sistema de arquivos, e comparações entre execuções | Linux/macOS com PTY nativo; [captura de processos](../process-capture.md) | A inspeção estática de JavaScript e .NET lê os arquivos fornecidos sem executar o aplicativo. A captura em execução inicia o alvo selecionado ou interage com ele usando as permissões do seu usuário; cada guia descreve seus efeitos. -Os formatos nativos e os sistemas anfitriões compatíveis variam por provedor. Consulte [configuração do Hopper e Ghidra](docs/installation.md#hopper), o [guia do IDA](docs/ida-provider.md) e o [suporte experimental ao Ghidra no Windows](docs/windows-ghidra-p0.md). O Ghidra também oferece [análise de DOS de 16 bits](docs/ghidra-dos.md). Para selecionar um provedor, consulte o [guia da CLI](docs/cli.md#choose-a-provider). Verifique a [disponibilidade nas versões](docs/installation.md#released-package-and-main) das funcionalidades adicionadas após a versão mais recente no npm. +Os formatos nativos e os sistemas anfitriões compatíveis variam por provedor. Consulte [configuração do Hopper e Ghidra](../installation.md#hopper), o [guia do IDA](../ida-provider.md) e o [suporte experimental ao Ghidra no Windows](../windows-ghidra-p0.md). O Ghidra também oferece [análise de DOS de 16 bits](../ghidra-dos.md). Para selecionar um provedor, consulte o [guia da CLI](../cli.md#choose-a-provider). Verifique a [disponibilidade nas versões](../installation.md#released-package-and-main) das funcionalidades adicionadas após a versão mais recente no npm. ## Casos de uso -[![Ilustrações dos estudos de panoramização de áudio do DX-Ball, da ponte da área de transferência do Notion e do anel de projéteis do TH04](docs/assets/rea-showcases.png)](https://rea.tools/pt-BR/showcase/) +[![Ilustrações dos estudos de panoramização de áudio do DX-Ball, da ponte da área de transferência do Notion e do anel de projéteis do TH04](../assets/rea-showcases.png)](https://rea.tools/pt-BR/showcase/) ### DX-Ball: reconstruir um cálculo de panorâmica de áudio @@ -187,28 +187,28 @@ Se você usou o REA em algo interessante, gostaríamos de conhecer. Compartilhe
Quais agentes podem usar o REA? -Qualquer agente que ofereça suporte a servidores MCP locais. A configuração prepara os [agentes compatíveis](docs/installation.md#supported-agents); outros clientes podem usar o [registro manual do MCP](docs/installation.md#mcp-registry). +Qualquer agente que ofereça suporte a servidores MCP locais. A configuração prepara os [agentes compatíveis](../installation.md#supported-agents); outros clientes podem usar o [registro manual do MCP](../installation.md#mcp-registry).
Preciso do Hopper, Ghidra ou IDA? -A análise nativa aprofundada usa um desses motores. A inspeção estática de JavaScript e .NET funciona sem um motor de análise nativa. A configuração pode instalar o Hopper após aprovação; Ghidra e IDA usam as instalações existentes. Consulte [configuração dos provedores](docs/installation.md#hopper). +A análise nativa aprofundada usa um desses motores. A inspeção estática de JavaScript e .NET funciona sem um motor de análise nativa. A configuração pode instalar o Hopper após aprovação; Ghidra e IDA usam as instalações existentes. Consulte [configuração dos provedores](../installation.md#hopper).
Preciso iniciar o Hopper antes? -O REA inicia o Hopper quando uma operação precisa dele. No macOS, um diálogo na primeira execução pode pedir que você escolha o modo de demonstração ou ative a licença. Consulte [inicialização do Hopper e solução de problemas](docs/installation.md#launcher-paths-and-troubleshooting). +O REA inicia o Hopper quando uma operação precisa dele. No macOS, um diálogo na primeira execução pode pedir que você escolha o modo de demonstração ou ative a licença. Consulte [inicialização do Hopper e solução de problemas](../installation.md#launcher-paths-and-troubleshooting).
O que faz a instalação da skill pelo skills.sh? -A skill fornece instruções de investigação ao seu agente. Use `npx rea-agents setup` para registrar o servidor MCP do REA e instalar as instruções correspondentes, depois reinicie seu agente. Consulte [instalação apenas da skill](docs/installation.md#skill-only-installation). +A skill fornece instruções de investigação ao seu agente. Use `npx rea-agents setup` para registrar o servidor MCP do REA e instalar as instruções correspondentes, depois reinicie seu agente. Consulte [instalação apenas da skill](../installation.md#skill-only-installation).
@@ -251,14 +251,14 @@ Se você usa um agente, conclua a [atualização da configuração](#atualize-o- Comece pelos [guias práticos](https://rea.tools/pt-BR/guides/) do site. Para opções exatas, pré-requisitos e contratos dos resultados: -- [Instalação e configuração](docs/installation.md): registro de agentes, configuração de provedores, atualizações e desinstalação. -- [Prontidão e solução de problemas](docs/installation.md#check-readiness-for-your-task): diagnóstico de um agente ou motor de análise específico. -- [CLI e Evidence](docs/cli.md): comandos, seleção de provedores, snapshots, importação/exportação e status de saída. -- [Contratos MCP](docs/mcp-contracts.md) e [prompts para agentes](docs/mcp-prompts.md): resultados das ferramentas, sessões e investigações guiadas. -- [Catálogo de ferramentas](docs/mcp-contracts.md#generated-catalog): inventário de ferramentas, provedores e comandos da CLI gerado durante o build. -- [Roadmap](docs/roadmap.md): trabalho planejado e acompanhamento das capacidades. +- [Instalação e configuração](../installation.md): registro de agentes, configuração de provedores, atualizações e desinstalação. +- [Prontidão e solução de problemas](../installation.md#check-readiness-for-your-task): diagnóstico de um agente ou motor de análise específico. +- [CLI e Evidence](../cli.md): comandos, seleção de provedores, snapshots, importação/exportação e status de saída. +- [Contratos MCP](../mcp-contracts.md) e [prompts para agentes](../mcp-prompts.md): resultados das ferramentas, sessões e investigações guiadas. +- [Catálogo de ferramentas](../mcp-contracts.md#generated-catalog): inventário de ferramentas, provedores e comandos da CLI gerado durante o build. +- [Roadmap](../roadmap.md): trabalho planejado e acompanhamento das capacidades. -Relate vulnerabilidades conforme [SECURITY.md](SECURITY.md). +Relate vulnerabilidades conforme [SECURITY.md](../../SECURITY.md). ## Histórico de estrelas @@ -276,7 +276,7 @@ Obrigado a todos que usam o REA, relatam bugs, sugerem funcionalidades, testam b ## Aviso legal -Consulte [DISCLAIMER.md](DISCLAIMER.md) para orientações sobre o uso responsável. +Consulte [DISCLAIMER.md](../../DISCLAIMER.md) para orientações sobre o uso responsável. O REA destina-se à pesquisa, à educação, à interoperabilidade, à preservação de software e à análise de software que você possui ou está autorizado a examinar, dentro da lei. @@ -288,14 +288,14 @@ applicable licenses. Sua ajuda com o REA é bem-vinda! [Abra uma issue](https://github.com/morluto/rea/issues) para relatar um bug ou sugerir uma funcionalidade, ou [envie um pull request](https://github.com/morluto/rea/pulls) para melhorar o código ou a documentação. -Consulte [CONTRIBUTING.md](CONTRIBUTING.md) para configurar o ambiente de desenvolvimento e as verificações, o [guia de testes](docs/testing.md) para os fluxos de verificação e o [mapa da arquitetura](docs/architecture.mermaid) para a estrutura do projeto. +Consulte [CONTRIBUTING.md](../../CONTRIBUTING.md) para configurar o ambiente de desenvolvimento e as verificações, o [guia de testes](../testing.md) para os fluxos de verificação e o [mapa da arquitetura](../architecture.mermaid) para a estrutura do projeto. ## Licença -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_ru.md b/docs/readme/README_ru.md similarity index 80% rename from README_ru.md rename to docs/readme/README_ru.md index 34e7d3af2..744e26f6c 100644 --- a/README_ru.md +++ b/docs/readme/README_ru.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · **Русский** · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · **Русский** · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA: Реверс-инжиниринг чего угодно @@ -9,10 +9,10 @@ **Нашли интересную функцию? Разберитесь, как она работает, вплоть до двоичного кода.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA запускает мост анализа внутри Hopper при исследовании нативного бинарного файла +REA запускает мост анализа внутри Hopper при исследовании нативного бинарного файла

@@ -33,7 +33,7 @@ - Discord
+ Discord
Присоединяйтесь к сообществу реверс-инжиниринга
Discord · Вопросы и ответы · Обмен результатами @@ -67,7 +67,7 @@ npx rea-agents setup Выберите агентов, проверьте предлагаемые изменения и подтвердите их. Настройка добавляет MCP-сервер REA и соответствующие рабочие инструкции, сохраняя резервные копии существующих настроек. После этого перезапустите агента. -Настройка поддерживает Claude Code, Codex, Cursor, Gemini CLI, Grok Build и [других агентов](docs/installation.md#supported-agents). Настройка провайдеров и ручная регистрация MCP описаны в разделе [установка и настройка](docs/installation.md). +Настройка поддерживает Claude Code, Codex, Cursor, Gemini CLI, Grok Build и [других агентов](../installation.md#supported-agents). Настройка провайдеров и ручная регистрация MCP описаны в разделе [установка и настройка](../installation.md). ### Задайте вопрос агенту @@ -94,7 +94,7 @@ npm install --global rea-agents rea --help ``` -Для нативного анализа сначала настройте провайдер. Нативные команды, выбор провайдера, снимки и работу со скриптами описывает [руководство по CLI и Evidence](docs/cli.md). +Для нативного анализа сначала настройте провайдер. Нативные команды, выбор провайдера, снимки и работу со скриптами описывает [руководство по CLI и Evidence](../cli.md). ### Обновите REA @@ -120,9 +120,9 @@ npx rea-agents@latest setup Агент вызывает REA через MCP, чтобы исследовать цель и проследить связанный код. REA возвращает результаты вместе с подтверждающими данными. Агент использует их для уточняющих вопросов, объяснения поведения или написания и тестирования реализации. Команды CLI используют те же рабочие процессы. -![Процесс исследования с REA: агент задаёт вопрос о локальной цели, REA исследует её инструментами анализа, а агент использует возвращённый код, ссылки и неизвестные сведения для объяснения, реализации и тестирования.](website/public/assets/figures/rea-investigation-flow.svg) +![Процесс исследования с REA: агент задаёт вопрос о локальной цели, REA исследует её инструментами анализа, а агент использует возвращённый код, ссылки и неизвестные сведения для объяснения, реализации и тестирования.](../../website/public/assets/figures/rea-investigation-flow.svg) -[Открыть рисунок в полном размере](website/public/assets/figures/rea-investigation-flow.svg). +[Открыть рисунок в полном размере](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -130,37 +130,37 @@ npx rea-agents@latest setup REA требует Node.js 22.x (>=22.19), 24.x (>=24.11) или 26+, а также npm. Дополнительные инструменты и поддерживаемые системы зависят от цели: -| Цель | Что возвращает REA | Требования и руководство | -| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | -| Нативные бинарные файлы | Псевдокод, ассемблер, строки, символы, вызовы и ссылки | Hopper, Ghidra или IDA; [нативный анализ](https://rea.tools/ru/guides/native/) | -| Структура ELF без запуска | Секции, сегменты, исходные символы/релокации и возможные механизмы защиты по данным статического анализа | pwntools, предоставленный вызывающей стороной, в Linux x64; [диагностика бинарных файлов](docs/binary-diagnostics.md) | -| Байт-код EVM | Селекторы диспетчеризации, смещения байтов, предполагаемые аргументы и изменяемость состояния | Локальные необработанные байты или шестнадцатеричное представление в Linux x64; [руководство по EVM без запуска](docs/evm-bytecode.md) | -| Записанные сбои Linux | Необработанные записи note, регистры/сигналы каждого записанного потока и необязательные кандидаты сопоставления с памятью | pwntools, предоставленный вызывающей стороной, в Linux x64; необязательные GDB/pwndbg; [записанные сбои](docs/recorded-crashes.md) | -| JavaScript / Electron | Модули, импорты, карты исходного кода, маршруты, IPC и связи с нативными дополнениями | Node.js и npm; [анализ приложений](https://rea.tools/ru/guides/javascript/) | -| Сайты | Структура страницы, скрипты, наблюдения за сетью и запрошенные снимки экрана | Браузер семейства Chrome; [анализ браузера](https://rea.tools/ru/guides/browser/) | -| Сохранённые сетевые захваты | Запросы, ответы, доступные полезные данные и позиции в источнике | HAR; mitmdump в Linux для захватов в собственном формате mitmproxy; [руководство по сетевым захватам](docs/web-network-captures.md) | -| Сборки .NET | Метаданные, инструкции CIL, объявленные нативные зависимости и сравнения сборок | Статическое исследование; [руководство по управляемому коду](docs/managed-code-analysis.md) | -| Android APK | Объявления манифеста, классы, декомпилированные методы и ссылки | JADX без графического интерфейса и полный JDK в Linux/macOS/Windows x64; [руководство по Android](docs/android-analysis.md) | -| Устройства Android | Устройства, пакеты, процессы, журналы, дампы, снимки экрана и передача APK/файлов | adb, предоставленный вызывающей стороной (эмулятор или устройство); [руководство по ADB](docs/adb-device-analysis.md) | -| Проекты JEB | Единицы проекта от движка, дайджесты артефактов и псевдокод типов/методов | Запущенный вызывающей стороной клиент JEB с интерфейсом MCP, выбирается через REA_JEB_MCP_URL; [руководство по JEB](docs/jeb-analysis.md) | -| Ресурсы Android | Декодированный манифест, таблицы строк, языки и сведения о версии | Запуск Apktool, предоставленный вызывающей стороной; [руководство по ресурсам](docs/apktool-resource-analysis.md) | -| Приложения Flutter | Хеш снапшота Dart, дайджесты библиотек по ABI и происхождение сборки движка | Не требуется (статический разбор APK); [руководство по Flutter](docs/flutter-analysis.md) | -| Прошивки | Области, результаты извлечения и передача в нативный анализ | Binwalk / Unblob в Linux; [руководство по прошивкам](docs/firmware-analysis.md) | -| Записанные пакеты | Выбранные поля, повторяющиеся значения, фильтры и правила декодирования | Linux: TShark 4.x + prlimit; [руководство по захватам](docs/packet-captures.md) | -| Захват пакетов | Конечные точки, DNS, TLS SNI, распределение протоколов и HTTP-запросы | Предоставленные вызывающей стороной tshark и capinfos; [руководство по pcap](docs/pcap-analysis.md) | -| Записанные логические сигналы | Аннотации декодера, диапазоны отсчётов и диагностика | Linux: sigrok-cli 0.7.2 + prlimit; [руководство по сигналам](docs/signal-recordings.md) | -| Пакеты и ресурсы | Списки файлов, хеши, plist, структура пакетов Apple и извлечённые ресурсы | [Руководство по артефактам и JavaScript](docs/javascript-artifact-reconstruction.md), [приложения Apple](docs/apple-application-analysis.md) | -| Поведение процессов | Вывод терминала, взаимодействия, наблюдения за завершением и файловой системой, сравнения запусков | Linux/macOS с нативным PTY; [захват процессов](docs/process-capture.md) | +| Цель | Что возвращает REA | Требования и руководство | +| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | +| Нативные бинарные файлы | Псевдокод, ассемблер, строки, символы, вызовы и ссылки | Hopper, Ghidra или IDA; [нативный анализ](https://rea.tools/ru/guides/native/) | +| Структура ELF без запуска | Секции, сегменты, исходные символы/релокации и возможные механизмы защиты по данным статического анализа | pwntools, предоставленный вызывающей стороной, в Linux x64; [диагностика бинарных файлов](../binary-diagnostics.md) | +| Байт-код EVM | Селекторы диспетчеризации, смещения байтов, предполагаемые аргументы и изменяемость состояния | Локальные необработанные байты или шестнадцатеричное представление в Linux x64; [руководство по EVM без запуска](../evm-bytecode.md) | +| Записанные сбои Linux | Необработанные записи note, регистры/сигналы каждого записанного потока и необязательные кандидаты сопоставления с памятью | pwntools, предоставленный вызывающей стороной, в Linux x64; необязательные GDB/pwndbg; [записанные сбои](../recorded-crashes.md) | +| JavaScript / Electron | Модули, импорты, карты исходного кода, маршруты, IPC и связи с нативными дополнениями | Node.js и npm; [анализ приложений](https://rea.tools/ru/guides/javascript/) | +| Сайты | Структура страницы, скрипты, наблюдения за сетью и запрошенные снимки экрана | Браузер семейства Chrome; [анализ браузера](https://rea.tools/ru/guides/browser/) | +| Сохранённые сетевые захваты | Запросы, ответы, доступные полезные данные и позиции в источнике | HAR; mitmdump в Linux для захватов в собственном формате mitmproxy; [руководство по сетевым захватам](../web-network-captures.md) | +| Сборки .NET | Метаданные, инструкции CIL, объявленные нативные зависимости и сравнения сборок | Статическое исследование; [руководство по управляемому коду](../managed-code-analysis.md) | +| Android APK | Объявления манифеста, классы, декомпилированные методы и ссылки | JADX без графического интерфейса и полный JDK в Linux/macOS/Windows x64; [руководство по Android](../android-analysis.md) | +| Устройства Android | Устройства, пакеты, процессы, журналы, дампы, снимки экрана и передача APK/файлов | adb, предоставленный вызывающей стороной (эмулятор или устройство); [руководство по ADB](../adb-device-analysis.md) | +| Проекты JEB | Единицы проекта от движка, дайджесты артефактов и псевдокод типов/методов | Запущенный вызывающей стороной клиент JEB с интерфейсом MCP, выбирается через REA_JEB_MCP_URL; [руководство по JEB](../jeb-analysis.md) | +| Ресурсы Android | Декодированный манифест, таблицы строк, языки и сведения о версии | Запуск Apktool, предоставленный вызывающей стороной; [руководство по ресурсам](../apktool-resource-analysis.md) | +| Приложения Flutter | Хеш снапшота Dart, дайджесты библиотек по ABI и происхождение сборки движка | Не требуется (статический разбор APK); [руководство по Flutter](../flutter-analysis.md) | +| Прошивки | Области, результаты извлечения и передача в нативный анализ | Binwalk / Unblob в Linux; [руководство по прошивкам](../firmware-analysis.md) | +| Записанные пакеты | Выбранные поля, повторяющиеся значения, фильтры и правила декодирования | Linux: TShark 4.x + prlimit; [руководство по захватам](../packet-captures.md) | +| Захват пакетов | Конечные точки, DNS, TLS SNI, распределение протоколов и HTTP-запросы | Предоставленные вызывающей стороной tshark и capinfos; [руководство по pcap](../pcap-analysis.md) | +| Записанные логические сигналы | Аннотации декодера, диапазоны отсчётов и диагностика | Linux: sigrok-cli 0.7.2 + prlimit; [руководство по сигналам](../signal-recordings.md) | +| Пакеты и ресурсы | Списки файлов, хеши, plist, структура пакетов Apple и извлечённые ресурсы | [Руководство по артефактам и JavaScript](../javascript-artifact-reconstruction.md), [приложения Apple](../apple-application-analysis.md) | +| Поведение процессов | Вывод терминала, взаимодействия, наблюдения за завершением и файловой системой, сравнения запусков | Linux/macOS с нативным PTY; [захват процессов](../process-capture.md) | Статическое исследование JavaScript и .NET читает предоставленные файлы, не запуская приложение. Захват во время выполнения запускает выбранную цель или взаимодействует с ней с правами вашего пользователя; соответствующие руководства описывают последствия. -Нативные форматы и поддерживаемые системы различаются у разных провайдеров. См. [настройку Hopper и Ghidra](docs/installation.md#hopper), [руководство по IDA](docs/ida-provider.md) и [экспериментальную поддержку Ghidra в Windows](docs/windows-ghidra-p0.md). Ghidra также поддерживает [анализ 16-битного DOS](docs/ghidra-dos.md). Выбор провайдера описан в [руководстве по CLI](docs/cli.md#choose-a-provider). Для функций, добавленных после последней версии в npm, проверьте [доступность в релизах](docs/installation.md#released-package-and-main). +Нативные форматы и поддерживаемые системы различаются у разных провайдеров. См. [настройку Hopper и Ghidra](../installation.md#hopper), [руководство по IDA](../ida-provider.md) и [экспериментальную поддержку Ghidra в Windows](../windows-ghidra-p0.md). Ghidra также поддерживает [анализ 16-битного DOS](../ghidra-dos.md). Выбор провайдера описан в [руководстве по CLI](../cli.md#choose-a-provider). Для функций, добавленных после последней версии в npm, проверьте [доступность в релизах](../installation.md#released-package-and-main). ## Примеры -[![Иллюстрации примеров панорамирования звука в DX-Ball, моста буфера обмена Notion и кольца пуль в TH04](docs/assets/rea-showcases.png)](https://rea.tools/ru/showcase/) +[![Иллюстрации примеров панорамирования звука в DX-Ball, моста буфера обмена Notion и кольца пуль в TH04](../assets/rea-showcases.png)](https://rea.tools/ru/showcase/) ### DX-Ball: восстановление расчёта звуковой панорамы @@ -187,28 +187,28 @@ REA требует Node.js 22.x (>=22.19), 24.x (>=24.11) или 26+, а так
Какие агенты могут использовать REA? -Любой агент с поддержкой локальных MCP-серверов. Настройка конфигурирует [поддерживаемых агентов](docs/installation.md#supported-agents); другие клиенты могут использовать [ручную регистрацию MCP](docs/installation.md#mcp-registry). +Любой агент с поддержкой локальных MCP-серверов. Настройка конфигурирует [поддерживаемых агентов](../installation.md#supported-agents); другие клиенты могут использовать [ручную регистрацию MCP](../installation.md#mcp-registry).
Нужны ли мне Hopper, Ghidra или IDA? -Углублённый нативный анализ использует один из этих инструментов. Статическое исследование JavaScript и .NET работает без движка нативного анализа. Настройка может установить Hopper после подтверждения; Ghidra и IDA используют существующие установки. См. [настройку провайдеров](docs/installation.md#hopper). +Углублённый нативный анализ использует один из этих инструментов. Статическое исследование JavaScript и .NET работает без движка нативного анализа. Настройка может установить Hopper после подтверждения; Ghidra и IDA используют существующие установки. См. [настройку провайдеров](../installation.md#hopper).
Нужно ли сначала запускать Hopper? -REA запускает Hopper, когда он нужен для операции. В macOS при первом запуске может появиться диалог выбора деморежима или активации лицензии. См. [запуск Hopper и устранение неполадок](docs/installation.md#launcher-paths-and-troubleshooting). +REA запускает Hopper, когда он нужен для операции. В macOS при первом запуске может появиться диалог выбора деморежима или активации лицензии. См. [запуск Hopper и устранение неполадок](../installation.md#launcher-paths-and-troubleshooting).
Что даёт установка skill с skills.sh? -Skill предоставляет агенту инструкции по исследованию. Используйте `npx rea-agents setup`, чтобы зарегистрировать MCP-сервер REA и установить соответствующие инструкции, затем перезапустите агента. См. [установку только skill](docs/installation.md#skill-only-installation). +Skill предоставляет агенту инструкции по исследованию. Используйте `npx rea-agents setup`, чтобы зарегистрировать MCP-сервер REA и установить соответствующие инструкции, затем перезапустите агента. См. [установку только skill](../installation.md#skill-only-installation).
@@ -251,14 +251,14 @@ npx rea-agents@latest setup Начните с [практических руководств](https://rea.tools/ru/guides/) на сайте. Точные параметры, требования и контракты результатов описаны здесь: -- [Установка и настройка](docs/installation.md): регистрация агентов, настройка провайдеров, обновление и удаление. -- [Готовность и устранение неполадок](docs/installation.md#check-readiness-for-your-task): диагностика конкретного агента или движка анализа. -- [CLI и Evidence](docs/cli.md): команды, выбор провайдера, снимки, импорт/экспорт и коды завершения. -- [Контракты MCP](docs/mcp-contracts.md) и [промпты агентов](docs/mcp-prompts.md): результаты инструментов, сеансы и исследования с подсказками. -- [Каталог инструментов](docs/mcp-contracts.md#generated-catalog): создаваемый при сборке перечень инструментов, провайдеров и команд CLI. -- [План развития](docs/roadmap.md): запланированные работы и отслеживание возможностей. +- [Установка и настройка](../installation.md): регистрация агентов, настройка провайдеров, обновление и удаление. +- [Готовность и устранение неполадок](../installation.md#check-readiness-for-your-task): диагностика конкретного агента или движка анализа. +- [CLI и Evidence](../cli.md): команды, выбор провайдера, снимки, импорт/экспорт и коды завершения. +- [Контракты MCP](../mcp-contracts.md) и [промпты агентов](../mcp-prompts.md): результаты инструментов, сеансы и исследования с подсказками. +- [Каталог инструментов](../mcp-contracts.md#generated-catalog): создаваемый при сборке перечень инструментов, провайдеров и команд CLI. +- [План развития](../roadmap.md): запланированные работы и отслеживание возможностей. -Сообщайте об уязвимостях согласно [SECURITY.md](SECURITY.md). +Сообщайте об уязвимостях согласно [SECURITY.md](../../SECURITY.md). ## История звёзд @@ -276,7 +276,7 @@ npx rea-agents@latest setup ## Отказ от ответственности -Рекомендации по ответственному использованию приведены в [DISCLAIMER.md](DISCLAIMER.md). +Рекомендации по ответственному использованию приведены в [DISCLAIMER.md](../../DISCLAIMER.md). REA предназначен для законных исследований, обучения, обеспечения совместимости, сохранения программного обеспечения и анализа программ, которыми вы владеете или которые вам разрешено изучать. @@ -288,14 +288,14 @@ applicable licenses. Будем рады вашей помощи с REA! [Откройте issue](https://github.com/morluto/rea/issues), чтобы сообщить об ошибке или предложить функцию, либо [отправьте pull request](https://github.com/morluto/rea/pulls) с улучшениями кода или документации. -Настройка среды разработки и проверки описаны в [CONTRIBUTING.md](CONTRIBUTING.md), направления проверки — в [руководстве по тестированию](docs/testing.md), а структура проекта — на [схеме архитектуры](docs/architecture.mermaid). +Настройка среды разработки и проверки описаны в [CONTRIBUTING.md](../../CONTRIBUTING.md), направления проверки — в [руководстве по тестированию](../testing.md), а структура проекта — на [схеме архитектуры](../architecture.mermaid). ## Лицензия -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_th.md b/docs/readme/README_th.md similarity index 81% rename from README_th.md rename to docs/readme/README_th.md index d07bbb6a3..734b28c86 100644 --- a/README_th.md +++ b/docs/readme/README_th.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · **ไทย** · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · **ไทย** · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA: Reverse Engineer Anything @@ -9,10 +9,10 @@ **เห็นฟีเจอร์ที่ชอบแล้วอยากรู้ว่าทำงานอย่างไร? เข้าใจได้ลึกถึงระดับไบนารี** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA เริ่มบริดจ์การวิเคราะห์ภายใน Hopper ขณะตรวจสอบไบนารีเนทีฟ +REA เริ่มบริดจ์การวิเคราะห์ภายใน Hopper ขณะตรวจสอบไบนารีเนทีฟ

@@ -33,7 +33,7 @@ - Discord
+ Discord
เข้าร่วมชุมชนวิศวกรรมย้อนกลับ
Discord · ถามตอบ · แบ่งปันผลงาน @@ -67,7 +67,7 @@ npx rea-agents setup เลือกเอเจนต์ ตรวจสอบการเปลี่ยนแปลงที่เสนอ แล้วอนุมัติ ขั้นตอนตั้งค่าจะเพิ่มเซิร์ฟเวอร์ MCP ของ REA และคำแนะนำการทำงานที่ตรงกัน พร้อมสำรองการตั้งค่าที่มีอยู่ จากนั้นเริ่มเอเจนต์ใหม่ -ขั้นตอนตั้งค่ารองรับ Claude Code, Codex, Cursor, Gemini CLI, Grok Build และ[เอเจนต์อื่น ๆ](docs/installation.md#supported-agents) ดู[การติดตั้งและตั้งค่า](docs/installation.md) สำหรับการตั้งค่าผู้ให้บริการวิเคราะห์และการลงทะเบียน MCP ด้วยตนเอง +ขั้นตอนตั้งค่ารองรับ Claude Code, Codex, Cursor, Gemini CLI, Grok Build และ[เอเจนต์อื่น ๆ](../installation.md#supported-agents) ดู[การติดตั้งและตั้งค่า](../installation.md) สำหรับการตั้งค่าผู้ให้บริการวิเคราะห์และการลงทะเบียน MCP ด้วยตนเอง ### ถามเอเจนต์ @@ -94,7 +94,7 @@ npm install --global rea-agents rea --help ``` -สำหรับการวิเคราะห์เนทีฟ ให้ตั้งค่าผู้ให้บริการวิเคราะห์ก่อน ดู[คู่มือ CLI และ Evidence](docs/cli.md) สำหรับคำสั่งเนทีฟ การเลือกผู้ให้บริการ สแนปช็อต และการใช้งานในสคริปต์ +สำหรับการวิเคราะห์เนทีฟ ให้ตั้งค่าผู้ให้บริการวิเคราะห์ก่อน ดู[คู่มือ CLI และ Evidence](../cli.md) สำหรับคำสั่งเนทีฟ การเลือกผู้ให้บริการ สแนปช็อต และการใช้งานในสคริปต์ ### อัปเดต REA @@ -120,9 +120,9 @@ npx rea-agents@latest setup เอเจนต์เรียก REA ผ่าน MCP เพื่อตรวจสอบเป้าหมายและติดตามโค้ดที่เกี่ยวข้อง REA ส่งข้อค้นพบกลับมาพร้อมหลักฐาน เอเจนต์ใช้ข้อมูลนี้เพื่อถามคำถามต่อ อธิบายพฤติกรรม หรือเขียนและทดสอบการนำไปใช้งาน คำสั่ง CLI ใช้กระบวนการเดียวกัน -![กระบวนการตรวจสอบด้วย REA: เอเจนต์ถามเกี่ยวกับเป้าหมายบนเครื่อง REA ตรวจสอบและติดตามด้วยเครื่องมือวิเคราะห์ แล้วเอเจนต์ใช้โค้ด การอ้างอิง และสิ่งที่ยังไม่ทราบที่ได้รับกลับมาเพื่ออธิบาย สร้าง และทดสอบ](website/public/assets/figures/rea-investigation-flow.svg) +![กระบวนการตรวจสอบด้วย REA: เอเจนต์ถามเกี่ยวกับเป้าหมายบนเครื่อง REA ตรวจสอบและติดตามด้วยเครื่องมือวิเคราะห์ แล้วเอเจนต์ใช้โค้ด การอ้างอิง และสิ่งที่ยังไม่ทราบที่ได้รับกลับมาเพื่ออธิบาย สร้าง และทดสอบ](../../website/public/assets/figures/rea-investigation-flow.svg) -[เปิดภาพขนาดเต็ม](website/public/assets/figures/rea-investigation-flow.svg) +[เปิดภาพขนาดเต็ม](../../website/public/assets/figures/rea-investigation-flow.svg) @@ -130,37 +130,37 @@ npx rea-agents@latest setup REA ต้องใช้ Node.js 22.x (>=22.19), 24.x (>=24.11) หรือ 26+ และ npm เครื่องมือเพิ่มเติมและแพลตฟอร์มที่รองรับขึ้นอยู่กับเป้าหมาย: -| เป้าหมาย | สิ่งที่ REA ส่งกลับ | ข้อกำหนดและคู่มือ | -| ---------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | -| ไบนารีเนทีฟ | โค้ดเทียม แอสเซมบลี สตริง สัญลักษณ์ การเรียก และการอ้างอิง | Hopper, Ghidra หรือ IDA; [การวิเคราะห์เนทีฟ](https://rea.tools/th/guides/native/) | -| โครงสร้าง ELF แบบออฟไลน์ | เซกชัน เซกเมนต์ สัญลักษณ์/รีโลเคชันต้นฉบับ และตัวเลือกกลไกป้องกันที่พบจากการวิเคราะห์แบบสถิต | pwntools ที่ผู้ใช้จัดเตรียมบน Linux x64; [การวินิจฉัยไบนารี](docs/binary-diagnostics.md) | -| ไบต์โค้ด EVM | ตัวเลือกฟังก์ชันสำหรับการเรียก ออฟเซ็ตไบต์ อาร์กิวเมนต์และความสามารถในการเปลี่ยนสถานะที่อนุมานได้ | ไฟล์บนเครื่องที่มีไบต์ดิบหรือข้อมูลเลขฐานสิบหกบน Linux x64; [คู่มือ EVM แบบออฟไลน์](docs/evm-bytecode.md) | -| ข้อมูลการแครชของ Linux ที่บันทึกไว้ | ระเบียน note ดิบ รีจิสเตอร์/สัญญาณของทุกเธรดที่บันทึกไว้ และตัวเลือกการแมปเพิ่มเติม | pwntools ที่ผู้ใช้จัดเตรียมบน Linux x64; เลือกใช้ GDB/pwndbg เพิ่มเติมได้; [การแครชที่บันทึกไว้](docs/recorded-crashes.md) | -| JavaScript / Electron | โมดูล การนำเข้า ซอร์สแมป เส้นทาง IPC และความสัมพันธ์กับส่วนเสริมเนทีฟ | Node.js และ npm; [การวิเคราะห์แอปพลิเคชัน](https://rea.tools/th/guides/javascript/) | -| เว็บไซต์ | โครงสร้างหน้า สคริปต์ ข้อมูลเครือข่ายที่สังเกตได้ และภาพหน้าจอที่ร้องขอ | เบราว์เซอร์ในตระกูล Chrome; [การวิเคราะห์ผ่านเบราว์เซอร์](https://rea.tools/th/guides/browser/) | -| ข้อมูลเครือข่ายที่บันทึกไว้ | คำขอ คำตอบ เพย์โหลดที่ปรากฏ และตำแหน่งต้นทาง | HAR; mitmdump บน Linux สำหรับข้อมูลที่บันทึกในรูปแบบเนทีฟของ mitmproxy; [คู่มือการบันทึกข้อมูล](docs/web-network-captures.md) | -| การจับแพ็กเก็ต | จุดเชื่อมต่อ, DNS, TLS SNI, การกระจายโปรโตคอล และคำขอ HTTP | tshark และ capinfos ที่ผู้เรียกจัดเตรียม; [คู่มือ pcap](docs/pcap-analysis.md) | -| แอสเซมบลี .NET | เมทาดาทา คำสั่ง CIL การพึ่งพาเนทีฟที่ประกาศไว้ และการเปรียบเทียบผลการบิลด์ | การตรวจสอบแบบสถิต; [คู่มือโค้ดแบบจัดการ](docs/managed-code-analysis.md) | -| APK ของ Android | การประกาศใน manifest คลาส เมธอดที่ดีคอมไพล์ และการอ้างอิง | JADX แบบไม่มี GUI และ JDK แบบเต็มบน Linux/macOS/Windows x64; [คู่มือ Android](docs/android-analysis.md) | -| อุปกรณ์ Android | อุปกรณ์ แพ็กเกจ โพรเซส บันทึก ดัมพ์ ภาพหน้าจอ และการถ่ายโอน APK/ไฟล์ | adb ที่ผู้เรียกจัดเตรียม (โปรแกรมจำลองหรืออุปกรณ์จริง); [คู่มือ ADB](docs/adb-device-analysis.md) | -| โปรเจกต์ JEB | หน่วยโปรเจกต์จากเอนจิน, แฮชของอาร์ติแฟกต์ และซูโดโค้ดของไทป์/เมท็อด | ไคลเอนต์ JEB ที่ผู้เรียกเริ่มและให้บริการ MCP เลือกด้วย REA_JEB_MCP_URL; [คู่มือ JEB](docs/jeb-analysis.md) | -| ทรัพยากร Android | ไฟล์ manifest ที่ถอดรหัสแล้ว ตารางสตริง ภาษา และข้อมูลเวอร์ชัน | ตัวเรียก Apktool ที่ผู้เรียกจัดเตรียม; [คู่มือทรัพยากร](docs/apktool-resource-analysis.md) | -| แอป Flutter | แฮชสแนปชอต Dart, แฮชไลบรารีราย ABI และที่มาของบิลด์เอนจิน | ไม่ต้อง (วิเคราะห์ APK แบบ static); [คู่มือ Flutter](docs/flutter-analysis.md) | -| เฟิร์มแวร์ | พื้นที่ข้อมูล ผลการแตกไฟล์ และการส่งต่อไปวิเคราะห์เนทีฟ | Binwalk / Unblob บน Linux; [คู่มือเฟิร์มแวร์](docs/firmware-analysis.md) | -| แพ็กเก็ตที่บันทึกไว้ | ฟิลด์ที่เลือก ค่าซ้ำ ตัวกรอง และกฎการถอดรหัส | Linux: TShark 4.x + prlimit; [คู่มือไฟล์แคปเจอร์](docs/packet-captures.md) | -| สัญญาณลอจิกที่บันทึกไว้ | คำอธิบายจากตัวถอดรหัส ช่วงตัวอย่าง และข้อมูลวินิจฉัย | Linux: sigrok-cli 0.7.2 + prlimit; [คู่มือสัญญาณ](docs/signal-recordings.md) | -| แพ็กเกจและทรัพยากร | รายการไฟล์ ค่าแฮช plist โครงสร้างบันเดิลของ Apple และทรัพยากรที่แตกออกมา | [คู่มืออาร์ติแฟกต์และ JavaScript](docs/javascript-artifact-reconstruction.md), [แอปพลิเคชัน Apple](docs/apple-application-analysis.md) | -| พฤติกรรมของโปรเซส | เอาต์พุตเทอร์มินัล การโต้ตอบ ข้อมูลการสิ้นสุดและระบบไฟล์ที่สังเกตได้ และการเปรียบเทียบการรัน | Linux/macOS ที่มี PTY เนทีฟ; [การบันทึกโปรเซส](docs/process-capture.md) | +| เป้าหมาย | สิ่งที่ REA ส่งกลับ | ข้อกำหนดและคู่มือ | +| ---------------------------- | ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | +| ไบนารีเนทีฟ | โค้ดเทียม แอสเซมบลี สตริง สัญลักษณ์ การเรียก และการอ้างอิง | Hopper, Ghidra หรือ IDA; [การวิเคราะห์เนทีฟ](https://rea.tools/th/guides/native/) | +| โครงสร้าง ELF แบบออฟไลน์ | เซกชัน เซกเมนต์ สัญลักษณ์/รีโลเคชันต้นฉบับ และตัวเลือกกลไกป้องกันที่พบจากการวิเคราะห์แบบสถิต | pwntools ที่ผู้ใช้จัดเตรียมบน Linux x64; [การวินิจฉัยไบนารี](../binary-diagnostics.md) | +| ไบต์โค้ด EVM | ตัวเลือกฟังก์ชันสำหรับการเรียก ออฟเซ็ตไบต์ อาร์กิวเมนต์และความสามารถในการเปลี่ยนสถานะที่อนุมานได้ | ไฟล์บนเครื่องที่มีไบต์ดิบหรือข้อมูลเลขฐานสิบหกบน Linux x64; [คู่มือ EVM แบบออฟไลน์](../evm-bytecode.md) | +| ข้อมูลการแครชของ Linux ที่บันทึกไว้ | ระเบียน note ดิบ รีจิสเตอร์/สัญญาณของทุกเธรดที่บันทึกไว้ และตัวเลือกการแมปเพิ่มเติม | pwntools ที่ผู้ใช้จัดเตรียมบน Linux x64; เลือกใช้ GDB/pwndbg เพิ่มเติมได้; [การแครชที่บันทึกไว้](../recorded-crashes.md) | +| JavaScript / Electron | โมดูล การนำเข้า ซอร์สแมป เส้นทาง IPC และความสัมพันธ์กับส่วนเสริมเนทีฟ | Node.js และ npm; [การวิเคราะห์แอปพลิเคชัน](https://rea.tools/th/guides/javascript/) | +| เว็บไซต์ | โครงสร้างหน้า สคริปต์ ข้อมูลเครือข่ายที่สังเกตได้ และภาพหน้าจอที่ร้องขอ | เบราว์เซอร์ในตระกูล Chrome; [การวิเคราะห์ผ่านเบราว์เซอร์](https://rea.tools/th/guides/browser/) | +| ข้อมูลเครือข่ายที่บันทึกไว้ | คำขอ คำตอบ เพย์โหลดที่ปรากฏ และตำแหน่งต้นทาง | HAR; mitmdump บน Linux สำหรับข้อมูลที่บันทึกในรูปแบบเนทีฟของ mitmproxy; [คู่มือการบันทึกข้อมูล](../web-network-captures.md) | +| การจับแพ็กเก็ต | จุดเชื่อมต่อ, DNS, TLS SNI, การกระจายโปรโตคอล และคำขอ HTTP | tshark และ capinfos ที่ผู้เรียกจัดเตรียม; [คู่มือ pcap](../pcap-analysis.md) | +| แอสเซมบลี .NET | เมทาดาทา คำสั่ง CIL การพึ่งพาเนทีฟที่ประกาศไว้ และการเปรียบเทียบผลการบิลด์ | การตรวจสอบแบบสถิต; [คู่มือโค้ดแบบจัดการ](../managed-code-analysis.md) | +| APK ของ Android | การประกาศใน manifest คลาส เมธอดที่ดีคอมไพล์ และการอ้างอิง | JADX แบบไม่มี GUI และ JDK แบบเต็มบน Linux/macOS/Windows x64; [คู่มือ Android](../android-analysis.md) | +| อุปกรณ์ Android | อุปกรณ์ แพ็กเกจ โพรเซส บันทึก ดัมพ์ ภาพหน้าจอ และการถ่ายโอน APK/ไฟล์ | adb ที่ผู้เรียกจัดเตรียม (โปรแกรมจำลองหรืออุปกรณ์จริง); [คู่มือ ADB](../adb-device-analysis.md) | +| โปรเจกต์ JEB | หน่วยโปรเจกต์จากเอนจิน, แฮชของอาร์ติแฟกต์ และซูโดโค้ดของไทป์/เมท็อด | ไคลเอนต์ JEB ที่ผู้เรียกเริ่มและให้บริการ MCP เลือกด้วย REA_JEB_MCP_URL; [คู่มือ JEB](../jeb-analysis.md) | +| ทรัพยากร Android | ไฟล์ manifest ที่ถอดรหัสแล้ว ตารางสตริง ภาษา และข้อมูลเวอร์ชัน | ตัวเรียก Apktool ที่ผู้เรียกจัดเตรียม; [คู่มือทรัพยากร](../apktool-resource-analysis.md) | +| แอป Flutter | แฮชสแนปชอต Dart, แฮชไลบรารีราย ABI และที่มาของบิลด์เอนจิน | ไม่ต้อง (วิเคราะห์ APK แบบ static); [คู่มือ Flutter](../flutter-analysis.md) | +| เฟิร์มแวร์ | พื้นที่ข้อมูล ผลการแตกไฟล์ และการส่งต่อไปวิเคราะห์เนทีฟ | Binwalk / Unblob บน Linux; [คู่มือเฟิร์มแวร์](../firmware-analysis.md) | +| แพ็กเก็ตที่บันทึกไว้ | ฟิลด์ที่เลือก ค่าซ้ำ ตัวกรอง และกฎการถอดรหัส | Linux: TShark 4.x + prlimit; [คู่มือไฟล์แคปเจอร์](../packet-captures.md) | +| สัญญาณลอจิกที่บันทึกไว้ | คำอธิบายจากตัวถอดรหัส ช่วงตัวอย่าง และข้อมูลวินิจฉัย | Linux: sigrok-cli 0.7.2 + prlimit; [คู่มือสัญญาณ](../signal-recordings.md) | +| แพ็กเกจและทรัพยากร | รายการไฟล์ ค่าแฮช plist โครงสร้างบันเดิลของ Apple และทรัพยากรที่แตกออกมา | [คู่มืออาร์ติแฟกต์และ JavaScript](../javascript-artifact-reconstruction.md), [แอปพลิเคชัน Apple](../apple-application-analysis.md) | +| พฤติกรรมของโปรเซส | เอาต์พุตเทอร์มินัล การโต้ตอบ ข้อมูลการสิ้นสุดและระบบไฟล์ที่สังเกตได้ และการเปรียบเทียบการรัน | Linux/macOS ที่มี PTY เนทีฟ; [การบันทึกโปรเซส](../process-capture.md) | การตรวจสอบ JavaScript และ .NET แบบสถิตอ่านไฟล์ที่ให้มาโดยไม่รันแอปพลิเคชัน การบันทึกขณะทำงานจะรันหรือโต้ตอบกับเป้าหมายที่เลือกโดยใช้สิทธิ์ของผู้ใช้คุณ คู่มือการทำงานแต่ละฉบับอธิบายผลที่เกิดขึ้นไว้ -รูปแบบไบนารีเนทีฟและแพลตฟอร์มที่รองรับแตกต่างกันตามผู้ให้บริการวิเคราะห์ ดู[การตั้งค่า Hopper และ Ghidra](docs/installation.md#hopper), [คู่มือ IDA](docs/ida-provider.md) และ[การรองรับ Ghidra บน Windows แบบทดลอง](docs/windows-ghidra-p0.md) Ghidra ยังรองรับ[การวิเคราะห์ DOS แบบ 16 บิต](docs/ghidra-dos.md) สำหรับการเลือกผู้ให้บริการ ดู[คู่มือ CLI](docs/cli.md#choose-a-provider) ตรวจสอบ[ความพร้อมใช้งานในแต่ละรุ่น](docs/installation.md#released-package-and-main) สำหรับฟีเจอร์ที่เพิ่มหลังรุ่นล่าสุดบน npm +รูปแบบไบนารีเนทีฟและแพลตฟอร์มที่รองรับแตกต่างกันตามผู้ให้บริการวิเคราะห์ ดู[การตั้งค่า Hopper และ Ghidra](../installation.md#hopper), [คู่มือ IDA](../ida-provider.md) และ[การรองรับ Ghidra บน Windows แบบทดลอง](../windows-ghidra-p0.md) Ghidra ยังรองรับ[การวิเคราะห์ DOS แบบ 16 บิต](../ghidra-dos.md) สำหรับการเลือกผู้ให้บริการ ดู[คู่มือ CLI](../cli.md#choose-a-provider) ตรวจสอบ[ความพร้อมใช้งานในแต่ละรุ่น](../installation.md#released-package-and-main) สำหรับฟีเจอร์ที่เพิ่มหลังรุ่นล่าสุดบน npm ## ตัวอย่างการใช้งาน -[![ภาพประกอบกรณีศึกษาการแพนเสียงใน DX-Ball บริดจ์คลิปบอร์ดของ Notion และวงแหวนกระสุนใน TH04](docs/assets/rea-showcases.png)](https://rea.tools/th/showcase/) +[![ภาพประกอบกรณีศึกษาการแพนเสียงใน DX-Ball บริดจ์คลิปบอร์ดของ Notion และวงแหวนกระสุนใน TH04](../assets/rea-showcases.png)](https://rea.tools/th/showcase/) ### DX-Ball: สร้างการคำนวณแพนเสียงขึ้นใหม่ @@ -187,28 +187,28 @@ REA ต้องใช้ Node.js 22.x (>=22.19), 24.x (>=24.11) หรือ 26
เอเจนต์ใดใช้ REA ได้บ้าง? -เอเจนต์ใดก็ตามที่รองรับเซิร์ฟเวอร์ MCP บนเครื่อง ขั้นตอนตั้งค่าจะกำหนดค่าให้กับ[เอเจนต์ที่รองรับ](docs/installation.md#supported-agents) ส่วนไคลเอนต์อื่นใช้[การลงทะเบียน MCP ด้วยตนเอง](docs/installation.md#mcp-registry) ได้ +เอเจนต์ใดก็ตามที่รองรับเซิร์ฟเวอร์ MCP บนเครื่อง ขั้นตอนตั้งค่าจะกำหนดค่าให้กับ[เอเจนต์ที่รองรับ](../installation.md#supported-agents) ส่วนไคลเอนต์อื่นใช้[การลงทะเบียน MCP ด้วยตนเอง](../installation.md#mcp-registry) ได้
จำเป็นต้องมี Hopper, Ghidra หรือ IDA หรือไม่? -การวิเคราะห์เนทีฟเชิงลึกต้องใช้หนึ่งในเครื่องมือเหล่านี้ การตรวจสอบ JavaScript และ .NET แบบสถิตทำงานได้โดยไม่มีเอนจินวิเคราะห์เนทีฟ ขั้นตอนตั้งค่าสามารถติดตั้ง Hopper หลังได้รับอนุมัติ ส่วน Ghidra และ IDA ใช้การติดตั้งที่คุณมีอยู่แล้ว ดู[การตั้งค่าผู้ให้บริการวิเคราะห์](docs/installation.md#hopper) +การวิเคราะห์เนทีฟเชิงลึกต้องใช้หนึ่งในเครื่องมือเหล่านี้ การตรวจสอบ JavaScript และ .NET แบบสถิตทำงานได้โดยไม่มีเอนจินวิเคราะห์เนทีฟ ขั้นตอนตั้งค่าสามารถติดตั้ง Hopper หลังได้รับอนุมัติ ส่วน Ghidra และ IDA ใช้การติดตั้งที่คุณมีอยู่แล้ว ดู[การตั้งค่าผู้ให้บริการวิเคราะห์](../installation.md#hopper)
ต้องเริ่ม Hopper ก่อนหรือไม่? -REA เริ่ม Hopper เมื่อการดำเนินการต้องใช้ บน macOS อาจมีหน้าต่างเมื่อรันครั้งแรกให้เลือกโหมดทดลองหรือเปิดใช้ใบอนุญาต ดู[การเริ่ม Hopper และการแก้ไขปัญหา](docs/installation.md#launcher-paths-and-troubleshooting) +REA เริ่ม Hopper เมื่อการดำเนินการต้องใช้ บน macOS อาจมีหน้าต่างเมื่อรันครั้งแรกให้เลือกโหมดทดลองหรือเปิดใช้ใบอนุญาต ดู[การเริ่ม Hopper และการแก้ไขปัญหา](../installation.md#launcher-paths-and-troubleshooting)
การติดตั้งสกิลจาก skills.sh ทำอะไร? -สกิลให้คำแนะนำการตรวจสอบแก่เอเจนต์ ใช้ `npx rea-agents setup` เพื่อลงทะเบียนเซิร์ฟเวอร์ MCP ของ REA และติดตั้งคำแนะนำที่ตรงกัน แล้วเริ่มเอเจนต์ใหม่ ดู[การติดตั้งเฉพาะสกิล](docs/installation.md#skill-only-installation) +สกิลให้คำแนะนำการตรวจสอบแก่เอเจนต์ ใช้ `npx rea-agents setup` เพื่อลงทะเบียนเซิร์ฟเวอร์ MCP ของ REA และติดตั้งคำแนะนำที่ตรงกัน แล้วเริ่มเอเจนต์ใหม่ ดู[การติดตั้งเฉพาะสกิล](../installation.md#skill-only-installation)
@@ -251,14 +251,14 @@ npx rea-agents@latest setup เริ่มจาก[คู่มือพร้อมตัวอย่าง](https://rea.tools/th/guides/) บนเว็บไซต์ สำหรับตัวเลือก ข้อกำหนดเบื้องต้น และสัญญาผลลัพธ์ที่แน่นอน: -- [การติดตั้งและตั้งค่า](docs/installation.md): การลงทะเบียนเอเจนต์ การตั้งค่าผู้ให้บริการ การอัปเดต และการถอนการติดตั้ง -- [ความพร้อมใช้งานและการแก้ไขปัญหา](docs/installation.md#check-readiness-for-your-task): ตรวจวินิจฉัยเอเจนต์หรือเอนจินวิเคราะห์หนึ่งตัว -- [CLI และ Evidence](docs/cli.md): คำสั่ง การเลือกผู้ให้บริการ สแนปช็อต การนำเข้า/ส่งออก และรหัสการสิ้นสุด -- [สัญญา MCP](docs/mcp-contracts.md) และ[พรอมป์ต์สำหรับเอเจนต์](docs/mcp-prompts.md): ผลลัพธ์เครื่องมือ เซสชัน และการตรวจสอบตามคำแนะนำ -- [แค็ตตาล็อกเครื่องมือ](docs/mcp-contracts.md#generated-catalog): รายการเครื่องมือ ผู้ให้บริการ และคำสั่ง CLI ที่สร้างระหว่างการบิลด์ -- [แผนพัฒนา](docs/roadmap.md): งานที่วางแผนไว้และรายการติดตามความสามารถ +- [การติดตั้งและตั้งค่า](../installation.md): การลงทะเบียนเอเจนต์ การตั้งค่าผู้ให้บริการ การอัปเดต และการถอนการติดตั้ง +- [ความพร้อมใช้งานและการแก้ไขปัญหา](../installation.md#check-readiness-for-your-task): ตรวจวินิจฉัยเอเจนต์หรือเอนจินวิเคราะห์หนึ่งตัว +- [CLI และ Evidence](../cli.md): คำสั่ง การเลือกผู้ให้บริการ สแนปช็อต การนำเข้า/ส่งออก และรหัสการสิ้นสุด +- [สัญญา MCP](../mcp-contracts.md) และ[พรอมป์ต์สำหรับเอเจนต์](../mcp-prompts.md): ผลลัพธ์เครื่องมือ เซสชัน และการตรวจสอบตามคำแนะนำ +- [แค็ตตาล็อกเครื่องมือ](../mcp-contracts.md#generated-catalog): รายการเครื่องมือ ผู้ให้บริการ และคำสั่ง CLI ที่สร้างระหว่างการบิลด์ +- [แผนพัฒนา](../roadmap.md): งานที่วางแผนไว้และรายการติดตามความสามารถ -รายงานช่องโหว่ผ่าน [SECURITY.md](SECURITY.md) +รายงานช่องโหว่ผ่าน [SECURITY.md](../../SECURITY.md) ## ประวัติดาว @@ -276,7 +276,7 @@ npx rea-agents@latest setup ## ข้อสงวนสิทธิ์ -ดูแนวทางการใช้งานอย่างรับผิดชอบใน [DISCLAIMER.md](DISCLAIMER.md) +ดูแนวทางการใช้งานอย่างรับผิดชอบใน [DISCLAIMER.md](../../DISCLAIMER.md) REA มีไว้สำหรับการวิจัย การศึกษา การทำงานร่วมกัน การอนุรักษ์ซอฟต์แวร์ และการวิเคราะห์ซอฟต์แวร์ที่คุณเป็นเจ้าของหรือได้รับอนุญาตให้ตรวจสอบ โดยปฏิบัติตามกฎหมาย @@ -288,14 +288,14 @@ applicable licenses. เรายินดีรับความช่วยเหลือในการพัฒนา REA! [เปิด issue](https://github.com/morluto/rea/issues) เพื่อรายงานบั๊กหรือเสนอฟีเจอร์ หรือ[ส่ง pull request](https://github.com/morluto/rea/pulls) เพื่อปรับปรุงโค้ดหรือเอกสาร -ดู [CONTRIBUTING.md](CONTRIBUTING.md) สำหรับการตั้งค่าสภาพแวดล้อมพัฒนาและการตรวจสอบ ดู[การทดสอบ](docs/testing.md) สำหรับแนวทางการตรวจยืนยัน และ[แผนผังสถาปัตยกรรม](docs/architecture.mermaid) สำหรับโครงสร้างโปรเจกต์ +ดู [CONTRIBUTING.md](../../CONTRIBUTING.md) สำหรับการตั้งค่าสภาพแวดล้อมพัฒนาและการตรวจสอบ ดู[การทดสอบ](../testing.md) สำหรับแนวทางการตรวจยืนยัน และ[แผนผังสถาปัตยกรรม](../architecture.mermaid) สำหรับโครงสร้างโปรเจกต์ ## สัญญาอนุญาต -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_tr.md b/docs/readme/README_tr.md similarity index 65% rename from README_tr.md rename to docs/readme/README_tr.md index 1daaaf9d0..152c81819 100644 --- a/README_tr.md +++ b/docs/readme/README_tr.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · **Türkçe** · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · **Türkçe** · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA: Her şeyi tersine mühendislikle inceleyin @@ -9,10 +9,10 @@ **Beğendiğiniz bir özellik mi gördünüz? İkili kod düzeyine kadar nasıl çalıştığını anlayın.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA, yerel bir ikili dosyayı incelerken Hopper içinde analiz köprüsünü başlatıyor +REA, yerel bir ikili dosyayı incelerken Hopper içinde analiz köprüsünü başlatıyor

@@ -33,7 +33,7 @@ - Discord
+ Discord
Tersine mühendislik topluluğuna katılın
Discord · Soru-cevap · Sonuç paylaşımı @@ -51,7 +51,7 @@ Bir uygulamada kendi ürününüze eklemek istediğiniz bir özellik mi gördün REA, ajanınızı yerel makine kodu içeren ikili dosyaları, JavaScript ve Electron uygulamalarını, .NET derlemelerini ve web sitelerini inceleyen araçlara bağlar. Aynı araçları terminalden de kullanabilirsiniz. Analiz yerel olarak çalışır; sonuçlar her çıkarımın dayandığı kanıtları ve sınırlamaları içerir. -Kurulum, REA'yı ajanınıza kaydeder ve uyumlu iş akışı talimatlarını yükler. Yerel makine kodu analizi mevcut bir Hopper veya Ghidra kurulumunu kullanabilir; kurulum işlemi, onayınızla isteğe bağlı olarak Hopper da yükleyebilir. Statik JavaScript analizi bu iki motora da ihtiyaç duymaz. +Kurulum, REA'yı ajanınıza kaydeder ve uyumlu iş akışı talimatlarını yükler. Yerel makine kodu analizi mevcut bir Hopper, Ghidra veya IDA kurulumunu kullanabilir; kurulum işlemi, onayınızla isteğe bağlı olarak Hopper da yükleyebilir. Statik JavaScript analizi bu yerel analiz motorlarından hiçbirine ihtiyaç duymaz. > Kurulum talimatları, görselli rehberler ve gerçek örnek çalışmalar için **[REA web sitesini ziyaret edin](https://rea.tools/tr/)**. @@ -67,7 +67,7 @@ npx rea-agents setup Ajanlarınızı seçin, önerilen değişiklikleri inceleyin ve onaylayın. Kurulum, mevcut yapılandırmaları yedekleyerek REA'nın MCP sunucusunu ve uyumlu iş akışı talimatlarını ekler. Ardından ajanınızı yeniden başlatın. -Kurulum Claude Code, Codex, Cursor, Gemini CLI, Grok Build ve [diğer ajanları](docs/installation.md#supported-agents) destekler. Sağlayıcı yapılandırması ve elle MCP kaydı için [yükleme ve kurulum](docs/installation.md) belgesine bakın. +Kurulum Claude Code, Codex, Cursor, Gemini CLI, Grok Build ve [diğer ajanları](../installation.md#supported-agents) destekler. Sağlayıcı yapılandırması ve elle MCP kaydı için [yükleme ve kurulum](../installation.md) belgesine bakın. ### Ajanınıza sorun @@ -94,18 +94,20 @@ npm install --global rea-agents rea --help ``` -Yerel makine kodu analizi için önce bir sağlayıcı yapılandırın. Yerel analiz komutları, sağlayıcı seçimi, anlık görüntüler ve betik kullanımı için [CLI ve Evidence rehberine](docs/cli.md) bakın. +Yerel makine kodu analizi için önce bir sağlayıcı yapılandırın. Yerel analiz komutları, sağlayıcı seçimi, anlık görüntüler ve betik kullanımı için [CLI ve Evidence rehberine](../cli.md) bakın. ### REA'yı güncelleyin REA hızla değişir ve yeni sürümler sık sık hata düzeltmeleri içerir. Kurulumunuzu güncel tutun. -npm ile yüklenen CLI için: +`npm install --global` veya `bun add --global` ile yüklenen CLI için: ```bash rea update ``` +Bun ile yapılan global güncellemeler için Bun 1.2.17 veya üzeri gerekir. Daha eski bir Bun sürümünü kullanıyorsanız `rea update` komutunu çalıştırmadan önce Bun'u manuel olarak yükseltin. + Ajan kayıtlarını ve skill'i güncellemek için güncellemenin çıktısında gösterilen kurulum komutunu çalıştırın. `npx` kullanıyorsanız ajan kurulumunu şu komutla güncelleyin: @@ -114,15 +116,15 @@ Ajan kayıtlarını ve skill'i güncellemek için güncellemenin çıktısında npx rea-agents@latest setup ``` -Kurulum değişikliklerini inceleyin ve ajanınızı yeniden başlatın. Tek seferlik CLI işlemleri için `npx rea-agents@latest` komutunun ardından istediğiniz komutu yazın. +Kurulum değişikliklerini inceleyin ve ajanınızı yeniden başlatın. Tek seferlik CLI işlemleri için `npx rea-agents@latest` komutunun ardından istediğiniz komutu yazın. Bun kullanıyorsanız `npx rea-agents@latest` yerine `bunx rea-agents@latest` kullanın. ## REA nasıl çalışır Ajanınız, hedefi incelemek ve ilgili kodu izlemek için MCP üzerinden REA'yı çağırır. REA bulguları kanıtlarıyla birlikte döndürür. Ajan bunları takip soruları sormak, davranışı açıklamak veya bir uygulama yazıp test etmek için kullanır. CLI komutları aynı iş akışlarını kullanır. -![REA araştırma akışı: ajanınız yerel bir hedef hakkında soru sorar, REA analiz araçlarıyla hedefi inceler ve izler; ajan döndürülen kodu, referansları ve bilinmeyenleri açıklama, geliştirme ve test için kullanır.](website/public/assets/figures/rea-investigation-flow.svg) +![REA araştırma akışı: ajanınız yerel bir hedef hakkında soru sorar, REA analiz araçlarıyla hedefi inceler ve izler; ajan döndürülen kodu, referansları ve bilinmeyenleri açıklama, geliştirme ve test için kullanır.](../../website/public/assets/figures/rea-investigation-flow.svg) -[Şekli tam boyutuyla açın](website/public/assets/figures/rea-investigation-flow.svg). +[Şekli tam boyutuyla açın](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -130,37 +132,40 @@ Ajanınız, hedefi incelemek ve ilgili kodu izlemek için MCP üzerinden REA'yı REA; Node.js 22.x (>=22.19), 24.x (>=24.11) veya 26+ ve npm gerektirir. Ek araçlar ve desteklenen ana bilgisayar sistemleri hedefe bağlıdır: -| Hedef | REA'nın döndürdükleri | Gereksinimler ve rehber | -| --------------------------------------- | ----------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | -| Yerel makine kodu içeren ikili dosyalar | Sözde kod, assembly kodu, dizeler, semboller, çağrılar ve referanslar | Hopper, Ghidra veya IDA; [yerel kod analizi](https://rea.tools/tr/guides/native/) | -| Çevrimdışı ELF yapısı | Bölümler, segmentler, özgün semboller/yeniden konumlandırmalar ve statik analizden olası güvenlik önlemleri | Linux x64 üzerinde çağıranın sağladığı pwntools; [ikili dosya tanılama](docs/binary-diagnostics.md) | -| EVM bayt kodu | Yönlendirme seçicileri, bayt ofsetleri, çıkarımla belirlenen argümanlar ve durum değiştirilebilirliği | Linux x64 üzerinde yerel ham bayt/onaltılık girdi; [çevrimdışı EVM rehberi](docs/evm-bytecode.md) | -| Kaydedilmiş Linux çökmeleri | Ham note kayıtları, kaydedilen her iş parçacığının yazmaçları/sinyalleri ve isteğe bağlı eşleme adayları | Linux x64 üzerinde çağıranın sağladığı pwntools; isteğe bağlı GDB/pwndbg; [kaydedilmiş çökmeler](docs/recorded-crashes.md) | -| JavaScript / Electron | Modüller, içe aktarmalar, kaynak haritaları, rotalar, IPC ve yerel eklenti ilişkileri | Node.js ve npm; [uygulama analizi](https://rea.tools/tr/guides/javascript/) | -| Web siteleri | Sayfa yapısı, betikler, ağ gözlemleri ve istenen ekran görüntüleri | Chrome ailesinden bir tarayıcı; [tarayıcı analizi](https://rea.tools/tr/guides/browser/) | -| Kaydedilmiş ağ yakalamaları | İstekler, yanıtlar, erişilebilir yük içerikleri ve kaynak konumları | HAR; mitmproxy'nin kendi biçimindeki yakalamalar için Linux üzerinde mitmdump; [ağ yakalama rehberi](docs/web-network-captures.md) | -| Paket yakalamaları | Uç noktalar, DNS, TLS SNI, protokol dağılımı ve HTTP istekleri | Çağıranın sağladığı tshark ve capinfos; [pcap rehberi](docs/pcap-analysis.md) | -| .NET derlemeleri | Meta veriler, CIL talimatları, bildirilmiş yerel bağımlılıklar ve derleme karşılaştırmaları | Statik inceleme; [yönetilen kod rehberi](docs/managed-code-analysis.md) | -| Android APK'leri | Manifest bildirimleri, sınıflar, tersine derlenmiş metotlar ve referanslar | Linux/macOS/Windows x64 üzerinde arayüzsüz JADX ve tam bir JDK; [Android rehberi](docs/android-analysis.md) | -| Android cihazları | Cihazlar, paketler, süreçler, günlükler, dökümler, ekran görüntüleri ve APK/dosya aktarımları | Çağıranın sağladığı adb (emülatör veya cihaz); [ADB rehberi](docs/adb-device-analysis.md) | -| JEB projeleri | Motor destekli proje birimleri, artefakt özetleri ve tür/metot sözdekodu | Çağıranın başlattığı, MCP sunan bir JEB istemcisi, REA_JEB_MCP_URL ile seçilir; [JEB rehberi](docs/jeb-analysis.md) | -| Android kaynakları | Çözülmüş manifest, dize tabloları, diller ve sürüm bilgileri | Çağıranın sağladığı Apktool başlatıcısı; [kaynak rehberi](docs/apktool-resource-analysis.md) | -| Flutter uygulamaları | Dart anlık görüntü karması, ABI başına kitaplık özetleri ve motor derleme kökeni | Yok (statik APK çözümleme); [Flutter rehberi](docs/flutter-analysis.md) | -| Donanım yazılımı | Bölgeler, çıkarma sonuçları ve yerel kod analizine devirler | Linux üzerinde Binwalk / Unblob; [donanım yazılımı rehberi](docs/firmware-analysis.md) | -| Kaydedilmiş paketler | Seçilen alanlar, yinelenen değerler, filtreler ve çözümleme kuralları | Linux: TShark 4.x + prlimit; [yakalama kılavuzu](docs/packet-captures.md) | -| Kaydedilmiş mantık sinyalleri | Kod çözücü açıklamaları, örnek aralıkları ve tanılama | Linux: sigrok-cli 0.7.2 + prlimit; [sinyal kılavuzu](docs/signal-recordings.md) | -| Paketler ve kaynaklar | Dosya envanterleri, özet değerleri, plist'ler, Apple paket yapısı ve çıkarılmış kaynaklar | [Artefakt ve JavaScript rehberi](docs/javascript-artifact-reconstruction.md), [Apple uygulamaları](docs/apple-application-analysis.md) | -| Süreç davranışı | Terminal çıktısı, etkileşimler, sonlanma ve dosya sistemi gözlemleri, çalıştırma karşılaştırmaları | Yerel PTY ile Linux/macOS; [süreç yakalama](docs/process-capture.md) | +| Hedef | REA'nın döndürdükleri | Gereksinimler ve rehber | +| --------------------------------------- | ----------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | +| Yerel makine kodu içeren ikili dosyalar | Sözde kod, assembly kodu, dizeler, semboller, çağrılar ve referanslar | Hopper, Ghidra veya IDA; [yerel kod analizi](https://rea.tools/tr/guides/native/) | +| Çevrimdışı ELF yapısı | Bölümler, segmentler, özgün semboller/yeniden konumlandırmalar ve statik analizden olası güvenlik önlemleri | Linux x64 üzerinde çağıranın sağladığı pwntools; [ikili dosya tanılama](../binary-diagnostics.md) | +| Go ikili dosyaları | Gömülü compiler sürümü, modüller, replacement'lar ve build ayarları | Yerleşik statik ELF/PE/thin Mach-O incelemesi; Go kurulumu gerekmez | +| WASM artefaktı | Doğrulama, bölümler, tam import/export WAT biçimleri ve decode edilmiş WAT | Çağıranın sağladığı WABT 1.0.42; `inspect_wasm_artifact` | +| EVM bayt kodu | Yönlendirme seçicileri, bayt ofsetleri, çıkarımla belirlenen argümanlar ve durum değiştirilebilirliği | Linux x64 üzerinde yerel ham bayt/onaltılık girdi; [çevrimdışı EVM rehberi](../evm-bytecode.md) | +| SQLite veritabanları | Tablolar, sütunlar, index'ler, view'lar, trigger'lar ve seçilen sıradan tablo satırları | Yerel database/WAL snapshot'ı; native SQLite sınırlarıyla Node.js 24.x >=24.15 veya 26+ | +| Kaydedilmiş Linux çökmeleri | Ham note kayıtları, kaydedilen her iş parçacığının yazmaçları/sinyalleri ve isteğe bağlı eşleme adayları | Linux x64 üzerinde çağıranın sağladığı pwntools; isteğe bağlı GDB/pwndbg; [kaydedilmiş çökmeler](../recorded-crashes.md) | +| JavaScript / Electron | Modüller, içe aktarmalar, kaynak haritaları, rotalar, IPC ve yerel eklenti ilişkileri | Node.js ve npm; [uygulama analizi](https://rea.tools/tr/guides/javascript/) | +| Web siteleri | Sayfa yapısı, betikler, ağ gözlemleri ve istenen ekran görüntüleri | Chrome ailesinden bir tarayıcı; [tarayıcı analizi](https://rea.tools/tr/guides/browser/) | +| Kaydedilmiş ağ yakalamaları | İstekler, yanıtlar, erişilebilir yük içerikleri ve kaynak konumları | HAR veya Fiddler Everywhere SAZ; Linux'ta native mitmproxy için mitmdump; [ağ yakalama rehberi](../web-network-captures.md) | +| .NET derlemeleri | Meta veriler, CIL talimatları, bildirilmiş yerel bağımlılıklar ve derleme karşılaştırmaları | Statik inceleme; [yönetilen kod rehberi](../managed-code-analysis.md) | +| Android APK'leri | Manifest bildirimleri, sınıflar, tersine derlenmiş metotlar ve referanslar | Linux/macOS/Windows x64 üzerinde arayüzsüz JADX ve tam bir JDK; [Android rehberi](../android-analysis.md) | +| Android cihazları | Cihazlar, paketler, süreçler, günlükler, dökümler, ekran görüntüleri ve APK/dosya aktarımları | Çağıranın sağladığı adb (emülatör veya cihaz); [ADB rehberi](../adb-device-analysis.md) | +| JEB projeleri | Motor destekli proje birimleri, artefakt özetleri ve tür/metot sözdekodu | Çağıranın başlattığı, MCP sunan bir JEB istemcisi, REA_JEB_MCP_URL ile seçilir; [JEB rehberi](../jeb-analysis.md) | +| Android kaynakları | Çözülmüş manifest, dize tabloları, diller ve sürüm bilgileri | Çağıranın sağladığı Apktool başlatıcısı; [kaynak rehberi](../apktool-resource-analysis.md) | +| Flutter uygulamaları | Dart anlık görüntü karması, ABI başına kitaplık özetleri ve motor derleme kökeni | Yok (statik APK çözümleme); [Flutter rehberi](../flutter-analysis.md) | +| Donanım yazılımı | Bölgeler, çıkarma sonuçları ve yerel kod analizine devirler | Linux üzerinde Binwalk / Unblob; [donanım yazılımı rehberi](../firmware-analysis.md) | +| Kaydedilmiş paketler | Seçilen alanlar, yinelenen değerler, filtreler ve çözümleme kuralları | Linux: TShark 4.x + prlimit; [yakalama kılavuzu](../packet-captures.md) | +| Paket özetleri | Uç noktalar, DNS, TLS SNI, protokol dağılımı ve HTTP istekleri | Çağıranın sağladığı tshark ve capinfos (Linux üzerinde doğrulandı); [pcap rehberi](../pcap-analysis.md) | +| Kaydedilmiş mantık sinyalleri | Kod çözücü açıklamaları, örnek aralıkları ve tanılama | Linux: sigrok-cli 0.7.2 + prlimit; [sinyal kılavuzu](../signal-recordings.md) | +| Paketler ve kaynaklar | Dosya envanterleri, özet değerleri, plist'ler, Apple paket yapısı ve çıkarılmış kaynaklar | [Artefakt ve JavaScript rehberi](../javascript-artifact-reconstruction.md), [Apple uygulamaları](../apple-application-analysis.md) | +| Süreç davranışı | Terminal çıktısı, etkileşimler, sonlanma ve dosya sistemi gözlemleri, çalıştırma karşılaştırmaları | Yerel PTY ile Linux/macOS; [süreç yakalama](../process-capture.md) | Statik JavaScript ve .NET incelemesi, uygulamayı çalıştırmadan sağlanan dosyaları okur. Çalışma zamanı yakalaması, seçilen hedefi kullanıcı izinlerinizle çalıştırır veya onunla etkileşime girer; her çalışma zamanı rehberi etkilerini açıklar. -Yerel makine kodu içeren ikili dosya biçimleri ve ana bilgisayar desteği sağlayıcıya göre değişir. [Hopper ve Ghidra kurulumu](docs/installation.md#hopper), [IDA rehberi](docs/ida-provider.md) ve [Windows için deneysel Ghidra desteği](docs/windows-ghidra-p0.md) belgelerine bakın. Ghidra ayrıca [16 bit DOS analizini](docs/ghidra-dos.md) destekler. Sağlayıcı seçimi için [CLI rehberine](docs/cli.md#choose-a-provider) bakın. En son npm sürümünden sonra eklenen özellikler için [sürümlerde kullanılabilirliği](docs/installation.md#released-package-and-main) kontrol edin. +Yerel makine kodu içeren ikili dosya biçimleri ve ana bilgisayar desteği sağlayıcıya göre değişir. [Hopper ve Ghidra kurulumu](../installation.md#hopper), isteğe bağlı native MCP bridge'i de açıklayan [IDA rehberi](../ida-provider.md) ve [Windows için deneysel Ghidra desteği](../windows-ghidra-p0.md) belgelerine bakın. Ghidra ayrıca [16 bit DOS analizini](../ghidra-dos.md) destekler. Büyük ikili dosyalar için sağlayıcının başlatma zaman aşımını `REA_GHIDRA_STARTUP_TIMEOUT_MS` veya `REA_HOPPER_STARTUP_TIMEOUT_MS` ile artırın. Sağlayıcı seçimi için [CLI rehberine](../cli.md#choose-a-provider) bakın. En son npm sürümünden sonra eklenen özellikler için [sürümlerde kullanılabilirliği](../installation.md#released-package-and-main) kontrol edin. ## Örnek çalışmalar -[![DX-Ball ses kaydırma, Notion pano köprüsü ve TH04 mermi halkası örneklerinin çizimleri](docs/assets/rea-showcases.png)](https://rea.tools/tr/showcase/) +[![DX-Ball ses kaydırma, Notion pano köprüsü ve TH04 mermi halkası örneklerinin çizimleri](../assets/rea-showcases.png)](https://rea.tools/tr/showcase/) ### DX-Ball: Sesin stereo konumlandırma hesabını yeniden oluşturma @@ -187,28 +192,28 @@ REA'yı ilginç bir konuda kullandıysanız görmek isteriz. Hedefi, sorunuzu, R
Hangi ajanlar REA'yı kullanabilir? -Yerel MCP sunucularını destekleyen her ajan REA'yı kullanabilir. Kurulum, [desteklenen ajanları](docs/installation.md#supported-agents) yapılandırır; diğer istemciler [elle MCP kaydını](docs/installation.md#mcp-registry) kullanabilir. +Yerel MCP sunucularını destekleyen her ajan REA'yı kullanabilir. Kurulum, [desteklenen ajanları](../installation.md#supported-agents) yapılandırır; diğer istemciler [elle MCP kaydını](../installation.md#mcp-registry) kullanabilir.
Hopper, Ghidra veya IDA gerekiyor mu? -Derin yerel kod analizi bunlardan birini kullanır. Statik JavaScript ve .NET incelemesi yerel analiz motoru olmadan çalışır. Kurulum onaydan sonra Hopper yükleyebilir; Ghidra ve IDA mevcut kurulumlarınızı kullanır. [Sağlayıcı kurulumuna](docs/installation.md#hopper) bakın. +Derin yerel kod analizi bunlardan birini kullanır. Statik JavaScript ve .NET incelemesi yerel analiz motoru olmadan çalışır. Kurulum onaydan sonra Hopper yükleyebilir; Ghidra ve IDA mevcut kurulumlarınızı kullanır. [Sağlayıcı kurulumuna](../installation.md#hopper) bakın.
Önce Hopper'ı başlatmam gerekiyor mu? -REA, bir işlem gerektirdiğinde Hopper'ı başlatır. macOS'ta ilk çalıştırmada demo modunu seçmenizi veya lisansınızı etkinleştirmenizi isteyen bir iletişim kutusu açılabilir. [Hopper başlatma ve sorun giderme](docs/installation.md#launcher-paths-and-troubleshooting) belgesine bakın. +REA, bir işlem gerektirdiğinde Hopper'ı başlatır. macOS'ta ilk çalıştırmada demo modunu seçmenizi veya lisansınızı etkinleştirmenizi isteyen bir iletişim kutusu açılabilir. [Hopper başlatma ve sorun giderme](../installation.md#launcher-paths-and-troubleshooting) belgesine bakın.
skills.sh üzerinden skill yüklemek ne yapar? -Skill, ajanınıza araştırma talimatları sağlar. REA'nın MCP sunucusunu kaydetmek ve uyumlu talimatları yüklemek için `npx rea-agents setup` kullanın, ardından ajanınızı yeniden başlatın. [Yalnızca skill yükleme](docs/installation.md#skill-only-installation) belgesine bakın. +Skill, ajanınıza araştırma talimatları sağlar. REA'nın MCP sunucusunu kaydetmek ve uyumlu talimatları yüklemek için `npx rea-agents setup` kullanın, ardından ajanınızı yeniden başlatın. [Yalnızca skill yükleme](../installation.md#skill-only-installation) belgesine bakın.
@@ -251,14 +256,14 @@ Ajan kullanıyorsanız [kurulum yenilemesini](#reayı-güncelleyin) tamamlayın Web sitesindeki [uygulamalı rehberlerden](https://rea.tools/tr/guides/) başlayın. Kesin seçenekler, ön koşullar ve sonuç sözleşmeleri için: -- [Yükleme ve kurulum](docs/installation.md): ajan kaydı, sağlayıcı yapılandırması, güncellemeler ve kaldırma. -- [Hazırlık ve sorun giderme](docs/installation.md#check-readiness-for-your-task): belirli bir ajanı veya analiz motorunu tanılama. -- [CLI ve Evidence](docs/cli.md): komutlar, sağlayıcı seçimi, anlık görüntüler, içe/dışa aktarma ve çıkış durumları. -- [MCP sözleşmeleri](docs/mcp-contracts.md) ve [ajan istemleri](docs/mcp-prompts.md): araç sonuçları, oturumlar ve yönlendirmeli araştırmalar. -- [Araç kataloğu](docs/mcp-contracts.md#generated-catalog): derleme sırasında oluşturulan araç, sağlayıcı ve CLI komutu envanteri. -- [Yol haritası](docs/roadmap.md): planlanan çalışmalar ve yetenek takibi. +- [Yükleme ve kurulum](../installation.md): ajan kaydı, sağlayıcı yapılandırması, güncellemeler ve kaldırma. +- [Hazırlık ve sorun giderme](../installation.md#check-readiness-for-your-task): belirli bir ajanı veya analiz motorunu tanılama. +- [CLI ve Evidence](../cli.md): komutlar, sağlayıcı seçimi, anlık görüntüler, içe/dışa aktarma ve çıkış durumları. +- [MCP sözleşmeleri](../mcp-contracts.md) ve [ajan istemleri](../mcp-prompts.md): araç sonuçları, oturumlar ve yönlendirmeli araştırmalar. +- [Araç kataloğu](../mcp-contracts.md#generated-catalog): derleme sırasında oluşturulan araç, sağlayıcı ve CLI komutu envanteri. +- [Yol haritası](../roadmap.md): planlanan çalışmalar ve yetenek takibi. -Güvenlik açıklarını [SECURITY.md](SECURITY.md) üzerinden bildirin. +Güvenlik açıklarını [SECURITY.md](../../SECURITY.md) üzerinden bildirin. ## Yıldız geçmişi @@ -276,32 +281,34 @@ REA'yı kullanan, hata bildiren, özellik talep eden, derlemeleri test eden ve d ## Sorumluluk reddi -Sorumlu kullanım yönergeleri için [DISCLAIMER.md](DISCLAIMER.md) dosyasına bakın. +Sorumlu kullanım yönergeleri için [DISCLAIMER.md](../../DISCLAIMER.md) dosyasına bakın. REA; yasal araştırma, eğitim, birlikte çalışabilirlik, yazılımın korunması ve sahibi olduğunuz ya da inceleme yetkiniz bulunan yazılımların analizi için tasarlanmıştır. -The project's position on illegal or unauthorized use covers all REA -versions, distribution channels, and platforms; each copy retains its -applicable licenses. +Projenin yasa dışı veya yetkisiz kullanıma ilişkin tutumu tüm REA sürümleri, +dağıtım kanalları ve platformlar için geçerlidir; her kopya geçerli lisanslarını +korur. ## Katkıda bulunma REA'ya katkınızı bekliyoruz! Hata bildirmek veya özellik önermek için bir [issue açın](https://github.com/morluto/rea/issues); kodu veya belgeleri geliştirmek için bir [pull request gönderin](https://github.com/morluto/rea/pulls). -Geliştirme ortamı ve kontroller için [CONTRIBUTING.md](CONTRIBUTING.md), doğrulama süreçleri için [test rehberi](docs/testing.md) ve proje yapısı için [mimari harita](docs/architecture.mermaid) belgelerine bakın. +Geliştirme ortamı ve kontroller için [CONTRIBUTING.md](../../CONTRIBUTING.md), doğrulama süreçleri için [test rehberi](../testing.md) ve proje yapısı için [mimari harita](../architecture.mermaid) belgelerine bakın. ## Lisans -[MIT](LICENSE) +[MIT](../../LICENSE) -REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) -for the dependency inventory, preserved license texts, and external-tool terms. +REA tarafından yazılan kod MIT lisansı altında lisanslanır. Üçüncü taraf +materyaller kendi lisanslarını ve telif hakkı bildirimlerini korur; bağımlılık +envanteri, korunan lisans metinleri ve harici araç koşulları için [Üçüncü taraf +bildirimleri](../../THIRD_PARTY_NOTICES.md) dosyasına bakın. -**You must legally obtain and be authorized to use commercial tools**, including -IDA Pro / Hex-Rays decompilers, Hopper, and JEB, under their applicable licenses. -REA grants no tool license, key, or activation entitlement. Trials and demos -remain subject to vendor terms, including user, feature, automation, and output -sharing restrictions. Cracked copies, stolen keys, unauthorized key sharing, -and bypassing license restrictions are not authorized. This guidance does not -amend MIT or third-party licenses. +**IDA Pro / Hex-Rays decompiler'ları, Hopper ve JEB dahil olmak üzere ticari +araçları yasal olarak edinmeli ve ilgili lisanslar kapsamında kullanma +yetkiniz olmalıdır.** REA hiçbir araç lisansı, anahtarı veya etkinleştirme hakkı +vermez. Deneme sürümleri ve demolar; kullanıcı, özellik, otomasyon ve çıktı +paylaşımı kısıtlamaları dahil olmak üzere satıcı koşullarına tabidir. Kırılmış +kopyalar, çalınmış anahtarlar, yetkisiz anahtar paylaşımı ve lisans +kısıtlamalarını aşma yetkili kullanım kapsamında değildir. Bu yönlendirme MIT +veya üçüncü taraf lisanslarını değiştirmez. diff --git a/README_uk.md b/docs/readme/README_uk.md similarity index 80% rename from README_uk.md rename to docs/readme/README_uk.md index 14c3d4984..5c77d80b0 100644 --- a/README_uk.md +++ b/docs/readme/README_uk.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · **Українська** · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · **Українська** · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA: Реверс-інжиніринг будь-чого @@ -9,10 +9,10 @@ **Побачили цікаву функцію? Зрозумійте, як вона працює, аж до рівня двійкового коду.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA запускає міст аналізу всередині Hopper під час дослідження нативного бінарного файла +REA запускає міст аналізу всередині Hopper під час дослідження нативного бінарного файла

@@ -33,7 +33,7 @@ - Discord
+ Discord
Приєднуйтеся до спільноти реверс-інжинірингу
Discord · Запитання й відповіді · Обмін результатами @@ -67,7 +67,7 @@ npx rea-agents setup Виберіть агентів, перевірте запропоновані зміни та схваліть їх. Налаштування додає MCP-сервер REA та відповідні інструкції робочих процесів, створюючи резервні копії наявної конфігурації. Після цього перезапустіть агента. -Налаштування підтримує Claude Code, Codex, Cursor, Gemini CLI, Grok Build та [інших агентів](docs/installation.md#supported-agents). Конфігурацію провайдерів і ручну реєстрацію MCP описано в розділі [встановлення й налаштування](docs/installation.md). +Налаштування підтримує Claude Code, Codex, Cursor, Gemini CLI, Grok Build та [інших агентів](../installation.md#supported-agents). Конфігурацію провайдерів і ручну реєстрацію MCP описано в розділі [встановлення й налаштування](../installation.md). ### Запитайте агента @@ -94,7 +94,7 @@ npm install --global rea-agents rea --help ``` -Для нативного аналізу спочатку налаштуйте провайдера. Нативні команди, вибір провайдера, знімки та використання у скриптах описує [посібник CLI та Evidence](docs/cli.md). +Для нативного аналізу спочатку налаштуйте провайдера. Нативні команди, вибір провайдера, знімки та використання у скриптах описує [посібник CLI та Evidence](../cli.md). ### Оновіть REA @@ -120,9 +120,9 @@ npx rea-agents@latest setup Агент викликає REA через MCP, щоб перевірити ціль і простежити пов'язаний код. REA повертає знахідки разом із підтвердженнями. Агент використовує їх для уточнювальних запитань, пояснення поведінки або написання й тестування реалізації. Команди CLI використовують ті самі робочі процеси. -![Процес дослідження з REA: агент запитує про локальну ціль, REA перевіряє й простежує її інструментами аналізу, а агент використовує повернений код, посилання та невідомі дані для пояснення, реалізації й тестування.](website/public/assets/figures/rea-investigation-flow.svg) +![Процес дослідження з REA: агент запитує про локальну ціль, REA перевіряє й простежує її інструментами аналізу, а агент використовує повернений код, посилання та невідомі дані для пояснення, реалізації й тестування.](../../website/public/assets/figures/rea-investigation-flow.svg) -[Відкрити зображення в повному розмірі](website/public/assets/figures/rea-investigation-flow.svg). +[Відкрити зображення в повному розмірі](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -130,37 +130,37 @@ npx rea-agents@latest setup REA потребує Node.js 22.x (>=22.19), 24.x (>=24.11) або 26+, а також npm. Додаткові інструменти й підтримувані системи залежать від цілі: -| Ціль | Що повертає REA | Вимоги й посібник | -| ------------------------------------ | ----------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | -| Нативні бінарні файли | Псевдокод, асемблерний код, рядки, символи, виклики й посилання | Hopper, Ghidra або IDA; [нативний аналіз](https://rea.tools/uk/guides/native/) | -| Структура ELF без запуску | Секції, сегменти, початкові символи/релокації та можливі захисні механізми за даними статичного аналізу | pwntools, наданий стороною, що викликає інструмент, у Linux x64; [діагностика бінарних файлів](docs/binary-diagnostics.md) | -| Байт-код EVM | Селектори диспетчеризації, зміщення байтів, виведені припущення про аргументи та змінюваність стану | Локальні необроблені байти або шістнадцяткове представлення у Linux x64; [посібник EVM без запуску](docs/evm-bytecode.md) | -| Записані аварійні завершення в Linux | Необроблені записи note, регістри/сигнали кожного записаного потоку й необов'язкові кандидати зіставлення пам'яті | pwntools, наданий стороною, що викликає інструмент, у Linux x64; необов'язкові GDB/pwndbg; [записані аварійні завершення](docs/recorded-crashes.md) | -| JavaScript / Electron | Модулі, імпорти, карти вихідного коду, маршрути, IPC і зв'язки з нативними доповненнями | Node.js і npm; [аналіз застосунків](https://rea.tools/uk/guides/javascript/) | -| Сайти | Структура сторінки, скрипти, мережеві спостереження й запитані знімки екрана | Браузер сімейства Chrome; [аналіз браузера](https://rea.tools/uk/guides/browser/) | -| Збережені мережеві захоплення | Запити, відповіді, доступні корисні дані та місця в джерелі | HAR; mitmdump у Linux для захоплень у власному форматі mitmproxy; [посібник мережевих захоплень](docs/web-network-captures.md) | -| Захоплення пакетів | Кінцеві точки, DNS, TLS SNI, розподіл протоколів та HTTP-запити | Надані викликаючою стороною tshark та capinfos; [підручник pcap](docs/pcap-analysis.md) | -| Збірки .NET | Метадані, інструкції CIL, оголошені нативні залежності й порівняння збірок | Статична перевірка; [посібник керованого коду](docs/managed-code-analysis.md) | -| Android APK | Оголошення маніфесту, класи, декомпільовані методи й посилання | JADX без графічного інтерфейсу й повний JDK у Linux/macOS/Windows x64; [посібник Android](docs/android-analysis.md) | -| Пристрої Android | Пристрої, пакети, процеси, журнали, дампи, знімки екрана та передавання APK/файлів | adb, наданий викликаючою стороною (емулятор або пристрій); [підручник ADB](docs/adb-device-analysis.md) | -| Проєкти JEB | Єдиниці проєкту від рушія, дайджести артефактів і псевдокод типів/методів | Запущений викликаною стороною клієнт JEB з MCP, обирається через REA_JEB_MCP_URL; [посібник JEB](docs/jeb-analysis.md) | -| Ресурси Android | Декодований маніфест, таблиці рядків, мови та відомості про версію | Засіб запуску Apktool, наданий викликаючою стороною; [підручник ресурсів](docs/apktool-resource-analysis.md) | -| Додатки Flutter | Хеш знімка Dart, дайджести бібліотек за ABI та походження збірки рушія | Не потрібні (статичний розбір APK); [підручник Flutter](docs/flutter-analysis.md) | -| Прошивки | Області, результати видобування й передавання до нативного аналізу | Binwalk / Unblob у Linux; [посібник прошивок](docs/firmware-analysis.md) | -| Записані пакети | Вибрані поля, повторювані значення, фільтри та правила декодування | Linux: TShark 4.x + prlimit; [посібник із записів пакетів](docs/packet-captures.md) | -| Записані логічні сигнали | Анотації декодера, діапазони відліків і діагностика | Linux: sigrok-cli 0.7.2 + prlimit; [посібник із сигналів](docs/signal-recordings.md) | -| Пакети й ресурси | Переліки файлів, хеші, plist, структура пакетів Apple та видобуті ресурси | [Посібник артефактів і JavaScript](docs/javascript-artifact-reconstruction.md), [застосунки Apple](docs/apple-application-analysis.md) | -| Поведінка процесів | Вивід термінала, взаємодії, спостереження за завершенням і файловою системою та порівняння запусків | Linux/macOS із нативним PTY; [захоплення процесів](docs/process-capture.md) | +| Ціль | Що повертає REA | Вимоги й посібник | +| ------------------------------------ | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | +| Нативні бінарні файли | Псевдокод, асемблерний код, рядки, символи, виклики й посилання | Hopper, Ghidra або IDA; [нативний аналіз](https://rea.tools/uk/guides/native/) | +| Структура ELF без запуску | Секції, сегменти, початкові символи/релокації та можливі захисні механізми за даними статичного аналізу | pwntools, наданий стороною, що викликає інструмент, у Linux x64; [діагностика бінарних файлів](../binary-diagnostics.md) | +| Байт-код EVM | Селектори диспетчеризації, зміщення байтів, виведені припущення про аргументи та змінюваність стану | Локальні необроблені байти або шістнадцяткове представлення у Linux x64; [посібник EVM без запуску](../evm-bytecode.md) | +| Записані аварійні завершення в Linux | Необроблені записи note, регістри/сигнали кожного записаного потоку й необов'язкові кандидати зіставлення пам'яті | pwntools, наданий стороною, що викликає інструмент, у Linux x64; необов'язкові GDB/pwndbg; [записані аварійні завершення](../recorded-crashes.md) | +| JavaScript / Electron | Модулі, імпорти, карти вихідного коду, маршрути, IPC і зв'язки з нативними доповненнями | Node.js і npm; [аналіз застосунків](https://rea.tools/uk/guides/javascript/) | +| Сайти | Структура сторінки, скрипти, мережеві спостереження й запитані знімки екрана | Браузер сімейства Chrome; [аналіз браузера](https://rea.tools/uk/guides/browser/) | +| Збережені мережеві захоплення | Запити, відповіді, доступні корисні дані та місця в джерелі | HAR; mitmdump у Linux для захоплень у власному форматі mitmproxy; [посібник мережевих захоплень](../web-network-captures.md) | +| Захоплення пакетів | Кінцеві точки, DNS, TLS SNI, розподіл протоколів та HTTP-запити | Надані викликаючою стороною tshark та capinfos; [підручник pcap](../pcap-analysis.md) | +| Збірки .NET | Метадані, інструкції CIL, оголошені нативні залежності й порівняння збірок | Статична перевірка; [посібник керованого коду](../managed-code-analysis.md) | +| Android APK | Оголошення маніфесту, класи, декомпільовані методи й посилання | JADX без графічного інтерфейсу й повний JDK у Linux/macOS/Windows x64; [посібник Android](../android-analysis.md) | +| Пристрої Android | Пристрої, пакети, процеси, журнали, дампи, знімки екрана та передавання APK/файлів | adb, наданий викликаючою стороною (емулятор або пристрій); [підручник ADB](../adb-device-analysis.md) | +| Проєкти JEB | Єдиниці проєкту від рушія, дайджести артефактів і псевдокод типів/методів | Запущений викликаною стороною клієнт JEB з MCP, обирається через REA_JEB_MCP_URL; [посібник JEB](../jeb-analysis.md) | +| Ресурси Android | Декодований маніфест, таблиці рядків, мови та відомості про версію | Засіб запуску Apktool, наданий викликаючою стороною; [підручник ресурсів](../apktool-resource-analysis.md) | +| Додатки Flutter | Хеш знімка Dart, дайджести бібліотек за ABI та походження збірки рушія | Не потрібні (статичний розбір APK); [підручник Flutter](../flutter-analysis.md) | +| Прошивки | Області, результати видобування й передавання до нативного аналізу | Binwalk / Unblob у Linux; [посібник прошивок](../firmware-analysis.md) | +| Записані пакети | Вибрані поля, повторювані значення, фільтри та правила декодування | Linux: TShark 4.x + prlimit; [посібник із записів пакетів](../packet-captures.md) | +| Записані логічні сигнали | Анотації декодера, діапазони відліків і діагностика | Linux: sigrok-cli 0.7.2 + prlimit; [посібник із сигналів](../signal-recordings.md) | +| Пакети й ресурси | Переліки файлів, хеші, plist, структура пакетів Apple та видобуті ресурси | [Посібник артефактів і JavaScript](../javascript-artifact-reconstruction.md), [застосунки Apple](../apple-application-analysis.md) | +| Поведінка процесів | Вивід термінала, взаємодії, спостереження за завершенням і файловою системою та порівняння запусків | Linux/macOS із нативним PTY; [захоплення процесів](../process-capture.md) | Статична перевірка JavaScript і .NET читає надані файли, не запускаючи застосунок. Захоплення під час виконання запускає вибрану ціль або взаємодіє з нею з правами вашого користувача; відповідні посібники описують його вплив. -Нативні формати й підтримка систем відрізняються залежно від провайдера. Дивіться [налаштування Hopper і Ghidra](docs/installation.md#hopper), [посібник IDA](docs/ida-provider.md) та [експериментальну підтримку Ghidra у Windows](docs/windows-ghidra-p0.md). Ghidra також підтримує [аналіз 16-бітного DOS](docs/ghidra-dos.md). Для вибору провайдера дивіться [посібник CLI](docs/cli.md#choose-a-provider). Для функцій, доданих після найновішої версії в npm, перевірте [доступність у випусках](docs/installation.md#released-package-and-main). +Нативні формати й підтримка систем відрізняються залежно від провайдера. Дивіться [налаштування Hopper і Ghidra](../installation.md#hopper), [посібник IDA](../ida-provider.md) та [експериментальну підтримку Ghidra у Windows](../windows-ghidra-p0.md). Ghidra також підтримує [аналіз 16-бітного DOS](../ghidra-dos.md). Для вибору провайдера дивіться [посібник CLI](../cli.md#choose-a-provider). Для функцій, доданих після найновішої версії в npm, перевірте [доступність у випусках](../installation.md#released-package-and-main). ## Приклади -[![Ілюстрації прикладів панорамування звуку в DX-Ball, мосту буфера обміну Notion та кільця куль у TH04](docs/assets/rea-showcases.png)](https://rea.tools/uk/showcase/) +[![Ілюстрації прикладів панорамування звуку в DX-Ball, мосту буфера обміну Notion та кільця куль у TH04](../assets/rea-showcases.png)](https://rea.tools/uk/showcase/) ### DX-Ball: відновлення розрахунку звукової панорами @@ -187,28 +187,28 @@ REA потребує Node.js 22.x (>=22.19), 24.x (>=24.11) або 26+, а та
Які агенти можуть використовувати REA? -Будь-який агент, що підтримує локальні MCP-сервери. Налаштування конфігурує [підтримуваних агентів](docs/installation.md#supported-agents); інші клієнти можуть використати [ручну реєстрацію MCP](docs/installation.md#mcp-registry). +Будь-який агент, що підтримує локальні MCP-сервери. Налаштування конфігурує [підтримуваних агентів](../installation.md#supported-agents); інші клієнти можуть використати [ручну реєстрацію MCP](../installation.md#mcp-registry).
Чи потрібні мені Hopper, Ghidra або IDA? -Поглиблений нативний аналіз використовує один із цих інструментів. Статична перевірка JavaScript і .NET працює без рушія нативного аналізу. Налаштування може встановити Hopper після схвалення; Ghidra й IDA використовують наявні встановлення. Дивіться [налаштування провайдерів](docs/installation.md#hopper). +Поглиблений нативний аналіз використовує один із цих інструментів. Статична перевірка JavaScript і .NET працює без рушія нативного аналізу. Налаштування може встановити Hopper після схвалення; Ghidra й IDA використовують наявні встановлення. Дивіться [налаштування провайдерів](../installation.md#hopper).
Чи потрібно спочатку запускати Hopper? -REA запускає Hopper, коли він потрібен для операції. У macOS під час першого запуску може з'явитися діалог вибору деморежиму або активації ліцензії. Дивіться [запуск Hopper та усунення неполадок](docs/installation.md#launcher-paths-and-troubleshooting). +REA запускає Hopper, коли він потрібен для операції. У macOS під час першого запуску може з'явитися діалог вибору деморежиму або активації ліцензії. Дивіться [запуск Hopper та усунення неполадок](../installation.md#launcher-paths-and-troubleshooting).
Що дає встановлення skill із skills.sh? -Skill надає агенту інструкції дослідження. Використовуйте `npx rea-agents setup`, щоб зареєструвати MCP-сервер REA та встановити відповідні інструкції, а потім перезапустіть агента. Дивіться [встановлення лише skill](docs/installation.md#skill-only-installation). +Skill надає агенту інструкції дослідження. Використовуйте `npx rea-agents setup`, щоб зареєструвати MCP-сервер REA та встановити відповідні інструкції, а потім перезапустіть агента. Дивіться [встановлення лише skill](../installation.md#skill-only-installation).
@@ -251,14 +251,14 @@ npx rea-agents@latest setup Почніть із [практичних посібників](https://rea.tools/uk/guides/) на сайті. Точні параметри, передумови й контракти результатів описано тут: -- [Встановлення й налаштування](docs/installation.md): реєстрація агентів, конфігурація провайдерів, оновлення та видалення. -- [Готовність та усунення неполадок](docs/installation.md#check-readiness-for-your-task): діагностика конкретного агента або рушія аналізу. -- [CLI та Evidence](docs/cli.md): команди, вибір провайдера, знімки, імпорт/експорт і статуси завершення. -- [Контракти MCP](docs/mcp-contracts.md) і [промпти агентів](docs/mcp-prompts.md): результати інструментів, сеанси й керовані дослідження. -- [Каталог інструментів](docs/mcp-contracts.md#generated-catalog): створюваний під час збірки перелік інструментів, провайдерів і команд CLI. -- [План розвитку](docs/roadmap.md): заплановані роботи й відстеження можливостей. +- [Встановлення й налаштування](../installation.md): реєстрація агентів, конфігурація провайдерів, оновлення та видалення. +- [Готовність та усунення неполадок](../installation.md#check-readiness-for-your-task): діагностика конкретного агента або рушія аналізу. +- [CLI та Evidence](../cli.md): команди, вибір провайдера, знімки, імпорт/експорт і статуси завершення. +- [Контракти MCP](../mcp-contracts.md) і [промпти агентів](../mcp-prompts.md): результати інструментів, сеанси й керовані дослідження. +- [Каталог інструментів](../mcp-contracts.md#generated-catalog): створюваний під час збірки перелік інструментів, провайдерів і команд CLI. +- [План розвитку](../roadmap.md): заплановані роботи й відстеження можливостей. -Повідомляйте про вразливості згідно з [SECURITY.md](SECURITY.md). +Повідомляйте про вразливості згідно з [SECURITY.md](../../SECURITY.md). ## Історія зірок @@ -276,7 +276,7 @@ npx rea-agents@latest setup ## Відмова від відповідальності -Рекомендації щодо відповідального використання наведено в [DISCLAIMER.md](DISCLAIMER.md). +Рекомендації щодо відповідального використання наведено в [DISCLAIMER.md](../../DISCLAIMER.md). REA призначений для законних досліджень, навчання, забезпечення сумісності, збереження програмного забезпечення й аналізу програм, якими ви володієте або які маєте дозвіл досліджувати. @@ -288,14 +288,14 @@ applicable licenses. Будемо раді вашій допомозі з REA! [Відкрийте issue](https://github.com/morluto/rea/issues), щоб повідомити про помилку чи запропонувати функцію, або [надішліть pull request](https://github.com/morluto/rea/pulls) з поліпшеннями коду чи документації. -Налаштування середовища розробки й перевірки описано в [CONTRIBUTING.md](CONTRIBUTING.md), шляхи верифікації — у [посібнику тестування](docs/testing.md), а структуру проєкту — на [схемі архітектури](docs/architecture.mermaid). +Налаштування середовища розробки й перевірки описано в [CONTRIBUTING.md](../../CONTRIBUTING.md), шляхи верифікації — у [посібнику тестування](../testing.md), а структуру проєкту — на [схемі архітектури](../architecture.mermaid). ## Ліцензія -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_vi.md b/docs/readme/README_vi.md similarity index 75% rename from README_vi.md rename to docs/readme/README_vi.md index 86544a535..3d2062da6 100644 --- a/README_vi.md +++ b/docs/readme/README_vi.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · **Tiếng Việt** · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · **Tiếng Việt** · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA: Kỹ thuật dịch ngược cho mọi thứ @@ -9,10 +9,10 @@ **Thấy một tính năng bạn thích? Hiểu cách nó hoạt động, đến tận mức mã nhị phân.** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA khởi chạy cầu nối phân tích bên trong Hopper khi kiểm tra tệp nhị phân mã máy +REA khởi chạy cầu nối phân tích bên trong Hopper khi kiểm tra tệp nhị phân mã máy

@@ -33,7 +33,7 @@ - Discord
+ Discord
Tham gia cộng đồng kỹ thuật dịch ngược
Discord · Hỏi đáp · Chia sẻ kết quả @@ -67,7 +67,7 @@ npx rea-agents setup Chọn các tác nhân, xem lại những thay đổi được đề xuất và chấp thuận. Quá trình thiết lập thêm máy chủ MCP của REA cùng hướng dẫn quy trình tương ứng, đồng thời sao lưu cấu hình hiện có. Sau đó khởi động lại tác nhân. -Quá trình thiết lập hỗ trợ Claude Code, Codex, Cursor, Gemini CLI, Grok Build và [các tác nhân khác](docs/installation.md#supported-agents). Xem [cài đặt và thiết lập](docs/installation.md) để cấu hình trình phân tích và đăng ký MCP thủ công. +Quá trình thiết lập hỗ trợ Claude Code, Codex, Cursor, Gemini CLI, Grok Build và [các tác nhân khác](../installation.md#supported-agents). Xem [cài đặt và thiết lập](../installation.md) để cấu hình trình phân tích và đăng ký MCP thủ công. ### Hỏi tác nhân @@ -94,7 +94,7 @@ npm install --global rea-agents rea --help ``` -Để phân tích mã máy, trước tiên hãy cấu hình trình phân tích. Xem [hướng dẫn CLI và Evidence](docs/cli.md) về các lệnh phân tích mã máy, lựa chọn trình phân tích, ảnh chụp trạng thái và sử dụng trong script. +Để phân tích mã máy, trước tiên hãy cấu hình trình phân tích. Xem [hướng dẫn CLI và Evidence](../cli.md) về các lệnh phân tích mã máy, lựa chọn trình phân tích, ảnh chụp trạng thái và sử dụng trong script. ### Cập nhật REA @@ -120,9 +120,9 @@ Xem lại thay đổi cấu hình và khởi động lại tác nhân. Với l Tác nhân gọi REA qua MCP để kiểm tra mục tiêu và lần theo mã liên quan. REA trả về các phát hiện kèm bằng chứng. Tác nhân dùng chúng để đặt câu hỏi tiếp theo, giải thích hành vi hoặc viết và kiểm thử một cách triển khai. Các lệnh CLI sử dụng cùng quy trình. -![Quy trình điều tra của REA: tác nhân hỏi về mục tiêu cục bộ, REA kiểm tra và lần theo mục tiêu bằng công cụ phân tích, rồi tác nhân dùng mã, tham chiếu và những điểm chưa biết được trả về để giải thích, triển khai và kiểm thử.](website/public/assets/figures/rea-investigation-flow.svg) +![Quy trình điều tra của REA: tác nhân hỏi về mục tiêu cục bộ, REA kiểm tra và lần theo mục tiêu bằng công cụ phân tích, rồi tác nhân dùng mã, tham chiếu và những điểm chưa biết được trả về để giải thích, triển khai và kiểm thử.](../../website/public/assets/figures/rea-investigation-flow.svg) -[Mở hình ở kích thước đầy đủ](website/public/assets/figures/rea-investigation-flow.svg). +[Mở hình ở kích thước đầy đủ](../../website/public/assets/figures/rea-investigation-flow.svg). @@ -130,37 +130,37 @@ Tác nhân gọi REA qua MCP để kiểm tra mục tiêu và lần theo mã li REA yêu cầu Node.js 22.x (>=22.19), 24.x (>=24.11) hoặc 26+, cùng với npm. Công cụ bổ sung và nền tảng chạy REA được hỗ trợ tùy thuộc vào mục tiêu: -| Mục tiêu | REA trả về gì | Yêu cầu và hướng dẫn | -| ----------------------------- | --------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | -| Tệp nhị phân mã máy | Mã giả, hợp ngữ, chuỗi, ký hiệu, lời gọi và tham chiếu | Hopper, Ghidra hoặc IDA; [phân tích mã máy](https://rea.tools/vi/guides/native/) | -| Bố cục ELF ngoại tuyến | Các section, segment, ký hiệu/thông tin tái định vị gốc và các biện pháp giảm thiểu có thể có từ phân tích tĩnh | pwntools do bên gọi cung cấp trên Linux x64; [chẩn đoán tệp nhị phân](docs/binary-diagnostics.md) | -| Bytecode EVM | Bộ chọn điều phối, độ lệch byte, tham số suy luận và khả năng thay đổi trạng thái | Đầu vào cục bộ chứa byte thô hoặc dạng thập lục phân trên Linux x64; [hướng dẫn EVM ngoại tuyến](docs/evm-bytecode.md) | -| Sự cố Linux đã ghi lại | Bản ghi note thô, thanh ghi/tín hiệu của mọi luồng được ghi lại và các ứng viên ánh xạ tùy chọn | pwntools do bên gọi cung cấp trên Linux x64; GDB/pwndbg tùy chọn; [sự cố đã ghi lại](docs/recorded-crashes.md) | -| JavaScript / Electron | Mô-đun, các khai báo import, source map, tuyến, IPC và quan hệ với tiện ích mở rộng mã máy | Node.js và npm; [phân tích ứng dụng](https://rea.tools/vi/guides/javascript/) | -| Trang web | Cấu trúc trang, script, quan sát mạng và ảnh chụp màn hình được yêu cầu | Trình duyệt thuộc họ Chrome; [phân tích trình duyệt](https://rea.tools/vi/guides/browser/) | -| Bản ghi lưu lượng mạng đã lưu | Yêu cầu, phản hồi, payload có thể truy cập và vị trí nguồn | HAR; mitmdump trên Linux cho bản ghi định dạng gốc của mitmproxy; [hướng dẫn bản ghi mạng](docs/web-network-captures.md) | -| Bắt gói tin | Điểm cuối, DNS, TLS SNI, phân bố giao thức và yêu cầu HTTP | tshark và capinfos do nơi gọi cung cấp; [hướng dẫn pcap](docs/pcap-analysis.md) | -| Assembly .NET | Siêu dữ liệu, lệnh CIL, các phụ thuộc mã máy được khai báo và so sánh bản dựng | Kiểm tra tĩnh; [hướng dẫn mã được quản lý](docs/managed-code-analysis.md) | -| APK Android | Khai báo manifest, lớp, phương thức được dịch ngược và tham chiếu | JADX không giao diện đồ họa và JDK đầy đủ trên Linux/macOS/Windows x64; [hướng dẫn Android](docs/android-analysis.md) | -| Thiết bị Android | Thiết bị, gói, tiến trình, nhật ký, dump, ảnh chụp màn hình và truyền APK/tệp | adb do nơi gọi cung cấp (trình giả lập hoặc thiết bị); [hướng dẫn ADB](docs/adb-device-analysis.md) | -| Dự án JEB | Đơn vị dự án từ engine, hash của artifact và mã giả của kiểu/phương thức | Một ứng dụng JEB do nơi gọi khởi động, phục vụ MCP, chọn bằng REA_JEB_MCP_URL; [hướng dẫn JEB](docs/jeb-analysis.md) | -| Tài nguyên Android | Manifest đã giải mã, bảng chuỗi, ngôn ngữ và thông tin phiên bản | Trình khởi chạy Apktool do nơi gọi cung cấp; [hướng dẫn tài nguyên](docs/apktool-resource-analysis.md) | -| Ứng dụng Flutter | Băm snapshot Dart, hash thư viện theo ABI và nguồn gốc build engine | Không cần (phân tích APK tĩnh); [hướng dẫn Flutter](docs/flutter-analysis.md) | -| Phần sụn | Các vùng, kết quả trích xuất và dữ liệu chuyển sang phân tích mã máy | Binwalk / Unblob trên Linux; [hướng dẫn phần sụn](docs/firmware-analysis.md) | -| Gói tin đã ghi | Trường được chọn, giá trị lặp, bộ lọc và quy tắc giải mã | Linux: TShark 4.x + prlimit; [hướng dẫn bản ghi gói tin](docs/packet-captures.md) | -| Tín hiệu logic đã ghi | Chú thích bộ giải mã, phạm vi mẫu và chẩn đoán | Linux: sigrok-cli 0.7.2 + prlimit; [hướng dẫn tín hiệu](docs/signal-recordings.md) | -| Gói và tài nguyên | Danh sách tệp, giá trị băm, plist, cấu trúc bundle Apple và tài nguyên được trích xuất | [Hướng dẫn hiện vật phần mềm và JavaScript](docs/javascript-artifact-reconstruction.md), [ứng dụng Apple](docs/apple-application-analysis.md) | -| Hành vi tiến trình | Đầu ra terminal, tương tác, quan sát khi thoát và hệ thống tệp, cùng so sánh các lần chạy | Linux/macOS có PTY gốc; [ghi lại tiến trình](docs/process-capture.md) | +| Mục tiêu | REA trả về gì | Yêu cầu và hướng dẫn | +| ----------------------------- | --------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | +| Tệp nhị phân mã máy | Mã giả, hợp ngữ, chuỗi, ký hiệu, lời gọi và tham chiếu | Hopper, Ghidra hoặc IDA; [phân tích mã máy](https://rea.tools/vi/guides/native/) | +| Bố cục ELF ngoại tuyến | Các section, segment, ký hiệu/thông tin tái định vị gốc và các biện pháp giảm thiểu có thể có từ phân tích tĩnh | pwntools do bên gọi cung cấp trên Linux x64; [chẩn đoán tệp nhị phân](../binary-diagnostics.md) | +| Bytecode EVM | Bộ chọn điều phối, độ lệch byte, tham số suy luận và khả năng thay đổi trạng thái | Đầu vào cục bộ chứa byte thô hoặc dạng thập lục phân trên Linux x64; [hướng dẫn EVM ngoại tuyến](../evm-bytecode.md) | +| Sự cố Linux đã ghi lại | Bản ghi note thô, thanh ghi/tín hiệu của mọi luồng được ghi lại và các ứng viên ánh xạ tùy chọn | pwntools do bên gọi cung cấp trên Linux x64; GDB/pwndbg tùy chọn; [sự cố đã ghi lại](../recorded-crashes.md) | +| JavaScript / Electron | Mô-đun, các khai báo import, source map, tuyến, IPC và quan hệ với tiện ích mở rộng mã máy | Node.js và npm; [phân tích ứng dụng](https://rea.tools/vi/guides/javascript/) | +| Trang web | Cấu trúc trang, script, quan sát mạng và ảnh chụp màn hình được yêu cầu | Trình duyệt thuộc họ Chrome; [phân tích trình duyệt](https://rea.tools/vi/guides/browser/) | +| Bản ghi lưu lượng mạng đã lưu | Yêu cầu, phản hồi, payload có thể truy cập và vị trí nguồn | HAR; mitmdump trên Linux cho bản ghi định dạng gốc của mitmproxy; [hướng dẫn bản ghi mạng](../web-network-captures.md) | +| Bắt gói tin | Điểm cuối, DNS, TLS SNI, phân bố giao thức và yêu cầu HTTP | tshark và capinfos do nơi gọi cung cấp; [hướng dẫn pcap](../pcap-analysis.md) | +| Assembly .NET | Siêu dữ liệu, lệnh CIL, các phụ thuộc mã máy được khai báo và so sánh bản dựng | Kiểm tra tĩnh; [hướng dẫn mã được quản lý](../managed-code-analysis.md) | +| APK Android | Khai báo manifest, lớp, phương thức được dịch ngược và tham chiếu | JADX không giao diện đồ họa và JDK đầy đủ trên Linux/macOS/Windows x64; [hướng dẫn Android](../android-analysis.md) | +| Thiết bị Android | Thiết bị, gói, tiến trình, nhật ký, dump, ảnh chụp màn hình và truyền APK/tệp | adb do nơi gọi cung cấp (trình giả lập hoặc thiết bị); [hướng dẫn ADB](../adb-device-analysis.md) | +| Dự án JEB | Đơn vị dự án từ engine, hash của artifact và mã giả của kiểu/phương thức | Một ứng dụng JEB do nơi gọi khởi động, phục vụ MCP, chọn bằng REA_JEB_MCP_URL; [hướng dẫn JEB](../jeb-analysis.md) | +| Tài nguyên Android | Manifest đã giải mã, bảng chuỗi, ngôn ngữ và thông tin phiên bản | Trình khởi chạy Apktool do nơi gọi cung cấp; [hướng dẫn tài nguyên](../apktool-resource-analysis.md) | +| Ứng dụng Flutter | Băm snapshot Dart, hash thư viện theo ABI và nguồn gốc build engine | Không cần (phân tích APK tĩnh); [hướng dẫn Flutter](../flutter-analysis.md) | +| Phần sụn | Các vùng, kết quả trích xuất và dữ liệu chuyển sang phân tích mã máy | Binwalk / Unblob trên Linux; [hướng dẫn phần sụn](../firmware-analysis.md) | +| Gói tin đã ghi | Trường được chọn, giá trị lặp, bộ lọc và quy tắc giải mã | Linux: TShark 4.x + prlimit; [hướng dẫn bản ghi gói tin](../packet-captures.md) | +| Tín hiệu logic đã ghi | Chú thích bộ giải mã, phạm vi mẫu và chẩn đoán | Linux: sigrok-cli 0.7.2 + prlimit; [hướng dẫn tín hiệu](../signal-recordings.md) | +| Gói và tài nguyên | Danh sách tệp, giá trị băm, plist, cấu trúc bundle Apple và tài nguyên được trích xuất | [Hướng dẫn hiện vật phần mềm và JavaScript](../javascript-artifact-reconstruction.md), [ứng dụng Apple](../apple-application-analysis.md) | +| Hành vi tiến trình | Đầu ra terminal, tương tác, quan sát khi thoát và hệ thống tệp, cùng so sánh các lần chạy | Linux/macOS có PTY gốc; [ghi lại tiến trình](../process-capture.md) | Kiểm tra tĩnh JavaScript và .NET đọc các tệp được cung cấp mà không chạy ứng dụng. Ghi lại khi chạy sẽ khởi chạy hoặc tương tác với mục tiêu đã chọn bằng quyền người dùng của bạn; từng hướng dẫn mô tả tác động của nó. -Định dạng mã máy và nền tảng chạy REA được hỗ trợ khác nhau tùy trình phân tích. Xem [thiết lập Hopper và Ghidra](docs/installation.md#hopper), [hướng dẫn IDA](docs/ida-provider.md) và [hỗ trợ Ghidra thử nghiệm trên Windows](docs/windows-ghidra-p0.md). Ghidra cũng hỗ trợ [phân tích DOS 16 bit](docs/ghidra-dos.md). Để chọn trình phân tích, xem [hướng dẫn CLI](docs/cli.md#choose-a-provider). Kiểm tra [khả năng sử dụng theo bản phát hành](docs/installation.md#released-package-and-main) đối với tính năng được bổ sung sau phiên bản npm mới nhất. +Định dạng mã máy và nền tảng chạy REA được hỗ trợ khác nhau tùy trình phân tích. Xem [thiết lập Hopper và Ghidra](../installation.md#hopper), [hướng dẫn IDA](../ida-provider.md) và [hỗ trợ Ghidra thử nghiệm trên Windows](../windows-ghidra-p0.md). Ghidra cũng hỗ trợ [phân tích DOS 16 bit](../ghidra-dos.md). Để chọn trình phân tích, xem [hướng dẫn CLI](../cli.md#choose-a-provider). Kiểm tra [khả năng sử dụng theo bản phát hành](../installation.md#released-package-and-main) đối với tính năng được bổ sung sau phiên bản npm mới nhất. ## Các nghiên cứu điển hình -[![Minh họa các ví dụ về cân bằng âm thanh DX-Ball, cầu nối bảng nhớ tạm Notion và vòng đạn TH04](docs/assets/rea-showcases.png)](https://rea.tools/vi/showcase/) +[![Minh họa các ví dụ về cân bằng âm thanh DX-Ball, cầu nối bảng nhớ tạm Notion và vòng đạn TH04](../assets/rea-showcases.png)](https://rea.tools/vi/showcase/) ### DX-Ball: tái tạo phép tính phân bố âm thanh trái phải @@ -187,28 +187,28 @@ Nếu bạn dùng REA cho một mục tiêu thú vị, chúng tôi muốn biết
Tác nhân nào có thể dùng REA? -Bất kỳ tác nhân nào hỗ trợ máy chủ MCP cục bộ. Quá trình thiết lập cấu hình các [tác nhân được hỗ trợ](docs/installation.md#supported-agents); ứng dụng khách khác có thể dùng [đăng ký MCP thủ công](docs/installation.md#mcp-registry). +Bất kỳ tác nhân nào hỗ trợ máy chủ MCP cục bộ. Quá trình thiết lập cấu hình các [tác nhân được hỗ trợ](../installation.md#supported-agents); ứng dụng khách khác có thể dùng [đăng ký MCP thủ công](../installation.md#mcp-registry).
Tôi có cần Hopper, Ghidra hoặc IDA không? -Phân tích mã máy chuyên sâu sử dụng một trong những công cụ đó. Kiểm tra tĩnh JavaScript và .NET hoạt động mà không cần bộ máy phân tích mã máy. Quá trình thiết lập có thể cài Hopper sau khi được chấp thuận; Ghidra và IDA dùng bản cài đặt sẵn có. Xem [thiết lập trình phân tích](docs/installation.md#hopper). +Phân tích mã máy chuyên sâu sử dụng một trong những công cụ đó. Kiểm tra tĩnh JavaScript và .NET hoạt động mà không cần bộ máy phân tích mã máy. Quá trình thiết lập có thể cài Hopper sau khi được chấp thuận; Ghidra và IDA dùng bản cài đặt sẵn có. Xem [thiết lập trình phân tích](../installation.md#hopper).
Tôi có cần khởi chạy Hopper trước không? -REA khởi chạy Hopper khi một thao tác cần đến nó. Trên macOS, hộp thoại lần chạy đầu có thể yêu cầu chọn chế độ dùng thử hoặc kích hoạt giấy phép. Xem [khởi động Hopper và khắc phục sự cố](docs/installation.md#launcher-paths-and-troubleshooting). +REA khởi chạy Hopper khi một thao tác cần đến nó. Trên macOS, hộp thoại lần chạy đầu có thể yêu cầu chọn chế độ dùng thử hoặc kích hoạt giấy phép. Xem [khởi động Hopper và khắc phục sự cố](../installation.md#launcher-paths-and-troubleshooting).
Cài skill từ skills.sh có tác dụng gì? -Skill cung cấp hướng dẫn điều tra cho tác nhân. Dùng `npx rea-agents setup` để đăng ký máy chủ MCP của REA và cài hướng dẫn tương ứng, sau đó khởi động lại tác nhân. Xem [chỉ cài skill](docs/installation.md#skill-only-installation). +Skill cung cấp hướng dẫn điều tra cho tác nhân. Dùng `npx rea-agents setup` để đăng ký máy chủ MCP của REA và cài hướng dẫn tương ứng, sau đó khởi động lại tác nhân. Xem [chỉ cài skill](../installation.md#skill-only-installation).
@@ -251,14 +251,14 @@ Nếu dùng tác nhân, hãy hoàn tất [làm mới thiết lập](#cập-nhậ Bắt đầu với [hướng dẫn thực hành](https://rea.tools/vi/guides/) trên trang web. Để biết tùy chọn cụ thể, điều kiện tiên quyết và hợp đồng kết quả: -- [Cài đặt và thiết lập](docs/installation.md): đăng ký tác nhân, cấu hình trình phân tích, cập nhật và gỡ cài đặt. -- [Kiểm tra mức sẵn sàng và khắc phục sự cố](docs/installation.md#check-readiness-for-your-task): chẩn đoán một tác nhân hoặc bộ máy phân tích cụ thể. -- [CLI và Evidence](docs/cli.md): lệnh, lựa chọn trình phân tích, ảnh chụp trạng thái, nhập/xuất và trạng thái thoát. -- [Hợp đồng MCP](docs/mcp-contracts.md) và [prompt cho tác nhân](docs/mcp-prompts.md): kết quả công cụ, phiên làm việc và điều tra có hướng dẫn. -- [Danh mục công cụ](docs/mcp-contracts.md#generated-catalog): danh sách công cụ, trình phân tích và lệnh CLI được tạo khi dựng dự án. -- [Lộ trình](docs/roadmap.md): công việc dự kiến và theo dõi các khả năng. +- [Cài đặt và thiết lập](../installation.md): đăng ký tác nhân, cấu hình trình phân tích, cập nhật và gỡ cài đặt. +- [Kiểm tra mức sẵn sàng và khắc phục sự cố](../installation.md#check-readiness-for-your-task): chẩn đoán một tác nhân hoặc bộ máy phân tích cụ thể. +- [CLI và Evidence](../cli.md): lệnh, lựa chọn trình phân tích, ảnh chụp trạng thái, nhập/xuất và trạng thái thoát. +- [Hợp đồng MCP](../mcp-contracts.md) và [prompt cho tác nhân](../mcp-prompts.md): kết quả công cụ, phiên làm việc và điều tra có hướng dẫn. +- [Danh mục công cụ](../mcp-contracts.md#generated-catalog): danh sách công cụ, trình phân tích và lệnh CLI được tạo khi dựng dự án. +- [Lộ trình](../roadmap.md): công việc dự kiến và theo dõi các khả năng. -Báo cáo lỗ hổng theo [SECURITY.md](SECURITY.md). +Báo cáo lỗ hổng theo [SECURITY.md](../../SECURITY.md). ## Lịch sử sao @@ -276,7 +276,7 @@ Cảm ơn mọi người đã sử dụng REA, báo lỗi, đề xuất tính n ## Miễn trừ trách nhiệm -Xem [DISCLAIMER.md](DISCLAIMER.md) để biết hướng dẫn sử dụng có trách nhiệm. +Xem [DISCLAIMER.md](../../DISCLAIMER.md) để biết hướng dẫn sử dụng có trách nhiệm. REA dành cho nghiên cứu, giáo dục, khả năng tương tác, bảo tồn phần mềm và phân tích phần mềm bạn sở hữu hoặc được phép kiểm tra, trong khuôn khổ pháp luật. @@ -288,14 +288,14 @@ applicable licenses. Chúng tôi hoan nghênh bạn giúp phát triển REA! [Mở issue](https://github.com/morluto/rea/issues) để báo lỗi hoặc đề xuất tính năng, hoặc [gửi pull request](https://github.com/morluto/rea/pulls) để cải thiện mã hay tài liệu. -Xem [CONTRIBUTING.md](CONTRIBUTING.md) về thiết lập môi trường phát triển và các bước kiểm tra, [hướng dẫn kiểm thử](docs/testing.md) về các luồng xác minh và [sơ đồ kiến trúc](docs/architecture.mermaid) về cấu trúc dự án. +Xem [CONTRIBUTING.md](../../CONTRIBUTING.md) về thiết lập môi trường phát triển và các bước kiểm tra, [hướng dẫn kiểm thử](../testing.md) về các luồng xác minh và [sơ đồ kiến trúc](../architecture.mermaid) về cấu trúc dự án. ## Giấy phép -[MIT](LICENSE) +[MIT](../../LICENSE) REA-authored code is licensed under MIT. Third-party material retains its own -licenses and copyright notices; see [Third-party notices](THIRD_PARTY_NOTICES.md) +licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md) for the dependency inventory, preserved license texts, and external-tool terms. **You must legally obtain and be authorized to use commercial tools**, including diff --git a/README_zh-TW.md b/docs/readme/README_zh-TW.md similarity index 69% rename from README_zh-TW.md rename to docs/readme/README_zh-TW.md index 62c9ab2f3..1ddbbd26e 100644 --- a/README_zh-TW.md +++ b/docs/readme/README_zh-TW.md @@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · **繁體中文** · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · [简体中文](README_zh.md) · **繁體中文** · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA：逆向一切 @@ -9,10 +9,10 @@ **看到喜歡的功能，弄清它的工作原理，深入到二進位層面。** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA 在 Hopper 中啟動分析橋，檢查原生二進位檔案 +REA 在 Hopper 中啟動分析橋，檢查原生二進位檔案

@@ -33,7 +33,7 @@ - Discord
+ Discord
加入逆向工程社群
Discord · 問答 · 成果分享 @@ -51,7 +51,7 @@ REA 將 AI 代理連接到分析工具，用於檢查原生二進位檔案、JavaScript 和 Electron 應用、.NET 程式集以及網站。你也可以在終端中使用同樣的工具。分析在本機執行，結果包含支撐各項結論的證據和相關限制。 -設定流程會向 AI 代理註冊 REA，並安裝對應的工作流程指引。原生分析可以使用已有的 Hopper 或 Ghidra；設定流程也可以在你核准後安裝 Hopper。靜態 JavaScript 分析不需要這兩個引擎。 +設定流程會向 AI 代理註冊 REA，並安裝對應的工作流程指引。原生分析可以使用已有的 Hopper、Ghidra 或 IDA；設定流程也可以在你核准後安裝 Hopper。靜態 JavaScript 分析不需要原生分析引擎。 > **[訪問 REA 網站](https://rea.tools/zh-TW/)**，查看設定說明、圖解指南和真實案例。 @@ -67,7 +67,7 @@ npx rea-agents setup 選擇 AI 代理，檢查計劃中的變更並核准。設定流程會添加 REA 的 MCP 服務和對應的工作流程指引，並備份已有組態。完成後重新啟動 AI 代理。 -設定流程支援 Claude Code、Codex、Cursor、Gemini CLI、Grok Build 和[其他 AI 代理](docs/installation.md#supported-agents)。提供者組態和手動註冊 MCP 的方法見[安裝與設定](docs/installation.md)。 +設定流程支援 Claude Code、Codex、Cursor、Gemini CLI、Grok Build 和[其他 AI 代理](../installation.md#supported-agents)。提供者組態和手動註冊 MCP 的方法見[安裝與設定](../installation.md)。 ### 詢問 AI 代理 @@ -94,18 +94,20 @@ npm install --global rea-agents rea --help ``` -原生分析需要先設定分析提供者。原生命令、提供者選擇、快照和腳本用法見 [CLI 與 Evidence 指南](docs/cli.md)。 +原生分析需要先設定分析提供者。原生命令、提供者選擇、快照和腳本用法見 [CLI 與 Evidence 指南](../cli.md)。 ### 更新 REA REA 更新較快，新版本經常包含錯誤修復。請將安裝的版本保持最新。 -通過 npm 安裝的 CLI 使用： +通過 `npm install --global` 或 `bun add --global` 安裝的 CLI 使用： ```bash rea update ``` +Bun 全域更新需要 Bun 1.2.17 或更新版本。如果使用較舊的 Bun 版本，請先手動升級 Bun，再執行 `rea update`。 + 要更新 AI 代理註冊和 skill，請執行更新命令輸出的設定命令。 如果使用 `npx`，通過以下命令更新 AI 代理設定： @@ -114,15 +116,15 @@ rea update npx rea-agents@latest setup ``` -檢查設定變更並重新啟動 AI 代理。單次 CLI 操作可使用 `npx rea-agents@latest`，後接所需命令。 +檢查設定變更並重新啟動 AI 代理。單次 CLI 操作可使用 `npx rea-agents@latest`，後接所需命令。使用 Bun 時，用 `bunx rea-agents@latest` 代替 `npx rea-agents@latest`。 ## REA 的工作原理 AI 代理通過 MCP 呼叫 REA，檢查目標並追蹤相關程式碼。REA 返回發現及對應證據。AI 代理據此提出後續問題、解釋行為，或編寫並測試實現。CLI 命令使用同樣的工作流程。 -![REA 調查流程：AI 代理詢問本地目標的行為，REA 使用分析工具檢查和追蹤目標，AI 代理再利用返回的程式碼、引用和未知項進行解釋、實現與測試。](website/public/assets/figures/rea-investigation-flow.svg) +![REA 調查流程：AI 代理詢問本地目標的行為，REA 使用分析工具檢查和追蹤目標，AI 代理再利用返回的程式碼、引用和未知項進行解釋、實現與測試。](../../website/public/assets/figures/rea-investigation-flow.svg) -[打開完整尺寸的圖示](website/public/assets/figures/rea-investigation-flow.svg)。 +[打開完整尺寸的圖示](../../website/public/assets/figures/rea-investigation-flow.svg)。 @@ -130,37 +132,39 @@ AI 代理通過 MCP 呼叫 REA，檢查目標並追蹤相關程式碼。REA 返 REA 需要 Node.js 22.x（>=22.19）、24.x（>=24.11）或 26+，以及 npm。額外工具和主機支援取決於分析目標： -| 目標 | REA 返回的內容 | 要求與指南 | -| --------------------- | --------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | -| 原生二進位檔案 | 偽程式碼、組合語言、字串、符號、呼叫和引用 | Hopper、Ghidra 或 IDA；[原生分析](https://rea.tools/zh-TW/guides/native/) | -| 離線 ELF 佈局 | 節、段、原始符號/重定位資訊及靜態防護機制候選項 | Linux x64 上由呼叫方提供的 pwntools；[二進位診斷](docs/binary-diagnostics.md) | -| EVM 位元組碼 | 分派選擇器、位元組偏移、推斷的參數和狀態可變性 | Linux x64 上的本地原始位元組/十六進位輸入載體；[離線 EVM 指南](docs/evm-bytecode.md) | -| 已記錄的 Linux 崩潰 | 原始 note 記錄、每個已記錄執行緒的暫存器/訊號及可選的對映候選項 | Linux x64 上由呼叫方提供的 pwntools；可選的 GDB/pwndbg；[已記錄崩潰](docs/recorded-crashes.md) | -| JavaScript / Electron | 模組、匯入、source map、路由、IPC 和原生附加元件關係 | Node.js 和 npm；[應用分析](https://rea.tools/zh-TW/guides/javascript/) | -| 網站 | 頁面結構、腳本、網路觀察結果和按請求獲取的螢幕擷取畫面 | Chrome 系瀏覽器；[瀏覽器分析](https://rea.tools/zh-TW/guides/browser/) | -| 已保存的網路擷取 | 請求、回應、可訪問的酬載和來源位置 | HAR；原生 mitmproxy 擷取需要 Linux 上的 mitmdump；[擷取指南](docs/web-network-captures.md) | -| 封包擷取 | 端點、DNS、TLS SNI、通訊協定分佈和 HTTP 請求 | 呼叫端提供的 tshark 和 capinfos；[pcap 指南](docs/pcap-analysis.md) | -| .NET 程式集 | 中繼資料、CIL 指令、聲明的原生依賴和建置對比 | 靜態檢查；[托管程式碼指南](docs/managed-code-analysis.md) | -| Android APK | 清單聲明、類別、反編譯的方法和引用 | Linux/macOS/Windows x64 上的無界面 JADX 和完整 JDK；[Android 指南](docs/android-analysis.md) | -| Android 裝置 | 裝置、套件、程序、日誌、傾印、螢幕擷取以及 APK/檔案傳輸 | 呼叫端提供的 adb（模擬器或實體裝置）；[ADB 指南](docs/adb-device-analysis.md) | -| JEB 專案 | 引擎支援的專案單元、成品摘要和型別/方法的虛擬碼 | 由呼叫端啟動、提供 MCP 的 JEB 用戶端，透過 REA_JEB_MCP_URL 選擇；[JEB 指南](docs/jeb-analysis.md) | -| Android 資源 | 解碼後的清單、字串表、語系和版本資訊 | 呼叫端提供的 Apktool 啟動器；[資源指南](docs/apktool-resource-analysis.md) | -| Flutter 應用 | Dart 快照雜湊、依 ABI 的程式庫雜湊及引擎建置來源 | 無（靜態 APK 解析）；[Flutter 指南](docs/flutter-analysis.md) | -| 韌體 | 區域、提取結果和轉交原生分析的內容 | Linux 上的 Binwalk / Unblob；[韌體指南](docs/firmware-analysis.md) | -| 封包記錄 | 指定欄位、重複值、篩選條件與解碼規則 | Linux: TShark 4.x + prlimit; [封包分析指南](docs/packet-captures.md) | -| 錄製的邏輯訊號 | 解碼註解、樣本範圍與診斷資訊 | Linux: sigrok-cli 0.7.2 + prlimit; [訊號記錄指南](docs/signal-recordings.md) | -| 軟體包與資源 | 檔案清單、摘要、plist、Apple bundle 結構和提取的資源 | [製品與 JavaScript 指南](docs/javascript-artifact-reconstruction.md)、[Apple 應用](docs/apple-application-analysis.md) | -| 處理程序行為 | 終端輸出、互動、退出和檔案系統觀察結果，以及執行對比 | 支援原生 PTY 的 Linux/macOS；[處理程序擷取](docs/process-capture.md) | +| 目標 | REA 返回的內容 | 要求與指南 | +| --------------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | +| 原生二進位檔案 | 偽程式碼、組合語言、字串、符號、呼叫和引用 | Hopper、Ghidra 或 IDA；[原生分析](https://rea.tools/zh-TW/guides/native/) | +| 離線 ELF 佈局 | 節、段、原始符號/重定位資訊及靜態防護機制候選項 | Linux x64 上由呼叫方提供的 pwntools；[二進位診斷](../binary-diagnostics.md) | +| Go 二進位檔案 | 內嵌編譯器版本、模組、替換項和建置設定 | 內建靜態 ELF/PE/thin Mach-O 檢查；無需安裝 Go | +| WASM 工件 | 驗證、節、精確的匯入/匯出 WAT 形式及解碼後的 WAT | 呼叫方提供的 WABT 1.0.42；`inspect_wasm_artifact` | +| EVM 位元組碼 | 分派選擇器、位元組偏移、推斷的參數和狀態可變性 | Linux x64 上的本地原始位元組/十六進位輸入載體；[離線 EVM 指南](../evm-bytecode.md) | +| 已記錄的 Linux 崩潰 | 原始 note 記錄、每個已記錄執行緒的暫存器/訊號及可選的對映候選項 | Linux x64 上由呼叫方提供的 pwntools；可選的 GDB/pwndbg；[已記錄崩潰](../recorded-crashes.md) | +| JavaScript / Electron | 模組、匯入、source map、路由、IPC 和原生附加元件關係 | Node.js 和 npm；[應用分析](https://rea.tools/zh-TW/guides/javascript/) | +| 網站 | 頁面結構、腳本、網路觀察結果和按請求獲取的螢幕擷取畫面 | Chrome 系瀏覽器；[瀏覽器分析](https://rea.tools/zh-TW/guides/browser/) | +| 已保存的網路擷取 | 請求、回應、可訪問的酬載和來源位置 | HAR；原生 mitmproxy 擷取需要 Linux 上的 mitmdump；[擷取指南](../web-network-captures.md) | +| 封包擷取 | 端點、DNS、TLS SNI、通訊協定分佈和 HTTP 請求 | 呼叫端提供的 tshark 和 capinfos；[pcap 指南](../pcap-analysis.md) | +| .NET 程式集 | 中繼資料、CIL 指令、聲明的原生依賴和建置對比 | 靜態檢查；[托管程式碼指南](../managed-code-analysis.md) | +| Android APK | 清單聲明、類別、反編譯的方法和引用 | Linux/macOS/Windows x64 上的無界面 JADX 和完整 JDK；[Android 指南](../android-analysis.md) | +| Android 裝置 | 裝置、套件、程序、日誌、傾印、螢幕擷取以及 APK/檔案傳輸 | 呼叫端提供的 adb（模擬器或實體裝置）；[ADB 指南](../adb-device-analysis.md) | +| JEB 專案 | 引擎支援的專案單元、成品摘要和型別/方法的虛擬碼 | 由呼叫端啟動、提供 MCP 的 JEB 用戶端，透過 REA_JEB_MCP_URL 選擇；[JEB 指南](../jeb-analysis.md) | +| Android 資源 | 解碼後的清單、字串表、語系和版本資訊 | 呼叫端提供的 Apktool 啟動器；[資源指南](../apktool-resource-analysis.md) | +| Flutter 應用 | Dart 快照雜湊、依 ABI 的程式庫雜湊及引擎建置來源 | 無（靜態 APK 解析）；[Flutter 指南](../flutter-analysis.md) | +| 韌體 | 區域、提取結果和轉交原生分析的內容 | Linux 上的 Binwalk / Unblob；[韌體指南](../firmware-analysis.md) | +| 封包記錄 | 指定欄位、重複值、篩選條件與解碼規則 | Linux: TShark 4.x + prlimit; [封包分析指南](../packet-captures.md) | +| 錄製的邏輯訊號 | 解碼註解、樣本範圍與診斷資訊 | Linux: sigrok-cli 0.7.2 + prlimit; [訊號記錄指南](../signal-recordings.md) | +| 軟體包與資源 | 檔案清單、摘要、plist、Apple bundle 結構和提取的資源 | [製品與 JavaScript 指南](../javascript-artifact-reconstruction.md)、[Apple 應用](../apple-application-analysis.md) | +| 處理程序行為 | 終端輸出、互動、退出和檔案系統觀察結果，以及執行對比 | 支援原生 PTY 的 Linux/macOS；[處理程序擷取](../process-capture.md) | 靜態 JavaScript 和 .NET 檢查讀取提供的檔案，不執行應用。執行階段擷取會以你的使用者權限執行目標或與之互動；各執行階段指南說明瞭具體影響。 -不同提供者支援的原生格式和主機系統有所不同。請參閱 [Hopper 與 Ghidra 設定](docs/installation.md#hopper)、[IDA 指南](docs/ida-provider.md)和[實驗性 Windows Ghidra 支援](docs/windows-ghidra-p0.md)。Ghidra 也支援 [16 位 DOS 分析](docs/ghidra-dos.md)。提供者選擇見 [CLI 指南](docs/cli.md#choose-a-provider)。對於最新 npm 版本發佈後新增的功能，請查看[發佈可用性](docs/installation.md#released-package-and-main)。 +不同提供者支援的原生格式和主機系統有所不同。請參閱 [Hopper 與 Ghidra 設定](../installation.md#hopper)、[IDA 指南](../ida-provider.md)和[實驗性 Windows Ghidra 支援](../windows-ghidra-p0.md)。Ghidra 也支援 [16 位 DOS 分析](../ghidra-dos.md)。提供者選擇見 [CLI 指南](../cli.md#choose-a-provider)。對於最新 npm 版本發佈後新增的功能，請查看[發佈可用性](../installation.md#released-package-and-main)。 ## 案例 -[![DX-Ball 聲音平移、Notion 剪貼簿橋接與 TH04 彈幕環案例的插圖](docs/assets/rea-showcases.png)](https://rea.tools/zh-TW/showcase/) +[![DX-Ball 聲音平移、Notion 剪貼簿橋接與 TH04 彈幕環案例的插圖](../assets/rea-showcases.png)](https://rea.tools/zh-TW/showcase/) ### DX-Ball：重建聲像計算 @@ -189,28 +193,28 @@ REA 需要 Node.js 22.x（>=22.19）、24.x（>=24.11）或 26+，以及 npm。
哪些 AI 代理可以使用 REA？ -任何支援本地 MCP 服務的 AI 代理都可以使用。設定流程會設定[受支援的 AI 代理](docs/installation.md#supported-agents)；其他客戶端可通過[手動註冊 MCP](docs/installation.md#mcp-registry) 使用。 +任何支援本地 MCP 服務的 AI 代理都可以使用。設定流程會設定[受支援的 AI 代理](../installation.md#supported-agents)；其他客戶端可通過[手動註冊 MCP](../installation.md#mcp-registry) 使用。
需要 Hopper、Ghidra 或 IDA 嗎？ -深入的原生分析需要其中一個引擎。靜態 JavaScript 和 .NET 檢查不需要原生分析引擎。設定流程可以在你核准後安裝 Hopper；Ghidra 和 IDA 則使用已有安裝。見[提供者設定](docs/installation.md#hopper)。 +深入的原生分析需要其中一個引擎。靜態 JavaScript 和 .NET 檢查不需要原生分析引擎。設定流程可以在你核准後安裝 Hopper；Ghidra 和 IDA 則使用已有安裝。見[提供者設定](../installation.md#hopper)。
需要先啟動 Hopper 嗎？ -REA 會在操作需要時啟動 Hopper。在 macOS 上，首次執行時可能會出現對話框，讓你選擇演示模式或啟用授權。見 [Hopper 啟動與故障排查](docs/installation.md#launcher-paths-and-troubleshooting)。 +REA 會在操作需要時啟動 Hopper。在 macOS 上，首次執行時可能會出現對話框，讓你選擇演示模式或啟用授權。見 [Hopper 啟動與故障排查](../installation.md#launcher-paths-and-troubleshooting)。
從 skills.sh 安裝 skill 有什麼作用？ -skill 為 AI 代理提供調查指引。使用 `npx rea-agents setup` 註冊 REA 的 MCP 服務並安裝對應的指引，然後重新啟動 AI 代理。見[僅安裝 skill](docs/installation.md#skill-only-installation)。 +skill 為 AI 代理提供調查指引。使用 `npx rea-agents setup` 註冊 REA 的 MCP 服務並安裝對應的指引，然後重新啟動 AI 代理。見[僅安裝 skill](../installation.md#skill-only-installation)。
@@ -253,14 +257,14 @@ npx rea-agents@latest setup 可以先閱讀網站上的[實操指南](https://rea.tools/zh-TW/guides/)。具體選項、前提條件和結果契約見： -- [安裝與設定](docs/installation.md)：AI 代理註冊、提供者組態、更新和卸載。 -- [就緒檢查與故障排查](docs/installation.md#check-readiness-for-your-task)：診斷特定 AI 代理或分析引擎。 -- [CLI 與 Evidence](docs/cli.md)：命令、提供者選擇、快照、匯入/匯出和退出狀態。 -- [MCP 契約](docs/mcp-contracts.md)和[AI 代理提示詞](docs/mcp-prompts.md)：工具結果、會話和引導式調查。 -- [工具目錄](docs/mcp-contracts.md#generated-catalog)：建置時生成的工具、提供者和 CLI 命令清單。 -- [路線圖](docs/roadmap.md)：計劃中的工作和能力跟蹤。 +- [安裝與設定](../installation.md)：AI 代理註冊、提供者組態、更新和卸載。 +- [就緒檢查與故障排查](../installation.md#check-readiness-for-your-task)：診斷特定 AI 代理或分析引擎。 +- [CLI 與 Evidence](../cli.md)：命令、提供者選擇、快照、匯入/匯出和退出狀態。 +- [MCP 契約](../mcp-contracts.md)和[AI 代理提示詞](../mcp-prompts.md)：工具結果、會話和引導式調查。 +- [工具目錄](../mcp-contracts.md#generated-catalog)：建置時生成的工具、提供者和 CLI 命令清單。 +- [路線圖](../roadmap.md)：計劃中的工作和能力跟蹤。 -請按照 [SECURITY.md](SECURITY.md) 報告漏洞。 +請按照 [SECURITY.md](../../SECURITY.md) 報告漏洞。 ## Star 歷史 @@ -278,7 +282,7 @@ npx rea-agents@latest setup ## 免責聲明 -負責任使用的相關指引請見 [DISCLAIMER.md](DISCLAIMER.md)。 +負責任使用的相關指引請見 [DISCLAIMER.md](../../DISCLAIMER.md)。 REA 用於合法的研究、教育、互通性、軟體保存，以及分析你擁有或獲准檢查的軟體。 @@ -289,14 +293,14 @@ REA 用於合法的研究、教育、互通性、軟體保存，以及分析你 歡迎幫助改進 REA！你可以[提交 issue](https://github.com/morluto/rea/issues) 報告錯誤或建議功能，也可以[提交 pull request](https://github.com/morluto/rea/pulls) 改進程式碼或文件。 -開發設定和檢查見 [CONTRIBUTING.md](CONTRIBUTING.md)，驗證流程見[測試指南](docs/testing.md)，專案結構見[架構圖](docs/architecture.mermaid)。 +開發設定和檢查見 [CONTRIBUTING.md](../../CONTRIBUTING.md)，驗證流程見[測試指南](../testing.md)，專案結構見[架構圖](../architecture.mermaid)。 ## 授權條款 -[MIT](LICENSE) +[MIT](../../LICENSE) REA 自行編寫的程式碼採用 MIT 授權；第三方內容保留各自的授權和著作權聲明。 -依賴清單、授權原文及外部工具條款見[第三方聲明](THIRD_PARTY_NOTICES.md)。 +依賴清單、授權原文及外部工具條款見[第三方聲明](../../THIRD_PARTY_NOTICES.md)。 **使用者須自行合法取得商用工具及所需使用權**，包括 IDA Pro / Hex-Rays 反編譯器、 Hopper 和 JEB，並遵守各自授權。REA 不提供工具授權、金鑰或啟用權利；試用版和 diff --git a/README_zh.md b/docs/readme/README_zh.md similarity index 69% rename from README_zh.md rename to docs/readme/README_zh.md index c8d66ce08..6e70fd3c3 100644 --- a/README_zh.md +++ b/docs/readme/README_zh.md @@ -1,6 +1,6 @@
-[English](README.md) · **简体中文** · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) +[English](../../README.md) · **简体中文** · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) # REA：逆向一切 @@ -9,10 +9,10 @@ **看到喜欢的功能，弄清它的工作原理，深入到二进制层面。** [![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents) -[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog) +[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](../mcp-contracts.md#generated-catalog) [![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#current-status) [![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything) -[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE) +[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](../../LICENSE) [![Discord](https://img.shields.io/discord/1556595354999332884?logo=discord&logoColor=white&label=Discord&color=5865F2)](https://discord.gg/GkcryMnJDM) morluto%2Frea | Trendshift @@ -25,7 +25,7 @@
-REA 在 Hopper 中启动分析桥，检查原生二进制文件 +REA 在 Hopper 中启动分析桥，检查原生二进制文件

@@ -33,7 +33,7 @@ - Discord
+ Discord
加入逆向工程社区
Discord · 问答 · 成果分享 @@ -51,7 +51,7 @@ REA 将智能体连接到分析工具，用于检查原生二进制文件、JavaScript 和 Electron 应用、.NET 程序集以及网站。你也可以在终端中使用同样的工具。分析在本机运行，结果包含支撑各项结论的证据和相关限制。 -设置流程会向智能体注册 REA，并安装匹配的工作流指引。原生分析可以使用已有的 Hopper 或 Ghidra；设置流程也可以在你批准后安装 Hopper。静态 JavaScript 分析不需要这两个引擎。 +设置流程会向智能体注册 REA，并安装匹配的工作流指引。原生分析可以使用已有的 Hopper、Ghidra 或 IDA；设置流程也可以在你批准后安装 Hopper。静态 JavaScript 分析不需要原生分析引擎。 > **[访问 REA 网站](https://rea.tools/zh-CN/)**，查看设置说明、图解指南和真实案例。 @@ -67,7 +67,7 @@ npx rea-agents setup 选择智能体，检查计划中的变更并批准。设置流程会添加 REA 的 MCP 服务和匹配的工作流指引，并备份已有配置。完成后重启智能体。 -设置流程支持 Claude Code、Codex、Cursor、Gemini CLI、Grok Build 和[其他智能体](docs/installation.md#supported-agents)。提供方配置和手动注册 MCP 的方法见[安装与设置](docs/installation.md)。 +设置流程支持 Claude Code、Codex、Cursor、Gemini CLI、Grok Build 和[其他智能体](../installation.md#supported-agents)。提供方配置和手动注册 MCP 的方法见[安装与设置](../installation.md)。 ### 询问智能体 @@ -94,18 +94,20 @@ npm install --global rea-agents rea --help ``` -原生分析需要先配置分析提供方。原生命令、提供方选择、快照和脚本用法见 [CLI 与 Evidence 指南](docs/cli.md)。 +原生分析需要先配置分析提供方。原生命令、提供方选择、快照和脚本用法见 [CLI 与 Evidence 指南](../cli.md)。 ### 更新 REA REA 更新较快，新版本经常包含错误修复。请保持安装的版本为最新。 -通过 npm 安装的 CLI 使用： +通过 `npm install --global` 或 `bun add --global` 安装的 CLI 使用： ```bash rea update ``` +Bun 全局更新需要 Bun 1.2.17 或更高版本。如果使用较旧的 Bun 版本，请先手动升级 Bun，再运行 `rea update`。 + 要更新智能体注册和 skill，请运行更新命令输出的设置命令。 如果使用 `npx`，通过以下命令更新智能体设置： @@ -114,15 +116,15 @@ rea update npx rea-agents@latest setup ``` -检查设置变更并重启智能体。单次 CLI 操作可使用 `npx rea-agents@latest`，后接所需命令。 +检查设置变更并重启智能体。单次 CLI 操作可使用 `npx rea-agents@latest`，后接所需命令。使用 Bun 时，用 `bunx rea-agents@latest` 代替 `npx rea-agents@latest`。 ## REA 的工作原理 智能体通过 MCP 调用 REA，检查目标并追踪相关代码。REA 返回发现及对应证据。智能体据此提出后续问题、解释行为，或编写并测试实现。CLI 命令使用同样的工作流。 -![REA 调查流程：智能体询问本地目标的行为，REA 使用分析工具检查和追踪目标，智能体再利用返回的代码、引用和未知项进行解释、实现与测试。](website/public/assets/figures/rea-investigation-flow.svg) +![REA 调查流程：智能体询问本地目标的行为，REA 使用分析工具检查和追踪目标，智能体再利用返回的代码、引用和未知项进行解释、实现与测试。](../../website/public/assets/figures/rea-investigation-flow.svg) -[打开完整尺寸的图示](website/public/assets/figures/rea-investigation-flow.svg)。 +[打开完整尺寸的图示](../../website/public/assets/figures/rea-investigation-flow.svg)。 @@ -130,37 +132,39 @@ npx rea-agents@latest setup REA 需要 Node.js 22.x（>=22.19）、24.x（>=24.11）或 26+，以及 npm。额外工具和主机支持取决于分析目标： -| 目标 | REA 返回的内容 | 要求与指南 | -| --------------------- | ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | -| 原生二进制文件 | 伪代码、汇编、字符串、符号、调用和引用 | Hopper、Ghidra 或 IDA；[原生分析](https://rea.tools/zh-CN/guides/native/) | -| 离线 ELF 布局 | 节、段、原始符号/重定位信息及静态防护机制候选项 | Linux x64 上由调用方提供的 pwntools；[二进制诊断](docs/binary-diagnostics.md) | -| EVM 字节码 | 分派选择器、字节偏移、推断的参数和状态可变性 | Linux x64 上的本地原始字节/十六进制输入载体；[离线 EVM 指南](docs/evm-bytecode.md) | -| 已记录的 Linux 崩溃 | 原始 note 记录、每个已记录线程的寄存器/信号及可选的映射候选项 | Linux x64 上由调用方提供的 pwntools；可选的 GDB/pwndbg；[已记录崩溃](docs/recorded-crashes.md) | -| JavaScript / Electron | 模块、导入、source map、路由、IPC 和原生扩展关系 | Node.js 和 npm；[应用分析](https://rea.tools/zh-CN/guides/javascript/) | -| 网站 | 页面结构、脚本、网络观察结果和按请求获取的截图 | Chrome 系浏览器；[浏览器分析](https://rea.tools/zh-CN/guides/browser/) | -| 已保存的网络捕获 | 请求、响应、可访问的载荷和来源位置 | HAR；原生 mitmproxy 捕获需要 Linux 上的 mitmdump；[捕获指南](docs/web-network-captures.md) | -| 数据包捕获 | 端点、DNS、TLS SNI、协议分布和 HTTP 请求 | 调用方提供的 tshark 和 capinfos；[pcap 指南](docs/pcap-analysis.md) | -| .NET 程序集 | 元数据、CIL 指令、声明的原生依赖和构建对比 | 静态检查；[托管代码指南](docs/managed-code-analysis.md) | -| Android APK | 清单声明、类、反编译的方法和引用 | Linux/macOS/Windows x64 上的无界面 JADX 和完整 JDK；[Android 指南](docs/android-analysis.md) | -| Android 设备 | 设备、软件包、进程、日志、转储、屏幕截图以及 APK/文件传输 | 调用方提供的 adb（模拟器或实体设备）；[ADB 指南](docs/adb-device-analysis.md) | -| JEB 项目 | 引擎支持的项目单元、工件摘要和类型/方法的伪代码 | 由调用方启动、提供 MCP 的 JEB 客户端，通过 REA_JEB_MCP_URL 选择；[JEB 指南](docs/jeb-analysis.md) | -| Android 资源 | 解码后的清单、字符串表、语言区域和版本信息 | 调用方提供的 Apktool 启动器；[资源指南](docs/apktool-resource-analysis.md) | -| Flutter 应用 | Dart 快照哈希、按 ABI 的库摘要及引擎构建溯源 | 无（静态 APK 解析）；[Flutter 指南](docs/flutter-analysis.md) | -| 固件 | 区域、提取结果和转交原生分析的内容 | Linux 上的 Binwalk / Unblob；[固件指南](docs/firmware-analysis.md) | -| 抓包文件 | 指定字段、重复值、过滤条件与解码规则 | Linux: TShark 4.x + prlimit; [抓包分析指南](docs/packet-captures.md) | -| 录制的逻辑信号 | 解码注释、样本范围和诊断信息 | Linux: sigrok-cli 0.7.2 + prlimit; [信号记录指南](docs/signal-recordings.md) | -| 软件包与资源 | 文件清单、摘要、plist、Apple bundle 结构和提取的资源 | [制品与 JavaScript 指南](docs/javascript-artifact-reconstruction.md)、[Apple 应用](docs/apple-application-analysis.md) | -| 进程行为 | 终端输出、交互、退出和文件系统观察结果，以及运行对比 | 支持原生 PTY 的 Linux/macOS；[进程捕获](docs/process-capture.md) | +| 目标 | REA 返回的内容 | 要求与指南 | +| --------------------- | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | +| 原生二进制文件 | 伪代码、汇编、字符串、符号、调用和引用 | Hopper、Ghidra 或 IDA；[原生分析](https://rea.tools/zh-CN/guides/native/) | +| 离线 ELF 布局 | 节、段、原始符号/重定位信息及静态防护机制候选项 | Linux x64 上由调用方提供的 pwntools；[二进制诊断](../binary-diagnostics.md) | +| Go 二进制文件 | 内嵌编译器版本、模块、替换项和构建设置 | 内置静态 ELF/PE/thin Mach-O 检查；无需安装 Go | +| WASM 工件 | 验证、节、精确的导入/导出 WAT 形式及解码后的 WAT | 调用方提供的 WABT 1.0.42；`inspect_wasm_artifact` | +| EVM 字节码 | 分派选择器、字节偏移、推断的参数和状态可变性 | Linux x64 上的本地原始字节/十六进制输入载体；[离线 EVM 指南](../evm-bytecode.md) | +| 已记录的 Linux 崩溃 | 原始 note 记录、每个已记录线程的寄存器/信号及可选的映射候选项 | Linux x64 上由调用方提供的 pwntools；可选的 GDB/pwndbg；[已记录崩溃](../recorded-crashes.md) | +| JavaScript / Electron | 模块、导入、source map、路由、IPC 和原生扩展关系 | Node.js 和 npm；[应用分析](https://rea.tools/zh-CN/guides/javascript/) | +| 网站 | 页面结构、脚本、网络观察结果和按请求获取的截图 | Chrome 系浏览器；[浏览器分析](https://rea.tools/zh-CN/guides/browser/) | +| 已保存的网络捕获 | 请求、响应、可访问的载荷和来源位置 | HAR；原生 mitmproxy 捕获需要 Linux 上的 mitmdump；[捕获指南](../web-network-captures.md) | +| 数据包捕获 | 端点、DNS、TLS SNI、协议分布和 HTTP 请求 | 调用方提供的 tshark 和 capinfos；[pcap 指南](../pcap-analysis.md) | +| .NET 程序集 | 元数据、CIL 指令、声明的原生依赖和构建对比 | 静态检查；[托管代码指南](../managed-code-analysis.md) | +| Android APK | 清单声明、类、反编译的方法和引用 | Linux/macOS/Windows x64 上的无界面 JADX 和完整 JDK；[Android 指南](../android-analysis.md) | +| Android 设备 | 设备、软件包、进程、日志、转储、屏幕截图以及 APK/文件传输 | 调用方提供的 adb（模拟器或实体设备）；[ADB 指南](../adb-device-analysis.md) | +| JEB 项目 | 引擎支持的项目单元、工件摘要和类型/方法的伪代码 | 由调用方启动、提供 MCP 的 JEB 客户端，通过 REA_JEB_MCP_URL 选择；[JEB 指南](../jeb-analysis.md) | +| Android 资源 | 解码后的清单、字符串表、语言区域和版本信息 | 调用方提供的 Apktool 启动器；[资源指南](../apktool-resource-analysis.md) | +| Flutter 应用 | Dart 快照哈希、按 ABI 的库摘要及引擎构建溯源 | 无（静态 APK 解析）；[Flutter 指南](../flutter-analysis.md) | +| 固件 | 区域、提取结果和转交原生分析的内容 | Linux 上的 Binwalk / Unblob；[固件指南](../firmware-analysis.md) | +| 抓包文件 | 指定字段、重复值、过滤条件与解码规则 | Linux: TShark 4.x + prlimit; [抓包分析指南](../packet-captures.md) | +| 录制的逻辑信号 | 解码注释、样本范围和诊断信息 | Linux: sigrok-cli 0.7.2 + prlimit; [信号记录指南](../signal-recordings.md) | +| 软件包与资源 | 文件清单、摘要、plist、Apple bundle 结构和提取的资源 | [制品与 JavaScript 指南](../javascript-artifact-reconstruction.md)、[Apple 应用](../apple-application-analysis.md) | +| 进程行为 | 终端输出、交互、退出和文件系统观察结果，以及运行对比 | 支持原生 PTY 的 Linux/macOS；[进程捕获](../process-capture.md) | 静态 JavaScript 和 .NET 检查读取提供的文件，不运行应用。运行时捕获会以你的用户权限运行目标或与之交互；各运行时指南说明了具体影响。 -不同提供方支持的原生格式和主机系统有所不同。请参阅 [Hopper 与 Ghidra 设置](docs/installation.md#hopper)、[IDA 指南](docs/ida-provider.md)和[实验性 Windows Ghidra 支持](docs/windows-ghidra-p0.md)。Ghidra 也支持 [16 位 DOS 分析](docs/ghidra-dos.md)。提供方选择见 [CLI 指南](docs/cli.md#choose-a-provider)。对于最新 npm 版本发布后新增的功能，请查看[发布可用性](docs/installation.md#released-package-and-main)。 +不同提供方支持的原生格式和主机系统有所不同。请参阅 [Hopper 与 Ghidra 设置](../installation.md#hopper)、[IDA 指南](../ida-provider.md)和[实验性 Windows Ghidra 支持](../windows-ghidra-p0.md)。Ghidra 也支持 [16 位 DOS 分析](../ghidra-dos.md)。提供方选择见 [CLI 指南](../cli.md#choose-a-provider)。对于最新 npm 版本发布后新增的功能，请查看[发布可用性](../installation.md#released-package-and-main)。 ## 案例 -[![DX-Ball 声音平移、Notion 剪贴板桥接与 TH04 弹幕环案例的插图](docs/assets/rea-showcases.png)](https://rea.tools/zh-CN/showcase/) +[![DX-Ball 声音平移、Notion 剪贴板桥接与 TH04 弹幕环案例的插图](../assets/rea-showcases.png)](https://rea.tools/zh-CN/showcase/) ### DX-Ball：重建声像计算 @@ -189,28 +193,28 @@ REA 需要 Node.js 22.x（>=22.19）、24.x（>=24.11）或 26+，以及 npm。
哪些智能体可以使用 REA？ -任何支持本地 MCP 服务的智能体都可以使用。设置流程会配置[受支持的智能体](docs/installation.md#supported-agents)；其他客户端可通过[手动注册 MCP](docs/installation.md#mcp-registry) 使用。 +任何支持本地 MCP 服务的智能体都可以使用。设置流程会配置[受支持的智能体](../installation.md#supported-agents)；其他客户端可通过[手动注册 MCP](../installation.md#mcp-registry) 使用。
需要 Hopper、Ghidra 或 IDA 吗？ -深入的原生分析需要其中一个引擎。静态 JavaScript 和 .NET 检查不需要原生分析引擎。设置流程可以在你批准后安装 Hopper；Ghidra 和 IDA 则使用已有安装。见[提供方设置](docs/installation.md#hopper)。 +深入的原生分析需要其中一个引擎。静态 JavaScript 和 .NET 检查不需要原生分析引擎。设置流程可以在你批准后安装 Hopper；Ghidra 和 IDA 则使用已有安装。见[提供方设置](../installation.md#hopper)。
需要先启动 Hopper 吗？ -REA 会在操作需要时启动 Hopper。在 macOS 上，首次运行时可能会出现对话框，让你选择演示模式或激活许可证。见 [Hopper 启动与故障排查](docs/installation.md#launcher-paths-and-troubleshooting)。 +REA 会在操作需要时启动 Hopper。在 macOS 上，首次运行时可能会出现对话框，让你选择演示模式或激活许可证。见 [Hopper 启动与故障排查](../installation.md#launcher-paths-and-troubleshooting)。
从 skills.sh 安装 skill 有什么作用？ -skill 为智能体提供调查指引。使用 `npx rea-agents setup` 注册 REA 的 MCP 服务并安装匹配的指引，然后重启智能体。见[仅安装 skill](docs/installation.md#skill-only-installation)。 +skill 为智能体提供调查指引。使用 `npx rea-agents setup` 注册 REA 的 MCP 服务并安装匹配的指引，然后重启智能体。见[仅安装 skill](../installation.md#skill-only-installation)。
@@ -253,14 +257,14 @@ npx rea-agents@latest setup 可以先阅读网站上的[实操指南](https://rea.tools/zh-CN/guides/)。具体选项、前提条件和结果契约见： -- [安装与设置](docs/installation.md)：智能体注册、提供方配置、更新和卸载。 -- [就绪检查与故障排查](docs/installation.md#check-readiness-for-your-task)：诊断特定智能体或分析引擎。 -- [CLI 与 Evidence](docs/cli.md)：命令、提供方选择、快照、导入/导出和退出状态。 -- [MCP 契约](docs/mcp-contracts.md)和[智能体提示词](docs/mcp-prompts.md)：工具结果、会话和引导式调查。 -- [工具目录](docs/mcp-contracts.md#generated-catalog)：构建时生成的工具、提供方和 CLI 命令清单。 -- [路线图](docs/roadmap.md)：计划中的工作和能力跟踪。 +- [安装与设置](../installation.md)：智能体注册、提供方配置、更新和卸载。 +- [就绪检查与故障排查](../installation.md#check-readiness-for-your-task)：诊断特定智能体或分析引擎。 +- [CLI 与 Evidence](../cli.md)：命令、提供方选择、快照、导入/导出和退出状态。 +- [MCP 契约](../mcp-contracts.md)和[智能体提示词](../mcp-prompts.md)：工具结果、会话和引导式调查。 +- [工具目录](../mcp-contracts.md#generated-catalog)：构建时生成的工具、提供方和 CLI 命令清单。 +- [路线图](../roadmap.md)：计划中的工作和能力跟踪。 -请按照 [SECURITY.md](SECURITY.md) 报告漏洞。 +请按照 [SECURITY.md](../../SECURITY.md) 报告漏洞。 ## Star 历史 @@ -278,7 +282,7 @@ npx rea-agents@latest setup ## 免责声明 -负责任使用的相关指引请见 [DISCLAIMER.md](DISCLAIMER.md)。 +负责任使用的相关指引请见 [DISCLAIMER.md](../../DISCLAIMER.md)。 REA 用于合法的研究、教育、互操作性、软件保存，以及分析你拥有或获准检查的软件。 @@ -289,14 +293,14 @@ REA 用于合法的研究、教育、互操作性、软件保存，以及分析 欢迎帮助改进 REA！你可以[提交 issue](https://github.com/morluto/rea/issues) 报告错误或建议功能，也可以[提交 pull request](https://github.com/morluto/rea/pulls) 改进代码或文档。 -开发设置和检查见 [CONTRIBUTING.md](CONTRIBUTING.md)，验证流程见[测试指南](docs/testing.md)，项目结构见[架构图](docs/architecture.mermaid)。 +开发设置和检查见 [CONTRIBUTING.md](../../CONTRIBUTING.md)，验证流程见[测试指南](../testing.md)，项目结构见[架构图](../architecture.mermaid)。 ## 许可证 -[MIT](LICENSE) +[MIT](../../LICENSE) REA 自行编写的代码采用 MIT 许可；第三方内容保留各自的许可和著作权声明。 -依赖清单、许可原文及外部工具条款见[第三方声明](THIRD_PARTY_NOTICES.md)。 +依赖清单、许可原文及外部工具条款见[第三方声明](../../THIRD_PARTY_NOTICES.md)。 **用户须自行合法取得商用工具及所需使用权**，包括 IDA Pro / Hex-Rays 反编译器、 Hopper 和 JEB，并遵守各自许可。REA 不提供工具许可、密钥或激活权利；试用版和 diff --git a/docs/recorded-crashes.md b/docs/recorded-crashes.md index 8b1a51662..bd508d051 100644 --- a/docs/recorded-crashes.md +++ b/docs/recorded-crashes.md @@ -1,7 +1,10 @@ # Recorded crash evidence `inspect_recorded_crash` / `rea inspect-recorded-crash` inspect a caller-selected -Linux ELF64 x86-64 little-endian core independently of an active binary target. +Linux ELF64 little-endian core independently of an active binary target. Two +architecture profiles are interpreted, x86-64 and AArch64; each binds registers +to its own `NT_PRSTATUS` layout, and any other `e_machine` is refused as +`unsupported_target` before note interpretation. ```sh export REA_PWNTOOLS_PYTHON=/absolute/existing/venv/bin/python @@ -10,8 +13,8 @@ rea inspect-recorded-crash ./crash.core --json The selected Python needs unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2. REA does not install these dependencies. ELFFile/its original ELF structures and -pwntools' original Linux amd64 ctypes provide parsing and ABI layout. The adapter -adds bounds, representation preservation and lifecycle ownership. +pwntools' original Linux amd64 and aarch64 ctypes provide parsing and ABI layout. +The adapter adds bounds, representation preservation and lifecycle ownership. The initial Linux note profile accepts System V/unspecified and GNU/Linux OSABI with ABI version zero. Other declared ABIs return `unsupported_target` before note interpretation; an unspecified OSABI alone does not prove OS origin. @@ -27,7 +30,8 @@ note interpretation; an unspecified OSABI alone does not prove OS origin. - Every exact Linux CORE PRSTATUS note as a separate recorded thread, with historical PID, signal and lossless uint64 registers bound to source bytes. Each supported note has exactly one interpretation; each thread contains the - complete 27-register ABI set, with names bound to their original ABI offsets. + complete ABI register set for its architecture (27 registers for amd64, + 34 for AArch64), with names bound to their original ABI offsets. Physical note ranges cannot overlap or leave segment gaps. ELF and note headers are independently bound to the snapshot, including extended program counts. - Signed SIGINFO number/code/errno. Initially only SIGSEGV codes 1/2 establish the diff --git a/docs/specs/2356-wasm-interface-comparison/plan.md b/docs/specs/2356-wasm-interface-comparison/plan.md new file mode 100644 index 000000000..a5b9e8ac3 --- /dev/null +++ b/docs/specs/2356-wasm-interface-comparison/plan.md @@ -0,0 +1,10 @@ +# Plan + +1. Check issue/timeline and complete open/closed PR searches; review #2406. +2. Extend the existing core-module WABT inspection in an isolated worktree. +3. Extend producer-owned inspection with optional typed interface facts; + preserve unknown unsupported signatures and exact WAT forms. +4. Add shared Evidence-only comparator and strict input/output contracts. +5. Register additive CLI/MCP workflow and documentation. +6. Validate original-module workflows, malformed/unknown/cancellation cases; + run checks, independent review, repair findings and rerun affected checks. diff --git a/docs/specs/2356-wasm-interface-comparison/spec.md b/docs/specs/2356-wasm-interface-comparison/spec.md new file mode 100644 index 000000000..b4dfb7778 --- /dev/null +++ b/docs/specs/2356-wasm-interface-comparison/spec.md @@ -0,0 +1,61 @@ +# Core WebAssembly interface comparison (#2356) + +## Outcome and scope + +Compare two exact authenticated `inspect_wasm_artifact` Evidence inputs, inline +or retained session references, under required `scope: imports | exports | both`. +No module acquisition or execution occurs during comparison. This is a derived +workflow within existing WABT inspection authority, not a new provider. + +Producer-owned typed `interface` facts are additive and optional in historical +inspection Evidence. WABT's validated canonical WAT owns decoding; exact original +forms and names remain available. Comparison never reparses retained display text. +Function/tag signatures resolve type uses; memory page size defaults to 65536 bytes and explicit custom page sizes are retained; table/memory limits and global types +are compared exactly. Unsupported type/proposal forms remain unknown rather than +being compared by raw text or inferred compatible. Names are exact decoded UTF-8, +without normalization, including empty names, escapes, punctuation and BOM. + +## Comparison contract + +Return both Evidence IDs and original artifact path/digest/length. Group imports +by exact module/name and exports by exact name, independently of kind, so a kind +change is reported as changed. Unique admitted signatures yield added, removed, +changed or unchanged. Duplicate keys retain every original occurrence and yield +unknown; no Cartesian pairing or deduplication. Missing historical facets and +incomplete inventories prevent absence claims. Unknown signatures do not erase +independently observed entries. Ordering is deterministic by direction and names. + +Both subject digest/path and inspection parameters must match normalized artifact +identity. Authenticate Evidence before projection; summary Evidence must be +resolved to its retained complete parent by the caller. Report unknown coverage +and explicit limitations. Exact static equality is not runtime compatibility, +ABI compatibility, linking success or source identity. + +## Authority and overlap + +Reads supplied Evidence only. MCP records derived Evidence through the existing +handler/ledger; CLI accepts portable inline Evidence from an explicit JSON input. +No new provider, implicit fetch, target mutation or foreign-code execution. +PR #2406 (head 86865035) adds component inspection, preamble routing and embedded +core-module inspection. Its component parser is not required or duplicated here; +shared contract/CLI registration changes remain additive. Core GC/recursive types +are outside admitted signature semantics and keep unknown mappings. + +## Resource admission + +Typed producer rows, names and declaration tokenization are admitted before +allocation against the existing 32 MiB WABT output representation capacity. +Comparison uses linear grouping and cooperative transport yields; it never +builds Cartesian duplicate pairings. CLI JSON input retains the shared file +admission gate. Token usage and peak process-family memory remain unmeasured. + +## Verification + +Original owned modules assembled using WABT 1.0.42 cover semantically equivalent +encodings, changes of kind/signature, names/duplicates, imported index spaces, +tables/memory/globals/tags, unknown types and historical incomplete Evidence. +Exercise authenticated identity, cancellation, retained lookup, CLI/MCP parity +and advertised schemas. Run focused existing WASM/Evidence tests, repository +checks and docs validation; independent subagent review before handoff. + +Format reference: https://webassembly.github.io/spec/core/binary/modules.html diff --git a/docs/specs/2356-wasm-interface-comparison/tasks.md b/docs/specs/2356-wasm-interface-comparison/tasks.md new file mode 100644 index 000000000..722ace2c9 --- /dev/null +++ b/docs/specs/2356-wasm-interface-comparison/tasks.md @@ -0,0 +1,8 @@ +# Tasks + +- [x] Confirm scope and overlap boundaries against #2406. +- [x] Record spec and implementation plan. +- [x] Implement typed producer interface and shared comparison. +- [x] Add CLI/MCP contracts and public workflow tests. +- [x] Run focused, static and documentation checks. +- [x] Resolve independent review findings and rerun relevant checks. diff --git a/docs/specs/2356-wasm-interface-comparison/verification.md b/docs/specs/2356-wasm-interface-comparison/verification.md new file mode 100644 index 000000000..ca531e8e4 --- /dev/null +++ b/docs/specs/2356-wasm-interface-comparison/verification.md @@ -0,0 +1,47 @@ +# Verification + +Verified base: a2d8b94dd50dab8d966239dd63ab84718aa8b4b4 (fetched main). +The change was reapplied without conflicts and relevant checks rerun. + +- Focused existing/new parser, provider, service, contract and CLI/MCP tests: + 8 files / 56 tests passed. +- After the final partial-uniqueness correction, reran service and compiled + CLI/MCP boundary tests: 2 files / 5 tests passed. +- `npm run check`: typecheck, lint, formatting, knip, module boundaries and + architecture guards passed. Lint reports warnings; no errors. +- `npm run docs:generate`, `npm run docs:check` and + `npm run verify:test-discovery` passed; every test file was discovered exactly once. +- Real WABT 1.0.42 comparison through compiled CLI and stdio MCP passed: + equivalent padded LEB128 original encodings, function signatures, all admitted + kinds, custom memory page size, escaped/BOM names, empty interfaces, explicit + scope, duplicate imports and unadmitted exception reference signatures. +- Existing real WABT artifact verifier passed after correcting its stale inline + partial-observation assumption to resolve the MCP retained reference from the + Evidence bundle and authenticate the original record. + +## Independent review + +Three findings repaired at their owning boundary: + +1. Memory page size omitted: now preserved, default 65536 canonicalized. +2. Shared signature expansion undercounted: charge every retained occurrence. + Reviewer independently measured the 1000-params/10000-exports case: original + 262956-byte WAT produced 61824014-byte JSON before correction. It now retains + 1949 observations in a partial facet, with 12046676-byte JSON and an explicit + capacity issue. Original WAT stays available; incomplete coverage prevents + absence and uniqueness claims. +3. Partial singleton uniqueness: comparison now returns unknown whenever either + selected inventory is incomplete, including matched names. + +Final independent source review reported no actionable findings. Review does not +replace the executed tests or real-tool evidence. + +## Scope and measurement limits + +On the original implementation base f254f5a, the complete comparison verifier reported 7.98 seconds elapsed, 6.56 user and +3.13 system seconds, and 429441024 bytes maximum RSS via macOS `/usr/bin/time -l`. +These are command-accounting measurements, not aggregate concurrent process-family +memory or model tokens. Token usage and full process-family peak are unmeasured. +Real-tool verification covers macOS arm64 only; Linux/Windows remain unverified. +No module was instantiated/executed and no source was fetched. Static equality +is not runtime compatibility. Full CI is separate from this local verification record. diff --git a/docs/specs/solidity/cbor-metadata/plan.md b/docs/specs/solidity/cbor-metadata/plan.md new file mode 100644 index 000000000..04b601b66 --- /dev/null +++ b/docs/specs/solidity/cbor-metadata/plan.md @@ -0,0 +1,33 @@ +# Implementation plan + +1. Use cborg 6.1.3 (Apache-2.0) as a pinned format decoder, with a bounded + tokenizer at its owning adapter boundary. Decode maps as Map, preserve integer + precision and strictly validate emitted text bytes before conversion. +2. Add the portable metadata-result schema in src/domain/evm/. Use exact byte + ranges into the already retained bytecode rather than copying an opaque CBOR + hex payload into every selected summary. +3. Add src/evm/SolidityMetadata.ts to interpret the terminal length/root-map + candidate. Return unavailable, decoded, partial or not-decoded facts; keep + unsupported nested values explicit instead of recursively expanding them. +4. Compute the facts in EvmoleInterfaceWorker from its selected bytes. The host + provider validates ranges/identity and projects the additive metadata field. + Preserve raw engine output without rewriting its unknown-precision metadata. +5. Ensure the canonical EVM bytecode shape carries the optional field into complete + and summary Evidence. Include decoding profile, coverage and limitations. +6. Add producer, worker, historical contract and public MCP boundary coverage. + Compile an owned Solidity fixture with a pinned caller-supplied solc module, + and verify its CBOR compiler version. Run relevant static/architecture checks. +7. Extend the existing real Linux CLI/MCP verifier with independent metadata + assertions. Run the compiler-setting matrix in its CI lane and classify its + verifier as EVM-owned, with a production CI-planner regression. + +No new execution provider or target authority is introduced: a format parser is +added inside the existing EVM worker. Linux x64 provider admission, target-free +semantics, cleanup, limits and complete original Evidence remain unchanged. +No metadata JSON or source URL is automatically acquired. + +The parser bounds nesting to 64 and token/collection growth to the selected +terminal payload's byte count (the format's terminal length is at most 65535). +Every data item consumes encoded bytes; broad declared collection lengths cannot +justify allocation without that lower bound. Expanded results still pass the +worker's existing 16 MiB output check. diff --git a/docs/specs/solidity/cbor-metadata/spec.md b/docs/specs/solidity/cbor-metadata/spec.md new file mode 100644 index 000000000..ea08dcc90 --- /dev/null +++ b/docs/specs/solidity/cbor-metadata/spec.md @@ -0,0 +1,43 @@ +# Solidity terminal metadata inspection + +Issue: https://github.com/morluto/rea/issues/2388 + +## Outcome + +The existing EVM interface operation should retain independently decoded terminal +CBOR candidate facts from the exact selected bytes instead of relying on precision +unknowns in the engine's reported metadata. This increment covers trailer ranges, +bounded decoding, exact integer/non-text scalar key evidence and compiler-version +representations. + +## Requirements + +- Preserve the existing carrier/decoded identities and original full bytecode. +- Report a plausible terminal root-map candidate separately from proof of Solidity + origin. Missing or non-map trailers remain unavailable, not proven absent. +- Interpret scalar root-map keys/values losslessly: integers as decimal strings, + bytes as hex, strings, booleans and null. Preserve undecoded/unsupported values + through the original bytecode plus exact candidate offset/length. +- Report partial and not-decoded coverage, parser/profile failures and unsupported + CBOR forms explicitly. Do not silently coerce unknown keys into object properties. +- Interpret admitted solc release/prerelease representations only from observed + entries; preserve ambiguity if duplicate compiler-version keys occur. +- Keep the additive field compatible with historical Evidence and retain + it through selected summary delivery and public CLI/MCP. + Historical omission parses as a null unknown facet without mutating the source. +- Bound nesting before decoder recursion and collection lengths before allocation. + Keep parsing inside the existing owned worker and its resource/output budgets. +- Do not fetch metadata or source content, contact RPC or execute contract code. + +## Acceptance and limits + +Owned compiler fixtures and malformed/synthetic trailers cover integers beyond +Number precision, non-text keys, nested/uninterpreted values, invalid UTF-8, +invalid lengths, unsupported tags and nesting bounds. Public boundary tests +preserve the field and older omission. A real pinned solc artifact verifies actual +metadata facts; the existing Linux-only EVM host profile remains unchanged. + +## Deferred issue criteria + +Binding a supplied metadata JSON to IPFS/Swarm commitments is a later independent +increment of #2388. This PR must reference rather than close the whole issue. diff --git a/docs/specs/solidity/cbor-metadata/tasks.md b/docs/specs/solidity/cbor-metadata/tasks.md new file mode 100644 index 000000000..e80aefa01 --- /dev/null +++ b/docs/specs/solidity/cbor-metadata/tasks.md @@ -0,0 +1,8 @@ +# Tasks + +- [x] Define single-feature requirements and implementation plan before code. +- [x] Add the pinned decoder and bounded owner boundary. +- [x] Implement lossless scalar metadata facts and compiler-version evidence. +- [x] Integrate worker/provider contracts and summary/historical compatibility. +- [x] Verify producer, worker and public boundary behavior with owned fixtures. +- [x] Run relevant repository checks and prepare a separately scoped PR. diff --git a/docs/specs/solidity/cbor-metadata/verification.md b/docs/specs/solidity/cbor-metadata/verification.md new file mode 100644 index 000000000..dae558d94 --- /dev/null +++ b/docs/specs/solidity/cbor-metadata/verification.md @@ -0,0 +1,43 @@ +# Verification + +After integrating upstream c462a4ebd, 47 tests passed +across the parser, domain contract, application service, compiled worker and MCP +boundary (five files). Compiled-worker goldens include BOM text, partial nested +values, unavailable trailers, malformed UTF-8 and unsupported floats. Five +upstream adapter tests also passed after its stricter integer validation landed. +All six static tasks and architecture guards +passed. After integration, test discovery found all 1004 files exactly once. +The CI-planner selection passed all 85 tests, including the new EVM-verifier case. + +The parser regressions cover exact large integers and typed keys, malformed UTF-8, +duplicate keys, excessive nesting, unsupported tags/floats, length bounds and +historical omission. A real defect in the decoder's default text handling removed +an initial U+FEFF. The owning tokenizer now validates and decodes the retained +text bytes with BOM preservation; independent fixtures check both values and +compiler-key identity. + +The owned fixture was compiled with caller-selected solc 0.8.30, using unoptimized, +optimized via-IR, content-hash-free and CBOR-free settings. All four metadata +checks passed again after integration. No contract was executed. Local first decode +took approximately 0.81 ms or less. The originally reported approximately 2.92 ms +for 100 warm iterations included equality assertions and is not a decode-only +measurement. The verifier now measures parsing separately from those assertions; +updated warm timings have not been collected. These historical figures describe +these fixtures and this host, not a general workload. +Peak process RSS +included the compiler (299536 KiB); complete process-family CPU/memory and agent +token costs were not measured. + +Compiled worker tests exercise unchanged EVMole WASM on this macOS host. MCP tests +use the production server and contracts with an injected provider seam. The full +Linux-only provider CLI/MCP, resource-limit and cleanup lane remains unverified +here. Metadata JSON commitment binding remains deferred under issue #2388. + +The Linux CLI/MCP verifier asserts exact compiler bytes, declared version and +candidate ranges for its existing owned raw/hex artifacts. Its CI lane additionally +runs the new compiler-setting matrix. A regression using the actual CI planner +checks that edits to the new verifier select the EVM lane. + +The later merge/review correction adds decoded-summary and omitted historical +metadata coverage. Tests and provider reruns for that correction were skipped at +the maintainer's request; the earlier runs above do not validate the changed head. diff --git a/docs/testing.md b/docs/testing.md index 7aa80d22a..213c48e44 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -646,6 +646,12 @@ records independent symbol/directory/SHA oracles; it never executes the target. Linux also builds stripped, ordinary-native and small malformed/ambiguous/layout negative inputs. The optional Windows fixture workflow builds a PE on a Windows runner; analyzing that PE on Linux does not verify a Windows Ghidra host. +On a local Windows host, the .NET 8 ILCompiler finds the linker only through +Visual Studio 2019 or 2022 (`vswhere -version [16,18)`). With only Visual +Studio Build Tools 2026 installed, open its x64 Developer Command Prompt. +In that same `cmd.exe` session, run `set IlcUseEnvironmentalTools=true` and +then `npm run build:fixtures:nativeaot`. Running `vcvars64.bat` as a child of +PowerShell does not preserve its environment in the parent PowerShell session. Build the clean pinned upstream adapter with `build:ghidra:nativeaot`, then set `REA_GHIDRA_NATIVEAOT_JAR`. Run `verify:ghidra:nativeaot -- symbols`, `-- stripped`, diff --git a/docs/user-visible-errors.csv b/docs/user-visible-errors.csv index a9dd0d05d..02237dcdc 100644 --- a/docs/user-visible-errors.csv +++ b/docs/user-visible-errors.csv @@ -1,4 +1,7 @@ harm_rank,location,trigger,current_copy,user_risk,proposed_replacement,implementation_status,verification_result +5,src/domain/native/bytePattern.ts,Unconstrained byte signature,Pattern must constrain at least one bit,Low risk; identifies input correction,Keep current copy,implemented,parser and MCP validation tests +5,src/domain/native/bytePattern.ts,Unequal mask length,Mask length must equal pattern length,Low risk; identifies input correction,Keep current copy,implemented,parser and MCP validation tests +5,bridge/ghidra/ReaGhidraBridge.java:searchMemoryBytes,Cursor reused with another query,Byte search cursor does not match the target and query,Low risk; identifies continuation mismatch,Keep current copy,implemented,real-provider verification required 1,src/domain/errors.ts:AnalysisCapabilityUnavailableError,Requested analysis unavailable,"Provider {providerId} cannot execute {operation}: {reason}",Exposes provider and internal operation/reason,"This analysis is unavailable for the current target. Choose another analysis or target.",implemented,focused assertions passed; full `npm run check` passed 2,src/domain/errors.ts:ProviderAdapterError,Provider adapter fails,"Provider {providerId} adapter failed during {operation}",Exposes provider and implementation boundary,"Analysis could not complete. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.",implemented,focused assertions passed; full `npm run check` passed 3,src/domain/errors.ts:HopperRemoteError,Remote analysis request fails,"Hopper request failed ({code}): {safeMessage}",Exposes provider name code and upstream text,"Analysis could not complete. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.",implemented,focused assertions passed; full `npm run check` passed diff --git a/docs/wasm-artifacts.md b/docs/wasm-artifacts.md index 6bc7c23ce..17b6c77f9 100644 --- a/docs/wasm-artifacts.md +++ b/docs/wasm-artifacts.md @@ -13,6 +13,11 @@ Check `binary_session` for configured capability availability. A recognized header is not validation of the complete module; missing WABT is reported separately as an unavailable capability, not as missing native engine support. +The recognized WASM header is checked before automatic database routing from a +.hop filename. Renaming a module to .hop therefore keeps the static-inspection +diagnostic. An explicit database selection or raw/COM format selection retains +the caller's requested interpretation; ordinary .hop database routing is unchanged. + Set `REA_WABT_BIN_DIRECTORY` to an absolute directory containing executable `wasm-validate`, `wasm-objdump` and `wasm2wat` (with `.exe` on Windows). Missing executables, unsupported banners and inaccessible tools produce actionable @@ -79,3 +84,46 @@ Both CLI and MCP use the same typed port and application service. MCP publishes SDK-compatible schemas and records Evidence for `get_evidence_bundle` readback. See [WABT provenance](https://github.com/morluto/rea/blob/main/third_party/wabt/README.md) and [testing](testing.md#offline-wasm-artifacts) for real-tool verification. + +## Compare core-module interfaces + +Inspection adds producer-owned typed `interface` facts while keeping exact WAT +import/export forms. Function/tag signatures resolve type indices; table/memory +limits, memory page size and global value type/mutability remain explicit. +Unadmitted types (including GC reference signatures) retain null signatures and +issues. Exact decoded names retain escapes, empty names, BOM and punctuation. +This facet is absent in historical Evidence that did not examine typed interfaces. + +`compare_wasm_interfaces` accepts two complete `inspect_wasm_artifact` Evidence +records, or exact retained references in the current MCP session, and required +`scope: "imports" | "exports" | "both"`. It reads no files and needs no active +WABT toolchain. CLI uses portable inline Evidence in one JSON document: + +```bash +rea compare-wasm-interfaces comparison-input.json --json +``` + +```json +{ + "left": { "kind": "retained-evidence", "evidence_id": "ev_" }, + "right": { "kind": "retained-evidence", "evidence_id": "ev_" }, + "scope": "both" +} +``` + +The references above are MCP-only; put complete Evidence objects in the CLI +input. Resolve summary delivery to its retained complete parent before comparing. +Results retain both input Evidence IDs, artifact paths/digests/lengths and every +selected occurrence. Unique names are added, removed, changed or unchanged; +duplicate names and unsupported signatures are unknown. Incomplete or historical +inventories never establish absence. Imports group by module/name, exports by +name, so a change of kind is visible. Output ordering is deterministic. + +Static type equality does not establish runtime compatibility, ABI compatibility +or linking success. Comparison covers core modules; WebAssembly components and +canonical ABI relationships belong to their separate inspection workflow. + +After building, run `REA_WABT_BIN_DIRECTORY=/absolute/wabt/bin node +scripts/verify-wasm-interface-comparison.mjs` for original-module comparison +through the compiled CLI and stdio MCP. It uses owned fixtures and WABT 1.0.42; +no WASM module is instantiated or executed. diff --git a/docs/windows-ghidra-p0.md b/docs/windows-ghidra-p0.md index 5def0dae4..78e11ee79 100644 --- a/docs/windows-ghidra-p0.md +++ b/docs/windows-ghidra-p0.md @@ -29,8 +29,10 @@ admitted raw NMOS 6502 mapping on Windows. Windows P0 admission remains limited to native PE and raw NMOS 6502 images. For x64 PE NativeAOT RTR 9.1, the read-only snapshot parser can report metadata through `inspect_native_load_image` and `inspect_native_data_type` without a JAR or -database mutation. A real Windows-host NativeAOT Ghidra run is still required -to establish the complete host workflow. +database mutation: MethodTable relationships, virtual slots and frozen strings. +`analyze_function` decompiles slot targets. Database reads of the rehydrated +bytes (`read_bytes`, `search_strings`) return nothing on this host. See +[NativeAOT](ghidra-nativeaot.md). This boundary does not establish general Windows feature parity. Ghidra GUI state, annotations, other target architectures and formats, and Hopper-only diff --git a/licenses/npm-dependencies.txt b/licenses/npm-dependencies.txt index ce45f7459..05fc2f173 100644 --- a/licenses/npm-dependencies.txt +++ b/licenses/npm-dependencies.txt @@ -2,7 +2,7 @@ REA npm dependency inventory Prepared from the root package-lock.json; refresh this inventory when locked dependencies change. License identifiers are upstream lockfile declarations, not a legal assessment. Runtime includes transitive and optional platform packages; development includes build, test and website tooling. -Locked package entries: 586; runtime: 174. +Locked package entries: 587; runtime: 175. Full runtime copyright/license/NOTICE texts: npm-runtime-notices.txt. @algolia/abtesting@1.25.0 | development | MIT @@ -1430,6 +1430,11 @@ canonicalize@3.0.0 | runtime | Apache-2.0 Package: https://www.npmjs.com/package/canonicalize/v/3.0.0 Archive: https://registry.npmjs.org/canonicalize/-/canonicalize-3.0.0.tgz Integrity: sha512-yYLfHyDMIXRyRqsKBRLX023riFLpXY2YOfdtqKXZRZy9qsfOJ9U+4F9YZL7MEzL5+ziN2x2nlBvY/Voi3EBljA== +cborg@6.1.3 | runtime | Apache-2.0 + Lockfile path: node_modules/cborg + Package: https://www.npmjs.com/package/cborg/v/6.1.3 + Archive: https://registry.npmjs.org/cborg/-/cborg-6.1.3.tgz + Integrity: sha512-Kb/wD/AFuVcqvVjR+HYNOjNiuhVJ2C+RWxQE2Yqq2WVlVkRjdvVLkkM6QsL3NMsBRh+rgNc+aXmcraA+X0HObQ== ccount@2.0.1 | development | MIT Lockfile path: node_modules/ccount Package: https://www.npmjs.com/package/ccount/v/2.0.1 diff --git a/licenses/npm-runtime-notices.txt b/licenses/npm-runtime-notices.txt index d981aa68a..2a0c45d24 100644 --- a/licenses/npm-runtime-notices.txt +++ b/licenses/npm-runtime-notices.txt @@ -1,7 +1,7 @@ REA third-party npm runtime copyright, license and NOTICE texts Collected from integrity-verified locked npm archives. Release-specific supplements and upstream notice gaps are identified in their entries. External engines are separately obtained; see THIRD_PARTY_NOTICES.md. -Runtime dependency fingerprint (SHA-256): 5de9cf97e7c327da1892efeff30b40507e4c7f8e496b5d305b8016c162bc15e3 +Runtime dependency fingerprint (SHA-256): 63d5b55e027a0cfb3e77989ca337aca905f1652d10cfbfc4d4dd876d3ecd4f3a ===== @babel/helper-string-parser@8.0.0 ===== Declared license: MIT @@ -2456,6 +2456,234 @@ Scope: npm runtime dependency, including optional platform packages. limitations under the License. +===== cborg@6.1.3 ===== +Declared license: Apache-2.0 +Archive: https://registry.npmjs.org/cborg/-/cborg-6.1.3.tgz +Integrity: sha512-Kb/wD/AFuVcqvVjR+HYNOjNiuhVJ2C+RWxQE2Yqq2WVlVkRjdvVLkkM6QsL3NMsBRh+rgNc+aXmcraA+X0HObQ== +Scope: npm runtime dependency, including optional platform packages. + +--- package/LICENSE --- + +Copyright 2020 Rod Vagg + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +--- Apache-2.0 full license (licenses/Apache-2.0.txt) --- + + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + ===== clean-git-ref@2.0.1 ===== Declared license: Apache-2.0 Archive: https://registry.npmjs.org/clean-git-ref/-/clean-git-ref-2.0.1.tgz diff --git a/native/windows/README.md b/native/windows/README.md index e2543260f..86103ce9a 100644 --- a/native/windows/README.md +++ b/native/windows/README.md @@ -90,8 +90,10 @@ without this lane reports Windows native controls as unavailable. root and ancestor leases for retry; a partially removed runtime accepts only cleanup, and its registry entry remains until removal succeeds. Normal close, cancellation, startup failure, and timeout release provider resources. Abrupt owner death - kills the owned job; it can leave private runtime files for later operator - cleanup and does not claim transactional deletion after a crash. + kills the owned job. The Android provider additionally uses a detached helper + that owns its private runtime directory and cleans it when the owner pipe + closes. Other providers can leave private runtime files after owner death; + helper death and host power loss are not transactional deletion. Windows failures retain their constraint, requested coordinate, Win32 code, and system message. Unsupported filesystems and reparse paths are distinct @@ -115,8 +117,10 @@ the supported device/UNC namespaces; existing admission checks still apply. The bearer-token descriptor remains under its immutable private lease until `runtime_close` on Windows; POSIX removes the descriptor after the bridge reads -it. Job ownership guarantees process termination on owner death, not deletion -of private files after a crash. SUBST drive aliases and changing DOS-device +it. Job ownership guarantees process termination on owner death. Android's +detached workspace helper provides a separate cleanup owner for that provider, +but simultaneous helper death or power loss can still leave private files. +SUBST drive aliases and changing DOS-device namespaces are outside the verified P0 scope and are not detected as a separate alias policy. Mounted-folder paths that report a reparse tag are rejected by component admission; broader namespace variants remain unverified. Preserve @@ -163,9 +167,14 @@ V8 heap flags cannot bound (limit notification, peak commit at the limit, all members settled, with the notification present in the first native exit report and the wrapper's exit event), and a control under the same limit that completes normally. -An optional independently built -`tests/fixtures/windows/processBoundary.cc` executable adds breakaway and -caller/child token observations and an in-place ancestor reparse attempt: +A separately built `tests/fixtures/windows/processBoundary.cc` executable adds +breakaway, caller/child token and environment semantics observations and an +in-place ancestor reparse attempt. The lane skips all of them when the +executable is absent, so both Windows workflows cross-build it in the same +MinGW job that builds their artifact and pass it: the capability and package +lane in `.github/workflows/ci.yml`, and the real Ghidra proof run in +`.github/workflows/real-ghidra-windows.yml`. Build it locally with any Windows +C++ compiler, or the same way they do, and pass its path: ```powershell node scripts/verify-windows-native.mjs . C:\fixtures\process-boundary.exe diff --git a/native/windows/src/filesystem.cc b/native/windows/src/filesystem.cc index c8f66e5c1..71c8bbd44 100644 --- a/native/windows/src/filesystem.cc +++ b/native/windows/src/filesystem.cc @@ -1,6 +1,7 @@ #include "authority.hpp" #include #include +#include namespace rea { @@ -165,7 +166,9 @@ static Handle createRelative(HANDLE parent, const std::wstring& name, DWORD acce IO_STATUS_BLOCK observation{}; HANDLE raw = INVALID_HANDLE_VALUE; const auto status = NtCreateFile(&raw, access | SYNCHRONIZE, &attributes, &observation, nullptr, - FILE_ATTRIBUTE_NORMAL, FILE_SHARE_READ, FILE_CREATE, + FILE_ATTRIBUTE_NORMAL, + directory ? FILE_SHARE_READ | FILE_SHARE_DELETE : FILE_SHARE_READ, + FILE_CREATE, FILE_SYNCHRONOUS_IO_NONALERT | FILE_OPEN_REPARSE_POINT | (directory ? FILE_DIRECTORY_FILE : FILE_NON_DIRECTORY_FILE), nullptr, 0); if (status < 0) throw Failure("Create private object relative to its admitted parent failed", path, @@ -176,6 +179,30 @@ static Handle createRelative(HANDLE parent, const std::wstring& name, DWORD acce return result; } +// Lookup relative to an already-owned directory; path is diagnostic only. +static Handle openRelative(HANDLE parent, const std::wstring& name, DWORD access, + DWORD sharing, ULONG kind, const std::wstring& path) { + require(!name.empty() && name.find_first_of(L"\\/:\"<>|?*") == std::wstring::npos && name.size() < 32768, + "Private lookup requires one filename component", path, ERROR_INVALID_PARAMETER); + UNICODE_STRING coordinate{}; + coordinate.Buffer = const_cast(name.data()); + coordinate.Length = static_cast(name.size() * sizeof(wchar_t)); + coordinate.MaximumLength = coordinate.Length; + OBJECT_ATTRIBUTES attributes; + InitializeObjectAttributes(&attributes, &coordinate, OBJ_CASE_INSENSITIVE, parent, nullptr); + IO_STATUS_BLOCK observation{}; + HANDLE raw = INVALID_HANDLE_VALUE; + const auto status = NtCreateFile(&raw, access | SYNCHRONIZE, &attributes, &observation, + nullptr, FILE_ATTRIBUTE_NORMAL, sharing, FILE_OPEN, + FILE_SYNCHRONOUS_IO_NONALERT | FILE_OPEN_REPARSE_POINT | kind, nullptr, 0); + if (status < 0) + throw Failure("Open private object relative to its directory handle failed", path, + RtlNtStatusToDosError(status)); + Handle result(raw); + require(result.valid(), "Private lookup returned an invalid handle", path); + return result; +} + static void verifyPrivate(HANDLE handle, const std::wstring& path, bool root) { Security expected; PSID owner = nullptr; @@ -258,7 +285,16 @@ static std::unique_ptr runtimeDirectory(Runtime& root, const std::wstring& size_t offset = root.path.size() + 1; while (offset < path.size()) { const auto end = path.find(L'\\', offset); - result->handles.push_back(openComponent(path.substr(0, end), 0, true)); + const auto name = path.substr(offset, end == std::wstring::npos ? end : end - offset); + auto child = openRelative(result->get(), name, + FILE_LIST_DIRECTORY | FILE_READ_ATTRIBUTES | READ_CONTROL, + FILE_SHARE_READ, FILE_DIRECTORY_FILE, path.substr(0, end)); + FILE_ATTRIBUTE_TAG_INFO info{}; + require(GetFileInformationByHandleEx(child.get(), FileAttributeTagInfo, &info, sizeof(info)), + "Read private directory tag failed", path); + require((info.FileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) == 0, + "Reparse private directory is unsupported", path, ERROR_NOT_SUPPORTED); + result->handles.push_back(std::move(child)); if (end == std::wstring::npos) break; offset = end + 1; } @@ -320,7 +356,9 @@ static void disposeSnapshot(Runtime& root, Handle& file, const std::wstring& pat require(GetFileInformationByHandleEx(file.get(), FileIdInfo, &expected, sizeof(expected)), "Read incomplete snapshot identity failed", path); file.reset(); - auto cleanup = openComponent(path, DELETE | GENERIC_READ | FILE_WRITE_ATTRIBUTES, false); + auto cleanup = openRelative(parent->get(), path.substr(path.find_last_of(L'\\') + 1), + DELETE | GENERIC_READ | FILE_WRITE_ATTRIBUTES, + FILE_SHARE_READ, FILE_NON_DIRECTORY_FILE, path); FILE_ID_INFO observed{}; require(GetFileInformationByHandleEx(cleanup.get(), FileIdInfo, &observed, sizeof(observed)), "Read snapshot cleanup identity failed", path); @@ -339,21 +377,51 @@ void closeRuntime(Runtime& root) { // cleanup. POSIX removes its consumed descriptor earlier. Abrupt owner death // closes process jobs but cannot run this walk, so private files can remain. root.immutableFiles.clear(); - // Each traversed directory stays locked against replacement. Reparse entries - // are unlinked as objects; their targets are never enumerated or deleted. + // Enumerate and open children through the retained directory handles. Paths + // below are diagnostic labels only: renames and ancestor junction replacement + // cannot redirect cleanup into an unrelated tree. struct Frame { Handle directory; - HANDLE search = INVALID_HANDLE_VALUE; std::wstring path; - WIN32_FIND_DATAW next{}; - bool pending = false; - Frame(Handle handle, std::wstring path) : directory(std::move(handle)), path(std::move(path)) { - search = FindFirstFileExW((L"\\\\?\\" + this->path + L"\\*").c_str(), FindExInfoBasic, &next, - FindExSearchNameMatch, nullptr, 0); - pending = search != INVALID_HANDLE_VALUE; - require(pending || GetLastError() == ERROR_FILE_NOT_FOUND, "Enumerate runtime cleanup failed", this->path); + // One NTFS component is at most 255 UTF-16 units; 1 KiB fits a record + // without retaining a large batch at every directory depth. + alignas(FILE_ID_BOTH_DIR_INFO) std::array entries{}; + size_t offset = 0; + bool buffered = false; + bool restart = true; + Frame(Handle handle, std::wstring path) : directory(std::move(handle)), path(std::move(path)) {} + bool next(std::wstring& name) { + for (;;) { + if (!buffered) { + const auto kind = restart ? FileIdBothDirectoryRestartInfo : FileIdBothDirectoryInfo; + restart = false; + if (!GetFileInformationByHandleEx(directory.get(), kind, entries.data(), + static_cast(entries.size()))) { + const auto code = GetLastError(); + if (code == ERROR_NO_MORE_FILES) return false; + throw Failure("Enumerate runtime cleanup by handle failed", path, code); + } + offset = 0; + buffered = true; + } + require(offset <= entries.size() - offsetof(FILE_ID_BOTH_DIR_INFO, FileName), + "Cleanup directory record is truncated", path, ERROR_INVALID_DATA); + const auto* entry = reinterpret_cast(entries.data() + offset); + require(entry->FileNameLength % sizeof(wchar_t) == 0 && + entry->FileNameLength <= entries.size() - offset - offsetof(FILE_ID_BOTH_DIR_INFO, FileName), + "Cleanup directory filename is truncated", path, ERROR_INVALID_DATA); + name.assign(entry->FileName, entry->FileNameLength / sizeof(wchar_t)); + if (entry->NextEntryOffset == 0) buffered = false; + else { + require(entry->NextEntryOffset % alignof(FILE_ID_BOTH_DIR_INFO) == 0 && + entry->NextEntryOffset >= offsetof(FILE_ID_BOTH_DIR_INFO, FileName) + entry->FileNameLength && + entry->NextEntryOffset <= entries.size() - offset - offsetof(FILE_ID_BOTH_DIR_INFO, FileName), + "Cleanup directory record offset is invalid", path, ERROR_INVALID_DATA); + offset += entry->NextEntryOffset; + } + if (name != L"." && name != L"..") return true; + } } - ~Frame() { if (search != INVALID_HANDLE_VALUE) FindClose(search); } }; std::vector> frames; HANDLE duplicate; @@ -362,21 +430,16 @@ void closeRuntime(Runtime& root) { frames.push_back(std::make_unique(Handle(duplicate), root.path)); while (!frames.empty()) { auto& frame = *frames.back(); - if (!frame.pending) { - if (frame.search != INVALID_HANDLE_VALUE) { FindClose(frame.search); frame.search = INVALID_HANDLE_VALUE; } + std::wstring name; + if (!frame.next(name)) { dispose(frame.directory.get(), frame.path); frames.pop_back(); continue; } - const auto name = std::wstring(frame.next.cFileName); - frame.pending = FindNextFileW(frame.search, &frame.next) != FALSE; - require(frame.pending || GetLastError() == ERROR_NO_MORE_FILES, "Continue runtime cleanup enumeration failed", frame.path); - if (name == L"." || name == L"..") continue; const auto path = frame.path + L"\\" + name; - Handle child(CreateFileW((L"\\\\?\\" + path).c_str(), DELETE | GENERIC_READ | FILE_WRITE_ATTRIBUTES, - FILE_SHARE_READ, nullptr, OPEN_EXISTING, - FILE_FLAG_OPEN_REPARSE_POINT | FILE_FLAG_BACKUP_SEMANTICS, nullptr)); - require(child.valid(), "Open owned cleanup object failed", path); + auto child = openRelative(frame.directory.get(), name, + DELETE | FILE_LIST_DIRECTORY | FILE_READ_ATTRIBUTES | FILE_WRITE_ATTRIBUTES, + FILE_SHARE_READ | FILE_SHARE_DELETE, 0, path); FILE_ATTRIBUTE_TAG_INFO info{}; require(GetFileInformationByHandleEx(child.get(), FileAttributeTagInfo, &info, sizeof(info)), "Read cleanup object tag failed", path); if ((info.FileAttributes & FILE_ATTRIBUTE_DIRECTORY) != 0 && (info.FileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) == 0) @@ -551,9 +614,16 @@ napi_value filesystemCall(napi_env env, const std::wstring& operation, const std // must share both to pass Windows' symmetric share check. Those retained // handles still omit FILE_SHARE_WRITE and FILE_SHARE_DELETE, blocking // competing writes and delete opens. - file->handles.push_back(openComponent(path, GENERIC_READ, false, - FILE_SHARE_READ | FILE_SHARE_WRITE | - FILE_SHARE_DELETE)); + auto child = openRelative(file->get(), path.substr(path.find_last_of(L'\\') + 1), + GENERIC_READ | FILE_READ_ATTRIBUTES | READ_CONTROL, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, + FILE_NON_DIRECTORY_FILE, path); + FILE_ATTRIBUTE_TAG_INFO info{}; + require(GetFileInformationByHandleEx(child.get(), FileAttributeTagInfo, &info, sizeof(info)), + "Read private file tag failed", path); + require((info.FileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) == 0, + "Reparse private file is unsupported", path, ERROR_NOT_SUPPORTED); + file->handles.push_back(std::move(child)); verifyPrivate(file->get(), path, false); auto result = identity(env, file->get(), path); set(env, result, "handle", wrap(env, std::move(file))); diff --git a/package-lock.json b/package-lock.json index 5af71e98e..7eae3799e 100644 --- a/package-lock.json +++ b/package-lock.json @@ -24,6 +24,7 @@ "ajv": "8.20.0", "ajv-formats": "3.0.1", "canonicalize": "3.0.0", + "cborg": "6.1.3", "cross-spawn": "7.0.6", "diff": "9.0.0", "evmole": "0.9.3", @@ -4181,6 +4182,15 @@ "node": ">=18" } }, + "node_modules/cborg": { + "version": "6.1.3", + "resolved": "https://registry.npmjs.org/cborg/-/cborg-6.1.3.tgz", + "integrity": "sha512-Kb/wD/AFuVcqvVjR+HYNOjNiuhVJ2C+RWxQE2Yqq2WVlVkRjdvVLkkM6QsL3NMsBRh+rgNc+aXmcraA+X0HObQ==", + "license": "Apache-2.0", + "bin": { + "cborg": "lib/bin.js" + } + }, "node_modules/ccount": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/ccount/-/ccount-2.0.1.tgz", diff --git a/package.json b/package.json index 7ee2b42ab..19e978205 100644 --- a/package.json +++ b/package.json @@ -62,6 +62,7 @@ "bridge/android/ReaJadxBridge.java", "bridge/mitmproxy/capture.py", "scripts/rea.mjs", + "scripts/rea-android-workspace-guardian.mjs", "scripts/electron-active-hook.cjs", "scripts/electron-active-hook-boundaries.cjs", "scripts/hopper-demo-x11.py", @@ -265,11 +266,14 @@ "verify:browser:source-maps": "npm run build:cached && node scripts/verify-browser-source-maps.mjs", "verify:browser:runtime": "npm run build:cached && node scripts/verify-browser-runtime.mjs", "verify:web:network-captures": "npm run build:cached && node scripts/verify-web-network-captures.mjs", + "verify:web:http3": "npm run build:cached && node scripts/verify-http3-network-captures.mjs", "verify:binary:layout": "npm run build:cached && node scripts/verify-binary-layout.mjs", "verify:go:binary": "npm run build:cached && node scripts/verify-go-binary.mjs", "verify:wasm:package": "npm run build:cached && node scripts/verify-packaged-wasm-artifact.mjs", + "verify:wasm-interface-comparison": "npm run build:cached && node scripts/verify-wasm-interface-comparison.mjs", "verify:wasm:artifact": "npm run build:cached && node scripts/verify-wasm-artifact.mjs", "verify:evm:interface": "npm run build:cached && node scripts/verify-evm-interface.mjs", + "verify:solidity:metadata": "npm run build:cached && node scripts/verify-solidity-metadata.mjs", "verify:recorded:crash": "npm run build:cached && node scripts/verify-recorded-crash.mjs", "mcp-catalog:check:built": "node scripts/generate-mcp-tool-catalog.mjs --check", "compile": "node scripts/clean-build-output.mjs && tsc -p tsconfig.build.json && node scripts/bundle-runtime.mjs", @@ -309,6 +313,7 @@ "ajv": "8.20.0", "ajv-formats": "3.0.1", "canonicalize": "3.0.0", + "cborg": "6.1.3", "cross-spawn": "7.0.6", "diff": "9.0.0", "evmole": "0.9.3", diff --git a/scripts/ci/scopes.mjs b/scripts/ci/scopes.mjs index ffe15cb3d..a4f0f0fac 100644 --- a/scripts/ci/scopes.mjs +++ b/scripts/ci/scopes.mjs @@ -55,6 +55,9 @@ export const SCOPES = { "tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts", "tests/boundary/filesystem/setupUnavailableProvider.test.ts", "tests/boundary/filesystem/windowsNpxRegistration.test.ts", + "tests/boundary/android/workspaceGuardianRetry.test.ts", + "src/android/JadxWorkspaceRoot.ts", + "scripts/rea-android-workspace-guardian.mjs", "tests/acceptance/setup/homeDirectoryFallback.test.ts", "tests/boundary/filesystem/javascriptApplicationWorkflows.test.ts", "src/domain/javascript/javascriptRuntimeStaticCandidates.ts", @@ -218,6 +221,7 @@ export const SCOPES = { "src/artifacts/readStableArtifact*.ts", "src/artifacts/ArtifactReader.ts", "scripts/verify-evm-interface.mjs", + "scripts/verify-solidity-metadata.mjs", "scripts/fixtures/evm-interface.sol", "third_party/evmole/**", ], @@ -359,6 +363,7 @@ export const SCOPES = { "website/**", "scripts/*website*.py", "README*.md", + "docs/readme/README*.md", ".github/workflows/website-check.yml", ".github/workflows/website-pages.yml", ], diff --git a/scripts/fixtures/http3-lab-requirements.txt b/scripts/fixtures/http3-lab-requirements.txt new file mode 100644 index 000000000..57e2af74a --- /dev/null +++ b/scripts/fixtures/http3-lab-requirements.txt @@ -0,0 +1,10 @@ +# Exercised runtime graph: CPython 3.14.7, Linux x86_64 guest. +aioquic==1.2.0 +attrs==26.1.0 +certifi==2026.7.22 +cffi==2.1.1 +cryptography==50.0.2 +pycparser==3.11 +pylsqpack==0.3.24 +pyOpenSSL==26.4.0 +service-identity==26.1.0 diff --git a/scripts/fixtures/http3-lab.py b/scripts/fixtures/http3-lab.py new file mode 100644 index 000000000..0f1ed7480 --- /dev/null +++ b/scripts/fixtures/http3-lab.py @@ -0,0 +1,416 @@ +"""Real loopback HTTP/3 laboratory using aioquic and caller-supplied mitmdump. + +Run inside a disposable Linux guest. All native records are written by +mitmproxy's real save addon, not constructed fixtures. Endpoint stream IDs are +observations scoped to that endpoint; no MITM stream mapping is claimed. +""" + +import argparse +import asyncio +import base64 +import hashlib +import json +import os +import signal +import ssl +import subprocess +import time +from pathlib import Path + +import aioquic +from aioquic.asyncio import QuicConnectionProtocol, connect, serve +from aioquic.h3.connection import H3Connection +from aioquic.h3.events import DataReceived, HeadersReceived +from aioquic.quic.configuration import QuicConfiguration +from aioquic.quic.events import ConnectionTerminated, HandshakeCompleted, StreamReset +from aioquic.quic.packet import QuicProtocolVersion + + +def binary(value): + return {"base64": base64.b64encode(value).decode("ascii"), "bytes": len(value), + "sha256": hashlib.sha256(value).hexdigest()} + + +def pairs(headers): + return [[binary(name), binary(value)] for name, value in headers] + + +class Journal: + def __init__(self, root): + self.root = root + self.events = [] + self.requests = {} + self.responses = {} + self.tasks = set() + self.children = [] + self.writers = [] + + def emit(self, event, **data): + value = {"event": event, "monotonic": time.monotonic(), **data} + self.events.append(value) + with (self.root / "endpoints.jsonl").open("a") as handle: + handle.write(json.dumps(value) + "\n") + + def task(self, coroutine): + task = asyncio.create_task(coroutine) + self.tasks.add(task) + return task + + + +class Endpoint(QuicConnectionProtocol): + def __init__(self, *args, journal, endpoint, **kwargs): + super().__init__(*args, **kwargs) + self.journal, self.endpoint = journal, endpoint + self.http = H3Connection(self._quic) + self.messages = {} + self.waiters = {} + self.concurrent = [] + self.identity = f"{endpoint}-{len(journal.events)}" + + def quic_event_received(self, event): + if isinstance(event, HandshakeCompleted): + # This is the negotiated runtime value in pinned aioquic 1.2.0, + # not configuration.supported_versions or mitmproxy's QUICv1 label. + self.journal.emit("handshake", endpoint=self.endpoint, connection=self.identity, + alpn=event.alpn_protocol, quic_version=int(self._quic._version)) + if isinstance(event, (StreamReset, ConnectionTerminated)): + self.journal.emit(type(event).__name__, endpoint=self.endpoint, + connection=self.identity, error_code=event.error_code, + stream_id=getattr(event, "stream_id", None), + reason=getattr(event, "reason_phrase", None)) + targets = self.waiters.values() if isinstance(event, ConnectionTerminated) else [self.waiters.get(event.stream_id)] + for waiter in targets: + if waiter is not None and not waiter.done(): + waiter.set_exception(ConnectionError(type(event).__name__)) + for item in self.http.handle_event(event): + if isinstance(item, (HeadersReceived, DataReceived)): + message = self.messages.setdefault(item.stream_id, {"headers": [], "body": bytearray()}) + if isinstance(item, HeadersReceived): + message["headers"].extend(item.headers) + else: + message["body"].extend(item.data) + if item.stream_ended: + self.completed(item.stream_id, message) + + def completed(self, stream_id, message): + body = bytes(message["body"]) + if self.endpoint == "client": + self.journal.emit("response-received", endpoint="client", connection=self.identity, + stream_id=stream_id, headers=pairs(message["headers"]), body=binary(body)) + self.waiters[stream_id].set_result(message) + return + headers = dict(message["headers"]) + case = headers.get(b"x-lab-case", b"curl").decode("ascii") + self.journal.requests[case] = {"headers": pairs(message["headers"]), "body": binary(body)} + self.journal.emit("request-received", endpoint="origin", connection=self.identity, + stream_id=stream_id, case=case, **self.journal.requests[case]) + if case.startswith("parallel-"): + self.concurrent.append((case, stream_id, body)) + if len(self.concurrent) == 3: + self.journal.task(self.answer_parallel()) + else: + self.answer(case, stream_id, b"\x00\xfeanswer:" + case.encode() + b":" + body[::-1]) + + async def answer_parallel(self): + for case, stream_id, body in reversed(self.concurrent): + self.answer(case, stream_id, b"\x00\xfeanswer:" + case.encode() + b":" + body[::-1]) + await asyncio.sleep(0.15) + + def answer(self, case, stream_id, body): + headers = [(b":status", b"200"), (b"content-type", b"application/octet-stream"), + (b"content-length", str(len(body)).encode()), + (b"x-lab-case", case.encode()), (b"x-duplicate", b"one"), (b"x-duplicate", b"two")] + self.journal.responses[case] = {"headers": pairs(headers), "body": binary(body), "finished": True} + self.journal.emit("response-sent", endpoint="origin", connection=self.identity, + stream_id=stream_id, case=case, **self.journal.responses[case]) + self.http.send_headers(stream_id, headers) + self.http.send_data(stream_id, body, end_stream=True) + self.transmit() + + def request(self, case, body): + stream_id = self._quic.get_next_available_stream_id() + headers = [(b":method", b"POST"), (b":scheme", b"https"), + (b":authority", b"lab.test:18443"), (b":path", b"/same?token=ordinary&x=%2F"), + (b"x-lab-case", case.encode()), (b"content-type", b"application/octet-stream"), + (b"content-length", str(len(body)).encode()), + (b"x-duplicate", b"first"), (b"x-duplicate", b"second")] + self.waiters[stream_id] = asyncio.get_running_loop().create_future() + self.journal.emit("request-sent", endpoint="client", connection=self.identity, + stream_id=stream_id, case=case, headers=pairs(headers), body=binary(body), finished=True) + self.http.send_headers(stream_id, headers) + self.http.send_data(stream_id, body, end_stream=True) + self.transmit() + return stream_id, self.waiters[stream_id] + + + +class Proxy: + def __init__(self, args, root, name, http3=True): + self.args, self.root, self.name, self.http3 = args, root, name, http3 + self.process = None + self.log = None + + async def start(self): + self.log = (self.root / f"{self.name}.proxy.log").open("wb") + command = [self.args.mitmdump, "--mode", "reverse:https://lab.test:19443@127.0.0.1:18443", + "--set", f"confdir={self.root / 'proxy-config'}", "--set", "connection_strategy=lazy", + "--set", "keep_host_header=true", "--set", "store_streamed_bodies=false", + "--set", f"rea_lab_log={self.root / (self.name + '.producer.jsonl')}", + "-w", str(self.root / (self.name + '.mitm')), + "-s", str(Path(__file__).with_name("observe-mitmproxy-http3.py"))] + command.extend(["--set", f"ssl_verify_upstream_trusted_ca={self.root / 'origin-ca.pem'}"]) + if not self.http3: + command.extend(["--set", "http3=false"]) + self.process = subprocess.Popen(command, stdout=self.log, stderr=subprocess.STDOUT, + start_new_session=True) + lease = {"runId": os.environ["REA_PROCESS_RUN_ID"], "leaderPid": self.process.pid, + "processGroupId": self.process.pid} + with (self.root / "process-groups.jsonl").open("a") as handle: + handle.write(json.dumps(lease) + "\n") + deadline = time.monotonic() + 15 + journal = self.root / f"{self.name}.producer.jsonl" + while time.monotonic() < deadline: + if self.process.poll() is not None: + raise RuntimeError(f"Proxy exited: {self.log.name}") + if journal.exists() and '"event": "ready"' in journal.read_text(): + return self + await asyncio.sleep(0.05) + raise TimeoutError(f"Proxy readiness: {self.log.name}") + + async def stop(self): + if self.process is not None and self.process.poll() is None: + # The standalone mitmdump distribution has a bootloader parent. + # Signal its owned session together, including the interpreter child. + os.killpg(self.process.pid, signal.SIGINT) + try: + await asyncio.wait_for(asyncio.to_thread(self.process.wait), 10) + except TimeoutError: + os.killpg(self.process.pid, signal.SIGKILL) + await asyncio.to_thread(self.process.wait) + raise + if self.process is not None: + for _ in range(40): + if group_is_empty(self.process.pid): + return + await asyncio.sleep(0.05) + raise RuntimeError(f"Owned proxy group still live: {self.process.pid}") + + async def close_log(self): + if self.log is not None: + self.log.close() + + +def certificates(root): + for name in ["origin"]: + subprocess.run(["openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes", + "-keyout", str(root / f"{name}-key.pem"), "-out", str(root / f"{name}-ca.pem"), + "-days", "2", "-subj", "/CN=lab.test", "-addext", "subjectAltName=DNS:lab.test", + "-addext", "basicConstraints=critical,CA:TRUE"], check=True, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + + +def client_config(root): + config = QuicConfiguration(is_client=True, alpn_protocols=["h3"], supported_versions=[QuicProtocolVersion.VERSION_1], + server_name="lab.test", idle_timeout=3) + config.load_verify_locations(cafile=str(root / "proxy-config/mitmproxy-ca-cert.pem")) + return config + + +def connection(root, journal): + return connect("127.0.0.1", 18443, configuration=client_config(root), + create_protocol=lambda *a, **kw: Endpoint(*a, journal=journal, endpoint="client", **kw)) + + +async def curl_request(root, journal, name, version, path="/curl", maximum=8): + child = await asyncio.create_subprocess_exec("curl", version, "--silent", "--show-error", + "--max-time", str(maximum), "--cacert", str(root / "proxy-config/mitmproxy-ca-cert.pem"), + "--resolve", "lab.test:18443:127.0.0.1", "--noproxy", "*", "-H", f"x-lab-case: {name}", + "--output", str(root / (name + ".body")), "--dump-header", str(root / (name + ".headers")), + "--write-out", "%{json}", "https://lab.test:18443" + path, + stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE) + journal.children.append(child) + out, err = await child.communicate() + (root / (name + ".json")).write_bytes(out) + (root / (name + ".stderr")).write_bytes(err) + return child.returncode, json.loads(out) + + +async def tcp_origin(reader, writer, journal): + journal.writers.append(writer) + try: + head = await asyncio.wait_for(reader.readuntil(b"\r\n\r\n"), 5) + case = "tcp" + for line in head.split(b"\r\n")[1:]: + if line.lower().startswith(b"x-lab-case:"): + case = line.split(b":", 1)[1].strip().decode("ascii") + body = b"\x00\xfftcp-origin" + journal.emit("tcp-request", case=case, alpn=writer.get_extra_info("ssl_object").selected_alpn_protocol(), + headers=binary(head), body=binary(body)) + writer.write(b"HTTP/1.1 200 OK\r\ncontent-type: application/octet-stream\r\ncontent-length: " + + str(len(body)).encode() + b"\r\nconnection: close\r\n\r\n" + body) + await writer.drain() + finally: + writer.close() + await writer.wait_closed() + + +def group_is_empty(group): + for entry in Path("/proc").iterdir(): + if not entry.name.isdigit(): + continue + try: + fields = (entry / "stat").read_text().rsplit(")", 1)[1].split() + except FileNotFoundError: + continue + if fields[0] != "Z" and int(fields[2]) == group: + return False + return True + + +async def attempt_all(actions): + failures = [] + for name, action in actions: + try: + await asyncio.wait_for(action(), 12) + except BaseException as error: + failures.append((name, error)) + return failures + + +async def close_resource(resource): + resource.close() + + +async def finish_child(child): + if child.returncode is None: + child.terminate() + try: + await asyncio.wait_for(child.wait(), 3) + except TimeoutError: + child.kill() + await child.wait() + + +async def finish_tasks(journal): + for task in journal.tasks: + if not task.done(): + task.cancel() + results = await asyncio.gather(*journal.tasks, return_exceptions=True) + failures = [error for error in results if isinstance(error, BaseException) + and not isinstance(error, asyncio.CancelledError)] + if failures: + raise BaseExceptionGroup("Endpoint tasks failed", failures) + + +async def run(args): + if aioquic.__version__ != "1.2.0": + raise RuntimeError("This laboratory requires aioquic 1.2.0") + root = Path(args.output).resolve() + root.mkdir(mode=0o700) + os.umask(0o077) + journal = Journal(root) + loop = asyncio.get_running_loop() + asynchronous_errors = [] + def unexpected_exception(active_loop, context): + asynchronous_errors.append(context.get("message", "Unexpected asynchronous error")) + active_loop.default_exception_handler(context) + loop.set_exception_handler(unexpected_exception) + certificates(root) + server = tcp_server = None + proxies = [] + scenarios = [] + primary_error = None + cleanup_errors = [] + try: + config = QuicConfiguration(is_client=False, alpn_protocols=["h3"], + supported_versions=[QuicProtocolVersion.VERSION_1], idle_timeout=3) + config.load_cert_chain(str(root / "origin-ca.pem"), str(root / "origin-key.pem")) + server = await serve("127.0.0.1", 19443, configuration=config, + create_protocol=lambda *a, **kw: Endpoint(*a, journal=journal, endpoint="origin", **kw)) + tls_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + tls_context.load_cert_chain(str(root / "origin-ca.pem"), str(root / "origin-key.pem")) + tls_context.set_alpn_protocols(["http/1.1"]) + tcp_server = await asyncio.start_server(lambda r, w: journal.task(tcp_origin(r, w, journal)), + "127.0.0.1", 19443, ssl=tls_context) + proxy = Proxy(args, root, "normal") + proxies.append(proxy) + await proxy.start() + async with connection(root, journal) as client: + requests = [client.request(f"parallel-{i}", bytes([0, 255, i]) + bytes(range(256))) for i in range(3)] + await asyncio.wait_for(asyncio.gather(*(future for _, future in requests)), 10) + # cURL supplies an independently implemented HTTP/3 smoke case. + code, result = await curl_request(root, journal, "curl", "--http3-only") + if code != 0: + raise RuntimeError("HTTP/3 cURL failed; inspect curl.stderr") + if result["http_version"] != "3": + raise AssertionError("cURL silently downgraded") + await proxy.stop() + scenarios.append({"name": "normal", "capture": "normal.mitm"}) + + proxy = Proxy(args, root, "fallback", http3=False) + proxies.append(proxy) + await proxy.start() + code, result = await curl_request(root, journal, "fallback", "--http3", path="/fallback") + if code != 0 or result["http_version"] != "2": + raise AssertionError(f"Fallback control: {result}") + await proxy.stop() + scenarios.append({"name": "fallback", "capture": "fallback.mitm"}) + + except BaseException as error: + primary_error = error + finally: + actions = [] + for proxy in proxies: + actions.extend([(f"proxy-{proxy.name}", proxy.stop), + (f"proxy-log-{proxy.name}", proxy.close_log)]) + if server is not None: + actions.append(("quic-server", lambda: close_resource(server))) + if tcp_server is not None: + actions.extend([("tcp-listener-close", lambda: close_resource(tcp_server)), + ("tcp-listener-wait", tcp_server.wait_closed)]) + for index, writer in enumerate(journal.writers): + actions.extend([(f"tcp-writer-close-{index}", lambda w=writer: close_resource(w)), + (f"tcp-writer-wait-{index}", writer.wait_closed)]) + actions.append(("endpoint-tasks", lambda: finish_tasks(journal))) + for index, child in enumerate(journal.children): + actions.append((f"curl-child-{index}", lambda c=child: finish_child(c))) + cleanup_errors = await attempt_all(actions) + # This runs after every close attempt, including failed setup. + groups = [] + async def audit_groups(): + for proxy in proxies: + if proxy.process is not None: + groups.append({"group": proxy.process.pid, "empty": group_is_empty(proxy.process.pid)}) + if any(not item["empty"] for item in groups): + raise RuntimeError("Proxy groups remain live") + cleanup_errors.extend(await attempt_all([("process-groups", audit_groups)])) + try: + (root / "producer-cleanup.json").write_text(json.dumps({ + "primary_error": None if primary_error is None else str(primary_error), + "cleanup_errors": [{"action": name, "type": type(error).__name__, "message": str(error)} + for name, error in cleanup_errors], "groups": groups, + "asynchronous_errors": asynchronous_errors}, indent=2)) + except BaseException as error: + cleanup_errors.append(("cleanup-report", error)) + if primary_error is not None: + for name, error in cleanup_errors: + primary_error.add_note(f"Cleanup {name}: {error}") + raise primary_error + if cleanup_errors: + raise BaseExceptionGroup("HTTP/3 producer cleanup failed", [error for _, error in cleanup_errors]) + if asynchronous_errors: + raise RuntimeError(f"Unexpected asynchronous errors: {asynchronous_errors}") + (root / "manifest.json").write_text(json.dumps({"profile": "http3-minimal-v1", "aioquic": aioquic.__version__, + "scenarios": scenarios, "requests": journal.requests, "responses": journal.responses, + "events": journal.events, "limitations": [ + "Endpoint stream IDs are scoped observations, not mitmproxy serialized IDs or mappings.", + "The fixture reads negotiated _version from pinned aioquic 1.2.0.", + "Only QUIC v1, Linux reverse proxy and the selected endpoints are exercised."]}, indent=2)) + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--mitmdump", required=True) + parser.add_argument("--output", required=True) + asyncio.run(run(parser.parse_args())) diff --git a/scripts/fixtures/http3-lab.test.py b/scripts/fixtures/http3-lab.test.py new file mode 100644 index 000000000..3bc8c1b7e --- /dev/null +++ b/scripts/fixtures/http3-lab.test.py @@ -0,0 +1,56 @@ +"""Lifecycle regressions; these tests do not establish HTTP/3 provider support.""" +import importlib.util +import json +import tempfile +import unittest +from pathlib import Path +from types import SimpleNamespace +from unittest.mock import patch + +spec = importlib.util.spec_from_file_location("http3_lab", Path(__file__).with_name("http3-lab.py")) +lab = importlib.util.module_from_spec(spec) +spec.loader.exec_module(lab) + + +class CleanupTests(unittest.IsolatedAsyncioTestCase): + async def test_all_actions_run_after_multiple_close_failures(self): + attempted = [] + async def fail(): + attempted.append("failed-close") + raise RuntimeError("close failed") + async def audit(): + attempted.append("audit") + failures = await lab.attempt_all([("one", fail), ("two", fail), ("audit", audit)]) + self.assertEqual(attempted, ["failed-close", "failed-close", "audit"]) + self.assertEqual([name for name, _ in failures], ["one", "two"]) + + async def test_partial_setup_keeps_primary_error_and_audits_after_close_failure(self): + original_serve = lab.serve + closed = [] + class ClosingServer: + def __init__(self, server): + self.server = server + def close(self): + self.server.close() + closed.append(True) + raise RuntimeError("injected close failure") + async def serve(*args, **kwargs): + return ClosingServer(await original_serve(*args, **kwargs)) + async def fail_tcp(*args, **kwargs): + raise RuntimeError("injected primary setup failure") + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) / "capture" + with patch.object(lab, "serve", serve), patch.object(lab.asyncio, "start_server", fail_tcp): + with self.assertRaisesRegex(RuntimeError, "injected primary setup failure") as caught: + await lab.run(SimpleNamespace(output=str(root), mitmdump="unused")) + self.assertEqual(closed, [True]) + self.assertIn("injected close failure", " ".join(caught.exception.__notes__)) + report = json.loads((root / "producer-cleanup.json").read_text()) + self.assertEqual(report["primary_error"], "injected primary setup failure") + self.assertEqual(report["cleanup_errors"][0]["action"], "quic-server") + self.assertEqual(report["groups"], []) + self.assertFalse((root / "manifest.json").exists()) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/fixtures/observe-mitmproxy-http3.py b/scripts/fixtures/observe-mitmproxy-http3.py new file mode 100644 index 000000000..003025a75 --- /dev/null +++ b/scripts/fixtures/observe-mitmproxy-http3.py @@ -0,0 +1,64 @@ +"""Test-only observation of real flows; never synthesizes flow or stream IDs.""" + +import base64 +import hashlib +import json +from pathlib import Path + +from mitmproxy import ctx + + +def bytes_view(value): + if value is None: + return None + return {"base64": base64.b64encode(value).decode("ascii"), + "bytes": len(value), "sha256": hashlib.sha256(value).hexdigest()} + + +class Observer: + def load(self, loader): + loader.add_option("rea_lab_log", str, "", "HTTP/3 verifier observations") + + def emit(self, event, **data): + with Path(ctx.options.rea_lab_log).open("a", encoding="utf-8") as handle: + handle.write(json.dumps({"event": event, **data}) + "\n") + + def running(self): + self.emit("ready") + + def connection(self, conn): + return {"id": conn.id, "transport_protocol": conn.transport_protocol, + "alpn": bytes_view(conn.alpn), "tls_version": conn.tls_version, + "timestamp_tls_setup": conn.timestamp_tls_setup, + "error": conn.error} + + def tls_established_client(self, data): + self.emit("tls-established-client", connection=self.connection(data.conn)) + + def tls_established_server(self, data): + self.emit("tls-established-server", connection=self.connection(data.conn)) + + def flow(self, event, flow): + def message(message): + if message is None: + return None + return {"http_version": message.http_version, + "content": bytes_view(message.raw_content), + "timestamp_end": message.timestamp_end} + self.emit(event, id=flow.id, case=flow.request.headers.get("x-lab-case", "curl"), + request=message(flow.request), response=message(flow.response), + error=None if flow.error is None else flow.error.msg, + client_conn=self.connection(flow.client_conn), + server_conn=self.connection(flow.server_conn)) + + def request(self, flow): + self.flow("request", flow) + + def response(self, flow): + self.flow("response", flow) + + def error(self, flow): + self.flow("error", flow) + + +addons = [Observer()] diff --git a/scripts/lib/browser-runtime-fixture-lifecycle.mjs b/scripts/lib/browser-runtime-fixture-lifecycle.mjs index d39617f58..88a76e573 100644 --- a/scripts/lib/browser-runtime-fixture-lifecycle.mjs +++ b/scripts/lib/browser-runtime-fixture-lifecycle.mjs @@ -8,7 +8,10 @@ import { spawnOwnedProviderProcess } from "../../dist/process/ProviderProcess.js import { createSystemProcessOwnershipHost } from "../../dist/process/ProcessOwnershipObservation.js"; import { liveProcesses } from "../../dist/process/ProcessOwnershipProcessTree.js"; import { CdpConnection } from "../../dist/browser/CdpConnection.js"; -import { waitForBrowserDevtoolsPort } from "../../dist/browser/BrowserProcessStartup.js"; +import { + BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS, + waitForBrowserDevtoolsPort, +} from "../../dist/browser/BrowserProcessStartup.js"; /** Launch the synthetic fixture browser and await CDP in its own verified process group. */ export async function startRuntimeFixtureBrowser(executable, origin) { @@ -50,7 +53,7 @@ export async function startRuntimeFixtureBrowser(executable, origin) { stderr: () => stderr, // Match the broader real-browser verifier's allowance for cold Chrome // startup on shared runners; early process failures still reject at once. - timeoutMs: 60_000, + timeoutMs: BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS, }); return { port, diff --git a/scripts/lib/browser-socket-e2e.mjs b/scripts/lib/browser-socket-e2e.mjs new file mode 100644 index 000000000..6e54e3d18 --- /dev/null +++ b/scripts/lib/browser-socket-e2e.mjs @@ -0,0 +1,100 @@ +import assert from "node:assert/strict"; +import { readFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { + inspectWebPageInputSchema, + listBrowserTargetsInputSchema, +} from "../../dist/domain/browserObservation.js"; +import { compareWebCapturesInputSchema } from "../../dist/domain/webCaptureDiffSchemas.js"; +import { artifactCliEvidence } from "./artifact-e2e.mjs"; + +/** Verify browser-socket discovery and inspection against HTTP discovery. */ +export async function verifyBrowserSocketTransport({ + provider, + profile, + endpoint, + target, + origin, +}) { + const [port, path] = ( + await readFile(join(profile, "DevToolsActivePort"), "utf8") + ).split("\n"); + const socket = `ws://127.0.0.1:${port}${path}`; + const version = await (await fetch(`${endpoint}/json/version`)).json(); + assert.equal( + new URL(version.webSocketDebuggerUrl).pathname, + path, + "DevToolsActivePort and /json/version named different browser sockets", + ); + const list = async (cdpEndpoint) => { + const listed = await provider.listTargets( + listBrowserTargetsInputSchema.parse({ + cdp_endpoint: cdpEndpoint, + allowed_origins: [origin], + }), + ); + if (!listed.ok) throw listed.error; + return listed.value; + }; + const throughHttp = await list(endpoint); + const throughSocket = await list(socket); + // Target.getTargets reports attachment state that /json/list may omit. + const withoutAttachment = (targets) => + targets.map(({ attached: _attached, ...rest }) => rest); + assert.ok(throughSocket.targets.length > 0, "Browser socket listed no page"); + assert.deepEqual( + withoutAttachment(throughSocket.targets), + withoutAttachment(throughHttp.targets), + ); + assert.ok( + throughSocket.targets.every( + ({ attached }) => typeof attached === "boolean", + ), + "Browser socket omitted target attachment state", + ); + assert.equal(throughSocket.browser.product, throughHttp.browser.product); + + const evidence = await artifactCliEvidence("inspect-web-page", socket, [ + target, + "--allowed-origins", + origin, + "--observation-ms", + "100", + ]); + assert.equal(evidence.normalized_result.target.target_id, target); + assert.ok( + evidence.normalized_result.dom.nodes.length > 0, + "Browser-socket inspection returned no DOM nodes", + ); + assert.equal( + evidence.parameters.cdp_endpoint, + `ws://127.0.0.1:${port}/devtools/browser/[REDACTED]`, + ); + assert.ok( + !JSON.stringify(evidence).includes(path), + "Browser-socket Evidence retained the socket identifier", + ); + const throughHttpInspection = await provider.inspectPage( + inspectWebPageInputSchema.parse({ + cdp_endpoint: endpoint, + allowed_origins: [origin], + target_id: target, + observation_ms: 100, + }), + ); + if (!throughHttpInspection.ok) throw throughHttpInspection.error; + const compared = await provider.compareCaptures( + compareWebCapturesInputSchema.parse({ + before: { inspection: throughHttpInspection.value }, + after: { inspection: evidence.normalized_result }, + }), + ); + if (!compared.ok) throw compared.error; + assert.equal(compared.value.dimensions.dom_structure.status, "unchanged"); + assert.equal(compared.value.dimensions.scripts.status, "unchanged"); + return { + targets: throughSocket.targets.length, + revision: throughSocket.browser.revision, + }; +} diff --git a/scripts/lib/catalog-builders.mjs b/scripts/lib/catalog-builders.mjs index 32ec51ee3..d924ef49c 100644 --- a/scripts/lib/catalog-builders.mjs +++ b/scripts/lib/catalog-builders.mjs @@ -213,7 +213,9 @@ export const providerCatalog = (sources) => { }, { identity: sources.wasmProvider.WABT_PROVIDER_IDENTITY, - contracts: sources.wasmContracts.WASM_TOOL_CONTRACTS, + contracts: sources.wasmContracts.WASM_TOOL_CONTRACTS.filter( + ({ kind }) => kind === "artifact-provider", + ), }, { identity: sources.evmProvider.EVMOLE_PROVIDER_IDENTITY, diff --git a/scripts/lib/clean-linux-hopper-runtime.mjs b/scripts/lib/clean-linux-hopper-runtime.mjs index b43eca7fc..41ccfed6c 100644 --- a/scripts/lib/clean-linux-hopper-runtime.mjs +++ b/scripts/lib/clean-linux-hopper-runtime.mjs @@ -68,18 +68,19 @@ export const assertCleanLinuxHopperRuntime = async ( "utf8", ); const arguments_ = commandLine.split("\0"); - let kernelCommand; - if (commandLine.length === 0) { - // Zombies have neither arguments nor an executable link, but keep - // their kernel command name until their parent reaps them. - kernelCommand = ( - await readFile(`${procRoot}/${entry.name}/comm`, "utf8") - ).replace(/\n$/u, ""); - if (/^python3(?:\.\d+)?$/u.test(kernelCommand)) - throw new Error( - "Cannot identify a command-less Python process on the clean Linux runner", - ); - } + // The kernel command field identifies renamed runtime processes such + // as Xvfb. Keep it separate from arguments so ordinary script text does + // not count as a retained runtime. + const kernelCommand = ( + await readFile(`${procRoot}/${entry.name}/comm`, "utf8") + ).replace(/\n$/u, ""); + if ( + commandLine.length === 0 && + /^python3(?:\.\d+)?$/u.test(kernelCommand) + ) + throw new Error( + "Cannot identify a command-less Python process on the clean Linux runner", + ); retainedRuntime = kernelCommand === "Hopper" || kernelCommand === "Xvfb" || diff --git a/scripts/lib/docs-facts.mjs b/scripts/lib/docs-facts.mjs index 26d9a4ac3..dda249aaa 100644 --- a/scripts/lib/docs-facts.mjs +++ b/scripts/lib/docs-facts.mjs @@ -1,13 +1,41 @@ import { readFile, readdir, stat } from "node:fs/promises"; import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; -const readmeLinkTargets = (content, language = "en") => - [...content.matchAll(/\]\(([^)\s]+)\)/gu)] - .map((match) => match[1] ?? "") +const repoRelative = (root, from, target) => { + const index = target.search(/[#?]/u); + const pathPart = index === -1 ? target : target.slice(0, index); + const suffix = index === -1 ? "" : target.slice(index); + if (pathPart.length === 0 || /^[a-z][a-z0-9+.-]*:/iu.test(pathPart)) + return target; + const absolute = resolve(root, dirname(from), pathPart); + const withinRepo = relative(root, absolute); + if ( + withinRepo === ".." || + withinRepo.startsWith(`..${sep}`) || + isAbsolute(withinRepo) + ) + return target; + return `${withinRepo.split(sep).join("/")}${suffix}`; +}; + +const isLanguageNavigation = (target) => { + const path = target.split(/[#?]/u)[0]; + return ( + path === "README.md" || + /^docs\/readme\/README_[A-Za-z0-9-]+\.md$/u.test(path) + ); +}; + +const readmeDestinations = (content) => [ + ...content.matchAll(/\]\(([^)\s]+)\)/gu), + ...content.matchAll(/\b(?:src|href)="([^"]+)"/gu), +]; + +const readmeLinkTargets = (root, from, content, language = "en") => + readmeDestinations(content) + .map((match) => repoRelative(root, from, match[1] ?? "")) .filter( - (target) => - !target.startsWith("#") && - !/^README(?:_[A-Za-z0-9-]+)?\.md$/u.test(target), + (target) => !target.startsWith("#") && !isLanguageNavigation(target), ) .map((target) => { if (language === "en" || !URL.canParse(target)) return target; @@ -70,9 +98,19 @@ export const documentationFactIssues = async (root, catalog) => { const issues = await skillReferenceIssues( join(root, "skills/reverse-engineer-anything"), ); - const readmePaths = (await readdir(root)) - .filter((path) => /^README(?:_[A-Za-z0-9-]+)?\.md$/u.test(path)) - .sort(); + const translatedFolder = join(root, "docs/readme"); + let translatedNames = []; + try { + translatedNames = await readdir(translatedFolder); + } catch (cause) { + if (cause?.code !== "ENOENT") throw cause; + } + const readmePaths = [ + "README.md", + ...translatedNames + .filter((name) => /^README_[A-Za-z0-9-]+\.md$/u.test(name)) + .map((name) => `docs/readme/${name}`), + ].sort(); const readmes = new Map(); for (const path of readmePaths) { const content = await readFile(join(root, path), "utf8"); @@ -94,7 +132,11 @@ export const documentationFactIssues = async (root, catalog) => { ); } } - const canonicalLinks = readmeLinkTargets(readmes.get("README.md") ?? ""); + const canonicalLinks = readmeLinkTargets( + root, + "README.md", + readmes.get("README.md") ?? "", + ); const languageInventory = JSON.parse( await readFile(join(root, "website/locales/languages.json"), "utf8"), ); @@ -104,7 +146,12 @@ export const documentationFactIssues = async (root, catalog) => { for (const path of readmePaths.filter((path) => path !== "README.md")) if ( JSON.stringify( - readmeLinkTargets(readmes.get(path) ?? "", readmeLanguages.get(path)), + readmeLinkTargets( + root, + path, + readmes.get(path) ?? "", + readmeLanguages.get(path), + ), ) !== JSON.stringify(canonicalLinks) ) issues.push(`${path}: documentation links differ from README.md`); diff --git a/scripts/lib/http3-capture-assertions.mjs b/scripts/lib/http3-capture-assertions.mjs new file mode 100644 index 000000000..ca08d32cc --- /dev/null +++ b/scripts/lib/http3-capture-assertions.mjs @@ -0,0 +1,243 @@ +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { isUtf8 } from "node:buffer"; +const digest = (bytes) => createHash("sha256").update(bytes).digest("hex"); +// The native decoder displays strict UTF-8 and retains invalid bytes as null text. +const textOf = (expected) => { + const value = Buffer.from(expected.base64, "base64"); + return isUtf8(value) ? value.toString("utf8") : null; +}; +const field = (record, pointer) => + record.binary_fields.find((item) => item.pointer === pointer); +const bytes = (record, pointer) => { + const value = field(record, pointer); + assert.equal( + value?.state, + "retained", + `Missing retained bytes at ${pointer}`, + ); + const decoded = Buffer.from(value.content_base64, "base64"); + assert.equal(value.bytes, decoded.length); + assert.equal(value.sha256, digest(decoded)); + return decoded; +}; +const binaryEqual = (record, pointer, expected) => { + const decoded = bytes(record, pointer); + assert.deepEqual(decoded, Buffer.from(expected.base64, "base64"), pointer); + assert.equal(field(record, pointer).sha256, expected.sha256); + assert.equal(field(record, pointer).bytes, expected.bytes); +}; +const caseOf = (record) => + record.reported.request.headers.find( + ([name]) => name === "x-lab-case", + )?.[1] ?? "curl"; +const byCase = (value, name) => { + const selected = value.records.filter((record) => caseOf(record) === name); + assert.equal( + selected.length, + 1, + `Expected exactly one producer record for ${name}`, + ); + return selected[0]; +}; +const connectionH3 = (record, side) => { + const conn = record.reported[side]; + assert.equal(conn.transport_protocol, "udp"); + assert.equal(bytes(record, `/${side}/alpn`).toString(), "h3"); + assert.equal(conn.alpn, "h3", `${side}.alpn readable value`); + assert.equal(typeof conn.timestamp_tls_setup, "number"); +}; + +function assertHeaders(record, side, headers) { + const regular = headers.filter( + ([name]) => !Buffer.from(name.base64, "base64").toString().startsWith(":"), + ); + assert.equal(record.reported[side].headers.length, regular.length); + for (const [index, pair] of regular.entries()) { + binaryEqual(record, `/${side}/headers/${index}/0`, pair[0]); + binaryEqual(record, `/${side}/headers/${index}/1`, pair[1]); + assert.deepEqual( + record.reported[side].headers[index], + pair.map(textOf), + `${side}.headers[${index}] readable values`, + ); + } +} + +export function assertEndpointHeaders(actual, expected) { + // The proxy rebuilds pseudo fields from its HTTP state. Compare their values + // by name while retaining the exact order and duplicates of regular headers. + const partition = (headers) => { + const pseudo = new Map(); + const regular = []; + for (const pair of headers) { + const name = Buffer.from(pair[0].base64, "base64").toString(); + if (name.startsWith(":")) { + assert.equal(pseudo.has(name), false); + pseudo.set(name, pair); + } else regular.push(pair); + } + return { pseudo, regular }; + }; + assert.deepEqual(partition(actual), partition(expected)); +} + +export function requireHttp3(record) { + for (const side of ["request", "response"]) + assert.equal(record.reported[side].http_version, "HTTP/3"); + connectionH3(record, "client_conn"); + connectionH3(record, "server_conn"); +} + +export function verifyNormalRecords(normal, manifest, finalProducer) { + for (const record of normal.records) { + const name = caseOf(record); + const producer = finalProducer.find( + (event) => event.id === record.reported.id, + ); + assert.ok(producer, `No actual producer observation for ${name}`); + assert.equal(record.reported.error, null); + assert.equal(producer.error, null); + assert.equal(record.reported.response.status_code, 200); + requireHttp3(record); + for (const side of ["request", "response"]) { + assert.equal( + record.reported[side].http_version, + producer[side].http_version, + ); + binaryEqual(record, `/${side}/content`, producer[side].content); + assertHeaders( + record, + side, + manifest[side === "request" ? "requests" : "responses"][name].headers, + ); + } + // Native UUIDs must be retained as UUIDs; the producer exports no stream mapping. + for (const side of ["client_conn", "server_conn"]) + for (const key of ["id", "tls_version", "timestamp_tls_setup", "error"]) + assert.equal( + record.reported[side][key], + producer[side][key], + `${side}.${key} differs from producer observation ${producer.id}`, + ); + binaryEqual(record, "/request/content", manifest.requests[name].body); + binaryEqual(record, "/response/content", manifest.responses[name].body); + const pseudo = { + ":method": "method", + ":scheme": "scheme", + ":authority": "authority", + ":path": "path", + }; + for (const [headerName, headerValue] of manifest.requests[name].headers) { + const decodedName = Buffer.from(headerName.base64, "base64").toString(); + if (pseudo[decodedName]) { + binaryEqual(record, `/request/${pseudo[decodedName]}`, headerValue); + assert.equal( + record.reported.request[pseudo[decodedName]], + textOf(headerValue), + `request.${pseudo[decodedName]} readable value`, + ); + } + } + } +} + +export { bytes, caseOf, byCase }; + +export function verifyConcurrency(normal, manifest) { + const parallelNames = ["parallel-0", "parallel-1", "parallel-2"]; + const sent = manifest.events.filter( + (event) => + event.event === "request-sent" && parallelNames.includes(event.case), + ); + const received = manifest.events.filter( + (event) => + event.event === "request-received" && parallelNames.includes(event.case), + ); + const responses = manifest.events.filter( + (event) => + event.event === "response-sent" && parallelNames.includes(event.case), + ); + assert.equal(new Set(sent.map((event) => event.connection)).size, 1); + assert.equal(new Set(received.map((event) => event.connection)).size, 1); + assert.equal(new Set(sent.map((event) => event.stream_id)).size, 3); + assert.equal(new Set(received.map((event) => event.stream_id)).size, 3); + assert.ok( + Math.max(...received.map((event) => event.monotonic)) < + responses[0].monotonic, + ); + assert.deepEqual( + responses.map((event) => event.case), + received.map((event) => event.case).reverse(), + ); + for (const side of ["client_conn", "server_conn"]) + assert.equal( + new Set( + parallelNames.map((name) => byCase(normal, name).reported[side].id), + ).size, + 1, + ); + const normalConnections = new Set( + [...sent, ...received].map((event) => event.connection), + ); + for (const connection of normalConnections) { + const handshakes = manifest.events.filter( + (event) => event.event === "handshake" && event.connection === connection, + ); + assert.equal(handshakes.length, 1); + assert.equal(handshakes[0].alpn, "h3"); + assert.equal(handshakes[0].quic_version, 1); + } + assert.equal( + new Set(normal.records.map((record) => record.reported.id)).size, + 4, + ); + assert.equal(sent.length, 3); + const receivedByClient = manifest.events.filter( + (event) => + event.event === "response-received" && + sent.some( + (request) => + request.connection === event.connection && + request.stream_id === event.stream_id, + ), + ); + assert.equal(receivedByClient.length, 3); + for (const request of sent) { + const response = receivedByClient.find( + (event) => + event.connection === request.connection && + event.stream_id === request.stream_id, + ); + assert.ok( + response, + "Each sent request needs an independently observed client response", + ); + assert.deepEqual(response.body, manifest.responses[request.case].body); + assert.deepEqual(request.body, manifest.requests[request.case].body); + assertEndpointHeaders( + request.headers, + manifest.requests[request.case].headers, + ); + assertEndpointHeaders( + response.headers, + manifest.responses[request.case].headers, + ); + } + return { + sent: sent.map((event) => event.case), + completed_at_origin: received.map((event) => event.case), + dispatched_at_origin: responses.map((event) => event.case), + completed_at_client: receivedByClient.map( + (event) => + sent.find( + (request) => + request.connection === event.connection && + request.stream_id === event.stream_id, + ).case, + ), + saved_in_capture: normal.records + .filter((record) => parallelNames.includes(caseOf(record))) + .map(caseOf), + }; +} diff --git a/scripts/lib/http3-capture-assertions.test.mjs b/scripts/lib/http3-capture-assertions.test.mjs new file mode 100644 index 000000000..bc7e3c602 --- /dev/null +++ b/scripts/lib/http3-capture-assertions.test.mjs @@ -0,0 +1,163 @@ +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { test } from "node:test"; +import { verifyNormalRecords } from "./http3-capture-assertions.mjs"; + +function connectionFactsFixture() { + const body = Buffer.from([0, 255, 128, 1]); + const view = (bytes) => ({ + base64: bytes.toString("base64"), + bytes: bytes.length, + sha256: createHash("sha256").update(bytes).digest("hex"), + }); + const producer = { + id: "flow-id", + error: null, + request: { http_version: "HTTP/3", content: view(body) }, + response: { http_version: "HTTP/3", content: view(body) }, + client_conn: { + id: "client-id", + transport_protocol: "udp", + tls_version: "QUICv1", + timestamp_tls_setup: 1_700_000_000.125678, + error: null, + }, + server_conn: { + id: "server-id", + transport_protocol: "udp", + tls_version: "QUICv1", + timestamp_tls_setup: 1_700_000_000.987654, + error: null, + }, + }; + const reported = structuredClone(producer); + const requestFields = { + method: "POST", + scheme: "https", + authority: "lab.test:18443", + path: "/echo?case=readable", + }; + Object.assign(reported.request, requestFields); + const headers = [ + [Buffer.from("x-repeat"), Buffer.from("first-α")], + [Buffer.from("x-repeat"), Buffer.from("second-β")], + [Buffer.from("x-binary"), Buffer.from([255])], + ]; + for (const side of ["request", "response"]) + reported[side].headers = [ + ["x-repeat", "first-α"], + ["x-repeat", "second-β"], + ["x-binary", null], + ]; + reported.client_conn.alpn = "h3"; + reported.server_conn.alpn = "h3"; + reported.response.status_code = 200; + const binary_fields = [ + ["/request/content", body], + ["/response/content", body], + ["/client_conn/alpn", Buffer.from("h3")], + ["/server_conn/alpn", Buffer.from("h3")], + ...Object.entries(requestFields).map(([name, value]) => [ + `/request/${name}`, + Buffer.from(value), + ]), + ...["request", "response"].flatMap((side) => + headers.flatMap((pair, index) => + pair.map((value, part) => [`/${side}/headers/${index}/${part}`, value]), + ), + ), + ].map(([pointer, bytes]) => { + const { base64, ...facts } = view(bytes); + return { + pointer, + representation: "producer-bytes", + state: "retained", + content_base64: base64, + ...facts, + }; + }); + return { + normal: { records: [{ reported, binary_fields }] }, + manifest: { + requests: { + curl: { + headers: [ + ...Object.entries(requestFields).map(([name, value]) => [ + view(Buffer.from(`:${name}`)), + view(Buffer.from(value)), + ]), + ...headers.map((pair) => pair.map(view)), + ], + body: view(body), + }, + }, + responses: { + curl: { + headers: headers.map((pair) => pair.map(view)), + body: view(body), + }, + }, + }, + producer: [producer], + }; +} + +test("positive assertions retain exact producer connection facts on both legs", () => { + const { normal, manifest, producer } = connectionFactsFixture(); + verifyNormalRecords(normal, manifest, producer); +}); + +for (const [field, value] of [ + ["method", "GET"], + ["scheme", "http"], + ["authority", "other.test:18443"], + ["path", "/different"], +]) + test(`positive assertions reject changed readable request ${field}`, () => { + const { normal, manifest, producer } = connectionFactsFixture(); + normal.records[0].reported.request[field] = value; + assert.throws(() => verifyNormalRecords(normal, manifest, producer), { + code: "ERR_ASSERTION", + }); + }); + +for (const side of ["client_conn", "server_conn"]) + test(`positive assertions reject null readable ${side} ALPN`, () => { + const { normal, manifest, producer } = connectionFactsFixture(); + normal.records[0].reported[side].alpn = null; + assert.throws(() => verifyNormalRecords(normal, manifest, producer), { + code: "ERR_ASSERTION", + }); + }); + +for (const side of ["request", "response"]) + for (const [index, part, value] of [ + [0, 0, "x-changed"], + [0, 1, "changed-first"], + [1, 1, "changed-second"], + [2, 1, "�"], + ]) + test(`positive assertions reject changed readable ${side} header ${index}/${part}`, () => { + const { normal, manifest, producer } = connectionFactsFixture(); + normal.records[0].reported[side].headers[index][part] = value; + assert.throws(() => verifyNormalRecords(normal, manifest, producer), { + code: "ERR_ASSERTION", + }); + }); + +for (const side of ["client_conn", "server_conn"]) + for (const [field, value] of [ + ["tls_version", null], + ["tls_version", "changed-producer-label"], + ["error", "changed-connection-error"], + ["error", undefined], + ["timestamp_tls_setup", 0], + ["timestamp_tls_setup", 1_700_000_000], + ]) + test(`positive assertions reject ${side}.${field} changed to ${String(value)}`, () => { + const { normal, manifest, producer } = connectionFactsFixture(); + normal.records[0].reported[side][field] = value; + assert.throws(() => verifyNormalRecords(normal, manifest, producer), { + code: "ERR_ASSERTION", + }); + }); diff --git a/scripts/lib/http3-verifier-lifecycle.mjs b/scripts/lib/http3-verifier-lifecycle.mjs new file mode 100644 index 000000000..d083bf8d6 --- /dev/null +++ b/scripts/lib/http3-verifier-lifecycle.mjs @@ -0,0 +1,143 @@ +import assert from "node:assert/strict"; +import { mkdir, readFile, rename, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { DatabaseSync } from "node:sqlite"; +import { setTimeout } from "node:timers/promises"; +import { cleanupOwnedProcessGroup } from "../../dist/process/ProcessOwnership.js"; +import { observeOwnedProcessGroups } from "../../dist/process/ProcessOwnershipObservation.js"; + +export const describeError = (error) => ({ + name: error?.name ?? "Error", + message: error?.message ?? String(error), + ...(error?.stack ? { stack: error.stack } : {}), +}); + +export async function attemptAll(actions) { + const failures = []; + for (const [name, action] of actions) { + try { + await action(); + } catch (error) { + failures.push({ action: name, ...describeError(error) }); + } + } + return failures; +} + +async function latest(root, report) { + const temporary = join(root, `.verification-${report.run_id}.tmp`); + await writeFile(temporary, JSON.stringify(report, null, 2), { flag: "wx" }); + await rename(temporary, join(root, "verification-latest.json")); +} + +async function withLatestLock(root, action) { + // Keep this pathname stable: removing it could split concurrent lock owners. + // SQLite releases its OS-backed lock when the connection/process closes. + const database = new DatabaseSync(join(root, ".verification-lock.sqlite")); + try { + for (;;) { + try { + database.exec("BEGIN EXCLUSIVE"); + break; + } catch (error) { + if ( + error.code !== "ERR_SQLITE_ERROR" || + (error.errcode !== 5 && error.errcode !== 6) + ) + throw error; + // Yield while another process (or async operation here) owns the lock. + // A synchronous busy timeout would block that operation's fs callbacks. + await setTimeout(10); + } + } + return await action(); + } finally { + database.close(); + } +} + +export async function prepareAttempt(root, inspectExisting, run) { + if (!inspectExisting) await mkdir(root, { mode: 0o700 }); + const directory = join(root, "verifications", run.run_id); + await mkdir(directory, { recursive: true, mode: 0o700 }); + await withLatestLock(root, async () => { + // Preserve older reports as history, without leaving an ambiguous success + // at the capture root. Registration is the shared marker's ordering point. + try { + await rename( + join(root, "verification.json"), + join(directory, "legacy-verification.json"), + ); + } catch (error) { + if (error.code !== "ENOENT") throw error; + } + await latest(root, { + run_id: run.run_id, + status: "running", + started_at: new Date().toISOString(), + directory, + }); + }); + return { directory, captureRoot: join(root, "captures") }; +} + +export async function publishAttempt(root, directory, report) { + await writeFile( + join(directory, "result.json"), + JSON.stringify(report, null, 2), + { flag: "wx" }, + ); + if (report.status === "passed") + await writeFile( + join(directory, "verification.json"), + JSON.stringify(report, null, 2), + { flag: "wx" }, + ); + await withLatestLock(root, async () => { + const current = JSON.parse( + await readFile(join(root, "verification-latest.json"), "utf8"), + ); + if (current.run_id !== report.run_id) return; + // Ownership check and replacement share one lock with attempt registration. + // An older attempt still keeps its reports, but cannot replace this marker. + await latest(root, { + run_id: report.run_id, + status: report.status, + directory, + finished_at: report.finished_at, + }); + }); +} + +export async function finishProducerGroups(captureRoot, runId) { + let leases; + try { + leases = (await readFile(join(captureRoot, "process-groups.jsonl"), "utf8")) + .trim() + .split("\n") + .filter(Boolean) + .map(JSON.parse); + } catch (error) { + if (error.code === "ENOENT") return { groups: [], observation: [] }; + throw error; + } + for (const lease of leases) assert.equal(lease.runId, runId); + const failures = await attemptAll( + leases.map((lease) => [ + `producer-group-${lease.processGroupId}`, + async () => { + const result = await cleanupOwnedProcessGroup(lease); + assert.equal(result.cleaned, true, JSON.stringify(result)); + }, + ]), + ); + // Inspect even when one close failed; the groups can outlive their bootloader + // parent and disappear from a plain descendant-only observation. + const observation = [ + ...(await observeOwnedProcessGroups( + runId, + leases.map((lease) => lease.processGroupId), + )), + ]; + return { groups: leases, failures, observation }; +} diff --git a/scripts/lib/http3-verifier-lifecycle.test.mjs b/scripts/lib/http3-verifier-lifecycle.test.mjs new file mode 100644 index 000000000..fc7c3f5d2 --- /dev/null +++ b/scripts/lib/http3-verifier-lifecycle.test.mjs @@ -0,0 +1,433 @@ +import assert from "node:assert/strict"; +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { test } from "node:test"; +import { spawn } from "node:child_process"; +import { randomUUID } from "node:crypto"; +import { once } from "node:events"; +import { setTimeout } from "node:timers/promises"; +import { DatabaseSync } from "node:sqlite"; +import { + attemptAll, + prepareAttempt, + publishAttempt, + finishProducerGroups, +} from "./http3-verifier-lifecycle.mjs"; +import { cleanupOwnedProcessGroup } from "../../dist/process/ProcessOwnership.js"; +import { runOwnedCommand } from "../../dist/process/OwnedCommand.js"; +import { verifyConcurrency } from "./http3-capture-assertions.mjs"; + +async function waitForFile(path) { + const signal = AbortSignal.timeout(5_000); + for (;;) { + try { + await readFile(path); + return; + } catch (error) { + if (error.code !== "ENOENT") throw error; + } + await setTimeout(10, undefined, { signal }); + } +} + +function startAttemptWorker(root, report) { + const worker = ` + import { readFile, writeFile } from "node:fs/promises"; + import { join } from "node:path"; + import { setTimeout } from "node:timers/promises"; + import { prepareAttempt, publishAttempt } from ${JSON.stringify( + new URL("./http3-verifier-lifecycle.mjs", import.meta.url).href, + )}; + const [root, serialized] = process.argv.slice(1); + const report = JSON.parse(serialized); + const attempt = await prepareAttempt(root, true, report); + await writeFile(join(root, report.run_id + ".ready"), ""); + for (;;) { + try { await readFile(join(root, report.run_id + ".finish")); break; } + catch (error) { if (error.code !== "ENOENT") throw error; } + await setTimeout(10); + } + await publishAttempt(root, attempt.directory, report); + `; + return runOwnedCommand( + { + command: process.execPath, + arguments: [ + "--input-type=module", + "--eval", + worker, + root, + JSON.stringify(report), + ], + cwd: root, + hostEnvironment: process.env, + runId: randomUUID(), + }, + { timeoutMs: 10_000, diagnosticBytes: 4096 }, + ).then( + (result) => ({ result }), + (error) => ({ error }), + ); +} + +test("cleanup continues through failed closes and reaches its audit", async () => { + const attempted = []; + const failures = await attemptAll( + ["client", "transport", "audit"].map((name) => [ + name, + async () => { + attempted.push(name); + if (name !== "audit") throw new Error(`${name} failed`); + }, + ]), + ); + assert.deepEqual(attempted, ["client", "transport", "audit"]); + assert.deepEqual( + failures.map((failure) => failure.action), + ["client", "transport"], + ); +}); + +test("a failed reinspection cannot leave the earlier success as the current result", async () => { + const root = await mkdtemp(join(tmpdir(), "rea-http3-results-")); + try { + await writeFile( + join(root, "verification.json"), + '{"status":"passed","run_id":"legacy"}', + ); + const first = await prepareAttempt(root, true, { run_id: "first" }); + await publishAttempt(root, first.directory, { + run_id: "first", + status: "passed", + }); + const next = await prepareAttempt(root, true, { run_id: "next" }); + assert.equal( + JSON.parse(await readFile(join(root, "verification-latest.json"))).status, + "running", + ); + await publishAttempt(root, next.directory, { + run_id: "next", + status: "failed", + error: { message: "new failure" }, + }); + assert.equal( + JSON.parse(await readFile(join(root, "verification-latest.json"))).run_id, + "next", + ); + assert.equal( + JSON.parse(await readFile(join(root, "verification-latest.json"))).status, + "failed", + ); + await assert.rejects(readFile(join(next.directory, "verification.json")), { + code: "ENOENT", + }); + await assert.rejects(readFile(join(root, "verification.json")), { + code: "ENOENT", + }); + assert.equal( + JSON.parse(await readFile(join(first.directory, "verification.json"))) + .status, + "passed", + ); + assert.equal( + JSON.parse( + await readFile(join(first.directory, "legacy-verification.json")), + ).run_id, + "legacy", + ); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test("separate overlapping attempts finishing in reverse order retain the newer failure", async () => { + const root = await mkdtemp(join(tmpdir(), "rea-http3-overlap-")); + const commands = []; + const failed = { + run_id: "newer", + status: "failed", + error: { message: "newer inspection failed" }, + cleanup_failures: [{ action: "transport-close", message: "close failed" }], + }; + let primaryError; + const start = (report) => { + const command = startAttemptWorker(root, report); + commands.push(command); + return command; + }; + try { + const older = start({ run_id: "older", status: "passed" }); + await waitForFile(join(root, "older.ready")); + const newer = start(failed); + await waitForFile(join(root, "newer.ready")); + await writeFile(join(root, "newer.finish"), ""); + const newerResult = await newer; + if (newerResult.error) throw newerResult.error; + assert.equal(newerResult.result.exitCode, 0); + assert.equal(newerResult.result.signal, null); + const marker = await readFile(join(root, "verification-latest.json")); + await writeFile(join(root, "older.finish"), ""); + const olderResult = await older; + if (olderResult.error) throw olderResult.error; + assert.equal(olderResult.result.exitCode, 0); + assert.equal(olderResult.result.signal, null); + assert.deepEqual( + await readFile(join(root, "verification-latest.json")), + marker, + ); + assert.deepEqual( + JSON.parse(await readFile(join(root, "verifications/newer/result.json"))), + failed, + ); + await assert.rejects( + readFile(join(root, "verifications/newer/verification.json")), + { + code: "ENOENT", + }, + ); + assert.equal( + JSON.parse( + await readFile(join(root, "verifications/older/verification.json")), + ).status, + "passed", + ); + } catch (error) { + primaryError = error; + } finally { + // Release both barriers even after an assertion/setup failure. Each command + // retains native supervision through termination before the root is removed. + const failures = await attemptAll( + ["older", "newer"].map((name) => [ + name, + () => writeFile(join(root, name + ".finish"), ""), + ]), + ); + const outcomes = await Promise.all(commands); + const retained = outcomes.filter( + ({ error }) => error?.cleanupFailure || error?.cleanupResult, + ); + if (retained.length) + primaryError = new AggregateError( + [primaryError, ...retained.map(({ error }) => error)].filter(Boolean), + `Worker cleanup failed; retained test evidence at ${root}`, + { cause: primaryError }, + ); + else + failures.push( + ...(await attemptAll([ + ["root", () => rm(root, { recursive: true, force: true })], + ])), + ); + if (failures.length) + primaryError = new AggregateError( + [primaryError, ...failures].filter(Boolean), + "Test cleanup failed", + { cause: primaryError }, + ); + } + if (primaryError) throw primaryError; +}); + +test("publication waits for the lock before reading the current owner", async () => { + const root = await mkdtemp(join(tmpdir(), "rea-http3-lock-")); + let database; + let publication; + let primaryError; + try { + const older = await prepareAttempt(root, true, { run_id: "older" }); + database = new DatabaseSync(join(root, ".verification-lock.sqlite")); + database.exec("BEGIN EXCLUSIVE"); + publication = publishAttempt(root, older.directory, { + run_id: "older", + status: "passed", + }); + await waitForFile(join(older.directory, "verification.json")); + const marker = join(root, "verification-latest.json"); + assert.equal(JSON.parse(await readFile(marker)).status, "running"); + // This connection represents another marker writer. Its replacement must + // be observed only after the blocked publisher acquires the same lock. + const newer = { run_id: "newer", status: "failed" }; + await writeFile(marker, JSON.stringify(newer)); + database.close(); + database = undefined; + await publication; + assert.deepEqual(JSON.parse(await readFile(marker)), newer); + } catch (error) { + primaryError = error; + } finally { + const failures = await attemptAll([ + ["lock", () => database?.close()], + [ + "publication", + async () => { + if (publication) await publication; + }, + ], + ["root", () => rm(root, { recursive: true, force: true })], + ]); + if (failures.length) + primaryError = new AggregateError( + [primaryError, ...failures].filter(Boolean), + "Test cleanup failed", + { cause: primaryError }, + ); + } + if (primaryError) throw primaryError; +}); + +test("a failed marker read releases the lock and preserves the attempt report", async () => { + const root = await mkdtemp(join(tmpdir(), "rea-http3-lock-failure-")); + try { + const older = await prepareAttempt(root, true, { run_id: "older" }); + await writeFile(join(root, "verification-latest.json"), "{"); + const report = { + run_id: "older", + status: "failed", + error: { message: "inspection failed" }, + }; + await assert.rejects( + publishAttempt(root, older.directory, report), + SyntaxError, + ); + assert.deepEqual( + JSON.parse(await readFile(join(older.directory, "result.json"))), + report, + ); + const next = await prepareAttempt(root, true, { run_id: "next" }); + await publishAttempt(root, next.directory, { + run_id: "next", + status: "passed", + }); + assert.equal( + JSON.parse(await readFile(join(root, "verification-latest.json"))).run_id, + "next", + ); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test("dispatch reverses origin completion; client and capture may arrive in other orders", () => { + const names = ["parallel-0", "parallel-1", "parallel-2"]; + const header = (text) => ({ base64: Buffer.from(text).toString("base64") }); + const messages = Object.fromEntries( + names.map((name) => [ + name, + { + headers: [[header("x-lab-case"), header(name)]], + body: { base64: header(name).base64 }, + }, + ]), + ); + const event = (kind, index, connection, monotonic) => ({ + event: kind, + case: names[index], + stream_id: index * 4, + connection, + monotonic, + ...messages[names[index]], + }); + const manifest = { + requests: messages, + responses: messages, + events: [ + ...[0, 1, 2].map((i) => event("request-sent", i, "client", i)), + ...[1, 2, 0].map((i) => event("request-received", i, "origin", 10)), + ...[0, 2, 1].map((i) => event("response-sent", i, "origin", 20)), + ...[2, 0, 1].map((i) => event("response-received", i, "client", 30)), + ...["client", "origin"].map((connection) => ({ + event: "handshake", + connection, + alpn: "h3", + quic_version: 1, + })), + ], + }; + const record = (name) => ({ + reported: { + id: name, + request: { headers: [["x-lab-case", name]] }, + client_conn: { id: "client-uuid" }, + server_conn: { id: "origin-uuid" }, + }, + }); + const normal = { + records: [...[1, 0, 2].map((i) => record(names[i])), record("curl")], + }; + const orders = verifyConcurrency(normal, manifest); + assert.deepEqual(orders.dispatched_at_origin, [ + "parallel-0", + "parallel-2", + "parallel-1", + ]); + manifest.events = manifest.events.filter( + (item) => item.event !== "response-sent", + ); + manifest.events.push( + ...[2, 1, 0].map((i) => event("response-sent", i, "origin", 20)), + ); + assert.throws(() => verifyConcurrency(normal, manifest), { + code: "ERR_ASSERTION", + }); +}); + +test("cleanup verifies and retires a separately owned producer group", async () => { + const root = await mkdtemp(join(tmpdir(), "rea-http3-group-")); + const runId = randomUUID(); + const child = spawn("/usr/bin/sleep", ["60"], { + detached: true, + env: { ...process.env, REA_PROCESS_RUN_ID: runId }, + stdio: "ignore", + }); + let primaryError; + let cleaned = false; + try { + await once(child, "spawn"); + await writeFile( + join(root, "process-groups.jsonl"), + JSON.stringify({ + runId, + leaderPid: child.pid, + processGroupId: child.pid, + }) + "\n", + ); + const result = await finishProducerGroups(root, runId); + assert.deepEqual(result.failures, []); + assert.equal(result.observation[0][1].state, "empty"); + } catch (error) { + primaryError = error; + } finally { + const failures = await attemptAll([ + [ + "process", + async () => { + const result = await cleanupOwnedProcessGroup({ + runId, + leaderPid: child.pid, + processGroupId: child.pid, + }); + assert.equal( + result.cleaned, + true, + `Retained ownership evidence at ${root}: ${JSON.stringify(result)}`, + ); + cleaned = true; + }, + ], + [ + "root", + async () => { + if (cleaned) await rm(root, { recursive: true, force: true }); + }, + ], + ]); + if (failures.length) + primaryError = new AggregateError( + [primaryError, ...failures].filter(Boolean), + "Test cleanup failed", + { cause: primaryError }, + ); + } + if (primaryError) throw primaryError; +}); diff --git a/scripts/rea-android-workspace-guardian.mjs b/scripts/rea-android-workspace-guardian.mjs new file mode 100644 index 000000000..93ba42d2d --- /dev/null +++ b/scripts/rea-android-workspace-guardian.mjs @@ -0,0 +1,46 @@ +#!/usr/bin/env node + +import { setTimeout as delay } from "node:timers/promises"; +import { WindowsPrivateRuntime } from "../dist/windows/WindowsPrivateRuntime.js"; + +if (process.platform !== "win32" || process.argv.length !== 3) + throw new Error( + "Android workspace guardian requires Windows and one parent directory", + ); + +// This process owns the native directory handle. Detached process creation +// keeps it alive when the REA owner is terminated without running JS cleanup. +const runtime = WindowsPrivateRuntime.create(process.argv[2], "rea-android-"); +const ready = await new Promise((resolve) => { + process.stdout.once("error", () => resolve(false)); + process.stdout.write( + JSON.stringify({ + path: runtime.observation.path, + volumeSerial: runtime.observation.volumeSerial, + fileId: runtime.observation.fileId, + privateDacl: runtime.observation.privateDacl, + }) + "\n", + (error) => resolve(error == null), + ); +}); + +// The owner holds the write end until close/death. A failed readiness pipe +// also grants cleanup authority, including owner death during startup. +if (ready) { + for await (const _ of process.stdin) { + // No commands are accepted; only EOF grants cleanup authority. + } +} else { + process.stdin.destroy(); +} + +// A failed native walk retains the directory identity lease for a later retry. +// Keep this process alive so a transient sharing violation cannot orphan it. +for (;;) { + try { + await runtime.close(); + process.exit(0); + } catch { + await delay(1000); + } +} diff --git a/scripts/test-website.py b/scripts/test-website.py index 5c1aef876..ba87c9f15 100644 --- a/scripts/test-website.py +++ b/scripts/test-website.py @@ -261,7 +261,9 @@ class WebsiteTranslationChecks(unittest.TestCase): inventory = [item for item in languages(ROOT) if item["code"] in {"en", "zh-CN", "ar"}] (root / "website/locales/languages.json").write_text(json.dumps(inventory)) for item in inventory: - (root / item["readme"]).write_text("# REA\n") + destination = root / item["readme"] + destination.parent.mkdir(parents=True, exist_ok=True) + destination.write_text("# REA\n") WEBSITE.sync_pages(root) extract_catalogs(root) yield root diff --git a/scripts/verify-agent-experience.mjs b/scripts/verify-agent-experience.mjs index 99ab060dd..71cd6383c 100644 --- a/scripts/verify-agent-experience.mjs +++ b/scripts/verify-agent-experience.mjs @@ -1,4 +1,5 @@ import { spawn } from "node:child_process"; +import { createHash } from "node:crypto"; import { chmod, copyFile, @@ -259,7 +260,7 @@ try { const execution = await runCodex( answerInstructions.length === 0 ? scenario.prompt - : `${scenario.prompt}\n\nThis closed factual rubric needs complete producer Evidence in the transcript. Request complete detail on each initial application analysis when supported; summary or view records alone are not accepted by this rubric.\n\n${answerInstructions}`, + : `${scenario.prompt}\n\nThis closed factual rubric needs complete producer Evidence in the transcript. Request complete detail on each initial inspection or analysis when supported; summary or view records alone are not accepted by this rubric.\n\n${answerInstructions}`, scenario.approvedTools ?? [], ); const transcriptDirectory = process.env.REA_AGENT_EVAL_TRANSCRIPT_DIR; @@ -440,6 +441,13 @@ async function createTargets(root, includeManaged) { ...fixtures, largeJavascript, managed: join(managedOutput, "AgentEval.dll"), + ...(includeManaged + ? { + managedSha256: createHash("sha256") + .update(await readFile(join(managedOutput, "AgentEval.dll"))) + .digest("hex"), + } + : {}), }; } diff --git a/scripts/verify-evm-interface.mjs b/scripts/verify-evm-interface.mjs index b0bfdafa4..4bacc4bdf 100644 --- a/scripts/verify-evm-interface.mjs +++ b/scripts/verify-evm-interface.mjs @@ -225,6 +225,26 @@ try { for (const mode of ["cli", "mcp"]) { const report = await inspect(mode, path, encoding); assert.equal(report.bytecode.hex, hex); + assert.equal(report.bytecode.metadata.status, "decoded"); + assert.deepEqual(report.bytecode.metadata.compiler_version, { + representation: "release-bytes", + value: "0.8.30", + }); + assert.equal( + report.bytecode.metadata.bytecode_offset + + report.bytecode.metadata.cbor_bytes + + 2, + Buffer.from(hex, "hex").length, + ); + assert.ok( + report.bytecode.metadata.entries.some( + ({ key, value }) => + key.kind === "text" && + key.value === "solc" && + value.kind === "bytes" && + value.value === "00081e", + ), + ); assert.deepEqual( report.functions .filter(({ dispatch }) => dispatch === "abi") diff --git a/scripts/verify-flutter.mjs b/scripts/verify-flutter.mjs index 061ad8b20..76b5a1fda 100644 --- a/scripts/verify-flutter.mjs +++ b/scripts/verify-flutter.mjs @@ -1,12 +1,12 @@ import assert from "node:assert/strict"; import { execFile } from "node:child_process"; -import { readdir, readFile } from "node:fs/promises"; -import { resolve } from "node:path"; +import { mkdtemp, readdir, readFile, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; import { promisify } from "node:util"; import { Client } from "@modelcontextprotocol/client"; import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; import { toolContract } from "../dist/contracts/toolContracts.js"; -import { PrivateRuntimeRoot } from "../dist/process/PrivateRuntimeRoot.js"; import { hashStableFile } from "../dist/filesystem/StableFileHash.js"; import { mcpTextValue } from "./lib/mcp-verifier-results.mjs"; @@ -27,17 +27,16 @@ const expectations = options.expectations === null ? null : JSON.parse(await readFile(options.expectations, "utf8")); -const root = await PrivateRuntimeRoot.create({ - prefix: "rea-flutter-verifier-", -}); +// This is a shared test parent, not a leased provider-owned Windows runtime. +const root = await mkdtemp(join(tmpdir(), "rea-flutter-verifier-")); const entrypoint = resolve("scripts/rea.mjs"); const environment = Object.fromEntries( Object.entries(process.env).filter(([, value]) => typeof value === "string"), ); Object.assign(environment, { - TMPDIR: root.path, - TMP: root.path, - TEMP: root.path, + TMPDIR: root, + TMP: root, + TEMP: root, }); const client = new Client({ name: "flutter-verifier", version: "1" }); const transport = new StdioClientTransport({ @@ -190,15 +189,18 @@ try { } finally { // Server shutdown retries retained provider cleanup and must settle before success. try { - await client.close(); - await transport.close(); + try { + await client.close(); + } finally { + await transport.close(); + } assert.deepEqual( - await readdir(root.path), + await readdir(root), [], - "Provider snapshots must be released before verification succeeds", + `Provider snapshots remain under ${root}`, ); } finally { - await root.close(); + await rm(root, { recursive: true, force: true }); } } process.stdout.write( diff --git a/scripts/verify-http3-network-captures.mjs b/scripts/verify-http3-network-captures.mjs new file mode 100644 index 000000000..2b3493821 --- /dev/null +++ b/scripts/verify-http3-network-captures.mjs @@ -0,0 +1,408 @@ +#!/usr/bin/env node +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { readFile, writeFile } from "node:fs/promises"; +import { isAbsolute, join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { Client } from "@modelcontextprotocol/client"; +import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; +import { Ajv2020 } from "ajv/dist/2020.js"; +import { runOwnedCommand } from "../dist/process/OwnedCommand.js"; +import { toolContract } from "../dist/contracts/toolContracts.js"; +import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs"; +import { + bytes, + byCase, + verifyNormalRecords, + requireHttp3, + verifyConcurrency, +} from "./lib/http3-capture-assertions.mjs"; +import { + attemptAll, + describeError, + prepareAttempt, + publishAttempt, + finishProducerGroups, +} from "./lib/http3-verifier-lifecycle.mjs"; +import { mcpTextValue } from "./lib/mcp-verifier-results.mjs"; + +const root = process.argv[2]; +const inspectExisting = process.argv[3] === "--inspect-existing"; +const mitmdump = process.env.REA_MITMDUMP_COMMAND; +const python = process.env.REA_HTTP3_PYTHON_COMMAND; +assert.equal( + process.platform, + "linux", + "HTTP/3 native verification requires Linux", +); +assert.ok( + root && isAbsolute(root), + "Select an absolute, new evidence directory", +); +assert.ok( + mitmdump && isAbsolute(mitmdump), + "Select caller-supplied mitmdump 12.2.3", +); +if (!inspectExisting) + assert.ok( + python && isAbsolute(python), + "Select a Python environment with aioquic 1.2.0", + ); +const run = createVerifierRun(); +const cwd = fileURLToPath(new URL("..", import.meta.url)); +const entrypoint = fileURLToPath(new URL("./rea.mjs", import.meta.url)); +const environment = Object.fromEntries( + Object.entries(process.env).filter(([, value]) => typeof value === "string"), +); +const client = new Client({ + name: "real-http3-capture-verifier", + version: "1", +}); +const transport = new StdioClientTransport({ + command: process.execPath, + args: [entrypoint, "mcp"], + env: environment, + stderr: "pipe", +}); +const contract = toolContract("inspect_web_network_capture"); +const maxMcpStderrBytes = 64 * 1024; +let mcpStderr = Buffer.alloc(0); +transport.stderr?.on("data", (chunk) => { + mcpStderr = Buffer.concat([mcpStderr, chunk]); + if (mcpStderr.length > maxMcpStderrBytes) + mcpStderr = mcpStderr.subarray(-maxMcpStderrBytes); +}); +const checks = []; +const retainedEvidence = []; +const originals = new Map(); +let directory; +let captureRoot = join(root, "captures"); +let primaryError; +let verifier; +let producerAudit; +let cleanupFailures = []; +let validateOutput; + +const digest = (bytes) => createHash("sha256").update(bytes).digest("hex"); +const readJson = async (path) => JSON.parse(await readFile(path, "utf8")); +const readLines = async (path) => + (await readFile(path, "utf8")) + .trim() + .split("\n") + .filter(Boolean) + .map(JSON.parse); + +async function cli(input, name) { + const command = await runOwnedCommand( + { + command: process.execPath, + arguments: [ + entrypoint, + "inspect-web-network-capture", + input.capture_path, + input.format, + "--json", + ], + cwd, + hostEnvironment: environment, + runId: run.run_id, + }, + { + timeoutMs: 40_000, + diagnosticBytes: 8 * 1024 * 1024, + }, + ); + assert.notEqual(command.diagnosticTruncated, true); + await writeFile( + join(directory, `${name}.cli.stdout.json`), + command.stdout.text, + ); + await writeFile( + join(directory, `${name}.cli.stderr.log`), + command.stderr.text, + ); + const value = JSON.parse(command.stdout.text); + return contract.outputSchema.parse(value); +} + +async function mcp(input, name) { + const reply = await client.callTool({ + name: contract.name, + arguments: input, + }); + await writeFile( + join(directory, `${name}.mcp.reply.json`), + JSON.stringify(reply, null, 2), + ); + const value = JSON.parse(mcpTextValue(reply)); + assert.notEqual(reply.isError, true, mcpTextValue(reply)); + assert.ok( + validateOutput(reply.structuredContent), + JSON.stringify(validateOutput.errors), + ); + const evidence = contract.outputSchema.parse(value); + assert.deepEqual(reply.structuredContent, value); + retainedEvidence.push(evidence); + return evidence; +} + +async function inspect(name) { + const input = { + capture_path: join(captureRoot, `${name}.mitm`), + format: "mitmproxy", + }; + const cliResult = await cli(input, name); + const mcpResult = await mcp(input, name); + assert.deepEqual(cliResult.normalized_result, mcpResult.normalized_result); + for (const evidence of [cliResult, mcpResult]) { + assert.equal(evidence.authority, "historical-reference"); + assert.equal(evidence.confidence, "observed"); + assert.equal(evidence.provider.version, "12.2.3"); + assert.equal(evidence.normalized_result.runtime_attribution, "unknown"); + assert.deepEqual(evidence.provider, evidence.normalized_result.decoder); + assert.equal( + evidence.subject.digest.sha256, + digest(await readFile(input.capture_path)), + ); + } + checks.push({ + case: name, + outcome: "passed", + records: mcpResult.normalized_result.records.length, + }); + return mcpResult.normalized_result; +} + +try { + ({ directory, captureRoot } = await prepareAttempt( + root, + inspectExisting, + run, + )); + if (!inspectExisting) { + const produced = await runOwnedCommand( + { + command: python, + arguments: [ + fileURLToPath(new URL("./fixtures/http3-lab.py", import.meta.url)), + "--mitmdump", + mitmdump, + "--output", + captureRoot, + ], + cwd, + hostEnvironment: environment, + runId: run.run_id, + }, + { timeoutMs: 120_000, diagnosticBytes: 1024 * 1024 }, + ); + await writeFile( + join(directory, "producer.stdout.log"), + produced.stdout.text, + ); + await writeFile( + join(directory, "producer.stderr.log"), + produced.stderr.text, + ); + } + const manifest = await readJson(join(captureRoot, "manifest.json")); + assert.equal(manifest.aioquic, "1.2.0"); + assert.equal(manifest.profile, "http3-minimal-v1"); + await client.connect(transport); + const advertised = (await client.listTools()).tools.find( + (tool) => tool.name === contract.name, + ); + assert.ok(advertised?.outputSchema, "Missing named advertised schema"); + const ajv = new Ajv2020({ strict: false, validateFormats: false }); + assert.ok(ajv.validateSchema(advertised.inputSchema)); + assert.ok(ajv.validateSchema(advertised.outputSchema)); + validateOutput = ajv.compile(advertised.outputSchema); + const inputValidator = ajv.compile(advertised.inputSchema); + assert.ok( + inputValidator({ + capture_path: join(captureRoot, "normal.mitm"), + format: "mitmproxy", + }), + ); + assert.equal( + inputValidator({ + capture_path: join(captureRoot, "normal.mitm"), + format: "mitmproxy", + stream_id: 0, + }), + false, + ); + await writeFile( + join(directory, "advertised-tool.json"), + JSON.stringify(advertised, null, 2), + ); + + for (const name of ["normal", "fallback"]) + originals.set(name, await readFile(join(captureRoot, `${name}.mitm`))); + const normal = await inspect("normal"); + assert.equal(normal.total_records, 4); + const finalProducer = ( + await readLines(join(captureRoot, "normal.producer.jsonl")) + ).filter((event) => ["response", "error"].includes(event.event)); + verifyNormalRecords(normal, manifest, finalProducer); + const orders = verifyConcurrency(normal, manifest); + assert.equal( + (await readJson(join(captureRoot, "curl.json"))).http_version, + "3", + ); + assert.deepEqual( + await readFile(join(captureRoot, "curl.body")), + bytes(byCase(normal, "curl"), "/response/content"), + ); + checks.push({ + case: "concurrency-bytes", + outcome: "passed", + parallel_streams: 3, + orders, + }); + const fallback = await inspect("fallback"); + assert.equal(fallback.total_records, 1); + const record = byCase(fallback, "fallback"); + assert.throws( + () => requireHttp3(record), + { code: "ERR_ASSERTION" }, + "The same positive HTTP/3 check must reject the fallback record", + ); + assert.equal(record.reported.client_conn.transport_protocol, "tcp"); + assert.equal(record.reported.server_conn.transport_protocol, "tcp"); + assert.equal(record.reported.request.http_version, "HTTP/2.0"); + assert.equal(record.reported.response.http_version, "HTTP/1.1"); + assert.equal(record.reported.server_conn.alpn, "http/1.1"); + assert.equal( + (await readJson(join(captureRoot, "fallback.json"))).http_version, + "2", + ); + checks.push({ + case: "fallback-detected", + outcome: "passed", + client_http: "2", + origin_http: "1.1", + }); + + const bundleReply = await client.callTool({ + name: "get_evidence_bundle", + arguments: {}, + }); + assert.notEqual(bundleReply.isError, true, mcpTextValue(bundleReply)); + const bundle = JSON.parse(mcpTextValue(bundleReply)).result; + const records = bundle.records; + assert.ok(Array.isArray(records), "Evidence bundle omitted its records"); + for (const evidence of retainedEvidence) { + const retained = records.find( + (record) => record.evidence_id === evidence.evidence_id, + ); + assert.deepEqual( + retained, + evidence, + `MCP must retain the complete Evidence for ${evidence.evidence_id}`, + ); + } + await writeFile( + join(directory, "session-bundle.reply.json"), + JSON.stringify(bundleReply, null, 2), + ); + checks.push({ + case: "mcp-session-retention", + outcome: "passed", + records: retainedEvidence.length, + }); +} catch (error) { + primaryError = error; +} finally { + cleanupFailures = await attemptAll([ + ["client-close", () => client.close()], + ["transport-close", () => transport.close()], + [ + "producer-groups", + async () => { + if (!inspectExisting) { + producerAudit = await finishProducerGroups(captureRoot, run.run_id); + assert.deepEqual(producerAudit.failures ?? [], []); + for (const [, observation] of producerAudit.observation) + assert.equal( + observation.state, + "empty", + JSON.stringify(observation), + ); + } + }, + ], + ...[...originals].map(([name, original]) => [ + `input-immutability-${name}`, + async () => { + assert.deepEqual( + await readFile(join(captureRoot, `${name}.mitm`)), + original, + ); + }, + ]), + [ + "mcp-stderr-diagnostics", + async () => { + if (directory) + await writeFile(join(directory, "mcp.stderr.log"), mcpStderr); + }, + ], + [ + "failed-command-diagnostics", + async () => { + if (directory && primaryError?.snapshot) { + await writeFile( + join(directory, "failed-command.stdout.log"), + primaryError.snapshot.stdout.text, + ); + await writeFile( + join(directory, "failed-command.stderr.log"), + primaryError.snapshot.stderr.text, + ); + } + }, + ], + [ + "final-lineage", + async () => { + verifier = await completeVerifierRun(run); + assert.equal(verifier.process_lineage.status, "verified"); + assert.deepEqual(verifier.process_lineage.descendants, []); + }, + ], + ]); +} +const report = { + run_id: run.run_id, + status: primaryError || cleanupFailures.length ? "failed" : "passed", + finished_at: new Date().toISOString(), + ...(primaryError + ? { + error: describeError(primaryError), + mcp_stderr_log: "mcp.stderr.log", + } + : {}), + cleanup_failures: cleanupFailures, + checks, + verifier, + producer_audit: producerAudit, + scope: + "Real Linux reverse HTTP/3, mitmproxy 12.2.3, aioquic 1.2.0 and guest cURL", + limitations: [ + "No local/WireGuard, browser, QUIC v2, migration, 0-RTT, mTLS or replay claim.", + "No HAR, interruption, certificate-failure or body-completeness coverage in this profile.", + ], +}; +if (directory) { + try { + await publishAttempt(root, directory, report); + } catch (error) { + console.error("Could not publish this attempt's result:", error); + primaryError ??= error; + } +} +if (primaryError) throw primaryError; +if (cleanupFailures.length) + throw new AggregateError(cleanupFailures, "HTTP/3 verifier cleanup failed"); +console.log(JSON.stringify(report, null, 2)); diff --git a/scripts/verify-package-environment.mjs b/scripts/verify-package-environment.mjs index cd84655b2..e57f7784b 100644 --- a/scripts/verify-package-environment.mjs +++ b/scripts/verify-package-environment.mjs @@ -76,6 +76,7 @@ export async function verifyPackageEnvironment({ HOME: home, USERPROFILE: home, APPDATA: join(home, "AppData", "Roaming"), + LOCALAPPDATA: join(home, "AppData", "Local"), CLAUDE_CONFIG_DIR: home, CODEX_HOME: codexDir, COPILOT_HOME: join(home, ".copilot"), diff --git a/scripts/verify-package-update.mjs b/scripts/verify-package-update.mjs index 5998ab77f..8d8582b44 100644 --- a/scripts/verify-package-update.mjs +++ b/scripts/verify-package-update.mjs @@ -98,6 +98,7 @@ export async function verifyPackageUpdate({ HOME: home, USERPROFILE: home, APPDATA: join(home, "AppData", "Roaming"), + LOCALAPPDATA: join(home, "AppData", "Local"), CLAUDE_CONFIG_DIR: home, CODEX_HOME: join(home, ".codex"), COPILOT_HOME: join(home, ".copilot"), diff --git a/scripts/verify-published-package.mjs b/scripts/verify-published-package.mjs index 8c0281d56..8a5eb48d8 100644 --- a/scripts/verify-published-package.mjs +++ b/scripts/verify-published-package.mjs @@ -118,6 +118,7 @@ async function verifyPublishedUpdate({ version: targetVersion, canaryRoot }) { HOME: home, USERPROFILE: home, APPDATA: join(home, "AppData", "Roaming"), + LOCALAPPDATA: join(home, "AppData", "Local"), CLAUDE_CONFIG_DIR: home, CODEX_HOME: join(home, ".codex"), COPILOT_HOME: join(home, ".copilot"), diff --git a/scripts/verify-real-browser.mjs b/scripts/verify-real-browser.mjs index 48c9bec08..158877bca 100644 --- a/scripts/verify-real-browser.mjs +++ b/scripts/verify-real-browser.mjs @@ -9,7 +9,10 @@ import { join } from "node:path"; import { fileURLToPath } from "node:url"; import { CdpBrowserProvider } from "../dist/browser/CdpBrowserProvider.js"; -import { waitForBrowserDevtoolsPort } from "../dist/browser/BrowserProcessStartup.js"; +import { + BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS, + waitForBrowserDevtoolsPort, +} from "../dist/browser/BrowserProcessStartup.js"; import { createBrowserScenarioProvider } from "../dist/composition/browserScenario.js"; import { inspectWebPageInputSchema, @@ -42,12 +45,12 @@ import { verifyBrowserNetworkEvidence } from "./lib/browser-network-e2e.mjs"; import { verifyBrowserScriptExport } from "./lib/browser-script-export-e2e.mjs"; import { verifyBrowserModules } from "./lib/browser-module-e2e.mjs"; import { verifyBrowserDomDestinations } from "./lib/browser-dom-destinations-e2e.mjs"; +import { verifyBrowserSocketTransport } from "./lib/browser-socket-e2e.mjs"; import { verifyBrowserCaptureMetadataBudget } from "./lib/browser-capture-metadata-budget-e2e.mjs"; import { artifactCliEvidence, artifactMcpResult } from "./lib/artifact-e2e.mjs"; import { verifyScenarioEnvironment } from "./lib/browser-scenario-environment-e2e.mjs"; import { verifyScenarioStorage } from "./lib/browser-scenario-storage-e2e.mjs"; -const REAL_BROWSER_STARTUP_TIMEOUT_MS = 60_000; const SCENARIO_SECRET_VALUE = "rea-browser-verifier-secret"; const SCENARIO_URL_SECRET_VALUE = "rea-browser-url-verifier-secret"; process.env.REA_BROWSER_VERIFIER_SECRET = SCENARIO_SECRET_VALUE; @@ -92,7 +95,7 @@ try { executable, activePortPath: join(profile, "DevToolsActivePort"), stderr: () => stderr, - timeoutMs: REAL_BROWSER_STARTUP_TIMEOUT_MS, + timeoutMs: BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS, }); const endpoint = `http://127.0.0.1:${String(port)}`; const provider = new CdpBrowserProvider(); @@ -232,6 +235,13 @@ try { pageProxy = await startPageCdpProxy(endpoint); await verifyPageScopedTransport(provider, pageProxy, site.origin); + const browserSocketTransport = await verifyBrowserSocketTransport({ + provider, + profile, + endpoint, + target, + origin: site.origin, + }); const attachedScenarioInput = browserScenario( { mode: "connect", @@ -375,6 +385,7 @@ try { bundle_cli_and_stdio_mcp: true, sessionEvents: session.value.timeline.length, pageScopedTransport: true, + browserSocketTransport, screenshotBytes: screenshot.value.artifact.bytes, largeScreenshot, popupEvents, diff --git a/scripts/verify-real-ghidra-mips.mjs b/scripts/verify-real-ghidra-mips.mjs index 1c04ea978..12c00366d 100644 --- a/scripts/verify-real-ghidra-mips.mjs +++ b/scripts/verify-real-ghidra-mips.mjs @@ -12,6 +12,7 @@ import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js"; import { inspectMipsReadelf, + inspectLegacyMipsReadelf, assertMipsLoadedImage, assertMipsProbe, assertMipsGlobal, @@ -122,6 +123,157 @@ async function verifySoftFloat(call, symbols, byteOrder) { const reports = []; let primaryFailure; try { + if (!psp) { + // Recordless legacy admission: ABI field 0, no ABI flags record, pinned + // 32-bit language. Independent GNU readelf checks the header interpretation. + for (const byteOrder of ["little", "big"]) { + const fixture = await buildMipsFixture(workspace, byteOrder, "hard", { + legacy: true, + }); + const legacy = await inspectLegacyMipsReadelf(fixture.path, byteOrder); + const resolved = await parseBinaryTarget(fixture.path); + if (!resolved.ok) throw resolved.error; + assert.equal(resolved.value.mips.flags, 0x50000001); + assert.equal(resolved.value.mips.abiFlags, null); + assert.deepEqual( + resolved.value.mips.limitations ?? [], + legacy.limitations, + "REA legacy ABI-record absence interpretation disagrees with GNU readelf", + ); + assert.equal(resolved.value.sha256, fixture.sha256); + const snapshotPath = join(workspace, `${byteOrder}-legacy.snapshot.json`); + const transport = new StdioClientTransport({ + command: process.execPath, + args: [entrypoint, "mcp"], + env, + stderr: "pipe", + }); + const client = new Client({ + name: "rea-real-mips-legacy-proof", + version: "1", + }); + let stderr = ""; + transport.stderr?.on("data", (chunk) => { + stderr = (stderr + chunk.toString()).slice(-65536); + }); + const call = async (name, args = {}) => { + const reply = await client.callTool( + { name, arguments: args }, + undefined, + { timeout: 180000 }, + ); + return requireMcpOperationResult(reply, name); + }; + try { + await client.connect(transport); + const selected = await call("open_binary", { + path: fixture.path, + provider_id: "ghidra", + }); + assert.equal(selected.architecture, "mips"); + assert.equal(selected.sha256, fixture.sha256); + const image = await call("inspect_native_load_image"); + assertMipsLoadedImage(image.observations, byteOrder, fixture.sha256); + // The pinned import language is the checked one, not an auto-selection. + assert.equal( + image.observations.language_id, + `MIPS:${byteOrder === "little" ? "LE" : "BE"}:32:default`, + ); + const procedures = await call("list_procedures"); + const entry = procedures.find((p) => p.value === "rea_mips_entry"); + assert.ok(entry, "Entry function missing from legacy inventory"); + assert.equal( + BigInt(entry.address), + BigInt(legacy.symbols.rea_mips_entry), + ); + const probe = await call("inspect_native_instruction", { + address: legacy.symbols.rea_mips_probe, + }); + const probeBranch = await call("inspect_native_instruction", { + address: `0x${(BigInt(legacy.symbols.rea_mips_probe) + 4n).toString(16)}`, + }); + assertMipsProbe( + probe, + probeBranch, + byteOrder, + legacy.symbols.rea_mips_probe, + ); + const dossier = functionDossierSchema.parse( + await call("analyze_function", { procedure: entry.address }), + ); + assert.ok(dossier.pseudocode.trim().length > 0); + const saved = await call("close_binary", { + snapshot_path: snapshotPath, + }); + assert.ok(saved.evidence_records > 0); + const snapshot = parseAnalysisSnapshot( + JSON.parse(await readFile(snapshotPath, "utf8")), + ); + assert.equal(snapshot.target.architecture, "mips"); + const parameters = snapshot.binding.analysis_profile.parameters; + assert.equal( + parameters.language_id, + `MIPS:${byteOrder === "little" ? "LE" : "BE"}:32:default`, + ); + assert.equal(parameters.language_selection, "provider-pinned-v1"); + assert.equal( + parameters.mips_support_lane, + "elf32-exec-arch32-legacy-pinned-v1", + ); + assert.equal(parameters.mips_abi_basis, "inferred-o32-assumption-v1"); + assert.ok( + snapshot.evidence_bundle.records.some((record) => + parseEvidence(record).limitations.some((item) => + item.includes("analysis assumption"), + ), + ), + "Legacy o32 assumption limitation missing from retained Evidence", + ); + assert.ok( + snapshot.evidence_bundle.records.some((record) => + parseEvidence(record).limitations.some((item) => + item.includes("Floating-point ABI semantics are unverified"), + ), + ), + "Legacy floating-point uncertainty missing from retained Evidence", + ); + await call("open_binary", { + path: fixture.path, + provider_id: "ghidra", + snapshot_path: snapshotPath, + }); + assert.deepEqual(await call("list_procedures"), procedures); + await call("close_binary"); + } catch (cause) { + throw new Error( + `MIPS legacy ${byteOrder} verification failed; MCP stderr: ${stderr}`, + { cause }, + ); + } finally { + try { + await client.close(); + } finally { + await transport.close(); + } + } + reports.push({ + ...fixture, + independent_reader: legacy, + variant: "recordless-legacy-pinned", + effective_language: `MIPS:${ + byteOrder === "little" ? "LE" : "BE" + }:32:default`, + procedures: "verified-through-snapshot", + checks: [ + "recordless header admitted with inferred o32 assumption", + "pinned 32-bit language checked after import", + "legacy limitations retained in Evidence", + "snapshot round-trip and profile parameters", + ], + status: "passed", + }); + } + } for (const [byteOrder, floatAbi] of psp ? [["little", "hard"]] : [ diff --git a/scripts/verify-real-ghidra-raw.mjs b/scripts/verify-real-ghidra-raw.mjs index befba80e0..4341b5cfd 100644 --- a/scripts/verify-real-ghidra-raw.mjs +++ b/scripts/verify-real-ghidra-raw.mjs @@ -114,6 +114,22 @@ try { length: bytes.length, }); assert.equal(memory.normalized_result.bytes_hex, bytes.toString("hex")); + const signature = await call("search_bytes", { + pattern: "1e ?? 2f e1", + limit: 1000, + }); + assert.equal(signature.normalized_result.complete, true); + assert.equal(signature.normalized_result.next_cursor, null); + assert.equal(signature.normalized_result.searched.bytes, bytes.length); + assert.deepEqual( + signature.normalized_result.matches.map((match) => match.address), + [entry], + ); + assert.equal(signature.normalized_result.matches[0].file_offset, 0); + assert.equal( + signature.normalized_result.matches[0].procedure.address, + entry, + ); const procedures = await call("list_procedures"); assert.ok( procedures.normalized_result.some( @@ -211,6 +227,69 @@ try { return_42: true, }); } + const repeatedPath = join(workspace, "signatures.bin"); + const repeatedBytes = Buffer.from("b82a000000c3".repeat(1005), "hex"); + const repeatedSha256 = createHash("sha256") + .update(repeatedBytes) + .digest("hex"); + await writeFile(repeatedPath, repeatedBytes); + await call("open_binary", { + path: repeatedPath, + format: "raw-binary", + architecture: "x86_64", + entry_point: "0x1000", + }); + // Import/analysis is complete before timing the first uncached search. + await call("read_bytes", { address: "0x1000", length: 6 }, repeatedSha256); + const started = performance.now(); + const page = await call( + "search_bytes", + { pattern: "b8 ?? 00 00 00 c3", limit: 1000 }, + repeatedSha256, + ); + const searchElapsedMs = performance.now() - started; + assert.equal(page.normalized_result.matches.length, 1000); + assert.equal(page.normalized_result.complete, false); + const rest = await call( + "search_bytes", + { + pattern: "b8 ?? 00 00 00 c3", + limit: 1000, + cursor: page.normalized_result.next_cursor, + }, + repeatedSha256, + ); + assert.equal(rest.normalized_result.matches.length, 5); + assert.equal( + rest.normalized_result.matches[0].address, + `0x${(0x1000 + 6000).toString(16)}`, + ); + assert.equal(rest.normalized_result.complete, true); + const masked = await call( + "search_bytes", + { pattern: "bf", mask: "f0", start: "0x1000", end: "0x1005" }, + repeatedSha256, + ); + assert.deepEqual( + masked.normalized_result.matches.map((match) => match.address), + ["0x1000"], + ); + const overlapping = await call( + "search_bytes", + { pattern: "00 00", start: "0x1000", end: "0x1005" }, + repeatedSha256, + ); + assert.deepEqual( + overlapping.normalized_result.matches.map((match) => match.address), + ["0x1002", "0x1003"], + ); + const wrongCursor = await client.callTool({ + name: "search_bytes", + arguments: { pattern: "c3", cursor: page.normalized_result.next_cursor }, + }); + assert.equal(wrongCursor.isError, true); + await call("close_binary"); + assert.deepEqual(await readFile(repeatedPath), repeatedBytes); report = { ok: true, mocked: false, @@ -231,6 +310,15 @@ try { load_images_verified: true, original_unchanged: true, x86_targets: x86Targets, + byte_search: { + matches: 1005, + page_size: 1000, + first_search_elapsed_ms: searchElapsedMs, + timing_scope: "MCP request after import; first uncached search", + token_usage: "unmeasured", + process_family_cpu_time: "unmeasured", + peak_process_family_memory: "unmeasured", + }, }; } catch (error) { if (stderr.trim()) process.stderr.write(stderr); diff --git a/scripts/verify-solidity-metadata.mjs b/scripts/verify-solidity-metadata.mjs new file mode 100644 index 000000000..f927477d8 --- /dev/null +++ b/scripts/verify-solidity-metadata.mjs @@ -0,0 +1,105 @@ +import assert from "node:assert/strict"; +import { readFile } from "node:fs/promises"; +import { isAbsolute } from "node:path"; +import { pathToFileURL } from "node:url"; +import { performance } from "node:perf_hooks"; +import { inspectSolidityMetadata } from "../dist/evm/SolidityMetadata.js"; + +const compilerModule = process.env.REA_VERIFY_SOLC_MODULE; +if (!isAbsolute(compilerModule ?? "")) + throw new Error( + "Select an absolute caller-installed solc 0.8.30 module with REA_VERIFY_SOLC_MODULE.", + ); +const solc = (await import(pathToFileURL(compilerModule).href)).default; +assert.equal(solc.version(), "0.8.30+commit.73712a01.Emscripten.clang"); +const source = await readFile( + new URL("./fixtures/evm-interface.sol", import.meta.url), + "utf8", +); +const cases = [ + { name: "unoptimized", optimizer: false, viaIR: false, metadata: {} }, + { name: "viaIR", optimizer: true, viaIR: true, metadata: {} }, + { + name: "without-content-hash", + optimizer: false, + viaIR: false, + metadata: { bytecodeHash: "none" }, + }, + { + name: "without-CBOR", + optimizer: false, + viaIR: false, + metadata: { appendCBOR: false }, + }, +]; +const reports = []; +for (const fixture of cases) { + const compiled = JSON.parse( + solc.compile( + JSON.stringify({ + language: "Solidity", + sources: { "fixture.sol": { content: source } }, + settings: { + optimizer: { enabled: fixture.optimizer }, + viaIR: fixture.viaIR, + metadata: fixture.metadata, + outputSelection: { "*": { "*": ["evm.deployedBytecode.object"] } }, + }, + }), + ), + ); + assert.ok( + !(compiled.errors ?? []).some((error) => error.severity === "error"), + JSON.stringify(compiled.errors), + ); + const bytes = Buffer.from( + compiled.contracts["fixture.sol"].InterfaceFixture.evm.deployedBytecode + .object, + "hex", + ); + const cpu = process.cpuUsage(); + const started = performance.now(); + const parsed = inspectSolidityMetadata(bytes); + const firstMs = performance.now() - started; + const firstCpu = process.cpuUsage(cpu); + if (fixture.metadata.appendCBOR === false) + assert.equal(parsed.status, "unavailable"); + else { + assert.equal(parsed.status, "decoded"); + assert.deepEqual(parsed.compiler_version, { + representation: "release-bytes", + value: "0.8.30", + }); + assert.equal(parsed.bytecode_offset + parsed.cbor_bytes + 2, bytes.length); + if (fixture.metadata.bytecodeHash === "none") + assert.deepEqual( + parsed.entries.map(({ key }) => key), + [{ kind: "text", value: "solc" }], + ); + } + const warmResults = new Array(100); + const warmStart = performance.now(); + for (let index = 0; index < warmResults.length; index++) + warmResults[index] = inspectSolidityMetadata(bytes); + const warmMs = performance.now() - warmStart; + for (const result of warmResults) assert.deepEqual(result, parsed); + reports.push({ + fixture: fixture.name, + bytes: bytes.length, + status: parsed.status, + first_decode_ms: firstMs, + first_decode_cpu_us: firstCpu, + warm_100_decode_ms: warmMs, + }); +} +console.log( + JSON.stringify({ + ok: true, + compiler_version: solc.version(), + contract_execution: "not-performed", + cases: reports, + process_peak_rss_kib: process.resourceUsage().maxRSS, + measurements: + "In-process decode timings; peak RSS includes the compiler. End-to-end Linux provider/CLI costs and agent tokens were not measured.", + }), +); diff --git a/scripts/verify-wasm-artifact.mjs b/scripts/verify-wasm-artifact.mjs index 1f3bde2ca..305063ccc 100644 --- a/scripts/verify-wasm-artifact.mjs +++ b/scripts/verify-wasm-artifact.mjs @@ -310,7 +310,23 @@ try { }), ); assert.equal(error.category, "invalid_input"); - const failed = parseEvidence(error.details.partial_observation); + const partial = error.details.partial_observation; + assert.equal(partial.kind, "retained-evidence"); + const failedBundleReply = await client.callTool({ + name: "get_evidence_bundle", + arguments: {}, + }); + assert.notEqual( + failedBundleReply.isError, + true, + mcpTextValue(failedBundleReply), + ); + const failedBundle = JSON.parse(mcpTextValue(failedBundleReply)).result; + const failed = parseEvidence( + failedBundle.records.find( + ({ evidence_id }) => evidence_id === partial.evidence_id, + ), + ); assert.equal(failed.subject.digest.sha256, sha256(invalidBytes)); assert.equal( failed.normalized_result.validation, diff --git a/scripts/verify-wasm-interface-comparison.mjs b/scripts/verify-wasm-interface-comparison.mjs new file mode 100644 index 000000000..6b215eef3 --- /dev/null +++ b/scripts/verify-wasm-interface-comparison.mjs @@ -0,0 +1,208 @@ +import assert from "node:assert/strict"; +import { execFile } from "node:child_process"; +import { mkdtemp, readFile, writeFile, rm } from "node:fs/promises"; +import { join, resolve, isAbsolute } from "node:path"; +import { tmpdir } from "node:os"; +import { promisify } from "node:util"; +import { Client } from "@modelcontextprotocol/client"; +import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; +import { parseEvidence } from "../dist/domain/evidence.js"; + +const execute = promisify(execFile); +const bin = process.env.REA_WABT_BIN_DIRECTORY; +if (!isAbsolute(bin ?? "")) + throw new Error("Supply absolute REA_WABT_BIN_DIRECTORY with WABT 1.0.42"); +const root = await mkdtemp(join(tmpdir(), "rea-wasm-interface-real-")); +const cli = resolve("scripts/rea.mjs"); +const environment = { ...process.env }; +const command = async (file, args) => + ( + await execute(file, args, { + env: environment, + cwd: root, + timeout: 120000, + maxBuffer: 64 * 1024 * 1024, + }) + ).stdout; +const client = new Client({ + name: "wasm-interface-real-verifier", + version: "1", +}); +const transport = new StdioClientTransport({ + command: process.execPath, + args: [cli, "mcp"], + env: environment, + stderr: "pipe", +}); +const source = (parameter) => + `(module (type (func (param ${parameter}) (result i64))) (import "env" "f" (func (type 0))) (table 1 3 funcref) (memory 1 2) (global (mut i64) (i64.const 0)) (tag (param i32)) (export "run\\22\\ef\\bb\\bf" (func 0)) (export "table" (table 0)) (export "memory" (memory 0)) (export "global" (global 0)) (export "tag" (tag 0)))`; +const assemble = async (name, wat) => { + await writeFile(join(root, `${name}.wat`), wat); + const path = join(root, `${name}.wasm`); + await command(join(bin, "wat2wasm"), [ + "--enable-all", + `${name}.wat`, + "-o", + path, + ]); + return path; +}; +const inspect = async (path) => { + const result = await client.callTool({ + name: "inspect_wasm_artifact", + arguments: { path }, + }); + assert.notEqual(result.isError, true, JSON.stringify(result)); + return parseEvidence(result.structuredContent); +}; +const compare = async (left, right, scope = "both") => { + const input = { + left: { kind: "retained-evidence", evidence_id: left.evidence_id }, + right: { kind: "retained-evidence", evidence_id: right.evidence_id }, + scope, + }; + const reply = await client.callTool({ + name: "compare_wasm_interfaces", + arguments: input, + }); + assert.notEqual(reply.isError, true, JSON.stringify(reply)); + const mcp = parseEvidence(reply.structuredContent); + const file = join(root, "comparison.json"); + await writeFile(file, JSON.stringify({ left, right, scope })); + const local = parseEvidence( + JSON.parse( + await command(process.execPath, [ + cli, + "compare-wasm-interfaces", + file, + "--json", + ]), + ), + ); + assert.deepEqual(local, mcp); + return mcp.normalized_result; +}; +try { + assert.equal( + (await command(join(bin, "wat2wasm"), ["--version"])).trim(), + "1.0.42", + ); + await client.connect(transport); + const path = await assemble("baseline", source("i32")); + const original = await readFile(path); + assert.equal(original[8], 1); + assert.ok(original[9] < 128); + // Section lengths allow padded u32 LEB128 encodings. Different original + // byte identities must compare equal after validated type resolution. + const equivalentPath = join(root, "equivalent.wasm"); + await writeFile( + equivalentPath, + Buffer.concat([ + original.subarray(0, 9), + Buffer.from([original[9] | 128, 0]), + original.subarray(10), + ]), + ); + const left = await inspect(path); + const equivalent = await inspect(equivalentPath); + assert.notEqual(left.subject.digest.sha256, equivalent.subject.digest.sha256); + const same = await compare(left, equivalent); + assert.deepEqual(same.summary, { + added: 0, + removed: 0, + changed: 0, + unchanged: 6, + unknown: 0, + }); + const facts = left.normalized_result.interface.entries; + assert.deepEqual(facts.find((e) => e.name === "memory").signature, { + kind: "memory", + page_size: "65536", + address_type: "i32", + minimum: "1", + maximum: "2", + shared: false, + }); + assert.deepEqual(facts.find((e) => e.name === "tag").signature, { + kind: "tag", + params: ["i32"], + results: [], + }); + assert.deepEqual(facts.find((e) => e.name === 'run"\ufeff').signature, { + kind: "func", + params: ["i32"], + results: ["i64"], + }); + const changed = await inspect(await assemble("changed", source("i64"))); + assert.deepEqual((await compare(left, changed)).summary, { + added: 0, + removed: 0, + changed: 2, + unchanged: 4, + unknown: 0, + }); + const smallPage = await inspect( + await assemble( + "small-page", + '(module (memory 1 2 (pagesize 1)) (export "memory" (memory 0)))', + ), + ); + const defaultPage = await inspect( + await assemble( + "default-page", + '(module (memory 1 2) (export "memory" (memory 0)))', + ), + ); + assert.equal((await compare(defaultPage, smallPage)).summary.changed, 1); + const empty = await inspect(await assemble("empty", "(module)")); + assert.equal((await compare(left, empty)).summary.removed, 6); + assert.equal((await compare(empty, left, "exports")).summary.added, 5); + const duplicate = await inspect( + await assemble( + "duplicate", + '(module (import "e" "same" (func (param i32))) (import "e" "same" (func (param i64))))', + ), + ); + const duplicates = await compare(duplicate, duplicate, "imports"); + assert.equal(duplicates.summary.unknown, 1); + assert.equal(duplicates.changes[0].left.length, 2); + const unsupported = await inspect( + await assemble( + "exnref", + '(module (import "e" "abstract-ref" (global exnref)))', + ), + ); + assert.equal((await compare(unsupported, unsupported)).summary.unknown, 1); + console.log( + JSON.stringify( + { + verified: true, + toolchain: "WABT 1.0.42", + transport: "compiled CLI + stdio MCP", + interface_fact_bytes: Buffer.byteLength( + JSON.stringify(left.normalized_result.interface), + ), + comparison_result_bytes: Buffer.byteLength(JSON.stringify(same)), + original_sha256: left.subject.digest.sha256, + equivalent_sha256: equivalent.subject.digest.sha256, + cases: [ + "equivalent padded LEB128", + "function signature change", + "all admitted kinds", + "custom memory page-size change", + "escaped/BOM name", + "empty interface additions/removals", + "scope", + "duplicate imports", + "unsupported exception reference", + ], + runtime_execution: "not-performed", + }, + null, + 2, + ), + ); +} finally { + await client.close(); + await rm(root, { recursive: true, force: true }); +} diff --git a/scripts/verify-windows-native.mjs b/scripts/verify-windows-native.mjs index 8623081f9..0a42cb604 100644 --- a/scripts/verify-windows-native.mjs +++ b/scripts/verify-windows-native.mjs @@ -472,10 +472,9 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe signal: AbortSignal.timeout(15_000), }); assert.equal(String(ready).trim(), "ready"); - await assert.rejects( - retryRuntime.close(), - /owned cleanup object|Delete owned object/u, - ); + // The reader denies delete sharing. Cleanup must retain ownership when + // its handle-relative open receives ERROR_SHARING_VIOLATION (32). + await assert.rejects(retryRuntime.close(), { win32Code: 32 }); assert.equal( windowsPrivateRuntime(retryRuntime.observation.path), retryRuntime, diff --git a/scripts/verify-windows-package.mjs b/scripts/verify-windows-package.mjs index 67dee06c2..425b5cbd3 100644 --- a/scripts/verify-windows-package.mjs +++ b/scripts/verify-windows-package.mjs @@ -52,8 +52,10 @@ try { "bridge/ghidra/ReaGhidraBridge.java", "native/windows/build/manifest.json", "native/windows/build/rea-windows-x64.node", + "dist/android/JadxWorkspaceRoot.js", "dist/main.js", "dist/cli.js", + "scripts/rea-android-workspace-guardian.mjs", "scripts/rea.mjs", ]) if (!packagedPaths.has(required)) @@ -108,6 +110,7 @@ try { HOME: home, USERPROFILE: home, APPDATA: join(home, "AppData", "Roaming"), + LOCALAPPDATA: join(home, "AppData", "Local"), CLAUDE_CONFIG_DIR: home, CODEX_HOME: join(home, ".codex"), XDG_CONFIG_HOME: join(home, ".config"), diff --git a/scripts/verify/android/analysis.mjs b/scripts/verify/android/analysis.mjs index 43c51d82b..30024b557 100644 --- a/scripts/verify/android/analysis.mjs +++ b/scripts/verify/android/analysis.mjs @@ -173,6 +173,50 @@ try { assert.deepEqual(structured.normalized_result, evidence.normalized_result); console.log(`PASS MCP/CLI parity ${task.name}`); } + const overloadedInput = { + path, + class_name: "androidx.activity.ComponentActivity", + method_name: "setContentView", + overload_index: 0, + }; + const overloadedCli = await execute( + process.execPath, + [ + entrypoint, + "trace-android-references", + path, + overloadedInput.class_name, + "--method-name", + overloadedInput.method_name, + "--overload-index", + "0", + "--format", + "json", + ], + { + cwd: repository, + env: environment, + timeout: 150_000, + maxBuffer: 8 * 1024 * 1024, + }, + ); + const overloadedEvidence = parseEvidence(JSON.parse(overloadedCli.stdout)); + assert.equal(overloadedEvidence.normalized_result.overload_index, 0); + assert.equal( + overloadedEvidence.normalized_result.reported_signature, + "setContentView(int): void", + ); + assert.equal(overloadedEvidence.normalized_result.total, 1); + const overloadedMcp = await client.callTool( + { name: "trace_android_references", arguments: overloadedInput }, + { timeout: 150_000 }, + ); + assert.notEqual(overloadedMcp.isError, true, JSON.stringify(overloadedMcp)); + assert.deepEqual( + parseEvidence(overloadedMcp.structuredContent).normalized_result, + overloadedEvidence.normalized_result, + ); + console.log("PASS real overloaded method references through CLI/MCP"); } catch (cause) { process.stderr.write(stderr); throw cause; @@ -205,7 +249,7 @@ assert.equal(method.body_status, "available"); assert.equal(method.source.status, "complete"); assert.ok(method.source.text.includes("getIntent()")); assert.ok(method.source.text.includes("setListAdapter")); -assert.equal(method.method.dex_descriptor, null); +assert.equal(method.method.dex_descriptor, "(Landroid/os/Bundle;)V"); const references = cliResults.get("trace_android_references").normalized_result; assert.ok(references.total > 0); assert.ok( @@ -264,3 +308,8 @@ if (process.env.REA_ANDROID_REPORT !== undefined) JSON.stringify(report, null, 2) + "\n", ); console.log(JSON.stringify(report, null, 2)); +assert.notEqual( + windowsLifecycle?.cli_abrupt_owner_exit_workspace_cleanup, + false, + "Android workspace remained after abrupt owner exit", +); diff --git a/scripts/verify/android/direct-invokes.mjs b/scripts/verify/android/direct-invokes.mjs new file mode 100644 index 000000000..7cb9972db --- /dev/null +++ b/scripts/verify/android/direct-invokes.mjs @@ -0,0 +1,112 @@ +import assert from "node:assert/strict"; +import { Client } from "@modelcontextprotocol/client"; +import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; +import { parseEvidence } from "../../../dist/domain/evidence.js"; +import { hashAndroidFile } from "./digest.mjs"; + +/** The public result keeps one using node and exposes its two raw DEX invokes. */ +export const verifyAndroidDirectInvokes = async ({ + apk, + dexEntry, + repository, + entrypoint, + environment, + execute, +}) => { + const digest = await hashAndroidFile(apk); + const check = (value) => { + const evidence = parseEvidence(value); + assert.equal(evidence.subject.digest.sha256, digest); + assert.equal(evidence.operation, "trace_android_references"); + const result = evidence.normalized_result; + assert.equal(result.class_name, "fixture.Target"); + assert.equal(result.method_name, "mark"); + assert.equal(result.dex_descriptor, "()V"); + assert.equal(result.reference_unit, "using_node"); + assert.equal(result.total, 1); + assert.equal(result.references.length, 1); + const source = result.references[0]; + assert.equal(source.reported_full_name, "fixture.Caller.twice"); + assert.deepEqual( + source.direct_invokes.map((site) => site.dex_code_unit_offset), + [0, 3], + ); + for (const site of source.direct_invokes) { + assert.equal(site.dex_entry, dexEntry); + assert.equal(site.source_raw_method_id, "fixture.Caller.twice()V"); + assert.equal(site.opcode, "INVOKE_STATIC"); + } + assert.equal( + source.direct_invokes[0].dex_method_index, + source.direct_invokes[1].dex_method_index, + ); + return evidence; + }; + const cliResponse = await execute( + process.execPath, + [ + entrypoint, + "trace-android-references", + apk, + "fixture.Target", + "--method-name", + "mark", + "--format", + "json", + ], + { + cwd: repository, + env: environment, + timeout: 150_000, + maxBuffer: 8 * 1024 * 1024, + }, + ); + const cli = check(JSON.parse(cliResponse.stdout)); + const transport = new StdioClientTransport({ + command: process.execPath, + args: [entrypoint, "mcp"], + cwd: repository, + env: environment, + stderr: "pipe", + }); + let stderr = ""; + transport.stderr?.on("data", (chunk) => { + stderr += chunk.toString(); + }); + const client = new Client({ + name: "rea-real-android-direct-invokes", + version: "1", + }); + try { + await client.connect(transport, { timeout: 30_000 }); + const response = await client.callTool( + { + name: "trace_android_references", + arguments: { + path: apk, + class_name: "fixture.Target", + method_name: "mark", + }, + }, + { timeout: 150_000 }, + ); + assert.notEqual(response.isError, true, JSON.stringify(response)); + const mcp = check(response.structuredContent); + assert.deepEqual(mcp.normalized_result, cli.normalized_result); + assert.deepEqual(mcp.parameters, cli.parameters); + assert.deepEqual(mcp.provider, cli.provider); + } catch (cause) { + process.stderr.write(stderr); + throw cause; + } finally { + await client.close(); + } + assert.equal( + await hashAndroidFile(apk), + digest, + "Static inspection changed fixture bytes", + ); + console.log( + `PASS real CLI/MCP direct-invoke offsets from ${dexEntry}, parity and unchanged APK`, + ); +}; diff --git a/scripts/verify/android/metadata.mjs b/scripts/verify/android/metadata.mjs index eebcaf812..a44da336c 100644 --- a/scripts/verify/android/metadata.mjs +++ b/scripts/verify/android/metadata.mjs @@ -3,6 +3,7 @@ import { mkdtemp, rm } from "node:fs/promises"; import { tmpdir } from "node:os"; import { delimiter, join, resolve } from "node:path"; import { verifyAndroidInitializers } from "./initializers.mjs"; +import { verifyAndroidDirectInvokes } from "./direct-invokes.mjs"; /** Prove cold metadata stays unmaterialized and overload identities survive code generation. */ export const verifyAndroidMetadata = async ({ @@ -60,6 +61,7 @@ export const verifyAndroidMetadata = async ({ apk, resolve(repository, "tests/fixtures/android/static-initializers"), classes, + resolve(repository, "tests/fixtures/android/direct-invokes"), ], { timeout: 120_000, maxBuffer: 8 * 1024 * 1024 }, ); @@ -71,12 +73,19 @@ export const verifyAndroidMetadata = async ({ result.stdout, /INITIALIZER_REGRESSION_PASS folded=2 bounded=true/u, ); + assert.match( + result.stdout, + /DIRECT_INVOKE_REGRESSION_PASS nodes=1 offsets=0,3 cross_dex=true/u, + ); console.log( "PASS real cold metadata, full generic types and stable overloaded method selection", ); console.log( "PASS real folded and mixed static initializers, absent bodies and bounded smali", ); + console.log( + "PASS real raw DEX direct-invoke offsets and cross-DEX source identity", + ); await verifyAndroidInitializers({ apk: resolve(classes, "initializers.apk"), repository, @@ -84,6 +93,22 @@ export const verifyAndroidMetadata = async ({ environment, execute, }); + await verifyAndroidDirectInvokes({ + apk: resolve(classes, "direct-invokes.apk"), + dexEntry: "classes.dex", + repository, + entrypoint, + environment, + execute, + }); + await verifyAndroidDirectInvokes({ + apk: resolve(classes, "direct-invokes-multidex.apk"), + dexEntry: "classes2.dex", + repository, + entrypoint, + environment, + execute, + }); } finally { await rm(classes, { recursive: true, force: true }); } diff --git a/scripts/verify/android/windows-guardian-isolation.mjs b/scripts/verify/android/windows-guardian-isolation.mjs new file mode 100644 index 000000000..1a6308970 --- /dev/null +++ b/scripts/verify/android/windows-guardian-isolation.mjs @@ -0,0 +1,105 @@ +import assert from "node:assert/strict"; +import { spawn } from "node:child_process"; +import { stat, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { createInterface } from "node:readline"; +import { setTimeout as delay } from "node:timers/promises"; +import { fileURLToPath } from "node:url"; +import { createJadxWorkspaceRoot } from "../../../dist/android/JadxWorkspaceRoot.js"; + +const source = fileURLToPath(import.meta.url); +const marker = "REA synthetic guardian isolation test\n"; +const exists = (path) => + stat(path).then( + () => true, + () => false, + ); + +if (process.argv[1] === source && process.argv[2] === "owner") { + const root = await createJadxWorkspaceRoot(); + await writeFile(join(root.path, "synthetic.txt"), marker); + process.stdout.write(JSON.stringify({ path: root.path }) + "\n"); + await new Promise(() => {}); +} + +const bounded = async (promise, milliseconds, label) => { + let timer; + try { + return await Promise.race([ + promise, + new Promise((_, reject) => { + timer = setTimeout(() => reject(new Error(label)), milliseconds); + }), + ]); + } finally { + clearTimeout(timer); + } +}; + +const launch = async () => { + const child = spawn(process.execPath, [source, "owner"], { + stdio: ["ignore", "pipe", "pipe"], + windowsHide: true, + }); + let stderr = ""; + child.stderr.on("data", (chunk) => { + stderr = (stderr + chunk.toString()).slice(-8192); + }); + const exit = new Promise((resolve, reject) => { + child.once("close", resolve); + child.once("error", reject); + }); + const lines = createInterface({ input: child.stdout, crlfDelay: Infinity }); + try { + const ready = await bounded( + lines[Symbol.asyncIterator]().next(), + 20000, + `Android guardian owner readiness timed out: ${stderr}`, + ); + assert.equal(ready.done, false, stderr); + const { path } = JSON.parse(ready.value); + assert.equal(typeof path, "string"); + assert.equal(await exists(join(path, "synthetic.txt")), true); + return { child, exit, path }; + } catch (cause) { + child.kill(); + await bounded(exit, 15000, "Android guardian owner did not exit"); + throw cause; + } +}; + +const stop = async ({ child, exit, path }) => { + assert.equal(child.kill(), true); + await bounded(exit, 15000, "Android guardian owner did not exit"); + for (let attempt = 0; attempt < 100; attempt++) { + if (!(await exists(path))) return; + await delay(100); + } + throw new Error(`Android guardian retained ${path} after owner death`); +}; + +/** Two independent owners must never delete one another's private workspace. */ +export async function verifyWindowsGuardianIsolation() { + assert.equal(process.platform, "win32"); + const first = await launch(); + let second; + try { + second = await launch(); + assert.notEqual(first.path, second.path); + await stop(first); + assert.equal( + await exists(join(second.path, "synthetic.txt")), + true, + "First owner deleted the second owner's workspace", + ); + await stop(second); + console.log("PASS Windows Android guardian two-owner cleanup isolation"); + } finally { + for (const owner of [first, second]) { + if (owner === undefined) continue; + if (owner.child.exitCode === null && owner.child.signalCode === null) + owner.child.kill(); + await bounded(owner.exit, 15000, "Android guardian owner did not exit"); + } + } +} diff --git a/scripts/verify/android/windows-lifecycle.mjs b/scripts/verify/android/windows-lifecycle.mjs index 5c1a9c02d..d2b281c60 100644 --- a/scripts/verify/android/windows-lifecycle.mjs +++ b/scripts/verify/android/windows-lifecycle.mjs @@ -17,6 +17,7 @@ import { resolveJadxConfigurationInputs, validateJadxConfiguration, } from "../../../dist/android/JadxConfiguration.js"; +import { verifyWindowsGuardianIsolation } from "./windows-guardian-isolation.mjs"; /** Verify real Windows cancellation, disconnect and abrupt CLI owner exit. */ export async function verifyWindowsAndroidLifecycle(options) { @@ -24,15 +25,16 @@ export async function verifyWindowsAndroidLifecycle(options) { await verifyJavaSelection(options.environment); const cancellation = await verifyMcp(options, "cancel"); await verifyMcp(options, "disconnect"); - await verifyCliOwnerExit(options); + const abruptWorkspaceRemoved = await verifyCliOwnerExit(options); + await verifyWindowsGuardianIsolation(); return { mcp_cancelled_engine_exited: true, mcp_cancelled_workspace_removed: true, mcp_ping_and_next_call: true, mcp_disconnect_engine_exited: true, cli_abrupt_owner_exit_engine_exited: true, - // A forced Windows process exit cannot execute JavaScript workspace cleanup. - cli_abrupt_owner_exit_workspace_cleanup: false, + cli_abrupt_owner_exit_workspace_cleanup: abruptWorkspaceRemoved, + guardian_two_owner_isolation: true, observed_resources: cancellation, java_home_and_quoted_case_insensitive_path: true, }; @@ -161,6 +163,7 @@ async function verifyCliOwnerExit({ path, repository, }) { + const baseline = new Set(await runtimeRoots()); const child = spawn( process.execPath, [entrypoint, "inspect-android-package", path, "--format", "json"], @@ -185,7 +188,40 @@ async function verifyCliOwnerExit({ await deadline(exited, 15000); await observer.verifyExit(); assert.equal(output, "", "Aborted CLI published successful Evidence"); - console.log("PASS Windows CLI abrupt owner exit: real Java job cleanup"); + const created = (await runtimeRoots()).filter( + (name) => !baseline.has(name), + ); + assert.ok( + created.length <= 1, + "Concurrent Android workspaces make abrupt cleanup observation ambiguous", + ); + if (created.length === 0) { + console.log( + "PASS Windows CLI abrupt owner exit: real Java job and workspace cleanup", + ); + return true; + } + const ownedName = created[0]; + try { + await waitFor( + async () => !(await runtimeRoots()).includes(ownedName), + 5000, + "Android workspace remained after abrupt owner exit", + ); + } catch (cause) { + if ( + !(cause instanceof Error) || + cause.message !== "Android workspace remained after abrupt owner exit" + ) + throw cause; + } + const removed = !(await runtimeRoots()).includes(ownedName); + console.log( + removed + ? "PASS Windows CLI abrupt owner exit: real Java job and workspace cleanup" + : "OBSERVED Windows CLI abrupt owner exit: Java job cleaned, workspace residue remains", + ); + return removed; } finally { if (child.exitCode === null && child.signalCode === null) child.kill(); await deadline(exited, 15000); diff --git a/scripts/verify/architecture-guards.mjs b/scripts/verify/architecture-guards.mjs index 967e210de..3ca254d7e 100644 --- a/scripts/verify/architecture-guards.mjs +++ b/scripts/verify/architecture-guards.mjs @@ -89,6 +89,30 @@ check( ["src/domain/javascript/javascriptSemanticTraversal.ts"], ); +// A producer that pins both the soft and hard ceiling of every resource must take +// the vector from one owner. Audited 2026-10-13 against the other prlimit lanes: +// src/pcap/PcapNativeBound.ts pairs soft and hard but budgets 300 CPU seconds and +// has no resident-input ceiling; src/evm/EvmoleInterfaceProvider.ts, +// src/firmware/FirmwareCommand.ts and src/javascript/recovery/WakaruCommand.ts +// spell soft-only forms (`--core=0:`, `--core=0`). Those four stay as they are; +// a new paired producer calls prlimitArgumentVector instead of spelling options. +check( + "inline paired prlimit ceilings (use prlimitArgumentVector from src/process/LinuxResourceLimits)", + files, + /--core=0:0/, + ["src/process/LinuxResourceLimits.ts", "src/pcap/PcapNativeBound.ts"], +); + +// Census is spelling-shaped, like the rules above: a lane that expresses the same +// ceilings differently (`1024 ** 3`, camelCase fields) is not caught here, which is +// why the four lanes above are listed rather than left to a future reader. +check( + "inline resident-input budget (use BOUNDED_PRODUCER_LIMITS from src/process/LinuxResourceLimits)", + files, + /input_bytes:\s*32 \* 1024 \* 1024/, + ["src/process/LinuxResourceLimits.ts"], +); + // Fixed-arity backtick keys audited 2026-10-10: every prefix field is NUL-free // (hex digests, file paths, enums, numbers, or compositeKey JSON which escapes // NUL), with free-form source strings only in final position. New backtick-\0 diff --git a/scripts/website_i18n.py b/scripts/website_i18n.py index 928594df7..0f93546f0 100644 --- a/scripts/website_i18n.py +++ b/scripts/website_i18n.py @@ -25,11 +25,27 @@ LITERALS = {"REA", "Reverse Engineer Anything", "REA · Reverse Engineer Anythin FALLBACK_NOTICE = "Translation is being updated. Read this page in English for now." +def readme_edition_paths(root): + """Return repo-relative README editions: root English plus docs/readme translations.""" + editions = set() + if (root / "README.md").is_file(): + editions.add("README.md") + folder = root / "docs" / "readme" + if folder.is_dir(): + editions.update( + f"docs/readme/{path.name}" + for path in folder.glob("README_*.md") + if path.is_file() + ) + return editions + + def languages(root): """Load the website language inventory and check it against README editions.""" path = root / "website/locales/languages.json" + editions = readme_edition_paths(root) if not path.exists(): - if any(root.glob("README_*.md")): + if any(edition != "README.md" for edition in editions): raise ValueError("Missing languages.json for the translated README editions") return [{"code": "en", "name": "English", "readme": "README.md", "dir": "ltr"}] data = json.loads(path.read_text(encoding="utf-8")) @@ -49,8 +65,7 @@ def languages(root): raise ValueError(f"Duplicate README edition: {item['readme']}") codes.add(item["code"]) filenames.add(item["readme"]) - readmes = {path.name for path in root.glob("README*.md")} - if readmes and ({item["readme"] for item in data} != readmes): + if editions and ({item["readme"] for item in data} != editions): raise ValueError("Website languages must match the README language editions") return data diff --git a/src/adb/AdbProvider.test.ts b/src/adb/AdbProvider.test.ts index 23645e456..3d15325fd 100644 --- a/src/adb/AdbProvider.test.ts +++ b/src/adb/AdbProvider.test.ts @@ -1127,6 +1127,68 @@ exit 1`, expect(projected.message).toContain("not installed for 0"); }); +// adb exits 1 on a refused package change: `adb install` after printing the +// status line to stderr, `cmd package uninstall` after printing it to stdout. +it("preserves the device's install refusal when adb exits nonzero", async () => { + const { binary, root } = await writeAdbStub( + () => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi +if [ "$1" = "-s" ] && [ "$3" = "install" ]; then + printf 'Performing Streamed Install\\n' + printf 'adb: failed to install app.apk: Failure [INSTALL_FAILED_ALREADY_EXISTS: Attempt to re-install com.example without first uninstalling.]\\n' >&2 + exit 1 +fi +exit 1`, + ); + const apk = join(root, "app.apk"); + await writeFile(apk, "apk"); + const projected = await refusedBy(providerOver(binary), { + operation: "install_adb_package", + input: { serial: "emulator-5554", apk_path: apk, replace: false }, + }); + expect(projected).toMatchObject({ + code: "unsupported_target", + details: { + reason: + "INSTALL_FAILED_ALREADY_EXISTS: Attempt to re-install com.example without first uninstalling.", + }, + }); + expect(projected.message).toContain( + "INSTALL_FAILED_ALREADY_EXISTS: Attempt to re-install com.example without first uninstalling.", + ); +}); + +it("preserves the device's uninstall refusal when adb exits nonzero", async () => { + const { binary } = await writeAdbStub( + () => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi +if [ "$1" = "-s" ] && [ "$3" = "uninstall" ]; then printf 'Failure [not installed for 0]\\n'; exit 1; fi +exit 1`, + ); + const projected = await refusedBy(providerOver(binary), { + operation: "uninstall_adb_package", + input: { serial: "emulator-5554", package: "com.absent" }, + }); + expect(projected).toMatchObject({ + code: "unsupported_target", + details: { reason: "not installed for 0" }, + }); + expect(projected.message).toContain("not installed for 0"); +}); + +it("keeps a nonzero install without a device verdict as an unreadable result", async () => { + const { binary, root } = await writeAdbStub( + () => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi +if [ "$1" = "-s" ] && [ "$3" = "install" ]; then printf 'adb: failed to install app.apk: closed\\n' >&2; exit 1; fi +exit 1`, + ); + const apk = join(root, "app.apk"); + await writeFile(apk, "apk"); + const projected = await refusedBy(providerOver(binary), { + operation: "install_adb_package", + input: { serial: "emulator-5554", apk_path: apk, replace: false }, + }); + expect(projected).toMatchObject({ code: "unreadable_output" }); +}); + it("starts an app through resolved launcher component and force-stops it", async () => { const { binary } = await writeAdbStub( shellStub({ diff --git a/src/adb/AdbProvider.ts b/src/adb/AdbProvider.ts index d26345eba..bce989aca 100644 --- a/src/adb/AdbProvider.ts +++ b/src/adb/AdbProvider.ts @@ -71,6 +71,7 @@ import { parseBugreportCopiedPath, parseDumpsysPackageOutput, parseFeatureListOutput, + packageChangeFailureReason, parsePackageChangeOutput, parseProcessListOutput, parsePngDimensions, @@ -248,6 +249,31 @@ const runAdb = async ( }); }; +/** + * adb exits nonzero when the device refuses a package change, printing the + * `Failure [...]` verdict to stderr (install) or stdout (uninstall). Read the + * refusal from that failed run so it is not reported as unreadable output. + */ +const packageChangeRefusal = ( + operation: string, + serial: string, + error: AnalysisError, +): Result | undefined => { + if (!(error instanceof AnalysisProtocolError)) return undefined; + const failure = execFileOutputFailure(error.cause); + if (failure === undefined) return undefined; + const reason = packageChangeFailureReason( + `${failure.stdout}\n${failure.stderr}`, + ); + return reason === null + ? undefined + : err( + new AnalysisUnsupportedTargetError(operation, serial, reason, { + cause: error, + }), + ); +}; + const clientIdentityResult = ( selection: AdbBinarySelection, identity: AdbClientIdentity, @@ -1754,7 +1780,11 @@ export class AdbProvider { { timeoutMs: PULL_TIMEOUT_MS, maxOutputBytes: PULL_MAX_OUTPUT_BYTES }, signal, ); - if (!installed.ok) return installed; + if (!installed.ok) + return ( + packageChangeRefusal(operation, input.serial, installed.error) ?? + installed + ); const verdict = parsePackageChangeOutput(installed.value.stdout); if (verdict.status === "failure") return err( @@ -1803,7 +1833,11 @@ export class AdbProvider { }, signal, ); - if (!uninstalled.ok) return uninstalled; + if (!uninstalled.ok) + return ( + packageChangeRefusal(operation, input.serial, uninstalled.error) ?? + uninstalled + ); const verdict = parsePackageChangeOutput(uninstalled.value.stdout); if (verdict.status === "failure") return err( diff --git a/src/adb/AdbShellOutput.ts b/src/adb/AdbShellOutput.ts index 726e6164d..6dac4cc0c 100644 --- a/src/adb/AdbShellOutput.ts +++ b/src/adb/AdbShellOutput.ts @@ -293,6 +293,10 @@ export const parseBugreportCopiedPath = (stdout: string): string | null => { return match === null ? null : match[1]!; }; +/** Read the device's `Failure [REASON]` verdict from package-change output. */ +export const packageChangeFailureReason = (output: string): string | null => + /Failure \[([^\]]+)\]/u.exec(output)?.[1] ?? null; + /** * Parse `adb install`/`adb uninstall` output. adb can print `Failure [...]` * with a zero exit code, so the text decides, not the exit status. @@ -305,10 +309,11 @@ export const parsePackageChangeOutput = ( } => { const text = stdout.trim(); if (/^Success$/mu.test(text)) return { status: "success", reason: null }; - const failure = /Failure \[([^\]]+)\]/u.exec(text); return { status: "failure", - reason: failure?.[1] ?? (text === "" ? null : text.slice(0, 512)), + reason: + packageChangeFailureReason(text) ?? + (text === "" ? null : text.slice(0, 512)), }; }; diff --git a/src/android/JadxAnalysis.ts b/src/android/JadxAnalysis.ts index d8afed20c..502c90a69 100644 --- a/src/android/JadxAnalysis.ts +++ b/src/android/JadxAnalysis.ts @@ -45,7 +45,7 @@ const methodsOf = (summary: ClassSummary) => { reported_signature: method.signature, overload_index: index, is_constructor: method.is_constructor, - dex_descriptor: null, + dex_descriptor: method.dex_descriptor, }; }); }; @@ -314,11 +314,25 @@ const traceReferences = async ( ): Promise => { const methodName = request.input.method_name; const candidates = matchingMethods(summary, request); - if (candidates.length > 1) - throw new AnalysisCapabilityUnavailableError( - "jadx", - request.operation, - `References require a unique method name; ${summary.full_name}.${methodName} has ${candidates.length} overloads. No unambiguous reference result is available.`, + if (methodName === undefined && request.input.overload_index !== undefined) + inputFailure( + request, + "overload_index", + "Method name is required for an overload index.", + ); + if (candidates.length > 1 && request.input.overload_index === undefined) + inputFailure( + request, + "overload_index", + `Select one of ${candidates.length} overloads: ${candidates.map((method) => `${method.overload_index}: ${method.reported_signature}`).join("; ")}`, + ); + const index = request.input.overload_index ?? 0; + const selected = methodName === undefined ? undefined : candidates[index]; + if (methodName !== undefined && selected === undefined) + inputFailure( + request, + "overload_index", + `Index ${index} is outside the ${candidates.length} matching overloads.`, ); const expectedTarget = methodName === undefined @@ -329,7 +343,9 @@ const traceReferences = async ( methodName === undefined ? "get_xrefs_to_class" : "get_xrefs_to_method", { class_name: summary.full_name, - ...(methodName === undefined ? {} : { method_name: methodName }), + ...(methodName === undefined + ? {} + : { method_name: methodName, overload_index: index }), limit: 2_147_483_647, resolve_line: false, }, @@ -337,6 +353,9 @@ const traceReferences = async ( ); if ( xrefs.target !== expectedTarget || + (selected !== undefined && + (xrefs.overload_index !== index || + xrefs.reported_signature !== selected.reported_signature)) || xrefs.total !== xrefs.count || xrefs.count !== xrefs.items.length ) @@ -348,6 +367,9 @@ const traceReferences = async ( engine, class_name: summary.full_name, method_name: methodName ?? null, + overload_index: selected?.overload_index ?? null, + reported_signature: selected?.reported_signature ?? null, + dex_descriptor: selected?.dex_descriptor ?? null, total: xrefs.total, references: xrefs.items.map((reference) => ({ kind: reference.kind, @@ -357,7 +379,10 @@ const traceReferences = async ( top_class: reference.top_class, source_line: null, dex_offset: null, + direct_invokes: reference.direct_invokes ?? null, + direct_invokes_exhaustive: reference.direct_invokes_exhaustive ?? null, })), coverage: "complete", + reference_unit: "using_node", }); }; diff --git a/src/android/JadxProtocol.ts b/src/android/JadxProtocol.ts index 804b7ffa7..22a457b0f 100644 --- a/src/android/JadxProtocol.ts +++ b/src/android/JadxProtocol.ts @@ -2,6 +2,7 @@ import { z } from "zod"; import { AnalysisOutputError } from "../domain/analysisErrorCore.js"; import { jsonValueSchema } from "../domain/jsonValue.js"; import type { AndroidOperation } from "../domain/android/androidAnalysis.js"; +import { dexDescriptorSchema } from "../domain/android/androidAnalysis.js"; const count = z.number().int().nonnegative(); /** Selector failures authored by REA's bridge, separate from arbitrary engine errors. */ @@ -50,6 +51,7 @@ export const jadxClassSchema = z.object({ z.object({ name: z.string(), signature: z.string(), + dex_descriptor: dexDescriptorSchema, is_constructor: z.boolean(), def_pos: z.number().int(), }), @@ -72,6 +74,8 @@ export const jadxMethodSchema = z.object({ }); export const jadxXrefsSchema = z.object({ target: z.string(), + overload_index: count.optional(), + reported_signature: z.string().optional(), total: count, count, resolve_line: z.literal(false), @@ -83,6 +87,18 @@ export const jadxXrefsSchema = z.object({ containing_class: z.string(), top_class: z.string(), def_pos: z.number().int(), + direct_invokes_exhaustive: z.boolean().optional(), + direct_invokes: z + .array( + z.object({ + dex_entry: z.string().min(1), + source_raw_method_id: z.string().min(1), + dex_code_unit_offset: count, + dex_method_index: count, + opcode: z.string().regex(/^INVOKE_/u), + }), + ) + .optional(), }), ), }); diff --git a/src/android/JadxProvider.ts b/src/android/JadxProvider.ts index 6fda5a143..3980ce318 100644 --- a/src/android/JadxProvider.ts +++ b/src/android/JadxProvider.ts @@ -18,7 +18,10 @@ import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; import type { AndroidRequest } from "../domain/android/androidAnalysis.js"; import type { BinaryTarget } from "../domain/binaryTargetTypes.js"; import { err, ok, type Result } from "../domain/result.js"; -import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js"; +import { + createJadxWorkspaceRoot, + type JadxWorkspaceRoot, +} from "./JadxWorkspaceRoot.js"; import { snapshotAndroidEngine, snapshotAndroidTarget, @@ -152,7 +155,7 @@ const hashAndroidSource = async ( interface PendingCleanup { session: JadxSession | undefined; - root: PrivateRuntimeRoot | undefined; + root: JadxWorkspaceRoot | undefined; readonly previousError: AnalysisError | undefined; } @@ -211,7 +214,7 @@ const cleanup = async ( interface RetainedSession { readonly key: string; - readonly root: PrivateRuntimeRoot; + readonly root: JadxWorkspaceRoot; readonly session: JadxSession; readonly snapshot: string; readonly jarHash: string; @@ -325,7 +328,7 @@ export class JadxProvider implements AndroidAnalysisPort { options?.signal === undefined ? timeout : AbortSignal.any([options.signal, timeout]); - let root: PrivateRuntimeRoot | undefined; + let root: JadxWorkspaceRoot | undefined; let session: JadxSession | undefined; try { if (target.format !== "apk") @@ -389,20 +392,24 @@ export class JadxProvider implements AndroidAnalysisPort { request.operation, signal, ); - root = await PrivateRuntimeRoot.create({ prefix: "rea-android-" }); + root = await createJadxWorkspaceRoot(); + const workspaceSignal = + root.failureSignal === undefined + ? signal + : AbortSignal.any([signal, root.failureSignal]); const engine = await snapshotAndroidEngine( configuration.jar, jarHash, root.path, request.operation, - signal, + workspaceSignal, ); const snapshot = await snapshotAndroidTarget( target.path, target.sha256, root.path, request.operation, - signal, + workspaceSignal, ); const bridge = join(root.path, "ReaJadxBridge.java"); await copyFile(BRIDGE_SOURCE, bridge); @@ -410,7 +417,7 @@ export class JadxProvider implements AndroidAnalysisPort { const copiedBridgeHash = await hashAndroidSource( bridge, request.operation, - signal, + workspaceSignal, ); if (copiedBridgeHash !== bridgeHash) throw androidSourceChanged( @@ -418,7 +425,7 @@ export class JadxProvider implements AndroidAnalysisPort { BRIDGE_SOURCE, undefined, ); - signal.throwIfAborted(); + workspaceSignal.throwIfAborted(); session = new JadxSession( { command: configuration.java, @@ -430,7 +437,7 @@ export class JadxProvider implements AndroidAnalysisPort { ], cwd: root.path, hostEnvironment: this.environment, - signal, + signal: workspaceSignal, }, this.launcher, ); @@ -445,6 +452,10 @@ export class JadxProvider implements AndroidAnalysisPort { } root = retained.root; session = retained.session; + const activeSignal = + root.failureSignal === undefined + ? signal + : AbortSignal.any([signal, root.failureSignal]); const { snapshot } = retained; const outcome = ok( await session.execute({ @@ -453,10 +464,10 @@ export class JadxProvider implements AndroidAnalysisPort { snapshot, jarHash: retained.jarHash, bridgeHash: retained.bridgeHash, - signal, + signal: activeSignal, }), ); - signal.throwIfAborted(); + activeSignal.throwIfAborted(); this.#retained = retained; return outcome; } catch (cause) { diff --git a/src/android/JadxRelease.ts b/src/android/JadxRelease.ts index b5b6bd058..8c7d16840 100644 --- a/src/android/JadxRelease.ts +++ b/src/android/JadxRelease.ts @@ -25,7 +25,8 @@ export const JADX_PROVIDER_IDENTITY = Object.freeze({ /** Audited engine coverage, kept outside portable Android semantics. */ export const JADX_LIMITATIONS = [ "Static decompiler observations do not establish runtime behavior; the APK is never executed.", - "Method signatures use provider display types; exact DEX descriptors and instruction offsets are unavailable. Overload indices are local to this engine and artifact.", + "Method signatures use provider display types; DEX prototypes come from frozen JADX metadata. Display names may be aliases, and overload indices are local to this engine and artifact.", "Class inventories use parsed metadata before code generation, including synthetic members that may be omitted from decompiled source. Source positions remain unknown.", + "Incoming reference totals count JADX using nodes. Direct invoke entries within a using method report raw DEX code-unit offsets and method indexes, but coverage is limited to those nodes and directly matched invokes; it does not establish every possible dynamic or reflective call site.", "Manifest summary fields are extracted by the upstream engine; decoded XML is retained for verification. APK signatures, split APKs, native libraries and Android runtime capture are not verified by this provider.", ] as const; diff --git a/src/android/JadxWorkspaceRoot.ts b/src/android/JadxWorkspaceRoot.ts new file mode 100644 index 000000000..15b862320 --- /dev/null +++ b/src/android/JadxWorkspaceRoot.ts @@ -0,0 +1,224 @@ +import { spawn, type ChildProcessWithoutNullStreams } from "node:child_process"; +import { lstat } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { win32 } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { z } from "zod"; + +import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js"; + +/** Match the native owner's ordinary spelling only for extended local drive paths. */ +const localDrivePath = (path: string): string => + path.replace(/^\\\\\?\\(?=[A-Za-z]:\\)/u, ""); + +const guardianSource = fileURLToPath( + new URL("../../scripts/rea-android-workspace-guardian.mjs", import.meta.url), +); +const readySchema = z.strictObject({ + path: z.string(), + volumeSerial: z.string().regex(/^[a-f0-9]{16}$/u), + fileId: z.string().regex(/^[a-f0-9]{32}$/u), + privateDacl: z.literal(true), +}); + +/** The Android provider needs a path and a cleanup owner, regardless of host. */ +export interface JadxWorkspaceRoot { + readonly path: string; + readonly failureSignal?: AbortSignal; + close(): Promise; +} + +type GuardianExit = { + readonly code: number | null; + readonly signal: NodeJS.Signals | null; + readonly error?: Error; +}; + +const boundedWait = async ( + promise: Promise, + milliseconds: number, + label: string, +): Promise => { + let timer: NodeJS.Timeout | undefined; + try { + return await Promise.race([ + promise, + new Promise((_resolve, reject) => { + timer = setTimeout(() => reject(new Error(label)), milliseconds); + }), + ]); + } finally { + clearTimeout(timer); + } +}; + +/** A detached guardian retains native cleanup ownership without keeping its caller alive. */ +const releaseGuardianReferences = ( + child: ChildProcessWithoutNullStreams, +): void => { + child.stdin.destroy(); + child.stdout.destroy(); + child.stderr.destroy(); + child.unref(); +}; + +class WindowsJadxWorkspaceRoot implements JadxWorkspaceRoot { + readonly #failure = new AbortController(); + readonly failureSignal = this.#failure.signal; + readonly #child: ChildProcessWithoutNullStreams; + readonly #exit: Promise; + readonly #diagnostics: () => string; + #closing = false; + #closePromise: Promise | undefined; + + private constructor( + readonly path: string, + child: ChildProcessWithoutNullStreams, + exit: Promise, + diagnostics: () => string, + ) { + this.#child = child; + this.#exit = exit; + this.#diagnostics = diagnostics; + void this.#exit.then((result) => { + if (!this.#closing) + this.#failure.abort( + new Error( + `Android workspace guardian exited before cleanup: ${result.error?.message ?? result.code ?? result.signal ?? "unknown"}`, + ), + ); + }); + } + + static async create(): Promise { + const parent = tmpdir(); + const child = spawn(process.execPath, [guardianSource, parent], { + detached: true, + windowsHide: true, + stdio: ["pipe", "pipe", "pipe"], + env: { + SystemRoot: process.env.SystemRoot, + WINDIR: process.env.WINDIR, + TEMP: parent, + TMP: parent, + }, + }); + let diagnostics = ""; + child.stderr.on("data", (chunk: Buffer) => { + diagnostics = (diagnostics + chunk.toString()).slice(-8192); + }); + const exit = new Promise((resolve) => { + child.once("error", (error: Error) => + resolve({ code: null, signal: null, error }), + ); + child.once("close", (code, signal) => resolve({ code, signal })); + }); + const ready = new Promise((resolve, reject) => { + let frame = ""; + child.stdout.on("data", (chunk: Buffer) => { + frame += chunk.toString(); + if (frame.length > 4096) { + reject( + new Error("Android workspace guardian readiness exceeded 4 KiB"), + ); + return; + } + const end = frame.indexOf("\n"); + if (end !== -1) resolve(frame.slice(0, end)); + }); + child.stdout.once("error", reject); + }); + try { + const line = await boundedWait( + Promise.race([ + ready, + exit.then((result) => { + throw new Error( + `Android workspace guardian exited during startup: ${result.error?.message ?? result.code ?? result.signal ?? "unknown"}; ${diagnostics}`, + ); + }), + ]), + 15000, + "Android workspace guardian startup timed out", + ); + const observation = readySchema.parse(JSON.parse(line)); + if ( + win32.dirname(localDrivePath(observation.path)).toLowerCase() !== + localDrivePath(win32.resolve(parent)).toLowerCase() || + !/^rea-android-[a-f0-9]{32}$/u.test(win32.basename(observation.path)) + ) + throw new Error( + "Android workspace guardian returned an unexpected root", + ); + child.stdout.resume(); + return new WindowsJadxWorkspaceRoot( + observation.path, + child, + exit, + () => diagnostics, + ); + } catch (cause) { + if (!child.stdin.destroyed) child.stdin.end(); + try { + await boundedWait( + exit, + 30000, + "Android workspace guardian did not settle after startup failure", + ); + } catch (cleanupFailure) { + throw new AggregateError( + [cause, cleanupFailure], + "Android workspace guardian startup and cleanup failed", + ); + } finally { + releaseGuardianReferences(child); + } + throw cause; + } + } + + close(): Promise { + this.#closing = true; + this.#closePromise ??= this.#finish().catch((cause: unknown) => { + this.#closePromise = undefined; + throw cause; + }); + return this.#closePromise; + } + + async #finish(): Promise { + if (!this.#child.stdin.destroyed) this.#child.stdin.end(); + let result: GuardianExit; + try { + result = await boundedWait( + this.#exit, + 120000, + `Android workspace guardian cleanup timed out for ${this.path}`, + ); + } catch (cause) { + releaseGuardianReferences(this.#child); + throw cause; + } + if (result.error !== undefined || result.code !== 0) + throw new Error( + `Android workspace guardian cleanup failed for ${this.path}: ${result.error?.message ?? result.code ?? result.signal ?? "unknown"}; ${this.#diagnostics()}`, + ); + try { + await lstat(this.path); + } catch (cause) { + if (cause instanceof Error && "code" in cause && cause.code === "ENOENT") + return; + throw cause; + } + throw new Error( + `Android workspace guardian exited but retained ${this.path}`, + ); + } +} + +/** POSIX retains its in-process root; Windows keeps a detached native owner. */ +export const createJadxWorkspaceRoot = (): Promise => + process.platform === "win32" + ? WindowsJadxWorkspaceRoot.create() + : PrivateRuntimeRoot.create({ prefix: "rea-android-" }); diff --git a/src/apktool/ApktoolConfiguration.test.ts b/src/apktool/ApktoolConfiguration.test.ts index d2d84874c..39cf1c652 100644 --- a/src/apktool/ApktoolConfiguration.test.ts +++ b/src/apktool/ApktoolConfiguration.test.ts @@ -1,4 +1,4 @@ -import { describe, expect, it } from "vitest"; +import { beforeEach, describe, expect, it, vi } from "vitest"; import { APKTOOL_COMMAND_ENV, @@ -10,29 +10,43 @@ import { resolveApktoolCommand, } from "./ApktoolConfiguration.js"; +const lookup = vi.hoisted(() => ({ stat: vi.fn() })); +vi.mock("node:fs/promises", async (importOriginal) => ({ + ...(await importOriginal()), + stat: lookup.stat, +})); +beforeEach(() => { + lookup.stat + .mockReset() + .mockRejectedValue(Object.assign(new Error("missing"), { code: "ENOENT" })); +}); + describe("resolveApktoolCommand", () => { - it("prefers an absolute REA_APKTOOL_COMMAND", () => { + it("prefers an absolute REA_APKTOOL_COMMAND", async () => { expect( - resolveApktoolCommand({ - [APKTOOL_COMMAND_ENV]: "/usr/local/bin/apktool", - }), + await resolveApktoolCommand( + { + [APKTOOL_COMMAND_ENV]: "/usr/local/bin/apktool", + }, + "linux", + ), ).toEqual({ command: "/usr/local/bin/apktool", commandSource: "environment", }); }); - it("falls back to PATH resolution without the environment key", () => { - expect(resolveApktoolCommand({})).toEqual({ + it("leaves POSIX PATH lookup to process creation", async () => { + expect(await resolveApktoolCommand({}, "linux")).toEqual({ command: "apktool", commandSource: "path", }); }); - it("rejects a relative environment path explicitly", () => { - expect(() => + it("rejects a relative environment path explicitly", async () => { + await expect( resolveApktoolCommand({ [APKTOOL_COMMAND_ENV]: "bin/apktool" }), - ).toThrow(/must be an absolute path/u); + ).rejects.toThrow(/must be an absolute path/u); }); }); @@ -119,3 +133,250 @@ describe("apktoolInvocation", () => { }); }); }); + +describe("Windows PATH launcher selection", () => { + const environment = { + Path: '"C:\\Selected Tools";C:\\Other Tools', + PathExt: ".EXE;.CMD;.BAT", + ComSpec: "C:\\Windows\\System32\\cmd.exe", + NoDefaultCurrentDirectoryInExePath: "", + }; + const files = (...paths: string[]) => { + const selected = new Set(paths.map((path) => path.toLowerCase())); + lookup.stat.mockImplementation(async (path: string) => { + if (selected.has(path.toLowerCase())) return { isFile: () => true }; + throw Object.assign(new Error("missing"), { code: "ENOENT" }); + }); + }; + + it.each([".BAT", ".CMD"])( + "resolves a default %s launcher before choosing cmd.exe for version and decode", + async (extension) => { + const command = `C:\\Selected Tools\\apktool${extension}`; + files(command); + const selection = await resolveApktoolCommand(environment, "win32"); + expect(selection).toEqual({ command, commandSource: "path" }); + for (const arguments_ of [ + ["--version"], + ["d", "-o", "C:\\output folder", "C:\\target.apk"], + ]) { + const invocation = apktoolInvocation( + selection, + arguments_, + environment, + "win32", + ); + expect(invocation).toEqual({ + command: environment.ComSpec, + arguments: [ + "/d", + "/e:on", + "/v:off", + "/s", + "/c", + `"${[command, ...arguments_].map((token) => `"${token}"`).join(" ")}"`, + ], + windowsVerbatimArguments: true, + }); + } + }, + ); + + it("honors PATH directory order before extension order", async () => { + files("C:\\Selected Tools\\apktool.CMD", "C:\\Other Tools\\apktool.EXE"); + expect(await resolveApktoolCommand(environment, "win32")).toMatchObject({ + command: "C:\\Selected Tools\\apktool.CMD", + }); + }); + + it("honors the selected PATHEXT ordering", async () => { + files("C:\\Selected Tools\\apktool.CMD", "C:\\Selected Tools\\apktool.EXE"); + expect( + await resolveApktoolCommand( + { ...environment, PathExt: ".CMD;.EXE" }, + "win32", + ), + ).toMatchObject({ command: "C:\\Selected Tools\\apktool.CMD" }); + }); + + it("keeps native executable arguments outside cmd.exe interpretation", async () => { + const command = "C:\\Selected Tools\\apktool.EXE"; + files(command); + const selection = await resolveApktoolCommand(environment, "win32"); + const arguments_ = ["d", "C:\\input & literal.apk"]; + expect( + apktoolInvocation(selection, arguments_, environment, "win32"), + ).toEqual({ + command, + arguments: arguments_, + windowsVerbatimArguments: false, + }); + }); + + it("preserves explicit launcher spelling without searching PATH", async () => { + const command = "C:\\selected alias\\APKTOOL.cmd"; + expect( + await resolveApktoolCommand( + { ...environment, [APKTOOL_COMMAND_ENV]: command }, + "win32", + ), + ).toEqual({ command, commandSource: "environment" }); + expect(lookup.stat).not.toHaveBeenCalled(); + }); + + it("does not inspect the current directory when the caller disables it", async () => { + await expect( + resolveApktoolCommand(environment, "win32"), + ).rejects.toMatchObject({ code: "command_missing" }); + expect( + lookup.stat.mock.calls.every( + ([path]) => + String(path).startsWith("C:\\Selected Tools\\") || + String(path).startsWith("C:\\Other Tools\\"), + ), + ).toBe(true); + }); + + it("does not turn an explicitly empty PATHEXT into default extensions", async () => { + files("C:\\Selected Tools\\apktool.EXE"); + await expect( + resolveApktoolCommand({ ...environment, PathExt: "" }, "win32"), + ).rejects.toMatchObject({ code: "command_missing" }); + expect(lookup.stat).not.toHaveBeenCalled(); + }); + + it("skips directories without treating them as executable launchers", async () => { + files("C:\\Other Tools\\apktool.EXE"); + const stat = lookup.stat.getMockImplementation()!; + lookup.stat.mockImplementation(async (path: string) => + path === "C:\\Selected Tools\\apktool.EXE" + ? { isFile: () => false } + : stat(path), + ); + expect(await resolveApktoolCommand(environment, "win32")).toMatchObject({ + command: "C:\\Other Tools\\apktool.EXE", + }); + }); + + it("preserves inspection failures instead of reporting a missing command or selecting a later file", async () => { + lookup.stat.mockRejectedValue( + Object.assign(new Error("denied"), { code: "EACCES" }), + ); + const failure = await resolveApktoolCommand(environment, "win32").catch( + (cause: unknown) => cause, + ); + expect(failure).toBeInstanceOf(ApktoolConfigurationFailure); + if (!(failure instanceof ApktoolConfigurationFailure)) throw failure; + expect(failure).toMatchObject({ + code: "command_unresolved", + diagnostics: { + candidate_path: "C:\\Selected Tools\\apktool.EXE", + filesystem_error: "EACCES", + }, + }); + expect(apktoolConfigurationAvailability(failure)).toMatchObject({ + code: "not_configured", + }); + expect(lookup.stat).toHaveBeenCalledTimes(1); + }); + + it("honors cancellation before inspecting launchers", async () => { + const controller = new AbortController(); + controller.abort(); + await expect( + resolveApktoolCommand(environment, "win32", controller.signal), + ).rejects.toMatchObject({ _tag: "AnalysisCancelledError" }); + expect(lookup.stat).not.toHaveBeenCalled(); + }); +}); + +it("does not select a launcher when cancellation arrives during lookup", async () => { + const controller = new AbortController(); + lookup.stat.mockImplementation(async () => { + controller.abort(); + return { isFile: () => true }; + }); + await expect( + resolveApktoolCommand( + { + PATH: "C:\\Tools", + PATHEXT: ".CMD", + NoDefaultCurrentDirectoryInExePath: "1", + }, + "win32", + controller.signal, + ), + ).rejects.toMatchObject({ _tag: "AnalysisCancelledError" }); + expect(lookup.stat).toHaveBeenCalledTimes(1); +}); + +it.each(["EXE", ".CMD\\..\\outside.exe", ".CMD\0"])( + "rejects malformed PATHEXT %j before path lookup", + async (PATHEXT) => { + await expect( + resolveApktoolCommand({ PATH: "C:\\Tools", PATHEXT }, "win32"), + ).rejects.toMatchObject({ + code: "command_unresolved", + diagnostics: { invalid_extension: PATHEXT }, + }); + expect(lookup.stat).not.toHaveBeenCalled(); + }, +); + +it("cancels a pending filesystem lookup and owns its eventual rejection", async () => { + const controller = new AbortController(); + let started!: () => void; + let rejectLookup!: (cause: Error) => void; + const pendingStat = new Promise((_resolve, reject) => { + rejectLookup = reject; + }); + const observed = new Promise((resolve) => { + started = resolve; + }); + lookup.stat.mockImplementation(() => { + started(); + return pendingStat; + }); + const pending = resolveApktoolCommand( + { + PATH: "C:\\Tools", + PATHEXT: ".CMD", + NODEFAULTCURRENTDIRECTORYINEXEPATH: "1", + }, + "win32", + controller.signal, + ); + await observed; + controller.abort(); + await expect(pending).rejects.toMatchObject({ + _tag: "AnalysisCancelledError", + }); + rejectLookup(new Error("filesystem completed after cancellation")); + await Promise.resolve(); + expect(lookup.stat).toHaveBeenCalledTimes(1); +}); + +it("bounds the entire pending PATH lookup by the selection deadline", async () => { + vi.useFakeTimers(); + try { + lookup.stat.mockReturnValue(new Promise(() => {})); + const failure = expect( + resolveApktoolCommand( + { + PATH: "C:\\Tools", + PATHEXT: ".CMD", + NODEFAULTCURRENTDIRECTORYINEXEPATH: "1", + }, + "win32", + ), + ).rejects.toMatchObject({ + _tag: "AnalysisTimeoutError", + timeoutMs: 20_000, + }); + await vi.advanceTimersByTimeAsync(20_000); + await failure; + expect(lookup.stat).toHaveBeenCalledTimes(1); + } finally { + vi.useRealTimers(); + } +}); diff --git a/src/apktool/ApktoolConfiguration.ts b/src/apktool/ApktoolConfiguration.ts index 12532b104..b1861f2ac 100644 --- a/src/apktool/ApktoolConfiguration.ts +++ b/src/apktool/ApktoolConfiguration.ts @@ -1,5 +1,12 @@ -import { AnalysisCancelledError } from "../domain/analysisErrorCore.js"; -import { isAbsolute } from "node:path"; +import { + AnalysisCancelledError, + AnalysisTimeoutError, +} from "../domain/analysisErrorCore.js"; +import { isAbsolute, win32 } from "node:path"; +import { stat } from "node:fs/promises"; +import { snapshotEnvironment } from "../process/snapshotEnvironment.js"; +import { ABORTED, waitForAbortable } from "../process/AbortablePromise.js"; +import { ProviderStartupDeadline } from "../process/ProviderDeadline.js"; import { execFileOutput, @@ -27,30 +34,119 @@ export interface ApktoolCommandSelection { } /** - * Resolve the caller-supplied apktool launcher. `REA_APKTOOL_COMMAND` must - * be absolute so the selection stays explicit; without it, apktool is - * resolved from `PATH` at run time. Both the upstream wrapper script and - * `java -jar` launcher scripts work: REA only runs ` --version` - * and ` d ...`. + * Preserve an explicit launcher, or resolve the default Windows command before + * choosing native versus batch process creation. POSIX keeps execFile's PATH + * lookup. The resolved selection is reused for the version and decode runs. */ -export const resolveApktoolCommand = ( +export const resolveApktoolCommand = async ( environment: Readonly>, -): ApktoolCommandSelection => { - const configured = environment[APKTOOL_COMMAND_ENV]; - if (configured === undefined || configured.trim() === "") + platform: NodeJS.Platform = process.platform, + signal?: AbortSignal, +): Promise => { + if (signal?.aborted === true) + throw new AnalysisCancelledError("inspect_apktool_client"); + const selectedEnvironment = snapshotEnvironment(environment, platform); + const configured = selectedEnvironment[APKTOOL_COMMAND_ENV]; + if (configured !== undefined && configured.trim() !== "") { + if ( + !(platform === "win32" + ? win32.isAbsolute(configured) + : isAbsolute(configured)) + ) + throw new ApktoolConfigurationFailure( + "not_absolute", + `${APKTOOL_COMMAND_ENV} must be an absolute path to the apktool launcher`, + { configured_command: configured }, + ); + return { command: configured, commandSource: "environment" }; + } + if (platform !== "win32") return { command: "apktool", commandSource: "path" }; - if (!isAbsolute(configured)) - throw new ApktoolConfigurationFailure( - "not_absolute", - `${APKTOOL_COMMAND_ENV} must be an absolute path to the apktool launcher`, - { configured_command: configured }, - ); - return { command: configured, commandSource: "environment" }; + const extensions = (selectedEnvironment.PATHEXT ?? ".EXE;.COM;.BAT;.CMD") + .split(";") + .filter((extension) => extension !== ""); + for (const extension of extensions) + if (!extension.startsWith(".") || /[\\/:\0]/u.test(extension)) + throw new ApktoolConfigurationFailure( + "command_unresolved", + "Windows PATHEXT must contain filename extensions, not paths", + { + selected_command: "apktool", + command_source: "path", + invalid_extension: extension, + }, + ); + const directories = (selectedEnvironment.PATH ?? "") + .split(";") + .filter((directory) => directory !== ""); + if (selectedEnvironment.NODEFAULTCURRENTDIRECTORYINEXEPATH === undefined) + directories.unshift(process.cwd()); + // A single deadline owns the whole PATH search, including filesystem waits. + const deadline = new ProviderStartupDeadline(VERSION_TIMEOUT_MS, signal); + const lookupInterruption = () => + deadline.interruption === "cancelled" + ? new AnalysisCancelledError("inspect_apktool_client") + : new AnalysisTimeoutError("inspect_apktool_client", VERSION_TIMEOUT_MS, { + diagnostics: { phase: "windows_path_lookup" }, + }); + try { + for (const entry of directories) { + const directory = + entry.startsWith('"') && entry.endsWith('"') + ? entry.slice(1, -1) + : entry; + for (const extension of extensions) { + if (deadline.interruption !== undefined) throw lookupInterruption(); + const candidate = win32.resolve(directory, `apktool${extension}`); + try { + const metadata = await waitForAbortable( + stat(candidate), + deadline.signal, + ); + if (metadata === ABORTED || deadline.interruption !== undefined) + throw lookupInterruption(); + if (metadata.isFile()) + return { command: candidate, commandSource: "path" }; + } catch (cause: unknown) { + if (deadline.interruption !== undefined) throw lookupInterruption(); + const code = + typeof cause === "object" && cause !== null + ? Reflect.get(cause, "code") + : undefined; + if (code === "ENOENT" || code === "ENOTDIR") continue; + throw new ApktoolConfigurationFailure( + "command_unresolved", + `The PATH-selected apktool launcher could not be inspected: ${candidate}`, + { + selected_command: "apktool", + command_source: "path", + candidate_path: candidate, + ...(typeof code === "string" ? { filesystem_error: code } : {}), + }, + { cause }, + ); + } + } + } + } finally { + deadline.dispose(); + } + + throw new ApktoolConfigurationFailure( + "command_missing", + "The apktool launcher was not found on the selected Windows PATH/PATHEXT", + { + selected_command: "apktool", + command_source: "path", + remediation: `Set ${APKTOOL_COMMAND_ENV} to an absolute apktool launcher, or add its directory and extension to PATH and PATHEXT`, + }, + ); }; export type ApktoolConfigurationFailureCode = | "not_absolute" | "command_missing" + | "command_unresolved" | "launcher_unsupported" | "version_unresolved"; @@ -121,7 +217,7 @@ export const inspectApktoolClient = async ( environment: Readonly> = process.env, platform: NodeJS.Platform = process.platform, ): Promise => { - const hostEnvironment = { ...process.env, ...environment }; + const hostEnvironment = snapshotEnvironment(environment, platform); let invocation: ApktoolInvocation; try { invocation = apktoolInvocation( @@ -195,7 +291,9 @@ const rejectionCodeOf = ( ): ProviderRejectionCode => code === "command_missing" ? "executable_missing" - : code === "not_absolute" || code === "launcher_unsupported" + : code === "not_absolute" || + code === "launcher_unsupported" || + code === "command_unresolved" ? "not_configured" : "version_unresolved"; diff --git a/src/apktool/ApktoolProvider.ts b/src/apktool/ApktoolProvider.ts index 1e1f187c0..c58ea1f49 100644 --- a/src/apktool/ApktoolProvider.ts +++ b/src/apktool/ApktoolProvider.ts @@ -51,6 +51,7 @@ import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js"; import { snapshotEnvironment } from "../process/snapshotEnvironment.js"; import { WindowsBatchCommandError } from "../process/WindowsBatchCommand.js"; import { + APKTOOL_COMMAND_ENV, ApktoolConfigurationFailure, apktoolConfigurationAvailability, apktoolInvocation, @@ -155,10 +156,9 @@ const runApktool = async ( ): Promise> => { if (signal?.aborted === true) return err(new AnalysisCancelledError(operation)); - const hostEnvironment = { ...process.env, ...environment }; let invocation: ReturnType; try { - invocation = apktoolInvocation(selection, arguments_, hostEnvironment); + invocation = apktoolInvocation(selection, arguments_, environment); } catch (cause) { if (!(cause instanceof WindowsBatchCommandError)) throw cause; const reason = `The apktool batch launcher cannot receive REA's decode paths through cmd.exe: ${cause.message}`; @@ -178,7 +178,7 @@ const runApktool = async ( arguments: invocation.arguments, windowsVerbatimArguments: invocation.windowsVerbatimArguments, runId: randomUUID(), - hostEnvironment, + hostEnvironment: environment, }, { timeoutMs: limits.timeoutMs, @@ -317,7 +317,9 @@ export interface ApktoolProviderOptions { * (`--no-src`) because Java-source recovery belongs to the JADX family. */ export class ApktoolProvider { - private readonly environment: Readonly>; + private readonly selectedEnvironment: + | Readonly> + | undefined; private readonly workspaces = new Set(); private readonly removeWorkspace: (path: string) => Promise; @@ -328,14 +330,25 @@ export class ApktoolProvider { #closePromise: Promise | undefined; constructor(options: ApktoolProviderOptions = {}) { - this.environment = snapshotEnvironment(options.environment ?? process.env); + this.selectedEnvironment = + options.environment === undefined + ? undefined + : snapshotEnvironment(options.environment); this.removeWorkspace = options.removeWorkspace ?? ((path) => rm(path, { recursive: true, force: true })); } - private selection(): ApktoolCommandSelection { - return resolveApktoolCommand(this.environment); + private operationEnvironment(): Readonly { + const ambient = snapshotEnvironment(process.env); + if (this.selectedEnvironment === undefined) return ambient; + // Child inheritance is sampled per operation. An explicit map keeps its + // launcher choice authoritative, including an omitted command selecting PATH. + return snapshotEnvironment({ + ...ambient, + ...this.selectedEnvironment, + [APKTOOL_COMMAND_ENV]: this.selectedEnvironment[APKTOOL_COMMAND_ENV], + }); } /** Probe the selected apktool launcher; decodes nothing. */ @@ -354,9 +367,14 @@ export class ApktoolProvider { async #inspectAvailability( signal: AbortSignal, ): Promise { + const environment = this.operationEnvironment(); try { - const selection = this.selection(); - await inspectApktoolClient(selection, signal, this.environment); + const selection = await resolveApktoolCommand( + environment, + process.platform, + signal, + ); + await inspectApktoolClient(selection, signal, environment); throwIfCancelled("inspect_apktool_client", signal); return { status: "available", @@ -406,17 +424,22 @@ export class ApktoolProvider { ): Promise> { if (signal?.aborted === true) return err(new AnalysisCancelledError(request.operation)); + const environment = this.operationEnvironment(); let selection: ApktoolCommandSelection; let identity: ApktoolClientIdentity; try { - selection = this.selection(); - identity = await inspectApktoolClient( - selection, + selection = await resolveApktoolCommand( + environment, + process.platform, signal, - this.environment, ); + identity = await inspectApktoolClient(selection, signal, environment); } catch (cause) { - if (cause instanceof AnalysisCancelledError) return err(cause); + if ( + cause instanceof AnalysisCancelledError || + cause instanceof AnalysisTimeoutError + ) + return err(cause); if (cause instanceof ApktoolConfigurationFailure) { return err( providerConfigurationRefusal( @@ -449,6 +472,7 @@ export class ApktoolProvider { return this.decodeResources( request.operation, selection, + environment, client, provider, request.input, @@ -500,6 +524,7 @@ export class ApktoolProvider { private async decodeResources( operation: "decode_android_resources", selection: ApktoolCommandSelection, + environment: Readonly, client: { command: string; command_source: string; @@ -628,7 +653,7 @@ export class ApktoolProvider { maxOutputBytes: DECODE_MAX_OUTPUT_BYTES, }, signal, - this.environment, + environment, (owner) => this.#pendingCommandOwners.set(owner, root), ); if (!decoded.ok) { diff --git a/src/application/AppBundleExecutable.test.ts b/src/application/AppBundleExecutable.test.ts new file mode 100644 index 000000000..942a00112 --- /dev/null +++ b/src/application/AppBundleExecutable.test.ts @@ -0,0 +1,34 @@ +import { mkdtemp, mkdir, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +import { describe, expect, it } from "vitest"; + +import { resolveAppBundleExecutable } from "./AppBundleExecutable.js"; + +describe("app bundle Info.plist magic", () => { + it("does not decode a high-bit fold of bplist as a binary Info.plist", async () => { + const root = await mkdtemp(join(tmpdir(), "rea-bplist-fold-")); + try { + const bundle = join(root, "Fixture.app"); + const plistPath = join(bundle, "Contents", "Info.plist"); + await mkdir(join(bundle, "Contents"), { recursive: true }); + const plist = Buffer.from("bplist00"); + plist[0] = (plist[0] ?? 0) | 0x80; + await writeFile(plistPath, plist); + let decoded = false; + const result = await resolveAppBundleExecutable(bundle, { + readFile: async () => plist, + realpath: async (path) => path, + decodeBinaryPlist: async () => { + decoded = true; + return "Fixture"; + }, + }); + expect(decoded).toBe(false); + expect(result.ok).toBe(false); + } finally { + await rm(root, { recursive: true, force: true }); + } + }); +}); diff --git a/src/application/BinaryTargetResolver.ts b/src/application/BinaryTargetResolver.ts index 6f1bc4366..b07364adf 100644 --- a/src/application/BinaryTargetResolver.ts +++ b/src/application/BinaryTargetResolver.ts @@ -162,6 +162,12 @@ export const parseBinaryTarget = async ( availableArchitectures: ["x86"], }); } + // Probe WASM content before suffix-only database routing; an explicit + // database selection keeps the caller's chosen interpretation. + const artifactFormat = + targetKind === "database" + ? undefined + : await detectArtifactFormat(path, handle, signal); if ( targetKind === "database" || (targetKind === undefined && path.toLowerCase().endsWith(".hop")) @@ -173,7 +179,6 @@ export const parseBinaryTarget = async ( kind: "database", format: "analysis-database", }); - const artifactFormat = await detectArtifactFormat(path, handle, signal); if (artifactFormat !== undefined) { const identity = { path, diff --git a/src/application/BrowserEvidence.ts b/src/application/BrowserEvidence.ts index d864fac9d..495f1f197 100644 --- a/src/application/BrowserEvidence.ts +++ b/src/application/BrowserEvidence.ts @@ -131,7 +131,7 @@ const browserParameters = ( } const observedOrigin = browserResultTargetOrigin(result); const scope = { - cdp_endpoint: input.cdp_endpoint, + cdp_endpoint: input.cdp_endpoint.recorded, allowed_origins: input.allowed_origins.length > 0 ? input.allowed_origins diff --git a/src/application/ClaudeDesktopConfigLocation.test.ts b/src/application/ClaudeDesktopConfigLocation.test.ts new file mode 100644 index 000000000..84d8ca8bc --- /dev/null +++ b/src/application/ClaudeDesktopConfigLocation.test.ts @@ -0,0 +1,341 @@ +import { basename, dirname, join, sep } from "node:path"; + +import { describe, expect, it } from "vitest"; + +import { + CLAUDE_DESKTOP_PACKAGE_FAMILY, + resolveWindowsClaudeDesktopLocation, + type ClaudeDesktopLocationFileSystem, +} from "./ClaudeDesktopConfigLocation.js"; +import { + packagePrivateLauncherBlocker, + packagePrivateStorageFamily, + redirectedWriteBlocker, + type MsixPackageView, +} from "./MsixPackageStorage.js"; + +const home = join("/profile", "a"); +const env = { + APPDATA: join(home, "AppData", "Roaming"), + LOCALAPPDATA: join(home, "AppData", "Local"), +}; +const packages = join(env.LOCALAPPDATA, "Packages"); +const appDataConfig = join(env.APPDATA, "Claude", "claude_desktop_config.json"); +const privateConfig = join( + packages, + CLAUDE_DESKTOP_PACKAGE_FAMILY, + "LocalCache", + "Roaming", + "Claude", + "claude_desktop_config.json", +); + +const failure = (code: string): Error => + Object.assign(new Error(`${code}: injected`), { code }); + +const file = { isFile: () => true }; +const privateDirectory = dirname(privateConfig); +const privatePreferences = join(privateDirectory, "Preferences"); +const appDataDirectory = dirname(appDataConfig); + +/** Package storage holding `privateFiles`; realpath decides the process view. */ +const packagedFileSystem = ( + realpath: ClaudeDesktopLocationFileSystem["realpath"], + privateFiles: readonly string[] = [privateConfig], +): ClaudeDesktopLocationFileSystem => ({ + readdir: (path) => { + if (path === packages) return [CLAUDE_DESKTOP_PACKAGE_FAMILY]; + if (path === privateDirectory) + return privateFiles.map((name) => basename(name)); + throw failure("ENOENT"); + }, + lstat: (path) => { + if (!privateFiles.includes(path)) throw failure("ENOENT"); + return file; + }, + realpath, +}); + +describe("Claude Desktop MSIX process view", () => { + it("recognizes a process that sees the private copy at the AppData path", () => { + const location = resolveWindowsClaudeDesktopLocation( + home, + env, + packagedFileSystem((path) => + path === appDataConfig ? privateConfig : path, + ), + ); + expect(location).toEqual({ + ok: true, + value: expect.objectContaining({ + configPath: privateConfig, + msixPackage: expect.objectContaining({ processView: "redirected" }), + }), + }); + }); + + it("treats an absent AppData file as an unredirected process", () => { + const location = resolveWindowsClaudeDesktopLocation( + home, + env, + packagedFileSystem((path) => { + if (path === appDataConfig) throw failure("ENOENT"); + return path; + }), + ); + expect(location.ok && location.value.msixPackage?.processView).toBe( + "direct", + ); + }); + + it("recognizes redirection through another private file before the configuration exists", () => { + const location = resolveWindowsClaudeDesktopLocation( + home, + env, + packagedFileSystem( + (path) => + path === join(appDataDirectory, "Preferences") + ? privatePreferences + : path, + [privatePreferences], + ), + ); + expect(location.ok && location.value.configPath).toBe(privateConfig); + expect(location.ok && location.value.msixPackage?.processView).toBe( + "redirected", + ); + }); + + it("continues after a listed private file disappears", () => { + const fs = packagedFileSystem( + (path) => + path === join(appDataDirectory, "Preferences") + ? privatePreferences + : path, + [privatePreferences], + ); + const location = resolveWindowsClaudeDesktopLocation(home, env, { + ...fs, + readdir: (path) => + path === privateDirectory + ? ["vanished", "Preferences"] + : fs.readdir(path), + }); + expect(location.ok && location.value.msixPackage?.processView).toBe( + "redirected", + ); + }); + + it("keeps the view unknown when the package has no private files", () => { + const location = resolveWindowsClaudeDesktopLocation( + home, + env, + packagedFileSystem((path) => path, []), + ); + expect(location.ok && location.value.msixPackage?.processView).toBe( + "unknown", + ); + }); + + it("keeps the view unknown when the AppData path cannot be resolved", () => { + const location = resolveWindowsClaudeDesktopLocation( + home, + env, + packagedFileSystem((path) => { + if (path === appDataConfig) throw failure("EACCES"); + return path; + }), + ); + expect(location.ok && location.value.msixPackage?.processView).toBe( + "unknown", + ); + }); +}); + +describe("unresolved Claude Desktop package identity", () => { + it("reports unreadable package storage instead of assuming AppData", () => { + const location = resolveWindowsClaudeDesktopLocation(home, env, { + readdir: () => { + throw failure("EACCES"); + }, + lstat: () => file, + realpath: (path) => path, + }); + expect(location).toEqual({ + ok: false, + error: { + path: packages, + message: expect.stringContaining("could not be inspected"), + }, + }); + }); + + it("reports an uninspectable private copy instead of choosing a file", () => { + const location = resolveWindowsClaudeDesktopLocation(home, env, { + readdir: () => [CLAUDE_DESKTOP_PACKAGE_FAMILY], + lstat: () => { + throw failure("EPERM"); + }, + realpath: (path) => path, + }); + expect(location.ok).toBe(false); + expect(!location.ok && location.error.path).toBe(privateConfig); + }); + + it("matches the package family case-insensitively, as Windows does", () => { + const location = resolveWindowsClaudeDesktopLocation(home, env, { + readdir: () => [CLAUDE_DESKTOP_PACKAGE_FAMILY.toUpperCase()], + lstat: () => { + throw failure("ENOENT"); + }, + realpath: (path) => path, + }); + expect(location.ok && location.value.msixPackage?.packageFamilyName).toBe( + CLAUDE_DESKTOP_PACKAGE_FAMILY.toUpperCase(), + ); + }); +}); + +describe("package-private REA launchers", () => { + const launcher = join( + env.APPDATA, + "npm", + "node_modules", + "rea-agents", + "scripts", + "rea.mjs", + ); + const stored = join( + packages, + CLAUDE_DESKTOP_PACKAGE_FAMILY, + "LocalCache", + "Roaming", + "npm", + "node_modules", + "rea-agents", + "scripts", + "rea.mjs", + ); + + it("names the package that privately stores a path", () => { + expect(packagePrivateStorageFamily(stored, packages)).toBe( + CLAUDE_DESKTOP_PACKAGE_FAMILY, + ); + expect(packagePrivateStorageFamily(launcher, packages)).toBeUndefined(); + expect( + packagePrivateStorageFamily( + join(packages, CLAUDE_DESKTOP_PACKAGE_FAMILY, "LocalState", "x"), + packages, + ), + ).toBeUndefined(); + }); + + it("blocks a launcher that only the package's processes can see", () => { + const blocker = packagePrivateLauncherBlocker( + [join("/Program Files", "nodejs", "node.exe"), launcher, "mcp"], + packages, + (path) => (path === launcher ? stored : path), + ); + expect(blocker).toContain(stored); + expect(blocker).toContain(CLAUDE_DESKTOP_PACKAGE_FAMILY); + expect(blocker).toContain("npm install -g rea-agents"); + }); + + it("compares the launcher with the canonical package storage path", () => { + const shortProfilePackages = join( + "/profile", + "A~1", + "AppData", + "Local", + "Packages", + ); + const blocker = packagePrivateLauncherBlocker( + [launcher, "mcp"], + shortProfilePackages, + (path) => + path === shortProfilePackages + ? packages + : path === launcher + ? stored + : path, + ); + expect(blocker).toContain(CLAUDE_DESKTOP_PACKAGE_FAMILY); + }); + + it("allows ordinary and unresolvable launchers", () => { + expect( + packagePrivateLauncherBlocker( + ["cmd.exe", "/d", "/c", "npx", "-y", "rea-agents@6.4.0", "mcp"], + packages, + () => { + throw new Error("package runners are not resolved"); + }, + ), + ).toBeUndefined(); + expect( + packagePrivateLauncherBlocker([launcher, "mcp"], packages, () => { + throw failure("ENOENT"); + }), + ).toBeUndefined(); + }); +}); + +describe("writes from a redirected AppData view", () => { + const view: MsixPackageView = { + packageFamilyName: CLAUDE_DESKTOP_PACKAGE_FAMILY, + privateConfigPath: privateConfig, + appDataConfigPath: appDataConfig, + virtualizedRoots: [env.APPDATA, env.LOCALAPPDATA], + processView: "redirected", + }; + const vscode = join(env.APPDATA, "Code", "User", "mcp.json"); + const hermes = join(env.LOCALAPPDATA, "hermes", "config.yaml"); + const codex = join(home, ".codex", "config.toml"); + + it("refuses only targets under the package's virtualized roots", () => { + const blocker = redirectedWriteBlocker( + view, + [vscode, codex, hermes], + "setup", + ); + expect(blocker).toContain(`${vscode}, ${hermes}`); + expect(blocker).not.toContain(codex); + expect(blocker).toContain("Rerun setup from a terminal"); + expect(redirectedWriteBlocker(view, [codex], "setup")).toBeUndefined(); + }); + + it("compares aliases through the existing parent of a new destination", () => { + const alias = join(home, "skill-alias"); + const target = join(alias, "skills", "reverse-engineer-anything"); + const blocker = redirectedWriteBlocker( + view, + [target], + "uninstall", + (path) => { + if (path === alias) return join(env.APPDATA, "dsh"); + if (path.startsWith(`${alias}${sep}`)) throw failure("ENOENT"); + return path; + }, + ); + expect(blocker).toContain(target); + expect(blocker).toContain("Rerun uninstall"); + }); + + it("keeps canonicalization failures visible instead of assuming a target is outside AppData", () => { + expect( + redirectedWriteBlocker(view, [codex], "setup", () => { + throw failure("EACCES"); + }), + ).toContain("Cannot establish the MSIX write location"); + }); + + it("allows every target without positive evidence of redirection", () => { + for (const processView of ["direct", "unknown"] as const) + expect( + redirectedWriteBlocker({ ...view, processView }, [vscode], "uninstall"), + ).toBeUndefined(); + expect( + redirectedWriteBlocker(undefined, [vscode], "setup"), + ).toBeUndefined(); + }); +}); diff --git a/src/application/ClaudeDesktopConfigLocation.ts b/src/application/ClaudeDesktopConfigLocation.ts new file mode 100644 index 000000000..899aadbbb --- /dev/null +++ b/src/application/ClaudeDesktopConfigLocation.ts @@ -0,0 +1,199 @@ +import { lstatSync, readdirSync, realpathSync } from "node:fs"; +import { join } from "node:path"; + +import { err, ok, type Result } from "../domain/result.js"; +import { + windowsAppDataRoots, + type MsixPackageView, +} from "./MsixPackageStorage.js"; + +const CONFIG_FILE = "claude_desktop_config.json"; + +/** Private files compared at most when looking for a redirected AppData view. */ +const PROCESS_VIEW_PROBES = 8; + +/** + * Claude Desktop's MSIX package family: package name `Claude` and the + * publisher ID Windows derives from `CN="Anthropic, PBC", O="Anthropic, PBC", + * L=San Francisco, S=California, C=US, …`. + */ +export const CLAUDE_DESKTOP_PACKAGE_FAMILY = "Claude_pzs8sxrjxfjjc"; + +/** `Claude_` plus a 13-character Crockford base32 MSIX publisher ID. */ +const CLAUDE_PACKAGE_FAMILY = /^Claude_[0-9a-hjkmnp-tv-z]{13}$/iu; + +/** The configuration Claude Desktop reads and the evidence that it is installed. */ +export interface ClaudeDesktopLocation { + readonly configPath: string; + readonly markerPath: string; + readonly msixPackage?: MsixPackageView; +} + +/** Unresolved Claude Desktop identity, reported instead of guessing a file. */ +export interface ClaudeDesktopLocationFailure { + readonly path: string; + readonly message: string; +} + +/** Synchronous probes; each throws for failures other than absence. */ +export interface ClaudeDesktopLocationFileSystem { + readonly readdir: (path: string) => readonly string[]; + readonly lstat: (path: string) => { isFile(): boolean }; + readonly realpath: (path: string) => string; +} + +const systemFileSystem: ClaudeDesktopLocationFileSystem = { + readdir: (path) => readdirSync(path), + lstat: (path) => lstatSync(path), + realpath: (path) => realpathSync.native(path), +}; + +/** + * Resolve the `claude_desktop_config.json` that Windows Claude Desktop reads. + * Without its MSIX package storage, that is the AppData file. With it, the + * package reads its private copy when one exists and the AppData file + * otherwise; a new registration goes to the private copy. A `Claude_*` + * package from another publisher leaves the reader unknown. + */ +export const resolveWindowsClaudeDesktopLocation = ( + home: string, + env: Readonly, + fileSystem: ClaudeDesktopLocationFileSystem = systemFileSystem, +): Result => { + const roots = windowsAppDataRoots(home, env); + const appDataDirectory = join(roots.roaming, "Claude"); + const appDataConfigPath = join(appDataDirectory, CONFIG_FILE); + const packagesDirectory = join(roots.local, "Packages"); + let families: readonly string[]; + try { + families = fileSystem + .readdir(packagesDirectory) + .filter((name) => CLAUDE_PACKAGE_FAMILY.test(name)); + } catch (cause: unknown) { + if (!isMissing(cause)) + return err({ + path: packagesDirectory, + message: `MSIX package storage ${packagesDirectory} could not be inspected, so REA cannot establish whether Claude Desktop reads ${appDataConfigPath} or a package-private copy: ${errorMessage(cause)}. Repair access to that directory, then rerun.`, + }); + families = []; + } + const foreign = families.filter( + (name) => + name.toLowerCase() !== CLAUDE_DESKTOP_PACKAGE_FAMILY.toLowerCase(), + ); + if (foreign.length > 0) + return err({ + path: join(packagesDirectory, foreign[0] ?? ""), + message: `MSIX package storage ${foreign.map((name) => join(packagesDirectory, name)).join(", ")} uses Claude Desktop's package name with an unrecognized publisher (Claude Desktop is ${CLAUDE_DESKTOP_PACKAGE_FAMILY}). REA cannot establish which ${CONFIG_FILE} that package reads and does not fall back to ${appDataConfigPath}. Remove the unrecognized package, or register REA in the configuration it reads.`, + }); + const family = families[0]; + if (family === undefined) + return ok({ configPath: appDataConfigPath, markerPath: appDataDirectory }); + const container = join(packagesDirectory, family); + const privateDirectory = join(container, "LocalCache", "Roaming", "Claude"); + const privateConfigPath = join(privateDirectory, CONFIG_FILE); + const privateCopy = present(fileSystem, privateConfigPath); + if (!privateCopy.ok) return privateCopy; + const appDataCopy = privateCopy.value + ? ok(false) + : present(fileSystem, appDataConfigPath); + if (!appDataCopy.ok) return appDataCopy; + return ok({ + configPath: + privateCopy.value || !appDataCopy.value + ? privateConfigPath + : appDataConfigPath, + markerPath: container, + msixPackage: { + packageFamilyName: family, + privateConfigPath, + appDataConfigPath, + virtualizedRoots: [roots.roaming, roots.local], + processView: processView( + fileSystem, + privateDirectory, + appDataDirectory, + privateCopy.value, + ), + }, + }); +}; + +/** + * A process the package started sees the package's private files at their + * AppData paths; any other process sees the real AppData files or nothing. + * Claude Desktop saves its user data there while it runs, so the + * configuration, or any other private file, can show which view this is. + */ +const processView = ( + fileSystem: ClaudeDesktopLocationFileSystem, + privateDirectory: string, + appDataDirectory: string, + privateConfig: boolean, +): MsixPackageView["processView"] => { + let probes: readonly string[]; + try { + probes = privateConfig + ? [CONFIG_FILE] + : fileSystem + .readdir(privateDirectory) + .filter((name) => { + try { + return fileSystem.lstat(join(privateDirectory, name)).isFile(); + } catch (cause: unknown) { + // A candidate may vanish or become unreadable while listed. + void cause; + return false; + } + }) + .slice(0, PROCESS_VIEW_PROBES); + } catch (cause: unknown) { + // No private file can be listed, so no view can be compared. + void cause; + return "unknown"; + } + let view: MsixPackageView["processView"] = "unknown"; + for (const name of probes) { + let privateTarget: string; + try { + privateTarget = fileSystem.realpath(join(privateDirectory, name)); + } catch (cause: unknown) { + // A private file that vanished cannot be compared; try another. + void cause; + continue; + } + try { + if (fileSystem.realpath(join(appDataDirectory, name)) === privateTarget) + return "redirected"; + view = "direct"; + } catch (cause: unknown) { + if (!isMissing(cause)) return "unknown"; + view = "direct"; + } + } + return view; +}; + +const present = ( + fileSystem: ClaudeDesktopLocationFileSystem, + path: string, +): Result => { + try { + fileSystem.lstat(path); + return ok(true); + } catch (cause: unknown) { + if (isMissing(cause)) return ok(false); + return err({ + path, + message: `${path} could not be inspected, so REA cannot establish which ${CONFIG_FILE} Claude Desktop's MSIX package reads: ${errorMessage(cause)}. Repair access to that path, then rerun.`, + }); + } +}; + +const isMissing = (cause: unknown): boolean => + cause instanceof Error && + "code" in cause && + (cause.code === "ENOENT" || cause.code === "ENOTDIR"); + +const errorMessage = (cause: unknown): string => + cause instanceof Error ? cause.message : String(cause); diff --git a/src/application/ClientConfigurationDocument.ts b/src/application/ClientConfigurationDocument.ts index 1b241a98b..a226f8cb6 100644 --- a/src/application/ClientConfigurationDocument.ts +++ b/src/application/ClientConfigurationDocument.ts @@ -110,9 +110,12 @@ const parseDocument = ( return requireObject(parseToml(text), "The configuration root"); // Pi loads `mcp.json` with a strict `JSON.parse` and no BOM tolerance, so // comments, trailing commas, a BOM, or an empty existing file must be - // rejected instead of registering into a document Pi cannot load. - if (format === "pi") + // rejected instead of registering into a document the client cannot load. + // Kimi also reads mcp.json with JSON.parse and an empty-file fallback. + if (format === "pi" || format === "kimi") { + if (format === "kimi" && text.trim().length === 0) return {}; return requireObject(JSON.parse(text), "The configuration root"); + } // An empty file, which some clients create before any server is added, // holds no settings to preserve. if (isEmptyClientConfigurationText(text)) return {}; @@ -1446,6 +1449,13 @@ export const clientRegistrationEntry = ( ? {} : { environment: { ...environment } }), }; + case "kimi": + // Kimi infers stdio from command; it uses `transport`, not `type`. + return { + command: executable, + args, + ...(Object.keys(environment).length === 0 ? {} : { env: environment }), + }; case "opencode_v2": // V2 connects servers unless `disabled` is true. return { diff --git a/src/application/ClientRegistrationStatus.ts b/src/application/ClientRegistrationStatus.ts index caeba4fb6..56a3e81ba 100644 --- a/src/application/ClientRegistrationStatus.ts +++ b/src/application/ClientRegistrationStatus.ts @@ -66,6 +66,8 @@ const registrationSchema = z command: z.string().min(1), args: z.array(z.string()).default([]), startup_timeout_sec: z.number().positive().optional(), + startupTimeoutMs: z.number().positive().optional(), + toolTimeoutMs: z.number().positive().optional(), type: z.string().optional(), transport: z.string().optional(), serverName: z.string().optional(), @@ -155,7 +157,7 @@ export const readClientRegistrationInspections = async ( } const parsed = effectiveClientConfiguration(files.value); if (parsed === undefined) { - record(unavailableStatus(client.name, client.configPath, "missing")); + record(unavailableStatus(client, "missing")); continue; } const policy = await readClientPolicyBlock( @@ -191,7 +193,7 @@ export const readClientRegistrationInspections = async ( }); continue; } - record(unavailableStatus(client.name, client.configPath, "missing")); + record(unavailableStatus(client, "missing")); continue; } const registration = parseRegistration(raw, client); @@ -202,8 +204,7 @@ export const readClientRegistrationInspections = async ( const activation = registrationActivation(registration, client, parsed); record( configuredStatus( - client.name, - client.configPath, + client, command, activation === "enabled" && registrationAligned( @@ -222,11 +223,7 @@ export const readClientRegistrationInspections = async ( ); } catch (cause: unknown) { record( - unavailableStatus( - client.name, - client.configPath, - isMissing(cause) ? "missing" : "invalid", - ), + unavailableStatus(client, isMissing(cause) ? "missing" : "invalid"), ); } } @@ -263,6 +260,21 @@ const registrationAligned = ( MCP_STARTUP_POLICY.codexStartupTimeoutSeconds ) return false; + if ( + client.name === "kimi" && + (registration.startupTimeoutMs !== + MCP_STARTUP_POLICY.kimiStartupTimeoutMs || + registration.toolTimeoutMs !== MCP_STARTUP_POLICY.kimiToolTimeoutMs) + ) + return false; + if ( + (client.format === "kimi" || + client.format === "omp" || + client.format === "pi") && + registration.type !== undefined && + registration.type !== "stdio" + ) + return false; if (client.format === "vscode" && registration.type !== "stdio") return false; // A Harness registration only connects as the rea stdio server; another // server name or transport leaves the session without REA tools. @@ -271,13 +283,6 @@ const registrationAligned = ( (registration.serverName !== "rea" || registration.transport !== "stdio") ) return false; - // OMP and Pi infer stdio for a command entry without an explicit type. - if ( - (client.format === "omp" || client.format === "pi") && - registration.type !== undefined && - registration.type !== "stdio" - ) - return false; if ( client.format === "copilot_cli" && (registration.type !== "stdio" || @@ -336,6 +341,12 @@ const parseRegistration = ( }); } const registration = registrationSchema.parse(value); + if ( + client.format === "kimi" && + registration.transport !== undefined && + registration.transport !== "stdio" + ) + throw new TypeError("Expected an stdio registration"); if ( (client.format === "vscode" || client.format === "copilot_cli") && registration.type !== "stdio" @@ -344,7 +355,9 @@ const parseRegistration = ( if (client.format === "commandcode" && registration.transport !== "stdio") throw new TypeError("Expected an stdio registration"); if ( - (client.format === "omp" || client.format === "pi") && + (client.format === "kimi" || + client.format === "omp" || + client.format === "pi") && registration.type !== undefined && registration.type !== "stdio" ) @@ -352,36 +365,45 @@ const parseRegistration = ( return registration; }; -const remediation = - "Run rea setup to refresh this registration, then restart the client."; +/** A client that rewrites its file from memory must be quit before setup. */ +const remediation = (client: SetupClient): string => { + if (client.quitBeforeSetup !== true) + return "Run rea setup to refresh this registration, then restart the client."; + const name = client.displayName ?? client.name; + return `Quit ${name} fully, run rea setup to refresh this registration, then start ${name} again. A running ${name} can rewrite its configuration from memory and drop the registration.`; +}; const configuredStatus = ( - client: string, - configPath: string, + client: SetupClient, command: RegistrationCommand, state: "aligned" | "stale", policyRemediation?: string, ): ClientRegistrationStatus => state === "aligned" - ? { client, config_path: configPath, command, state, remediation: null } - : { - client, - config_path: configPath, + ? { + client: client.name, + config_path: client.configPath, command, state, - remediation: policyRemediation ?? remediation, + remediation: null, + } + : { + client: client.name, + config_path: client.configPath, + command, + state, + remediation: policyRemediation ?? remediation(client), }; const unavailableStatus = ( - client: string, - configPath: string, + client: SetupClient, state: "missing" | "invalid", ): ClientRegistrationStatus => ({ - client, - config_path: configPath, + client: client.name, + config_path: client.configPath, command: [], state, - remediation, + remediation: remediation(client), }); const clientEvidencePresent = async (client: SetupClient): Promise => { diff --git a/src/application/DirectAnalysis.ts b/src/application/DirectAnalysis.ts index 0bdb69864..8a7874598 100644 --- a/src/application/DirectAnalysis.ts +++ b/src/application/DirectAnalysis.ts @@ -60,6 +60,7 @@ type DirectAnalysisTool = | "annotate_native_function" | "inspect_native_load_image" | "read_bytes" + | "search_bytes" | "address_to_file_offset" | "binary_overview" | "procedure_pseudo_code" diff --git a/src/application/Doctor.test.ts b/src/application/Doctor.test.ts index 4251bbaf2..a39f81e35 100644 --- a/src/application/Doctor.test.ts +++ b/src/application/Doctor.test.ts @@ -74,7 +74,7 @@ describe("doctor", () => { ), ).toEqual({ node: `Install ${SUPPORTED_NODE_VERSION_PROSE}.`, - host: "REA supports macOS 12+, Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, CachyOS, Omarchy, and the experimental Windows x64 Ghidra P0 boundary.", + host: "REA supports macOS 12+, Linux distributions with a recognized apt, dnf, or pacman package family, and the experimental Windows x64 Ghidra P0 boundary; automated Hopper setup additionally requires Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, CachyOS, or Omarchy.", hopper: "Run rea setup to install Hopper, or set HOPPER_LAUNCHER_PATH.", target: "Supply a readable local app or program path.", }); @@ -418,10 +418,9 @@ describe("doctor scoped readiness", () => { platform: "linux", linuxDistribution: () => Promise.resolve({ - id: "debian", - versionId: "13", - packageFamily: "deb", - supported: false, + id: "nixos", + packageFamily: undefined, + hopperSetupSupported: false, }), clientRegistrations: () => Promise.resolve([ @@ -451,6 +450,31 @@ describe("doctor scoped readiness", () => { ); }); + it("accepts a Debian-family host that has no tested Hopper package", async () => { + const result = await runDoctor( + undefined, + host({ + platform: "linux", + macosVersion: () => Promise.resolve(undefined), + linuxDistribution: () => + Promise.resolve({ + id: "kali", + versionId: "2026.3", + packageFamily: "deb", + hopperSetupSupported: false, + }), + }), + ); + + expect(result.checks).toContainEqual( + expect.objectContaining({ + name: "host", + ok: true, + detail: "kali 2026.3", + }), + ); + }); + it("requires explicitly selected providers instead of any available provider", async () => { const ghidra = { id: "ghidra", @@ -502,7 +526,7 @@ describe("doctor Linux and Hopper discovery", () => { id: "ubuntu", versionId: "24.04", packageFamily: "deb", - supported: true, + hopperSetupSupported: true, }), configuredHopperPath: path, executable: (candidate) => Promise.resolve(candidate === path), @@ -522,7 +546,7 @@ describe("doctor Linux and Hopper discovery", () => { id: "ubuntu", versionId: "24.04", packageFamily: "deb", - supported: true, + hopperSetupSupported: true, }), linuxDemoRuntimeCheck: () => Promise.resolve({ @@ -554,7 +578,7 @@ describe("doctor Linux and Hopper discovery", () => { id: "ubuntu", versionId: "24.04", packageFamily: "deb", - supported: true, + hopperSetupSupported: true, }), executable: (candidate) => Promise.resolve(candidate === path), supportedLinuxHopper: () => Promise.resolve(false), diff --git a/src/application/DoctorDiagnostics.ts b/src/application/DoctorDiagnostics.ts index 354d08636..1d8a5e621 100644 --- a/src/application/DoctorDiagnostics.ts +++ b/src/application/DoctorDiagnostics.ts @@ -2,7 +2,7 @@ import { SUPPORTED_NODE_VERSION_PROSE, supportsNodeVersion, } from "../domain/runtimeVersion.js"; -import { linuxHopperLauncherPath } from "./LinuxHopper.js"; +import { linuxHopperLauncherPath, supportsReaHost } from "./LinuxHopper.js"; import type { DoctorCheck, DoctorHost, @@ -129,7 +129,7 @@ const hostCheck = async (host: DoctorHost): Promise => { host.platform === "linux" ? await host.linuxDistribution() : undefined; const supported = (host.platform === "darwin" && parseMajor(macosVersion) >= 12) || - (host.platform === "linux" && linuxDistribution?.supported === true) || + (host.platform === "linux" && supportsReaHost(linuxDistribution)) || (host.platform === "win32" && host.architecture === "x64"); const detail = macosVersion ?? @@ -138,7 +138,7 @@ const hostCheck = async (host: DoctorHost): Promise => { : `${linuxDistribution.id} ${linuxDistribution.versionId ?? "unknown"}`); return check("host", supported, detail, { remediation: - "REA supports macOS 12+, Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, CachyOS, Omarchy, and the experimental Windows x64 Ghidra P0 boundary.", + "REA supports macOS 12+, Linux distributions with a recognized apt, dnf, or pacman package family, and the experimental Windows x64 Ghidra P0 boundary; automated Hopper setup additionally requires Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, CachyOS, or Omarchy.", classification: "unsupported_host", }); }; diff --git a/src/application/DshCordisPatch.ts b/src/application/DshCordisPatch.ts index 5e0fc10ec..0e34da23e 100644 --- a/src/application/DshCordisPatch.ts +++ b/src/application/DshCordisPatch.ts @@ -8,6 +8,7 @@ import { CordisExpression, DshCordisYaml } from "./DshCordisYaml.js"; export const DSH_ENTRY_ID = "rea-mcp"; export const DSH_CLIENT_PACKAGE = "@deepseek-ai/dsh-mcp-client"; const LAUNCHER_KEYS = new Set(["serverName", "transport", "command", "args"]); +const SELECTOR_KEYS = new Set(["id", "name"]); const isRecord = (value: unknown): value is Record => typeof value === "object" && value !== null && @@ -295,10 +296,15 @@ export class DshCordisDocument { carried = true; return [this.#entry(desired, invalidated)] as typeof inserted.items; }); - replacements.set( - op.source, - inserted.items.length === 0 ? undefined : copy, - ); + if (inserted.items.length === 0) copy.delete("insert"); + // An emptied operation keeps caller fields unless it would patch REA's row. + const kept = + inserted.items.length > 0 || + (this.#id(op.map) !== DSH_ENTRY_ID && + copy.items.some( + (pair) => !SELECTOR_KEYS.has(this.#yaml.value(pair.key) as string), + )); + replacements.set(op.source, kept ? copy : undefined); } const output = this.#yaml.document.clone(); const outputRoot = output.contents; diff --git a/src/application/EvidenceBundleCommands.ts b/src/application/EvidenceBundleCommands.ts index a72d17cb5..1255ea68f 100644 --- a/src/application/EvidenceBundleCommands.ts +++ b/src/application/EvidenceBundleCommands.ts @@ -11,6 +11,7 @@ import { compareBundles } from "../domain/bundleComparison.js"; import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; import { readEvidenceBundle, + readImmutableEvidenceBundle, writeEvidenceBundle, } from "./EvidenceBundleFiles.js"; @@ -33,7 +34,8 @@ export const exportEvidenceBundleCommand = async ( outputPath: string, overwrite: boolean, ): Promise> => { - const loaded = await readEvidenceBundle(sourcePath); + // The sealed snapshot lets the writer reuse this authentication. + const loaded = await readImmutableEvidenceBundle(sourcePath); if (!loaded.ok) return loaded; const resources = new ArtifactResourceScope(); const written = await writeEvidenceBundle( @@ -69,9 +71,11 @@ export const compareEvidenceBundlesCommand = async (input: { readonly leftPath: string; readonly rightPath: string; }): Promise> => { + // Sealed snapshots let comparison and its bundle digests reuse this + // authentication instead of re-validating every record twice more. const [left, right] = await Promise.all([ - readEvidenceBundle(input.leftPath), - readEvidenceBundle(input.rightPath), + readImmutableEvidenceBundle(input.leftPath), + readImmutableEvidenceBundle(input.rightPath), ]); if (!left.ok) return left; if (!right.ok) return right; diff --git a/src/application/LinuxHopper.test.ts b/src/application/LinuxHopper.test.ts index b5cf2d38f..1c37a03ea 100644 --- a/src/application/LinuxHopper.test.ts +++ b/src/application/LinuxHopper.test.ts @@ -7,6 +7,7 @@ import { linuxPackageManagerCommands, linuxSharedLibrariesAvailable, parseLinuxDistribution, + supportsReaHost, type LinuxDistribution, type LinuxHopperDownload, type LinuxHopperInstallHost, @@ -19,7 +20,7 @@ class RecordingLinuxHost implements LinuxHopperInstallHost { id: "ubuntu", versionId: "24.04", packageFamily: "deb", - supported: true, + hopperSetupSupported: true, }; archive = new Uint8Array([1, 2, 3]); archiveOk = true; @@ -66,7 +67,7 @@ describe("Linux Hopper host classification", () => { (document, family) => { expect(parseLinuxDistribution(document)).toMatchObject({ packageFamily: family, - supported: true, + hopperSetupSupported: true, }); }, ); @@ -99,8 +100,32 @@ describe("Linux Hopper host classification", () => { "ID=garuda\nID_LIKE=arch\n", "ID=omarchy\n", "ID=omarchy\nID_LIKE=debian\n", - ])("rejects unsupported vendor/version combinations", (document) => { - expect(parseLinuxDistribution(document).supported).toBe(false); + ])( + "rejects vendor/version combinations without a tested Hopper installer", + (document) => { + expect(parseLinuxDistribution(document).hopperSetupSupported).toBe(false); + }, + ); + + it.each([ + 'ID=kali\nVERSION_ID="2026.3"\nID_LIKE=debian\n', + 'ID=debian\nVERSION_ID="13"\nID_LIKE=debian\n', + 'ID=linuxmint\nVERSION_ID="22.1"\nID_LIKE="ubuntu debian"\n', + "ID=manjaro\nID_LIKE=arch\n", + "ID=derivative\nID_LIKE='debian'\n", + "ID='debian'\nVERSION_ID='13'\n", + ])( + "hosts REA on a distribution Hopper has no tested package for: %s", + (document) => { + const distribution = parseLinuxDistribution(document); + expect(distribution.hopperSetupSupported).toBe(false); + expect(supportsReaHost(distribution)).toBe(true); + }, + ); + + it("withholds host support from a distribution with no package family", () => { + expect(supportsReaHost(parseLinuxDistribution("ID=nixos\n"))).toBe(false); + expect(supportsReaHost(undefined)).toBe(false); }); }); @@ -166,7 +191,7 @@ describe("Linux Hopper installation", () => { host.distributionValue = { id: family, packageFamily: family, - supported: true, + hopperSetupSupported: true, }; const result = await installFixture(host); expect(result).toEqual({ @@ -194,7 +219,7 @@ describe("Linux Hopper installation", () => { host.distributionValue = { id: "debian", packageFamily: "deb", - supported: false, + hopperSetupSupported: false, }; if (scenario === "package request failure") host.archiveOk = false; if (scenario === "checksum mismatch") host.archive = new Uint8Array([9]); diff --git a/src/application/LinuxHopper.ts b/src/application/LinuxHopper.ts index 3a46106bb..765f31d29 100644 --- a/src/application/LinuxHopper.ts +++ b/src/application/LinuxHopper.ts @@ -76,19 +76,35 @@ interface LinuxDistributionIdentity { readonly versionId?: string; } -/** Parsed Linux distribution information used for support and package selection. */ +/** + * Parsed Linux distribution information used for package selection. + * + * `hopperSetupSupported` is the automated Hopper installer's tested list. It is + * narrower than the hosts where REA itself runs, which {@link supportsReaHost} + * answers from the package family. + */ export type LinuxDistribution = LinuxDistributionIdentity & ( | { readonly packageFamily: LinuxPackageFamily; - readonly supported: true; + readonly hopperSetupSupported: true; } | { readonly packageFamily: LinuxPackageFamily | undefined; - readonly supported: false; + readonly hopperSetupSupported: false; } ); +/** + * Whether REA's analysis providers run on this distribution. A recognized + * package family means a glibc host with a known package layout, which is what + * the providers need; downloading and installing the official Hopper package + * additionally requires a distribution on its tested list. + */ +export const supportsReaHost = ( + distribution: LinuxDistribution | undefined, +): boolean => distribution?.packageFamily !== undefined; + /** Download response kept independent of Node's global fetch implementation. */ export interface LinuxHopperDownload { readonly ok: boolean; @@ -141,16 +157,19 @@ export const parseLinuxDistribution = (text: string): LinuxDistribution => { const key = match[1]; const encoded = match[2]; if (key === undefined || encoded === undefined) continue; - const value = - encoded.startsWith('"') && encoded.endsWith('"') - ? encoded.slice(1, -1).replaceAll('\\"', '"').replaceAll("\\\\", "\\") - : encoded; + const quote = encoded[0]; + const quoted = (quote === "'" || quote === '"') && encoded.endsWith(quote); + const value = !quoted + ? encoded + : quote === "'" + ? encoded.slice(1, -1) + : encoded.slice(1, -1).replace(/\\(["\\$`])/gu, "$1"); fields.set(key, value); } const id = (fields.get("ID") ?? "unknown").toLowerCase(); const versionId = fields.get("VERSION_ID"); const packageFamily = packageFamilyFor(id, fields.get("ID_LIKE")); - const supported = + const hopperSetupSupported = packageFamily !== undefined && (id === "arch" || id === "cachyos" || @@ -162,9 +181,9 @@ export const parseLinuxDistribution = (text: string): LinuxDistribution => { id, ...(versionId === undefined ? {} : { versionId }), }; - return supported - ? { ...identity, packageFamily, supported: true } - : { ...identity, packageFamily, supported: false }; + return hopperSetupSupported + ? { ...identity, packageFamily, hopperSetupSupported: true } + : { ...identity, packageFamily, hopperSetupSupported: false }; }; /** Read and classify the current Linux distribution. */ @@ -193,7 +212,7 @@ export const installLinuxHopper = async ( if (options.signal?.aborted === true) return { status: "failed", reason: "cancelled" }; const distribution = await host.distribution(); - if (distribution?.supported !== true) + if (distribution?.hopperSetupSupported !== true) return { status: "failed", reason: "unsupported_host" }; let temporary: string | undefined; try { diff --git a/src/application/MsixPackageStorage.ts b/src/application/MsixPackageStorage.ts new file mode 100644 index 000000000..098b2f1db --- /dev/null +++ b/src/application/MsixPackageStorage.ts @@ -0,0 +1,187 @@ +import { lstat } from "node:fs/promises"; +import { realpathSync } from "node:fs"; +import { basename, dirname, isAbsolute, join, relative, sep } from "node:path"; +import { err, ok, type Result } from "../domain/result.js"; + +/** + * How one MSIX-packaged client reads its AppData configuration. A package + * with write virtualization saves AppData files to its private + * `LocalCache\Roaming` copy, then reads that copy whenever it exists and the + * ordinary AppData file otherwise. + */ +export interface MsixPackageView { + readonly packageFamilyName: string; + /** The package-private copy, which shadows the AppData file once it exists. */ + readonly privateConfigPath: string; + /** The AppData path that unpackaged processes read and write. */ + readonly appDataConfigPath: string; + /** Roots whose writes the package redirects for processes it starts. */ + readonly virtualizedRoots: readonly string[]; + /** + * `redirected`: this process sees one of the package's private files at its + * AppData path, so the running package started it. `direct`: it does not. + * `unknown`: no private file could be compared. + */ + readonly processView: "redirected" | "direct" | "unknown"; +} + +/** Windows' roaming and local application data roots; empty means unset. */ +export const windowsAppDataRoots = ( + home: string, + env: Readonly, +): { readonly roaming: string; readonly local: string } => ({ + roaming: env.APPDATA?.trim() || join(home, "AppData", "Roaming"), + local: env.LOCALAPPDATA?.trim() || join(home, "AppData", "Local"), +}); + +/** The per-user root of MSIX package data containers. */ +export const msixPackagesDirectory = ( + home: string, + env: Readonly, +): string => join(windowsAppDataRoots(home, env).local, "Packages"); + +/** Whether `path` is `root` or inside it, using the host's path comparison. */ +const pathWithin = (path: string, root: string): boolean => { + const inner = relative(root, path); + return ( + inner === "" || + (inner !== ".." && !inner.startsWith(`..${sep}`) && !isAbsolute(inner)) + ); +}; + +/** + * Name the package whose private `LocalCache` holds a canonical path. Its + * processes see those files at virtual AppData locations; other processes do + * not, and Windows removes them with the package. + */ +export const packagePrivateStorageFamily = ( + canonicalPath: string, + packagesDirectory: string, +): string | undefined => { + if (!pathWithin(canonicalPath, packagesDirectory)) return undefined; + const [family, storage] = relative(packagesDirectory, canonicalPath).split( + /[\\/]/u, + ); + return family !== undefined && + family !== "" && + storage?.toLowerCase() === "localcache" + ? family + : undefined; +}; + +/** + * Explain why setup must not register a launcher that exists only in one + * package's private storage. Other clients could not start it, and doctor + * outside that package could not verify it. + */ +export const packagePrivateLauncherBlocker = ( + command: readonly string[], + packagesDirectory: string, + canonicalPath: (path: string) => string = realpathSync.native, +): string | undefined => { + // Compare canonical forms: a short-name, junctioned or substituted profile + // spells the same directory differently. + let packages = packagesDirectory; + try { + packages = canonicalPath(packagesDirectory); + } catch (cause: unknown) { + // Without package storage, the spelled path is the only comparison. + void cause; + } + for (const part of command) { + if (!isAbsolute(part)) continue; + let resolved: string; + try { + resolved = canonicalPath(part); + } catch (cause: unknown) { + // A launcher that cannot be resolved is not evidence of package storage. + void cause; + continue; + } + const family = packagePrivateStorageFamily(resolved, packages); + if (family !== undefined) + return `The REA launcher ${part} is stored at ${resolved}, inside the private storage of MSIX package ${family}. Only processes started by that app can see it at ${part}, and Windows removes it with the package, so other clients could not start this registration. Install REA from a terminal that the app did not start (for example with \`npm install -g rea-agents\`), then rerun setup from that terminal.`; + } + return undefined; +}; + +/** + * Explain why this process must not write `targets`: the running package + * started it, so writes under its virtualized roots land in that package's + * private storage, which the targets' owners never read. + */ +const canonicalDestination = ( + path: string, + canonicalPath: (path: string) => string, +): Result => { + let ancestor = path; + const missing: string[] = []; + while (true) { + try { + return ok(join(canonicalPath(ancestor), ...missing)); + } catch (cause: unknown) { + if ( + cause instanceof Error && + "code" in cause && + (cause.code === "ENOENT" || cause.code === "ENOTDIR") && + dirname(ancestor) !== ancestor + ) { + missing.unshift(basename(ancestor)); + ancestor = dirname(ancestor); + continue; + } + return err({ + path, + detail: cause instanceof Error ? cause.message : String(cause), + }); + } + } +}; + +export const redirectedWriteBlocker = ( + view: MsixPackageView | undefined, + targets: readonly string[], + operation: "setup" | "uninstall", + canonicalPath: (path: string) => string = realpathSync.native, +): string | undefined => { + if (view?.processView !== "redirected" || targets.length === 0) + return undefined; + const roots: string[] = []; + const redirected: string[] = []; + for (const root of view.virtualizedRoots) { + const resolved = canonicalDestination(root, canonicalPath); + if (!resolved.ok) + return `Cannot establish the MSIX write location for ${resolved.error.path}: ${resolved.error.detail}. Rerun ${operation} from a terminal that this app did not start.`; + roots.push(resolved.value); + } + for (const target of targets) { + const resolved = canonicalDestination(target, canonicalPath); + if (!resolved.ok) + return `Cannot establish the MSIX write location for ${resolved.error.path}: ${resolved.error.detail}. Rerun ${operation} from a terminal that this app did not start.`; + if (roots.some((root) => pathWithin(resolved.value, root))) + redirected.push(target); + } + if (redirected.length === 0) return undefined; + return `This process was started by MSIX package ${view.packageFamilyName}, which redirects its writes under ${view.virtualizedRoots.join(" and ")} into the package's private storage. Changes to ${redirected.join(", ")} would not reach the files other apps read. Rerun ${operation} from a terminal that this app did not start.`; +}; + +/** + * Whether the package began reading its private copy after the AppData file + * was chosen, for example by saving its settings when it quit. + */ +export const packageReadTargetChanged = async ( + view: MsixPackageView, + configPath: string, +): Promise => { + if (configPath !== view.appDataConfigPath) return false; + try { + await lstat(view.privateConfigPath); + return true; + } catch (cause: unknown) { + return !( + cause instanceof Error && + "code" in cause && + (cause.code === "ENOENT" || cause.code === "ENOTDIR") + ); + } +}; diff --git a/src/application/ProviderOperationExecution.ts b/src/application/ProviderOperationExecution.ts index bf4ad4b0e..966a43f29 100644 --- a/src/application/ProviderOperationExecution.ts +++ b/src/application/ProviderOperationExecution.ts @@ -43,7 +43,7 @@ export const executeProviderOperation = async < provider: ProviderOperationPort, confidence: "observed" | "derived", ): Promise> => { - if (options?.signal?.aborted === true) + if (options?.signal?.aborted) return err(new AnalysisCancelledError(operation)); const request = schema.safeParse({ operation, input }); if (!request.success) @@ -51,8 +51,10 @@ export const executeProviderOperation = async < const executed = await provider.execute(request.data, options); if (!executed.ok) return executed; const execution = executed.value; - return ok( - createEvidence(execution.subject ?? undefined, execution.provider, { + const evidence = createEvidence( + execution.subject ?? undefined, + execution.provider, + { operation, parameters: jsonObjectSchema.parse(request.data.input), result: execution.result, @@ -60,6 +62,16 @@ export const executeProviderOperation = async < limitations: execution.limitations, locations: execution.locations, confidence, - }), + }, ); + // A provider that cannot observe the signal may finish its work after the + // caller cancelled, so re-check rather than reporting a completed operation + // the caller no longer wants. The observation it completed is still this + // provider's Evidence, so the cancellation carries it as the partial + // observation instead of discarding a finished result. + if (options?.signal?.aborted) + return err( + new AnalysisCancelledError(operation, { partialObservation: evidence }), + ); + return ok(evidence); }; diff --git a/src/application/Setup.configuration.test.ts b/src/application/Setup.configuration.test.ts index 4ba11f622..a3f46ed6f 100644 --- a/src/application/Setup.configuration.test.ts +++ b/src/application/Setup.configuration.test.ts @@ -1,9 +1,43 @@ import { describe, expect, it } from "vitest"; -import { resolve } from "node:path"; +import { join, resolve } from "node:path"; import { FakeSetupHost, options } from "./Setup.fixture.js"; import { runSetup } from "./Setup.js"; import { setupRegistrationCommand } from "./SetupHost.js"; +import type { MsixPackageView } from "./MsixPackageStorage.js"; +import type { SetupClient } from "./SupportedClients.js"; + +// Host joins keep containment checks meaningful on POSIX test hosts too. +const PROFILE = String.raw`C:\Users\a`; +const ROAMING = join(PROFILE, "AppData", "Roaming"); +const LOCAL = join(PROFILE, "AppData", "Local"); +const PRIVATE_CONFIG = join( + LOCAL, + "Packages", + "Claude_pzs8sxrjxfjjc", + "LocalCache", + "Roaming", + "Claude", + "claude_desktop_config.json", +); +const APPDATA_CONFIG = join(ROAMING, "Claude", "claude_desktop_config.json"); + +/** Claude Desktop as setup resolves it from its MSIX package storage. */ +const msixClaudeDesktop = ( + processView: MsixPackageView["processView"], +): SetupClient => ({ + name: "claude_desktop", + displayName: "Claude Desktop", + configPath: PRIVATE_CONFIG, + quitBeforeSetup: true, + msixPackage: { + packageFamilyName: "Claude_pzs8sxrjxfjjc", + privateConfigPath: PRIVATE_CONFIG, + appDataConfigPath: APPDATA_CONFIG, + virtualizedRoots: [ROAMING, LOCAL], + processView, + }, +}); describe("setup workflow", () => { it("omits an aligned managed skill from an otherwise empty plan", async () => { @@ -23,7 +57,7 @@ describe("setup workflow", () => { id: "ubuntu", versionId: "24.04", packageFamily: "deb", - supported: true, + hopperSetupSupported: true, }; host.hopper = "/opt/hopper/bin/Hopper"; host.linuxDemoRuntimeMissing = true; @@ -45,7 +79,7 @@ describe("setup workflow", () => { id: "ubuntu", versionId: "24.04", packageFamily: "deb", - supported: true, + hopperSetupSupported: true, }; host.hopper = "/custom/Hopper"; host.unsupportedHopperVersion = true; @@ -293,3 +327,85 @@ describe("setup workflow action selection", () => { expect(result.remediation).not.toContain("internal_client_key"); }); }); + +describe("MSIX Claude Desktop setup", () => { + it("discloses which file MSIX Claude Desktop reads and that it must be quit", async () => { + const host = new FakeSetupHost("win32"); + host.hopper = join(PROFILE, "Hopper"); + host.skill = "unchanged"; + host.clients = [msixClaudeDesktop("direct")]; + + const result = await runSetup( + { ...options(false), clientIds: ["claude_desktop"], dryRun: true }, + host, + ); + + expect(result.status).toBe("planned"); + const [action] = result.plannedActions; + expect(action?.target).toBe(PRIVATE_CONFIG); + expect(action?.detail).toContain( + `Claude Desktop is installed as MSIX package Claude_pzs8sxrjxfjjc, which reads this package-private file instead of ${APPDATA_CONFIG}.`, + ); + expect(action?.detail).toContain( + "Quit Claude Desktop fully before approving; closing its window is not enough.", + ); + }); + + it("refuses AppData writes from a process the MSIX package started", async () => { + const host = new FakeSetupHost("win32"); + host.hopper = join(PROFILE, "Hopper"); + host.skill = "unchanged"; + const vscode = join(ROAMING, "Code", "User", "mcp.json"); + host.clients = [ + msixClaudeDesktop("redirected"), + { name: "vscode", displayName: "VS Code", configPath: vscode }, + { + name: "codex", + displayName: "Codex", + configPath: join(PROFILE, ".codex", "config.toml"), + }, + ]; + + const blocked = await runSetup( + { ...options(true), clientIds: ["vscode", "codex"] }, + host, + ); + expect(blocked.status).toBe("needs_human"); + expect(blocked.remediation).toContain( + `Changes to ${vscode} would not reach the files other apps read.`, + ); + expect(host.configurations).toBe(0); + + const homeOnly = await runSetup( + { ...options(true), clientIds: ["codex"] }, + host, + ); + expect(homeOnly.status).toBe("ready"); + expect(host.configurations).toBe(1); + }); + + it("refuses registrations whose launcher only one MSIX package can see", async () => { + const host = new FakeSetupHost("win32"); + host.hopper = join(PROFILE, "Hopper"); + host.registrationCommandBlocker = "Install REA outside the package."; + host.clients = [ + { name: "codex", displayName: "Codex", configPath: "/codex.toml" }, + ]; + + const blocked = await runSetup( + { ...options(true), clientIds: ["codex"] }, + host, + ); + expect(blocked.status).toBe("needs_human"); + expect(blocked.remediation).toBe("Install REA outside the package."); + expect(host.configurations).toBe(0); + expect(host.skillInstalls).toBe(0); + + const skillOnly = await runSetup( + { ...options(true), clientIds: [], installSkill: true }, + host, + ); + expect(skillOnly.status).toBe("ready"); + expect(host.skillInstalls).toBe(1); + }); +}); diff --git a/src/application/Setup.fixture.ts b/src/application/Setup.fixture.ts index a99897494..3c7c21d35 100644 --- a/src/application/Setup.fixture.ts +++ b/src/application/Setup.fixture.ts @@ -25,6 +25,7 @@ export class FakeSetupHost implements SetupHost { return skillDestinations(this.homeDirectory, clientIds); } readonly registrationCommand: readonly string[]; + registrationCommandBlocker?: string; nodeVersion = "24.18.0"; version: string | undefined = "14.5"; distribution: LinuxDistribution | undefined; diff --git a/src/application/Setup.planning.test.ts b/src/application/Setup.planning.test.ts index d1ca6baea..2548772c9 100644 --- a/src/application/Setup.planning.test.ts +++ b/src/application/Setup.planning.test.ts @@ -70,7 +70,7 @@ describe("setup workflow", () => { id: "ubuntu", versionId: "24.04", packageFamily: "deb", - supported: true, + hopperSetupSupported: true, }; const result = await runSetup( { ...options(false), structured: false }, @@ -162,7 +162,7 @@ describe("setup workflow", () => { id: "ubuntu", versionId: "24.04", packageFamily: "deb", - supported: true, + hopperSetupSupported: true, }; const result = await runSetup(options(true, true), host); expect(result.appliedActions).toEqual(["installed_hopper"]); @@ -236,7 +236,7 @@ describe("setup workflow lifecycle and provider planning", () => { id: "ubuntu", versionId: "24.04", packageFamily: "deb", - supported: true, + hopperSetupSupported: true, }; host.ghidra = "/opt/ghidra_12.1.4_PUBLIC"; host.javaHome = "/usr/lib/jvm/jdk-21"; diff --git a/src/application/Setup.recovery.test.ts b/src/application/Setup.recovery.test.ts index f29c9d375..244aee26b 100644 --- a/src/application/Setup.recovery.test.ts +++ b/src/application/Setup.recovery.test.ts @@ -310,7 +310,7 @@ describe("setup scoped readiness", () => { id: "debian", versionId: "13", packageFamily: "deb", - supported: false, + hopperSetupSupported: false, }; const result = await runSetup(options(true, true), host); expect(result.status).toBe("needs_human"); @@ -326,7 +326,7 @@ describe("setup scoped readiness", () => { id: "debian", versionId: "13", packageFamily: "deb", - supported: false, + hopperSetupSupported: false, }; host.ghidra = "/opt/ghidra_12.1.4_PUBLIC"; host.javaHome = "/usr/lib/jvm/jdk-21"; diff --git a/src/application/SetupClientConfiguration.ts b/src/application/SetupClientConfiguration.ts index 981b829a4..90f4bf4ca 100644 --- a/src/application/SetupClientConfiguration.ts +++ b/src/application/SetupClientConfiguration.ts @@ -29,6 +29,7 @@ import { PRODUCT_IDENTITY } from "../identity.js"; import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js"; import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js"; import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js"; +import { packageReadTargetChanged } from "./MsixPackageStorage.js"; import { readRegularFile, readRegularFileText, @@ -40,7 +41,10 @@ import type { ClientConfigurationResult, SetupProviderEnvironment, } from "./SetupTypes.js"; -import type { SetupClient } from "./SupportedClients.js"; +import { + registrationWriteBlocker, + type SetupClient, +} from "./SupportedClients.js"; export class ClientConfigurationCleanupRollbackFailure extends RegularFileCleanupFailure { constructor( @@ -187,6 +191,13 @@ const persistClientConfiguration = async ({ desired, fallbackServers, }: ClientConfigurationUpdate): Promise => { + // The plan chose this file; an MSIX package may have switched to its + // private copy since, for example when it quit. + if ( + client.msixPackage !== undefined && + (await packageReadTargetChanged(client.msixPackage, client.configPath)) + ) + return { status: "failed", reason: "target_changed" }; const backupPath = original === undefined ? undefined : `${client.configPath}.rea.backup`; let recoveryBackupPath: string | undefined; @@ -358,6 +369,25 @@ export const inspectClientConfiguration = async ( client: SetupClient, providerEnvironment: SetupProviderEnvironment, command: readonly string[], +): Promise => { + const inspection = await inspectClientDocument( + client, + providerEnvironment, + command, + ); + if (inspection.status !== "create" && inspection.status !== "update") + return inspection; + // A change the running client would overwrite is not a persisted registration. + const blocker = registrationWriteBlocker(client, "setup"); + return blocker === undefined + ? inspection + : { status: "invalid", remediation: blocker }; +}; + +const inspectClientDocument = async ( + client: SetupClient, + providerEnvironment: SetupProviderEnvironment, + command: readonly string[], ): Promise => { if (client.configPathError !== undefined) return { status: "invalid", remediation: client.configPathError }; @@ -674,6 +704,14 @@ const clientConfigurationDesired = ( launcher, environment.value, ); + const previous = parsed?.servers[PRODUCT_IDENTITY.mcpServerKey]; + if ( + client.name === "kimi" && + typeof previous === "object" && + previous !== null && + "enabledTools" in previous + ) + registration.enabledTools = previous.enabledTools; return { ...registration, ...(client.name === "codex" || client.name === "grok_build" @@ -681,5 +719,11 @@ const clientConfigurationDesired = ( startup_timeout_sec: MCP_STARTUP_POLICY.codexStartupTimeoutSeconds, } : {}), + ...(client.name === "kimi" + ? { + startupTimeoutMs: MCP_STARTUP_POLICY.kimiStartupTimeoutMs, + toolTimeoutMs: MCP_STARTUP_POLICY.kimiToolTimeoutMs, + } + : {}), }; }; diff --git a/src/application/SetupClients.ts b/src/application/SetupClients.ts index d86d04b9e..5ceccb21a 100644 --- a/src/application/SetupClients.ts +++ b/src/application/SetupClients.ts @@ -9,6 +9,8 @@ import type { SetupClient } from "./SupportedClients.js"; export const failedConfigurationMessage = ( result: Extract, ): string => { + if (result.reason === "target_changed") + return "The client began reading a different configuration file after the plan was shown, so no change was made. Rerun setup to review the new target."; if (result.reason === "path") return "Agent configuration path could not be safely verified. Check its permissions and, if it is a symbolic link, verify that the link resolves to a regular file owned by the current user, then rerun setup."; if (result.reason === "backup") diff --git a/src/application/SetupHost.ts b/src/application/SetupHost.ts index 7ac6828fd..84d41229e 100644 --- a/src/application/SetupHost.ts +++ b/src/application/SetupHost.ts @@ -44,6 +44,10 @@ import { retryRegularFileCleanup, } from "./RegularFileRead.js"; import { providerRegistrationEnvironment } from "./SetupRegistrationEnvironment.js"; +import { + msixPackagesDirectory, + packagePrivateLauncherBlocker, +} from "./MsixPackageStorage.js"; import type { SetupHost, SetupInitialState, @@ -66,6 +70,31 @@ export const setupRegistrationCommand = ( ] : [resolve(process.argv[1] ?? PRODUCT_IDENTITY.cliBinary), "mcp"]; +/** + * The launcher setup registers, and why it must not be registered when it + * exists only inside one MSIX package's private storage. + */ +const setupRegistration = ( + platform: NodeJS.Platform, + homeDirectory: string, + environment: Readonly, +): Pick => { + const registrationCommand = setupRegistrationCommand( + platform, + invokingPackageRunner(environment, platform), + ); + const registrationCommandBlocker = + platform === "win32" + ? packagePrivateLauncherBlocker( + registrationCommand, + msixPackagesDirectory(homeDirectory, environment), + ) + : undefined; + return registrationCommandBlocker === undefined + ? { registrationCommand } + : { registrationCommand, registrationCommandBlocker }; +}; + export const filterClientsNeedingConfigure = async ( host: SetupHost, detectedClients: readonly SetupClient[], @@ -95,7 +124,8 @@ export const hostRemediation = async ( ? "Upgrade to macOS 12 or newer." : undefined; } - if ((await host.linuxDistribution())?.supported === true) return undefined; + if ((await host.linuxDistribution())?.hopperSetupSupported === true) + return undefined; return "Automated Hopper setup supports Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, CachyOS, and Omarchy; configure an existing supported provider instead."; }; @@ -117,10 +147,7 @@ export const systemSetupHost = ( homeDirectory, skillDestinations: (clientIds) => skillDestinations(homeDirectory, clientIds, environment, platform), - registrationCommand: setupRegistrationCommand( - platform, - invokingPackageRunner(environment, platform), - ), + ...setupRegistration(platform, homeDirectory, environment), nodeVersion: process.versions.node, macosVersion: () => doctorHost.macosVersion(), linuxDistribution: readLinuxDistribution, diff --git a/src/application/SetupPlan.ts b/src/application/SetupPlan.ts index a78697c80..f428ecade 100644 --- a/src/application/SetupPlan.ts +++ b/src/application/SetupPlan.ts @@ -8,12 +8,18 @@ import { import type { SkillDestination } from "./SetupSkill.js"; import { macHopperInstallDisclosure } from "./MacHopper.js"; import type { + ClientConfigurationInspection, SetupAction, SetupClientState, SetupHost, SetupProviderEnvironment, } from "./SetupTypes.js"; -import type { SetupClient } from "./SupportedClients.js"; +import { + appDataRedirection, + registrationPersistenceDetail, + type SetupClient, +} from "./SupportedClients.js"; +import { redirectedWriteBlocker } from "./MsixPackageStorage.js"; import type { DoctorReport } from "./Doctor.js"; import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js"; @@ -50,7 +56,8 @@ export const discoverSetupState = async (input: { const hopperInstallSupported = (input.host.platform === "darwin" && majorVersion((await input.host.macosVersion()) ?? "0") >= 12) || - (input.host.platform === "linux" && linuxDistribution?.supported === true); + (input.host.platform === "linux" && + linuxDistribution?.hopperSetupSupported === true); const installHopper = input.forceHopperInstall || (hopperInstallSupported && @@ -161,12 +168,58 @@ export const planSetupActions = async (input: { blocker: `${invalid.client.displayName ?? invalid.client.name}: ${invalid.inspection.remediation}`, }; } - const clientPlans = inspections.flatMap(({ client, inspection }) => + const clientPlans = plannedClientWrites( + inspections, + input.discovery.installHopper, + ); + const plannedActions = setupPlan({ + platform: input.host.platform, + homeDirectory: input.host.homeDirectory, + installHopper: input.discovery.installHopper, + installSkill: input.installSkill, + skillClientIds: input.skillClientIds, + skillDestinations: input.host.skillDestinations(input.skillClientIds), + clients: clientPlans, + providerEnvironment: input.providerEnvironment, + ...(input.discovery.linuxPackageFamily === undefined + ? {} + : { linuxPackageFamily: input.discovery.linuxPackageFamily }), + command: input.command, + }); + const blocker = + (clientPlans.length > 0 + ? input.host.registrationCommandBlocker + : undefined) ?? + redirectedWriteBlocker( + appDataRedirection(input.discovery.clients), + plannedActions + .filter(({ kind }) => kind !== "install_hopper") + .map(({ target }) => target), + "setup", + ); + return { + plannedActions, + selectedClients, + clientsToConfigure: selectedClients.filter((client) => + clientPlans.some(({ client: planned }) => planned.name === client.name), + ), + ...(blocker === undefined ? {} : { blocker }), + }; +}; + +/** Selected clients whose registration setup writes, with the disclosed operation. */ +const plannedClientWrites = ( + inspections: readonly { + readonly client: SetupClient; + readonly inspection: ClientConfigurationInspection; + }[], + installHopper: boolean, +) => + inspections.flatMap(({ client, inspection }) => client.format !== "unsupported" && (inspection.status === "create" || inspection.status === "update" || - (input.discovery.installHopper && - inspection.status === "already_current")) + (installHopper && inspection.status === "already_current")) ? [ { client, @@ -182,27 +235,6 @@ export const planSetupActions = async (input: { ] : [], ); - return { - plannedActions: setupPlan({ - platform: input.host.platform, - homeDirectory: input.host.homeDirectory, - installHopper: input.discovery.installHopper, - installSkill: input.installSkill, - skillClientIds: input.skillClientIds, - skillDestinations: input.host.skillDestinations(input.skillClientIds), - clients: clientPlans, - providerEnvironment: input.providerEnvironment, - ...(input.discovery.linuxPackageFamily === undefined - ? {} - : { linuxPackageFamily: input.discovery.linuxPackageFamily }), - command: input.command, - }), - selectedClients, - clientsToConfigure: selectedClients.filter((client) => - clientPlans.some(({ client: planned }) => planned.name === client.name), - ), - }; -}; /** Describe every selected setup mutation before user approval. */ const setupPlan = (input: { @@ -259,7 +291,7 @@ const setupPlan = (input: { label: client.displayName ?? client.name, target: client.configPath, detail: clientConfigurationDetail( - client.displayName ?? client.name, + client, input.command, input.providerEnvironment, input.installHopper, @@ -303,7 +335,7 @@ const majorVersion = (version: string): number => Number.parseInt(version.split(".")[0] ?? "0", 10); const clientConfigurationDetail = ( - client: string, + client: SetupClient, command: readonly string[], environment: SetupProviderEnvironment, installHopper: boolean, @@ -320,5 +352,6 @@ const clientConfigurationDetail = ( ? " If this plan installs Hopper, REA will add HOPPER_LAUNCHER_PATH using the launcher path reported by that installation." : " The HOPPER_LAUNCHER_PATH value shown above is the current fallback; if this plan installs Hopper, REA will replace it with the launcher path reported by that installation." : ""; - return `Add the REA MCP registration for ${client}; preserve unrelated configuration. The backup path reflects the configuration inspected for this plan; if that file changes before execution, setup will select a recovery path for its current contents. Command: ${command.map((part) => JSON.stringify(part)).join(" ")}.${environmentDetail}${hopperDetail}`; + const persistence = registrationPersistenceDetail(client); + return `Add the REA MCP registration for ${client.displayName ?? client.name}; preserve unrelated configuration. The backup path reflects the configuration inspected for this plan; if that file changes before execution, setup will select a recovery path for its current contents. Command: ${command.map((part) => JSON.stringify(part)).join(" ")}.${environmentDetail}${hopperDetail}${persistence === undefined ? "" : ` ${persistence}`}`; }; diff --git a/src/application/SetupSkill.ts b/src/application/SetupSkill.ts index 83ba96c5a..794202b48 100644 --- a/src/application/SetupSkill.ts +++ b/src/application/SetupSkill.ts @@ -72,6 +72,15 @@ interface CanonicalSkillFile { readonly original: SkillFileState | undefined; } +const canonicalSkillContent = (relativePath: string): Promise => + readFile( + new URL( + `../../skills/${PRODUCT_IDENTITY.skillName}/${relativePath}`, + import.meta.url, + ), + "utf8", + ); + export interface SkillInstallFileOperations { readonly mkdir: (path: string) => Promise; readonly backup: (source: string, destination: string) => Promise; @@ -244,19 +253,36 @@ const canonicalSkillFiles = async ( const destination = join(root, relativePath); return { destination, - content: await readFile( - new URL( - `../../skills/${PRODUCT_IDENTITY.skillName}/${relativePath}`, - import.meta.url, - ), - "utf8", - ), + content: await canonicalSkillContent(relativePath), original: await readOptionalSkillState(destination, resources), }; }), ), ); +const canonicalSkillNeedsInstallByReading = async ( + home: string, + clientIds: readonly string[], + environment: Readonly, + platform: NodeJS.Platform, + resources?: ArtifactResourceScope, +): Promise => { + for (const { path: root } of skillDestinations( + home, + clientIds, + environment, + platform, + )) { + for (const relativePath of MANAGED_SKILL_FILES) { + const destination = join(root, relativePath); + const content = await canonicalSkillContent(relativePath); + const original = await readOptionalSkillState(destination, resources); + if (original?.content !== content) return true; + } + } + return false; +}; + /** Report whether setup would change any file in the managed REA skill bundle. */ export const canonicalSkillNeedsInstall = async ( home: string, @@ -266,15 +292,13 @@ export const canonicalSkillNeedsInstall = async ( resources?: ArtifactResourceScope, ): Promise => { try { - return ( - await canonicalSkillFiles( - home, - clientIds, - environment, - platform, - resources, - ) - ).some(({ content, original }) => original?.content !== content); + return await canonicalSkillNeedsInstallByReading( + home, + clientIds, + environment, + platform, + resources, + ); } catch (cause: unknown) { if ( cause instanceof RegularFileCleanupFailure || diff --git a/src/application/SetupTypes.ts b/src/application/SetupTypes.ts index 241e0f6dd..f0cf61c59 100644 --- a/src/application/SetupTypes.ts +++ b/src/application/SetupTypes.ts @@ -25,7 +25,12 @@ export type ClientConfigurationResult = } | { readonly status: "failed"; - readonly reason: "path" | "backup" | "write" | "readback"; + readonly reason: + | "path" + | "target_changed" + | "backup" + | "write" + | "readback"; } | { readonly status: "failed"; @@ -55,6 +60,8 @@ export interface SetupHost { readonly homeDirectory: string; skillDestinations(clientIds: readonly string[]): readonly SkillDestination[]; readonly registrationCommand: readonly string[]; + /** Why no client may be registered with `registrationCommand` from this host. */ + readonly registrationCommandBlocker?: string; readonly nodeVersion: string; macosVersion(): Promise; linuxDistribution(): Promise; diff --git a/src/application/SupportedClients.ts b/src/application/SupportedClients.ts index 0a7352627..55a651a52 100644 --- a/src/application/SupportedClients.ts +++ b/src/application/SupportedClients.ts @@ -7,6 +7,8 @@ import { resolveHermesClientHome, } from "./HermesClientHome.js"; import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js"; +import { resolveWindowsClaudeDesktopLocation } from "./ClaudeDesktopConfigLocation.js"; +import type { MsixPackageView } from "./MsixPackageStorage.js"; /** One supported client configuration location. */ export interface SetupClient { @@ -29,6 +31,10 @@ export interface SetupClient { /** Same-scope fallback ZCode stops loading after its first native server. */ readonly zcodeFallbackPath?: string; readonly markerPath?: string; + /** MSIX package storage that decides which configuration the client reads. */ + readonly msixPackage?: MsixPackageView; + /** A running client can rewrite its configuration from memory. */ + readonly quitBeforeSetup?: boolean; readonly format?: | "json" | "toml" @@ -39,6 +45,7 @@ export interface SetupClient { | "grok" | "omp" | "pi" + | "kimi" | "hermes" | "zcode" | "dsh" @@ -61,12 +68,35 @@ interface ClientPathContext { readonly env: Readonly; } -interface ClientDefinition { +/** One resolved configuration location and its detection evidence. */ +interface ClientLocation { + readonly configPath: string; + readonly markerPath: string; + readonly msixPackage?: MsixPackageView; +} + +type ClientDefinition = ClientDefinitionBase & + ( + | { + readonly configPath: ClientPath; + readonly markerPath: ClientPath; + readonly location?: undefined; + } + | { + /** Resolve configuration and marker together when they share probes. */ + readonly location: ( + context: ClientPathContext, + ) => Result; + readonly configPath?: undefined; + readonly markerPath?: undefined; + } + ); + +interface ClientDefinitionBase { readonly name: string; readonly displayName: string; - readonly configPath: ClientPath; - readonly markerPath: ClientPath; readonly format: NonNullable; + readonly quitBeforeSetup?: boolean; readonly zcodeFallbackPath?: (context: ClientPathContext) => string; readonly skillPath?: ClientPath; readonly configPaths?: (context: ClientPathContext) => readonly string[]; @@ -111,16 +141,20 @@ const traeUserDirectory = ({ return join(xdgConfigHome({ home, platform, env }), "Trae", "User"); }; -const claudeDesktopDirectory = ({ - home, - platform, - env, -}: ClientPathContext): string => { - if (platform === "win32") - return join(env.APPDATA ?? join(home, "AppData", "Roaming"), "Claude"); - if (platform === "linux") - return join(xdgConfigHome({ home, platform, env }), "Claude"); - return join(home, "Library", "Application Support", "Claude"); +/** Windows packaging decides which file Claude Desktop reads; see its resolver. */ +const claudeDesktopLocation = ( + context: ClientPathContext, +): Result => { + if (context.platform === "win32") + return resolveWindowsClaudeDesktopLocation(context.home, context.env); + const directory = + context.platform === "linux" + ? join(xdgConfigHome(context), "Claude") + : join(context.home, "Library", "Application Support", "Claude"); + return ok({ + configPath: join(directory, "claude_desktop_config.json"), + markerPath: directory, + }); }; const claudeCodeConfigDirectory = ({ home, env }: ClientPathContext): string => @@ -132,6 +166,10 @@ const claudeCodeMarkerDirectory = ({ home, env }: ClientPathContext): string => const codexDirectory = ({ home, env }: ClientPathContext): string => env.CODEX_HOME || join(home, ".codex"); +/** Kimi Code stores MCP declarations beside the rest of its home data. */ +const kimiDirectory = ({ home, env }: ClientPathContext): string => + env.KIMI_CODE_HOME || join(home, ".kimi-code"); + const qoderDirectory = ({ home, env }: ClientPathContext): string => env.QODER_CONFIG_DIR || join(home, ".qoder"); @@ -211,6 +249,55 @@ export const manualRegistrationRemediation = ( ? GROK_BOT_MANUAL_REGISTRATION_REMEDIATION : undefined; +/** + * Facts a setup plan must disclose so an approved registration persists: + * which file an MSIX package reads, and that a running client can overwrite it. + */ +export const registrationPersistenceDetail = ( + client: SetupClient, +): string | undefined => { + const name = client.displayName ?? client.name; + const msix = client.msixPackage; + const packageDetail = + msix === undefined + ? undefined + : client.configPath === msix.privateConfigPath + ? `${name} is installed as MSIX package ${msix.packageFamilyName}, which reads this package-private file instead of ${msix.appDataConfigPath}.` + : `${name} is installed as MSIX package ${msix.packageFamilyName}, which reads this file until it saves its own copy at ${msix.privateConfigPath}.`; + const quitDetail = + client.quitBeforeSetup === true + ? `Quit ${name} fully before approving; closing its window is not enough. A running ${name} can rewrite this file from memory and drop the registration. Start it again after setup.` + : undefined; + const details = [packageDetail, quitDetail].filter( + (detail): detail is string => detail !== undefined, + ); + return details.length === 0 ? undefined : details.join(" "); +}; + +/** + * Why this process must not change an MSIX client's configuration. A process + * the running app started sees its redirected AppData, and the app would + * overwrite the change from memory; an unknown view cannot rule that out. + */ +export const registrationWriteBlocker = ( + client: SetupClient, + operation: "setup" | "uninstall", +): string | undefined => { + const msix = client.msixPackage; + if (msix === undefined || msix.processView === "direct") return undefined; + const name = client.displayName ?? client.name; + return msix.processView === "redirected" + ? `This process was started from ${name}'s MSIX package ${msix.packageFamilyName}: its AppData is redirected into that package, so ${name} is running and would overwrite ${msix.privateConfigPath} from memory. Quit ${name} fully, then rerun ${operation} from a terminal that ${name} did not start.` + : `REA cannot tell whether ${name}'s MSIX package ${msix.packageFamilyName} started this process, because none of its files under ${dirname(msix.privateConfigPath)} could be compared with ${dirname(msix.appDataConfigPath)}. A running ${name} would overwrite the change. Open ${name} once if it has never run, quit it fully, then rerun ${operation} from a terminal that ${name} did not start.`; +}; + +/** The MSIX package view, if any, that redirects this process's AppData writes. */ +export const appDataRedirection = ( + clients: readonly SetupClient[], +): MsixPackageView | undefined => + clients.find(({ msixPackage }) => msixPackage?.processView === "redirected") + ?.msixPackage; + const OMP_PROFILE_NAME = /^[a-z0-9][a-z0-9._-]{0,63}$/u; const OMP_RESERVED_PROFILE = /^(?:CON|PRN|AUX|NUL|COM[0-9]|LPT[0-9])(?:\..*)?$/iu; @@ -408,9 +495,8 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [ { name: "claude_desktop", displayName: "Claude Desktop", - configPath: (context: ClientPathContext) => - join(claudeDesktopDirectory(context), "claude_desktop_config.json"), - markerPath: claudeDesktopDirectory, + location: claudeDesktopLocation, + quitBeforeSetup: true, format: "json", }, { @@ -495,6 +581,14 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [ markerPath: qwenCodeDirectory, format: "json", }, + { + name: "kimi", + displayName: "Kimi Code", + configPath: (context: ClientPathContext) => + join(kimiDirectory(context), "mcp.json"), + markerPath: kimiDirectory, + format: "kimi", + }, { name: "vscode", displayName: "VS Code", @@ -627,6 +721,19 @@ const resolvePath = ( return typeof resolved === "string" ? ok(resolved) : resolved; }; +const resolveLocation = ( + definition: ClientDefinition, + context: ClientPathContext, +): Result => { + if (definition.location !== undefined) return definition.location(context); + const config = resolvePath(definition.configPath, context); + if (!config.ok) return config; + const marker = resolvePath(definition.markerPath, context); + return marker.ok + ? ok({ configPath: config.value, markerPath: marker.value }) + : marker; +}; + /** Describe every client location that setup, doctor, or uninstall may inspect. */ export const supportedClients = ( home: string, @@ -637,27 +744,16 @@ export const supportedClients = ( const definitions: readonly ClientDefinition[] = SUPPORTED_CLIENT_DEFINITIONS; return definitions.map((definition) => { try { - const config = resolvePath(definition.configPath, context); - const marker = config.ok - ? resolvePath(definition.markerPath, context) - : config; - if (!config.ok) + const location = resolveLocation(definition, context); + if (!location.ok) return { name: definition.name, displayName: definition.displayName, format: definition.format, - configPath: config.error.path, - configPathError: config.error.message, + configPath: location.error.path, + configPathError: location.error.message, }; - if (!marker.ok) - return { - name: definition.name, - displayName: definition.displayName, - format: definition.format, - configPath: marker.error.path, - configPathError: marker.error.message, - }; - const configPath = config.value; + const { configPath, markerPath, msixPackage } = location.value; return { name: definition.name, displayName: definition.displayName, @@ -675,7 +771,11 @@ export const supportedClients = ( ...(definition.zcodeFallbackPath === undefined ? {} : { zcodeFallbackPath: definition.zcodeFallbackPath(context) }), - markerPath: marker.value, + markerPath, + ...(msixPackage === undefined ? {} : { msixPackage }), + ...(definition.quitBeforeSetup === true + ? { quitBeforeSetup: true } + : {}), format: definition.format, }; } catch (cause: unknown) { diff --git a/src/application/Uninstall.ts b/src/application/Uninstall.ts index a7f051a49..eac36c73d 100644 --- a/src/application/Uninstall.ts +++ b/src/application/Uninstall.ts @@ -32,10 +32,16 @@ import { } from "./RegularFileRead.js"; import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; import { + appDataRedirection, manualRegistrationRemediation, + registrationWriteBlocker, supportedClients, type SetupClient, } from "./SupportedClients.js"; +import { + redirectedWriteBlocker, + type MsixPackageView, +} from "./MsixPackageStorage.js"; export { isUninstallFailure } from "./UninstallFailure.js"; interface ManagedPathStats { @@ -93,6 +99,7 @@ export interface UninstallHost { /** Report a configuration that cannot be read safely, before any removal. */ inspectClient(client: SetupClient): Promise; removeClient(client: SetupClient): Promise; + inspectSkill?(): Promise; removeSkill(): Promise; purgeData(): Promise; close?(): Promise; @@ -118,6 +125,8 @@ export const runUninstall = async ( const blocking = await host.inspectClient(client); if (blocking !== undefined) blocked.push(blocking); } + const skillBlocker = await host.inspectSkill?.(); + if (skillBlocker !== undefined) blocked.push(skillBlocker); const items: UninstallItem[] = blocked.length > 0 ? [ @@ -209,27 +218,70 @@ export const systemUninstallHost = ( selectedHome ?? homeDirectoryFromEnvironment(environment, platform); const resources = new ArtifactResourceScope(); const fileSystem = selectedFileSystem ?? systemFileSystem(resources); + // Set with the client list: the package view that redirects this + // process's AppData writes, if any. + let redirection: MsixPackageView | undefined; + const clientWriteBlocker = (client: SetupClient): string | undefined => + registrationWriteBlocker(client, "uninstall") ?? + redirectedWriteBlocker(redirection, [client.configPath], "uninstall"); + const skillWriteBlocker = (path: string): string | undefined => + redirectedWriteBlocker(redirection, [path], "uninstall"); return { - clients: () => - Promise.resolve( - supportedClients(home, platform, environment).flatMap((client) => - client.configPaths === undefined - ? [client] - : client.configPaths.map((configPath) => ({ - ...client, - configPath, - })), + clients: () => { + const clients = supportedClients(home, platform, environment); + redirection = appDataRedirection(clients); + return Promise.resolve( + clients.flatMap((client) => + uninstallConfigPaths(client).map((configPath) => ({ + ...client, + configPath, + })), ), - ), + ); + }, inspectClient: async (client) => { const read = await readClientConfiguration(client, fileSystem); - return read.kind === "item" && read.item.status === "failed" - ? read.item - : undefined; + if (read.kind === "item") + return read.item.status === "failed" ? read.item : undefined; + const removal = clientRegistrationRemoval(read.parsed, client); + if (!removal.removeNative && removal.legacyPath === undefined) + return undefined; + // A removal the running package would revert, or redirect into its + // private storage, would leave the registration in place. + const blocker = clientWriteBlocker(client); + return blocker === undefined + ? undefined + : item(client.name, "failed", blocker); + }, + removeClient: (client) => + removeClient(client, fileSystem, () => clientWriteBlocker(client)), + inspectSkill: async () => { + for (const destination of skillDestinations( + home, + undefined, + environment, + platform, + )) { + const inspected = await inspectManagedSkill( + destination.path, + destination.client, + fileSystem, + ); + if (inspected?.status === "failed") return inspected; + if (inspected !== undefined) continue; + const blocker = skillWriteBlocker(destination.path); + if (blocker !== undefined) return item("skill", "failed", blocker); + } + return undefined; }, - removeClient: (client) => removeClient(client, fileSystem), removeSkill: () => - removeManagedSkills(home, fileSystem, environment, platform), + removeManagedSkills( + home, + fileSystem, + environment, + platform, + skillWriteBlocker, + ), purgeData: () => purgeManagedData(join(home, ".rea"), fileSystem), close: async () => { try { @@ -242,6 +294,36 @@ export const systemUninstallHost = ( }; }; +/** + * Every user file that can hold REA's registration for one client. An MSIX + * package can read either its private copy or the AppData file, and earlier + * setups wrote the AppData file, so both lose REA's entry. + */ +const uninstallConfigPaths = (client: SetupClient): readonly string[] => [ + ...new Set([ + ...(client.configPaths ?? [client.configPath]), + ...(client.msixPackage === undefined + ? [] + : [ + client.msixPackage.privateConfigPath, + client.msixPackage.appDataConfigPath, + ]), + ]), +]; + +/** Whether removal would change this configuration. */ +const registrationPresent = (parsed: ClientConfigurationDocument): boolean => { + const name = PRODUCT_IDENTITY.mcpServerKey; + return ( + Object.hasOwn(parsed.servers, name) || + // Beside a native OpenCode V2 table, OpenCode also loads a V1 `mcp.rea`. + legacyClientServerPath(parsed, name) !== undefined || + // Harness ownership is the stable entry id. A stale or malformed launcher + // still belongs to that id, and so does a direct patch that projects no row. + (parsed.dialect === "dsh" && dshDocument(parsed.document).carriesEntry) + ); +}; + type ClientConfigurationRead = | { readonly kind: "item"; readonly item: UninstallItem } | { @@ -308,21 +390,47 @@ const itemRead = (result: UninstallItem): ClientConfigurationRead => ({ item: result, }); +/** + * Share the exact ownership and applicability decision with preflight. A + * removable legacy entry carries its document path, so removal edits exactly + * the entry this decision admitted. + */ +const clientRegistrationRemoval = ( + parsed: ClientConfigurationDocument, + client: SetupClient, +): { + readonly removeNative: boolean; + readonly legacyPath: readonly string[] | undefined; +} => { + const name = PRODUCT_IDENTITY.mcpServerKey; + if ( + parsed.dialect === "dsh" && + dshDocument(parsed.document).hasUnknownStructure + ) + return { removeNative: false, legacyPath: undefined }; + const legacyPath = legacyClientServerPath(parsed, name); + return { + removeNative: + (parsed.dialect === "dsh" && dshDocument(parsed.document).carriesEntry) || + (Object.hasOwn(parsed.servers, name) && + isOwnedRegistration(parsed.servers[name], client)), + legacyPath: + legacyPath !== undefined && + isOwnedRegistration(parsed.legacyServers[name], client) + ? legacyPath + : undefined, + }; +}; + const removeClient = async ( client: SetupClient, fileSystem: UninstallFileSystem, + writeBlocker?: () => string | undefined, ): Promise => { const read = await readClientConfiguration(client, fileSystem); if (read.kind === "item") return read.item; const { transactionPath, original, parsed } = read; const name = PRODUCT_IDENTITY.mcpServerKey; - // Beside a native OpenCode V2 table, OpenCode also loads a V1 `mcp.rea`. - const legacyPath = legacyClientServerPath(parsed, name); - const hasNative = Object.hasOwn(parsed.servers, name); - // Harness ownership is the stable entry id. A stale or malformed launcher - // still belongs to that id, and so does a direct patch that projects no row. - const carriesDshEntry = - parsed.dialect === "dsh" && dshDocument(parsed.document).carriesEntry; if ( parsed.dialect === "dsh" && dshDocument(parsed.document).hasUnknownStructure @@ -332,20 +440,21 @@ const removeClient = async ( "retained", "A Cordis expression may produce registration rows, so the patch was retained without changes.", ); - if (!hasNative && legacyPath === undefined && !carriesDshEntry) + if (!registrationPresent(parsed)) return item(client.name, "skipped", "REA registration is absent."); - const removeNative = - carriesDshEntry || - (hasNative && isOwnedRegistration(parsed.servers[name], client)); - const removeLegacy = - legacyPath !== undefined && - isOwnedRegistration(parsed.legacyServers[name], client); + const { removeNative, legacyPath } = clientRegistrationRemoval( + parsed, + client, + ); + const removeLegacy = legacyPath !== undefined; if (!removeNative && !removeLegacy) return item( client.name, "retained", "The rea-named registration is not REA-owned.", ); + const blocker = writeBlocker?.(); + if (blocker !== undefined) return item(client.name, "failed", blocker); const remaining = { ...parsed.servers }; if (removeNative) delete remaining[name]; const document = withClientServers( @@ -355,7 +464,7 @@ const removeClient = async ( ); const editedPaths = [ ...(removeNative ? [clientServerPath(parsed, name)] : []), - ...(removeLegacy ? [legacyPath] : []), + ...(legacyPath === undefined ? [] : [legacyPath]), ]; const backupPath = `${client.configPath}.rea.backup`; try { @@ -516,12 +625,18 @@ const removeManagedSkills = async ( fileSystem: UninstallFileSystem, environment: Readonly, platform: NodeJS.Platform, + writeBlocker?: (path: string) => string | undefined, ): Promise => { - // Every destination shares one summary item, so each result is labeled by - // the directory it refers to. + // Every destination shares one summary item; preserve both client and path. const results = await Promise.all( - skillDestinations(home, undefined, environment, platform).map(({ path }) => - removeManagedSkill(path, path, fileSystem), + skillDestinations(home, undefined, environment, platform).map( + ({ client, path }) => + removeManagedSkill( + path, + `${client} (${path})`, + fileSystem, + writeBlocker, + ), ), ); const failed = results.find(({ status }) => status === "failed"); @@ -552,11 +667,11 @@ const removeManagedSkills = async ( ); }; -const removeManagedSkill = async ( +const inspectManagedSkill = async ( path: string, name: string, fileSystem: UninstallFileSystem, -): Promise => { +): Promise => { let root: ManagedPathStats; try { root = await fileSystem.stat(path); @@ -594,6 +709,20 @@ const removeManagedSkill = async ( if (!isManagedSkillManifest(content)) return item(name, "retained", "Skill manifest is not REA-owned."); + return undefined; +}; + +const removeManagedSkill = async ( + path: string, + name: string, + fileSystem: UninstallFileSystem, + writeBlocker?: (path: string) => string | undefined, +): Promise => { + const inspection = await inspectManagedSkill(path, name, fileSystem); + if (inspection !== undefined) return inspection; + const blocker = writeBlocker?.(path); + if (blocker !== undefined) return item(name, "failed", blocker); + const manifestName = "SKILL.md"; const otherFiles = MANAGED_SKILL_FILES.filter( (relativePath) => relativePath !== manifestName, diff --git a/src/application/apktool/ApktoolResourceAnalysisService.test.ts b/src/application/apktool/ApktoolResourceAnalysisService.test.ts index 6242bb54d..2220ce9e6 100644 --- a/src/application/apktool/ApktoolResourceAnalysisService.test.ts +++ b/src/application/apktool/ApktoolResourceAnalysisService.test.ts @@ -83,3 +83,37 @@ it("preserves cancellation before execution", async () => { if (result.ok) return; expect(result.error).toBeInstanceOf(AnalysisCancelledError); }); + +it("preserves a cancellation that races a completed observation", async () => { + const controller = new AbortController(); + const racingPort: ApktoolResourceAnalysisPort = { + async inspectAvailability() { + return { status: "available", code: null, reason: null, diagnostics: {} }; + }, + async close() {}, + async execute() { + // The provider may finish its work before the abort is delivered, so a + // successful result is not by itself proof the caller still wants it. + controller.abort(); + return ok(execution); + }, + }; + const result = await new ApktoolResourceAnalysisService(racingPort).execute( + "decode_android_resources", + { path: "/targets/app.apk", include_strings: true }, + { signal: controller.signal }, + ); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error).toBeInstanceOf(AnalysisCancelledError); + // The observation the provider completed is retained on the cancellation, so + // a session still records the Evidence instead of losing finished work. + const retained = parseEvidence(result.error.partialObservation); + expect(retained.operation).toBe("decode_android_resources"); + expect(retained.provider).toEqual(identity); + expect(retained.parameters).toEqual({ + path: "/targets/app.apk", + include_strings: true, + }); + expect(retained.confidence).toBe("observed"); +}); diff --git a/src/application/binary/AnalysisProviderEvaluation.ts b/src/application/binary/AnalysisProviderEvaluation.ts index 1654ab783..27292ed31 100644 --- a/src/application/binary/AnalysisProviderEvaluation.ts +++ b/src/application/binary/AnalysisProviderEvaluation.ts @@ -12,7 +12,7 @@ import { jsonObjectSchema, type JsonValue } from "../../domain/jsonValue.js"; import { err, ok, type Result } from "../../domain/result.js"; import type { AnalysisProviderCandidate } from "../AnalysisProvider.js"; import type { AnalysisProviderCandidateStatus } from "./AnalysisProviderRegistry.js"; -import { ABORTED, waitForAbortable } from "./AbortablePromise.js"; +import { ABORTED, waitForAbortable } from "../../process/AbortablePromise.js"; /** Profile-resolution truth for one discovered provider candidate. */ export interface AnalysisProviderCandidateEvaluation { diff --git a/src/application/binary/BinarySession.ts b/src/application/binary/BinarySession.ts index 3da571832..b151958e7 100644 --- a/src/application/binary/BinarySession.ts +++ b/src/application/binary/BinarySession.ts @@ -283,7 +283,9 @@ export class BinarySession previous.route.identity.id, ); if (!closed.ok) { - this.clearSessionRecords(); + // A failed open retires the provider cache, not the investigation. + // Preserve Evidence and Unknown history for export and recovery. + this.clearSnapshot(); this.#clearRuntimeAvailability(); return closed; } diff --git a/src/application/binary/LazyAnalysisProvider.ts b/src/application/binary/LazyAnalysisProvider.ts index a87e77445..095acf5ed 100644 --- a/src/application/binary/LazyAnalysisProvider.ts +++ b/src/application/binary/LazyAnalysisProvider.ts @@ -5,7 +5,7 @@ import { AnalysisError } from "../../domain/analysisErrorBase.js"; import { ProviderAdapterError } from "../../domain/providerAdapterError.js"; import type { ProviderOperationHealth } from "../../domain/providerOperationHealth.js"; import { err, ok } from "../../domain/result.js"; -import { ABORTED, waitForAbortable } from "./AbortablePromise.js"; +import { ABORTED, waitForAbortable } from "../../process/AbortablePromise.js"; import type { AnalysisClient, AnalysisClientContext, diff --git a/src/application/binaryDiagnostics/RecordedCrashService.test.ts b/src/application/binaryDiagnostics/RecordedCrashService.test.ts index cdf7cbd8f..8d8f8e94c 100644 --- a/src/application/binaryDiagnostics/RecordedCrashService.test.ts +++ b/src/application/binaryDiagnostics/RecordedCrashService.test.ts @@ -30,6 +30,29 @@ it("labels an envelope containing debugger-derived maps as derived", async () => expect(evidence.raw_result).toBeNull(); }); +it.each([ + ["x86_64-little-endian", "x86_64"], + ["aarch64-little-endian", "arm64"], +] as const)( + "names the CPU family carried by a %s core on created Evidence", + async (architecture, expected) => { + const fixture = recordedCrashFixture("/selected.core", architecture); + const service = new RecordedCrashService({ + inspectAndClose(input, options) { + return this.inspect(input, options); + }, + identity: RECORDED_CRASH_TEST_PROVIDER, + inspect: () => Promise.resolve(ok(fixture)), + }); + const result = await service.inspect({ + path: fixture.artifact.path, + include_debugger_context: false, + }); + if (!result.ok) throw result.error; + expect(parseEvidence(result.value).subject?.architecture).toBe(expected); + }, +); + it.each([ { path: "relative.elf" }, { path: "/selected.core", approval: true }, diff --git a/src/application/binaryDiagnostics/RecordedCrashService.ts b/src/application/binaryDiagnostics/RecordedCrashService.ts index df137f83e..67a57ca9a 100644 --- a/src/application/binaryDiagnostics/RecordedCrashService.ts +++ b/src/application/binaryDiagnostics/RecordedCrashService.ts @@ -10,6 +10,7 @@ import { import { createEvidence, type Evidence } from "../../domain/evidence.js"; import { inspectRecordedCrashInputSchema, + RECORDED_LINUX_CORE_PROFILES, recordedCrashSchema, } from "../../domain/native/recordedCrash.js"; import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js"; @@ -101,7 +102,9 @@ export class RecordedCrashService { { path: value.artifact.path, format: "elf", - architecture: "x86_64", + architecture: + RECORDED_LINUX_CORE_PROFILES[value.architecture] + .evidenceArchitecture, sha256: value.artifact.sha256, }, this.provider.identity, diff --git a/src/application/javascript/ElectronBoundaryGraphCallPaths.ts b/src/application/javascript/ElectronBoundaryGraphCallPaths.ts index 4447d59db..b815e82bf 100644 --- a/src/application/javascript/ElectronBoundaryGraphCallPaths.ts +++ b/src/application/javascript/ElectronBoundaryGraphCallPaths.ts @@ -51,6 +51,9 @@ export const addElectronCallPaths = ( properties: { path_id: pathId, entry: path.entry, + ...(path.entry_path === undefined + ? {} + : { entry_path: [...path.entry_path] }), entry_kind: path.entry_kind, channel: path.channel, terminal: path.terminal, @@ -97,6 +100,9 @@ export const addElectronCallPaths = ( properties: { path_id: pathId, entry: path.entry, + ...(path.entry_path === undefined + ? {} + : { entry_path: [...path.entry_path] }), resolution: "candidate", }, }); @@ -119,6 +125,9 @@ export const addElectronCallPaths = ( properties: { path_id: pathId, entry: path.entry, + ...(path.entry_path === undefined + ? {} + : { entry_path: [...path.entry_path] }), channel: path.channel, resolution: "candidate", conditions: path.conditions, diff --git a/src/application/javascript/JavaScriptApplicationEvidence.ts b/src/application/javascript/JavaScriptApplicationEvidence.ts index b812f237e..ca20b813f 100644 --- a/src/application/javascript/JavaScriptApplicationEvidence.ts +++ b/src/application/javascript/JavaScriptApplicationEvidence.ts @@ -100,4 +100,10 @@ const parameters = ( ...(input.source_paths === undefined ? {} : { source_paths: input.source_paths }), + ...(input.max_heap_mb === undefined + ? {} + : { max_heap_mb: input.max_heap_mb }), + ...(input.analysis_timeout_ms === undefined + ? {} + : { analysis_timeout_ms: input.analysis_timeout_ms }), }); diff --git a/src/application/javascript/JavaScriptApplicationService.ts b/src/application/javascript/JavaScriptApplicationService.ts index ea56ce591..fa66a5521 100644 --- a/src/application/javascript/JavaScriptApplicationService.ts +++ b/src/application/javascript/JavaScriptApplicationService.ts @@ -62,26 +62,7 @@ export const analyzeJavaScriptApplicationValidated = async ( }); await checkpointJavaScriptAnalysis(options.signal); reconstructed = await reconstructJavaScriptArtifact( - { - input_path: input.input_path, - format: input.format, - integrity_policy: input.integrity_policy, - ...(input.max_semantic_nodes === undefined - ? {} - : { max_semantic_nodes: input.max_semantic_nodes }), - ...(input.max_semantic_nodes_per_file === undefined - ? {} - : { max_semantic_nodes_per_file: input.max_semantic_nodes_per_file }), - ...(input.source_paths === undefined - ? {} - : { source_paths: input.source_paths }), - ...(input.max_heap_mb === undefined - ? {} - : { max_heap_mb: input.max_heap_mb }), - ...(input.analysis_timeout_ms === undefined - ? {} - : { analysis_timeout_ms: input.analysis_timeout_ms }), - }, + input, scope, options.signal, options.progress, diff --git a/src/application/javascript/JavaScriptArtifactAnalysis.ts b/src/application/javascript/JavaScriptArtifactAnalysis.ts index e61d3ae94..6b48b448c 100644 --- a/src/application/javascript/JavaScriptArtifactAnalysis.ts +++ b/src/application/javascript/JavaScriptArtifactAnalysis.ts @@ -553,10 +553,10 @@ const collectSourceMapOriginals = ( if (!Array.isArray(names)) return invalidSourceMap(file, "Source map has no sources array."); const contents = map.sourcesContent; - if (!hasValidSourceMapContents(names.length, contents)) + if (!hasValidSourceMapContents(contents)) return invalidSourceMap( file, - "Source map sourcesContent must contain one string or null entry per source.", + "Source map sourcesContent must be an array of strings or null entries.", ); if (map.sourceRoot != null && typeof map.sourceRoot !== "string") return invalidSourceMap( diff --git a/src/application/javascript/JavaScriptArtifactPathResolution.ts b/src/application/javascript/JavaScriptArtifactPathResolution.ts index cb29ab2a9..88aa1508f 100644 --- a/src/application/javascript/JavaScriptArtifactPathResolution.ts +++ b/src/application/javascript/JavaScriptArtifactPathResolution.ts @@ -457,9 +457,8 @@ const bareModuleCandidate = ( dependencyDirectory = posix.dirname(confined); } const source = input.files.get(input.sourcePath); - let directory = posix.dirname(input.sourcePath); let nearestNotFound: CandidateResolution | null = null; - while (true) { + for (const directory of packageLookupDirectories(input)) { const candidate = posix.join(directory, dependencyDirectory); const metadata = readPackageMetadata(input, candidate); const continueCommonJsLookup = @@ -487,8 +486,6 @@ const bareModuleCandidate = ( nearestNotFound ??= packageResolution; } else return packageResolution; } - if (directory === "." || directory === "") break; - directory = posix.dirname(directory); } if (nearestNotFound !== null) return nearestNotFound; return unresolvedCandidate("external", [ @@ -496,6 +493,21 @@ const bareModuleCandidate = ( ]); }; +function* packageLookupDirectories( + input: ResolveArtifactPathInput, +): Generator { + let directory = posix.dirname(input.sourcePath); + while (true) { + if ( + input.moduleKind !== "require" || + posix.basename(directory) !== "node_modules" + ) + yield directory; + if (directory === "." || directory === "") return; + directory = posix.dirname(directory); + } +} + /** * Nearest same-container package.json, stopping at a `node_modules` directory * so a nested package cannot inherit an ancestor's map. @@ -960,15 +972,29 @@ const resolveCandidate = ( candidate: string, packageChain: ReadonlySet, emittedSourceCandidates: readonly TypeScriptEmittedSourceCandidate[] = [], -): CandidateResolution => - resolveFileCandidates(input, fileCandidates(candidate, input.moduleKind)) ?? - resolveRuntimeSourceCandidates(input, candidate, emittedSourceCandidates) ?? - resolveDirectory( +): CandidateResolution => { + if (!commonJsDirectoryRequest(input)) { + const file = + resolveFileCandidates( + input, + fileCandidates(candidate, input.moduleKind), + ) ?? + resolveRuntimeSourceCandidates(input, candidate, emittedSourceCandidates); + if (file !== null) return file; + } + return resolveDirectory( input, candidate, readPackageMetadata(input, candidate), packageChain, ); +}; + +const commonJsDirectoryRequest = (input: ResolveArtifactPathInput): boolean => + input.context === "module-specifier" && + input.moduleKind === "require" && + (input.declaredPath.endsWith("/") || + /(?:^|\/)\.{1,2}$/u.test(input.declaredPath)); const resolveDirectory = ( input: ResolveArtifactPathInput, diff --git a/src/application/javascript/JavaScriptArtifactReconstruction.ts b/src/application/javascript/JavaScriptArtifactReconstruction.ts index bd95415c6..a5528ee10 100644 --- a/src/application/javascript/JavaScriptArtifactReconstruction.ts +++ b/src/application/javascript/JavaScriptArtifactReconstruction.ts @@ -277,15 +277,9 @@ export const reconstructJavaScriptArtifact = async ( limitations: completed.graph.limitations, }), ); - const { max_heap_mb, analysis_timeout_ms, ...coreInput } = input; - const parameters = { - ...coreInput, - ...(max_heap_mb === undefined ? {} : { max_heap_mb }), - ...(analysis_timeout_ms === undefined ? {} : { analysis_timeout_ms }), - }; const evidence = await createOwnedJavaScriptApplicationEvidenceCooperatively( - parameters, + input, partial, ); throw interruption.retainPartialObservation(evidence); diff --git a/src/application/javascript/JavaScriptSemanticGraphElectronProjection.ts b/src/application/javascript/JavaScriptSemanticGraphElectronProjection.ts index 53b5bdce6..c10e8f632 100644 --- a/src/application/javascript/JavaScriptSemanticGraphElectronProjection.ts +++ b/src/application/javascript/JavaScriptSemanticGraphElectronProjection.ts @@ -34,6 +34,9 @@ export function* projectElectronSemanticCallPaths( properties: { path_id: pathId, entry: path.entry, + ...(path.entry_path === undefined + ? {} + : { entry_path: [...path.entry_path] }), channel: path.channel, terminal: path.terminal, step_kind: step.kind, @@ -62,6 +65,9 @@ export function* projectElectronSemanticCallPaths( properties: { path_id: pathId, entry: path.entry, + ...(path.entry_path === undefined + ? {} + : { entry_path: [...path.entry_path] }), channel: path.channel, basis: "lexical-wrapper-substitution", }, diff --git a/src/application/native/EnhancedNativeInvestigation.test.ts b/src/application/native/EnhancedNativeInvestigation.test.ts index 087b5981f..1e1b517b6 100644 --- a/src/application/native/EnhancedNativeInvestigation.test.ts +++ b/src/application/native/EnhancedNativeInvestigation.test.ts @@ -23,6 +23,20 @@ import type { Result } from "../../domain/result.js"; const target = "a".repeat(64); const provider = { id: "fixture", name: "Fixture", version: "1" }; +const methodName = "-[BuildViewController buildTapped:]"; +const methodSymbol = { + primary: true, + dynamic: false, + external: false, + type: "function", + source: "user_defined", +}; +const methodInventoryEntry = { + address: "0x1000", + value: methodName, + symbol: methodSymbol, +}; + const evidence = [ { kind: "interface_builder_resource" as const, @@ -199,6 +213,49 @@ const testAnalysis = ( }); describe("native UI action trace", () => { + it("retains Hopper targetless call evidence alongside fallback callee routes", async () => { + const result = await new EnhancedTools( + testAnalysis({ + procedure_address: () => execution("procedure_address", "0x1000"), + procedure_references: (parameters) => + ok( + createAnalysisExecution( + { + reference_kinds_available: false, + references: [], + unresolved_calls: + parameters.procedure === "0x1000" + ? [{ address: "0x1008", reason: "No static target" }] + : [], + }, + { id: "hopper", name: "Hopper", version: "fixture" }, + ), + ), + }), + ).execute("trace_native_ui_action", { action: "0x1000" }); + if (!result.ok) throw result.error; + const trace = nativeInvestigationTraceSchema.parse(result.value); + expect(trace.edges).toContainEqual( + expect.objectContaining({ + relation: "provider_call", + to: "native:function:0x2000", + }), + ); + expect(trace.unresolved).toContainEqual( + expect.objectContaining({ + from: "native:function:0x1000", + to: null, + reason: "No static target", + evidence: [ + expect.objectContaining({ + description: "Hopper reported an unresolved call: No static target", + location: { address: "0x1008", file_offset: null }, + artifact_sha256: target, + }), + ], + }), + ); + }); it("accepts an explicit native seed without an authored UI connection", async () => { const result = await new EnhancedTools( testAnalysis({ @@ -215,8 +272,12 @@ describe("native UI action trace", () => { }, }); }); - it("builds the evidence path from the active app and native provider", async () => { - const tools = new EnhancedTools(testAnalysis()); + it("builds the evidence path from provider names with declared symbol facts", async () => { + const tools = new EnhancedTools( + testAnalysis({ + list_names: () => execution("list_names", [methodInventoryEntry]), + }), + ); const result = await tools.execute("trace_native_ui_action", { action: "buildTapped:", }); @@ -349,18 +410,117 @@ describe("bounded native UI tracing", () => { error: { _tag: "AnalysisInputError" }, }); }); +}); - it("binds dispatch metadata to the active target digest", async () => { - const result = await new EnhancedTools(testAnalysis()).execute( - "inspect_native_dispatch_metadata", - { max_records: 10 }, - ); +describe("native dispatch inventory", () => { + it.each([ + { label: "plain value", entry: { address: "0x1000", value: methodName } }, + { label: "value with symbol facts", entry: methodInventoryEntry }, + { + label: "legacy name with symbol facts", + entry: { address: "0x1000", name: methodName, symbol: methodSymbol }, + }, + ])( + "binds $label inventory to the target without overstating dispatch coverage", + async ({ entry }) => { + const result = await new EnhancedTools( + testAnalysis({ + list_names: () => execution("list_names", [entry]), + }), + ).execute("inspect_native_dispatch_metadata", { max_records: 10 }); - expect(result).toMatchObject({ - ok: true, - value: { target_sha256: target }, - }); - }); + expect(result).toMatchObject({ + ok: true, + value: { + target_sha256: target, + provider, + result: { + objc_dispatch_implementations: [ + { + class_name: "BuildViewController", + selector: "buildTapped:", + implementation_address: "0x1000", + decode: { status: "partial", reason: "symbol_name_only" }, + }, + ], + coverage: expect.arrayContaining([ + expect.objectContaining({ + facet: "objc_dispatch_implementations", + status: "partial", + }), + expect.objectContaining({ + facet: "swift_dispatch_tables", + status: "unsupported", + }), + expect.objectContaining({ + facet: "relative_pointers", + status: "unsupported", + }), + ]), + }, + }, + }); + }, + ); + + it.each([ + { + label: "malformed symbol flag", + entry: { + ...methodInventoryEntry, + symbol: { ...methodSymbol, primary: "true" }, + }, + }, + { + label: "unknown symbol source", + entry: { + ...methodInventoryEntry, + symbol: { ...methodSymbol, source: "unknown" }, + }, + }, + { + label: "incomplete symbol", + entry: { ...methodInventoryEntry, symbol: { primary: true } }, + }, + { + label: "malformed legacy symbol", + entry: { address: "0x1000", name: methodName, symbol: null }, + }, + { + label: "malformed address", + entry: { ...methodInventoryEntry, address: 4096 }, + }, + { + label: "malformed value with a legacy name", + entry: { ...methodInventoryEntry, value: 42, name: methodName }, + }, + { + label: "undeclared field", + entry: { address: "0x1000", value: methodName, unexpected: true }, + }, + ])( + "rejects $label inventory in both dispatch consumers", + async ({ entry }) => { + const tools = new EnhancedTools( + testAnalysis({ + list_names: () => execution("list_names", [entry]), + }), + ); + for (const [operation, parameters] of [ + ["inspect_native_dispatch_metadata", { max_records: 10 }], + ["trace_native_ui_action", { action: "buildTapped:" }], + ] as const) { + expect(await tools.execute(operation, parameters)).toMatchObject({ + ok: false, + error: { + _tag: "AnalysisOutputError", + operation: "list_names", + reason: "provider returned an invalid inventory", + }, + }); + } + }, + ); }); describe("native UI query outcomes", () => { diff --git a/src/application/native/NativeCallRoutes.test.ts b/src/application/native/NativeCallRoutes.test.ts index e54a55dc5..135807c3e 100644 --- a/src/application/native/NativeCallRoutes.test.ts +++ b/src/application/native/NativeCallRoutes.test.ts @@ -7,6 +7,59 @@ import { err, ok } from "../../domain/result.js"; import { AnalysisCancelledError } from "../../domain/analysisErrorCore.js"; const provider = { id: "fixture", name: "Fixture", version: "1" }; +describe("untyped native call route fallback", () => { + it.each([{ callees: [] }, { callees: ["0x2000"] }])( + "preserves targetless calls and their provider when fallback callees are $callees", + async ({ callees }) => { + const calls: string[] = []; + const referenceProvider = { ...provider, id: "hopper", name: "Hopper" }; + const fallbackProvider = { ...provider, id: "fallback" }; + const signal = new AbortController().signal; + const unknown = { address: "0x1008", reason: "No static target" }; + const result = await readNativeCallRoutes( + { + execute: async (operation, parameters, options) => { + calls.push(operation); + expect(parameters.procedure).toBe("0x1000"); + expect(options).toEqual({ signal, selectorOrigin: "provider" }); + return ok( + createAnalysisExecution( + operation === "procedure_references" + ? { + reference_kinds_available: false, + references: [], + unresolved_calls: [unknown], + } + : callees, + operation === "procedure_references" + ? referenceProvider + : fallbackProvider, + ), + ); + }, + }, + "0x1000", + signal, + "provider", + ); + expect(calls).toEqual(["procedure_references", "procedure_callees"]); + expect(result).toEqual({ + ok: true, + value: { + provider: fallbackProvider, + routes: callees.map((address) => ({ + address, + site: "0x1000", + relation: "provider_call", + resolution: "inferred", + })), + unknowns: [{ ...unknown, provider: referenceProvider }], + }, + }); + }, + ); +}); + describe("typed native call routes", () => { it("preserves computed targets, ambiguity and targetless call sites", async () => { const reference = jsonValueSchema.parse(ghidraReferenceEdge()); diff --git a/src/application/native/NativeCallRoutes.ts b/src/application/native/NativeCallRoutes.ts index fe77f61e9..9848ab86d 100644 --- a/src/application/native/NativeCallRoutes.ts +++ b/src/application/native/NativeCallRoutes.ts @@ -2,12 +2,12 @@ import { z } from "zod"; import type { AnalysisOperationPort, ExecutionOptions, + ProviderIdentity, } from "../AnalysisProvider.js"; import { parseRelatedAddresses, referenceKindSchema, } from "../../domain/hopperValues.js"; -import { err, ok } from "../../domain/result.js"; import { AnalysisOutputError } from "../../domain/analysisErrorCore.js"; const referencesSchema = z.object({ @@ -40,15 +40,29 @@ export const readNativeCallRoutes = async ( { procedure, direction: "outgoing" }, options, ); + let unknowns: Array<{ + address: string; + reason: string; + provider: ProviderIdentity; + }> = []; if (execution.ok) { const parsed = referencesSchema.safeParse(execution.value.result); if (!parsed.success) - return err( - new AnalysisOutputError( + return { + ok: false as const, + error: new AnalysisOutputError( "procedure_references", "Invalid typed call references", ), - ); + unknowns, + }; + // Missing detailed reference flags do not invalidate observed targetless + // calls. Keep their producing provider even if another provider supplies + // the fallback callee inventory. + unknowns = parsed.data.unresolved_calls.map((call) => ({ + ...call, + provider: execution.value.provider, + })); const references = parsed.data.references; if ( parsed.data.reference_kinds_available === true || @@ -64,45 +78,52 @@ export const readNativeCallRoutes = async ( targets.add(call.target_address); countBySite.set(call.source_address, targets); } - return ok({ - provider: execution.value.provider, - routes: calls.map((call) => ({ - address: call.target_procedure?.address ?? call.target_address, - site: call.source_address, - relation: - call.kind.available && (call.kind.computed || call.kind.indirect) - ? ("indirect_call" as const) - : ("direct_call" as const), - resolution: - (countBySite.get(call.source_address)?.size ?? 0) > 1 - ? ("ambiguous" as const) - : call.kind.available && - (call.kind.computed || call.kind.indirect) - ? ("resolved" as const) - : ("observed" as const), - })), - unknowns: parsed.data.unresolved_calls, - }); + return { + ok: true as const, + value: { + provider: execution.value.provider, + routes: calls.map((call) => ({ + address: call.target_procedure?.address ?? call.target_address, + site: call.source_address, + relation: + call.kind.available && (call.kind.computed || call.kind.indirect) + ? ("indirect_call" as const) + : ("direct_call" as const), + resolution: + (countBySite.get(call.source_address)?.size ?? 0) > 1 + ? ("ambiguous" as const) + : call.kind.available && + (call.kind.computed || call.kind.indirect) + ? ("resolved" as const) + : ("observed" as const), + })), + unknowns, + }, + }; } } else if (execution.error._tag !== "AnalysisCapabilityUnavailableError") - return err(execution.error); + return { ok: false as const, error: execution.error, unknowns }; const callees = await analysis.execute( "procedure_callees", { procedure }, options, ); - if (!callees.ok) return callees; + if (!callees.ok) + return { ok: false as const, error: callees.error, unknowns }; const parsed = parseRelatedAddresses(callees.value.result, "callees"); return parsed.ok - ? ok({ - provider: callees.value.provider, - routes: parsed.value.map((address) => ({ - address, - site: procedure, - relation: "provider_call" as const, - resolution: "inferred" as const, - })), - unknowns: [], - }) - : parsed; + ? { + ok: true as const, + value: { + provider: callees.value.provider, + routes: parsed.value.map((address) => ({ + address, + site: procedure, + relation: "provider_call" as const, + resolution: "inferred" as const, + })), + unknowns, + }, + } + : { ok: false as const, error: parsed.error, unknowns }; }; diff --git a/src/application/native/NativeDispatchMetadataInspection.ts b/src/application/native/NativeDispatchMetadataInspection.ts index 85c1a69c3..e72d17dbf 100644 --- a/src/application/native/NativeDispatchMetadataInspection.ts +++ b/src/application/native/NativeDispatchMetadataInspection.ts @@ -1,4 +1,5 @@ import { z } from "zod"; +import { addressedValue } from "../../contracts/toolOutputSchemaPrimitives.js"; import { nativeDispatchMetadataResultSchema, inspectNativeDispatchMetadata, @@ -36,8 +37,11 @@ export const inspectNativeDispatch = async ( const names = z .array( z.union([ - z.strictObject({ address: z.string(), value: z.string() }), - z.strictObject({ address: z.string(), name: z.string() }), + addressedValue.strict(), + addressedValue + .omit({ value: true }) + .extend({ name: z.string() }) + .strict(), ]), ) .safeParse(execution.value.result); diff --git a/src/application/native/NativeUiActionTrace.ts b/src/application/native/NativeUiActionTrace.ts index 141b31bc0..8a6affdf7 100644 --- a/src/application/native/NativeUiActionTrace.ts +++ b/src/application/native/NativeUiActionTrace.ts @@ -414,9 +414,44 @@ const addDirectCallees = async (input: { "provider", ); examined += 1; + if (!execution.ok && execution.error._tag === "AnalysisCancelledError") + return { error: execution.error }; + const unknowns = execution.ok + ? execution.value.unknowns + : execution.unknowns; + for (const [index, unknown] of unknowns.entries()) { + if (edges.size >= input.input.max_edges) { + truncated = true; + omitted += unknowns.length - index; + break; + } + const id = `unknown-dispatch:${current.address}:${unknown.address}`; + edges.set(id, { + id, + from: current.node.id, + to: null, + relation: "indirect_call", + resolution: "unresolved", + reason: unknown.reason, + evidence: [ + { + kind: "reference", + provider: unknown.provider, + description: `${unknown.provider.name} reported an unresolved call: ${unknown.reason}`, + location: { address: unknown.address, file_offset: null }, + artifact_path: null, + artifact_sha256: input.targetSha256, + }, + ], + limitations: [], + }); + } if (!execution.ok) { - if (execution.error._tag === "AnalysisCancelledError") - return { error: execution.error }; + if (edges.size >= input.input.max_edges) { + truncated = true; + omitted += 1; + continue; + } const id = `unknown-callees:${current.address}`; edges.set(id, { id, @@ -432,32 +467,6 @@ const addDirectCallees = async (input: { }); continue; } - for (const unknown of execution.value.unknowns) { - if (edges.size >= input.input.max_edges) { - truncated = true; - omitted++; - break; - } - const id = `unknown-dispatch:${current.address}:${unknown.address}`; - edges.set(id, { - id, - from: current.node.id, - to: null, - relation: "indirect_call", - resolution: "unresolved", - reason: unknown.reason, - evidence: [ - { - kind: "reference", - description: unknown.reason, - location: { address: unknown.address, file_offset: null }, - artifact_path: null, - artifact_sha256: input.targetSha256, - }, - ], - limitations: [], - }); - } for (const [index, route] of execution.value.routes.entries()) { const address = route.address; if (edges.size >= input.input.max_edges) { @@ -469,6 +478,7 @@ const addDirectCallees = async (input: { const evidence = [ { kind: "reference" as const, + provider: execution.value.provider, description: `${execution.value.provider.name} reported ${route.relation} from ${current.address} to ${address}`, location: { address: route.site, file_offset: null }, artifact_path: null, diff --git a/src/application/wasm/WasmInterfaceComparisonService.test.ts b/src/application/wasm/WasmInterfaceComparisonService.test.ts new file mode 100644 index 000000000..9936c7530 --- /dev/null +++ b/src/application/wasm/WasmInterfaceComparisonService.test.ts @@ -0,0 +1,198 @@ +import { expect, it } from "vitest"; +import { createEvidence } from "../../domain/evidence.js"; +import { wasmInterfaceComparisonSchema } from "../../contracts/wasm/wasmComparisonContracts.js"; +import { compareWasmInterfaces } from "./WasmInterfaceComparisonService.js"; +import type { WasmInterface } from "../../domain/wasm/wasmInterface.js"; +import { wasmArtifactFixture } from "../../../tests/fixtures/wasm/artifact.js"; + +const facts = ( + name: string, + parameter: "i32" | "i64" = "i32", +): WasmInterface => ({ + state: "complete", + issues: [], + entries: [ + { + direction: "export", + module: null, + name, + kind: "func", + index: 0, + signature: { kind: "func", params: [parameter], results: [] }, + wat_form: `(export "${name}" (func 0))`, + issue: null, + }, + ], +}); +const evidence = (interfaceFacts?: WasmInterface) => { + const report = wasmArtifactFixture(); + if (interfaceFacts !== undefined) report.interface = interfaceFacts; + return createEvidence( + { + path: report.artifact.path, + sha256: report.artifact.sha256, + format: "file", + }, + { id: "fixture", name: "typed fixture", version: "1" }, + { + operation: "inspect_wasm_artifact", + parameters: { path: report.artifact.path }, + result: report, + confidence: "observed", + }, + ); +}; +const compare = async ( + left: ReturnType, + right: ReturnType, +) => { + const result = await compareWasmInterfaces({ left, right, scope: "both" }); + if (!result.ok) throw result.error; + return wasmInterfaceComparisonSchema.parse(result.value.normalized_result); +}; + +it("reports all five comparison states, including duplicate ambiguity and explicit selected scope", async () => { + const left = facts("changed"); + const right = facts("changed", "i64"); + left.entries.push( + ...facts("unchanged").entries, + ...facts("removed").entries, + ...facts("dup").entries, + ...facts("dup").entries, + ); + right.entries.push( + ...facts("unchanged").entries, + ...facts("added").entries, + ...facts("dup").entries, + ); + const report = await compare(evidence(left), evidence(right)); + expect(report.summary).toEqual({ + changed: 1, + unchanged: 1, + removed: 1, + added: 1, + unknown: 1, + }); + expect(report.changes.find((c) => c.name === "dup")).toMatchObject({ + state: "unknown", + left: [expect.anything(), expect.anything()], + right: [expect.anything()], + }); + const selected = await compareWasmInterfaces({ + left: evidence(left), + right: evidence(right), + scope: "imports", + }); + if (!selected.ok) throw selected.error; + expect(selected.value.normalized_result).toMatchObject({ + changes: [], + coverage: "complete-within-inputs", + }); +}); + +it("withholds absence claims for historical or partial inventories and unsupported signatures", async () => { + const known = facts("f"); + const absent = await compare(evidence(), evidence(known)); + expect(absent).toMatchObject({ + coverage: "partial", + changes: [{ state: "unknown" }], + }); + const partial: WasmInterface = { + state: "partial", + issues: ["unread import table"], + entries: [], + }; + expect(await compare(evidence(partial), evidence(known))).toMatchObject({ + changes: [{ state: "unknown" }], + }); + const unsupported = facts("f"); + unsupported.entries[0]!.signature = null; + expect(await compare(evidence(known), evidence(unsupported))).toMatchObject({ + changes: [{ state: "unknown" }], + }); + const partialMatch = facts("f"); + partialMatch.state = "partial"; + partialMatch.issues = ["Later imports were not examined"]; + expect(await compare(evidence(partialMatch), evidence(known))).toMatchObject({ + changes: [{ state: "unknown" }], + }); + expect(await compare(evidence(), evidence())).toMatchObject({ + coverage: "unknown", + changes: [], + }); +}); + +it("authenticates semantic Evidence and binds display subject and selection identity", async () => { + const left = JSON.parse(JSON.stringify(evidence(facts("f")))); + left.normalized_result.interface.entries[0].name = "tampered"; + expect( + await compareWasmInterfaces({ + left, + right: evidence(facts("f")), + scope: "both", + }), + ).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } }); + const rebound = JSON.parse(JSON.stringify(evidence(facts("f")))); + rebound.subject.local_path = "/other.wasm"; + expect( + await compareWasmInterfaces({ + left: rebound, + right: evidence(facts("f")), + scope: "both", + }), + ).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } }); +}); + +it("resolves exact retained references and permits cancellation between comparison batches", async () => { + const left = evidence(facts("f")); + const right = evidence(facts("f", "i64")); + const references = { + left: { kind: "retained-evidence", evidence_id: left.evidence_id }, + right: { kind: "retained-evidence", evidence_id: right.evidence_id }, + scope: "both", + }; + expect(await compareWasmInterfaces(references)).toMatchObject({ + ok: false, + error: { _tag: "EvidenceIntegrityError" }, + }); + const result = await compareWasmInterfaces(references, {}, (id) => + id === left.evidence_id ? left : right, + ); + if (!result.ok) throw result.error; + expect(result.value.evidence_links).toEqual([ + left.evidence_id, + right.evidence_id, + ]); + const large = facts("f"); + large.entries.push( + ...Array.from({ length: 2000 }, (_, i) => facts(`f${i}`).entries[0]!), + ); + const controller = new AbortController(); + const pending = compareWasmInterfaces( + { left: evidence(large), right: evidence(large), scope: "both" }, + { signal: controller.signal }, + ); + setImmediate(() => controller.abort()); + expect(await pending).toMatchObject({ + ok: false, + error: { _tag: "AnalysisCancelledError" }, + }); +}); + +it.each([false, true])( + "keeps a one-sided unsupported interface unknown with reverse=%s", + async (reverse) => { + const empty: WasmInterface = { state: "complete", issues: [], entries: [] }; + const unsupported = facts("exception"); + unsupported.entries[0]!.signature = null; + unsupported.entries[0]!.issue = "Unsupported exnref global type"; + const sides = reverse ? [unsupported, empty] : [empty, unsupported]; + expect(await compare(evidence(sides[0]), evidence(sides[1]))).toMatchObject( + { + coverage: "partial", + summary: { added: 0, removed: 0, unknown: 1 }, + changes: [{ state: "unknown" }], + }, + ); + }, +); diff --git a/src/application/wasm/WasmInterfaceComparisonService.ts b/src/application/wasm/WasmInterfaceComparisonService.ts new file mode 100644 index 000000000..f920a5dea --- /dev/null +++ b/src/application/wasm/WasmInterfaceComparisonService.ts @@ -0,0 +1,250 @@ +import { + WASM_INTERFACE_REPRESENTATION_BYTES, + wasmInterfaceEntryRepresentationBytes, +} from "../../domain/wasm/wasmInterface.js"; +import { setImmediate } from "node:timers/promises"; +import { + compareWasmInterfacesInputSchema, + wasmInterfaceComparisonSchema, +} from "../../contracts/wasm/wasmComparisonContracts.js"; +import { + createEvidence, + parseEvidence, + type Evidence, +} from "../../domain/evidence.js"; +import { + wasmArtifactSchema, + type WasmArtifact, +} from "../../domain/wasm/wasmArtifact.js"; +import type { WasmInterfaceEntry } from "../../domain/wasm/wasmInterface.js"; +import { + AnalysisCancelledError, + AnalysisInputError, + AnalysisResourceConstraintError, +} from "../../domain/analysisErrorCore.js"; +import { AnalysisError } from "../../domain/analysisErrorBase.js"; +import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js"; +import { err, ok, type Result } from "../../domain/result.js"; +import { ARTIFACT_COMPARISON_PROVIDER } from "../InvestigationProviders.js"; +import { + resolvePairedEvidenceRequest, + type EvidenceLookup, +} from "../EvidenceInputResolver.js"; +import type { ExecutionOptions } from "../AnalysisProvider.js"; +import type { z } from "zod"; + +const OPERATION = "compare_wasm_interfaces"; +const parseInspection = ( + raw: unknown, +): { evidence: Evidence; result: WasmArtifact } => { + const evidence = parseEvidence(raw); + if (evidence.operation !== "inspect_wasm_artifact") + throw new TypeError("Expected complete inspect_wasm_artifact Evidence"); + const result = wasmArtifactSchema.parse(evidence.normalized_result); + if ( + evidence.subject?.digest.sha256 !== result.artifact.sha256 || + evidence.subject.local_path !== result.artifact.path || + evidence.parameters.path !== result.artifact.path + ) + throw new TypeError( + "WASM Evidence subject/selection does not match normalized artifact identity", + ); + return { evidence, result }; +}; + +/** Compare only producer-owned typed facts; no raw WAT or original file is reread. */ +export const compareWasmInterfaces = async ( + rawInput: unknown, + options?: ExecutionOptions, + lookup?: EvidenceLookup, +): Promise> => { + if (options?.signal?.aborted) + return err(new AnalysisCancelledError(OPERATION)); + const parsed = compareWasmInterfacesInputSchema.safeParse(rawInput); + if (!parsed.success) + return err( + analysisInputErrorFromIssues(OPERATION, parsed.error.issues, rawInput), + ); + const resolved = resolvePairedEvidenceRequest(parsed.data, lookup); + if (!resolved.ok) return resolved; + try { + await setImmediate(); + options?.signal?.throwIfAborted(); + const left = parseInspection(resolved.value.left); + const right = parseInspection(resolved.value.right); + const scope = parsed.data.scope; + const selected = (entry: WasmInterfaceEntry): boolean => + scope === "both" || + entry.direction === (scope === "imports" ? "import" : "export"); + const groups = new Map< + string, + { + left: WasmInterfaceEntry[]; + right: WasmInterfaceEntry[]; + entry: WasmInterfaceEntry; + } + >(); + let work = 0; + for (const [side, inspection] of [ + ["left", left], + ["right", right], + ] as const) { + for (const entry of inspection.result.interface?.entries ?? []) { + if (++work % 256 === 0) { + await setImmediate(); + options?.signal?.throwIfAborted(); + } + if (!selected(entry)) continue; + const key = JSON.stringify([entry.direction, entry.module, entry.name]); + let group = groups.get(key); + if (group === undefined) { + group = { left: [], right: [], entry }; + groups.set(key, group); + } + group[side].push(entry); + } + } + const knownInventory = (inspection: typeof left): boolean => + inspection.result.interface?.state === "complete"; + const changes: z.infer["changes"] = + []; + const summary = { + added: 0, + removed: 0, + changed: 0, + unchanged: 0, + unknown: 0, + }; + let remaining = WASM_INTERFACE_REPRESENTATION_BYTES; + for (const group of groups.values()) { + remaining -= + 512 + + group.left.reduce( + (sum, entry) => sum + wasmInterfaceEntryRepresentationBytes(entry), + 0, + ) + + group.right.reduce( + (sum, entry) => sum + wasmInterfaceEntryRepresentationBytes(entry), + 0, + ); + if (remaining < 0) + throw new AnalysisResourceConstraintError( + OPERATION, + "capacity", + "WASM comparison exceeds typed interface representation capacity", + { maximum_bytes: WASM_INTERFACE_REPRESENTATION_BYTES }, + ); + if (++work % 256 === 0) { + await setImmediate(); + options?.signal?.throwIfAborted(); + } + let state: keyof typeof summary; + let reason: string | null = null; + const l = group.left[0]; + const r = group.right[0]; + if (group.left.length > 1 || group.right.length > 1) { + state = "unknown"; + reason = + "Duplicate interface names are ambiguous; all occurrences retained"; + } else if (!knownInventory(left) || !knownInventory(right)) { + state = "unknown"; + reason = + "Incomplete interface inventories cannot establish per-key uniqueness or absence"; + } else if ( + [l, r].some( + (entry) => + entry !== undefined && + (entry.signature === null || + entry.issue !== null || + entry.kind === "unknown"), + ) + ) { + state = "unknown"; + reason = + "Signature/type semantics were not admitted for every present entry"; + } else if (l === undefined) state = "added"; + else if (r === undefined) state = "removed"; + else if (l.kind !== r.kind) state = "changed"; + else + state = + JSON.stringify(l.signature) === JSON.stringify(r.signature) + ? "unchanged" + : "changed"; + summary[state]++; + changes.push({ + direction: group.entry.direction, + module: group.entry.module, + name: group.entry.name, + state, + left: group.left, + right: group.right, + reason, + }); + } + const compareText = (a: string, b: string): number => + a < b ? -1 : a > b ? 1 : 0; + changes.sort( + (a, b) => + compareText(a.direction, b.direction) || + compareText(a.module ?? "", b.module ?? "") || + compareText(a.name, b.name), + ); + const coverage = + left.result.interface === undefined && + right.result.interface === undefined + ? "unknown" + : !knownInventory(left) || !knownInventory(right) || summary.unknown > 0 + ? "partial" + : "complete-within-inputs"; + const limitations = [ + "Exact declared interface equality is not runtime/ABI compatibility, successful linking or original-source identity.", + "Typed facts come from the selected validated WABT decoding; unsupported signature semantics and duplicate names remain unknown.", + ]; + const report = wasmInterfaceComparisonSchema.parse({ + left: { + evidence_id: left.evidence.evidence_id, + artifact: left.result.artifact, + }, + right: { + evidence_id: right.evidence.evidence_id, + artifact: right.result.artifact, + }, + scope, + coverage, + changes, + summary, + limitations, + }); + await setImmediate(); + options?.signal?.throwIfAborted(); + return ok( + createEvidence(undefined, ARTIFACT_COMPARISON_PROVIDER, { + operation: OPERATION, + predicateType: "rea.wasm-interface-comparison", + parameters: { + left_evidence_id: left.evidence.evidence_id, + right_evidence_id: right.evidence.evidence_id, + scope, + }, + result: report, + confidence: "derived", + authority: "analyst-inference", + limitations, + evidenceLinks: [left.evidence.evidence_id, right.evidence.evidence_id], + }), + ); + } catch (cause: unknown) { + if (options?.signal?.aborted) + return err(new AnalysisCancelledError(OPERATION)); + if (cause instanceof AnalysisError) return err(cause); + return err( + new AnalysisInputError(OPERATION, { cause }, [ + { + path: [], + reason: "invalid_value", + message: cause instanceof Error ? cause.message : String(cause), + }, + ]), + ); + } +}; diff --git a/src/artifacts/AsarArtifactReader.test.ts b/src/artifacts/AsarArtifactReader.test.ts deleted file mode 100644 index 54eb78cd4..000000000 --- a/src/artifacts/AsarArtifactReader.test.ts +++ /dev/null @@ -1,25 +0,0 @@ -import { describe, expect, it } from "vitest"; - -import { orderAsarListing } from "./AsarArtifactReader.js"; - -describe("orderAsarListing", () => { - it("orders host-spelled listings by their portable archive path", () => { - const posix = orderAsarListing(["/a0", "/a", "/a/b"], "/"); - const windows = orderAsarListing(["\\a0", "\\a", "\\a\\b"], "\\"); - - // "/" (0x2F) precedes "0" (0x30); "\" (0x5C) would follow it. - expect(posix.map(({ path }) => path)).toEqual(["a", "a/b", "a0"]); - expect(windows.map(({ path }) => path)).toEqual(["a", "a/b", "a0"]); - expect(windows.map(({ listed }) => listed)).toEqual([ - "\\a", - "\\a\\b", - "\\a0", - ]); - }); - - it("keeps POSIX backslashes as literal filename characters", () => { - expect(orderAsarListing(["/a\\b"], "/")).toEqual([ - { listed: "/a\\b", path: "a\\b" }, - ]); - }); -}); diff --git a/src/artifacts/AsarArtifactReader.ts b/src/artifacts/AsarArtifactReader.ts index b7a235f4b..c988ba056 100644 --- a/src/artifacts/AsarArtifactReader.ts +++ b/src/artifacts/AsarArtifactReader.ts @@ -13,7 +13,7 @@ import { isAbsolute, join, relative, sep } from "node:path"; import { Readable, Transform } from "node:stream"; import { pipeline } from "node:stream/promises"; -import { listPackage, statFile, uncache } from "@electron/asar"; +import { getRawHeader, uncache } from "@electron/asar"; import { ArtifactReaderFailure, @@ -26,7 +26,11 @@ import { closeAsarHandle, readValidatedAsarEntry } from "./AsarEntryStream.js"; import { admitAsarHeader } from "./AsarHeader.js"; import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js"; import { readFileHandleChunks } from "../filesystem/readFileHandleChunks.js"; -import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js"; +import { + readAsarListing, + type AsarListedEntry, + type AsarMetadata, +} from "./AsarListing.js"; /** * Official Electron ASAR adapter with range-streamed member reads. @@ -51,7 +55,7 @@ export class AsarArtifactReader implements ArtifactReader { ) {} async *entries(signal?: AbortSignal): AsyncIterable { - let paths: readonly AsarListedPath[]; + let paths: readonly AsarListedEntry[]; try { await this.prepareContainer(undefined, signal); paths = await this.#withSnapshotOperation(async () => { @@ -62,8 +66,9 @@ export class AsarArtifactReader implements ArtifactReader { this.#archiveSize = admitted.archiveBytes; this.#headerSize = admitted.headerBytes; abortIfNeeded(signal); - const ordered = orderAsarListing( - listPackage(snapshotPath, { isPack: false }), + const ordered = await readAsarListing( + getRawHeader(snapshotPath).header, + signal, ); const archiveMetadata = await lstat(snapshotPath); if (!archiveMetadata.isFile() || archiveMetadata.isSymbolicLink()) @@ -77,34 +82,25 @@ export class AsarArtifactReader implements ArtifactReader { } catch (cause: unknown) { throw asarFailure(this.path, "inventory", cause); } - for (const { listed, path } of paths) { + for (const { metadata, path } of paths) { abortIfNeeded(signal); - if (path.length === 0) continue; - const providerPath = listed.startsWith(sep) - ? listed.slice(sep.length) - : listed; - let metadata: ReturnType; - try { - metadata = statFile(this.#snapshotPath(), providerPath, false); - } catch (cause: unknown) { - throw asarFailure(this.path, `stat ${path}`, cause); - } + const providerPath = path.split("/").join(sep); const kind = - "files" in metadata + metadata.files !== undefined ? "directory" - : "link" in metadata + : metadata.link !== undefined ? "symlink" : "file"; const entry: ArtifactEntry = { path, kind, - declaredSize: "size" in metadata ? metadata.size : null, + declaredSize: metadata.size ?? null, compressedSize: null, - executable: "executable" in metadata && metadata.executable, + executable: metadata.executable === true, encrypted: false, byteOffset: null, declaredSha256: - "integrity" in metadata && + metadata.integrity !== undefined && metadata.integrity.algorithm === "SHA256" && /^[a-f0-9]{64}$/u.test(metadata.integrity.hash) ? metadata.integrity.hash @@ -603,7 +599,7 @@ export class AsarArtifactReader implements ArtifactReader { } } -type AsarFileMetadata = Extract, { size: number }>; +type AsarFileMetadata = AsarMetadata & { readonly size: number }; type AsarEntryState = { readonly metadata: AsarFileMetadata; readonly entry: ArtifactEntry; @@ -652,12 +648,11 @@ const asarUnpackedEntryFailure = ( }; const isAsarFileMetadata = ( - metadata: ReturnType, + metadata: AsarMetadata, ): metadata is AsarFileMetadata => - "size" in metadata && typeof metadata.size === "number" && - !("files" in metadata) && - !("link" in metadata); + metadata.files === undefined && + metadata.link === undefined; const parseArchiveOffset = (value: string): number | undefined => { if (!/^(?:0|[1-9][0-9]*)$/u.test(value)) return undefined; @@ -699,33 +694,6 @@ const waitForOperation = async ( }); }; -interface AsarListedPath { - readonly listed: string; - readonly path: string; -} - -/** - * Pair each host-spelled listing with its portable archive path, ordered by - * the portable path so Windows and POSIX hosts agree. - * - * @electron/asar returns paths assembled with the host path module. Keep that - * spelling for its API calls, but expose archive paths with portable - * separators. On POSIX, backslashes can be literal filename characters and - * must remain untouched. - */ -export const orderAsarListing = ( - listed: readonly string[], - separator: string = sep, -): AsarListedPath[] => - listed - .map((entry) => ({ listed: entry, path: toArtifactPath(entry, separator) })) - .sort((left, right) => compareUnicodeCodePoints(left.path, right.path)); - -const toArtifactPath = (listed: string, separator: string): string => { - const portable = separator === "\\" ? listed.replaceAll("\\", "/") : listed; - return portable.replace(/^\/+|\/+$/gu, ""); -}; - const asarFailure = ( path: string, operation: string, diff --git a/src/artifacts/AsarListing.ts b/src/artifacts/AsarListing.ts new file mode 100644 index 000000000..60875e93b --- /dev/null +++ b/src/artifacts/AsarListing.ts @@ -0,0 +1,110 @@ +import { setImmediate } from "node:timers/promises"; +import { z } from "zod"; + +import { ArtifactReaderFailure } from "./ArtifactReader.js"; +import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js"; + +/** Capacity for expanded paths and listing records, independent of packed bytes. */ +const MAX_LISTING_BYTES = 64 * 1024 * 1024; +const nodeSchema = z.looseObject({ + files: z + .custom>( + (value) => + value !== null && typeof value === "object" && !Array.isArray(value), + ) + .optional(), + link: z.string().optional(), + size: z.number().nonnegative().safe().optional(), + offset: z.string().optional(), + unpacked: z.boolean().optional(), + executable: z.boolean().optional(), + integrity: z + .looseObject({ algorithm: z.string(), hash: z.string() }) + .optional(), +}); +export type AsarMetadata = z.infer; +export interface AsarListedEntry { + readonly path: string; + readonly metadata: AsarMetadata; +} + +function* ownChildren(files: Record) { + for (const name in files) + if (Object.hasOwn(files, name)) yield [name, files[name]] as const; +} + +/** Decode each node once; admit expanded paths before concatenation and retain no recursive lookup. */ +export const readAsarListing = async ( + header: unknown, + signal?: AbortSignal, +): Promise => { + const root = nodeSchema.parse(header); + if (root.files === undefined) + throw new ArtifactReaderFailure("format", "ASAR root has no files table"); + const entries: AsarListedEntry[] = []; + const pending = [{ parent: "", children: ownChildren(root.files) }]; + let retainedBytes = 0; + let examined = 0; + while (pending.length > 0) { + if (examined++ % 128 === 0) { + await setImmediate(); + if (signal?.aborted) + throw new ArtifactReaderFailure( + "cancelled", + "ASAR operation cancelled", + ); + } + const frame = pending.at(-1); + if (frame === undefined) break; + const next = frame.children.next(); + if (next.done) { + pending.pop(); + continue; + } + const [name, raw] = next.value; + // A header member is one component, not a host path to normalize. POSIX + // backslashes remain literal; Windows cannot represent them as components. + if ( + name.length === 0 || + name === "." || + name === ".." || + name.includes("/") || + name.includes("\0") || + (process.platform === "win32" && name.includes("\\")) + ) + throw new ArtifactReaderFailure( + "path", + "ASAR header has an unsafe path component", + ); + const units = + frame.parent.length + name.length + (frame.parent.length === 0 ? 0 : 1); + // UTF-16 storage, plus records and traversal bookkeeping. Charge every + // occurrence even when the engine can share a string prefix. + retainedBytes += units * 2 + 256; + if (retainedBytes > MAX_LISTING_BYTES) + throw new ArtifactReaderFailure( + "limit", + "ASAR expanded listing exceeds its 64 MiB representation budget", + ); + const path = frame.parent.length === 0 ? name : `${frame.parent}/${name}`; + const metadata = nodeSchema.parse(raw); + if ( + metadata.files === undefined && + metadata.link === undefined && + metadata.size === undefined + ) + throw new ArtifactReaderFailure( + "format", + "ASAR entry has no directory, link or file metadata", + ); + entries.push({ path, metadata }); + if (metadata.files !== undefined) + pending.push({ + parent: path, + children: ownChildren(metadata.files), + }); + } + return entries.sort((left, right) => + compareUnicodeCodePoints(left.path, right.path), + ); +}; diff --git a/src/artifacts/RegularFileAdmission.test.ts b/src/artifacts/RegularFileAdmission.test.ts index 17926799a..97f61fc34 100644 --- a/src/artifacts/RegularFileAdmission.test.ts +++ b/src/artifacts/RegularFileAdmission.test.ts @@ -1,5 +1,6 @@ import type { BigIntStats, Stats } from "node:fs"; import type { FileHandle } from "node:fs/promises"; +import { tmpdir } from "node:os"; import { afterEach, expect, it, vi } from "vitest"; import { @@ -88,3 +89,36 @@ it("compares available nanosecond timestamps without rounding away source change sameRegularFileState(before, { ...before, ctimeNs: before.ctimeNs + 1n }), ).toBe(false); }); + +it.each([ + ["EACCES", -13, "keeps"], + ["ENXIO", -6, "reclassifies"], +] as const)( + "%s from opening a non-regular path: admission %s the failure", + async (code, errno, outcome) => { + const failure = Object.assign(new Error(`open ${code}`), { code, errno }); + mockOpen.mockRejectedValue(failure); + // The selected path itself is a directory that stat can still describe. + const cause = await openRegularFile(tmpdir(), { symlinks: "follow" }).catch( + (rejected: unknown) => rejected, + ); + if (outcome === "keeps") expect(cause).toBe(failure); + else expect(cause).toBeInstanceOf(NonRegularFileReadError); + }, +); + +it("keeps cancellation that arrives during the non-regular metadata probe", async () => { + const controller = new AbortController(); + const cancellation = new Error("cancelled during probe"); + mockOpen.mockImplementation(async () => { + // The probe stats the path after this rejection; abort before it settles. + queueMicrotask(() => controller.abort(cancellation)); + throw Object.assign(new Error("open ENXIO"), { code: "ENXIO", errno: -6 }); + }); + await expect( + openRegularFile(tmpdir(), { + symlinks: "follow", + signal: controller.signal, + }), + ).rejects.toBe(cancellation); +}); diff --git a/src/artifacts/apple/NibArchive.test.ts b/src/artifacts/apple/NibArchive.test.ts index 10579d011..665617d28 100644 --- a/src/artifacts/apple/NibArchive.test.ts +++ b/src/artifacts/apple/NibArchive.test.ts @@ -36,6 +36,22 @@ const archiveWithOverlappingObjectRanges = (objectCount: number): Buffer => { }; describe("NIBArchive decoder", () => { + it("keeps leading U+FEFF in length-prefixed keys and class names", () => { + const archive = encodeArchive({ + classes: ["\uFEFFWidget"], + objects: [{ classIndex: 0, values: { count: 1, "\uFEFFcount": 2 } }], + }); + expect(decodeNibArchive(archive).objects).toEqual([ + { + id: 0, + class_name: "\uFEFFWidget", + values: { count: 1, "\uFEFFcount": 2 }, + }, + ]); + const invalid = Buffer.from(archive); + invalid[invalid.indexOf(Buffer.from("\uFEFFcount"))] = 0xff; + expect(() => decodeNibArchive(invalid)).toThrow(/not valid UTF-8/u); + }); it("decodes bounded object, key, class, and reference tables", () => { const archive = encodeArchive({ classes: ["NSView\0", "NSString\0"], diff --git a/src/browser/BrowserProcessStartup.ts b/src/browser/BrowserProcessStartup.ts index 85eae0553..48367c432 100644 --- a/src/browser/BrowserProcessStartup.ts +++ b/src/browser/BrowserProcessStartup.ts @@ -14,6 +14,7 @@ export class BrowserStartupError extends Error { readonly failure: BrowserStartupFailure; readonly executable: string; readonly elapsedMs: number; + readonly startupTimeoutMs: number; readonly exitCode: number | null; readonly signalCode: NodeJS.Signals | null; readonly stderr: string; @@ -23,6 +24,7 @@ export class BrowserStartupError extends Error { readonly failure: BrowserStartupFailure; readonly executable: string; readonly elapsedMs: number; + readonly startupTimeoutMs: number; readonly exitCode: number | null; readonly signalCode: NodeJS.Signals | null; readonly stderr: string; @@ -33,12 +35,22 @@ export class BrowserStartupError extends Error { this.failure = diagnostics.failure; this.executable = diagnostics.executable; this.elapsedMs = diagnostics.elapsedMs; + this.startupTimeoutMs = diagnostics.startupTimeoutMs; this.exitCode = diagnostics.exitCode; this.signalCode = diagnostics.signalCode; this.stderr = diagnostics.stderr; } } +/** + * Single owner of the wait for Chrome to publish its DevToolsActivePort. + * + * Callers that need a different budget pass `timeoutMs`; nobody else should + * restate this number, because a per-caller literal silently changes how long + * a cold Chrome launch is tolerated before the failure names its ceiling. + */ +export const BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS = 60_000; + /** Owned child process and startup policy for Chrome CDP discovery. */ export interface BrowserStartupOptions { readonly child: ChildProcess; @@ -53,23 +65,23 @@ export interface BrowserStartupOptions { export const waitForBrowserDevtoolsPort = async ( options: BrowserStartupOptions, ): Promise => { - const timeoutMs = options.timeoutMs ?? 15_000; + const timeoutMs = options.timeoutMs ?? BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS; const pollIntervalMs = options.pollIntervalMs ?? 25; const startedAt = performance.now(); - const terminal = childTermination(options, startedAt); + const terminal = childTermination(options, startedAt, timeoutMs); const deadline = startedAt + timeoutMs; while (true) { const terminalState = currentTerminalState(options.child); if (terminalState !== null) - throw startupError(options, startedAt, terminalState); + throw startupError(options, startedAt, timeoutMs, terminalState); const port = await readDevtoolsPort(options.activePortPath); if (port !== null) return port; const remainingMs = deadline - performance.now(); if (remainingMs <= 0) - throw startupError(options, startedAt, { failure: "timeout" }); + throw startupError(options, startedAt, timeoutMs, { failure: "timeout" }); const outcome = await Promise.race([ terminal, @@ -88,11 +100,12 @@ type TerminalState = const childTermination = ( options: BrowserStartupOptions, startedAt: number, + startupTimeoutMs: number, ): Promise => new Promise((resolve) => { options.child.once("error", (cause: unknown) => { resolve( - startupError(options, startedAt, { + startupError(options, startedAt, startupTimeoutMs, { failure: "spawn-error", cause, }), @@ -103,6 +116,7 @@ const childTermination = ( startupError( options, startedAt, + startupTimeoutMs, options.child.signalCode === null ? { failure: "exited" } : { failure: "signalled" }, @@ -120,6 +134,7 @@ const currentTerminalState = (child: ChildProcess): TerminalState | null => { const startupError = ( options: BrowserStartupOptions, startedAt: number, + startupTimeoutMs: number, state: TerminalState, ): BrowserStartupError => new BrowserStartupError( @@ -127,6 +142,7 @@ const startupError = ( failure: state.failure, executable: options.executable, elapsedMs: Math.max(0, Math.round(performance.now() - startedAt)), + startupTimeoutMs, exitCode: options.child.exitCode, signalCode: options.child.signalCode, stderr: options.stderr(), @@ -156,9 +172,10 @@ const browserStartupMessage = ( diagnostics: { readonly executable: string; readonly elapsedMs: number; + readonly startupTimeoutMs: number; readonly exitCode: number | null; readonly signalCode: NodeJS.Signals | null; readonly stderr: string; }, ): string => - `Chrome CDP startup ${failure}; executable=${diagnostics.executable}; elapsed_ms=${String(diagnostics.elapsedMs)}; exit_code=${diagnostics.exitCode === null ? "null" : String(diagnostics.exitCode)}; signal=${diagnostics.signalCode ?? "null"}; stderr=${diagnostics.stderr.length === 0 ? "" : diagnostics.stderr}`; + `Chrome CDP startup ${failure}; executable=${diagnostics.executable}; elapsed_ms=${String(diagnostics.elapsedMs)}; startup_timeout_ms=${String(diagnostics.startupTimeoutMs)}; exit_code=${diagnostics.exitCode === null ? "null" : String(diagnostics.exitCode)}; signal=${diagnostics.signalCode ?? "null"}; stderr=${diagnostics.stderr.length === 0 ? "" : diagnostics.stderr}`; diff --git a/src/browser/CdpBrowserProvider.ts b/src/browser/CdpBrowserProvider.ts index f4d898a68..744a82983 100644 --- a/src/browser/CdpBrowserProvider.ts +++ b/src/browser/CdpBrowserProvider.ts @@ -49,13 +49,14 @@ import { ProviderAdapterError } from "../domain/providerAdapterError.js"; import type { BrowserObservationOperation } from "../domain/browserObservationErrors.js"; import { err, ok, type Result } from "../domain/result.js"; import { - discoverCdpEndpoint, hasCdpTargetWebSocket, type CdpEndpointDiscovery, type CdpEndpointTarget, } from "./CdpEndpoint.js"; import { + attachCdpTargetSession, closeCdpTargetSession, + openCdpEndpoint, openCdpTargetSession, type CdpTargetSession, } from "./CdpTargetSession.js"; @@ -82,11 +83,13 @@ export class CdpBrowserProvider implements BrowserObservationPort { options: ExecutionOptions = {}, ): Promise> { try { - const discovery = await discoverCdpEndpoint( + const endpoint = await openCdpEndpoint( input.cdp_endpoint, "list_browser_targets", options.signal, ); + await endpoint.connection?.close(); + const { discovery } = endpoint; const selected = selectTargets( discovery, input.allowed_origins.length === 0 @@ -169,45 +172,19 @@ export class CdpBrowserProvider implements BrowserObservationPort { input: ObserveWebSessionInput, options: ExecutionOptions = {}, ): Promise> { - let targetSession: CdpTargetSession | undefined; try { - const discovery = await discoverCdpEndpoint( - input.cdp_endpoint, - "observe_web_session", - options.signal, - ); - const target = authorizeTarget(discovery, input); - const scopedInput = scopeToTargetOrigin(input, target); - targetSession = await openCdpTargetSession( - discovery, - target, - "observe_web_session", - options.signal, - ); return ok( webObservationSessionSchema.parse( - await observeCdpSession({ - connection: targetSession.connection, - sessionId: targetSession.sessionId, - discovery, - target, - input: scopedInput, - ...(options.signal === undefined ? {} : { signal: options.signal }), - ...(options.progress === undefined - ? {} - : { progress: options.progress }), - }), + await withTargetSession( + input, + "observe_web_session", + options, + observeCdpSession, + ), ), ); } catch (cause: unknown) { return err(providerError(cause, "observe_web_session")); - } finally { - if (targetSession !== undefined) - await closeCdpTargetSession( - targetSession, - CLEANUP_DOMAINS, - options.signal, - ); } } @@ -215,45 +192,19 @@ export class CdpBrowserProvider implements BrowserObservationPort { input: DiscoverWebMcpToolsInput, options: ExecutionOptions = {}, ): Promise> { - let targetSession: CdpTargetSession | undefined; try { - const discovery = await discoverCdpEndpoint( - input.cdp_endpoint, - "discover_webmcp_tools", - options.signal, - ); - const target = authorizeTarget(discovery, input); - const scopedInput = scopeToTargetOrigin(input, target); - targetSession = await openCdpTargetSession( - discovery, - target, - "discover_webmcp_tools", - options.signal, - ); return ok( webMcpDiscoverySchema.parse( - await discoverWebMcp({ - connection: targetSession.connection, - sessionId: targetSession.sessionId, - discovery, - target, - input: scopedInput, - ...(options.signal === undefined ? {} : { signal: options.signal }), - ...(options.progress === undefined - ? {} - : { progress: options.progress }), - }), + await withTargetSession( + input, + "discover_webmcp_tools", + options, + discoverWebMcp, + ), ), ); } catch (cause: unknown) { return err(providerError(cause, "discover_webmcp_tools")); - } finally { - if (targetSession !== undefined) - await closeCdpTargetSession( - targetSession, - CLEANUP_DOMAINS, - options.signal, - ); } } @@ -273,45 +224,19 @@ export class CdpBrowserProvider implements BrowserObservationPort { input: CaptureWebScreenshotInput, options: ExecutionOptions = {}, ): Promise> { - let targetSession: CdpTargetSession | undefined; try { - const discovery = await discoverCdpEndpoint( - input.cdp_endpoint, - "capture_web_screenshot", - options.signal, - ); - const target = authorizeTarget(discovery, input); - const scopedInput = scopeToTargetOrigin(input, target); - targetSession = await openCdpTargetSession( - discovery, - target, - "capture_web_screenshot", - options.signal, - ); return ok( webScreenshotSchema.parse( - await captureCdpScreenshot({ - connection: targetSession.connection, - sessionId: targetSession.sessionId, - discovery, - target, - input: scopedInput, - ...(options.signal === undefined ? {} : { signal: options.signal }), - ...(options.progress === undefined - ? {} - : { progress: options.progress }), - }), + await withTargetSession( + input, + "capture_web_screenshot", + options, + captureCdpScreenshot, + ), ), ); } catch (cause: unknown) { return err(providerError(cause, "capture_web_screenshot")); - } finally { - if (targetSession !== undefined) - await closeCdpTargetSession( - targetSession, - CLEANUP_DOMAINS, - options.signal, - ); } } @@ -336,44 +261,83 @@ export class CdpBrowserProvider implements BrowserObservationPort { options: ExecutionOptions, operation: "inspect_web_page" | "analyze_web_bundle", ): Promise { - let targetSession: CdpTargetSession | undefined; - try { - const discovery = await discoverCdpEndpoint( - input.cdp_endpoint, - operation, - options.signal, - ); - const target = authorizeTarget(discovery, input); - const scopedInput = scopeToTargetOrigin(input, target); - targetSession = await openCdpTargetSession( - discovery, - target, - operation, - options.signal, - ); - return await capturePage({ - connection: targetSession.connection, - sessionId: targetSession.sessionId, - operation, - discovery, - target, - input: scopedInput, - ...(options.signal === undefined ? {} : { signal: options.signal }), - ...(options.progress === undefined - ? {} - : { progress: options.progress }), - }); - } finally { - if (targetSession !== undefined) - await closeCdpTargetSession( - targetSession, - CLEANUP_DOMAINS, - options.signal, - ); - } + return await withTargetSession( + input, + operation, + options, + async (session) => await capturePage({ ...session, operation }), + ); } } +/** Facts and transport handed to one target operation. */ +interface CdpTargetOperationSession { + readonly connection: CdpTargetSession["connection"]; + readonly sessionId: CdpTargetSession["sessionId"]; + readonly discovery: CdpEndpointDiscovery; + readonly target: CdpEndpointTarget; + readonly input: Input; + readonly signal?: AbortSignal; + readonly progress?: NonNullable; +} + +/** Authorize one discovered target, run work in its session, and close REA's socket. */ +const withTargetSession = async < + Input extends Pick< + InspectWebPageInput, + "cdp_endpoint" | "target_id" | "allowed_origins" + >, + Output, +>( + input: Input, + operation: BrowserObservationOperation, + options: ExecutionOptions, + work: (session: CdpTargetOperationSession) => Promise, +): Promise => { + const endpoint = await openCdpEndpoint( + input.cdp_endpoint, + operation, + options.signal, + ); + let targetSession: CdpTargetSession | undefined; + try { + const target = authorizeTarget(endpoint.discovery, input); + const scopedInput = scopeToTargetOrigin(input, target); + targetSession = + endpoint.connection === undefined + ? await openCdpTargetSession( + endpoint.discovery, + target, + operation, + options.signal, + ) + : await attachCdpTargetSession( + endpoint.connection, + target, + operation, + options.signal, + ); + return await work({ + connection: targetSession.connection, + sessionId: targetSession.sessionId, + discovery: endpoint.discovery, + target, + input: scopedInput, + ...(options.signal === undefined ? {} : { signal: options.signal }), + ...(options.progress === undefined ? {} : { progress: options.progress }), + }); + } finally { + if (targetSession !== undefined) + await closeCdpTargetSession( + targetSession, + CLEANUP_DOMAINS, + options.signal, + ); + // An unadopted browser socket is closed here; closing is idempotent. + await endpoint.connection?.close(); + } +}; + const selectTargets = ( discovery: CdpEndpointDiscovery, allowedOrigins: ReadonlySet | undefined, diff --git a/src/browser/CdpCaptureStorage.test.ts b/src/browser/CdpCaptureStorage.test.ts new file mode 100644 index 000000000..51bb49a08 --- /dev/null +++ b/src/browser/CdpCaptureStorage.test.ts @@ -0,0 +1,84 @@ +import { expect, it } from "vitest"; + +import { inspectWebPageInputSchema } from "../domain/browserObservation.js"; +import { captureStorage } from "./CdpCaptureStorage.js"; +import { CdpCommandRejection } from "./CdpCommandRejection.js"; +import type { CdpConnection } from "./CdpConnection.js"; + +const NO_CACHE_COMMAND = Symbol("unsupported"); + +const captureWithCacheResult = async ( + cacheResult: unknown, +): Promise>["value"]> => { + const connection = { + send(method: string): Promise { + switch (method) { + case "Storage.getUsageAndQuota": + return Promise.resolve({ usage: 1, quota: 10 }); + case "DOMStorage.getDOMStorageItems": + return Promise.resolve({ entries: [] }); + case "IndexedDB.requestDatabaseNames": + return Promise.resolve({ databaseNames: [] }); + case "Network.getCookies": + return Promise.resolve({ cookies: [] }); + case "CacheStorage.requestCacheNames": + return cacheResult === NO_CACHE_COMMAND + ? Promise.reject( + new CdpCommandRejection( + "inspect_web_page", + method, + -32601, + "Method not found", + ), + ) + : Promise.resolve(cacheResult); + case "CacheStorage.requestEntries": + return Promise.resolve({ cacheDataEntries: [], returnCount: 0 }); + default: + return Promise.reject(new Error(`Unexpected command: ${method}`)); + } + }, + } as unknown as CdpConnection; + const { value } = await captureStorage( + { + connection, + sessionId: undefined, + input: inspectWebPageInputSchema.parse({ + cdp_endpoint: "http://127.0.0.1:9222", + target_id: "fixture-page", + include_storage_keys: true, + include_storage_fingerprints: true, + }), + }, + "https://example.test/", + [], + ); + return value; +}; + +it.each([ + ["an observed empty cache list", { caches: [] }, true], + ["a response without a cache list", {}, false], + ["a non-array cache list", { caches: "nope" }, false], + ["a cache list with an unusable entry", { caches: [5] }, false], + ["an unsupported cache command", NO_CACHE_COMMAND, false], +] as const)( + "reports fingerprint completeness for %s", + async (_label, cacheResult, complete) => { + const value = await captureWithCacheResult(cacheResult); + expect(value.cache_names).toEqual([]); + expect(value.fingerprints_complete).toBe(complete); + }, +); + +it.each([false, true])( + "preserves valid cache evidence beside an unusable entry: %s", + async (mixed) => { + const cache = { cacheName: "fixture-cache", cacheId: "fixture-cache-id" }; + const value = await captureWithCacheResult({ + caches: mixed ? [cache, 5] : [cache], + }); + expect(value.cache_names).toEqual(["fixture-cache"]); + expect(value.fingerprints_complete).toBe(!mixed); + }, +); diff --git a/src/browser/CdpCaptureStorage.ts b/src/browser/CdpCaptureStorage.ts index 8ddcea9a8..1e88ca74a 100644 --- a/src/browser/CdpCaptureStorage.ts +++ b/src/browser/CdpCaptureStorage.ts @@ -60,8 +60,8 @@ export const captureStorage = async ( limitations, ) : undefined; - const cacheValues = recordsValue(recordValue(cacheRaw)?.caches); - const caches = capturedCaches(cacheValues); + const cacheField = recordValue(cacheRaw)?.caches; + const caches = capturedCaches(recordsValue(cacheField)); const fingerprints = context.input.include_storage_fingerprints ? await captureStorageFingerprints({ context, @@ -78,8 +78,10 @@ export const captureStorage = async ( indexedRaw !== undefined && Array.isArray(indexedValues) && indexed.length === indexedValues.length && - cacheRaw !== undefined && - caches.length === cacheValues.length; + // Like IndexedDB names, a missing, non-array or partly unusable cache + // list is an incomplete enumeration rather than an empty one. + Array.isArray(cacheField) && + caches.length === cacheField.length; return { value: { origin, diff --git a/src/browser/CdpEndpoint.ts b/src/browser/CdpEndpoint.ts index cf78df183..3db49d28c 100644 --- a/src/browser/CdpEndpoint.ts +++ b/src/browser/CdpEndpoint.ts @@ -9,6 +9,7 @@ import type { BrowserObservationOperation } from "../domain/browserObservationEr import { sanitizeBrowserUrl } from "../domain/browserObservation.js"; import { urlDeclaresQueryOrFragment } from "../domain/urlComponents.js"; import { safeParseJson } from "../domain/safeJson.js"; +import type { CdpConnection } from "./CdpConnection.js"; const endpointVersionSchema = z.object({ Browser: z.string().min(1), @@ -40,6 +41,26 @@ interface CdpDiscoveryResponseLimits { readonly timeoutMs: number; } +const browserSocketVersionSchema = z.object({ + protocolVersion: z.string().min(1), + product: z.string().min(1), + revision: z.string(), + userAgent: z.string(), + jsVersion: z.string(), +}); + +const browserSocketTargetsSchema = z.object({ + targetInfos: z.array( + z.object({ + targetId: z.string().min(1), + type: z.string().min(1), + title: z.string().default(""), + url: z.string(), + attached: z.boolean().optional(), + }), + ), +}); + /** Validated direct CDP WebSocket bound to one discovered page target. */ interface CdpPageWebSocketEndpoint { readonly scope: "page"; @@ -124,6 +145,88 @@ export const discoverCdpEndpoint = async ( }; }; +/** + * Run browser-socket discovery work, including its WebSocket handshake, under + * the HTTP discovery deadline. Reaching the deadline is a timeout, not a + * caller cancellation, so the internal abort is not carried as its cause. + */ +export const withCdpDiscoveryDeadline = async ( + operation: BrowserObservationOperation, + signal: AbortSignal | undefined, + work: (deadlineSignal: AbortSignal) => Promise, + timeoutMs: number = CDP_DISCOVERY_RESPONSE_LIMITS.timeoutMs, +): Promise => { + const deadline = new AbortController(); + const timer = setTimeout(() => deadline.abort(), timeoutMs); + try { + return await work( + signal === undefined + ? deadline.signal + : AbortSignal.any([signal, deadline.signal]), + ); + } catch (cause: unknown) { + if (deadline.signal.aborted && signal?.aborted !== true) + throw new BrowserObservationError(operation, "timeout", { + detail: `CDP browser-socket discovery exceeded its ${String(timeoutMs)} ms deadline.`, + }); + throw cause; + } finally { + clearTimeout(timer); + } +}; + +/** + * Read discovery facts over a browser socket that has no HTTP discovery. + * Each reply is bounded by the connection's protocol message limit; callers + * apply the discovery deadline through `signal`. + */ +export const discoverCdpBrowserSocket = async ( + connection: CdpConnection, + url: string, + operation: BrowserObservationOperation, + signal?: AbortSignal, +): Promise => { + const versionInput = await connection.send( + "Browser.getVersion", + {}, + undefined, + signal, + ); + const targetsInput = await connection.send( + "Target.getTargets", + {}, + undefined, + signal, + ); + const version = parseCdpEndpointValue( + browserSocketVersionSchema, + versionInput, + operation, + ); + const targets = parseCdpEndpointValue( + browserSocketTargetsSchema, + targetsInput, + operation, + ); + return { + webSocket: { scope: "browser", url }, + version: { + product: version.product, + protocol_version: version.protocolVersion, + revision: version.revision, + user_agent: version.userAgent, + js_version: version.jsVersion, + }, + targets: targets.targetInfos.map((target) => ({ + id: target.targetId, + type: target.type, + title: sanitizeTargetTitle(target.title, target.url), + url: target.url, + attached: target.attached ?? null, + })), + }; +}; + const sanitizeTargetTitle = (value: string, targetUrl: string): string => { let url: URL; try { diff --git a/src/browser/CdpSafeMetadata.test.ts b/src/browser/CdpSafeMetadata.test.ts index 0cff4e094..ef9c42e14 100644 --- a/src/browser/CdpSafeMetadata.test.ts +++ b/src/browser/CdpSafeMetadata.test.ts @@ -289,4 +289,25 @@ describe("safe CDP response metadata referrer policy", () => { expect(captured.response.policies.referrer_policy).toBe(expected); expect(captured.response.policies.coop).toBe("same-origin"); }); + + it.each<[string | number | undefined, number | null]>([ + ["100", 100], + [" 0 ", 0], + [100, 100], + ["-5", null], + ["+12", null], + ["0x10", null], + ["1e3", null], + ["9007199254740991", 9007199254740991], + ["9007199254740992", null], + [undefined, null], + ])("declares Content-Length %j only for decimal digits", (raw, expected) => { + const captured = safeResponseMetadata( + "request-length", + `${origin}/api`, + { headers: raw === undefined ? {} : { "Content-Length": raw } }, + new Set([origin]), + ); + expect(captured.response.content_length).toBe(expected); + }); }); diff --git a/src/browser/CdpSafeMetadata.ts b/src/browser/CdpSafeMetadata.ts index 589c00367..1347cdcca 100644 --- a/src/browser/CdpSafeMetadata.ts +++ b/src/browser/CdpSafeMetadata.ts @@ -1,10 +1,6 @@ import type { WebPageInspection } from "../domain/browserObservationSchemas.js"; import { sanitizeBrowserUrl } from "../domain/browserObservation.js"; -import { - numberValue, - cdpStringValue, - type UnknownRecord, -} from "./CdpCaptureValues.js"; +import { cdpStringValue, type UnknownRecord } from "./CdpCaptureValues.js"; type ResponseMetadata = WebPageInspection["metadata"]["responses"][number]; type LinkMetadata = ResponseMetadata["links"][number]; @@ -43,7 +39,7 @@ export const safeResponseMetadata = ( request_id: requestId, url, mime_type: boundedHeader(cdpStringValue(response.mimeType)), - content_length: nonnegativeInteger(headers.get("content-length")), + content_length: contentLength(headers.get("content-length")), content_encoding: boundedHeader(headers.get("content-encoding")), csp, links, @@ -343,9 +339,12 @@ const policyToken = (value: string | undefined): string | null => { return /^[a-z][a-z0-9_.\x2d]*$/u.test(token) ? token : null; }; -const nonnegativeInteger = (value: string | undefined): number | null => { - const number = numberValue(value === undefined ? undefined : Number(value)); - return number === undefined ? null : Math.max(0, Math.trunc(number)); +// RFC 9110 section 8.6: Content-Length = 1*DIGIT. Other spellings declare no length. +const contentLength = (value: string | undefined): number | null => { + const token = (value ?? "").trim(); + if (!/^[0-9]+$/u.test(token)) return null; + const length = Number(token); + return Number.isSafeInteger(length) ? length : null; }; const boundedHeader = (value: string | undefined): string | null => diff --git a/src/browser/CdpTargetSession.ts b/src/browser/CdpTargetSession.ts index 3e1f6eb43..7355e142e 100644 --- a/src/browser/CdpTargetSession.ts +++ b/src/browser/CdpTargetSession.ts @@ -1,8 +1,12 @@ +import type { BrowserObservationEndpoint } from "../domain/browserObservation.js"; import { BrowserObservationError } from "../domain/browserObservationError.js"; import type { BrowserObservationOperation } from "../domain/browserObservationErrors.js"; import { CdpConnection } from "./CdpConnection.js"; import { cdpTargetWebSocket, + discoverCdpBrowserSocket, + discoverCdpEndpoint, + withCdpDiscoveryDeadline, type CdpEndpointDiscovery, type CdpEndpointTarget, } from "./CdpEndpoint.js"; @@ -13,6 +17,59 @@ export interface CdpTargetSession { readonly sessionId: string | undefined; } +/** Discovery for one caller-selected endpoint and the browser socket that read it. */ +export interface CdpOpenedEndpoint { + readonly discovery: CdpEndpointDiscovery; + /** + * Browser socket kept for the target session, because a browser may ask + * the user to approve each socket. HTTP discovery opens no socket. + */ + readonly connection: CdpConnection | undefined; +} + +/** + * Discover one passive browser endpoint through its selected transport. A + * browser socket's handshake and discovery share one discovery deadline. + */ +export const openCdpEndpoint = async ( + endpoint: BrowserObservationEndpoint, + operation: BrowserObservationOperation, + signal?: AbortSignal, + timeoutMs?: number, +): Promise => { + if (endpoint.transport === "http_discovery") + return { + discovery: await discoverCdpEndpoint(endpoint.url, operation, signal), + connection: undefined, + }; + return await withCdpDiscoveryDeadline( + operation, + signal, + async (deadlineSignal) => { + const connection = await CdpConnection.connect( + endpoint.url, + operation, + deadlineSignal, + ); + try { + return { + discovery: await discoverCdpBrowserSocket( + connection, + endpoint.url, + operation, + deadlineSignal, + ), + connection, + }; + } catch (cause: unknown) { + await connection.close(); + throw cause; + } + }, + timeoutMs, + ); +}; + /** Open one authorized target through its supported browser or page transport. */ export const openCdpTargetSession = async ( discovery: CdpEndpointDiscovery, @@ -29,6 +86,16 @@ export const openCdpTargetSession = async ( limits, ); if (webSocket.scope === "page") return { connection, sessionId: undefined }; + return await attachCdpTargetSession(connection, target, operation, signal); +}; + +/** Attach one target through an open browser socket, which the session then owns. */ +export const attachCdpTargetSession = async ( + connection: CdpConnection, + target: CdpEndpointTarget, + operation: BrowserObservationOperation, + signal?: AbortSignal, +): Promise => { try { const attached = await connection.send( "Target.attachToTarget", diff --git a/src/browser/PlaywrightScenarioBrowser.test.ts b/src/browser/PlaywrightScenarioBrowser.test.ts index 79e903975..ffd2dba95 100644 --- a/src/browser/PlaywrightScenarioBrowser.test.ts +++ b/src/browser/PlaywrightScenarioBrowser.test.ts @@ -164,7 +164,15 @@ it("keeps attached-target ownership after the cleanup response timeout", async ( cleanup: { reason: expect.stringContaining("timed out") }, }); await closeStarted.promise; - await vi.advanceTimersByTimeAsync(1_000); + let responseSettled = false; + void timedOutClose + .finally(() => { + responseSettled = true; + }) + .catch(() => {}); + await vi.advanceTimersByTimeAsync(29_999); + expect(responseSettled).toBe(false); + await vi.advanceTimersByTimeAsync(1); await timeoutExpectation; expect(settled).toEqual([]); diff --git a/src/browser/PlaywrightScenarioBrowser.ts b/src/browser/PlaywrightScenarioBrowser.ts index 615ad055e..3af9ade9c 100644 --- a/src/browser/PlaywrightScenarioBrowser.ts +++ b/src/browser/PlaywrightScenarioBrowser.ts @@ -11,6 +11,9 @@ import { BrowserObservationError } from "../domain/browserObservationError.js"; import { withPlaywrightExecutionBoundary } from "./PlaywrightExecutionBoundary.js"; const OPERATION = "capture_browser_scenario" as const; +// Closing a real Chrome persistent context can take several seconds even when +// it succeeds. The shorter operation deadline must not misreport cleanup. +const CLEANUP_CONFIRMATION_TIMEOUT_MS = 30_000; const throwIfScenarioCancelled = (signal: AbortSignal | undefined): void => { if (signal?.aborted === true) @@ -71,7 +74,11 @@ export class PlaywrightScenarioBrowserCleanupOwner { signal?: AbortSignal, ): Promise { try { - await withPlaywrightExecutionBoundary(() => cleanup, 1_000, signal); + await withPlaywrightExecutionBoundary( + () => cleanup, + CLEANUP_CONFIRMATION_TIMEOUT_MS, + signal, + ); } catch (cause: unknown) { throw new BrowserObservationError(OPERATION, "cleanup_failed", { cause, diff --git a/src/browser/PlaywrightScenarioEvents.sharedContext.test.ts b/src/browser/PlaywrightScenarioEvents.sharedContext.test.ts new file mode 100644 index 000000000..8e20a83ae --- /dev/null +++ b/src/browser/PlaywrightScenarioEvents.sharedContext.test.ts @@ -0,0 +1,130 @@ +import { EventEmitter } from "node:events"; +import type { Page } from "playwright-core"; +import { expect, it } from "vitest"; + +import { browserScenarioSchema } from "../domain/browserScenario.js"; +import { BrowserScenarioSecrets } from "./BrowserScenarioSecrets.js"; +import { PlaywrightScenarioEvents } from "./PlaywrightScenarioEvents.js"; + +it("omits a request without a frame in a shared context", () => { + const context = new EventEmitter(); + const page = new EventEmitter() as unknown as Page; + const scenario = browserScenarioSchema.parse({ + browser: { mode: "launch", executable_path: "/opt/chromium" }, + start_url: { url: "https://app.example.test/" }, + actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }], + capture: { events: ["network"] }, + }); + const secrets = BrowserScenarioSecrets.resolve(scenario, {}); + if (secrets === undefined) throw new Error("Expected resolved secrets"); + const events = new PlaywrightScenarioEvents({ + page, + context: context as never, + ownsContext: false, + enabled: new Set(["network"]), + secrets, + }); + const unknownRequest = { + frame: () => { + throw new Error("Frame for this navigation request is unavailable"); + }, + url: () => "https://unrelated.example.test/private", + }; + context.emit("request", unknownRequest); + expect(events.result().items).toEqual([]); + expect(events.limitations()).toContain( + "Shared-context network capture excludes requests without a known root or descendant Page frame.", + ); +}); + +it("detaches root and popup listeners when a scenario finishes", async () => { + const context = new EventEmitter(); + const page = new EventEmitter() as unknown as Page; + const popup = Object.assign(new EventEmitter(), { + url: () => "https://app.example.test/popup", + }); + const scenario = browserScenarioSchema.parse({ + browser: { mode: "launch", executable_path: "/opt/chromium" }, + start_url: { url: "https://app.example.test/" }, + actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }], + capture: { events: ["console", "popups"] }, + }); + const secrets = BrowserScenarioSecrets.resolve(scenario, {}); + if (secrets === undefined) throw new Error("Expected resolved secrets"); + const events = new PlaywrightScenarioEvents({ + page, + context: context as never, + ownsContext: false, + enabled: new Set(["console", "popups"]), + secrets, + }); + (page as unknown as EventEmitter).emit("popup", popup); + expect((page as unknown as EventEmitter).listenerCount("console")).toBe(1); + expect(popup.listenerCount("console")).toBe(1); + // Popup closure is recorded, then its page-owned listeners are released. + expect(popup.listenerCount("close")).toBe(2); + await events.finish(); + expect((page as unknown as EventEmitter).listenerCount("console")).toBe(0); + expect((page as unknown as EventEmitter).listenerCount("popup")).toBe(0); + expect(popup.listenerCount("console")).toBe(0); + expect(popup.listenerCount("popup")).toBe(0); + expect(popup.listenerCount("close")).toBe(0); +}); + +it("detaches worker and websocket listeners when a scenario finishes", async () => { + const context = new EventEmitter(); + const page = new EventEmitter() as unknown as Page; + const socket = Object.assign(new EventEmitter(), { + url: () => "wss://app.example.test/socket", + }); + const worker = Object.assign(new EventEmitter(), { + url: () => "https://app.example.test/worker.js", + }); + const scenario = browserScenarioSchema.parse({ + browser: { mode: "launch", executable_path: "/opt/chromium" }, + start_url: { url: "https://app.example.test/" }, + actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }], + capture: { events: ["websockets", "workers"] }, + }); + const secrets = BrowserScenarioSecrets.resolve(scenario, {}); + if (secrets === undefined) throw new Error("Expected resolved secrets"); + const events = new PlaywrightScenarioEvents({ + page, + context: context as never, + ownsContext: false, + enabled: new Set(["websockets", "workers"]), + secrets, + }); + (page as unknown as EventEmitter).emit("websocket", socket); + (page as unknown as EventEmitter).emit("worker", worker); + expect(socket.listenerCount("framesent")).toBe(1); + expect(worker.listenerCount("close")).toBe(1); + await events.finish(); + expect(socket.listenerCount("framesent")).toBe(0); + expect(socket.listenerCount("framereceived")).toBe(0); + expect(socket.listenerCount("close")).toBe(0); + expect(worker.listenerCount("close")).toBe(0); +}); + +it("detaches owned-context popup discovery when a scenario finishes", async () => { + const context = new EventEmitter(); + const page = new EventEmitter() as unknown as Page; + const scenario = browserScenarioSchema.parse({ + browser: { mode: "launch", executable_path: "/opt/chromium" }, + start_url: { url: "https://app.example.test/" }, + actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }], + capture: { events: ["console"] }, + }); + const secrets = BrowserScenarioSecrets.resolve(scenario, {}); + if (secrets === undefined) throw new Error("Expected resolved secrets"); + const events = new PlaywrightScenarioEvents({ + page, + context: context as never, + ownsContext: true, + enabled: new Set(["console"]), + secrets, + }); + expect(context.listenerCount("page")).toBe(1); + await events.finish(); + expect(context.listenerCount("page")).toBe(0); +}); diff --git a/src/browser/PlaywrightScenarioEvents.ts b/src/browser/PlaywrightScenarioEvents.ts index 18e1f0e3b..aee9db630 100644 --- a/src/browser/PlaywrightScenarioEvents.ts +++ b/src/browser/PlaywrightScenarioEvents.ts @@ -54,10 +54,11 @@ export class PlaywrightScenarioEvents { private readonly enabled: ReadonlySet; private readonly secrets: BrowserScenarioSecrets; private readonly ownsContext: boolean; - private readonly pages = new Set(); + private pages = new WeakSet(); private readonly incompleteFamilies = new Set(); private readonly network: PlaywrightScenarioNetwork | undefined; private stopNetwork: (() => void) | undefined; + private readonly stopListeners = new Map void>>(); private sealed = false; constructor(options: EventCaptureOptions) { @@ -72,10 +73,12 @@ export class PlaywrightScenarioEvents { this.secrets = options.secrets; this.ownsContext = options.ownsContext === true; this.pages.add(options.page); - if (this.ownsContext && this.enabled.size > 0) - (options.context ?? options.page.context()).on("page", (page) => - this.popup(page), - ); + if (this.ownsContext && this.enabled.size > 0) { + const context = options.context ?? options.page.context(); + const onPage = (page: Page) => this.popup(page); + context.on("page", onPage); + this.retainListener(context, () => context.off("page", onPage)); + } if (this.enabled.has("network")) { const context = options.context ?? options.page.context(); const collector = new PlaywrightScenarioNetwork({ @@ -138,6 +141,8 @@ export class PlaywrightScenarioEvents { async finish(): Promise { this.stopNetwork?.(); this.stopNetwork = undefined; + for (const owner of this.stopListeners.keys()) this.releaseListeners(owner); + this.pages = new WeakSet(); try { await this.network?.finish(); } finally { @@ -184,34 +189,81 @@ export class PlaywrightScenarioEvents { return this.sequence; } + private retainListener(owner: object, stop: () => void): void { + let stops = this.stopListeners.get(owner); + if (stops === undefined) { + stops = new Set(); + this.stopListeners.set(owner, stops); + } + stops.add(stop); + } + + private releaseListeners(owner: object): void { + const stops = this.stopListeners.get(owner); + this.stopListeners.delete(owner); + for (const stop of stops ?? []) stop(); + } + private observePage(page: Page): void { - if (this.enabled.has("console")) - page.on("console", (message) => this.console(message)); - if (this.enabled.has("page-errors")) - page.on("pageerror", (error) => { + if (this.enabled.size > 0) { + const onClose = () => { + this.releaseListeners(page); + }; + page.on("close", onClose); + this.retainListener(page, () => page.off("close", onClose)); + } + + if (this.enabled.has("console")) { + const onConsole = (message: ConsoleMessage) => this.console(message); + page.on("console", onConsole); + this.retainListener(page, () => page.off("console", onConsole)); + } + if (this.enabled.has("page-errors")) { + const onError = (error: Error) => { this.push({ kind: "page-error", message: this.secrets.redact(error.message), stack: error.stack === undefined ? null : this.secrets.redact(error.stack), }); - }); - if (this.enabled.has("websockets")) - page.on("websocket", (socket) => this.webSocket(socket)); - if (this.enabled.has("frames")) { - page.on("frameattached", (frame) => this.frame("frame-attached", frame)); - page.on("framedetached", (frame) => this.frame("frame-detached", frame)); - page.on("framenavigated", (frame) => - this.frame("frame-navigated", frame), - ); + }; + page.on("pageerror", onError); + this.retainListener(page, () => page.off("pageerror", onError)); + } + if (this.enabled.has("websockets")) { + const onWebSocket = (socket: WebSocket) => this.webSocket(socket); + page.on("websocket", onWebSocket); + this.retainListener(page, () => page.off("websocket", onWebSocket)); + } + if (this.enabled.has("frames")) { + const onAttached = (frame: Frame) => this.frame("frame-attached", frame); + const onDetached = (frame: Frame) => this.frame("frame-detached", frame); + const onNavigated = (frame: Frame) => + this.frame("frame-navigated", frame); + page.on("frameattached", onAttached); + page.on("framedetached", onDetached); + page.on("framenavigated", onNavigated); + this.retainListener(page, () => page.off("frameattached", onAttached)); + this.retainListener(page, () => page.off("framedetached", onDetached)); + this.retainListener(page, () => page.off("framenavigated", onNavigated)); + } + if (this.enabled.has("workers")) { + const onWorker = (worker: Worker) => this.worker(worker); + page.on("worker", onWorker); + this.retainListener(page, () => page.off("worker", onWorker)); } - if (this.enabled.has("workers")) - page.on("worker", (worker) => this.worker(worker)); // Any selected family needs descendant discovery in shared contexts. // Popup lifecycle records remain gated on their own selector below. - if (this.enabled.size > 0) page.on("popup", (popup) => this.popup(popup)); - if (this.enabled.has("downloads")) - page.on("download", (download) => this.download(download)); + if (this.enabled.size > 0) { + const onPopup = (popup: Page) => this.popup(popup); + page.on("popup", onPopup); + this.retainListener(page, () => page.off("popup", onPopup)); + } + if (this.enabled.has("downloads")) { + const onDownload = (download: Download) => this.download(download); + page.on("download", onDownload); + this.retainListener(page, () => page.off("download", onDownload)); + } } private safeUrl(value: string) { @@ -231,13 +283,20 @@ export class PlaywrightScenarioEvents { private webSocket(socket: WebSocket): void { const url = this.safeUrl(socket.url()); this.push({ kind: "websocket-opened", url }); - socket.on("framesent", ({ payload }) => - this.webSocketFrame("websocket-frame-sent", url, payload), - ); - socket.on("framereceived", ({ payload }) => - this.webSocketFrame("websocket-frame-received", url, payload), - ); - socket.on("close", () => this.push({ kind: "websocket-closed", url })); + const onSent = ({ payload }: { payload: string | Buffer }) => + this.webSocketFrame("websocket-frame-sent", url, payload); + const onReceived = ({ payload }: { payload: string | Buffer }) => + this.webSocketFrame("websocket-frame-received", url, payload); + const onClose = () => { + this.push({ kind: "websocket-closed", url }); + this.releaseListeners(socket); + }; + socket.on("framesent", onSent); + socket.on("framereceived", onReceived); + socket.on("close", onClose); + this.retainListener(socket, () => socket.off("framesent", onSent)); + this.retainListener(socket, () => socket.off("framereceived", onReceived)); + this.retainListener(socket, () => socket.off("close", onClose)); } private webSocketFrame( @@ -286,7 +345,12 @@ export class PlaywrightScenarioEvents { name: null, }; this.push({ kind: "worker-created", ...details }); - worker.on("close", () => this.push({ kind: "worker-closed", ...details })); + const onClose = () => { + this.push({ kind: "worker-closed", ...details }); + this.releaseListeners(worker); + }; + worker.on("close", onClose); + this.retainListener(worker, () => worker.off("close", onClose)); } private popup(page: Page): void { @@ -305,13 +369,14 @@ export class PlaywrightScenarioEvents { }; if (this.enabled.has("popups")) { this.push({ kind: "popup-opened", ...opened }); - page.on("close", () => + const onClose = () => this.push({ kind: "popup-closed", url: page.url() === "" ? null : this.safeUrl(page.url()), name: null, - }), - ); + }); + page.on("close", onClose); + this.retainListener(page, () => page.off("close", onClose)); } this.observePage(page); } diff --git a/src/browser/WebSourceMapFetcher.ts b/src/browser/WebSourceMapFetcher.ts index c4bae782a..5c8f86ea0 100644 --- a/src/browser/WebSourceMapFetcher.ts +++ b/src/browser/WebSourceMapFetcher.ts @@ -743,8 +743,8 @@ const originalModuleEdges = ( check(); const dependency = originalDependency(node); if (dependency === null) return; - const { kind, specifier } = dependency; - if (kind === "require" && t.isCallExpression(node)) { + const { kind, specifier, requireCallee } = dependency; + if (requireCallee !== undefined) { unboundRequires ??= new Set( analyzeParsedJavaScriptReferences(parsed, "require") .filter( @@ -756,7 +756,7 @@ const originalModuleEdges = ( `${String(location.start.line)}:${String(location.start.column)}`, ), ); - const location = node.callee.loc?.start; + const location = requireCallee.loc?.start; if ( location === undefined || !unboundRequires.has( @@ -1039,6 +1039,7 @@ const originalDependency = ( ): { readonly kind: ParsedSourceMapItem["original_module_edges"][number]["kind"]; readonly specifier: string; + readonly requireCallee?: t.Identifier; } | null => { if ( t.isImportDeclaration(node) || @@ -1051,11 +1052,15 @@ const originalDependency = ( if (t.isImportExpression(node) && t.isStringLiteral(node.source)) return { kind: "dynamic_import", specifier: node.source.value }; if ( - t.isCallExpression(node) && + (t.isCallExpression(node) || t.isOptionalCallExpression(node)) && t.isIdentifier(node.callee, { name: "require" }) && t.isStringLiteral(node.arguments[0]) ) - return { kind: "require", specifier: node.arguments[0].value }; + return { + kind: "require", + specifier: node.arguments[0].value, + requireCallee: node.callee, + }; // `import x = require("m")` is only legal at module top level and always // refers to the host loader, so it needs no unbound-`require` check. The // `moduleReference` is an entity name for a local alias instead, which diff --git a/src/captures/TsharkCaptureProvider.ts b/src/captures/TsharkCaptureProvider.ts index d22515929..c4c89b88e 100644 --- a/src/captures/TsharkCaptureProvider.ts +++ b/src/captures/TsharkCaptureProvider.ts @@ -8,6 +8,8 @@ import { FIELD_RECORD_BUDGET, } from "./parseFieldDiscovery.js"; import { + BOUNDED_PRODUCER_LIMITS as LIMITS, + prlimitArgumentVector, readLinuxResourceLimits, type LinuxResourceLimits, } from "../process/LinuxResourceLimits.js"; @@ -60,15 +62,6 @@ export const TSHARK_PROVIDER_IDENTITY = { name: "REA TShark capture adapter", version: "tshark@4", } as const; -// Bound the resident input copy and retained text before allocation/collection. -const LIMITS = { - input_bytes: 32 * 1024 * 1024, - output_bytes: 4 * 1024 * 1024, - command_timeout_ms: 30_000, - address_space_bytes: 1024 * 1024 * 1024, - file_bytes: 64 * 1024 * 1024, - cpu_seconds: 30, -} as const; type Launcher = NonNullable[2]>["launcher"]; /** Inspect selected capture files only, retaining cleanup ownership across failures. */ @@ -165,17 +158,10 @@ export class TsharkCaptureProvider implements PacketCapturePort { options?.signal, ); effectiveLimits = await this.#resources.readLimits(limits); - const limitArguments = [ - `--as=${effectiveLimits.address_space_bytes}:${effectiveLimits.address_space_bytes}`, - `--fsize=${effectiveLimits.file_bytes}:${effectiveLimits.file_bytes}`, - `--cpu=${effectiveLimits.cpu_seconds}:${effectiveLimits.cpu_seconds}`, - "--core=0:0", - "--", - command, - ]; + const limitArguments = prlimitArgumentVector(effectiveLimits, command); phase = "workspace"; root = await this.createRuntime(); - this.#resources.retain(root, undefined, operation); + this.#resources.retain(root, undefined); const snapshotPath = join(root.path, "capture"); const configurationPath = join(root.path, "configuration"); await mkdir(configurationPath, { mode: 0o700 }); @@ -406,7 +392,7 @@ export class TsharkCaptureProvider implements PacketCapturePort { cause instanceof OwnedCommandFailure && cause.cleanupOwner !== undefined ) - this.#resources.retain(root, cause.cleanupOwner, operation); + this.#resources.retain(root, cause.cleanupOwner); result = err( this.#failure( cause, diff --git a/src/cli/androidCommands.ts b/src/cli/androidCommands.ts index 80a71b235..540dd295d 100644 --- a/src/cli/androidCommands.ts +++ b/src/cli/androidCommands.ts @@ -99,13 +99,20 @@ export const registerAndroidCommands = ( }), }); cli.command(CLI_COMMANDS.traceAndroidReferences, { - description: "Trace incoming APK class or unique-method references", + description: + "Trace incoming APK using nodes for a class or selected method", args: z.object({ path, className }), options: z.object({ methodName: z .string() .optional() - .describe("Unique declared method name; omit for class references"), + .describe("Declared method name; omit for class references"), + overloadIndex: z + .number() + .int() + .nonnegative() + .optional() + .describe("Index among same-name methods from inspect-android-class"), }), run: ({ args, options }) => execute(CLI_COMMANDS.traceAndroidReferences, "trace_android_references", { @@ -114,6 +121,9 @@ export const registerAndroidCommands = ( ...(options.methodName === undefined ? {} : { method_name: options.methodName }), + ...(options.overloadIndex === undefined + ? {} + : { overload_index: options.overloadIndex }), }), }); }; diff --git a/src/cli/apktoolCommands.ts b/src/cli/apktoolCommands.ts index c7795f9d4..217889f5a 100644 --- a/src/cli/apktoolCommands.ts +++ b/src/cli/apktoolCommands.ts @@ -6,7 +6,10 @@ import { createApktoolResourceAnalysisProvider } from "../composition/apktool.js import { CLI_COMMANDS } from "../cliCommandNames.js"; import { logCliCommand } from "../cliLogging.js"; import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; -import type { ApktoolOperation } from "../domain/apktool/apktoolResourceAnalysis.js"; +import { + apktoolInputSchemas, + type ApktoolOperation, +} from "../domain/apktool/apktoolResourceAnalysis.js"; import type { Logger } from "pino"; import type { CliInstance } from "./types.js"; import { withCommandCancellation } from "./commandCancellation.js"; @@ -49,22 +52,19 @@ export const registerApktoolCommands = ( args: z.object({ path: z.string().min(1).describe("Local APK file to decode"), }), + // The locale grammar and the string-projection default belong to the + // apktool input owner. Read them from that schema rather than restating a + // narrower locale pattern, which refused the owner's own advertised + // `b+zh+Hans+CN` form at the CLI boundary. options: z.object({ - locale: z - .string() - .min(2) - .max(16) - .regex( - /^[a-z]{2,3}(-[A-Za-z0-9]{2,8})*$/u, - "Locale must look like de or pt-BR", - ) - .optional() - .describe( - "Project strings from res/values- instead of the default", - ), + locale: apktoolInputSchemas.decode_android_resources.shape.locale, strings: z .boolean() - .default(true) + .default( + apktoolInputSchemas.decode_android_resources.shape.include_strings.parse( + undefined, + ), + ) .describe( "Include string resources; use --no-strings to skip projecting them", ), diff --git a/src/cli/artifactCommands.ts b/src/cli/artifactCommands.ts index 6b6b452e4..af2eab068 100644 --- a/src/cli/artifactCommands.ts +++ b/src/cli/artifactCommands.ts @@ -225,7 +225,7 @@ const registerInspectionCommand = ( description: "Inspect one artifact and return all available observations and next probes", args: z.object({ - path: z.string().describe("Application or package path"), + path: z.string().min(1).describe("Application or package path"), }), options: z.object({ integrityPolicy: z diff --git a/src/cli/browserAdvancedCommands.ts b/src/cli/browserAdvancedCommands.ts index 50c9ce446..dc0c268d1 100644 --- a/src/cli/browserAdvancedCommands.ts +++ b/src/cli/browserAdvancedCommands.ts @@ -46,7 +46,9 @@ const registerWebMcp = ( cli.command(CLI_COMMANDS.discoverWebMcpTools, { description: "Passively discover page-declared WebMCP tools", args: z.object({ - endpoint: z.string().describe("Configured loopback CDP HTTP endpoint"), + endpoint: z + .string() + .describe("Loopback CDP HTTP endpoint or browser WebSocket URL"), targetId: z.string().describe("Target ID from list-browser-targets"), }), options: z.object({ @@ -150,7 +152,9 @@ const registerScreenshot = ( description: "Capture one visible page viewport within the supplied origin scope", args: z.object({ - endpoint: z.string().describe("Configured loopback CDP HTTP endpoint"), + endpoint: z + .string() + .describe("Loopback CDP HTTP endpoint or browser WebSocket URL"), targetId: z.string().describe("Target ID from list-browser-targets"), }), options: z.object({ diff --git a/src/cli/browserCommands.ts b/src/cli/browserCommands.ts index aae6714d2..e9043ec80 100644 --- a/src/cli/browserCommands.ts +++ b/src/cli/browserCommands.ts @@ -42,7 +42,9 @@ const registerTargetList = ( description: "List page targets from a selected loopback CDP browser, optionally filtered by origin", args: z.object({ - endpoint: z.string().describe("Configured loopback CDP HTTP endpoint"), + endpoint: z + .string() + .describe("Loopback CDP HTTP endpoint or browser WebSocket URL"), }), options: z.object({ ...browserScopeOptions }), run: ({ args, options }) => @@ -75,7 +77,9 @@ const registerPageInspection = ( description: "Passively inspect one page through CDP, defaulting to its current origin", args: z.object({ - endpoint: z.string().describe("Configured loopback CDP HTTP endpoint"), + endpoint: z + .string() + .describe("Loopback CDP HTTP endpoint or browser WebSocket URL"), targetId: z.string().describe("Target ID from list-browser-targets"), }), options: browserPageInspectionOptions, @@ -118,7 +122,9 @@ const registerBundleAnalysis = ( description: "Capture and statically analyze a page bundle, defaulting to the selected target origin", args: z.object({ - endpoint: z.string().describe("Configured loopback CDP HTTP endpoint"), + endpoint: z + .string() + .describe("Loopback CDP HTTP endpoint or browser WebSocket URL"), targetId: z.string().describe("Target ID from list-browser-targets"), }), options: z.object({ @@ -162,7 +168,9 @@ const registerObservationSession = ( cli.command(CLI_COMMANDS.observeWebSession, { description: "Observe user-driven page navigation during a chosen window", args: z.object({ - endpoint: z.string().describe("Configured loopback CDP HTTP endpoint"), + endpoint: z + .string() + .describe("Loopback CDP HTTP endpoint or browser WebSocket URL"), targetId: z.string().describe("Target ID from list-browser-targets"), }), options: z.object({ diff --git a/src/cli/coreBinaryCommands.ts b/src/cli/coreBinaryCommands.ts index 710b39181..8902cf4e9 100644 --- a/src/cli/coreBinaryCommands.ts +++ b/src/cli/coreBinaryCommands.ts @@ -48,7 +48,10 @@ const registerOverviewCommands = ( cli.command(CLI_COMMANDS.analyze, { description: "Get an overview of an app", args: z.object({ - path: z.string().describe("App, program, or analysis database path"), + path: z + .string() + .min(1) + .describe("App, program, or analysis database path"), }), options: overviewOptions.extend({ integrityPolicy: javascriptApplicationOptions.shape.integrityPolicy, diff --git a/src/cli/coreNativeCommands.ts b/src/cli/coreNativeCommands.ts index 016eedbc0..80d07187f 100644 --- a/src/cli/coreNativeCommands.ts +++ b/src/cli/coreNativeCommands.ts @@ -17,6 +17,46 @@ export const registerCoreNativeCommands = ( runDirectAnalysis: DirectAnalysis["runDirectAnalysis"], ): void => { registerAnnotationCommand(cli, logger, runDirectAnalysis); + cli.command(CLI_COMMANDS.searchBytes, { + description: "Find masked byte patterns in initialized provider memory", + args: z.object({ + path: z.string().describe("Local executable path"), + pattern: z.string().describe("Hexadecimal byte pairs with ?? wildcards"), + }), + options: z.object({ + mask: z.string().optional().describe("Equal-length hexadecimal bit mask"), + start: z.string().optional().describe("Inclusive start address"), + end: z.string().optional().describe("Inclusive end address"), + cursor: z + .string() + .optional() + .describe("Continuation from the same query's next_cursor"), + limit: z + .number() + .int() + .min(1) + .max(1000) + .default(100) + .describe("Maximum matches on this page"), + ...nativeOpenOptionFields, + }), + run: ({ args, options }) => + logCliCommand(logger, CLI_COMMANDS.searchBytes, () => + runDirectAnalysis( + args.path, + "search_bytes", + { + pattern: args.pattern, + limit: options.limit, + ...(options.mask === undefined ? {} : { mask: options.mask }), + ...(options.start === undefined ? {} : { start: options.start }), + ...(options.end === undefined ? {} : { end: options.end }), + ...(options.cursor === undefined ? {} : { cursor: options.cursor }), + }, + directAnalysisOptionsFromCli(logger, options), + ), + ), + }); cli.command(CLI_COMMANDS.inspectNativeLoadImage, { description: "Verify loaded native bytes, source mappings, relocations and entry", diff --git a/src/cli/utilityCommands.ts b/src/cli/utilityCommands.ts index 33099ee28..a5da57ac5 100644 --- a/src/cli/utilityCommands.ts +++ b/src/cli/utilityCommands.ts @@ -2,6 +2,7 @@ import { z } from "incur"; import type { DirectAnalysis } from "../composition/directAnalysis.js"; import type { ReferenceSourcePolicy } from "../domain/referenceSourcePolicy.js"; +import { createHistoricalSourceSummary } from "../domain/referenceSourceSummary.js"; import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js"; import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; import { importReferenceSource } from "../application/ReferenceSourceImport.js"; @@ -32,89 +33,116 @@ export const registerUtilityCommands = ( registerCapabilityCommands(cli, logger, analysis); registerNativeCommands(cli, logger, runProviderAnalysis); registerNativeCallCommands(cli, logger, runProviderAnalysis); - for (const [command, operation] of [ - [CLI_COMMANDS.observeNativeUi, "observe_native_ui"], - [CLI_COMMANDS.captureNativeUiScenario, "capture_native_ui_scenario"], - ] as const) { - cli.command(command, { - description: - "Observe or run a scenario in one exact native application window", - args: z.object({ - path: z.string().describe("Local native executable or app path"), - }), - options: z.object({ - pid: z.number().int().positive().describe("Existing application PID"), - windowId: z - .number() - .int() - .positive() - .describe("Exact selected application window ID"), - steps: z - .string() - .optional() - .describe("JSON array of declarative AX actions or waits"), - screenshot: z - .boolean() - .default(true) - .describe("Capture only the selected window"), - accessibility: z - .boolean() - .default(true) - .describe("Read the selected window accessibility tree"), - maxNodes: z - .number() - .int() - .positive() - .safe() - .default(500) - .describe("Maximum accessibility nodes per capture"), - }), - run: ({ args, options }) => - logCliCommand(logger, command, async () => { - const parameters = { - pid: options.pid, - window_id: options.windowId, - screenshot: options.screenshot, - accessibility: options.accessibility, - max_nodes: options.maxNodes, - }; - if (operation === "capture_native_ui_scenario") { - const decoded = - options.steps === undefined - ? { ok: true as const, value: undefined } - : safeParseJson(options.steps); - if (!decoded.ok) - return invalidNativeUiScenarioInput(operation, [ - { path: ["steps"], reason: "invalid_format", expected: "JSON" }, - ]); - const parsed = nativeUiScenarioInputSchema.safeParse({ - ...parameters, - steps: decoded.value, - }); - if (!parsed.success) - return invalidNativeUiScenarioInput( - operation, - projectInputIssues(parsed.error.issues, { - ...parameters, - ...(options.steps === undefined - ? {} - : { steps: decoded.value }), - }), - ); - return runProviderAnalysis( - args.path, - operation, - parsed.data, - logger, - ); - } - return runProviderAnalysis(args.path, operation, parameters, logger); - }), - }); - } + registerNativeUiCommands(cli, logger, runProviderAnalysis); registerReferenceSourceCommand(cli, logger, environment); }; +const nativeUiScopeOptions = { + pid: z.number().int().positive().describe("Existing application PID"), + windowId: z + .number() + .int() + .positive() + .describe("Exact selected application window ID"), + screenshot: z + .boolean() + .default(true) + .describe("Capture only the selected window"), + accessibility: z + .boolean() + .default(true) + .describe("Read the selected window accessibility tree"), + maxNodes: z + .number() + .int() + .positive() + .safe() + .default(500) + .describe("Maximum accessibility nodes per capture"), +}; + +const nativeUiScope = (options: { + readonly pid: number; + readonly windowId: number; + readonly screenshot: boolean; + readonly accessibility: boolean; + readonly maxNodes: number; +}) => ({ + pid: options.pid, + window_id: options.windowId, + screenshot: options.screenshot, + accessibility: options.accessibility, + max_nodes: options.maxNodes, +}); + +/** + * Passive observation and active scenarios have different input contracts. + * Only the scenario command accepts steps, so passive observation rejects + * them as an unknown flag instead of silently dropping them. + */ +const registerNativeUiCommands = ( + cli: CliInstance, + logger: Logger, + runProviderAnalysis: DirectAnalysis["runProviderAnalysis"], +): void => { + const pathArgument = z.object({ + path: z.string().describe("Local native executable or app path"), + }); + cli.command(CLI_COMMANDS.observeNativeUi, { + description: + "Observe one already-running native application window by PID and window ID", + args: pathArgument, + options: z.object(nativeUiScopeOptions), + run: ({ args, options }) => + logCliCommand(logger, CLI_COMMANDS.observeNativeUi, () => + runProviderAnalysis( + args.path, + "observe_native_ui", + nativeUiScope(options), + logger, + ), + ), + }); + const operation = "capture_native_ui_scenario"; + cli.command(CLI_COMMANDS.captureNativeUiScenario, { + description: + "Run declarative AX actions or waits in one exact native application window", + args: pathArgument, + options: z.object({ + ...nativeUiScopeOptions, + steps: z + .string() + .optional() + .describe("JSON array of declarative AX actions or waits"), + }), + run: ({ args, options }) => + logCliCommand(logger, CLI_COMMANDS.captureNativeUiScenario, async () => { + const parameters = nativeUiScope(options); + const decoded = + options.steps === undefined + ? { ok: true as const, value: undefined } + : safeParseJson(options.steps); + if (!decoded.ok) + return invalidNativeUiScenarioInput(operation, [ + { path: ["steps"], reason: "invalid_format", expected: "JSON" }, + ]); + const parsed = nativeUiScenarioInputSchema.safeParse({ + ...parameters, + steps: decoded.value, + }); + if (!parsed.success) + return invalidNativeUiScenarioInput( + operation, + projectInputIssues(parsed.error.issues, { + ...parameters, + ...(options.steps === undefined ? {} : { steps: decoded.value }), + }), + ); + return runProviderAnalysis(args.path, operation, parsed.data, logger); + }), + }); +}; + const invalidNativeUiScenarioInput = ( operation: string, issues: ConstructorParameters[2], @@ -214,11 +242,48 @@ const registerReferenceSourceCommand = ( .string() .describe("Local source directory to import as historical reference"), }), - run: ({ args }) => + options: z.object({ + detail: z + .enum(["full", "summary"]) + .default("full") + .describe( + "Complete committed graph or compact historical-source summary", + ), + outlineDepth: z + .number() + .int() + .nonnegative() + .safe() + .optional() + .describe( + "Summary outline path depth (default: 2; 0 omits entries); requires --detail summary", + ), + }), + run: ({ args, options }) => logCliCommand( logger, "import-reference-source", async () => { + if ( + options.outlineDepth !== undefined && + options.detail !== "summary" + ) + return { + error: "Import failed", + ...projectAnalysisError( + new AnalysisInputError( + "import-reference-source", + { userMessage: "--outline-depth requires --detail summary" }, + [ + { + path: ["outlineDepth"], + reason: "invalid_value", + message: "--outline-depth requires --detail summary", + }, + ], + ), + ), + }; const config = parseConfig(environment); if (!config.ok) return { @@ -228,6 +293,9 @@ const registerReferenceSourceCommand = ( return runReferenceSourceImport( args.root, config.value.referenceSourcePolicy, + options.detail === "summary" + ? (options.outlineDepth ?? 2) + : undefined, ); }, isReferenceSourceImportCliFailure, @@ -238,6 +306,7 @@ const registerReferenceSourceCommand = ( const runReferenceSourceImport = async ( root: string, policy: ReferenceSourcePolicy, + outlineDepth: number | undefined, ): Promise => { const resources = new ArtifactResourceScope(); let outcome: @@ -319,7 +388,9 @@ const runReferenceSourceImport = async ( cause: closeOutcome.cause, }), }; - return imported.value; + return outlineDepth === undefined + ? imported.value + : createHistoricalSourceSummary(imported.value, outlineDepth); }; const mergeReferenceSourceCleanup = ( diff --git a/src/cli/wasmCommands.ts b/src/cli/wasmCommands.ts index b91bda033..73b8f8d3f 100644 --- a/src/cli/wasmCommands.ts +++ b/src/cli/wasmCommands.ts @@ -1,3 +1,6 @@ +import { compareWasmInterfaces } from "../application/wasm/WasmInterfaceComparisonService.js"; +import { parseCliJsonInput } from "../cliJsonInput.js"; +import { CLI_COMMANDS } from "../cliCommandNames.js"; import { z } from "incur"; import { resolve } from "node:path"; import { wasmArtifactModule } from "../composition/wasm.js"; @@ -13,6 +16,29 @@ export const registerWasmCommands = ( logger: Logger, environment: Readonly, ): void => { + cli.command(CLI_COMMANDS.compareWasmInterfaces, { + description: + "Compare core WASM interfaces from two exact inspection Evidence records", + args: z.object({ + input: z + .string() + .describe( + "Inline JSON or input file containing left/right complete Evidence and required scope", + ), + }), + run: ({ args }) => + withCommandCancellation(async (signal) => + logCliCommand(logger, CLI_COMMANDS.compareWasmInterfaces, async () => { + const input = await parseCliJsonInput( + args.input, + "compare_wasm_interfaces", + ); + if (!input.ok) return input.error; + const result = await compareWasmInterfaces(input.value, { signal }); + return result.ok ? result.value : projectAnalysisError(result.error); + }), + ), + }); const service = wasmArtifactModule.create(environment); cli.command(wasmArtifactModule.cliCommand, { description: "Inspect an explicit local WASM artifact with WABT", diff --git a/src/cliCommandNames.ts b/src/cliCommandNames.ts index d63a9a2bc..ff208c62e 100644 --- a/src/cliCommandNames.ts +++ b/src/cliCommandNames.ts @@ -4,6 +4,7 @@ export const CLI_COMMANDS = Object.freeze({ inspectPacketCapture: "inspect-packet-capture", inspectSignalDecoders: "inspect-signal-decoders", decodeSignalRecording: "decode-signal-recording", + compareWasmInterfaces: "compare-wasm-interfaces", inspectWasmArtifact: "inspect-wasm-artifact", inspectSqliteDatabase: "inspect-sqlite-database", inspectEvmInterface: "inspect-evm-interface", @@ -39,6 +40,7 @@ export const CLI_COMMANDS = Object.freeze({ inspectNativeDataType: "inspect-native-data-type", inspectNativeLoadImage: "inspect-native-load-image", readBytes: "read-bytes", + searchBytes: "search-bytes", addressToFileOffset: "address-to-file-offset", traceNativeValues: "trace-native-values", inspectNativeInstruction: "inspect-native-instruction", diff --git a/src/cliJsonInput.ts b/src/cliJsonInput.ts index 4dd38a90e..cf1b6ea72 100644 --- a/src/cliJsonInput.ts +++ b/src/cliJsonInput.ts @@ -65,12 +65,19 @@ const accessDeniedSystemCode = ( : undefined; }; +/** + * A `file:` URL or `scheme://` URL is not an operator-relative filesystem path. + * One-letter prefixes stay paths so Windows drive paths resolve normally. + */ +const URL_REFERENCE = /^(?:file:|[A-Za-z][A-Za-z0-9+.-]+:\/\/)/iu; + /** * Resolve named string fields of CLI-supplied JSON against the operator * working directory. Each entry is a key path from the document root; only - * fields holding nonblank strings are resolved, everything else passes through - * untouched so shared schemas can reject blank paths. Shared MCP contracts reject - * relative paths, so CLI workflows resolve operator-relative values before validation. + * fields holding nonblank, non-URL strings are resolved, everything else passes + * through untouched so shared schemas can reject blank paths and URLs. Shared MCP + * contracts reject relative paths, so CLI workflows resolve operator-relative + * values before validation. */ export const resolveCliJsonPaths = ( value: unknown, @@ -94,7 +101,9 @@ const resolveJsonPath = ( if (tail.length === 0) return [ key, - typeof entry === "string" && entry.trim().length > 0 + typeof entry === "string" && + entry.trim().length > 0 && + !URL_REFERENCE.test(entry) ? resolve(entry) : entry, ]; diff --git a/src/cliJsonPaths.test.ts b/src/cliJsonPaths.test.ts index 936bfacea..ed9e18642 100644 --- a/src/cliJsonPaths.test.ts +++ b/src/cliJsonPaths.test.ts @@ -56,4 +56,38 @@ describe("CLI JSON path normalization", () => { }); expect(input.browser.executable_path).toBe(" browser "); }); + + it.each([ + "file:///opt/electron", + "FILE:electron", + "https://example.test/app", + ])("keeps URL %j distinguishable from a local path", (value) => { + const input = { + executable_path: value, + application_path: resolve("/synthetic/app"), + }; + const normalized = resolveCliJsonPaths(input, paths); + expect(normalized).toEqual(input); + const parsed = electronActiveObservationInputSchema.safeParse(normalized); + expect(parsed.success).toBe(false); + if (parsed.success) throw new Error("Expected URL path rejection"); + expect(parsed.error.issues).toEqual([ + expect.objectContaining({ + path: ["executable_path"], + message: expect.stringContaining( + "must be absolute local filesystem paths", + ), + }), + ]); + }); + + it.each([ + "bin/electron", + "release:v1/electron", + "C:\\Electron\\electron.exe", + ])("keeps resolving local path %j", (value) => { + expect(resolveCliJsonPaths({ executable_path: value }, paths)).toEqual({ + executable_path: resolve(value), + }); + }); }); diff --git a/src/contracts/android/androidToolContracts.ts b/src/contracts/android/androidToolContracts.ts index b555fcbea..2710e0faa 100644 --- a/src/contracts/android/androidToolContracts.ts +++ b/src/contracts/android/androidToolContracts.ts @@ -51,7 +51,7 @@ export const ANDROID_TOOL_CONTRACTS = [ name: "inspect_android_class", ...toolContractMetadata("inspect_android_class"), description: - "Inspect one exact Android class's parsed methods, fields and inner classes, including synthetic members. Returns display signatures and explicit unknown DEX descriptors without generating source. Overload indices remain consistent within the artifact and engine session.", + "Inspect one exact Android class's parsed methods, fields and inner classes, including synthetic members. Returns display signatures and exact DEX prototypes without generating source. A prototype is not a complete raw method ID; overload indices remain consistent within the artifact and engine session.", kind: "android-provider", inputSchema: androidInputSchemas.inspect_android_class, outputSchema: evidenceResultOf(androidResultSchemas.inspect_android_class), @@ -66,7 +66,7 @@ export const ANDROID_TOOL_CONTRACTS = [ name: "inspect_android_method", ...toolContractMetadata("inspect_android_method"), description: - "Decompile one Android method with smali fallback. Requires explicit overload_index when multiple methods share a name. Reports representation, fallback markers and source truncation; never treats display signatures as exact DEX identities.", + "Decompile one Android method with smali fallback. Requires explicit overload_index when multiple methods share a name. Reports its DEX prototype, representation, fallback markers and source truncation; display signatures may be aliases.", kind: "android-provider", inputSchema: androidInputSchemas.inspect_android_method, outputSchema: evidenceResultOf(androidResultSchemas.inspect_android_method), @@ -85,7 +85,7 @@ export const ANDROID_TOOL_CONTRACTS = [ name: "trace_android_references", ...toolContractMetadata("trace_android_references"), description: - "Trace incoming static references to an exact Android class or unique method. Overloaded method reference queries are explicitly unsupported by the current engine. Source lines and DEX offsets remain unknown; no runtime calls are inferred.", + "Trace incoming static using nodes to an exact Android class or selected method overload. The total counts containing nodes, not individual invoke instructions. Use inspect_android_class to choose an overload_index when names collide. Source lines and the containing node dex_offset remain unknown. direct_invokes carries exact DEX code-unit instruction offsets when available; direct_invokes_exhaustive false marks byte-bounded partial instruction evidence, and null means coverage is unknown. Using-node coverage remains separate; no runtime calls are inferred.", kind: "android-provider", inputSchema: androidInputSchemas.trace_android_references, outputSchema: evidenceResultOf( diff --git a/src/contracts/functionComparisonExample.ts b/src/contracts/functionComparisonExample.ts index 39dfe6f92..aefc75e1a 100644 --- a/src/contracts/functionComparisonExample.ts +++ b/src/contracts/functionComparisonExample.ts @@ -16,7 +16,7 @@ const dossier = (text: string) => locals: [], }, pseudocode: text, - assembly: [], + assembly: ["ret"], comments: [], callers: [], callees: [], @@ -27,6 +27,16 @@ const dossier = (text: string) => basic_blocks: [{ start: "0x1000", end: "0x1001", successors: [] }], native_api: null, native_value_flow: null, + facet_availability: { + pseudocode: { available: true }, + assembly: { available: true }, + comments: { available: true }, + callers: { available: true }, + callees: { available: true }, + references: { available: true }, + strings_names: { available: true }, + cfg: { available: true }, + }, limitations: ["The fixture omits native API and value-flow observations."], }); diff --git a/src/contracts/hopperMemoryContracts.ts b/src/contracts/hopperMemoryContracts.ts index e4c49788a..df7e43169 100644 --- a/src/contracts/hopperMemoryContracts.ts +++ b/src/contracts/hopperMemoryContracts.ts @@ -1,8 +1,27 @@ import { z } from "zod"; import { address, document } from "./toolContractHelpers.js"; +import { + bytePatternFields, + validateBytePattern, +} from "../domain/native/bytePattern.js"; /** Hopper memory and file-mapping contracts. */ export const HOPPER_MEMORY_TOOL_DEFINITIONS = [ + { + name: "search_bytes", + description: + "Find masked byte signatures in initialized analyzed memory. Hex byte pairs and ?? wildcards are supported; an optional equal-length hex mask selects bits. Bounds are inclusive. Matches are ordered by provider address and never span blocks. Coverage describes this page; reuse the same query with next_cursor to continue. Uninitialized blocks are listed separately. Ghidra only; Hopper is unavailable.", + inputSchema: z + .strictObject({ + ...bytePatternFields, + document, + start: address.optional(), + end: address.optional(), + limit: z.number().int().min(1).max(1000).default(100), + cursor: z.string().min(1).optional(), + }) + .superRefine(validateBytePattern), + }, { name: "read_bytes", description: diff --git a/src/contracts/inputSchemaPresentation.ts b/src/contracts/inputSchemaPresentation.ts index 9663dd832..22ad0ae30 100644 --- a/src/contracts/inputSchemaPresentation.ts +++ b/src/contracts/inputSchemaPresentation.ts @@ -30,6 +30,7 @@ const resolveReference = ( root: Readonly>, reference: string, ): unknown => { + if (reference === "#") return root; if (!reference.startsWith("#/")) return undefined; let value: unknown = root; for (const token of reference.slice(2).split("/")) { @@ -229,3 +230,76 @@ export const presentInputJsonSchema = ( const described = describeSchema(flattened, flattened, describeProperty); return isObject(described) ? described : flattened; }; + +/** + * Clients that render advertised inputs into SDK declarations expand + * schema-local references with a shared per-input budget: Pi 1.1.0 code-mode + * allows 32 expansions and degrades every later reference to `unknown` + * (morluto/rea#1632), hiding the fields those references describe. The budget + * counts conservatively — every resolvable, non-cyclic reference occurrence + * in every schema position — so any reference-resolving client stays within + * its measured limit. + */ +export const advertisedReferenceExpansionBudget = 32; + +const definitionKeys = new Set(["$defs", "definitions"]); + +const countReferenceExpansions = ( + node: unknown, + root: Readonly>, + active: ReadonlySet, + count: { value: number }, +): void => { + if (Array.isArray(node)) { + node.forEach((child) => + countReferenceExpansions(child, root, active, count), + ); + return; + } + if (!isObject(node)) return; + if (typeof node.$ref === "string" && !active.has(node.$ref)) { + const target = resolveReference(root, node.$ref); + if (target !== undefined) { + count.value += 1; + countReferenceExpansions( + target, + root, + new Set([...active, node.$ref]), + count, + ); + return; + } + } + for (const [key, value] of Object.entries(node)) { + if (definitionKeys.has(key)) continue; + if (schemaMaps.has(key) && isObject(value)) { + for (const child of Object.values(value)) + countReferenceExpansions(child, root, active, count); + } else if (schemaArrays.has(key) || schemaChildren.has(key)) { + countReferenceExpansions(value, root, active, count); + } + } +}; + +/** Conservative client expansion count over every schema position. */ +export const countLocalReferenceExpansions = ( + schema: Record, +): number => { + const count = { value: 0 }; + countReferenceExpansions(schema, schema, new Set(), count); + return count.value; +}; + +/** + * Keep the shared-definition form of an advertised input while clients can + * still resolve it. Once rendering would exhaust the reference budget, fall + * back to the inline projection, which carries no shared definitions for the + * nonrecursive schemas this repository advertises, so every field renders + * with complete structure in every reference-resolving client. + */ +export const inputSchemaWithinReferenceExpansionBudget = ( + shared: Record, + inline: () => Record, + budget: number, +): Record => + countLocalReferenceExpansions(shared) <= budget ? shared : inline(); diff --git a/src/contracts/managed/managedWorkflowToolContracts.ts b/src/contracts/managed/managedWorkflowToolContracts.ts index 3842c3814..519c2e125 100644 --- a/src/contracts/managed/managedWorkflowToolContracts.ts +++ b/src/contracts/managed/managedWorkflowToolContracts.ts @@ -1,5 +1,6 @@ import { z } from "zod"; import { evidenceSchema } from "../../domain/evidence.js"; +import { managedNativeVerificationInputSchema } from "../../domain/managed/managedNativeVerificationSchemas.js"; import { managedReconstructionImportInputSchema } from "../../domain/managed/managedReconstruction.js"; import type { ToolContract } from "../toolContractTypes.js"; @@ -34,35 +35,6 @@ export const managedReconstructionReferenceInputSchema = static_members: evidenceSchema, }); -/** Inline Evidence inputs for managed/native verification. */ -export const managedNativeVerificationReferenceInputSchema = z - .strictObject({ - managed_boundaries: evidenceSchema, - native_observations: z - .array(evidenceSchema) - .min(1) - .describe("Native function or export Evidence records"), - }) - .superRefine((input, context) => { - const ids = new Set([input.managed_boundaries.evidence_id]); - for (const [index, evidence] of input.native_observations.entries()) { - if (evidence.evidence_id === input.managed_boundaries.evidence_id) - context.addIssue({ - code: "custom", - path: ["native_observations", index], - message: - "Native observation Evidence must be distinct from managed boundary Evidence", - }); - if (ids.has(evidence.evidence_id)) - context.addIssue({ - code: "custom", - path: ["native_observations", index], - message: "Native observation Evidence IDs must be unique", - }); - ids.add(evidence.evidence_id); - } - }); - const managedApplicationGraphReferenceFacts = {} as const; const managedApplicationGraphReferenceShape = z.strictObject({ @@ -146,7 +118,7 @@ export const MANAGED_WORKFLOW_TOOL_CONTRACTS = [ description: "Verify managed P/Invoke/native-boundary declarations against authenticated native export or function Evidence without executing managed code or translating managed metadata tokens into native addresses. The workflow preserves declaration-only, verified, inferred, contradicted, and unresolved states.", kind: "application", - inputSchema: managedNativeVerificationReferenceInputSchema, + inputSchema: managedNativeVerificationInputSchema, outputSchema: nativeVerificationOutputSchema, examples: [ { diff --git a/src/contracts/native/binaryDiagnosticsToolContracts.ts b/src/contracts/native/binaryDiagnosticsToolContracts.ts index 64a80e42d..8c872f9d9 100644 --- a/src/contracts/native/binaryDiagnosticsToolContracts.ts +++ b/src/contracts/native/binaryDiagnosticsToolContracts.ts @@ -67,7 +67,7 @@ export const BINARY_DIAGNOSTICS_TOOL_CONTRACTS = [ ...toolContractMetadata("inspect_recorded_crash"), kind: "native-provider", description: - "Inspect a supplied Linux ELF64 amd64 little-endian core without launching its target or attaching to any live PID. Returns all recorded segments, raw note owners/descriptors and source ranges, every Linux CORE PRSTATUS thread/register and SIGINFO fields inline with SHA-256 Evidence. Historical PID is recorded metadata; current process/file identity, note coverage and signal-thread associations stay unknown. Optionally add derived core-only GDB/pwndbg mapping candidates through explicit BYO configuration; zero map flags mean unknown permissions. Uses the same stable snapshot for both stages, bounded complete output and owned cleanup. Initial Linux x64 profile: unchanged pwntools 4.15.0/pyelftools 0.33/Unicorn 2.1.2; optional unchanged pwndbg 2026.09.15. No implicit executable/library pairing, GDB command language, host configuration changes or dependency installation.", + "Inspect a supplied Linux ELF64 amd64 or AArch64 little-endian core without launching its target or attaching to any live PID. Returns all recorded segments, raw note owners/descriptors and source ranges, every Linux CORE PRSTATUS thread/register and SIGINFO fields inline with SHA-256 Evidence. Historical PID is recorded metadata; current process/file identity, note coverage and signal-thread associations stay unknown. Optionally add derived core-only GDB/pwndbg mapping candidates through explicit BYO configuration; zero map flags mean unknown permissions. Uses the same stable snapshot for both stages, bounded complete output and owned cleanup. Linux x64 host profile: unchanged pwntools 4.15.0/pyelftools 0.33/Unicorn 2.1.2; optional unchanged pwndbg 2026.09.15. No implicit executable/library pairing, GDB command language, host configuration changes or dependency installation.", inputSchema: inspectRecordedCrashInputSchema, outputSchema: evidenceResultOf(recordedCrashSchema), examples: [ diff --git a/src/contracts/toolContractExamples.ts b/src/contracts/toolContractExamples.ts index 66b0f3518..50a745e75 100644 --- a/src/contracts/toolContractExamples.ts +++ b/src/contracts/toolContractExamples.ts @@ -26,6 +26,7 @@ export const TOOL_EXAMPLE_OVERRIDES: Readonly< resolve_native_call_targets: { address: "0x1000" }, read_function_instructions: { procedure: "main" }, read_bytes: { address: "0x1000", length: 16 }, + search_bytes: { pattern: "48 8B ?? ?? 89" }, address_to_file_offset: { address: "0x1000" }, procedure_references: { procedure: "main" }, procedure_pseudo_code: { procedure: "main" }, diff --git a/src/contracts/toolEffects.test.ts b/src/contracts/toolEffects.test.ts index fe18c96b3..3a4acca99 100644 --- a/src/contracts/toolEffects.test.ts +++ b/src/contracts/toolEffects.test.ts @@ -27,6 +27,8 @@ const EXECUTION_WORDING_EXCEPTIONS: Readonly> = { inspect_macho: "runs host Mach-O inspection tools, not the target", inspect_evm_interface: "decodes bytecode in REA's bundled worker without executing it", + compare_wasm_interfaces: + "compares retained inspection Evidence without executing modules", inspect_wasm_artifact: "WABT validates and decompiles the module without executing it", identify_flutter_build: diff --git a/src/contracts/toolEffects.ts b/src/contracts/toolEffects.ts index 70a40c6a1..377905480 100644 --- a/src/contracts/toolEffects.ts +++ b/src/contracts/toolEffects.ts @@ -438,6 +438,7 @@ export const TOOL_EFFECTS: Readonly> = { inspect_native_data_type: evidence, resolve_native_call_targets: evidence, read_bytes: evidence, + search_bytes: evidence, address_to_file_offset: evidence, procedure_references: evidence, procedure_pseudo_code: evidence, @@ -622,6 +623,7 @@ export const TOOL_EFFECTS: Readonly> = { }), compare_process_captures: sessionEvidence, compare_artifacts: sessionEvidence, + compare_wasm_interfaces: sessionEvidence, compare_functions: sessionEvidence, compare_bundles: effects({ mutatesSession: true }), find_changed_behavior: sessionEvidence, diff --git a/src/contracts/toolInputReferences.test.ts b/src/contracts/toolInputReferences.test.ts index beb63b7e7..f75a06531 100644 --- a/src/contracts/toolInputReferences.test.ts +++ b/src/contracts/toolInputReferences.test.ts @@ -3,7 +3,12 @@ import { Ajv2020 } from "ajv/dist/2020.js"; import { describe, expect, it } from "vitest"; import { z } from "zod"; -import { presentInputJsonSchema } from "./inputSchemaPresentation.js"; +import { + advertisedReferenceExpansionBudget, + countLocalReferenceExpansions, + inputSchemaWithinReferenceExpansionBudget, + presentInputJsonSchema, +} from "./inputSchemaPresentation.js"; import { toolContract } from "./toolContracts.js"; import { toolInputSchemaWithMetadata } from "./toolSchemaMetadata.js"; import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "./javascript/javascriptRuntimeReconciliationExample.js"; @@ -166,6 +171,105 @@ describe("advertised input references", () => { ); }); +describe("advertised reference expansion budget", () => { + it("counts references in keyword-named fields, excluding literals", () => { + const fields = ["default", "enum", "examples", "const", "$defs"]; + const schema = { + type: "object", + properties: Object.fromEntries( + fields.map((name) => [name, { $ref: "#/$defs/value" }]), + ), + $defs: { value: { type: "string" } }, + default: { $ref: "#/$defs/value" }, + examples: [{ $ref: "#/$defs/value" }], + }; + expect(countLocalReferenceExpansions(schema)).toBe(fields.length); + }); + const reusedComponent = z + .object({ kind: z.literal("rec"), value: z.string() }) + .strict(); + const inputWithReusedFields = (count: number) => + z.strictObject( + Object.fromEntries( + Array.from({ length: count }, (_, index) => [ + `field_${String(index).padStart(2, "0")}`, + reusedComponent, + ]), + ), + ); + const advertisedInput = ( + canonical: z.ZodType>>, + ) => + toolInputSchemaWithMetadata({ + ...toolContract("open_binary"), + inputSchema: canonical, + })["~standard"].jsonSchema.input({ target: "draft-2020-12" }); + + it("keeps shared definitions while every client can resolve them", () => { + const advertised = advertisedInput(inputWithReusedFields(8)); + expect(advertised.$defs).toBeDefined(); + expect(countLocalReferenceExpansions(advertised)).toBe(8); + }); + + it("keeps shared definitions exactly at the budget", () => { + const advertised = advertisedInput( + inputWithReusedFields(advertisedReferenceExpansionBudget), + ); + expect(advertised.$defs).toBeDefined(); + expect(countLocalReferenceExpansions(advertised)).toBe( + advertisedReferenceExpansionBudget, + ); + }); + + it("re-projects without shared definitions once the budget is exceeded", () => { + const advertised = advertisedInput( + inputWithReusedFields(advertisedReferenceExpansionBudget + 1), + ); + expect(advertised.$defs).toBeUndefined(); + expect(countLocalReferenceExpansions(advertised)).toBe(0); + const properties = advertised.properties as Record; + expect(properties.field_00).toMatchObject({ + type: "object", + properties: { kind: { const: "rec" }, value: { type: "string" } }, + required: ["kind", "value"], + }); + }); + + it("keeps the shared form untouched when it fits and swaps it when not", () => { + const shared = { + type: "object", + properties: { value: { $ref: "#/$defs/shared" } }, + $defs: { shared: { type: "string" } }, + }; + const inlined = { + type: "object", + properties: { value: { type: "string" } }, + }; + expect( + inputSchemaWithinReferenceExpansionBudget( + shared, + () => inlined, + advertisedReferenceExpansionBudget, + ), + ).toBe(shared); + expect( + inputSchemaWithinReferenceExpansionBudget( + { + ...shared, + properties: Object.fromEntries( + Array.from({ length: 33 }, (_, index) => [ + `field_${index}`, + { $ref: "#/$defs/shared" }, + ]), + ), + }, + () => inlined, + advertisedReferenceExpansionBudget, + ), + ).toBe(inlined); + }); +}); + describe("root input presentation", () => { it("preserves prototype-named properties in object-root presentation", () => { const fields = Object.fromEntries([ diff --git a/src/contracts/toolOutputSchemaGroups.ts b/src/contracts/toolOutputSchemaGroups.ts index 76294e2db..1a36c0896 100644 --- a/src/contracts/toolOutputSchemaGroups.ts +++ b/src/contracts/toolOutputSchemaGroups.ts @@ -1,4 +1,5 @@ import { z } from "zod"; +import { byteSearchResultSchema } from "../domain/native/bytePattern.js"; import { artifactInventorySummaryViewResultSchema, machoSummaryViewResultSchema, @@ -108,6 +109,7 @@ const bookmarkFacetSchema = z.discriminatedUnion("state", [ /** Exact structured-content schemas shared by direct analysis providers. */ export const officialOutputSchemas: Readonly> = { + search_bytes: evidenceResultOf(byteSearchResultSchema), annotate_native_function: evidenceResultOf(nativeFunctionAnnotationsSchema), inspect_native_load_image: evidenceResultOf(nativeLoadImageSchema), inspect_native_data_type: evidenceResultOf(nativeDataTypeSchema), diff --git a/src/contracts/toolSchemaMetadata.ts b/src/contracts/toolSchemaMetadata.ts index 5ed63db5b..3034b8354 100644 --- a/src/contracts/toolSchemaMetadata.ts +++ b/src/contracts/toolSchemaMetadata.ts @@ -2,7 +2,11 @@ import { z } from "zod"; import type { ToolContract } from "./toolContractTypes.js"; import { advertisedInputExamples } from "./advertisedInputExamples.js"; -import { presentInputJsonSchema } from "./inputSchemaPresentation.js"; +import { + advertisedReferenceExpansionBudget, + inputSchemaWithinReferenceExpansionBudget, + presentInputJsonSchema, +} from "./inputSchemaPresentation.js"; import { digestCanonicalValue } from "../domain/canonicalDigest.js"; const PROPERTY_DESCRIPTIONS: Readonly> = { @@ -66,7 +70,25 @@ export const toolInputSchemaWithMetadata = ( }); const examples = advertisedInputExamples(contract); return { - ...presentInputJsonSchema(shared, fallbackPropertyDescription), + // Sharing definitions only helps clients that resolve them. A client + // rendering declarations from a shared expansion budget degrades + // every reference past it to `unknown`, so an input whose shared form + // exceeds the budget is re-projected without shared definitions. + ...inputSchemaWithinReferenceExpansionBudget( + presentInputJsonSchema(shared, fallbackPropertyDescription), + () => + presentInputJsonSchema( + project({ + ...options, + libraryOptions: { + ...options.libraryOptions, + reused: "inline", + }, + }), + fallbackPropertyDescription, + ), + advertisedReferenceExpansionBudget, + ), ...(examples.length === 0 ? {} : { examples }), }; }, diff --git a/src/contracts/wasm/wasmComparisonContracts.ts b/src/contracts/wasm/wasmComparisonContracts.ts new file mode 100644 index 000000000..7eae508b7 --- /dev/null +++ b/src/contracts/wasm/wasmComparisonContracts.ts @@ -0,0 +1,63 @@ +import { evidenceResultOf } from "../toolOutputSchemaPrimitives.js"; +import { z } from "zod"; +import { evidenceInputSchema } from "../evidenceInputContracts.js"; +import { wasmInterfaceSchema } from "../../domain/wasm/wasmInterface.js"; +import { toolContractMetadata } from "../toolEffects.js"; +import type { ToolContract } from "../toolContractTypes.js"; + +export const compareWasmInterfacesInputSchema = z.strictObject({ + left: evidenceInputSchema.describe( + "Exact complete inspect_wasm_artifact Evidence or retained session reference", + ), + right: evidenceInputSchema.describe( + "Exact complete inspect_wasm_artifact Evidence or retained session reference", + ), + scope: z + .enum(["imports", "exports", "both"]) + .describe( + "Explicit interface selection; static equality does not establish runtime compatibility", + ), +}); +const identity = z.object({ + evidence_id: z.string(), + artifact: z.object({ + path: z.string(), + sha256: z.string(), + bytes: z.number().int().nonnegative(), + }), +}); +export const wasmInterfaceComparisonSchema = z.object({ + left: identity, + right: identity, + scope: z.enum(["imports", "exports", "both"]), + coverage: z.enum(["complete-within-inputs", "partial", "unknown"]), + changes: z.array( + z.object({ + direction: z.enum(["import", "export"]), + module: z.string().nullable(), + name: z.string(), + state: z.enum(["added", "removed", "changed", "unchanged", "unknown"]), + left: wasmInterfaceSchema.shape.entries, + right: wasmInterfaceSchema.shape.entries, + reason: z.string().nullable(), + }), + ), + summary: z.object({ + added: z.number(), + removed: z.number(), + changed: z.number(), + unchanged: z.number(), + unknown: z.number(), + }), + limitations: z.array(z.string()), +}); +export const COMPARE_WASM_INTERFACES_CONTRACT = { + name: "compare_wasm_interfaces", + ...toolContractMetadata("compare_wasm_interfaces"), + kind: "application", + description: + "Compare typed core WASM import/export names, kinds and exact signatures from two authenticated inspect_wasm_artifact Evidence records. Select imports, exports or both explicitly. Retained references resolve only in the current session; CLI accepts portable inline Evidence. Duplicate names, unsupported types and missing historical interface facets remain unknown. Preserve original module digests and every occurrence; no source acquisition, module execution or runtime-compatibility inference.", + inputSchema: compareWasmInterfacesInputSchema, + outputSchema: evidenceResultOf(wasmInterfaceComparisonSchema), + examples: [], +} as const satisfies ToolContract; diff --git a/src/contracts/wasm/wasmToolContracts.ts b/src/contracts/wasm/wasmToolContracts.ts index 34f26f11d..cf05166d2 100644 --- a/src/contracts/wasm/wasmToolContracts.ts +++ b/src/contracts/wasm/wasmToolContracts.ts @@ -1,3 +1,4 @@ +import { COMPARE_WASM_INTERFACES_CONTRACT } from "./wasmComparisonContracts.js"; import { inspectWasmArtifactInputSchema, wasmArtifactSchema, @@ -40,4 +41,7 @@ export const INSPECT_WASM_ARTIFACT_CONTRACT = { } as const satisfies ToolContract; /** Keep the public catalog in its existing order, using its named contract. */ -export const WASM_TOOL_CONTRACTS = [INSPECT_WASM_ARTIFACT_CONTRACT] as const; +export const WASM_TOOL_CONTRACTS = [ + INSPECT_WASM_ARTIFACT_CONTRACT, + COMPARE_WASM_INTERFACES_CONTRACT, +] as const; diff --git a/src/contracts/workflowInputContracts.test.ts b/src/contracts/workflowInputContracts.test.ts index 32e1ddf02..237469487 100644 --- a/src/contracts/workflowInputContracts.test.ts +++ b/src/contracts/workflowInputContracts.test.ts @@ -1,5 +1,9 @@ import { describe, expect, it } from "vitest"; +import { managedNativeVerificationInputSchema } from "../domain/managed/managedNativeVerificationSchemas.js"; +import { toolContract } from "./toolContracts.js"; +import { toolInputSchemaWithMetadata } from "./toolSchemaMetadata.js"; + import { traceJavaScriptSemanticsRequestSchema } from "./javascript/applicationWorkflowInputContracts.js"; import { JAVASCRIPT_FEATURE_TRACE_EXAMPLE } from "./javascript/javascriptApplicationWorkflowExamples.js"; import { @@ -10,7 +14,6 @@ import { import { compareManagedMembersReferenceInputSchema, managedApplicationGraphReferenceInputSchema, - managedNativeVerificationReferenceInputSchema, } from "./managed/managedWorkflowToolContracts.js"; describe("workflow input contracts", () => { @@ -41,17 +44,6 @@ describe("workflow input contracts", () => { }).success, ).toBe(false); - expect( - managedNativeVerificationReferenceInputSchema.safeParse({ - managed_boundaries: - MANAGED_NATIVE_VERIFICATION_EXAMPLE.managed_boundaries, - native_observations: [ - MANAGED_NATIVE_VERIFICATION_EXAMPLE.native_observations[0], - MANAGED_NATIVE_VERIFICATION_EXAMPLE.native_observations[0], - ], - }).success, - ).toBe(false); - expect( managedApplicationGraphReferenceInputSchema.safeParse({ managed_artifact: MANAGED_APPLICATION_GRAPH_EXAMPLE.managed_members, @@ -62,4 +54,80 @@ describe("workflow input contracts", () => { managedApplicationGraphReferenceInputSchema.safeParse({}).success, ).toBe(false); }); + + it.each([ + { + name: "one observation reusing the boundary", + observations: [MANAGED_NATIVE_VERIFICATION_EXAMPLE.managed_boundaries], + issues: [ + { + path: ["native_observations", 0], + message: + "Native observation Evidence must be distinct from managed boundary Evidence", + }, + ], + }, + { + name: "repeated native observations", + observations: [ + MANAGED_NATIVE_VERIFICATION_EXAMPLE.native_observations[0], + MANAGED_NATIVE_VERIFICATION_EXAMPLE.native_observations[0], + ], + issues: [ + { + path: ["native_observations", 1], + message: "Native observation Evidence IDs must be unique", + }, + ], + }, + { + name: "repeated observations reusing the boundary", + observations: [ + MANAGED_NATIVE_VERIFICATION_EXAMPLE.managed_boundaries, + MANAGED_NATIVE_VERIFICATION_EXAMPLE.managed_boundaries, + ], + issues: [ + { + path: ["native_observations", 0], + message: + "Native observation Evidence must be distinct from managed boundary Evidence", + }, + { + path: ["native_observations", 1], + message: + "Native observation Evidence must be distinct from managed boundary Evidence", + }, + { + path: ["native_observations", 1], + message: "Native observation Evidence IDs must be unique", + }, + ], + }, + { + name: "distinct native observations", + observations: MANAGED_NATIVE_VERIFICATION_EXAMPLE.native_observations, + issues: [], + }, + ])( + "preserves managed/native diagnostics for $name", + async ({ observations, issues }) => { + const input = { + managed_boundaries: + MANAGED_NATIVE_VERIFICATION_EXAMPLE.managed_boundaries, + native_observations: observations, + }; + const contract = toolContract("verify_managed_native_boundaries"); + for (const schema of [ + managedNativeVerificationInputSchema, + contract.inputSchema, + toolInputSchemaWithMetadata(contract), + ]) { + const result = await schema["~standard"].validate(input); + expect( + result.issues?.map(({ path, message }) => ({ path, message })) ?? [], + ).toEqual(issues); + if (issues.length === 0) expect(result).toEqual({ value: input }); + } + }, + ); }); diff --git a/src/domain/analysisView/analysisView.test.ts b/src/domain/analysisView/analysisView.test.ts index 07e6909c3..4bf16e4b4 100644 --- a/src/domain/analysisView/analysisView.test.ts +++ b/src/domain/analysisView/analysisView.test.ts @@ -792,3 +792,29 @@ it("rejects unsupported parent operations and incompatible collections", () => { error: { _tag: "AnalysisInputError" }, }); }); + +it("preserves selected native facet unavailability instead of an exhausted empty page", () => { + const parent = nativeParent({ + ...functionDossierSchema.parse(ghidraFunctionDossier()), + facet_availability: { + comments: { available: false, reason: "Typed comments unavailable" }, + references: { available: false, reason: "Typed references unavailable" }, + }, + }); + for (const [facet, reason] of [ + ["comments", "Typed comments unavailable"], + ["incoming_references", "Typed references unavailable"], + ] as const) { + const result = projectAnalysisView(parent, { + kind: "native", + facet, + offset: 0, + limit: 8, + }); + if (!result.ok) throw result.error; + expect(result.value).toMatchObject({ + item: { available: false, reason }, + unknowns: expect.arrayContaining([reason]), + }); + } +}); diff --git a/src/domain/analysisView/nativeFunctionView.ts b/src/domain/analysisView/nativeFunctionView.ts index b56de24a1..cac7ef82f 100644 --- a/src/domain/analysisView/nativeFunctionView.ts +++ b/src/domain/analysisView/nativeFunctionView.ts @@ -110,6 +110,22 @@ export const projectNativeFunctionView = ( ]), ); const common = parentFields(parent, dossier, artifact); + const availabilityKeys: Readonly> = { + incoming_references: "references", + outgoing_references: "references", + referenced_strings: "strings_names", + referenced_names: "strings_names", + basic_blocks: "cfg", + }; + const availabilityKey = availabilityKeys[view.facet] ?? view.facet; + const availability = + dossier.facet_availability?.[ + availabilityKey as keyof NonNullable< + FunctionDossier["facet_availability"] + > + ]; + if (availability?.available === false) + common.unknowns = uniqueSorted([...common.unknowns, availability.reason]); const singleton = (item: unknown): UnsignedAnalysisView => ({ kind: "native", view, @@ -117,6 +133,7 @@ export const projectNativeFunctionView = ( coverage: completeWithinViewCoverage(1, 1), ...common, }); + if (availability?.available === false) return ok(singleton(availability)); const flow = dossier.native_value_flow; if ( view.facet === "procedure" || diff --git a/src/domain/android/androidAnalysis.ts b/src/domain/android/androidAnalysis.ts index ca4d6db59..caa3bf5ee 100644 --- a/src/domain/android/androidAnalysis.ts +++ b/src/domain/android/androidAnalysis.ts @@ -40,7 +40,15 @@ export const androidInputSchemas = { .min(1) .optional() .describe( - "Omit for class references; ambiguous overloaded methods are unsupported", + "Omit for class references; select ambiguous methods with overload_index", + ), + overload_index: z + .number() + .int() + .nonnegative() + .optional() + .describe( + "Index among same-name methods from inspect_android_class; requires method_name", ), }), } as const; @@ -90,12 +98,19 @@ const textCoverage = z.strictObject({ text: z.string(), reported_total_bytes: z.number().int().nonnegative().nullable(), }); +/** A DEX prototype: parameter descriptors followed by the return descriptor. */ +export const dexDescriptorSchema = z + .string() + .regex( + /^\((?:\[*(?:[ZBSCIJFD]|L[^;\x28\x29]+;))*\)(?:V|\[*(?:[ZBSCIJFD]|L[^;\x28\x29]+;))$/u, + ); + const method = z.strictObject({ name: z.string(), reported_signature: z.string(), overload_index: z.number().int().nonnegative(), is_constructor: z.boolean(), - dex_descriptor: z.null(), + dex_descriptor: dexDescriptorSchema, }); const engine = z.strictObject({ name: z.string().min(1), @@ -159,6 +174,9 @@ export const androidResultSchemas = { ...common, class_name: z.string(), method_name: z.string().nullable(), + overload_index: z.number().int().nonnegative().nullable(), + reported_signature: z.string().nullable(), + dex_descriptor: dexDescriptorSchema.nullable(), total: z.number().int().nonnegative(), references: z.array( z.strictObject({ @@ -169,8 +187,23 @@ export const androidResultSchemas = { top_class: z.string(), source_line: z.null(), dex_offset: z.null(), + /** false means transport-bounded instruction evidence; null/omitted is unknown. */ + direct_invokes_exhaustive: z.boolean().nullable().default(null), + /** Raw direct invoke instructions inside this using node, when available. */ + direct_invokes: z + .array( + z.strictObject({ + dex_entry: z.string().min(1), + source_raw_method_id: z.string().min(1), + dex_code_unit_offset: z.number().int().nonnegative(), + dex_method_index: z.number().int().nonnegative(), + opcode: z.string().regex(/^INVOKE_/u), + }), + ) + .nullable(), }), ), coverage: z.literal("complete"), + reference_unit: z.literal("using_node"), }), } as const; diff --git a/src/domain/apple/appleApplication.test.ts b/src/domain/apple/appleApplication.test.ts index 9caf21fcb..3f8906e2b 100644 --- a/src/domain/apple/appleApplication.test.ts +++ b/src/domain/apple/appleApplication.test.ts @@ -417,6 +417,34 @@ describe("Apple bundle layouts and containers", () => { }); }); +describe("Apple IPA components", () => { + it("excludes AppleDouble sidecars from IPA components", () => { + const result = project( + inventoryEvidence("ipa", "Fixture.ipa", [ + { path: "Payload/Fixture.app/Info.plist" }, + macho("Payload/Fixture.app/Fixture"), + { path: "Payload/Fixture.app/Contents/Resources/._libfoo.dylib" }, + { path: "Payload/Fixture.app/Frameworks/Foo.framework/._Foo" }, + { path: "__MACOSX/Payload/Fixture.app/libghost.dylib" }, + macho("Payload/Fixture.app/Contents/Resources/libreal.dylib"), + macho("SwiftSupport/iphoneos/libswiftCore.dylib"), + ]), + ); + expect(result.components.native_libraries.map(({ path }) => path)).toEqual([ + "Payload/Fixture.app/Contents/Resources/libreal.dylib", + "SwiftSupport/iphoneos/libswiftCore.dylib", + ]); + expect(result.components.frameworks).toEqual([]); + expect(result.components.executables.map(({ path }) => path)).toEqual([ + "Payload/Fixture.app/Contents/Resources/libreal.dylib", + "Payload/Fixture.app/Fixture", + ]); + expect(result.limitations).toContainEqual( + expect.stringContaining("AppleDouble sidecar entries"), + ); + }); +}); + describe("macOS application scope", () => { it("attributes only components inside the application roots", () => { const result = project( diff --git a/src/domain/apple/appleApplication.ts b/src/domain/apple/appleApplication.ts index d46ddf26b..dfb34e3c5 100644 --- a/src/domain/apple/appleApplication.ts +++ b/src/domain/apple/appleApplication.ts @@ -218,8 +218,9 @@ export const appleProjectionId = (projection: unknown): string => `aap_${digest(projection)}`; /** - * IPA projection keeps every archive component, as before. Directory, ZIP, - * and DMG inventories can hold installers and other siblings, so only + * AppleDouble sidecars describe neighbouring files and are never components. + * IPA projection keeps every other archive component, as before. Directory, + * ZIP, and DMG inventories can hold installers and other siblings, so only * components inside an application root are attributed to it. */ const attributeComponents = ( @@ -227,8 +228,8 @@ const attributeComponents = ( roots: readonly string[], rootFormat: RootFormat, ): { readonly all: Component[]; readonly unattributed: number } => { - if (rootFormat === "ipa") return { all: [...inventoried], unattributed: 0 }; const visible = inventoried.filter(({ path }) => !isSidecar(path)); + if (rootFormat === "ipa") return { all: visible, unattributed: 0 }; const all = visible.filter(({ path }) => roots.some((root) => isWithin(path, root)), ); diff --git a/src/domain/browserObservation.test.ts b/src/domain/browserObservation.test.ts index c5eb73287..6bcf09f68 100644 --- a/src/domain/browserObservation.test.ts +++ b/src/domain/browserObservation.test.ts @@ -2,7 +2,9 @@ import { describe, expect, it } from "vitest"; import { browserEndpointSchema, + browserObservationEndpointSchema, browserOriginSchema, + recordedBrowserObservationEndpointSchema, sanitizeEndpointCandidate, sanitizeBrowserUrl, } from "./browserObservation.js"; @@ -45,6 +47,58 @@ describe("browser observation contracts", () => { ]) expect(browserEndpointSchema.safeParse(value).success, value).toBe(false); }); + + it("accepts a loopback browser socket and records it without its identifier", () => { + expect( + browserObservationEndpointSchema.parse("http://127.0.0.1:9222/"), + ).toEqual({ + transport: "http_discovery", + url: "http://127.0.0.1:9222", + recorded: "http://127.0.0.1:9222", + }); + expect( + browserObservationEndpointSchema.parse( + "ws://[::1]:9222/devtools/browser/0b1c-socket-id", + ), + ).toEqual({ + transport: "browser_websocket", + url: "ws://[::1]:9222/devtools/browser/0b1c-socket-id", + recorded: "ws://[::1]:9222/devtools/browser/[REDACTED]", + }); + for (const value of [ + "ws://127.0.0.1/devtools/browser/id", + "ws://localhost:9222/devtools/browser/id", + "wss://127.0.0.1:9222/devtools/browser/id", + "ws://user:pass@127.0.0.1:9222/devtools/browser/id", + "ws://127.0.0.1:9222/devtools/browser/id?token=x", + "ws://127.0.0.1:9222/devtools/browser/id#fragment", + "ws://127.0.0.1:9222/devtools/browser/id?", + "ws://127.0.0.1:9222/devtools/browser/id#", + "ws://127.0.0.1:9222/devtools/browser/", + "ws://127.0.0.1:9222/devtools/browser/id/extra", + "ws://127.0.0.1:9222/devtools/page/id", + ]) + expect( + browserObservationEndpointSchema.safeParse(value).success, + value, + ).toBe(false); + }); + + it("accepts recorded browser sockets only in their redacted form", () => { + for (const value of [ + "http://127.0.0.1:9222", + "ws://127.0.0.1:9222/devtools/browser/[REDACTED]", + ]) + expect( + recordedBrowserObservationEndpointSchema.safeParse(value).success, + value, + ).toBe(true); + expect( + recordedBrowserObservationEndpointSchema.safeParse( + "ws://127.0.0.1:9222/devtools/browser/0b1c-socket-id", + ).success, + ).toBe(false); + }); }); describe("browser observation sensitive surfaces and retention", () => { diff --git a/src/domain/browserObservation.ts b/src/domain/browserObservation.ts index f4b8984b6..e15265305 100644 --- a/src/domain/browserObservation.ts +++ b/src/domain/browserObservation.ts @@ -43,46 +43,125 @@ export const browserOriginSchema = z export const isLiteralLoopbackHostname = (hostname: string): boolean => hostname === "127.0.0.1" || hostname === "[::1]" || hostname === "::1"; +const HTTP_ENDPOINT_MESSAGE = + "CDP endpoint must be an explicit-port HTTP URL on 127.0.0.1 or ::1"; +const BROWSER_SOCKET_PATH = "/devtools/browser/"; +const REDACTED_BROWSER_SOCKET_PATH = `${BROWSER_SOCKET_PATH}[REDACTED]`; + +/** Normalized origin of an accepted loopback CDP HTTP discovery endpoint. */ +const httpDiscoveryOrigin = (url: URL): string | undefined => + url.protocol === "http:" && + isLiteralLoopbackHostname(url.hostname) && + url.port !== "" && + url.username === "" && + url.password === "" && + url.pathname === "/" && + !urlDeclaresQueryOrFragment(url) + ? url.origin + : undefined; + +/** Scheme and authority of an accepted loopback CDP browser WebSocket. */ +const browserSocketAuthority = (url: URL): string | undefined => { + const id = url.pathname.slice(BROWSER_SOCKET_PATH.length); + return url.protocol === "ws:" && + isLiteralLoopbackHostname(url.hostname) && + url.port !== "" && + url.username === "" && + url.password === "" && + !urlDeclaresQueryOrFragment(url) && + url.pathname.startsWith(BROWSER_SOCKET_PATH) && + id.length > 0 && + !id.includes("/") + ? `${url.protocol}//${url.host}` + : undefined; +}; + /** Loopback-only HTTP endpoint accepted for a user-owned CDP browser. */ export const browserEndpointSchema = z .string() .min(1) .transform((value, context) => { - let url: URL; - try { - url = new URL(value); - } catch (cause: unknown) { - // Invalid input is reported through the zod issue. - void cause; + const url = URL.parse(value); + const origin = url === null ? undefined : httpDiscoveryOrigin(url); + if (origin !== undefined) return origin; + context.addIssue({ + code: "custom", + message: + url === null ? "Invalid CDP endpoint URL" : HTTP_ENDPOINT_MESSAGE, + }); + return z.NEVER; + }); + +/** + * Caller-selected endpoint for passive browser observation. A browser socket + * skips HTTP discovery; its identifier gates access, so `recorded` omits it. + */ +export interface BrowserObservationEndpoint { + readonly transport: "http_discovery" | "browser_websocket"; + readonly url: string; + readonly recorded: string; +} + +/** Loopback CDP HTTP discovery origin or browser WebSocket URL. */ +export const browserObservationEndpointSchema = z + .string() + .min(1) + .describe( + "Literal loopback CDP endpoint: http://127.0.0.1:PORT, or ws://127.0.0.1:PORT/devtools/browser/ID.", + ) + .transform((value, context): BrowserObservationEndpoint => { + const url = URL.parse(value); + if (url === null) { context.addIssue({ code: "custom", message: "Invalid CDP endpoint URL" }); return z.NEVER; } - if ( - url.protocol !== "http:" || - !isLiteralLoopbackHostname(url.hostname) || - url.port === "" || - url.username !== "" || - url.password !== "" || - url.pathname !== "/" || - urlDeclaresQueryOrFragment(url) - ) { - context.addIssue({ - code: "custom", - message: - "CDP endpoint must be an explicit-port HTTP URL on 127.0.0.1 or ::1", - }); - return z.NEVER; - } - return url.origin; + const origin = httpDiscoveryOrigin(url); + if (origin !== undefined) + return { transport: "http_discovery", url: origin, recorded: origin }; + const authority = browserSocketAuthority(url); + if (authority !== undefined) + return { + transport: "browser_websocket", + url: url.href, + recorded: `${authority}${REDACTED_BROWSER_SOCKET_PATH}`, + }; + context.addIssue({ + code: "custom", + message: + url.protocol === "ws:" + ? "CDP browser WebSocket must be ws://127.0.0.1:PORT/devtools/browser/ID or ws://[::1]:PORT/devtools/browser/ID" + : HTTP_ENDPOINT_MESSAGE, + }); + return z.NEVER; }); +/** Evidence form of a passive browser endpoint, with any socket identifier redacted. */ +export const recordedBrowserObservationEndpointSchema = z + .string() + .min(1) + .refine( + (value) => { + const url = URL.parse(value); + return ( + url !== null && + (httpDiscoveryOrigin(url) !== undefined || + (browserSocketAuthority(url) !== undefined && + url.pathname === REDACTED_BROWSER_SOCKET_PATH)) + ); + }, + { + message: + "Recorded CDP endpoint must be a loopback HTTP origin or a redacted browser WebSocket URL", + }, + ); + export const browserAllowedOriginsSchema = z .array(browserOriginSchema) .transform((origins) => Array.from(new Set(origins)).sort()) .default([]); const browserInput = { - cdp_endpoint: browserEndpointSchema, + cdp_endpoint: browserObservationEndpointSchema, allowed_origins: browserAllowedOriginsSchema, }; diff --git a/src/domain/browserSession.ts b/src/domain/browserSession.ts index 17ecfa8db..c2f00e9a2 100644 --- a/src/domain/browserSession.ts +++ b/src/domain/browserSession.ts @@ -2,14 +2,14 @@ import { z } from "zod"; import { browserAllowedOriginsSchema, - browserEndpointSchema, + browserObservationEndpointSchema, } from "./browserObservation.js"; import { browserCompletenessSchema } from "./browserCompleteness.js"; import { browserVersionSchema } from "./browserObservationSchemas.js"; /** Input for a navigation-aware browser observation window. */ export const observeWebSessionInputSchema = z.strictObject({ - cdp_endpoint: browserEndpointSchema, + cdp_endpoint: browserObservationEndpointSchema, allowed_origins: browserAllowedOriginsSchema, target_id: z.string().trim().min(1), observation_ms: z.number().int().min(1).default(10_000), diff --git a/src/domain/callPath.test.ts b/src/domain/callPath.test.ts index e78a5f936..80a260ac3 100644 --- a/src/domain/callPath.test.ts +++ b/src/domain/callPath.test.ts @@ -15,6 +15,7 @@ const observe = ( readonly evidenceLimitations?: readonly string[]; readonly dossierLimitations?: readonly string[]; readonly relationshipCoverage?: RelationshipCoverage; + readonly calleeUnavailable?: string; } = {}, ): Evidence => { const value = { @@ -52,6 +53,13 @@ const observe = ( native_value_flow: null, basic_blocks: [], limitations: [...(options.dossierLimitations ?? [])], + ...(options.calleeUnavailable === undefined + ? {} + : { + facet_availability: { + callees: { available: false, reason: options.calleeUnavailable }, + }, + }), } satisfies JsonValue; return createEvidence( { path: "/tmp/a", sha256: "a".repeat(64), format: "mach-o" }, @@ -541,3 +549,13 @@ describe("call path evidence admission", () => { ); }); }); + +it("keeps an absent path unknown when only callee facet availability is declared", () => { + const result = run([ + observe("0x1000", [], "rea-workflow", { + calleeUnavailable: "Graph callees unavailable", + }), + ]); + expect(result.status).toBe("unknown"); + expect(result.limitations).toContain("Graph callees unavailable"); +}); diff --git a/src/domain/callPath.ts b/src/domain/callPath.ts index 0bd5e88e1..3ffbc2356 100644 --- a/src/domain/callPath.ts +++ b/src/domain/callPath.ts @@ -253,6 +253,11 @@ const inspectSearch = ({ // cannot add a path to the goal, so its dossier limits are contextual // but do not make the searched start-to-goal closure incomplete. if (node === goal) continue; + const callees = snapshot.dossier.facet_availability?.callees; + if (callees?.available === false) { + limitations.push(callees.reason); + incompleteExpandedCoverage = true; + } // Explicit, Evidence-bound relationship coverage owns completeness; // generic provider/dossier limitations remain contextual. Legacy Evidence // without that metadata retains its conservative limitation-based rule. diff --git a/src/domain/evm/evmInterface.test.ts b/src/domain/evm/evmInterface.test.ts index 93ab32add..1b2a36de6 100644 --- a/src/domain/evm/evmInterface.test.ts +++ b/src/domain/evm/evmInterface.test.ts @@ -2,6 +2,14 @@ import { expect, it } from "vitest"; import { evmInterfaceSchema } from "./evmInterface.js"; import { evmInterfaceFixture } from "../../../tests/fixtures/evm/interface.js"; +it("accepts historical metadata omission as an unknown facet without changing the source", () => { + const value = evmInterfaceFixture(); + const { metadata: _metadata, ...oldBytecode } = value.bytecode; + const old = { ...value, bytecode: oldBytecode }; + expect(evmInterfaceSchema.parse(old).bytecode.metadata).toBeNull(); + expect(old.bytecode).not.toHaveProperty("metadata"); +}); + it.each(["byte-count", "body-offset", "selector", "authenticity"])( "rejects false interface evidence: %s", (problem) => { diff --git a/src/domain/evm/evmInterface.ts b/src/domain/evm/evmInterface.ts index 6db3f7e16..bde6ecba7 100644 --- a/src/domain/evm/evmInterface.ts +++ b/src/domain/evm/evmInterface.ts @@ -1,6 +1,7 @@ import { z } from "zod"; import { localPathStringSchema } from "../localPath.js"; +import { solidityMetadataSchema } from "./solidityMetadata.js"; const byteOffset = z.number().int().nonnegative(); const sha256 = z.string().regex(/^[a-f0-9]{64}$/); @@ -34,6 +35,7 @@ export const evmInterfaceSchema = z kind: z.literal("unknown"), deployment_authenticity: z.literal("unknown"), hardfork: z.literal("unknown"), + metadata: solidityMetadataSchema.nullable().default(null), }), evidence_kind: z.literal("inferred"), functions: z.array( @@ -62,6 +64,23 @@ export const evmInterfaceSchema = z message: "Selected byte count does not match the bytecode hex representation.", }); + const metadata = value.bytecode.metadata; + if ( + metadata !== null && + metadata.status !== "unavailable" && + (metadata.bytecode_offset === null || + metadata.cbor_bytes === null || + metadata.bytecode_offset + metadata.cbor_bytes + 2 !== + value.bytecode.bytes || + metadata.cbor_bytes !== + Number.parseInt(value.bytecode.hex.slice(-4), 16)) + ) + context.addIssue({ + code: "custom", + path: ["bytecode", "metadata"], + message: + "Metadata range must match the terminal length suffix in selected bytecode.", + }); for (const [index, fn] of value.functions.entries()) if (fn.bytecode_offset >= value.bytecode.bytes) context.addIssue({ diff --git a/src/domain/evm/solidityMetadata.ts b/src/domain/evm/solidityMetadata.ts new file mode 100644 index 000000000..28cdee95f --- /dev/null +++ b/src/domain/evm/solidityMetadata.ts @@ -0,0 +1,89 @@ +import { z } from "zod"; + +/** Scalar view of CBOR; uninterpreted containers remain in original bytecode. */ +export const solidityMetadataScalarSchema = z.discriminatedUnion("kind", [ + z.strictObject({ kind: z.literal("text"), value: z.string().max(65535) }), + z.strictObject({ + kind: z.literal("integer"), + value: z + .string() + .max(21) + .regex(/^-?(?:0|[1-9][0-9]*)$/), + }), + z.strictObject({ + kind: z.literal("bytes"), + value: z + .string() + .max(131070) + .regex(/^(?:[a-f0-9]{2})*$/), + }), + z.strictObject({ kind: z.literal("boolean"), value: z.boolean() }), + z.strictObject({ kind: z.literal("null") }), + z.strictObject({ + kind: z.literal("uninterpreted"), + container: z.enum(["array", "map"]), + }), +]); + +/** A trailer candidate, not authentication of Solidity origin or deployment. */ +export const solidityMetadataSchema = z + .strictObject({ + profile: z.literal("terminal-cbor-scalars-v1"), + status: z.enum(["unavailable", "decoded", "partial", "not-decoded"]), + bytecode_offset: z.number().int().nonnegative().nullable(), + cbor_bytes: z.number().int().positive().max(65535).nullable(), + entries: z + .array( + z.strictObject({ + key: solidityMetadataScalarSchema, + value: solidityMetadataScalarSchema, + }), + ) + .max(32767), + compiler_version: z + .strictObject({ + representation: z.enum(["release-bytes", "prerelease-text"]), + value: z.string().min(1).max(65535), + }) + .nullable(), + diagnostic: z.string().max(512).nullable(), + }) + .superRefine((value, context) => { + const unavailable = value.status === "unavailable"; + if ( + unavailable + ? value.bytecode_offset !== null || value.cbor_bytes !== null + : value.bytecode_offset === null || value.cbor_bytes === null + ) + context.addIssue({ + code: "custom", + message: "Only unavailable metadata omits the candidate range.", + }); + const unexamined = unavailable || value.status === "not-decoded"; + if ( + unexamined && + (value.entries.length !== 0 || value.compiler_version !== null) + ) + context.addIssue({ + code: "custom", + message: + "Unavailable interpretation cannot report decoded entries or compiler identity.", + }); + const uninterpreted = value.entries.some( + ({ key, value: entryValue }) => + key.kind === "uninterpreted" || entryValue.kind === "uninterpreted", + ); + if ( + (value.status === "decoded" && uninterpreted) || + (value.status === "partial" && !uninterpreted) + ) + context.addIssue({ + code: "custom", + message: "Scalar coverage must preserve uninterpreted containers.", + }); + }); + +export type SolidityMetadata = z.infer; +export type SolidityMetadataScalar = z.infer< + typeof solidityMetadataScalarSchema +>; diff --git a/src/domain/functionComparison.test.ts b/src/domain/functionComparison.test.ts index 677fdc0cc..16f8282c9 100644 --- a/src/domain/functionComparison.test.ts +++ b/src/domain/functionComparison.test.ts @@ -44,6 +44,16 @@ const dossier = ( referenced_names: [], native_api: null, native_value_flow: null, + facet_availability: { + pseudocode: { available: true }, + assembly: { available: true }, + comments: { available: true }, + callers: { available: true }, + callees: { available: true }, + references: { available: true }, + strings_names: { available: true }, + cfg: { available: true }, + }, basic_blocks: [ { start: base, @@ -118,34 +128,118 @@ const observe = ( }, ); -describe("function comparison normalized identity", () => { - it("keeps differing generated names ambiguous and stable names mismatched", () => { - const named = (base: "0x1000" | "0x2000", name: string) => - functionDossierSchema.parse({ - ...dossier("return 0;", base), - procedure: { - ...procedure(base, name), - signature: "int helper(void)", - locals: [], - }, - }); - const generated = compareFunctions( - observe("1", named("0x1000", "sub_1000")), - observe("2", named("0x2000", "sub_2000")), - ); - expect(generated.function_match.status).toBe("ambiguous"); - expect(generated.status).toBe("unknown"); - expect( - generated.dimensions.find(({ dimension }) => dimension === "identity"), - ).toMatchObject({ status: "unknown" }); +const rejectsMalformedFacetAvailability = (): void => { + expect( + functionDossierSchema.safeParse({ + ...dossier("return 0;", "0x1000"), + facet_availability: { cfg: { available: false, reason: "" } }, + }).success, + ).toBe(false); + expect( + functionDossierSchema.safeParse({ + ...dossier("return 0;", "0x1000"), + facet_availability: { cfg: undefined }, + }).success, + ).toBe(false); +}; - const stable = compareFunctions( - observe("3", named("0x1000", "firstFunction")), - observe("4", named("0x2000", "secondFunction")), +const keepsGeneratedNamesAmbiguous = (): void => { + const named = (base: "0x1000" | "0x2000", name: string) => + functionDossierSchema.parse({ + ...dossier("return 0;", base), + procedure: { + ...procedure(base, name), + signature: "int helper(void)", + locals: [], + }, + }); + const generated = compareFunctions( + observe("1", named("0x1000", "sub_1000")), + observe("2", named("0x2000", "sub_2000")), + ); + expect(generated.function_match.status).toBe("ambiguous"); + expect(generated.status).toBe("unknown"); + expect( + generated.dimensions.find(({ dimension }) => dimension === "identity"), + ).toMatchObject({ status: "unknown" }); + + const stable = compareFunctions( + observe("3", named("0x1000", "firstFunction")), + observe("4", named("0x2000", "secondFunction")), + ); + expect(stable.function_match.status).toBe("mismatched"); + expect(stable.status).toBe("changed"); +}; + +describe("function comparison normalized identity", () => { + it("retains unresolved call details alongside missing facet coverage", () => { + const value: Record = { + ...dossier("return 0;", "0x1000"), + unresolved_calls: [ + { address: "0x1010", reason: "Unknown indirect target" }, + ], + }; + delete value.facet_availability; + const legacy = observe("a", functionDossierSchema.parse(value), "ida"); + const calls = compareFunctions(legacy, legacy).dimensions.find( + ({ dimension }) => dimension === "calls", ); - expect(stable.function_match.status).toBe("mismatched"); - expect(stable.status).toBe("changed"); + expect(calls).toMatchObject({ + status: "unknown", + limitations: expect.arrayContaining([ + "Left: Unresolved call at 0x1010: Unknown indirect target", + "Left: callers: Provider did not declare coverage for this empty facet.", + ]), + }); }); + it("keeps empty legacy facets without coverage declarations unknown", () => { + const value: Record = { + ...dossier("return 0;", "0x1000"), + basic_blocks: [], + }; + delete value.facet_availability; + const legacy = observe("a", functionDossierSchema.parse(value), "ida"); + const result = compareFunctions(legacy, legacy); + for (const dimension of [ + "comments", + "references", + "strings_names", + "cfg", + "calls", + ]) { + expect( + result.dimensions.find((item) => item.dimension === dimension), + ).toMatchObject({ status: "unknown" }); + } + }); + it("keeps unavailable facets unknown even when one side supplies values", () => { + const left = observe( + "a", + functionDossierSchema.parse({ + ...dossier("return 0;", "0x1000"), + facet_availability: { + comments: { available: false, reason: "Not collected" }, + }, + comments: [ + { address: "0x1000", kind: "comment", text: "partial record" }, + ], + }), + ); + const right = observe("b", dossier("return 0;", "0x2000")); + expect( + compareFunctions(left, right).dimensions.find( + ({ dimension }) => dimension === "comments", + ), + ).toMatchObject({ status: "unknown", left_count: null, right_count: null }); + }); + it( + "rejects malformed facet availability instead of treating it as absence", + rejectsMalformedFacetAvailability, + ); + it( + "keeps differing generated names ambiguous and stable names mismatched", + keepsGeneratedNamesAmbiguous, + ); it.each(["sub_deallocate", "sub_deadbeef_handler", "fcn.dispatch"])( "recognizes the explicit symbol %s without a generated-name prefix false positive", @@ -999,3 +1093,51 @@ const expectFunctionDimensionAlgebra = ( }).success, ).toBe(false); }; + +it.each([ + { match: "mismatched", changedDimensions: false, expected: "changed" }, + { match: "ambiguous", changedDimensions: false, expected: "unknown" }, + { match: "ambiguous", changedDimensions: true, expected: "unknown" }, +] as const)( + "validates $match match precedence with changed dimensions=$changedDimensions", + ({ match, changedDimensions, expected }) => { + const result = compareFunctions( + FUNCTION_COMPARISON_EXAMPLE.left, + changedDimensions + ? FUNCTION_COMPARISON_EXAMPLE.right + : FUNCTION_COMPARISON_EXAMPLE.left, + ); + const candidate = { + ...result, + function_match: { + status: match, + method: "explicit", + left_name: "main", + right_name: "other", + }, + }; + expect( + functionComparisonResultSchema.safeParse({ + ...candidate, + status: expected, + }).success, + ).toBe(true); + for (const status of ["unchanged", "changed", "unknown"] as const) { + if (status === expected) continue; + const parsed = functionComparisonResultSchema.safeParse({ + ...candidate, + status, + }); + expect(parsed.success).toBe(false); + if (!parsed.success) + expect(parsed.error.issues).toMatchObject([ + { + path: ["status"], + code: "custom", + message: + "Function comparison status contradicts its dimensions or function match.", + }, + ]); + } + }, +); diff --git a/src/domain/functionComparisonDimensions.ts b/src/domain/functionComparisonDimensions.ts index c5a0f373f..f91057a6b 100644 --- a/src/domain/functionComparisonDimensions.ts +++ b/src/domain/functionComparisonDimensions.ts @@ -52,7 +52,67 @@ export const compareDimensions = ( compareReferences(left, right, context), compareStringsAndNames(left, right, context), compareCfg(left, right, context), - ]; + ].map((result) => { + const limitations = [ + ...facetLimitations(left, result.dimension).map( + (reason) => `Left: ${reason}`, + ), + ...facetLimitations(right, result.dimension).map( + (reason) => `Right: ${reason}`, + ), + ]; + return limitations.length === 0 + ? result + : unresolvedDimension({ + dimension: result.dimension, + links: context.links, + leftCount: null, + rightCount: null, + limitations: [...result.limitations, ...limitations], + }); + }); +}; + +/** An empty undeclared facet is not evidence of observed absence. */ +const facetLimitations = ( + snapshot: FunctionSnapshot, + dimension: DimensionName, +): string[] => { + const { dossier } = snapshot; + const fields = { + identity: [], + pseudocode: [["pseudocode", dossier.pseudocode.length]], + assembly: [["assembly", dossier.assembly.length]], + comments: [["comments", dossier.comments.length]], + calls: [ + ["callers", dossier.callers.length], + ["callees", dossier.callees.length], + ], + references: [ + [ + "references", + dossier.incoming_references.length + dossier.outgoing_references.length, + ], + ], + strings_names: [ + [ + "strings_names", + dossier.referenced_strings.length + dossier.referenced_names.length, + ], + ], + cfg: [["cfg", dossier.basic_blocks.length]], + } satisfies Record< + DimensionName, + [keyof NonNullable, number][] + >; + return fields[dimension].flatMap(([field, count]) => { + const availability = dossier.facet_availability?.[field]; + if (availability?.available === false) + return [`${field}: ${availability.reason}`]; + return availability === undefined && count === 0 + ? [`${field}: Provider did not declare coverage for this empty facet.`] + : []; + }); }; const compareText = ( diff --git a/src/domain/functionComparisonSchemas.ts b/src/domain/functionComparisonSchemas.ts index bafc3def1..844bdc9b9 100644 --- a/src/domain/functionComparisonSchemas.ts +++ b/src/domain/functionComparisonSchemas.ts @@ -1,4 +1,5 @@ import { z } from "zod"; +import { overallStatus } from "./functionComparisonResults.js"; import { evidenceSchema } from "./evidence.js"; import { digestSchema } from "./../domain/digests.js"; import { prefixedDigestSchema } from "./../domain/digests.js"; @@ -79,20 +80,33 @@ const functionMatchSchema = z.union([ ]); /** Deterministic dimension-classified function comparison Evidence payload. */ -export const functionComparisonResultSchema = z.object({ - status: comparisonStatusSchema, - function_match: functionMatchSchema, - left_subject_sha256: digestSchema, - right_subject_sha256: digestSchema, - summary: z.object({ - unchanged: z.number().int().min(0), - changed: z.number().int().min(0), - unknown: z.number().int().min(0), - }), - dimensions: z.array(functionDimensionSchema).length(8), - changes: z.array(functionDimensionSchema), - limitations: z.array(z.string()), -}); +export const functionComparisonResultSchema = z + .object({ + status: comparisonStatusSchema, + function_match: functionMatchSchema, + left_subject_sha256: digestSchema, + right_subject_sha256: digestSchema, + summary: z.object({ + unchanged: z.number().int().min(0), + changed: z.number().int().min(0), + unknown: z.number().int().min(0), + }), + dimensions: z.array(functionDimensionSchema).length(8), + changes: z.array(functionDimensionSchema), + limitations: z.array(z.string()), + }) + .superRefine((result, context) => { + if ( + result.status !== + overallStatus(result.dimensions, result.function_match.status) + ) + context.addIssue({ + code: "custom", + path: ["status"], + message: + "Function comparison status contradicts its dimensions or function match.", + }); + }); export type FunctionComparisonResult = z.infer< typeof functionComparisonResultSchema diff --git a/src/domain/hopperValues.ts b/src/domain/hopperValues.ts index d6344ff3e..b42a98383 100644 --- a/src/domain/hopperValues.ts +++ b/src/domain/hopperValues.ts @@ -1,4 +1,5 @@ import { z } from "zod"; +import { addressCoordinate } from "./native/addressCoordinate.js"; import type { JsonValue } from "./jsonValue.js"; @@ -38,16 +39,6 @@ export const addressDistance = (start: string, end: string): number => { ); }; -const addressCoordinate = ( - value: string, -): { readonly space: string; readonly offset: bigint } | null => { - const separator = value.lastIndexOf(":0x"); - const space = separator < 0 ? "default" : value.slice(0, separator); - const offset = separator < 0 ? value : value.slice(separator + 1); - if (!/^0x[0-9a-f]+$/u.test(offset)) return null; - return { space, offset: BigInt(offset) }; -}; - const procedureMapSchema = z.record(z.string(), z.string()); const addressedNamesSchema = z.array( z.object({ address: z.string(), name: z.string() }), @@ -128,9 +119,9 @@ export const functionBodySchema = z.discriminatedUnion("available", [ }) .superRefine((body, context) => { let total = 0n; - const bounds = new Map(); - const closedSpaces = new Set(); - let previousSpace: string | null = null; + const bounds = new Map(); + const closedSpaces = new Set(); + let previousSpace: string | null | undefined; for (const range of body.ranges) { const start = addressCoordinate(range.start); const end = addressCoordinate(range.end); @@ -146,7 +137,7 @@ export const functionBodySchema = z.discriminatedUnion("available", [ }); return; } - if (previousSpace !== null && previousSpace !== start.space) + if (previousSpace !== undefined && previousSpace !== start.space) closedSpaces.add(previousSpace); const previous = bounds.get(start.space); if ( @@ -371,6 +362,11 @@ export const referenceEdgeSchema = z .exactOptional(), }) .strict(); +const facetAvailabilitySchema = z.discriminatedUnion("available", [ + z.object({ available: z.literal(true) }).strict(), + z.object({ available: z.literal(false), reason: z.string().min(1) }).strict(), +]); + export const functionDossierSchema = z .object({ procedure: procedureIdentitySchema.safeExtend({ @@ -428,6 +424,19 @@ export const functionDossierSchema = z ), native_api: nativeApiBoundarySchema.nullable(), native_value_flow: nativeValueFlowSchema.nullable(), + facet_availability: z + .object({ + pseudocode: facetAvailabilitySchema.exactOptional(), + assembly: facetAvailabilitySchema.exactOptional(), + comments: facetAvailabilitySchema.exactOptional(), + callers: facetAvailabilitySchema.exactOptional(), + callees: facetAvailabilitySchema.exactOptional(), + references: facetAvailabilitySchema.exactOptional(), + strings_names: facetAvailabilitySchema.exactOptional(), + cfg: facetAvailabilitySchema.exactOptional(), + }) + .strict() + .exactOptional(), limitations: z.array(z.string()), }) .strict(); diff --git a/src/domain/javascript/childProcessOptionsRegression.test.ts b/src/domain/javascript/childProcessOptionsRegression.test.ts new file mode 100644 index 000000000..8c18fe830 --- /dev/null +++ b/src/domain/javascript/childProcessOptionsRegression.test.ts @@ -0,0 +1,97 @@ +import { describe, expect, it } from "vitest"; +import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js"; + +describe("child-process options detection", () => { + it("detects options without an args array", () => { + const ir = analyzeJavaScriptSemantics(` + import { spawn, execFile, fork } from "node:child_process"; + + spawn("tool", { + env: { A: "1" }, + stdio: "pipe" + }); + + execFile("tool", { + env: { A: "1" } + }); + + fork("worker.js", { + env: { A: "1" }, + stdio: ["pipe", "pipe", "pipe", "ipc"] + }); + `); + + expect( + ir.childProcessSpawns.map( + ({ method, environmentSupplied, stdioMode, resolution }) => ({ + method, + environmentSupplied, + stdioMode, + resolution, + }), + ), + ).toEqual([ + { + method: "spawn", + environmentSupplied: true, + stdioMode: "pipe", + resolution: "complete", + }, + { + method: "execFile", + environmentSupplied: true, + stdioMode: "default", + resolution: "complete", + }, + { + method: "fork", + environmentSupplied: true, + stdioMode: "array", + resolution: "complete", + }, + ]); + }); + + it("preserves behavior with explicit args arrays", () => { + const ir = analyzeJavaScriptSemantics(` + import { spawn, execFile, fork } from "node:child_process"; + + spawn("tool", [], { env: { A: "1" }, stdio: "pipe" }); + execFile("tool", [], { env: { A: "1" } }); + fork("worker.js", [], { + env: { A: "1" }, + stdio: ["pipe", "pipe", "pipe", "ipc"] + }); + `); + + expect( + ir.childProcessSpawns.map(({ environmentSupplied, stdioMode }) => ({ + environmentSupplied, + stdioMode, + })), + ).toEqual([ + { environmentSupplied: true, stdioMode: "pipe" }, + { environmentSupplied: true, stdioMode: "default" }, + { environmentSupplied: true, stdioMode: "array" }, + ]); + }); + + it("does not invent options when omitted", () => { + const ir = analyzeJavaScriptSemantics(` + import { spawn, execFile, fork } from "node:child_process"; + + spawn("tool"); + execFile("tool"); + fork("worker.js"); + `); + + expect( + ir.childProcessSpawns.every( + ({ environmentSupplied, stdioMode }) => + environmentSupplied === false && stdioMode === "default", + ), + ).toBe(true); + + expect(ir.childProcessSpawns).toHaveLength(3); + }); +}); diff --git a/src/domain/javascript/electronStaticAnalysisTypes.ts b/src/domain/javascript/electronStaticAnalysisTypes.ts index b8d2c03fb..97cd32375 100644 --- a/src/domain/javascript/electronStaticAnalysisTypes.ts +++ b/src/domain/javascript/electronStaticAnalysisTypes.ts @@ -219,6 +219,8 @@ export interface ElectronCallPathStep { export interface ElectronCallPath { readonly entry_kind: "bridge-member" | "ipc-handler"; readonly entry: string; + /** Exact bridge API and property segments; `entry` remains display text. */ + readonly entry_path?: readonly string[]; readonly channel: string | null; readonly terminal: string; readonly resolution: "candidate"; diff --git a/src/domain/javascript/electronStaticAnalysisValues.ts b/src/domain/javascript/electronStaticAnalysisValues.ts index 2fe9d5858..90ef1f0a5 100644 --- a/src/domain/javascript/electronStaticAnalysisValues.ts +++ b/src/domain/javascript/electronStaticAnalysisValues.ts @@ -153,6 +153,13 @@ const collectMembersAt = ( state.unknown += 1; continue; } + if ( + t.isObjectProperty(property) && + !property.computed && + !property.shorthand && + name === "__proto__" + ) + continue; const segments = [...prefix, name]; const path = segments.join("."); state.members.push(path); diff --git a/src/domain/javascript/electronWrappedCallPaths.test.ts b/src/domain/javascript/electronWrappedCallPaths.test.ts index 2740d7915..6d30a10f5 100644 --- a/src/domain/javascript/electronWrappedCallPaths.test.ts +++ b/src/domain/javascript/electronWrappedCallPaths.test.ts @@ -357,6 +357,83 @@ it("preserves guarded alternatives in direct exposed object literals", () => { ]), ); }); +it("preserves bridge member segments and excludes prototype setters", () => { + const result = analyzeJavaScriptStaticSource(` + const e=require("electron"); + const write=()=>e.ipcRenderer.send("write"); + e.contextBridge.exposeInMainWorld("desktop", { + "nested.write": write, + nested: { write }, + __proto__: { inherited: write }, + ["__proto__"]: { own: write }, + }); + `); + expect(result.electron.call_paths).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + entry: "desktop.nested.write", + entry_path: ["desktop", "nested.write"], + }), + expect.objectContaining({ + entry: "desktop.nested.write", + entry_path: ["desktop", "nested", "write"], + }), + expect.objectContaining({ + entry: "desktop.__proto__.own", + entry_path: ["desktop", "__proto__", "own"], + }), + ]), + ); + expect(result.electron.call_paths).not.toEqual( + expect.arrayContaining([ + expect.objectContaining({ + entry_path: ["desktop", "__proto__", "inherited"], + }), + ]), + ); +}); +it("follows an explicit object-literal prototype lookup in a bridge wrapper", () => { + const result = analyzeJavaScriptStaticSource(` + const e=require("electron"); + const prototype={send:()=>e.ipcRenderer.send("through-prototype")}; + const object={__proto__:prototype}; + e.contextBridge.exposeInMainWorld("desktop", { + action:()=>object.__proto__.send(), + }); + `); + expect(result.electron.call_paths).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + entry_path: ["desktop", "action"], + channel: "through-prototype", + }), + ]), + ); +}); +it("does not spread a large nested bridge member list into function arguments", () => { + const members = Array.from( + { length: 125_000 }, + (_, index) => `m${index}(){}`, + ).join(","); + const source = `const e=require("electron");e.contextBridge.exposeInMainWorld("desktop",{nested:{${members}}});`; + expect(() => analyzeJavaScriptStaticSource(source)).not.toThrow(); +}); +it("reports nested bridge depth truncation while retaining shallow paths", () => { + const nested = + Array.from({ length: 130 }, (_, index) => `level${index}:{`).join("") + + `deep:()=>e.ipcRenderer.send("deep")` + + "}".repeat(130); + const result = analyzeJavaScriptStaticSource(` + const e=require("electron"); + e.contextBridge.exposeInMainWorld("desktop", { + shallow:()=>e.ipcRenderer.send("shallow"), ${nested} + }); + `); + expect(result.electron.call_paths).toContainEqual( + expect.objectContaining({ entry: "desktop.shallow", channel: "shallow" }), + ); + expect(result.limitations.join(" ")).toContain("depth 128"); +}); it("preserves conditions when a wrapper dispatches through conditional external methods", () => { const result = analyzeJavaScriptStaticSource(` const e=require("electron"); diff --git a/src/domain/javascript/electronWrappedCallPaths.ts b/src/domain/javascript/electronWrappedCallPaths.ts index 048ad30f5..051595eb6 100644 --- a/src/domain/javascript/electronWrappedCallPaths.ts +++ b/src/domain/javascript/electronWrappedCallPaths.ts @@ -44,6 +44,7 @@ const callLabel = (node: t.Node): string => interface PathContext { readonly entryNode: t.Node; readonly entry: string; + readonly entryPath?: readonly string[]; readonly entryKind: ElectronCallPath["entry_kind"] | null; readonly channel: string | null; readonly steps: readonly ElectronCallPathStep[]; @@ -122,6 +123,37 @@ export const collectElectronWrappedCallPaths = ( }; const hasConditions = (value: SymbolicValue): boolean => symbolicVariants(value).some((variant) => variant.conditions?.length); + function* bridgeMemberFunctions( + value: SymbolicValue, + segments: readonly string[] = [], + active = new Set(), + ): Generator<{ + readonly name: string; + readonly path: readonly string[]; + readonly fn: SymbolicFunction; + }> { + if (active.has(value)) return; + if (active.size >= 128) { + accumulator.unknownFindings += 1; + accumulator.electronCallPathLimitReached = true; + return; + } + const nested = new Set(active); + nested.add(value); + for (const member of [...resolver.members(value)].sort()) { + if (--remainingVisits < 0) return; + const path = [...segments, member]; + const name = path.join("."); + for (const variant of symbolicVariants( + resolver.property(value, member), + )) { + if (variant.kind === "function") yield { name, path, fn: variant }; + else if (resolver.members(variant).length > 0) + yield* bridgeMemberFunctions(variant, path, nested); + } + if (remainingVisits < 0) return; + } + } const isolatedBridgeCall = ( node: t.CallExpression | t.OptionalCallExpression, ): @@ -167,6 +199,10 @@ export const collectElectronWrappedCallPaths = ( const path: ElectronCallPath = { entry_kind: context.entryKind, entry: context.entry, + ...(context.entryKind !== "bridge-member" || + context.entryPath === undefined + ? {} + : { entry_path: context.entryPath }), channel, terminal, resolution: "candidate", @@ -405,27 +441,25 @@ export const collectElectronWrappedCallPaths = ( offset: node.start ?? -1, value: finding, }); - for (const name of members) { - const value = resolver.property(object, name); - for (const fn of symbolicVariants(value)) - if (fn.kind === "function") { - const memberContext = metadataContext(keyContext, fn); - if (!memberContext) continue; - functionBody( - fn, - fn.node.params.map(() => symbolicUnknown("public API argument")), - { - ...memberContext, - entryNode: node, - entryKind: "bridge-member", - entry: `${key.value}.${name}`, - channel: null, - steps: [step(node, "exposure", `${key.value}.${name}`)], - active: new Set(), - }, - node, - ); - } + for (const { name, path, fn } of bridgeMemberFunctions(object)) { + const memberContext = metadataContext(keyContext, fn); + if (!memberContext) continue; + const entry = `${key.value}.${name}`; + functionBody( + fn, + fn.node.params.map(() => symbolicUnknown("public API argument")), + { + ...memberContext, + entryNode: node, + entryKind: "bridge-member", + entry, + entryPath: [key.value, ...path], + channel: null, + steps: [step(node, "exposure", entry)], + active: new Set(), + }, + node, + ); } }; const call = ( diff --git a/src/domain/javascript/javascriptFeatureSeed.array_members.test.ts b/src/domain/javascript/javascriptFeatureSeed.array_members.test.ts index 9cab84239..50f6ceb09 100644 --- a/src/domain/javascript/javascriptFeatureSeed.array_members.test.ts +++ b/src/domain/javascript/javascriptFeatureSeed.array_members.test.ts @@ -11,7 +11,10 @@ import { } from "./javascriptApplicationGraph.fixture.js"; import { findApplicationFeatureSeeds } from "./javascriptFeatureSeed.js"; import { traceApplicationFeature } from "./javascriptFeatureTrace.js"; -import type { ApplicationFeatureSeed } from "./javascriptFeatureTraceSchemas.js"; +import { + applicationFeatureSeedSchema, + type ApplicationFeatureSeed, +} from "./javascriptFeatureTraceSchemas.js"; import { analyzeJavaScriptStaticSource } from "./javascriptStaticAnalysis.js"; import type { JsonValue } from "../jsonValue.js"; @@ -268,6 +271,32 @@ describe("feature seed selection controls", () => { }); describe("public feature trace from contextBridge source members", () => { + it.each(["openProject", "missingProject"])( + "normalizes optional seed defaults before deriving trace identity for %s", + (value) => { + const input = { + sourceEvidenceId, + graph: contextBridgeGraph(), + nativeEvidence: [], + direction: "both" as const, + }; + const omitted = traceApplicationFeature({ + ...input, + seed: { kind: "api", value }, + }); + const explicit = traceApplicationFeature({ + ...input, + seed: { + kind: "api", + value, + match: null, + case_sensitive: false, + }, + }); + expect(omitted).toEqual(explicit); + }, + ); + it("traces one exact member seed through the public feature workflow", () => { const mode = { value: "openProject", @@ -381,12 +410,13 @@ const sourceEvidenceId = `ev_${"b".repeat(64)}`; const seed = ( kind: ApplicationFeatureSeed["kind"], value = "openProject", -): ApplicationFeatureSeed => ({ - kind, - value, - match: "exact", - case_sensitive: true, -}); +): ApplicationFeatureSeed => + applicationFeatureSeedSchema.parse({ + kind, + value, + match: "exact", + case_sensitive: true, + }); const memberNode = ( kind: ApplicationNode["kind"], diff --git a/src/domain/javascript/javascriptFeatureSeed.ts b/src/domain/javascript/javascriptFeatureSeed.ts index 929733f00..d5f06b817 100644 --- a/src/domain/javascript/javascriptFeatureSeed.ts +++ b/src/domain/javascript/javascriptFeatureSeed.ts @@ -52,10 +52,28 @@ const seedMatch = ( ] : []; if (!kindMatches(node, seed.kind)) return []; + const requestedEntryPath = + seed.kind === "api" && "entry_path" in seed ? seed.entry_path : undefined; + if ( + seed.kind === "api" && + requestedEntryPath !== undefined && + !node.observations.some(({ properties }) => { + const entryPath = properties.entry_path; + return ( + Array.isArray(entryPath) && + entryPath.length === requestedEntryPath.length && + entryPath.every( + (segment, index) => segment === requestedEntryPath[index], + ) + ); + }) + ) + return []; const mode = featureSeedMatchMode(seed); - const query = normalized(seed.value, seed.case_sensitive); + const caseSensitive = seed.case_sensitive; + const query = normalized(seed.value, caseSensitive); const match = candidateFields(node, seed.kind).find((candidate) => { - const value = normalized(candidate.value, seed.case_sensitive); + const value = normalized(candidate.value, caseSensitive); return mode === "exact" ? value === query : value.includes(query); }); return match === undefined diff --git a/src/domain/javascript/javascriptFeatureTrace.ts b/src/domain/javascript/javascriptFeatureTrace.ts index 47333bab6..e4109b85d 100644 --- a/src/domain/javascript/javascriptFeatureTrace.ts +++ b/src/domain/javascript/javascriptFeatureTrace.ts @@ -12,6 +12,8 @@ import type { } from "./javascriptApplicationGraphSchemas.js"; import { applicationFeatureTraceResultSchema, + applicationFeatureSeedSchema, + type ApplicationFeatureSeedInput, type ApplicationFeatureTraceResult, type TraceApplicationFeatureInput, } from "./javascriptFeatureTraceSchemas.js"; @@ -30,14 +32,18 @@ export interface ApplicationFeatureTraceProjectionInput { readonly sourceEvidenceId: string; readonly graph: JavaScriptApplicationGraph; readonly nativeEvidence: readonly Evidence[]; - readonly seed: TraceApplicationFeatureInput["seed"]; + readonly seed: ApplicationFeatureSeedInput; readonly direction: TraceApplicationFeatureInput["direction"]; } /** Trace a literal feature through its complete reachable, authority-preserving JAG. */ export const traceApplicationFeature = ( - input: ApplicationFeatureTraceProjectionInput, + source: ApplicationFeatureTraceProjectionInput, ): ApplicationFeatureTraceResult => { + const input = { + ...source, + seed: applicationFeatureSeedSchema.parse(source.seed), + }; const allSeedMatches = findApplicationFeatureSeeds( input.graph.nodes, input.seed, diff --git a/src/domain/javascript/javascriptFeatureTraceSchemas.ts b/src/domain/javascript/javascriptFeatureTraceSchemas.ts index ee34bad09..b341ce142 100644 --- a/src/domain/javascript/javascriptFeatureTraceSchemas.ts +++ b/src/domain/javascript/javascriptFeatureTraceSchemas.ts @@ -17,33 +17,65 @@ export const featureSeedMatchMode = (seed: { }): "exact" | "contains" => seed.match ?? (seed.kind === "string" ? "contains" : "exact"); -const literalSeedKinds = [ - "route", - "string", - "api", - "channel", - "module", - "native-export", -] as const; +const apiSeedSchema = z.union([ + z.strictObject({ + kind: z.literal("api"), + value: boundedTextSchema, + match: z.enum(["exact", "contains"]).nullable().default(null), + case_sensitive: z.boolean().default(false), + }), + z.strictObject({ + kind: z.literal("api"), + value: z.literal(""), + match: z.literal("exact"), + case_sensitive: z.boolean().default(false), + }), +]); + +const apiPathSeedSchema = z.union([ + z.strictObject({ + kind: z.literal("api"), + value: boundedTextSchema, + match: z.enum(["exact", "contains"]).nullable().default(null), + case_sensitive: z.boolean().default(false), + entry_path: z.array(z.string()).min(1), + }), + z.strictObject({ + kind: z.literal("api"), + value: z.literal(""), + match: z.literal("exact"), + case_sensitive: z.boolean().default(false), + entry_path: z.array(z.string()).min(1), + }), +]); /** * Literal starting point for one application feature trace. An empty value, * such as the channel in `ipcMain.handle("")`, must be an explicit exact * literal seed: empty containment would select every node. */ -const applicationFeatureSeedSchema = z.union([ +export const applicationFeatureSeedSchema = z.union([ z.strictObject({ - kind: z.enum(["node-id", ...literalSeedKinds]), + kind: z.enum([ + "node-id", + "route", + "string", + "channel", + "module", + "native-export", + ]), value: boundedTextSchema, match: z.enum(["exact", "contains"]).nullable().default(null), case_sensitive: z.boolean().default(false), }), z.strictObject({ - kind: z.enum(literalSeedKinds), + kind: z.enum(["route", "string", "channel", "module", "native-export"]), value: z.literal(""), match: z.literal("exact"), case_sensitive: z.boolean().default(false), }), + apiSeedSchema, + apiPathSeedSchema, ]); /** Check unique native identities before projecting an application trace. */ @@ -156,7 +188,10 @@ export const applicationFeatureTraceResultSchema = z.strictObject({ export type TraceApplicationFeatureInput = z.infer< typeof traceApplicationFeatureInputSchema >; -export type ApplicationFeatureSeed = z.infer< +export type ApplicationFeatureSeed = z.output< + typeof applicationFeatureSeedSchema +>; +export type ApplicationFeatureSeedInput = z.input< typeof applicationFeatureSeedSchema >; export type ApplicationFeatureTraceResult = z.infer< diff --git a/src/domain/javascript/javascriptRuntimeReconciliationCaptureParsing.ts b/src/domain/javascript/javascriptRuntimeReconciliationCaptureParsing.ts index baba3395d..be428b06c 100644 --- a/src/domain/javascript/javascriptRuntimeReconciliationCaptureParsing.ts +++ b/src/domain/javascript/javascriptRuntimeReconciliationCaptureParsing.ts @@ -3,6 +3,7 @@ import { z } from "zod"; import { browserAllowedOriginsSchema, browserEndpointSchema, + recordedBrowserObservationEndpointSchema, } from "../browserObservation.js"; import { webPageInspectionSchema, @@ -424,14 +425,17 @@ const assertRuntimeParameters = ( ): void => { const common = { target_id: z.string().trim().min(1), - cdp_endpoint: browserEndpointSchema, include_script_sources: z.boolean(), }; if (expected.kind === "browser") { const parameters = parseAtPath( (value) => z - .object({ ...common, allowed_origins: browserAllowedOriginsSchema }) + .object({ + ...common, + cdp_endpoint: recordedBrowserObservationEndpointSchema, + allowed_origins: browserAllowedOriginsSchema, + }) .passthrough() .parse(value), evidence.parameters, @@ -455,7 +459,11 @@ const assertRuntimeParameters = ( return; } const parameters = parseAtPath( - (value) => z.object(common).passthrough().parse(value), + (value) => + z + .object({ ...common, cdp_endpoint: browserEndpointSchema }) + .passthrough() + .parse(value), evidence.parameters, [...path, "parameters"], ); diff --git a/src/domain/javascript/javascriptSemanticAnalysis.logical_assignment.test.ts b/src/domain/javascript/javascriptSemanticAnalysis.logical_assignment.test.ts new file mode 100644 index 000000000..255376cc1 --- /dev/null +++ b/src/domain/javascript/javascriptSemanticAnalysis.logical_assignment.test.ts @@ -0,0 +1,96 @@ +import { describe, expect, it } from "vitest"; +import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js"; + +const observe = (source: string) => { + const ir = analyzeJavaScriptSemantics(source); + return { + binding: ir.bindings.find((item) => item.name === "x")?.value, + returned: ir.callables.find((item) => item.name === "read")?.returnSites[0] + ?.value, + }; +}; + +describe("JavaScript semantic analysis: logical assignment", () => { + it.each(["||=", "??=", "&&="])( + "keeps a parameter uncertain after %s", + (operator) => { + const { binding, returned } = observe( + `function read(x) { x ${operator} 2; return x; }`, + ); + expect(binding?.status).toBe("unknown"); + expect(returned?.status).toBe("unknown"); + }, + ); + + it.each(["||=", "??=", "&&="])( + "keeps a destructured parameter uncertain after %s", + (operator) => { + const { binding, returned } = observe( + `function read({ x }) { x ${operator} 2; return x; }`, + ); + expect(binding?.status).toBe("unknown"); + expect(returned?.status).toBe("unknown"); + }, + ); + + it.each(["||=", "??=", "&&="])( + "keeps a catch binding uncertain after %s", + (operator) => { + const { binding, returned } = observe( + `function read() { try { run(); } catch (x) { x ${operator} 2; return x; } }`, + ); + expect(binding?.status).toBe("unknown"); + expect(returned?.status).toBe("unknown"); + }, + ); + + it.each([ + ["let", "||="], + ["let", "??="], + ["var", "||="], + ["var", "??="], + ])( + "keeps an uninitialized %s local exact after %s", + (declaration, operator) => { + const { binding, returned } = observe( + `function read() { ${declaration} x; x ${operator} 2; return x; }`, + ); + expect(binding).toEqual({ status: "literal", value: 2 }); + expect(returned).toEqual({ status: "literal", value: 2 }); + }, + ); + + it("keeps an unconditional parameter assignment exact", () => { + const { binding, returned } = observe( + "function read(x) { x = 2; return x; }", + ); + expect(binding).toEqual({ status: "literal", value: 2 }); + expect(returned).toEqual({ status: "literal", value: 2 }); + }); + + it("keeps an unchanged parameter unknown", () => { + expect(observe("function read(x) { return x; }").returned?.status).toBe( + "unknown", + ); + }); + + it("reproduces the issue #2646 inputs without an exact literal", () => { + for (const operator of ["||=", "??="]) { + const { binding, returned } = observe( + `function read(x) { x ${operator} 2; return x; }`, + ); + expect(binding).not.toEqual({ status: "literal", value: 2 }); + expect(returned).not.toEqual({ status: "literal", value: 2 }); + } + const readOr = (x: number) => { + x ||= 2; + return x; + }; + const readNullish = (x: number) => { + x ??= 2; + return x; + }; + expect(readOr(5)).toBe(5); + expect(readNullish(5)).toBe(5); + }); +}); diff --git a/src/domain/javascript/javascriptSemanticAnalysis.ts b/src/domain/javascript/javascriptSemanticAnalysis.ts index f6c6db25d..119fbdd45 100644 --- a/src/domain/javascript/javascriptSemanticAnalysis.ts +++ b/src/domain/javascript/javascriptSemanticAnalysis.ts @@ -247,19 +247,24 @@ export function* classifyParsedJavaScriptRpcClientRootsSteps( return roots; } -/** Receiver identity facts for the overloaded `.open` syntax only. */ +/** + * Receiver identity facts for the overloaded `.open` syntax, plus + * `configuration` for `.get` reads of a `getConfiguration(...)` result. + */ export type JavaScriptOpenReceiverFact = | "window" | "indexed-db" | "cache-storage" | "local" | "local-indexed-db" - | "local-cache-storage"; + | "local-cache-storage" + | "configuration"; /** * Resolve only the lexical receiver identity needed to distinguish browser - * `window.open` from XHR and global storage `.open` calls. This deliberately - * builds definitions without running semantic value or provenance analysis. + * `window.open` from XHR and global storage `.open` calls, and configuration + * `.get` reads from HTTP client calls. This deliberately builds definitions + * without running semantic value or provenance analysis. */ export const classifyParsedJavaScriptOpenReceivers = ( file: ParsedJavaScriptSource, @@ -278,14 +283,19 @@ export function* classifyParsedJavaScriptOpenReceiversSteps( if ( (!t.isCallExpression(node) && !t.isOptionalCallExpression(node)) || (!t.isMemberExpression(node.callee) && - !t.isOptionalMemberExpression(node.callee)) || - semanticStaticPropertyKey( - node.callee.property, - node.callee.computed, - ) !== "open" || - !t.isIdentifier(node.callee.object) + !t.isOptionalMemberExpression(node.callee)) ) return; + const method = semanticStaticPropertyKey( + node.callee.property, + node.callee.computed, + ); + if (method === "get") { + if (isConfigurationReceiver(state, node.callee.object)) + facts.set(node.start ?? -1, "configuration"); + return; + } + if (method !== "open" || !t.isIdentifier(node.callee.object)) return; const receiver = node.callee.object.name; const binding = resolveSemanticBindingState(state, node, receiver); if (binding === undefined) { @@ -351,6 +361,82 @@ export function* classifyParsedJavaScriptOpenReceiversSteps( return facts; } +/** + * Whether a call spells VS Code's `workspace.getConfiguration(section)`, whose + * result `.get` reads setting keys rather than request paths. The `workspace` + * member is required: a bare or differently scoped `getConfiguration` may be + * an application's own HTTP client factory. Minifiers keep property names, so + * bundled `x.workspace.getConfiguration` still matches. + */ +const isConfigurationAccessorCall = ( + state: JavaScriptSemanticAnalysisState, + node: t.Node, +): boolean => { + if (!t.isCallExpression(node) && !t.isOptionalCallExpression(node)) + return false; + const callee = node.callee; + if ( + (!t.isMemberExpression(callee) && !t.isOptionalMemberExpression(callee)) || + semanticStaticPropertyKey(callee.property, callee.computed) !== + "getConfiguration" + ) + return false; + const host = callee.object; + const matchesSpelling = t.isIdentifier(host) + ? host.name === "workspace" + : (t.isMemberExpression(host) || t.isOptionalMemberExpression(host)) && + semanticStaticPropertyKey(host.property, host.computed) === "workspace"; + if (!matchesSpelling) return false; + const root = t.isIdentifier(host) + ? host + : t.isMemberExpression(host) || t.isOptionalMemberExpression(host) + ? host.object + : undefined; + if (!t.isIdentifier(root)) return false; + const binding = resolveSemanticBindingState(state, root, root.name); + if (binding === undefined) + return !semanticResolutionBlocked(state, root, root.name); + const initializer = semanticStableInitializer(binding, state); + const origins = [ + ...binding.directOrigins, + ...(initializer === undefined + ? [] + : [semanticRequireOrigin(initializer.node, state)?.origin]), + ]; + return origins.some( + (origin) => + origin?.specifier === "vscode" && + (t.isIdentifier(host) + ? origin.importedPath.at(-1) === "workspace" || + initializer?.projection.at(-1) === "workspace" + : origin.importedPath.length === 0 || + (origin.importedPath.length === 1 && + (origin.importedPath[0] === "*" || + origin.importedPath[0] === "default"))), + ); +}; + +/** + * Whether a `.get` receiver is a configuration accessor result, either inline + * or through a binding whose only visible write is one unconditional + * initializer. Minifiers emit `let` for such bindings, so declaration kind + * alone does not decide. + */ +const isConfigurationReceiver = ( + state: JavaScriptSemanticAnalysisState, + receiver: t.Node, +): boolean => { + if (isConfigurationAccessorCall(state, receiver)) return true; + if (!t.isIdentifier(receiver)) return false; + const binding = resolveSemanticBindingState(state, receiver, receiver.name); + const initializer = semanticStableInitializer(binding, state); + return ( + initializer !== undefined && + initializer.projection.length === 0 && + isConfigurationAccessorCall(state, initializer.node) + ); +}; + const openGlobalFact = ( name: string, ): JavaScriptOpenReceiverFact | undefined => { @@ -833,6 +919,21 @@ const bindImportEquals = ( }); }; +// A parameter or catch binding starts with a caller-supplied value that a +// conditional logical assignment can retain, so its RHS is not exact. +const hasIncomingValue = ( + identifier: t.Identifier, + scope: JavaScriptSemanticScopeState, + state: JavaScriptSemanticAnalysisState, +): boolean => { + const kind = resolveSemanticBindingFromScope( + scope, + identifier.name, + state, + )?.kind; + return kind === "parameter" || kind === "catch"; +}; + const bindAssignment = ( node: t.Node, scope: JavaScriptSemanticScopeState, @@ -843,8 +944,8 @@ const bindAssignment = ( addAssignment( node.left, node.operator === "=" || - node.operator === "||=" || - node.operator === "??=" + ((node.operator === "||=" || node.operator === "??=") && + !hasIncomingValue(node.left, scope, state)) ? node.right : node, scope, diff --git a/src/domain/javascript/javascriptSemanticChildProcesses.ts b/src/domain/javascript/javascriptSemanticChildProcesses.ts index fe7b5e802..1cffb5164 100644 --- a/src/domain/javascript/javascriptSemanticChildProcesses.ts +++ b/src/domain/javascript/javascriptSemanticChildProcesses.ts @@ -324,17 +324,25 @@ const resolveChildCandidates = ( ? [] : (spawnsByBinding.get(binding.bindingId) ?? []); }; - const spawnOptions = (candidate: SpawnCandidate): t.ObjectExpression | null => { - const index = candidate.method === "exec" ? 1 : 2; - const value = candidate.node.arguments[index]; + const args = candidate.node.arguments; + + const value = + candidate.method === "exec" + ? args[1] + : t.isObjectExpression(args[1]) + ? args[1] + : args[2]; + return t.isObjectExpression(value) ? value : null; }; - const spawnArgvCount = (candidate: SpawnCandidate): number | null => { if (candidate.method === "exec") return null; const args = candidate.node.arguments[1]; - return t.isArrayExpression(args) ? args.elements.length : null; + return t.isArrayExpression(args) && + !args.elements.some((element) => t.isSpreadElement(element)) + ? args.elements.length + : null; }; const objectHasProperty = ( diff --git a/src/domain/javascript/javascriptSemanticDataEffectHelpers.ts b/src/domain/javascript/javascriptSemanticDataEffectHelpers.ts index 5485fcde9..0f7a6d942 100644 --- a/src/domain/javascript/javascriptSemanticDataEffectHelpers.ts +++ b/src/domain/javascript/javascriptSemanticDataEffectHelpers.ts @@ -275,12 +275,3 @@ export const dataEffectMemberCallee = ( t.isMemberExpression(node.callee) || t.isOptionalMemberExpression(node.callee) ? node.callee : null; - -/** Read a parent member's object when present. */ -export const dataEffectMemberObject = ( - node: t.Node | null, -): t.Expression | t.Super | null => - node !== null && - (t.isMemberExpression(node) || t.isOptionalMemberExpression(node)) - ? node.object - : null; diff --git a/src/domain/javascript/javascriptSemanticDataEffects.ts b/src/domain/javascript/javascriptSemanticDataEffects.ts index 445cd2e5e..607843aab 100644 --- a/src/domain/javascript/javascriptSemanticDataEffects.ts +++ b/src/domain/javascript/javascriptSemanticDataEffects.ts @@ -24,7 +24,6 @@ import { dataEffectArgumentBindingId as argumentBindingId, dataEffectLiteralString as literalString, dataEffectMemberCallee as memberCallee, - dataEffectMemberObject as memberObject, dataEffectPrimitive as primitiveValue, isSemanticProcessObject as isProcessObject, outerDataEffectBinding as outerAssignedBinding, @@ -83,8 +82,8 @@ const collectConfigurations = ( context: TraversalContext, ): ConfigurationCandidate[] => { const output: ConfigurationCandidate[] = []; - traverseWithContext(program, context, (node, parent) => { - const details = configurationDetails(node, parent, context); + traverseWithContext(program, context, (node) => { + const details = configurationDetails(node, context); if (details === null) return; const operation: JavaScriptSemanticConfigurationOperation = { configId: `config:${details.kind}:${String(node.start ?? -1)}:${String(node.end ?? -1)}`, @@ -104,17 +103,13 @@ const collectConfigurations = ( const configurationDetails = ( node: t.Node, - parent: t.Node | null, context: TraversalContext, ): { readonly kind: "environment" | "argv" | "file"; readonly key: string | null; readonly resolution: "complete" | "partial"; } | null => { - if ( - (t.isMemberExpression(node) || t.isOptionalMemberExpression(node)) && - !(memberObject(parent) === node) - ) { + if (t.isMemberExpression(node) || t.isOptionalMemberExpression(node)) { const processMember = processConfigurationMember(node, context.state); if (processMember !== null) return processMember; } diff --git a/src/domain/javascript/javascriptStaticAnalysisCalls.ts b/src/domain/javascript/javascriptStaticAnalysisCalls.ts index 148f7fb7e..b098b3c26 100644 --- a/src/domain/javascript/javascriptStaticAnalysisCalls.ts +++ b/src/domain/javascript/javascriptStaticAnalysisCalls.ts @@ -429,7 +429,12 @@ export const addSourceMapDirectives = ( end === null ) continue; - const commentSource = source.slice(start, end); + // The parser owns comment boundaries. Exclude only the closing block + // delimiter; an asterisk elsewhere remains part of the declared URL. + const commentSource = source.slice( + start, + comment.type === "CommentBlock" ? end - 2 : end, + ); if ( !(commentSource.startsWith("//") || commentSource.startsWith("/*")) || !(commentSource[2] === "#" || commentSource[2] === "@") @@ -449,9 +454,7 @@ export const addSourceMapDirectives = ( const urlOffset = commentSource.slice(equalsStart + 1).search(/\S/u); if (urlOffset === -1) continue; const declaredStart = equalsStart + 1 + urlOffset; - const terminator = commentSource - .slice(declaredStart) - .search(commentSource.startsWith("//") ? /\s/u : /[\s*]/u); + const terminator = commentSource.slice(declaredStart).search(/\s/u); const declaredEnd = terminator === -1 ? commentSource.length : declaredStart + terminator; if (declaredEnd === declaredStart) continue; diff --git a/src/domain/javascript/javascriptStaticAnalysisHelpers.ts b/src/domain/javascript/javascriptStaticAnalysisHelpers.ts index f2045a1eb..71628bc8c 100644 --- a/src/domain/javascript/javascriptStaticAnalysisHelpers.ts +++ b/src/domain/javascript/javascriptStaticAnalysisHelpers.ts @@ -17,6 +17,7 @@ import { semanticStaticPropertyName, unwrapJavaScriptExpression, } from "./javascriptAstValues.js"; +import type { JavaScriptOpenReceiverFact } from "./javascriptSemanticAnalysis.js"; import { compareUnicodeCodePoints, compositeKey, @@ -470,13 +471,7 @@ const xhrOpenUrl = ( name: string, methodNode: t.Node | null | undefined, urlNode: t.Node | null | undefined, - receiverFact?: - | "window" - | "indexed-db" - | "cache-storage" - | "local" - | "local-indexed-db" - | "local-cache-storage", + receiverFact?: JavaScriptOpenReceiverFact, ): string | undefined => { const url = stringValue(urlNode); if (url === undefined) return undefined; @@ -555,13 +550,7 @@ export const endpointArgument = ( | t.ArgumentPlaceholder )[], callee: t.Node, - receiverFact?: - | "window" - | "indexed-db" - | "cache-storage" - | "local" - | "local-indexed-db" - | "local-cache-storage", + receiverFact?: JavaScriptOpenReceiverFact, ): string | undefined => { if (receiverFact === "window") return undefined; if (name === "fetch" || name.endsWith(".fetch") || name === "WebSocket") @@ -575,10 +564,12 @@ export const endpointArgument = ( const value = stringValue(args[0]); // Only get and delete also belong to the keyed-collection APIs, whose // keys are header, field, and parameter names rather than URLs or paths. + // Configuration `get` reads setting keys. return value !== undefined && (method === "get" || method === "delete") && !isRequestUrlLiteral(value) && - isKeyedCollectionReceiver(callee) + (isKeyedCollectionReceiver(callee) || + (method === "get" && receiverFact === "configuration")) ? undefined : value; } diff --git a/src/domain/javascript/javascriptStaticAnalysisSourceMaps.test.ts b/src/domain/javascript/javascriptStaticAnalysisSourceMaps.test.ts index f5664f73c..6779d35bd 100644 --- a/src/domain/javascript/javascriptStaticAnalysisSourceMaps.test.ts +++ b/src/domain/javascript/javascriptStaticAnalysisSourceMaps.test.ts @@ -72,10 +72,10 @@ describe("source map directives", () => { it("preserves Unicode whitespace and URL terminator boundaries", () => { const source = "ğ/*@\u00a0sourceMappingURL\u2003=\ufeffapp.js.map* ignored */"; - const declaredEnd = source.indexOf("* ignored"); + const declaredEnd = source.indexOf(" ignored"); expect(analyzeJavaScriptStaticSource(source).source_map_urls).toEqual([ { - declared_url: "app.js.map", + declared_url: "app.js.map*", location: { start: { line: 1, column: 1 }, end: { line: 1, column: declaredEnd }, @@ -84,6 +84,29 @@ describe("source map directives", () => { ]); }); + it.each([ + ["/*#", "app*part.js.map", " */"], + ["/*@", "*part.js.map", "*/"], + ["/*#", "app.js.map*", "*/"], + ["/*#", "**part.js.map", " */"], + ["/*#", "app.js.map", "*/"], + ["//#", "app*part.js.map", ""], + ])("preserves the URL and range in %s %s%s", (marker, url, suffix) => { + const directive = marker + " sourceMappingURL=" + url; + const result = analyzeJavaScriptStaticSource( + "export const value = 1;\n" + directive + suffix, + ); + expect(result.source_map_urls).toEqual([ + { + declared_url: url, + location: { + start: { line: 2, column: 0 }, + end: { line: 2, column: directive.length }, + }, + }, + ]); + }); + it("reports nothing when no directive is present", () => { expect(sourceMapUrls("const value = 1;")).toEqual([]); }); diff --git a/src/domain/javascript/javascriptStaticAnalysisState.ts b/src/domain/javascript/javascriptStaticAnalysisState.ts index 3c93d2fa3..5173362fa 100644 --- a/src/domain/javascript/javascriptStaticAnalysisState.ts +++ b/src/domain/javascript/javascriptStaticAnalysisState.ts @@ -1,4 +1,5 @@ import type * as t from "@babel/types"; +import type { JavaScriptOpenReceiverFact } from "./javascriptSemanticAnalysis.js"; import type { JavaScriptRpcScan } from "./javascriptRpcEndpoints.js"; import type { @@ -64,15 +65,7 @@ export interface JavaScriptFindingContext { readonly rpcScan: JavaScriptRpcScan; readonly source: string; readonly accumulator: JavaScriptAnalysisAccumulator; - readonly openReceiverFacts?: ReadonlyMap< - number, - | "window" - | "indexed-db" - | "cache-storage" - | "local" - | "local-indexed-db" - | "local-cache-storage" - >; + readonly openReceiverFacts?: ReadonlyMap; /** Proven Electron export path segments by root offset. */ readonly electronBindings?: ReadonlyMap; } diff --git a/src/domain/javascript/javascriptSymbolicBindings.ts b/src/domain/javascript/javascriptSymbolicBindings.ts index b1f37269f..62c70f953 100644 --- a/src/domain/javascript/javascriptSymbolicBindings.ts +++ b/src/domain/javascript/javascriptSymbolicBindings.ts @@ -47,6 +47,14 @@ interface SymbolicReturnFlow { readonly fallsThrough: boolean; } const emptyEnvironment: SymbolicEnvironment = { values: new Map() }; +const isObjectPrototypeSetter = ( + property: t.ObjectMethod | t.ObjectProperty, + name: string, +): boolean => + t.isObjectProperty(property) && + !property.computed && + !property.shorthand && + name === "__proto__"; /** Demand-driven substitutions for local bindings, closures and literal module tables. */ export class JavaScriptSymbolicBindings { @@ -243,15 +251,30 @@ export class JavaScriptSymbolicBindings { index = null; break; } + if (isObjectPrototypeSetter(property, name)) continue; properties.set(name, property); } this.#literalPropertyIndex.set(value.node, index); } if (index !== null) { const property = index.get(key); - return property === undefined - ? symbolicUnknown("missing property") - : this.#objectPropertyValue(property, value.environment); + if (property !== undefined) + return this.#objectPropertyValue(property, value.environment); + if (key === "__proto__") { + const prototypeSetter = [...value.node.properties] + .reverse() + .filter( + (candidate): candidate is t.ObjectMethod | t.ObjectProperty => + !t.isSpreadElement(candidate), + ) + .find((candidate) => isObjectPrototypeSetter(candidate, key)); + if (prototypeSetter !== undefined) + return this.#objectPropertyValue( + prototypeSetter, + value.environment, + ); + } + return symbolicUnknown("missing property"); } for (const p of [...value.node.properties].reverse()) { if (t.isSpreadElement(p)) { @@ -314,7 +337,8 @@ export class JavaScriptSymbolicBindings { t.isSpreadElement(p) ? this.members(this.evaluate(p.argument, value.environment), active) : [semanticStaticPropertyKey(p.key, p.computed)].filter( - (k): k is string => k !== null, + (k): k is string => + k !== null && !isObjectPrototypeSetter(p, k), ), ), ), diff --git a/src/domain/managed/managedNativeVerificationSchemas.ts b/src/domain/managed/managedNativeVerificationSchemas.ts index 8e543bdab..16afe4048 100644 --- a/src/domain/managed/managedNativeVerificationSchemas.ts +++ b/src/domain/managed/managedNativeVerificationSchemas.ts @@ -92,7 +92,10 @@ export type PinvokeVerification = z.infer; export const managedNativeVerificationInputSchema = z .strictObject({ managed_boundaries: evidenceSchema, - native_observations: z.array(evidenceSchema).min(1), + native_observations: z + .array(evidenceSchema) + .min(1) + .describe("Native function or export Evidence records"), }) .superRefine((input, context) => { const ids = new Set(); diff --git a/src/domain/native/addressCoordinate.ts b/src/domain/native/addressCoordinate.ts new file mode 100644 index 000000000..d43b1b4f1 --- /dev/null +++ b/src/domain/native/addressCoordinate.ts @@ -0,0 +1,11 @@ +/** Parse canonical native offsets without discarding their address space. */ +export const addressCoordinate = ( + value: string, +): { readonly space: string | null; readonly offset: bigint } | null => { + const separator = value.lastIndexOf(":0x"); + if (separator === 0) return null; + const space = separator < 0 ? null : value.slice(0, separator); + const offset = separator < 0 ? value : value.slice(separator + 1); + if (!/^0x[0-9a-f]+$/u.test(offset)) return null; + return { space, offset: BigInt(offset) }; +}; diff --git a/src/domain/native/bytePattern.test.ts b/src/domain/native/bytePattern.test.ts new file mode 100644 index 000000000..cc3b6dd9a --- /dev/null +++ b/src/domain/native/bytePattern.test.ts @@ -0,0 +1,33 @@ +import { describe, expect, it } from "vitest"; +import { parseBytePattern } from "./bytePattern.js"; + +describe("masked byte patterns", () => { + it("preserves whole-byte wildcards and combines explicit bit masks", () => { + const parsed = parseBytePattern("48 8b\t??89", "ff f0 ff 0f"); + if (!parsed.ok) throw parsed.error; + expect(parsed.value).toEqual({ + bytes: [0x48, 0x8b, 0, 0x89], + masks: [255, 240, 0, 15], + }); + const exact = parseBytePattern("Aa\u00a000"); + if (!exact.ok) throw exact.error; + expect(exact.value).toEqual({ bytes: [170, 0], masks: [255, 255] }); + }); + + it.each([ + ["", undefined, "pattern"], + ["4", undefined, "pattern"], + ["4?", undefined, "pattern"], + ["gg", undefined, "pattern"], + ["?? ??", undefined, "pattern"], + ["ff", "00", "pattern"], + ["ffff", "ff", "mask"], + ["ff", "??", "mask"], + ])("rejects invalid pattern %s with typed issues", (pattern, mask, field) => { + const parsed = parseBytePattern(pattern, mask); + if (parsed.ok) throw new Error("Expected a rejected pattern"); + expect(parsed.error.issues).toMatchObject([ + { path: [field], reason: "invalid_value" }, + ]); + }); +}); diff --git a/src/domain/native/bytePattern.ts b/src/domain/native/bytePattern.ts new file mode 100644 index 000000000..add5bb7ef --- /dev/null +++ b/src/domain/native/bytePattern.ts @@ -0,0 +1,125 @@ +import { z } from "zod"; +import { addressCoordinate } from "./addressCoordinate.js"; +import { AnalysisInputError } from "../analysisErrorCore.js"; +import { err, ok, type Result } from "../result.js"; + +/** Byte pairs, with whole-byte wildcards; explicit masks can select individual bits. */ +export const bytePatternFields = { + pattern: z + .string() + .regex(/^([0-9a-fA-F]{2}|\?\?)(\s*([0-9a-fA-F]{2}|\?\?))*$/u), + mask: z + .string() + .regex(/^[0-9a-fA-F]{2}(\s*[0-9a-fA-F]{2})*$/u) + .optional(), +}; + +/** Parse once at the input boundary, retaining typed correction issues. */ +export const parseBytePattern = ( + pattern: string, + mask?: string, +): Result<{ bytes: number[]; masks: number[] }, AnalysisInputError> => { + const invalid = (field: string, message: string) => + err( + new AnalysisInputError("search_bytes", undefined, [ + { path: [field], reason: "invalid_value", message }, + ]), + ); + if (!bytePatternFields.pattern.safeParse(pattern).success) + return invalid("pattern", "Use hexadecimal byte pairs or ?? wildcards"); + if (mask !== undefined && !bytePatternFields.mask.safeParse(mask).success) + return invalid("mask", "Use hexadecimal mask byte pairs"); + const pairs = pattern.replace(/\s/gu, "").match(/../gu) ?? []; + const maskPairs = mask?.replace(/\s/gu, "").match(/../gu); + if (maskPairs && maskPairs.length !== pairs.length) + return invalid("mask", "Mask length must equal pattern length"); + const bytes = pairs.map((pair) => + pair === "??" ? 0 : Number.parseInt(pair, 16), + ); + const masks = pairs.map((pair, index) => + pair === "??" ? 0 : Number.parseInt(maskPairs?.[index] ?? "ff", 16), + ); + if (!masks.some((value) => value !== 0)) + return invalid("pattern", "Pattern must constrain at least one bit"); + return ok({ bytes, masks }); +}; + +/** Shared semantic validation for public and provider inputs. */ +export const validateBytePattern = ( + value: { pattern: string; mask?: string | undefined }, + context: z.RefinementCtx, +): void => { + const parsed = parseBytePattern(value.pattern, value.mask); + if (!parsed.ok) + for (const issue of parsed.error.issues) + context.addIssue({ + code: "custom", + path: [...issue.path], + message: issue.message ?? "Invalid byte pattern", + }); +}; + +const coverageBlock = z + .strictObject({ + segment: z.string(), + start: z.string(), + end: z.string(), + bytes: z.number().int().nonnegative().max(Number.MAX_SAFE_INTEGER), + }) + .superRefine((value, context) => { + const start = addressCoordinate(value.start); + const end = addressCoordinate(value.end); + if ( + start === null || + end === null || + start.space !== end.space || + start.offset > end.offset || + end.offset - start.offset + 1n !== BigInt(value.bytes) + ) + context.addIssue({ + code: "custom", + message: + "Byte coverage must equal its inclusive address range in one space", + }); + }); + +/** Inclusive page coverage; matches never span memory blocks. */ +export const byteSearchResultSchema = z + .strictObject({ + matches: z + .array( + z.strictObject({ + address: z.string(), + segment: z.string(), + file_offset: z + .number() + .int() + .nonnegative() + .max(Number.MAX_SAFE_INTEGER) + .nullable(), + procedure: z + .strictObject({ address: z.string(), name: z.string() }) + .nullable(), + }), + ) + .max(1000), + searched: z.strictObject({ + blocks: z.array(coverageBlock), + bytes: z.number().int().nonnegative().max(Number.MAX_SAFE_INTEGER), + skipped_uninitialized: z.array(coverageBlock), + skipped_not_loaded: z.array(coverageBlock), + }), + complete: z.boolean(), + next_cursor: z.string().min(1).nullable(), + }) + .superRefine((value, context) => { + if ( + value.complete !== (value.next_cursor === null) || + value.searched.bytes !== + value.searched.blocks.reduce((sum, block) => sum + block.bytes, 0) + ) + context.addIssue({ + code: "custom", + message: "Byte search coverage and continuation disagree", + }); + }); diff --git a/src/domain/native/objcSwiftMetadata.ts b/src/domain/native/objcSwiftMetadata.ts index 5778e5dd7..a360c3202 100644 --- a/src/domain/native/objcSwiftMetadata.ts +++ b/src/domain/native/objcSwiftMetadata.ts @@ -1,4 +1,5 @@ import { z } from "zod"; +import { providerSchema } from "../evidence.js"; import { compareUnicodeCodePoints, @@ -115,6 +116,7 @@ export const nativeMetadataEvidenceSchema = z.strictObject({ "decompiler", ]), description: z.string().min(1), + provider: providerSchema.optional(), location: nativeMetadataLocationSchema, artifact_path: z.string().min(1).nullable().default(null), artifact_sha256: z diff --git a/src/domain/native/recordedCrash.test.ts b/src/domain/native/recordedCrash.test.ts index 7a050d417..18414128b 100644 --- a/src/domain/native/recordedCrash.test.ts +++ b/src/domain/native/recordedCrash.test.ts @@ -1,7 +1,8 @@ -import { expect, it } from "vitest"; +import { describe, expect, it } from "vitest"; import { inspectRecordedCrashInputSchema, recordedCrashSchema, + type RecordedCrash, } from "./recordedCrash.js"; import { recordedCrashFixture, @@ -282,3 +283,95 @@ it("keeps absent map flags unknown and validates reported permission bits", () = }).success, ).toBe(false); }); + +describe("recorded AArch64 core profile", () => { + it("accepts the complete AArch64 register window in ABI order", () => { + const value = recordedCrashSchema.parse( + recordedCrashFixture("/selected/aarch64.core", "aarch64-little-endian"), + ); + expect(value.architecture).toBe("aarch64-little-endian"); + const registers = value.threads[0]?.registers ?? []; + expect(registers.map(({ name }) => name)).toEqual([ + ...Array.from({ length: 31 }, (_, index) => `x${String(index)}`), + "sp", + "pc", + "pstate", + ]); + expect(registers[14]?.value).toBe("0x1122334455667788"); + expect(registers.every(({ location }) => location.bytes === "0x8")).toBe( + true, + ); + }); + + it("keeps the AArch64 note ranges inside their recorded segment", () => { + const value = recordedCrashSchema.parse( + recordedCrashSignalFixture(undefined, "aarch64-little-endian"), + ); + const note = value.notes[0]; + const segment = value.segments[0]; + if (note === undefined || segment === undefined) + throw new Error("Expected recorded notes"); + expect( + BigInt(note.location.offset) + BigInt(note.location.bytes), + ).toBeLessThanOrEqual(BigInt(segment.offset) + BigInt(segment.file_size)); + expect(value.signals[0]?.fault_address).toBe("0x1122334455667788"); + }); + + it.each([ + [ + "an amd64 register name inside an AArch64 window", + (v: RecordedCrash) => ({ + ...v, + threads: v.threads.map((thread) => ({ + ...thread, + registers: thread.registers.map((register, index) => + index === 0 ? { ...register, name: "rax" } : register, + ), + })), + }), + ], + [ + "the amd64 register count", + (v: RecordedCrash) => ({ + ...v, + threads: v.threads.map((thread) => ({ + ...thread, + registers: thread.registers.slice(0, 27), + })), + }), + ], + [ + "a 4-byte AArch64 register source", + (v: RecordedCrash) => ({ + ...v, + threads: v.threads.map((thread) => ({ + ...thread, + registers: thread.registers.map((register, index) => + index === 0 + ? { + ...register, + location: { ...register.location, bytes: "0x4" }, + } + : register, + ), + })), + }), + ], + ] as const)("rejects %s", (_name, change) => { + const value = recordedCrashFixture( + "/selected/aarch64.core", + "aarch64-little-endian", + ); + expect(recordedCrashSchema.safeParse(change(value)).success).toBe(false); + }); + + it("rejects an architecture outside the interpreted profiles", () => { + const value = recordedCrashFixture(); + expect( + recordedCrashSchema.safeParse({ + ...value, + architecture: "arm-little-endian", + }).success, + ).toBe(false); + }); +}); diff --git a/src/domain/native/recordedCrash.ts b/src/domain/native/recordedCrash.ts index d4b533838..8bd04cac1 100644 --- a/src/domain/native/recordedCrash.ts +++ b/src/domain/native/recordedCrash.ts @@ -1,5 +1,6 @@ import { z } from "zod"; +import type { BinaryArchitecture } from "../binaryTargetTypes.js"; import { localPathStringSchema } from "../localPath.js"; import { CANONICAL_BASE64_PATTERN } from "../stringPatterns.js"; import { recordedNoteCoverageIssues } from "./recordedCrashNoteCoverage.js"; @@ -18,7 +19,7 @@ const diagnostics = z.strictObject({ const LINUX_CORE_OWNER_BASE64 = "Q09SRQA="; /** Complete register identities in the supported Linux amd64 PRSTATUS ABI order. */ -export const RECORDED_LINUX_AMD64_REGISTERS: readonly string[] = [ +const RECORDED_LINUX_AMD64_REGISTERS: readonly string[] = [ "r15", "r14", "r13", @@ -47,7 +48,95 @@ export const RECORDED_LINUX_AMD64_REGISTERS: readonly string[] = [ "fs", "gs", ]; -const supportedRegisters = new Set(RECORDED_LINUX_AMD64_REGISTERS); +const RECORDED_LINUX_AARCH64_REGISTERS: readonly string[] = [ + "x0", + "x1", + "x2", + "x3", + "x4", + "x5", + "x6", + "x7", + "x8", + "x9", + "x10", + "x11", + "x12", + "x13", + "x14", + "x15", + "x16", + "x17", + "x18", + "x19", + "x20", + "x21", + "x22", + "x23", + "x24", + "x25", + "x26", + "x27", + "x28", + "x29", + "x30", + "sp", + "pc", + "pstate", +]; + +/** Recorded-core architectures whose Linux note ABI is interpreted. */ +export const RECORDED_LINUX_CORE_ARCHITECTURES = [ + "x86_64-little-endian", + "aarch64-little-endian", +] as const; + +export type RecordedCrashArchitecture = + (typeof RECORDED_LINUX_CORE_ARCHITECTURES)[number]; + +/** + * ABI facts for one recorded-core architecture, measured from unchanged pwntools + * 4.15.0 `elf_prstatus_*` and `user_regs_struct_*` structures and mirrored by + * `bridge/pwntools/recorded_crash_impl.py`. Both profiles place `pr_cursig` at + * 12, `pr_pid` at 32 and the register window at 112; the ELF machine, the window + * contents and the Evidence CPU family differ per profile. + */ +export const RECORDED_LINUX_CORE_PROFILES: Record< + RecordedCrashArchitecture, + { + readonly machine: number; + readonly evidenceArchitecture: BinaryArchitecture; + readonly registers: readonly string[]; + readonly registerBytes: number; + readonly registerOffset: number; + readonly prstatusBytes: number; + } +> = { + "x86_64-little-endian": { + machine: 62, + evidenceArchitecture: "x86_64", + registers: RECORDED_LINUX_AMD64_REGISTERS, + registerBytes: 8, + registerOffset: 112, + prstatusBytes: 336, + }, + "aarch64-little-endian": { + machine: 183, + evidenceArchitecture: "arm64", + registers: RECORDED_LINUX_AARCH64_REGISTERS, + registerBytes: 8, + registerOffset: 112, + prstatusBytes: 392, + }, +}; + +const recordedCoreRegisterSets: Record< + RecordedCrashArchitecture, + ReadonlySet +> = { + "x86_64-little-endian": new Set(RECORDED_LINUX_AMD64_REGISTERS), + "aarch64-little-endian": new Set(RECORDED_LINUX_AARCH64_REGISTERS), +}; /** Select an explicit recording and optionally request debugger-derived mapping context. */ export const inspectRecordedCrashInputSchema = z.strictObject({ @@ -108,7 +197,7 @@ const recordedCrashObjectSchema = z.strictObject({ bytes: index, }), format: z.literal("elf-core"), - architecture: z.literal("x86_64-little-endian"), + architecture: z.enum(RECORDED_LINUX_CORE_ARCHITECTURES), target_execution: z.literal("not-performed"), live_process_identity: z.literal("unknown"), segments: z.array( @@ -284,6 +373,8 @@ export const recordedCrashSchema = recordedCrashObjectSchema.superRefine( "Each supported raw note requires exactly one corresponding interpretation.", ); } + const profile = RECORDED_LINUX_CORE_PROFILES[value.architecture]; + const profileRegisters = recordedCoreRegisterSets[value.architecture]; for (const thread of value.threads) { const note = value.notes[thread.note_index]; if ( @@ -295,8 +386,8 @@ export const recordedCrashSchema = recordedCrashObjectSchema.superRefine( continue; } if ( - thread.registers.length !== RECORDED_LINUX_AMD64_REGISTERS.length || - !thread.registers.every(({ name }) => supportedRegisters.has(name)) || + thread.registers.length !== profile.registers.length || + !thread.registers.every(({ name }) => profileRegisters.has(name)) || new Set(thread.registers.map(({ name }) => name)).size !== thread.registers.length ) @@ -305,7 +396,7 @@ export const recordedCrashSchema = recordedCrashObjectSchema.superRefine( ); for (const register of thread.registers) if ( - BigInt(register.location.bytes) !== 8n || + BigInt(register.location.bytes) !== BigInt(profile.registerBytes) || !within( register.location, BigInt(note.descriptor_location.offset), diff --git a/src/domain/process/processCapture.comparison.test.ts b/src/domain/process/processCapture.comparison.test.ts index 4d238da2d..0bc69335d 100644 --- a/src/domain/process/processCapture.comparison.test.ts +++ b/src/domain/process/processCapture.comparison.test.ts @@ -192,3 +192,75 @@ it("keeps filesystem comparison unknown without coverage and reports observed ch }), ).toMatchObject({ filesystem: "changed", status: "changed" }); }); + +it.each([ + [500, "interaction", 6], + [4, "process", 4], +] as const)( + "orders an interaction difference by its dispatch time (process sample at %i ms)", + (processTime, dimension, at) => { + const capture = (side: "left" | "right") => { + const base = emptyCapture(); + const journal = [ + { collection: "filesystem_checkpoints" as const, index: 0 }, + ...(processTime < 6 + ? [{ collection: "process_samples" as const, index: 0 }] + : []), + { collection: "interaction_events" as const, index: 0 }, + ...(processTime >= 6 + ? [{ collection: "process_samples" as const, index: 0 }] + : []), + { collection: "lifecycle" as const, index: 0 }, + { collection: "lifecycle" as const, index: 1 }, + { collection: "filesystem_checkpoints" as const, index: 1 }, + ].map((entry, capture_order) => ({ ...entry, capture_order })); + return { + ...base, + manifest: { + ...base.manifest, + completed_at: "2026-01-01T00:00:01.000Z", + }, + settlement: { ...base.settlement, elapsed_ms: 1000 }, + filesystem_checkpoints: base.filesystem_checkpoints.map( + (checkpoint, index) => ({ + ...checkpoint, + at_ms: index === 0 ? 0 : 1000, + }), + ), + interaction_events: [ + { + sequence: 0, + scheduled_at_ms: 5, + dispatched_at_ms: 6, + type: "input" as const, + data: "hello", + outcome: + side === "left" ? ("dispatched" as const) : ("failed" as const), + }, + ], + process_samples: [ + { + at_ms: processTime, + pid: 2, + parent_pid: 1, + command: side === "left" ? "worker-a" : "worker-b", + process_group_id: 1, + session_id: 1, + }, + ], + event_journal: journal, + }; + }; + const comparison = compareFullyObserved(capture("left"), capture("right")); + expect(comparison).toMatchObject({ + interaction: "changed", + process: "changed", + first_divergence: { + status: "found", + dimension, + left_at_ms: at, + right_at_ms: at, + }, + }); + }, +); diff --git a/src/domain/process/processCapture.validation.test.ts b/src/domain/process/processCapture.validation.test.ts index df0344b7c..db36b2c64 100644 --- a/src/domain/process/processCapture.validation.test.ts +++ b/src/domain/process/processCapture.validation.test.ts @@ -839,3 +839,27 @@ it("requires compatible contracts and enforces capture age through a clock seam" }), ); }); + +it("requires interaction dispatch times to follow their sequence", () => { + const capture = emptyCapture(); + const interaction = (sequence: number, dispatched: number) => ({ + sequence, + scheduled_at_ms: 0, + dispatched_at_ms: dispatched, + type: "input" as const, + data: "x", + outcome: "dispatched" as const, + }); + const issues = (dispatched: readonly number[]) => + processCaptureIssues({ + ...capture, + interaction_events: dispatched.map((at, index) => interaction(index, at)), + }).filter(({ message }) => message.includes("dispatch timestamps")); + expect(issues([5, 5, 10])).toEqual([]); + expect(issues([10, 5])).toEqual([ + { + path: "interaction_events", + message: expect.stringContaining("dispatch timestamps must be ordered"), + }, + ]); +}); diff --git a/src/domain/process/processCaptureValidation.ts b/src/domain/process/processCaptureValidation.ts index 33c6d5483..537f6f92f 100644 --- a/src/domain/process/processCaptureValidation.ts +++ b/src/domain/process/processCaptureValidation.ts @@ -17,12 +17,10 @@ type RequireInvariant = ( message: string, ) => void; -const orderedTimestamps = ( - values: readonly { readonly at_ms: number }[], -): boolean => +const orderedTimestamps = (values: readonly number[]): boolean => values.every((value, index) => { const previous = values[index - 1]; - return previous === undefined || value.at_ms >= previous.at_ms; + return previous === undefined || value >= previous; }); const validateCommitments = ( @@ -66,7 +64,14 @@ const validateOrdering = ( ["rendered_frames", capture.rendered_frames], ["process_samples", capture.process_samples], ] as const) - require(orderedTimestamps(values), name, "timestamps must be ordered"); + require(orderedTimestamps( + values.map(({ at_ms }) => at_ms), + ), name, "timestamps must be ordered"); + // Interactions are dispatched in sequence, so their recorded dispatch + // times cannot decrease; comparison ranks a changed interaction by it. + require(orderedTimestamps( + capture.interaction_events.map(({ dispatched_at_ms }) => dispatched_at_ms), + ), "interaction_events", "dispatch timestamps must be ordered"); }; const validateEventJournal = ( @@ -242,7 +247,7 @@ const validateLifecycle = ( require(capture.filesystem_checkpoints.map(({ name }) => name).join(",") === "before,after_settlement", "filesystem_checkpoints", "capture must include initial and final filesystem snapshots"); require(orderedTimestamps( - capture.filesystem_checkpoints, + capture.filesystem_checkpoints.map(({ at_ms }) => at_ms), ), "filesystem_checkpoints", "filesystem snapshot times must be ordered"); require(!capture.filesystem_checkpoints.some(({ truncated }) => truncated) || capture.truncated, "truncated", "filesystem snapshot truncation must propagate to the capture"); diff --git a/src/domain/process/processComparison.ts b/src/domain/process/processComparison.ts index 21c7bd80b..eafcd4fd8 100644 --- a/src/domain/process/processComparison.ts +++ b/src/domain/process/processComparison.ts @@ -165,17 +165,25 @@ type DivergenceDimension = Exclude< { readonly status: "none" | "unknown" } >["dimension"]; -const eventTime = (value: unknown): number | null => { - if ( - typeof value === "object" && - value !== null && - "at_ms" in value && - typeof value.at_ms === "number" && - Number.isSafeInteger(value.at_ms) && - value.at_ms >= 0 - ) - return value.at_ms; - return null; +// Interactions record when their outcome was observed as the dispatch time; +// every other timed observation uses at_ms. +const EVENT_TIME_FIELD: Readonly> = { + terminal: "at_ms", + interaction: "dispatched_at_ms", + exit: "at_ms", + filesystem: "at_ms", + process: "at_ms", +}; + +const eventTime = ( + dimension: DivergenceDimension, + value: unknown, +): number | null => { + if (typeof value !== "object" || value === null) return null; + const time: unknown = Reflect.get(value, EVENT_TIME_FIELD[dimension]); + return typeof time === "number" && Number.isSafeInteger(time) && time >= 0 + ? time + : null; }; const firstCollectionDivergence = ( @@ -195,8 +203,8 @@ const firstCollectionDivergence = ( status: "found", dimension, index, - left_at_ms: eventTime(leftValue), - right_at_ms: eventTime(rightValue), + left_at_ms: eventTime(dimension, leftValue), + right_at_ms: eventTime(dimension, rightValue), left: jsonValueSchema.parse(leftValue), right: jsonValueSchema.parse(rightValue), }; diff --git a/src/domain/reconstructionCoverage.fixture.ts b/src/domain/reconstructionCoverage.fixture.ts index 0a51546b6..2d1e37a19 100644 --- a/src/domain/reconstructionCoverage.fixture.ts +++ b/src/domain/reconstructionCoverage.fixture.ts @@ -143,3 +143,13 @@ export const completeReconstructionCoverageData = () => { ], }); }; + +/** Complete workspace with its first verifier result and boundary narrowed once. */ +export const completeReconstructionCoverageFixture = () => { + const workspace = completeReconstructionCoverageData(); + const result = workspace.verifier_results[0]; + const boundary = workspace.boundaries[0]; + if (result === undefined || boundary === undefined) + throw new Error("Expected complete reconstruction coverage fixture"); + return { workspace, result, boundary }; +}; diff --git a/src/domain/reconstructionCoverage.test.ts b/src/domain/reconstructionCoverage.test.ts index ac7552571..628ad67c1 100644 --- a/src/domain/reconstructionCoverage.test.ts +++ b/src/domain/reconstructionCoverage.test.ts @@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest"; import { completeReconstructionCoverageData, + completeReconstructionCoverageFixture, RECONSTRUCTION_COVERAGE_NOW, reconstructionCoverageEvidenceId, } from "./reconstructionCoverage.fixture.js"; @@ -288,11 +289,8 @@ describe("reconstruction coverage verifier selection", () => { describe("reconstruction coverage conflicting latest verifier observations", () => { it("keeps conflicting tied assessments unknown regardless of input order", () => { - const workspace = completeReconstructionCoverageData(); - const result = workspace.verifier_results[0]; - const boundary = workspace.boundaries[0]; - if (result === undefined || boundary === undefined) - throw new Error("Expected complete reconstruction coverage fixture"); + const { workspace, result, boundary } = + completeReconstructionCoverageFixture(); const failed = { ...result, status: "fail" as const, @@ -338,11 +336,8 @@ describe("reconstruction coverage conflicting latest verifier observations", () describe("reconstruction coverage equivalent latest verifier observations", () => { it("coalesces equivalent equal-instant observations and retains all evidence", () => { - const workspace = completeReconstructionCoverageData(); - const result = workspace.verifier_results[0]; - const boundary = workspace.boundaries[0]; - if (result === undefined || boundary === undefined) - throw new Error("Expected complete reconstruction coverage fixture"); + const { workspace, result, boundary } = + completeReconstructionCoverageFixture(); const first = { ...result, observed_at: "2026-07-16T11:00:00.123456789012345678900Z", @@ -388,11 +383,8 @@ describe("reconstruction coverage equivalent latest verifier observations", () = describe("reconstruction coverage verifier timestamp selection", () => { it("lets a later unambiguous observation supersede an older tie", () => { - const workspace = completeReconstructionCoverageData(); - const result = workspace.verifier_results[0]; - const boundary = workspace.boundaries[0]; - if (result === undefined || boundary === undefined) - throw new Error("Expected complete reconstruction coverage fixture"); + const { workspace, result, boundary } = + completeReconstructionCoverageFixture(); const failed = { ...result, status: "fail" as const }; const latestPass = { ...result, @@ -412,50 +404,14 @@ describe("reconstruction coverage verifier timestamp selection", () => { ).toMatchObject({ status: "ready", reasons: [] }); }); - it("orders offset-bearing verifier timestamps chronologically", () => { - const workspace = completeReconstructionCoverageData(); - const result = workspace.verifier_results[0]; - if (result === undefined) - throw new Error("Expected fixture verifier result"); - const semantic = workspace; - const changed = createReconstructionCoverageData({ - ...semantic, - verifier_results: [ - { ...result, observed_at: "2026-07-16T08:00:00.000Z" }, - { - ...result, - observed_at: "2026-07-16T04:00:00.000-05:00", - status: "fail", - }, - ], - }); - - expect( - evaluateReconstructionClosure( - changed, - "replacement.cli", - RECONSTRUCTION_COVERAGE_NOW, - ), - ).toMatchObject({ - status: "failed", - reasons: [expect.objectContaining({ code: "verifier-failed" })], - }); - }); - - it("selects the later result when timestamps differ below millisecond precision", () => { - const workspace = completeReconstructionCoverageData(); - const result = workspace.verifier_results[0]; - if (result === undefined) - throw new Error("Expected fixture verifier result"); + /** Evaluate a workspace whose newest verifier observation fails the closure. */ + const expectNewestObservationFails = (passedAt: string, failedAt: string) => { + const { workspace, result } = completeReconstructionCoverageFixture(); const changed = createReconstructionCoverageData({ ...workspace, verifier_results: [ - { ...result, observed_at: "2026-07-16T11:00:00.1231Z" }, - { - ...result, - observed_at: "2026-07-16T11:00:00.1239Z", - status: "fail", - }, + { ...result, observed_at: passedAt }, + { ...result, observed_at: failedAt, status: "fail" }, ], }); @@ -469,15 +425,26 @@ describe("reconstruction coverage verifier timestamp selection", () => { status: "failed", reasons: [expect.objectContaining({ code: "verifier-failed" })], }); + }; + + it("orders offset-bearing verifier timestamps chronologically", () => { + expectNewestObservationFails( + "2026-07-16T08:00:00.000Z", + "2026-07-16T04:00:00.000-05:00", + ); + }); + + it("selects the later result when timestamps differ below millisecond precision", () => { + expectNewestObservationFails( + "2026-07-16T11:00:00.1231Z", + "2026-07-16T11:00:00.1239Z", + ); }); }); describe("reconstruction coverage verifier future timestamps", () => { it("treats an observation beyond millisecond-precision now as future", () => { - const workspace = completeReconstructionCoverageData(); - const result = workspace.verifier_results[0]; - if (result === undefined) - throw new Error("Expected fixture verifier result"); + const { workspace, result } = completeReconstructionCoverageFixture(); const changed = createReconstructionCoverageData({ ...workspace, verifier_results: [ @@ -505,10 +472,7 @@ describe("reconstruction coverage verifier future timestamps", () => { }); it("rejects verifier observations from the future", () => { - const workspace = completeReconstructionCoverageData(); - const result = workspace.verifier_results[0]; - if (result === undefined) - throw new Error("Expected fixture verifier result"); + const { workspace, result } = completeReconstructionCoverageFixture(); const semantic = workspace; const changed = createReconstructionCoverageData({ ...semantic, diff --git a/src/domain/reconstructionVerification.functionSources.test.ts b/src/domain/reconstructionVerification.functionSources.test.ts index 5dc8db589..e23d37d74 100644 --- a/src/domain/reconstructionVerification.functionSources.test.ts +++ b/src/domain/reconstructionVerification.functionSources.test.ts @@ -53,6 +53,7 @@ const source = ( digit: string, observation: Partial = {}, hasSubject = true, + hasCoverage = true, ): Evidence => createEvidence( hasSubject @@ -69,7 +70,9 @@ const source = ( result: dossier, confidence: "derived", authority: "shipped-artifact", - relationshipCoverage: { status: "complete" }, + ...(hasCoverage + ? { relationshipCoverage: { status: "complete" as const } } + : {}), ...observation, }, ); @@ -197,6 +200,118 @@ describe("function reconstruction source parameters", () => { }); }); +describe("historical calls-only reconstruction changes", () => { + it.each(["missing", "unknown", "partial", "unresolved"] as const)( + "keeps historical calls-only changes unknown with %s call coverage", + (coverage) => { + const changedDossier = functionDossierSchema.parse({ + ...functionDossierSchema.parse(dossier), + callees: [ + { + address: "0x2000", + name: "puts", + classification: null, + body: { available: false, reason: "External function." }, + }, + ], + }); + const complete = source("8", { + result: jsonValueSchema.parse(changedDossier), + }); + const incomplete = source( + "9", + { + result: jsonValueSchema.parse({ + ...changedDossier, + ...(coverage === "unresolved" + ? { + unresolved_calls: [ + { + address: "0x1004", + reason: "Computed target unresolved.", + }, + ], + } + : {}), + }), + ...(coverage === "missing" + ? {} + : { + relationshipCoverage: + coverage === "unknown" + ? { status: "unknown", reason: "Call graph unavailable." } + : coverage === "partial" + ? { + status: "partial", + limitations: ["Additional edges omitted."], + } + : { status: "complete" }, + }), + }, + true, + coverage !== "missing", + ); + const historicalResult = compareFunctions(left, complete); + expect(historicalResult).toMatchObject({ + status: "changed", + summary: { changed: 1 }, + }); + expect(historicalResult.changes).toMatchObject([ + { dimension: "calls", status: "changed" }, + ]); + const completeComparison = comparison( + { + left_evidence_id: left.evidence_id, + right_evidence_id: complete.evidence_id, + }, + {}, + [left, complete], + {}, + historicalResult, + ); + expect( + admittedVerification(completeComparison, [left, complete], "overall")() + .status, + ).toBe("fail"); + const evidence = comparison( + { + left_evidence_id: left.evidence_id, + right_evidence_id: incomplete.evidence_id, + }, + {}, + [left, incomplete], + {}, + historicalResult, + ); + const before = JSON.stringify(evidence); + for (const dimension of ["calls", "overall"] as const) { + expect( + admittedVerification(evidence, [left, incomplete], dimension)(), + ).toMatchObject({ + status: "unknown", + summary: { total: 1, passed: 0, failed: 0, unknown: 1 }, + claims: { + items: [ + expect.objectContaining({ + status: "unknown", + observed_status: "changed", + limitations: expect.arrayContaining([ + expect.stringContaining(`Source ${incomplete.evidence_id}:`), + ]), + }), + ], + }, + }); + } + expect( + admittedVerification(evidence, [left, incomplete], "pseudocode")() + .status, + ).toBe("pass"); + expect(JSON.stringify(evidence)).toBe(before); + }, + ); +}); + describe("function reconstruction call coverage", () => { it("rechecks old unchanged comparisons without affecting unrelated finite dimensions", () => { const incomplete = source("7", { diff --git a/src/domain/reconstructionVerification.ts b/src/domain/reconstructionVerification.ts index dd300a0a2..6f693a158 100644 --- a/src/domain/reconstructionVerification.ts +++ b/src/domain/reconstructionVerification.ts @@ -144,10 +144,15 @@ const evaluateClaim = ( return source; }); const functionSourceFacts = readValidatedSourceFacts(claim, sources); - const observed = observedStatus(claim, comparison); + const observation = comparisonObservation(claim, comparison); + const observed = observation.status; const limitations = [ ...authorityLimitations(claim, sources, sides), - ...sourceCallCoverageLimitations(claim, functionSourceFacts, observed), + ...sourceCallCoverageLimitations( + claim, + functionSourceFacts, + observation.hasIndependentFunctionChange === true, + ), ]; const relatedUnknowns = reconstructionClaimUnknowns( heads, @@ -235,21 +240,40 @@ const validateComparisonIdentity = (claim: Claim, evidence: Evidence): void => { throw new TypeError("Comparison Evidence identity or authority disagrees"); }; -const observedStatus = (claim: Claim, evidence: Evidence): ObservedStatus => { +const comparisonObservation = ( + claim: Claim, + evidence: Evidence, +): { + readonly status: ObservedStatus; + readonly hasIndependentFunctionChange?: boolean; +} => { if (claim.kind === "behavioral") { const result = processCaptureComparisonSchema.parse( evidence.normalized_result, ); validateProcessStatus(result); - return claim.dimension === "overall" - ? result.status - : result[claim.dimension]; + return { + status: + claim.dimension === "overall" ? result.status : result[claim.dimension], + }; } if (claim.kind === "structural-function") { const result = functionComparisonResultSchema.parse( evidence.normalized_result, ); - if (claim.dimension === "overall") return result.status; + if (claim.dimension === "overall") + return { + status: result.status, + // Historical overall changes may come solely from incomplete call sets. + hasIndependentFunctionChange: + result.function_match.status === "mismatched" || + (result.function_match.status === "matched" && + result.dimensions.some( + (dimension) => + dimension.dimension !== "calls" && + dimension.status === "changed", + )), + }; const dimension = result.dimensions.find( (item) => item.dimension === claim.dimension, ); @@ -263,7 +287,7 @@ const observedStatus = (claim: Claim, evidence: Evidence): ObservedStatus => { throw new TypeError( "Function dimension cites Evidence outside comparison closure", ); - return dimension.status; + return { status: dimension.status }; } const result = artifactComparisonResultSchema.parse( evidence.normalized_result, @@ -276,7 +300,7 @@ const observedStatus = (claim: Claim, evidence: Evidence): ObservedStatus => { throw new TypeError( "Artifact change cites Evidence outside comparison closure", ); - return result.status; + return { status: result.status }; }; const validateProcessStatus = ( @@ -340,12 +364,12 @@ const authorityLimitations = ( const sourceCallCoverageLimitations = ( claim: Claim, functionSources: readonly FunctionSourceFacts[], - observed: ObservedStatus, + hasIndependentFunctionChange: boolean, ): string[] => { if ( claim.kind !== "structural-function" || (claim.dimension !== "calls" && - !(claim.dimension === "overall" && observed !== "changed")) + !(claim.dimension === "overall" && !hasIndependentFunctionChange)) ) return []; return functionSources.flatMap(({ evidence_id, callCoverageLimitations }) => diff --git a/src/domain/referenceSourceSummary.test.ts b/src/domain/referenceSourceSummary.test.ts new file mode 100644 index 000000000..a0d23b0e1 --- /dev/null +++ b/src/domain/referenceSourceSummary.test.ts @@ -0,0 +1,104 @@ +import { describe, expect, it } from "vitest"; + +import { + createHistoricalSourceGraph, + createHistoricalSourceManifest, + type HistoricalSourceGraphInput, +} from "./referenceSourceGraph.js"; +import { createHistoricalSourceSummary } from "./referenceSourceSummary.js"; + +const input = ( + entries: HistoricalSourceGraphInput["entries"], +): HistoricalSourceGraphInput => ({ + schema: "HistoricalSourceGraph", + authority: "historical-reference", + root_alias: "$REFERENCE_ROOT", + inventory_state: "unknown", + entries, + relationships: [], + parse_failures: [], + exclusions: [], + languages: [], + manifests: [], + vcs: null, + provenance: { importer: "fixture", importer_version: null, caller: "test" }, + limitations: ["Inventory coverage is unknown"], +}); + +describe("committed historical-source summaries", () => { + it("preserves unknown coverage, sizes, directories and missing symlinks without mutating the graph", () => { + const graph = createHistoricalSourceGraph( + input([ + { + path: "a", + kind: "directory", + tree_state: "unknown", + classifications: ["unknown"], + limitations: ["Not enumerated"], + }, + { + path: "b", + kind: "file", + sha256: null, + size: null, + content_state: "unreadable", + language: null, + classifications: ["unknown"], + limitations: ["Not readable"], + }, + { + path: "c", + kind: "symlink", + target: "missing", + target_state: "missing", + classifications: ["unknown"], + limitations: [], + }, + ]), + ); + const original = structuredClone(graph); + const summary = createHistoricalSourceSummary(graph, 1); + expect(summary).toMatchObject({ + manifest: createHistoricalSourceManifest(original), + inventory_state: "unknown", + vcs: null, + provenance: original.provenance, + file_sizes: { known_bytes: "0", unknown_file_count: 1 }, + counts: { entries: 3, files: 1, directories: 1, symlinks: 1 }, + directory_tree_states: { unknown: 1, enumerated: 0 }, + file_content_states: { unreadable: 1, hashed: 0 }, + limitations: original.limitations, + outline: { + entries: [ + original.entries[0], + expect.objectContaining({ size: null, content_state: "unreadable" }), + original.entries[2], + ], + omitted_entry_count: 0, + }, + }); + expect(summary.outline.entries[1]).not.toHaveProperty("sha256"); + expect(graph).toEqual(original); + }); + + it("sums known bytes losslessly beyond the JSON safe integer range", () => { + const graph = createHistoricalSourceGraph( + input( + ["a", "b"].map((path) => ({ + path, + kind: "file", + sha256: "a".repeat(64), + size: Number.MAX_SAFE_INTEGER, + content_state: "hashed", + language: null, + classifications: ["unknown"], + limitations: [], + })), + ), + ); + expect(createHistoricalSourceSummary(graph, 0)).toMatchObject({ + file_sizes: { known_bytes: "18014398509481982", unknown_file_count: 0 }, + outline: { entries: [], omitted_entry_count: 2 }, + }); + }); +}); diff --git a/src/domain/referenceSourceSummary.ts b/src/domain/referenceSourceSummary.ts new file mode 100644 index 000000000..e7e99e3fa --- /dev/null +++ b/src/domain/referenceSourceSummary.ts @@ -0,0 +1,93 @@ +import { + createHistoricalSourceManifest, + type HistoricalSourceGraph, +} from "./referenceSourceGraph.js"; + +type Entry = HistoricalSourceGraph["entries"][number]; + +const outlineEntry = (entry: Entry) => { + if (entry.kind !== "file") return entry; + return { + path: entry.path, + kind: entry.kind, + language: entry.language, + size: entry.size, + content_state: entry.content_state, + classifications: entry.classifications, + limitations: entry.limitations, + }; +}; + +/** Project one committed inventory without rereading or reinterpreting source. */ +export const createHistoricalSourceSummary = ( + graph: HistoricalSourceGraph, + outlineDepth: number, +) => { + const manifest = createHistoricalSourceManifest(graph); + const counts = { + entries: graph.entries.length, + files: 0, + directories: 0, + symlinks: 0, + relationships: graph.relationships.length, + parse_failures: graph.parse_failures.length, + exclusions: graph.exclusions.length, + }; + const contentStates = { + hashed: 0, + "redacted-secret": 0, + excluded: 0, + unreadable: 0, + unknown: 0, + }; + const treeStates = { + enumerated: 0, + partial: 0, + excluded: 0, + unreadable: 0, + unknown: 0, + }; + let knownBytes = 0n; + let unknownSizeFiles = 0; + const outline: ReturnType[] = []; + for (const entry of graph.entries) { + if (entry.kind === "file") { + counts.files += 1; + contentStates[entry.content_state] += 1; + if (entry.size === null) unknownSizeFiles += 1; + else knownBytes += BigInt(entry.size); + } else if (entry.kind === "directory") { + counts.directories += 1; + treeStates[entry.tree_state] += 1; + } else counts.symlinks += 1; + // Canonical entries already use Unicode code-point order; filtering preserves it. + if (entry.path.split("/").length <= outlineDepth) + outline.push(outlineEntry(entry)); + } + return { + schema: "HistoricalSourceSummary" as const, + authority: graph.authority, + root_alias: graph.root_alias, + manifest, + inventory_state: graph.inventory_state, + provenance: graph.provenance, + vcs: graph.vcs, + languages: graph.languages, + manifests: graph.manifests, + counts, + file_content_states: contentStates, + directory_tree_states: treeStates, + file_sizes: { + // Decimal text keeps large totals exact; this covers only known inventoried sizes. + known_bytes: knownBytes.toString(), + unknown_file_count: unknownSizeFiles, + }, + exclusions: graph.exclusions, + limitations: graph.limitations, + outline: { + depth: outlineDepth, + entries: outline, + omitted_entry_count: graph.entries.length - outline.length, + }, + }; +}; diff --git a/src/domain/sourceMapContents.test.ts b/src/domain/sourceMapContents.test.ts index 06f07197b..dd80f39fb 100644 --- a/src/domain/sourceMapContents.test.ts +++ b/src/domain/sourceMapContents.test.ts @@ -2,10 +2,18 @@ import { expect, it } from "vitest"; import { hasValidSourceMapContents } from "./sourceMapContents.js"; -it("accepts absent map-level source contents and aligned nullable slots", () => { - expect(hasValidSourceMapContents(2, undefined)).toBe(true); - expect(hasValidSourceMapContents(2, null)).toBe(true); - expect(hasValidSourceMapContents(2, [null, "retained source"])).toBe(true); - expect(hasValidSourceMapContents(2, ["retained source"])).toBe(false); - expect(hasValidSourceMapContents(2, [null, 42])).toBe(false); +it("accepts absent map-level source contents and nullable string slots", () => { + expect(hasValidSourceMapContents(undefined)).toBe(true); + expect(hasValidSourceMapContents(null)).toBe(true); + expect(hasValidSourceMapContents([null, "retained source"])).toBe(true); + expect(hasValidSourceMapContents([null, 42])).toBe(false); + expect(hasValidSourceMapContents("retained source")).toBe(false); +}); + +it("leaves sourcesContent length to source decoding (ECMA-426 DecodeSourceMapSources)", () => { + expect(hasValidSourceMapContents([])).toBe(true); + expect(hasValidSourceMapContents(["retained source"])).toBe(true); + expect( + hasValidSourceMapContents(["a", null, "unused trailing content"]), + ).toBe(true); }); diff --git a/src/domain/sourceMapContents.ts b/src/domain/sourceMapContents.ts index 5d55810a9..c6a621f1a 100644 --- a/src/domain/sourceMapContents.ts +++ b/src/domain/sourceMapContents.ts @@ -1,12 +1,15 @@ -/** Validate optional source-map content against its source inventory. */ -export const hasValidSourceMapContents = ( - sourceCount: number, - contents: unknown, -): boolean => +/** + * Validate optional source-map content against its source inventory. + * + * ECMA-426 DecodeSourceMapSources reads a content slot only while its index is + * within `sourcesContent`, so a shorter array leaves trailing sources without + * content and a longer array contributes no extra sources. Only the entry types + * are checked here. + */ +export const hasValidSourceMapContents = (contents: unknown): boolean => contents === undefined || contents === null || (Array.isArray(contents) && - contents.length === sourceCount && contents.every( (content: unknown) => content === null || typeof content === "string", )); diff --git a/src/domain/wasm/wasmArtifact.ts b/src/domain/wasm/wasmArtifact.ts index d8c89ddf3..a15735125 100644 --- a/src/domain/wasm/wasmArtifact.ts +++ b/src/domain/wasm/wasmArtifact.ts @@ -1,3 +1,4 @@ +import { wasmInterfaceSchema } from "./wasmInterface.js"; import { z } from "zod"; import { createHash } from "node:crypto"; import type { EvidenceSubjectTarget } from "../evidence.js"; @@ -32,6 +33,7 @@ export const wasmArtifactSchema = z description: z.string(), }), ), + interface: wasmInterfaceSchema.exactOptional(), imports: z.array(z.string()), exports: z.array(z.string()), headers: z.string(), diff --git a/src/domain/wasm/wasmInterface.ts b/src/domain/wasm/wasmInterface.ts new file mode 100644 index 000000000..f5559269c --- /dev/null +++ b/src/domain/wasm/wasmInterface.ts @@ -0,0 +1,109 @@ +import { z } from "zod"; + +const kind = z.enum(["func", "table", "memory", "global", "tag", "unknown"]); +const valueType = z.enum([ + "i32", + "i64", + "f32", + "f64", + "v128", + "funcref", + "externref", +]); +const limit = z.string().regex(/^(?:0|[1-9][0-9]*)$/u); +const signature = z.discriminatedUnion("kind", [ + z.object({ + kind: z.literal("func"), + params: z.array(valueType), + results: z.array(valueType), + }), + z.object({ + kind: z.literal("tag"), + params: z.array(valueType), + results: z.array(valueType), + }), + z.object({ + kind: z.literal("table"), + element: z.enum(["funcref", "externref"]), + address_type: z.enum(["i32", "i64"]), + minimum: limit, + maximum: limit.nullable(), + shared: z.boolean(), + }), + z.object({ + kind: z.literal("memory"), + page_size: limit, + address_type: z.enum(["i32", "i64"]), + minimum: limit, + maximum: limit.nullable(), + shared: z.boolean(), + }), + z.object({ + kind: z.literal("global"), + value_type: valueType, + mutable: z.boolean(), + }), +]); + +/** Typed observations decoded once at the WABT producer boundary. */ +export const wasmInterfaceSchema = z.object({ + state: z.enum(["complete", "partial"]), + entries: z.array( + z + .object({ + direction: z.enum(["import", "export"]), + module: z.string().nullable(), + name: z.string(), + kind, + index: z.number().int().nonnegative().max(0xffff_ffff).nullable(), + signature: signature.nullable(), + wat_form: z.string(), + issue: z.string().nullable(), + }) + .superRefine((entry, context) => { + if ( + (entry.direction === "export" && entry.module !== null) || + (entry.direction === "import" && entry.module === null) + ) + context.addIssue({ + code: "custom", + message: "Interface direction does not match module-name presence", + }); + if (entry.signature !== null && entry.signature.kind !== entry.kind) + context.addIssue({ + code: "custom", + message: "Interface kind does not match its typed signature", + }); + }), + ), + issues: z.array(z.string()), +}); +export type WasmInterface = z.infer; +export type WasmInterfaceEntry = WasmInterface["entries"][number]; +export type WasmInterfaceSignature = NonNullable< + WasmInterfaceEntry["signature"] +>; + +export type WasmInterfaceValueType = z.infer; + +/** Bound eager typed output and repeated signature expansion independently of WAT bytes. */ +export const WASM_INTERFACE_REPRESENTATION_BYTES = 32 * 1024 * 1024; + +/** Conservative JSON/retention charge; shared arrays still expand at every occurrence. */ +export const wasmInterfaceEntryRepresentationBytes = ( + entry: WasmInterfaceEntry, +): number => { + let bytes = + 1024 + + [entry.module, entry.name, entry.wat_form, entry.issue].reduce( + (sum, value) => sum + (value?.length ?? 0) * 6, + 0, + ); + const signature = entry.signature; + if (signature?.kind === "func" || signature?.kind === "tag") + bytes += (signature.params.length + signature.results.length) * 16; + else if (signature !== null) + for (const value of Object.values(signature)) + if (typeof value === "string") bytes += value.length * 6; + return bytes; +}; diff --git a/src/domain/webBundleAnalyzer.test.ts b/src/domain/webBundleAnalyzer.test.ts index 5913b0f40..edcb37a20 100644 --- a/src/domain/webBundleAnalyzer.test.ts +++ b/src/domain/webBundleAnalyzer.test.ts @@ -8,6 +8,48 @@ import { createWebTextArtifact } from "./webContentArtifact.js"; const origin = "https://app.example.test"; +it.each([ + { + kind: "recovered", + source: "function legacy(a, a) { return a; } fetch('/partial');", + parsed: 2, + failures: 0, + reason: "recovered", + endpoints: ["/partial", "/complete"], + }, + { + kind: "failed", + source: "function broken( {", + parsed: 1, + failures: 1, + reason: "could not be parsed", + endpoints: ["/complete"], + }, +])( + "attributes $kind parse uncertainty only to its affected source", + ({ source, parsed, failures, reason, endpoints }) => { + const captured = inspection(source); + const good = inspection("fetch('/complete');").scripts.items[0]; + if (good === undefined) throw new Error("Missing valid script fixture"); + captured.scripts.items.push({ ...good, script_key: "valid-script" }); + captured.scripts.total += 1; + const result = analyzeCapturedWebBundle(captured); + expect(result.completeness).toMatchObject({ + status: "partial", + parsed_scripts: parsed, + parse_failures: failures, + }); + expect(result.unknowns).toContainEqual({ + dimension: "javascript_ast", + reason: expect.stringContaining(reason), + affected_script_keys: [captured.scripts.items[0]?.script_key], + }); + expect(result.observations.endpoints.map(({ value }) => value)).toEqual( + endpoints, + ); + }, +); + describe("web bundle analyzer", () => { it("extracts graph, route, endpoint, vendor, and WebMCP evidence", () => { const source = ` @@ -320,6 +362,63 @@ describe("web bundle API classification parity", () => { "users", ]); }); +}); + +describe("web bundle configuration and navigation parity", () => { + it("keeps configuration setting reads out of endpoints in both analyzers", () => { + const source = ` + import vscodeDefault from "vscode"; + vscodeDefault.workspace.getConfiguration("git").get("default-setting"); + vscode.workspace.getConfiguration("git").get("autofetch"); + workspace.getConfiguration("git").get("enabled", true); + const config = vscode.workspace.getConfiguration("git"); + config.get("autofetchPeriod", 180); + config.get("/v1/settings"); + let c = Mt.workspace.getConfiguration("git", uri), l = c.get("timeline.date"); + let changed = vscode.workspace.getConfiguration("git"); + changed = api; + changed.get("reassigned"); + const maybe = flag ? vscode.workspace.getConfiguration("git") : api; + maybe.get("conditional"); + { + const config = api; + config.get("shadowed"); + } + api.get("users"); + const client = getConfiguration("/api/v1"); + client.get("accounts"); + http.getConfiguration(baseUrl).get("orders"); + { + const workspace = { getConfiguration: () => api }; + workspace.getConfiguration("/api").get("local-users"); + } + { + const vscode = { workspace: { getConfiguration: () => api } }; + vscode.workspace.getConfiguration("/api").get("local-orders"); + } + `; + const expected = [ + "/v1/settings", + "accounts", + "conditional", + "local-orders", + "local-users", + "orders", + "reassigned", + "shadowed", + "users", + ]; + expect( + analyzeCapturedWebBundle(inspection(source)) + .observations.endpoints.map(({ value }) => value) + .sort(), + ).toEqual(expected); + expect( + analyzeJavaScriptStaticSource(source) + .endpoints.map(({ value }) => value) + .sort(), + ).toEqual(expected); + }); it("recognizes loadURL endpoints like static analysis", () => { const result = analyzeCapturedWebBundle( @@ -657,3 +756,54 @@ it("keeps RPC lexical identity and bundled factory recognition in captured scrip ]); expect(result.completeness.status).toBe("complete"); }); + +it("retains distinct WebMCP descriptions and names while deduplicating equivalent properties", () => { + const result = analyzeCapturedWebBundle( + inspection(` + document.modelContext.registerTool({ name: "lookup", description: "first", inputSchema: { properties: { alpha: {} } } }); + document.modelContext.registerTool({ name: "lookup", description: "second", inputSchema: { properties: { alpha: {} } } }); + document.modelContext.registerTool({ name: "equivalent", inputSchema: { properties: { alpha: {}, beta: {} } } }); + document.modelContext.registerTool({ name: "equivalent", inputSchema: { properties: { beta: {}, alpha: {}, alpha: {} } } }); + document.modelContext.registerTool({ description: "unnamed" }); + document.modelContext.registerTool({ name: "", description: "unnamed" }); + `), + ); + + expect( + result.observations.webmcp_declarations.map( + ({ name, description, schema_property_names }) => ({ + name, + description, + schema_property_names, + }), + ), + ).toEqual([ + { + name: "lookup", + description: "first", + schema_property_names: ["alpha"], + }, + { + name: "lookup", + description: "second", + schema_property_names: ["alpha"], + }, + { + name: "equivalent", + description: null, + schema_property_names: ["alpha", "beta"], + }, + { name: null, description: "unnamed", schema_property_names: [] }, + { name: "", description: "unnamed", schema_property_names: [] }, + ]); + expect(result.observations.webmcp_declarations[2]?.location).toEqual({ + script_key: `scr_${"1".repeat(64)}`, + line: 4, + column: 6, + }); + expect( + result.observations.webmcp_declarations.every( + ({ trust }) => trust === "page-declared-untrusted", + ), + ).toBe(true); +}); diff --git a/src/domain/webBundleAnalyzerInspection.ts b/src/domain/webBundleAnalyzerInspection.ts index fd8c541ac..12c35109c 100644 --- a/src/domain/webBundleAnalyzerInspection.ts +++ b/src/domain/webBundleAnalyzerInspection.ts @@ -50,7 +50,8 @@ export interface AnalysisAccumulator { readonly seen: Set; visitedNodes: number; parsedScripts: number; - parseFailures: number; + readonly failedScripts: string[]; + readonly recoveredScripts: string[]; } export const analyzeScript = ( @@ -67,10 +68,12 @@ export const analyzeScript = ( } catch (cause: unknown) { // Unparseable scripts are counted; the failure needs no extra detail. void cause; - accumulator.parseFailures += 1; + accumulator.failedScripts.push(script.script_key); return; } accumulator.parsedScripts += 1; + if ((file.errors?.length ?? 0) > 0) + accumulator.recoveredScripts.push(script.script_key); detectVendorFingerprints(script, accumulator); const openReceiverFacts = classifyParsedJavaScriptOpenReceivers(file); const rpc = createJavaScriptRpcScan( @@ -293,19 +296,18 @@ const addWebMcpDeclaration = ( : [], ) : []; + const metadata = { + name: name ?? null, + description: description ?? null, + schema_property_names: [...new Set(schemaPropertyNames)].sort(), + }; + // Deduplicate the metadata we return, preserving omission and descriptions. // Property names are a variable-length source-derived list: NUL-joins // would collapse ["a\0b"] with ["a", "b"] and drop a declaration. - const key = compositeKey([ - "webmcp", - script.script_key, - name ?? "", - schemaPropertyNames, - ]); + const key = compositeKey(["webmcp", script.script_key, metadata]); addUnique(accumulator, key, () => accumulator.webMcp.push({ - name: name ?? null, - description: description ?? null, - schema_property_names: [...new Set(schemaPropertyNames)].sort(), + ...metadata, trust: "page-declared-untrusted", location: location(script.script_key, node), }), @@ -387,7 +389,8 @@ export const emptyAccumulator = (): AnalysisAccumulator => ({ seen: new Set(), visitedNodes: 0, parsedScripts: 0, - parseFailures: 0, + failedScripts: [], + recoveredScripts: [], }); export const isIncludedScript = ( diff --git a/src/domain/webBundleAnalyzerResult.ts b/src/domain/webBundleAnalyzerResult.ts index 1e08f0f11..4d2509675 100644 --- a/src/domain/webBundleAnalyzerResult.ts +++ b/src/domain/webBundleAnalyzerResult.ts @@ -25,7 +25,8 @@ export const buildWebBundleAnalysis = ( const sourceMapIncomplete = sourceMaps.status === "partial" || sourceMaps.status === "unavailable"; const partial = - accumulator.parseFailures > 0 || + accumulator.failedScripts.length > 0 || + accumulator.recoveredScripts.length > 0 || unavailable.length > 0 || sourceMapIncomplete || omittedScripts !== undefined || @@ -44,9 +45,9 @@ export const buildWebBundleAnalysis = ( }, inferences: accumulator.inferences, unknowns: buildUnknowns({ - sourceScripts, unavailable, - parseFailures: accumulator.parseFailures, + failedScripts: accumulator.failedScripts, + recoveredScripts: accumulator.recoveredScripts, sourceMaps, omittedScripts, omittedSourceMaps, @@ -54,7 +55,7 @@ export const buildWebBundleAnalysis = ( completeness: buildCompleteness({ partial, parsedScripts: accumulator.parsedScripts, - parseFailures: accumulator.parseFailures, + parseFailures: accumulator.failedScripts.length, visitedNodes: accumulator.visitedNodes, }), limitations: bundleLimitations(), @@ -88,9 +89,9 @@ const chunkNode = (script: IncludedScript) => { }; interface UnknownsInput { - readonly sourceScripts: IncludedScript[]; readonly unavailable: string[]; - readonly parseFailures: number; + readonly failedScripts: readonly string[]; + readonly recoveredScripts: readonly string[]; readonly sourceMaps: WebBundleAnalysis["observations"]["source_maps"]; readonly omittedScripts: number | null | undefined; readonly omittedSourceMaps: number | null | undefined; @@ -115,15 +116,23 @@ const buildUnknowns = (input: UnknownsInput): WebBundleAnalysis["unknowns"] => [ affected_script_keys: input.unavailable, }, ]), - ...(input.parseFailures === 0 + ...(input.failedScripts.length === 0 ? [] : [ { dimension: "javascript_ast" as const, reason: "One or more source artifacts could not be parsed", - affected_script_keys: input.sourceScripts.map( - ({ script_key }) => script_key, - ), + affected_script_keys: [...input.failedScripts], + }, + ]), + ...(input.recoveredScripts.length === 0 + ? [] + : [ + { + dimension: "javascript_ast" as const, + reason: + "The parser recovered from errors in one or more source artifacts; findings may be incomplete", + affected_script_keys: [...input.recoveredScripts], }, ]), ...(input.omittedSourceMaps === undefined diff --git a/src/domain/webMcpDiscovery.ts b/src/domain/webMcpDiscovery.ts index f1a3f3b12..0924f5b78 100644 --- a/src/domain/webMcpDiscovery.ts +++ b/src/domain/webMcpDiscovery.ts @@ -3,7 +3,7 @@ import { z } from "zod"; import { browserCompletenessSchema } from "./browserCompleteness.js"; import { browserAllowedOriginsSchema, - browserEndpointSchema, + browserObservationEndpointSchema, } from "./browserObservation.js"; import { browserVersionSchema } from "./browserObservationSchemas.js"; import { jsonValueSchema } from "./jsonValue.js"; @@ -11,7 +11,7 @@ import { digestSchema, prefixedDigestSchema } from "./../domain/digests.js"; /** Input for passive discovery of page-registered WebMCP tools. */ export const discoverWebMcpToolsInputSchema = z.strictObject({ - cdp_endpoint: browserEndpointSchema, + cdp_endpoint: browserObservationEndpointSchema, allowed_origins: browserAllowedOriginsSchema, target_id: z.string().trim().min(1), observation_ms: z.number().int().min(0).default(100), diff --git a/src/domain/webScreenshot.ts b/src/domain/webScreenshot.ts index 57ee35161..d477bdaeb 100644 --- a/src/domain/webScreenshot.ts +++ b/src/domain/webScreenshot.ts @@ -5,7 +5,7 @@ import { z } from "zod"; import { browserCompletenessSchema } from "./browserCompleteness.js"; import { browserAllowedOriginsSchema, - browserEndpointSchema, + browserObservationEndpointSchema, } from "./browserObservation.js"; import { browserVersionSchema } from "./browserObservationSchemas.js"; @@ -34,7 +34,7 @@ export type WebScreenshotArtifact = z.infer; /** Input for one read-only visible-viewport screenshot. */ export const captureWebScreenshotInputSchema = z.strictObject({ - cdp_endpoint: browserEndpointSchema, + cdp_endpoint: browserObservationEndpointSchema, allowed_origins: browserAllowedOriginsSchema, target_id: z.string().trim().min(1), }); diff --git a/src/evaluation/AgentEvaluationFixtures.ts b/src/evaluation/AgentEvaluationFixtures.ts index b7ef17efe..b0535a96e 100644 --- a/src/evaluation/AgentEvaluationFixtures.ts +++ b/src/evaluation/AgentEvaluationFixtures.ts @@ -8,7 +8,19 @@ import type { JsonValue } from "../domain/jsonValue.js"; import { javascriptApplicationAnalysisResultSchema } from "../domain/javascript/javascriptApplicationAnalysis.js"; import type { ApplicationNode } from "../domain/javascript/javascriptApplicationGraphSchemas.js"; import { javaScriptExportShapeComparisonResultSchema } from "../domain/javascript/javascriptExportShapeComparisonSchemas.js"; -import type { FixtureClaimExpectation } from "./KnownAnswerEvaluation.js"; +import { + managedArtifactInspectionSchema, + managedMemberInspectionSchema, +} from "../domain/managed/managedArtifact.js"; +import { + managedTokenScopesMatch, + parseManagedInspectionEvidence, +} from "../domain/managed/managedInspectionEvidence.js"; +import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js"; +import type { + FixtureClaimContext, + FixtureClaimExpectation, +} from "./KnownAnswerEvaluation.js"; /** Source-owned artifact paths used by the configured known-answer benchmarks. */ export interface AgentEvaluationFixtureTargets { @@ -16,6 +28,8 @@ export interface AgentEvaluationFixtureTargets { readonly javascriptSha256: string; readonly javascriptShapeLeft: string; readonly javascriptShapeRight: string; + readonly managed?: string; + readonly managedSha256?: string; } const desktopFixtureFiles = { @@ -277,17 +291,122 @@ const parserClaims = ( ]; }; +const managedMembers = ({ evidence }: FixtureClaimContext) => { + const { result } = parseManagedInspectionEvidence( + evidence, + "inspect_managed_members", + managedMemberInspectionSchema, + ); + if ( + result.coverage.state !== "complete" || + result.metadata.status !== "complete" || + !managedTokenScopesMatch( + result.identity_scope.requires_mvid, + result.module?.mvid ?? null, + ) + ) + return undefined; + return result; +}; + +const managedClaims = ( + targets: AgentEvaluationFixtureTargets, +): readonly FixtureClaimExpectation[] => { + if (targets.managed === undefined || targets.managedSha256 === undefined) + throw new TypeError( + "Managed fixture grading requires the built DLL path and SHA-256", + ); + const source = { + operation: "inspect_managed_members", + authority: "shipped-artifact", + confidence: "observed", + subject: { path: targets.managed, sha256: targets.managedSha256 }, + } satisfies Omit; + return [ + { + id: "public_types", + expectedValue: ["AgentEval.Profile", "AgentEval.Program"], + source: { + ...source, + select: (_value, context) => { + const members = managedMembers(context); + if ( + members === undefined || + members.selection !== null || + Object.hasOwn(context.evidence.parameters, "type_names") + ) + return undefined; + return members.types + .filter((type) => (type.flags & 7) === 1) + .map((type) => type.full_name) + .sort(compareUnicodeCodePoints); + }, + }, + }, + { + id: "entry_point", + expectedValue: { declaring_type: "AgentEval.Program", name: "Main" }, + source: { + ...source, + select: (_value, context) => { + const members = managedMembers(context); + if (members === undefined) return undefined; + const artifacts = context.deliveredEvidence.filter( + (evidence) => + evidence.operation === "inspect_managed_artifact" && + evidence.authority === source.authority && + evidence.confidence === source.confidence && + evidence.subject?.local_path === source.subject.path && + evidence.subject.digest.sha256 === source.subject.sha256, + ); + if (artifacts.length !== 1) return undefined; + const { result: artifact } = parseManagedInspectionEvidence( + artifacts[0], + "inspect_managed_artifact", + managedArtifactInspectionSchema, + ); + if ( + artifact.coverage.state !== "complete" || + artifact.metadata.status !== "complete" || + !managedTokenScopesMatch( + artifact.module?.mvid ?? null, + members.module?.mvid ?? null, + ) || + artifact.pe.cli?.entry_point.kind !== "metadata-token" + ) + return undefined; + const methods = members.methods.filter( + (method) => method.token === artifact.pe.cli?.entry_point.value, + ); + const method = methods.length === 1 ? methods[0] : undefined; + return method === undefined + ? undefined + : { + declaring_type: method.declaring_type, + name: method.name, + }; + }, + }, + }, + ]; +}; + /** Return the closed known-answer rubric, or undefined for routing-only scenarios. */ export const agentFixtureClaims = ( scenarioId: string, targets: AgentEvaluationFixtureTargets, ): readonly FixtureClaimExpectation[] | undefined => { if (scenarioId === "asar") return asarClaims(targets); + if (scenarioId === "managed") return managedClaims(targets); if (scenarioId === "javascript-export-shape") return parserClaims(targets); return undefined; }; const answerValueDescriptions: Readonly> = { + managed: [ + "public_types: string[], listing the exact public top-level type full names in Unicode code-point order from unfiltered member Evidence (omit type_names).", + 'entry_point: {"declaring_type": string, "name": string}, resolving the CLI metadata entry-point token against the member inventory from the same artifact SHA-256 and module MVID. Cite member Evidence; both artifact and member Evidence must be delivered.', + ], asar: [ 'renderer_api: {"key": string, "members": string[], "world": string, "source": string}, describing the exposed bridge API, its members, world, and source file.', 'renderer_ipc: {"channel": string, "operation": string, "source": string}, describing the renderer-side IPC operation and its source file.', diff --git a/src/evaluation/KnownAnswerEvaluation.ts b/src/evaluation/KnownAnswerEvaluation.ts index c6e4af72b..fe71302eb 100644 --- a/src/evaluation/KnownAnswerEvaluation.ts +++ b/src/evaluation/KnownAnswerEvaluation.ts @@ -5,6 +5,12 @@ import { parseEvidence, type Evidence } from "../domain/evidence.js"; import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js"; import { canonicalJson } from "../domain/comparisonSemantics.js"; +/** Only authenticated, successful, non-conflicting transcript deliveries. */ +export interface FixtureClaimContext { + readonly evidence: Evidence; + readonly deliveredEvidence: readonly Evidence[]; +} + /** One closed fixture assertion and the producer facts that must support it. */ export interface FixtureClaimExpectation { readonly id: string; @@ -20,7 +26,10 @@ export interface FixtureClaimExpectation { readonly path: string; readonly sha256?: string; }[]; - readonly select: (normalizedResult: JsonValue) => JsonValue | undefined; + readonly select: ( + normalizedResult: JsonValue, + context: FixtureClaimContext, + ) => JsonValue | undefined; }; } @@ -430,7 +439,12 @@ const assessClaim = ( try { if ( !sameJson( - expected.source.select(evidence.normalized_result), + expected.source.select(evidence.normalized_result, { + evidence, + deliveredEvidence: [...delivered.records.values()].filter( + (record) => !delivered.conflicts.has(record.evidence_id), + ), + }), expected.expectedValue, ) ) diff --git a/src/evm/EvmoleInterfaceProvider.ts b/src/evm/EvmoleInterfaceProvider.ts index c59427e6a..cbcb31615 100644 --- a/src/evm/EvmoleInterfaceProvider.ts +++ b/src/evm/EvmoleInterfaceProvider.ts @@ -30,6 +30,7 @@ import { import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; import type { InspectEvmInterfaceInput } from "../domain/evm/evmInterface.js"; import { evmInterfaceSchema } from "../domain/evm/evmInterface.js"; +import { solidityMetadataSchema } from "../domain/evm/solidityMetadata.js"; import { ProviderAdapterError } from "../domain/providerAdapterError.js"; import { ProviderCleanupError } from "../domain/providerCleanupError.js"; import { err, ok, type Result } from "../domain/result.js"; @@ -67,6 +68,7 @@ const replySchema = z.discriminatedUnion("ok", [ sha256: z.string().regex(/^[a-f0-9]{64}$/), bytes: z.number().int().nonnegative(), hex: z.string().regex(/^(?:[a-f0-9]{2})*$/), + metadata: solidityMetadataSchema.optional(), }), raw: evmoleInterfaceReplySchema, }), @@ -331,6 +333,7 @@ export class EvmoleInterfaceProvider implements EvmInterfacePort { "Recovered selectors, argument strings and mutability are upstream inferences, not verified ABI signatures or names. Empty results do not prove the absence of callable dispatch.", "Bytecode kind, hardfork and deployed-contract authenticity are unknown. This inspection performs no target runtime execution, chain/RPC lookup or network request.", "Original carrier and decoded bytecode SHA-256 are distinct identities; SHA-256 is not the Ethereum Keccak code hash.", + "Terminal CBOR metadata is an independently decoded candidate, not proof of Solidity origin, source identity or deployment. Nested or unsupported values retain explicit unavailable interpretation and original byte ranges.", "The owned worker has a 256 MiB JavaScript heap, 3 GiB virtual address-space limit and 30-second deadline. Virtual address space is distinct from resident memory; limits fail without partial success.", ]; const validated = evmInterfaceSchema.safeParse({ diff --git a/src/evm/EvmoleInterfaceWorker.ts b/src/evm/EvmoleInterfaceWorker.ts index 061ceb47e..17e50f7be 100644 --- a/src/evm/EvmoleInterfaceWorker.ts +++ b/src/evm/EvmoleInterfaceWorker.ts @@ -6,6 +6,7 @@ import { EvmCarrierFailure, } from "./EvmBytecodeCarrier.js"; import { projectEvmoleInterface } from "./EvmoleInterfaceAdapter.js"; +import { inspectSolidityMetadata } from "./SolidityMetadata.js"; import { EVM_INTERFACE_LIMITS, EVM_FILE_SIZE_FAILURE_EXIT, @@ -64,6 +65,7 @@ const main = async (): Promise => { sha256: createHash("sha256").update(bytes).digest("hex"), bytes: bytes.length, hex: Buffer.from(bytes).toString("hex"), + metadata: inspectSolidityMetadata(bytes), }, raw: inferred.raw, }; diff --git a/src/evm/SolidityMetadata.test.ts b/src/evm/SolidityMetadata.test.ts new file mode 100644 index 000000000..1ca056f18 --- /dev/null +++ b/src/evm/SolidityMetadata.test.ts @@ -0,0 +1,96 @@ +import { expect, it } from "vitest"; +import { inspectSolidityMetadata } from "./SolidityMetadata.js"; + +const bytecode = (hex: string) => { + const payload = Buffer.from(hex, "hex"); + const length = Buffer.alloc(2); + length.writeUInt16BE(payload.length); + return Buffer.concat([Buffer.from("00", "hex"), payload, length]); +}; + +it("reads a compiler version from actual trailer bytes without a magic-prefix assumption", () => { + const result = inspectSolidityMetadata(bytecode("a164736f6c634300081e")); + expect(result).toMatchObject({ + status: "decoded", + bytecode_offset: 1, + cbor_bytes: 10, + compiler_version: { representation: "release-bytes", value: "0.8.30" }, + entries: [ + { + key: { kind: "text", value: "solc" }, + value: { kind: "bytes", value: "00081e" }, + }, + ], + }); +}); + +it("preserves integer and non-text key precision beyond Number", () => { + expect( + inspectSolidityMetadata(bytecode("a1011bffffffffffffffff")).entries, + ).toEqual([ + { + key: { kind: "integer", value: "1" }, + value: { kind: "integer", value: "18446744073709551615" }, + }, + ]); + expect( + inspectSolidityMetadata(bytecode("a1003bffffffffffffffff")).entries[0] + ?.value, + ).toEqual({ + kind: "integer", + value: "-18446744073709551616", + }); +}); + +it("retains opaque nested values as partial rather than fabricating scalar values", () => { + expect(inspectSolidityMetadata(bytecode("a161788100"))).toMatchObject({ + status: "partial", + entries: [ + { + key: { kind: "text", value: "x" }, + value: { kind: "uninterpreted", container: "array" }, + }, + ], + }); +}); + +it("preserves a UTF-8 BOM inside a CBOR text value rather than stripping it", () => { + expect( + inspectSolidityMetadata(bytecode("a1617864efbbbf61")).entries[0]?.value, + ).toEqual({ + kind: "text", + value: "\ufeffa", + }); + expect( + inspectSolidityMetadata(bytecode("a167efbbbf736f6c634300081e")) + .compiler_version, + ).toBeNull(); +}); + +it.each([ + ["invalid text", "a1617861ff"], + ["truncated value", "a16178"], + ["unsupported tag", "a16178c000"], + ["floating point", "a16178fa3f800000"], + ["duplicate key", "a2617801617802"], + ["excessive nesting", "a16178" + "81".repeat(65) + "00"], + ["impossible collection", "a161788a00"], +])("keeps exact ranges and unavailable interpretation for %s", (_name, hex) => { + expect(inspectSolidityMetadata(bytecode(hex))).toMatchObject({ + status: "not-decoded", + bytecode_offset: 1, + entries: [], + compiler_version: null, + }); +}); + +it.each(["", "60006000f3", "000000", "00010001"])( + "does not claim absent Solidity metadata for an unavailable trailer: %s", + (hex) => + expect(inspectSolidityMetadata(Buffer.from(hex, "hex"))).toMatchObject({ + status: "unavailable", + bytecode_offset: null, + cbor_bytes: null, + entries: [], + }), +); diff --git a/src/evm/SolidityMetadata.ts b/src/evm/SolidityMetadata.ts new file mode 100644 index 000000000..5b24ce245 --- /dev/null +++ b/src/evm/SolidityMetadata.ts @@ -0,0 +1,158 @@ +import { decode, Tokenizer, Type, type Token } from "cborg"; +import type { + SolidityMetadata, + SolidityMetadataScalar, +} from "../domain/evm/solidityMetadata.js"; + +const PROFILE = "terminal-cbor-scalars-v1"; +const MAX_DEPTH = 64; +const utf8 = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }); +const decodeOptions = { + useMaps: true, + allowBigInt: true, + allowIndefinite: false, + allowUndefined: false, + allowInfinity: false, + allowNaN: false, + rejectDuplicateMapKeys: true, + retainStringBytes: true, +} as const; + +/** Bound recursive decoder entry before collection interpretation/allocation. */ +class BoundedMetadataTokenizer extends Tokenizer { + readonly remaining: number[] = []; + constructor(bytes: Uint8Array) { + super(bytes, decodeOptions); + } + + override next(): Token { + while (this.remaining.at(-1) === 0) this.remaining.pop(); + const token = super.next(); + if ( + Type.equals(token.type, Type.tag) || + Type.equals(token.type, Type.float) + ) + throw new Error( + "CBOR tags and floating-point values are outside this metadata profile.", + ); + if (Type.equals(token.type, Type.string)) { + if (token.byteValue === undefined) + throw new Error( + "CBOR text bytes were unavailable for strict UTF-8 validation.", + ); + token.value = utf8.decode(token.byteValue); + } + const parent = this.remaining.length - 1; + if (parent >= 0) this.remaining[parent]!--; + if ( + Type.equals(token.type, Type.map) || + Type.equals(token.type, Type.array) + ) { + const count: unknown = token.value; + const children = + typeof count === "number" + ? count * (Type.equals(token.type, Type.map) ? 2 : 1) + : NaN; + if ( + !Number.isSafeInteger(children) || + children < 0 || + children > this.data.length + ) + throw new Error( + "Declared CBOR collection exceeds the encoded item-byte bound.", + ); + if (this.remaining.length >= MAX_DEPTH) + throw new Error("CBOR nesting exceeds the 64-level metadata profile."); + this.remaining.push(children); + } + return token; + } +} + +const scalar = (value: unknown): SolidityMetadataScalar => { + if (typeof value === "string") return { kind: "text", value }; + if (typeof value === "bigint") + return { kind: "integer", value: value.toString() }; + if (typeof value === "number" && Number.isSafeInteger(value)) + return { kind: "integer", value: String(value) }; + if (typeof value === "boolean") return { kind: "boolean", value }; + if (value === null) return { kind: "null" }; + if (value instanceof Uint8Array) + return { kind: "bytes", value: Buffer.from(value).toString("hex") }; + if (value instanceof Map) return { kind: "uninterpreted", container: "map" }; + if (Array.isArray(value)) + return { kind: "uninterpreted", container: "array" }; + throw new Error( + "CBOR decoder returned a value outside the admitted profile.", + ); +}; + +/** Parse only a plausible terminal root-map candidate; never establish absence. */ +export const inspectSolidityMetadata = ( + bytes: Uint8Array, +): SolidityMetadata => { + const empty: SolidityMetadata = { + profile: PROFILE, + status: "unavailable", + bytecode_offset: null, + cbor_bytes: null, + entries: [], + compiler_version: null, + diagnostic: + "No plausible terminal CBOR map was identified; Solidity origin and metadata absence remain unknown.", + }; + if (bytes.length < 3) return empty; + const length = bytes[bytes.length - 2]! * 256 + bytes[bytes.length - 1]!; + const offset = bytes.length - 2 - length; + if (length === 0 || offset < 0 || bytes[offset]! >> 5 !== 5) return empty; + const selected = bytes.subarray(offset, bytes.length - 2); + const result: SolidityMetadata = { + ...empty, + status: "not-decoded", + bytecode_offset: offset, + cbor_bytes: length, + }; + try { + const raw: unknown = decode(selected, { + ...decodeOptions, + tokenizer: new BoundedMetadataTokenizer(selected), + }); + if (!(raw instanceof Map)) + throw new Error("Terminal CBOR candidate did not decode to a root map."); + for (const [key, value] of raw as Map) + result.entries.push({ key: scalar(key), value: scalar(value) }); + result.status = result.entries.some( + ({ key, value }) => + key.kind === "uninterpreted" || value.kind === "uninterpreted", + ) + ? "partial" + : "decoded"; + result.diagnostic = + result.status === "partial" + ? "Nested containers are uninterpreted; their exact bytes remain at the recorded range in original bytecode." + : null; + const version: unknown = raw.get("solc"); + if (version instanceof Uint8Array && version.length === 3) + result.compiler_version = { + representation: "release-bytes", + value: [...version].join("."), + }; + else if (typeof version === "string" && version.length > 0) + result.compiler_version = { + representation: "prerelease-text", + value: version, + }; + return result; + } catch (cause: unknown) { + return { + ...result, + status: "not-decoded", + entries: [], + compiler_version: null, + diagnostic: (cause instanceof Error + ? cause.message + : String(cause) + ).slice(0, 512), + }; + } +}; diff --git a/src/filesystem/RegularFile.ts b/src/filesystem/RegularFile.ts index 4942e2e46..47fdfa1d8 100644 --- a/src/filesystem/RegularFile.ts +++ b/src/filesystem/RegularFile.ts @@ -1,6 +1,7 @@ import { constants } from "node:fs"; +import { constants as osConstants } from "node:os"; import type { BigIntStats, Stats } from "node:fs"; -import { lstat, open, type FileHandle } from "node:fs/promises"; +import { lstat, open, stat, type FileHandle } from "node:fs/promises"; import { OwnedFileHandle } from "./OwnedFileHandle.js"; @@ -66,7 +67,17 @@ export const openRegularFile = async ( (constants.O_NONBLOCK ?? 0) | (constants.O_NOCTTY ?? 0) | (options.symlinks === "reject" ? (constants.O_NOFOLLOW ?? 0) : 0), - ); + ).catch(async (cause: unknown) => { + const nonRegular = await nonRegularOpenFailure( + path, + options.symlinks, + cause, + ); + if (nonRegular === undefined) throw cause; + // Cancellation during the metadata probe outranks the reclassification. + options.signal?.throwIfAborted(); + throw nonRegular; + }); const owner = new OwnedFileHandle(handle); try { options.signal?.throwIfAborted(); @@ -95,6 +106,41 @@ export const openRegularFile = async ( } }; +/** + * Some non-regular files cannot be opened at all: a Unix socket fails with + * ENXIO on Linux and EOPNOTSUPP on macOS. Confirm such a failure against the + * path's own metadata before reporting a non-regular selection. + */ +const nonRegularOpenFailure = async ( + path: string, + symlinks: "follow" | "reject", + cause: unknown, +): Promise => { + // Only errors meaning "this file type cannot be opened" are reclassified; + // permission, symlink-rejection and resource failures keep their codes. + if (!isUnopenableFileTypeError(cause)) return undefined; + const metadata = await (symlinks === "reject" ? lstat : stat)(path).catch( + () => undefined, + ); + if (metadata === undefined || metadata.isFile()) return undefined; + return new NonRegularFileReadError(path, metadata.isDirectory()); +}; + +// Match by errno: libuv has no name for macOS EOPNOTSUPP (102) and reports +// "Unknown system error -102" as its code. +const errnos: Partial> = osConstants.errno; +const UNOPENABLE_FILE_TYPE_ERRNOS: ReadonlySet = new Set( + [errnos.ENXIO, errnos.EOPNOTSUPP, errnos.ENOTSUP].flatMap((errno) => + errno === undefined ? [] : [-errno], + ), +); + +const isUnopenableFileTypeError = (cause: unknown): boolean => + cause instanceof Error && + "errno" in cause && + typeof cause.errno === "number" && + UNOPENABLE_FILE_TYPE_ERRNOS.has(cause.errno); + /** Compare one regular file's identity and mutable metadata across a read. */ export const sameRegularFileState = ( expected: Stats | BigIntStats, diff --git a/src/flutter/DartElf.test.ts b/src/flutter/DartElf.test.ts index b2b777de3..ff2724695 100644 --- a/src/flutter/DartElf.test.ts +++ b/src/flutter/DartElf.test.ts @@ -6,6 +6,7 @@ import { buildElf64Image, buildStrippedElf64Image, snapshotSection, + appendElfRanges, } from "./FlutterBuild.fixture.js"; const readElfSymbol = async (bytes: Buffer, name: string) => { @@ -143,6 +144,96 @@ describe("readElfSymbol", () => { }); describe("ELF symbol-table ownership", () => { + it.each([ + { address: 0x400, offset: 0x700, size: 52 }, + { address: 0x410, offset: 0x710, size: 4 }, + { address: 0x3f0, offset: 0x6f0, size: 20 }, + ])( + "rejects conflicting interpretations of any symbol byte: %j", + async (range) => { + const image = appendElfRanges(imageWithSections(), "program", [ + { type: 1, ...range }, + ]); + const names = ["_kDartVmSnapshotData", "_kDartIsolateSnapshotData"]; + const result = await readElfSymbols(image, names); + expect(result.symbols).toEqual([ + { name: names[1], offset: 0x800, size: 52 }, + ]); + expect(result.failures).toContainEqual({ + reason: expect.stringContaining("conflicting PT_LOAD"), + }); + const phoff = Number(image.readBigUInt64LE(0x20)); + const first = Buffer.from(image.subarray(phoff, phoff + 56)); + image.copy(image, phoff, phoff + 56, phoff + 112); + first.copy(image, phoff + 56); + expect(await readElfSymbols(image, names)).toEqual(result); + }, + ); + + it("accepts equivalent overlaps and excludes adjacent unrelated mappings", async () => { + const image = appendElfRanges(imageWithSections(), "program", [ + { type: 1, address: 0x410, offset: 0x410, size: 4 }, + { type: 1, address: 0x400 + 52, offset: 0x700, size: 4 }, + ]); + expect(await readElfSymbols(image, ["_kDartVmSnapshotData"])).toEqual({ + symbols: [{ name: "_kDartVmSnapshotData", offset: 0x400, size: 52 }], + failures: [], + }); + }); + + it("does not publish an earlier definition when another definition has unresolved mapping", async () => { + const image = imageWithSections(); + const shoff = Number(image.readBigUInt64LE(0x28)); + const symbols = Number(image.readBigUInt64LE(shoff + 64 + 0x18)); + image.writeUInt32LE(image.readUInt32LE(symbols), symbols + 24); + const conflict = appendElfRanges(image, "program", [ + { type: 1, address: 0x810, offset: 0x710, size: 4 }, + ]); + const result = await readElfSymbols(conflict, ["_kDartVmSnapshotData"]); + expect(result.symbols).toEqual([]); + expect(result.failures).toEqual([ + { + reason: "Symbol _kDartVmSnapshotData has conflicting PT_LOAD mappings", + }, + ]); + }); + + it("handles zero-width symbols at a unique segment end without choosing a conflicting shared endpoint", async () => { + const image = imageWithSections(); + const shoff = Number(image.readBigUInt64LE(0x28)); + const symbols = Number(image.readBigUInt64LE(shoff + 64 + 0x18)); + const at = Number(image.readBigUInt64LE(64 + 0x20)); + image.writeBigUInt64LE(BigInt(at), symbols + 8); + image.writeBigUInt64LE(0n, symbols + 16); + expect( + (await readElfSymbols(image, ["_kDartVmSnapshotData"])).symbols, + ).toEqual([{ name: "_kDartVmSnapshotData", offset: at, size: 0 }]); + const ambiguous = appendElfRanges(image, "program", [ + { type: 1, address: at, offset: 0x700, size: 4 }, + ]); + expect( + (await readElfSymbols(ambiguous, ["_kDartVmSnapshotData"])).symbols, + ).toEqual([]); + }); + + it.each(["hash", "symbols", "strings"] as const)( + "rejects partially conflicting dynamic %s mappings", + async (table) => { + const image = strippedImageWithSections(); + const dynamic = Number(image.readBigUInt64LE(64 + 56 + 8)); + const tagIndex = table === "hash" ? 4 : table === "symbols" ? 0 : 1; + const at = Number(image.readBigUInt64LE(dynamic + tagIndex * 16 + 8)); + const conflict = appendElfRanges(image, "program", [ + { type: 1, address: at + 1, offset: 0x700, size: 1 }, + ]); + const result = await readElfSymbols(conflict, ["_kDartVmSnapshotData"]); + expect(result.symbols).toEqual([]); + expect(result.failures).toContainEqual({ + reason: expect.stringContaining("conflicting PT_LOAD"), + }); + }, + ); + it.each([false, true])( "keeps a symbol terminator inside dynstr (stripped: %s)", async (stripped) => { diff --git a/src/flutter/DartElf.ts b/src/flutter/DartElf.ts index a9bf00fa5..b1c6fc584 100644 --- a/src/flutter/DartElf.ts +++ b/src/flutter/DartElf.ts @@ -1,5 +1,6 @@ import { Buffer } from "node:buffer"; import { setImmediate as yieldToEventLoop } from "node:timers/promises"; +import { createElfLoadMap } from "./ElfLoadMap.js"; import { SNAPSHOT_HASH_LENGTH, SNAPSHOT_HASH_OFFSET, @@ -205,6 +206,7 @@ export const readElfSymbols = async ( vaddr === null || offset === null || filesz === null || + !Number.isSafeInteger(vaddr + filesz) || offset > bytes.length || filesz > bytes.length - offset ) @@ -214,17 +216,7 @@ export const readElfSymbols = async ( return failure("Multiple PT_DYNAMIC segments are ambiguous"); else dynamicSegment = { offset, filesz }; } - const toOffset = (vaddr: number, size: number): number | null => { - const segment = segments.find( - (candidate) => - vaddr >= candidate.vaddr && - vaddr - candidate.vaddr <= candidate.filesz && - size <= candidate.filesz - (vaddr - candidate.vaddr), - ); - return segment === undefined - ? null - : segment.offset + vaddr - segment.vaddr; - }; + const toOffset = await createElfLoadMap(segments, signal); if (segments.length === 0) return failure("ELF image has no PT_LOAD segments"); const tables: { @@ -335,26 +327,25 @@ export const readElfSymbols = async ( const gnu = tags.get(0x6ffffef5); if (sysv !== undefined) { const at = toOffset(sysv, 8); - if (at === null) return failure("DT_HASH header is not file-backed"); - const buckets = reader.u32(at)!; - count = reader.u32(at + 4)!; - if (toOffset(sysv, 8 + 4 * (buckets + count)) === null) - return failure("DT_HASH table is truncated"); + if (!at.ok) return failure(`DT_HASH header ${at.error.reason}`); + const buckets = reader.u32(at.value)!; + count = reader.u32(at.value + 4)!; + const table = toOffset(sysv, 8 + 4 * (buckets + count)); + if (!table.ok) return failure(`DT_HASH table ${table.error.reason}`); } else if (gnu !== undefined) { const at = toOffset(gnu, 16); - if (at === null) return failure("DT_GNU_HASH header is not file-backed"); - const buckets = reader.u32(at)!; - const firstSymbol = reader.u32(at + 4)!; - const bloomWords = reader.u32(at + 8)!; + if (!at.ok) return failure(`DT_GNU_HASH header ${at.error.reason}`); + const buckets = reader.u32(at.value)!; + const firstSymbol = reader.u32(at.value + 4)!; + const bloomWords = reader.u32(at.value + 8)!; const prefixBytes = 16 + bloomWords * (is64 ? 8 : 4) + buckets * 4; - if ( - buckets === 0 || - bloomWords === 0 || - toOffset(gnu, prefixBytes) === null - ) + if (buckets === 0 || bloomWords === 0) return failure("DT_GNU_HASH buckets are malformed"); + const prefix = toOffset(gnu, prefixBytes); + if (!prefix.ok) + return failure(`DT_GNU_HASH buckets ${prefix.error.reason}`); let last = 0; - const bucketsAt = at + 16 + bloomWords * (is64 ? 8 : 4); + const bucketsAt = at.value + 16 + bloomWords * (is64 ? 8 : 4); for (let index = 0; index < buckets; index += 1) { if (index % 4096 === 0) { await yieldToEventLoop(); @@ -374,9 +365,10 @@ export const readElfSymbols = async ( signal?.throwIfAborted(); } const span = prefixBytes + (index - firstSymbol + 1) * 4; - if (toOffset(gnu, span) === null) - return failure("DT_GNU_HASH chain is unterminated"); - const chain = reader.u32(at + span - 4)!; + const range = toOffset(gnu, span); + if (!range.ok) + return failure(`DT_GNU_HASH chain ${range.error.reason}`); + const chain = reader.u32(at.value + span - 4)!; index += 1; if ((chain & 1) !== 0) { count = index; @@ -388,14 +380,22 @@ export const readElfSymbols = async ( return failure("Stripped ELF has no DT_HASH or DT_GNU_HASH symbol count"); const offset = toOffset(sym, count * entrySize); const stringOffset = toOffset(str, stringSize); - if (offset === null || stringOffset === null) - return failure("Dynamic symbol or string table is not file-backed"); - tables.push({ offset, count, stringOffset, stringSize }); + if (!offset.ok) + return failure(`Dynamic symbol table ${offset.error.reason}`); + if (!stringOffset.ok) + return failure(`Dynamic string table ${stringOffset.error.reason}`); + tables.push({ + offset: offset.value, + count, + stringOffset: stringOffset.value, + stringSize, + }); } const wanted = new Set(symbolNames); const symbols = new Map(); const ambiguous = new Set(); - const failures: ElfParseFailure[] = []; + const unresolved = new Set(); + const failures = new Set(); for (const table of tables) { for (let index = 0; index < table.count; index += 1) { if (index % 4096 === 0) { @@ -419,33 +419,37 @@ export const readElfSymbols = async ( const value = is64 ? reader.u64(base + 8) : reader.u32(base + 4); const size = is64 ? reader.u64(base + 16) : reader.u32(base + 8); if (value === null || size === null) { - failures.push({ reason: `Symbol ${name} has unrepresentable bounds` }); + failures.add(`Symbol ${name} has unrepresentable bounds`); + unresolved.add(name); continue; } const offset = toOffset(value, size); - if (offset === null) { - failures.push({ reason: `Symbol ${name} has no file-backed range` }); + if (!offset.ok) { + failures.add(`Symbol ${name} ${offset.error.reason}`); + unresolved.add(name); continue; } const previous = symbols.get(name); if ( previous !== undefined && - (previous.offset !== offset || previous.size !== size) + (previous.offset !== offset.value || previous.size !== size) ) ambiguous.add(name); - else symbols.set(name, { name, offset, size }); + else symbols.set(name, { name, offset: offset.value, size }); } } for (const name of ambiguous) { symbols.delete(name); - failures.push({ reason: `Symbol ${name} has conflicting definitions` }); + failures.add(`Symbol ${name} has conflicting definitions`); } + for (const name of unresolved) symbols.delete(name); for (const name of wanted) - if (!symbols.has(name)) - failures.push({ - reason: `symbol ${name} is not in the dynamic symbol table`, - }); - return { symbols: [...symbols.values()], failures }; + if (!symbols.has(name) && !ambiguous.has(name) && !unresolved.has(name)) + failures.add(`symbol ${name} is not in the dynamic symbol table`); + return { + symbols: [...symbols.values()], + failures: [...failures].map((reason) => ({ reason })), + }; }; /** Read one Dart snapshot section header from its file offset. */ diff --git a/src/flutter/ElfBuildId.ts b/src/flutter/ElfBuildId.ts index 3c04d0473..a7eb6145d 100644 --- a/src/flutter/ElfBuildId.ts +++ b/src/flutter/ElfBuildId.ts @@ -8,6 +8,9 @@ import { const MAX_METADATA_BYTES = 8 * 1024 * 1024; const MAX_BUILD_IDS = 64; +// Range bookkeeping and separate positional reads need a budget independent +// of payload bytes: tiny/overlapping declarations can otherwise fan out I/O. +const MAX_NOTE_RANGES = 1024; export interface ElfBuildIds { readonly candidates: readonly string[]; @@ -21,9 +24,10 @@ export const readElfBuildIds = async ( signal: AbortSignal | undefined, ): Promise => { const candidates = new Set(); + const failures: ElfParseFailure[] = []; const failure = (reason: string): ElfBuildIds => ({ candidates: [...candidates], - failures: [{ reason }], + failures: [...failures, { reason }], }); let examined = 0; const read = async (offset: number, size: number): Promise => { @@ -59,7 +63,7 @@ export const readElfBuildIds = async ( if (parsed.failure !== undefined) return failure(parsed.failure.reason); const h = parsed.header; const ranges = new Map(); - for (const table of [ + tables: for (const table of [ { offset: h.e_phoff, width: h.e_phentsize, @@ -105,7 +109,17 @@ export const readElfBuildIds = async ( size > totalBytes - offset ) return failure("ELF note range is not file-backed"); - ranges.set(`${offset}:${size}`, { offset, size }); + if (size === 0) continue; + const key = `${offset}:${size}`; + if (ranges.has(key)) continue; + if (ranges.size === MAX_NOTE_RANGES) { + failures.push({ + reason: + "ELF notes exceed the 1024-range note-range inventory/read budget", + }); + break tables; + } + ranges.set(key, { offset, size }); } } const align4 = (value: number): number => Math.ceil(value / 4) * 4; @@ -154,5 +168,5 @@ export const readElfBuildIds = async ( } } signal?.throwIfAborted(); - return { candidates: [...candidates], failures: [] }; + return { candidates: [...candidates], failures }; }; diff --git a/src/flutter/ElfLoadMap.ts b/src/flutter/ElfLoadMap.ts new file mode 100644 index 000000000..541e2dfdf --- /dev/null +++ b/src/flutter/ElfLoadMap.ts @@ -0,0 +1,101 @@ +import { setImmediate as yieldToEventLoop } from "node:timers/promises"; +import { err, ok, type Result } from "../domain/result.js"; + +interface LoadSegment { + readonly vaddr: number; + readonly offset: number; + readonly filesz: number; +} + +/** Normalize overlapping file-backed interpretations once, before any lookups. */ +export const createElfLoadMap = async ( + segments: readonly LoadSegment[], + signal?: AbortSignal, +): Promise< + (address: number, size: number) => Result +> => { + const events = segments + .filter((segment) => segment.filesz > 0) + .flatMap((segment) => [ + { at: segment.vaddr, delta: segment.offset - segment.vaddr, change: 1 }, + { + at: segment.vaddr + segment.filesz, + delta: segment.offset - segment.vaddr, + change: -1, + }, + ]) + .sort((left, right) => left.at - right.at); + const active = new Map(); + const spans: { start: number; end: number; delta: number | null }[] = []; + let previous = 0; + for (let index = 0; index < events.length; index += 1) { + if (index % 4096 === 0) { + await yieldToEventLoop(); + signal?.throwIfAborted(); + } + const event = events[index]!; + if (event.at > previous && active.size > 0) { + const delta = active.size === 1 ? active.keys().next().value! : null; + const last = spans.at(-1); + if (last?.end === previous && last.delta === delta) last.end = event.at; + else spans.push({ start: previous, end: event.at, delta }); + } + const count = (active.get(event.delta) ?? 0) + event.change; + if (count === 0) active.delete(event.delta); + else active.set(event.delta, count); + previous = event.at; + } + // Prefix counts keep growing GNU-hash range checks logarithmic even with + // many load segments. Null denotes conflicting interpretations, not a hole. + const conflicts = [0]; + for (const span of spans) + conflicts.push(conflicts.at(-1)! + Number(span.delta === null)); + const locate = (address: number): number => { + let low = 0, + high = spans.length; + while (low < high) { + const middle = Math.floor((low + high) / 2); + if (spans[middle]!.start <= address) low = middle + 1; + else high = middle; + } + return low - 1; + }; + signal?.throwIfAborted(); + return (address, size) => { + const missing = () => + err({ reason: "has no contiguous file-backed PT_LOAD range" }); + const ambiguous = () => err({ reason: "has conflicting PT_LOAD mappings" }); + if ( + !Number.isSafeInteger(address) || + !Number.isSafeInteger(size) || + address < 0 || + size < 0 || + !Number.isSafeInteger(address + size) + ) + return missing(); + const first = locate(address); + const span = spans[first]; + if (span === undefined || (size > 0 && address >= span.end)) + return missing(); + if (size === 0) { + // A zero-width symbol at a shared endpoint still needs one file offset. + if (address > span.end) return missing(); + const left = spans[first - 1]; + if ( + span.delta === null || + (span.start === address && + left?.end === address && + left.delta !== span.delta) + ) + return ambiguous(); + return ok(address + span.delta); + } + const last = locate(address + size - 1); + if (conflicts[last + 1]! !== conflicts[first]!) return ambiguous(); + return span.delta !== null && + address < span.end && + size <= span.end - address + ? ok(address + span.delta) + : missing(); + }; +}; diff --git a/src/flutter/FlutterBuild.fixture.ts b/src/flutter/FlutterBuild.fixture.ts index 635ea04fb..27c43c3bb 100644 --- a/src/flutter/FlutterBuild.fixture.ts +++ b/src/flutter/FlutterBuild.fixture.ts @@ -265,3 +265,50 @@ export const engineFixtureImage = (options: { labels.copy(image, noteAt + options.notes.length); return image; }; + +/** Relocate one table and append declarations without disturbing payload bytes. */ +export const appendElfRanges = ( + image: Buffer, + table: "program" | "section", + ranges: readonly { + type: number; + offset: number; + size: number; + address?: number; + }[], +): Buffer => { + const is64 = image[4] === 2; + const program = table === "program"; + const offsetAt = program ? (is64 ? 0x20 : 0x1c) : is64 ? 0x28 : 0x20; + const widthAt = program ? (is64 ? 0x36 : 0x2a) : is64 ? 0x3a : 0x2e; + const countAt = widthAt + 2; + const oldOffset = is64 + ? Number(image.readBigUInt64LE(offsetAt)) + : image.readUInt32LE(offsetAt); + const oldWidth = image.readUInt16LE(widthAt); + const oldCount = image.readUInt16LE(countAt); + const width = oldWidth || (is64 ? 64 : 40); + const count = oldCount || 1; // A section table starts with its null entry. + const offset = Math.ceil(image.length / 8) * 8; + const result = Buffer.alloc(offset + (count + ranges.length) * width); + image.copy(result); + image.copy(result, offset, oldOffset, oldOffset + oldCount * oldWidth); + const word = (value: number, at: number): void => { + if (is64) result.writeBigUInt64LE(BigInt(value), at); + else result.writeUInt32LE(value, at); + }; + word(offset, offsetAt); + result.writeUInt16LE(width, widthAt); + result.writeUInt16LE(count + ranges.length, countAt); + ranges.forEach((range, index) => { + const at = offset + (count + index) * width; + result.writeUInt32LE(range.type, at + (program ? 0 : 4)); + word(range.offset, at + (program ? (is64 ? 8 : 4) : is64 ? 0x18 : 0x10)); + word( + range.size, + at + (program ? (is64 ? 0x20 : 0x10) : is64 ? 0x20 : 0x14), + ); + if (program) word(range.address ?? 0, at + (is64 ? 0x10 : 8)); + }); + return result; +}; diff --git a/src/flutter/FlutterBuildProvider.ts b/src/flutter/FlutterBuildProvider.ts index 9a6e6fe50..ea5d56025 100644 --- a/src/flutter/FlutterBuildProvider.ts +++ b/src/flutter/FlutterBuildProvider.ts @@ -33,6 +33,7 @@ import { import { ZipArtifactReader } from "../artifacts/ZipArtifactReader.js"; import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js"; import { + NonRegularFileReadError, openRegularFile, RegularFileAdmissionFailure, sameRegularFileState, @@ -181,16 +182,29 @@ const mapInputFailure = ( { max_metadata_read_bytes: APK_METADATA_READ_MAX_BYTES }, { cause }, ); - const message = cause instanceof Error ? cause.message : String(cause); return new AnalysisInputError(operation, { cause }, [ { path: ["path"], reason: "invalid_format", - message: `The APK could not be read as a package: ${path}: ${message}`, + message: inputFailureMessage(path, cause), }, ]); }; +/** Name a missing or non-file selection instead of blaming package bytes. */ +const inputFailureMessage = (path: string, cause: unknown): string => { + if ( + cause instanceof Error && + "code" in cause && + (cause.code === "ENOENT" || cause.code === "ENOTDIR") + ) + return `Selected APK does not exist: ${path}`; + if (cause instanceof NonRegularFileReadError) + return `Selected APK is not a regular file: ${path}`; + const message = cause instanceof Error ? cause.message : String(cause); + return `The APK could not be read as a package: ${path}: ${message}`; +}; + /** Provider-owned storage failures must never be attributed to caller input. */ const mapOwnedIoFailure = ( operation: string, diff --git a/src/flutter/FlutterPayloadScan.test.ts b/src/flutter/FlutterPayloadScan.test.ts index 681a373a1..13bb4f743 100644 --- a/src/flutter/FlutterPayloadScan.test.ts +++ b/src/flutter/FlutterPayloadScan.test.ts @@ -70,6 +70,37 @@ describe("SnapshotHashScanner", () => { }); }); + it.each([49, 50, 51])( + "finds a section %i bytes after a rejected record at any chunking", + (start) => { + // A stray magic whose hash field is not hex, then a real section that + // starts inside the stray record's magic-to-hash span. + const payload = new Uint8Array(start + 52); + payload.set([0xf5, 0xf5, 0xdc, 0xdc], 0); + payload.set( + realSnapshotSection("65817c30a78bb44c3dc3771876b6010a"), + start, + ); + const whole = new SnapshotHashScanner(); + whole.push(payload); + expect(whole.result()).toEqual({ + capped: false, + sources: 1, + candidates: ["65817c30a78bb44c3dc3771876b6010a"], + }); + for (let split = 1; split < payload.length; split += 1) { + const scanner = new SnapshotHashScanner(); + scanner.push(payload.subarray(0, split)); + scanner.push(payload.subarray(split)); + expect(scanner.result(), `split at ${split}`).toEqual(whole.result()); + } + const bytewise = new SnapshotHashScanner(); + for (let offset = 0; offset < payload.length; offset += 1) + bytewise.push(payload.subarray(offset, offset + 1)); + expect(bytewise.result()).toEqual(whole.result()); + }, + ); + it("ignores a magic without a following hash", () => { const scanner = new SnapshotHashScanner(); scanner.push(Uint8Array.from([0xf5, 0xf5, 0xdc, 0xdc, 0xff, 0xff, 0xff])); diff --git a/src/flutter/FlutterPayloadScan.ts b/src/flutter/FlutterPayloadScan.ts index e1a69d594..a58ee5cc4 100644 --- a/src/flutter/FlutterPayloadScan.ts +++ b/src/flutter/FlutterPayloadScan.ts @@ -58,17 +58,16 @@ export class SnapshotHashScanner { private readonly candidates: string[] = []; private readonly candidateSet = new Set(); #buffer = new Uint8Array(0); - /** Absolute stream offset from which unprocessed bytes begin. */ - #base = 0; - #absoluteScanFrom = 0; push(chunk: Uint8Array): void { const joined = new Uint8Array(this.#buffer.length + chunk.length); joined.set(this.#buffer, 0); joined.set(chunk, this.#buffer.length); - // Resume after the last fully processed hash so carried bytes are not - // counted twice across chunk boundaries. - let offset = Math.max(0, this.#absoluteScanFrom - this.#base); + // Carried bytes are an unprocessed pending magic or too short to hold a + // magic, so rescanning them never counts a record twice. Within a chunk + // the scan resumes after each magic, so a magic inside a previous + // record's span is found wherever the chunk boundaries fall. + let offset = 0; let pendingMagic: number | null = null; for (;;) { const magic = findBytes(joined, DART_SNAPSHOT_MAGIC, offset); @@ -96,21 +95,14 @@ export class SnapshotHashScanner { // magic record whose field is not lowercase hex is not a source. if (isValidHash) this.#sources += 1; offset = magic + DART_SNAPSHOT_MAGIC.length; - this.#absoluteScanFrom = - this.#base + magic + SNAPSHOT_HASH_OFFSET + SNAPSHOT_HASH_LENGTH; } // Keep a pending magic for the next chunk; otherwise keep only the // trailing bytes where a magic could still begin, so magic-free // streams never accumulate. const keepFrom = pendingMagic ?? - Math.max( - this.#absoluteScanFrom - this.#base, - joined.length - (DART_SNAPSHOT_MAGIC.length - 1), - ); + Math.max(0, joined.length - (DART_SNAPSHOT_MAGIC.length - 1)); this.#buffer = joined.slice(keepFrom); - this.#base += keepFrom; - if (pendingMagic !== null) this.#absoluteScanFrom = this.#base; } result(): SnapshotHashScan { diff --git a/src/ghidra/GhidraAnalysisProfile.ts b/src/ghidra/GhidraAnalysisProfile.ts index 81defabd3..d4bd5a433 100644 --- a/src/ghidra/GhidraAnalysisProfile.ts +++ b/src/ghidra/GhidraAnalysisProfile.ts @@ -15,7 +15,7 @@ import { err, ok, type Result } from "../domain/result.js"; import type { GhidraInstallationInspection } from "./GhidraInstallation.js"; import { ghidraMipsProfileParameters, - ghidraMipsSharedLanguageOverride, + ghidraMipsLanguageOverride, } from "./GhidraMipsProfile.js"; import { ghidraRiscvLanguageOverride, @@ -44,8 +44,7 @@ import { export const ghidraPinnedLanguage = ( target: BinaryTarget, ): GhidraLanguageOverride | undefined => - ghidraMipsSharedLanguageOverride(target) ?? - ghidraRiscvLanguageOverride(target); + ghidraMipsLanguageOverride(target) ?? ghidraRiscvLanguageOverride(target); /** Resolve version-bound, deterministic semantics before Ghidra imports a target. */ export const resolveGhidraAnalysisProfile = async ( @@ -168,7 +167,7 @@ export const resolveGhidraAnalysisProfile = async ( `The caller-selected processor or compiler specification is outside the verified ${ target.architecture === "riscv" ? "generic little-endian RISC-V" - : "endian-matched MIPS32 shared-image" + : "endian-matched MIPS32" } profile. The requested override has not been changed.`, ), ); @@ -207,6 +206,9 @@ export const resolveGhidraAnalysisProfile = async ( ...(target.availableArchitectures ?? []), ].sort(), import_mode: "ephemeral-source-immutable", + ...(target.format === "pe" + ? { pdb_observations: "program-info-v1" } + : {}), annotation_policy: "atomic-function-entry-metadata-v1", load_image_observations: "source-mappings-entry-context-v2", function_body_evidence: "complete-inclusive-ranges-v1", @@ -273,7 +275,7 @@ export const resolveGhidraAnalysisProfile = async ( } : {}), ...analyzerParameters, - ...ghidraMipsProfileParameters(target), + ...ghidraMipsProfileParameters(target, languageOverride), ...ghidraRiscvProfileParameters(target), ...(pspExtension?.ok === true ? { diff --git a/src/ghidra/GhidraDoctor.test.ts b/src/ghidra/GhidraDoctor.test.ts index 69b9acb62..844438408 100644 --- a/src/ghidra/GhidraDoctor.test.ts +++ b/src/ghidra/GhidraDoctor.test.ts @@ -61,7 +61,7 @@ const doctorHost = ( id: "ubuntu", versionId: "24.04", packageFamily: "deb", - supported: true, + hopperSetupSupported: true, }), validTarget: () => Promise.resolve(true), executable: (path) => diff --git a/src/ghidra/GhidraFunctionValues.test.ts b/src/ghidra/GhidraFunctionValues.test.ts index e54692840..e7e5d8e9d 100644 --- a/src/ghidra/GhidraFunctionValues.test.ts +++ b/src/ghidra/GhidraFunctionValues.test.ts @@ -117,7 +117,9 @@ describe("Ghidra function-analysis result values", () => { }); }, ); +}); +describe("Ghidra reference and value-flow completeness", () => { it.each(["reference_kinds_available", "unresolved_calls"])( "rejects omitted %s instead of inventing complete reference observations", (field) => { diff --git a/src/ghidra/GhidraInventoryValues.ts b/src/ghidra/GhidraInventoryValues.ts index 86c17a7fb..85a8b8fbe 100644 --- a/src/ghidra/GhidraInventoryValues.ts +++ b/src/ghidra/GhidraInventoryValues.ts @@ -1,4 +1,9 @@ import { z } from "zod"; +import { + bytePatternFields, + validateBytePattern, + byteSearchResultSchema, +} from "../domain/native/bytePattern.js"; import { nativeLoadImageObservationSchema } from "../domain/native/nativeLoadImage.js"; import { @@ -20,6 +25,7 @@ import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js" export const GHIDRA_INVENTORY_OPERATIONS = [ "inspect_native_load_image", "read_bytes", + "search_bytes", "address_to_file_offset", "address_name", "list_documents", @@ -72,6 +78,16 @@ const searchInput = { }; const inputSchemas = { + search_bytes: z + .strictObject({ + ...bytePatternFields, + document, + start: explicitAddress.optional(), + end: explicitAddress.optional(), + limit: z.number().int().min(1).max(1000).default(100), + cursor: z.string().min(1).optional(), + }) + .superRefine(validateBytePattern), inspect_native_load_image: z.strictObject({}), read_bytes: z.strictObject({ document, @@ -271,6 +287,24 @@ const containingProcedure = z.discriminatedUnion("found", [ ]); const resultSchemas = { + search_bytes: byteSearchResultSchema.superRefine((value, context) => { + const addresses = [ + ...value.matches.flatMap((match) => [ + match.address, + ...(match.procedure === null ? [] : [match.procedure.address]), + ]), + ...[ + ...value.searched.blocks, + ...value.searched.skipped_uninitialized, + ...value.searched.skipped_not_loaded, + ].flatMap((block) => [block.start, block.end]), + ]; + if (addresses.some((value) => !canonicalAddress.safeParse(value).success)) + context.addIssue({ + code: "custom", + message: "Byte search addresses must be canonical", + }); + }), inspect_native_load_image: nativeLoadImageObservationSchema, read_bytes: z .strictObject({ diff --git a/src/ghidra/GhidraMipsProfile.test.ts b/src/ghidra/GhidraMipsProfile.test.ts index ae72a3069..800de9490 100644 --- a/src/ghidra/GhidraMipsProfile.test.ts +++ b/src/ghidra/GhidraMipsProfile.test.ts @@ -10,10 +10,17 @@ import { HopperProvider } from "../hopper/HopperProvider.js"; import { hopperLoaderArgsForTarget } from "../hopper/HopperAnalysisProfile.js"; import { IdaProvider } from "../ida/IdaProvider.js"; import { GhidraProvider } from "./GhidraProvider.js"; +import { + ghidraMipsProfileParameters, + ghidraPinnedMipsLanguage, +} from "./GhidraMipsProfile.js"; import type { GhidraInstallationHost } from "./GhidraInstallation.js"; import { GHIDRA_SESSION_CAPABILITIES } from "./GhidraSessionValues.js"; import type { GhidraMipsSoftFloatReport } from "./GhidraMipsProfile.js"; -import { ghidraProfileLanguageOverride } from "./GhidraAnalysisProfile.js"; +import { + ghidraPinnedLanguage, + ghidraProfileLanguageOverride, +} from "./GhidraAnalysisProfile.js"; // Fake installation inspection tests admission/profile contracts, not Ghidra. const host: GhidraInstallationHost = { @@ -161,6 +168,126 @@ describe("bounded Ghidra MIPS admission", () => { }); }); +describe("recordless legacy MIPS32 admission", () => { + it("keeps recordless relocatable objects off the inferred-o32 lane", async () => { + const ghidra = new GhidraProvider(config, silentLogger, {}, host); + const object = target({ + flags: 0x50000001, + abiFlags: null, + type: 1, + }); + expect(ghidra.inspectTargetSupport(object)).toMatchObject({ + status: "unsupported", + }); + const resolved = await ghidra.resolveAnalysisProfile(object); + expect(resolved.ok).toBe(false); + }); + + it.each([ + [{ flags: 0x50000001 }, "EF_MIPS_ARCH_32"], + [{ flags: 0x70000001 }, "EF_MIPS_ARCH_32R2"], + ] satisfies [Partial, string][])( + "admits a recordless legacy header (%s) with a pinned 32-bit language", + async (change) => { + const ghidra = new GhidraProvider(config, silentLogger, {}, host); + const little = target({ ...change, abiFlags: null }); + const big = target({ ...change, abiFlags: null, byteOrder: "big" }); + expect(ghidra.inspectTargetSupport(little).status).toBe("supported"); + expect(ghidra.inspectTargetSupport(big).status).toBe("supported"); + const resolved = await ghidra.resolveAnalysisProfile(little); + if (!resolved.ok) throw resolved.error; + expect(resolved.value.profile?.parameters).toMatchObject({ + mips_support_lane: "elf32-exec-arch32-legacy-pinned-v1", + mips_abi_basis: "inferred-o32-assumption-v1", + language_id: "MIPS:LE:32:default", + compiler_spec_id: "default", + language_selection: "provider-pinned-v1", + }); + const resolvedBig = await ghidra.resolveAnalysisProfile(big); + if (!resolvedBig.ok) throw resolvedBig.error; + expect(resolvedBig.value.profile?.parameters).toMatchObject({ + language_id: "MIPS:BE:32:default", + }); + }, + ); + + it("requires an inspected absence; a merely unobserved record stays refused", () => { + const ghidra = new GhidraProvider(config, silentLogger, {}, host); + const withRecord = target({ flags: 0x50000001 }) as BinaryTarget & { + mips: MipsElfMetadata; + }; + const { abiFlags: _record, ...unobservedMips } = withRecord.mips; + void _record; + expect( + ghidra.inspectTargetSupport({ + ...withRecord, + mips: unobservedMips, + } as BinaryTarget), + ).toMatchObject({ + status: "unsupported", + reason: expect.stringContaining("recordless legacy header"), + }); + }); + + it.each([ + [{ flags: 0x50000021 }, "n32"], + [{ flags: 0x50000041 }, "EF_MIPS_ABI_ON32"], + [{ flags: 0x50000101 }, "EF_MIPS_32BITMODE"], + [{ flags: 0x70000101 }, "EF_MIPS_32BITMODE"], + [ + { flags: 0x50001001 }, + "EF_MIPS_ARCH_32R2 for headers that declare an ABI", + ], + [{ flags: 0x50000201 }, "EF_MIPS_FP64"], + [{ flags: 0x50000401 }, "EF_MIPS_NAN2008"], + [{ flags: 0x60000001 }, "32R2"], + [ + { + flags: 0x50000001, + limitations: [ + "MIPS section header table at file offset 96 extends beyond the captured file: examined 2 of 5 complete headers. ABI declarations in unavailable headers remain unknown.", + ], + }, + "complete ABI-record inspection", + ], + ] satisfies [Partial, string][])( + "keeps refusing unverified recordless variants %j", + (change, reason) => { + const ghidra = new GhidraProvider(config, silentLogger, {}, host); + expect( + ghidra.inspectTargetSupport(target({ ...change, abiFlags: null })), + ).toMatchObject({ + status: "unsupported", + reason: expect.stringContaining(reason), + }); + }, + ); + + it("a caller-selected language wins over the provider pin", () => { + const parameters = ghidraMipsProfileParameters( + target({ flags: 0x50000001, abiFlags: null }), + { languageId: "MIPS:LE:32:default", compilerSpecId: "default" }, + ); + expect(parameters.language_selection).toBeUndefined(); + expect(parameters.language_id).toBeUndefined(); + expect(parameters).toMatchObject({ + mips_support_lane: "elf32-exec-arch32-legacy-pinned-v1", + }); + }); + + it("reads the pinned import language back only from pinned profiles", () => { + const pinned = ghidraMipsProfileParameters( + target({ flags: 0x50000001, abiFlags: null }), + ); + expect(ghidraPinnedMipsLanguage(pinned)).toEqual({ + languageId: "MIPS:LE:32:default", + compilerSpecId: "default", + }); + const standard = ghidraMipsProfileParameters(target()); + expect(ghidraPinnedMipsLanguage(standard)).toBeUndefined(); + }); +}); + const report: GhidraMipsSoftFloatReport = { policy: "o32-soft-float-prototype-v1", functions: 5, @@ -218,6 +345,56 @@ const session = async ( }; }; +describe("legacy profile bindings", () => { + it("refuses a legacy endian pin committed for a different target", async () => { + const { counts, client } = await session( + target({ flags: 0x50000001, abiFlags: null, byteOrder: "big" }), + undefined, + target({ flags: 0x50000001, abiFlags: null, byteOrder: "little" }), + ); + expect((await client.execute("health", {})).ok).toBe(false); + expect(counts.starts).toBe(0); + }); + it.each([0x50000201, 0x50000401, 0x54000001, 0x50910001])( + "refuses unsupported same-endian legacy flags %i before startup", + async (flags) => { + const value = target({ flags, abiFlags: null }); + expect(ghidraPinnedLanguage(value)).toBeUndefined(); + const { counts, client } = await session( + value, + undefined, + target({ flags: 0x50000001, abiFlags: null }), + ); + expect((await client.execute("health", {})).ok).toBe(false); + expect(counts.starts).toBe(0); + }, + ); + const unpinnedTargets: BinaryTarget[] = [ + target(), + { + path: "/fixture.elf", + sha256: "a".repeat(64), + kind: "executable", + format: "elf", + architecture: "x86_64", + availableArchitectures: ["x86_64"], + }, + ]; + it.each(unpinnedTargets)( + "refuses a pinned profile for an unpinned target $architecture", + async (value) => { + expect(ghidraPinnedLanguage(value)).toBeUndefined(); + const { counts, client } = await session( + value, + undefined, + target({ flags: 0x50000001, abiFlags: null }), + ); + expect((await client.execute("health", {})).ok).toBe(false); + expect(counts.starts).toBe(0); + }, + ); +}); + describe("MIPS16 shared-image profile", () => { const shared = (change: Partial = {}) => target({ diff --git a/src/ghidra/GhidraMipsProfile.ts b/src/ghidra/GhidraMipsProfile.ts index 031e273f1..76a753ccb 100644 --- a/src/ghidra/GhidraMipsProfile.ts +++ b/src/ghidra/GhidraMipsProfile.ts @@ -1,6 +1,6 @@ +import type { GhidraLanguageOverride } from "../config/ghidraLanguageOverride.js"; import type { BinaryTarget } from "../domain/binaryTargetTypes.js"; import type { JsonValue } from "../domain/jsonValue.js"; -import type { GhidraLanguageOverride } from "../config/ghidraLanguageOverride.js"; const MIPS_FP_ABI_SOFT = 3; const MIPS_ET_REL = 1; @@ -15,17 +15,56 @@ const isGhidraMips16SharedTarget = (target: BinaryTarget): boolean => target.mips.type === MIPS_ET_DYN && (target.mips.flags & 0x0f000000) === MIPS16_HEADER_ASE; -/** Generic MIPS32 permits per-function ISA_MODE; a global 16e processor does not. */ -export const ghidraMipsSharedLanguageOverride = ( +/** Derive the verified MIPS32 import pin from this target's observed byte order. */ +export const ghidraMipsLanguageOverride = ( target: BinaryTarget, ): GhidraLanguageOverride | undefined => - isGhidraMips16SharedTarget(target) && target.architecture === "mips" + (isGhidraMips16SharedTarget(target) || hasLegacyMipsHeaderShape(target)) && + ghidraMipsUnsupportedReason(target) === null && + target.architecture === "mips" ? { languageId: `MIPS:${target.mips.byteOrder === "little" ? "LE" : "BE"}:32:default`, compilerSpecId: "default", } : undefined; +/** + * Recordless legacy ELF32 MIPS admission: the header declares no ABI field and + * carries no ABI flags record, so o32 is an analysis assumption derived from + * ELF32/ET_EXEC structure, never a declared fact. Ghidra's header + * auto-selection picks a 64-bit language for exactly these files, so the lane + * pins the 32-bit language instead. + */ +const hasLegacyMipsHeaderShape = (target: BinaryTarget): boolean => { + if ( + target.kind !== "executable" || + target.architecture !== "mips" || + target.format !== "elf" || + target.mips === undefined + ) + return false; + const { flags, abiFlags, limitations } = target.mips; + const arch = flags & 0xf0000000; + return ( + // The lane and its verification cover executable firmware images only; + // recordless relocatable objects stay outside the inferred-o32 assumption. + target.mips.elfClass === 32 && + target.mips.type === MIPS_ET_EXEC && + (arch === 0x50000000 || arch === 0x70000000) && + (flags & 0x0000f000) === 0 && + (flags & 0x20) === 0 && + // EF_MIPS_ABI_ON32 (bit 0x40) declares the old n32 ABI; it is not an + // unspecified-ABI header and must not inherit the o32 assumption. + (flags & 0x40) === 0 && + (flags & 0x100) === 0 && + // Undefined means the record was not inspected; only an inspected + // absence establishes the recordless legacy admission. An inspection + // that could not examine every declaration source stays unknown. + abiFlags === null && + (limitations === undefined || limitations.length === 0) + ); +}; + /** Keep family recognition separate from the deliberately bounded Ghidra lane. */ export const ghidraMipsUnsupportedReason = ( target: BinaryTarget, @@ -48,13 +87,41 @@ export const ghidraMipsUnsupportedReason = ( return "MIPS16, microMIPS and other ASE encodings are outside this standard-instruction MIPS lane."; if ((metadata.flags & 0x00ff0000) !== 0) return "Machine-specific MIPS variants, including PSP Allegrex, need a separately supported processor profile."; - if ((metadata.flags & 0xf0000000) !== 0x70000000) - return "This Ghidra MIPS lane requires EF_MIPS_ARCH_32R2; other ISA revisions need separate verification."; + const arch = metadata.flags & 0xf0000000; + if (arch !== 0x70000000 && arch !== 0x50000000) + return "This Ghidra MIPS lane requires EF_MIPS_ARCH_32R2 or a recordless legacy EF_MIPS_ARCH_32 header; other ISA revisions need separate verification."; + const recordlessShape = + (metadata.flags & 0x0000f000) === 0 && + (metadata.flags & 0x20) === 0 && + metadata.abiFlags === null; + if (recordlessShape) { + if ((metadata.flags & 0x100) !== 0) + return "EF_MIPS_32BITMODE declares code for a 64-bit processor in 32-bit mode; it is outside the recordless MIPS32 admission."; + if ((metadata.flags & 0x40) !== 0) + return "EF_MIPS_ABI_ON32 declares the n32 ABI; recordless headers carrying it are outside the recordless legacy admission."; + if ((metadata.flags & 0x00000200) !== 0) + return "EF_MIPS_FP64 requires separate verification of 64-bit floating-point register semantics in this Ghidra lane."; + if ((metadata.flags & 0x00000400) !== 0) + return "EF_MIPS_NAN2008 requires separate verification of NaN encoding semantics in this Ghidra lane."; + if (hasLegacyMipsHeaderShape(target)) return null; + // A recordless relocatable object is not an incomplete inspection; the + // lane is scoped to executable firmware images by design. + if (metadata.type !== MIPS_ET_EXEC) + return "The recordless legacy admission covers ET_EXEC firmware images only; recordless relocatable objects need their own verified build, and the declared-ABI lane requires an ABI flags record."; + // The inspection could not examine every ABI declaration source, so the + // recordless admission is not established for this header. + return [ + "This recordless legacy admission requires a complete ABI-record inspection; ABI declarations in unavailable headers remain unknown.", + ...(metadata.limitations ?? []), + ].join(" "); + } if ( (metadata.flags & 0x0000f000) !== 0x00001000 || (metadata.flags & 0x20) !== 0 ) - return "This Ghidra MIPS lane requires an explicit o32 ABI; unspecified, n32 and EABI targets remain unsupported."; + return "This Ghidra MIPS lane requires an explicit o32 ABI or a recordless legacy header with an unspecified ABI; n32, EABI and mixed declarations remain unsupported."; + if (arch !== 0x70000000) + return "This Ghidra MIPS lane requires EF_MIPS_ARCH_32R2 for headers that declare an ABI; EF_MIPS_ARCH_32 headers are admitted only through the recordless legacy admission."; if ((metadata.flags & 0x00000200) !== 0) return "EF_MIPS_FP64 requires separate verification of 64-bit floating-point register semantics in this Ghidra lane."; if ((metadata.flags & 0x00000400) !== 0) @@ -98,6 +165,11 @@ export const ghidraMipsUnsupportedReason = ( return null; }; +/** A legacy lane is established only after every MIPS declaration is admitted. */ +export const isLegacyMipsAdmission = (target: BinaryTarget): boolean => + hasLegacyMipsHeaderShape(target) && + ghidraMipsUnsupportedReason(target) === null; + /** Soft-float prototype policy applied by ReaGhidraMipsSoftFloat.java after analysis. */ export const GHIDRA_MIPS_SOFT_FLOAT_STORAGE = "o32-soft-float-prototype-v1"; @@ -174,6 +246,7 @@ export const ghidraMipsSharedLimitations = ( /** Commit source ELF interpretation without confusing target ISA with host CPU. */ export const ghidraMipsProfileParameters = ( target: BinaryTarget, + languageOverride?: GhidraLanguageOverride, ): Readonly> => { if ( target.kind !== "executable" || @@ -182,6 +255,18 @@ export const ghidraMipsProfileParameters = ( ) return {}; const abi = target.mips.abiFlags; + // A deliberate caller-selected language wins over the legacy pin; the pin + // exists because header auto-selection misselects a 64-bit language. + const legacy = isLegacyMipsAdmission(target); + const language = ghidraMipsLanguageOverride(target); + const pinnedLanguage = + legacy && languageOverride === undefined && language !== undefined + ? { + language_id: language.languageId, + compiler_spec_id: language.compilerSpecId ?? "default", + language_selection: "provider-pinned-v1", + } + : {}; return { mips_elf: { elf_class: target.mips.elfClass, @@ -231,10 +316,28 @@ export const ghidraMipsProfileParameters = ( ? { mips_soft_float_storage: GHIDRA_MIPS_SOFT_FLOAT_STORAGE } : {}), instruction_context_evidence: "per-address-isa-mode-v1", - mips_support_lane: isGhidraMips16SharedTarget(target) - ? "elf32-dyn-o32-arch32r2-mips16-local-rel32-v1" - : target.mips.type === MIPS_ET_REL - ? "elf32-rel-o32-arch32r2-standard-v1" - : "elf32-exec-o32-arch32r2-standard-v3", + mips_support_lane: legacy + ? "elf32-exec-arch32-legacy-pinned-v1" + : isGhidraMips16SharedTarget(target) + ? "elf32-dyn-o32-arch32r2-mips16-local-rel32-v1" + : target.mips.type === MIPS_ET_REL + ? "elf32-rel-o32-arch32r2-standard-v1" + : "elf32-exec-o32-arch32r2-standard-v3", + ...(legacy ? { mips_abi_basis: "inferred-o32-assumption-v1" } : {}), + ...pinnedLanguage, }; }; + +/** Read the provider-pinned import language back from a resolved legacy profile. */ +export const ghidraPinnedMipsLanguage = ( + parameters: Readonly>, +): GhidraLanguageOverride | undefined => { + if (parameters.language_selection !== "provider-pinned-v1") return undefined; + const languageId = parameters.language_id; + const compilerSpecId = parameters.compiler_spec_id; + if (typeof languageId !== "string" || typeof compilerSpecId !== "string") + return undefined; + return compilerSpecId === "language-default" + ? { languageId } + : { languageId, compilerSpecId }; +}; diff --git a/src/ghidra/GhidraProvider.behavior.test.ts b/src/ghidra/GhidraProvider.behavior.test.ts index c9084905b..ccdfd96f7 100644 --- a/src/ghidra/GhidraProvider.behavior.test.ts +++ b/src/ghidra/GhidraProvider.behavior.test.ts @@ -1541,6 +1541,113 @@ const sessionInfo = () => ({ }, }); +describe("Ghidra PDB coverage", () => { + it.each([ + { name: "missing bridge report", pdb: undefined, warns: true }, + { + name: "report without a PDB reference", + pdb: { + file: null, + guid: null, + age: null, + signature: null, + version: null, + loaded: null, + }, + warns: false, + }, + ])("distinguishes $name in direct PE searches", async ({ pdb, warns }) => { + const ghidra = provider(installationHost(), () => ({ + start: () => + Promise.resolve( + ok(pdb === undefined ? sessionInfo() : { ...sessionInfo(), pdb }), + ), + callTool: () => Promise.resolve(ok([])), + close: () => Promise.resolve(ok(null)), + })); + const target = executableTarget("pe", "x86_64"); + const resolved = await ghidra.resolveAnalysisProfile(target); + if (!resolved.ok || resolved.value.profile === null) + throw new Error("Expected a PE profile"); + const client = ghidra.createClient(target, resolved.value.profile); + try { + const searched = await client.execute("search_procedures", { + pattern: "add_numbers", + }); + if (!searched.ok) throw searched.error; + expect(searched.value.result).toEqual([]); + const warnings = searched.value.limitations.filter((limitation) => + limitation.includes("PDB"), + ); + expect(warnings).toEqual( + warns + ? [ + "Ghidra did not report PDB attributes; debug-symbol coverage is unknown.", + ] + : [], + ); + } finally { + await client.close(); + } + }); + const pdb = { + file: "C:\\build\\fixture.pdb", + guid: "01234567-89AB-CDEF-0123-456789ABCDEF", + age: "a", + signature: null, + version: "RSDS", + loaded: false as boolean | null, + }; + it.each([false, null, true])( + "carries PDB load state %s into direct search and health Evidence", + async (loaded) => { + let starts = 0; + const ghidra = provider(installationHost(), () => ({ + start: () => { + starts += 1; + return Promise.resolve( + ok({ ...sessionInfo(), pdb: { ...pdb, loaded } }), + ); + }, + callTool: () => Promise.resolve(ok([])), + close: () => Promise.resolve(ok(null)), + })); + const target = executableTarget("pe", "x86_64"); + const resolved = await ghidra.resolveAnalysisProfile(target); + if (!resolved.ok || resolved.value.profile === null) + throw new Error("Expected a PE profile"); + const client = ghidra.createClient(target, resolved.value.profile); + // No preceding health call: a name search must establish symbol coverage itself. + const searched = await client.execute("search_procedures", { + pattern: "add_numbers", + }); + expect(starts).toBe(1); + if (!searched.ok) throw searched.error; + expect(searched.value.result).toEqual([]); + const health = await client.execute("health", {}); + if (!health.ok) throw health.error; + expect(health.value.result).toMatchObject({ pdb: { ...pdb, loaded } }); + for (const observation of [searched.value, health.value]) { + const limitations = observation.limitations.join("\n"); + if (loaded === true) { + expect(limitations).toContain("loaded PDB"); + expect(limitations).toContain("not an independent PDB identity"); + expect(limitations).toContain("empty name search does not prove"); + } else { + expect(limitations).toContain(pdb.file); + expect(limitations).toContain(pdb.guid); + expect(limitations).toContain("age (hex) a"); + expect(limitations).toContain("list_procedures"); + expect(limitations).toContain( + loaded === null ? "unknown" : "not loaded", + ); + } + } + await client.close(); + }, + ); +}); + const executableTarget = ( format: "mach-o" | "elf" | "pe", architecture: Extract["architecture"], diff --git a/src/ghidra/GhidraProviderCapabilities.ts b/src/ghidra/GhidraProviderCapabilities.ts index 68c3045ce..0714d1ce0 100644 --- a/src/ghidra/GhidraProviderCapabilities.ts +++ b/src/ghidra/GhidraProviderCapabilities.ts @@ -90,6 +90,11 @@ export const limitationsFor = (operation: string): readonly string[] => { ...common, "Bytes reflect Ghidra's initialized memory, including loader fixups. Reads stop at an unmapped or uninitialized byte and report completeness explicitly.", ]; + case "search_bytes": + return [ + ...common, + "Searches initialized loader-adjusted memory independently per block, in provider address order. Bounds and page coverage ends are inclusive. Uninitialized blocks are listed separately. Overlapping matches are included; patterns never cross block boundaries. A cursor resumes the same target, profile and query. Cancellation discards the active search and returns no successful partial page.", + ]; case "address_to_file_offset": return [ ...common, diff --git a/src/ghidra/GhidraProviderClient.ts b/src/ghidra/GhidraProviderClient.ts index 3a15dbc4e..25838453e 100644 --- a/src/ghidra/GhidraProviderClient.ts +++ b/src/ghidra/GhidraProviderClient.ts @@ -20,6 +20,7 @@ import { sameGhidraDisabledAnalyzers, sameGhidraLanguageOverride, } from "./GhidraAnalysisProfile.js"; +import { ghidraPinnedMipsLanguage } from "./GhidraMipsProfile.js"; import type { AnalysisProfileCommitment } from "../domain/analysisProfile.js"; import type { BinaryTarget } from "../domain/binaryTargetTypes.js"; import { @@ -77,10 +78,7 @@ import { parseNativeAotPe, type NativeAotPeResult, } from "../domain/native/nativeAotPe.js"; -import { - ABORTED, - waitForAbortable, -} from "../application/binary/AbortablePromise.js"; +import { ABORTED, waitForAbortable } from "../process/AbortablePromise.js"; import { nativeLoadImageObservationSchema } from "../domain/native/nativeLoadImage.js"; import { GHIDRA_PROVIDER_IDENTITY, @@ -90,7 +88,10 @@ import { relationshipCoverageFor, } from "./GhidraProviderCapabilities.js"; import type { GhidraSessionError } from "./GhidraSessionError.js"; -import type { GhidraSessionInfo } from "./GhidraSessionValues.js"; +import { + ghidraPdbLimitations, + type GhidraSessionInfo, +} from "./GhidraSessionValues.js"; import { ghidraExtensionFailure } from "./extensions/GhidraExtensionFailures.js"; import { ghidraExtensionSchema, @@ -449,13 +450,39 @@ export const createGhidraProviderClient = (input: { }), ); const seeds = seedProfile.data; - const languageOverride = ghidraProfileLanguageOverride( + const parameterLanguage = ghidraProfileLanguageOverride( committedProfile.parameters, ); + const pinnedLanguage = ghidraPinnedMipsLanguage(committedProfile.parameters); + const targetLanguage = ghidraPinnedLanguage(target); + if (pinnedLanguage !== undefined && targetLanguage === undefined) + return unavailableClient( + new ProviderAdapterError("ghidra", "open_binary", { + diagnostics: { + reason: + "The committed profile pins a MIPS language that the current target does not establish; reopen an admitted target to resolve a new profile.", + }, + }), + ); + // Provider-pinned profiles declare their language through the same + // parameter slot caller-configured profiles use, so only an environment + // override conflicts with the legacy pin; the slot itself carries it. + if ( + pinnedLanguage !== undefined && + config.ghidraLanguageOverride !== undefined + ) + return unavailableClient( + new ProviderAdapterError("ghidra", "open_binary", { + diagnostics: { + reason: + "The committed legacy MIPS profile pins its import language; REA_GHIDRA_LANGUAGE_ID/REA_GHIDRA_COMPILER_SPEC_ID cannot be combined with it. Reopen the target without the override.", + }, + }), + ); if ( !sameGhidraLanguageOverride( - languageOverride, - config.ghidraLanguageOverride ?? ghidraPinnedLanguage(target), + parameterLanguage, + config.ghidraLanguageOverride ?? targetLanguage, ) ) return unavailableClient( @@ -466,6 +493,9 @@ export const createGhidraProviderClient = (input: { }, }), ); + // Both caller-selected and provider-pinned languages use the validated + // committed slot; the profile cannot supply its own applicability proof. + const importLanguage = parameterLanguage; const disabledAnalyzers = ghidraProfileDisabledAnalyzers( committedProfile.parameters, ); @@ -483,6 +513,7 @@ export const createGhidraProviderClient = (input: { }, }), ); + const invalidProfile = validateGhidraExtensionProfile( extensions, config, @@ -498,6 +529,7 @@ export const createGhidraProviderClient = (input: { let extensionFailure: AnalysisError | undefined; // Replaced by the reported outcome once the session handshake is checked. const seedLimitations: string[] = [...ghidraSeedLimitations(seeds)]; + const pdbLimitations: string[] = []; // Replaced by the reported storage outcome once the handshake is checked. const softFloatLimitations: string[] = [ ...ghidraMipsSoftFloatLimitations(softFloat), @@ -523,6 +555,13 @@ export const createGhidraProviderClient = (input: { ...(target.kind === "executable" && target.architecture === "mips" ? (target.mips.limitations ?? []) : []), + ...(committedProfile.parameters.mips_abi_basis === + "inferred-o32-assumption-v1" + ? [ + "The ELF header declares no ABI field and no ABI flags record was observed; o32 calling conventions are an analysis assumption of this lane, not a declared fact.", + "Floating-point ABI semantics are unverified for this recordless legacy admission.", + ] + : []), ...ghidraMipsRelocatableLimitations(target), ...ghidraMipsSharedLimitations(target), ...ghidraRiscvLimitations(target), @@ -563,7 +602,9 @@ export const createGhidraProviderClient = (input: { baseAddr: target.entryPoint, }, }), - ...(languageOverride === undefined ? {} : { languageOverride }), + ...(importLanguage === undefined + ? {} + : { languageOverride: importLanguage }), ...(config.ghidraDisabledAnalyzers === undefined ? {} : { disabledAnalyzers: config.ghidraDisabledAnalyzers }), @@ -588,13 +629,13 @@ export const createGhidraProviderClient = (input: { expectedCompilerSpecId: "default", } : rawLanguage === undefined - ? languageOverride === undefined + ? importLanguage === undefined ? {} : { - expectedLanguageId: languageOverride.languageId, - ...(languageOverride.compilerSpecId === undefined + expectedLanguageId: importLanguage.languageId, + ...(importLanguage.compilerSpecId === undefined ? {} - : { expectedCompilerSpecId: languageOverride.compilerSpecId }), + : { expectedCompilerSpecId: importLanguage.compilerSpecId }), } : { expectedLanguageId: rawLanguage.languageId, @@ -609,10 +650,16 @@ export const createGhidraProviderClient = (input: { ...(context === undefined ? {} : { runId: context.runId }), logger: logger.child({ layer: "ghidra-bridge" }), }); - const checkExtensions = async ( + const checkSessionObservations = async ( operation: AnalysisOperation, info: GhidraSessionInfo, ): Promise => { + if (target.format === "pe") + pdbLimitations.splice( + 0, + pdbLimitations.length, + ...ghidraPdbLimitations(info.pdb), + ); const softFloatReason = ghidraMipsSoftFloatFailure( softFloat, info.mips_soft_float_storage, @@ -664,6 +711,7 @@ export const createGhidraProviderClient = (input: { ...softFloatLimitations, ...ghidraExtensionLimitations(extensions), ...seedLimitations, + ...pdbLimitations, ...(releaseLimitation === undefined ? [] : [releaseLimitation]), ]; const nativeAotOwner = @@ -735,7 +783,7 @@ export const createGhidraProviderClient = (input: { config.ghidraStartupTimeoutMs, ), ); - const failed = await checkExtensions(operation, started.value); + const failed = await checkSessionObservations(operation, started.value); if (failed !== undefined) return err(failed); return ok( createAnalysisExecution(started.value, committedProfile.provider, { @@ -748,7 +796,12 @@ export const createGhidraProviderClient = (input: { ? parseGhidraFunctionInput(operation, parameters) : parseGhidraInventoryInput(operation, parameters); if (!input.ok) return input; - if (extensions.length > 0 || seeds !== undefined || softFloat) { + if ( + target.format === "pe" || + extensions.length > 0 || + seeds !== undefined || + softFloat + ) { const started = await client.start(options?.signal, options?.progress); if (!started.ok) return err( @@ -758,7 +811,7 @@ export const createGhidraProviderClient = (input: { config.ghidraStartupTimeoutMs, ), ); - const failed = await checkExtensions(operation, started.value); + const failed = await checkSessionObservations(operation, started.value); if (failed !== undefined) return err(failed); } const called = await client.callTool(operation, input.value, { diff --git a/src/ghidra/GhidraSessionValues.test.ts b/src/ghidra/GhidraSessionValues.test.ts index a0430f35e..a68ed1615 100644 --- a/src/ghidra/GhidraSessionValues.test.ts +++ b/src/ghidra/GhidraSessionValues.test.ts @@ -52,6 +52,49 @@ describe("Ghidra DOS import commitment", () => { }); }); +describe("Ghidra PDB observations", () => { + const pdb = { + file: "C:\\build\\fixture.pdb", + guid: "01234567-89AB-CDEF-0123-456789ABCDEF", + age: "a", + signature: null, + version: "RSDS", + loaded: false, + }; + it("retains the referenced identity and observed load state", () => { + const parsed = parseGhidraSessionInfo({ ...session(), pdb }, expected); + if (!parsed.ok) throw parsed.error; + expect(parsed.value.pdb).toEqual(pdb); + }); + it("preserves missing attributes as unknown", () => { + const unknown = Object.fromEntries( + Object.keys(pdb).map((key) => [key, null]), + ); + const parsed = parseGhidraSessionInfo( + { ...session(), pdb: unknown }, + expected, + ); + if (!parsed.ok) throw parsed.error; + expect(parsed.value.pdb).toEqual(unknown); + }); + it.each([ + { ...pdb, loaded: "false" }, + { ...pdb, age: 1 }, + { + guid: pdb.guid, + age: pdb.age, + signature: null, + version: "RSDS", + loaded: false, + }, + { ...pdb, matched: true }, + ])("rejects malformed PDB reports", (report) => { + expect( + parseGhidraSessionInfo({ ...session(), pdb: report }, expected).ok, + ).toBe(false); + }); +}); + describe("Ghidra MIPS soft-float storage report", () => { const valid = { policy: "o32-soft-float-prototype-v1", diff --git a/src/ghidra/GhidraSessionValues.ts b/src/ghidra/GhidraSessionValues.ts index 0ee2a6bf1..e370892f3 100644 --- a/src/ghidra/GhidraSessionValues.ts +++ b/src/ghidra/GhidraSessionValues.ts @@ -25,6 +25,13 @@ export const GHIDRA_MUTATING_OPERATIONS: ReadonlySet = new Set([ ...GHIDRA_NAMING_OPERATIONS, ]); +// Shared wire budget: Java bounds the aggregate before serializing the handshake. +const PDB_ATTRIBUTE_UTF16_BUDGET = 4096; +const pdbAttributeSchema = z + .string() + .max(PDB_ATTRIBUTE_UTF16_BUDGET) + .nullable(); + const capabilitySchema = z.enum(GHIDRA_SESSION_CAPABILITIES); const sessionInfoSchema = z @@ -41,6 +48,39 @@ const sessionInfoSchema = z read_only: z.boolean(), analysis_complete: z.boolean(), analysis_timed_out: z.boolean(), + /** Program Information attributes; absent load state remains unknown. */ + pdb: z + .strictObject({ + file: pdbAttributeSchema, + guid: pdbAttributeSchema, + /** Ghidra stores CodeView age as a hexadecimal string. */ + age: pdbAttributeSchema, + signature: pdbAttributeSchema, + version: pdbAttributeSchema, + loaded: z.boolean().nullable(), + resource_limit: z + .strictObject({ + utf16_budget: z.literal(PDB_ATTRIBUTE_UTF16_BUDGET), + omitted_attributes: z + .array(z.enum(["file", "guid", "age", "signature", "version"])) + .min(1) + .max(5), + }) + .optional(), + }) + .refine( + (pdb) => + [ + pdb.file, + pdb.guid, + pdb.age, + pdb.signature, + pdb.version, + ].reduce((sum, value) => sum + (value?.length ?? 0), 0) <= + PDB_ATTRIBUTE_UTF16_BUDGET, + "PDB attributes exceed the aggregate wire budget", + ) + .optional(), analysis_extensions: z.array(ghidraExtensionResultSchema).optional(), analysis_seeds: ghidraSeedReportSchema.optional(), mips_soft_float_storage: z @@ -68,6 +108,37 @@ const sessionInfoSchema = z /** Authenticated metadata produced after Ghidra finishes auto-analysis. */ export type GhidraSessionInfo = z.infer; +/** Describe missing PDB authority without claiming all names are generated. */ +export const ghidraPdbLimitations = ( + pdb: GhidraSessionInfo["pdb"], +): readonly string[] => { + if (pdb === undefined) + return [ + "Ghidra did not report PDB attributes; debug-symbol coverage is unknown.", + ]; + const limitations: string[] = []; + if (pdb.resource_limit !== undefined) + limitations.push( + `PDB attributes ${pdb.resource_limit.omitted_attributes.join(", ")} were omitted because the handshake's ${pdb.resource_limit.utf16_budget}-UTF-16-unit PDB attribute budget was exhausted. Omitted values remain unknown; they were not truncated or reported as absent.`, + ); + if (pdb.loaded === true) + return [ + ...limitations, + "Ghidra reports a loaded PDB; this is not an independent PDB identity or complete symbol/type-coverage attestation. Original names and types may still be missing; an empty name search does not prove a procedure is absent. Use list_procedures and address-based analysis to inspect recovered functions.", + ]; + if ( + [pdb.file, pdb.guid, pdb.age, pdb.signature, pdb.version].every( + (value) => value === null, + ) + ) + return limitations; + const state = pdb.loaded === false ? "not loaded" : "load state unknown"; + return [ + ...limitations, + `The target references PDB ${pdb.file ?? "(file unknown)"} (GUID ${pdb.guid ?? "unknown"}, age (hex) ${pdb.age ?? "unknown"}, signature ${pdb.signature ?? "unknown"}, version ${pdb.version ?? "unknown"}); Ghidra reports ${state}. Original names and types may be missing; an empty name search does not prove a procedure is absent. Use list_procedures and address-based analysis to inspect recovered functions. REA does not admit a caller-selected PDB companion.`, + ]; +}; + /** Require exact run, provider, and profile identity from the Java bridge. */ export const parseGhidraSessionInfo = ( value: JsonValue, diff --git a/src/hopper/HopperProviderCapabilities.ts b/src/hopper/HopperProviderCapabilities.ts index 00f89fcdb..2dbac4659 100644 --- a/src/hopper/HopperProviderCapabilities.ts +++ b/src/hopper/HopperProviderCapabilities.ts @@ -62,12 +62,16 @@ const MUTATING_OPERATIONS = new Set([ /** Source-owned capabilities without acquiring or probing a Hopper session. */ export const CAPABILITIES: readonly CapabilityDescriptor[] = Object.freeze( - HOPPER_OPERATIONS.map((operation) => + [...HOPPER_OPERATIONS, "search_bytes" as const].map((operation) => Object.freeze({ provider: HOPPER_PROVIDER_IDENTITY, operation, - available: true, - reason: null, + ...(operation === "search_bytes" + ? { + available: false as const, + reason: "Masked byte search has not been verified for Hopper.", + } + : { available: true as const, reason: null }), effects: Object.freeze({ mutatesArtifact: MUTATING_OPERATIONS.has(operation), launchesProcess: true, diff --git a/src/ida/IdaMcpConnection.ts b/src/ida/IdaMcpConnection.ts index d532a66e8..8137edcc9 100644 --- a/src/ida/IdaMcpConnection.ts +++ b/src/ida/IdaMcpConnection.ts @@ -13,6 +13,7 @@ import { z } from "zod"; import type { IdaConfiguration } from "./IdaConfiguration.js"; import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js"; import { AnalysisProtocolError } from "../domain/analysisErrorCore.js"; +import { ProviderAdapterError } from "../domain/providerAdapterError.js"; /** Injectable MCP boundary used by the provider and transport conformance tests. */ export interface IdaMcpConnection { @@ -142,8 +143,30 @@ export const createIdaMcpConnection = ( environment: Readonly, ): IdaMcpConnection => { const client = new Client({ name: "rea-ida-provider", version: "1" }); + let stderr = Buffer.alloc(0); + let stderrBytes = 0; const transportFailure = (cause: unknown): never => { - throw redactIdaTransportFailure(cause, config); + const failure = redactIdaTransportFailure(cause, config); + if (stderrBytes === 0) throw failure; + let start = 0; + // The byte tail may start inside a UTF-8 code point. Retain whole units. + if (stderrBytes > stderr.length) + while (start < stderr.length && (stderr[start]! & 0xc0) === 0x80) start++; + const diagnostic = redactIdaTransportFailure( + new Error(stderr.subarray(start).toString("utf8")), + config, + ).message; + throw new ProviderAdapterError("ida", "transport", { + cause: failure, + diagnostics: { reason: failure.message }, + capturedOutput: { + stdout: "", + stderr: diagnostic, + truncated: stderrBytes > stderr.length - start, + stdout_bytes: 0, + stderr_bytes: stderrBytes, + }, + }); }; const transport = "command" in config @@ -169,9 +192,15 @@ export const createIdaMcpConnection = ( : new StreamableHTTPClientTransport(new URL(config.url), { requestInit: { headers: config.headers, redirect: "error" }, }); - // Drain proxy diagnostics without persisting ambient environment or transport credentials. + // Keep a bounded diagnostic tail for genuine failures; never persist ambient variables. if (transport instanceof StdioClientTransport) - transport.stderr?.on("data", () => undefined); + transport.stderr?.on("data", (chunk: Buffer) => { + stderrBytes += chunk.length; + const combined = Buffer.concat([stderr, chunk]); + stderr = Buffer.from( + combined.subarray(Math.max(0, combined.length - 16_384)), + ); + }); return { async connect() { await client diff --git a/src/ida/IdaNativeProtocol.test.ts b/src/ida/IdaNativeProtocol.test.ts index dc95727f6..466a09ad5 100644 --- a/src/ida/IdaNativeProtocol.test.ts +++ b/src/ida/IdaNativeProtocol.test.ts @@ -243,6 +243,10 @@ describe("native IDA provider observations", () => { expect(dossier.value.result).toMatchObject({ callers: [], callees: [{ address: "0x2000", name: "puts" }], + facet_availability: { + callers: { available: false }, + callees: { available: true }, + }, }); if (more) expect(dossier.value.limitations).toContain( @@ -275,6 +279,11 @@ describe("native IDA provider observations", () => { ); if (!result.ok) throw result.error; expect(result.value.provider.name).toBe("IDA MCP adapter"); + if (operation === "xrefs") + expect( + result.value.referencesTruncated, + "the native producer's explicit cap flag is forwarded", + ).toBe(false); if (operation === "procedure_assembly") expect(result.value.result).toBe("0x1000: ret"); if (operation === "procedure_pseudo_code") { diff --git a/src/ida/IdaOperationRunner.ts b/src/ida/IdaOperationRunner.ts index 25a49c268..576dac1e7 100644 --- a/src/ida/IdaOperationRunner.ts +++ b/src/ida/IdaOperationRunner.ts @@ -70,6 +70,9 @@ const identity = (fn: IdaFunction) => ({ interface IdaRelationships { readonly items: ReturnType[]; readonly coverage: RelationshipCoverage; + readonly availability: + | { readonly available: true } + | { readonly available: false; readonly reason: string }; } /** The only lookup answers upstream reports for a name or address that is not a function. */ @@ -469,7 +472,14 @@ export class IdaOperationRunner { }; if (this.readsCallGraph(direction)) { coverage = await this.addCallGraphCallees(fn, result); - return { items: [...result.values()], coverage }; + return { + items: [...result.values()], + coverage, + availability: + coverage.status === "unknown" + ? { available: false, reason: coverage.reason } + : { available: true }, + }; } if (this.protocol === "legacy") { const value = await this.call( @@ -491,6 +501,11 @@ export class IdaOperationRunner { ); return { items: [], + availability: { + available: false, + reason: + "Upstream xrefs do not establish direct caller relationships.", + }, coverage: { status: "unknown", reason: @@ -555,7 +570,11 @@ export class IdaOperationRunner { result.set(resolved.address, resolved); } } - return { items: [...result.values()], coverage }; + return { + items: [...result.values()], + coverage, + availability: { available: true }, + }; } async run( @@ -694,6 +713,32 @@ export class IdaOperationRunner { basic_blocks: [], native_api: null, native_value_flow: null, + facet_availability: { + pseudocode: + pseudocode === null + ? { available: false, reason: "IDA did not supply pseudocode." } + : { available: true }, + assembly: { available: true }, + comments: { + available: false, + reason: "IDA adapter does not expose typed comments.", + }, + callers: callers.availability, + callees: callees.availability, + references: { + available: false, + reason: "IDA adapter does not expose typed reference edges.", + }, + strings_names: { + available: false, + reason: + "IDA adapter does not expose function-referenced strings and names.", + }, + cfg: { + available: false, + reason: "IDA adapter does not expose basic blocks.", + }, + }, limitations: [ ...IDA_LIMITATIONS, ...this.limitations, diff --git a/src/ida/IdaSessionClient.test.ts b/src/ida/IdaSessionClient.test.ts index 7fa366bce..1a35dd7c7 100644 --- a/src/ida/IdaSessionClient.test.ts +++ b/src/ida/IdaSessionClient.test.ts @@ -9,6 +9,7 @@ import { import { IdaSessionClient } from "./IdaSessionClient.js"; import { functionDossierSchema } from "../domain/hopperValues.js"; import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js"; +import { ProviderAdapterError } from "../domain/providerAdapterError.js"; import { projectAnalysisError } from "../domain/analysisErrorProjection.js"; const roots: string[] = []; @@ -31,6 +32,7 @@ const LIVE_PEER = { const fixture = async ( mode: "attached" | "headless" = "attached", timeoutMs = 1000, + protocol?: "native", ) => { const { root, target } = await createIdaTarget(); roots.push(root); @@ -41,6 +43,7 @@ const fixture = async ( args: [], env: {}, mode, + ...(protocol === undefined ? {} : { protocol }), timeoutMs, workspaceRoot: root, }, @@ -709,6 +712,74 @@ describe("headless IDA non-busy health payloads", () => { }); }); +describe("IDA compatibility profile mismatch", () => { + it("reports the native profile for an incompatible catalog", async () => { + const { producer, client } = await fixture("attached", 1000, "native"); + producer.tools = []; + const result = await client.execute("health", {}); + expect(result.ok).toBe(false); + if (result.ok) throw new Error("Expected incompatible native catalog"); + expect(projectAnalysisError(result.error).details).toMatchObject({ + diagnostics: { + mode: "attached", + expected_profile: "native", + missing_tools: expect.arrayContaining(["decompile"]), + }, + }); + expect(producer.calls).toEqual([]); + expect((await client.close()).ok).toBe(true); + }); + + it("names every missing tool so the caller can identify the upstream profile", async () => { + const { producer, client } = await fixture("headless"); + producer.tools = [ + "idb_open", + "idb_list", + "server_health", + "list_funcs", + "lookup_funcs", + "find_regex", + "decompile", + "disasm", + "callees", + "xrefs_to", + ]; + const result = await client.execute("health", {}); + expect(result.ok, "an incomplete profile cannot serve analysis").toBe( + false, + ); + if (result.ok) return; + const projected = projectAnalysisError(result.error); + expect( + projected.details, + "the absent tool and the expected profile are both reported", + ).toMatchObject({ + provider_id: "ida", + diagnostics: { + mode: "headless", + expected_profile: "database-supervisor", + missing_tools: ["idb_close"], + }, + }); + }); + + it("reports the legacy profile when an attached producer serves modern tools", async () => { + const { producer, client } = await fixture("attached"); + producer.tools = ["server_health", "lookup_funcs", "decompile", "disasm"]; + const result = await client.execute("health", {}); + expect(result.ok, "an incomplete profile cannot serve analysis").toBe( + false, + ); + if (result.ok) return; + expect( + projectAnalysisError(result.error).details, + "attached mode names the legacy profile it expected", + ).toMatchObject({ + diagnostics: { mode: "attached", expected_profile: "legacy-1.4" }, + }); + }); +}); + describe("headless IDA lifecycle boundaries", () => { it("never closes an unowned worker even when open echoes the requested private identity", async () => { const { producer, client } = await fixture("headless"); @@ -1006,6 +1077,7 @@ describe("headless IDA cleanup diagnostics", () => { backend: "worker", owned: false, input_path: producer.inputPath, + is_active: false, }, ), ); @@ -1296,3 +1368,162 @@ describe("headless IDA cleanup verification", () => { expect(await readdir(root)).toEqual(["target.elf"]); }); }); + +it("retains a conflicting requested session ID during pre-close recovery", async () => { + const { producer, client, root } = await fixture("headless"); + producer.overrideCall = (name) => + name === "idb_list" + ? { + sessions: [ + { + session_id: String(producer.database), + input_path: "/conflicting/input", + owned: true, + backend: "worker", + is_active: false, + }, + ], + } + : undefined; + expect((await client.execute("health", {})).ok).toBe(false); + const requested = String(producer.database); + const result = await client.close(); + expect(result).toMatchObject( + reportsSessions({ + session_id: requested, + input_path: "/conflicting/input", + owned: true, + backend: "worker", + is_active: false, + }), + ); + expect(producer.calls.some(({ name }) => name === "idb_close")).toBe(false); + expect( + (await readdir(root)).some((name) => name.startsWith("rea-ida-")), + ).toBe(true); +}); + +it.each(["unavailable", "malformed"] as const)( + "does not invent a worker after explicit rejection and %s inventory", + async (kind) => { + const { producer, client } = await fixture("headless"); + producer.overrideCall = (name) => + name === "idb_open" + ? { error: "rejected before worker creation" } + : name === "idb_list" && kind === "malformed" + ? {} + : undefined; + expect((await client.execute("health", {})).ok).toBe(false); + producer.beforeCall = async (name) => { + if (name === "idb_list" && kind === "unavailable") + throw new Error("inventory unavailable"); + }; + const result = await client.close(); + expect(result.ok).toBe(false); + if (result.ok) throw new Error("Expected retained cleanup"); + expect(result.error).not.toHaveProperty("diagnostics.upstream_sessions"); + }, +); + +it.each(["idb_close", "idb_list", "connection_close"])( + "preserves transport stderr through cleanup at %s", + async (stage) => { + const { producer, client } = await fixture("headless"); + expect((await client.execute("health", {})).ok).toBe(true); + const failure = new ProviderAdapterError("ida", "transport", { + diagnostics: { + reason: "upstream transport closed", + }, + capturedOutput: { + stdout: "", + stderr: "IDA shutdown diagnostic", + truncated: false, + }, + }); + const originalClose = producer.close.bind(producer); + producer.beforeCall = async (name) => { + if (name === stage) throw failure; + }; + if (stage === "connection_close") + producer.close = async () => { + throw failure; + }; + try { + const result = await client.close(); + if (result.ok) throw new Error("expected cleanup failure"); + expect(result.error.capturedOutput?.stderr).toBe( + "IDA shutdown diagnostic", + ); + expect(result.error.diagnostics).toMatchObject({ + failure_reason: "upstream transport closed", + }); + expect(result.error.cleanupIncomplete).toBe(true); + } finally { + delete producer.beforeCall; + producer.close = originalClose; + await client.close(); + } + }, +); + +it("binds retained transport diagnostics to the requested operation and recovery", async () => { + const { producer, client } = await fixture("attached"); + await client.execute("health", {}); + producer.beforeCall = async (name) => { + if (name === "list_functions") + throw new ProviderAdapterError("ida", "transport", { + diagnostics: { reason: "upstream closed" }, + capturedOutput: { + stdout: "", + stderr: "IDA diagnostic", + truncated: false, + }, + }); + }; + const result = await client.execute("list_procedures", {}); + if (result.ok) throw new Error("expected upstream failure"); + expect(projectAnalysisError(result.error)).toMatchObject({ + details: { + operation: "list_procedures", + diagnostics: { + reason: "upstream closed", + recovery: expect.stringContaining("docs/ida-provider.md"), + }, + captured_output: { stderr: "IDA diagnostic" }, + }, + }); + await client.close(); +}); + +it("retains transport diagnostics on a cancellation regardless of stderr", async () => { + const { producer, client } = await fixture("attached"); + await client.execute("health", {}); + const controller = new AbortController(); + producer.beforeCall = async (name) => { + if (name === "list_functions") { + controller.abort(); + throw new ProviderAdapterError("ida", "transport", { + diagnostics: { reason: "upstream closed after cancellation" }, + capturedOutput: { + stdout: "", + stderr: "shutdown diagnostic", + truncated: false, + }, + }); + } + }; + const result = await client.execute( + "list_procedures", + {}, + { signal: controller.signal }, + ); + if (result.ok) throw new Error("expected cancellation"); + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "cancelled", + details: { + diagnostics: { reason: "upstream closed after cancellation" }, + captured_output: { stderr: "shutdown diagnostic" }, + }, + }); + await client.close(); +}); diff --git a/src/ida/IdaSessionClient.ts b/src/ida/IdaSessionClient.ts index dc0ae5d6f..d157b5209 100644 --- a/src/ida/IdaSessionClient.ts +++ b/src/ida/IdaSessionClient.ts @@ -1,8 +1,5 @@ import { z } from "zod"; -import { - ABORTED, - waitForAbortable, -} from "../application/binary/AbortablePromise.js"; +import { ABORTED, waitForAbortable } from "../process/AbortablePromise.js"; import { createAnalysisExecution, type AnalysisClient, @@ -101,6 +98,8 @@ interface UpstreamSession { readonly owned: boolean | null; /** Last inventoried input binding; null until upstream reports it. */ readonly input_path: string | null; + /** Last inventoried activity; null before upstream reports it. */ + readonly is_active: boolean | null; } /** A failed cleanup retains its private workspace and reports the remaining resource. */ @@ -114,10 +113,18 @@ class IdaCleanupError extends ProviderAdapterError { ) { super("ida", "close", { cause, + capturedOutput: + cause instanceof AnalysisError ? cause.capturedOutput : undefined, diagnostics: { + ...(cause instanceof AnalysisError ? cause.diagnostics : undefined), reason: "IDA cleanup could not be verified; the remaining connection or private workspace is retained for retry.", - failure_reason: cause instanceof Error ? cause.message : String(cause), + failure_reason: + cause instanceof AnalysisError + ? (cause.diagnostics?.reason ?? cause.message) + : cause instanceof Error + ? cause.message + : String(cause), workspace: path, ...(sessions === undefined ? {} @@ -125,15 +132,16 @@ class IdaCleanupError extends ProviderAdapterError { upstream_sessions: { state: "unverified", sessions: sessions.map( - ({ session_id, backend, owned, input_path }) => ({ + ({ session_id, backend, owned, input_path, is_active }) => ({ session_id, backend, owned, input_path, + is_active, }), ), reason: - "Release of upstream sessions matching the requested session ID or private input was not confirmed; input_path preserves the last observed binding and is null when no inventory has reported it. A worker outlives its supervisor and may keep running until it finishes and reaches its idle timeout; exiting REA does not stop it. A gui session belongs to an IDA instance REA does not control. An empty session_id means upstream reported none; owned is null when no inventory has reported ownership; an empty list means the requested worker's close was confirmed but no inventory has identified any remaining session.", + "Release of upstream sessions matching the requested session ID or private input was not confirmed; input_path preserves the last observed binding and is null when no inventory has reported it. is_active preserves registry activity and is null when unknown; it does not establish process liveness. A worker outlives its supervisor and may keep running until it finishes and reaches its idle timeout; exiting REA does not stop it. A gui session belongs to an IDA instance REA does not control. An empty session_id means upstream reported none; owned is null when no inventory has reported ownership; an empty list means the requested worker's close was confirmed but no inventory has identified any remaining session.", }, }), }, @@ -378,15 +386,47 @@ export class IdaSessionClient implements AnalysisClient { context: IdaOperationFailureContext, ) { let failure: AnalysisError; - if ( + if (wasCancelled(options)) + failure = new AnalysisCancelledError(operation, { + cause, + ...(cause instanceof AnalysisError + ? { + capturedOutput: cause.capturedOutput, + diagnostics: cause.diagnostics, + partialObservation: cause.partialObservation, + cleanup: + cause.cleanup ?? + (cause.cleanupIncomplete + ? { + reason: cause.message, + resources: cause.cleanupResources, + } + : undefined), + } + : {}), + }); + else if ( + cause instanceof ProviderAdapterError && + cause.operation === "transport" + ) + failure = new ProviderAdapterError("ida", operation, { + cause, + capturedOutput: cause.capturedOutput, + partialObservation: cause.partialObservation, + cleanup: cause.cleanup, + diagnostics: { + ...cause.diagnostics, + recovery: + "Check the configured upstream MCP registration and selected target; see docs/ida-provider.md.", + }, + }); + else if ( cause instanceof AnalysisError && (cause.partialObservation !== undefined || cause.cleanupIncomplete || cause.capturedOutput !== undefined) ) failure = cause; - else if (wasCancelled(options)) - failure = new AnalysisCancelledError(operation); else if (cause instanceof AnalysisError) failure = cause; else if (cause instanceof z.ZodError) failure = new AnalysisOutputError( @@ -483,10 +523,27 @@ export class IdaSessionClient implements AnalysisClient { ? legacyTools : modernTools; const missing = expected.filter((name) => !tools.has(name)); - if (missing.length > 0) - throw new AnalysisProtocolError( - `IDA MCP ${this.config.protocol === "native" ? "native attached" : this.config.mode} compatibility profile is missing tools: ${missing.join(", ")}. Native attached mode requires the native IDA MCP tool catalog; legacy attached mode requires the legacy 1.4 profile; headless mode requires the database supervisor profile. See docs/ida-provider.md.`, - ); + if (missing.length > 0) { + const summary = `IDA MCP ${this.config.protocol === "native" ? "native attached" : this.config.mode} compatibility profile is missing tools: ${missing.join(", ")}. Native attached mode requires the native IDA MCP tool catalog; legacy attached mode requires the legacy 1.4 profile; headless mode requires the database supervisor profile. See docs/ida-provider.md.`; + // The caller cannot act on a profile mismatch it never sees, so the + // missing names travel as adapter diagnostics rather than a bare message. + throw new ProviderAdapterError("ida", "initialize", { + userMessage: summary, + diagnostics: { + reason: summary, + mode: this.config.mode, + expected_profile: + this.config.protocol === "native" + ? "native" + : this.config.mode === "attached" + ? "legacy-1.4" + : "database-supervisor", + missing_tools: [...missing], + recovery: + "Install an IDA MCP build that serves the selected profile; see docs/ida-provider.md.", + }, + }); + } if (this.config.mode === "headless") { this.#workspace = await IdaWorkspace.allocate( this.target, @@ -637,7 +694,12 @@ export class IdaSessionClient implements AnalysisClient { */ #unreleasedSessions(): readonly UpstreamSession[] | undefined { if (this.#blockingSessions !== undefined) return this.#blockingSessions; - if (!this.#openAttempted || this.#workspace === undefined) return undefined; + if ( + !this.#openAttempted || + this.#workspace === undefined || + this.#openRejected + ) + return undefined; return this.#databaseCloseCompleted ? [] : [ @@ -646,6 +708,7 @@ export class IdaSessionClient implements AnalysisClient { backend: "worker", owned: null, input_path: null, + is_active: null, }, ]; } @@ -665,12 +728,15 @@ export class IdaSessionClient implements AnalysisClient { ): boolean { this.#blockingSessions = entries.length > 0 - ? entries.map(({ session_id, backend, owned, input_path }) => ({ - session_id, - backend, - owned, - input_path, - })) + ? entries.map( + ({ session_id, backend, owned, input_path, is_active }) => ({ + session_id, + backend, + owned, + input_path, + is_active, + }), + ) : undefined; return entries.length > 0; } @@ -727,7 +793,7 @@ export class IdaSessionClient implements AnalysisClient { ); if (this.#database === undefined && blocked) throw new AnalysisProtocolError( - "An unowned worker still uses the private workspace; it was retained.", + "A registered session still matches the requested ID or private input without a verified private worker binding; the workspace was retained.", ); if ( this.#database === undefined && diff --git a/src/javascript/analysis/JavaScriptAnalysisWorker.ts b/src/javascript/analysis/JavaScriptAnalysisWorker.ts index 6a587915b..de7ac6c42 100644 --- a/src/javascript/analysis/JavaScriptAnalysisWorker.ts +++ b/src/javascript/analysis/JavaScriptAnalysisWorker.ts @@ -83,6 +83,7 @@ export class JavaScriptAnalysisWorker { readonly #resultBudgetBytes: number; /** Null when the runtime enforces the V8 heap flags instead. */ readonly #processMemoryLimitBytes: number | null; + readonly #onStdinError = (): void => {}; readonly #onStdout = (chunk: Buffer | string): void => { try { this.#pending += @@ -784,20 +785,40 @@ export class JavaScriptAnalysisWorker { if (this.#supervisor !== undefined) { const supervisor = this.#supervisor; const stopped = await supervisor.stop(); + supervisor.launch.process.stdout?.off("data", this.#onStdout); if (stopped.status === "incomplete") { + try { + const stdin = supervisor.launch.process.stdin; + if ( + stdin !== undefined && + stdin !== null && + !stdin.destroyed && + !stdin.writableEnded + ) { + stdin.off("error", this.#onStdinError); + stdin.once("error", this.#onStdinError); + stdin.end(); + } + } catch { + // Cooperative protocol stdin may already be closed or broken. + } + this.#resetState(); pendingCleanup.add(this); return this.#cleanupError(stopped.reason, original); } - supervisor.launch.process.stdout?.off("data", this.#onStdout); supervisor.dispose(); this.#supervisor = undefined; } + this.#resetState(); + return undefined; + } + + #resetState(): void { this.#ready = false; this.#frames = []; this.#pending = ""; this.#decoder = new StringDecoder("utf8"); this.#protocolFailure = undefined; - return undefined; } } diff --git a/src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.ts b/src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.ts index 84a63d79b..40073f1ae 100644 --- a/src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.ts +++ b/src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.ts @@ -157,6 +157,7 @@ const electron = z z.strictObject({ entry_kind: z.enum(["bridge-member", "ipc-handler"]), entry: z.string(), + entry_path: strings.optional(), channel: z.string().nullable(), terminal: z.string(), resolution: z.literal("candidate"), @@ -181,6 +182,13 @@ const electron = z ), }), ) + .transform((paths) => + paths.map(({ entry_kind, entry, entry_path, ...rest }) => + entry_path === undefined + ? { entry_kind, entry, ...rest } + : { entry_kind, entry, entry_path, ...rest }, + ), + ) .optional(), browser_windows: z.array( z.union([ diff --git a/src/javascript/sourceMaps/SourceMapFormat.ts b/src/javascript/sourceMaps/SourceMapFormat.ts index fc3d3e8cd..a2edcab52 100644 --- a/src/javascript/sourceMaps/SourceMapFormat.ts +++ b/src/javascript/sourceMaps/SourceMapFormat.ts @@ -1,6 +1,5 @@ import { z } from "zod"; import { WEB_SOURCE_MAP_LIMITS } from "../../domain/webSourceLocation.js"; -import { hasValidSourceMapContents } from "../../domain/sourceMapContents.js"; import { sourceMapJsonText } from "../../domain/sourceMapEnvelope.js"; import type { EncodedSourceMapXInput } from "@jridgewell/trace-mapping"; @@ -267,23 +266,6 @@ const inspectLeaf = ( consumeRecords(budget, Array.isArray(rawNames) ? rawNames.length : 0); consumeRecords(budget, Array.isArray(rawContents) ? rawContents.length : 0); options.check?.(); - if (options.profile === "browser-collection" && !Array.isArray(rawNames)) - throw new SourceMapFormatFailure( - "format", - "Source-map leaf requires a names array.", - ); - if ( - options.profile === "browser-collection" && - !hasValidSourceMapContents( - Array.isArray(rawSources) ? rawSources.length : 0, - rawContents, - ) - ) - throw new SourceMapFormatFailure( - "format", - "Source-map sourcesContent must align with its source declarations.", - ); - options.check?.(); const leaf = leafSchema.parse(node.value); const layout = inspectMappingLayout(leaf.mappings, options.check); consumeRecords(budget, layout.segments + layout.lines); diff --git a/src/mcpStartupPolicy.ts b/src/mcpStartupPolicy.ts index 280a328c2..dc31114c1 100644 --- a/src/mcpStartupPolicy.ts +++ b/src/mcpStartupPolicy.ts @@ -7,6 +7,8 @@ */ export const MCP_STARTUP_POLICY = { codexStartupTimeoutSeconds: 30, + kimiStartupTimeoutMs: 30_000, + kimiToolTimeoutMs: 120_000, initializeBudgetMs: 15_000, toolsListBudgetMs: 5_000, firstCatalogBudgetMs: 20_000, diff --git a/src/native/CommandRunner.ts b/src/native/CommandRunner.ts index ccd49c570..b5393190c 100644 --- a/src/native/CommandRunner.ts +++ b/src/native/CommandRunner.ts @@ -15,11 +15,18 @@ import type { ProviderProcessSupervisor, SpawnedOwnedProviderProcess, } from "../process/ProviderProcess.js"; +import { boundedUtf8Text } from "../process/boundedUtf8Text.js"; /** Maximum wall-clock duration for one native command. */ export const NATIVE_COMMAND_TIMEOUT_MS = 60_000; /** Retain complete command output up to a bounded aggregate size. */ export const NATIVE_COMMAND_OUTPUT_BUDGET_BYTES = 64 * 1024 * 1024; +/** + * Failure diagnostics keep only an excerpt of each stream. Successful + * analysis needs complete output, but a failed command's error would + * otherwise repeat up to the whole output budget to the caller. + */ +export const NATIVE_FAILURE_DIAGNOSTIC_BYTES = 64 * 1024; export interface NativeCommandCapture { readonly tool: string; @@ -383,18 +390,30 @@ type ProcessCapture = Omit< | "arguments" >; -const failureCapture = ( +/** Bound one command's output for failure diagnostics, keeping observed byte totals. */ +export const failureCapture = ( capture: ProcessCapture, truncated: boolean, -): NativeCommandFailureCapture => ({ - stdout: capture.stdout, - stderr: capture.stderr, - stdoutBytes: capture.stdoutBytes, - stderrBytes: capture.stderrBytes, - exitCode: capture.exitCode, - signal: capture.signal, - truncated, -}); +): NativeCommandFailureCapture => { + const stdout = boundedUtf8Text( + capture.stdout, + NATIVE_FAILURE_DIAGNOSTIC_BYTES, + ); + const stderr = boundedUtf8Text( + capture.stderr, + NATIVE_FAILURE_DIAGNOSTIC_BYTES, + ); + return { + stdout: stdout.text, + stderr: stderr.text, + // Byte counts stay the observed totals, not the excerpt lengths. + stdoutBytes: capture.stdoutBytes, + stderrBytes: capture.stderrBytes, + exitCode: capture.exitCode, + signal: capture.signal, + truncated: truncated || stdout.truncated || stderr.truncated, + }; +}; const captureProcess = async ( executable: string, diff --git a/src/native/NativeMacOSProvider.ts b/src/native/NativeMacOSProvider.ts index 333569e47..f4d06ae84 100644 --- a/src/native/NativeMacOSProvider.ts +++ b/src/native/NativeMacOSProvider.ts @@ -62,6 +62,7 @@ import { NATIVE_COMMAND_OUTPUT_BUDGET_BYTES, NativeCommandFailure, XcrunCommandRunner, + failureCapture, type NativeCommandCleanupFailure, type NativeCommandCapture, type NativeCommandRunner, @@ -997,15 +998,8 @@ const translateCodeSignExitFailure = ( "nonzero-exit", capture.exitCode, undefined, - { - stdout: capture.stdout, - stderr: capture.stderr, - stdoutBytes: capture.stdoutBytes, - stderrBytes: capture.stderrBytes, - exitCode: capture.exitCode, - signal: capture.signal, - truncated: false, - }, + // Accepted output stays complete until the provider decides it failed. + failureCapture(capture, false), ), ); diff --git a/src/native/NativeMachoInspection.ts b/src/native/NativeMachoInspection.ts index bb1966c3d..8453f38a8 100644 --- a/src/native/NativeMachoInspection.ts +++ b/src/native/NativeMachoInspection.ts @@ -266,6 +266,11 @@ const normalizeMacho = ( ]), exports: covered(exports, false, [ "Merged nm/dyld_info results may be incomplete for stripped binaries.", + ...(dyldExports.some(({ sourceSymbol }) => sourceSymbol !== undefined) + ? [ + "Renamed re-exports are listed under their exported name; the source library's symbol name is not reported.", + ] + : []), ]), segments: covered( load.segments.map((segment) => ({ diff --git a/src/native/NativeUiObservation.test.ts b/src/native/NativeUiObservation.test.ts index 527afd8db..0d02693d8 100644 --- a/src/native/NativeUiObservation.test.ts +++ b/src/native/NativeUiObservation.test.ts @@ -35,6 +35,28 @@ const validPng = Buffer.from( "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+jv4sAAAAASUVORK5CYII=", "base64", ); + +/** Screenshot payload as the native helper reports one PNG buffer. */ +const screenshotOf = (bytes: Buffer) => ({ + mime_type: "image/png", + base64: bytes.toString("base64"), + sha256: createHash("sha256").update(bytes).digest("hex"), + width: bytes.readUInt32BE(16), + height: bytes.readUInt32BE(20), +}); + +/** Observe the fixture snapshot carrying one screenshot payload. */ +const observeWithScreenshot = (screenshot: Record) => + observeNativeUi( + target, + "observe_native_ui", + { ...scope, screenshot: true }, + { + environment: {}, + invoke: async () => ({ ok: true, result: { ...snapshot, screenshot } }), + }, + ); + it.each([ { offset: 8, value: 12 }, { offset: 16, value: 0x8000_0000 }, @@ -44,49 +66,16 @@ it.each([ async ({ offset, value }) => { const bytes = Buffer.from(validPng); bytes.writeUInt32BE(value, offset); - const screenshot = { - mime_type: "image/png", - base64: bytes.toString("base64"), - sha256: createHash("sha256").update(bytes).digest("hex"), - width: bytes.readUInt32BE(16), - height: bytes.readUInt32BE(20), - }; - const result = await observeNativeUi( - target, - "observe_native_ui", - { ...scope, screenshot: true }, - { - environment: {}, - invoke: async () => ({ - ok: true, - result: { ...snapshot, screenshot }, - }), - }, - ); + const result = await observeWithScreenshot(screenshotOf(bytes)); expect(result.ok).toBe(false); }, ); describe("native UI screenshot validation", () => { it("rejects malformed or mismatched screenshot bytes from the helper", async () => { - const screenshot = { - mime_type: "image/png", - base64: validPng.toString("base64"), + const result = await observeWithScreenshot({ + ...screenshotOf(validPng), sha256: "0".repeat(64), - width: 1, - height: 1, - }; - const result = await observeNativeUi( - target, - "observe_native_ui", - { ...scope, screenshot: true }, - { - environment: {}, - invoke: async () => ({ - ok: true, - result: { ...snapshot, screenshot }, - }), - }, - ); + }); expect(result.ok).toBe(false); }); @@ -97,26 +86,10 @@ describe("native UI screenshot validation", () => { ])( "rejects invalid screenshot field variants from the helper", async (change) => { - const screenshot = { - mime_type: "image/png", - base64: validPng.toString("base64"), - sha256: createHash("sha256").update(validPng).digest("hex"), - width: 1, - height: 1, + const result = await observeWithScreenshot({ + ...screenshotOf(validPng), ...change, - }; - const result = await observeNativeUi( - target, - "observe_native_ui", - { ...scope, screenshot: true }, - { - environment: {}, - invoke: async () => ({ - ok: true, - result: { ...snapshot, screenshot }, - }), - }, - ); + }); expect(result.ok).toBe(false); }, ); @@ -126,49 +99,12 @@ describe("native UI screenshot validation", () => { const type = bytes.subarray(12, 16); type[0] = (type[0] ?? 0) | 0x80; expect(type.toString("ascii")).toBe("IHDR"); - const screenshot = { - mime_type: "image/png", - base64: bytes.toString("base64"), - sha256: createHash("sha256").update(bytes).digest("hex"), - width: 1, - height: 1, - }; - const result = await observeNativeUi( - target, - "observe_native_ui", - { ...scope, screenshot: true }, - { - environment: {}, - invoke: async () => ({ - ok: true, - result: { ...snapshot, screenshot }, - }), - }, - ); + const result = await observeWithScreenshot(screenshotOf(bytes)); expect(result.ok).toBe(false); }); it("accepts a self-consistent PNG screenshot from the helper", async () => { - const bytes = validPng; - const screenshot = { - mime_type: "image/png", - base64: bytes.toString("base64"), - sha256: createHash("sha256").update(bytes).digest("hex"), - width: 1, - height: 1, - }; - const result = await observeNativeUi( - target, - "observe_native_ui", - { ...scope, screenshot: true }, - { - environment: {}, - invoke: async () => ({ - ok: true, - result: { ...snapshot, screenshot }, - }), - }, - ); + const result = await observeWithScreenshot(screenshotOf(validPng)); expect(result.ok).toBe(true); }); diff --git a/src/native/parsers/dyldInfo.ts b/src/native/parsers/dyldInfo.ts index f08dc8bdd..9742c66a5 100644 --- a/src/native/parsers/dyldInfo.ts +++ b/src/native/parsers/dyldInfo.ts @@ -4,6 +4,8 @@ interface ParsedDyldSymbol { readonly weak: boolean | null; readonly reexport: boolean | null; readonly source: string | null; + /** The source library's name for a symbol it re-exports under another name. */ + readonly sourceSymbol?: string; } /** Decode Apple dyld inventory rows, projecting export offsets to VM addresses. */ @@ -33,15 +35,23 @@ export const parseDyldSymbols = ( ]; return []; } - const reexport = /^\[re-export\]\s+(.+?)(?:\s+\(from (.+)\))?$/u.exec(line); - if (reexport?.[1] !== undefined) + // dyld_info prints `_sym (from lib)`, or `_new (_old from lib)` when the + // source library names the symbol differently. Split the trailing + // parenthesized suffix with string scans so a library path containing + // " from ", or a source symbol containing parentheses such as + // `operator()`, cannot pull the match into nested lazy quantifiers. + const reexport = parseReexportRow(line, knownExportNames); + if (reexport !== null) return [ { - name: reexport[1], + name: reexport.name, address: null, weak: null, reexport: true, - source: reexport[2] ?? null, + source: reexport.source, + ...(reexport.sourceSymbol === undefined + ? {} + : { sourceSymbol: reexport.sourceSymbol }), }, ]; const exported = /^(0x[\da-f]+)\s+(.+?)(?:\s+\[([^\]]+)\])?$/iu.exec(line); @@ -78,3 +88,70 @@ export const parseDyldSymbols = ( const canonicalHex = (value: string | undefined): string | null => value === undefined ? null : `0x${BigInt(value).toString(16)}`; + +interface ReexportRow { + readonly name: string; + readonly source: string | null; + readonly sourceSymbol?: string; +} + +/** + * Split a `[re-export]` row without nested lazy quantifiers. + * + * Returns null when the line is not a re-export row so callers fall through + * to the plain export parser. Known names disambiguate nested parentheses; + * when that inventory is incomplete, retain the suffix-based fallback. + */ +const parseReexportRow = ( + line: string, + knownNames: ReadonlySet, +): ReexportRow | null => { + const prefix = /^\[re-export\]\s+/u.exec(line); + if (prefix === null) return null; + const rest = line.slice(prefix[0].length); + if (rest.length === 0) return null; + if (!rest.endsWith(")")) return { name: rest, source: null }; + const candidates: ReexportRow[] = []; + for (const match of rest.matchAll(/\s+\(/gu)) { + const name = rest.slice(0, match.index); + if (!knownNames.has(name)) continue; + const parsed = reexportSuffix( + name, + rest.slice(match.index + match[0].length, -1), + ); + if (parsed !== null) candidates.push(parsed); + } + // Exact nm identities establish a boundary even when names contain parentheses. + // Competing known interpretations remain unknown rather than creating a new name. + if (candidates.length > 0) + return candidates.length === 1 ? candidates[0]! : null; + if (knownNames.has(rest)) return { name: rest, source: null }; + let depth = 0; + for (let index = rest.length - 1; index >= 0; index--) { + if (rest[index] === ")") depth++; + else if (rest[index] === "(" && --depth === 0) { + if (index === 0 || !/\s/u.test(rest[index - 1]!)) break; + const name = rest.slice(0, index).trimEnd(); + return ( + reexportSuffix(name, rest.slice(index + 1, -1)) ?? { + name: rest, + source: null, + } + ); + } + } + return { name: rest, source: null }; +}; + +const reexportSuffix = (name: string, inner: string): ReexportRow | null => { + if (name.length === 0) return null; + if (inner.startsWith("from ") && inner.length > 5) + return { name, source: inner.slice(5) }; + const separator = /\s+from /u.exec(inner); + if (separator === null) return null; + const sourceSymbol = inner.slice(0, separator.index); + const source = inner.slice(separator.index + separator[0].length); + return sourceSymbol.length > 0 && source.length > 0 + ? { name, source, sourceSymbol } + : null; +}; diff --git a/src/native/parsers/otool.ts b/src/native/parsers/otool.ts index 3e1c3a90f..8f1ed9dda 100644 --- a/src/native/parsers/otool.ts +++ b/src/native/parsers/otool.ts @@ -63,6 +63,56 @@ const DYLIB_FIELDS = [ const DYLIB_ENCODING_FIELDS = ["time stamp", "options"]; const EMPTY_VALUE_FIELDS = new Set(["options", "segname", "sectname"]); +/** + * Non-verbose otool prints LC_BUILD_VERSION platform and tool codes as + * numbers. Name the codes declared in the SDK's , without + * the PLATFORM_/TOOL_ prefix, so build metadata uses the same symbolic style + * as LC_VERSION_MIN_* platforms. Unknown codes keep their printed value. + */ +const BUILD_PLATFORM_NAMES: ReadonlyMap = new Map([ + ["1", "MACOS"], + ["2", "IOS"], + ["3", "TVOS"], + ["4", "WATCHOS"], + ["5", "BRIDGEOS"], + ["6", "MACCATALYST"], + ["7", "IOSSIMULATOR"], + ["8", "TVOSSIMULATOR"], + ["9", "WATCHOSSIMULATOR"], + ["10", "DRIVERKIT"], + ["11", "VISIONOS"], + ["12", "VISIONOSSIMULATOR"], + ["13", "FIRMWARE"], + ["14", "SEPOS"], + ["15", "MACOS_EXCLAVECORE"], + ["16", "MACOS_EXCLAVEKIT"], + ["17", "IOS_EXCLAVECORE"], + ["18", "IOS_EXCLAVEKIT"], + ["19", "TVOS_EXCLAVECORE"], + ["20", "TVOS_EXCLAVEKIT"], + ["21", "WATCHOS_EXCLAVECORE"], + ["22", "WATCHOS_EXCLAVEKIT"], + ["23", "VISIONOS_EXCLAVECORE"], + ["24", "VISIONOS_EXCLAVEKIT"], +]); +const BUILD_TOOL_NAMES: ReadonlyMap = new Map([ + ["1", "CLANG"], + ["2", "SWIFT"], + ["3", "LD"], + ["4", "LLD"], + ["1024", "METAL"], + ["1025", "AIRLLD"], + ["1026", "AIRNT"], + ["1027", "AIRNT_PLUGIN"], + ["1028", "AIRPACK"], + ["1031", "GPUARCHIVER"], + ["1032", "METAL_FRAMEWORK"], +]); +const buildCodeName = ( + names: ReadonlyMap, + value: string | null, +): string | null => (value === null ? null : (names.get(value) ?? value)); + /** * otool emits LF, but a capture replayed from another host or toolchain can * carry CRLF. The section/command boundaries below match a bare `\n`, so a @@ -295,7 +345,10 @@ const collectBuild = ( if (numberField(fields, "ntools") !== tools.length) throw new TypeError("otool build-tool count does not match ntools"); state.builds.push({ - platform: stringField(fields, "platform"), + platform: buildCodeName( + BUILD_PLATFORM_NAMES, + stringField(fields, "platform"), + ), minimum_os: stringField(fields, "minos"), sdk: stringField(fields, "sdk"), tools, @@ -434,7 +487,7 @@ const parseBuildTools = ( const tool = /^tool\s+(.+)$/u.exec(lines[index] ?? "")?.[1]; const version = /^version\s+(.+)$/u.exec(lines[index + 1] ?? "")?.[1]; if (tool !== undefined && version !== undefined) - tools.push({ name: tool, version }); + tools.push({ name: BUILD_TOOL_NAMES.get(tool) ?? tool, version }); } return tools; }; diff --git a/src/native/pwntools/RecordedCrashSourceBindings.test.ts b/src/native/pwntools/RecordedCrashSourceBindings.test.ts new file mode 100644 index 000000000..892fc1049 --- /dev/null +++ b/src/native/pwntools/RecordedCrashSourceBindings.test.ts @@ -0,0 +1,135 @@ +import { describe, expect, it } from "vitest"; + +import { + RECORDED_LINUX_CORE_ARCHITECTURES, + RECORDED_LINUX_CORE_PROFILES, +} from "../../domain/native/recordedCrash.js"; +import { + recordedCrashFixture, + recordedCrashFixtureBytes, +} from "../../../tests/fixtures/binaryDiagnostics/recordedCrash.js"; +import { validateRecordedCrashSources } from "./RecordedCrashSourceBindings.js"; + +const diagnostics = { stdout: "", stderr: "", truncated: false } as const; + +const bind = ( + architecture?: "x86_64-little-endian" | "aarch64-little-endian", +) => { + const report = recordedCrashFixture("/selected.core", architecture); + return { report, bytes: recordedCrashFixtureBytes(report) }; +}; + +describe("recorded core source bindings", () => { + it.each(RECORDED_LINUX_CORE_ARCHITECTURES)( + "binds each %s register to its own source bytes", + (architecture) => { + const { report, bytes } = bind(architecture); + expect(() => + validateRecordedCrashSources(report, bytes, diagnostics), + ).not.toThrow(); + }, + ); + + it("measures the AArch64 register window one slot wide per name", () => { + const profile = RECORDED_LINUX_CORE_PROFILES["aarch64-little-endian"]; + expect(profile.registers.length).toBe(34); + expect(profile.prstatusBytes).toBe(392); + expect(profile.machine).toBe(183); + // The fixtures and the bindings both read these numbers, so they are pinned + // here against the ABI record rather than only used consistently. + expect(profile.registerOffset).toBe(112); + expect(profile.registerBytes).toBe(8); + const amd64 = RECORDED_LINUX_CORE_PROFILES["x86_64-little-endian"]; + expect(amd64.registers.length).toBe(27); + expect(amd64.prstatusBytes).toBe(336); + expect(amd64.machine).toBe(62); + expect(amd64.registerOffset).toBe(112); + expect(amd64.registerBytes).toBe(8); + }); + + it("stamps each crafted core with the machine its profile declares", () => { + for (const architecture of RECORDED_LINUX_CORE_ARCHITECTURES) { + const bytes = bind(architecture).bytes; + expect(bytes.readUInt16LE(18)).toBe( + RECORDED_LINUX_CORE_PROFILES[architecture].machine, + ); + } + }); + + it("rejects an AArch64 register bound to another slot", () => { + const { report, bytes } = bind("aarch64-little-endian"); + const shifted = { + ...report, + threads: report.threads.map((thread) => ({ + ...thread, + registers: thread.registers.map((register, index) => + index === 3 + ? { + ...register, + location: { + ...register.location, + offset: `0x${(BigInt(register.location.offset) + 8n).toString( + 16, + )}`, + }, + } + : register, + ), + })), + }; + expect(() => + validateRecordedCrashSources(shifted, bytes, diagnostics), + ).toThrow(/aarch64-little-endian register identity/); + }); + + it("rejects an AArch64 note shorter than the profile descriptor", () => { + const { report, bytes } = bind("aarch64-little-endian"); + const shortened = { + ...report, + notes: report.notes.map((note, index) => + index === 0 + ? { + ...note, + descriptor_location: { + ...note.descriptor_location, + bytes: "0x150", + }, + } + : note, + ), + }; + expect(() => + validateRecordedCrashSources(shortened, bytes, diagnostics), + ).toThrow(/aarch64-little-endian source bytes/); + }); + + it("rejects an amd64 register name reported for AArch64", () => { + const { report, bytes } = bind("aarch64-little-endian"); + const renamed = { + ...report, + threads: report.threads.map((thread) => ({ + ...thread, + registers: thread.registers.map((register, index) => + index === 0 ? { ...register, name: "rax" } : register, + ), + })), + }; + expect(() => + validateRecordedCrashSources(renamed, bytes, diagnostics), + ).toThrow(/aarch64-little-endian register identity/); + }); + + it("rejects a reported PID that differs from the recorded source bytes", () => { + const { report, bytes } = bind("aarch64-little-endian"); + const wrongPid = { + ...report, + threads: report.threads.map((thread) => ({ + ...thread, + historical_pid: thread.historical_pid + 1, + })), + }; + expect(() => + validateRecordedCrashSources(wrongPid, bytes, diagnostics), + ).toThrow(/aarch64-little-endian source bytes/); + }); +}); diff --git a/src/native/pwntools/RecordedCrashSourceBindings.ts b/src/native/pwntools/RecordedCrashSourceBindings.ts index 4de2555d2..1a08e8721 100644 --- a/src/native/pwntools/RecordedCrashSourceBindings.ts +++ b/src/native/pwntools/RecordedCrashSourceBindings.ts @@ -1,23 +1,26 @@ import { AnalysisOutputError } from "../../domain/analysisErrorCore.js"; import { - RECORDED_LINUX_AMD64_REGISTERS, + RECORDED_LINUX_CORE_PROFILES, type RecordedCrash, } from "../../domain/native/recordedCrash.js"; -const registerOffsets = new Map( - RECORDED_LINUX_AMD64_REGISTERS.map((name, index) => [name, 112 + index * 8]), -); - -/** Bind interpreted values and register identities to Linux amd64 source bytes. */ +/** Bind interpreted values and register identities to recorded source bytes. */ export function validateRecordedCrashSources( report: RecordedCrash, snapshot: Buffer, diagnostics: RecordedCrash["decoder_diagnostics"], ): void { + const profile = RECORDED_LINUX_CORE_PROFILES[report.architecture]; + const registerOffsets = new Map( + profile.registers.map((name, index) => [ + name, + profile.registerOffset + index * profile.registerBytes, + ]), + ); const fail = () => { throw new AnalysisOutputError( "inspect_recorded_crash", - "Reported PID or signal value differs from its recorded Linux amd64 source bytes.", + `Reported PID or signal value differs from its recorded ${report.architecture} source bytes.`, { capturedOutput: diagnostics }, ); }; @@ -29,10 +32,10 @@ export function validateRecordedCrashSources( if (length < minimumBytes) return fail(); return snapshot.subarray(start, start + length); }; - // Fixed offsets independently checked against unchanged pwntools 4.15.0 - // elf_prstatus_amd64 / elf_siginfo_64; this is the sole supported note ABI. + // Offsets come from unchanged pwntools 4.15.0 elf_prstatus_amd64 / + // elf_prstatus_aarch64 and elf_siginfo_64 through the shared domain profile. for (const thread of report.threads) { - const bytes = descriptor(thread.note_index, 336); + const bytes = descriptor(thread.note_index, profile.prstatusBytes); if ( thread.historical_pid !== bytes.readInt32LE(32) || thread.recorded_current_signal !== bytes.readInt16LE(12) @@ -49,7 +52,7 @@ export function validateRecordedCrashSources( ) throw new AnalysisOutputError( "inspect_recorded_crash", - "Reported register source does not match its Linux amd64 register identity.", + `Reported register source does not match its ${report.architecture} register identity.`, { capturedOutput: diagnostics }, ); } diff --git a/src/native/pwntools/RecordedCrashStructureBindings.ts b/src/native/pwntools/RecordedCrashStructureBindings.ts index 3d1cacce6..c1e2790b2 100644 --- a/src/native/pwntools/RecordedCrashStructureBindings.ts +++ b/src/native/pwntools/RecordedCrashStructureBindings.ts @@ -1,5 +1,8 @@ import { AnalysisOutputError } from "../../domain/analysisErrorCore.js"; -import type { RecordedCrash } from "../../domain/native/recordedCrash.js"; +import { + RECORDED_LINUX_CORE_PROFILES, + type RecordedCrash, +} from "../../domain/native/recordedCrash.js"; // Exact amd64/core enum projections in the pinned pyelftools 0.33 profile. const segmentKinds = new Map([ @@ -81,6 +84,8 @@ export function validateRecordedCrashStructure( return snapshot.subarray(Number(offset), Number(offset) + bytes); }; const header = bytesAt(0n, 64); + const expectedMachine = + RECORDED_LINUX_CORE_PROFILES[report.architecture].machine; if ( !header .subarray(0, 6) @@ -90,7 +95,7 @@ export function validateRecordedCrashStructure( ![0, 3].includes(header[7] ?? -1) || header[8] !== 0 || header.readUInt16LE(16) !== 4 || - header.readUInt16LE(18) !== 62 + header.readUInt16LE(18) !== expectedMachine ) fail(); const table = header.readBigUInt64LE(32); diff --git a/src/application/binary/AbortablePromise.ts b/src/process/AbortablePromise.ts similarity index 100% rename from src/application/binary/AbortablePromise.ts rename to src/process/AbortablePromise.ts diff --git a/src/process/ArtifactAnalysisResources.ts b/src/process/ArtifactAnalysisResources.ts index b0083e05b..ae3d9f803 100644 --- a/src/process/ArtifactAnalysisResources.ts +++ b/src/process/ArtifactAnalysisResources.ts @@ -33,7 +33,10 @@ export type ArtifactAnalysisRuntime = Pick< /** Own admission, artifact descriptors, producer cleanup and private workspaces together. */ export class ArtifactAnalysisResources { - readonly #artifacts = new ArtifactResourceScope(); + readonly #artifacts: { + readonly scope: ArtifactResourceScope; + operation: string; + }; readonly #pending = new Map< ArtifactAnalysisRuntime, { @@ -54,7 +57,9 @@ export class ArtifactAnalysisResources { readonly message: string; readonly details: Readonly>; }, - ) {} + ) { + this.#artifacts = { scope: new ArtifactResourceScope(), operation }; + } run( work: () => Promise>, @@ -76,8 +81,15 @@ export class ArtifactAnalysisResources { ); this.#busy = true; const running = this.#tail - .then(() => this.#artifacts.run(work)) - .catch((cause: unknown) => err(this.#artifactFailure(cause, operation))) + .then(() => + this.#artifacts.scope.run(() => { + // Failed descriptor cleanup still belongs to the preceding request until + // this scope has drained it and admits acquisition for the new request. + this.#artifacts.operation = operation; + return work(); + }), + ) + .catch((cause: unknown) => err(this.#artifactFailure(cause))) .finally(() => { this.#busy = false; }); @@ -94,20 +106,23 @@ export class ArtifactAnalysisResources { maximumBytes, signal, undefined, - this.#artifacts, + this.#artifacts.scope, ); } readLimits(limits: LinuxResourceLimits): Promise { - return this.readKernelLimits(limits, this.#artifacts); + return this.readKernelLimits(limits, this.#artifacts.scope); } retain( root: ArtifactAnalysisRuntime, owner?: ProviderProcessSupervisor, - operation = this.operation, ): void { - this.#pending.set(root, { owner, operation }); + this.#pending.set(root, { + owner, + operation: + this.#pending.get(root)?.operation ?? this.#artifacts.operation, + }); } get pendingPaths(): readonly string[] { @@ -251,7 +266,7 @@ export class ArtifactAnalysisResources { async #closeResources(): Promise { let descriptorFailure: AnalysisError | undefined; try { - await this.#artifacts.close(); + await this.#artifacts.scope.close(); } catch (cause: unknown) { descriptorFailure = this.#artifactFailure(cause); } @@ -262,14 +277,17 @@ export class ArtifactAnalysisResources { throw analysisErrorWithCleanupFailure( descriptorFailure, cause, - this.operation, + this.#artifacts.operation, ); throw cause; } if (descriptorFailure !== undefined) throw descriptorFailure; } - #artifactFailure(cause: unknown, operation = this.operation): AnalysisError { + #artifactFailure( + cause: unknown, + operation = this.#artifacts.operation, + ): AnalysisError { if (cause instanceof AnalysisError) return cause; if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined) return this.cleanupFailure(cause.cleanup, cause, undefined, operation); diff --git a/src/process/LinuxResourceLimits.test.ts b/src/process/LinuxResourceLimits.test.ts index 69a584d74..0b19f773e 100644 --- a/src/process/LinuxResourceLimits.test.ts +++ b/src/process/LinuxResourceLimits.test.ts @@ -1,5 +1,9 @@ import { expect, it } from "vitest"; -import { clampLinuxResourceLimits } from "./LinuxResourceLimits.js"; +import { + BOUNDED_PRODUCER_LIMITS, + clampLinuxResourceLimits, + prlimitArgumentVector, +} from "./LinuxResourceLimits.js"; const configured = { address_space_bytes: 1073741824, file_bytes: 67108864, @@ -41,3 +45,38 @@ it("refuses missing, ambiguous or malformed kernel limits before launch", () => ]) expect(() => clampLinuxResourceLimits(configured, value)).toThrow(); }); +it("pins the shared producer budget in one place", () => { + expect(BOUNDED_PRODUCER_LIMITS).toEqual({ + input_bytes: 33554432, + output_bytes: 4194304, + command_timeout_ms: 30000, + address_space_bytes: 1073741824, + file_bytes: 67108864, + cpu_seconds: 30, + }); +}); +it("pairs each effective ceiling with itself before the producer argv", () => { + expect( + prlimitArgumentVector(BOUNDED_PRODUCER_LIMITS, "/bin/producer"), + ).toEqual([ + "--as=1073741824:1073741824", + "--fsize=67108864:67108864", + "--cpu=30:30", + "--core=0:0", + "--", + "/bin/producer", + ]); + expect( + prlimitArgumentVector( + clampLinuxResourceLimits(BOUNDED_PRODUCER_LIMITS, report("10 20")), + "/bin/producer", + ), + ).toEqual([ + "--as=10:10", + "--fsize=67108864:67108864", + "--cpu=30:30", + "--core=0:0", + "--", + "/bin/producer", + ]); +}); diff --git a/src/process/LinuxResourceLimits.ts b/src/process/LinuxResourceLimits.ts index e6eefcefe..414eb13bb 100644 --- a/src/process/LinuxResourceLimits.ts +++ b/src/process/LinuxResourceLimits.ts @@ -15,6 +15,29 @@ const rows = { cpu_seconds: "Max cpu time", } as const; +/** Bound the resident input copy and retained text before allocation/collection. */ +export const BOUNDED_PRODUCER_LIMITS = { + input_bytes: 32 * 1024 * 1024, + output_bytes: 4 * 1024 * 1024, + command_timeout_ms: 30_000, + address_space_bytes: 1024 * 1024 * 1024, + file_bytes: 64 * 1024 * 1024, + cpu_seconds: 30, +} as const; + +/** Pin soft and hard ceilings together, then hand the producer argv to the limiter. */ +export const prlimitArgumentVector = ( + limits: LinuxResourceLimits, + command: string, +): string[] => [ + `--as=${limits.address_space_bytes}:${limits.address_space_bytes}`, + `--fsize=${limits.file_bytes}:${limits.file_bytes}`, + `--cpu=${limits.cpu_seconds}:${limits.cpu_seconds}`, + "--core=0:0", + "--", + command, +]; + /** Never raise either inherited soft policy or hard ceiling. */ export const clampLinuxResourceLimits = ( configured: LinuxResourceLimits, diff --git a/src/process/ProviderDeadline.test.ts b/src/process/ProviderDeadline.test.ts index b43ce6654..ca83d98ab 100644 --- a/src/process/ProviderDeadline.test.ts +++ b/src/process/ProviderDeadline.test.ts @@ -33,3 +33,171 @@ it("aborts a bounded wait when its real monotonic deadline expires", async () => deadline.dispose(); } }); + +it("does not expire a future monotonic deadline on an early timer callback", async () => { + vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] }); + let monotonic = 0; + vi.spyOn(performance, "now").mockImplementation(() => monotonic); + const deadline = new ProviderStartupDeadline(40); + try { + monotonic = 39; + await vi.advanceTimersByTimeAsync(40); + expect(deadline.remainingMs()).toBe(1); + expect(deadline.signal.aborted).toBe(false); + expect(deadline.interruption).toBeUndefined(); + } finally { + deadline.dispose(); + vi.useRealTimers(); + } +}); + +it("aborts when the re-armed timer reaches the absolute deadline", async () => { + vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] }); + let monotonic = 0; + vi.spyOn(performance, "now").mockImplementation(() => monotonic); + const deadline = new ProviderStartupDeadline(40); + try { + monotonic = 39; + await vi.advanceTimersByTimeAsync(40); + expect(deadline.signal.aborted).toBe(false); + monotonic = 40; + await vi.advanceTimersByTimeAsync(1); + expect(deadline.signal.aborted).toBe(true); + expect(deadline.interruption).toBe("timeout"); + expect(deadline.remainingMs()).toBe(0); + } finally { + deadline.dispose(); + vi.useRealTimers(); + } +}); + +it("reports a wait that resolves at the deadline as aborted, not elapsed", async () => { + vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] }); + let monotonic = 0; + vi.spyOn(performance, "now").mockImplementation(() => monotonic); + const deadline = new ProviderStartupDeadline(40); + try { + const pending = deadline.wait(500); + monotonic = 40; + await vi.advanceTimersByTimeAsync(40); + await expect(pending).resolves.toBe("aborted"); + } finally { + deadline.dispose(); + vi.useRealTimers(); + } +}); + +it("resolves a polling interval shorter than the budget as elapsed", async () => { + vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] }); + let monotonic = 0; + vi.spyOn(performance, "now").mockImplementation(() => monotonic); + const deadline = new ProviderStartupDeadline(40); + try { + const pending = deadline.wait(10); + monotonic = 10; + await vi.advanceTimersByTimeAsync(10); + await expect(pending).resolves.toBe("elapsed"); + expect(deadline.signal.aborted).toBe(false); + expect(deadline.remainingMs()).toBe(30); + } finally { + deadline.dispose(); + vi.useRealTimers(); + } +}); + +it("keeps caller cancellation classified without reporting an exhausted budget", () => { + vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] }); + let monotonic = 0; + vi.spyOn(performance, "now").mockImplementation(() => monotonic); + const caller = new AbortController(); + const deadline = new ProviderStartupDeadline(40, caller.signal); + try { + monotonic = 20; + caller.abort(new Error("caller stopped startup")); + expect(deadline.signal.aborted).toBe(true); + expect(deadline.interruption).toBe("cancelled"); + expect(deadline.remainingMs()).toBe(20); + } finally { + deadline.dispose(); + vi.useRealTimers(); + } +}); + +// Probe from issue #2760 review: both the abort timer and the clamped polling +// timer can fire early at the same monotonic instant. The full-budget wait must +// stay pending until the absolute deadline expires. +it("keeps a full-budget wait pending when both timers fire early", async () => { + vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] }); + let monotonic = 0; + vi.spyOn(performance, "now").mockImplementation(() => monotonic); + const caller = new AbortController(); + const deadline = new ProviderStartupDeadline(40, caller.signal); + let outcome: string | undefined; + const pending = deadline.wait(500).then((result) => { + outcome = result; + return result; + }); + try { + monotonic = 39; + await vi.advanceTimersByTimeAsync(40); + expect(deadline.signal.aborted).toBe(false); + expect(deadline.remainingMs()).toBe(1); + expect(outcome).toBeUndefined(); + monotonic = 40; + await vi.advanceTimersByTimeAsync(1); + await expect(pending).resolves.toBe("aborted"); + expect(deadline.remainingMs()).toBe(0); + } finally { + caller.abort(); + deadline.dispose(); + vi.useRealTimers(); + } +}); + +it("releases the timer and listener on dispose with a pending wait", async () => { + vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] }); + let monotonic = 0; + vi.spyOn(performance, "now").mockImplementation(() => monotonic); + const caller = new AbortController(); + const deadline = new ProviderStartupDeadline(40, caller.signal); + try { + const pending = deadline.wait(500); + deadline.dispose(); + // The external abort listener is removed: caller cancellation after + // dispose must not abort the deadline or classify it as cancelled. + caller.abort(new Error("caller stopped startup")); + expect(deadline.signal.aborted).toBe(false); + expect(deadline.interruption).toBeUndefined(); + monotonic = 100; + await vi.advanceTimersByTimeAsync(100); + expect(deadline.signal.aborted).toBe(false); + // The deadline expired while disposed, so the clamped wait resolves + // "aborted" from the monotonic clock instead of polling past expiry. + await expect(pending).resolves.toBe("aborted"); + } finally { + deadline.dispose(); + vi.useRealTimers(); + } +}); + +// Preserve the authored fractional-clock control while following the current +// owning deadline's re-arm contract for an early native timer callback. +it("reports zero after the re-armed deadline reaches a fractional early callback", async () => { + vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] }); + const now = vi.spyOn(performance, "now").mockReturnValue(1_000); + const deadline = new ProviderStartupDeadline(40); + try { + now.mockReturnValue(1_039.6); + await vi.advanceTimersByTimeAsync(40); + expect(deadline.signal.aborted).toBe(false); + expect(deadline.remainingMs()).toBe(1); + now.mockReturnValue(1_040); + await vi.advanceTimersByTimeAsync(1); + expect(deadline.signal.aborted).toBe(true); + expect(deadline.interruption).toBe("timeout"); + expect(deadline.remainingMs()).toBe(0); + } finally { + deadline.dispose(); + vi.useRealTimers(); + } +}); diff --git a/src/process/ProviderDeadline.ts b/src/process/ProviderDeadline.ts index 33cb3a8d0..7681ecf62 100644 --- a/src/process/ProviderDeadline.ts +++ b/src/process/ProviderDeadline.ts @@ -61,13 +61,7 @@ export class ProviderStartupDeadline { return; } signal?.addEventListener("abort", this.#onExternalAbort, { once: true }); - this.#timer = setTimeout(() => { - if (this.#controller.signal.aborted) return; - this.#interruption = "timeout"; - this.#abort( - new DOMException("Provider startup deadline elapsed", "TimeoutError"), - ); - }, timeoutMs); + this.#armDeadlineTimer(timeoutMs); } /** Composite signal aborted by either the caller or the absolute deadline. */ @@ -85,15 +79,42 @@ export class ProviderStartupDeadline { /** Milliseconds still available to the next startup phase. */ remainingMs(): number { + // Node timers can fire a fraction of a millisecond before the monotonic + // deadline, so an elapsed timer must not report time left to spend. + if (this.#interruption === "timeout") return 0; return Math.max(0, Math.ceil(this.#deadlineAt - performance.now())); } - /** Wait for at most one polling interval under the shared deadline signal. */ + /** + * Wait for at most one polling interval under the shared deadline signal. + * + * Both timers may fire before their monotonic targets. Retain the earlier + * of the polling target and deadline, and wait again for any residual time. + */ wait(milliseconds: number): Promise { - return waitForAbortableDelay( - Math.min(milliseconds, this.remainingMs()), - this.signal, - ); + const startedAt = performance.now(); + // Absolute target: the earlier of the requested polling interval and the + // absolute monotonic deadline. + const targetAt = Math.min(startedAt + milliseconds, this.#deadlineAt); + const poll = (): Promise => { + const budget = this.remainingMs(); + const residual = Math.max(0, targetAt - performance.now()); + return waitForAbortableDelay( + Math.min(residual, budget), + this.signal, + ).then((result) => { + if (result === "aborted") return "aborted"; + const now = performance.now(); + // The monotonic deadline expired: a full-budget wait resolves as an + // abort so polling stops at the boundary. + if (now >= this.#deadlineAt) return "aborted"; + // The polling interval elapsed without expiring the deadline. + if (now >= targetAt) return "elapsed"; + // A timer fired before the absolute target; keep waiting. + return poll(); + }); + }; + return poll(); } /** Release the deadline timer and external abort listener. */ @@ -112,4 +133,25 @@ export class ProviderStartupDeadline { if (this.#timer !== undefined) clearTimeout(this.#timer); this.#timer = undefined; } + + /** + * Arm (or re-arm) the timer that expires the absolute monotonic deadline. + * + * Re-check the clock before classifying expiry, and retain the residual + * budget when the timer arrives early. + */ + #armDeadlineTimer(milliseconds: number): void { + this.#timer = setTimeout(() => { + this.#timer = undefined; + if (this.#controller.signal.aborted) return; + if (performance.now() >= this.#deadlineAt) { + this.#interruption = "timeout"; + this.#abort( + new DOMException("Provider startup deadline elapsed", "TimeoutError"), + ); + return; + } + this.#armDeadlineTimer(this.remainingMs()); + }, milliseconds); + } } diff --git a/src/process/ProviderProcess.ts b/src/process/ProviderProcess.ts index d7b95613f..8e650898b 100644 --- a/src/process/ProviderProcess.ts +++ b/src/process/ProviderProcess.ts @@ -61,6 +61,8 @@ export interface ProviderProcessHandle extends Pick< /** True once an enforced process memory limit reported an allocation failure. */ readonly memoryLimitReached?: boolean; kill(signal?: NodeJS.Signals | number): boolean; + unref?(): void; + ref?(): void; } /** Process handle returned by a provider-specific launcher. */ @@ -371,13 +373,33 @@ export class ProviderProcessSupervisor { this.launch.process.off("error", this.#onError); } + /** Release event-loop references so incomplete cleanup does not prevent host exit. */ + unref(): void { + const process = this.launch.process; + process.unref?.(); + (process.stdin as { unref?: () => void } | null | undefined)?.unref?.(); + (process.stdout as { unref?: () => void } | null | undefined)?.unref?.(); + (process.stderr as { unref?: () => void } | null | undefined)?.unref?.(); + } + + /** Restore event-loop references when retrying supervised process cleanup. */ + ref(): void { + const process = this.launch.process; + process.ref?.(); + (process.stdin as { ref?: () => void } | null | undefined)?.ref?.(); + (process.stdout as { ref?: () => void } | null | undefined)?.ref?.(); + (process.stderr as { ref?: () => void } | null | undefined)?.ref?.(); + } + async #stop( options: ProviderProcessStopOptions, ): Promise { + this.ref(); let completed = false; try { const result = await this.#stopWithin(options); completed = result.status !== "incomplete"; + if (!completed) this.unref(); return result; } finally { if (completed) this.dispose(); diff --git a/src/process/ProviderProcessOutput.test.ts b/src/process/ProviderProcessOutput.test.ts index 28dc87dff..3ca2621f0 100644 --- a/src/process/ProviderProcessOutput.test.ts +++ b/src/process/ProviderProcessOutput.test.ts @@ -121,3 +121,40 @@ it("reports a reached process memory limit only when the handle observed one", ( supervisor.dispose(); } }); + +it("unreferences process and stream handles when stop reports incomplete cleanup", async () => { + const unrefProcess = vi.fn(); + const unrefStdin = vi.fn(); + const unrefStdout = vi.fn(); + const unrefStderr = vi.fn(); + const producer = Object.assign(new EventEmitter(), { + stdout: Object.assign(new PassThrough(), { unref: unrefStdout }), + stdin: Object.assign(new PassThrough(), { unref: unrefStdin }), + stderr: Object.assign(new PassThrough(), { unref: unrefStderr }), + exitCode: null, + signalCode: null, + kill: () => false, + unref: unrefProcess, + }); + const supervisor = new ProviderProcessSupervisor({ + process: producer, + ownsProcessLifetime: true, + cleanup: async () => ({ + cleaned: false, + reason: "process table could not be inspected: spawn EPERM", + }), + }); + try { + const result = await supervisor.stop(); + expect(result).toMatchObject({ + status: "incomplete", + reason: "process table could not be inspected: spawn EPERM", + }); + expect(unrefProcess).toHaveBeenCalledTimes(1); + expect(unrefStdin).toHaveBeenCalledTimes(1); + expect(unrefStdout).toHaveBeenCalledTimes(1); + expect(unrefStderr).toHaveBeenCalledTimes(1); + } finally { + supervisor.dispose(); + } +}); diff --git a/src/server/EvidenceMcpServer.ts b/src/server/EvidenceMcpServer.ts index 2691ec422..f42fe9de6 100644 --- a/src/server/EvidenceMcpServer.ts +++ b/src/server/EvidenceMcpServer.ts @@ -278,8 +278,13 @@ class ErrorEvidenceTransport implements Transport { // Count the provenance marker exactly as the projected error did, and keep // it on the replacement: the retained original may hold target content. const targetContent = carriesTargetContentMarker(message.result); - // The delivered error omits structuredContent, so only its text counts. - const encoded = encodeToolResult(structured, budget, "text", targetContent); + // The delivered error omits structuredContent, so its text and error flag count. + const encoded = encodeToolResult( + structured, + budget, + "error-text", + targetContent, + ); if (encoded.ok) return message; const operation = tool ?? "mcp_tool_error"; const originalError = analysisErrorProjectionSchema.safeParse( diff --git a/src/server/registerBundleComparisonTool.ts b/src/server/registerBundleComparisonTool.ts index 75ef2a618..3dbc92298 100644 --- a/src/server/registerBundleComparisonTool.ts +++ b/src/server/registerBundleComparisonTool.ts @@ -1,7 +1,7 @@ import type { EvidenceMcpServer } from "./EvidenceMcpServer.js"; import type { BinarySessionPort } from "../application/binary/BinarySessionPort.js"; -import { readEvidenceBundle } from "../application/EvidenceBundleFiles.js"; +import { readImmutableEvidenceBundle } from "../application/EvidenceBundleFiles.js"; import { toolContract } from "../contracts/toolContracts.js"; import { compareBundles } from "../domain/bundleComparison.js"; import { createEvidence } from "../domain/evidence.js"; @@ -33,12 +33,12 @@ export const registerBundleComparisonTool = ( context.mcpReq.signal, async () => { const [left, right] = await Promise.all([ - readEvidenceBundle( + readImmutableEvidenceBundle( input.left_bundle_path, context.mcpReq.signal, resources, ), - readEvidenceBundle( + readImmutableEvidenceBundle( input.right_bundle_path, context.mcpReq.signal, resources, diff --git a/src/server/registerWasmTools.ts b/src/server/registerWasmTools.ts index 494972b2f..a7a056cc7 100644 --- a/src/server/registerWasmTools.ts +++ b/src/server/registerWasmTools.ts @@ -1,3 +1,7 @@ +import { compareWasmInterfaces } from "../application/wasm/WasmInterfaceComparisonService.js"; +import { COMPARE_WASM_INTERFACES_CONTRACT } from "../contracts/wasm/wasmComparisonContracts.js"; +import { resolvePairedEvidenceRequest } from "../application/EvidenceInputResolver.js"; +import { recordSessionEvidenceSources } from "./sessionEvidence.js"; import type { EvidenceMcpServer } from "./EvidenceMcpServer.js"; import type { WasmArtifactService } from "../application/wasm/WasmArtifactService.js"; @@ -21,6 +25,38 @@ export const registerWasmTools = ( withAdmittedAnalysis?: WithAdmittedAnalysis, evidenceById?: EvidenceLookup, ): void => { + const comparison = COMPARE_WASM_INTERFACES_CONTRACT; + const compareHandler = createEvidenceToolHandler( + server, + async (rawInput, options) => { + const input = comparison.inputSchema.safeParse(rawInput); + if (!input.success) + return err( + analysisInputErrorFromIssues( + comparison.name, + input.error.issues, + rawInput, + ), + ); + const sources = resolvePairedEvidenceRequest(input.data, evidenceById); + if (!sources.ok) return sources; + const result = await compareWasmInterfaces(sources.value, options); + if (!result.ok) return result; + const recorded = recordSessionEvidenceSources(recordEvidence, [ + sources.value.left, + sources.value.right, + ]); + return recorded.ok ? result : recorded; + }, + logger, + recordEvidence, + withAdmittedAnalysis, + ); + server.registerTool( + comparison.name, + toolRegistrationOptions(comparison), + compareHandler(comparison), + ); const contract = wasmArtifactModule.contract; const handler = createEvidenceToolHandler( server, diff --git a/src/server/toolResult.ts b/src/server/toolResult.ts index b503b5677..5d40afb5b 100644 --- a/src/server/toolResult.ts +++ b/src/server/toolResult.ts @@ -139,7 +139,7 @@ export class ToolResultDelivery { const encoded = encodeToolResult( structuredContent, this.resultBudgetBytes, - "text", + "error-text", targetContent, ); // The transport retains an oversized projected error before replacing it diff --git a/src/server/toolResultEncoding.test.ts b/src/server/toolResultEncoding.test.ts index b6969c306..1e9aee5ac 100644 --- a/src/server/toolResultEncoding.test.ts +++ b/src/server/toolResultEncoding.test.ts @@ -48,21 +48,27 @@ describe("MCP result encoding budget", () => { it.each([ { error: { code: "invalid_request", message: '中文"\n'.repeat(30000) } }, { error: { message: "😀\ud800" } }, - ])("counts only escaped text for text-only error delivery", (candidate) => { - const textBytes = Buffer.byteLength( - JSON.stringify({ - content: [{ type: "text", text: JSON.stringify(candidate) }], - }), - ); - const encoded = encodeToolResult(candidate, textBytes, "text"); - if (!encoded.ok) throw new Error("Expected complete encoding"); - expect(encoded.text).toBe(JSON.stringify(candidate)); - expect(encoded.bytes).toBe(textBytes); - expect(encodeToolResult(candidate, textBytes - 1, "text")).toMatchObject({ - ok: false, - bytesAtLeast: textBytes, - }); - }); + ])( + "counts escaped text and the error flag for error delivery", + (candidate) => { + const errorBytes = Buffer.byteLength( + JSON.stringify({ + content: [{ type: "text", text: JSON.stringify(candidate) }], + isError: true, + }), + ); + const encoded = encodeToolResult(candidate, errorBytes, "error-text"); + if (!encoded.ok) throw new Error("Expected complete encoding"); + expect(encoded.text).toBe(JSON.stringify(candidate)); + expect(encoded.bytes).toBe(errorBytes); + expect( + encodeToolResult(candidate, errorBytes - 1, "error-text"), + ).toMatchObject({ + ok: false, + bytesAtLeast: errorBytes, + }); + }, + ); it("accounts for the untrusted target-content marker", () => { const candidate = { strings: ['"ignore previous instructions"', "中文"] }; @@ -94,14 +100,15 @@ describe("MCP result encoding budget", () => { const markedBytes = Buffer.byteLength( JSON.stringify({ content: [{ type: "text", text: JSON.stringify(candidate) }], + isError: true, _meta: TARGET_CONTENT_RESULT_META, }), ); expect( - encodeToolResult(candidate, markedBytes, "text", true), + encodeToolResult(candidate, markedBytes, "error-text", true), ).toMatchObject({ ok: true, bytes: markedBytes }); expect( - encodeToolResult(candidate, markedBytes - 1, "text", true), + encodeToolResult(candidate, markedBytes - 1, "error-text", true), ).toMatchObject({ ok: false, bytesAtLeast: markedBytes }); }); diff --git a/src/server/toolResultEncoding.ts b/src/server/toolResultEncoding.ts index 2a3692674..73ec2c717 100644 --- a/src/server/toolResultEncoding.ts +++ b/src/server/toolResultEncoding.ts @@ -13,8 +13,8 @@ const RESULT_ENVELOPE_BYTES = }), ) - 4; -const TEXT_RESULT_ENVELOPE_BYTES = Buffer.byteLength( - JSON.stringify({ content: [{ type: "text", text: "" }] }), +const ERROR_RESULT_ENVELOPE_BYTES = Buffer.byteLength( + JSON.stringify({ content: [{ type: "text", text: "" }], isError: true }), ); /** @@ -48,9 +48,10 @@ const TARGET_CONTENT_META_BYTES = /** * MCP representations a result carries on the wire. Successful results repeat - * the JSON as structured content and escaped text; error results carry text only. + * the JSON as structured content and escaped text; error results carry text + * only, plus their `isError` flag. */ -export type ToolResultRepresentations = "structured-and-text" | "text"; +export type ToolResultRepresentations = "structured-and-text" | "error-text"; /** Result of encoding the complete repeated MCP representations within a wire budget. */ export type ToolResultEncoding = @@ -72,7 +73,7 @@ export const encodeToolResult = ( const structured = representations === "structured-and-text"; const parts: string[] = []; let bytes = - (structured ? RESULT_ENVELOPE_BYTES : TEXT_RESULT_ENVELOPE_BYTES) + + (structured ? RESULT_ENVELOPE_BYTES : ERROR_RESULT_ENVELOPE_BYTES) + (targetContent ? TARGET_CONTENT_META_BYTES : 0); let codeUnits = bytes; for (const part of bufferedJsonParts(jsonParts(value))) { diff --git a/src/signals/SigrokRecordingProvider.ts b/src/signals/SigrokRecordingProvider.ts index 2e903d4c6..541138a6a 100644 --- a/src/signals/SigrokRecordingProvider.ts +++ b/src/signals/SigrokRecordingProvider.ts @@ -7,6 +7,8 @@ import { parseDecoderDiscovery, } from "./parseDecoderDiscovery.js"; import { + BOUNDED_PRODUCER_LIMITS as LIMITS, + prlimitArgumentVector, readLinuxResourceLimits, type LinuxResourceLimits, } from "../process/LinuxResourceLimits.js"; @@ -59,15 +61,6 @@ export const SIGROK_PROVIDER_IDENTITY = { name: "REA sigrok recording adapter", version: "sigrok-cli@0.7.2", } as const; -// Bound the resident input copy and retained text before allocation/collection. -const LIMITS = { - input_bytes: 32 * 1024 * 1024, - output_bytes: 4 * 1024 * 1024, - command_timeout_ms: 30_000, - address_space_bytes: 1024 * 1024 * 1024, - file_bytes: 64 * 1024 * 1024, - cpu_seconds: 30, -} as const; type Launcher = NonNullable[2]>["launcher"]; /** Decode selected recordings only, retaining cleanup ownership across failures. */ @@ -163,17 +156,10 @@ export class SigrokRecordingProvider implements SignalRecordingPort { options?.signal, ); effectiveLimits = await this.#resources.readLimits(LIMITS); - const limitArguments = [ - `--as=${effectiveLimits.address_space_bytes}:${effectiveLimits.address_space_bytes}`, - `--fsize=${effectiveLimits.file_bytes}:${effectiveLimits.file_bytes}`, - `--cpu=${effectiveLimits.cpu_seconds}:${effectiveLimits.cpu_seconds}`, - "--core=0:0", - "--", - command, - ]; + const limitArguments = prlimitArgumentVector(effectiveLimits, command); phase = "workspace"; root = await this.createRuntime(); - this.#resources.retain(root, undefined, operation); + this.#resources.retain(root, undefined); const snapshotPath = join(root.path, "recording"); const execute = async (args: readonly string[]) => { const output = await runOwnedCommand( @@ -371,7 +357,7 @@ export class SigrokRecordingProvider implements SignalRecordingPort { cause instanceof OwnedCommandFailure && cause.cleanupOwner !== undefined ) - this.#resources.retain(root, cause.cleanupOwner, operation); + this.#resources.retain(root, cause.cleanupOwner); result = err( this.#failure( cause, diff --git a/src/sqlite/SqliteDatabaseInspection.ts b/src/sqlite/SqliteDatabaseInspection.ts index 510fbaafc..941634dd5 100644 --- a/src/sqlite/SqliteDatabaseInspection.ts +++ b/src/sqlite/SqliteDatabaseInspection.ts @@ -109,6 +109,11 @@ class OutputBudget { } const quoteIdentifier = (name: string): string => `"${name.replaceAll('"', '""')}"`; +const missingSelectedTable = (name: string): SqliteInspectionFailure => + new SqliteInspectionFailure( + "selection", + `Selected ordinary table does not exist with that exact name: ${name}`, + ); // SQLite folds only ASCII letters when matching virtual/shadow table names. const sqliteIdentifierKey = (name: string): string => name.replace(/[A-Z]/g, (letter) => letter.toLowerCase()); @@ -236,12 +241,49 @@ const readSchema = ( metadata.set(value.name, value); } } + // A trigger stores its target as written in CREATE TRIGGER, which SQLite + // resolves by ASCII case folding; report the stored table or view name. + // A selected-table scan returns only the selected table's dependents, and + // other scopes fold stored names once, only when a spelling differs. + let foldedNames: Map | undefined; + const triggerTable = (written: string): string => { + if (selectedName !== undefined) return selectedName; + if (metadata.has(written)) return written; + foldedNames ??= new Map( + [...metadata.keys()].map((name) => [sqliteIdentifierKey(name), name]), + ); + return foldedNames.get(sqliteIdentifierKey(written)) ?? written; + }; const unresolvedVirtualTables = [...metadata.values()] .filter((value) => value.type === "virtual" && value.ncol === 0) .map((value) => ({ name: value.name, prefix: sqliteIdentifierKey(`${value.name}_`), })); + const shadowParentOf = ( + name: string, + listed: z.output, + ): (typeof unresolvedVirtualTables)[number] | undefined => { + if (listed.type !== "table") return undefined; + const identifier = sqliteIdentifierKey(name); + return unresolvedVirtualTables.find((parent) => + identifier.startsWith(parent.prefix), + ); + }; + const selected = + selectedName === undefined ? undefined : metadata.get(selectedName); + if (selectedName !== undefined && selected === undefined) + throw missingSelectedTable(selectedName); + const selectedShadowParent = + selectedName === undefined || selected === undefined + ? undefined + : shadowParentOf(selectedName, selected); + // NOCASE folds only ASCII letters, as SQLite does when it resolves the + // table named by an index or trigger. Fold only for an ordinary table whose + // rows can be selected; a selection refused later keeps the exact filter, + // so its failure does not depend on declarations under another spelling. + const foldDependents = + selected?.type === "table" && selectedShadowParent === undefined; const schema: SqliteDatabase["schema"] = { scope, completeness: "complete", @@ -254,7 +296,9 @@ const readSchema = ( // duplicate large names or definitions beyond SQLite's native record limit. const declaration = scope === "inventory" ? "" : ", CAST(sql AS BLOB) AS sql"; const selection = - scope === "selected_table" ? " WHERE tbl_name = ? COLLATE BINARY" : ""; + scope === "selected_table" + ? ` WHERE tbl_name = ? COLLATE ${foldDependents ? "NOCASE" : "BINARY"}` + : ""; const statement = db.prepare( `SELECT type, CAST(name AS BLOB) AS name, CAST(tbl_name AS BLOB) AS tbl_name, rootpage${declaration} FROM main.sqlite_schema${selection}`, ); @@ -280,13 +324,10 @@ const readSchema = ( "format", `SQLite schema/table inventory mismatch: ${value.name}`, ); - const identifier = sqliteIdentifierKey(value.name); const possibleShadowParent = - listed.type === "table" - ? unresolvedVirtualTables.find((parent) => - identifier.startsWith(parent.prefix), - ) - : undefined; + value.name === selectedName + ? selectedShadowParent + : shadowParentOf(value.name, listed); const table: SqliteDatabase["schema"]["tables"][number] = { name: value.name, ...(scope === "inventory" ? {} : { sql: value.sql }), @@ -349,7 +390,7 @@ const readSchema = ( } else { const trigger = { name: value.name, - table: value.tbl_name, + table: triggerTable(value.tbl_name), ...(scope === "inventory" ? {} : { sql: value.sql }), }; budget.retain(trigger); @@ -370,11 +411,7 @@ const readRows = ( ): SqliteDatabase["rows"] => { const { encoding, budget } = context; if (input.table === undefined) return null; - if (table === undefined) - throw new SqliteInspectionFailure( - "selection", - `Selected ordinary table does not exist with that exact name: ${input.table}`, - ); + if (table === undefined) throw missingSelectedTable(input.table); if (table.kind === "unknown") throw new SqliteInspectionFailure( "selection", diff --git a/src/sqlite/SqliteDatabaseSnapshot.ts b/src/sqlite/SqliteDatabaseSnapshot.ts index 4acf89611..1384fbdb6 100644 --- a/src/sqlite/SqliteDatabaseSnapshot.ts +++ b/src/sqlite/SqliteDatabaseSnapshot.ts @@ -3,6 +3,7 @@ import { constants, type BigIntStats } from "node:fs"; import { join } from "node:path"; import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js"; import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; +import { AnalysisInputError } from "../domain/analysisErrorCore.js"; import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js"; import { readStableArtifact, @@ -166,6 +167,16 @@ const verifyInactiveJournal = async ( } }; +// Absence is a caller value to correct, not malformed path or content. +const missingDatabase = (path: string): AnalysisInputError => + new AnalysisInputError("inspect_sqlite_database", undefined, [ + { + path: ["path"], + reason: "invalid_value", + message: `Selected SQLite database does not exist: ${path}`, + }, + ]); + /** Freeze a stable offline file set without ever opening the source through SQLite. */ export const captureSqliteDatabaseSnapshot = async ( path: string, @@ -183,11 +194,7 @@ export const captureSqliteDatabaseSnapshot = async ( const wal = before[1]; if (database === undefined || wal === undefined || before[2] === undefined) throw new Error("Incomplete database file-set observation"); - if (database === null) - throw new ArtifactReaderFailure( - "path", - `Selected SQLite database does not exist: ${path}`, - ); + if (database === null) throw missingDatabase(path); if (before[2] !== null) await verifyInactiveJournal( `${path}-journal`, diff --git a/src/wasm/WabtArtifactProvider.ts b/src/wasm/WabtArtifactProvider.ts index c5f0aef53..3be21b2a1 100644 --- a/src/wasm/WabtArtifactProvider.ts +++ b/src/wasm/WabtArtifactProvider.ts @@ -362,8 +362,13 @@ export class WabtArtifactProvider implements WasmArtifactPort { OPERATION, "Unexpected WABT decoding output.", ); - const { imports, exports, customSectionNames, debugModuleNames } = - parseWabtWat(wat.stdout.text); + const { + imports, + exports, + customSectionNames, + debugModuleNames, + interface: interfaceFacts, + } = parseWabtWat(wat.stdout.text); capturedOutput = { stdout: dump.stdout.text, stderr: dump.stderr.text, @@ -388,6 +393,7 @@ export class WabtArtifactProvider implements WasmArtifactPort { sections, imports, exports, + interface: interfaceFacts, headers, details, wat: { diff --git a/src/wasm/WabtInterface.test.ts b/src/wasm/WabtInterface.test.ts new file mode 100644 index 000000000..68505f3ba --- /dev/null +++ b/src/wasm/WabtInterface.test.ts @@ -0,0 +1,116 @@ +import { expect, it } from "vitest"; +import { parseWabtWat } from "./WabtWat.js"; + +it("resolves imported index spaces and decoded escaped names without depending on body/type numbering", () => { + const report = parseWabtWat(`(module + (type (;0;) (func (param i32) (result i64))) + (import "a\\22b" "\\ef\\bb\\bf\\00" (func (;0;) (type 0))) + (func (;1;) (type 0) (param i32) (result i64) local.get 0 i64.extend_i32_s) + (export "" (func 1)))`).interface; + expect(report.entries).toMatchObject([ + { + direction: "import", + module: 'a"b', + name: "\ufeff\0", + index: 0, + signature: { kind: "func", params: ["i32"], results: ["i64"] }, + }, + { + direction: "export", + name: "", + index: 1, + signature: { kind: "func", params: ["i32"], results: ["i64"] }, + }, + ]); +}); + +it("retains exact memory/table/global/tag declarations and keeps unsupported references unknown", () => { + const report = parseWabtWat(`(module + (type (func (param i32))) + (type (struct (field i32))) + (import "env" "table" (table 1 3 funcref)) + (memory i64 2 4 shared) + (global (mut i64) (i64.const 1)) + (tag (type 0)) + (func (type 0)) + (func (param (ref null 1))) + (export "memory" (memory 0)) (export "global" (global 0)) + (export "tag" (tag 0)) (export "gc" (func 1)))`).interface; + expect(report.entries.map((e) => e.signature)).toEqual([ + { + kind: "table", + element: "funcref", + address_type: "i32", + minimum: "1", + maximum: "3", + shared: false, + }, + { + kind: "memory", + page_size: "65536", + address_type: "i64", + minimum: "2", + maximum: "4", + shared: true, + }, + { kind: "global", value_type: "i64", mutable: true }, + { kind: "tag", params: ["i32"], results: [] }, + null, + ]); + expect(report.entries[4]?.issue).toBeTruthy(); +}); + +it("does not bind recursive-type index spaces by guessing their cardinality", () => { + const report = parseWabtWat( + `(module (rec (type (struct)) (type (func))) (func (type 1)) (export "f" (func 0)))`, + ).interface; + expect(report.entries).toMatchObject([ + { name: "f", signature: null, issue: expect.any(String) }, + ]); +}); + +it("preserves custom memory page sizes and treats unadmitted nested declarations as unknown", () => { + const custom = parseWabtWat( + '(module (memory 1 2 (pagesize 1)) (export "m" (memory 0)))', + ).interface; + expect(custom.entries[0]?.signature).toMatchObject({ + kind: "memory", + page_size: "1", + }); + const unknown = parseWabtWat( + '(module (memory 1 (future-proposal 0)) (export "m" (memory 0)))', + ).interface; + expect(unknown.entries[0]?.signature).toBeNull(); +}); + +it("refuses typed name expansion within the source WABT byte cap before retaining interface output", () => { + const name = "x".repeat(6 * 1024 * 1024); + expect( + parseWabtWat(`(module (import "${name}" "f" (func)))`).interface, + ).toMatchObject({ + state: "partial", + entries: [], + issues: [expect.stringMatching(/representation budget/u)], + }); +}); + +it("charges shared type signatures at every retained occurrence and preserves bounded partial facts", () => { + const params = Array(1000).fill("i32").join(" "); + const exports = Array.from( + { length: 10000 }, + (_, i) => `(export "f${i}" (func 0))`, + ).join(" "); + const report = parseWabtWat( + `(module (type (func (param ${params}))) (import "e" "f" (func (type 0))) ${exports})`, + ).interface; + expect(report.state).toBe("partial"); + expect(report.entries.length).toBeGreaterThan(1); + expect(report.entries.length).toBeLessThan(10001); + expect(report.entries[0]?.signature).toMatchObject({ + kind: "func", + params: expect.any(Array), + }); + expect(Buffer.byteLength(JSON.stringify(report))).toBeLessThan( + 32 * 1024 * 1024, + ); +}); diff --git a/src/wasm/WabtInterface.ts b/src/wasm/WabtInterface.ts new file mode 100644 index 000000000..65e6fc0fa --- /dev/null +++ b/src/wasm/WabtInterface.ts @@ -0,0 +1,325 @@ +import { AnalysisResourceConstraintError } from "../domain/analysisErrorCore.js"; +import { wasmInterfaceEntryRepresentationBytes } from "../domain/wasm/wasmInterface.js"; +import { WasmInterfaceBudget } from "./WasmInterfaceBudget.js"; +import type { + WasmInterface, + WasmInterfaceEntry, + WasmInterfaceSignature, + WasmInterfaceValueType, +} from "../domain/wasm/wasmInterface.js"; +import { watString } from "./WabtString.js"; + +/** Split direct WAT children only; function instruction bodies are never materialized. */ +const parts = (form: string): string[] => { + const budget = new WasmInterfaceBudget(); + const items: string[] = []; + let i = 1; + while (i < form.length - 1) { + if (/\s/u.test(form[i] ?? "")) { + i++; + continue; + } + if (form.startsWith("(;", i)) { + let depth = 1; + i += 2; + while (i < form.length && depth > 0) { + if (form.startsWith("(;", i)) { + depth++; + i += 2; + } else if (form.startsWith(";)", i)) { + depth--; + i += 2; + } else i++; + } + continue; + } + if (form.startsWith(";;", i)) { + const end = form.indexOf("\n", i); + i = end < 0 ? form.length : end + 1; + continue; + } + const start = i; + if (form[i] === '"') { + i = watString(form, i + 1).end; + } else if (form[i] === "(") { + let depth = 1; + i++; + while (i < form.length && depth > 0) { + if (form[i] === '"') i = watString(form, i + 1).end; + else { + if (form[i] === "(") depth++; + else if (form[i] === ")") depth--; + i++; + } + } + } else { + while (i < form.length && !/[\s()]/u.test(form[i] ?? "")) i++; + const atom = form.slice(start, i); + if (items[0] === "func" && items.length > 1 && !atom.startsWith("$")) + break; + } + if (i <= start) throw new Error("Malformed WABT form"); + budget.reserve(64 + (i - start) * 2); + items.push(form.slice(start, i)); + } + return items; +}; +const head = (form: string): string | undefined => parts(form)[0]; +const admittedValues = new Set([ + "i32", + "i64", + "f32", + "f64", + "v128", + "funcref", + "externref", +]); +const isValueType = (type: string): type is WasmInterfaceValueType => + admittedValues.has(type); +const values = ( + form: string, + category: string, +): WasmInterfaceValueType[] | null => { + const result: WasmInterfaceValueType[] = []; + for (const item of parts(form)) { + if (!item.startsWith("(")) continue; + const group = parts(item); + if (group[0] !== category) continue; + for (const type of group.slice(1).filter((t) => !t.startsWith("$"))) { + if (!isValueType(type)) return null; + result.push(type); + } + } + return result; +}; +const functionType = ( + form: string, + kind: "func" | "tag", +): WasmInterfaceSignature | null => { + const params = values(form, "param"); + const results = values(form, "result"); + return params === null || results === null ? null : { kind, params, results }; +}; +const name = (text: string | undefined): string => { + if (text?.startsWith('"') !== true) + throw new Error("WABT interface name is missing"); + return new TextDecoder("utf8", { fatal: true, ignoreBOM: true }).decode( + watString(text, 1).bytes, + ); +}; +const integer = (text: string): string | null => { + if (!/^(?:\d+|0x[\da-f]+)$/iu.test(text)) return null; + return BigInt(text).toString(); +}; + +/** Resolve index spaces at the producer, not from retained raw WAT in consumers. */ +const decodeWabtInterface = ( + forms: readonly string[], + result: WasmInterface, +): WasmInterface => { + const budget = new WasmInterfaceBudget(); + const types: (WasmInterfaceSignature | null)[] = []; + const typeNames = new Map(); + const definitions = new Map< + string, + { signature: WasmInterfaceSignature | null; label: string | undefined }[] + >(); + const typeSpaceUnknown = forms.some((f) => head(f) === "rec"); + for (const form of forms) { + const p = parts(form); + if (p[0] !== "type") continue; + if (p[1]?.startsWith("$") === true) typeNames.set(p[1], types.length); + const body = p.find((item) => item.startsWith("(")); + budget.reserve(256 + (body?.length ?? 0) * 6); + types.push( + body !== undefined && head(body) === "func" && !typeSpaceUnknown + ? functionType(body, "func") + : null, + ); + } + const signature = (form: string): WasmInterfaceSignature | null => { + const p = parts(form); + const kind = p[0]; + if (kind === "func" || kind === "tag") { + const use = p.find((t) => t.startsWith("(type ")); + if (use === undefined) return functionType(form, kind); + const reference = parts(use)[1]; + const index = + reference === undefined + ? undefined + : reference.startsWith("$") + ? typeNames.get(reference) + : Number(reference); + const type = + index === undefined || typeSpaceUnknown ? undefined : types[index]; + return type?.kind === "func" ? { ...type, kind } : null; + } + if (kind === "memory" || kind === "table") { + const groups = p.slice(1).filter((t) => t.startsWith("(")); + let pageSize = "65536"; + for (const group of groups) { + const declaration = parts(group); + if ( + kind !== "memory" || + declaration[0] !== "pagesize" || + declaration.length !== 2 + ) + return null; + const parsedPageSize = integer(declaration[1] ?? ""); + if (parsedPageSize === null) return null; + pageSize = parsedPageSize; + } + const tokens = p + .slice(1) + .filter((t) => !t.startsWith("$") && !t.startsWith("(")); + const address_type = tokens[0] === "i64" ? "i64" : "i32"; + if (tokens[0] === "i64" || tokens[0] === "i32") tokens.shift(); + const minimum = integer(tokens.shift() ?? ""); + const possibleMaximum = + tokens[0] === undefined ? null : integer(tokens[0]); + if (possibleMaximum !== null) tokens.shift(); + const shared = tokens[0] === "shared"; + if (shared) tokens.shift(); + if (minimum === null) return null; + if (kind === "memory") + return tokens.length === 0 + ? { + kind, + page_size: pageSize, + address_type, + minimum, + maximum: possibleMaximum, + shared, + } + : null; + const element = tokens.shift(); + return tokens.length === 0 && + element !== undefined && + (element === "funcref" || element === "externref") + ? { + kind, + address_type, + minimum, + maximum: possibleMaximum, + shared, + element, + } + : null; + } + if (kind === "global") { + const declaration = p[1]?.startsWith("$") === true ? p[2] : p[1]; + const mutable = declaration?.startsWith("(mut ") === true; + const value_type = + mutable && declaration !== undefined + ? parts(declaration)[1] + : declaration; + return value_type !== undefined && isValueType(value_type) + ? { kind, value_type, mutable } + : null; + } + return null; + }; + const addDefinition = ( + form: string, + ): { + index: number; + kind: WasmInterfaceEntry["kind"]; + signature: WasmInterfaceSignature | null; + } => { + const p = parts(form); + const rawKind = p[0] ?? "unknown"; + const kind = ["func", "table", "memory", "global", "tag"].includes(rawKind) + ? (rawKind as WasmInterfaceEntry["kind"]) + : "unknown"; + const space = definitions.get(kind) ?? []; + const item = { + signature: signature(form), + label: p[1]?.startsWith("$") === true ? p[1] : undefined, + }; + budget.reserve(256); + const index = space.length; + space.push(item); + definitions.set(kind, space); + return { index, kind, signature: item.signature }; + }; + for (const form of forms) { + const p = parts(form); + if (p[0] !== "import") continue; + const body = p[3]; + if (body === undefined) throw new Error("Incomplete WABT import"); + const observed = addDefinition(body); + const entry: WasmInterfaceEntry = { + direction: "import", + module: name(p[1]), + name: name(p[2]), + ...observed, + wat_form: form, + issue: + observed.signature === null + ? "Import type is outside admitted core signature semantics" + : null, + }; + budget.reserve(wasmInterfaceEntryRepresentationBytes(entry)); + result.entries.push(entry); + } + for (const form of forms) { + const kind = head(form); + if ( + kind !== undefined && + ["func", "table", "memory", "global", "tag"].includes(kind) + ) + addDefinition(form); + } + for (const form of forms) { + const p = parts(form); + if (p[0] !== "export") continue; + const body = p[2]; + if (body === undefined) throw new Error("Incomplete WABT export"); + const descriptor = parts(body); + const rawKind = descriptor[0] ?? "unknown"; + const kind = ["func", "table", "memory", "global", "tag"].includes(rawKind) + ? (rawKind as WasmInterfaceEntry["kind"]) + : "unknown"; + const reference = descriptor[1]; + const space = definitions.get(kind) ?? []; + const index = + reference === undefined + ? -1 + : reference.startsWith("$") + ? space.findIndex((d) => d.label === reference) + : Number(reference); + const observed = + Number.isSafeInteger(index) && index >= 0 ? space[index] : undefined; + const entry: WasmInterfaceEntry = { + direction: "export", + module: null, + name: name(p[1]), + kind, + index: observed === undefined ? null : index, + signature: observed?.signature ?? null, + wat_form: form, + issue: + observed?.signature == null + ? "Export type/index is outside admitted core signature semantics" + : null, + }; + budget.reserve(wasmInterfaceEntryRepresentationBytes(entry)); + result.entries.push(entry); + } + // Unsupported signatures leave inventory completeness intact. A missing + // signature is unknown per entry, not proof that a name is absent. + return result; +}; + +/** New typed coverage may be incomplete without discarding existing decoded WAT Evidence. */ +export const parseWabtInterface = (forms: readonly string[]): WasmInterface => { + const result: WasmInterface = { state: "complete", entries: [], issues: [] }; + try { + return decodeWabtInterface(forms, result); + } catch (cause: unknown) { + if (!(cause instanceof AnalysisResourceConstraintError)) throw cause; + result.state = "partial"; + result.issues.push(cause.message); + return result; + } +}; diff --git a/src/wasm/WabtOutput.test.ts b/src/wasm/WabtOutput.test.ts index abd20d848..91740f09d 100644 --- a/src/wasm/WabtOutput.test.ts +++ b/src/wasm/WabtOutput.test.ts @@ -229,7 +229,7 @@ it("extracts escaped WAT import/export forms while retaining multiline producer new URL("../../tests/fixtures/wasm/decoded.wat", import.meta.url), "utf8", ); - expect(parseWabtWat(text)).toEqual({ + expect(parseWabtWat(text)).toMatchObject({ imports: ['(import "env" "foo" (func (;0;) (type 0)))'], exports: ['(export "memory" (memory 0))', '(export "run" (func 1))'], customSectionNames: ["test"], @@ -239,7 +239,7 @@ it("extracts escaped WAT import/export forms while retaining multiline producer parseWabtWat( '(module (import "a.b" "c\\0aline" (func)) (export "fake\\22Export[1]:" (func 0)))', ), - ).toEqual({ + ).toMatchObject({ imports: ['(import "a.b" "c\\0aline" (func))'], exports: ['(export "fake\\22Export[1]:" (func 0))'], customSectionNames: [], diff --git a/src/wasm/WabtString.ts b/src/wasm/WabtString.ts new file mode 100644 index 000000000..79615b193 --- /dev/null +++ b/src/wasm/WabtString.ts @@ -0,0 +1,46 @@ +/** Decode a producer WAT string to frame objdump's raw display text. */ +export const watString = (form: string, start: number) => { + let end = start; + while (end < form.length && form[end] !== '"') { + if (form[end] === "\\") end++; + end++; + } + if (end === form.length) + throw new Error("Incomplete WABT custom-section name."); + const name = form.slice(start, end); + // Escapes shrink; literal UTF-8 has the same byte extent as its source. + const bytes = Buffer.alloc(Buffer.byteLength(name, "utf8")); + let length = 0; + for (let index = 0; index < name.length;) { + if (name[index] !== "\\") { + const next = name.indexOf("\\", index); + const end = next < 0 ? name.length : next; + length += bytes.write(name.slice(index, end), length, "utf8"); + index = end; + continue; + } + const hex = name.slice(index + 1, index + 3); + if (/^[\da-f]{2}$/iu.test(hex)) { + bytes[length++] = Number.parseInt(hex, 16); + index += 3; + } else { + const escape = name[index + 1]; + const value = + escape === "t" + ? 9 + : escape === "n" + ? 10 + : escape === "r" + ? 13 + : escape === '"' + ? 34 + : escape === "\\" + ? 92 + : undefined; + if (value === undefined) throw new Error("Unsupported WABT name escape."); + bytes[length++] = value; + index += 2; + } + } + return { bytes: bytes.subarray(0, length), end: end + 1 }; +}; diff --git a/src/wasm/WabtWat.ts b/src/wasm/WabtWat.ts index d766fe617..01b9f16bc 100644 --- a/src/wasm/WabtWat.ts +++ b/src/wasm/WabtWat.ts @@ -1,49 +1,5 @@ -/** Decode a producer WAT string to frame objdump's raw display text. */ -const watString = (form: string, start: number) => { - let end = start; - while (end < form.length && form[end] !== '"') { - if (form[end] === "\\") end++; - end++; - } - if (end === form.length) - throw new Error("Incomplete WABT custom-section name."); - const name = form.slice(start, end); - // Escapes shrink; literal UTF-8 has the same byte extent as its source. - const bytes = Buffer.alloc(Buffer.byteLength(name, "utf8")); - let length = 0; - for (let index = 0; index < name.length;) { - if (name[index] !== "\\") { - const next = name.indexOf("\\", index); - const end = next < 0 ? name.length : next; - length += bytes.write(name.slice(index, end), length, "utf8"); - index = end; - continue; - } - const hex = name.slice(index + 1, index + 3); - if (/^[\da-f]{2}$/iu.test(hex)) { - bytes[length++] = Number.parseInt(hex, 16); - index += 3; - } else { - const escape = name[index + 1]; - const value = - escape === "t" - ? 9 - : escape === "n" - ? 10 - : escape === "r" - ? 13 - : escape === '"' - ? 34 - : escape === "\\" - ? 92 - : undefined; - if (value === undefined) throw new Error("Unsupported WABT name escape."); - bytes[length++] = value; - index += 2; - } - } - return { bytes: bytes.subarray(0, length), end: end + 1 }; -}; +import { parseWabtInterface } from "./WabtInterface.js"; +import { watString } from "./WabtString.js"; /** Read only module-name metadata from WABT's preserved name-section payload. */ const moduleNames = (bytes: Uint8Array): string[] => { @@ -85,6 +41,7 @@ const moduleNames = (bytes: Uint8Array): string[] => { export const parseWabtWat = (text: string) => { if (!/^\(module(?:\s|\))/u.test(text)) throw new Error("Unsupported WABT WAT module prefix."); + const forms: string[] = []; const imports: string[] = []; const exports: string[] = []; const customSectionNames: string[] = []; @@ -140,6 +97,12 @@ export const parseWabtWat = (text: string) => { } else if (char === ")") { if (depth === 2) { const form = text.slice(start, index + 1); + if ( + /^\((?:type|rec|import|export|func|table|memory|global|tag)\s/u.test( + form, + ) + ) + forms.push(form); if (/^\(import\s/u.test(form)) imports.push(form); else if (/^\(export\s/u.test(form)) exports.push(form); else if (/^\(@custom\s/u.test(form)) { @@ -172,5 +135,11 @@ export const parseWabtWat = (text: string) => { } if (depth !== 0 || quoted || commentDepth !== 0) throw new Error("Incomplete WABT WAT output."); - return { imports, exports, customSectionNames, debugModuleNames }; + return { + imports, + exports, + customSectionNames, + debugModuleNames, + interface: parseWabtInterface(forms), + }; }; diff --git a/src/wasm/WasmInterfaceBudget.ts b/src/wasm/WasmInterfaceBudget.ts new file mode 100644 index 000000000..0d928b7ff --- /dev/null +++ b/src/wasm/WasmInterfaceBudget.ts @@ -0,0 +1,17 @@ +import { AnalysisResourceConstraintError } from "../domain/analysisErrorCore.js"; +import { WASM_INTERFACE_REPRESENTATION_BYTES } from "../domain/wasm/wasmInterface.js"; + +/** Eager typed observations share the existing WABT output capacity, not an agent token estimate. */ +export class WasmInterfaceBudget { + #remaining = WASM_INTERFACE_REPRESENTATION_BYTES; + reserve(bytes: number): void { + if (!Number.isSafeInteger(bytes) || bytes < 0 || bytes > this.#remaining) + throw new AnalysisResourceConstraintError( + "inspect_wasm_artifact", + "capacity", + "Typed WASM interface decoding exceeds the WABT output representation budget", + { maximum_bytes: WASM_INTERFACE_REPRESENTATION_BYTES }, + ); + this.#remaining -= bytes; + } +} diff --git a/src/windows/WindowsOwnedProcess.ts b/src/windows/WindowsOwnedProcess.ts index 1b96db867..b6a530461 100644 --- a/src/windows/WindowsOwnedProcess.ts +++ b/src/windows/WindowsOwnedProcess.ts @@ -67,6 +67,7 @@ export class WindowsOwnedProcess extends EventEmitter { #closed = false; #failure: string | undefined; #timer: NodeJS.Timeout | undefined; + #unreferenced = false; #cleanup: Promise | undefined; readonly #memoryLimitBytes: number | undefined; #memoryLimitReached = false; @@ -222,6 +223,18 @@ export class WindowsOwnedProcess extends EventEmitter { return this.#cleanup; } + /** Release event-loop references so unsettled jobs do not prevent host exit. */ + unref(): void { + this.#unreferenced = true; + this.#timer?.unref(); + } + + /** Restore event-loop references when retrying process lifecycle observation. */ + ref(): void { + this.#unreferenced = false; + this.#timer?.ref(); + } + async #stop(): Promise { if (this.#closed) return { cleaned: true, signaled: false }; let timer: NodeJS.Timeout | undefined; @@ -255,6 +268,7 @@ export class WindowsOwnedProcess extends EventEmitter { #schedule(): void { if (this.#closed || this.#timer !== undefined) return; this.#timer = setTimeout(() => this.#poll(), 25); + if (this.#unreferenced) this.#timer.unref(); } #poll(): void { diff --git a/tests/acceptance/applications/blockSourceMapCli.test.ts b/tests/acceptance/applications/blockSourceMapCli.test.ts new file mode 100644 index 000000000..957cd49c2 --- /dev/null +++ b/tests/acceptance/applications/blockSourceMapCli.test.ts @@ -0,0 +1,99 @@ +import { createHash } from "node:crypto"; +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { join, sep } from "node:path"; + +import { createPackageWithOptions, listPackage } from "@electron/asar"; +import { expect } from "vitest"; +import { z } from "zod"; + +import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js"; +import { cliTest } from "../../support/cli/cliFixture.js"; + +cliTest.for(["CLI", "stdio MCP"] as const)( + "links a block source-map URL containing an asterisk through %s", + async (surface, { cli }) => { + const root = await createTestTempDirectory("rea-block-map-"); + const directory = join(root, "input"); + await mkdir(directory); + const url = "app*part.js.map"; + const directive = "/*# sourceMappingURL=" + url; + await writeFile(join(directory, "app.js"), directive + "*/"); + await writeFile( + join(directory, "app_part.js.map"), + JSON.stringify({ version: 3, sources: [], names: [], mappings: "" }), + ); + const archive = join(root, "app.asar"); + await createPackageWithOptions(directory, archive, {}); + // ASAR logical names can contain '*' even on Windows. Rename this header + // key without changing entry bytes, sizes or payload offsets. + const bytes = await readFile(archive); + const key = Buffer.from('"app_part.js.map"'); + const offset = bytes.indexOf(key); + expect(offset).toBeGreaterThan(0); + expect(bytes.indexOf(key, offset + key.length)).toBe(-1); + Buffer.from('"' + url + '"').copy(bytes, offset); + await writeFile(archive, bytes); + expect(listPackage(archive, { isPack: false })).toContain(sep + url); + let document: unknown; + if (surface === "CLI") { + const result = await cli.run({ + arguments: ["analyze-javascript-application", archive, "--json"], + }); + expect(result.exitCode, result.stderr).toBe(0); + document = result.json; + } else { + const { call } = await connectLocalToolsMcp(); + const result = await call("analyze_javascript_application", { + input_path: archive, + format: "asar", + }); + expect(result.isError).not.toBe(true); + document = result.structuredContent; + } + const { + normalized_result: { graph }, + } = z + .object({ + normalized_result: javascriptApplicationAnalysisResultSchema, + }) + .parse(document); + const digest = createHash("sha256").update(url).digest("hex"); + const map = graph.nodes.find(({ observations }) => + observations.some(({ properties }) => properties.declared_url === url), + ); + expect( + map, + "the complete declared URL must identify the map", + ).toBeDefined(); + expect(map?.observations).toContainEqual( + expect.objectContaining({ + properties: { + declared_url: url, + declared_url_sha256: digest, + resolved_path: url, + available: true, + }, + evidence: expect.objectContaining({ + location: { + available: true, + value: { + kind: "source-range", + source: "app.js", + start: { line: 1, column: 0 }, + end: { line: 1, column: directive.length }, + }, + }, + }), + }), + ); + expect(graph.edges).toContainEqual( + expect.objectContaining({ + relation: "maps_to", + target_node_id: map?.node_id, + properties: { declared_url_sha256: digest, resolved_path: url }, + }), + ); + }, +); diff --git a/tests/acceptance/applications/electronInvalidPaths.test.ts b/tests/acceptance/applications/electronInvalidPaths.test.ts index d74c04b2b..e7d5935e2 100644 --- a/tests/acceptance/applications/electronInvalidPaths.test.ts +++ b/tests/acceptance/applications/electronInvalidPaths.test.ts @@ -67,3 +67,40 @@ cliTest( expect(session.isError, JSON.stringify(session)).not.toBe(true); }, ); + +cliTest( + "rejects a file URL path field without resolving an invented local path", + async ({ cli }) => { + const root = await createTestTempDirectory("rea-electron-file-url-"); + const application = join(root, "main.js"); + await writeFile(application, "module.exports = {};\n"); + const response = await cli.run({ + arguments: [ + "capture-electron-scenario", + JSON.stringify({ + executable_path: "file:///opt/electron", + application_path: application, + }), + "--format", + "json", + ], + environment: { REA_LOG_LEVEL: "silent" }, + }); + expect(response.exitCode).toBe(1); + expect(response.json).toMatchObject({ + code: "invalid_request", + details: { + issues: [ + { + path: ["executable_path"], + message: expect.stringContaining( + "must be absolute local filesystem paths", + ), + }, + ], + }, + }); + expect(response.stdout).not.toContain("ENOENT"); + expect(response.stdout).not.toContain("file:/opt/electron"); + }, +); diff --git a/tests/acceptance/applications/javascriptAnalysisLimits.test.ts b/tests/acceptance/applications/javascriptAnalysisLimits.test.ts new file mode 100644 index 000000000..6f0ba8cef --- /dev/null +++ b/tests/acceptance/applications/javascriptAnalysisLimits.test.ts @@ -0,0 +1,156 @@ +import type { Client } from "@modelcontextprotocol/client"; +import { readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { expect, it } from "vitest"; +import { z } from "zod"; + +import { parseEvidence } from "../../../src/domain/evidence.js"; +import { parseEvidenceBundle } from "../../../src/domain/evidenceBundle.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { cliTest } from "../../support/cli/cliFixture.js"; + +const cases = [ + { + name: "both limits", + args: ["--max-heap-mb", "128", "--analysis-timeout-ms", "30000"], + controls: { max_heap_mb: 128, analysis_timeout_ms: 30000 }, + }, + { + name: "heap limit alone", + args: ["--max-heap-mb", "256"], + controls: { max_heap_mb: 256 }, + }, + { + name: "timeout alone", + args: ["--analysis-timeout-ms", "40000"], + controls: { analysis_timeout_ms: 40000 }, + }, + { + name: "omitted limits", + args: [], + controls: {}, + }, +]; + +cliTest.for(cases)( + "preserves $name in CLI Evidence without adding ambient defaults", + async ({ args, controls }, { cli }) => { + const root = await createTestTempDirectory("rea-js-limit-provenance-"); + await writeFile(join(root, "main.js"), "export const observed = 7;\n"); + const response = await cli.run({ + arguments: [ + "analyze-javascript-application", + root, + ...args, + "--format", + "json", + ], + environment: { NODE_OPTIONS: "--max-old-space-size=512" }, + }); + expect(response.exitCode, response.stdout + response.stderr).toBe(0); + const evidence = parseEvidence(response.json); + expect(evidence.parameters).toEqual({ + format: "auto", + integrity_policy: "fail", + ...controls, + }); + }, +); + +it("preserves MCP analysis limits in retained Evidence and an exported bundle round trip", async () => { + const root = await createTestTempDirectory("rea-js-limit-mcp-"); + await writeFile(join(root, "main.js"), "export const observed = 7;\n"); + const bundleRoot = await createTestTempDirectory("rea-js-limit-bundle-"); + const path = join(bundleRoot, "evidence.json"); + const { client } = await connectLocalToolsMcp({ + NODE_OPTIONS: "--max-old-space-size=512", + }); + const expected = new Map(); + for (const { controls } of cases) { + const response = await client.callTool({ + name: "analyze_javascript_application", + arguments: { input_path: root, ...controls, detail: "summary" }, + }); + expect(response.isError, JSON.stringify(response.content)).not.toBe(true); + const { parent_evidence_id } = z + .object({ + normalized_result: z.object({ parent_evidence_id: z.string() }), + }) + .parse(response.structuredContent).normalized_result; + expected.set(parent_evidence_id, { + format: "auto", + integrity_policy: "fail", + ...controls, + }); + } + expect(expected.size).toBe(cases.length); + const retained = await getBundle(client); + for (const [evidence_id, parameters] of expected) + expect( + retained.records.find((record) => record.evidence_id === evidence_id) + ?.parameters, + ).toEqual(parameters); + const exported = await client.callTool({ + name: "export_evidence_bundle", + arguments: { path }, + }); + expect(exported.isError).not.toBe(true); + expect(parseEvidenceBundle(JSON.parse(await readFile(path, "utf8")))).toEqual( + retained, + ); + const closed = await client.callTool({ + name: "close_binary", + arguments: {}, + }); + expect(closed.isError).not.toBe(true); + expect((await getBundle(client)).records).toEqual([]); + const imported = await client.callTool({ + name: "import_evidence_bundle", + arguments: { path }, + }); + expect(imported.isError, JSON.stringify(imported.content)).not.toBe(true); + expect(await getBundle(client)).toEqual(retained); +}, 60_000); + +it("retains caller limits in partial Evidence after an actual analysis timeout", async () => { + const root = await createTestTempDirectory("rea-js-limit-timeout-"); + await writeFile(join(root, "main.js"), "export const observed = 7;\n"); + const { client } = await connectLocalToolsMcp({ + NODE_OPTIONS: "--max-old-space-size=512", + }); + const response = await client.callTool({ + name: "analyze_javascript_application", + arguments: { + input_path: root, + max_heap_mb: 128, + analysis_timeout_ms: 1, + }, + }); + expect(parseMcpToolError(response)).toMatchObject({ + error: { code: "provider_timeout" }, + }); + const bundle = await getBundle(client); + const partial = bundle.records.find( + (record) => record.operation === "analyze_javascript_application", + ); + expect(partial?.parameters).toEqual({ + format: "auto", + integrity_policy: "fail", + max_heap_mb: 128, + analysis_timeout_ms: 1, + }); + await client.ping(); +}, 30_000); + +async function getBundle(client: Client) { + const response = await client.callTool({ + name: "get_evidence_bundle", + arguments: { detail: "complete" }, + }); + expect(response.isError).not.toBe(true); + return parseEvidenceBundle( + z.object({ result: z.unknown() }).parse(response.structuredContent).result, + ); +} diff --git a/tests/acceptance/applications/javascriptChildProcessArgumentsCli.test.ts b/tests/acceptance/applications/javascriptChildProcessArgumentsCli.test.ts new file mode 100644 index 000000000..84bfca030 --- /dev/null +++ b/tests/acceptance/applications/javascriptChildProcessArgumentsCli.test.ts @@ -0,0 +1,72 @@ +import { join } from "node:path"; + +import { expect } from "vitest"; + +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { cliTest } from "../../support/cli/cliFixture.js"; +import { analyzeJavaScriptDirectory } from "../../support/cli/javascriptApplication.js"; +import { writeFixtureFiles } from "../../support/javascriptApplicationFixture.js"; + +cliTest.for(["CLI", "stdio MCP"] as const)( + "keeps child argv counts unknown after spread through %s", + async (surface, { cli, processes }) => { + const root = await createTestTempDirectory("rea-child-argv-"); + await writeFixtureFiles(root, { + "entry.mjs": [ + 'import { spawn, execFile, fork } from "node:child_process";', + 'const args = ["a", "b"];', + 'spawn("spread-spawn", [...args]);', + 'execFile("spread-exec", ["prefix", ...args]);', + 'fork("spread-fork", [...args, "suffix"]);', + 'spawn("fixed", ["a", "b"]);', + 'execFile("empty", []);', + ].join("\n"), + "child.cjs": "console.log(process.argv.slice(2).length);", + }); + const args = ["a", "b"]; + const counts: number[] = []; + for (const values of [ + [...args], + ["prefix", ...args], + [...args, "suffix"], + ["a", "b"], + [], + ]) { + const control = await processes.run( + process.execPath, + [join(root, "child.cjs"), ...values], + { timeoutMs: 5_000 }, + ); + expect(control.exitCode, control.stderr).toBe(0); + counts.push(Number(control.stdout)); + } + expect(counts).toEqual([2, 3, 3, 2, 0]); + const graph = (await analyzeJavaScriptDirectory(surface, cli, root)) + .semantic_graph; + const children = graph.nodes.filter( + (node) => node.kind === "child-process", + ); + expect(children).toHaveLength(5); + for (const command of ["spread-spawn", "spread-exec", "spread-fork"]) { + const child = children.find( + (node) => node.properties.command === command, + ); + expect(child?.properties.argv_count, command).toBeNull(); + expect( + graph.relations.filter( + (relation) => + relation.relation === "supplies-argv" && + relation.target_node_id === child?.node_id, + ), + ).toEqual([]); + } + expect( + children.find((node) => node.properties.command === "fixed")?.properties + .argv_count, + ).toBe(2); + expect( + children.find((node) => node.properties.command === "empty")?.properties + .argv_count, + ).toBe(0); + }, +); diff --git a/tests/acceptance/applications/javascriptCommonJsDirectoryLookup.test.ts b/tests/acceptance/applications/javascriptCommonJsDirectoryLookup.test.ts new file mode 100644 index 000000000..da9fcee86 --- /dev/null +++ b/tests/acceptance/applications/javascriptCommonJsDirectoryLookup.test.ts @@ -0,0 +1,188 @@ +import { execFile } from "node:child_process"; +import { createRequire } from "node:module"; +import { join, relative, sep } from "node:path"; +import { fileURLToPath } from "node:url"; +import { promisify } from "node:util"; + +import { expect } from "vitest"; + +import type { JavaScriptApplicationGraph } from "../../../src/domain/javascript/javascriptApplicationGraph.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { cliTest } from "../../support/cli/cliFixture.js"; +import { analyzeJavaScriptDirectory } from "../../support/cli/javascriptApplication.js"; +import { writeFixtureFiles } from "../../support/javascriptApplicationFixture.js"; + +const execute = promisify(execFile); + +const logicalPath = (root: string, path: string): string => + relative(root, path).split(sep).join("/"); + +const requireSources = ( + requests: readonly (readonly [string, string, string | null])[], +): Readonly> => + Object.fromEntries( + [...new Set(requests.map(([source]) => source))].map((source) => [ + source, + requests + .filter(([path]) => path === source) + .map(([, specifier]) => `require(${JSON.stringify(specifier)});`) + .join("\n"), + ]), + ); + +const expectResolution = ( + graph: JavaScriptApplicationGraph, + source: string, + specifier: string, + kind: "require" | "import", + resolvedPath: string | null, +): void => { + const edges = graph.edges.filter( + ({ relation, properties, evidence }) => + relation === "imports" && + properties.specifier === specifier && + evidence.location.available && + evidence.location.value.kind === "source-range" && + evidence.location.value.source === source, + ); + expect(edges).toHaveLength(2); + expect(edges.map(({ properties }) => properties)).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + kind: kind === "require" ? "require" : "static-import", + resolved_path: resolvedPath, + }), + expect.objectContaining({ + module_link_kind: kind, + resolved_path: resolvedPath, + resolution_status: resolvedPath === null ? "not-found" : "resolved", + }), + ]), + ); +}; + +cliTest.for(["CLI", "stdio MCP"] as const)( + "preserves CommonJS directory requests before path normalization through %s", + async (surface, { cli }) => { + const root = await createTestTempDirectory("rea-cjs-directory-lookup-"); + const requests = [ + ["main.cjs", "./dep/.", "dep/index.js"], + ["main.cjs", "./dep/", "dep/index.js"], + ["main.cjs", "./parent/child/..", "parent/index.js"], + ["main.cjs", "./dep", "dep.js"], + ["main.cjs", "./entry/.", "entry/dep.js"], + ["main.cjs", "./missing/.", null], + ["main.cjs", "directory/", "node_modules/directory/index.js"], + ["main.cjs", "directory/.", "node_modules/directory/index.js"], + ["main.cjs", "package/dep/.", "node_modules/package/dep/index.js"], + ["parent/child/main.cjs", ".", "parent/child/index.js"], + ["parent/child/main.cjs", "..", "parent/index.js"], + ["typed.cts", "./typed.js/.", "typed.js/index.js"], + ] as const; + await writeFixtureFiles(root, { + "package.json": '{"type":"commonjs"}', + "dep.js": "module.exports = 'file';", + "dep/index.js": "module.exports = 'directory';", + "entry/package.json": '{"main":"./dep/."}', + "entry/dep.js": "module.exports = 'main file';", + "entry/dep/index.js": "module.exports = 'main directory decoy';", + "missing.js": "module.exports = 'missing directory decoy';", + "parent.js": "module.exports = 'parent file';", + "parent/index.js": "module.exports = 'parent directory';", + "parent/child.js": "module.exports = 'child file';", + "parent/child/index.js": "module.exports = 'child directory';", + "node_modules/directory.js": "module.exports = 'package file';", + "node_modules/directory/index.js": + "module.exports = 'package directory';", + "node_modules/package/dep.js": "module.exports = 'subpath file';", + "node_modules/package/dep/index.js": + "module.exports = 'subpath directory';", + "typed.ts": "module.exports = 'emitted source decoy';", + "typed.js/index.js": "module.exports = 'typed request directory';", + ...requireSources(requests), + }); + for (const [source, specifier, expected] of requests) { + const require = createRequire(join(root, source)); + if (expected === null) + expect(() => require.resolve(specifier)).toThrow( + expect.objectContaining({ code: "MODULE_NOT_FOUND" }), + ); + else expect(logicalPath(root, require.resolve(specifier))).toBe(expected); + } + const { graph } = await analyzeJavaScriptDirectory(surface, cli, root); + for (const [source, specifier, expected] of requests) + expectResolution(graph, source, specifier, "require", expected); + expect(graph.coverage.status).toBe("complete"); + }, +); + +cliTest.for(["CLI", "stdio MCP"] as const)( + "preserves loader-specific node_modules ancestors through %s", + async (surface, { cli }) => { + const root = await createTestTempDirectory("rea-cjs-ancestor-lookup-"); + const requests = [ + [ + "node_modules/client/main.cjs", + "shared", + "node_modules/shared/index.js", + ], + [ + "node_modules/client/main.cjs", + "@scope/shared", + "node_modules/@scope/shared/index.js", + ], + [ + "node_modules/client/main.cjs", + "private", + "node_modules/client/node_modules/private/index.js", + ], + [ + "node_modules/@scope/client/main.cjs", + "shared", + "node_modules/shared/index.js", + ], + [ + "node_modules/parent/node_modules/client/main.cjs", + "shared", + "node_modules/parent/node_modules/shared/index.js", + ], + ] as const; + const esmSource = "node_modules/client/main.mjs"; + const esmTarget = "node_modules/node_modules/shared/index.js"; + await writeFixtureFiles(root, { + "package.json": '{"type":"commonjs"}', + "node_modules/shared/index.js": "module.exports = 'outer dependency';", + "node_modules/@scope/shared/index.js": + "module.exports = 'scoped dependency';", + "node_modules/node_modules/shared/index.js": + "module.exports = 'ESM dependency';", + "node_modules/node_modules/@scope/shared/index.js": + "module.exports = 'scoped decoy';", + "node_modules/client/node_modules/private/index.js": + "module.exports = 'private dependency';", + "node_modules/parent/node_modules/shared/index.js": + "module.exports = 'nested dependency';", + "node_modules/parent/node_modules/node_modules/shared/index.js": + "module.exports = 'nested decoy';", + [esmSource]: + 'import "shared"; console.log(JSON.stringify(import.meta.resolve("shared")));', + ...requireSources(requests), + }); + for (const [source, specifier, expected] of requests) + expect( + logicalPath(root, createRequire(join(root, source)).resolve(specifier)), + ).toBe(expected); + const control = await execute(process.execPath, [join(root, esmSource)], { + cwd: root, + timeout: 5_000, + }); + expect( + logicalPath(root, fileURLToPath(JSON.parse(control.stdout) as string)), + ).toBe(esmTarget); + const { graph } = await analyzeJavaScriptDirectory(surface, cli, root); + for (const [source, specifier, expected] of requests) + expectResolution(graph, source, specifier, "require", expected); + expectResolution(graph, esmSource, "shared", "import", esmTarget); + expect(graph.coverage.status).toBe("complete"); + }, +); diff --git a/tests/acceptance/applications/javascriptConfigurationReadsCli.test.ts b/tests/acceptance/applications/javascriptConfigurationReadsCli.test.ts new file mode 100644 index 000000000..8b9ba1abc --- /dev/null +++ b/tests/acceptance/applications/javascriptConfigurationReadsCli.test.ts @@ -0,0 +1,78 @@ +import { join } from "node:path"; + +import { expect } from "vitest"; + +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { cliTest } from "../../support/cli/cliFixture.js"; +import { analyzeJavaScriptDirectory } from "../../support/cli/javascriptApplication.js"; +import { writeFixtureFiles } from "../../support/javascriptApplicationFixture.js"; + +cliTest.for(["CLI", "stdio MCP"] as const)( + "retains configuration sources consumed by member access through %s", + async (surface, { cli, processes }) => { + const root = await createTestTempDirectory("rea-config-reads-"); + await writeFixtureFiles(root, { + "entry.mjs": [ + "const length = process.env.REA_ENDPOINT.length;", + "const endpoint = process.env.REA_ENDPOINT;", + "const argumentLength = process.argv[2].length;", + "const mode = process.argv[3].trim();", + "const optional = process.env.REA_MISSING?.length;", + "function shadow(process) { return process.env.LOCAL.length + process.argv[2].length; }", + 'const local = { env: { FAKE: "fake" }, argv: [0, 1, "abc"] };', + 'const control = shadow({ env: { LOCAL: "local" }, argv: [0, 1, "abc"] });', + "console.log(JSON.stringify([length, endpoint, argumentLength, mode, optional, control, local.env.FAKE.length]));", + ].join("\n"), + }); + const control = await processes.run( + process.execPath, + [join(root, "entry.mjs"), "abc", " mode "], + { env: { REA_ENDPOINT: "endpoint" }, timeoutMs: 15_000 }, + ); + expect(control.exitCode, control.stderr).toBe(0); + expect(JSON.parse(control.stdout)).toEqual([ + 8, + "endpoint", + 3, + "mode", + null, + 8, + 4, + ]); + const graph = (await analyzeJavaScriptDirectory(surface, cli, root)) + .semantic_graph; + const sources = graph.nodes.filter((node) => node.kind === "config-source"); + expect( + sources + .map((node) => [node.properties.source_kind, node.properties.key]) + .sort(), + ).toEqual([ + ["argv", "2"], + ["argv", "3"], + ["environment", "REA_ENDPOINT"], + ["environment", "REA_ENDPOINT"], + ["environment", "REA_MISSING"], + ]); + expect(sources.every((node) => node.properties.value === null)).toBe(true); + const endpoint = graph.nodes.find( + (node) => node.kind === "binding" && node.label === "endpoint", + ); + expect(endpoint).toBeDefined(); + for (const source of sources) { + const bindingTargets = graph.relations + .filter((relation) => relation.source_node_id === source.node_id) + .flatMap((relation) => + graph.nodes.filter( + (node) => + node.node_id === relation.target_node_id && + node.kind === "binding", + ), + ); + expect(bindingTargets.map((node) => node.node_id)).toEqual( + source.identity.source_range?.start.line === 2 + ? [endpoint?.node_id] + : [], + ); + } + }, +); diff --git a/tests/acceptance/emptyTargetPathCli.test.ts b/tests/acceptance/emptyTargetPathCli.test.ts new file mode 100644 index 000000000..6f6210d31 --- /dev/null +++ b/tests/acceptance/emptyTargetPathCli.test.ts @@ -0,0 +1,35 @@ +import { expect } from "vitest"; + +import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js"; +import { cliTest } from "../support/cli/cliFixture.js"; + +cliTest.for(["analyze", "inspect-artifact"])( + "%s rejects an explicitly empty target before resolving it against cwd", + async (command, { cli }) => { + const cwd = await createTestTempDirectory("rea-empty-target-"); + const result = await cli.run({ + arguments: [command, "", "--json"], + cwd, + }); + expect(result.exitCode).toBe(1); + expect(result.json).toMatchObject({ code: "VALIDATION_ERROR" }); + expect(result.stdout).toContain("path"); + expect(result.stdout).not.toContain("directory_requires_file"); + }, +); + +cliTest( + "inspect-artifact still admits an explicitly selected current directory", + async ({ cli }) => { + const cwd = await createTestTempDirectory("rea-explicit-target-"); + const result = await cli.run({ + arguments: ["inspect-artifact", ".", "--json"], + cwd, + }); + expect(result.exitCode).toBe(1); + expect(result.json).toMatchObject({ + code: "target_unavailable", + details: { path: cwd, constraint: "directory_requires_file" }, + }); + }, +); diff --git a/tests/acceptance/flutterAnalysis.test.ts b/tests/acceptance/flutterAnalysis.test.ts index 11671785e..5e9a8889b 100644 --- a/tests/acceptance/flutterAnalysis.test.ts +++ b/tests/acceptance/flutterAnalysis.test.ts @@ -7,6 +7,7 @@ import { aotFixtureImage, engineFixtureImage, REFERENCE_HASH, + appendElfRanges, } from "../../src/flutter/FlutterBuild.fixture.js"; import { createFlutterApk, @@ -159,3 +160,88 @@ cliTest( ).toEqual([4, 4]); }, ); + +cliTest( + "CLI and MCP qualify conflicting load mappings and bounded ELF notes", + async ({ cli }) => { + const note = Buffer.from( + "040000000800000003000000474e55000123456789abcdef", + "hex", + ); + const engine = engineFixtureImage({ + notes: Buffer.concat([note, Buffer.alloc(2000)]), + }); + const denseEngine = appendElfRanges( + engine, + "program", + Array.from({ length: 2000 }, (_, index) => ({ + type: 4, + offset: 0x400 + note.length + index, + size: 1, + })), + ); + const { call } = await connectLocalToolsMcp(); + for (const conflict of [false, true]) { + const image = appendElfRanges(aotFixtureImage(), "program", [ + { + type: 1, + address: 0x2010, + offset: conflict ? 0x1010 : 0x2010, + size: 4, + }, + ]); + const path = await createFlutterApk(`load-${String(conflict)}.apk`, [ + { path: "lib/x86_64/libapp.so", bytes: image }, + { + path: "lib/x86_64/libflutter.so", + bytes: conflict ? denseEngine : engine, + }, + ]); + for (const name of [ + "inspect_dart_aot", + "identify_flutter_build", + ] as const) { + const result = await cli.run({ + arguments: [name.replaceAll("_", "-"), path, "--json"], + }); + expect(result.exitCode, result.stderr).toBe(0); + const evidence = toolContract(name).outputSchema.parse(result.json); + const response = await call(name, { path }); + expect(response.isError, JSON.stringify(response)).not.toBe(true); + const mcp = parseEvidence(response.structuredContent); + expect(mcp.normalized_result).toEqual(evidence.normalized_result); + expect(mcp.evidence_id).toBe(evidence.evidence_id); + expect(mcp.subject).toEqual(evidence.subject); + expect(mcp.limitations).toEqual(evidence.limitations); + expect(evidence.normalized_result.coverage).toBe( + conflict ? "partial" : "complete", + ); + if (name === "inspect_dart_aot") { + const facts = flutterResultSchemas[name].parse( + evidence.normalized_result, + ); + expect(facts.libapp.sections).toHaveLength(conflict ? 3 : 4); + expect(facts.string_pool.coverage).toBe( + conflict ? "partial" : "complete", + ); + if (conflict) { + expect(facts.string_pool.package_uris).toEqual([]); + expect(evidence.limitations.join("\n")).toContain( + "conflicting PT_LOAD", + ); + } + } else { + const facts = flutterResultSchemas[name].parse( + evidence.normalized_result, + ); + expect(facts.abis[0]?.libflutter).toMatchObject({ + build_id: conflict ? null : "0123456789abcdef", + build_id_candidates: ["0123456789abcdef"], + }); + if (conflict) + expect(evidence.limitations.join("\n")).toContain("note-range"); + } + } + } + }, +); diff --git a/tests/acceptance/wasmInterfaceComparison.test.ts b/tests/acceptance/wasmInterfaceComparison.test.ts new file mode 100644 index 000000000..77f6c4319 --- /dev/null +++ b/tests/acceptance/wasmInterfaceComparison.test.ts @@ -0,0 +1,130 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import { Ajv2020 } from "ajv/dist/2020.js"; +import { expect } from "vitest"; +import { createEvidence, parseEvidence } from "../../src/domain/evidence.js"; +import { wasmInterfaceComparisonSchema } from "../../src/contracts/wasm/wasmComparisonContracts.js"; +import { composeBinarySession } from "../../src/application/binary/BinarySessionComposition.js"; +import { AnalysisProviderRegistry } from "../../src/application/binary/AnalysisProviderRegistry.js"; +import { createServer } from "../../src/server/createServer.js"; +import { wasmArtifactFixture } from "../fixtures/wasm/artifact.js"; +import { cliTest } from "../support/cli/cliFixture.js"; +import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js"; + +cliTest( + "compares portable Evidence identically through CLI and retained MCP with schema/provenance", + async ({ cli }) => { + const sources = (["i32", "i64"] as const).map((parameter) => { + const report = wasmArtifactFixture(); + report.interface = { + state: "complete", + issues: [], + entries: [ + { + direction: "export", + module: null, + name: 'quoted"\ufeffname', + kind: "func", + index: 0, + signature: { kind: "func", params: [parameter], results: [] }, + wat_form: '(export "quoted\\22\\ef\\bb\\bfname" (func 0))', + issue: null, + }, + ], + }; + return createEvidence( + { + path: report.artifact.path, + sha256: report.artifact.sha256, + format: "file", + }, + { id: "typed-fixture", name: "Typed fixture", version: "1" }, + { + operation: "inspect_wasm_artifact", + parameters: { path: report.artifact.path }, + result: report, + confidence: "observed", + }, + ); + }); + const [left, right] = sources; + if (left === undefined || right === undefined) + throw new Error("Missing paired fixture"); + const root = await createTestTempDirectory("rea-wasm-comparison-"); + const path = join(root, "input.json"); + const input = { left, right, scope: "both" }; + await writeFile(path, JSON.stringify(input)); + const cliResult = await cli.run({ + arguments: ["compare-wasm-interfaces", path, "--json"], + }); + expect(cliResult.exitCode).toBe(0); + const cliEvidence = parseEvidence(cliResult.json); + const cliReport = wasmInterfaceComparisonSchema.parse( + cliEvidence.normalized_result, + ); + expect(cliReport).toMatchObject({ + coverage: "complete-within-inputs", + changes: [{ name: 'quoted"\ufeffname', state: "changed" }], + left: { + evidence_id: left.evidence_id, + artifact: { + path: "/artifacts/module.wasm", + sha256: "a".repeat(64), + bytes: 8, + }, + }, + }); + const session = composeBinarySession(new AnalysisProviderRegistry([])); + const server = createServer({ kind: "session", session }); + const client = new Client({ name: "wasm-interface-parity", version: "1" }); + const [ct, st] = InMemoryTransport.createLinkedPair(); + try { + await server.connect(st); + await client.connect(ct); + const tool = (await client.listTools()).tools.find( + (t) => t.name === "compare_wasm_interfaces", + ); + if (tool?.outputSchema === undefined) + throw new Error("Missing comparison schema"); + const outputSchema: Record = tool.outputSchema; + const ajv = new Ajv2020({ strict: false, validateFormats: false }); + expect(ajv.validate(tool.inputSchema, input)).toBe(true); + expect(ajv.validate(tool.inputSchema, { left, right })).toBe(false); + const mcp = await client.callTool({ name: tool.name, arguments: input }); + expect(mcp.isError, JSON.stringify(mcp)).not.toBe(true); + expect( + ajv.validate(outputSchema, mcp.structuredContent), + JSON.stringify(ajv.errors), + ).toBe(true); + const mcpEvidence = parseEvidence(mcp.structuredContent); + expect(mcpEvidence.normalized_result).toEqual(cliReport); + expect(mcpEvidence.evidence_id).toBe(cliEvidence.evidence_id); + expect(mcpEvidence.evidence_links).toEqual([ + left.evidence_id, + right.evidence_id, + ]); + const retained = await client.callTool({ + name: tool.name, + arguments: { + scope: "both", + left: { kind: "retained-evidence", evidence_id: left.evidence_id }, + right: { kind: "retained-evidence", evidence_id: right.evidence_id }, + }, + }); + expect(retained.isError).not.toBe(true); + expect(parseEvidence(retained.structuredContent).evidence_id).toBe( + mcpEvidence.evidence_id, + ); + const invalid = await client.callTool({ + name: tool.name, + arguments: { ...input, scope: "all" }, + }); + expect(invalid.isError).toBe(true); + } finally { + await client.close(); + await server.close(); + await session.close(); + } + }, +); diff --git a/tests/boundary/android/JadxAnalysis.test.ts b/tests/boundary/android/JadxAnalysis.test.ts new file mode 100644 index 000000000..36b0ab9d9 --- /dev/null +++ b/tests/boundary/android/JadxAnalysis.test.ts @@ -0,0 +1,304 @@ +import { expect, it } from "vitest"; + +import { + analyzeJadxRequest, + type JadxToolPort, +} from "../../../src/android/JadxAnalysis.js"; +import { androidInputSchemas } from "../../../src/domain/android/androidAnalysis.js"; + +const summary = { + full_name: "fixture.Target", + name: "Target", + method_count: 2, + field_count: 0, + inner_class_count: 0, + methods: [ + { + name: "choose", + signature: "choose(String): void", + dex_descriptor: "(Ljava/lang/String;)V", + is_constructor: false, + def_pos: 0, + }, + { + name: "choose", + signature: "choose(int): void", + dex_descriptor: "(I)V", + is_constructor: false, + def_pos: 0, + }, + ], + fields: [], + inner_classes: [], +}; + +const engine = { + name: "jadx", + version: "0.7.1", + artifact_sha256: "a".repeat(64), + source_revision: null, + worker_count: 1, + heap_limit_mib: null, +}; + +it("publishes exact DEX prototypes instead of discarding bridge metadata", async () => { + const tools: JadxToolPort = { + json: () => Promise.resolve(summary), + text: () => Promise.reject(new Error("unexpected text call")), + }; + const result = await analyzeJadxRequest( + tools, + { + operation: "inspect_android_class", + input: androidInputSchemas.inspect_android_class.parse({ + path: "/fixture.apk", + class_name: "fixture.Target", + }), + }, + {} as never, + engine, + ); + expect(result).toMatchObject({ + methods: [ + { name: "choose", dex_descriptor: "(Ljava/lang/String;)V" }, + { name: "choose", dex_descriptor: "(I)V" }, + ], + }); +}); + +it("traces only the selected overloaded method and checks the producer identity", async () => { + const calls: Record[] = []; + const tools: JadxToolPort = { + json: (name, input) => { + calls.push({ name, ...input }); + if (name === "get_class_summary") return Promise.resolve(summary); + return Promise.resolve({ + target: "fixture.Target.choose", + overload_index: 1, + reported_signature: "choose(int): void", + total: 1, + count: 1, + resolve_line: false, + items: [ + { + kind: "method", + name: "caller", + full_name: "fixture.Other.caller", + containing_class: "fixture.Other", + top_class: "fixture.Other", + def_pos: 0, + direct_invokes: [ + { + dex_entry: "classes.dex", + source_raw_method_id: "fixture.Other.caller()V", + dex_code_unit_offset: 0, + dex_method_index: 1, + opcode: "INVOKE_STATIC", + }, + { + dex_entry: "classes.dex", + source_raw_method_id: "fixture.Other.caller()V", + dex_code_unit_offset: 3, + dex_method_index: 1, + opcode: "INVOKE_STATIC", + }, + ], + }, + ], + }); + }, + text: () => Promise.reject(new Error("unexpected text call")), + }; + const request = { + operation: "trace_android_references", + input: androidInputSchemas.trace_android_references.parse({ + path: "/fixture.apk", + class_name: "fixture.Target", + method_name: "choose", + overload_index: 1, + }), + } as const; + const result = await analyzeJadxRequest(tools, request, {} as never, engine); + expect(result).toMatchObject({ + method_name: "choose", + overload_index: 1, + reported_signature: "choose(int): void", + dex_descriptor: "(I)V", + total: 1, + reference_unit: "using_node", + references: [ + { + direct_invokes: [ + { dex_code_unit_offset: 0 }, + { dex_code_unit_offset: 3 }, + ], + }, + ], + }); + expect(calls[1]).toMatchObject({ + name: "get_xrefs_to_method", + overload_index: 1, + }); +}); + +it("rejects an ambiguous method and a producer response for a different overload", async () => { + const xrefs = { + target: "fixture.Target.choose", + overload_index: 0, + reported_signature: "choose(String): void", + total: 0, + count: 0, + resolve_line: false, + items: [], + }; + const tools: JadxToolPort = { + json: (name) => + Promise.resolve(name === "get_class_summary" ? summary : xrefs), + text: () => Promise.reject(new Error("unexpected text call")), + }; + const input = { + path: "/fixture.apk", + class_name: "fixture.Target", + method_name: "choose", + }; + await expect( + analyzeJadxRequest( + tools, + { + operation: "trace_android_references", + input, + }, + {} as never, + engine, + ), + ).rejects.toMatchObject({ + _tag: "AnalysisInputError", + }); + await expect( + analyzeJadxRequest( + tools, + { + operation: "trace_android_references", + input: { ...input, overload_index: 1 }, + }, + {} as never, + engine, + ), + ).rejects.toMatchObject({ + _tag: "AnalysisOutputError", + }); +}); + +const invokeSite = { + dex_entry: "classes.dex", + source_raw_method_id: "fixture.Other.caller()V", + dex_code_unit_offset: 0, + dex_method_index: 1, + opcode: "INVOKE_STATIC", +}; +const traceWithReference = (fields: Record) => + analyzeJadxRequest( + { + json: (name) => + Promise.resolve( + name === "get_class_summary" + ? summary + : { + target: "fixture.Target.choose", + overload_index: 0, + reported_signature: "choose(String): void", + total: 1, + count: 1, + resolve_line: false, + items: [ + { + kind: "method", + name: "caller", + full_name: "fixture.Other.caller", + containing_class: "fixture.Other", + top_class: "fixture.Other", + def_pos: 0, + ...fields, + }, + ], + }, + ), + text: () => Promise.reject(new Error("unexpected text call")), + }, + { + operation: "trace_android_references", + input: { + path: "/fixture.apk", + class_name: "fixture.Target", + method_name: "choose", + overload_index: 0, + }, + }, + {} as never, + engine, + ); + +it.each([ + [{}, null], + [{ direct_invokes: [] }, []], +])( + "preserves unavailable versus observed-empty invoke sites: %j", + async (fields, expected) => { + const result = await traceWithReference(fields as Record); + expect(result).toMatchObject({ + references: [{ direct_invokes: expected }], + }); + }, +); + +it.each([ + { ...invokeSite, dex_code_unit_offset: -1 }, + { + dex_entry: "classes.dex", + source_raw_method_id: "fixture.Other.caller()V", + dex_code_unit_offset: 0, + dex_method_index: 1, + }, +])("rejects malformed direct invoke producer evidence: %j", async (site) => { + await expect( + traceWithReference({ direct_invokes: [site] }), + ).rejects.toThrow(); +}); + +it("preserves bounded instruction coverage separately from using-node coverage", async () => { + const result = await traceWithReference({ + direct_invokes: [invokeSite], + direct_invokes_exhaustive: false, + }); + expect(result).toMatchObject({ + coverage: "complete", + reference_unit: "using_node", + references: [ + { direct_invokes: [invokeSite], direct_invokes_exhaustive: false }, + ], + }); +}); + +it("rejects malformed method descriptors at the Android adapter boundary", async () => { + const invalidSummary = { + ...summary, + methods: summary.methods.map((method) => ({ + ...method, + dex_descriptor: "not-a-dex-descriptor", + })), + }; + await expect( + analyzeJadxRequest( + { + json: () => Promise.resolve(invalidSummary), + text: () => Promise.reject(new Error("unexpected text call")), + }, + { + operation: "inspect_android_class", + input: { path: "/fixture.apk", class_name: "fixture.Target" }, + }, + {} as never, + engine, + ), + ).rejects.toThrow(); +}); diff --git a/tests/boundary/android/workspaceGuardianRetry.test.ts b/tests/boundary/android/workspaceGuardianRetry.test.ts new file mode 100644 index 000000000..fda07135c --- /dev/null +++ b/tests/boundary/android/workspaceGuardianRetry.test.ts @@ -0,0 +1,60 @@ +import { spawn } from "node:child_process"; +import { readFile } from "node:fs/promises"; + +import { expect, it } from "vitest"; + +it.skipIf(process.platform !== "win32")( + "retains guardian cleanup ownership through more than 100 transient failures", + async () => { + const guardian = await readFile( + new URL( + "../../../scripts/rea-android-workspace-guardian.mjs", + import.meta.url, + ), + "utf8", + ); + const withoutTimer = guardian + .replace(/^#!.*\r?\n/u, "") + .replace( + 'import { setTimeout as delay } from "node:timers/promises";', + "const delay = async () => {};", + ); + const injected = withoutTimer.replace( + 'import { WindowsPrivateRuntime } from "../dist/windows/WindowsPrivateRuntime.js";', + `let attempts = 0; + const WindowsPrivateRuntime = { + create: () => ({ + observation: { path: "synthetic-root", volumeSerial: "0", fileId: "0", privateDacl: true }, + close: async () => { if (++attempts <= 101) throw new Error("transient cleanup failure"); }, + }), + };`, + ); + expect(withoutTimer).not.toBe(guardian); + expect(injected).not.toBe(withoutTimer); + const child = spawn( + process.execPath, + [ + "--input-type=module", + "-e", + 'process.argv.splice(1, 0, "guardian.mjs");\n' + injected, + "synthetic-parent", + ], + { signal: AbortSignal.timeout(3000) }, + ); + let stdout = ""; + let stderr = ""; + child.stdout.on("data", (chunk: Buffer) => { + stdout += chunk.toString(); + }); + child.stderr.on("data", (chunk: Buffer) => { + stderr += chunk.toString(); + }); + child.stdin.end(); + const code = await new Promise((resolve, reject) => { + child.once("close", resolve); + child.once("error", reject); + }); + expect(code, stderr).toBe(0); + expect(JSON.parse(stdout.trim())).toMatchObject({ path: "synthetic-root" }); + }, +); diff --git a/tests/boundary/binaryDiagnostics/RecordedCrashStructureBindings.test.ts b/tests/boundary/binaryDiagnostics/RecordedCrashStructureBindings.test.ts index 5b98e4241..970f57f5d 100644 --- a/tests/boundary/binaryDiagnostics/RecordedCrashStructureBindings.test.ts +++ b/tests/boundary/binaryDiagnostics/RecordedCrashStructureBindings.test.ts @@ -4,6 +4,40 @@ import { recordedCrashFixtureBytes, } from "../../fixtures/binaryDiagnostics/recordedCrash.js"; import { validateRecordedCrashStructure } from "../../../src/native/pwntools/RecordedCrashStructureBindings.js"; +import { RECORDED_LINUX_CORE_PROFILES } from "../../../src/domain/native/recordedCrash.js"; + +it.each(["x86_64-little-endian", "aarch64-little-endian"] as const)( + "admits a %s core whose header carries that profile's e_machine", + (architecture) => { + const report = recordedCrashFixture(undefined, architecture); + expect(() => + validateRecordedCrashStructure( + report, + recordedCrashFixtureBytes(report), + report.decoder_diagnostics, + ), + ).not.toThrow(); + }, +); + +it.each([ + ["x86_64-little-endian", "aarch64-little-endian"], + ["aarch64-little-endian", "x86_64-little-endian"], +] as const)( + "rejects a %s report whose header carries the other profile's e_machine", + (reported, embedded) => { + const report = recordedCrashFixture(undefined, reported); + const snapshot = recordedCrashFixtureBytes(report); + snapshot.writeUInt16LE(RECORDED_LINUX_CORE_PROFILES[embedded].machine, 18); + expect(() => + validateRecordedCrashStructure( + report, + snapshot, + report.decoder_diagnostics, + ), + ).toThrow(/Reported core structure differs/); + }, +); it.each([ [6, 0], diff --git a/tests/boundary/binaryDiagnostics/recordedCrashProvider.test.ts b/tests/boundary/binaryDiagnostics/recordedCrashProvider.test.ts index de422334b..16c355789 100644 --- a/tests/boundary/binaryDiagnostics/recordedCrashProvider.test.ts +++ b/tests/boundary/binaryDiagnostics/recordedCrashProvider.test.ts @@ -1,5 +1,8 @@ import type { AnalysisError } from "../../../src/domain/analysisErrorBase.js"; -import type { RecordedCrash } from "../../../src/domain/native/recordedCrash.js"; +import type { + RecordedCrash, + RecordedCrashArchitecture, +} from "../../../src/domain/native/recordedCrash.js"; import { access, mkdir, readFile, writeFile } from "node:fs/promises"; import { join } from "node:path"; import { expect, it, onTestFinished } from "vitest"; @@ -48,6 +51,7 @@ it .runIf(host) .each([ "success", + "success-aarch64", "wrong-register", "wrong-register-source", "missing-thread", @@ -77,9 +81,13 @@ it ] as const)( "validates recorded source bytes and cleanup: %s", async (scenario) => { + const architecture = scenario.endsWith("aarch64") + ? "aarch64-little-endian" + : "x86_64-little-endian"; const { path, value, bytes } = await fixture( true, scenario === "wrong-owner-display", + architecture, ); if (scenario.startsWith("wrong-ei-version-")) bytes[6] = Number(scenario.at(-1)); @@ -123,8 +131,9 @@ it path, include_debugger_context: false, }); - if (scenario === "success") { + if (scenario === "success" || scenario === "success-aarch64") { if (!result.ok) throw result.error; + expect(result.value.architecture).toBe(architecture); expect(result.value.threads).toEqual(value.threads); expect(result.value.diagnostics.stdout).toBe("recorded warning"); } else @@ -340,13 +349,17 @@ const expectOptionalContextFailure = ( }); }; -const fixture = async (includeSignal = false, opaqueOwner = false) => { +const fixture = async ( + includeSignal = false, + opaqueOwner = false, + architecture: RecordedCrashArchitecture = "x86_64-little-endian", +) => { const workspace = await createTestWorkspace("rea-recorded-core-seam-"); onTestFinished(() => removeTestWorkspace(workspace.root)); const path = join(workspace.root, "recorded.core"); const value = includeSignal - ? recordedCrashSignalFixture(path) - : recordedCrashFixture(path); + ? recordedCrashSignalFixture(path, architecture) + : recordedCrashFixture(path, architecture); if (opaqueOwner) { value.notes = value.notes.map((note) => note.index === 0 diff --git a/tests/boundary/browser/browserProcessStartup.test.ts b/tests/boundary/browser/browserProcessStartup.test.ts index 91c4f58e9..b1cfaeafe 100644 --- a/tests/boundary/browser/browserProcessStartup.test.ts +++ b/tests/boundary/browser/browserProcessStartup.test.ts @@ -1,3 +1,4 @@ +import { readFile } from "node:fs/promises"; import { join } from "node:path"; import { describe, expect } from "vitest"; @@ -9,6 +10,7 @@ import { } from "../../support/process/processFixture.js"; import { + BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS, BrowserStartupError, waitForBrowserDevtoolsPort, } from "../../../src/browser/BrowserProcessStartup.js"; @@ -86,15 +88,68 @@ describe("browser process startup", () => { failure: "timeout", exitCode: null, signalCode: null, + startupTimeoutMs: 20, }); expect(String(failure)).toContain("stderr="); + expect(String(failure)).toContain("startup_timeout_ms=20"); } finally { child.kill("SIGKILL"); expect(await waitForExit(child, 5_000)).toBe(true); } }); + + it("names the owner's deadline when a caller omits one", async ({ + processes, + }) => { + const root = await temporaryRoot(); + const child = processes.spawn(process.execPath, [ + "-e", + "setTimeout(() => process.exit(3), 10)", + ]); + + const failure = await waitForBrowserDevtoolsPort({ + child, + executable: process.execPath, + activePortPath: join(root, "DevToolsActivePort"), + stderr: () => "owner default fixture", + pollIntervalMs: 5, + }).catch((cause: unknown) => cause); + + expect(failure).toBeInstanceOf(BrowserStartupError); + expect(failure).toMatchObject({ + failure: "exited", + startupTimeoutMs: BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS, + }); + expect(String(failure)).toContain( + `startup_timeout_ms=${String(BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS)}`, + ); + }); + + it("keeps one owner for the DevToolsActivePort deadline", async () => { + expect(BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS).toBe(60_000); + const callers = [ + "../../../scripts/verify-real-browser.mjs", + "../../../scripts/lib/browser-runtime-fixture-lifecycle.mjs", + ]; + for (const caller of callers) { + const source = await readFile(new URL(caller, import.meta.url), "utf8"); + expect(devtoolsPortDeadlines(source)).toEqual([ + ["BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS"], + ]); + } + }); }); const temporaryRoot = async (): Promise => { return createTestTempDirectory("rea-browser-startup-"); }; + +const devtoolsPortDeadlines = (source: string): string[][] => { + const calls = /waitForBrowserDevtoolsPort\(\{(?[\s\S]*?)\}\)/gu; + const deadlines = /\btimeoutMs:\s*(?[^\n,]+)/gu; + return [...source.matchAll(calls)].map((call) => + [...String(call.groups?.body).matchAll(deadlines)].map((deadline) => + String(deadline.groups?.value).trim(), + ), + ); +}; diff --git a/tests/boundary/browser/cdpBrowserProvider.browserSocket.test.ts b/tests/boundary/browser/cdpBrowserProvider.browserSocket.test.ts new file mode 100644 index 000000000..f04b8210d --- /dev/null +++ b/tests/boundary/browser/cdpBrowserProvider.browserSocket.test.ts @@ -0,0 +1,232 @@ +import { createServer, type Socket } from "node:net"; + +import { describe, expect, it, vi } from "vitest"; + +import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js"; +import { openCdpEndpoint } from "../../../src/browser/CdpTargetSession.js"; +import { AnalysisError } from "../../../src/domain/analysisErrorBase.js"; +import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; +import { + browserObservationEndpointSchema, + inspectWebPageInputSchema, + listBrowserTargetsInputSchema, +} from "../../../src/domain/browserObservation.js"; +import { startFakeCdpBrowser } from "../../fixtures/fakeCdpBrowser.js"; +import { trackBrowser } from "./cdpBrowserProvider.support.js"; + +describe("CdpBrowserProvider: browser socket without HTTP discovery", () => { + it("discovers and inspects a page through one browser socket", async () => { + const browser = await startFakeCdpBrowser(); + trackBrowser(browser); + const provider = new CdpBrowserProvider(); + const throughHttp = await provider.listTargets( + listBrowserTargetsInputSchema.parse({ cdp_endpoint: browser.endpoint }), + ); + if (!throughHttp.ok) throw throughHttp.error; + const httpRequests = browser.httpRequests.length; + const openedBefore = browser.openedConnectionCount(); + + const listed = await provider.listTargets( + listBrowserTargetsInputSchema.parse({ + cdp_endpoint: browser.browserWebSocketUrl, + }), + ); + if (!listed.ok) throw listed.error; + expect(listed.value.targets).toEqual(throughHttp.value.targets); + expect(listed.value.excluded).toEqual(throughHttp.value.excluded); + expect(listed.value.browser).toEqual({ + ...throughHttp.value.browser, + revision: "@fake-revision", + }); + + const commandsBefore = browser.commands.length; + const inspected = await provider.inspectPage( + inspectWebPageInputSchema.parse({ + cdp_endpoint: browser.browserWebSocketUrl, + target_id: "allowed-page", + observation_ms: 0, + }), + ); + if (!inspected.ok) throw inspected.error; + expect(inspected.value.target.origin).toBe(browser.allowedOrigin); + expect( + browser.commands + .slice(commandsBefore, commandsBefore + 3) + .map(({ method, params }) => ({ method, params })), + ).toEqual([ + { method: "Browser.getVersion", params: {} }, + { method: "Target.getTargets", params: {} }, + { + method: "Target.attachToTarget", + params: { targetId: "allowed-page", flatten: true }, + }, + ]); + expect(browser.httpRequests).toHaveLength(httpRequests); + expect(browser.openedConnectionCount()).toBe(openedBefore + 2); + await vi.waitFor(() => expect(browser.activeConnectionCount()).toBe(0)); + }); +}); + +describe("CdpBrowserProvider: browser socket failures and cleanup", () => { + it("closes an unadopted browser socket when the target is missing", async () => { + const browser = await startFakeCdpBrowser(); + trackBrowser(browser); + const result = await new CdpBrowserProvider().inspectPage( + inspectWebPageInputSchema.parse({ + cdp_endpoint: browser.browserWebSocketUrl, + target_id: "missing", + observation_ms: 0, + }), + ); + + expect(result).toMatchObject({ + ok: false, + error: { _tag: "BrowserObservationError", reason: "target_not_found" }, + }); + expect(browser.commands.map(({ method }) => method)).toEqual([ + "Browser.getVersion", + "Target.getTargets", + ]); + await vi.waitFor(() => expect(browser.activeConnectionCount()).toBe(0)); + }); + + it("rejects malformed browser-socket discovery and closes the socket", async () => { + const browser = await startFakeCdpBrowser({ + commandResult: ({ method }) => + method === "Browser.getVersion" + ? { product: "FakeChrome/1.0" } + : undefined, + }); + trackBrowser(browser); + const result = await new CdpBrowserProvider().listTargets( + listBrowserTargetsInputSchema.parse({ + cdp_endpoint: browser.browserWebSocketUrl, + }), + ); + + expect(result).toMatchObject({ + ok: false, + error: { + _tag: "BrowserObservationError", + operation: "list_browser_targets", + reason: "invalid_endpoint_response", + }, + }); + await vi.waitFor(() => expect(browser.activeConnectionCount()).toBe(0)); + }); + + it("reports a refused browser socket as unreachable", async () => { + const browser = await startFakeCdpBrowser(); + trackBrowser(browser); + const result = await new CdpBrowserProvider().listTargets( + listBrowserTargetsInputSchema.parse({ + cdp_endpoint: browser.browserWebSocketUrl.replace( + /fake$/u, + "not-this-browser", + ), + }), + ); + + expect(result).toMatchObject({ + ok: false, + error: { + _tag: "BrowserObservationError", + reason: "endpoint_unreachable", + }, + }); + expect(JSON.stringify(result)).not.toContain("not-this-browser"); + }); + + it("cancels browser-socket discovery and releases the socket", async () => { + const browser = await startFakeCdpBrowser({ + hangOnMethod: "Target.getTargets", + }); + trackBrowser(browser); + const controller = new AbortController(); + const pending = new CdpBrowserProvider().inspectPage( + inspectWebPageInputSchema.parse({ + cdp_endpoint: browser.browserWebSocketUrl, + target_id: "allowed-page", + observation_ms: 0, + }), + { signal: controller.signal }, + ); + await vi.waitFor(() => + expect(browser.commands.map(({ method }) => method)).toContain( + "Target.getTargets", + ), + ); + controller.abort(); + + expect(await pending).toMatchObject({ + ok: false, + error: { _tag: "AnalysisCancelledError", operation: "inspect_web_page" }, + }); + expect(browser.commands.map(({ method }) => method)).not.toContain( + "Target.attachToTarget", + ); + await vi.waitFor(() => expect(browser.activeConnectionCount()).toBe(0)); + }); +}); + +/** Expect a discovery timeout that projects publicly as a provider timeout. */ +const expectDiscoveryTimeout = async (url: string): Promise => { + let failure: unknown; + try { + await openCdpEndpoint( + browserObservationEndpointSchema.parse(url), + "list_browser_targets", + undefined, + 50, + ); + } catch (cause: unknown) { + failure = cause; + } + expect(failure).toMatchObject({ + _tag: "BrowserObservationError", + reason: "timeout", + }); + if (!(failure instanceof AnalysisError)) + throw new Error("Expected a typed discovery failure"); + expect(projectAnalysisError(failure)).toMatchObject({ + code: "provider_timeout", + category: "timeout", + }); +}; + +describe("CdpBrowserProvider: browser socket discovery deadline", () => { + it("times out a discovery reply that never arrives and closes the socket", async () => { + const browser = await startFakeCdpBrowser({ + hangOnMethod: "Target.getTargets", + }); + trackBrowser(browser); + await expectDiscoveryTimeout(browser.browserWebSocketUrl); + await vi.waitFor(() => expect(browser.activeConnectionCount()).toBe(0)); + }); + + it("times out a WebSocket handshake that never completes", async () => { + const sockets = new Set(); + // Read the upgrade request but never answer it; reading lets the + // client's close be observed here. + const server = createServer((socket) => { + sockets.add(socket); + socket.on("close", () => sockets.delete(socket)); + socket.resume(); + }); + await new Promise((resolve) => + server.listen(0, "127.0.0.1", resolve), + ); + const address = server.address(); + if (address === null || typeof address === "string") + throw new Error("Expected a TCP listener address"); + try { + await expectDiscoveryTimeout( + `ws://127.0.0.1:${String(address.port)}/devtools/browser/stalled`, + ); + await vi.waitFor(() => expect(sockets.size).toBe(0)); + } finally { + for (const socket of sockets) socket.destroy(); + await new Promise((resolve) => server.close(() => resolve())); + } + }); +}); diff --git a/tests/boundary/browser/cdpBrowserProvider.targetInventory.test.ts b/tests/boundary/browser/cdpBrowserProvider.targetInventory.test.ts index 81fe60759..9b2053d39 100644 --- a/tests/boundary/browser/cdpBrowserProvider.targetInventory.test.ts +++ b/tests/boundary/browser/cdpBrowserProvider.targetInventory.test.ts @@ -3,6 +3,7 @@ import { createServer } from "node:http"; import { describe, expect, it } from "vitest"; import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js"; +import { listBrowserTargetsInputSchema } from "../../../src/domain/browserObservation.js"; describe("CDP target inventory", () => { it("returns every allowed target within the bounded discovery payload", async () => { @@ -51,10 +52,12 @@ describe("CDP target inventory", () => { port = address.port; const endpoint = `http://127.0.0.1:${String(port)}`; - const result = await new CdpBrowserProvider().listTargets({ - cdp_endpoint: endpoint, - allowed_origins: ["https://allowed.example"], - }); + const result = await new CdpBrowserProvider().listTargets( + listBrowserTargetsInputSchema.parse({ + cdp_endpoint: endpoint, + allowed_origins: ["https://allowed.example"], + }), + ); if (!result.ok) throw result.error; expect(result.value.targets).toHaveLength(targets.length); diff --git a/tests/boundary/browser/cdpHttpComparison.test.ts b/tests/boundary/browser/cdpHttpComparison.test.ts new file mode 100644 index 000000000..2f4ad689c --- /dev/null +++ b/tests/boundary/browser/cdpHttpComparison.test.ts @@ -0,0 +1,255 @@ +import { describe, expect, it } from "vitest"; + +import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js"; +import { inspectWebPageInputSchema } from "../../../src/domain/browserObservation.js"; +import type { WebPageInspection } from "../../../src/domain/browserObservationSchemas.js"; +import { compareWebCaptures } from "../../../src/domain/webCaptureDiff.js"; +import { compareWebCapturesInputSchema } from "../../../src/domain/webCaptureDiffSchemas.js"; +import type { FakeCdpBrowser } from "../../fixtures/fakeCdpBrowser.js"; +import { + startHttpComparisonBrowser, + type HttpRequestObservation, +} from "../../fixtures/httpComparisonBrowser.js"; +import { trackBrowser } from "./cdpBrowserProvider.support.js"; + +const inspect = async (browser: FakeCdpBrowser): Promise => { + const result = await new CdpBrowserProvider().inspectPage( + inspectWebPageInputSchema.parse({ + cdp_endpoint: browser.endpoint, + allowed_origins: [browser.allowedOrigin], + target_id: "allowed-page", + observation_ms: 0, + }), + ); + if (!result.ok) throw result.error; + return result.value; +}; + +const capturePair = async ( + before: readonly HttpRequestObservation[], + after: readonly HttpRequestObservation[], +): Promise => { + const fixture = await startHttpComparisonBrowser(before); + trackBrowser(fixture.browser); + const left = await inspect(fixture.browser); + fixture.setRequests(after); + return [left, await inspect(fixture.browser)]; +}; + +const compareNetwork = (before: WebPageInspection, after: WebPageInspection) => + compareWebCaptures( + compareWebCapturesInputSchema.parse({ + before: { inspection: before }, + after: { inspection: after }, + }), + ).dimensions.network; + +const response = { + requestId: "request-before", + status: 200, + mimeType: "application/json", +} as const; + +describe("HTTP comparison response coverage", () => { + it("does not compare an unavailable byte count with a completed response", async () => { + const [before, after] = await capturePair( + [response], + [{ ...response, requestId: "request-after", encodedDataLength: 100 }], + ); + expect(before.network.requests[0]?.encoded_data_length).toBeNull(); + expect(after.network.requests[0]?.encoded_data_length).toBe(100); + for (const [left, right] of [ + [before, after], + [after, before], + ] as const) + expect(compareNetwork(left, right)).toMatchObject({ + status: "unknown", + total_changes: 0, + changes: [], + }); + }); + + it("does not treat an unobserved response as a changed response", async () => { + const [before, after] = await capturePair( + [{ requestId: "request-before" }], + [{ ...response, encodedDataLength: 100 }], + ); + expect(before.network.requests).toEqual([ + expect.objectContaining({ + status: null, + mime_type: null, + encoded_data_length: null, + }), + ]); + for (const [left, right] of [ + [before, after], + [after, before], + ] as const) + expect(compareNetwork(left, right)).toMatchObject({ + status: "unknown", + total_changes: 0, + }); + }); + + it("retains observed status changes when response bytes are unavailable", async () => { + const [before, after] = await capturePair( + [response], + [{ ...response, status: 500 }], + ); + expect(compareNetwork(before, after)).toMatchObject({ + status: "changed", + changes: [expect.objectContaining({ change: "modified" })], + }); + }); + + it.each([ + { mimeType: "text/plain", encodedDataLength: 100 }, + { mimeType: "application/json", encodedDataLength: 200 }, + ])("retains observed response changes: %j", async (change) => { + const [before, after] = await capturePair( + [{ ...response, encodedDataLength: 100 }], + [{ ...response, ...change }], + ); + expect(compareNetwork(before, after).status).toBe("changed"); + }); +}); + +describe("HTTP comparison repeated request coverage", () => { + it("does not assign missing evidence by request order or transient ID", async () => { + const [before, after] = await capturePair( + [ + { ...response, encodedDataLength: 100 }, + { ...response, requestId: "second-before" }, + ], + [ + { ...response, requestId: "second-after", encodedDataLength: 200 }, + { ...response, requestId: "first-after", encodedDataLength: 100 }, + ], + ); + expect(before.network.requests).toHaveLength(2); + expect(after.network.requests).toHaveLength(2); + expect(compareNetwork(before, after)).toMatchObject({ + status: "unknown", + total_changes: 0, + }); + }); + + it("compares complete byte counts even when another field is incomplete", async () => { + const [before, after] = await capturePair( + [ + { ...response, encodedDataLength: 100 }, + { requestId: "second-before", encodedDataLength: 200 }, + ], + [ + { requestId: "second-after", encodedDataLength: 300 }, + { ...response, requestId: "first-after", encodedDataLength: 100 }, + ], + ); + expect(compareNetwork(before, after).status).toBe("changed"); + }); + + it("preserves the association between complete response fields", async () => { + const [before, after] = await capturePair( + [ + { ...response, encodedDataLength: 100 }, + { + ...response, + requestId: "second-before", + status: 201, + encodedDataLength: 200, + }, + ], + [ + { ...response, requestId: "first-after", encodedDataLength: 200 }, + { + ...response, + requestId: "second-after", + status: 201, + encodedDataLength: 100, + }, + ], + ); + expect(compareNetwork(before, after).status).toBe("changed"); + }); +}); + +it.each([ + { + name: "a pending request in an empty inventory", + before: [], + after: [{ requestId: "pending" }], + }, + { + name: "a known request beside an existing pending request", + before: [{ requestId: "pending" }], + after: [ + { requestId: "pending" }, + { + ...response, + requestId: "new", + path: "/api/new", + encodedDataLength: 100, + }, + ], + }, +])("preserves additions and removals of $name", async (observations) => { + const [before, after] = await capturePair( + observations.before, + observations.after, + ); + // Live CDP observations have attach-limited inventories. Their absence + // remains unknown even when a request is seen in the other capture. + for (const [left, right] of [ + [before, after], + [after, before], + ] as const) + expect(compareNetwork(left, right)).toMatchObject({ + status: "unknown", + total_changes: 0, + changes: [], + }); + + // Project explicitly complete inventory coverage without changing the + // producer's request facts; response fields remain independently unknown. + markNetworkInventoriesComplete(before, after); + for (const [left, right, change] of [ + [before, after, "added"], + [after, before, "removed"], + ] as const) + expect(compareNetwork(left, right)).toMatchObject({ + status: "changed", + total_changes: 1, + changes: [expect.objectContaining({ change })], + }); +}); + +it("requires response field coverage even when capture sections are complete", async () => { + const [before, after] = await capturePair( + [response], + [{ ...response, encodedDataLength: 100 }], + ); + markNetworkInventoriesComplete(before, after); + expect(compareNetwork(before, before).status).toBe("unknown"); + expect(compareNetwork(before, after).status).toBe("unknown"); + expect(compareNetwork(after, after).status).toBe("unchanged"); +}); + +const markNetworkInventoriesComplete = ( + ...inspections: WebPageInspection[] +): void => { + const sections = new Set([ + "network_requests", + "websocket_connections", + "websocket_frames", + ]); + for (const inspection of inspections) + for (const key of [ + "policy_filtered_sections", + "attach_limited_sections", + "truncated_sections", + "unavailable_sections", + ] as const) + inspection.completeness[key] = inspection.completeness[key].filter( + (section) => !sections.has(section), + ); +}; diff --git a/tests/boundary/browser/optionalSourceMapRequire.test.ts b/tests/boundary/browser/optionalSourceMapRequire.test.ts new file mode 100644 index 000000000..e7e861d77 --- /dev/null +++ b/tests/boundary/browser/optionalSourceMapRequire.test.ts @@ -0,0 +1,69 @@ +import { createHash } from "node:crypto"; +import { createServer } from "node:http"; + +import { expect, it } from "vitest"; + +import { fetchWebSourceMaps } from "../../../src/browser/WebSourceMapFetcher.js"; +import { analyzeWebBundleInputSchema } from "../../../src/domain/webBundleAnalysis.js"; + +it("retains unshadowed optional require dependencies from HTTP source-map originals", async () => { + const source = [ + 'require("./normal.cjs");', + 'require?.("./optional.cjs");', + 'function local(require) { require?.("./shadowed.cjs"); }', + "require?.(selectedModule);", + ].join("\n"); + const text = JSON.stringify({ + version: 3, + names: [], + mappings: "AAAA", + sources: ["main.cjs"], + sourcesContent: [source], + }); + const server = createServer((_request, response) => response.end(text)); + try { + await new Promise((resolve, reject) => { + server.once("error", reject); + server.listen(0, "127.0.0.1", resolve); + }); + const address = server.address(); + if (address === null || typeof address === "string") + throw new TypeError("Expected a TCP listener address"); + const origin = `http://127.0.0.1:${String(address.port)}`; + const url = `${origin}/app.js.map`; + const result = await fetchWebSourceMaps( + [{ scriptKey: `scr_${"1".repeat(64)}`, declaredUrl: url, fetchUrl: url }], + analyzeWebBundleInputSchema.parse({ + cdp_endpoint: origin, + target_id: "fixture", + allowed_origins: [origin], + fetch_source_maps: true, + }), + ); + expect(result.items[0]).toMatchObject({ + status: "included", + artifact: { + text, + bytes: Buffer.byteLength(text), + sha256: createHash("sha256").update(text).digest("hex"), + }, + original_sources: [{ artifact: { text: source } }], + original_module_edges: ["./normal.cjs", "./optional.cjs"].map( + (specifier) => ({ + from_source: `${origin}/main.cjs`, + kind: "require", + specifier, + resolved_source: new URL(specifier, origin).href, + }), + ), + }); + } finally { + server.closeAllConnections(); + if (server.listening) + await new Promise((resolve, reject) => { + server.close((error) => + error === undefined ? resolve() : reject(error), + ); + }); + } +}); diff --git a/tests/boundary/browser/webCaptureDiff.test.ts b/tests/boundary/browser/webCaptureDiff.test.ts index 5c2ce8785..3680f7dfa 100644 --- a/tests/boundary/browser/webCaptureDiff.test.ts +++ b/tests/boundary/browser/webCaptureDiff.test.ts @@ -446,91 +446,50 @@ describe("web capture WebSocket payload byte coverage", () => { }); describe("web capture diff", () => { - it("treats unavailable HTTP response facts as unknown while retaining observed changes", async () => { - const browser = await startFakeCdpBrowser(); - browsers.push(browser); - const captured = await new CdpBrowserProvider().inspectPage( - inspectWebPageInputSchema.parse({ - cdp_endpoint: browser.endpoint, - allowed_origins: [browser.allowedOrigin], - target_id: "allowed-page", - observation_ms: 0, - }), - ); - if (!captured.ok) throw captured.error; - const before = structuredClone(captured.value); - const request = before.network.requests[0]; - if (request === undefined) throw new Error("Missing network request"); - request.status = null; - request.mime_type = null; - request.encoded_data_length = null; - markSectionsComplete(before, [ - "network_requests", - "websocket_connections", - "websocket_frames", - "json_body_shapes", - ]); - for (const item of before.network.requests) - item.body_shapes.status = "not_approved"; + it.each(["status", "mime_type"] as const)( + "keeps an independently unavailable HTTP %s unknown", + async (field) => { + const browser = await startFakeCdpBrowser(); + browsers.push(browser); + const captured = await new CdpBrowserProvider().inspectPage( + inspectWebPageInputSchema.parse({ + cdp_endpoint: browser.endpoint, + allowed_origins: [browser.allowedOrigin], + target_id: "allowed-page", + observation_ms: 0, + }), + ); + if (!captured.ok) throw captured.error; + const observed = structuredClone(captured.value); + const request = observed.network.requests[0]; + if (request === undefined) throw new Error("Missing network request"); + request.status = 200; + request.mime_type = "application/json"; + request.encoded_data_length = 100; + markSectionsComplete(observed, [ + "network_requests", + "websocket_connections", + "websocket_frames", + "json_body_shapes", + ]); + for (const item of observed.network.requests) + item.body_shapes.status = "not_approved"; + expect(compareNetwork(observed, observed).status).toBe("unchanged"); - const after = structuredClone(before); - const responseObserved = after.network.requests[0]; - if (responseObserved === undefined) - throw new Error("Missing network request"); - responseObserved.status = 200; - responseObserved.mime_type = "application/json"; - responseObserved.encoded_data_length = 100; - expect(compareNetwork(before, after)).toMatchObject({ - status: "unknown", - total_changes: 0, - changes: [], - }); - expect(compareNetwork(after, before)).toMatchObject({ - status: "unknown", - total_changes: 0, - }); - expect(compareNetwork(before, before).status).toBe("unknown"); - - for (const field of [ - "status", - "mime_type", - "encoded_data_length", - ] as const) { - const partial = structuredClone(after); + const partial = structuredClone(observed); partial.network.requests[0]![field] = null; - expect(compareNetwork(partial, after).status).toBe("unknown"); - expect(compareNetwork(after, partial).status).toBe("unknown"); - } - - const inventoryBefore = structuredClone(before); - inventoryBefore.network.requests = []; - expect(compareNetwork(inventoryBefore, before)).toMatchObject({ - status: "changed", - total_changes: 1, - changes: [expect.objectContaining({ change: "added" })], - }); - - const statusChanged = structuredClone(after); - statusChanged.network.requests[0]!.status = 500; - expect(compareNetwork(after, statusChanged)).toMatchObject({ - status: "changed", - changes: [expect.objectContaining({ change: "modified" })], - }); - after.network.requests[0]!.encoded_data_length = null; - expect(compareNetwork(after, statusChanged).status).toBe("changed"); - - const repeated = structuredClone(statusChanged); - const duplicate = structuredClone(repeated.network.requests[0]!); - duplicate.status = 200; - duplicate.mime_type = "text/plain"; - repeated.network.requests.push(duplicate); - const reassociated = structuredClone(repeated); - reassociated.network.requests[0]!.mime_type = "text/plain"; - reassociated.network.requests.at(-1)!.mime_type = "application/json"; - expect(compareNetwork(repeated, reassociated).status).toBe("changed"); - repeated.network.requests[0]!.status = null; - expect(compareNetwork(repeated, reassociated).status).toBe("unknown"); - }); + for (const [left, right] of [ + [partial, observed], + [observed, partial], + [partial, partial], + ] as const) + expect(compareNetwork(left, right)).toMatchObject({ + status: "unknown", + total_changes: 0, + changes: [], + }); + }, + ); it("ignores transient request IDs and capture-approval state", async () => { const browser = await startFakeCdpBrowser(); diff --git a/tests/boundary/browser/webSourceMapFetcher.test.ts b/tests/boundary/browser/webSourceMapFetcher.test.ts index b1227fa26..5fab66f58 100644 --- a/tests/boundary/browser/webSourceMapFetcher.test.ts +++ b/tests/boundary/browser/webSourceMapFetcher.test.ts @@ -29,6 +29,104 @@ const expectInvalidIncludedSourceMaps = (result: WebSourceMaps): void => { expectInvalidSourceMaps({ ...result, status: "unavailable" }); }; +it.each(["regular", "indexed"])( + "collects an HTTP %s source map with omitted names without accepting invalid names", + async (format) => { + const leaf = { + version: 3, + sources: ["src/main.ts"], + sourcesContent: ["import './dependency.ts';"], + mappings: "AAAA", + }; + const texts = [ + leaf, + { ...leaf, names: [] }, + { ...leaf, names: null }, + { ...leaf, names: 42 }, + { ...leaf, names: [42] }, + { ...leaf, mappings: "AAAAA" }, + ].map((map) => + JSON.stringify( + format === "indexed" + ? { + version: 3, + sections: [{ offset: { line: 2, column: 5 }, map }], + } + : map, + ), + ); + const server = createServer((incoming, response) => { + response.setHeader("content-type", "application/source-map+json"); + response.end(texts[Number(incoming.url?.slice(1))]); + }); + try { + await new Promise((resolve, reject) => { + server.once("error", reject); + server.listen(0, "127.0.0.1", resolve); + }); + const address = server.address(); + if (address === null || typeof address === "string") + throw new TypeError("Expected a TCP listener address"); + const localOrigin = `http://127.0.0.1:${String(address.port)}`; + const result = await fetchWebSourceMaps( + texts.map((_, index) => ({ + ...request, + fetchUrl: `${localOrigin}/${index}`, + declaredUrl: `${localOrigin}/${index}`, + })), + input({ allowed_origins: [localOrigin] }), + ); + expect(result.items.map(({ status }) => status)).toEqual([ + "included", + "included", + "invalid", + "invalid", + "invalid", + "invalid", + ]); + for (const [index, text] of texts.slice(0, 2).entries()) + expect(result.items[index]).toMatchObject({ + artifact: { + text, + bytes: Buffer.byteLength(text), + sha256: createHash("sha256").update(text).digest("hex"), + }, + original_sources: [{ source: `${localOrigin}/src/main.ts` }], + mappings: [ + { + generated_line: format === "indexed" ? 3 : 1, + generated_column: format === "indexed" ? 5 : 0, + source: `${localOrigin}/src/main.ts`, + original_line: 1, + original_column: 0, + name: null, + }, + ], + original_module_edges: [ + { + from_source: `${localOrigin}/src/main.ts`, + resolved_source: `${localOrigin}/src/dependency.ts`, + }, + ], + }); + for (const item of result.items.slice(2)) + expect(item).toMatchObject({ + artifact: null, + mappings: [], + original_sources: [], + }); + } finally { + server.closeAllConnections(); + if (server.listening) + await new Promise((resolve, reject) => { + server.close((error) => + error === undefined ? resolve() : reject(error), + ); + }); + } + }, +); + it("parses ECMA-426 HTTP source-map prefixes while retaining exact response evidence", async () => { const map = JSON.stringify({ version: 3, @@ -712,7 +810,6 @@ describe("web source-map sourcesContent validation", () => { it.each([ ["a scalar sourcesContent value", { sourcesContent: "source" }], ["a non-string, non-null source entry", { sourcesContent: [42] }], - ["a sourcesContent array with the wrong length", { sourcesContent: [] }], ])("reports %s as invalid", async (_description, content) => { const result = await fetchWebSourceMaps([request], input(), undefined, { fetch: () => @@ -743,6 +840,50 @@ describe("web source-map sourcesContent validation", () => { }); }); + it.each([ + ["shorter", ["import './a.ts';"]], + ["longer", ["import './a.ts';", null, "unused trailing content"]], + ])( + "keeps every source declaration when sourcesContent is %s than sources", + async (_length, sourcesContent) => { + const result = await fetchWebSourceMaps([request], input(), undefined, { + fetch: () => + Promise.resolve( + new Response( + JSON.stringify({ + version: 3, + sources: ["a.ts", "b.ts"], + sourcesContent, + names: [], + mappings: "", + }), + { status: 200 }, + ), + ), + }); + + expect(result.items[0]).toMatchObject({ + status: "included", + limitation: null, + original_sources: [ + { + source_index: 0, + declaration_index: 0, + source: `${origin}/assets/a.ts`, + artifact: { text: "import './a.ts';" }, + }, + { + source_index: 1, + declaration_index: 1, + source: `${origin}/assets/b.ts`, + artifact: null, + }, + ], + }); + expect(result.items[0]?.original_sources).toHaveLength(2); + }, + ); + it("accepts null source-map source contents", async () => { const result = await fetchWebSourceMaps([request], input(), undefined, { fetch: () => diff --git a/tests/boundary/captures/captureDescriptorCleanup.test.ts b/tests/boundary/captures/captureDescriptorCleanup.test.ts index a1ef5efc1..852d06e22 100644 --- a/tests/boundary/captures/captureDescriptorCleanup.test.ts +++ b/tests/boundary/captures/captureDescriptorCleanup.test.ts @@ -1,3 +1,7 @@ +import { + descriptorFailurePath, + expectRetainedDescriptor, +} from "../../fixtures/descriptorCloseFailure.js"; import { expect, it, onTestFinished, vi } from "vitest"; import { writeFile } from "node:fs/promises"; import type { FileHandle } from "node:fs/promises"; @@ -15,27 +19,22 @@ const cleanup = vi.hoisted(() => ({ })); vi.mock("node:fs/promises", async (importOriginal) => { const filesystem = await importOriginal(); - return { - ...filesystem, - open: async (...args: Parameters) => { - const handle = await filesystem.open(...args); - if (args[0] === cleanup.path) { - cleanup.handles.push(handle); - const close = handle.close.bind(handle); - handle.close = async () => { - if (cleanup.denied && handle.fd >= 0) - throw new Error("Descriptor close deliberately denied"); - await close(); - }; - } - return handle; - }, - }; + const { withDescriptorCloseFailure } = + await import("../../fixtures/descriptorCloseFailure.js"); + return withDescriptorCloseFailure(filesystem, cleanup); }); it .runIf(process.platform === "linux") - .each(["capture", "tool", "limiter", "kernel"])( + .each([ + "capture", + "tool", + "limiter", + "kernel", + "field-tool", + "field-limiter", + "field-kernel", + ])( "retains failed %s descriptors and retries them before further acquisition and shutdown", async (member) => { const directory = await createTestTempDirectory("rea-capture-descriptor-"); @@ -67,14 +66,16 @@ it }, () => PrivateRuntimeRoot.create({ parent: directory }), ); - cleanup.path = - member === "capture" - ? path - : member === "tool" - ? command - : member === "limiter" - ? limiter - : "/proc/self/limits"; + const discovery = member.startsWith("field-"); + const resource = discovery ? member.slice(6) : member; + const operation = discovery + ? "inspect_packet_fields" + : "inspect_packet_capture"; + cleanup.path = descriptorFailurePath(resource, { + capture: path, + tool: command, + limiter, + }); cleanup.denied = true; cleanup.handles = []; onTestFinished(async () => { @@ -84,24 +85,45 @@ it cleanup.path = ""; }); const service = new PacketCaptureService(provider); - const first = await service.inspect({ path, fields: ["frame.number"] }); - expect(first).toMatchObject({ - ok: false, - error: { cleanupIncomplete: true, cleanupResources: [cleanup.path] }, - }); - expect(cleanup.handles).toHaveLength(1); - expect(cleanup.handles[0]?.fd).toBeGreaterThanOrEqual(0); + const first = discovery + ? await service.inspectFields({ query: "usb" }) + : await service.inspect({ path, fields: ["frame.number"] }); + expectRetainedDescriptor(first, cleanup, operation); const launchCount = launches; expect( await service.inspect({ path, fields: ["frame.number"] }), - ).toMatchObject({ ok: false, error: { cleanupIncomplete: true } }); + ).toMatchObject({ + ok: false, + error: { cleanupIncomplete: true, operation }, + }); expect(cleanup.handles).toHaveLength(1); expect(launches).toBe(launchCount); + let closingOperation = operation; + if (discovery) { + cleanup.denied = false; + const recovered = await service.inspect({ + path, + fields: ["frame.number"], + }); + expect(recovered.ok).toBe(true); + expect(cleanup.handles.every((handle) => handle.fd === -1)).toBe(true); + expect(launches).toBeGreaterThan(launchCount); + cleanup.denied = true; + expect( + await service.inspect({ path, fields: ["frame.number"] }), + ).toMatchObject({ + ok: false, + error: { cleanupIncomplete: true, operation: "inspect_packet_capture" }, + }); + expect(cleanup.handles.at(-1)?.fd).toBeGreaterThanOrEqual(0); + closingOperation = "inspect_packet_capture"; + } await expect(provider.close()).rejects.toMatchObject({ cleanupIncomplete: true, + operation: closingOperation, }); cleanup.denied = false; await provider.close(); - expect(cleanup.handles[0]?.fd).toBe(-1); + expect(cleanup.handles.every((handle) => handle.fd === -1)).toBe(true); }, ); diff --git a/tests/boundary/captures/captureProvider.test.ts b/tests/boundary/captures/captureProvider.test.ts index 2428b39d4..881b43e63 100644 --- a/tests/boundary/captures/captureProvider.test.ts +++ b/tests/boundary/captures/captureProvider.test.ts @@ -11,6 +11,9 @@ import { PrivateRuntimeRoot } from "../../../src/process/PrivateRuntimeRoot.js"; import { spawnOwnedProviderProcess } from "../../../src/process/ProviderProcess.js"; import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { createDeferred } from "../../fixtures/binarySession.js"; +import { Cli } from "incur"; +import { registerCaptureCommands } from "../../../src/cli/captureCommands.js"; +import { silentLogger } from "../../../src/logger.js"; // A command-boundary simulator: real process ownership/filesystem, not real TShark coverage. const fixture = async (mode = "ok", failCleanup = false) => { @@ -625,3 +628,34 @@ it("retains completed discovery on late cancellation and cleanup failure", async await f.service.close(); } }); + +it("preserves discovery resource ownership through CLI shutdown retries", async () => { + const f = await fixture("cleanup", true); + const cli = Cli.create("discovery-cleanup", { sync: false }); + registerCaptureCommands(cli, silentLogger, {}, f.service); + let stdout = ""; + try { + await cli.serve( + ["inspect-packet-fields", "--query", "usb", "--format", "json"], + { + stdout: (text) => { + stdout += text; + }, + exit: () => {}, + }, + ); + expect(JSON.parse(stdout)).toMatchObject({ + code: "cleanup_incomplete", + details: { + diagnostics: { + cleanup_error: { + details: { operation: "inspect_packet_fields" }, + }, + }, + }, + }); + } finally { + f.allowCleanup(); + await f.service.close(); + } +}); diff --git a/tests/boundary/cli/apktoolLocaleOption.test.ts b/tests/boundary/cli/apktoolLocaleOption.test.ts new file mode 100644 index 000000000..4bc596695 --- /dev/null +++ b/tests/boundary/cli/apktoolLocaleOption.test.ts @@ -0,0 +1,92 @@ +import { expect, it } from "vitest"; +import { Cli } from "incur"; +import pino from "pino"; + +import type { ApktoolResourceAnalysisPort } from "../../../src/application/apktool/ApktoolResourceAnalysisPort.js"; +import { createAnalysisExecution } from "../../../src/application/AnalysisProvider.js"; +import { registerApktoolCommands } from "../../../src/cli/apktoolCommands.js"; +import { presentOmittableDefaults } from "../../../src/cli/omittableDefaults.js"; +import type { ApktoolRequest } from "../../../src/domain/apktool/apktoolResourceAnalysis.js"; +import { ok } from "../../../src/domain/result.js"; + +const invoke = async (argv: readonly string[]) => { + const requests: ApktoolRequest[] = []; + const provider: ApktoolResourceAnalysisPort = { + async inspectAvailability() { + throw new Error("unexpected provider probe"); + }, + async close() {}, + async execute(request) { + requests.push(request); + return ok( + createAnalysisExecution( + { strings: [] }, + { id: "apktool", name: "Apktool", version: null }, + ), + ); + }, + }; + const cli = Cli.create("rea", { sync: false }); + registerApktoolCommands(cli, pino({ level: "silent" }), {}, provider); + presentOmittableDefaults(cli); + + let stdout = ""; + let exitCode = 0; + await cli.serve([...argv], { + env: {}, + exit: (code) => { + exitCode = code; + }, + stdout: (text) => { + stdout += text; + }, + }); + return { exitCode, stdout, requests }; +}; + +/** + * Locales the apktool input owner admits. The owner's own rejection message + * advertises `b+zh+Hans+CN`, so the CLI cannot be narrower than this list + * without refusing a caller-selected locale the decoding lane supports. + */ +const OWNER_ADMITTED_LOCALES = [ + "de", + "pt-BR", + "pt-rBR", + "zh-rCN", + "en-419", + "b+zh+Hans+CN", +] as const; + +it("forwards every locale the apktool input owner admits", async () => { + for (const locale of OWNER_ADMITTED_LOCALES) { + const result = await invoke([ + "decode-android-resources", + "/targets/app.apk", + "--locale", + locale, + "--json", + ]); + + expect(result.exitCode, `exit code for locale ${locale}`).toBe(0); + expect(result.requests, `request for locale ${locale}`).toEqual([ + { + operation: "decode_android_resources", + input: { path: "/targets/app.apk", include_strings: true, locale }, + }, + ]); + } +}); + +it("still refuses a locale the apktool input owner refuses", async () => { + const result = await invoke([ + "decode-android-resources", + "/targets/app.apk", + "--locale", + "not a locale", + "--json", + ]); + + expect(result.exitCode).not.toBe(0); + expect(result.requests).toEqual([]); +}); diff --git a/tests/boundary/cli/browserCli.test.ts b/tests/boundary/cli/browserCli.test.ts index 5f8193fce..351d07512 100644 --- a/tests/boundary/cli/browserCli.test.ts +++ b/tests/boundary/cli/browserCli.test.ts @@ -183,6 +183,54 @@ it( INTEGRATION_TEST_TIMEOUT_MS, ); +it( + "browser CLI attaches through a browser socket and records it redacted", + async () => { + const browser = await startFakeCdpBrowser(); + browsers.push(browser); + const inspected = await runCli( + [ + "inspect-web-page", + browser.browserWebSocketUrl, + "allowed-page", + "--observation-ms", + "0", + "--json", + ], + process.env, + ); + expect(inspected).toMatchObject({ + operation: "inspect_web_page", + parameters: { + cdp_endpoint: browser.browserWebSocketUrl.replace( + /fake$/u, + "[REDACTED]", + ), + }, + normalized_result: { target: { target_id: "allowed-page" } }, + }); + expect(JSON.stringify(inspected)).not.toContain("/devtools/browser/fake"); + + const refused = await runCli( + [ + "list-browser-targets", + browser.browserWebSocketUrl.replace(/fake$/u, "not-this-browser"), + "--json", + ], + process.env, + ); + expect(refused).toMatchObject({ + code: "provider_unavailable", + details: { + operation: "list_browser_targets", + reason: "endpoint_unreachable", + }, + }); + expect(JSON.stringify(refused)).not.toContain("not-this-browser"); + }, + INTEGRATION_TEST_TIMEOUT_MS, +); + describe("scenario comparison CLI diagnostics", () => { const rule = { rule_id: "domain", diff --git a/tests/boundary/cli/captureCleanup.test.ts b/tests/boundary/cli/captureCleanup.test.ts index 6e5882cd4..40013d6a1 100644 --- a/tests/boundary/cli/captureCleanup.test.ts +++ b/tests/boundary/cli/captureCleanup.test.ts @@ -188,8 +188,12 @@ it("closes the service and retains an unexpected producer rejection", async () = }); }); -it("rejects an oversized selector budget before the CLI invokes its service", async () => { - const cli = Cli.create("capture-selector-test", { sync: false }); +/** Serve one inspection request whose service must never be reached. */ +const rejectBeforeService = async ( + name: string, + args: string[], +): Promise => { + const cli = Cli.create(name, { sync: false }); let called = false; registerCaptureCommands( cli, @@ -207,66 +211,41 @@ it("rejects an oversized selector budget before the CLI invokes its service", as }, ); let stdout = ""; - await cli.serve( - [ - "inspect-packet-capture", - "recording.pcap", - "--fields", - "x".repeat(8192), - "--display_filter", - "界".repeat(8192), - "--format", - "json", - ], - { - stdout: (text) => { - stdout += text; - }, - exit: () => {}, + await cli.serve(args, { + stdout: (text) => { + stdout += text; }, - ); + exit: () => {}, + }); expect(called).toBe(false); + return stdout; +}; + +it("rejects an oversized selector budget before the CLI invokes its service", async () => { + const stdout = await rejectBeforeService("capture-selector-test", [ + "inspect-packet-capture", + "recording.pcap", + "--fields", + "x".repeat(8192), + "--display_filter", + "界".repeat(8192), + "--format", + "json", + ]); expect(stdout).toContain("32 KiB"); }); it("rejects conflicting output options before the CLI invokes its service", async () => { - const cli = Cli.create("capture-window-test", { sync: false }); - let called = false; - registerCaptureCommands( - cli, - silentLogger, - {}, - { - inspectFields: async () => { - throw new Error("Unexpected field discovery call"); - }, - inspect: async () => { - called = true; - throw new Error("Must not inspect"); - }, - close: async () => {}, - }, - ); - let stdout = ""; - await cli.serve( - [ - "inspect-packet-capture", - "recording.pcap", - "--fields", - "ip.addr", - "--all_matches", - "--match_limit", - "20", - "--format", - "json", - ], - { - stdout: (text) => { - stdout += text; - }, - exit: () => {}, - }, - ); - expect(called).toBe(false); + const stdout = await rejectBeforeService("capture-window-test", [ + "inspect-packet-capture", + "recording.pcap", + "--fields", + "ip.addr", + "--all_matches", + "--match_limit", + "20", + "--format", + "json", + ]); expect(stdout).toContain("mutually exclusive"); }); diff --git a/tests/boundary/cli/clientProfileLifecycle.test.ts b/tests/boundary/cli/clientProfileLifecycle.test.ts index 4fd566fd7..1ceaf49cd 100644 --- a/tests/boundary/cli/clientProfileLifecycle.test.ts +++ b/tests/boundary/cli/clientProfileLifecycle.test.ts @@ -287,9 +287,12 @@ cliTest("uninstall names each retained skill directory", async ({ cli }) => { }; const skill = report.items?.find(({ name }) => name === "skill"); expect(skill?.status).toBe("retained"); - for (const directory of [shared, claude]) { + for (const [client, directory] of [ + ["shared", shared], + ["claude_code", claude], + ] as const) { expect(skill?.detail).toContain( - `${directory}: retained (Skill manifest is not REA-owned.)`, + `${client} (${directory}): retained (Skill manifest is not REA-owned.)`, ); expect(await readFile(join(directory, "SKILL.md"), "utf8")).toBe( "---\nname: unrelated\n---\n", diff --git a/tests/boundary/cli/electronWrappedCallPaths.test.ts b/tests/boundary/cli/electronWrappedCallPaths.test.ts index af61f0885..4dd0d02a8 100644 --- a/tests/boundary/cli/electronWrappedCallPaths.test.ts +++ b/tests/boundary/cli/electronWrappedCallPaths.test.ts @@ -23,7 +23,7 @@ const analyze = async (cli: TestCli, extra = "") => { const output = await cli.run({ arguments: ["analyze-javascript-application", root, "--json"], }); - expect(output.exitCode).toBe(0); + expect(output.exitCode, output.stderr).toBe(0); const { normalized_result: result } = z .object({ normalized_result: javascriptApplicationAnalysisResultSchema, @@ -122,6 +122,190 @@ cliTest( }, ); +cliTest( + "traces a nested contextBridge member through a unique IPC channel", + async ({ cli }) => { + const root = await createTestTempDirectory("rea-electron-nested-bridge-"); + await writeFixtureFiles(root, { + "preload.js": `const { contextBridge, ipcRenderer } = require("electron"); + contextBridge.exposeInMainWorld("api", { + nested: { write: (value) => ipcRenderer.invoke("nested:write", value) } + });`, + "main.js": `const { ipcMain, clipboard } = require("electron"); + ipcMain.handle("nested:write", (event, value) => clipboard.writeText(value));`, + }); + const output = await cli.run({ + arguments: ["analyze-javascript-application", root, "--json"], + }); + expect(output.exitCode, output.stderr).toBe(0); + const { normalized_result: application } = z + .object({ normalized_result: javascriptApplicationAnalysisResultSchema }) + .parse(output.json); + const { graph } = application; + const paths = graph.nodes + .flatMap((node) => node.observations) + .filter( + (observation) => + observation.properties.entry === "api.nested.write" && + observation.properties.entry_kind === "bridge-member", + ); + expect( + paths.some( + (observation) => observation.properties.step_kind === "exposure", + ), + ).toBe(true); + expect( + paths.some( + (observation) => observation.properties.step_kind === "terminal", + ), + ).toBe(true); + const joins = graph.edges.filter( + (edge) => + edge.evidence.extractor.operation === "join-electron-wrapper-paths", + ); + expect(joins.map((edge) => edge.properties.channel)).toEqual([ + "nested:write", + ]); + + const traceInput = join(root, "trace-nested-input.json"); + await writeFile( + traceInput, + JSON.stringify({ + application: output.json, + seed: { kind: "api", value: "api.nested.write", match: "exact" }, + direction: "outgoing", + }), + ); + const traced = await cli.run({ + arguments: ["trace-application-feature", traceInput, "--json"], + }); + expect(traced.exitCode).toBe(0); + const { normalized_result: trace } = z + .object({ normalized_result: applicationFeatureTraceResultSchema }) + .parse(traced.json); + expect(trace.seed_matches).toHaveLength(1); + expect(trace.paths.some((path) => path.contains_inference)).toBe(true); + }, +); + +cliTest( + "preserves exact bridge member segments through the public CLI", + async ({ cli }) => { + const root = await createTestTempDirectory("rea-electron-bridge-segments-"); + await writeFixtureFiles(root, { + "preload.js": `const { contextBridge, ipcRenderer } = require("electron"); + const write = () => ipcRenderer.send("write"); + contextBridge.exposeInMainWorld("desktop", { + "nested.write": write, + nested: { write }, + __proto__: { inherited: write }, + ["__proto__"]: { own: write }, + });`, + "main.js": `const { ipcMain } = require("electron"); + ipcMain.on("write", (event) => event.sender.copy());`, + }); + const output = await cli.run({ + arguments: ["analyze-javascript-application", root, "--json"], + }); + expect(output.exitCode).toBe(0); + const { normalized_result: application } = z + .object({ normalized_result: javascriptApplicationAnalysisResultSchema }) + .parse(output.json); + const observations = application.graph.nodes.flatMap((node) => + node.observations.filter( + (observation) => + observation.properties.entry_kind === "bridge-member" && + observation.properties.step_kind === "exposure", + ), + ); + const bridge = application.graph.nodes + .filter((node) => node.kind === "context-bridge-api") + .flatMap((node) => node.observations) + .find((observation) => observation.properties.api_key === "desktop"); + expect(bridge?.properties.members).not.toContain("__proto__.inherited"); + expect(bridge?.properties.members).toContain("__proto__.own"); + const paths = observations.map( + (observation) => observation.properties.entry_path, + ); + expect(paths).toEqual( + expect.arrayContaining([ + ["desktop", "nested.write"], + ["desktop", "nested", "write"], + ["desktop", "__proto__", "own"], + ]), + ); + expect(paths).not.toContainEqual(["desktop", "__proto__", "inherited"]); + const handlerPaths = application.graph.nodes + .flatMap((node) => node.observations) + .filter( + (observation) => observation.properties.entry_kind === "ipc-handler", + ); + expect(handlerPaths.length).toBeGreaterThan(0); + expect( + handlerPaths.every( + (observation) => !("entry_path" in observation.properties), + ), + ).toBe(true); + }, +); + +cliTest( + "retains a nested bridge path when its IPC handler has no external terminal", + async ({ cli }) => { + const root = await createTestTempDirectory("rea-electron-nested-return-"); + await writeFixtureFiles(root, { + "preload.js": `const { contextBridge, ipcRenderer } = require("electron"); + contextBridge.exposeInMainWorld("api", { + nested: { write: (value) => ipcRenderer.invoke("nested:write", value) } + });`, + "main.js": `const { ipcMain } = require("electron"); + ipcMain.handle("nested:write", (event, value) => value);`, + }); + const output = await cli.run({ + arguments: ["analyze-javascript-application", root, "--json"], + }); + expect(output.exitCode).toBe(0); + const { normalized_result: application } = z + .object({ normalized_result: javascriptApplicationAnalysisResultSchema }) + .parse(output.json); + const { graph } = application; + expect( + graph.nodes.some((node) => + node.observations.some( + (observation) => + observation.properties.entry === "api.nested.write" && + observation.properties.step_kind === "exposure", + ), + ), + ).toBe(true); + expect( + graph.edges.some( + (edge) => + edge.evidence.extractor.operation === "join-electron-wrapper-paths", + ), + ).toBe(false); + + const traceInput = join(root, "trace-nested-input.json"); + await writeFile( + traceInput, + JSON.stringify({ + application: output.json, + seed: { kind: "api", value: "api.nested.write", match: "exact" }, + direction: "outgoing", + }), + ); + const traced = await cli.run({ + arguments: ["trace-application-feature", traceInput, "--json"], + }); + expect(traced.exitCode).toBe(0); + const { normalized_result: trace } = z + .object({ normalized_result: applicationFeatureTraceResultSchema }) + .parse(traced.json); + expect(trace.seed_matches).toHaveLength(1); + expect(trace.graph?.nodes.length).toBeGreaterThan(1); + }, +); + for (const surface of ["CLI", "stdio MCP"] as const) { cliTest( `preserves handler ambiguity and short-circuit factory paths through ${surface}`, @@ -225,3 +409,89 @@ cliTest( expect(joins.map((e) => e.properties.channel)).toEqual(["test:cut"]); }, ); + +cliTest( + "keeps shallow bridge members when a sibling exceeds the nesting limit", + async ({ cli }) => { + const root = await createTestTempDirectory("rea-electron-deep-bridge-"); + let deepMember = '() => e.ipcRenderer.send("deep:run")'; + for (let index = 0; index < 129; index += 1) + deepMember = `{ level${index}: ${deepMember} }`; + await writeFixtureFiles(root, { + "preload.js": `const e=require("electron"); + e.contextBridge.exposeInMainWorld("desktop", { + deep: ${deepMember}, + shallow: () => e.ipcRenderer.send("shallow:run") + });`, + "main.js": `const e=require("electron"); + e.ipcMain.on("shallow:run", event => event.sender.copy()); + e.ipcMain.on("deep:run", event => event.sender.copy());`, + }); + const output = await cli.run({ + arguments: ["analyze-javascript-application", root, "--json"], + }); + expect(output.exitCode).toBe(0); + const { normalized_result: application } = z + .object({ normalized_result: javascriptApplicationAnalysisResultSchema }) + .parse(output.json); + const { graph } = application; + const joins = graph.edges.filter( + (edge) => + edge.evidence.extractor.operation === "join-electron-wrapper-paths", + ); + expect(joins.map((edge) => edge.properties.channel)).toEqual([ + "shallow:run", + ]); + expect( + graph.nodes + .flatMap((node) => node.observations) + .some( + (observation) => + observation.properties.entry === "desktop.shallow" && + observation.properties.step_kind === "exposure", + ), + ).toBe(true); + }, +); + +cliTest( + "preserves empty bridge member segments in nested paths", + async ({ cli }) => { + const root = await createTestTempDirectory( + "rea-electron-empty-bridge-key-", + ); + await writeFixtureFiles(root, { + "preload.js": `const { contextBridge, ipcRenderer } = require("electron"); + contextBridge.exposeInMainWorld("desktop", { + "": { nested: { call: () => ipcRenderer.send("empty:key") } } + });`, + "main.js": `const { ipcMain } = require("electron"); + ipcMain.on("empty:key", event => event.sender.copy());`, + }); + const output = await cli.run({ + arguments: ["analyze-javascript-application", root, "--json"], + }); + expect(output.exitCode).toBe(0); + const { normalized_result: application } = z + .object({ normalized_result: javascriptApplicationAnalysisResultSchema }) + .parse(output.json); + const observations = application.graph.nodes.flatMap( + (node) => node.observations, + ); + expect( + observations.some( + (observation) => + observation.properties.entry === "desktop..nested.call" && + observation.properties.step_kind === "exposure", + ), + ).toBe(true); + expect( + application.graph.edges + .filter( + (edge) => + edge.evidence.extractor.operation === "join-electron-wrapper-paths", + ) + .map((edge) => edge.properties.channel), + ).toEqual(["empty:key"]); + }, +); diff --git a/tests/boundary/cli/emptyHomeStartup.test.ts b/tests/boundary/cli/emptyHomeStartup.test.ts new file mode 100644 index 000000000..f1c73734f --- /dev/null +++ b/tests/boundary/cli/emptyHomeStartup.test.ts @@ -0,0 +1,32 @@ +import { readFile } from "node:fs/promises"; + +import { expect } from "vitest"; +import { z } from "zod"; + +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { cliTest } from "../../support/cli/cliFixture.js"; + +const packageSchema = z.object({ version: z.string() }); + +// Node's Windows home lookup throws for an empty USERPROFILE, so a dependency +// that resolves the home during import used to stop every command there. +cliTest("starts when the selected home is empty", async ({ cli }) => { + const { version } = packageSchema.parse( + JSON.parse(await readFile("package.json", "utf8")), + ); + const cwd = await createTestTempDirectory("rea-empty-home-startup-"); + const environment = { HOME: "", USERPROFILE: "" }; + + const reported = await cli.run({ + arguments: ["--version"], + cwd, + environment, + }); + expect(reported.stderr).toBe(""); + expect(reported.exitCode).toBe(0); + expect(reported.stdout.trim()).toBe(version); + + const help = await cli.run({ arguments: ["--help"], cwd, environment }); + expect(help.stderr).not.toContain("uv_os_homedir"); + expect(help.exitCode).toBe(0); +}); diff --git a/tests/boundary/cli/httpCaptureComparison.test.ts b/tests/boundary/cli/httpCaptureComparison.test.ts new file mode 100644 index 000000000..10d373351 --- /dev/null +++ b/tests/boundary/cli/httpCaptureComparison.test.ts @@ -0,0 +1,77 @@ +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { expect, it } from "vitest"; + +import { webPageInspectionSchema } from "../../../src/domain/browserObservationSchemas.js"; +import { parseEvidence } from "../../../src/domain/evidence.js"; +import { runCliJson } from "../../fixtures/cliJsonProcess.js"; +import { startHttpComparisonBrowser } from "../../fixtures/httpComparisonBrowser.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +it("CLI compares incomplete HTTP responses without inventing changes", async () => { + const root = await createTestTempDirectory("rea-http-comparison-"); + const response = { + requestId: "request-1", + status: 200, + mimeType: "application/json", + }; + const fixture = await startHttpComparisonBrowser([response]); + try { + const inspect = async () => + webPageInspectionSchema.parse( + parseEvidence( + await runCliJson([ + "inspect-web-page", + fixture.browser.endpoint, + "allowed-page", + "--observation-ms", + "0", + "--json", + ]), + ).normalized_result, + ); + const pending = await inspect(); + fixture.setRequests([{ ...response, encodedDataLength: 100 }]); + const finished = await inspect(); + fixture.setRequests([{ ...response, status: 500, encodedDataLength: 100 }]); + const changed = await inspect(); + expect(pending.network.requests).toMatchObject([ + { status: 200, mime_type: "application/json", encoded_data_length: null }, + ]); + expect(finished.network.requests).toMatchObject([ + { status: 200, mime_type: "application/json", encoded_data_length: 100 }, + ]); + const paths = [pending, finished, changed].map((_, index) => + join(root, `capture-${String(index)}.json`), + ); + await Promise.all( + [pending, finished, changed].map((inspection, index) => + writeFile(paths[index]!, JSON.stringify({ inspection })), + ), + ); + for (const [left, right, status, count] of [ + [0, 1, "unknown", 0], + [1, 0, "unknown", 0], + [0, 2, "changed", 1], + ] as const) { + const compared = await runCliJson([ + "compare-web-captures", + paths[left]!, + paths[right]!, + "--json", + ]); + expect(compared).toMatchObject({ + operation: "compare_web_captures", + normalized_result: { + overall_status: status, + dimensions: { + network: { status, total_changes: count }, + }, + }, + }); + } + } finally { + await fixture.browser.close(); + } +}, 60_000); diff --git a/tests/boundary/cli/idaRetainedCleanupCli.test.ts b/tests/boundary/cli/idaRetainedCleanupCli.test.ts index b31903489..826bb7f1d 100644 --- a/tests/boundary/cli/idaRetainedCleanupCli.test.ts +++ b/tests/boundary/cli/idaRetainedCleanupCli.test.ts @@ -79,6 +79,7 @@ describe("headless IDA timeout through the one-shot CLI", () => { backend: "worker", owned: null, input_path: null, + is_active: null, }, ], }, diff --git a/tests/boundary/cli/nativeUiScenarioInputCli.test.ts b/tests/boundary/cli/nativeUiScenarioInputCli.test.ts index 766b77a79..7bbd2e2bc 100644 --- a/tests/boundary/cli/nativeUiScenarioInputCli.test.ts +++ b/tests/boundary/cli/nativeUiScenarioInputCli.test.ts @@ -71,4 +71,27 @@ describe("native UI scenario CLI input", () => { }); }, ); + + cliTest( + "refuses scenario steps on passive observation instead of dropping them", + async ({ cli }) => { + const result = await cli.run({ + arguments: [ + "observe-native-ui", + "/provider-must-not-be-invoked", + "--pid", + "1", + "--window-id", + "1", + "--steps", + '[{"kind":"wait","milliseconds":0}]', + "--json", + ], + environment: { REA_LOG_LEVEL: "silent" }, + }); + + expect(result.exitCode).not.toBe(0); + expect(result.stdout).toContain("Unknown flag: --steps"); + }, + ); }); diff --git a/tests/boundary/cli/referenceSourceSummary.test.ts b/tests/boundary/cli/referenceSourceSummary.test.ts new file mode 100644 index 000000000..aed8cb871 --- /dev/null +++ b/tests/boundary/cli/referenceSourceSummary.test.ts @@ -0,0 +1,190 @@ +import { mkdir, writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { describe, expect } from "vitest"; + +import { + createHistoricalSourceManifest, + parseHistoricalSourceGraph, +} from "../../../src/domain/referenceSourceGraph.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { cliTest } from "../../support/cli/cliFixture.js"; + +describe.skipIf(process.platform === "win32")( + "compiled historical-source summaries", + () => { + cliTest( + "selects a committed summary without changing complete output", + async ({ cli }) => { + const root = await createTestTempDirectory("rea-reference-summary-"); + await mkdir(join(root, "src", "nested"), { recursive: true }); + await Promise.all([ + writeFile(join(root, "package.json"), '{"name":"fixture"}\n'), + writeFile( + join(root, "src", "main.js"), + 'import "./nested/value.js";\n', + ), + writeFile( + join(root, "src", "nested", "value.js"), + "export const value = 1;\n", + ), + writeFile(join(root, "\uE000.js"), "export const first = 1;\n"), + writeFile(join(root, "\u{10000}.js"), "export const last = 2;\n"), + ]); + const run = (...options: string[]) => + cli.run({ + arguments: ["import-reference-source", root, "--json", ...options], + cwd: root, + }); + const complete = await run(); + expect(complete.exitCode).toBe(0); + const graph = parseHistoricalSourceGraph(complete.json); + expect((await run("--detail", "full")).stdout).toBe(complete.stdout); + for (const depth of [undefined, 0, 1, 10]) { + const result = await run( + "--detail", + "summary", + ...(depth === undefined ? [] : ["--outline-depth", String(depth)]), + ); + expect(result.exitCode).toBe(0); + const selected = graph.entries.filter( + (entry) => entry.path.split("/").length <= (depth ?? 2), + ); + expect(result.json).toMatchObject({ + schema: "HistoricalSourceSummary", + authority: graph.authority, + root_alias: graph.root_alias, + manifest: createHistoricalSourceManifest(graph), + inventory_state: graph.inventory_state, + provenance: graph.provenance, + vcs: graph.vcs, + languages: graph.languages, + manifests: graph.manifests, + counts: { + entries: 7, + files: 5, + directories: 2, + symlinks: 0, + relationships: 1, + parse_failures: 0, + exclusions: 0, + }, + outline: { + depth: depth ?? 2, + entries: selected.map(({ path, kind }) => + expect.objectContaining({ path, kind }), + ), + omitted_entry_count: graph.entries.length - selected.length, + }, + }); + expect(result.json).not.toHaveProperty("relationships"); + } + }, + ); + + cliTest( + "retains exclusions and parse coverage independently of a hidden outline", + async ({ cli }) => { + const root = await createTestTempDirectory( + "rea-reference-partial-summary-", + ); + await Promise.all([ + writeFile(join(root, ".gitignore"), "ignored.txt\n"), + writeFile(join(root, "ignored.txt"), "ignored\n"), + writeFile(join(root, "broken.js"), "export const = ;\n"), + ]); + const result = await cli.run({ + arguments: [ + "import-reference-source", + root, + "--detail", + "summary", + "--outline-depth", + "0", + "--json", + ], + cwd: root, + }); + expect(result.exitCode).toBe(0); + expect(result.json).toMatchObject({ + inventory_state: "partial", + counts: { parse_failures: 1, exclusions: 1 }, + exclusions: [ + { + path: "ignored.txt", + reason: "project-ignored", + pattern: "ignored.txt", + }, + ], + outline: { depth: 0, entries: [], omitted_entry_count: 2 }, + }); + }, + ); + }, +); + +describe("compiled historical-source summary input boundaries", () => { + cliTest( + "documents detail and depth in public command help", + async ({ cli }) => { + const result = await cli.run({ + arguments: ["import-reference-source", "--help"], + }); + expect(result.exitCode).toBe(0); + expect(result.stdout).toContain("--detail"); + expect(result.stdout).toContain("--outline-depth"); + expect(result.stdout).toContain("requires --detail summary"); + }, + ); + + for (const options of [ + ["--detail", "summry"], + ["--detail", "summary", "--outline-depth", "-1"], + ["--detail", "summary", "--outline-depth", "1.5"], + ["--detail", "summary", "--outline-depth", "NaN"], + ["--detail", "summary", "--outline-depth", "9007199254740992"], + ["--outline-depth", "1"], + ["--detail", "full", "--outline-depth", "1"], + ]) { + cliTest(`rejects ${options.join(" ")}`, async ({ cli }) => { + const root = await createTestTempDirectory( + "rea-reference-summary-input-", + ); + const result = await cli.run({ + arguments: ["import-reference-source", root, ...options, "--json"], + cwd: root, + }); + expect(result.exitCode).toBe(1); + expect(result.stdout).not.toContain("HistoricalSourceGraph"); + expect(result.stdout).not.toContain("HistoricalSourceSummary"); + expect(result.stdout).toMatch(/detail|outline/u); + }); + } + + cliTest( + "preserves the missing-root failure in summary mode", + async ({ cli }) => { + const root = await createTestTempDirectory( + "rea-reference-summary-missing-", + ); + const result = await cli.run({ + arguments: [ + "import-reference-source", + join(root, "missing"), + "--detail", + "summary", + "--json", + ], + cwd: root, + }); + expect(result.exitCode).toBe(1); + expect(result.json).toMatchObject({ + error: "Import failed", + category: "invalid_input", + message: expect.stringContaining( + "Reference source root could not be resolved", + ), + }); + }, + ); +}); diff --git a/tests/boundary/evm/solidityMetadataWorker.test.ts b/tests/boundary/evm/solidityMetadataWorker.test.ts new file mode 100644 index 000000000..0577983c4 --- /dev/null +++ b/tests/boundary/evm/solidityMetadataWorker.test.ts @@ -0,0 +1,75 @@ +import { execFile } from "node:child_process"; +import { readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { promisify } from "node:util"; +import { expect, it } from "vitest"; +import { solidityMetadataSchema } from "../../../src/domain/evm/solidityMetadata.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +/** Real compiled worker and unchanged WASM engine; no Linux provider-admission claim. */ +it.each([ + ["compiler", "00a164736f6c634300081e000a", "decoded"], + ["integer", "00a1011bffffffffffffffff000b", "decoded"], + ["BOM text", "00a1617864efbbbf610008", "decoded"], + ["nested value", "00a1617881000005", "partial"], + ["unavailable trailer", "00", "unavailable"], + ["unsupported tag", "00a16178c0000005", "not-decoded"], + ["floating point", "00a16178fa3f8000000008", "not-decoded"], + ["invalid UTF-8", "00a1617861ff0005", "not-decoded"], +])( + "retains independent %s metadata through the actual worker", + async (kind, hex, status) => { + const directory = await createTestTempDirectory("rea-solidity-worker-"); + const snapshot = join(directory, "selected.hex"); + const reply = join(directory, "reply.json"); + const request = join(directory, "request.json"); + await writeFile(snapshot, hex); + await writeFile( + request, + JSON.stringify({ + snapshot_path: snapshot, + reply_path: reply, + failure_marker_path: join(directory, "resource.failure"), + encoding: "hex", + }), + ); + await promisify(execFile)( + process.execPath, + [ + "--max-old-space-size=256", + "--disable-wasm-trap-handler", + new URL("../../../dist/evm/EvmoleInterfaceWorker.js", import.meta.url) + .pathname, + request, + ], + { timeout: 30_000, maxBuffer: 1024 * 1024 }, + ); + const value = JSON.parse(await readFile(reply, "utf8")); + expect(value.ok, JSON.stringify(value)).toBe(true); + expect(value.bytecode.hex).toBe(hex); + const metadata = solidityMetadataSchema.parse(value.bytecode.metadata); + expect(metadata.status).toBe(status); + if (kind === "integer") + expect(metadata.entries[0]?.value).toEqual({ + kind: "integer", + value: "18446744073709551615", + }); + if (kind === "compiler") + expect(metadata.compiler_version?.value).toBe("0.8.30"); + if (kind === "BOM text") + expect(metadata.entries[0]?.value).toEqual({ + kind: "text", + value: "\ufeffa", + }); + if (kind === "nested value") + expect(metadata.entries[0]?.value).toEqual({ + kind: "uninterpreted", + container: "array", + }); + if (kind === "unavailable trailer") { + expect(metadata.bytecode_offset).toBeNull(); + expect(metadata.entries).toEqual([]); + expect(metadata.compiler_version).toBeNull(); + } + }, +); diff --git a/tests/boundary/filesystem/asarListingAdmission.test.ts b/tests/boundary/filesystem/asarListingAdmission.test.ts new file mode 100644 index 000000000..f5ebaa870 --- /dev/null +++ b/tests/boundary/filesystem/asarListingAdmission.test.ts @@ -0,0 +1,130 @@ +import { readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { buffer } from "node:stream/consumers"; +import { setImmediate } from "node:timers/promises"; +import { expect, it } from "vitest"; + +import { AsarArtifactReader } from "../../../src/artifacts/AsarArtifactReader.js"; +import { readAsarListing } from "../../../src/artifacts/AsarListing.js"; +import { + asarHeaderBytes, + nestedAsarHeader, +} from "../../fixtures/asarHeader.js"; +import { inventoryArtifact } from "../../fixtures/artifactInventory.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +const archiveWith = async (bytes: Buffer) => { + const root = await createTestTempDirectory("rea-asar-listing-"); + const path = join(root, "app.asar"); + await writeFile(path, bytes); + return path; +}; + +it("interrupts expanded listing admission between traversal batches", async () => { + const controller = new AbortController(); + const header = { + files: Object.fromEntries( + Array.from({ length: 4000 }, (_, index) => [ + `f${index}`, + { size: 0, offset: "0" }, + ]), + ), + }; + const listing = readAsarListing(header, controller.signal); + await setImmediate(); + controller.abort(); + await expect(listing).rejects.toMatchObject({ reason: "cancelled" }); +}); + +it("rejects path expansion before publishing any inventory, preserving the input", async () => { + const bytes = nestedAsarHeader(64, "d".repeat(256), 4000); + expect(bytes.length).toBeLessThan(150_000); + const path = await archiveWith(bytes); + await expect(inventoryArtifact(path)).rejects.toMatchObject({ + reason: "limit", + message: "ASAR expanded listing exceeds its 64 MiB representation budget", + }); + expect(await readFile(path)).toEqual(bytes); +}); + +it("reads deeply nested packed metadata without recursive listing or member lookup", async () => { + const path = await archiveWith(nestedAsarHeader(3500, "d", 1)); + const reader = new AsarArtifactReader(path); + try { + let count = 0; + for await (const entry of reader.entries()) { + count++; + if (entry.kind === "file") { + expect(entry.path).toBe(`${"d/".repeat(3500)}f0`); + expect(await buffer(await reader.open(entry))).toEqual(Buffer.alloc(0)); + } + } + expect(count).toBe(3501); + } finally { + await reader.close(); + } +}); + +it.each(["..", ".", "", "a/../b", "a/b", "a\0b"])( + "rejects raw ASAR component %j before host normalization can alias another entry", + async (component) => { + const json = JSON.stringify({ + files: { [component]: { size: 0, offset: "0" } }, + }); + await expect( + inventoryArtifact(await archiveWith(asarHeaderBytes(json))), + ).rejects.toMatchObject({ reason: "path" }); + }, +); + +it("orders raw names portably and retains prototype-named entries and links", async () => { + const path = await archiveWith( + asarHeaderBytes( + '{"files":{"a0":{"size":0,"offset":"0"},"__proto__":{"size":0,"offset":"0"},"a":{"files":{"b":{"link":"a0"}}},"😀":{"size":0,"offset":"0"},"\uE000":{"size":0,"offset":"0"}}}', + ), + ); + const reader = new AsarArtifactReader(path); + try { + const entries = []; + for await (const entry of reader.entries()) + entries.push([entry.path, entry.kind]); + expect(entries).toEqual([ + ["__proto__", "file"], + ["a", "directory"], + ["a/b", "symlink"], + ["a0", "file"], + ["\uE000", "file"], + ["😀", "file"], + ]); + } finally { + await reader.close(); + } +}); + +it("preserves POSIX literal backslashes and rejects Windows component separators", async () => { + const path = await archiveWith( + asarHeaderBytes( + JSON.stringify({ files: { "a\\b": { size: 0, offset: "0" } } }), + ), + ); + const reader = new AsarArtifactReader(path); + try { + const list = async () => { + const entries = []; + for await (const entry of reader.entries()) entries.push(entry); + return entries; + }; + if (process.platform === "win32") { + await expect(list()).rejects.toMatchObject({ reason: "path" }); + } else { + const entries = await list(); + expect(entries).toHaveLength(1); + expect(entries[0]?.path).toBe("a\\b"); + expect(await buffer(await reader.open(entries[0]!))).toEqual( + Buffer.alloc(0), + ); + } + } finally { + await reader.close(); + } +}); diff --git a/tests/boundary/filesystem/ciChangeScope.test.ts b/tests/boundary/filesystem/ciChangeScope.test.ts index 321160a36..fd7abf65b 100644 --- a/tests/boundary/filesystem/ciChangeScope.test.ts +++ b/tests/boundary/filesystem/ciChangeScope.test.ts @@ -118,14 +118,21 @@ it("reuses only scheduled qualification and preserves every explicit full reques ).toBe("fast"); }); -it("keeps every translated root README and authored guide on the documentation lane", async () => { +it("keeps every translated README and authored guide on the documentation lane", async () => { const { directory, base } = await fixture(); const readmes = (await readdir(new URL("../../../", import.meta.url))).filter( (name) => /^README(?:_.*)?\.md$/u.test(name), ); + const translated = ( + await readdir(new URL("../../../docs/readme/", import.meta.url)) + ) + .filter((name) => /^README_.*\.md$/u.test(name)) + .map((name) => `docs/readme/${name}`); expect(readmes).toContain("README.md"); + expect(translated.length).toBeGreaterThan(0); const head = await change(directory, [ ...readmes, + ...translated, "docs/guide.md", "AGENTS.md", "CONTRIBUTING.md", @@ -198,7 +205,7 @@ it("emits an observational owner proposal without changing ordinary or full port it("does not charge a documentation PR for unrelated implementation updates on main", async () => { const { directory, base } = await fixture(); await git(directory, ["switch", "-c", "docs/change"]); - const head = await change(directory, ["README_zh-TW.md"]); + const head = await change(directory, ["docs/readme/README_zh-TW.md"]); await git(directory, ["switch", "main"]); const advancedBase = await change(directory, ["src/anotherProvider.ts"]); expect(advancedBase).not.toBe(base); @@ -240,7 +247,7 @@ it("retains complete checks when implementation is renamed into documentation", it("keeps main pushes scoped when a successful ancestor baseline is available", async () => { const { directory, base } = await fixture(); - const head = await change(directory, ["README_ja.md"]); + const head = await change(directory, ["docs/readme/README_ja.md"]); expect(await classify(directory, base, head, "push")).toMatchObject({ tier: "fast", code: false, @@ -449,6 +456,10 @@ it.each([ sqlite: false, }, ], + [ + "scripts/verify-solidity-metadata.mjs", + { code: true, evm: true, browser: false, package: false }, + ], [ "src/browser/execution/evaluate.ts", { @@ -470,7 +481,10 @@ it.each([ { code: false, website: true, package: false }, ], ["scripts/website_i18n.py", { code: false, website: true, package: false }], - ["README_ja.md", { code: false, docs: true, website: true, package: false }], + [ + "docs/readme/README_ja.md", + { code: false, docs: true, website: true, package: false }, + ], [ ".github/workflows/website-pages.yml", { code: false, website: true, package: false }, @@ -924,7 +938,7 @@ it.each(["workflow_dispatch", "schedule"])( it("honors ci:full for successive PR revisions", async () => { const { directory, base } = await fixture(); const first = await change(directory, ["README.md"]); - const second = await change(directory, ["README_ja.md"]); + const second = await change(directory, ["docs/readme/README_ja.md"]); for (const head of [first, second]) { expect( await classify(directory, base, head, "pull_request", true), diff --git a/tests/boundary/filesystem/ciRequiredLanes.test.ts b/tests/boundary/filesystem/ciRequiredLanes.test.ts index da6c43186..4c23844e7 100644 --- a/tests/boundary/filesystem/ciRequiredLanes.test.ts +++ b/tests/boundary/filesystem/ciRequiredLanes.test.ts @@ -365,7 +365,9 @@ it("keeps fast feedback separate from complete qualification and release proof", expect(ci.concurrency["cancel-in-progress"]).toBe( "${{ github.event_name == 'pull_request' && inputs.source_sha == '' && !contains(github.event.pull_request.labels.*.name, 'ci:full') }}", ); - expect(ci.jobs.required?.if).toBe("${{ always() }}"); + // Failed dependencies still reach the gate, but superseded PR runs must + // release their concurrency slot without waiting for another runner. + expect(ci.jobs.required?.if).toBe("${{ !cancelled() }}"); const shards = ci.jobs["test-shard"]?.steps ?? []; const fast = shards.find((step) => step.run?.includes("npm run test:run")); const full = shards.find((step) => step.run?.includes("test:ci:shard:run")); diff --git a/tests/boundary/filesystem/claudeDesktopClientConfiguration.test.ts b/tests/boundary/filesystem/claudeDesktopClientConfiguration.test.ts new file mode 100644 index 000000000..b411aeb4b --- /dev/null +++ b/tests/boundary/filesystem/claudeDesktopClientConfiguration.test.ts @@ -0,0 +1,412 @@ +import { + access, + mkdir, + readFile, + rm, + symlink, + writeFile, +} from "node:fs/promises"; +import { dirname, join } from "node:path"; + +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +import { CLAUDE_DESKTOP_PACKAGE_FAMILY } from "../../../src/application/ClaudeDesktopConfigLocation.js"; +import { readClientRegistrationStatuses } from "../../../src/application/ClientRegistrationStatus.js"; +import { npxRegistrationCommand } from "../../../src/application/ClientRegistrationIdentity.js"; +import { + configureClientConfiguration, + inspectClientConfiguration, +} from "../../../src/application/SetupClientConfiguration.js"; +import { detectClients } from "../../../src/application/SetupHost.js"; +import { + supportedClients, + type SetupClient, +} from "../../../src/application/SupportedClients.js"; +import { + runUninstall, + systemUninstallHost, +} from "../../../src/application/Uninstall.js"; +import { clearClientLocationEnvironment } from "../../fixtures/clientEnvironment.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +/** The Windows registration setup writes, independent of the test host. */ +const command = npxRegistrationCommand("win32"); +const CONFIG = "claude_desktop_config.json"; + +const roots: string[] = []; +beforeEach(clearClientLocationEnvironment); +afterEach(async () => { + vi.unstubAllEnvs(); + await Promise.all( + roots.splice(0).map((root) => rm(root, { recursive: true, force: true })), + ); +}); + +/** A Windows profile with separate roaming and local AppData roots. */ +const windowsProfile = async () => { + const home = await createTestTempDirectory("rea-claude-desktop-"); + roots.push(home); + const env = { + APPDATA: join(home, "AppData", "Roaming"), + LOCALAPPDATA: join(home, "AppData", "Local"), + }; + const appDataConfig = join(env.APPDATA, "Claude", CONFIG); + const container = join( + env.LOCALAPPDATA, + "Packages", + CLAUDE_DESKTOP_PACKAGE_FAMILY, + ); + const privateConfig = join( + container, + "LocalCache", + "Roaming", + "Claude", + CONFIG, + ); + return { home, env, appDataConfig, container, privateConfig }; +}; + +const claudeDesktop = ( + home: string, + env: Readonly>, +): SetupClient => { + const client = supportedClients(home, "win32", env).find( + ({ name }) => name === "claude_desktop", + ); + if (client === undefined) throw new Error("claude_desktop is not supported"); + return client; +}; + +const writeJson = async (path: string, value: unknown): Promise => { + await mkdir(dirname(path), { recursive: true }); + await writeFile(path, `${JSON.stringify(value, null, 2)}\n`); +}; + +const readJson = async (path: string): Promise> => + JSON.parse(await readFile(path, "utf8")) as Record; + +const reaEntry = { command: command[0], args: command.slice(1) }; + +describe("Windows Claude Desktop configuration location", () => { + it("uses AppData when no MSIX package storage exists", async () => { + const { home, env, appDataConfig } = await windowsProfile(); + await mkdir( + join(env.LOCALAPPDATA, "Packages", "Contoso.App_8wekyb3d8bbwe"), + { + recursive: true, + }, + ); + + const client = claudeDesktop(home, env); + + expect(client.configPath).toBe(appDataConfig); + expect(client.markerPath).toBe(dirname(appDataConfig)); + expect(client.msixPackage).toBeUndefined(); + expect(client.quitBeforeSetup).toBe(true); + }); + + it("uses the package-private copy that the MSIX package reads", async () => { + const { home, env, appDataConfig, container, privateConfig } = + await windowsProfile(); + await writeJson(privateConfig, { preferences: { theme: "dark" } }); + await writeJson(appDataConfig, { mcpServers: {} }); + + const client = claudeDesktop(home, env); + + expect(client.configPath).toBe(privateConfig); + expect(client.markerPath).toBe(container); + expect(client.msixPackage).toEqual({ + packageFamilyName: CLAUDE_DESKTOP_PACKAGE_FAMILY, + privateConfigPath: privateConfig, + appDataConfigPath: appDataConfig, + virtualizedRoots: [env.APPDATA, env.LOCALAPPDATA], + processView: "direct", + }); + expect( + (await detectClients(home, "win32", env)).map(({ name }) => name), + ).toContain("claude_desktop"); + }); + + it("uses AppData while the MSIX package has no private copy", async () => { + const { home, env, appDataConfig, container, privateConfig } = + await windowsProfile(); + await mkdir(container, { recursive: true }); + await writeJson(appDataConfig, { preferences: {} }); + + const client = claudeDesktop(home, env); + + expect(client.configPath).toBe(appDataConfig); + expect(client.msixPackage).toMatchObject({ + privateConfigPath: privateConfig, + processView: "unknown", + }); + // No private file shows whether the running package started this process. + const inspection = await inspectClientConfiguration(client, {}, command); + expect(inspection.status).toBe("invalid"); + expect(inspection.status === "invalid" && inspection.remediation).toContain( + "Open Claude Desktop once if it has never run, quit it fully, then rerun setup", + ); + }); + + it("creates the private copy when the MSIX package has no configuration", async () => { + const { home, env, privateConfig } = await windowsProfile(); + await writeJson(join(dirname(privateConfig), "Local State"), {}); + + const client = claudeDesktop(home, env); + expect(client.configPath).toBe(privateConfig); + expect(client.msixPackage?.processView).toBe("direct"); + await expect( + inspectClientConfiguration(client, {}, command), + ).resolves.toEqual({ status: "create" }); + await expect( + configureClientConfiguration(client, {}, command), + ).resolves.toEqual({ status: "configured" }); + expect(await readJson(privateConfig)).toEqual({ + mcpServers: { rea: reaEntry }, + }); + }); + + it("reports an unrecognized Claude package instead of falling back to AppData", async () => { + const { home, env, appDataConfig } = await windowsProfile(); + const foreign = join(env.LOCALAPPDATA, "Packages", "Claude_abcdefghjkmnp"); + await mkdir(foreign, { recursive: true }); + await writeJson(appDataConfig, { mcpServers: {} }); + + const client = claudeDesktop(home, env); + + expect(client.configPath).toBe(foreign); + expect(client.configPathError).toContain("unrecognized publisher"); + expect(client.configPathError).toContain(CLAUDE_DESKTOP_PACKAGE_FAMILY); + expect( + (await detectClients(home, "win32", env)).map(({ name }) => name), + ).not.toContain("claude_desktop"); + const status = ( + await readClientRegistrationStatuses(home, "unused", { + platform: "win32", + environment: env, + }) + ).find(({ client: name }) => name === "claude_desktop"); + expect(status).toMatchObject({ state: "invalid", config_path: foreign }); + await expect( + inspectClientConfiguration(client, {}, command), + ).resolves.toMatchObject({ status: "invalid" }); + }); +}); + +describe("MSIX Claude Desktop registration lifecycle", () => { + it("registers in the private copy and reports an AppData-only entry as missing", async () => { + const { home, env, appDataConfig, privateConfig } = await windowsProfile(); + const privateOriginal = { + coworkUserFilesPath: "C:\\Users\\a\\Claude", + preferences: { menuBarEnabled: false }, + mcpServers: { other: { command: "other", args: ["serve"] } }, + }; + await writeJson(privateConfig, privateOriginal); + // Earlier setups wrote AppData, which the package never reads. + await writeJson(appDataConfig, { mcpServers: { rea: reaEntry } }); + const appDataOriginal = await readFile(appDataConfig, "utf8"); + const status = async () => + ( + await readClientRegistrationStatuses(home, "unused", { + platform: "win32", + environment: env, + }) + ).find(({ client }) => client === "claude_desktop"); + + expect(await status()).toMatchObject({ + state: "missing", + config_path: privateConfig, + }); + expect((await status())?.remediation).toContain( + "Quit Claude Desktop fully", + ); + + const client = claudeDesktop(home, env); + await expect( + inspectClientConfiguration(client, {}, command), + ).resolves.toEqual({ + status: "update", + backupPath: `${privateConfig}.rea.backup`, + }); + await expect( + configureClientConfiguration(client, {}, command), + ).resolves.toEqual({ + status: "configured", + backupPath: `${privateConfig}.rea.backup`, + }); + + expect(await readJson(privateConfig)).toEqual({ + ...privateOriginal, + mcpServers: { + ...privateOriginal.mcpServers, + rea: reaEntry, + }, + }); + expect(await readJson(`${privateConfig}.rea.backup`)).toEqual( + privateOriginal, + ); + expect(await readFile(appDataConfig, "utf8")).toBe(appDataOriginal); + expect(await status()).toMatchObject({ + state: "aligned", + config_path: privateConfig, + }); + await expect( + configureClientConfiguration(client, {}, command), + ).resolves.toEqual({ status: "unchanged" }); + }); + + it("refuses to write from a process the running package started", async () => { + const { home, env, privateConfig } = await windowsProfile(); + await writeJson(privateConfig, { preferences: {} }); + const resolved = claudeDesktop(home, env); + if (resolved.msixPackage === undefined) + throw new Error("expected MSIX package storage"); + // Only a packaged process sees AppData redirected; model that observation. + const client: SetupClient = { + ...resolved, + msixPackage: { ...resolved.msixPackage, processView: "redirected" }, + }; + + const inspection = await inspectClientConfiguration(client, {}, command); + + expect(inspection.status).toBe("invalid"); + expect(inspection.status === "invalid" && inspection.remediation).toContain( + "Quit Claude Desktop fully, then rerun setup from a terminal that Claude Desktop did not start", + ); + expect(await readJson(privateConfig)).toEqual({ preferences: {} }); + + await configureClientConfiguration(resolved, {}, command); + await expect( + inspectClientConfiguration(client, {}, command), + ).resolves.toEqual({ status: "already_current" }); + }); + + it("refuses a write once the package reads a newer private copy", async () => { + const { home, env, appDataConfig, privateConfig } = await windowsProfile(); + await writeJson(join(dirname(privateConfig), "Preferences"), {}); + await writeJson(appDataConfig, { preferences: {} }); + const client = claudeDesktop(home, env); + expect(client.configPath).toBe(appDataConfig); + // Quitting the app after the plan was shown can save its private copy. + await writeJson(privateConfig, { preferences: {} }); + + await expect( + configureClientConfiguration(client, {}, command), + ).resolves.toEqual({ status: "failed", reason: "target_changed" }); + expect(await readJson(appDataConfig)).toEqual({ preferences: {} }); + expect(await readJson(privateConfig)).toEqual({ preferences: {} }); + }); + + it("registers in AppData while the package has no private copy", async () => { + const { home, env, appDataConfig, privateConfig } = await windowsProfile(); + await writeJson(join(dirname(privateConfig), "Preferences"), {}); + await writeJson(appDataConfig, { preferences: { theme: "light" } }); + + const client = claudeDesktop(home, env); + expect(client.configPath).toBe(appDataConfig); + expect(client.msixPackage?.processView).toBe("direct"); + await expect( + configureClientConfiguration(client, {}, command), + ).resolves.toMatchObject({ status: "configured" }); + + expect(await readJson(appDataConfig)).toEqual({ + preferences: { theme: "light" }, + mcpServers: { rea: reaEntry }, + }); + await expect(access(privateConfig)).rejects.toMatchObject({ + code: "ENOENT", + }); + }); +}); + +describe("MSIX Claude Desktop uninstall", () => { + it("refuses uninstall when the package view is unknown", async () => { + const { home, env, appDataConfig, container } = await windowsProfile(); + await mkdir(container, { recursive: true }); + await writeJson(appDataConfig, { mcpServers: { rea: reaEntry } }); + + const result = await runUninstall( + false, + systemUninstallHost(home, undefined, env, "win32"), + ); + + expect(result.status).toBe("failed"); + expect(result.items).toContainEqual( + expect.objectContaining({ + name: "claude_desktop", + status: "failed", + detail: expect.stringContaining("rerun uninstall"), + }), + ); + expect(await readJson(appDataConfig)).toEqual({ + mcpServers: { rea: reaEntry }, + }); + }); + + it("retains an unrelated rea-named entry when the package view is unknown", async () => { + const { home, env, appDataConfig, container } = await windowsProfile(); + await mkdir(container, { recursive: true }); + const unrelated = { mcpServers: { rea: { command: "unrelated-server" } } }; + await writeJson(appDataConfig, unrelated); + const result = await runUninstall( + false, + systemUninstallHost(home, undefined, env, "win32"), + ); + expect(result.status).toBe("complete"); + expect(await readJson(appDataConfig)).toEqual(unrelated); + }); + + it("blocks managed skill removal through redirected AppData before any removal", async () => { + const { home, env, appDataConfig, privateConfig } = await windowsProfile(); + await writeJson(join(dirname(privateConfig), "Preferences"), {}); + await mkdir(dirname(dirname(appDataConfig)), { recursive: true }); + await symlink(dirname(privateConfig), dirname(appDataConfig), "dir"); + const agents = join(env.APPDATA, "dsh", "agents"); + const manifest = join( + agents, + "skills", + "reverse-engineer-anything", + "SKILL.md", + ); + await mkdir(dirname(manifest), { recursive: true }); + await writeFile( + manifest, + "---\nname: reverse-engineer-anything\n---\nmanaged fixture\n", + ); + const environment = { ...env, DSH_AGENTS_HOME: agents }; + const result = await runUninstall( + false, + systemUninstallHost(home, undefined, environment, "win32"), + ); + expect(result.status).toBe("failed"); + expect(result.items).toContainEqual( + expect.objectContaining({ + name: "skill", + status: "failed", + detail: expect.stringContaining("redirects its writes"), + }), + ); + expect(await readFile(manifest, "utf8")).toContain("managed fixture"); + }); + + it("removes REA from both the private copy and AppData on uninstall", async () => { + const { home, env, appDataConfig, privateConfig } = await windowsProfile(); + await writeJson(privateConfig, { + preferences: {}, + mcpServers: { other: { command: "other" }, rea: reaEntry }, + }); + await writeJson(appDataConfig, { mcpServers: { rea: reaEntry } }); + + const result = await runUninstall( + false, + systemUninstallHost(home, undefined, env, "win32"), + ); + + expect(result.status).toBe("complete"); + expect(await readJson(privateConfig)).toEqual({ + preferences: {}, + mcpServers: { other: { command: "other" } }, + }); + expect(await readJson(appDataConfig)).toEqual({ mcpServers: {} }); + }); +}); diff --git a/tests/boundary/filesystem/clientLifecycle.test.ts b/tests/boundary/filesystem/clientLifecycle.test.ts index 63ca0667a..0e4054ae6 100644 --- a/tests/boundary/filesystem/clientLifecycle.test.ts +++ b/tests/boundary/filesystem/clientLifecycle.test.ts @@ -64,6 +64,7 @@ describe("client configuration filesystem lifecycle", () => { "copilot_cli", "commandcode", "qwen_code", + "kimi", "vscode", "trae", "grok_build", @@ -1895,14 +1896,11 @@ describe("client configuration filesystem failure reporting", () => { remove: () => Promise.reject(new Error("SECRET removal failure")), }), ); - expect(result.items).toContainEqual( - expect.objectContaining({ - name: "skill", - status: "failed", - detail: expect.stringContaining( - `Managed file could not be removed: ${join(skillRoot, "SKILL.md")}`, - ), - }), + const skillItem = result.items.find(({ name }) => name === "skill"); + expect(skillItem?.status).toBe("failed"); + expect(skillItem?.detail).toContain(`shared (${skillRoot}): failed (`); + expect(skillItem?.detail).toContain( + `Managed file could not be removed: ${join(skillRoot, "SKILL.md")}`, ); expect(JSON.stringify(result)).not.toContain("SECRET"); expect(await readFile(join(skillRoot, "SKILL.md"), "utf8")).toBe( @@ -1944,6 +1942,42 @@ describe("client configuration filesystem failure reporting", () => { }); }); +describe("skill root client identifier labels", () => { + it("labels each retained or removed skill root by its client identifier", async () => { + const home = await createTestTempDirectory("rea-uninstall-skill-labels-"); + roots.push(home); + const customClaudeDirectory = join(home, "claude-config"); + const sharedRoot = join(home, ".agents/skills/reverse-engineer-anything"); + const claudeRoot = join( + customClaudeDirectory, + "skills/reverse-engineer-anything", + ); + await mkdir(sharedRoot, { recursive: true }); + await mkdir(claudeRoot, { recursive: true }); + await writeFile( + join(sharedRoot, "SKILL.md"), + "---\nname: another-skill\n---\n# Not REA-owned\n", + ); + await writeFile(join(claudeRoot, "SKILL.md"), ownedSkillManifest); + + const skillItem = await systemUninstallHost(home, testFileSystem, { + CLAUDE_CONFIG_DIR: customClaudeDirectory, + }).removeSkill(); + + expect(skillItem.name).toBe("skill"); + expect(skillItem.detail).toContain(`shared (${sharedRoot}): retained (`); + expect(skillItem.detail).toContain( + `claude_code (${claudeRoot}): removed (`, + ); + await expect( + readFile(join(claudeRoot, "SKILL.md"), "utf8"), + ).rejects.toMatchObject({ code: "ENOENT" }); + expect(await readFile(join(sharedRoot, "SKILL.md"), "utf8")).toContain( + "another-skill", + ); + }); +}); + const testFileSystem: UninstallFileSystem = { readText: (path) => readFile(path, "utf8"), copy: (source, destination) => diff --git a/tests/boundary/filesystem/dshClientConfiguration.test.ts b/tests/boundary/filesystem/dshClientConfiguration.test.ts index c0315f67b..fd65e6b00 100644 --- a/tests/boundary/filesystem/dshClientConfiguration.test.ts +++ b/tests/boundary/filesystem/dshClientConfiguration.test.ts @@ -2195,3 +2195,135 @@ describe("DeepSeek Harness caller node metadata", () => { }, ); }); + +describe("DeepSeek Harness uninstall of a shared operation", () => { + it("keeps a group patch's own fields and comment when its only row was REA's", async () => { + const { home, client } = await harnessClient(); + const entry = canonicalEntry(); + await writeFile( + client.configPath, + `- insert: + - id: user-a + name: plugin-a +# caller group patch +- id: tools + config: + sandbox: strict + insert: + - id: rea-mcp + name: "${entry.name}" + config: + serverName: rea + transport: stdio + command: ${entry.config.command} + args: ${JSON.stringify(entry.config.args)} +`, + ); + + expect((await systemUninstallHost(home).removeClient(client)).status).toBe( + "removed", + ); + const text = await readFile(client.configPath, "utf8"); + expect(parseYaml(text)).toEqual([ + { insert: [{ id: "user-a", name: "plugin-a" }] }, + { id: "tools", config: { sandbox: "strict" } }, + ]); + expect(text).toContain("# caller group patch"); + }); + + it("keeps an operation's own fields when setup removes a second REA row from it", async () => { + // Setup refuses a REA row in a group patch, so the setup path meets the + // keep rule only on an operation without a selector. + const { home, client } = await harnessClient(); + const entry = canonicalEntry(); + const row = (launcher: string) => ` - id: rea-mcp + name: "${entry.name}" + config: + serverName: rea + transport: stdio + command: ${launcher} + args: [mcp] +`; + await writeFile( + client.configPath, + `- insert: +${row("first-launcher")}# caller operation +- disabled: false + insert: +${row("second-launcher")}`, + ); + + expect( + (await configureClientConfiguration(client, {}, command)).status, + ).toBe("configured"); + const text = await readFile(client.configPath, "utf8"); + expect(text).toContain("# caller operation"); + expect(loadedInserts(text)).toHaveLength(1); + expect((parseYaml(text) as unknown[]).at(-1)).toEqual({ disabled: false }); + expect(await dshStatus(home)).toMatchObject({ state: "aligned" }); + }); + + it("drops an emptied operation that targets REA's own row", async () => { + const { home, client } = await harnessClient(); + await writeFile( + client.configPath, + stringifyYaml([ + { insert: [{ id: "user-a", name: "plugin-a" }] }, + { + id: "rea-mcp", + config: { serverName: "rea" }, + insert: [canonicalEntry()], + }, + ]), + ); + + const host = systemUninstallHost(home); + expect((await host.removeClient(client)).status).toBe("removed"); + expect(parseYaml(await readFile(client.configPath, "utf8"))).toEqual([ + { insert: [{ id: "user-a", name: "plugin-a" }] }, + ]); + }); + + it.each([ + { selector: { id: "tools" } }, + { selector: { name: "tools" } }, + { selector: { id: "tools", name: "tools" } }, + ])( + "drops an emptied operation left with only its target selector $selector", + async ({ selector }) => { + const { home, client } = await harnessClient(); + await writeFile( + client.configPath, + stringifyYaml([ + { insert: [{ id: "user-a", name: "plugin-a" }] }, + { ...selector, insert: [canonicalEntry()] }, + ]), + ); + + expect( + (await systemUninstallHost(home).removeClient(client)).status, + ).toBe("removed"); + expect(parseYaml(await readFile(client.configPath, "utf8"))).toEqual([ + { insert: [{ id: "user-a", name: "plugin-a" }] }, + ]); + }, + ); + + it("still drops an insert-only operation that REA's row emptied", async () => { + const { home, client } = await harnessClient(); + await writeFile( + client.configPath, + stringifyYaml([ + { insert: [{ id: "user-a", name: "plugin-a" }] }, + { insert: [canonicalEntry()] }, + ]), + ); + + expect((await systemUninstallHost(home).removeClient(client)).status).toBe( + "removed", + ); + expect(parseYaml(await readFile(client.configPath, "utf8"))).toEqual([ + { insert: [{ id: "user-a", name: "plugin-a" }] }, + ]); + }); +}); diff --git a/tests/boundary/filesystem/flutterBuildProvider.test.ts b/tests/boundary/filesystem/flutterBuildProvider.test.ts index 4a1c8248f..3773378f2 100644 --- a/tests/boundary/filesystem/flutterBuildProvider.test.ts +++ b/tests/boundary/filesystem/flutterBuildProvider.test.ts @@ -1,5 +1,6 @@ import { createReadStream } from "node:fs"; import { open, readFile, rm, stat, utimes, writeFile } from "node:fs/promises"; +import { createServer } from "node:net"; import { join } from "node:path"; import { Readable } from "node:stream"; import { Uint8ArrayReader, Uint8ArrayWriter, ZipWriter } from "@zip.js/zip.js"; @@ -1122,11 +1123,16 @@ it.each(["identify_flutter_build", "inspect_dart_aot"] as const)( if (result.ok) throw new Error("Duplicate libraries must be refused"); expect(projectAnalysisError(result.error)).toMatchObject({ code: "invalid_request", + details: { + issues: [ + { + message: expect.stringContaining( + `duplicate Flutter library members: ${path}:`, + ), + }, + ], + }, }); - expect(JSON.stringify(projectAnalysisError(result.error))).toContain( - "duplicate Flutter library members", - ); - expect(JSON.stringify(projectAnalysisError(result.error))).toContain(path); expect(extract).not.toHaveBeenCalled(); }, ); @@ -1171,6 +1177,69 @@ it.each(["identify_flutter_build", "inspect_dart_aot"] as const)( }, ); +const expectSelectionIssue = async ( + provider: FlutterBuildProvider, + path: string, + message: string, +): Promise => { + for (const operation of [ + "identify_flutter_build", + "inspect_dart_aot", + ] as const) { + const result = await provider.execute({ operation, input: { path } }); + if (result.ok) throw new Error("A non-file APK selection must fail"); + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "invalid_request", + details: { + issues: [ + { + path: ["path"], + reason: "invalid_format", + message: `${message}: ${path}`, + }, + ], + }, + }); + } +}; + +it("names a missing or non-file APK selection instead of blaming package bytes", async () => { + const root = await createTestTempDirectory("rea-flutter-selection-"); + const provider = new FlutterBuildProvider(); + onTestFinished(() => provider.close()); + await expectSelectionIssue( + provider, + join(root, "absent.apk"), + "Selected APK does not exist", + ); + await expectSelectionIssue( + provider, + root, + "Selected APK is not a regular file", + ); +}); + +// Windows listens on named pipes rather than filesystem sockets. +it.skipIf(process.platform === "win32")( + "names an unopenable Unix socket APK selection as a non-file", + async () => { + const root = await createTestTempDirectory("rea-flutter-socket-"); + const socketPath = join(root, "s.sock"); + const socket = createServer(); + await new Promise((resolve) => socket.listen(socketPath, resolve)); + onTestFinished( + () => new Promise((resolve) => socket.close(() => resolve())), + ); + const provider = new FlutterBuildProvider(); + onTestFinished(() => provider.close()); + await expectSelectionIssue( + provider, + socketPath, + "Selected APK is not a regular file", + ); + }, +); + it.each(["EACCES", "EPERM"] as const)( "preserves %s at the shared source admission boundary", async (code) => { diff --git a/tests/boundary/filesystem/flutterElfNotes.test.ts b/tests/boundary/filesystem/flutterElfNotes.test.ts new file mode 100644 index 000000000..bdf461117 --- /dev/null +++ b/tests/boundary/filesystem/flutterElfNotes.test.ts @@ -0,0 +1,79 @@ +import { open, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { expect, it, vi } from "vitest"; +import { readElfBuildIds } from "../../../src/flutter/ElfBuildId.js"; +import { + appendElfRanges, + engineFixtureImage, +} from "../../../src/flutter/FlutterBuild.fixture.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +const note = Buffer.from( + "040000000800000003000000474e55000123456789abcdef", + "hex", +); + +it.each([ + { bits: 32, table: "program" }, + { bits: 64, table: "section" }, +] as const)( + "bounds dense $bits-bit $table note ranges while retaining examined IDs", + async ({ bits, table }) => { + const count = 30_000; + const image = appendElfRanges( + engineFixtureImage({ + bits, + notes: Buffer.concat([note, Buffer.alloc(count)]), + ownership: "both", + }), + table, + Array.from({ length: count }, (_, index) => ({ + type: table === "program" ? 4 : 7, + offset: 0x400 + note.length + index, + size: 1, + })), + ); + const root = await createTestTempDirectory("rea-flutter-notes-"); + const path = join(root, "engine.so"); + await writeFile(path, image); + const file = await open(path, "r"); + const read = vi.spyOn(file, "read"); + try { + const result = await readElfBuildIds(file, image.length, undefined); + expect(result.candidates).toEqual(["0123456789abcdef"]); + expect(result.failures).toContainEqual({ + reason: expect.stringContaining("note-range"), + }); + expect(read.mock.calls.length).toBeLessThanOrEqual(1027); + } finally { + await file.close(); + } + }, +); + +it("deduplicates shared program/section notes and skips empty ranges before charging inventory", async () => { + const ranges = Array.from({ length: 2000 }, (_, index) => ({ + type: 7, + offset: index % 2 === 0 ? 0x400 : index, + size: index % 2 === 0 ? note.length : 0, + })); + const image = appendElfRanges( + engineFixtureImage({ notes: note, ownership: "both" }), + "section", + ranges, + ); + const root = await createTestTempDirectory("rea-flutter-notes-"); + const path = join(root, "engine.so"); + await writeFile(path, image); + const file = await open(path, "r"); + const read = vi.spyOn(file, "read"); + try { + expect(await readElfBuildIds(file, image.length, undefined)).toEqual({ + candidates: ["0123456789abcdef"], + failures: [], + }); + expect(read.mock.calls.length).toBe(4); + } finally { + await file.close(); + } +}); diff --git a/tests/boundary/filesystem/installScript.test.ts b/tests/boundary/filesystem/installScript.test.ts index 0f2e14730..d7e7f78a3 100644 --- a/tests/boundary/filesystem/installScript.test.ts +++ b/tests/boundary/filesystem/installScript.test.ts @@ -209,13 +209,7 @@ describe("installer TTY fallback", { timeout: 20_000 }, () => { cwd: process.cwd(), env: { ...process.env, - HOME: fixture.home, - TMPDIR: fixture.temporary, - PATH: `${fixture.bin}:/usr/bin:/bin`, - FAKE_NPM_LOG: fixture.npmLog, - FAKE_REA_LOG: fixture.reaLog, - FAKE_REA_SOURCE: join(fixture.bin, "rea-source"), - FAKE_NPM_PREFIX: join(fixture.home, ".npm-global"), + ...installerEnvironment(fixture), TERM: "xterm-256color", }, name: "xterm-256color", @@ -581,6 +575,19 @@ const executable = async (path: string, source: string): Promise => { await chmod(path, 0o755); }; +/** Fixture-owned installer environment shared by the exec, pipe and PTY runs. */ +const installerEnvironment = ( + fixture: InstallerFixture, +): Record => ({ + HOME: fixture.home, + TMPDIR: fixture.temporary, + PATH: `${fixture.bin}:/usr/bin:/bin`, + FAKE_NPM_LOG: fixture.npmLog, + FAKE_REA_LOG: fixture.reaLog, + FAKE_REA_SOURCE: join(fixture.bin, "rea-source"), + FAKE_NPM_PREFIX: join(fixture.home, ".npm-global"), +}); + const runInstaller = async ( fixture: InstallerFixture, args: readonly string[] = [], @@ -590,13 +597,7 @@ const runInstaller = async ( execFileAsync("/bin/bash", [script, ...args], { env: { ...process.env, - HOME: fixture.home, - TMPDIR: fixture.temporary, - PATH: `${fixture.bin}:/usr/bin:/bin`, - FAKE_NPM_LOG: fixture.npmLog, - FAKE_REA_LOG: fixture.reaLog, - FAKE_REA_SOURCE: join(fixture.bin, "rea-source"), - FAKE_NPM_PREFIX: join(fixture.home, ".npm-global"), + ...installerEnvironment(fixture), ...overrides, }, }); @@ -618,13 +619,7 @@ const runPipedInstaller = async ( new Promise((resolve, reject) => { const child = spawn("/bin/bash", ["-s", "--", ...args], { env: { - HOME: fixture.home, - TMPDIR: fixture.temporary, - PATH: `${fixture.bin}:/usr/bin:/bin`, - FAKE_NPM_LOG: fixture.npmLog, - FAKE_REA_LOG: fixture.reaLog, - FAKE_REA_SOURCE: join(fixture.bin, "rea-source"), - FAKE_NPM_PREFIX: join(fixture.home, ".npm-global"), + ...installerEnvironment(fixture), ...overrides, }, stdio: ["pipe", "pipe", "pipe"], diff --git a/tests/boundary/filesystem/interfaceBuilderBundle.test.ts b/tests/boundary/filesystem/interfaceBuilderBundle.test.ts index d584aaf3c..543248e63 100644 --- a/tests/boundary/filesystem/interfaceBuilderBundle.test.ts +++ b/tests/boundary/filesystem/interfaceBuilderBundle.test.ts @@ -405,12 +405,16 @@ describe("keyed archive hierarchy coverage", () => { const resources = join(bundle, "Contents", "Resources"); await mkdir(resources, { recursive: true }); const archive = join(resources, "Foundation.nib"); - await compile("/usr/bin/xcrun", [ - "swift", - "tests/conformance/native/keyed-archive-hierarchy.swift", - archive, - "40", - ]); + await compile( + "/usr/bin/xcrun", + [ + "swift", + "tests/conformance/native/keyed-archive-hierarchy.swift", + archive, + "40", + ], + { timeout: 90_000 }, + ); const analysis = await analyzeInterfaceBuilderBundle({ bundlePath: bundle, @@ -431,6 +435,7 @@ describe("keyed archive hierarchy coverage", () => { ), ).toHaveLength(41); }, + 120_000, ); }); @@ -696,6 +701,49 @@ describe("NIB projection byte amplification", () => { }); }); +cliTest( + "compiled CLI preserves distinct NIB keys and class identity", + async ({ cli }) => { + const root = await createTestTempDirectory("rea-nib-name-identity-"); + const bundle = join(root, "Example.app"); + const resources = join(bundle, "Contents", "Resources"); + await mkdir(resources, { recursive: true }); + await mkdir(join(bundle, "Contents", "MacOS"), { recursive: true }); + await writeFile( + join(bundle, "Contents", "MacOS", "App"), + thinMach(0xfeedfacf, 0x0100000c), + ); + await writeFile( + join(bundle, "Contents", "Info.plist"), + "CFBundleExecutableApp", + ); + await writeFile( + join(resources, "Identity.nib"), + encodeNibArchiveFixture({ + classes: ["\uFEFFWidget"], + objects: [{ classIndex: 0, values: { count: 1, "\uFEFFcount": 2 } }], + }), + ); + const output = await cli.run({ + arguments: ["decode-interface-builder", bundle, "--json"], + environment: { REA_LOG_LEVEL: "silent", REA_ANALYSIS_PROVIDER: "auto" }, + }); + expect(output.exitCode, output.stderr).toBe(0); + const result = interfaceBuilderAnalysisSchema.parse( + parseEvidence(output.json).normalized_result, + ); + expect(result.graph.nodes).toContainEqual( + expect.objectContaining({ + attributes: expect.objectContaining({ + class_name: "\uFEFFWidget", + nibValues: { count: 1, "\uFEFFcount": 2 }, + }), + }), + ); + expect(JSON.stringify(result)).toContain("\uFEFFWidget"); + }, +); + cliTest( "compiled CLI preserves a good archive and reports repeated key and data omissions", async ({ cli }) => { diff --git a/tests/boundary/filesystem/javascript-applications/javascriptApplicationEmptyLiterals.test.ts b/tests/boundary/filesystem/javascript-applications/javascriptApplicationEmptyLiterals.test.ts index 0306f250f..0490ae211 100644 --- a/tests/boundary/filesystem/javascript-applications/javascriptApplicationEmptyLiterals.test.ts +++ b/tests/boundary/filesystem/javascript-applications/javascriptApplicationEmptyLiterals.test.ts @@ -9,7 +9,10 @@ import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js import { javascriptApplicationAnalysisResultSchema } from "../../../../src/domain/javascript/javascriptApplicationAnalysis.js"; import { matchJavaScriptApplicationVersions } from "../../../../src/domain/javascript/javascriptApplicationVersionKeys.js"; import { findApplicationFeatureSeeds } from "../../../../src/domain/javascript/javascriptFeatureSeed.js"; -import { traceApplicationFeatureInputSchema } from "../../../../src/domain/javascript/javascriptFeatureTraceSchemas.js"; +import { + applicationFeatureSeedSchema, + traceApplicationFeatureInputSchema, +} from "../../../../src/domain/javascript/javascriptFeatureTraceSchemas.js"; import { analyzeJavaScriptApplication } from "../../../support/javascriptApplicationScope.js"; const analyzeSource = async ( @@ -76,6 +79,59 @@ it("retains empty literal evidence across the application boundary surfaces", as ); }); +it("keeps bridge entry paths stable through worker validation and API seed matching", async () => { + const output = await analyzeSource(` + const { contextBridge, ipcRenderer } = require("electron"); + const literal = () => ipcRenderer.send("literal"); + const nested = () => ipcRenderer.send("nested"); + contextBridge.exposeInMainWorld("desktop", { + "nested.write": literal, + nested: { write: nested }, + }); + `); + const semanticPathIds = new Set( + (output.semantic_graph?.nodes ?? []) + .map(({ properties }) => properties.path_id) + .filter((pathId): pathId is string => typeof pathId === "string"), + ); + const applicationPathIds = new Set( + output.graph.nodes.flatMap(({ observations }) => + observations + .map(({ properties }) => properties.path_id) + .filter((pathId): pathId is string => typeof pathId === "string"), + ), + ); + expect(semanticPathIds).toEqual(applicationPathIds); + + const matches = findApplicationFeatureSeeds( + output.graph.nodes, + applicationFeatureSeedSchema.parse({ + kind: "api", + value: "desktop.nested.write", + entry_path: ["desktop", "nested.write"], + }), + ); + expect(matches).toHaveLength(1); + expect( + output.graph.nodes + .find(({ node_id: nodeId }) => nodeId === matches[0]?.node_id) + ?.observations.some( + ({ properties }) => + Array.isArray(properties.entry_path) && + properties.entry_path.join("\0") === "desktop\0nested.write", + ), + ).toBe(true); + expect( + findApplicationFeatureSeeds( + output.graph.nodes, + applicationFeatureSeedSchema.parse({ + kind: "api", + value: "desktop.nested.write", + }), + ), + ).toHaveLength(2); +}); + it("analyzes a plain long-line loader without package metadata", async () => { const output = await analyzeSource( `var Module = {}; fetch(""); new Worker(""); var payload = "${"x".repeat(65_536)}";`, @@ -240,8 +296,18 @@ it.each([ [{ kind: "channel", value: "" }, false], [{ kind: "channel", value: "", match: "contains" }, false], [{ kind: "node-id", value: "", match: "exact" }, false], + [{ kind: "api", value: "desktop.nested.write" }, true], + [ + { + kind: "api", + value: "desktop.nested.write", + entry_path: ["desktop", "nested.write"], + }, + true, + ], + [{ kind: "channel", value: "channel", entry_path: ["channel"] }, false], ] as const)( - "validates empty seed %j at runtime and in the advertised schema", + "validates seed %j at runtime and in the advertised schema", (seed, accepted) => { expect(seedSchema.safeParse(seed).success).toBe(accepted); expect(validateAdvertisedSeed(seed)).toBe(accepted); diff --git a/tests/boundary/filesystem/javascriptArtifactReconstruction.test.ts b/tests/boundary/filesystem/javascriptArtifactReconstruction.test.ts index aabfdd19b..1254d11dd 100644 --- a/tests/boundary/filesystem/javascriptArtifactReconstruction.test.ts +++ b/tests/boundary/filesystem/javascriptArtifactReconstruction.test.ts @@ -135,7 +135,6 @@ it("retains every source declared by every included local source map", async () it.each([ ["a non-array sourcesContent value", { sourcesContent: "source" }], ["a non-string, non-null source entry", { sourcesContent: [42] }], - ["a source-content array with the wrong length", { sourcesContent: [] }], ])( "reports %s as an invalid local source map", async (_description, content) => { @@ -210,6 +209,51 @@ it("accepts omitted and explicitly null source-map source contents", async () => expect(result.graph.coverage.status).toBe("complete"); }); +it.each([ + ["shorter", ["export const answer = 42;"]], + ["longer", ["export const answer = 42;", null, "unused trailing content"]], +])( + "keeps every source declaration when sourcesContent is %s than sources", + async (length, sourcesContent) => { + const root = await createTestTempDirectory( + `rea-javascript-${length}-source-content-`, + ); + await writeFile( + join(root, "bundle.js.map"), + JSON.stringify({ + version: 3, + sources: ["a.ts", "b.ts"], + sourcesContent, + names: [], + mappings: "AAAA;ACAA", + }), + ); + + const result = await reconstructJavaScriptArtifact({ input_path: root }); + const originals = result.graph.nodes.filter( + ({ identity }) => identity.strategy === "source-map-original", + ); + + expect( + originals + .map(({ identity, observations }) => ({ + original_source: + "original_source" in identity ? identity.original_source : null, + source_index: observations[0]?.properties.source_index, + content_available: observations[0]?.properties.content_available, + })) + .sort( + (left, right) => + Number(left.source_index) - Number(right.source_index), + ), + ).toEqual([ + { original_source: "a.ts", source_index: 0, content_available: true }, + { original_source: "b.ts", source_index: 1, content_available: false }, + ]); + expect(result.graph.coverage.status).toBe("complete"); + }, +); + it("retains unnamed source declarations without shifting later source evidence", async () => { const root = await createTestTempDirectory("rea-javascript-unnamed-source-"); const text = JSON.stringify({ diff --git a/tests/boundary/filesystem/kimiClientConfiguration.test.ts b/tests/boundary/filesystem/kimiClientConfiguration.test.ts new file mode 100644 index 000000000..e0580031a --- /dev/null +++ b/tests/boundary/filesystem/kimiClientConfiguration.test.ts @@ -0,0 +1,384 @@ +import { access, mkdir, readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +import { readClientRegistrationStatuses } from "../../../src/application/ClientRegistrationStatus.js"; +import { setupRegistrationCommand } from "../../../src/application/SetupHost.js"; +import { configureClientConfiguration } from "../../../src/application/SetupClientConfiguration.js"; +import { + clientSkillDirectories, + supportedClients, +} from "../../../src/application/SupportedClients.js"; +import { systemUninstallHost } from "../../../src/application/Uninstall.js"; +import { MCP_STARTUP_POLICY } from "../../../src/mcpStartupPolicy.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { + clearClientLocationEnvironment, + NPX_REGISTRATION_COMMAND, +} from "../../fixtures/clientEnvironment.js"; + +const command = NPX_REGISTRATION_COMMAND; + +beforeEach(clearClientLocationEnvironment); +afterEach(() => vi.unstubAllEnvs()); + +const kimiPath = ( + home: string, + platform: NodeJS.Platform, + env: Parameters[2] = {}, +) => supportedClients(home, platform, env).find(({ name }) => name === "kimi"); + +const exists = async (path: string): Promise => { + try { + await access(path); + return true; + } catch { + return false; + } +}; + +const expectedRegistration = (launcher: readonly string[]) => ({ + command: launcher[0], + args: launcher.slice(1), + startupTimeoutMs: MCP_STARTUP_POLICY.kimiStartupTimeoutMs, + toolTimeoutMs: MCP_STARTUP_POLICY.kimiToolTimeoutMs, +}); + +describe("Kimi Code configuration paths and skills", () => { + it.each([ + [undefined, join("/home/a/.kimi-code")], + ["", join("/home/a/.kimi-code")], + [join("/tmp", "isolated-kimi"), join("/tmp", "isolated-kimi")], + ])("resolves KIMI_CODE_HOME for %j", (override, directory) => { + const client = kimiPath( + "/home/a", + "linux", + override === undefined ? {} : { KIMI_CODE_HOME: override }, + ); + expect(client).toEqual({ + name: "kimi", + displayName: "Kimi Code", + configPath: join(directory, "mcp.json"), + markerPath: directory, + format: "kimi", + }); + }); + + it("uses the existing shared skill destination without a separate skill root", () => { + expect(clientSkillDirectories("/home/a", ["kimi"])).toEqual([ + { client: "shared", directory: "/home/a/.agents/skills" }, + ]); + }); +}); + +describe("Kimi Code registration lifecycle", () => { + const kimiClient = async () => { + const home = await createTestTempDirectory("rea-kimi-"); + const client = kimiPath(home, process.platform, {}); + if (client?.markerPath === undefined) + throw new Error("missing Kimi Code client"); + await mkdir(client.markerPath, { recursive: true }); + return { home, client }; + }; + + it("preserves the caller's enabledTools selection during refresh", async () => { + const { client } = await kimiClient(); + const enabledTools = ["binary_session", "analyze_javascript_application"]; + await writeFile( + client.configPath, + JSON.stringify({ + mcpServers: { + rea: { command: "old-rea", args: ["mcp"], enabledTools }, + }, + }), + ); + expect( + (await configureClientConfiguration(client, {}, command)).status, + ).toBe("configured"); + expect( + JSON.parse(await readFile(client.configPath, "utf8")).mcpServers.rea, + ).toEqual({ + ...expectedRegistration(command), + enabledTools, + }); + expect( + (await configureClientConfiguration(client, {}, command)).status, + ).toBe("unchanged"); + }); + + it("registers without type, preserves unrelated settings, backs up, and is idempotent", async () => { + const { home, client } = await kimiClient(); + const original = `{ + "customSetting": { "enabled": true }, + "mcpServers": { + "other": { "command": "other-server", "startupTimeoutMs": 5000 } + } +} +`; + await writeFile(client.configPath, original); + + expect(await configureClientConfiguration(client, {}, command)).toEqual({ + status: "configured", + backupPath: `${client.configPath}.rea.backup`, + }); + const text = await readFile(client.configPath, "utf8"); + expect(JSON.parse(text)).toEqual({ + customSetting: { enabled: true }, + mcpServers: { + other: { command: "other-server", startupTimeoutMs: 5000 }, + rea: expectedRegistration(command), + }, + }); + expect(text).not.toContain('"type"'); + expect(text).not.toContain('"transport"'); + expect(await readFile(`${client.configPath}.rea.backup`, "utf8")).toBe( + original, + ); + expect(await configureClientConfiguration(client, {}, command)).toEqual({ + status: "unchanged", + }); + expect(await readFile(client.configPath, "utf8")).toBe(text); + expect( + ( + await readClientRegistrationStatuses(home, undefined, { + environment: {}, + }) + ).find(({ client: name }) => name === "kimi"), + ).toMatchObject({ state: "aligned", command }); + + expect((await systemUninstallHost(home).removeClient(client)).status).toBe( + "removed", + ); + expect(JSON.parse(await readFile(client.configPath, "utf8"))).toEqual({ + customSetting: { enabled: true }, + mcpServers: { + other: { command: "other-server", startupTimeoutMs: 5000 }, + }, + }); + }); +}); + +describe("Kimi Code package launchers", () => { + const kimiClient = async () => { + const home = await createTestTempDirectory("rea-kimi-launcher-"); + const client = kimiPath(home, process.platform, {}); + if (client?.markerPath === undefined) + throw new Error("missing Kimi Code client"); + await mkdir(client.markerPath, { recursive: true }); + return { home, client } as const; + }; + + it.each([ + ["npx", "linux"], + ["bunx", "linux"], + ["npx", "win32"], + ["bunx", "win32"], + ] as const)( + "keeps the %s launcher aligned on %s", + async (runner, platform) => { + const { home, client } = await kimiClient(); + const launcher = setupRegistrationCommand(platform, runner); + expect( + (await configureClientConfiguration(client, {}, launcher)).status, + ).toBe("configured"); + expect( + JSON.parse(await readFile(client.configPath, "utf8")).mcpServers.rea, + ).toEqual(expectedRegistration(launcher)); + expect( + ( + await readClientRegistrationStatuses(home, undefined, { + platform, + environment: {}, + }) + ).find(({ client: name }) => name === "kimi"), + ).toMatchObject({ state: "aligned", command: launcher }); + expect(await configureClientConfiguration(client, {}, launcher)).toEqual({ + status: "unchanged", + }); + }, + ); +}); + +describe("Kimi Code transport validation", () => { + const kimiClient = async () => { + const home = await createTestTempDirectory("rea-kimi-validation-"); + const client = kimiPath(home, process.platform, {}); + if (client?.markerPath === undefined) + throw new Error("missing Kimi Code client"); + await mkdir(client.markerPath, { recursive: true }); + return { home, client }; + }; + + it("creates a registration in a zero-byte file and accepts whitespace-only files", async () => { + const { home, client } = await kimiClient(); + for (const original of ["", " \n\t"]) { + await writeFile(client.configPath, original); + expect( + (await configureClientConfiguration(client, {}, command)).status, + ).toBe("configured"); + expect( + JSON.parse(await readFile(client.configPath, "utf8")).mcpServers.rea, + ).toEqual(expectedRegistration(command)); + expect(await configureClientConfiguration(client, {}, command)).toEqual({ + status: "unchanged", + }); + expect( + (await systemUninstallHost(home).removeClient(client)).status, + ).toBe("removed"); + } + }); + + it.each([ + '{"mcpServers": {', + '{/* comment */ "mcpServers": {}}', + '{"mcpServers": {},}', + '\uFEFF{"mcpServers": {}}', + ])("rejects invalid strict JSON %j without writes", async (original) => { + const { client } = await kimiClient(); + await writeFile(client.configPath, original); + expect(await configureClientConfiguration(client, {}, command)).toEqual({ + status: "failed", + reason: "readback", + }); + expect(await readFile(client.configPath, "utf8")).toBe(original); + expect(await exists(`${client.configPath}.rea.backup`)).toBe(false); + }); + + it("accepts explicit stdio transport without causing repeated rewrites", async () => { + const { home, client } = await kimiClient(); + await writeFile( + client.configPath, + JSON.stringify({ + mcpServers: { + rea: { + transport: "stdio", + ...expectedRegistration(command), + }, + }, + }), + ); + expect( + ( + await readClientRegistrationStatuses(home, undefined, { + environment: {}, + }) + ).find(({ client: name }) => name === "kimi"), + ).toMatchObject({ state: "aligned", command }); + expect( + (await configureClientConfiguration(client, {}, command)).status, + ).toBe("configured"); + expect(await configureClientConfiguration(client, {}, command)).toEqual({ + status: "unchanged", + }); + expect(await configureClientConfiguration(client, {}, command)).toEqual({ + status: "unchanged", + }); + expect( + ( + await readClientRegistrationStatuses(home, undefined, { + environment: {}, + }) + ).find(({ client: name }) => name === "kimi"), + ).toMatchObject({ state: "aligned", command }); + }); + + it.each(["sse"])( + "rejects explicit transport %j instead of silently accepting another protocol", + async (transport) => { + const { home, client } = await kimiClient(); + await writeFile( + client.configPath, + JSON.stringify({ + mcpServers: { + rea: { + transport, + ...expectedRegistration(command), + }, + }, + }), + ); + expect( + ( + await readClientRegistrationStatuses(home, undefined, { + environment: {}, + }) + ).find(({ client: name }) => name === "kimi"), + ).toMatchObject({ state: "invalid", command: [] }); + }, + ); +}); + +describe("Kimi Code timeout validation", () => { + const kimiClient = async () => { + const home = await createTestTempDirectory("rea-kimi-timeout-"); + const client = kimiPath(home, process.platform, {}); + if (client?.markerPath === undefined) + throw new Error("missing Kimi Code client"); + await mkdir(client.markerPath, { recursive: true }); + return { home, client } as const; + }; + + it.each([ + ["startupTimeoutMs", 1], + ["toolTimeoutMs", 1], + ["startupTimeoutMs", undefined], + ["toolTimeoutMs", undefined], + ] as const)("reports stale when %s is %j", async (field, value) => { + const { home, client } = await kimiClient(); + const registration = { + ...expectedRegistration(command), + [field]: value, + }; + await writeFile( + client.configPath, + JSON.stringify({ mcpServers: { rea: registration } }), + ); + expect( + ( + await readClientRegistrationStatuses(home, undefined, { + environment: {}, + }) + ).find(({ client: name }) => name === "kimi"), + ).toMatchObject({ state: "stale" }); + }); + + it("uses a direct installed REA command without requiring a package runner", async () => { + const { home, client } = await kimiClient(); + const entry = join(home, "scripts", "rea.mjs"); + const launcher = [process.execPath, entry, "mcp"]; + expect( + (await configureClientConfiguration(client, {}, launcher)).status, + ).toBe("configured"); + expect( + ( + await readClientRegistrationStatuses(home, entry, { + environment: {}, + }) + ).find(({ client: name }) => name === "kimi"), + ).toMatchObject({ state: "aligned", command: launcher }); + }); + + it("leaves a foreign rea-key registration out of aligned state", async () => { + const { home, client } = await kimiClient(); + await writeFile( + client.configPath, + JSON.stringify({ + mcpServers: { + rea: { + ...expectedRegistration(command), + command: "another-package", + args: ["mcp"], + }, + }, + }), + ); + expect( + ( + await readClientRegistrationStatuses(home, undefined, { + environment: {}, + }) + ).find(({ client: name }) => name === "kimi"), + ).toMatchObject({ state: "stale" }); + }); +}); diff --git a/tests/boundary/filesystem/productCatalog.test.ts b/tests/boundary/filesystem/productCatalog.test.ts index 1c5f3ebcd..2d79506f6 100644 --- a/tests/boundary/filesystem/productCatalog.test.ts +++ b/tests/boundary/filesystem/productCatalog.test.ts @@ -37,6 +37,53 @@ import { const root = fileURLToPath(new URL("../../..", import.meta.url)); const temporaryRoots: string[] = []; +const README_FACT_FIXTURE_PATHS = [ + "README.md", + "docs/readme/README_zh.md", + "docs/readme/README_ja.md", + "docs/readme/README_ko.md", + "docs/readme/README_ar.md", + "website/locales/languages.json", + "docs/installation.md", + "AGENTS.md", + ".github/pull_request_template.md", +] as const; + +const README_LINK_DRIFT_MUTATIONS = [ + [ + "(../installation.md#supported-agents)", + "(../installation.md#missing-agents)", + ], + [ + "https://rea.tools/zh-CN/guides/native/", + "https://rea.tools/zh-CN/missing/", + ], + [ + "https://rea.tools/zh-CN/guides/native/", + "https://rea.tools/zh-TW/guides/native/", + ], + [ + "https://rea.tools/zh-CN/guides/native/", + "https://rea.tools/zh-CN/guides/native/?other=1", + ], + [ + 'src="../assets/rea-hopper-analysis.png"', + 'src="../assets/missing-hopper.png"', + ], + ['href="https://discord.gg/GkcryMnJDM"', 'href="https://discord.gg/missing"'], +] as const; + +const seedReadmeFactFixture = async (directory: string) => { + for (const path of README_FACT_FIXTURE_PATHS) { + const destination = join(directory, path); + await mkdir(dirname(destination), { recursive: true }); + await copyFile(join(root, path), destination); + } + await mkdir(join(directory, "skills/reverse-engineer-anything"), { + recursive: true, + }); +}; + afterEach(async () => { await Promise.all( temporaryRoots @@ -176,49 +223,14 @@ describe("canonical product catalog drift", () => { it("keeps translated readmes linked to canonical setup and tool details", async () => { const directory = await createTestTempDirectory("rea-readme-facts-"); temporaryRoots.push(directory); - const paths = [ - "README.md", - "README_zh.md", - "README_ja.md", - "README_ko.md", - "README_ar.md", - "website/locales/languages.json", - "docs/installation.md", - "AGENTS.md", - ".github/pull_request_template.md", - ]; - for (const path of paths) { - const destination = join(directory, path); - await mkdir(dirname(destination), { recursive: true }); - await copyFile(join(root, path), destination); - } - await mkdir(join(directory, "skills/reverse-engineer-anything"), { - recursive: true, - }); - const translatedPath = join(directory, "README_zh.md"); + await seedReadmeFactFixture(directory); + const translatedPath = join(directory, "docs/readme/README_zh.md"); const translated = await readFile(translatedPath, "utf8"); const catalog = await createProductCatalog(root); await expect(documentationFactIssues(directory, catalog)).resolves.toEqual( [], ); - for (const [original, changed] of [ - [ - "(docs/installation.md#supported-agents)", - "(docs/installation.md#missing-agents)", - ], - [ - "https://rea.tools/zh-CN/guides/native/", - "https://rea.tools/zh-CN/missing/", - ], - [ - "https://rea.tools/zh-CN/guides/native/", - "https://rea.tools/zh-TW/guides/native/", - ], - [ - "https://rea.tools/zh-CN/guides/native/", - "https://rea.tools/zh-CN/guides/native/?other=1", - ], - ] as const) { + for (const [original, changed] of README_LINK_DRIFT_MUTATIONS) { expect(translated).toContain(original); await writeFile( translatedPath, @@ -228,11 +240,10 @@ describe("canonical product catalog drift", () => { await expect( documentationFactIssues(directory, catalog), ).resolves.toEqual([ - "README_zh.md: documentation links differ from README.md", + "docs/readme/README_zh.md: documentation links differ from README.md", ]); } }); - it("fails check mode without rewriting a stale generated artifact", async () => { const directory = await createTestTempDirectory("rea-generated-check-"); temporaryRoots.push(directory); diff --git a/tests/boundary/filesystem/setupClientConfigCleanupOwnership.test.ts b/tests/boundary/filesystem/setupClientConfigCleanupOwnership.test.ts index ab2b5e5a7..bf7de8819 100644 --- a/tests/boundary/filesystem/setupClientConfigCleanupOwnership.test.ts +++ b/tests/boundary/filesystem/setupClientConfigCleanupOwnership.test.ts @@ -54,6 +54,42 @@ const holdReadHandle = (path: string, reason: string) => { }; }; +/** Linux setup host with its VS Code client configuration directory prepared. */ +const preparedVsCodeSetupHost = async (prefix: string) => { + const home = await createTestTempDirectory(prefix); + const host = systemSetupHost( + { platform: "linux", homeDirectory: home } as DoctorHost, + { HOME: home, XDG_CONFIG_HOME: join(home, "xdg") }, + ); + const client = (await host.supportedClients()).find( + ({ name }) => name === "vscode", + ); + if (client === undefined) throw new Error("VS Code setup client missing"); + await mkdir(dirname(client.configPath), { recursive: true }); + return { home, host, client }; +}; + +/** Release the held read handle and show host cleanup reuses it as a third attempt. */ +const expectCloseReclaimsHandle = async ( + host: { close?: () => Promise }, + closeAttempts: readonly object[], + release: () => void, +) => { + release(); + expect(await host.close?.()).toBeUndefined(); + expect(closeAttempts).toHaveLength(3); + expect(closeAttempts[2]).toBe(closeAttempts[0]); +}; + +/** Give up on the assertion path: release the handle and let cleanup close it. */ +const cleanupHeldHandle = async ( + host: { close?: () => Promise }, + release: () => void, +) => { + release(); + await host.close?.(); +}; + it("retains invalid UTF-8 diagnostics and the uninstall read owner when close fails", async () => { const home = await createTestTempDirectory("rea-uninstall-utf8-close-"); const path = join(home, "client.json"); @@ -77,13 +113,9 @@ it("retains invalid UTF-8 diagnostics and the uninstall read owner when close fa }); expect(await readFile(path)).toEqual(original); expect(closeAttempts).toHaveLength(2); - release(); - expect(await host.close?.()).toBeUndefined(); - expect(closeAttempts).toHaveLength(3); - expect(closeAttempts[2]).toBe(closeAttempts[0]); + await expectCloseReclaimsHandle(host, closeAttempts, release); } finally { - release(); - await host.close?.(); + await cleanupHeldHandle(host, release); } }); @@ -126,13 +158,9 @@ describe("ZCode fallback cleanup ownership", () => { await expect(readFile(client.configPath)).rejects.toMatchObject({ code: "ENOENT", }); - release(); - expect(await host.close?.()).toBeUndefined(); - expect(closeAttempts).toHaveLength(3); - expect(closeAttempts[2]).toBe(closeAttempts[0]); + await expectCloseReclaimsHandle(host, closeAttempts, release); } finally { - release(); - await host.close?.(); + await cleanupHeldHandle(host, release); } }, ); @@ -145,20 +173,9 @@ describe("setup client configuration cleanup ownership", () => { ] as const)( "stops configuration and retains the %s read owner until host close", async (encoding, original) => { - const home = await createTestTempDirectory("rea-setup-config-close-"); - const environment = { - HOME: home, - XDG_CONFIG_HOME: join(home, "xdg"), - }; - const host = systemSetupHost( - { platform: "linux", homeDirectory: home } as DoctorHost, - environment, + const { host, client } = await preparedVsCodeSetupHost( + "rea-setup-config-close-", ); - const client = (await host.supportedClients()).find( - ({ name }) => name === "vscode", - ); - if (client === undefined) throw new Error("VS Code setup client missing"); - await mkdir(dirname(client.configPath), { recursive: true }); await writeFile(client.configPath, original); const { closeAttempts, release } = holdReadHandle( @@ -210,32 +227,17 @@ describe("setup client configuration cleanup ownership", () => { expect(closeAttempts[0]).toBe(closeAttempts[1]); expect(await readFile(client.configPath)).toEqual(original); - release(); - expect(await host.close?.()).toBeUndefined(); - expect(closeAttempts).toHaveLength(3); - expect(closeAttempts[2]).toBe(closeAttempts[0]); + await expectCloseReclaimsHandle(host, closeAttempts, release); } finally { - release(); - await host.close?.(); + await cleanupHeldHandle(host, release); } }, ); it("waits for an admitted configuration read and rejects later admissions", async () => { - const home = await createTestTempDirectory("rea-setup-config-inflight-"); - const environment = { - HOME: home, - XDG_CONFIG_HOME: join(home, "xdg"), - }; - const host = systemSetupHost( - { platform: "linux", homeDirectory: home } as DoctorHost, - environment, + const { host, client } = await preparedVsCodeSetupHost( + "rea-setup-config-inflight-", ); - const client = (await host.supportedClients()).find( - ({ name }) => name === "vscode", - ); - if (client === undefined) throw new Error("VS Code setup client missing"); - await mkdir(dirname(client.configPath), { recursive: true }); await writeFile(client.configPath, '{"servers": {}}\n'); let beginRead!: () => void; diff --git a/tests/boundary/filesystem/setupSkillCleanupOwnership.test.ts b/tests/boundary/filesystem/setupSkillCleanupOwnership.test.ts index dfb472972..4ca9ea407 100644 --- a/tests/boundary/filesystem/setupSkillCleanupOwnership.test.ts +++ b/tests/boundary/filesystem/setupSkillCleanupOwnership.test.ts @@ -70,7 +70,7 @@ const failFirstCloses = (path: string) => { describe("skill regular-file cleanup ownership", () => { it("retains a failed setup skill read until setup host disposal", async () => { const home = await createTestTempDirectory("rea-setup-skill-close-"); - const [destination] = skillDestinations(home, []); + const [destination] = skillDestinations(home, [], {}, "linux"); if (destination === undefined) throw new Error("Shared skill root missing"); const manifest = join(destination.path, "SKILL.md"); await mkdir(dirname(manifest), { recursive: true }); diff --git a/tests/boundary/filesystem/targetDiagnostics.test.ts b/tests/boundary/filesystem/targetDiagnostics.test.ts index dacd21687..998657082 100644 --- a/tests/boundary/filesystem/targetDiagnostics.test.ts +++ b/tests/boundary/filesystem/targetDiagnostics.test.ts @@ -1,4 +1,4 @@ -import { chmod, realpath, writeFile } from "node:fs/promises"; +import { chmod, readFile, realpath, writeFile } from "node:fs/promises"; import { join } from "node:path"; import { describe, expect, it } from "vitest"; import { systemDoctorHost } from "../../../src/application/Doctor.js"; @@ -78,7 +78,7 @@ describe("executable format refusal diagnostics", () => { }); describe("target route diagnostics", () => { - it.each(["module.wasm", "module.bin", "misleading.js"])( + it.each(["module.wasm", "module.bin", "misleading.js", "misleading.hop"])( "identifies a WASM v1 header in %s without suggesting native provider installation", async (name) => { const directory = await createTestTempDirectory("rea-wasm-target-"); @@ -346,3 +346,68 @@ describe("raw-binary load rejection diagnostics", () => { }); }); }); + +describe("caller-selected and inferred target interpretation", () => { + it.each(["archive.hop", "archive.HOP"])( + "preserves inferred database routing for ZIP content named %s", + async (name) => { + const directory = await createTestTempDirectory("rea-zip-database-"); + const bytes = Buffer.alloc(22); + bytes.writeUInt32LE(0x06054b50, 0); + const zipPath = join(directory, "archive.zip"); + const databasePath = join(directory, name); + await writeFile(zipPath, bytes); + await writeFile(databasePath, bytes); + expect(await parseBinaryTarget(zipPath)).toMatchObject({ + ok: true, + value: { kind: "archive", format: "zip" }, + }); + expect(await parseBinaryTarget(databasePath)).toMatchObject({ + ok: true, + value: { kind: "database", format: "analysis-database" }, + }); + expect(await readFile(databasePath)).toEqual(bytes); + }, + ); + + it("preserves explicit DOS COM interpretation for bytes with a WASM header", async () => { + const directory = await createTestTempDirectory("rea-wasm-explicit-com-"); + const path = join(directory, "module.hop"); + await writeFile(path, Buffer.from([0, 0x61, 0x73, 0x6d, 1, 0, 0, 0])); + expect( + await parseBinaryTarget(path, { formatHint: "dos-com" }), + ).toMatchObject({ + ok: true, + value: { kind: "executable", format: "dos-com", architecture: "x86" }, + }); + }); + it.each(["module.wasm", "misleading.hop"])( + "preserves explicitly selected database routing for %s", + async (name) => { + const directory = await createTestTempDirectory( + "rea-wasm-explicit-database-", + ); + const path = join(directory, name); + await writeFile(path, Buffer.from([0, 0x61, 0x73, 0x6d, 1, 0, 0, 0])); + expect( + await parseBinaryTarget(path, { targetKind: "database" }), + ).toMatchObject({ + ok: true, + value: { kind: "database", format: "analysis-database" }, + }); + }, + ); + + it.each(["ordinary.hop", "ordinary.HOP"])( + "retains automatic database routing for non-WASM %s", + async (name) => { + const directory = await createTestTempDirectory("rea-ordinary-database-"); + const path = join(directory, name); + await writeFile(path, "existing database content"); + expect(await parseBinaryTarget(path)).toMatchObject({ + ok: true, + value: { kind: "database", format: "analysis-database" }, + }); + }, + ); +}); diff --git a/tests/boundary/idaStartupDiagnostics.test.ts b/tests/boundary/idaStartupDiagnostics.test.ts new file mode 100644 index 000000000..8f36543b0 --- /dev/null +++ b/tests/boundary/idaStartupDiagnostics.test.ts @@ -0,0 +1,111 @@ +import { describe, expect, it } from "vitest"; +import { createIdaMcpConnection } from "../../src/ida/IdaMcpConnection.js"; + +describe("IDA startup diagnostics", () => { + it("preserves the failing launcher's stderr instead of reporting only a closed transport", async () => { + const connection = createIdaMcpConnection( + { + command: process.execPath, + args: [ + "-e", + 'process.stderr.write("FATAL: IDA runtime initialization failed\\n", () => process.exit(2));', + ], + env: {}, + mode: "headless", + timeoutMs: 1000, + }, + {}, + ); + try { + await expect(connection.connect()).rejects.toMatchObject({ + _tag: "ProviderAdapterError", + capturedOutput: { + stderr: expect.stringContaining("IDA runtime initialization failed"), + truncated: false, + }, + }); + } finally { + await connection.close(); + } + }); + + it("retains only the bounded stderr tail and reports the original byte count", async () => { + const connection = createIdaMcpConnection( + { + command: process.execPath, + args: [ + "-e", + 'process.stderr.write("x".repeat(20000) + "FATAL-END", () => process.exit(2));', + ], + env: {}, + mode: "headless", + timeoutMs: 1000, + }, + {}, + ); + try { + await expect(connection.connect()).rejects.toMatchObject({ + _tag: "ProviderAdapterError", + capturedOutput: { + stderr: "x".repeat(16384 - "FATAL-END".length) + "FATAL-END", + stderr_bytes: 20000 + "FATAL-END".length, + truncated: true, + }, + }); + } finally { + await connection.close(); + } + }); +}); + +it("drops a leading partial UTF-8 character from the bounded stderr tail", async () => { + const connection = createIdaMcpConnection( + { + command: process.execPath, + args: [ + "-e", + 'process.stderr.write("€".repeat(6000), () => process.exit(2));', + ], + env: {}, + mode: "headless", + timeoutMs: 1000, + }, + {}, + ); + try { + await expect(connection.connect()).rejects.toMatchObject({ + capturedOutput: { + stderr: "€".repeat(Math.floor(16384 / 3)), + stderr_bytes: 18000, + truncated: true, + }, + }); + } finally { + await connection.close(); + } +}); + +it("retains an in-budget undecodable leading byte without claiming tail truncation", async () => { + const connection = createIdaMcpConnection( + { + command: process.execPath, + args: [ + "-e", + "process.stderr.write(Buffer.from([0x80, 0x61, 0x62]), () => process.exit(2));", + ], + env: {}, + mode: "headless", + timeoutMs: 1000, + }, + {}, + ); + try { + const failure = await connection.connect().catch((cause: unknown) => cause); + expect(failure).toMatchObject({ + capturedOutput: { stderr: "�ab", stderr_bytes: 3, truncated: false }, + }); + expect(failure).not.toHaveProperty("diagnostics.upstream_stderr"); + } finally { + await connection.close(); + } +}); diff --git a/tests/boundary/javascript/javascriptSourceAnalysisWorker.test.ts b/tests/boundary/javascript/javascriptSourceAnalysisWorker.test.ts index 0e41a11bb..04ba5201f 100644 --- a/tests/boundary/javascript/javascriptSourceAnalysisWorker.test.ts +++ b/tests/boundary/javascript/javascriptSourceAnalysisWorker.test.ts @@ -1,7 +1,13 @@ import { createHash } from "node:crypto"; +import { execFile } from "node:child_process"; +import { readFile } from "node:fs/promises"; +import { join } from "node:path"; +import { promisify } from "node:util"; import { expect, it } from "vitest"; import { JavaScriptAnalysisWorker } from "../../../src/javascript/analysis/JavaScriptAnalysisWorker.js"; import { cleanupOwnedProcessGroup } from "../../../src/process/ProcessOwnership.js"; +import type { OwnedProcessGroup } from "../../../src/process/ProcessOwnership.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; import { ProviderCleanupError } from "../../../src/domain/providerCleanupError.js"; import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js"; import { spawnOwnedProviderProcess } from "../../../src/process/ProviderProcess.js"; @@ -151,6 +157,121 @@ it("preserves a failed owned cleanup retry when the next caller cancels before a } }); +it("ends worker stdin and unreferences process handles when cleanup reports inspection uncertainty", async () => { + let stdinEnded = false; + let unreferenced = false; + let permitCleanup = false; + let childStdin: typeof process.stdin | undefined; + const worker = new JavaScriptAnalysisWorker( + { heapMb: 128, timeoutMs: 30_000 }, + async (request) => { + const launched = await spawnOwnedProviderProcess(request); + const originalUnref = launched.process.unref; + launched.process.unref = () => { + unreferenced = true; + originalUnref?.call(launched.process); + }; + childStdin = launched.process.stdin as unknown as typeof process.stdin; + launched.process.stdin?.once("finish", () => { + stdinEnded = true; + }); + return { + ...launched, + cleanup: async () => + permitCleanup + ? launched.cleanup === undefined + ? cleanupOwnedProcessGroup(launched.ownership) + : launched.cleanup() + : { + cleaned: false, + reason: "process table could not be inspected: spawn EPERM", + }, + }; + }, + ); + try { + const file = source("app.mjs", "export const observed = 7;"); + const result = await worker.analyze(file, 100); + if (!result.ok) throw result.error.error; + const closing = worker.close(); + await expect(closing).rejects.toBeInstanceOf(ProviderCleanupError); + await expect(closing).rejects.toMatchObject({ + cause: { + message: "process table could not be inspected: spawn EPERM", + }, + }); + expect(childStdin?.writableEnded || stdinEnded).toBe(true); + expect(unreferenced).toBe(true); + const errorListeners = childStdin?.listenerCount("error"); + for (let retry = 0; retry < 3; retry++) + await expect(worker.close()).rejects.toBeInstanceOf(ProviderCleanupError); + expect(childStdin?.listenerCount("error")).toBeLessThanOrEqual( + errorListeners ?? 0, + ); + permitCleanup = true; + await worker.close(); + } finally { + permitCleanup = true; + await worker.close().catch(() => {}); + } +}); + +it("lets a separate host exit while its live worker cleanup remains incomplete", async () => { + const root = await createTestTempDirectory("rea-worker-incomplete-host-"); + const ownershipPath = join(root, "ownership.json"); + const workerUrl = new URL( + "../../../dist/javascript/analysis/JavaScriptAnalysisWorker.js", + import.meta.url, + ).href; + const launcherUrl = new URL( + "../../../dist/process/ProviderProcess.js", + import.meta.url, + ).href; + const script = ` +import { writeFile } from "node:fs/promises"; +import { pathToFileURL } from "node:url"; +import { JavaScriptAnalysisWorker } from ${JSON.stringify(workerUrl)}; +import { spawnOwnedProviderProcess } from ${JSON.stringify(launcherUrl)}; +const worker = new JavaScriptAnalysisWorker({ heapMb: 128, timeoutMs: 10000 }, async request => { + const entry = pathToFileURL(request.arguments.at(-1)).href; + const launched = await spawnOwnedProviderProcess({ ...request, arguments: [ + ...request.arguments.slice(0, -1), "--input-type=module", "-e", + "const interval = setInterval(() => {}, 1000); setTimeout(() => clearInterval(interval), 60000); await import(" + JSON.stringify(entry) + ");" + ] }); + // The launcher is deliberately orphaned when this host exits; cleanup still + // requires its run token and executable identity, rather than its old parent. + const { expectedParentPid, ...ownership } = launched.ownership; + await writeFile(${JSON.stringify(ownershipPath)}, JSON.stringify(ownership)); + return { ...launched, cleanup: async () => ({ cleaned: false, reason: "fixture inspection unavailable" }) }; +}); +const analyzed = await worker.analyze(${JSON.stringify(source("app.mjs", "export const value = 1;"))}, 100); +if (!analyzed.ok) throw analyzed.error.error; +try { await worker.close(); throw new Error("cleanup unexpectedly completed"); } +catch (error) { + if (error.cleanupIncomplete !== true) throw error; + process.stdout.write(JSON.stringify({ code: "cleanup_incomplete", reason: error.cause.message }) + "\\n"); +} +`; + try { + const result = await promisify(execFile)( + process.execPath, + ["--input-type=module", "-e", script], + { timeout: 10_000 }, + ); + expect(JSON.parse(result.stdout)).toMatchObject({ + code: "cleanup_incomplete", + reason: "fixture inspection unavailable", + }); + } finally { + const ownership = JSON.parse( + await readFile(ownershipPath, "utf8"), + ) as OwnedProcessGroup; + expect(await cleanupOwnedProcessGroup(ownership)).toMatchObject({ + cleaned: true, + }); + } +}); + // The handshake mirrors the real server: only Node.js reports an enforced V8 heap limit. const resourceFailureWorker = (heapLimit: string) => new JavaScriptAnalysisWorker({ heapMb: 128, timeoutMs: 30_000 }, (request) => diff --git a/tests/boundary/mcp/analysisViewMcp.test.ts b/tests/boundary/mcp/analysisViewMcp.test.ts index b95aebe57..8fa3f0fb7 100644 --- a/tests/boundary/mcp/analysisViewMcp.test.ts +++ b/tests/boundary/mcp/analysisViewMcp.test.ts @@ -1,3 +1,4 @@ +import { advertisedMcpToolSchemas } from "../../fixtures/advertisedMcpTool.js"; import { API_SURFACE_EXAMPLE } from "../../../src/contracts/analysisViewApiSurfaceExample.js"; import { createHash } from "node:crypto"; import { readFile, writeFile } from "node:fs/promises"; @@ -212,16 +213,10 @@ it("advertises exact schemas and projects one layout section from retained Evide inspect: () => Promise.resolve(ok(layout)), }); const { client, session } = await connect(service); - const advertised = (await client.listTools()).tools.find( - (tool) => tool.name === "inspect_analysis_view", + const { ajv, inputSchema, outputSchema } = await advertisedMcpToolSchemas( + client, + "inspect_analysis_view", ); - if (advertised?.outputSchema === undefined) - throw new Error("inspect_analysis_view must publish both schemas"); - const ajv = new Ajv2020({ strict: false, validateFormats: false }); - const inputSchema: Record = advertised.inputSchema; - const outputSchema: Record = advertised.outputSchema; - expect(ajv.validateSchema(inputSchema)).toBe(true); - expect(ajv.validateSchema(outputSchema)).toBe(true); expect( ajv.validate(inputSchema, { source: { kind: "inline", evidence: analysisViewLayoutEvidence() }, diff --git a/tests/boundary/mcp/asarListingAdmissionMcp.test.ts b/tests/boundary/mcp/asarListingAdmissionMcp.test.ts new file mode 100644 index 000000000..c8db05e92 --- /dev/null +++ b/tests/boundary/mcp/asarListingAdmissionMcp.test.ts @@ -0,0 +1,60 @@ +import { readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import { expect, it } from "vitest"; + +import { ArtifactProvider } from "../../../src/artifacts/ArtifactProvider.js"; +import { createServer } from "../../../src/server/createServer.js"; +import { nestedAsarHeader } from "../../fixtures/asarHeader.js"; +import { createTestBinarySession } from "../../fixtures/binarySession.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; + +it("reports expanded ASAR admission failure through MCP and permits a valid retry", async () => { + const root = await createTestTempDirectory("rea-asar-listing-mcp-"); + const path = join(root, "app.asar"); + const bytes = nestedAsarHeader(64, "d".repeat(256), 4000); + await writeFile(path, bytes); + const session = createTestBinarySession(new ArtifactProvider(process.env)); + const server = createServer({ kind: "session", session }); + const client = new Client({ name: "asar-listing-test", version: "1" }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + try { + await server.connect(serverTransport); + await client.connect(clientTransport); + expect( + (await client.callTool({ name: "open_binary", arguments: { path } })) + .isError, + ).not.toBe(true); + const result = await client.callTool({ + name: "inspect_artifact", + arguments: {}, + }); + expect(result.isError).toBe(true); + expect(parseMcpToolError(result)).toMatchObject({ + error: { + code: "truncated", + category: "truncated", + }, + }); + expect(result.structuredContent).toBeUndefined(); + expect(await readFile(path)).toEqual(bytes); + const valid = join(root, "valid.asar"); + await writeFile(valid, nestedAsarHeader(1, "lib", 1)); + expect( + ( + await client.callTool({ + name: "open_binary", + arguments: { path: valid }, + }) + ).isError, + ).not.toBe(true); + expect( + (await client.callTool({ name: "inspect_artifact", arguments: {} })) + .isError, + ).not.toBe(true); + } finally { + await Promise.allSettled([client.close(), server.close(), session.close()]); + } +}); diff --git a/tests/boundary/mcp/binaryDiagnosticsMcp.test.ts b/tests/boundary/mcp/binaryDiagnosticsMcp.test.ts index bea7c89dd..51bda12c7 100644 --- a/tests/boundary/mcp/binaryDiagnosticsMcp.test.ts +++ b/tests/boundary/mcp/binaryDiagnosticsMcp.test.ts @@ -1,9 +1,11 @@ +import { + advertisedMcpToolSchemas, + assertRecordedMcpEvidence, +} from "../../fixtures/advertisedMcpTool.js"; import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; -import { Ajv2020 } from "ajv/dist/2020.js"; import { expect, it, onTestFinished } from "vitest"; import { BinaryLayoutService } from "../../../src/application/binaryDiagnostics/BinaryLayoutService.js"; import { toolContract } from "../../../src/contracts/toolContracts.js"; -import { parseEvidence } from "../../../src/domain/evidence.js"; import { ok } from "../../../src/domain/result.js"; import { createServer } from "../../../src/server/createServer.js"; import { createTestBinarySession } from "../../fixtures/binarySession.js"; @@ -43,16 +45,11 @@ it("advertises exact valid schemas and records inline layout without a binary ta InMemoryTransport.createLinkedPair(); await server.connect(serverTransport); await client.connect(clientTransport); - const advertised = (await client.listTools()).tools.find( - (tool) => tool.name === "inspect_binary_layout", + const schemas = await advertisedMcpToolSchemas( + client, + "inspect_binary_layout", ); - if (advertised?.outputSchema === undefined) - throw new Error("layout must publish both schemas"); - const ajv = new Ajv2020({ strict: false, validateFormats: false }); - const inputSchema: Record = advertised.inputSchema; - const outputSchema: Record = advertised.outputSchema; - expect(ajv.validateSchema(inputSchema)).toBe(true); - expect(ajv.validateSchema(outputSchema)).toBe(true); + const { ajv, inputSchema } = schemas; expect(ajv.validate(inputSchema, { path: value.artifact.path })).toBe(true); expect( ajv.validate(inputSchema, { @@ -64,19 +61,14 @@ it("advertises exact valid schemas and records inline layout without a binary ta name: "inspect_binary_layout", arguments: { path: value.artifact.path }, }); - expect(response.isError).not.toBe(true); - expect( - ajv.validate(outputSchema, response.structuredContent), - JSON.stringify(ajv.errors), - ).toBe(true); - const parsed = toolContract("inspect_binary_layout").outputSchema.parse( - response.structuredContent, + const evidence = assertRecordedMcpEvidence( + response, + "inspect_binary_layout", + schemas, + session, + value, + "observed", ); - const evidence = parseEvidence(parsed); - expect(evidence.confidence).toBe("observed"); - expect(evidence.subject?.digest.sha256).toBe(value.artifact.sha256); - expect(parsed.normalized_result).toEqual(value); - expect(session.evidenceById(evidence.evidence_id)).toEqual(evidence); const viewed = await client.callTool({ name: "inspect_analysis_view", arguments: { diff --git a/tests/boundary/mcp/browserMcp.test.ts b/tests/boundary/mcp/browserMcp.test.ts index a8a780d61..b13d17df5 100644 --- a/tests/boundary/mcp/browserMcp.test.ts +++ b/tests/boundary/mcp/browserMcp.test.ts @@ -424,6 +424,47 @@ it("delivers the source-map response URL through authenticated MCP Evidence", as ).toBe(true); }); +it("records a browser-socket endpoint redacted and reconciles its Evidence", async () => { + const browser = await startFakeCdpBrowser(); + browsers.push(browser); + const connected = await connectBrowser(); + const tool = (await connected.client.listTools()).tools.find( + ({ name }) => name === "inspect_web_page", + ); + expect(tool?.inputSchema.properties?.cdp_endpoint).toMatchObject({ + description: expect.stringContaining("/devtools/browser/ID"), + }); + const inspected = await connected.client.callTool({ + name: "inspect_web_page", + arguments: { + cdp_endpoint: browser.browserWebSocketUrl, + target_id: "allowed-page", + observation_ms: 0, + include_script_sources: true, + }, + }); + expect(inspected.isError).not.toBe(true); + expect(inspected.structuredContent).toMatchObject({ + parameters: { + cdp_endpoint: browser.browserWebSocketUrl.replace(/fake$/u, "[REDACTED]"), + }, + }); + expect(JSON.stringify(inspected.structuredContent)).not.toContain( + "/devtools/browser/fake", + ); + const reconciled = await connected.client.callTool({ + name: "reconcile_javascript_runtime", + arguments: { + static_layers: JAVASCRIPT_RUNTIME_RECONCILIATION_EXAMPLE.static_layers, + runtime_observations: [evidenceFor(inspected.structuredContent)], + }, + }); + expect(reconciled.isError).not.toBe(true); + expect(reconciled.structuredContent).toMatchObject({ + normalized_result: { summary: { runtime_scripts: 1 } }, + }); +}); + it("does not attach to a target outside the request's allowed origin scope", async () => { const browser = await startFakeCdpBrowser(); browsers.push(browser); diff --git a/tests/boundary/mcp/comparisonUnknownsMcp.test.ts b/tests/boundary/mcp/comparisonUnknownsMcp.test.ts index bf950f83b..f872d2f15 100644 --- a/tests/boundary/mcp/comparisonUnknownsMcp.test.ts +++ b/tests/boundary/mcp/comparisonUnknownsMcp.test.ts @@ -49,6 +49,9 @@ const observeAgain = ( result, confidence: source.confidence, authority: source.authority, + ...(source.relationship_coverage === null + ? {} + : { relationshipCoverage: source.relationship_coverage }), }, ); diff --git a/tests/boundary/mcp/contractToolHandlerTargetContent.test.ts b/tests/boundary/mcp/contractToolHandlerTargetContent.test.ts new file mode 100644 index 000000000..9e8a9d24b --- /dev/null +++ b/tests/boundary/mcp/contractToolHandlerTargetContent.test.ts @@ -0,0 +1,86 @@ +import { describe, expect, it } from "vitest"; + +import type { + CallToolResult, + ServerContext, +} from "@modelcontextprotocol/server"; +import { STDIO_DEFAULT_MAX_BUFFER_SIZE } from "@modelcontextprotocol/server"; + +import { toolContract } from "../../../src/contracts/toolContracts.js"; +import { AnalysisCancelledError } from "../../../src/domain/analysisErrorCore.js"; +import { createEvidence } from "../../../src/domain/evidence.js"; +import type { Evidence } from "../../../src/domain/evidence.js"; +import { err, ok } from "../../../src/domain/result.js"; +import { silentLogger } from "../../../src/logger.js"; +import { createEvidenceToolHandler } from "../../../src/server/contractToolHandler.js"; +import { EvidenceMcpServer } from "../../../src/server/EvidenceMcpServer.js"; +import { ToolResultDelivery } from "../../../src/server/toolResult.js"; + +const UNTRUSTED_TARGET_CONTENT = { + provenance: { untrusted_content: true, source: "analysis_target" }, +}; + +const context = { + mcpReq: { signal: new AbortController().signal }, +} as ServerContext; + +/** + * Deliver one retained partial-observation failure through the shared Evidence + * handler. The handler is the producer of the result: `EvidenceMcpServer` only + * adds the marker later for results that bypass `ToolResultDelivery`, so a + * missing marker here also undercounts the wire budget by the marker's bytes. + */ +const deliverPartialFailure = async (): Promise<{ + readonly result: CallToolResult; + readonly evidence: Evidence; +}> => { + const evidence = createEvidence( + undefined, + { id: "fixture", name: "Completed capture fixture", version: "1" }, + { + operation: "inspect_packet_capture", + parameters: { path: "recording.pcap" }, + result: { packets: [{ "frame.number": ["1"] }] }, + confidence: "observed", + }, + ); + const failure = new AnalysisCancelledError("inspect_packet_capture", { + partialObservation: evidence, + }); + const writer = { recordEvidence: () => ok("added" as const) }; + const server = new EvidenceMcpServer( + { name: "partial-error-framing-test", version: "1" }, + { capabilities: {} }, + writer.recordEvidence, + new ToolResultDelivery(STDIO_DEFAULT_MAX_BUFFER_SIZE), + ); + const contract = toolContract("inspect_packet_capture"); + const handler = createEvidenceToolHandler( + server, + async () => err(failure), + silentLogger, + writer.recordEvidence, + ); + return { + result: await handler(contract)({ path: "recording.pcap" }, context), + evidence, + }; +}; + +describe("shared Evidence handler target-content framing", () => { + it("marks a retained partial-observation error as untrusted target content", async () => { + const { result, evidence } = await deliverPartialFailure(); + expect(result.isError).toBe(true); + expect(result.structuredContent).toMatchObject({ + error: { + details: { + partial_observation: { + kind: "retained-evidence", + evidence_id: evidence.evidence_id, + }, + }, + }, + }); + expect(result._meta).toEqual(UNTRUSTED_TARGET_CONTENT); + }); +}); diff --git a/tests/boundary/mcp/enhancedTools.01-lists-the-complete-target-open.test.ts b/tests/boundary/mcp/enhancedTools.01-lists-the-complete-target-open.test.ts index 62246a85c..add15a6b2 100644 --- a/tests/boundary/mcp/enhancedTools.01-lists-the-complete-target-open.test.ts +++ b/tests/boundary/mcp/enhancedTools.01-lists-the-complete-target-open.test.ts @@ -104,7 +104,16 @@ const fixturePort = (): AnalysisOperationPort => ({ "0x11": "_OBJC_CLASS_$_Fixture", "0x12": "_OBJC_PROTOCOL_$_FixtureDelegate", "0x13": "entry", - }), + }).map((entry) => ({ + ...entry, + symbol: { + primary: true, + dynamic: false, + external: false, + type: "label", + source: "analysis", + }, + })), ), ); case "decode_interface_builder": diff --git a/tests/boundary/mcp/evmInterfaceMcp.test.ts b/tests/boundary/mcp/evmInterfaceMcp.test.ts index 823a4d3cf..2bfed5c81 100644 --- a/tests/boundary/mcp/evmInterfaceMcp.test.ts +++ b/tests/boundary/mcp/evmInterfaceMcp.test.ts @@ -1,4 +1,9 @@ +import { + advertisedMcpToolSchemas, + assertRecordedMcpEvidence, +} from "../../fixtures/advertisedMcpTool.js"; import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import { createHash } from "node:crypto"; import { Ajv2020 } from "ajv/dist/2020.js"; import { expect, it, onTestFinished, vi } from "vitest"; import { EvmInterfaceService } from "../../../src/application/evm/EvmInterfaceService.js"; @@ -10,6 +15,7 @@ import { } from "../../../src/domain/analysisErrorCore.js"; import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js"; import { evmInterfaceSummaryViewResultSchema } from "../../../src/domain/analysisView/analysisView.js"; +import { inspectSolidityMetadata } from "../../../src/evm/SolidityMetadata.js"; import { err, ok } from "../../../src/domain/result.js"; import { silentLogger } from "../../../src/logger.js"; import { createServer } from "../../../src/server/createServer.js"; @@ -26,8 +32,21 @@ import { evmInterfaceExecution, } from "../../fixtures/evm/interface.js"; -it("advertises exact valid schemas and records inline EVM interface evidence without a binary target", async () => { +const decodedMetadataFixture = () => { + const bytes = Buffer.from("00a1011bffffffffffffffff000b", "hex"); const value = evmInterfaceFixture(); + value.bytecode = { + ...value.bytecode, + hex: bytes.toString("hex"), + bytes: bytes.length, + sha256: createHash("sha256").update(bytes).digest("hex"), + metadata: inspectSolidityMetadata(bytes), + }; + return value; +}; + +it("advertises exact valid schemas and records inline EVM interface evidence without a binary target", async () => { + const value = decodedMetadataFixture(); const inspect = vi.fn(() => Promise.resolve(ok(evmInterfaceExecution(value))), ); @@ -51,16 +70,11 @@ it("advertises exact valid schemas and records inline EVM interface evidence wit InMemoryTransport.createLinkedPair(); await server.connect(serverTransport); await client.connect(clientTransport); - const advertised = (await client.listTools()).tools.find( - (tool) => tool.name === "inspect_evm_interface", + const schemas = await advertisedMcpToolSchemas( + client, + "inspect_evm_interface", ); - if (advertised?.outputSchema === undefined) - throw new Error("EVM interface inspection must publish both schemas"); - const ajv = new Ajv2020({ strict: false, validateFormats: false }); - const inputSchema: Record = advertised.inputSchema; - const outputSchema: Record = advertised.outputSchema; - expect(ajv.validateSchema(inputSchema)).toBe(true); - expect(ajv.validateSchema(outputSchema)).toBe(true); + const { ajv, inputSchema } = schemas; expect( ajv.validate(inputSchema, { path: value.artifact.path, encoding: "hex" }), ).toBe(true); @@ -84,19 +98,14 @@ it("advertises exact valid schemas and records inline EVM interface evidence wit name: "inspect_evm_interface", arguments: { path: value.artifact.path, encoding: "hex" }, }); - expect(response.isError).not.toBe(true); - expect( - ajv.validate(outputSchema, response.structuredContent), - JSON.stringify(ajv.errors), - ).toBe(true); - const parsed = toolContract("inspect_evm_interface").outputSchema.parse( - response.structuredContent, + assertRecordedMcpEvidence( + response, + "inspect_evm_interface", + schemas, + session, + value, + "inferred", ); - const evidence = parseEvidence(parsed); - expect(evidence.confidence).toBe("inferred"); - expect(evidence.subject?.digest.sha256).toBe(value.artifact.sha256); - expect(parsed.normalized_result).toEqual(value); - expect(session.evidenceById(evidence.evidence_id)).toEqual(evidence); expect( ( await client.callTool({ @@ -280,7 +289,8 @@ it("settles the admitted EVM operation after SDK cancellation and accepts a subs }); it("returns the exact inferred interface summary while preserving complete defaults and retained source", async () => { - const value = evmInterfaceFixture(); + const value = decodedMetadataFixture(); + expect(value.bytecode.metadata?.status).toBe("decoded"); value.functions = [ { selector: "0x11223344", @@ -419,6 +429,7 @@ it.each(["valid", "wrong-subject", "no-subject"])( "preserves historical portable EVM Evidence and rejects incompatible identity: %s", async (scenario) => { const value = evmInterfaceFixture(); + const { metadata: _metadata, ...historicalBytecode } = value.bytecode; const source = createEvidence( scenario === "no-subject" ? undefined @@ -434,7 +445,7 @@ it.each(["valid", "wrong-subject", "no-subject"])( { operation: "inspect_evm_interface", parameters: { path: value.artifact.path, encoding: "hex" }, - result: value, + result: { ...value, bytecode: historicalBytecode }, confidence: "inferred", rawResult: { original: true }, }, @@ -463,6 +474,7 @@ it.each(["valid", "wrong-subject", "no-subject"])( expect(view.parent_evidence_id).toBe(source.evidence_id); expect(view.summary.functions).toEqual([]); expect(view.summary.discovery_completeness).toBe("unknown"); + expect(view.summary.bytecode.metadata).toBeNull(); expect(view.coverage).toMatchObject({ total: 0, examined: 0, diff --git a/tests/boundary/mcp/functionComparisonMcp.test.ts b/tests/boundary/mcp/functionComparisonMcp.test.ts index 8ec10f402..97f07dc1e 100644 --- a/tests/boundary/mcp/functionComparisonMcp.test.ts +++ b/tests/boundary/mcp/functionComparisonMcp.test.ts @@ -1,6 +1,14 @@ import { ok as resultOk } from "../../../src/domain/result.js"; import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; -import { describe, expect, it } from "vitest"; +import { describe, expect, it, onTestFinished } from "vitest"; +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; +import { createEvidenceBundle } from "../../../src/domain/evidenceBundle.js"; +import { compareFunctions } from "../../../src/domain/functionComparison.js"; +import { jsonValueSchema } from "../../../src/domain/jsonValue.js"; import { createTestBinarySession } from "../../fixtures/binarySession.js"; import { FUNCTION_COMPARISON_EXAMPLE } from "../../../src/contracts/functionComparisonExample.js"; @@ -271,3 +279,130 @@ describe("function comparison MCP integration", () => { } }); }); + +it.each([ + { status: "changed", expected: "fail" }, + { status: "unknown", expected: "unknown" }, + { status: "unchanged", expected: "pass" }, +] as const)( + "checks the aggregate status of imported $status function comparisons", + async ({ status, expected }) => { + const left = FUNCTION_COMPARISON_EXAMPLE.left; + const right = + status === "changed" + ? FUNCTION_COMPARISON_EXAMPLE.right + : createEvidence( + { + path: "/fixture/function-right", + sha256: "3".repeat(64), + format: "mach-o", + }, + status === "unknown" + ? { ...left.provider, id: "other-provider" } + : left.provider, + { + operation: left.operation, + parameters: left.parameters, + result: left.normalized_result, + confidence: left.confidence, + authority: left.authority, + relationshipCoverage: { status: "complete" }, + }, + ); + const result = compareFunctions(left, right); + expect(result.status).toBe(status); + const provider = { + id: "rea-function-comparison", + name: "REA function comparison", + version: "1", + }; + const observation = { + operation: "compare_functions", + predicateType: "rea.function-comparison", + parameters: { + left_evidence_id: left.evidence_id, + right_evidence_id: right.evidence_id, + }, + confidence: "derived" as const, + authority: "analyst-inference" as const, + evidenceLinks: [left.evidence_id, right.evidence_id], + }; + const consistent = createEvidence(undefined, provider, { + ...observation, + result: jsonValueSchema.parse(result), + }); + const inconsistent = + status === "unchanged" + ? null + : createEvidence(undefined, provider, { + ...observation, + result: jsonValueSchema.parse({ ...result, status: "unchanged" }), + }); + const directory = await createTestTempDirectory("rea-function-status-"); + const path = join(directory, "bundle.json"); + await writeFile( + path, + JSON.stringify( + createEvidenceBundle([ + left, + right, + consistent, + ...(inconsistent === null ? [] : [inconsistent]), + ]), + ), + ); + const session = createTestBinarySession(() => { + throw new Error("Imported comparison must not launch a provider"); + }); + const harness = await createApplicationMcpHarness({ session }); + onTestFinished(harness.close); + const imported = await harness.client.callTool({ + name: "import_evidence_bundle", + arguments: { path }, + }); + expect(imported.isError).not.toBe(true); + const verification = (evidence_id: string) => + harness.client.callTool({ + name: "verify_reconstruction", + arguments: { + specification: { + name: "Function aggregate", + claims: [ + { + kind: "structural-function", + claim_id: "overall", + title: "Function agrees", + comparison_evidence_id: evidence_id, + dimension: "overall", + }, + ], + }, + }, + }); + if (inconsistent !== null) { + const rejected = await verification(inconsistent.evidence_id); + expect(parseMcpToolError(rejected).error).toMatchObject({ + code: "invalid_request", + details: { + issues: [ + { + reason: "invalid_value", + message: expect.stringContaining( + "Function comparison status contradicts its dimensions or function match", + ), + }, + ], + }, + }); + expect(harness.session.evidenceById(inconsistent.evidence_id)).toEqual( + inconsistent, + ); + } + const verified = await verification(consistent.evidence_id); + expect(verified.isError).not.toBe(true); + expect( + parseEvidence(verified.structuredContent).normalized_result, + ).toMatchObject({ status: expected }); + await harness.client.ping(); + }, +); diff --git a/tests/boundary/mcp/ghidraInventoryMcp.test.ts b/tests/boundary/mcp/ghidraInventoryMcp.test.ts index b24dac153..ac42b9388 100644 --- a/tests/boundary/mcp/ghidraInventoryMcp.test.ts +++ b/tests/boundary/mcp/ghidraInventoryMcp.test.ts @@ -1,9 +1,352 @@ import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; import { expect, it } from "vitest"; +import { writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { nativeAotPeFixture } from "../../fixtures/nativeaotPe.js"; import { jsonValueSchema } from "../../../src/domain/jsonValue.js"; -import { ok } from "../../../src/domain/result.js"; +import { err, ok } from "../../../src/domain/result.js"; import { ghidraFunctionIdentity } from "../../../src/domain/ghidraValues.fixture.js"; -import { connectGhidraMcp, sessionEvidence } from "./ghidraMcpHarness.js"; +import { + connectGhidraMcp, + ghidraInstallationHost, + sessionEvidence, +} from "./ghidraMcpHarness.js"; +import { spawn } from "node:child_process"; +import { mkdir } from "node:fs/promises"; +import { fileURLToPath } from "node:url"; +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import { GhidraClient } from "../../../src/ghidra/GhidraClient.js"; +import { GhidraProvider } from "../../../src/ghidra/GhidraProvider.js"; +import { parseConfig } from "../../../src/config/parseConfig.js"; +import { silentLogger } from "../../../src/logger.js"; +import { composeBinarySession } from "../../../src/application/binary/BinarySessionComposition.js"; +import { AnalysisProviderRegistry } from "../../../src/application/binary/AnalysisProviderRegistry.js"; +import { createServer } from "../../../src/server/createServer.js"; +import { waitForExit } from "../../support/process/processFixture.js"; +import { GhidraSessionError } from "../../../src/ghidra/GhidraSessionError.js"; + +it("retains byte-search coverage and rejects malformed pages before creating Evidence", async () => { + const page = { + matches: [ + { + address: "0x401000", + segment: ".text", + file_offset: null, + procedure: null, + }, + ], + searched: { + blocks: [ + { segment: ".text", start: "0x401000", end: "0x40100f", bytes: 16 }, + ], + bytes: 16, + skipped_uninitialized: [ + { segment: ".bss", start: "0x402000", end: "0x40200f", bytes: 16 }, + ], + skipped_not_loaded: [ + { segment: ".symtab", start: "0x0", end: "0x17f", bytes: 384 }, + ], + }, + complete: false, + next_cursor: "continuation", + }; + let output = jsonValueSchema.parse(page); + const harness = await connectGhidraMcp("byte-search-projection", () => + Promise.resolve(ok(output)), + ); + try { + const first = await harness.mcp.callTool({ + name: "search_bytes", + arguments: { pattern: "48 ??", limit: 1 }, + }); + expect(first.isError).not.toBe(true); + expect( + sessionEvidence(harness.session, first.structuredContent) + .normalized_result, + ).toEqual(page); + output = jsonValueSchema.parse({ + ...page, + matches: [], + complete: true, + next_cursor: null, + }); + const last = await harness.mcp.callTool({ + name: "search_bytes", + arguments: { pattern: "48 ??", cursor: "continuation" }, + }); + expect(last.isError).not.toBe(true); + expect( + sessionEvidence(harness.session, last.structuredContent) + .normalized_result, + ).toEqual(output); + output = jsonValueSchema.parse({ ...page, complete: true }); + const malformed = await harness.mcp.callTool({ + name: "search_bytes", + arguments: { pattern: "49 ??" }, + }); + expect(parseMcpToolError(malformed)).toMatchObject({ + error: { code: "unreadable_output" }, + }); + for (const block of [ + { segment: ".text", start: "0x1000", end: "0x1009", bytes: 1 }, + { segment: ".text", start: "0x1009", end: "0x1000", bytes: 1 }, + { segment: ".text", start: "overlay:0x1000", end: "0x1000", bytes: 1 }, + { segment: ".text", start: "invalid", end: "0x1000", bytes: 1 }, + ]) { + output = jsonValueSchema.parse({ + ...page, + searched: { ...page.searched, blocks: [block], bytes: block.bytes }, + }); + const invalidCoverage = await harness.mcp.callTool({ + name: "search_bytes", + arguments: { pattern: "48 ??" }, + }); + expect(parseMcpToolError(invalidCoverage)).toMatchObject({ + error: { code: "unreadable_output" }, + }); + expect(invalidCoverage.structuredContent).toBeUndefined(); + } + const calls = harness.calls.length; + for (const args of [{ pattern: "??" }, { pattern: "48", mask: "ffff" }]) { + const invalid = await harness.mcp.callTool({ + name: "search_bytes", + arguments: args, + }); + expect(invalid.isError).toBe(true); + expect(JSON.stringify(invalid.content)).toContain( + "Input validation error", + ); + } + expect(harness.calls).toHaveLength(calls); + } finally { + await harness.close(); + } +}); + +it("projects cancelled byte searches as typed failures without successful partial Evidence", async () => { + const harness = await connectGhidraMcp("byte-search-cancelled", () => + Promise.resolve( + err(new GhidraSessionError("cancelled", "Search cancelled")), + ), + ); + try { + const reply = await harness.mcp.callTool({ + name: "search_bytes", + arguments: { pattern: "48 8b" }, + }); + expect(reply.isError).toBe(true); + expect(parseMcpToolError(reply)).toMatchObject({ + error: { code: "cancelled" }, + }); + expect(reply.structuredContent).toBeUndefined(); + } finally { + await harness.close(); + } +}); + +it("searches bytes through MCP and the socket bridge with pagination and skipped coverage", async () => { + const config = parseConfig({ GHIDRA_INSTALL_DIR: "/fixture/ghidra" }); + if (!config.ok) throw config.error; + const provider = new GhidraProvider( + config.value, + silentLogger, + {}, + ghidraInstallationHost(), + (options) => + new GhidraClient({ + ...options, + platform: process.platform, + startupTimeoutMs: 5000, + launcher: { + launch: async (launch) => { + const projectRoot = join(launch.runtimeRoot, "project"); + await mkdir(projectRoot); + const child = spawn( + process.execPath, + [ + fileURLToPath( + new URL("../../fixtures/fakeGhidra.mjs", import.meta.url), + ), + ], + { + stdio: ["ignore", "pipe", "pipe"], + env: { + ...process.env, + REA_GHIDRA_FIXTURE_CONFIG: JSON.stringify({ + endpointPath: launch.endpointPath, + token: launch.token, + runId: launch.runId, + providerVersion: launch.providerVersion, + profileDigest: launch.profileDigest, + targetSha256: launch.targetSha256, + transport: launch.transport, + }), + }, + }, + ); + return ok({ + process: child, + ownsProcessLifetime: true as const, + projectRoot, + ghidraLogPath: join(launch.runtimeRoot, "ghidra.log"), + scriptLogPath: join(launch.runtimeRoot, "script.log"), + cleanup: async () => { + const signaled = + child.exitCode === null && child.signalCode === null; + if (signaled) child.kill("SIGTERM"); + return (await waitForExit(child, 2000)) + ? { cleaned: true as const, signaled } + : { cleaned: false as const, reason: "Fixture did not exit" }; + }, + }); + }, + }, + }), + ); + const session = composeBinarySession( + new AnalysisProviderRegistry([provider]), + ); + const server = createServer( + { kind: "session", session }, + { logger: silentLogger }, + ); + const client = new Client({ name: "byte-search", version: "1" }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + try { + await server.connect(serverTransport); + await client.connect(clientTransport); + const opened = await client.callTool({ + name: "open_binary", + arguments: { path: process.execPath }, + }); + expect(opened.isError).not.toBe(true); + const first = await client.callTool({ + name: "search_bytes", + arguments: { pattern: "48 8b ?? ?? 89", limit: 1 }, + }); + expect(first.isError, JSON.stringify(first)).not.toBe(true); + const firstEvidence = sessionEvidence(session, first.structuredContent); + expect(firstEvidence.normalized_result).toMatchObject({ + matches: [ + { + address: "0x1000", + segment: ".text", + file_offset: 0, + procedure: { name: "fixture_main" }, + }, + ], + searched: { + bytes: 10, + skipped_uninitialized: [{ segment: ".bss", bytes: 16 }], + }, + complete: false, + next_cursor: "fixture-second-page", + }); + const second = await client.callTool({ + name: "search_bytes", + arguments: { + pattern: "48 8b ?? ?? 89", + limit: 1, + cursor: "fixture-second-page", + }, + }); + expect(second.isError).not.toBe(true); + expect( + sessionEvidence(session, second.structuredContent).normalized_result, + ).toMatchObject({ + matches: [{ address: "0x1005", file_offset: 5 }], + complete: true, + next_cursor: null, + }); + const invalid = await client.callTool({ + name: "search_bytes", + arguments: { pattern: "??" }, + }); + expect(invalid.isError).toBe(true); + expect(JSON.stringify(invalid.content)).toContain( + "Pattern must constrain at least one bit", + ); + } finally { + await client.close(); + await server.close(); + const closed = await session.close(); + if (!closed.ok) throw closed.error; + } +}); + +it("retains missing PDB coverage in empty PE name-search Evidence and snapshot replay", async () => { + const root = await createTestTempDirectory("rea-pdb-evidence-"); + const targetPath = join(root, "fixture.exe"); + await writeFile(targetPath, nativeAotPeFixture()); + // Synthetic post-analysis facts at the real provider boundary; no engine is simulated as verified. + const pdb = { + file: "C:\\build\\fixture.pdb", + guid: "01234567-89AB-CDEF-0123-456789ABCDEF", + age: "a", + signature: null, + version: "RSDS", + loaded: false, + }; + const harness = await connectGhidraMcp( + "ghidra-pdb-coverage", + (operation) => + Promise.resolve( + ok( + operation === "procedure_pseudo_code" + ? { + value: "int FUN_140001000(void) { return 0; }", + limitations: [], + } + : [], + ), + ), + { + targetPath, + pdb, + }, + ); + try { + const reply = await harness.mcp.callTool({ + name: "search_procedures", + arguments: { pattern: "add_numbers" }, + }); + expect(reply.isError).not.toBe(true); + const evidence = sessionEvidence(harness.session, reply.structuredContent); + expect(evidence.normalized_result).toEqual([]); + expect(evidence.analysis_profile?.parameters.pdb_observations).toBe( + "program-info-v1", + ); + const limitation = evidence.limitations.find((value) => + value.includes(pdb.file), + ); + expect(limitation).toContain(pdb.guid); + expect(limitation).toContain("age (hex) a"); + expect(limitation).toContain("not loaded"); + const decompiled = await harness.mcp.callTool({ + name: "procedure_pseudo_code", + arguments: { procedure: "0x140001000", document: "fixture" }, + }); + expect(decompiled.isError).not.toBe(true); + const functionEvidence = sessionEvidence( + harness.session, + decompiled.structuredContent, + ); + expect(functionEvidence.limitations).toContain(limitation); + const replay = await harness.session.execute( + "procedure_pseudo_code", + functionEvidence.parameters, + ); + if (!replay.ok) throw replay.error; + expect(replay.value.limitations).toContain(limitation); + expect(replay.value.limitations).toContainEqual( + expect.stringContaining("Loaded from a local REA analysis snapshot"), + ); + } finally { + await harness.close(); + } +}); it("retains defined-string coverage in empty and matching search Evidence", async () => { let output = jsonValueSchema.parse([]); diff --git a/tests/boundary/mcp/ghidraMcpHarness.ts b/tests/boundary/mcp/ghidraMcpHarness.ts index a9b2248dc..fea27b037 100644 --- a/tests/boundary/mcp/ghidraMcpHarness.ts +++ b/tests/boundary/mcp/ghidraMcpHarness.ts @@ -12,6 +12,7 @@ import type { GhidraOperation } from "../../../src/ghidra/GhidraClient.js"; import { GhidraProvider } from "../../../src/ghidra/GhidraProvider.js"; import type { GhidraProviderClientFactory } from "../../../src/ghidra/GhidraProviderClient.js"; import { GHIDRA_SESSION_CAPABILITIES } from "../../../src/ghidra/GhidraSessionValues.js"; +import type { GhidraSessionInfo } from "../../../src/ghidra/GhidraSessionValues.js"; import { silentLogger } from "../../../src/logger.js"; import { createServer } from "../../../src/server/createServer.js"; import { @@ -26,12 +27,19 @@ const INSTALL = "/opt/ghidra_12.1.4_PUBLIC"; export const connectGhidraMcp = async ( name: string, execute?: ReturnType["callTool"], + observations: { + readonly targetPath?: string; + readonly pdb?: GhidraSessionInfo["pdb"]; + } = {}, ) => { const calls: GhidraOperation[] = []; const factory: GhidraProviderClientFactory = (options) => ({ start: () => Promise.resolve( - ok(sessionInfo(options.profileDigest, options.targetSha256)), + ok({ + ...sessionInfo(options.profileDigest, options.targetSha256), + ...(observations.pdb === undefined ? {} : { pdb: observations.pdb }), + }), ), callTool: (operation, input, options) => { calls.push(operation); @@ -46,7 +54,7 @@ export const connectGhidraMcp = async ( config.value, silentLogger, {}, - installationHost(), + ghidraInstallationHost(), factory, ); const session = composeBinarySession( @@ -68,7 +76,7 @@ export const connectGhidraMcp = async ( await mcp.connect(clientTransport); const opened = await mcp.callTool({ name: "open_binary", - arguments: { path: process.execPath }, + arguments: { path: observations.targetPath ?? process.execPath }, }); if (opened.isError === true) throw new Error( @@ -92,7 +100,7 @@ export const sessionEvidence = (session: BinarySession, value: unknown) => { return evidence; }; -const installationHost = (): GhidraInstallationHost => ({ +export const ghidraInstallationHost = (): GhidraInstallationHost => ({ platform: "linux", architecture: "x64", readText: () => "application.version=12.1.4\napplication.java.min=21\n", diff --git a/tests/boundary/mcp/officialArguments.test.ts b/tests/boundary/mcp/officialArguments.test.ts index ff0284ac0..a5fe0f2b1 100644 --- a/tests/boundary/mcp/officialArguments.test.ts +++ b/tests/boundary/mcp/officialArguments.test.ts @@ -43,7 +43,23 @@ describe("official operation arguments", () => { const execute: AnalysisOperationPort["execute"] = (name, parameters) => { calls.push({ name, parameters }); return Promise.resolve( - observed(name === "search_strings" ? [] : "int main() {}"), + observed( + name === "search_bytes" + ? { + matches: [], + searched: { + blocks: [], + bytes: 0, + skipped_uninitialized: [], + skipped_not_loaded: [], + }, + complete: true, + next_cursor: null, + } + : name === "search_strings" + ? [] + : "int main() {}", + ), ); }; const createSession = () => @@ -59,6 +75,7 @@ describe("official operation arguments", () => { const client = await connect({ execute }); for (const [name, mcpParameters, cliParameters] of [ ["procedure_pseudo_code", { procedure: "main" }, { procedure: "main" }], + ["search_bytes", { pattern: "48 ??" }, { pattern: "48 ??", limit: 100 }], [ "search_strings", { pattern: "entry" }, @@ -93,6 +110,11 @@ describe("official operation arguments", () => { ["list_procedures", { document: null }], ["procedure_pseudo_code", { procedure: "main", document: 3 }], ["search_strings", { pattern: "entry", case_sensitive: null }], + ["search_bytes", { pattern: "??" }], + ["search_bytes", { pattern: "48 8B", mask: "ff" }], + ["search_bytes", { pattern: "4?" }], + ["search_bytes", { pattern: "48", limit: 1001 }], + ["search_bytes", { pattern: "48", cursor: null }], ] as const) { const result = await client.callTool({ name, arguments: parameters }); expect(result.isError).toBe(true); diff --git a/tests/boundary/mcp/officialTools.test.ts b/tests/boundary/mcp/officialTools.test.ts index 70fbf8587..e28639dd9 100644 --- a/tests/boundary/mcp/officialTools.test.ts +++ b/tests/boundary/mcp/officialTools.test.ts @@ -10,6 +10,12 @@ import { err } from "../../../src/domain/result.js"; import { observed as ok } from "../../fixtures/analysisExecution.js"; import type { JsonValue } from "../../../src/domain/jsonValue.js"; import { createServer } from "../../../src/server/createServer.js"; +import { HopperProvider } from "../../../src/hopper/HopperProvider.js"; +import { parseConfig } from "../../../src/config/parseConfig.js"; +import { silentLogger } from "../../../src/logger.js"; +import { composeBinarySession } from "../../../src/application/binary/BinarySessionComposition.js"; +import { AnalysisProviderRegistry } from "../../../src/application/binary/AnalysisProviderRegistry.js"; +import { createBinarySessionTargets } from "../../fixtures/binarySession.js"; interface Invocation { readonly name: string; @@ -18,6 +24,67 @@ interface Invocation { const resources: Array<{ close(): Promise }> = []; +it("returns typed Hopper byte-search unavailability without launching the provider", async () => { + const [target] = await createBinarySessionTargets(); + const config = parseConfig({ HOPPER_LAUNCHER_PATH: process.execPath }); + if (!config.ok) throw config.error; + const provider = new HopperProvider(config.value, silentLogger, {}, "linux"); + const session = composeBinarySession( + new AnalysisProviderRegistry([provider]), + ); + const server = createServer( + { kind: "session", session }, + { logger: silentLogger }, + ); + const client = new Client({ + name: "hopper-byte-search-unavailable", + version: "1", + }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + try { + await server.connect(serverTransport); + await client.connect(clientTransport); + const opened = await client.callTool({ + name: "open_binary", + arguments: { path: target }, + }); + expect(opened.isError).not.toBe(true); + const result = await client.callTool({ + name: "search_bytes", + arguments: { pattern: "48 8b" }, + }); + expect(result.isError).toBe(true); + expect(parseMcpToolError(result)).toMatchObject({ + error: { + code: "capability_unavailable", + details: { provider_id: "hopper", operation: "search_bytes" }, + }, + }); + } finally { + await client.close(); + await server.close(); + const closed = await session.close(); + if (!closed.ok) throw closed.error; + } +}); + +it("advertises strict masked-byte arguments and bounded pages", () => { + const contract = OFFICIAL_TOOL_CONTRACTS.find( + ({ name }) => name === "search_bytes", + ); + expect( + contract?.inputSchema.parse({ pattern: "48 8B ?? ?? 89" }), + ).toMatchObject({ limit: 100 }); + expect( + contract?.inputSchema.safeParse({ pattern: "48", ignored: true }).success, + ).toBe(false); + expect( + contract?.inputSchema.safeParse({ pattern: "48", mask: "f0", limit: 1000 }) + .success, + ).toBe(true); +}); + describe("read_bytes contract", () => { it("accepts lengths above the former arbitrary ceiling and rejects zero", () => { const contract = OFFICIAL_TOOL_CONTRACTS.find( diff --git a/tests/boundary/mcp/oversizedErrorDeliveryMcp.test.ts b/tests/boundary/mcp/oversizedErrorDeliveryMcp.test.ts index f03d349b2..10d081d84 100644 --- a/tests/boundary/mcp/oversizedErrorDeliveryMcp.test.ts +++ b/tests/boundary/mcp/oversizedErrorDeliveryMcp.test.ts @@ -257,3 +257,72 @@ it("keeps the untrusted target-content marker on an oversized error's replacemen await Promise.all([client.close(), server.close()]); } }); + +// The delivered error wire omits only the private structured carrier; its +// `isError` flag and any provenance marker count against the result budget. +const deliveredErrorBytes = (nameBytes: number, targetContent: boolean) => { + const { structuredContent: _carrier, ...wire } = failureWithName( + nameBytes, + targetContent, + ); + return Buffer.byteLength(JSON.stringify(wire)); +}; + +it.each([false, true])( + "applies the result budget to the complete delivered error (target content: %s)", + async (targetContent) => { + const fittingName = + delivery.resultBudgetBytes - deliveredErrorBytes(0, targetContent); + expect(deliveredErrorBytes(fittingName, targetContent)).toBe( + delivery.resultBudgetBytes, + ); + const server = new EvidenceMcpServer( + { name: "error-budget-test", version: "1" }, + { capabilities: {} }, + undefined, + delivery, + ); + server.registerTool( + "fitting_failure", + { inputSchema: z.object({}) }, + async () => failureWithName(fittingName, targetContent), + ); + server.registerTool( + "one_byte_over_failure", + { inputSchema: z.object({}) }, + async () => failureWithName(fittingName + 1, targetContent), + ); + const client = new Client({ name: "error-budget-client", version: "1" }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + try { + await server.connect(serverTransport); + await client.connect(clientTransport); + const fitting = await client.callTool({ + name: "fitting_failure", + arguments: {}, + }); + expect(fitting.isError).toBe(true); + expect(parseMcpToolError(fitting)).toEqual( + failureWithName(fittingName, targetContent).structuredContent, + ); + const over = await client.callTool({ + name: "one_byte_over_failure", + arguments: {}, + }); + expect(over.isError).toBe(true); + expect(over.structuredContent).toBeUndefined(); + expect(parseMcpToolError(over)).toMatchObject({ + error: { + code: "resource_constraint", + details: { + reported_limits: { original_error_code: "invalid_request" }, + }, + }, + }); + } finally { + await Promise.all([client.close(), server.close()]); + } + }, + 20000, +); diff --git a/tests/boundary/mcp/sessionEvidenceRecovery.test.ts b/tests/boundary/mcp/sessionEvidenceRecovery.test.ts new file mode 100644 index 000000000..dab264dbc --- /dev/null +++ b/tests/boundary/mcp/sessionEvidenceRecovery.test.ts @@ -0,0 +1,184 @@ +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import type { CallToolResult } from "@modelcontextprotocol/server"; +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { join } from "node:path"; +import { expect, it } from "vitest"; +import { z } from "zod"; + +import { createServer } from "../../../src/server/createServer.js"; +import { evidenceBundleSchema } from "../../../src/domain/evidenceBundle.js"; +import { ProviderCleanupError } from "../../../src/domain/providerCleanupError.js"; +import { err, ok } from "../../../src/domain/result.js"; +import { + createCacheProvider, + createTestBinarySession, +} from "../../fixtures/binarySession.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { parseMcpToolError } from "../../fixtures/mcpToolError.js"; + +const bundleResult = (reply: CallToolResult) => + z.object({ result: evidenceBundleSchema }).parse(reply.structuredContent) + .result; + +const successfulTool = async ( + client: Client, + name: string, + arguments_: Record = {}, +): Promise => { + const reply = await client.callTool({ name, arguments: arguments_ }); + expect(reply.isError, `${name}: ${JSON.stringify(reply.content)}`).not.toBe( + true, + ); + return reply; +}; + +const recordUnknownHistory = async (client: Client): Promise => { + const fields = { + severity: "medium", + required_authority: "shipped-artifact", + required_confidence: "observed", + required_environment: null, + recommended_probes: [], + relationships: [], + }; + const unknown = await successfulTool(client, "record_unknown", { + ...fields, + question: "Which source revision produced this target?", + domain: "provenance", + }); + const recorded = z + .object({ result: z.object({ unknown_id: z.string() }) }) + .parse(unknown.structuredContent).result; + await successfulTool(client, "update_unknown", { + ...fields, + unknown_id: recorded.unknown_id, + expected_revision: 1, + status: "investigating", + supporting_evidence_ids: [], + contradicting_evidence_ids: [], + resolution: null, + }); +}; + +it.each(["switch", "restart"] as const)( + "preserves investigation records through failed target %s and cleanup retries", + async (transition) => { + const root = await createTestTempDirectory("rea-mcp-switch-evidence-"); + const first = join(root, "first.hop"); + const second = join(root, "second.hop"); + await writeFile(first, "first source-owned target"); + await writeFile(second, "second source-owned target"); + const application = join(root, "application"); + await mkdir(application); + await writeFile( + join(application, "index.js"), + 'export const label = "retained";', + ); + const nextTarget = transition === "switch" ? second : first; + const calls: string[] = []; + const base = createCacheProvider(calls); + let cleanupAllowed = true; + let created = 0; + const failure = new ProviderCleanupError( + "fixture", + [join(root, "runtime")], + { + reason: "fixture cleanup temporarily unavailable", + }, + ); + const session = createTestBinarySession({ + ...base, + createClient: (target, profile, context) => { + created += 1; + return { + ...base.createClient(target, profile, context), + operationHealthSnapshot: () => ({ + state: cleanupAllowed ? "idle" : "exited", + stage: "analysis", + retryAction: cleanupAllowed ? null : "restart_provider", + exitCode: cleanupAllowed ? null : 1, + requests: [], + }), + close: () => + Promise.resolve(cleanupAllowed ? ok(null) : err(failure)), + }; + }, + }); + const server = createServer( + { kind: "session", session }, + { environment: {} }, + ); + const client = new Client({ + name: "switch-evidence-recovery", + version: "1", + }); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + const call = (name: string, arguments_: Record = {}) => + successfulTool(client, name, arguments_); + const query = { address: "0x1000", document: "fixture" }; + try { + await server.connect(serverTransport); + await client.connect(clientTransport); + await call("open_binary", { path: first }); + // The cached primitive must be discarded when the provider is retired, + // independently of the investigation's Evidence and Unknown history. + await call("address_name", query); + const analyzed = await call("analyze_javascript_application", { + input_path: application, + format: "directory", + max_heap_mb: 128, + }); + const applicationId = z + .object({ evidence_id: z.string() }) + .parse(analyzed.structuredContent).evidence_id; + await recordUnknownHistory(client); + const before = bundleResult(await call("get_evidence_bundle")); + expect(before.records).toHaveLength(4); + expect(before.unknowns).toHaveLength(2); + + cleanupAllowed = false; + for (let attempt = 0; attempt < 2; attempt += 1) { + const replacement = await client.callTool({ + name: "open_binary", + arguments: { path: nextTarget }, + }); + expect(parseMcpToolError(replacement)).toMatchObject({ + error: { code: "cleanup_incomplete" }, + }); + expect(session.activeTarget()).toBeUndefined(); + expect(created).toBe(1); + expect(bundleResult(await call("get_evidence_bundle"))).toEqual(before); + } + const exportedPath = join(root, "retained.json"); + await call("export_evidence_bundle", { path: exportedPath }); + expect( + evidenceBundleSchema.parse( + JSON.parse(await readFile(exportedPath, "utf8")), + ), + ).toEqual(before); + + cleanupAllowed = true; + await call("open_binary", { path: nextTarget }); + expect(bundleResult(await call("get_evidence_bundle"))).toEqual(before); + await call("address_name", query); + expect( + calls.filter((operation) => operation === "address_name"), + ).toHaveLength(2); + expect(created).toBe(2); + await call("inspect_analysis_view", { + source: { kind: "retained-evidence", evidence_id: applicationId }, + view: { kind: "page", collection: "modules", offset: 0, limit: 10 }, + }); + await call("close_binary"); + const closed = bundleResult(await call("get_evidence_bundle")); + expect(closed.records).toEqual([]); + expect(closed.unknowns).toEqual([]); + } finally { + cleanupAllowed = true; + await client.close(); + await server.close(); + await session.close(); + } + }, +); diff --git a/tests/boundary/mcp/toolSchemaValidity.test.ts b/tests/boundary/mcp/toolSchemaValidity.test.ts index 5a94d27bf..e9d5e944a 100644 --- a/tests/boundary/mcp/toolSchemaValidity.test.ts +++ b/tests/boundary/mcp/toolSchemaValidity.test.ts @@ -217,6 +217,46 @@ function resolveReference(schema: unknown, reference: string): unknown { return value; } +function localReferenceExpansions(schema: Record): number { + // Clients that render SDK declarations from the advertised input expand + // schema-local references against a shared per-input budget; Pi 1.1.0 + // code-mode allows 32 and renders every later reference as `unknown`. + // Count conservatively: every resolvable, non-cyclic reference occurrence + // reached while walking the input, never the definitions themselves. + let count = 0; + const visit = (node: unknown, active: ReadonlySet): void => { + if (Array.isArray(node)) { + node.forEach((child) => visit(child, active)); + return; + } + if (!isRecord(node)) return; + if (typeof node.$ref === "string" && !active.has(node.$ref)) { + const target = resolveReference(schema, node.$ref); + if (target !== undefined) { + count += 1; + visit(target, new Set([...active, node.$ref])); + return; + } + } + for (const [key, value] of Object.entries(node)) { + if ( + [ + "$defs", + "definitions", + "examples", + "default", + "const", + "enum", + ].includes(key) + ) + continue; + visit(value, active); + } + }; + visit(schema, new Set()); + return count; +} + function recursiveReferences(tools: readonly ToolSchemas[]): string[] { return tools.flatMap((tool) => (["inputSchema", "outputSchema"] as const).flatMap((kind) => { @@ -674,6 +714,14 @@ it("ships nonrecursive input and output schemas in the generated catalog", () => expect(recursiveReferences(GENERATED_MCP_TOOL_CATALOG)).toEqual([]); }); +it("keeps advertised input references within the client expansion budget", () => { + for (const tool of GENERATED_MCP_TOOL_CATALOG) + expect( + localReferenceExpansions(tool.inputSchema), + `${tool.name} exhausts the shared reference-expansion budget`, + ).toBeLessThanOrEqual(32); +}); + it("rejects unresolved output-schema references even for uncalled tools", () => { const errors = schemaErrors([ { diff --git a/tests/boundary/mcp/wasmArtifactMcp.test.ts b/tests/boundary/mcp/wasmArtifactMcp.test.ts index 003a37cba..04c5e59da 100644 --- a/tests/boundary/mcp/wasmArtifactMcp.test.ts +++ b/tests/boundary/mcp/wasmArtifactMcp.test.ts @@ -1,5 +1,9 @@ -import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; import { Ajv2020 } from "ajv/dist/2020.js"; +import { + advertisedMcpToolSchemas, + assertRecordedMcpEvidence, +} from "../../fixtures/advertisedMcpTool.js"; +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; import { expect, it, onTestFinished, vi } from "vitest"; import { readFile, realpath, writeFile } from "node:fs/promises"; import { join } from "node:path"; @@ -20,20 +24,6 @@ import { wasmArtifactExecution, } from "../../fixtures/wasm/artifact.js"; -const wasmToolSchemas = async (client: Client) => { - const advertised = (await client.listTools()).tools.find( - (tool) => tool.name === "inspect_wasm_artifact", - ); - if (advertised?.outputSchema === undefined) - throw new Error("WASM recovery must name an advertised static tool"); - const ajv = new Ajv2020({ strict: false, validateFormats: false }); - const inputSchema: Record = advertised.inputSchema; - const outputSchema: Record = advertised.outputSchema; - expect(ajv.validateSchema(inputSchema)).toBe(true); - expect(ajv.validateSchema(outputSchema)).toBe(true); - return { name: advertised.name, ajv, inputSchema, outputSchema }; -}; - const assertFirstSummaryDelivery = async ( harness: Awaited>, value: ReturnType, @@ -43,7 +33,7 @@ const assertFirstSummaryDelivery = async ( ajv: wasmAjv, inputSchema, outputSchema, - } = await wasmToolSchemas(harness.client); + } = await advertisedMcpToolSchemas(harness.client, "inspect_wasm_artifact"); const summaryInput = { path: value.artifact.path, detail: "summary" }; expect(wasmAjv.validate(inputSchema, summaryInput)).toBe(true); expect( @@ -91,73 +81,86 @@ const assertFirstSummaryDelivery = async ( return { summary, retained }; }; -it("directs rejected native WASM opening to the advertised static tool while preserving missing-WABT diagnostics", async () => { - const directory = await createTestTempDirectory("rea-wasm-recovery-"); - const path = join( - await realpath(directory), - process.platform === "win32" ? "module space.bin" : 'module "quoted".bin', - ); - await writeFile(path, Buffer.from([0, 0x61, 0x73, 0x6d, 1, 0, 0, 0])); - const createClient = vi.fn(() => { - throw new Error("native provider must not start for WASM"); - }); - const session = createTestBinarySession(createClient); - const harness = await createApplicationMcpHarness({ - session, - serverOptions: { environment: {} }, - }); - onTestFinished(harness.close); - const { client } = harness; - const { name, ajv, inputSchema } = await wasmToolSchemas(client); - const input = { path }; - expect( - toolContract("inspect_wasm_artifact").inputSchema.safeParse(input).success, - ).toBe(true); - const opening = await client.callTool({ - name: "open_binary", - arguments: input, - }); - expect(opening.isError).toBe(true); - const failure = parseMcpToolError(opening); - expect(failure).toMatchObject({ - error: { - code: "target_unavailable", - category: "unsupported_target", - details: { constraint: "wasm_requires_static_inspection" }, - }, - }); - expect(failure.error.remediation.action).toContain(JSON.stringify({ path })); - expect(ajv.validate(inputSchema, input)).toBe(true); - const inspection = await client.callTool({ - name, - arguments: input, - }); - expect(inspection.isError).toBe(true); - expect(parseMcpToolError(inspection)).toMatchObject({ - error: { - code: "capability_unavailable", - details: { - operation: "inspect_wasm_artifact", - reason: expect.stringContaining("REA_WABT_BIN_DIRECTORY"), +it.each(["bin", "hop"])( + "directs rejected native WASM opening (%s suffix) to the advertised static tool while preserving missing-WABT diagnostics", + async (extension) => { + const directory = await createTestTempDirectory("rea-wasm-recovery-"); + const path = join( + await realpath(directory), + process.platform === "win32" + ? `module space.${extension}` + : `module "quoted".${extension}`, + ); + await writeFile(path, Buffer.from([0, 0x61, 0x73, 0x6d, 1, 0, 0, 0])); + const createClient = vi.fn(() => { + throw new Error("native provider must not start for WASM"); + }); + const session = createTestBinarySession(createClient); + const harness = await createApplicationMcpHarness({ + session, + serverOptions: { environment: {} }, + }); + onTestFinished(harness.close); + const { client } = harness; + const { name, ajv, inputSchema } = await advertisedMcpToolSchemas( + client, + "inspect_wasm_artifact", + ); + const input = { path }; + const opening = await client.callTool({ + name: "open_binary", + arguments: input, + }); + expect(opening.isError).toBe(true); + const failure = parseMcpToolError(opening); + expect(failure).toMatchObject({ + error: { + code: "target_unavailable", + category: "unsupported_target", + details: { constraint: "wasm_requires_static_inspection" }, }, - }, - }); - const summary = await client.callTool({ - name, - arguments: { ...input, detail: "summary" }, - }); - expect(parseMcpToolError(summary)).toMatchObject({ - error: { - code: "capability_unavailable", - details: { reason: expect.stringContaining("REA_WABT_BIN_DIRECTORY") }, - }, - }); - await client.ping(); - expect(createClient).not.toHaveBeenCalled(); - expect(await readFile(path)).toEqual( - Buffer.from([0, 0x61, 0x73, 0x6d, 1, 0, 0, 0]), - ); -}); + }); + const [, recoveryName, serializedInput] = + failure.error.remediation.action.match( + /call (\w+) with (\{.*\}) or run/, + ) ?? []; + expect(recoveryName).toBe(name); + const recoveryInput = z + .object({ path: z.literal(path) }) + .strict() + .parse(JSON.parse(serializedInput ?? "null")); + expect(ajv.validate(inputSchema, recoveryInput)).toBe(true); + const inspection = await client.callTool({ + name: recoveryName ?? "", + arguments: recoveryInput, + }); + expect(inspection.isError).toBe(true); + expect(parseMcpToolError(inspection)).toMatchObject({ + error: { + code: "capability_unavailable", + details: { + operation: "inspect_wasm_artifact", + reason: expect.stringContaining("REA_WABT_BIN_DIRECTORY"), + }, + }, + }); + const summary = await client.callTool({ + name, + arguments: { ...input, detail: "summary" }, + }); + expect(parseMcpToolError(summary)).toMatchObject({ + error: { + code: "capability_unavailable", + details: { reason: expect.stringContaining("REA_WABT_BIN_DIRECTORY") }, + }, + }); + await client.ping(); + expect(createClient).not.toHaveBeenCalled(); + expect(await readFile(path)).toEqual( + Buffer.from([0, 0x61, 0x73, 0x6d, 1, 0, 0, 0]), + ); + }, +); it("advertises exact valid schemas and records inline WASM interface evidence without a binary target", async () => { const value = wasmArtifactFixture(); @@ -175,7 +178,11 @@ it("advertises exact valid schemas and records inline WASM interface evidence wi }); onTestFinished(harness.close); const { client } = harness; - const { ajv, inputSchema, outputSchema } = await wasmToolSchemas(client); + const schemas = await advertisedMcpToolSchemas( + client, + "inspect_wasm_artifact", + ); + const { ajv, inputSchema } = schemas; expect(ajv.validate(inputSchema, { path: value.artifact.path })).toBe(true); expect( ajv.validate(inputSchema, { @@ -188,19 +195,14 @@ it("advertises exact valid schemas and records inline WASM interface evidence wi name: "inspect_wasm_artifact", arguments: { path: value.artifact.path }, }); - expect(response.isError).not.toBe(true); - expect( - ajv.validate(outputSchema, response.structuredContent), - JSON.stringify(ajv.errors), - ).toBe(true); - const parsed = toolContract("inspect_wasm_artifact").outputSchema.parse( - response.structuredContent, + assertRecordedMcpEvidence( + response, + "inspect_wasm_artifact", + schemas, + session, + value, + "observed", ); - const evidence = parseEvidence(parsed); - expect(evidence.confidence).toBe("observed"); - expect(evidence.subject?.digest.sha256).toBe(value.artifact.sha256); - expect(parsed.normalized_result).toEqual(value); - expect(session.evidenceById(evidence.evidence_id)).toEqual(evidence); expect( ( await client.callTool({ diff --git a/tests/boundary/process/cleanLinuxHopperRuntime.test.ts b/tests/boundary/process/cleanLinuxHopperRuntime.test.ts index 79a536f37..b296ca15a 100644 --- a/tests/boundary/process/cleanLinuxHopperRuntime.test.ts +++ b/tests/boundary/process/cleanLinuxHopperRuntime.test.ts @@ -7,6 +7,7 @@ import { readFile, symlink, unlink, + writeFile, } from "node:fs/promises"; import { join } from "node:path"; import { promisify } from "node:util"; diff --git a/tests/boundary/process/jadxIntegration.test.ts b/tests/boundary/process/jadxIntegration.test.ts index 86e2c5ab7..173f5b522 100644 --- a/tests/boundary/process/jadxIntegration.test.ts +++ b/tests/boundary/process/jadxIntegration.test.ts @@ -342,17 +342,30 @@ it("exposes overload candidates and executes only an explicitly selected ambiguo expect(result.method).toMatchObject({ overload_index: 1, reported_signature: "choose(int): void", - dex_descriptor: null, + dex_descriptor: "(I)V", }); expect(result.source.text).toContain("overload 1"); const references = await service.execute("trace_android_references", input); expect(references).toMatchObject({ ok: false, error: { - _tag: "AnalysisCapabilityUnavailableError", - reason: expect.stringContaining("unique method name"), + _tag: "AnalysisInputError", + issues: [expect.objectContaining({ path: ["overload_index"] })], }, }); + const selectedReferences = await service.execute("trace_android_references", { + ...input, + overload_index: 1, + }); + if (!selectedReferences.ok) throw selectedReferences.error; + expect( + androidResultSchemas.trace_android_references.parse( + selectedReferences.value.normalized_result, + ), + ).toMatchObject({ + overload_index: 1, + reported_signature: "choose(int): void", + }); await provider.close(); await verifyCleanup(launches); }); diff --git a/tests/boundary/providers/ghidra/ghidraLauncher.test.ts b/tests/boundary/providers/ghidra/ghidraLauncher.test.ts index ea8fb964c..98c231240 100644 --- a/tests/boundary/providers/ghidra/ghidraLauncher.test.ts +++ b/tests/boundary/providers/ghidra/ghidraLauncher.test.ts @@ -486,7 +486,9 @@ describe("Ghidra Windows launcher", () => { }), ).toThrow(/metacharacters/u); }); +}); +describe("Ghidra Windows launcher runtime", () => { it.skipIf(process.platform !== "win32").each([ { properties: diff --git a/tests/boundary/providers/ida/calleeRelationships.test.ts b/tests/boundary/providers/ida/calleeRelationships.test.ts index 1204b4d10..523304dac 100644 --- a/tests/boundary/providers/ida/calleeRelationships.test.ts +++ b/tests/boundary/providers/ida/calleeRelationships.test.ts @@ -208,6 +208,22 @@ it("preserves missing callgraph coverage through direct, graph, and trace workfl ]), ); + const dossier = await client.execute("analyze_function", { + procedure: "main", + }); + if (!dossier.ok) throw dossier.error; + expect(dossier.value.result).toMatchObject({ + callers: [], + callees: [], + facet_availability: { + callers: { available: false }, + callees: { + available: false, + reason: expect.stringContaining("callgraph was unavailable"), + }, + }, + }); + const application = new EnhancedTools(client); const graph = await application.execute("get_call_graph", { address: "main", diff --git a/tests/boundary/providers/native/nativeInventory.golden.test.ts b/tests/boundary/providers/native/nativeInventory.golden.test.ts index 5ac80e778..934d24137 100644 --- a/tests/boundary/providers/native/nativeInventory.golden.test.ts +++ b/tests/boundary/providers/native/nativeInventory.golden.test.ts @@ -225,6 +225,79 @@ it("retains native inventory facts from captured Apple tool output", async () => }); }); +it("reports a renamed re-export once under its exported name", async () => { + const client = new NativeMacOSProvider( + {}, + new FixtureRunner({ + "dyld_info:-exports": await fixture( + "dyld-inventory/renamed-reexports.txt", + ), + nm: "_foo_renamed\n_own\n", + }), + "darwin", + ).createClient(machoTarget("/private/fixture")); + const execution = await client.execute("inspect_macho", {}); + if (!execution.ok) throw execution.error; + expect(execution.value.result).toMatchObject({ + exports: { + items: [ + { + name: "_foo_renamed", + address: null, + weak: null, + reexport: true, + source: "libfoo", + }, + { name: "_own", reexport: false, source: null }, + ], + limitations: [ + "Merged nm/dyld_info results may be incomplete for stripped binaries.", + "Renamed re-exports are listed under their exported name; the source library's symbol name is not reported.", + ], + }, + }); +}); + +it("splits re-export rows at the source library", () => { + // dyld_info -exports rows for synthetic dylibs: a source symbol named + // `operator()`, and a library whose install name contains ` from `. + expect( + parseDyldSymbols( + [ + "[re-export] _call_renamed (operator() from libfoo)", + "[re-export] _g (from lib from foo)", + "[re-export] _nested (from lib (from foo))", + ].join("\n"), + "exports", + null, + new Set(["_call_renamed", "_g", "_nested"]), + ), + ).toEqual([ + { + name: "_call_renamed", + address: null, + weak: null, + reexport: true, + source: "libfoo", + sourceSymbol: "operator()", + }, + { + name: "_g", + address: null, + weak: null, + reexport: true, + source: "lib from foo", + }, + { + name: "_nested", + address: null, + weak: null, + reexport: true, + source: "lib (from foo)", + }, + ]); +}); + it("retains symbol names containing spaces in dyld inventory rows", () => { expect( parseDyldSymbols(" 0x0000 _entry with space (from fixture)", "imports"), @@ -449,6 +522,25 @@ it("leaves an ambiguous short and literal name to the caller's nm inventory", () ).toEqual([]); }); +it("retains unmatched re-export rows when the known export inventory is incomplete", () => { + expect( + parseDyldSymbols( + "[re-export] _g (from libfoo)", + "exports", + null, + new Set(["_own"]), + ), + ).toEqual([ + { + name: "_g", + address: null, + weak: null, + reexport: true, + source: "libfoo", + }, + ]); +}); + it("retains annotation parsing without nm name evidence", () => { expect(parseDyldSymbols("0x120 _weak [weak-def]", "exports")).toEqual([ { diff --git a/tests/boundary/providers/native/nativeLoadKinds.golden.test.ts b/tests/boundary/providers/native/nativeLoadKinds.golden.test.ts index 87912a31d..dd83a1b3c 100644 --- a/tests/boundary/providers/native/nativeLoadKinds.golden.test.ts +++ b/tests/boundary/providers/native/nativeLoadKinds.golden.test.ts @@ -229,6 +229,62 @@ it("normalizes legacy version-minimum commands into build metadata", () => { ]); }); +it("names numeric build-version platforms and tools", () => { + const load = parseOtoolLoadCommands( + withOtoolMachHeader( + [ + "Load command 0", + " cmd LC_BUILD_VERSION", + " cmdsize 48", + " platform 1", + " minos 11.0", + " sdk 26.5", + " ntools 2", + " tool 3", + " version 1267.0", + " tool 1029", + " version 1.0", + "Load command 1", + " cmd LC_BUILD_VERSION", + " cmdsize 24", + " platform 99", + " minos 1.0", + " sdk 1.0", + " ntools 0", + "Load command 2", + " cmd LC_BUILD_VERSION", + " cmdsize 32", + " platform MACCATALYST", + " minos 14.0", + " sdk 14.2", + " ntools 1", + " tool SWIFT", + " version 5.9", + "", + ].join("\n"), + ), + ); + expect(load.builds).toEqual([ + { + platform: "MACOS", + minimum_os: "11.0", + sdk: "26.5", + tools: [ + { name: "LD", version: "1267.0" }, + { name: "1029", version: "1.0" }, + ], + }, + { platform: "99", minimum_os: "1.0", sdk: "1.0", tools: [] }, + { + platform: "MACCATALYST", + minimum_os: "14.0", + sdk: "14.2", + tools: [{ name: "SWIFT", version: "5.9" }], + }, + ]); + expect(load.commands[0]?.fields.platform).toBe(1); +}); + it("does not classify LC_LOAD_DYLINKER as a dylib dependency", async () => { const load = parseOtoolLoadCommands( await nativeFixture("native-macho-header/otool-header-load.txt"), @@ -273,3 +329,78 @@ it("limits thread-state entrypoints instead of reporting none silently", async ( "Thread-state entrypoints (LC_UNIXTHREAD/LC_THREAD) are retained as raw load commands; file offsets are not derived.", ); }); + +it.each([ + ["1", "MACOS"], + ["2", "IOS"], + ["3", "TVOS"], + ["4", "WATCHOS"], + ["5", "BRIDGEOS"], + ["6", "MACCATALYST"], + ["7", "IOSSIMULATOR"], + ["8", "TVOSSIMULATOR"], + ["9", "WATCHOSSIMULATOR"], + ["10", "DRIVERKIT"], + ["11", "VISIONOS"], + ["12", "VISIONOSSIMULATOR"], + ["13", "FIRMWARE"], + ["14", "SEPOS"], + ["15", "MACOS_EXCLAVECORE"], + ["16", "MACOS_EXCLAVEKIT"], + ["17", "IOS_EXCLAVECORE"], + ["18", "IOS_EXCLAVEKIT"], + ["19", "TVOS_EXCLAVECORE"], + ["20", "TVOS_EXCLAVEKIT"], + ["21", "WATCHOS_EXCLAVECORE"], + ["22", "WATCHOS_EXCLAVEKIT"], + ["23", "VISIONOS_EXCLAVECORE"], + ["24", "VISIONOS_EXCLAVEKIT"], +] as const)("names build platform %s as %s", (code, name) => { + const load = parseOtoolLoadCommands( + withOtoolMachHeader( + [ + "Load command 0", + "cmd LC_BUILD_VERSION", + "cmdsize 24", + `platform ${code}`, + "minos 1.0", + "sdk 1.0", + "ntools 0", + "", + ].join("\n"), + ), + ); + expect(load.builds[0]?.platform).toBe(name); +}); + +it.each([ + ["1", "CLANG"], + ["2", "SWIFT"], + ["3", "LD"], + ["4", "LLD"], + ["1024", "METAL"], + ["1025", "AIRLLD"], + ["1026", "AIRNT"], + ["1027", "AIRNT_PLUGIN"], + ["1028", "AIRPACK"], + ["1031", "GPUARCHIVER"], + ["1032", "METAL_FRAMEWORK"], +] as const)("names build tool %s as %s", (code, name) => { + const load = parseOtoolLoadCommands( + withOtoolMachHeader( + [ + "Load command 0", + "cmd LC_BUILD_VERSION", + "cmdsize 32", + "platform 1", + "minos 1.0", + "sdk 1.0", + "ntools 1", + `tool ${code}`, + "version 1.0", + "", + ].join("\n"), + ), + ); + expect(load.builds[0]?.tools).toEqual([{ name, version: "1.0" }]); +}); diff --git a/tests/boundary/providers/native/nativeProvider.test.ts b/tests/boundary/providers/native/nativeProvider.test.ts index a1e7d63ce..426b2f9f3 100644 --- a/tests/boundary/providers/native/nativeProvider.test.ts +++ b/tests/boundary/providers/native/nativeProvider.test.ts @@ -12,6 +12,7 @@ import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js import { NativeMacOSProvider } from "../../../../src/native/NativeMacOSProvider.js"; import { NATIVE_COMMAND_OUTPUT_BUDGET_BYTES, + NATIVE_FAILURE_DIAGNOSTIC_BYTES, NativeCommandFailure, XcrunCommandRunner, type NativeCommandRunner, @@ -381,6 +382,36 @@ describe("native command failure capture and limits", () => { }); }); + it("bounds both failure streams at a UTF-8 boundary while keeping their totals", async () => { + const runner = new XcrunCommandRunner({}, () => + Promise.resolve(ok({ path: process.execPath, sha256: "a".repeat(64) })), + ); + // "é" is two bytes and straddles the excerpt boundary, so it is dropped whole. + const stdout = `${"x".repeat(NATIVE_FAILURE_DIAGNOSTIC_BYTES - 1)}é${"y".repeat(1024)}`; + const stderr = "z".repeat(NATIVE_FAILURE_DIAGNOSTIC_BYTES + 1024); + // Build the output in the child: one Linux argument is limited to 128 KiB. + const result = await runner.run( + "file", + [ + "-e", + `const stdout = "x".repeat(${String(NATIVE_FAILURE_DIAGNOSTIC_BYTES - 1)}) + "\u00e9" + "y".repeat(1024); const stderr = "z".repeat(${String(NATIVE_FAILURE_DIAGNOSTIC_BYTES + 1024)}); process.stdout.write(stdout, () => process.stderr.write(stderr, () => process.exit(3)));`, + ], + {}, + ); + + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.reason).toBe("nonzero-exit"); + expect(result.error.capture).toMatchObject({ + stdout: "x".repeat(NATIVE_FAILURE_DIAGNOSTIC_BYTES - 1), + stderr: "z".repeat(NATIVE_FAILURE_DIAGNOSTIC_BYTES), + stdoutBytes: Buffer.byteLength(stdout), + stderrBytes: Buffer.byteLength(stderr), + exitCode: 3, + truncated: true, + }); + }); + it("rejects output beyond the complete native-command budget", async () => { const runner = new XcrunCommandRunner({}, () => Promise.resolve(ok({ path: process.execPath, sha256: "a".repeat(64) })), @@ -404,6 +435,13 @@ describe("native command failure capture and limits", () => { (result.error.capture?.stdoutBytes ?? 0) + (result.error.capture?.stderrBytes ?? 0), ).toBeLessThanOrEqual(NATIVE_COMMAND_OUTPUT_BUDGET_BYTES); + // The failure carries an excerpt, not the whole retained output budget. + expect(result.error.capture?.stdout.length).toBe( + NATIVE_FAILURE_DIAGNOSTIC_BYTES, + ); + expect(result.error.capture?.stdoutBytes).toBeGreaterThan( + NATIVE_FAILURE_DIAGNOSTIC_BYTES, + ); }); }); @@ -834,6 +872,43 @@ describe("native signature entitlements", () => { }); }); +/** Fail every codesign call nonzero after writing more than one excerpt. */ +class VerboseCodeSignFailureRunner extends FixtureRunner { + override async run(tool: string, arguments_: readonly string[]) { + const result = await super.run(tool, arguments_); + if (!result.ok || tool !== "codesign") return result; + const stderr = "e".repeat(NATIVE_FAILURE_DIAGNOSTIC_BYTES * 2); + return ok({ + ...result.value, + stderr, + stderrBytes: Buffer.byteLength(stderr), + exitCode: 1, + }); + } +} + +describe("native signature failure diagnostics", () => { + it("bounds accepted nonzero codesign output once it becomes a failure", async () => { + directory = await createTestTempDirectory("rea-signature-failure-"); + const executable = join(directory, "fixture"); + await writeFile(executable, "fixture"); + const client = new NativeMacOSProvider( + {}, + new VerboseCodeSignFailureRunner(), + "darwin", + ).createClient(await nativeMachoTargetForFile(executable)); + + const signature = await client.execute("inspect_signature", {}); + + if (signature.ok) throw new Error("A failed codesign run must fail"); + expect(signature.error.capturedOutput).toMatchObject({ + stderr: "e".repeat(NATIVE_FAILURE_DIAGNOSTIC_BYTES), + stderr_bytes: NATIVE_FAILURE_DIAGNOSTIC_BYTES * 2, + truncated: true, + }); + }); +}); + describe("native dispatch metadata error results", () => { it("preserves tagged cancellation and integrity failures", async () => { directory = await createTestTempDirectory("rea-dispatch-errors-"); diff --git a/tests/boundary/signals/recordingDescriptorCleanup.test.ts b/tests/boundary/signals/recordingDescriptorCleanup.test.ts index 3a1927975..f600f0477 100644 --- a/tests/boundary/signals/recordingDescriptorCleanup.test.ts +++ b/tests/boundary/signals/recordingDescriptorCleanup.test.ts @@ -1,3 +1,7 @@ +import { + descriptorFailurePath, + expectRetainedDescriptor, +} from "../../fixtures/descriptorCloseFailure.js"; import { expect, it, onTestFinished, vi } from "vitest"; import { writeFile } from "node:fs/promises"; import type { FileHandle } from "node:fs/promises"; @@ -15,27 +19,22 @@ const cleanup = vi.hoisted(() => ({ })); vi.mock("node:fs/promises", async (importOriginal) => { const filesystem = await importOriginal(); - return { - ...filesystem, - open: async (...args: Parameters) => { - const handle = await filesystem.open(...args); - if (args[0] === cleanup.path) { - cleanup.handles.push(handle); - const close = handle.close.bind(handle); - handle.close = async () => { - if (cleanup.denied && handle.fd >= 0) - throw new Error("Descriptor close deliberately denied"); - await close(); - }; - } - return handle; - }, - }; + const { withDescriptorCloseFailure } = + await import("../../fixtures/descriptorCloseFailure.js"); + return withDescriptorCloseFailure(filesystem, cleanup); }); it .runIf(process.platform === "linux") - .each(["capture", "tool", "limiter", "kernel"])( + .each([ + "capture", + "tool", + "limiter", + "kernel", + "field-tool", + "field-limiter", + "field-kernel", + ])( "retains failed %s descriptors and retries them before further acquisition and shutdown", async (member) => { const directory = await createTestTempDirectory("rea-signal-descriptor-"); @@ -67,14 +66,16 @@ it }, () => PrivateRuntimeRoot.create({ parent: directory }), ); - cleanup.path = - member === "capture" - ? path - : member === "tool" - ? command - : member === "limiter" - ? limiter - : "/proc/self/limits"; + const discovery = member.startsWith("field-"); + const resource = discovery ? member.slice(6) : member; + const operation = discovery + ? "inspect_signal_decoders" + : "decode_signal_recording"; + cleanup.path = descriptorFailurePath(resource, { + capture: path, + tool: command, + limiter, + }); cleanup.denied = true; cleanup.handles = []; onTestFinished(async () => { @@ -84,22 +85,20 @@ it cleanup.path = ""; }); const service = new SignalRecordingService(provider); - const first = await service.decode({ path, decoders: "uart" }); - expect(first).toMatchObject({ - ok: false, - error: { cleanupIncomplete: true, cleanupResources: [cleanup.path] }, - }); - expect(cleanup.handles).toHaveLength(1); - expect(cleanup.handles[0]?.fd).toBeGreaterThanOrEqual(0); + const first = discovery + ? await service.inspectDecoders({ query: "uart" }) + : await service.decode({ path, decoders: "uart" }); + expectRetainedDescriptor(first, cleanup, operation); const launchCount = launches; expect(await service.decode({ path, decoders: "uart" })).toMatchObject({ ok: false, - error: { cleanupIncomplete: true }, + error: { cleanupIncomplete: true, operation }, }); expect(cleanup.handles).toHaveLength(1); expect(launches).toBe(launchCount); await expect(provider.close()).rejects.toMatchObject({ cleanupIncomplete: true, + operation, }); cleanup.denied = false; await provider.close(); diff --git a/tests/boundary/sqlite/sqliteDatabaseInspection.test.ts b/tests/boundary/sqlite/sqliteDatabaseInspection.test.ts index ba94792a5..26fab7d6a 100644 --- a/tests/boundary/sqlite/sqliteDatabaseInspection.test.ts +++ b/tests/boundary/sqlite/sqliteDatabaseInspection.test.ts @@ -130,6 +130,71 @@ it("accounts for JSON escape expansion of a stored declaration in the reply limi ]); }); +it("attributes triggers declared with another ASCII case to the stored table name", async () => { + const path = await createDatabase(` + CREATE TABLE Users(id INTEGER PRIMARY KEY, email TEXT); + CREATE INDEX users_email ON users(email); + CREATE TRIGGER audit_insert AFTER INSERT ON users BEGIN SELECT 1; END; + CREATE TRIGGER audit_delete AFTER DELETE ON USERS BEGIN SELECT 1; END; + CREATE VIEW recent AS SELECT 1; + CREATE TRIGGER recent_insert INSTEAD OF INSERT ON Recent BEGIN SELECT 1; END; + `); + const selected = inspectSqliteDatabaseSnapshot(path, { + path, + table: "Users", + }); + expect(selected.schema).toMatchObject({ + scope: "selected_table", + completeness: "complete", + tables: [{ name: "Users" }], + indexes: [{ name: "users_email", table: "Users" }], + views: [], + }); + expect( + selected.schema.triggers.map(({ name, table }) => ({ name, table })), + ).toEqual([ + { name: "audit_insert", table: "Users" }, + { name: "audit_delete", table: "Users" }, + ]); + const inventory = inspectSqliteDatabaseSnapshot(path, { path }); + expect(inventory.schema.triggers).toEqual([ + { name: "audit_insert", table: "Users" }, + { name: "audit_delete", table: "Users" }, + { name: "recent_insert", table: "recent" }, + ]); +}); + +it.each([ + { + selection: "a differently cased table", + schema: "CREATE TABLE Users(id INTEGER PRIMARY KEY)", + trigger: "AFTER INSERT ON Users", + table: "users", + }, + { + selection: "an exact view", + schema: "CREATE VIEW Recent AS SELECT 1 AS id", + trigger: "INSTEAD OF INSERT ON recent", + table: "Recent", + }, +])( + "refuses $selection before expanding declarations under another spelling", + async ({ schema, trigger, table }) => { + const path = await createDatabase(` + ${schema}; + CREATE TRIGGER expanded ${trigger} BEGIN /*${"\u0001".repeat(3 * 1024 * 1024)}*/ SELECT 1; END; + `); + expect(() => + inspectSqliteDatabaseSnapshot(path, { path, table }), + ).toThrowError( + expect.objectContaining({ + reason: "selection", + message: `Selected ordinary table does not exist with that exact name: ${table}`, + }), + ); + }, +); + it.each(["UTF-8", "UTF-16le", "UTF-16be"])( "preserves exact %s schema text when decoding native bytes", async (encoding) => { diff --git a/tests/boundary/sqlite/sqliteDatabaseProvider.test.ts b/tests/boundary/sqlite/sqliteDatabaseProvider.test.ts index b5adf8297..7805552c9 100644 --- a/tests/boundary/sqlite/sqliteDatabaseProvider.test.ts +++ b/tests/boundary/sqlite/sqliteDatabaseProvider.test.ts @@ -357,6 +357,30 @@ it("does not execute views when selecting rows", async () => { }); }); +nativeTest.each(["absent.db", "absent-parent/absent.db"])( + "classifies a missing %s as an invalid value", + async (name) => { + const workspace = await createTestWorkspace("rea-sqlite-missing-"); + onTestFinished(() => removeTestWorkspace(workspace.root)); + const path = workspace.path(name); + const provider = createProvider(); + onTestFinished(() => provider.close()); + const result = await provider.inspect({ path }); + if (result.ok) throw new Error("Expected a missing database rejection"); + expect(projectAnalysisError(result.error)).toMatchObject({ + details: { + issues: [ + { + path: ["path"], + reason: "invalid_value", + message: `Selected SQLite database does not exist: ${path}`, + }, + ], + }, + }); + }, +); + it("attributes table and column selection failures to their own arguments", async () => { const { database } = await createFixture(); const provider = createProvider(); diff --git a/tests/composition/idaProvider.test.ts b/tests/composition/idaProvider.test.ts index 9a702bb45..1fd17171d 100644 --- a/tests/composition/idaProvider.test.ts +++ b/tests/composition/idaProvider.test.ts @@ -1,6 +1,7 @@ import { rm, writeFile } from "node:fs/promises"; import { join } from "node:path"; import { afterEach, describe, expect, it } from "vitest"; +import { projectAnalysisError } from "../../src/domain/analysisErrorProjection.js"; import { IdaProvider } from "../../src/ida/IdaProvider.js"; import { createIdaTarget, RecordingIdaMcp } from "../fixtures/idaMcp.js"; import { createTestBinarySession } from "../fixtures/binarySession.js"; @@ -21,6 +22,73 @@ afterEach(async () => { await rm(root, { recursive: true, force: true }); }); +const reobservesExternallyMutableState = async (): Promise => { + const { root, target } = await createIdaTarget(); + roots.push(root); + const path = join(root, "registration.json"); + await writeFile(path, JSON.stringify({ command: "fixture" })); + const config = parseConfig({ REA_IDA_MCP_CONFIG: path }); + if (!config.ok) throw config.error; + const producer = new RecordingIdaMcp(target); + const session = createTestBinarySession( + new IdaProvider(config.value, {}, () => producer), + ); + expect((await session.open(target.path)).ok).toBe(true); + const query = { procedure: "main", document: target.path }; + const first = await session.execute("procedure_pseudo_code", query); + expect(first.ok && first.value.result).toContain("42"); + const snapshot = session.exportAnalysisSnapshot(); + if (!snapshot.ok) throw snapshot.error; + expect(snapshot.value.entries).toEqual([]); + const profile = session.analysisProfile(); + if (profile === undefined) throw new Error("Missing profile"); + const cached = createAnalysisExecution( + "stale output", + IDA_PROVIDER_IDENTITY, + { + analysisProfile: profile, + subject: { + path: target.path, + sha256: target.sha256, + format: target.format, + ...(target.architecture === undefined + ? {} + : { architecture: target.architecture }), + }, + }, + ); + snapshot.value.entries.push( + createAnalysisSnapshotEntry({ + target: snapshotTarget(target), + binding: snapshotBinding(profile), + operation: "procedure_pseudo_code", + parameters: query, + execution: cached, + }), + ); + snapshot.value.evidence_bundle = createEvidenceBundle([ + ...snapshot.value.evidence_bundle.records, + createEvidence(target, IDA_PROVIDER_IDENTITY, { + operation: "procedure_pseudo_code", + parameters: query, + result: cached.result, + rawResult: cached.rawResult, + analysisProfile: profile, + }), + ]); + expect(session.importAnalysisSnapshot(snapshot.value).ok).toBe(true); + expect(session.allowsSnapshotReplay("procedure_pseudo_code")).toBe(false); + expect(session.allowsSnapshotReplay("binary_overview")).toBe(false); + producer.pseudocode = "int main(void) { return 99; }"; + const second = await session.execute("procedure_pseudo_code", query); + expect(second.ok && second.value.result).toContain("99"); + expect( + producer.calls.filter(({ name }) => name === "decompile_function"), + ).toHaveLength(2); + await session.close(); + expect(producer.closes).toBe(1); +}; + describe("IDA provider composition", () => { it("discovers without launching and commits adapter semantics without persisting credentials", async () => { const { root, target } = await createIdaTarget(); @@ -58,72 +126,10 @@ describe("IDA provider composition", () => { .find(({ operation }) => operation === "procedure_callers")?.available, ).toBe(false); }); - it("re-observes externally mutable state even when a document is explicit and a snapshot is imported", async () => { - const { root, target } = await createIdaTarget(); - roots.push(root); - const path = join(root, "registration.json"); - await writeFile(path, JSON.stringify({ command: "fixture" })); - const config = parseConfig({ REA_IDA_MCP_CONFIG: path }); - if (!config.ok) throw config.error; - const producer = new RecordingIdaMcp(target); - const session = createTestBinarySession( - new IdaProvider(config.value, {}, () => producer), - ); - expect((await session.open(target.path)).ok).toBe(true); - const query = { procedure: "main", document: target.path }; - const first = await session.execute("procedure_pseudo_code", query); - expect(first.ok && first.value.result).toContain("42"); - const snapshot = session.exportAnalysisSnapshot(); - if (!snapshot.ok) throw snapshot.error; - expect(snapshot.value.entries).toEqual([]); - const profile = session.analysisProfile(); - if (profile === undefined) throw new Error("Missing profile"); - const cached = createAnalysisExecution( - "stale output", - IDA_PROVIDER_IDENTITY, - { - analysisProfile: profile, - subject: { - path: target.path, - sha256: target.sha256, - format: target.format, - ...(target.architecture === undefined - ? {} - : { architecture: target.architecture }), - }, - }, - ); - snapshot.value.entries.push( - createAnalysisSnapshotEntry({ - target: snapshotTarget(target), - binding: snapshotBinding(profile), - operation: "procedure_pseudo_code", - parameters: query, - execution: cached, - }), - ); - snapshot.value.evidence_bundle = createEvidenceBundle([ - ...snapshot.value.evidence_bundle.records, - createEvidence(target, IDA_PROVIDER_IDENTITY, { - operation: "procedure_pseudo_code", - parameters: query, - result: cached.result, - rawResult: cached.rawResult, - analysisProfile: profile, - }), - ]); - expect(session.importAnalysisSnapshot(snapshot.value).ok).toBe(true); - expect(session.allowsSnapshotReplay("procedure_pseudo_code")).toBe(false); - expect(session.allowsSnapshotReplay("binary_overview")).toBe(false); - producer.pseudocode = "int main(void) { return 99; }"; - const second = await session.execute("procedure_pseudo_code", query); - expect(second.ok && second.value.result).toContain("99"); - expect( - producer.calls.filter(({ name }) => name === "decompile_function"), - ).toHaveLength(2); - await session.close(); - expect(producer.closes).toBe(1); - }); + it( + "re-observes externally mutable state even when a document is explicit and a snapshot is imported", + reobservesExternallyMutableState, + ); it("rejects unsupported target kinds and reports incomplete upstream profiles on first analysis", async () => { const { root, target } = await createIdaTarget(); roots.push(root); @@ -157,12 +163,21 @@ describe("IDA provider composition", () => { expect(analyzed).toMatchObject({ ok: false, error: { - _tag: "AnalysisProtocolError", - message: expect.stringContaining( + _tag: "ProviderAdapterError", + userMessage: expect.stringContaining( "headless compatibility profile is missing tools: idb_open", ), }, }); + if (analyzed.ok) throw new Error("Expected incompatible IDA profile"); + expect(projectAnalysisError(analyzed.error).details).toMatchObject({ + provider_id: "ida", + diagnostics: { + mode: "headless", + expected_profile: "database-supervisor", + missing_tools: expect.arrayContaining(["idb_open"]), + }, + }); expect(producer.connects).toBe(1); expect((await session.close()).ok).toBe(true); expect(producer.closes).toBe(1); diff --git a/tests/conformance/android/ReaJadxMetadataRegression.java b/tests/conformance/android/ReaJadxMetadataRegression.java index a0cbd9364..7e21dd4fc 100644 --- a/tests/conformance/android/ReaJadxMetadataRegression.java +++ b/tests/conformance/android/ReaJadxMetadataRegression.java @@ -60,6 +60,78 @@ public class ReaJadxMetadataRegression { return apk; } + private static void verifyDirectInvokes(SessionHolder holder, SessionConfig config, + Constructor constructor, Method handle, Path sources, Path output) throws Exception { + SmaliOptions options = new SmaliOptions(); + options.apiLevel = 28; + options.jobs = 1; + options.outputDexFile = output.resolve("direct-invokes.dex").toString(); + require(Smali.assemble(options, List.of( + sources.resolve("Caller.smali").toString(), sources.resolve("Target.smali").toString())), + "Direct-invoke fixture assembly failed"); + Path apk = output.resolve("direct-invokes.apk"); + try (ZipOutputStream zip = new ZipOutputStream(Files.newOutputStream(apk))) { + zip.putNextEntry(new ZipEntry("classes.dex")); + Files.copy(Path.of(options.outputDexFile), zip); + zip.closeEntry(); + } + BuildersKt.runBlocking(EmptyCoroutineContext.INSTANCE, + (scope, continuation) -> holder.unload(continuation)); + BuildersKt.runBlocking(EmptyCoroutineContext.INSTANCE, + (scope, continuation) -> holder.load(apk.toString(), config, continuation)); + JsonObject refs = call(handle, constructor.newInstance(holder), "get_xrefs_to_method", + methodArguments("fixture.Target", "mark")); + checkDirectInvokes(refs, "classes.dex"); + + SmaliOptions targetOptions = new SmaliOptions(); + targetOptions.apiLevel = 28; + targetOptions.jobs = 1; + targetOptions.outputDexFile = output.resolve("direct-target.dex").toString(); + SmaliOptions callerOptions = new SmaliOptions(); + callerOptions.apiLevel = 28; + callerOptions.jobs = 1; + callerOptions.outputDexFile = output.resolve("direct-caller.dex").toString(); + require(Smali.assemble(targetOptions, List.of(sources.resolve("Target.smali").toString())), + "Target DEX assembly failed"); + require(Smali.assemble(callerOptions, List.of(sources.resolve("Caller.smali").toString())), + "Caller DEX assembly failed"); + Path splitApk = output.resolve("direct-invokes-multidex.apk"); + try (ZipOutputStream zip = new ZipOutputStream(Files.newOutputStream(splitApk))) { + zip.putNextEntry(new ZipEntry("classes.dex")); + Files.copy(Path.of(targetOptions.outputDexFile), zip); + zip.closeEntry(); + zip.putNextEntry(new ZipEntry("classes2.dex")); + Files.copy(Path.of(callerOptions.outputDexFile), zip); + zip.closeEntry(); + } + BuildersKt.runBlocking(EmptyCoroutineContext.INSTANCE, + (scope, continuation) -> holder.unload(continuation)); + BuildersKt.runBlocking(EmptyCoroutineContext.INSTANCE, + (scope, continuation) -> holder.load(splitApk.toString(), config, continuation)); + JsonObject crossDex = call(handle, constructor.newInstance(holder), "get_xrefs_to_method", + methodArguments("fixture.Target", "mark")); + checkDirectInvokes(crossDex, "classes2.dex"); + System.out.println("DIRECT_INVOKE_REGRESSION_PASS nodes=1 offsets=0,3 cross_dex=true"); + } + + private static void checkDirectInvokes(JsonObject refs, String dexEntry) { + require(refs.get("total").getAsInt() == 1, "JADX using-node count changed"); + JsonArray sites = refs.getAsJsonArray("items").get(0).getAsJsonObject() + .getAsJsonArray("direct_invokes"); + require(sites.size() == 2, "Raw DEX invoke count differs from the fixture: " + sites); + for (int i = 0; i < sites.size(); i++) { + JsonObject site = sites.get(i).getAsJsonObject(); + require(site.get("dex_entry").getAsString().equals(dexEntry), "Wrong DEX entry"); + require(site.get("source_raw_method_id").getAsString().equals("fixture.Caller.twice()V"), + "Wrong raw source method"); + require(site.get("dex_code_unit_offset").getAsInt() == i * 3, "Wrong DEX offset"); + require(site.get("opcode").getAsString().equals("INVOKE_STATIC"), "Wrong opcode"); + } + require(sites.get(0).getAsJsonObject().get("dex_method_index").getAsInt() + == sites.get(1).getAsJsonObject().get("dex_method_index").getAsInt(), + "Two invokes did not reference the same DEX method ID"); + } + private static JsonObject methodArguments(String className, String methodName) { JsonObject value = arguments(className); value.addProperty("method_name", methodName); @@ -152,13 +224,20 @@ public class ReaJadxMetadataRegression { JsonObject componentArgs = arguments("androidx.activity.ComponentActivity"); JsonObject componentSummary = call(handle, bridge, "get_class_summary", componentArgs); List signatures = new ArrayList<>(); + List descriptors = new ArrayList<>(); for (JsonElement entry : componentSummary.getAsJsonArray("methods")) { JsonObject method = entry.getAsJsonObject(); - if (method.get("name").getAsString().equals("setContentView")) + if (method.get("name").getAsString().equals("setContentView")) { signatures.add(method.get("signature").getAsString()); + descriptors.add(method.get("dex_descriptor").getAsString()); + } } require(signatures.size() == 3, "Fixture must contain three setContentView overloads"); + require(descriptors.equals(List.of("(I)V", "(Landroid/view/View;)V", + "(Landroid/view/View;Landroid/view/ViewGroup$LayoutParams;)V")), + "Frozen DEX prototypes did not identify the three overloads: " + descriptors); String[] declarations = { "setContentView(int ", "setContentView(View ", "setContentView(View view, ViewGroup.LayoutParams " }; + List referenceCounts = new ArrayList<>(); for (int index = 0; index < signatures.size(); index++) { JsonObject methodArgs = arguments("androidx.activity.ComponentActivity"); methodArgs.addProperty("method_name", "setContentView"); @@ -169,12 +248,35 @@ public class ReaJadxMetadataRegression { "Overload envelope mismatch"); require(first.get("mode").getAsString().equals("java"), "Fixture did not produce Java"); require(first.get("body").getAsString().contains(declarations[index]), "Wrong overload: " + signatures.get(index)); + JsonObject references = call(handle, bridge, "get_xrefs_to_method", methodArgs); + require(references.get("overload_index").getAsInt() == index, + "Reference result did not preserve selected overload index"); + require(references.get("reported_signature").getAsString().equals(signatures.get(index)), + "Reference result did not preserve selected overload signature"); + if (index == 0) { + JsonArray invokes = references.getAsJsonArray("items").get(0).getAsJsonObject() + .getAsJsonArray("direct_invokes"); + require(invokes.size() == 1, "Selected overload must have one raw direct invoke"); + JsonObject invoke = invokes.get(0).getAsJsonObject(); + require(invoke.get("source_raw_method_id").getAsString() + .equals("androidx.activity.ComponentActivity.onCreate(Landroid/os/Bundle;)V"), + "Wrong source method for overloaded target"); + require(invoke.get("dex_code_unit_offset").getAsInt() == 37 + && invoke.get("opcode").getAsString().equals("INVOKE_VIRTUAL"), + "Wrong direct invoke for selected overloaded target"); + } + referenceCounts.add(references.get("count").getAsInt()); } + require(referenceCounts.equals(List.of(1, 0, 0)), + "Incoming references were not isolated to the selected overload: " + referenceCounts); require(componentSummary.equals(call(handle, bridge, "get_class_summary", componentArgs)), "Frozen summary changed after code generation"); require(apiSummary.equals(call(handle, bridge, "get_class_summary", apiArgs)), "Unrelated frozen summary changed"); - System.out.println("METADATA_REGRESSION_PASS classes=" + states.size() + " overloads=" + signatures.size()); + System.out.println("METADATA_REGRESSION_PASS classes=" + states.size() + " overloads=" + signatures.size() + + " reference_counts=" + referenceCounts); verifyInitializers(holder, config, constructor, handle, initializerFixture(args[1], args[2])); + verifyDirectInvokes(holder, config, constructor, handle, + Path.of(args[3]), Path.of(args[2])); } finally { BuildersKt.runBlocking(EmptyCoroutineContext.INSTANCE, (scope, continuation) -> holder.unload(continuation)); diff --git a/tests/conformance/ghidra/mips-checks.mjs b/tests/conformance/ghidra/mips-checks.mjs index 3e6af3733..e5c6d335c 100644 --- a/tests/conformance/ghidra/mips-checks.mjs +++ b/tests/conformance/ghidra/mips-checks.mjs @@ -160,7 +160,89 @@ export async function inspectMipsReadelf(path, byteOrder, floatAbi = "hard") { }; } -/** Check effective Ghidra observations, not the requested analysis profile. */ +/** Independent GNU readelf interpretation for the recordless legacy fixture. */ +export async function inspectLegacyMipsReadelf(path, byteOrder) { + const command = process.env.REA_MIPS_READELF ?? "readelf"; + const options = { + env: { ...process.env, LC_ALL: "C" }, + timeout: 10000, + maxBuffer: 1024 * 1024, + }; + const version = await execute(command, ["--version"], options); + assert.match( + version.stdout, + /^GNU readelf\b/u, + "This lane requires GNU readelf", + ); + const header = await execute(command, ["-h", "-W", path], options); + assert.equal( + header.stderr.trim(), + "", + `readelf diagnostics: ${header.stderr}`, + ); + const field = (expression, label) => { + const matches = [...header.stdout.matchAll(expression)]; + assert.equal(matches.length, 1, `Missing or ambiguous readelf ${label}`); + return matches[0][1].trim(); + }; + assert.equal(field(/^\s*Class:\s*(.+)$/gmu, "class"), "ELF32"); + assert.equal( + field(/^\s*Data:.*?,\s*(little|big) endian\s*$/gmu, "byte order"), + byteOrder, + ); + assert.match(field(/^\s*Type:\s*(.+)$/gmu, "type"), /^EXEC\b/u); + assert.match(field(/^\s*Machine:\s*(.+)$/gmu, "machine"), /^MIPS\b/u); + const flags = field(/^\s*Flags:\s*(0x[\da-f]+)\b.*$/gmu, "flags"); + assert.equal( + Number(flags), + 0x50000001, + "Legacy fixture must carry EF_MIPS_ARCH_32 with an unspecified ABI field", + ); + const abi = await execute(command, ["-A", path], options); + assert.equal(abi.stderr.trim(), "", `readelf diagnostics: ${abi.stderr}`); + // -A also prints unrelated MIPS facts such as the static GOT; only the ABI + // flags declaration itself must be gone from the recordless fixture. + assert.doesNotMatch( + abi.stdout, + /MIPS ABI Flags Version/u, + "GNU readelf still observes an ABI flags declaration in the legacy fixture", + ); + const symbols = await execute(command, ["-s", "-W", path], options); + assert.equal( + symbols.stderr.trim(), + "", + `readelf diagnostics: ${symbols.stderr}`, + ); + const parsed = {}; + for (const name of ["rea_mips_entry", "rea_mips_probe"]) { + const matches = [ + ...symbols.stdout.matchAll( + new RegExp( + `^\\s*\\d+:\\s+([\\da-f]+)\\s+\\d+\\s+FUNC\\s+GLOBAL\\s+\\S+\\s+\\d+\\s+${name}\\s*$`, + "gmu", + ), + ), + ]; + assert.equal( + matches.length, + 1, + `Missing or ambiguous readelf symbol ${name}`, + ); + parsed[name] = `0x${BigInt(`0x${matches[0][1]}`).toString(16)}`; + } + return { + flags: Number(flags), + limitations: [], + symbols: parsed, + command, + version: version.stdout.split("\n")[0], + raw: { + header: header.stdout, + abi: abi.stdout, + symbols: symbols.stdout, + }, + }; +} export function assertMipsLoadedImage(observations, byteOrder, sha256) { assert.equal(observations.language_id, language(byteOrder)); assert.equal(observations.compiler_spec_id, "default"); diff --git a/tests/conformance/ghidra/mips-fixture.mjs b/tests/conformance/ghidra/mips-fixture.mjs index b88669e31..1d19aaa57 100644 --- a/tests/conformance/ghidra/mips-fixture.mjs +++ b/tests/conformance/ghidra/mips-fixture.mjs @@ -1,6 +1,6 @@ import { execFile } from "node:child_process"; import { createHash } from "node:crypto"; -import { readFile } from "node:fs/promises"; +import { readFile, writeFile } from "node:fs/promises"; import { join } from "node:path"; import { fileURLToPath } from "node:url"; import { promisify } from "node:util"; @@ -12,11 +12,13 @@ export async function buildMipsFixture( directory, byteOrder, floatAbi = "hard", + options = {}, ) { if (byteOrder !== "little" && byteOrder !== "big") throw new Error("MIPS fixture byte order must be little or big"); if (floatAbi !== "hard" && floatAbi !== "soft") throw new Error("MIPS fixture float ABI must be hard or soft"); + const legacy = options.legacy === true; const soft = floatAbi === "soft"; const target = byteOrder === "little" ? "mipsel-linux-gnu" : "mips-linux-gnu"; const source = fileURLToPath(new URL("../c/mips.c", import.meta.url)); @@ -24,10 +26,13 @@ export async function buildMipsFixture( new URL("../c/mips-soft-float.c", import.meta.url), ); const probe = fileURLToPath(new URL("./mips-probe.S", import.meta.url)); - const path = join(directory, `${target}${soft ? "-soft-float" : ""}.elf`); + const path = join( + directory, + `${target}${soft ? "-soft-float" : ""}${legacy ? "-legacy" : ""}.elf`, + ); const args = [ `--target=${target}`, - "-march=mips32r2", + `-march=${legacy ? "mips32" : "mips32r2"}`, "-mabi=32", ...(soft ? ["-msoft-float"] : []), "-mno-abicalls", @@ -45,8 +50,88 @@ export async function buildMipsFixture( "-o", path, ]; + const compiled = await compileMipsFixture(path, byteOrder, 2, args, { + expectedFlags: legacy ? 0x50001001 : 0x70001001, + }); + // Legacy variant: retype the ABI record declarations out of the image and + // zero the header ABI field, matching pre-ABI-flags 2000s firmware toolchains + // (their compilers predate the record; the header leaves the ABI unspecified). + let attested = compiled; + if (legacy) { + const raw = await readFile(path); + const write16 = (offset, value) => + byteOrder === "little" + ? raw.writeUInt16LE(value, offset) + : raw.writeUInt16BE(value, offset); + const write32 = (offset, value) => + byteOrder === "little" + ? raw.writeUInt32LE(value, offset) + : raw.writeUInt32BE(value, offset); + const read16 = (offset) => + byteOrder === "little" + ? raw.readUInt16LE(offset) + : raw.readUInt16BE(offset); + const read32 = (offset) => + byteOrder === "little" + ? raw.readUInt32LE(offset) + : raw.readUInt32BE(offset); + const flags = read32(36); + if ((flags & 0xf0000000) !== 0x50000000) + throw new Error("Legacy fixture did not build as EF_MIPS_ARCH_32"); + write32(36, flags & ~0x0000f000); + const programOffset = read32(28); + const programCount = read16(44); + if (read16(42) !== 32) + throw new Error("Legacy fixture has non-standard program headers"); + for (let index = 0; index < programCount; index += 1) { + const offset = programOffset + index * 32; + if (read32(offset) === 0x70000003) write32(offset, 0); + } + const sectionOffset = read32(32); + const sectionCount = read16(48); + if (read16(46) !== 40) + throw new Error("Legacy fixture has non-standard section headers"); + for (let index = 0; index < sectionCount; index += 1) { + const offset = sectionOffset + index * 40; + if (read32(offset + 4) === 0x7000002a) write32(offset + 4, 1); + } + // GNU readelf resolves the record by section name, not type, so a retyped + // header alone keeps a named declaration. Replace the name in the string + // table in place, preserving every remaining name and offset. + const stringTableIndex = read16(50); + const stringTableOffset = + stringTableIndex < sectionCount + ? read32(sectionOffset + stringTableIndex * 40 + 16) + : -1; + const named = + stringTableOffset === -1 + ? -1 + : raw.indexOf( + Buffer.from(".MIPS.abiflags", "ascii"), + stringTableOffset, + ); + if (named === -1) + throw new Error("Legacy fixture lacks the expected ABI section name"); + raw.write("x.rea.no.flags", named, "ascii"); + await writeFile(path, raw); + const patched = await readFile(path); + const patchedFlags = + byteOrder === "little" + ? patched.readUInt32LE(36) + : patched.readUInt32BE(36); + if (patchedFlags !== 0x50000001) + throw new Error( + "Legacy fixture did not end with the recordless o32 declaration", + ); + attested = { + ...compiled, + flags: patchedFlags, + sha256: createHash("sha256").update(patched).digest("hex"), + }; + } return { - ...(await compileMipsFixture(path, byteOrder, 2, args)), + ...attested, + legacy, float_abi: floatAbi, source_sha256: createHash("sha256") .update(await readFile(source)) @@ -108,7 +193,13 @@ export async function buildMipsRelocatableFixture( } /** Compile and attest the shared ELF32/o32/MIPS32r2 fixture boundary. */ -async function compileMipsFixture(path, byteOrder, objectType, args) { +async function compileMipsFixture( + path, + byteOrder, + objectType, + args, + options = {}, +) { const compiler = process.env.REA_MIPS_CLANG ?? "clang"; const version = await execute(compiler, ["--version"], { timeout: 10000 }); await execute(compiler, args, { timeout: 60000 }); @@ -128,7 +219,7 @@ async function compileMipsFixture(path, byteOrder, objectType, args) { bytes[5] !== (byteOrder === "little" ? 1 : 2) || read16(16) !== objectType || read16(18) !== 8 || - read32(36) !== 0x70001001 + read32(36) !== (options.expectedFlags ?? 0x70001001) ) throw new Error( "Compiler did not produce the declared ELF32/o32/MIPS32r2 fixture", diff --git a/tests/conformance/readmeConsistency.test.ts b/tests/conformance/readmeConsistency.test.ts index 90f8e5886..18d5eeae5 100644 --- a/tests/conformance/readmeConsistency.test.ts +++ b/tests/conformance/readmeConsistency.test.ts @@ -4,26 +4,6 @@ import { resolve } from "node:path"; import { describe, expect, it } from "vitest"; import { PRODUCT_IDENTITY } from "../../src/identity.js"; -const translatedReadmes = [ - "README_zh.md", - "README_zh-TW.md", - "README_ja.md", - "README_ko.md", - "README_tr.md", - "README_ru.md", - "README_vi.md", - "README_th.md", - "README_de.md", - "README_es.md", - "README_fr.md", - "README_uk.md", - "README_pl.md", - "README_pt-BR.md", - "README_ar.md", - "README_fa.md", - "README_id.md", -] as const; - const jsonExamples = (content: string): unknown[] => [...content.matchAll(/```json\s*([\s\S]*?)```/gu)].map((match): unknown => JSON.parse(match[1] ?? ""), @@ -64,6 +44,14 @@ describe("onboarding documentation product facts", () => { }); it("keeps translated onboarding commands aligned with the English examples", async () => { + const inventory = JSON.parse( + await readFile(resolve("website/locales/languages.json"), "utf8"), + ) as { code: string; readme: string }[]; + const translatedReadmes = inventory + .filter((language) => language.code !== "en") + .map((language) => language.readme); + expect(translatedReadmes).toContain("docs/readme/README_it.md"); + expect(translatedReadmes).toHaveLength(18); const english = shellExamples(await readFile(resolve("README.md"), "utf8")); for (const path of translatedReadmes) expect( @@ -71,4 +59,21 @@ describe("onboarding documentation product facts", () => { path, ).toEqual(english); }); + + it("preserves Persian joiners in links to GitHub-rendered headings", async () => { + const persian = await readFile(resolve("docs/readme/README_fa.md"), "utf8"); + // Confirmed against GitHub's rendered README, which retains U+200C. + for (const heading of [ + "REA چگونه کار می‌کند", + "چه چیزهایی را می‌توان تحلیل کرد", + "نمونه‌های عملی", + "پرسش‌های متداول", + "به‌روزرسانی REA", + ]) { + expect(persian).toContain(`## ${heading}`); + expect(persian).toContain( + `](#${heading.toLowerCase().replaceAll(" ", "-")})`, + ); + } + }); }); diff --git a/tests/evaluation/model-evals/knownAnswerEvaluation.test.ts b/tests/evaluation/model-evals/knownAnswerEvaluation.test.ts index cbe59dd7b..daed23cac 100644 --- a/tests/evaluation/model-evals/knownAnswerEvaluation.test.ts +++ b/tests/evaluation/model-evals/knownAnswerEvaluation.test.ts @@ -5,6 +5,16 @@ import { toolContract } from "../../../src/contracts/toolContracts.js"; import { createEvidence, type Evidence } from "../../../src/domain/evidence.js"; import type { JsonValue } from "../../../src/domain/jsonValue.js"; import { evaluateCodexEvents } from "../../../src/evaluation/CodexAgentEval.js"; +import { + agentFixtureAnswerInstructions, + agentFixtureClaims, +} from "../../../src/evaluation/AgentEvaluationFixtures.js"; +import { inspectManagedArtifactBytes } from "../../../src/dotnet/ManagedArtifactInspector.js"; +import { inspectManagedMembersBytes } from "../../../src/dotnet/ManagedMemberInspector.js"; +import { + buildManagedPeFixture, + managedPeFixtureTarget, +} from "../../../src/dotnet/ManagedPe.fixture.js"; import type { FixtureClaimExpectation } from "../../../src/evaluation/KnownAnswerEvaluation.js"; import { ToolResultDelivery } from "../../../src/server/toolResult.js"; @@ -33,7 +43,13 @@ const createObservation = ( const resultFor = (evidence: Evidence) => delivery.toEvidenceToolResult( evidence, - toolContract("analyze_javascript_application"), + toolContract( + evidence.operation === "inspect_managed_artifact" + ? "inspect_managed_artifact" + : evidence.operation === "inspect_managed_members" + ? "inspect_managed_members" + : "analyze_javascript_application", + ), undefined, ); const delivered = ( @@ -95,6 +111,336 @@ const assess = ( ); const answer = (claims: readonly unknown[]) => JSON.stringify({ claims }); +// Synthetic transcript cases exercise grading, not an SDK-built assembly oracle. +const managedTranscript = () => { + const bytes = buildManagedPeFixture({ + typeNamespace: "AgentEval", + typeName: "Program", + methodName: "Main", + }); + const target = managedPeFixtureTarget(bytes, "/targets/AgentEval.dll"); + const artifact = inspectManagedArtifactBytes(bytes, target); + const members = inspectManagedMembersBytes(bytes, target); + const program = members.types[0]; + if (program === undefined) throw new Error("Fixture type missing"); + const result = { + ...members, + types: [ + program, + { + ...program, + token: "0x02000002", + name: "Profile", + full_name: "AgentEval.Profile", + }, + ], + }; + const rubric = agentFixtureClaims("managed", { + javascript: "", + javascriptSha256: "", + javascriptShapeLeft: "", + javascriptShapeRight: "", + managed: target.path, + managedSha256: target.sha256, + }); + if (rubric === undefined) throw new Error("Managed rubric missing"); + const evidenceFor = ( + operation: string, + value: JsonValue, + overrides: Record = {}, + ) => + createEvidence(target, provider, { + operation, + parameters: {}, + result: value, + confidence: "observed", + authority: "shipped-artifact", + ...overrides, + }); + const artifactEvidence = evidenceFor("inspect_managed_artifact", artifact); + const memberEvidence = evidenceFor("inspect_managed_members", result); + const textFor = (evidence: Evidence) => + answer([ + { + id: "public_types", + value: ["AgentEval.Profile", "AgentEval.Program"], + evidence_id: evidence.evidence_id, + confidence: "observed", + authority: "shipped-artifact", + }, + { + id: "entry_point", + value: { declaring_type: "AgentEval.Program", name: "Main" }, + evidence_id: evidence.evidence_id, + confidence: "observed", + authority: "shipped-artifact", + }, + ]); + const grade = ( + events = [delivered(artifactEvidence), delivered(memberEvidence)], + text = textFor(memberEvidence), + ) => + evaluateCodexEvents([...events, final(text)], "inspect_managed_artifact", { + fixtureClaims: rubric, + }); + return { + artifact, + result, + artifactEvidence, + memberEvidence, + evidenceFor, + textFor, + grade, + }; +}; + +describe("managed known-answer fixture grading", () => { + it("grades public types independent of row order and resolves the header token", () => { + expect(managedTranscript().grade().factualCorrectness).toBe("passed"); + }); + + it("does not pin compiler-local entry tokens or MVIDs", () => { + const fixture = managedTranscript(); + const artifact = structuredClone(fixture.artifact); + const members = structuredClone(fixture.result); + if ( + artifact.pe.cli === null || + artifact.module === null || + members.module === null || + members.methods[0] === undefined + ) + throw new Error("Fixture identity missing"); + artifact.pe.cli.entry_point.value = "0x06000009"; + members.methods[0].token = "0x06000009"; + const mvid = "11111111-2222-3333-4444-555555555555"; + artifact.module.mvid = mvid; + members.module.mvid = mvid; + members.identity_scope.requires_mvid = mvid; + members.types.reverse(); + const artifactEvidence = fixture.evidenceFor( + "inspect_managed_artifact", + artifact, + ); + const memberEvidence = fixture.evidenceFor( + "inspect_managed_members", + members, + ); + expect( + fixture.grade( + [delivered(artifactEvidence), delivered(memberEvidence)], + fixture.textFor(memberEvidence), + ).factualCorrectness, + ).toBe("passed"); + }); + + it.each([ + "wrong-token", + "native-rva", + "mvid", + "null-mvid", + "digest", + "partial", + ])("rejects an unsupported artifact entry point: %s", (kind) => { + const fixture = managedTranscript(); + const artifact = structuredClone(fixture.artifact); + if (artifact.pe.cli === null || artifact.module === null) + throw new Error("Fixture identity missing"); + if (kind === "wrong-token") + artifact.pe.cli.entry_point.value = "0x06000002"; + if (kind === "native-rva") artifact.pe.cli.entry_point.kind = "native-rva"; + if (kind === "mvid") + artifact.module.mvid = "11111111-2222-3333-4444-555555555555"; + if (kind === "null-mvid") artifact.module.mvid = null; + if (kind === "digest") artifact.artifact.sha256 = "b".repeat(64); + if (kind === "partial") artifact.coverage.state = "partial"; + const evidence = fixture.evidenceFor("inspect_managed_artifact", artifact); + expect( + fixture.grade([delivered(evidence), delivered(fixture.memberEvidence)]) + .factualCorrectness, + ).toBe("failed"); + }); +}); + +describe("managed inventory scope", () => { + it.each(["selection", "parameters", "both"])( + "rejects selected public-type inventories while retaining entry-point grounding: %s", + (kind) => { + const fixture = managedTranscript(); + const typeNames = ["AgentEval.Profile", "AgentEval.Program"]; + const members = { + ...fixture.result, + selection: + kind === "parameters" + ? null + : { type_names: typeNames, unmatched_type_names: [] }, + }; + const evidence = fixture.evidenceFor("inspect_managed_members", members, { + parameters: kind === "selection" ? {} : { type_names: typeNames }, + }); + expect( + fixture.grade( + [delivered(fixture.artifactEvidence), delivered(evidence)], + fixture.textFor(evidence), + ), + ).toMatchObject({ + factualCorrectness: "failed", + factualAssessment: { + claims: [ + { id: "public_types", status: "failed" }, + { id: "entry_point", status: "passed" }, + ], + }, + }); + }, + ); +}); + +describe("managed member grounding", () => { + it.each([ + "extra-public-type", + "missing-public-type", + "duplicate-method", + "decoy-main", + "scope-digest", + "scope-mvid", + "null-scope", + "partial", + "unavailable", + ])("rejects unsupported member facts: %s", (kind) => { + const fixture = managedTranscript(); + const members = structuredClone(fixture.result); + const method = members.methods[0]; + const type = members.types[0]; + if (method === undefined || type === undefined) + throw new Error("Fixture members missing"); + if (kind === "extra-public-type") + members.types.push({ + ...type, + name: "Extra", + full_name: "AgentEval.Extra", + }); + if (kind === "missing-public-type") members.types.pop(); + if (kind === "duplicate-method") members.methods.push(method); + if (kind === "decoy-main") { + method.name = "Other"; + members.methods.push({ ...method, token: "0x06000002", name: "Main" }); + } + if (kind === "scope-digest") + members.identity_scope.requires_artifact_sha256 = "b".repeat(64); + if (kind === "scope-mvid") + members.identity_scope.requires_mvid = + "11111111-2222-3333-4444-555555555555"; + if (kind === "null-scope") members.identity_scope.requires_mvid = null; + if (kind === "partial") members.metadata.status = "partial"; + if (kind === "unavailable") members.coverage.state = "unavailable"; + const evidence = fixture.evidenceFor("inspect_managed_members", members); + expect( + fixture.grade( + [delivered(fixture.artifactEvidence), delivered(evidence)], + fixture.textFor(evidence), + ).factualCorrectness, + ).toBe("failed"); + }); + + it("rejects distinct authenticated matching artifact deliveries", () => { + const fixture = managedTranscript(); + const secondArtifact = fixture.evidenceFor( + "inspect_managed_artifact", + fixture.artifact, + { parameters: { path: "/targets/AgentEval.dll" } }, + ); + expect(secondArtifact.evidence_id).not.toBe( + fixture.artifactEvidence.evidence_id, + ); + const member = delivered(fixture.memberEvidence); + expect( + fixture.grade([delivered(secondArtifact), member]).factualCorrectness, + ).toBe("passed"); + expect( + fixture.grade([ + delivered(fixture.artifactEvidence), + delivered(secondArtifact), + member, + ]).factualCorrectness, + ).toBe("failed"); + }); + + it("accepts identical repeated artifact deliveries", () => { + const fixture = managedTranscript(); + expect( + fixture.grade([ + delivered(fixture.artifactEvidence), + delivered(fixture.artifactEvidence), + delivered(fixture.memberEvidence), + ]).factualCorrectness, + ).toBe("passed"); + }); + + it("requires companion delivery and rejects conflicting or failed artifact deliveries", () => { + const fixture = managedTranscript(); + const member = delivered(fixture.memberEvidence); + expect(fixture.grade([member]).factualCorrectness).toBe("failed"); + expect( + fixture.grade([ + delivered(fixture.artifactEvidence, { status: "failed" }), + member, + ]).factualCorrectness, + ).toBe("failed"); + const conflicting = { + ...fixture.artifactEvidence, + subject: + fixture.memberEvidence.subject === null + ? null + : { ...fixture.memberEvidence.subject, local_path: "/other.dll" }, + }; + expect( + fixture.grade([ + delivered(fixture.artifactEvidence), + delivered(conflicting), + member, + ]).factualCorrectness, + ).toBe("failed"); + }); + + it("binds the digest, rejects fabricated values and cites the supporting operation", () => { + const fixture = managedTranscript(); + expect( + fixture.grade(undefined, fixture.textFor(fixture.artifactEvidence)) + .factualCorrectness, + ).toBe("failed"); + expect( + fixture.grade( + undefined, + fixture.textFor(fixture.memberEvidence).replace('"Main"', '"Invented"'), + ).factualCorrectness, + ).toBe("failed"); + const other = createEvidence( + { path: "/targets/AgentEval.dll", sha256: "b".repeat(64), format: "pe" }, + provider, + { + operation: "inspect_managed_members", + parameters: {}, + result: fixture.result, + confidence: "observed", + authority: "shipped-artifact", + }, + ); + expect( + fixture.grade( + [delivered(fixture.artifactEvidence), delivered(other)], + fixture.textFor(other), + ).factualCorrectness, + ).toBe("failed"); + }); + + it("keeps golden names out of instructions and preserves routing-only scenarios", () => { + const instructions = agentFixtureAnswerInstructions("managed"); + expect(instructions).toContain("do not read target files directly"); + expect(instructions).not.toMatch(/AgentEval|Profile|Program|Main/u); + expect(agentFixtureAnswerInstructions("native")).toBe(""); + }); +}); + // Removing exact fixture comparison or accepted-source validation must break these regressions. describe("known-answer fixture grading", () => { it("passes closed claims against authenticated production MCP Evidence", () => { diff --git a/tests/fixtures/advertisedMcpTool.ts b/tests/fixtures/advertisedMcpTool.ts new file mode 100644 index 000000000..854d272ba --- /dev/null +++ b/tests/fixtures/advertisedMcpTool.ts @@ -0,0 +1,52 @@ +import type { Client } from "@modelcontextprotocol/client"; +import { Ajv2020 } from "ajv/dist/2020.js"; +import { expect } from "vitest"; +import { toolContract } from "../../src/contracts/toolContracts.js"; +import { parseEvidence } from "../../src/domain/evidence.js"; +import type { BinarySession } from "../../src/application/binary/BinarySession.js"; + +/** Verify the actual advertised schemas, retaining a private Ajv registry. */ +export const advertisedMcpToolSchemas = async ( + client: Client, + name: string, +) => { + const advertised = (await client.listTools()).tools.find( + (tool) => tool.name === name, + ); + if (advertised?.outputSchema === undefined) + throw new Error(`${name} must publish both schemas`); + const ajv = new Ajv2020({ strict: false, validateFormats: false }); + const inputSchema: Record = advertised.inputSchema; + const outputSchema: Record = advertised.outputSchema; + expect(ajv.validateSchema(inputSchema)).toBe(true); + expect(ajv.validateSchema(outputSchema)).toBe(true); + return { name: advertised.name, ajv, inputSchema, outputSchema }; +}; + +/** Keep public output, provenance and session retention checks together. */ +export const assertRecordedMcpEvidence = ( + response: Awaited>, + name: + | "inspect_wasm_artifact" + | "inspect_binary_layout" + | "inspect_evm_interface", + schemas: Awaited>, + session: BinarySession, + expected: { readonly artifact: { readonly sha256: string } }, + confidence: ReturnType["confidence"], +) => { + expect(response.isError).not.toBe(true); + expect( + schemas.ajv.validate(schemas.outputSchema, response.structuredContent), + JSON.stringify(schemas.ajv.errors), + ).toBe(true); + const parsed = toolContract(name).outputSchema.parse( + response.structuredContent, + ); + const evidence = parseEvidence(parsed); + expect(evidence.confidence).toBe(confidence); + expect(evidence.subject?.digest.sha256).toBe(expected.artifact.sha256); + expect(parsed.normalized_result).toEqual(expected); + expect(session.evidenceById(evidence.evidence_id)).toEqual(evidence); + return evidence; +}; diff --git a/tests/fixtures/android/direct-invokes/Caller.smali b/tests/fixtures/android/direct-invokes/Caller.smali new file mode 100644 index 000000000..4a565cca5 --- /dev/null +++ b/tests/fixtures/android/direct-invokes/Caller.smali @@ -0,0 +1,9 @@ +.class public Lfixture/Caller; +.super Ljava/lang/Object; + +.method public static twice()V + .registers 0 + invoke-static {}, Lfixture/Target;->mark()V + invoke-static {}, Lfixture/Target;->mark()V + return-void +.end method diff --git a/tests/fixtures/android/direct-invokes/Target.smali b/tests/fixtures/android/direct-invokes/Target.smali new file mode 100644 index 000000000..463234abf --- /dev/null +++ b/tests/fixtures/android/direct-invokes/Target.smali @@ -0,0 +1,7 @@ +.class public Lfixture/Target; +.super Ljava/lang/Object; + +.method public static mark()V + .registers 0 + return-void +.end method diff --git a/tests/fixtures/android/jadx-mcp.mjs b/tests/fixtures/android/jadx-mcp.mjs index 7319a5501..8b37ae54d 100644 --- a/tests/fixtures/android/jadx-mcp.mjs +++ b/tests/fixtures/android/jadx-mcp.mjs @@ -32,18 +32,21 @@ const methods = [ { name: "onCreate", signature: "onCreate(Bundle): void", + dex_descriptor: "(Landroid/os/Bundle;)V", is_constructor: false, def_pos: 0, }, { name: "choose", signature: "choose(String): void", + dex_descriptor: "(Ljava/lang/String;)V", is_constructor: false, def_pos: 0, }, { name: "choose", signature: "choose(int): void", + dex_descriptor: "(I)V", is_constructor: false, def_pos: 0, }, @@ -139,10 +142,18 @@ register("get_method_body", ({ method_name, overload_index }) => }), ); for (const name of ["get_xrefs_to_class", "get_xrefs_to_method"]) - register(name, ({ class_name, method_name }) => + register(name, ({ class_name, method_name, overload_index }) => json({ target: method_name === undefined ? class_name : `${class_name}.${method_name}`, + ...(method_name === undefined + ? {} + : { + overload_index, + reported_signature: methods.filter( + (method) => method.name === method_name, + )[overload_index]?.signature, + }), total: mode === "partial-xrefs" ? 2 : 1, count: 1, resolve_line: false, diff --git a/tests/fixtures/asarHeader.ts b/tests/fixtures/asarHeader.ts new file mode 100644 index 000000000..65fc0601b --- /dev/null +++ b/tests/fixtures/asarHeader.ts @@ -0,0 +1,27 @@ +/** Serialize a header-only ASAR, including Chromium pickle alignment. */ +export const asarHeaderBytes = (json: string): Buffer => { + const text = Buffer.from(json); + const headerSize = 8 + Math.ceil(text.length / 4) * 4; + const bytes = Buffer.alloc(8 + headerSize); + bytes.writeUInt32LE(4, 0); + bytes.writeUInt32LE(headerSize, 4); + bytes.writeUInt32LE(headerSize - 4, 8); + bytes.writeUInt32LE(text.length, 12); + text.copy(bytes, 16); + return bytes; +}; + +/** Nested metadata without a recursive fixture builder or real directories. */ +export const nestedAsarHeader = ( + depth: number, + name: string, + leaves: number, +): Buffer => { + let json = `{"files":{${Array.from( + { length: leaves }, + (_, index) => `"f${index}":{"size":0,"offset":"0"}`, + ).join(",")}}}`; + for (let index = 0; index < depth; index++) + json = `{"files":{${JSON.stringify(name)}:${json}}}`; + return asarHeaderBytes(json); +}; diff --git a/tests/fixtures/binaryDiagnostics/recordedCrash.ts b/tests/fixtures/binaryDiagnostics/recordedCrash.ts index 518b2d330..55698aac7 100644 --- a/tests/fixtures/binaryDiagnostics/recordedCrash.ts +++ b/tests/fixtures/binaryDiagnostics/recordedCrash.ts @@ -1,17 +1,41 @@ -import type { RecordedCrash } from "../../../src/domain/native/recordedCrash.js"; +import { + RECORDED_LINUX_CORE_PROFILES, + type RecordedCrash, + type RecordedCrashArchitecture, +} from "../../../src/domain/native/recordedCrash.js"; + +// One note is a 12-byte header, the "CORE\0" owner padded to 8 bytes, then the +// descriptor. The first note starts at the end of the owned header and phdr. +const NOTE_START = 0x80; +const OWNER_START = NOTE_START + 12; +const DESCRIPTOR_START = OWNER_START + 8; +const NOTE_HEADER_BYTES = 20; + +const hex = (value: number): string => `0x${value.toString(16)}`; /** Source-owned observation seam; these bytes do not simulate a verified ELF engine. */ export const recordedCrashFixture = ( path = "/artifacts/source-owned.core", + architecture: RecordedCrashArchitecture = "x86_64-little-endian", ): RecordedCrash => { - const descriptor = Buffer.alloc(336); + const profile = RECORDED_LINUX_CORE_PROFILES[architecture]; + const noteBytes = NOTE_HEADER_BYTES + profile.prstatusBytes; + const descriptor = Buffer.alloc(profile.prstatusBytes); descriptor.writeInt32LE(123, 32); descriptor.writeInt16LE(11, 12); - descriptor.writeBigUInt64LE(0x1122334455667788n, 224); + // One distinguishable register carries a full uint64 so retention is asserted. + descriptor.writeBigUInt64LE( + 0x1122334455667788n, + profile.registerOffset + 14 * profile.registerBytes, + ); return { - artifact: { path, sha256: "a".repeat(64), bytes: 512 }, + artifact: { + path, + sha256: "a".repeat(64), + bytes: Math.max(512, DESCRIPTOR_START + profile.prstatusBytes), + }, format: "elf-core", - architecture: "x86_64-little-endian", + architecture, target_execution: "not-performed", live_process_identity: "unknown", segments: [ @@ -20,7 +44,7 @@ export const recordedCrashFixture = ( type: "PT_NOTE", header_location: { offset: "0x40", bytes: "0x38" }, offset: "0x80", - file_size: "0x164", + file_size: hex(noteBytes), memory_size: "0x0", virtual_address: "0x0", physical_address: "0x0", @@ -34,11 +58,14 @@ export const recordedCrashFixture = ( index: 0, segment_index: 0, type: "NT_PRSTATUS", - location: { offset: "0x80", bytes: "0x164" }, - owner_location: { offset: "0x8c", bytes: "0x5" }, + location: { offset: hex(NOTE_START), bytes: hex(noteBytes) }, + owner_location: { offset: hex(OWNER_START), bytes: "0x5" }, owner_bytes_base64: Buffer.from("CORE\0").toString("base64"), owner_display: "CORE", - descriptor_location: { offset: "0x94", bytes: "0x150" }, + descriptor_location: { + offset: hex(DESCRIPTOR_START), + bytes: hex(profile.prstatusBytes), + }, descriptor_bytes_base64: descriptor.toString("base64"), }, ], @@ -48,40 +75,16 @@ export const recordedCrashFixture = ( note_index: 0, historical_pid: 123, recorded_current_signal: 11, - registers: [ - "r15", - "r14", - "r13", - "r12", - "rbp", - "rbx", - "r11", - "r10", - "r9", - "r8", - "rax", - "rcx", - "rdx", - "rsi", - "rdi", - "orig_rax", - "rip", - "cs", - "eflags", - "rsp", - "ss", - "fs_base", - "gs_base", - "ds", - "es", - "fs", - "gs", - ].map((name, index) => ({ + registers: profile.registers.map((name, index) => ({ name, - value: name === "rdi" ? "0x1122334455667788" : "0x0", + value: index === 14 ? "0x1122334455667788" : "0x0", location: { - offset: `0x${(148 + 112 + index * 8).toString(16)}`, - bytes: "0x8", + offset: hex( + DESCRIPTOR_START + + profile.registerOffset + + index * profile.registerBytes, + ), + bytes: hex(profile.registerBytes), }, })), }, @@ -100,27 +103,38 @@ export const recordedCrashFixture = ( }; /** Source-owned SIGSEGV descriptor for scalar/source binding regressions. */ -export const recordedCrashSignalFixture = (path?: string): RecordedCrash => { - const value = recordedCrashFixture(path); +export const recordedCrashSignalFixture = ( + path?: string, + architecture: RecordedCrashArchitecture = "x86_64-little-endian", +): RecordedCrash => { + const value = recordedCrashFixture(path, architecture); + const profile = RECORDED_LINUX_CORE_PROFILES[architecture]; const descriptor = Buffer.alloc(32); descriptor.writeInt32LE(11, 0); descriptor.writeInt32LE(-2, 4); descriptor.writeInt32LE(1, 8); descriptor.writeBigUInt64LE(0x1122334455667788n, 16); - value.artifact.bytes = 640; + const prstatusNoteBytes = NOTE_HEADER_BYTES + profile.prstatusBytes; + const signalNoteStart = NOTE_START + prstatusNoteBytes; + const signalDescriptorStart = signalNoteStart + NOTE_HEADER_BYTES; + const signalNoteBytes = NOTE_HEADER_BYTES + 32; + value.artifact.bytes = Math.max(640, signalDescriptorStart + 32); value.segments = value.segments.map((segment) => ({ ...segment, - file_size: "0x198", + file_size: hex(prstatusNoteBytes + signalNoteBytes), })); value.notes.push({ index: 1, segment_index: 0, type: "NT_SIGINFO", - location: { offset: "0x1e4", bytes: "0x34" }, - owner_location: { offset: "0x1f0", bytes: "0x5" }, + location: { offset: hex(signalNoteStart), bytes: hex(signalNoteBytes) }, + owner_location: { offset: hex(signalNoteStart + 12), bytes: "0x5" }, owner_bytes_base64: Buffer.from("CORE\0").toString("base64"), owner_display: "CORE", - descriptor_location: { offset: "0x1f8", bytes: "0x20" }, + descriptor_location: { + offset: hex(signalDescriptorStart), + bytes: "0x20", + }, descriptor_bytes_base64: descriptor.toString("base64"), }); value.signals.push({ @@ -141,7 +155,10 @@ export const recordedCrashFixtureBytes = (value: RecordedCrash): Buffer => { bytes.set([0x7f, 0x45, 0x4c, 0x46, 2, 1, 1]); bytes.writeUInt32LE(1, 20); bytes.writeUInt16LE(4, 16); - bytes.writeUInt16LE(62, 18); + bytes.writeUInt16LE( + RECORDED_LINUX_CORE_PROFILES[value.architecture].machine, + 18, + ); bytes.writeBigUInt64LE(64n, 32); bytes.writeUInt16LE(56, 54); bytes.writeUInt16LE(value.segments.length, 56); diff --git a/tests/fixtures/clientEnvironment.ts b/tests/fixtures/clientEnvironment.ts index 9825caeec..5e2ba94c2 100644 --- a/tests/fixtures/clientEnvironment.ts +++ b/tests/fixtures/clientEnvironment.ts @@ -27,6 +27,7 @@ export const CLIENT_LOCATION_ENVIRONMENT = [ "GEMINI_CLI_SYSTEM_DEFAULTS_PATH", "HERMES_DATA_DIR_SUFFIX", "HERMES_HOME", + "KIMI_CODE_HOME", "LOCALAPPDATA", "OMP_PROFILE", "OPENCODE_CONFIG", diff --git a/tests/fixtures/descriptorCloseFailure.ts b/tests/fixtures/descriptorCloseFailure.ts new file mode 100644 index 000000000..04cf29cde --- /dev/null +++ b/tests/fixtures/descriptorCloseFailure.ts @@ -0,0 +1,65 @@ +import type { FileHandle } from "node:fs/promises"; +import { expect } from "vitest"; +import type { Result } from "../../src/domain/result.js"; +import type { AnalysisError } from "../../src/domain/analysisErrorBase.js"; + +export interface DescriptorCloseFailure { + path: string; + denied: boolean; + handles: FileHandle[]; +} + +/** Public failure attribution and a still-live OS handle establish retained ownership. */ +export const expectRetainedDescriptor = ( + result: Result, + cleanup: DescriptorCloseFailure, + operation: string, +): void => { + expect(result).toMatchObject({ + ok: false, + error: { + cleanupIncomplete: true, + cleanupResources: [cleanup.path], + operation, + }, + }); + expect(cleanup.handles).toHaveLength(1); + expect(cleanup.handles[0]?.fd).toBeGreaterThanOrEqual(0); +}; + +export const descriptorFailurePath = ( + resource: string, + paths: Readonly>, +): string => { + switch (resource) { + case "capture": + case "tool": + case "limiter": + return paths[resource]; + case "kernel": + return "/proc/self/limits"; + default: + throw new Error(`Unknown descriptor failure resource: ${resource}`); + } +}; + +/** Inject a retryable close failure while retaining the actual OS descriptor. */ +export const withDescriptorCloseFailure = ( + filesystem: typeof import("node:fs/promises"), + cleanup: DescriptorCloseFailure, +) => ({ + ...filesystem, + open: async (...args: Parameters) => { + const handle = await filesystem.open(...args); + if (args[0] === cleanup.path) { + cleanup.handles.push(handle); + const close = handle.close.bind(handle); + handle.close = async () => { + if (cleanup.denied && handle.fd >= 0) + throw new Error("Descriptor close deliberately denied"); + await close(); + }; + } + return handle; + }, +}); diff --git a/tests/fixtures/evm/interface.ts b/tests/fixtures/evm/interface.ts index f0f1960ba..0c876c37b 100644 --- a/tests/fixtures/evm/interface.ts +++ b/tests/fixtures/evm/interface.ts @@ -18,6 +18,7 @@ export const evmInterfaceFixture = ( kind: "unknown", deployment_authenticity: "unknown", hardfork: "unknown", + metadata: null, }, evidence_kind: "inferred", functions: [], diff --git a/tests/fixtures/fakeCdpBrowser.ts b/tests/fixtures/fakeCdpBrowser.ts index 191b4569c..aa476a58b 100644 --- a/tests/fixtures/fakeCdpBrowser.ts +++ b/tests/fixtures/fakeCdpBrowser.ts @@ -28,6 +28,7 @@ export const startFakeCdpBrowser = async ( referer: string | undefined; }[] = []; const sockets = new Set(); + let openedConnections = 0; let frameTreeReads = 0; let port = 0; const http = createServer((request, response) => { @@ -85,6 +86,7 @@ export const startFakeCdpBrowser = async ( }); webSockets.on("connection", (socket: WebSocket, request: IncomingMessage) => { const directPage = request.url?.startsWith("/devtools/page/") === true; + openedConnections += 1; sockets.add(socket); socket.on("close", () => sockets.delete(socket)); socket.on("message", (raw) => { @@ -142,6 +144,7 @@ export const startFakeCdpBrowser = async ( allowedOrigin: endpoint, commands, activeConnectionCount: () => sockets.size, + openedConnectionCount: () => openedConnections, emitRawMessage(message) { for (const socket of sockets) socket.send(message); }, diff --git a/tests/fixtures/fakeCdpBrowserResults.ts b/tests/fixtures/fakeCdpBrowserResults.ts index 908db238e..4a489227f 100644 --- a/tests/fixtures/fakeCdpBrowserResults.ts +++ b/tests/fixtures/fakeCdpBrowserResults.ts @@ -24,6 +24,8 @@ export const resultFor = ( return responseBodyResult(options); case "Page.captureScreenshot": return screenshotResult(); + case "Browser.getVersion": + return browserVersionResult(); case "Target.getTargets": return { targetInfos: targets(port, options).map(endpointTargetToInfo) }; case "Storage.getUsageAndQuota": @@ -66,6 +68,15 @@ export const resultFor = ( } }; +/** Chrome's browser-socket reply shape, mirroring the fake's `/json/version`. */ +const browserVersionResult = () => ({ + protocolVersion: "1.3", + product: "FakeChrome/1.0", + revision: "@fake-revision", + userAgent: "FakeChrome", + jsVersion: "13.0", +}); + const attachToTargetResult = (options: FakeOptions) => ({ sessionId: options.invalidAttachedSession ? "" : "session-1", }); diff --git a/tests/fixtures/fakeCdpBrowserTypes.ts b/tests/fixtures/fakeCdpBrowserTypes.ts index 07a882a1e..e75e0dc48 100644 --- a/tests/fixtures/fakeCdpBrowserTypes.ts +++ b/tests/fixtures/fakeCdpBrowserTypes.ts @@ -13,6 +13,8 @@ export interface FakeCdpBrowser { readonly commands: readonly FakeCdpCommand[]; /** Actual WebSocket connections still held by the wire fixture. */ activeConnectionCount(): number; + /** WebSocket upgrades accepted over the fixture's lifetime. */ + openedConnectionCount(): number; /** Deliver actual malformed wire data without bypassing the production transport parser. */ emitRawMessage(message: string): void; emitEvent(event: { diff --git a/tests/fixtures/fakeGhidra.mjs b/tests/fixtures/fakeGhidra.mjs index b08410793..0a130c744 100644 --- a/tests/fixtures/fakeGhidra.mjs +++ b/tests/fixtures/fakeGhidra.mjs @@ -121,7 +121,7 @@ const handleRequest = (socket, server, request, state) => { ? [stringItem("0x2000", "fixture 雪🦊")] : request.method === "list_strings" && state.mode === "oversized_result" ? [stringItem("0x2000", "x".repeat(64 * 1024 * 1024 + 1))] - : inventoryResultFor(request.method); + : inventoryResultFor(request.method, request.params); if (inventory !== undefined) { const response = `${JSON.stringify({ id: request.id, ok: true, result: inventory })}\n`; if (state.mode === "fragmented") { @@ -228,6 +228,7 @@ const sessionInfo = ({ "shutdown", "inspect_native_load_image", "read_bytes", + "search_bytes", "address_to_file_offset", "address_name", "list_documents", @@ -273,8 +274,42 @@ const sessionInfo = ({ }, }); -const inventoryResultFor = (method) => { +const inventoryResultFor = (method, params) => { switch (method) { + case "search_bytes": { + const continued = params.cursor === "fixture-second-page"; + const paged = params.limit === 1 && !continued; + const addresses = continued + ? ["0x1005"] + : paged + ? ["0x1000"] + : ["0x1000", "0x1005"]; + return { + matches: addresses.map((address) => ({ + address, + segment: ".text", + file_offset: address === "0x1000" ? 0 : 5, + procedure: { address: "0x1000", name: "fixture_main" }, + })), + searched: { + blocks: [ + { + segment: ".text", + start: continued ? "0x1005" : "0x1000", + end: "0x1009", + bytes: continued ? 5 : 10, + }, + ], + bytes: continued ? 5 : 10, + skipped_uninitialized: [ + { segment: ".bss", start: "0x2000", end: "0x200f", bytes: 16 }, + ], + skipped_not_loaded: [], + }, + complete: !paged, + next_cursor: paged ? "fixture-second-page" : null, + }; + } case "address_name": return "fixture_main"; case "list_documents": diff --git a/tests/fixtures/httpComparisonBrowser.ts b/tests/fixtures/httpComparisonBrowser.ts new file mode 100644 index 000000000..94c7ed856 --- /dev/null +++ b/tests/fixtures/httpComparisonBrowser.ts @@ -0,0 +1,83 @@ +import { startFakeCdpBrowser, type FakeCdpBrowser } from "./fakeCdpBrowser.js"; + +export type HttpRequestObservation = { + readonly requestId: string; + readonly path?: string; + readonly status?: number; + readonly mimeType?: string; + readonly encodedDataLength?: number; +}; + +type CdpEvent = Parameters[0]; + +const requestEvents = ( + observation: HttpRequestObservation, + origin: string, + sessionId: string | undefined, +): CdpEvent[] => { + const session = sessionId === undefined ? {} : { sessionId }; + const url = `${origin}${observation.path ?? "/api/items"}`; + const { requestId, status, mimeType, encodedDataLength } = observation; + const events: CdpEvent[] = [ + { + ...session, + method: "Network.requestWillBeSent", + params: { + requestId, + timestamp: 1, + type: "Fetch", + request: { url, method: "GET", headers: {} }, + initiator: { type: "other" }, + }, + }, + ]; + if (status !== undefined || mimeType !== undefined) + events.push({ + ...session, + method: "Network.responseReceived", + params: { + requestId, + timestamp: 2, + type: "Fetch", + response: { + url, + headers: {}, + ...(status === undefined ? {} : { status }), + ...(mimeType === undefined ? {} : { mimeType }), + }, + }, + }); + if (encodedDataLength !== undefined) + events.push({ + ...session, + method: "Network.loadingFinished", + params: { requestId, timestamp: 3, encodedDataLength }, + }); + return events; +}; + +/** Capture changing response coverage through the real CDP wire boundary. */ +export const startHttpComparisonBrowser = async ( + initial: readonly HttpRequestObservation[], +): Promise<{ + readonly browser: FakeCdpBrowser; + setRequests(next: readonly HttpRequestObservation[]): void; +}> => { + let requests = initial; + const browser = await startFakeCdpBrowser({ + eventsBeforeReply: ["Network.enable"], + commandEvents(command, origin) { + return command.method === "Network.enable" + ? requests.flatMap((request) => + requestEvents(request, origin, command.sessionId), + ) + : undefined; + }, + }); + return { + browser, + setRequests(next) { + requests = next; + }, + }; +}; diff --git a/tests/fixtures/javaClassFile.ts b/tests/fixtures/javaClassFile.ts index 30f5ffc37..29fdace98 100644 --- a/tests/fixtures/javaClassFile.ts +++ b/tests/fixtures/javaClassFile.ts @@ -1,5 +1,8 @@ /** Build a valid minimal JVM class file, optionally with an unused UTF-8 constant. */ -export const javaClassFile = (extraUtf8Bytes = 0, fatCpuLookalike = false): Buffer => { +export const javaClassFile = ( + extraUtf8Bytes = 0, + fatCpuLookalike = false, +): Buffer => { const utf8 = (value: string): Buffer => { const bytes = Buffer.from(value, "utf8"); const entry = Buffer.alloc(3 + bytes.length); diff --git a/tests/fixtures/native-macos/dyld-inventory/README.md b/tests/fixtures/native-macos/dyld-inventory/README.md index af4744fe8..2c9b27743 100644 --- a/tests/fixtures/native-macos/dyld-inventory/README.md +++ b/tests/fixtures/native-macos/dyld-inventory/README.md @@ -37,6 +37,19 @@ import rows from `xcrun dyld_info -imports /usr/lib/libobjc.A.dylib`. `xcrun dyld_info -exports /usr/lib/libc++.1.dylib`. These are separate library captures used to check their respective row formats. +`renamed-reexports.txt` is `xcrun dyld_info -exports` for a dylib that +re-exports `_foo` from `libfoo` under the name `_foo_renamed`: + +```sh +printf 'int foo(void){return 1;}\n' > /tmp/foo.c +printf 'int own(void){return 2;}\n' > /tmp/re.c +xcrun clang -dynamiclib /tmp/foo.c -o /tmp/libfoo.dylib -install_name @rpath/libfoo.dylib +xcrun clang -dynamiclib /tmp/re.c -o /tmp/libre.dylib -install_name @rpath/libre.dylib \ + /tmp/libfoo.dylib -Wl,-alias,_foo,_foo_renamed +xcrun dyld_info -exports /tmp/libre.dylib +xcrun nm -gjU /tmp/libre.dylib +``` + `codesign-missing.txt` is stderr from `codesign -d --verbose=4` on a nonexistent path (exit 1). `codesign-unsigned.txt` is stderr from the same command on a copy of the fixture after `codesign --remove-signature` (exit 1). Only the latter is diff --git a/tests/fixtures/native-macos/dyld-inventory/renamed-reexports.txt b/tests/fixtures/native-macos/dyld-inventory/renamed-reexports.txt new file mode 100644 index 000000000..f401afcf5 --- /dev/null +++ b/tests/fixtures/native-macos/dyld-inventory/renamed-reexports.txt @@ -0,0 +1,5 @@ +fixture [arm64]: + -exports: + offset symbol + [re-export] _foo_renamed (_foo from libfoo) + 0x00000308 _own diff --git a/tests/process-global/adbStartMcp.test.ts b/tests/process-global/adbStartMcp.test.ts new file mode 100644 index 000000000..ac34b61e6 --- /dev/null +++ b/tests/process-global/adbStartMcp.test.ts @@ -0,0 +1,147 @@ +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import { expect, it, onTestFinished, vi } from "vitest"; + +import { AdbProvider } from "../../src/adb/AdbProvider.js"; +import { parseEvidence } from "../../src/domain/evidence.js"; +import { createServer } from "../../src/server/createServer.js"; +import { createTestBinarySession } from "../fixtures/binarySession.js"; +import { + execFileBytes, + execFileOutput, +} from "../../src/process/ExecFileOutput.js"; +import { toolContract } from "../../src/contracts/toolContracts.js"; + +vi.mock("../../src/process/ExecFileOutput.js", async (importOriginal) => ({ + ...(await importOriginal< + typeof import("../../src/process/ExecFileOutput.js") + >()), + execFileOutput: vi.fn(), + execFileBytes: vi.fn(), +})); + +it.each(["start_adb_app", "start_adb_activity"] as const)( + "%s retains launch evidence through MCP with isolated process mocks", + async (operation) => { + const starting = + "Starting: Intent { act=android.intent.action.MAIN cmp=com.example/.Main }\n"; + let stdout = `${starting}Status: ok\nActivity: com.example/.Main\nWaitTime: 315\nComplete\n`; + let stderr = ""; + vi.mocked(execFileOutput).mockResolvedValue({ + stdout: "Android Debug Bridge version 1.0.41\nVersion 34.0.5-stub\n", + stderr: "", + }); + const calls: string[][] = []; + vi.mocked(execFileBytes).mockImplementation(async (_binary, args) => { + calls.push([...args]); + return args.includes("resolve-activity") + ? { + stdout: Buffer.from("com.example/.Main\n"), + stderr: Buffer.alloc(0), + } + : { stdout: Buffer.from(stdout), stderr: Buffer.from(stderr) }; + }); + onTestFinished(() => { + vi.resetAllMocks(); + }); + const server = createServer( + { + kind: "session", + session: createTestBinarySession(() => { + throw new Error("must not start deep provider"); + }), + }, + { adbDeviceAnalysis: new AdbProvider({ environment: {} }) }, + ); + onTestFinished(() => server.close()); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + await server.connect(serverTransport); + const client = new Client({ name: "adb-start-boundary", version: "0.0.0" }); + onTestFinished(() => client.close()); + await client.connect(clientTransport); + const input = + operation === "start_adb_app" + ? { serial: "emulator-5554", package: "com.example" } + : { + serial: "emulator-5554", + action: "android.intent.action.MAIN", + component: "com.example/.Main", + }; + const successful = await client.callTool({ + name: operation, + arguments: input, + }); + expect(successful.isError).not.toBe(true); + const evidence = toolContract(operation).outputSchema.parse( + parseEvidence( + JSON.parse( + (successful.content as { text?: string }[]) + .map((part) => part.text ?? "") + .join(""), + ), + ), + ); + expect(evidence.confidence).toBe("observed"); + expect(evidence.normalized_result).toMatchObject({ + started_activity: "act=android.intent.action.MAIN cmp=com.example/.Main", + reported_activity: "com.example/.Main", + reported_status: "ok", + }); + expect(evidence.raw_result).toEqual({ stdout, stderr }); + + stdout = `${starting}Status: timeout\nActivity: com.example/.Main\nWaitTime: 10004\nComplete\n`; + const timedOut = await client.callTool({ + name: operation, + arguments: input, + }); + expect(timedOut.isError).toBe(true); + const failure = JSON.parse( + (timedOut.content as { text?: string }[]) + .map((part) => part.text ?? "") + .join(""), + ); + expect(failure.error).toMatchObject({ + code: "unsupported_target", + details: { + captured_output: { stdout, stderr, truncated: false }, + partial_observation: { + provider_id: "adb", + operation, + result: { + started_activity: + "act=android.intent.action.MAIN cmp=com.example/.Main", + reported_activity: "com.example/.Main", + reported_status: "timeout", + }, + }, + }, + }); + expect(failure).not.toHaveProperty("confidence"); + + stdout = starting; + stderr = "Error: Activity class does not exist\n"; + const refused = await client.callTool({ + name: operation, + arguments: input, + }); + expect(refused.isError).toBe(true); + expect( + (refused.content as { text?: string }[]) + .map((part) => part.text ?? "") + .join(""), + ).toContain("Activity class does not exist"); + expect(calls.filter((args) => args.includes("start"))).toEqual( + Array.from({ length: 3 }, () => [ + "-s", + "emulator-5554", + "shell", + "am", + "start", + "-W", + ...(operation === "start_adb_app" + ? ["-n", "com.example/.Main"] + : ["-a", "android.intent.action.MAIN", "-n", "com.example/.Main"]), + ]), + ); + }, +); diff --git a/tests/process-global/apktoolEnvironment.test.ts b/tests/process-global/apktoolEnvironment.test.ts new file mode 100644 index 000000000..90f4d483c --- /dev/null +++ b/tests/process-global/apktoolEnvironment.test.ts @@ -0,0 +1,53 @@ +import { afterEach, expect, it, onTestFinished, vi } from "vitest"; + +import { ApktoolProvider } from "../../src/apktool/ApktoolProvider.js"; +import { execFileOutput } from "../../src/process/ExecFileOutput.js"; + +vi.mock("../../src/process/ExecFileOutput.js", async (importOriginal) => ({ + ...(await importOriginal< + typeof import("../../src/process/ExecFileOutput.js") + >()), + execFileOutput: vi.fn(async () => ({ stdout: "2.7.0-stub\n", stderr: "" })), +})); +afterEach(() => { + vi.unstubAllEnvs(); + vi.mocked(execFileOutput).mockClear(); +}); + +it.skipIf(process.platform === "win32")( + "keeps an explicit environment's PATH choice and samples ambient inheritance per operation", + async () => { + vi.stubEnv("REA_APKTOOL_COMMAND", "/ambient/initial-apktool"); + vi.stubEnv("REA_APKTOOL_TEST_INHERITED_VALUE", "initial"); + const provider = new ApktoolProvider({ environment: {} }); + onTestFinished(() => provider.close()); + vi.stubEnv("REA_APKTOOL_COMMAND", "/ambient/changed-apktool"); + vi.stubEnv("REA_APKTOOL_TEST_INHERITED_VALUE", "changed"); + await provider.inspectAvailability(); + expect(vi.mocked(execFileOutput).mock.lastCall?.[0]).toBe("apktool"); + expect(vi.mocked(execFileOutput).mock.lastCall?.[2]?.env).toMatchObject({ + REA_APKTOOL_COMMAND: undefined, + REA_APKTOOL_TEST_INHERITED_VALUE: "changed", + }); + vi.stubEnv("REA_APKTOOL_TEST_INHERITED_VALUE", undefined); + await provider.inspectAvailability(); + expect( + vi.mocked(execFileOutput).mock.lastCall?.[2]?.env + ?.REA_APKTOOL_TEST_INHERITED_VALUE, + ).toBeUndefined(); + }, +); + +it.skipIf(process.platform === "win32")( + "inherits the current default launcher when no environment was selected", + async () => { + vi.stubEnv("REA_APKTOOL_COMMAND", "/ambient/initial-apktool"); + const provider = new ApktoolProvider(); + onTestFinished(() => provider.close()); + vi.stubEnv("REA_APKTOOL_COMMAND", "/ambient/later-apktool"); + await provider.inspectAvailability(); + expect(vi.mocked(execFileOutput).mock.lastCall?.[0]).toBe( + "/ambient/later-apktool", + ); + }, +); diff --git a/tests/process-global/browserScenarioCleanup.test.ts b/tests/process-global/browserScenarioCleanup.test.ts index b2acfd02e..498940e53 100644 --- a/tests/process-global/browserScenarioCleanup.test.ts +++ b/tests/process-global/browserScenarioCleanup.test.ts @@ -1,4 +1,5 @@ import { Cli } from "incur"; +import { resolve } from "node:path"; import { expect, it, vi } from "vitest"; import type { BrowserScenarioCapturePort } from "../../src/application/BrowserScenarioCapturePort.js"; @@ -32,7 +33,7 @@ vi.mock("../../src/composition/browserScenario.js", () => ({ })); const scenario = browserScenarioSchema.parse({ - browser: { mode: "launch", executable_path: "/fixture/chromium" }, + browser: { mode: "launch", executable_path: resolve("/fixture/chromium") }, start_url: { url: "https://app.example.test/" }, actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }], storage: {}, @@ -111,6 +112,22 @@ class RetrySession implements BrowserScenarioSessionPort { } } +it("waits for a slow but successful browser close before removing its profile", async () => { + const stages: string[] = []; + const cleanup = new PlaywrightScenarioBrowserCleanupOwner({ + closeBrowser: async () => { + await new Promise((resolve) => setTimeout(resolve, 1_250)); + stages.push("browser-closed"); + }, + removeProfile: async () => { + stages.push("profile-removed"); + }, + }); + + await cleanup.close(); + expect(stages).toEqual(["browser-closed", "profile-removed"]); +}); + const serveScenario = async () => { const cli = Cli.create("browser-cleanup-test", { sync: false }); registerBrowserScenarioCommands(cli, silentLogger, {}); diff --git a/tests/process-global/cleanLinuxHopperRuntime.test.ts b/tests/process-global/cleanLinuxHopperRuntime.test.ts new file mode 100644 index 000000000..8b7be17da --- /dev/null +++ b/tests/process-global/cleanLinuxHopperRuntime.test.ts @@ -0,0 +1,136 @@ +import { execFile } from "node:child_process"; +import { randomUUID } from "node:crypto"; +import { + chmod, + copyFile, + mkdir, + readFile, + symlink, + unlink, +} from "node:fs/promises"; +import { join } from "node:path"; +import { promisify } from "node:util"; + +import { expect, it } from "vitest"; + +import { assertCleanLinuxHopperRuntime } from "../../scripts/lib/clean-linux-hopper-runtime.mjs"; +import { completeVerifierRun } from "../../scripts/lib/verifier-run.mjs"; +import { cleanupOwnedProcessGroup } from "../../src/process/ProcessOwnership.js"; +import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js"; + +const execFileAsync = promisify(execFile); + +it + .runIf(process.platform === "linux") + .each(["demo helper", "alternate launcher"])( + "rejects a reparented %s without signaling it", + async (mode) => { + const runId = randomUUID(); + const childCode = + "process.title = 'Xvfb'; setInterval(() => {}, 1000); process.send('ready');"; + const directory = await createTestTempDirectory("rea-clean-runtime-test-"); + const launcher = join(directory, "Hopper"); + const alternate = join(directory, "Hopper-alternate"); + const procRoot = join(directory, "proc"); + await mkdir(procRoot); + const alternateMode = mode === "alternate launcher"; + if (alternateMode) { + await copyFile("/usr/bin/sleep", launcher); + await chmod(launcher, 0o700); + await symlink(launcher, alternate); + } + // A rejection must come from the fixture we are about to launch. An + // existing Xvfb would otherwise mask a broken executable-identity check. + await assertCleanLinuxHopperRuntime( + alternateMode ? [launcher] : [], + procRoot, + ); + const command = alternateMode ? alternate : process.execPath; + const arguments_ = alternateMode ? ["30"] : ["-e", childCode]; + const parentCode = ` + const { spawn } = require('node:child_process'); + const child = spawn(${JSON.stringify(command)}, ${JSON.stringify(arguments_)}, { + detached: true, stdio: ${JSON.stringify(alternateMode ? "ignore" : ["ignore", "ignore", "ignore", "ipc"])} + }); + child.once('error', () => process.exit(1)); + child.once(${JSON.stringify(alternateMode ? "spawn" : "message")}, () => { + process.stdout.write(JSON.stringify({ pid: child.pid, parentPid: process.pid })); + if (child.connected) child.disconnect(); + child.unref(); + }); + `; + const parent = execFileAsync(process.execPath, ["-e", parentCode], { + env: { ...process.env, REA_PROCESS_RUN_ID: runId }, + timeout: 10_000, + }); + const parentPid = parent.child.pid; + if (parentPid === undefined) + throw new Error("Fixture parent did not spawn"); + let orphanPid: number | undefined; + try { + const { stdout } = await parent; + const { pid } = JSON.parse(stdout) as { pid: number }; + orphanPid = pid; + if (alternateMode) { + const processEntry = join(procRoot, String(pid)); + await mkdir(processEntry); + await symlink(`/proc/${String(pid)}/exe`, join(processEntry, "exe")); + } + const { stdout: coordinates } = await execFileAsync("ps", [ + "-p", + String(pid), + "-o", + "ppid=,pgid=", + ]); + const [currentParent, group] = coordinates + .trim() + .split(/\s+/u) + .map(Number); + expect(currentParent).not.toBe(parentPid); + expect(group).toBe(pid); + + const lineage = await completeVerifierRun({ + run_id: runId, + verifier_pid: process.pid, + parent_pid: process.ppid, + }); + expect(lineage.process_lineage.status).toBe("verified"); + if (lineage.process_lineage.status === "verified") + expect( + lineage.process_lineage.descendants.some((p) => p.pid === pid), + ).toBe(false); + + if (alternateMode) { + const commandLine = await readFile( + `/proc/${String(pid)}/cmdline`, + "utf8", + ); + expect(commandLine.split("\0")[0]).toBe(alternate); + await unlink(alternate); + } + await expect( + assertCleanLinuxHopperRuntime( + alternateMode ? [launcher] : [], + procRoot, + ), + ).rejects.toThrow("Clean Linux runner retained a Hopper demo process"); + // Observation must not terminate the fixture, even though it resembles Xvfb. + const { stdout: alive } = await execFileAsync("ps", [ + "-p", + String(pid), + "-o", + "stat=", + ]); + expect(alive.trim()).not.toMatch(/^Z/u); + } finally { + if (orphanPid !== undefined) { + const cleanup = await cleanupOwnedProcessGroup({ + runId, + leaderPid: orphanPid, + processGroupId: orphanPid, + }); + expect(cleanup.cleaned, JSON.stringify(cleanup)).toBe(true); + } + } + }, +); diff --git a/tests/process-global/updateMaintenance.test.ts b/tests/process-global/updateMaintenance.test.ts index ae057bab2..171761478 100644 --- a/tests/process-global/updateMaintenance.test.ts +++ b/tests/process-global/updateMaintenance.test.ts @@ -25,6 +25,7 @@ beforeEach(async () => { HOME: home, USERPROFILE: home, APPDATA: join(home, "AppData", "Roaming"), + LOCALAPPDATA: join(home, "AppData", "Local"), CLAUDE_CONFIG_DIR: home, CODEX_HOME: join(home, ".codex"), COPILOT_HOME: join(home, ".copilot"), diff --git a/website/README.md b/website/README.md index f09c7f603..6c260affd 100644 --- a/website/README.md +++ b/website/README.md @@ -33,8 +33,43 @@ After editing source HTML or a catalog, run `python3 scripts/prepare-website.py` to rebuild the preview. `serve` prepares the site once when it starts. Keep the virtual environment active when preparing assets. CairoSVG needs the native Cairo library; the sharing card uses DejaVu Sans. On Debian or Ubuntu, -install them with `sudo apt-get install libcairo2 fonts-dejavu-core`. For other -systems, see [CairoSVG's installation instructions](https://cairosvg.org/documentation/#installation). +install them with `sudo apt-get install libcairo2 fonts-dejavu-core`. On an +Apple Silicon Mac, enter an ARM shell first with `/usr/bin/arch -arm64 /bin/zsh`. Install +the ARM build with `/opt/homebrew/bin/brew install cairo pkg-config`. Install +the sharing-card font with `/opt/homebrew/bin/brew install --cask font-dejavu`. +The card in `public/assets/social-card.svg` requests DejaVu Sans, so Cairo output +uses a fallback font without it. Verify with `fc-match "DejaVu Sans"`. The result +should name DejaVu. A plain +`brew` call can select the Intel brew when `/usr/local/bin` comes first or the +terminal runs under Rosetta. Check that +`/opt/homebrew/lib/libcairo.dylib` reports `arm64`. An old Intel build in +`/usr/local/lib` shadows the ARM build. The preview then fails with +`incompatible architecture (have 'x86_64', need 'arm64')`. Keep +`/usr/local/lib` intact. Do not move its files and do not link ARM files into +it. That breaks Intel programs that use the same names. Use an unprotected ARM +Python, not Apple `/usr/bin/python3`. A venv from the Apple interpreter stays +protected and purges `DYLD` at launch. Deactivate any active venv, confirm the +shell reports `arm64`, then recreate the venv with the ARM interpreter by its +full path: + +```sh +deactivate || true +/usr/bin/arch +/opt/homebrew/bin/brew install python3 +/opt/homebrew/bin/python3 -m venv website/.venv --clear +source website/.venv/bin/activate +python3 -m pip install -r website/requirements.txt +``` + +A rebuilt venv is empty +without this step. Export `DYLD_LIBRARY_PATH=/opt/homebrew/lib` and +`DYLD_FALLBACK_LIBRARY_PATH=/opt/homebrew/lib` inside the ARM shell. Verify +with `python3 -c "import ctypes.util; +print(ctypes.util.find_library('cairo'))"`. The result should point at +`/opt/homebrew`. Run the preview from that same shell with +`python3 scripts/website.py serve` and no `arch` prefix. A `DYLD` export +before `arch` gets purged and the preview fails again. For other systems, see +[CairoSVG's installation instructions](https://cairosvg.org/documentation/#installation). CI installs both packages explicitly before rendering the card. To test the GitHub Pages prefix, use diff --git a/website/locales/README.md b/website/locales/README.md index de337f6cc..49f3c0d6e 100644 --- a/website/locales/README.md +++ b/website/locales/README.md @@ -113,7 +113,7 @@ translated fragments. ## Language coverage and initial translations -`languages.json` lists the same editions as the root README, with native names, +`languages.json` lists the same editions as the English root README and the translations in `docs/readme/`, with native names, standard language tags and text direction. English keeps its existing routes; other editions use a language prefix such as `/ja/` or `/pt-BR/`. Arabic and Persian use right-to-left page layouts; code and numeric demos remain left to diff --git a/website/locales/languages.json b/website/locales/languages.json index 49e004304..50c110e8b 100644 --- a/website/locales/languages.json +++ b/website/locales/languages.json @@ -3,49 +3,109 @@ { "code": "zh-CN", "name": "简体中文", - "readme": "README_zh.md", + "readme": "docs/readme/README_zh.md", "dir": "ltr" }, { "code": "zh-TW", "name": "繁體中文", - "readme": "README_zh-TW.md", + "readme": "docs/readme/README_zh-TW.md", + "dir": "ltr" + }, + { + "code": "ja", + "name": "日本語", + "readme": "docs/readme/README_ja.md", + "dir": "ltr" + }, + { + "code": "ko", + "name": "한국어", + "readme": "docs/readme/README_ko.md", + "dir": "ltr" + }, + { + "code": "tr", + "name": "Türkçe", + "readme": "docs/readme/README_tr.md", + "dir": "ltr" + }, + { + "code": "ru", + "name": "Русский", + "readme": "docs/readme/README_ru.md", "dir": "ltr" }, - { "code": "ja", "name": "日本語", "readme": "README_ja.md", "dir": "ltr" }, - { "code": "ko", "name": "한국어", "readme": "README_ko.md", "dir": "ltr" }, - { "code": "tr", "name": "Türkçe", "readme": "README_tr.md", "dir": "ltr" }, - { "code": "ru", "name": "Русский", "readme": "README_ru.md", "dir": "ltr" }, { "code": "vi", "name": "Tiếng Việt", - "readme": "README_vi.md", + "readme": "docs/readme/README_vi.md", + "dir": "ltr" + }, + { + "code": "th", + "name": "ไทย", + "readme": "docs/readme/README_th.md", + "dir": "ltr" + }, + { + "code": "de", + "name": "Deutsch", + "readme": "docs/readme/README_de.md", + "dir": "ltr" + }, + { + "code": "es", + "name": "Español", + "readme": "docs/readme/README_es.md", + "dir": "ltr" + }, + { + "code": "fr", + "name": "Français", + "readme": "docs/readme/README_fr.md", "dir": "ltr" }, - { "code": "th", "name": "ไทย", "readme": "README_th.md", "dir": "ltr" }, - { "code": "de", "name": "Deutsch", "readme": "README_de.md", "dir": "ltr" }, - { "code": "es", "name": "Español", "readme": "README_es.md", "dir": "ltr" }, - { "code": "fr", "name": "Français", "readme": "README_fr.md", "dir": "ltr" }, { "code": "uk", "name": "Українська", - "readme": "README_uk.md", + "readme": "docs/readme/README_uk.md", + "dir": "ltr" + }, + { + "code": "pl", + "name": "Polski", + "readme": "docs/readme/README_pl.md", "dir": "ltr" }, - { "code": "pl", "name": "Polski", "readme": "README_pl.md", "dir": "ltr" }, { "code": "pt-BR", "name": "Português (Brasil)", - "readme": "README_pt-BR.md", + "readme": "docs/readme/README_pt-BR.md", "dir": "ltr" }, - { "code": "ar", "name": "العربية", "readme": "README_ar.md", "dir": "rtl" }, - { "code": "fa", "name": "فارسی", "readme": "README_fa.md", "dir": "rtl" }, + { + "code": "ar", + "name": "العربية", + "readme": "docs/readme/README_ar.md", + "dir": "rtl" + }, + { + "code": "fa", + "name": "فارسی", + "readme": "docs/readme/README_fa.md", + "dir": "rtl" + }, { "code": "id", "name": "Bahasa Indonesia", - "readme": "README_id.md", + "readme": "docs/readme/README_id.md", "dir": "ltr" }, - { "code": "it", "name": "Italiano", "readme": "README_it.md", "dir": "ltr" } + { + "code": "it", + "name": "Italiano", + "readme": "docs/readme/README_it.md", + "dir": "ltr" + } ]