Compare commits

...

30 Commits

Author SHA1 Message Date
Kaoru0822 935b62249b fix(managed): keep delivery detail out of retained Evidence identity
Summary, omitted and complete delivery of the same managed inspection
now retain one parent record, as inspect_macho already does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 01:41:55 +00:00
Kaoru0822 1f4ee2dcda Merge origin/main into perf/managed-member-summary
Resolve analysis-view conflicts with the Interface Builder views, and
adopt main's single owners: parseAnalysisViewResult for the member parent
parse and retainedSummaryDelivery for managed summary delivery.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 01:24:01 +00:00
Matt Van Horn bcc141549a
feat(electron): inspect packaged Electron fuse state without execution (#2399)
Add bounded, read-only inspection of packaged Electron fuse state for ELF, PE, and Mach-O applications through CLI and MCP workflows.

Preserve artifact provenance and unknown values, resolve app bundles without execution, reject malformed or overlapping containers before scanning, and keep output compact for agent use.

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-14 09:16:33 +08:00
morluto fab33a66c6 test(package): select the intended Hopper smoke-test provider
Keep provider discovery independent from Hopper-specific packaged workflows. Installed Ghidra launchers on PATH must not turn the fixture into an ambiguous auto-selection request; target-free inventory and unknown-provider checks remain explicit.
2026-10-14 09:09:02 +08:00
morluto 86afc040e5 fix(ghidra): discover installations from resolved PATH launchers
Reuse installation inspection in doctor and provider readiness. Keep explicit configuration authoritative and validate discovered roots without executing launchers. Clarify the existing per-operation availability contract.
2026-10-14 09:05:53 +08:00
morluto 127c621328 Merge remote-tracking branch 'origin/main' 2026-10-14 08:59:31 +08:00
morluto 4d4631ac94 test: harden verification fixtures and runtime coverage
Use exact discovered test identities, production MCP sessions and byte-exact setup backups. Add CLI signal cleanup coverage and independent Ghidra, Go and firmware verification assertions; prune redundant test configuration and preserve platform limits.
2026-10-14 08:59:27 +08:00
morluto 1b14870ace refactor(evidence): remove superseded ordering wrapper and document exclusions 2026-10-14 08:58:53 +08:00
morluto a688c79784 docs(ida): describe catalog-based lifecycle compatibility 2026-10-14 08:58:46 +08:00
morluto 537467ab9e fix(test-fixtures): keep managed metadata and method bodies disjoint
Lay out resources, ReadyToRun data and IL bodies from their actual extents. Preserve RVA zero for bodyless methods and verify adjacent bodies through the production inspector.
2026-10-14 08:58:34 +08:00
morluto a769915f6e refactor(javascript): reuse canonical AST traversal
Remove the separate Vite walker and raw visitor-key require scan. Keep subtree selection at the shared traversal owner. All 103 focused builtin/Vite tests and the complete static and architecture checks pass.
2026-10-14 08:51:32 +08:00
morluto b58adafa4f fix(javascript): align module lookup and source evidence
Preserve CommonJS literal paths and confined ESM URL semantics, including package main fallback, directory intent, Unicode identity, and present opaque modules. Parse extraction inputs at their AST/source owner, retain skipped HTML and source-map coverage, and distinguish endpoint candidates from proven receiver facts. Version structural fingerprints with computed literal contents and propagate them through worker contracts.

Reviewed fixes include #1810, #1840, #2642, #2700, #2702, #2714, #2803, #2960, #2968, #2986, #3035, and #3036.

Validation: 158 resolver tests passed; public CLI/MCP module lookup controls, package-main/Unicode workflows, and six builtin workflows passed in the current compiled lane. Worker schema/fingerprint regressions passed. Large public pdf.js completion and real Electron analysis remain unverified.
2026-10-14 08:46:50 +08:00
William Blair 41a2e84647
docs(installation): limit REA to one project or agent profile
Document existing Claude Code profile and per-project installation workflows, including scoped maintenance, backups, caller metadata and Windows commands. Clarify authentication storage and the filesystem/network restrictions required for enforced separation.

Validated 68 boundary tests, docs checks, formatting, typecheck/lint and two real REA setup dry-runs with zero applied actions or profile writes. Native Claude sessions, PowerShell execution and enforcement policies were not tested. Setup automation remains separate in #2187.

Co-authored-by: William Blair <60581672+williamblair333@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-10-14 08:45:50 +08:00
morluto 04672f2b91 fix(javascript): preserve builtin identity and share alias traversal
Track module writes and escapes at their owner, preserve capture and reassignment ordering, and share CommonJS/default namespace identity without erasing parent-slot changes. Carry write sites into ambient alias resolution and use parsed write targets across consumers. Deduplicate returned-reference work with cycle context; retain unchanged temporal selection identity so provenance memoization stays effective.

Reviewed fixes include #2041, #2465, #2830, #2881, and #2948. Remove superseded mutation representations and the ineffective per-markValue dedup layer.

Validation: 380 focused JavaScript tests passed under a 384 MiB heap cap, including the previously aborting 30-level/128-call alias regression. Six public CLI/MCP builtin tests passed. Exact pdf.js completed-workflow performance remains unmeasured.
2026-10-14 08:44:52 +08:00
morluto 0f4c484be5 fix(reference): preserve missing symlink target facts
Treat ENOTDIR while resolving a symlink target as missing target evidence, preserving its spelling and partial inventory. Keep graph-level excluded-ancestor rejection separate from import success. The compiled boundary regression passes.
2026-10-14 08:43:59 +08:00
Ysopwnable 6a7f70fd54
test(windows): verify retained inspection ownership after rename (#2902)
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 17:39:38 -07:00
morluto 3ea5fdd7cf chore: reconcile current main fixes and client support 2026-10-14 08:36:38 +08:00
Kaoru0822 ff56f78594
fix(uninstall): remove the REA-owned ZCode fallback registration (#2783)
* fix(uninstall): remove the REA-owned ZCode fallback registration

ZCode loads the user-scope `~/.agents/mcp.json` `mcpServers` fallback
whenever its native `mcp.servers` table is empty. Uninstall removed REA
only from the native table, so a REA-owned fallback entry became eligible
to load again once the last native server was gone, while uninstall
reported ZCode as removed and complete.

Also visit the ZCode fallback as a standard `mcpServers` JSON
configuration in the uninstall host. It goes through the same
ownership check, `.rea.backup`, readback and restore path as every other
client configuration, so other fallback servers and settings, and a
foreign `rea` entry, are preserved. Its result merges into the ZCode item.

Fixes #2664

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(uninstall): preserve the strict ZCode fallback dialect

Address review findings at their producing boundaries and preserve directly affected consumers. Source inspection and git diff --check completed. Tests and automated checks were not run at the user’s explicit request.

* fix(comparison): recheck historical calls-only changes (#2223)

* fix(comparison): preserve incomplete function call evidence

Keep missing, unknown, partial and unresolved call relationships unknown in function comparisons and reconstruction claims. Aggregate IDA caller and callee coverage without overwriting uncertainty.

Validate with 144 focused tests, typecheck/lint, docs checks and test discovery. Real providers were not verified; the full headless IDA session suite requires unavailable Windows native controls.

* fix(comparison): recheck historical calls-only changes

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>

* fix(ci): isolate Linux Hopper process fixture inventory

* fix(android): make DEX descriptor pattern portable

* fix: repair the static-check failures on the default branch (#2895)

* fix: repair the static-check failures on the default branch

The default branch fails its own required `Static checks` job three ways, so
every pull request reports that job as failed regardless of its own change.

- `npm run format:check` reports `src/domain/functionComparison.test.ts` and
  `src/ida/IdaOperationRunner.ts`.
- `npm run lint` reports a duplicate `import { join } from "node:path"` in
  `tests/boundary/mcp/ghidraInventoryMcp.test.ts` (lines 4 and 17).
- `npm run typecheck` reports `format: "devin"` missing from the client format
  union in `src/application/SupportedClients.ts`, and an unchecked index in
  `tests/boundary/cli/clientProfileLifecycle.test.ts`, whose pair array is
  inferred as `string[][]`.

Apply the pinned formatter, drop the duplicate import, list the `"devin"` format
that its own definition already uses, and mark the pair array `as const`.

Validation: `npm run check:fast` and `npm run format:check` pass.

* fix(setup): keep Devin on the JSON configuration dialect

---------

Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: Lucien <164857476+dakjdakd@users.noreply.github.com>
Co-authored-by: 吴佳浩Alben <35291372+XiaoBinGan@users.noreply.github.com>
Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
2026-10-13 17:33:26 -07:00
Kaoru0822 679658b287
fix(reference): report entries beyond the path limit as I/O failures (#2716)
* fix(reference): report entries beyond the path limit as I/O failures

Reference-source traversal joins each child onto the root path. A valid
tree whose nested entries exceed the platform's absolute path limit
(PATH_MAX 1024 on macOS) makes lstat fail with ENAMETOOLONG. The
reader's filesystem error allow-list lacked that code, so
filesystemFailureDetail returned undefined and the native rejection
escaped both readReferenceSource and importReferenceSource instead of
becoming the existing failed `io` entry.

Add ENAMETOOLONG to the allow-list, matching the JavaScript application
reader's list. Readable siblings stay in the graph and the unreachable
entry is recorded with its OS message.

Fixes #2652

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(reference): assert the imported graph keeps readable and failed entries

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(process): report no remaining time after a provider deadline fires

Node timers can fire while performance.now() is still a fraction of a
millisecond short of the monotonic deadline. ProviderStartupDeadline then
reported interruption "timeout" and an aborted signal while remainingMs()
returned ceil(0.x) = 1, so polling loops guarded by remainingMs() > 0
(Hopper and Ghidra startup) could run past an elapsed deadline, and the
real-timer test "aborts a bounded wait when its real monotonic deadline
expires" failed intermittently in CI (expected 1 to be 0).

Once the deadline's own timer has classified a timeout, report zero
remaining time. A deterministic regression test fires the timer with the
monotonic clock 0.4 ms short of the deadline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(reference): preserve path limit entry uncertainty

* fix(reference): preserve unknown entry failures and identity

* fix(reference): type unknown-kind exclusions and keep them out of symlink counts

The metadata-failure path offers "unknown" to exclusion predicates, but
the predicate type only admitted concrete kinds, which broke the build.
Directory-only secret and ignore rules now also match an unknown-kind
entry (it may be a directory), and the summary counts unknown entries
separately instead of as symlinks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: Codex <codex@users.noreply.github.com>
2026-10-13 17:33:09 -07:00
morluto 5b7fe94710 fix(analysis): preserve extraction gaps and database failure reasons
Derive firmware completeness from examined coverage, retain linked continuation evidence, and preserve reported extraction limits. Keep Android package attribution and Go image fixtures coherent, and classify SQLite busy failures without discarding descriptor cleanup facts.

Reviewed fixes include firmware, ADB, SQLite, and recorded-transport bug reports and their focused PRs.

Validation: firmware report source tests passed 21 cases; prior focused artifact/provider boundaries passed. Real Android devices and external firmware tools were not rerun.
2026-10-14 08:32:15 +08:00
morluto 80de3145e4 fix(mcp): align advertised contracts and retain workflow evidence
Use one named input-schema projection for clients and catalog identity, preserve dialect declarations, and bound compact schema depth. Keep canonical SDK output validation when output advertisements are disabled. Release request-owned state on cancellation and carry upstream limitations through composed workflows. Preserve doctor full-output envelopes and validate navigation facts at their owner.

Reviewed fixes: #2088, #2108, #2513, #2684, #2693, #2695, #2836, #2864, #2868, #2877, #2933, #2998, #3002, #3015.

Validation: focused schema, SDK output-validation, context, and workflow tests passed; no schema dialect or canonical output validation was relaxed.
2026-10-14 08:32:00 +08:00
morluto f6905719c3 fix(browser): bound capture work and preserve observation gaps
Own service-worker CDP registrations and retry failed teardown. Admit redaction and network metadata against remaining capture capacity, retain explicit omission reasons, and preserve source URL, redirect ancestry, caller-selected content, and historical capture values. Forward CLI cancellation through provider cleanup.

Reviewed fixes: #2263, #2637, #2767, #2845, #2859, #2863, #2872, #2874, #2876, #2879, #2955, #2957, #2961, #3033.

Validation: 25 current event/network source tests passed; earlier focused browser lanes passed 201 source tests and 176 boundary tests with two skips, plus 35 Python tests. Real browser/Windows lanes were not rerun. Playwright allocation before admission remains unbounded by these retention limits.
2026-10-14 08:31:29 +08:00
Danish Rafie Ekaputra a6c3d3a365
docs(skill): route browser actions to scenario capture
Route user browser actions to passive observation and agent actions to declared scenarios. Clarify requested artifacts, step outcomes, connect navigation and cleanup, and the limits of listener and request-initiator evidence. Distinguish discarded raw headers from retained response metadata. Update the bundled skill to version 42 with matching package metadata.

Validation on the synchronized revision: docs (5 tests), typecheck/lint, formatting, 61 focused tests, and real Linux Chromium launch/connect and passive CLI probes. Verified declared steps, requested artifacts, digests, external page preservation, temporary form reset and cleanup. Native Windows/macOS and separate follow-up tools were not rerun. Package install/update and CLI/MCP verification passed earlier on 738d02a, before this main sync.

Co-authored-by: 0xRafie <155859586+0xRafie@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-10-14 08:31:16 +08:00
morluto ec271121e4 fix(artifacts): preserve path identity and exclusion applicability
Normalize path facts once at the owning filesystem boundary, preserve caller spelling and unavailable inventory members, and retain typed read failures. Record exclusion entry kinds so only proven directories can establish descendant symlink targets; omitted legacy kinds stay unknown.

Reviewed fixes include #1969, #2642, #2759, and #2994.

Validation: 26 reference graph source tests passed; prior artifact checks passed 275 tests with one Windows-only skip. Updated compiled import workflows are included in final validation after regeneration.
2026-10-14 08:31:16 +08:00
morluto 55dd21a5e2 fix(native): retain partial format facts and explicit coverage
Preserve XML and Interface Builder identity at their parsing boundary, retain usable Objective-C slots with precise coverage, and keep native trace failures distinct from absence. Add caller-selected Interface Builder summaries backed by complete retained Evidence, and preserve comparison limitations.

Reviewed fixes: #1969, #2088, #2108, #2513, #2639, #2684, #2731, #2759, #2875, #2888, #2889, #2970, #2971, #3002.

Validation: relevant native/domain focused lane passed 142 tests; earlier Interface Builder/MCP checks passed 24 tests. Real native UI and Ghidra producer coverage was not repeated.
2026-10-14 08:29:16 +08:00
Ysopwnable d82ad1faf0
test(android): preserve malformed DEX prototype rejection (#2917)
Preserve malformed DEX parameter/return rejection with tagged producer errors and exact issue paths.

The equivalent portable descriptor fix is already on main via #2910.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-14 08:28:23 +08:00
morluto d4bd65c53c fix(providers)!: preserve readiness, coverage, and diagnostic ownership
Share tagged readiness observations, retain partial provider facts, authenticate bridge identity, bound diagnostics, and preserve cancellation and lost-session reasons. Fail closed for unparsed MIPS ABI attributes and macOS saved Hopper databases without a script-capable bridge route.

Reviewed fixes: #2010, #2108, #2513, #2695, #2731, #2771, #2828, #2867, #2868, #2870, #2880, #2919, #2945, #2961, #2981, #2998, #3000, #3002.

Validation: 285 focused source/boundary tests passed. Real Ghidra, IDA, Linux Hopper, and Windows provider runs were not repeated.

BREAKING CHANGE: MIPS targets with unresolved GNU ABI attributes and saved Hopper databases on macOS are refused rather than advertised as supported.
2026-10-14 08:28:14 +08:00
JANG BYUNGKUN 2dc89f81be
docs(readme): sync Korean README with the English README (#3046) 2026-10-14 08:27:37 +08:00
morluto ec56c1da04 fix(setup): preserve registration identity and recovery ownership
Keep layered environment choices, current byte-exact recovery snapshots, and package-manager failure details. Report pinned package drift without claiming runner resolution. Retain failed Linux installer cleanup for host-owned retry while preserving the install outcome.

Reviewed fixes include #2632, #2769, #2783, #2959, #2979, #2983, #2995, #3009, and #2559.

Validation: 98 focused setup/registration tests passed; 45 Linux installer tests passed. Real Linux package installation remains unverified.
2026-10-14 08:25:53 +08:00
morluto 599a8ee11d fix(process): preserve cleanup ownership and normalize captures once
Retain unresolved process identity and launch evidence through retryable cleanup. Share capture path normalization and preserve failed filesystem observations. Bound child output and drain Windows pipes without one-chunk throttling.

Reviewed fixes include #1628, #2632, #2644, #2784, #2928, #2931, #2965, #2992, and #3011.

Validation: 92 focused source tests passed; four platform-specific skips. Real Windows and Hopper lifecycle coverage was not rerun.
2026-10-14 08:19:07 +08:00
561 changed files with 38436 additions and 3610 deletions

View File

@ -2,7 +2,7 @@
name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "41"
version: "42"
---
# REA
@ -104,6 +104,10 @@ explicit path or endpoint and do not need it.
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
optional metadata recovery and its supported host/layout boundary.
- User-owned browser page already open: `list_browser_targets`.
- Browser action the agent must perform (click, type, navigate):
`capture_browser_scenario` with declared actions. See
[runtime observation](references/runtime-observation.md) for connect/launch
and the focused follow-up browser tools.
- Fiddler Everywhere traffic or a native `.saz` capture: read the
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
@ -118,7 +122,7 @@ explicit path or endpoint and do not need it.
pages; select one trace, stream ID and direction for contribution/range pages.
Facts retain all-recorded-events-before-selection scope. Other qlog layouts
and PCAP correlation are unsupported.
- User-owned Electron runtime already open: `list_electron_targets`.
- User-owned Electron runtime already open: `list_electron_targets`. Before an electron-main Inspector attempt, check packaged fuses with `inspect_electron_fuses`; disabled `EnableNodeCliInspectArguments` means the app ignores `--inspect`.
- Packet-field discovery: `inspect_packet_fields` searches installed protocols
and field names/types before choosing fields for `inspect_packet_capture`.
Query/page the relevant records; registrations do not establish observed traffic.
@ -323,7 +327,8 @@ as complete while required questions remain open.
[references/javascript-applications.md](references/javascript-applications.md)
- Android APK declarations, classes, methods, and static references:
[references/android-applications.md](references/android-applications.md)
- Passive browser/Electron observation and static/runtime reconciliation:
- Browser/Electron observation (passive and scenario) and static/runtime
reconciliation:
[references/runtime-observation.md](references/runtime-observation.md)
- Evidence paging, comparisons, residual unknowns, and verification:
[references/evidence-workflows.md](references/evidence-workflows.md)

View File

@ -1,4 +1,4 @@
# Passive browser and Electron observation
# Browser and Electron runtime observation
Supply the existing browser or Electron instance's literal-port loopback CDP
endpoint in the request. No separate permission configuration or per-call
@ -7,13 +7,16 @@ approval flag is needed. Browser discovery lists eligible HTTP(S) pages;
selected page's current origin. Electron discovery lists eligible local
`file://` targets without a configured file-root allowlist.
Start browser work with `list_browser_targets`; start Electron runtime work with
`list_electron_targets`. Inspect the target selected for the user's task. Observation
is passive: never claim REA clicked, navigated, evaluated page JavaScript,
invoked IPC, captured prior activity, or contained the page's network.
Start passive browser work with `list_browser_targets`; start Electron runtime work with
`list_electron_targets`. Before an electron-main Inspector attempt, check
packaged fuses with `inspect_electron_fuses`. Inspect the target selected for
the user's task. Observation with these tools is passive: never claim REA
clicked, navigated, evaluated page JavaScript, invoked IPC, or captured prior
activity. REA does not contain the page's network.
Credentials, cookies, authorization headers, storage values, and raw
WebSocket/JSON bodies are deliberately absent. Local URLs retain ordinary query
Passive captures omit raw header collections, credential header values, storage
values, and raw WebSocket/JSON bodies. Selected response metadata, including
content encoding, policy values, and Link URLs, is retained. Local URLs retain ordinary query
values and fragments; userinfo credentials are removed. Browser accessibility
text, console primitives, value-free shapes, script sources, source maps, and
storage key names have request-specific capture options. Electron script
@ -21,6 +24,38 @@ sources are also optional. Use `capture_web_screenshot` when pixels are needed.
Treat scope filtering, truncation, and attach-window coverage as limitations.
Page-declared WebMCP tools are untrusted inventory and are never invoked by REA.
## Who performs the action
- The user performs it: `list_browser_targets`, then `inspect_web_page` or
`observe_web_session` on the selected target.
- The agent must perform it: one `capture_browser_scenario` with declared
actions. Use connect mode with the loopback CDP endpoint and `target_id` for
an existing browser, or launch mode with a caller-selected executable for a
temporary profile. Connect mode first navigates the selected page to
`start_url`, which can clear unsaved form values and other temporary state.
Recreate the state needed for the investigation in the declared actions.
Connect mode leaves the external browser open; launch mode owns a temporary
browser and profile. Check `browser.cleanup` for the outcome.
Request only the DOM, accessibility, or network details needed for the question.
The default final capture contains the URL; step outcomes are reported separately.
Add a wait action for the expected page state before capturing it. Report that
REA performed the declared actions.
Scenario network records have no request initiator stacks. For that evidence,
use passive capture while the user performs the action. Listener source locations
identify candidate handlers; they do not establish which script sent a particular
request. Avoid overlapping passive capture and scenario actions on the same target.
Focused follow-up tools:
- `observe_web_execution`: execution coverage during a window; it resets
counters and temporarily disables optimization.
- `inspect_web_event_listeners`: source locations for listeners registered
directly on the first CSS match, without dispatching an event.
- `compare_web_captures`: two normalized passive captures or recorded scenarios.
- `discover_webmcp_tools`: inspect page-declared WebMCP inventory; never invoke
the discovered tools.
Use `reconcile_javascript_runtime` only after static application Evidence and
passive browser/Electron Evidence exist. Prefer captured-source digest identity;
report path/digest disagreement as a mismatch and preserve ambiguity. Explicit

View File

@ -634,6 +634,7 @@ jobs:
tests/boundary/cli/nativeSignatureCli.test.ts
tests/boundary/mcp/nativeSignatureTargetBindingMcp.test.ts
src/process/capture/FilesystemSnapshot.test.ts
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
tests/boundary/process/darwinProcessRunTokenReader.test.ts
tests/boundary/process/directAnalysisSignals.test.ts
tests/boundary/process/processCaptureLifecycle.test.ts
@ -714,6 +715,8 @@ jobs:
- name: Run curated Windows capability tests
run: >-
npx vitest run --coverage.enabled=false
src/process/capture/FilesystemSnapshot.test.ts
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
tests/boundary/filesystem/binaryTarget.test.ts
tests/boundary/filesystem/safeOutputTree.test.ts
tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts

View File

@ -5,9 +5,11 @@ import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.math.BigInteger;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.SocketTimeoutException;
import java.net.StandardProtocolFamily;
import java.net.UnixDomainSocketAddress;
import java.nio.channels.Channels;
@ -298,7 +300,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
Files.move(pendingPath, endpointPath);
}
}
acceptClient(server, descriptor);
acceptAuthenticatedLoopbackClient(server, descriptor);
}
finally {
Files.deleteIfExists(pendingPath);
@ -310,6 +312,50 @@ public final class ReaGhidraBridge extends HeadlessScript {
return System.getProperty("os.name", "").startsWith("Windows");
}
private static String readLoopbackHandshake(
SocketChannel client, int maxBytes, long timeoutMillis) throws IOException {
long deadline = System.nanoTime() + timeoutMillis * 1_000_000L;
InputStream input = client.socket().getInputStream();
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
while (true) {
long remaining = deadline - System.nanoTime();
if (remaining <= 0) throw new SocketTimeoutException("Bridge authentication timed out");
client.socket().setSoTimeout((int) Math.max(1,
Math.min(Integer.MAX_VALUE, (remaining + 999_999L) / 1_000_000L)));
int value = input.read();
if (value == -1) return null;
if (value == '\n') return bytes.toString(StandardCharsets.UTF_8);
if (bytes.size() == maxBytes) throw new IOException("Bridge authentication frame is too large");
bytes.write(value);
}
}
private void acceptAuthenticatedLoopbackClient(
ServerSocketChannel server, SessionDescriptor descriptor) throws Exception {
while (true) {
try (SocketChannel client = server.accept()) {
Request first;
try {
String line = readLoopbackHandshake(client, 64 * 1024, 1_000);
if (line == null) continue;
first = parseRequest(line, descriptor.token);
}
catch (IOException | RuntimeException rejected) {
// A rejected unauthenticated peer cannot consume the bridge session.
continue;
}
client.socket().setSoTimeout(0);
try (BufferedReader reader = new BufferedReader(
Channels.newReader(client, StandardCharsets.UTF_8));
BufferedWriter writer = new BufferedWriter(
Channels.newWriter(client, StandardCharsets.UTF_8))) {
serveClient(descriptor, reader, writer, first);
}
return;
}
}
}
private void acceptClient(
ServerSocketChannel server,
SessionDescriptor descriptor) throws Exception {
@ -320,26 +366,28 @@ public final class ReaGhidraBridge extends HeadlessScript {
BufferedWriter writer = new BufferedWriter(
Channels.newWriter(client, StandardCharsets.UTF_8)
)) {
serveClient(descriptor, reader, writer);
serveClient(descriptor, reader, writer, null);
}
}
private void serveClient(
SessionDescriptor descriptor,
BufferedReader reader,
BufferedWriter writer) throws Exception {
BufferedWriter writer,
Request first) throws Exception {
while (true) {
String line = reader.readLine();
if (line == null) {
return;
}
Request request;
try {
request = parseRequest(line, descriptor.token);
}
catch (RuntimeException exception) {
writeFailure(writer, 1, "invalid_request", "Bridge request is invalid");
return;
Request request = first;
first = null;
if (request == null) {
String line = reader.readLine();
if (line == null) return;
try {
request = parseRequest(line, descriptor.token);
}
catch (RuntimeException exception) {
writeFailure(writer, 1, "invalid_request", "Bridge request is invalid");
return;
}
}
try {
if (request.method.equals("shutdown")) {
@ -887,6 +935,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
JsonObject result = functionObservation(
value == null ? JsonNull.INSTANCE : GSON.toJsonTree(value),
noReturnCallLimitations(
function,
currentProgram.getListing().getInstructions(function.getBody(), true)
)
);
@ -978,7 +1027,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
return functionObservation(
GSON.toJsonTree(renderAssembly(scan.instructions)),
noReturnCallLimitations(scan.instructions)
noReturnCallLimitations(function, scan.instructions)
);
}
@ -997,7 +1046,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
limitations.add(
"The fast path does not invoke the decompiler or scan whole-program names and strings."
);
List<String> boundaryLimitations = noReturnCallLimitations(scan.instructions);
List<String> boundaryLimitations = noReturnCallLimitations(function, scan.instructions);
for (String limitation : boundaryLimitations) limitations.add(limitation);
JsonObject result = new JsonObject();
result.add("procedure", procedureIdentity(function));
@ -1020,7 +1069,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
result.add(canonicalAddress(item.getEntryPoint()));
}
if (callers) {
JsonObject observation = functionObservation(result, noReturnCallLimitations(
JsonObject observation = functionObservation(result, noReturnCallLimitations(function,
currentProgram.getListing().getInstructions(function.getBody(), true)
));
observation.add("relationship_coverage", callRelationshipCoverage(null,
@ -1051,6 +1100,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
return functionObservation(
result,
noReturnCallLimitations(
function,
currentProgram.getListing().getInstructions(function.getBody(), true)
)
);
@ -1571,7 +1621,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
);
if (swift && aarch64) {
limitations.add(
"Swift String literals are decoded from constant register pairs at calls, returns and paired stores; literals stored one word at a time, chosen by conditional selects, passed through unresolved jumps, or built at runtime are not reported."
"Swift String literals are decoded from constant register pairs at calls, returns and paired stores. Literals stored one word at a time, chosen by conditional selects, or built at runtime are not reported; candidate pairs at unresolved jumps are retained as unknown omissions."
);
if (swiftLiterals.omissions().size() > 0) {
limitations.add(
@ -3380,13 +3430,23 @@ public final class ReaGhidraBridge extends HeadlessScript {
return omitted[0];
}
private List<String> noReturnCallLimitations(Iterable<Instruction> instructions)
private List<String> noReturnCallLimitations(
Function function, Iterable<Instruction> instructions)
throws Exception {
List<String> limitations = new ArrayList<>();
Instruction beforePrevious = null;
Instruction previous = null;
for (Instruction instruction : instructions) {
monitor.checkCancelled();
String limitation = terminalCallLimitation(instruction);
if (limitation != null) limitations.add(limitation);
if (beforePrevious != null && previous != null) {
String seh = borlandSehHandlerLimitation(
function, beforePrevious, previous, instruction);
if (seh != null) limitations.add(seh);
}
beforePrevious = previous;
previous = instruction;
}
return limitations;
}
@ -3397,6 +3457,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
throws Exception {
List<String> limitations = new ArrayList<>();
CallBoundaryReferences calls = new CallBoundaryReferences(returnedCallees, includeUnresolvedSites);
Instruction beforePrevious = null;
Instruction previous = null;
if (observedReferences == null) visitReferences(function, "outgoing", calls::observe);
else for (Reference reference : observedReferences) {
monitor.checkCancelled();
@ -3406,6 +3468,13 @@ public final class ReaGhidraBridge extends HeadlessScript {
monitor.checkCancelled();
String limitation = terminalCallLimitation(instruction);
if (limitation != null) limitations.add(limitation);
if (beforePrevious != null && previous != null) {
String seh = borlandSehHandlerLimitation(
function, beforePrevious, previous, instruction);
if (seh != null) limitations.add(seh);
}
beforePrevious = previous;
previous = instruction;
if (!instruction.getFlowType().isCall()) continue;
boolean destination = false;
for (Reference reference : instruction.getReferencesFrom()) {
@ -3423,6 +3492,61 @@ public final class ReaGhidraBridge extends HeadlessScript {
returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls());
}
private String borlandSehHandlerLimitation(
Function function, Instruction handlerPush, Instruction framePush,
Instruction frameLink) {
if (!immediatelyPrecedes(handlerPush, framePush) ||
!immediatelyPrecedes(framePush, frameLink) ||
!handlerPush.getMnemonicString().equalsIgnoreCase("PUSH") ||
handlerPush.getNumOperands() != 1 ||
!framePush.getMnemonicString().equalsIgnoreCase("PUSH") ||
framePush.getNumOperands() != 1 ||
!frameLink.getMnemonicString().equalsIgnoreCase("MOV") ||
frameLink.getNumOperands() != 2 ||
!sameFsMemoryOperand(framePush, frameLink) ||
!hasRegisterOperand(frameLink, 1, "ESP"))
return null;
Object[] handlerOperand = handlerPush.getOpObjects(0);
if (handlerOperand.length != 1 || !(handlerOperand[0] instanceof Scalar scalar) ||
scalar.bitLength() != 32)
return null;
Address handler = currentProgram.getAddressFactory().getDefaultAddressSpace()
.getAddress(scalar.getUnsignedValue());
if (function.getBody().contains(handler)) return null;
return "Ghidra Listing shows a Borland/Delphi-style SEH frame registration at " +
canonicalAddress(frameLink.getAddress()) + " (push " + canonicalAddress(handler) +
" followed by fs-segment stack chaining); handler " + canonicalAddress(handler) +
" lies outside this procedure's body, so callees, strings, and cross-references may omit the handler region.";
}
private static boolean immediatelyPrecedes(Instruction first, Instruction second) {
Address fallThrough = first.getFallThrough();
return fallThrough != null && fallThrough.equals(second.getAddress());
}
private static boolean sameFsMemoryOperand(Instruction first, Instruction second) {
String firstOperand = normalizedFsMemoryOperand(first);
String secondOperand = normalizedFsMemoryOperand(second);
return firstOperand != null && firstOperand.equalsIgnoreCase(secondOperand);
}
private static String normalizedFsMemoryOperand(Instruction instruction) {
String representation = instruction.getDefaultOperandRepresentation(0);
representation = representation.replaceAll("(?i)\\b(?:byte|word|dword|qword)\\s+ptr\\s+", "")
.replaceAll("\\s+", "");
return representation.regionMatches(true, 0, "fs:", 0, 3) &&
representation.indexOf('[') >= 0 ? representation : null;
}
private static boolean hasRegisterOperand(
Instruction instruction, int operand, String name) {
for (Object object : instruction.getOpObjects(operand)) {
if (object instanceof Register register && register.getName().equalsIgnoreCase(name))
return true;
}
return false;
}
private final class CallBoundaryReferences {
private final Set<Function> returnedCallees;
private final TreeMap<Address, String> unresolved;
@ -3827,13 +3951,20 @@ public final class ReaGhidraBridge extends HeadlessScript {
* The canonical empty string is a small literal with a zero count.
*/
private SwiftStringLiteralDecode swiftStringLiterals(Function function) throws Exception {
return swiftStringLiterals(function, new SwiftLiteralBudget());
}
// An explicit budget makes capacity omissions deterministic in producer tests.
private SwiftStringLiteralDecode swiftStringLiterals(
Function function,
SwiftLiteralBudget budget
) throws Exception {
Register[] registers = new Register[31];
for (int index = 0; index < registers.length; index++) {
registers[index] = currentProgram.getRegister("x" + index);
}
TreeMap<String, JsonObject> observed = new TreeMap<>();
TreeMap<String, JsonObject> omitted = new TreeMap<>();
SwiftLiteralBudget budget = new SwiftLiteralBudget();
Map<String, Boolean> selectedValues = new HashMap<>();
SymbolicPropogator propagator = new SymbolicPropogator(currentProgram, false);
propagator.flowConstants(function.getEntryPoint(), function.getBody(), new ContextEvaluatorAdapter() {
@ -3841,6 +3972,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
public boolean evaluateContextBefore(VarnodeContext context, Instruction instruction) {
// Returning true ends only the current flow, so every queued flow checks the shared stop.
if (budget.stopped()) return true;
FlowType flow = instruction.getFlowType();
Address[] flowTargets = instruction.getFlows();
boolean unresolvedJump = flow.isJump() && !flow.isConditional() && flowTargets.length == 0;
for (Register[] pair : swiftStringPairs(function, instruction, registers)) {
Long word = registerConstant(context, pair[0]);
Long object = registerConstant(context, pair[1]);
@ -3890,12 +4024,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
JsonObject item = overCapacity
? swiftStringOmission(layout, word, object)
: swiftStringItem(value, layout, word, object);
if (unresolvedJump && !overCapacity)
item = swiftStringOmission(layout, word, object, "unresolved_jump");
item.addProperty("source_address", canonicalAddress(instruction.getAddress()));
JsonArray pairRegisters = new JsonArray();
pairRegisters.add(pair[0].getName());
pairRegisters.add(pair[1].getName());
item.add("registers", pairRegisters);
(overCapacity ? omitted : observed).put(key, item);
(overCapacity || unresolvedJump ? omitted : observed).put(key, item);
if (overCapacity) {
budget.stop();
return true;
@ -3931,11 +4067,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (operation.getOpcode() == PcodeOp.RETURN) returns = true;
}
// A jump is a proven tail call only when it has resolved targets and all leave the
// body. A targetless computed jump can still dispatch inside the function.
// body. For unresolved targetless jumps, candidate literals remain unknown omissions.
Address[] targets = instruction.getFlows();
boolean tailCall = flow.isJump() && !flow.isConditional() && targets.length > 0 &&
Arrays.stream(targets).allMatch(target -> !function.getBody().contains(target));
if (!flow.isCall() && !returns && !tailCall) return pairs;
boolean unresolvedJump = flow.isJump() && !flow.isConditional() && targets.length == 0;
if (!flow.isCall() && !returns && !tailCall && !unresolvedJump) return pairs;
// A String occupies two consecutive registers among x0-x7, or x0 and x1 when returned.
int lastFirstRegister = returns ? 0 : 6;
for (int index = 0; index <= lastFirstRegister; index++) {
@ -4078,6 +4215,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
remaining = Math.max(0, (runtime.maxMemory() - runtime.totalMemory() + runtime.freeMemory()) / 2);
}
SwiftLiteralBudget(long remaining) {
if (remaining < 0) throw new IllegalArgumentException("Swift literal budget must be nonnegative");
this.remaining = remaining;
}
boolean takeState() {
if (remaining < BYTES_PER_RECORD) return false;
remaining -= BYTES_PER_RECORD;
@ -4116,6 +4258,15 @@ public final class ReaGhidraBridge extends HeadlessScript {
}
private static JsonObject swiftStringOmission(SwiftLiteralLayout layout, long word, long object) {
return swiftStringOmission(layout, word, object, "capacity");
}
private static JsonObject swiftStringOmission(
SwiftLiteralLayout layout,
long word,
long object,
String reason
) {
JsonObject item = new JsonObject();
item.addProperty("representation", layout.representation());
if (layout.address() == null) {
@ -4124,7 +4275,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
item.addProperty("address", canonicalAddress(layout.address()));
}
item.addProperty("byte_length", layout.byteLength());
item.addProperty("reason", "capacity");
item.addProperty("reason", reason);
item.add("words", swiftStringWords(word, object));
return item;
}

View File

@ -736,7 +736,12 @@ def _analyze_function(document, params):
def _dispatch(method, params):
"""Dispatch only the closed operation set implemented by REA's public tools."""
if method == "health":
return {"name": "REA Hopper bridge", "version": "1.0.0", "run_id": REA_RUN_ID}
return {
"name": "REA Hopper bridge",
"version": "1.0.0",
"run_id": REA_RUN_ID,
"process_id": os.getpid(),
}
if method in ("shutdown", "shutdown_document"):
document = _session_document()
if document is None:

View File

@ -41,6 +41,87 @@ class BoundedReader:
return self.handle.read(count)
SPECIAL_SCHEMES = {"ftp", "file", "http", "https", "ws", "wss"}
SCHEME_PREFIX = r"^[\x00-\x20]*([a-z][a-z0-9+.-]*):"
NETWORK_PREFIX = r"^[\x00-\x20]*[/\\]{2}"
URL_IGNORABLE_CONTROLS = "\t\n\r"
URL_CONTROL_TRANSLATION = {ord(char): None for char in URL_IGNORABLE_CONTROLS}
def authority_start(text, base_scheme=None):
"""Return where an authority begins and whether it is special, or None.
Follows WHATWG URL parsing as src/domain/recordedTransportUrl.ts does:
special schemes (and scheme-relative references) skip every leading "/"
or "\\" before the authority, so `https://\\user:pass@host` still has
userinfo. File URLs carry no userinfo and keep their "//" boundary.
"""
scheme = re.match(SCHEME_PREFIX, text, flags=re.I)
name = scheme.group(1).lower() if scheme is not None else None
if name is not None and name in SPECIAL_SCHEMES and name != "file":
# A same-scheme Location without two slash-equivalent separators is
# relative to the recorded request, so its @ belongs to the path.
following = text[scheme.end():scheme.end() + 2]
if name == base_scheme and (len(following) < 2 or any(char not in "/\\" for char in following)):
return None
start = scheme.end()
else:
if scheme is not None:
if text[scheme.end():scheme.end() + 2] != "//":
return None
return scheme.end() + 2, name in SPECIAL_SCHEMES
# A non-special base treats backslashes as authority data and only
# recognizes two forward slashes as a network reference.
if base_scheme is not None and base_scheme not in SPECIAL_SCHEMES:
prefix = re.match(r"^[\x00-\x20]*//", text)
return None if prefix is None else (prefix.end(), False)
prefix = re.match(NETWORK_PREFIX, text)
if prefix is None:
return None
start = prefix.end()
if base_scheme == "file":
return start, True
while start < len(text) and text[start] in "/\\":
start += 1
return start, True
def without_userinfo(value, base_scheme=None):
"""Remove an authority's userinfo using the WHATWG authority boundary.
Special schemes and scheme-relative references also end the authority
at a backslash, so later path text containing "@" is not userinfo.
"""
binary = isinstance(value, bytes)
text = value.decode("latin-1") if binary else value
# WHATWG removes these controls before interpreting scheme/separators.
# Keep that parse view separate from the exact source spelling and bytes.
parsed_text = text.translate(URL_CONTROL_TRANSLATION)
located = authority_start(parsed_text, base_scheme)
if located is None:
return value
start, special = located
terminators = "/?#\\" if special else "/?#"
end = start
while end < len(parsed_text) and parsed_text[end] not in terminators:
end += 1
at = parsed_text.rfind("@", start, end)
if at < start:
return value
parsed_offset = 0
raw_start = 0
for raw_offset, char in enumerate(text):
if char in URL_IGNORABLE_CONTROLS:
continue
if parsed_offset == start:
raw_start = raw_offset
if parsed_offset == at:
safe = text[:raw_start] + text[raw_offset + 1 :]
return safe.encode("latin-1") if binary else safe
parsed_offset += 1
raise CaptureFailure("decoder", "Normalized URL authority could not be bound to its source.")
def project_record(state, secrets):
binaries, numbers, redactions = [], [], []
byte_secrets = []
@ -58,16 +139,23 @@ def project_record(state, secrets):
return None
return text
def transport_value(value, path, transport_url):
def transport_value(value, path, transport_url, base_scheme):
if not (transport_url or re.fullmatch(r"/(?:backup/)*request/(?:path|authority)", path)):
return value, False
binary = isinstance(value, bytes)
pattern = rb"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@" if binary else r"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@"
replacement = rb"\1" if binary else r"\1"
separator = b"@" if binary else "@"
safe_url = re.sub(pattern, replacement, value, flags=re.I)
if path.endswith("/authority") and separator in safe_url:
safe_url = safe_url.rsplit(separator, 1)[1]
if transport_url:
safe_url = without_userinfo(value, base_scheme)
elif path.endswith("/authority"):
safe_url = value.rsplit(separator, 1)[-1]
else:
# Native request.path is origin-form unless it explicitly carries
# a scheme (absolute-form). Leading separators are path evidence,
# not a header URL reference's network authority.
text = value.decode("latin-1") if binary else value
safe_url = without_userinfo(value) if re.match(
SCHEME_PREFIX, text.translate(URL_CONTROL_TRANSLATION), flags=re.I
) else value
changed = safe_url != value
if changed:
redactions.append({"pointer": path, "reason": "transport-credential"})
@ -80,7 +168,7 @@ def project_record(state, secrets):
del numbers[before[1]:]
del redactions[before[2]:]
def visit(value, path, depth=0, credential=False, transport_url=False):
def visit(value, path, depth=0, credential=False, transport_url=False, base_scheme=None):
if depth > 64:
raise CaptureFailure("input-limit", "Native capture exceeds the 64-level evidence nesting budget.", path)
if credential:
@ -90,7 +178,7 @@ def project_record(state, secrets):
binaries.append({"pointer": path, "representation": "producer-bytes", "state": "redacted", "content_base64": None, "bytes": None, "sha256": None})
return None
if isinstance(value, bytes):
safe_bytes, credential_url = transport_value(value, path, transport_url)
safe_bytes, credential_url = transport_value(value, path, transport_url, base_scheme)
content_base64 = base64.b64encode(value).decode("ascii")
digest = hashlib.sha256(value).hexdigest()
hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest or secret in str(len(value)) for secret in secrets)
@ -107,7 +195,7 @@ def project_record(state, secrets):
return None
return safe_text
if isinstance(value, str):
safe_text, _ = transport_value(value, path, transport_url)
safe_text, _ = transport_value(value, path, transport_url, base_scheme)
return redact(safe_text, path, value)
if isinstance(value, bool) or value is None:
return value
@ -125,10 +213,16 @@ def project_record(state, secrets):
for index, field in enumerate(value):
if not isinstance(field, (list, tuple)) or len(field) != 2 or not isinstance(field[0], bytes):
raise CaptureFailure("format", "Native header/trailer is not an ordered byte pair.", pointer(path, index))
result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"})])
result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"}, base_scheme if field[0].lower() == b"location" else None)])
return result
return [visit(item, pointer(path, index), depth + 1) for index, item in enumerate(value)]
return [visit(item, pointer(path, index), depth + 1, base_scheme=base_scheme) for index, item in enumerate(value)]
if isinstance(value, dict):
if path == "" or re.fullmatch(r"/backup(?:/backup)*", path):
request = value.get("request")
scheme = request.get("scheme") if isinstance(request, dict) else None
if isinstance(scheme, bytes):
scheme = scheme.decode("latin-1")
base_scheme = scheme.lower() if isinstance(scheme, str) and re.fullmatch(r"[a-z][a-z0-9+.-]*", scheme, flags=re.I) else None
result = {}
for key, item in value.items():
if not isinstance(key, str):
@ -139,7 +233,7 @@ def project_record(state, secrets):
validate_omitted(item, pointer(path, key), depth + 1)
redactions.append({"pointer": path, "reason": "explicit-sensitive-value", "scope": "property-name"})
continue
result[key] = visit(item, pointer(path, key), depth + 1)
result[key] = visit(item, pointer(path, key), depth + 1, base_scheme=base_scheme)
return result
raise CaptureFailure("decoder", "Native parser returned an unexpected value type: " + type(value).__name__ + ".", path)

View File

@ -0,0 +1,174 @@
"""Source-owned tests for the mitmproxy adapter's record projection."""
import importlib.util
import os
import sys
import types
import unittest
from unittest import mock
def load_capture():
"""Load the adapter with fresh mitmproxy stand-ins scoped to its import."""
fake_mitmproxy = types.ModuleType("mitmproxy")
fake_mitmproxy.ctx = types.SimpleNamespace()
fake_mitmproxy.version = types.SimpleNamespace(VERSION="12.2.3")
fake_io = types.ModuleType("mitmproxy.io")
fake_io.tnetstring = types.ModuleType("mitmproxy.io.tnetstring")
stand_ins = {
"mitmproxy": fake_mitmproxy,
"mitmproxy.io": fake_io,
"mitmproxy.io.tnetstring": fake_io.tnetstring,
}
spec = importlib.util.spec_from_file_location(
"rea_mitmproxy_capture", os.path.join(os.path.dirname(__file__), "capture.py")
)
module = importlib.util.module_from_spec(spec)
with mock.patch.dict(sys.modules, stand_ins):
spec.loader.exec_module(module)
return module
capture = load_capture()
def location(value, scheme=None):
state = {"response": {"headers": [[b"Location", value]]}}
if scheme is not None:
state["request"] = {"scheme": scheme}
record = capture.project_record(state, [])
sidecar = next(
field for field in record["binary_fields"] if field["pointer"] == "/response/headers/0/1"
)
return record["reported"]["response"]["headers"][0][1], sidecar, record["redactions"]
class TransportUserinfoTests(unittest.TestCase):
def test_ignorable_controls_cannot_hide_authority_credentials(self):
for control in [b"\t", b"\n", b"\r"]:
for value in [b"https://" + control + b"/user:pass@host/path", b"ht" + control + b"tps://user:" + control + b"pass@host/path"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"https")
self.assertNotIn("user", text)
self.assertNotIn("pass", text)
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_mixed_network_separators_remove_credentials(self):
for prefix in [b"\\\\", b"/\\", b"\\/"]:
with self.subTest(prefix=prefix):
text, sidecar, redactions = location(prefix + b"user:pass@host/path", b"https")
self.assertEqual(text, prefix.decode("ascii") + "host/path")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_non_special_base_keeps_backslashes_inside_network_authorities(self):
text, sidecar, redactions = location(b"//user\\name:pass@host/x", b"foo")
self.assertEqual(text, "//host/x")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
text, sidecar, redactions = location(b"/\\invoice@customer", b"foo")
self.assertEqual(text, "/\\invoice@customer")
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_native_request_paths_preserve_origin_form_and_redact_absolute_form(self):
for value in [b"/\\invoice@customer", b"//invoice@customer", b"\\\\invoice@customer"]:
with self.subTest(value=value):
record = capture.project_record({"request": {"path": value}}, [])
self.assertEqual(record["reported"]["request"]["path"], value.decode("ascii"))
self.assertEqual(record["binary_fields"][0]["state"], "retained")
self.assertEqual(record["redactions"], [])
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [])
self.assertEqual(record["reported"]["request"]["path"], "https://host/x")
self.assertEqual(record["binary_fields"][0]["state"], "redacted")
def test_same_scheme_locations_preserve_relative_path_text_and_bytes(self):
for value in [b"https:invoice@customer", b"https:/invoice@customer", b"https:\\invoice@customer"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"https")
self.assertEqual(text, value.decode("ascii"))
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_location_authorities_still_remove_credentials_with_a_base_scheme(self):
for value, scheme, expected in [
(b"https://user:pass@host/path", b"https", "https://host/path"),
(b"https:\\\\user:pass@host/path", b"https", "https:\\\\host/path"),
(b"https:user:pass@host/path", b"http", "https:host/path"),
]:
with self.subTest(value=value, scheme=scheme):
text, sidecar, redactions = location(value, scheme)
self.assertEqual(text, expected)
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_backup_location_uses_its_own_request_scheme(self):
record = capture.project_record({
"request": {"scheme": b"http"},
"backup": {
"request": {"scheme": b"https"},
"response": {"headers": [[b"Location", b"https:invoice@customer"]]},
},
}, [])
self.assertEqual(record["reported"]["backup"]["response"]["headers"][0][1], "https:invoice@customer")
self.assertEqual(record["redactions"], [])
def test_backslash_ends_a_special_scheme_authority(self):
text, sidecar, redactions = location(b"https://example.test\\part@tail")
self.assertEqual(text, "https://example.test\\part@tail")
self.assertEqual(sidecar["state"], "retained")
self.assertIsNotNone(sidecar["content_base64"])
self.assertEqual(redactions, [])
def test_userinfo_before_a_backslash_is_still_removed(self):
text, sidecar, redactions = location(b"https://user:pw@example.test\\path")
self.assertEqual(text, "https://example.test\\path")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(
redactions,
[{"pointer": "/response/headers/0/1", "reason": "transport-credential"}],
)
def test_ordinary_userinfo_is_removed(self):
text, _sidecar, _redactions = location(b"https://user:pw@example.test/a@b")
self.assertEqual(text, "https://example.test/a@b")
def test_empty_userinfo_marker_is_removed(self):
self.assertEqual(
capture.without_userinfo("https://@host"), "https://host"
)
def test_special_schemes_skip_leading_slashes_before_userinfo(self):
for value, expected in [
(b"https://\\user:pass@host/path", b"https://\\host/path"),
(b"https:\\\\user:pass@host/path", b"https:\\\\host/path"),
(b"https:user:pass@host", b"https:host"),
(b"//\\user:pass@host", b"//\\host"),
]:
with self.subTest(value=value):
self.assertEqual(capture.without_userinfo(value), expected)
text, sidecar, redactions = location(b"https://\\user:pass@host/path")
self.assertEqual(text, "https://\\host/path")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_file_base_keeps_paths_after_the_fixed_network_boundary(self):
for value in [b"///u@h", b"////u@h"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"file")
self.assertEqual(text, value.decode("ascii"))
self.assertEqual(sidecar["state"], "retained")
self.assertIsNotNone(sidecar["content_base64"])
self.assertEqual(redactions, [])
def test_file_urls_keep_path_text(self):
self.assertEqual(capture.without_userinfo("file:///a@b"), "file:///a@b")
def test_non_special_schemes_keep_backslashes_in_the_authority(self):
self.assertEqual(capture.without_userinfo("foo://a\\b@host/x"), "foo://host/x")
self.assertEqual(capture.without_userinfo(b"//a\\b@host"), b"//a\\b@host")
if __name__ == "__main__":
unittest.main()

View File

@ -44,6 +44,7 @@ export default defineConfig({
link: "/web-network-captures",
},
{ text: "Electron observation", link: "/electron-observation" },
{ text: "Electron fuses", link: "/electron-fuses" },
{ text: "Process capture", link: "/process-capture" },
{ text: ".NET assemblies", link: "/managed-code-analysis" },
{ text: "Android applications", link: "/android-analysis" },

View File

@ -73,32 +73,32 @@ existing analysis families directly, for example
## Operations
| Tool | Kind | What it does |
| ----------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `inspect_adb_client` | read | Resolved binary, version, install path; never contacts the server |
| `list_adb_devices` | read | Serial, state, transport, model metadata, inferred emulator/physical kind with basis |
| `inspect_adb_device` | read | Fixed `getprop` whitelist: build, SDK, patch, fingerprint, product, ABIs, debuggable, `ro.kernel.qemu` |
| `inspect_adb_display` | read | `wm size` and `wm density` as declared |
| `list_adb_features` | read | `pm list features` (hardware/software features, hex GL versions) |
| `list_adb_services` | read | `service list` binder services and interfaces |
| `list_adb_processes` | read | `ps -A`: user, pid, ppid, rss, state, name |
| `inspect_adb_window` | read | `dumpsys window` focus declarations (what is on screen) |
| `read_adb_setting` | read | One `settings get` value; device-side null is reported as such |
| `read_adb_logcat` | read | Bounded `logcat -d -t` dump for one buffer, optional pid filter |
| `list_adb_packages` | read | `pm list packages -f` with scope; base APK device paths |
| `resolve_adb_packages` | read | Case-insensitive package-name substring search with exact-match flag |
| `inspect_adb_package` | read | `dumpsys package` projection: versions, install times, installer, flags, permission count |
| `list_adb_directory` | read | NUL-framed immediate children, including hidden entries: kinds, permissions, owner, group, sizes, symlink targets |
| `pull_adb_package` | read | Complete split APK set from `pm path`, each file digested |
| `pull_adb_file` | read | One absolute device path pulled and digested |
| `capture_adb_screen` | read | One `screencap -p` frame (exec-out, no device-side file) with dimensions |
| `collect_adb_bugreport` | read | Device-generated bugreport zip, digested |
| `push_adb_file` | **mutates device** | One local file to a device path (e.g. pushing tools into `/data/local/tmp`); refuses existing targets without explicit overwrite; verifies via the device's own sha256sum |
| `install_adb_package` | **mutates device** | Local APK via `adb install [-r]`; preserves the device's Success/Failure verdict |
| `uninstall_adb_package` | **mutates device** | Removes a package and its data for user 0 |
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -n`) |
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
| Tool | Kind | What it does |
| ----------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `inspect_adb_client` | read | Resolved binary, version, install path; never contacts the server |
| `list_adb_devices` | read | Serial, state, transport, model metadata, inferred emulator/physical kind with basis |
| `inspect_adb_device` | read | Fixed `getprop` whitelist: build, SDK, patch, fingerprint, product, ABIs, debuggable, `ro.kernel.qemu` |
| `inspect_adb_display` | read | `wm size` and `wm density` as declared |
| `list_adb_features` | read | `pm list features` (hardware/software features, hex GL versions) |
| `list_adb_services` | read | `service list` binder services and interfaces |
| `list_adb_processes` | read | `ps -A`: user, pid, ppid, rss, state, name |
| `inspect_adb_window` | read | `dumpsys window` focus declarations (what is on screen) |
| `read_adb_setting` | read | One `settings get` value; device-side null is reported as such |
| `read_adb_logcat` | read | Bounded `logcat -d -t` dump for one buffer, optional pid filter |
| `list_adb_packages` | read | `pm list packages -f` with scope; base APK device paths |
| `resolve_adb_packages` | read | Case-insensitive package-name substring search with exact-match flag |
| `inspect_adb_package` | read | `dumpsys package` projection: versions, install times, installer, flags, permission count |
| `list_adb_directory` | read | NUL-framed immediate children, including hidden entries: kinds, permissions, owner, group, sizes, symlink targets |
| `pull_adb_package` | read | Complete split APK set from `pm path`, each file digested |
| `pull_adb_file` | read | One absolute device path pulled and digested |
| `capture_adb_screen` | read | One `screencap -p` frame (exec-out, no device-side file) with dimensions |
| `collect_adb_bugreport` | read | Device-generated bugreport zip, digested |
| `push_adb_file` | **mutates device** | One local file to a device path (e.g. pushing tools into `/data/local/tmp`); checks for existing targets and refuses them without explicit overwrite (the check is not atomic); compares digests when device `sha256sum` is available |
| `install_adb_package` | **mutates device** | Local APK via `adb install [-r]`; preserves the device's Success/Failure verdict |
| `uninstall_adb_package` | **mutates device** | Removes a package and its data for user 0 |
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -n`) |
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
## Interpret the result
@ -118,6 +118,16 @@ existing analysis families directly, for example
while POSIX hosts retain valid POSIX filenames such as names with spaces or
colons. A package pull checks reported names against the destination
filesystem before pulling, and refuses names that would share a local path.
- **Package path refusals are diagnosable.** Refusals from `pull_adb_package`
return `unreadable_output` with `retryable: false`, keep the `pm path` output
in `details.captured_output`, and name the refusal in `details.reason`.
`unsafe_device_path` lists refused entries in
`details.diagnostics.rejected_paths` as `{device_path, reason}`;
`duplicate_local_destination` reports the shared `file_name` and both
`device_paths`; `local_destination_unavailable` reports the `device_path`,
`file_name`, and filesystem `error_code`. Diagnostics retain raw device
strings, while the caller-facing message escapes invisible and bidirectional
characters so they cannot hide text or reorder the path.
- **Splits are part of the answer.** `pm path` enumerates the base APK and
every installed split; pulls cover the whole set. System packages whose
APKs keep other names report the `unknown` role rather than guessing.
@ -127,8 +137,10 @@ existing analysis families directly, for example
partial coverage instead of dropping rows.
- **Mutations are explicit.** `push_adb_file`, `install_adb_package`, and
`uninstall_adb_package` declare `mutatesTarget` and never run implicitly:
pushes refuse existing device files unless `overwrite` is set and verify
the transfer with the device's own `sha256sum`; installs preserve the
pushes check for existing device files before transfer and refuse them
unless `overwrite` is set. The check is not atomic, so a different process
can create the path before `adb push` writes it. When available, the device's
`sha256sum` verifies the transfer; installs preserve the
device's `Failure [...]` reason verbatim.
- **Producer identity stays attached.** Every result embeds the producing
adb client (`binary_path`, `path_source`, `version`, `installed_path`).

View File

@ -18,7 +18,7 @@ Inspect native SAZ or HAR 1.2 offline with REA, or connect Everywhere's separate
- `inspect_web_page` / `rea inspect-web-page` captures DOM structure, accessibility nodes, scripts, resources, safe response/DOM metadata, attach-window network and console metadata, WebSocket frame sizes, workers, quota, and optionally selected storage key names, redacted storage-content fingerprints, or script sources.
- `analyze_web_bundle` / `rea analyze-web-bundle` parses selected script artifacts without execution and derives chunk edges, route and endpoint candidates, vendor fingerprints, static WebMCP declarations, and optionally source-map/original-source evidence.
- `observe_web_session` / `rea observe-web-session` arms an observation window of the requested duration for an external user action and records ordered reload, SPA navigation, redirect, failure, lifecycle, and target-termination metadata.
- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. REA never exposes `WebMCP.invokeTool`; malformed protocol payloads fail explicitly instead of returning incomplete tool metadata.
- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. Declared `readOnly`, `untrustedContent`, `consequential`, and `autosubmit` annotation hints are reported per registration: `true`/`false` are declared values, `null` means the declaration did not provide the hint in that capture, and a missing `consequential` key means the hint was not recorded, either in a historical capture or because the observed annotation was invalid. Consult discovery completeness and exclusions to distinguish these cases. Malformed annotation values retain the registration with incomplete coverage and an `invalid_protocol_value` exclusion. Other malformed protocol payloads can fail discovery explicitly. REA never exposes `WebMCP.invokeTool`.
- `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence.
- `capture_web_screenshot` / `rea capture-web-screenshot` returns a content-addressed visible-viewport PNG.
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
@ -190,6 +190,36 @@ rea discover-webmcp-tools http://127.0.0.1:9222 TARGET_ID \
--json
```
### WebMCP discovery prerequisites
`discover_webmcp_tools` reports only what the attached browser's experimental
`WebMCP` CDP domain exposes. When a browser rejects the unknown domain, the
result reports `status: "unavailable"`. Whether a page can register tools at
all depends on several conditions that are not interchangeable:
- A site's own origin-trial token can enable the API for that origin
**provided the trial is still active, the browser version and platform
support it, and user settings permit it** — token presence alone is not a
guarantee. The trial is announced for Chrome 149 and later.
- The `WebMCPTesting` feature — `--enable-features=WebMCPTesting` at launch, or
`chrome://flags/#enable-webmcp-testing` in a normal profile — enables the
API page-side for that browser instance, independent of per-site tokens.
In one pilot run, a site whose registration code was present but whose
served token was scoped for third-party use registered 0 tools under a
default launch and 4 under the flag; that the token did not enroll the
first-party document is consistent with this but is an inference, not a
measured Chrome behavior.
In Chrome 154 specifically (one tested version), `WebMCP.enable` answered
under a default launch — the debug domain was not flag-gated in that run.
Other versions may differ, so an unattributed zero-tool result should be
read as registration state only after domain availability is confirmed.
Discovery is attach-only: the command needs a reachable loopback CDP endpoint
and a target ID from `list_browser_targets`, and it never navigates, reloads,
or evaluates the page. Observe the page in the state you want to measure;
registered tools reflect the document as loaded.
Accessibility names/descriptions, console primitive text, JSON body shapes, WebSocket shapes, script content, and storage key names are selected independently:
```bash

View File

@ -15,6 +15,12 @@ there is no separate origin allowlist or request-interception layer. Environment
settings (including service workers blocked by default), empty storage, and a
final sanitized URL capture are supplied by default. Set
`environment.service_workers` to `allow` when the application requires them.
For attached browsers, `block` rejects captures when the selected page or its
frames are controlled by a service worker or already have a registration,
including after navigation. Existing caller registrations are never removed.
Navigation can run destination code before a newly discovered worker is rejected;
that rejection does not roll back browser effects. REA removes its owned
future-document scripts, restores live frame methods, then disconnects.
Add secret declarations, storage seeds, and additional artifact or event
capture only when needed. Action, secret, and storage counts are not capped.
Duration, action, and navigation
@ -249,8 +255,12 @@ Each observed request object receives a capture-scoped `transaction_id`.
Requests sharing a URL remain distinct. Redirects link to the preceding
transaction only when it was observed; a null predecessor does not establish
that no earlier hop existed. Request events require both transaction fields;
an absent ID is malformed capture data, while a null redirect predecessor is
an explicit unknown.
an absent ID is malformed capture data. `redirected_from_transaction_state`
records `root` when Playwright observed no predecessor and `unknown` when a
predecessor exists but was outside the capture. A linked predecessor is carried
by `redirected_from_transaction_id` with no state field. Older captures without
the state field retain the conservative meaning that a null predecessor is
unknown.
`scenario.network_content` records the resolved content selection. Comparing
captures with different selections reports network evidence as unknown with a
@ -284,8 +294,10 @@ browser cleanup, with a provider-owned five-second timeout per read and
cancellation support. Failed or unexposed reads retain per-content states and make selected
event coverage incomplete. Body events can arrive during later steps or final
drain; event bounds describe receipt during the step, while the source event
reference establishes the transaction phase. Comparison attributes content to
its source event's step and excludes the sequence reference from normalization.
reference establishes the transaction phase. Comparison attributes content, like
every other event of its transaction, to the step that sent the request (the
receipt step when that request was not observed) and excludes the sequence
reference from normalization.
Playwright scenario capture does not supply initiator stacks; passive CDP
observation retains its own initiator evidence.
@ -300,7 +312,22 @@ an unchanged claim; observed differences may still prove a changed result.
The comparison records its full normalization commitment and SHA-256 digest.
Built-in rules exclude step elapsed time and event sequence/index fields,
compare screenshots by content digest, and compare DOM/accessibility captures
by normalized text. Optional caller rules are bounded exact-literal
by normalized text. Network events are grouped by transaction without their
capture-local transaction IDs, attributed to the step that sent the request,
and compared as an unordered collection after caller rules apply, because
concurrent responses arrive in any order and can land in a later step; a
redirect retains the complete predecessor ancestry, including each hop's own
request, response outcome, and selected network content. Each transaction's
capture-local IDs, sequence/index fields, and content source-sequence anchor
are removed. Caller rules normalize retained strings before hashing. The
`preserve_redirect_transaction_ancestry` built-in rule fingerprints each hop
as canonical SHA-256 of `{ events, redirected_from }`: `events` is the sorted
list of its normalized event SHA-256 digests, and `redirected_from` is its
predecessor fingerprint or null for a root. Selected transaction fingerprints
are sorted for the step's network comparison. Temporary numeric graph indices
are lookup references only and do not enter the final digest. Missing or cyclic
predecessor evidence yields unknown comparability instead of an unchanged claim.
Other events keep receipt order. Optional caller rules are bounded exact-literal
replacements over named artifact kinds. Rules and artifact-kind lists are
sorted canonically before application, making the same policy reproducible
regardless of input order. Rules operate only on already-redacted durable
@ -318,3 +345,7 @@ selection. Captures with different selections report event evidence as unknown
with a recovery instruction, even when no selected family emitted an event.
Captures written before this field existed omit it, so their event evidence is
also unknown.
Network normalization also commits the removal of capture-local transaction IDs,
attribution to the requesting step, and unordered transaction comparison. These
built-in rule identities participate in the normalization digest.

View File

@ -263,7 +263,12 @@ tree matches that commit remains unknown. It honors the source tree's `.gitignor
and applies default exclusions for common generated and dependency paths such
as `node_modules/`, `dist/`, and `*.log`. Pattern-based exclusions record the
matched pattern and whether it came from project, default, caller, or sensitive
path policy. Set `REA_REFERENCE_SECRET_PATTERNS_JSON` to a JSON array of ignore
path policy. Links into excluded paths retain their internal target metadata;
the exclusion explains why the target has no inventory entry, and the inventory
remains partial. Exclusions record `entry_kind` when the importer observed it;
older saved exclusions without that field have unknown kind. An excluded
ancestor proves an internal descendant only when its kind is `directory`.
Set `REA_REFERENCE_SECRET_PATTERNS_JSON` to a JSON array of ignore
patterns when you want to mark selected paths as sensitive; those patterns are
reported separately as configured-secret exclusions.
@ -287,6 +292,7 @@ Choose the guide for the target and intended interaction:
- [Browser observation](browser-observation.md): inspect a selected page through CDP.
- [Browser scenarios](browser-scenario-contract.md): run declared interactions and capture their results.
- [Electron observation](electron-observation.md): inspect an Electron renderer or capture an application scenario.
- [Electron fuses](electron-fuses.md): read packaged fuse bytes before expecting `--inspect` to reach an Electron main process.
- [Node/Electron Inspector](javascript-runtime-observation.md): record script locations and execution contexts.
- [Process capture](process-capture.md): run an executable and compare terminal, exit and filesystem observations.

View File

@ -129,6 +129,12 @@ and Claude Code reports that autocompact is thrashing. To avoid this:
The sizes above come from REA's own `tools/list`. A live Claude Code session
against DeepSeek's API is unverified.
For a client whose stdio reader rejects the complete catalog frame, add
`"REA_MCP_OUTPUT_SCHEMA_PROFILE": "off"` to REA's `env` object and restart the
client. This omits only advertised output schemas; REA still validates complete
structured results against its canonical contracts. See [MCP output schema
profiles](mcp-contracts.md#output-schemas).
## Kimi Code
Setup registers `rea` in `~/.kimi-code/mcp.json`, or
@ -294,7 +300,11 @@ the first native server into an empty table, it copies the fallback's
`mcpServers` entries into the native table so they keep loading, and leaves
the fallback file untouched. Existing REA environment overrides are preserved
when refreshing its command. A malformed or unreadable fallback file fails setup instead of
being silently suppressed; repair or remove it and rerun. ZCode disables a
being silently suppressed; repair or remove it and rerun. Uninstall removes an
REA-owned `rea` entry from both the native table and the fallback, backing up
each changed file, so an emptied native table cannot load a leftover fallback
registration; other fallback servers, settings and a foreign `rea` entry are
kept. ZCode disables a
server with `enable: false` on its entry; doctor reports such a registration
as stale, and rerunning setup removes the flag to repair it. Native configuration
uses strict JSON; setup refuses comments, trailing commas and a BOM rather than

82
docs/electron-fuses.md Normal file
View File

@ -0,0 +1,82 @@
# Electron fuse state
`inspect_electron_fuses` / `inspect-electron-fuses` answers whether a packaged
Electron app will honor `--inspect` and `--inspect-brk`, `ELECTRON_RUN_AS_NODE`,
embedded ASAR integrity validation, and loading application code only from
`app.asar`. It reads the fuse wire stored in the Electron binary. It does not
launch the app, modify fuse bytes, or use the network.
```bash
rea inspect-electron-fuses /Applications/Example.app --json
```
The CLI resolves a relative path against the caller's working directory. MCP
requires an absolute filesystem path and preserves that exact selection:
```json
{
"name": "inspect_electron_fuses",
"arguments": { "path": "/Applications/Example.app" }
}
```
Check this before `observe_javascript_runtime` with `electron-main`. When
`EnableNodeCliInspectArguments` is disabled or removed, Electron ignores
Inspector flags, and a later target listing can simply show no main-process
target.
## What is read
On macOS, pass the `.app` bundle or the Electron Framework binary. Bundle and
framework selection first tries the top-level `Electron Framework` binary,
then `Versions/Current/Electron Framework`, then
`Versions/A/Electron Framework`. Symlinks are followed only while they stay
inside that framework.
On Windows and Linux, pass the main Electron executable. The wire lives in
that executable rather than in `app.asar`.
The inline Evidence result contains:
- the selected path, the resolved regular file, its SHA-256, size, and container
format (`mach-o`, `mach-o-universal`, `pe`, or `elf`);
- one slice entry for a thin Mach-O, PE, or ELF file, and one entry per slice of
a universal Mach-O, with file-absolute `sentinel_offset` values;
- `wire_version`, `declared_length`, and ordered fuses `{ index, name, state, raw_byte }`;
- `sentinel_count`, including zero and more than one;
- `runtime_implications`, each marked `relationship: "derivation"` and tied to
the fuse index it came from.
Version 1 names, in index order, are `RunAsNode`, `EnableCookieEncryption`,
`EnableNodeOptionsEnvironmentVariable`, `EnableNodeCliInspectArguments`,
`EnableEmbeddedAsarIntegrityValidation`, `OnlyLoadAppFromAsar`,
`LoadBrowserProcessSpecificV8Snapshot`, `GrantFileProtocolExtraPrivileges`,
and `WasmTrapHandlers`. An index past that table, or any index on another wire
version, keeps `name: null` and the raw byte.
Wire bytes are `0x30` disabled, `0x31` enabled, and `0x72` removed. Any other
byte is `unknown`. A removed fuse is not enabled: Inspector flags and
`ELECTRON_RUN_AS_NODE` are not honored, and application code is not restricted
to `app.asar`. On macOS and Windows, a removed or disabled
`EnableEmbeddedAsarIntegrityValidation` byte means embedded ASAR integrity is
not enforced, and an enabled byte means it is enforced. Electron compiles that
validation only on macOS and Windows. On a Linux ELF executable the observed
byte is preserved and the enforcement outcome is `unknown`.
## Unknown and multiple sentinels
No sentinel means the fuse state is unknown. It does not mean the fuses are
disabled. A wire that continues past the slice is reported as truncated, with
only the fuse bytes that were present. Two or more sentinels in one slice are
all reported. REA does not pick one.
Fuse bytes do not prove that the application enables Inspector some other way,
for example through `app.commandLine` or a remote-debugging switch in
application code.
The selected file must be a regular file, or a bundle that resolves to one.
Symlinks are followed only inside `Electron Framework.framework`. The reader
admits at most 1 GiB, searches with 256 KiB windows, and keeps at most 32
sentinels per slice. A real nine-fuse, one-slice result is about 1,200 CLI
tokens; a two-slice result is about 1,300 tokens. Exact size varies with path
lengths and limitations. This inspection creates no temporary files and does
not modify the selected bytes.

View File

@ -7,6 +7,10 @@ This passive runtime surface is distinct from the target-free static
workflow. Static analysis reads a local ASAR or extracted directory without
executing application code. Passive observation inspects an existing Electron
page. The results keep static inferences and runtime observations distinct.
Packaged fuse state is a separate static read:
[`inspect_electron_fuses`](electron-fuses.md) reports whether this binary will
honor Inspector flags, `ELECTRON_RUN_AS_NODE`, embedded ASAR integrity, and
`app.asar`-only loading.
Use the separate
[`reconcile_javascript_runtime`](javascript-runtime-reconciliation.md) workflow
when both Evidence sets already exist.

View File

@ -103,6 +103,23 @@ build's source revision, so that field remains unknown.
engine finished its configured extraction without those reported conditions;
default skip rules and format coverage still apply. It does not establish
complete firmware reconstruction.
- `extraction_gaps` makes those conditions recoverable: a row identifies an
unextracted region or a failed task/stage. A chunk retains its identity and
byte range, plus its original-file range when it belongs to the selected input.
A multi-file stage retains its reported ID and handler with unknown byte ranges;
a standalone task failure keeps unavailable stage identity null. Rows retain exact dependency names from
the producer's missing-dependency reports; an accepted report with no names
remains a failure with unknown dependencies; no executable identity or specific installation step is inferred. Each row's `continuation_id`
references one instruction in `extraction_continuations`, so many regions
share recovery prose without losing any region identity. Instructions use
already configured capabilities where one exists. Every extraction rerun
requires a fresh absent `output_directory`, preserving the prior evidence.
Nested unknown chunks use `inspect_firmware_regions` on the actual published
child rather than repeating the same extraction with unchanged handlers.
Inspection requires separately configured Binwalk (`REA_BINWALK_COMMAND`);
without it, the inspection step is unavailable and the region remains unexamined. A listed gap does not
mean the target is corrupt or that a retained region is empty; REA installs
nothing.
- The destination must be absent, absolute, with an existing parent. Publication
uses exclusive, symlink-resistant writes and durable digest readback. Existing
destinations are never overwritten. This is not an atomic directory rename:
@ -167,6 +184,12 @@ records gzip offset/length and extracted file hashes. Real verification checks
CLI/MCP, full-file and selected-range extraction, unknown chunks, depth limits,
content parity and unchanged source bytes. No binary fixture is committed.
The optional missing-extractor lane asserts dependency names and the owning
handler in `extraction_gaps`. Those assertions must be run against the pinned
Unblob release to establish real-provider coverage; synthetic producer fixtures
alone do not verify Unblob's report nesting or stage identities. The gap ownership
claims remain unverified against real Unblob until that lane runs.
Additional lanes are explicit, so the base lane stays small:
```sh

View File

@ -52,13 +52,15 @@ section names or the record being inside the initial 4 KiB header probe.
Program and section declarations must agree; duplicate, conflicting, truncated
or out-of-file records are rejected. Stripped section tables and section-only
records are supported. Extended table numbering is explicitly unsupported.
For carved ELFs whose section table extends beyond the captured file, REA
examines every complete captured section header and records the unavailable
headers as a limitation. A valid program-header ABI record can still establish
the supported declaration. Available section declarations must agree with it;
their duplicate, conflicting or unreadable records remain errors. A missing
program-header record does not make ABI declarations in uncaptured sections
absent or supply a default ABI.
GNU `SHT_GNU_ATTRIBUTES` sections are refused by numeric section type until REA
parses and reconciles their declarations; section names and payload sizes do not
control detection or allocation. For carved ELFs whose section table extends
beyond the captured file, REA examines every complete captured section header
and records the unavailable headers as a limitation. The provider refuses an
incomplete ABI declaration inspection even when a program-header ABI record is
valid, because an unavailable section header could carry an unsupported or
conflicting declaration. A missing program-header record does not make ABI
declarations in uncaptured sections absent or supply a default ABI.
The field layout follows
[`Elf_MIPS_ABIFlags_v0`](https://github.com/bminor/glibc/blob/master/elf/elf.h).

View File

@ -154,12 +154,15 @@ An inferred no-return call can terminate that body before subsequent bytes.
Pseudocode, assembly, procedure info, instruction windows, callee and caller
reads, and dossiers report observed terminal call sites in Evidence limitations,
including resolved callee names, addresses, and FunctionManager flags when
available. Instruction windows
and dossiers also retain these warnings in their result limitations. An
unresolved callee or unestablished terminal-flow basis remains explicit. These
observations qualify the analysis; they do not prove that a no-return flag is
incorrect or recover excluded code. Analysis profiles version this observation
contract so older snapshots cannot replay unqualified results.
available. On x86-32, the same views also report the exact Borland/Delphi-style
FS exception-registration pattern when its handler address falls outside the
function body. This is a static registration observation; it does not establish
handler execution or change the body. Instruction windows and dossiers also
retain these warnings in their result limitations. An unresolved callee or
unestablished terminal-flow basis remains explicit. These observations qualify
the analysis; they do not prove that a no-return flag is incorrect or recover
excluded code. Analysis profiles version this observation contract so older
snapshots cannot replay unqualified results.
On non-Windows hosts, the ephemeral import repair clears no-return flags on the
known returning external ABI names `malloc`, `__tls_get_addr`, `memcpy`, `memset`,
@ -189,7 +192,7 @@ host/provider acceptance run with only the host compiler.
`npm run verify:ghidra:windows` checks source-owned native x86-64 PE applications;
`-- --x86` selects x86 and `-- --dll` selects DLLs, and the flags can be combined.
These lanes check all 25 read-only operations, target/snapshot/import SHA-256 linkage,
These lanes check all 26 read-only operations, target/snapshot/import SHA-256 linkage,
and project, endpoint, process, and runtime cleanup. The independent native
lane checks DACLs, handle admission, cancellation, and Job Object lifecycle.
`npm run verify:ghidra:windows:package` packs and installs REA into an isolated

View File

@ -58,6 +58,12 @@ original stable file snapshot, including pointer-backed strings. It does not
scan arbitrary byte strings as compiler metadata or interpret a fat Mach-O
container as one selected architecture.
For ELF images without a named build-info section, discovery checks each
nonempty writable, non-executable load segment in program-header order. This
includes sectionless PIE images with metadata in a later data segment. Named
sections retain priority, and alignment, file mapping and record validation
remain required.
Historical moduleless Go binaries can store their empty module-string header in
declared zero-initialized virtual memory rather than file-backed bytes. The reader
recognizes this only when the complete optional module header lies in that memory
@ -78,9 +84,13 @@ host permission denial, changed artifacts and cancellation retain distinct
errors. The selected file is never modified; this inspection creates no
temporary files, starts no subprocesses and accesses no network endpoint.
Resource guards bound the selected file to 256 MiB, aggregate embedded strings
Resource guards bound the selected file and cumulative header searches to 256 MiB
each, aggregate embedded strings
to 1 MiB, structural table decoding to 16 MiB and the complete normalized metadata
report to 8 MiB. Oversized results fail without partial success. MCP also applies
report to 8 MiB. Identical search spans with the same virtual alignment are
searched once; partially overlapping spans consume the cumulative search budget
and fail with a CPU resource constraint when it is exhausted. Oversized results
fail without partial success. MCP also applies
its [response transport budget](mcp-contracts.md), which counts both text and
structured content; a report within the metadata limit can still require export
from retained Evidence. These limits are not operating-system memory or CPU

View File

@ -6,7 +6,9 @@ Choose one lifecycle mode in an upstream MCP JSON registration. Set `REA_IDA_MCP
## Supported upstream versions
Each mode accepts one upstream tool profile, checked against `tools/list` before any analysis runs. Upstream's `main` branch has since moved to 2.0.0, which renamed the GUI tools and dropped `idb_close`, so it serves neither profile. Installing from that branch fails at startup with the missing tool names in the error diagnostics.
Each mode accepts one upstream tool profile, checked against `tools/list` before any analysis runs. The attached profile requires the legacy GUI tool names; upstream 2.0.0 renamed those tools, and REA has not established an equivalent attached-mode lifecycle and ownership contract for that catalog. It therefore fails closed with the missing tool names in the error diagnostics.
The headless profile is defined by the database-supervisor API, not by an upstream version string. The upstream source snapshot [`c133c385`](https://github.com/mrexodia/ida-pro-mcp/tree/c133c3853faa111a9b00ee615c013b720d0c4acd) includes `idb_close` (introduced by [upstream PR #495](https://github.com/mrexodia/ida-pro-mcp/pull/495)) alongside `idb_open` and `idb_list`. Earlier builds may report the same `2.0.0` version while lacking that tool, so REA checks the live catalog rather than treating a version number or branch name as a compatibility guarantee.
| Mode | Upstream profile | Known working |
| -------- | ------------------- | ------------------------------------------------------ |
@ -137,11 +139,11 @@ For guided registration, set `REA_IDA_MCP_CONFIG` before `rea setup --client <cl
All profiles support procedure/string inventory and literal or regex search, procedure resolution, pseudocode, assembly, raw function instructions, resolved callees, incoming address references, and a function dossier. Inventories and modern pseudocode/assembly follow upstream pagination. Upstream limits on modern xrefs and callees are retained as explicit truncation limitations. In the modern headless profile, targets that are not proven x86-only use processor-neutral `callgraph` edges instead of the x86 instruction checks in upstream `callees` (mrexodia/ida-pro-mcp#549). REA requests depth zero: the pinned upstream emits the root's outgoing edges without scanning child functions, then REA batches function-name lookup for those addresses. The graph covers recognized function destinations only; imports and other non-function targets remain unknown. A server without `callgraph` returns an empty relationship list with an explicit coverage limitation. Graph and path workflows retain those limitations alongside observed edges. A failed or inconsistent graph reply remains an error. A transport-level truncated preview is rejected rather than reported as complete evidence; the error preserves upstream retrieval guidance.
The modern headless profile also supports `resolve_containing_procedure` and `list_documents`. An address resolves through upstream `lookup_funcs` to the canonical entry of the function that contains it; the body stays unavailable like other IDA identities. When no function contains it, REA reads the database-scoped minimal `survey_binary` and reports `not_in_procedure` for an address inside one of its segments (ends are exclusive) and `outside_segments` otherwise. A missing, truncated or inconsistent survey fails the read instead of guessing a reason. `list_documents` returns the single admitted program path, so `inspect_address_context` and `trace_feature` need no explicit document. `trace_feature` marks a trace truncated, with a residual unknown for that address, when upstream caps the references it returned. The legacy and native profiles do not declare these operations.
All profiles expose `list_documents` as the single target admitted for the session. Attached mode verifies its digest against IDA metadata before returning it; headless mode binds the digest-checked private input. `inspect_address_context`, `trace_feature`, and `binary_overview` can therefore resolve that identity without upstream document-listing tools. Segment enumeration is optional: when the selected profile cannot provide it, `binary_overview` still returns document and procedure/string counts and omits `segments` and `segment_count`; omission means unobserved, not empty. The modern headless profile also supports `resolve_containing_procedure`. An address resolves through upstream `lookup_funcs` to the canonical entry of the function that contains it; the body stays unavailable like other IDA identities. When no function contains it, REA reads the database-scoped minimal `survey_binary` and reports `not_in_procedure` for an address inside one of its segments (ends are exclusive) and `outside_segments` otherwise. A missing, truncated or inconsistent survey fails the read instead of guessing a reason. `trace_feature` marks a trace truncated, with a residual unknown for that address, when upstream caps the references it returned. Attached profiles do not declare address containment resolution.
The legacy profile additionally supports direct callers. Its producer may report call-site or interior addresses; REA resolves the reported function symbol to its canonical entry. The modern and native profiles do not prove direct callers from its code/data xref labels, so that capability is unavailable and the dossier marks it unknown.
Complete body ranges, typed reference edges, CFG, referenced data, source locals, comment kinds, and database revision are unavailable. Empty unsupported dossier facets mean **unknown**, not observed absence. `binary_overview`, whole-symbol inventory, mutation, debugging, arbitrary Python forwarding, and requested GUI navigation are outside this adapter's coverage. Check session availability before composing a workflow that needs those facets.
Complete body ranges, typed reference edges, CFG, referenced data, source locals, comment kinds, and database revision are unavailable. Empty unsupported dossier facets mean **unknown**, not observed absence. Whole-symbol inventory, mutation, debugging, arbitrary Python forwarding, and requested GUI navigation are outside this adapter's coverage. Check session availability before composing a workflow that needs those facets.
`provider.version` identifies the REA adapter; the profile explicitly records `version_scope: "rea-ida-adapter"`. The MCP handshake's `serverInfo` is separate and does not establish IDA or Python distribution versions. Raw observations preserve producer metadata and output. Attached hashes identify the recorded original input; headless identity uses the digest-verified private copy and observed worker input path. Neither proves source-code recovery or an unmodified analysis database.

View File

@ -52,7 +52,7 @@ For runtime instrumentation, see the [Frida guide](./frida-instrumentation.md).
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [JEB](jeb-analysis.md) |
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md) |
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
| Inspect assemblies, packages, or firmware | [.NET](managed-code-analysis.md), [Android](android-analysis.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |

View File

@ -64,11 +64,21 @@ preserves a verified `.rea.backup.<sha256>.recovery` copy of the current bytes.
Both snapshots remain available; a failed rollback reports the current recovery
copy's path and the restore failure.
Setup also retains valid environment overrides in REA's existing server entry,
such as `REA_MCP_INPUT_SCHEMA_PROFILE` and `REA_MCP_MAX_RESPONSE_BYTES`.
such as `REA_MCP_INPUT_SCHEMA_PROFILE`, `REA_MCP_OUTPUT_SCHEMA_PROFILE`, and
`REA_MCP_MAX_RESPONSE_BYTES`.
Detected provider settings replace only matching environment keys.
Setup and doctor report a malformed REA environment map or invalid runtime
setting before setup writes files. Repair the named setting in the client
configuration; setup does not silently discard the rest of the map.
This includes malformed loader/exclusion JSON arrays and a Ghidra compiler
specification without its required language identifier. Valid overrides keep
their original strings, including empty Hopper arguments. DeepSeek Harness
`!!js` environment expressions remain unevaluated and preserved; registration
alignment does not verify the runtime values those expressions will produce.
For layered OpenCode files, each file's values are checked before merging, while
setting dependencies are checked against the effective merged environment.
Setup preserves each layer's overrides instead of copying inherited settings
into the file it updates.
After setup, REA reports which agents, analysis tools, and workflow files passed
its final checks. Restart any agent named in the completion message, then begin
@ -129,6 +139,150 @@ Setup pins package-runner MCP registrations to the exact installed REA version,
installs the matching skill and on-demand references in the same plan, and adds
`startup_timeout_sec = 30` for Codex and Grok Build.
## Limit REA to one project or agent profile
Setup registers REA for the whole client. For Claude Code it writes
`mcpServers.rea` in `~/.claude.json` and installs the bundled skill under
`~/.claude/skills`, so every project that Claude Code opens can call REA. To
offer REA's tools only where they're wanted, use one of the routes below.
Both routes control which sessions discover REA's MCP server and skill. They
leave shell commands, filesystem access and network access available. An agent
under the same OS account can run `rea` from `PATH` or `npx` and read other
profiles and REA output. For clean-room separation, use an implementation
environment with filesystem and network restrictions that prevent reading or
fetching REA, the target and analysis output, including through shared
installations and package registries. A separate account or container must
enforce those restrictions.
For Claude Code, check existing default-profile state before choosing either
route. Setup is additive: a previous user-level REA registration and personal
skill remain active in other projects. Back up the default configuration and
REA skill first, then, with `CLAUDE_CONFIG_DIR` unset, remove the REA user
registration with `claude mcp remove --scope user rea` and remove the known
REA-installed `~/.claude/skills/reverse-engineer-anything` directory. Preserve
unrelated registrations and skills. `rea uninstall` applies to other clients
as well, so it is broader than this Claude-only migration.
**A separate Claude Code profile.** Claude Code keeps settings, skills and session
history under `CLAUDE_CONFIG_DIR`. Setup follows the same
variable: it writes the registration to `$CLAUDE_CONFIG_DIR/.claude.json` and
the skill to `$CLAUDE_CONFIG_DIR/skills`.
```bash
CLAUDE_CONFIG_DIR=~/.claude-rea npx rea-agents setup --client claude_code
CLAUDE_CONFIG_DIR=~/.claude-rea claude
```
A new profile may need authentication. Console sign-ins without an API key are
stored outside `CLAUDE_CONFIG_DIR`; see
[Claude Code authentication](https://code.claude.com/docs/en/authentication#log-in-with-multiple-accounts).
Sessions started without the variable use the default profile's registration,
skill and history.
Inline assignments apply only to that command. Keep the same prefix for every
profile check, setup rerun, update, or removal; bare commands use the current
environment and can otherwise inspect or recreate default-profile state:
```bash
CLAUDE_CONFIG_DIR=~/.claude-rea claude mcp list
CLAUDE_CONFIG_DIR=~/.claude-rea npx rea-agents doctor --client claude_code --skill --json
CLAUDE_CONFIG_DIR=~/.claude-rea npx -y rea-agents@latest setup --client claude_code --dry-run
```
After reviewing an update plan, rerun that same prefixed setup command without
`--dry-run` and approve its exact changes. Profile removal likewise needs the
prefix on `claude mcp remove --scope user rea` and any `rea uninstall` command;
keep the broader client scope of uninstall in mind. Restart the selected
profile after changing its pinned registration.
`claude mcp remove` removes only the registration. To remove REA from this
profile without running the broader uninstall flow, first back up the profile's
REA-managed skill directory, then remove
`$CLAUDE_CONFIG_DIR/skills/reverse-engineer-anything` as well. Keep other skills
in the profile intact.
The commands above use POSIX shell assignment syntax. In native Windows
PowerShell, select the profile for the whole setup/launch/check session and
restore the previous selection afterward:
```powershell
$previousClaudeConfigDir = $env:CLAUDE_CONFIG_DIR
try {
$env:CLAUDE_CONFIG_DIR = Join-Path $HOME ".claude-rea"
npx rea-agents setup --client claude_code
claude mcp list
npx rea-agents doctor --client claude_code --skill --json
npx -y rea-agents@latest setup --client claude_code --dry-run
# After reviewing the plan, repeat setup without --dry-run and approve it.
claude
# Profile removal, when intended, also belongs inside this selected scope:
# claude mcp remove --scope user rea
} finally {
if ($null -eq $previousClaudeConfigDir) {
Remove-Item Env:CLAUDE_CONFIG_DIR -ErrorAction SilentlyContinue
} else {
$env:CLAUDE_CONFIG_DIR = $previousClaudeConfigDir
}
}
```
Use PowerShell for these Windows examples rather than pasting POSIX assignments
into cmd.exe. Keep profile updates, readiness checks and removal inside the
same selected environment; the `finally` block restores the previous profile
or unsets the variable when it was absent.
**One project.** Install without configuring Claude Code: use the curl
installer's `--no-setup`, or run setup only for other clients. Then register
REA from the project directory at Claude Code's local scope.
This route uses the default Claude profile. Keep `CLAUDE_CONFIG_DIR` unset for
the entire local lifecycle: registration, backup, listing, version replacement
and removal. In POSIX shells run `unset CLAUDE_CONFIG_DIR`; in PowerShell run
`Remove-Item Env:CLAUDE_CONFIG_DIR -ErrorAction SilentlyContinue` before these
commands. Record an existing selection first if you intend to restore it after
finishing. The configuration path below is the default profile's
`~/.claude.json`; a different active profile uses its own config root instead.
Register with the variable unset:
<!-- x-release-please-start-version -->
```bash
claude mcp add --scope local rea -- npx -y rea-agents@6.4.0 mcp
```
On native Windows, use the same project directory and wrap the package-runner
shim so the stdio client can launch it:
```powershell
claude mcp add --scope local rea -- cmd.exe /d /c npx -y rea-agents@6.4.0 mcp
```
<!-- x-release-please-end -->
Local scope stores the entry under that directory's path in `~/.claude.json`,
so other projects don't load it. This route doesn't install the bundled skill.
Avoid `--scope project`, which writes `.mcp.json` into the repository where it
can be committed or copied. `rea doctor` and `rea uninstall` read only the
user-level entry, so they don't report or remove a local-scope registration;
remove it with `claude mcp remove --scope local rea` from the same directory.
`rea update` doesn't refresh it either, so it stays on the version it names.
To move it to a newer release, first back up the project entry in
`~/.claude.json` and record its environment and other metadata. In particular,
preserve caller-selected overrides such as `REA_MCP_INPUT_SCHEMA_PROFILE` and
`REA_MCP_MAX_RESPONSE_BYTES`. Run `claude mcp remove --scope local rea` from the
same directory, then recreate it with the platform-appropriate command above
and the new version. Reapply every recorded environment value with Claude
Code's `mcp add --env NAME=value` options and restore any other retained entry
metadata before restarting the project session. Removal and re-addition do not
preserve these values automatically. Running setup for Claude Code instead
would add a user-level entry for every project.
Check the separate profile with the prefixed `claude mcp list` above and compare
with the default profile with `CLAUDE_CONFIG_DIR` unset. For local scope, run
`claude mcp list` from the selected project and again from another project.
## Check readiness for your task
Run `rea doctor` when you need diagnosis. It reads host prerequisites and agent
@ -136,6 +290,15 @@ configuration without changing them. Without options, it audits every detected
registration, the installed skill and optional analysis engines; its overall
`healthy` value can be false while your chosen workflow works.
Doctor's `providerInspections` are host-level engine and registration checks,
not a readiness row for every advertised MCP adapter surface. `tools/list`
remains the complete stable catalog. Call `binary_session` with `{}` before
choosing an operation; it works before a target is opened, and
`result.tool_availability` reports per-operation availability, reasons and
client requirements for the current session state. It does not guarantee that
a particular caller-supplied input will be valid. See
[MCP discovery](mcp-contracts.md#identity-and-discovery).
Select the part you want to check:
| Task | Readiness check |
@ -335,6 +498,20 @@ document for the same target. REA does not currently identify, focus, or reuse
that existing GUI document, and the supported launcher exposes no attach or
reuse action for it.
REA currently declines saved Hopper database (`.hop`) targets on macOS because
the native `open database` launch route does not run REA's bridge script. To
analyze that program with REA, select its original executable as the target and
open it through Hopper; this starts the supported bridge route. This does not
recover database-only annotations or analysis changes. Saved-database analysis
on Linux continues to use its supported demo-launch route.
Hopper lease directories must be readable and owned by the current user.
REA validates and secures an opened directory before reserving a lease. If
directory access fails, the error identifies the path and recovery steps;
REA leaves its permissions unchanged. Check ownership and parent-directory
access, then restore owner-only read, write, and search permissions (`0700`)
on a directory you own before retrying.
On supported Linux distributions, approved setup verifies Hopper's official
`.deb`, `.rpm`, or Arch package before invoking the native package manager.
REA runs the supported demo build on a private Xvfb display and selects Hopper's
@ -347,6 +524,13 @@ host mount and the rest of `/tmp` remain unchanged; this fallback never invokes
`sudo`. `rea doctor --provider hopper --json` reports the selected
strategy and both host and effective mount facts.
When the package-manager step fails, setup reports the failing command and its
captured output, so a package conflict is distinguishable from an authorization
failure. On CachyOS, an unversioned compatibility package such as
`zlib-ng-compat` may conflict with Hopper's versioned `zlib` dependency. Resolve
the package conflict yourself or install Hopper's official package manually,
then rerun `rea setup`.
The supported Linux demo forwards additional launches into its existing
application, even across private displays. REA therefore reserves one Linux
Hopper application per user across cooperating REA processes. A competing
@ -371,8 +555,9 @@ sessions before removing that recovery directory and retrying.
On macOS, REA uses
`/Applications/Hopper Disassembler.app/Contents/MacOS/hopper` by default.
On Linux, it prefers executable `/opt/hopper/bin/Hopper`, then executable
`~/.local/share/rea/hopper/bin/Hopper`; if neither exists, the former remains
the diagnostic fallback. `HOPPER_LAUNCHER_PATH` overrides those choices:
`~/.local/share/rea/hopper/bin/Hopper`, followed by `/opt/hopper-v4/bin/hopper`
and `/usr/bin/hopper`; if none exists, the former remains the diagnostic
fallback. `HOPPER_LAUNCHER_PATH` overrides those choices:
```bash
export HOPPER_LAUNCHER_PATH=/absolute/path/to/Hopper
@ -512,7 +697,7 @@ reported as incompatible instead of being admitted from its directory name, and
a header that cannot be read or recognized is reported as unknown. REA does not
build or install native tools, or change Gatekeeper quarantine settings.
The adapter exposes 25 read-only operations: thirteen inventory/name/search
The adapter exposes 26 read-only operations: fourteen inventory/name/search
operations and twelve function-analysis operations. These cover metadata,
decompilation, assembly, resolved calls, typed references, xrefs, function
dossiers, instructions, recovered data types, measured load mappings, loaded
@ -560,6 +745,14 @@ installation remains unavailable on Windows. Setup never installs Ghidra,
Java, or Python. It preserves valid detected Ghidra/JDK settings in agent
registrations. See [Windows Ghidra P0](windows-ghidra-p0.md) for provider diagnostics.
When `GHIDRA_INSTALL_DIR` is unset, doctor and the Ghidra provider also search
`PATH` for `ghidra` or `ghidraRun`, resolve the selected launcher through its
real path, and accept it only when that location belongs to a Ghidra tree with
the application metadata and headless launcher. An explicitly supplied
`GHIDRA_INSTALL_DIR` always takes precedence, including when it is invalid;
REA reports that error instead of silently selecting another installation.
This discovery is read-only and does not execute the Ghidra launcher.
Doctor validates the platform, architecture, Ghidra 12.1.x application version,
`support/analyzeHeadless` or `support/analyzeHeadless.bat`, the installation's
Java major range, 64-bit JDK bitness, and the presence of `javac`/`javac.exe`.
@ -800,7 +993,7 @@ On a controlled Windows x64 runner, use
`npm run verify:ghidra:windows`. The verifier generates a deterministic native
PE fixture from source bytes and requires the Windows native authority before
opening the provider. The controls are implemented on main; see the
[release boundary](#released-package-and-main). The lane verifies all 25 admitted
[release boundary](#released-package-and-main). The lane verifies all 26 admitted
read-only operations, digest identity, and runtime cleanup. The separate
`verify:ghidra:windows:package` lane checks the packed CLI/MCP boundary with an
ordinary user. It requires the built Windows native bundle and an existing

View File

@ -282,9 +282,14 @@ suffix. Utility calls such as `useUtils().invoice.list.fetch()`, `useQueries`,
and links whose callee was renamed by a minifier are not recognized.
Each recovered bundle module retains the exact factory-source digest. A complete
bounded AST also receives a `babel-ast-v1` structural fingerprint that ignores
bounded AST also receives a `babel-ast-v2` structural fingerprint that ignores
ordinary identifier names while retaining syntax, literals, operators, object
keys, and member properties. If fingerprint construction reaches its structural
keys, and member properties. Computed member identifiers retain their access
syntax without treating a local variable's spelling as a fixed property; template
literal contents remain part of the fingerprint. Version comparison keeps the
algorithm in its matching key, so retained `babel-ast-v1` fingerprints cannot match
these new fingerprints. Re-analyze both snapshots to compare their new structure.
If fingerprint construction reaches its structural
bound, the fingerprint is unavailable and its status is `truncated`; a bounded
prefix is never presented as a complete stable fingerprint.
@ -619,3 +624,11 @@ Unresolved HTML references retain their declaration, source range, and resolutio
reason in the renderer observations.
HTML script source ranges follow the HTML parser across LF, CRLF, and bare CR
line endings, preserving UTF-16 columns.
Inline JavaScript in local HTML, meaning classic or module `<script>` bodies
without a `src` attribute, is not analyzed. Each HTML file that has one gets an
unknown `parse-html-inline-scripts` scope with the exact body count and the
script type and source range of the first 100 bodies; beyond that, coverage is
truncated under the `html_inline_script_locations_per_file` limit, and the
scope's evidence counts the omitted locations. The graph names the
file and up to 10 starting lines in its limitations, and coverage is partial. Import maps, speculation rules, data blocks, and blank
bodies are not JavaScript and are not reported.

View File

@ -36,6 +36,12 @@ application starts, and REA never resumes it or sends
`Runtime.runIfWaitingForDebugger`. Resume a paused target with a separate
caller-owned debugger before requesting application runtime observations.
For a packaged Electron app, read
[`inspect_electron_fuses`](electron-fuses.md) before starting an `electron-main`
target. When `EnableNodeCliInspectArguments` is disabled or removed, Electron
ignores `--inspect` and `--inspect-brk`. A missing main-process target after
launch does not, by itself, show that fuse.
The caller supplies the literal-loopback HTTP endpoint directly. Selecting an endpoint exposes all targets it serves and their local script locations or renderer origins. A target must match the exact requested target ID and expose a same-port WebSocket reported by that endpoint. Targets explicitly marked attached are rejected rather than displacing another debugger. Node's `/json/list` omits attachment state, so REA preserves it as `null` (unknown), reports that limitation, and permits observation unless the endpoint explicitly reports `attached: true`. Local paths are canonicalized after symlink resolution.
The `attached` field in JavaScript runtime and browser target results is nullable: `true` and `false` are reported only when discovery supplied a boolean; `null` means the endpoint omitted or could not report the state. CDP `TargetInfo` worker records require an `attached` boolean; REA excludes a worker record with a missing or malformed value as an invalid protocol value.

View File

@ -193,7 +193,8 @@ after target content was read.
## Retained application Evidence inputs
`analyze_javascript_application`, `inspect_binary_layout`, `inspect_artifact`,
`inspect_wasm_artifact`, and `inspect_macho` accept `detail`:
`inspect_wasm_artifact`, `inspect_macho`, and `decode_interface_builder`
accept `detail`:
`complete` (default) returns the complete analysis Evidence; `summary` records
that complete Evidence in the session and returns the derived
`inspect_analysis_view` summary Evidence instead. Its
@ -276,6 +277,17 @@ symbols (Electron Framework: 2.5 MB of imports and exports, 3.3 KB as a
summary), so select `detail: "summary"` on the first call. Delivery detail
never reaches the native tools or changes the source Evidence ID.
Completed `decode_interface_builder` Evidence supports a summary and
`view: {kind: "interface-builder", collection: "nodes", offset: 0, limit: 64}`.
Collections are `documents`, `nodes`, `edges`, and `coverage`; for nodes and
edges an optional `document` keeps rows whose evidence names that exact decoded
`relative_path`, and an undecoded path is refused. The summary keeps every
decoded document, the provider, node-kind, edge-relation and coverage-status
counts, and whether the decoded graph is incomplete. Each page repeats
`source_truncated`. A complete app graph often exceeds the default MCP response
budget (XLD.app: 6.7 MB), so select `detail: "summary"` on the first call.
Delivery detail never reaches the decoder or changes the source Evidence ID.
`trace_application_feature`, `trace_javascript_semantics`,
`compare_application_versions`, `compare_source_to_bundle`, and
`compare_javascript_export_shapes` accept complete inline application Evidence
@ -671,7 +683,13 @@ advertised input schema within a 13,312-byte budget, measured to sit below
Moonshot's observed cap (community measurements: a 13.7 KB schema is accepted
while a ~15 KB one is rejected). The setting is read at startup; restart the
server to apply changes, as with `REA_MCP_MAX_RESPONSE_BYTES`. The default
value `full` keeps the complete advertisement unchanged.
value `full` keeps the complete advertisement unchanged. Compact presentation
also reduces schemas whose expanded JSON tree exceeds ten levels, the limit
reported by strict-depth MCP clients in [#2813](https://github.com/morluto/rea/issues/2813).
WorkBuddy/CodeBuddy gateways have also rejected full REA catalogs with an
opaque `11133` error while accepting the compact profile
([#2999](https://github.com/morluto/rea/issues/2999)); compatibility depends on
the selected gateway and model.
The compact profile renders the same tool set, output schemas, annotations,
and canonical server-side validation. It changes only the advertised input
@ -696,6 +714,9 @@ JSON Schema:
canonical schema still rejects malformed calls server-side, so callers
following the reduced advertisement receive the canonical typed error
instead of silent acceptance.
- Schemas deeper than ten nodes use the same reduced projection, with
root-property names, types, guidance, and required fields retained. The
server continues to enforce the full canonical input contract.
Boundary tests pin every advertised compact schema to the budget, keep them
valid JSON Schema that still accepts every canonical example. Client verification
@ -711,6 +732,30 @@ identity. Explicit schema IDs and relative external reference bases are
preserved. This keeps every tool and validation rule in discovery and does not
change the SDK's catalog invalidation or availability checks.
Some stdio hosts reject a complete `tools/list` frame before they load any
tools. The opt-in `REA_MCP_OUTPUT_SCHEMA_PROFILE=off` setting omits `outputSchema`
from the advertised catalog for hosts with a small frame limit (the measured
pi-web-ui failure and catalog breakdown are in [#2512](https://github.com/morluto/rea/issues/2512)).
The default `full` profile is unchanged. With `off`, REA validates successful
structured results against the same named canonical tool contract before
delivery; it still returns complete `structuredContent` and text results.
Only client-side schema discovery and validation are omitted, so hosts using
this profile should rely on REA's validation and retain their usual response
size limits.
### Function comparison limitations
`compare_functions` compares the `pseudocode` and `assembly` dimensions as
exact provider text. A `changed` status establishes a textual difference, not a
behavior change. Addresses and generated names are not normalized, so relocated
labels can change these dimensions while `cfg` remains unchanged. Both text
dimensions carry this limitation inline. Numeric constants remain part of the
exact comparison. Reconstruction verification and static/runtime correlation
retain limitations from the selected dimension; an overall function claim
includes all dimension limitations. Equal CFGs establish only equal observed
block sizes and successor relationships. Text from different provider
identities remains unknown.
### Comparison input shapes
`compare_web_captures` accepts exactly one of two input shapes:

View File

@ -227,6 +227,10 @@ nodes and edges. Node payloads are bounded to 8 MiB and serialized results to
32 MiB. Pagination returns edges whose source is on the node page and preserves
stable IDs for endpoints outside that page.
Within one value trace, each exact procedure selector is attempted at most once,
including failed analyses. Failures remain unknown observations and do not
establish canonical function identity. A new trace can retry those selectors.
The raw Java `is_dead` flag can remain set after Ghidra decodes a live block;
`block_membership` reports actual syntax-tree membership separately.
@ -319,7 +323,16 @@ complete observation.
digest are preserved; key interpretation follows the pinned plist decoder.
- `decode_interface_builder` / `rea decode-interface-builder <app>` reads
compiled storyboard and nib resources into objects, outlets, actions,
connections and controller/class names. `verify:interface-builder` compiles
connections and controller/class names. A full app graph is often several
megabytes (XLD.app: 6.7 MB, about 2M tokens, which also exceeds the default
MCP response budget), so MCP callers should pass `detail: "summary"`. The
summary keeps every decoded document and node/edge/coverage counts and
retains the complete Evidence; page rows with `inspect_analysis_view`
`view: {kind: "interface-builder", collection: "nodes", document:
"Contents/Resources/Base.lproj/MainMenu.nib", offset: 0, limit: 64}`.
`document` selects rows whose evidence names that exact
`documents[].relative_path` (nodes and edges only); `source_truncated` keeps
graph incompleteness visible on every page. `verify:interface-builder` compiles
a source-owned AppKit XIB with `ibtool`. Storyboards require an installed
iOS platform; unsupported archive forms remain explicit.
XML plist archives accept UTF-8 and BOM-marked UTF-16 in either byte order.

View File

@ -380,6 +380,9 @@ Every capture requires `truncation_details`, with separate accounting for:
When a directory exceeds that capacity, the retained subset follows filesystem
enumeration order and is then sorted; it is explicitly incomplete and cannot
establish that an omitted path is absent.
An unreadable directory is recorded in `enumeration_omissions` with its
root-aliased path and operating-system error code. Entries observed before
that directory remain available, while descendants beneath it stay unknown.
Each retained regular file without a digest has an aliased path, size,
remaining budget and reason: `file_bytes_budget`,
`file_changed_or_short_read`, or `file_unavailable`. Its `system_code` is

View File

@ -20,7 +20,7 @@
**[웹사이트](https://rea.tools/ko/) · [가이드](https://rea.tools/ko/guides/) · [사례](https://rea.tools/ko/showcase/)**
[빠른 시작](#빠른-시작) · [REA의 작동 방식](#rea의-작동-방식) · [분석할 수 있는 대상](#분석할-수-있는-대상) · [사례](#사례) · [자주 묻는 질문](#자주-묻는-질문) · [GNU objdump 및 Rizin 도구](../reverse-engineering-tools.md) · [문서](#문서)
[빠른 시작](#빠른-시작) · [REA의 작동 방식](#rea의-작동-방식) · [분석할 수 있는 대상](#분석할-수-있는-대상) · [사례](#사례) · [자주 묻는 질문](#자주-묻는-질문) · [문서](#문서)
<code>npx rea-agents setup</code>
@ -56,7 +56,7 @@
REA는 에이전트를 분석 도구에 연결해 네이티브 바이너리, JavaScript와 Electron 앱, .NET 어셈블리, 웹사이트를 검사할 수 있게 합니다. 터미널에서도 같은 도구를 사용할 수 있습니다. 분석은 로컬에서 실행되며, 결과에는 각 결론의 근거와 한계가 포함됩니다.
설정 과정은 에이전트에 REA를 등록하고 해당 버전에 맞는 워크플로 지침을 설치합니다. 네이티브 분석에는 기존 Hopper 또는 Ghidra 설치를 사용할 수 있습니다. 승인하면 설정 과정에서 Hopper를 설치할 수도 있습니다. 정적 JavaScript 분석에는 두 엔진 모두 필요하지 않습니다.
설정 과정은 에이전트에 REA를 등록하고 해당 버전에 맞는 워크플로 지침을 설치합니다. 네이티브 분석에는 기존 Hopper, Ghidra 또는 IDA 설치를 사용할 수 있습니다. 승인하면 설정 과정에서 Hopper를 설치할 수도 있습니다. 정적 JavaScript 분석에는 네이티브 분석 엔진이 필요하지 않습니다.
> **[REA 웹사이트](https://rea.tools/ko/)**에서 설정 방법, 그림으로 설명한 가이드, 실제 사례를 확인하세요.
@ -105,12 +105,14 @@ rea --help
REA는 빠르게 바뀌며 새 릴리스에는 버그 수정이 자주 포함됩니다. 설치된 버전을 최신으로 유지하세요.
npm으로 설치한 CLI에서는 다음 명령을 사용합니다.
`npm install --global` 또는 `bun add --global`로 설치한 CLI에서는 다음 명령을 사용합니다.
```bash
rea update
```
Bun 전역 업데이트에는 Bun 1.2.17 이상이 필요합니다. 이전 버전의 Bun을 사용한다면 `rea update`를 실행하기 전에 Bun을 직접 업그레이드하세요.
에이전트 등록과 스킬을 갱신하려면 업데이트 과정에서 출력된 설정 명령을 실행하세요.
`npx`를 사용한다면 다음 명령으로 에이전트 설정을 갱신하세요.
@ -119,7 +121,7 @@ rea update
npx rea-agents@latest setup
```
설정 변경 사항을 검토하고 에이전트를 다시 시작하세요. 일회성 CLI 작업에는 `npx rea-agents@latest` 뒤에 필요한 명령을 붙여 실행하세요.
설정 변경 사항을 검토하고 에이전트를 다시 시작하세요. 일회성 CLI 작업에는 `npx rea-agents@latest` 뒤에 필요한 명령을 붙여 실행하세요. Bun을 사용한다면 위 명령의 `npx rea-agents@latest`를 `bunx rea-agents@latest`로 바꿔 실행하세요.
## REA의 작동 방식
@ -139,14 +141,16 @@ REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니
| --------------------- | ----------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| 네이티브 바이너리 | 의사 코드, 어셈블리, 문자열, 심볼, 호출, 참조 | Hopper, Ghidra 또는 IDA; [네이티브 분석](https://rea.tools/ko/guides/native/) |
| 오프라인 ELF 레이아웃 | 섹션, 세그먼트, 원본 심볼/재배치 정보, 정적 분석으로 얻은 보안 완화 기법 후보 | Linux x64에서 호출자가 제공하는 pwntools; [바이너리 진단](../binary-diagnostics.md) |
| Go 바이너리 | 내장된 컴파일러 버전, 모듈, `replace` 지시어, 빌드 설정 | REA 내장 ELF/PE/thin Mach-O 정적 검사; Go 설치 불필요 |
| Rust 바이너리 | cargo-auditable이 내장한 크레이트 이름, 버전, 소스, 의존성 관계 | REA 내장 ELF/PE/thin Mach-O 정적 검사; [Rust 바이너리](../rust-binaries.md) |
| WASM 아티팩트 | 유효성 검사, 섹션, import/export의 정확한 WAT 형식, 디코딩된 WAT | 호출자 제공 WABT 1.0.42; `inspect_wasm_artifact` |
| EVM 바이트코드 | 디스패치 선택자, 바이트 오프셋, 추론한 인자와 상태 변경 가능 여부 | Linux x64의 로컬 원시 바이트/16진수 입력 파일; [오프라인 EVM 가이드](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| SQLite 데이터베이스 | 테이블, 열, 인덱스, 뷰, 트리거, 선택한 일반 테이블의 행 | 로컬 데이터베이스/WAL 스냅샷; Node.js 24.15 이상의 24.x 또는 26 이상(네이티브 SQLite 한도 적용) |
| 기록된 Linux 크래시 | 원시 note 레코드, 기록된 모든 스레드의 레지스터/시그널, 선택적 매핑 후보 | Linux x64에서 호출자가 제공하는 pwntools; GDB/pwndbg는 선택 사항; [기록된 크래시](../recorded-crashes.md) |
| JavaScript / Electron | 모듈, 임포트, 소스 맵, 라우트, IPC, 네이티브 애드온 관계 | Node.js와 npm; [애플리케이션 분석](https://rea.tools/ko/guides/javascript/) |
| 웹사이트 | 페이지 구조, 스크립트, 네트워크 관찰 결과, 요청한 스크린샷 | Chrome 계열 브라우저; [브라우저 분석](https://rea.tools/ko/guides/browser/) |
| 저장된 네트워크 캡처 | 요청, 응답, 접근 가능한 페이로드, 소스 위치 | HAR; mitmproxy 네이티브 캡처에는 Linux의 mitmdump; [캡처 가이드](../web-network-captures.md) |
| 패킷 캡처 | 엔드포인트, DNS, TLS SNI, 프로토콜 분포, HTTP 요청 | 호출자 제공 tshark 및 capinfos; [pcap 가이드](../pcap-analysis.md) |
| .NET 어셈블리 | 메타데이터, CIL 명령, 선언된 네이티브 의존성, 빌드 비교 | 정적 검사; [관리 코드 가이드](../managed-code-analysis.md) |
| 저장된 네트워크 캡처 | 요청, 응답, 접근 가능한 페이로드, 소스 위치 | HAR 또는 Fiddler Everywhere SAZ; mitmproxy 네이티브 캡처에는 Linux의 mitmdump; [캡처 가이드](../web-network-captures.md) |
| .NET 어셈블리 | 메타데이터, CIL 명령, 선언된 네이티브 의존성, 빌드 비교 | 별도 도구 불필요(정적 검사만); [관리 코드 가이드](../managed-code-analysis.md) |
| Android APK | 매니페스트 선언, 클래스, 디컴파일된 메서드, 참조 | Linux/macOS/Windows x64의 헤드리스 JADX와 전체 JDK; [Android 가이드](../android-analysis.md) |
| Android 기기 | 기기, 패키지, 프로세스, 로그, 덤프, 화면 캡처, APK/파일 전송 | 호출자 제공 adb(에뮬레이터 또는 실제 기기); [ADB 가이드](../adb-device-analysis.md) |
| JEB 프로젝트 | 엔진 기반 프로젝트 유닛, 아티팩트 해시, 타입/메서드 의사코드 | 호출자가 시작하고 MCP를 제공하는 JEB 클라이언트(REA_JEB_MCP_URL로 선택); [JEB 가이드](../jeb-analysis.md) |
@ -154,15 +158,18 @@ REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니
| Flutter 앱 | Dart 스냅샷 해시, ABI별 라이브러리 해시, 엔진 빌드 출처 | 없음 (정적 APK 파싱); [Flutter 가이드](../flutter-analysis.md) |
| 펌웨어 | 영역, 추출 결과, 네이티브 분석으로 전달할 정보 | Linux의 Binwalk / Unblob; [펌웨어 가이드](../firmware-analysis.md) |
| 기록된 패킷 | 선택 필드, 반복 값, 필터 및 디코딩 규칙 | Linux: TShark 4.x + prlimit; [캡처 가이드](../packet-captures.md) |
| 패킷 요약 | 엔드포인트, DNS, TLS SNI, 프로토콜 분포, HTTP 요청 | 호출자 제공 tshark 및 capinfos(Linux에서 검증됨); [pcap 가이드](../pcap-analysis.md) |
| 기록된 논리 신호 | 디코더 주석, 샘플 범위, 진단 정보 | Linux: sigrok-cli 0.7.2 + prlimit; [신호 기록 가이드](../signal-recordings.md) |
| 패키지 및 리소스 | 파일 목록, 다이제스트, plist, Apple 번들 구조, 추출한 리소스 | [아티팩트 및 JavaScript 가이드](../javascript-artifact-reconstruction.md), [Apple 애플리케이션](../apple-application-analysis.md) |
| 프로세스 동작 | 터미널 출력, 상호작용, 종료 및 파일 시스템 관찰 결과, 실행 비교 | 네이티브 PTY를 지원하는 Linux/macOS; [프로세스 캡처](../process-capture.md) |
정적 JavaScript 및 .NET 검사는 제공된 파일을 읽으며 애플리케이션을 실행하지 않습니다. 런타임 캡처는 사용자의 권한으로 선택한 대상을 실행하거나 대상과 상호작용합니다. 각 런타임 가이드에서 그 영향을 설명합니다.
오프라인 네이티브 바이너리 검사에는 호출자가 제공하는 GNU objdump와 Rizin도 사용할 수 있습니다. 제공자 설정, CLI 명령, Evidence의 한계는 [리버스 엔지니어링 도구 가이드](../reverse-engineering-tools.md)를 참고하세요. REA는 objdump와 Rizin을 설치하거나 업데이트하지 않습니다.
<a id="choosing-a-deep-analysis-provider"></a>
지원하는 네이티브 형식과 호스트는 제공자마다 다릅니다. [Hopper 및 Ghidra 설정](../installation.md#hopper), [IDA 가이드](../ida-provider.md), [실험적 Windows Ghidra 지원](../windows-ghidra-p0.md)을 참고하세요. Ghidra는 [16비트 DOS 분석](../ghidra-dos.md)도 지원합니다. 제공자 선택은 [CLI 가이드](../cli.md#choose-a-provider)를 참고하세요. 최신 npm 릴리스 이후 추가된 기능은 [릴리스에서의 사용 가능 여부](../installation.md#released-package-and-main)를 확인하세요.
지원하는 네이티브 형식과 호스트는 제공자마다 다릅니다. [Hopper 및 Ghidra 설정](../installation.md#hopper), [IDA 가이드](../ida-provider.md)(선택 사항인 네이티브 MCP 브리지 포함), [실험적 Windows Ghidra 지원](../windows-ghidra-p0.md)을 참고하세요. Ghidra는 [16비트 DOS 분석](../ghidra-dos.md)도 지원합니다. 큰 바이너리를 분석할 때는 `REA_GHIDRA_STARTUP_TIMEOUT_MS` 또는 `REA_HOPPER_STARTUP_TIMEOUT_MS`로 제공자의 시작 제한 시간을 늘리세요. 제공자 선택은 [CLI 가이드](../cli.md#choose-a-provider)를 참고하세요. 최신 npm 릴리스 이후 추가된 기능은 [릴리스에서의 사용 가능 여부](../installation.md#released-package-and-main)를 확인하세요.
## 사례
@ -209,7 +216,7 @@ REA로 흥미로운 대상을 분석했다면 알려 주세요. [issue](https://
<details>
<summary><strong>Hopper를 먼저 실행해야 하나요?</strong></summary>
REA는 작업에 필요할 때 Hopper를 실행합니다. macOS에서는 처음 실행할 때 데모 모드를 선택하거나 라이선스를 활성화하라는 대화 상자가 나타날 수 있습니다. [Hopper 실행 및 문제 해결](../installation.md#launcher-paths-and-troubleshooting)을 참고하세요.
REA는 작업에 필요할 때 Hopper를 실행합니다. macOS에서는 처음 실행할 때 데모 모드를 선택하거나 라이선스를 활성화하라는 대화 상자가 나타날 수 있습니다. 첫 CLI/MCP 작업이나 무인 작업을 실행하기 전에 같은 사용자 세션에서 이 대화 상자를 완료하세요. 필요하면 Hopper를 직접 여세요. [Hopper 실행 및 문제 해결](../installation.md#launcher-paths-and-troubleshooting)을 참고하세요.
</details>
@ -239,7 +246,7 @@ REA는 대상을 로컬에서 분석합니다. 에이전트는 도구 결과를
먼저 업데이트하세요. 최근 릴리스에서 이미 수정되었을 수 있습니다.
npm으로 설치한 CLI에서는 다음 명령을 사용합니다.
`npm install --global` 또는 `bun add --global`로 설치한 CLI에서는 다음 명령을 사용합니다.
```bash
rea update
@ -286,11 +293,9 @@ REA를 사용하고, 버그를 보고하고, 기능을 요청하고, 빌드를
책임 있는 사용에 관한 안내는 [DISCLAIMER.md](../../DISCLAIMER.md)를 참고하세요.
REA는 법을 준수하는 연구, 교육, 상호운용성 확보, 소프트웨어 보존, 그리고 본인이 소유하거나 조사 권한을 가진 소프트웨어의 분석을 위한 도구입니다.
불법적이거나 허가되지 않은 사용에 대한 프로젝트의 입장은 모든 REA 버전, 배포 채널, 플랫폼에 적용되며, 각 사본에는 해당 라이선스가 그대로 유지됩니다.
The project's position on illegal or unauthorized use covers all REA
versions, distribution channels, and platforms; each copy retains its
applicable licenses.
REA는 법을 준수하는 연구, 교육, 상호운용성 확보, 소프트웨어 보존, 그리고 본인이 소유하거나 조사 권한을 가진 소프트웨어의 분석을 위한 도구입니다.
## 기여
@ -302,14 +307,6 @@ REA에 도움을 보태 주세요! [issue를 등록](https://github.com/morluto/
[MIT](../../LICENSE)
REA-authored code is licensed under MIT. Third-party material retains its own
licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md)
for the dependency inventory, preserved license texts, and external-tool terms.
REA가 작성한 코드는 MIT 라이선스를 따릅니다. 제3자 자료에는 각자의 라이선스와 저작권 고지가 그대로 유지됩니다. 의존성 목록, 보존된 라이선스 원문, 외부 도구 약관은 [제3자 고지](../../THIRD_PARTY_NOTICES.md)를 참고하세요.
**You must legally obtain and be authorized to use commercial tools**, including
IDA Pro / Hex-Rays decompilers, Hopper, and JEB, under their applicable licenses.
REA grants no tool license, key, or activation entitlement. Trials and demos
remain subject to vendor terms, including user, feature, automation, and output
sharing restrictions. Cracked copies, stolen keys, unauthorized key sharing,
and bypassing license restrictions are not authorized. This guidance does not
amend MIT or third-party licenses.
IDA Pro / Hex-Rays 디컴파일러, Hopper, JEB 등 상용 도구는 해당 라이선스에 따라 **합법적으로 취득하고 사용 권한을 갖춰야 합니다.** REA는 어떤 도구의 라이선스, 키, 활성화 권한도 부여하지 않습니다. 체험판과 데모는 사용자, 기능, 자동화, 출력 공유 제한을 포함한 공급업체 약관을 따릅니다. 크랙 버전, 도난 키, 무단 키 공유, 라이선스 제한 우회는 허용되지 않습니다. 이 안내는 MIT 라이선스나 제3자 라이선스를 변경하지 않습니다.

View File

@ -19,7 +19,7 @@ behavior, and complete CI evidence before merging.
| `npm run test:boundary` | Boundary, CLI boundary, MCP boundary, process boundary, and process-global projects |
| `npm run test:mcp` | MCP boundary project |
| `npm run test:acceptance` | Complete compiled CLI and MCP acceptance workflows |
| `npm run test:python` | LLDB producer retention and launch contracts with a fake LLDB API; no native debugger required |
| `npm run test:python` | LLDB and mitmproxy producer contracts plus analysis-efficiency checks; no real debugger or proxy required |
| `npm run test:watch` | Dirty source tests in watch mode, without build |
| `npm run test:watch:all` | Changed tests from every project; builds at startup, so rebuild after production edits before relying on compiled tests |
| `npm run check:changed` | Cached typecheck/lint and branch-related source feedback |
@ -449,8 +449,9 @@ adds in-place reparse, breakaway, environment, and token observations.
The separate `scripts/verify-windows-native-inspection.mjs` lane uses the
`.cache/windows-native-inspection-fixture` artifact built on Linux with
`node scripts/build-windows-native.mjs --inspection-fixture`. Its controlled
first deletion failure checks retained NTFS ownership, later-query recovery,
successful-admission caching, and environment-teardown cleanup. It neither
first deletion failure checks retained NTFS ownership across a same-volume
rename, later-query handle-based recovery, successful-admission caching, and
environment-teardown cleanup. It neither
establishes the cause of the intermittent #1701 host failure nor replaces
verification with the shipped production artifact.
`npm run verify:ghidra:windows:package` additionally packs and installs REA into

View File

@ -19,7 +19,7 @@ additional permission flag or degraded mode is required.
- a 64-bit full JDK inside that installation's declared Java range (JDK 21 or newer, with no maximum, for current 12.1 releases);
- an explicit native, non-managed x86 or x86-64 PE application/DLL, or a
[raw NMOS 6502 image](raw-binary.md#nmos-6502) with a caller-supplied entry; and
- the 25 read-only Ghidra inventory, memory, and function-analysis operations.
- the 26 read-only Ghidra inventory, memory, and function-analysis operations.
Loaded memory reads and file offsets preserve Ghidra's observed source mappings.
PE load-image inspection returns an explicit unsupported attestation status.
@ -78,6 +78,19 @@ OS failures. It checks the Ghidra release, headless launcher, full JDK, x64
host, and native controls. A missing or mismatched addon requires the matching
Windows x64 package; changing MCP registration cannot supply that artifact.
A filesystem or access failure identifies the failed native constraint.
An incompatible host uses `unsupported_host`. Missing, mismatched or unloadable
native artifacts use `runtime_missing` and doctor classification
`missing_dependency`. A failed native invocation or invalid inspection uses
`runtime_unavailable` and doctor classification `runtime_failure`; it does not
mean Windows x64 itself is unsupported. Provider diagnostics retain the failure
stage under `windows_native_failure`, together with the corresponding remediation.
Self-test failures preserve the native operation and OS error text. Check the
process's writable local NTFS temporary directory and the reported filesystem,
DACL, Job Object or cleanup constraint, then rerun doctor after resolving it.
Ghidra or Java reinstallation cannot resolve a native self-test failure. The
provider remains unavailable until all native controls are proven; a later query
may retry a failed native invocation, while artifact or malformed-inspection
failures remain stable for the process lifetime.
Other doctor checks, including unconfigured agent integrations, can remain
unready while Ghidra itself is available.

View File

@ -16,6 +16,7 @@ prerelease_identifier="(${numeric_identifier}|[0-9A-Za-z-]*[A-Za-z-][0-9A-Za-z-]
build_identifier='[0-9A-Za-z-]+'
version_pattern="${numeric_identifier}\\.${numeric_identifier}\\.${numeric_identifier}(-${prerelease_identifier}(\\.${prerelease_identifier})*)?(\\+${build_identifier}(\\.${build_identifier})*)?"
version="${REA_VERSION:-}"
version_requested=false
dry_run=false
start_setup=true
verbose=false
@ -81,6 +82,7 @@ main() {
--version)
(($# >= 2)) || fail "--version requires a value."
version="$2"
version_requested=true
shift 2
;;
--dry-run)
@ -121,7 +123,7 @@ main() {
fail_node_version "$node_version"
fi
if [ -n "$version" ]; then
if [ "$version_requested" = true ] || [ -n "$version" ]; then
matches "$version" "^${version_pattern}$" || fail "version must be an exact semantic version."
else
release_json="$(curl -fsSL --proto '=https' --tlsv1.2 --retry 3 -H 'Accept: application/vnd.github+json' "https://api.github.com/repos/$REPOSITORY/releases/latest")" || fail "the latest REA release could not be resolved. Check network access or pass --version VERSION, then retry."
@ -164,6 +166,7 @@ main() {
npm "${install_args[@]}" || fail "npm could not install REA. Check registry access and npm permissions, then retry."
[ -x "$install_bin" ] || fail "npm completed without installing the rea command. Check the npm global bin directory and PATH, then retry."
installed_version="$("$install_bin" --version 2>/dev/null | tr -d '[:space:]')" || fail "the installed REA version could not be read. Reinstall the requested version, then retry."
[ -n "$installed_version" ] || fail "the installed REA version could not be read. Reinstall the requested version, then retry."
[ "$installed_version" = "$version" ] || fail "installed version $installed_version does not match $version. Reinstall the requested version, then retry."
printf 'REA %s is installed.\n' "$version"

View File

@ -9,8 +9,8 @@
"src/application/RealHopperAssertions.ts",
"scripts/verify/fridaBindingsCompatibility.ts",
"scripts/verify/termux/browser.mjs",
"scripts/verify-real-cutter.mjs",
"scripts/verify-real-ghidra-noreturn.mjs",
"scripts/run-exact-tests.mjs",
"tests/processOwnershipRunner.ts"
],
"project": ["src/**/*.ts", "tests/**/*.ts", "scripts/**/*.mjs"],

View File

@ -214,18 +214,32 @@ std::unique_ptr<Process> spawnProcess(const std::wstring& command, const std::ws
}
static napi_value readOutput(napi_env env, Handle& read, bool& ended) {
std::array<unsigned char, 65536> bytes;
DWORD available = 0, count = 0;
constexpr size_t kMaximumReadPerPoll = 1024 * 1024;
std::array<unsigned char, 65536> chunk;
std::vector<unsigned char> bytes;
if (!read.valid()) ended = true;
else if (!PeekNamedPipe(read.get(), nullptr, 0, nullptr, &available, nullptr)) {
require(GetLastError() == ERROR_BROKEN_PIPE, "Observe owned process output failed");
read.reset(); ended = true;
} else if (available > 0) {
require(ReadFile(read.get(), bytes.data(), std::min<DWORD>(available, bytes.size()), &count, nullptr),
"Read owned process output failed");
else {
while (bytes.size() < kMaximumReadPerPoll) {
DWORD available = 0;
if (!PeekNamedPipe(read.get(), nullptr, 0, nullptr, &available, nullptr)) {
require(GetLastError() == ERROR_BROKEN_PIPE,
"Observe owned process output failed");
read.reset();
ended = true;
break;
}
if (available == 0) break;
const auto requested = static_cast<DWORD>(std::min<size_t>(
{available, chunk.size(), kMaximumReadPerPoll - bytes.size()}));
DWORD count = 0;
require(ReadFile(read.get(), chunk.data(), requested, &count, nullptr) &&
count > 0,
"Read owned process output failed");
bytes.insert(bytes.end(), chunk.begin(), chunk.begin() + count);
}
}
napi_value result;
check(napi_create_buffer_copy(env, count, bytes.data(), nullptr, &result));
check(napi_create_buffer_copy(env, bytes.size(), bytes.data(), nullptr, &result));
return result;
}

View File

@ -138,7 +138,7 @@
"test": "npm run test:with-build",
"test:with-build": "npm run test:prepare && vitest run",
"test:run": "vitest run",
"test:python": "python3 -m unittest bridge/native/test_rea_lldb_tracer.py -v && python3 -m unittest discover -s experiments/analysis-efficiency -p 'test_*.py' -v",
"test:python": "python3 -m unittest bridge/native/test_rea_lldb_tracer.py bridge/mitmproxy/test_capture.py -v && python3 -m unittest discover -s experiments/analysis-efficiency -p 'test_*.py' -v",
"test:local": "node scripts/run-development-tests.mjs local",
"test:fast": "node scripts/ci/test-group.mjs source",
"test:built": "node scripts/ci/test-group.mjs built",

View File

@ -202,6 +202,9 @@ if (tier === "reused") {
if (GUARDED_WORKFLOWS.has(file)) selected.code = true;
if (file !== ".github/workflows/ci.yml") continue;
}
// The Notes example is also the source-owned agent evaluation fixture.
if (file.startsWith("website/public/examples/notes-electron/"))
selected.code = true;
if (owners.length === 1 && owners[0] === "website") continue;
if (/^src\/(contracts|domain)\//.test(file)) selected.docs = true;
if (file === "package.json") {

View File

@ -79,6 +79,7 @@ export const SCOPES = {
"tests/boundary/cli/nativeSignatureCli.test.ts",
"tests/boundary/mcp/nativeSignatureTargetBindingMcp.test.ts",
"src/process/capture/FilesystemSnapshot.test.ts",
"tests/process-global/filesystemSnapshotFallbackCancellation.test.ts",
"tests/boundary/process/darwinProcessRunTokenReader.test.ts",
"tests/boundary/process/directAnalysisSignals.test.ts",
"tests/boundary/process/processCaptureLifecycle.test.ts",
@ -89,6 +90,9 @@ export const SCOPES = {
package: [
"src/windows/**",
"native/windows/**",
"src/process/capture/FilesystemSnapshot.ts",
"src/process/capture/FilesystemSnapshot.test.ts",
"tests/process-global/filesystemSnapshotFallbackCancellation.test.ts",
"src/inspector/**",
"scripts/verify/inspector/**",
"scripts/verify-package.mjs",

View File

@ -1,6 +1,7 @@
import { mkdir, writeFile } from "node:fs/promises";
import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { writeWindowsGhidraByteSearchFixture } from "./lib/ghidra-windows-byte-search.mjs";
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
// Source-owned static fixtures: no compiler, target loading or execution.
@ -61,7 +62,7 @@ for (const architecture of ["x86_64", "x86"]) {
}
bytes.write(".text", sectionTable, "ascii");
bytes.writeUInt32LE(dll ? 0x180 : 0x50, sectionTable + 8);
bytes.writeUInt32LE(0x200, sectionTable + 8);
bytes.writeUInt32LE(0x1000, sectionTable + 12);
bytes.writeUInt32LE(0x200, sectionTable + 16);
bytes.writeUInt32LE(0x200, sectionTable + 20);
@ -136,6 +137,8 @@ for (const architecture of ["x86_64", "x86"]) {
bytes.write("fixture_return42\0", 0x340, "binary");
}
writeWindowsGhidraByteSearchFixture(bytes);
await mkdir(dirname(output), { recursive: true });
await writeFile(output, bytes);
if (!x86 && !dll) process.stdout.write(`${output}\n`);

View File

@ -3,6 +3,8 @@ export interface ManagedPeFixtureMethod {
readonly flags?: number;
readonly implFlags?: number;
readonly name: string;
/** An explicit metadata RVA; zero represents a method without an IL body. */
readonly rva?: number;
}
export interface ManagedPeFixtureOptions {
@ -19,6 +21,7 @@ export interface ManagedPeFixtureOptions {
readonly pinvoke?: {
readonly importName?: string;
readonly mappingFlags?: number;
readonly memberRow?: number;
readonly moduleName?: string;
};
readonly readyToRun?: boolean;

View File

@ -280,7 +280,7 @@ const buildManagedMetadata = (options, methods) => {
methodDefRow({
name: stringsForMethods[index],
signature: methodSignature,
rva: method.rva ?? 0x2800 + index * 0x20,
rva: method.rva,
flags: method.flags ?? 0x0016,
implFlags: method.implFlags ?? 0,
}),
@ -325,13 +325,64 @@ const buildManagedMetadata = (options, methods) => {
return metadata;
};
const align = (value, boundary) => Math.ceil(value / boundary) * boundary;
const managedFixtureLayout = (options, methods, metadataLength) => {
const resourceLength =
4 + (options.resourceData ?? Buffer.from("resource-data")).length;
const metadataOffset = 0x0300;
const resourceOffset = Math.max(
0x0800,
align(metadataOffset + metadataLength, 0x200),
);
const readyToRunOffset = Math.max(
0x0900,
align(resourceOffset + resourceLength, 0x200),
);
const methodOffsets = [];
let nextMethodOffset = Math.max(
0x0a00,
align(readyToRunOffset + (options.readyToRun === true ? 4 : 0), 0x200),
);
for (const method of methods) {
// RVA zero denotes a method without a mapped IL body, such as P/Invoke.
if (method.rva === 0) {
methodOffsets.push(null);
continue;
}
const offset =
method.rva === undefined
? nextMethodOffset
: 0x0200 + (method.rva - 0x2000);
methodOffsets.push(offset);
nextMethodOffset =
Math.max(offset, nextMethodOffset) +
align(Math.max(0x20, (method.body ?? retOnlyIlBody).length), 0x20);
}
return { metadataOffset, resourceOffset, readyToRunOffset, methodOffsets };
};
const buildManagedImage = (options, methods, metadata) => {
const resourceData = options.resourceData ?? Buffer.from("resource-data");
const resourceDirectory = Buffer.concat([
u32(resourceData.length),
resourceData,
]);
const image = Buffer.alloc(0x1000);
const { metadataOffset, resourceOffset, readyToRunOffset, methodOffsets } =
managedFixtureLayout(options, methods, metadata.length);
const contentEnd = Math.max(
metadataOffset + metadata.length,
resourceOffset + resourceDirectory.length,
readyToRunOffset + (options.readyToRun === true ? 4 : 0),
...methodOffsets.flatMap((offset, index) =>
offset === null
? []
: [offset + (methods[index].body ?? retOnlyIlBody).length],
),
0x1000,
);
const imageSize = align(contentEnd, 0x200);
const image = Buffer.alloc(imageSize);
image.write("MZ", 0, "ascii");
image.writeUInt32LE(0x80, 0x3c);
image.writeUInt32LE(0x0000_4550, 0x80);
@ -351,16 +402,16 @@ const buildManagedImage = (options, methods, metadata) => {
else image.writeUInt32LE(0x0040_0000, optional + 28);
image.writeUInt32LE(0x1000, optional + 32);
image.writeUInt32LE(0x200, optional + 36);
image.writeUInt32LE(0x3000, optional + 56);
image.writeUInt32LE(align(0x2000 + imageSize - 0x200, 0x1000), optional + 56);
image.writeUInt32LE(0x200, optional + 60);
image.writeUInt32LE(16, optional + directoryOffset - 4);
image.writeUInt32LE(0x2000, optional + directoryOffset + 14 * 8);
image.writeUInt32LE(72, optional + directoryOffset + 14 * 8 + 4);
const section = optional + optionalSize;
image.write(".text\0\0\0", section, "ascii");
image.writeUInt32LE(0x1000, section + 8);
image.writeUInt32LE(imageSize - 0x200, section + 8);
image.writeUInt32LE(0x2000, section + 12);
image.writeUInt32LE(0x0e00, section + 16);
image.writeUInt32LE(imageSize - 0x200, section + 16);
image.writeUInt32LE(0x0200, section + 20);
image.writeUInt32LE(0x6000_0020, section + 36);
const cli = 0x0200;
@ -371,27 +422,41 @@ const buildManagedImage = (options, methods, metadata) => {
image.writeUInt32LE(metadata.length, cli + 12);
image.writeUInt32LE(options.cliFlags ?? 1, cli + 16);
image.writeUInt32LE(0x0600_0001, cli + 20);
image.writeUInt32LE(0x2600, cli + 24);
image.writeUInt32LE(0x2000 + (resourceOffset - 0x0200), cli + 24);
image.writeUInt32LE(resourceDirectory.length, cli + 28);
if (options.readyToRun === true) {
image.writeUInt32LE(0x2700, cli + 64);
image.writeUInt32LE(0x2000 + (readyToRunOffset - 0x0200), cli + 64);
image.writeUInt32LE(4, cli + 68);
image.write("RTR\0", 0x0900, "ascii");
image.write("RTR\0", readyToRunOffset, "ascii");
}
methods.forEach((method, index) => {
const offset = 0x0a00 + index * 0x20;
const offset = methodOffsets[index];
if (offset === null) return;
(method.body ?? retOnlyIlBody).copy(image, offset);
});
metadata.copy(image, 0x0300);
resourceDirectory.copy(image, 0x0800);
metadata.copy(image, metadataOffset);
resourceDirectory.copy(image, resourceOffset);
return image;
};
/** Build a source-owned PE/CLI fixture without committing compiled binaries. */
export const buildManagedPeFixture = (options = {}) => {
const methods = managedFixtureMethods(options);
const metadata = buildManagedMetadata(options, methods);
return buildManagedImage(options, methods, metadata);
const provisionalMetadata = buildManagedMetadata(
options,
methods.map((method) => ({ ...method, rva: method.rva ?? 0x2800 })),
);
const layout = managedFixtureLayout(
options,
methods,
provisionalMetadata.length,
);
const methodsWithRvas = methods.map((method, index) => ({
...method,
rva: method.rva ?? 0x2000 + (layout.methodOffsets[index] - 0x0200),
}));
const metadata = buildManagedMetadata(options, methodsWithRvas);
return buildManagedImage(options, methodsWithRvas, metadata);
};
export const buildNativePeFixture = () => {
const image = buildManagedPeFixture();

View File

@ -54,10 +54,12 @@ export async function verifyBrowserCaptureMetadataBudget(endpoint) {
if (address === null || typeof address === "string")
throw new Error("Browser metadata fixture did not bind a TCP port");
fixtureOrigin = `http://127.0.0.1:${String(address.port)}`;
// Each independent capture opens a new Debugger session. Keep each script
// reachable so V8 cannot collect it before the later sessions attach.
const annotations = Array.from(
{ length: SCRIPT_COUNT },
(_, index) =>
`<script>globalThis["${marker}${index}"]=true;\n//# sourceMappingURL=${fixtureOrigin}/maps/${String(index)}-${"x".repeat(ANNOTATION_BYTES)}.map\n</script>`,
`<script>globalThis["${marker}${index}"]=()=>true;\n//# sourceMappingURL=${fixtureOrigin}/maps/${String(index)}-${"x".repeat(ANNOTATION_BYTES)}.map\n</script>`,
).join("");
page = `<!doctype html><meta charset="utf-8">${annotations}<script>fetch("/capture-metadata-ready")</script>`;
const fixtureUrl = `${fixtureOrigin}/capture-metadata-budget`;

View File

@ -0,0 +1,260 @@
import assert from "node:assert/strict";
import { once } from "node:events";
import { createServer, request } from "node:http";
import { connect } from "node:net";
import { createBrowserScenarioProvider } from "../../dist/composition/browserScenario.js";
import { browserScenarioSchema } from "../../dist/domain/browserScenario.js";
import { PlaywrightScenarioServiceWorkers } from "../../dist/browser/PlaywrightScenarioServiceWorkers.js";
const beforeDeadline = async (promise, label, timeoutMs = 2_000) => {
let timer;
try {
return await Promise.race([
promise,
new Promise((_, reject) => {
timer = setTimeout(
() => reject(new Error(`${label} did not settle`)),
timeoutMs,
);
}),
]);
} finally {
clearTimeout(timer);
}
};
// Real loopback proxy: Playwright's first transport is forwarded to Chrome,
// then only the policy owner's discovery/handshake is deliberately withheld.
const startProxy = async (endpoint, mode, ownerOnly = false) => {
const upstream = new URL(endpoint);
const sockets = new Set();
let versions = 0;
let upgrades = 0;
let enabled = true;
let entered;
const stalled = new Promise((resolve) => {
entered = resolve;
});
let closed;
let stalledClose;
const stall = (socket) => {
socket.resume();
socket.once("end", () => socket.end());
stalledClose = new Promise((resolve) => {
closed = resolve;
});
socket.once("close", () => closed());
entered();
};
const server = createServer((incoming, response) => {
const versionRequest = incoming.url.replace(/\/$/, "") === "/json/version";
if (versionRequest) versions += 1;
if (
enabled &&
mode === "http-discovery" &&
versions === 2 &&
versionRequest
) {
stall(incoming.socket);
return;
}
const outgoing = request(new URL(incoming.url, upstream), (received) => {
const chunks = [];
received.on("data", (chunk) => chunks.push(chunk));
received.on("end", () => {
const content = Buffer.concat(chunks).toString("utf8");
response.writeHead(received.statusCode, {
"content-type": "application/json",
});
response.end(
content.replaceAll(
`127.0.0.1:${upstream.port}`,
`127.0.0.1:${server.address().port}`,
),
);
});
});
outgoing.on("error", () => response.destroy());
response.on("close", () => outgoing.destroy());
outgoing.end();
});
server.on("connection", (socket) => {
sockets.add(socket);
socket.on("close", () => sockets.delete(socket));
});
server.on("upgrade", (incoming, socket, head) => {
upgrades += 1;
if (
enabled &&
mode !== "http-discovery" &&
upgrades === (ownerOnly ? 1 : 2)
) {
stall(socket);
return;
}
const outgoing = connect(Number(upstream.port), "127.0.0.1", () => {
const headers = incoming.rawHeaders.flatMap((value, index, all) =>
index % 2 === 0 ? [`${value}: ${all[index + 1]}`] : [],
);
outgoing.write(
`${incoming.method} ${incoming.url} HTTP/${incoming.httpVersion}\r\n${headers.join("\r\n")}\r\n\r\n`,
);
if (head.length > 0) outgoing.write(head);
outgoing.pipe(socket);
socket.pipe(outgoing);
});
outgoing.on("error", () => socket.destroy());
socket.on("error", () => outgoing.destroy());
socket.on("close", () => outgoing.destroy());
});
server.listen(0, "127.0.0.1");
await once(server, "listening");
const origin = `http://127.0.0.1:${server.address().port}`;
let proxyEndpoint = origin;
if (mode === "browser-websocket") {
const version = await (await fetch(`${endpoint}/json/version`)).json();
proxyEndpoint = version.webSocketDebuggerUrl.replace(
`127.0.0.1:${upstream.port}`,
`127.0.0.1:${server.address().port}`,
);
}
return {
endpoint: proxyEndpoint,
stalled,
waitClosed: () => stalledClose,
resume: () => {
enabled = false;
},
close: async () => {
const done = once(server, "close");
server.close();
for (const socket of sockets) socket.destroy();
await done;
},
};
};
export const verifyScenarioPolicySetup = async (
endpoint,
targetId,
origin,
page,
) => {
const result = {};
for (const mode of ["http-websocket", "http-discovery", "deadline"]) {
const proxy = await startProxy(endpoint, mode);
const provider = createBrowserScenarioProvider(process.env);
const scenario = browserScenarioSchema.parse({
browser: {
mode: "connect",
cdp_endpoint: proxy.endpoint,
target_id: targetId,
},
start_url: { url: origin },
environment: { service_workers: "block" },
actions: [
{ step_id: "settle", action: "wait_for_timeout", duration_ms: 1 },
],
capture: { events: [], after_each_step: ["dom"], at_end: ["dom"] },
});
const cancellation = new AbortController();
let failure;
try {
const pending = provider.captureScenario(scenario, {
signal: cancellation.signal,
});
await beforeDeadline(proxy.stalled, `${mode} setup`);
if (mode !== "deadline") cancellation.abort();
const cancelled = await beforeDeadline(
pending,
`${mode} termination`,
mode === "deadline" ? 12_000 : 2_000,
);
assert.equal(cancelled.ok, false);
assert.equal(cancelled.error.cleanupIncomplete, false);
if (mode === "deadline") assert.equal(cancelled.error.reason, "timeout");
await beforeDeadline(proxy.waitClosed(), `${mode} socket release`);
assert.equal(
await page.evaluate(() =>
navigator.serviceWorker.register.toString().includes("[native code]"),
),
true,
);
proxy.resume();
const next = await beforeDeadline(
provider.captureScenario(scenario),
`${mode} target reacquisition`,
);
if (!next.ok) throw next.error;
assert.equal(next.value.browser.cleanup, "disconnected-external");
result[mode] = {
...(mode === "deadline" ? { timed_out: true } : { cancelled: true }),
socket_closed: true,
target_reused: true,
};
} catch (cause) {
failure = cause;
throw cause;
} finally {
cancellation.abort();
try {
await proxy.close();
} finally {
try {
await provider.close();
} catch (cleanupFailure) {
if (failure !== undefined)
throw new AggregateError([failure, cleanupFailure]);
throw cleanupFailure;
}
}
}
}
const proxy = await startProxy(endpoint, "browser-websocket", true);
const policy = new PlaywrightScenarioServiceWorkers(
page,
proxy.endpoint,
targetId,
);
try {
const rejected = assert.rejects(
policy.initialize(),
(error) => error._tag === "AnalysisCancelledError",
);
await beforeDeadline(proxy.stalled, "browser websocket approval handshake");
await beforeDeadline(policy.close(), "browser websocket cleanup");
await rejected;
await beforeDeadline(
proxy.waitClosed(),
"browser websocket socket release",
);
proxy.resume();
const next = new PlaywrightScenarioServiceWorkers(
page,
proxy.endpoint,
targetId,
);
try {
await beforeDeadline(
next.initialize(),
"browser websocket reacquisition",
);
} finally {
await next.close();
}
result["browser-websocket"] = {
cancelled: true,
socket_closed: true,
target_reused: true,
direct_owner: true,
};
} finally {
try {
await policy.close();
} finally {
await proxy.close();
}
}
return result;
};

View File

@ -0,0 +1,605 @@
import assert from "node:assert/strict";
import { once } from "node:events";
import { createServer } from "node:http";
import { setTimeout as delay } from "node:timers/promises";
import { chromium } from "playwright-core";
import { PlaywrightScenarioSession } from "../../dist/browser/PlaywrightScenarioSession.js";
import { PlaywrightScenarioServiceWorkers } from "../../dist/browser/PlaywrightScenarioServiceWorkers.js";
import { createBrowserScenarioProvider } from "../../dist/composition/browserScenario.js";
import { browserScenarioSchema } from "../../dist/domain/browserScenario.js";
import { runScenarioCli } from "./browser-scenario-verifier.mjs";
import { verifyScenarioPolicySetup } from "./browser-scenario-service-worker-setup-e2e.mjs";
const startSite = async () => {
let otherOrigin;
const server = createServer((request, response) => {
const path = new URL(request.url, "http://localhost").pathname;
if (path === "/sw.js") {
response.setHeader("content-type", "application/javascript");
response.end(`self.addEventListener("install", () => self.skipWaiting());
self.addEventListener("activate", event => event.waitUntil(clients.claim()));
self.addEventListener("fetch", event => {
if (new URL(event.request.url).pathname === "/probe")
event.respondWith(new Response("worker"));
});`);
} else if (path === "/probe") {
response.end("network");
} else if (path === "/failed-navigation") {
response.destroy();
} else if (path === "/stalled-image.png") {
response.writeHead(200, { "content-type": "image/png" });
response.flushHeaders();
} else if (path === "/pending-image" || path === "/pending-child") {
response.setHeader("content-type", "text/html");
response.end(
'<!doctype html><img id="pending-image" src="/stalled-image.png">',
);
} else if (path === "/pending-frame") {
response.setHeader("content-type", "text/html");
response.end(
'<!doctype html><iframe id="pending-frame" src="/pending-child"></iframe>',
);
} else {
response.setHeader("content-type", "text/html");
response.end(`<!doctype html><p id="state">ready</p>
<a id="navigate" href="${otherOrigin}/">Navigate</a>
${path === "/frame" ? `<iframe src="${otherOrigin}/"></iframe>` : ""}
${
path === "/register"
? `<script>
navigator.serviceWorker.register('/sw.js').then(
() => document.querySelector('#state').textContent = 'registered',
() => {
document.querySelector('#state').textContent = 'blocked';
document.querySelector('#state').dataset.blocked = 'true';
});
</script>`
: ""
}`);
}
});
server.listen(0, "127.0.0.1");
await once(server, "listening");
return {
origin: `http://127.0.0.1:${server.address().port}`,
setOtherOrigin: (origin) => {
otherOrigin = origin;
},
close: async () => {
const closed = once(server, "close");
server.close();
server.closeAllConnections();
await closed;
},
};
};
/** Real attached-page policy checks, with caller-owned workers on a second origin. */
export async function verifyScenarioServiceWorkers(endpoint, targetId) {
const first = await startSite();
const second = await startSite();
first.setOtherOrigin(second.origin);
second.setOtherOrigin(first.origin);
const observer = await chromium.connectOverCDP(endpoint);
let provider = createBrowserScenarioProvider(process.env);
let session;
let failure;
try {
let page;
for (const context of observer.contexts()) {
for (const candidate of context.pages()) {
const cdp = await context.newCDPSession(candidate);
const { targetInfo } = await cdp.send("Target.getTargetInfo");
await cdp.detach();
if (targetInfo.targetId === targetId) page = candidate;
}
}
assert.ok(page, "External fixture page disappeared");
const scenarioFor = (url, actions = [], policy = "block") =>
browserScenarioSchema.parse({
browser: {
mode: "connect",
cdp_endpoint: endpoint,
target_id: targetId,
},
start_url: { url },
environment: { service_workers: policy },
actions:
actions.length === 0
? [
{
step_id: "settle",
action: "wait_for_timeout",
duration_ms: 1,
},
]
: actions,
capture: { events: [], after_each_step: ["dom"], at_end: ["dom"] },
});
const assertRestored = async () => {
assert.equal(page.isClosed(), false);
assert.equal(
await page.evaluate(() =>
ServiceWorkerContainer.prototype.register
.toString()
.includes("[native code]"),
),
true,
);
};
const assertRejected = async (scenario) => {
const result = await provider.captureScenario(scenario);
assert.equal(
result.ok,
false,
"Worker-influenced capture unexpectedly succeeded",
);
assert.equal(result.error.reason, "target_not_allowed");
assert.equal(
result.error.partialObservation.capture.browser.cleanup,
"disconnected-external",
);
await assertRestored();
};
// A fresh attached origin blocks registration, then restores both the
// current document and future documents when its transport disconnects.
const clean = scenarioFor(`${first.origin}/register`, [
{
step_id: "registration-rejected",
action: "wait_for",
locator: { kind: "css", selector: "#state[data-blocked='true']" },
state: "visible",
},
]);
const cleanCapture = (await runScenarioCli(clean)).normalized_result;
assert.ok(cleanCapture.steps.every(({ status }) => status === "completed"));
assert.equal(cleanCapture.browser.cleanup, "disconnected-external");
await assertRestored();
await page.reload();
await page.getByText("registered", { exact: true }).waitFor();
await page.evaluate(async () => {
for (const registration of await navigator.serviceWorker.getRegistrations())
await registration.unregister();
});
await page.goto("about:blank");
// The caller installs a real worker at a different origin, then moves the
// selected target away before REA attaches. Its registration must survive.
await page.goto(second.origin);
await page.evaluate(async () => {
await navigator.serviceWorker.register("/sw.js");
await navigator.serviceWorker.ready;
});
await page.reload();
assert.equal(
await page.evaluate(async () => (await fetch("/probe")).text()),
"worker",
);
await page.goto("about:blank");
await assertRejected(scenarioFor(second.origin));
await page.goto("about:blank");
await assertRejected(
scenarioFor(first.origin, [
{
step_id: "cross-origin",
action: "goto",
destination: { url: second.origin },
wait_until: "load",
},
]),
);
await page.goto("about:blank");
await assertRejected(
scenarioFor(first.origin, [
{
step_id: "link-navigation",
action: "click",
locator: { kind: "css", selector: "#navigate" },
},
{ step_id: "loaded", action: "wait_for_timeout", duration_ms: 100 },
]),
);
await page.goto("about:blank");
await assertRejected(scenarioFor(`${first.origin}/frame`));
const allowed = await provider.captureScenario(
scenarioFor(second.origin, [], "allow"),
);
if (!allowed.ok) throw allowed.error;
assert.ok(
allowed.value.steps.every(({ status }) => status === "completed"),
);
assert.equal(
await page.evaluate(async () => (await fetch("/probe")).text()),
"worker",
);
assert.equal(
await page.evaluate(
async () => (await navigator.serviceWorker.getRegistrations()).length,
),
1,
);
// Restoration also runs after navigation failure and active cancellation.
await page.goto("about:blank");
const failed = await provider.captureScenario(
scenarioFor(`${first.origin}/failed-navigation`),
);
assert.equal(failed.ok, false);
// Chromium completes its error-document transition after goto rejects.
await page.waitForSelector("#main-frame-error", { state: "attached" });
// A context replacement can defer restoration. Retained cleanup must
// restore the replacement document before releasing the external page.
await provider.close();
provider = createBrowserScenarioProvider(process.env);
await page.goto(first.origin);
await assertRestored();
const cancellation = new AbortController();
const pending = provider.captureScenario(
scenarioFor(`${first.origin}/register`, [
{
step_id: "wait",
action: "wait_for",
locator: { kind: "css", selector: "#absent" },
state: "visible",
timeout_ms: 30_000,
},
]),
{ signal: cancellation.signal },
);
try {
const deadline = Date.now() + 10_000;
while ((await page.locator("#state").textContent()) !== "blocked") {
assert.ok(
Date.now() < deadline,
"Scenario never applied registration policy",
);
await delay(25);
}
} finally {
cancellation.abort();
}
const cancelled = await pending;
assert.ok(
!cancelled.ok ||
cancelled.value.steps.some(({ status }) => status === "cancelled"),
);
await assertRestored();
// Caller changes made after installation belong to the caller, too.
session = await PlaywrightScenarioSession.open(
scenarioFor(first.origin),
process.env,
{},
);
await page.evaluate(() => {
ServiceWorkerContainer.prototype.register =
function callerReplacement() {};
});
await session.close();
session = undefined;
assert.equal(
await page.evaluate(() => ServiceWorkerContainer.prototype.register.name),
"callerReplacement",
);
await page.reload();
await assertRestored();
await page.evaluate(() => {
delete ServiceWorkerContainer.prototype.register;
});
const policyBrowser = await chromium.connectOverCDP(endpoint);
try {
let policyPage;
for (const context of policyBrowser.contexts()) {
for (const candidate of context.pages()) {
const cdp = await context.newCDPSession(candidate);
const { targetInfo } = await cdp.send("Target.getTargetInfo");
await cdp.detach();
if (targetInfo.targetId === targetId) policyPage = candidate;
}
}
assert.ok(policyPage);
const policy = new PlaywrightScenarioServiceWorkers(
policyPage,
endpoint,
targetId,
);
await policy.initialize();
await policy.close();
} finally {
await policyBrowser.close();
}
assert.equal(
await page.evaluate(() =>
Object.hasOwn(ServiceWorkerContainer.prototype, "register"),
),
false,
);
await page.reload();
await assertRestored();
const policyCleanup = await verifyPolicyCleanup({
page,
endpoint,
targetId,
provider,
scenarioFor,
origin: first.origin,
});
return {
mocked: false,
cli: true,
production_provider: true,
initial_navigation: true,
action_navigation: true,
cross_origin_frame: true,
allow_preserves_existing_worker: true,
registration_blocked: true,
live_document_and_reload_restored: true,
failure_and_cancellation_cleanup: true,
caller_replacement_preserved: true,
caller_original_absence_preserved: true,
policyCleanup,
external_target_preserved: true,
};
} catch (cause) {
failure = cause;
throw cause;
} finally {
await session?.close();
try {
await provider.close();
} catch (cleanupFailure) {
if (failure !== undefined)
throw new AggregateError([failure, cleanupFailure]);
throw cleanupFailure;
} finally {
await observer.close();
await first.close();
await second.close();
}
}
}
const verifyPolicyCleanup = async ({
page,
endpoint,
targetId,
provider,
scenarioFor,
origin,
}) => {
const nativeRegister = async () =>
page.evaluate(() =>
navigator.serviceWorker.register.toString().includes("[native code]"),
);
for (const path of ["/pending-image", "/pending-frame"]) {
const cancellation = new AbortController();
const pending = provider.captureScenario(scenarioFor(`${origin}${path}`), {
signal: cancellation.signal,
});
try {
await page
.locator(
path === "/pending-image" ? "#pending-image" : "#pending-frame",
)
.waitFor({ state: "attached" });
if (path === "/pending-frame") {
const child = page
.frames()
.find((frame) => frame.parentFrame() !== null);
assert.ok(child);
await child.waitForURL(`${origin}/pending-child`, {
waitUntil: "domcontentloaded",
});
}
} finally {
cancellation.abort();
}
const result = await pending;
assert.equal(result.ok, false);
assert.equal(
result.error.partialObservation.capture.browser.cleanup,
"disconnected-external",
);
assert.equal(await nativeRegister(), true);
const next = await provider.captureScenario(scenarioFor(origin));
if (!next.ok) throw next.error;
assert.equal(next.value.browser.cleanup, "disconnected-external");
}
await page.addInitScript("globalThis.reaCallerInitScript = 'preserved'");
const policy = new PlaywrightScenarioServiceWorkers(page, endpoint, targetId);
await policy.initialize();
const exposure = await page.evaluate(async () => {
const register = navigator.serviceWorker.register;
const originals = Object.getOwnPropertySymbols(register).map(
(symbol) => register[symbol],
);
const restore = Object.getOwnPropertyNames(globalThis).find((name) =>
name.startsWith("rea-restore-"),
);
const denied = globalThis[restore]("not-the-session-secret");
let blocked = false;
try {
await navigator.serviceWorker.register("/sw.js");
} catch {
blocked = true;
}
return {
exposesOriginal: originals.some(
(value) => typeof value?.value === "function",
),
denied,
blocked,
registrations: (await navigator.serviceWorker.getRegistrations()).length,
};
});
assert.deepEqual(exposure, {
exposesOriginal: false,
denied: false,
blocked: true,
registrations: 0,
});
// Crossing the renderer boundary must not layer a wrapper over an earlier
// wrapper in a surviving realm, or leave an OOPIF unprotected.
const crossSite = `${origin.replace("127.0.0.1", "localhost")}/register`;
await page.evaluate((url) => {
const child = document.createElement("iframe");
child.id = "cross-site";
child.src = url;
document.body.append(child);
}, crossSite);
const child = page.frames().find((frame) => frame.parentFrame() !== null);
assert.ok(child);
await child.waitForURL(crossSite, { waitUntil: "load" });
assert.equal(await child.locator("#state").textContent(), "blocked");
await page.locator("#cross-site").evaluate((frame, url) => {
frame.src = url;
}, `${origin}/register`);
await child.waitForURL(`${origin}/register`, { waitUntil: "load" });
assert.equal(await child.locator("#state").textContent(), "blocked");
// Gate one real frame evaluation to force a new main document between live
// frame restorations. Browser replies and navigation remain real.
const originalEvaluate = child.evaluate.bind(child);
let release;
const gate = new Promise((resolve) => {
release = resolve;
});
let entered;
const started = new Promise((resolve) => {
entered = resolve;
});
child.evaluate = async (...args) => {
entered();
await gate;
return originalEvaluate(...args);
};
const closing = policy.close();
void closing.catch(() => undefined);
try {
await started;
await page.goto(origin);
} finally {
release();
child.evaluate = originalEvaluate;
}
await closing;
assert.equal(await nativeRegister(), true);
await page.reload();
assert.equal(await nativeRegister(), true);
assert.equal(
await page.evaluate(() => globalThis.reaCallerInitScript),
"preserved",
);
assert.equal(
await page.evaluate(() =>
Object.getOwnPropertyNames(globalThis).some((name) =>
name.startsWith("rea-restore-"),
),
),
false,
);
await verifyRejectedPolicyInstallation(page.context(), endpoint, origin);
const setup = await verifyScenarioPolicySetup(
endpoint,
targetId,
origin,
page,
);
return {
stalled_image_cancelled: true,
stalled_frame_cancelled: true,
target_reused_after_cancel: true,
private_original_descriptor: true,
wrong_restore_token_rejected: true,
renderer_transition: true,
navigation_during_restoration: true,
caller_init_script_preserved: true,
failed_main_installation_rejected: true,
failed_accessor_installation_rejected: true,
failed_iframe_installation_rejected: true,
replaced_iframe_policy_rejected: true,
setup,
};
};
const verifyRejectedPolicyInstallation = async (context, endpoint, origin) => {
for (const realm of ["main", "accessor", "existing-iframe", "new-iframe"]) {
const page = await context.newPage();
let policy;
try {
await page.goto(origin);
const cdp = await context.newCDPSession(page);
const { targetInfo } = await cdp.send("Target.getTargetInfo");
await cdp.detach();
policy = new PlaywrightScenarioServiceWorkers(
page,
endpoint,
targetInfo.targetId,
);
if (realm === "new-iframe") await policy.initialize();
let frame = page.mainFrame();
if (realm === "existing-iframe" || realm === "new-iframe") {
const crossSite = origin.replace("127.0.0.1", "localhost");
await page.evaluate((url) => {
const child = document.createElement("iframe");
child.src = url;
document.body.append(child);
}, crossSite);
frame = page
.frames()
.find((candidate) => candidate.parentFrame() !== null);
assert.ok(frame);
await frame.waitForURL(crossSite, { waitUntil: "load" });
}
await frame.evaluate((kind) => {
const prototype = ServiceWorkerContainer.prototype;
const descriptor = Object.getOwnPropertyDescriptor(
prototype,
"register",
);
if (kind === "accessor") {
const original = descriptor.value;
Object.defineProperty(prototype, "register", {
configurable: true,
enumerable: descriptor.enumerable,
get: () => original,
});
return;
}
Object.defineProperty(prototype, "register", {
...descriptor,
configurable: false,
...(kind === "new-iframe"
? { value: function callerReplacement() {} }
: {}),
});
}, realm);
await assert.rejects(
realm === "new-iframe" ? policy.assertBlocked() : policy.initialize(),
(error) => error.reason === "target_not_allowed",
);
await policy.close();
assert.equal(
await frame.evaluate((replaced) => {
const register = navigator.serviceWorker.register;
return replaced
? register.name === "callerReplacement"
: register.toString().includes("[native code]");
}, realm === "new-iframe"),
true,
);
} finally {
try {
await policy?.close();
} finally {
await page.close();
}
}
}
};

View File

@ -167,6 +167,7 @@ export const providerCatalog = (sources) => {
reconciliationContracts,
observationContracts,
activeContracts,
fuseContracts,
} = electronContractSlices(sources);
return [
{
@ -261,6 +262,10 @@ export const providerCatalog = (sources) => {
identity: sources.rustProvider.RUST_BINARY_PROVIDER_IDENTITY,
contracts: sources.rustContracts.RUST_TOOL_CONTRACTS,
},
{
identity: sources.electronFuseProvider.ELECTRON_FUSE_PROVIDER_IDENTITY,
contracts: fuseContracts,
},
{
identity: sources.sqliteDomain.SQLITE_PROVIDER_IDENTITY,
contracts: sources.sqliteContracts.SQLITE_TOOL_CONTRACTS,
@ -425,17 +430,23 @@ const electronContractSlices = (sources) => {
({ name }) =>
name !== "analyze_javascript_application" &&
name !== "reconcile_javascript_runtime" &&
name !== "capture_electron_scenario",
name !== "capture_electron_scenario" &&
name !== "inspect_electron_fuses",
);
const activeContracts =
sources.electronContracts.ELECTRON_TOOL_CONTRACTS.filter(
({ name }) => name === "capture_electron_scenario",
);
const fuseContracts =
sources.electronContracts.ELECTRON_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_electron_fuses",
);
if (
applicationContracts.length !== 1 ||
reconciliationContracts.length !== 1 ||
observationContracts.length !== 2 ||
activeContracts.length !== 1
activeContracts.length !== 1 ||
fuseContracts.length !== 1
)
throw new Error("Electron provider capability ownership drifted");
return {
@ -443,5 +454,6 @@ const electronContractSlices = (sources) => {
reconciliationContracts,
observationContracts,
activeContracts,
fuseContracts,
};
};

View File

@ -96,6 +96,7 @@ const SOURCE_PATHS = {
webNetworkCaptureProvider: "dist/browser/history/CaptureRelease.js",
browserScenarioContracts: "dist/contracts/browserScenarioToolContracts.js",
electronContracts: "dist/contracts/javascript/electronToolContracts.js",
electronFuseProvider: "dist/electron/ElectronFuseProvider.js",
javascriptRuntimeObservationContracts:
"dist/contracts/javascript/javascriptRuntimeObservationToolContracts.js",
applicationContracts: "dist/contracts/applicationToolContracts.js",

View File

@ -1,8 +1,36 @@
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { readdir, stat } from "node:fs/promises";
import { readFile, readdir, stat } from "node:fs/promises";
import { basename } from "node:path";
const execFileAsync = promisify(execFile);
// CPython's interface option ends interpreter parsing; following values are
// script arguments. -W/-X consume either an attached value or the next operand.
// https://docs.python.org/3.12/using/cmdline.html
const pythonScriptOperand = (arguments_) => {
for (let index = 1; index < arguments_.length; index++) {
const option = arguments_[index];
if (option === "--") return arguments_[index + 1];
if (option === "-" || option === "") return undefined;
if (!option.startsWith("-")) return option;
if (option === "--check-hash-based-pycs") {
index++;
continue;
}
if (/^--(?:help(?:-env|-xoptions|-all)?|version)$/u.test(option))
return undefined;
if (option.startsWith("--"))
throw new Error(`Unknown Python helper invocation option: ${option}`);
for (let character = 1; character < option.length; character++) {
const flag = option[character];
if ("cmh?V".includes(flag)) return undefined;
if (flag === "W" || flag === "X") {
if (character === option.length - 1) index++;
break;
}
if (!"bBdEiIOPqRsSuvx".includes(flag))
throw new Error(`Unknown Python helper invocation option: ${option}`);
}
}
return undefined;
};
/**
* Observe leftover demo processes on a dedicated clean Linux runner, including
@ -18,9 +46,6 @@ export const assertCleanLinuxHopperRuntime = async (
],
procRoot = "/proc",
) => {
const { stdout } = await execFileAsync("ps", ["-ax", "-o", "command="]);
if (/\/opt\/hopper\/bin\/Hopper|\bXvfb\b|hopper-demo-x11\.py/u.test(stdout))
throw new Error("Clean Linux runner retained a Hopper demo process");
const launchers = [];
for (const path of launcherPaths) {
try {
@ -29,20 +54,55 @@ export const assertCleanLinuxHopperRuntime = async (
if (error?.code !== "ENOENT") throw error;
}
}
if (launchers.length === 0) return;
// /proc exposes the executed file even when argv[0] names a removed symlink.
// Compare file identity, not a display path or a process name.
let inspectionError;
for (const entry of await readdir(procRoot, { withFileTypes: true })) {
if (!entry.isDirectory() || !/^\d+$/u.test(entry.name)) continue;
let executable;
let retainedRuntime;
try {
executable = await stat(`${procRoot}/${entry.name}/exe`, {
bigint: true,
});
// procfs preserves argument boundaries. A caller merely mentioning a
// runtime in its script text or other arguments is not that runtime.
const commandLine = await readFile(
`${procRoot}/${entry.name}/cmdline`,
"utf8",
);
const arguments_ = commandLine.split("\0");
// The kernel command field identifies renamed runtime processes such
// as Xvfb. Keep it separate from arguments so ordinary script text does
// not count as a retained runtime.
const kernelCommand = (
await readFile(`${procRoot}/${entry.name}/comm`, "utf8")
).replace(/\n$/u, "");
if (
commandLine.length === 0 &&
/^python3(?:\.\d+)?$/u.test(kernelCommand)
)
throw new Error(
"Cannot identify a command-less Python process on the clean Linux runner",
);
retainedRuntime =
kernelCommand === "Hopper" ||
kernelCommand === "Xvfb" ||
arguments_[0] === "/opt/hopper/bin/Hopper" ||
basename(arguments_[0] ?? "") === "Xvfb" ||
(/^python3(?:\.\d+)?$/u.test(basename(arguments_[0] ?? "")) &&
arguments_.some(
(value) => basename(value) === "hopper-demo-x11.py",
) &&
basename(pythonScriptOperand(arguments_) ?? "") ===
"hopper-demo-x11.py");
if (!retainedRuntime && launchers.length > 0) {
const executable = await stat(`${procRoot}/${entry.name}/exe`, {
bigint: true,
});
retainedRuntime = launchers.some(
({ dev, ino }) => dev === executable.dev && ino === executable.ino,
);
}
} catch (error) {
if (error?.code === "ENOENT" || error?.code === "ESRCH") continue;
// Other users' executables may be unreadable. The setup lane launches
// Other users' process facts may be unreadable. The setup lane launches
// providers as the caller; unreadable caller-owned processes fail closed.
if (error?.code === "EACCES") {
const owner = await stat(`${procRoot}/${entry.name}`).catch((cause) => {
@ -56,11 +116,7 @@ export const assertCleanLinuxHopperRuntime = async (
inspectionError ??= error;
continue;
}
if (
launchers.some(
({ dev, ino }) => dev === executable.dev && ino === executable.ino,
)
)
if (retainedRuntime)
throw new Error("Clean Linux runner retained a Hopper demo process");
}
if (inspectionError !== undefined) throw inspectionError;

View File

@ -0,0 +1,149 @@
import assert from "node:assert/strict";
export const WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE = Object.freeze({
pattern: "a0 ?? c0",
mask: "f0 ff f0",
startFileOffset: 0x380,
endFileOffset: 0x392,
markerOffsets: Object.freeze([0x380, 0x390]),
});
/** Put two masked, wildcarded signatures in loaded, source-owned PE bytes. */
export function writeWindowsGhidraByteSearchFixture(bytes) {
bytes.set([0xa1, 0xb2, 0xc3], 0x380);
bytes.set([0xa4, 0xb5, 0xc6], 0x390);
}
/** Verify real Ghidra search pages against the immutable source fixture. */
export async function verifyWindowsGhidraByteSearch({
invoke,
sourceBytes,
architecture,
sourceOwnedFixture = true,
fallbackAddress,
fallbackFileOffset,
runCliPage,
}) {
const imageBase = architecture === "x86" ? 0x400000n : 0x140000000n;
const query = sourceOwnedFixture
? {
pattern: WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE.pattern,
mask: WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE.mask,
start: addressAtFileOffset(
imageBase,
WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE.startFileOffset,
),
end: addressAtFileOffset(
imageBase,
WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE.endFileOffset,
),
limit: 1,
}
: {
pattern: sourceBytes
.subarray(fallbackFileOffset, fallbackFileOffset + 1)
.toString("hex"),
start: fallbackAddress,
end: fallbackAddress,
limit: 1,
};
if (!sourceOwnedFixture) {
const result = await invoke("search_bytes", query);
assert.equal(result.complete, true);
assert.equal(result.next_cursor, null);
assert.deepEqual(
result.matches.map(({ address, file_offset }) => ({
address,
file_offset,
})),
[{ address: fallbackAddress, file_offset: fallbackFileOffset }],
);
assert.ok(result.searched.bytes > 0);
return { bounded_source_match: true };
}
const { startFileOffset, endFileOffset } = WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE;
const expected = [];
for (
let fileOffset = startFileOffset;
fileOffset + 2 <= endFileOffset;
fileOffset += 1
) {
const first = sourceBytes[fileOffset];
const last = sourceBytes[fileOffset + 2];
if (first === undefined || last === undefined) break;
if ((first & 0xf0) === 0xa0 && (last & 0xf0) === 0xc0)
expected.push({
address: addressAtFileOffset(imageBase, fileOffset),
file_offset: fileOffset,
});
}
assert.deepEqual(
expected.map(({ file_offset }) => file_offset),
[...WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE.markerOffsets],
"The source-owned byte oracle must retain both distinct signatures",
);
const first = await invoke("search_bytes", query);
assert.deepEqual(
first.matches.map(({ address, file_offset }) => ({ address, file_offset })),
[expected[0]],
);
assert.equal(first.complete, false);
assert.equal(typeof first.next_cursor, "string");
assert.ok(first.searched.bytes > 0);
if (runCliPage !== undefined) {
const cli = await runCliPage(query);
assert.deepEqual(cli.matches, first.matches);
assert.deepEqual(cli.searched, first.searched);
assert.equal(cli.complete, first.complete);
}
const second = await invoke("search_bytes", {
...query,
cursor: first.next_cursor,
});
assert.deepEqual(
second.matches.map(({ address, file_offset }) => ({
address,
file_offset,
})),
[expected[1]],
);
assert.equal(second.complete, true);
assert.equal(second.next_cursor, null);
assert.ok(second.searched.bytes > 0);
const covered = new Set();
for (const page of [first, second]) {
assert.equal(
page.searched.bytes,
page.searched.blocks.reduce((sum, block) => sum + block.bytes, 0),
);
for (const block of page.searched.blocks) {
const start = BigInt(block.start);
const end = BigInt(block.end);
assert.ok(start >= BigInt(query.start) && end <= BigInt(query.end));
for (let offset = start; offset <= end; offset += 1n)
covered.add(offset.toString());
}
}
assert.equal(
covered.size,
endFileOffset - startFileOffset + 1,
"Paged byte-search coverage must account for the complete bounded source range",
);
return {
bounded: true,
wildcardAndMask: true,
matches: expected,
continuation: true,
examinedBytes: covered.size,
cliMcpParity: runCliPage !== undefined,
};
}
function addressAtFileOffset(imageBase, fileOffset) {
return `0x${(imageBase + 0x1000n + BigInt(fileOffset - 0x200)).toString(16)}`;
}

View File

@ -62,36 +62,68 @@ export async function verifyGhidraAddressBoundaries(client, entry, cli) {
await query("read_bytes", { address: padded, length: 1 }),
await query("read_bytes", { address: entry, length: 1 }),
);
const aliasKeys = [entry, padded];
assert.deepEqual(
await query("set_addresses_names", {
names: {
[entry]: "rea_same_address_alias",
[padded]: "rea_same_address_alias",
},
}),
Object.fromEntries(aliasKeys.map((address) => [address, true])),
"Equivalent address spellings with one requested name share one edit",
);
const nameBeforeAliasedBatch = await query("address_name", {
address: entry,
});
assert.equal(nameBeforeAliasedBatch, "rea_same_address_alias");
assert.deepEqual(
await query("set_addresses_names", {
names: {
[entry]: "rea_first_address_alias",
[padded]: "rea_second_address_alias",
},
}),
Object.fromEntries(aliasKeys.map((address) => [address, false])),
"Conflicting spellings of one Ghidra address must be rejected together",
);
const originalName = await query("address_name", { address: entry });
assert.equal(
await query("address_name", { address: entry }),
nameBeforeAliasedBatch,
"A rejected alias collision must leave the preexisting target name unchanged",
typeof originalName,
"string",
"The function entry needs an observed name that can be restored",
);
const aliasKeys = [entry, padded];
let primaryFailure;
try {
assert.deepEqual(
await query("set_addresses_names", {
names: {
[entry]: "rea_same_address_alias",
[padded]: "rea_same_address_alias",
},
}),
Object.fromEntries(aliasKeys.map((address) => [address, true])),
"Equivalent address spellings with one requested name share one edit",
);
const nameBeforeAliasedBatch = await query("address_name", {
address: entry,
});
assert.equal(nameBeforeAliasedBatch, "rea_same_address_alias");
assert.deepEqual(
await query("set_addresses_names", {
names: {
[entry]: "rea_first_address_alias",
[padded]: "rea_second_address_alias",
},
}),
Object.fromEntries(aliasKeys.map((address) => [address, false])),
"Conflicting spellings of one Ghidra address must be rejected together",
);
assert.equal(
await query("address_name", { address: entry }),
nameBeforeAliasedBatch,
"A rejected alias collision must leave the preexisting target name unchanged",
);
} catch (cause) {
primaryFailure = cause;
throw cause;
} finally {
try {
assert.equal(
await query("set_address_name", { address: entry, name: originalName }),
true,
"The temporary alias edit must be restored",
);
assert.equal(
await query("address_name", { address: entry }),
originalName,
"The original function-entry name must survive restoration",
);
} catch (cleanupCause) {
if (primaryFailure !== undefined)
throw new AggregateError(
[primaryFailure, cleanupCause],
"Ghidra address-boundary verification and alias restoration failed",
);
throw cleanupCause;
}
}
for (const [command, flags, operation] of [
["read-bytes", ["--length", "1"], "read_bytes"],
[

View File

@ -1,3 +1,7 @@
import type { z } from "zod";
import type { analysisStringSchema } from "../../src/domain/hopperValues.js";
type VerifierCall = (
name: string,
arguments_?: Record<string, unknown>,
@ -44,6 +48,3 @@ export function verifyHopperStringObjects(
}[];
readonly displayEscapesExcludedFromMatching: true;
}>;
import type { z } from "zod";
import type { analysisStringSchema } from "../../src/domain/hopperValues.js";

View File

@ -17,7 +17,7 @@ export const SOURCE_TEST_PROJECTS = [
group: "source",
include: [
"src/*.test.ts",
"src/{apktool,jeb,adb,artifacts,browser,dotnet,evm,firmware,flutter,pcap,ghidra,go,hopper,ida,inspector,javascript,native,process,reference,rust,server,wasm,windows}/**/*.test.ts",
"src/{apktool,jeb,adb,artifacts,browser,dotnet,electron,evm,firmware,flutter,pcap,ghidra,go,hopper,ida,inspector,javascript,native,process,reference,rust,server,wasm,windows}/**/*.test.ts",
],
},
{

View File

@ -68,7 +68,11 @@ try {
}
if (!process.exitCode && !shutdownSignal) {
const tested = await run(process.execPath, [
resolve("node_modules/vitest/vitest.mjs"),
resolve(
request.paths.length > 0
? "scripts/run-exact-tests.mjs"
: "node_modules/vitest/vitest.mjs",
),
...plan.vitestArguments,
]);
process.exitCode = tested.code;

134
scripts/run-exact-tests.mjs Normal file
View File

@ -0,0 +1,134 @@
import { lstat, realpath, stat } from "node:fs/promises";
import { parse, relative, resolve, sep } from "node:path";
import { createVitest, parseCLI } from "vitest/node";
// CLI file filters are substring matches. Select discovered specifications by
// identity instead, retaining each project's include/exclude and runner settings.
const { filter, options } = parseCLI(["vitest", ...process.argv.slice(2)]);
const vitest = await createVitest({ ...options, watch: false });
const signalExitCodes = { SIGINT: 130, SIGTERM: 143 };
let shutdownSignal;
let closePromise;
const close = () => (closePromise ??= vitest.close());
for (const signal of Object.keys(signalExitCodes))
process.on(signal, () => {
if (shutdownSignal !== undefined) return;
shutdownSignal = signal;
void close().catch((cause) => {
console.error(cause instanceof Error ? cause.message : String(cause));
process.exitCode = 1;
});
});
try {
if (shutdownSignal === undefined) await vitest.standalone();
if (shutdownSignal === undefined) {
const discovered = new Map();
for (const specification of await vitest.globTestSpecifications()) {
const path = resolve(specification.moduleId);
const specifications = discovered.get(path) ?? [];
specifications.push(specification);
discovered.set(path, specifications);
}
const selected = new Set();
const missing = [];
let discoveredByIdentity;
for (const path of filter) {
const absolute = resolve(path);
let matches = discovered.get(absolute) ?? [];
if (matches.length === 0) {
try {
await lstat(absolute);
} catch (cause) {
if (cause?.code === "ENOENT" || cause?.code === "ENOTDIR") {
missing.push(path);
continue;
}
throw cause;
}
const [canonical, selectedIdentity] = await Promise.all([
realpath(absolute),
stat(absolute, { bigint: true }),
]);
if (discoveredByIdentity === undefined) {
discoveredByIdentity = new Map();
for (const [candidate, specifications] of discovered) {
try {
const entry = await stat(candidate, { bigint: true });
const identity = `${entry.dev}:${entry.ino}`;
const entries = discoveredByIdentity.get(identity) ?? [];
entries.push([candidate, specifications]);
discoveredByIdentity.set(identity, entries);
} catch (cause) {
if (cause?.code === "ENOENT" || cause?.code === "ENOTDIR")
continue;
throw cause;
}
}
}
const identity = `${selectedIdentity.dev}:${selectedIdentity.ino}`;
for (const [candidate, specifications] of discoveredByIdentity.get(
identity,
) ?? []) {
try {
await lstat(candidate);
} catch (cause) {
if (cause?.code === "ENOENT" || cause?.code === "ENOTDIR") continue;
throw cause;
}
if (
!(await hasCaseMismatchedSymlinkComponent(absolute, candidate)) &&
(await realpath(candidate)) === canonical
)
matches = [...matches, ...specifications];
}
}
if (matches.length === 0) missing.push(path);
for (const specification of matches) selected.add(specification);
}
if (missing.length > 0 || selected.size === 0)
throw new Error(
`Selected test files were not discovered: ${missing.join(", ")}`,
);
if (shutdownSignal === undefined)
await vitest.runTestSpecifications([...selected]);
}
} finally {
await close();
}
if (shutdownSignal !== undefined)
process.exitCode = signalExitCodes[shutdownSignal];
async function hasCaseMismatchedSymlinkComponent(selected, candidate) {
const selectedAbsolute = resolve(selected);
const candidateAbsolute = resolve(candidate);
const root = parse(selectedAbsolute).root;
if (root !== parse(candidateAbsolute).root) return true;
const selectedParts = relative(root, selectedAbsolute).split(sep);
const candidateParts = relative(root, candidateAbsolute).split(sep);
let selectedPath = root;
let candidatePath = root;
for (let index = 0; index < selectedParts.length; index += 1) {
const selectedPart = selectedParts[index];
const candidatePart = candidateParts[index];
if (selectedPart === undefined || candidatePart === undefined) return true;
selectedPath = selectedPath
? resolve(selectedPath, selectedPart)
: selectedPart;
candidatePath = candidatePath
? resolve(candidatePath, candidatePart)
: candidatePart;
if (
selectedPart !== candidatePart &&
selectedPart.toLowerCase() === candidatePart.toLowerCase()
) {
const [selectedEntry, candidateEntry] = await Promise.all([
lstat(selectedPath),
lstat(candidatePath),
]);
if (selectedEntry.isSymbolicLink() || candidateEntry.isSymbolicLink())
return true;
}
}
return false;
}

View File

@ -155,6 +155,18 @@ try {
requiredAnswerTermGroups: [["profileapi"], ["profile:read"]],
prompt: `請靜態分析已交付的 Electron 程式 ${targets.javascript},說明 renderer 如何透過 preload 讀取 profile,以及 main handler 的關係。不要執行目標或安裝分析引擎;引用完整的產出 Evidence,保留 runtime 未知事項。`,
},
{
id: "notes-export",
expectedFirstTool: "analyze_javascript_application",
requiresEvidence: true,
requiredAnswerTermGroups: [],
requiredToolSubsequence: [
"analyze_javascript_application",
"trace_application_feature",
"trace_javascript_semantics",
],
prompt: `Use REA on the shipped Electron application at ${targets.notesElectron} to investigate its notes export API, preload exposure and main handler. Do not read target files directly or launch Electron. Use the returned application Evidence to trace the exact notes:export IPC channel in both directions, then query semantic relations for the property invoke with forward influence, excluding ambiguous dynamic edges and without an expected sink. Report the selected static facts, source locations and coverage limits using the closed claim contract.`,
},
{
id: "javascript-module-view",
expectedFirstTool: "analyze_javascript_application",

View File

@ -3,7 +3,7 @@ import assert from "node:assert/strict";
import { isUtf8 } from "node:buffer";
import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { lstat, mkdir, readFile, writeFile } from "node:fs/promises";
import { isAbsolute, join } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
@ -24,6 +24,8 @@ if (process.env.REA_VERIFY_GO_COMMAND !== undefined && !isAbsolute(compiler))
let compilerVersion = null;
const run = createVerifierRun();
const root = await PrivateRuntimeRoot.create({ prefix: "rea-go-verifier-" });
// Keep Go's directory enumeration below the private root's Windows ownership lease.
const sourceRoot = join(root.path, "source");
const entrypoint = fileURLToPath(new URL("./rea.mjs", import.meta.url));
const environment = Object.fromEntries(
Object.entries(process.env).filter(([, value]) => typeof value === "string"),
@ -115,14 +117,54 @@ try {
],
{ GOOS: os, GOARCH: arch },
);
await verify(path, target, oracle, null, "stripped");
const strippedInfo = await verify(path, target, oracle, null, "stripped");
if (os === "linux" && arch === "amd64") {
const sectionless = join(root.path, "linux-amd64-sectionless");
const bytes = Buffer.from(await readFile(path));
bytes.fill(0, 40, 48); // ELF64 e_shoff.
bytes.fill(0, 58, 64); // e_shentsize, e_shnum and e_shstrndx.
await writeFile(sectionless, bytes);
await verify(sectionless, target, oracle, null, "sectionless");
const sectionlessInfo = await verify(
sectionless,
target,
oracle,
null,
"sectionless",
path,
);
assert.deepEqual(sectionlessInfo, strippedInfo);
const pie = join(root.path, "linux-amd64-pie");
await go(
[
"build",
"-trimpath",
"-buildvcs=false",
"-ldflags=-s -w",
"-buildmode=pie",
"-o",
pie,
".",
],
{ GOOS: os, GOARCH: arch },
);
const originalInfo = await verify(pie, target, oracle, null, "pie");
const pieBytes = Buffer.from(await readFile(pie));
pieBytes.fill(0, 40, 48);
pieBytes.fill(0, 58, 64);
const pieSectionless = `${pie}-sectionless`;
await writeFile(pieSectionless, pieBytes);
// Keep the producer image as the independent oracle. Removing section
// headers must preserve every metadata field and original source span.
const retainedInfo = await verify(
pieSectionless,
target,
oracle,
null,
"sectionless-pie",
pie,
);
assert.deepEqual(retainedInfo, originalInfo);
}
}
for (const target of targets.filter(([os]) => os === "darwin")) {
@ -143,7 +185,7 @@ try {
const expected = JSON.parse(
(
await execute(oracle, ["--build-byte-fixture", compiler, path], {
cwd: root.path,
cwd: sourceRoot,
env: { ...goEnvironment, GOOS: "linux", GOARCH: "amd64" },
timeout: 120_000,
maxBuffer: 1024 * 1024,
@ -169,7 +211,7 @@ try {
}
const verifier = await completeVerifierRun(run);
try {
await assert.rejects(readFile(join(root.path, "go.mod")), { code: "ENOENT" });
await assert.rejects(lstat(root.path), { code: "ENOENT" });
if (verifier.process_lineage.status === "verified")
assert.deepEqual(verifier.process_lineage.descendants, []);
} catch (cause) {
@ -214,7 +256,7 @@ console.log(
async function go(args, overrides = {}) {
return execute(compiler, args, {
cwd: root.path,
cwd: sourceRoot,
env: { ...goEnvironment, ...overrides },
timeout: 120_000,
maxBuffer: 1024 * 1024,
@ -222,29 +264,30 @@ async function go(args, overrides = {}) {
}
async function prepareFixture() {
await mkdir(sourceRoot);
await mkdir(join(root.path, "tmp"));
await mkdir(join(root.path, "telemetry"));
await writeFile(join(root.path, "telemetry", "mode"), "off\n");
await mkdir(join(root.path, "dependency"));
await mkdir(join(root.path, "oracle"));
await mkdir(join(sourceRoot, "dependency"));
await mkdir(join(sourceRoot, "oracle"));
await writeFile(
join(root.path, "go.mod"),
join(sourceRoot, "go.mod"),
"module example.test/rea-go-fixture\n\ngo 1.20\n\nrequire example.test/dependency v1.2.3\nreplace example.test/dependency => ./dependency\n",
);
await writeFile(
join(root.path, "dependency", "go.mod"),
join(sourceRoot, "dependency", "go.mod"),
"module example.test/dependency\n\ngo 1.20\n",
);
await writeFile(
join(root.path, "dependency", "dependency.go"),
join(sourceRoot, "dependency", "dependency.go"),
'package dependency\nfunc Value() string { return "REA fixture" }\n',
);
await writeFile(
join(root.path, "main.go"),
join(sourceRoot, "main.go"),
'package main\nimport ("fmt"; "example.test/dependency")\nfunc main() { fmt.Println(dependency.Value()) }\n',
);
await writeFile(
join(root.path, "oracle", "main.go"),
join(sourceRoot, "oracle", "main.go"),
String.raw`package main
import (
@ -332,13 +375,14 @@ async function verify(
oracle,
byteExpected = null,
fileKind = "stripped",
oraclePath = path,
) {
const bytes = await readFile(path);
const expected =
byteExpected ??
JSON.parse(
(
await execute(oracle, [path], {
await execute(oracle, [oraclePath], {
timeout: 10_000,
maxBuffer: 1024 * 1024,
})
@ -408,6 +452,7 @@ async function verify(
? {}
: { go_version_bytes_base64: expected.GoVersionBytesBase64 }),
});
return reports[0].build_info;
}
function verifyByteStrings(info, expected) {

View File

@ -1,6 +1,6 @@
import { execFile } from "node:child_process";
import { readFile } from "node:fs/promises";
import { resolve } from "node:path";
import { readdir, readFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { promisify } from "node:util";
import { inspectModuleBoundaries } from "./lib/module-boundaries.mjs";
@ -11,24 +11,44 @@ const selected = process.argv.slice(2);
for (const file of selected)
if (!file.endsWith(".ts"))
throw new TypeError(`Expected an exact TypeScript source file: ${file}`);
const files =
selected.length > 0
? selected
: (
await execute(
"git",
[
"ls-files",
"-z",
"--cached",
"--others",
"--exclude-standard",
"--",
"src",
],
{ cwd: root, maxBuffer: 8 * 1024 * 1024 },
)
).stdout.split("\0");
const enumerateSourceFiles = async () => {
if (selected.length > 0) return selected;
let isGitRoot = false;
try {
const { stdout } = await execute("git", ["rev-parse", "--show-cdup"], {
cwd: root,
maxBuffer: 8 * 1024 * 1024,
});
isGitRoot = stdout.trim() === "";
} catch {
// Archives and source snapshots may not be in a Git worktree.
}
if (isGitRoot) {
try {
const { stdout } = await execute(
"git",
[
"ls-files",
"-z",
"--cached",
"--others",
"--exclude-standard",
"--",
"src",
],
{ cwd: root, maxBuffer: 8 * 1024 * 1024 },
);
return stdout.split("\0");
} catch {
// Fall back to disk when Git's index is unavailable or unreadable.
}
}
const entries = await readdir(resolve(root, "src"), { recursive: true });
return entries.map((entry) => join("src", entry));
};
const files = await enumerateSourceFiles();
let checked = 0;
for (const file of new Set(files.filter((path) => path.endsWith(".ts")))) {
@ -36,7 +56,11 @@ for (const file of new Set(files.filter((path) => path.endsWith(".ts")))) {
try {
source = await readFile(resolve(root, file), "utf8");
} catch (cause) {
if (selected.length === 0 && cause.code === "ENOENT") continue;
if (
selected.length === 0 &&
(cause.code === "ENOENT" || cause.code === "EISDIR")
)
continue;
throw cause;
}
checked += 1;

View File

@ -2,7 +2,7 @@ import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { isAbsolute, join, relative, sep } from "node:path";
import { fileURLToPath } from "node:url";
import Ajv2020 from "ajv/dist/2020.js";
import {
@ -42,6 +42,21 @@ const skillDirectory = join(
"skills",
"reverse-engineer-anything",
);
const isManagedSkillTarget = (target) => {
const value = relative(skillDirectory, target);
return (
value === "" ||
(value !== ".." && !value.startsWith(`..${sep}`) && !isAbsolute(value))
);
};
assert.equal(
isManagedSkillTarget(`${skillDirectory}-sibling/AGENTS.md`),
false,
);
assert.equal(
isManagedSkillTarget(join(skillDirectory, "..", "outside", "AGENTS.md")),
false,
);
await Promise.all([mkdir(account), mkdir(profile), mkdir(workspace)]);
await exec("git", ["init", "--quiet", workspace], { timeout: 30_000 });
const environment = {
@ -264,8 +279,7 @@ try {
assert(parsed.plannedActions.some((a) => a.target === skillDirectory));
assert(
parsed.plannedActions.every(
(a) =>
a.target === configPath || a.target.startsWith(skillDirectory),
(a) => a.target === configPath || isManagedSkillTarget(a.target),
),
);
}

View File

@ -31,7 +31,14 @@ export async function verifyPackageCapabilitiesAndSearch({ cli, environment }) {
const searchResult = json(
await run(
cli,
["search", process.execPath, "fixture", "--json"],
[
"search",
process.execPath,
"fixture",
"--provider",
"hopper",
"--json",
],
environment,
),
);

View File

@ -10,7 +10,7 @@ export async function verifyPackagePlatform({
if (platform === "linux") {
const unsupportedExecution = await runWithStatus(
cli,
["analyze", process.execPath, "--json"],
["analyze", process.execPath, "--provider", "hopper", "--json"],
environment,
);
assertLinuxPackageProviderFailure(unsupportedExecution);
@ -18,7 +18,7 @@ export async function verifyPackagePlatform({
const overview = json(
await runCommand(
cli,
["analyze", process.execPath, "--json"],
["analyze", process.execPath, "--provider", "hopper", "--json"],
environment,
),
);
@ -29,7 +29,7 @@ export async function verifyPackagePlatform({
const inspected = json(
await runCommand(
cli,
["inspect", process.execPath, "--json"],
["inspect", process.execPath, "--provider", "hopper", "--json"],
environment,
),
);
@ -38,7 +38,14 @@ export async function verifyPackagePlatform({
const functionResult = json(
await runCommand(
cli,
["function", process.execPath, "0x1000", "--json"],
[
"function",
process.execPath,
"0x1000",
"--provider",
"hopper",
"--json",
],
environment,
),
);
@ -53,7 +60,7 @@ export async function verifyPackagePlatform({
const xrefs = json(
await runCommand(
cli,
["xrefs", process.execPath, "0x1000", "--json"],
["xrefs", process.execPath, "0x1000", "--provider", "hopper", "--json"],
environment,
),
);
@ -65,7 +72,14 @@ export async function verifyPackagePlatform({
const trace = json(
await runCommand(
cli,
["trace", process.execPath, "fixture", "--json"],
[
"trace",
process.execPath,
"fixture",
"--provider",
"hopper",
"--json",
],
environment,
),
);

View File

@ -0,0 +1,6 @@
/** Validate a live registration and its byte-identical pre-setup backup. */
export function assertClientConfig(
cli: string,
configPath: string,
original: Uint8Array,
): Promise<void>;

View File

@ -414,7 +414,8 @@ const verifySetupApply = async ({
};
};
const assertClientConfig = async (cli, configPath) => {
/** Verify the live registration and exact pre-setup backup bytes. */
export const assertClientConfig = async (cli, configPath, original) => {
const config = json(await readFile(configPath, "utf8"));
if (
config.existing !== true ||
@ -422,9 +423,7 @@ const assertClientConfig = async (cli, configPath) => {
JSON.stringify(config.mcpServers?.rea?.args) !== JSON.stringify(["mcp"])
)
throw new Error("packaged client readback failed");
if (
!(await readFile(`${configPath}.rea.backup`, "utf8")).includes("existing")
)
if (!(await readFile(`${configPath}.rea.backup`)).equals(original))
throw new Error("packaged client backup failed");
};
@ -467,9 +466,7 @@ const assertCodexSymlink = async ({
!codex.includes(`command = "${cli}"`) ||
!codex.includes("startup_timeout_sec = 30") ||
!codex.startsWith(CODEX_ORIGINAL) ||
(await readFile(`${codexConfig}.rea.backup`, "utf8")) !== CODEX_ORIGINAL ||
(await readFile(`${cursorConfig}.rea.backup`, "utf8")) !==
'{"existing":true}\n'
(await readFile(`${codexConfig}.rea.backup`, "utf8")) !== CODEX_ORIGINAL
)
throw new Error("packaged setup did not preserve config symlinks");
};
@ -534,9 +531,11 @@ const verifyConfigReadback = async ({
siblingSkillPath,
opencodeConfig,
opencodeOriginal,
claudeOriginal,
cursorOriginal,
}) => {
await assertClientConfig(cli, claudeConfig);
await assertClientConfig(cli, cursorConfig);
await assertClientConfig(cli, claudeConfig, claudeOriginal);
await assertClientConfig(cli, cursorConfig, cursorOriginal);
await assertCodexSymlink({
cli,
codexConfig,
@ -729,6 +728,9 @@ export async function verifyPackageSetup({
opencodeConfig,
skillPath,
];
const [claudeOriginal, cursorOriginal] = supportedSetupHost
? await Promise.all([readFile(claudeConfig), readFile(cursorConfig)])
: [];
await verifyInteractiveSetup({
command: packageRunnerCli,
environment,
@ -772,6 +774,8 @@ export async function verifyPackageSetup({
siblingSkillPath,
opencodeConfig,
opencodeOriginal: OPENCODE_ORIGINAL,
claudeOriginal,
cursorOriginal,
});
await verifySetupFailureRecovery({
cli,

View File

@ -6,6 +6,7 @@ import {
requireMcpOperationResult,
} from "./lib/mcp-verifier-results.mjs";
import { verifyGhidraCompactReferences } from "./lib/real-ghidra-compact-references.mjs";
import { verifyWindowsGhidraByteSearch } from "./lib/ghidra-windows-byte-search.mjs";
import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import {
@ -358,11 +359,12 @@ try {
const mapping = await call("address_to_file_offset", {
address: procedure.address,
});
const sourceBytes = await readFile(target);
assert.equal(memory.complete, true);
assert.equal(memory.returned_bytes, 16);
assert.equal(
memory.bytes_hex,
(await readFile(target))
sourceBytes
.subarray(mapping.file_offset, mapping.file_offset + 16)
.toString("hex"),
);
@ -374,6 +376,35 @@ try {
await call("resolve_containing_procedure", { address: procedure.address });
await call("search_procedures", { pattern: procedure.value });
await call("search_strings", { pattern: "rea" });
report.byteSearch = await verifyWindowsGhidraByteSearch({
invoke: call,
sourceBytes,
architecture: parsedTarget.value.architecture,
sourceOwnedFixture: targetArgument === undefined,
fallbackAddress: procedure.address,
fallbackFileOffset: mapping.file_offset,
runCliPage: async (query) => {
const cliArguments = [entry, "search-bytes", target, query.pattern];
for (const [option, value] of [
["--mask", query.mask],
["--start", query.start],
["--end", query.end],
["--limit", String(query.limit)],
]) {
if (value !== undefined) cliArguments.push(option, value);
}
cliArguments.push("--provider", "ghidra", "--json");
const result = await exec(process.execPath, cliArguments, {
env: environment,
cwd: callerDirectory,
timeout: 360_000,
maxBuffer: 8 * 1024 * 1024,
});
const evidence = JSON.parse(result.stdout);
assert.ok(evidence.error === undefined, "CLI byte search failed");
return evidence.normalized_result;
},
});
for (const name of [
"procedure_assembly",
"procedure_callees",
@ -429,9 +460,24 @@ try {
name !== "shutdown" &&
!GHIDRA_MUTATING_OPERATIONS.has(name),
);
assert.ok(
expected.every((name) => report.mcpOperations.includes(name)),
"Packaged verification omitted an admitted operation.",
const expectedOperations = new Set([
...expected,
"open_binary",
"binary_overview",
"binary_session",
"close_binary",
]);
const observedOperations = new Set(report.mcpOperations);
const missingOperations = expected.filter(
(name) => !observedOperations.has(name),
);
const unexpectedOperations = [...observedOperations].filter(
(name) => !expectedOperations.has(name),
);
assert.deepEqual(
{ missing: missingOperations, unexpected: unexpectedOperations },
{ missing: [], unexpected: [] },
"Packaged verification operation coverage drifted.",
);
const session = await call("binary_session", {});
assert.equal(

View File

@ -50,6 +50,7 @@ import { verifyBrowserCaptureMetadataBudget } from "./lib/browser-capture-metada
import { artifactCliEvidence, artifactMcpResult } from "./lib/artifact-e2e.mjs";
import { verifyScenarioEnvironment } from "./lib/browser-scenario-environment-e2e.mjs";
import { verifyScenarioStorage } from "./lib/browser-scenario-storage-e2e.mjs";
import { verifyScenarioServiceWorkers } from "./lib/browser-scenario-service-workers-e2e.mjs";
const SCENARIO_SECRET_VALUE = "rea-browser-verifier-secret";
const SCENARIO_URL_SECRET_VALUE = "rea-browser-url-verifier-secret";
@ -299,6 +300,11 @@ try {
endpoint,
targetId: target,
});
process.stderr.write("Browser verifier: scenario service workers\n");
const scenarioServiceWorkers = await verifyScenarioServiceWorkers(
endpoint,
target,
);
const launchedScenario = await createBrowserScenarioProvider(
process.env,
).captureScenario(
@ -400,6 +406,7 @@ try {
scenarioFailure,
scenarioEnvironment,
scenarioStorage,
scenarioServiceWorkers,
verified: true,
};
} finally {

View File

@ -1,7 +1,14 @@
#!/usr/bin/env node
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import { copyFile, mkdir, mkdtemp, rm } from "node:fs/promises";
import {
copyFile,
mkdir,
mkdtemp,
readFile,
readdir,
rm,
} from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
@ -9,6 +16,11 @@ import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { inspectGhidraInstallation } from "../dist/ghidra/GhidraInstallation.js";
import { cleanupOwnedProcessGroup } from "../dist/process/ProcessOwnership.js";
import {
ProviderProcessSupervisor,
spawnOwnedProviderProcess,
} from "../dist/process/ProviderProcess.js";
import { toolContract } from "../dist/contracts/toolContracts.js";
import { requireMcpOperationResult } from "./lib/mcp-verifier-results.mjs";
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
@ -48,10 +60,21 @@ const source = fileURLToPath(
const literalSource = fileURLToPath(
new URL("../tests/conformance/swift/string-literals.swift", import.meta.url),
);
const unresolvedJumpSource = fileURLToPath(
new URL(
"../tests/conformance/swift/unresolved-string-jump.s",
import.meta.url,
),
);
const binary = join(root, "swift-inventory");
const literalBinary = join(root, "swift-literals");
const literalAssembly = fileURLToPath(
new URL("../tests/conformance/swift/decoder-safeguards.S", import.meta.url),
);
const literalObject = join(root, "decoder-safeguards.o");
const reports = [];
let literalReport;
let decoderCleaned = true;
try {
await execute("swiftc", ["-O", "-g", source, "-o", binary], {
timeout: 60_000,
@ -59,6 +82,19 @@ try {
await mkdir(join(root, "copied"));
const copied = join(root, "copied", "swift-inventory");
await copyFile(binary, copied);
await execute(
"xcrun",
[
"clang",
"-target",
"arm64-apple-macosx13.0",
"-c",
literalAssembly,
"-o",
literalObject,
],
{ timeout: 60_000 },
);
// Decoding covers AArch64 register pairs, so build that slice on any Mac host.
await execute(
"swiftc",
@ -67,11 +103,14 @@ try {
"-target",
"arm64-apple-macosx13.0",
literalSource,
literalObject,
unresolvedJumpSource,
"-o",
literalBinary,
],
{ timeout: 60_000 },
);
await verifyDecoderSafeguards();
await client.connect(transport);
for (const [variant, path] of [
["debug", binary],
@ -154,7 +193,7 @@ try {
} finally {
await client.close();
await transport.close();
await rm(root, { recursive: true, force: true });
if (decoderCleaned) await rm(root, { recursive: true, force: true });
}
console.log(
JSON.stringify(
@ -222,8 +261,144 @@ async function verifyStringLiterals() {
}
decoded[name] = observed.length;
}
const unresolvedJump = procedures.find(({ value }) =>
value.includes("rea_unresolved_string_jump"),
);
assert(
unresolvedJump,
"unresolved jump fixture is missing from the procedure inventory",
);
const unresolvedDossier = await call("analyze_function", {
procedure: unresolvedJump.address,
});
assert.deepEqual(unresolvedDossier.swift_string_literals, []);
assert.deepEqual(
unresolvedDossier.swift_string_literal_omissions.map(
({ reason }) => reason,
),
["unresolved_jump"],
);
decoded.unresolvedStringJump =
unresolvedDossier.swift_string_literal_omissions.length;
return decoded;
} finally {
await call("close_binary", {});
}
}
// Exercise the Java producer inside an ephemeral Ghidra database. The explicit
// budget seam makes exhaustion deterministic without exposing a runtime option.
async function verifyDecoderSafeguards() {
const scripts = join(root, "decoder-scripts");
await mkdir(scripts);
const bridgeSources = fileURLToPath(
new URL("../bridge/ghidra/", import.meta.url),
);
for (const name of await readdir(bridgeSources)) {
if (name.endsWith(".java"))
await copyFile(join(bridgeSources, name), join(scripts, name));
}
await copyFile(
fileURLToPath(
new URL(
"../tests/conformance/swift/ReaSwiftDecoderRegressions.java",
import.meta.url,
),
),
join(scripts, "ReaSwiftDecoderRegressions.java"),
);
const report = join(root, "decoder-safeguards.txt");
await runDecoderProbe(scripts, report);
assert.equal(
await readFile(report, "utf8"),
"REA_SWIFT_DECODER_SAFEGUARDS_OK\n",
);
}
async function runDecoderProbe(scripts, report) {
const controller = new AbortController();
let interrupt;
const interrupted = new Promise((resolve) => {
interrupt = () => {
controller.abort(new Error("Swift decoder probe interrupted"));
resolve(false);
};
});
process.on("SIGINT", interrupt);
process.on("SIGTERM", interrupt);
let supervisor;
let launch;
try {
launch = await spawnOwnedProviderProcess({
command: installation.analyzeHeadlessPath,
arguments: [
root,
"decoder-regressions",
"-import",
literalBinary,
"-scriptPath",
scripts,
"-postScript",
"ReaSwiftDecoderRegressions.java",
report,
"-deleteProject",
"-max-cpu",
"2",
],
runId: run.run_id,
expectedCommand: null,
env: { JAVA_HOME: installation.javaHome },
signal: controller.signal,
});
decoderCleaned = false;
supervisor = new ProviderProcessSupervisor(
{
...launch,
ownsProcessLifetime: true,
cleanup: () => cleanupOwnedProcessGroup(launch.ownership),
},
{ maxDiagnosticBytes: 8 * 1024 * 1024 },
);
assert.ok(
await Promise.race([supervisor.waitForExit(120_000), interrupted]),
controller.signal.aborted
? "Swift decoder probe interrupted"
: "Swift decoder probe timed out",
);
assert.ok(
await supervisor.waitForOutputClose(1_000),
"Swift decoder probe output did not close",
);
const snapshot = supervisor.snapshot();
assert.equal(
snapshot.exitCode,
0,
`${snapshot.stdout.text}\n${snapshot.stderr.text}`,
);
assert.ok(
!snapshot.diagnosticTruncated,
"Swift decoder probe exceeded its diagnostic capacity",
);
} finally {
try {
if (supervisor !== undefined) {
const stopped = await supervisor.stop();
decoderCleaned = stopped.status === "verified-cleanup";
assert.ok(
decoderCleaned,
`Owned Ghidra cleanup incomplete; retained ${root}: ${JSON.stringify(stopped)}`,
);
} else if (launch !== undefined) {
const stopped = await cleanupOwnedProcessGroup(launch.ownership);
decoderCleaned = stopped.cleaned;
assert.ok(
decoderCleaned,
`Owned Ghidra cleanup incomplete; retained ${root}: ${JSON.stringify(stopped)}`,
);
}
} finally {
process.off("SIGINT", interrupt);
process.off("SIGTERM", interrupt);
}
}
}

View File

@ -29,6 +29,7 @@ import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
import { verifyGhidraCompactReferences } from "./lib/real-ghidra-compact-references.mjs";
import { verifyGhidraFileOffsets } from "./lib/real-ghidra-file-offsets.mjs";
import { verifyWindowsGhidraByteSearch } from "./lib/ghidra-windows-byte-search.mjs";
const verifierRun = createVerifierRun();
const SESSION_CAPABILITIES = GHIDRA_SESSION_CAPABILITIES.filter(
@ -215,6 +216,11 @@ try {
});
await inventory("search_procedures", { pattern: procedure.value });
await inventory("search_strings", { pattern: "rea" });
const byteSearch = await verifyWindowsGhidraByteSearch({
invoke: inventory,
sourceBytes,
architecture,
});
for (const operation of [
"procedure_assembly",
@ -242,12 +248,18 @@ try {
...GHIDRA_INVENTORY_OPERATIONS,
...GHIDRA_FUNCTION_OPERATIONS,
].filter((name) => !GHIDRA_MUTATING_OPERATIONS.has(name));
const expectedOperations = new Set(expected);
const missingOperations = expected.filter((name) => !observed.has(name));
const unexpectedOperations = [...observed].filter(
(name) => !expectedOperations.has(name),
);
if (
observed.size !== expected.length ||
expected.some((operation) => !observed.has(operation))
missingOperations.length > 0 ||
unexpectedOperations.length > 0 ||
observed.size !== expectedOperations.size
)
throw new Error(
`Windows Ghidra operation proof was incomplete: ${JSON.stringify([...observed])}`,
`Windows Ghidra operation proof was incomplete: ${JSON.stringify({ missing: missingOperations, unexpected: unexpectedOperations })}`,
);
// The reference controls occupy the EXE fixture's additional function.
@ -271,6 +283,7 @@ try {
},
transport: "authenticated-loopback-tcp",
operations: [...observed].sort((left, right) => left.localeCompare(right)),
byte_search: byteSearch,
cleanup: "complete",
compact_references: compactReferences,
native_controls: {

View File

@ -1,11 +1,12 @@
import assert from "node:assert/strict";
import { execFileSync } from "node:child_process";
import { spawnSync } from "node:child_process";
import {
access,
copyFile,
mkdir,
mkdtemp,
readFile,
readdir,
rmdir,
rm,
stat,
symlink,
@ -54,7 +55,7 @@ try {
join(workspace, "dist", "windows", "WindowsNativeLoader.js"),
).href;
const source = `import assert from 'node:assert/strict';
import {existsSync,mkdirSync,readdirSync,renameSync,statSync} from 'node:fs';
import {existsSync,mkdirSync,readFileSync,readdirSync,renameSync,statSync,writeFileSync} from 'node:fs';
import {join} from 'node:path';
import {systemWindowsNativeAuthority} from ${JSON.stringify(loader)};
const temp=process.env.TEMP;
@ -64,46 +65,61 @@ assert.equal(first.retryable,true);
assert.match(first.reason,/Delete owned object by handle failed.*Win32 5/s);
const [name,...extra]=readdirSync(temp);assert.ok(name);assert.equal(extra.length,0);
const path=join(temp,name);const identity=statSync(path,{bigint:true});
// Ownership follows the retained NTFS handle even when the directory moves.
// A replacement at its old pathname must remain caller-owned.
const moved=path+'-renamed';renameSync(path,moved);
assert.equal(statSync(moved,{bigint:true}).ino,identity.ino);
// Delete sharing permits rename; ownership follows the retained NTFS handle.
writeFileSync(join(path,'owned-child.txt'),'owned child');
const renamed=path+'-renamed';renameSync(path,renamed);
const moved=statSync(renamed,{bigint:true});
assert.equal(moved.dev,identity.dev);assert.equal(moved.ino,identity.ino);
mkdirSync(path);const replacement=statSync(path,{bigint:true});
assert.notEqual(replacement.ino,identity.ino);
const sentinel=join(path,'unrelated.txt');writeFileSync(sentinel,'unrelated replacement');
if(process.argv[2]!=='teardown'){
const second=systemWindowsNativeAuthority();assert.equal(second.available,true);
assert.equal(existsSync(moved),false);
assert.equal(existsSync(renamed),false);
assert.deepEqual(readdirSync(temp),[name]);
assert.equal(readFileSync(sentinel,'utf8'),'unrelated replacement');
assert.equal(statSync(path,{bigint:true}).ino,replacement.ino);
assert.equal(systemWindowsNativeAuthority(),second);
}
console.log(JSON.stringify({mode:process.argv[2],firstFailed:true,retainedOwner:true,inode:identity.ino.toString(),replacementName:name,replacementInode:replacement.ino.toString()}));
console.log(JSON.stringify({mode:process.argv[2],firstFailed:true,retainedOwner:true,name,renamed,device:identity.dev.toString(),inode:identity.ino.toString(),replacementInode:replacement.ino.toString()}));
`;
const fixture = join(workspace, "probe.mjs");
await writeFile(fixture, source);
for (const mode of ["retry", "teardown"]) {
const temp = join(workspace, mode);
await mkdir(temp);
const observed = JSON.parse(
execFileSync(process.execPath, [fixture, mode], {
encoding: "utf8",
timeout: 20_000,
windowsHide: true,
env: { ...process.env, TEMP: temp, TMP: temp },
}),
);
assert.equal(observed.retainedOwner, true);
assert.deepEqual(await readdir(temp), [observed.replacementName]);
const replacement = join(temp, observed.replacementName);
const child = spawnSync(process.execPath, [fixture, mode], {
encoding: "utf8",
timeout: 20_000,
windowsHide: true,
env: { ...process.env, TEMP: temp, TMP: temp },
});
if (child.error !== undefined) throw child.error;
assert.equal(child.status, 0, child.stderr);
assert.equal(
(await stat(replacement, { bigint: true })).ino.toString(),
observed.replacementInode,
child.stderr,
"",
"Native environment teardown reported a cleanup failure",
);
const observed = JSON.parse(child.stdout);
assert.equal(observed.mode, mode);
assert.equal(observed.firstFailed, true);
assert.equal(observed.retainedOwner, true);
await assert.rejects(access(observed.renamed), { code: "ENOENT" });
assert.deepEqual(await readdir(temp), [observed.name]);
const replacement = join(temp, observed.name);
const identity = await stat(replacement, { bigint: true });
assert.equal(identity.dev.toString(), observed.device);
assert.equal(identity.ino.toString(), observed.replacementInode);
assert.notEqual(identity.ino.toString(), observed.inode);
assert.deepEqual(await readdir(replacement), ["unrelated.txt"]);
assert.equal(
await readFile(join(replacement, "unrelated.txt"), "utf8"),
"unrelated replacement",
);
await rmdir(replacement);
assert.deepEqual(await readdir(temp), []);
}
process.stdout.write(
`${JSON.stringify({ ok: true, controlledDispositionFailure: true, retainedOwner: true, renamedOwnerCleanup: true, replacementPreserved: true, retryRecovers: true, successfulAdmissionCached: true, environmentTeardownCleansProbe: true, issue1701HostCauseReproduced: false })}\n`,
`${JSON.stringify({ ok: true, controlledDispositionFailure: true, retainedOwner: true, renamedOwnerCleansChildren: true, unrelatedReplacementPreserved: true, retryRecovers: true, successfulAdmissionCached: true, environmentTeardownCleansProbe: true, issue1701HostCauseReproduced: false })}\n`,
);
} finally {
await rm(workspace, { recursive: true, force: true });

View File

@ -0,0 +1,21 @@
export interface OpenAiModelFixture {
readonly baseUrl: string;
readonly requests: readonly { readonly path: string }[];
readonly probes: readonly {
readonly path: string | undefined;
readonly status: 404;
}[];
readonly failure: string | undefined;
close(): Promise<void>;
}
export function createOpenAiModelFixture(options: {
readonly directory: string;
readonly prefix: string;
readonly onRequest: () =>
| {
readonly role: "assistant";
readonly content: string;
}
| Promise<{ readonly role: "assistant"; readonly content: string }>;
}): Promise<OpenAiModelFixture>;

View File

@ -15,7 +15,8 @@ export const createOpenAiModelFixture = async ({
let failure;
const server = createServer(async (request, response) => {
try {
if (request.method === "GET" && request.url?.endsWith("/models")) {
const path = new URL(request.url ?? "/", "http://localhost").pathname;
if (request.method === "GET" && path.endsWith("/models")) {
response.setHeader("Content-Type", "application/json");
response.end(
JSON.stringify({
@ -47,7 +48,7 @@ export const createOpenAiModelFixture = async ({
join(directory, `${prefix}-incoming-${++incoming}.json`),
JSON.stringify({ method: request.method, path: request.url, body }),
);
if (request.url !== "/v1/chat/completions") {
if (path !== "/v1/chat/completions") {
// Hermes probes custom endpoints for Ollama/llama.cpp metadata.
// Unsupported HTTP endpoints return 404, as on an OpenAI-only server.
probes.push({

View File

@ -201,6 +201,16 @@ async function stageCli() {
),
);
assert.ok(
files.extraction_gaps.some(
(gap) =>
gap.kind === "unknown-chunk-handler" &&
gap.range !== null &&
gap.range.offset === 0 &&
gap.range.length === 64,
),
"The unknown chunk must carry its explicit recovery row",
);
return { regions, files };
}
@ -247,6 +257,20 @@ async function stageMcp(cli) {
);
assert.equal(depthParsed.coverage, "partial");
assert.ok(depthParsed.depth_limited_paths.length > 0);
assert.ok(
depthParsed.extraction_gaps.some(
(gap) =>
gap.kind === "depth-limited" &&
gap.dependencies.length === 0 &&
depthParsed.extraction_continuations.some(
({ id, instruction }) =>
id === gap.continuation_id &&
instruction.includes("max_depth") &&
instruction.includes("fresh output_directory"),
),
),
"Depth-limited paths must carry the explicit depth continuation",
);
console.log(
"PASS real Binwalk/Unblob: CLI, MCP, independent offsets/digests, range extraction, unknown chunks and depth limits",
@ -292,8 +316,17 @@ async function stageExt4() {
);
assert.equal(degradedParsed.coverage, "partial");
assert.ok(JSON.stringify(degradedParsed.diagnostics).includes("debugfs"));
assert.ok(
degradedParsed.extraction_gaps.some(
(gap) =>
gap.kind === "missing-extractor-dependency" &&
gap.dependencies.includes("debugfs") &&
gap.handler === "extfs",
),
"The missing extractor must be named in its explicit recovery row",
);
console.log(
"PASS real missing-extractor diagnostics retained despite Unblob exit 1",
"PASS real missing-extractor diagnostics and recovery row retained despite Unblob exit 1",
);
}

View File

@ -192,6 +192,25 @@ export const parsePackageListOutput = (
return { packages, unparsedLines };
};
/** Quote raw device paths for prose without allowing bidi or invisible text. */
export const displayDevicePath = (devicePath: string): string => {
let excerpt = "";
let count = 0;
for (const character of devicePath) {
if (count === 160) break;
excerpt += character;
count += 1;
}
const quoted = JSON.stringify(excerpt).replace(
/[\p{C}\p{Z}\p{Default_Ignorable_Code_Point}\u2800]/gu,
(character) =>
character === " "
? character
: `\\u{${character.codePointAt(0)!.toString(16).toUpperCase()}}`,
);
return excerpt.length === devicePath.length ? quoted : `${quoted} (excerpt)`;
};
/** One APK of an installed package exactly as `pm path` reports it. */
export interface AdbPackagePathEntry {
readonly device_path: string;

View File

@ -240,11 +240,12 @@ const shellStub = (
): ((root: string) => string) => {
const cases = Object.entries(dispatch)
.map(([key, output]) => {
const [first, second] = key.split(":");
const guard =
second === undefined
? `[ "$4" = "${first}" ]`
: `[ "$4" = "${first}" ] && [ "$5" = "${second}" ]`;
const [first, second, third] = key.split(":");
const guard = [
`[ "$4" = "${first}" ]`,
...(second === undefined ? [] : [`[ "$5" = "${second}" ]`]),
...(third === undefined ? [] : [`[ "$6" = "${third}" ]`]),
].join(" && ");
return ` if ${guard}; then printf '%b' '${output.replaceAll("\\", "\\\\").replaceAll("\0", "\\000").replace(/'/gu, "'\\''")}'; exit 0; fi`;
})
.join("\n");
@ -379,6 +380,30 @@ exit 1`,
expect(result.error.capturedOutput?.stdout).toContain(
pmOutput.split("\n")[0],
);
const projected = projectAnalysisError(result.error);
const duplicate = pmOutput.includes("\n");
expect(projected).toMatchObject({
code: "unreadable_output",
message: expect.stringContaining("pm path for com.example"),
remediation: { action: expect.stringContaining("details.diagnostics") },
details: {
reason: duplicate ? "duplicate_local_destination" : "unsafe_device_path",
diagnostics: {
package_name: "com.example",
...(duplicate
? { device_paths: ["/data/app/x/base.apk", "/data/app/y/base.apk"] }
: {
rejected_paths: [
{
device_path: "/data/app/x/",
reason: "empty_file_name",
},
],
}),
},
captured_output: { stdout: expect.stringContaining("package:") },
},
});
await expect(readFile(join(root, "pull-ran"))).rejects.toThrow();
await expect(lstat(output)).rejects.toMatchObject({ code: "ENOENT" });
});
@ -410,7 +435,15 @@ it.each([
if (aliases) {
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error.message).toContain("unique local destinations");
expect(projectAnalysisError(result.error)).toMatchObject({
message: expect.stringContaining("unique local destination"),
details: {
reason: "local_destination_unavailable",
diagnostics: {
device_path: `/data/app/y/${secondName}`,
},
},
});
await expect(readFile(pullLog)).rejects.toThrow();
await expect(lstat(packageDirectory)).rejects.toThrow();
return;
@ -1015,6 +1048,60 @@ it("inspects the display and window focus", async () => {
);
});
it("reads Android 16 focus declarations from displays when windows omits them", async () => {
const { binary } = await writeAdbStub(
shellStub({
"dumpsys:window:windows": "Window state has no focus fields\\n",
"dumpsys:window:displays":
"mCurrentFocus=Window{abc u0 com.example/.Main}\nmFocusedApp=ActivityRecord{def u0 com.example/.Main t1}\n",
}),
);
const result = await evidenceResult(providerOver(binary), {
operation: "inspect_adb_window",
input: { serial: "emulator-5554" },
});
expect(result).toMatchObject({
focused_window: "Window{abc u0 com.example/.Main}",
focused_app: "ActivityRecord{def u0 com.example/.Main t1}",
observed_lines: [
"Window{abc u0 com.example/.Main}",
"ActivityRecord{def u0 com.example/.Main t1}",
],
});
const execution = await providerOver(binary).execute({
operation: "inspect_adb_window",
input: { serial: "emulator-5554" },
});
expect(execution.ok).toBe(true);
if (!execution.ok) throw execution.error;
expect(execution.value.limitations).toContainEqual(
expect.stringContaining("dumpsys window displays"),
);
});
it("describes missing window focus as unknown after both sections omit declarations", async () => {
const { binary } = await writeAdbStub(
shellStub({
"dumpsys:window:windows": "no focus\\n",
"dumpsys:window:displays": "no focus\\n",
}),
);
const result = await evidenceResult(providerOver(binary), {
operation: "inspect_adb_window",
input: { serial: "emulator-5554" },
});
expect(result).toMatchObject({ focused_window: null, focused_app: null });
const execution = await providerOver(binary).execute({
operation: "inspect_adb_window",
input: { serial: "emulator-5554" },
});
expect(execution.ok).toBe(true);
if (!execution.ok) throw execution.error;
expect(execution.value.limitations).toContainEqual(
expect.stringContaining("focus is unknown"),
);
});
it("reads one settings value and reports device-side nulls", async () => {
const { binary } = await writeAdbStub(shellStub({ "settings:get": "1\n" }));
const enabled = await evidenceResult(providerOver(binary), {

View File

@ -56,6 +56,7 @@ import {
type AdbClientIdentity,
} from "./AdbConfiguration.js";
import {
displayDevicePath,
parseAdbDevicesOutput,
parseGetpropOutput,
parsePackageListOutput,
@ -417,8 +418,12 @@ const removeOwnedOutputDirectory = async (
const reservePackageDestinations = async (
operation: string,
packageName: string,
packageDirectory: string,
entries: readonly { readonly file_name: string }[],
entries: readonly {
readonly file_name: string;
readonly device_path: string;
}[],
capturedOutput: {
readonly stdout: string;
readonly stderr: string;
@ -427,6 +432,8 @@ const reservePackageDestinations = async (
): Promise<AnalysisOutputError | undefined> => {
const reservations: {
readonly path: string;
readonly file_name: string;
readonly device_path: string;
readonly identity: OutputDirectoryIdentity;
}[] = [];
for (const entry of entries) {
@ -438,6 +445,8 @@ const reservePackageDestinations = async (
throw new Error(`Could not reserve local filename ${entry.file_name}`);
reservations.push({
path,
file_name: entry.file_name,
device_path: entry.device_path,
identity: {
dev: created.dev,
ino: created.ino,
@ -445,10 +454,28 @@ const reservePackageDestinations = async (
},
});
} catch (cause: unknown) {
const errorCode =
cause instanceof Error &&
"code" in cause &&
typeof cause.code === "string"
? cause.code
: null;
return new AnalysisOutputError(
operation,
`Package Manager paths cannot be written to unique local destinations at ${JSON.stringify(entry.file_name)}; refusing before pulling any files`,
{ cause, capturedOutput },
"local_destination_unavailable",
{
cause,
capturedOutput,
userMessage: `pm path for ${packageName} reported ${displayDevicePath(entry.device_path)}, whose file name ${displayDevicePath(entry.file_name)} cannot be written to a unique local destination (${errorCode ?? "unknown error"}). No APK was pulled.`,
remediationAction:
"Review details.diagnostics and details.captured_output for the refused destination. Correct the output filesystem conflict before retrying; REA does not rename APKs.",
diagnostics: {
package_name: packageName,
device_path: entry.device_path,
file_name: entry.file_name,
error_code: errorCode,
},
},
);
}
}
@ -467,10 +494,28 @@ const reservePackageDestinations = async (
);
await rmdir(reservation.path);
} catch (cause: unknown) {
const errorCode =
cause instanceof Error &&
"code" in cause &&
typeof cause.code === "string"
? cause.code
: null;
return new AnalysisOutputError(
operation,
`Could not release a local filename reservation before pulling: ${reservation.path}`,
{ cause, capturedOutput },
"local_destination_unavailable",
{
cause,
capturedOutput,
userMessage: `Could not release the local filename reservation for ${displayDevicePath(reservation.device_path)} (${errorCode ?? "unknown error"}); no APK was pulled.`,
remediationAction:
"Review details.diagnostics and details.captured_output for the refused destination. Correct the output filesystem conflict before retrying; REA does not rename APKs.",
diagnostics: {
package_name: packageName,
device_path: reservation.device_path,
file_name: reservation.file_name,
error_code: errorCode,
},
},
);
}
}
@ -1095,14 +1140,27 @@ export class AdbProvider {
);
}
case "inspect_adb_window": {
const ran = await shell(["dumpsys", "window", "windows"], {
let section = "windows";
const ran = await shell(["dumpsys", "window", section], {
timeoutMs: SHELL_TIMEOUT_MS,
maxOutputBytes: DUMPSYS_WINDOW_MAX_OUTPUT_BYTES,
});
if (!ran.ok) return ran;
const { focusedWindow, focusedApp } = parseWindowFocusOutput(
let { focusedWindow, focusedApp } = parseWindowFocusOutput(
ran.value.stdout,
);
if (focusedWindow === null && focusedApp === null) {
section = "displays";
const fallback = await shell(["dumpsys", "window", section], {
timeoutMs: SHELL_TIMEOUT_MS,
maxOutputBytes: DUMPSYS_WINDOW_MAX_OUTPUT_BYTES,
});
if (!fallback.ok) return fallback;
({ focusedWindow, focusedApp } = parseWindowFocusOutput(
fallback.value.stdout,
));
}
const declared = focusedWindow !== null || focusedApp !== null;
return ok(
createAnalysisExecution(
{
@ -1119,6 +1177,9 @@ export class AdbProvider {
{
limitations: [
"Focus values are the window manager's declarations at read time; no interaction occurred",
declared
? `Focus declarations were read from \`dumpsys window ${section}\``
: "Neither `dumpsys window windows` nor `dumpsys window displays` declared mCurrentFocus or mFocusedApp; focus is unknown, not absent",
],
},
),
@ -1269,14 +1330,27 @@ export class AdbProvider {
stderr: paths.value.stderr,
truncated: false,
};
if (parsed.rejectedPaths.length > 0)
if (parsed.rejectedPaths.length > 0) {
const first = parsed.rejectedPaths[0]!;
const otherCount = parsed.rejectedPaths.length - 1;
return err(
new AnalysisOutputError(
operation,
`Package Manager reported paths that cannot be represented as host files: ${parsed.rejectedPaths.map((entry) => `${JSON.stringify(entry.device_path)} (${entry.reason})`).join(", ")}`,
{ capturedOutput },
),
new AnalysisOutputError(operation, "unsafe_device_path", {
capturedOutput,
userMessage: `pm path for ${packageName} reported an APK path that cannot be written as a file on this host: ${displayDevicePath(first.device_path)} (${first.reason})${otherCount > 0 ? ` and ${String(otherCount)} other path(s)` : ""}. No APK was pulled.`,
remediationAction:
"Review details.diagnostics for the refused device paths and details.captured_output for the retained pm path output. REA does not rename device-reported APK paths; correct the device path policy before retrying.",
diagnostics: {
package_name: packageName,
rejected_paths: parsed.rejectedPaths.map(
({ device_path, reason }) => ({
device_path,
reason,
}),
),
},
}),
);
}
if (parsed.paths.length === 0 && parsed.unparsedLines.length === 0)
return err(
new AnalysisUnsupportedTargetError(
@ -1296,11 +1370,19 @@ export class AdbProvider {
for (const entry of parsed.paths) {
if (destinationNames.has(entry.file_name))
return err(
new AnalysisOutputError(
operation,
`Package Manager reported multiple paths with the same local destination ${JSON.stringify(entry.file_name)}; refusing before pulling any files`,
{ capturedOutput },
),
new AnalysisOutputError(operation, "duplicate_local_destination", {
capturedOutput,
userMessage: `pm path for ${packageName} reported multiple APK paths with the same local file name ${displayDevicePath(entry.file_name)}; no APK was pulled.`,
remediationAction:
"Review details.diagnostics and details.captured_output for the conflicting device paths. REA does not rename device-reported paths; correct Package Manager's path response before retrying.",
diagnostics: {
package_name: packageName,
file_name: entry.file_name,
device_paths: parsed.paths
.filter((candidate) => candidate.file_name === entry.file_name)
.map((candidate) => candidate.device_path),
},
}),
);
destinationNames.add(entry.file_name);
}
@ -1395,6 +1477,7 @@ export class AdbProvider {
};
const destinationError = await reservePackageDestinations(
operation,
packageName,
packageDirectory,
parsed.paths,
capturedOutput,

View File

@ -172,7 +172,7 @@ export const parseWmDensityOutput = (
return { physical, override };
};
/** Parse the focus declarations from `dumpsys window windows`. */
/** Parse focus declarations from `dumpsys window windows` or `displays`. */
export const parseWindowFocusOutput = (
stdout: string,
): {

View File

@ -1,6 +1,7 @@
import { constants } from "node:fs";
import { access, realpath, stat } from "node:fs/promises";
import { delimiter, isAbsolute, join, resolve } from "node:path";
import { inspectWindowsNativeReadiness } from "../process/WindowsAuthority.js";
import { systemWindowsNativeAuthority } from "../windows/WindowsNativeLoader.js";
import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore.js";
import type { AndroidOperation } from "../domain/android/androidAnalysis.js";
@ -11,7 +12,15 @@ import {
execFileOutput,
execFileOutputFailure,
} from "../process/ExecFileOutput.js";
import { JADX_JAR_CONFIGURATION_REMEDIATION } from "./JadxRelease.js";
import type { DoctorProviderCheck } from "../application/Doctor.js";
import {
JADX_JAR_CONFIGURATION_REMEDIATION,
JADX_JAVA_PROBE_REMEDIATION,
JADX_JAVA_RUNTIME_REMEDIATION,
JADX_JVM_OPTIONS_REMEDIATION,
JADX_UNSUPPORTED_HOST_REMEDIATION,
JADX_WINDOWS_NATIVE_REMEDIATION,
} from "./JadxRelease.js";
const JAVA_MODULE_LISTING_TIMEOUT_MS = 10_000;
const JAVA_MODULE_LISTING_MAX_OUTPUT_BYTES = 1024 * 1024;
@ -30,16 +39,110 @@ type ConfigurationFailureCode =
| "not_configured"
| "executable_missing"
| "runtime_missing"
| "runtime_unavailable"
| "unsupported_host"
| "unsupported_version"
| "version_unresolved"
| "open_options_invalid";
/**
* Failure kinds tagged at the JADX configuration boundary. Consumers (the
* doctor adapter) project the boundary's remediation and classification for
* a kind; they must not reinterpret the phase/setting evidence to derive
* them.
*/
export type JadxFailureKind =
| "windows_native_artifact"
| "windows_native_runtime"
| "java_runtime"
| "java_home"
| "jvm_options"
| "jar"
| "java_probe"
| "unsupported_host";
/** Runtime mirror of JadxFailureKind for validating the loose diagnostics tag. */
export const JADX_FAILURE_KINDS: readonly JadxFailureKind[] = [
"windows_native_artifact",
"windows_native_runtime",
"java_runtime",
"java_home",
"jvm_options",
"jar",
"java_probe",
"unsupported_host",
];
export interface JadxFailureOutcome {
readonly remediation: string;
readonly classification: DoctorProviderCheck["classification"];
}
/**
* Project the boundary-owned remediation and classification for a failure
* kind. A null kind (missing or corrupt tag) keeps JAR guidance, matching
* the previous fallthrough for unclassified failures.
*/
export const projectJadxFailureOutcome = (
kind: JadxFailureKind | null,
configured: boolean,
): JadxFailureOutcome => {
const drift = configured ? "config_drift" : "missing_analysis_engine";
switch (kind) {
case "windows_native_artifact":
return {
remediation: JADX_WINDOWS_NATIVE_REMEDIATION,
classification: "missing_dependency",
};
case "windows_native_runtime":
return {
remediation: JADX_WINDOWS_NATIVE_REMEDIATION,
classification: "runtime_failure",
};
case "java_runtime":
return {
remediation: JADX_JAVA_RUNTIME_REMEDIATION,
classification: "missing_dependency",
};
case "java_home":
return {
remediation: JADX_JAVA_RUNTIME_REMEDIATION,
classification: drift,
};
case "jvm_options":
return {
remediation: JADX_JVM_OPTIONS_REMEDIATION,
classification: drift,
};
case "jar":
return {
remediation: JADX_JAR_CONFIGURATION_REMEDIATION,
classification: drift,
};
case "java_probe":
return {
remediation: JADX_JAVA_PROBE_REMEDIATION,
classification: drift,
};
case "unsupported_host":
return {
remediation: JADX_UNSUPPORTED_HOST_REMEDIATION,
classification: "unsupported_host",
};
case null:
return {
remediation: JADX_JAR_CONFIGURATION_REMEDIATION,
classification: drift,
};
}
};
class JadxConfigurationFailure extends Error {
constructor(
readonly code: ConfigurationFailureCode,
message: string,
readonly diagnostics: Readonly<Record<string, JsonValue>>,
readonly kind: JadxFailureKind,
options?: ErrorOptions,
) {
super(message, options);
@ -75,7 +178,7 @@ export const inspectJadxAvailability = async (
status: "unavailable",
code: cause.code,
reason: cause.message,
diagnostics: cause.diagnostics,
diagnostics: { ...cause.diagnostics, failure_kind: cause.kind },
};
throw cause;
}
@ -191,6 +294,7 @@ const requireEngineArchive = async (
archive_error: reason,
maximum_metadata_read_bytes: JAR_METADATA_READ_MAX_BYTES,
},
"jar",
cause,
);
} finally {
@ -200,6 +304,7 @@ const requireEngineArchive = async (
"unsupported_version",
`REA_JADX_MCP_JAR ${jar} does not contain the required metadata bridge classes: ${[...missing].join(", ")}. ${JADX_JAR_CONFIGURATION_REMEDIATION}`,
{ jar_path: jar, phase: "jar-inspection", missing_classes: [...missing] },
"jar",
);
};
@ -207,25 +312,52 @@ const requireSupportedHost = (): void => {
if (process.platform === "linux" || process.platform === "darwin") return;
if (process.platform === "win32" && process.arch === "x64") {
const loaded = systemWindowsNativeAuthority();
if (loaded.available && loaded.authority.inspection.protocolStdin === true)
return;
if (loaded.available) {
if (loaded.authority.inspection.protocolStdin === true) return;
throw configurationFailure(
"runtime_missing",
"The bundled Windows native artifact lacks owned protocol stdin; use a matching current Windows x64 REA package.",
{
platform: process.platform,
architecture: process.arch,
phase: "process-ownership",
owned_protocol_stdin: false,
},
"windows_native_artifact",
);
}
const readiness = inspectWindowsNativeReadiness(process.platform, loaded);
throw configurationFailure(
"runtime_missing",
loaded.available
? "The bundled Windows native artifact lacks owned protocol stdin; use a matching current Windows x64 REA package."
: loaded.reason,
readiness.available ? "runtime_unavailable" : readiness.code,
readiness.available
? "Windows native readiness disagreed with the native loader result."
: readiness.reason,
{
platform: process.platform,
architecture: process.arch,
phase: "process-ownership",
owned_protocol_stdin: false,
...(readiness.available
? {}
: {
failure_stage: readiness.failureStage,
remediation: readiness.remediation,
}),
},
readiness.available
? "windows_native_runtime"
: readiness.code === "unsupported_host"
? "unsupported_host"
: readiness.code === "runtime_missing"
? "windows_native_artifact"
: "windows_native_runtime",
);
}
throw configurationFailure(
"unsupported_host",
`Android JADX subprocess ownership is unsupported on ${process.platform}/${process.arch}; use Linux, macOS, or Windows x64 with its bundled native controls.`,
{ platform: process.platform },
"unsupported_host",
);
};
@ -238,6 +370,7 @@ const resolveJar = async (
"not_configured",
JADX_JAR_CONFIGURATION_REMEDIATION,
{ jar_configured: false },
"jar",
);
try {
await access(jar, constants.R_OK);
@ -246,6 +379,7 @@ const resolveJar = async (
"executable_missing",
`REA_JADX_MCP_JAR is not a regular file: ${jar}`,
{ jar_path: jar },
"jar",
);
return await realpath(jar);
} catch (cause) {
@ -254,6 +388,7 @@ const resolveJar = async (
"executable_missing",
`Cannot read REA_JADX_MCP_JAR ${jar}: ${cause instanceof Error ? cause.message : String(cause)}`,
{ jar_path: jar },
"jar",
cause,
);
}
@ -293,6 +428,7 @@ const resolveJava = async (
"runtime_missing",
`Cannot inspect selected Java executable ${candidate}: ${errorText(cause)}`,
{ java_executable: candidate },
"java_runtime",
cause,
);
}
@ -301,6 +437,7 @@ const resolveJava = async (
"runtime_missing",
"Cannot find java.exe on the selected Windows PATH. Select a full JDK with JAVA_HOME or PATH, then verify it with java --list-modules. REA does not install Java.",
{ java_executable: executable },
"java_runtime",
);
}
if (isAbsolute(environment.JAVA_HOME))
@ -309,6 +446,7 @@ const resolveJava = async (
"open_options_invalid",
"JAVA_HOME must select an existing JDK by absolute path; omit it to use java on PATH.",
{ java_home: environment.JAVA_HOME },
"java_home",
);
};
@ -324,6 +462,7 @@ const requireJavaHomeExecutable = async (
"runtime_missing",
`JAVA_HOME java is not a regular file: ${java}`,
{ java_executable: java },
"java_runtime",
);
} catch (cause) {
if (cause instanceof JadxConfigurationFailure) throw cause;
@ -331,6 +470,7 @@ const requireJavaHomeExecutable = async (
"runtime_missing",
`Cannot execute JAVA_HOME java ${java}: ${cause instanceof Error ? cause.message : String(cause)}`,
{ java_executable: java },
"java_runtime",
cause,
);
}
@ -351,6 +491,7 @@ const resolveJvmArguments = (
"open_options_invalid",
`${name} must be a positive safe integer; received ${value}.`,
{ setting: name, value },
"jvm_options",
);
if (
name === "REA_JADX_ACTIVE_PROCESSOR_COUNT" &&
@ -360,6 +501,7 @@ const resolveJvmArguments = (
"open_options_invalid",
`${name} exceeds the JVM's signed 32-bit processor-count range.`,
{ setting: name, value },
"jvm_options",
);
result.push(`${prefix}${value}${suffix}`);
}
@ -405,6 +547,7 @@ const inspectJavaRuntime = async (
failure: "output_limit",
output_limit_bytes: JAVA_MODULE_LISTING_MAX_OUTPUT_BYTES,
},
"java_probe",
cause,
);
if (output.killed)
@ -416,12 +559,14 @@ const inspectJavaRuntime = async (
failure: "probe_terminated",
deadline_ms: JAVA_MODULE_LISTING_TIMEOUT_MS,
},
"java_probe",
cause,
);
throw configurationFailure(
"runtime_missing",
`Cannot inspect selected Java runtime ${java}: ${output.stderr.trim() || errorText(cause)}. Select a full JDK including jdk.compiler via JAVA_HOME or PATH; verify it with java --list-modules.`,
diagnostics,
"java_runtime",
cause,
);
}
@ -438,6 +583,7 @@ const inspectJavaRuntime = async (
stderr,
output_truncated: false,
},
"java_runtime",
);
const major = /^java\.base@(\d+)(?:[.+_-]|$)/mu.exec(stdout)?.[1];
if (major === undefined)
@ -453,6 +599,7 @@ const inspectJavaRuntime = async (
stderr,
output_truncated: false,
},
"java_probe",
);
if (Number(major) < 17)
throw configurationFailure(
@ -467,6 +614,7 @@ const inspectJavaRuntime = async (
stderr,
output_truncated: false,
},
"java_runtime",
);
};
@ -474,12 +622,14 @@ const configurationFailure = (
code: ConfigurationFailureCode,
message: string,
diagnostics: Readonly<Record<string, JsonValue>>,
kind: JadxFailureKind,
cause?: unknown,
): JadxConfigurationFailure =>
new JadxConfigurationFailure(
code,
message,
diagnostics,
kind,
cause === undefined ? undefined : { cause },
);

155
src/android/JadxDoctor.ts Normal file
View File

@ -0,0 +1,155 @@
import type {
DoctorProviderCheck,
DoctorProviderInspection,
} from "../application/Doctor.js";
import type { ProviderAvailability } from "../application/AnalysisProvider.js";
import type { JsonValue } from "../domain/jsonValue.js";
import {
inspectJadxAvailability,
JADX_FAILURE_KINDS,
projectJadxFailureOutcome,
type JadxFailureKind,
} from "./JadxConfiguration.js";
/** Project the validated JADX configuration into the doctor contract. */
export const inspectJadxProvider = async (
environment: Readonly<Record<string, string | undefined>>,
): Promise<DoctorProviderInspection> => {
const configured = environment.REA_JADX_MCP_JAR !== undefined;
const availability = await inspectJadxAvailability(environment);
// diagnostics is a loosely-typed JsonValue map; guard the published path
// even though the success path always sets jar_path today.
const jarPath =
availability.status === "available" &&
typeof availability.diagnostics["jar_path"] === "string"
? availability.diagnostics["jar_path"]
: undefined;
const available =
availability.status === "available" && jarPath !== undefined;
// Preserve the caller-selected JAR path (it may be a stable symlink) and
// the validated JAVA_HOME selection; the resolved jarPath stays in
// diagnostics as inspection evidence.
const configuredJar = environment.REA_JADX_MCP_JAR;
return {
id: "jadx",
configured,
available,
providerVersion: null,
registrationEnvironment:
jarPath === undefined || configuredJar === undefined
? {}
: {
REA_JADX_MCP_JAR: configuredJar,
...optionalEntry("JAVA_HOME", environment.JAVA_HOME),
...optionalEntry(
"REA_JADX_HEAP_MIB",
environment.REA_JADX_HEAP_MIB,
),
...optionalEntry(
"REA_JADX_ACTIVE_PROCESSOR_COUNT",
environment.REA_JADX_ACTIVE_PROCESSOR_COUNT,
),
},
checks: [configurationCheck(configured, availability, jarPath)],
};
};
/**
* Publish a caller-selected variable only when explicitly set; never persist
* ambient inheritance. Values are validated before this point.
*/
const optionalEntry = (
name: string,
value: string | undefined,
): Record<string, string> => (value === undefined ? {} : { [name]: value });
const configurationCheck = (
configured: boolean,
availability: ProviderAvailability,
jarPath: string | undefined,
): DoctorProviderCheck => {
const code = availability.status === "unavailable" ? availability.code : null;
if (availability.status === "available" && jarPath !== undefined) {
return {
name: "configuration",
ok: true,
code: null,
detail: `JADX MCP bridge JAR validated at ${jarPath}; its path is recorded for agent registrations.`,
remediation: null,
classification: "config_drift",
};
}
// The failure kind is tagged at the configuration boundary; project its
// remediation and classification here instead of reinterpreting the
// phase/setting evidence. Every failed check keeps a remediation.
const kind =
availability.status === "unavailable"
? asFailureKind(availability.diagnostics["failure_kind"])
: null;
const outcome = projectJadxFailureOutcome(kind, configured);
return {
name: "configuration",
ok: false,
code,
detail:
availability.status === "available"
? "Validated JADX configuration reported no JAR path."
: availability.reason,
remediation: outcome.remediation,
classification: outcome.classification,
...(kind === "java_probe" && availability.status === "unavailable"
? { details: probeDetails(availability.diagnostics) }
: kind !== null &&
kind.startsWith("windows_native_") &&
availability.status === "unavailable"
? { details: windowsNativeFailureDetails(availability.diagnostics) }
: {}),
};
};
const windowsNativeFailureDetails = (
diagnostics: Readonly<Record<string, JsonValue>>,
): Record<string, JsonValue> => {
const details: Record<string, JsonValue> = {};
for (const key of ["failure_stage", "remediation"] as const) {
const value = diagnostics[key];
if (value !== undefined) details[key] = value;
}
return details;
};
/** Validate the loose diagnostics tag against the boundary's failure kinds. */
const asFailureKind = (value: JsonValue | undefined): JadxFailureKind | null =>
typeof value === "string" &&
(JADX_FAILURE_KINDS as readonly string[]).includes(value)
? (value as JadxFailureKind)
: null;
/**
* Project bounded Java probe metadata for the doctor report. The full
* stdout/stderr can reach the 1 MiB probe budget, so only the bounded
* outcome metadata is carried; the message already references the retained
* output.
*/
const probeDetails = (
diagnostics: Readonly<Record<string, JsonValue>>,
): Record<string, JsonValue> => {
const details: Record<string, JsonValue> = {};
for (const key of [
"java_executable",
"failure",
"output_truncated",
"exit_code",
"process_error_code",
"signal",
"killed",
"output_limit_bytes",
"deadline_ms",
"jdk_compiler",
"java_major",
] as const) {
const value = diagnostics[key];
if (value !== undefined) details[key] = value;
}
return details;
};

View File

@ -15,6 +15,26 @@ export const JADX_BRIDGE_IDENTITY = Object.freeze({
/** Remediation when `REA_JADX_MCP_JAR` is missing or not an absolute path. */
export const JADX_JAR_CONFIGURATION_REMEDIATION = `Set REA_JADX_MCP_JAR to the absolute path of a caller-supplied jadx-headless-mcp ${JADX_RELEASE.version} JAR. REA does not download or install it.`;
/** Remediation when the selected Java runtime cannot satisfy the JADX bridge. */
export const JADX_JAVA_RUNTIME_REMEDIATION =
"Select a full JDK including jdk.compiler via JAVA_HOME or PATH, then verify it with java --list-modules. REA does not install Java.";
/** Remediation when a JADX JVM resource override is malformed. */
export const JADX_JVM_OPTIONS_REMEDIATION =
"Set REA_JADX_HEAP_MIB to a positive integer (heap size in MiB) and REA_JADX_ACTIVE_PROCESSOR_COUNT to a positive integer within the JVM's signed 32-bit range.";
/** Remediation when the host cannot run the Android JADX subprocess. */
export const JADX_UNSUPPORTED_HOST_REMEDIATION =
"Use Linux, macOS, or Windows x64 with its bundled native controls for Android JADX analysis.";
/** Remediation when the Windows native backend cannot take process ownership. */
export const JADX_WINDOWS_NATIVE_REMEDIATION =
"Use a matching current Windows x64 REA package with its bundled native controls. The native failure above identifies the failed constraint; Java installation or registration changes cannot enable a missing native backend.";
/** Remediation when the Java module-listing probe cannot establish the runtime. */
export const JADX_JAVA_PROBE_REMEDIATION =
"Verify the selected Java launcher with java --list-modules: it must start promptly and emit a concise module listing including a parseable java.base version. REA does not install Java.";
/** Producing engine identity used by Evidence and the generated product catalog. */
export const JADX_PROVIDER_IDENTITY = Object.freeze({
id: "jadx",

View File

@ -100,10 +100,12 @@ const decodeStub = (
readonly manifestFile?: string;
readonly stringsFile?: string;
readonly stringsXml?: string;
readonly extraValuesDirectories?: readonly string[];
} = {},
): ((root: string) => string) => {
const withStrings = options.withStrings ?? true;
const withGerman = options.withGerman ?? true;
const extraValuesDirectories = options.extraValuesDirectories ?? [];
const stringsXml = options.stringsXml ?? REAL_STRINGS;
const ymlWrite =
options.ymlFile === undefined
@ -117,6 +119,12 @@ const decodeStub = (
options.stringsFile === undefined
? `printf '%s\\n' '${stringsXml.replace(/'/gu, "'\\''")}' > "$5/res/values/strings.xml"`
: `cp '${options.stringsFile}' "$5/res/values/strings.xml"`;
const qualifierStrings = extraValuesDirectories
.map(
(name) =>
`printf '%s\\n' '<resources><string name="wrong">Wrong</string></resources>' > "$5/res/values-${name}/strings.xml"`,
)
.join("\\n ");
return () => `if [ "$1" = "--version" ]; then ${VERSION_BLOCK}; exit 0; fi
if [ "$1" = "d" ]; then
${
@ -124,9 +132,10 @@ if [ "$1" = "d" ]; then
? `printf 'Exception in thread "main" brut.androlib.AndrolibException: Could not decode the APK\\n' >&2; exit 1;`
: `${ymlWrite}
${manifestWrite}
mkdir -p "$5/res/values"${withGerman ? ' "$5/res/values-de"' : ""}
mkdir -p "$5/res/values"${withGerman ? ' "$5/res/values-de"' : ""}${extraValuesDirectories.map((name) => ` "$5/res/values-${name}"`).join("")}
${withStrings ? stringsWrite : ""}
${withGerman ? `printf '%s\\n' '${DE_STRINGS.replace(/'/gu, "'\\''")}' > "$5/res/values-de/strings.xml"` : ""}
${qualifierStrings}
printf 'I: Using Apktool on stub\\n'
exit 0`
}
@ -254,6 +263,30 @@ it("preserves empty resource strings and reports markup omissions as partial", a
expect(result.coverage).toBe("partial");
});
it("ignores Android resource qualifiers when discovering locales", async () => {
const { command, root } = await writeApktoolStub(
decodeStub({ extraValuesDirectories: ["car", "night", "v21"] }),
);
const apk = await apkFixture(root);
const result = await evidenceResult(providerOver(command), {
operation: "decode_android_resources",
input: { path: apk.path, include_strings: true },
});
expect(result.locales).toEqual(["de"]);
});
it.each(["car", "night", "v21"])(
"rejects %s as a requested locale",
(locale) => {
expect(
apktoolRequestSchema.safeParse({
operation: "decode_android_resources",
input: { path: "/tmp/probe.apk", locale },
}).success,
).toBe(false);
},
);
it("projects one locale's strings on request", async () => {
const { command, root } = await writeApktoolStub(decodeStub());
const apk = await apkFixture(root);

View File

@ -25,7 +25,10 @@ import {
createAnalysisExecution,
type AnalysisExecution,
} from "../application/AnalysisProvider.js";
import type { ApktoolRequest } from "../domain/apktool/apktoolResourceAnalysis.js";
import {
androidLocaleQualifierSchema,
type ApktoolRequest,
} from "../domain/apktool/apktoolResourceAnalysis.js";
import {
AnalysisCapabilityUnavailableError,
AnalysisCancelledError,
@ -291,6 +294,7 @@ const countDecodedFiles = async (
const match = /^values-(.+)$/u.exec(entry.name);
if (
match !== null &&
androidLocaleQualifierSchema.safeParse(match[1]).success &&
(await access(join(path, "strings.xml")).then(
() => true,
() => false,

View File

@ -10,6 +10,7 @@ import {
} from "./AnalysisProvider.js";
import {
AnalysisCancelledError,
AnalysisOutputError,
AnalysisProtocolError,
} from "../domain/analysisErrorCore.js";
import { err, ok } from "../domain/result.js";
@ -17,6 +18,35 @@ import { err, ok } from "../domain/result.js";
const provider = { id: "fixture", name: "Fixture", version: "1" };
describe("analysis context queries: navigation", () => {
it.each([
[null, "0x401000", "current_document"],
["fixture", null, "current_address"],
] as const)(
"rejects malformed %s / %s provider selection before returning success",
async (document, address, operation) => {
const analysis: AnalysisOperationPort = {
execute: async (name) =>
ok(
createAnalysisExecution(
name === "current_document" ? document : address,
provider,
),
),
};
const result = await getNavigationContext(analysis, {});
expect(result).toEqual(
err(
new AnalysisOutputError(
operation,
operation === "current_document"
? "expected a document identity string"
: "expected an address string",
),
),
);
},
);
it("aggregates volatile navigation and treats a missing procedure as null", async () => {
const calls: {
operation: string;
@ -70,6 +100,23 @@ describe("analysis context queries: navigation", () => {
});
});
it("preserves empty string selections accepted by the output contract", async () => {
const analysis: AnalysisOperationPort = {
execute: async (operation) =>
ok(
createAnalysisExecution(
operation === "resolve_containing_procedure"
? { found: false, procedure: null, reason: "not_in_procedure" }
: "",
provider,
),
),
};
await expect(getNavigationContext(analysis, {})).resolves.toEqual(
ok({ document: "", address: "", procedure: null }),
);
});
it("uses an explicit document without passing it to current_document and preserves procedure failures", async () => {
const calls: string[] = [];
const failure = new AnalysisProtocolError("provider response malformed");

View File

@ -4,10 +4,11 @@ import type {
AnalysisOperationPort,
} from "./AnalysisProvider.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { analysisBookmarkSchema } from "../domain/hopperValues.js";
import { resolveAnalysisDocument } from "./AnalysisDocument.js";
import { ok, type Result } from "../domain/result.js";
import { err, ok, type Result } from "../domain/result.js";
type Facet =
| { readonly state: "available"; readonly value: JsonValue }
@ -42,8 +43,14 @@ export const getNavigationContext = async (
if (typeof document !== "string" && !document.ok) return document;
const documentName =
typeof document === "string" ? document : document.value.result;
const documentArgs =
typeof documentName === "string" ? parameters(documentName) : {};
if (typeof documentName !== "string")
return err(
new AnalysisOutputError(
"current_document",
"expected a document identity string",
),
);
const documentArgs = parameters(documentName);
const address = await analysis.execute(
"current_address",
documentArgs,
@ -51,17 +58,12 @@ export const getNavigationContext = async (
);
if (!address.ok) return address;
if (typeof address.value.result !== "string")
return ok({
document: documentName,
address: address.value.result,
procedure: null,
});
return err(
new AnalysisOutputError("current_address", "expected an address string"),
);
const procedure = await analysis.execute(
"resolve_containing_procedure",
parameters(
typeof documentName === "string" ? documentName : undefined,
address.value.result,
),
parameters(documentName, address.value.result),
executionOptions(signal),
);
if (!procedure.ok) {

View File

@ -275,6 +275,12 @@ export type ProviderTargetSupport =
readonly diagnostics: Readonly<Record<string, JsonValue>>;
};
/** One owner-observed availability/capability snapshot for provider discovery. */
export interface AnalysisProviderReadinessSnapshot {
readonly availability: ProviderAvailability;
readonly capabilities: readonly CapabilityDescriptor[];
}
/** Deep provider candidate discoverable before one target-bound route is chosen. */
export interface AnalysisProviderCandidate extends AnalysisProvider {
resolveAnalysisProfile(
@ -282,5 +288,6 @@ export interface AnalysisProviderCandidate extends AnalysisProvider {
options?: AnalysisProfileResolutionOptions,
): Promise<Result<AnalysisProfileResolution, AnalysisError>>;
inspectAvailability(): ProviderAvailability;
inspectReadinessSnapshot?(): AnalysisProviderReadinessSnapshot;
inspectTargetSupport(target: BinaryTarget): ProviderTargetSupport;
}

View File

@ -0,0 +1,60 @@
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { resolveAppBundleExecutable } from "./AppBundleExecutable.js";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
describe("OpenStep Info.plist nesting during app target resolution", () => {
it.each([127, 128, 3000])(
"applies the shared bound to %s nested payload arrays",
async (depth) => {
const root = await createTestTempDirectory("rea-app-openstep-depth-");
const app = join(root, "Example.app");
const programs = join(app, "Contents", "MacOS");
await mkdir(programs, { recursive: true });
await writeFile(join(programs, "App"), "inert program-path fixture");
await writeFile(
join(app, "Contents", "Info.plist"),
`{ CFBundleExecutable = App; payload = ${"(".repeat(depth)}x${")".repeat(depth)}; }`,
);
const result = await resolveAppBundleExecutable(app);
if (depth === 127) {
expect(result).toMatchObject({
ok: true,
value: { executable: join(programs, "App") },
});
} else {
expect(result).toMatchObject({
ok: false,
error: {
path: join(app, "Contents", "Info.plist"),
cause: expect.objectContaining({
message: "Plist exceeds 128 nesting levels",
}),
},
});
}
},
);
it("resolves the first Info.plist root without decoding deeply nested trailing text", async () => {
const root = await createTestTempDirectory("rea-app-openstep-trailing-");
const app = join(root, "Example.app");
const programs = join(app, "Contents", "MacOS");
await mkdir(programs, { recursive: true });
await writeFile(join(programs, "App"), "inert program-path fixture");
await writeFile(
join(app, "Contents", "Info.plist"),
`{ CFBundleExecutable = App; }${"(".repeat(3000)}x${")".repeat(3000)}`,
);
expect(await resolveAppBundleExecutable(app)).toMatchObject({
ok: true,
value: { executable: join(programs, "App") },
});
});
});

View File

@ -32,7 +32,11 @@ import {
} from "../domain/rawBinary.js";
import type { BinaryArchitecture } from "../domain/binaryTargetTypes.js";
import { observeMzWindowsHeader } from "../domain/dosMz.js";
import { hasBinaryPlistSignature } from "../domain/apple/appleArchiveFormat.js";
import {
createXmlPlistSignatureProbe,
hasBinaryPlistSignature,
MAX_XML_PLIST_PROBE_BYTES,
} from "../domain/apple/appleArchiveFormat.js";
import {
hasZipSignature,
isZipPackageFormat,
@ -49,6 +53,8 @@ import {
} from "../domain/binaryTarget.js";
import type { PeCodeViewRecord } from "../domain/binaryTargetTypes.js";
const XML_PLIST_PROBE_CHUNK_BYTES = 64 * 1024;
/** Options for resolving a local binary target and its stable identity. */
export interface BinaryTargetResolverOptions {
readonly cwd?: string;
@ -288,12 +294,12 @@ const detectArtifactFormat = async (
> => {
throwIfTargetResolutionCancelled(signal);
const lower = path.toLowerCase();
const magic = Buffer.alloc(8);
const magic = Buffer.alloc(4096);
const observed = await handle.read(magic, 0, magic.length, 0);
throwIfTargetResolutionCancelled(signal);
if (
observed.bytesRead === 8 &&
magic.equals(Buffer.from([0, 0x61, 0x73, 0x6d, 1, 0, 0, 0]))
observed.bytesRead >= 8 &&
magic.subarray(0, 8).equals(Buffer.from([0, 0x61, 0x73, 0x6d, 1, 0, 0, 0]))
)
throw new BinaryTargetError(
path,
@ -329,6 +335,37 @@ const detectArtifactFormat = async (
return "dmg";
}
}
// XML property lists can use archive-specific suffixes such as .archive.
// The named-format check above keeps .js, .map, and .asar classifications.
const xmlProbe = createXmlPlistSignatureProbe();
let xmlState = xmlProbe.write(magic.subarray(0, observed.bytesRead));
if (xmlState === "matched") return "plist";
const chunk = Buffer.alloc(XML_PLIST_PROBE_CHUNK_BYTES);
let bytesRead = observed.bytesRead;
while (xmlState === "pending" && bytesRead < MAX_XML_PLIST_PROBE_BYTES) {
throwIfTargetResolutionCancelled(signal);
const read = await handle.read(
chunk,
0,
Math.min(chunk.length, MAX_XML_PLIST_PROBE_BYTES - bytesRead),
bytesRead,
);
throwIfTargetResolutionCancelled(signal);
if (read.bytesRead === 0) break;
bytesRead += read.bytesRead;
xmlState = xmlProbe.write(chunk.subarray(0, read.bytesRead));
if (xmlState === "matched") return "plist";
}
if (bytesRead === MAX_XML_PLIST_PROBE_BYTES) {
const { size } = await handle.stat();
throwIfTargetResolutionCancelled(signal);
if (size > bytesRead)
throw new BinaryTargetError(
path,
"XML plist prolog was not found within the 1 MiB format probe limit",
{ constraint: "file_format_rejected" },
);
}
return undefined;
};

View File

@ -11,7 +11,10 @@ import type {
CaptureWebScreenshotInput,
CompareWebScreenshotsInput,
} from "../domain/webScreenshot.js";
import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore.js";
import {
AnalysisCancelledError,
AnalysisCapabilityUnavailableError,
} from "../domain/analysisErrorCore.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { err, ok, type Result } from "../domain/result.js";
import type { ExecutionOptions } from "./AnalysisProvider.js";
@ -27,6 +30,8 @@ export const listBrowserTargets = async (
const ready = requireBrowser(browser, "list_browser_targets");
if (!ready.ok) return ready;
const result = await ready.value.listTargets(input, options);
if (result.ok && options.signal?.aborted === true)
return err(new AnalysisCancelledError("list_browser_targets"));
return result.ok
? ok(
createBrowserEvidence(
@ -48,6 +53,8 @@ export const inspectWebPage = async (
const ready = requireBrowser(browser, "inspect_web_page");
if (!ready.ok) return ready;
const result = await ready.value.inspectPage(input, options);
if (result.ok && options.signal?.aborted === true)
return err(new AnalysisCancelledError("inspect_web_page"));
return result.ok
? ok(
createBrowserEvidence(
@ -69,6 +76,8 @@ export const analyzeWebBundle = async (
const ready = requireBrowser(browser, "analyze_web_bundle");
if (!ready.ok) return ready;
const analyzed = await ready.value.analyzeBundle(input, options);
if (analyzed.ok && options.signal?.aborted === true)
return err(new AnalysisCancelledError("analyze_web_bundle"));
if (!analyzed.ok) return analyzed;
return ok(
createBrowserEvidence(
@ -89,6 +98,8 @@ export const observeWebSession = async (
const ready = requireBrowser(browser, "observe_web_session");
if (!ready.ok) return ready;
const observed = await ready.value.observeSession(input, options);
if (observed.ok && options.signal?.aborted === true)
return err(new AnalysisCancelledError("observe_web_session"));
return observed.ok
? ok(
createBrowserEvidence(
@ -110,6 +121,8 @@ export const discoverWebMcpTools = async (
const ready = requireBrowser(browser, "discover_webmcp_tools");
if (!ready.ok) return ready;
const discovered = await ready.value.discoverWebMcpTools(input, options);
if (discovered.ok && options.signal?.aborted === true)
return err(new AnalysisCancelledError("discover_webmcp_tools"));
return discovered.ok
? ok(
createBrowserEvidence(
@ -151,6 +164,8 @@ export const captureWebScreenshot = async (
const ready = requireBrowser(browser, "capture_web_screenshot");
if (!ready.ok) return ready;
const captured = await ready.value.captureScreenshot(input, options);
if (captured.ok && options.signal?.aborted === true)
return err(new AnalysisCancelledError("capture_web_screenshot"));
return captured.ok
? ok(
createBrowserEvidence(

View File

@ -98,6 +98,21 @@ const entry = (
};
describe("capability failure reasons", () => {
it("keeps an unavailable Android runtime distinct from a missing provider", () => {
const inventory = buildCapabilityInventory(status(), {
...enabledPolicy,
androidAnalysisAvailability: {
status: "unavailable",
code: "runtime_unavailable",
reason: "The JADX native self-test failed.",
diagnostics: {},
},
});
expect(
inventory.find(({ name }) => name === "inspect_android_package"),
).toMatchObject({ available: false, reason: "provider_unavailable" });
});
it("admits physical universal Mach-O targets for native Mach-O operations", () => {
expect(
entry(
@ -501,7 +516,6 @@ it("reports host denial before target mismatch but keeps composed workflow prere
describe("binary_overview capability requirements", () => {
it("admits a provider with current_document and other mandatory capabilities", () => {
const capabilities = [
"list_segments",
"current_document",
"list_procedures",
"list_strings",
@ -520,7 +534,6 @@ describe("binary_overview capability requirements", () => {
it("admits a provider with list_documents and other mandatory capabilities", () => {
const capabilities = [
"list_segments",
"list_documents",
"list_procedures",
"list_strings",
@ -556,7 +569,7 @@ describe("binary_overview capability requirements", () => {
});
});
it("reports missing segment enumeration independently when segments are missing", () => {
it("admits a provider without optional segment enumeration", () => {
const capabilities = [
"current_document",
"list_procedures",
@ -568,12 +581,31 @@ describe("binary_overview capability requirements", () => {
status({ open: true, kind: "executable", capabilities }),
),
).toMatchObject({
available: false,
reason: "provider_missing",
remediation: "Configure a provider for required operation list_segments.",
available: true,
reason: "available",
remediation: null,
});
});
it("keeps binary overview available when a declared segment facet is unavailable", () => {
const capabilities = [
{
operation: "list_segments",
available: false,
reason: "This upstream profile does not expose segment enumeration.",
},
{ operation: "list_documents", available: true, reason: null },
{ operation: "list_procedures", available: true, reason: null },
{ operation: "list_strings", available: true, reason: null },
];
expect(
entry(
"binary_overview",
status({ open: true, kind: "executable", capabilities }),
),
).toMatchObject({ available: true, reason: "available" });
});
it("reports provider_unavailable when declared document selection capability is unavailable", () => {
const capabilities = [
{ operation: "list_segments", available: true, reason: null },

View File

@ -115,7 +115,6 @@ const ENHANCED_REQUIREMENTS: Readonly<
analyze_swift_types: ["list_procedures"],
find_xrefs_to_name: ["list_names", "xrefs"],
binary_overview: [
"list_segments",
["current_document", "list_documents"],
"list_procedures",
"list_strings",
@ -345,7 +344,11 @@ const workflowAvailabilityFor = ({
kind,
policy,
}: AvailabilityContext): Availability | null => {
if (name === "inspect_go_binary" || name === "inspect_rust_binary")
if (
name === "inspect_go_binary" ||
name === "inspect_rust_binary" ||
name === "inspect_electron_fuses"
)
return { reason: "available", remediation: null };
if (name === "inspect_sqlite_database") {
const capability = sqliteDatabaseCapability(process.versions.node);
@ -462,7 +465,10 @@ const workflowAvailabilityFor = ({
reason:
policy.androidAnalysisAvailability?.code === "unsupported_host"
? "unsupported_host"
: "provider_missing",
: policy.androidAnalysisAvailability?.code ===
"runtime_unavailable"
? "provider_unavailable"
: "provider_missing",
remediation:
policy.androidAnalysisAvailability?.reason ??
"Set REA_JADX_MCP_JAR to a caller-supplied jadx-headless-mcp 0.7.1 JAR and provide a full JDK on Linux, macOS, or Windows x64 with its matching bundled native controls.",

View File

@ -167,7 +167,7 @@ export const parseZcodeFallbackConfiguration = (
? {}
: requireObject(document.mcpServers, "ZCode fallback mcpServers"),
serversPath: ["mcpServers"],
dialect: "json",
dialect: "zcode_fallback",
legacyServers: {},
};
};
@ -183,6 +183,7 @@ export const parseClientConfiguration = (
if (format === undefined || format === "unsupported")
throw new TypeError("client does not have a supported MCP config format");
// DeepSeek Harness stores a Cordis patch list, not a server table.
if (format === "zcode_fallback") return parseZcodeFallbackConfiguration(text);
if (format === "dsh") return parseDshCordisDocument(text);
const document = parseDocument(text, format);
const serversKey = clientConfigurationServersKey(format);

View File

@ -11,6 +11,21 @@ import {
import type { SetupClient } from "./SupportedClients.js";
import { clientRegistrationEnvironment } from "./ClientRegistrationEnvironment.js";
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import type { ConfigurationError } from "../domain/configurationErrors.js";
/** A parsed layer retains its path so rereading the write target replaces only that layer. */
export interface ClientConfigurationSource extends ClientConfigurationDocument {
readonly sourcePath: string;
}
const invalidEnvironment = (
path: string,
error: ConfigurationError,
): ClientConfigurationFileError => ({
kind: "malformed",
path,
detail: `Configuration ${path} has an invalid REA server environment: ${error.settings.map(({ setting, constraint }) => `${setting}: ${constraint}`).join("; ")}. Repair it before rerunning setup.`,
});
/** A named source failure that setup can explain without losing the bad path. */
export interface ClientConfigurationFileError {
@ -74,9 +89,9 @@ export const readClientConfigurationFiles = async (
client: SetupClient,
resources?: ArtifactResourceScope,
): Promise<
Result<readonly ClientConfigurationDocument[], ClientConfigurationFileError>
Result<readonly ClientConfigurationSource[], ClientConfigurationFileError>
> => {
const documents: ClientConfigurationDocument[] = [];
const documents: ClientConfigurationSource[] = [];
for (const path of client.configPaths ?? [client.configPath]) {
const document = await readConfigurationFile(
path,
@ -85,16 +100,21 @@ export const readClientConfigurationFiles = async (
);
if (!document.ok) return document;
if (document.value !== undefined) {
const environment = clientRegistrationEnvironment(document.value);
const environment = clientRegistrationEnvironment(
document.value,
{},
"fragment",
);
if (!environment.ok)
return err({
kind: "malformed",
path,
detail: `Configuration ${path} has an invalid REA server environment: ${environment.error.settings.map(({ setting, constraint }) => `${setting}: ${constraint}`).join("; ")}. Repair it before rerunning setup.`,
});
documents.push(document.value);
return err(invalidEnvironment(path, environment.error));
documents.push({ ...document.value, sourcePath: path });
}
}
const environment = clientRegistrationEnvironment(
effectiveClientConfiguration(documents),
);
if (!environment.ok)
return err(invalidEnvironment(client.configPath, environment.error));
return ok(documents);
};
@ -117,8 +137,24 @@ const isObject = (value: unknown): value is Record<string, unknown> =>
/** Project a client's effective user registration without modifying its source files. */
export const effectiveClientConfiguration = (
documents: readonly ClientConfigurationDocument[],
sources: readonly ClientConfigurationSource[],
target?: {
readonly client: Pick<SetupClient, "configPath" | "configPaths">;
readonly document: ClientConfigurationDocument | undefined;
},
): ClientConfigurationDocument | undefined => {
const documents =
target === undefined
? sources
: (target.client.configPaths ?? [target.client.configPath]).flatMap(
(path) => {
const document =
path === target.client.configPath
? target.document
: sources.find(({ sourcePath }) => sourcePath === path);
return document === undefined ? [] : [document];
},
);
const last = documents.at(-1);
if (last === undefined) return undefined;
if (documents.length === 1) return last;

View File

@ -1,6 +1,9 @@
import { z } from "zod";
import { parseEnvironment } from "../config/environment.js";
import {
parseEnvironment,
parseEnvironmentValues,
} from "../config/environment.js";
import { ConfigurationError } from "../domain/configurationErrors.js";
import { err, ok, type Result } from "../domain/result.js";
import {
@ -15,9 +18,10 @@ import { dshDocument } from "./DshCordisPatch.js";
export const clientRegistrationEnvironment = (
parsed: ClientConfigurationDocument | undefined,
overrides: Readonly<Record<string, string>> = {},
validation: "fragment" | "effective" = "effective",
): Result<Readonly<Record<string, string>>, ConfigurationError> => {
if (parsed?.dialect === "dsh")
return dshDocument(parsed.document).environment(overrides);
return dshDocument(parsed.document).environment(overrides, validation);
const key =
parsed?.dialect === "opencode" || parsed?.dialect === "opencode_v2"
? "environment"
@ -46,7 +50,10 @@ export const clientRegistrationEnvironment = (
// Validate known settings with the runtime's parser, but retain the original
// string overrides, including unknown client-specific keys and no defaults.
const environment = { ...existing.data, ...overrides };
const validated = parseEnvironment(environment);
const validated =
validation === "fragment"
? parseEnvironmentValues(environment)
: parseEnvironment(environment);
if (!validated.ok) return validated;
return ok(
Object.fromEntries(

View File

@ -76,18 +76,24 @@ const unwrapCmdCommand = (command: readonly string[]): readonly string[] => {
return [];
};
const packageRunnerReference = (
/** Preserve the package argument from a supported package-runner command. */
export const clientRegistrationPackageReference = (
command: readonly string[],
): string | undefined => {
const arguments_ = unwrapCmdCommand(command);
if (
command.length === 4 &&
command[0] === "npx" &&
command[1] === "-y" &&
command[3] === "mcp"
arguments_.length === 4 &&
arguments_[0] === "npx" &&
arguments_[1] === "-y" &&
arguments_[3] === "mcp"
)
return command[2];
if (command.length === 3 && command[0] === "bunx" && command[2] === "mcp")
return command[1];
return arguments_[2];
if (
arguments_.length === 3 &&
arguments_[0] === "bunx" &&
arguments_[2] === "mcp"
)
return arguments_[1];
return undefined;
};
@ -129,7 +135,7 @@ export const isOwnedClientRegistrationCommand = (
return true;
}
const packageReference = packageRunnerReference(unwrapCmdCommand(command));
const packageReference = clientRegistrationPackageReference(command);
return (
packageReference !== undefined && isOwnedPackageReference(packageReference)
);

View File

@ -13,10 +13,12 @@ import { access } from "node:fs/promises";
import { resolve } from "node:path";
import { z } from "zod";
import { valid } from "semver";
import { PRODUCT_IDENTITY } from "../identity.js";
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
import {
clientRegistrationPackageReference,
isOwnedClientRegistrationCommand,
packageRunnerRegistrationCommand,
} from "./ClientRegistrationIdentity.js";
@ -33,6 +35,8 @@ import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.j
interface ClientRegistrationStatusBase {
readonly client: string;
readonly config_path: string;
/** Exact package reference requested by npx/bunx, when registration uses one. */
readonly package_reference?: string;
}
type RegistrationCommand = readonly [string, ...string[]];
@ -201,12 +205,18 @@ export const readClientRegistrationInspections = async (
registration.command,
...registration.args,
];
const packageReference = clientRegistrationPackageReference(command);
const activation = registrationActivation(registration, client, parsed);
const packageVersion = exactRegisteredPackageVersion(packageReference);
const packageVersionDrift =
packageVersion !== undefined &&
packageVersion !== PRODUCT_IDENTITY.packageVersion;
record(
configuredStatus(
client,
command,
activation === "enabled" &&
!packageVersionDrift &&
registrationAligned(
registration,
client,
@ -217,7 +227,11 @@ export const readClientRegistrationInspections = async (
policyBlock === undefined
? "aligned"
: "stale",
policyBlock,
policyBlock ??
(packageVersionDrift
? packageVersionRemediation(packageReference ?? "", client)
: undefined),
packageReference,
),
activation,
);
@ -378,12 +392,16 @@ const configuredStatus = (
command: RegistrationCommand,
state: "aligned" | "stale",
policyRemediation?: string,
packageReference?: string,
): ClientRegistrationStatus =>
state === "aligned"
? {
client: client.name,
config_path: client.configPath,
command,
...(packageReference === undefined
? {}
: { package_reference: packageReference }),
state,
remediation: null,
}
@ -391,10 +409,29 @@ const configuredStatus = (
client: client.name,
config_path: client.configPath,
command,
...(packageReference === undefined
? {}
: { package_reference: packageReference }),
state,
remediation: policyRemediation ?? remediation(client),
};
const exactRegisteredPackageVersion = (
packageReference: string | undefined,
): string | undefined => {
if (packageReference === undefined) return undefined;
const prefix = `${PRODUCT_IDENTITY.packageName}@`;
if (!packageReference.startsWith(prefix)) return undefined;
const version = packageReference.slice(prefix.length);
return valid(version) === version ? version : undefined;
};
const packageVersionRemediation = (
packageReference: string,
client: SetupClient,
): string =>
`This registration requests ${packageReference}, while the current REA CLI package is ${PRODUCT_IDENTITY.packageVersion}. REA does not resolve package-runner requests during doctor; update the registration with rea setup --client ${client.name}, then restart the client.`;
const unavailableStatus = (
client: SetupClient,
state: "missing" | "invalid",

View File

@ -42,6 +42,7 @@ import {
workflowAnalysisProfile,
} from "./InvestigationProviders.js";
import {
executeComposedWorkflow,
createWorkflowEvidence,
workflowSnapshotRecord,
recordWorkflowUnknowns,
@ -401,18 +402,17 @@ const executeAnalysisTool = async (input: {
tool === "trace_native_ui_action"
) {
const { EnhancedTools } = await import("./EnhancedTools.js");
const result = await new EnhancedTools(session).execute(
tool,
input.arguments,
signal,
const result = await executeComposedWorkflow(session, (observed) =>
new EnhancedTools(observed).execute(tool, input.arguments, signal),
);
if (!result.ok) return { output: cliError(result.error) };
const evidence = createWorkflowEvidence({
target: input.openedTarget,
operation: tool,
parameters: input.arguments,
result: result.value,
result: result.value.result,
upstreamProfile: evidenceProfile,
upstreamLimitations: result.value.limitations,
});
return { output: evidence, evidence };
}

View File

@ -588,6 +588,9 @@ describe("doctor Linux and Hopper discovery", () => {
result.checks.find(({ name }) => name === "hopper-version")?.remediation,
).toContain("Unset or update HOPPER_LAUNCHER_PATH");
});
});
describe("doctor installed Hopper discovery", () => {
it("detects a Homebrew cask installed outside /Applications", async () => {
const path =
"/opt/homebrew/Caskroom/hopper/Hopper Disassembler.app/Contents/MacOS/hopper";
@ -612,7 +615,9 @@ describe("doctor Linux and Hopper discovery", () => {
);
expect(result.hopperPath).toBe(path);
});
});
describe("doctor target admission on Linux", () => {
it("uses shared app target validation when a target is supplied", async () => {
const result = await runDoctor(
"/Applications/Notes.app",
@ -652,6 +657,30 @@ describe("doctor Linux and Hopper discovery", () => {
});
});
describe("doctor standard Linux Hopper paths", () => {
it.each(["/opt/hopper-v4/bin/hopper", "/usr/bin/hopper"])(
"detects Hopper at the standard path %s",
async (path) => {
const result = await runDoctor(
undefined,
host({
platform: "linux",
macosVersion: () => Promise.resolve(undefined),
linuxDistribution: () =>
Promise.resolve({
id: "ubuntu",
versionId: "24.04",
packageFamily: "deb",
hopperSetupSupported: true,
}),
executable: (candidate) => Promise.resolve(candidate === path),
}),
);
expect(result.hopperPath).toBe(path);
},
);
});
describe("doctor target admission diagnostics", () => {
it("preserves route-specific target diagnostics from the host", async () => {
const targetCheck = {

View File

@ -48,6 +48,7 @@ export interface DoctorCheck {
readonly classification:
| "healthy"
| "missing_dependency"
| "runtime_failure"
| "unsupported_host"
| "unsupported_target"
| "missing_analysis_engine"
@ -65,6 +66,7 @@ export interface DoctorProviderCheck {
readonly detail: string;
readonly remediation: string | null;
readonly classification: Exclude<DoctorCheck["classification"], "healthy">;
readonly details?: Readonly<Record<string, JsonValue>>;
}
/** Read-only provider inspection supplied by an outer composition adapter. */

View File

@ -2,7 +2,11 @@ import {
SUPPORTED_NODE_VERSION_PROSE,
supportsNodeVersion,
} from "../domain/runtimeVersion.js";
import { linuxHopperLauncherPath, supportsReaHost } from "./LinuxHopper.js";
import {
linuxHopperLauncherCandidates,
systemLinuxHopperLauncherCandidates,
supportsReaHost,
} from "./LinuxHopper.js";
import type {
DoctorCheck,
DoctorHost,
@ -11,7 +15,6 @@ import type {
const DEFAULT_HOPPER =
"/Applications/Hopper Disassembler.app/Contents/MacOS/hopper";
const SYSTEM_LINUX_HOPPER = "/opt/hopper/bin/Hopper";
/** Ordered environment diagnostics consumed by the Doctor report assembler. */
export interface DoctorDiagnostics {
@ -148,8 +151,9 @@ const findHopper = async (host: DoctorHost): Promise<string | undefined> => {
host.configuredHopperPath === undefined
? [
DEFAULT_HOPPER,
SYSTEM_LINUX_HOPPER,
linuxHopperLauncherPath(host.homeDirectory),
...(host.platform === "linux"
? linuxHopperLauncherCandidates(host.homeDirectory)
: [...systemLinuxHopperLauncherCandidates]),
...(await host.manualHopperPaths()),
]
: [host.configuredHopperPath];
@ -280,6 +284,7 @@ const providerDoctorChecks = (
...(candidate.remediation === null
? {}
: { remediation: candidate.remediation }),
...(candidate.details === undefined ? {} : { details: candidate.details }),
}));
const unsupportedHopperRemediation = (configured: boolean): string =>

View File

@ -1,2 +1,2 @@
/** Deep-analysis providers whose readiness can be selected explicitly by doctor. */
export const DOCTOR_PROVIDER_IDS = ["hopper", "ghidra", "ida"] as const;
export const DOCTOR_PROVIDER_IDS = ["hopper", "ghidra", "ida", "jadx"] as const;

View File

@ -1,5 +1,8 @@
import { isMap, isSeq, type Node, type YAMLMap } from "yaml";
import { parseEnvironment } from "../config/environment.js";
import {
parseEnvironment,
parseEnvironmentValues,
} from "../config/environment.js";
import { ConfigurationError } from "../domain/configurationErrors.js";
import { err, ok, type Result } from "../domain/result.js";
import type { ClientConfigurationDocument } from "./ClientConfigurationDocument.js";
@ -181,6 +184,7 @@ export class DshCordisDocument {
/** Validate literals without treating expression source as an environment value. */
environment(
overrides: Readonly<Record<string, string>> = {},
validation: "fragment" | "effective" = "effective",
): Result<Readonly<Record<string, string>>, ConfigurationError> {
const config = this.#field("config");
if (config instanceof CordisExpression)
@ -215,8 +219,12 @@ export class DshCordisDocument {
new ConfigurationError("Cordis server environment must be a mapping"),
);
const literals: Record<string, string> = {};
const unresolvedSettings = new Set<string>();
for (const [key, value] of Object.entries(raw ?? {})) {
if (value instanceof CordisExpression) continue;
if (value instanceof CordisExpression) {
if (overrides[key] === undefined) unresolvedSettings.add(key);
continue;
}
if (typeof value !== "string")
return err(
new ConfigurationError("Invalid Cordis server environment", {
@ -232,7 +240,10 @@ export class DshCordisDocument {
literals[key] = value;
}
const environment = { ...literals, ...overrides };
const validated = parseEnvironment(environment);
const validated =
validation === "fragment"
? parseEnvironmentValues(environment)
: parseEnvironment(environment, unresolvedSettings);
return validated.ok ? ok(environment) : validated;
}

View File

@ -525,33 +525,47 @@ export class EnhancedTools {
this.#call("list_strings", {}, signal),
],
);
if (!segmentsResult.ok) return segmentsResult;
if (
!segmentsResult.ok &&
!(segmentsResult.error instanceof AnalysisCapabilityUnavailableError)
)
return segmentsResult;
if (!proceduresResult.ok) return proceduresResult;
if (!stringsResult.ok) return stringsResult;
const segments = parseSegments(segmentsResult.value);
if (!segments.ok) return segments;
const segments = segmentsResult.ok
? parseSegments(segmentsResult.value)
: undefined;
if (segments !== undefined && !segments.ok) return segments;
const procedureCount = parseListCount(proceduresResult.value, "procedures");
if (!procedureCount.ok) return procedureCount;
const stringCount = parseListCount(stringsResult.value, "strings");
if (!stringCount.ok) return stringCount;
const imageBase = parseSegmentImageBase(segments.value);
const observedSegments = segments?.ok ? segments.value : undefined;
const imageBase =
observedSegments === undefined
? undefined
: parseSegmentImageBase(observedSegments);
return ok({
document: document.value,
...(imageBase === undefined ? {} : { image_base: imageBase }),
segments: segments.value.map(
({ name, start, end, image_base, address_space }) => ({
name,
start,
end,
length: addressDistance(start, end),
...(image_base === undefined ? {} : { image_base }),
...(address_space === undefined ? {} : { address_space }),
}),
),
segment_count: segments.value.length,
...(observedSegments === undefined
? {}
: {
segments: observedSegments.map(
({ name, start, end, image_base, address_space }) => ({
name,
start,
end,
length: addressDistance(start, end),
...(image_base === undefined ? {} : { image_base }),
...(address_space === undefined ? {} : { address_space }),
}),
),
segment_count: observedSegments.length,
}),
procedure_count: procedureCount.value,
string_count: stringCount.value,
});

View File

@ -2,19 +2,36 @@ import { createHash } from "node:crypto";
import { describe, expect, it } from "vitest";
import {
formatCommandForDisplay,
installLinuxHopper,
linuxHopperLauncherDigestSupported,
linuxPackageManagerCommands,
linuxSharedLibrariesAvailable,
parseLinuxDistribution,
supportsReaHost,
LinuxHopperTemporaryCleanupOwner,
type LinuxDistribution,
type LinuxHopperDownload,
type LinuxHopperInstallHost,
type LinuxHopperLauncherStatus,
type LinuxPackageFamily,
type LinuxPackageInstallFailure,
type LinuxPackageInstallOutcome,
} from "./LinuxHopper.js";
describe("Linux package-manager command display", () => {
it("preserves argument boundaries for paths with spaces and quotes", () => {
expect(
formatCommandForDisplay("pkexec", [
"pacman",
"-U",
"/tmp/rea test/hopper.arch",
"apostrophe'path",
]),
).toBe('pkexec pacman -U "/tmp/rea test/hopper.arch" "apostrophe\'path"');
});
});
class RecordingLinuxHost implements LinuxHopperInstallHost {
distributionValue: LinuxDistribution | undefined = {
id: "ubuntu",
@ -25,6 +42,14 @@ class RecordingLinuxHost implements LinuxHopperInstallHost {
archive = new Uint8Array([1, 2, 3]);
archiveOk = true;
installSucceeds = true;
cleanupFails = false;
packageFailure: LinuxPackageInstallFailure = {
command: "pacman -U --noconfirm /tmp/rea test/hopper.arch",
exitCode: 1,
stdout: "",
stderr: "error: failed to prepare transaction",
outputTruncated: false,
};
launcherStatusValue: LinuxHopperLauncherStatus = "ready";
downloads: string[] = [];
installed: Array<{ family: LinuxPackageFamily; archive: string }> = [];
@ -42,15 +67,20 @@ class RecordingLinuxHost implements LinuxHopperInstallHost {
installPackage = (
family: LinuxPackageFamily,
archive: string,
): Promise<boolean> => {
): Promise<LinuxPackageInstallOutcome> => {
this.installed.push({ family, archive });
return Promise.resolve(this.installSucceeds);
const outcome: LinuxPackageInstallOutcome = this.installSucceeds
? { status: "installed" }
: { status: "failed", failure: this.packageFailure };
return Promise.resolve(outcome);
};
launcherStatus = (): Promise<LinuxHopperLauncherStatus> =>
Promise.resolve(this.launcherStatusValue);
cleanup = (path: string): Promise<void> => {
this.cleaned.push(path);
return Promise.resolve();
return this.cleanupFails
? Promise.reject(new Error("temporary path is busy"))
: Promise.resolve();
};
}
@ -183,7 +213,9 @@ describe("Linux Hopper installation", () => {
);
},
);
});
describe("Linux Hopper package installation", () => {
it.each(["deb", "rpm", "arch"] as const)(
"downloads, verifies, installs, reads back, and cleans a %s package",
async (family) => {
@ -204,7 +236,9 @@ describe("Linux Hopper installation", () => {
expect(host.cleaned).toEqual(["/tmp/rea test"]);
},
);
});
describe("Linux Hopper installation failure classification", () => {
it.each([
["unsupported host", "unsupported_host"],
["package request failure", "download"],
@ -232,11 +266,65 @@ describe("Linux Hopper installation", () => {
expect(await installFixture(host)).toEqual({
status: "failed",
reason,
...(scenario === "package manager failure"
? { packageFailure: host.packageFailure }
: {}),
});
if (host.installed.length > 0)
expect(host.cleaned).toEqual(["/tmp/rea test"]);
});
});
describe("Linux Hopper cleanup and failure details", () => {
it("preserves the failing package-manager command and its output", async () => {
const host = new RecordingLinuxHost();
host.installSucceeds = false;
host.packageFailure = {
command: "pkexec pacman -U --noconfirm /tmp/rea test/hopper.arch",
exitCode: 1,
stdout: "loading packages...",
stderr:
"\u001b[31merror: unresolvable package conflicts detected\u001b[0m",
outputTruncated: false,
};
expect(await installFixture(host)).toEqual({
status: "failed",
reason: "authorization_or_package_manager",
packageFailure: host.packageFailure,
});
expect(host.cleaned).toEqual(["/tmp/rea test"]);
});
it("preserves a successful install result when temporary cleanup fails", async () => {
const host = new RecordingLinuxHost();
host.cleanupFails = true;
expect(await installFixture(host)).toEqual({
status: "installed",
launcherPath: "/opt/hopper/bin/Hopper",
cleanupFailure: {
path: "/tmp/rea test",
reason: "temporary path is busy",
},
});
});
it("preserves an install failure when temporary cleanup also fails", async () => {
const host = new RecordingLinuxHost();
host.installSucceeds = false;
host.cleanupFails = true;
expect(await installFixture(host)).toMatchObject({
status: "failed",
reason: "authorization_or_package_manager",
packageFailure: host.packageFailure,
cleanupFailure: {
path: "/tmp/rea test",
reason: "temporary path is busy",
},
});
});
});
describe("Linux Hopper cancellation", () => {
it("cancels before network or mutation", async () => {
const host = new RecordingLinuxHost();
const controller = new AbortController();
@ -252,6 +340,27 @@ describe("Linux Hopper installation", () => {
});
});
describe("Linux Hopper temporary cleanup ownership", () => {
it("retains a failed cleanup for a later successful retry", async () => {
const owner = new LinuxHopperTemporaryCleanupOwner();
owner.retain({ path: "/tmp/rea-hopper", reason: "busy" });
let attempts = 0;
expect(
await owner.retry(async () => {
attempts += 1;
throw new Error("still busy");
}),
).toContain("/tmp/rea-hopper");
expect(
await owner.retry(async () => {
attempts += 1;
}),
).toBeUndefined();
expect(attempts).toBe(2);
});
});
function releasesFor(
bytes: Uint8Array,
): Record<

View File

@ -1,18 +1,18 @@
import { createHash } from "node:crypto";
import { execFile } from "node:child_process";
import { createReadStream } from "node:fs";
import { access, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { promisify } from "node:util";
import { execFileBoundedOutput } from "../process/ExecFileBoundedOutput.js";
import { execFileOutput } from "../process/ExecFileOutput.js";
import {
downloadHopperPackage,
HOPPER_DOWNLOAD_BYTES,
} from "./HopperDownload.js";
const execFileAsync = promisify(execFile);
/** Package-manager output retained per stream without stopping the command. */
const PACKAGE_MANAGER_OUTPUT_BYTES = 64 * 1024;
const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/";
const SUPPORTED_HOPPER_SHA256 = new Set([
"0294ced141cc373468ee22d8343e7dac41980cb05a937994ca81c9f09afe7ded",
@ -67,7 +67,7 @@ export const linuxHopperInstallDisclosure = (
family,
`<verified-hopper.${family}>`,
isRoot,
).map(({ executable, args }) => `${executable} ${args.join(" ")}`),
).map(({ executable, args }) => formatCommandForDisplay(executable, args)),
};
};
@ -111,6 +111,25 @@ export interface LinuxHopperDownload {
readonly bytes: Uint8Array;
}
/** One failed package-manager command, with the output that explains it. */
export interface LinuxPackageInstallFailure {
/** The executed executable and arguments, joined for display. */
readonly command: string;
readonly exitCode: string | number | null;
readonly stdout: string;
readonly stderr: string;
/** Whether the command produced more output than was retained. */
readonly outputTruncated: boolean;
}
/** Whether the package-manager step installed the verified package. */
export type LinuxPackageInstallOutcome =
| { readonly status: "installed" }
| {
readonly status: "failed";
readonly failure: LinuxPackageInstallFailure;
};
/** External effects required to install the official Linux package. */
export interface LinuxHopperInstallHost {
distribution(): Promise<LinuxDistribution | undefined>;
@ -120,15 +139,20 @@ export interface LinuxHopperInstallHost {
): Promise<LinuxHopperDownload>;
createTemporaryDirectory(): Promise<string>;
writeArchive(path: string, bytes: Uint8Array): Promise<void>;
installPackage(family: LinuxPackageFamily, archive: string): Promise<boolean>;
installPackage(
family: LinuxPackageFamily,
archive: string,
): Promise<LinuxPackageInstallOutcome>;
launcherStatus(path: string): Promise<LinuxHopperLauncherStatus>;
cleanup(path: string): Promise<void>;
}
/** Structured outcome for every expected Linux installation failure. */
export type LinuxHopperInstallResult =
| { readonly status: "installed"; readonly launcherPath: string }
| {
| ({ readonly status: "installed"; readonly launcherPath: string } & {
readonly cleanupFailure?: LinuxHopperTemporaryCleanupFailure;
})
| ({
readonly status: "failed";
readonly reason:
| "unsupported_host"
@ -140,7 +164,49 @@ export type LinuxHopperInstallResult =
| "runtime_dependencies"
| "unsupported_hopper_build"
| "cancelled";
};
/**
* The failing package-manager command and its output. Present only when
* the package-manager step itself failed, so setup can report what the
* package manager said instead of assuming an authorization problem.
*/
readonly packageFailure?: LinuxPackageInstallFailure;
} & { readonly cleanupFailure?: LinuxHopperTemporaryCleanupFailure });
/** Failed removal of an owned temporary install directory. */
export interface LinuxHopperTemporaryCleanupFailure {
readonly path: string;
readonly reason: string;
}
/** Own failed temporary cleanup until a later retry succeeds. */
export class LinuxHopperTemporaryCleanupOwner {
private readonly pending = new Map<
string,
LinuxHopperTemporaryCleanupFailure
>();
retain(failure: LinuxHopperTemporaryCleanupFailure): void {
this.pending.set(failure.path, failure);
}
async retry(
cleanup: (path: string) => Promise<void>,
): Promise<string | undefined> {
const failures: string[] = [];
for (const path of this.pending.keys()) {
try {
await cleanup(path);
this.pending.delete(path);
} catch (cause: unknown) {
const reason = cause instanceof Error ? cause.message : String(cause);
failures.push(
`Linux Hopper temporary cleanup failed for ${path}: ${reason}`,
);
}
}
return failures.length === 0 ? undefined : failures.join("; ");
}
}
export type LinuxHopperLauncherStatus =
| "ready"
@ -215,33 +281,57 @@ export const installLinuxHopper = async (
if (distribution?.hopperSetupSupported !== true)
return { status: "failed", reason: "unsupported_host" };
let temporary: string | undefined;
let result: LinuxHopperInstallResult;
try {
const release = (options.releases ?? SUPPORTED_RELEASES)[
distribution.packageFamily
];
const archiveDownload = await host.download(release.filename, options);
if (!archiveDownload.ok) return { status: "failed", reason: "download" };
if (!packageIntegrityMatches(archiveDownload.bytes, release))
return { status: "failed", reason: "integrity" };
temporary = await host.createTemporaryDirectory();
const archive = join(temporary, `hopper.${distribution.packageFamily}`);
await host.writeArchive(archive, archiveDownload.bytes);
if (!(await host.installPackage(distribution.packageFamily, archive)))
return { status: "failed", reason: "authorization_or_package_manager" };
const launcherPath = "/opt/hopper/bin/Hopper";
const launcherStatus = await host.launcherStatus(launcherPath);
if (launcherStatus !== "ready")
return {
status: "failed",
reason:
launcherStatus === "missing" ? "launcher_missing" : launcherStatus,
};
return { status: "installed", launcherPath };
result = await (async (): Promise<LinuxHopperInstallResult> => {
const release = (options.releases ?? SUPPORTED_RELEASES)[
distribution.packageFamily
];
const archiveDownload = await host.download(release.filename, options);
if (!archiveDownload.ok) return { status: "failed", reason: "download" };
if (!packageIntegrityMatches(archiveDownload.bytes, release))
return { status: "failed", reason: "integrity" };
temporary = await host.createTemporaryDirectory();
const archive = join(temporary, `hopper.${distribution.packageFamily}`);
await host.writeArchive(archive, archiveDownload.bytes);
const install = await host.installPackage(
distribution.packageFamily,
archive,
);
if (install.status === "failed")
return {
status: "failed",
reason: "authorization_or_package_manager",
packageFailure: install.failure,
};
const launcherPath = "/opt/hopper/bin/Hopper";
const launcherStatus = await host.launcherStatus(launcherPath);
if (launcherStatus !== "ready")
return {
status: "failed",
reason:
launcherStatus === "missing" ? "launcher_missing" : launcherStatus,
};
return { status: "installed", launcherPath };
})();
} catch (cause: unknown) {
if (isAbortCause(cause)) return { status: "failed", reason: "cancelled" };
return { status: "failed", reason: "download" };
} finally {
if (temporary !== undefined) await host.cleanup(temporary);
result = {
status: "failed",
reason: isAbortCause(cause) ? "cancelled" : "download",
};
}
if (temporary === undefined) return result;
try {
await host.cleanup(temporary);
return result;
} catch (cause: unknown) {
return {
...result,
cleanupFailure: {
path: temporary,
reason: cause instanceof Error ? cause.message : String(cause),
},
};
}
};
@ -249,6 +339,24 @@ export const installLinuxHopper = async (
export const linuxHopperLauncherPath = (home: string): string =>
join(home, ".local/share/rea/hopper/bin/Hopper");
/** REA's managed Linux install path, preferred when no explicit launcher is set. */
export const REA_MANAGED_LINUX_HOPPER_LAUNCHER = "/opt/hopper/bin/Hopper";
/** Other system-wide Hopper paths used by package installations. */
export const systemLinuxHopperLauncherCandidates: readonly string[] = [
"/opt/hopper-v4/bin/hopper",
"/usr/bin/hopper",
];
/** Probe REA-managed and user-local installs before arbitrary system builds. */
export const linuxHopperLauncherCandidates = (
home: string,
): readonly string[] => [
REA_MANAGED_LINUX_HOPPER_LAUNCHER,
linuxHopperLauncherPath(home),
...systemLinuxHopperLauncherCandidates,
];
/** Compose package-manager and linker checks from the selected caller environment. */
export const systemLinuxHopperInstallHost = (
environment: Readonly<NodeJS.ProcessEnv>,
@ -304,24 +412,49 @@ const installSystemPackage = async (
family: LinuxPackageFamily,
archive: string,
environment: Readonly<NodeJS.ProcessEnv>,
): Promise<boolean> => {
try {
for (const command of linuxPackageManagerCommands(
family,
archive,
process.getuid?.() === 0,
))
await execFileAsync(command.executable, command.args, {
): Promise<LinuxPackageInstallOutcome> => {
for (const command of linuxPackageManagerCommands(
family,
archive,
process.getuid?.() === 0,
)) {
const output = await execFileBoundedOutput(
command.executable,
command.args,
{
env: environment,
});
return true;
} catch (cause: unknown) {
// Package-manager failure is reported by the false return.
void cause;
return false;
limit: PACKAGE_MANAGER_OUTPUT_BYTES,
},
);
if (output.exitCode !== 0) {
return {
status: "failed",
failure: {
command: formatCommandForDisplay(command.executable, command.args),
exitCode: output.exitCode,
// The streams stay separate; their original interleaved order
// is not recoverable after capture.
stdout: output.stdout,
stderr: output.stderr,
outputTruncated: output.outputTruncated,
},
};
}
}
return { status: "installed" };
};
/** Preserve argument boundaries in a command shown to the user. */
export const formatCommandForDisplay = (
executable: string,
args: readonly string[],
): string =>
[executable, ...args]
.map((part) =>
/^[A-Za-z0-9_./:@%+=,-]+$/u.test(part) ? part : JSON.stringify(part),
)
.join(" ");
/** Select the native package manager and authorization boundary without shell evaluation. */
export const linuxPackageManagerCommands = (
family: LinuxPackageFamily,

View File

@ -10,6 +10,8 @@ import {
type MipsElfRelocationSection,
} from "./MipsElfSharedLoad.js";
const SHT_GNU_ATTRIBUTES = 0x6ffffff5;
/** Observed ABI and load declarations, retaining unavailable source coverage. */
export interface MipsElfInspection {
readonly abiFlags: MipsAbiFlags | null;
@ -151,6 +153,10 @@ export const readMipsElfMetadata = async (
checkCancelled();
const position = i * table.entrySize;
const type = u32(contents.value, position + table.typeOffset);
if (table.optional && type === SHT_GNU_ATTRIBUTES)
return err(
"MIPS ELF declares GNU ABI attributes that REA does not inspect; refusing provider selection until the declared ABI is parsed and reconciled",
);
if (metadata.type === 3 && table.optional && (type === 4 || type === 9))
relocationSections.push({
type,

View File

@ -12,7 +12,7 @@ import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.j
import type { AnalysisCleanupObservation } from "../domain/analysisErrorBase.js";
import { readReferenceSource } from "../reference/ReferenceSourceReader.js";
import type {
ReferenceSourceEntryKind,
ReferenceSourceExclusionKind,
ReferenceSourceRead,
ReferenceSourceReaderError,
} from "../reference/ReferenceSourceReaderTypes.js";
@ -111,7 +111,8 @@ const deriveInventoryState = (
entry.limitations.length > 0 ||
(entry.kind === "file" && entry.content_state !== "hashed") ||
(entry.kind === "directory" && entry.tree_state !== "enumerated") ||
(entry.kind === "symlink" && entry.target_state !== "internal"),
(entry.kind === "symlink" && entry.target_state !== "internal") ||
entry.kind === "unknown",
);
if (partialEntry) return "partial";
const partialRelationship = input.relationships.some(
@ -142,22 +143,36 @@ const createShouldExclude =
exclusions: HistoricalSourceGraphInput["exclusions"],
secrets: PreparedReferenceSourceImport["secrets"],
ignored: PreparedReferenceSourceImport["ignored"],
): ((path: string, kind: ReferenceSourceEntryKind) => boolean) =>
): ((path: string, kind: ReferenceSourceExclusionKind) => boolean) =>
(path, kind) => {
const patternPath = kind === "directory" ? `${path}/` : path;
const secretMatch = secrets.test(patternPath);
if (secretMatch.ignored) {
// An entry of unknown kind may be a directory, so directory-only rules
// must still exclude it; this fails closed for caller-marked secrets.
const patternPaths =
kind === "directory"
? [`${path}/`]
: kind === "unknown"
? [path, `${path}/`]
: [path];
const firstMatch = (matcher: typeof secrets) => {
const results = patternPaths.map((patternPath) =>
matcher.test(patternPath),
);
return results.find(({ ignored }) => ignored) ?? results[0];
};
const secretMatch = firstMatch(secrets);
if (secretMatch?.ignored === true) {
if (!secretMatch.rule)
throw new Error(`Ignored secret path has no matching rule: ${path}`);
exclusions.push({
path,
entry_kind: kind,
reason: "configured-secret",
pattern: secretMatch.rule.pattern,
});
return true;
}
const match = ignored.test(patternPath);
if (!match.ignored) return false;
const match = firstMatch(ignored);
if (match?.ignored !== true) return false;
if (!match.rule)
throw new Error(`Ignored path has no matching rule: ${path}`);
const reason = match.rule.mark;
@ -167,7 +182,12 @@ const createShouldExclude =
reason !== "caller-excluded"
)
throw new Error(`Ignored path has unknown rule origin: ${path}`);
exclusions.push({ path, reason, pattern: match.rule.pattern });
exclusions.push({
path,
entry_kind: kind,
reason,
pattern: match.rule.pattern,
});
return true;
};

Some files were not shown because too many files have changed in this diff Show More