diff --git a/README.md b/README.md index 370e0974e..4c303628d 100644 --- a/README.md +++ b/README.md @@ -460,17 +460,14 @@ Attach-only JavaScript runtime observation is separately disabled by default: ```bash export REA_V8_INSPECTOR_OBSERVE_ENABLED=true -export REA_V8_INSPECTOR_ENDPOINTS_JSON='["http://127.0.0.1:9229"]' -export REA_V8_INSPECTOR_FILE_ROOTS_JSON='["/absolute/path/to/app"]' -export REA_V8_INSPECTOR_ALLOWED_ORIGINS_JSON='[]' -rea list-javascript-runtime-targets http://127.0.0.1:9229 --approved --json +rea list-javascript-runtime-targets http://127.0.0.1:9229 --json rea observe-javascript-runtime http://127.0.0.1:9229 TARGET_ID \ - --runtime-kind node --approved --json + --runtime-kind node --json ``` -REA sends only `Runtime.enable` and `Debugger.enable`. It retains bounded, -scope-authorized script locations and execution-context lifecycle events; +REA sends only `Runtime.enable` and `Debugger.enable`. It retains validated +script locations and execution-context lifecycle events; require/import edges, EventEmitter activity, Electron IPC, PID identity, and Electron role identity stay explicit unknowns. See [passive Node and Electron runtime observation](docs/javascript-runtime-observation.md). diff --git a/docs/browser-observation.md b/docs/browser-observation.md index 5f70641bf..865197760 100644 --- a/docs/browser-observation.md +++ b/docs/browser-observation.md @@ -8,16 +8,14 @@ REA can attach to a user-owned Chrome-family browser through the Chrome DevTools - `inspect_web_page` / `rea inspect-web-page` captures DOM structure, accessibility nodes, scripts, resources, safe response/DOM metadata, attach-window network and console metadata, WebSocket frame sizes, workers, quota, and optionally storage key names, redacted storage-content fingerprints, or script sources. - `analyze_web_bundle` / `rea analyze-web-bundle` parses approved script artifacts without execution and derives chunk edges, route and endpoint candidates, vendor fingerprints, static WebMCP declarations, and optionally approved source-map/original-source evidence. - `observe_web_session` / `rea observe-web-session` arms an observation window of the requested duration for an external user action and records ordered reload, SPA navigation, redirect, failure, lifecycle, and target-termination metadata. -- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete structural input-schema summary inline. REA never exposes `WebMCP.invokeTool`; malformed or over-limit protocol payloads fail explicitly instead of returning truncated tool metadata. +- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete structural input-schema summary inline. REA never exposes `WebMCP.invokeTool`; malformed protocol payloads fail explicitly instead of returning incomplete tool metadata. - `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence. - `capture_web_screenshot` / `rea capture-web-screenshot` returns an explicitly approved, content-addressed visible-viewport PNG. - `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services. - Every surface has equivalent CLI and MCP contracts and returns Evidence provenance. - MCP tools return complete results inline and include the Evidence ID. Session evidence can be exported with `export_evidence_bundle`. -Target discovery returns the complete approved target array in one result. The -CDP endpoint's response-size ceiling remains internal; an oversized discovery -response fails explicitly instead of appearing complete. +Target discovery returns the complete approved target array in one result. Electron `file://` pages use a separate provider, permission capability, and root model; see [electron-observation.md](electron-observation.md). Existing static application Evidence and passive web/Electron captures can be diff --git a/docs/electron-observation.md b/docs/electron-observation.md index 5dcc96af2..e8c3a0a14 100644 --- a/docs/electron-observation.md +++ b/docs/electron-observation.md @@ -1,6 +1,6 @@ # Electron file-page observation -REA can attach to a user-owned Electron/Chromium CDP endpoint and inspect existing `file://` renderer pages without evaluating JavaScript or invoking Electron APIs. Electron observation is separate from website observation because filesystem roots, not HTTP origins, define its authority. +REA can attach to a user-owned Electron/Chromium CDP endpoint and inspect existing `file://` renderer pages without evaluating JavaScript or invoking Electron APIs. The caller supplies the endpoint directly; REA accepts only literal-port loopback HTTP endpoints. Selecting an endpoint exposes every Electron target it serves, including local file paths and page metadata. This passive runtime surface is distinct from the target-free static [`analyze_javascript_application`](javascript-artifact-reconstruction.md) @@ -16,23 +16,15 @@ The capability is disabled by default: ```bash export REA_ELECTRON_OBSERVE_ENABLED=true -export REA_ELECTRON_CDP_ENDPOINTS_JSON='["http://127.0.0.1:9223"]' -export REA_ELECTRON_FILE_ROOTS_JSON='["/Applications/Example.app/Contents/Resources"]' ``` -Endpoints accept only explicit-port loopback HTTP URLs. Roots are canonicalized by the shared permission authority. Each target, frame, script, and resource is independently converted from a hostless `file://` URL to a real path and checked after symlink resolution. UNC hosts, percent-encoded path separators, nonexistent paths, and root escapes are rejected. +The capability is disabled by default. Its permission ceiling is loopback-only. REA accepts local hostless `file://` URLs that resolve to regular files; remote hosts, encoded path separators, and nonexistent paths are rejected. There is no separate filesystem-root configuration. ## Workflow -`list_electron_targets` returns every target inside the approved roots in one -inline array. CDP response-size limits remain internal and oversized discovery -responses fail explicitly. +`list_electron_targets` returns every eligible target served by the supplied endpoint in one inline array. -For the MCP follow-up call, pass the discovery Evidence ID and selected target -ID to `inspect_electron_page`. REA reuses the endpoint and authorized roots from -that retained discovery Evidence, checks the selected ID against its target -list, and reapplies permission policy. The agent does not repeat the roots. The -CLI continues to accept its endpoint and roots directly. +For the MCP follow-up call, pass the same literal-loopback endpoint and the selected target ID to `inspect_electron_page`. REA rediscovers targets and validates the ID against the live endpoint before inspection. ```bash rea list-electron-targets http://127.0.0.1:9223 --json @@ -40,7 +32,7 @@ rea inspect-electron-page http://127.0.0.1:9223 TARGET_ID \ --observation-ms 100 --json ``` -Script content is excluded by default. Requesting it includes every authorized script source whose individual source stays within the per-script validation bound: +Script content is excluded by default. Requesting it includes every authorized script source: ```bash rea inspect-electron-page http://127.0.0.1:9223 TARGET_ID \ @@ -56,8 +48,8 @@ inventories authorized worker, service-worker, and shared-worker targets with validated opener-target and parent-frame IDs. Worker discovery uses passive target metadata; REA does not attach to or execute code in those targets. Collection counts and aggregate script-source bytes are not capped. Like every -target, frame, script, and resource, a worker URL must resolve beneath an -approved canonical root before it is retained. Relationship IDs improve +target, frame, script, and resource, a worker URL must resolve to a local +file before it is retained. Relationship IDs improve attribution but do not prove which static module started a worker or that its work completed. diff --git a/docs/installation.md b/docs/installation.md index 0a8f3fb32..9b2929942 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -234,17 +234,17 @@ environment entry in its plan, writes only after approval, and never downloads, installs, upgrades, or modifies Ghidra or Java. Each verified session uses an ephemeral temporary project and isolated -home/cache/config/temp paths. REA passes `-readOnly`, `-deleteProject`, a -per-file analysis timeout, CPU and heap bounds, and its packaged Java bridge via -`-scriptPath`; it never opens an existing user project. Linux uses a +home/cache/config/temp paths. REA passes `-readOnly`, `-deleteProject`, uses +Ghidra's default analysis and resource settings, and loads its packaged Java +bridge via `-scriptPath`; it never opens an existing user project. Linux uses a current-user-only local bridge socket and descriptor. Windows P0 uses token-authenticated IPv4 loopback, a token-free endpoint record, and bounded process-tree termination. It does not yet prove private DACL, reparse-point-safe, or Job Object semantics; use only approved non-sensitive fixtures. -Operations begin only after default auto-analysis completes. A timeout fails -the open rather than exposing partial analysis. One session contains exactly +Operations begin only after default auto-analysis completes. The provider +startup deadline fails the open rather than exposing partial analysis. One session contains exactly one imported Program; use `provider_id: "ghidra"`, `--provider ghidra`, or `REA_ANALYSIS_PROVIDER=ghidra` when both Hopper and Ghidra support the target. One persistent decompiler is owned by the Program, Ghidra API calls cross a diff --git a/docs/javascript-application-workflows.md b/docs/javascript-application-workflows.md index 8ebdeb799..16414e0e1 100644 --- a/docs/javascript-application-workflows.md +++ b/docs/javascript-application-workflows.md @@ -138,8 +138,9 @@ replay recommendation; it does not execute JavaScript. ## CLI and verification -All five CLI commands accept inline JSON or a JSON file up to 64 MiB. The input -is the same object used by the corresponding MCP tool. +All five CLI commands accept inline JSON or a path to a JSON file. Put the full +Evidence records in the workflow input, as in the corresponding MCP tool; the +CLI does not resolve Evidence IDs from a separate bundle. For two operator-provided directories or ASARs, run: diff --git a/docs/javascript-runtime-observation.md b/docs/javascript-runtime-observation.md index a36e971fa..daf981f82 100644 --- a/docs/javascript-runtime-observation.md +++ b/docs/javascript-runtime-observation.md @@ -13,9 +13,6 @@ replay: ```bash export REA_V8_INSPECTOR_OBSERVE_ENABLED=true -export REA_V8_INSPECTOR_ENDPOINTS_JSON='["http://127.0.0.1:9229"]' -export REA_V8_INSPECTOR_FILE_ROOTS_JSON='["/absolute/path/to/app"]' -export REA_V8_INSPECTOR_ALLOWED_ORIGINS_JSON='[]' rea list-javascript-runtime-targets http://127.0.0.1:9229 \ --json @@ -23,23 +20,12 @@ rea observe-javascript-runtime http://127.0.0.1:9229 TARGET_ID \ --runtime-kind node --json ``` -At least one canonical file root or exact HTTP(S) origin is required. -Discovery is restricted to a literal-loopback HTTP endpoint. A target must -match the exact requested target ID, expose a same-port WebSocket reported by -the approved discovery endpoint, and remain inside the requested file or -origin scope. Targets already marked attached are rejected rather than -displacing another debugger. Canonical file paths are checked after symlink resolution. -Excluded locations are counted but never retained. +The caller supplies the literal-loopback HTTP endpoint directly. Selecting an endpoint exposes all targets it serves and their local script locations or renderer origins. A target must match the exact requested target ID and expose a same-port WebSocket reported by that endpoint. Targets already marked attached are rejected rather than displacing another debugger. Local paths are canonicalized after symlink resolution. -`list_javascript_runtime_targets` returns every approved target in one inline -array. The Inspector discovery response-size ceiling remains internal; an -oversized response fails explicitly. +`list_javascript_runtime_targets` returns every available target in one inline +array. -For the MCP follow-up call, pass the discovery Evidence ID and selected target -ID to `observe_javascript_runtime`. REA reuses the endpoint and file/origin -scope from that retained discovery Evidence, checks the selected ID against its -target list, and reapplies permission policy. The agent does not repeat roots or -origins. The CLI continues to accept its endpoint and scope arguments directly. +For the MCP follow-up call, pass the same literal-loopback endpoint and the selected target ID to `observe_javascript_runtime`. REA rediscovers targets and validates the ID against the live endpoint before observation. ## Passive protocol boundary diff --git a/docs/product-catalog.json b/docs/product-catalog.json index 4e40e7c63..fffb089a2 100644 --- a/docs/product-catalog.json +++ b/docs/product-catalog.json @@ -501,9 +501,9 @@ "mcp_prompts": 6 }, "digests": { - "tools_sha256": "6847abaeec3a3c0b962edbb3f61301db4c7c05b1caa9e517484331516e14b324", - "prompts_sha256": "7f7f15d84efdce3487412ea64fdf4b83d3d6cf01b42c7fadc3e046799a2b0b15", - "combined_sha256": "5efc89a50de2cdd829729292ad0ab65dee6032b55f22293e35f08e33018955ec", + "tools_sha256": "160dcf163a82ce2bc841622077bd069d1ef29d02095a7ad44b294217c84e92b8", + "prompts_sha256": "32a133bf3cbd4485e38e3b29da401626d142a8ca9b1f11d3ac002a1dd1d318b8", + "combined_sha256": "ea88133456deb4be91d59a64184624fdce5edad8d8e5e6d25f6abbcfa002e0a2", "providers_sha256": "7247eece8fad09c8023af8d5c682d223d7ed7ebd68eac79e9154962a7e03b66b" } } diff --git a/docs/provider-evaluation.md b/docs/provider-evaluation.md index e87342acd..c6aafe7e1 100644 --- a/docs/provider-evaluation.md +++ b/docs/provider-evaluation.md @@ -34,7 +34,7 @@ follow. - Start with bring-your-own Ghidra and a compatible Java runtime. Setup must not install or upgrade Java. - Run Ghidra headlessly in an owned process with a private temporary project, - bounded startup/analysis/request deadlines, cancellation, and cleanup. + startup and cleanup deadlines, caller cancellation, and complete replies. - Prefer a packaged Java bridge loaded through Ghidra's script path. PyGhidra remains useful for prototypes but is not a mandatory production dependency. - Treat authenticated `ping` and `shutdown` as lifecycle proof only. Inventory, @@ -61,9 +61,9 @@ other formats, and other architectures before launch. The launcher creates one ephemeral runtime root with project, home/cache/config/data/temp, logs, descriptor, endpoint, target snapshot, and ownership manifest. -It passes `-readOnly`, `-deleteProject`, a 300-second per-file analysis limit, -two CPUs, and a 2 GiB heap; inherited Java option injection variables are -cleared. On Linux, the mode-0600 descriptor carries the random token without +It passes `-readOnly` and `-deleteProject`, and uses Ghidra's default analysis +and resource settings; inherited Java option injection variables are cleared. +On Linux, the mode-0600 descriptor carries the random token without exposing it in argv or environment and the Java bridge binds a mode-restricted Unix socket. Windows uses authenticated IPv4 loopback with a strict token-free endpoint record. Both transports report actual @@ -92,31 +92,31 @@ capability from a successful import. | Strings | Only Ghidra-defined string `Data` is observed. Items report charset, byte length, and whether a required null terminator is missing. The API cannot distinguish a present terminator from fixed/Pascal layouts when no terminator is missing, so that state is named `present_or_not_required`. | | Memory | Memory-block end addresses are exclusive. Read/write/execute, initialization, overlay, address space, and image base are direct Ghidra observations. | | Inventory | Procedure, symbol, and string listings plus searches return the complete matching collection in one response; callers do not provide offsets or limits. The bridge retains a 64 MiB internal process-line safety bound for exceptional responses. | -| Search | Literal and Java-regex searches scan the complete immutable inventory and return all matching entries inline. Search uses the established-request deadline; there are no caller-supplied offsets or result-count limits. | -| Analysis state | The socket is exposed only after default auto-analysis. A 300-second analysis timeout fails target opening; ordinary established requests have a 10-second client deadline. The internal response-line guard is sized for complete analysis output. | +| Search | Literal and Java-regex searches scan the complete immutable inventory and return all matching entries inline. Search waits for a reply or caller cancellation; there are no caller-supplied offsets or result-count limits. | +| Analysis state | The socket is exposed only after default auto-analysis completes. Established operations wait for their reply or caller cancellation. The internal response-line guard is sized for complete analysis output. | ## Admitted function-analysis semantics -| Concern | Ghidra contract | -| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Decompiler lifetime | One persistent `DecompInterface` is opened for the imported Program and disposed during bridge shutdown. Each native decompile has a 30-second deadline; the socket leaves a bounded 35-second projection window. External functions or functions without bodies return `null`; timeout, cancellation, and native failure remain distinct. | -| Serialization | A bounded FIFO admits at most 32 active-plus-queued requests and sends one Program request at a time. Queue wait counts against the caller's deadline, and queued cancellation is prompt. This is an adapter safety commitment, not a claim that every Ghidra API is thread-safe. | -| Function identity | Every function result carries the entry address and Ghidra FunctionManager classification for external, thunk, and resolved thunk target. These are observations, not proof that unresolved targetless calls have been recovered. | -| Assembly and pseudocode | Assembly is complete Ghidra Listing text; pseudocode is Ghidra decompiler output. Neither is original source, and cross-provider comparison never treats Hopper and Ghidra text as equal or unequal semantic facts. | -| Instruction fast path | `read_function_instructions` returns every raw Listing instruction for the requested function without invoking the decompiler or whole-program name/string inventories. | -| Calls and references | Callers/callees contain only resolved functions. Reference edges preserve exact ReferenceManager type and call/jump/data/read/write/indirect/computed/conditional/terminal/external facts. Targetless computed flow remains unknown. Synthetic entry-point references without actionable memory sources are omitted explicitly. | -| CFG | Dossiers use `BasicBlockModel` and retain only non-call successors inside the function body. CFG topology is address-normalized for comparison; provider-specific block construction remains a declared difference. | -| Bounds | Function instruction scans and native API boundary observations do not use item-count caps or caller pagination. A 64 MiB internal process-line guard fails exceptionally large individual responses with an explicit output error; it does not intentionally truncate ordinary full analysis. | +| Concern | Ghidra contract | +| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Decompiler lifetime | One persistent `DecompInterface` is opened for the imported Program and disposed during bridge shutdown. Native decompilation has no fixed per-function deadline. External functions or functions without bodies return `null`; cancellation and native failure remain distinct. | +| Serialization | A FIFO sends one Program request at a time. Caller cancellation removes queued work promptly and is passed to active socket waits. This is an adapter safety commitment, not a claim that every Ghidra API is thread-safe. | +| Function identity | Every function result carries the entry address and Ghidra FunctionManager classification for external, thunk, and resolved thunk target. These are observations, not proof that unresolved targetless calls have been recovered. | +| Assembly and pseudocode | Assembly is complete Ghidra Listing text; pseudocode is Ghidra decompiler output. Neither is original source, and cross-provider comparison never treats Hopper and Ghidra text as equal or unequal semantic facts. | +| Instruction fast path | `read_function_instructions` returns every raw Listing instruction for the requested function without invoking the decompiler or whole-program name/string inventories. | +| Calls and references | Callers/callees contain only resolved functions. Reference edges preserve exact ReferenceManager type and call/jump/data/read/write/indirect/computed/conditional/terminal/external facts. Targetless computed flow remains unknown. Synthetic entry-point references without actionable memory sources are omitted explicitly. | +| CFG | Dossiers use `BasicBlockModel` and retain only non-call successors inside the function body. CFG topology is address-normalized for comparison; provider-specific block construction remains a declared difference. | +| Bounds | Function instruction scans and native API boundary observations do not use item-count caps or caller pagination. A 64 MiB internal process-line guard fails exceptionally large individual responses with an explicit output error; it does not intentionally truncate ordinary full analysis. | `npm run verify:ghidra` compiles the versioned C oracles into x86-64 debug and stripped ELF, AArch64 ELF, x86-64 PE, and x86-64 Mach-O targets. It proves all 19 operations, external functions, resolved thunks, exports, stripped-name behavior, direct and targetless indirect calls, typed references, strings/xrefs, -multi-block CFG, semantic enhanced workflows, cancellation, deadlines, +multi-block CFG, semantic enhanced workflows, cancellation, startup deadlines, serialized concurrency, malformed-target rejection, profile identity, and process/project cleanup against real Ghidra 12.1.2. Unit fixtures separately -cover analysis/decompile timeouts, process exit, queue saturation, and malformed -wire output. +cover startup deadlines, process exit, queued cancellation, and malformed wire +output. `npm run verify:ghidra:windows` generates a deterministic source-owned native x86-64 PE application, proves all 19 operations through the production Windows @@ -130,7 +130,7 @@ gates. `src/process/` now provides the mechanisms that a long-lived Hopper or Ghidra adapter genuinely shares: POSIX run-token-authenticated process-group ownership, ephemeral temporary runtime roots, one absolute startup deadline, correlated -request timeout/cancellation cleanup, bounded stdout and stderr retention with +request cancellation and lifecycle-deadline cleanup, bounded stdout and stderr retention with exact byte counts, process-exit diagnostics, and bounded TERM-to-KILL shutdown. Reusable fixtures exercise exit, timeout, cancellation, graceful termination, forced termination, double-close, spawn failure, and resource release. Windows diff --git a/docs/verification/managed-conformance-manifest.json b/docs/verification/managed-conformance-manifest.json index cc0813e60..68d099c7b 100644 --- a/docs/verification/managed-conformance-manifest.json +++ b/docs/verification/managed-conformance-manifest.json @@ -1,5 +1,5 @@ { - "manifest_id": "ecm_209d5126691ff80c6be13d0dc12f0062f5f9a7370d501a7f908aa0ee7d822576", + "manifest_id": "ecm_5f5b8318487ce373d4b6e6c96254032bed01153b295a720254a3e70411dd84da", "verifier": { "id": "rea-managed-conformance", "version": "1" @@ -13,7 +13,7 @@ "skill_digests": [ { "skill_id": "reverse-engineer-anything", - "sha256": "3540554ed6ca8c5906eb691a324c4a7f9d94c1a858c9b12c62b6c764ee56dc5e" + "sha256": "8da549dc6b0ca7141293dab23706064ab12a880475e7c5860256daf751e11cd8" } ], "claims": [ diff --git a/skills/reverse-engineer-anything/SKILL.md b/skills/reverse-engineer-anything/SKILL.md index 6dff73fe0..17db7b112 100644 --- a/skills/reverse-engineer-anything/SKILL.md +++ b/skills/reverse-engineer-anything/SKILL.md @@ -4,7 +4,7 @@ description: Reverse engineer native, managed, Electron/JavaScript, packaged, an metadata: version: "23" tool_count: 111 - catalog_digest: "5efc89a50de2cdd829729292ad0ab65dee6032b55f22293e35f08e33018955ec" + catalog_digest: "ea88133456deb4be91d59a64184624fdce5edad8d8e5e6d25f6abbcfa002e0a2" --- # REA